90 percent of everything : Usability Blog

Written by Harry Brignull

Earlier this month, Carsten Schmitt & Poppy James of Bunnyfoot gave an Eye-Tracking demo at UX Brighton. People have been asking us for some of the heatmaps that were generated during the demo, so here they are.

Before you look at them, a word of warning: these heatmaps are entirely inconclusive and non-scientific. They are the result of a small handful of people trying out an Eye-Tracker in a noisy environment. The purpose of showing them here is to raise questions rather than answer them.

Clooney or Crook: which one do people prefer?

Judging from the heatmaps, which actor do you think was preferred by our 9 volunteers? Handsome Clooney or strange looking Crook? Actually, it’s a trick question. Eye-Tracking heatmaps only tell you where your participants fixate – they do not tell you what they are thinking. This has very important implications for when you use Eye-Tracking on your own sites. If you notice a hotspot (i.e. gaze fixations) on a certain area of a homepage, you don’t know whether this is because users are understanding it and lapping it up, or whether they are thinking “What the hell does that mean, I’d better read that again!” For this very reason, eye-tracking studies are normally paired with an analysis of the user’s behaviour and some form of retrospective think-aloud (in which the user tells the researcher what they were thinking during the test). This data is crucial in explaining what the heatmaps actually mean.

Michelangelo’s David

As you can see above, people almost always focus in on faces, breasts and genital areas. This is common knowledge and often exploited by advertisers, but what’s interesting is that it seems to have nothing to do with sexual orientation – in our little test, even the straightest male participant couldn’t help but cop a quick look at David’s manhood.

Cennydd Bowles: Tourdust

Moving on to something less frivolous, here’s a heatmap from Cennydd Bowles’s Tourdust. If you take a look at the live site, you’ll see that he’s employed a JavaScript progressive disclosure technique to hide/reveal the site’s faceted navigation system. While usefully compact, the risk with such an approach is that users might not notice the call-to-action. To counter this, he’s added an instruction on the right that reads “Click these to find what you’re looking for”. Judging from the heatmap above it seems to have been effective.

Guardian Jobs

Above you can see a heatmap of Guardian Jobs. There’s a hotspot at the top right-hand side, relating to an eye-catching orange rectangle that reads “Recruiters, post your jobs online”. It’s worth noting that this works because it’s the only such element on the page. Repeat this trick too many times, and you’d create visual clutter with too many brash elements vying for the user’s attention.

So, that’s all I’ve got for you today. Hopefully you found that interesting, even though it was inherently inconclusive. By the way, if you fancy getting an eye tracker of your own, you should start saving up now – Tobii Eye-Trackers weigh in at roughly $35,000 [PDF] for a high end model.

Rob Gillham of HFI gave a presentation on Persuasion, Emotion and Trust at UX Brighton earlier this month (Aug ‘09). Here are the slides from his talk.

Many thanks to HFI, Bunnyfoot, iCrossing, Madgex and everyone who attended for making the event such a success!

→ View Rob’s presentation on slideshare

[Amendment 22-Aug-09: the sideshare embed code was removed from this page because it was causing problems]

When you’re designing a website, the range of things a user can do at any one time is fairly limited – you have quite a clear idea of the paths most users will take and this makes your life relatively easy. Video game designers don’t have this luxury. When they put together NPC AI, physics systems, combinable objects, “go-anywhere” environments and unpredictable user desires, they can’t really be sure how it will all play out until it’s all put together. The whole is much more than the sum of the parts, and a huge amount of tuning is required to ensure a game is as good as it can be. Put this together with the sheer scale of the industry (Grand Theft Auto IV took in over $500 million in sales during its opening week!) – and it’s obvious that gameplay research has a critical role in ensuring success and profitability.

It’s a really interesting space to work in right now, and I’ve arranged a special event on gameplay research for next month’s UX Brighton (8-Sept-09). The line-up includes GiGi Demming (User Testing Manager at Sony Computer Entertainment Europe), Graham McCalister (Director of Vertical Slice), and one or two other mystery guests (tbc). Tickets aren’t yet available, but they will be soon.

In the mean time, here’s a taster of the kind of research that people are doing in this space right now. Most of these images are cribbed (with permission) from a slide deck used in the “Game Metrics and Biometrics” panel at Future play 2009 (Nacke, Ambinder, Canossa, Mandryk, & Stach).

Heatmap showing player deaths

This heatmap is a visualisation of the locations where players tend to die on a level map (Ref: Tychsen & Canossa, 2008; Tychsen, 2008). This gives very useful pointers for how makes it easy to diagnose and fix areas of poor design where the player needs some kind of “boost” to get through. It’s all about balancing – players hate games to be too easy or too hard.

Flower of Death Information Visualisation

This visualisation from the game ‘Kane & Lynch: Fragile Alliance’ shows player role at time of death against the cause of death. It’s intended to be interactive so a doesn’t make much sense in a static image, but its purpose is allow a researcher to ascertain whether certain roles are weaker or stronger in certain scenarios. (Ref: Tychsen & Canossa, 2008; Tychsen, 2008).

A typical Galvanic Skin Response (GSR) unit

This is a typical Galvanic Skin Response (GSR) device. (Image credit: Shiori Saito). The amount of moisture on a person’s skin is a tacit, non-conscious indicator of their emotional state. This simple device measures the conductivity on a person’s hand. The trouble with GSR is that sweat builds up over time – so for example, it can erroneously show a high level of arousal during a boring cut scene that followed an exciting end-of-level shootout, simply because they player’s hands are still sweaty.

Comparing GSR for human-human gameplay against human-computer

This graph (ref: Mandryk, et al. 2006) shows the marked difference between scoring a goal against the computer (boring) versus scoring a goal against a friend (kick-ass exciting). This is probably quite useful when play testing users without think-aloud protocol, and aggregating the data for multiple participants.

A “full” biometric research lab set-up

An extensive set of kit is used in this picture: an eye tracker, screen recording software, facial expression recording via webcam, plus EEG/EMG and GSR devices (Ref: Nacke & Lindley, 2008). I don’t envy their research analysts who must have to crunch vast reams of data for each participant.

Example output from an EEG session

Crazy stuff isn’t it. The real challenge with EEG analysis is to take this data output and translate it into actionable findings that can be used to optimise the game design.

The jury is still out regarding biometrics – some gameplay researchers doubt the practical value, suggesting such tools have a big wow factor but can be expensive and time consuming to use. Although biometrics may not yet be proven as a core tool in gameplay research, it’s great that people are working on it. This is how new research methods are born.

Special thanks to Lennart Nacke et al for use of their images in this article. If you fancy reading more on the academic side of things, check out Lennart Nacke’s and Regan Mandryk’s research publications.

Bruce Archer is widely recognised as one of the founding fathers of Design Research. It’s worth reading his wikipedia entry. Here’s a fascinating excerpt about his initial experiences in using research to improve the design of hospital equipment. You’ll notice that although his design recommendations were at first rejected by the funding body (who couldn’t understand his unconventional approach), today they are familiar features in hospitals all over the world.

“In 1961 Misha Black was appointed head of industrial design at the Royal College of Art and asked him to lead a research project called Studies in the function and design of non-surgical hospital equipment, being funded by the Nuffield Foundation. Archer returned in the summer of 1962 and, with a small multi-disciplinary team, identified four urgent design problems: a receptacle for soiled dressings, a means of reducing incorrect dispensing of medicines to ward patients, the need for a standard design for hospital beds, and a way to prevent smoke control doors being routinely propped open. They presented their report at the end of the first year to the Nuffield Foundation. Unfortunately:

‘They hated it. They’d expected beautifully presented designs for funny looking cutlery for hospital patients to use in bed. That was what art schools did.’

Nuffield refused to fund a second year, leaving both Archer and Misha Black stunned. Undaunted, he took a job at the Eldorado ice cream factory in Southwark, loading ice cream into refrigerated vans every night and working at the College unpaid during the day. Eventually commercial funding was found for the soiled dressings receptacle, and in 1963 he gave up his evening job when support was obtained from the King Edward’s Hospital Fund for London to study the medicine-dispensing problem. A radical solution was devised – a medicine trolley on wheels which could be securely padlocked to a wall when not in use. The hospital bed problem was also re-examined. The King Edward’s Hospital Fund became the King’s Fund and was seeking a major exercise to promote its new nationwide role. It took on the standardisation of the hospital bed. Archer was appointed to a Working Party, and in due course won a contract for a standard specification and a prototype design. After widespread consultation, evidence gathering through direct observations, and extensive field trials using mock-ups and test devices, the specification was adopted by the Kings Fund and became a British Standard; a successful prototype was also developed by Kenneth Agnew at the College for a commercial bed manufacturer. The hospital bed project has been documented by an historian [4][5]. The fire door problem was solved by the use of electro-magnetic door-holders wired to the fire alarm, which released the doors when the alarm was triggered. So solutions to all four of the original projects were delivered. In the process, Archer had demonstrated that work study, systems analysis, and ergonomics, were proper tools for use by designers, and that systematic methods were not inimical to creativity in design, but essential contributors to it.”

If you find this stuff interesting, you might also want to take a look at this article by Bruce Archer: “A place for design in management education“. Considering it was written in 1967, it covers some impressively contemporary topics (and is refreshingly free of iPhone references).

If you want to go even deeper, you might want to take a look at Ghislaine Lawrence’s PhD Thesis (2001), an account of the origins and early years of ‘King’s Fund Beds’ (mentioned in the excerpt above). Unsurprisingly, it’s fairly heavy-going, but worth a read if you dare.