90 percent of everything : Usability Blog

Written by Harry Brignull

When you’re designing a website, the range of things a user can do at any one time is fairly limited – you have quite a clear idea of the paths most users will take and this makes your life relatively easy. Video game designers don’t have this luxury. When they put together NPC AI, physics systems, combinable objects, “go-anywhere” environments and unpredictable user desires, they can’t really be sure how it will all play out until it’s all put together. The whole is much more than the sum of the parts, and a huge amount of tuning is required to ensure a game is as good as it can be. Put this together with the sheer scale of the industry (Grand Theft Auto IV took in over $500 million in sales during its opening week!) – and it’s obvious that gameplay research has a critical role in ensuring success and profitability.

It’s a really interesting space to work in right now, and I’ve arranged a special event on gameplay research for next month’s UX Brighton (8-Sept-09). The line-up includes GiGi Demming (User Testing Manager at Sony Computer Entertainment Europe), Graham McCalister (Director of Vertical Slice), and one or two other mystery guests (tbc). Tickets aren’t yet available, but they will be soon.

In the mean time, here’s a taster of the kind of research that people are doing in this space right now. Most of these images are cribbed (with permission) from a slide deck used in the “Game Metrics and Biometrics” panel at Future play 2009 (Nacke, Ambinder, Canossa, Mandryk, & Stach).

Heatmap showing player deaths

This heatmap is a visualisation of the locations where players tend to die on a level map (Ref: Tychsen & Canossa, 2008; Tychsen, 2008). This gives very useful pointers for how makes it easy to diagnose and fix areas of poor design where the player needs some kind of “boost” to get through. It’s all about balancing – players hate games to be too easy or too hard.

Flower of Death Information Visualisation

This visualisation from the game ‘Kane & Lynch: Fragile Alliance’ shows player role at time of death against the cause of death. It’s intended to be interactive so a doesn’t make much sense in a static image, but its purpose is allow a researcher to ascertain whether certain roles are weaker or stronger in certain scenarios. (Ref: Tychsen & Canossa, 2008; Tychsen, 2008).

A typical Galvanic Skin Response (GSR) unit

This is a typical Galvanic Skin Response (GSR) device. (Image credit: Shiori Saito). The amount of moisture on a person’s skin is a tacit, non-conscious indicator of their emotional state. This simple device measures the conductivity on a person’s hand. The trouble with GSR is that sweat builds up over time – so for example, it can erroneously show a high level of arousal during a boring cut scene that followed an exciting end-of-level shootout, simply because they player’s hands are still sweaty.

Comparing GSR for human-human gameplay against human-computer

This graph (ref: Mandryk, et al. 2006) shows the marked difference between scoring a goal against the computer (boring) versus scoring a goal against a friend (kick-ass exciting). This is probably quite useful when play testing users without think-aloud protocol, and aggregating the data for multiple participants.

A “full” biometric research lab set-up

An extensive set of kit is used in this picture: an eye tracker, screen recording software, facial expression recording via webcam, plus EEG/EMG and GSR devices (Ref: Nacke & Lindley, 2008). I don’t envy their research analysts who must have to crunch vast reams of data for each participant.

Example output from an EEG session

Crazy stuff isn’t it. The real challenge with EEG analysis is to take this data output and translate it into actionable findings that can be used to optimise the game design.

The jury is still out regarding biometrics – some gameplay researchers doubt the practical value, suggesting such tools have a big wow factor but can be expensive and time consuming to use. Although biometrics may not yet be proven as a core tool in gameplay research, it’s great that people are working on it. This is how new research methods are born.

Special thanks to Lennart Nacke et al for use of their images in this article. If you fancy reading more on the academic side of things, check out Lennart Nacke’s and Regan Mandryk’s research publications.

Bruce Archer is widely recognised as one of the founding fathers of Design Research. It’s worth reading his wikipedia entry. Here’s a fascinating excerpt about his initial experiences in using research to improve the design of hospital equipment. You’ll notice that although his design recommendations were at first rejected by the funding body (who couldn’t understand his unconventional approach), today they are familiar features in hospitals all over the world.

“In 1961 Misha Black was appointed head of industrial design at the Royal College of Art and asked him to lead a research project called Studies in the function and design of non-surgical hospital equipment, being funded by the Nuffield Foundation. Archer returned in the summer of 1962 and, with a small multi-disciplinary team, identified four urgent design problems: a receptacle for soiled dressings, a means of reducing incorrect dispensing of medicines to ward patients, the need for a standard design for hospital beds, and a way to prevent smoke control doors being routinely propped open. They presented their report at the end of the first year to the Nuffield Foundation. Unfortunately:

‘They hated it. They’d expected beautifully presented designs for funny looking cutlery for hospital patients to use in bed. That was what art schools did.’

Nuffield refused to fund a second year, leaving both Archer and Misha Black stunned. Undaunted, he took a job at the Eldorado ice cream factory in Southwark, loading ice cream into refrigerated vans every night and working at the College unpaid during the day. Eventually commercial funding was found for the soiled dressings receptacle, and in 1963 he gave up his evening job when support was obtained from the King Edward’s Hospital Fund for London to study the medicine-dispensing problem. A radical solution was devised – a medicine trolley on wheels which could be securely padlocked to a wall when not in use. The hospital bed problem was also re-examined. The King Edward’s Hospital Fund became the King’s Fund and was seeking a major exercise to promote its new nationwide role. It took on the standardisation of the hospital bed. Archer was appointed to a Working Party, and in due course won a contract for a standard specification and a prototype design. After widespread consultation, evidence gathering through direct observations, and extensive field trials using mock-ups and test devices, the specification was adopted by the Kings Fund and became a British Standard; a successful prototype was also developed by Kenneth Agnew at the College for a commercial bed manufacturer. The hospital bed project has been documented by an historian [4][5]. The fire door problem was solved by the use of electro-magnetic door-holders wired to the fire alarm, which released the doors when the alarm was triggered. So solutions to all four of the original projects were delivered. In the process, Archer had demonstrated that work study, systems analysis, and ergonomics, were proper tools for use by designers, and that systematic methods were not inimical to creativity in design, but essential contributors to it.”

If you find this stuff interesting, you might also want to take a look at this article by Bruce Archer: “A place for design in management education“. Considering it was written in 1967, it covers some impressively contemporary topics (and is refreshingly free of iPhone references).

If you want to go even deeper, you might want to take a look at Ghislaine Lawrence’s PhD Thesis (2001), an account of the origins and early years of ‘King’s Fund Beds’ (mentioned in the excerpt above). Unsurprisingly, it’s fairly heavy-going, but worth a read if you dare.