90 percent of everything : Usability Blog

Written by Harry Brignull

User research. It’s right there in the name. A user is someone who actually uses your service. Equally valid is the idea of a “target user” – someone who doesn’t yet use your service, but has a genuine need that it would fulfil.

User research has to involve these people. Otherwise, by definition, it’s not user research.

When you use a panel provided by remote usability testing service, you end up gathering data from a bunch of freelancers – professional participants who know the kind of things they’re expected to say. At best, they’ll make a real effort to take the tasks seriously and you feedback. At worst, they’re just actors, going through the motions to get paid.

Real user research involves recruiting participants who honestly care about the problem your service is trying to solve. They’ll engage with the information, weigh up their options in difficult decisions, and carefully consider implications. Most importantly, they’ll draw upon their life experiences to weigh up the benefits of your service against their current practices and other competitors.

One of the biggest strengths of remote research is the fact that you can cast your recruitment net far more widely than with face-to-face research. Remote research is well suited to sourcing real users. If you use remote research techniques and then test irrelevant participants, you’re missing the point of it all.


——

Edit 1: This post isn’t intended as a criticism of remote usability testing services per se. Most services offer a predefined panel alongside the option to recruit your own users. I’m advising you to recruit your own, and to take care in doing so – it’s as simple as that.

Edit 2: Nate Bolt has reminded me that ethnio is a good tool for recruiting your own users, including the ability to intercept users live off your own site. I’ve used it a few times, and it gets my vote.

These days we all know how easy it is to record usability testing sessions on a desktop computer. You can use Silverback on a Mac ($69.95), Morae on PC ($1,495) or you can try one of the many other screen recording tools on the market today.

If you need to record research footage from a device that doesn’t support software recording (e.g. a Kindle), then you’ll need to point a camera at its screen. This is simple if you don’t want picture-in-picture: you can make a sled like this, mount a webcam on it and record the footage using any free recording app (like Quicktime). If you want picture-in-picture, then it’s a little more complicated but still doable. On a Mac, you can fudge Silverback to record from a second webcam by using an app like macam to display that webcam’s footage on your desktop. If you’re on PC, you can use AmCap ($29) in a similar manner or Morae in its “mobile device study” mode [PDF].

However, specialist UX research agencies often use “Pro” video production hardware for multi-camera recordings. Even though this is far more expensive than software, the results can be better – often giving higher frame rates, better resolution, and less chance of drop-outs when transmitting live footage.

Speccing up your own hardware rig can be an intimidating prospect. Luckily, my friends at Amberlight (a London-based UX research consultancy) have agreed to share the specs of their rig, so you can create your own.

Before I start, I want to preface this little instructable with an important point: the quality of your research is determined by the skills of your team, not by the recording equipment they use. The point of a rig like this is simply to ensure that stakeholders can watch the live session clearly from another room, and so that good quality highlights can be shared. This helps with buy-in and decision-making, but don’t loose any sleep if you can’t afford it.

The first component we have here is an ELMO P30s document camera. You may have seen this sort of thing on lecterns in lecture theatres. It costs about $2000, but with the price tag, you get a good lens, good resolution (1280 x 720) and a high frame rate (30 fps).

Some researchers don’t like document cameras, saying that they cause participants to hold the test devices in an an unnatural fashion, and that they often hold them out of shot or at the wrong angle for the camera. This is true, but document cameras are still useful because they are so general purpose. You can shove anything underneath one start recording straight away.

In the picture below, you can see the next component – an HD DV camera on a tripod, pointed at the participant’s face. This gives you the second video feed for your picture-in-picture output.

Both video feeds are run into an HD Video Mixer (a “TV One C2-6104a”, approx. $4200) shown below.

This video mixer is used to create the picture-in-picture (PiP) composition. It’s important to use an HD mixer so the resolution of your footage doesn’t get reduced at this stage. The audio from two boundary microphones is also mixed together, using an audio mixer (a Focusrite Pro Studio Audio Interface).

Next, we get the picture-in-picture footage output as HDMI, which is fed into an HDMI splitter (pictured below). One feed is sent to a recording device, the other is run into another room in the building for live viewing.

Since HDMI cables are expensive and don’t work well over long distances, the cable is fed into a CAT extender (pictured below), so that the signal is transmitted over a standard ethernet cable.

In the picture below (front right), there is a small HDMI hard disk recorder (an Atomos “Ninja”, approx. $1000). This little thing takes the video input and records it directly to hard drive, no laptop need. Alternatively you can record your footage onto a PC or Mac using an HD Video Capture Card.

Also of note in the Amberlight lab is a couple of femtocell boxes, pictured below. These are mobile network signal extenders, so they ensure good 3G coverage within the lab. To test low connectivity they simply turn these boxes off.

So that’s pretty much it. Huge thanks to Amberlight for being so open and willing to share! If you have any suggestions about better/cheaper/different rigs then please post them in the comments below.