90 percent of everything : Usability Blog

Written by Harry Brignull

“The first business computer anyone can learn to use overnight.” (1984)

Try to think back to the first time you used a brand-new, ground-breaking, disruptive piece of technology. For me, I’ll never forget sitting in my bedroom in the early 1990s, going through the training application on my brand new Macintosh Classic. It explained how to lift my mouse when it got to the side of the mouse mat and position it back into the centre. It taught me the different functions of single-click and double-click, and how to select icons by clicking and dragging.

It was so new and so different, it didn’t actually feel intuitive to me. I could see that the Macintosh GUI was going to be far easier to learn than DOS, but at the same time there was still a fair amount of work involved. Ultimately, I didn’t care – it promised something so useful that the learning it was trivial in comparison to the benefits I stood to gain.

Now think back to when you got your first iPhone. Admit it – it wasn’t instantly easy to use when you first got it out of the box. Work was involved in setting it up. Sure, it was easy to unlock, flick left and right, start apps, and so on – but actually using it to get work done, that took some learning. Cancelling auto-corrected words. Copying and pasting. Creating bookmarklets in Safari. Entering the Euro or Yen symbol from the keyboard. Learning the work-arounds for having no file manager. They were lots of little mysteries and idiosyncrasies to start off with. Although it feels intuitive now, that’s learned intuitiveness, which, if you think about it, is a strangely wooly concept. Any interface can feel like second nature with the right amount of effort spent learning it.

My point is that disruptive technology is highly likely to be unfamiliar and will therefore have a notable learning curve. This raises some interesting problems for designers. For example, if your product has a painful learning curve, how can you convince people to invest time and effort in overcoming it? Back in 1984, Apple had to let people take home a Macintosh completely free for 24 hours, along with a 16 page instruction booklet (see Macintosh Ad above). Think about that for a moment – imagine designing something that required 24 hours of exposure for end-users to overcome bafflement and “get it”, let alone become a competent user. Web designers these days have it easy in comparison.

When the telephone was invented, Bell Co. tried to sell the patent to the British Post Office but they weren’t interested. The Chief Engineer famously responded “…we have plenty of messenger boys”. Bell then realised, like Apple did 100 years later, that people had to experience real usage of their product in order to “get it”. So Bell developed an aggressive adoption strategy to get telephones into peoples’ hands. They put phones in hotel rooms for calling the front desk, in offices as a replacement to intercoms, and near lunch counters in diners – a brilliant idea that not only got people to use them but also ensured people saw others using them in real life. It’s interesting to consider that today, Apple’s TV ads draw upon the same underlying strategy – the ads are presented like actual product demos. The viewer gets to vicariously experience real usage. The same goes for the hands-on nature of Apple Retail Stores, where anyone can walk in and play with any of the hardware for as long as they want. It’s not an act of kindness, it’s an adoption strategy.

Another problem you might face is knowing whether your product has a noticeable learning curve because it’s ground-breaking or just because it’s badly designed. The good news is that you can approach this with a simple heuristic: if it requires effort to learn, always start by assuming it’s just badly designed and let user research be your guide. Bear in mind, you’ll need to engage in longitudinal research to evaluate it properly. Typical usability testing, MVT and analytics doesn’t cut it because the time-frame is usually too short. You need to look at adoption over a period of weeks or months to get a true picture. Open and closed Betas are easy enough to implement with webapps, but physical products bring a whole different set of challenges.

What’s necessary is that you give your participants enough time to experience the strengths and weaknesses of your product in their own lives, and let them work out for themselves whether the benefits outweigh the costs of learning how to use it.

At UX Week 2010, Facebook Product Designer Adam Mosseri gave a presentation called Data Informed, Not Data Driven. It’s an excellent talk and Adam gave some really good examples demonstrating how data-driven design can take you into “local maximum”, which is essentially a design cul-de-sac caused by a blinkered over-emphasis on Analytics and MVT. In this situation, no matter how many small metrics-driven improvements you make, you’re unlikely to ever make a big creative leap to a substantially better design. Joshua Porter explains Local Maxima really well in his article The Local Maximum on 52weeksofux.com.

Local Maxima diagram by Joshua Porter

With data-driven design you can end up wasting time trying to reach the peak of the hill on the left, when there is actually a huge mountain of improvement over on the right that you’re not even aware of because it involves a radically different design approach. Let’s look at two real life examples from Facebook. Quoting from Adam Mosseri’s UX Week 2010 presentation:

“…we created a team we called the engagement team, which was tasked with understanding engagement and increasing it significantly [...] our first attempt at quantifying engagement was “R.A.W.” – reads and writes. [...] Writes are creations of either objects or connections between objects. And reads are what they sound like: reads of that information.

And so we just decided to treat all writes equal and all reads equal, and start to try to optimize for that. We did this over the past few months, and we ended up with products like comment liking. Comment liking is what it sounds like. [...] [It] allowed you to quickly and easily like a comment. [...] This RAW metric was wildly successful. It produced an 11 percent increase in likes throughout the entire system. [...] But there was a feeling within the team [...] that this really might not be the best thing to optimize for. We sort of got what we asked for. This type of write, the fact that you like the comment, is obviously less valuable than you telling us that you had a baby or that you switched jobs or that you moved companies. So clearly, all writes weren’t created equal, and we started to struggle with this.” – Adam Mosseri

This is a great example of how Facebook started off with a poor KPI, and if they had blindly stuck with it, they would have rapidly found themselves in a local maximum where they were optimising for a pretty line graph rather than for the overall user experience. Comment liking is such a low value write activity that it would have simply been a source of noise, covering up the impact of high value writes elsewhere in the system (i.e. other kinds of content publishing). In other words, the RAW metric was raw by nature as well as by name – it needed splitting down into some meaningful consituents. Back in October 2010 the Facebook engagement team was working on developing new metrics to supersede RAW – but the very fact that they recognised the problem means they were already half-way there. Adam goes onto elaborate:

“As we scale, a division of labor becomes invariably more intense, and you have different people representing different interests. We have a Photos team; we have a growth team; we have an engagement team; we have a News Feed team, etc. And all of these teams optimize in good faith for their own interests. But sometimes these interests can be sort of opposing or distracting from each other, and sometimes you can get lost in the specifics of a decision and sort of miss what we think of as the big picture.” – Adam Mosseri

He then goes on to give an example of how this caused a similar “local maximum” problem in the design of the Facebook homepage. Back in early 2008, the Facebook homepage had all of the primary navigation items on the left-hand side. Importantly, this was how you navigated to applications, which of course are an important source of revenue. Later on in 2008, there was a radical redesign of the homepage and the applications menu went from being an exposed, highly visible list (below left) to a drop-down menu (below right). This resulted in a significant decrease in traffic to applications. They tried a few different tricks to increase usage of applications within this “top navigation” page layout, and some limited improvements were delivered.

So, it turned out the top navigation approach (shown above right) was causing a local maximum, and Facebook had to basically bin that layout and create a completely new design in order to deliver an application usage uplift. The key point here is that the analysis that got them out of this rut was not data-driven. It took a creative leap to conclude that the uplifts they were getting weren’t good enough, and they needed to try something completely different.

“What we were doing here is we were optimizing for a local maximum. Within this framework, there was only so much traffic we could funnel to applications. And what we needed was a structural change. Our premise was off. Our interests were leading us down the wrong path. We didn’t realize it [...] we were optimizing for something locally, and we needed to be somewhat disruptive to sort of get out of it.” – Adam Mosseri

It’s fantastic that Facebook have the guts to share information like this, and let’s hope it continues. I guess there’s a certain level of confidence you get from from having 500 million active users that spend over 700 billion minutes per month using your site.

Some quotes in this article were paraphrased for readability. Read full transcript of quoted items