90 percent of everything : Usability Blog

Written by Harry Brignull

Back in January, Paul Thurston (@paulthurston) and Nick Marsh (@choosenick) gave a great talk on Service Design at UX Brighton. They make some really interesting points about the differences between UX & SD, and strategic (“thinking”) vs tactical (“doing”) work. Here are the slides:

Using a feed reader and can’t see the slides?

Huge thanks to Nick and Paul for putting this together!

I think we all “get” simplicity these days, but nevertheless this quote from Paul Graham really sums it up:

It seems strange to have to emphasize simplicity. You’d think simple would be the default. Ornate is more work. But something seems to come over people when they try to be creative. Beginning writers adopt a pompous tone that doesn’t sound anything like the way they speak. Designers trying to be artistic resort to swooshes and curlicues. Painters discover that they’re expressionists. It’s all evasion. Underneath the long words or the “expressive” brush strokes, there is not much going on, and that’s frightening.

When you’re forced to be simple, you’re forced to face the real problem. When you can’t deliver ornament, you have to deliver substance.

From Paul Graham’s article Taste For Makers (Feb 2002). Paul Graham is one of the partners at Y Combinator.

Romeo and Juliet by Frank Dicksee

When a barrier is placed between a person and their desires, those desires become intensified. This is because the barrier prevents them from experiencing their desires in the flesh – warts and all – and instead causes them to long for their fantasy of the desired object, rather than the object itself. Fantasies are more alluring because we gloss over the bad bits, and generally the desired object takes on a mystical perfection.

This is known as the Romeo and Juliet effect. I think everyone’s familiar with the idea of it applying in romance between people, but it’s actually a useful concept to extend to design.

When creating something, it’s easy to fall in love with your ideas while they are at a nascent, intangible stage. Potential and promise are a heady mix, and it’s easy to hang around at this early stage far longer than you should do. You end up having meetings where you and your stakeholders add more and more layers onto the concept. And why not? In fantasy land, everything’s possible. Criticisms can be dodged with a swift verbal replies. Ideas are bullet proof, shape-shifting, teflon-covered wonderments.

Everything’s peachy except for one thing: it’s all bullshit.

Iterative, low fidelity prototyping is the perfect remedy, which is exactly why it’s so popular these days. You make your ideas real early on, so you can reveal the warts and deal with them as soon as possible.

The thing is, even though you may be doing your best with your lo-fi prototypes, are you revealing all the warts you should be? Are you using Lorem Ipsum when you should be giving your best shot at copywriting? Are you just designing the “happy path” through your system and forgetting about all the crucially important error conditions and error messages? In other words, even though you’re doing all the “right” things by prototyping and iterating, are you still leaving out too much detail and falling foul of the Romeo and Juliet effect?

It’s great to start off with sketches, and to build sketchy wireframes at the beginning of a project. But don’t stay in that phase. Whatever you leave out, you leave to chance.

I’ve always liked this story about Betty Crocker and how General Mills took such care in designing the experience of making a cake. They’d been espousing speed and ease in the kitchen since 1931 with products like Bisquick, but this story originates from a cake mix they launched in 1952, almost 2 decades later.

To quote from Finding Betty Crocker by Susan Marks:

“At this time, the company was still refining their approach to marketing. While they sought to promote a quick and easy product that still retained a “fresh, ‘home-made’” quality, ‘the market was slow to mature’ (p. 168). The company called upon the market research of Dr. Burleigh Gardner and Dr. Ernet Dichter, both business psychologists:

‘The problem, according to psychologists, was eggs. Dichter, in particular, believed that powdered eggs, often used in cake mixes, should be left out, so women could add a few fresh eggs into the batter, giving them a sense of creative contribution.’

As a result, General Mills (who own Betty Crocker) altered their product, abandoning the powdered egg in their mixes. The requirement to add eggs at home was marketed as a benefit, conferring the quality of ‘home-made’ authenticity upon the box cake mix. (Whether using fresh eggs instead of powdered eggs actually enhanced taste was beside the point.)”

I like this story because it nicely sums up the progression in thinking from ‘just designing for ease and speed’ (old-school usability) to ‘designing an entire experience’ (new-school experience design).

Old-school Usability espouses the idea that user activities are onerous tasks that they want to get out of the way as soon as possible. While this is true in some cases, usability is now widely understood to be more of a hygiene factor – something that can cause dissatisfaction if missing, but its presence cannot take you beyond lack of dissatisfaction.

Image credit: Jonny Baker

By now you’re probably thinking “Yeah, I get it already – usability is just one of the components of good design”. So let’s move on to my second point. In the Betty Crocker example, the psychologists realized the customer wanted to play the role of a successful home-maker and cook. We could even go so far to argue that their customers may have felt societal pressures to perform this role well. The egg, therefore, becomes more than an ingredient, and more than just an extra pleasurable step. It becomes a prop, enabling the customer to play a social role. Goffman’s theory of Dramaturgy explains this to some extent. It’s worth reading about if you don’t have much background in Social Psychology or Sociology.

Now, I’m not saying that knowing a bit of sociology is going to make you into a great designer. However, it does give you a nice underpinning perspective for when you carry out field research.

Amendment: this post has been edited for typos: Thanks Carlo!

Unless you’re a mathematically-minded person, it’s easy to get muddled up when people talk about conversion rates and conversion rate “uplift”. Here’s a quick explanation:

• If you had 1000 users visit your site yesterday, and 1 of them made a purchase, you’d have a conversion rate of 1:1000 (also referred to as “0.1%” or “0.001”).
• If today you had 1000 visitors and 3 of them made a purchase, what’s the uplift? It’s 200%. In other words, the uplift is a percentage of the increase, not the increase itself (which is 2:1000 people more).

200% sounds HUGE, doesn’t it. Companies like to talk about it that way for two reasons. Firstly, if the underlying numbers are poor (as in this example), it makes them sound impressive. It doesn’t sound quite so good to say “1:1000 more users made a purchase today”. Secondly, they might simply want to hide the details from their competitors. They could have great figures, but don’t want their competitors to know exactly how they are doing. This is understandable – it’s often very sensitive information.

Here are some questions to consider when you’re shown percentage uplift figures within your own organization:

1. Establish whether AB / Multivariate testing was carried out. Some people talk about uplift when comparing two time periods, e.g. last month vs. this month. The problem with that is time-based variation. For example, you will sell more ice-cream in the summer months, regardless of your sales pitch.
2. Establish the period of the test. AB / Multivariate testing randomises the samples on the fly, so it removes the effect of “before and after” time-based variations. However, the test itself is still time-bound: if your test was run on christmas day alone, then you should have doubts about whether the findings will generalize to every day of the coming year. Why? Because you will be getting a different sample of users who will be behaving differently to normal.
3. Get access to the real numbers, not just the uplift percentages. Ideally, take a look at the reports produced by the testing platform itself. For example, GWO produces unintimidating reports.
4. Find out what the statistical significance is. If you toss a coin twice today and then do it again tomorrow, you might get a 200% uplift in incidence of heads. On the face of it, you might be tempted to generalize that your technique is much better today. Of course, that’s plain wrong – you just need a much larger sample size. (Check out User Effect’s split test calculator)
5. Find out how the conversion goals are defined. For example, is the goal “the number of people who enter the checkout process” or “the number of people who actually complete a purchase”? There’s a big difference.
6. Find out what the research objectives were. Why did they run the study? What were the hypotheses? Were the alternative designs based upon any prior research or theory? Have the findings confirmed expectations or confused them? Answers to questions like this are solid gold – they will extend your organization’s understanding of the relationship between design, psychology and user behaviour within the unique context of your own offering.

Lets look at an example: 37signals reportedly achieved a 200% uplift in an A/B test on Highrise registrations when they tested a call-to-action that read “Sign up for a Free Trial” against an alternative wording: “See Plans and Pricing”. (Read more on Ryan Carson’s blog.) Because it’s 37signals, we can assume they’ve got ample traffic and good conversion rates to start off with, which implies that their uplift is probably very impressive. But hang on a minute – if we’re put our scientist hats back on, we’ll remember that this is only an assumption. We don’t know the answers to many of the questions above. We haven’t been given enough information to evaluate the meaning of the percentage given. It’s intriguing enough for a press release, but it wont wash when communicating within your own organization.

Oh, and by the way, just because they got a 200% uplift for that copy change, it doesn’t mean you will. You can’t reliably generalize AB test results between sites – you have different users, in different states of mind, looking at different page designs. It might even have the opposite effect in your context, so it’s worth doing some user testing of your own. Don’t forget to report it properly, though!

A few weeks ago, I invited Filip Healy and Ian Collingwood of Amberlight down to give a talk at UX Brighton. The topic they chose was “Mental Models, Service Design & The Problem With Convergence”.

What do consumers expect when they buy a bundle of services like Internet, Telephone and TV from a converged provider? Is there a gap between their expectations and what they actually get? And what can the providers do about it?

Using a feedreader and can’t see the slides?

Many thanks to Filip and Ian for all their great insights, to all the other UX Brighton organisers, particularly Danny Hope, for making the event happen, and to The Werks for providing the venue.

UX Brighton is a free event that happens on the second tuesday of every month. The next one is a film night at The Eagle. Coming up this year we also have Human Factors International, Bunnyfoot, CX Partners, We are: London and various others pencilled in. If you want to be on that list, get in touch!