What if Usability Tests Don’t Work?

5 min.

Could you ever think that the change of a single button may bring a $300 mln revenue growth? Yet it may. I was impressed to have heard about a fateful change of the checkout form that brought unprecedented profit to an e-commerce website. By replacing the button “Register” with the “Checkout without Registration”, the company released first-time shoppers from the obligatory registration routine, allowing them to proceed directly to what they came for.

Fantastic though the story may sound, usability testing repeatedly proves its paramount importance, especially for traffic-dependent B2C web services providers, as it helps to find and fix problems hampering conversion rate, verify concepts and even take some inspiration from the target audience. At present, there is a plethora of UX testing methodologies, all of them agreeing at exploring the behavior of the target audience with respect to the following aspects:

Aspects of Testing

However, what if these methodologies don’t reveal users’ actual needs, or the findings are not enough to make decisions? Let’s try to go deeper than the warned-out questionnaires and interviews.

Involve Broader Audience

There is a solid conviction that your target group’s representatives are the best reporters on the product’s usability. Of course, it is true, when you want to kill 2 birds with one stone – find UX bottlenecks and collect opinion from the target group. Remember, usability research is about catching and eliminating usability defects, not about understanding your target audience, so you do not necessarily need to involve mainstream users to find and fix usability problems.

The best-kept secret of usability testing is the extent to which it doesn't much matter who you test. For most sites, all you really need are people who have used the Web enough to know the basics.

Steve Krug

For example, if we take a consumer electronics online shop, the elderly and teens located on the margin of the target audience are a valuable source for usability research. Their physical, sensory and cognitive abilities due to age can uncover the trivial mistakes you would never catch with the mainstream users.

TArget Audience

Similarly, there is a definite sense in engaging versatile public with diverse professional background and experience, and even potential haters of your product.  Although recruiting the participants is sometimes challenging, you are likely to reveal unexpected UX bottlenecks and widen your target audience by making the interface logical, simple and accessible for everybody.

Go Beyond Words

If you google “Usability Testing”, you will probably see a lot of workshops promoting contextual inquiries and feedback forms of different kind as the most fruitful testing tools. Yet, there is a huge gap between what people say and what they actually feel, when they are aware of being studied. In social psychology, this phenomenon is called the Hawthorne Effect – aspiration for improving one’s behavior under observation.

Unnatural environment, careful observation and straightforward questions make people feel strained and give misleading answers. Moreover, they usually can’t explain their motives accurately in words. For this reason, I recommend prioritizing the participants’ natural behavior and uncontrolled reaction, when making conclusions about your product’s usability.

Click Tracking & Eye Tracking

Probably, the most reliable way of estimating, whether your message reaches users. Evolving from click tracking, the method consists in capturing the user’s eye movement during interaction with a web site and generating the so-called heatmaps demonstrating the degree of human sight fixation on every particular object. All you need for eye tracking is a web camera and specific analytical software.

Screenshot from uxcam.com

Quick Memory Test

Users’ first impression can say a lot about the efficiency of the information architecture and attractiveness of the UI design.  Show a screenshot to the participant for 5 seconds before it is removed. Then ask the person to recall, what elements outstood most, quickly and without thinking it over. During the 5 sec observation, the participant will get just a glimpse and provide valuable feedback for evaluating your content layout.

Make it More Natural

Despite usability research is an inherently artificial process, it is likely to give better results, when organized as close to reality as possible. The strains of usability testing can be relieved by the following actions.

  • Minimize the observer effect. Use unobvious or hidden cameras, take as little notes as possible, use sound recording instead. If it is possible, let users perform tasks alone in an isolated room.
  • Conduct field studies. People feel far more comfortable, working at their familiar environment, using their own devices. Observing users at their working place or home eliminates a lot of lab-based artificiality.
  • Launch unmoderated tests. In some cases, you do not necessarily need to know the users’ background, to collect valuable feedback. Let people remain incognito filling automated tests, so they could speak frankly, what they think.

In a nutshell, when usability research ceased providing grounds for UI enhancement, I recommend you to revise the 2 major points of your strategy – who you test and how you test. Certainly, you will bear extra costs, but the price of not knowing your users is far higher.