- 5 User Testing Mistakes That Even Experienced Startups Make - September 21, 2017
“We had a big idea and we freaking went for it,” Kash Shaikh, Besomebody CEO, wistfully told the company’s community earlier this year. The Boston-based business, Shaikh continued, would then shut its doors the very next day.
It’s a story that no startup wants to hear. After appearing last November on ABC’s “Shark Tank,” Besomebody briefly gained fame by helping users explore their passions. Unfortunately, however, it had forgotten something important: user testing.
For the model to work, according to Shaikh, tens of millions of people needed to book experiences annually. Had Shaikh done his homework before betting big, he’d have discovered that the demand simply wasn’t there.
Test early, test often
Unfortunately, Besomebody’s story isn’t all that surprising. Startups fail for all sorts of reasons, but product-market problems are chief culprits. Guess what topped the list when Harvard Business Review studied the top 40 reasons why products fail? No market research.
In a world full of factors outside of your control, user testing is your startup’s super power. It provides real insights into whether customers want what you’re selling and, if not, what they might want instead. In fact, companies that practice “human-centered” design realize 228 percent greater return on investment than the S&P 500.
The bottom line? Test early, and test often. The more you do, the better you’ll be able to craft your product to meet users’ needs and motivations.
Top user testing mistakes
User testing is essential, but even experienced entrepreneurs sometimes get it wrong. When putting your product to the test, avoid these five common mistakes:
You skipped user testing entirely
Skipping user testing is beyond risky. Without it, you’re basing your product’s UX on your own opinions or, at best, those of an imaginary user. Neither is a reliable guide.
Icons8, a digital design startup, recently learned this lesson. Hoping a modern spin would spur sales, it redesigned its website’s “Request Icon” button. But to its chagrin, icon requests dropped by 47 percent. Icon8 misunderstood what its users wanted because it didn’t bother to ask them.
Budget for user testing from day one. Pitch it to stakeholders as a non-negotiable part of product development. Rather than focus on cost, however, lead with your objective: to confirm product-market fit and create a user-driven UX.
You didn’t talk to the right users
When it comes to user testing, quality trumps quantity. Sample size doesn’t matter if, for example, you tested a children’s educational app on business executives.
This might sound like an easy mistake to avoid, but it’s not. If you’ve got an existing user base, start there. These people are already incentivized to make your product better. Before doing any formal testing, interview people to ensure a representative sample. Eliminate outliers from your test group. If your users are geographically distributed or otherwise difficult to reach, consider conducting the test online.
What if you don’t have existing users? Describe the characteristics, such as skills, experiences and demographics of people who might find value in your product. Then, recruit people who closely fit those characteristics. As before, don’t forget to screen out non-representative users.
You didn’t understand users’ underlying motivations
Even startups that test (including those that select the right users) don’t always design their tests correctly. They focus on peripheral details rather than the qualities that actually motivate people to use their product.
Users are driven by pain points and emotions. An app like Uber checks both of those boxes: It solves a problem (users need a way to get around), but it also speaks to an emotional need (people want to travel easily, spontaneously, stylishly and with frictionless payments).
Prior to testing, contact your customer support team. Ask what problems users mention the most. Develop a list of feedback to dig into more deeply. If, for example, users complain that they can’t find the login button, don’t write it off as a minor design flaw. What emotion does it boil down to? Are users frustrated and wasting time navigating your app?
You valued all feedback equally
Not all feedback is created equal. You want as much of it as possible, of course, but certain feedback will inevitably prove more useful than others.
If you test correctly, you’ll end up with a long list of problems and suggestions. Go through it with a fine-tooth comb and distinguish the meaningful ones. Ignore requests for features unrelated to your product’s core purpose. Highlight those based on emotions, which speak to users’ true needs.
How should you prioritize from there? Focus on issues that affect the widest group of people first (i.e. core functionality problems) rather than those only experienced by a subsection of users.
You talked more than you listened
It’s natural to be excited about your product, but don’t let it hamper your test. Some startups fail to create a realistic test environment because they’re so keen to talk about their product.
User testing is most useful when it models how users behave alone with your product. You want to see how they navigate, where they find value, and where they lose interest. If you have to show them how it works, there’s already something wrong with the design.
Start by briefly introducing your product. Next, explain what you’d like users to do. For example, you might tell them to download and start using your app. After that, embrace the silence. Speak only to spur conversation or to understand and clarify. Even if you’re asked for your opinion, reply with, “Well, what do you think?” The user’s opinion is the one that counts.
What happens if users don’t like your product? Don’t get frustrated; be grateful for the feedback. Industry research shows that every dollar spent on UX returns between $2 and $100.
Why the range? User testing improves every product. Sometimes, though, it’s the difference between Besomebody and being somebody with a product customers crave.