By Dave Bailey
Common wisdom says to speak with potential users to check that people want something before building it. So I did — but it didn’t go according to plan.
Several years ago, I created a mobile app called Spotnight that showed you every nightlife event across a city on any given night.
The idea came about when I was with a group of friends at a bar, trying to figure out where to go next.
“Wouldn’t it be cool,” someone said, “if there was an app that listed all the different night spots and told you where your friends were going?” The response was an exciting and unanimous, “Yes!” I’d found my startup idea.
Getting Out of the House
It was 2012 and I was living in Rio de Janeiro. There was no shortage of potential users around me.
I asked questions to dozens of my friends, and their friends, and random people that I’d meet.
Here’s roughly how it went:
Q: “How often do you miss great nights out because you’re unaware they’re happening?”
A: “Pretty much all the time!”
Q: “What do you think about an app that shows you all the parties — and where your friends are going?”
A: “Sounds really awesome!”
Q: “Would you use an app like that often?”
A: “Definitely! In fact, I’d use it to find out where I’m going this weekend — I don’t have any plans yet! Where do I sign up?”
There was clearly a market for the app, and excitement fueled me as I worked night and day to build it.
After three months of hard labor (okay, it was probably more like six months), the Spotnight app arrived on the app store with a big cheer from my friends.
Everyone downloaded it and used it. The end.
Reality Check
Wait a minute. Not so fast.
Was there a problem with the activity logs? Far fewer people were continuing to use the app than I’d expected. More baffling still, the people that said they’d use it most weren’t actually using it at all!
I got in touch with our early users to find out what was going on. Each time, I heard a similar excuse: “I haven’t had a chance to use it yet. But it looks really great and I definitely plan to use it soon.”
Were they lying? Or was something else going on?
The Mom Test
I discovered a technique called ‘The Mom Test’, devised by Rob Fitzpatrick, and it became clear that my friends weren’t to blame. I’d been asking the wrong questions.
The premise behind The Mom Test is that everyone is innocently lying to you all the time, for a wide variety of reasons.
And the person who lies to you most is your mother because she loves you and doesn’t want to hurt your feelings.
To pass The Mom Test, you have to ask questions that even your mother can’t lie about.
Rob advises:
1. Talk about the customer’s life instead of your idea.
2. Ask about specifics in the past instead of generics or opinions about the future.
3. Talk less and listen more.
My questions had focused on generic statements, hypotheticals, and opinions, not on facts.
But it turns out that the best way to predict future behavior is to study past behavior — not to ask people for their best guess.
Generic Questions are Bad!
“How often do you…”
“Normally, what do you . . . ”
“In a typical week, how many . . .”
Questions that include words like these are highly unreliable.
For example, ask someone how often they go to the gym and they might say, “I go twice a week.”
But ask the same person specifically when they last went to the gym, and it might turn out that they haven’t been in a month!
Rephrasing generic questions to refer to previous events or actions is likely to yield more reliable answers.
Asking for Opinions is Worse!
Questions that begin with, “What do you think of . . .” will give you opinions, not facts.
Opinions sound a lot like validation but they aren’t — and this makes them very dangerous.
When it comes to businesses and products, even seasoned experts are wrong most of the time.
Just speak to any angel investor, or serial entrepreneur, or product manager about their success rate.
I’d recommend ignoring opinions completely. Stick to the facts when you’re making product decisions.
Asking Hypotheticals is the Worst!
The worst questions of all are those which come so naturally when validating an idea: “Would you?”; “Could you?”; “Will you?”
We are all dreadful at predicting how we’ll behave in the future. A simple audit of last year’s New Year’s resolutions should illustrate this.
Hypotheticals are guesses and it’s an awful idea to build a product or business based on a guess.
Always Ask About Past Experiences
The whole point of interviewing customers is to gather facts about them, not your product.
I ditched my old questions and started with a different question — one that avoided guesswork altogether:
“Would you walk me through the last time you [performed an activity]?”
In my case, I wanted to know about the last time people went out with friends.
My goal was to discover specific triggers, motivations and behaviors.
I was hunting for insights into why they would — or wouldn’t — use my product.
– What triggered this?
– What were you hoping to get done?
– When was this?
– Where were you?
– Who was with you?
– What exactly did you do?
– What happened afterwards?
At every stage, I stayed curious and asked more questions to deepen my understanding.
– Why does that matter?
– Was that frustrating?
– Tell me more about that?
– What makes you say that?
– Can you help me understand your thinking there?
– What other options did you consider?
– What else?
After five or six interviews, new patterns started emerging.
For example, it turns out that finding somewhere for a night out is most problematic while in a group. And we’ve all been there — someone makes a suggestion, it’s accepted, and it turns out to be awful.
This small insight had a completely unexpected consequence.
GroupThink is a tendency for groups to ignore external influences and use their existing knowledge to make decisions.
My app was an external influence, so groups would be unlikely to use it in practice. No wonder people were using it less than even they expected.
Epilogue
Ultimately, Spotnight pivoted to another idea. In retrospect, I didn’t need to build the product to learn the customer insights that undermined its usefulness; I just needed to ask the right questions from Day One.
If I’d focused on my customers’ behaviours rather than their words, I could have saved a great deal of wasted time and effort.
It takes a lot of practice to avoid generics, opinions and hypotheticals.
We naturally want to know if a potential likes the idea, or if they would use it. But no matter how easy they are to ask, and how exciting their answers sound, through no fault of their own they are not to be trusted.
This article first appeared on Medium under the same headline
