The focus of the Demand Validation is Product Management episode is about answering the second question of these two standard questions:
- Is this product useable?
- Do people want to use it? – This is Demand Validation
The guest is Steve Cohn, Founder of Validately.
The rest of this blog post contains pretty much raw notes. The notes contain a mixture of direct quotes from the podcast, my own thoughts, and sometimes ideas that came to mind while listening.
Notes
Steve Cohn got a lot of false positives. It begs the question: “How is great innovation brought to market?” It’s relatively easy to define and validate the problem space.
There is a gap between “Is the problem worth it?” and “Solution you have in mind.” Do people want to pay for or talk about your solution?
In comes Demand Validation. Validate the solution space. Figure out ways to run rapid prototypes.
Back to the false positives mentioned above: 50% of the features go unused by customers. “They passed the internal sniff test.” 30% of the work that a product team does is rework. Product teams ask customers “would you use this?” which is a bad indicator of demand.
People are not misleading you on purpose. There’s a gap between the hypothetical desire and committing to the solution. The “cost question” is better. Until you make saying yes cost something, their answer is meaningless. It forces the consumer to make the mental tradeoffs. For the “free products”, instead of costing money we go with time and reputation. So, the three things of possible interest are:
- time – Is the problem deep enough to give their time for free to help you “create” the product?
- money – Money comes in prepaying or signing a contract.
- reputation – Social sharing is the reputation.
If people care about the problem and are compelled by the solution, they will work with you to create your solution.
Money comes in prepaying or signing a contract. Kickstarter is an example for consumers. For businesses, you can pre-sell all the time.
If you get a “no it’s not worth their time / money / reputation”, ask “why.” Knowing why is important. Customers are good at telling you what is not compelling about what is in front of them.
Backlog Prioritization
Product Management is harder at bigger companies. Some examples include marketing, finance and executive. Marketing is saying there is a feature buzzing out there in the marketplace and we need to do this. Sales say a big customer needs this. Finance says where is the ROI for this feature? Shouldn’t you build another one? Executive says they had a great idea in the shower.
There is a saying mentioned: If we have data, let’s go with data. If we have opinions, go with mine.
Solution: Do Demand Validation Tests at the prototype stage with your backlog. We’re talking about rapid experimentation and getting the data. Then you can respond with the data. Yes, we prototyped it. We tested the idea against the customers and this is what they said. It went poorly so why should we build it? If it tested well, build it ASAP.
A feature prioritization process built on opinions will fail. You need to prioritize based on data.
MVP vs MVE
The true concept of the MVP is brilliant. The problem is when there is a misunderstanding about MVP. He likes to use the term MVE, Minimal Viable Experiment. We’re learning through this experiment. It’s understood that an experiment yields learning. When people hear product, they think growth. So, instead of MVP let’s call it MVE.
We’re in the home-run business. We have to consistently hit home-runs. There’s only one way to do that. The home-runs strike out a lot. So, get lots of trips to the plate. Take lots of swings. Run experiments. You will get a lot of strikeouts. However, you have to do that to get the home-runs. Give the team flexibility to try crazy things.
When Do Product Managers Know They’ve Hit It Big?
Alpha-UX are thought leaders on this. Spit-testing prototypes. They use high volume. They shoot for statistical significance.
The thing you want to understand about validation is that it’s not a guarantee. Lean is about reducing waste. Removing things that definitely won’t work. Prioritize those things that have the higher probability of achieving market success.
Summary
- Do Demand Validation Testing
- Experiment often
- Use data to help prioritize the backlog
- Use the term MVE instead of MVP to set expectations
