The ultimate guide to Product Reviews: good vs bad review
What makes a truly great Product Review?
Let’s look at Solve stage (vs. Scale, Sustain): MVP! = minimum viable process
How to form a product review questions:
- Who is the target customer
- What is the goal (acquisition, engagement, retention..?) Pick one!
The goal must be outcome focused and measurable (quantitative & qualitative)
3. Once completed, remove yourself and look at it from various angles: engineering, designer, the product overall perspectives…Get the relevant team involved incl. user research and design leads.
4. Check for the alignment, i.e. anchoring about the same exact experience which company wants for the customers.
ProTip: usually the problem is talking about generalities.
4. Then check against the company goal
- is this going to increase the conversion of the specific type of user through this flow
- is this going to reduce the contact rate
- is this going to increase the adoption of the new feature
Scale stage (vs. Solve, Sustain)
ProTip: Product lead owns the agenda. Lead with the data, metrics, frameworks. Get the relevant team involved incl. user research and design leads.
What are the biggest mistakes that product leaders make when leading product reviews? i.e. Bad product review:
- Lack of clarity in
- what’s the agenda
- is it decision making
- why everyone is there for?
2. Macro level
- slow velocity
- team doesn’t convey strong conviction and trying to get blessing from the leadership (the team should stand up for conviction and articulate clearly their ground)
3. Vibe
- leadership giving directives instead of feedback
- PM should set the tone! Explain how to interpret each one’s feedback
Who should be invited to the product review? How does this change with scale? How does this change in a world of remote work and Zoom?
Initial product reviews should be with the design and product team.
Engineers there to discuss the cost of smth. Also they assess for
- inefficiency
- missing APIs
Disney had 3 rooms:
most amazing things we can come up with
slaughter house — what’s wrong with those ideas
reconciliation room to bring all together.
Who should set the agenda for the product review?
PM! Have to oversee that everyone capturing their part.
The difference between success and failure — capturing things.
Focus group went wrong -> we don’t need this! Getting them back on track → Ask what they are struggling with!
3 different archetypes of reviews
- Early-hypothesis stage. Feedback on strategy, early thoughts.
- Experiential product review with a Demo
- Post-launch review. To understand how is it working?
All attendees can be split to:
- Team — to get feedback from leadership
- Leadership — to check alignment on direction, dynamics of the team, velocity
How do you advise on prioritisation as there are so many ideas and feedback?
Good problem to have! :)
Do a debrief with the leadership team after the review. What’s the most valuable feedback collected today? Then putting it in few bullets and passing it as artefacts to the team to act upon.
You cannot react to everything, you filter and if have to, ignore some.
That’s how leaders can assign accountability and ensure that the follow-ups from product reviews are executed on.
Credit: 20Product and
Scott Belsky is an entrepreneur, master of product reviews, author, investor, and currently serves as Adobe’s Chief Product Officer and Executive Vice President, Creative Cloud.
Tony Fadell, often referred to as the father of the iPod is one of the leading product thinkers of the last 30 years as one of the makers of some of the most game-changing products in society from the iPhone and iPod to more recently founding Nest.
Lenny Rachitsky is one of the OGs of product, having spent over 7 years at Airbnb as a product lead he left to start his newsletter, find it here.
Kayvon Beykpour is one of the most prominent product leaders of the last decade. For the last 7 years, Kayvon has been at Twitter where he led all of the teams across Product, Engineering, Design, Research and Customer Service & Operations.
Aparna Chennapragada is Chief Product Officer @ Robinhood, the company revolutionising consumer finance with commission-free investing.
