The focus of the User Science is Product Management episode is on User Science. It covers what User Science means, the tools involved and advice around implementing it. The guest is Brent Tworetzky, the Executive Vice President of Product at XO Group. Brent is on Twitter and Medium.
User Science is the study of users. It’s the combination of user research and user analytics. This directly aligns itself with interests that I have. Throughout my career and personal time, I have continuously sought to get closer to the end users of technological solutions. Through the power of enjoyable yet easy to use technology, I like to empower and enrich the lives of others. So, the topic of User Science is an area of deep fascination.
The rest of this blog post contains pretty much raw notes. The notes contain a mixture of direct quotes from the podcast, my own thoughts, and sometimes ideas that came to mind while listening.
Notes
Brent started as an engineer. He didn’t like the day-to-day of programming. So, he got involved in strategy oriented careers such as management consulting. However, he felt “removed from the action.” Product Management lets him put all the things he loves together into one role. As a side note, Brent goes into detail about the Product Management role in his article What Does a Product Manager Do?
Brent has his roots in Product Management from working with others who used to work for eBay, Netflix, and Google. With philosophies that came from those places combined, he has been exposed to different approaches that cover being business case driven, user research oriented, and the use of quick iterations. This all has been foundational to how he applies User Science.
Applying User Science at his organization answers the question of whether to continue to iterate on a product or to go a completely different direction. User Science is a combination of understanding the users’ intent and their user behavior.
Intent vs Action (Behavioral)
The tools involved with Intent vs Action can be expressed in this 2 x 2 grid that Brent provides in his article titled A Product Manager’s Superpower: User Science.
Product User Science Grid
Intent
The purpose of exploring a users intent is to understand what problems the users are trying to solve. Using the intent tools, one is trying to nail down the important problems to solve.
Action / User Behavioral
Once you have the problem to solve, it’s time to switch to the User Behavioral side of things. This is where you can do things like prototype and do usability testing. You can do A/B testing or bucket testing. You can also do aggregate analytics.
From these tools you learn things like:
- Do we have something that still works for what the user wants to do?
- Should we iterate on this or is there stuff to throw away?
Diary Study, A Special User Science Tool
The Diary Study uncovers actual user behavior over an extended period of time. To do the study, they collect a small group of users that agree to share their thoughts and challenges. For example, wedding planning. They check-in every week or every two weeks over a two to twelve month period. The diary captures the needs and challenges of the user each week.
Through this tool, they learn what tools the user uses, what problems they have, and maybe what ideas they might have. Through all of this, the Product Manager can discover whether or not the user has experienced the kind of problem that is being looked for. In other words, if the users are experiencing the same problem again and again, that’s an interesting thing to learn and maybe you’ve found a problem to solve.
Implementation Challenges of User Science
User Science is a field that needs to be learned. It doesn’t come naturally or by accident. When new Product Management team members join their team, they explain and walk through the related philosophies. They ask them to put the tools into their hands and practice right away on practice tests. These new members also get tutorials.
The new team member gets tutorials from the User Research or Product Analytics team. For A/B testing, they use Optimizely. For usability testing, they use UserTesting. They run practice tests and get feedback about those tests. This lets the new team member get familiar with the tools and gain muscle memory about those tools.
On top of all that, their company also brings in guest speakers. The speakers show the capabilities of new tools and how to use the new tools to make “magical” things happen.
Some User Science Tools Need Others to Support
Using some tools requires help from others. For example, diary tests is something that takes a lot of time. His company has a special team to implement those kinds of things.
The trick thing about the tools is that we can’t just tell people about the tools. User Science is a field that needs to be learned. There are four learning curves / stages to User Science.
Four Learning Curves for User Science
One needs to learn:
- The tools themselves. For example, surveys, one on ones, A/B tests and more.
- How to do quality tests; Rookie mistakes are common. It’s easy to accidentally bias a test.
- What test to apply where. Is this the right test for a specific situation?
- How to use tests in a valuable way. How to gain value from the tests you do. For example, some beginning Product Managers will always use an A/B test for a new idea. That’s their go-to. Brent encourages the Product Managers to use tests to be more “provocative”. Since it’s a test on a small subgroup, the gain is so much greater than the risk.
For Those Getting Started
The User Science Field is advancing quickly. So his advice consists of:
- Hire fantastic people. Help make people great and create an environment to succeed.
- Share a clear vision and mission so all are moving in the same direction.
- Make it clear what success looks like. He recommends Daniel Pink’s book Drive. It covers autonomy, the opportunity for mastery and a rich sense of purpose. From this, one can create a great team environment.
- Focus on outcomes. Make things that matter.
- Communicate, communicate, communicate. There’s no such thing as over communicating especially when companies grow.
Summary
As you can see above, the User Science is Product Management podcast episode is filled with great insight on what makes up User Science, the related tools, and some practical tips on its implementation. I highly recommend you listen to the podcast episode as well as the other episodes at This is Product Management.
