Notes on “User Science is Product Management”

The focus of the User Science is Product Management episode is on User Science. It covers what User Science means, the tools involved and advice around implementing it. The guest is Brent Tworetzky, the Executive Vice President of Product at XO Group. Brent is on Twitter and Medium.

User Science is the study of users. It’s the combination of user research and user analytics. This directly aligns itself with interests that I have. Throughout my career and personal time, I have continuously sought to get closer to the end users of technological solutions. Through the power of enjoyable yet easy to use technology, I like to empower and enrich the lives of others. So, the topic of User Science is an area of deep fascination.

The rest of this blog post contains pretty much raw notes. The notes contain a mixture of direct quotes from the podcast, my own thoughts, and sometimes ideas that came to mind while listening.

Notes

Brent started as an engineer. He didn’t like the day-to-day of programming. So, he got involved in strategy oriented careers such as management consulting. However, he felt “removed from the action.” Product Management lets him put all the things he loves together into one role. As a side note, Brent goes into detail about the Product Management role in his article What Does a Product Manager Do?

Brent has his roots in Product Management from working with others who used to work for eBay, Netflix, and Google. With philosophies that came from those places combined, he has been exposed to different approaches that cover being business case driven, user research oriented, and the use of quick iterations. This all has been foundational to how he applies User Science.

Applying User Science at his organization answers the question of whether to continue to iterate on a product or to go a completely different direction. User Science is a combination of understanding the users’ intent and their user behavior.

Intent vs Action (Behavioral)

The tools involved with Intent vs Action can be expressed in this 2 x 2 grid that Brent provides in his article titled A Product Manager’s Superpower: User Science.

Product User Science Grid

Product User Science Grid

Intent

The purpose of exploring a users intent is to understand what problems the users are trying to solve. Using the intent tools, one is trying to nail down the important problems to solve.

Action / User Behavioral

Once you have the problem to solve, it’s time to switch to the User Behavioral side of things. This is where you can do things like prototype and do usability testing. You can do A/B testing or bucket testing. You can also do aggregate analytics.

From these tools you learn things like:

  • Do we have something that still works for what the user wants to do?
  • Should we iterate on this or is there stuff to throw away?

Diary Study, A Special User Science Tool

The Diary Study uncovers actual user behavior over an extended period of time. To do the study, they collect a small group of users that agree to share their thoughts and challenges. For example, wedding planning. They check-in every week or every two weeks over a two to twelve month period. The diary captures the needs and challenges of the user each week.

Through this tool, they learn what tools the user uses, what problems they have, and maybe what ideas they might have. Through all of this, the Product Manager can discover whether or not the user has experienced the kind of problem that is being looked for. In other words, if the users are experiencing the same problem again and again, that’s an interesting thing to learn and maybe you’ve found a problem to solve.

Implementation Challenges of User Science

User Science is a field that needs to be learned. It doesn’t come naturally or by accident. When new Product Management team members join their team, they explain and walk through the related philosophies. They ask them to put the tools into their hands and practice right away on practice tests. These new members also get tutorials.

The new team member gets tutorials from the User Research or Product Analytics team. For A/B testing, they use Optimizely. For usability testing, they use UserTesting. They run practice tests and get feedback about those tests. This lets the new team member get familiar with the tools and gain muscle memory about those tools.

On top of all that, their company also brings in guest speakers. The speakers show the capabilities of new tools and how to use the new tools to make “magical” things happen.

Some User Science Tools Need Others to Support

Using some tools requires help from others. For example, diary tests is something that takes a lot of time. His company has a special team to implement those kinds of things.

The trick thing about the tools is that we can’t just tell people about the tools. User Science is a field that needs to be learned. There are four learning curves / stages to User Science.

Four Learning Curves for User Science

One needs to learn:

  1. The tools themselves. For example, surveys, one on ones, A/B tests and more.
  2. How to do quality tests; Rookie mistakes are common. It’s easy to accidentally bias a test.
  3. What test to apply where. Is this the right test for a specific situation?
  4. How to use tests in a valuable way. How to gain value from the tests you do. For example, some beginning Product Managers will always use an A/B test for a new idea. That’s their go-to. Brent encourages the Product Managers to use tests to be more “provocative”. Since it’s a test on a small subgroup, the gain is so much greater than the risk.

For Those Getting Started

The User Science Field is advancing quickly. So his advice consists of:

  • Hire fantastic people. Help make people great and create an environment to succeed.
  • Share a clear vision and mission so all are moving in the same direction.
  • Make it clear what success looks like. He recommends Daniel Pink’s book Drive. It covers autonomy, the opportunity for mastery and a rich sense of purpose. From this, one can create a great team environment.
  • Focus on outcomes. Make things that matter.
  • Communicate, communicate, communicate. There’s no such thing as over communicating especially when companies grow.

Summary

As you can see above, the User Science is Product Management podcast episode is filled with great insight on what makes up User Science, the related tools, and some practical tips on its implementation. I highly recommend you listen to the podcast episode as well as the other episodes at This is Product Management.

 

 

Notes on “Mentorship is Product Management”

The focus of the Mentorship is Product Management podcast episode is on how to form valuable mentorship relationships between a product manager and a mentor. The guest is Ben Foster, author of Most Product Managers Suck.

Throughout my life, I have sought out different ways of empowering people through knowledge sharing. Among other things, I founded a Technology Mentor Program at CARFAX  So, this podcast was of particular interest to me.

The rest of this blog post contains pretty much raw notes. The notes contain a mixture of direct quotes from the podcast, my own thoughts, and sometimes ideas that came to mind while listening.

Notes

Ben had a statistics degree. He got a job doing QA, filed bugs, and got some enhancement requests. The product managers at his company invited him to become a product manager. That was in 1998 in San Francisco.

He joined eBay in 2001. The company grew fast. Ben rose in the ranks of Product Management. (How did he do that?) At eBay, he was in charge of merchandising which is all about “Would you like fries with that?”

He later became VP of product management at CoPower, an energy efficiency company for about four years. They took the company public in 2014.

Now, Ben helps founders and product managers. Every product manager can benefit from having a mentor. Decision making is what a product manager does. They have backlogs.They have strategies. They have to decide how far out to communicate a roadmap. They need good judgement and wisdom. Mentors help with the wisdom part. Mentors have:

  • Prior experience
  • Functional know how
  • Domain expertise

What’s Better, a Formal or Informal Relationship?

There are many different types of relationships. Paid vs. not-paid. Where the mentor is extremely senior or a peer. It depends on the needs of the mentee and the needs of the mentor. What are they trying to get from the relationship?

For the mentee, are they looking to do a current job better or take that next step in the career? Do they want formal or informal? Do they want a really good peer or someone more senior? Do they want someone in the same organization or outside of the company?

If you get a mentor that is inside the same company, the mentor has the context and is already familiar with what’s going on. If the mentor is outside of the company, they can offer benchmarks on what you are doing well or not based on comparisons to other companies.

Common Use Cases for Mentees:

  • Some have laser focus. They want a promotion.
  • Others are brand new. How can I do my current job better?
  • Some just want to learn from other people’s failures and their successes.

Mentors

Usually, you don’t go from not being a mentor to being one. It’s a progression. Someone asks 1, 2, or 3 questions and then the relationship evolves into a mentor / mentee relationship. Mentors can sometimes “pattern match” meaning that they can apply solutions that they have learned through other contacts. Mentors can create a general solution framework. The more the mentor can create those kinds of things the more people will want to reach out to you. It’s important to make it clear to the mentee what it is you want out of the relationship. Otherwise, it’s a one sided relationship. Both parties must understand each other’s reasons so it is win / win.

Not all mentors are the same. The good mentors will actively reach out to their mentees. It’s on the mentor to provide actionable and real world information. Theory needs to be backed up with practicality. How they communicate and how often can be dictated by what is currently going on for the mentee. The example given is around stakeholder communication. That might require frequent phone calls from the mentor. The communication should be constructive and direct.

Good mentors have a network. They proactively send out articles to the mentees that may serve them well. They can tap their network for answers they don’t know.

Best Mentor / Mentees Matches And Linking Up

An ideal mentor has worked with many employees that went from career spot A to career spot B just like the mentee wants to do. So, how can a product manager connect to a mentor?

Can start small. LinkedIn and other sources of groups. (User groups in town or online groups?) If you are seeking UX, Behance or other sources are discussed on Quora.

How To Approach The Mentor?

Say you are impressed by what they are doing. Tell them it would be helpful for you if they could advise you on this one specific problem.

People are sometimes happy to do a 20 minute phone call. Offering a free lunch can be good.

Good Questions To Ask

Good example of a specific question to ask is something tangible. Something like you are faced with a specific challenge. For example, you want to communicate the roadmap to the sales team better. Right now, there is a mismatch. What are some best practices in this specific situation?

An example of a bad question is like “What are the three most important things for a product manager”? It’s not actionable because people work with different constraints and issues.

In Ben’s career, he got mentors that helped fill gaps that he had. His “council” of people that he could go to consisted of CEOs, VPs, VCs, investors, and so on.

Stay in touch with your network. Over time, it’s amazing the value the network will provide.

If Ben Could Change Something

A mistake he made: He spent a couple of years making incremental improvements. He looked back two years later and asked himself “what did I really build?” “Where’s the innovation?” Getting leapfrogged by someone else is bad. The specific example is eBay vs Amazon. It’s like they “Won the battles and lost the war”

Ben is Reading

Summary

As you can see above, the Mentorship is Product Management podcast episode is filled with great advice when it comes to understanding, defining, and establishing mentor / mentee relationships. I highly recommend you listen to the podcast as well as the other podcasts at This is Product Management.