Pretty candle balls sparkling in the snow

How Big Is The Ball of Wax For Quality Assurance?

At CARFAX, we have awesome Quality Assurance (QA) support for the CARFAX iOS app and other apps. In order to do an awesome job, QA needs adequate time to do adequate regression testing before the next release of an app. The question is how much time is needed?

I like to think about this in terms of balls of wax. Amongst other things, the total effort of a project includes both an amount of coding plus a certain amount of testing. Therefore, the coding-ball-of-wax plus the testing-ball-of-wax equals the whole ball-of-wax.

In order to give QA a feel for how long it will take to regression test at the end of a project, they need some data. One idea that I’ve heard a few times is to use Story Points. What Are Story Points?

As I put it once, Story Points are how much brain-power plus actual coding is required to make the change. Do Story Points help give QA a feel for how much functionality change is being introduced? Not always.

A Story Point size is reflective of how much effort a developer has to put into making the change. It’s not reflective of how much testing is needed for that specifically changed feature.

How can we give QA the data they need in order for them to size the amount of QA work? If one feeds into the process, a “Feature Change Amount“ that will give them the information they need. The Feature Change Amount could be sized the same way as Story Points are sized. As opposed to Story Points however, a Feature Change Amount directly reflects something useful by QA so they know how much QAing is needed.

How does your QA get a feel for how long it takes to QA a project before it’s released for your customers to use? Feel free to share it with me on Twitter!

Pretty candle balls sparkling in the snow

Love Apple Seed

When it comes to the Apple company, I love the infant Apple, the Apple seed. I love Apple’s vision and passion of long ago. The Apple where Steve Jobs was figuring out how to make positive impacts on the world while still on this planet.

Investopedia says it well where:

..what satisfied him [Steve Jobs] most was watching kids use Apple products in the classroom

Steve Jobs wanted to improve people’s life experiences. He saw that he could do that through Apple products. Steve Jobs is gone, but it’s my hope that the momentum of his positive vision continues through the decades.

There is a new hope. Tim Cook spoke about the general philosophy of Apple and it’s captured nicely in the article Apple’s mission statement (Tim Cook text). I recommend one checks out the “The most interesting lines” section. It captures the concepts of keeping it simple, make a difference, keep focused, be excellent, and make excellent products.

So, keep the faith when it comes to your own personal mission statement and carry on! What is your personal personal mission statement? Feel free to share it with me on Twitter and let’s make paradise on earth happen together!

Notes on “Self-Awareness is Product Management”

Although the focus of the Self-Awareness is Product Management episode starts with self-awareness, it also covers communication skills, habits, and relationships between people.

The guest is Kelley Amadei, co-author of Founder of SparkShift. The rest of this blog post contains pretty much raw notes. The notes contain a mixture of direct quotes from the podcast, my own thoughts, and sometimes ideas that came to mind while listening.

Foundation for Leadership

“Self awareness is the foundation for leadership.” By being self-aware, one can be vulnerable in a good way and recognize their impact on others. Usually self-awareness is low in a person. From a cultural standpoint, people typically learn content specific for their job and push through as they climb the corporate ladder. Thus, the concepts of empathy, trust, compassion, and deeply understanding others / ourselves is weird to many.

If people don’t understand what is in it for them, they won’t do what you need. Understanding your people requires self awareness. It’s basic and foundational. It’s not even hard. One acknowledges their new found understanding externally to just trusted people initially and then expands into the “greater organization.”

If leaders focused on self awareness, they would be more effective. To do this, one needs to be a researcher and an antropologist. We should be objective. Let’s not get self-judgmental. Self-mastery is the goal. Pay attention. She teaches this to all kinds of people such as executives and people in grad-schools.

Beginning the Process

To begin ask yourself, what habits do I have that serve me and what habits should I drop?

Only choose one thing. “Identify the one thing that’s getting in your way.” Identify the thing that makes you cringe. Check it out with people you trust. Perhaps even a boss if there is a trusted relationship. “Is this the right area to focus on?”

Most common symptom of not having self-awareness is not listening. It is common with senior executives. It’s understandable though. If you’re a leader, you feel like you are supposed to know and quick to respond to questions. There are cultural things that get in the way:

  • Supposed to know
  • To appear quick, one usually is formulating a response while listening. That’s actually counterproductive.
  • You feel like you are not supposed to hesitate.
  • You feel like you’re not supposed to even take a moment to reflect.

Validate the behavior you want to work on: Don’t have to pick the hardest. Can pick the easiest one to gain confidence. Once you figured out the one thing to work on. There are strategies to apply.

Observing

For example, let’s say it has to do with eating. People usually will want to jump on the thing they want to change right away. “Don’t do that.” Don’t immediately jump into action. Watch and observe yourself. Side note: this sounds quite close to what was said on the 10% Happier podcast episode #61 titled Dr. Judson Brewer, Using Mindfulness to Beat Addiction.

When interacting with your direct reports, in meetings, and all areas of life observe yourself. Watch for when you are listening and not listening. Side note: reminds me of Stephen Covey, Seek first to understand, then to be understood.

The approach:

  • Pick a week or ten days.
  • Log it. Collect data.
  • You will see a pattern of when you are not listening and you will understand what is motivating the behavior.

Example: Direct report comes into the office. They have something to tell me. Trigger: they walk in. Response: Ready to tell them something first before listening. They are telling me and I am waiting to share what’s on my mind. Reward:I got stuff done. However, it could have been better.

Psychology and the Brain 

She is trying to drive a wedge between the trigger and response. We have less than a 1/2 second to work with. The first 1/3 second is physical. Most of what we do is an automatic response. Science says 90% of what we do is automated behavior, driving, swallowing, and so on. The brain starts on pathway and creates a groove (wrinkle). The brain says this works. So, it keeps doing it.

Another side note: This sounds similar to the aforementioned 10% Happier podcast, Stephen Covey as described by Michael Hyatt, and Dr. Victor Frankl. As said in Don’t Just React: Choose Your Response:

“Between stimulus and response there is a space.  In that space is our power to choose our response.  In our response lies our growth and our freedom.”   -Victor Frankl

If you are trying to stop an automated behavior, you have a small fraction of a second. Side note: Nir Eyal talks about habits all the time.

We’re driving a wedge between trigger and response. You can choose to do it the old way or when a direct report comes in, one can say: “I will shut my mouth, experience the uncomfortable feeling (it’s normal), and learn from it.” Try it for a couple weeks.

From a chemical standpoint, a habit is rewarded when dopamine gets released in the brain. That’s when the habit loop starts. Whether taking a drink or exhibiting a behavior in an organization, you are fighting the same battle.

Leaders are rewarded by getting things done themselves and they stop trusting others. They stop collaborating and delegating effectively. They second guess a lot.

A habit of not trusting is rooted in the fear of being let down. If I trust you and you let me down, it reflects on me. Trust requires vulnerability and understanding. A lot of leaders don’t feel they have time for that.

New VP of Operations Example

Example: There was a VP of operations. He focused on lean strategy and efficiency, efficiency, efficiency. Everything was through a very logical approach. He was 36 years old. His approach was by blowing things up (organizationally) and make things better.

When he got the VP position, he went in assuming that no-one knew what they are doing and so on. He built a habit of mistrust. He assumed that he had the only solutions. It would take too long to get things done. Now that he’s the VP he could no longer blow things up. Behaviorally, he was:

  • Too direct
  • Aggressive
  • Didn’t listen
  • Didn’t trust
  • He would second guess their solutions
  • He should have let his seasoned and senior people be part of the solution

His people went to the CEO and said this “guy’s a jerk.” He wasn’t listening and trusting. They brought Kelley Amadei in.

That’s one example of how something that serves a leader in the past can backfire later in their career.

Support the Right Culture and Also Build Authentic Relationships

Google has spent a lot of time learning about the successful attributes around leaders (related). Most psychology books would call some of this psychological safety.  Do I feel safe? Are my ideas heard? Can we have constructive conflict? Can we disagree and leave the relationship intact? One should allow constructive conflict. That’s where great ideas come from. Steps to make that happen:

  1. Encourage authentic connection between people.
  2. Never interrupt. When running a team meeting, don’t ever interrupt. Encourage team to do the  same.
  3. Allow everyone to be heard. If someone is in the corner, call them out. They are probably have something to say and are being squished by others.
  4. Known conflicts should come out in the open. Otherwise, the unresolved conflicts will waste people’s time later when talking in subgroups later.

Most people think they have a work persona. Can you get people / teammates to connect as real people?

Have an offsite and bring some questions. Examples:

  • What are you most afraid of?
  • Want do you most want out of your life?
  • What’s something difficult you have overcome?

Summary

There is plenty of great information in this podcast episode around self-awareness, communication, group relationship building, and even psychology. It’s clear Kelley Amadei is passionate about her work. She has great strategies for helping organizations improve their productivity and undoubtedly their overall happiness. I recommend you listen to the podcast episode yourself.

Sticky Notes

Sticky Notes by Gabriel Toro

Notes on “Jobs to Be Done is Product Management”

The focus of the Jobs To Be Done is Product Management episode is about:

  1. Disruption Theory and Innovation
  2. Current Innovation Approaches Established Companies Do
  3. How “jobs to be done” Drives New Features and Product Development
  4. Companies that Embraced “jobs to be done” Either Explicitly or Implicitly
  5. A Lens To Look Through

The guest is Karen Dillon, co-author of Competing Against Luck.

The rest of this blog post contains pretty much raw notes. The notes contain a mixture of direct quotes from the podcast, my own thoughts, and sometimes ideas that came to mind while listening.

Disruption Theory and Innovation

Clayton Christensen is known for the disruption theory aka disrupting the established company. The theory of disruption doesn’t cover everything. It covers the established company. The startup is not covered by the theory.

“Getting innovation right” is not just luck. The “theory of jobs to be done” is the “casual mechanism” of success. You need to understand it.

Current Innovation Approaches Established Companies Do

  • They gather lots of data, but the evidence is correlational as opposed to based on causation
  • Most innovations fail which may mean they are not in the field a year later
  • They don’t understand the “causal mechanism”

How “jobs to be done” Drives New Features and Product Development

As given 8:17 into the podcast episode, the definition of “jobs to be done” is:

The progress someone is trying to make in particular circumstances.

Very key: We pick a product or service based on our social and emotional reasons.

We should view things from the perspective of the progress that someone is trying to make. We have to understand the social and emotional circumstances along with the functional reasons for choosing a service / product. They use the language of “hiring” instead of “buying.”

Companies that Embraced “jobs to be done” Either Explicitly or Implicitly

AirBnB

They are a good example of a company that applied the “jobs to be done” approach successfully. Competition and solving the problem.

Intercom

  • In 2001, they did things based on various personae. That’s changed.
  • They even changed how they organized their company based on the “jobs to be done” framework:
    • They now have different physical teams around those “jobs to be done” which includes technology, marketing and so on.
    • They did great by offering prices for specific jobs and made more money overall.
    • They target the job to be done as opposed to the personae.

Intuit

“Help me get my accounting done now” is the “job to be done.” The up-sells they used to do were counter productive and interfering with the “job to be done.”

Amazon

They get “it.” They are focused on the “customer’s job to be done.”

They measure the speed of delivery from the moment when the customer makes the order to when the item arrives.

A Lens To Look Through

Looking at things through this “jobs to be done” lens, highlights what to do in regards to:

  • Customer service
  • Billing
  • How to renew and talk with people

It’s hard to find the right KPIs related to “jobs to be done” to show it’s working. Also, “jobs to be done” has to be integrated throughout the organization to succeed.

Summary

The “causal mechanism” is used often. It’s clearly an important part of the “jobs to be done” theory. Although there is concern about coming up with KPIs that demonstrate the successfulness of the approach, it absolutely makes sense. I encourage you to listen to the podcast episode yourself.

Notes on “Empathy is Product Management”

The focus of the Empathy is Product Management episode is about:

  1. Advantages of being new to an industry
  2. How to get industry knowledge
  3. Strategies for keeping empathy
  4. Formula?
  5. They have users near them
  6. Career transition

The guest is Jon Stross, a Co-Founder of Greenhouse.

The rest of this blog post contains pretty much raw notes. The notes contain a mixture of direct quotes from the podcast, my own thoughts, and sometimes ideas that came to mind while listening.

Advantages of Being New to an Industry

The advantages of being a product manager in an industry where you don’t have any experience:

  • Can stay objective. Can have a wider perspective.
  • Requires you to get out of the office.
  • You are not “..blinded by your biases.”

How to Get Industry Knowledge

(Talking to Potential New Customers)

Need to talk to many people. For them, it was recruiters. They would ask them, what do you like or hate about your tools and jobs? His business partner was an expert. He ran a team of recruiters.

They went and said “we want to teach a course on how to make recruiting a strength of your company.” They did a handful of classes. They figured out what resonated and what did not. They looked to see who showed up. They wanted to bounce ideas off of them. It was “great market research.”

They created a paper product. It was made of notecards and more. “They walked around to people’s offices and tried it out with them.” Before they wrote code or even made a company, they had a pretty good idea that the idea would work.

Strategies for Keeping Empathy

  • They hire out of their consumer service department. The tech team gets a sense of which of those “get it.” So, technical expertise and domain expertise is there.
  • They have a user experience research group.
  • They break out the different segments.
  • Sales and Consumer Service provide input.
  • They look at user data.
  • Product Managers synthesize all of those inputs and decide which ideas to go with.

Formula?

There’s no formula that they use. Their agile philosophy helps a lot. If they can get twenty things done and eighteen of those things are right, they are doing fine. They keep things scaled small. They manage scope and have things be smaller. Six months is too long.

They Have Users Physically Close To Them

They used their network. They can easily find folks. It helps that recruiters are real public. They want to be easy to reach.

They have lots of data. The hard part is figuring out who to listen to and “the patterns that matter.”

Career Transition

Product Management is a natural step towards becoming a CEO or general management.

Summary

Lots of great things in here! Clearly, Jon and his partners have figured out a way to dive into a domain and come out with awesome products. I really enjoyed the podcast episode and invite you to listen to it yourself.

CARROT iPhone App Video Marketing Done Right

The maker of the CARROT TODO app does user experience and marketing right in so many ways. To keep this post short, let’s just focus on one bit of video marketing titled CARROT Launch Trailer. The video is fun enough that I even saved it to watch again!

The video starts with some exciting music and a strong message similar to this: “the todo list with a personality.” It’s a good clear message which shows how it distinguishes itself and what the app does.

With the blue circle pulsing, the video moves on to the playful and humorous statements of “Let’s play a game. We’ll call it ‘Don’t Suck at Life.'” It shows some functionality of the app and then moves on to the rules of the game. Get stuff done or CARROT will get mad!

As someone who has used the app for years, I guarantee she (CARROT) will get mad. CARROT is often referred to as the user’s “new mistress.” Here’s the video:

One thing the video doesn’t show you is the funny and sometimes creepy (in a humorous way) push notifications that the app receives. I highly encourage you to turn the notifications on. Although the humor is often dark, it is fantastically funny. Check out the app for yourself!

Notes on “Networking Is Still Not Working”

The focus of the Networking Is Still Not Working episode is about:

  1. Business indifference is a huge issue
  2. Nourishing new relationships versus existing relationships
  3. Dormant ties and what that means
  4. Optimizing a networking event
  5. “Host a different kind of networking event as a way to offer value to the people you already know and work with.”

The guest is Derek Coburn, author of Networking Is Not Working: Stop Collecting Business Cards and Start Making Meaningful Connections.

The rest of this blog post contains pretty much raw notes. The notes contain a mixture of direct quotes from the podcast, my own thoughts, and sometimes ideas that came to mind while listening.

Notes

Leaping into indifference from a software developer point of view, a supervisor might see people as fungible software developers. How to disrupt this? Just going heads down and working harder doesn’t help.

Financial Advisors have a 94% retention rate. 75% of the time it’s because they provide great client service. The way to disrupt things is to focus on providing more value for your existing clients beyond just doing a good job. You can help them network and make introductions.

Strategically and effectively communicating how you are different will give you great opportunities. It’s better than waiting around for people to contact you.

Create a different rung of service that people have ever thought about or ever seen as necessary in their job. Don’t just be someone who does their job.

Derek’s general example: You can be someone who generates trust and rapport and makes the client feel safe by sending them something in the mail. Specific example:

Here are five reasons you don’t have to worry about Brexit.

[or] I’ve already come up with something in order to help you deal with Brexit.

Be an extension of your client’s development and marketing team. Derek positions himself to work with business owners.

Nourish Existing Network

Existing relationships are very important. Among other things, it’s important for job searching. Nourish your existing network. Existing clients are a good place to start. You can say to your client “I want to know more about your ideal client. What are your trigger phrases?” What is meant by trigger phrases is that certain phrases provide a clue that someone may need your services now or at some point.

A trigger phrase would reflect perhaps a triggering event. Perhaps, something that happened in life. You can share this with clients and others. Example: Someone is getting ready to sell a business. They said this 3 to 6 months before they know they even need you.

Figure out good triggering phrases for your business and then help your clients figure out triggering phrases to listen for for their own business. You can teach your clients how to do this by giving examples of what phrases you would be looking for. This also teaches the client to listen for trigger phrases on your behalf. Even though the client interviews are for learning about your clients, you can teach clients and friends how to help you.

Teaching Others How To Help You

Generic questions of “How can I help you?” doesn’t work. Having something ready to share with those who honestly want to help you is helpful.

This is what I am looking for. If you know someone that knows someone that would help.

Networking

Don’t just go to a networking event. Go to an event that has a great speaker and content. Invite a client. Tell the client to bring a friend.

Setting Up an Event Yourself

You can host round table discussions. Facilitate events. Examples: Poker or wine tasting events (for a client and someone else.) You can tell them: “Don’t feel like you have to bring a prospective client for me.” This plants a seed. Initially, they thought about their wife or friend. With that seed planted, they may bring a prospective client.

Start with a 5 to 10 minutes idea. You can share an idea and not a sales pitch. Think about sharing a couple things that are not directly related to the thing you provide.

After sharing a bit of information and sharing wine, you’re the thing in there they all have in common.

Hosting a technology event can get you in touch with the recruiters. You’re the hub. You know all the recruiters / head-hunters. Tons of lateral connections or non-lateral connections in your industry.

How often events? Roundtables are easy to set up. With Open Table, you can search for all the restaurants that have private rooms. Contact the restaurant; can they do separate checks? Invite people to come together and have lunch. You can facilitate a conversation about whatever you want to have a conversation about.

You can start with 4 to 5 people and 2 of those people can be your friends to get feedback.

He stumbled into great success with this. He invited a few influencers that are influential in the industry. This made the events very popular.

Summary

Lots of good gold nuggets in here. How to stand out and be seen as essential by providing extra value. How to nourish the existing network. How to set up an event. There are more tips in this podcast that I couldn’t capture in this post such as don’t just reach out to someone periodically with a template. I recommend listening to the podcast episode.

 

Notes on “Demand Validation is Product Management”

The focus of the Demand Validation is Product Management episode is about answering the second question of these two standard questions:

  • Is this product useable?
  • Do people want to use it? – This is Demand Validation

The guest is Steve Cohn, Founder of Validately.

The rest of this blog post contains pretty much raw notes. The notes contain a mixture of direct quotes from the podcast, my own thoughts, and sometimes ideas that came to mind while listening.

Notes

Steve Cohn got a lot of false positives. It begs the question: “How is great innovation brought to market?” It’s relatively easy to define and validate the problem space.

There is a gap between “Is the problem worth it?” and “Solution you have in mind.” Do people want to pay for or talk about your solution?

In comes Demand Validation. Validate the solution space. Figure out ways to run rapid prototypes.

Back to the false positives mentioned above: 50% of the features go unused by customers. “They passed the internal sniff test.” 30% of the work that a product team does is rework. Product teams ask customers “would you use this?” which is a bad indicator of demand.

People are not misleading you on purpose. There’s a gap between the hypothetical desire and committing to the solution. The “cost question” is better. Until you make saying yes cost something, their answer is meaningless. It forces the consumer to make the mental tradeoffs. For the “free products”, instead of costing money we go with time and reputation. So, the three things of possible interest are:

  • time – Is the problem deep enough to give their time for free to help you “create” the product?
  • money – Money comes in prepaying or signing a contract.
  • reputation – Social sharing is the reputation.

If people care about the problem and are compelled by the solution, they will work with you to create your solution.

Money comes in prepaying or signing a contract. Kickstarter is an example for consumers. For businesses, you can pre-sell all the time.

If you get a “no it’s not worth their time / money / reputation”, ask “why.” Knowing why is important. Customers are good at telling you what is not compelling about what is in front of them.

Backlog Prioritization 

Product Management is harder at bigger companies. Some examples include marketing, finance and executive. Marketing is saying there is a feature buzzing out there in the marketplace and we need to do this. Sales say a big customer needs this. Finance says where is the ROI for this feature? Shouldn’t you build another one? Executive says they had a great idea in the shower.

There is a saying mentioned: If we have data, let’s go with data. If we have opinions, go with mine.

Solution: Do Demand Validation Tests at the prototype stage with your backlog. We’re talking about rapid experimentation and getting the data. Then you can respond with the data. Yes, we prototyped it. We tested the idea against the customers and this is what they said. It went poorly so why should we build it? If it tested well, build it ASAP.

A feature prioritization process built on opinions will fail. You need to prioritize based on data.

MVP vs MVE

The true concept of the MVP is brilliant. The problem is when there is a misunderstanding about MVP. He likes to use the term MVE, Minimal Viable Experiment. We’re learning through this experiment. It’s understood that an experiment yields learning. When people hear product, they think growth. So, instead of MVP let’s call it MVE.

We’re in the home-run business. We have to consistently hit home-runs. There’s only one way to do that. The home-runs strike out a lot. So, get lots of trips to the plate. Take lots of swings. Run experiments. You will get a lot of strikeouts. However, you have to do that to get the home-runs. Give the team flexibility to try crazy things.

When Do Product Managers Know They’ve Hit It Big?

Alpha-UX are thought leaders on this. Spit-testing prototypes. They use high volume. They shoot for statistical significance.

The thing you want to understand about validation is that it’s not a guarantee. Lean is about reducing waste. Removing things that definitely won’t work. Prioritize those things that have the higher probability of achieving market success.

Summary

  • Do Demand Validation Testing
  • Experiment often
  • Use data to help prioritize the backlog
  • Use the term MVE instead of MVP to set expectations

Notes on “User Research Is Product Management”

The focus of the User Research Is Product Management episode is about going beyond simply User Experience. The guest is Jack Cole, a director of design. User Research is critical to product management decision making. It’s beyond just trying to make “cool stuff.”

The rest of this blog post contains pretty much raw notes. The notes contain a mixture of direct quotes from the podcast, my own thoughts, and sometimes ideas that came to mind while listening.

Notes

User research is a through line throughout any type of product lifecycle. — Jack Cole

We focus on users of a certain age or economic background. We do interviews with people, uncover their goals and understand where they are trying to get some additional value. We’re trying to understand the “opportunity space” that is there. We want to create “habit forming” products.

We break it down around the key factors / features that “certain user groups would be be looking for.” We want to go for trends. We uncover them. Next, we go for those identified age ranges, economic backgrounds or areas of the world.

Formal interviews are not the only way. His company has built out friendship groups. It’s more of a casual atmphosphere. They build on a conversation and a level of trust to identify where the opportunities are.

Discover existing pain points and trends. This is done before proposing new ideas and value propositions. Hoping and Praying that things are on target is not fun.

How to Find the Right Users? Who To Talk To?

Envelope yourself in the project. Start gorilla style. Ask around.

Metrics and Data Points

It’s about trying to identify themes in the discussions. Map them out. You can see opportunities based on what is common in the responses. See an opportunity? Build on a hypothesis and on those elements. Objectively identify themes and narratives in data. It’s about quantitive and qualitative information. Shake out the truth vs what is an anomaly. Build out ideas and test.

Common Problem

Trusting in the first interview you hear is a problem. See the themes and through lines. Don’t trust just one voice.

B2Bs count too. Even if the customer base is established, we can achieve better adoption through a deeper understanding of the end user. Come at this through a path of humility. You don’t know everything. There’s always something new that can be pulled out. New innovations can spring forth.

Risks of Not Doing User Research?

  • Loss of time, money, and resources.
  • If the culture doesn’t embrace it then you get Product Manager burnout.

Evangelizing Tips

It doesn’t have to be time consuming or extensive. Engaging your user community is a form of research.

Reading

Podcasts

He also suggests checking out the blog posts at motivatedesign.com

Summary

  • Do user research by doing interviews.
  • Map out themes from the research.
  • Evangelize through community

There’s certainly overlap in this episode with other episodes made by This is Product Management. However, this episode is highly focused which is refreshing. As listed above, the podcasts shared and things he is reading are certainly worth looking into. From what I have seen where I work at CARFAX, understanding the users through user research is a must have. CARFAX gets it. An understood user is a user who will rave about your company.

Notes on “User Experience Is Product Management”

The focus of the User Experience Is Product Management episode is on User Experience and things that impact such work. The guest is Sarah Doody, a user experience designer.

The rest of this blog post contains pretty much raw notes. The notes contain a mixture of direct quotes from the podcast, my own thoughts, and sometimes ideas that came to mind while listening.

Notes

Sarah works 50% on startups and 50% on companies that are in the market already. She helps them figure out where they should optimize the user experience. She does teaching and writing on the side. She even will be launching some curriculum in 2016 hopefully.

Her Background

She has always been interested in the tech scene. When growing up, she was interested in neuroscience. Specifically, biofeedback. However, that requires medical school. She realized she couldn’t make it through medical school. It’s too gory.

She went to Texas and attended a community college. She did internships. She did web design which was good because her left and right sides of the brain were both used. She took business courses. She took Graphic Design. She did some front end work.

Eventually, she got a full time position in Oregon on a web team doing maintenance of sales sites. She didn’t have a degree yet. At some point, she was asked to do banner ads. It was then she realized this career path was not going to suite her well down the road. He dived into learning about user experience.

She did a startup for five years. After that, she “started her own thing” in 2012.

Intersection of UX and Product Management

User experience involves all the touch points with a product or service. Although there’s the online experience, there could be offline too. There was an example of a frame company. They had a photo of the frame. The product was packaged in beautiful paper. There was a hand written note. It was a great experience. The experience doesn’t end when one leaves the website.

Role Confusion

There’s a difference between UX and UI. UI deals with what does the site look like? It deals with icon colors. UX is stuff before that. How did they get there? What tasks do they want to do? The whole customer journey.

Data Informs How?

Don’t sketch until you’ve talked with people. You don’t have to have a complicated setup with a user lab. You can do research without too much time or budget.

Here’s an example. For one project, she started with doing twitter, message boards, and Quora searches. She tried to answer:

  • What are people doing to solve the problem now?
  • What websites are they using?
  • She spoke with x people in Twitter and x people in Quora.

For her current project, she just got done with a two week research trip to Atlanta, GA. They interviewed people personally in their own environments. They will now take hours of interviews and comb through them and find common themes (Personal thought: Is there an AI opportunity here of finding themes?). They spot themes that come up in each interview. Just spot patterns and understand people’s behaviors at the macro level.

Mistakes Teams Make

Classic mistakes deal with testing on their site. They test too many things at once. Control your variables. Choose one or two things. Otherwise, “the data you get is garbage”.

A tool she likes is Optomizely. It’s simple and you don’t know have to know how to code. She used it on her blog for example. She would test and tweak the location of a social image or button.

Another tool is Crazy Egg. Can see people’s scroll or mouse patterns. It’s good to use a variety of tools. She might use both. That way you get diversified data.

There’s a misconception that you need a massive sample size for interviews. (Like one that needs 100s of people.) She just spoke with 16 people. After the 4th person, they are seeing patterns. The patterns are the same patterns over and over. By the 15th person, you are feeling real done.

If you are doing A / B testing, you need a minimum of a couple of 100 people. It all depends.

Microfeedback

Micro-UX? It’s a small touchpoint that adds a delight factor. Delight in the sense of the experience just works. There’s less thinking required by the user.

An example is the user types “c..a..m.. and predictive text fills in the rest. It helps you get to what you want fast. Another example is with a password form field where there is a show link. You click the show link and you can see the password you just typed in.

Grab the Feedback in a Timely Fashion

It’s important to rapidly grab the feedback from the user after they just did an important task.

For example: She rode in an Uber. A little survey popped up which would let me rate them. That’s micro-feedback. She completed an important task with a product. So, get the feedback close to the point of transaction so you have a chance of getting any feedback at all and it will be better data.

Here’s an example of doing it wrong: Delta. They asked for feedback 3+ days later. They sent an email. They asked “How was the flight?” Click here for a survey. It was 15 to 20 questions long. It’s too long. It looks bad on mobile. They should just send a text message. It should be something like: I saw you landed on Boston. How was your flight: 1 – 5. Done.

Story Telling

She wrote in 2011 Why We Need Storytellers at the heart of product development. Based on a rant on her blog.

She noticed teams that she worked with that the team itself would be over budget, over timeline, bloated with features or sometimes too little features, and the users would not adopt the features. Why is this happening over and over?

It’s like this. There’s a beautiful vision in mind and then what comes out is nothing like what you had in mind in the first place. On Pinterest, you see cakes and decorations and then you try to mimic what you see.

The reason is because we don’t spend enough time in storytelling. It’s because we feel progress writing stuff. However, it’s not real progress.

Spend more time working on the story up front. People will start to embrace it.

Idea: How to apply storytelling? Use storyboarding in the Product Management process. Imagine you are crafting a film. Scene by scene. They are in this particular environment. What are they doing and what’s their problem? Another scene, they have this problem. You have the solution. It helps you think about all the other factors. Example: A location based mapping thing. People are using it outside. It might be sunny. It could impact the colors. So, it might need less words. Consider the whole experience and the whole environment and not just the individual screens.

Collaborate 

She wrote an article called 6 mistakes companies make when working with a UX agency or consultant

Be specific with the feedback. Example of bad feedback: I don’t like orange and green because I don’t like peas and carrots.

Be as specific as possible with the requirements as possible. (Personal thought: What about Agile and incremental / iterative development?)

Be actionable. How to draw out the requirements very very soon is important. Do more of the complex screens up front. It’s going to trigger 1000 questions for them and the teammates.

Don’t just say what you don’t like, say why you don’t like it. I don’t like this because (insert some real business reason).

Don’t hold feedback back. 1/2 way through is too late. Don’t say at the end, I haven’t liked everything up to this point. That doesn’t work. Don’t delay the feedback because that will increase cost.

The cost of change. Changes in UX; factor of 1. Changes in design; factor of 5. Changes in coding; factor in much more.

Don’t race towards the launch date. People see the holidays. A commerce site example: We need to launch this big thing by November. Running toward the launch date causes two things to happen:

  1. They end up launching a product with all the features, but all features are 70% quality.
  2. You start having to do feature slashing. However, you slashed so many features it’s worthless.

Don’t isolate your developers. Let the developers be involved as much as possible. They are close to the data. They have a good eye for UX sometimes. Look to them for ideas. Have them involved early on and they can help you figure out your time line. Example: Custom chat messaging vs using something out of the box.

Benchmark Question: How does she eat her own dog food?

Amusement park project example will explain it best. She doesn’t have kids. So, she reached out to a few friends. She asked: Ever lost a kid at the grocery store? She used her personal network. She Googled. She looked for products that existed. She spent an afternoon doing the research. She made a storyboard.

In her storyboard, her first screen box was them at a park. Second screen they discover the kid is missing. Next screen they want to take action. What’s the first thing you would want to do? First thing you want to do is report the kid is missing. Keep in mind they need to conserve battery because they are at a park. There was a total of 16 scenes.

That enabled a user flow document. She sketched out what screens would be needed. Similar to sitemap diagrams. Now it’s user flows or screen flows. From there, that allowed her to do wireframes for each screen.

She would bounce between the following as needed: storyboarding, storytelling, user flow / screen flow, wire framing. The main point is that you don’t want to start with wire framing first.

Benchmark Question: How does she get out of the office?

She did formal user analysis. It was too formal. So they developed a process to get information in a less formal and fruitful way.

They created a screener to identify the exact people to talk to. A recruiting firm was hired to find all those people. The quality of recruits is 100% dependent on the screener. Hours were spent on the screener. More time into the screener the better quality of recruits. A discussion guide was created. This consisted of a list of rapid fire questions. The goal was to make interviews conversational and informal yet the questions are in the back pocket.

She talked with people in their own environment. You get more information from people if it is their home or office. For activities on the computer, that helps. They are with the computer they are familiar with. They are familiar with the environment and devices. So she did research in people’s homes and offices.

Benchmark Question: Reading?

She has two books on the go always. For pleasure, My Paris Dream.

Second book: It’s more business oriented. The Shallows by Nicholas Carr. She’s not finished. It’s kind of around the idea that our brains are changing and are not static. What is the influence of the internet on our brains? Multi-tasking is bad. She has learned and confirmed content from there. The internet might not be good for our brains. It’s harder to absorb long form content. Reflecting and focusing on things is a challenge. We’re in distraction mode. We’re bombarded with content. The content may be swishing by our brains. It’s a hard read. However, it ties into her love of studying the brain.

Benchmark Question: Recurring Nightmare

Having projects and clients that lack requirements. There’s a lot of first time founders. People come into product management as UX or from another field. There’s junior product management that is good at the day-to-day. However, there is a product strategy gap. UX is filling that gap. It’s important to identify the requirements and create product roadmaps. As consultant, you want to have those things up front. She may want to figure out how to put the creation of requirements and product roadmaps into her contract.

Summary

The identification of requirements through a process of storytelling is consistent with great advice I have heard from other sources. It sounds like User Experience roles are still being confused with User Interface roles. Finally, the knowledge of how to get valuable insights and feedback from the user are invaluable. Having looked through her website, there are many other gold nuggets to be found at the Sarah Doody website. Also, don’t forget to listen to the User Experience Is Product Management episode made by This is Product Management.

Sticky Notes

Sticky Notes by Gabriel Toro