Effective Product Roadmaps

Product Roadmaps, in general, are confusing. The truth? Even the most experienced Product Managers still don’t have them fully figured out.

Three years ago I wrote a blog post about Rethinking the Product Roadmap, in which I advocated for a focus on solving customer problems instead of listing out features with deadlines. I received lots of feedback on the post, much of it praise, but also some criticism. The biggest takeaway comment for me was, “This is great for new products in the Discovery Phase, but what about accounting for products that we have already validated and need to build?”

In the time since, I’ve been working with clients to find what works best in real life situations. I want to share with you the framework that I have found, which allows for a balance of discovery and delivery and takes us out of that laundry list of features. But first, let’s talk about Roadmaps in general, and how they got us on the wrong track to begin with.

Think back, before Google Maps, to what a real roadmap was- those things with tiny, tiny city names and thousands of squiggly lines.  The ones that made it so you could never drive alone because you needed a copilot, someone to wrangle the giant map and dictate to you how to navigate from point A to point B. Yeah THOSE things.


Well that’s where the term Roadmap actually comes from.  When using a roadmap, you always knew where you were and where you wanted to go, and you had all these different routes to choose from. Some routes were shorter, some were longer but more scenic. It was ultimately up to you to choose your optimal path. Product Roadmaps should work just like their namesake, but their intended purpose was lost somewhere along the way. Now, Roadmaps function more like Gantt Charts.

Gantt Charts are a project management tool used to keep track of timelines and due dates. While Gantt Charts are fine for large manufacturing projects, they do not work well for most software situations because of the uncertainty around development.

When we’re first deciding what software product we should be exploring, we don’t know what our users want, and thus we are not exactly sure what the finished product will look like. This makes it nearly impossible to estimate how long the product will take to build. Yet, we still use this Gantt Chart model to plan our work, which locks us into delivering solutions by certain dates. Teams get frustrated that there is no perceived flexibility in these plans, and management gets mad when teams don’t deliver on time. The result is a bunch of feature releases that don’t make a difference for our users.

Instead, we need a model that helps us take into account uncertainty and allows us the flexibility for both discovery and delivery. To do this, we focus on Product Roadmaps that are made up of outcomes, a theme, and hypotheses, and we leave out any unvalidated solutions or features.

Let’s look at an example I created of a completed Product Roadmap for one team. I’m using the New York Health Exchange’s website as an example of a product as I can easily pin down things that need to be fixed- like, lots of things. I’ve never worked with them, and I don’t claim to have knowledge about how they function internally, but I am a customer of the website. This is how I would approach improving the product if I were working with them.



The top section of our Roadmap outlines our Vision, Challenge, and Target Condition, which reflect the goals outlined in my post on Product Strategy.  A Product Roadmap should be aligned around these same goals and help communicate the tactics to reach them.

According to this Roadmap, Team X is working towards the overarching goal (or Challenge) of getting 99% of people who create an account on NYS of Health to sign up for insurance.

Now that we have our Challenge set, the team analyzes the customer problems standing in the way of them reaching that goal. This is the same process we outlined in the original Roadmap post. Let’s say that after user research, we find that the biggest problem for NYS of Health’s website users is that it takes a very long time to search for and find an ideal health insurance plan. It’s a confusing process and incredibly frustrating. It causes people to give up instead of sign up.

So Team X decides to tackle this problem, and sets a Target Condition that will tell them when they have reached a successful outcome. “Decrease the average time to find and purchase health insurance from 3 weeks to 2 days in Q2.” While users can only sign up once a year for health insurance, there are plenty of changes that could be implemented to make it easier once that enrollment period opens.

From there, Team X would figure out the top areas they want to explore or deliver on in Q2 to reach that Target Condition. We call these Themes. For example, the “Search Enhancements” Theme will include all the UX work needed to simplify the search for health insurance options.

In order to answer the question, “Why are we exploring this theme?” we write a Hypothesis or Problem Statement. When products are still in the Discovery Phase, we are still validating the problem and solution, so we want to state what we know and what we don’t know. Our Hypothesis should support that we believe our solution will solve the biggest problem.

So with our Search Enhancements Theme we are in Delivery Phase, and our Hypothesis is “We believe that by enhancing the filter options to reflect the top deciding factors that shoppers make when choosing a health insurance, we will make it easier for them to narrow down their choices and find the right health insurance.” We have already validated that people have problems narrowing down their choices to find the right health insurance, and we think that offering some enhancements to the filter options can solve that problem.

Our last theme, Subsidy Communication, is an example of a problem that is still very early in Discovery Phase. We have identified that there is a very low conversion rate for people who qualify for Subsidies, and we want to investigate why that is. We have a few ideas but we have not yet validated them. So we need to clearly communicate our Hypothesis, highlighting what we know (“the people who would get Health Insurance for almost free are converting at a very low rate”) and what we don’t know (“we are not sure if people leave before they know if they whether or not they qualify for subsidies, or if they just don’t understand our explanation of subsidies”).

After the Hypothesis section comes the “Outcomes” – the most important part of our Roadmap. Here, we state what we hope to achieve by solving this problem or proving this Hypothesis. Each Outcome should relate back to the Target Condition in some way. All of our work is going to help us reach the Target Condition, so it would be a pretty boring Roadmap if every Outcome for each theme is just a repetition of that goal. Instead, think about how each of those changes or improvements signals a move towards your goal.

For example, with the Search Enhancements, we can tell if we are making it easier for people to search by measuring the qualitative components of frustration and the quantitative components of decreasing the time to identify an option. You want to make sure you are focusing on a good combination of both business and user outcomes in these sections, and make them measurable.

The last two sections deal with status and priority. The Status should state where you are in the Discovery versus Delivery phase. If you are in the Discovery Phase, you’re delivering to learn, so your focus is on experimentation over releasing features.  In the Delivery phase, you’ll be delivering valuable features and solutions to users. Sit down with your team and define what this means within your company, and then share these definitions with the rest of the company so there is mutual understanding.

Notice here that our Roadmap does not include any specifics on how we plan on tackling these problems.  This is because we are still experimenting with options- we haven’t laid out a plan to implement a set of features or solution components.

The above example is only one team’s Roadmap. There might be many other teams working towards this same Challenge, but they would all have different Target Conditions. These Roadmaps could then be combined into a Portfolio Roadmap that could be communicated across different levels of the business, like so:

Good Roadmap - FDS portfolio.007

It’s important to remember that a Roadmap is a communication tool first and foremost. It’s meant to align teams across time horizons and goals. Management should be goal and Vision setting with a timeline of  6 months to a few years, while teams should be looking at a smaller timeline, from quarters up to 6 months. There should be an accompanying strategy document that addresses how these goals help the company reach its Vision, but the Roadmap itself does not serve that purpose.

Roadmaps are a living document. If you invalidate a theme, update it. If you’ve changed a goal, update it. There is no reason you should be struggling to work around a Roadmap that does not work for you. When you’re communicating with customers, focus on how you are going to solve their problems, not how you are going to deliver their features. Your customers care first and foremost about the value they are receiving.




We teach Product Roadmaps in our 10 week online course at

Product Institute.

The next class will start in April 2017. Use APRIL17 for $100 off.

Sign up for our mailing list on the top, right hand side of this page to be notified when we launch our next class.



Stop Blaming the User

I’m finally going home from a long business trip and I’m very excited. But my experience with United Airline’s customer service this morning completely killed my good mood.

They keep blaming me for their mistake. It’s a story I know well because I see it so commonly in Product Management too.

When I was booking my return ticket from London to Newark, the cheapest option was for something called “Mixed Cabin”. I could fly Business Class between London to Dublin and then Economy from Dublin to Newark. Since I had to get up at the ungodly hour of 5am to get to the airport, I thought that could be a nice treat on my first leg. So I booked it.

Here’s what my reservation looks like from a few minutes ago:

Imagine how surprised I was when I checked in for the first leg on Aer Lingus and they gave me seat 27B. “There must be a mistake,”,I thought. So I told the flight attendant that I was booked through United in Business. She replied, “We don’t have Business Class on any of our flights.”

What a bait and switch! United is selling me something that doesn’t exist. It’s only a one-hour flight, so I’m not going to hem and haw all day, but I wanted to get to the bottom of it. So, I tweeted… because that’s what you do when you’re angry. What happened next, pissed me off more than the switcheroo.

The customer service agents considered this my fault, and acted like it wasn’t a big deal.


As someone who doesn’t work for an airline, how am I supposed to know that Z class doesn’t exist on Aer Lingus? When you purchase your ticket and it says “Business”, that’s what you expect. I don’t think I am wrong for being upset; yet United is telling me I am. If I went to a restaurant and ordered the duck, the waiter wouldn’t say “I’m sorry, we actually meant steak when we wrote duck. Here’s your steak, deal with it.”

I’m not telling this story to complain about United (well, maybe that’s part of the reason). The moral of my story is this: I see product teams commiting this classic mistake all the time. Product Managers love to blame the user. If the user doesn’t understand something, they are just stupid. If the user won’t tell us what they want, they are difficult. If they are calling to complain, they are ungrateful.

This is a dangerous mindset, yet it’s so prevalent. “Users don’t know what they want,” is the battle cry of Product Managers all over the world. It’s their excuse not to talk to them, because what could a user tell them that they don’t already know?

The problem isn’t with the user. It’s with you.

The Curse of Knowledge Bias makes you forget that as a Product Manager, you have more knowledge than the user when it comes to your product. You can’t understand how they could find something difficult when you can do it so easily. Users become just a thorn in your side, and if they went away and took all their complaints and wishy-washy answers to your questions with them, you could build THE BEST PRODUCT IN THE WORLD! You forget that if users went away, you would be out of business.

Recently, I was training Product Managers at a very large company. I always start off with a major focus on the user and trying to uncover their problems. We did a Product Kata workshop to help demonstrate this. The idea behind Product Kata is to take a step back, figure out what you need to learn, and create a test or step to learn it. In this case, the teams had to create a “paper pizza” for the user. Chris Matts helped out by being our user. They had 45 minutes to sell Chris $10 worth of pizza. Everyone immediately jumped in and started asking him “What do you want?” He answered, “I don’t know what I want. I like these things…”

After a while, everyone hated Chris. Someone even said, “I am going to pivot my pizza business to martial arts because Chris better learn to defend himself.” That was pretty funny, but also a prime example of the problem. This happens all the time in real life. The reason people were getting frustrated is because they were asking Chris, “What do you want?”

It is not the user’s job to know what they want.

It‘s their job to know what problems they have. You need to understand how to question them about their context, problems, and needs. If you cannot do this well, you will end up getting frustrated with the user. When I interrupted the teams from running around like chickens with their heads cut off, I asked them “What do you need to learn?” No one had even thought of that question (even though I just taught them they should!). They just set to work to immediately build a bunch of crap hoping he would buy it.

When they stopped and realized they needed to learn the context in which he was eating the pizza, they were able to solve it within 5 minutes. After 30 minutes of wasting a bunch of paper pizzas.

Sound familiar?

It is easy to blame the user, but it is so dangerous for organizations. It leads to culture of disdain for our customers and whole lot of arrogance on our part.

In large companies this is particularly true. When you have layers of bureaucracy, very long-term employees, and a million different departments separating you from the user, it’s easy to forget about them. Product Managers in these companies say, “We can’t talk to the user, that’s the job of another department.”

There are always ways to learn more about your customers, even if you have different departments. For example:

  1. Go talk to the customer service team. When I was working with a client at the beginning of the year, we had trouble finding users to talk to. We could easily talk to people who already signed up, but we couldn’t get in touch with people who dropped off. Customer service could, though. We talked to them about what we were trying to learn, and gave them questions to ask. Then, we met with them weekly to go over what they learned from the users. This helped a lot. If you can, even go so far as to take customer service calls a few times a month.
  2. We designed ways to learn more from our user in the above case, as well. We implemented things like Qualaroo to get feedback from users when they were dropping off. They were allowed to type whatever they wanted in a box asking, “What’s stopping you from signing up today?” We got thousands of responses and learned the biggest problems.
  3. When I was working for a B2B company, I was told I wasn’t allowed to talk to users because we would disturb them. I was the head of UX there. I fought for that ability, and was allowed to go with a sales team member. In that interview, I was able to prove that what we were building was not what the user expected. I brought that back to the team, and it gave me more buy in to keep doing more interviews. We ended up setting up a little feedback group of some willing customers and gave them access to tools earlier than others, and some extra hands on support. This proved invaluable for our discovery methods.

It’s easy to forget that the reason our businesses exist is because someone is buying our product. Someone. Not a nameless persona full of stats, but actual human beings. Human beings have feelings and needs. When I am paying United for a service and they are telling me, effectively, that I’m wrong, I feel frustrated. I feel wronged. That leads me to start to look for their competitors. I’ve bet you felt that too on just about any airline. How many times have you tweeted that they don’t care about you? Felt that they don’t care about their customers in general?

It’s the same for your users. How do they feel when you do this? Blaming them is not the answer. I never said it was easy to talk to users, but it is the only way you can understand and build empathy for them. This is what makes a good Product Manager. If you don’t want to talk to users, get another job. If you’re willing to give it a shot, remember to take a step back and put yourself in their shoes.

What is Good Product Strategy?

strategy dilbert

“What is your Product Strategy? YOU NEED A STRATEGY.”

When I replay this scene in my head, I can hear the CTO very audibly yelling (slash pleading) with our product team. He was on edge. We had been experimenting towards a very concrete goal for two months, and had made a lot of progress. We had learned so much about what was preventing users from signing up on the site, and it was a lot clearer which direction in which we should be going. BUT we still had to test our ideas.

This didn’t sit well with the CTO because in reality he didn’t want a strategy, he wanted a plan. He wanted a list of what we were going to build, and when we were going to build it. He wanted to feel certain about what we were doing when we all came in tomorrow, so he could measure our progress based on how much we built. It’s not his fault though. This is the way we were taught to think about Product Strategy.

Most companies fall into the trap of thinking about Product Strategy as a plan to build certain features and capabilities. We often say our Product Strategy are things like:

  • “To create a platform that allows music producers to upload and share their music.”
  • “To create a backend system that will allow the sales team to manage their leads.”
  • “To create a front of the funnel website that markets to our target users and converts them.”

This isn’t a strategy, this is a plan. The problem is that when we treat a product strategy like a plan, it will almost always fail. Plans do not account for uncertainty or change. They give us a false sense of security. “If we just follow the plan, we’ll succeed!” Unfortunately, there is no guarantee of success here. (I wish there was, our jobs would be SO much easier!)

These product initiatives aren’t bad, they are just communicated at the wrong time and with the wrong intentions. When we lock ourselves into planning to build a set of features (ehem, Roadmaps), we rarely stop to question if those features are the right things to build to reach our goals. We stop focusing on the outcomes, and judge success of teams by outputs.

We need to have a plan but the plan shouldn’t be “build feature x.” Our plan should be to reach our business goals. We need to switch from thinking about Product Strategy as something that is dictated from top to bottom, and instead something that is uncovered as we learn what will help us achieve our objectives.

Product Strategy is a system of achievable goals and visions that work together to align the team around desirable outcomes for both the business and your customers.

Product Strategy emerges from experimentation towards a goal. Initiatives around features, products, and platforms are proven this way. Those KPIs, OKRs, and other metrics you are setting for your teams are part of the Product Strategy. But, they cannot create a successful strategy on their own.

We need a few core things for our Product Strategy to be successful:

Vision: The vision is your high level, ultimate view of where the company or business line is going. In large corporations, you want to narrow this to the business line or customer journey. In smaller companies, this will be your company and product’s overall vision. Think long term here, and keep it qualitative. This is a good chance to talk about competitors, how customers will see you, and ambitions for expansion.

Challenge: The challenge is the first Business goal you have to achieve on the way to your longer term vision. Which area of your customer journey or funnel needs to be optimized first? It’s communicated as a strategic objective that helps align and focus your team around a certain aspect of product development. This can be qualitative or quantitative. Try to keep these still in broad, high level terms. This one is the hardest for me to personally wrap my head around, but check out the example below for some clarity.

Target Condition: The target condition helps break down the Challenge. Challenges are made up of smaller problems you need to tackle along the way. These are set in terms of achievable, measurable metrics. When you set a target condition, your team shouldn’t know exactly how to reach it tomorrow. They should have a good idea of where to start looking through.

Current State: This is what the current reality is compared to the Target Condition. It should be measured and quantified before the work starts to achieve the first target condition.


Product Strategy Animation

These all contribute to something called “Unified Field Theory”, which is explained very nicely here by Bill Costantino and Mike Rother from Toyota Kata. When we are building products, we have a threshold of knowledge. We cannot start on Day 1 and exactly plan to reach our vision. There are too many unknowns and variables. Instead, we set goals along the way, then remove obstacles through experimentation until we reach our vision.

This is best explained through an example, so we’re going to use Uber. Let’s pretend you’re a Product Manager working on the platform that allows drivers to sign up.

Please note: I have no affiliation with Uber so this is a guess as how they might line up their strategy if they chose to use it. The vision is from a statement their CEO made in an interview. The rest I am improvising for the example.

The CEO has stated that the company’s vision is to make Uber the cheap and efficient alternative to both owning a vehicle and taking public transportation. (He really said this in an interview, but everything after this is hypothetical).

So if we understand the Vision correctly, Uber wants people to use them as their sole source of transportation. They would first want to look at why other people are taking other transportation methods now instead of Uber. They may go out and interview people and find that in certain cities where Uber isn’t as popular, there is a very long wait time to get a car. They would compare this to other problems and determine how big it is in comparison. Let’s say it’s the biggest challenge at the moment.
So the first goals they may want to tackle is decreasing the wait times in cities where it’s exceedingly long. Let’s say anything over 10 minutes on average is too long, and we want to decrease that down to 5 minutes or less because we’ve seen in cities with those wait times, people are 80% more likely to use Uber.

This is now our Challenge: Decrease wait times in cities where it is over 10 minutes to less than 5 minutes by January 30, 2018.

Target Condition
As a Product Manager, you now want to figure out what is causing that long wait time. The problem in this case might be that there are not enough cars to serve that area. So our metric we care about now Acquisition of new drivers.

Our goal for our team should be measurable and achievable, something like: We want to onboard at least one driver for every 50 people in each city by January 30, 2017.

As the Product Manager responsible for the onboarding process for new drivers you would be tasked with contributing to that acquisition. You would first measure how many drivers per people in each city you currently have (Current Condition), then find the obstacles that are preventing new drivers from signing up today. Next, you experiment to remove each obstacle until you successfully hit your goal. The Product Kata explains how to do that.

So let’s step back and take a look at all that:

(You can download a blank copy of the Product Strategy canvas here.)

As a Product Manager, you don’t have control over all these numbers. The vision is set by CEOs, CPOs, Boards, and other C-Level Executives. The challenge is set by the next level of management (VP of Product for Each Journey or Business Line).

Direct Managers help their teams set effective Target Conditions. At the beginning these may need to be handed down from management. As teams get used to this way of working, the managers and team can work together to set them.

Once these four items are set and communicated, the team can start applying the Product Kata method to figure out how to reach the goals. It’s the Product Manager and their team’s responsibility to determine the user problems and other obstacles standing in the way of achieving that Target Condition. Then they experiment to solve them.

This aligns everyone around a strategic goal and vision. Every level of the portfolio has their objectives. Teams are held accountable for their progress towards the goal they are responsible for reaching.

Now you have probably looked at this and said “well this isn’t a product strategy, it’s a business strategy.” Yes, this does come off looking like a bunch of business objectives, but isn’t that why we built a product? To reach our business objectives? Product Management is the art of solving your customer’s problems to reach your business objectives. If you’re not doing both of those things, your product is just a fancy piece of code for show.

The next post will be on how to format your Product Roadmaps around these strategies, and how to communicate with stakeholders to encourage experimentation.


How to Educate Stakeholders

There is one question I always get during every Product Management workshop:

How do I convince/teach/educate stakeholders to work this way?


We need to change this perception.

The class has bought in on validation before building, a focus on problems, and defined metrics. They can see the value, but they are afraid they will return to work and get shot down. Often they do. Unfortunately, there’s really not much you can do about it as a Product Manager. Your job is not to educate and train your entire company on this way of working. It’s to create valuable products for your customers and your business.

The way I approach stakeholders is the same way I approach customers – empathize. Stakeholders aren’t at this company to make your life miserable. They are there to do a job. So start to ask questions and learn about them. How are your stakeholders measured for success? Usually they have concrete goals they have to hit. Learn what those are. What are your stakeholders problems? How can you solve them?

When you start empathizing with stakeholders, you realize that the things they are demanding or requesting relate back to assumptions. These assumptions are based on what they think will help them solve their problems. Become a Product Manager for your stakeholder. Explain how this way of working helps them achieve their goals. This has worked very well for me in the past, even with really difficult stakeholders.

Now, if you stakeholder still doesn’t understand this way of thinking and is not bought in, you need to change your approach. If you are having meetings where you are asking for feature requests, stop. Stop it right now. The meetings you set with Stakeholders and the way you run those meetings set the tone for your relationship. If the purpose of all your meetings are to gather requirements (aka feature requests) and communicate deadlines, this is the way your Stakeholders will view you – as a place to dump their requests and get status updates.

Flip those meetings on their heads. Use them to learn more about their business and the problems they are currently trying to solve. Give status updates on what you are currently learning and the goals you’re working towards, not hard dates on shipments. Changing the conversation allows your team to come up with the best solutions, instead of building just what the stakeholders or customers ask for.

Remember, you’re job is not to teach or educate. You need to explain clearly why you do certain things, but that explanation is enough. Start to draw the line. Work with your manager on approaches and tactics so you have support from above.

Have you tried all this and it’s still not working? I’m sorry. That happens a lot. Honestly, until stakeholders, managers, and team leaders want to change, they won’t. You can encourage them to start coming to the same conferences and workshops you do. That will help build a shared context. Many of my workshop attendees leave and say “I wish my manager was here for that.” If you are a manager, I would take that sentence to heart. You’re never too busy to spend time educating and improving yourself. But, if none of this works, you have to make a tough call.

Last night I was talking to James Royal-Lawson who put it the best way I’ve heard yet. When someone asks how to convince their managers or stakeholders, he replies:

What do you want to do with your career? Do you want to go into training and change management? Or do you want to be awesome at your job? If you don’t want to go into change management, find a job that allows you to be the best you can be. If you want to do change management, then we’ll talk.

If you want to change a company, that’s going to be where you spend all of your time. This is what I do every single day. You have to be ready for it. It’s a long hard role. If you want to be the best Product Manager, then try some of the above, or find a place that will let you grow.

Finding the Truth Behind MVPs

If you’re interested in learning more about MVPs, I’m running a one day workshop in Paris on June 14, 2016. More information here.

A Successful Start

I learned about Minimum Viable Products like 99% of other Product Managers – through The Lean Startup by Eric Ries. When I happened upon the book and Eric’s method, I thought, “YES! This is what I’ve been searching for. This makes so much sense.” Testing products before you build them? What a novel idea! I was excited. I was energized! I was now going to build things that mattered to my customers.

Frankly, this came at the perfect time for me. I was tired of building products that no one used. Watching and waiting for the numbers to go up in Google Analytics, only to be let down again. It was getting old. My team and I spent months building products we thought would be successful, only to be disappointed. When I had the chance to try the MVP approach on a new product, I jumped on it.

The CEO of our ecommerce company approached me with a new product idea that was going to increase engagement and sell more items. He wanted to implement a Twitter-like feed so that the celebrities, who sold products on our site, could also post about their lives. This idea was prime for testing. It was so ripe with assumptions: “Do our customers want to hear about what are celebrities are up to directly in our platform? Will this sell more items or increase retention?”

I went to the engineers and asked them how long it would take to fully implement this idea from scratch, the way our CEO was proposing. With rough estimates, I went back to our fearless leader and told him “This is going to cost us $75,000 to fully build, and we’re not even sure our customers want it. I can prove in a week with $2,000 if this is going to move the needle.” Just like that, I had buy in.

Within one week, we proved this was a terrible idea. A week after that we found a different solution that increased clickthrough rate threefold on our products. We also doubled conversion rate. The whole company was hooked, and we were allowed to keep experimenting. “Great,” I thought, “everyone can see the value of this! It’s a no brainer.”

Eventually, I moved on to another job. I was excited about bringing the concept of MVPs to this team as well. Honestly, I was kind of shocked that everyone in that company wasn’t using them already. Did they just not know about this wonderful witchcraft? I was so confident that everything would go exactly as it had in my previous company.

“We Don’t Do That Here”

When the words “Minimum Viable Product” left my mouth for the first time, the reaction in the room was quite different than I expected. You would have thought I recited every curse word in the English dictionary. Like I just busted out a Biggie Smalls song in the middle of the meeting, not bleeping out any of the colorful lines. They responded as if I had offended their ancestors. Finally the CMO broke the silence with, “We don’t do that here. We don’t ship terrible products.”

Over the next few years I experienced this same reaction countless times.

I’ve learned that Minimum Viable Products are widely misunderstood. Some people are afraid to try building MVPs because of preconceived notions. Others use the word so much that it’s lost all meaning. “We should MVP that!” has become a battle cry in product development that just means make it minimal, make it cheap, and make it fast.

How do people end up here? The story is almost always the same. Someone picked up The Lean Startup, had their mind blown, and said “We should do that here!” They saw a quick and cheap way to execute on a product without fully understanding what the purpose of an MVP was or how to make one well. In my particularly jaded company, a developer ended up creating a hack to test a new feature that broke every time someone went to use it. Customers were pissed. The company blamed it on MVPs as a whole, rather than sloppy development.

It’s not the MVP’s fault. The problems stem from misinterpretation of what an MVP is and from miscommunications along the way.

What is an MVP?

My definition of a Minimum Viable Product is the smallest amount of effort you can do to learn. When I teach this in workshops, I’m usually met with disagreement. “MVPs are the smallest feature set you can build and sell! Not just an experiment.”

So what is the truth? Is an MVP a product, a subset of a product, or just an experiment?

The Minimum Viable Product was first coined by Steve Blank and then made popular by Eric Ries in The Lean Startup. I went back to research how these two experts, and a few others, defined the term.

“The minimum viable product is that product which has just those features and no more that allows you to ship a product that early adopters see and, at least some of whom resonate with, pay you money for, and start to give you feedback on.” – Eric Ries, 2009

“Minimum feature set (“minimum viable product”) is a Customer Development tactic to reduce engineering waste and to get product in the hands of Earlyvangelists soonest.” – Steve Blank, 2010

“A minimum viable product (MVP) is not always a smaller/cheaper version of your final product.” – Steve Blank, 2013

“An MVP is not just a product with half of the features chopped out, or a way to get the product out the door a little earlier. In fact, the MVP doesn’t have to be a product at all. And it’s not something you build only once, and then consider the job done.” – Yevgeniy Brikman, Y Combinator, 2016

Confusing, yes? The one thing that was clear to me through this research is that the definition of an MVP has evolved. In the beginning, we talked about this concept as something to validate startup ideas. All these products were searching for product-market fit. I learned about the Concierge Experiment and Wizard of Oz in those days, which helped shape my definition and understanding. As I continued to use these methods as a Product Manager in enterprises and other more mature companies, I had to customize both my definition and the practice of building Minimum Viable Products. What I’ve learned is that you need both – the concept of experimenting and building a minimum feature set to be successful.

While there’s tons of dissent on the definition of an MVP, everyone pretty much agrees on the goal. The goal of an Minimum Viable Product is to rapidly learn what your customers want. You want to do this as quickly as possible so you can focus on building the right thing. So let’s get rid of the buzzword and focus on that premise. Let’s stop arguing about what an MVP is and start talking about what we need to learn as a company.

How and When to Learn

When we start off building a new feature or product, there are a million questions to answer. “Is this solving the customer’s problem? Does this problem really exist? What does the user expect to gain with the end result?” We have to find the answers to these questions before committing ourselves to building a solution.

This is why starting with a minimum feature set is dangerous. When you jump into building a version one of a new product or feature you forget to learn. Experimenting helps you discover your customer’s problems and the appropriate solutions for them by answering these questions. It also doesn’t end with just one experiment. You should have multiple follow-ups that keep answering questions. The more you answer before committing yourself to the final solution, the less uncertainty there is around whether users will want or use it.

Once you have proven that a user wants your product, it’s time to investigate a minimum feature set. Now we can start to find a product that is marketable and sellable, but also addresses the user’s needs that were uncovered through experimentation. Delivering this product to market as fast as possible is the ultimate goal, so you can get feedback from customers and iterate. But, you have to be careful to deliver a quality product, even if it’s tiny. Broken products do not produce value for your customers, only headaches. Any version of a product that does not deliver value is useless.

How does this look in practice? In one SAAS company I was working at, we had to create a new feature that would help our customers forecast their goals. We were given input by the customer’s sales team from their conversation with the customer. After reviewing the information, we knew we had to learn more.

We met with the customer to understand what they were looking for in this forecaster. Once we thought we had a good grasp of the needs, we built them a spreadsheet and dumped their current data into it. This took us less than a week. We presented the spreadsheet to the customer and let them use it for a week before getting their feedback. We didn’t get it right the first time, or the second, or the third. But, on the fourth iteration, we were able to deliver exactly the results the customer was looking for. We did the same process with a few other customers to make sure this scaled.

While the spreadsheet was providing immediate value to some of our customers, we didn’t have the resources to do it for all of them. So, we had to build a software solution. We started exploring the Minimum Feature Set, using the feedback we received on the spreadsheet. There were plenty of other bells and whistles the customers wanted, but we paired it back to the essentials in the first version. We spent a few weeks getting the first version to work and include the most important pieces. Then we turned it on for the clients with the spreadsheet to get their feedback. After iterating a few times, we began selling it to others.

This process is will help your company find problem-solution fit. If you are creating a new feature or a new business line that solves a different problem for your user, this method can help ensure you’re building the right thing for your customer. But what if you have a mature product and are not starting from scratch?

Experimenting in Enterprises

Many enterprises today are introduced to the Minimum Viable Product by consultancies who propose creating an entirely new product from scratch. This is may not be the best idea. When your company already has product-market fit, you have already built a product that customers are using. You do not need to rebuild your product, you need to improve it. The methods should to be adapted for this case.

Something that sets these two methods apart is the goal. When searching for product-market fit, you want the user to adopt and probably pay for your product. This is not always the goal when improving existing products. It could be to improve retention or increase engagement with certain parts of your product. Whatever you decide your goal is, it should be clear to the team and informing all their decisions.

Once the goal is clear, you again focus on learning. What do you need to learn before committing to a solution. Write out these hypotheses on what you think will move the needle, and then design a Product Experiment to test it. You don’t have to create an entirely new product. Maybe you will find out a new product is necessary while experimenting, but that should not be the end goal.

A company I coached wanted to increase their conversion rate across the site. They already had a popular ecommerce subscription product with thousands of users. Traffic was coming in, but users were not converting as much as projected. Nothing had moved the needle when it came to offer testing. They dug into the sources of where great customers were coming from and found that many came through referrals. But, only a few people were actually sending referrals. Through user research we discovered two main reasons why: they didn’t know they could actually send referrals, and they were not sure what the referral gave their friends.

The team decided to tackle the first problem with the hypothesis, “If we let users know clearly that they had referrals available, they would send them.” The first experiment involved showing a pop up that encouraged users to send referrals when they next logged in to the site. Referrals sends went up 30%, leading to an increase in conversion rate! They did not have to implement a whole new program here; they just had to create ways to make it visible.

This team continued to dive into problems around conversion. They learned that the top three problems on the direct-to-site experience were:

  1. Customers were not sure how the service worked.
  2. Customers wanted to know what specific items came in the subscription.
  3. Customers were not sure why the product was priced higher than competitors.

The next step to solving this problem was to see if they could deliver value and learn at the same time. They created the hypothesis, “If we give users the information they are searching for in the sign-up flow, they will convert more.”  They also wanted to learn which questions people would click on most to see which problem was the strongest. They planned a simple way to get people the information they needed while signing up: adding a few links into the sign-up flow, echoing the questions back to users. When the links were clicked, it showed a pop up  explaining the answer to the question. At the end of the week of building and testing, they could see the experiment increased conversion rate closer to our original goal. They also learned that showing the information about what exactly came in the subscription was the most important thing. The team continued to learn what was preventing prospects from signing up, and systematically answered those questions through experimentation.

Caution Ahead

One mistake companies make when dealing with Product Experiments is keeping them in play once you have learned. These features then break and cause problems for your users. You are designing to learn and move on, not to implement something that will last forever.

With the team above, they learned that the information they provided was helping prospects answer their questions, but not enough people were seeing that solution. After experimenting more, they learned that a more robust solution would be needed.

It was time to start planning a sustainable solution that incorporated the learning from the experiments. Moving away from Product Experiments to the next phase is not an excuse to stop measuring. This team was still releasing components in small batches, but those batches were complete with beautiful design and a more holistic vision. After every release, which happened biweekly, they would measure the effect it had on conversion and test it in front of customers. The feedback would help them iterate towards the product that would reach the conversion rate goal.

Chris Matts has eloquently named this the Minimum Viable Investment. He’s also pointed out that you should not only be looking at improving your user facing products, but the infrastructure that helps you create those products quickly. The team above is also improving their site architecture to experiment faster while waiting for test results. I introduced the Product Kata to teams improving their products to help them find structure through Product Experiments and Minimum Viable Investments.

Learning is the Goal

One of the scariest parts of this process for companies is releasing things that are not perfect. It’s important to balance good design with fast design and good development with fast development. The best way to do this is getting UI designers and developers to pair together. After defining what the goal is for the iteration or experiment, sit down together and talk through ideas on how to execute. If we design in a slightly different way, is it just as useful to the user, but easier to build? Prototype together. Sketch together. Work side by side and talk about trade offs the whole time. This is how good teams move quickly and avoid rework.

By learning what the user wanted early, I’ve avoided countless hours of rework and throwing out features all together. This is why it’s so important for teams to experiment, whether they are B2B or B2C. Give the product teams access to users. I’ve seen fear in companies that their employees will say or do things that will upset users. If you teach your product teams the right way to communicate and experiment, this will not happen. Train the teams in user research. Don’t release experiments to everyone. Create a Feedback Group with a subset of users. Build infrastructure so you can turn on experiments and features just for smaller group. These users will guide you to create features that will fit their needs.

I dream of a day when I can walk into a company and mention “MVP” and not hear, “We don’t do that here.” While the definition of Minimum Viable Product may work us into a tizzy, the goal behind it is extremely valuable for product companies. If you’re having trouble implementing these practices inside a company, try leaving out the buzzwords. Use terms like experimenting and focus on the premise. Learning what your users want before you build it is good product development. Make sure when you do invest in a feature or solution, it’s the right one.

This post was originally published on InfoQ on April 18, 2016.