Rethinking the Product Roadmap

[After reading this article, check out our latest thoughts on this topic here.]

I walked into the conference room on the first day of work at one of my previous jobs and looked at a huge list of features on the whiteboard. There were maybe 20 in total, with a few scribbles on the side that looked like the beginnings of more. The other PMs explained to me that these were the items that were in their Product Roadmap the previous year, but they had run out of time to develop them all. The dev teams were crunching for the first few weeks of the new year to finish these features and ship them to customers. To me it seemed that some of these features were unnecessary and there was more important work to do, but I quickly discovered that a lot of the clients had these written into their contracts… and we had not fulfilled those obligations on time. I watched over the next month and a half as the teams battled an exhausting schedule, fixing terribly buggy code and working through botched releases to get the products shipped before the next phase of the Product Roadmap started.

Now this case was a very extreme version of when “Product Roadmaps Go Wild” (TV series anyone?), but it’s not uncommon. I’ve seen unrealistic roadmaps destroy teams and good products time and time again. Many problems could be solved if we stop considering Product Roadmaps to be a Gantt Chart rather than a high-level guide, but the way we make traditional product roadmaps prevent teams from being innovative and successful. Here’s why:

Estimations are arbitrary.

Estimating the time it takes to create a feature usually ends up in a guessing game. I’ve even seen product roadmaps get built without developer input (note: this is a terrible idea). Typically developers will listen to a high level overview of which features will be built, and then assign some kind of value that will indicate how long it will take to build them. These products are popped into time slots and something else gets put right behind it – a la gantt chart.

Now when teams are two weeks away from the end of those X weeks and it’s obvious the product isn’t going to be finished, one of two things happen. One is the development time runs over and the next product is pushed off. This usually results in lots of screaming by management “we missed our deadlines” or “you didn’t prioritize correctly”. Or, the other – the team ends up rushing to release whatever they have so they can make the deadline and avoid the chastising. Number two can be an even bigger problem, as it usually results in buggy code being pushed out and little testing. More work piles up to fix the product, but that becomes impossible since the product roadmap is full and we need to start the next epic. Your product now becomes the king of the bugs.

It’s nearly impossible to estimate large chunks of work without planning out and thinking through the features, which we do not do when we plan roadmaps so far in advance. See the first example of this post.

Your product.

Your product – King of the Bugs!

Time for validation and research is rarely budgeted in.

Developing an excellent product needs extensive research, product strategy, a great UX, solid development, plenty of testing, and feedback from customers. Estimations in roadmaps usually leave out at least two of these: testing and research. When companies estimate how long a product will take, they usually tack on a week or two at the beginning for wireframe and design. Because little research is done, we often end up building products that do not solve users’ problems. We don’t find this out until we’re working on the next product in the Product Roadmap because we shipped it at the end of the time frame and did not include time for feedback. Then time is rarely budgeted in to go back and fix the previous product since our roadmap is full. Which brings me to the next problem…

There’s little room for change.

When we fill up 100% of our allotted time in the Roadmap with the development efforts, we leave no time to act on the feedback and make improvements on existing features. Every time we release a product, we learn new information about the customer’s problems, frustrations, likes, and dislikes. That information should go back into making our products better, but all too often it does not because by the time we receive enough information to act, we’ve moved on to the next product.

We may also find out during one of these feedback sessions that there is a bigger problem we should be tackling. In fear that changing course will screw up our nicely planned roadmap, the normal response to this is “we’ll slot it in when we go over the roadmap again”. Unfortunately most companies revisit roadmaps twice to four times per year which is far too infrequent to respond to change fast enough.

 

A lot of the problems I mentioned above can be mitigated or lessened if you are constantly revisiting your roadmap, doing estimations correctly, and have a great UX practice. I know a few companies who are doing these things and seeing some success. But there is one fatal flaw I see in product roadmaps that we cannot correct with these processes.

Product Roadmaps focus solely on solutions and features, ignore problems.

Product Roadmaps plan out which large feature sets will be built over a long period of time, called “Epics”. Since we plan Roadmaps well in advance, by the time we reach the start of an Epic we usually don’t have much up to date research or information on what we’re building. It could even be irrelevant or the problem has been solved. But, we rarely ever scrap these features. Why? We completely miss the point of building features, which is to solve problems for our users. We focus so much on planning and speccing out these elaborate features that we forget why we’re building them in the first place. When the speccing phase starts on the Epic, we generally don’t spend too much time really analyzing if the problem is still relevant because it had been so obvious when we planned the roadmap. Unfortunately this is the nature of thinking of solutions without carefully analyzing the problem first – you get wrapped up in the bells and whistles of feature sets, the original problem starts becoming more of a rumor than a motivation to solve something.

productfeatures

I see this as a major problem especially for startups that are transitioning to young businesses. They are putting more processes in place so they can scale well, but they still want to hold on to their competitive edge – innovating rapidly. When these startups start implementing product roadmaps, they find themselves moving out of problem solving mode and into feature overload territory. This can stifle innovation faster than anything I know. When the team is more focused on the processes of a company and meeting deadlines than creating value for a user, that team is in trouble.

So how do we give these teams some structure and direction, without stopping the innovation?

The Problem Roadmap

I’m suggesting that teams I work with start to treat Product Roadmaps as Problem Roadmaps. Instead of focusing on features to be developed, we focus on problems to be solved.

product-roadmap

The Problem Roadmap can be planned over any amount of time, depending on what is good for your business. In my example I use Quarters, which I think gives adequate time to work through a test and a solution. It’s broken into two phases: Discover & Experiment and Build & Validate.

Each cross-functional team works on a problem to be solved during the time frame. Their goal is to the solve the problem. We assign a KPI to each team which signifies if the problem has been solved successfully or not.

During the Discover & Experiment phase, the teams are focusing on customer research and validating that the problem exists. Once they have gathered enough research, they will start to build an MVP to test in the market. This part of the process should be done rapidly but carefully, with extensive customer interaction. After the MVP has proved successful, they move on to the Build & Validate phase.

In the Build & Validate phase, teams focus on minimum solutions (feature sets) to solve the core problem for the user. They are constantly releasing to customers and receiving feedback. They plan what they can build in the time frame, keeping the features essential and useful. When they have feedback, they can add more features and iterate.

If a product needs to be built out more at the end of the time frame, they can add it to the prioritized problems and weigh it against other problems that need to be solved using metrics. If it’s high priority that these features be developed, they can continue the Build & Validate phase.

How to plan a Problem Roadmap:

  • As a team/company, list out the top problems customers are facing now.
  • Prioritize the list of problems with the strongest at the top.
  • Assign each cross functional team a problem to tackle in that quarter and KPI to measure their progress in solving the problem.
  • Each team is responsible for validating that their problem is still a strong issue for the customer. If a problem is no longer an issue, they will move on to the next problem on the list and validate that.
  • Once a problem has been validated, the teams will conduct research on the problem by directly interacting with the customers.
  • Teams will start to develop MVPs after initial research and test in the market.
  • Once an MVP has been validated (has met the goals set by the team), the teams will start building a solution that can be made in the time frame left. They will focus on Minimum Feature Sets, with just the essentials. Teams will focus on releasing often for customer feedback and iterations.
  • At the end of the quarter, the teams can decide whether more investment is needed in adding to the solution after they have received customer feedback in the initial stages. The roadmap will be adjusted for this.
  • At the beginning of the next cycle, the process repeats. We revisit the list of problems and re-prioritize them based on customer needs, or add in new problems.

This Problem Roadmap solves many of the problems above. Instead of thinking in features, we think about how we can deliver the most value to our customers by solving their problems. Estimation for complex feature sets is not needed, since we focus on minimum solutions that can be built and released in short time frames to get rapid customer feedback. The roadmap is flexible since problems are constantly being evaluated, and irrelevant problems will be squashed at the beginning of each cycle. We can communicate the problems we plan on tackling to the team and our customers without committing to solutions we do not know will solve the problem.

What do you think? What have you done with Product Roadmaps that has been successful? If you decide to try Problem Roadmaps, I’d love to hear more about it!

 

Please Stop Misusing Buzzwords

Please. Please, just stop. Stop using buzzwords if you do not know the meaning. Somewhere in the past couple of years “agile”, “lean”, “mvp”, and “scrum” became the “it” thing in the tech world. Everyone wanted to be agile. Every manager wanted to say they use scrum. I’m a huge proponent for these methodologies and practices – look above it says lean all over my page. I think they can really turn companies around and produce great products when implemented and used correctly. But, I have seen a fundamental problem with a lot of teams and people who claim to be agile or lean: they have absolutely no idea what these words actually mean.

Here’s one example of the wrong use of a buzzword gone haywire. During August at Techpeaks, the staff announced they were implementing a “scrum of scrums” every Wednesday. Upon hearing this new plan, I asked myself “What the hell, is that even possible?” If you know what scrum is, you will understand that this term doesn’t even make sense. It got better when they explained that during this “scrum of scrums” we would have a mandatory 2 hour meeting with all 20 teams at Techpeaks where each team would stand up and say what they did last week, what they’re doing next week, and what problems they need help on. At this point, I audibly said “what the hell?” Obviously, this is just a progress meeting to keep track of what the teams are doing, but by using the term “scrum” a whole abyss of misunderstanding opened up. Teams started freaking out that the day for scrum didn’t match up with schedule for scrums that people were doing in their teams. Huge debates started about teams changing their scrum schedules to match the Wednesday end date, or move the Wednesday date to Friday so they can report on what they did. This discussion went on for weeks and weeks, and “scrum of scrums” (which ironically got shortened to “sos”) became a huge point of contention and anger for all the participants.

The misuse of one word, “scrum”, started a landslide of problems. Our common language started to unravel. One group was no longer on the same page as the other, and working relationships became utterly confusing. This is the danger of using buzzwords without understanding the meaning. The words become nothing more than a shiny badge of honor for being “cool” in this industry. Congrats, you are a lean startup and the new hip place to work. But what happens when you advertise this and attract people who actually know lean startup, and want to conduct lean experiments? They will be unhappy, because you will have promised them something different than what they understood. I know, it’s happened to me.

It’s imperative that we cut the crap with buzzwords and start using them properly. Simply launching a landing page does not necessarily make you “lean”. Deciding to chunk up a large project over 4 weeks does not necessarily make you “agile”. There are theories behind all these practices that were lost along the way. So here is my quick refresher on what these common buzzwords really mean and some common misconceptions:

Agile

Agile is a set of software development methods that focuses on iterating products incrementally. Agile values continuous delivery, personal interactions over documentation, and close collaboration between cross-functional teams to plan and evolve requirements and solutions. You can read the Agile Manifesto here and the 12 principles of Agile here.

One common misunderstanding I see is when developers require Product Managers to write out a complete, multipage, highly detailed spec before starting development on any products. The first half of the Agile manifesto states “Individuals and interactions over processes and tools. Working software over comprehensive documentation.“ Working face to face inside the team rather than handing off documentation not only creates a more collaborative environment, but it surfaces problems earlier.

Scrum

Scrum is an Agile framework that embodies the Agile manifesto in its practice. Scrum contains 3 core roles: the Product Owner, the Scrum Master, and the development team. Work is defined and agreed upon by the team and then sectioned off into sprints of a certain length. Scrum values teams working together to complete a common goal, responding to customer’s changing needs, and delivering quickly. You can read more about Scrum practices and values here.

Scrum is not a rigid process, but many teams understand it to be. One very common statement I hear from teams practicing scrum is that once the sprint is planned, nothing can be changed. This directly contradicts the flexible nature of scrum and one of the principles of Agile: “Welcome changing requirements, even late in development.“ This is not to say that you should be changing what you’re building all the time, but learning to be open to changes in product backlogs and deciding if they are a bigger priority is key.

Lean

Lean is most commonly used in our industry referring to Lean Startup principles, but Lean originated from the Toyota Production System and the term is also used for Lean Software Development. Using the term just “lean” can get really confusing depending on who you are talking to. For our sake, we’ll stick with Lean Startup Methodology when we refer to Lean here. Lean Startup Methodology focuses on building what customers will use, not what you think they will use. It values eliminating waste, maximizing learning, and measuring results. Iterating through the Build-Measure-Learn loop quickly is the goal.

Many folks miss the maximizing learning part of Lean. They build products quickly, release them to customers, and ask them if they like it. As soon as they get a yes or see a bit of traction, they think they are done. They stop talking to customers, and just keep building. Soon customers are out of the feedback loop. If you are focusing more on building your solution than learning from your customers, you are not Lean.

MVP

“Minimum Viable Product” is the least amount of work you can do to learn from your customers. MVPs are experiments, not minimum feature sets. In fact, an MVP can be manually delivered without a website at all. The main goal of an MVP is to learn if customers have the problem you assume and want your solution.

Many people think an MVP is just the minimum website they can create for their product, most often a landing page. While a landing page is a good tool to gauge customer interest, every MVP is not a landing page. A landing page is just one piece of the experiment. Targeting the right subset of users, driving traffic to the page, and delivering results is the entire MVP.

One day, these buzzwords will die down and others will emerge. This whole cycle will start again – learn, practice, fad, misunderstanding. It’s always a good move to educate ourselves and others on new trends, to make sure we understand the deeper meaning behind these words and keep a common language in our workplaces.

Why You Should Talk About Your Ideas

“Can you please help me figure out how to prove my idea with Lean?”

“Sure, what are you working on.”

“Oh, I can’t tell you. It’s confidential.”

I’ve had this conversation at least 10 times in the past year with an aspiring entrepreneur. They wanted advice and guidance on how to use lean with their idea, then refused to share what they were working on. One person wanted me to sign an NDA before we chatted… at a bar… at midnight. What??

People who do not talk about their ideas are at a sore disadvantage. Now I’m not talking about broadcasting your grand plan to a million people, but picking a few connections that you trust and respect can yield helpful advice and feedback.

I am currently validating an idea with my team here at Techpeaks. I’ve reached out to a few people back in NYC to receive their feedback and do a few interviews. Their insights have been inspiring and helpful.

Tim introduced me to the Jobs to Be Done framework and to help me align my idea around it.

Tohm shared his experience at his company and helped me think through the challenges and benefits of our current idea for selling to the potential customers.

Adam helped us streamline our goals and tests so we could focus on the technical feasibility of the project and not get bogged down by going too broad, too fast.

Jon and Pamela gave us a great way to test our idea very lean so we can begin starting experiments this week.

Some of my assumptions have been squashed. Some of them have been validated (so far!). It feels good to get the feedback, “Oh that’s a good idea”, but it’s been more helpful and constructive when people say, “Stop. The first part of that sounds interesting. The rest is crap. Focus on the beginning.” I find myself eagerly waiting for those moments to come up in the conversation.

So to those of you out there saying, “I can’t tell you what I’m working on, but it’s going to be awesome”… is it? Who is going to buy it? Have you asked your potential customers what they think? My advice to you is:

  • Do interviews.
  • Ask for advice.
  • Don’t broadcast your idea to the entire world, but tell someone.
  • Once you ask for advice, shut up.
  • Listen to what the person has to say.
  • Don’t get defensive, just explain your thoughts.
  • Expect criticism, it can only help you.

I’m overwhelmed by just how smart the people I know are. Thank you to everyone who has helped me, guided me, and have been truly interested in what I’m doing. I can’t wait to return the favor one day.

Lean Product Management on a DIME

While setting up my team to use Lean practices at OpenSky and Conductor, I had a hard time figuring out exactly what the role of a Product Manager was on the team. I knew that I was still in charge of the vision of the product, and thinking about where we would take it in the future. But, the main function of my job – coming up with what to build – was now something the whole team contributed to. Then there was the speccing, something that took up 80% of my day. Lean and Agile practice collaboration over documentation, so what happens to those documents I had been making? What exactly is a Lean Product Manager responsible for on a day to day basis?

I’ve found the daily role of Product Managers shifts away from churning out solutions and specs, and into that of a facilitator and team-leader. I became responsible for setting up the experiment, empowering the team to solve problems, and making the hard definitive calls, all while steering the vision of the product. In my class, I call this process DIME because it’s easy to remember:

    • Define the problem
    • Investigate assumptions
    • Measure results
    • Evaluate success

Defining the Problem:

Product Managers bring the problem to the team and rally them around it. They are responsible for explaining why it is important to solve the problem, how customers are experiencing problems now, and what their potential solutions could mean for the company. When I have the original kickoff meeting with my developers, I try to tell them a good story about the problem. I get pretty dramatic for emphasis. (Thanks High School drama club.) The better story you tell, the more everyone gets excited about what you are going to build. Driving passion in the team to solve the problem is one of the most important jobs a PM can do.

Investigate Assumptions:

When we’re talking about the problem, I’ll use that meeting with my developers to talk about our assumptions. We’ll discuss how we believe the users interact with our product now, what we believe they do in their daily jobs, and other things we’re not sure about. After the team lists their assumptions on sticky notes, I’ll the charge to explore if these are correct or not. I always include my developers in this discovery process, so they hear first hand what the customer is saying. This makes them feel closer to the problem and gets them excited about coming up with the solution.

Measure Results:

Every problem that is getting solved should be tied to a KPI. The PM figures out what data to collect during the experiment, and then tracks it. I talk to the upper level management to figure out what are our business’s key goals for this feature. Do they want to see higher revenue? Do they want to increase engagement? What is it about this feature that will change our business as we know it today. Metrics can drive so much insight to help your decision making in a startup, but you’d be surprised how many companies have none implemented.

Evaluate Success:

Finally, I reflect on my feature experiments and decide whether it was successful or not. This can be a hard call to make. Sometimes the data is juuuust off the marks you wanted to hit. Sometimes everything is a complete flop and you’re not even sure where you went wrong. It’s up to the PM to lead the discussion with the team and make the ultimate call around the success of your project. If it failed, I’ll investigate the cause of the failure to determine the next move, usually by talking to customers. If it succeeded, I will drive the initiative to build out a full product (remember an MVP is only a test!)

When you work in small teams, everyone wears many hats. Sometimes it can seem a bit chaotic, and you’re not sure who is in charge of what on a daily basis. It’s important to have guidelines on who is in charge of certain aspects of the development cycle so they can lead the discussion around decisions and make the calls in time of dissent. Using DIME has helped me define my day to day job as a Product Manager in a Lean team, while keeping our teams balanced and focused on the vision.

Lean UX NYC Recap

When Will asked me to speak at Lean UX NYC on Lean Product Management, I was both excited and confused. It didn’t make sense. This was a UX design conference, what would he want with PMs?

When the first day of the conference was over, no one thought it was just a UX design conference. There were speakers on every section of the business. Johanna Kollman talked about how lean makes learning the new deliverable for design consultants. Adrian Howard explained that the lean process is actually brand discovery for startups. Bill Beard spoke on how copy should be integrated into Lean UX to enhance the user experience rather than be used as a crutch. Grace Ng showed how UX designers can leverage the concierge method to do Minimum Viable UX before designing the whole product. Tomer Sharon gave user research tips for engineers: “Watch what the user is doing, instead of asking the user what they need.”

Throughout this whole conference there was one major theme: collaboration. The reason lean is so powerful is that it incorporates the entire team in solution generation. There is no longer a product “owner”. Virginia Cagwin talked about how you need to break down the silos in typical enterprise businesses and form balanced teams to produce sexy ideas. This is a huge culture change that is very hard for waterfall companies to understand and adopt, but it can be done. Ari Font Llitos showed how they do this at IBM by creating these small teams, and then teaching them design thinking.

Now once we have those small, balanced teams, the question becomes what process do we follow? Jonathan Berger brought up great points about creating sustainable pace for designers when working with agile development teams. Designers have been asked to work at the fast paced tempo of developers’ sprint cycles, which leaves them feeling overwhelmed and disempowered. Lane Halley and Courtney Hemphill spoke about how they are practicing ways that can help solve this problem at Carbon 5. They practice short, weeklong sprints that run through the build-measure-learn loop. The whole team participates in discovery, building, and testing during the sprint, eliminating the need for extensive documentation and designs. I think what made this talk even more powerful was hearing Courtney, a developer, endorse the integration of LeanUX into the agile workflow.

Will Evans wrapped up the conference with a great talk on modeling leadership. What really resonated with me was his call for creating a shared language for new processes rather than trying to distort the language of older processes to fit a model they don’t belong to. Designers and developers are subject to using industry jargon that doesn’t make sense to people outside those roles. Being able to effectively communicate with team members aligns everyone around a common goal. I picked up ways to communicate through two workshops last weekend. Jacklyn Burgan’s workshop on “Sketch Before You Etch” taught me a way to generate ideas with my team through simple sketches, something everyone can relate to. Lane and Courtney’s workshop on “Conversation, Cadence, and Culture” simulated the conversations and language I should use when sketching and building with the team. I just started working with a much larger development team, and effective communication has been a challenge for us. I’ve already started integrating these techniques in our new project and I’m seeing great results in just a week.

By Saturday, everyone from doctors to developers had given a talk. In fact, we could all probably form our own lean startup and make kickass products without needing to fill any roles. I realized now why the name of the conference makes sense. That’s what UX is: User Experience. It’s all the pieces of the business working together to make a unique, compelling experience for the people who use your product.

This is the first conference I’ve attended where people are still talking about it a week later, which only goes to show you just how much useful information came out of it. I’m still pumped. I met so many wonderful people last weekend, and I’m still riding the high of it. Balanced Team brunch with the speakers was just icing on the cake the next day, and I can’t wait to do it again soon. Thank you Will, for including me in an epic weekend.

If you did not go this year, make sure you check it out next year (it’s already posted). The sheer amount of knowledge you’ll walk away with is well worth the ticket price and will continue to help you for years. I’ll be there, mumbling my speech to myself and shoving macaroons in my face. You can find me by the cookie table.