You can watch my interview on Applying Lean Startup in Different Organizations live on InfoQ.
I walked into the conference room on the first day of work at one of my previous jobs and looked at a huge list of features on the whiteboard. There were maybe 20 in total, with a few scribbles on the side that looked like the beginnings of more. The other PMs explained to me that these were the items that were in their Product Roadmap the previous year, but they had run out of time to develop them all. The dev teams were crunching for the first few weeks of the new year to finish these features and ship them to customers. To me it seemed that some of these features were unnecessary and there was more important work to do, but I quickly discovered that a lot of the clients had these written into their contracts… and we had not fulfilled those obligations on time. I watched over the next month and a half as the teams battled an exhausting schedule, fixing terribly buggy code and working through botched releases to get the products shipped before the next phase of the Product Roadmap started.
Now this case was a very extreme version of when “Product Roadmaps Go Wild” (TV series anyone?), but it’s not uncommon. I’ve seen unrealistic roadmaps destroy teams and good products time and time again. Many problems could be solved if we stop considering Product Roadmaps to be a Gantt Chart rather than a high-level guide, but the way we make traditional product roadmaps prevent teams from being innovative and successful. Here’s why:
Estimations are arbitrary.
Estimating the time it takes to create a feature usually ends up in a guessing game. I’ve even seen product roadmaps get built without developer input (note: this is a terrible idea). Typically developers will listen to a high level overview of which features will be built, and then assign some kind of value that will indicate how long it will take to build them. These products are popped into time slots and something else gets put right behind it – a la gantt chart.
Now when teams are two weeks away from the end of those X weeks and it’s obvious the product isn’t going to be finished, one of two things happen. One is the development time runs over and the next product is pushed off. This usually results in lots of screaming by management “we missed our deadlines” or “you didn’t prioritize correctly”. Or, the other – the team ends up rushing to release whatever they have so they can make the deadline and avoid the chastising. Number two can be an even bigger problem, as it usually results in buggy code being pushed out and little testing. More work piles up to fix the product, but that becomes impossible since the product roadmap is full and we need to start the next epic. Your product now becomes the king of the bugs.
It’s nearly impossible to estimate large chunks of work without planning out and thinking through the features, which we do not do when we plan roadmaps so far in advance. See the first example of this post.
Time for validation and research is rarely budgeted in.
Developing an excellent product needs extensive research, product strategy, a great UX, solid development, plenty of testing, and feedback from customers. Estimations in roadmaps usually leave out at least two of these: testing and research. When companies estimate how long a product will take, they usually tack on a week or two at the beginning for wireframe and design. Because little research is done, we often end up building products that do not solve users’ problems. We don’t find this out until we’re working on the next product in the Product Roadmap because we shipped it at the end of the time frame and did not include time for feedback. Then time is rarely budgeted in to go back and fix the previous product since our roadmap is full. Which brings me to the next problem…
There’s little room for change.
When we fill up 100% of our allotted time in the Roadmap with the development efforts, we leave no time to act on the feedback and make improvements on existing features. Every time we release a product, we learn new information about the customer’s problems, frustrations, likes, and dislikes. That information should go back into making our products better, but all too often it does not because by the time we receive enough information to act, we’ve moved on to the next product.
We may also find out during one of these feedback sessions that there is a bigger problem we should be tackling. In fear that changing course will screw up our nicely planned roadmap, the normal response to this is “we’ll slot it in when we go over the roadmap again”. Unfortunately most companies revisit roadmaps twice to four times per year which is far too infrequent to respond to change fast enough.
A lot of the problems I mentioned above can be mitigated or lessened if you are constantly revisiting your roadmap, doing estimations correctly, and have a great UX practice. I know a few companies who are doing these things and seeing some success. But there is one fatal flaw I see in product roadmaps that we cannot correct with these processes.
Product Roadmaps focus solely on solutions and features, ignore problems.
Product Roadmaps plan out which large feature sets will be built over a long period of time, called “Epics”. Since we plan Roadmaps well in advance, by the time we reach the start of an Epic the feature we usually don’t have much up to date research or information on what we’re building. It could even be irrelevant or the problem has been solved. But, we rarely ever scrap these features. Why? We completely miss the point of building features, which is to solve problems for our users. We focus so much on planning and speccing out these elaborate features that we forget why we’re building them in the first place. When the speccing phase starts on the Epic, we generally don’t spend too much time really analyzing if the problem is still relevant because it had been so obvious when we planned the roadmap. Unfortunately this is the nature of thinking of solutions without carefully analyzing the problem first – you get wrapped up in the bells and whistles of feature sets, the original problem starts becoming more of a rumor than a motivation to solve something.
I see this as a major problem especially for startups that are transitioning to young businesses. They are putting more processes in place so they can scale well, but they still want to hold on to their competitive edge – innovating rapidly. When these startups start implementing product roadmaps, they find themselves moving out of problem solving mode and into feature overload territory. This can stifle innovation faster than anything I know. When the team is more focused on the processes of a company and meeting deadlines than creating value for a user, that team is in trouble.
So how do we give these teams some structure and direction, without stopping the innovation?
The Problem Roadmap
I’m suggesting that teams I work with start to treat Product Roadmaps as Problem Roadmaps. Instead of focusing on features to be developed, we focus on problems to be solved.
The Problem Roadmap can be planned over any amount of time, depending on what is good for your business. In my example I use Quarters, which I think gives adequate time to work through a test and a solution. It’s broken into two phases: Discover & Experiment and Build & Validate.
Each cross-functional team works on a problem to be solved during the time frame. Their goal is to the solve the problem. We assign a KPI to each team which signifies if the problem has been solved successfully or not.
During the Discover & Experiment phase, the teams are focusing on customer research and validating that the problem exists. Once they have gathered enough research, they will start to build an MVP to test in the market. This part of the process should be done rapidly but carefully, with extensive customer interaction. After the MVP has proved successful, they move on to the Build & Validate phase.
In the Build & Validate phase, teams focus on minimum solutions (feature sets) to solve the core problem for the user. They are constantly releasing to customers and receiving feedback. They plan what they can build in the time frame, keeping the features essential and useful. When they have feedback, they can add more features and iterate.
If a product needs to be built out more at the end of the time frame, they can add it to the prioritized problems and weigh it against other problems that need to be solved using metrics. If it’s high priority that these features be developed, they can continue the Build & Validate phase.
How to plan a Problem Roadmap:
- As a team/company, list out the top problems customers are facing now.
- Prioritize the list of problems with the strongest at the top.
- Assign each cross functional team a problem to tackle in that quarter and KPI to measure their progress in solving the problem.
- Each team is responsible for validating that their problem is still a strong issue for the customer. If a problem is no longer an issue, they will move on to the next problem on the list and validate that.
- Once a problem has been validated, the teams will conduct research on the problem by directly interacting with the customers.
- Teams will start to develop MVPs after initial research and test in the market.
- Once an MVP has been validated (has met the goals set by the team), the teams will start building a solution that can be made in the time frame left. They will focus on Minimum Feature Sets, with just the essentials. Teams will focus on releasing often for customer feedback and iterations.
- At the end of the quarter, the teams can decide whether more investment is needed in adding to the solution after they have received customer feedback in the initial stages. The roadmap will be adjusted for this.
- At the beginning of the next cycle, the process repeats. We revisit the list of problems and re-prioritize them based on customer needs, or add in new problems.
This Problem Roadmap solves many of the problems above. Instead of thinking in features, we think about how we can deliver the most value to our customers by solving their problems. Estimation for complex feature sets is not needed, since we focus on minimum solutions that can be built and released in short time frames to get rapid customer feedback. The roadmap is flexible since problems are constantly being evaluated, and irrelevant problems will be squashed at the beginning of each cycle. We can communicate the problems we plan on tackling to the team and our customers without committing to solutions we do not know will solve the problem.
What do you think? What have you done with Product Roadmaps that has been successful? If you decide to try Problem Roadmaps, I’d love to hear more about it!
“We’re the Amazon of cats.”
“We’re the Uber of drinks.”
“We’re the Netflix of hair care.”
“We’re the X of Y” seems to be all I am hearing startups describe themselves as these days. It’s become so prevalent, you can find youtube videos or dumb HBO shows that use these sayings in dialogue to mock entrepreneurs. When I was learning to pitch a startup at Techpeaks, we were encouraged to use these analogies to describe our new ideas. We were told that investors would understand us faster. I think overall this is actually hurting the quality and originality of products in the long run.
When someone asks what your company does, you should communicate the value to the user immediately and clearly. For example, if you are a wine delivery service that sent a few new bottles every month to my users to try so they can discover new ones, would you want to be remembered as:
“Netflix for wines.”
“Winernator – A way to experience and discover new and unique wines every month.”
Every description of the company should be about the value to the user, not somebody else’s valuable company that kind of resembles yours. But this isn’t the part that’s incredibly harmful. The problem is that these comparisons are starting to take a toll on the product and user experience.
As a UX consultant, I’ve been helping startups/ larger companies come up with their product strategy and create new business lines. I recently had a situation where my team and I had been working with a client who wanted to be the Skyscanner* of Mortgages*. When we asked the founder what value did he want to provide to his users, he talked a lot about the current problems in the market. He explained that he wanted Skyscanner because it allowed people to drill down on the right flights for them, something he wanted to do for his financial products.
Great. We had the information we needed to make a great experience for his users based off this problem of drilling down to the right products for the right users. We presented him with three different options for their site. He wasn’t satisfied. After a lot of back and forth he said he did not want these experiences, he wanted Skyscanner. So we made something that was a bit more similar to skyscanner with their filters and took it back. No. He wanted Skyscanner.
He literally wanted the entire experience and layout to be a duplicate of Skyscanner… but for mortgages.
The experience for Skyscanner was made for people searching for flights, not mortgages. Users don’t search for mortgages the same way they search for flights. They will be in a different setting, in a different circumstance, with different concerns. You have to qualify people for mortgages. You don’t qualify people for flights. The requirements and the use cases are different, and the experiences don’t match up.
Unfortunately this isn’t the first time I’ve been in this position. This is what happens when you let another company’s value become YOUR product’s value. Founders think about their pitches and product positions so many times a day, that it becomes ingrained in them. If you are staring at other people’s products all day and reciting yours as an analogy to them, you are going to blur the lines between what is your product and what is theirs.
I am not saying don’t use a business model that works. I’m saying don’t think because it worked in one vertical it will work in another. Each product we build needs to be unique and original for our targeted audience. Every user’s reaction differs depending on what their situation is at that moment of need. If they are filling out very important forms, they might be incredibly careful, articulate, and neat. If they are taking notes in a meeting, they might be rushing to get things done. This doesn’t mean because they are writing that the same product will work or even be needed in both cases.
I would love to hear founders drop the X of Y analogies and start explaining their products for the value that they convey, not the other successes they wish to imitate. If your product is solving a problem in an original way, let that shine through. To me, that’s the better pitch.
[*Note: Analogies changed for anonymity's sake.]
This past week I had the opportunity to work with Rovio’s team, teaching them about Lean Startup. While getting the hang of Build, Measure, Learn, my team was saying that Lean Startup couldn’t apply to them because they are making entertainment products, not solving a customer problem. I’ve heard that argument before not only from entertainment companies, but also internal teams trying to solve problems for the business. Yes, these teams are not usually solving the dire needs of their users. The user is either delighted by their experience (entertainment) or tolerant of your business optimization (increasing revenue).
When I was starting out with Lean at OpenSky, I ran into this problem. As a Product Manager, I would get handed themes or features about what we should build next from upper management. The goal was always to increase revenue. I was struggling with how to apply this exact model to solve our business problems. My “customer” (OpenSky) and my “end user” (actual users) were not the same people. How do you apply lean to get people to invite their friends? How do you figure out an end user problem to “we need customers to buy more things”? Those are purely business problems. No customer is banging down the “door” of a clothing shop saying pllllease take more of my paycheck.
BUT you can still use lean in these cases. This is how.
Do Build, Measure, Learn in reverse.
Instead of focusing first on what your customer problem is (and here you will get stuck for reasons I said above) start with this question: What do we want to learn?
Ask your team and managers what do we want to learn about our users in the context of this business problem. For example, when a manager tells you they want users to visit your site more frequently, you turn that into: “Why are my users not coming back to my site more than twice a month?”
Now you can call your customers who are not returning and ask them why. Are they having specific problems with the site? Learn what they come to the site for, why they come when they do, and what prompts them to revisit sites in general. Sometimes this act alone will help you discover a big problem for the customer. Maybe the user wanted to come back to the site but always forgot during their busy schedule. You can start with your build measure learn loop right there.
Most of the time, you are going to struggle to find that “dire” problem the customer has that solves your business need. From this question, I’d expect to hear “I forget, but it’s not such a big deal for me. I like shopping when I remember to.” So after researching, you build an experiment around the how. “What can I do to get users to come back to my site more than twice a month?”
Figure out how you would measure that lesson. In this example, we would want to track how many times per month users visit the site through our vetted channels.
Take what you learned about users and create a hypothesis and a test about what they will do and enjoy. “My customers come to our site when we send emails. My users are always trying to keep in touch with their friends. I think if we send them an email once a week with activity their friends had on the site, they will come back.”
Set your minimum success criteria to gauge your progress. We’ll send this email to 100 users for one month. If we get 60/100 people to click through and visit once a week, this will be a success. Then run and pivot as you would with any Lean concept.
Doing Lean inside an existing business is actually EASIER than at a startup. You already have customers to tap into for running your experiments. There are actual users to call for research when they do not respond to your initial tests. This makes doing Lean a million times easier, and should not be put to waste! Lean is not just for startups.
Try this and let me know how it goes.
In 2013 I spent 210 days in Europe. Lived in 2 countries. Visited 8 countries, 5 of them for the first time. Had 3 different jobs. Founded 1 company. Raised €25.000. Spoke at 2 conferences, 3 meetups, 2 accelerators. Made friends from 19 different countries. Flew my dog 8000 miles. Went to 1 hackathon, came in 2nd. Took a spontaneous trip to Slovenia. Skied the alps once. Participated in Oktoberfest. Moved across the world, and then came back home.
2013 was a rollercoaster of a ride. I got the chance to do many things I’ve always wanted to, and for that I’m thankful. There was also a lot of hard lessons to be learned along the way, making it a pretty stressful and by no means easy year. Worth it though? Yes. Keeping the same tradition as last year, here’s 5 of the biggest lessons I’ve learned along the way.
1. Trust your gut
I have always given things a fighting chance to work out. Jobs, relationships. I am a fixer by nature, and I try to fix broken things. I keep telling myself “but it might get better and then you’ll want to be here so try your best to figure out what can be mended.” I do like the concept of giving things a fighting chance, but somewhere along the way, I started giving things a chance for too long. I stopped listening to my gut.
In Italy, I formed a company with three other people. One left at the very beginning and we were down to three of us. Soon, problems started arising with one of the cofounders. Big problems that just couldn’t be shoved under a rug. I stuck around for about 4 months after the warning signs emerged, hoping things would get better, trying to make them so. My subconscious was telling me it never would, but I thought in my head, if we could just fix this relationship, we could make a successful business and keep going. It came to a point where I had to make a decision to finally trust my gut or keep fixing. I chose to leave. The next morning, I felt like a load of weight had been finally lifted from my shoulders. I started to feel better too, less stressed and more healthy. I started hanging out with a lot of other people in the program, and made many friends I intend on visiting and working with in the future. Everything finally got better.
Your body can tell you a lot that your mind can’t. Have you ever made a decision but felt sick to your stomach? I have. Something inside you is telling you that it’s the wrong decision. Don’t ignore it. I spent too much time listening to my head this year. Next year, I’m going to listen more to what my gut tells me.
2. The grass may be greener on the other side, but its not your grass
I always wanted to live in Italy. It’s a more relaxing life there. There’s thousands of years worth of history that we don’t have here. In these cases it is better than the US, but we always want what we don’t have. Living in Italy was a great opportunity. I learned a lot about people. About cultures. About bureaucracy. I would never have been exposed to these things had I stayed in NYC. But, I also learned a lot of what we have right here is better. I’m thankful I can come home to that.
When you work in startups, like me, the US provides you with so many more opportunities than many countries in Europe. The bureaucracy I witnessed there was astounding. You’d go broke just trying to incorporate a company in Italy. Most of the people I worked with were trying to come here. I wish our immigration laws would let them. There’s a slew of talented people who could be creating jobs with their startups in this country. For me, I learned this is the right place to be when I start a company.
I by no means want to stop traveling. In fact, I want to travel more. There are still places I want to see, cultures I want to experience. I want to live somewhere else for a while too, experience that city, it’s highs and lows. Berlin is looking pretty good. But I will not take for granted the excellent opportunities we have in the US when it comes to smoother processes, salaries, and starting companies.
3. You don’t have to settle
For the first time in my life, I’m not 100% sure what I will be doing next. I’m fortunate to have many opportunities, and I want to be sure I pick the best one before I dive in with both feet. 2 years ago (hell, last January) not having a full time job lined up would have scared me to death. I couldn’t fathom leaving one company without knowing where I would be next. When I was no longer happy at OpenSky, I started looking for my next opportunity. At first I couldn’t find anything I liked. I had a list of things that were very important to me, but none of the companies I interviewed with fit the bill. After a few months of searching, countless interviews, and many offers turned down, I was offered a job at Conductor. I had reservations about the company culture and the size, but for the most part it looked like a really good fit and I was tired of searching. I took the offer and started quickly after, excited to get to work. After only a month though, I could tell it was the wrong place for me. The culture and the size turned out to be quite different from the environments in which I thrive. It just wasn’t a match. When the opportunity to go to Italy arose, I jumped at a chance to build something of my own.
This time I’m not settling for anything less than “Hell Yes!” I want to find something to work on that keeps me up at night. For now, I am consulting on UX Design, Product Strategy, and Lean Methodologies, something I really love doing (so reach out to me if you want to chat!). I’ve been advising various startups and teaching classes and workshops. By doing the things I love, I’m confident I’ll find something I really want to be a part of instead of settling for something less.
4. Surround yourself with people who make you better
Ever have a friend that always said they would hang out and never followed through? How about a coworker who always put down your ideas? I’ve had quite a few in both categories. This year I learned to cut them loose.
I met a lot of different people this year. Some awesome, some not so awesome. I met people who made me want to stay up until 4am trying to hack things together. I had my first brush with men who could not stand to have a woman CEO. I made some friends who’d let you invite yourself over for dinner on a bad day without warning. I saw firsthand people who would do anything to be perceived as important. There’s a lot of people in the world, and who you choose to associate with has a huge effect on your happiness and productivity. I spent quite some time this year on people who brought me down. They chided my ideas. They ignored my calls. They didn’t support me.
Near the end of the year, I started making some changes and cutting out these people. Turns out, once you get rid of those toxic people, you have room to surround yourself with people who make you better.
5. Let go of your things
I lived for 6 months out of 2 suitcases I brought to Italy with me. When I got there I bought a hair dryer, a toothbrush, and necessarily toiletries. Oh and a fan, because it was DAMN hot in August. Wish I bought that sooner. I didn’t wear some of those clothes because, well, startup life. Also, I only had 2 pairs of shoes once it got cold, but if I had one more pair I probably would have been set.
I opened up my storage when I got home and felt like someone from Hoarders… and it’s really just a small storage! But now all the things I own seem to weigh me down. Why do I keep all those college t-shirts from slope day? Every time I thought about moving in the past I always had some excuse to tie me down. My apartment is too awesome, and I love living with my roommates. I have too many things and I’ve been moving the past 9 years. I REALLY need a tortilla press and 7 coffee makers, so obviously I should just stay here. All these things ended up being a huge burden while I was away, even with a lot of help from some very special people. There were arguments, payments on storage, and just a ton of stress. If I didn’t own so many things, where could I go? Where could I move to if I wasn’t worrying about them?
I’m taking a look at everything I own this year and reducing the clutter. I’m keeping my Cole Haan boots though and by god I will wear them all over NYC, even if I want to sit down, rub my feet, and cry a bit every 5 blocks.