Jeremy Jarrell

Agile Made Simple

Tag: lean startup

Stop Doing Experiments and Start Making Bets

Many teams have embraced the experiment mindset, which teaches that you can learn more from simply trying out an idea and seeing the results than you would ever learn from…

Many teams have embraced the experiment mindset, which teaches that you can learn more from simply trying out an idea and seeing the results than you would ever learn from intense upfront analysis.

While this mindset can help your team make rapid gains, there’s also a dark side to this approach.  Although experimentation can yield faster and more in-depth learning than traditional analysis, many teams forget that some initial lightweight analysis can also be useful for helping them choose those experiments that are the most likely to succeed.  As a result, these teams embark upon poorly planned and risky experiments without first considering what they stand to gain…or lose.

The root of the problem lies in the word “experiment”.  By their nature, experiments can be very open ended.  This can make them a bad model for change since the lack of a defined end state can lead to a lack of clarity on how to proceed, or even a lack of understanding of what the team stands to gain from the experiment.

Playing the Game

A better model is a bet.  Imagine playing a game of poker but with no chips.  Without a need to commit chips then each hand becomes nothing more than a simple experiment with no potential upside or downside.  This means that you could play forever and still have no real investment in the game.

But adding chips to the hand changes the game.  When you’re working with a finite amount of chips you start to become very aware of the real possibility of gain and the real possibility of loss that hides in each hand.  

This forces you to weigh your options much more carefully.

Watch a great poker player play the game and you may be surprised at what you see.  Great poker players don’t play every hand…in fact, many even sit out more hands than they actually play.

Why?  Because great poker players know that they have a finite amount of chips and, if they want to play the game long enough for that big pay out, then they need to protect the chips they have.  This means that they evaluate each hand at the outset and decide if it’s a hand they stand a chance at winning.  If they’re not likely to win, then they simply bow out and wait for a better opportunity.  This is because they know that every hand has a cost, and the more bad hands that they pass on, the more likely they’ll still be in the game when those truly great hands come along.

Betting on Change

So should you abandon experiments altogether?  Not so fast.  A true embracement of an experiment mindset is one of the hallmarks that separates the best teams from the mediocre ones.  The difference, is that rather than simply diving headfirst into every opportunity, the best teams evaluate each of the possible experiments on their plates and choose to focus on those experiments that have the most potential for a positive upside, given the investment they’ll need to make.

For example, imagine that during the most recent retrospective your team complained that their Product Owner is not easily accessible throughout the sprint to provide feedback or clarification on items that are in progress.  As a result, your team has repeatedly found themselves reviewing items at the Sprint Review that turned out to not be what the Product Owner had in mind.  This means that they’ve had to invest additional time in the next sprint to rework those items, which has slowed the team down as a whole.

After a few minutes of brainstorming your team comes up with several experiments they could run which could help address the problem.  For example, maybe the Product Owner could appoint a proxy Product Owner on the team that would be empowered to provide feedback and make decisions on the Product Owner’s behalf whenever he’s unavailable.  Or, perhaps the Product Owner could make more of an effort to attend the team’s standup whenever possible so he’s available to provide feedback to the team as items are complete.  Of, maybe he could even keep regular “office hours” in the team room so he’s more accessible to the team a few days per week.

Placing the Right Bet

Any of these options could be worth pursuing, and when each is considered as an experiment they all look equally promising.  It’s only when we look at these options through the lens of making a bet do the tradeoffs between each become more clear.

The table below demonstrates a simple framework for evaluating different experiments through the lens of a bet.

Bet Potential Upside Potential Downside Odds of Success 
Proxy Product Owner Quick response to questions

Proxy may provide wrong responses leading to future rework

Proxy role becomes a distraction from that team member’s responsibilities

Product Owner attends standups

Slightly more availability to team

Quicker response to questions, if asked during standup

Greater impact on Product Owner’s time Possible
Product Owner office hours

Tighter collaboration with team

Quicker response to questions

Less impact on Product Owner’s time

Different way of working for Product Owner Likely

When a team starts to look at experiments through the lens of a bet, not only does the potential for gain become clearer, but they also better understand the investment that they’ll need to make to carry out each experiment.  This can help them better separate those experiments that are likely to fail from those that have a better chance of success so they’ll know where to focus their efforts.

Knowing the Cost

An experiment mindset can be a powerful tool for teams who desire to improve their way of working.  But, while experiments may appear to be free, few truly are.  Most experiments carry with them very real costs, but by evaluating the potential for loss or gain of each your team can focus on those experiments with the greatest likelihood for success…meaning that they’ll be around to play the game for a long time to come.

Want to see more about how to make textbook agile work on real teams? Check out my course, Agile in the Real World, for tips and techniques for making agile really work in your organization.

Don't have a Pluralsight membership yet? Try the entire Pluralsight course catalog free for 10 days here.

No Comments on Stop Doing Experiments and Start Making Bets

Why Your Minimum Viable Products Probably Are Not Viable nor Even a Product

Founders pay a lot of attention to Minimum Viable Products, or “MVPs”.  However, for all of the attention that’s paid there seems to be just as much confusion about what…

Founders pay a lot of attention to Minimum Viable Products, or “MVPs”.  However, for all of the attention that’s paid there seems to be just as much confusion about what an MVP exactly is.

Despite the name, MVPs aren’t intended to actually be viable products.  MVPs serve one purpose and one purpose only…to validate a specific hypothesis.  Sometimes that means you need to release a product to the market to validate that hypothesis, but usually not.

cup-of-coffee-and-napkin-sketchWhat this means is that most early MVPs actually aren’t viable in the marketplace.  In fact, they’re usually not even products.

Let’s dive a bit deeper into this.  Often when you first begin work on your product the hypotheses that you’re validating are less about specific features that would be tested with a build and more about your overall product direction or business model.  These are the “would you pay for this questions” that underly all assumptions about your product from the outset.  And, these are the most important assumptions that you need to be validating with your early MVPs.

So, how do you validate these early hypotheses?  You do so with the most lightweight MVP possible…a conversation.  You don’t need a functioning app, or even a prototype, to be able to have an early discussion with a potential customer about the validity of your idea.  Often, you just need a list of questions and a sketch on a napkin.  If this list of questions and sketch helps you to test whether or not your hypothesis is valid then this is your MVP.

Could this list of questions and sketch be considered a product?  Probably not.  Would it be viable in the marketplace?  Absolutely not.  But, it’s well suited to validating the specific hypothesis that you have at that moment because this hypothesis is about your overall vision for your product and where it fits in the market…not about specific features.

Meet the MMP

So, if an MVP is not always viable in the market then what is?  The MMP.

An MMP, or Minimum Marketable Product, embodies what people often think of when it comes to MVPs.  An MMP is the smallest possible product increment that could conceivably be viable in the marketplace.

Is a list of questions and a napkin sketch a viable product in the marketplace?  No.  But the first build of that product containing the bare minimum of features may be.  Often when we discuss adding only enough features into our product to create an MVP, we’re actually referring to an MMP.

All MMPs are MVPs

We use MVPs to prove a hypothesis that we’ve set for our product.  Depending on the nature of that hypothesis our MVP may be a sketch on a napkin, a landing page that collects email sign ups, or even just a cup of coffee and conversation with a potentialmvp-mmp customer.  It doesn’t matter what form they take, as long as they can be used to validate a hypothesis and move your product forward.

Once our hypotheses have matured to the point that we’re now testing actual features our MVPs start to become viable product builds, often in the form of MMPs.  Although an MMP will be more refined than the MVPs described above it can be just as useful for validating a hypothesis about your product.  The point we must remember, however, is that shipping a full fledged product to validate a hypothesis will take much more time and effort than walking a napkin sketch to a coffee shop.  This means that we should always be asking ourselves if we’re using the absolute lightest weight MVP possible to validate that hypothesis.

Choosing the Right Path

In the end, it all comes down to the nature of the hypotheses that you wish to validate and where you are in your product life cycle.   Early hypotheses concerning your vision or market fit can be much more effectively validated using a simple MVP.  This is because their lighter weight and more malleable nature will allow you to get feedback sooner and more easily adjust course if you take a misstep.  Later hypotheses concerning specific features, distribution channels, or platform decisions may warrant the investment in an MMP.  But your goal is to delay that investment as long as possible until you’ve proven as many of your initial hypotheses as possible using other means.

Want to learn more about choosing the right features for each step of your release?  Check out my course Agile Release Planning on Front Row Agile to learn more.

No Comments on Why Your Minimum Viable Products Probably Are Not Viable nor Even a Product

Beware Your Early Adopters

If you’re building a product then you want users…it’s that simple.   And you want these users as soon as possible because the earlier you can get users, the sooner…

If you’re building a product then you want users…it’s that simple.  

And you want these users as soon as possible because the earlier you can get users, the sooner you can get start to get that feedback that will be so crucial to your product’s success.

These first users are your early adopters and the feedback that they provide will be instrumental in the early stages of your product.  But as much as you need your early adopters there’s one thing that you must remember about them:  your early adopters are not your mainstream users.

Sure, there may be overlap.  And sure, with any luck, some of your early adopters will grow alongside your product into your mainstream users as your product matures.  But, more often than not, early adopters are an entirely different breed than your mainstream users.

Inside the Mind of the Early Adopter

Early adopters tend to have different needs than mainstream customers.  On the surface, both groups may be solving the same problems but by their very nature early adopters are more adventurous and are willing to put in more of an effort to do so.  And, while everyone wants a great experience, early adopters tend to be more tolerant of the occasional bug or odd usability choice.

But there’s an emotional component at play here, as well.  Often early adopters are just as attracted to your product for the fact that it’s new as they are for the underlying problem that it solves.  This newness is exciting in its own right and, in large part, drives their use of the product.  But this “I used your product before it was cool” quality isn’t a quality that you can count on to drive usage as a your product matures.

Inside the Mind of the Mainstream User

Mainstream customers are less drawn to a product for its newness but are also less drawn by the product’s feature set.  Since this may be surprising, let me explain.  Mainstream customers don’t buy a product because of its specific features, they buy a product because of the specific problems it solves.  This doesn’t mean that all of the same features that were compelling to early adopters aren’t also compelling to your mainstream customers, but it does mean that these feature aren’t likely to be the deciding factor to whether they make the jump.

Expanding on this, your mainstream customers will have much mess tolerance for the same bugs and usability issues that your early adopters had.  Whereas your early adopters viewed rough corners and hiccups with your product as a fair trade for getting access to a new and exciting product early your mainstream customers won’t be so kind.  These users will expect a complete and polished product that functions out of the box.  And, to the same end, while an early adopter may be willing to invest some time fiddling with and configuring their own experience, a mainstream customer will want an experience that fits their needs on day one.

Making the Jump

So, why does this matter?  This matters because if you expect your product to grow you’ll ultimately need to make the jump from your early adopters to your mainstream customers.  The catch, however, it that the same qualities that appealed to your early adopters may not entirely align with the same qualities that will appeal to your mainstream users.  Therefore, you may have to make the decision to alienate some of your early adopters, who have been with since the beginning, in order to grow to the mainstream.

To this end, you’ll also want to be cognizant of catering too much to the early adopters in your roadmap.  You’ll want to measure every step of the way once your product hits the market but beware of blindly following those metrics.  Don’t double down on features or trends that have had a lot of uptake in your product by your early adopters but aren’t likely to appeal to your mainstream customers.  By the same token, evaluate judiciously the decision to drop any feature with little uptake by your early adopters that potentially may be a play towards the mainstream market.

Product development is a journey.  On that journey you’ll encounter many customers all with different needs.  Don’t forget your early customers and the lessons they taught you at the beginning but don’t become trapped by those same customers, either.  By learning to tell the difference between the needs of your early adopters and the needs of your mainstream customers you’ll know when it’s time to take the next step in your product’s journey.

Want to learn more about how to grow your product from inception to the mainstream?  Check out my course Agile Release Planning on Front Row Agile to learn concrete strategies for planning your product's evolution.

No Comments on Beware Your Early Adopters

Type on the field below and hit Enter/Return to search