Embrace uncertainty to successfully pilot new offerings
Large corporations, under increasing pressure to make bigger and bolder shifts, have started to turn to pilots as a way to launch products that are very different from their core offerings. The buzz and expectation of the last few years that pilots can help a company successfully launch innovations is one of the reasons companies are turning to them more today.
The “Lean Startup” by Eric Ries and “The Innovator’s Method: Bringing The Lean Startup Into Your Organization” by Nathan Furr and Jeff Dyer are both great examples of some of the latest thinking around pilots and a lean startup approach to developing and launching new offerings. What is underemphasized, though, is the need for mature, established companies to fully embrace the high degree of uncertainty in bringing an innovative offering to market. This means shifting to a totally different mental model to guide development.
When attempting to launch new products and run pilot programs, most established companies apply the same mental model of development they use for their core offering, which is often based on tangible and low risk uncertainties, known upfront, that don’t radically shift throughout development. This type of development process is very linear and sets the budget, timeline and structure of a pilot early without straying much from the original scope. In applying this same mental model of development to offerings that have a high degree of uncertainty and risk, companies don’t incorporate methods to de-risk a new offering before running a pilot. And they also lock in the scope and structure of a pilot based on set timelines and budgets versus what it will really take to successfully de-risk the offering.
Ultimately, this approach either causes companies to pull the plug on launching something new because there are too many uncertainties left to address, or they prematurely launch an offering that is doomed to fail in the market.
To increase the odds of a successful pilot, companies should follow a process that actively surfaces the uncertainties of the offering early, which can then be addressed throughout the development process and guide the scope and structure of a pilot. This can be done in the following three ways:
1. Start de-risking before a pilot
De-risking an offering can start very early in concept development. Conducting iterative qualitative customer research loops using low-fidelity prototypes is one of the most effective and cheap methods for starting to understand and solve for basic uncertainties.
In “The Lean Startup,” Eric Ries illustrated what can happen if you don’t build in early customer feedback loops. In 2004, Ries’s team was working on incorporating Instant Messaging (a huge market at that time) into a virtual world video game with the hope of scaling the offering virally through users’ IM friend networks. They fully developed the offering and launched it without getting any input from customers – and as a result, it failed to get any traction when it launched. His team then went back and discovered through qualitative research with customers that “the IM concept was fundamentally flawed.” Customers did not want to use their existing IM accounts for the game; they wanted the game to provide its own IM network that wasn’t limited to their friends, but allowed them to play the game with strangers, as well.
This kind of learning is a good thing when it happens in early concept development; it’s a bad thing when it happens after launch. Fundamental uncertainties, especially around what the customer desires, can be understood and solved early in the development process – well before a pilot gets underway.
2. Assess risk level to determine pilot scope
Once an offering has been more fully defined through the concept development process, assess its level of risk by asking yourself about its relative “newness” (both in terms of the company’s current capabilities and the current market it targets). If the offering requires both new company capabilities and a new target market (versus only one or the other), then the level of risk is very high – and so are the uncertainties, which can require a more intensive pilot program in terms of timeline and cost.
Google Glass’s risk level is quite high because it not only requires new technical capabilities, but the target market is also completely unknown. Google has been smart in not applying a fixed pilot scope that locks in a pre-determined timeline and launch date for the product because it knows it’s a high-risk offering that needs to go through not one pilot test but many. Google started early pilot testing in 2012 with just developers as the first beta test group. It then expanded testing to a wider user pool called “Explorers” in 2013 and continued to add users in 2014 for further testing. The company has still not announced a launch date.
If Google had locked in a pilot timeline and launch date early on, before starting to experiment and test with users, the timeline would have most likely been too short to de-risk the uncertainties of such a high-risk offering.
3. Outline critical uncertainties to determine pilot structure
In order to decide how to structure a pilot, outline the critical uncertainties of the offering and then structure the pilot to focus on testing those uncertainties. It’s important that you outline the uncertainties with the most risk associated – i.e. the ones that can ultimately make or break the success of an offering if you don’t solve for them before launch. It won’t be a successful pilot if the uncertainties you focus on are relatively unimportant.
In “The Up Side of Down: Why Failing Well Is The Key To Success,” Megan McArdle writes about “The Peril of the Promising Pilot.” She uses the example of the New Coke launch to illustrate the limits of experimentation. While pilot testing does have some limits, it wasn’t the limits of testing itself that led to the now infamous market failure of New Coke. Coca-Cola didn’t structure its testing to address the critical uncertainties of the potential new product, which was one of the contributing factors to its failure.
Coca-Cola spent millions on testing New Coke, which included testing market samples with customers at different venues. While it did receive positive input from the New Coke samples, providing samples to customers was not an effective testing structure to answer some of the critical uncertainties such as: “Do customers like New Coke more than Classic Coke?” and “Will customers drink New Coke in the same quantities as Classic Coke?” To understand those two uncertainties, Coca-Cola could have designed a more in-depth pilot test with current Coke customers that looked to understand their consumption rates of both products over a set period of time.
Uncertainty is what most companies spend a lot of energy trying to avoid through their existing development processes. However, to bring a truly new and innovative offering to market that is both meaningful to customers and adds value to the company, uncertainty should be embraced early and throughout development to increase the probability of success.