A recent column in Wired magazine recounted the story of the 5,127 prototypes used to create the first Dyson vacuum cleaner. In this column, Sir James Dyson notes his similarity to Thomas Edison, who said, “I have not failed. I’ve just found 10,000 ways that won’t work.” Dyson appears to take pride in his 5,127 prototypes as emblematic of the persistence and fortitude of an entrepreneur. In contrast, I think this extraordinary number of unsuccessful trials may illustrate a very fundamental misconception about innovation.
First, I should point out that I think Dyson is a brilliant entrepreneur who has created a very successful company. I also greatly admire his advocacy of design and engineering education. He gets an extraordinary number of things right. Nevertheless, I believe his approach to innovation, brute force trial and error, has severe weaknesses. While Dyson says that, “…each failure brought me closer to solving the problem,” it is not clear that his 5,127 prototype, 15 year journey should be used as a model.
I agree that if Edison were alive today, he would undoubtedly use Dyson’s approach. The real question is whether the approach of Edison and Dyson, is good engineering. While Edison may be a deity of innovation to the lay public, not all engineers share this view. Consider the viewpoint of Edison’s contemporary and former employee, Nikola Tesla. Tesla was the technical genius behind alternating current (AC) power. This is dominant form of power distribution today, and it became dominant because of its compelling advantages over direct current (DC) power. (DC power was tirelessly advocated for by Edison.) Like Edison, Tesla was a creative genius; unlike Edison he was a skilled engineer. What did Tesla think of Edison’s brute force, trial and error approach?
“If Edison had a needle to find in a haystack, he would proceed at once with the diligence of the bee to examine straw after straw until he found the object of his search…. I was a sorry witness of such doings, knowing that a little theory and calculation would have saved him ninety per cent of his labor.”
– Nikola Tesla, New York Times, October 19, 1931
I believe that Tesla would have the same opinion of Dyson’s 5,127 prototypes. In 30 years working with engineering organizations I have never seen a product, innovative or otherwise, come within an order of magnitude of this number of prototypes. Why is this the case? Because great engineering organizations don’t just design products, they also design efficient strategies to find solutions. Unfortunately, these strategies are much less visible than the products they produce.
What do we mean by an efficient strategy? It is one that generates the maximum valuable information with the minimum expenditure of time and money. There is a science behind generating information and it is called information theory. Why is it relevant? Because design processes must remove risk, and removing risk requires generating information. Information theory shows us that the information generated from a pass/fail test is most efficiently generated at a 50 percent failure rate. In fact, the two worst places to operate a design process are at 0 percent failure rate and at 100 percent failure rate. A very high failure rate is as dysfunctional as a very low failure rate.
How do great engineering organizations achieve optimum failure rates?
- The direction of their first step is determined by a hypothesis as to where a solution may be found. This step is the easiest.
- The magnitude of their first step is chosen to create a 50 percent chance of failure. When steps are either too small, or too large, they will be inefficient at generating information. Unfortunately most companies gravitate to failure rates that are either too low or too high.
- Finally, the information generated by each experiment must be carefully analyzed and used to modify the search strategy. Each chunk of new information alters conditional probabilities and thus suggests a new direction for the next trial. This is challenging.
You may recognize this as the winning strategy in playing the game of Twenty Questions. If each question is carefully chosen to have a 50 percent chance of being correct, then over 1,000,000 possible alternatives can be explored in twenty questions. Success arrives by proceeding from general to specific questions; if you begin with specific questions you will never win.
So please do not help to perpetuate the myth that success at innovation is due to brute-force trial and error. Successful innovators, from Henry Ford to present day Internet entrepreneurs, explore possibilities systematically. With each result they modify their next move.
When I began consulting in product development 30 years ago a skilled entrepreneur told me success at innovation came from a willingness to, “…build a tall junk pile.” I now realize that the height of the junk was not as important as the underlying logic behind each experiment. All observers will notice the height of the junk pile; only the most discerning will spot the careful logic behind the entrepreneur’s search strategy. Try to be in the second group.
Originally published in May 2011