It's almost a truism that we should judge the quality of a strategy and the projects and initiatives that come from it based on their outcomes. We're results oriented after all, aren't we? Unfortunately using outcomes as your sole measure is usually a terrible way to judge effectiveness. Outcome-based measures are usually the holy grail of performance measurement but if our goal in the end is to make our decision making process repeatable and to learn from past experience, judging solely on the basis of a few outcome data points won't get us very far.
What we need to do is focus on the quality of the strategy and decision-making process that led to the outcome. Our first question should always be whether the process was sound using the available data to make as informed a decision as possible. As Michael Mauboussin shows in his book The Success Equation, there is a continuum in most things in life between skill and luck. Most endeavors combine elements of the two. We can and should use skill to make decisions but in the end luck will act on those decisions in ways we cannot predict. The more complex and multi-faceted the issue and the more players there are involve, the greater the role that luck plays.1
As a result, luck can change the outcome or any strategy or project radically. If we rely on those outcomes to judge success then we will draw the wrong conclusions about the strategy and make worse decisions in the future. Over time we'll see the average outcome get worse.
As a personal example, my parents took me on a trip to the UK when I was 8. I was fascinated by the slot machines in all the pubs and restaurants since we didn't have them where I grew up. After a lot of pressing, my parents gave me a pound to play one of the machines as a way to teach me about how gambling leads to loss. I put in the pound, pulled the lever, and I hit the jackpot. As the coins poured out of the machine I could have confused strategy with outcome. That would have set me on a course to be a life long gambler. Fortunately I got the basic point from my parents without having to read The Success Equation and almost never gamble today. I was lucky not good at pulling a lever.
We obviously cannot ignore outcomes altogether and certainly we need to ameliorate bad outcomes and attempt to replicate good ones. But I urge leaders to probe a bit further. What was the strategy and decision making process you or your team used? Was it well thought out, inclusive, and structured? Is it repeatable? How many data points to you really have? Over a sustained period, a sound strategy and decision-making process will eventually swamp luck.
We all know that soccer and football games can turn on the way a player feels that day or the way the wind blows. Maybe we should have all invested in Bitcoin or listened better to the stock tip in the elevator. Luck plays a big role in life and it is tempting to try to draw conclusions from a single success or failure. Resist the temptation and focus on implementing a sound strategy and planning process. It will serve you much better in the long run.
1Interestingly, he also points out that the greater the collective skill of all the participants, the greater role luck plays. For example, when Roger Federer plays Nadal the role of luck is far greater between these two highly skilled players than it is when he plays an amateur with a vastly inferior skill level. In the later case the amateur loses entirely based on Federer's superior skill.