No One Ever Tests Counterintuitive Things

From "Alchemy" by Rory Sutherland:

"The problem that bedevils organizations once they reach a certain size is that narrow, conventional logic is the natural mode of thinking for the risk-averse bureaucrat or executive. There is a simple reason for this: you can never be fired for being logical. If your reasoning is sound and unimaginative, even if you fail, it is unlikely you will attract much blame. It is much easier to be fired for being illogical than it is for being unimaginative.

The fatal issue is that logic always gets you to exactly the place as your competitors. At Ogilvy, I founded a division that employs psychology graduates to look at behavioral change problems through a new lens. Our mantra is, 'Test counterintuitive things, because no one else ever does.' Why is this necessary? In short, the world runs on two operating systems. The much smaller of them runs on conventional logic. If you are building a bridge or building a road, there is a definition of success that is independent of perception. [...]

This may be true when you are building a road, but it is not true when you are paintings the lines on it. Here, you have to consider the more complex component of how people respond to informational cues in their environment. For instance, if you want vehicles to slow down, painting parallel lines across the road in the approach to a junction at increasing smaller intervals will help, since the narrowing gaps between the lines will create the sensation that the car is slowing less than it really is.

A great way of saying that (a) "No one gets fired for buying IBM" (or SalesForce, nowadays) – following logical paths derisks them, but also (b) there are many things that are not logical and you should test that because – by not being logical – other people won't stumble across those conclusions. A personal example of this is raising prices in the DTC space to increase sales. Most people would assume that decreasing prices would make a product more affordable and hence accessible, and therefore drive more sales. However, I found that doubling the price of a specific item increased sales as well, this time at a much better profit margin as well. This is because increasing the price changed who we were viewed as competing against and put us into a totally new category of products and shopping behaviors, and increased the perception of quality of our product.

The virtue of logic is that anyone following it will get to the same result. But the drawback of logic is that anyone following it will get to the same result. If you're looking for reproducible things, follow logic. If you're looking to innovate, do the exact opposite.

Update on Feb 22, 2022:

Revisiting this a year later, I realize there's a bigger "thing" here. In most business management, the goal is to increase confidence and decrease risk (something I'm writing about in a soon-to-come blog post).

In the meantime, pulling from "The Product Management Reading List", I came across another article that highlights a similar argument from a different perspective:

• How to be led astray by your analytics (Roger Martin)

To quote:

Analysis is useful only to the extent there is a theory that is being tested. And that theory must be stated with sufficient precision that we can declare in advance, not after-the-fact, that the theory holds if we observe a particular and specific pattern of data. Instead, lots of data mining occurs and as they say, if you torture the data enough, it will give you something — just not anything useful.

As at the time of its analysis, all data is from the past. The goal of DA (Data Analysis) is to know something about the future. Thus 100% of DA is an act of extrapolating the past into the future. As explained by the father of DA — 4th century BC Greek philosopher Aristotle, who created the scientific method of analysis — DA works only when the past data is representative of future data. That only happens when the future is guaranteed to be identical to the past, which is the case with many attributes of our physical world — e.g., with respect to gravity, or the boiling point of water, or the rotation of the earth. But as I have pointed out before, the vast majority of the business world in no way features this characteristic. It is always changing. Thus, any sample of data fails to be representative of the universe that includes the future. Yet the DA aficionados keep analyzing the past with techniques that assume — utterly without reason — that the sample of past data will enable valid inferences about the future. Aristotle figured out that was a dumb idea 2500 years ago. One would have thought that the business world would have figured it out by now.

Worse still, doing DA will convince you that the future will be identical to the past — because that is the only thing DA can do. DA can’t tell you the ways in which the future will be different than the past, because it crunches data entirely from the past. This is a reason why all of those startups invent the future long before the powerful incumbents. It isn’t despite your DA: it is because of your DA. They imagine a future while you analyze the past. As American pragmatist philosopher Charles Sanders Peirce pointed out, no new idea in the history of the world has been proven in advance analytically.

Essentially: counterintuitive things are counterintuitive because they don't show up in the data.

(Last updated: April 16, 2022)