Correlations hurt your decision-making (not just your portfolio)

Decision-making in contexts that aren’t always easy to predict —such as investment—is challenging, especially during uncertain times. But one often underestimated habit can be helpful: gathering information from many diverse sources.

It’s not just common sense. Mathematics supports our suggestion. Information relying on only a few sources is more prone to be biased, or to point towards a specific trend even though the reality is different. When we analyze biased information, we are more likely to find correlations. This implies actual decision-making risk.

Consider the striking correlation between the per capita consumption of cheese and the revenue generated by golf courses in the US between 2000 and 2009. Shall we conclude that a golf entrepreneur should invest in cheese futures? Of course not. That reasoning is grounded on very limited data, which then immediately leads to drawing the wrong conclusions. Indeed, the origin of this sort of correlation is typically due to unobserved confounding factors, which are not apparent at the time of the examination.

The role played by unobserved causal processes is even more apparent if we look at the strong correlation between the average number of shark attacks and ice cream consumption. While this observation by itself leaves us wondering whether ice cream make people tastier, the truth is both events just happen to occur together more frequently in summer than in winter months. In other words, the correlation is explained by seasonality, while there’s no direct influence of sharks over ice cream sales or the other way round.

Gathering more diverse information invariably helps. In 1973, UC Berkeley’s graduate school was sued for gender discrimination, as the admission rate had been higher for men applicants (44%) than for women (35%). However, closer inspection exposed a different story: it turned out that women had just applied more often for departments with lower admission rates. Indeed, when considering one department at a time, a significant gender bias in favor of women was discovered for 4 out of 6 departments.

Reducing correlations in the input information is also a key prescription in training artificial intelligence algorithms: an image detector needs to be trained with a very large and varied set of chair pictures if the goal is identifying both four-leg chairs and swivel chairs.

The rationale is the same that also underpins the importance of diversification to reduce risk in portfolio management. Risk increases if you hold a similar position in a bunch of stocks following analogous trends. In this scenario, a way to decrease a portfolio’s volatility is to blend in a good quantity of uncorrelated or negatively correlated assets.

The takeaway is that if your sources of information are correlated and limited to start with, no post-processing can heal that. In extreme cases, you might fall victim to Simpson’s paradox, where the analysis of limited samples supports deductions that are completely opposite to the conclusions backed by more extensive data.

According to the words of the Nobel Prize-winning economist Harry Markowitz, diversification is “the only free lunch in finance.” We suggest you spend time and resources diversifying the ways you gather information.