Pick two headlines. We will tie them together.

Why the string always holds

The code knows six kinds of link. Two headlines can share a word. They can add up to the same number if you count a as 1, b as 2 and so on. They can have the same number of words or letters, or start or end with the same letter. Each kind of link is rare. With six kinds and twelve headlines, some chain nearly always turns up. On this board every pair is tied, most of them in one or two steps.

A conspiracy board works the same way. Pin up enough facts and look at enough ways they could match, and links appear. Each link is true. The plot is not.

In 1958 the psychiatrist Klaus Conrad gave this a name, apophenia: seeing connections where there is no reason for them, together with a strong feeling that they mean something. Michael Shermer later called the everyday version patternicity.

The same game in a trading room

David Leinweber wanted to show what happens when you go fishing in data. He took the yearly closing level of the S&P 500 from 1983 to 1993 and searched a United Nations CD-ROM full of data series from 140 countries. Butter production in Bangladesh, he found, "explains 75% of the variation in the S&P 500 over ten years."

Then he added United States cheese production and got to 95%. With sheep numbers he reached 99%. He called the result "utterly useless for anything outside the fitted period." It was a lucky fishing trip, and with enough series to choose from, one would always turn up.

Our desk does the same thing with 500 random series that mean nothing. In most runs the best three explain well over 90% of the past. Then the future comes, and in about three runs out of four the formula misses by more than a forecast of "no change" would. A fund that hires on a backtest like this is buying butter in Bangladesh.

Sources

1. David J. Leinweber, "Stupid Data Miner Tricks: Overfitting the S&P 500" (first written 1995; published in The Journal of Investing, Spring 2007).
2. Klaus Conrad, Die beginnende Schizophrenie (1958), where the word apophenia first appears.
3. Michael Shermer, "Patternicity: Finding Meaningful Patterns in Meaningless Noise", Scientific American 299, no. 6 (2008), 48.
4. The headlines are made up. The links are checked by code in your browser, in a fixed order: a shared word of four letters or more, the same letter sum, the same number of words, the same first letter, the same number of letters, the same last letter. The index is a random walk with about 15% of movement a year. Each of the 500 series is a random walk too. The code picks the series that fits years 1 to 11 best, then adds the two that help most, by least squares.