The tale of the largest and most inaccurate poll ever published for a U.S. Presidential election

Mark Roulston, Senior Data Scientist •

Blog posts 1

On 3rd November 1936 Franklin Roosevelt was elected to his second term as President of the United States. Roosevelt won the election by a landslide, winning over 60% of the popular vote and carrying 46 of the then 48 states.

The result came as quite a surprise to the Literary Digest, a weekly magazine that had mailed out over 10 million cards in the run up to the election to find out how people intended to vote. They received 2.4 million responses from which they predicted that Roosevelt’s Republican opponent, Kansas Governor Alf Landon, would win with a comfortable 57% of the vote compared to Roosevelt’s 43%.

The magazine never really recovered from the debacle which has the dubious distinction of being both the largest and most inaccurate poll ever published for a U.S. Presidential election. In fact, it was so discredited by this failure that it folded within two years.

It was also the catalyst for a considerable refinement of public opinion polling techniques, and later came to be regarded as ushering in the era of modern scientific public opinion research.

What was the Literary Digest’s poll biggest blunder?

The Literary Digest poll suffered from two major sources: it sampled Literary Digest readers and two other available lists: registered telephone users and registered automobiles owners who, in 1936, were not representative of the U.S. electorate. It also overlooked the ‘non-response bias’ which in this case meant most of the 7.6 million people who did not respond to the poll turned out to think very differently to the 2.4 million who did.

The tale of the Literary Digest opinion poll has been told many times to generations of statistics students. The story is worth repeating, however, because when it comes to data there are still people who subscribe to the old adage that “Quantity is its own quality”.

These people are wrong. One clear moral from the Literary Digest opinion poll story is that a badly chosen big sample is much worse than a well-chosen small sample.