One of the most famous failures of political polling occurred in the 1948 presidential election between Harry Truman (the democratic incumbent) and Thomas Dewey (the republican challenger). There were also a couple of third party candidates. The Gallup, Roper, and Crossley polls all predicted that Dewey would defeat Truman by a significant margin, but in fact, just the opposite happened. The results of the polls and the actual election results (in percentages) are given in the following table:
| Candidate | Crossley | Gallup | Roper | Election |
|---|---|---|---|---|
| Truman | 45 | 44 | 38 | 50 |
| Dewey | 50 | 50 | 53 | 45 |
| Others | 5 | 6 | 9 | 5 |
The Crossley, Gallup, and Roper organizations all used quota sampling. Each interviewer was assigned a specified number of subjects to interview. Moreover, the interviewer was required to interview specified numbers of subjects in various categories, based on residential area, sex, age, race, economic status, and other variables. The intent of quota sampling is to ensure that the sample represents the population in all essential respects. The Chicago Tribune felt so confident in the polls that the night of the election, the paper went ahead and printed the following morning's edition with a banner headline that Dewey had defeated Truman. The picture of Truman holding the paper is one of the most famous images in American politics.
What went wrong? Outside of the quota constraints, each interviewer was free to pick his subjects any way that he pleased. It is now generally accepted that this freedom of choice created selection bias in favor of Dewey. At the time, evidently, republicans were slightly easier to find and interview than democrats.