Description

The table below gives the results of the 1954 field trials to test the Salk polio vaccine (named for the developer, Jonas Salk), conducted by the National Foundation for Infantile Paralysis (NFIP). It is adapted from data in an article in the American Journal of Public Health in 1955 by Thomas Francis, Robert Korn et al. There were actually two trials, corresponding to two statistical designs

The original design called for vaccination of second-graders at selected schools in selected areas of the country (with the consent of the children's parents, of course). The vaccinated second-graders formed the treatment group. The first and third-graders at the schools were not given the vaccination, and formed the control group. This design was known as the observed control experiment

In the second design, children were selected (again in various schools in various areas), all of whose parents consented to vaccination. The sample was randomly divided into treatment and control groups. Children in the treatment group were given the polio vaccination, of course. Children in the control group were given a placebo that looked just like the real vaccine. This type of design is known as a randomized control experiment. The experiment was also double blind: neither the parents of a child in the study nor the doctors treating the child knew which group the child belonged to.

Data

Table Notes

• The Exp column refers to the experiment: Observed Control (OC) or Randomized Control (RC)
• In the Polio columns, P refers to Paralytic and NP to Non-Paralyitic.
• The False Reports column refers to cases that were initially reported as polio, but were later determined not to be polio.
• The Not Inoculated row refers to the children in the designated grades who did not participate in the experiment, and includes 8,577 children who received one or two (but not three) injections of placebo.
• The Incomplete Vaccinations row in both experiments refers to children who received one or two (but not three) injections of the vaccine.
• The Controls row refers to the total first and third grade populations used as controls in the observed control experiment.

Discussion

The first design, the observed control experiment, suffers from selection bias and diagnostic bias.

In terms of selection, the treatment and control groups are different with respect to at least two variables: age and parental consent for vaccination. The critical issue is whether these variables are related to the variable of interest, namely contracting polio. The answer for the second variable, at least, is yes. Parents who consented to vaccination were on average better educated and more affluent, and lived in more sanitary conditions. As counterintuitive as it might seem, the children of these parents were slightly more likely to contract polio. Thus, this confounding variable causes bias against the vaccine.

In terms of diagnosis, the problem is that mild cases of polio resemble influenza and other common diseases. Doctors, who generally believe in the value of vaccines, would tend to diagnose polio slightly more often for the unvaccinated children in the control group than for the vaccinated children in the treatment group. Thus, the diagnostic bias is in favor of the vaccine.

The net effect of the two confounding variables in the observed control experiment is very difficult to determine, but could easily swamp the effect of the vaccine in a rare disease such as polio.

The second experiment was designed to overcome the deficiencies of the first design. Although the sample of children selected did not represent the population of children as a whole, the critical fact is that the treatment and control groups were essentially identical to each other in all respects except the variable being tested: vaccination. The double-blind, randomized control experiment is considered the gold standard of statistical experiments.

In an invited review article for the Journal of the American Statistical Association in 1955, KA Brownlee did not mince words in his scathing criticism of the first design, calling it stupid and futile and the results worthless.

Download

• Polio.tsv

Sources

• Statistics of the 1954 Polio Vaccine Trials. KA Brownlee
• An Evaluation of the 1954 Poliomyelitis Vaccine Trials, Thomas Francis, Robert Korn, et al.