Cargo Cult Science

 

cargo cult by Michael Paukner

Cargo cult science refers to practices that have the semblance of being scientific, but do not in fact follow the scientific method. The term was first used by the physicist Richard Feynman during his commencement address at the California Institute of Technology in 1974. The speech is reproduced in the book ‘Surely You’re Joking, Mr. Feynman!’ He based the phrase on a concept in anthropology, the cargo cult, which describes how some pre-scientific cultures interpreted technologically advanced visitors as religious or supernatural figures who brought boons of cargo.

Later, in an effort to call for a second visit the natives would develop and engage in complex religious rituals, mirroring the previously observed behavior of the visitors manipulating mock machines but without understanding the true nature of those tasks. Just as cargo cultists create mock airports that fail to produce airplanes, cargo cult scientists conduct flawed research that superficially resembles the scientific method, but which fails to produce scientifically useful results.

Feynman cautioned that to avoid becoming cargo cult scientists, researchers must first of all avoid fooling themselves, be willing to question and doubt their own theories and their own results, and investigate possible flaws in a theory or an experiment. He recommended that researchers adopt an unusually high level of honesty which is rarely encountered in everyday life, and gives examples from advertising, politics, and behavioral psychology to illustrate the everyday dishonesty which should be unacceptable in science. Feynman cautions: ‘We’ve learned from experience that the truth will come out.

Other experimenters will repeat your experiment and find out whether you were wrong or right. Nature’s phenomena will agree or they’ll disagree with your theory. And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven’t tried to be very careful in this kind of work. And it’s this type of integrity, this kind of care not to fool yourself, that is missing to a large extent in much of the research in Cargo Cult Science.’

An example of cargo cult science is an experiment that uses another researcher’s results in lieu of an experimental control. Since the other researcher’s conditions might differ from those of the present experiment in unknown ways, differences in the outcome might have no relation to the independent variable under consideration. Other examples, given by Feynman, are from educational research, psychology (particularly parapsychology), and physics. He also mentions other kinds of dishonesty, for example, falsely promoting one’s research to secure funding.

The history of published results for a famous experiment (the ‘oil drop experiment’) is an example given in ‘Surely You’re Joking, Mr. Feynman!’ Feynman noted: ‘We have learned a lot from experience about how to handle some of the ways we fool ourselves. One example: Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It’s a little bit off because he had the incorrect value for the viscosity of air. It’s interesting to look at the history of measurements of the charge of an electron, after Millikan. If you plot them as a function of time, you find that one is a little bit bigger than Millikan’s, and the next one’s a little bit bigger than that, and the next one’s a little bit bigger than that, until finally they settle down to a number which is higher.’

‘Why didn’t they discover the new number was higher right away? It’s a thing that scientists are ashamed of – this history – because it’s apparent that people did things like this: When they got a number that was too high above Millikan’s, they thought something must be wrong – and they would look for and find a reason why something might be wrong. When they got a number close to Millikan’s value they didn’t look so hard. And so they eliminated the numbers that were too far off, and did other things like that…’ Thus, each new publication slowly and quietly drifted more and more away from the initial (erroneous) values given by Robert Millikan, rather than all having a random distribution from the start around what we now believe the correct result is. This slow and linear drift in the chronological history of results is unnatural and suggests that nobody wanted to contradict the previous one, instead only submitting for publication ‘agreeable’ results.

Physician Raymond Tallis describes the psychoanalytic school established by Jacques Lacan as an example of cargo cult science. Tallis argues that Lacan, who was poorly trained in both traditional medicine and psychoanalysis, superficially mimicked medicine and science, and that Lacan’s later devotees similarly mimic their guru’s confused concepts. The Cranfield University report ‘Aircraft Cabin Air Sampling Study’ provides a more recent example of cargo cult science. The government-sponsored study purported to measure the concentrations of toxic substances in aircraft cabin air but used such inappropriate[vague] methodology the results were of little value. Nevertheless, they were used to make comparisons with domestic environments of dubious relevance in order to assert that ‘…there was no evidence for target pollutants occurring in the cabin air at levels exceeding available health and safety standards and guidelines.’

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s