I recently received a letter from an anthropologist commenting on a new paper in the Proceedings of the Royal Society. The topic of the report was the Bigfoot, or rather, the genetic analysis of different hairs, which, according to people at different times, belong to a giant, hairy and unidentified primate.
An international team of scientists, led by Oxford University geneticist Bryan Sykes, has found no evidence that the DNA of these hairs belongs to a mysterious primate. Most of these hairs belonged to completely non-mysterious mammals such as porcupines, raccoons and cows.
The author of the letter expressed his opinion on this matter very sparingly and expressively: "Well, of course."
The new report will not go down in history as one of the greatest scientific studies of all times and peoples. He will not change our ideas about the natural world or about ourselves. But he shows the illogicality and paradoxicality with which modern science operates.
People often think that the task of scientists is to prove the truth of hypotheses - about the existence of electrons, for example, or about the ability of drugs to cure cancer. However, scientists very often do exactly the opposite: they refute hypotheses.
It took them many decades to develop this technique, but one day in the early 1920s holds a special place in history. At an experimental agricultural station in England, three scientists decided to take a break and drink tea. A statistician named Ronald Fisher poured a cup and offered it to his colleague Muriel Bristol.
She refused. She liked the taste of the tea in the cup where the milk was first poured.
“Nonsense,” Fischer said. "Of course, there is no difference here."
Promotional video:
But Bristol persisted, claiming she felt the difference.
A third scientist from the company named William Roach proposed an experiment. (Actually, there was a moment of scientific flirting here, for Roach and Bristol were married in 1923.) But how can you verify Bristol's claim? The easiest thing that Fisher and Roach could do was pour a cup of tea without her seeing it, give her a taste and offer to guess in what order it was poured.
But if Bristol had given the right answer, it wouldn’t be considered proof that she had any supernatural powers about tea. Since the chances of a correct answer were 50 percent, she could very well have given it by accident.
A few years later, in 1935, Fisher wrote The Design of Experiments, where he explained how one could test such a claim. Instead of trying to prove that Bristol is able to tell the difference between two cups of tea, one should try to refute the hypothesis that she makes her choice at random. "We can talk about such a hypothesis as a 'null hypothesis" - wrote Fischer. - The null hypothesis cannot be proved or substantiated, but it can be refuted in the course of experiments. We can say that every experiment exists only to give facts a chance to refute the null hypothesis."
Fischer outlined a way to disprove the null hypothesis that the choice of Bristol is random. It is necessary to prepare eight cups, first pour milk into the first four, and tea first into the second four. Then mix the cups and invite Bristol to taste the tea from each one in turn. As a result, she has to divide the cups into two groups: the first, where milk was first poured, and the second, where milk was poured after tea.
Bristol is said to have passed the exam brilliantly, having correctly identified all eight cups. Thanks to the design of Fischer's experiment, the chances of her correctly dividing the eight cups into two groups were slim. She had 70 different ways of dividing the eight cups into two groups of four; and this means that Bristol could determine the cups correctly by chance only in one case out of 70 attempts.
The Fisher test could not completely eliminate the possibility that Bristol was acting on guesswork. He was simply showing that the chances of her guessing by chance were insignificant. Fischer could have lessened those chances even further by suggesting that Bristol taste more cups of tea, but he couldn’t reduce the chance of her guessing to zero.
Since absolute proof was not possible, Fischer preferred practicality in his experiments. In the laboratory where he worked with Bristol, Fischer was responsible for analyzing decades of data to determine if the information could provide an indication of any details such as the optimal fertilizer composition for crops.
Scientists could use this data to design ever larger experiments with ever more accurate results. Fischer believed that it was pointless to design an experiment that took centuries to produce results. At some point, Fischer believed, scientists would simply have to stop it.
He believed that a reasonable threshold was five percent. If we assume that the null hypothesis is true, and find that the chances of scientific observation on these data are less than five percent, then we can safely refuse such an experiment. In the Bristol story, the odds were well below Fisher's threshold at just 1.4 percent.
Thanks largely to Fischer, the null hypothesis has become an important vehicle in scientific discovery. Null-hypothesis tests can be found today in all branches of scientific knowledge, from psychology and virology to cosmology. And the scientists are following Fischer's recommendation using a five percent threshold.
But back to Bigfoot
Humans have claimed for decades that they have observed hairy humanoids. They show grainy photographs, questionable footprints, and mysterious tufts of hair. In recent years, they have even tried to extract DNA from these hairs, but scientists have dismissed such genetic studies because they lack the standard precautions commonly used in studies of this kind.
Proponents of the Bigfoot hypothesis have repeatedly argued that professional scientists willfully ignore compelling evidence. But in reality, the problem is that these proponents do not approach the question of the existence of Bigfoot from a scientific standpoint. Therefore, two years ago, Sykes and his colleagues decided to conduct a scientific study of this "anomalous primate" hair. To do this, it was necessary to create a null hypothesis in order to try to refute it.
They developed the following null hypothesis. Hair, allegedly belonging to Bigfoot (yeti, bigfoot or whatever this creature is called in different places), does not belong to some previously unknown primate, but to famous mammals. They extracted DNA fragments from 30 different hair samples and managed to isolate the same short pieces of DNA from each other. They then compared such a stretch with the corresponding DNA stretch sequenced in many living mammals.
The result was clear and understandable: scientists found exact matches for all 30 samples, finding them in known mammals.
Have Sykes and his colleagues proved that Bigfoot does not exist? Not. It only means that Sykes, unlike Fischer with his tea experiment, was unable to refute the null hypothesis. The question remains open, and if Bigfoot does not exist, it will remain unanswered forever.
I must say that Sykes' experiment presented some surprises. Two hair samples from the Himalayas matched the DNA base sequence that was recovered from a 40,000-year-old polar bear fossil. Even stranger was the fact that his DNA does not match the DNA of living polar bears.
In their talk, Sykes and his colleagues propose a scenario of how such a result could have happened. It is possible that ancient polar bears and brown bears interbred, and some of the bears living in the Himalayas still have some DNA from ancient polar bears.
Some skeptics put forward a different explanation for Sykes' findings. It is possible that the DNA that supposedly belongs to a polar bear actually belongs to some living mammal - perhaps a brown bear - that underwent several mutations that created a false similarity to the DNA of an ancient polar bear.
It turns out that these skeptics essentially created a null hypothesis. And there is a simple and sure way to refute it. Scientists will need to find more DNA from these mysterious bears. If other regions of DNA also match the DNA of an ancient polar bear, then scientists can disprove the null hypothesis.
This is how science moves, from one null hypothesis to another.
Karl Zimmer, New York Times columnist and author of 12 books, including A Planet of Viruses.