Benjamin Franklin once said that any fool can criticize, judge, and complain - and most fools do just that. Richard Feynman once said about the scientific process: the first principle is not to deceive yourself - and you are the easiest to deceive. Skeptics believe that scientists can deceive themselves (either out of ignorance, or to keep their jobs), and often blame them for this - climatologists, cosmologists, anyone. In principle, it is easy to dismiss such criticism as unfounded, but an interesting question arises: how can we make sure that we are not deceiving ourselves?
There is a popular opinion in science that experiments should be possible to repeat and falsify. If you have a scientific model, that model should make clear predictions, and those predictions should be testable in a way that confirms or refutes your model. Sometimes critics understand this to mean that true science is accomplished only in laboratory conditions, but this is only part of the story. Observational science like cosmology also obeys this rule, as new observations can potentially disprove our current theories. If, for example, I observe a thousand white swans, I can assume that all the swans are white. Seeing a black swan will change my speculation. A scientific theory cannot be absolute, it is always preliminary, it changes when new evidence appears.
While this is technically correct, it is a little unfair to call well-established theories “tentative”. For example, Newton's theory of universal gravitation existed for several centuries before it was supplanted by Einstein's general theory of relativity. And if we can say today that Newtonian gravity is wrong, it works the same way it always did. We now know that Newton created an approximate model describing the gravitational interaction of masses, but close to reality so accurately that we can still use it to calculate orbital trajectories. It is only when we expand our observations beyond the (very large) range of situations in which Newton was right that we need Einstein's help.
When we collect evidence to support a scientific theory, we can be confident that it works with a small window for new evidence. In other words, a theory may be considered “true” in the range over which it was qualitatively tested, but new conditions may unexpectedly reveal behavior that will lead to a broader and more complete picture. Our scientific theories are inherently preliminary, but not to the extent that we cannot rely on their accuracy. And this is the problem with well-established theories. Since we can never know for sure that our experimental results are "real", how do we know that we are simply not passing off the desired answer as valid?
Measurements of the speed of light in different years
This kind of thinking appears in elementary students. They are tasked with measuring some experimental values like the acceleration of gravity or the wavelength of a laser. As newbies, they often make the simplest mistakes and get results that don't match the "generally accepted" meaning. When this happens, they go back and look for mistakes in their work. But if they make mistakes in such a way that they balance out or turn out to be non-obvious, they won't double-check their work. Since their result is close to the expected value, they think they did everything right. This prejudice is shared by all of us, and sometimes by distinguished scientists. Historically, this has happened both with the speed of light and with the charge of an electron.
Promotional video:
Currently, there is a model in cosmology that agrees well with observations. This is the ΛCDM model, whose name is made up of the Greek letter "lambda" and cold dark matter (CDM). Most of the refinements of this model include making more accurate measurements of the parameters of this model, such as the age of the universe, the Hubble parameter and the density of dark matter. If the lambda-CDM model truly accurately describes the universe, then an unbiased measurement of these parameters must follow a statistical pattern. By studying the historical values of these parameters, we can measure how biased the measurements were.
To understand how this works, imagine a dozen students measuring the length of a chalk board. Statistically, some students get a value that is greater or less than the present. According to the usual distribution, if the real length of the board is 183 centimeters with a standard deviation per centimeter, then eight students will receive a result in the range of 182-184 centimeters. But imagine that all students are within this range. In this case, you have the right to suspect some measurement errors. For example, the students heard that the board was about eighty two and a half meters, so they took measurements, rounding the result to 183. Paradoxically, if their experimental results were too good, one suspects the initial bias in the experiment.
In cosmology, various parameters are well known. Therefore, when a group of scientists conducts a new experiment, they already know which result is generally accepted. It turns out that the results of the experiments are "infected" with the previous results? One of the most recent papers by the Quarterly Physics Review addresses this very issue. By studying 637 measurements of 12 different cosmological parameters, they figured out how the results were statistically distributed. Since the "real" values of these parameters are unknown, the authors used the WMAP 7 results as "true". And they found out that the distribution of the results was more accurate than it should have been. The effect is small, so it could be attributed to biased expectation, but it was also very different from the expected effect, which may indicate an overestimation of experimental uncertainties.
This does not mean that our current cosmological model is wrong, but it does mean that we need to be a little more careful about our confidence in the accuracy of our cosmological parameters. Fortunately, there are ways to improve measurement accuracy. Cosmologists are not fooling themselves and us, there is simply still a lot of room to improve and correct the data, methods and analyzes they use.