There’s a recent post on Watts Up With That (WUWT) called Can we actually even tell if humans are affecting the climate? What if we did nothing at all?. It’s part of an essay by someone called Charlie Martin.
The post has a set of clear points about the practice of science
1. We generate a number of alternate explanations, hypotheses, that might explain the phenomenon.
2. For each hypothesis, we come up with an experiment which will prove the hypothesis wrong. That is, not one that “proves the hypothesis”, but one which, if successful, would disprove or falsify the hypothesis. (Sir Karl Popper argued in his book The Logic of Scientific Discovery that this falsification was the core of scientific knowledge.)
3. We do the experiments. If an experiment falsifies a hypothesis, we discard it ruthlessly. Then we go back to (1) and try again.
I was thinking about this a little yesterday because I came across a post called evidence, absence and the Type II monster on a blog called In The Dark. I found this quite an interesting post, but should acknowledge that I’m not an expert at this type of statistical thinking.
The basic points made in the WUWT post (shown above) are probably, strictly speaking, correct but rather overstate the case. There are known errors associated with testing the null hypothesis. A type I error is one in which you reject the null even though it is correct (false positive). A type II error is when the null hypothesis is not rejected even though it’s false (false negative). So simply testing the null hypothesis and getting a result does not immediately indicate that one has proven or disproven a hypothesis. One needs to check on the chance of false positives or false negatives. Furthermore, I think point 3 above rather oversimplifies the actual process. One could run an experiment that turns out to be marginally consistent with the null hypothesis. Therefore, the null hypothesis cannot be rejected. This doesn’t immediately mean that the original hypothesis is false. It could simply be that more data is needed so as to improve the statistical confidence. So the data being consistent with the null hypothesis may simply mean that one cannot make any strong statements about the original hypothesis. Of course, if the data is highly consistent with the null and the chance of a false positive is very low, then one may well conclude that the original hypothesis is false.
So, this essay by Charlie Martin goes on to consider climate change. To paraphrase, he suggests that the hypothesis is that humans are emitting CO2 into the atmosphere and that this is leading to global warming. Let me simplify that to the planet is currently undergoing global warming. His essay then includes the following figure.
He then goes on to say that the observed temperature is outside the 95% confidence interval and hence (I assume) that this is consistent with the null hypothesis (no warming) and therefore it’s time for some new hypotheses.
This is, in my opinion, a great example of why one has to careful about how to apply the null hypothesis. As pointed out in the In The Dark blogpost, the null hypothesis has to be well-defined in terms of the model. The hypothesis that is being tested is not that global surface temperatures will rise. The hypothesis being tested is, the Earth is undergoing global warming. Global warming is about energy, not simply surface temperatures. To test the null hypothesis one has to consider the energy in the climate system, not simply the global surface temperature anomaly. If one were to do this properly one would consider the measured top-of-the atmosphere energy imbalance, the ocean heat content, the various temperature anomaly datasets (land, sea surface, troposphere), the evolution of arctic sea ice. The null hypothesis would be that, on average, the energy in the climate system has not increased. Well, you only need to look at the various datasets for it to be clear that the data is not consistent with the null hypothesis and hence we can conclude that global warming is indeed taking place (I appreciate that this isn’t really a proper statistical test, but it should be obvious. Also, strictly speaking, one would reject the null in favour of the alternative, rather than simply accepting the alternative).
I appreciate that above I haven’t been considering the full hypothesis (global warming is happening and is anthropogenic) but that’s because that just adds an extra level of complexity. It could be tested in the same way but one would need a null hypothesis that considered how the energy in the climate system would evolve if there was no anthropogenic influence and then compare this with the actual data. It must be possible to do this, but simply comparing global surface temperature anomalies with model predictions is certainly not the correct way to do so.