There is a new post on Watts Up With That (WUWT) by Cato Boffins Patrick J. Michaels and Paul C. “Chip” Knappenberger. The post is called anti-information in climate models. The post considers the two models evaluated by the first “National Assessment” of climate change impacts in the United States in the 21st century, published by the U.S. Global Change Research Program (USGCRP) in 2000. One is the Candian Climate Model and the other the model from the Hadley Centre at the UK Met. Office.
So, they test these models by comparing them with the ten-year running means of the temperature of the lower 48 states of the USA. Now, I don’t know these models particularly well but I thought they were normally used to produce global surface temperature anomalies. I presume that they can also produce temperatures for the contiguous USA. They can’t have been silly enough to have compared global surface temperatures with temperatures in the US, can they?
Anyway, they go on to say
One standard method used to determine the utility of a model is to compare the “residuals”, or the differences between what is predicted and what is observed, to the original data. Specifically, if the variability of the residuals is less than that of the raw data, then the model has explained a portion of the behavior of the raw data and the model can continue to be tested and entertained.
A model can’t do worse than explaining nothing, right?
Not these models! The differences between their predictions and the observed temperatures were significantly greater (by a factor of two) than what one would get just applying random numbers.
Now, I found this a bit confusing for a couple of reasons. Firstly, what do they mean by the residual has to be less than the raw data? The data is typically temperature anomalies which are relative to some long term mean. The values can go from being negative, to being close to zero to being positive. That seems to suggest that the success of the model (as defined by them) depends on the year that they’re comparing with. Also, I couldn’t find data for the USA, but if one looks at a comparison of climate models with global surface temperatures, the mean is quite a good fit. This is shown in the figure below. If the observations were to be smoothed over a longer period, it seems clear that the fit would be quite impressive. I can’t quite believe that the residual that they get is typically bigger than the observed value. Surely they’re comparing the ten-year running mean of the measured values with the ten-year running mean of the model values. Anything else would be crazy.
So, what else do they say? Well they claim that the model fit is twice as bad as one would get if one simply applied a random number generator. If one simply chose the temperature anomaly randomly, surely the fit would be awful. What I assume they mean is if you randomly perturb the known observed temperature anomaly values. Well, sure. I could make that a fantastic fit if I simply perturbed it by a tiny amount. Their “random number model” – that is supposedly better than a climate model – already “knows” the values of the measured temperature anomalies. The climate models do not. That’s the point of climate models.
They then go on to say that it is twice as bad as a random number generator because the climate models were only “correct” 12.5% of the time, while a random number generator would be correct 25% of the time. This is based on an analogy in which the model is expected to predict the temperature 100 times but in which there are 4 possible choices for each temperature. A random choice would therefore be correct one time out of 4 (hence 25%). But there aren’t only 4 possible values for each temperature anomaly. Their analogy simply makes no sense at all! You can’t compare a climate model with a multiple choice test with 100 questions and 4 answers per question.
I really don’t know what else to say about this. Either the definition of the term Boffin used at the beginning of the WUWT post differs from what I thought it meant or these are two people who are very bright but who are knowingly intending to mislead the people who read their post. It’s just an absurd post that really makes no sense at all.