Tests of a climate forecast

Thanks to a comment by Rachel, I’ve been made aware of a recent paper by Myles Allen called Tests of a decadal climate forecast. The paper is actually a short comment to the Editor and presents a comparison of the HadCM2 climate model predictions for the last decade (2001 to 2010), with observed global surface temperature anomalies.

One of the main focus points for those who claim that there is no global warming or that the science isn’t settled, is that the climate models have failed. This is mainly based on comparing the IPCC’s CMIP5 ensemble with observations and is also mainly based on a leaked draft of the next IPCC report. So what do Myles Allen & Co. actually do in this paper? Well what they seem to do is to take their climate model (HadCM2) and set it up using data only up until 1996, and then run it forward until 2012. They also use the IS92a scenario of relatively high greenhouse gas and anthropogenic sulphate forcing. They then compare what the model predicts with what is observed. They also compares the observations with what would be expected if the temperature fluctuations were simply following a random walk.

So, the main figure is shown below. The left-hand panel shows the model prediction with the black line being the ensemble mean and the grey area the 5% – 95% confidence interval. The red line is the running decadal mean of the observations, while the yellow diamonds are the annual temperatures during the forecast period. The middle panel is simply the period 1990 – 2040 shown relative to the period 1986 – 1996. The green vertical lines are the range based on a random walk, while the blue vertical lines are estimates from the IPCC’s CMIP5 model ensemble. The right hand panel simply shows the distribution of predicted temperatures (green – random walk, black – Allen et al., blue – CMIP5).

Comparison of model predictions with observations from Allen, Mitchell & Stott 2013.

Comparison of model predictions with observations from Allen, Mitchell & Stott 2013.

So, as far as I can tell, the HadCM2 model has done a remarkably good job of predicting global temperatures for the last decade using only data prior to 1996. It’s clear (from the right-hand panel in the above figure) that the CMIP5 ensemble does worse than HadCM2 alone but, according to Allen et al. (2013), still does a better job than a simple random walk. Just in case you think there is something funny going on here, I include below the figure from Allen et al. (1999) on which this current comparison is based. As far as I can see, it is essentially the same as the left-hand panel of the figure above, so they haven’t done anything to adjust the model prediction that they made in their 1999 paper.
Model prediction from Allen et al. (1999).

Model prediction from Allen et al. (1999).

So, Allen et al. (2013) seem to show – quite convincingly – that climate models have done a remarkably good job of predicting the evolution of the global surface temperatures over the last decade. I’ll finish with a quote from Allen et al. (2013) about what would need to happen in order for these model predictions to be falsified at the 10% level,

Even if temperatures for the decade 2007–2016 remain no higher than those for the decade 2002–2011, the 1999 forecast would still not be falsified at the 10% confidence level. However, it would no longer be substantially better than the random walk. If, however, temperatures have still not risen above those of the most recent decade by 2017–2026, in the absence of an explosive volcanic eruption, asteroid strike, nuclear exchange or other neglected short-term climate forcing, then the observations will fall outside the range of the dotted lines in Fig. 1b and the forecast2 will have been falsified at the 10% level.

This entry was posted in Climate change, Climate sensitivity, Global warming and tagged , , , , , , . Bookmark the permalink.

18 Responses to Tests of a climate forecast

  1. Skeptikal says:

    Okay, so you’ve found one scenario from one climate model that hasn’t strayed too far from the observations… yet.

    Can you really use that one scenario from that one computer model to say “that climate models have done a remarkably good job of predicting the evolution of the global surface temperatures over the last decade”?

    Wouldn’t it be more accurate to say that climate models in general have performed poorly, but here is the one exception?

  2. Possibly, but it seems as though there is at least one model with predictions made in 1999 that is still consistent with the observations. I would imagine (although I can’t say for sure) that there is probably more than one given that the IPCC model is a multi-model ensemble. Also – strictly speaking – the observations are still within the range of the multi-model ensemble so, I would argue, that even this can’t be rejected yet.

  3. I should add that we’re considering here only the climate model’s effectiveness at predicting global surface temperatures. We’re not considering how well they model the TOA energy imbalance or the evolution of ocean heat content. I believe they do quite well, but have yet to have this confirmed.

  4. Very informative post. I’ve decided to promote your blog from my home page. Keep up the great work!

  5. Thanks. Trying to do my best, but it is trying at times 🙂

  6. Pingback: Global warming predictions spot-on | quakerattled

  7. BBD says:


    Do you recall Hansen’s comments on how he was *sceptical* about models? Do you remember what he said?

    He said that we know that the climate is sensitive to radiative perturbation because of the way it has behaved in the past. Models are just tools for exploring the fine detail, not the basis on which we *know* that dF = dT.

    Do you really grasp this? I begin to wonder.

  8. BBD says:

    Think about what separates the Holocene from the LGM. Think about it.

    Roughly 6W/m^2.

    Think about it.

  9. Pingback: Watt about Chip and Paddy? | Wotts Up With That Blog

  10. Skeptikal says:

    Models are just tools for exploring the fine detail

    The performance of most climate models would suggest that they are tools that don’t work for their intended purpose… they only serve to make astrology look respectable.

  11. Skeptikal says:

    but it is trying at times

    Would that be in any way directed toward my contributions to the discussions on your site?

  12. BBD says:

    I suggested that you *thought* about something. I repeat that suggestion, with a little more emphasis.

    Your “scepticsm” is getting a little bit tiresome.

  13. Don’t worry so much about what people say in the comments. Just keep on an even keel and keep doing what you’re doing.

    The climate change deniers like to attack the models (which get the trend right but often underestimate the rate of change) as a means to attempt to invalidate all of climate science. This is nonsense, malarkey, and tom foolery at its worst. The models are valuable tools showing that we understand the climate system enough to determine trend. They are also valuable, and rather terrifying, in that they often show that Earth systems are more sensitive to global warming forcing than we originally assumed.

    As such, they establish a base-line for prediction and comparison with events. So they are a very valuable part of the science even when they aren’t perfectly accurate.

    To this point, the models absolutely have, as Rachel and you demonstrate, gotten the big variable — rate of temperature change — right. And that’s a pretty big deal because it is the primary driver of all other climate impacts.

    Skeptikal is moronic to denigrate this major contribution to the science and to compare this major achievement of atmospheric prediction to ‘astrology.’ His buffoonery here doesn’t add much to the discourse and his observations are in direct opposition to a massive and established body of evidence. My opinion is that he’s an agitator from one of these climate change denial sites and that he will continue to do everything he can to troll these forums and demoralize you. So you have to think about this:

    Does his contribution here have any merit?

    It may, as an abject display of ignorance typical to the climate change denial movement. You may well want to display his clowning and foolishness for all rational people to see. But to provide any justification for his nonsense to have equal weight with actual climate science would be harmful.

    Perhaps you believe I have been too vicious? So be it. I’m tired of coddling fools.

  14. Skeptical, no I wasn’t aiming that at you are at anyone in particular. I quite enjoy engaging in these discussions.

    I also don’t have any moderation policy so have allowed all comments made to stand. Robertscribbler’s comment is quite strongly worded. I will acknowledge that I have some sympathy with his views, in the sense that some of what is said by those who question the standard results of climate science is sometimes so odd that it’s hard no to be insulting. There are also occasions when it is clear that something is wrong and you still can’t get someone to acknowledge that this is the case. I will say, however, that I would prefer that this site did not become a place where people simply fling insults at each other.

  15. BBD says:

    I would prefer that this site did not become a place where people simply fling insults at each other.

    Of course. There are venues elsewhere for bare-knuckle boxing 😉

  16. Hola! I’ve been following your weblog for a while now and finally got the bravery to go ahead and give you a shout out from Porter Tx! Just wanted to mention keep up the great job!

  17. Pingback: Another Week of GW News, June 30, 2013 – A Few Things Ill Considered

  18. Pingback: Another Week of GW News, June 30, 2013 [A Few Things Ill Considered] | Blog Submit

Comments are closed.