Experimental Great Barrier Reef SST Anomaly Forecast Skill

To evaluate the accuracy of model forecasts and provide a measure of the skill of the model, hindcasts of SST anomalies are compared to observed SST anomalies for the same period. SST anomalies are calculated for both the model forecasts and observed values as the difference between SST values and the relevant climatology. The climatology is the monthly mean SST over the period 1982-2006, computed relative to start month and lead-time for the model, and removing this from SST values reduces the effects of any model bias (Stockdale 1997). Skill is calculated by correlating model anomalies with observed anomalies in both space and time. The correlation coefficient (r) is defined as the ratio of the covariance of the sample populations to the product of their standard deviations, with a skill value of 1.0 indicating a perfect fit between model and observed values. For more information see Spillman and Alves (2009).

The first plot shows the spatial skill of model ensemble mean SST anomaly forecasts for the target three month season January-February-March (JFM) for up to 5 months prior to the forecast season (1982-2006). Skill is reasonably high for up to 2 months prior to the forecast season. Model skill appears to be higher in the northern reaches of the GBR region than in the south, with the exception of an area of lower skill along the northern coast at lead-time of 1 month. Skill of the model decreases significantly at lead-times of 2 months in the southeast quadrant of the region. The higher skill in the northern parts compared to the southern area is likely due to the larger influence of tropical variability, principally ENSO (Spillman and Alves 2009).

The plot below shows regional skill of the model ensemble mean GBR Index, persistence and potential predictability for all lead-times for (a) forecasts starting all months, and for (b) target season JFM 1982-2006 starting at different lead-times. In both cases, the model skill exceeds that of persistence, indicating the model forecasts have useful value. As expected, the skill of model forecasts decreases with lead-time, with skill of JFM forecasts generally lower than that for all months. Model skill still exceeds 0.5 for lead-times of 0-2 months and is an improvement over persistence forecasts at greater lead-times (Spillman and Alves 2009).

Persistence is used as a minimum skill forecast to assess the predictive value of the model. A forecast of persistence simply uses current observed anomaly conditions as a predictor of future conditions e.g. for a forecast beginning on 1 March, the SST anomaly for February is used as the forecast and persisted for the duration of the forecast period. Persistence forecasts are then correlated with observed values with skill compared to that of the model. For the season January-Feburary-March, model skill exceeds that of persistence at almost all lead-times and locations (Spillman and Alves 2009).

Potential predictability is the upper level of skill that can be achieved for a model forecast given a perfect model and initial conditions (Griffies and Bryan 1997). It is calculated by using one ensemble member as a reference and calculating the skill of the mean of the remaining ensemble members in predicting it. This is repeated using different ensemble members as the reference ensemble member. Skill is never perfect as the chaotic component in the system leads to ensemble spread, which in turn limits predictability (Spillman and Alves 2009).

The ROC curve plots hit rate against false alarm rate for probabilistic forecasts of a discrete event. The area under the ROC curve is interpreted as a measure of discrimination, or signal detection, ability.

A summary of POAMA SST predictions and model performance for Summer 2008/2009 is also available.

References

• Griffies S.M. and Bryan K., 1997. A predictability study of North Atlantic multidecadal variability. Climate Dynamics 13:459-487
• Spillman C. and Alves O., 2008. Dynamical seasonal prediction of summer sea surface temperatures in the Great Barrier Reef. Coral Reefs, 28:197-206.
• Spillman C.M., Alves O., Hudson D.A. and Charles A.N., 2009. POAMA SST predictions for the Great Barrier Reef: Summer 2008/2009. CAWCR Research Letters, 2:30-34.
• Stockdale T.N., 1997. Coupled ocean-atmosphere forecasts in the presence of climate drift. Monthly Weather Review 125:809-818

Last updated: 8 October 2009

 

For further information: email the POAMA group

© Copyright Commonwealth of Australia 2008
Centre for Australian Weather and Climate Research (CAWCR)
A partnership between the Australian Bureau of Meteorology and CSIRO.
Users of these Web pages are deemed to have read and accepted the conditions
described in the copyright declaration.