Verification of ACCESS-S1

These are working level pages generated by the Coupled Modeling and Climate Processes teams of the BoM Science to Services Program.

The purpose of these pages are to share verification results within the Bureau and with research partners.

These pages and their contents are NOT to be distributed without permission. If you have any comments or queries, please send an email to POAMA

Background

The ACCESS-S1 evaluation is based on the hindcast set:


NOTE: most of the assessment shown is based on forecasts starting on the 1st of the month. In some of the monthly and seasonal forecast verification, a 22-member lagged ensemble has been used (combining the 11 members of the 1st with the 11 members from the 25th of the month prior).

Where comparisons are made with POAMA, the evaluation is done over the common hindcast years (1990-2012), using the closest matching start dates (i.e. the 1st and 26th from POAMA) and only 11-members per start date from POAMA.

Is 23 years and 11 ensemble members enough for a robust estimation of regional skill?
For some pointers on the impact of sampling on skill calculations (and other key considerations), see here.

For details about the ACCESS-S forecast system, see:

Hudson, D., Alves, O., Hendon, H.H., Lim, E., Liu, G., Luo J.-J., MacLachlan, C., Marshall, A.G., Shi, L., Wang, G., Wedd, R., Young, G., Zhao, M., Zhou X., 2017: ACCESS-S1: The new Bureau of Meteorology multi-week to seasonal prediction system. Journal of Southern Hemisphere Earth Systems Science, 67:3 132-159 doi: 10.22499/3.6703.001.

Mean State Bias (Systematic Errors)

SST MSLP U200 Rainfall

Climate Drivers

ENSOIODSAMMJO
Stddev Stddev    
Skill Skill Monthly/seas skill               Daily skill Skill

Australian Climate

Verification on model grid (~60km)

Tmax, Tmin & Rainfall

Climatologies
Climate Mean Ensemble standard deviation

Overall skill summary (all seasons combined)
Maps Regions

Skill for different seasons and lead times
RMS Error Correlation Accuracy above Median
Generalized Discrimination Score Reliability and ROC curves  

Verification on 5km grid (work in progress)

Tmax, Tmin, Rainfall, SW Radiation, Wind Speed, Evaporation, Vapour pressure

Climatologies
Climate Mean Ensemble standard deviation Percentile thresholds

Overall skill summary (all seasons combined)
Maps and Regions

Skill for different seasons and lead times
RMS Error Correlation Accuracy above Median
Generalized Discrimination Score Ranked Probability Skill Score Area averaged skill

Teleconnections to Australian rainfall

AUS rainfall
NINO3 EMI IOD EIO WIO SAM

Global Climate

Skill for Rainfall, Tmax and Tmin. Verified against ERA-Interim and CRU TS respectively.

Correlation Skill Accuracy above Median

Global Teleconnections

SSTRainfallMSLPZ200
NINO3 NINO3 NINO3 NINO3
  EMI EMI EMI
IOD IOD IOD IOD
EIO EIO EIO EIO
WIO WIO WIO WIO
  SAM SAM SAM