NOAA Logo

NWS Logo

Organizations

Space Weather Prediction Center

National Oceanic and Atmospheric Administration

Thursday, November 21, 2024 21:19:04

Main menu

NOAA Scales mini

minimize icon
Space Weather Conditions
24-Hour Observed Maximums
R
no data
S
no data
G
no data
Latest Observed
R
no data
S
no data
G
no data
R1-R2 --
R3-R5 --
S1 or greater --
G
no data
R1-R2 --
R3-R5 --
S1 or greater --
G
no data
R1-R2 --
R3-R5 --
S1 or greater --
G
no data
maximize icon
R
no data
S
no data
G
no data
Current Space Weather Conditions
R1 (Minor) Radio Blackout Impacts
close
HF Radio: Weak or minor degradation of HF radio communication on sunlit side, occasional loss of radio contact.
Navigation: Low-frequency navigation signals degraded for brief intervals.
More about the NOAA Space Weather Scales

ROC (Receiver Operating Characteristic) curves

A ROC curve is a plot of the false alarm rate (also known as probability of false detection or POFD) on the x-axis, versus the hit-rate (also known as probability of detection-yes or PODy) on the y-axis.  
False Alarm Rate and Hit Rate are defined as:

Hit Rate (PODy) = a/(a+c)
False Alarm Rate (POFD) = b/(b+d)

based on this contingency table:

You can read more about creating ROC curves in this presentation, slides 50-53.  Finally, a thorough treatment is presented in a Weather and Forecasting article.

Here's an example:

So, notice that you want your curve for whatever forecast you're making to be above the diagonal, otherwise, you have no skill.  If it's under the diagonal, your forecast is worse than a random guess!  The closer your curve is to the upper left corner, the better your forecast.

If the number of events  (e.g. M-flares) is low, this interpretation will be a bit more tenuous, so the number of events with each plot are included.  

The effort to develop ROC curves was spawned by conversations during the ISES Verification Workshop held a couple of years ago and they were recently highlighted in a presentation at this year's European Space Weather Workshop about a flare forecast scoreboard being established at CCMC to mirror their CME prediction scoreboard.

Notice the AUC (area under the curve) is mentioned as a potentially useful measure.

 

ROC (receiver operating characteristic) curves for SWPC flare forecasts (1996 to the present)

(blank plots indicate no data was available)

ROC Curve Plots   Statistical Data Files  
1996   test_stats_1996  
1997   test_stats_1997  
1998   test_stats_1998  
1999   test_stats_1999  
2000   test_stats_2000  
2001   test_stats_2001  
2002   test_stats_2002  
2003   test_stats_2003  
2004   test_stats_2004  
2005   test_stats_2005  
2006   test_stats_2006  
2007   test_stats_2007  
2008   test_stats_2008  
2009   test_stats_2009  
2010   test_stats_2010  
2011   test_stats_2011  
2012   test_stats_2012  
2013   test_stats_2013  
2014   test_stats_2014  
2015   test_stats_2015  
2016   test_stats_2016