03 April 2012

Rating Systems Challenge 2012: Conclusion

It was official on Sunday--the Survival Model was the best in the 2012 NCAA Tournament--but Kentucky's national championship puts the icing on the cake. While nearly every system performed better than last year (see below), only four systems had more correct picks than chalk.
 
Model 2011 2012 ∆%
ESPN Decision Tree 32 41 9 28%
AP Preseason Poll 33 42 9 27%
Jeff Sagarin 34 43 9 26%
Vegas 34 40 6 18%
Lunardi RPI 33 38 5 15%
LRMC Bayesian 34 39 5 15%
Sonny Moore 35 40 5 14%
ESPN National Bracket 36 41 5 14%
Pomeroy 37 41 4 11%
RPI* 33 36 3 9%
Pure Chalk 38 41 3 8%
FiveThirtyEight 38 40 2 5%
Nolan Power 35 35 0 0%





*NCAA RPI in 2011, ESPN InsideRPI in 2012.
Results should be identical.

Considering that there weren't as many Final Four surprises this year as last (where no system picked any finalist correctly), twelve of the thirteen systems that competed in both Rating System Challenges improved on their 2011 records. The one exception is the Nolan Power Index, which performed as poorly this year as last. This is the second year in a row in which Nolan Power ranked last in correct picks.

Thanks again for following along. Check out the final rankings after the jump.

Click here to review RPBlog's March Madness 2012 coverage and Rating Systems Challenge results.

02 April 2012

Rating Systems Challenge 2012: Survival Model Locks It Up

Image credit: Wikipedia
Click here for the most up-to-date March Madness 2012 coverage and Rating Systems Challenge results from RPBlog.

The ESPN Simulation Model, which had been performing dismally throughout the tournament, shot up into second place due to its strong performance in the Final Four. This model was the only system among the nineteen that picked both finalists correctly. Every other system, save Sonny Moore, picked 1 Kentucky but not 2 Kansas.

This exceptional rise was not enough to unseat the Survival Model. With Kentucky's advance to the final game, the Survival Model maintained pole position. And it will keep its lead through the finish, as no system has Kansas winning the national championship. As a result, the only possible change in standings is whether the brackets that picked the Wildcats will surpass the brackets that picked UNC, Syracuse or Ohio State.

And so, the final results are just a formality, but be sure to tune in for some final commentary and comparisons with last year's Rating Systems Challenge results.

Rank System R64 R32 S16 E8 FF NCG Champion PPR Total Pct Correct
1 Survival Model 250 200 280 240 160 0 Kentucky 320 1130 96.9 74.2%
2 ESPN Simulation Model 220 180 160 240 320 0 Kentucky 320 1120 96.5 64.5%
3 ESPN/USA Today Preseason 230 200 280 240 160 0 UNC 0 1110 96.0 71.0%
4 Jeff Sagarin 230 200 200 240 160 0 Kentucky 320 1030 91.3 67.7%
5 AP Preseason 210 220 280 160 160 0 UNC 0 1030 91.3 67.7%
6 Pomeroy 220 180 200 240 160 0 Kentucky 320 1000 88.7 64.5%
7 LRMC Bayesian 200 180 200 240 160 0 Kentucky 320 980 86.6 61.3%
8 ESPN Decision Tree 210 220 200 160 160 0 Kentucky 320 950 82.7 64.5%
9 ESPN National Bracket 220 200 200 160 160 0 Kentucky 320 940 81.2 64.5%
10 FiveThirtyEight 220 180 200 160 160 0 Kentucky 320 920 77.8 62.9%
11 Pure Chalk 220 220 200 80 160 0 Kentucky 320 880 70.3 64.5%
11 AP Postseason 220 220 200 80 160 0 Kentucky 320 880 70.3 64.5%
13 Lunardi RPI 200 200 240 80 160 0 UNC 0 880 70.3 61.3%
14 Vegas 210 220 200 80 160 0 Kentucky 320 870 67.5 62.9%
15 ESPN CBPI 220 200 200 80 160 0 Kentucky 320 860 65.4 62.9%
15 ESPN/USA Today Postseason 220 200 200 80 160 0 Kentucky 320 860 65.4 62.9%
17 ESPN InsideRPI 200 180 200 80 160 0 Syracuse 0 820 57.0 58.1%
18 Nolan Power Index 190 160 200 80 160 0 Kentucky 320 790 50.7 54.8%
19 Sonny Moore 230 200 200 160 0 0 Ohio St 0 790 50.7 64.5%

*Rank sorted by total points (Total), possible points remaining (PPR) and percentage of correct picks (Correct). "Pct" denotes where the brackets rank among the rest in the ESPN Tournament Challenge.


Rank System South East West Midwest Correct PPR
1 Survival Model UK OSU MSU KU 74.2% 320
2 ESPN Simulation Model UK OSU MSU KU 64.5% 320
3 ESPN/USA Today Preseason UK OSU L'Ville UNC 71.0% 0
4 Jeff Sagarin UK OSU MSU UNC 67.7% 320
5 AP Preseason UK Cuse Mizzou KU 67.7% 0
6 Pomeroy UK OSU MSU KU 64.5% 320
7 LRMC Bayesian UK OSU MSU KU 61.3% 320
8 ESPN Decision Tree UK OSU Mizzou UNC 64.5% 320
9 ESPN National Bracket UK OSU Mizzou UNC 64.5% 320
10 FiveThirtyEight UK OSU Mizzou UNC 62.9% 320
11 Pure Chalk UK Cuse MSU UNC 64.5% 320
11 AP Postseason UK OSU Florida UNC 64.5% 320
13 Lunardi RPI UK Cuse MSU UNC 61.3% 0
14 Vegas UK Cuse MSU UNC 62.9% 320
15 ESPN CBPI UK Cuse MSU UNC 62.9% 320
15 ESPN/USA Today Postseason UK Cuse Mizzou UNC 62.9% 320
17 ESPN InsideRPI UK Cuse MSU UNC 58.1% 0
18 Nolan Power Index UK Cuse Murray St UNC 54.8% 320
19 Sonny Moore UK OSU MSU UNC 64.5% 0