03 April 2012

Rating Systems Challenge 2012: Conclusion

It was official on Sunday--the Survival Model was the best in the 2012 NCAA Tournament--but Kentucky's national championship puts the icing on the cake. While nearly every system performed better than last year (see below), only four systems had more correct picks than chalk.
 
Model 2011 2012 ∆%
ESPN Decision Tree 32 41 9 28%
AP Preseason Poll 33 42 9 27%
Jeff Sagarin 34 43 9 26%
Vegas 34 40 6 18%
Lunardi RPI 33 38 5 15%
LRMC Bayesian 34 39 5 15%
Sonny Moore 35 40 5 14%
ESPN National Bracket 36 41 5 14%
Pomeroy 37 41 4 11%
RPI* 33 36 3 9%
Pure Chalk 38 41 3 8%
FiveThirtyEight 38 40 2 5%
Nolan Power 35 35 0 0%





*NCAA RPI in 2011, ESPN InsideRPI in 2012.
Results should be identical.

Considering that there weren't as many Final Four surprises this year as last (where no system picked any finalist correctly), twelve of the thirteen systems that competed in both Rating System Challenges improved on their 2011 records. The one exception is the Nolan Power Index, which performed as poorly this year as last. This is the second year in a row in which Nolan Power ranked last in correct picks.

Thanks again for following along. Check out the final rankings after the jump.

Click here to review RPBlog's March Madness 2012 coverage and Rating Systems Challenge results.

Rank System R64 R32 S16 E8 FF NCG Champion PPR Total Pct Correct
1 Survival Model 250 200 280 240 160 320 Kentucky 0 1450 97.7 74.6%
2 ESPN Simulation Model 220 180 160 240 320 320 Kentucky 0 1440 97.4 65.1%
3 Jeff Sagarin 230 200 200 240 160 320 Kentucky 0 1350 94.0 68.3%
4 Pomeroy 220 180 200 240 160 320 Kentucky 0 1320 92.4 65.1%
5 LRMC Bayesian 200 180 200 240 160 320 Kentucky 0 1300 91.2 61.9%
6 ESPN Decision Tree 210 220 200 160 160 320 Kentucky 0 1270 88.9 65.1%
7 ESPN National Bracket 220 200 200 160 160 320 Kentucky 0 1260 88.0 65.1%
8 FiveThirtyEight 220 180 200 160 160 320 Kentucky 0 1240 86.0 63.5%
9 Pure Chalk 220 220 200 80 160 320 Kentucky 0 1200 81.8 65.1%
9 AP Postseason 220 220 200 80 160 320 Kentucky 0 1200 81.8 65.1%
11 Vegas 210 220 200 80 160 320 Kentucky 0 1190 80.0 63.5%
12 ESPN CBPI 220 200 200 80 160 320 Kentucky 0 1180 78.9 63.5%
12 ESPN/USA Today Postseason 220 200 200 80 160 320 Kentucky 0 1180 78.9 63.5%
14 ESPN/USA Today Preseason 230 200 280 240 160 0 UNC 0 1110 71.0 69.8%
14 Nolan Power Index 190 160 200 80 160 320 Kentucky 0 1110 71.0 55.6%
16 AP Preseason 210 220 280 160 160 0 UNC 0 1030 64.2 66.7%
17 Lunardi RPI 200 200 240 80 160 0 UNC 0 880 53.2 60.3%
18 ESPN InsideRPI 200 180 200 80 160 0 Syracuse 0 820 46.9 57.1%
19 Sonny Moore 230 200 200 160 0 0 Ohio St 0 790 43.5 63.5%

*Rank sorted by total points (Total), possible points remaining (PPR) and percentage of correct picks (Correct). "Pct" denotes where the brackets rank among the rest in the ESPN Tournament Challenge.

No comments:

Post a Comment

Please Be Civil.