23 March 2012

Rating Systems Challenge 2012: Preseason Polls Got It Right the First Time

7 Florida is headed to the Elite 8
Image credit: Wikipedia
Click here for the most up-to-date March Madness 2012 coverage and Rating Systems Challenge results from RPBlog.

After fifty-two NCAA Tournament games (not counting the play-in round), we're right back where we started in November. Halfway through the Sweet 16, the two best performing systems—and the only two brackets that have outperformed chalk—are the AP and ESPN/USA Today preseason polls.

This may surprise some of you, but in fact this is not uncommon. As Nate Silver finds, teams that outperform their preseason rankings tend to underperform in the NCAA Tournament. Why? Preseason rankings lack the recency bias that plagues in-season polls (as well as algorithms in some rating systems).

And so it is that the preseason polls are the only two systems to surpass Pure Chalk in terms of ESPN Tourney Challenge points and percentile, in terms of their share of correct picks, and in possible points remaining. And with Michigan State's defeat at the hands of Louisville, the ESPN/USA Today preseason poll is one of two remaining systems with their Final Fours intact.

Rank System South East West Midwest Correct PPR
1 ESPN/USA Today Preseason UK OSU L'Ville UNC 71.2% 1080
2 AP Preseason UK Cuse Mizzou UNC 69.2% 1080
3 Pure Chalk UK Cuse MSU UNC 67.3% 1000
3 AP Postseason UK OSU Florida UNC 67.3% 1000
5 Jeff Sagarin UK OSU MSU UNC 67.3% 1000
6 ESPN Decision Tree UK OSU Mizzou UNC 65.4% 1000
6 Vegas UK Cuse MSU UNC 65.4% 1000
8 Sonny Moore UK OSU MSU UNC 67.3% 840
9 ESPN CBPI UK Cuse MSU UNC 65.4% 1000
9 ESPN National Bracket UK OSU Mizzou UNC 65.4% 1000
9 ESPN/USA Today Postseason UK Cuse Mizzou UNC 65.4% 1000
12 Lunardi RPI UK Cuse MSU UNC 61.5% 1040
13 FiveThirtyEight UK OSU Mizzou UNC 63.5% 1000
14 ESPN InsideRPI UK Cuse MSU UNC 59.6% 1000
14 LRMC Bayesian UK OSU MSU KU 59.6% 1000
16 Pomeroy UK OSU MSU KU 61.5% 1040
17 ESPN Simulation Model UK OSU MSU KU 61.5% 1000
18 Nolan Power Index UK Cuse Murray St UNC 55.8% 1000


Not to pile on, but most systems came up short on last night's contests. The preseason polls were the only systems that went 4/4 in the first day of the Sweet 16.

One other system has tied Pure Chalk, but that system is another poll: the AP postseason poll. Two other systems have equaled chalk in the number of correct picks: Jeff Sagarin and Sonny Moore. Their smart picks were in earlier rounds, however, and therefore not as valuable in the Tournament Challenge format.

Rank System R64 R32 S16 E8 FF NCG Champion PPR Total Pct Correct
1 ESPN/USA Today Preseason 230 200 160 0 0 0 UNC 1080 590 99.7 71.2%
2 AP Preseason 210 220 160 0 0 0 UNC 1080 590 99.7 69.2%
3 Pure Chalk 220 220 80 0 0 0 Kentucky 1000 520 93.2 67.3%
3 AP Postseason 220 220 80 0 0 0 Kentucky 1000 520 93.2 67.3%
5 Jeff Sagarin 230 200 80 0 0 0 Kentucky 1000 510 89.3 67.3%
6 ESPN Decision Tree 210 220 80 0 0 0 Kentucky 1000 510 89.3 65.4%
6 Vegas 210 220 80 0 0 0 Kentucky 1000 510 89.3 65.4%
8 Sonny Moore 230 200 80 0 0 0 Ohio St 840 510 89.3 67.3%
9 ESPN CBPI 220 200 80 0 0 0 Kentucky 1000 500 85.7 65.4%
9 ESPN National Bracket 220 200 80 0 0 0 Kentucky 1000 500 85.7 65.4%
9 ESPN/USA Today Postseason 220 200 80 0 0 0 Kentucky 1000 500 85.7 65.4%
12 Lunardi RPI 200 200 80 0 0 0 UNC 1040 480 76.5 61.5%
13 FiveThirtyEight 220 180 80 0 0 0 Kentucky 1000 480 76.5 63.5%
14 ESPN InsideRPI 200 180 80 0 0 0 Syracuse 1000 460 65.0 59.6%
14 LRMC Bayesian 200 180 80 0 0 0 Kentucky 1000 460 65.0 59.6%
16 Pomeroy 220 180 40 0 0 0 Kentucky 1040 440 52.2 61.5%
17 ESPN Simulation Model 220 180 40 0 0 0 Kentucky 1000 440 52.2 61.5%
18 Nolan Power Index 190 160 80 0 0 0 Kentucky 1000 430 45.7 55.8%

*Rank sorted by total points (Total), possible points remaining (PPR) and percentage of correct picks (Correct). "Pct" denotes where the brackets rank among the rest in the ESPN Tournament Challenge.

Tonight presents a new opportunity for major upsets and bracket carnage. Three double-digit seeds—Xavier (10), NC State (11) and Ohio (13)—will fight for a spot among the Elite. Meanwhile, one-seeds Kentucky and UNC, as well as two-seed Kansas, appear in the Final Four on at least one system-driven bracket. If any one of these powerhouses fall, check in tomorrow to see how it impacts the 2012 Rating Systems Challenge.

And if not, check in anyway—this is prime time for major shifts in everyone's NCAA Tournament pool.

No comments:

Post a Comment

Please Be Civil.