Rating Systems Challenge 2012: Preseason Polls Got It Right the First Time

7 Florida is headed to the Elite 8
Image credit: Wikipedia

Click here for the most up-to-date March Madness 2012 coverage and Rating Systems Challenge results from RPBlog.

After fifty-two NCAA Tournament games (not counting the play-in round), we're right back where we started in November. Halfway through the Sweet 16, the two best performing systems—and the only two brackets that have outperformed chalk—are the AP and ESPN/USA Today preseason polls.

This may surprise some of you, but in fact this is not uncommon. As Nate Silver finds, teams that outperform their preseason rankings tend to underperform in the NCAA Tournament. Why? Preseason rankings lack the recency bias that plagues in-season polls (as well as algorithms in some rating systems).

And so it is that the preseason polls are the only two systems to surpass Pure Chalk in terms of ESPN Tourney Challenge points and percentile, in terms of their share of correct picks, and in possible points remaining. And with Michigan State's defeat at the hands of Louisville, the ESPN/USA Today preseason poll is one of two remaining systems with their Final Fours intact.

Rank	System	South	East	West	Midwest	Correct	PPR
1	ESPN/USA Today Preseason	UK	OSU	L'Ville	UNC	71.2%	1080
2	AP Preseason	UK	Cuse	Mizzou	UNC	69.2%	1080
3	Pure Chalk	UK	Cuse	MSU	UNC	67.3%	1000
3	AP Postseason	UK	OSU	Florida	UNC	67.3%	1000
5	Jeff Sagarin	UK	OSU	MSU	UNC	67.3%	1000
6	ESPN Decision Tree	UK	OSU	Mizzou	UNC	65.4%	1000
6	Vegas	UK	Cuse	MSU	UNC	65.4%	1000
8	Sonny Moore	UK	OSU	MSU	UNC	67.3%	840
9	ESPN CBPI	UK	Cuse	MSU	UNC	65.4%	1000
9	ESPN National Bracket	UK	OSU	Mizzou	UNC	65.4%	1000
9	ESPN/USA Today Postseason	UK	Cuse	Mizzou	UNC	65.4%	1000
12	Lunardi RPI	UK	Cuse	MSU	UNC	61.5%	1040
13	FiveThirtyEight	UK	OSU	Mizzou	UNC	63.5%	1000
14	ESPN InsideRPI	UK	Cuse	MSU	UNC	59.6%	1000
14	LRMC Bayesian	UK	OSU	MSU	KU	59.6%	1000
16	Pomeroy	UK	OSU	MSU	KU	61.5%	1040
17	ESPN Simulation Model	UK	OSU	MSU	KU	61.5%	1000
18	Nolan Power Index	UK	Cuse	Murray St	UNC	55.8%	1000

Not to pile on, but most systems came up short on last night's contests. The preseason polls were the only systems that went 4/4 in the first day of the Sweet 16.

One other system has tied Pure Chalk, but that system is another poll: the AP postseason poll. Two other systems have equaled chalk in the number of correct picks: Jeff Sagarin and Sonny Moore. Their smart picks were in earlier rounds, however, and therefore not as valuable in the Tournament Challenge format.

Rank	System	R64	R32	S16	E8	FF	NCG	Champion	PPR	Total	Pct	Correct
1	ESPN/USA Today Preseason	230	200	160	0	0	0	UNC	1080	590	99.7	71.2%
2	AP Preseason	210	220	160	0	0	0	UNC	1080	590	99.7	69.2%
3	Pure Chalk	220	220	80	0	0	0	Kentucky	1000	520	93.2	67.3%
3	AP Postseason	220	220	80	0	0	0	Kentucky	1000	520	93.2	67.3%
5	Jeff Sagarin	230	200	80	0	0	0	Kentucky	1000	510	89.3	67.3%
6	ESPN Decision Tree	210	220	80	0	0	0	Kentucky	1000	510	89.3	65.4%
6	Vegas	210	220	80	0	0	0	Kentucky	1000	510	89.3	65.4%
8	Sonny Moore	230	200	80	0	0	0	Ohio St	840	510	89.3	67.3%
9	ESPN CBPI	220	200	80	0	0	0	Kentucky	1000	500	85.7	65.4%
9	ESPN National Bracket	220	200	80	0	0	0	Kentucky	1000	500	85.7	65.4%
9	ESPN/USA Today Postseason	220	200	80	0	0	0	Kentucky	1000	500	85.7	65.4%
12	Lunardi RPI	200	200	80	0	0	0	UNC	1040	480	76.5	61.5%
13	FiveThirtyEight	220	180	80	0	0	0	Kentucky	1000	480	76.5	63.5%
14	ESPN InsideRPI	200	180	80	0	0	0	Syracuse	1000	460	65.0	59.6%
14	LRMC Bayesian	200	180	80	0	0	0	Kentucky	1000	460	65.0	59.6%
16	Pomeroy	220	180	40	0	0	0	Kentucky	1040	440	52.2	61.5%
17	ESPN Simulation Model	220	180	40	0	0	0	Kentucky	1000	440	52.2	61.5%
18	Nolan Power Index	190	160	80	0	0	0	Kentucky	1000	430	45.7	55.8%

*Rank sorted by total points (Total), possible points remaining (PPR) and percentage of correct picks (Correct). "Pct" denotes where the brackets rank among the rest in the ESPN Tournament Challenge.

Tonight presents a new opportunity for major upsets and bracket carnage. Three double-digit seeds—Xavier (10), NC State (11) and Ohio (13)—will fight for a spot among the Elite. Meanwhile, one-seeds Kentucky and UNC, as well as two-seed Kansas, appear in the Final Four on at least one system-driven bracket. If any one of these powerhouses fall, check in tomorrow to see how it impacts the 2012 Rating Systems Challenge.

And if not, check in anyway—this is prime time for major shifts in everyone's NCAA Tournament pool.