Rating Systems Challenge 2012: Chalk Prevails through Round of 32

13 Ohio Bobcats logo.
Schools from Ohio are 8-0 so far.
Image credit: Wikipedia

Click here for the most up-to-date March Madness 2012 coverage and Rating Systems Challenge results from RPBlog.

Once again, chalk rules despite the prevalence of upsets in the 2012 NCAA Tournament (the AP Postseason bracket is tied with Pure Chalk in terms of total points, but trails in possible points remaining). It's not that the other systems didn't pick any upsets, it's that they picked the wrong upsets.

Turns out, it's very hard to pick the big upsets. A few systems will get the 8-9 and 7-10 match-ups right. And as we saw a couple days ago, quite a few of the brackets successfully picked NC State, an 11 seed now headed to the Sweet Sixteen.

But only one system (Nolan Power Index) had 13 Ohio beating 4 Michigan, and that system is still dead last. No systems predicted the success of 15 Lehigh, 13 Norfolk State or 12 USF.

Moreover, most of the upsets that the least chalky brackets picked didn't come through. Nolan Power and AP Preseason put 14 Belmont in the Sweet Sixteen. Nolan Power also picked 12 Harvard to upset 5 Vanderbilt and 13 Davidson to oust 4 Louisville while sending Murray State to the Final Four. ESPN/USA Today's Preseason Poll put 12 VCU in the Sweet Sixteen.

Those were the big upset picks of our three least chalky systems, and none of these predictions came true (although ESPN/USA Today's Preseason Poll almost hit paydirt with VCU). Only the AP Preseason poll hasn't suffered for it, so far.

Rank	System	R64	R32	S16	E8	FF	NCG	Champion	PPR	Total	Pct
1	Pure Chalk	220	220	0	0	0	0	Kentucky	1200	440	92.4
2	AP Postseason	220	220	0	0	0	0	Kentucky	1120	440	92.4
3	AP Preseason	210	220	0	0	0	0	UNC	1240	430	86.1
3	ESPN/USA Today Preseason	230	200	0	0	0	0	UNC	1240	430	86.1
5	Jeff Sagarin	230	200	0	0	0	0	Kentucky	1200	430	86.1
5	Sonny Moore	230	200	0	0	0	0	Ohio St	1200	430	86.1
5	Vegas	210	220	0	0	0	0	Kentucky	1200	430	86.1
8	ESPN Decision Tree	210	220	0	0	0	0	Kentucky	1120	430	86.1
9	ESPN CBPI	220	200	0	0	0	0	Kentucky	1200	420	79.0
9	ESPN National Bracket	220	200	0	0	0	0	Kentucky	1120	420	79.0
11	ESPN/USA Today Postseason	220	200	0	0	0	0	Kentucky	1120	420	79.0
11	Lunardi RPI	200	200	0	0	0	0	UNC	1240	400	61.0
13	Pomeroy	220	180	0	0	0	0	Kentucky	1240	400	61.0
14	ESPN Simulation Model	220	180	0	0	0	0	Kentucky	1200	400	61.0
15	FiveThirtyEight	220	180	0	0	0	0	Kentucky	1120	400	61.0
16	ESPN InsideRPI	200	180	0	0	0	0	Syracuse	1240	380	41.8
17	LRMC Bayesian	200	180	0	0	0	0	Kentucky	1200	380	41.8
18	Nolan Power Index	190	160	0	0	0	0	Kentucky	1120	350	19.5

*Rank sorted by total points (Total) and possible points remaining (PPR). "Pct" denotes where the brackets rank among the rest in the ESPN Tournament Challenge.

Considering that, last year, the only system to outperform chalk was the ESPN National Bracket, the evidence seems to indicate that bracket entrants shouldn't rely on the advanced systems alone to pick their upsets for them.