26 March 2011

Rating System Challenge: Pomeroy Slips, ESPN Users Looking Sharp

After a crazy Sweet Sixteen in which yours truly picked one—yes, one—winner correctly, there's been a bit of a shake-up in the Rating System Challenge standings. Once in first place and in the 98th percentile of ESPN Tourney Challenge Brackets, Pomeroy has tumbled to seventh and 55th.

Pomeroy switched places with the ESPN National Bracket, which clearly indicates that most ESPN users didn't rely all that heavily on the ESPN Decision Tree tool (dead last).

Just as last week, FiveThirtyEight remains tied with Pure Chalk for second place, although their ranking among ESPN users has dropped six percentiles. This tells much of the story with these ratings systems. The best among among them ranks only in the 93rd percentile, and it quickly drops off from there.

Moreover, picking the best system through the Sweet Sixteen would have won you only 10 extra points over chalk. Not very impressive.

RNK ENTRY R64 R32 S16 E8 FF NCG CHAMPION PPR TOTAL PCT
1 ESPN National Bracket 230 200 160 0 0 0 Ohio St 240 580 93.2
2 FiveThirtyEight 250 200 120 0 0 0 Ohio St 240 570 90.6
2 Pure Chalk 250 200 120 0 0 0 Ohio St 240 570 90.6
4 Vegas 210 200 120 0 0 0 Ohio St 240 530 72.5
5 Nolan Power Index 230 200 80 0 0 0 Kansas 560 510 61.1
5 Sonny Moore 230 200 80 0 0 0 Ohio St 240 510 61.1
7 Pomeroy 260 200 40 0 0 0 Ohio St 240 500 55.0
8 Lunardi RPI 220 180 80 0 0 0 Kansas 560 480 42.7
8 NCAA Official RPI 220 180 80 0 0 0 Kansas 560 480 42.7
10 Jeff Sagarin 230 200 40 0 0 0 Ohio St 240 470 36.9
11 Preseason AP Poll 240 140 80 0 0 0 Duke 80 460 31.4
12 LRMC Bayesian 250 160 40 0 0 0 Ohio St 240 450 26.4
13 ESPN Decision Tree 230 160 40 0 0 0 Kansas 560 430 17.8

More analysis after the jump.

We should expect the Nolan Power Index make a run for first place in the final three rounds, as they're the only system in the top 50% of users that has their national champion pick intact. With three of the four #1 seeds out of the tournament, late-round chalkiness has resulted in a Final Four disintegration for our systems.

Final Four Picks
East
West
Southwest
Southeast
Vegas
OSU (1)
Duke (1)
Kansas (1)
Pitt (1)
ESPN National Bracket
OSU (1)
Duke (1)
Kansas (1)
Pitt (1)
Sonny Moore
OSU (1)
Duke (1)
Kansas (1)
Pitt (1)
FiveThirtyEight
OSU (1)
Duke (1)
Kansas (1)
Pitt (1)
Jeff Sagarin
OSU (1)
Duke (1)
Kansas (1)
Pitt (1)
Pomeroy
OSU (1)
Duke (1)
Kansas (1)
Pitt (1)
NCAA
OSU (1)
SDSU (2)
Kansas (1)
BYU (3)
ESPN Decision Tree
OSU (1)
Duke (1)
Kansas (1)
BYU (3)
Lunardi RPI
OSU (1)
Texas (4)
Kansas (1)
BYU (3)
Nolan Power Index
OSU (1)
SDSU (2)
Kansas (1)
BYU (3)
LRMC Bayesian
OSU (1)
Duke (1)
Kansas (1)
Belmont (13)

Image credit: Wikipedia

No comments: