15 March 2011

Rating Systems and March Madness Brackets, Part II

This series continues in Part III.

Yesterday, we initiated our coverage of March Madness by looking at three different ratings systems and determining their chalkiness when used as the sole determinant for bracket choices. Yes, this is an incredibly boring way to fill out your bracket, but it's a useful way to cross-compare before the tournament begins (and I mean the real tournament, not including play-in games).

The method revealed some interesting results, like the fact that Pomeroy was even chalkier than the official NCAA Official RPI, which seems a bit counter-intuitive.

Today, we added more ratings systems for a broader picture. We added two more "power" formulae--Sonny Moore and Jeff Sagarin--to complement Pomeroy. Incidentally, these are three systems that Nate Silver's FiveThirtyEight bracket uses, in addition to LRMC Bayesian (he also includes preseason rankings, which are surprisingly accurate, and tweaks the bracket for personnel changes).

Finally, we added the ESPN Decision Tree, which is a tool that ESPN Insiders can use to pick their Tournament Challenge brackets. Results are presented below. Lower numbers equal more chalk


Upset Index (Seed Differential * [2 ^ Round / 2])







Field 64 Field 32 Sweet 16 Elite 8 Overall
Sonny Moore 8 0 4 0 12
FiveThirtyEight 14 0 4 0 18
Jeff Sagarin 14 0 8 0 22
Pomeroy 25 0 12 0 37
NCAA* 9 16 4 24 53
ESPN Decision Tree 27 22 12 16 77
Lunardi RPI 25 8 40 8 81
LRMC Bayesian 34 12 72 80 198
Mean 20 7 20 16 62


*NCAA Official RPI does not account for Sunday games.

As you can see, the three power formulae (and the FiveThirtyEight system that is heavily dependent on them) are the chalkiest. The NCAA Official RPI formula is pretty much in the middle. The ESPN Decision Tree's advice is a tad aggressive and the LRMC Bayesian approach really loves upsets.

None of the power rankings nor FiveThirtyEight predict any upsets in the field of thirty-two or in the Elite 8. None of the systems see any seeding upsets in the Final Four or National Championship game. And while LRMC's results seem to come straight out of left field, you'll note below the source of the lion's share of their Upset Index.

The following table plots each system's Final Four picks, with finalists in bold and champions in bold and blue.

Final Four Picks







East West Southwest Southeast
Sonny Moore OSU (1) Duke (1) Kansas (1) Pitt (1)
FiveThirtyEight OSU (1) Duke (1) Kansas (1) Pitt (1)
Jeff Sagarin OSU (1) Duke (1) Kansas (1) Pitt (1)
Pomeroy OSU (1) Duke (1) Kansas (1) Pitt (1)
NCAA OSU (1) SDSU (2) Kansas (1) BYU (3)
ESPN Decision Tree OSU (1) Duke (1) Kansas (1) BYU (3)
Lunardi RPI OSU (1) Texas (4) Kansas (1) BYU (3)
LRMC Bayesian OSU (1) Duke (1) Kansas (1) Belmont (13)

Yes, that's Belmont that the LRMC Bayesian formula has in the Final Four (after beating Wisconsin, Utah State, Pittsburgh and Brigham Young).

Beyond LRMC, however, most of the systems are rather conservative when it comes to the finals and semifinals. All four power and power-based systems send all #1 seeds to the Final Four. Only Lunardi and the NCAA official formula pick fewer than two #1s. Even LRMC Bayesian is pure chalk outside of the Southeast region.

None of the systems pick any team to play in the National Championship game other than Kansas or Ohio State, and only two of them pick anyone but OSU to win it all.

Who knows, maybe chalk is the best bracket strategy this year. We'll see going forward, as I'll be playing all of these brackets in an ESPN Tournament Challenge group so that we can follow their successes and failures. If you'd like to take a closer look at each system's upset picks, continue reading.

FiveThirtyEight: 18




Field 64 Field 32 Sweet 16 Elite 8
Villanova (9) None Purdue (3) None
George Mason (8)
Notre Dame (2)




Tennessee (9)


Michigan (8)






Illinois (9)


UNLV (8)






Florida St (10)


Texas A&M (7)






Gonzaga (11)


St. John's (6)






Michigan St (10)


UCLA (7)










Jeff Sagarin: 22




Field 64 Field 32 Sweet 16 Elite 8
Villanova (9) None Syracuse (3) None
George Mason (8)
UNC (2)




Marquette (11)
BYU (3)
Xavier (6)
Florida (2)




Old Dominion (9)


Butler (8)






Utah St (12)


Kansas St (5)










LRMC Bayesian: 198




Field 64 Field 32 Sweet 16 Elite 8
Marquette (11) Washington (7) Washington (7) Belmont (13)
Xavier (6) North Carolina (2) Syracuse (3) BYU (3)




Tennessee (9) Belmont (13) Purdue (3)
Michigan (8) Utah St (12) Notre Dame (2)




Illinois (9)
Belmont (13)
UNLV (8)
Pittsburgh (1)




Florida St (10)
BYU (3)
Texas A&M (7)
Florida (2)




Utah St (12)


Kansas St (5)






Belmont (13)


Wisconsin (4)






Gonzaga (11)


St. John's (6)






Michigan St (10)


UCLA (7)










Sonny Moore: 12




Field 64 Field 32 Sweet 16 Elite 8
Villanova (9) None Syracuse (3) None
Mason (8)
North Carolina (2)




Penn St (10)


Temple (7)






Illinois (9)


UNLV (8)






Michigan St (10)


UCLA (7)










ESPN Decision Tree: 77




Field 64 Field 32 Sweet 16 Elite 8
Tennessee (9) Cincinatti (6) Syracuse (3) BYU (3)
Michigan (8) Connecticut (3) North Carolina (2) Pittsburgh (1)




Richmond (12) Utah St (12) Purdue (3)
Vanderbilt (5) Wisconsin (4) Notre Dame (2)




Old Dominion (9)
BYU (3)
Butler (8)
Florida (2)




Utah St (12)


Kansas St (5)






Gonzaga (11)


St. John's (6)






Michigan St (10)


UCLA (7)



Lunardi RPI, Pomeroy and NCAA Official RPI upsets are available in yesterday's Part I post.

Image credit: DowntownHouston.org

No comments: