One of my favorite parts of the thought experiment I call the Rating Systems Challenge is discovering how well the AP and USA Today voters performed when they submitted their preseason ballots. The answer, more often than not, is "pretty well." Take this year for example. Through two rounds, the preseason polls are ranked first and second in both points thus far and most possible points through the National Championship.
For those of you who are unfamiliar with this project, it's rather simple. I look at NCAA men's basketball rating systems, simulators and betting markets. I take the data they provide and construct an NCAA Tournament bracket based on these data. For polls and rating systems, it's rather simple: the higher ranked school advances over the lower ranked school. With simulators and betting markets, the team with the highest projected likelihood of advancing is the team that advances. In cases that require a tie breaker, I rely on chalk (the better seeded team advances).
As a collective, the systems picked three different schools to win the South, West and Midwest regions and two to win the Midwest. In total, the systems have 11 different schools reaching the Final Four, six schools reaching the National Championship game and four winning it all. These are above average numbers that reflect the parity at the top of the men's game in 2015-16.
At the time of posting, three of these 11 schools have already exited the tournament, most shockingly Michigan State. Fourteen schools put the Spartans (who lost in the first round) in the Final Four; ten had them losing to the eventual national champion. Four systems, including Sonny Moore, Sports Reference's adjusted net margin ratings, Christopher Long's simulation and ESPN Insider's simulation, had the two-seed cutting down the nets this year in Houston.
It's early yet, and no system has been eliminated from claiming the imaginary title of Best NCAA Tournament Rating System 2016, but some are in better position than others. Interestingly, all but four systems are an improvement on chalk through the Round of 32. As we get deeper into the tournament, I will call out the systems that have performed best and heap (sometimes undeserved) derision on those that have not, but for now I'd like to direct your attention to the Nolan Power Index.
Historically, Nolan Power Index (NPI) does a terrible job of building a bracket. That said, NPI has done something that none of the other systems have done two years in a row: successfully projected Stephen F. Austin to advance. This may be a case of the broken clock that's right twice a day, and considering NPI's past performance this is the likely explanation. However, it is still worth considering that maybe, just maybe, Warren Nolan knows something the rest of the systems (and most of America) do not. Nevertheless, I derive a not insignificant bit of joy when these projected long-shots pay off.
Check my work:
- Constructed Brackets
- Links to Data
2 comments:
Awesome! I want to thank you:
1) For doing this. I always love seeing how these systems perform in your nice, concise format.
2) For keeping the graphics from the first post separate from the second post. It makes it easier for me to look back later and see how things progressed.
3) For the links! They'll be useful when I want to pick a system to copy for next year's bracket :).
Thanks, Mark. All a byproduct of moving the whole process in-house rather than dumping it into an ESPN Tourney Challenge pool.
Post a Comment