29 March 2014

Rating Systems Challenge: 'Cats are Wild as Vegas Takes Lead


This post is part of a series tracking the successes and failures of various NCAA Men's Basketball ranking systems and bracket models throughout the 2014 NCAA Tournament. Click here to check out the full series.

Last night, twenty Final Four picks, eight finalist picks and five National Championship picks died in the span of just a couple of minutes when 4 Louisville and 1 Virginia were sent packing by 8 Kentucky and 4 Michigan State. And while they didn't shake up the top of the Rating Systems Challenge leaderboard too much, those upsets have major implications for most systems going forward.



As you can see, Vegas and FiveThirtyEight still maintain the points lead, though Vegas takes sole possession of first place by virtue of points remaining. With Louisville no longer competing, prospects for Nate Silver's system look bleak. It's going to take an upset of 1 Florida for them to have a shot at the crown this year. On the other hand, the ESPN National Bracket and AP Postseason Poll are well-positioned moving forward. Just short of the top, they have the strongest 1-seed in the tournament as their championship pick.

What's almost as interesting is how poorly all of these systems are performing right now. Only two are beating chalk, and on average they only rank in the 64th percentile of ESPN Tournament Challenge brackets. The leaders have performed well against the field, but not great. The best-regarded rating systems—Pomeroy and Sagarin—are closer to the bottom than they are the top. One of the best performing models of the past, the Survival Model, is next-to-last. With the exception of FiveThirtyEight, it has not been a great March for the computers just yet.


There's a lot more grey on that board than there was yesterday. Ten systems had Virginia winning the East (but advancing no farther). Another ten had Louisville winning the Midwest, eight of which picked them to reach the finals and five of which picked them to win it all. No more.

Today's Elite Eight games all have Final Four implications (somewhat obviously). Every system in the Challenge has 1 Arizona winning this one, but a 2 Wisconsin victory would hurt some systems more than others. Pure Chalk and RPI expect Zona to reach the finals and go no further; BPI, Lunardi RPI, Pomeroy and Sagarin expect them to win it all.

Sixteen systems expect Florida to advance, all of which expect them to reach the finals and seven of which expect the Gators to win it all. A shocker by 11 Daytona would put the biggest hurt on the ESPN National Bracket, Vegas, the Survival Model, RPI, Pure Chalk and the postseason polls. It would also level the playing field for the preseason polls (which picked Kansas).

How will today's games affect the rating systems' chances going forward? Check back tomorrow—or follow along on Twitter—to find out.

No comments: