Rating Systems Challenge: Equations are Red, Experts are Blue

Georgia Dome
Site of the 2013 Final Four
Source: Wikimedia Commons

This post is one in a series investigating how the various college basketball rating systems predict the men's NCAA Tournament. Click here to read the rest.

While tonight's contest will feature a battle between two squads of college students who represent the pinnacle of their sport, the winner will also determine whether the best prognosticators of NCAA Tournament Basketball were a council of experts—the survey samples for the ESPN and AP Preseason Polls—or advanced models developed by Nate Silver and ESPN.

In The Signal and the Noise, author Nate Silver makes a convincing argument that experts are typically bad prognosticators and groups of experts are seldom better. Instead, we should look at rigorous, testable models based on objective, verifiable data. However, even advanced models can fail to predict outcomes in low-information systems where minor variations can cascade into major disruptions.

Weather forecasting is one attempt to simplify such a system. Forecasting tournament play in sport is another.

Rating Systems Challenge: Final Four Preview

This post is one in a series investigating how the various college basketball rating systems predict the men's NCAA Tournament. Click here to read the rest.

The 2013 Rating Systems Challenge heads into the Final Four with only one top seed and two original picks remaining: 1 Louisville, which eighteen of twenty systems expected to make it this far, and 4 Michigan, which only the preseason polls expected to survive. Joining them are 4 Syracuse and, 2013's Cinderella school, 9 Wichita State. It should be pretty obvious by now which school the systems expect to win it all.

The implications are now clear for the Rating Systems Challenge since the systems that picked Michigan to reach the Final Four do not expect the Wolverines to advance further, and since those are the systems that are currently winning. If the Cardinals fall short of winning the National Championship, the ESPN Preseason Poll will have been 2013's best prognosticator. If Louisville does win it all, that honor will fall to the FiveThirtyEight and the ESPN Computer systems.

Rating Systems Challenge: Mascot Pun Does Basketball Thing to Math

This post is one in a series investigating how the various college basketball rating systems predict the men's NCAA Tournament. Click here to read the rest.

Yes, that's a lazy title, but it is about as much as my fatigued, basketball-ingesting mind can regurgitate this morning.

It would have made sense to come up with some sort of title illustrating how the upset victories of 9 Wichita State and 4 Syracuse threw the rating systems for a loop, but I'm not sure anything makes sense in sports at this point. Besides, my mind is still far too blown to think through anything of the sort. Just imagine a post title that incorporates the word "shocking" and, maybe, "Otto."

I'm not sure which damage is more irreparable: that to the pleasure center of my brain or to the nation's brackets. Probably the latter, and we have evidence. RP Blog is tracking twenty rating systems, yielding forty picks for East and West representatives in the Final Four. These forty picks represented five different schools.

Today, halfway through the Regional Finals, each and every one of these five schools has been eliminated. Inspect the wreckage below:

The last school standing from this half of the bracket was 2 Ohio State, which The Wichita State University took care of last night after deflecting a late Buckeye charge. What sort of effect did this have on the standings in the Rating Systems Challenge? Check the leaderboard below the jump.

Rating Systems Challenge: Preseason Polls Dominating? Don't Be So A-maized.

This post is one in a series investigating how the various college basketball rating systems predict the men's NCAA Tournament. Click here to read the rest.

Was that awful pun really necessary? Yes, yes it was.

Okay, no it wasn't, but I hope it drew your attention to something that seems amazing but is actually rather common. The preseason polls from AP and ESPN tend to do rather well in forecasting NCAA Tournament results. Last year, the AP Preseason Poll finished fourth overall in the Rating Systems challenge, six spots ahead of the AP Postseason Poll.

On average, the preseason polls are actually slightly more accurate than postseason polls at predicting postseason success. Perhaps the preseason polls are better at measuring the baseline talent level. Perhaps the postseason polls are poisoned by recency bias. Either way, this is the pattern, and it's a pattern that is playing out again this season. Largely thanks to 4 Michigan's (get it? maize?) upset of 1 Kansas, the ESPN and AP Preseason Polls rank #1 and #3, respectively.

Will their lead hold up? Maybe, but most likely not. Read why after the jump.

Rating Systems Challenge: The Bitter Taste of Bracket l'Orange

This post is one in a series investigating how the various college basketball rating systems predict the men's NCAA Tournament. Click here to read the rest.

The Shockers of 9 Wichita State may have been the lowest seed to advance last night, but 4 Syracuse's upset of 1 Indiana was by far the most disruptive. All twenty systems picked Indiana to advance to the Elite Eight; nineteen picked them to reach the Final Four. Nine had the Hoosiers in the Finals, and three systems—the preseason polls and the ESPN Decision Tree—picked Indiana to win it all.

The only system to pick a different team to win the East Regional was NCAA RPI. That system picked 2 Miami, which fell tonight in another upset to 3 Marquette. In short, all of the East Regional champion picks by the twenty rating systems have been eliminated.

Rating Systems Challenge: Regional Preview

This post is one in a series investigating how the various college basketball rating systems predict the men's NCAA Tournament. Click here to read the rest.

The Men's NCAA Tournament returns tonight at 7PM EST, bringing with it the potential for some exciting upsets on the way to the Elite Eight. Ten schools that were not the top seed in their region when the tournament began have a chance to advance. Two of those original top seeds—1 Gonzaga and 2 Georgetown—have been eliminated, clearing the way for Cinderellas such as 9 Wichita State, 13 La Salle and 15 Florida Gulf Coast.

In fact, with Wichita State and La Salle facing off against one-another, we are guaranteed to see a 9-seed or lower advance to the Elite Eight for the first time since, well, 11-seed VCU reached the Final Four two years ago.

Unfortunately—but unsurprisingly—none of the rating systems under review picked any of those underdogs to advance that far. The gutsiest picks of these systems were 4 Michigan (picked by the two preseason polls and Sports Reference), 3 Marquette (picked by the ESPN Preseason Poll) and 3 Florida.*

Rating Systems Challenge: Eagles Soar, Explorers Explore, Brackets Sore

This post is one in a series investigating how the various college basketball rating systems predict the men's NCAA Tournament. Click here to read the rest.

While last night was one of the more exciting nights in recent history for the NCAA Tournament, it was not a particularly interesting night for the Rating Systems Challenge. Our leaderboard remains essentially unchanged.

How could it be otherwise? None of the systems expected a 13 LaSalle / 12 Ole Miss match-up in the Round of 32, let alone that the Explorers would advance. And they're joining Florida Gulf Coast as the first 15-seed to reach the Sweet Sixteen?! Please. If anyone picked this--human or computer--you're not smart, just lucky (or a loyal alum).

Today's leader is the same as yesterday's leader (Sports Reference efficiency rating) and today's laggard is the same as yesterday's laggard (Nolan Power Index). With all the surprises, the disparity between best and worst is only five picks out of forty-eight. Read more about those surprises after the jump.