30 September 2019

Validating 2019 Projections: Swings and Misses

Me in March:

That's right, I just projected that the New York Mets would win top seed in the National League. You're probably thinking that there's something wrong with my model. I'd say, yeah, you're probably right, but I don't have time to look for whatever it is.

Well, I spent six months looking and, no, I still don't know what went wrong there. I more or less nailed the projection of how many games the Mets would win, but obviously 87 wins was not enough to win the NL (nor did I expect it would be).

On average, I missed a team's final win rate by .056, or about 9 wins. That's… not great. The 80th percentile error was .080, or 13 wins. I projected an 80% confidence interval of 12.2 games back in March, so I didn't limbo under that bar, either.

Five of the seven teams that finished outside my margin of error had either historically great (Los Angeles, Minnesota) or historically awful (Kansas City, Detroit, Baltimore) seasons. My simulator, which tunes the amount of error it injects into the Monte Carlo simulation to match historical margins of error, might not have been ready for a season in which three American League teams outright tanked.

I expect the Phillies, Cincinnati Reds and Kansas City Royals to be much improved as well

No, kinda, and no. The Phillies and KC were one win better than 2018. Cinci added eight wins. My projections missed by five, eight and 16 wins.

On the flip side, the Cubs, Seattle Mariners and Oakland Athletics are three teams I expect to tumble this season.

Yep, yep and absolutely not. Oakland won 97 games last season. This season I projected them to win 80. Instead, they won… 97 games. But the Cubs and M's did tumble—I missed their final win totals by three games apiece.

Finally, my playoff projections were a mixed bag. I correctly pegged the Astros as the overall #1, the Yankees as the AL's #2, and the Rays as a strong wild card team. I also had the Dodgers winning the NL West and the Nationals in the wild card slot.

But I also had the Indians winning the AL Central, the Red Sox and Phillies landing wild card berths, the Mets winning the NL (bumping Atlanta into the WC slot), the Brewers winning the Central, and the Cardinals missing the playoffs altogether.

I didn't see the A's coming on this strong at all.

So, not one of my best preseason performances. On the other hand, my season-long 60% correct pick rate is one of my best yet, so I have high hopes for October. You can follow along with my 2019 postseason projections here at Rational Pastime.

No comments: