The Player Empirical Comparison and Optimization Test Algorithm, better known by the backronym PECOTA, was developed in the early aughts by Nate Silver, at the time a Baseball Prospectus writer and now better known for his work in political forecasting. Since then, it has predicted player and team performance in Major League Baseball. Like Silver’s political forecasting, the system does a reasonably good job, and there is no chance I could devise a formula to predict performance so well without constantly reconfiguring the results to fit my preconceived notions. And like Silver’s political forecasting, those behind it are stridently stubborn about it when they are wrong.
And perhaps they should be! If a forecast is wrong one time, the worst thing a statistically-inclined person can do is over-correct. But it can be a bit tedious. When Nate Silver defends himself against accusations that he botched projections of the 2016 presidential election by noting that he gave Donald Trump a much better chance (28.6%) than most other political talking heads, he isn’t wrong–after all, if a player with a .286 on-base percentage (no active player has this, but Scott Kingery has a career .284, so let’s just say him) gets on base, it isn’t the outcome you expect, but you aren’t going to be astonished when it happens–but to not at least consider the potential shortcomings of your system is malpractice. No, you shouldn’t just add a few percentage points to balance out where you went wrong, but you should also be mindful of what may have gone wrong in your calculations.
No individual player more befuddles PECOTA, and most projection systems really, than Chicago Cubs pitcher Kyle Hendricks. Every year, Kyle Hendricks, who has been the eighth-best starter in baseball by earned-run average over the last half-decade, is projected to come crashing back down to Earth, and every year, he continues to be one of the sport’s most effective pitchers. Hendricks was the kind of pitcher that sabermetricians throughout most of the 21st century have warned was overrated because his ERAs are typically lower than his FIPs–he doesn’t strike guys out at an elite level, had an elite defensive infield behind him, and wasn’t much of a prospect. But he keeps doing it. And even still, he is routinely projected not only to regress by ERA–an outcome which, for a variety of reasons more befitting explanation in an article specifically about Kyle Hendricks–but by FIP, to levels that he has never reached previously.
The strength of a projection system is that it is free from personal biases, but the weakness is that it, to assure its strength, must be free from discretion. If Craig Goldstein or Dan Szymborski believed Kyle Hendricks was better than PECOTA or ZiPS said and manipulated his numbers, they may come up with a more accurate result, but what’s to stop them from doing so with every player, to a point where a system we praise for its neutrality is in reality based on the (highly educated) whims of one man? They would never do this, and as a whole this is a good thing, but it does demonstrate the limitations of any computer projection system.
There is a tradition, usually shortly after the Super Bowl, where Baseball Prospectus publishes its projected team standings by PECOTA, and in turn, St. Louis Cardinals fans complain that the system is biased and wrong. The system isn’t biased–quite to the contrary, it is frigid and algorithmic. But it is overwhelmingly wrong about the Cardinals. To a point where the 80.6 win projection given to the Cardinals for 2021 isn’t worth your concern.
Over the last decade, Baseball Prospectus has published projections for the Cardinals. Eight times, the Cardinals won more games than the projection. One time (last season), the projections actually oversold the Cardinals, but this is easy to declare a one-time fluke–PECOTA projected 31 wins in a 60-game schedule and the Cardinals won 30 in 58, which was a 31.03 win pace. And in 2012, the system projected the Cardinals to win 88 games, and they nailed it.
PECOTA has missed badly on the Cardinals a few times. The 2013, 2015, and 2017 teams were projected for 84, 89, and 76 wins and they ended up with 97, 100, and 83. Over the last decade, if you adjust for 2020 scheduling weirdness (which makes PECOTA look better than if you don’t, which runs counter to my point, but I am a man of the scientific method), the system has averaged underselling the Cardinals by 4.8 wins per season.
And if you were to add 4.8 wins to the 80.6 win projection, putting the Cardinals at 85.4 wins, this would surely annoy some partisans, but this seems about right. Nolan Arenado is more valuable to the Cardinals than Kolten Wong, though probably not by as much as most people think. Dexter Fowler will be replaced by a roughly analogous player. Miles Mikolas comes back while Dakota Hudson is lost, which is an upgrade, but otherwise, the roster is mostly the same, and, with vanishingly few exceptions, is mostly slightly-to-way past its prime. And the Cardinals played at an 84-win pace last season, so even though they would have still made the postseason under the five-teams-per-league normal, they weren’t exactly a juggernaut.
The 2021 St. Louis Cardinals have improved with the recent acquisition of Nolan Arenado and re-acquisitions of Adam Wainwright and Yadier Molina, but this is still a flawed team. And perhaps the virtue of PECOTA in the context of the Cardinals is that it could force people to consider that the team could win 80-81 games and finish third in the division. But I certainly wouldn’t predict it. If I were a betting man, I would bet heavily on the over in a 162-game season. And for a decade, history has been on my side.