One of the easiest and most common criticisms of Aaron Boone, and any manager really, is over bullpen usage. With the benefit of hindsight, it’s so easy for us to armchair manage and say what we would have done differently to win a game. Crucially though, bullpen usage was a strength of Joe Girardi, and I think there may be something here about poor bullpen usage.
It is easy to anecdotally critique a decision based on how it works out or if it seems to make little sense, but it is harder to quantify bullpen decision making. To do so, we’re going to rely on two statistics that can help tell us which relievers are brought into important situations, and how they have performed in those contexts.
Leverage Index
The first stat is known as leverage index (LI). Basically, this number tells you how important the game situation is when a reliever is brought in. An LI of 1.0 is an average situation. Anything below 1.0 is low leverage, and anything above is considered high leverage. Naturally, you want your best relievers pitching when the leverage index is highest. Let’s see which Yankee pitchers are brought into high leverage situations so far this year:
Pay attention to the highlighted row and column. Adam Ottavino and Aroldis Chapman have been used in the highest leverage situations which sounds good. One of the best closers in baseball and the guy brought in to be the new fireman. The problem we have here is in the highlighted row, Jonathan Holder. Holder tends to come in during the middle innings before going to the big guns later in the game. Unfortunately, games are often won and lost during these times. And based on who is in the Yankee bullpen, it seems problematic to have Holder coming into the 3rd most important situations. Next let’s take a look at performance to see if these guys have been stepping up when they needed to.
Win Probability Added
Win Probability Added (WPA) is a measure of how well a reliever did in a game relative to increasing or decreasing the chance of winning. Measuring reliever value or performance is so hard because of small samples, so sometimes it is easier and better to see if they improved or worsened a team’s chance of winning the game. Crucially, WPA takes leverage into account which is helpful for our purposes so we can see if guys are performing when they need to. Here is a table, per FanGraphs, of Yankee reliever WPA:
A few things stick out there right away. First off, look at Tommy Kahnle coming up huge thus far and being far and away the best reliever based on WPA. After his struggles last season, that is awesome to see. I would not put much stock in Chapman’s low WPA because as a closer, the team already has a high chance of winning whenever he enters a game. Increasing the win probability from 95% to 100% is not that big a deal. However, when you blow a save, that can lead to a huge spike downwards. If blowing a save leads to a 90% decrease in WPA (which it easily could if you give up a walk-off), you would need somewhere between 10-20 saves to counteract that. Thus, I’m not concerned about Chapman based on this stat.
What I am concerned about, and should have been apparent, is that Holder has the lowest WPA on the team of qualified relievers, yet is brought into the 3rd most important situations! How many times do we need to see Holder come into a game with a 1-run deficit and let the game get away before the Yankees realize to use a different strategy?
This analysis also shows us one way in which Zack Britton has struggled this year. He clearly is not back to the guy he was before the injuries, and it remains to be seen whether signing him instead of re-signing David Robertson was a good idea.
Concluding Thoughts
The biggest takeaway for me in doing this is let’s get Tommy Kahnle some higher leverage innings. All those close game appearances going to Holder can easily go to Kahnle, and so far he has shown he is up to the task. Right now, Kahnle on average comes in for below average leverage outings, and I hope that changes moving forward.
As for Holder, I wrote a similar article last year about how Boone needs to reevaluate his usage. And then he went off and threw 20 straight scoreless innings to make me look really dumb. Here’s to hoping Boone and the giant analytics staff know way more than I do again and that the bullpen starts performing better and more in-line with how pitchers are used.
You can contact Rohan on Twitter @rohanarcot20
