Pre-Pitch Predictions

To me, the best part about baseball is the game within the game that the batter plays with the pitcher between every pitch. I’ve been making videos to learn more about this aspect of the game and the more I learn, the cooler it is. When casual fans call baseball boring, I get it. If you’re turning on a random baseball game to see athleticism, power, and action, it may be a while before you see any of those things. But if you look closer, you’ll notice the chess match. The pitcher decides which pitch type to throw. The batters thinks along with the pitcher. Being able to consistently guess right could be the difference between millions of dollars in career earnings.

Some hitters engineer their swings in a way that gives them the best chance to succeed even if they guess wrong. Other hitters say they always sit Fastball and adjust to non-Fastballs. Hitting is hard. But what if there was a way to help hitters correctly guess what pitch is coming more often? Let’s dive in.

Theory

In 2016, right when I was getting serious about pursuing a career in baseball analytics, I came across an article titled “10 Things I Believe About Baseball Without Evidence” by Ken Arneson. Only after rediscovering the article and re-reading it recently did I realize that Ken’s points about the batter-pitcher interaction (#4 and #5 on his list) almost perfectly put into words a theory that I had in my head but couldn’t quite figure out how to verbalize. Here’s how he put it:

“I believe that before any given pitch, the batter is in some sort of Prediction State for the next pitch. After each pitch, the batter then moves into a different Prediction State.”

The idea is that before every pitch, the hitter has an idea of what might be coming. It could be conscious (“He’s probably throwing me a fastball here”) or a more vague subconscious thought about what is coming based on their previous experiences (but I can’t claim to know how this prediction is made). Ken gave the following example of what each batter’s pre-pitch prediction might look like:

Ken’s implication is that the more accurate the batter’s pre-pitch prediction is, the more success he’ll have at hitting that pitch. It’s easier to hit a pitch you are expecting than one you are not!

In this article, I want to estimate what the batter’s pre-pitch prediction should look like for every pitch. To simplify, I am only going to look at pitch type (the first line of Ken’s above example) and leave pitch location for a future project.

To do this, I will create a model that will predict the distribution of pitch types for the next pitch that the pitcher is about to throw, mimicking what hitters do in their head, consciously or unconsciously, before every pitch they see.

Thought Experiment

Let’s have some fun for a second. Given the following image from a pitch in a 2019 Regular Season MLB game, guess what pitch is thrown in the video (Fastball, Slider, Changeup, or Curveball). No cheating!

Stripped of all relevant details, your pre-pitch pitch type prediction (catchy name, isn’t it?!) probably looks something like this:

I bet you picked fastball. Because I’m an oracle? No! Because 57% of pitches in Major Leagues last year were fastballs, making it the best guess for any random pitch in the absence of all other information. This distribution is called the marginal distribution, meaning it is the distribution of pitch types that you should guess if you know absolutely nothing else.

Now, if you could only ask for one piece of information from me in order to improve your prediction, what would it be? The inning? The date of the game? The temperature outside? NO! You probably would ask me for the identity of the pitcher.

So i’ll tell you. It’s Diego Castillo.

Now, if I asked you to guess what pitch would be shown in the video, I bet your opinion has changed to something like this:

Since Castillo exclusively throws two pitches with almost equal frequency, I bet you assigned a much higher probability to the pitch being a Slider. The table above represents the conditional distribution of pitch types given that we know Diego Castillo is the pitcher. That really changed our guess, huh? Because information about who the pitcher is can change our guess a lot from the marginal distribution, the identity of the pitcher is said to be important in predicting which pitch type will come next. (If I gave you information on a variable that did not change our guess, like the stadium temperature at the time of the pitch, that variable would be unimportant.)

What is the next piece of information you would like from me to improve your guess? When I did this exercise, I decided that the second most important piece of information would be the handedness of the batter since many pitchers pitch Righties and Lefties very differently.

This pitch was against a Righty. Here is how our pre-pitch guess could look given the pitcher and the handedness of the hitter:

Now instead of a coin flip, we have a little more confidence in a Slider being thrown here, but it doesn’t quite change the prediction as much as the last piece of information did. Each time new information is added, our guess changes.

I hope you get the point. We could keep adding more context to the situation to improve our model, and each addition would increase our guess a little less than the previous one did. Finally, suppose I tell you the count and the identity of the hitter.

Our mystery pitch is a 3–2 pitch from Diego Castillo to Righty Aaron Judge. Oh, and the previous pitch was a Slider. Is that enough info for you to make a good guess as to which pitch Castillo throws in this at-bat? For computers, it is.

Model Background

I want to note something important here. Every time we added new information during our thought experiment, the sample size of the conditional distribution gets smaller and smaller, hurting our confidence in our guess. With no information, we looked at the distribution of more than 700,000 pitches which was every pitch in the 2019 MLB Regular Season. When we looked at Diego Castillo’s overall distribution, that data included every pitch he threw which was only 1,119 pitches. Our guess given Diego Castillo against a Righty was only based on 709 pitches. Now, looking at all pitches in 2019 thrown to Judge by Castillo in a 3–2 count gives us a sample size of 1.

Subsetting down the data after every time you get new information is, in my understanding, how a Frequentist statistician would approach this problem. However, this sample size issue can be avoided using a Bayesian approach.

I am no Bayesian expert, but my understanding is that a Bayesian model begins with a set of prior beliefs (called priors) and incorporates new information (called likelihood) to make a prediction (called a posterior), and that is what my model will do. So instead of subsetting down to Castillo’s pitches against Righties with a 3–2 count, where the last pitch was a Slider, my model will take Castillo’s distribution against all Righties on 3–2 counts and combine that with Castillo’s pitch mix distribution on pitches after Sliders (using a weighted average to combine the two distributions).

Features

So to recap, the pieces of information I am including in this model are Pitcher Identity, Count, Batter Handedness, and Previous Pitch Type. I am including previous pitch type because hitters are thought to incorporate information about previous pitches into their conception of what pitch is coming next. I agree with Ken Arneson that pitches do not happen in isolation (#2 on his list) so adding information about the previous pitch is a step toward acknowledging that fact.

You may also notice that I did not include Batter Identity in my model. This is simply because I do not have the computational resources to add this variable (and adding it increases the time to run the model exponentially), though I did find evidence that including this variable would improve the model, as would including game situation (inning, outs, runners on, leverage, times through the order, result of the last pitch, etc.). This is a type of model that could be run in parallel, so it could be scaled up to include these things with more computational resources.

Data Cleaning

As you may have noticed by now, I grouped pitch types in my data into four categories. 4-Seam Fastballs, 2-Seam Fastballs, Sinkers, and Cutters are “Fastballs”, Changeups and Splitters are “Changeups”, Curveballs and Knuckle Curves are “Curveballs” and Sliders are “Sliders” with Eephuses, Knuckleballs, and Pitchouts being removed. I did this to mimic the brain of a hitter, who is likely not differentiating between a 4-Seamer and 2-Seamer, for example, in their mental pre-pitch prediction since both are going to be similar speeds. Since hitting is so hard already, I think this step will prevent any unnecessary over-complications.

Results

All of this theory is great, but nothing matters unless the model actually works.

The output of my model gives a distribution of pitch types with posterior probabilities for each. For the pitch in the thought experiment, here is what the model predicted:

So for this pitch, I would say the model has 65% “confidence” in a Slider being thrown and 35% “confidence” in a Fastball being thrown in this scenario.

The actual pitch type for the pitch was a slider! (Watch here!)

So the model was right for that pitch, but let’s check out how accurate the model is overall. If you simply look at how often the model correctly identified the actual pitch type as the one most likely to have been thrown, the model is accurate for 60% of pitches in 2019. Now I realize that’s not an eye-popping number. If I made a model that predicted every pitch would be a Fastball no matter what, it would have 57% accuracy. But there’s hope.

What if we only looked at pitches the model predicts with over 50% confidence? 75% confidence? 99% confidence? A Fastball-only model would not improve in these cases, but my model does.

The more confident my model is, the better it performs. That’s how things should be! In the worst case, the model is 60% accurate and in the best case, the model is 99.98% accurate. Here is how the model performs at various confidence levels:

So is it possible to accurately predict which pitch type is coming on the very next pitch using only Pitcher Identity, Count, Batter Handedness, and Previous Pitch Type? Yes. And the model would be better if I had the computational resources to include more variables like Batter Identity, Second Most Recent Pitch, Game State, etc.

Applications of the Model

We love a good model, but I have realized recently that a lot of my research projects on this blog end after the conclusion without addressing the crucial question “How is this project going to help the team?” With this project, I have a definitive answer to how this could immediately impact a team’s results on the field.

If the model’s confidence is above a certain threshold, relay the most likely predicted pitch type to the batter.

This may trigger alarm bells and PTSD flashbacks to trash can banging in Houston, but let me explain why I think this is different. Firstly, this data is publicly available, and every team can freely make their own similar model. Secondly, the predictions are not guarantees (like the stolen signs were), so the team using this strategy is taking on risk that the model is wrong. Lastly, this model is just a more complex version of reminding a hitter in the box that the pitcher on the mound loves throwing his Fastball when behind in the count or that he loves doubling up on Curveballs. These are things that are already relayed to the batter, so I don’t see how this more complex message is morally any different.

When I asked an established and well respected coach in an MLB organization if relaying the results of this model to a hitter would be unethical, he simply replied, “No.” If you are reading this and disagree though, I would love to hear your opinion on this!

So assuming it is ethical and breaks no written rule, relaying the model’s predicted pitch type to the hitter could be extremely lucrative. Basic logic would suggest that batters are more successful against pitches they are more prepared for. I looked at the average linear weight of pitches above each pitch probability cutoff and found that batters tended to do better on pitches the model assigned higher probability to in 2019:

Values above the red line are beneficial to the offense

This lends evidence to the claim that hitters performed better on pitches with higher probability of being thrown last season.

The Strategy

In a world where all of this is true, a team could adopt the offensive philosophy that when the model confidence is below the threshold (let’s say 70%, which the graph above suggests is the optimal cutoff value), the batter will hit like normal according to his usual approach. But when the model confidence is above 75%, the batter will be relayed the model’s pitch type prediction (Fastball, Slider, Changeup, or Curveball) and sell out for that pitch paying to attention to the probability of other pitch types being thrown.

As shown in the chart above, if every team used this strategy with 75% confidence as its cutoff, they hypothetically would have signaled a pitch type to a batter on about 14% of pitches in 2019 and would have been right 92,031 times. That’s a lot of potential for increased damage and the positive benefit should far outweigh the 14,697 pitches that would have been predicted incorrectly.

If a team wanted to increase their risk tolerance (getting them more correct predictions but also more incorrect predictions), they could lower the confidence cutoff to below 75%. Risk averse teams could use this method with a confidence cutoff of way more than 75%, virtually guaranteeing the prediction would be right but only on about 3% of total pitches.

Additionally, the model could be used to tell the hitter which pitch likely is not coming which would help the hitter narrow down their prediction from three pitches to two. In 2019, pitches that the model assigned less than 5% probability of being thrown were only thrown 0.1% of the time, meaning that hitters could safely rule out these pitches without much consequence. However, there is less evidence that using the model to identify which pitch will not be thrown would be beneficial than using the model to identify the pitch that will be thrown.

Using this strategy could help hitters hone in on which pitch type is coming on the next pitch, allowing them to “sell out” and thus achieve better outcomes.

Secondary Applications

In addition to the use of this model laid out above, I think the results of this model have additional uses. Chiefly, it can be used by the pitching team to quantify how predictable their pitcher’s pitch mix is. If the model shows high confidence that a pitcher is about to throw a certain pitch type, his team may want to adjust their pitch calling to remain a step ahead of the hitter. Secondly, I think further research can be done regarding the affect that predictability and adhering to the pre-pitch expectation of the batter has on pitcher and hitter performance using this data.

Limitations

Not every pitch is predicted equally. In the interest of full disclosure, I want to make sure to note that among pitches with 90%+ confidence in 2019, about 96% were fastballs.

Furthermore, many of the correct predictions at the 90% confidence level were either on the first pitch of the plate appearance or when the hitter was ahead in the count (when Fastballs are more common).

Although the model’s successes were not randomly distributed among all pitches, I still think the results are potentially beneficial in identifying when fastballs are coming or when they are not in the minority of cases. Even if this model is correctly predicting the most easy pitches to predict, those predictions can still be helpful if acted on appropriately.

In addition, other limitations of this model include its lack of interesting variables like Batter Identity, Game State, Previous Pitch Result, and Times Through the Order; the fact that the prediction does not include pitch location; the time the model takes to run; and its lack of accuracy at low confidence levels.

Note: I tested a version of this model that included Batter Identity on every pitch thrown in 2019 by a random sample of 13 pitchers. That model performed about 3 percentage points better than my baseline model at every confidence level, lending evidence that more variables would improve this model further.

Conclusion

There is an offensive advantage to be had by MLB teams using a predictive model to decrease batter uncertainty about which pitch type they are about to see. A Bayesian model that incorporates many of the most important variables to predict the pitch type of the next pitch appears to be fairly accurate at worst and very accurate at best, depending on how much risk the user wants to take while relying on the model. A team using this model selectively to inform the hitter of what pitch type is coming has the potential to see improved offensive outcomes. When baseball comes back, I would love to see a team give this a shot.

As always, please reach out to me on Twitter or DM me @Moore_Stats with any questions or feedback!