Are you kidding me? Can framing be worth that much?

Posted: July 6, 2018 in Catching, Projections, Uncategorized

I created a bit of controversy on Twitter a few days ago (imagine that) when I tweeted my top 10 to-date 2018 projections for the total value of position players, including batting, base running, and defense, including positional adjustments. Four of my top 10 were catchers, Posey, Flowers (WTF?), Grandal, and Barnes. How can that be? Framing, my son, framing. All of those catchers in addition to being good hitters, are excellent framers, according to Baseball Prospectus catcher framing numbers. I use their season numbers to craft a framing projection for each catcher, using a basic Marcel methodology – 4 years’ weighted and regressed toward a population mean, zero in this case.

When doing this, the spread of purported framing talent is quite large. Among the 30 catchers going into 2018 with the most playing time (minors and majors), the standard deviation of talent (my projection) is 7.6 runs. That’s a lot. Among the leaders in projected runs per 130 games are Barnes at +18 runs, and Grandal and Flowers at +21. Some of poor framers include such luminaries as Anthony Recker, Ramon Cabrera, and Tomas Telis (who are these guys?) at -18, -15, and -18, respectively. Most of your everyday catchers these days are decent (or a little on the bad side, like Kurt Suzuki) or very good framers. Gone are the days when Ryan Doumit (terrible framer) was a full-timer and Jose Molina (great framer) a backup.

Anyway, the beef on twitter was that surely framing can’t be worth so much that 4 of the top 10 all-around players in baseball are catchers. To be honest, that makes little sense to me either. If that were true, then catchers are underrepresented in baseball. In other words, there must be catchers in the minor leagues who should be in the majors, presumably because they are good framers though not necessarily good hitters or in other arenas like throwing, blocking pitches, and calling games. If this beef is valid, then either my projection methodology for framing is too strong, i.e., not enough regression, or BP’s numbers lack some integrity.

As a good sabermetricians should be wont to do, I set out to find out the truth. Or at least find evidence supporting the truth. Here’s what I did:

I did a WOWY (without and with you – invented by the illustrious Tom Tango) to compare every catcher’s walk and strikeout rate with each pitcher they worked with to that of the the same pitchers working with other catchers – the without. I did not adjust for the framing value of the other catchers. Presumably for a good framing catcher they should be slightly bad, framing-wise, and vice versa for bad-framing catchers, so that there will be a slight double counting. I did this for each projected season 2014-2017, or 4 seasons.

I split the projected catchers into 3 groups, Group I were projected at greater than 10 runs per 150 games (8.67 per 130), Group II at less than -10 runs, and Group III, all the rest. Here is the data for 2014-2017 combined. Remember I am using, for example, 2017 pre-season projections, and then comparing that to a WOWY for that same year.

Total PA Mean Proj per 130 g W/ BB rate WO/ BB rate Diff W/ SO rate WO/SO rate Diff
74,221 -12.6 .082 .077 .005 .197 .206 -..009
107,535 +13.3 .073 .078 -.005 .215 .212 .003
227,842 -.2 .078 .078 0 .213 .212 .001

 

We can clearly see that we’re on the right track. The catchers projected to be bad framers had more BB and fewer SO than average and the good framers had more SO and fewer BB. That shouldn’t be surprising. The question is how accurate are our projections in terms of runs. To answer that, we need to convert those BB and SO rates into runs. There are around 38 PA per game, so for 130 games, we have 4,940 PA. Let’s turn those rate differences into runs per 130 games by multiplying them by 4,940 and then by .57 runs which is the value of a walk plus an out, which assumes that every other component stays the same, other than outs. My presumption is that an out is turned into a walk or a walk is turned into an out. A walk as compared to a neutral PA is worth around .31 runs and an out around .26 runs.

Total PA Mean Proj per 130 g W/ BB rate WO/ BB rate Diff in runs/130 W/ SO rate WO/SO rate Diff
74,221 -12.6 .082 .077 +14.0 .197 .206 -.009
107,535 +13.3 .073 .078 -14.0 .215 .212 .003
227,842 -.2 .078 .078 0 .213 .212 .001

 

Let’s make sure that my presumption is correct before we get tool excited with those numbers. Namely that an out really is turning into a walk and vice versa due to framing. Changes in strikeout rate are mostly irrelevant in terms of translating into runs, assuming that the only other changes are in outs and walks (strikeouts are worth about the same as a batted ball out).

Total PA Mean Proj W/ HR WO/HR Diff W/ Hits WO/Hits Diff W/ Outs WO/

Outs

Diff
74,221 -12.6 .028 .028 0 .204 .203 .001 .675 .681 -.006
107,535 +13.3 .029 .029 0 .200 .198 .002 .689 .685 .004
227,842 -.2 .029 .029 0 .199 .200 -.001 .685 .683 .002

 

So, HR is not affected at all. Interestingly, both good and bad framers give up slightly more non-HR hits. This is likely just noise. As I presumed, the bad framers are not only allowing more walks and fewer strikeouts, but they’re also allowing fewer outs. The good framers are producing more outs. So this does in fact suggest that the walks are being converted into outs, strikeouts and/or batted ball outs and vice versa.

If we chalk up the difference in hits between the with and the without to noise (if you want to include that, that’s fine – both the good and bad framers lose a little, the good framers losing more), we’re left with outs and walks. Let’s translate each one into runs separately using .31 runs for the walks and .26 runs for the outs. Those are the run values compared to a neutral PA.

Total PA Mean Proj per 130 g W/ BB rate WO/ BB rate Diff in runs/130 W/ Outs WO/

Outs

Diff
74,221 -12.6 .082 .077 +7.7 .675 .681 +7.7
107,535 +13.3 .073 .078 -7.7 .689 .685 -5.1
227,842 -.2 .078 .078 0 .685 .683 -2.6

 

So our bad framers are allowing 15.4 runs more per 130 games than the average catcher or than their others at least, in terms of fewer outs and more BB. The good framers are allowing 12.8 fewer runs per 130 games. Compare that to our projections, and I think we’re in the same ballpark.

It appears from this data that we have pretty strong evidence that framing is worth a lot and our four catchers should be in the top 10 players in all of baseball.

Advertisements
Comments
  1. Elliott Andrew Frankfother says:

    I am sorry if I appear to be wielding his name as though it were a cudgel of some kind, but on the MLB Network “Top 10 Right Now” show (which I know you have some dislike for) Bill James did not have Flowers in his top 10 catchers, saying that he did not believe that catchers “sneaking” extra strikes was worth THAT much. Is this a difference that you two have? Would you know if James does not buy into framing as much? I’m not certain, but I think that you two know each other pretty well. Thank you!

    • Mitchel Gordon Lichtman says:

      Never had any discussions with him about that. My research speaks for itself. He’s welcome to comment on it. No one else that I know of, at least publicly, includes any semblance of a catcher framing projection into their “best of” lists.

  2. Guy Molyneux says:

    I’m not sure this data supports the projections that well. Let’s focus on the good framers. I could be thinking about this wrong, but it seems to me that counting both the run value of their extra outs *and* the value of fewer BBs is basically counting the same thing twice. Good framers are reducing walks allowed, which increases the likelihood of an out. The bottom line is that good framers are reducing hitters’ OBP from .315 to .311, and that’s worth about 5 runs. Isn’t that basically all we care about?

    Now, it appears that in addition to lowering OBP by 4 points, these catchers are trading a few walks for hits (in a bad way). IF that is true, that basically wipes out the 5 run value of reducing OBP. But even if we accept your speculation that the hit data is just noise, aren’t the good framers worth about 5 runs?

    • Mitchel Gordon Lichtman says:

      Guy, as I said on Twitter, it’s not double counting if you use the .31 and .26 for the values of the walk and out, respectively. If you trade a walk for an out, that’s a difference of .57 runs. I could have used that value and then just used the increased or decreased walks. As far as the increased number of non-HR hits for good framers, I can think of no credible reason why the number of hits would increase as a result. Can you?

      • Guy Molyneux says:

        OK, I agree that it’s not double-counting. But I do think you need to include the difference in hits allowed in the analysis. The framing metric you are using does not credit catchers only for increasing Ks or reducing BBs, it ascribes a value to every extra strike based on the change in run expectancy. That value includes the impact of the count on hits allowed. The fact that your study doesn’t show good framers reducing hits allowed — in fact, the reverse — is certainly germane. Ignoring those outcomes is no more correct than studying sac bunts and looking only at plays where the batter is out at first.

        Now, do I think good framers really allow a higher BABIP? Probably not. But it may also be the case that their true impact on Ks and BBs is smaller than you found. I think the best approach is to measure everything. Doing that, it does appear the good framers save a significant number of runs, but perhaps 20-30% less than projected.

        • Guy Molyneux says:

          Actually, the story for the good framers is not as good as I first thought. Look at the average framers: they also increase the out% compared to their backups, and they actually allow slightly fewer hits (and same number of HRs). By my back-of-envelope estimate, the average framers are +9 runs compared to their backups over 130 games. But the good framers are only +11 runs compared to their backups, so we can only attribute an advantage of 2 runs to framing. (You can give us more precise estimates by calculating component ERAs.) So most of what you are seeing with the from the good framers may just be that backup catchers decrease pitcher performance a bit (because of less familiarity, pitching mainly in day games, or whatever).

          • Mitchel Gordon Lichtman says:

            Did you see my latest tweet? I calculated the linear weights and the wOBA of both good and bad framers, which, as you say, other than noise problems, is the only thing that counts. Those numbers support my thesis that framing talent, based on performance compared to projections, is huge.

            Bad framers: w/wOBA .335 w/o wOBA .329
            Good framers: w/wOBA .323 w/o wOBA .326
            The rest: w/wOBA .327 w/o wOBA .328

  3. Guy Molyneux says:

    So we’re saying an average framer is better than his backups by .001 wOBA, while the good framers are better by .003. That suggests framing is reducing wOBA by .002. I believe that’s equivalent to about .06 RA9, or 7.8 runs over 130 games. So I agree there is a real signal in the projection, but this study suggests the value may be only about 60% of the projection. Obviously, there’s measurement error here, so maybe good framers are really worth 90% or 100% of the projection — but it could also be 40%. It would take other/larger studies to know.

    On the other end of the spectrum, this study seems to suggest there are some truly bad framers. Based on wOBA allowed, the bad framers are costing their teams .20 RA9, or -27 runs (!) over 130 games.

    • Mitchel Gordon Lichtman says:

      Correct on all accounts. There should be little difference between projecting good and bad framers so I don’t know why one is 8 runs and the other is 27.

      I think the takeaway from the numbers is that we have evidence (empirical data only gives us evidence and never “proof”) that the impact of framing from what we can project is likely large.

      Maybe I’ll throw some other years in it to increase the sample size. Not sure how far back the BP data goes.

    • Mitchel Gordon Lichtman says:

      Expanded the database to 2010-2017.

      Bad framers mean projection was -19.6 runs per 150 games (5700 PA caught). Their wOBA with was .3296 and without was .3256 for a difference of .004 or -20.3 runs.

      Good framers mean projections was +18.6. wOBA with was .3202 and without was .3249 for a difference of .0053 or +26.9 runs.

      The rest were -.2 runs projected, with wOBA of .3272 and without, .3274. or +1 run.

      • Guy Molyneux says:

        Good stuff. Makes more sense than the earlier/smaller study.

        Small question: how are you converting wOBA to runs? I thought the formula (for good framers) would be .0053/1.27 * 38 PA * 130 G = 20.6 runs.

        • Mitchel Gordon Lichtman says:

          I was using .87 as the multiplier rather than .79 which is 1/1.27. Thanks for all the good feedback, BTW.

  4. C Welsh says:

    Interesting analysis here but if you talk to MLB coaches, they are convinced that the best pitch framers are among the overall worst catches from a catch & throw standpoint. If you talk to MLB umpires, they will tell you the best pitch framers are the nicest guys. I have a hunch that there is more human factor involved here than meets the eye. Studies have shown a natural reluctance of umpires (pro and amateur) to make a call that actually changes the at bat, like ball four on a borderline 3-0 pitch and strike three on a close 0-2 pitch.

    • Mitchel Gordon Lichtman says:

      Interesting. Perhaps part of the framing “skill” is being such a nice guy that the umpire gives you the benefit of the doubt. As far as good framers being poor catch and throw guys, I have not looked at the correlation, but I doubt it. Also, throwing ability is not worth much anymore as no one steals much compared to the 80’s and 90’s and much of the success of base stealers is the responsibility of the pitcher.

  5. Servicetrf says:

    Where is admin?
    I’ts important.
    Thank.

  6. Ronald Myers says:

    I studied this last year before I blew up my old MLB handicapping model (now I just steal the numbers from BR). I used a gam model to estimate the probability a pitch was called a strike and compared actuals to expectation to measure framing ability. I had the best catchers in the league coming in at around +1.5% strikes called above average, based on their performance in the current season-to-date and the season before, after regressing everyone to a mean of 1000 called pitches of average production. I recall having most of the same names you list here at the top of my leaderboard as well. Flowers was a dominator.

    Putting the to-date stats into a regression model using the last five years of games, the model told me that for every 1% difference in the strikes called above average of the two starting catchers one would predict a difference in team run differential of about 0.1 runs per game. So for example if you had a catcher with a called strikes projection of 1% above average who caught 130 games in a season you would wind up with a 13 run impact over the course of a year. This seems to line up pretty well with what you show here, the best catchers worth 15-20 runs over the course of a season over average.

    The other item of note was that framing seemed to have fallen in effectiveness in the past couple of years compared to the period before. In earlier years (2011-2015), the coefficient was closer to 0.15 runs per game per 1%. If I had to guess this was caused by teams valuing framers more, meaning bad framers played less, meaning in later years what differences in framing there were were more likely to be caused by variance. There is also the possibility that umpires, knowing who the good framers were, started changing their calls in response.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s