Beating a Dead Horse, pt. 1

Last summer, I had a post that I described as a rant about OPS, and my problems with the stat. I got (relatively) a lot of feedback about that piece, and you can add that fact to other evidence that shows OPS is still a topic of interest among people who are interested in sabermetrics. If you go to the kinds of discussion forums in which baseball fans of a sabermetric bent, but not themselves hard-core sabermetricians post, various combinations of OBA and SLG, along with rate stats revolving around bases gained, remain a popular topic.

More than one of the respondents to my post felt that it was in essence beating a dead horse, because all of the serious sabermetricians of the world have moved on and found more important things to worry about, and that enough has already been written on the topic to satisfy the inquiries of the less-informed. I largely agree with this; however, I am fairly stubborn by nature, and I like to express myself on sabermetric topics myself, even if others have already covered the matter thoroughly. So despite the fact that I agree that OPS is a topic whose time has passed, I am going to write a little series on it and its cousins. Also, I feel that if people are going to use OPS, they might as well understand how it relates to run scoring rates.

I would also add that by posting about it here, I do not expect anyone to pay attention to it. If you feel that horse has been bloodied enough, then by all means, pay no attention to this post. If no one else ever reads this, that would be just fine with me.

The earlier linked post did not deal a lot with numbers, which these will have more of. I don’t want to rewrite the same thing over again, but I do want to summarize my main points:

1) OPS is a quick and dirty statistic, and it is alright for this purpose. I am not saying that you should never, ever look at OPS.

2) OPS is a statistic which is unitless. If you write the formula over a common denominator, you have ((H + W)*AB + TB*(AB + W))/(AB*(AB + W)), which is a bunch of gobbledy-gook.

3) OPS also does not come in an estimated unit. Runs Created is an example of a stat that does; (H+W)*TB/(AB+W) is an estimate of runs scored. Obviously the result is not actually runs scored, thus it is an estimated unit (see my previous post for more on this). OPS does not have an estimated unit. “The higher the OPS, the better” or “OPS is an estimate of a player’s offensive productivity” are both true statements, but they do not confer a unit upon the stat.

4) OBA is a fundamental baseball statistic. If OBA did not already exist, you would want to invent it. SLG is not a fundamental baseball statistic, because it does not measure any fundamental quantity--it measures “bases gained by the batter on hits”. This is a unit, and it is a useful thing to know, but it is not fundamental. If this does not convince you, ask yourself the question, “What does SLG represent?” Some people will be tempted to say “power”, but that’s obviously false since it includes singles. Some people will say “advancement of baserunners”, which can be true, but it is also true that it is not even close the being the most accurate estimate of advancement. In contrast, there is no definition for a statistic that would better define what OBA attempts to define than OBA itself, at least that can be derived from the official statistics.

5) Because of points 2, 3, and 4, there is nothing special about OPS. When Pete Palmer comes around and invents “OPS+”, it may make sense to claim that the name is a little misleading, but it doesn’t make any sense to complain that it causes distortions in measuring performance because it deviates from OPS. Of course, it makes even less sense to complain about it when it was Palmer who invented both OPS and OPS+.

With that out of the way, let’s talk math. Throughout this series, I have defined OBA as (H + W)/(AB + W); I did not mess with HB or SF, so keep that in mind. Also, I will be focusing on things on the team level, and running a lot of regressions. Then I will be testing the accuracy of the equations on the same sample from which I derived them. I recognize that this is not the best approach to take, but I think that if you focus on the relative accuracy of the formulas to each other rather than the absolute RMSE figures, you will not be mislead too far.

Let’s start with the premise that we have OBA and SLG data for a large group of teams, and we want to estimate a run scoring rate from them. In this case, our large sample will be all teams 1961-2002, except 1981 and 1994. We will also look at using relative OBA, SLG, or OPS to predict relative runs scored, as established by the composite average for the dataset. I am doing it that way because that is how OPS+ is expressed, and because using a constant league average will wash out the adjustments.

Let’s define aOBA as OBA/LgOBA, aSLG as SLG/LgSLG, aOPS as OPS/LgOPS (this is what I called “SOPS+” in my earlier post), aR/P as (R/PA)/Lg(R/PA), and aR/O as (R/O)/Lg(R/O). OPS+ is OBA/LgOBA + SLG/LgSLG - 1, which is the same as aOBA + aSLG - 1.

When we regress OPS to estimate runs, what run rate should we regress to, R/PA or R/O? We can all agree that the most important thing to know about a team’s offense is its R/O, so that would seem to be the right choice. While this does not transfer perfectly to individuals, it is still true that R/O is more telling for them than R/PA is, and R/O is generally fine to use as an individual rate stat.

So let’s look at the equations to predict aR/O from aOPS and OPS+ for the sample in question:

aR/O = 2.06(aOPS) - 1.06
aR/O = 1.06(OPS+) - .06

Here we can see that OPS has a 2:1 relationship to runs scored. If you are 5% better than the league average in OPS, you will be approximately 10% better in runs scored per out (and, by extension, in runs scored). On the other hand, OPS+ has an almost 1:1 relationship to runs scored.

The practical implications of this are that if you see a player listed with an OPS+ of 125, you can interpret this as “the player is estimated to create 25% more runs/out than the league average.” It is of course an estimate, and it may not be as accurate as other estimates, but it does scale properly.

You cannot do the same thing with aOPS (and since aOPS is simply OPS divided by a constant, the same goes for OPS as well). If you have a batter with an OPS of 900 in a league with an OPS of 750, saying that his aOPS is 120 means nothing other than that his OPS is 20% higher than the league average. It does not mean that he created 20% more runs--in fact, he created something close to 40% more runs.

As I mentioned in the older piece, people are conditioned to expect that when they see a stat called X+, it will be calculated as X/LgX. OPS+ breaks this convention, and it is curious why Pete Palmer chose to name it as he did (originally it was PRO+, but OPS itself was PRO, so it was the same situation). However, Pete’s choice to use OPS+ instead of aOPS was a good one, as it is more accurate and expressed in estimated units that have meaning.

Once we have the above equations, we can estimate team runs scored and see how accurate the estimates are (keeping in mind the caveats about using the regression on the sample it was derived from). We can find Runs = aR/O*Lg(R/O)*O. We know that for our sample, BA = .258, OBA = .324, SLG = .391, R/PA = .117, and R/O = .172. Plugging everything in, the RMSE against actual runs scored is 25.72 for the aOPS equation and 24.88 for the OPS+ equation.

What would happen if we tried to predict R/PA, rather than R/O? We would get these equations:

aR/P = 1.74(aOPS) - .74
aR/P = .90(OPS+) + .10

The estimate for team runs scored will be Runs = aR/P*Lg(R/PA)*PA. The RMSEs in this case are 23.76 for the aOPS equation and 23.64 for the OPS+ equation--over a run better than the R/O predictions. Why is this?

First, let me claim without presenting any evidence that most statistics do better at predicting team runs scored when estimating R/PA than it does when estimating R/O. To understand why this is, we need to remind ourselves about the relationship between R/PA and R/O. Assuming, as we have in this case, that the only outs are batting outs and that there are no ways to reach base that are not included in OBA, the relationship R/O = (R/PA)/(1 - OBA) holds. This is not an “estimate”; it is a demonstrable mathematical truth. As you can see, the On Base Average is key, since it is the complement of the rate at which outs are made. It is better to estimate R/PA from OBA and SLG, then convert it to R/O by dividing by (1 - OBA). Instead of doing a regression to try to incorporate the value of OBA, you are better off to use OBA directly for that purpose.

So it is in fact more accurate to estimate R/PA from OPS than it is to estimate R/O. However, if you agree with the premise that individual productivity should be measured in terms of runs/out, and you use OPS or OPS+ as your rate stat of choice, you are in essence locking yourself in to considering the less accurate R/O relationships.