Run Estimation Stuff, pt. 3

Base Runs and Runs Created have many similarities that are easily apparent. Each has an A factor that represents baserunners and a B factor that represents advancement. Each has a C factor that in some way reflects what Bill James called “opportunity”, but James used plate appearances while Smyth uses outs.

The two fundamental differences between the models are 1) their treatment of the home run and 2) how they estimate score rate (the proportion of baserunners that score). BsR recognizes that a home run always results in at least one run, and thus splits them off into a D factor. This modification not only credits each homer with creating at least one run, but it alters the BsR A factor. Since the run scored by the batter who hits the longball has already been accounted for, home runs are removed from baserunners.

Then, since BsR estimates score rate as a proportion (B/(B + C)) rather than as a ratio (B/C), it adheres to the obvious constraint that “runs scored by baserunners <= baserunners”, which RC does not adhere to. If B>C, than RC predicts that more runs will score than their were runners on base.

This is not to say that BsR is a perfect model, because of course it is not. For example, BsR does not adhere to the rule that you can leave a maximum of three on base per inning. In some extreme situations it could return a negative run estimate (although the simple versions that do not subtract anything from the B factor have a lower bound of zero). But these things are also true of Runs Created.

Another flaw of BsR is that in some extreme cases, the intrinsic weight of a triple is greater than that of a home run. This is obviously absurd, although Tango Tiger has developed a workaround for this situation. However, while RC does not have that flaw, it is not necessarily a point in RC’s favor. It’s better to have the triple be weighted higher than a homer, but both weighted somewhat in the vicinity of their actual value than to have both wildly overvalued. That is not to say the triple problem is a good thing for BsR; it’s a clear shortcoming. It just doesn’t hurt its standing against RC, as any time BsR says that a triple is worth more runs than a home run, RC will be telling you that they are both are much more valuable than they actually are.

The only claim here is that BsR is a better model than RC, in almost all regards. You might be able to find an extreme situation that a particular RC version will handle better than a particular BsR version, but the vast majority of cases go to BsR and the logic of the model is a clear advantage for BsR. There is also no fundamental rule of baseball (such as the maximum three runners stranded per inning) that RC adheres to when BsR does not, at least not that I am aware of. Base Runs is the best simple, dynamic model for run scoring that we have. Of course, I grant that simulations and Markov models can be superior if designed properly. However, those take massive spreadsheets or stand-alone programs to implement. As far as formulas that can be implemented quickly on a spreadsheet (and explained in English to those unfamiliar with linear algebra or computer programming) go, Base Runs is the best model.

Anyway, what I want to do here is to examine RC and BsR head-to-head in the manner that Tango suggested here. First, let’s look at Bill James’ latest RC version, remembering that RC is always A*B/C:

A = H + W + HB - DP - CS
B = 1.125S + 1.69D + 3.02T + 3.73HR + .29(W - IW + HB) + .492(SB + SH + SF) - .04K
C = AB + W + HB + SH + SF

As will be demonstrated in the next segment, this formula is more accurate than any of the BsR versions we came up with when used on the 1990-2005 team data. I don’t put a lot of stock in this, but I feel obliged to mention it in the interests of full disclosure (lest I be accused of being a BsR “fanboy” as I have been in the past, by one of the idiots who posts at a certain site that sometimes links to stuff like this, although thankfully not as much as they did in the past).

Let’s take a look at the intrinsic weights generated by this formula for the 1960-2004 period, compared to the target weights from Ruane’s research:



I will let that stand without comment; if the arguments of others as well as my own feeble efforts have not yet convinced you of the importance of approximating empirical linear weight values, nothing I can say here will help. If you hold that view, this table should be very revealing with respect to the Runs Created model, or at the very least the manifestation of it that is currently in use.

Just as we did for BsR, we can find the B coefficients for RC necessary to match our target linear weights. They are:

B = .634S + 1.640D + 2.594T + 3.844HR + .105(W - IW) - .337IW + .189HB + .652SB + .416CS + .157(AB - H - K - DP) + .025K + .146DP + .629SH + .911SF

You can see that these weights are superficially similar to the BsR B weights; of course the home run is a big exception, and there certainly are some discrepancies, but they are in the same ballpark.

We can also “dumb down” Base Runs in order to compare it to the Runs Created model. David Smyth has recently suggested this type of approach as the most basic Base Runs model. What we will do is not give the home run the special treatment of being split off from baserunners and always credited with creating at least one run.

This is not to say that the home run should not be treated as BsR does. The home run, as opposed to other (or possible) categories that always indicate a run scored, like the sacrifice fly, is unique not as a scoring category but as an event defined by the rules of the game. A sacrifice fly, or a RBI groundout, is just an accounting category--it's a way of more specifically describing the outcome of a given event that could have also been classified an out, a groundout, etc. The home run is not just an accounting category; it is an event set aside in the rules of baseball that entitles the batter and any runner to score automatically without the threat of being put out. While there are other events that allow one to score without risk (a bases loaded walk, for instance), the home run is the only event that can do so independently of what precedes or follows it in an inning.

That digression aside, a simple version of BsR suggested by Smyth is:
A = H + W
B = (2*TB - H + HR)*.75
C = AB - H
BsR = A*B/(B + C)

Another version he offered is B = (2*TB - H)*.79.

We could optimize this equation to match target linear weights in the same way that we did in the previous installment, but that would be overkill. It defeats the point of a “simple” formula to include small fractional weights for walks and outs in the B factor.

The second version in particular does a decent job of estimating linear weight values: .52, .83, 1.14, 1.45, .36, -.11 for 1960-2004 (S, D, T, HR, W, O). To me, this in an illustration of the fact that BsR is a step forward over RC in its approach to estimating score rate, without even considering its treatment of the home run.

We have looked at how we can move RC closer to BsR by switching to a proportional estimate of score rate. We could also move RC closer to BsR by retaining a ratio estimate of score rate, but instead dealing with the home runs specially. This would give us a model along the lines of Runs = A*B/C + D.

In fact, this is a construct that has been used for a run estimator, namely Eric Van’s Contextual Runs. Here we will make a generic version to would match our target linear weights:

A = H + W - HR + HB
B = .349S + 1.018D + 1.652T + .915HR - .004(W - IW) - .297IW + .053HB + .434SB - .598CS + .153(AB - H - K - DP) + .065K - .730DP + .467SH + .654SF
C = AB - H + SH + SF

Van uses an initial baserunner, full out construct, but I just want to illustrate how the coefficients directly compare to the RC and BsR versions here. Obviously I am not attempting to supersede his definition of his own estimator; consider this a generic, nameless A*B/C + D model. However, it would be disingenuous to present such a model without acknowledging Van's work.

The bottom line is that all of these models are inferior to the full Base Runs model of A*B/(B + C) + D. Is it possible that someone could develop a better estimator of score rate? Sure, it’s possible. However, it won’t be easy to find, and it is almost certain that it would be a lot more complex than B/(B + C).

When you look at the measures to which Bill James has resorted to prop up the accuracy of RC, you have to ask why not just take one more step and take HR out of A? That is the only difference between the two formulas at this point, once James abandoned using TB in the B factor (instead considering S, D, T, and HR separately) and went to two decimal place weights.

The claim that RC is more simple than BsR, computationally, is now hinging just a few additional mathematical operations (subtracting HR from A, adding them in D, adding B and C, and depending on which BsR version you use, adding and multiplying by a coefficient for a few miscellaneous events in B). If it is really simplicity that you crave, a simple linear weights formula can’t be beat. If you are willing to put up with the “complexity” of Runs Created, it seems very odd to me that you would find Base Runs to be a bridge too far.

I do not really care if Bill James wants to continue to use RC; after all, he developed it, and it has influenced the superior estimators that followed. I imagine that it’s hard to let go of your own creation that was once considered the gold standard (I wouldn’t know because I am no Bill James, not by a longshot, and have never been in that position). What I fail to understand, though, is why anyone else continues to use Runs Created (other than its most simple incarnations for quick and dirty estimates in the same vein as OPS and its cousins).