Tripod: Appraised Runs

See the first paragraph of this post for an explanation of this series. The content of this article is also the topic of the more recent post here.

Mike Gimbel's stat, Run Production Average, is a very unique look at runs created, that, though published almost a decade ago, has gotten very little attention from other sabermetricians. RPA uses an initial RC formula based on what Gimbel calls Run Driving values, that underweight walks and overrate extra base hits. But Gimbel accounts for this with a set-up rating which evaluates the added impact of extra base hits in removing baserunners for the following batters. Gimbel's method has tested accuracy, with real teams, very similar to that of Runs Created, Base Runs, and Linear Weights. That it does not hold up at the extremes like Base Runs prevents it from being the best structure for RC we have, but it is an interesting alternative to RC. This is a compilation of some posts from FanHome on my knockoff of RPA, Appraised Runs. RPA uses some categories, like balks and wild pitches, that we do not have readily available. So I rigged up, following Gimbel's example, a similar formula using the typical categories. In doing so, I probably lost some of the true nature of Gimbel's creation. Gimbel obviously is the expert on his own stat, but hopefully AR is not too flawed to be useful in looking at the concept of RPA. This is Gimbel's RPA article.

Here is a compilation of posts from a thread on AR on FanHome. You can see the errors I made the first time, although I am not sure that the second version is much of an improvement.

Patriot - Dec 31, 2000

Mike Gimbel has a stat called Run Production Average. It is basically a R:PA method, but the way he gets runs is unlike any other construct I've seen. He starts by using a set of LW that reflect advancement values(or Run Driving), not total run scoring like most LW formulas. Than he adjusts half of this for the batter's Set Up ability, representing runners on base for the following batters. It is an interesting concept, but his formula has all sorts of variables that aren't available, like ROE, Balks, and WPs. So I tried to replicate his work.

As a starting point I used the Runs Produced formula laid out by Steve Mann in the 1994 Mann Fantasy Baseball Guide. The weights are a little high compared to other LW formulas, but oh well:

RP=.575S+.805D+1.035T+1.265HR+.345W-.115(AB-H)

Working with this formula, I saw that Gimbel's weights were similar to the (event value-walk value), and that Gimbel's walk value was similar to the (RP run value/2) The HR value seems to be kept.. This gives Run Driving, RD, as: .23S+.46D+.69T+1.265HR+.138W

The set-up values were similar to 1-Run Driving value, so the Set-Up Rating, which I'll call UP is (.77S+.54D+.31T-.265HR+.862W)/(AB-H) Gimbel used (AB+W) in the denominator, but outs works better.

Then Gimbel would take UP/LgUP*RD*.5+RD*.5, thus weighting half of the RD by the adjusted UP. But I found that UP correlated better with runs scored than RD, so we get:

AR = UP/LgUP*RD*.747+RD*.390

Where AR is Appraised Runs, the name I gave to this thing. LgUP can be constant @ .325 if you like it better.

Anyway, this had an AvgE in predicting team runs of 18.72, which is a little bit better than RC. So it appears as if Gimbel's work can be taken seriously as an alternative Run Production formula, like RC, LW, or BsR.

Please note that I am not endorsing this method. I'm just playing with it.

David Smyth - Jan 1, 2001

There is no doubt that Gimbel's method was ahead of its time, and that it can, properly updated, be as accurate as any other RC method. It has a unique advantage in being equally applicable to any entity (league, team, or individual), I think.

I support your effort to work on it a bit, and get rid of the odd categories he includes.

Basically what he was saying is that part of scoring is linear, and part is not. This is in between all-linear formulas such as XR, and all non-linear ones such as RC and BsR. The new RC is 89% linear and 11% non-linear, I recall. I'm not sure what the percentage is for RPA. As a team formula, it's certainly not perfect; at theoretical extremes it will break down. The only team formula I'm aware of which doesn't have that problem is BsR. There is probably a 'compromise' between RPA and BsR which would be great. IOW, you could probably use the fixed drive-in portion from RPA, and a modification of BsR for the non-linear part.

My position on these things is that both parts of the complete method--the run part and the win part--should be consistent with each other. For example, XR is linear and XW is non-linear. BsR is non-linear and BsW is linear. That bothers me, so I've chosen to go with a linear run estimator and BsW. Linear-linear. It's not so much a question of which is 'right'; it's a question of which frame of reference is preferable. If you want an individual frame of reference, go with RC or BsR, OWP, Off. W/L record, etc. If you want a team frame of reference, go with RPA or the new RC and XW. If you want a global (league or group of leagues) frame of reference, go with an XR-type formula and BsW. IMO, global has a simplicity and elegance which is unmatchable. Global would also include the Palmer/mgl LWts, using the -.30 type out value--another excellent choice.

There are also methods with enhanced accuracy such as Value Added Runs, and Base Production (Tuttle). These methods require tons of data. It's all a question of where to draw the line between accuracy, the amount of work, and what you're trying to measure. I tend to draw the line in favor of simplicity, because I've yet to be convinced that great complexity really pays off.

Patriot - Jan 2, 2001(clipped)

Anyway, since I have it here, this is the AR stolen base version:
RD = .23S+.46D+.69T+1.265HR+.138W+.092SB
UP = (.77S+.54D+.31T-.265HR+.862W+.092SB-.173CS)/(AB-H+CS)
AR = UP/LgUP*RD*.737+RD*.381
LgUP can be held constant @ .325

Patriot - Jun 13, 2001(clipped)

I have been working with this again, not because I endorse the construct or method but because the first time I did one amazingly crappy job.

For example, Ruth in 1920 has 205 RC, 191 BsR, and 167 RP. And 248 AR! Now, we don't know for sure how many runs Ruth would have created on his own, but anything that's 21% higher than RC makes me immediately suspicious.

Anyway, the problem comes from the UP term mostly. Gimbel used AB+W as the denominator and I used AB-H. Neither of us were right. Gimbel's method doesn't give enough penalty for outs, and mine overemphasizes out making to put too much emphasis on a high OBA. The solution is to subtract .115(that is the value from RP which I based everything on) times outs from the UP numerator because every out(or at least every third out) reduces the number of runners on base to zero.

Gimbel's RD values were also meant to estimate actual runs scored. So I applied a fudge factor to my RD to make it do the same. Anyway, this is the new Appraised Runs method:

RD = .262S+.523D+.785T+1.44HR+.157W
UP = (.77S+.54D+.31T-.265HR+.862W-.115(AB-H))/(AB+W)
AR = UP/AvgUP*RD*.5+RD*.5 AvgUP can be held @.145

This decreases the RMSE of the formula and also makes a better estimate IMO for extreme teams. Ruth now has 205 AR, more in line with the other estimators, although if you wanted to apply this method TT is the way to go.

The new AR stolen base version is:

RD = .262S+.523D+.785T+1.44HR+.157W+.079SB-.157CS

UP = (.77S+.54D+.31T-.265HR+.862W-.115(AB-H)+.262SB-CS)/(AB+W)

AR = UP/AvgUP*RD*.5+RD*.5 AvgUP can be held @ .140

Corrections - July 2002

I have had those Appraised Runs formulas for over a year now, and never bothered to check and see if they held up to the LW test. Here are the LW for AR from the +1 method for the long term ML stats(the display is S,D,T,HR,W,SB,CS,O): .52,.69,.86,1.28,.46,.19,-.57,-.106

You can see that we have some serious problems. The single, steal, and out are pegged pretty much perfectly. But extra base hits are definitely undervalued and the CS is wildly overvalued. So, I tried to revise the formula to improve these areas.

And I got nowhere. Eventually I scrapped everything I had, and went back to Gimbel's original values, and just corrected it for the fact that we didn't have some of his data. His RD portion worked fine, but I couldn't get his UP to work at all. Finally, I scrapped UP altogether. I decided instead to focus on the UP ratio(UP/AvgUP). This value is multiplied by half of the RD, and added to the other half of the RD to get AR. We'll call the UP/AvgUP ratio X. If you know RD, which I did based on Gimbel's work(I used his RD exactly except with a fudge factor to make it equate with runs scored, and dropping the events I didn't want/have), you have this equation:

R = RD*.5+RD*.5*X

Rearranging this equation to solve for X, you have:

X = R/(RD*.5)-1

So, with the actual X value for each team known, I set off to find a good way to estimate X. I didn't want to compare to the average anymore-if you think about it, it doesn't matter what the LgUP is, the number of baserunners on should depend only on the team's stats. So I did some regressions, found one that worked well, streamlined and edited the numbers, and wound up with these equations for AR:

RD1 = .289S+.408D+.697T+1.433HR+.164W

UP1 = (5.7S+8.6(D+T)+1.44HR+5W)/(AB+W)-.821

AR1 = UP*RD*.5 + RD*.5

RD2 = .288S+.407D+.694T+1.428HR+.164W+.099SB-.164CS

UP2 = (5.7S+8.6(D+T)+1.44HR+5W+1.5SB-3CS)/(AB+W)-.818

AR2 = UP*RD*.5 + RD*.5

These equations had RMSEs on the data for 1970-1989 of 22.64 and 21.79 respectively. For comparison, Basic RC was at 24.93 and Basic ERP was at 23.08, so the formulas are quite accurate when used for real teams. The linear values were: .51,.80,1.09,1.42,.35,.187,-.339,-.106

When applied to Babe Ruth, 1920, he had 205 AR, which is a reasonable value for an RC-like formula. Hopefully this new version of AR will turn out to be one that I can actually keep-maybe the third time is a charm.