Tripod: Runs Created

See the first paragraph of this post for an explanation of this series.

Bill James' Runs Created remains the most used run estimator, although there is no good reason for that being the case. It is odd that sabermetricians, generally a group inclined to fight preconceived notions and to not worship tradition for the heck of it, continue to use a method like RC.

Let me be clear: Bill James is my favorite author, he is the most influential and important(and one of the very best) sabermetricians of all-time. When he developed RC, it was just about as good as anything that anybody else had developed to estimate team runs scored and the thought process that went into developing it was great. But the field moves forward and it has left RC behind it.

I will now go into looking at the theory of RC and get back to explaining the alternative methods and the deficiencies of the method later. The basic structure of Runs Created is that runs are scored by first getting runners on base and then driving them in, all occurring within an opportunity space. Since getting runners on base and advancing them is an interactive process(if there is no one on base to drive in, all the advancement in the world will get you know where and getting runners on base but not driving them in will not score many runs either), the on base component and the advancement component are multiplied and divided by the opportunity component. A represents on base, B represents advancement, and C represents opportunity. The construct of RC is A*B/C.

No matter how many elements are introduced into the formula, it maintains the A*B/C structure. The first version of the formula, the basic version, is very straightforward. A = H+W, B = TB, and C = AB+W, or RC = (H+W)*TB/(AB+W). This simple formula is fairly accurate in predicting runs, with a RMSE in the neighborhood of 25(when I refer to accuracy right now I'm talking solely about predicting runs for normal major league teams).

The basic form of RC has several useful properties. The math simplifies so that it can be written as OBA*SLG*AB, which is also OBA*TB. Or if you define TB/PA as Total Base Average, you can write it as OBA*TBA*(AB+W). Also, RC/(AB-H), runs/out, is OBA*SLG/(1-BA).

The basic rate rewrite for RC is useful, (A/C)*(B/C)*C, which is easily seen to be A*B/C. If you call A/C modified OBA(MOBA) and B/C modified TBA(MTBA), you can write all versions of RC as MOBA*MTBA*C and as we will see, this will come in handy later.

James' next incarnation was to include SB and CS in the formula as they are fairly basic offensive stats. A became H+W-CS, B became TB+.7*SB, and C became AB+W+CS.

A couple years later (in the 1983 Abstract to be precise), James introduce an "advanced" version of the formula that included just about all of the official offensive statistics. This method was constructed using the same reasoning as the stolen base version. Baserunners lost are subtracted from the A factor, events like sacrifice flies that advance runners are credited in the B factor, and all plate appearances and extra outs consumed(like CS and DP) are counted as opportunity in the C factor.

A = H+W+HB-CS
B = TB+.65(SB+SH+SF)
C = AB+W+HB+SH+SF+CS+DP

In his 1984 book, though, James rolled out a new SB and technical version, citing their higher accuracy and structural problems in his previous formulas. The key structural problem was including outs like CS and DP in the C factor. This makes a CS too costly. As we will see later in calculating the linear weights, the value of a CS in the original SB version is -.475 runs(using the 1990 NL for the event frequencies). The revision cuts this to -.363 runs. That revision is:

A = H+W-CS
B = TB+.55*SB
C = AB+W

In addition to being more accurate and more logical, the new version is also simpler. The revision to the technical formula would stand as the state of RC for over ten years and was figured thusly:

A = H+W+HB-CS-DP
B = TB+.26(W+HB-IW)+.52(SB+SH+SF)
C = AB+W+HB+SH+SF

Additionally, walks are introduced into the B factor; obviously walks have advancement value, but including them in the basic version would have ruined the elegance of OBA*TB. With the added complexity of the new formula, James apparently saw no reason not to include walks in B.

The technical formula above is sometimes called TECH-1 because of a corresponding series of 14 technical RC formulas designed to give estimates for the majors since 1900.

Around 1997, James made additional changes to the formula, including strikeouts in the formula for the first time, introducing adjustments for performance in two "clutch" hitting situations, reconciling individual RC figures to equal team runs scored, and figuring individual RC within a "theoretical team" context. James also introduced 23 other formulas to cover all of major league history. The modern formula is also known as HDG-1(for Historical Data Group). The changes to the regular formula itself were quite minor and I will put them down without comment:

A = H+W+HB-CS-DP
B = TB+.24(W+HB-IW)+.5(SH+SF)+.62SB-.03K
C = AB+W+HB+SH+SF

Whether or not the clutch adjustments are appropriate is an ability v. value question. Value-wise, there is nothing wrong with taking clutch performance into account. James gives credit for hitting homers with men on base at a higher rate then for overall performance, and for batting average with runners in scoring position against overall batting average. The nature of these adjustments seems quite arbitrary to this observer--one run for each excess home run or hit. With all of the precision in the rest of the RC formula, hundredth place coefficients, you would think that there would be some more rigorous calculations to make the situational adjustments. These are added to the basic RC figure--except the basic RC no longer comes from A*B/C it comes from (A+2.4C)(B+3C)/(9C)-.9C(more on this in a moment). That figure is rounded to a whole number, the situational adjustments are added, then the figures for each hitter on the team are summed. This sum is divided into the team runs scored total to get the reconciliation factor, which is then multiplied by each individual's RC, which is once again rounded to a whole number to get the final Runs Created figure.

Quite a mouthful. Team reconciliation is another area that falls into the broad ability v. value decision. It is certainly appropriate in some cases and inappropriate in others. For Bill James' purpose of using the RC figures in a larger value method(Win Shares), in this observer's eyes they are perfectly appropriate. Whether they work or not is a question I'll touch on after explaining the theoretical team method.

The idea behind the theoretical team is to correct one of the most basic flaws of Runs Created, one that Bill James had noticed at least as early in 1985. In the context of introducing Paul Johnson's ERP, a linear method(although curiously it is an open question whether James noticed this at the time, as he railed against Pete Palmer's Batting Runs in the Historical Abstract), James wrote: "I've known for a little over a year that the runs created formula had a problem with players who combined high on-base percentages and high slugging percentages—-he is certainly correct about that—and at the time that I heard from him I was toying with options to correct these problems. The reasons that this happens is that the players' individual totals do not occur in an individual context...the increase in runs created that results from the extension of the one[on base or advancement ability] acting upon the extension of the other is not real; it is a flaw in the run created method, resulting from the player's offense being placed in an individual context."

The basic point is that RC is a method designed to estimate team runs scored. By putting a player's statistics in a method designed to estimate team runs scored, you are introducing problems. Each member of the team's offensive production interacts with the other eight players. But Jim Edmonds' offense does not interact with itself; it interacts with that of the entire team. A good offensive player like Edmonds, who has superior OBA and TBA, benefits by having them multiplied. But in actuality, his production should be considered within the context of the whole team. The team OBA with Edmonds added is much smaller then Edmonds' personal OBA, and the same for TBA.

So the solution(one which I am quite fond of and, following the lead of James, David Tate, Keith Woolner, and David Smyth among others have applied to Base Runs) that James uses is to add the player to a team of fairly average OBA and TBA, and to calculate the difference between the number of runs scored with the player and the runs scored without the player, and call this the player's Runs Created. This introduces the possibility of negative RC figures. This is one of those things that is difficult to explain but has some theoretical basis. Mathematically, negative RC must be possible in any linear run estimation method. It is beyond the scope of this review of Runs Created to get into this issue in depth.

The theoretical team is made up of eight players plus the player whose RC we are calculating. The A component of the team is (A+2.4C). This is the player's A, plus 2.4/8=.3 A/PA for the other players. Remember, A/PA is MOBA(and B/PA is MTBA). So the eight other players have a MOBA of .300. The B component of the team is (B+3C), so 3/8=.375 B/PA or a .375 MTBA for the remainder of the team. Each of the eight players has C number of plate appearances(or the player in question's actual PA), so the team has 9C plate appearances, and their RC estimate is (A+2.4C)(B+3C)/(9C). The team without the player has an A of 2.4C, a B of 3C, and a C of 8C, giving 2.4C*3C/8C=.9C runs created. Without adding the ninth player, the team will score .9C runs. So this is subtracted, and the difference is Runs Created.

James does not do this, but it is easy to change the subtracted value to give runs above average(just use nine players with MOBA .300 and MTBA .375, or adjust these values to the league or some other entity's norms, and then run them through the procedure above). Generally, we can write TT RC as:

(A+LgMOBA*C)(B+LgMTBA*C)/(9C)-LgMOBA*LgMTBA*8C(or 9C for average)

This step of the RC process is correct in my opinion, or at least justifiable. But one question that I do have for Mr. James is why always .300/.375? Why not have this value vary by the actual league averages, or some other criteria? It is true that slight changes in the range of major league MOBA and MTBA values will not have a large effect on the RC estimates, but if everything is going to be so precise, why not put precision in the TT step? If we are going to try to estimate how many runs Jim Edmonds created for the 2004 Cardinals, why not start the process by measuring how Jim Edmonds would effect a team with the exact offensive capabilities of the 2004 Cardinals? Then when you note the amount of precision(at least computationally if not logically) in Win Shares, you wonder even more. Sure, it is a small thing, but there are a lot of small things that are carefully corrected for in the Win Share method.

Just to illustrate the slight differences, let's take a player with a MOBA of .400 and a MTBA of .500 in 500 PA and calculate his TT RC in two situations. One is on the team James uses--.300/.375. His RC will be (.400*500+.300*500*8)(.500*500+.375*500*8)/(9*500)-.9*500, or 94.44. On a .350/.425 team(a large difference of 32% more runs/plate appearance), his RC figured analogously will be 98.33. A difference of less then four runs for a huge difference in teams. So while ignoring this probably does not cause any noticeable problems for either RC or WS estimates, it does seem a little inconsistent.

But while the TT procedure is mathematically correct and sabermetrically justifiable, it does not address the larger problem of RC construction. Neither does Bill's latest tweak to the formula, published in the 2005 Bill James Handbook. He cites declining accuracy of the original formula in the current high-home run era and proposes this new B factor:

B = 1.125S+1.69D+3.02T+3.73HR+.29(W-IW+HB)+.492(SB+SH+SF)-.04K

None of these changes corrects the most basic, most distorting flaw of Runs Created. That is its treatment of home runs. David Smyth developed Base Runs in the 1990s to correct this flaw. He actually tried to work with the RC form to develop BsR, but couldn't get it to work. So instead he came up with a different construct(A*B/(B+C)+D) that was still inspired by the idea of Runs Created. Once again, James' ideas have been an important building block for run estimation thinking. RC was fine in its time. But its accuracy has been surpassed and its structure has been improved upon.

A home run always produces at least one run, no matter what. In RC, a team with 1 HR and 100 outs will be projected to score 1*4/101 runs, a far cry from the one run that we know will score. And in an offensive context where no outs are made, all runners will eventually score, and each event, be it a walk, a single, a home run--any on base event at all--will be worth precisely one run. In a 1.000 OBA context, RC puts a HR at 1*4/1 = 4 runs. This flaw is painfully obvious at that kind of extreme point, but the distorting effects begin long before that. The end result is that RC is too optimistic for high OBA, high SLG teams and too pessimistic for low OBA, low SLG teams. The home run flaw is one of the reason why James proposed the new B factor in 2004--but that may cause more problems in other areas as we will see.

One way to evaluate Runs Created formulas is to see what kind of inherent linear weights they use. We know, based on empirical study, very good values for the linear weight of each offensive event. Using calculus, we can find precisely, for the statistics of any entity, the linear weights that any RC formula is using in that case. I'll skip the calculus, but for those who are interested, it involves partial derivatives.

LW = (C(Ab + Ba) - ABc)/C^2

Where A, B, and C are the total calculated A, B, and C factors for the entity in question, and a, b, and c are the coefficients for the event in question(single, walk, out, etc.) in the RC formula being used. This can be written as:

LW = (B/C)*a + (A/C)*b - (A/C)*(B/C)*c
= MTBA(a) + MOBA(b) - MOBA*MTBA*c

Take a team with a .350 MOBA and a .425 MTBA. For the basic RC formula, the coefficients for a single in the formula are a = 1, b = 1, c = 1, so the linear weight of a single is .425*1 + .350*1 - .425*.350*1 = .626 runs. Or a batting out, which is a = 0, b = 0, c = 1 is worth -.425*.350*1 = -.149 runs.

Let's use this approach with a fairly typical league(the 1990 NL) to generate the Linear Weight values given by three different RC constructs: basic, TECH-1, and the 2004 update.

Single: .558, .564, .598
Double: .879, .855, .763
Triple: 1.199, 1.146, 1.150
Home Run: 1.520, 1.437, 1.356
Walk/Hit Batter: .238, .348, .355
Intentional Walk: N/A, .273, .271
Steal: N/A, .151, .143
Caught Stealing: N/A, -.384, -.382
Sacrifice Hit: N/A, .039, .032
Sacrifice Fly: N/A, .039, .032
Double Play: N/A, -.384, -.382
Batting Out(AB-H): -.112, -.112, N/A
In Play Out(AB-H-K): N/A, N/A, -.111
Strikeout: N/A, N/A, -.123

Comparing these values to empirical LW formulas and other good linear formulas like ERP, we see, starting with the Basic version, that all of the hits are overemphasized while walks are severely underemphasized. The TECH-1 version brings the values of all hit types in line(EXCEPT singles), and fixes the walk problems. The values generated by TECH-1, with the glaring exception of the single, really aren't that bad. However, the 2004 version grossly understates the impact of extra base hits. I don't doubt James claim that it gives a lower RMSE for normal major league teams then the previous versions, but theoretically, it is a step backwards in my opinion.

You can use these linear values as a traditional linear weight equation if you want, but they are at odds in many cases with empirical weights and those generated through a similar process by BsR. One good thing is that Theoretical Team RC is equal to 1/9 times traditional RC plus 8/9 of linear RC. Traditional RC is the classic A*B/C construct, whereas the linear RC must be appropriate for the reference team used in the TT formula.