On Run Distributions, pt. 5: Estimating Variance

So far, I’ve demonstrated only that the zero-modified negative binomial distribution (which I’m referring to as the Enby distribution for the sake of my sanity) can provide a decent fit to actual scoring patterns when the sample variance of runs per game is known. In order for this approach to have value with teams for which we don’t know the variance (and that’s the whole point of the exercise--estimating what the distribution should be based on the average R/G rather than simply regurgitating a sample distribution), we need a way to approximate the variance (As a tease, some work being done independently should allow for a better approximation of the variance than what I've come up with here. At some point in the future, I will incorporate that method into my methodology).

Allow me to issue this disclaimer up front: the formula I’m proposing here is woefully inadequate. If the Enby distribution is to have any real value to sabermetricians, someone will have to come along and clean this part up (while I’m making a wish list, a better way of adjusting the parameters to return the correct average R/G after the zero modification would be nice too).

The scatterplot below shows the variance of R/G plotted against average R/G for each major league team, 1981-1996. You can see there is clearly a positive correlation between the two: the higher the average, the higher the variance:



There is no clear pattern that will help me in attempting to develop a function to estimate variance from the mean. In fact, if you plot the ratio of variance to mean against the mean, you get a big clump:



There is a positive correlation (r = +.27) between the ratio of variance/mean and the mean. However, using a regression equation to describe the relationship between the mean and the variance introduces the problem of illogical results at the extremes.
For example, a linear regression yields the following equation for variance as a function of mean:

Variance = 2.637*mean - 2.670

For any team scoring less than 1.013 R/G, this formula will predict a negative variance, which is obviously impossible. Granted, I’m under no delusions that the final method I offer will be of any use outside of the normal scoring range of major league teams, but I cringe at laying out a method that obviously cannot work at such extremes.

Another option is to estimate the variance ratio as a function of the mean. The benefit here is that this constrains the estimated variance; if a team scores zero runs, the estimated variance will be zero. The estimated variance can never be negative:

Variance/mean = .1345*mean + 1.430
So Variance = mean*(.1345*mean + 1.430) = 1.430*mean + .1345*mean^2

Neither of these equations comes close to matching the aggregate results at 4.46 R/G, which is troublesome. But that value is itself an amalgamation of hundreds of individual team seasons, each recorded by teams that theoretically follow their own distributions of runs scored, and so I’m not sure a failure to match the result should be a death knell. Further complicating matters is that the estimate of variance will only be used to fit initial r and B parameters, with r then being varied to ensure that the mean of the distribution equals the actual mean.

Let me try using the second formula and run through the whole process to generate the expected run distribution for the aggregate 4.4558 R/G:

1. Estimate the variance of R/G from the mean:

Variance = 1.43*mean + .1345*mean^2 = 1.43*4.4558 + .1345*4.4558^2 = 9.042

2. Fit the parameters of a Enby distribution assuming no zero-modification:

B = variance/mean - 1 = 9.042/4.4558 - 1 = 1.029
r = mean/B = 4.4558/1.029 = 4.33

3. Estimate the parameter z (variable RI = (R/G)/9 = 4.4558/9 = .4951):

z = (RI/(RI + .767*RI^2))^9 = (.4951/(.4951 + .767*.4951^2))^9 = .0552

4. Calculate the probability for k runs (where k >=1) using the zero-modified formula, then find a new value for r that sets the mean of the zero-modified distribution equal to the desired mean (4.4558).

This step can only be done via some kind of computer algorithm; I used trial and error and get r = 4.364.

For the first time in the series, we have a version of the Enby distribution that is not blatantly cheating compared to other methods: I am not treating the variance or the probability of being shutout as known values, but rather am estimating them. We’re still not flying completely solo--the formula for estimating variance from mean was based on the same data that’s been aggregated, but we’re getting closer.

What would the estimates look like if we tried to apply them to a really extreme team? I do not expect the Enby to perform well at all, but it’s worth checking to confirm. I’ll try to estimate the run distribution first for a team that averages 1.5 R/G, then for a team that averages 10. I didn’t select these numbers for any particular reason other than that they are extreme, without being so crazy as to be beyond the range that anyone could possibly care (for practical rather than theoretical reasons) if the method worked for that point.

Obviously we don’t have an actual run distribution to compare to, but I’ll compare to the Tango-Ben distribution which, while also untested at these extremes, would be a better bet. First, the 1.5 R/G team (parameters z = .3387, B = .6318, r = 2.5706):



And the 10 R/G team (parameters z = .0039, B = 1.775, r = 5.638):



I was honestly surprised when I saw how closely Enby tracked Tango-Ben. Pleasantly so, of course, but surprised nonetheless.

I’ve said this before in different ways, but it’s worth repeating: my experience working with probability distributions is either theoretical (I took a Stats course once where I rarely wrote a real number down for the entire class) or working with fixed distributions (i.e. you are given a Poisson distribution with parameter h = 2.05. What is the probability of three or fewer occurrences?) I have no practical knowledge of how to best fit a zero-modified distribution to sample data, and thus my work product here will be of little value. Hopefully I’ve provided enough promising results to encourage those of you who are skilled at this type of problem to consider the negative binomial as a model for runs per game.

On Run Distributions, pt. 4: Tango Distribution

At this point in the series, I need to take a detour from discussing run per game distributions and get a little more fundamental--runs per inning distributions. Thankfully, this will not involve any painful attempts to develop a model, as a good one already exists: the Tango Distribution.

In order to implement the zero-modified negative binomial model I’ve discussed, I need a way to estimate the probability of zero runs being scored in a game. While converting a runs per inning distribution to a runs per game distribution involves a mess of combination math, estimating the probability of a shutout from an inning distribution is much simpler. Theoretically, the probability of a shutout should be the probability of failing to score in a single inning to the ninth power (that is, the probability of failing to score in nine consecutive innings).

Of course, this requires making some assumptions, most notably that there are nine innings per game (in fact, if both teams fail to score, then in at least 50% of such games a team will fail to score in the first nine innings and still not get whitewashed for the game) and that the run distribution in each inning is independent.

The Tango Distribution uses Runs/Inning (RI) as one parameter, and a control value c as the other. c is set equal to .767 if looking at one team independently and .852 for teams in a head-to-head game. Then:

a = c*RI^2
f(0) = RI/(RI + a)
d = 1 - c*f(0)
f(1) = (1 - f(0))*(1 - d)
f(k) = f(k - 1)*d for k >= 2

I’ll assume that RI = (R/G)/9, so if we have a team that averages 4.4558 R/G, their RI = .4951.

a = .767*.4951^2 = .188
f(0) = .4951/(.4951 + .188) = .7248
probability(0 runs in game) = p(0) = f(0)^9 = .7248^9 = .0552

Our empirical probability of a shutout for teams averaging 4.4558 R/G was .0578.

We will use the Tango Distribution to estimate the parameter z for the Enby distribution. I’ll restate the formula here, omitting the superfluous formulas for the probabilities of scoring in the inning:

z = (RI/(RI + c*RI^2))^9 where default is c = .767

As I mentioned earlier, Ben Vollmayr-Lee demonstrated how to use the Tango Distribution to estimate the per game run distribution, assuming that runs are independent across innings. His explanation is available here (zip link).

This provides us an alternative means of estimating the runs per game distribution. It’s really not important to me at this whether the converted Tango Distribution or the Enby distribution does better, as the latter is easier to implement and I’ve surely not been able to optimize it. In any event, the Tango Distribution is a very valuable tool to have as well, certainly for estimating the runs per inning distribution but also for the runs per game distribution.

I have a spreadsheet which implements Vollmayr-Lee’s approach, and used it to generate estimated runs per game distributions for the three samples I’ve referenced throughout the series. I’ll give you the graphs here as I did for the Enby distribution. First, for all teams 1981-1996:



On the eyeball test, it certainly does seem as if the Tango-Ben method is a better fit than the Enby distribution, especially when we consider that the latter has a number of artificial advantages in this test. Here are the 25 lowest scoring teams:



This doesn’t look as good, but remember the sample size caveat and it still looks a little better than the Enby distribution. For the 25 highest scoring teams:



Pretty good, again appearing to be a better fit than Enby. Again, I want to stress that I don’t think Enby is any kind of silver bullet; its potential value is due to its being a distribution that returns a direct result in terms of runs per game and is a well-known distribution.

On Run Distributions, pt. 3: Zero Modification

It appears that the negative binomial distribution provides a decent starting point for estimating the per game run distribution for typical major league teams. However, even a quick glance at the results suggests that it underestimates the probability of a team being shutout, and overestimates the probability of scoring a moderate number of runs (2-4).

While what I’m going to propose in this piece may appear to just be a crude fudge factor, but in fact distributions with modifications at zero are in use (Google “zero modified distributions” or something similar if you have to see for yourself). Modifying the negative binomial distribution to allow more probability to be placed at the point 0 runs will obviously improve our estimate of that probability, but should also result in more accurate estimates across the board.

I’ll let the variable a represent the probability of zero using the negative binomial distribution without modification, while z will represent the new probability that we wish to place on the zero point (in effect, it is now a third parameter along with r and B). So:

a = (1 + B)^(-r)

To calculate the probability of j runs (where j >=1) in this new distribution, all we do is assume that the share of non-zero probability assigned to each j stays the same--the only difference is that there is now a different amount of non-zero probability available. Recall that:

q(k) = (r)(r + 1)(r + 2)(r + 3)...(r + k - 1)*B^k/(k!*(1 + B)^(r + k)) for k >=1

So:

p(k) = z for k = 0
p(k) = (1 - z)*q(k)/(1 - a) for k >=1

In order to demonstrate this, I’m going to look at the three groupings of 1981-1996 teams I used last time: all teams and the 25 lowest and highest scoring teams. In this case I will set z equal to the actual observed frequency of scoring zero runs. Obviously, this is cheating if we are trying to establish the accuracy of the model and not helpful in cases in which we don’t know the empirical probability of zero runs. Again, though, I’m doing it this way as a quick check of whether or not we’re wasting our time. If you can’t make material gains in the accuracy with a known value, good luck trying to do so while estimating that value.

For the average teams, z = .0578 and a was previously calculated to be .0495. We previously estimated that the probability of scoring 3 runs was .1468, so the new estimated probability of scoring 3 runs is:

p(3) = (1 - .0578)*.1468/(1 - .0495) = .1455

Here’s a chart:



The model is still guessing too high on the probability of scoring a moderate number of runs, but at least now we no longer have one point which is an obvious problem. Going back to the X^2 statistic I discussed previously, the X^2 (excluding zero runs, since we’ve set this up in such a manner that the error at the point is zero by definition) for the zero-modified model is 63 compared to 68 for the unmodified distribution. That’s a slight gain in accuracy; in order for it to be worth the extra effort, the gain in better estimating shutouts has to be worth the extra work. For the lowest and highest scoring teams, the gains are similar.

Unfortunately, the zero-modification has a side effect which reduces the accuracy and (more importantly) destroys a fundamental property that our run distribution model must adhere to. I’d posit there are two such fundamental properties--obviously there are other aspects to consider, but these two are non-negotiable:

1. The model must return a total probability of 1. That is, p(0) + p(1) + p(2) + … + p (infinity) = 1. This property is not affected by the zero-modification.

2. The model must return the correct average runs scored. That is, 0*p(0) + 1*p(1) + 2*p(2) + 3*p(3) + ... = R/G. This property is not preserved by the zero-modified distribution.

Remember that when I fit the parameters initially, I used the method of moments, which relies on the mean of the distribution. The mean of a negative binomial distribution is r*B, and by using the method of moments I ensured that the mean and variance of the fitted distribution would equal the mean and variance of the sample.

In the literature on zero-modified distributions (see here for an example) one can find formulas for the mean and variance of the modified distributions. Ideally, we could just use the method of moments in conjunction with these formulas and fit the distributions properly. Unfortunately, that’s not possible:

mean = (1 - z)*r*B/(1 - (1 + B)^(-r))
variance = (1 - z)*rB*((1 + B) - (1 + B + r*B)*(1 + B)^(-r))/((1 - (1 + B)^(-r))^2 + z*(1 - z)*((1 - z)*r*B/(1 - (1 + B)^(-r)))^2

In fact, even the formula for the mean makes it impossible to algebraically force the mean of the modified distribution to equal the sample mean. Even if you hold either r or B constant, there’s no simple algebraic solution.

This complicates the decision between the modified and standard distribution considerably; if we use the standard distribution, we know that we’ll underestimate the probability of a shutout for all manner of teams. If we use the modified distribution, we won’t retain the same mean and variance as observed in the sample. What to do?

From my limited testing, it appears as if the least disruptive approach is to set the value of r such that the mean is equal to the sample mean. By least disruptive, I mean that the variance of the modeled distribution is closer to the variance of the sample when r is changed rather than B. For the sample of all teams, the revised value of r necessary is approximately 4.09, which results in a variance of 9.5663 (the sample variance is 9.3603). Changing B would result in a new B value of approximately 1.112 with a variance of 9.61.

Ideally both r and B would be selected in such a way that would maximize accuracy, but that is beyond my capabilities given the complexity of the model and three parameters in play. So I’ll have to settle for a crude solution, which will be to vary r.

Since there is no algebraic solution to calculate the necessary value of r, this will require an ad hoc approach. For those that are interested, I have gone through the trouble of figuring the parameters for common R/G levels and will present those at the end of this series.

Here is a graph of the modified distribution:



We’re still underestimating the likelihood of 2-4 runs and overestimating the likelihood of 5-7 runs, but I’m unaware of any other discrete distribution that comes close to this level of accuracy.

I’m getting a little tired of writing "negative binomial" and "modified", and so I will slap a cutesy name on this method. I’m not going to name it after myself, of course (for the record, I didn’t push the use of “Pythagenpat”, and this is only a new application of a common probability distribution discovered centuries ago). Until someone else decides they want to use this method and give it a better name, I’ll go with the Enby Distribution. Abbreviate “negative binomial” to “NB” and try spelling it as a word, and you get Enby.

On Run Distributions, pt. 2: Negative Binomial

There does exist a common discrete distribution that can be useful in modeling the per game run distribution. It doesn’t necessarily seem like it should work, since the questions that the distribution is typically employed to answer (at least in the sort of non-expert presentations most people are exposed to) doesn’t have an obvious connection with expected runs scored in baseball. Is it a perfect fit? Of course not; there are no perfect models anyway, but even granting that, it has flaws. Would it work in extreme circumstances? I don’t really know; over the next several articles I’ll take a stab at answering these questions, but obviously I have my limits as an analyst.

I did a couple of internet searches to try to see if there were any published papers or articles using the Negative Binomial distribution to model runs scored. There were a few that seemed to mention it, and there was an article about cricket scoring, but I didn’t see anything using it for the exact purpose I am. That does not mean none exist.

In any event, I know that I am not the first person to use the distribution to model runs per game, because I was sent an unpublished paper about it several years ago by Phil Melita. Melita sent it to me since he had cited Pythagenpat in the article, and we traded a couple emails about it. It was never published in By the Numbers, though, and I’m not sure why. In any event, Melita didn’t present his work in the same manner that I am here, but it was clearly along the same lines and I can take no credit for any originality here. I hadn’t thought about it in a few years, but as soon as I tried the negative binomial, I remembered Melita’s article.

To be honest, the negative binomial is not one of my favorite distributions. From my educational experiences, it is associated with two things in my mind: 1) the nice, simple Geometric distribution (a special case of negative binomial) run amuck and 2) Poisson-Gamma compound processes. It does have other obvious baseball uses, but it’s just not as intuitive to me as Poisson, Geometric, or Binomial.

The way the negative binomial distribution is usually presented masks its ability to deal with fractional parameters, hidden by the use of factorials and the aversion of everyone but mathematicians to gamma functions. And the problems to which the negative binomial is usually applied are along the lines of “What is the probability of 5 successes before the eighth failure?”, which further reduces the likelihood that people will consider it as a model for run scoring. However, the pmf for the negative binomial can be written thusly:

p(k) = (1 + B)^(-r) for k = 0
p(k) = (r)(r + 1)(r + 2)(r + 3)…(r + k - 1)*B^k/(k!*(1 + B)^(r + k)) for k >=1

where r and B are the parameters of the distribution and k is the point at which we are calculating the probability.

Of course, in order to actually use this or any other distribution, we need to be able to define the parameters. Everything that follows in this series of posts needs to be prefaced by this caveat--I am not a statistician or a probability expert. Compared to the average person, I have a lot of knowledge through my own curiosity and my studies/profession. Compared to the average person who dabbles in sabermetrics, though, my knowledge doesn’t stand out, and obviously I am a kindergartner compared to the experts in these fields.

So here I will take the easy step of fitting the parameters of this distribution using the method of moments. There may well be a better, more precise method out there (maximum likelihood estimation always comes to mind, but is a nightmare for negative binomial). In any event, the most important simple properties of any distribution are its mean and variance. For negative binomial, these are computed as follows:

mean (u) = r*B
variance(v) = r*B*(1 + B)

The method of moments uses the first two central moments (mean and variance to estimate the parameters). If we take the ratio of v to u, we can estimate B:

v/u = r*B*(1 + B)/(r*B) = 1 + B
Ergo B =v/u - 1 and r = u/B = u^2/(v - u).

For 1981-96, the average runs scored per game was 4.4558 with a variance of 9.3603, which leads to an estimate that B = 9.3603/4.4558 - 1 = 1.1007 and r = 4.4558/1.1007 = 4.0480.

Given those parameters, here are the estimates for the first few run totals:

p(0 runs) = (1 + B)^(-r) = (1 + 1.1007)^(-4.048) = .0496
p(1 run) = r*B^k/(k!*(1 + B)^(r + k)) = 4.048*1.1007/(1!*(1 + 1.1007)^(4.408 + 1)) = .1051
p(2 runs) = r*(r + 1)*B^k/(k!*(1 + B)^(r + k)) = 4.408*(4.408 + 1)*1.1007^2/(2!*(1 + 1.1007)^(4.408 + 2)) = .1390
p(3 runs) = r*(r + 1)*(r + 2)*B^k/(k!*(1 + B)^(r + k)) = 4.408*(4.408 + 1)*(4.408 + 2)*1.1007^3/(3!*(1 + 1.1007)^(4.408 + 3)) = .1468

The r*(r + 1)*(r + 2)*(r + 3)*(r + 4)*... formula gets annoying, but with a spreadsheet it is easy to implement. Here is the complete runout (to 16 runs), along with a graph comparing the estimate probabilities to the observed frequencies:





When I look at that graph, I see a pretty decent fit, one that makes me believe that the negative binomial distribution may have some usefulness in estimating run distributions. Is it perfect? No; as you can see, it is noticeably low on shutouts (an issue that I will go into detail about in the next post), and it’s too high for some of the other common scores (2-4). Using the X^2 test statistic described in the previous post, this scores a 176, which is easily better than the other crude models I considered.

Of course, I’ve only dealt with scoring at normal levels. In order to estimate run distributions for other scoring levels, we’ll have to change the parameters. The mean number of runs per game is always known, but the variance is a quantity that we can only know if we have prior knowledge of the empirical run distribution. Since both r and B are dependent on both the mean and variance, they will be constantly changing when the mean and variance of the distribution we are fitting to changes.

Before worrying about how to tackle that problem, we first need to make sure it would be worth attempting to adapt to non-average situations. I will consider the top and bottom 5% of teams by R/G for the data in Davenport’s spreadsheet--the 25 lowest and highest scoring teams of 1981-2006. The bottom 25 teams averaged 3.49 R/G with a variance of 6.6206, which produces estimates of B = .897 and r = 3.891. The results:



This doesn’t look as nice, but it still appears as if the model does a decent job. Remember that the sample size is not that large (25 teams is a smaller sample than a full major league season), so there’s the possibility of some random fluctuation. Again, the point here is not to perform a rigorous test, but rather to make sure the model isn’t completely out in left field.

The 25 highest scoring teams averaged 5.7 R/G with a variance of 12.4 for B = 1.175 and r = 4.849:



Not bad. Next time, I’ll discuss a workaround for the underestimation at zero runs (albeit one that introduces a new stumbling block).

On Run Distributions, pt.1: Introduction

Let’s suppose that you were going to conjure up a model that fit the distribution of team runs per game. What properties would you like it to exhibit (ignoring for the moment whether a model with such properties is actually feasible)?

I would suggest the following:

1. It should be discrete. Runs scored in a game is a discrete quantity. There’s no such thing as a fractional number of runs scored. There’s nothing wrong with suggesting that a certain combination of events will generally create, say, 4.35 runs, but that 4.35 runs represents a weighted average of all the times that combination produces 0, 1, 2, 3, 4, 5, …, runs. The distribution function should reflect those probabilities.

2. It should be a standard, well-known probability distribution.

3. It should be accurate, and adaptable across a wide range of scoring contexts. This is the most important, but also should go without saying.

4. If possible, it should be simple. I am not suggesting trading accuracy or theoretical purity for simplicity, but this is a wish list.

Again, that is a wish list. In the real world, you have to model reality. That might mean accepting a continuous distribution as a model of a discrete function, or having a method that is useful in normal contexts but has some structural deficiencies that limits its utility at the extremes.

If we want to find a distribution that meets some of the criteria on the list, we have to start with discrete distributions. I would assume that the first such distribution that comes to mind for most people with any background in probability or statistics is the Poisson distribution. The Poisson distribution is simple, adaptable, and has a cool name and a cool Greek letter for a parameter. Unfortunately, it doesn’t work as a model of runs scored in baseball, despite some noble efforts to make it so.

Poisson does work fairly well (at least to the extent of my knowledge, which is limited) as a model of soccer and hockey scoring. Given that it is often used for situations in which events are expected to occur over a certain period of time (or area and other physical intervals), it makes intuitive sense that it wouldn’t fit baseball run scoring, since time is not a factor. A Poisson process should be able to be extended to time periods other than the basis--that is, if your mean is 3 goals per game for hockey, you should be able to use the distribution to estimate goals per period (with mean 1) or goals per ten games (with mean 30), or goals per 3714 games (with mean 11142).

This property obviously does not carry over to baseball due to the nature of the game. One could picture it working for converting between innings and games, but once you get to the inning level and start splitting into outs, the pattern of runs scored is not the same independent of the number of outs in the inning (i.e. more runs are scored with two out than zero out).

As you can see in the chart in the Clay Davenport article I linked above, the Poisson distribution severely underestimates the proportion of shutouts. This is an issue that can be sidestepped by using a modified value at zero, but the slope at non-zero runs is not close enough to reality to salvage a Poisson approach, and the relationship of the mean of runs scored in relation to the variance is not a match for Poisson.

In lieu of a simple distribution, those examining baseball scoring have had to be creative. The three-parameter Weibull distribution (a continuous distribution) has been used with good results by Dr. Steven Miller and Sal Baxamusa to model runs scored. The Tango Distribution, which models runs scored per inning, can be used to estimate the per game distribution with the use of some combination math published by Ben Vollmayr-Lee (zip link) (as an aside, one of the topics in statistics that confuses me the most is combinations and permutations. Sure, I understand why 9! gives you the number of possible batting orders for nine players, and I understand the binomial coefficient, but start asking more detailed questions about pulling colored balls out of urns with and without replacement and I started to wish that I was studying liberal arts.)

With the recent publication of Bill James’ old Baseball Analyst newsletters, I was able to read Dallas Adams’ article in the June 1982 issue in which he uses the following functions to estimate scoring distribution as a function of R/G:

When N is 5 runs or fewer: P(N) = A + B*N + C^2*N

where A = .38498 - .10839 (R/G) + .0080055(R/G)^2
B = .0010306 + .024139 (R/G) - .002943(R/G)^2
C = -.01879 + .002514(R/G) - .00001506(R/G)^2

When N is 6 runs or more: P(N) = D(E^N)
where D = 6.6210 - 2.4965(R/G) + .27518(R/G)^2
E = .058479 + .24022(R/G) - .022291(R/G)^2

That’s the kind of thing I’d like to avoid if at all possible, no offense to Mr. Adams and the others who have made the effort to work on this problem.

Let me close out this post by offering a quick look at the Adams formula and the failures of Poisson, while introducing a set of data that I’ll use to fit a model in the next installment. I have a spreadsheet (I believe it was put together by Clay Davenport or Keith Woolner, but I can’t find the original link) with the runs per game for each major league team 1981-96. The aggregate distribution for these teams was:



The easiest way to fit a Poisson model to this data is to use the maximum likelihood estimate that the parameter h is equal to the sample mean (this is also equivalent to the method of moments estimate in this case). In this case the mean R/G is 4.456 and the variance is 9.36 (those of you familiar with Poisson can already see this is not going to end well as a Poisson distribution has a mean equal to its variance).

The probability mass function for Poisson is:
P(x) = e^(-h)*h^x/x!

For example, the estimated probability of scoring 3 runs is e^(-4.456)*4.456^3/3! = .1712. The empirical probability in this case is ,1417, and this is one of the more accurate points:




As you can see, the Poisson model bunches too much probability around the mean, leaving estimates of high scoring games too low and estimates of low scoring games way too low.

I will also look at Adams’ formula just for kicks. The constants for 4.456 R/G are A = .0610, B = .0502, C = -.0079, D = .9606, E = .6863, yielding these estimated probabilities:




Adams better matches the actual distribution for normal amounts of runs scored, but overestimates the likelihood of extremely high run output. Since he uses formulas that don’t tie to a true probability distribution which adheres to the requirements thereof (most importantly that the sum of all probabilities is one), the estimated probabilities total to 1.03 and runs per game to 4.61.

My point is not to pick on a thirty-year old formula from a pioneering sabermetrician--the fact that luminaries such as Adams and Davenport have not been able to come up with a satisfying approach to this problem serves to illustrate its difficulty, and hopefully to cause you not to judge too harshly the method that I offer in the next post as flawed but useful in certain circumstances.

Before closing this post, let me provide a crude measure of the relative accuracy of the models. Basically, it is the chi-square test statistic. For each number of runs scored (0, 1, 2, etc.), calculate (Expected games with n runs - actual games with n runs)/Expected^2. Here, I will group all games of 15+ runs together, which could be a problem if we encounter a distribution that overpredicts extremely high scores.

I will be doing this test (which I’ll call X^2) without any of the trappings of an actual chi-square test: degrees of freedom, critical values, and the like. The point is not to perform a real significance test--it's to get a quick idea of how these models stack up against one another. I will at various points in this series be testing the model with the same data that was used to fit its parameters--obviously a no-no if you want an unbiased test. However, the first basic test for a model is whether or not it can describe the data it was fitted with. If it can’t do that, there’s absolutely no point in worrying about how it handle out of sample data. None of the models I’m going to be offering up in this series are clearly good enough to move on to that next step.

So, for the Poisson, Adams, and modified Adams (I rescaled so that the total probability would be equal to one) distributions, here are the X^2 values for the model versus the 1981-96 data:
Poisson 91996
Adams 371
Modified Adams 305

Poisson with no fudging obviously doesn’t match runs scored, but is provided here so that you can see how a truly worthless model fares.

Meanderings

* As sabermetrics has matured, many of us have come to recognize to a greater extent that information asymmetry hinders our analysis of transactions and other team decisions. Teams certainly operate with much greater information about their own players than their opponents do, and much, much greater information than we could have hope to possess as outsiders. This is a positive step for sabermetrics, one that I certainly needed to heed more in my earlier days pretending to be more of a know-it-all than I do now.

However, one can go too far, and use information asymmetry as an excuse for questionable decision-making. For the 2012 MLB season, I would make the Reds usage of Aroldis Chapman Exhibit A. Chapman was groomed throughout the winter and the spring as a starter, and seemed to be handling it well. There was really no reason to believe that he wouldn’t, since he had previously started both in Cuba and in Louisville in 2010. Then the Reds suffered injuries to Nick Masset, Bill Bray, and Ryan Madson, and Chapman was moved to the bullpen.

It’s possible, of course, that the Reds always had grave reservations about utilizing Chapman as a starter, and that the bullpen injuries tipped the scale in a direction it was already beginning to go. But given the timing, it seems very unlikely.

Now the Reds have made a second decision, to use Chapman as the closer, for which there is again no evidence that it was prompted by inside information. Instead, the cause is pretty straightforward--continuing poor performance in that role by Sean Marshall, capped by a game in which he had to be rescued by Jose Arredondo.


* With I’ll Have Another set to make a bid for the Triple Crown, there will be two opposing yet equally silly points of view on display among the lesser lights of sportswriting.

The first is that I’ll Have Another cannot be beaten. They will point out the perceived weak Belmont field that does not include Bodemeister, talk about the horse’s “will to win”, and say that they just can’t imagine how he gets beat. Of course, the fact that similar things were written about Smarty Jones and Big Brown will be completely ignored, as sportswriters of this class are incapable of learning. The obvious fact of the matter that if Northern Dancer, Majestic Prince, Spectacular Bid, Alysheba, and Sunday Silence can all fail to win the Belmont, then any horse could as well (not to mention losses in other races, like Native Dancer and Point Given in the Kentucky Derby) will also be ignored.

The second is that I’ll Have Another can’t win because he is running against 34 years of failure and the ten horses in that period that have won the Derby and the Preakness before losing the Belmont. This is also silly; one need only to look at the margin by which Victory Gallop denied Real Quiet to know that the Triple Crown is winnable, not to mention trying to explain who would have beaten Sunday Silence if not for another all-time great in Easy Goer, and many other similar arguments.

Then there is the matter of what it would mean if I’ll Have Another wins the Belmont. Where would that rank him historically? For those that know nothing about horse racing other than the Triple Crown (which is a large share of the general public), it would obviously put him in an elite group of twelve. For those of us that have a broader knowledge about the sport, it wouldn’t really mean a whole lot for his historical standing. It would certainly make him the most famous American horse of recent memory, but he would have to achieve more in his career to be noted as particularly remarkable.

That is not my attempt to downplay the Triple Crown--there's no professional sports outcome I’d rather see other than the Indians winning the World Series and the Browns winning the Super Bowl. But the Triple Crown is three races run over a month and a half in the middle portion of a horse’s three-year old campaign. It is but a small portion of what makes an all-time great race horse. Many all-time greats didn’t contest the Triple Crown, but went on to achieve great things later in their three-year old seasons (see Buckpasser for an example, or Tiznow in more recent history). Many more were non-factors at three, but achieved great things later in their careers (listing examples here would be silly as there are so many).

Of course, evaluating a horse from a historical perspective must include consideration of context. Today’s thoroughbreds race much shorter schedules and are less likely to return to run at age four and later if they are stars at three. Still, even from the perspective of three-year old stars of the last fifteen years, even a Triple Crown will not raise I’ll Have Another above the very best. Point Given may have lost the Kentucky Derby, but he came back in the summer to win the Travers at the same distance, and ran one of the best Belmonts of recent memory. That his “Triple Crown” runs Preakness-Belmont-Travers doesn’t make it much less of an achievement. Curlin may have lost the Derby and the Belmont, but he came back in the fall to win the Jockey Club Gold Cup and the Breeders’ Cup Classic against the best older horses in the country.

On the other hand, the same can be written about some of the other horses that have won the Triple Crown. It is easy to believe that the eleven Triple Crown winners represent the greatest American thoroughbreds, but it really isn’t the case. Certainly all eleven are well-regarded historically and are in the Hall of Fame, but in a couple of cases, that is mostly the result of winning the Triple Crown and not anything else they did.

- Sir Barton won the Triple Crown before anyone talked about such a thing (that would start with the second winner, Gallant Fox). For his career he won 13 of 31 starts. He was Horse of the Year in 1919, and was ranked by Blood-Horse as the #49 American thoroughbred of the Twentieth Century.

- Omaha was sired by the 1930 Triple Crown winner Gallant Fox, but he wasn’t even named Horse of the Year in 1935. He won 9 of 22 career starts and was ranked #61 by Blood-Horse.

It’s not a knock on I’ll Have Another to say that he’d be more Omaha than Count Fleet should he win the Belmont. And if the horse can stay in training and win the Breeder’s Cup Classic or another key race against older horses, I’d be inclined to change my opinion. But that’s a hard task for a three-year old, and the horses that can do it are the ones that distinguish themselves as the best, Triple Crown or no.

* Speaking of horse racing and circling back to baseball, with the season roughly 25% complete I have heard multiple announcers referring to the “quarter pole”. I find this extremely aggravating (which is admittedly a bit of an overreaction). Unless there is some non-horse racing definition of the term “quarter pole” of which I am unaware, using the term in this manner is completely wrong. The quarter pole does not have anything to do with one quarter of the race--it is the pole marking a quarter mile to the finish line. And as you can see from that description, it is oriented to the end rather than the beginning of the race. There is no use for miles in baseball, so I suppose the most appropriate use of the term would be after 120 or so games had been played--a quarter of the season remaining.

I realize that I am complaining extensively about a very silly thing, but when a term from one of my favorite sports gets twisted by people I am predisposed to complain about anyway, I can’t help myself. Off the top of my head, I can’t think of a similar example of a baseball term used in other contexts that so completely misses the point (which isn’t to say one doesn’t exist). The closest I can come up with is “balanced scorecard”, which has been turned into some sort of silly business jargon, kind of how statisticians must bristle when they hear about “Six Sigma”.

Scoring Self-Indulgence, pt. 9

Finally, it’s over. No more boring you with detailed explanations of a fairly conventional scoring system as if it is unique and deserving of great attention. I will close with a pitch-by-pitch translation of a scoresheet into text. Truly fascinating, I know.

This game was played on July 24, 2010 between Los Angeles and Texas. The lineups were as follows:

Umpires
H--Holbrook
1--Gibson
2--Knight
3--Davis

Los Angeles
1. 6 Aybar
2. 4 Izturis
3. 9 Abreu
4. 8 Hunter
5. D Matsui
6. 5 Callaspo
7. 3 Napoli
8. 2 Mathis
9. 7 Willits
P: Santana

Texas
1. 6 Andrus
2. 5 Young
3. 4 Kinsler
4. D Guerrero
5. 7 Hamilton
6. 9 Cruz
7. 2 Molina
8. 3 Davis
9. 8 Borbon
P: Feldman



LAA First
Aybar: Called strike, foul, ball, ball, ball; flied to left (along line)
Izturis: Called strike, ball, foul, ball, foul; foul to third
Abreu: Ball, called strike; lined to right
0 runs, 0 hits, 0 walks, 0 left on base, 15 pitches

TEX First
Andrus: Ball, called strike, called strike, ball; grounded to first (pitcher covering)
Young: Called strike, called strike, ball; lined to right
Kinsler: Called strike, swinging strike, foul; struck out swinging
0 runs, 0 hits, 0 walks, 0 left on base, 13 pitches

LAA Second
Hunter: Ball, foul, swinging strike, foul, ball; flied to right
Matsui: Lined to right
Callaspo: Ball, called strike, ball, grounded to first (pitcher covering)
0 runs, 0 hits, 0 walks, 0 left on base, 11 pitches (26 for game)

TEX Second
Guerrero: Ball; grounded to short
Hamilton: Ball, ball; grounded to first
Cruz: Flied to center
0 runs, 0 hits, 0 walks, 0 left on base, 6 pitches (19 for game)

LAA Third
Napoli: Called strike; flied to center
Mathis: Called strike, swinging strike, foul, ball; flied to center in center-right
Willits: Called strike; flied to left
0 runs, 0 hits, 0 walks, 0 left on base, 9 pitches (35 for game)

TEX Third
Molina: Called strike, ball, called strike; struck out swinging
Davis: Ball, called strike, ball, ball; doubled to right-center very deep on line drive
Borbon: Ball, called strike, foul, ball, ball; flied to center
Andrus: Ball, called strike, foul, ball; grounded to short
0 runs, 1 hit, 0 walks, 1 left on base, 20 pitches (39 for game)

LAA Fourth
Aybar: Bunt foul; singled to left along line, thrown out stretching to second (left to second)
Izturis: Called strike showing bunt, ball; flied to left
Abreu: Called strike, ball, ball, ball; walk
Hunter: Ball; singled to right on flyball, advanced to second on fielding error by right fielder, Abreu to third
Matsui: Called strike, ball; popped to shortstop in shallow left
0 runs, 2 hits, 1 walk, 2 left on base, 15 pitches (50 for game)

TEX Fourth
Young: Grounded to second
Kinsler: Ball, ball, called strike, ball; walk
Guerrero: Grounded into double play, third to second to first, Kinsler retired at second
0 runs, 0 hits, 1 walk, 0 left on base, 7 pitches (46 for game)

LAA Fifth
Callaspo: Ball; grounded to pitcher
Napoli: Ball, called strike, foul, ball; grounded to third
Mathis: Called strike showing bunt, ball, ball, ball; homered to left on flyball, RBI
Willits: Called strike; singled to right on flyball
Aybar: Ball, ball; tripled to right along line on line drive, Willits scored, RBI
Izturis: Ball, foul, foul; flied to right in right-center
2 runs, 3 hits, 0 walks, 1 left on base, 21 pitches (71 for game)

TEX Fifth
Hamilton: Foul; flied to center
Cruz: Foul, swinging strike, ball; struck out swinging
Molina: Foul; singled to center on groundball
Davis: Called strike, foul; struck out swinging
0 runs, 1 hit, 0 walks, 1 left on base, 11 pitches (57 for game)

LAA Sixth
Abreu: Grounded to second
Hunter: Foul, ball, singled to center on groundball
Matusi: Ball, Ball, called strike, foul; singled to right-center, Hunter to third
Ogando relieved Feldman (9 pitches in inning, 80 for game)
Callaspo: Called strike, singled to shallow center on flyball, Hunter scored, Matsui to second, RBI
Napoli: Swinging strike, ball, swinging strike, foul, ball, ball, foul, foul, foul, foul, foul; struck out looking
Mathis: Ball, swinging strike, foul; grounded into fielder’s choice at third, Callaspo retired at second (third to second)
1 run, 3 hits, 0 walks, 2 left on base

TEX Sixth
Borbon: Ball, called strike showing bunt, swinging strike, ball; struck out swinging
Andrus: Ball, ball; flied to right along line
Young: Called strike, ball, ball; homered to left-center on flyball, RBI
Kinsler: Ball, swinging strike, foul, ball, ball; walk
Guerrero: Ball (Young stole second); grounded to third
1 run, 1 hit, 1 walk, 1 left on base, 20 pitches (77 for game)

LAA Seventh
Oliver relieved Ogando
Willits: Ball, called strike, ball; singled to center on flyball
Aybar: Bunt foul, bunt foul; singled to right on groundball, Willits to third
Izturis: Called strike, ball; singled to right on groundball, Willits scored, Aybar to second, RBI
Abreu: Bunt foul; doubled down the right field line on a groundball, Aybar scored, Izturis to third, RBI
Hunter: Intentional ball, intentional ball, intentional ball; intentional walk
Matsui: Foul, ball; grounded into fielder’s choice at first, Izturis retired at home (first to catcher), Abreu to third, Hunter to second
Callaspo: Foul, ball, called strike; flied to center in center-right, Abreu scored, sacrifice fly, RBI
O’Day relieved Ogando
Napoli: Ball, ball, ball, called strike; grounded into fielder’s choice at shortstop, Matsui retired at second (shortstop to second)
3 runs, 4 hits, 1 walk, 2 left on base

TEX Seventh
Hamilton: Flied to center in deep center-left
Cruz: Ball, called strike; homered to left on flyball, RBI
Molina: Flied to deep right
Borbon: Ball, called strike, swinging strike; struck out swinging
1 run, 1 hit, 0 walks, 0 left on base, 9 pitches (86 for game)

LAA Eighth
Harrison relieved O’Day
Mathis: Ball, called strike; singled to left on flyball
Willits: Foul bunt; bunted into fielder’s choice at catcher, Mathis retired at second (catcher to shortstop)
Aybar: Swinging strike, ball, ball, swinging strike (Willits caught stealing catcher to second), ball, foul; walk
Izturis: Ball, called strike; popped to second in shallow right
0 runs, 1 hit, 1 walk, 1 left on base

TEX Eighth
Borbon: Called strike; singled to shallow left along line on flyball
Andrus: Ball, called strike, called strike, foul; struck out swinging
Young: Ball, ball, called strike, ball, called strike, foul; struck out looking
Kinsler: Called strike, swinging strike; popped to first
0 runs, 1 hit, 0 walk, 1 left on base, 17 pitches (103 for game)

LAA Ninth
Mathis relieved Harrison
Abreu: Ball, ball, called strike, ball; walk
Hunter: Ball; reached on fielding error by second baseman, Abreu to second
Matsui: Ball, foul, ball, ball; grounded to second, Abreu to third, Hunter to second
Callaspo: Popped to short
Napoli: Ball, ball, ball, called strike, foul, foul; looped to second
0 runs, 0 hits, 1 walk, 2 left on base

TEX Ninth
Rodney relieved Santana
Guerrero: Ball, ball, ball, called strike, called strike; struck out looking
Hamilton: Ball, called strike, swinging strike, foul, ball; doubled to center-right on flyball
Cruz: Called strike, foul, foul, ball, ball; struck out swinging
Molina: Called strike, foul, ball, ball, ball, foul, foul; struck out looking
0 runs, 1 hit, 0 walks, 1 left on base
LAA 6, TEX 2

Great Moments in Yahoo! Box Scores

This is from the same game I posted earlier.  Yahoo! has corrected the score, but remain confused with respect to the participants in the game.