"Statistical Analysis" v. "Sabermetrics"

In the last week, there was a lively discussion on the mailing list of the SABR Statistical Analysis Committee on a proposal to change the name of the committee to the Sabermetric Committee or something similar. Some of the debate has been based on how the term “sabermetrics” is viewed by other members of the organization, and other internal organizational politics-type stuff. I know little about SABR politics and care little about SABR politics, so I will ignore that element of it.

As to which is the better name for the committee, though, I believe 100% that it is “sabermetrics”. One objection has been that there is no readily agreed upon definition of sabermetrics, and that the definitions often employed, especially by those who are not themselves sabermetricians, are essentially equivalent to “statistical analysis.” This is true, although I don’t believe that a misunderstanding of what sabermetrics is by the uninformed should change the way the term is used by those who understand it.

Bill James had two explicitly stated definitions for sabermetrics that he published in the Abstracts. The first was “the mathematical and statistical analysis of baseball records”. Oops. But Bill quickly realized that this was not adequate, so he used “the search for objective knowledge about baseball”.

That definition is too broad in my opinion, as there are many things that are objectively knowable that do not really fall under the purview of sabermetrics. For example, a list of owners of the Red Sox is objective knowledge about baseball, but under this broad definition practically all baseball research is sabermetrics.

Personally, I think that the best single definition of sabermetrics was that presented by Craig Wright in his forward to the 1985 Abstract, which was “the scientific research of the available evidence, to identify, study, and measure forces in professional baseball.” My only quibble here is that I believe the “professional” qualifier is unnecessary. I especially like the use of the word “measure”--James' word, if not his definition, already captured this by using the suffix “-metrics”. It measurement that sets a list of Red Sox owners, or Cal McLish’s full name, and other such things outside of the realm of sabermetrics and into the other categories of baseball research.

I also believe that the “objective knowledge” part of the James definition is best left by the wayside. Much of the statistical data that we work with is not purely objective. Take the distinction between a hit and an error, between a wild pitch and a passed ball, etc. These are subjectively determined, although there is an established framework by which one is supposed to make the distinction. However, scouting reports could also be incorporated in some way into evaluation, if the biases and error ranges are considered--just as we should for our statistical methods. That is not to say that all information is equally valid or equally useful, but we should not throw it all out the window right off the bat.

Wright also included some clarifications about the properties of sabermetrics which are very instructive in this particular debate: “A sabermetrician is not a statistician. Sabermetricians do not study baseball statistics. Sabermetricians are actually involved in research, scientific study, and the object is baseball.” (emphasis in original)

Statistical analysis and sabermetrics are not synonymous. Much of the statistical analysis that is done can certainly be put under the umbrella of sabermetrics--but it is a subset of sabermetrics, not the definition of it. I have always been uncomfortable with the term “statistical analysis” for other reasons--it implies a rigorousness and an approach that is not inherent to sabermetrics.

Statistics is a science unto itself, with its’ own principles and its’ own approach to questions. If a statistician is presented with the problem of estimating team runs from component statistics, his first move is not going to be to sit down and try to build a theoretical model of how runs are scored. He will likely look at correlations between the events, run regressions, etc. I am not here to say that those approaches are bad, just that sabermetrics encompasses more then that. Statistical tools are not going to allow you to create a method like Base Runs. If you created the model though, they will help you validate it.

At the risk of repeating myself, statistical analysis is certainly an important part of sabermetrics, but it is a poor choice of term to describe everything that sabermetricians do. And therefore, I don’t see why the committee should carry that name.