Gelf Magazine - Looking over the overlooked

Books | Sports

December 4, 2007

Enlightening Football's Dark Ages

Football Outsiders founder Aaron Schatz tries to make sense of the NFL's numbers, and its known unknowns.

Eriq Gardner

In 2003, Michael Lewis published Moneyball, the account of how Oakland Athletics general manager Billy Beane used a statistical approach to find underappreciated assets that could help his low-budget team be competitive. The book came as a revelation to many sports fans (and, of course, fantasy sports fans), and created a demand for better statistics in other sports.

Applying Moneyball concepts to the NFL wasn't easy, though. Baseball had a strong statistical tradition. Football was in the stone age. Aaron Schatz sought to change that.

"The Patriots are the best team ever, as of now, but let's see what they do in the playoffs before we officially give them the title."—Aaron Schatz

As a side gig to his day job analyzing the popularity of websites for Lycos, Schatz started Football Outsiders in 2003. Taking his cue from Lewis, and his inspiration from statistical pioneer Bill James (author of The Bill James Baseball Abstract), Schatz set out not only to find new methods for analyzing the performance of football players, but also to foster a community of statistically-inclined “outsiders” who could help him.

The site took off quickly. Football is a sport with a quick season and a small, finite number of plays per game, each of which seem to mean life or death for a team’s playoff hopes. Football coaches get routinely booed for passing when they should be running, and for playing cover when they should be going man defense. Schatz’s quests, such as examining conventional wisdom about running the ball early in a game, hit a nerve. The site also scored a publicity coup when it managed to snag ESPN’s Gregg Easterbrook for a few columns.

Football Outsider’s success led to a now-annual book that comes out before the start of each football season. Pro Football Prospectus 2007: The Essential Guide to the 2007 Pro Football Season is a collection of essays by Football Outsider contributors and long-detailed scouting reports on players and teams. Gelf got Schatz's thoughts on the state of statistical analysis in the NFL, as well as his thoughts on the current season: Are the Patriots the best team ever? And who will win the NFC? This interview has been edited for clarity. (You can hear Schatz and other sports writers read from and talk about their works at the free Varsity Letters event presented by Gelf on Thursday, December 6, in New York's Lower East Side. UPDATE: Aaron Schatz is ill and won't be able to attend.)

Gelf Magazine: In baseball, we've read about the rivalries between the stat-heads and the scouts. Of course, much of that is overblown, but in your view, what has been the perception in football circles to a more analytical approach to the game, and how has that evolved since you started Football Outsiders and published Football Prospectus?

Aaron Schatz: Football teams have always been more accepting of objective analysis than other sports, since the days when Gil Brandt fed scouting information into a computer to figure out who Dallas should draft. Film study is, in effect, objective analysis. "Hey, quality control guy, tell me, when Detroit is on third down, how often are they lined up in Ace and how often in Tiger?" or some such thing. Front offices use it to different degrees, coaches to different degrees, but there isn't the kind of negativity you always saw in baseball. (Another reason for this: Most football teams were family-owned for a long time, so teams were often run by businessmen, not former players.)
At the same time, I haven't been in contact with many teams, because the fact that the NFL has done this stuff more than other sports also means teams are much more secretive. There's no community that crosses over between front-office people and outsiders like myself, the way you have with SABR. Everything the teams do is very proprietary.

GM: To paraphrase our former Secretary of Defense, there are things we don't know and things we know we don't know. What aspects of football would you identify as being the next frontiers of statistical research?

AS: To paraphrase him again, we have lots of "known unknowns." Football research is still in the dark ages compared to baseball research, primarily because the information is just not there. It's hard to research stats when the stats don't exist. That's why we started the FO Game Charting Project, and even that is extremely imperfect. I would love to be able to get my hands on coaches' film to create stats on blocking, defensive coverage, and so on.

GM: Do you think head coaches get too little or too much credit for the successes and failures of their teams?

AS: Um, probably neither. It's probably about right. Again, it is something we just can't quantify yet due to lack of information.

GM: With your fingertips on the leading edge of statistical analysis, how unconventional would you be as an NFL head coach? Would you, for example, go for it on fourth down at most opportunities?

AS: Hmmm, I don't know how unconventional I would be. I would definitely go for it on fourth down as much as Bill Belichick does, maybe a little more. I wouldn't constantly play it safe with lame draws on third-and-long. I would use a running-back committee, and make sure at least one of the two backs could throw the ball a bit—then I would use the halfback option roughly once a week, so opposing defenses would always have to worry about whether they were leaving a receiver open whenever my back took a pitch to the outside. I would make sure I had depth with players who were good on kickoff and punt coverage, and I would have a kicker with the strongest possible kickoffs. I would trade down into the late first round or second round whenever possible, but also trade lower picks to move up into the third round more often.

"If a robo-punter existed, would you take him with the first overall pick? Of course, robots are not legal in the NFL. Also, they are fictional."
GM: Do you think that teams look at the right performance metrics when analyzing football players out of college?

AS: What performance metrics? There are no performance metrics. There are no college stats for most positions, and the stats we do have for “skill positions" need to be looked at in the context of the style of football played in each conference. For example, we've found that running backs from the Pac-10 seem to be underrated a bit, and receivers overrated, due in part to the fact that the Pac-10 has more passing than the other BCS conferences.

GM: Ever since reading David Foster Wallace’s Infinite Jest, the transformational value of punters has been a favorite topic of mine. Do you think we’ll ever see a punter who changes the way that football is played?

AS: There are limits to what the human body can accomplish. That's why the people on my site joke about "ROBO-PUNTER," who lands every punt coffin corner at the one-yard line. The question is, if such a punter existed, would you take him with the first overall pick? Of course, robots are not legal in the NFL. Also, they are fictional.

GM: Because there are only 16 games in a season, not including the playoffs, how do you think luck and chance factors into team and individual performance?

AS: A ton. A ridiculous amount. Far more than in other sports. Getting people to understand that is one of the most important things we try to do at Football Outsiders.

GM: Given the opportunity, is there anything in Football Prospectus 2007 you’d like to take back?

AS: Dude, do you want a list?
There are two kinds of predictions that go wrong. Some predictions made a lot of sense based on the information we had at the time, and based on previous trends. There was no way to expect Randy Moss to have a season like this because no receiver that age whose numbers had declined like Moss's had ever came back to this extent. We were completely wrong about the Dallas Cowboys, but there were a lot of reasons to think that team would decline, and they all still make sense in retrospect (Cowboys collapsed at end of previous season, new offensive coordinator, Tony Romo still a big question mark before season began, etc.). It just so happens that probability is not certainty.
The second kind of prediction is the one where you look back and say, well, I should have seen that coming, and we need to be more aware of that in the future. For example, I think we were a little too quick to say "ACL injuries are not as damning as they were in the past," based on the year Javon Walker had in Denver. Clearly, in the case of Donovan McNabb, this is not true, and our projection for McNabb and for the Eagles as a whole was too high because of this.
There were places where we just completely misread a coach's intentions for usage of a player. Laurence Maroney is the best example. The fact is, first-round backs with numbers like Maroney's in year one usually become studs in year two, but that wasn't going to happen given the way Belichick likes to use his backs. I thought this would be different and I was wrong. And we were too hyper on expectations when it came to emerging young running backs in committees (Jerious Norwood, DeAngelo Williams, etc.).
In general, I don't feel too bad about the fact that many of the fantasy football projections were way off, because if you go around the Web and look at all the preseason magazines, everybody's projections were way off. Everyone else was wrong about Frank Gore and Steven Jackson, too. At least we were right about Larry Johnson and Shaun Alexander. Finally, I also wish we had trusted our numbers on the Saints and Bucs a little more.

GM: The talk of the season so far has been the dominance of the New England Patriots. How good is this team, in your opinion? The best ever, or the beneficiaries of a weak schedule?

AS: They're the best team ever, as of now, but let's see what they do in the playoffs before we officially give them the title. Many of us believe the 1985 Bears are the greatest team ever, in part because of how they dominated the playoffs. However, the idea that the Pats have a weak schedule is nuts. It comes from only looking at their division. Remember, they have to play all four NFC East teams, and all those teams are better than average, even the ones with losing records. They also have Pittsburgh, Indianapolis, and San Diego. If they go perfect, they will do it against a schedule much harder than the schedule played by the 1972 Dolphins.

"Chris Chambers runs awful routes and drops balls but makes the highlights because his athletic ability lets him make a great play or two most weeks."
GM: OK, time to put yourself on the line here: Who is going to win the NFC?

AS: Whoever won tomorrow night's game by the time people read this. [Eds. note: That would be the Cowboys.] I do think Dallas is better, primarily due to depth—in particular, they have two tight ends and two running backs better than anyone Green Bay has at either position—but the teams are close enough in quality that home-field advantage will make one team the favorite over the other.

GM: Who is the most underappreciated player in the NFL and why? The most overrated?

AS: The most overrated is San Diego receiver Chris Chambers, who runs awful routes and drops balls but makes the highlights because his athletic ability lets him make a great play or two most weeks. The most underrated players are offensive linemen and defensive linemen in 3-4 schemes who don't pick up stats. Let's say… defensive end Aaron Smith of the Steelers and center Dan Koppen of the Patriots.

Related in Gelf

• Two backgammon experts claim that their computer modeling program can help NFL teams win at least one more game per year.

Post a comment

Comment Rules

The following HTML is allowed in comments:
Bold: <b>Text</b>
Italic: <i>Text</i>
<a href="URL">Text</a>


Article by Eriq Gardner

Contact this author


Hate to miss out? Enter your email for occasional Gelf news flashes.


Gelf t-shirt

The picture is on the front of the shirt, the words are on the back. You can be in between.