Introduction

Football announcers are fond of saying that an outstanding quarterback is “consistently excellent” — or conversely bashing an underachieving quarterback by saying he “needs to improve his consistency.”

Based on comments like this, there seems to be a prevailing theory out there that the elite quarterbacks in the NFL are separated from the pack by their consistency.  Do the statistics back this assertion up?  Let’s dive in and take a look.

Because I am a giant stats nerd, I just had to dig into this and see if there was any truth to the conventional wisdom. This is the first installment of a two-part series in which I will examine the consistency of NFL quarterbacks. This installment will explore league-wide trends, while the second part will focus on the four NFC North quarterbacks.

To the statsmobile!

Numbers and tables

To find the simplest method possible of measuring quarterback consistency, I looked at the standard deviation of game-by-game passer ratings for quarterbacks in the last two years. Ignoring playoff games, I included only quarterbacks who made at least 20 regular-season starts for the same team between 2011 and 2012. A total of 24 players qualified for this study. These numbers were chosen to find the best middle ground between getting a large enough pool to do a meaningful study and having a large enough sample size for each individual to get meaningful results.

The table below shows the 24 quarterbacks examined, along with their passer rating over the last two years and the standard deviation of their single-game passer ratings. A lower standard deviation means there is less variability — or in other words, that a quarterback is more consistent.

Looking at the spread of this data, we can see that it shows what statisticians call a “normal distribution,” which is a fancy way of saying that the frequency of the numbers peaks around the mean (average) and tapers off toward either end of the range. In this case, the range is roughly 20 to 30.

Lessons

A quick glance also tells us that more consistent does not necessarily equal to better: five of the six least consistent quarterbacks rank in the top ten for passer rating over the last two years. Indeed, there is almost no correlation between a quarterback’s average passer rating and his standard deviation(r²=0.002). In other words, there is no indication that consistency is necessarily a marker of excellence, for a quarterback can just as easily be consistently bad as consistently good.

This makes intuitive sense. St. Louis Rams quarterback Sam Bradford has put up the most consistent passer rating in the NFL the last two years — but no one would accuse him of being an excellent quarterback. There is no value found in being consistently mediocre.

The next two most consistent quarterbacks, however, have been two of the best in recent years. Ask the average NFL fan to name the best two quarterbacks of 2011 and 2012, and the most common answers would likely be Aaron Rodgers and Tom Brady (Drew Brees would be in the mix as well), and indeed, they rank as two of the top three in passer rating. Interestingly, though, seven of the top ten highest-rated quarterbacks rank in the bottom half for consistency.

This suggests that, among the top quarterbacks, what separates the truly elite players from the rest is their ability to avoid bad games and be consistently excellent. The numbers suggest that may be possible, as the correlation between passer rating and standard deviation rises to 0.38 if the sample size is limited to the top ten highest passer ratings It should be noted, however, that 0.38 still represents a relatively low correlation.

Now I want to look briefly at one interesting case for a quarterback who did not qualify for this study because he missed the 2011 season: Peyton Manning. I am looking at quarterbacks playing for the same team, so rather than combine his 2010 and 2012 numbers (which featured him plying his trade in two different cities), I decided to look at his numbers from 2009 and 2010. In those years, Manning posted a passer rating of 95.6 with a standard deviation of 22.2. His passer rating would rank only sixth in this study — likely because he missed out on the passing bonanza that was 2011, when defenses suffered tremendously from the lockout and shortened training camp — while his standard deviation would score him as the fifth most consistent quarterback.

These numbers actually muddy the waters a little bit: Manning falls short of the elite benchmark in passer rating but is still relatively consistent (similar to Ben Roethlisberger).

Conclusion

Those of you who don’t like wading through a whole long article full of numbers — in other words, all of you — can skip straight to this point and read what I learned in easy-to-digest bullet points!

• There is no statistical relationship between consistency and effectiveness for NFL quarterbacks.
• It looks possible, but is not proven, that the truly elite quarterbacks are separated from the merely good ones by being consistently good.
• Second-tier quarterbacks are some of the most inconsistent in the NFL, probably because they fluctuate so often between good and mediocre.

Stay tuned for an upcoming look at the NFC North quarterbacks!