My (Small) Issue With a Stat-Based Football Site

Statistics in sports is a relatively new concept (stop reading this and go read Moneyball if you haven’t already done so) and offers a different perspective that’s not always noticed by the untrained eye. In a sport such as baseball, it’s easier to use analytics for talent evaluation because the sport largely focuses on individual contributions (hitting, pitching, fielding etc.). Also of note, nearly every player on the field is going to touch the ball at some point. So it’s much more difficult in a situation like football where most of the players on the field will never come in contact with the ball.

For my third installment of blog life, I am going to complain about a minuscule issue I have with a stat site that I very much like, Pro Football Focus. They have garnered a good reputation over the past few years with their system of grading players through a means of statistical analysis. And it’s well deserved, they do a lot of great work, even if I don’t agree with everything they do (I do have a paid subscription to the site). The main issue I have with this site is they consider their system as the quintessential model of player evaluation, which I personally don’t agree with. I believe statistics are important, but they don’t tell you the whole picture, especially in football.

For example: their pass rush productivity grade. For reference a sack = 1 point, a QB “hit” = .75 points, and a QB “hurry” = .75 points (hurries = pressures). A QB hit occurs when the QB is hit by a pass rusher, but does not have the ball in his possession or is releasing the ball from his possession.  A QB hurry (or pressure) occurs when a QB is flushed out of the pocket. Here is how they achieve their pass rushing grade:

Total sacks + (total hits x .75) + (total pressures x .75) = Pass Rushing Points

(Pass Rushing Points/number of pass rushing snaps) x 100 = PRP rating

The second half of that equation is irrelevant for my argument. As you will see in the third graphic below, they are only going to use the pass rushing points and other metrics to grade the players. My issue with this metric is that they have 1 QB hit = 3/4 of a sack. A sack is a definitive loss of yards and a loss of a down for the offense. A hit is very much not that. A hit usually occurs after a QB has thrown the football, although it occasionally occurs while the QB is throwing the ball. Now I know a lot of this is subjective, and QB hits are still an important statistic. However, I can’t fathom how something that occurs after the QB has let go of the football, regardless if it was a completed pass or not, is worth 3/4 of a sack.

First I will cover the QB pressure metric. Just to go off PFF, below is their grades/rankings of quarterbacks for this past season.  Colorful, I know. The column I want to highlight is completion percentage, where the quarterbacks are ranked from highest to lowest .

Screen Shot 2015-02-12 at 5.50.33 PM
2014 Quarterback stats, please look at the “Comp %” column

It’s no secret that a QB’s completion percentage drops when he is under pressure, but take a look at the list below, specifically the completion percentage when the quarterback is under pressure. I hope you noticed, but the completion % numbers have dropped all across the board (many by 10-20% lower). Interestingly enough, Ben Roethlisberger seems to be the only one to not have a dramatic drop-off.

Screen Shot 2015-02-12 at 5.51.51 PM
Once again, check out the “Comp %” column

Aaron Rodgers, the highest graded QB of 2014 by PFF, saw his completion percentage drop from 65.6% to 45.7% when under pressure, which is huge for someone of his caliber. Due to this severe drop in completion percentage, I’m okay with a QB hurry/pressure = 3/4 of a sack because I believe it has a measurable effect.

Now let’s revisit QB hits. Generally speaking, a hit will occur while the quarterback is under pressure. That’s not always the case though, as quarterbacks often can’t tell if they are receiving pressure from their blindside or behind them (I realize they can assume the pressure is there, but it’s not always that simple). I’m a Jets fan, so I’ve heard on many occasions, “Tom Brady does not like to get hit.” And it’s mostly true (much to the chagrin of Patriots fans who like to tell you that you are in fact wrong). In games where the pressure gets to him, he often looks shaky and attempts to get the ball out sooner. However, that’s completely subjective. A hit occurs after the QB has thrown the ball, so how do we measure how it affected the outcome of the play? If the QB gets the ball out quicker on his next throw, how do we know it’s because of a hit on a previous play, or just an entirely different factor (i.e. sees a tight window, pre-designed route that requires a quick release etc.)? We can make an inference that QB hits are making an impact, but there’s no real way to measure how much it affects the outcome of the game.

In my experience, I’ve noticed a great deal of QBs hold on to the ball for a second longer before throwing, even if they know a hit is coming. They would rather absorb some contact to give their receiver a chance to break free or let the play develop a little longer if it means completing a pass (i.e. Brett Favre). So while I do believe that QB hits are an important metric, in no way should it be equivalent to 3/4 of a sack. (Below I included the grades for 3-4 defensive ends for this past season)

Screen Shot 2015-02-12 at 12.58.33 PM
JJ Watt = Still good

I’ll get it out there now. I think JJ Watt is really good, for a 3-4 DE to put up the numbers that he does, is an incredible feat. Despite being one of the least fun people in the world, he is a force to be reckoned with. While I do believe he is the best defensive end in football (both 3-4 and 4-3 DEs included), I think his PFF stats are a little inflated due to the issues I raised earlier. He had a whopping overall grade of 107.5 (almost 70 points higher than my boy Sheldon Richardson), but he also had 44 QB hits. That’s 29 more than the next on the list, Jurrell Casey. If you do the math (44 x .75) that’s 33 points added to his grade based off of a statistic that has a subjective influence on the game. Does that take away from JJ Watt’s game? Absolutely not. He has 55 sacks over the past three seasons, he’s earned his praise. I think he just needs to come back down to earth a little bit (in PFF’s eyes anyway).

So what was the point of this post? I felt PFF had a flawed metric they heavily use and base a lot of their work off of and I believe it could use some improvements. It may surprise you, but this is actually something that bothered me and I wanted to tell other people about it, and I have now found the proper channel to do so. Now I realize the irony in criticizing how a stat site calculates their stats, and then going ahead and using them anyway. So while sites like PFF grow in popularity (which I’m all for), it’s important that we take everything with a grain of salt. They still provide a lot of great analysis, but like with everything humans do, it has its flaws. There is no perfect model and that’s okay. As the game evolves, so will the ways we visualize and measure it. Statistics are important, but remember, they don’t always tell you the whole picture.

If you enjoyed my post, please feel free to take the poll below telling me how awesome I am or leave a comment. If not, well here are some animal facts that you might find interesting.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s