Tebow is Still Producing Miracles: An Update

By Chris Bruce and Andrew Mooney

A couple weeks ago we did some analysis on Tim Tebow, attempting to explain how he’s been winning games despite his poor statistical performance. Simply put, he has been performing poorly overall but has saved his best for the most crucial game situations. After a couple more wins from Tebow, we thought it deserved another look.

A word on the methods used: each potential game situation (down, distance, and yard line) carries with it a certain number of expected points –– the amount of points, on average, a team scores (or allows) from that position, based on NFL play-by-play data. Similarly, every play in a football game adds (or subtracts) these expected points by altering the game situation. For example, a 1st and 10 on one’s own 20 yard line is worth 0.4 Expected Points, and a 1st and 10 on the 20 yard line of the opposition is worth 4.0 Expected Points. A 60-yard bomb on first down following a touchback, then, would be worth 3.6 Expected Points Added, the change in Expected Points generated by the play’s outcome. This last statistic, EPA, is what we used to evaluate Tebow’s raw numbers, free of the context of timing within the game.

But that context can also provide meaningful information about how players affected a game’s outcome, so we also examined Tebow’s Win Probability Added, or how much his plays contributed to the Broncos’ chances of victory. WPA works a lot like EPA –– it calculates the change in a team’s probability of victory following each play –– except that it also factors in time remaining in the game. A 20 yard touchdown pass to take the lead as time expires will be worth much more WPA than a 20 yard touchdown pass to take the lead in the 1st quarter because, in the second scenario, there is still a sizable chance that the opponent comes back to win. (Note: credit for both EPA and WPA goes to advancednflstats.com, which compiles both figures on its website)

Our findings from a couple weeks ago revealed that Tebow’s EPA to that point in the season (-13.4) was worse than Kyle Orton’s (-0.3) during his tenure as the Broncos’ starter, meaning that he objectively produced worse results over all of his plays. However, Tebow recorded a significantly better, though still negative, WPA (-0.06) than Orton (-0.45) –– his plays contributed more to the chances of the Broncos winning. The majority of his positive plays occurred at crucial points in the game, when they had a much larger impact on the outcome: his 20 yard TD run for the lead with 58 seconds left against the Jets, for instance. Likewise, Tebow’s negative plays have come at points where their effect was less harmful, like his entire first half against the Jets.

As one might expect, EPA and WPA for individual players are highly correlated; they measure very similar things, with the only difference being WPA’s allowance for the mystical “clutch” factor. With regard to Tebow this season, his WPA is above and beyond

what you would expect, given the raw results of the plays he has executed on the field. In fact, the difference between his performance and expectation is statistically significant at the 90% level. Other players, like Tom Brady and Aaron Rodgers for example, have had similar discrepancies in their first years, but over time the differences have regressed toward the mean, ultimately coming more in line with expectations.

Yet, in the interim, Tebow apparently chose to ignore our column and win two more games. Did anything about his performance change?

In Week 12, the Broncos beat San Diego with a game-winning field goal in overtime. Tebow’s WPA (still negative at -0.23) was actually slightly lower than expected, given his EPA (-0.2). This means that, as before, Tebow’s plays negatively affected the Broncos’ chances of winning. In previous weeks, Tebow was able to come through with a big play at a crucial moment, thereby disproportionately raising his WPA, but that was not the case here –– Denver tied the game on a field goal with 1:34 left and won with another field goal in overtime (after the Chargers bungled a field goal chance of their own). Though he deserves credit for the game-tying drive he engineered at the end of regulation, the plays that had the greatest effect on the Broncos’ win probability –– San Diego’s missed 53-yard field goal in overtime (which also swung field position dramatically), Willis McGahee’s 24-yard run to set up the winning field goal –– occurred with little to no help from Tebow.

In Week 13, Tebow played arguably the best game in his short career, throwing for 202 yards with a completion percentage of 67%. This resulted in an EPA of +2.0, a positive contribution to his team’s success. Using our regression from last week, however, his WPA of +0.24 was still higher than expected (by 0.31), given his raw on-field performance. Again, this was a product of a pattern Tebow has followed quite often: playing poorly early in the game and coming through late. After throwing for no touchdowns in the first half, he tossed two in the 3rd quarter and rhino-charged in a two point conversion to tie the game in the 4th. And, once more, Tebow was the beneficiary of a healthy dose of late game fairy dust: Vikings quarterback Christian Ponder’s interception, returned to the 15-yard line with 1:33 remaining, which set up a chip-shot, game-winning field goal by Matt Prater.

So, broadly, Tebow has indeed exhibited “clutch” performance –– performing pretty ordinarily overall, but at a higher level when it matters most. Given our previous analysis of other quarterbacks who did this in their first year (including Tom Brady, Aaron Rodgers and Drew Brees), it’s unlikely that this is a sustainable course in the long run. However, it

should be encouraging for Tebow fans that his performance has been improving in absolute terms. Whether he exceeds the expectations of our model or not, you can expect him to win more games if he keeps playing at a higher level.

As an addendum, we’d like to clarify some things about our analysis that may have caused some confusion. First EPA and WPA take into account both turnovers and time of possession. Turning the ball over has a negative expected point value and a negative impact on the player’s team’s win probability, accounted for in the individual player’s accumulated EPA and WPA stats. Additionally, WPA takes into account the timing of plays in the game, so if a team possesses the ball for a longer period of time, it will inherently be represented in WPA. Secondly, it is true that the two stats we use, EPA and WPA, don’t take into account read option plays where Tebow may actually have an impact on the play’s outcome despite not accumulating stats (this not being represented in traditional stats, either). However, this also doesn’t help explain the Broncos’ recent winning ways because the running backs to whom Tebow is handing off have actually performed worse during Tebow’s starts, have a negative WPA, and, like Tebow, are exceeding EPA-based expectations. 

About these ads
This entry was posted in NFL Football. Bookmark the permalink.

6 Responses to Tebow is Still Producing Miracles: An Update

  1. tunesmith says:

    What of the possibility that at times other than when Denver needs to “come back”, Denver’s play selection has less EPA/WPA upside, in favor of picking plays with less variance? That would roughly correspond to when plays have less leverage.

    Over the course of a game or season, especially with an offense that isn’t “over performing” when the playbook is opened up, (meaning, generally executing as the offense is capable of for those higher-upside plays), wouldn’t that correspond to WPA outperforming EPA over time?

  2. Pingback: The Curious Case of Tim Tebow « Notes from the Sports Nerds

  3. Todd S. says:

    While I’m not sure if there’s enough data yet for a robust conclusion, Football Outsiders has found there is a tendency for strong running quarterbacks to improve performance of running backs. So Tebow may have had something to do with McGahee’s 24-yard run

  4. Pingback: A Statistical Analysis of the Miracles of Tim Tebow | Harvard College Sports Analysis Collective

  5. The Brucester says:

    OMG! You guys miss the point completely (and I don’t mean the extra point)! You’ve obviously not factored in the God variable. Tim Tebow is incredibly devout, always pointing above when he scores and does something magnificent to thank the Almighty and leading the after-game prayer. So you must absolutely work in the God variable when you do the statistical analysis. To do that you must calculate how devout Tebow is compared to Kyle Orton, which is the cross symbol (+) divided by 2 and multiplied by a factor of .00312, or the equivalent of how many times a devout person prays while summoning up the courage to play a game in which the opposing linebackers would like to do nothing more than tear the head off your shoulders and eat it for breakfast with salsa. C’mon, guys! Wake up! The God variable! Get with the program!

  6. Gator Joe says:

    I watched the Broncos / Bears game yesterday. If you are a statistics purist, then the hair on your arm should be standing up on top of goose bumps right now. Is God the unforseen variable? Probably not. If an all powerful being wanted the Broncos to win the game, why wouldn’t they be winning 40-0 in regulation instead of 13-10 in OT?

    More likely, we’re using the wrong database to explain the correlations. The defensive coordinator (and defense) is doing well to limit the opposing team’s offense. Tebow might be figuring out defenses as the game progresses. The kicker is awesome. Try to quantify these variables rather than using “expected points added” and “win probability.”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s