By Chris Bruce and Andrew Mooney
A couple weeks ago we did some analysis on Tim Tebow, attempting to explain how he’s been winning games despite his poor statistical performance. Simply put, he has been performing poorly overall but has saved his best for the most crucial game situations. After a couple more wins from Tebow, we thought it deserved another look.
A word on the methods used: each potential game situation (down, distance, and yard line) carries with it a certain number of expected points –– the amount of points, on average, a team scores (or allows) from that position, based on NFL play-by-play data. Similarly, every play in a football game adds (or subtracts) these expected points by altering the game situation. For example, a 1st and 10 on one’s own 20 yard line is worth 0.4 Expected Points, and a 1st and 10 on the 20 yard line of the opposition is worth 4.0 Expected Points. A 60-yard bomb on first down following a touchback, then, would be worth 3.6 Expected Points Added, the change in Expected Points generated by the play’s outcome. This last statistic, EPA, is what we used to evaluate Tebow’s raw numbers, free of the context of timing within the game.
But that context can also provide meaningful information about how players affected a game’s outcome, so we also examined Tebow’s Win Probability Added, or how much his plays contributed to the Broncos’ chances of victory. WPA works a lot like EPA –– it calculates the change in a team’s probability of victory following each play –– except that it also factors in time remaining in the game. A 20 yard touchdown pass to take the lead as time expires will be worth much more WPA than a 20 yard touchdown pass to take the lead in the 1st quarter because, in the second scenario, there is still a sizable chance that the opponent comes back to win. (Note: credit for both EPA and WPA goes to advancednflstats.com, which compiles both figures on its website)
Our findings from a couple weeks ago revealed that Tebow’s EPA to that point in the season (-13.4) was worse than Kyle Orton’s (-0.3) during his tenure as the Broncos’ starter, meaning that he objectively produced worse results over all of his plays. However, Tebow recorded a significantly better, though still negative, WPA (-0.06) than Orton (-0.45) –– his plays contributed more to the chances of the Broncos winning. The majority of his positive plays occurred at crucial points in the game, when they had a much larger impact on the outcome: his 20 yard TD run for the lead with 58 seconds left against the Jets, for instance. Likewise, Tebow’s negative plays have come at points where their effect was less harmful, like his entire first half against the Jets.
As one might expect, EPA and WPA for individual players are highly correlated; they measure very similar things, with the only difference being WPA’s allowance for the mystical “clutch” factor. With regard to Tebow this season, his WPA is above and beyond
what you would expect, given the raw results of the plays he has executed on the field. In fact, the difference between his performance and expectation is statistically significant at the 90% level. Other players, like Tom Brady and Aaron Rodgers for example, have had similar discrepancies in their first years, but over time the differences have regressed toward the mean, ultimately coming more in line with expectations.
Yet, in the interim, Tebow apparently chose to ignore our column and win two more games. Did anything about his performance change?
In Week 12, the Broncos beat San Diego with a game-winning field goal in overtime. Tebow’s WPA (still negative at -0.23) was actually slightly lower than expected, given his EPA (-0.2). This means that, as before, Tebow’s plays negatively affected the Broncos’ chances of winning. In previous weeks, Tebow was able to come through with a big play at a crucial moment, thereby disproportionately raising his WPA, but that was not the case here –– Denver tied the game on a field goal with 1:34 left and won with another field goal in overtime (after the Chargers bungled a field goal chance of their own). Though he deserves credit for the game-tying drive he engineered at the end of regulation, the plays that had the greatest effect on the Broncos’ win probability –– San Diego’s missed 53-yard field goal in overtime (which also swung field position dramatically), Willis McGahee’s 24-yard run to set up the winning field goal –– occurred with little to no help from Tebow.
In Week 13, Tebow played arguably the best game in his short career, throwing for 202 yards with a completion percentage of 67%. This resulted in an EPA of +2.0, a positive contribution to his team’s success. Using our regression from last week, however, his WPA of +0.24 was still higher than expected (by 0.31), given his raw on-field performance. Again, this was a product of a pattern Tebow has followed quite often: playing poorly early in the game and coming through late. After throwing for no touchdowns in the first half, he tossed two in the 3rd quarter and rhino-charged in a two point conversion to tie the game in the 4th. And, once more, Tebow was the beneficiary of a healthy dose of late game fairy dust: Vikings quarterback Christian Ponder’s interception, returned to the 15-yard line with 1:33 remaining, which set up a chip-shot, game-winning field goal by Matt Prater.
So, broadly, Tebow has indeed exhibited “clutch” performance –– performing pretty ordinarily overall, but at a higher level when it matters most. Given our previous analysis of other quarterbacks who did this in their first year (including Tom Brady, Aaron Rodgers and Drew Brees), it’s unlikely that this is a sustainable course in the long run. However, it
should be encouraging for Tebow fans that his performance has been improving in absolute terms. Whether he exceeds the expectations of our model or not, you can expect him to win more games if he keeps playing at a higher level.
As an addendum, we’d like to clarify some things about our analysis that may have caused some confusion. First EPA and WPA take into account both turnovers and time of possession. Turning the ball over has a negative expected point value and a negative impact on the player’s team’s win probability, accounted for in the individual player’s accumulated EPA and WPA stats. Additionally, WPA takes into account the timing of plays in the game, so if a team possesses the ball for a longer period of time, it will inherently be represented in WPA. Secondly, it is true that the two stats we use, EPA and WPA, don’t take into account read option plays where Tebow may actually have an impact on the play’s outcome despite not accumulating stats (this not being represented in traditional stats, either). However, this also doesn’t help explain the Broncos’ recent winning ways because the running backs to whom Tebow is handing off have actually performed worse during Tebow’s starts, have a negative WPA, and, like Tebow, are exceeding EPA-based expectations.