With the new collective bargaining agreement came the rookie wage scale and a lengthy discussion of how the rookie wage scale would change draft strategy. Using salary data from www.spotrac.com, this analysis presents two models for rookie contract value based on overall pick drafted before and after the new CBA. Combining these models with my previous analysis of Career Approximate Value (a metric from www.profootballreference.com), I present a model for CAVOA/dollar both before and after the 2011 CBA to see how the rookie wage scale affected draft efficiency. (Note that throughout this post, salaries are in tens of thousands of dollars.)

Let’s start by modeling rookie salaries before the 2011 CBA. Salaries peak at the first overall pick, and rapidly decline until pick 34, after which salaries stay fairly constant. This sharp change makes a corner in the salary curve, so we need two equations to accurately model rookie salaries:

If pick ≤ 34, salary = -2197 * ln(pick)+8211.7 (R²: 0.92)

If pick > 34,  salary = -127.1 * ln(pick) + 843.74 (R²: 0.68)

Combining these equations, we get the model below (“eDollars” is expected salary):

Having accurately modeled rookie salaries, we can get CAVOA/dollar at each pick simply by dividing the expected CAVOA by the expected salary of each pick, which results in this graph:

CAVOA/dollar starts at 0.060 at the first overall pick and rises to 0.499 at pick 56 where it begins to fall until the end of the draft. The first 21 overall picks are the most inefficient of any in the draft, but the expected value of those picks is so large that teams accepted the relative inefficiency of those picks. However, it is clear that early first round picks are extremely inefficient compared to the rest of the draft. This inefficiency was one of the reasons that the league instituted the rookie wage scale. But how successful was this policy in making early draft picks more efficient?

The 2011 draft provides some answers. Like in 2010, there is a kink in the salary curve, but significantly later in the draft at pick 68. After this point, salaries become almost constant. Because of this corner, there are two equations to model salaries before and after pick 60:

If pick ≤ 62, salary = -529.2 * ln(pick) + 2451 (R²: 0.97)

If pick > 62, salary = -68.67 * ln(pick) + 574.29 (R²: 0.80)

Combined, these equations form the model below.

Rookie salaries under the rookie wage scale have a very similar shape to salaries before the 2011 CBA. However, the first overall pick’s expected salary is over $55,000,000 less under the 2011 CBA. Because of that shift, rookie salaries are significantly lower at every pick than in previous years.

Having created this model, all that is left to do to get Career Approximate Value Over Average/dollar is divide expected CAVOA by the expected salary of each pick to get CAVOA/dollar. The result is below:

CAVOA/dollar begins at 0.203, and steadily rises until the 61^st pick, where it peaks at 0.497. From pick 62 on, CAVOA/dollar falls to 0.112 at pick 252 (the final pick of last year’s draft). So picks become more and more efficient until the 61^stoverall pick, where efficiency begins falling. The average CAVOA/dollar is 0.294 and the median is 0.288, which makes most of the first round relatively inefficient. Picks 25-147, on the other hand, are all relatively efficient. Given their inefficiency and low expected value, picks after 147 have severely limited value. It is not the case that no individual picked after 147 can be valuable. Rather, those picks have a low CAVOA and are financially inefficient. In terms of trading draft picks, trading out of the first round and into the second definitely increases the efficiency of a team’s draft. However, trading back after the 61^st pick sacrifices both overall value and efficiency, which is the opposite of what teams, in general, want to do.

There are some significant differences between these two CAVOA/dollar models. Earlier draft picks are much more efficient under the 2011 CBA than in previous years. Efficiency starts higher and peaks later in the draft. However, the new CBA makes late round draft picks much less efficient. This change is great for teams, as they are better able to invest significant money in top draft picks without sacrificing as much efficiency as they had to before. While teams should love this change, future early round draft picks lost a lot of money because of the rookie wage scale. All in all, the league accomplished its goal of making rookie contracts more efficient.

You can also read this post on the Boston Globe’s website here

Bill Belichick and the postgame podium are notorious for being a lethally boring tandem. Week after week, he stymies reporters with concise non-answers, vague summaries, and robotic praise of players, giving them nothing of substance to write about his team.

Belichick’s public persona may be as inscrutable as his gameplans, but that doesn’t mean we can’t try to break it down, as he would do to an opposing offense. To get a more in-depth look at the man, we analyzed Belichick’s speaking patterns from the transcripts of his post-game press conferences this season and, for purposes of comparison, gave the same treatment to the more media-friendly Tom Brady.

We started by looking at how long each of them is willing to stand in front of the podium. There’s actually not much variation between the two; Belichick answers questions with an average of 72 words, whereas Brady’s answers average 60 words. However, the difference in the two personalities becomes apparent when we split these averages up by wins and losses. Brady’s answer length barely changes; after a win, his average response length is 59 words, as compared to 62 words following a loss. Belichick is quite a bit less eager to talk to the media after a defeat. Though he averages 82-word answers following a win, he cuts his responses to an average of 25 words after a loss, less than one-third of his post-victory average.

We also examined the word frequencies for each speaker. Of course, the most commonly used words were ordinary ones like ‘a’, ‘the’, and ‘it’. It is more interesting to look at which words had the largest difference in frequency between the two (i.e. which words had the greatest ratio difference between two). Below are the top ten words used most commonly by Belichick and Brady in proportion to one another. (Note: words used less than three times by any one of the speakers were not counted.)

Unsurprisingly, the results for Belichick include a list of profoundly uninteresting words, matching his style at the podium. And Brady, though his list includes a few more syllables, isn’t conveying much more meaning; “We gave them another chance” tells me just as much (or little) as “We didn’t take advantage of our opportunities.”

We also wondered if it was possible to determine the speaker at each press conference just by the style of their speech, examining the frequency of the various words used. Statistician Frederick Mostellar famously used this method to ascertain if Alexander Hamilton, James Madison, or John Jay was the author of the articles of the anonymously published Federalist Papers. At HSAC, we also used it last year to determine the authorship of sports articles, based on individual writing styles. Using these same probabilistic methods, we randomly selected four post-game press conferences (Brady’s and Belichick’s post-game press conferences from last season’s victory over the Packers and loss to the Browns) and tested the question: can a set of equations detect the true speaker?

The answer is a solid affirmative. The model works by producing a probability that a given set of words belongs to either Brady or Belichick, given the speaking patterns analyzed above. In all four cases, the generated probabilities were lopsided (more than 99.9 percent in favor of one speaker), and in all four cases the model was correct in identifying the true speaker.

All in all, this underscores just how little can actually be learned from sports interviews, given the short, recycled nature of their content. Personally, we appreciate Belichick’s approach. The Hoodie has no intention of giving comprehensive answers to anyone’s questions, but at least he doesn’t pretend to by dragging out the same timeworn clichés heard all over the sporting world. His concern is winning football games, and the less other people know about how he does it, the better.

(Huge thanks to TeamRankings for providing the data used in this post)

Last November, I wrote a post on how the lockout did not significantly affect injuries in the NFL this season. Now that the regular season is over, I wanted to take another look. The much discussed lock out could have exacerbated or limited the mounting “wear and tear” and “buildup of cuts and bruises” that develop throughout the season. In reality it did neither. There was almost no difference in injuries this season when compared to the past two years. For various reasons, there has been a lot more attention paid to the injuries that did happen. That increased attention has made it appear that there were a lot more injuries this season, despite efforts by the NFL to limit concussions and hits on defenseless players. If the league is serious about reducing injuries, that attention needs to focus on prevention going into next season.

As the graph below clearly shows, injuries this season are almost exactly the same as the past two years. This constant level of injuries suggests that neither the lockout nor the recent rule changes have had any significant effect on the number of injuries in the NFL.

While it appears that the past two seasons have had a lot fewer injuries than the 2009-2010 season, that difference stems from an extremely high injury rate in the first two weeks of the preseason that year. Removing the preseason, it is clear how little the number of injuries has changed over the past three years.

To accompany this consistent injury rate, injury severity has not changed either – despite the NFL’s recent efforts to penalize and fine players for dangerous hits. Those policies are good for the health of the players, but more should be done if the NFL wants to protect the men on the playing field. To calculate severity, I created a system where a player listed as “probable” had a severity of 0, “questionable” equaled 1, “doubtful” was 2, “out” equaled 3, and “injured reserve” was 4. A metric like “weeks missed” would be better than this system, but those data, unfortunately, are hard to come by. Below is a graph of average severity in each week of the season. As you can see, severity is well within the normal range of the past couple of seasons.

Furthermore, through the end of the regular season, 308 players have been placed on I-R, compared to 259 in the 2009-2010 season and 338 in 2010-2011. Even the most serious injuries have not fallen significantly.

With so many big name players seriously injured this season and recent studies on the effects of concussions on players’ long term health, talk about injuries has become commonplace this season. It may simply be that football as it is played requires this level of injuries. It is up to people who know much more about football than I do how much, if at all,  the NFL should change the game to ensure the safety of its players. What I am putting forward here is simple: neither the lockout nor recent rule changes has significantly changed the injury rate or the severity of those injuries. For that to change, the NFL must continue to change its rules.

Using my previous analysis of career approximate value (CAV) on how to value draft picks, this study analyzes how the results apply to the quarterback position. As the five-month debate on what the Colts should do viz. Peyton Manning and Andrew Luck is just beginning, these findings can help inform the Colts’ decision. As with the entire draft, quarterbacks selected first among quarterbacks and earlier in the draft have more expected upside and more expected value. The following analysis looks at the quarterback position in terms of both order drafted and overall pick.

Let’s first look at quarterbacks by order drafted. Both the upside and expected value of the first overall pick stand out here. The upside of drafting the first overall quarterback taken is two and a half times greater than the downside; these both fall very rapidly as more quarterbacks are drafted. Here’s the graph:

The median is a much better metric to use here because there are so many outliers that skew the mean up. For the first overall quarterback taken, however, the mean and median are both 59, just a little worse than David Garrard at 61. The upside, however, is unparalleled. The upper bound is 152, just three below Brett Favre at 155. This upside drops immediately, falling to 116 and 96 at the second and third overall quarterbacks selected respectively. Those are still great picks: Tom Brady has a CAV of 115 and Mark Brunell has a CAV of 96; however, this is the absolute best-case scenario for these picks. They are much more likely to be closer to the median quarterbacks selected second and third overall: second quarterback should be right below Tim Couch at 31 CAV; the third should be right above Patrick Ramsey at 14 CAV.

These findings also imply that, in general, NFL teams are good at identifying players who will become good quarterbacks. If they did not, the above graphs would have trendlines sloping up.  Given the strong negative slope of the median curve above, it is clear that CAV falls continuously as the draft continues. There are obvious exceptions to this trend: Alex Smith going over Aaron Rodgers; Tom Brady drafted after Chad Pennington, Giovanni Carmazzi, Chris Redman, Tee Martin, Marc Bulger, and Spergon Wynn. However, perspective is important. These are the exceptions; the rule is that teams are good at identifying talent at the quarterback position.

Order drafted is not the only important way to analyze the quarterback position. Overall draft pick is also important. Looking at the data from draft pick instead of order selected requires grouping picks into 16-pick (half round) buckets; without this grouping the sample sizes are way too small. Even with these groups, the models are very noisy. The graph and analysis follow below.

The large groupings hide the drop in value from early to late round picks in the mean and median graphs. That said, there is still a clear downward trend from early rounds to later ones. The upper bound line is the most important line here. The extreme drop after the 33-48 bucket is the really significant point to take from this chart. Before the 48^th overall pick, any quarterback has the potential to be the next Jim Kelly or Drew Bledsoe. After that pick, the upside vanishes. For the rest of the draft, the maximum upside of every quarterback is right around the David Garrard range. Mark Brunell solely causes the peak at the 113-128 bucket, which demonstrates that it is possible to find a talented quarterback late; it’s simply highly unlikely.

Tom Brady

The conclusion here is simple: if you need a quarterback, draft a one as early as possible. If you draft a quarterback after the 48^th overall pick, temper expectations. There’s a reason Tom Brady’s story stands out: he’s the only quarterback of 262 drafted after pick 100 since 1980 to have a CAV over 100. Only Mark Brunell and Trent Green are close, but Brady’s CAV/year is a full 4 points higher than theirs, putting him in a class apart. If teams want a good quarterback, they must draft one early.

Conventional wisdom and anecdotal evidence holds that these breaks, and the distractions around this time of year, can lead to letdowns, especially for favored teams. In fact, just last week Pittsburgh lost to Wagner, #4 Louisville trailed at home to both College of Charleston and Western Kentucky, Michigan struggled with Bradley, and Kansas lost to Davidson. All of these teams were heavily favored at home. Jeff Goodman summed up this view succinctly, tweeting that “Louisville’s players look as though they have already left for Christmas. Down early to WKU but Cards should come back.”

But do BCS home teams really perform worse than usual during this exam break? Do they schedule easier competition to account for poorer play? Looking at my database of 35,000 college basketball betting lines and results from 1996-2011, this does not appear to be the case. As I have explained before, the Vegas line, while imperfect, is still the best predictor of game results. It also is a great way for us to judge performance: if teams really do play worse over this break, we should either see them doing significantly worse relative to the lines. In my dataset, that worse performance simply is not there.

I defined my timeframe as games played between December 19th and December 28th. These specific dates could have been moved a day one way or the other, but the range covers the first game back for most teams after exams, and any games played around the Christmas break. Most teams let players go home for Christmas itself before having them back to campus for the next game. If there is a “Christmas/exams” letdown, it would appear in these games.

I compared performance in these games to performance in non-conference games prior to  exams (November-December) because other non-conference games are a fair peer group  at a similar level of competition–conference play can differ dramatically. The first issue to look at is whether teams schedule easier games during this period. The raw data is summarized in a table below:

BCS Non-conference home game spreads

Unsurprisingly, BCS teams are usually big favorites in non-conference home games. They are, on average, a point bigger favorites (-9.3 vs. -8.3) in Christmastime/ exam period games, but this difference is not statistically significant (using both t-tests and a Kolmogorov-Smirnov test of equivalence of distributions). While the level of competition during the exams/Christmas period is on average a bit easier, it does not appear that teams are scheduling down significantly during this period.

What about performance against that expectation? Again, there does not appear to be a significant divergence in results over exams/Christmastime. At home, BCS teams outperformed the line by an average of 0.5 points in the rest of non-conference play. Over the exam/Christmas period, they outperformed the line by an average of 0.8 points, a small and insignificant difference. The two samples also have almost identical standard deviations, meaning results are not spread out more from the line over the Christmas period.

One might argue that looking at simply the average and standard deviation is not enough. Maybe on average teams do well, but a portion of the BCS favorites simply collapse and lose big at home, missing the line substantially. Let’s look the two distributions of results  compared to the Vegas line (positive numbers indicate that the home team outperformed the line, negative that they underperformed):

Visually, it does not appear that the two distributions differ much at all. This is confirmed by the Kolmogorov-Smirnov test. What this means is that, relative to the Vegas line, the distribution of a BCS home team’s performance is the same in the rest of the non-conference as it is over the exams/Christmas period. There does not appear to be any evidence for a Christmas letdown or worse performance.

While this analysis focused on the home teams, it also tells us about the converse: road performance over the period in question. The teams that travel to play BCS opponents over Christmas do not do significantly better than their counterparts over the rest of the non-conference. This lack of significance extends across the country: including non-BCS home games over the break does not change the finding.

The numbers are convincing, but do not tell the whole story. One caveat may be that Vegas prices in some poorer performance, and thus the spreads for exams/Christmas games are artificially suppressed. More qualitatively, it does seem that teams do perform more sloppily after long breaks. What is clear, however, is that this sloppiness is not somehow only limited to favored home teams.

Your favorite team may lose a shocker over this Christmas and exams period, but if they do, please buck the conventional wisdom and do not blame it on the season.

Big news hit the baseball world yesterday as the Los Angeles Angels announced a $254 million, 10-year contract for 31-year-old Albert Pujols, a strikingly large contract for such an old player, even if Pujols is still a perennial all star and MVP-candidate. It’s pretty clear that the contract is long and a lot of money for a player so late in his career, but just how much is he getting overpaid? Let’s take a look.

To get a sense of what the rest of Pujols’ career will look like we can examine how other top players’ performance typically drops off at the end of their career. Looking at other top first basemen from the past, we can map out a typical trajectory for a player in Pujols’ position, and then make a reasonable estimate of what will happen for him. We use Wins Above Replacement as a proxy for overall production (not a perfect proxy, but a good estimation of each player’s value to the team – FYI, we use Baseball Reference‘s version of WAR). On average, top first basemen start to decline precipitously after age 32. Using this trajectory, we can expect Pujols to contribute approximately 47 WAR over the rest of his 10-year contract. (Here we are generous in assuming that he won’t have any wrist, or other, issues and will be able to bounce back to a high level this year like his counterparts from the past instead of continuing his decline from the past few years. There is a bit of sample bias here in Pujols’ favor because we have used as comparables other Hall of Fame / MVP / All Star caliber players who played out their career and did not see it cut short by injury. We also assume he’s actually 31-years-old… unlike some people)

(Note: it’s strikingly difficult to find a large sample of first basemen who ended their career recently, performed at a near-MVP level and haven’t been suspected of steroid use. I used the group consisting of Jeff Bagwell, Frank Thomas, Mark Grace, John Olerud, Jim Thome, and Will Clark – not all perfect comparables to Pujols, but they still should provide a good sample of how performance degrades over time, including some players that transitioned to DH to extend their productivity)

So how much is 47 WAR worth? Many writers cite a value of $5 million for a marginal win on the free agent market today, implying that Pujols’ remaining years should be worth approximately $235 million to over $254 million depending on salary inflation – right around his contract with the Angels. However, looking at the 2011 salaries and performance of all non-pitchers in the league (excluding those who did not play), teams will pay, on average, $600,000 for a replacement level player (WAR = 0) and $3 million for each incremental win above replacement (including players on pre-arbitration salaries). Given these market values and annual salary increases of about 2.7%, the Angels should theoretically be able to find the performance that Pujols is bringing to the team over the next 10 years with $159 million – overpaying by a whopping $95 million. [Additional note: this does not include the fact that having Pujols would free up talent coming up through their farm system to be traded, so the net overpayment would be somewhat lower.]

This has also just looked at the baseball side of things. It’s also possible that the Angels expect to garner significantly more ticket sales or merchandise revenue from Pujols’ arrival besides just what he is adding on the field. To look at Pujols like the Angels’ front office, additional revenue would need to be considered, not just WAR. But a gap of up to $95 million is a large gap to fill, especially if you only expect another 3-4 quality years from him. (Additionally, the market rates we’ve used to calculate the market value of a WAR should already take into account teams “overpaying” to get additional off-the-field revenues).

Albert Pujols is an extraordinary player, and it’s certainly possible that he could extend his career and play at a high level longer than others have in the past, but if using history as a guide and being a little less rosy about how his career will progress, the Angels could end up being stuck with a $100 million mistake.

A couple weeks ago we did some analysis on Tim Tebow, attempting to explain how he’s been winning games despite his poor statistical performance. Simply put, he has been performing poorly overall but has saved his best for the most crucial game situations. After a couple more wins from Tebow, we thought it deserved another look.

A word on the methods used: each potential game situation (down, distance, and yard line) carries with it a certain number of expected points –– the amount of points, on average, a team scores (or allows) from that position, based on NFL play-by-play data. Similarly, every play in a football game adds (or subtracts) these expected points by altering the game situation. For example, a 1^st and 10 on one’s own 20 yard line is worth 0.4 Expected Points, and a 1^st and 10 on the 20 yard line of the opposition is worth 4.0 Expected Points. A 60-yard bomb on first down following a touchback, then, would be worth 3.6 Expected Points Added, the change in Expected Points generated by the play’s outcome. This last statistic, EPA, is what we used to evaluate Tebow’s raw numbers, free of the context of timing within the game.

But that context can also provide meaningful information about how players affected a game’s outcome, so we also examined Tebow’s Win Probability Added, or how much his plays contributed to the Broncos’ chances of victory. WPA works a lot like EPA –– it calculates the change in a team’s probability of victory following each play –– except that it also factors in time remaining in the game. A 20 yard touchdown pass to take the lead as time expires will be worth much more WPA than a 20 yard touchdown pass to take the lead in the 1st quarter because, in the second scenario, there is still a sizable chance that the opponent comes back to win. (Note: credit for both EPA and WPA goes to advancednflstats.com, which compiles both figures on its website)

Our findings from a couple weeks ago revealed that Tebow’s EPA to that point in the season (-13.4) was worse than Kyle Orton’s (-0.3) during his tenure as the Broncos’ starter, meaning that he objectively produced worse results over all of his plays. However, Tebow recorded a significantly better, though still negative, WPA (-0.06) than Orton (-0.45) –– his plays contributed more to the chances of the Broncos winning. The majority of his positive plays occurred at crucial points in the game, when they had a much larger impact on the outcome: his 20 yard TD run for the lead with 58 seconds left against the Jets, for instance. Likewise, Tebow’s negative plays have come at points where their effect was less harmful, like his entire first half against the Jets.

As one might expect, EPA and WPA for individual players are highly correlated; they measure very similar things, with the only difference being WPA’s allowance for the mystical “clutch” factor. With regard to Tebow this season, his WPA is above and beyond

what you would expect, given the raw results of the plays he has executed on the field. In fact, the difference between his performance and expectation is statistically significant at the 90% level. Other players, like Tom Brady and Aaron Rodgers for example, have had similar discrepancies in their first years, but over time the differences have regressed toward the mean, ultimately coming more in line with expectations.

Yet, in the interim, Tebow apparently chose to ignore our column and win two more games. Did anything about his performance change?

In Week 12, the Broncos beat San Diego with a game-winning field goal in overtime. Tebow’s WPA (still negative at -0.23) was actually slightly lower than expected, given his EPA (-0.2). This means that, as before, Tebow’s plays negatively affected the Broncos’ chances of winning. In previous weeks, Tebow was able to come through with a big play at a crucial moment, thereby disproportionately raising his WPA, but that was not the case here –– Denver tied the game on a field goal with 1:34 left and won with another field goal in overtime (after the Chargers bungled a field goal chance of their own). Though he deserves credit for the game-tying drive he engineered at the end of regulation, the plays that had the greatest effect on the Broncos’ win probability –– San Diego’s missed 53-yard field goal in overtime (which also swung field position dramatically), Willis McGahee’s 24-yard run to set up the winning field goal –– occurred with little to no help from Tebow.

In Week 13, Tebow played arguably the best game in his short career, throwing for 202 yards with a completion percentage of 67%. This resulted in an EPA of +2.0, a positive contribution to his team’s success. Using our regression from last week, however, his WPA of +0.24 was still higher than expected (by 0.31), given his raw on-field performance. Again, this was a product of a pattern Tebow has followed quite often: playing poorly early in the game and coming through late. After throwing for no touchdowns in the first half, he tossed two in the 3rd quarter and rhino-charged in a two point conversion to tie the game in the 4th. And, once more, Tebow was the beneficiary of a healthy dose of late game fairy dust: Vikings quarterback Christian Ponder’s interception, returned to the 15-yard line with 1:33 remaining, which set up a chip-shot, game-winning field goal by Matt Prater.

So, broadly, Tebow has indeed exhibited “clutch” performance –– performing pretty ordinarily overall, but at a higher level when it matters most. Given our previous analysis of other quarterbacks who did this in their first year (including Tom Brady, Aaron Rodgers and Drew Brees), it’s unlikely that this is a sustainable course in the long run. However, it

should be encouraging for Tebow fans that his performance has been improving in absolute terms. Whether he exceeds the expectations of our model or not, you can expect him to win more games if he keeps playing at a higher level.

As an addendum, we’d like to clarify some things about our analysis that may have caused some confusion. First EPA and WPA take into account both turnovers and time of possession. Turning the ball over has a negative expected point value and a negative impact on the player’s team’s win probability, accounted for in the individual player’s accumulated EPA and WPA stats. Additionally, WPA takes into account the timing of plays in the game, so if a team possesses the ball for a longer period of time, it will inherently be represented in WPA. Secondly, it is true that the two stats we use, EPA and WPA, don’t take into account read option plays where Tebow may actually have an impact on the play’s outcome despite not accumulating stats (this not being represented in traditional stats, either). However, this also doesn’t help explain the Broncos’ recent winning ways because the running backs to whom Tebow is handing off have actually performed worse during Tebow’s starts, have a negative WPA, and, like Tebow, are exceeding EPA-based expectations. 

Last week we debuted our BCS prediction model, using it to predict what would happen in the BCS rankings under certain scenarios for the week’s games. Turns out there wasn’t too much chaos as all the favorites at the top of the rankings won, but that doesn’t mean we won’t see some interesting scenarios play out this weekend. Below we let you know how the race for the BCS championship will likely play out depending on the results of the SEC Championship between LSU and Georgia and the Bedlam between Oklahoma and Oklahoma State. For each scenario we show the probability that it will occur, according to TeamRankings predictions, in parentheses.

Scenario 1: LSU and Oklahoma State win (49%)

The lists below are based on my previous analysis on the NFL draft using the Career Approximate Value from www.pro-football-reference.com.

The Overachievers

Tom Brady

* Denotes an active player

Of course Tom Brady has been the highest performing pick in history. I think it’s crazy that Green Bay and Denver make up 7 of the best 10 draft picks ever. Looking through draft data by team will show whether or not those picks indicate a superior front office or just luck. It’s also interesting that no player selected before the 150^th overall pick is here, but that should be expected given that projections for late round picks are so much lower than the expected value of higher round picks.

The Worst Underachievers

To be clear, these data mean that Akili Smith provided 1.6% of the expected value of a 3^rd overall pick. That’s just horrendously terrible. He was almost twice as bad as Ryan Leaf. No other player in history has performed below 2% of his expected value. A singular talent, that Akili Smith. The 1997-9 drafts should not be proud to represent over half of the worst picks in NFL history.

The Most Overachieving Top 10 Picks

* Denotes active player

Perhaps worthy of an honorable mention is Peyton Manning who was a number one pick

Rob Woodson

and has a CAV/Expected CAV of 2.1

I think it’s amazing that there are three active players in the top 11 here. No surprise that Peyton Manning is the only first overall pick close to the top 10. The relatively low CAV/Expected CAV really demonstrates the different expectations of a top 10 pick from the rest of the draft, especially compared to the biggest overachievers in history.

The Most Underachieving Top 10 Picks

Honoree Mention in 14th place is JaMarcus Russel who was the first overall pick and has a CAV/Expected CAV of .094.

Hard to believe JaMarcus comes in at #14… or that he was almost six times better than Akili Smith. The fact that Mike Junkin and Kelly Stouffer were back-to-back historically awful picks is endlessly entertaining. I wonder if there has ever been a worse series of picks in any professional draft. (Edit: As commenters have pointed out, Reggie Rogers was picked directly after Junkin and Stouffer. This makes the series even more remarkable.)

JaMarcus Russell

Commissioner Roger Goodell at the NFL Draft

by Kevin Meers

Through 12 weeks of the season, most franchises in the National Football League know whether they are in the playoff hunt or if they need to start prepping for the draft: The Green Bay Packers and San Francisco 49ers have almost clinched playoff berths while the Indianapolis Colts have just about locked up the number one overall draft pick. As the Superbowl hopefuls get excited for playoff runs, most of the league has to sit and wait for April. For five months, most teams have to sit and wait. They are on the outside looking in, watching the playoffs on TV. Eventually, only one team will win while the rest of the league can do nothing but watch. That team will be the undisputed winners, and everyone else will have officially lost.

Until draft day. On draft day, every franchise wins. No matter what grades draft analysts give out, the draft gives every franchise hope.

Every team, every fan, has this hope that their franchise will land the next great player. Everyone has the same dream, that their fourth round draft pick will turn into a Hall-of-Famer. Yet they have little idea what that fourth round draft pick is actually worth. For years, teams used something like the chart Jimmy Johnson developed as the coach of the Dallas Cowboys in the 1980s to value picks against each other.

These values are completely arbitrary: there is no statistical evidence to back up the relative values of these draft picks. There is no reason why the 156^th pick is 100 times less valuable than the first overall pick. “The Chart” simply dictates how much each pick is worth. These values also have no grounding in the real worth of the players drafted at a given pick. This system is a ridiculous way to value picks because there is no reason behind the values it gives. There must be a better approach.

Taking data from www.pro-football-reference.com, I have created a much better system that more accurately values each pick in the NFL draft, similar to the work done by Chase Stuart. Pro-Football-Reference uses a metric called Career Approximate Value (CAV) that allows one to compare players across seasons and positions. It is not meant as the ultimate NFL statistic. It is useful for comparing large groups of players across time and positions, which is exactly the objective here. Using data from 1980 through 2005, I analyzed each overall draft pick from 1 to 224 (the 32^nd pick of the 7^th round in today’s draft). I found the mean, median, and standard deviation of the CAV for each pick from those 25 years, creating one set of data that represented the historical value of each pick. I then found the mean, median and standard deviation for this new dataset to determine the expected value of a normal draft pick. I then used that normal pick to standardize my data, finding the percentage value over average, or Career Approximate Value Over Average (CAVOA), for every pick in the draft.

For example, the first overall pick, historically, has had a mean CAV of 66.7. The standard draft pick had a mean CAV of 15.03. . Thus the first overall pick was 443.39% more valuable than the standard pick. Using this method, I found the CAVOA of every pick in the draft, and then regressed it against overall pick number. The regression equation was  with an R² of 0.91599. The R² means that the variance in overall pick number explains 91.599% of the variance in CAVOA. Using this equation, I found the expected CAVOA for every pick in the draft.

The CAVOA is the comparative value of each pick versus the normal pick and is based off of real, historical, on-field performance. This non-arbitrary statistic is a massive improvement over the old draft chart. To compare my system with the old one, I transformed the old system into a percentage over average as well. The results are below.

The old system massively over values the earliest picks and significantly undervalues mid-to-late round picks. The regression line is clearly a better predictor of future value than the old chart.

Quarterback Josh McCown

But what do these numbers actually mean in practice? The 94^th pick is as close to the normal pick, having a draft value of 100.3 and a CAV of 15. If Josh McCown had retired instead of signing as Caleb Hanie’s backup with the Chicago Bears last week, he would have ended his career with a CAV of 15.  So if a team uses the 94^th overall pick, it should expect to draft a player with a comparable value to Josh McCown: a marginal backup who may spend some time in the United Football League, but will also have some productive outings in the NFL.

In contrast, the first overall pick has a value of 494.6, almost five times greater than the

Safety Rodney Harrison

94^thpick; so one should expect a player with a CAV of 74. Brad Johnson, Rodney Harrison,and Corey Dillon all had CAV’s of 74. Given the first overall pick, that is the kind caliber player one should expect to draft: Pro-Bowlers who can help lead their team to a Super Bowl victory. One cannot expect to get a player like Peyton Manning, although it is a possibility; instead, one should anticipate a player of Harrison, Dillon, and Johnson’s caliber.

The distribution around each pick is also interesting. The standard deviation of the expected CAV decreases as the pick number increases, suggesting that picks become less and less variable as the draft continues. While this conclusion is accurate, the coefficient of variation is a better measure of variance than standard deviation in this case. Because there is a separate data set for every pick, one has to compare the standard deviation compared the average of each data set. While the standard deviation decreases as the draft continues, the coefficient of variation is constantly increasing, suggesting that picks become much more variable relative to their mean as the draft continues. This finding suggests that one can be more certain of an earlier draft pick’s value than of a later draft pick.

That said, it is impossible to predict with any accuracy what a given pick’s CAV will be, as the standard deviations are very large. Below is a graph of the 95% confidence interval for every draft pick. The real confidence interval would include negative values for a number of picks, but since this is impossible, I list them as 0. Each pick’s confidence interval includes 0, meaning that no matter where in the draft one picks, there is always the risk of the player picked being a complete bust. However, the upside of picking a superstar diminishes rapidly as the draft unfolds, making earlier picks extremely valuable. This subject, however, is for another study.

This study is the tip of the iceberg of analysis that is now possible to do. There is a ton of work to be done now that we have an accurate representation the value of every draft pick in terms of both on-field value (expected Career Approximate Value) and in terms of other draft picks (Career Approximate Value Over Average). One can use these data to evaluate any trade involving draft picks, come up with a draft strategy that maximizes total expected Career Approximate Value, and much more.