by Robert Bedetti
ESPN is once again allowing Mel Kiper to flaunt his homemade haircut and scowl on live television, which can only mean one thing: draft day is approaching. As we count down the days to the start of the draft, the talking heads continue to speculate on where certain players will be drafted and where they should be drafted according to their value. Analysts will construct, dissemble, and reconstruct countless draft boards between now and draft day. The ‘experts’ may mention a coach’s drafting tendencies or evaluate last year’s stars and flops, but they are unlikely to produce any meaningful analysis based on the history of the draft, other than mentioning a few notorious marquee trades.
Especially in the first round of the draft, the reasoning behind a team’s selection is sometimes obvious. Like the 2012 Colts and Redskins, the organization may have an urgent need to fill a certain position (often the quarterback). In other cases, teams are attracted to an exceptional talent that they may seek to develop (e.g. Tim Tebow in Denver before Josh McDaniels left) and possibly trade. Often, however, an NFL team selects its first pick from a handful of equally appealing candidates, each ranked at the top of his position. While talent evaluation is an inexact science, historical data can tell us which positions are the riskiest and which are the most productive to draft in the first round.
Inspired by Kevin’s post last fall, How to Value NFL Draft Picks, I used his data from Pro-Football-Reference, specifically the Career Approximate Value (CAV) for each draft pick from 1980 through 2005. As Kevin aptly noted, CAV is by no means the ultimate NFL statistic, but it is very useful for comparing large groups of players across seasons and positions. I used this data to find the mean, median, and standard deviation of both the CAV and Actual CAV – Expected CAV (Actual-Expected Difference, or AED) for every first round draft pick across all positions in the dataset, besides kickers (sorry Al Davis). The AED measures the difference between each player’s career performance and the mean performance of all players selected in his draft slot, which is the best tool we have for statistically measuring the merit of a draft pick. Since Kevin found a steep decline in value from the top half of the first round to the second, I analyzed each half of the first round separately.
Traditional football wisdom posits that quarterbacks are the riskiest first round picks. I found that this notion holds true, as quarterbacks have a noticeably higher standard deviation in CAV than all other positions. Most of this discrepancy can be attributed to the multitude of quarterback busts that were drafted exceptionally high, and some to the Hall of Famers’ exceptional careers. It is not worth dwelling on the standard deviation or focusing on mean CAV values, since the standard deviation is exceptionally high across positions.
To better represent the riskiness of different positions, I graphed boxplots of the AED across positions. The red lines represent the median AED for each position, but I want to focus on the size of the boxes and their whiskers.
Again, you can see that the quarterback position is by far one of the least consistent in terms of career production relative to draft position. The IQR for quarterbacks drafted with picks 1-16 is about 66 CAV, or the difference between Vinny Testaverde and Tim Couch. Even so, the IQR for offensive linemen chosen with the same picks is about 50 CAV, or the difference between Orlando Pace and Tom Ricketts (granted, Pace was drafted first overall, so his expected CAV was very high). Tight ends are noticeably the most consistent in their performance, but teams rarely draft them in the first round.
I found much disparity between the consistencies in the performance of most positions between the two halves of the first round. Linebackers and defensive backs are comparable to offensive linemen in both halves. While running backs drafted in the top half of the first round vary widely, those in the bottom half are consistent despite a few outliers. Unlike other positions, wide receivers are not more consistent when drafted later in the first round.
In terms of consistency, one can conclude that picks within all positions vary greatly in AED, but that does not mean that every position is equally risky.
In looking at the median AED for each position, I found significant differences across positions.
All of the values are obviously negative, showing that most first round picks in the dataset did/have (some are still playing) not lived up to the average expectation of their draft slots. The expected CAV, however, is skewed to the right, or driven up by a few outstanding superstars, so it is no surprise that all of the values are negative.
Again, as the quarterback position is abounding with high-profile busts, the AED for quarterbacks is the lowest of any position. In the spirit of traditional wisdom, positions on the offensive line have the highest AED in the top of the first round and the second highest in the bottom.
No draft pick is a sure thing, regardless of position or pick number (see: JaMarcus Russell, QB and Tony Mandarich, OL). No scout or organization evaluates its players the same as the next one either. Every team has its own plan and priorities that take may take precedence over scouting reports or historical evidence. When evaluating first round draft picks, history can tell us a lot about how different positions are valued and how they tend to live up to (or fall short of) expectations.
I have only barely scratched the surface of possible uses of this data. There are a ton of possible uses for CAV and AED to evaluate trades, maximize draft value, and model the most efficient strategies. Interviews, workouts, and game film will always be paramount to the NFL’s scouting and drafting process, yet it is clear that NFL teams are, in general, poor drafters. The data from past drafts can be invaluable to the evaluation and selection process that no team has yet to master.