Amateur Hour: Pre-Professional Success as a Predictor of Professional Success in Football, Basketball and Baseball

By Brian Furey

Bryan Bullington was the first overall pick in the 2002 MLB draft. In his senior year with Ball State he went 11-3 and set an NCAA record for strikeouts with 139. In the majors Bullington went 1-9 with an ERA of 5.62. He now pitches for the Hiroshima Toyo Carp.

It is often taken for granted that superstar college and amateur athletes will continue on to professional greatness.  Clearly this is not always the case.  Sports fans every year witness innumerable flops and countless dark horses.  The imperfection of pre-professional success as a predictor of professional success is universal.  A neglected aspect of this phenomenon, however, is the extent to which this is the case in various sports.  In an attempt to address this question, I took data on professional football, basketball and baseball players, seeking validation of the claim that professional success in baseball is most difficult to forecast.

A simple approach is to find, for each sport, the proportion of players said to have attained pre-professional success (been college or amateur stars) that have continued on to professional success (become stars in the NFL, NBA and MLB).  Defining criteria for pre-professional success is somewhat tricky because of sample diversity (players coming from different levels, leagues, countries, etc.) and statistical limitation (intangibles that can’t exactly be quantified).  I decided to use a player’s draft pick as a measure of his pre-professional success.  Although draft picks are ultimately subjective, driven to a certain extent by team needs, and not exhaustive (not all players get drafted), they incorporate vast information, both statistical and non-statistical, and essentially summarize a player’s past success.

Defining criteria for professional success encounters similar issues with statistical limitation.  A logical solution is to use all-star/pro-bowl appearances as evidence of professional success.  Again, this approach is not perfect: all-stars are chosen subjectively (usually by fans, players, and/or coaches); all-star teams limit selection by position (preventing, say, the league’s tenth best player but fifth best at a particular position from being chosen); and, in baseball, all-star teams have the requirement that each team is represented by at least one player, which may cause the selection of someone undeserving and thus the non-selection of an elite player.  However, by and large, all-stars and pro-bowlers are the top tier professionals both in light of, and sometimes despite, the statistics.

Because different sports (and positions within sports) warrant careers of varying lengths, it would be unfair to simply correlate one’s draft pick number with his number of all-star selections.  For this reason, I defined professional success as a player’s having achieved at least a single all-star selection.  To prevent the “fluke season” argument and to provide further insight, I then changed the definition of professional success to a player’s having achieved at least two and, finally, at least three all-star seasons.

With the intention of being both relevant and accurate, I looked at the top draft picks in football and basketball from 1996 through 2005 and in baseball from 1993 through 2002.  The definition of “top pick” is different for each sport because the all-star teams differ greatly in size.  The all-star teams for football, basketball and baseball average roughly 45, 13, and 34 members respectively.  If the data set were comprised of the same number of picks for each sport, the proportion of football players to achieve all-star status would dominate for the simple reason that a team of 45 is easier to qualify for than a team of 13 or 34.  Thus I looked at the top 45 football picks, the top 13 basketball picks, and the top 34 baseball picks each year.  I then recorded how many all-star games each top pick was selected to, and therefore found which players satisfied the definitions of professional success.  For each definition of professional success I found the proportion of top picks who achieved it in each sport.  I then performed hypothesis tests with the alternative hypothesis being that this proportion for baseball was less than those for football and basketball.

The results are summarized in Table 1.  Among the 450 football players said to have achieved pre-professional success (been a top 45 draft pick), 159 of them (35.3%) have gone on to be selected for at least one pro-bowl game.  Among the 130 top tier college/amateur basketball players, 33 (25.4%) have been selected for at least one all-star game.  Among the 340 baseball players studied, 51 (15%) have been voted all-stars at least once.

Table 1

Sport # of Players to Achieve Pre-Professional Success % to Achieve Professional Success (1X All-Star/Pro-Bowl)
Football

450

35.33%

Basketball

130

25.38%

Baseball

340

15%

With this information I performed hypothesis tests for differences in proportions.  This information is summarized in Test 1.

Test 1

Null Hypothesis Alternative Hypothesis P-Value Conclusion

<.0001

Reject Null,Accept Alternative

.0043

Reject Null,Accept Alternative

The findings are similar when the definition of professional success is changed to at least two all-star selections and, finally, at least three all-star selections.  These data are summarized in Table 2 and Table 3.

Table 2

Sport

# of Players to Achieve Pre-Professional Success

% to Achieve Professional Success (2X All-Star/Pro-Bowl)

Football

450

24.44%

Basketball

130

22.31%

Baseball

340

8.82%

Table 3

Sport

# of Players to Achieve Pre-Professional Success

% to Achieve Professional Success (3X All-Star/Pro-Bowl)

Football

450

18.22%

Basketball

130

17.69%

Baseball

340

5.59%

In both cases, performing hypothesis tests for differences in proportions yields p-values less than .0001.  This validates the claim that top baseball draft picks have a lower likelihood of multiple all-star selections than top picks in football and basketball.  If we accept the definition of pre-professional success as a player’s being a top pick and the definition of professional success as a player’s being a one-time, two-time, or three-time all-star, then the data show that pre-professional success in baseball is a weaker predictor of professional success than it is in football and basketball.

This analysis is by no means perfect.  As has already been mentioned, the chosen definitions of pre-professional and professional success are subjective and flawed for various reasons.  The choice of ten year spans was somewhat arbitrary, selected under the notion that 920 players was a large enough sample and that all football and basketball players drafted since 2005, and all baseball players drafted since 2002, have had enough time to achieve professional success if they have deserved it.  Perhaps the greatest weakness is the inability to reconcile the different roles drafts play in each sport – some sports rely more heavily on drafts while others sign talent without serious dependence on them.  Nonetheless, the message of this report is simple and clear: in comparison to football and basketball, it is especially difficult to predict how even the most successful college and amateur baseball players will fare at the professional level.

About these ads
This entry was posted in MLB Baseball, NBA Basketball, NFL Football. Bookmark the permalink.

5 Responses to Amateur Hour: Pre-Professional Success as a Predictor of Professional Success in Football, Basketball and Baseball

  1. David Pinto says:

    Table 1 appears to be incomplete.

  2. Gordon Nowlan says:

    I wonder what the results would be for NHL players

  3. Pingback: Amateur Hour: Pre-Professional Success as a Predictor of Professional Success in Football, Basketball and Baseball » Stathead » Blog Archive

  4. Enrique says:

    One thing to take into account that could really impact your findings is the popularity of the draft. Many people follow the NBA draft and even more follow the NFL draft. This doesn’t happen with baseball. The main reason this would have an impact is the fan vote. A lot of players are recognizable from the hype that draft day brought in for them, attracting more votes.

  5. Dave says:

    Interesting analysis, but I’d call it perceived professional success vs achieved professional success. Pre-professional success is poorly correlated to draft location in my opinion. Heisman winner QBs have been drafted in the 5th round or later (most recently Troy Smith) . In the NBA, Marvin Williams was not even a starter for UNC his final year, but was drafted 2nd overall based on potential.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s