By John Ezekowitz
Effective Field Goal Percentage. Turnover Percentage. Offensive Rebounding Rate. Free Throw Rate.
Ever since Dean Oliver posited these statistics as the most important for explaining how teams score and defend in basketball, these so-called Four Factors have been the gospel of the tempo-free college basketball stats community. According to Oliver, the Four Factors are listed in order of their relative importance, with Free Throw Rate being the least important of the four. FT Rate, Free Throws Attempted divided by Field Goals Attempted, measures how often a team gets to the line. Oliver was very insistent on this definition, saying, “the biggest aspect of ‘free throws’ is actually attempting them, not making them. Teams that get to the line more are more effective than teams that make a higher percentage of their free throws…”
This has always struck me as odd. Sure, getting to the line is very important (it leads to all sorts of beneficial effects of getting the other team in foul trouble), but free throw makes matter, too. Could there be a statistic that better measures both? Now, with seven full years of college basketball data at hand, we have a dataset big enough to investigate the question. And the answers are surprising. It’s time to re-evaluate that Fourth Factor.
Two weeks ago, I unveiled a new metric I called Free Throw Plus. The full methodology is detailed here, but the more I thought about it, the more one of the inputs really interested me. To calculate FT+, I needed first to calculate Free Throws Made per 100 Possessions, or as I call it, FTM/Poss. FTM/Poss is an interesting stat because it takes made free throws into account in a tempo-free manner. Just like Turnover Percentage, FTM/Poss takes a useful counting stat and makes it better by controlling for the pace at which teams play. My friends at Team Rankings have calculated the FTM/Poss standings for 2011 here.
But is it a better stat than Free Throw Rate? To answer this, I first had to specify some criteria. First, I decided to look at offense only. The role that the free throw line plays in defense is much more complicated — while a team can control its own free throw shooting, it can do little (other than fouling big men who shoot worse from the line) to control that of its opponents. The defensive implications of FTM/Poss will be tackled in another post.
To analyze the relative merits of the two stats, we should start with whether FT Rate or FTM/Poss is a better predictor of offensive efficiency. Oliver defines his Four Factors as the components that make up offensive and defensive efficiency. Oliver’s pioneering offensive and defensive ratings, which measure points scored and allowed per 100 possessions (or points per possession, if you prefer), were the measures of efficiency. If one of the two stats is clearly superior in predicting offensive efficiency, that has large importance in Oliver’s framework. Additionally, it is important to look at how well these two stats measure getting to the free throw line.
On the first count, the answer is clear. Using team data compiled from Ken Pomeroy and Statsheet.com for all teams from 2004-2010, I ran a series of regressions controlling for the other three offensive factors and defensive efficiency. The results are below:
This regression used FTM/Poss instead of FTRate. The Adjusted R^2 was .8179, and the root mean square error (variance of the error, a lower number represents a better fit) was 3.826.
This regression used FT Rate. The Adjusted R^2 was .7912, and the RMSE was 3.992.
As you can see, while both are significant predictors of Adjusted Offensive Efficiency, FTM/Poss is a slightly better predictor than FT Rate. The difference is not huge, but it is still apparent. Including both FT Rate and FTM/Poss in a single regression shows that both are significant predictors, but that FTM/Poss has a higher t statistic (5.55 to -4.43). Thus in terms of predicting offensive efficiency, FTM/Poss appears to be a better choice for the Fourth Factor.
But, of course, there is more than just predicting offensive efficiency. We need to know how often teams get to the free throw line. As you might expect, the two metrics are highly collinear, with a correlation of 85 percent. But what of the 15 percent variance? Is FTM/Poss missing some important measure of how many times a team gets to the line? Theoretically, and as we observe in the data, FTM/Poss should increase as FT Rate increases: more chances will lead to more makes. So where is the variance? Take a look at this scatter plot of FTM/Poss and FT Rate for the 2010 season.
Unfortunately, the data was packed too tightly to get coherent team labels into the graph. Nevertheless, we can examine the nature of the variance by looking at teams (datapoints) that are relatively far away from the best fit line.
First, a team that has a relatively high FT Rate, and a relatively low FTM/Poss. In 2010, Northwestern State of the Southland Conference had an FT Rate of 49.63, getting a free throw attempt for every two of their shots. Kansas State had an almost identical FT Rate of 49.86 percent. Yet while Kansas State scored 27 Free Throws per 100 possessions, putting them right on the best fit line, NW State scored only 23 FTs per 100 possessions. That is a four-point difference in offensive ratings over every 100 possessions, holding all other things equal. This is an example of a comparison that FT Rate misses (it views Kansas State’s and Northwestern State’s free throw performance as identical), but FTM/Poss gives better information on.
How about a case where FTM/Poss views two teams as identical, but there is a difference in FT Rate? For this, we can compare two teams from 2010: the national champion Duke Blue Devils, and the Oakland Golden Grizzlies. Both Duke and Oakland made 25.6 Free Throws per 100 possessions in 2010, yet Oakland had an FT Rate of 41 percent, whereas Duke’s FT Rate was 37 percent. A 4 percent difference might seem like something significant, but as it turns out, it really is not. In fact, the Blue Devils actually drew more fouls per game and per 100 possessions than the Golden Grizzlies, but come out with a lower FT Rate because they simply took more shots. This was probably a function of Duke having a lower Turnover Rate and a higher Offensive Rebound % than Oakland.
So which variance explains more of a team’s offensive success: a case where FTM/Poss identifies an extra four points per 100 possessions that FT Rate does not see, or the case where FT Rate finds that a team gets to the line more often simply because they took fewer shots? I actually did not cherry-pick these examples: I just chose two data points that were relatively far away from the best fit line where FT Rate = FTM/Poss to examine. I think both examples, and the correlation between the two statistics, show that FTM/Poss captures almost all of the “getting to the line” effects that FT Rate measures, while also measuring how much teams actually score once they get there.
The positive effects of getting to the line, and thus getting your opponent in foul trouble, should also extend to defense for two reasons: foul trouble causes coaches to sit their starters and put in reserves, and because teams (even bad free throw shooting teams) score more effectively from the line, it allows them on average to build leads and causing the opponents to play from behind. We can (admittedly crudely) test the ability of FT Rate and FTM/Poss to measure the “getting to the line” effect by looking at how well they predict defensive efficiency. As it turns out, when controlling for the defensive Four Factors and overall offensive ability, FTM/Poss is again just as good a predictor of defensive efficiency (t stat= 6.20, p<0.001) than FT Rate (t stat=3.20, p<0.001).
After weighing all of this evidence, it seems hard to conclude that Free Throw Rate is a better offensive Four Factor than FTM/Poss. FTM/Poss is a better predictor of offensive efficiency, and also seems to capture the effects of getting to the line well. Theoretically, the best single free throw stat would be able to show not only how many points a team gets from going to the line, but also indicate some of the second-order effects that come from getting the other team in foul trouble. FTM/Poss fits that description far more closely than Free Throw Rate.
While this analysis may not be a decisive and thorough victory for FTM/Poss, I think it certainly provides a strong case for revamping the Fourth Factor. Now if only we could come up with a snappier name for Free Throws Made per 100 Possessions…
Pingback: Tweets that mention Re-Examining the Four Factors: The Case for Free Throws Made per 100 Possessions | Harvard Sports Analysis Collective -- Topsy.com
This is a great post; thanks for taking the time to do this. I don’t have a (very) substantial response yet, but here are some quick reactions:
– the Four Factors are mostly just a way to organize basketball events into workable categories… Specifically the events that have an impact on the score, either by adding points to it or changing who has the possession and thus the chance to score. Some people prefer to look at free throws made/field goals attempted as the fourth factor , and that’s legit too.
-I’m probably missing something, but choosing between FTM/Poss and FGA/FTA comes down to whether or not you want turnovers in your denominator. Once you include turnovers to make it truly ‘per possession’, you have to reconcile the relative possession cost of FGA and FTA, either estimated with a coefficient or through play-by-play data to parse out the FTs that end possessions from the bonus FTs. FGA/FTA does ‘control for the pace at which teams play’ in a meaningful way, even if it is an incomplete picture without TOV. There might be reasons to look specifically at field goals and free throws or reasons against a FTM/Poss approach.
“Theoretically, the best single free throw stat would be able to show not only how many points a team gets from going to the line, but also indicate some of the second-order effects that come from getting the other team in foul trouble.”
This is a good point. I would have no idea where to begin, and it seems like there are some prerequisite steps before this could happen. I’m not sure we understand foul trouble to begin with. It is something else to keep in mind when actual basketball is being played.
Thank you for taking the time to not only read, but comment on it. The denominator distinction is a good one to make. I’ve been experimenting with Non-FT Offensive ratings and Free Throw Offensive Ratings, and it wouldn’t be too hard to extend that concept to FGA-only Offensive Rating. That might give us a better idea of the relative possession cost of FTA and FGA. I guess the highest level point of my post is that using FTA in the denominating really doesn’t make a lot of sense to me. The “getting to the line” argument doesn’t seem to outweigh the value of knowing how many points you are actually scoring.
I agree that foul trouble is not well understood right now. Longer term, I’d like to think about ways to quantify it.
I may be missing something, but I notice that in Dean Oliver’s article he also writes: “But I often prefer to make some account for ability to make the foul shots, too, and use FTM/FGA”. That is the most common stat used to determine the “Fourth Factor” that I have seen. Did you test that version as well?
I was focusing on FTA/FGA because that’s how the main college stat guys define it (Ken Pomeroy, Statsheet, Team Rankings, etc). I just ran the numbers for FTM/FGA in college from 2004 til 2010 in the same format as the regressions above and it was slightly worse (but very close to) FTM/Poss. The R^2 was .8044.
FTM/Poss and FTM/FGA are very, very close, and both seem to be superior to FTA/FGA. I do think there is some added benefit to the denominator being possessions instead of FGA, however.
The main problem people have using FTM is not on the offensive side, but on defense – if you use FTM, you’re explicitly assigning credit/blame to the defense based partly on something that is completely out of its control (opponent FT%).
Ideally you’d use FTM on offense and FTA on defense, but the four factors work better when parallel. I feel like the best way might be to keep the traditional four but add a ‘fifth factor’ on offense, FT%.
As far as the denominator is concerned, FGA is a better measure because it is independent of any of the other three factors. If you use possessions as the denominator, you’ll conflate FT rate with turnovers (more possessions without a real chance to draw free throws, since most FTA come on shot attempts). No other factors are interconnected like that; sure, a good ORB team will probably have a better eFG%, because it gets more shots on the inside, but that’s not a direct connection like turnovers and possessions. I suppose it comes down to how likely shooting fouls on non-shot attempts are – with more of them, FGA is a less accurate denominator; with fewer, more accurate – but I think I prefer FGA to keep the factors independent.
After thinking some more and looking over your stuff again, I’m actually agnostic on the FTA vs possessions debate for the denominator, at least until I see a comparison of shooting-foul-FTA vs non-shooting-FTA. But I still support the ‘fifth factor’ idea.
Agree about the complications on defense. It is an interesting question: should we give up some predictive power on the offensive side to keep symmetry?
As for the argument about the independence of the four factors, we can actually partially test that using interaction terms. The correlation between FTM/Poss and TO% is -.137. A regression like the one used above with an interaction term TO% times FTM/Poss shows that the interaction term is not a significant predictor of Off. Rating (p=.081). The correlation between FTA/Poss and FTA/FGA (simply adding turnovers to the denominator) is .935. Could you maybe say more about why you think using possessions in the denominator might bias a FT Rate stat? Perhaps find a team that fits or propose a hypothetical team?
I agree that theoretically, all of the four factors should be independent. FT Rate is actually more highly correlated with offensive rebounding (.23) than any other factor. Using possessions in the denominator does not seem to make the Free Throw factor much more highly correlated with the other three than FTA/FGA is.
I agree with the various comments made by other commenters: the Poss is not a good idea, because TO are now also included, removing the independence of the 4 factors. Perhaps FTM/FGA on the offensive end… but the defensive end is different, because FT% against is scarcely controlled by the team.
I wish I had seen this before responding to Kevin’s comment. Again, maybe it’s just me, but I’m having trouble seeing how possessions in the denominator removes the independence of the four factors. Can you give an example of a hypothetical team where this would bias the FT component? Or maybe propose a way to test this empirically? I just don’t see it in the data.
Basically, we’d like to have orthogonal components, right?
TO% is not related to EFG% or to ORB%. Those 3 are definitely orthogonal to one another.
TO% is related strongly to FGA/Poss, as is ORB%. If you measure FT/Poss, you are measuring something related to TO% and to ORB%.
Looking at it from a “tree” perspective: TO% measures how many first shots you take on a possession. ORB% measures how often you get a chance at second shots. (Hmmm… TO% may be related to that after all, since you can get a shot, rebound it, and then have a turnover which ends up shown as a possession ending in a TO). EFG% measures what happens when you shoot it. FT_/FGA measures, basically, how many free points you get besides that influenced by EFG%.
Personally, I’d rather use 3 factors: TO%, ORB%, and TS%. Forget the confusion!
Thanks for the prompt reply. Everything you say makes sense in theory. TO% should not be related to eFG%, but in my data set, it is. The correlation is -.291. There must be some lurking, unobserved variable (something about the skill of the players) that relates the two. Very interestingly, if I restrict the sample to those teams that have an adjusted offensive rating of 105 or greater (call them above average offensive teams), the correlation goes to 0 as you would expect (0.0081). It seems like eFG% and TO% are not orthogonal for bad offensive teams.
I guess what I’m saying is that theoretically the Four Factors are pretty much unrelated, but at least in the college data, there are some relations. I just don’t see the theoretical concern about double counting as being borne out in the data.
It seems like all of the FT metrics are flawed. There must be a better way that we just haven’t hit upon/don’t have the data for yet.
Or maybe bad teams are just bad at everything?
Real quick, in response to comments above by Kevin and DSMok1:
I don’t think it is the case that teams have no control over opponent FT%. Teams and players sometimes come to the decision to foul out in normal defensive strategy, and you can rate teams as ‘good’ or ‘smart’ in this category if they send worse shooters to the line and avoid technicals and fouls on threes.
There are coaches who are famous for using fouls as part of defensive strategy like Sloan and Pat Riley. Since they call for rougher defense on certain parts of the floor, against certain kinds of players or in certain situations, you might find a persistent effect on opponent FT% from the change in who’s being fouled.
What sort of algorithm are you using to calculate your possessions?
I’ve seen this:
FGA – ORB + TO + .475 (FTA)
… but I’m not really sure where the .475 coefficient comes from. I’ve seen .4 as well… but I have no idea how accurate these are.
Can you provide any assistance on that?
Pingback: Site Update: New College Basketball Stats & College Basketball Section Refresh « Notes from the Sports Nerds
As an intro, I am a former college player, current coach, undergrad math major, MIT grad degree who has been dabbling in hoop stats as a practical indicator for how to advise playing strategy. My college coach was calculating PPP in the early seventies so I have been very aware of the tempo approach from long before Dean’s book. My biggest issue with his fourth factor is that FGA is a misleading data element because of the convention that a “FGA when fouled” is only an official FGA when it goes in. It is better to look at basket-attempts as a denominator. I invite the stats community to look at finding/using a constant that splits FTA into shooting fouls and other fta and use that to get tot the real decisions a player and coach make about emphasizing drives/inside play vs jumpers. Check the results from a season of intense charting by the Kings posted on 82games.com to see what a huge data difference there is in shot type and getting to the line. The defensive advantage to FTA is also real especially when FGM. The differences are much greater as you go lower in the hoop chain from pro to college to HS to AAU to youth. Hoop stats won’t get the relavance deserved until kids’ coach’s grasp the simplest messages in the way the OBP message has finally gotten to the little league coaches. If I have missed that these points are in the literature, please help me as a math literate coach to find them.