Monday, April 5, 2010

The Effect of the NFL Draft on College Teams

Alrighty, what we're talking about here is the seemingly logical and oft-cited idea that the more players a college team loses in the draft, the worse they'll do the next season. Phil Steele posted in March that from his analysis, "7 out of every 10 teams that rank among the top in the NCAA in players drafted struggle the next year and have a weaker record". On the surface it makes sense, but as you'll see it's not as simple as team - good players = more losses. Let's start at the beginning.

No, there's not a concrete direct correlation between number of draftees and how much a team falls the next year. If there was, they wouldn't need to play the games. But at the same time, it's pretty logical that the better a team performs, the more players it'll have drafted, the more talent they'll have to replace. So all of these things are related, but very loosely. Another thing that we have to take into account are all the players who graduate or end their college careers without being drafted. The average FBS team has 1.95 players drafted each year, but loses anywhere between 18-26 players annually due to graduation or used-up eligibility. So of the players that a team must replace, probably less than 10% of those are drafted. Does that mean the other 90% aren't any good? No, it just means they weren't in the upper echelon at their position. Here's some other things to know about the draft for context:

In any given year, draftees usually come from around 100 different schools, 80% of those being DI-A (FBS) schools, the other 20% being DI-AA, DII & DIII. But since the BCS began, only 28 teams have had at least one player drafted every year. The average number of players drafted per team per year comes out to about 1.95 for the FBS schools - for the non-FBS schools, the average is .18.

So now with that settled, let me explain my process a bit more. I'm only going back 11 years here, starting with 1999 NFL draft, mainly because it covers the whole BCS era and should help us zero in on more recent trends. The main stat I'm using is losses, not wins or winning percentage or any other "winning" measurement. But negative numbers signal more losses, so if you see -2, it means that a team had 2 more losses the following season, not that their total number of losses went down 2.

The first thing I set out to do was double-check Steele's 7 of 10 stat, not only because he doesn't provide any explanation of how he crunched the numbers but also because I like seeing the data for myself. Roughly, when he's talking about the teams that "rank among the top in the NCAA in players drafted", I'm going to take that to mean teams that had 6 or more players drafted in any given year. It's basically a Top 10, if you average it out.

Draftees 1999-2009
Year # of team's with 6+ draftees Teams
1999 12
8: Ohio St, Florida, Nebraska
7: Notre Dame
6: Georgia, Tennessee, Virginia, Texas A&M, Kansas St, North Carolina, Clemson, West Virginia
2000 8 9: Tennessee
7: Arkansas, Florida St, Michigan St
6: Ohio St, Texas A&M, Michigan, Arizona St, Florida
2001 9 9: Florida St
8: Miami (FL), Wisconsin
7: Nebraska
6: Ohio St, Kansas St, Georgia, Notre Dame, TCU
2002 12 11: Miami (FL)
10: Tennessee
8: Ohio St, Georgia, Florida, Virginia Tech
6: Kansas St, Notre Dame, North Carolina
2003 10 8: Miami (FL), Tennessee, Florida
7: Georgia, Notre Dame
6: Colorado, Michigan, Florida St, Texas A&M
2004 7 14: Ohio St
9: Miami (FL), Purdue
7: LSU
6: Nebraska, Pittsburgh, Arkansas
2005 7 11: Oklahoma
9: Florida St
7: Wisconsin, Virginia
6: Georgia, Stanford, Louisville
2006 11 11: USC
9: Miami (FL), Ohio St, Virginia Tech
8: Florida St
7: Georgia, LSU
6: Oklahoma, Texas
2007 6 9: Florida
8: Ohio St
7: Michigan, Texas, Notre Dame
6: Tennessee
2008 7 10: USC
8: Texas, Virginia Tech
7: LSU
6: Michigan, California, Arkansas
2009 9 11: USC
7: Ohio St, Oregon St, South Carolina
6: LSU, Georgia, Oregon, Cincinnati, Missouri

If we examine how each of these teams did the following season, here's what we get (along with the other categories of teams, just to be complete).

Losses the Following Season, 1998-2009 DI-A: by draftees
Draftees # of team-seasons More L's following Same L's following Fewer L's following % more % fewer avg # of losses the following season
6+ 98 65 12 21 66.3% 21.4% -1.2
2-5 494 225 83 186 45.5% 37.7% -0.3
1 290 120 45 125 41.4% 43.1% 0.0
0 404 135 67 202 33.4% 50.0% 0.5

It appears that this method pretty much confirms Steele's 7 of 10 assertion.

But as I said, that's not the whole story. I can go along with the 7 of 10 statistic, but there's a few caveats that we definitely have to add on.

The first is that even though there might be better than a 66% chance that a team with 6+ draft picks will lose more games next year, that number is made up of teams that have a penchant for staying on top. Of those 98 teams-seasons in the table above with 6 or more players drafted in a single year since 1999, there are 39 teams represented. Of those 39, 24 of them had at least one season where they had 6 or more players drafted but lost the same or fewer games the next season. (7 of those 24 had multiple season like that, including an amazing 4 such seasons from Ohio State.) So if your team is good enough to have 6+ players drafted, they're also probably good enough to overcome it.

And you've got to remember too, we're dealing with the cream of the crop here. That means that even if they technically have more losses the following season, it is most likely a drop from going 12-1 or 11-2 to 10-3 or 9-4. The vast majority (86%) of those 98 teams that lost 6+ draftees lost 6 games or less the following season, and the majority (56%) lost 4 games or less.

Overall, of the 117 DI-A teams that have had players drafted since 1999, 107 of them have had at least one year in which they did better (fewer losses) after having 2 or more players taken in the draft. Remember the average is 1.95 players per draft, so even after losing more talent than average, they still won more. (For clarification, just because 107 teams have done it doesn't mean it's really frequent - just that it's possible for any team. Adding up some of the above numbers, teams do better after losing 2+ draftees about 35% of the time.)

My second caveat to the 7 of 10 stat is this - there are a lot of other stats to look at that correlate better with the next season's losses than draftees does.

I looked at nearly 200 different seasonal variables, tracking their correlation with number of losses the following season. And as I said at the beginning, the correlations aren't significantly strong, but they're stronger with some stats than with others. I pared down the list due to a lot of overlap (for instance, if we're looking at # of TD's scored, we don't really need to look at # of extra points attempted as well). So here are the results of the relevant stats:

*blue = offensive stats, red = defensive stats, green = draft stats
*O/D Pairing refers to which stat's variable, offense or defense, has the higher correlation

Correlations with Losses the Following Season
Rank Variable Correlation O/D Pairing
1 Losses .462
2 Win% -.447
3 Wins -.427
4 avg Margin of Victory -.330
5 Points per game -.299 off
6 Points per Play -.295 off
7 Points -.292 off
8 TO+/- -.287
9 avg spread +/- .284
10 Total TD's -.271 off
11 Yards per Pass -.255 off
12 Spreads covered -.245
13 Opp. Points per Play .241 off
14 Opp. Points per game .239 off
15 Yards per Play -.231 off
16 Total Yards -.231 off
17 Spread cover % -.227
18 Opp. Total TD's .224 off
19 Opp. Interceptions -.221 def
20 Opp. Points .219 off
21 Opp. Rushes .213 def
22 Opp. Yards per Pass .213 off
23 Rush TD's -.212 off
24 Total First Downs -.206 off
25 Opp. Rush-Net .205 def
26 Opp. Rush-Gain .202 def
27 drafted by position -.201
28 draft multiplier -.199
29 Opp. Rush TD's .198 off
30 drafted by weight -.193
31 # drafted -.193
32 Spreads not covered .193
33 Interceptions .193 def
34 Opp. Yards per Play .184 off
35 Opp. Total Yards .178 off
36 Punts .175 off
37 Punt Yards .168 off
38 Opp. Pass Attempts -.162* def
39 Opp. Yards per Rush .161 def
40 Rush-Net -.152 def
41 Yards per Rush -.150 def
42 Rush-Gain -.145 def
43 Opp. Punts -.142 off
44 Opp. Punt Yards -.138* off
45 Opp. Total First Downs .132 off
46 Rushes -.113 def
47 Fumbles Lost .105 off
48 Fumbles .084 off
49 Opp. Fumbles Lost -.067 off
50 Penalty Yards -.059* off
51 Opp. Pass Completions -.044 def
52 Opp. Penalty Yards .043* off
53 Opp. Fumbles -.042 off
54 Pass Attempts .035 def
55 Pass Completions -.035 def
56 Penalties -.030* off
57 Opp. Penalties .019* off

Right off the bat you can see that there's a whole lot of variables that correlate with the next season's losses a lot better than # drafted. Losses, Wins, and Winning% are all above .4, right at the top, then there's a pretty big drop down to Avg. MoV, the only other variable above .3.

In addition to the simple # drafted, I've also added the variables of drafted by position, drafted by weight, and draft multiplier. (For drafted by position, I assigned each position a number of importance 10-1, based on their average draft spot. For drafted by weight, the #1 pick scored a 100, then on down exponentially to the low teens for the last players taken. For the multiplier, I just multiplied the position number and the weight number.) Interestingly, of those four variables, straight # drafted had the lowest correlation.

Even more interstingly, all of the top-ranked variables have the opposite correlation you might expect. For instance, since Points per game (-.299) is negative, that means the more points per game a team scored this year, the more losses they're likely to have next year. Or since Opp. Total TD's (.224) is positive, that means the more TD's a teams' opponents scored, the fewer losses they're likely to have next year. The only relevant stats that correlate the other way, (pass attempts, penalties, and penalty yards), have asterisks. So not too generally, the better you do this year, the worse you're likely to do next year. (But again, these correlations aren't too strong overall.)

Let's look at that most relevant correlation, between losses this year and losses next year, a bit more in depth. We know that it's hard to stay at the top season after season, and that's in part because the natural pull for all teams is .500. Here's the proof:

Losses the Following Season, 1998-2009 DI-A: by previous season losses
L's per season # of team-seasons More L's following Same L's following Fewer L's following % more % fewer avg # of losses the following season
undefeated (0) 13 13 0 0 100% 0% -3.2
1 loss 40 30 5 5 75.0% 12.5% -2.0
2 losses 73 51 12 10 69.9% 13.7% -1.7
3 losses 107 62 21 24 57.9% 22.4% -1.3
4 losses 150 98 18 34 65.3% 22.7% -0.9
5 losses 205 87 47 71 42.4% 34.6% -0.2
6 losses 196 80 37 79 40.8% 40.3% 0.0
7 losses 170 48 33 90 28.2% 52.9% 0.5
8 losses 131 43 10 78 32.8% 59.5% 0.8
9 losses 104 24 14 66 23.1% 63.5% 1.2
10 losses 58 8 6 44 13.8% 75.9% 1.9
11 losses 32 1 5 26 3.1% 81.3% 2.8
12 losses 6 0 0 6 0% 100% 3.8
13 losses 1 0 0 1 0% 100% 4.0

1) The number of seasons when a team loses between 4-8 games are the majority (852 of 1,286, or 66%), and the seasons when a team loses 2 or fewer or more than 10 games are exceedingly rare (165 of 1,286, or 13%).
2) the fewer losses you have in a year the better your chances of having more losses the next season, and vice versa. (Meaning that in a 12-game season, if you lose 5 or fewer games, you're more likely to lose more than that the next season, and if you lose 7 or more games, you're more likely to lose fewer than that the next season.)
3) the number of losses you'll incur goes higher or lower as you get further away from 6, as seen in the last column.
4) all of this makes sense when you realize that a teams that wins 0 or 1 game(s) has little chance to do worse, and teams that win 12 or 13 games have little chance to do better - that's just pure probability.

So my point is that you can't really use number of players drafted as an accurate barometer of how well your team is going to do next season, but if you insist on some sort of measurement, just know that there are a lot of options that are better than that one.

(And if you're wondering, here's the table with teams, draft picks, each year, and whether or not they did better (blue) or worse (red) the following season.)

1998-2009 DI-A: draftees / # losses following season
Conf Team 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 total draftees avg per year
Big10 Ohio St 8 6 6 8 5 14 3 9 8 3 7 77 7.0
ACC Miami (FL) 3 5 8 11 8 9 5 9 5 3 1 67 6.1
Pac10 USC 5 5 3 2 5 4 5 11 5 10 11 66 6.0
SEC Georgia 6 3 6 8 7 4 6 7 4 4 6 61 5.5
SEC Tennessee 6 9 5 10 8 4 3 5 6 3 1 60 5.5
ACC Florida St 4 7 9 2 6 5 9 8 5 3 1 59 5.4
SEC Florida 8 5 4 8 8 5 3 3 9 2 3 58 5.3
SEC LSU 3 2 3 5 4 7 4 7 5 7 6 53 4.8
Big10 Michigan 4 6 5 4 6 4 3 4 7 6 2 51 4.6
Big12 Texas 3 4 4 2 4 5 3 6 7 8 4 50 4.5
Big12 Nebraska 8 3 7 4 4 6 4 4 4 3 3 50 4.5
Big10 Wisconsin 4 4 8 4 4 3 7 5 2 4 4 49 4.5
ACC Virginia Tech 2 5 3 8 2 5 3 9 3 8 1 49 4.5
Indy Notre Dame 7 1 6 6 7 5 2 3 7 4 1 49 4.5
Big12 Oklahoma 3 2 2 2 4 3 11 6 3 4 5 45 4.1
Big10 Penn St 2 4 4 2 6 4 0 6 5 2 5 40 3.6
Pac10 California 5 4 2 2 4 2 5 3 4 6 3 40 3.6
SEC Alabama 2 4 4 4 5 4 4 5 3 0 4 39 3.5
ACC Virginia 6 4 1 3 2 1 7 5 2 3 4 38 3.5
Big12 Texas A&M 6 6 5 2 6 1 3 2 0 5 2 38 3.5
SEC Auburn 2 4 3 3 0 4 5 4 5 5 3 38 3.5
Pac10 Oregon 3 2 1 6 3 4 2 4 3 3 6 37 3.4
Big12 Kansas St 6 5 6 6 4 2 1 1 3 2 1 37 3.4
SEC Arkansas 5 7 2 1 1 6 2 1 4 6 1 36 3.3
Pac10 Arizona St 2 6 4 4 3 2 3 2 2 5 2 35 3.2
Big10 Purdue 2 2 5 2 1 9 1 4 3 3 2 34 3.1
Big10 Iowa 2 2 1 3 5 5 4 2 3 3 4 34 3.1
Big10 Michigan St 2 7 3 5 1 2 4 2 3 3 1 33 3.0
ACC North Carolina 6 1 4 6 1 3 3 1 0 2 5 32 2.9
Pac10 Oregon St 3 0 4 1 3 4 3 2 2 2 7 31 2.8
Pac10 Stanford 0 2 2 6 4 3 6 4 3 0 0 30 2.7
ACC Clemson 6 0 3 0 2 2 3 3 5 2 4 30 2.7
ACC NC State 2 2 2 2 2 3 3 6 3 2 2 29 2.6
Big12 Colorado 3 4 0 5 6 2 0 4 2 2 1 29 2.6
BigEast Louisville 0 2 1 1 4 0 6 4 4 5 2 29 2.6
SEC South Carolina 0 2 0 5 3 3 3 2 2 1 7 28 2.5
Pac10 UCLA 3 1 1 6 2 4 4 3 1 3 0 28 2.5
ACC Maryland 2 2 2 3 1 5 2 3 2 1 5 28 2.5
BigEast Pittsburgh 0 1 1 3 3 6 1 2 3 3 4 27 2.5
BigEast Syracuse 3 2 4 2 2 3 1 4 2 0 2 25 2.3
ACC Boston College 3 3 1 2 3 2 0 3 2 3 2 24 2.2
ACC Georgia Tech 3 3 0 2 0 4 0 3 2 3 4 24 2.2
Pac10 Arizona 4 5 1 0 3 0 0 1 4 4 2 24 2.2
MtnWest Brigham Young 3 3 2 5 2 1 3 1 1 1 2 24 2.2
MtnWest Utah 1 3 2 2 3 0 5 2 2 0 4 24 2.2
BigEast West Virginia 6 4 0 0 2 1 3 1 0 3 3 23 2.1
MtnWest TCU 0 0 6 1 2 1 1 3 2 0 5 21 1.9
Big12 Texas Tech 2 2 1 1 2 2 3 1 3 0 4 21 1.9
SEC Mississippi 0 3 3 1 1 4 3 0 2 0 4 21 1.9
Pac10 Washington 2 2 5 3 0 4 2 1 2 0 0 21 1.9
SEC Mississippi St 4 4 4 1 3 0 2 2 1 0 0 21 1.9
BigEast Cincinnati 1 0 1 1 3 0 4 0 2 2 6 20 1.8
WAC Hawaii 0 3 1 1 3 2 1 0 5 1 3 20 1.8
Big10 Illinois 1 2 2 1 5 2 2 0 1 1 3 20 1.8
CUSA Southern Miss 1 4 4 2 1 3 2 0 0 1 2 20 1.8
WAC Fresno St 1 0 1 4 1 1 2 3 4 1 2 20 1.8
Big12 Missouri 1 1 1 0 2 0 3 2 1 2 6 19 1.7
MtnWest San Diego St 0 3 0 3 1 1 3 2 0 5 1 19 1.7
Big10 Minnesota 0 2 4 3 1 1 1 4 1 1 0 18 1.6
ACC Wake Forest 1 3 0 0 3 0 1 1 2 3 4 18 1.6
Big10 Northwestern 3 1 2 3 1 0 3 3 0 0 0 16 1.5
MtnWest Colorado St 4 2 2 0 0 3 1 1 1 0 1 15 1.4
SEC Kentucky 2 1 2 1 2 1 1 0 0 4 1 15 1.4
CUSA UCF 2 1 0 1 4 1 0 1 1 3 1 15 1.4
Pac10 Washington St 1 1 1 2 2 1 3 1 1 1 1 15 1.4
BigEast Rutgers 0 0 1 0 1 2 0 0 3 2 5 14 1.3
Big12 Oklahoma St 0 2 0 0 1 3 3 1 2 0 1 13 1.2
SEC Vanderbilt 1 0 4 0 1 0 2 1 0 3 1 13 1.2
MtnWest UNLV 1 1 0 4 0 1 2 0 1 1 1 12 1.1
WAC Boise St 0 1 1 2 1 0 0 1 4 2 0 12 1.1
MtnWest New Mexico 0 2 1 1 0 0 2 2 1 1 2 12 1.1
CUSA Marshall 0 4 1 1 3 1 1 0 1 0 0 12 1.1
CUSA Memphis 1 0 3 0 2 1 0 2 1 0 1 11 1.0
WAC San Jose St 2 0 1 0 1 1 0 0 2 1 3 11 1.0
BigEast Connecticut 0 0 0 0 0 0 2 0 1 2 4 9 1.0
BigEast South Florida 0 0 0 0 3 1 0 0 1 2 1 8 1.0
Big12 Iowa St 0 0 3 1 1 0 1 1 1 2 0 10 0.9
MAC Miami (OH) 2 2 0 0 0 2 1 1 1 0 1 10 0.9
CUSA Tulane 1 1 1 1 0 2 1 0 0 1 1 9 0.8
Big12 Kansas 0 0 1 2 0 1 1 0 0 4 0 9 0.8
Big12 Baylor 2 0 1 0 1 0 1 1 2 0 1 9 0.8
CUSA East Carolina 2 0 0 2 0 1 0 1 1 1 1 9 0.8
SunBelt Troy 0 0 0 0 2 0 1 0 1 1 1 6 0.8
WAC Louisiana Tech 1 2 0 1 0 2 0 0 1 0 0 7 0.6
CUSA Houston 0 1 0 0 1 1 1 0 1 1 1 7 0.6
MAC Northern Illinois 0 0 2 1 0 1 0 0 2 0 1 7 0.6
Big10 Indiana 0 0 0 1 1 0 1 2 0 2 0 7 0.6
CUSA UTEP 0 0 1 1 0 0 1 1 2 0 0 6 0.5
MAC Akron 0 0 1 1 0 0 1 1 1 1 0 6 0.5
WAC Idaho 0 1 2 0 0 1 0 0 0 1 1 6 0.5
MAC Toledo 0 0 0 1 1 0 1 1 0 2 0 6 0.5
MAC Temple 0 0 1 1 2 0 1 0 0 0 1 6 0.5
SunBelt Florida Intl 0 0 0 0 0 0 0 0 2 0 0 2 0.5
MAC Ball St 0 0 0 0 0 0 3 0 0 0 2 5 0.5
CUSA SMU 2 0 0 0 1 0 0 0 1 0 1 5 0.5
MAC Kent St 1 1 0 0 0 0 0 0 1 1 1 5 0.5
CUSA Tulsa 0 1 0 1 0 0 1 1 0 1 0 5 0.5
MAC Central Michigan 0 0 0 0 0 0 2 0 3 0 0 5 0.5
SunBelt LA-Lafayette 1 0 0 1 2 0 1 0 0 0 0 5 0.5
WAC Nevada 0 0 0 0 0 2 0 1 1 0 1 5 0.5
WAC Utah St 1 0 0 0 1 1 0 0 0 2 0 5 0.5
MAC Western Michigan 0 0 0 0 0 1 0 2 0 0 2 5 0.5
MAC Eastern Michigan 1 0 0 0 1 0 1 0 0 1 1 5 0.5
CUSA Rice 0 0 0 0 2 0 0 0 0 0 2 4 0.4
MAC Bowling Green 0 0 0 0 0 1 1 1 0 1 0 4 0.4
MtnWest Wyoming 0 0 1 0 0 1 0 1 1 0 0 4 0.4
SunBelt Arkansas St 0 0 0 0 0 1 0 0 1 1 1 4 0.4
CUSA UAB 0 0 1 2 0 0 1 0 0 0 0 4 0.4
SunBelt Mid Tenn St 0 0 0 1 1 0 0 0 0 1 0 3 0.3
MAC Buffalo 0 1 0 0 0 0 0 0 0 2 0 3 0.3
SunBelt LA-Monroe 0 1 0 0 0 0 1 0 1 0 0 3 0.3
MAC Ohio 0 0 0 1 0 0 0 0 0 1 1 3 0.3
ACC Duke 1 1 0 0 0 1 0 0 0 0 0 3 0.3
SunBelt New Mexico St 0 0 1 0 1 0 0 0 0 0 0 2 0.2
WAC Air Force 1 0 0 0 0 0 0 0 0 0 0 1 0.1
Indy Army 0 0 0 0 0 0 0 0 0 1 0 1 0.1
SunBelt North Texas 0 0 0 0 0 1 0 0 0 0 0 1 0.1

2 comments:

Anonymous said...

I know that this would require more research.... but should draft rd be a variable? Theory dominant players, difference makers go higher than marginal role players - later rd draftees, which would be more likely, easier to replace thus having little effect on # L's.

I wonder if there is some noise in the "draft varible" that needs to be tone down.

Ed Gunther said...

Yeah, I kinda glossed over that part a bit because the correlations for anything draft related weren't very high. But basically "drafted by weight" is what you're talking about. I didn't go by round, since that'd only give seven different "levels" (and I think would skew things, since there's a big difference between rd 1, pick 1 & rd 1, pick 32). But starting at #1 pick = 100, it goes down from there: #10 is a 92, #32 is 74, #97 is a 37, on down to 13 for the low.

What I found really interesting though, was that all of the draft correlations were within .1 of each other, while there were big variances among offensive & defensive stats (and even between the offensive & defensive versions of the same stat). And again, draftees are usually only 10-20% of a team's yearly turnover, so it could be that other turnover factors play a bigger role. So I don't think there'd be many ways to adjust or tinker with the draft variables to get them to correlate more.

Thanks for the note.