Fairness and Rankings

One of the reasons people accept playoffs as a post-season solution is because the sports and leagues which hold them are able to fairly rank and include the most deserving teams. If this wasn’t the case there would be a lot more arguing and outrage when seeds and brackets came out. Sure you might hear some grumbling during March about teams that should’ve been in the NCAA Basketball Tournament but weren’t picked for whatever reason. But for the most part, fans are fine with the post-season setup most sports use. So how are these other sports able to fairly rank and include the most deserving teams in their playoff? And why can’t college football do the same?

My argument here is that a low number of games, a high number of teams, and only playing once a week separates college football from other sports, and the combination of these things makes it nearly impossible to implement a fair playoff. I’ll break this argument down in a moment, but first a few things about the word “fair”.

Fairness, defined here as free from injustice, is an extremely subjective idea, especially when dealing with a competitive topic like sports. Because it’s subjective, there’s no such thing as 100% fair when it comes to rankings or deciding a national champion. There’s varying degrees of fairness, mostly dependent on the person or people using the term. By using the term “fair” in reference to rankings or a playoff etc., I’m simply referring to a situation in which most reasonable people would agree that the most deserving teams are ranked properly or included in the playoff. In addition, using the term “fair playoff” doesn’t mean that I think that playoffs are inherently fair or even more fair than any of the other definitions in the previous section. So with that clarification out of the way...

In order to rank teams fairly, you need information about how they’ve fared in relation to one another. Single games between two teams can be used to determine who is better on any given day, but many games are needed to determine who is better over the course of a whole season. It stands that the more games that are played, the better the chance is of being able to fairly rank teams because there’s more information available. At the same time, the opposite is true when dealing with the number of teams involved. As the number of teams goes up, the chances of putting them all into a fair ranking goes down. To illustrate this idea, let's take a look at some examples.

The table below lists all of the team sports which the NCAA awards national championships to as well as the four major professional sports. They're ranked by the average number of games they play in a season from highest to lowest.

Sport	# Teams	# Games	# Playoff Teams	% of Teams
MLB	30	162	8	26.67%
NBA	30	82	16	53.33%
NHL	30	82	16	53.33%
Baseball (M)	284	55	64	22.54%
Softball (W)	269	50	64	23.79%
Ice Hockey (M)	58	40	16	27.59%
Ice Hockey (W)	33	34	8	24.24%
Basketball (W)	325	32	64	19.69%
Basketball (M)	326	32	65	19.94%
Volleyball (M)	22	30	4	18.18%
Volleyball (W)	315	30	64	20.32%
Water Polo (M)	21	28	4	19.05%
Water Polo (W)	31	28	8	25.81%
Field Hockey (W)	78	21	16	20.51%
Soccer (W)	301	20	64	21.26%
Lacrosse (W)	81	18	16	19.75%
Soccer (M)	196	18	48	24.49%
NFL	32	16	12	37.50%
Lacrosse (M)	56	13	16	28.57%
D-1A Football (M)	118	12	-	-
D1-AA Football (M)	116	11	16	13.79%
D-II Football (M)	156	11	24	15.38%
D-III Football (M)	235	10	32	13.62%

In Major League Baseball (MLB) there are 30 teams which play 162 games apiece – lots of opportunities to compare a relatively small number of teams. On the other hand, NCAA Women's Soccer has 301 teams playing 20 games each – too few opportunities to compare too many teams. A ranking of MLB teams has a better chance to be more accurate, more fair, and less controversial than a ranking of Women’s Soccer teams simply because there’s a lot more information available to analyze when comparing the teams. Fair rankings = fair playoff.

So what about those sports with a low number of games? If there’s not enough information to make their rankings reasonably fair, then how do they ensure that that they draw the cutoff line for their playoff fairly and include most (if not all) of the deserving teams? By increasing the number of teams that get into the playoff.

The aforementioned Women’s Soccer lets 64 teams into its playoff, over 21% of its sport’s teams. So even if their initial ranking of teams isn’t that fair, letting so many teams into the postseason makes their playoff reasonably fair. Thus a high number of playoff teams acts as a failsafe to catch teams that an unfair ranking might let slip through. Excluding college football, the lowest percentage of playoff teams is Men’s Volleyball at a little over 18%. But because their 22 teams play 30 games each, there is little chance that their rankings will be unfair, thus there’s less need for more playoff teams. All of the sports or leagues listed either play a lot of games, have few teams, or include a large number of their teams in the playoff. All of the sports except college football that is.

The four divisions of college football play the fewest games on the list as well as having the lowest percentages of playoff teams. Part of the reason for these low numbers is the third factor – that college football teams only play one game per week. This limits both the number of games that can be played during the regular season and the number of teams that can realistically be included in a playoff. A four or six round playoff in other sports might take two or three weeks, but in college football a four week playoff takes four weeks. This might not seem like a big difference, but since most playoffs are single-elimination, the process is exponential. Two rounds is the difference between having a 16 team playoff and a 64 team playoff.

(I know what you might be thinking – what about the NFL? They only play once a week. True, but take a look at their number of games, number of teams, and percentage of playoff teams. They have far fewer teams, (college football has nearly 4 times as many teams in even the smallest division), and they play four or five more games than college football teams do, giving them anywhere from 33%-60% more information to use. If they used rankings they would be fairer to begin with, and they have a solid failsafe by including a high percentage of their teams. But the whole point is rather moot because their playoff teams are chosen based on winning percentage alone. All of those things contribute to the fairness in their playoff system.)

So basically, when compared to other professional and NCAA team sports, college football has too many teams playing too few games to come up with a reasonably fair ranking, and it cannot create a large enough playoff to act as a failsafe for the unfairness of those rankings. A lot of people moaned and groaned before the BCS about the perceived unfairness of two national co-champions, and even more people have cried out over the unfairness of the BCS. But if the unfairness of the current system is one of the reasons you're calling for the demise of the BCS, calling for a playoff (which would be just as unfair) is quite hypocritical.

Here’s the big thing – rankings are more important to college football than to any other sport. This is partly because they are a tradition of the sport, giving people something to think about, talk about, and argue about as the season progresses. The rankings have considerable power, but as we’ve seen, they are also inherently flawed. Not a good combination. Is there a single ranking system out there that everyone wholeheartedly agrees upon? No way – if there was, we’d use it. There never was, and there never will be. The closest we’ve ever got was using both the AP & Coach’s polls (see Definition #3 on my Versions of the BCS page and its corresponding issue). In a way, college football rankings are like the metaphoric super-athlete: you cannot stop them, you can only hope to contain them.

So if we want to reduce the importance of these flawed rankings, all we have to do is institute a playoff, right? Wrong – so very, very wrong. Before the late 1990’s we were able to contain the rankings by giving them only one task – choose a single team to be the national champion. Sometimes this didn’t work, and the polls chose two teams. But those years were few and far between (only 4 times in the past 30 years for a failure rate of 13%). But then the BCS came along, and the rankings went from being the real Bo Jackson to the Bo Jackson in Tecmo Bowl – uncontainable. We gave the rankings more power, not less, because we doubled their task: choose the top two teams instead of just the top one. Most everyone would agree that the BCS rankings have not worked most of the time. In 9 years they’ve failed to produce top 2 teams a majority of people could agree upon 5 times, a failure rate of 56%. There’s mayhem and chaos when the BCS tries to rank the top 2 teams – can you imagine the uproar that would ensue if we doubled or quadrupled their nearly impossible task yet again, giving them 4, 8, or even 16 teams to rank? There wouldn’t be a single year in which a majority of people agreed that the rankings got it right. The result would be like a whole team of Tecmo Bowl Bo Jacksons – completely unstoppable and so unfair you might as well not even play the game. We’d focus our attention on them even more, giving them increased power and exposing their significant flaws at every turn. I’m all for letting teams sort it out on the field, but in order to do that properly you have to rank them fairly first, something we’ve proved rather inept at doing.

Yes, you over there – you have a question? What? What about Divisions I-AA, II, & III? Ah, yes. All three of them do hold playoffs, don’t they. If it’s good enough for them, why isn’t it good enough for D.I-A? And doesn’t that contradict everything I’ve been writing in this section, blowing my whole “a college football playoff can’t be set up fairly” argument out of the water? I’m glad you asked. The next sections will explain how the fact that the three lower divisions hold playoffs can be used to support my argument that playoffs are not suitable for D.I-A.

Defining a Champion < Top > Tradition