The BCS Computer Rankings
Now that the first BCS rankings are out, it’s time to refresh what we know about what some people see as their more mysterious aspect – the computer rankings. Mainly because they can take a lot more raw data into account when comparing teams and deciding how to rank them, I’m definitely in favor of having computers play some role in the BCS standings. But a lot of people aren’t, quite possibly in part because they don’t understand where they’re coming from. And it’s true, you do have to be able to look at things from a different perspective to understand the computers. So let’s take a look now and see if we can’t quelch some of the screaming that’s already started, shall we?
Just like human pollsters, the computers rarely agree with each other completely when it comes to rankings, which is both a good thing and a bad thing. Good in that it helps average everything out, making the computer component more of a consensus vote than if one computer had all the power, bad in that people can pick apart and denigrate each computer ranking individually. Even though there are flaws with them, just as there are with the humans who choose rankings in the polls, one computer ranking isn’t heads above the others in terms of correctness or reasonability. They all have their flaws as well as they’re strengths.
Let’s take a look at the six computer rankings that the BCS is currently using, explaining and assessing them a bit. By no means is this a comprehensive look - this is just to help you understand a bit more about them, and I hope it'll get you to go to the individual rankings' websites where there's a lot more information. (As always, the links to all of the computer ranking pages are to the left under “A Few Good College Football Links”.) Of course the exact formulas the computer rankings use are pretty well guarded by their creators, understandably, but most of them do publish descriptions of their methods. I’ll try to decipher some of them here - if I seem to misrepresent your formula it's unintentional, so please let me know.
Below is a table (imagine that) detailing the final regular season rankings for the two main polls and the current six computers. If a team is in red, it means they were NOT conference champions or co-champions – all other teams (in black) were.
| BCS Computers Final Regular Season Rankings | ||||||||
|---|---|---|---|---|---|---|---|---|
| Yr/Rk | AP | Coach's | Anderson & Hester | Billingsley | Colley | Massey | Sagarin | Wolfe |
| 1998 #1 | Tennessee
(12-0) | Tennessee
(12-0) | Tennessee
(12-0) | Florida St
(11-1) | Tennessee
(12-0) | Florida St
(11-1) | Kansas St
(11-1) |
|
| #2 | Florida St
(11-1) | Florida St
(11-1) | Florida St
(11-1) | Tennessee
(12-0) | Florida St
(11-1) | Tennessee
(12-0) | Tennessee
(12-0) |
|
| #3 | Ohio St
(10-1) | Ohio St
(10-1) | UCLA
(10-1) | Ohio St
(10-1) | UCLA
(10-1) | UCLA
(10-1) | Florida St
(11-1) |
|
| #4 | Kansas St
(11-1) | Kansas St
(11-1) | Kansas St
(11-1) | Kansas St
(11-1) | Texas A&M
(11-2) | Texas A&M
(11-2) | UCLA
(10-1) |
|
| #5 | Arizona
(11-1) | UCLA
(10-1) | Arizona
(11-1) | Wisconsin
(10-1) | Kansas St
(11-1) | Kansas St
(11-1) | Texas A&M
(11-2) |
|
| 1999 #1 | Florida St
(11-0) | Florida St
(11-0) | Florida St
(11-0) | Florida St
(11-0) | Florida St
(11-0) | Florida St
(11-0) | Florida St
(11-0) | Florida St
(11-0) |
| #2 | VA Tech
(11-0) | VA Tech
(11-0) | Nebraska
(11-1) | VA Tech
(11-0) | VA Tech
(11-0) | VA Tech
(11-0) | VA Tech
(11-0) | VA Tech
(11-0) |
| #3 | Nebraska
(11-1) | Nebraska
(11-1) | VA Tech
(11-0) | Nebraska
(11-1) | Alabama
(10-2) | Nebraska
(11-1) | Nebraska
(11-1) | Nebraska
(11-1) |
| #4 | Wisconsin
(9-2) | Wisconsin
(9-2) | Alabama
(10-2) | Kansas St
(10-1) | Nebraska
(11-1) | Kansas St
(10-1) | Alabama
(10-2) | Alabama
(10-2) |
| #5 | Alabama
(10-2) | Tennessee
(9-2) | Kansas St
(10-1) | Alabama
(10-2) | Michigan
(9-2) | Tennessee
(9-2) | Tennessee
(9-2) | Tennessee
(9-2) |
| 2000 #1 | Oklahoma
(12-0) | Oklahoma
(12-0) | Oklahoma
(12-0) | Oklahoma
(12-0) | Florida St
(11-1) | Florida St
(11-1) | Florida St
(11-1) | Florida St
(11-1) |
| #2 | Miami (FL)
(10-1) | Miami (FL)
(10-1) | Washington
(10-1) | Florida St
(11-1) | Oklahoma
(12-0) | Oklahoma
(12-0) | Miami (FL)
(10-1) | Washington
(10-1) |
| #3 | Florida St
(11-1) | Florida St
(11-1) | Florida St
(11-1) | Miami (FL)
(10-1) | Washington
(10-1) | Miami (FL)
(10-1) | Oklahoma
(12-0) | Oklahoma
(12-0) |
| #4 | Washington
(10-1) | Washington
(10-1) | Miami (FL)
(10-1) | Florida
(10-2) | Miami (FL)
(10-1) | VA Tech
(10-1) | Nebraska
(9-2) | Miami (FL)
(10-1) |
| #5 | Oregon St
(10-1) | VA Tech
(10-1) | Oregon St
(10-1) | VA Tech
(10-1) | VA Tech
(10-1) | Washington
(10-1) | VA Tech
(10-1) | Oregon St
(10-1) |
| 2001 #1 | Miami (FL)
(11-0) | Miami (FL)
(11-0) | Miami (FL)
(11-0) | Miami (FL)
(11-0) | Miami (FL)
(11-0) | Miami (FL)
(11-0) | Miami (FL)
(11-0) | Miami (FL)
(11-0) |
| #2 | Oregon
(10-1) | Oregon
(10-1) | Nebraska
(11-1) | Nebraska
(11-1) | Nebraska
(11-1) | Oregon
(10-1) | Florida
(9-2) | Nebraska
(11-1) |
| #3 | Colorado
(10-2) | Colorado
(10-2) | Oregon
(10-1) | Oregon
(10-1) | Oregon
(10-1) | Nebraska
(11-1) | Nebraska
(11-1) | Colorado
(10-2) |
| #4 | Nebraska
(11-1) | Nebraska
(11-1) | Colorado
(10-2) | Colorado
(10-2) | Tennessee
(10-2) | Colorado
(10-2) | Texas
(10-2) | Tennessee
(10-2) |
| #5 | Florida
(9-2) | Florida
(9-2) | Tennessee
(10-2) | Maryland
(10-1) | Colorado
(10-2) | Stanford
(9-2) | Colorado
(10-2) | Florida
(9-2) |
| 2002 #1 | Miami (FL)
(12-0) | Miami (FL)
(12-0) | Ohio St
(13-0) | Miami (FL)
(12-0) | Miami (FL)
(12-0) | Miami (FL)
(12-0) | Miami (FL)
(12-0) | Ohio St
(13-0) |
| #2 | Ohio St
(13-0) | Ohio St
(13-0) | Miami (FL)
(12-0) | Ohio St
(13-0) | Ohio St
(13-0) | Ohio St
(13-0) | Ohio St
(13-0) | Miami (FL)
(12-0) |
| #3 | Iowa
(11-1) | Iowa
(11-1) | Georgia
(12-1) | Georgia
(12-1) | Georgia
(12-1) | USC
(10-2) | Georgia
(12-1) | Georgia
(12-1) |
| #4 | Georgia
(12-1) | Georgia
(12-1) | Iowa
(11-1) | Oklahoma
(11-2) | USC
(10-2) | Georgia
(12-1) | USC
(10-2) | USC
(10-2) |
| #5 | USC
(10-2) | USC
(10-2) | USC
(10-2) | Iowa
(11-1) | Iowa
(11-1) | Wash. St
(10-2) | Iowa
(11-1) | Iowa
(11-1) |
| 2003 #1 | USC
(11-1) | USC
(11-1) | Oklahoma
(12-1) | Oklahoma
(12-1) | Oklahoma
(12-1) | LSU
(12-1) | Oklahoma
(12-1) | Oklahoma
(12-1) |
| #2 | LSU
(12-1) | LSU
(12-1) | LSU
(12-1) | LSU
(12-1) | LSU
(12-1) | Oklahoma
(12-1) | LSU
(12-1) | LSU
(12-1) |
| #3 | Oklahoma
(12-1) | Oklahoma
(12-1) | USC
(11-1) | USC
(11-1) | USC
(11-1) | USC
(11-1) | Miami (OH)
(12-1) | USC
(11-1) |
| #4 | Michigan
(10-2) | Michigan
(10-2) | Miami (OH)
(12-1) | Michigan
(10-2) | Ohio St
(10-2) | Ohio St
(10-2) | USC
(11-1) | Miami (OH)
(12-1) |
| #5 | Texas
(10-2) | Texas
(10-2) | Texas
(10-2) | Miami (FL)
(10-2) | Florida St
(10-2) | Michigan
(10-2) | Michigan
(10-2) | Michigan
(10-2) |
| 2004 #1 | USC
(12-0) | USC
(12-0) | Oklahoma
(12-0) | Oklahoma
(12-0) | USC
(12-0) | USC
(12-0) | Oklahoma
(12-0) | Oklahoma
(12-0) |
| #2 | Oklahoma
(12-0) | Oklahoma
(12-0) | USC
(12-0) | USC
(12-0) | Oklahoma
(12-0) | Oklahoma
(12-0) | USC
(12-0) | USC
(12-0) |
| #3 | Auburn
(12-0) | Auburn
(12-0) | Auburn
(12-0) | Auburn
(12-0) | Auburn
(12-0) | Auburn
(12-0) | Auburn
(12-0) | Auburn
(12-0) |
| #4 | California
(10-1) | California
(10-1) | Utah
(11-0) | Texas
(10-1) | Texas
(10-1) | Texas
(10-1) | Texas
(10-1) | Texas
(10-1) |
| #5 | Texas
(10-1) | Texas
(10-1) | Texas
(10-1) | Boise St
(11-0) | Utah
(11-0) | Utah
(11-0) | California
(10-1) | Utah
(11-0) |
| 2005 #1 | USC
(12-0) | USC
(12-0) | USC
(12-0) | USC
(12-0) | Texas
(12-0) | Texas
(12-0) | Texas
(12-0) | Texas
(12-0) |
| #2 | Texas
(12-0) | Texas
(12-0) | Texas
(12-0) | Texas
(12-0) | USC
(12-0) | USC
(12-0) | USC
(12-0) | USC
(12-0) |
| #3 | Penn St
(10-1) | Penn St
(10-1) | Penn St
(10-1) | Penn St
(10-1) | Penn St
(10-1) | Penn St
(10-1) | Penn St
(10-1) | Penn St
(10-1) |
| #4 | Ohio St
(9-2) | Ohio St
(9-2) | Ohio St
(9-2) | Georgia
(10-2) | Ohio St
(9-2) | Ohio St
(9-2) | Ohio St
(9-2) | Ohio St
(9-2) |
| #5 | Notre Dame
(9-2) | Oregon
(10-1) | Oregon
(10-1) | Auburn
(9-2) | Oregon
(10-1) | VA Tech
(10-2) | Oregon
(10-1) | Oregon
(10-1) |
| 2006 #1 | Ohio St
(12-0) | Ohio St
(12-0) | Ohio St
(12-0) | Ohio St
(12-0) | Florida
(12-1) | Ohio St
(12-0) | Ohio St
(12-0) | Ohio St
(12-0) |
| #2 | Florida
(12-1) | Florida
(12-1) | Florida
(12-1) | Michigan
(11-1) | Ohio St
(12-0) | Florida
(12-1) | Michigan
(11-1) | Michigan
(11-1) |
| #3 | Michigan
(11-1) | Michigan
(11-1) | Michigan
(11-1) | Florida
(12-1) | Michigan
(11-1) | Michigan
(11-1) | Florida
(12-1) | Florida
(12-1) |
| #4 | LSU
(10-2) | LSU
(10-2) | LSU
(10-2) | Louisville
(11-1) | USC
(10-2) | USC
(10-2) | USC
(10-2) | USC
(10-2) |
| #5 | Louisville
(11-1) | Wisconsin
(11-1) | Louisville
(11-1) | Wisconsin
(11-1) | Louisville
(11-1) | LSU
(10-2) | LSU
(10-2) | LSU
(10-2) |
| 2007 #1 | Ohio St
(11-1) | Ohio St
(11-1) | Ohio St
(11-1) | Ohio St
(11-1) | LSU
(11-2) | VA Tech
(11-2) | VA Tech
(11-2) | VA Tech
(11-2) |
| #2 | LSU
(11-2) | LSU
(11-2) | Missouri
(11-2) | LSU
(11-2) | VA Tech
(11-2) | LSU
(11-2) | LSU
(11-2) | Oklahoma
(11-2) |
| #3 | Oklahoma
(11-2) | Oklahoma
(11-2) | Kansas
(11-1) | USC
(10-2) | USC
(10-2) | Ohio St
(11-1) | Oklahoma
(11-2) | LSU
(11-2) |
| #4 | Georgia
(10-2) | Georgia
(10-2) | VA Tech
(11-2) | VA Tech
(11-2) | Missouri
(11-2) | Georgia
(10-2) | Ohio St
(11-1) | Missouri
(11-2) |
| #5 | VA Tech
(11-2) | VA Tech
(11-2) | LSU
(11-2) | Oklahoma
(11-2) | Ohio St
(11-1) | Missouri
(11-2) | Kansas
(11-1) | Ohio St
(11-1) |
Anderson & Hester: This one used to go under the moniker of the Seattle-Times poll until the 2001 season, and is one of two computer rankings to be used in every BCS ranking. It uses the tagline “Showing Which Teams Have Accomplished the Most to Date”, a goal I myself subscribe to. Their site lists four things which make the Anderson & Hester rankings different, though realistically we can only learn about what makes the A&H formula unique from one of them. “1) Unlike the polls, these rankings do not reward teams for running up scores. Teams are rewarded for beating quality opponents, which is the object of the game.” This makes it different than the polls, but not the other computers, since the BCS mandates that none of the computer ranking take Margin of Victory into account. “2) Unlike the polls, these rankings do not prejudge teams.” Not starting the rankings until October helps with some of the early-season shock that most people get when looking at computer rankings.“3) These rankings compute the most accurate strength of schedule ratings” and “4) These rankings provide the most accurate conference ratings. These last two claims are debatable and don't really help, but they’re covering the bases.
So overall, we don’t know too much about how they use the numbers. In looking at their rankings over the past ten seasons, we see a few things. First, you can see their reliance on SoS and figuring out which teams have beaten quality opponents. Undefeated BCS teams are always ranked higher than one-loss BCS teams except for one year – in 1999 they were the only computer to have 11-1 Nebraska at #2 and undefeated Virginia Tech at #3. In that year though, it’s pretty apparent that Nebraska had the tougher schedule and bigger wins, which counts for something. In 2001 they kept Nebraska at #2 over #3 Oregon & #4 Colorado, but lots of other computers did the same. Same thing with Oklahoma at #1 in 2003. Last year they had Missouri at #2, which is fine, but LSU is sitting at #5 one behind a team they beat (#4 Virginia Tech) that had the same record. Kinda iffy. But overall, it’s solid.
Billingsley: Richard explains his system with the following: “My rankings are in effect, a “power rating” and it is possible to derive a projected point spread from them by subtracting ratings, dividing by three and adding three points to the home team, however, I’m not as concerned about predicting future outcomes as I am honoring what transpired most recently on the field of play.” He adds, “I guess in a sense, my rankings are not only about who the “best team” is, but also about who is the “most deserving” team”.
He has seven distinct components, namely 1) Starting Position, 2) Accumulation of Points, 3) Strength of Opponent, 4) Deductions for Losses, 5) Rewarding Defense, 6) Game Site, and 7) Head-to-Head Rules. For a detailed breakdown of these, just go to his site – it’s a well thought out and interesting read. He does start his rankings with teams original positions based on the previous season’s rankings, but unlike other rankings, he does not discard these original positions once the season is well under way. Also, his “Accumulating” value system means that once a team earns points for beating an opponent, those points are set in stone no matter what that opponent does the rest of the year, whether they win a lot or lose a lot. Undefeated teams get much more credit than ones that have lost, and he gives bonuses for how many points teams hold their opponents to.
Looking at the rankings, everything seems pretty well in place. He had 11-1 Florida State at #1 over 12-0 Tennessee in 1998, but undefeateds always took the top spots in the years after. He also kept Michigan at #2 over Florida in 2006.
Colley: Wes’s system is pretty heavy on the math, but he explains it well, saying, “The scheme adjusts effectively for strength of schedule, in a way that is free of bias toward conference, tradition, or region. Comparison of rankings produced by this method to those produced by the press polls shows that despite its simplicity, the scheme produces common sense results.” In the paper explaining his matrix, his goal is to convince the reader that his system “has no bias toward conference, tradition, history, etc., (and, hence, has no pre-season poll), is reproducible, uses a minimum of assumptions, uses no ad hoc adjustments, nonetheless adjusts for strength of schedule, ignores runaway scores, and produces common sense results.” He also covers the reasoning behind his methods and the overall effect of certain methods on football, which is interesting as well. The equations get tougher as you go along, but it’s a good read – get as far as you can. One of the interesting things about Colley’s rankings is that they don’t include results from games against DI-AA teams, so all you cupcake eaters out there are out of luck.
In examining the rankings, Colley’s have the usual blips, but nothing too out there. He had Florida State at #1 over undefeated Oklahoma in 2000, Nebraska at #2 in 2001, and Oklahoma at #1 in 2003, but so do most other computers. Interestingly, he had Florida at #1 over undefeated Ohio State in 2006, and LSU at #1 in 2007 with Ohio State #5. It seems that SoS is pretty big with his rankings, for the most part.
Massey: Kenneth takes a different approach to the rankings, stating that, “The first challenge for any computer rating system is to account for the variability in performance. A team will not always play up to its full potential. Other random factors (officiating, bounce of the ball) may also affect the outcome of a game. The computer must somehow eliminate the "noise" which obscures the true strength of a team.” So it seems that his rankings are more a measure of actual power instead of performance. As such, he also measures things such as offensive power, defensive power, home field advantage, and SoS. But interestingly, in measuring power his rankings look solely at past performance and aren’t trying to be predictive.
Kenneth’s ratings seem to sway heavily towards the SoS component – in two years, 1998 & 2000, he has one-loss Florida State ranked higher than undefeated teams who (seemingly) played an easier schedule. He’s the only one who has Oregon at #2 in 2001, and the only computer who had LSU at #1 instead of Oklahoma in 2003. Last year, probably because of the SoS, he has Virginia Tech at #1 over the LSU team which beat them. So in general, it seems that playing a really tough schedule is the one things that can trump going undefeated in Massey’s rankings.
Sagarin: Jeff Sagarin’s rankings are probably the most famous of all the BCS computer rankings, and they’ve been included in the official measures since the beginning. I don’t know too much about Sagarin’s stuff, but here’s some tidbits: the Sagarin rankings are weighted at the beginning of the season, but once the games get going and all teams are connected he throws the initial weights out. Also, there are two different versions of his rankings – the Elo-Chess do not include margin of victory and are used in the BCS, while the Predictor does include MoV and is more accurate in predicting who will win matchups between teams. It makes for interesting reading comparing these two systems. For instance, right now (week 8, 2008) USC is #10 in the Elo-Chess, but #1 in the Predictor, which, in thinking about the margins of the Trojans’ victories, makes a lot of sense. Alas, the only thing I know about the ins-and-outs of the Elo-Chess is that it only takes wins and losses into account.
Something jumps off the page right away looking at Sagarin’s rankings – he had Kansas State #1 in 1998, which was interesting because Tennessee was undefeated and Kansas State lost (albeit barely) in the Big12 Championship game. UCLA, who also was undefeated before losing in the last week, comes in #4. In 1999 Florida State and Miami (FL) both were ranked ahead of undefeated Oklahoma, while in 2001 Florida was #2 over Nebraska, Colorado, or Oregon. Since then, his rankings have normaled out a bit, staying within the main range of computers.
Wolfe: And finally the good doctor. Peter Wolfe’s rankings are unique because they take sportsmanship into account, calling it, “A significant but hard-to-measure factor in comparing teams”. His rankings look at “all varsity teams of four year colleges that can be connected by mutual opponents”, and the method is called a Maximum Likelihood Estimate. So his rankings are power rankings in the same sense of Massey’s and Sagarin’s in that the probability is that the higher ranked team will beat the lower one.
The rankings show that Wolfe’s rankings are normal as far as computers go. He had Florida State and Washington ranked over undefeated Oklahoma in 2000, and Virginia Tech ranked #1 last year, but nothing else too out of the ordinary.
So that's a quick and dirty rundown of the computer rankings, each one unique in it’s own way and showing that it’s possible to come up with similar results from a lot of different methods. They’re not so scary now, are they?
6 comments:
This year, eight computer rankings as opposed to three last season go into helping figure out the teams that will be part of the BCS. The BCS guarantees a matchup of the two best teams as determined by a four-part point-system using national polls, computer rankings, strength of schedule and won-loss record.
-----------------------
oliviaharis
Internet Marketing
By "This year" I'm assuming you mean 1999. Either that or you're able to send messages nine years into the future.
I'm pretty sure that Colley actually includes games against I-AA opponents now; he has a pretty nice system for grouping the I-AA teams and letting the groups serve as a I-A "teams" that have played an appropriate number of games against I-A opponents.
This aspect might not be updated in the file describing the system, but it should be somewhere on the Colley Matrix website.
Yeah, you're right, OT - good catch. I went back to the website and found his explanation of why he started to include I-AA games in 2007. Here's the link for anyone who wants the specifics - http://colleyrankings.com/iaagroups.html
After a quick scan I noticed an interesting trend. The whole reason for the BCS was to handle the apparent problem of poll disagreement. In the late 80's and early 90's you had a huge ammount of disagreement. Even as late as 1997 we had Michigan and Nebraska.
But look at the polls since then. Total agreement in spots 1 and 2. Meaning that every contraversy since then has been computer related.
Yup, dethwing, that's about it. And there's a general trend back towards the polls since the beginning of the BCS too, making you wonder if the whole thing was even worth it in the first place. Check out the "Conclusions" page in my "Versions of the BCS" section for a breakdown of the numbers.
Post a Comment