Reasonable Rankings for College Football: Part II
In response to reader comments, I conclusively demonstrate that CFP rankings ignore home field advantage and scoring margin.
A few weeks ago I proposed reasonable rankings for college football, which used least squares regression with a logistic function of the scoring margin. The ranking system boiled down to a single parameter, 𝜆, which represented how much the scoring margin actually mattered. When 𝜆 was small, the scoring margin mattered much less than getting a win; when 𝜆 was large, the scoring margin mattered more. In the rankings I published, I used a value of 1 point for 𝜆 based on “feel.”
There were some great reader comments on the piece, but one in particular inspired this sequel: “I’m curious what 𝜆 minimizes the error function of the linear regression. I wonder how the rankings would ‘feel’ with this 𝜆.” When I tried out larger values, the rankings frankly didn’t look right to me, as they seemed to unfairly reward teams who ran up the score against their opponents. But this reader was quite right—what made 1 a better choice for 𝜆 than, say, 3, or 7?
Inspired by this question, I looked at how rankings based on different values of 𝜆 agreed or disagreed with the CFP rankings. While I was at it, I also revisited my assertion that home field advantage didn’t matter when it came to the rankings.
But before we get to all that, let’s revisit some of the games that were played since I last wrote on this subject.
Playoff Results
Four CFP First Round games were played on December 20 and 21. Three of these pitted an unambiguously stronger team against a weaker team (the numbers in parentheses represent each team’s seed in the playoff):
(5) Texas defeated (12) Clemson, 38-24. Prior to this game, Texas was 3rd in the CFP rankings while Clemson was 16th, having limped in as the ACC conference champion. These rankings closely mirrored my own—I had Texas 4th and Clemson 15th.
(6) Penn State defeated (11) SMU, 38-10. For these two teams, the CFP and reasonable rankings were again quite close. Penn State was 4th in the CFP rankings and 5th in mine. SMU was 10th in the CFP rankings and 8th in mine.
(7) Notre Dame defeated (10) Indiana, 27-17. The committee ranked Notre Dame 5th, while I had them 3rd. They ranked Indiana 8th, while I had them 6th.
However, I found the fourth game to be much more interesting with respect to the rankings:
(8) Ohio State defeated (9) Tennessee, 42-17. According to the CFP rankings, this game shouldn’t have been a blowout. Ohio State was ranked 6th, having gone 10-2 in the Big Ten, while Tennessee was ranked 7th, having gone 10-2 in the vaunted SEC. But my rankings strongly disagreed with those of the CFP—for both teams. I had Ohio State ranked 2nd, thanks to their two narrow losses to great (Oregon) and better-than-you-think (Michigan) teams. And I had Tennessee ranked 23rd, since their 10-2 record was more a product of who they played rather than how they played.
So yeah, I was quite right about Tennessee in my last column.
Home Field Advantage
Okay, let’s talk about home field advantage. If there were no such thing as home field advantage, then if you looked at the distribution of the home team’s score minus the visiting team’s score, you’d expect to find a mean of zero. In my last column, I mentioned that in 2024 this mean favored the home team by almost 9 points—a clear home field advantage.
As noted by a reader, this figure was skewed in part by some extreme cases in which the strongest teams invited non-conference (or even FCS) opponents to their home turf for a beating. Neil Paine reported a more modest home field advantage of roughly 3 points for in-conference games.
That said, I offered anecdotal evidence that the CFP committee didn’t actually care about home field advantage. Exhibit A was when, in week 7, Oregon defeated Ohio State at home by just 1 point—a result that would have corresponded to a loss for Oregon at a neutral site. But no committee would discredit Oregon for a win, albeit the narrowest one possible and on home turf, and so Oregon moved up in the rankings while Ohio State moved down.
But let’s transcend the anecdotal evidence, and look instead for statistical evidence. To do this, we can assume different values for home field advantage and see which produce reasonable rankings that are most consistent with the CFP rankings. To measure consistency, I compute the root mean square error in the ordinal rankings. For example, if Ohio State is ranked 2nd given a particular home field advantage, while the CFP committee has Ohio State 6th, then the contribution to the square error is (2−6)2, or 16. I add these contributions up for all 25 CFP-ranked teams, divide by 25 to get the mean square error, and take the square root. A lower value means a stronger correspondence.
The graph below shows consistency with CFP rankings immediately preceding bowl games from 2014 to 2024, assuming various values for home field advantage between 0 and 10 points. To be clear, a home field advantage of H meant that H points were removed from a team’s score when they played at home, H was added to a team’s score when they were away, and scores were unchanged at neutral sites. Throughout, I used a very small value 𝜆 (0.01), prioritizing wins over margin of victory.
The average of these (shown in red) made the overall trend quite clear: This variation was minimized when H was near zero. In other words, as I’ve been saying all along, the CFP committee does not seriously care about home field advantage.
Scoring Margin
The original reader suggestion was to conduct a similar analysis for 𝜆. That is, for which value of 𝜆 did the reasonable rankings best resemble those of the CFP committee? Ignoring home field advantage (and rightly so, as we just saw), here was the consistency with the CFP rankings when 𝜆 was between 0 (i.e., very small but positive) and 14:
For a couple of years (2015 and 2019), a value of 3 seemed appropriate. In other years (2022, in particular), a value of 0 resulted in the greatest consistency with the CFP committee by far.
The average of these results (shown in red) revealed a trend that values of 𝜆 closer to 0 were more consistent with the CFP rankings. On average, the committee cares about wins a lot more than scoring margins.
That said, I would still advocate for a modest, nonzero value like 1. Ignoring scoring margins completely means discarding meaningful information.
Outlook
As we have seen, the CFP committee does not appear to care about home field advantage. They also seem to care a lot more about wins than scoring margins. None of this surprised me, but I found this a worthwhile exercise.
So, what does the remainder of the playoff look like, now that the first round is complete? Here are the current top 25 teams according to their reasonable ranking, based on all games (including bowl games) played through Sunday, December 29th:
The lackluster first round playoff performances by Clemson, SMU, and Indiana dragged these teams down, with SMU taking the biggest hit. Meanwhile, Tennessee performed as expected (by the reasonable rankings), so it remained at #23.
Of the four upcoming quarterfinal matches, I find two of them rather uninteresting (again, numbers in parentheses indicate playoff seed):
(6) Penn State vs. (3) Boise State. I have Penn State as the 5th best team to Boise State’s 13th best. I wouldn’t be surprised if Penn State runs away with it, as they did with SMU.
(5) Texas vs. (4) Arizona State. Texas is the stronger team according to my rankings. I, along with almost everyone else, expect a Texas victory.
The other two matches are more intriguing in my opinion:
(7) Notre Dame vs. (2) Georgia. I find this one interesting because SEC-champion Georgia is the consensus #2 team in the country. Meanwhile, I now have them at #6, three slots below Notre Dame. I’d personally expect a Notre Dame victory, with the belief that Georgia is buoyed in the rankings due to SEC bias. Of course, either team can win (duh), while the betting lines appear to be about even.
(8) Ohio State vs. (1) Oregon. This is the most interesting game to me, since I have these as the top two teams in the country (again, when 𝜆 = 1). Oregon’s index is a good deal higher than Ohio State’s, but it’s nevertheless a pity that these two teams are meeting in a quarterfinal, rather than in a semifinal or final. (And Nick Saban agrees.)
One more game to keep an eye on is Alabama vs. Michigan at the ReliaQuest Bowl on New Year’s Eve. Alabama (9–3) was ranked 11th by the CFP committee, while Michigan (7–5) was unranked, despite its victory over Ohio State. I have these two teams ranked much closer (with Alabama currently at #12 and Michigan currently at #18), due to my lack of bias regarding Alabama and my giving some actual credit to Michigan for their victory over Ohio State.
Projecting matchups is all good fun, but I am at risk of straying from my thesis. So here it is, one more time: Statistical rankings can be generated in a straightforward way, and can reflect expert opinions to a reasonable degree. Such reasonable rankings represent an improvement over the implicit bias of a selection committee.