After my first glance analysis of the 2014 FIFA World Cup Draw on Monday, a friend told me about some other soccer rankings besides FIFA’s, namely the World Football Elo Ratings system, which like my friend is super terrific. There are other methods, such as least squares rankings (similar to football’s Pythagorean wins), or the FIFA Women’s ranking system1, or Nate Silver’s ESPN Soccer Power Index. But the Elo system is probably the best.
Dr. Arpad Elo developed the rating system used in the international chess federation, and in the 1990s Bob Runyan made some tweaks, adapting it to international soccer. It has since adapted further, incorporating the stakes of the match (a world cup match counts much more than a friendly), the relative strength of the opponent (if 233rd ranked Palau and second ranked Spain both defeat first ranked Brazil, Palau gets a much bigger boost), the margin of victory (winning 10-0 is better than winning 10-9), and home field advantage (a little more complicated, but not much: just read the formula yourself). My first impression was that Elo would be a little skewed by past results; after all, matches from the 1930s would be in there, but what does that have to do with how good a team is today? In fact that’s not the case. The relative strength of a team’s opponent firmly controls for more recent results, and if you start beating good teams (or losing to bad ones) on a regular basis, your rating starts to reflect that pretty quickly. It’s expected that after 30 games, the rating is accurate. Which is why Bosnia-Herzegovina, which after the dissolution of Yugoslavia played its first FIFA game in 1995, has been able to climb to 24th in the Elo Ratings since that time.
So, is Elo really the best? Yeah, pretty much. Its predictive power is measurably stronger than the other ranking systems in use. Jan Lasek, Zoltan Szlavik, and Sandjai Bhulai, mathematics and computer science students at various European universities, conclude that Elo is the best in their 2012 paper “The Predictive Power of Ranking Systems in Association Football”. Over all international FIFA Men’s matches played between 2006-2012, the Elo system was the best predictor of outcomes, closely followed by the FIFA Women’s system (which itself just uses different tweaks on a basic Elo model). Many other ranking systems were better than the FIFA Men’s system at predicting the outcome of matches.
So back to the results of the draw last Friday, this time armed with FIFA rankings, Elo ratings, and Nate Silver’s SPI because it wasn’t examined in that European study and Nate Silver is a pretty smart dude.
Remember when I said that Pot 3, consisting of Asia and North and Central American teams, clearly contained the worst teams, even after saying that FIFA rankings were silly? Well, that was silly of me, because the Elo system suggests it’s actually Pot 2, containing the non-seeded South American and all qualifying African nations, with an average Elo rating of 1,726 (average rank 32.1), to Pot 3’s average of 1,732 (avg. rank 29.8). That’s a very slight difference, while the Europeans in Pot 4 average is 1,856 (13.6), and the seeded European and South American teams in Pot 1 average is 1,961 (7.1). The best teams are in South America and Europe. The trouble is that the non-seeded South American teams go into Pot 2 with the African teams. Remember how Pots 2 and 3 had really similar average ratings? The standard deviation in Pot 3 is 55, while in Pot 2 it’s nearly twice that, 105, indicating much more variation among the quality of its members. In Pot 2 a seeded team could draw Algeria, Elo rating rank 59th, or Cameroon, 56th, but also Chile, 10th, or Ecuador, 16th. With five teams from Africa, but only four teams from Asia or North and Central America, it’s not as easy as switching them, but I suspect FIFA could do a little bit better.
As for the original question, which group is toughest? I called it a draw earlier between Group D (Uruguay, Costa Rica, England, Italy) and Group G (Germany, Portugal, Ghana, USA), but now I’m calling it Group D, straight-up like. The average Elo rating of teams in D is 1,852, behind G’s 1,877 and B’s (Spain, Holland, Chile, Australia) 1,916. But the standard deviation in D is only 78, while in G it’s 129 and in B 136. Two groups have lower standard deviations than D: E (Switzerland, Ecuador, France, Honduras) with 74, and C (Colombia, Greece, Ivory Coast, Japan) with 61. The teams in C & E aren’t as good though, as those groups rank in the lower half of average rating among the eight groups. Group D is the only one in the top half of quality (3rd best teams of any group) and equality (3rd most evenly matched).
Nate Silver’s algorithm strongly backs this up. Using his Soccer Power Index, teams in Group D combine for an average ranking of 14th in the world, ahead of G’s 15.3. Group D also has the lowest standard deviation, 6.6, while G has the second lowest, 7.2. So according to Silver’s numbers, D has the best teams and is the most evenly matched; G is a close second on both counts. I guess you could call Group G a group of death, but Group D is The Group of Death.
Just for fun, what is the worst that could have happened to the USA? According to the Elo system, it’s getting drawn by Brazil (Elo rank 1st), along with Netherlands (5th) and England (7th). According to Nate Silver, it’s getting drawn by Brazil (Silver’s rank 1st), along with Chile (5th) and France (7th). Of course, according to FIFA it’s Spain (1st), Portugal (5th), and Italy (7th). None of those groups happened, but they all could have, and were about as likely as the USA getting Belgium (Elo rank 18th), Algeria (59th), and Bosnia-Herzegovina (24th), their easiest possible group according to Elo.2
Last but not least, for which countries did the rankings diverge, and for which did they agree? Algeria was 59th in Elo, 69th in Silver’s, and 26th in FIFA, for a standard deviation of 18.4, easily the highest among all 32 teams. Here are the Top 5 most divisive teams (standard deviation of different world ranks in parentheses):
- 1. Algeria (18.4)
- 2. Australia (11.6)
- 3. Japan (9.6)
- 4. Cameroon & Korea Republic (7.1)
And here are the Top 5 teams the various ranking systems agreed on the most:
- 1. Argentina, Germany, and Spain (0.8)
- 4. Colombia (0.9)
- 5. Uruguay (1.2)
Actually, the USA was the sixth most agreed upon, with a standard deviation of its various world rankings at only 1.7. And that’s with an average rank of 14.7. A reason for optimism come June? Maybe. No matter what, USA ALL THE WAY! WHOOOOOOOOOOOOOOOOOOOO! Over and out.