Yesterday I was conflicted. The 49ers-Cardinals and Packers-Bears games were simultaneously going down to the wire. I sat there, enthralled watching both games, with pretty much no idea what I was rooting for. I have both Packers’ and Bears’ fans as friends, and liked the potential Rodgers’ story line, but also enjoyed the completely unrecognizable Bears under new head coach Mark Tresman. Meanwhile I wanted the 49ers, who’d already clinched a playoff spot, to get the most favorable postseason match-up possible. At that point the 49ers would be either the 5th seed (if we won or tied) or 6th (if we lost). Losing meant playing Philadelphia/Chicago/Dallas (in event of PHI win/CHI win + DAL win/GB win + DAL win, respectively), while winning meant playing Green Bay/Chicago/Dallas (GB win/CHI win + PHI win/CHI win + DAL win). With no evidence, I felt that going to Lambeau Field offered the smallest chance of winning (albeit still a non-zero one), and that’s what would happen if both the Packers and 49ers won, and that’s exactly what did happen.1
The 49ers had just pulled off a game winning field goal with no time left, on the road against an Arizona Cardinals team that had just beaten the Seahawks in Seattle, and we’re getting rewarded by going to Lambeau next week and facing a freshly-returned-from-seven-weeks-healing-his-broken-collarbone Aaron Rodgers? Hmph. My mother, ever optimistic, offered the following consoling words:
[It’s] always better going to the playoffs off a win, right?
To which I replied:
I believe [that’s a] playoff myth.
Well, today I investigated that belief, my belief that winning the final game of the regular season has nothing to do with playoff success. The theoretical thinking here is that teams have pretty-well defined themselves by the of the regular season, and don’t get significantly better or worse between then and the playoffs. They’ve got a certain probability of winning their playoff games, advancing to the Super Bowl, etc, and these outcomes are independent of the final game of the regular season. I recalled that just last year, the Ravens lost their final game before winning out through the playoffs and Super Bowl.[^2] But that’s an anecdote. What does the data suggest?
I didn’t take this question too seriously, at least at first, because I don’t think the findings will reveal too much. A lot of things go into winning football games, so even if I were to find that Week 17 results had enormous predictive power of playoff results, it’s the underlying probabilities that drive them both: good teams are more likely to win in general, as well as in the playoffs. But I pulled a few basic numbers anyway. These figures go back to 2002, the first season after the NFL’s realignment into its current format. That’s 132 observations, 12 teams in the playoffs each season through 11 seasons.
Of the 132 teams, 68% had won their final game of the season. Not surprising; teams that make the playoffs usually win more than they lose, in any given week of the season. Of the 68% (90/132) of playoff teams that did win in Week 17, 58% (52/90) won their first playoff game. Of the 32% (42/132) that lost in Week 17, 52% (22/42) won their first playoff game. And playoff teams that won their final regular season game went on to win the Super Bowl 9% (8/90) of the time; playoff losers of final regular season games won the Super Bowl 7% (3/42) of the time. So there’s a difference, right? The playoff teams who win in Week 17 are a little more successful? Well, maybe.
A statistical test called the “Student’s t-test“[^3] is used to determine if the averages of two different samples are statistically different. Applied here, it answers the question:
What are the chances that playoff teams who won in Week 17 actually go on to win their first playoff game (or the Super Bowl) more than playoff teams who lost in Week 17, given that we’ve observed the above numbers?
p>I ran this test on both a team’s chances of winning its first playoff game, and ultimately winning the Super Bowl. The results? These numbers suggest there is a 63% chance that playoff winners in Week 17 actually have a better shot at winning their first playoff game than playoff losers in Week 17 do. For the Super Bowl, the odds that Week 17 winners have a better chance of winning it all are also 63%. That may sound like a lot, but it really, really isn’t. There’s a 37% chance these numbers don’t tell us anything about the true probabilities. In academic circles, the rule of thumb for statistically significant findings is when there’s a 5% chance or less that the results are due to randomness. We’re nowhere close.
Even if we were– let’s say we were 99% certain winners in Week 17 had a higher probability of playoff success– the substantive findings would still be very slight. Week 17 winners might be more successful, but probably to the tune of two or three percent, the difference of winning one or two more games more every several seasons. It may be something to hope for, but not something to pin your hopes on. In reality, Week 17 results tell us very, very little about what will happen in the playoffs. I’m just guessing, but I’d bet quite a lot that all the regular season results, and measures of play that go beyond wins and losses, such as DVOA or Pythagorean Winning Percentage, are far, far more telling than whether a playoff team won or lost its last regular season game.
So if your team is in the playoffs, great! If they won yesterday, that’s cool. And if they didn’t (actually, that’s just you, Kansas City fans), don’t fret. It’s how your team plays the next game that matters. And how they played yesterday has very little to do with that.
- As it turns out, my Packers’ fan friends are taking me to the freaking game (!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!) so I think everything worked out pretty well. Of course I also figure this means the 49ers are sure to lose. But then I also figure I can use this insider knowledge to my advantage and put some moolah on the Packers, and OH DID YOU KNOW THEY WERE GETTING THREE POINTS? It’s all falling into place. ↩