Popular Harvard Sports Analytic Collective Study of NFL Playoff Possibilities Misses the Odds

Near the beginning of preseason, a Harvard Sports Analysis Collective (HSAC) study projected each NFL team’s percentage chances of making it into the 2015 NFL playoffs.

The study reached numbers that appear to carry the credibility of tested data and analysis. Because of this, along with the school name and the study’s use of assessments from Pro Football Reference and statistical behemoth FiveThirtyEight, it garnered a lot of attention.

Unfortunately, many of its numbers are heavily flawed. (I’ve compared them in here with better playoff chances in part I-covering teams 1-10; II-covering teams 11-20; and III-covering teams 21-32, and will look at both during the course of the season and run a comparison at season end. Anything can happen, but barring a statistical aberration, the Harvard Collective’s study numbers will fare worse.)

The HSAC study made several compounding assumptions. And not only did this lead to some results that may not represent the best assessment of that team’s actual playoff chances, it also led to some statistically questionable, and even unsupportable ones.

For instance, it pegged the Seahawks at a ridiculous 99% to make the playoffs. (The Seahawks were originally 95% to make the playoffs – still too high. But apparently to normalize outcomes so an average of six teams from each conference would make it into the postseason each year, their number was adjusted upward.) There’s far too much variability, uncertainty, as well as general parity in the NFL for any team to have a 99 out of 100 chance to make the playoffs, before the season even begins.

The HSAC study also pegged the Titans at 2% to make the playoffs, and originally the Ravens at 9%. Both of these are also unrealistic given basic NFL variance in the case of the Titans; and in the case of the Ravens, also given the fact they have made the playoffs 6 out of the last 7 seasons, and have more playoff wins than any team in the entire NFL since 2008; the year quarterback Joe Flacco entered the league and John Harbaugh became their head coach.

And it pegged the Raiders at a ridiculous, almost ludicrously low .003 (.3%) – that’s 3 out of 1000 times – chance of making the playoffs.

Along with a few other probabilities that push the boundaries of statistical reasonableness, and several others that probably don’t represent particularly great assessments, the study also pegged the Miami Dolphins as having the highest chances of making the playoffs out of the entire AFC.

That’s not a totally wacky pick. Miami was one of my dark horse teams to take the next step this year; as it was for several other people. But it still seems a little odd that since this study has come out, Miami is now often thrown into the mix of AFC, and even possible Super Bowl contenders.

There’s a good chance this is simply a coincidence. After all, Miami as a dark horse team (among several) was not a novel idea. They have some good players, a potentially excellent quarterback, and showed occassional signs the last two years of being able to play at a near elite team level. (Albeit several teams have. For instance, watch out for the Chiefs this year as much as if not more than the Dolphins. Another AFC dark horse that may surprise, if that defense really pulls together and QB Hoyer throws as accurately as he did the first half of last season and not the second half, is Houston. The Bills are also at least on par with Miami, and probably more likely to make the playoffs.)

But it could also be that a reasonably well publicized Harvard study floating around out there, that pegged Miami as the top team in the AFC, also didn’t hurt – no matter how goofy some of its numbers upon closer analysis.

And some of its numbers, as suggested above, are goofy. For instance, pegging the Seahawks at 99% to make the playoffs defies football reality, and at least relative NFL parity and uncertainty.

One of the only ways to really show this point is for the Seahawks to miss the postseason. (Though it wouldn’t technically prove that the “99%” probability was wrong, since, though a long shot, such an outcome could still just be a “1 in 100” fluke, it would certaintly help suggest it.)

But the problem is the Seahawks are likely to make the postseason.They’re just not “99 out of 100 times” likely to make the playoffs. And no team in modern NFL history has been. Ever.
_______

Essentially, the HSAC study used a multiple step interpretive statistics process to come up with a methodology that appears sound, but isn’t.

The study used Pro Football Reference’s approximate value statistics for players, then assessed team strength by relying on them for “core players.” But the valuations are still subjective. And more importantly, football is a team game, not a core player game.

The model results were also “tested” by running last year’s data, and comparing it to last year’s end of season FiveThirtyEight ELO ratings. But reasonable correlation with these ratings doesn’t imply the probabilities are robust; just that they may be more accurate than merely throwing darts at a random board of probability numbers.

The ELO assessments also reflect a limiting system of assumptions as well – one that tries to arrive at the “better team” in terms of overall performance, including in large part how much a team wins games by, etc. But this also doesn’t mean correlation with the highest chances of making the playoffs – just again, something superior to throwing darts at a board.

First off, some teams know how to protect leads; others how to do so and pull out close games when behind; still others manage to stay tight and lose but can win by blowouts, etc. (And even if to some extent these things can factor in to win totals, it gets heavily skewed by score differentials, what team was coming off of what games, and most importantly what actually happened in each game.)  And it doesn’t take into account the odds that particular team faces – the makeup of their division, what other divisions they have to play, etc.

So rather than test the model compared to last season’s rankings, as noted above we’ll compare its probabilities to how the season actually works out for the 32 team’s ranked, as well as how it does in comparison to the non statistical generic evaluation of each team’s playoff possibilities assesed here. My prediction is that the Harvard study, although it got a lot more publicity, is going to show worse results than the assessments made here in parts I II, and III.

In addition to the fact that grading core players rather than the full team is incomplete, and that player grades, even for all players, is still not necessarily equal to a team grade, part of the study’s flaws is that grading players relative to each other in terms of win probability is also very difficult. If one player is a 10 and another is an 8 (just for a scale of comparison), what does that mean?

Is the difference between 10 and 8 that big of a gap that surrounding “non core” players, coaching ability – beyond its small reflection in that team’s player ranking to begin with – overall team chemistry, cohesion, or heart, don’t matter as much?

Of course if we can assess the general quality of multiple key positions, statistically at least we can at least get a feel for the team. (And in many of the skill positions particularly, the study’s overall ranking, even if unintentionally, will be affected by the overall quality of a team, with receivers with great quarterbacks and solid offensive lines and great offensive coordinators getting higher ratings, for instance, than if they had played their last several years on a different team, etc.)

But that’s all the study really does. Which around the middle of the pack is enough to put forth numbers that aren’t consistently outlandish, but not at the high, and in particular low, ends.

Think what you will of the Rams, for instance, but assessing them as having only a 1 in 10 chance of making the playoffs, before the season even starts, and with an upgrade at quarterback; another year for their many young players; an improving team; a good head coach; and when 12 of 32 teams reach the playoffs, is just not realistic.

For this same reason, almost all of the low end, and particularly the very low end numbers, are not just too low, but become increasingly preposterous, no matter how bad seeming the teams. Even Tennessee, and even Oakland.

And, frankly, who knows. either could be a decent team this year. (with Oakland probably having a slightly better chance, although they’re in a tougher division and face a tougher outside the division schedule, which will hurt them in the getting to the playoffs sweepstakes.).

Also notice Oakland’s pattern last year after beating the Chiefs to bolt their record up to a whopping 1-10 in week 12. They took it light – obviously – and got pounded 52-0 by the Rams, then pulled it together and back at home surprised again, legitimately beating the still tough 49ers – and doing so as large underdogs – 24-13, before then, same pattern, getting pounded yet again, and this time by the Chiefs in a rematch in Kansas City, 31-13. Then guess what. Same pattern still: They won again, and again against a good team. That is, by late last season the Bills were a very good football team, and probably taking the Raiders lightly, and on a cross country trip fell to those same Raiders 26-24. And yet after pulling out that win, Oakland continued its pattern as well, getting pounded by Denver in a season ending game, 47-14.

Again, we’ll examine the outcomes at the end of the season, but it will be very surprising if the Harvard numbers don’t fare much worse overall than the numbers given here. In the meantime, again, two sets of playoff odds for all teams in the NFL, one by the Harvard Sports Collective study, and one by this site along with some of the key reasons for the numbers given, are set forth in parts I, II, and III.

Advertisements

Week 14 NFL Picks, Against the Spread

History to date: A bit long, but a nice recap of all the weeks records prior to last week (as well as last week’s abominable picks) and some keys on assessing football games, is reviewed here.

There was no pick for the Thursday night game this week. In last weeks picks, recap, and assessment of Thanksgiving Thursday’s plate of three excellent games just linked to, this blog said the following about this past Thursday night Cowboys at Bears match-up:

Now the Bears face Dallas (next week), so they’re perhaps finally ready to let it all hang out, or their claims they “haven’t given up” are probably tepid. I have no pick on that game. I’m always wrong on the Bears. But hey, this is a blog, and if forced, until they do show that passion that should have come last week, here’s the pick for next week: Dallas.  (Though do recall that Dallas typically falls apart in late November and December, so this ought to be a good one.)

Two problems existed. One is too much faith in even repeatedly passionless teams to pull it together, get jazzed and play hard when it would seem teams would.  With the Bears this is a bad mistake to make, since one would have made that mistake about 5 or 6 big games in a row with them now, in games against good teams. And I think I’ve made it the majority of those times.

The second was a little too little faith in the Cowboys’ December abilities.

But in hindsight, the game materializes nicely, once the “is this the game the Bears finally rear up and play like a team and not a bunch of energy less piddlers” question was (once again), emphatically answered:

Dallas has been good this season.  They got embarrassed at home by the Eagles Thanksgiving day. With that loss their backs are against the wall. Past season tendencies are not a full indication of future performance. And they’ve played well on the road this season.

Meanwhile, putting idealistic guesswork on the Bear’s motivation aside, a callous examination of the Bears’ games indicates a decent team that has listlessly spiraled into a fairly bad one, while others are playing strong.

Hence, a team that is good, good on the road, with a very good quarterback – who unlike the week before had his normal full week to rest his ailing back – in a very important game for them, in front of a national TV audience, coming off of an embarrassing home loss in front of a national (thanksgiving afternoon no less) audience, against a fairly bad, and worsening, team, is likely going to win, and win strong.

Nice analysis, huh.  Be a lot nicer if saw it and provided it before the game.

As always, the following picks are only for the purposes of raising massive funds in Las Vegas in order to buy copious amounts of food for the homeless in Fort Lauderdale and numerous other cities, and then be able to afford a bevy of lawyers for a defense when the police – charged with enforcing an inane law against feeding homeless people in public in order to – get this – reduce homelessness (“Charlie, no one’s feeding me, I think I’ll buy my house back now!”) – arrest you for helping the neediest of the needy.

Ravens (+3) at Dolphins

It’s going to be a little warm in Miami for this game. That is what it does in Miami: Stays warm. And sometimes during the colder months visiting teams have a harder time with this, as during colder months the blood thickens and the body accommodates to colder temperatures.

Teams have gotten pretty good at adjusting for this though, with wet air blowing sideline fans, etc. But the Ravens will still likely be the team in dark, sunlight absorbing jerseys, and the forecast is for mostly sunny.

And while in the world of climate change “debate” an increase in absorbed or even captured and re radiated energy can be dismissed via belief under the grand complexity of long term global impact, in the more simplistic world of incoming solar radiation upon a colored surface, or, shirt, we don’t debate it any more.

Going back over several years it would seem Baltimore is the better team. But this is now. And this is a heck of a football game, in terms of the AFC, and by this point in the season it will say a lot about who has a chance. While Baltimore still has a shot for their division (provided they win this game), the loser of this game will probably be out unless they run the table. And unless the game is closely played, the loser probably isn’t strong playoff caliber anyway.

The Dolphins got slightly outplayed in a 3 point win at the Jets last week that was misleading – as once in a while the Jets play like a strong playoff team. The rest of the time they play like a team with almost no wins. Which is pretty much what they are.

The Ravens were at home last week. And as six point favorites (in a game that this blog, despite a bunch of bad picks against the spread last week, specifically called the upset on), they lost late by a point to the Chargers. But they probably slightly outplayed San Diego in the game.

Then again, Baltimore also tends to be a fairly strong home team, and not a very strong road team. And whether they “slightly outplayed” the Chargers or not: it wasn’t by much, and they didn’t win, at home.

Going with the Raven’s history, and perhaps not giving enough credit to the Dolphins’ recent overall performances (it’s hard to see Baltimore going into Denver in week 12 and only losing by 3 in a game they led a good portion of the way, as the Dolphins for instance did in week 12), the call this week, perhaps questionably, is:

Pick: Ravens, but it’s close. If there’s any pick here you just want to skip, or ignore my take completely on it (apart from maybe all of them), this might be the one.

Steelers (+3) at Bengals

It took a while before the Bengals could finally beat the Steelers in a game that mattered. And if they don’t win this, it’s right back to the old ways.

But the Bengals are the better team. And either this is their moment to peak, or they might as well pack it in once again as a pretty good, but just can’t quite get it done, team. (Though if they do win this game they still have the playoffs to not get it done, as they now have five out of five times, going 0-5 so far in the playoffs under head coach Marvin Lewis: the second longest tenured head coach in the NFL, after some random guy no one’s ever heard of – and who certainly has never won anything – who coaches some team out of Boston.)

The Bengals are simply a better football team. The question to be answered here is whether they have as much heart as Pittsburgh traditionally shows. Because Pittsburgh, more likely than not, will show it here.

Pick: Bengals 

Panthers (+10) at Saints

A few weeks ago, the Panthers hosted the Saints, and despite playing well in the first half, fell solidly behind by halftime. Then when they needed to atone in the second half, they did so by playing worse.

Carolina has shown itself to not necessarily be the type of team that will rise up with passion in key games where it is really needed. (Or perhaps they play too tight, and too worried about losing. Or both. See links below.) And on some level they may have half packed it in for this season already.

Whereas New Orleans is playing pretty well again, and is hard to beat at home. But Carolina still has a game left against the Falcons, who “lead” the division along with the Saints at 5-7, one and a half games ahead of the Panthers.

The Panthers have also not beaten, nor in most cases even come very close to beating, a good team this season, with the exception of a 24-7 win at home against Detroit in week 2. (At a time when they sill had linebacker Greg Hardy, a key cog in that defense.) And a tie, in a wacky week 6 game against the Bengals, at a time when the Bengals were not playing all that great. (And which took a Bengals miss of a short 36 yard field goal at the end of overtime for the Panthers to pull out the tie.)

They’re just not very good.

And after turtling up just a little bit at the end and losing in a battle for the division lead to Atlanta, the Panthers had a bye to get all nice and rested, and then came out and lost 31-13 to Minnesota. Minnesota.

Not that Minnesota’s not good, but other than a few fluke seasons here and there, the last time the Vikings were otherwise a strong team, they were purple people eaters. (And even if the Vikings two blocked punts for scores in that game are removed – though failing to block well enough and giving up blocked punts are part of football – they still lose 17-13. Plus they would have gotten the ball two extra times, because after each blocked punt and score by the Vikings, the Panthers get the ball back.)

Sometimes however, even if not as much this season, the Panthers play tough in hard games.

They just don’t seem to be showing much capacity to do so this year. And aside from the still underrated loss of Hardy (on the NFL’s exempt list, practicing some rap while hoping badly to beat the rap against him on appeal from an initial trial with no jury, when the now delayed case finally goes to a jury trial after the NFL season), and the seemingly overly tense and mechanical play of quarterback Cam Newton, this team may have made some poor offseason moves, and they’ve lost a few key players to injury.

Nevertheless, this is a division game. It’s not a near foregone conclusion that the Saints win. They’ve battled the Saints tough before when they’ve been a worse team than the Saints; and the Saints, playing well right now, have been up and down this season.

Though they don’t show anywhere near championship caliber energy – one of the many things that distinguishes the Panthers from the Seahawks, who they keep losing late leads and close games at home to – this is a chance at some redemption against these Saints for the big cats from Charlotte; and a shot, long shot though it may be with a team that is not very good, at pulling something out this season.

This game could be all over the place, and it’s not a great call given the Panthers’ lackluster play and the Saints home record and general focus. But this is the Panthers’ game, or it’s their season. They do have the capacity to play well, and despite losses, their defense still has the capacity to hold the Saints in check.

Keep in mind that if the Panthers do somehow win, it would be the fourth consecutive home loss for the Saints. Who, until this season (two weeks ago, specifically) had never lost three home games in a row under head coach Sean Payton, now in his 9th NFL season. (Though one of those seasons Payton was required to sit out for some Bountygate hanky panky, for which NFL Roger Commissioner was later scolded, and Payton somewhat exonerated).

Still, here’s to the big cat underdogs to show some divisional spark:

Pick: Panthers

Buccaneers (+10) at Lions

The Bucs are at the bottom of the NFC South, where the two division leaders are at 5-7, and the third place team, the aforesaid Panthers, have 3 wins and 1 tie.

Still, the Bucs, even under former head coach Greg Schiano, were a team that on several occasions went into the stadiums of much better teams, and battled close or won outright. (In fact, late in quarterback Russell Wilson’s second season last year, the otherwise lagging Bucs were almost the first team to beat him in Seattle since he entered the league, losing in overtime.)

And while former Bears head coach Lovie Smith might have been a little overrated, and proclamations of how good the Bucs were going to be this season (seemingly long forgotten, like sand beach castles down by the edge of an incoming tide, washed over and gone from memory) greatly inaccurate, they have shown a few signs of playing some good teams tough, and might finally be improving.

No upset picks this week. But this would be the type of game – though in a tough battle for the division or a wild card, Detroit can’t really afford it – where an unexpected upset could happen. A 10 point (or more) loss looks very plausible. But so does a close game.

Pick: Buccaneers

Seahawks (+1.5) at Eagles

Two birds going at it. This is the game of games. And Philadelphia, who has played well enough to be the standard 3 point home favorite here, doesn’t get much respect.

Still, the Seahawks, after what they did at the end of 2012 (blowing nearly everybody out of the water and then barely missing out on advancing to the AFC championship game), last year (a near dominant Super Bowl run and win) and the fact they are playing well of late, are a tough team to pick against right now.

If they are still for real, they have to win this game. While the Eagles, conceivably, do not.

No pick, it’s just a great game, and one that will tell a lot. And one in which the suddenly vulnerable Arizona Cardinals, who are just a game up on Seattle at this point but have already lost to them once, and who are playing the Chiefs at home and aren’t even 3 point favorites themselves, no doubt have a rooting interest in as well.

And which leads us into a game which, back to common practice, will yield a pick.

Chiefs (-1) at Cardinals

Perhaps the curtain is being pulled back on the Cardinals, who didn’t build as much as other teams in the offseason, and have suffered some key injuries. And who now have Drew Stanton at quarterback –  who throughout his career as a backup, and now in place of, is so far no Carson Palmer, by a long shot.

And the Chiefs are in a tough division, and a good team.

But don’t count these birds out yet.  They’ll give it all they’ve got, and should be a favorite to win the game.

Pick: Cardinals 

Patriots (-4) at Chargers, Sunday Night Football

Are the otherwise red hot Patriots, who once again seem to be one of the two or three best teams in the league right now, really going to lose twice in a row?

Maybe.

Straight up, this might be a Patriots pick. But the Chargers have the home field advantage. And while it doesn’t mean too much for current performance, compared to their other games, the Chargers simply have a near phenomenal record under quarterback Philip Rivers in December. (Albeit not as good as the Patriots record in December under Tom Brady.)

The Chargers are also a little handicapped though. They were down to their 4th string center, having lost a remarkable three centers to season ending injuries this year, and then their 4th center, Chris Watt, was injured in the close win at Baltimore in week 13.

This is not good. Also not good: Their starting right tackle, DJ Fluker, was drafted 11 overall last year. And he may be better than the 1st and 2nd overall picks in that draft, both also offensive tackles. Yet Fluker suffered a concussion this week. In practice.

If it was really a concussion, it is hard to see how he could sensibly play. But he was back at practice on Friday, as was Watt, who may play. (Although center number 5, Trevor Robinson, signed off the Bengals practice squad in week 7, seemed to do all-right against the Ravens last week.) Defensive tackle Corey Liuget, who leads the Chargers in tackles for a loss (14), may also not play.

But 4 points is still relevant in a what could easily a close game, or even a Chargers win.

Pick: Chargers

Falcons (+13) at Packers, Monday Night Football

Sure, the Packers have been blowing nearly everyone out of the water at Green Bay, including some teams better than the Falcons.

But give Atlanta some credit here.  They may not be any good any more, but they are a tough football team under head coach Mike Smith, and less apt to completely fall apart in games.

True, they could not fall apart and still lose by 27 to the Packers up in Green Bay. But it’s time for these ridiculously lopsided games up there in Wisconsin to stop. At least that’s what the Falcons are thinking, right?

Or maybe the team from George is instead thinking. “It’s cold up here, when do we get to go home.”

Pick: Falcons

Miami Dolphins Make a Typical, but Truly Awful Strategic Decision in last Minute of the Half Against the Buffalo Bills

This blog is about to share something with you, that, if you happen to come across this post in your online football travels, from a football strategy perspective will absolutely blow you away. Continue reading