CITP Blog - Formerly Freedom to Tinker

In Partial Defense of the Seahawks' Play Calling

– by

Comments

The conventional wisdom about last night’s Super Bowl is that the Seahawks made a game-losing mistake by running a passing play from the Patriots’ one yard line in the closing seconds. Some are calling it the worst Super Bowl play call ever.

I disagree. I won’t claim it was the right call, but I do think it was reasonable. Let me explain why.

To analyze the decision we have to put ourselves in the shoes of the Seahawks’ coaches at the time. They did not know that an opposing defender would make a spectacular interception. They knew that was possible—and needed to take it into account—but a fair analysis of the decision can’t use the hindsight knowledge we have now.

With that established, let’s make a simple model of the Seahawks’ strategic choices. They needed a touchdown to win. It was second down, so they could run three plays. The clock was running down, so let’s assume that if they run two running plays, the clock will expire before they can get a third play off; but an incomplete pass on the first or second play will stop the clock and give them time to run a third play. There are three play sequences they can use: run-run, pass-run-run, run-pass-run. (Passing more than once is bad strategy.)

Suppose that a run play with Marshawn Lynch scores 85% of the time, and gets stuffed at the line 15% of the time. If you run twice, there is a 2.25% chance you’ll get stuffed twice, so you win the game with 97.75% probability.

Suppose that passing on second down has these results: score: 50%, incomplete: 49%, interception: 1%. So if you call the pass-run-run sequence, the game outcome probabilities are: score: 97.90%, stopped short: 1.10%, interception: 1%. The odds of winning are a tiny bit better than if you just ran twice.

It’s counterintuitive that passing might be the right choice even though a running play is more likely to score. The reason it comes out this way is that you’re not passing instead of running, you’re passing because passing gets you an extra play and you can still try to run twice, absent a spectacular interception play by the opponent.

Now you can quibble with these probability estimates; and you can argue that the Seahawks might have had time to do three run plays. Change these assumptions, and the strategic calculations are different. But the argument so far should establish that the Seahawks weren’t crazy to pass.

The real kicker comes, though, when we consider the remaining option of run-pass-run. If the outcomes of a pass are still 50/49/1 on third down, then run-pass-run is a clear winner. But maybe a pass comes as less of a surprise on third down, so the outcomes of a pass might be worse. Even so, run-pass-run turns out to be the best strategy. For example, if the outcomes of a third-down pass are score: 25%, incomplete: 73%, interception: 2%, the run-pass-run strategy still scores 98.06% of the time, which is better than either of the other options.

The conclusion that run-pass-run is the best sequence is fairly robust against changes in the probability assumptions. If it’s wrong, it’s probably because of the assumption that run-run-run isn’t an option.

The Seahawks’ decision to pass on second down wasn’t crazy, but a better choice would have been to pass on third down. Announcers who said “just run twice” were giving bad advice. The Seahawks didn’t make a terrible play call; they made a reasonable choice but were defeated by a great defensive play.

Comments

16 responses to “In Partial Defense of the Seahawks' Play Calling”

  1. Justin Zamora

    But the Seahawks had a timeout, so they could have run three plays regardless. Also, Seattle’s offensive coordinator says they called the pass in order to *use* more clock, apparently to prevent the Patriots from completing a miracle play after a Seattle TD.

    Reply
    1. Ed Felten

      Even with a timeout, I think it’s doubtful they could have gotten a third play off after two stuffed runs. The runs themselves take a few seconds, and after a stuffed run there is a big pile of players to untangle—and half of them are stalling. Pete Carroll’s comments after the game indicated that he thought of the second-down pass as an extra play.

      Reply
      1. D

        You can call timeout when there’s a scrum. As soon as the ref blows the whistle you can call timeout. Run-run-run is definitely an option and is the best option.

        Reply
      2. Anonymous

        But if you count the time from the end of the Lynch run to the 1, not the time of the pass, there was time for three runs. The Seahawks were very slow to the line. I think they were expecting the Patriots to call timeout.

        Reply
  2. Justin Zamora

    The Seattle OC explains the reasoning at http://m.espn.go.com/nfl/story?storyId=12266535

    Reply
  3. Blake Reid

    Ed, I don’t necessarily disagree with your analysis. But the real problem was the particular pass they called; a route across the middle is particularly susceptible to an interception if the defender can beat the pick. There were any number of other passes they could have called with a comparatively lower chance of interception, such as a corner or fade route. Throwing a pass on second down wasn’t necessarily a terrible idea, but I think that particular one was a bad call.

    Reply
  4. Joseph Bonneau

    You missed the most important aspect of game-theoretic analysis here, Ed. Football is about deception, you want the defense to not know if you are going to run or pass, and that requires sometimes calling a pass or a run even if the other play has a higher net yardage.

    This answers the question: why don’t teams simply pass on every down? Every professional team gains a greater average number of yards per play when they pass then run, so why run at all? The point of running (besides some short-yardage situations where it is lower variance) is that you need to run to keep the defense honest. If a team never runs, the defense can adjust and the passing success will go down.

    This applies to goal-line too, which means every team will have a number of pass plays on the goal line installed. Over the course of the season, it’s important to run some pass plays from the 1 yard line to ensure defenses stay honest and respect the chance that you might pass. So they have had this play in their goal-line package all year, I’ll be interested to see if they ran it before and how well it did.

    Of course, at this point the repeated-game aspects are starting to go out the window. Arguably by passing on second down (even if it fails) you keep the defense honest for third and fourth down. But the long-term repeated game is over at this point and it’s better to just run your highest-percentage plays which may be runs. Especially given that there was tight coverage on this receiver. While it was a spectacular interception, I think it was a major error by the QB not changing away from that play given that the defense was not lined up in a “stop the run at all costs” formation.

    Reply
  5. Chris S

    According to twittter ESPN Stats & Info (‏@ESPNStatsInfo), Marshawn Lynch had 5 previous runs from the 1 this season and had scored only one touchdown.

    If my key running back is 1 for 6 at that field position, that might influence my decision about what play to call next.

    Reply
  6. pete.d

    Interesting take.

    I’m curious about the 1% interception chance. That would mean that of 100 passes, only one is intercepted. That sounds much lower than my vague recollection of the actual odds.

    In fact, the Seahawks web site says that Wilson has a 1.5% interception rate in the regular season, but (more telling) a 7% rate in the post-season (i.e. playing against better teams, like, oh I don’t know, the AFC _champions_).

    So the odds of an interception in this case were a) a lot higher overall, and b) as you note, even higher because of the specific play.

    I think it was an honest mistake. I don’t think any less of the Seahawks coaching or players for the outcome. But it _definitely_ seems like a mistake to me, and they absolutely should view it that way. “Lesson learned”, I hope, rather than the all-too-common “wasn’t my fault!”

    Reply
    1. Ed Felten

      1% seems like a reasonable estimate to me. Over the 2014 regular season, Russell Wilson was intercepted on 1.5% of his pass attempts. Here, he knew an interception was disastrous so he should throw the ball away if in doubt, making the risk a bit lower than normal.

      Reply
      1. Dillon Reisman

        Not to quibble, but isn’t ‘getting sacked’ a fourth possible outcome when passing that hugely changes the benefits of going pass-run-run? Putting aside the fact that Lynch isn’t as great from 1-yard out as people seem to suggest [1], he definitely have even less of a chance at success if the first attempted pass came with a loss of yards.

        1. http://www.washingtonpost.com/blogs/early-lead/wp/2015/02/02/seahawks-marshawn-lynch-isnt-exactly-money-from-the-1-yard-line/

        Reply
      2. mind

        See also 538 for another analysis justifying Carroll’s call (and explaining why Belichick should have called a time out).

        http://fivethirtyeight.com/features/a-head-coach-botched-the-end-of-the-super-bowl-and-it-wasnt-pete-carroll/

        It was just a bad pass (too high) combined with a great defensive play by Butler (who anticipated the exact play from studying films).

        Reply
  7. Fumbles Happen

    The biggest missing piece from the analysis is that running isn’t risk-free either. In fact, the chances of fumbling on a run from the one are about the same as the chances of an interception, based on Bill Barnwell’s analysis of NFL data at grantland.com (Not so surprising a result: when the runner is struggling for that extra couple inches, and defenders are trying to strip him, balls can come out fairly often).

    Note that the 49ers played it super-safe earlier this season and even avoided the chance of fumbling the hand-off by running the QB. He fumbled just before the goal line.

    You should re-run the numbers including a 1% chance of fumbling on any run.

    Reply
    1. Peter Wayner

      Just the week before the SuperBowl, my son was just 20 seconds from kicking a field goal that would win the game and end it, all at the same time. So he ran the ball and the running back fumbled. Game over. (BTW, this was in Madden Ultimate.) It happens in the running game too.

      I tend to think of any football game that comes down to the last possession as a tie.

      Reply
  8. NathanT

    Darn, was in middle of a nice post when my browser crashed. Oh well. I will shorten my post.

    Normally I don’t follow nor comment on sports articles. However, to see a blog post from a man whose opinion always find interesting to read and often value; I decided I would read.

    I wanted to comment on that even before I got to the end I was thinking run-pass-run to be the best option (at least between running and passing plays). I used to watch college football as a child (with my dad) and the family team was winning a lot; though loses as often as wins these days. I have come to reason why they used to win so much; they would pass when it was not the expected play to make; and they would make use of the fourth downs (especially when not expected).

    Given all the assumptions you made, I too would have suggested a pass. But, I think I may have suggested the “dive-pass-run” or perhaps the “run-pass-dive” option.

    What is a dive? It is sort of a running play, but there is no running. I understand their are some rules about who can handle the ball (and forgive me if pros have different rules than colleges); but the idea behind the dive play is simply you hand the ball to the runner who instead of running just takes a flying leap head first (usually over the bodies of the front line) into the end zone. With the front lineman basically blocking the opposing team (keeping them closer to the ground). I seen it work many times in the scenario you said occurred. I don’t know if I would have called a dive first, or last; but I certainly would have put the passing in there. I have found more often than not, passing is the way to win the game of football.

    What is sad, the current coach for the family team (not that I cheer for that team anymore for other reasons) is that he would likely just go for a field goal on a third down after attempting a run — given the same scenario in this article.

    Reply
  9. Raphael

    See also:

    “Why the Pass at the End of Super Bowl XLIX was the Right Call” by Joel C. Adams
    http://cacm.acm.org/blogs/blog-cacm/182941

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

About

The CITP Blog is hosted by Princeton University’s Center for Information Technology Policy, a research center that studies digital technologies in public life. Here you’ll find comment and analysis from the digital frontier, written by the Center’s faculty, students, and friends.

Categories