Though you may not know it, when you watch the Super Bowl on Sunday–especially if you actually watch the game and not just the commercials– you’ll likely be engaging in a lot of what psychological scientists call counterfactual thinking: thoughts about “what might have been,” “at least,” and “if only.”

Research on counterfactual thought explains all kinds of surprising reactions to sporting events, and a couple of ways in which your judgment about the big game may not be as clear as you think.

1. It actually feels worse to lose by a little than a lot. We’re most likely to think about “what might have been” when it’s easy to imagine something turning out differently. It’s much easier to imagine how a game that ended with only a one point difference might have had a different winner than to imagine how the losing team could have made up a 20 point difference. Because counterfactual thoughts are so closely tied to emotions like regret, having easier-to-imagine counterfactuals makes us likely to feel worse. For instance, research in the Journal of Personality and Social Psychology found that students who got an 89%, just missing out on an A-, felt less satisfied with their grades than students who got an 87%. In general, we’ll actually wind up feeling worse when victory falls just out of reach than when it’s a hopeless blowout.

2. … but it feels better to be down at halftime. The exception to this “doing better but feeling worse” rule is when we’re in the middle of an event. After an event is over, thinking about what might have been is mostly important to our emotions. During the event, however, these thoughts can help us understand our performance and serve as a motivator, like how runners might focus on the fact that they only have one more mile to go to encourage themselves to pick up the pace. In one study, researchers had students read the play-by-play of a basketball game. As expected, those reading about a missed shot in the final minutes of a close game felt worse when it was their team that lost rather than won the game. However, when they were told that the story described the last minutes of the first half of the game, the fans of the team that was down by one actually felt better than the team that was ahead by one, because the former team had reason to feel hopeful and the latter had reasons to feel worried.

3. You only think you knew how that play would go. Fans engaging in “Monday morning quarterbacking” are usually certain that they knew all along how things would turn out. It turns out, though, that we’re likely far too confident in our post-game estimates of what we actually knew at the time. Researchers have termed this exaggeration of what we actually knew at the time “hindsight bias.” In the initial demonstration of this effect, participants read one of two versions of a story that described the events leading up to a crucial battle between the British and their adversaries, the Gurkas. One version of the story noted that the British had won the battle, and the other was identical except that it omitted the outcome. The first group of participants was asked to imagine that they didn’t know how the battle turned out and to do their best to ignore this information and estimate how likely they would have thought the British were to win if they weren’t aware of the outcome (that is, to imagine that they were in the shoes of the second, “unknown outcome” group). Compared to people who really didn’t know who had won the battle, people who were trying to ignore their knowledge of the real victor still said that it was much more likely the British would win. Once we actually know how something has turned out, it’s easy for us to start to focus on the information that we knew that was consistent with this information (the quarterback is known for his passing game), and ignore the other information that would have suggested another outcome (during the playoffs, he’d bruised his shoulder and was throwing inconsistently), making what actually happened seem much more certain than it really felt at the time.

4. Counterfactuals explain what we think was “the play of the game.” As noted, counterfactuals are more likely when it’s easy to imagine an alternative. This means that events that are unusual tend to make us think counterfactually: it’s easier to imagine how we might have avoided food poisoning if we ate a new dish at an unfamiliar restaurant than if we went to the same neighborhood place and got the usual. Counterfactual thinking explains why we focus on surprising outcomes, like a team intercepting a field goal and returning it 100 yards for a touchdown, when talking about the “deciding play.” Likewise, because it’s easier to mentally rewrite just the last few minutes of play by focusing on events late in the game than to imagine how the whole game would have unfolded differently if events in the first quarter had gone differently, we tend to focus on these late events. Of course, every touchdown has the same effect on the score, so no one play is objectively any more decisive than another–but because of the ways counterfactual thought works it certainly doesn’t feel this way.

As you watch the big game this weekend, you’ll probably catch yourself and your friends doing at least one of these things. (Pro tip: you may not want to actually point out every time this happens, if your friends and family are anything like mine.) As you enjoy the football, commercials, and/or the half-time show, take a minute to appreciate the cognitive quirks of counterfactual thinking that shape how we feel and respond to events.