It happens to us all: you spend all day avoiding the cookie jar at work, but when you pass by it in the late afternoon, your hand reaches in and grabs a cookie. Before you know it, you’re wiping crumbs from your mouth. Or, on the way to the new job, you get distracted by a news story on the radio. When the story ends, you look up and you’re at your old job, wondering how you got there.
What’s going on? How can we perform complicated behaviors without even realizing what we’re doing? Neuroscience and psychology help answer this question. The first step is to think of the brain not as a single entity, but rather as several learning and control processes that can each control behavior. In other words, we are composed of several different “selves” (“I can’t believe I let myself eat that cookie!”) and those selves sometimes come into conflict.
We can better understand two of these processes, goal-directed (or “conscious”) and habitual (or “automatic”), by understanding how neuroscientists think of them. A series of elegant experiments conducted by Anthony Dickinson and colleagues in the early 1980s at the University of Cambridge in England clearly exposes the behavioral differences between goal-directed and habitual processes. Basically, in the training phase, a rat was trained to press a lever in order to receive some food. Then, in a second phase, the rat was placed in a different cage without a lever and was given the food, but it was made ill whenever it ate the food. This caused the rat to “devalue” the food, because it associated the food with being ill, without directly associating the action of pressing the lever with being ill. Finally, in the test phase, the rat was placed in the original cage with the lever. (To prevent additional learning, no food was delivered in the test phase.) Rats that had undergone an extensive training phase continued to press the lever in the test phase even though the food was devalued; their behavior was called habitual. Rats that had undergone a moderate training phase did not, and their behavior was called goal-directed.
Referring to Figure 1, goal-directed behavior is explained by the rat using an explicit prediction of the consequence, or outcome, of an action to select that action. If the rat wants the food, it presses the lever, because it predicts that pressing the lever will deliver the food. If the food has been devalued, the rat will not press the lever. Habitual behavior is explained by a strong association between an action and the situation from which the action was executed. The rat presses the lever when it sees the lever, not because of the predicted outcome.
More recent work by David Neal (while at the University of Southern California) and colleagues illustrate this same type of behavior in humans in a more palatable scenario: eating popcorn at the movies. Some participants were given freshly popped popcorn, while others were given stale popcorn. Most participants that were given fresh popcorn ate the fresh popcorn. Out of those that were given stale popcorn, those that usually ate popcorn at the movies ate the stale popcorn—they acted habitually. Those that usually did not eat popcorn at the movies ate less of the stale popcorn—they acted under goal-directed control.
The two separate processes are mediated by mostly-separate networks in the brain. Damage to the network involving goal-directed processes, which includes the prefrontal cortex and the dorsomedial striatum of the basal ganglia, results in habitual processes dominating behavior. This makes sense because the prefrontal cortex, which is located behind your forehead, has working memory capabilities that allow it to temporarily represent the desired outcome, such as the food reward or popcorn, which elicits the action. On the other hand, damage to the network involving habitual processes, which includes sensorimotor cortices and the dorsolateral striatum, results in goal-directed processes dominating behavior.
Goal-directed and habitual processes can each generate behavior. So, why do we have both of them? From a functional point of view, they have different advantages and disadvantages, and thus are useful for controlling behavior under different circumstances. Goal-directed processes use a model of the environment to predict the possible outcomes of each action. They are deliberative in that you use the model to mentally go through different actions and even sequences of actions, and then execute the one that you predict will deliver the desired outcome. This is like using a map of a city—where the map is the model of the environment—to navigate from point A to point B (see Figure 2). The advantages of goal-directed control are that it’s very flexible and it doesn’t require much experience: as long as your map is accurate, you can navigate from any point to any other, even in a city you’ve never visited. (This scheme is similar to the use of cognitive maps to control behavior as described by the American psychologist Edward Tolman in the late 1940s.)
Of course, the versatility afforded by goal-directed control comes with a price: it require cognitive effort—memory and computations—in order to represent the model of the environment and to search through the model to choose the best actions. Habitual control, on the other hand, is much simpler because it relies on just an association between the “state,” or environmental situation, you’re in and the action. The stronger that association is, the more likely it is that you’ll select that action from that state.
Habitual processes are trained by experience: if you frequently execute an action from the same state (like taking a left at the intersection of Pleasant Street and Main Street), and the outcome of that action is good (you reach your destination), the strength of the association between the state and action will increase (see Figure 2). (This training scheme is similar to the Law of Effect described by the American psychologist Edward Thorndike in the early 1900s.) Goal-directed processes can generate this experience until habitual processes are trained enough, but experience can also be generated by randomly trying out different actions.
If, after you execute the action, you wind up in another state in which there is a strong association with another action, and from there you wind up in yet another state in which there is a strong association with an action, habitual processes allow you to execute a sequence of actions without much cognitive effort. Thus, even complicated behavior, like taking a series of turns while driving to work, can continue under habitual control as long as things proceed as expected. However, if you wind up in an unexpected state, as may be the case if part of your usual route was closed due to construction, you may revert to goal-directed control.
We’re all familiar with acting with little conscious effort (when was the last time you thought about typing each character of your password?) and early psychologists noted the advantages of such control. In his seminal 1890 book The Principles of Psychology, the American psychologist and philosopher William James noted that, without habitual processes, “we could not accomplish everyday tasks as each little act, like tying our shoes or dressing ourselves, would require so much conscious effort that we will be exhausted. … The more of the details of our daily life we can hand over to the effortless custody of automatism, the more our higher powers of mind will be set free for their own proper work.” By using habitual processes to control routine behavior, we can use our goal-directed processes for other purposes, like reasoning about longer-term plans or executing other behaviors in parallel.
So, goal-directed processes are deliberative, flexible, and don’t require much training, but they require much cognitive effort. Habitual processes require little cognitive effort, but they are not as flexible and they require some training. The former is better when flexibility is needed, and when the unexpected happens, but the latter is better when behavior is routine and things proceed in a consistent manner. Assuming both processes are available, how does the brain arbitrate between them? This is an important question that remains largely unanswered, but theoretical studies offer possible explanations. Nathaniel Daw at New York University and colleagues suggest that a measure of certainty or confidence is associated with each process, and a separate arbitration process chooses the more certain of the two in any given situation. A study of mine suggests that the simplicity of the habitual processes causes them to elicit actions, if they are trained enough, faster than goal-directed processes. Dickinson and colleagues suggest that habitual processes dominate when the rate of outcome delivery (such as food) no longer increases as rate of behavior (such as lever pressing) increases. In all three cases, control is transferred from goal-directed processes to habitual processes as experience is accrued. Also, goal-directed processes can actively suppress habitual processes, though suppressing the habitual process as well as determining the appropriate action likely requires more cognitive effort than just determining the appropriate action.
Finally, some disorders in behavior can be explained by an imbalance between goal-directed and habitual processes. Some symptoms of obsessive compulsive disorder or drug addiction may be due to overly-strong habitual processes, and even overly-strong habits of thought: you “know” that you don’t have to flip the light switch a third time or that you shouldn’t reach for the cigarette, but you do so anyway. In the other direction, a recent theory of Parkinson’s disease noted that damage to dopamine projections to the dorsolateral striatum, which mediates habitual processes, occurs earlier than projections to dorsomedial striatum, which mediates goal-directed processes. As the disease progresses, fewer behaviors would be control by habitual processes, and even routine behaviors would have to be executed with relatively slow, deliberative goal-directed processes.
Having a better understanding of the differences between goal-directed and habitual processes gives us a better understanding of our own behavior. You might think about changing your behavior in terms of goal-directed processes and leaving it at that: “I shouldn’t eat cookies, so I won’t put my hand in the cookie jar.” However, you might have a previously established habit of executing the action of reaching into the cookie jar whenever you’re in the state of seeing the cookie jar in the break room at work. So, knowing that it takes extra cognitive effort to suppress the habit, it makes sense to get enough rest so as to ensure that you can exert the required effort to suppress the habit. Or, even better, you should try to avoid the state of being in the break room altogether so that the habit won’t be triggered. Even a small change in state might suffice: David Neal and colleagues showed that when their participants were given the stale popcorn in a room other than the movie theater, even the habitual popcorn eaters didn’t eat as much. As another example, you may have recently changed jobs and now have to take a slightly different route to work. Your previously established habits of taking certain turns at particular intersections may be triggered if you’re distracted. It may make sense to allow goal-directed processes to dominate control by not listening to the radio during the first few days at the new job.
So, if you want to change your behavior, perhaps the question shouldn’t always be, “how do I convince myself” to act a certain way. Instead, a better question might be “which process do I want to control my behavior?”
Here are some nice review papers that focus on the neuroscience of goal-directed and habitual control:
Ray J. Dolan and Peter Dayan (2013). Goals and Habits in the Brain. Neuron, volume 80, pages 312—325.
Ann M. Graybiel (2008). Habits, Rituals, and the Evaluative Brain. Annual Reviews Neuroscience, volume 31, pages 359—387.
Peter Redgrave, Manuel Rodriguez, Yoland Smith, Maria C. Rodriguez-Oroz, Stephane Lehericy, Hahai Bergman, Yves Agid, Mahlon R. DeLong, and Jose A. Obeso (2010). Goal-directed and Habitual Control in the Basal Ganglia: Implications for Parkinson’s Disease. Nature Reviews Neuroscience, volume 11, pages 760—772.
Henry H. Yin and Barbara J. Knowlton (2006). The Role of the Basal Ganglia in Habit Formation. Nature Reviews Neuroscience, volume 7, pages 464—476.
Here are citations for the specific studies I described:
Nathaniel D. Daw, Yael Niv, and Peter Dayan (2005). Uncertainty-based Competition Between Prefrontal and Dorsolateral Striatal Systems for Behavioral Control. Nature Neuroscience, volume 8, pages 1704—1711.
Anthony Dickinson (1985). Actions and Habits: The Development of Behavioural Autonomy. Philosophical Transactions of the Royal Society B: Biological Sciences, volume 308, pages 67—78.
David T. Neal, Wendy Wood, Mengju Wu, and David Kurlander (2011). The Pull of the Past: When Do Habits Persist Despite Conflict With Motives? Personality and Social Psychology Bulletin, volume 37, pages 1428—1437.
Ashvin Shah and Andrew G. Barto (2009). Effect on Movement Selection of an Evolving Sensory Representation: A Multiple Controller Model of Skill Acquisition. Brain Research, volume 1299, pages 55—73.
Secrets of the Universe: Past, Present, FutureX