ADVERTISEMENT
  About the SA Blog Network













Observations

Observations


Opinion, arguments & analyses from the editors of Scientific American
Observations HomeAboutContact

How a Computer Game Is Reinventing the Science of Expertise [Video]

The views expressed are those of the author and are not necessarily those of Scientific American.


Email   PrintPrint



A crowd observes the match playing on the main stage at the StarCraft 2 championships in Providence, RI. Credit: Major League Gaming

If there is one general rule about the limitations of the human mind, it is that we are terrible at multitasking. The old phrase “united we stand, divided we fall” applies equally well to the mechanisms of attention as it does to a patriotic cause. When devoted to a single task, the brain excels; when several goals splinter its focus, errors become unavoidable.

But clear exceptions challenge that general rule. Two weeks ago, thousands of computer game enthusiasts descended on a convention center in downtown Providence, Rhode Island, to observe some of these exceptions in action. They were attending the championships of one of the world’s hottest computer games, StarCraft 2. Hands fluttered over keyboards like hummingbirds mid-hover at about fifty computers set up in a dimly lit open hall. Players, many of whom flew in from South Korea to compete, vied to advance through their brackets to the finals. This game is no joke, with the prize money to prove it—$50,000 went to the winner, a 16-year-old Korean who goes by the name Leenock. The agility on display in Providence —as seen in the players’ multitasking, their nonstop decision-making, and the stunning speed of their fingers—has not gone unnoticed by cognitive scientists.

For decades, a different game, chess, has held the exalted position of “the drosophila of cognitive science”—the model organism that scientists could poke and prod to learn what makes experts better than the rest of us. StarCraft 2, however, might be emerging as the rhesus macaque: its added complexity may confound researchers initially, but the answers could ultimately be more telling.

This real-time strategy game demands the frenetic pursuit of numerous simultaneous goals, any of which can change in the blink of an eye. Players play a god-like role over a cluster of creatures, leading them to develop their economy and prepare them for skirmishes with a neighboring society. Wildly popular among gamers—StarCraft 2 was the top-selling computer game in 2010, the year it was released—for researchers the appeal lies in the data each game generates. When two players face off, their computers each produce a record of the actions taken during the game. Called replay files, those logs reflect what a gamer was thinking at every stage of play. “I can’t think of a cognitive process that’s not involved in StarCraft,” says Mark Blair, a cognitive scientist at Simon Fraser University. “It’s working memory. It’s decision making. It involves very precise motor skills. Everything is important and everything needs to work together.”

That intellectual rigor and the corresponding data trail, multiplied across hundreds of thousands of players worldwide, makes StarCraft an unparalleled resource that scientists are only now tapping for the study of attention, multitasking, and learning. (As a rough estimate, at the time of writing about 11,000 games are being played on the servers of Blizzard Entertainment, the company that created StarCraft.) Recent experiments on computer games are beginning to suggest that players develop skills that could be useful in other contexts—skills that might allow those individuals to cope better with certain types of information overload. Thousands of these gamers are now contributing to a project under Blair’s watch, called SkillCraft, to learn what separates experts from novices and everyone in between. By all appearances this study of StarCraft players is the world’s biggest experiment on how expertise develops and, ultimately, on how we learn.

Video game research has reached a turning point: psychologists are no longer asking only whether the violence in some games corrupts young minds, or whether games are dangerously addictive, thus corrupting young minds. At least in some cases, gamers are being recognized for the specific forms of learning they cultivate, with the data trail that could finally unravel some of the major mysteries of the human brain.

Why StarCraft

To really appreciate why scientists are turning to StarCraft 2, you need to know a few basics of the game. Two players, connected over the internet, fight for control of a territory of which they have an aerial view. Each player begins with a small base of one of three species—terran (humans), zerg (insectoid creatures), or protoss (photosynthetic aliens). To win, one species’ army must defeat the other, a simple enough goal. But the number of variables that can shift during the game is enormous, demanding constant slight adjustments to strategy. Unlike in a board game, StarCraft players don’t take turns—they simply do as much as they possibly can and hope their opponent is not as fast.

The species’ first task is to start extracting minerals and a fictitious vespene gas, which form the foundation of a StarCraft economy. Each player must balance building up his or her economic production with developing fighters, defenses, and eventually more bases. A player must also try to discern the economic and military strategy of the opponent, whose base is initially hidden from view. As mineral and gas production ramps up, the gamer gains capital to spend on developing more advanced technology, a larger economy, or more fighters. StarCraft 2’s overarching strategic challenge is to decide how much time and money to devote to building up either an economy or an army.

But that’s just one level of play. When you attack, you do not simply dispatch fighters—you manually control them by using your mouse to click on them and set them in motion. A skilled player will monitor the health of individual fighters in the frontlines of a battle and pull them back to the rear as they lose strength, giving them time to recover while fresher troops bear the brunt of the attack. At any given time, you can have several dozen fighters with different abilities pottering around a map, numerous laborers busily mining minerals and gas, and various facilities producing new tools and resources that should be deployed immediately. With so many moving parts, even a top-level player can succumb to paralysis.

A screenshot from a StarCraft 2 game.

Translating those goals into cognitive load, the brain’s executive functions manage most of the game’s demands. Several types of memory may be engaged to keep track of the weapons at one’s disposal and the locations of multiple objects on a map; attentional systems allow a player to plan future moves, switch focus to different activities around the map, and evaluate the enemy’s strategy. Motor skills are needed to rapidly click around the map to move and implement actions.

In short, the game is a relentless exercise in multitasking and constant decision-making. The winner, often, is the person who can make the most moves—an elite player can perform about 5 or 6 actions a second, which translates into a flurry of key presses and mouse maneuvers (see video, above).

Commentators react to events in a game. Credit: Major League Gaming

But you don’t have to be elite to be intriguing from a psychological perspective. According to the Entertainment Software Association, 72 percent of American households play computer or video games. With people engaging with games at all ages, scientists have become increasingly curious about how visually arresting—and in the case of StarCraft and similar games, cognitively demanding—software might be interacting with the brain. “From the perspective of the cognitive motor system, StarCraft is the most interesting thing you could do online,” Blair says.

The Trouble with Brain Training

The question now tantalizing psychologists is whether the rest of us can learn anything from these hyper-specialized multitasking gamers. Perhaps we, too, can accomplish some 300 things each minute—such productivity! Maybe we can learn to pick up new skills faster or use such games to stave off aging.

Given appropriate practice, humans seem to improve on almost any task they tackle. Ask us to sculpt a cake in the shape of Pinocchio, and with sufficient time and motivation we probably will. But multitasking abilities tend to resist practice. Perhaps Starcraft, somehow, possesses the elixir that can morph us into successful multitaskers.

Part of the problem is that once developed, human skills generally stay specific to the original task. Expert chess players, for example, have significantly superior recall of the positions of chess pieces on a board after a brief exposure than non-experts. That is as you might expect: years and years of practice have produced deep familiarity with the arrangements of pawns, rooks, and knights and what strategic opportunities they represent. But chess players turn out to be no better than others when asked to remember the arrangements of chess pieces placed at random, in configurations that could not appear in a game. Experiments in numerous other domains have demonstrated a similar lack of transfer.

In the last decade, however, some experiments have begun to suggest that video games might indeed teach transferable skills. Cognitive scientist Daphne Bavelier at the University of Rochester and her colleagues have used video games to investigate what kinds of learning humans are good at, and along the way they’ve turned up some promising, if modest, examples of brain training. (When scientists search for newly acquired abilities that transfer from one domain to another, they boil them down to skills so basic that you could be forgiven for responding with the raise of a single eyebrow.)

Early results suggest that gamers may have faster visual reaction times, enhanced visuomotor coordination, and heightened ability to visualize spatial arrangements. They may also be better at rotating an object in their minds and may distinguish more deftly between the trajectories of moving objects. Players might also have an edge when paying attention to several objects at once.

Because the tests did not mimic exactly the characteristics of the games participants played, the researchers are optimistic that more general skills were heightened. “There are few obvious links between chasing monsters across a star-spotted “spacescape” and determining the orientation of a single black ‘T’ on a uniform gray background, or between driving a car through a crowded cityscape while shooting at rival vehicles and counting the number of white squares that are quickly flashed against a black background,” Bavelier and colleagues wrote in support of the transfer effects in a 2008 article in Psychology and Aging.

The literature, however, is no Greek chorus of collective ascent. Some studies have failed to replicate the benefits of gaming, as a recent review article points out, and most experiments in the field have struggled to prove that they’ve lopped off all of the placebo effect’s nefarious tentacles.

The vast majority of these studies focused on first-person-shooter games, which may share only a few characteristics with real-time strategy games. Nonetheless, small hints from the StarCraft 2 community also suggest that players might be developing some generalizable expertise. Most current top players initially participated in the original StarCraft, an obviously related but nonetheless distinct game, or Warcraft, another variation on the real-time-strategy theme. Of course, a selection bias may be muddying the waters: the elite players who are drawn to these types of games may already have stronger-than-average multitasking skills. Even so, the fact that long-time players entered the new game at a substantial advantage suggests that they had honed some sort of Starcraftian skill.

In a paper published this year, cognitive scientist Joshua Lewis and colleagues at the University of California – San Diego analyzed what actions players took in 2000 games to see if certain capabilities stood out as hallmarks of success.  Unlike previous studies, which tested participants before and after they played games to see if their behaviors changed, the approach taken by Lewis and colleagues allowed them to look for specific differences in what players are doing and perceiving.

They tracked several measures, including how many actions players took per minute and the distances between the locations where actions occurred across the map. Not surprisingly, they found that players who made the most moves tended to win. Of more interest was the second calculation. Distributing actions more widely across a map, which the authors argue reflects a player’s ability to distribute attention, also correlated highly with winning.

Now the question is whether people can learn to divide their attention more effectively. Professional Starcraft players belong to teams, with coaches and practice schedules, and they devote the majority of their time to developing their abilities. “If there is some methodology for building up multitasking skills, we might be able to figure out a way to train people to better distribute their attention,” Lewis says. “Maybe these teams have learned that implicitly.”

A New Model Organism?

In a traditional experiment on expertise, investigators corral about ten highly ranked professionals into a study and compare them with a similar number of novices. With StarCraft 2, scientists can mine the replay files of players at all stages, from chumps up to champions.

Blair, the Simon Fraser University scientist running the SkillCraft project, asked gamers at all ability levels to submit their replay files. He and his colleagues collected more than 4500 files, of which at least 3500 turned out to usable. “What we’ve got is a satellite view of expertise that no one was able to get before,” he says. “We have hundreds of players at the basic levels, then hundreds more at level slightly better, and so on, in 8 different categories of players.” By comparing the techniques and attributes of low-level players with other gamers up the chain of ability, they can start to discern how skills develop—and perhaps, over the long run, identify the most efficient training regimen.

LosirA, a 19-year-old multitasking phenom, merely thinks for a moment. Credit: Major League Gaming

Both Blair and Lewis see parallels between the game and emergency management systems. In a high-stress crisis situation, the people in charge of coordinating a response may find themselves facing competing demands. Alarms might be alerting them to a fire burning in one part of town, a riot breaking out a few streets over, and the contamination of drinking water elsewhere. The mental task of keeping cool and distributing attention among equally urgent activities might closely resemble the core challenge of Starcraft 2. “For emergencies, you don’t get to train eight hours a day. You get two emergencies in your life but you better be good because lives are at stake,” Blair says. “Training in something like Starcraft could be really useful.”

Indeed. But they know what we really want. Even more useful would be stumbling on the secrets of multitasking hidden in the replay files, then distilling them into helpful hints for the rest of us.





Rights & Permissions

Comments 23 Comments

Add Comment
  1. 1. vegastarcraft 11:12 pm 12/1/2011

    Very interesting article. As a sc2 player I’ll be interested to see what results are generated from this study and I will definitely be attempting to participate.

    Link to this
  2. 2. thetoyshoppee.com 9:37 am 12/2/2011

    Very interesting i feel that video game players able to multitask with ease an the speed in which they learn new skills is in some cases amazing i will be interested to see the results when published

    Link to this
  3. 3. pingywen 10:20 am 12/2/2011

    I would like to comment that the 3500 replay files these researchers have received, while provide significant amount of data, do not provide enough to represent the millions of people that play, and the millions of replays that are available. There are many pseudostudies by the Starcraft 2 enthusiasts with significantly more data. Perhaps they could team up with some of these enthusiasts and refine their studies and achieve more accurate and intensive data.

    Link to this
  4. 4. jennybates 6:05 pm 12/2/2011

    The author (and probably for that matter, scientists) doesn’t have a very good understanding of how advanced players play RTSes. Most players I know find a method to reach a certain goal within the least amount of time possible, usually some kind of tech or building. These players aren’t multitasking; they’re actually mindlessly clicking on the buildings and items in order to reach a certain goal. They’ve programmed their brains (and their hands) to do certain things in a certain order and respond instinctively to certain events in the game (like random events or surprise attacks). How do I know? Ask the advanced players to change just one step in their strategy, and see how long it takes them to adjust. If they are multitasking, it shouldn’t take long at all for them to adjust, as their brains are thinking about strategy constantly. But in reality, it will take quite awhile, because the player will have to think to remember what the change was. It’s no longer habit.

    Also, ask an advanced player how many different buildings he has. The answer probably is surprisingly small, and most of those may not even be functional, but are more likely to be support structures or structures needed to build other structures. Again, the brain is focusing on as few variables as possible; this is very similar to the brain’s ability to follow only three musical lines at the same time.

    Is this a wonder of the mind? Sure. But I wouldn’t call it a wonder of multitasking. It’s more like the skill a ballerina dancer uses to perform. Is she thinking about the placement of every single limb and digit? No, she’s gained muscle memory and only focuses on the parts she needs to. Same with video gamers. You learn a skill to the point that it’s muscle-memory and then pay attention to what you need to.

    Link to this
  5. 5. Blckmgc 7:31 pm 12/2/2011

    One point of correction – the Protoss are psionic, not photosynthetic.

    Link to this
  6. 6. jonloi 11:15 pm 12/2/2011

    @jennybates this isn’t multitasking?
    have you even experienced the game? or do you consider all games with the definition RTS equal?

    this is not pattern or habitual reproduction, there are various actions to perform in the game, as the author mentions, the ability to perform more actions per minute (defined as APM) is advantageous and directly translates to higher success/winning rates.

    a good example is as you move your built forces across the map to engage/destroy your opponent’s army/base structures, you have to simultaneously build more structures,set up ambushes, reposition forces, continue production or switch production based on your opponent’s army composition (build units to counter opponent’s units) and pursue unlocking/climbing technology trees.

    a constant eye has to be kept on your various armies while doing the above IS multitasking. Walking into an ambush or engaging at a disadvantageous position will cost the player the game and it is easy to do so if you are engrossed in the various base activities.

    vice versa, solely controlling your army to micromanagement perfection will lose you the production(macro) war as your opponent can overrun you with superior economy and production even, essentially you can win a battle but lose the war.

    I consider juggling a habitual pattern programming ability.

    Link to this
  7. 7. Gfletchaa 12:27 am 12/3/2011

    Blckmgc : Actually, they are photosynthetic. Check the link.

    Link to this
  8. 8. cogwheel 3:54 am 12/3/2011

    @pingywen: They were looking to collect 20,000 replays according to their original post on the StarCraft 2 forum:

    http://us.battle.net/sc2/en/forum/topic/2973252447#1

    Link to this
  9. 9. jvin248 9:25 am 12/3/2011

    Having played these games myself too, JennyB above is closer to the solution. The more a player can build habits on the background mechanics the better they can focus on the important events. Like keeping production going while an enemy is attacking; you sweep the secondary circuit on a regular frequency (“do I have any idle workers?” “are they ready for the next building?”). But you keep focused on that front line and the tactics to win there.

    I would suggest that when the researchers peg what they consider the best players of StarCraft 2, that they take those players and drop them into an unfamiliar but similar game. A couple of open source candidates, because they are free and easy to obtain, may be Glest/MegaGlest and manybe the Battle for Wesnoth. All are available on the Linuxgamers live DVD so it’s easy to get games running.

    I would also caution the researchers that if they believe that winning these games is due to exceptionally high Actions Per Minute, then they need to verify the hardware and internet bandwidth used by all the players. Is there a ‘ping’ speed or latency data bit encoded in the saved play files? Are hardware characteristics stored there like CPU, Ram, Graphics CPU, ethernet port speed? What gaming controller are they using (mouse and keyboard or something more advanced)? Faster feedback loops for players means they can assess and make adjustments quicker.

    .

    Link to this
  10. 10. jennybates 5:55 pm 12/3/2011

    @JonLoi – I have a very varied gaming background, everything from the Sims to Counter-Strike. On RTS, the list includes Alpha Centauri, Red Alert, Tib Sun, Star Trek: Armada, so on. Haven’t touched the SC franchise, but from what other players tell me, my background fits.

    As far as APM go, that actually points more towards a gamer using his instinct. If you have to think, you take too long, and the APM will actually be lower. Think about a brand new player who has to sit and think about every action they take. Their APM will be in the toilet. An experienced player who automatically knows where on the screen to click, what to click, and when, will execute such actions automatically, without thinking.

    Ever seen Top Gun? This convo applies:
    “What were you thinking?”
    “Up there, you don’t have time to think. If you think, you’re dead.”

    Same here. Thinking takes to long. You go through training like flying an airplane, playing a game, playing a musical instrument, it’s all based off learning something to the point it becomes automatic and instinctual. Hence the phrase, you never forget how to ride a bike. The involuntary, subconscious part of your brain may be active, but you aren’t consciously focused on it.

    Link to this
  11. 11. kelly.truckdriver 10:46 pm 12/3/2011

    @jennybates 1. Alpha Centauri was turn-based, not RTS. All Sid Meiers’ games are turn-based, like a board game. 2. If it were a case of memorizing what to do and a build order, then it wouldn’t be the most popular and highest paying e-sport. The fact is these guys; the real pros, adjust their build order on what they’ve scouted, how large the map is, how close they are spawned to their opponent, what faction they are playing and playing against, what their opponent seems to be up to, what they want their opponent to think they are up to.. and they can do all this while skirmishing or harassing away from their base. It is incredible multi-tasking and quick decision making. You or I might have to memorize 2 or 3 or even 10 build orders, but you’re looking at the pros here who have hundreds and are able to change on the fly, all while invading, defending, or harassing their opponents production. It is amazing to watch, please try to watch it at gomtv.net or something before you pass judgement and try to make other people think something on a game you have not played or watched yet. It is simply un-scientific to post judgement or opinion on a game you have not admittedly played.

    Link to this
  12. 12. Magallanes 12:15 pm 12/5/2011

    I agree that it is not multitasking.
    The process to master it is :

    a) learn about the units.
    b) learn some statistic approach mostly by force brute but also for external help. I.e. you learn that (for example) zerg are quick to build by practice but other units are not.
    c) Apply some pattern.

    In Starcraft, you can control up to 100 units (more or less), however, the trick is that you can control a group at once and leaving some units working automatically. In fact, most of the time, you are controlling two or three army, your base and nothing else much.

    Link to this
  13. 13. MadBoat 3:08 pm 12/5/2011

    I wonder if watching Day9′s newbie lessons would be of any help to this study? http://www.youtube.com/watch?v=-jxdX3514t0&feature=fvsr

    The thing that sticks in my head from Day9 is his advice to create a mental checklist, and every so often run through that checklist. (idle workers? no. Production buildings? producing and not queued. good. Now what was I doing? oh right, trying not to die.)

    Link to this
  14. 14. obryan 9:44 am 12/6/2011

    @jennybates On the contrary, I believe that to mindlessly click buildings repetitiously from instinct/habit, and then to pay attention to “other things” within the game (in order to adjust to the opponent, etc) is the definition of multi-tasking. You are literally doing two things at once, accounting for multiple scenarios simultaneously.

    You say it is not the a multi-tasking thing but something else. I disagree.

    Link to this
  15. 15. nhat11 12:30 pm 12/6/2011

    @jennybates What you descirbed is multitasking…. Doing multiple actions all at the same time. Managing econ, creating strats, keeping army upkeep, attacking, defendings all at the same time. How is that not multitasking?

    Link to this
  16. 16. jgrosay 1:49 pm 12/8/2011

    It was said that Napoleon Bonaparte was able to give attention to six conversations simultaneously. This ability seems linked also to multiple personalities being present in the same man/woman, multiple personalities being a psychiatric disorder. As it has been shown that in people with hydrocephalus, a brain cortex as thin as less than one centimeter is able to carry on all activities of day living, without a noticeable impairment, our brain has probably room for much more than 6 personalities, or to pay attention to much more than two things at the same time. Someone has a good user manual for human mind ? Some years ago, a book with a similar title sold many copies.

    Link to this
  17. 17. Strmcrw 2:29 pm 12/8/2011

    Nobody seems to realise that all pilots are excellent multitaskers. This applies to private pilots all the way up to the Captain of a 747. Private pilots have to navigate, map-read, watch where they are relative to the runway, etc, watch the condition of the engine(s) by sound as well as by instruments, communicate with air-traffic control, oh, and, by the way, they have to fly the plane. A 747 Captain has to do all this stuff, as well as pass a checkout, every six months, in a (very realistic) simulator, where he may get an engine fire as he lifts the nose wheels on takeoff. If this isn’t multi-tasking, I don’t know what is. Pilots fly in the real world: there are crashes every day in the real world. These computer games are a parlour game/walk in the park.

    Link to this
  18. 18. redeux 3:41 pm 12/8/2011

    I must’ve missed the part where someone failed to recognize pilots as excellent multi-taskers. I’m sorry but you’re the epitome of a typical internet user who feels the need to shit on someone else’s viewpoint to build up their own. No one diminished what pilots do and the only thing you’re doing here is diminishing what we, esports, do. You said it yourself, if that’s not multi-tasking then you don’t know what is. Just because we don’t put hundreds of lives at stake everytime we practice doesn’t mean we don’t multi-task. If anything, SC2 players can PRACTICE multi-tasking hundreds of times easier than a pilot. Professional gamers such as the ones mentioned in this thread live in team houses where they practice 12+ hours a day. Now, tell me the total flight time a pilot has and imagine someone such as SlayerSBoxer who has played starcraft for years as a career? I’d daresay 이묘환 has more practice with his multi-tasking than _most_ pilots.

    Now, I’m not diminishing what pilots do, but please, manner the fuck up before you shit on esports. Taking a crap on someone elses viewpoint does not make your argument correct. Goodbye sir.

    Link to this
  19. 19. haleti 2:34 am 12/9/2011

    Hi,
    “If there is one general rule about the limitations of the human mind, it is that we are terrible at multitasking”.  This not quit true, a single neuron is computational very weak, its power and that of the human mind is in its connectivity and thus multitasking. A distinction between behaviourally multitasking which is mechanical must be made.
    Regards Dr. Terence Hal

    Link to this
  20. 20. rroto1 4:09 pm 12/9/2011

    The flaw is in SC2 is a very poorly written game. Its AI for the individual units is poorly written. They were not programed to fight in the most optimal manner. Smart players must overcome this by overriding the programing and saving them to fight another day.
    I find this type of poorly programed game very boring. You have to waste your time in trivial pursuits But…. for 50K it could become interesting.
    As a student of video game AI I learned that a human opponent is not boring, but SC2 is.

    Link to this
  21. 21. Aweather 1:30 pm 03/5/2012

    I have to agree with rroto1. SC2 is a very poorly written game. However, it is the poorly written part that made the game popular. I believe a well written AI can beat at least high level players in SC2 though they don’t have a API (only for SC1) yet so I wasn’t able to develop one.

    SC2 strategy wise is simple compared to the most complicated turn based games on the world (go, for example). If SC2 has very good AI and controls, the optimal build (or their counter) will just be used by everyone and the game will be like paper-scissor-rock.

    Just my two cents.

    Link to this
  22. 22. cinque_verdi 9:10 am 04/22/2012

    Very interesting article.

    I would like to share some of the content on my blog at http://www.airjig.com/ if you don’t mind.

    I agree with haleti, we should make a distinction of mechanical multitasking.

    Link to this
  23. 23. kamalmoonlight 2:59 pm 05/21/2014

    hi for everyone I want to show u the link to Download free FIFA 2013 Full version with Crack to this link:
    http://okayfiles.com/file/04L6F1

    Link to this

Add a Comment
You must sign in or register as a ScientificAmerican.com member to submit a comment.

More from Scientific American

Scientific American Holiday Sale

Limited Time Only!

Get 50% off Digital Gifts

Hurry sale ends 12/31 >

X

Email this Article

X