October 14, 2011 | 11
As soon as I saw the headline “Research sheds light on origins of greatness” , my interest was piqued. The article is referring to a new paper in Current Directions in Psychological Science, so I immediately downloaded that paper and left the press release open to the side. I’m wary of press releases with these sorts of headlines so best to go right to the source. Scanning the paper, which is coauthored by David Z. Hambrick and Elizabeth J. Meinz, I realize it’s a summary of research they’ve already conducted (some published, some not). As I read about their studies I noticed that not one of them actually looked at greatness.
In a nutshell, their impressive body of research shows that working memory—the ability to simultaneously hold information in memory while processing other information—is correlated with performance on different “complex tasks” in the laboratory, including remembering baseball information, Texas Hold’em poker performance (their manuscript on this topic is submitted for publication), memory for the movement of spaceships and baseball players, and piano sight-reading performance. What’s more, working memory performance is still correlated with these “complex tasks” even among individuals with high levels of specific experience and knowledge for the domain. Hambrick and Meinz conclude “although deliberate practice may well be necessary to reach a very high level of skill, it is not always sufficient.”
I thought about this statement, and how the media interpreted it. Something just didn’t feel right. It seemed like they were setting up a straw man. Who really thinks that deliberate practice is completely sufficient for greatness? As if it’s even a possibility that we’re just machines, and the only thing we do is input knowledge and export greatness. Surely Hambrick and Meinz meant that deliberate practice might not be sufficient without accompanying high ability, not that deliberate practice might not be the only factor contributing to greatness. Even Anders Ericsson and his colleagues—who study the importance of deliberate practice for attaining elite performance— would have to admit that other factors come into play, such as inspiration, motivation, passion, perseverance, mindset, and self-belief. Environmental circumstances also of course play a huge role, as does just plain luck (i.e., being in the right or wrong place at the right or wrong time).
So I decided to flip the question on its head. The most interesting question isn’t whether deliberate practice is both necessary and sufficient for elite performance, but whether high ability is necessary for greatness. My reading of the literature is that Ericsson’s specific argument is that (with the exception of innate constraints on body size and height) genetic endowment doesn’t put limits on the ultimate level of elite performance healthy individuals are capable of attaining. This is a different question than asking whether differences in ability contribute to differences in elite performance. Genes contribute to all of our traits (including our tendency for perseverance), and all of our traits contribute to our greatness. But this doesn’t mean that our genes necessarily constrain our potential for greatness.
The correlations found by Hambrick and Meinz, which were far from perfect, suggest that ability contributes to greatness, not constrains it. Take their study on piano sight-reading skill. They found that deliberate practice was correlated .67 (p < .01) with sight-reading performance, while working memory was only correlated with sight-reading skill .28 (p < .05). After looking at both variables simultaneously (in a regression model), they found that deliberate practice accounted for about half of the differences (45.1%) in sight-reading performance whereas working memory accounted for an additional 7.4% of the differences in sight-reading performance above and beyond deliberate practice.
A few things to note:
- The researchers call this 7.4% increase in prediction “significant”. While they are referring to significance in the statistical sense, keep in mind that we’re talking about explaining differences. The correlation between working memory and sight-reading was .28, so 7.8% of the differences in sight-reading were explained by working memory. This means that 92.2% of the differences in sight-reading skill were not explained by differences in working memory performance. I’d say 92.2% is much more “significant” than 7.8%, at least in terms of practical meaning!
- Deliberate practice explained a much larger (45.1% vs. 7.4%) amount of the differences in sight-reading performance compared to working memory when both variables were looked at simultaneously. The facts are quite different than the media would have us believe. Look at this headline, from Psych Central: “Key to Greatness is Working Memory, Not Practice”. First of all, their studies actually show that both working memory and practice contribute to differences in performance, not one or the other. Second of all, we already determined that their studies don’t actually involve greatness. Third of all, practice is actually a much better predictor of “complex task” performance than working memory. That headline is wrong on so many levels, I could hurl!
So again I ask: is high working memory necessary for greatness?
To explore this question, I decided to look at a dataset I already had sitting on my computer. A few years back I went to Cambridge, England and administered psychological tests on high school (“Sixth Form”) students as part of a larger study I was conducting. I had measured working memory using the Operation Span Task), which asks participants to remember words while simultaneously verifying whether easy mathematical equations are correct (e.g., “Is 2 + 4 =6?”). Working memory can become more burdened by increasing the number of words to remember. The minimum number of words I had participants remember was 2, and the maximum was 6. It’s actually pretty darn difficult to keep that many words in your head while you’re also processing math equations!
I also assessed creative achievement across 10 different arts and sciences domains (Visual Arts, Music, Dance, Architectural Design, Creative Writing, Humor, Inventions, Scientific Discovery, Theater and Film, and Culinary Arts) using the Creative Achievement Questionnaire. For each domain, participants had to check off their level of achievement. For instance, for Inventions, the scale ranged from “I do not have recognized talent in this area” to “I have sold one of my inventions to a manufacturing firm”. While these were just high school students, some had accomplished quite a bit (this may have to do with the intense specialization of the British school system).
Once I dug into my data, I realized a few things that make this analysis very tricky. Working memory and greatness have very different distributions in the general population. Hambrick and his colleagues can find linear correlations because they are comparing two different variables that are both normally distributed. Any complex cognitive task that is administered in the laboratory is probably going to be normally distributed (and is also going to have a good chance of being positively correlated with each other, as well). Greatness, on the other hand, is a whole other ball of wax.
By definition, greatness is rare; only very few people reach greatness in any domain. Greatness is also combinatorial: as already mentioned, it consists of many, many traits and environmental influences that interact. Sight-reading skill may contribute to musical greatness, but musical greatness is much, much more than sight-reading skill! And anyway, sight-reading skill isn’t even necessary for musical greatness. Lots of great musicians—especially jazz musicians—can’t even read music. Some, like Ray Charles, even have learning disabilities that physically prevent them from sight-reading music.
To illustrate the different distributions of working memory and greatness, I plotted some graphs. Here’s the distribution of working memory scores in my sample of 177 British high school students:
See how it conforms pretty well to the bell curve? This is roughly what a normal distribution looks like.
Now let’s look at the distribution of greatness:
See the difference? Greatness is not normally distributed among the students in my sample (this distribution is similar to what others looking at creative achievement have found). Consistent with the idea that greatness is rare, greatness scores are highly skewed to the right. Most people fall within a particular range, and only very few are at the very right of the tail. Again, this is because greatness is combinatorial, comprising many different traits and environmental conditions that interact with each other. It’s a lot easier to memorize information or sight-read music if you’re a professional musician than it is to be a great musician!
The differing distributions make my analysis tricky because whatever relationship between working memory and greatness does exist, it ain’t gonna be all that linear. So I did the best with what I had and calculated my correlations using a Spearman rho rank correlation coefficient).
What did I find? Lo and behold, working memory was not statistically significantly correlated with total greatness scores. So I looked at separate domains. Things weren’t looking too promising here either; working memory didn’t correlate with creative achievement in most domains (which were mostly focused on the arts). That is, until I got to scientific discovery. Aha—a significant correlation! Very small (ρ= .16), but statistically significant nonetheless. (I should note that while the correlation between working memory and music creative achievement was not statistically significant, the correlation did approach statistical significance).
So I zoomed in on scientific discovery, by making a scatterplot of the relationship between working memory scores and scientific discovery greatness scores. Here it is:
Like the total greatness scores, scientific discovery is also highly skewed to the right. 88.1% of all my participants fit within levels 0 and 1 of scientific discovery greatness. In fact, 97.2% fit within levels 0-4. As suspected, whatever linear relationship does exist is very minimal. Those circles to the right seem mighty lonely, don’t they?
Those aren’t just circles, by the way. Each one represents a different participant—a unique living, breathing, human being, to be exact. There were 177 total human beings in my sample, and each circle represents their level of working memory and greatness for scientific discovery. See all those circles at the top left corner? That’s right, those represent the slackers—the ‘underachievers’, some would say. Their working memory scores are high, but they aren’t achieving much in scientific discovery (although, of course, not all my participants were equally motivated to achieve in science, particularly those at level 0, and their greatness may lie in other domains.)
But, do you see that big circle— the one that looks like a smiley face? Well, that represents a person too. And why did I make that one person stand out among all the rest? Because that person is my existence proof that a high level of working memory isn’t a necessary condition for greatness.
Let’s see why that’s the case. The average working memory score in my sample was 44.63 and the standard deviation (or how much variation there is from the average) was 8.56. The minimum working memory score in my sample was 24 and the maximum was 60 (which was also the maximum for the test). This smiley, blue circle represents a person “who won a prize at a science fair or other local competition”, and also “won a national prize in the field of science or medicine”. Quite impressive for a high school student! What’s more, this person won these prizes with a lower than average working memory score (35). Technically, about 68% of the other students in my sample scored higher in working memory than this person, but this person still stands heads and shoulders above virtually all the other students in terms of scientific discovery greatness!
Of course, it’s entirely possible that this smiley face kid either lied about his scientific discovery creative achievement or didn’t try very hard on the working memory task. So don’t take this data as gospel (in fact, you shouldn’t take any single dataset as gospel)! Still, I hope this illustrates a very important point: it’s so easy for psychologists, desperately trying to find linear, straight lines, to miss this person. Clearly, whatever this person lacks in working memory (or at least in his or her ability to do well on that task that day), they make up for in other ways. This person, in my opinion, is a true outlier.
A few years back, I was kindly invited by one of Ericsson’s former students, Edward Cokely, to give a talk at the Max Planck Institute for Human Development, where he was working at the time as a postdoctoral Fellow. At dinner, we were discussing Anders Ericsson’s theory of deliberate practice and I was questioning how it could possibly be true. I had been conducting my own research on the predictive value of cognitive ability and was finding statistically significant correlations with fancy outcomes such as academic achievement and even creative cognition as measured in the laboratory.
Then we started discussing individual cases of people who surmounted seemingly insurmountable physical and cognitive limitations, only to become great in their field. Edward described these people as “existence proofs”; they were proof that greatness is possible, despite their apparent weaknesses. I never forgot that term, which is why I used it to describe my blue-face outlier (whose real identity I will never know). For these existence proofs, their lower ability does not constrain their ultimate levels of performance; they are able to overcome their limitations. Perhaps they even attained greatness because of their limitations!
At one level of analysis—the group level—Hambrick and his colleagues are surely scientifically correct: ability matters all across the performance spectrum. Differences in working memory performance are slightly to moderately but significantly (at least statistically), related to performance on measures of complex cognition under laboratory conditions, and these effects don’t weaken at the high ends of domain-specific knowledge. Research also shows that early ability matters. In a recent review, Kimberley Robertson and her colleagues showed that among a group of adolescence, both domain-general and domain-specific abilities even at the very top 1% were statistically predictive of the likelihood of educational, occupational, and creative outcomes decades later in life (although independent of that, measures of educational-vocational interest and lifestyle preferences also had a significant effect). These are certainly interesting findings, and popular writers such as Malcolm Gladwell or David Brooks are technically (or scientifically) incorrect if they claim that ability does not matter. Perhaps a more nuanced view is that the importance of different abilities and traits depends on the domain in question. For the arts, the type of ability measured on IQ tests appears to be less important than for scientific discovery. At least when looking at group averages.
But at another level of analysis, this debate breaks down. While it’s fun for scientists to find order among apparent chaos, let’s remember that we’re talking about chaos among human beings, not fractals. Each participant comes to the experiment with their own unique constellation of traits, abilities, inspirations, motivations, passions, desires, life circumstances, and life experiences. There are so many different paths to greatness. The name of the game is strengthening what you’re good at, and compensating for your weaknesses. This is actually part of the definition of intelligence, at least as defined by my former advisor Robert J. Sternberg.
Everyone who is alive has at least some level of working memory, and they can use it the best they can, while trying to reduce their mental burden through the use of technology and collaboration. Even though working memory is heritable, this doesn’t mean it’s fixed at birth. Recent research suggests that working memory can be improved through training. Psychological experiments that intentionally make it as hard as possible for mostly uninterested participants to solve a task underestimate the power and possibility of compensation in the real world given sufficient motivation.
So next time you see a study that says some ability is necessary for some form of greatness, remember that this isn’t necessarily the case. You can personally get there, regardless of the group trend. After all, working memory is common, but greatness is rare.
Note: In their paper, David Z. Hambrick and Elizabeth J. Meinz refer to “working memory capacity”. Here, I am referring to the same construct when I talk about “working memory”. I dropped the “capacity” bit since some researchers have convincingly argued (at least to me) that working memory is not a capacity, per se, but more of an ability for attentional control.
Give a 1 year subscription as low as $14.99X