ADVERTISEMENT
  About the SA Blog Network













Beautiful Minds

Beautiful Minds


Insights into intelligence, creativity, and the mind
Beautiful Minds Home

In Defense of Intelligent Testing

The views expressed are those of the author and are not necessarily those of Scientific American.


Email   PrintPrint



I recently published a book called Ungifted: Intelligence Redefined. With a title like that, you’d think the book is one big anti-IQ, anti-testing manifesto. It isn’t. While the IQ test certainly has a less than pretty history of abuse and misuse, we can learn a lot about a child’s educational needs through the responsible use of intelligent testing.

A generation ago, Alan S. Kaufman introduced the notion of intelligent testing, which represented his philosophy of how IQ tests should be used. According to this approach:

“The focus is on the child, with…communication of the test results in the context of the child’s particular background, behaviors, and approach to the test items as the main goals. Global scores are deemphasized, flexibility and insight on the part of the examiner are demanded, and the test is perceived as a dynamic helping agent rather than an instrument for placement, labeling, or other types of academic oppression. In short, intelligent testing is the key.”

In a 2009 edited volume in honor of Alan Kaufman (Intelligent Testing: Integrating Psychological Theory and Clinical Practice), the neuropsychologist Elaine Fletcher-Janzen noted how Kaufman’s Intelligent Testing approach “became the gold standard for psychometric test interpretation and clinical assessment.”

Unfortunately, not all clinicians apply the gold standard when testing a child. In Ungifted, I describe a traumatic situation in high school when a school psychologist provided the bronze standard. This experience, along with the many others that came before it, left me feeling ungifted despite my high academic performance. These unfortunate educational experiences set off a downward spiral of self-esteem and anxiety that affected my actual possibilities in life.

But my personal experiences don’t invalidate the potential utility of IQ testing. My views on the utility of IQ testing have softened over the years, even though I still have a healthy dose of skepticism for its value. I’m against “intelligence” testing, in which we use the tests for the purpose of sorting the “gifted” from the “ungifted”. I’d like for more dynamic testing, and am against comparing each person’s level of intelligence to each other based on a single standardized metric, such as a global IQ score, measured at a single moment in time. I’d like for us to take into greater consideration each person’s unique package of personal characteristics and personal goals. I’d like to see creativity and imagination better measured and appreciated. Same for unconscious, intuitive modes of thought. I’d like the tests to only help, and never limit. I’m not convinced the type of problems on the tests are as exciting as can be to motivate students to want to solve them. I have many other criticisms, which I mention in Ungifted.

But after spending years conducting research on intelligence, I have come to realize a few things. There is no doubt that there are some shoddy test administrators and intelligence researchers who are biased by their prior beliefs and are out of touch with the latest research. But to the credit to IQ test makers– the field is rapidly evolving. IQ test constructors have tried to address the critics on many key points. On the whole, they are moving away from a focus on a single IQ score. Instead, they are relying on updated models of intelligence, such as the Cattell-Horn-Carrol (CHC) theory that emphasizes multiple cognitive abilities. In fact, there is a trend now to not even refer to IQ tests as measures of “intelligence” but instead refer to them as tests of “cognitive ability”. Some tests are even based on neurological functioning.

Most (but not all) researchers fully admit that IQ tests only measure a piece of the intelligence pie (although they argue it is an important piece). Most intelligence researchers are actually pretty nice, sensitive individuals who sincerely want to help the welfare of children. They want to create tests that help teachers identify the special needs of students and then select the right intervention. They are scoring tests using sophisticated statistics and encouraging the use of the intelligent testing approach and the use of cross-battery assessment techniques to increase fairness in measuring the cognitive abilities of those from diverse backgrounds.

So is there any utility to modern IQ tests? This is an important question still open to discussion, but I think past American Psychological Association President Diane Halpern put it quite reasonably when she said:

“We will always need some way of making intelligent decisions about people. We’re not all the same; we have different skills and abilities. What’s wrong is thinking of intelligence as a fixed, innate ability, instead of something that develops in a context.”

I have particular respect for Alan and Nadeen Kaufman (they are no relation to me*), whose tests have been on the forefront of the evolving nature of IQ tests. Alan’s intelligent testing approach makes sense to me. It elevates the clinician above the test and enables the test administrator to treat each child as an individual. If we have to use a test, it allows us to use that test usefully– to go beyond a single IQ score as the clinician observes the child’s profile of cognitive processing strengths and weaknesses with the goal of custom tailoring a program to help that particular child.

In honor of Alan’s intelligent testing approach, and his 2009 book IQ Testing 101, which provides a nice summary of the current state of the field of intelligence testing, I conducted a short interview with him. What follows are words from an actual IQ test maker.

What is the main tenet of the intelligent testing approach?

To me, the main tenet is that there is a hierarchy between the IQ test and the clinician who administers and interprets the test–and it is the clinician and not the test that is at the top of the hierarchy. Yes, it is important to use carefully-developed, well-standardized, theory-based tests of intellectual development. But the scores on such tests are meaningless unless they are interpreted by sophisticated clinicians who are totally knowledgeable in pertinent research on child and adult development, cognitive psychology, and neuropsychology, and who are astute observers and interpreters of behaviors (such as the person’s approach to problem solving or the degree to which anxiety or distractibility might have compromised the person’s test scores). Intelligent testers also must have a healthy respect for what tests cannot do, such as measure two-thirds of Sternberg’s triarchic theory of successful intelligence (i.e., practical intelligence and creativity).

In what ways is modern IQ testing misunderstood by the general public?

The general public tends to misunderstand the kinds of IQ tests that are administered by psychologists to children or adults who are referred for evaluation. Such referrals are for a diversity of reasons such as suspected brain damage, learning disabilities, intellectual disabilities, behavioral disorders or intellectual giftedness. The tests are clinical instruments that are administered one-on-one for an hour and a half or two and can only be given by professionals (usually psychologists) who have a high degree of supervised training in intellectual assessment. I believe that most people tend to hear the words IQ test and think of the kind of paper-and-pencil IQ tests that they took in school. Such group-administered IQ tests are still commonly used, but not for making important real-life decisions. There is also a second level of misunderstanding by the more sophisticated lay public–and sometimes by psychologists whose field does not include clinical assessment, or by special educators who generally are not qualified to administer clinical tests of intelligence. And that is the notion that IQ test = Wechsler scale (or sometimes the old Stanford-Binet) and that the most important scores to interpret are the Verbal IQ, Performance IQ, and Full Scale IQ. In fact, Wechsler’s scales remain the most popular, but there are many other widely-used tests to choose from, and these tests uniformly are based on theories of intelligence and neuropsychological processing. Furthermore, these tests emphasize a person’s profile of theory-based cognitive abilities, not their global intelligence. And even the latest versions of the Wechsler scales are based on theories and research in cognitive neuroscience and have eliminated Verbal and Performance IQs in favor of standard scores on four theory-driven indexes.

Individually-administered IQ tests are still used to make important real-life decisions though, right?

Individually-administered tests are ordinarily used for important decisions such as diagnosis or placement in special education, but some people still take short cuts. Group IQ tests such as the Otis-Lennon are sometimes used to identify gifted students.  In fact, New York city uses the Otis-Lennon along with an individual screening test for gifted assessment. My own opinion is that NO major decision about a person’s life–child or adult–should be based on a group-administered IQ test.

Is it ever appropriate for a global-IQ score or specific cognitive ability score to be used as a cut-off to get into gifted education or to make any other important real-decisions?

Cut-off scores of any sort are a violation of what we have known for years about errors of measurement. The only situations where cut-off scores make any sense are for the diagnosis of intellectual disability (which used to be known as mental retardation). That diagnostic category is defined specifically by low intellectual functioning. However, even in such diagnostic circumstances, errors of measurement must be taken into account, and an IQ cut=off is not sufficient for the diagnosis–the person must also be shown to have very low adaptive behavior (akin to social intelligence).

Are global-IQ or specific cognitive ability scores ever invalid for an individual? In what ways could they be invalid?

It is common for IQs or scores on separate scales to be invalid for an individual. the tests are administered individually, and rapport with the examiner is important. If the examiner doesn’t succeed in establishing and maintaining rapport with a child or adult, then that person might not give full effort. Apart from low motivation, IQs are sometimes invalid estimates of a person’s true functioning because behaviors such as distractibility, anxiety, or low frustration tolerance can interfere with test performance and lead to a low IQ or a low standard score on a test of working memory–even when the person truly has exceptional ability.

How much can these scores change over a person’s lifetime, and how limiting are a person’s scores for obtaining what they want out of life?

For groups of individuals, IQs are fairly stable between childhood and adulthood, but for specific individuals within a group, IQs can–and do–vary greatly over a lifetime. The IQs will vary as a result of specific interventions (such as preschool enrichment programs), quality education (or the lack of it), injuries that affect brain functioning, and other aspects of the environment that either enhance or diminish one’s cognitive ability. In addition, errors of measurement are much larger than people tend to think, and, therefore, an individual’s IQs will vary from time to time–sometimes substantially–simply due to the chance fluctuations that accompany any repeated measurement. And, there is more to life success than the ability to score high on IQ tests. People can be successful based on their creativity, street smarts, and personality variables.

How can IQ tests be misused?

IQ tests are misused when a person is given a number or a set of numbers that supposedly define their intellect, but are not given an explanation of what the numbers mean. They are also misused whenever IQs and other standard scores are presented without providing confidence intervals around the obtained scores. They are misused when they are used as the sole method for making a decision, such as special education placement or being hired for a job or gaining entrance into a prestigious program.

It seems to me that since IQ-global scores or even specific cognitive ability scores derived from individually-administered testing sessions are still used to make important real-life decisions–even today– it is all the more reason for clinicians to adopt the intelligent testing approach. I’m sure you agree?

I do agree, naturally, because that is what I have been “preaching” for 30 years.

© 2013 Scott Barry Kaufman, All Rights Reserved

Portions of this article originally appeared at Psychology Today on October 25, 2009.

 

* Personal Aside: Life is funny. When I was a child, my bad experience was with the Wechsler Intelligence Scale for Children (WISC). This experience contributed to my decision to become an intelligence and creativity researcher. As it turns out, when Alan Kaufman was at The Psychological Corporation he worked with David Wechsler on the revision of the WISC– the exact test that was the bane of my existence. Here I am, years later, collaborating with Alan and his son James. I am thankful that they have been so supportive of my career and hold them both in very high esteem.


Scott Barry Kaufman About the Author: Scott Barry Kaufman is Scientific Director of The Imagination Institute in the Positive Psychology Center at the University of Pennsylvania. Follow on Twitter @sbkaufman.

The views expressed are those of the author and are not necessarily those of Scientific American.





Rights & Permissions

Comments 9 Comments

Add Comment
  1. 1. David Edenden 12:12 pm 07/15/2013

    The “Intelligence Quotient” (IQ) test should be re-named the the Knowledge Quotient (KQ).

    I know a person who speaks (well) and reads (basic) three languages with three different alphabets. His children and grandchildren include college teachers, a doctor and high school teachers, stock broker etc.

    However the only form of schooling he has had was grade 6 in one language. He speaks all three languages with accents showing him to be “uneducated”. His acquisition of knowledge is low.

    He is very intelligent, but would be classified as [mentally disabled*] by current the “IQ” standard since he could not complete the test in any language.

    His KQ results would also show him as having a very low score (because lack of formal education) without casting aspersions to his intelligence, mental capacity, DNA, culture, etc.

    I think I have solved the problem. Don’t thank me know, thank me later.

    * Word changed by the author to reflect the official phrasing of the term.

    Link to this
  2. 2. TTLG 1:35 pm 07/15/2013

    It sounds like the understanding of human intelligence is getting better, but I think that it still has far to go. For instance, it looks to me that people who grow up in different environments, such as dangerous or violent areas, do not have stunted mental development, but instead develop in very different ways from those of us who grew up in safe and intellectual-oriented environments. Take a person who grew up in one environment and put them in the other, and I think that that person would be very unsuccessful no matter which direction the change was. But since the people who make the various intelligence tests grew up in the intellectual environment, they do not understand or value the skills needed to succeed in other circumstances. I think that having a more open-minded approach to understanding the type of thinking that goes on in people who grow up in different ways may greatly increase our understanding of our mental abilities.

    Link to this
  3. 3. Scienceisnotagenda 3:57 pm 07/15/2013

    Science is science. This article like many on human intelligence is a mishmash of facts bundled in political correctness.

    Human intelligence is a product of evolution. Not magical. It is chemistry…governed by the properties of matter and energy…like everything else in the Universe.

    Psychology is part science, part agenda. It just doesn’t make the grade as a full science.

    As a geologist I don’t justify my findings beyond the evidence to be acceptable to other geologists or the public. Psychologists need to stop looking over their shoulders and just do the nitty gritty research.

    Link to this
  4. 4. Scienceisnotagenda 4:01 pm 07/15/2013

    Re Davis above. No, it’s about intelligence, not knowledge. Stuffing more facts into a computer doesn’t increase it’s hard drive. Our intelligence doesn’t increase by learning more facts. Intelligence is the capacity to learn the facts and use them.

    Link to this
  5. 5. FOOZLER8 4:32 pm 07/16/2013

    Traditional IQ scores are correlationally related to more other factors/traits than any other measure we have. Getting rid of it deprives us of our greatest instrument for measuring cognitive skills.

    Do we need to take into account creativity? Absolutely, but so far there are no good tests.

    As Greg Allman says: “Don’t mess up a good thing.”

    My Ph. D. is in psychology

    Link to this
  6. 6. mah3md 6:10 pm 07/17/2013

    Stephen Jay Gould extensively discussed in his book “The Mismeasure of Man” the origins and development of what today we called “the IQ test.” Binet never intended for the test to “measure” intelligence but to detect those school kids that were left behind because of their shortcomings in learning and thus be able to help them. Distortions by others followed, and intelligence was defined by a numbers (the higher ones) and the others, the lower ones tended towards “retardation.” It would be great if we return to Binet’s idea.
    By the way, Gould’s book should be read by anybody interested in how the history of some ideas on the workings of the brain evolved and how ideologies, particularly racist ones, masqueraded as Science and interfered with scientific developments. Gould was (is) one of our greatest treasures of thought.

    Link to this
  7. 7. quincygtg 4:08 pm 07/18/2013

    As a clinical psychologist trained under the “Boulder Model”, which promotes a scientist-practitioner approach, I think human beings are inherently programmed to measure things. It serves a very useful purpose. And standardized measures allow all to have a common understanding of things such as how tall a person is and their weight. (I do wish the US would convert to the metric system since it is the world standard and used so frequently in our everyday lives such as the measure of medication. A similar issue is our use of DSM [Diagnostic Statistical Manual] versus ICD [International Classification of Diseases]. Both create confusion for Americans.) But I digress.
    I feel that IQ is better thought of as a measure of cognitive abilities as does the author. I also agree with the author that individual standardized IQ tests when administered by a trained professional are the “gold” standard. (Mensa has moved away from using some group tests such as the ACT.) In my opinion, the Wecshler tests are the “gold” standard of individual standardized IQ tests. They provide individual cognitive ability measures and a general IQ number. And these are pragmatically useful. They can be used to identify a person’s strengths that will aid the person in pursuing a successful area of education, work and hopefully, life satisfaction.
    As a clinician, I have focused on a person’s strengths when treating behavioral health disorders. Focusing on a person’s strengths maximizes the resources available to an individual to deal with whatever they need to address.
    In summary, IQ is useful when measured and used appropriately.

    Link to this
  8. 8. stuger 3:23 pm 07/22/2013

    I liked your book (Ungifted intelligence redefined) very much. It is a very good and nuanced presentation of the field of intelligence research. However I think psychologists underestimate the impact of cultural stereotypes (you also adress this issue)related with IQ measurements. These sensitive topics (IQ and creativity)in a information age create for the ‘high IQ scoring’ groups achievement peer pressures and for the ‘low IQ scoring’ groups structural multi generational intellectual inferiorty stigma and according behavioral problems. I think that much can be gained by instructing low performing IQ groups to emulate the cultural characteristics of succesfull socioeconomic groups because I think low group IQ scores is a cultural problem. For that matter I think Carol Dweck’s research is the best I have seen.

    Link to this
  9. 9. bucketofsquid 5:25 pm 07/23/2013

    @Scienceisnotagenda – The vast majority of your posts display a particular agenda that is easy to see for even average laymen such as myself. I’m starting to wonder if you are a sociopath. I have a fair number of issues with psychology/psychiatry myself but to discard an area of research just because you don’t like it is silly.

    Also – your analogy is flawed. You should have used a more complex analogy involving RAM, the CPU and the clock speed as well as the hard drive. Other than that I generally agree with your separation of knowledge and intelligence.

    @FOOZLER8 – Traditional IQ scores always put me at the genius to super genius range. Based on the idiotic things I’ve done in my life and the fact that I’m not a rich copyright or patent holder, I’d have to say that you are very wrong and doing a disservice to your patients. The 1950s really were pretty sucky so you may want to move past them and join the modern era.

    Link to this

Add a Comment
You must sign in or register as a ScientificAmerican.com member to submit a comment.

More from Scientific American

Scientific American Back To School

Back to School Sale!

12 Digital Issues + 4 Years of Archive Access just $19.99

Order Now >

X

Email this Article



This function is currently unavailable

X