This article was published in Scientific American’s former blog network and reflects the views of the author, not necessarily those of Scientific American
In the late 19th century, the English social reformer Charles Booth went door-to-door with a small group of researchers asking London residents questions about their socioeconomic status. He published his findings in a series of maps that used an eight-point color scale to illustrate wealth on a street-by-street level. A few years later, Parliament passed the Old-Age Pensions Act, which provided financial aid for people over the age of 70. The new laws paved the way for modern social welfare in the United Kingdom.
Although collecting data on populations predates the 19th century—the concept of a census is a few thousand years old—Booth played a special role by recording social and economic data of a population and using the findings to influence legislation. Subsequently, surveys became one of the most important features of the modern era. They were the first scientific tool we used to study ourselves in groups, and our new self-knowledge has paid off enormously. The social progress, economic growth, improvements in education and advances in health of the last 150 years were made possible, at least in part, by survey work. Progress became a lot easier once we could measure the things we were trying to change.
Over the past few years, I’ve conducted about 300 market research surveys while working in a creative advertising agency. As I was reading about Charles Booth it occurred to me that there are two types of survey questions: those that extract and those that evoke. Questions that extract involve collecting information; this is Booth on the street asking people about their income. Questions that evoke involve eliciting a reaction. Think about a focus group moderator versus a standup comedian. Both care about what you think. But the stand-up comedian goes one step further to provoke a response.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
Questions that extract represent the vast majority of survey questions. But in the market research world, questions that evoke are more valuable. Brands revolve around consumer insights akin to comedic bits—observations that make you grin, nod your head and say: “I hadn’t thought about it that way.” In an era in which data are seen as key to understanding people, brands still crave deeper emotional responses. Not data per se but a firm grip on emotional levers that, once pulled, generate a perspective-alerting response, the kind of response people have when they’re exposed to a story or piece art that changes them—permanently.
In the decades after Booth published his findings, academic psychologists began measuring attitude—Rensis Likert proposed his famous five-point scale in 1932—and businesses started employing full-time researchers. Sadly, market research cut itself off from the creativity it should have achieved by following the formal thinking of Booth and Likert rather than the informal but socially attentive judgment of writers and artists of the day. Questions that extract are important—Booth was the first person to accurately calculate the poverty rate in London—but they became a norm that few people in the field thought to challenge.
Making it a goal to elicit a response changes how you draft survey questions. Instead of thinking about what information you want to collect, you’re forced to think about the origins of your own impressions. The idea is to interrogate perception and use survey questions to surface a new observation, much like a comedian uses a story to set up a joke.
For example, a few months ago, the managers of a toilet paper brand were interested in learning more about what people thought about their product. I considered what might happen if my company switched toilet paper brands. I had a neutral view of toilet paper but noticed that I was disappointed about the hypothetical change. As the scenario played out in my mind, I wanted the old brand back.
Consider these two questions.
“How likely are you to buy toilet paper brand X in the future?”
“How much would you be willing to pay for toilet paper brand X?”
These are questions that extract. Now, consider this question.
“If your office switched toilet paper brands and began using [insert brand] in the bathrooms, how excited would you be?”
This is a question that evokes. If you filled in “toilet paper brand X” with an expensive or cheap brand, you’d react. And by swapping in different brands and measuring how people respond, you can see how people perceive each brand relative to one another. Questions that extract still matter—asking people in a separate survey what toilet paper brand they would buy for their home bathroom could reveal a difference between stated versus revealed preference—but they play a supporting role in the research process.
We ended up conducting a survey (n=1,200) with this question. We found something the client had suspected but had struggled to validate. Although their product was seen as a little pricey, people still preferred it. In fact, it was the brand that people were most excited to have in the office. With a few edits to the survey question, we recommended a new tagline that inspired the subsequent creative work. “The toilet paper brand everyone wants in their office bathroom.”
Thinking about toilet paper in terms of questions that evoke turned a relatively mundane item into an object of interest. Toilet paper, it turns out, carries cultural weight. It works on us as travelers and consumers and not just homeowners. “Don’t bias the respondent” is perfectly responsible advice for questions that extract. But for questions that evoke, contemplating how the wording of a question may influence respondents is the last thing you want to think about. Introspection and curiosity need to guide the research process, not rigid questionnaire principles.
The advertising legend David Ogilvy once complained that advertising agencies “use research as a drunkard uses lamppost—not for illumination but for support.” It’s a great metaphor because it’s true. You think you’re uncovering new perspectives, but, actually, you’re just stuck in the old one. Questions that evoke, in this sense, illuminate. They shed light on the details of daily life that we notice and feel but don’t consciously acknowledge. Until we’re prompted to.