It's not really news when a journalist goes cherry-picking for juicy tidbits to fit a narrative, is it? We all fall into the trap of going too easy on the things we want to believe.
So what is it about a piece about vacations in Vox that got me rushing to the keyboard?
Science journalist Joseph Stromberg is asking whether vacations increase your creativity in the workplace. Here's how the article starts:
"The evidence is pretty clear that taking a vacation makes people more productive when they get back to work.
But could they be more creative too?
That's the claim of a new study, in which Dutch researchers gave creativity tests to workers before and after they took a trip and found they scored better afterward.
Still, it's a very small study and may not provide an accurate measure of the kind of real-world creativity that yields actual benefits. Let's take a closer look."
Yep, I sure did take a closer look.
The evidence is most definitely not clear that vacations make you more productive. The Vox links take you to news stories. In those, the only research that could be the basis for that specific statement is one vaguely described and unreferenced 2006 internal study of employees at one company. Nor do the authors of the new study, Jessica de Bloom and colleagues, make the claim that they've shown people are more creative after a vacation.
De Bloom's study of 46 people who took a test two weeks before their vacation, and another one week after, didn't find increased creativity. Of two domains of creativity they looked at, one came up with a "statistically significant" result and the other didn't. This could be a real difference. But it could also be a statistical artifact.
The "significant" result was small-ish: it had a Cohen's d test result of 0.32. (Cohen's rule of thumb is that 0.2 is a small effect and 0.5 is a medium one.)
Some of De Bloom's other work on vacations features in a post I wrote here about a year ago summarizing the evidence about vacations. I'm an Australian who moved to Germany about a decade ago - and then to the U.S. 3 years ago. That means I went from decades of having 4 weeks (plus extras) a year, first to a country with much more - and now to one with much less. The differences, and the culture around it, fascinate me. So does the research in this field.
Since my post last year, de Bloom has also reported on the design of a trial she and her colleagues have underway about different ways of spending a lunch break. That's important to watch out for, because what we do day-in, day-out is likely to be more important for our wellbeing than something we do once a year.
Amanda Gilbert and colleagues reported the results of a trial comparing going to a resort, to doing a meditation retreat at the resort. (Warning for those with a low tolerance: that one's the Chopra Center.) However, that's just a small abstract and there's too little information there to make much of a judgment about it. I think last year's post remains a reasonable round-up of what we know. And yes, there are some takeaway messages there.
But let's get back to what vexed me about the Vox piece. It wasn't just that it was a highly biased piece with a patina of science: what Charles Seife calls "proofiness" in his wonderful book of that name (more on that below).
Focusing on work-related benefits of vacations from that work bothers me. Does there have to be a benefit in "real-world creativity" back on the factory floor or at the computer for the research to mean something?
Many people fought long and hard for the entitlements workers have. Many are fighting those battles still. The point of vacations isn't to benefit work. It's to live a fuller life.
Interested in Charles Seife's excellent and entertaining book, Proofiness? Here's a review by mathematician John Allen Paulos in The Washington Post. And one by another mathematician: Steven Strogatz in The New York Times Sunday Book Review.
Interested in reading more about the science of evaluating bodies of evidence? Here's a post from me on 5 key things to know about meta-analysis.
The cartoon is my own. (There are more of my cartoons at Statistically Funny.)
The 8-hour day banner from Melbourne in 1856 is by an unknown artist, via Wikimedia Commons.
(And with thanks to Deborah Mayo for pointing out an error in the original - updated shortly after posting.)
* The thoughts Hilda Bastian expresses here at Absolutely Maybe are personal, and do not necessarily reflect the views of the National Institutes of Health or the U.S. Department of Health and Human Services.