Skip to main content

When We’re Wrong, It’s Our Responsibility as Scientists to Say So

We tried to reproduce our 2012 paper on how to make people report their income more honestly—and we ended up refuting it

This article was published in Scientific American’s former blog network and reflects the views of the author, not necessarily those of Scientific American


Tax season is upon us. Although 95 percent of Americans believe it is their civic duty to pay their fair share of taxes, each year, billions of dollars are lost to fraudulent tax reporting. In fact, according to the IRS, during 2008–2010 (the most recent data), $387 billion was lost each year to underreporting, with individual taxpayers accounting for $264 billion of this. What simple, costless interventions can we use to try to reduce tax fraud?

As behavioral scientists, we tried to answer this question using what we already know from psychology: People want to see themselves as good. So we thought that by reminding people of being truthful before reporting their income, they would be more honest. Building on this idea, in 2012, we came up with a seemingly costless simple intervention: Get people to sign a tax or insurance audit form before they reported critical information (versus after, the common business practice).

We ran studies showing that when people signed an honesty declaration before reporting information, they thought about how they were honest people, and were less likely to misreport compared to when they signed after they had filled out the form. While our original set of studies found that this intervention worked in the lab and in one field experiment, we no longer believe that signing before versus after is a simple costless fix.


On supporting science journalism

If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.


In one of our original studies, we brought 101 participants into the lab and asked them to solve 20 math problems. For every problem they scored correctly, they would earn $1 (subject to a 20 percent tax). Participants were then asked to fill out a tax form reporting their earnings. Some people had to report their earnings after reading and signing a veracity statement (“I declare that I carefully examined this return and that to the best of my knowledge and belief it is correct and complete”); others did so before reading and signing this veracity statement. In this original study, people who were asked to read and sign the statement at the top (versus at the bottom) of the tax form were 53 percent less likely to cheat.

We later replicated this effect a couple of more times (one of these replications was a field experiment with automobile insurance customers) and published our results in the Proceedings of the National Academy of Science, a widely read, interdisciplinary journal.

Seven years and hundreds of citations and media mentions later, we want to update the record. Based on research we recently conducted—with a larger number of people—we found abundant evidence that signing a veracity statement at the beginning of a form does not increase honesty compared to signing at the end.

Why are we updating the record? In an attempt to replicate and extend our original findings, three people on our team (Kristal, Whillans and Bazerman) found no evidence for the observed effects across five studies with 4,559 participants. We brought the original team together and reran an identical lab experiment from the original paper (Experiment 1). The only thing we changed was the sample size: we had 20 times more participants per condition. And we found no difference in the amount of cheating between signing at the top of the form and signing at the bottom.

In light of these findings, we reanalyzed the field study in the original paper and became concerned with a failure of random assignment (such that the number of miles driven before the intervention was delivered was significantly different between the two groups). What we originally thought to be a reporting difference (between customers who signed at the top versus bottom of the form) now seems more likely to be a difference in actual driving behavior—not the honest or dishonest reporting of it. Therefore, we have further lost confidence in the conclusions made in the original paper.

This matters because governments worldwide have spent considerable money and time trying to put this intervention into practice with limited success. In particular, failed attempts have been reported in several countries, and thousands of dollars were spent when one of us (Whillans) was working with a government in Canada to attempt to change their tax forms.

It is important to note that our research looked at whether the location of an honesty prompt and signature made a difference in honest reporting (based on our latest findings, it doesn’t). However, our research is not about the effectiveness of honesty prompts and signatures more broadly. Also, there may be more promising psychological interventions that could increase honesty, like making public commitments, which researchers should continue to explore. In light of the examples of failures in the field and this rigorous direct replication, we recommend that policymakers and industry leaders stop pursuing the specific intervention of moving signing to the top of a form.

We also hope that this collaboration serves as a positive example, whereby upon learning that something they had been promoting for nearly a decade may not be true, the original authors confronted the issue directly by running new and more rigorous studies, and the original journal was open to publishing a new peer-reviewed article documenting the correction.

Moreover, the fact that this prestigious journal accepted this new publication to update the record is important for scientific progress. We believe that incentives need to continue to change in research, such that researchers are able to publish what they find and that the rigor and usefulness of their results, not their sensationalism, is what is rewarded.

While we may have lost confidence in the effect of signing first (versus last) being the simple fix that we thought we found, this collaboration has strengthened our belief that psychology, like all good science, is continuously updating and self-correcting and that it is up to all of us to maintain this growing positive trend.