July 16, 2011 | 17
Have you heard about the highly endangered tree octopus of the forests of the Pacific Northwest? "Ridiculous," you say? But I found a whole webpage devoted to saving them, so they must exist, right? This website has been used to intentionally mislead students (see news article) as part of a scientific study, and it points out a real problem our society faces in the digital information age: anyone can put anything on the internet.[Ed. this sentence edited for clarification]
The internet empowers us to educate ourselves and make more informed choices and decisions without leaving our couches. But if we believe everything we find on the internet, we are likely to wind up making some very poor decisions. In this new digital information age, how do we keep from being misinformed? As a skeptical environmental research scientist and educator I have picked up a few tricks that anyone can use to find and select high-quality information from the internet.
#1 Don’t be scared of scientific papers
Aren’t scientific papers too hard to read?
While scientific papers may be filled with difficult to understand jargon, anyone can scan them and get the gist (pdf); just skip the confusing words or look them up. If you are still confused and it is important that you understand completely, ask for help from someone trained in the field of interest. For instance, if it is a medical paper, you might take the article to a physician, but not all doctors have the necessary level of scientific literacy to understand the article and you many need a second opinion.
You might also contact a medical researcher you know or are able to find using the internet and ask for their help in understanding what the article means. You could also take advantage of the power of social media and ask questions on twitter, adding hashtags for science or medical topics. You can even find and follow individual scientists in particular fields of interest on twitter using sciencepond.com.
Additionally, certain websites may allow you to ask questions directly of health professionals or scientists. Alternatively, you may be able to email the lead author of the study directly. Their address is usually listed on the publication and they are often thrilled to have someone interested in their research.
Why use scientific papers?
Scientific papers are the best source of information on the internet. These papers use rigorous experimental and statistical methods that improve validity of the evidence presented within them. For instance, scientific studies often allow the clear identification of cause and effect because only one factor is different at a time, so that the effect of that single factor can be determined. Additionally, these studies often repeat the tests many times using randomization of specific aspects of the experimental design, which helps reduce the likelihood that the results were due to chance or unknown factors.
Scientific papers are also peer-reviewed. Peer-review is a process by which other scientists in the field are asked to anonymously and critically evaluate the research presented within a paper. The editor ultimately makes the decision on whether or not to publish the articles, based on feedback from reviewers and his or her own critique.
This usually ensures that the information published in these papers has been obtained through rigorous, high-quality techniques and that the authors do not make overly speculative conclusions that do not match the data. However, this is not always true. With the proliferation of peer-reviewed journals, it may be that erroneous or poorly conducted research sometimes gets published. Even high profile journals sometimes make mistakes, but with a few techniques, you can make sure these publications don’t trick you.
How do I find appropriate scientific papers?
Google scholar allows anyone to search directly for scientific papers. This is an incredibly powerful tool. When I am trying to find information on a topic that is really important, I use several simple tricks to ensure that I am not misled by a single erroneous publication.
First, I add the word "review" to my search. Review articles generally consider most, if not all studies published on a topic and summarize them, often weighing the trade-offs between these studies, and giving the opinion of the authors about the relative weight of the evidence.
Second, if no recent review exists, I scan the top 10-50 articles and make a list, conducting my own mini-review on the topic. How do I read so many studies you ask? I don’t. I read the titles and abstracts first. Abstracts are a brief, simplified summary of the information contained within the article. Scanning these abstracts is a handy way to quickly examine a large collection of papers for their key points.
Another trick I use to obtain the best information is to obtain the most recent information. Google scholar gives you an option to narrow your search to a range of years. A drop down box allows you to specify how far back you want to search for articles. If you narrow your search to the last 1, 2, or 5 years, you are likely to get the most recent papers, which often will present the best summaries of the current understanding of the topic.
In general, abstracts of papers are available to everyone. This is not true for the full papers. Some journals or specific papers will be "open-access" and available to everyone, and these are becoming more common, but many papers are still not publicly available. Google scholar will give you links to the papers that are available in-full, online, but some will be unavailable with this tool. However, there are many other methods of obtaining these papers. For instance, you can ask for the paper on twitter using the hashtag #icanhazpdf, ask for it on friendfeed, email the author (all three of these methods ultimately result in one person e-mailing the paper to one other person, thus not violating copyright) of the paper asking them for it respectfully, or go to the nearest university library.
#2 Not all websites are created equal
Can I find good information on regular websites?
Using websites is like talking to friends. While we all have friends that are truly knowledgeable and can be very helpful, we all also have friends who are often wrong, but never in doubt. Some of our friends may tell us things that have mixtures of truth and fiction at the same time. Most people know to be skeptical of what some of their friends tell them and we should hold websites to the same standards. However, there are a few guidelines that can help you to separate the informative from the misleading.
Commercial websites that are trying to sell a product cannot be trusted to tell you the truth about issues related to that product. You may be able to gain understanding of a product and be able to compare products using these sites, but any claims should be highly suspect. Never forget they are trying to sell you something. Selling you something is the website’s goal, not educating you.
Websites sponsored by particular commercial, political, or other entities should be viewed with suspicion as well. Like commercial websites, these sites also have a goal of "selling" you something, it just may be a particular belief rather than a physical product. Check the "about us" page to see who sponsors the website. You are more likely to get high-quality information from websites affiliated with universities, government agencies (but not politicians), and some news organizations or non-profit associations devoted to public education.
Beyond these categories, we enter realm of news and magazine websites, blogs, and Wikipedia. All of these reporting platforms are valuable and can be trustworthy. Many respectable news and magazine websites now employ bloggers, hired because they are good journalists, with editors in some cases reviewing blog posts prior to publication. So blogs may be of equal or higher quality than many paper publications.
However, there is more variability among unaffiliated individual bloggers. Commonly, no one reviews the material in these blogs prior to posting. However, readers may comment on their posts and these comments can be informative. Among science blogs, individuals and organizations who regularly produce high-quality work are now being recognized and included in science blog aggregators like ResearchBlogging.org, Scienceblogging.org, and ScienceSeeker.org. Many of these blogs can be regarded as trustworthy sources of information and are often better than the coverage by the main stream media (see this post).
Wikipedia can also be very valuable. Although Wikipedia can be edited by anyone, it has a large group of active moderators/editors who remove erroneous information and protect some controversial websites from being altered too often or rapidly. Additionally, Wikipedia now flags pages that have fewer citations or might have poorer quality information with warning messages at the top of the page. Multiple approaches work together to ensure the overall accuracy of Wikipedia pages. Even before many of these policies were implemented, Wikipedia was found to be relatively equal in quality to Encyclopedia Britannica (see this study (pdf)). Thus, in its current form, Wikipedia is a fairly trustworthy source of information. Some individual pages may still need caution, but for these, you will get a warning right at the top.
#3 Checking the facts
How do I know if a particular piece of information is valid?
In general, webpages with more citations and links, or from more unbiased, reputable organizations, will provide you with better information. However, it is also a good idea to try spot-checking some of what is on the more suspicious websites, like commercial sites or some individual blogs, with other trusted webpages, or by using Google scholar or a fact-checking website.
Fact checking websites claim to deliver unbiased evaluations of recent internet rumors or statements by politicians. A couple of my favorites are factcheck.org and snopes.com. These sites are a great way to check whether that email claiming to be donating to a sick child or offering strange advice is a fake. You can also ask experts on twitter or other social media websites your questions. The more different sources you have that all say the same thing, the more confident you can be of the validity of that information.
Note: many thanks to Bora Zivkovic for his helpful suggestions.
About the Author: Kevin McCluney is a postdoctoral research associate in the School of Life Sciences at Arizona State University, also receiving his PhD in biology from this institution in 2010. McCluney conducts research on rivers and streamside animal communities and how water structures these communities. Through a series of peer-reviewed publications, he has shown that many food webs are really water webs, with animals essentially drinking their food rather than eating it. McCluney has also taught college level classes for seven years and mentored students from 6th grade through the graduate level in their efforts to conduct independent research projects, with students winning many local and international awards and publishing peer-reviewed papers.
The views expressed are those of the author and are not necessarily those of Scientific American.