Skip to main content

Never Perfect: Making Research Decisions about Close Enough for Now

Imagine you have been plopped in the middle of a desert and asked to make a geologic map of the surrounding square mile over the next seven hours. What would you do? How would you start? Making decisions in this kind of setting depends on interpreting your surroundings enough to create working hypotheses that will help you decide where to go, when and what to look for once you get there

One approach for getting a handle on a new field location is to climb up to a higher point and get a sense of things from above. 

This article was published in Scientific American’s former blog network and reflects the views of the author, not necessarily those of Scientific American


Imagine you have been plopped in the middle of a desert and asked to make a geologic map of the surrounding square mile over the next seven hours. What would you do? How would you start? Making decisions in this kind of setting depends on interpreting your surroundings enough to create working hypotheses that will help you decide where to go, when, and what to look for once you get there.

Each scientific field has its own set of skills that is specific to its discipline. When learning to conduct scientific research, most people start out with critical analysis and defining research questions. But eventually the skills that need to develop become a little less traditional: instantly recognizing bird calls, fixing scientific instruments mid-flight, or even hiking for eight hours per day. These skills and experiences combine into more than just a toolbox for researchers, but into a way of thinking and approaching problems.

In the world of field geology, there has been a recent push to start studying the way that geoscientists process and approach problems in the field – a process called geocognition. Common geologist adages suggest that these skills are simply built through experience, with phrases floating around like, “The best geologist is the one who has seen the most rocks.” But in addition to exposure and experience, something else must be gained in this process that makes experienced field geologists better able to ask the right questions, get to the right places, and make the best possible use of their time. By studying the movements and motivations of a range of people – from students to experts – researchers are starting to identify some of the skills and approaches that make up this particular approach to the natural landscape.


On supporting science journalism

If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.


When trying to categorize the type of cognitive tasks that geologists face in the field, researchers have compared them to other people tasked with Naturalistic Decision Making (NDM) – situations in which decisions must be made or problems solved iteratively in data-poor situations with limited time. Some of the other fields where NDM has been studied include firefighting, military command, and air traffic control. These may not be the skills that one traditionally thinks of as part of science, but they are certainly relevant.

None

Examples of walking paths and resultant maps from one novice and one expert (Source: Petcovic, Libarkin, & Baker 2009)

The preliminary geocognition studies vary somewhat in their approaches, each trying to determine what kinds of data will provide insight into the decision-making processes of geologists in practice. In one study, a series of students were given just the task from our opening paragraph – seven hours to produce a geologic map of ~2km2. Their motions were tracked via GPS and they were given scores based on the overall accuracy of their maps, whether their path took them to areas that contained key pieces of data, and the efficiency with which they moved through the region. Another study involved both students and experts, and included an additional Think-Aloud audio log for some of the participants as they mapped a smaller area in Michigan.

While the findings in these studies are preliminary and merit further investigation, there are already some interesting points worth noting.  

The less experienced mappers tended to move through space much less efficiently – doubling back over areas or being forced to take a more rapid pace near the end. Less experienced mappers could also miss critical areas, or fail to understand the significance of data in those locations. Even if these mappers did find and correctly measure important structures, they were less likely to place it in the right location on the map.

Some of these results can be attributed to lack of familiarity with rock types, geologic structures, and even lack of experience with relating a real landscape to a map representation. However, the researchers identified another underlying difference that can also affect each of these points: whether or not the mapper had a working hypothesis about the overall context of the region.

When more experienced mappers would think aloud or explain their maps, there was often a consideration of what they thought could potentially be going on beneath the surface or what overall patterns they were seeing in the structures of the area. As they took measurements and encountered new structures, they would acknowledge that some hypotheses became stronger and others less likely. As information accumulated into this overall mental model, future decisions and problems were faced according to the most likely scenario.

None

Researchers captured where mappers went, how quickly they got there, and whether they passed through certain areas more than once (Source: Riggs, Lieder & Balliet 2009)

In the absence of such hypotheses, the paths and measurements were more arbitrary and more likely to be consolidated for potential interpretation at the end.

Though these geocognition studies serve as an example of what science-based Naturalistic Decision Making may look like in a short window, it seems reasonable that such decision making would be prevalent throughout the world of science. When instruments fail, natural disasters strike, diseases are diagnosed, or satellites only get a single pass at a distant planet, decisions have to be made. There will never be the chance for all of the desired information to be available for an ideal decision, but the ability to integrate all of the available information into a meaningful set of models makes it possible to at least make a decision you can stand by.

References:

Heather L. Petcovic, Julie C. Libarkin, and Kathleen M. Baker (2009) An Empirical Methodology for Investigating Geocognition in the Field. Journal of Geoscience Education: September 2009, Vol. 57, No. 4, pp. 316-328.

Eric M. Riggs, Christopher C. Lieder, and Russell Balliet (2009) Geologic Problem Solving in the Field: Analysis of Field Navigation and Mapping by Advanced Undergraduates. Journal of Geoscience Education: January 2009, Vol. 57, No. 1, pp. 48-63.

Amanda Baker is a science communicator and outreach advocate. She has a geoscience PhD from Cornell University and has managed open-access, academic journals as well as the outreach journal Frontiers for Young Minds. She is currently writing and editing science content for kids, from curriculum materials to magazines like Smore. She has served as a Science Olympiad national event supervisor and taught a first-year writing seminar on sustainable earth systems while at Cornell.

More by Amanda Baker