Today’s workforce is in need of workers who are capable of strategic information gathering, flexible thinking, and adaptive approaches to problem-solving (Partnership for 21st Century Skills 2009)—all hallmark aspects of critical thinking. But how can we effectively target and build these complex thinking and reasoning skills in the classroom? Moreover, how can we help students recognize, document, and use processes of critical thinking as they solve authentic scientific problems?

An important first step is to define more precisely what we mean by “critical thinking.” Critical thinking in science education is characterized by a broad set of cognitive processes that require “reflection on and evaluation of available evidence” (Wentzel 2014, p. 579). As such, there are at least seven composite skills involved in critical thinking across multiple disciplines: observing, problem-finding, connecting, flexible thinking, comparing and contrasting, evaluating, and interpreting (Butcher, Hudson, and Runburg 2018; Butcher, Runburg, and Altizer 2017). Breaking down critical thinking into these observable skills serves as the foundation for structuring phenomena-centered activities that require critical thinking.

In years past, our classroom experiences with middle school students demonstrated that students participated in rich collaborative discussions that reflected many processes of critical thinking. However, the multitude of evidence and ideas that students generated during collaborative discussion was not documented in a manner that allowed students to reflect on their evidence in rigorous and systematic ways. In essence, a great deal of critical-thinking products generated by students (including detailed observations, comparison across different sources of evidence, and alternative hypotheses) were lost to the moment, creating significant difficulties when it came time for students to reflect on their findings and develop detailed, evidence-based arguments that indicated the progress of their thinking and a clear rationale for their conclusions. Instead, students tended to seize upon a single idea early in their investigation and evaluated selected evidence against their chosen idea rather than keeping multiple ideas active and choosing the idea that was best supported by available evidence. This approach is consistent with the well-established psychological phenomenon of confirmation bias, where individuals seek out evidence or interpret existing evidence in ways that are biased to support existing beliefs or expectations (Nickerson 1998; Wason and Johnson-Laird 1972).

Structured documentation helps expose student thinking and helps students manage the complexities and demands of critical thinking, allowing them to engage more fully in the full range of critical-thinking skills expressed in their curriculum standards and encompassed by the science and engineering practices of the Next Generation Science Standards (NGSS). The term structured documentation has been used in technical writing (Murphy 2000), medicine (Joyner et al. 2017), and computer programming (Holt, Boehm-Davis, and Schultz 1989) to refer to any documentation approach where the format and/or organization of documentation is prescribed, but the content within the documentation varies by individual case. In the current work, we use structured documentation to help students record and organize the products of their critical- thinking processes. Without structured support for documenting evidence in ways that support reflection and analysis of multiple possibilities, we have found students tend to rely upon vague rationale and references to global similarities when developing their arguments (see Figure 1). As seen in Figure 1, students rarely articulated the factual details driving their arguments and often failed to weigh the strength of confirming and disconfirming evidence, providing arguments such as “because we have three things that tell us what dinosaur it is” and “because most all of the stuff fits” (thereby discounting the stuff that does not). These examples are consistent with our observations that students had difficulty understanding the distinction between strong, weak, and disconfirming evidence and how these varied forms of evidence should be combined to form a strong, evidence-based argument.

We have developed a three-phased approach to help students document their critical thinking and create a visible record of their work for analysis and reflection. This three-phased approach helps move students from more intuitive skills in critical thinking (observing, connecting, and problem-finding) to more difficult and complex critical-thinking skills (particularly: flexible thinking, evaluation using evidence, and interpretation). In phase 1, students gather evidence and generate multiple ideas, questions, and hypotheses. In phase 2, students move on to evaluate evidence systematically across multiple ideas or hypotheses. In phase 3, students interpret their findings and communicate an evidence-based argument.

The context for our work is best described through our use of free online, object-based investigations called “Research Quest” (see Resources). Research Quest investigations were developed by the Natural History Museum of Utah (NHMU) to engage students in critical thinking using authentic scientific questions and digitized versions of real collections objects. Research Quest currently hosts three investigations situated in Utah’s world-renowned dig site: the Cleveland–Lloyd Dinosaur Quarry (CLDQ). In this article, we use the first of these investigations to illustrate the three-phased approach to support effective and reflective critical thinking through structured documentation. Our materials can be easily adapted to other phenomena-centered lessons that require reflection or analysis of evidence to derive well-supported conclusions.

In CLDQ investigation 1, students work in collaborative pairs to explore a fundamental question for scientists at any dig site where fossils are found: “What dinosaur did these bones come from?” Each student completes their own documentation of the investigation using a provided Research Assistant Notebook. During the investigation, students use high-quality, 3-D scans of dinosaur fossils from the museum’s collections to develop an evidence-based argument for the order or suborder and species of three mystery fossils at the heart of this investigation. Students first determine the type of bones represented by the mystery fossils, then identify the suborder (e.g., theropod) and species (e.g., Allosaurus fragilis) of dinosaur by comparing the mystery fossils to known fossils in the NHMU collection. Across this investigation, students engage in the full range of critical-thinking skills as they gather evidence, develop conclusions, and prepare their oral argument (see Figure 2).

In this CLDQ investigation, some observations about the mystery claw (its strong curve and sharp point) serve as strong evidence that the dinosaur comes from one branch of the dinosaur family tree (theropods) and disconfirming evidence for other branches (e.g., ornithischian). However, the same observations about the mystery claw serve only as weak evidence for the species of dinosaur (all theropod species have sharp, curved claws). Detailed observations about the mystery jaw fossil (its shape and unique curve) provide the strongest evidence for determining the particular species of theropod (Allosaurus fragilis). Our approach provides scaffolding for students to document and analyze their observations in ways that help them understand how and when patterns of evidence support valid conclusions.

Recording observations is an important place to start students in a phenomena-centered activity, as the concrete process of gathering empirical evidence is the entry point to most critical thinking. Observations may be recorded free-form (including drawings for students or English language learners who may find it helpful to visualize details), to encourage rich generation, varied thinking, and unencumbered note-taking (Figure 3). Regardless of format, students should be encouraged to record as much specific detail as possible during their collaborative work, as this documentation serves as the basis for future steps of critical thinking. Students who do not take detailed notes about their observations will have limited evidence for reflection and analysis in phase 2. Other than expert video models in Research Quest (where a paleontologist models her thinking as she develops detailed observations for a fossil) and a text prompt to encourage detailed notes, we do not use additional structure or scaffolding in phase 1 documentation materials to encourage as many student ideas and observations as possible. As students make their observations, they naturally connect their observations to prior knowledge (“This part could be the root of the tooth” or “A predator would have a sharp claw!”). They also will engage in problem-finding—suggesting ideas and hypotheses related to the investigation that can be evaluated (e.g., “I think it’s a claw” or “Maybe it’s a tooth”).

Flexible thinking (considering multiple possibilities until sufficient evidence is gathered) is a hallmark process of critical thinking, but one that is especially difficult for students. Like the rest of us, students have intuitions and they generally want to be right. Thus, their natural tendency is to choose a single idea early during observation (phase 1) and find confirmatory evidence for that idea (often ignoring disconfirming evidence and failing to consider that confirmatory evidence may also support other ideas). Getting students to systematically evaluate evidence across multiple possibilities and to avoid committing to a single idea before this process is complete can be difficult. However, we have had good success with using visually-organized documentation to scaffold students’ thinking (see Figure 4). Multiple ideas are listed in columns, and detailed observations (evidence) are listed in rows. As students discuss the evidence they have collected, they mark individual cells in the table to show whether the evidence fits the idea (a checkmark) or whether the evidence does not fit the idea (an “X”).

A key benefit of visually organized documentation for critical thinking is that it not only helps students engage in flexible thinking, evaluation, and comparison and contrast, but it also provides students with a clear, visual way to recognize and understand different types of evidence. Disconfirming evidence is represented in cells with an “X”—this evidence does not fit the idea and can be used to argue against it. Confirming evidence can be either weak (supporting multiple ideas) or strong (supporting one idea). Students look across the cells in a row to determine the strength of evidence. Rows with multiple checkmarks represent weak evidence because they fit multiple ideas. Rows with a single checkmark represent strong evidence because they fit one idea. By examining the pattern of evidence they have collected, students systematically determine which idea is best supported by evidence and which ideas should be ruled out. At this point, some students will be surprised by their results—like any of us, if they find their favorite idea is being ruled out, they often return to check the gathered evidence for accuracy and completeness (an important process for any scientist!).

As students become accustomed to evaluating the quality of detailed evidence, they begin to realize that not all evidence is created equal. In this investigation, the claw (mystery fossil 1) is strong evidence to narrow down the order or suborder of dinosaur (ornithischian, sauropod, theropod), but the femur (mystery fossil 2) provides only weak evidence—ornithischians, sauropods, and theropods of dinosaurs have similarly shaped femurs. This is reflected in the patterns of evidence that students document for mystery fossil 1 (Figure 4a) and mystery fossil 2 (Figure 4b). Identifying less useful forms of evidence during an investigation helps prevent floundering and supports students in developing stronger arguments.

As you develop evidence tables to structure students’ critical thinking in this phase, you can choose how much information is generated by students. We started by providing students with the different orders and suborders of dinosaurs to which they should compare their own observed evidence (drawn from the free-form notes) to ensure flexible thinking at this stage of the investigation (Figure 4). After the order or suborder of dinosaur is chosen, students list their own set of multiple species for comparison (Figure 5). This serves two purposes: (1) students gradually take on more of the documentation task, and (2) it does not presume that students will choose a specific order or suborder of dinosaur during the course of the investigation. Students can (and do) come up with different conclusions (arguing for a different order or suborder and species of dinosaur), leading to vigorous classroom debates based on the quality and sufficiency of evidence for the different conclusions.

After phase 2, students are ready to interpret findings and communicate their conclusions to others. Thus, the final phase of support for critical thinking in phenomena-centered investigations is to help students organize their evidence in a way that allows them to communicate the results of their critical thinking. Phase 3 documentation should support students in discussing how the combination of strong, weak, and disconfirming evidence they gathered justifies their conclusion (helping students avoid the natural tendency to list confirmatory evidence only, without regard for its strength). By building upon phase 2 documentation, students build an evidence-based argument focused on specific details and patterns of evidence. This has helped students we work with move past vaguely articulated “matching” arguments (Figure 1) to detailed arguments that use varied forms of evidence to justify a conclusion and encourage meaningful scientific debate (Figure 6). Students benefit from seeing other pairs of students arrive at the same conclusion with different evidence or arrive at another conclusion using the same evidence, leading to important discussions about the sufficiency of evidence and the accuracy of observations. This directly supports NGSS science and engineering practice 7: Engaging in argument from evidence. Students present their arguments to classmates, detailing the strong, weak, and disconfirming evidence that led to their conclusions and responding to questions and challenges from other groups who may have emphasized different evidence or interpreted similar evidence in different ways. Students respectfully critique the sufficiency of other groups’ evidence and debate whether inferences are justified. When students use similar evidence to develop different arguments, the full class has the opportunity to compare and critique the ways that evidence was recorded, weighed, and interpreted and to determine whether each argument is equally valid given relevant observations and details.

When using structured documentation to support critical thinking, teachers should: Align student work in an investigation to the component skills of critical thinking. While students may move iteratively through the skills, the activity should ensure that students advance from intuitive critical-thinking skills (e.g., observing, problem-finding) to more complex critical-thinking skills (flexible thinking, evaluation, comparing and contrasting). Documentation should be structured so that students can track, reflect, and analyze the progress of their thinking. For phase 1, emphasize detailed documentation of varied evidence. In phase 2, use structured documentation (such as evidence tables) to ensure students engage in flexible thinking, systematic evaluation, and comparison and contrast. In phase 3, use structures that help students organize their analysis from phase 2 to make an evidence-based argument that can be clearly communicated to others. Ensure that documentation does not constrain students to correct answers. Incorrect conclusions and different explanations can lead to meaningful debates about the quality and adequacy of evidence—a superb activity for engaged critical thinkers! Over time, require students to generate more components of the documentation and even the documentation structure itself.

Although all three phases of documentation are an effective way to help students leverage critical-thinking discussions during phenomena-centered investigations, we believe the visually organized evidence tables are a critical part of this process. Evidence tables are valuable to students because they create a clear bridge between students’ observations and arguments. They support student engagement in more complex forms of critical thinking and serve as a visual record of students’ critical-thinking processes that is then available for further analysis and reflection. They also serve as the key foundation for resolving disagreements, allowing students to go back to pinpoint where they used evidence in different ways and to reconsider evidence to resolve conflicting views. Evidence tables in particular—and structured documentation in general—give students an important tool to capture their process and articulate their ideas, providing them a firm foundation for effective critical thinking.

Teachers can register for a free Research Quest account on www.researchquest.org. The “Teacher Support” tab provides PDF downloads for all supplemental materials, including instructional guides, assessment tools, teacher and student rubrics, and the student Research Assistant Notebook. All documentation materials (including evidence tables) are provided in the Research Assistant Notebook.