Students arrive in science classrooms with a range of ideas based on prior knowledge and experiences (NRC, 2000). These ideas or preconceptions are the basis of students’ mental models, which are conceptual models about how the world around them works (Norman, 1983). Conceptual change theorists disagree about the importance of extinguishing or distinguishing novice ideas (Linn, 2008). For this study, conceptual change is identified as an individual’s trajectory away from novice-like ways of conceptualizing a given phenomenon toward more expert-like ways.

The phenomenon of groundwater is critical in geoscience education. Yet, there is evidence that students across age groups and regions hold persistent novice conceptions about groundwater (Arthurs & Elwonger, 2018). For some time, it was opined these ideas could be used as instructional tools (Bar, 1989; Meyer, 1987) but it was not until recently that education research illustrated how this could be accomplished. The present study is part of a larger project undertaken in response to the question of how students’ conceptions about groundwater could be used as instructional tools. While a previous study describes how these conceptions can be incorporated into classroom instruction as tools for learning about groundwater (Arthurs, 2019), the present study examines the impact of soliciting student conceptions and actively incorporating them into classroom instruction. The driving research question is: How effective is a preconceptions-based instructional sequence at facilitating conceptual change of students’ mental models about groundwater?

Theoretical framework

The knowledge integration perspective of conceptual change (Linn, 2008) provides the theoretical framework for this study. This perspective brings together cognitive, temporal, and social considerations underpinning a fuller understanding of conceptual change. It argues that several practices promote student learning and should be incorporated into classroom instruction. They include (1) using personally relevant problems, (2) making individual student thinking visible; (3) enabling students to learn from one another by sharing, discussing, and evaluating one another’s ideas; and (4) providing students with opportunities to reflect on and monitor their performance.

Methodology

Design study

The goal of research conducted using the design study methodology is to “use the close study of [teaching and learning] in naturalistic contexts, to develop new theories, artifacts, and practices that can be generalized to other schools and classrooms” (Barab, 2012, p. 153). According to Confrey (2012), such research is an “investigation of educational interactions provoked by use of a carefully sequenced and typically novel set of designed curricular tasks studying how some conceptual field, or set of proficiencies and interests, are learned through interactions among learners with guidance” (p. 135–136). Furthermore, student work, classroom assessments, and instructional records are used to document “prior knowledge the students bring to the task, how students and teachers interact, how records and inscriptions are created, how conceptions emerge and change, what resources are used and how teaching is accomplished over the course of instruction” (p. 136).

These studies are like case studies because they examine a single bounded case of complex interactions in detail over extended periods of time (Confrey, 2012; Yin, 2013). They are also akin to ethnographies because the researcher is a participant observer (Barab, 2012; Confrey, 2012). Peer review by an expert in mixed methods research and external to the project indicated that the selected methodology and methods are appropriate for answering the stated research question.

Methods

Locating the study

This study was approved by the Institutional Review Board. The focal case of this study is a week-long instructional sequence about groundwater resources, which constituted part of a five-week module on natural resources in an introductory-level college geoscience course. The sequence consists of three 50-minute class meetings. The details of its iterative design over five years at two large universities in the United States are described in Arthurs (2019). As the focal case for the present study, the sequence was implemented in two iterations (i.e., two different semesters) at the same large public midwestern university.

Sample population

During one iteration, 29 out of 48 students both (1) consented to their course work being used for research and (2) completed the pre- and post-instruction assessments. In another iteration, 32 out of 40 consented and completed assessments. Although all students completed the postinstruction assessment, not all students provided consent and not all students were in class on the day of the preinstruction assessment. Thus, 61 students compose the study’s sample population and its demographics are summarized in Table 1.

Data sources

To address the cognitive, temporal, and social considerations associated with the knowledge integration perspective of conceptual change and consistent with a design study methodology, the following sources of data were used: instructor lesson plans; student responses to paper-and-pencil in-class activities; instructor notes about in-class activities and discussions; and student responses to homework, exams, and pre- and postcourse surveys. Classroom observations were also recorded.

The cognitive dimension of conceptual change herein is conceptual learning gains. They were estimated using responses to a free-response item before and after the instructional sequence (see supplemental Table 1 at https://www.nsta.org/online-connections-journal-college-science-teaching). Students completed an in-class activity as a gauge of their prior knowledge a week before the instructional sequence began. Students addressed a similar item in the course’s final exam. The items consisted of a prompt and a large blank space in which to draw and label a sketch. In this study, such an activity is referred to as a free-sketch activity because students begin with an entirely blank space where they are free to construct a sketch from scratch.

The temporal dimension of conceptual change herein is the longitudinal development of students’ mental models. They were assessed at four different times during the course (see supplemental Table 1 at https://www.nsta.org/online-connections-journal-college-science-teaching). The first time (T1) was the week before the instructional sequence began. T2 was during the instructional sequence, after a mini lecture that separately described three main types of aquifers. T3 was three weeks after the instructional sequence ended (during a mid-term exam). The last time (T4) was eight weeks after the instructional sequence ended (during the final exam).

In contrast to the free-sketch activities at T1 and T4, the assessments at T2 and T3 were delimited-sketch activities. A delimited-sketch activity begins with a partial sketch already provided, which we refer to as a base-form sketch. Students add to the base-form sketch by drawing additional features, amending existing features, and labeling their sketch to help clarify their ideas.

The formative assessments administered at T1 and T2 were in-class activities. As with other in-class activities in the instructional sequence and course, the instructor let students know that their responses would help her better understand what they know and their current thinking, which would assist her in helping them take what they already learned to the next level. The instructor also reminded students they would earn two points toward their in-class activity component of the course for completing their work clearly and demonstrating a good-faith effort at communicating their ideas, not for correctness. As formative assessments, anonymized responses were displayed and discussed in subsequent lessons in the instructional sequence to summarize the diversity of ideas expressed and to build on those ideas through follow-up in-class activities and discussions.

The summative assessments administered at T3 and T4 were exams. All exam items were worth up to two points each, and the items used in this study were worth two points. As summative assessments, the results of these exams were not discussed as part of any follow-up instruction in class. Responses to other homework and exam items that were not sketch based and the instructor’s notes about in-class discussions and interactions with the students provide additional information about the longitudinal development of students’ mental models.

The social dimension of conceptual change herein addresses primarily student interactions during the focal instructional sequence. To explore these interactions, sources of data include: the instructor’s lesson plans and notes about student interactions during in-class activities and discussions, student responses to pre- and postcourse survey items (see supplemental Table 2 at https://www.nsta.org/online-connections-journal-college-science-teaching), and classroom observations. Two trained observers external to the course and department used the Classroom Observation Protocol for Undergraduate Science (COPUS) (Smith et al., 2013) to observe the instructional sequence.

Data analyses

Analyses of concept sketch items

Sketches generated during free- and delimited-sketch activities were analyzed using diagrammatic (Gobert, 2000) and textual content analysis (Sapsford, 1999). Analyses were formalized with a rubric (see supplemental Table 3 and 4 at https://www.nsta.org/online-connections-journal-college-science-teaching). The rubric was developed to evaluate specific features in the concept sketches against an expert standard for comparison. The expert standard used for this study is described in the course textbook and describes three types of aquifers in which groundwater resides: perched aquifers, unconfined aquifers, and confined aquifers (Reichard, 2010). Using this description, one researcher developed a scoring rubric in which the highest score that could be assigned to a concept sketch is “6” and these scores are then translated into percentages (six points = 100%). The closer to 100% a concept sketch is, the more expert-like the communicated mental model about groundwater is. After the scoring rubric was developed, two research assistants with a geology background critiqued the rubric for content, clarity, and organization. The critiques contributed to the rubric’s trustworthiness (Guba, 1990).

One researcher applied the rubric to all concept sketches using a process of double coding to ensure the repeatability of coding results (Krefting, 1991). The two coding sessions were spaced approximately a week apart, so the coding results from the first session were forgotten. There was more than 95% agreement between results from both coding sessions. The researcher contemplated disparities and assigned a code.

To further check and recheck the coding results, another researcher independently applied the rubric to all concept sketches. The two researchers then compared their coding results to achieve interrater agreement (LeBreton & Senter, 2008). A comparison of scores assigned by the two researchers resulted in more than 84% interrater agreement prior to discussion, and after discussion resulted in 100% interrater agreement. The final concept sketch scores were statistically analyzed.

Data collected at T1, T2, T3, and T4 were statistically analyzed to examine the overall development of students’ mental models over time. Potential statistical differences from one time period to another was determined using a two-tailed t-test and the effect size was determined by calculating Cohen’s d. The same analyses were performed to determine potential differential impacts based on gender and race.

Analysis of other items

Additional insights on the evolution of students’ ideas about groundwater were obtained via answers to groundwater-related items in homework and exams. These items were mainly multiple-choice items and a few were open-ended items.

Classroom observations

Two trained observers used the COPUS to observe all three days of the focal instructional sequence. They produced a report describing their observations. Information in their report is used in conjunction with the instructor’s lesson plans and notes to help characterize the social dimensions of conceptual change.

Pre- and postcourse surveys

Additional insights on the social dimensions of conceptual change were obtained via responses to items on the pre- and postcourse surveys. Matching Likert-scale items were analyzed to determine whether any shifts in students’ attitudes toward working alone and with others might have occurred. A free-response item in the postcourse survey was analyzed to determine the frequency with which social aspects of the course were mentioned by students.

Results

Cognitive dimension

The overall conceptual learning gains were determined by comparing rubric C scores for the concept sketches created at T1 and T4 (Table 2). The results show a significant positive shift toward more expert-like mental models for all students in the study group (p = < 0 .00001, d = 4.460217).

Temporal dimension

The most prevalent preinstructional mental model of groundwater is that it exists as large open underground pockets or reservoirs; whereas, in fact, only a small percentage of ground water is found in such reservoirs. The results show a gradual change toward more expert-like mental models over time. Evidence of students’ conceptual change as a function of time were determined by applying the rubric to concept sketches collected at T1, T2, T3, an T4 (Figure 1). Additional evidence of a shift toward more expert-like ways of conceptualizing groundwater comes from performance on groundwater-related items in homework and exams, which have a combined average of more than 80%. The instructional sequence had a positive impact on students’ conceptual development for male and female students as well as Caucasian and non-Caucasian students, with no statistically significant difference.

Social dimension

The instructor’s lesson plans, instructor notes, and external observers’ report show that the instructional sequence is best described as an interactive lecture with a conversational tone, with back-and-forth sharing of ideas between students and between students and instructor. Each class meeting comprised 19 or 20 PowerPoint slides. Of them, 40% were used to facilitate in-class activities and discussions, 35% were used to transfer information via lecture, 15% engaged students as part of lecture, and 10% were used as visual transitions from one topic to another and/or as announcements. During times of lecture, students were actively engaged in note taking, listening, and asking questions. During times of individual work, they were engaged in thinking and committing their thinking to paper. During group work and class discussion, the room was alive with audible discussion, inquiry, and even laughter.

Based on results of the pre- and postcourse survey items, there were no shifts in the extent to which the sample population liked working alone, working with other people, and their preferences for working in one way or another. The free-response item on the postcourse survey revealed 3% of the students wanted “fewer in-class activities” or “less participation.” Meanwhile, 52% of students noted they enjoyed one or more social aspects of the course. Of these students, 12% (n = 6) provided suggestions for doing more.

Discussion

Cognitive implications

The dominant preconception of groundwater existing as large underground pockets or reservoirs of water is shared by students across grade levels and regions (Arthurs & Elwonger, 2018). The pre- and postinstruction results indicate the preconceptions-based instructional sequence was very effective at facilitating a high degree of overall conceptual change for most students (Table 2). This is counterintuitive to notions that novice conceptions are potential barriers to learning, potentially harmful to the learning process, and should be extinguished.

The empirically derived results suggest deliberate solicitation and engagement of students’ preconceptions can be effective as instructional tools. This is consistent with the knowledge integration perspective of conceptual change, which argues “variability in student ideas is fundamentally a valuable feature and that instruction designed to capitalize on the variability … has potential for facilitating conceptual change” (Linn, 2008, p. 715). It also promotes the practice of characterizing the repertoire of student-held ideas and “adding the right ideas to the mix held by students … as a way to increase the efficiency and effectiveness of instruction” (Linn, 2008, p. 716).

Temporal implications

The postlecture results (T2) suggest lecture alone has limited impact on student learning. The learning gains made between T2 and T3 provide evidence that actively engaging students’ preconceptions and ideas facilitates conceptual change. Furthermore, the results from T3 to T4 suggest the learning gains persisted even two months after the instructional sequence. The same pattern of general improvement is similar between male and female students as well as Caucasian and non-Caucasian students, suggesting the engagement of student preconceptions is an instructional tool that benefits students from diverse backgrounds.

Also, the results from T1 to T4 (Figure 1) indicate conceptual change occurs at different rates for different students and is, overall, a more gradual process rather than a revolutionary process. These empirical data support the knowledge integration perspective, which suggests learning is gradual (Linn, 2008). Gradual conceptual changes occur because students need to grapple with their own perhaps confusing and conflicting ideas (Linn, 2008). Indeed, it takes time for students to more fully understand their own ideas and how they integrate or not with the ideas learned in class.

Social implications

The COPUS results corroborate the opportunities described in the instructor’s lesson plans and notes. Consistent with the knowledge integration perspective of conceptual change, such opportunities allowed students to (1) individually engage in a personally relevant issue (i.e., groundwater as a drinking water source); (2) make their individual thinking visible to themselves, their peers, and instructor; (3) learn from one another by sharing, discussing, and evaluating one another’s ideas; and (4) reflecting on and monitoring their performance.

Although not specific to the focal instructional sequence, the results of the free-response item in the postcourse survey indicate that many students valued various social aspects of the course and their learning experiences in it. These responses provide some affective insights into how students value social interactions as a part of their learning experiences. The course had no statistically significant impact on how the sample population liked working individually and with others. Their views and their preferences in this regard remained largely unchanged. The instructional sequence positively impacted students who preferred to work alone, those who preferred to work with others, and those who had no preference.

Limitations

In this study, mental models were characterized using concept sketches. The free- and delimited-sketch activities were designed to elicit two different but complementary ways of communicating students’ ideas: diagrammatic and textual communication. Although a concept sketch can communicate key elements of an individual’s mental model, it does not necessarily communicate all elements that may be present (Clement, 1982; Henriques, 2002; Osborne & Wittrock, 1983). Drawing to communicate one’s ideas has similar goals and limitations to communicating verbally—the goal of clarity and the potential for imperfect conveyance of ideas. Despite its limitations, drawing has a long and demonstrated history as a useful tool for studying mental models and cognitive development (e.g., Piaget, 1956).

Research based on the design study methodology has local impact, and the challenge is scaling up in such a way that the findings can inform implementation in other contexts (Dede, 2012). A perceived limitation of the design study methodology is that complex learning environments contain multiple interacting variables that may be confounding. Despite these criticisms, design studies provide valuable perspectives and insights that cannot be obtained under contrived and controlled laboratory conditions because, as Barab (2012) notes, “If researchers only study that which takes place in controlled conditions, they run the risk of developing artificial meanings and interactive dynamics that are so free of contextual realities that they may not be able to inform real-world practice” (p. 154).

Generalizability

Design studies have constraints to generalizability as a “whole package;” however, the developed theories, artifacts, and practices can be generalizable for other instructors and students (Barab, 2012). The results of this study provide empirical support for the knowledge integration perspective of conceptual change, the artifacts it produced are mental models of groundwater that are comparable to other studies, the practices developed include the instructional sequence itself and the parts that comprise it (e.g., base-form sketch). Evidence of the instructional sequence’s potential generalizability rests in its development taking place in courses taught at two different institutions with enrollments ranging from 48 to 312 (Arthurs, 2019). Additional evidence of the potential generalizability rests in the knowledge that the preinstructional mental models held by students in the sample population are held in common with students across grade levels and regions (Arthurs & Elwonger, 2018). The evidence suggests that the instructional sequence, or parts of it, may have applicability and utility in other science courses that study groundwater, at different grade levels, and in different regions.

Conclusion

The theoretical value of using student ideas to facilitate conceptual change (Linn, 2008) and student-held conceptions as instructional tools for developing deeper understandings about groundwater (Bar, 1989; Meyer, 1987) were empirically tested in this study. The findings of this study help connect theory and practice. Consistent with the knowledge integration perspective of conceptual change and the design study methodology, this study:

• provides empirical support for the knowledge integration perspective of conceptual change by showing students become more expert-like when their preconceptions and evolving ideas are engaged during instruction.
• illustrates how student-produced artifacts (e.g., concept sketches) can be used to further understand their mental models and gain insights into their developmental trajectories.
• evaluates the impact a preconceptions-based instructional sequence had on students’ conceptual change, to help inform others’ decisions about using the instructional sequence or parts of it for instruction and/or research.

Furthermore, this study provides evidence that preconceptions-based instruction benefits diverse students including males and females, Caucasians and non-Caucasians, and students who prefer to work alone and those who prefer to work with others. ■

Leilani A. Arthurs (leilani.arthurs@colorado.edu) is an assistant professor in the Department of Geological Sciences at the University of Boulder. Justin Elwonger was an undergraduate research assistant in the Department of Earth and Atmospheric Sciences at the University of Nebraska–Lincoln; currently, he is a science teacher in Shickley, Nebraska. Chelsie M. Kowalski is a research assistant in the Department of Geological Sciences at the University of Colorado at Boulder.