Participating in citizen science—where members of the public collect data for use by scientists—has become popular over the past 5 to 10 years, with teachers increasingly looking to incorporate citizen science into lessons (Gray, Nicosia, and Jordan 2012). Unfortunately, implementation of projects in the classroom can be difficult. Many teachers I have worked with report limited time to investigate a project and prepare lessons for it. They must “teach to the test” or adhere to strict curriculum goals that exclude the possibility of implementing citizen science, or have a difficult time getting administrative approval for implementation. This is unfortunate, as K–12 students represent a huge population of potential citizen scientists who could: (a) collect large data sets, (b) learn science by doing science, and, perhaps most importantly, (c) gain a better understanding of how science truly works through participation in projects where the ultimate outcome of the research is unknown (Shah and Martinez 2016).

As evidenced by the widespread adoption of the Next Generation Science Standards (NGSS Lead States 2013) and Common Core State Standards (NGAC and CCSSO 2010), educators are moving away from cookie-cutter demonstrations of science and moving instead toward an inquiry-based approach, where students ask questions and learn how to answer them. These standards engage students in the processes employed by professional scientists, training them to ask and answer questions themselves rather than simply observe demonstrations of scientific concepts and arrive at known answers after following step-by-step instructions. Citizen science projects provide students the opportunity to ask and answer questions, and experience true scientific discovery and the joy of learning something completely new—qualities often stripped from science education that relies on demonstrations of concepts rather than inquiry and scientific skill-building.

Given the challenges teachers face when implementing citizen science in the classroom, the opportunity exists for informal science education organizations, such as museums, science centers, and environmental education centers, to offer these experiences to K–12 students during out-of-school time (Bonney et al. 2009). If thoughtfully designed, such programs can augment science learning in formal education settings and help prepare students to become, if not scientists themselves, scientifically literate adults who understand the process of science sufficiently to make informed decisions.

Program overview

Dragonfly Detectives is an afterschool citizen science program for children in grades 4–8 offered by the North Carolina Museum of Natural Sciences (NCMNS). The goals of the program are threefold:

1. to provide opportunities for North Carolina’s children to learn about nature in nature,
2. to allow students to participate in almost all stages of the scientific process, and
3. to allow students to collect publication-quality data as citizen scientists.

We use a distributed model of leadership for Dragonfly Detectives, recruiting educators at outdoor sites, such as environmental education centers and state, county, and local parks, that are interested in offering the program. We then train educators in the Dragonfly Detectives curriculum and scientific protocols during a required full-day training, which includes an overview of the agendas for each day of the youth program, practice collecting data for each of the citizen science projects, a lesson on dragonfly identification, and instructions for how to submit data. We also distribute all the materials they need to implement the program at their site. This model allows the two NCMNS staff dedicated to Dragonfly Detectives to focus on training and support for program educators, rather than leading every program ourselves. We are able to reach far more students with less funding with this distributed model than we could relying on our staff alone. Although we cannot guarantee that every child participating in Dragonfly Detectives receives the exact same experience and coordinating educators at so many external sites is complex, we feel that the benefits outweigh these concerns.

At each site, Dragonfly Detectives program educators work with groups of up to 15 students for two hours per day, once a week, over six weeks.

Dragonfly Detectives has similarities to the University of Minnesota Extension’s Driven to Discover program but was developed without prior knowledge of that program. The Dragonfly Detectives program is based on a custom-built citizen science project, whereas Driven to Discover uses large-scale, pre-existing citizen science projects.

Day 1

The program educator travels to the school to work with students. Students are given the information they need for the remainder of the program, such as:

• an overview of citizen science and the three dragonfly citizen science projects they will participate in (Dragonfly Pond Watch, Dragonfly Swarm Project, and the main project, Dragonfly Detectives),
• how to identify the five focal species of dragonfly they will study and how to use the equipment, and
• a “meet the scientist” video that introduces them to the scientist who will be using their Dragonfly Detectives data.

After learning how to identify the dragonflies, students take a dragonfly identification pretest to see how effectively they identify the dragonflies. Students then review the scientific process and make a hypothesis as a group that addresses the main Dragonfly Detectives question: What are the best and worst weather conditions for dragonfly flight? Students typically hypothesize that dragonflies prefer hot, calm, and sunny weather and avoid flying on cold, windy, and cloudy days, but the exact hypothesis varies from group to group.

Days 2–5

Students travel to a nearby outdoor site. They collect data for the three citizen science projects during their visits, including the primary data collection effort for Dragonfly Detectives. This project investigates the role that weather plays in influencing dragonfly flight activity, so we divide students into two groups: A small group records weather data while the rest line up shoulder to shoulder near the pond and count the number of common whitetail dragonflies that cross their field of view (Figure 1). Students count silently for a three-minute period and compare their count to one or two other people afterward. To minimize the “competitive counting” that can occur (students often like to see who can get the biggest count and count poorly or even make up numbers to do so), we focus on students getting the same count as their neighbors. Everyone then combines the weather and count data in their workbooks and compares the current count to counts they have made before so they can begin to see the influence of weather on dragonfly flight.

During the outdoor days, students also participate in several fun, educational activities that focus on a variety of topics related to dragonflies, including dragonfly life cycles, aquatic insects, aquatic habitats, water quality, adaptations, and migrations. On two days, participants catch, mark, and release dragonflies, so they get an up-close look at the animals they are studying. Students also write a weekly journal entry that encourages them to use what they learned for the day and apply it in a new way (Figure 2).

Day 6

The program educator travels to the school to help students analyze the Dragonfly Detectives data they collected throughout the program and determine whether their initial hypothesis was correct. As a group, students then create a scientific poster that illustrates their hypothesis, methods, results, and conclusions. Students learn about careers in science and take another dragonfly identification quiz to measure improvement after their field experience. All data that students collect are then submitted to the appropriate projects so that their data become part of the scientific knowledge of dragonflies and contribute to publishable research. Preliminary results of the Dragonfly Detectives project suggest that temperature, light intensity, and humidity are most strongly correlated with dragonfly flight, but a complete analysis of the data has yet to be completed.

Summer camp variation

An alternative version of the program is offered through outdoor sites that wish to offer Dragonfly Detectives in summer camp settings. Students typically complete the program over five days instead of six in summer camps, but because they spend the same number of hours in the program as in the afterschool format, the program is nearly identical.

BugFest

All students who participate in Dragonfly Detectives are invited to exhibit their posters at BugFest at NCMNS as an optional concluding activity. Students who attend BugFest talk to attendees, either by staffing a dragonfly information table or by standing by their posters and talking to visitors about what they did in the field (Figure 3). BugFest routinely attracts over 30,000 people, so the youth who choose to attend show their work to thousands of people in a very public venue.

Program funding

Dragonfly Detectives is funded by a Student Science Enrichment Program (SSEP) grant awarded by the Burroughs-Wellcome Fund. The grant provides all the materials that educators at an outdoor site need to implement the program, so that the only cost is staff time. The grant also provides transportation from schools to the outdoor facility, snacks, and equipment that students use and keep at the end of the program, including a professional insect net, a dragonfly field guide, a water bottle, and the workbook containing their data and journal entries from the program.

Between 2015 and 2017, we worked with nearly 500 children at 25 different sites in North Carolina. Participants span grades 4–8 and represent a wide range of races and socioeconomic statuses. Most students attend public school or are homeschooled, including some students with special needs such as children with autism or attention deficit hyperactivity disorder. Due to the successful completion of the program, as indicated by an overall increase in dragonfly identification scores, accurately collected data (see “Can Children in Grades 4–8 Collect Publication-Quality Data?” below), and evidence of understanding as gauged by journal entries, by nearly every student, we feel that it is inclusive and works effectively for the age range we target.

Connection to schools

Dragonfly Detectives was designed to augment and support the lessons students learn during their in-school time. Our lessons include at least one connection to the North Carolina state science curriculum for each grade level. For example, fourth-grade participants learn about adaptations in aquatic insects and are asked to write about an adaptation in a live pond insect they observe. The fourth-grade curriculum also directs teachers to introduce the use of more complex instruments, so students use weather meters and light meters to measure weather parameters.

The teachers who oversee the groups we work with nearly universally report that their students talk about what they learn through Dragonfly Detectives in school. Participants identify the dragonflies they see on their school grounds for their classmates and bring up what they learned in class. One teacher reported that students who participated in the program sometimes talked about their experiences to an excessive degree, causing the teacher to ask them to refrain from sharing as often during lessons.

We took teacher feedback into account as we assessed year 1 of the program and made improvements in years 2 and 3. Informal surveys of teachers at all sites were conducted in person or via e-mail to determine which parts of the program they liked, and we solicited recommendations for improvement. Although the teachers largely approved of the program, they did make valuable suggestions for improvement. For example, in year 1, all students were given a composition book in which to keep their data and journal entries. It became immediately apparent that these blank notebooks were confusing to most students, so teachers recommended a more targeted workbook. The workbook currently in use includes copies of the data sheets for each day of the program, pages for each journal entry, and all the information they learn through the activities (e.g., how purple martins, monarchs, and green darner dragonflies migrate), so they can refer back to it after the end of the program. The workbooks also act as a source of ideas for extension after completion of the program, including additional data sheets for Dragonfly Pond Watch, suggestions for other dragonfly-related studies students can undertake, coloring pages, and other resources. Feedback from teachers was instrumental in adding this resource to our materials, adjusting the materials we offered to participants at the end of the program, adjusting our expectations of how best to offer transportation, and other changes that we feel improved the program dramatically from the first to the second year.

Because we ask students to participate in all aspects of the scientific method apart from developing a question and a step-by-step investigation, students learn about the process of science by participating in that process firsthand. We believe that students take away a better understanding of how science works in the real world due to their participation in Dragonfly Detectives, though more data is required to test whether this truly happens and how. However, anecdotal support comes from the surveys administered during the program as part of the Burroughs-Wellcome Fund SSEP grant obligations, where 89.6% of students reported that they have improved their understanding of science due to their participation in Dragonfly Detectives.

Driven to Discover reported similarly high levels of understanding of the scientific method in their program, with statistically significant increases in self-reported inquiry skills between pre- and postprogram assessments (Garibay Group 2015).

Student enthusiasm

The Dragonfly Detectives program offers students an opportunity to collect data for a local scientist as they learn about a group of organisms that is new to most participants. Although we ask that they participate in some activities that they may find tedious (counting dragonflies and creating their scientific posters, for example), the program results in a high level of student enthusiasm. Evaluation of journal entries includes looking for keywords in the written text and assessing the overall student attitudes toward the program. These data suggest that participants feel proud that they contribute data to a real scientific project and of their learning during the program (Figure 4 contains examples of text contributed by participants). Program educators often talk with students before they begin their activities and students routinely share dragonfly encounters, how they are teaching their family and friends to identify area dragonflies, and new things they learned about dragonflies through independent investigation. We administer a parent survey during the program and parents confirm what the students tell us: Participants speak enthusiastically about the program and dragonflies, and are excited by the things they learned (average score of 4.52 out of a possible score of 5, with 5 being the most positive, n = 75). Ninety-five percent of Driven to Discover participants reported high enthusiasm for their program (Garibay Group 2015), suggesting that this approach to teaching science is one that students enjoy.

Students who participate in BugFest confirm these observations. Most BugFest participants complete their program several months before the event, so they are coming back to their posters after significant time away. It is clear from observing students as they talk about their work with thousands of visitors that they remember what they learned, are proud of what they accomplished (no matter how tedious preparing their posters may have been!), and are proud that they were asked to speak about dragonflies as an expert. Students at BugFest quickly notice that they know far more about dragonflies than most of the visitors and that they have information about a popular insect that they can share with interested attendees. Visitors often make a point of telling us that they enjoyed learning from the children and are impressed with their knowledge. In fact, some of the children are so excited about sharing that they occasionally trap visitors in conversation and must be coaxed away to allow the hapless visitor to escape!

We have limited evidence that some Dragonfly Detectives participants continue their dragonfly investigations well beyond the end of their program. Several BugFest participants tell us about dragonfly experiences they have had since their program ended, including teaching people how to identify common local dragonflies, making dragonfly observations near their homes or schools, and even continuing to participate in the Dragonfly Pond Watch project. One student went a step further and developed a Dragonfly Club for her homeschool group that meets at a neighborhood pond once a week during the warm months to observe dragonflies and ask and answer their own dragonfly questions.

Skill retention

Although we do not track the students beyond the end of the program to see how long they retain their knowledge and skills, we have some evidence that the program produces lasting results for some participants. Although most students who participate in BugFest are ones who especially loved Dragonfly Detectives and are therefore most likely to retain what they learned, the youth can immediately identify the focal species when we informally quiz them and recall the results of their study. Most BugFest participants completed the program between one and five months prior to the event, so they are clearly able to retain the information for at least a few months.

In 2017, NCMNS invited all Dragonfly Detectives participants from the first three years of the program to a social gathering. One activity was a trivia game during which the children competed for prizes. The speed with which students recalled specific information, such as which characteristics to look for to identify a wandering glider or how to differentiate black saddlebags (one of their focal species) from Carolina saddlebags (not a focal species but present in large numbers at almost all sites) was impressive. All attendees were able to answer enough questions to take home a prize, which provides anecdotal evidence that at least some participants retain what they learn for two to three years.

One teacher whose students participated three years previously recently told us that her students are still talking about dragonflies, still helping their classmates identify the species on their school grounds, and using the equipment they took home. It is heartening to hear anecdotes such as this, but it also suggests that the experience was sufficiently impactful to result in lasting effects.

Can children in grades 4–8 collect publication-quality data?

One of the goals of Dragonfly Detectives is to investigate how effective children are at collecting publication-quality data. Although citizen science is not a new form of scientific discovery, science in the past 200 years has evolved into an exclusively professional enterprise (Cooper 2016). As a result, amateur-generated data are often met with skepticism and distrust such that citizen scientist–generated data are scrutinized more heavily than scientist-generated data (Bonney et al. 2014). Proof of sound data must be offered for results to be taken seriously and considered valid. Although there is ample evidence that citizen science data are of about equal quality to that collected by professional scientists—so long as the project leader is realistic about who the citizen scientist audience will be, what tasks they can perform well, and the training required (Kosmala et al. 2016)—the need to illustrate data quality places a significant burden on scientists or citizen science project leaders as they seek to publish their research.

Citizen science is increasingly used as an educational tool in K–12 settings, but the effectiveness of these programs remains understudied (Ballard, Dixon, and Harris 2016). Even less well known is how effective children are at collecting data as citizen scientists. Given the concern that many scientists have for data collected by adults, we investigated how well children collect data, especially given the complex tasks many citizen science projects require of them.

To answer this question, we look at several measures of effectiveness. The dragonfly identification quizzes compare the identification scores of the participants on their first day (immediately after learning how to identify the five focal species of dragonflies) to their scores on the last day (after spending several days in the field observing dragonflies in nature). Because we ask students to collect data for two citizen science projects that depend on their accurately identifying specific dragonfly species, this skill is essential if they are to collect useable data. Data show that students increase their identification scores from the first day to the last from 70% to 90%, and nearly all students are able to accurately identify all focal species by the end of the program (Goforth and Peterman, Forthcoming). We do not submit data from the first day for the Dragonfly Pond Watch to ensure that only quality data are shared with the project leaders. In contrast, students are nearly universally able to identify the common whitetail dragonfly from the first day, so the Dragonfly Detectives data are not hindered by ability to accurately identify the focal species for that project.

One potential issue we anticipated was that participants might not accurately count the dragonflies for Dragonfly Detectives. Because we ask them to stand still and silently count dragonflies for a block of time, however small, we worried that students would become distracted or start competing with each other to get the biggest counts, whether they were accurate or not. The heavy emphasis we place on precision, or getting the same score as your neighbors, has helped keep the children focused with less urge to compete. This measure does not specifically address accuracy, however, as students can get the same count as their neighbors without accurately counting dragonflies. Starting in year 2, we video-recorded the pond at one site as the participants counted dragonflies. The lead scientist counted the dragonflies captured on the videos and we compared her counts to those of students. On the first day, a small percentage of student counts approached the expert count, but by the last day most student counts were close or identical to expert counts (Goforth and Peterman, Forthcoming). The data students gather the first (and sometimes second) field day is low-quality and cannot be used for publication, but by the third and fourth day, students collect data that we consider publication-quality. These data become a part of the research submitted for publication.

We have more data to incorporate into our analyses, and the accuracy data presented here is from only one site that has the most contact with the project leaders and scientist. Other sites further removed from the project scientists could show different results, but we feel confident that our Dragonfly Detectives participants collect high-quality, useful data for at least some portion of their time in the program.

Conclusion

We believe that Dragonfly Detectives is an example of a citizen science program that effectively engages children in authentic science. Through the program evaluations, we have learned that students enjoy participating in science when they know they are doing something real. Similar results were reported for Driven to Discover (Garibay Group 2015).

We were impressed with the results of the preliminary data analyses that showed how effectively students identify the focal dragonfly species and accurately count dragonflies. Tasks such as these are hard for many adults and we asked children as young as nine to attempt them. Happily, with simple weekly reviews of the dragonfly species before starting data collection, an emphasis on collecting precise and accurate data, and extensive training in the data collection protocols, students were able to complete these complex tasks. Furthermore, they completed these tasks with sufficient skill by the end of the program that they felt confident enough to teach adults at a public event such as BugFest.

One aspect of Dragonfly Detectives we feel has particularly helped our students succeed is that students are reminded repeatedly that the data they collect are valuable and are going to someone who intends to use them. Students “meet” the scientist through a video on the first day so they can see and hear the person who collects their Dragonfly Detectives data, and many students meet her in person at some point because she lives and works nearby. Students work throughout the program to collect data they will submit to three projects, all of which generate scientific publications based on citizen science data. They also know that their results will be viewed by the general public. Although some sites are sufficiently far from NCMNS that participants cannot easily attend BugFest in person, all posters are on display during the event. We believe that a combination of seeing a local scientist who will use the data they collect and knowing that everyone can see their final results encourages participants to collect the best data possible throughout the program.

Overall, we feel that thoughtful use of citizen science projects can support science education offered during in-school time and is well within reach of many informal science education institutions. Although we have grant funding to cover the materials costs, we have been able to reach many students throughout our state with minimal staff time. The participants are enthusiastic about what they learn and love sharing their knowledge. They are proud of the science they do and feel empowered to continue their investigations on their own after the end of the program. We believe that Dragonfly Detectives has been a good experience for everyone—our staff, our educators, our participants, and our scientist. Driven to Discover has had similar successes and has reached even more participants than we have, making it an even greater model of the success of this approach. As a result, we encourage other institutions to develop similar projects that allow children to explore science through thoughtfully created citizen science programs that allow participants to experience as much of the scientific process as possible. By building on the science students learn in school while providing an authentic science experience that allows students to experience the joys of scientific discovery, we have the potential to start building a more scientifically literate generation who can make better informed decisions well into the future.

Christine L. Goforth (chris.goforth@naturalsciences.org) is head of citizen science at the North Carolina Museum of Natural Sciences in Raleigh, North Carolina.