For the summers of 2020 and 2021, COVID-19 produced a “wicked problem” (Braund 2021) for sustaining established STEM programming. The challenges included the transfer of hands-on educational activities to virtual, poor internet infrastructure in rural and less-advantaged communities, and lack of experience conducting virtual STEM programs. Many programs, including those described here, made adaptations to continue their programs (Allen 2020).

This article discusses six summer STEM programs in Arizona that developed ways to meet the challenges. The authors, from the Community Engagement Core of the Southwest Environmental Health Sciences Center, have been conducting summer STEM programs for 15 years. These programs provide pathways for students from middle school to community college to explore STEM topics and careers. We have found the environmental health lens to be effective for exploring many STEM fields, degrees, and careers. It is a broad and interdisciplinary area of study that is important to the students and communities we serve, which are primarily American Indian (Lindsey et al. 2021), Latinx, and less advantaged socioeconomically. During the pandemic, the topic of environmental health became especially important as it taught students about epidemiology, interpreting data, understanding the research process, identifying bias, and the importance of science literacy because of its relevance to the direct experience of students and their communities.

The original programs and their virtual adaptations were developed in consultation with educational partners including teachers, administrators, and workforce development professionals. In planning the 2020 virtual summer programs, collaborating teachers and community partners noted that participants were poorly motivated by remote learning during their spring 2020 classes, which led to the essential adaptations described in this article.    

Further, lessons learned in the summer of 2020 informed the summer 2021 programming, some of which continued to be virtual. Findings from program experience and evaluation, educational research, and education literature concerning Latinx, and Native American students have led the authors to several critical insights for optimal learning. These include the participants

• needing repeated exposures to skills and information to be able to use the information (Lewin 1997);
• learning best with culturally relevant content, wholistic learning, and as much choice in projects as possible (Lindsey 2000);
• succeeding through opportunities for visual and auditory learning, personal relationships between teachers and students, and information essential to functioning in dominant society (Lindsey 2000); and
• thriving in opportunities to work cooperatively and to develop feelings of competence (Lindsey 2000).

The programs have varying structures but share the following common processes based on educational theories and experiences: (1) science literacy development, (2) college readiness experiences, (3) lab safety and research methods training, (3) active involvement in conducting science, (4) relationships with multiple mentors to guide participants, and (5) engagement in a reflective process that encourages learning. Figure 1 shows how proud students are at the end of the programs when they demonstrate the competencies developed during the programs.

Collaborations

Community partners and teachers were involved in the curriculum development and logistic planning of these programs in various ways. Specific community partners and collaborating teacher types are identified in Table 1.

For example, high school biotechnology teachers informed the curriculum and taught specific lessons for the initial weeks of Keep Engaging Youth in Science (KEYS) (Ingram et al. 2019) and Steps 2 STEM. They designed and helped teach lab techniques, biotechnology concepts, and how to read research papers. Community partners were also heavily involved in both the curriculum covered and logistical planning for the Toxic Detectives and Our Land, Education and Health (OLEH) programs. Their collaboration was essential to ensure the curriculum was culturally relevant and responsive for student participants and that the timing of the programs did not create unneeded barriers for student participation. The curriculum for A Student’s Journey (ASJ) was largely designed in partnership between University of Arizona and Tohono O’odham Community College staff members, with the program design heavily informed by community members, former students, and community college teachers. Together, all six programs serve at least 130 students each summer.

Programming Rationale and Goals

All six programs aim to accomplish three primary goals: (1) to advance student science knowledge, critical thinking, and skills; (2) to empower students to develop authentic self-confidence; and (3) to introduce students to the university environment and the skills needed to matriculate.

University introduction topics are addressed because over half of the students are from underserved communities and schools, and these students are likely to be the first in their families to attend college. They benefit from guidance about academic culture. The programs provide age-appropriate and culturally relevant science experiences.

From program evaluations over the ye­­­ars, we have learned that participants see themselves “doing science” and “being a scientist” in very real ways (see Figure 2). They show their comprehension and expertise through hands-on mind-on activities (Young 2002) and through regular reflective learning and reading, writing, and speaking about science. Participants critically reflect on the real-world science activities and the people they met to consider how they personally were changing and growing because of program activities. The intention of the programs is to create a pipeline for students to follow as they explore a future in STEM.

Reflective learning is founded on the work of John Dewey (1980). The prompts incorporate student responses to the experiences. Reviews of hundreds of participants’ reflections have consistently found them describing their work in highly detailed “science speak” as casually as they narrate the fun they had at a social gathering. Many also comment, “I really am a researcher!”

The science foundation for all programs is environmental health, which combines environmental science, biology, chemistry, and risk communication (Table 2).

Lab safety trainings (Bloodborne Pathogens and Universal Precautions (OSHA Compliant), General Lab Chemical Safety, Fire Safety Awareness, Basic Biosafety) are conducted through the university online portal. Guest speakers, professionals, faculty, and graduate students from Native American and Latinx communities are included as exemplars and role models to develop the students’ sense of belonging on a university campus (Weiss 2021). Thus, the students view themselves as scientists from these experiences (see Figure 3).

Many mentors surround each participant with a “constellation” of mentoring relationships (Mittan 2018). World-class scientists, post-doctoral researchers, graduate and undergraduate students, near-peer student paraprofessionals and program peers provide support. Vygotsky (Taber 2020) describes the learning situation for students with a “more capable peer” can achieve more than when working in isolation. Students can do more at a higher level than they thought possible because they are given opportunities to do difficult things with a guide on their side.

The science content of each program is appropriate to the preparation level of its participants and designed to meet their needs (see Table 3).

Three of the programs are aimed at less-experienced students and thus involve informal science activities, faculty and graduate student interviews, and lab visits (see Figure 4). The other three programs, for more advanced students, focus on research internships—hands-on lab work on real science in process.

For all programs, guest experts present workshops about environmental health topics, cultural connections, academic skills, STEM careers, and academic pathways. When appropriate and if scheduling permits, students from more than one program participate in the same workshop, working with students of different ages and seeing the progression of the six programs. All programs include time for participants to practice their final oral presentations in private (Lindsey 2000), receiving peer and mentor feedback before their public performance.

2020 COVID-19 Adaptations

Programs across the nation faced similar challenges of accessibility, online fatigue, and keeping lessons fun and different from online school. Using the adaptations described below encouraged authentic learning and enjoyment of the summer programs in 2020. While some programs focused on pandemic-related science (Sadler 2020), the programs described here focused on environmental health and social justice.

While creating virtual adaptations for programming in 2020, some features typical of face-to-face programming were retained to engage students and improve their learning. These include internships; independent scientific discovery; environmental health science education; a seminar approach; developing academic and professional skills; mentoring, especially from university faculty and staff of similar cultural backgrounds; prompts for reflective learning (Thorpe 2004); and final projects. Student final projects are developed through a series of weekly mini assignments. Thus, student success is enhanced using a Zone of Proximal Development or scaffold approach (Billings 2018).

The program adaptations during 2020 occurred in four distinct categories, described for each program in Table 4. The categories include (1) encouraging peer-to-peer interactions, (2) reducing social barriers, (3) providing virtual mentorship, and (4) using time and energy effectively. A blended approach to STEM learning, including reading and writing literacy and academic skills, proved effective by assisting students from non-dominant cultures in understanding the importance of STEM within environmental and social circumstances.

One key effect of transforming in-person programming to virtual formats was program staff needing to make adaptations. They developed the changes by doing daily evaluations of the programs and collaborating with each other to address the challenges created by the pandemic. One example of adaptation was staggering the program schedules to accommodate collaboration among the programs. Because staff members were constantly learning what worked, many adaptations developed spontaneously. The following sections include adaptation details and impacts collected via insights and feedback from students, program staff, and collaborating partners.

Peer to Peer Interactions

The virtual programs all incorporated peer-to-peer interaction to facilitate public speaking, self- advocacy, friendships, and networking skills development. These interactions also promote community building and allow participants to do low-stakes practice prior to experiences with researchers, professionals in respective fields, and final presentations. The following strategies produced good results in the online format.

• Popcorning – promoting group discussion by having a student share and then call on another student to communicate their point of view or ideas. This allowed students to practice speaking in front of others while building comradery with classmates. This method of discussion promoted diversity of thought and collaboration among peers, and was a way to ensure students were engaged with the material. It also promoted getting to know others and learning names (see Popcorning image in Appendix A).

• Planned Competition – any activity where students must compete with each other, either individually or within small teams. It took many forms such as quiz games, Jeopardy, or Bingo. In one program, the final projects took the form of “shark tank” presentations and a panel of judges awarded a prize to the winning project. In many competitive activities, students communicated, worked as a team to accomplish a common goal, and ultimately seemed more motivated and engaged (see Planned Competition image in Appendix A).

• Breakout Rooms – facilitated small group discussions. Students were able to work effectively as a team with minimal assistance from program staff. They engaged with each other, built relationships, and developed their skills for working in groups. It was helpful to assign roles to each group member such as group leader, note taker, or speaker to ensure quality interactions. In larger programs, breakout groups helped participants feel a sense of community (see Breakout Rooms image in Appendix A).

Reducing Social Barriers

Because many of the students in these programs had difficulties with online programming, staff sought creative ways to reduce the barriers for students’ participation. These adaptations were critical to students’ experiences and program success. Students often were reticent to inform programs about their barriers; therefore, program staff sought out each student individually to discuss how their needs might be met. With many students living in rural areas, two large barriers included inadequate internet connectivity and a lack of computers that could use all Zoom features. Strategies to address barriers included:

• Laptops – If the student did not have access to an adequate computer, the program helped them obtain one through the college, the collaborating department, or the school district.
• Hotspots – The university was able to lend mobile hot spots to students who did not have access to the internet.
• Packets – Program staff found that the best way to create engaging virtual programs was to send packets with materials and tools students would need to complete hands-on activities. Students were encouraged to develop hypotheses and to work with materials in novel ways to answer questions they developed. Program staff distributed packets by delivering at drive-through pop-ups, sending through the mail, meeting students and parents at convenient and socially distant spaces, and even hand delivering to students’ doorsteps (see Packets image in Appendix A).

Virtual Programming

Staff designed the programs to maintain the important element of giving students individualized help with assignments, professional writing, and skills development that had been part of the in-person programming. This level of mentorship was vital for students to gain confidence in their ability to perform efficiently in academic and professional settings.

Virtual programming mentorship and interviews became especially important to help facilitate hands-on practice and interactions. The fun social events and informal conversations between activities or while moving across campus all but disappeared in the virtual programs, so adaptations were made (see Virtual Programming image in Appendix A).

• Office Hours – During a Zoom session, staff members announced their availability outside of program hours. At any point during Zoom office hours students could log on to ask questions and engage in reflection discussions about the material. Office hours allowed interaction with the instructional team and fostered mentoring relationships as well as preparing students for university attendance. Virtual mentoring also encouraged students to practice self-advocacy, creating a comfortable space for students to ask questions or express concerns they were too shy to do in group settings (see Office Hours image in Appendix A).

Time Use

Nearly everyone experienced “Zoom fatigue” (see Figure 5) or struggled to stay engaged with virtual presentations and activities. While it was a challenge to get students to interact, it was even harder to help them stay attentive in the virtual format. The following adaptations helped maximize student engagement without making them feel overworked or overwhelmed.

• Short Days – Students noted that it was tiring to pay attention in virtual classrooms. Programs found it beneficial to shorten the daily schedule to maintain student focus, prevent boredom, and allow students time to work on their own or use office hours. Shortening days helped everyone deal with mental and physical fatigue (see Short Days image in Appendix A).
• Breaks – Virtual programs required more frequent breaks than in-person ones. Frequent breaks allowed students to step away from their computers and regain their focus. Students opted to get a snack, step outside, close their eyes, etc. Some programs enhanced break times with guided stretching or breathing exercises, encouraging students to continue with the next activities (see Breaks image in Appendix A).
• Alternate Days – Similar to shortening the duration of program days, alternating days was an effective adaptation for some programs. This schedule gave students a mental break so they could return fresh and focused on the next day of programming. One program that is normally one week long became a two-week virtual program that operated Monday, Wednesday, and Friday.
• Self-Paced Working – In conjunction with shortening or alternating days, incorporating self-paced content allowed students to work individually while still engaging with the program’s content. This was ideal in a virtual setting because students could complete the work on their own schedule and take breaks as needed without feeling overworked. One program employed a morning of virtual programming and self-paced work in the afternoon. This strategy was particularly helpful to students in rural communities whose internet availability was limited later in the day (see Self-Paced Working image in Appendix A).

While all these adaptations were helpful, some topics and activities did not lend themselves easily to virtual interaction among participants. To combat this issue, staff rethought activities to make them more interactive. For example, using activity packets (see Figure 6) began after staff wondered, “How are we going to keep students busy and engaged while looking at a computer screen?”

Program staff also found that, no matter how hard they tried, they could not make every piece of programming highly engaging; however, they could mix things up by

• shortening the more passive components,
• sandwiching other components between highly active and engaging ones, and
• scheduling an active break in the middle of a less-palatable session.

The “popcorn” strategy shifted discussions from passive listening to active and engaged listening.

Evaluation Method

All programs are evaluated formatively in the moment, at benchmarks within the session, and summatively at program end. Staff gather a wide variety of materials, documents, and observations that reveal to what extent and how participants achieve the goals. Written and spoken reflections provide rich documentation (see Appendix A). Prompts for written reflections elicit specific examples of participants’ experiences and explanation of how those experiences increased their knowledge, self-efficacy, and abilities.

Spoken reflections include short debriefs following presentations and immediate group discussions following activities. Final presentations reveal participants’ level of self-confidence, as well as their knowledge, as they discuss how program experiences have impacted them, their sense of accomplishment, and their desire to pursue higher education. Electronic satisfaction and impact surveys at the end of each program ask participants to rate activities and experiences and suggest improvements for every aspect of the program. Viewing the successful presentations (see Figure 7) also demonstrated the impact of the programs on students of all ages and backgrounds.

During post-program debrief sessions, staff used SWOT analysis (Strengths, Weaknesses, Opportunities and Threats) to collaboratively identify student overall achievement and plan future development of the programs. Table 5 presents these outcomes.

Strengths identified the focus on “what really matters” to program participants. Weaknesses focused on the difficulty of doing purely fun, social activities online. The Opportunities identified that the adaptations could be used during in-person programming, particularly using virtual tools to introduce and communicate with experts and mentors who cannot be physically present. Threats were primarily those that everyone experienced during the pandemic: Zoom fatigue and limited social interaction.

Outcomes and Feedback From Students

The most common feedback participants offered during Summer 2020 programs involved gratitude that the program was held, even if it had to be done online. Exit surveys in 2020 show that the vast majority of participants (above 90%) agreed that the program increased their science knowledge and their ability to do science. Interestingly, a high majority (nearing 100%) agreed that their 2020 program experience improved their ability to learn using online environments, an unexpected and welcome outcome. Perhaps most surprising, the majority of participants (73%) whose hands-on, in-person lab internship was shifted completely to an online experience agreed that, if the opportunity became available, they would do another “virtual” STEM program.

The 2020 program participants valued who they met as much or more than what activities they did. Online discussions with world-class researchers, public health experts, and undergraduate and graduate students working in research labs consistently earned strongly positive feedback. One student indicated that the student panel “answered lots of questions and were able to give us tips for when we [are] in college.” Other participants noted that learning about career options, science, and research fields, as well as “the social life behind the research field” gave them an understanding of what those experiences might be like in the future.

Participants from underrepresented groups commented on the impact of learning about environmental justice issues from “one of our own people” and to hear firsthand about their work in science and health fields. Participants often described “inspiring” activities in which mentors shared their personal histories, journeys to their current position, as well as their passion for their work. Table 6 presents sample student comments.

Going Back: Returning to In-Person Formats

In 2020 every program was forced to find a way to be successful in a virtual format. While it was difficult at first and challenges remained, it did not take long to get accustomed to the new lifestyle. Some students preferred this new routine because it offered convenience and a more self-dictated pace. However, it was also a way to hide behind a camera, an excuse not to participate, and a crutch in social situations.

These negative consequences created unique challenges when some programs—Toxic Detectives, Step2STEM, Environmental Scholars, and A Student’s Journey—returned to an in-person format in 2021. For example, staff noted students were shyer than they had been in previous years, taking a couple of weeks to gain pre-pandemic social confidence. Lunchtime is usually a time for students to interact and tell stories about their experiences, but in one program staff noticed that students sat quietly, often seeking tables to eat alone. Staff’s purposeful efforts to initiate conversation at lunch was helpful. Purposeful use of icebreakers throughout each day was helpful for bringing students together. Competition was another useful adaptation from virtual programming that was useful for the in-person environment. These strategies facilitated social interaction, breaking down the social walls that students had built up while learning online at home.

Hybrid formats, which had students come in person some days and attend virtually on others, was not a productive way to structure days. Sometimes, students would be confused about the schedule and where they needed to be. Upon observation of student performance, staff members reflected that in-person days were often the more productive days and that students treated virtual days as a break.

Occasionally changing the classroom location helped keep students focused: using a different classroom if available, going on a tour with a guest speaker, or doing activities outside if weather permitted. The change of scenery seemed to motivate students, increase their interest, and keep their focus. Overall, when going from a virtual format to an in-person format, it was important to be even more intentional in facilitating social interactions, committing to a full in-person format, and being purposeful about using multiple settings.

Weaknesses that were addressed included limited and difficult social interactions, virtual tours not giving students a real on-campus experience, and mental and physical exhaustion. Results of the adaptations proved successful enough in overcoming the challenges so that there were few dropouts and students were able to complete the expected work.

Recommendations

In 2020 we found that the community partners and students appreciated that the six programs were not cancelled. In 2021 returning to in-person programming provided new insights into best practices for summer STEM programs. In both years, program staff had to think on their feet and make adaptations on the fly. The summative program evaluations each year led to ongoing program improvement, while the all-staff debriefing with the external evaluator led to changes for the future.

Below are recommendations for both going to virtual programming and then going back to an in-person format. They serve as guidance for pathway development over middle school to junior college, providing a pipeline to STEM education and careers.

Going There: Recommendations for Virtual Programming

1. Review program elements and ensure intentionality in program activities.
2. Build in and value informal social time. Downtime and casual conversation often yield the most effective mentoring.
3. Develop more independent, hands-on, and active individual learning. Self-directed, individual learning can include, for example, preprogram self-directed activities or postprogram community engagement.
4. Silence does not always mean reticence; sometimes it is the sound of thinking. Give students time and space to consider and formulate responses and questions.
5. Provide frequent breaks for students to ensure they do not get burnt out or fatigued. Program staff need breaks, too.
6. Use competitive group activities to keep students engaged with programming material.
7. Adapt to the ways students best participate. Use the chat feature in Zoom for students who do not feel comfortable or confident speaking up or asking questions on camera.

Going Back: Recommendations for In-Person Programming

1. Encourage social interactions by having more icebreakers and structured discussions.
2. Avoid hybrid formatting that rotates in-person and virtual days and commit to one format.
3. Include different learning settings, using different classrooms, meeting outside.
4. Provide frequent breaks to help everyone rest their brains, shift their bodies, and regain focus.

Conclusions and Lessons Learned

Students and staff were able to shift to virtual activities and gain new knowledge, develop new science and academic skills, express positive self-confidence, and view themselves as university students and scientists. Table 7 summarizes how the programs benefited from evaluation and reflection to improve going forward, creating a pipeline of programming to interest students—especially non-dominant culture students—in STEM and environmental health.

It was exciting to see the STEM showcases at the end of each program, where students presented about environmental health topics they studied. Done virtually, programs gained an impressive record of student outcomes.

Marti Lindsey is the Retired Community Engagement Director and founder of the KEYS High School Research Internship at University of Arizona Medical Center, College of Pharmacy in Tucson, Arizona. Ben Richmond, MPH, is Associate Director, Community Engagement Core, founder of Steps 2 STEM, Alex Benavides is Program Coordinator, Senior, Coordinator Students Journey, Graciela "Zonnie" Olivas, Tribal Affiliation: Navajo (Diné), is Coordinator of Our Lands, Education and Health, Program Coordinator, Leesa Lyons is Assistant Program Coordinator, Coordinator of Environmental Scholars, Lexy Havunen is Assistant Program Coordinator, Coordinator Toxic Detectives, Mayra Vargas is Program Coordinator, Coordinator of Steps 2 STEM, all at the Southwest Environmental Health Sciences Center, University of Arizona Medical Center, College of Pharmacy in Tucson, Arizona. Kelle Hyland is Program Manager, Outreach and Engagement, Coordinator, KEYS High School Student Research Internship, BIO5 Institute, at the University of Arizona. Brook Moreno is the Title III Project Manager and past Coordinator of the KEYS Program. Bob Mittan is President, RKKM Consulting, LLC, External Evaluator in Tucson, Arizona.

citation: Lindsey, M. , B. Richmond, A. Benavides, G. Olivas, K. Hyland, L. Lyons, L. Havunen, M. Vargas, B. Moreno, and B. Mittan. 2022. There and back: How a global pandemic shaped youth programming. Connected Science Learning 4 (6). https://www.nsta.org/connected-science-learning/connected-science-learning-november-december-2022/there-and-back-how