COVID-19 forced many informal education programs to quickly adapt by either altering, canceling, or moving their offerings to a remote format. We studied the effectiveness of the transition of one such program: a selective six-week academic immersion program for rising high school seniors with demonstrated aptitude in science, technology, engineering, and mathematics (STEM) disciplines. The program redesign for Summer 2020 aligned with principles of connected learning (Ito et al. 2013; Maul et al. 2017) to preserve student perceptions of research experience authenticity (Burgin 2020; Rowland et al. 2016) and their sense of belonging (Anderson-Butcher and Conroy 2002). We amended our standard program evaluation procedures to collect relevant data from program completers, examining the extent to which connected learning principles can facilitate the transition of an informal education program to a virtual environment, providing students with an authentic research experience and instilling a sense of belonging among participants in a fully remote six-week academic summer camp.

This work differs from that of others who work to connect high school students and scientists remotely, in that the program in question was not originally designed to be delivered remotely. In fact, the program’s strengths were believed to be derived from in-person interaction, as the program came with a rich tradition, spanning decades, of facilitating in-person interaction between students and scientists. Transitioning a program to a remote format in response to an emergent situation is fundamentally different than designing a remote program, and this study speaks specifically to how one program made that transition. We also share lessons learned, which may be applicable to practitioners creating remote programming, whether by design or in response to outside circumstances.           

Review of Literature

Previous studies have argued that learning to do research is “a product of the experiential interaction between person and environment” that is “emergent, adaptive, and unpredictable” (Emo et al. 2015, p. 339). Traditionally, the environment in which students learn to do research relies heavily on in-person interaction between a community of individuals, including developing researchers, their mentors, and other resources (Emo et al. 2015; Barab and Hay 2001), presenting obvious challenges when attempting to reap the same benefits in a virtual context.

The Connected Learning Framework identifies three crucial contexts for learning (peer supported, interest powered, and academically oriented) and three core properties of connected learning experiences (production centered, shared purpose, openly networked) to suggest design principles that inform the intentional connecting of learning environments: everyone can participate, learning happens by doing, challenge is a constant, and everything is interconnected (Ito et al. 2013). Principles of connected learning provide a framework to translate informal education to a remote setting (Maul et al. 2017), in this case by prioritizing the retention of aspects of the learning environment that encourage students to follow their interests, work with their peers, and develop their academic skills in pursuit of a shared purpose of producing novel academic research using open access resources.

While “authenticity” is difficult to define (Rowland et al. 2016; Burgin 2020), obtaining some measure of student perceptions of authenticity is helpful in evaluating the effectiveness of academic research experiences in a remote setting, as a defining feature of such an informal educational experience is its perceived authenticity (Barab and Hay 2001; Markowitz 2004; Hsu and Roth 2010). Another defining feature is the promotion of students’ sense of belonging (Hsu and Roth 2010), which can be measured by Anderson-Butcher and Conroy’s (2002) valid and reliable measure of belonging. Informed by this literature, we describe our methodology for collecting data about student perceptions of authenticity and sense of belonging.

Method

Participants

Forty rising high school seniors enrolled in an academic summer camp (hereafter referred to as “the program”) at a large, state-funded public university in the southeastern United States, participating in synchronous remote instruction during summer 2020 and completing follow-up surveys during the subsequent fall semester.

Participant selection for the program was competitive and based on traditional college-selection criteria. There were 226 highly qualified applicants for 40 positions, with an admission yield of 100% and 100% program completion. Participants were high-achieving students from across the region with demonstrated interest in STEM, with 82% of the students ranking in the top 5% of their high school classes (23 of 28 students whose schools provide class rank). The participants included 24 individuals who identified as Asian/Pacific Islander (60%), 10 who identified as Caucasian (25%), 3 who identified as Hispanic (7.5%), 2 who identified as African American (5%), and 1 who identified as Other/Multiple Races (2.5%). Three participants (7.5%) demonstrated significant financial need and were eligible for supplemental stipends to support their attendance. Participants were intentionally selected for gender parity (i.e., 20 male and 20 female) and all participants were between 16 and 17 years old.

Program Description

In its usual format, the program is inherently experiential and strives to be inclusive. The program has always been accessible for any student to attend, regardless of financial circumstance. Students typically live together in a dormitory with four college-age counselors for six weeks, eating meals together, engaging in social activities and weekend trips, taking academic courses designed specifically for the program, and completing Independent Research Projects (IRPs) in groups of two or more under the supervision of volunteer faculty mentors. In Summer 2020, the residential and research components of the program were cancelled and severely disrupted, respectively, while the academic courses were retained but moved to an online/synchronous format.

In a typical year, students engage in program-specific coursework, research, and enrichment activities, attending in-person academic courses three days per week and working in faculty laboratories two days per week. In Summer 2020, we offered three mornings per week of program-specific academic courses (“Probability and Simulation” or “Mathematical Models and Problem Solving” for mathematics, and “Scientific Computing with Python” for programming) and two days per week of "Open Access Research Seminar" (OARS) via Canvas/Zoom synchronously. Creating OARS involved unprecedented levels of collaboration across and beyond the university, including with the NSF National Ecological Observatory Network (NEON), SciTeens.org, the university’s STEM Libraries, Information and Technology Services, and faculty members from several STEM disciplines (Biological Science, Chemical and Biomedical Engineering, Chemistry and Biochemistry, Medicine, Physics, and Scientific Computing).

Instead of working on ongoing research of faculty members as usual, in 2020 students worked in peer groups of five to design, execute, and communicate the findings for original research questions using open access data sets through OARS, with a variety of structural supports, including customized workshops from NEON and the university’s STEM libraries, and asynchronous mentorship from STEM faculty facilitated through SciTeens.org. The research activities culminated in a Research Symposium, attended by over 110 individuals synchronously via Zoom, including scientists from around the university and recent program alumni, with over 500 additional asynchronous views.

For OARS, students attended two Zoom meetings each week (Tuesday/Thursday 9:45–11:45 a.m.) for a total of four hours of synchronous instruction and activities, along with optional office hours. In general, on Tuesdays the OARS instructor led activities, and on Thursdays there were invited guest speakers from the STEM Libraries. The focus of the first week was Orientation and Introduction to Tableau Public; the second week was Guided Exploration of NEON Data Sets and Communicating Results; the theme of the third week was Developing Independent Research Questions; and weeks four through five focused on execution of a research project, interpretation of results, and preparation and delivery of a final presentation. Students were assigned to groups of five, and each group was assigned a faculty mentor to provide feedback on research question and investigation design, which were principally student-driven. The amount of oversight varied significantly across groups, with some operating more and some operating less independently.

Data Collection and Analysis

Prior to its administration, the university’s Institutional Review Board approved the survey instrument, which consisted of the usual annual program evaluation questions, plus additional items intended to measure students’ perceptions of the extent to which connected learning principles were present, the authenticity of the academic research experience, and their sense of belonging. There was a 97.5% survey response rate among program participants (39 of 40 participants completed the survey) with the survey administered asynchronously via Qualtrics, between 12 and 14 weeks after the program’s conclusion.

To determine the extent to which the remote version of the program embodied the principles of connected learning (Ito et al. 2013), we used the survey instrument developed by Maul et al. (2017) that “elicits information regarding the connections between youths’ interest-related pursuits and academic goals, the involvement of adults as co-participants in pursuits, and youths’ access to technology tools they deem necessary for their pursuits” (p. 1). We did not include the academically oriented principle because Maul et al. (2017) were unable to develop a reliable measure for this principle. That said, the experience of the students was inherently academically oriented by virtue of the program being an academic summer camp. We report on survey results for the other five principles: shared purpose, production centered, openly networked, interest powered, and peer supported.

To evaluate students’ perceptions of the authenticity of their academic research experience, we used the framing developed by Burgin (2020) to create quantitative questions where students ranked the presence of Burgin’s dimensions of authenticity on a Likert scale (“During the program I… (1) gained authentic research experience, (2) engaged in practices likely to be used in professional science, (3) produced results that were significant to others, and (4) carried out an investigation that was meaningful to me;” with multiple-choice options “Strongly agree,” “Somewhat agree,” “Neither agree nor disagree,” “Somewhat disagree,” or “Strongly disagree”). We also created questions where students ranked the extent to which they gained experience in each of the Next Generation Science Standards practices (NGSS Lead States 2013) on the same Likert scale (“During the program I gained experience in… (1) asking scientific questions, (2) developing and using models, (3) planning and carrying out scientific investigations, (4) analyzing and interpreting data, (5) using mathematics and computational thinking, (6) constructing scientific explanations, (7) engaging in argument from evidence, and (8) obtaining, evaluating, and communicating information”). To address students’ perceptions of authenticity, we also elicited open-ended comments mirroring “rose and thorn” procedures developed by Rowland et al. (2016).

Finally, to diagnose students’ sense of belonging, we used Anderson-Butcher and Conroy’s (2002) measure, asking students to use the same Likert scale as above to evaluate the following statements: (1) “I didn’t have many friends at the program,” (2) “I felt comfortable at the program,” (3) “The leaders at the program made me feel wanted and accepted,” (4) “I felt like I was an important member of the program,” (5) “I wish I had not been part of the program,” (6) “I was disliked by kids at the program,” (7) “I was part of the program,” (8) “I was committed to the program,” (9) “I was supported at the program,” and (10) “I was accepted at the program.” We also prompted students to provide open-ended comments in response to the question, “Do you feel like you developed a sense of belonging during the program? Comment on why or why not.”

Results

We reviewed students’ responses to the survey to determine the extent to which the implementation of the program aligned with the principles of connected learning, the extent to which students believed they gained authentic research experience during the program, and the extent to which students felt a sense of belonging.

Connected Learning Principles

Using the survey instrument developed by Maul et al. (2017), we examined how well the students’ perceptions of the program implementation aligned with the principles of connected learning, specifically the shared purpose, production centered, openly networked, interest powered, and peer supported principles. We report relevant results below, to provide context of students’ perspectives, as well as to diagnose the extent to which our implementation abided by the connected learning principles.

Results indicate that there was exceptionally strong agreement among our program participants that they had a shared purpose, with clear goals and ample opportunities for all students to contribute, as at least 86% of respondents agreed or strongly agreed they experienced each of the measures of shared purpose (see Figure 1). While students reported a wide range of frequency of perceived opportunity to think about or create something that would be interesting to others or make a difference in the world (see Figure 2a), the vast majority of students felt they were able to get help from adults, use the tools professionals use, and learn about the ways professionals use tools to create or design things at least weekly (see Figure 2b), indicating that the learning experience of the students was at least somewhat production centered.

Despite working remotely from their homes, all but one to two students felt they had access to the hardware and technology they needed. Further, students reported unanimous agreement that they had the software tools they needed, and all students agreed they were able to get support and feedback from others, demonstrating that the learning experience successfully aligned with the openly networked principle of connected learning.

The experience also proved interest powered, in that all but one student reported having looked for further information related to academic research on their own in the 12 weeks following the end of the program. While the program did not seem to have a marked impact in changing students’ minds about what they want to do in the future (see Figure 3), the vast majority of students knew they wanted to pursue academic research in the future at the start of the program. Even so, most students reported having discovered an interest or talent they did not previously know they had (65% agreed or strongly agreed with this measure, see Figure 3) and having become more curious about things in which they were not previously interested (84% agreed or strongly agreed with this measure, see Figure 3).

Finally, the students reported experiencing strong peer support by receiving helpful suggestions when stuck, getting constructive feedback, being invited to try new things that pushed them to grow, and receiving support from other participants as well as from those managing the program, where at least 90% of students agreed or strongly agreed they experienced each of these peer supported principles of connected learning (see Figure 4). Of the students who responded to the question about who encouraged them to participate in the program, 22 reported being encouraged by a parent (34%), 11 by a mentor (17%), 6 by a brother or sister (9%), and 17 by one or more friends (27%). In addition, all but two students reported developing and improving relationships with an instructor or instructors, program leader or leaders, friend or friends, and research mentor or mentors because of their participation in the program, having good conversations and coming to feel more supported.

Authenticity of Research Experience

All but three students who responded to the survey question about the authenticity of their research experience during the program reported having gained authentic research experience (34 out of 37, or 92%), with all but two students agreeing that they engaged in practices likely to be used in professional science (35 out of 37, or 95%). The vast majority reported that they agreed or strongly agreed that they produced results that were significant to others (84%) and that they believed they carried out an investigation that was meaningful to themselves (81%). At a more granular level, students agreed they gained experience in the practices used in professional science, with at least 89% of participants agreeing or strongly agreeing they experienced each practice: asking scientific questions; developing and using models; planning and carrying out scientific investigations; analyzing and interpreting data; using mathematics and computational thinking; constructing scientific explanations; engaging in argument from evidence; and obtaining, evaluating, and communicating information (see Figure 5).

When asked to provide examples of how they felt the program provided authentic research experience, common themes emerged among students’ free-response answers, including: learning how to use professional tools/software for visualization and data analysis; learning how to use authentic open access data; formulating an independent research question and/or writing a research paper; presenting data analysis and results at a research symposium in front of professionals; learning how to find, read, and/or cite scientific literature; and working together with peers (see Table 1).

Sense of Belonging

All respondents reported feeling comfortable, accepted, committed to, and part of the program, while none reported wishing they had not been part of the program or feeling disliked (see Figure 6). Students varied most in the extent to which they felt they had many friends at the program, but the majority (71%) disagreed that they “didn’t have many friends at the program.”

When asked to comment on whether they felt they developed a sense of belonging during the program, student free-response answers indicated that they “developed friendships with other participants and formed bonds with professors and mentors,” that their “fellow scholars were extremely friendly and accepting,” and several students reported continuing to be in touch with many of the participants over three months after the end of the program. The students gave some specific indications of the things they felt contributed to their sense of belonging, highlighting both platforms/tools (e.g., “Canvas discussion boards, thoughtful Zoom breakout room discussions, and external means of communications in group chats”) and people/attitudes (e.g., “Everyone came together to create a very welcoming environment”), with 74% (26 out of 35 responses) incorporating at least one of these themes. One student summed it up especially nicely, saying, “I, like many, was skeptical about whether a sense of community and belonging would emerge in such an isolated format. However, over time, the involvement of the instructors and counselors broke the ice and allowed for a lot of true friendships to emerge among the students. […] I was thoroughly impressed and appreciative of the opportunity to experience and become a part of such a gifted group of students and mentors.”

There were also students who admitted to feeling more disconnected than they believed they otherwise would have been in-person, with comments such as, “I think some of my difficulties stemmed from the online environment since I was, in a way, limited to the students that had a more active presence in group chats, but I did not share non-academic related interests with them. […] Additionally, I felt a little out of place because of my lack of experience with academic research, competitions and clubs since I don’t have much access to these resources in my community.” The above response provides an example of a phenomenon commonly observed during the program in-person as well, where many students anecdotally display symptoms of or express feelings consistent with “imposter syndrome” (Parkman 2016). While some students attributed such symptoms to the virtual format in their survey responses this year, it is unclear to what extent the virtual format and the struggles are related. Another student reported: “The only part where I feel like I didn’t belong was like the smartness level. Everyone is so talented, takes rigorous courses and participates in the coolest extracurriculars. Compared to what I had at my school I did feel a little less prepared but I think others were in the same boat as me, so that helped me feel better.” Several students also reported struggling with the online format because of natural introversion, with or without reference to symptoms of imposter syndrome.

Discussion

In general, we found that the remote implementation of the program, as experienced and reported by the participants, aligned well with the principles of connected learning. Students reported feeling that they had a shared purpose, and the experience was production centered, openly networked, interest powered, and peer supported. Having established that the principles of connected learning were indeed present in the program implementation leaves us to address the extent to which the program was able to instill a sense of belonging among students and facilitate an authentic academic research experience from the perspective of students.

The survey data strongly suggests the vast majority of participants believe they had an authentic academic research experience, despite the limitations and challenges of the short-notice pivot of the program to a remote format. In their survey responses, the students identified some key ingredients that may have helped boost perceived authenticity, including learning how to use professional tools/software for visualization and data analysis; learning how to use and access authentic open access data; formulating an independent research question and/or writing a research paper; presenting data analysis and results at a research symposium in front of professionals; learning how to find, read, and/or cite scientific literature; and working together with peers. The extent to which these ingredients are necessary and/or sufficient for student perceptions of authenticity is an area for future study, but preliminarily, they do seem to have contributed to the students’ overall positive experiences.

Both the students and the researchers were surprised by the extent to which individuals felt a sense of belonging in the fully remote context of the program, when such a large part of the traditional program relies heavily on physical collocation. That said, this may not be as surprising as we initially thought because there are inherent commonalities between the principles of connected learning, to which the online implementation adhered, and the sense of belonging instrument we used. Anderson-Butcher and Conroy (2002) separate a sense of belonging into three facets: commitment, engagement, and connectedness. Commitment mirrors the shared purpose and interest powered principles through the focus on “perceived importance of one’s work” (Anderson-Butcher and Conroy 2002, p. 859) and “level of interest in the activity” (Anderson-Butcher and Conroy 2002, p. 859), respectively. Engagement is most closely related to the interest powered and academically oriented principles, focusing on student beliefs about whether the activities are “fun and exciting or if youth look forward to learning new things” (Anderson-Butcher and Conroy 2002, p. 859). Finally, connectedness reflects the peer supported and openly networked principles, focusing on student connections with peers and support from adults. A common theme among the open-ended student responses was that the use of tools/platforms to facilitate online communication between students contributed to the sense of belonging. This highlights the importance of the openly networked principle of connected learning, particularly in a virtual format.

Overall, the program was able to create a remote connected learning environment, instill a sense of belonging among the students, and maintain an authentic academic research experience from the perspective of the students, but questions remain that may suggest directions for future research. First, the generalizability of these results may be tempered by the contextual factors of the situation. In the unique circumstances of 2020, most other programs were cancelled, and many students had few other options for either socializing with peers or seeking career development. It is unclear whether the positive results—particularly those regarding a sense of belonging—would persist under more typical circumstances when students had other options. Another question that this work raises is whether and how remote settings might impact students’ experiences with imposter syndrome and/or introversion. While some students attributed their symptoms of imposter syndrome to the virtual environment, we have found anecdotally these symptoms are common in person as well.

While the generalizability of this study is limited by the unique circumstances in which it took place, the results do provide evidence that it is possible for a traditionally residential academic program to pivot to a remote format on short notice by incorporating connected learning principles, and students are able to feel a sense of belonging and perceive that they have been involved with authentic research even in a remote environment. We hope this study might inform other informal education programs faced with similar decisions, or may suggest an alternate model for residential research programs to reach students who are unable to attend in person, especially when due to issues of access and equity.

Implications for Practice

The topic of this study is the original transition of a residential and immersive summer program into a remote format in the year 2020. In 2021 the program was held remotely for a second time, so in this section, we provide insight into what we took away from this study, specifically regarding what we kept the same and what we did differently the following year.

Given student responses about how they felt the 2020 iteration of the program created authentic research experiences, in the second iteration of our implementation, we continued to include structured opportunities for all students to learn how to use professional tools/software for visualization and data analysis, and introduced multiple open access databases. Second, we created additional opportunities and supports for students to find, read, and/or cite scientific literature. Finally, we provided scaffolding for students to produce and present data analysis and results in front of peers and professionals, expanding opportunities for participants to connect with varied professionals.

In both years, we relied on a significant amount of synchronous meeting time (4–6 hours per week), along with a combination of individual and group assignments asynchronously. The assignments were formalized the second year to provide more coherent structure, and each student was responsible for creating a “Datasets, Analysis Tools, and Visualization Software Toolbox,” a “Career Exploration Journal,” and writing a final research abstract. In research groups of five, students submitted “Annotated Bibliography,” “Research Proposal,” and “Final Presentation” assignments.

Because students especially appreciated learning about authentic tools and data sets, we dedicated more time to this in the second year to deliver a more uniform experience for all students before forming longer-term project groups. During the first three weeks of the program, students worked in rotating small groups to survey an array of tools/skills and explore data set options for the final research project. In the second three weeks of the program, students were again assigned to a group of five to do more in-depth analysis on a single topic/data set, learning the skills necessary to design, execute, and communicate data-driven, open access research projects to a broad audience of scientific experts. This was achieved through a combination of support from the OARS course instructor during synchronous class time and through optional meetings with assigned research mentors (i.e., STEM faculty and graduate student volunteers).

The overarching advice we have for others designing virtual research experiences for high school students is not to lose sight of the importance of students’ sense of belonging. We feel that a key element of our success both years was cultivating connections between participants and their peers, between current participants and recent alumni of the program, and between participants and STEM faculty and graduate students. Whether in a face-to-face or remote setting, we have found that imposter syndrome is exacerbated when students do not feel they belong. Providing many opportunities for students to work in groups in and out of class time, recruiting program alumni to participate as assistants in the delivery of instruction (e.g., circulating through breakout rooms during small-group activities, leading segments of whole-class synchronous instruction on specific content in which they have expertise, and answering student questions about their experiences in STEM in higher education), and having past program completers return to synchronously watch student presentations and ask questions created a rich social tapestry where students could see themselves as part of a larger community.

Erica M. Staehling is the Associate Director at the Center for the Advancement of Teaching, Hannah R. Hiester is Co-Director of the Young Scholars Program, and Barbara M. Shoplock is Co-Director of the Young Scholars Program, all at Florida State University in Tallahassee, Florida.

citation: Staehling, E.M., H.R. Hiester, and B.M. Shoplock. 2023. Using connected learning principles to foster authentic STEM research experiences remotely. Connected Science Learning 5 (2). https://www.nsta.org/connected-science-learning/connected-science-learning-april-2023/using-connected-learning