“Humans are conscious creatures; in fact, philosophers have argued that the ability to reflect and take perspective is what makes humans unique.” —Introduction to “Reflection,” Clover Model

Reflection is an essential social-emotional strategy and 21st-century skill in the field of science, technology, engineering, and mathematics (STEM). For educators, focused reflection, self-evaluation, and continuous improvement can be aided by standardized classroom observations and feedback sessions (Gitomer 2021). When reflecting on their teaching, educators engage their STEM skills to collect and analyze evidence to determine whether their students are learning and using STEM concepts and practices. Teachers also use social-emotional skills to understand each student’s interests and needs, consider ways to make STEM accessible and personally relevant, and determine how best to provide what their students need to progress in STEM.

For over a decade, the Dimensions of Success (DoS) quality framework, observation tool, and feedback system have facilitated this reflection process for out-of-school-time (OST) STEM program staff. DoS-OST was initially introduced by Dr. Gil Noam and The PEAR Institute: Partnerships in Education and Resilience at McLean Hospital and Harvard Medical School (now named the Institute for the Study of Resilience and Youth (ISRY), a teaching and research unit of Harvard that is separate from PEAR, Inc., the educational services benefit corporation that spun off from McLean in 2020). DoS-OST was developed to address the challenges of defining quality STEM learning and identifying observable indicators of social-emotional development (SED) in STEM programs (Shah et al. 2018). As the reach and impact of the DoS framework increased, our team received requests from various stakeholders to expand the tool into formal K–12 settings and align it with current educational standards (e.g., Next Generation Science Standards) and advancements in research and practice. In addition to its continued use for informal settings, DoS is now being redesigned for use in formal school settings to support middle school classroom teachers in identifying their strengths and opportunities for growth through an integrated lens of STEM and SED, an approach that does not currently exist in any other validated observation framework.  

In this article we include four scenarios that illustrate how SED can be integrated in middle-school STEM classrooms, based on real, anonymized observations conducted by our research team. We will discuss these vignettes using the Clover Model (Noam and Triggs 2018) to demonstrate how the DoS tool can frame feedback to teachers about how they incorporate social-emotional principles into high-quality STEM learning. This will provide teachers with concrete strategies to integrate SED and STEM in their classrooms and demonstrate the value of this pedagogical approach: Engaging in authentic STEM practices provides opportunities for students to build social-emotional skills, and fostering SED enhances STEM learning. For clarity, we use the term SED to emphasize the growth in social-emotional skills that occurs through social-emotional learning (SEL) over time, with different skills prioritized at different ages and in different social contexts (home, school, work). For continuity and conciseness in this article, the term SED is meant to encompass SEL, social, emotional, and academic development (SEAD), and the 21st-century skills that today’s children need to succeed now and in the future.

Dimensions of Success—Observing SED and STEM During and Outside School Time

The DoS framework includes an observation tool organized into 12 dimensions of quality across four domains (Figure 1). It is used by certified observers who record field notes about the student and teacher/facilitator interactions they see and hear during a STEM learning experience. Then, observers score each dimension on a 4-point scale, from 1 (evidence absent) to 4 (compelling evidence). Quantitative scores help educators, program staff, coaches, or administrators aggregate results and identify trends in quality. Qualitative details describing teachers’ and students’ interactions during STEM activities provide specific examples for teachers to reflect on their practice. For example, one middle school science teacher stated, “I appreciate the specificity of the feedback—concrete ideas and small tweaks.”

Training in the DoS-OST framework and use of the DoS-OST tool is currently available via PEAR Inc. Completing a certification process is required to reliably perform DoS-OST observations in the field. People who are interested in receiving training or becoming certified in using the DoS-OST tool can enroll on PEAR’s website.

Since the DoS-OST framework and observation tool was validated (Shah et al. 2018), it has become the standard tool for observing OST STEM teaching and learning in informal settings, such as museums, summer camps, and afterschool programs. In addition to established psychometric properties, our research demonstrates that STEM program quality, measured using DoS-OST, is positively associated with youth STEM-related outcomes, including STEM engagement, career interest and knowledge, and identity (Allen et al. 2019). Studies also demonstrate how evidence and feedback from the DoS-OST tool support continuous improvement (e.g., identifying persistently challenging dimensions for which professional development may be beneficial) (Browne, Allen, and Noam 2021) and provide a common language for discussing quality STEM learning (Allen, Lewis-Warner, and Noam 2019).

DoS is now entering an exciting period of transformation. With funding from the National Science Foundation, our team, in collaboration with stakeholder groups, is updating the DoS tool for middle-school science and engineering classrooms. After the DoS-MSSE tool is validated, our team will offer free training on the DoS-MSSE framework to 500 people, including classroom teachers, instructional coaches, administrators, or researchers/evaluators. Open-access information about DoS-MSSE is currently available on ISRY’s website, including a summary of our collaborative project activities and an interest form for readers to request more information about the project and training opportunities. Future resources will be added as they are published.

The Clover Model — Mapping Elements of SED to Quality STEM Learning

The Clover Model, also known as a research-based Developmental Process Theory (DPT), was used to inform on SED in the development of DoS-OST and DoS-MSSE (Noam and Triggs 2018; Noam and Shah 2014). This model consists of four leaves, or domains: Active Engagement, Assertiveness, Belonging, and Reflection (Figure 2). At different times, youth may “specialize” in one domain, but all four areas are important for balance and positive youth development (Noam and Triggs 2018). Ideally, productive OST and school-based STEM learning strengthens all four leaves. Although some teachers may focus their instruction on one or two domains, DoS-MSSE feedback is aligned to all four Clover domains, with the aim of supporting a balanced classroom where fun, productive, and deep learning of all STEM phenomena is part of the curriculum.

DoS was created using the lens of the Clover Model to define indicators, or evidence, that allow observers to determine if SED is incorporated into a STEM lesson. As we update DoS, we are sharpening our definitions of the 12 dimensions of quality STEM learning as they relate to SED and more explicitly incorporate elements of DEIA. Table 1 maps the 12 dimensions of the revised DoS-MSSE tool to the Clover Model’s four SED domains. Four vignettes—based on real, anonymized observations of middle school classrooms—are provided to illustrate how the DoS tool strengthens STEM learning experiences while also supporting SED. Although multiple DoS dimensions map to each Clover domain, this article includes only a few for brevity.

Active Engagement

The first domain of the Clover Model to which the DoS tool maps is Active Engagement. This domain involves connecting the physical body to the world through movement and experiential learning (Noam and Shah 2014; Noam and Triggs 2018). In middle school, students develop their identity in relation to their physical abilities and begin to assert themselves while moving throughout the world (Noam and Shah 2014; Noam and Triggs 2018). Two dimensions in the DoS tool that integrate elements of Active Engagement are Space Utilization and Engagement with STEM.

Consider the following scenario:

Students in Ms. G.’s sixth-grade life science class are learning about interactions among organisms in a pond environment. To identify interdependent relationships among animals, plants, and fungi, students work in small groups to create their own food web model using pre-labeled cards. For the lesson introduction, students are seated in rows facing Ms. G. and a projector screen. Before students begin the modeling activity, Ms. G. asks them to turn their desks to form groups of four. Each student stands around their desks and places the cards in the food web model to identify an organism and its relation to others in the pond. Students discuss the activity in small groups and discuss with one another where each card should be placed and the reasons why.

In this scenario, Ms. G. demonstrates how adjusting the physical space in a classroom supports her students’ Active Engagement during the modeling activity. An observer using the DoS tool in this classroom might highlight this moment as a strength under the Space Utilization dimension. By rearranging desks from rows to small groups, Ms. G. encourages students to explore the food web cards in a hands-on way. Students are also better able to discuss the activity and collaborate with peers because they are facing each other.

The Engagement with STEM dimension would also be highlighted as a strength for Ms. G. Through her facilitation, Ms. G. ensures that students can move their bodies in a way that supported their Active Engagement while learning a life science concept. Students are physically engaged in the activity through their hands-on interaction with the cards and by moving around the desks to view the card web. They are also cognitively engaged in a small-group discussion about where to place the cards in the food web model. In this activity, students work together by asking questions and making suggestions. They are developing their identities as “scientists” who collaborate to explain phenomena (i.e., how energy flows in a food web).

Assertiveness

The next domain of the Clover Model, Assertiveness, involves expressing ideas (voice) and making decisions (choice). In middle school, it is particularly important for students to feel that their voices are heard and that they can make decisions for themselves (Noam and Shah 2014; Noam and Triggs 2018). Assertiveness is integrated within the Authentic STEM Practices and Youth Voice dimensions of the DoS tool.

Consider the following scenario:

Students in Mr. T.’s eighth-grade engineering class are building three types of model bridges to discover how and where tension and compression forces are acting. In groups of four, students brainstorm and design their bridge models based on established needs and constraints. Then, students move into the construction phase. Mr. T. reminds students about the requirements and limitations of the bridge designs and gives students freedom to sketch and build their ideas. Midway through the activity, Mr. T. has one member of each group examine their peers’ designs and report back on something they might want to incorporate into their design. After constructing their bridges, students test their designs using incremental weight increases. Mr. T. then encourages all groups to discuss why they think their bridge failed or passed the test and to revise their design for adjustments or rebuilding.

Mr. T. uses an engineering design challenge that encourages students to reason and make decisions, providing a strong example of the DoS Authentic STEM Practices dimension. Although Mr. T. guides his students through the engineering design process of brainstorming, designing, testing, redesigning, and retesting, the students also engage in critical thinking and decision making. Mr. T. encourages the first aspect of Assertiveness, student choice, and allows students to decide how they design and redesign their bridges. This scaffolded support builds students’ confidence in their ability to make decisions (Chen and Terada 2021; NRC 2012).

At multiple points during this activity, Mr. T. also encourages student voice, the second aspect of Assertiveness. In the DoS tool, “voice” is metaphorical through the expression of ideas and can be verbal or nonverbal. For example, Mr. T. has students share their ideas verbally about their initial design, their peers’ designs, and potential redesigns for their bridges. Additionally, students share their ideas on paper by sketching their designs. These strategies might be highlighted as strengths under the Youth Voice dimension. By promoting a student-centered classroom, Mr. T. is listening to their voices and allowing his students to lead, while helping them learn to balance the opinions and ideas of their classmates (Laux 2018).

Belonging

The Belonging domain of the Clover Model involves building strong relationships with peers and adults, group acceptance, and group identity (Noam and Shah 2014; Noam and Triggs 2018). In middle school, students recognize their commonalities with one another and connect with classmates and teachers about shared experiences (Noam and Shah 2014; Noam and Triggs 2018). Belonging is integrated within the Access and Relationship dimensions of the DoS tool.

Consider the following scenario:

Mrs. A.’s seventh-grade engineering class is building a tower that can support the weight of a stuffed animal tiger, their school’s mascot, for display in the school’s lobby. To build their tower, each group of students is given 50 index cards, one foot of tape, scissors, and a ruler (Engineering is Elementary 2012). While the groups build, Mrs. A. notices a group of three students struggle to work together. One female student is not physically handling the materials because the other two male students dominate the construction of the tower. Mrs. A. stops the group and says, “I see that only two of you are handling materials. What is each person’s role in constructing the model?” The students then assign roles with one another. Mrs. A. reminds them, “Make sure you are communicating with your teammates, being respectful, and sharing materials and roles. It’s okay if you don’t finish or if your prototypes are not successful while testing. It is more important that I see you using good teamwork skills.”

In this scenario, Mrs. A. notices inequity (two students dominating the activity, limiting the third student’s engagement) and addresses it during the lesson to ensure all students participate in the tower building exercise. This moment might be highlighted as a strength under the Access dimension. By addressing this moment of inequity in student participation, Mrs. A. builds a sense of belonging for this student and the entire class. She shows the female student that her participation is important and reinforces the expectations for respectful collaboration among all students by having the group work together to assign roles. Thus, Mrs. A. makes it clear that everyone is expected to participate in the activity, while providing scaffolding when needed.

For students who are not inherently interested in STEM or who have experienced exclusion in STEM, building a sense of belonging can stimulate their engagement in STEM inside and outside the classroom (Mulvey et al. 2022). National efforts—such as the U.S. Department of Education’s YOU Belong in STEM and the Beyond 100K initiative—are also calling attention to the importance of building belonging in a STEM classroom to support equity and inclusion in STEM. In this example, Mrs. A. helps each student identify their “role” in the group, which shows each student that they belong in this STEM classroom now, a connection that ideally will continue in their future classrooms and careers.

Mrs. A. also fosters mutual respect and trust that encourages students to share their ideas and participate in STEM practices. Under the Relationship dimension in the DoS tool, this is a way of building belonging. For example, Mrs. A. gently reminds her students how to collaborate effectively and be respectful when disagreeing with peers. Through these actions, Mrs. A. tells all students that their participation in the STEM activity is valuable and helps them feel connected to their learning and peers. Over time, this sense of belonging may contribute to her students’ continued participation and interest in STEM.

Reflection

The final Clover domain, Reflection, involves making sense of one’s experiences, thoughts, emotions, and the larger world. This allows students to think introspectively and take on the perspectives of others (Noam and Shah 2014; Noam and Triggs 2018). In middle school, students reflect on their own thoughts, emotions, and actions and begin to understand how their insights can impact the people around them. Reflection is integrated into the STEM Concepts, Reflection, and Relevance dimensions of the DoS tool.

Consider the following scenario:

Students in Mr. M.’s eighth-grade engineering class have been growing plants in their school garden for several weeks. The week before Spring Break, students are asked to design a device to water the plants while they are away (Engineering is Elementary, n.d.). Mr. M. has students brainstorm their designs for such a device, first verbally in pairs and then by writing on a class idea board. During a whole-group discussion, one student states that monetary cost will probably be a factor in what materials they select. Mr. M. then asks students to discuss and draw in small groups what an affordable device might look like. Mr. M. prompts students by asking, “Think of what materials may be used to drip water slowly over time but not so slow that the plants dry out if it gets too hot.” One student responds, “I think something like a water bottle with a hole in the cap would let water flow through slowly if it were turned upside down. I’ve seen my neighbor do something like this in his garden over the summer.” This introductory discussion leads students to research and develop their own prototypes using household materials.

By grounding this unit in a question that is relevant for students, Mr. M. helps them connect to the world around them. He starts by asking students to think about their school garden, reflect on how to water the plants slowly, and then consider how to construct a device to solve this problem. These strategies are examples of the Relevance dimension because students are connecting concepts and practices to their lives or broader context to promote personal meaning-making and supporting identity. In middle school, when youth development is focused on building strong relationships with peers and adults, it is particularly important for students to recognize that they can impact and influence others by making connections with their classmates and teachers (Noam and Shah 2014; Noam and Triggs 2018). In this scenario, students are guided to consider the potential real-world impact of the concepts and practices they are learning, which may increase their motivation to solve engineering design challenges in future lessons. As one middle-school teacher said, “Every kid should be able to have their own ‘why’ at some point to latch onto, but then come to a consensus that we are trying to figure something out as a class.”

In addition to having students reflect on the personal relevance of building a slow-release watering device, Mr. M. has students reflect on the properties of materials and the importance of selecting appropriate materials for different designs. Mr. M. then allows his students to share their reflections in multiple formats (e.g., writing/drawing, in pairs/small groups, in large-group discussion). This prompt from Mr. M., and the responses from his students, might be highlighted as a strength under the Reflection and STEM Concepts dimensions in DoS. Reflection involves students making their thoughts “public” and sharing their ideas, opinions, and questions. By encouraging multiple moments of reflection, Mr. M. is teaching his students how to communicate their thoughts to others.

Our Efforts Toward Continuous Improvement for All

In this article we provide educators with examples of how their colleagues across the country are integrating SED and STEM in their middle-school classrooms. By sharing these scenarios in the context of the DoS framework, observation tool, and feedback system, we hope readers will reflect on their teaching and identify ways they may be able to enhance the SED of their students through engaging in high-quality STEM activities. Through this reflection, teachers can consider their own skills, and those of their students, and identify strengths and areas where they might better support young people. When asked about the potential utility of the DoS tool, one middle-school teacher responded, “We’re science teachers. We want the evidence—show me why I got the score I did, and what I can do better.”

As our team continues to refine the middle-school version of the DoS tool, we encourage teachers, STEM coaches, school/district administrators, researchers, and educators to help inform and support our work. In the spirit of collaboration, we seek feedback on ISRY’s website from all stakeholders about your experiences with SED in STEM. We will use this information to ensure that the DoS-MSSE indicators are accurate and meaningful to users, and to develop training materials that address what educators want to know about SED in STEM. We invite educators, practitioners, and policy makers to partner with us as we develop the DoS-MSSE tool—we hope to provide public access to this work, and we will share progress updates and resources to anyone who indicates interest on the ISRY website.

“Science is a human process, and the experiences of scientists can’t be separated [from that]. [DoS] is a STEM tool for coaches, but [takes] into account that learners are humans.” —Science Instructional Coach, Midwestern United States

Acknowledgments

The development of the DoS-MSSE tool is supported by the National Science Foundation (Award #2101554). The opinions, findings, and conclusions or recommendations expressed are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. We thank Ashima M. Shah, PhD; Sara M. Hoots, Ed.M.; and Colleen McDermott, BS, for their careful review of this manuscript. We also thank our teacher participants and advisors for their feedback and thoughtful contributions. We appreciate our school partners and students who have participated in classroom observations throughout this project.

Virginia Andrews is a Research Project Manager, Victoria Oliveira is a Research Assistant, Patricia J. Allen is the Director of Research and an Instructor, and Gil G. Noam is the Director of ISRY and an Associate Professor, all at the Institute for the Study of Resilience in Youth (ISRY) at McLean Hospital and Harvard Medical School in Belmont, Massachusetts. Drew H. Gitomer is the Rose & Nicholas DeMarzo Chair in Education at Rutgers University – Graduate School of Education in New Brunswick, New Jersey.

citation: Andrews, V., V. Oliveira, P.J. Allen, G.G. Noam, and D.H.Gitomer. 2023. Reflecting on STEM classroom experiences: The power of an observation tool with an integrated STEM/SED lens. Connected Science Learning 5 (3). https://www.nsta.org/connected-science-learning/connected-science-learning-may-june-2023/reflecting-stem-classroom