In 2012, the President’s Council of Advisors on Science and Technology reported that the United States would need 1 million additional STEM graduates within the next decade, an increase of 34% annually. Although some progress is evident for increasing the overall number of STEM graduates, too many students do not complete STEM programs they have begun—particularly students from minority and diverse backgrounds. The National Research Council (2012) also reported that the research “clearly shows that research-based instructional strategies are more effective than traditional lecture in improving conceptual knowledge and attitudes about learning” (p. 3). These strategies are often referred to as student-centered learning; however, we use the term evidence-based instructional practices (EBIPs) to emphasize the strong foundation in research demonstrating their effectiveness.

Despite frequent calls for more integration of EBIPs into the classrooms of higher education, the barriers for change are high. Borrego and Henderson (2014) reviewed 191 articles on instructional change, published between 1995 and 2008, and reported their conclusion of four categories of strategies for creating effective instructional change: (a) diffusion (increasing awareness of quality strategies), (b) implementation (putting the theories into practice), (c) teacher reflection, and (d) policy change at all levels of an academic institution to encourage new teaching. At our university, we focused on these four strategies to create a multifaceted approach to the challenge, which would be durable and lasting. This effort led to the creation of STEMFI: the STEM Faculty Institute.

Components and scheduling

Supported by multiyear funding from the National Science Foundation, STEMFI provides diffusion of EBIP strategies by conducting a summer workshop, implementing these strategies through the revision of faculty courses, enabling teacher reflection supported by focused one-to-one mentoring through the year of change, and supporting environmental change through administrative support and policies. In this article, we describe critical factors to the STEMFI approach, then report on the initial effectiveness we have observed with the first cohort.

Administrative leadership support

Lasting instructional change requires initial “buy-in” from university professors and administrators. Initial support for STEMFI was achieved by conducting two focus group interviews with approximately 30 faculty and administrators who provided feedback on the proposed design and implementation. The feedback was used to further refine the curriculum materials and research protocols.

STEMFI was designed primarily for the faculty of the colleges of life science, engineering and technology, and physical and mathematical sciences. The STEMFI team furnished the modified STEMFI documents to each of the college deans and department heads whose units would be involved, as well as the university’s central administration for approval. The ensuing letters of support helped assure faculty that STEMFI was encouraged at all levels of the institution, along with providing documentation of the value of STEMFI that faculty could include in their tenure and promotion process. The deans helped disseminate STEMFI by having each of their department heads send personal notifications encouraging their faculty to enroll.

Administrative support and leadership for STEMFI have been critical to its success. In both STEMFI cohorts to date, there have been more applicants than spaces available. As faculty have been concerned about ways pedagogical change might affect their ability to meet promotion and tenure requirements, this support from all levels of the institution has been valuable.

Focused faculty mentoring

Mentoring has been vital to the success of the STEMFI program. Strong mentoring relationships are essential for faculty to successfully overcome personal and social barriers to student-centered teaching and implement lasting change. Researchers at the Summer Institute for Undergraduate Education in Biology (sponsored by the National Academies, Howard Hughes Medical Institute, and the University of Wisconsin-Madison) concluded from their project that feedback from mentors was necessary for supporting change (Ebert-May et al., 2011; Pfund et al., 2009).

Each STEMFI workshop participant has been assigned a mentor from the same discipline who assists the participant for one academic year with designing and implementing active learning strategies in the participant’s course. The mentors in the first cohort were members of the STEMFI team, along with four senior faculty identified as master teachers. All mentors have gone through an intensive 1-day training prior to the initial workshop through Brigham Young University’s Center for Teaching and Learning.

During the last half of the workshop, mentors are paired with participants to assist with redesigning their courses and learning activities. Mentors continue their support after the workshop by providing one-on-one attention, attending the mentee’s class when an activity is piloted, and meeting with their mentee after the class to review the implementation, including celebrating what went well and strategizing about how to improve.

In addition, mentors have helped their mentee prepare a presentation for one of the monthly cohort meetings to share a new activity they have implemented. The cohort meetings enable participants to reconnect and peer mentor each other. The general cohort meeting structure is a modified consultancy and tuning protocol (Allen & Blythe, 2004), in which the presenter shares concerns and shows a short clip of the EBIP executed in class. The cohort discusses the issues and the presenter concludes with a response to the feedback and final comments.

The selection, training, and role of the mentors during the second year have been improved using feedback from the first cohort. We reimagined the role of the mentor and changed the name to peer-teaching partner (PTP). We nominated exceptional faculty graduates from the first STEMFI cohort to be PTPs for the second cohort, training them during a breakout session during the second summer workshop. These mentors participated in a panel for the new cohort, ate lunch together, and worked with their participants to develop activities. An important component was for the PTPs to attend the culminating workshop activity in which the participants taught a newly created activity to the group. These changes have created more space and time for the participants and the PTPs to build relationships, and both groups have been invested to work together throughout the year.

Course redesign and summer workshop

The summer workshop was intended to address personal factors relevant to pedagogical change, which often begins with redesigning a course. The workshop has promoted learner-centered beliefs by engaging the participants as learners (Loucks-Horsley et al., 2003) and has fostered positive attitudes about EBIPs by sharing a deep understanding of the theoretical underpinnings of student-centered teaching (Prather & Brissenden, 2008). It has also improved self-efficacy by providing opportunities during the workshop and the mentored implementation phase for faculty to experiment, receive feedback, and modify their instruction. To address pedagogical dissatisfaction that “results when one recognizes the mismatch between the stated teaching beliefs, goals, instructional practices, and student learning outcomes” (Gess-Newsome et al., 2003, p. 762), the workshops have provided training for faculty in articulating student learning goals and sharing their personal classroom observation data, as well as time to reflect on the ways their actions might not match their beliefs and goals.

The workshop was also designed to enable participants to learn how to transform their teaching with EBIPs. Instruction has included introducing Bloom’s taxonomy (Anderson & Krathwohl, 2001) and the scientific teaching worksheet (Handelsman et al., 2007) to help participants write better learning outcomes to align their classroom activities with their goals for student learning. A broad range of EBIPs have been incorporated to help faculty design more effective and engaging instruction for their courses: think-pair-share (Allen & Tanner, 2002); paired verbal fluency (Llewellyn, 2013); peer instruction (Crouch & Mazur, 2001); the 5E learning cycle (Bybee, 1993); “I do, we do, you do” (Bowgren & Sever, 2010); spaced retrieval and interleaved practice (Hopkins et al., 2016; Rohrer et al., 2015); jigsaw (Doymus, 2008); one-minute papers (Nilson, 2010); and decision-based learning (Sansom et al., 2019).

The STEMFI summer workshops have consisted of 5 days of teaching, practicing, and developing a course. Table 1 provides a brief outline of the schedule used for the first cohort. In the workshop, the participants have explored evidence supporting student-centered teaching with readings and discussions each day. Every concept or activity has been taught and modeled using EBIPs, followed by participant reflection, so participants have experienced EBIPs while learning the workshop content. Each morning has begun with an EBIP strategy requiring participants to work in small groups to summarize the readings from the night before (e.g., four-sentence summaries, jigsaw).

The final day of the workshop explored assessment strategies, teaching the participants about backward design in education research (Jensen et al., 2017) for assessing their teaching scientifically. Formative assessment strategies such as using classroom response systems (e.g., ABCD cards, clickers; Bruff, 2009), writing exam questions (Tarrant & Ware, 2008), and implementing exit tickets (Dixson & Worrell, 2016) were demonstrated and emphasized as crucial for learning how to gauge student understanding during a unit and modify instruction accordingly.

Summative assessment strategies to help faculty measure student learning at the end of a unit were briefly addressed, including peer social assessment (Panadero & Jonsson, 2013), calibrated peer review (Robinson, 2001), and assessment blueprints (Fowell et al., 1999)—strategies that help faculty visualize the distribution of learning outcomes and levels of thinking (Bloom’s taxonomy) across an assessment. The three-dimensional learning assessment protocol can be used for both formative and summative assessments to ensure that items address core ideas in addition to science and engineering practices and crosscutting concepts (Laverty et al., 2016).

We improved the workshop structure and scope during the second year. We began the 5-day workshop on a Wednesday so that a weekend separated the first 3 days from the last 2. We reduced the number of activities and EBIPs demonstrated and incorporated opportunities for participants to demonstrate their teaching each day of the workshop, culminating in a realistic teaching experience on the last day. This restructuring allowed participants more time to process their own learning and manage the intense experience. Combined with the changes we made in mentoring, our design-based research cycle is helping us iteratively improve the STEMFI experience for faculty.

Data-driven instruction and practice

Part of the STEMFI approach to supporting change among faculty has been helping them better understand their teaching practices and beliefs and recognize through design-based research how the STEMFI intervention has affected these practices and beliefs. To provide participants with formative feedback and gather summative information about the program’s success, we utilized the Classroom Observation Protocol for Undergraduate STEM (COPUS; Smith et al., 2013). This protocol involves trained observers recording behaviors according to a detailed rubric every 2 minutes throughout the class period, giving a code for both student and instructor behaviors. Results of the COPUS instrument give the instructor (and the observer) a quantitative picture of time spent in three main categories: presenting/receiving (a more teacher-centered approach), guiding/having students work (a more student-centered approach), and other (e.g., administrating, waiting, and conducting non-course-related activities).

We observed faculty participants three or four times before they participated in the summer workshop, where we gathered baseline data on their typical teaching approaches. We showed these data to participants on Day 5 of the workshop to help them reflect on their teaching behaviors and recognize areas for improvement. We measured participants four additional times during the semester of reform, uploading successive data to COPUSprofiles.org (Stains et al., 2018) and analyzing for changes in overall teaching behavior. These data were provided to faculty participants and included in a letter of support for rank advancement, as well as used to validate the effectiveness of the STEMFI program.

To ensure that the workshop was responsive to participant needs, in addition to considering the COPUS data, we interviewed every faculty member prior to the workshop to discuss factors that influenced their instructional decisions (see Figure 1 for interview questions). Broadly, these included personal factors (e.g., attitudes, beliefs, self-efficacy), social factors (e.g., pressure from students, colleagues, administration), and environmental factors (e.g., time, resources, student characteristics). Based on the responses, we created a participant profile for each faculty member that included a 2-page summary of the individual’s personal, social, and environmental factors. The purpose of this profile was to create a single narrative for each faculty member about their challenges, beliefs, and goals. For example, some participants might have positive attitudes about EBIPs but fear low student ratings that would influence their tenure or promotion. We also analyzed the interviews to suggest which types of interventions would be most useful or helpful to the interviewees, and we included this information in the profile so the faculty members’ mentors or PTPs could understand more accurately how to support their growth. Each participant was furnished with a profile copy and asked to read it and modify it if needed for accuracy.

The primary purpose of these profiles, as explained, was to assist us in preparing the workshop to meet participants’ needs and to assist mentors in understanding how to support the faculty members. In addition, we did closing interviews with each faculty member and asked questions similar to those from the pre-workshop interview. Comparing these closing interviews to the initial interviews and profiles helped us see patterns in faculty growth through the course of the STEMFI intervention.

Findings on STEMFI success

Results of workshop surveys

Participants completed a survey at the end of the weeklong workshop to provide affective data about their impressions and changes in beliefs, attitudes, and confidence. Figure 2 shows responses to a selected set of questions concerning the overall quality of the workshop. In the second part of the survey, participants reflected on their attitudinal changes. To guide them in doing this, we asked them to compare their impressions prior to the STEMFI workshop with their impressions after having completed it. As these responses were collected simultaneously, reports of prior attitudes may be influenced by current attitudes; however, we consider this adequate for allowing participants to express the change they felt they experienced. Figure 3 shows participants’ reported shifts in motivation, confidence, and support, and Figure 4 shows their shifts in knowledge and skill regarding student-centered practices, resources available, and beliefs about the effectiveness of student-centered teaching. As shown in Figures 3 and 4, on average, participants expressed positive shifts in each of these factors, leading us to conclude that the workshop was successful in accomplishing its goals.

Findings from COPUS profiles

Copusprofiles.org performs a latent profile analysis grouping COPUS observations into clusters defining a particular way of teaching (Stains et al., 2018). For the first STEMFI cohort, the seven resulting clusters were grouped into three broad teaching styles: (a) didactic teaching, defined by 80% or more lecture; (b) interactive lecture, defined as mostly lecture but including student-centered strategies such as clicker questions or group work added to the lecture period; and (c) student-centered teaching, defined as incorporating significant student-centered strategies into large portions of the class. From this cohort, the 14 participants’ pretreatment and posttreatment observations were analyzed, clustered, and compared to assess change. Eight participants made significant shifts toward student-centered teaching, with two moving from solely lecture to interactive lecture, three moving from solely lecture to student-centered teaching, and three moving from interactive lecture to student-centered teaching (see Figure 5). Four participants were already implementing significant student-centered practices; they continued student-centered teaching after STEMFI. Two participants were difficult to profile, having primarily didactic profiles with occasional student-centered practice—both moving to a mix of didactic practice and interactive lecture. Thus, findings indicated that 86% of participants remained in or moved to more student-centered teaching profiles.

Conclusions

In this article, we have discussed the need for and initial design of STEMFI—an initiative to support faculty in significant change toward more evidence-based instructional practices. We explained the key elements of STEMFI, including administrative support, faculty mentoring, a weeklong summer workshop, and data-driven reflection and practice.

Our initial findings after the first cohort were that STEMFI has been successful. Most faculty experienced significant shifts in their teaching toward more student-centered practices. Feedback on the survey after the workshop indicated participants felt the material was helpful and engaging, encouraging a shift in their attitudes toward supporting EBIPs in their classes.

We are actively developing the intervention through several enhancements. First, we are developing a STEMFI workshop to enhance administrators’ understanding of EBIPs, including how EBIPs can be effective within each college and how administrators can support their faculty in adopting these practices. We also plan to develop a self-sustaining interdisciplinary faculty community of practice that can continue processes of peer mentoring, support, and feedback as faculty continue evolving in their EBIP implementation. In addition, we are transforming the role of the peer-teaching partners (i.e., mentors) to support their progression in this crucial role. Finally, we plan to collect resources along with models of effective EBIPs that can be showcased and disseminated throughout the university community via a website and thus shared with the larger STEM community.

Acknowledgments

Funding for this project was provided by the National Science Foundation.

Richard E. West (rickwest@byu.edu) is a professor in the Department of Instructional Psychology and Technology, Jamie L. Jensen is an associate professor in the Department of Biology, Michael Johnson is a teaching and learning consultant with the Center for Teaching and Learning, Jennifer Nielson is the associate dean of the College of Physical and Mathematical Sciences, Rebecca Sansom is an associate teaching professor in the Department of Chemistry and Biochemistry, and Geoffrey Wright is an associate professor in the Department of Technology and Engineering Studies, all at Brigham Young University in Provo, Utah.