Pacific Northwest National Laboratory created a network focused on collaboration in STEM education to design and deliver projects, resources, and professional learning opportunities in a testbed environment.

How do you uncover and fill gaps in equitable access to high-quality science, technology, engineering, and math (STEM) education offerings in your local region? Where might you deploy strategies to improve STEM workforce preparation and increase public understanding of STEM-oriented issues? And how can you help to ensure that students, educators, parents, and the community are aware of these programs and know how to access them in schools, colleges, and community venues?

If you are the Pacific Northwest National Laboratory (PNNL), you take on the huge goal of designing and implementing an innovative STEM education collaboration project that impacts all levels of local education, both inside and outside of school settings. PNNL is one of the 17 national laboratories funded by the U.S. Department of Energy. Operated by Battelle, PNNL has a vested interest in preparing the next generation of scientists and engineers for their future careers, thus building a STEM-capable workforce and creating a STEM-literate community. One of Battelle’s core principles is a commitment to STEM education and its role in business competitiveness and quality of life. PNNL has been active in STEM education for decades, providing paid internships for future scientists, giving educators in-house lab experiences, and engaging its researchers in STEM outreach activities in classrooms and the community.

The Collaboratory is a relatively recent outcome of Battelle’s longstanding efforts in STEM education. The original Collaboratory planning documents, developed by PNNL’s Office of STEM Education (OSE), state the objective to “design, implement, and mature a local STEM education collaboration zone that highlights the power of PNNL and Battelle to impact the educational ecosystem and serve as a model for amplifying and accelerating progress in addressing our STEM education and workforce challenges” (PNNL 2013). In other words, we (the OSE) sought to create a zone of collaboration in which members codesign and deliver STEM education programs, share findings and lessons learned from their experiences, and comanage and sustain the organization.

We started by identifying possible collaborators located nearby in the largely rural southeast corner of Washington State. Recognizing that our potential collaborators had differing norms, values, and relationships within the community, as well as their own areas of expertise and purpose, we convened representatives from K–12 public and private schools, higher education, community-based learning providers, and local business and industry to brainstorm a unified vision to resolve gaps in local STEM education needs.

Through discussions with these collaborators, we started hunting for gaps where STEM efforts were lacking but a desire to improve existed. We gave ourselves permission to try things out and built a testbed space where we could experiment with new ideas, gather evidence of feasibility, and treat failures as constructive learning opportunities. Through this generative process and with seed funding from Battelle, interorganizational teams now work together, both virtually and in real time, to develop, test, and deploy resources to support student success, educators’ effectiveness, and community engagement in STEM. Thus, the Mid-Columbia STEM Education Collaboratory (Collaboratory) was born.

This is the story of our beginnings: our challenges, our lessons learned, and emerging indicators of success. For those interested in launching an education–business–community STEM learning ecosystem, we share our story.

A Collaborative Framework

We envisioned the Collaboratory as a network of independent entities collaborating to investigate the potential of promising efforts in local STEM education. As described by Wulf (1989), collaboratory is a blending of the words “collaboration” and “laboratory.” We designed an environment where multiple entities could work together to provide a creative space for testing, analyzing, and drawing inferences about new educational practices, ventures, or programs in STEM. Cogburn (2003) further clarifies the concept as “a new networked organizational form that includes social processes, collaboration techniques, formal and informal communication, and agreements on norms, principles, values, and rules.” The scientific perspective inherent in a collaboratory also implies a data-driven, analytical approach to evaluating the impact of the organization itself and the impact of individual programs and projects.

A number of STEM Learning Ecosystems have emerged in recent years (NRC 2014; ODE 2013), and much has been learned about creating a successful regional system (SFN 2017). Our version exemplifies five of the 10 Design Principles recommended by the STEM Ecosystem Initiative, although the others are also represented. The Collaboratory is a good example of:

1. allowing an ecosystem to emerge organically,
2. having a highly engaged lead organization strongly committed to collaborative practice,
3. valuing and respecting the roles of diverse partners across sectors,
4. valuing transparency and the need for data sharing and data-based decision making, and
5. encouraging constant reflection, peer exchange, and constructive feedback among and between cross-sector teams.

Coalitions between formal and informal education providers and business or community can come in many forms. As described by Solomon (2016), types of coalitions generally fall into four basic structures depending on the initiating entity: government top-down, funder/business, higher education, or intermediary. Structurally, the Collaboratory belongs to the funder/business category with the following key features:

• Invites stakeholders to be part of the coalition
• Respects enlightened self-interest
• Has a broad focus, often with workforce outcome goals
• Partners with nonprofits to deliver programming (Solomon 2016, p. 23)

 Establishing the Structure

In late 2012, the OSE, part of PNNL’s External Affairs Division, took the initiative to gather potential participants together to explore interest in establishing a regional Collaboratory. We started by using our connections with the Southeast Washington Leadership and Assistance for Science Education Reform Alliance (LASER); the Yakima Valley/Tri-Cities Mathematics, Engineering, and Science Achievement Center (MESA); and Delta High School (a local STEM-focused school) to form the nucleus of a new organization. We invited representatives from several school districts, local colleges and universities, businesses and industry, and community groups to develop a common vision for STEM education in the Mid-Columbia region. Representatives from 13 entities became the Design Team. The Design Team negotiated high-level goals that all participants could commit to and established an organizational structure that encouraged collaboration among members.

The organizational structure that emerged specifies a seven-member governing Council and two operations teams: Communications & Outreach and Research & Evaluation. Key features of our collaborative framework, specified in more detail in the Collaboratory Business Plan (MCSEC 2014) and Bylaws (MCSEC 2015a), include the following components:

• A membership body, including school and nonschool entities, that commits to shared goals, provides expertise and human and fiscal resources, and cooperates in publicizing Collaboratory efforts and eliciting participation in operational functions and testbed projects.
• A governing council with members from different entities representing both school and nonschool settings, responsible for selecting, funding, and supporting testbed projects and sustaining the Collaboratory.
• Operations teams comprised of individuals from multiple entities, responsible for providing Collaboratory support services and for evaluating Collaboratory progress and impacts.
• Testbed projects led by multientity teams to investigate the viability of new or expanded STEM education activities or programs.
• Explication of values and goals in the form of mission and vision statements, a logic model, and a letter of agreement to join the Collaboratory.
• Procedures, processes, roles, and responsibilities.

The Collaboratory mission is to “amplify, accelerate, and inform innovative and sustainable improvements in STEM education,” whereas its longer-term vision is to produce “students who are inspired and prepared for tomorrow’s challenges through transformative experiences and authentic connections with the community” (MCSEC 2014, p. 19). In signing the Collaboratory letter of agreement, members agree to promote the four overarching goals established by the Collaboratory Design Team. These goals are to:

• enable high standards,
• support excellence,
• spread innovation, and
• meet future workforce needs (Raab 2014).

Progress toward achieving these goals is accomplished through a combination of efforts among Collaboratory operations teams and testbed projects. The funding of STEM education projects, one of three major activities of the Collaboratory, contributes to achieving the following outcomes (direct impacts) outlined in the logic model:

• Local educators and community leaders are more aware of best practices for teaching and learning STEM in schools and the community.
• Educators are using more effective strategies in STEM education.
• Local students, educators, and community members are experiencing higher quality education programs in STEM (MCSEC 2015a).

Consistent with the funder/business structure, backbone functions of the Collaboratory are managed and funded by PNNL as an outreach and community sponsorship effort, while the Council, which manages all aspects of the Collaboratory including selecting and supporting testbed projects, is comprised of six non-PNNL members and one PNNL representative. Thus far, Battelle provides the majority of direct funding for projects and PNNL provides in-kind support by incorporating Collaboratory responsibilities into several PNNL employees’ workloads. Other Collaboratory members offer in-kind support through providing employee salaries for time spent on Collaboratory activities, allowing use of facilities and expertise for Collaboratory projects and meetings, and locally sustaining successful projects (for example, school districts purchase Family Engineering materials, support Code.org professional development, and provide makerspace areas).

The Players

In addition to PNNL, the other founding members of the Collaboratory—Delta High School, LASER, and MESA—are all interested in promoting STEM education in the Mid-Columbia region and beyond and have been successful in doing so within their niche areas.

Delta High School is a public STEM high school in Pasco, Washington. It accepts 120 freshmen each year from three area school districts through a lottery process that any student can enter. One indicator of its success is that its students have exceeded average Washington State assessment results since 2011. In addition, the school has won multiple awards.

LASER, originating at the Smithsonian Science Education Center in Washington, D.C., has developed a wide system of support and resources to help educators successfully integrate STEM subjects into their classrooms. LASER assists districts in identifying and implementing locally appropriate strategies for their student populations (e.g., implementing the Science Resource Center, which provides ready-to-use science modules for teachers in K–8 classrooms throughout southeast Washington). Districts select those strategies that best fit the needs of their students and curricula.

MESA focuses on building pathways to college and careers in STEM fields for students historically underrepresented in these areas. MESA’s work involves outreach that helps students, educators, and families learn about and plan to pursue careers in STEM, focusing especially on students from families where no one has previously pursued any postsecondary education. The organization also seeks to challenge societal stereotypes about the “face of STEM.”

Since the public launch of the Collaboratory in 2013, it has expanded to a total of 17 members. In addition to those described above, member organizations include Washington State University Tri-Cities, Columbia Basin College, Laser Interferometer Gravitational–Wave Observatory (LIGO) Hanford Observatory, REACH Museum, Educational Service District 123, Richland Public Library, Christ the King School, and six Mid-Columbia school districts—Columbia, Finley, Kennewick, Othello, Pasco, and Richland. Membership expands primarily through word-of-mouth. After hearing about, or participating in, a Collaboratory-sponsored event, an individual may want to have a similar event held at his or her organization or wish to help plan and present a similar event. The individual may then pursue joining the Collaboratory through his or her organization.

Projects Inside and Outside of School Settings

The Collaboratory creates and evaluates projects that aim to fill a gap in local STEM education and address the outcomes and impacts as specified in the Collaboratory logic model. Project proposals must “describe the need for the project in light of the current context (community and educational factors that are the current reality).” Going further, the proposal evaluation rubric instructs reviewers to rate the degree to which the proposal clearly states the “environment and conditions creating the need for the project” (MCSEC 2015b, p. 1). In addition to addressing these goals, projects must include a collaborative leadership team.

The first projects selected in 2013 started their work in spring 2014 and expanded their scopes in 2016 with additional funding from the Collaboratory. Two new projects were also taken on by the Collaboratory in 2016. Each Collaboratory project is organized by a diverse team of representatives from formal schooling and business/industry stakeholders, and two include representatives from informal community settings such as libraries and museums. These STEM education projects directly impact students, educators, families, and community professionals (e.g., scientists from PNNL, librarians, locally employed computer scientists). None of the projects would exist without Collaboratory funding, which thus far has ranged from $5,000 to $15,000 per project per year.

Computing the Future works toward ensuring that all Mid-Columbia students develop the computational thinking and fundamental computer science (CS) skills necessary for success in our technology-rich world. Working with Code.org, OSPI, and other formal and informal education groups, the project team seeks to dramatically increase Mid-Columbia participation in CS learning. The project provides professional development and implementation support to educators, CS summer camps for K–12 students from underrepresented populations, and technical expertise to national and state CS policy efforts (e.g., CS framework, standards, teacher certification). It also responds to emerging workforce needs, such as cybersecurity, by helping local schools and colleges develop associated programs. Current Collaboratory organizers come from PNNL, Delta High School, LASER, Pasco and Richland school districts, and Educational Service District #123, with the LIGO joining the project this year.

Experiencing the STEM Spectrum is designed to generate excitement and support for STEM education through authentic, hands-on STEM learning experiences, such as Family Engineering (a hands-on event showcasing engineering design challenges), for students, educators, families, and community members. Their work integrates engineering education with microscopy and materials science to exemplify the Next Generation Science Standards. Events are conducted at local schools, Boys & Girls Clubs, the YMCA, public libraries, and PNNL. Current Collaboratory organizers come from PNNL, Delta High School, LASER, MESA, Finley and Richland school districts, Christ the King School, and Education Service District #123, with Columbia Basin College and the REACH Museum joining the project this year.

STEM Flicks was designed in response to a request from local educators to help students deepen their awareness of local STEM careers and pathways. It was envisioned as a video series created by K–12, higher education, and community partners to bring real-world examples of how STEM workers in our region are finding creative solutions to today’s tough scientific challenges. The videos depict the nature of STEM-oriented teams where each member brings essential expertise to the problem at hand, regardless of age, gender, ethnicity, or educational attainment. Current Collaboratory organizers come from PNNL, LASER, MESA, Pasco School District, Washington State University Tri-Cities, and LIGO.

The Collaboratory’s Makerspace Project has a special focus on underrepresented populations and seeks to spark creative and innovative ideas so that students and their families see STEM fields as viable opportunities. Launched in spring 2016 at Pasco Elementary School and the Richland Public Library, Makerspace has plans to operate in three school districts, the Boys & Girls Club, and the regional library system within the year.

STEM-IT Now, a validation research study based on two prior U.S. Department of Education projects, combines authentic, real-world STEM experiences with targeted professional learning for K–8 educators. The experience consists of two integrated components. The first involves partnering scientists and engineers with elementary classroom teachers, and the second combines those experiences with a full year of professional learning on how to implement integrated STEM education in their classrooms. The first cohort of teachers launched in August 2016, with a second cohort scheduled to begin in August 2017. Participating teachers become local teacher-leaders and trainers on using the materials they have come to know so well. Participants include PNNL, LASER, Columbia, Finley, Othello, Pasco and Richland school districts, and Education Service District #123.

Early Lessons About Collaborating

The anticipated benefits of the collaborative approach, compared to organizations working independently, included sharing specialized expertise, improving coordination of joint activities, receiving constructive feedback from external perspectives, and stimulating the development of new projects. Collaboratory-sponsored projects, with their accompanying organizational supports, were expected to amplify and accelerate their impacts and outcomes in the Mid-Columbia region (and perhaps beyond) because of their collaborative components. Scientific, technical, and organizational expertise are in relatively short supply and often require time-consuming research to identify individuals with specific qualifications. Collaboratory members benefit when contacts and experiences are shared. By working together to coordinate event dates, identify target audiences, and publicize activities, members help one another get the word out and optimize attendance at their educational events. Establishing regular communications channels among members facilitates conveyance of both Collaboratory and individual information. Joint project-team strategy meetings and guided review and critique of project progress provides meaningful feedback to organizers to improve outcomes. And regular sharing of plans, activities, and experiences helps members hone in on areas of need and opportunities to build on earlier efforts of other members.

These benefits were not realized immediately but required persistence and belief in the effort. In its first year, the Collaboratory council had difficulty estimating and allocating sufficient time for fulfilling essential tasks. Members were challenged to add more responsibilities to their already crowded schedules as they sought to balance their “in-kind” contribution of time to the Collaboratory given their existing workloads. Potential conflicts also emerged around licensing and ownership of shared resources, requiring another layer of planning and communication as products were readied for public distribution. It was also difficult to distribute leadership responsibilities; the “lone leader” approach led to some council members feeling like it was a PNNL project rather than a true collaborative effort with shared leadership. Other challenges revolved around planning for growth and sustainability and determining meaningful measures of success (MCSEC 2013). Adjustments to operational procedures resolved some of the challenges. Some aspects of time management were improved by using online collaboration tools (e.g., SharePoint, GoogleDocs, Skype) to jointly generate and edit documents, allowing team members to reflect and respond in their own time, rather than having to always meet in person. However, members lacking time for Collaboratory work still seems to be one of our greatest challenges.

To address problems of time and task management, the council started planning its meetings several months in advance with a formal, time-delimited agenda and introduced an Action Items Snapshot that is updated monthly. The document became a running record of tasks and activities specifying who is responsible for each one, the anticipated timeline, and the status (i.e., planned, in process, completed, or postponed) of each. In addition, special-purpose meetings were added between regular monthly meetings, as needed, to address unique issues that would otherwise overwhelm the normal meeting agenda. These might include guests with specialized expertise, such as fund raising. Jurisdiction and intellectual property issues are now considered in advance and addressed as part of vetting materials that are to be shared publicly. In addition, although the preponderance of expertise continues to reside with PNNL, Collaboratory membership has increased, expanding available expertise and the distribution of operational responsibilities.

Preliminary Signs of Impact

In the early years, the launch and collaborative structure itself were the focus of evaluation with meeting minutes, end-of-year interviews, and trimester progress reports comprising the body of evidence. Testbed projects, which were proposed concurrently with the design of the Collaboratory, tested the viability of multientity management structures and collective management by the council. Some of the early findings around collaboration were described in the “Early Lessons” section earlier in this article. In addition, we found varying degrees of participants’ comfort in using collaborative tools, such as GoogleDocs, and in their experience gathering evidence of impact. This was true among council members, operations teams, and testbed project teams. In most projects, leaders did not understand the importance the Collaboratory placed on evaluating their impact; instead, leaders focused their energy and resources on providing the programming they had proposed. They needed assistance in planning, gathering, and interpreting meaningful data to document and learn from their experiences.

To address these needs, the Research & Evaluation team created

1. new reporting templates to facilitate consistent sharing of experiences and findings and to encourage reflection,
2. formal review and feedback procedures using GoogleDocs, and
3. short presentations to introduce teams to the evaluation model used across projects.

Both operational and project teams now complete brief monthly status reports and trimester progress summary reports to engage in regular reflection and document challenges and successes as they occur when events are fresh in their minds. Council members and members of the Research & Evaluation team read these reports and provide constructive feedback, using GoogleDocs comments, so authors can see the feedback immediately, engage in an online conversation with the reader, and take action as needed in a timely manner. By selecting council members with expertise relevant to a project as that project’s lead reviewer, we have been able to support projects better than before. In addition, regular reporting generates a relatively complete history of how a project or activity has unfolded, and this becomes very useful data for discerning lessons learned, challenges, and successful strategies that can benefit others. To help projects gather evidence of impact, we provided teams with data gathering templates and suggestions for using them in different settings. We now remind project managers of this need to gather evidence when providing feedback on their reports. Although some progress has been made, such as counting participants and collecting evaluations, project leads are still challenged to systematically follow participants to determine impact after participating.

As described earlier, the Collaboratory logic model specifies three short-term outcomes anticipated from the support of testbed projects. Despite limited impact data gathered by the projects from 2014 to 2016, there are indicators that the projects are having an impact by expanding access to high-quality STEM education in formal and informal educational settings in the region. Thus far, projects have analyzed headcounts, provided summaries of workshop participant evaluations, and identified testimonials from participants (workshop evaluation data have been gathered from several events but have not yet been analyzed). Some of the findings are described below.

Increased Awareness of Best Practices

In 2016 more than 50 STEM learning opportunities were sponsored by the Collaboratory; none would have been possible without Collaboratory support. In those events, more than 200 educators, including K–12 teachers from 21 school districts and volunteers from local community organizations (e.g., Boys & Girls Clubs, museums, libraries), participated to increase their computer science literacy. More than 250 educators and community leaders from organizations such as the Richland Public Library, REACH Museum, and Boys & Girls Club, engaged in hands-on Family Engineering activities, and the Collaboratory provided public access to more than 100 field-tested and documented tools for teaching STEM via their website and social media tools.

Use of Effective Strategies

Local teachers are introducing new STEM concepts to their students and using new strategies for teaching those concepts. In 2016 first-time teaching of computer science concepts and introduction of computer science activities was evident in K–12 classrooms in eight local school districts. New Family Engineering events or story-related engineering design challenges were offered in K–12 classrooms in four districts. New science-based engineering design challenges were offered in 22 K–6 classrooms across six districts. And new makerspace activities were made available to the community in one elementary school and its local public library.

Availability of High-Quality STEM Experiences

In 2016 more than 850 local students attended STEM workshops offered by Computing the Future (provides computer science workshops with updated curricular materials) and Experiencing the STEM Spectrum (holds Family Engineering events and innovative, literature-based engineering design challenges), often with their families, and occasionally in community venues such as Boys & Girls Clubs and public libraries. In addition, more than 500 students directly participated in hands-on makerspace activities in their school library. And at least 1,200 middle and high school students learned about a wide array of STEM careers as a result of viewing STEM Flicks career videos in their classrooms. In many cases, community members were actively engaged in planning and delivering project activities, and according to a few individual testimonials, some deepened their own understanding of school and community settings and needs, inspiring future volunteering. Even more importantly, all of these efforts have expanded further in 2017, involving more schools, colleges, and community venues.

Keys to Success

A collaboratory does not magically appear out of nowhere. In our experience, the keys to successfully launching and sustaining our work included the following actions:

• convening a broad array of stakeholders to jointly design an organization from a shared vision of what was required to meet local needs in STEM;
• focusing on identifying the gaps in STEM education that we could adequately address with sufficient expertise and funding;
• identifying and staffing critical operating functions to fulfill backbone organizational tasks, including the council chair, communications, and evaluation components;
• acquiring seed funding for pilot projects in a testbed environment;
• establishing an operational structure with specific membership goals, norms, values, and procedures;
• emphasizing building public awareness and familiarity with the Collaboratory to help sustain and broaden impact in the community via web pages and social media;
• planning for sustaining successful innovations by establishing evidence-gathering protocols to facilitate publicizing successes and thereby encourage local support in the future; and
• engaging in continuous self-assessment and making adjustments as we encountered challenges and learned from working together.

Is This a Model for Others?

The Design Team envisioned the Collaboratory as a replicable model that could be implemented in other regions that

1. have an established base of STEM enterprise,
2. appreciate and value the contribution of local educational institutions, and
3. want to build relationships among research and development organizations, industry, community, and educational entities.

We learned many lessons along the way to achieving our initial successes. Together, we have been able to improve local STEM education in formal and informal school settings, addressed our local workforce needs, and observed our community becoming more engaged in STEM learning and literacy. Our collaborators agree that participating in the Collaboratory has strengthened their community relationships, increased their recognition as STEM education leaders, and enhanced their professional reputations.

Peggy Harris Willcuts (peggy.willcuts@pnnl.gov) is senior STEM education consultant in the Office of STEM Education at Pacific Northwest National Laboratory in Richland, Washington. Cathleen A. Kennedy (cathy@kacgroup.com) is an independent evaluator and education research consultant in San Mateo, California.