The Importance of Measuring Collaboration in Higher Education Reform Initiatives

Increasingly, higher education STEM reform change initiatives seek to advance systems change, often through the proliferation of multi-institutional and multi-sector collaborative reform strategies. These initiatives believe that by working together, they can affect greater change than individual or organizational efforts. However, evaluation strategies and evaluation capacity are often not designed to address the complexity of these initiatives, suggesting the need for new and innovative approaches to measure success and guide ongoing improvement. Evaluators typically focus on intended project outcomes, but often overlook theories of change that include collaborative dynamics as a key determinant that influences how the project team functions, which then affects what change activities are pursued as well as what is accomplished. The purpose of this session is to explore the measurement of collaboration in two large, NSF-funded initiatives focused on inclusive teaching and broadening participation in STEM. The presenters will review the development and implementation of metrics and how they have engaged the projects in reviewing and exploring their respective collaborative dynamics. In addition, the presenters will explore a systems mapping strategy for how to connect and integrate collaborative dynamics in the examination of a project or initiative's reform strategies and associated outputs and outcomes.