Institutional Strategies to Transform Teaching Evaluation: Developing Theories of Change

We share findings from a study of 16 colleges and universities convened as a learning community cluster (LCC) to pursue a common goal: transforming institutional systems of teaching evaluation to recognize and value inclusive teaching in STEM. Though they shared this general goal and a commitment to inclusion, the 16 institutional teams were convened by their funder, rather than self-assembling, and had to learn to "row in the same direction." They began their collaborative work with a wide range of campus contexts and histories of teaching evaluation reform. Team members also had diverse experiences of leading change in their institutions, and diverse ideas about what changes were needed and what levers might be applied to accomplish those changes. In our capacity as developmental evaluators and researchers working with the LCC, we asked: What is the added value for institutional change teams to work with the learning community? What promotes or inhibits the development of added value? To answer these questions, we studied the organizational change strategies of both the local campus change projects and the LCC as a whole. Working as embedded researchers, we gathered information from team documents, focus groups, surveys, and participant observation. In this analysis, we summarize the types of interventions the institutional teams pursued, with what rationales. We analyze patterns in these interventions and rationales to develop prototypical theories of change that reflect some contextual and experiential factors that shaped institutional teams' choices, such as institutional size. We also offer observations on the collaborative work that aided teams in articulating, refining and implementing their local theories of change. Sadly, the work of this community will be interrupted by early termination of their funding. Our analysis thus focuses on the pitfalls and promises of their initial theories of change, offering emergent insights intended to assist future projects.