Assessing impact and visualizing progress of a campus-wide initiative to transform teaching evaluation

The Teaching Quality Framework Initiative (TQF) is an ongoing action research project at the University of Colorado Boulder (part of the multi-institutional NSF-funded TEval project) that engages multiple stakeholders in the process of transforming teaching evaluation across campus. We approach this transformation primarily through a process of facilitated departmental working groups in combination with cross-departmental and cross-campus stakeholder engagement. This approach is grounded in higher education scholarship, institutional and organizational change in academia, and scholarly approaches to teaching and learning. Predicted outcomes of the TQF Initiative span multiple scales and include (but are not limited to):

1. Departments that form TQF working groups will adopt and implement better teaching evaluation tools and practices
2. These tools and practices will be aligned with a common scholarly framework (the TQF rubric), more explicitly attend to equity and inclusion, and rely on multiple measures from three voices (self, peer, student)
3. As better tools and practices are developed and implemented within departments, they will be taken up by other units across campus
4. This work will help departments/the university establish improved, transparent guidelines for defining quality teaching and how to recognize and evaluate it for merit, reappointment, tenure, and promotion
5. This work will enhance the visibility and value of teaching campus-wide.

To assess whether these outcomes are being met, we are gathering and analyzing data to systematically evaluate the effectiveness of the TQF approach to transforming teaching evaluation. These data include interviews with TQF project leads and departmental working group participants, pre/post comparison of evaluation tools/systems, meeting notes/artifacts, etc. In this presentation we will discuss the forms of data we collect to assess impact and how these are represented, share these representations for others engaged in similar transformation efforts, and share preliminary findings from data analyses.