Building a better SET: An "outside-in" approach to developing and implementing improved student evaluations of teaching

Student evaluations of teaching (SETs) are a contentious, but ubiquitous, tool in teaching assessment. Wide-ranging stakeholders--from faculty assemblies to national organizing bodies to the courts--have recently argued that current methods are of dubious validity and exhibit unacceptable levels of bias. Yet, many of these efforts to change SETs neglect to answer the question, what is a better alternative? We describe a multi-pronged initiative to develop and enact improved SETs on our campus, and present analyses and evidence from two pilot studies testing improved student evaluation measures. Three key principles underlie this work: (a) Attend to local context: we build on prior reform attempts to engage existing change agents, while also addressing shortcomings and the concerns of those who challenged them (e.g. we recognize the role of institutional culture, address challenges to implementation, and leverage additional supportive evidence, such as in-depth testing and analysis); (b) Employ an "outside-in" model for institutional change: build support within interested departments ("bottom-up") through opt-in/voluntary opportunities to engage while also mobilizing "top-down" institutional entities, such as the faculty assembly and administration; (c) Utilize parallel approaches to improving the use of SETs--build a better evaluation tool (i.e. the measures) and promote improved interpretation of SETs regardless of the tool--so that success on one does not hinge on the other and multiple paths to success are available. Finally, we describe the development of new measures for SETs, two phased pilot studies testing those measures, and analyses related to validity, non-response (who completes optional SETs, the impact on the data), bias, and interpretation of SETs. Ultimately, we argue that implementing better student evaluations of teaching requires both a better process for navigating a potentially contentious change to institutional practice and a better tool worthy of the support of its constituents.