Reducing implicit bias in student evaluations of teaching: observations of high-impact classroom practices

We present on progress made in developing a student evaluation of teaching (SET) inventory that reduces implicit bias, is instructor-actionable, and is reliable across student evaluators. Our approach has focused on elevating faculty doing the "invisible work" disproportionately done by minorities and/or women that evidence demonstrates is required for underrepresented student success, but is rarely rewarded by traditional university structures. In particular, SETs focused on the affective-domain have been criticized for being race and gender biased and for failing to measure the underlying construct: teaching effectiveness. For example, although research-validated high-impact teaching practices (HIPs) are more likely to be used in courses with minority and/or women instructors in a variety of fields, this population often scores lower on SETs, which are frequently used in personnel decisions. To construct an SET, we started with the Faculty Inventory of Methods and Practices Associated with Competent Teaching (F-IMPACT), which is a validated instructor self-report of research-based HIPs usage. Language was modified to turn the F-IMPACT into a student observation protocol, where over the course of 15 prompts students identify HIPs used in the course. In contrast to affective-domain SETs, this approach was designed to minimize subjective judgements. Either a student did or did not observe a specific practice. Initial face validity of the SET has been established via student focus groups and clarity surveys (N = 22), and reliability and congruent validity studies are reported from a single institution pilot (N = 512). We also discuss the validity of the underlying F-IMPACT instrument, SET pilot efforts currently underway at several institutions, and efforts made to include faculty concerns and perception of barriers.