Session I

This session will engage participants around the ongoing efforts of 8 campuses working collaboratively to improve persistence and address disparities for students who are low-income, first-generation, or underrepresented minorities by eliminating equity gaps in grade patterns in gateway courses as part of the Association of Public & Land Grant Universities' (APLU) Powered by Publics Big Ten Cluster. This Cluster's improvement aim builds upon the group's previous examination of student course data and the practices/ policies affecting success rates in entry-level courses.

The Cluster has played an important role for participating institutions in serving to keep student success in gateway courses centered as a priority by sharing ideas, finding commonalities, and challenging each other to keep making progress. Recently, the Cluster developed a Logic Model to map and support each institution's efforts in building a climate that informs, engages, and empowers stakeholders to collectively share and use data to end cycles of inequity in gateway courses. This session will share the Logic Model "roadmap," how it is being utilized, and provide an opportunity for participants to assess their own campus' progress.

Work at the University of Minnesota Twin Cities will serve as a case study for this session. Our campus has seen an evolution from a small group of individuals discussing gateway courses, to a larger committee, and now campus-wide conversations with increased access to student course outcome and degree progress data. Contextualizing our work in the Logic Model has served to ground the work and provide clearer direction for next steps.

This symposium shares the journey of an NSF-funded Research Coordination Network in uniting a group of instructors, education researchers, psychologists, and students to empower institutional change. FLAMEnet – Factors influencing Learning, Attitudes, and Mindsets in Education – was established in 2018 to explore undergraduate STEM students' orientations towards challenges. Initially, we aimed to involve instructors from multiple institution types and settings in a) developing classroom interventions, b) conducting large-scale research to assess the efficacy of those interventions, and c) disseminating said interventions within their departmental and institutional contexts. These initiatives were based on a theoretical model developed by network leadership from existing literature on intrapersonal factors (e.g., growth mindset, fear of failure) of interest

Conversations with network members during COVID lockdown elicited dual frustrations: a) roadblocks disseminating information and materials (e.g., lack of buy-in) and b) desires for the network to be more flexible in terms of individual needs and goals. That is, FLAMEnet members wished to explore empirical testing of alternative theoretical constructs (e.g., student belonging, instructor trust) as possible influences on students' outcomes within their specific educational context(s). Members also sought specific advice for identifying and navigating challenges related to their change initiatives.

In response, the network's steering team recognized a unifying drive to change departmental processes. To address member needs, the network pivoted to utilizing network resources to support member-proposed research projects and scaffold change-related initiatives inspired by specific educational contexts of our institutional teams. For example, our most recent annual workshop was centered around evidence-based approaches to change processes.

In this symposium, in addition to summarizing this history of FLAMEnet, we will provide information and prompt discussion on the usefulness of FLAMEnet's core constructs to drive change in undergraduate education and the potential of research-focused networks to empower members to become effective change leaders.

Most STEM departments have inadequate teaching evaluation practices. As a result, they are largely unable to recognize and reward evidence-based teaching. Yet widespread adoption of evidence-based teaching likely depends on systems that incentivize these approaches. With this in mind, we undertook the DeLTA project at the University of Georgia. DeLTA (Departmental and Leadership Teams for Action) aims to transform how STEM departments evaluate teaching. DeLTA advocates for using three voices to inform teaching evaluation: students, trained peers, and instructor self-reflection.

The DeLTA project convened 12 STEM department heads to enact new teaching evaluation practices in their departments. They gathered for facilitated meetings five times per year for three years. Drawing on a social cognition theoretical perspective, we created opportunities in meetings for heads to recognize and critically evaluate their ideas about teaching evaluation and to construct new knowledge. Drawing on a cultural perspective on change, we created opportunities for heads to uncover underlying assumptions and values and craft new departmental practices. Meetings created space and impetus for reflection that otherwise would not occur. Both meetings and tailored consultations provided scaffolding and differentiated support for advancing teaching evaluation practices.

We studied the impact of DeLTA on teaching evaluation practices. We systematically documented changes in departmental teaching evaluation practices for the three voices between 2019 and 2022. We did so by developing the Guides to Advance Teaching Evaluation (GATEs), which articulates a list of best practices for using each voice. We gathered data about departmental evaluation practices using interviews and analyzed data using the GATEs. Some departments made major changes by advancing practices for at least two voices and enacting multiple best practices. Other departments made minor changes, generally trying out one new voice without attention to best practices. A few departments pursued no change, despite continued involvement for three years.

We aimed to uncover differences in the thinking of department chairs who pursued more and less change. We characterized each department chair's thinking about changing teaching evaluation practices using the readiness for change theoretical framework. Readiness for change is a cognitive precursor to engaging productively in the change process. We gathered data using interviews and recordings of the facilitated meetings. We conducted qualitative content analysis using both deductive and inductive coding. We created profiles for each department chair and contrasted those who pursued major changes, minor changes, and no change to teaching evaluation practices. These analyses generated hypotheses about the thinking that motivates department chairs to pursue reforms to teaching evaluation. For example, department chairs who pursued major changes to teaching evaluation practices expressed considerable dissatisfaction with the status quo of teaching being seen as second-class work that goes largely unrecognized.