Session VIII

Over a decade of research on learning mindsets – such as belonging, growth mindset, and purpose – demonstrates that positive experiences of community, belonging, and academic support on campus and in the classroom can increase a student's likelihood of persisting through academic challenges toward graduation. This is especially important for students from structurally disadvantaged or numerically underrepresented groups, who are paying particular attention to signals from the institution, their professors, and the other students about whether they, or people like them, belong on campus, and are capable of succeeding. But how do we apply this scholarship to create systemic change, particularly now as students face unprecedented strain and disruption to learning?

The Student Experience Project (SEP) is a national network of scholars and campuses seeking to increase equity in college success by transforming students' experience in STEM introductory courses. The SEP has engaged over 150 faculty in a continuous improvement approach to introduce new classroom practices and gather frequent student feedback on the learning environment in order to improve student experiences. Since this work began in Spring 2020, SEP campuses have expanded participation from small communities of STEM faculty to entire courses, departments, and soon, colleges. During this session, campus leaders from two urban-serving universities participating in the Student Experience Project will share successful strategies for driving this systemic change. Participants will learn about scalable practices they can use to support their own efforts to transform STEM education - whether in-person or in virtual settings.

Structured classroom observations are widely used in research and evaluation of STEM classrooms to characterize teaching and learning activities. Because conducting observations is typically resource intensive, it is important that inferences about change involving pre/post and between-group comparisons are made confidently. While much attention about observational data focuses on interrater reliability (the agreement between independent raters), the reliability of a single-class measure over the course of a semester receives less attention. We examined the use and limitations of observational data for characterizing and evaluating teaching practices. We wanted to know how many observations are needed during a typical course to make confident inferences about teaching practices. We conducted two studies based in generalizability theory to calculate reliabilities given class-to-class variation in teaching over a semester. We used 177 in-class observations from 32 undergraduate mathematics courses using the TAMI-OP observational protocol which collects 2-minute observations of 11 student and 9 instructor behaviors. Eleven observations of class periods over the length of a semester were needed to achieve a reliable measure for our data as defined by a reliability coefficient of G = .8 or greater. Additionally, we found that different activity codes varied on the level of rater agreement needed to achieve a reliable measure. Agreement on what constitutes teacher and student questions showed the lowest agreement. The number of observations is many more than the one-to-four class periods typically observed in the literature. Comparing our results to other studies for the amount of class-to-class variability responsible for reliability suggested that our data were slightly more variable than other studies, but needing eleven observations was not an outlier. Findings suggest practitioners may need to devote more resources than anticipated to achieve reliable measures and comparisons in research and evaluation that incorporates observational data.  

This presentation will highlight the process the COMunities for Mathematics Inquiry in Teaching Network (COMMIT) developed to ensure diversification in their second round of on-boarding regional COMMITs. This presentation is geared toward anyone interested in expanding and diversifying their current professional learning teams, whether they be comprised of single institutions or multiple institutions. COMMIT is a national network of regional Communities of Practice centered around integrating active learning into the undergraduate Mathematics classroom that receives funding through a NSF grant. Communities of Practice are important for pedagogical change to be accepted and implemented with fidelity (Austin, 2011; Gehrke & Kezar, 2016). This presentation will describe how COMMIT national leaders identified a need to increase the diversity of the regional COMMITs with which they work through the following: development of a process to clarify their current demographic status, identify gaps they needed to address, expansion of their recruitment efforts, and data-based decisions for selecting new regional COMMITs. This presentation also highlights how the national network is using Social Network Analysis (SNA) to longitudinally measure the demographic makeup of individuals that interact with one another both intra- and inter-regionally to determine the amount of homophily present among the network members. This information will then be used to identify possible interventions the national COMMIT leadership team can use to help increase the diversity of the interactions themselves. The format of this session will be a 15 minute overview of the project including 10 minutes of Q and A and discussion specifically aimed at how this process might be integrated for different contexts.