Working Groups 2 & 4: Costs, Benefits, and Demonstrating Impact
Working Group 2:
This working group will explore the financial and social costs and benefits of implementing, scaling and sustaining instructional changes in undergraduate STEM education. In particular, techniques and expertise from economics and related social sciences will be applied to STEM higher education to evaluate and promote change initiatives aimed at improving student success.
Working Group 4:
Using data for a change: This group's mission is to identify, explain and disseminate information on metrics that hold the potential to document, foster, accelerate, and communicate systemic change. Documenting, by creating empirical evidence for the effectiveness of various approaches for changes in undergraduate STEM education. Fostering, by enabling stakeholders to reflect upon and assess current conditions. Accelerating, by using existing measures more broadly and effectively and by identifying important elements of change that are not being adequately assessed. Communicating, by establishing a common language and set of expectations for studying change. Identification involves both cataloging what is available and suggesting what might be usefully developed, including the potential for a set of common change metrics. Explanation includes a discussion of the implications of each metric, illustrated by cases that describe how metrics have been used successfully to foster change. Dissemination comprises making this work readily available in an accessible format via a community-curated website.
Knowledge Claims and Established Ideas
Working Group 2:
Change initiatives related to pedagogical innovation are typically analyzed in terms of financial costs to the institution and initial time investments by the faculty, while the benefits are framed in terms of improved student outcomes (generally at the course level). What is generally missing from these analyses is the potential financial benefit to the institution of improved student performance, e.g. improved retention, persistence in a major, and graduation rates, among other possible measures. Typical analyses of pedagogical change initiatives also fall short in capturing the ongoing cost of faculty time investment to use improved teaching methods beyond the initial learning curve.
This working group is focused on exploring the costs and benefits of change at multiple levels: individual, department, and institution. When considering the impact of change in higher education, it is important to include both economic and social costs/benefits, including effects on equity and inclusion.
Costs and Benefits of Individual and Department-Level Change. Previous research on change initiatives related to pedagogical innovation has focused primarily on costs and benefits to individuals (e.g., time costs to instructors, incentives and rewards for changing teaching practices) and groups of students (e.g. changes in learning outcomes in a class section or course), although there is increased research activity analyzing efforts to implement and assess pedagogical innovation at the department level.
Costs and Benefits of Institutional Change. There has been relatively less work done on costs and benefits of change at the institution level. The working group views this as a critical area of emphasis, given the lack of previous research and practical guidance for higher education leaders, as well as potential impact on student success.
Working Group 2:
- What kinds of institutional data inform and influence change and in what ways? How does this question inform what kinds of data should be collected to promote institutional change?
- How do institutions appropriately measure the financial and social costs and benefits of institutional change initiatives aimed at improving student learning outcomes, persistence, and graduation rates? What data on the financial and social costs and benefits of institutional change efforts are needed by chancellors/presidents, provosts, and deans to effectively evaluate potential projects and initiatives aimed at improving student success at an institution?
- How do faculty experience their day-to-day work and how does that experience affect their interest in adopting and promoting evidence-based teaching practices? What is the incentive for individual faculty members to improve teaching and learning? How is teaching (and the improvement of teaching) valued by the department and institution? What constraints do faculty feel (in terms of incentives, alternate uses of time, colleagues' approval, etc.) when considering making changes to their teaching practices? How can change leaders make use of this knowledge to promote change at the individual and department level?
- How does the "change process" differ across institutional types - community colleges, public four-year institutions, private four-year institutions, R1 and comprehensive regional institutions - and how does institutional context regarding costs and benefits of change influence "what works" for promoting change at the personal, department, and institutional level?
Working Group 4:
- Who are the stakeholders who need to be informed of the metrics that can and should used to establish baselines and monitor progress?
- How do we inform these stakeholders of what can and should be measured?
- What are the impediments to the use of existing metrics and what can be done to reduce them?
- What metrics do the stakeholders believe they need and currently engage, and how does this vary by institution and student type?
- What metrics do we believe they need, and how does this vary by institution and student type?
- What do we know about how these metrics are now being measured, and which instruments will need to be built or improved?
- How can we help stakeholders effectively engage with these metrics?
- How do we infuse this entire process with the importance of inclusivity?
- How do we assess our own effectiveness in providing these resources?
- Pamela Brown, CUNY New York City College of Technology (pbrown [at] citytech [dot] cuny [dot] edu)
- Archie Holmes, University of Texas (aholmes [at] utsystem [dot] edu)
- Lorne Whitehead, University of British Columbia, Vancouver, BC (CA) (lorne.whitehead [at] ubc [dot] ca)