Assessment of Change Resources

Contribute Resource »

Tools: Measuring student learning

National Survey of Student Engagement (NSSE) (Organization Website). The NSSE is an organization providing information to institutions about student engagement in effective educational practice, primarily through the College Student Report: A survey of student time and effort, and institutional resources. Also includes a Faculty Survey of Student Engagement (FSSE), Beginning College Survey of Student Engagement (BCSSE). Products include reports, engagement indicators, high impact practices, and survey results. Survey items reflect "good practices" in undergraduate education.

Learning Assistant Alliance (website). Includes information on supporting and training undergraduate Learning Assistants (LAs), and joining with other institutions to promote this change program. Allows uploading of concept inventory data for comparison to other institutions through LASSO. Also included in: Assessment, networks.

AAC&U Value Rubrics Set of rubrics for evaluating various student learning outcomes, including inquiry, integrated learning, critical thinking, and ethics.

SERU (Student Experience in the Research University) (instrument). The Student Experience in the Research University (SERU) survey' goal is to provide data to promote institutional self-improvement by serving research, policy analysis, and programmatic assessment needs. Areas addressed include academic engagement, students' majors, global skills and awareness, climate for diversity, community and civic engagement, and student development.

Promoting Instructor and Department Action via Simple, Actionable Tools and Analysis(Conference paper, 2016). Describes recent work to provide tools to support evidence based instruction, including Know Your Students, Departmental Diagnostic Dashboard, and Ribbon Tool. Includes description of how these measurements have inspired change.

Developing Indicators for Undergraduate STEM Education (Public report draft). Documenting the current state of undergraduate STEM education indicators, currently open for comments.

STEM Evaluation Repository of AEA (Resource list). Resource list of STEM Evaluation metrics from the American Evaluation Association STEM topical group. Not well populated at this time but could be a growing resource. Add a resource through this form

Understanding Interventions (website). Understanding interventions that broaden participation in science careers. Includes a conference and community resources. Lots of great data at the UI Index on the top tab.

Survey of Undergraduate Research Experiences (SURE) (Survey). This is a survey for undergraduates who have recently completed a summer undergraduate research experience. SURE, and the related tool, the Classroom Undergraduate Research Experiences (CURE). The CURE may be used as a pretest-posttest or posttest-only survey to measure student experiences in "research-like" or other science courses.

The ETS® Major Field Tests (assessment). These are comprehensive undergraduate outcomes assessments designed to measure the critical knowledge and understanding obtained by students in a major field of study, including biology, chemistry, mathematics and physics.

The ETS the HEIghten™ Outcomes Assessment Suite. This can be used to assess students' skill levels in critical general education areas including critical thinking and quantitative literacy. Results provide evidence of student outcomes, to inform decisions to enhance curriculum, improve the overall learning experience and achieve institutional goals, and evaluate improvement over time.

Concept inventories

Student attitudes

  • PhysPort (Website). This site hosts information and guidance for physics faculty on using evidence-based teaching methods, including lists of validated assessments and a Data Explorer for uploading and analyzing data, including student attitudes about science instruments. Also included in: Measurement. Relevant for: Novices.
  • The Field Tested Learning Assessment Guide (FLAG) and Views About Science Survey (VASS). (Survey). The VASS is designed to survey student views about knowing and learning science, and to assess the relation of these views to student understanding of science and course achievement (Grades 8-16).
  • Colorado Learning Attitudes about Science Surveys . A suite of surveys for measuring student attitudes towards Physics, Chemistry, and Biology.
  • E-CLASS Survey. The Colorado Learning Attitudes about Science Survey for Experimental Physics assesses student views about habits of mind about experimental science.

Tools: Measuring classroom practice

The Postsecondary Instructional Practices Instrument (PIPS) (website and instrument). Western Michigan has piloted survey measures to measure teaching and learning approaches.

Research based methods of instructional reform (source). A list of measures of instructional reform from the Stains research group, including classroom observation protocol, self-report surveys, and classroom artifacts.

Bay View Alliance (Network). A network of universities working to improve teaching and learning in higher education. Includes three Research Action Clusters: Entry level courses, intellectual skills in multiple disciplines, measurement and analytics, and professional development. Includes a Teaching Practices Survey (modified from Wieman et al) and a departmental case study protocol for assessing change. Also included in: Change networks, measurement.

Assessing quality of teacher prep (Report). Development of the Teacher Education Program Assessment (TEPA) tool.

Ethnology and Evaluation Research at CU Boulder (website). E&ER conducts research and evaluation on a variety of issues in STEM education, including student and faculty outcomes, women in science, under-represented minorities, professional development, student-centered education, and out of school learning. Also included in: Cross cutting issues, leadership. Relevant for: Experts.

Learning Spaces Toolkit (website) is a resource for assessing technology-rich informal learning spaces, in particular to examine the kinds of activities envisioned for the space(s) and to identify outcomes that will indicate how well these activities are supported over time. The Needs Assessment protocol will help in the design of effective spaces, and will inform decisions throughout the space design and service design processes.

The Learning Spaces Collaboratory (LSC) Guide: Planning for Assessing 21st Century Spaces for 21st Century Learners (Guide). This guide, developed by a working group convened by the LSC, is a template for measuring return on investment of time, energy and funds expended in shaping physical environments serving 21st century learners and examining how space influences the nature of the learning experience. It is intended to advance efforts on individual campuses to imagine and shape new spaces, reimagine and repurpose existing spaces; it is intended to inform the national dialogue about why attention to physical learning environments matters.

Observational protocols

The Best of Both Worlds: Building on the COPUS and RTOP Observation Protocols to Easily and Reliably Measure Various Levels of Reformed Instructional Practice (Journal article, 2015) Lund et al. Cluster analysis of RTOP and COPUS to identify 10 profiles. These profiles and their use are outlined here: COPUS Profiles

Tools for Evidence-based Action (TEA) (Project). The mission of the Tools for Evidence-based Action (TEA) community is to enhance evidence-based teaching methodology and policy by developing and sharing new technologies. To start, we have developed a visualization system to understand the pathways that students take through a university (Ribbon Tool) and an online classroom observation platform (GORP) (including several observation protocols) which can be used for researching teaching methodology and teacher development and training.

Teaching Dimensions Observation Protocol (TDOP) (Instrument website). An online classroom observation protocol designed to capture aspects of classroom instruction at fine-grain size (teaching methods, cognitive engagement, technology, student-teacher interactions). Adaptations of the full TDOP instrument include COPUS. Website includes access to protocol, user manuals, and related publications

Toward a descriptive science of teaching: How the TDOP illuminates the multidimensional nature of active learning in postsecondary classrooms (article, 2015). This empirical paper describes the use of the instructional systems-of-practice framework and the Teaching Dimensions Observation Protocol (TDOP) to advance an approach to thinking about teaching in science classrooms in more multidimensional terms than the common lecturing v. active learning binary. Results indicate variation across disciplines and course contexts, that active learning is embedded within PowerPoint lectures, and that small group work exercises are not synonymous with constructivist activities.

PORTAAL, the Practical Observation Rubric To Assess Active Learning (journal article, 2015). Sarah Eddy and colleagues. The paper describes its development and validation. It is designed to systematically document evidence-based teaching practices, including the extent to which instructors expect students to practice what they are learning, to engage in higher level thinking, and to be accountable for course content as well as instructor practices aimed at reducing student apprehension.

Tools: Measuring organizational change

The Survey of Climate for Instructional Improvement (SCII) (website and instrument). Western Michigan has piloted survey measures to measure departmental climate towards teaching.

Strategic Toolkit: Strategies for Effecting Gender Equity and Institutional Change (Toolkit).The Toolkit includes 13 Strategic Intervention Briefs that each explain a specific intervention to foster change for greater campus inclusiveness and diversity, including addressing purposes, models, affordances, limitations, and relevant evaluation and research.

Bay View Alliance (Network). A network of universities working to improve teaching and learning in higher education. Includes three Research Action Clusters: Entry level courses, intellectual skills in multiple disciplines, measurement and analytics, and professional development. Includes a Teaching Practices Survey (modified from Wieman et al) and a departmental case study protocol for assessing change. Also included in: Change networks, measurement.

Partnership for Undergraduate Life Science Education (PULSE) Rubrics (program, website). Rubrics for assessing departmental readiness for change and enactment of Vision and Change, including faculty practice, assessment, and more. Also included in: Strategies, leadership, networks.

SERU (Student Experience in the Research University) (instrument). The Student Experience in the Research University (SERU) survey' goal is to provide data to promote institutional self-improvement by serving research, policy analysis, and programmatic assessment needs. Areas addressed include academic engagement, students' majors, global skills and awareness, climate for diversity, community and civic engagement, and student development.

Promoting Instructor and Department Action via Simple, Actionable Tools and Analysis(Conference paper, 2016). Describes recent work to provide tools to support evidence based instruction, including Know Your Students, Departmental Diagnostic Dashboard, and Ribbon Tool. Includes description of how these measurements have inspired change.

Understanding Interventions (website). Understanding interventions that broaden participation in science careers. Includes a conference and community resources. Lots of great data at the UI Index on the top tab.

Assessing departmental readiness.

There is a growing need, and set of resources, for assessing the level of departmental readiness for change in order to inform change strategies. A list of such resources, most in development:

General assessment

Transforming Institutions (Book, 2015). This book brings together chapters from scholars in the 2011 and 2014 Transforming Institutions conferences. It provides an overview of the context and challenges in STEM higher education, descriptions of programs and research, and summary of lessons learned, plus next steps. Also included in: Model and theory, Recommendations for leaders, cost benefit, assessment.

American Evaluation Association (Organization). Network and professional society for evaluators, including ability to find evaluators, and many resources for performing evaluation and communicating results to stakeholders. Includes a topical group for STEM education evaluation.

Searching for better approaches: Effective evaluation of teaching and learning in STEM (Report, Research corporation for scientific advancement, 2015). Summary publication of a two day workshop, Effective Evaluation of Teaching and Learning, organized by Cottrell Scholars and AAU on effective evaluation of teaching and learning in STEM.

National Institute for Learning Outcomes Assessment (NILOA) (Organization website). NILOA helps others in assessing college student learning outcomes. It is a project of U. Illinois and Indiana University. It conducts periodic surveys of campus assessment efforts, producing commissioned papers and case studies.

Innovation and Quality in Teacher prep (Report, 2012). APLU assessment framework for the Science & Math Teacher Imperative (SMTI) to allow program leaders to assess policies and practices and better determine program coherency.

Evaluation of teacher prep advancements project (Report). Summary text. Also Included in: Relevant for:

Evaluation of undergrad teaching in STEM (NAP Report, 2003). Offers a vision for systematic evaluation of teaching practices and academic programs, with recommendations to the various stakeholders in higher education about how to achieve change, focusing on assessment of teaching effectiveness.

Scientific Teaching: Defining a Taxonomy of Observable Practices (Article, 2014). Couch et al. Describes development of a taxonomy of the core elements of scientific teaching, including pedagogical goals, practices, behaviors, artifacts and features releated to scientific teaching. Provides a framework for researchers studying course transformation, and guide for faculty. Also Included in: Theory.

Describing & Measuring Undergraduate STEM Teaching Practices (Conference report, AAAS, 2013). Describes practices for measurement of teaching excellence in STEM.Also included in: Assessing Change.

Why theories of change matter (WCER Working Paper No. 2015-2): Connolly, M. R., & Seymour, E. (2015). (Working Paper, 2015). Connolly and Seymour. Argues that making programmatic theories of change explicit, by helping organizers specify their reason for their theory of change or action, can be useful. Also included in: Measurement. Relevant for: Experts, evaluators.

Faculty Development and Student Learning: Condon et al., 2015 (Book, 2015). Condon, Iverson, Manduca et al. reports the results of a multi-year study undertaken by faculty at Carleton College and Washington State University to assess how students' learning is affected by faculty members' efforts to become better teachers, including assessment of faculty development effectiveness. Also Included in: Costs and Benefits, measurement.

Undergrad chemistry education: A workshop summary (NAP report, 2014). Summary of a workshop convened in May 2013 by the Chemical Science Roundtable of the National Research Council to explore the current state of undergraduate chemistry education. Discusses large-scale innovations that are transferable, widely applicable, and/or proven successful, with specific consideration of drivers and metrics of change, barriers to implementation of changes, and examples of innovation in the classroom. Also included in: cross cutting issues, theory. Relevant for: Faculty, change agents.

Communication to stakeholders

Enhancing use of learning sciences research in planning for and supporting educational change: Leveraging and building social networks (Journal article, 2016). Penuel, Bell, Bevan, Buffington and Falk, Journal of Educational Change. 17,2,251-278. Interesting paper on efforts to leverage recent research on social networks to inform Next Generation Science Standards implementation across a set of professional associations and school districts. Although this paper involves K-12 teachers and administrators, it may be of value in our effective communication work.