Evaluation & Assessment

2021-2022 Annual Report

This team encompasses two important areas of work in CTSI: the operation of the centralized online course evaluation system, including how to meaningfully interpret and use student feedback, and support for the scholarship of teaching and learning. The Course Evaluation operations team supports the intake and modification of data that populates the online evaluation system, contributing to an essential component of the University’s commitment to teaching excellence. The Scholarship of Teaching and Learning (SoTL) entails the study of student learning and the public sharing of these investigations. The CTSI SoTL team provides instructor support through programs, consultations and project planning, and community building opportunities. Learn more about the team’s work in Intersections.

Glossary of Abbreviations

ACT: Academic and Collaborative Technology
ACUE: Association of College and University Educators
ALC: Active Learning Classroom
AODA: Accessibility for Ontarians with Disabilities Act
CE: Course Evaluations team
DLI: Digital Learning Innovation
E&A: Evaluation and Assessment team
FAS: Faculty of Arts & Science
FASE: Faculty of Applied Science & Engineering
IRDG: Institutional Research & Data Governance Office
ITS: Information Technology Services
KPE: Faculty of Kinesiology & Physical Education
OISE: Ontario Institute for Studies in Education
OVPIUE: Office of the Vice-Provost, Innovations in Undergraduate Education
P&C: Programming and Communication team
SGS: School of Graduate Studies
SoTL: Scholarship of Teaching & Learning
TATP: Teaching Assistants’ Training Program
TFM: Temerty Faculty of Medicine
TLT: Teaching, Learning and Technology team
TRC: Truth and Reconciliation Commission
UTM: University of Toronto Mississauga
UTSC: University of Toronto Scarborough

As an archaeologist, I was unfamiliar with how best to approach qualitative survey data that I collected for teaching research. I really needed someone to confirm for me that I was on the right track. Kyle gave me the confidence that I needed to move ahead with this project. I'm enormously grateful to him and to CTSI for last year's workshops and journal club.

Working with CTSI helped me understand how learning at U of T works, what are the institutional expectations, and how to create an enjoyable and effective learning experience for students. Specifically, I appreciate comments and feedback provided by Kyle Turner who helped me process student evaluations to improve my teaching in future offerings.

Course Evaluations Highlights

Piloting 1) Dynamic Report Access (DRA) and 2) automatic preparation of ROSI data:: With the DRA, the CE operations team can grant back access to historic evaluation reports for incoming administrators, and efficiently manage appropriate access for administrators. For the ROSI data, a PHP script that manipulates CSV files was developed as a precursor to the design of a specialized CE middleware database application that greatly improves the intake and processing of course evaluation data.
Course Evaluation Institute: The team relaunched the Course Evaluations Institute, bringing together 39 course evaluation professionals from 20 unique institutions. Across three weeks, participants took part in six panel discussions, with topics ranging from operations and statistics to equity and communications.
Bluenotes Conference: E&A team members presented a framework for integrating educational development with course evaluation operations in their presentation, “Feedback from Form Fill-out to Faculty Development: Fostering a Data Culture to Integrate Perspectives and Lessons across Multiple Roles”, at the Bluenotes Global 2021 conference, and also facilitated and participated in the cross-institutional panel, “Discussing and Addressing Bias in Course Evaluations” (awarded Best Presentation).
CE Report Study: The team received a grant from the course evaluation system vendor, Explorance, to conduct an internal improvement investigation on how course evaluation reports are used and understood. Led by the CTSI Director and Principal Investigator of the project, CE Operations and SoTL team members collaboratively developed the survey. Close to 600 faculty responded and findings were presented at the annual Bluenotes conference in Chicago in August 2022.

Scholarship of Teaching & Learning

SoTL Journal Club: The SoTL team invited instructors to facilitate a monthly SoTL Journal Club session (7 held between Sept 2021-April 2022). Instructors selected the scholarly article and developed discussion prompts. Webinars were consistently well-attended and received excellent evaluations.
Pilot SoTL Writing cohort: Launched in October 2021 with 12 participants, this 7-month cohort program offered instructors a reciprocal, peer-driven hybrid learning and support environment as they worked in small groups, guided by an evidence-based writing structure. Many participants successfully drafted various dissemination publications, and evaluations have informed planning to offer future cohorts.
Building/enhancing the SoTL Hub: The team continues to carefully curate resources, including documents, links, and programming that encompasses all stages of the research process. Along with programming and consultations, this Hub is part of a broader CTSI support system for all members of the university community who are interested in undertaking a SoTL project. Membership in the SoTL Hub has more than doubled over the past year.

I arranged a one-on-one Dossier Consultation with Cora McCloy to discuss how elements of the ACUE course, of which she was a facilitator, could be integrated into my Teaching Dossier. We discussed which modules of the ACUE course inspired me and complemented my teaching philosophy. Cora’s expertise was essential in guiding me through developing a narrative that communicated the considerable amount of work undertaken in the ACUE course, how particular elements were being incorporated into my current teaching, and how I envision their role (and others) in my future development. My experiences working with CTSI, and in particular Cora, have been invaluable.

I had the privilege of participating in the SoTL journal club series this year, both as a participant and as a contributor, and the experience had a tremendous impact on prompting me to actively engage in SoTL and informed my experimental design & data collection.

The presentations and facilitated discussions, as well as the workshops provided by Cora and Kyle were engaging and stimulating. I also greatly appreciated the fact that Cora and Kyle offered that Prof Maria Asif and I work collaboratively, bridging the two disciplines, Life Science and English, and fostering a broader sense of community.