The work put into establishing the vision, purpose, and goals of the program should help provide direction for creating program outcomes. Basically, it is necessary to take those goals and translate them into tangible, ‘measurable’ outcome statements. Biggs and Tang (8) describe an outcome as
…a statement of how we would recognize if or how well students have learned what is intended they should learn…[it] tells us what, and how well, students are able to do something that they were unable, or only partially able, to do before teaching. Good teachers have always had some idea of that…in outcomes-based teaching and learning, we are simply making that as explicit as we can – always allowing for unintended but desirable outcomes.
Biggs and Tang were referring to course-level outcomes in this quote; however, the principle is relevant to outcomes at any level. It is important to make explicit our expectations about what we want students to take away from a course or program. The last part of the quote is especially important – it is not possible to anticipate or account for everything a student is going to learn, and that is not what learning outcomes seek to do. The purpose of outcomes is to make the expectations and priorities clear, with the knowledge that there will be other things students take away from courses and programs.
Advantages of Learning Outcomes
- Transparency to students – outcomes statements (at the program, course, or lesson-level) articulate expectations for students, which helps them understand the focus of the experience, and what will be required of them.
- Focused and strategic teaching and assessment planning – by articulating what students should be able to accomplish by the end of an experience, other elements of the experience can be strategically aligned with those outcomes.
Outcomes can be the driver for:
- Assignments and tests
- Choice of instructional mode
- Choice of course content
- Assessment and feedback
- Accreditation or program assessment
- Curriculum development
- Better learning and better performance on assignments and tests – while this influence is indirect, the relevance driven by the alignment and transparency noted above leads to more focused and motivated work on the part of students, which improves performance on assessments (9).
- Meeting internal requirements (UTQAP) and external requirements (e.g., professional accrediting bodies) – internal and external quality assurance processes require units to have program outcomes and show how the program has been designed to support those outcomes.
Challenges and Limitations of Learning Outcomes
- Reductionist – by nature, outcomes are meant to be straightforward, active, and ‘measurable’ (i.e., assessable); therefore, they cannot easily capture amorphous, big-picture goals (e.g., developing a love of learning). As addressed earlier in the section, learning outcomes should be used with the acknowledgement that they cannot capture all learning that happens in a course or program.
- An exclusive focus on measurement is results or product-oriented – program outcomes are focused on the end goal of an experience, and therefore neglect what students take away from the process of achieving that end goal.
- Can be perfunctory if not engaged with in a meaningful way – it is entirely possible to write a list of outcomes just for the sake of it. Unless outcomes are tied to assessment, and are used to drive teaching and learning methods, they do not serve a real purpose.
- Potentially challenging learning curve and required commitment to establishing a new approach to course design. Orienting course design to a learning outcomes framework is a different way of thinking about a course for many instructors, which requires time and practice to feel comfortable (10).
- Orienting and educating students on outcomes – understanding learning outcomes statements is not necessarily intuitive, so it often takes a bit of work to walk students through outcomes so they understand how to read them and make use of them.
In program planning, outcomes should be aligned, or connected, with divisional and institutional goals, and with the Degree Level Expectations established by all University of Toronto divisions. Further along the renewal process, the program outcomes will be aligned with course level outcomes, as described in the introduction to this guide. The program outcomes are specific enough to explain how those broad expectations are accomplished within a given program, and course outcomes will specify what expectations an instructor has for the course, which are related to one or more program outcomes.
In the previous section, a sample vision statement was provided for the Scandinavian Department. The unit established that research needed to be a priority for the Swedish Studies program, and consequently made research a priority in the program vision. Now that the unit is determining learning outcomes for the program, they recognize the need to have one or two outcomes related to research skills to ensure the expectations for graduates represent the vision of the program. Starting from the broadest level, the DLEs, the unit would determine which expectation category is the most appropriate representation of the goals related to research (see the Vice-Provost, Academic Programs website for more information on the DLEs). Here’s an example of how the Scandinavian Department might break down the research goal at different learning outcome levels:
Level: Institution-Level Outcome (i.e., DLE category)
Example: Knowledge of Methodologies
Level: Program-Level Outcomes
Example: Students will be able to use research within the field to make evidence-based decisions
Level: Course-Level Outcomes
Example: Students will be able to critique the findings of a peer reviewed academic article
Note: the Scandinavian Department might have additional program outcomes related to research, and in turn would have several more course-level outcomes present in a variety of courses that would align with the full suite of research-based program-level outcomes.
It is unnecessary to think about course-specific outcomes at this stage in the process. However, when creating program-level outcomes, it is useful to think about how students might demonstrate achievement of them – e.g., by the time they graduate, what will they have done throughout the program to show that they’re able to use research within the field to make evidence-based decisions? Perhaps they will have created solutions for mock clients based on research findings. Perhaps they will have taken on a field research project with a faculty member and a team of students. Etc.
See Appendix A for examples of program outcomes.
Any program that has been through a UTQAP process (review or academic change) likely already has established learning outcomes. It will be essential to use those as a starting point in the curriculum renewal process and to document any changes to the learning outcomes as part of the “major modification” proposal. The Dean’s Office can provide previous UTQAP self-studies or proposals containing these learning outcomes.