Assessing Online Learning

Assessing Online Learning

Assessments are a crucial component of student learning. In offering students feedback and grades, we gauge their understanding of the material at hand and support their development in our courses. The best practices of assessment design are equally applicable to the creation of your online assignments.

Online Assessments and Accessibility Resource (excel file)

Designing Online Projects and Assignments

  • Provide clear and concise assignment instructions. Details are important but also consider the value of being concise to ensure students will be able to easily read and follow the instructions.
  • Provide examples or models of assessments, both high- and low-performance, to give students a clear idea of your expectations.

Feedback and Checks for Understanding

In addition to graded assignments and tests providing feedback and checks for understanding can help both you and your students evaluate their progress and understanding of the online course material.

  • Students can assess their own work through reflection and assignment guides.
  • Create activities where students can receive feedback from you as the instructor and from their peers.
  • Include short content quizzes (graded or ungraded) to check for understanding after watching a segment of a recorded lecture.
  • Use peer-review assessments to allow students to develop skills in giving and receiving feedback from peers.
  • Use the discussion board to receive student responses and feedback.

Designing Online Tests and Exams

  • Shift focus from recall-oriented questions (which may be easier to search within a text or online) to application and analysis-oriented questions which do not have an easily searchable answer.
  • Instead of one or two large, high-stakes exams, break assessment into smaller, lower-stakes tests or quizzes.
  • If using an open-book test, students will need to be prepared not only for the structure and logistics but also the purpose and intent of the concept of open-book – in other words, ensure they understand the nature of the questions that will be asked, and that open-book does not inherently mean ‘easy.’
  • Having students collaborate in small groups to complete the test can support better learning. A systematic review of Collaborative Testing offers benefits and challenges to this approach.
  • Connect exam questions closely to the learning outcomes and materials of the course, which will make it more challenging for students to locate pre-fabricated solutions or responses on the internet.
  • Design questions based on real-world or fictitious scenarios, requiring students to draw on the knowledge and skills of the course to work through the problem/case.
    • Cases or scenarios can serve as a trigger for a series of close-ended questions (e.g., multiple choice) and/or open-ended questions (e.g., short answer).
  • Provide relevant quantitative or qualitative data and ask students to interpret it – they can respond to what the data show, the relevance of the data to a problem or scenario, factors that may affect the data, etc.
  • Ensure questions are written clearly in straightforward language to reduce the time students need to make sense of what is being asked of them.
  • Have students submit their draft work/notes/etc. along with their final product (e.g., test or quiz).

Designing an online final exam

In general, CTSI does not recommend the use of Quercus online tools alone as a simple replacement for final or other high-stakes exams, as this strategy does not meet the University of Toronto policy requirements for exam-taking conditions. Attempts to replicate a traditional online exam in Quercus may lead to academic integrity breaches related to test-taker identity; access to non-approved resources and aids; and collaboration among peers; alternative final assessment approaches are recommended whenever possible. The following guidance will help you consider the potential challenges for students in order to help you determine whether a final exam is necessary, and if so, to provide guidance for configuring and administering the test to best support academic integrity.

  • Given issues related to accessibility and equity can be exacerbated in a time-constrained, ‘closed-book’ or ‘locked down’ online exam, the following elements should factor into your decision-making about developing a high-stakes online final exam:
    • Technical requirements – will your students have access to a reliable internet connection and appropriate technical set-up (hardware and software)?
    • Academic accommodations – are you able to account for any individualized academic accommodation plans for relevant students in your course?
    • Personal circumstances – are students able to set aside family and other responsibilities for the required timeframe? Have you considered the time-zone your students are working from?
    • Consider online proctoring only for unique situations or needs given the additional logistical challenges and potential equity issues related to accessing the needed technology.
  • Use test configuration options to deter academic integrity infractions:
    • Randomize questions
    • Build a test bank and schedule multiple versions of the test so students do not answer all the same questions (support considerations of equity by ensuring each version is equally challenging)
    • Set a time limit per question (e.g., 2 mins per multiple choice question)

Online tools for creating test and exams

The University offers a variety of online tools to assist with assessments for your course.

Quizzes in Quercus

The Quercus Quizzes tool can be used for automated online tests, including multiple choice and other common question types. Sharing of answers among students can be deterred through quiz configuration strategies including:

  • Randomization of questions from a large question bank
  • Configuration of time limits
  • Modularized test segments released in sequence

However, it is recommended that the Quizzes tool be used for lower-stakes or open-book assessment, or assigned less weight in the grading scheme, given completion is unsupervised and there is potential for academic integrity offences. For guidance on how to support academic integrity in online assessments, please see Academic Integrity and Teaching Online/Remotely.

Accessibility and Accommodation Options in Quercus Quizzes

 

[link to this content – UofT.me/Quercus-Quiz-Accommodation]

 

Assignments in Quercus

 

The Quercus Assignments tool can be used to create a drop-box for students to submit digital responses to one or more test or essay questions designed as “take-home” or “open-book” assessments. Assignments settings can be configured to accept a variety of file types.

Administer Take-home Exams Using the Assignments Tool on Quercus

Final Exams

 

In general, CTSI does not recommend the use of Quercus online tools alone for final or other high-stakes exams based on U of T policy, given the challenge in ensuring the appropriate conditions required by policy. Redesign of final assessments and/or adjustment of grading weighting is recommended. Notwithstanding the availability of new technologies, developing and preparing a computer-based exam, especially an online exam, requires a significant amount of advance preparation. In addition to preparing the actual assessment, instructors should consider the need for administrative logistics related to online proctoring or other computer-based assessment methods should those options be pursued to ensure the integrity of the assessment.

University of Toronto institutional policy anticipates that final exams will be conducted in a manner such that the identity of students writing exams can be verified in an appropriate manner. Further, is it important that students are only able to access approved supports or aids during the exam, as a means of ensuring no academic offences occur.

Note: Divisions or faculties who are developing procedures or strategies specific to their unique needs may reference the U of T Assessment and Grading Policy and the guidance around online proctoring offered in the Academic Integrity and Teaching Online/Remotely guide.

The University of Toronto does have recommended vendors with preferred rates forprogram areas that plan to contract a third-party platform to add additional support for academic integrity. These services may be appropriate in some program contexts.

  • Online proctoring is available through two recommended vendors, Examity and ProctorU. See Online Proctoring Guidelines for more information.
  • Computer-based testing with enhanced features has been adopted by some units within the University of Toronto through under contract with ExamSoft to provide an advanced solution for high-stakes assessments.

Grades in Quercus

Exam and final course grades should not be posted in Quercus courses.

Grades posted in Quercus allow students to view their progress, and do not represent their official final marks. Students can view their final grades on ACORN.

[link: uoft.me/q-finalgrades]

Back to Top