This information is also available to download as a printable pdf:
Multiple-Choice-Question-and-Quiz-Tips.pdf
Vocabulary
Multiple choice questions consist of a stem or question, and alternatives or options including the correct response and incorrect distractors.
Stem or Question
Narrow openings of the eyelids, deep set eyes, a shortened distance between the nose and upper lip, and a thin upper lip in a young child are typical of:
Alternative or Options
A) Myhre syndrome
B) the effects of thalidomide
C) Down’s syndrome
D) Rubella during the child’s prenatal development
The Correct Response or Keyed-Response (the best answer) is A) Myhre syndrome.
The Incorrect Responses or Distractions are B) the effects of thalidomide, C) Down’s syndrome, and D) Rubella during the child’s prenatal development.
Best Practices: The Stem
The stem is the portion of a multiple choice question that poses a problem. The stem should be best solved or answered by only one of the options presented. It is normally either written as a question or an incomplete statement. Questions are preferred over incomplete statements as they decrease the likelihood of creating test item flaws that will clue test-wise students.
Best practices for writing question stems include:
A good stem is one that clearly poses a problem or question. After reading a stem, students should be able to come up with the correct answer on their own because the question and purpose of the question is clear and complete.
Poor stem: What did Freud believe?
Better stem: What did Freud believe dreams represent?
Wording stems negatively (e.g., using “NOT” in the question) can be unnecessarily confusing, causing the question to become an exercise in reading comprehension rather than an assessment of learning. If a question must be stated negatively, be sure to EMPHASIZE the negative statement. Use bold, all caps, and underline or italics to have it stand out.
Poor stem: What is not the function of the superego?
Better stem: What is the function of the superego?
When using incomplete statements, position the blank at the end of the stem. Putting the blank in the beginning or middle of the stem increases the cognitive load of completing the question.
Poor stem: The ______ psychosexual stage is when the superego develops.
Better stem: The psychosexual stage in which the superego develops is the ______.
The stem should contain only information that is necessary to answer the question. Including extraneous information extends the time required for students to read the question and can make it less clear what the question is hoping to address.
Poor stem: The latent stage of psychosexual development is one in which the libido is dormant. It is during this stage that children can put their energy toward school, play, and hobbies. At what age does this stage occur?
Better stem: At what age does the latent stage of psychosexual development occur?
Best Practices: The Alternatives
The alternatives or options are the full suite of possible answers made available to students. This includes one correct response or keyed-response (best answer) and the incorrect responses or distractors.
Writing plausible distractors is the most difficult part of writing multiple choice questions. It is important that all alternatives are plausible.
While only 3 alternatives are needed for a multiple choice question, 4 alternatives is standard practice. Writing additional plausible distractors can be challenging. Adding a 5th distractor only decreases the probability of students guessing the correct answer by 5%.
Be sure to alternate the location of the correct response and put the alternatives in a meaningful order. If there is a meaningful order to the alternatives (e.g., numeric, chronological), organize the alternatives in that order. Or use alphabetical order to help ensure that the alternatives are randomized.
Common Multiple Choice Flaws
Common multiple choice flaws clue test-wise students to the correct answer—without reliance on their learning. Being aware of these mistakes can help ensure that your quizzes are assessing student learning, rather than student test-taking skills.
Some of the most common flaws, as presented by Berk (1998)(3), are shown below. Note that an asterisk (*) is used to emphasize test item flaws.
Repeating actual or associated words in the stem and correct response. Ensure words are not repeated or that all alternatives use related words.
1. Charles Dicken’s Hard Times deals with
a) the difficult life of a factory worker.*
b) the politics of the French Chateau country.
c) the court of Kind Edward’s existentialism.
Ensure that the grammatical structure between the stem and all alternatives is appropriate.
A biologist who specializes in the study of the relationships of an organism to its environment is known as an:
a) Ecologist*
b) Structuralist
c) Taxonomist
d) Naturalist
The correct response is often longer and more complex than the distractors. Ensure that all alternatives are of similar length and complexity.
One of the main reasons for Custer’s defeat at the Little Big Horn was the fact that:
a) He split his troops into three groups, making each group too small to effectively fight*
b) The weather was too hot
c) He had sick horses
d) He wanted to be president
If two alternatives are opposites, this may indicate that one of them is correct. If two alternatives are synonyms it indicates that both are incorrect.
The treaty of Brest Litovsk was ratified by Moscow because:
a) Tzar Alexander I wanted to prevent Napoleon’s invasion of Russia.
b) Russia was unable to keep up with the armament manufacture of Austria.
c) Russia could not keep pace with the military production of Austria.
d) Nicolai Lenin wanted to get the Soviet Union out of WWI.*
The planarian has:
a) an anterior brain.*
b) red eyes.
c) three legs.
d) a posterior brain.
If the alternatives are from different cateogries, it can clue testwise students. They need to look only for the alternative that is not related to the other options. Ensure that all alternatives are in the same category.
Which Canadian castle has served as a filming location for the movies X-men, Chicago, Scott Pilgrim vs the World and Titans?
a) Mont Saint-Michel
b) Edinburgh Castle
c) Windsor Castle
d) Casa Loma*
Overlapping choices can lead to confusion. Ensure that alternatives are mutually exclusive.
In which age range does the oral stage of psychosexual development take place?
a) Birth to 1 year old*
b) 1 to 2 years old
c) 1 to 3 years old
d) 3 to 6 years old
e) 5 years to adolescence
Use of absolute terms (e.g., always, never, only) often clues an incorrect response. Either avoid the use of absolute terms or ensure they are applied in each alternatives.
To avoid infection after receiving a puncture wound to the hand, you should:
a) Always go to the immunization center to receive a tetanus shot.
b) Be treated with an antibiotic only if the wound is painful.
c) Ensure that there are no foreign objects left in the wound.*
d) Never wipe the wound with alcohol unless it is still bleeding.
“All of the above” is sometimes used when the test writer is struggling to develop distractors. When applied as the correct response, students who can identify two likely alternatives will be inclined to select “all of the above”. When “all of the above” is presented as an incorrect response, students who identify one alternative as incorrect can also rule out “all of the above”.
“None of the above,” as either the correct or incorrect response, increases question difficulty without increasing discrimination between high and low-scoring students. If students can identify a likely option, “none of the above” is eliminated, reducing the number of distractors.
Writing Multiple-Choice Items that Demand Critical Thinking
Questions that call on students to demonstrate critical thinking and deep learning ask students to apply their learning to new contexts, analyze situations and draw connections among ideas or facts, demonstrate a rationale for a chosen approach, and solve real–world problems.
When designing such questions, it can be helpful to think through what is the desired impact of your course on students 2-3 years into the future and how they will use what they have learned in their professional roles. This can help in designing questions that emphasize critical thinking, creative use of course knowledge in disciplinary contexts and solving real-world problems.
Some possible question formats include:
Present a mini-case study to students. A single, well-written paragraph can provide material for several follow-up questions. Questions can focus on an aspect of the case study, an interpretation of evidence, or require students to evaluate possible solutions.
Use pictures, diagrams, charts, tables or figures that require students to interpret, apply course concepts, or explain relationships between components.
Present a problem and proposed solution in the stem that students are asked to evaluate based on criteria provided. Alternatively, have students analyze the problem and evaluate the correct solution or next step in a process.
Example 1: Students are asked to evaluate the response to a problem.
A student was asked the following question: “Briefly list and explain the various stages of the creative process.”
As an answer, this student wrote the following:
“The creative process is believed to take place in five stages, in the following order: orientation, when the problem must be identified and defined; preparation, when all the possible information about the problem is collected; incubation, when no solution seems in sight and the person is often busy with other tasks; illumination, when the person experiences a general idea of how to arrive at a solution to the problem; and finally verification, when the person determines whether the solution is the right one for the problem.”
How would you judge this student’s answer?
a) EXCELLENT (all stages correct in the right order with clear and correct explanations)
b) GOOD (all stages correct in the right order, but the explanations are not as clear as they should be)
c) MEDIOCRE (one or two stages are missing OR the stages are in the wrong order, OR the explanations are not clear OR the explanations are irrelevant)*
d) UNACCEPTABLE (more than two stages are missing AND the order is incorrect AND the explanations are not clear AND/OR they are irrelevant)
Example 2: Students are asked to interpret a ficticious situation and apply their course learning to that context.
Wilma and Fred were arguing over the furniture while they were settling their divorce agreement. Wilma said to Fred, “I never like that chair anyway.” At which of Knapp’s relational conversation levels is this statement?
a) Integrating
b) Differentiating*
c) Sustaining
d) Initiating
Multiple Choice Test Design
When integrating multiple choice questions into assessments that form a significant component of the course grade, it is important to carefully think through the test structure. Much of this advice can also be applied, in a limited way, to low and no stakes quizzes.
To determine how many questions should focus on each topic, begin by making a list of the topics and their corresponding importance in the course. Ensure the proportion of test questions associated with that topic aligns with the importance and/or class time devoted to the topic.
Create test items over a term and build on them through successive course offerings. Because these questions can be difficult to write well, it is best not to attempt to write them in a single sitting. If the course text includes a question bank, carefully read over the questions for test item flaws before adopting them in your course.
Testing can be particularly stress-inducing for students. Help to reduce this stress by ensuring that the first and last questions on the test are questions that most students can answer correctly.
When students study, they tend to focus on one course topic at a time in the order they were presented in the course. Mimic this with your tests by presenting all questions related to one topic before moving on to the next. This is also best practice for accessibility. Consider grouping related items under headings to help students see the overall test structure. If you have multiple versions of a test, maintain the structure but vary questions within each grouping or shift the placement of the correct response in multiple choice items.
When considering how challenging multiple choice questions should be, ensure that there is sufficient variety within your test so that all students have the opportunity to demonstrate their learning. A rule of thumb is that approximately 10% of students should be able to answer the most challenging questions that form approximately 5% of the test, and 90% of students should be able to answer the easiest questions that form approximately 5% of the test.
DuQuesne University (n.d.) Good, Better, Best: Multiple Choice Exam Construction. Retrieved Feb 7, 2019.
Brame, C., (2013) Writing good multiple choice test questions. Retrieved February 11, 2019.
Berk, R. (1998) A Humorous Account of 10 Multiple-Choice Test-Item Flaws that Clue Testwise Students. Journal on Excellence in College Teaching 9(2), 93-117.
Pachai, M. V., DiBattista, D., & Kim, J. A. (2015) A Systematic Assessment of ‘None of the Above’ on Multiple Choice Tests in a First Year Psychology Classroom. The Canadian Journal for the Scholarship of Teaching and Learning, 6(3).
Bethel Academic Enrichment & Support Centre (n.d.) Test-Wiseness Clues. [PDF file]. Retrieved Feb 7, 2019.
University of Oregon Teaching and Learning Center (n.d.) Writing Multiple-Choice Questions that Demand Critical Thinking.
Center for Distributed Learning, University of Florida. (n.d.) 10 Examples of Question Improvements. [PDF file].