Missing the mark with multiple guess

It is now exam time for colleges and universities.The increased public focus on our educational systemhas increased the focus on exams, and a particularly pernicious aspect of exams:Multiple-choicequestions.

• Alberta math, science exams to be multiple choice

Worshipping a false prophet

I spent more thana decade as a college instructor, and I got to see first-hand the damage that examinationsdo to education.This damage takes many forms: the administrative workload associated with preparing,safely storing and marking exams, the psychological stress imposed on students, teachersandadministratorsand of course the blunted prospects for students who did not do well on the exams.

• After student's complaint, mental health gets a rethink at York University

But don't we need exams to make sure that students know something when they graduate?Yes,students need to "know something."

But what do students know when they do well on exams,particularly thosewithmultiple-choice questions? The simple and most accurate answer is thatthe students who do well know how to successfullywrite anexam, and of course they may have some knowledge of the subject just not as much knowledge as you might think.

Putting the guesswork into education

In my undergraduate days, we used to love what we called multiple-guess exams, because that's whatthey were.If you were a skilled guesser or knew how to skim a textbook or lecture notes, it wasrelatively easy to cram for an exam.

Remembering anything useful a week later was highly unlikely, but apassing grade on a transcript was one more step on the road to success. We'd save for later theembarrassment of actually holding down a job and having a manager or co-worker look at the fruits ofour labour and ask, "Where did you go to school?"

The thing is, there are strategies for maximizing the odds of finding the right answer. One thing you canbe certain of in a multiple-choice exam is that the correct answer is in there somewhere; you just haveto find it. The so-called 'advantage' of multiple-choice exams is that they are efficient: it does not takelong to write oneand with modern technology it takes even less time to mark one or 100.

Thedisadvantage is that it is very difficult to write "good" exam questions, and even if you do, it is evenmore difficult to write questions that evaluate high-level learning. And then there's the drawback(some would say the benefit) that you really don't need to know much to pass a multiple-choice exam.

Exams as lottery tickets

If a six-year-oldwrotea university-levelexam that had100 multiple-choice questions, each withfourpossible answers, the childcould expecttoscore 25 per cent on averageby filling inone box for each question. (Of course, it is not likelythat a six-year-old would score50 per cent or more, but as we shall see, that does not add much to the integrity of the process).

Given that random selection will yield the correctanswer 25 per cent of the time, how much do youactually have to know to score 50 per cent?

I will save you the algebra by telling you that if you know theanswer to 34 questions and guess on the other 66, odds are that you will score a 50 or better. Knowingbarely a third of the material, you get a marginal creditassuming that the questions accurately assesssubject knowledge.

OK, you might say, this regime would still result in a failing grade for 50 percent ofthe students who have mastered only a third of the course material, but why pass any of them?

But it's worse than that. Most multiple-choice exams follow a pattern of having one ridiculous answer,one highly unlikely answer, and two that require some expertise, either in language or subjectareaknowledge to sort out which is correct.

A person who has been moderately attentive during the schoolyear and done a reasonable amount of cramming will be able to find the final two pretty quickly, makingthe guessing part a 50-50 proposition.

Just what are we measuring?

Yes, there is a technique to writing exams, and students who learn itcan outperformmore skilled or knowledgeable students. Educators regard exam-writing skills as so important that in mytime at Red River College, first-year students received a booklet called Making your Markthat gave tipson how to study and how to write examswith a lot of detail on multiple-choice and true orfalse questions.

Essentially this well-meaning practice helped many students to falsely inflate their grades.Gradeinflation is only one side of this darkcoin, however; some students are not skilled at exam writingor arepsyched out by the process and do not choose options wisely. These students' grades devalue theamount of learning they have done.

I'd be upset about this inflation if I believed that exams were a reliable or even useful tool for evaluatingstudent learning. But if you believe it is a useful exercise to spend as much as a quarter of the timeallotted in a semester to writing and marking exams, ask yourself how comfortable you would beifbefore going under for heart surgery, your surgeon were to tell you, "Don't worry.I scored 96 per centon a multiple choice exam on bypass surgery."

Kevin Longfield is a retired college instructor who relied on guesswork for at least part of his universitydegrees.