Recently, a program director who works with me asked why he needed to proctor his students when they take an exam. The test is administered to students across the country, once a year. In his words “these aren’t kids; they’re adults in a graduate training program. Why do they need a babysitter?”
I was a little stunned. Like most test developers, I spend a lot of time making sure I communicate and follow “the rules” of testing. I work hard to make sure item writers are well trained, test items are strong, users and administrators have the materials they need to administer the exam, and results analyzed correctly. Clearly, I’ve been taking some things for granted. Here’s the answer I would give if I had time to do it over (or write a blog before I answered):
“Without a proctor monitoring the test environment, there is no way to guarantee that we are making fair comparisons of test takers.”
We do a lot of analyzing to make sure our tests are valid (measure what we say they measure) and reliable (measure the same construct, consistently).
But, standardizing test administrations is critical to the inferences we make about tests and proctoring is the key to ensuring that standardization.
Don’t just take my word for it. Here’s what the Standards for Educational and Psychological testing1 (aka, The Standards) say:
“When directions to examinees, testing conditions, and scoring procedures follow the same detailed procedures, the test is said to be standardized. Without such standardization, the accuracy and comparability of score interpretations would be reduced. For tests designed to assess the examinee’s knowledge, skills, or abilities, standardization helps to ensure that all examinees have the same opportunity to demonstrate their competencies. (Standard 6)”
The Standards go on to say
“Maintaining test security also helps to ensure that no one has an unfair advantage.”
In other words, to interpret test scores similarly, it is important that all students are given equal opportunity to do their best on the test and, regardless of where or when they take the test, students receive the same instructions, have the same amount of time to complete it, complete it in similar environments (e.g., quiet and light), and interpret the questions in the same way.
And yes, Virginia, sometimes even adults cheat.
There is an industry built around detecting and preventing cheating and it is successful because there is an industry built around cheating. Researchers find that cheating begins as early as elementary school and it is not just the students. In certification and licensure, there are highly organized theft rings that take exams to memorize and/or record content and sell it online. Content experts impersonate and take the test for exam candidates.
So, why proctor high stakes tests?
Having trained test proctors is the best way to ensure that the test administration is standardized. Scores are only as good as the conditions in which the tests were taken. After all the care and vigilance we put into developing assessments, why wouldn’t we want to administer them with the same care and vigilance?
1American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (1999). The Standards for Educational and Psychological Testing. Washington, DC: AERA.