Thursday, April 25, 2013

How to beat online exam proctoring


Part of the potential of online education is to break the "credentialing chokehold" of traditional universities. But in order for a credential to have value, one has to be sure that the holder has actually mastered the subject matter. Thus, security in testing is important. Certainly, students can cheat at traditional universities, but the problem becomes much more severe for online-only education in which the educational institution may never have physical contact with the student. A security hole in an online proctoring system can be exploited wholesale, by thousands of people in different locations. (See also Magical Mystery Moocs and Whither Higher Education?)

Here is how to beat the online proctoring systems described in a recent Chronicle of Higher Education article.

Attach a second monitor to the test taker's computer (e.g., via a long cable) which is visible only to a hidden confederate, or otherwise arrange to have the image on the main screen visible to the hidden confederate.

The confederate works out problems and transmits answers via, e.g., tapping on the test-taker's leg, using a long stick (not visible to webcam). Another possibility is for the confederate to use a laser pointer (or other projection device) pointed at the wall behind the test taker's monitor. The confederate could also just hide under the desk/table at which the test taker sits. Most of these methods will work on multiple choice tests, but the projection method could even work on essay or programming tests. A commenter also suggests running multiple virtual machines (VMs) on the test taker's computer, one for the testing app and the other for secret communication. A quick toggle on the keyboard will make this difficult to detect by existing security measures.

Eventually I can imagine students setting up special "test (cheating) rooms" for this purpose. Ideally the monitoring company should obtain POV data from the test taker to defeat these methods.
Chronicle: ... The old biases against online education have begun to erode, but companies that offer remote-proctoring services still face an uphill battle in persuading skeptics, many of whom believe that the duty of preserving academic integrity should not be entrusted to online watchers who are often thousands of miles from the test-takers. So ProctorU and other players have installed a battery of protocols aimed at making their systems as airtight as possible.

The result is a monitoring regime that can seem a bit Orwellian. Rather than one proctor sitting at the head of a physical classroom and roaming the aisles every once in a while, remote proctors peer into a student's home, seize control of her computer, and stare at her face for the duration of a test, reading her body language for signs of impropriety. ...

15 comments:

Paul said...

A Google Glass-style setup would do the trick. Equip the testing rooms with cell jammers?

steve hsu said...

Yes, every "test (cheating) center" should offer machines with multiple VMs and keyboard toggle that switches the screen :-)

Javier said...

As an avid consumer of MOOCs, I think cheating concerns are overblown and online test proctoring is a waste of resources. For example, a typical Computer Science student will take dozens of tests throughout the course of completing a degree. Yes, someone could cheat on all those tests, but a gross cheater would then be hard pressed to pass a good 30-min technical interview. Instead of proctoring every one of those tests with imperfect methods, it makes more sense to infrequently interview students to ensure their skills are consistent with their coursework.

A separate issue is that cheating concerns interfere with optimal learning. A great way to learn how to code is to study what other coders have produced, after you have worked on the problem yourself. However, most online courses discourage students from posting code on the forums, even after the deadlines have passed. This is annoying. I wish that more MOOCs adopt the philosophy: "If you cheat, you're only cheating yourself".

Hao Ye said...

"For example, a typical Computer Science student will take dozens of tests throughout the course of completing a degree. Yes, someone could cheat on all those tests, but a gross cheater would then be hard pressed to pass a good 30-min technical interview. Instead of proctoring every one of those tests with imperfect methods, it makes more sense to infrequently interview students to ensure their skills are consistent with their coursework."

This is fine for employers, but bad for schools that offer online courses -- if the only test that matters is the interview, then being credentialed by the school is meaningless and fewer people will pay for courses. I agree that cheating concerns are probably overblown, but I do think there are economic incentives to try and catch cheaters (or at least to deter would-be cheaters).

Hauser Quaid said...

I know people who cheated on ALL exams on my university, and most students have cheated at least once.

Bobdisqus said...

Who would be so dumb as to run their system on common cell frequencies? A modest fee puts a pair of Asian PHDs to the task looking over the button hole video feed.

Better go to at least a Faraday cage and RF monitoring systems. Think of the money to be made selling that QE system that can beat it.



http://www.imdb.com/title/tt0090685/

Javier said...

I agree that the credential would lose value, but the education still holds a lot of value for students if that is what enables them to land and keep a job. Also, schools could separate the credential from the teaching, as Salman Khan has argued. Just a few tests per discipline could serve as a credential; there's no need to proctor every test or censor discussion in the forums.

gide07 said...

It was a lot harder to take my GMAT than get on a plane.

cnnm said...

Pretty dumb article. Cheating doesnt have to be that complicated (Like you said tapping on legs, using laser pointers, wtf haha)... All you can do is mute your mic and the person can damn shout the answers to you like I did.

zxoiesru said...

Hello, I am a smart person and I never cheated on a test in my life. However, due to this extreme violation of privacy, I am considering cheating on ProctorU just to prove to myself that I can. So congratulations. Because you assumed all students are guilty and violated their privacy trying to get them to not cheat, even though 99% of people don't actually cheat, well guess what you have INCREASED cheating and by making students think harder you're going to make it easier to get away with it and less likely to "slip up".

Gene said...

but isnt proctoru able to see your monitor remotely?
what if another person remotely logs into your computer while youre taking the test. then proctoru also logs into your computer. will your friendly remote be detected by proctoru?
thanks

Anonymous said...

Not if you use the same program they do .

kiyoko960 said...

I work at ProctorU! We have a video that when you think it's paused or muted, it is, but on your end. But We can still see and hear everything. If you are gonna invest all that energy in cheating, why not switch it up and invest it in studying. What's the point in a higher education if you aren't really trying to learn?!

MHDVM said...

My son just had to take a test last night with Proctor U. He knew the material. He does not test well due to ADD and a tendency to panic at the first unsure answer. I don't use the "F" bomb often
but he was so "F" intimidated by the pre test exam of his surroundings and the fact that whatever they use slowed our internet speed so much that he was knocked off line 3 times during the test, that he totally jacked-up the test. We thought this methodology would be less of a problem for him than classroom testing but the crime scene type investigation and overall assumption that he or anyone like him is guilty of cheating before any evidence of that happening left him, as I am sure many others, too stressed to do even close to their best on the exam being proctored. I have gone through 8 years of college and quite a a bit more continuing ed since and I saw cheating all along the way. You know where most of thise individuals are now ? not where they wanted to be because they did not have the knowledge to survive the real world. My feeling is that they are only hurting themselves and the fruit of their labor will end in a dismal employment downhill spiral. And to those who would counter by saying let people steal or commit other crimes - those are crimes affecting others and should be prevented and punished - not the same issue at all.

MHDVM said...

Bottom line - why Proctor at all?

Blog Archive

Labels