Robots Proctor Online Exams

For background on this subject, see How to beat online exam proctoring. It is easy for clever students to beat existing security systems for online exams. Enterprising students could even set up "cheating rooms" that make it easy for test takers to cheat. Judging by the amount of traffic this old post gets, cheating on online exams is a serious problem.

Machine learning to the rescue! :-) The machines don't have to be 100% accurate in detection -- they can merely flag suspicious moments in the data and ask a human proctor to look more carefully. This makes the overall system much more scalable.

The monitoring data (e.g., video from webcam + pov cam) from a particular exam could potentially be stored forever. In an extreme case, a potential employer who wants to be sure that Johnny passed the Python coding (or psychometric g) exam for real could be granted access to the stored data by Johnny, to see for themselves.

Automated Online Exam Proctoring
Atoum, Chen, Liu, Hsu, and Liu
IEEE Transactions on Multimedia

Abstract:
Massive open online courses (MOOCs) and other forms of remote education continue to increase in popularity and reach. The ability to efficiently proctor remote online examinations is an important limiting factor to the scalability of this next stage in education. Presently, human proctoring is the most common approach of evaluation, by either requiring the test taker to visit an examination center, or by monitoring them visually and acoustically during exams via a webcam. However, such methods are labor-intensive and costly. In this paper, we present a multimedia analytics system that performs automatic online exam proctoring. The system hardware includes one webcam, one wearcam, and a microphone, for the purpose of monitoring the visual and acoustic environment of the testing location. The system includes six basic components that continuously estimate the key behavior cues: user verification, text detection, voice detection, active window detection, gaze estimation and phone detection. By combining the continuous estimation components, and applying a temporal sliding window, we design higher level features to classify whether the test taker is cheating at any moment during the exam. To evaluate our proposed system, we collect multimedia (audio and visual) data from 24 subjects performing various types of cheating while taking online exams. Extensive experimental results demonstrate the accuracy, robustness, and efficiency of our online exam proctoring system.

This work is related to the issued patent

Online examination proctoring system
WO 2014130769 A1

ABSTRACT
The system to proctor an examination includes a first camera (10) worn by the examination taking subject (12) and directed to capture images in subject's field of vision. A second camera (14) is positioned to record an image of the subject's face during the examination. A microphone (26) captures sounds within the room, which are analyzed to detect speech utterances. The computer system (8) is programmed to store captured images from said first camera. The computer (16) is also programmed to issue prompting events instructing the subject to look in a direction specified by the computer at event intervals not disclosed to subject in advance and to index for analysis the captured images in association with indicia corresponding to the prompting events.

Publication number WO2014130769 A1
Publication type Application
Application number PCT/US2014/017584
Publication date Aug 28, 2014
Filing date Feb 21, 2014
Priority date Feb 25, 2013
Also published as US9154748, US20140240507
Inventors Stephen Hsu, Xiaoming Liu, Xiangyang Alexander LIU
Applicant Board Of Trustees Of Michigan State University