Cheating detection in applicant selection

Since the usage of unproctored internet testing (UIT) is increasing rapidly, it is of interest to find a reliable way to detect cheating in those tests. Paradata, which records when the testee leaves the testpage, could be used for cheating detection in applicant selection.

In online testing, cheating can be a serious issue. The testee can switch tabs or browser windows, and can use powerful search engines like google or libraries like Wikipedia to quickly find the answers to knowledge questions. The speed in which this is possible, and the fact that the testee needs an online computer to work with the Hogrefe Testsystem (HTS) in the first place can pose a serious incentive to cheating. Cheating is easy, and it cannot be controlled for. The HTS is basically a web application. It's a highly sophisticated one, using almost all the ressources available to a modern browser, but it still suffers the limitations a web page has. As such, it cannot control anything outside it's own window. There is no way to find out if or which other browser windows are open, or what the user does besides taking the test (chatting, listening to music or of course cheating).

One thing a web page can control for, however, is if it's currently focused. That means: if the window is the user's current window. Since the HTS runs fullscreen, the only way to loose focus is if the user "tabs" out using a predefined global hotkey. If the user leaves the window, an "onblur" event is called. If the window regains the focus, an "onfocus" event is called. By logging these events, we could theoretically gather information about how often, during which items, and how long the user changed windows. This information might give us clues on whether the testee cheated during the test.

A panel study was conducted as part of a master thesis at the University of Goettingen. The aim of this study was to evaluate within an experimental design whether PageFocus data proves to be a valid measurement for cheating in tests for applicant selection. The experiment reveals that PageFocus could be used as an additional indicator for cheating detection. Furthermore, it shows that PageFocus can be found, as predicted, within performance tests which are commonly used in selection settings. It can be seen as a generalized phenomenon and is not restricted to special types of tasks like knowlegde questions.

Seminare zum Thema

Drawbacks and interpretation

This technology can in no way identify cheaters for sure. Since not every tab event is an attempted cheat, it is difficult to interpret them. The Testee might be reading his ICQ messages. Or he might tab out to turn off his music and messengers, so he can concentrate on the test. Also, it doesn't prevent users from working on two computers (e.g. a laptop and a desktop PC, or a Pc and a mobile phone) simultaneously. It is up to the diagnostician and his judgement of the situation. Implementing a system to help identifying cheaters will in many ways contribute to improve the HTS.

What could give us hints on whether the testee cheated is the pattern of his tabs. The testee will most probably show one of two possible patterns: he will either tab out most of the items, to be absolutely sure to get correct answers on every item. Or, especially if a test has limited time, he will answer easier questions himself, showing no tabs on easy items, sporadical tabs on medium-difficulty items, and a lot of tabs on difficult items. In many tests, items increase in difficulty. So the testee will show more tabs as he progresses through the items. Tab times might - depending on the test - get longer, as the answers get more complex and require more reading. A careful investigation and consideration of the pattern of tabs might actually be our best bet to see if a testee has cheated.

 

Read more: https://hts.hogrefe.de/group/issues/6372