Security: it’s about validity!

Posted by Emily Barnes on Mar 1, 2014, 12:58:20 AM

March 2014
Security: it’s about validity!

 

Many testing programs recognize the importance of security, but do not take action against individual test takers because of the time and cost incurred in ‘proving’ allegations such as fraud, cheating, or misconduct. By definition, fraud and cheating require intent; which can be very difficult to establish. Further complicating the matter, aspects of cheating, misconduct, inappropriateness, and unfairness are interpreted differently across regional, cultural, and even generational lines. (For example, “piracy” and “theft” to some are “sharing” and “collaboration” to others.)

Although taking action against an individual test taker can be onerous, allowing a test taker to achieve a score through fraudulent means or other forms of what the testing program would consider unfair advantage is also problematic. A program could have compelling validity evidence that provides a high degree of confidence in the interpretations and uses of test scores. But if a test taker achieves a score through fraudulent or other unfair or unanticipated advantage, the validity of the interpretations and uses of that candidate’s test score is compromised. Over time, as test taker after test taker slips through, the program damages its reputation and ultimately defeats the purpose of using the test.

The Standards[1] state, “Validity refers to the degree to which evidence and theory support the interpretations of test scores entailed by the proposed uses of the tests” (p. 9). Furthermore, “Test validity rests on the assumption that a test taker has earned fairly a particular score or pass/fail decision” (p. 86) and “Any form of cheating or behavior that reduces the validity and fairness of test results should be investigated promptly, and appropriate action taken” (p. 89). Through these statements, and many others not cited here, the Standards contemplate the possibility of a candidate compromising the validity of the interpretations of that candidate’s score and expect the test sponsor to take appropriate action.

If programs recognize the primary goal of security is to promote fairness and validity, programs can better focus detection and enforcement efforts. Testing programs can systematically require additional validity evidence or implement other enforcement actions for candidates based on concerns related to the validity of the interpretation of the candidate’s individual test score. Such an approach eliminates the time, cost, and difficulty of establishing the test taker’s intent and/or debating differences across regions, cultures, and generations. It bolsters fairness because all candidates are treated the same for violation of a priori parameters regardless of the root cause and whether an advantage was gained knowingly through intentional wrong doing or unwittingly without intentional wrong doing. Ultimately, it supports the goal of promoting validity without the burden of having to ‘prove’ cheating, fraud, or misconduct.

Additional posts will continue the discussion of security and look in more detail at prevention, detection, enforcement, policy, and legal considerations.

 


[1] American Educational Research Association, American Psychological Association, National Council on Measurement in Education (1999). Standards for Educational and Psychological Testing. Washington, DC: AERA

Brian Adams is the President and CEO of Alpine Testing Solutions, Inc. His interests include performance testing solutions, efficient test development processes, test security solutions, and finding the mythical meeting ground of business constraints and sound testing practices.

 

Topics: Our Perspective

Emily Barnes

Written by Emily Barnes