Accessibility and Exam Security: Competitors or Collaborators?

Posted by Jill van den Heuvel on Dec 9, 2016 2:48:27 PM

Authors: Jill R. van den Heuvel, Ph.D., Sheryl S. Lazarus, Ph.D., and Martha L. Thurlow, Ph.D.

Note: Content is an adaptation of an article that previously appeared in Certification Magazine 

Assessments must be accessible to individuals with disabilities and non-native English language speakers while remaining secure. Credentialing organizations receive frequent exam accommodation requests. Some credentialing programs have blanket policies, such as “all candidates testing in non-English speaking countries and taking an English version of an examination receive 30 additional minutes to complete the exam.However, such blanket policies can create legal and security issues as not all candidates require the provided accommodation. Conversely, programs lacking clearly defined accommodation policies risk allegations that the program does not provide legally required accommodations.

Exam accessibility is of utmost importance to ensure valid scores; however, there are often questions about whether accommodations compromise exam security. Two vital components of ensuring exam score validity are exam security and providing accommodations to appropriate individuals.

Exam accessibility is legally required to allow all candidates possessing the required knowledge to earn a credential demonstrating that knowledge. Accommodations are changes made to “standardized” conditions to enable individuals with needs and barriers to meaningfully access an exam. Accommodations can provide equitable access and ensure a valid measure of exam constructs, which enables individuals with disabilities and English learners to demonstrate their knowledge and skills.

Laws and Professional Standards

The Americans with Disabilities Act (ADA) determined that “reasonable” accommodations must be provided to individuals with disabilities. Technical assistance guidance from the U.S.

Department of Justice in 2015 clarified that assessments must be offered in a manner that is accessible to individuals with disabilities. This includes “exams administered by any private, state, or local government entity related to applications, licensing, certification, or credentialing for secondary or postsecondary education, professional, or trade purposes (http://www.ada.gov/regs2014/testing_accommodations.html). The guidance noted that the absence of previous formal testing accommodations does not preclude a person from receiving testing accommodations.

Additionally, recent action by the U.S. Equal Employment Opportunity Commission (EEOC) has affected special populations. The Commission began a public comment period on February 24, 2016, on a proposed rule related to Section 501 of the Rehabilitation Act of 1973 related to affirmative action in employment for individuals with disabilities. The current Section 501 regulations indicate that the federal government should be a “model employer of individuals with disabilities”; however, the guidelines regarding the specifics of this statement are insufficient. The proposed rule would more clearly define this statement to include the percentage representation of individuals with disabilities across pay levels (to include higher and lower levels of employment) to be employed by the federal government and federal government contractors. The proposed rule would also require agencies to include disabilities in anti-harassment policies. This proposed rule could require more individuals with disabilities to seek certification in various areas; thus, it is critical to address this potential increase in the number of candidates who request accommodations proactively.

In addition to federal laws and court cases, several sets of professional standards address the issue of accessibility. The 2014 revision of the Standards for Educational and Psychological Testing, published jointly by the American Educational Research Association (AERA), American Psychological.

Association (APA), and National Council on Measurement in Education (NCME), indicated that comparability in the interpretation of scores from accommodated and non-accommodated exams is essential, and rests on clearly defining the intended construct to be measured.

Two primary sets of standards are specific to credentialing exams: the National Commission for Certifying Agencies (NCCA) (the accreditation division of the Institute for Credentialing Excellence [ICE]) and the American National Standards Institute (ANSI). ICE released a revised version of the Standards for the Accreditation of Certification Programs in 2014 for use beginning in 2016; these standards contain requirements for accommodations and accommodation requests. In 2004, ANSI published the Guidance on Psychometric Requirements (2004), which provides accreditation standards guidance related to ISO/IEC 17024 (Section 4.3.6). These standards state, “A policy regarding the implications of testing irregularities, and a policy regarding accommodations, modifications, and adaptations, is essential. Guidelines are required for understanding the most likely effects of irregularities, accommodations, or changes upon the interpretation of resultant scores (p. 5).

Lessons Learned from Educational Assessments

Little research has been conducted specific to licensure and certification exams on how to maintain test security while providing accessibility. Therefore, it is useful to look at the educational assessment policy analysis that the National Center on Educational Outcomes (NCEO) conducted in 2014. (The complete analysis is available at: http://www.cehd.umn.edu/NCEO/OnlinePubs/Synthesis95/SynthesisReport95.pdf.)

At the K-12 level, exam security policies often address how to administer accommodations without compromising exam security. These policies can provide insights into how certification exam policies might be structured. NCEO found that the most common accommodations addressed in test

security policies were braille, transcription and scribing, read aloud by a human reader, and sign language interpretation. Slightly fewer than half of the states included extended time in their policies. The analysis also found that the policies of about half the states addressed training requirements for accommodations providers and administrators of accommodated tests.

Recommendations

The answer to “What should I do to ensure that I have a method to grant accommodation requests appropriately and maintain the security of my exam?” is straight-forward: have policies and processes in place to handle accommodation requests, and ensure the delivery vendor can provide accommodations in a secure environment. All test administrators, accommodations providers, and other parties involved in preparing and administrating accommodated assessments should be required to sign test security and confidentiality agreements to help improve test security. Additionally, coordinate with the delivery vendor to ensure that common accommodation software is available on machines in accessible locations. This software must maintain appropriate functionality without modifying other settings (i.e., accessing the Internet). In cases where a human access assistant is required, make sure that this assistant is appropriately trained, has signed relevant documents, and understands the effects of compromising item security and modifying items during delivery.

Develop accommodation policies during exam development and refine them throughout the exam lifecycle. Carefully consider the items and structure, types of accommodations likely to be requested/needed, and implementation of accommodations without sacrificing security or the intended meaning of exam scores. Consult content and accommodation/disability experts. Content experts can evaluate construct changes as a result of an accommodation, and disability/accommodation experts can document best practices for incorporating appropriate accommodations without risking exam security.

Lessons learned from educational assessment security policies, as well as the literature, can provide some basic guidelines for including accommodations in policies. These lessons help to ensure security in many ways; one of the most overarching ways is a shared understanding of the role of accommodations within assessment. Once the test owner understands the need for accommodations, it is easy to apply security policies that work in coordination with the accommodation policies. Some of the lessons include:

Publish requirements regarding accommodations (e.g., dates, required materials, and policies for challenging decisions) in an area available to all eligible candidates.

Standardize the request and review process.

Document the accommodation request review process (e.g., required materials/information, inappropriate/irrelevant information, communication of decisions, standardized application to all requests).

Log all requests and review the logs with experts to refine policies/procedures.

Individually review and process all requests within the defined accommodations policies. All candidates should be treated the same. However, blanket policies should be avoided because they could include individuals who do not require a particular accommodation.

Author Bios:

Jill R. van den Heuvel, Ph.D., is a Psychometrician with Alpine Testing Solutions, Inc. who focuses on Information Technology certification exams. Her research interests include: accommodations, security, and psychometrics applied to small samples.

Sheryl S. Lazarus, Ph.D., is a senior research associate at the National Center on Educational Outcomes (NCEO) at the University of Minnesota, and is the co-principal investigator for the National Assessment Center. Her research interests include accessibility and accommodations, and test security.

Martha L. Thurlow, Ph.D., is a senior research associate and the director of the National Center on Educational Outcomes (NCEO) at the University of Minnesota. In this position, she addresses the implications of contemporary U.S. policy and practice for students with disabilities and English Learners, including national and statewide assessment policies and practices, standard setting efforts, and graduation requirements.

Jill van den Heuvel

Written by Jill van den Heuvel