UNITED STATES DEPARTMENT OF EDUCATION OFFICE FOR CIVIL RIGHTS

THE ASSISTANT SECRETARY

DECEMBER 2000

Adherence to good test use practices in education is a shared goal of government officials, policy-makers, educators, parents, and students. In an era of school reforms that place increasing emphasis on measures of accountability, such as the use of tests as part of decision-making that has high-stakes consequences for students,* the need to provide practical information about good testing practices is well documented. In January 1999, the National Research Council (NRC) observed that we, in the education community, should work to better disseminate information related to good testing practices with a focus on the standards of testing professionals and the relevant legal principles that, together, “reflect many common concerns.”

Sound educational policies and federal nondiscrimination laws can work together to promote educational excellence for all students and ensure that educational practices do not — intentionally or otherwise — unfairly deny educational opportunities to students based upon their race, national origin, sex, or disability. In short, federal civil rights laws affirm good test use practices. Thus, an understanding of the measurement principles related to the use of tests for high-stakes purposes is an essential foundation to better understanding the federal legal standards that are significantly informed by those measurement principles.

In order to further the goal of accurate and fair judgments in high-stakes decision-making that involves the use of tests, we are pleased to provide you with this copy of The Use of Tests as Part of High-Stakes Decision-Making for Students: A Resource Guide for Educators and Policy-Makers. This guide provides important information about the professional standards relating to the use of tests for high-stakes purposes, the relevant federal laws that apply to such practices, and references that can help shape educationally sound and legally appropriate practices.

There are few simple or definitive answers to questions about the use of tests for high-stakes purposes. Tests are a means to an end and, as such, can be understood only in the context in which they are used. The education context — in which the relationship (and attendant obligations) of the educator to the student is frequently more complex than that between employer and employee — shows time and again that any decision regarding the legality of a use of a test for high-stakes purposes under federal nondiscrimination laws cannot be made without regard to the educational interests and judgments upon which the test use is premised.

Background

Throughout the 1990s, national, state, and local education leaders focused on raising education standards and establishing strategies to promote accountability in education. In fact, the promotion of challenging learning standards for all students — coupled with assessment systems that monitor progress and hold schools accountable — has been the centerpiece of the education policy agenda of the federal government as well as many states.

At the same time, the use of tests as part of high-stakes decision-making for students is on the rise. For example, the number of states using tests as a condition for high school graduation is increasing, with a majority of states projected to use tests as conditions for graduation by 2003 and several states now using tests as conditions for grade promotion.

Recently, more and more educators and policy-makers have requested advice and technical assistance from the U.S. Department of Education regarding test use in the context of standard-based reforms. The Department’s Office for Civil Rights (OCR) is also addressing testing issues in a more extensive array of complaints of discrimination being filed with our office, most of them in a K-12 setting with implications for high-standards learning. OCR has responsibility for enforcing Title VI of the Civil Rights Act of 1964, Title IX of the Education Amendments of 1972, Section 504 of the Rehabilitation Act of 1973, and Title II of the Americans with Disabilities Act of 1990. These statutes prohibit discrimination on the basis of race, color, national origin, sex, and disability by educational institutions that receive federal funds.

In a similar vein, institutions in the post-secondary community in recent years have engaged in a thoughtful dialogue and analysis regarding merit in admissions and the appropriate use of tests as part of the process for making high-stakes admissions decisions. In some states, the use of tests in connection with admissions decisions has been an important element in public post-secondary education reform.

These trends highlight the salience of two recent conclusions of the NRC’s Board on Testing and Assessment. The NRC observed that many policy-makers and educators are unaware of the test measurement standards that should inform testing policies and practices. These standards include the Standards for Educational and Psychological Testing (Joint Standards), prepared by a joint committee of the American Psychological Association (APA), the American Educational Research Association (AERA), and the National Council on Measurement in Education (NCME). The NRC also concluded that it “is essential that educators and policy-makers alike be aware of both the letter of the laws and their implications for test takers and test users.” [National Research Council, High Stakes: Testing for Tracking, Promotion and Graduation, p.68 (Heubert and Hauser, eds., 1999).]

The Resource Guide

Toward this end, OCR has prepared this guide in an effort to assemble the best information regarding test measurement standards, legal principles, and resources to help educators and policy-makers frame strategies and programs that promote learning to high standards in ways consistent with federal nondiscrimination laws. Our goal is to inform decisions related to the use of tests as part of decision-making that has high-stakes consequences for students, such as when they move from grade to grade or graduate from high school. Just as we know that good test use practices can advance high standards for learning and equal opportunity, we know that educationally inappropriate uses of tests do not. If we want this generation of test-taking students (and their teachers and schools) to meet high standards, then we should insist that the tests they take meet high standards. When tests are used in ways that profoundly shape the lives of students, they must also be used in ways that accurately reflect educational standards and that do not deny opportunities or benefits to students based on their race, national origin (including limited English proficiency), sex, or disability.

The guide is organized to provide practical guidance related to the test measurement principles and applicable federal laws that guide the use of tests as part of decision-making that has high-stakes consequences for students. The Introduction to the guide provides a broad, conceptual overview of relevant principles so that those who are not familiar with test measurement principles or applicable federal laws can better understand the kinds of issues that relate to the use of tests in many contexts. Chapter One of the guide provides a detailed discussion of the test measurement principles that provide a foundation for making well-informed decisions related to the use of tests for high-stakes purposes. The Joint Standards, which has been approved by the APA, AERA, and NCME, is discussed in detail in this chapter. Adherence to relevant professional standards can help reduce the risk of legal liability when schools are using assessments for high-stakes purposes. Chapter Two provides an overview of the existing legal principles that have guided federal courts and OCR when analyzing claims of race, national origin, sex, and disability discrimination related to the use of tests for high-stakes purposes. These principles, as applied by the courts and OCR, underscore the importance of adhering to educationally sound testing practices. The Appendix includes a Glossary of Legal Terms, a Glossary of Test Measurement Terms, a list of Accommodations Used by States, a Compendium of Federal Statutes and Regulations, and a Resources and References section.

Central Principles

There are several central principles reflected in the text of this guide.

First, the goals of promoting high educational standards and ensuring nondiscrimination are complementary objectives. The ultimate question regarding the use of tests for high-stakes purposes, as a matter of federal nondiscrimination law and sound educational policy, centers on educational sufficiency: Is the test appropriate for the purposes used? That is, are the inferences derived from test scores, and the high-stakes decisions based on those inferences, valid, reliable, and fair for all students? In applying civil rights laws to education cases, federal courts recognize the importance of providing appropriate deference to the educational judgments of educators and policy-makers. These inquiries are not an effort to lower academic standards or alter core education objectives integral to academic admissions or other educational decisions. Rather, these inquires focus the educator and policy-maker on ensuring that uses of tests with high-stakes consequences for students are educationally sound and legally appropriate.

Second, when tests, including large-scale standardized tests, are used in valid, reliable, and educationally appropriate ways, their use is not inconsistent with federal nondiscrimination laws. Importantly, tests can help indicate inequalities in the kinds of educational opportunities students are receiving, and, in turn, may stimulate efforts to ensure that all students have equal opportunity to achieve high standards. When tests accurately indicate performance gaps, it is important to focus on the quality of educational opportunities afforded to under-performing students. The key question in the context of standards-based reforms and the use of tests as measures of student accountability is: Have all students been provided quality instruction, sufficient resources, and the kind of learning environment that would foster success?

Third, a test score disparity among groups of students does not alone constitute discrimination under federal law. The guarantee under federal law is for equal opportunity, not equal results. Test results indicating that groups of students perform differently should be a cause for further inquiry and examination, with a focus upon the relevant educational programs and testing practices at issue. The legal nondiscrimination standard regarding neutral practices (referred to by the courts as the “disparate impact” standard) provides that if the education decisions based upon test scores reflect significant disparities based on race, national origin, sex, or disability in the kinds of educational benefits afforded to students, then questions about the education practices at issue (including testing practices) should be thoroughly examined to ensure that they are in fact nondiscriminatory and educationally sound.

In short, the goal of the federal legal standards is to help promote accurate and fair decisions that have real consequences for students, not to water down academic standards or deter educators from establishing and applying sensible and rigorous standards. In fact, properly understood, the legal standards are an aid to meaningful education reform — by helping to ensure that instruction and assessments are aligned and structured to promote the high-level skills and knowledge that rigorous standards seek for all children.

Finally, while this guide focuses on the use of tests, similar principles apply to the overall decision-making process used to make high-stakes decisions for students. In fact, the NRC, APA, AERA, NCME, and others caution against making high-stakes decisions based on a single test score. “Other relevant information should be taken into account if it will enhance the overall validity of the decision.”[ Joint Standards, p.146 (1999).]

Conclusion

Recognizing the responsibility that educators and policy-makers must shoulder in making the promise of high-standards learning a reality, U.S. Secretary of Education Richard Riley in his commemoration of the 45th anniversary of the Brown v. Board of Education decision said, “A quality education must be considered a key civil right for the twenty-first century.” This is the driving force behind OCR’s continuing effort to provide assistance to policy-makers and educators as we continue to enforce federal laws that prohibit discrimination against students. Rather than creating false and polarizing “win-lose” choices on this all-important set of issues, we need to, as Secretary Riley noted, “search for common ground” — ground, that is, in this case, expansive.

We have worked with literally dozens of groups and individuals, including educators, parents, teachers, business leaders, policy-makers, test publishers, individual members of Congress, and others, to solicit input and advice regarding the scope, framing, and kinds of resources to include in this guide, and we are grateful for their time and assistance. The first draft of the testing guide was released in April 1999 and was the subject of substantial comments leading to extensive revisions. The second draft was released in December 1999 and once again received substantial comments. That draft also was independently reviewed by the NRC’s Board on Testing and Assessment, which held a hearing earlier this year to discuss the draft guide and issued a letter report in June 2000 commenting on the draft. We are grateful for the NRC’s tireless efforts. The third draft was released for public comment in July 2000, this time with notice of availability in the Federal Register. OCR has made numerous changes throughout the guide in response to comments seeking to clarify, make more accurate, or expand key sections. It is important to keep in mind that the guide is not designed to answer all questions related to the use of tests when making high-stakes decisions for students. However, working together with our education partners, we believe that we are providing a useful resource that will serve the education community as it addresses the very complex and important questions that stem from the institution of high standards and accountability systems designed to promote the best schools in the world.

Very truly yours,

/s/ Norma V. Cantú

Norma V. Cantú

* As explained throughout the guide, the primary focus is the use of standardized tests or assessments (referred to in the guide as tests) used to make decisions with important consequences for individual students. Examples of high-stakes decisions include: student placement in gifted and talented programs or in programs serving students with limited English proficiency; determinations of disability and eligibility to receive special education services; student promotion from one grade level to another; graduation from high school and diploma awards; and admission decisions and scholarship awards. The guide does not address teacher-created tests that are used for individual classroom purposes. BACK