On MCAS and Climate Change, Trust the Experts

by Louis J. Kruger, Professor Emeritus, Northeastern University

“It is… a mistake to accept observed test scores as either infallible or immutable” (National Research Council, 1999)

For more than two decades, controversy has swirled around the MCAS graduation requirement. At present, Massachusetts public high school students must pass tests in English language arts, mathematics and science to receive a diploma. The controversy has now reached a crescendo because the graduation requirement is a ballot question in this year’s election season. Both advocates and opponents of eliminating the graduation requirement have cited data and research in support of their positions. 

Given the confusion that can arise from conflicting claims, it is important to ask if there are trusted sources of guidance on this issue. The answer is an unequivocal ‘Yes.’ Just as climate scientists from different research organizations reached consensus on human-caused global warming, the leading experts on assessment have reached consensus on the appropriate use of educational tests, including high-stakes tests, such as the high school MCAS tests. 

This consensus is evident in the position statements published by several highly respected national professional and research groups: American Educational Research Association, American Psychological Association, National Association of School Psychologists, National Council on Measurement in Education, American Evaluation Association and National Research Council. None of them are unions. The organizations’ members include many of the leading educational assessment experts in the nation and the world. All of them have members who have conducted research on assessment and tests. The first four of these organizations publish scholarly journals, which include research articles on assessment and tests. 

These organizations are far from being anti-testing. The American Psychological Association, for example, included this preface to its position statement on high-stakes testing, “Tests, when used properly, are among the most sound and objective ways to measure student performance. But, when test results are used inappropriately or as a single measure of performance (emphasis added), they can have unintended adverse consequences.” 

Here is what these esteemed organizations said about using a test as the deciding factor in a potentially life-changing decision, such as denying a student a high school diploma.

National Research Council (NRC)

“An educational decision that will have a major impact on a test taker should not be made solely or automatically on the basis of a single test score. Other relevant information about the student’s knowledge and skills should also be taken into account.”

American Psychological Association (APA)

“…high-stakes decisions should not be made on the basis of a single test score, because a single test can only provide a ‘snapshot’ of student achievement and may not accurately reflect an entire year’s worth of student progress and achievement.”

National Association of School Psychologists (NASP)

“NASP strongly opposes the use of large scale testing as the sole determinant for making critical high-stakes decisions about students…including…receipt of a diploma…”

American Educational Research Association (AERA)

“Decisions that affect individual students’ life chances…should not be made on the basis of test scores alone.” 

American Evaluation Association (AEA)

“…important decisions…about students…should not be made on the basis of any single test or test battery, no matter how many times it may be taken.”

In addition, the APA, AERA and the National Council of Measurement in Education (NCME) have jointly developed multiple iterations of the reference book, Standards for Educational and Psychological Testing. It has been referred to as “the gold standard in guidance on testing in the United States and in many other countries” (APA, 2014). One of the principles enunciated in this book is that “a decision…that will have a major impact on a student should take into consideration not just scores from a single test but other relevant information” (Standard 12.10).

The message from the leading assessment experts could not be any clearer. A test, such as MCAS, should not be used as the sole determining factor in denying a student a high school diploma. 

Proponents of the MCAS graduation requirement often point out that high school students have multiple opportunities to pass the tests. However, that does not improve the limited validity of basing an important decision on a group-administered test. NASP stated that, “…multiple administrations of the same type of measure do not improve the reliability of the scores or reduce the general limitations of such testing” and “…myriad factors can impact the performance of any one student at a single point in time, significantly reducing the reliability of test scores.” 

Even well designed tests provide only a partial understanding of what students know and can do, and for some students a test score can be a misleading indicator of their knowledge and skills. The NRC (1997), for example. asserted that “for some students with disabilities, …a standard assessment may yield scores that are not comparable in meaning to those obtained by other students because the disability itself biases the score.“ The inherent limitations of any single test is one of the reasons the above organizations and groups advocate for the use of multiple sources of information when making high stakes decisions that impact K-12 students.

Based on reviews of relevant data and research, the organizations also voiced concern about the unintended negative consequences of using test scores to deny diplomas. 

“High stakes testing leads to under-serving or mis-serving all students, especially the most needy and vulnerable, thereby violating the principle of ‘do no harm.’” (AEA)

“High stakes testing programs can also have unintended but negative effects on the education provided to all students by narrowing the curriculum and unduly emphasizing basic skills to the exclusion of the arts …and humanities; creating a culture of “teach-to-the-test”; increasing the psychological stress on children and families; and decreasing teacher job satisfaction.” (NASP).

“The potential problem with the current increased emphasis on testing is… the instances when tests have unintended and potentially negative consequences for individual students, groups of students, or the educational system more broadly.” (APA).

“…students may be placed at increased risk of educational failure and dropping out; teachers may be blamed or punished for inequitable resources over which they have no control; and curriculum and instruction may be severely distorted if high test scores per se, rather than learning, become the overriding goal of classroom instruction.” (AERA)

Because of these concerns, the National Research Council (1999) recommended that “policy-makers should monitor both the intended and unintended consequences of high-stakes assessments on all students and on significant subgroups of students, including ..English-language learners, and students with disabilities.” In addition, according to the Standards for Educational and Psychological Testing, “it is the responsibility of those who mandate the use of tests to …minimize potential negative consequences…”.  (AERA, APA & NCME, 2014).

In Massachusetts, one documented unintended consequence is the disproportionate number of English learners, students with disabilities, and economically disadvantaged, Latinx and African-American students being denied a diploma solely because of not passing MCAS (Curtin, 2024). Another documented unintended consequence of high stakes MCAS is a disproportionately high dropout rate among students who fail the tests, especially among urban and low income students (Kruger, 2023; Papay et al., 2010 ). Other likely, unintended outcomes of the MCAS graduation requirement are the large graduation gaps among Massachusetts’ historically underserved students. Massachusetts’ graduation gap between Latinx and White students, for example, was the 5th largest in the country in 2019 (Atwell et al. 2021). In that same year, Massachusetts ranked an embarrassing 26th in the nation with respect to the on-time graduation rate of economically disadvantaged students (Atwell et al., 2021).

Two more potential unintended consequences of high stakes testing are narrowing of the curriculum and student mental health problems associated with taking and failing the high stakes tests. To my knowledge, neither has been systematically investigated in our state. After two decades of high stakes testing in Massachusetts, those who have mandated this type of testing have yet to satisfactorily investigate and address the negative consequences of their decision.

It is hard to imagine Massachusetts’ officials being summarily dismissive of the consensus of climate scientists on global warming. Then, why do they feel so emboldened to ignore the clear consensus of the leading experts on educational tests? Regardless of the reason, when it comes to MCAS, state officials have sadly abandoned their ethical responsibility ‘to do no harm’. Please vote ‘YES’ to eliminate the MCAS graduation requirement.

References

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (2014). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association.

https://www.aera.net/Publications/Books/Standards-for-Educational-Psychological-Testing-2014-Edition

American Educational Research Association (2000). Position Statement on High-Stakes Testing. Retrieved from https://www.aera.net/About-AERA/AERA-Rules-Policies/Association-Policies/Position-Statement-on-High-Stakes-Testing

American Psychological Association (2000). Appropriate Use of High-Stakes Testing in Our Nation’s Schools. Retrieved from https://www.apa.org/topics/schools-classrooms/high-stakes-testing

American Evaluation Association (2002). Position Statement of High-Stakes Testing in PreK-12 Education. Retrieved from https://www.eval.org/Policy-Advocacy/Policy-Statements/High-Stakes-Testing

Atwell, M., Balfanz, R., Manspile, E., Byrnes, V., & Bridgeland, J. (2021) Building a Grad Nation: Progress and Challenge in Raising High School Graduation Rates. Retrieved from https://eric.ed.gov/?id=ED617355

Curtin, R. (2024, March 4). Testimony at the Hearing of the Joint Committee on Initiative Petitions, Massachusetts State House. https://malegislature.gov/Events/Hearings/Detail/4883

Kruger, L. J. (2023, October). The MCAS, Dropouts and Tortured Logic. Unpublished manuscript. https://www.citizensforpublicschools.org/the-mcas-dropouts-and-tortured-logic/

National Association of School Psychologists (2003). Using Large-Scale Assessment for High-Stakes Decisions. Retrieved from https://casponline.org/pdfs/pdfs/nasp11.pdf

National Research Council (1999). High Stakes: Testing for Tracking, Promotion, and Graduation. Washington, DC: The National Academies Press. Retrieved from https://doi.org/10.17226/6336.

National Research Council (1997). Educating One and All:  Students with Disabilities and Standards-Based Reform, L.M. McDonnell, M.L. McLaughlin, and P. Morison, eds.  Committee on Goals 2000 and the Inclusion of Students with Disabilities, Board on Testing and Assessment.  Washington, DC:  National Academy Press. 

Papay, J. P., Murnane, R. J., & Willett, J. B. (2010). The consequences of high school exit examinations for low-performing urban students: Evidence from Massachusetts. Educational Evaluation and Policy Analysis, 32(1), 5–23.