The Test Taker’s Right to Know

Continued article from the January 2005 Issue of The Behavioral Measurement Letter, Vol. 8, No. 2  Winter 2005

Robert Perloff, PhD

It is my powerful intuition, reinforced subjectively by anecdotes and hearsay, that in many quarters there is oracular resistance to the test-taker’s/examinee’s right to know how he or she did, performed, stacked up, scored, or was assessed on a psychological test, attitude survey, personality checklist, diagnostic tool or other (generally) paper-and-pencil instrument designed to describe, measure, or evaluate how test­ takers acquitted themselves on instruments of one sort or another.

Opposition to the Release of Test Scores

As a fellow of Division 5 (Evaluation, Measurement, and Statistics) of the American Psychological Association (APA), an erstwhile specialist in psychometrics, and a former APA President, but most of all as a fervid champion of civil rights and of the first Amendment to the United States constitution, I have long been persuaded that the pas de deux between an individual’s personhood and his or her test scores or results (even if provided in a typical way) of a medical examination, e.g., blood pressure, electrocardiogram, or urinalysis, is binding, inviolate, never to be severed, never!

And yet, while I have not gathered empirical evidence supporting my contention, I am convinced—indeed, dead certain—that many aficionados in evaluation and measurement frown upon providing a person with a frank and unexpurgated, though properly explained to, and interpreted for the lay test-taker, an account of his or her test score. A likely reason offered for this resistance is that a test score is complex—but an electrocardiogram isn’t complex? —and it would confuse the test-taker to be confronted with a perplexing array of “ifs, ands, or buts.” Why throw test-takers into a deep funk by frightening them with the showy psychometrician’s jargon of the likes of standard deviation, stanines, percentiles, probable error, and reliability?’ In my judgment, these terms should be viewed by the tester as a challenge, not as a difficulty or as a reason to conceal test scores from the test-taker. If the tester is unable to describe a test score to a high school student, a college sophomore, a soccer mom, or to a guy who, at another time of the week is a patron of a sports bar watching Monday Night Football, then the tester should refrain from administering the test in the first place!

The only legitimate exception against providing test­ takers with their test scores would be if test-takers themselves, of their own volition—without being prompted by the tester to refrain from seeing their test scores—opted not to be given their test scores. Of course, this mandate to provide test feedback applies only to tests of established validity, and not to tests of unknown validity. After all, if it is unclear what the test is measuring, then there is no trustworthy feedback to communicate to the test­ taker.

The Case for Release of Test Scores

Test results should be unexceptionally (save for the above reason) given to test-takers, jargon-free and comprehensibly. But for the foregoing exception there is no condition, circumstance, or context in which test-takers should not be given their test results. “I’ll let you take a snapshot of me. but you’ll have to show me the picture”—otherwise, no deal. Either provide me with comprehensible test feedback, or I will not take the test. It’s as simple as that.

After all, a test is invasive, puncturing the test-taker’s behavioral or psychological epidermis; it probes for, measures, and reports intimate and significant aspects of one’s core being. Would it not be objectionable, if the results of medical tests were withheld from the patient? Likewise, it is objectionable that psychological test scores would be withheld from the test-taker. An organization or institution afraid or reluctant to reveal test scores is exposing itself to the accusation that it doubts the validity, relevance, or dependability of the test score. If you’re confident about the verisimilitude of the procedure, you should share your findings with the examinee. What’s to hide? Let it all hang out, the good, the bad, and the indifferent. It’s “show and tell” time: if test-takers are willing to show their responses, then the tester is obliged to” tell test-takers what scores their responses were transformed into, and what the scores mean.

In today’s zeitgeist of transparency, accountability, and full disclosure—nothing hidden or kept back—it is the universal and unalienable right that test­ takers must have guaranteed for them by those of us who poke around in every nook and cranny of the test-taker’s imperfect and sometimes gnarled, wrinkled, and remorseful (or remorseless) psyches, their right to know what bits of information we uncovered about them.

Summary

In this paper, I have hoisted a flag on behalf of the proposition that unless test-takers distinctly specify that their test results not be shown to them, then, testers, test administrators, and organizations—colleges, universities, employing organizations, and other manner of assessors—responsible for administering tests are obliged to share test results with test-takers, test-takers (examinees), and college sophomores who are research participants.

The information provided to test-takers should be simple, clear, unambiguously interpreted, and impervious to misunderstanding. It is unmitigated arrogance, unforgivable imperiousness, despicably authoritarian, and palpably undemocratic for testers to presume to withhold test results from test-takers.

Robert Perloff, Ph.D., is a Distinguished Service Professor Emeritus of Business Administration and of Psychology in the Joseph M. Katz Graduate School of Business at the University of Pittsburgh. Dr. Perloff is a past President of the American Psychological Association and a recipient in 2000 of the American Psychological Foundation’s Gold Medal Award for Lifetime Achievement in Psychology in the Public Interest. Email: rperloff@katz.pitt.edu

“There is no science which does not begin by requiring you to believe the incredible.”

– Henry Adams

Editorial Afterthoughts

Fred B. Bryant, PhD

In his newsletter article, Dr. Perloff advocates a general principle that social and behavioral scientists have largely ignored—namely, the unalienable right of test-takers to receive feedback about their test scores. His article reflects an impo1tant and thought­provoking perspective that is bound to create controversy. Dr. Perloff’s ideas also raise many questions that his article does not address (given the space limitations of this newsletter), questions that must be answered, however, if we are to implement effectively the policies he advocates. As Dr. Perloff argues, these technicalities should be seen not as insurmountable obstacles, but rather as challenges to be faced and overcome. To stimulate further discussion of his ideas, below we note six specific sets of questions that arise in response to Dr. Perloff’s article. Although we offer no answers to these questions, we invite readers to respond with their own ideas.

1. What specific information should be communicated to test-takers? Dr. Perloff makes a strong case_ for the general rule that test administrators should report “test scores” or “test results” to provide “comprehensible feedback” to test-takers, but he does not specify exactly what form this report should take. Although he exhorts test administrators to avoid “showy psychometrician’s jargon” and “let it all hang out, the good, the bad, and the indifferent,” this test feedback could take many different forms with varying levels of specificity. For example, imagine a respondent completed the Thematic Apperception Test (TAT) in the context of a clinical interview. Should the test administrator tell the respondent the clinical interpretation of each response to each TAT stimulus card, or merely the overall score and what it indicates? Should clinicians tell test-takers where their scores fall in a normative sample, in terms of the percentage of the general population that scores higher or lower? What sort of qualifications, if any, should clinicians include regarding the reliability and validity of any one test-taker’s score? Should test administrators report this feedback verbally or in writing? Dr. Perloff’s brief article is not intended to provide definitive answers to these questions, but rather to stimulate further thought about the issues that must be addressed directly if his ideas are to be implemented effectively.
2. Who is responsible for providing test feedback? Exactly whose responsibility is it to report test feedback to test-takers? Is this always the responsibility of the individual or organization administering the test? Imagine, for example, an industrial-organizational setting in which a business company retains the services of a professional test publishing company to analyze and interpret scores on a pre-employment screening test, and to make hire/no hire recommendations based on test scores. Whose responsibility is it to provide test feedback to test-takers: (a) the organization that administers the test, or (b) the test publisher who developed, validated, analyzed, and interpreted the test scores for the organization? In some cases, the organization may rely entirely on an outside test developer to score and interpret the test, and in these cases, no one in the organization may be qualified to provide test feedback. In such a case, whose responsibility is it-the tester’s or the test developer’s-to provide test feedback?
3. Who should receive test feedback? In situations in which the test-taker is a minor (e.g., a young child) or a person with impaired capacity to comprehend test feedback, then who should receive the feedback about test scores? Should test administrators be required to provide young children with feedback that these children can understand? Should parents or guardians receive this test feedback instead? Should both children and parents receive test feedback?
4. How can test administrators be sure that test­ takers actually understand the test feedback provided? Just because test-takers receive test feedback does not guarantee that they understand it. In general, how can test administrators know that test-takers have obtained an adequate understanding of their test scores? Should test administrators be required to evaluate the level of comprehension of test feedback? Should test administrators be required to continue their efforts to provide feedback, if evidence indicates test-takers do not understand? Is it enough simply to provide test feedback, or must test administrators persist in their communications until each test-taker understands test results to some minimally acceptable degree? And how would one determine whether or not one has established this minimally acceptable degree of comprehension?
5. Are there possible exceptions in specific assessment situations? What about the proprietary rights associated with certain test interpretations (such as the Rorschach, TAT, or employee honesty tests), in which test administrators have at stake the loss of a “trade secret,” if they were to reveal their interpretations of test responses to test-takers? Indeed, test developers may have spent a great deal of money to develop test items and sco1ing protocols that allow valid prediction of on-the-job performance. Because providing feedback about test-score interpretation could potentially destroy the utility of the test (if test-takers can then coach others on how scores are interpreted), it might be argued that withholding such feedback from test-takers in such cases is a “business necessity.” Don’t test-takers give up their unalienable right to test feedback when they take such tests in the workplace, particularly if they are informed of this restriction in freedom of information beforehand? Can’t an employer make the absence of test feedback to the test-taker a precondition to taking the test, and tell test-takers they are free to walk away and look for another job elsewhere, if they wish not to take the test under such conditions? The key here is that test-takers understand when they apply for the job that they are required to waive their right to test feedback, and that if they do not wish to waive this right, then they are free to look for work elsewhere.
6. Is there a possible compromise position? To extend Dr. Perloff’s suggestions, a researcher or other test administrator who wants to withhold test feedback from respondents should be required to add a clause to the informed consent form notifying potential respondents, before they decide whether to take the test, that they will not receive an interpretation of their test scores; and that if they object to this arrangement, then they are free to terminate participation without penalty. This might offer a reasonable compromise to the rule of always requiring feedback about test scores. It would also make for a more “informed” consent in which potential respondents are told, before deciding whether or not to take a test (not afterwards, as is currently done), that they either will or will not receive test feedback. To deny respondents the right to such feedback later, without explicitly informing them of this fact before they consent to take the test, is a breach of APA’s ethical principles concerning the need for fully informed consent.

“The Nazz, stompin’ on a sweet, swingin’ beat, goin’ down the road. The Nazz talkin’ about

How pretty the flowers,

how pretty the hours,

how pretty me, how pretty you,

how pretty he, how pretty she.

Nazz had them pretty eyes. He wanted everybody to see through his eyes so they could see how pretty it was.”

– Lord Buckley

Read additional articles from this newsletter:

Comparing Measurement in the Natural and Behavioral Sciences

Doris Bloch Remembered

Selecting Tools For Use in Cross-Cultural Measurement