Developments on the Inkblot Test: Scoring

After Hermann Rorschach’s sudden death in 1922, his inkblot test began to take on a life of its own as it traveled around the world. Psychologist David Levy is credited with bringing the Rorschach to America in 1927, after he learned about the test while studying abroad with Rorschach’s friend, psychoanalyst Emil Oberholzer. Then in the 1930s, two psychologists located in the United States—Samuel Beck, who studied under Levy, and Bruno Klopfer—created divergent scoring systems for the Rorschach test. 

An American Evolution

Samuel Beck valued a quantitative approach, and used the Rorschach in the practice of orthopsychiatry at the Boston Psychopathic Hospital (Beck, 1930). He created ratios of color and movement responses in an attempt to quantify the perceptions made from inkblots. Beck’s formal method was first outlined in the 1937 monograph, Introduction to the Rorschach method: a manual of personality study. In 1944, Beck published part II. 

On the other hand, Bruno Klopfer valued a qualitative approach. Inspired by the work of Carl Jung, Klopfer focused more on symbolism and phenomenological experiences (Weiner, 2018). In 1942, he published his scoring system in the book, The Rorschach technique. Kloper and Beck’s opposing interpretations did not go unnoticed, which led to a professional rift between the two. Notable tiffs included Kloper’s critical review of Beck’s 1937 book on the Rorschach method, which garnered a reply article from Beck, straining the relationship between the two (Weiner, 2018). 

Over the years, several other psychologists developed their own interpretations of the Rorschach. And by 1957 five different Rorschach scoring systems (Beck 1937, Klopfer-Kelley 1942, Hertz 1952, Rapaport-Schaffer 1954, & Piotrowski 1957) were in circulation, all originating from the United States (Flanagan, 2006).  

The Exner System

In the 1960s, John Exner began to study the Rorschach technique. His research resulted in the 1969 book, The Rorschach Systems. Exner—executive director of the Rorschach Research Foundation—observed the strengths and weaknesses of the five major scoring systems. He found that the systems used different procedures, data collection, and scoring methods. The Rorschach Research Foundation also surveyed clinician’s use of the Rorschach and found that there was little consistency in scoring practices and methodology (Stricker and Lally, 2015). 

Then in 1974, Exner published his own scoring system under the title, The Rorschach: A comprehensive system. In doing so, Exner integrated bits and parts of the five major scoring systems in an attempt at unifying the field of practice (Zaccario and Gordon, 2018). The comprehensive system improved the test’s empirical and psychometric standards, and would become widely accepted as the gold standard for Rorschach interpretation. The system underwent extensive revisions that spanned four editions, including two volumes on advanced interpretation (Exner 1974, 1978, 1986, 1991, 1993, 2003). 

Psychologist Rosemary Flanagan wrote a review on the fourth edition of the comprehensive system and highlighted the systemic and data-driven approach to the Rorschach test. From an administrative standpoint, the practitioner collected initial responses to the inkblots, and then specified the data based on an inquiry phase. The second inquiry phase helped to identify the location of the subject’s perception on the inkblot and the determinants of the content (Flanagan 2006, Jenny, 2011). Additionally, the comprehensive system detailed a procedure for data interpretation. 

All in all, Flanagan concluded that while the system was not without its flaws—a lack of availability and transparency of psychometric data—the comprehensive system added a systematic, data driven approach to the Rorschach—a test that’s invaluable due to its in vivo method in the study of personality (Flanagan, 2006). 

Critically Acclaimed

James Wood—a prominent critic of the comprehensive system—conceded that Exner’s various editions:

• “established detailed, objective rules for administration, scoring, and interpretation of the Rorschach; 

  • For example, the systematic approach detailed in the response and inquiry phases enabled more consistent data acquisition and scoring procedures, which reduced subjective elements of interpretation. 

• cataloged extensive data regarding the interrater reliability of the scales; 

  • Standardization of the Rorschach procedure helped raters obtain more consistent results both within and across groups of examiners/practitioners. 

• provided norms and reference data for numerous psychiatric and nonpsychiatric groups, including children; and 

  • Normative scores created a benchmark for comparative studies and statistical operations related to analysis and interpretation of individuals, groups, and cultures. 

• cited numerous empirical studies to support the validity of Comprehensive System scores.” (Wood et al., 1996, p. 3)

  • This issue is not unique to the Rorschach test or any of its scoring systems. Researchers often publish results from a single study with less than optimal sample sizes. Validity should be a reproducible measure. 

Holtzman Inkblot Technique 

Outside of the core Rorschach developments, other psychologists were interested in the use of inkblot tests for the study of personality. In 1961 psychologist Wayne Holtzman and colleagues published their inkblot technique in the book, Inkblot perception and personality: Holtzman Inkblot Technique. 

Their main goal was to preserve the qualitative richness of the original Rorschach test, and also “overcome psychometric limitations in the Rorschach by constructing completely new sets of inkblots” (Holtzman, 1968, p. 136). Specifically, Holtzman summarized four major quantitative issues with the Rorschach method at the time: 

• Inconsistencies in examiner administration and inquiry styles. There were five major scoring systems for the Rorschach by the late 1950s. The lack of a universally accepted Rorschach method led to vast confusion and severe irregularities when the test was empirically verified. And yet, this same flexibility led to its favorability in clinical practices. However, variable inquiry phases—which ultimately identify scoring categories—destabilized the Rorschach’s consistency and reliability. 

• Poor measures of internal consistency—the degree to which test items correlate in measuring the same construct—or retest reliability—the correlation between test scores over time. These issues apparently stemmed from the immense variability (i.e., lack of equivalency) of the original Rorschach inkblots themselves. Another compounding factor was the lack of standardized parallel forms. 

• Disagreement in scoring criteria for specific categories. Perhaps the Rorschach’s reported poor measures of interrater reliability were symptoms of rater scoring disagreement.  

• Too much variation in the number of responses to the original set of ten inkblot cards (Holtzman, 1968). This resulted in a lack of norms as well as variable interpretations. 

7 Key Differences

Holtzman noted seven critical distinctions between his inkblot technique and the original Rorschach. 

• Number of inkblots: While the traditional Rorschach test has ten inkblots, the Holtzman Inkblot Technique has forty-five blots, and two practice cards (Holtzman, 1968, p. 136). The addition of more inkblots increased overall reliability of the test and afforded more information for the examiner. 

• Parallel forms: Holtzman’s technique has two parallel forms, resulting in a total of ninety-two potential standardized responses (forty-five cards + two practice cards + A & B forms) that fit into twenty-two scoring categories. These parallel forms enabled test-retest measures of reliability to be conducted as well as measures of individual response change. 

• Characteristics of inkblots: The Holtzman inkblots have greater variation in color, form, and shading; making them a comparatively richer form of test stimuli. The addition of varied inkblot characteristics increased the richness of potential responses. 

• Symmetry of inkblots: The Holtzman inkblots are not all symmetrical. This created a distinct metric for inkblot analysis. On the other hand, the original Rorschach inkblots were created with symmetry as a prerequisite for proper artistic composition, with Hermann Rorschach himself noting that subjects would commonly reject (i.e. provide an inadequate response) to asymmetrical images (Rorschach et al., 1942). The new variable of balance was created by comparing measures of symmetry and asymmetry among the inkblots. This was measured by a subject’s expressed concern for an inkblot’s given symmetry or asymmetry (Holtzman, 1968). 

• Responses per card: One psychometric “issue” with the Rorschach stems from the variability of responses to a given card. Unlike the Rorschach—where subjects respond at will—the Holtzman technique only demands a single response. The limitation of responses (i.e., consistency)  helped to produce stable norms for individuals and populations. 

• Inquiry phase: The original manuscript on the Rorschach’s procedure does not explicitly detail an ‘inquiry phase’, while the Exner comprehensive system’s inquiry phase occurs after completion of the response phase. Holtzman’s technique conducts each inquiry immediately after each response phase. Holtzman argued that an immediate inquiry phase enabled the examiner to better encode the location of the subject’s percept on the inkblot and the determinants of the content. 

• Standardized percentile norms: Holtzman reported percentile norms across diverse populations. This enabled the analysis of percentile ranks and standard deviations. 

• Methods of administration: The Holtzman technique can be administered to groups of individuals. Ironically, the inkblot cards were projected onto a screen, as participants wrote down responses in a standardized workbook. Group studies enabled rapid testing within and across cultures at large scales. 

Universal Dimensions of Inkblot Perception

Perhaps the most interesting aspect of Holtzman’s technique was the results produced by a factor-analysis on the test’s variables. Holtzman found that a set of common dimensions accounted for nearly “all the significant common variance present among the twenty-two variables.” (Holtzman, 1968, p. 150)

In plain English, Holztman’s twenty-two variables—regardless of culture or age—were consistently found to produce the following dimensions in the perception of his inkblots across samples (Darolia, 2016). 

• Perceptual maturity and integrated ideational activity. Holtzman reported that a high score in this dimension would reflect imaginative capacity, distinct ego boundaries, and awareness of conventional concepts (Holtzman, 1968). This dimension has been correlated with “cognitive ability, creativity potential, reading comprehension among children” (Darolia, 2016, p. 271). 

• Perceptual sensitivity. Color and shading were noted to have an inverse relationship with form definiteness, meaning that emphasis on one tends to result in a lack of emphasis on the other. For example, a person who is not concerned with form as a determinant from the physical inkblot should also overreact to color and shading. 

• Psychopathology of thought. For Holztman, high levels of this dimension indicated disordered thought, bizarre perception, and active fantasy life. Interestingly, studies showed that high scores of bizarre perception in adults indicated either schizophrenia or a response with artistic content (Darolia, 2016). 

• Perceptual differentiation. As detailed by professor of psychology C.R. Darolia, “Holtzman et al. (1961) believed that fragmentation of the inkblot into smaller parts generally tends to be associated with more accurate form, and a greater variety of concepts with good form are possible only when smaller areas of the inkblot are used.” (2016, p. 272) For example, an inkblot will tell you about a subject’s perceptions of form, color, and shading, as well as the manner in which they organize these perceptions into personal meanings. In short, the authors propose that a variety of meanings tend to come from the smaller details of the inkblot, rather than the whole interpretation.

• Perceptual inability. This dimension addressed a subject’s response time and whether or not they rejected a given inkblot. This is a given similarity between Holtzman’s technique and Rorschach’s original method for testing inkblots.

References:

Beck, S. J. (1930). Personality diagnosis by means of the Rorschach test. American Journal of Orthopsychiatry, 1(1), 81–88.

Beck, S. J. (1937). Introduction to the Rorschach method: a manual of personality study. American Orthopsychiatric Association Monographs, 1, xv + 278.

Beck, S. J. (1944). Rorschach's test. I. Basic processes. Grune & Stratton.

Darolia, C. R. (2016). Holtzman Inkblot Technique and Personality Assessment. The Wiley handbook of personality assessment. Wiley Blackwell. 264-268.

Exner, J. E. (1969). The Rorschach systems. New York, Grune & Stratton.

Exner, J. E. (1974). The Rorschach: A comprehensive system. John Wiley & Sons. 

Exner, J. E.. (1978). The Rorschach: A comprehensive system: Vol. 2. Current research and advanced interpretation. New York: John Wiley & Sons.

Exner, J. E. (1986). The Rorschach: A Comprehensive System (2nd ed.). New York: John Wiley. 

Exner, J. E. (1991). The Rorschach: A Comprehensive System. Volume 2: Interpretation (2nd ed.). New York: John Wiley. 

Exner, J. E. (1993). The Rorschach: A Comprehensive System (3rd ed.). New York: John Wiley. 

Exner, J. E. (2003). The Rorschach: A Comprehensive System (4th ed.). New York: John Wiley. 

Flanagan, R. (2006). Review of The Rorschach: A Comprehensive System [Review of the book The rorschach: A comprehensive system, by J. E. Exner]. Journal of Psychoeducational Assessment, 24(2), 166–171.

Jenny. E. B. (2011). Rorschach Inkblot Method. In: Goldstein S., Naglieri J.A. (eds) Encyclopedia of Child Behavior and Development. Springer, Boston, MA. 

Klopfer, B., & Kelley, D. M. (1942). The Rorschach technique. World Book. 

Hertz, M. R. (1952). Frequency tables for scoring Rorschach responses. Cleveland, OH: Western Reserve University Press.

Holtzman, W. H., Thorpe, J. S., Swartz, J. D., & Herron, E. W. (1961). Inkblot perception and personality: Holtzman Inkblot Technique. Texas Press.

Holztman, W. H. (1968). Projective Techniques in Personality Assessment: Holtzman Inkblot Technique. Germany: Springer Berlin Heidelberg. 136-172. 

Meyer, G. J., Hilsenroth, M. J., Baxter, D., Exner, J. E., Jr, Fowler, J. C., Piers, C. C., & Resnick, J. (2002). An examination of interrater reliability for scoring the Rorschach Comprehensive System in eight data sets. Journal of personality assessment, 78(2), 219–274.

Mihura, J., Bombel, G., Dumitrascu,N., Roy, M., & Meadows, E. (2019). Why We Need a Formal Systematic Approach to Validating Psychological Tests: The Case of the Rorschach Comprehensive System, Journal of Personality Assessment, 101:4, 374-392. 

Piotrowski, Z. (1957). Perceptanalysis. New York: Macmillan.

Rorschach, H., Lemkau, P., Kronenberg, B. (1942). Psychodiagnostics : a diagnostic test based on perception, including Rorschach's paper. The application of the form interpretation test (published posthumously by Dr. Emil Oberholzer). Berne, Switzerland: H. Huber; New York: Grune & Stratton.

Schaefer, R. (1954). Psychoanalytic interpretation in Rorschach testing. New York: Grune & Stratton.

Stricker, G., Lally, S. (2015). Projective methods in psychology. International Encyclopedia of the Social & Behavioral Sciences (Second Edition). 129-134. 

Weiner, I. (2018). Society for Personality Assessment/Journal of Personality Assessment: A History, Journal of Personality Assessment, 100:1, 2-15.

Wood, J. M., Nezworski, M. T., & Stejskal, W. J. (1996). The comprehensive system for the Rorschach: A critical examination. Psychological Science, 7(1), 3–10.

Zaccario M.L., Gordon R.M. (2018). Rorschach. Encyclopedia of Clinical Neuropsychology. Springer, Cham.