"To give, say, a detailed exegesis of Locke's empiricism or Engel's dialectical materialism without carefully identifying the empirical and conceptual problems that those doctrines were designed to resolve is not unlike playing one of those parlor games in which one is given an answer (often a bizarre one), without knowing the question to which it is an answer! One can only understand a system of ideas when one knows, in detail, the problems to which it was addressed" (Laudan, 1977, pp. 175-176)
In order to assess progress in measurement theory, it is important to identify the perennial problems encountered in the assessment of human characteristics. Though precise definitions of measurement problems are dependent on the research tradition within which the measurement theorist is working, these problems can be used to define criteria for comparing different measurement theories, and also provide a framework for examining progress toward their own solution.
E. L. Thorndike (1904, 1916) stands out as a major source of ideas about educational and psychological measurement, and he identifies some of the fundamental problems in this area. "In the mental sciences, as in the physical, we have to measure things, differences, changes and relations" (1904, p. 4). In order to develop scales like those in the physical sciences, Thorndike (1916) proposed five essentials of valid scales: (1) objectivity, (2) consistency, (3) definiteness of the facts and their differences, (4) comparability with the facts to be measured, and (5) reference to a defined zero point.
One of the major measurement problems encountered in the behavioral sciences is the lack of objectivity in scales. For Thorndike, a perfectly objective scale is one on the meaning of which all competent thinkers agree. In order to obtain objective scales, he envisioned the construction of a set of standard items calibrated onto a scale that would be used as a common measuring stick. He describes a scale developed by Hillegas (1912) for the measurement of quality in English composition as an example of a "foot-rule for merit in English composition" (1916 p. 18). Hillegas selected a set of essays and had them rated by a group of experts ("competent thinkers") in order to calibrate them.
"The series of facts used as a scale must be varying amounts of the same sort of thing or quality" (1916, p. 13). For Thorndike, measurement problems related to consistency were so obvious that they did not need further comment. From a current perspective, consistency seems closest to the concept of unidimensionality.
Thorndike's third essential of a valid scale is definiteness. He sought the development of ideal scales with steps of equal difficulty between the calibrated objects on the scale. He also recognized that equal distances were not necessary, so long as the differences were fixed and known on some well-defined scale.
Comparability addresses measurement problems related to the use of calibrated scales. The usefulness of a scale is increased when the objects to be measured can be compared easily with the calibrated objects that define the scale. Comparisons are more precise and accurate when the objects to be measured are similar. "To measure the beauty of a drawing of an eagle by comparing it with the series [of drawings] a, b, c, etc., and observing to which point of the series it was nearest in respect to beauty, would be much easier if the series consisted of drawings of the same eagle, than if the series consisted of drawings of ships" (1916 p.15).
Finally, Thorndike points out that it is essential to know how the zero point is defined. Is it an arbitrary or theoretically defined zero? Clearly, the definition and location of the zero point will influence the interpretation of the scores obtained with a scale.
Thorndike's approach places him in the "scaling" tradition with which Rasch measurement is closely connected. In contrast, in the next column, I will examine Wood's principles of educational measurement. Though these were intended to be an elaboration of Thorndike's essentials, Wood subtly changed the definitions of these problems to initiate what today we call the "test score" tradition.
Hillegas MB 1912. A scale for the measurement of quality in English composition by young people. New York:Teachers College
Laudan L 1977. Progress and its problems: Towards a theory of scientific growth. Berkeley, CA:University of California Press
Thorndike EL 1904.An introduction to the theory of mental and social measurements. New York:Teachers College, Columbia University. 1916. Revised and enlarged edition.
Thorndike's Essentials of a Valid Scale, G Engelhard Jr. Rasch Measurement Transactions, 1991, 5:3 p. 170
|Rasch Measurement Transactions (free, online)||Rasch Measurement research papers (free, online)||Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch||Applying the Rasch Model 3rd. Ed., Bond & Fox||Best Test Design, Wright & Stone|
|Rating Scale Analysis, Wright & Masters||Introduction to Rasch Measurement, E. Smith & R. Smith||Introduction to Many-Facet Rasch Measurement, Thomas Eckes||Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr.||Statistical Analyses for Language Testers, Rita Green|
|Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar||Journal of Applied Measurement||Rasch models for measurement, David Andrich||Constructing Measures, Mark Wilson||Rasch Analysis in the Human Sciences, Boone, Stave, Yale|
|in Spanish:||Análisis de Rasch para todos, Agustín Tristán||Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez|
|Forum||Rasch Measurement Forum to discuss any Rasch-related topic|
Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement
Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.
|Coming Rasch-related Events|
|Feb. 28 - June 18, 2022, Mon.-Sat.||On-line course: Introduction to Classical and Rasch Measurement Theories (D. Andrich, I. Marais, RUMM), The Psychometric Laboratory at UWA, Australia|
|Feb. 28 - June 18, 2022, Mon.-Sat.||On-line course: Advanced Course in Rasch Measurement Theory (D. Andrich, I. Marais, RUMM), The Psychometric Laboratory at UWA, Australia|
|May 20 - June 17, 2022, Fri.-Fri.||On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com|
|June 24 - July 22, 2022, Fri.-Fri.||On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com|
|Aug. 12 - Sept. 9, 2022, Fri.-Fri.||On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com|
|Oct. 7 - Nov. 4, 2022, Fri.-Fri.||On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com|
|June 23 - July 21, 2023, Fri.-Fri.||On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com|
|Aug. 11 - Sept. 8, 2023, Fri.-Fri.||On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com|
The URL of this page is www.rasch.org/rmt/rmt53j.htm