Meaningfulness, Measurement and Item Response Theory (IRT)

There is a basic principle of meaningfulness accepted by a wide cross-section of different philosophical viewpoints that justifies the use of fundamental measurement models like Rasch's to the exclusion of IRT models. That principle states that meaning is an abstract fictional ideal approximated to the extent that it is separable from the (metaphorical, geometrical, numerical, historical, etc.) figures representing it. According to the deconstructionist Jacques Derrida (1982, p. 229), for instance, "... the sense aimed at through these figures [of metaphor] is an essence rigorously independent of that which transports it." Similarly, the hermeneuticist Paul Ricoeur (1977, p. 293) concurs, saying "No philosophical discourse would be possible, not even a discourse of deconstruction, if we ceased to assume what Derrida justly holds to be 'the sole thesis of philosophy,' namely 'that the meaning aimed at through these figures is an essence rigorously independent of that which carries it over.'" Gadamer (1980, p. 100) also agrees, saying "It is clear to us that the figure which we draw to illustrate a mathematical relationship visually is not the mathematical relationship itself. ...in a manner of speaking one looks right through the drawn circle and keeps the pure thought of the circle in mind."

We find the same principle at work again in Mundy's (1986, p. 392) general theory of meaningful representation: "The hallmark of a meaningless proposition is that its truth-value depends on what scale or coordinate system is employed, whereas meaningful propositions have truth-value independent of the choice of representation, within certain limits. The formal analysis of this distinction leads, in all three areas [measurement theory, geometry, and relativity], to a rather involved technical apparatus focusing upon invariance under changes of scale or changes of coordinate system." The same focus on the independence of figure and meaning, or scale and proportion, emerges in a wide variety of other works on the creation of qualitative mathematical meaning (Heidegger, 1967; Luce, 1978; Narens, 1981, 2002; Roberts, 1985, 1994, 1999).

These issues of meaningfulness and measurement are explored at length by Fisher (2003a, 2003b, 2004). The basic point is that the content of tests and surveys ought to be used to illustrate mathematical relationships between abilities and difficulties visually and conceptually without confusing them with the mathematical relationships themselves. We need to look right through the sample of items used to illustrate the construct and keep the pure thought of the construct in mind, in the manner of the numerical and geometrical figures that are understood to paradigmatically define meaningful representation by, again, a wide range of diverse philosophers (Derrida, 1989, p. 66; Descartes, 1961, p. 8; Gadamer, 1989, pp. 412-3; Kant, 1970, p. 7; see Michell, 1990, pp. 6-8 for more).

Meaningful measurement requires what Rasch (1960) called parameter separation, what Ronald Fisher (1922) called statistical sufficiency, and what Luce and Tukey (1964) called conjoint additivity; all of these can be identified in Rasch models, but not in IRT models (Wright 1984, 1999). In purporting to produce meaningful results, IRT models assume, but do not test or establish, the separation of figure and meaning. Wood (1978, p. 31), accordingly, found himself "persuaded by Lumsden (Lumsden 1978) that two- and three-parameter models are not the answer - test scaling models are self-contradictory if they assert both unidimensionality and different slopes for the item characteristic curves." In David Andrich's (1988, p. 67) study of measurement, he notes that "[the 2-parameter IRT model] attempts to capture the differences in discriminations of the ICCs. The model destroys the possibility of explicit invariance of the estimates of the person and item parameters and will, therefore, not be pursued here."

Because of its internal contradictions, IRT discrimination and guessing parameter estimation requires much larger samples than are required for Rasch models (Lord, 1983). Even with very large samples, the estimation process may diverge instead of converge, a failure prevented by the authors of one popular IRT software program by eliminating the additional item parameters and reducing to a Rasch model on alternate iterations (Stocking, 1989). Given all of these issues, Lumsden (1978, p. 22) accordingly contends that "The two- and three-parameter logistic and normal ogive scaling models should be abandoned since, if the unidimensionality requirement is met, the Rasch (1960) one-parameter model will be realized."

It has lately been noted that some IRT advocates mistakenly think that unidimensionality is not tested in Rasch model applications, but is merely assumed. This has never been the case, but may be a perceptual by-product of a failure to recognize and accept the paradigmatic difference between the measurement perspective (data are fit to models specifying the relational structures needed for meaningfulness) and the statistical perspective (models are fit to data as a means of describing the data). IRT's item discrimination parameter captures and describes interactions between items and respondents in an essentially statistical exercise that compromises the requirements of measurement. This is, of course, quite a reasonable way to proceed in contexts where measurement has already been accomplished, and understanding of the relations between measures is at issue. But the need for information on these interactions does not require that they be estimated at the same time that item calibrations and respondent measures are estimated.

On the contrary, the internal contradictions and estimation problems introduced by the additional item parameter(s) are unnecessary complications that are easily overcome by removing them from the measurement environment and placing them in the statistical environment where they belong. Model fit statistics, for instance, usually correlate very highly (0.95 and up) with the discrimination parameter, do not confound the estimation process, have been available in Rasch software since the early 1970s, and are routinely employed in evaluating data quality. The fit statistics isolate specific scale dependencies that render measures and/or calibrations meaningless, and in doing so facilitate the deconstruction of the original research question in a critical search for a more fundamental articulation capable of supporting figure-meaning separation. As Derrida (in Wood and Bernasconi 1988, p. 88-9) put it, "I try to place myself at a certain point at which -- and this would be the very 'content' of what I would like to 'signify' -- the thing signified is no longer easily separable from the signifier." Model fit statistics flag failures of invariance in which the clarity of mathematical representation is compromised by content-dependencies that prevent the separation of the amount measured (the signified) from the number representing it (the signifier).

A popular Rasch measurement software package, Winsteps (Linacre, 2005), now makes it possible to estimate IRT item parameters without confounding the estimation of measures and calibrations (Linacre, 2004). The model fit statistics routinely produced in Rasch applications enable the study of individual residual differences between observed and expected responses, and are widely recognized for their diagnostic utility. The high correlation of the IRT discrimination parameter estimates with the fit statistics simply reproduces information already available and does not offer additional information beyond what is provided in the Rasch residuals. The recent inclusion of the IRT parameter estimates in Winsteps, then, provides an instructive point of contrast between the measurement and IRT perspectives, but does not enhance the existing substantive value of the information previously provided (Wright, 1992).

William P. Fisher, Jr.

Andrich, D. (1988). Rasch models for measurement. Beverly Hills, California: Sage Publications.

Derrida, J. (1982). Margins of philosophy. Chicago, Illinois: University of Chicago Press.

Derrida, J. (1989). Edmund Husserl's Origin of Geometry: An introduction. Lincoln: University of Nebraska Press.

Descartes, R. (1961). Rules for the direction of the mind (L. J. Lafleur, Trans.). The Library of Liberal Arts. Indianapolis: Bobbs-Merrill.

Fisher, R. A. (1922). On the mathematical foundations of theoretical statistics. Philosophical Transactions of the Royal Society of London, A, 222, 309-368.

Fisher, W. P., Jr. (2003, December). Mathematics, measurement, metaphor, metaphysics: Part I. Implications for method in postmodern science. Theory & Psychology, 13(6), 753-90.

Fisher, W. P., Jr. (2003, December). Mathematics, measurement, metaphor, metaphysics: Part II. Accounting for Galileo's "fateful omission." Theory & Psychology, 13(6), 791-828.

Fisher, W. P., Jr. (2004, October). Meaning and method in the social sciences. Human Studies: A Journal for Philosophy and the Social Sciences, 27(4), 429-54.

Gadamer, H.-G. (1980). Dialogue and dialectic: Eight hermeneutical studies on Plato (P. C. Smith, Trans.). New Haven: Yale University Press.

Gadamer, H.-G. (1989). Truth and method (J. Weinsheimer & D. G. Marshall, Trans.) (Rev. ed.). New York: Crossroad (Original work published 1960).

Heidegger, M. (1967). What is a thing? (W. B. Barton, Jr. & V. Deutsch, Trans.). South Bend, Indiana: Regnery/Gateway.

Kant, I. (1786). (1970). Metaphysical foundations of natural science (J. Ellington, Trans.). Indianapolis, Indiana: Bobbs-Merrill.

Lord, F. M. (1983). Small N justifies Rasch model. In D. J. Weiss (Ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 51-61). New York: Academic Press, Inc.

Luce, R. D. (1978). Dimensionally invariant numerical laws correspond to meaningful qualitative relations. Philosophy of Science, 45, 1-16.

Luce, R. D., & Tukey, J. W. (1964). Simultaneous conjoint measurement: A new kind of fundamental measurement. Journal of Mathematical Psychology, 1(1), 1-27.

Lumsden, J. (1978). Tests are perfectly reliable. British Journal of Mathematical and Statistical Psychology, 31, 19-26.

Michell, J. (1990). An introduction to the logic of psychological measurement. Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Mundy, B. (1986). On the general theory of meaningful representation. Synthese, 67, 391-437.

Narens, L. (1981). A general theory of ratio scalability with remarks about the measurement-theoretic concept of meaningfulness. Theory and Decision: An International Journal for Philosophy and Methodology of the Social Sciences, 13, 1-70.

Narens, L. (2002). Theories of meaningfulness (S. W. Link & J. T. Townsend, Eds.). Scientific Psychology Series. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Ricoeur, P. (1977). The rule of metaphor: Multi-disciplinary studies of the creation of meaning in language (R. Czerny, Trans.). Toronto: University of Toronto Press.

Roberts, F. S. (1985). Applications of the theory of meaningfulness to psychology. Journal of Mathematical Psychology, 29, 311-32.

Roberts, F. S. (1994). Limitations on conclusions using scales of measurement. In A. Barnett, S. Pollock & M. Rothkopf (Eds.), Operations research and the public sector (pp. 621-671). Amsterdam, The Netherlands: Elsevier.

Roberts, F. S. (1999). Meaningless statements. In R. Graham, J. Kratochvil, J. Nesetril & F. Roberts (Eds.), Contemporary trends in discrete mathematics, DIMACS Series, Volume 49 (pp. 257-274). Providence, RI: American Mathematical Society.

Stocking, M. L. (1989). Empirical estimation errors in item response theory as a function of test properties. Princeton, New Jersey: Educational Testing Service. ETS Research Reports.

Wood, R. (1978). Fitting the Rasch model: A heady tale. British Journal of Mathematical and Statistical Psychology, 31, 27-32.

Wood, D., & Bernasconi, R. (1988). Derrida and differance. Evanston, Illinois: Northwestern University Press.

Wright, B. D. (1984). Despair and hope for educational measurement. Contemporary Education Review, 3(1), 281-288.

Wright, B. D. (1992, Spring). IRT in the 1990's: Which models work best? Rasch Measurement Transactions, pp. 196-200 www.rasch.org/rmt/rmt61.htm].

Wright, B. D. (1999). Fundamental measurement for psychology. In S. E. Embretson & S. L. Hershberger (Eds.), The new rules of measurement: What every educator and psychologist should know (pp. 65-104 [www.rasch.org/memo64.htm]). Hillsdale, New Jersey: Lawrence Erlbaum Associates.


Meaningfulness, Measurement and Item Response Theory (IRT). Fisher W.P. Jr.… Rasch Measurement Transactions, 2005, 19:2 p. 1018-20

Please help with Standard Dataset 4: Andrich Rating Scale Model



Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr. Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez

To be emailed about new material on www.rasch.org
please enter your email address here:

I want to Subscribe: & click below
I want to Unsubscribe: & click below

Please set your SPAM filter to accept emails from Rasch.org

www.rasch.org welcomes your comments:

Your email address (if you want us to reply):

 

ForumRasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

Coming Rasch-related Events
March 31, 2017, Fri. Conference: 11th UK Rasch Day, Warwick, UK, www.rasch.org.uk
April 2-3, 2017, Sun.-Mon. Conference: Validity Evidence for Measurement in Mathematics Education (V-M2Ed), San Antonio, TX, Information
April 26-30, 2017, Wed.-Sun. NCME, San Antonio, TX, www.ncme.org - April 29: Ben Wright book
April 27 - May 1, 2017, Thur.-Mon. AERA, San Antonio, TX, www.aera.net
May 26 - June 23, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 30 - July 29, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
July 31 - Aug. 3, 2017, Mon.-Thurs. Joint IMEKO TC1-TC7-TC13 Symposium 2017: Measurement Science challenges in Natural and Social Sciences, Rio de Janeiro, Brazil, imeko-tc7-rio.org.br
Aug. 7-9, 2017, Mon-Wed. In-person workshop and research coloquium: Effect size of family and school indexes in writing competence using TERCE data (C. Pardo, A. Atorressi, Winsteps), Bariloche Argentina. Carlos Pardo, Universidad Catòlica de Colombia
Aug. 7-9, 2017, Mon-Wed. PROMS 2017: Pacific Rim Objective Measurement Symposium, Sabah, Borneo, Malaysia, proms.promsociety.org/2017/
Aug. 10, 2017, Thurs. In-person Winsteps Training Workshop (M. Linacre, Winsteps), Sydney, Australia. www.winsteps.com/sydneyws.htm
Aug. 11 - Sept. 8, 2017, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Aug. 18-21, 2017, Fri.-Mon. IACAT 2017: International Association for Computerized Adaptive Testing, Niigata, Japan, iacat.org
Sept. 15-16, 2017, Fri.-Sat. IOMC 2017: International Outcome Measurement Conference, Chicago, jampress.org/iomc2017.htm
Oct. 13 - Nov. 10, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 5 - Feb. 2, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 10-16, 2018, Wed.-Tues. In-person workshop: Advanced Course in Rasch Measurement Theory and the application of RUMM2030, Perth, Australia (D. Andrich), Announcement
Jan. 17-19, 2018, Wed.-Fri. Rasch Conference: Seventh International Conference on Probabilistic Models for Measurement, Matilda Bay Club, Perth, Australia, Website
May 25 - June 22, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 29 - July 27, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
Aug. 10 - Sept. 7, 2018, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Oct. 12 - Nov. 9, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
The HTML to add "Coming Rasch-related Events" to your webpage is:
<script type="text/javascript" src="http://www.rasch.org/events.txt"></script>

 

The URL of this page is www.rasch.org/rmt/rmt192d.htm

Website: www.rasch.org/rmt/contents.htm