The Necessity of Construct Theory

Measurement is the process of converting observations (e.g., counts) into measures (quantities) via a construct theory. The Rasch Model states a requirement for the way observations and construct theory combine in a probability model to make measures. There is no other combination of observation and theory that produces sufficiency invariance and objectivity in the resultant measures.

As scientists, we cherish parsimony (do much with little) and as statisticians, we cherish uniqueness. This is why we speak of data that fit the Rasch Model - not of fitting some model, Rasch or otherwise, to data. The Rasch Model combines the three components in the definition of measurement (observation, theory and measure) into a simple, elegant and, in more important respects, unique representation.

It is unfortunately too easy to apply the Rasch Model without a construct theory. Rasch programs enable practitioners to estimate item difficulties and person measures on a logit or transformed logit scale. Item maps can be built and variables named, all without any a priori of our intention. In my own case, this practice reminds me of how quickly I could name a factor structure coming out of an "exploratory" factor analysis. Never once did I specify, prior to data collection, what I expected to find in the data. To compound the heresy, I wrote a suite of Educational and Psychological Measurement articles on factor replicability and factor invariance.

If we are to have human science(s), we must know enough about the constructs we intend to measure to be able to specify item calibrations prior to data collection. These item calibrations come from a construct theory. Theory and item engineering improve as we bring observed item difficulties and theory-based item calibrations into closer and closer coincidence. The just announced "temporal thermometers" are a recent example of this kind of improvement in the measurement of human temperature.

"Exploratory" Rasch analysis is no substitute, because, without a construct theory and associated specification equation, the data does not "bite". Just like in exploratory factor analysis, exploratory Rasch analysis produces too many successful studies. We need to increase our failure rate and the shortest route is to make our intentions explicit and then see if the data confirm to the theoretical expectations.

What price do we pay for extensive reliance on exploratory Rasch analysis? We make it almost impossible to demonstrate how a common theory and specification equation can unify the diverse instruments purporting to measure a single construct.

Two hundred and fifty instruments (tests) for measuring "reading comprehension" each with proprietary non-exchangeable scales can be unified only if there is
(1) a compelling demonstration that most of these instruments measure a single unidimensional construct called "reading comprehension",
(2) a common supplemental metric is proposed that is based on a construct theory and supported by a specification equation capable of explaining high proportions of item difficulty variance within and across the candidate tests,
(3) there is some compelling advantage beyond parsimony to encourage the "community" to adopt the standard (the possibility of linking test scores to books is such an application for reading comprehension), and
(4) a business model is proposed that makes it advantageous for the power elite (e.g., publishers, academicians, key users, policy makers) to support unification.

Adoption of the metric system in the US in the 1970s failed criteria 3 and 4 and, ergo, no metric system in our everyday lives despite a huge expenditure.

There is a simple thought experiment that can inform us regarding how well we understand the construct under study. If presented with an instrument purportedly measuring the construct, can we use our knowledge about the construct-associated (construct theory) specification/calibration equation(s) together with item engineering rules to produce a clone or copy of the instrument - such that the score-to-measure table for the clone is identical to that of the original instrument?

Moving now from theory to experiment, if the clone produces measures that are statistically equivalent to those produced by the original, then we have demonstrated that we understand what we are measuring. We can reproduce the phenomena experimentally and we are in control of the major contaminants.

The first time we attempted the above protocol with "reading comprehension", we failed. After some four years and 30 attempts, we are getting pretty good at building reading comprehension tests based on theory alone that produce raw score-to-measure correspondences that conform to design requirements. With this kind of control, it is practically feasible to contemplate unifying the measurement of reading.

Finally, William P. Fisher, Jr. states: "I think that the need for these metrological networks will eventually be seen simply because of the way independent experiments addressing the same variable are converging on common constructs!" I want this to happen because human science is impossible without it. But, the unification process that precedes consensus on a unit of measurement for a construct will be built on improved construct theories and resultant improved instrumentation, not more exploratory Rasch analysis.

Exploratory studies don't converge on anything. As always, the great unifier is substantive theory. See www.lexile.com for a report card on the unification of the measurement of reading comprehension in English and Spanish.

Jack Stenner

Trevor Bond & Christine Fox (2001, 2001, pp.xxi-xxii) comment:
"The Rasch approach to investigating measures illustrates an appreciation of the dialectical nature of the measurement process: Theory informs practice, and practice informs theory. ... From our viewpoint, every item you write puts either the theory, your understanding of it, or your ability to translate it into an empirical indicator on the line. Tossing out items might be fine if you have trawled up a huge bunch of possible items and are looking for the probables to keep in your test. However, if you have developed items that encapsulate for you the very essence of the theory or construct to which you are attached, then all the items are probables. In fact, you might consider all of them as somehow indispensable! Thus, whereas your theory tells you how to write the item, the item's performance should tell you about your theory, your understanding of it, or your item-writing skills."

The Necessity of Construct Theory. Stenner J. … Rasch Measurement Transactions, 2001, 15:1 p.804-5




Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr. Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez

To be emailed about new material on www.rasch.org
please enter your email address here:

I want to Subscribe: & click below
I want to Unsubscribe: & click below

Please set your SPAM filter to accept emails from Rasch.org

www.rasch.org welcomes your comments:

Your email address (if you want us to reply):

 

ForumRasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

Coming Rasch-related Events
May 17 - June 21, 2024, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 12 - 14, 2024, Wed.-Fri. 1st Scandinavian Applied Measurement Conference, Kristianstad University, Kristianstad, Sweden http://www.hkr.se/samc2024
June 21 - July 19, 2024, Fri.-Fri. On-line workshop: Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
Aug. 5 - Aug. 6, 2024, Fri.-Fri. 2024 Inaugural Conference of the Society for the Study of Measurement (Berkeley, CA), Call for Proposals
Aug. 9 - Sept. 6, 2024, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Oct. 4 - Nov. 8, 2024, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 17 - Feb. 21, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
May 16 - June 20, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 20 - July 18, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Further Topics (E. Smith, Facets), www.statistics.com
Oct. 3 - Nov. 7, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com

 

The URL of this page is www.rasch.org/rmt/rmt151q.htm

Website: www.rasch.org/rmt/contents.htm