Why measure more precisely? Here are the reasons given by a physicist (G. Feinberg, 1992):
"Why should one wish to make measurements with ever increasing precision? Because the whole history of physics proves that a new discovery is likely to be found lurking in the next decimal place. (Floyd K. Richtmyer, 1931)"
Social Science does not yet capitalize on precise measurement in the building of theory. Indeed "qualitative" methodologists reject quantification.
"Great precision in measurement contributes to knowing both the great unifying principles, such as the law of conservation of energy, and to know the detailed particulars about individual phenomena. Most basically, it helps to shrink the range of our ignorance."
The unifying principles of social science, such as "People are scared of losing", are equivalent to pre-quantitative statements such as "What goes up must come down". Without an encompassing quantitative framework, more precise details increase confusion, not knowledge.
"The criterion of what constitutes a precision measurement changes over time ... Michelson's measurements of the speed of light, which represented that best that could be done in the early part of the twentieth century, are now rivaled in demonstrations done for first year physics students."
The general precision of social science measurement has changed little in 75 years (since the introduction of MCQs), though technological advances suggest that it should have.
"The contrasts between measurements which can be compared to a precise theoretical prediction, and measurements for which no accurate theory is available, is sometimes expressed through the distinction between precision tests of theory and precision measurements from data."
Nearly all social science measurement is in the category of precision measurement. An example of a precision test is the Lexile theoretical determination of reading difficulty and its prediction of empirical results.
"When an accurate theoretical prediction is available for comparisons with some precision measurement it is relatively easy to know when a problem exists, either with the measurement or with the theory."
Evaluation of construct validity, though usually far short of accurate prediction, often detects this type of problem.
"There is in general no simple relation between the precision with which measurements can, have, and should be done, and the accuracy of the theory used to understand those measurements."
In the Social Sciences, the precision of measurements is generally far higher than the accuracy of theory. In Physics, vice-versa.
"I think it can be argued that many of the profoundest experimental discoveries are made when there is no theoretical guidance available about what to expect."
There is hope here for Social Science!
"There are probably many other, yet unrecognized instances in which precisely measurable quantities involving macroscopic [large-scale] systems are related to simple properties of their components, and it seems likely that the search for and measurement of such quantities will be at the frontiers of future research."
Physics looks for simplicity in the midst of every-increasing complexity. Social Science in contrast is preoccupied with rejecting simplicity and looking for complexity, perhaps to disguise its lack of useful theory.
"Another point to what is worth measuring precisely involves the contrast between the permanent properties of systems [cf. test items], and the measurement of variable qualities [cf. examinees]. Physicists are usually interested in accurate [i.e., very precise] measurement of quantities whose value does not change with time."
In Social Science, a high precision measure of examinee ability is demanded for high-stakes pass-fail decisions, though it is evident that the examinee will not remain at that ability for long, perhaps only instantaneously.
"There has over time been a systematic increase in the precision of the estimates and a systematic tendency to underestimate errors. Of course the situation has been much worse for some results that do not qualify as precision measurements."
Social Science relies heavily on reliability coefficients and standard-error-based effect-sizes,
yet it is doubtful if the underlying measures attain to the status of "precision measurements" as viewed by physicists. Consequently, Social Science findings are more insecure even than "Hubble constant" for the expansion of the Universe. In 1929, Edwin Hubble estimated this to be 500±60, but it is now 75±25. Social Science is just starting on the road to "constants" with the rule-of-thumb "one logit per grade-level".
"Perhaps the main lesson to be learned is that we should be somewhat skeptical of the precise values and their associated errors, but that we can rely on the prospect that future experiments, by physicists who are dedicated to pushing the envelope of what can be measured, will surely produce values of considerably greater precision."
Social Science! Here we are!
Feinberg, G. (1992) On knowing things better and better. In M.E. Zeller (Ed.) A Festschrift in honor of Vernon W. Hughes. Singapore: World Scientific.
Feinberg, G. The Rationale for Precision Measurements. … Rasch Measurement Transactions, 2000, 14:3 p.764-5
Rasch Publications | ||||
---|---|---|---|---|
Rasch Measurement Transactions (free, online) | Rasch Measurement research papers (free, online) | Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch | Applying the Rasch Model 3rd. Ed., Bond & Fox | Best Test Design, Wright & Stone |
Rating Scale Analysis, Wright & Masters | Introduction to Rasch Measurement, E. Smith & R. Smith | Introduction to Many-Facet Rasch Measurement, Thomas Eckes | Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr. | Statistical Analyses for Language Testers, Rita Green |
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar | Journal of Applied Measurement | Rasch models for measurement, David Andrich | Constructing Measures, Mark Wilson | Rasch Analysis in the Human Sciences, Boone, Stave, Yale |
in Spanish: | Análisis de Rasch para todos, Agustín Tristán | Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez |
Forum | Rasch Measurement Forum to discuss any Rasch-related topic |
Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement
Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.
Coming Rasch-related Events | |
---|---|
June 26 - July 24, 2020, Fri.-Fri. | On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com |
June 29 - July 1, 2020, Mon.-Wed. | Measurement at the Crossroads 2020, Milan, Italy , https://convegni.unicatt.it/mac-home |
July - November, 2020 | On-line course: An Introduction to Rasch Measurement Theory and RUMM2030Plus (Andrich & Marais), http://www.education.uwa.edu.au/ppl/courses |
July 1 - July 3, 2020, Wed.-Fri. | International Measurement Confederation (IMEKO) Joint Symposium, Warsaw, Poland, http://www.imeko-warsaw-2020.org/ |
Aug. 7 - Sept. 4, 2020, Fri.-Fri. | On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com |
Oct. 9 - Nov. 6, 2020, Fri.-Fri. | On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com |
June 25 - July 23, 2021, Fri.-Fri. | On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com |
The URL of this page is www.rasch.org/rmt/rmt143h.htm
Website: www.rasch.org/rmt/contents.htm