The Rationale for Precision Measurements

Why measure more precisely? Here are the reasons given by a physicist (G. Feinberg, 1992):

"Why should one wish to make measurements with ever increasing precision? Because the whole history of physics proves that a new discovery is likely to be found lurking in the next decimal place. (Floyd K. Richtmyer, 1931)"

Social Science does not yet capitalize on precise measurement in the building of theory. Indeed "qualitative" methodologists reject quantification.

"Great precision in measurement contributes to knowing both the great unifying principles, such as the law of conservation of energy, and to know the detailed particulars about individual phenomena. Most basically, it helps to shrink the range of our ignorance."

The unifying principles of social science, such as "People are scared of losing", are equivalent to pre-quantitative statements such as "What goes up must come down". Without an encompassing quantitative framework, more precise details increase confusion, not knowledge.

"The criterion of what constitutes a precision measurement changes over time ... Michelson's measurements of the speed of light, which represented that best that could be done in the early part of the twentieth century, are now rivaled in demonstrations done for first year physics students."

The general precision of social science measurement has changed little in 75 years (since the introduction of MCQs), though technological advances suggest that it should have.

The Relevance of Theory to Precision Measurement

"The contrasts between measurements which can be compared to a precise theoretical prediction, and measurements for which no accurate theory is available, is sometimes expressed through the distinction between precision tests of theory and precision measurements from data."

Nearly all social science measurement is in the category of precision measurement. An example of a precision test is the Lexile theoretical determination of reading difficulty and its prediction of empirical results.

"When an accurate theoretical prediction is available for comparisons with some precision measurement it is relatively easy to know when a problem exists, either with the measurement or with the theory."

Evaluation of construct validity, though usually far short of accurate prediction, often detects this type of problem.

"There is in general no simple relation between the precision with which measurements can, have, and should be done, and the accuracy of the theory used to understand those measurements."

In the Social Sciences, the precision of measurements is generally far higher than the accuracy of theory. In Physics, vice-versa.

"I think it can be argued that many of the profoundest experimental discoveries are made when there is no theoretical guidance available about what to expect."

There is hope here for Social Science!

"There are probably many other, yet unrecognized instances in which precisely measurable quantities involving macroscopic [large-scale] systems are related to simple properties of their components, and it seems likely that the search for and measurement of such quantities will be at the frontiers of future research."

Physics looks for simplicity in the midst of every-increasing complexity. Social Science in contrast is preoccupied with rejecting simplicity and looking for complexity, perhaps to disguise its lack of useful theory.

Permanent Properties vs. Contingent Qualities

"Another point to what is worth measuring precisely involves the contrast between the permanent properties of systems [cf. test items], and the measurement of variable qualities [cf. examinees]. Physicists are usually interested in accurate [i.e., very precise] measurement of quantities whose value does not change with time."

In Social Science, a high precision measure of examinee ability is demanded for high-stakes pass-fail decisions, though it is evident that the examinee will not remain at that ability for long, perhaps only instantaneously.

How Reliable are Precision Measurements?

"There has over time been a systematic increase in the precision of the estimates and a systematic tendency to underestimate errors. Of course the situation has been much worse for some results that do not qualify as precision measurements."

Social Science relies heavily on reliability coefficients and standard-error-based effect-sizes,

yet it is doubtful if the underlying measures attain to the status of "precision measurements" as viewed by physicists. Consequently, Social Science findings are more insecure even than "Hubble constant" for the expansion of the Universe. In 1929, Edwin Hubble estimated this to be 500±60, but it is now 75±25. Social Science is just starting on the road to "constants" with the rule-of-thumb "one logit per grade-level".

"Perhaps the main lesson to be learned is that we should be somewhat skeptical of the precise values and their associated errors, but that we can rely on the prospect that future experiments, by physicists who are dedicated to pushing the envelope of what can be measured, will surely produce values of considerably greater precision."

Social Science! Here we are!

Feinberg, G. (1992) On knowing things better and better. In M.E. Zeller (Ed.) A Festschrift in honor of Vernon W. Hughes. Singapore: World Scientific.

Feinberg, G. The Rationale for Precision Measurements. … Rasch Measurement Transactions, 2000, 14:3 p.764-5


Please help with Standard Dataset 4: Andrich Rating Scale Model



Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr. Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez

To be emailed about new material on www.rasch.org
please enter your email address here:

I want to Subscribe: & click below
I want to Unsubscribe: & click below

Please set your SPAM filter to accept emails from Rasch.org

www.rasch.org welcomes your comments:

Your email address (if you want us to reply):

 

ForumRasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

Coming Rasch-related Events
April 26-30, 2017, Wed.-Sun. NCME, San Antonio, TX, www.ncme.org - April 29: Ben Wright book
April 27 - May 1, 2017, Thur.-Mon. AERA, San Antonio, TX, www.aera.net
April 29, 2017, Sat., 16:35 to 18:05. NCME Presidents Invitational Symposium: a new book commemorating Ben Wright's life and career, 16:35 to 18:05, San Antonio, TX, www.ncme.org
May 26 - June 23, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 30 - July 29, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
July 31 - Aug. 3, 2017, Mon.-Thurs. Joint IMEKO TC1-TC7-TC13 Symposium 2017: Measurement Science challenges in Natural and Social Sciences, Rio de Janeiro, Brazil, imeko-tc7-rio.org.br
Aug. 7-9, 2017, Mon-Wed. In-person workshop and research coloquium: Effect size of family and school indexes in writing competence using TERCE data (C. Pardo, A. Atorressi, Winsteps), Bariloche Argentina. Carlos Pardo, Universidad Catòlica de Colombia
Aug. 7-9, 2017, Mon-Wed. PROMS 2017: Pacific Rim Objective Measurement Symposium, Sabah, Borneo, Malaysia, proms.promsociety.org/2017/
Aug. 10, 2017, Thurs. In-person Winsteps Training Workshop (M. Linacre, Winsteps), Sydney, Australia. www.winsteps.com/sydneyws.htm
Aug. 11 - Sept. 8, 2017, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Aug. 18-21, 2017, Fri.-Mon. IACAT 2017: International Association for Computerized Adaptive Testing, Niigata, Japan, iacat.org
Sept. 15-16, 2017, Fri.-Sat. IOMC 2017: International Outcome Measurement Conference, Chicago, jampress.org/iomc2017.htm
Oct. 13 - Nov. 10, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 5 - Feb. 2, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 10-16, 2018, Wed.-Tues. In-person workshop: Advanced Course in Rasch Measurement Theory and the application of RUMM2030, Perth, Australia (D. Andrich), Announcement
Jan. 17-19, 2018, Wed.-Fri. Rasch Conference: Seventh International Conference on Probabilistic Models for Measurement, Matilda Bay Club, Perth, Australia, Website
May 25 - June 22, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 29 - July 27, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
Aug. 10 - Sept. 7, 2018, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Oct. 12 - Nov. 9, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
The HTML to add "Coming Rasch-related Events" to your webpage is:
<script type="text/javascript" src="http://www.rasch.org/events.txt"></script>

 

The URL of this page is www.rasch.org/rmt/rmt143h.htm

Website: www.rasch.org/rmt/contents.htm