A measurement ideal is that each respondent uses the rating scale in the same way. But this doesn't always happen. Does allowing for personal idiosyncrasies solve the problem?
I asked 62 hikers to rate the difficulty of 20 mountains on the Appalachian Trail, using a 5-point rating scale, from 1=easy to 5=extremely hard. The respondents rated each of the 20 mountains in comparison to some other "standard" mountain of the respondent's choice. It was clear from the comments of the hikers that they were answering this item idiosyncratically. One hiker said that, compared to the Rocky Mountains, the mountains on the Appalachian Trail were easy, and so he never used the 4 and 5 ratings. Another hiker said that there is no such thing as an easy mountain, and never used the 1 and 2 ratings.
When I modelled all the hikers to use the rating scale in the same way, I found that 13 of the hikers showed noticeable misfit, with infit or outfit greater than 1.4 (see Figure). This motivated me to hope that the misfit was caused by each hiker defining a separate rating scale.
In a new calibration, in which the 13 misfitting hikers were modelled to have their own rating scales, there was some overall improvement in fit and statistical separations improved slightly, but most misfit was still unexplained. It looks like the real solution to personal idiosyncrasies is to rewrite my questionnaire.
Ryan Bowles
Modeling person response styles. Bowles R. … Rasch Measurement Transactions, 2000, 14:1 p.739
| Rasch Publications | ||||
|---|---|---|---|---|
| Rasch Measurement Transactions (free, online) | Rasch Measurement research papers (free, online) | Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch | Applying the Rasch Model 3rd. Ed., Bond & Fox | Best Test Design, Wright & Stone |
| Rating Scale Analysis, Wright & Masters | Introduction to Rasch Measurement, E. Smith & R. Smith | Introduction to Many-Facet Rasch Measurement, Thomas Eckes | Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr. | Statistical Analyses for Language Testers, Rita Green |
| Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar | Journal of Applied Measurement | Rasch models for measurement, David Andrich | Constructing Measures, Mark Wilson | Rasch Analysis in the Human Sciences, Boone, Stave, Yale |
| in Spanish: | Análisis de Rasch para todos, Agustín Tristán | Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez | ||
| Forum | Rasch Measurement Forum to discuss any Rasch-related topic |
Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement
Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.
| Coming Rasch-related Events | |
|---|---|
| Aug. 11 - Sept. 8, 2023, Fri.-Fri. | On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com |
| Aug. 29 - 30, 2023, Tue.-Wed. | Pacific Rim Objective Measurement Society (PROMS), World Sports University, Macau, SAR, China https://thewsu.org/en/proms-2023 |
| Oct. 6 - Nov. 3, 2023, Fri.-Fri. | On-line workshop: Rasch Measurement - Core Topics (E. Smith, Facets), www.statistics.com |
| June 12 - 14, 2024, Wed.-Fri. | 1st Scandinavian Applied Measurement Conference, Kristianstad University, Kristianstad, Sweden http://www.hkr.se/samc2024 |
The URL of this page is www.rasch.org/rmt/rmt141m.htm
Website: www.rasch.org/rmt/contents.htm