Guessing and the Rasch Model

There is a misunderstanding in the psychometric community that the Rasch model "does not account for guessing" while the three-parameter model does. The truth is the opposite.

Every one agrees that random responses provide no information about ability. Correct responses due to random guessing are quite different from correct responses where some choices have been eliminated before guessing. "Informed guessing" is not "random guessing" because these responses contain information about item difficulty and person ability.

The measurement task is to differentiate between random guessing, which contains no information, and informed guessing, which contains some information. This cannot be accomplished by modelling guessing as an item or even as a person parameter. The pseudo-guessing item parameter in the three-parameter model is useless. It mistakes guessing as a function solely of the item, when, in fact, guessing is an interaction between item propensity to provoke guessing and person proclivity to guess. In addition, the parameterization of guessing obliterates differentiation between random and informed guessing.

A unique strength of the Rasch model is its requirement that the outcome of any interaction between person and item be solely determined by just two parameters, the ability of the person and the difficulty of the item. This requirement establishes a strong framework against which to test data for the presence of anomalous behavior that may influence the estimation of item and person parameters. This identification of anomalies is not restricted to guessing, but addresses any potential measurement disturbance.

Here are three examples that show how Rasch measurement detects guessing in person response records. These analyses were done by IPARM (Smith, 1991) and are used in large scale testing programs like the Dental Admission Test to differentiate between those with low ability who use random responses to complete their test and those with high ability who do not capitalize on chance to improve their scores.

Person 1540 illustrates random guessing. On 6 of the 8 items answered correctly she had less than 0.1 chance of success. On the other 2, less than 0.2. This improbable pattern produced an UnWeighted Total Fit (OUTFIT) of 2.03 and a Weighted Total Fit (INFIT) of 1.63. When items were grouped by difficulty, she failed all 15 easiest items (Subgroup 1), 17 of the 21 medium items (Subgroup 2), but only 10 of the 14 hardest items (Subgroup 3). This produced an Item Difficulty Between Fit of 4.19, a strong indication that her pattern of successes is meaningless.

Person 217 produced fit statistics similar to Person 1540. But his lucky guesses are at the end of his test. This implies that he did not start wholesale guessing until he was out of time at about item 31, at which point he filled out the rest of his answer sheet with "1"s whenever a "right" choice was not immediately apparent to him. His response pattern suggests that a useful estimate of his ability might be obtained from his performance on the first 30 items.

Person 2233 brings out the difference between filling out an answer sheet to complete the test and plodding. Persons 217 and 2233 have similar raw scores (14 and 13) and total test ability estimates (-1.11 and -1.23). But when omits are identified and left unscored a difference emerges.

Person 2233 left her last 19 responses blank, but Person 217's reliance on "1" to fill out his answer sheet beyond item 30 results in 7 additional improbable correct responses. This produces an unweighted total fit of 3.62 and a difficulty between fit of 3.57.

The total test abilities for these persons may imply equality. But their response patterns show that Person 217's random guessing produced a substantial over-estimate of his ability, while Person 2233, when scored only on the 31 items she tried, earns an ability of -0.76, a standard error higher than her whole test -1.23.

The fit analyses which brought out the randomness of Person 1540's responses, the potential utility of Person 217's responses to the first 30 items and the clear differentiation of Persons 217 and 2233 depends on NOT spoiling the frame of reference by forcing an item guessing parameter into the model. That is how the Rasch model "accounts for guessing".

Smith, R.M. (1991), IPARM: Item and person analysis with the Rasch Model. Chicago: MESA Press.









Guessing and the Rasch Model, R Smith … Rasch Measurement Transactions, 1993, 6:4 p. 262-3


Please help with Standard Dataset 4: Andrich Rating Scale Model



Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr. Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez

To be emailed about new material on www.rasch.org
please enter your email address here:

I want to Subscribe: & click below
I want to Unsubscribe: & click below

Please set your SPAM filter to accept emails from Rasch.org

www.rasch.org welcomes your comments:

Your email address (if you want us to reply):

 

ForumRasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

Coming Rasch-related Events
April 26-30, 2017, Wed.-Sun. NCME, San Antonio, TX, www.ncme.org - April 29: Ben Wright book
April 27 - May 1, 2017, Thur.-Mon. AERA, San Antonio, TX, www.aera.net
April 29, 2017, Sat., 16:35 to 18:05. NCME Presidents Invitational Symposium: a new book commemorating Ben Wright's life and career, 16:35 to 18:05, San Antonio, TX, www.ncme.org
May 26 - June 23, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 30 - July 29, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
July 31 - Aug. 3, 2017, Mon.-Thurs. Joint IMEKO TC1-TC7-TC13 Symposium 2017: Measurement Science challenges in Natural and Social Sciences, Rio de Janeiro, Brazil, imeko-tc7-rio.org.br
Aug. 7-9, 2017, Mon-Wed. In-person workshop and research coloquium: Effect size of family and school indexes in writing competence using TERCE data (C. Pardo, A. Atorressi, Winsteps), Bariloche Argentina. Carlos Pardo, Universidad Catòlica de Colombia
Aug. 7-9, 2017, Mon-Wed. PROMS 2017: Pacific Rim Objective Measurement Symposium, Sabah, Borneo, Malaysia, proms.promsociety.org/2017/
Aug. 10, 2017, Thurs. In-person Winsteps Training Workshop (M. Linacre, Winsteps), Sydney, Australia. www.winsteps.com/sydneyws.htm
Aug. 11 - Sept. 8, 2017, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Aug. 18-21, 2017, Fri.-Mon. IACAT 2017: International Association for Computerized Adaptive Testing, Niigata, Japan, iacat.org
Sept. 15-16, 2017, Fri.-Sat. IOMC 2017: International Outcome Measurement Conference, Chicago, jampress.org/iomc2017.htm
Oct. 13 - Nov. 10, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 5 - Feb. 2, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 10-16, 2018, Wed.-Tues. In-person workshop: Advanced Course in Rasch Measurement Theory and the application of RUMM2030, Perth, Australia (D. Andrich), Announcement
Jan. 17-19, 2018, Wed.-Fri. Rasch Conference: Seventh International Conference on Probabilistic Models for Measurement, Matilda Bay Club, Perth, Australia, Website
May 25 - June 22, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 29 - July 27, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
Aug. 10 - Sept. 7, 2018, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Oct. 12 - Nov. 9, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
The HTML to add "Coming Rasch-related Events" to your webpage is:
<script type="text/javascript" src="http://www.rasch.org/events.txt"></script>

 

The URL of this page is www.rasch.org/rmt/rmt64l.htm

Website: www.rasch.org/rmt/contents.htm