There is a misunderstanding in the psychometric community that the Rasch model "does not account for guessing" while the three-parameter model does. The truth is the opposite.
Every one agrees that random responses provide no information about ability. Correct responses due to random guessing are quite different from correct responses where some choices have been eliminated before guessing. "Informed guessing" is not "random guessing" because these responses contain information about item difficulty and person ability.
The measurement task is to differentiate between random guessing, which contains no information, and informed guessing, which contains some information. This cannot be accomplished by modelling guessing as an item or even as a person parameter. The pseudo-guessing item parameter in the three-parameter model is useless. It mistakes guessing as a function solely of the item, when, in fact, guessing is an interaction between item propensity to provoke guessing and person proclivity to guess. In addition, the parameterization of guessing obliterates differentiation between random and informed guessing.
A unique strength of the Rasch model is its requirement that the outcome of any interaction between person and item be solely determined by just two parameters, the ability of the person and the difficulty of the item. This requirement establishes a strong framework against which to test data for the presence of anomalous behavior that may influence the estimation of item and person parameters. This identification of anomalies is not restricted to guessing, but addresses any potential measurement disturbance.
Here are three examples that show how Rasch measurement detects guessing in person response records. These analyses were done by IPARM (Smith, 1991) and are used in large scale testing programs like the Dental Admission Test to differentiate between those with low ability who use random responses to complete their test and those with high ability who do not capitalize on chance to improve their scores.
Person 1540 illustrates random guessing. On 6 of the 8 items answered correctly she had less than 0.1 chance of success. On the other 2, less than 0.2. This improbable pattern produced an UnWeighted Total Fit (OUTFIT) of 2.03 and a Weighted Total Fit (INFIT) of 1.63. When items were grouped by difficulty, she failed all 15 easiest items (Subgroup 1), 17 of the 21 medium items (Subgroup 2), but only 10 of the 14 hardest items (Subgroup 3). This produced an Item Difficulty Between Fit of 4.19, a strong indication that her pattern of successes is meaningless.
Person 217 produced fit statistics similar to Person 1540. But his lucky guesses are at the end of his test. This implies that he did not start wholesale guessing until he was out of time at about item 31, at which point he filled out the rest of his answer sheet with "1"s whenever a "right" choice was not immediately apparent to him. His response pattern suggests that a useful estimate of his ability might be obtained from his performance on the first 30 items.
Person 2233 brings out the difference between filling out an answer sheet to complete the test and plodding. Persons 217 and 2233 have similar raw scores (14 and 13) and total test ability estimates (-1.11 and -1.23). But when omits are identified and left unscored a difference emerges.
Person 2233 left her last 19 responses blank, but Person 217's reliance on "1" to fill out his answer sheet beyond item 30 results in 7 additional improbable correct responses. This produces an unweighted total fit of 3.62 and a difficulty between fit of 3.57.
The total test abilities for these persons may imply equality. But their response patterns show that Person 217's random guessing produced a substantial over-estimate of his ability, while Person 2233, when scored only on the 31 items she tried, earns an ability of -0.76, a standard error higher than her whole test -1.23.
The fit analyses which brought out the randomness of Person 1540's responses, the potential utility of Person 217's responses to the first 30 items and the clear differentiation of Persons 217 and 2233 depends on NOT spoiling the frame of reference by forcing an item guessing parameter into the model. That is how the Rasch model "accounts for guessing".
Smith, R.M. (1991), IPARM: Item and person analysis with the Rasch Model. Chicago: MESA Press.
Guessing and the Rasch Model, R Smith Rasch Measurement Transactions, 1993, 6:4 p. 262-3
Forum | Rasch Measurement Forum to discuss any Rasch-related topic |
Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement
Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.
Coming Rasch-related Events | |
---|---|
Oct. 4 - Nov. 8, 2024, Fri.-Fri. | On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com |
Jan. 17 - Feb. 21, 2025, Fri.-Fri. | On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com |
May 16 - June 20, 2025, Fri.-Fri. | On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com |
June 20 - July 18, 2025, Fri.-Fri. | On-line workshop: Rasch Measurement - Further Topics (E. Smith, Facets), www.statistics.com |
Oct. 3 - Nov. 7, 2025, Fri.-Fri. | On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com |
The URL of this page is www.rasch.org/rmt/rmt64l.htm
Website: www.rasch.org/rmt/contents.htm