What can be reported to a CAT examinee? Raw scores are meaningless. Percentiles provide no substantive guidance on strengths and weaknesses. Even measures have little value without a context.
Instructionally useful diagnostic information can be given, however, by using a variation of KIDMAP output (Wright, Mead, and Ludlow, 1980). This is implemented in the IPARM program (Smith, 1991). This program performs item and person analysis at a higher level of detail than calibration programs. The "IPARMM" option creates detailed maps of examinee performance for fixed length or computer adaptive tests.
Abbreviated versions of two person maps are shown. The top of each map summarizes the person's test performance: raw score, number of items attempted, logit ability, and standard error of measurement. Conventionally, the center of the scale, 0.0, represents the average item difficulty for the bank of test items. In practice, logits are usually rescaled into positive integers for published reports.
The center section of each map shows item level performance. The items answered correctly are shown in the top half, those answered incorrectly in the bottom half. Items are identified by five character item names supplied by the user. Here, item names are textbook chapter numbers followed by objectives within chapters.
The band that separates the correct and incorrect responses indicates the person's ability estimate by "®" with a ±1 standard error band around that estimated ability indicated by "-".
The line of symbols beneath the ability estimate indicates the expected success rate for this examinee. The probabilities of correctly answering an item in the indicated ranges are: 80%-100%: "///", 65%-80%: "<lt;lt;", 35%-65%: "===", 20%-35%: ">gt;gt;", 0%-20%:"\\\". These symbols provide a frame of reference for the person's mastery of the material and for the consistency (fit) of the person's performance.
The histogram below the item section shows the performance of all persons taking the examination. Each "*" represents one person. This provides a normative interpretation of the measure.
In the first example, Person 1 has a low score, falling in the lower quartile of the class performance. The CAT test covered a wide range of item difficulties, from -4.0 to +2.0 logits. There were no surprising responses. The map of wrong responses shows what objectives this person missed and forms a guide for further study.
In the second example, Person 38 has a much higher ability estimate, third in the class. This test also covers a wide range of item difficulties, -3.0 to +3.0 logits. Here there are three unexpected incorrect responses to what should be easy items for this person. Person 38 had more than an 80 percent chance of answering items 02-33, 02-39, and 02-40 correctly. It seems that even this able examinee would benefit from further study of parts of Chapter 2!
Richard M. Smith 1994 RMT 8:1 p. 344-5
Smith RM (1991) IPARM Computer Program. Chicago: MESA Press
Wright BD, Mead RJ, Ludlow LH (1980) KIDMAP: Person-by-Item Interaction Mapping. MESA Memorandum #29. Chicago: MESA Press
Reporting candidate performance on computer-adaptive tests. Smith RM. … Rasch Measurement Transactions, 1994, 8:1 p.344-5
| Rasch Publications | ||
|---|---|---|
| Rasch Measurement Transactions (free, online) | Rasch Measurement research papers (free, online) | Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch |
| Applying the Rasch Model 2nd. Ed., Bond & Fox | Best Test Design, Wright & Stone | Rating Scale Analysis, Wright & Masters |
| Introduction to Rasch Measurement, E. Smith & R. Smith | Introduction to Many-Facet Rasch Measurement, Thomas Eckes | Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr. |
| Statistical Analyses for Language Testers, Rita Green | Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar | Journal of Applied Measurement |
|
|
|
| Forum | Rasch Measurement Forum to discuss any Rasch-related topic |
Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement
Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.
| Coming Rasch-related Events | |
|---|---|
| May 15-17, 2013, Wed.-Fri. | In-person workshop: Introductory Rasch (A. Tennant, RUMM), Leeds, UK, www.leeds.ac.uk/medicine/rehabmed/psychometric |
| May 20-22, 2013, Mon.-Wed. | In-person workshop: Intermediate Rasch (A. Tennant, RUMM), Leeds, UK, www.leeds.ac.uk/medicine/rehabmed/psychometric |
| May 31 - June 28, 2013, Fri.-Fri. | On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com |
| June 7, 2013, Fri. | SPHERE workshop: Response-Shift and subjective measures in health science, Nantes, France, www.sphere-nantes.fr/ |
| June 19-21, 2013, Wed.-Fri. | SIS 2013 Conference on Advances in Latent Variables: Methods, Models and Applications, Brescia, Italy, meetings.sis-statistica.org/index.php/sis2013/ALV |
| July 1 - Nov. 30, 2013, Mon.-Sun. | Online Course: Introduction to Rasch Measurement Theory (D. Andrich, RUMM), uwa.edu.au |
| July 5 - Aug. 2, 2013, Fri.-Fri. | On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com |
| Aug.1-5, 2013, Thur.-Mon. | TERA-PROMS Annual Meeting, Kaohsiung, Taiwan, tera.education.nsysu.edu.tw |
| Aug. 9 - Sept. 6, 2013, Fri.-Fri. | On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com |
| Aug. 22, 2013, Thursday. | Symposium in honor of Svend Kreiner, Copenhagen, Denmark, biostat.ku.dk/kreinersymposium |
| Sept. 4-6, 2013, Wed.-Fri. | IMEKO TC1-TC7-TC13 Symposium: Measurement Across Physical and Behavioural Sciences, Genoa, Italy, www.imeko-genoa-2013.it |
| Sept. 13 - Oct. 11, 2013, Fri.-Fri. | On-line workshop: Rasch Applications in Clinical Assessment, Survey Research, and Educational Measurement (W.P. Fisher), www.statistics.com |
| Sept. 18-20, 2013, Wed.-Fri. | In-person workshop: Introductory Rasch (A. Tennant, RUMM), Leeds, UK, www.leeds.ac.uk/medicine/rehabmed/psychometric |
| Sept. 23-25, 2013, Mon.-Wed. | In-person workshop: Intermediate Rasch (A. Tennant, RUMM), Leeds, UK, www.leeds.ac.uk/medicine/rehabmed/psychometric |
| Sept. 26-27, 2013, Thurs.-Fri. | In-person workshop: Advanced Rasch (A. Tennant, RUMM), Leeds, UK, www.leeds.ac.uk/medicine/rehabmed/psychometric |
| Oct. 18 - Nov. 15, 2013, Fri.-Fri. | On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com |
| Oct. 20 - Oct. 25, 2013, Sun.-Fri. | International Association for Educational Assessment (IAEA) 39th Annual Conference, Tel Aviv, Israel, www.iaea-2013.com |
| Dec. 11-13, 2013, Wed.-Fri. | In-person workshop: Introductory Rasch (A. Tennant, RUMM), Leeds, UK, www.leeds.ac.uk/medicine/rehabmed/psychometric |
| March 12-14, 2014, Wed.-Fri. | In-person workshop: Introductory Rasch (A. Tennant, RUMM), Leeds, UK, www.leeds.ac.uk/medicine/rehabmed/psychometric |
| May 14-16, 2014, Wed.-Fri. | In-person workshop: Introductory Rasch (A. Tennant, RUMM), Leeds, UK, www.leeds.ac.uk/medicine/rehabmed/psychometric |
| May 19-21, 2013, Mon.-Wed. | In-person workshop: Intermediate Rasch (A. Tennant, RUMM), Leeds, UK, www.leeds.ac.uk/medicine/rehabmed/psychometric |
| July 4 - Aug. 1, 2014, Fri.-Fri. | On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com |
| Aug. 8 - Sept. 5, 2014, Fri.-Fri. | On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com |
| Sept. 10-12, 2014, Wed.-Fri. | In-person workshop: Introductory Rasch (A. Tennant, RUMM), Leeds, UK, www.leeds.ac.uk/medicine/rehabmed/psychometric |
| Sept. 12 - Oct. 10, 2014, Fri.-Fri. | On-line workshop: Rasch Applications in Clinical Assessment, Survey Research, and Educational Measurement (W.P. Fisher), www.statistics.com |
| Sept. 15-17, 2014, Mon.-Wed. | In-person workshop: Intermediate Rasch (A. Tennant, RUMM), Leeds, UK, www.leeds.ac.uk/medicine/rehabmed/psychometric |
| Sept. 18-19, 2014, Thurs.-Fri. | In-person workshop: Advanced Rasch (A. Tennant, RUMM), Leeds, UK, www.leeds.ac.uk/medicine/rehabmed/psychometric |
| The javascript to add "Coming Rasch-related Events" to your webpage is: <script type="text/javascript" src="http://www.rasch.org/events.txt"></script> | |
The URL of this page is www.rasch.org/rmt/rmt81m.htm
Website: www.rasch.org/rmt/contents.htm