Estimating measures from data containing rating scale, partial credit, polytomous and other ordinal structures, with categories that are not observed, is awkward.
Structural and incidental zeroes
Unobserved categories can be of two types: structural zeroes and incidental or sampling zeroes. Structural zero means that
the category is unobserved because it is defined not to exist. For instance, a rating scale may be defined to consist of the
4 categories 10, 20, 30, 40. Categories 11, 12, 13, .... cannot be observed. They are merely an artifact of the category
numbering system. For analysis purposes, 10, 20, 30, 40 represent categories 1, 2, 3, 4. Consequently either by direct
recoding, or automatically within the software, this numerical transformation must be performed. If it is necessary to report
adjusted raw scores in the original numbering system, a reverse transformation is required.
Incidental zeroes correspond to categories that are defined to exist and are expected to be observed with some samples, but are not observed with this particular sample. They correspond to performance levels that are within the range of this sample, but do not happen to have been manifested. This is likely to occur with long rating scales (such as percentages) used with small samples, but may happen, by chance, with any ordinal scale with any sample. In this case, unobserved categories correspond to actual performance levels. They must be maintained in order for the ordinal structure to keep its integrity, despite their lack of observations.
Structural and incidental zeroes can occur simultaneously. If a 4 category rating scale has categories numbered 10, 20, 30, 40, perhaps only categories 10, 30, 40 are observed this time. Then to eliminate structural zeroes, the scale is renumbered, for estimation purposes, 1, 2, 3, 4. In the original data, 20 was not observed, so, for the purposes of estimation, the rating scale becomes, 1, 3, 4 with 2 an incidental zero.
Remedying structural zeroes
Rasch polytomous analysis (rating scale, partial credit, Poisson, etc.) proceeds on the basis that each advance of one
qualitative level up the polytomy is represented by a one-point ordinal advance. This requires that structural zeroes be
eliminated, and the qualitative levels renumbered cardinally. Renumbering may be performed automatically by software
or may require explicit data recoding, particularly if the software demands that the cardinal numbers start at 0.
Remedying incidental zeroes: dummy data: approximate, but effective
Incidental zeroes correspond to categories that could be observed, but aren't. A consequence will be that comparison with
other analyses, in which all or different categories are observed, will be difficult. The simplest remedy is to include
some dummy data records which include those unobserved categories. For instance, suppose that one category
of a 7 category rating scale is not observed in this data set. Construct and include a reasonable data record which contains
the missing category. If it is the lowest category that is missing, the data record would have next-but-lowest categories
for all items except the hardest, which would have the lowest, otherwise unobserved, category. For an intermediate
category, a mid-difficulty item would have the unobserved category. Easier items the next higher category. Harder items
the next lower category. In general, a few dummy data records would have minimal direct impact on fit statistics or
summary statistics. If necessary, the dummy data records can be used to produce rating scale anchor values, then the rating
scales can be anchored and the dummy records dropped for final reporting.
Incidental extreme zeroes: no exact remedy
Consider unobserved extreme (high or low) categories. These correspond to performance levels outside that of the current
sample. This sample provides no information to estimate their probability of occurrence. Accordingly unobserved extreme
high and low categories are ignored for exact estimation based only on this sample. Thus a rating scale may be defined
with categories 1, 2, 3, 4, 5, 6 but if categories 1, 2, and 6 are not observed in this dataset, then analysis proceeds as
though the rating scale is defined to be 3, 4, 5. The advancing integers correspond to advancing performance levels. The
choice of initial integer, 3 in this case, makes no difference to the estimation, but may be constrained by software
considerations to 0. In which case, 3, 4, 5 must be renumbered 0, 1, 2.
Incidental intermediate zeroes: a zero-probability remedy: Wilson's method
Rasch Estimation with Unobserved or Null Intermediate Categories, M Wilson ... Rasch Measurement Transactions, 1991, 5:1 p. 128
Incidental zeroes: an almost exact remedy, useful for anchoring
Dropping extreme unobserved categories and flagging intermediate unobserved categories is awkward, perhaps impossible
if not supported by software. It is also not transportable, in terms of anchor values, to other analyses in which those
categories are observed. Accordingly, the rating scale structure from the "exact remedy" can be modified to include the
unobserved categories directly. Unobserved categories have not been observed and so the inference is that they must have
a very low probability of being observed.
For intermediate categories, this corresponds to a very high value of F_{gj} for the unobserved category, and a very low value of F_{g(j+1)} for the next category. In practice, "very high" means "add 40 logits", very low means "subtract 40 logits". Applied to the 4, 2, 0, 4 example above, the parameter estimates become: F_{1} = log_{e}(2), F_{2} = 40, F_{3} = -40-log_{e}(2), with no category flagged or dropped.
Multiple incidentally unobserved categories can be anchored using the same approach of adding 40 at the low end and subtracting 40 at the high end. If the frequencies of categories 0-6 are 12, 2, 0, 0, 0, 16, 5, then F_{1} = 1.6, F_{2} = 40, F_{3} = 0, F_{4} = 0, F_{5} = -42.6, F_{6 }= 1.0. The basis for inference becomes weaker the more unobserved categories there are.
For an extreme unobserved bottom category, consider category frequencies 0, 1, 2, 1. Then F_{1} = -40, F_{2} = 20-log_{e}(2), F_{3} = 20+log_{e}(2). The first observed category has a very low parameter estimate, but the relationship between other estimates is unchanged, and their overall sum remains zero.
For an extreme unobserved top category, consider category frequencies 1, 2, 1, 0. Then F_{1} = -20-log_{e}(2), F_{2} = -20+log_{e}(2), F_{3} =40. The unobserved top category has a very high parameter estimate, but the relationship between other estimates is unchanged, and their overall sum remains zero.
Incidental zeroes: a curve-fitting approach
The Guttman-component technique (D. Andrich & G. Luo, 2003, Conditional pairwise..., Journal of Applied Measurement
4:3, 205-221) is one technique that can bridge over unobserved categories by modeling all categories to be part of a
smooth process. This is particularly powerful for long rating scales, such as percentages, with many incidental-zero
categories .
John Michael Linacre
Unobserved categories: Estimating and anchoring Rasch Measures. Linacre J.M. … 17:2 p. 924-925
Rasch Publications | ||||
---|---|---|---|---|
Rasch Measurement Transactions (free, online) | Rasch Measurement research papers (free, online) | Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch | Applying the Rasch Model 3rd. Ed., Bond & Fox | Best Test Design, Wright & Stone |
Rating Scale Analysis, Wright & Masters | Introduction to Rasch Measurement, E. Smith & R. Smith | Introduction to Many-Facet Rasch Measurement, Thomas Eckes | Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr. | Statistical Analyses for Language Testers, Rita Green |
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar | Journal of Applied Measurement | Rasch models for measurement, David Andrich | Constructing Measures, Mark Wilson | Rasch Analysis in the Human Sciences, Boone, Stave, Yale |
in Spanish: | Análisis de Rasch para todos, Agustín Tristán | Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez |
Forum | Rasch Measurement Forum to discuss any Rasch-related topic |
Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement
Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.
Coming Rasch-related Events | |
---|---|
Apr. 14-17, 2020, Tue.-Fri. | International Objective Measurement Workshop (IOMW), University of California, Berkeley, https://www.iomw.org/ |
May 22 - June 19, 2020, Fri.-Fri. | On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com |
June 26 - July 24, 2020, Fri.-Fri. | On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com |
June 29 - July 1, 2020, Mon.-Wed. | Measurement at the Crossroads 2020, Milan, Italy , https://convegni.unicatt.it/mac-home |
July - November, 2020 | On-line course: An Introduction to Rasch Measurement Theory and RUMM2030Plus (Andrich & Marais), http://www.education.uwa.edu.au/ppl/courses |
July 1 - July 3, 2020, Wed.-Fri. | International Measurement Confederation (IMEKO) Joint Symposium, Warsaw, Poland, http://www.imeko-warsaw-2020.org/ |
Aug. 7 - Sept. 4, 2020, Fri.-Fri. | On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com |
Oct. 9 - Nov. 6, 2020, Fri.-Fri. | On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com |
June 25 - July 23, 2021, Fri.-Fri. | On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com |
The URL of this page is www.rasch.org/rmt/rmt172e.htm
Website: www.rasch.org/rmt/contents.htm