Unobserved Categories: Estimating and Anchoring Rasch Measures

Estimating measures from data containing rating scale, partial credit, polytomous and other ordinal structures, with categories that are not observed, is awkward.

Structural and incidental zeroes
Unobserved categories can be of two types: structural zeroes and incidental or sampling zeroes. Structural zero means that the category is unobserved because it is defined not to exist. For instance, a rating scale may be defined to consist of the 4 categories 10, 20, 30, 40. Categories 11, 12, 13, .... cannot be observed. They are merely an artifact of the category numbering system. For analysis purposes, 10, 20, 30, 40 represent categories 1, 2, 3, 4. Consequently either by direct recoding, or automatically within the software, this numerical transformation must be performed. If it is necessary to report adjusted raw scores in the original numbering system, a reverse transformation is required.

Incidental zeroes correspond to categories that are defined to exist and are expected to be observed with some samples, but are not observed with this particular sample. They correspond to performance levels that are within the range of this sample, but do not happen to have been manifested. This is likely to occur with long rating scales (such as percentages) used with small samples, but may happen, by chance, with any ordinal scale with any sample. In this case, unobserved categories correspond to actual performance levels. They must be maintained in order for the ordinal structure to keep its integrity, despite their lack of observations.

Structural and incidental zeroes can occur simultaneously. If a 4 category rating scale has categories numbered 10, 20, 30, 40, perhaps only categories 10, 30, 40 are observed this time. Then to eliminate structural zeroes, the scale is renumbered, for estimation purposes, 1, 2, 3, 4. In the original data, 20 was not observed, so, for the purposes of estimation, the rating scale becomes, 1, 3, 4 with 2 an incidental zero.

Remedying structural zeroes
Rasch polytomous analysis (rating scale, partial credit, Poisson, etc.) proceeds on the basis that each advance of one qualitative level up the polytomy is represented by a one-point ordinal advance. This requires that structural zeroes be eliminated, and the qualitative levels renumbered cardinally. Renumbering may be performed automatically by software or may require explicit data recoding, particularly if the software demands that the cardinal numbers start at 0.

Remedying incidental zeroes: dummy data: approximate, but effective
Incidental zeroes correspond to categories that could be observed, but aren't. A consequence will be that comparison with other analyses, in which all or different categories are observed, will be difficult. The simplest remedy is to include some dummy data records which include those unobserved categories. For instance, suppose that one category of a 7 category rating scale is not observed in this data set. Construct and include a reasonable data record which contains the missing category. If it is the lowest category that is missing, the data record would have next-but-lowest categories for all items except the hardest, which would have the lowest, otherwise unobserved, category. For an intermediate category, a mid-difficulty item would have the unobserved category. Easier items the next higher category. Harder items the next lower category. In general, a few dummy data records would have minimal direct impact on fit statistics or summary statistics. If necessary, the dummy data records can be used to produce rating scale anchor values, then the rating scales can be anchored and the dummy records dropped for final reporting.

The dummy-data method:
1. Construct some reasonable dummy data that includes the unobserved categories and other non-extreme categories
2. Analyze the dataset+dummy data
3. Output the Andrich thresholds (Winsteps SFILE=sf.txt)
4. Reanalyze the dataset without the dummy data. Anchor Andrich thresholds at their values from 3 (SAFILE=sf.txt)

Incidental extreme zeroes: no exact remedy
Consider unobserved extreme (high or low) categories. These correspond to performance levels outside that of the current sample. This sample provides no information to estimate their probability of occurrence. Accordingly unobserved extreme high and low categories are ignored for exact estimation based only on this sample. Thus a rating scale may be defined with categories 1, 2, 3, 4, 5, 6 but if categories 1, 2, and 6 are not observed in this dataset, then analysis proceeds as though the rating scale is defined to be 3, 4, 5. The advancing integers correspond to advancing performance levels. The choice of initial integer, 3 in this case, makes no difference to the estimation, but may be constrained by software considerations to 0. In which case, 3, 4, 5 must be renumbered 0, 1, 2.

Incidental intermediate zeroes: a zero-probability remedy: Wilson's method
Rasch Estimation with Unobserved or Null Intermediate Categories, M Wilson ... Rasch Measurement Transactions, 1991, 5:1 p. 128

Incidental zeroes: an almost exact remedy, useful for anchoring
Dropping extreme unobserved categories and flagging intermediate unobserved categories is awkward, perhaps impossible if not supported by software. It is also not transportable, in terms of anchor values, to other analyses in which those categories are observed. Accordingly, the rating scale structure from the "exact remedy" can be modified to include the unobserved categories directly. Unobserved categories have not been observed and so the inference is that they must have a very low probability of being observed.

For intermediate categories, this corresponds to a very high value of F_gj for the unobserved category, and a very low value of F_g(j+1) for the next category. In practice, "very high" means "add 40 logits", very low means "subtract 40 logits". Applied to the 4, 2, 0, 4 example above, the parameter estimates become: F₁ = log_e(2), F₂ = 40, F₃ = -40-log_e(2), with no category flagged or dropped.

Multiple incidentally unobserved categories can be anchored using the same approach of adding 40 at the low end and subtracting 40 at the high end. If the frequencies of categories 0-6 are 12, 2, 0, 0, 0, 16, 5, then F₁ = 1.6, F₂ = 40, F₃ = 0, F₄ = 0, F₅ = -42.6, F₆ = 1.0. The basis for inference becomes weaker the more unobserved categories there are.

For an extreme unobserved bottom category, consider category frequencies 0, 1, 2, 1. Then F₁ = -40, F₂ = 20-log_e(2), F₃ = 20+log_e(2). The first observed category has a very low parameter estimate, but the relationship between other estimates is unchanged, and their overall sum remains zero.

For an extreme unobserved top category, consider category frequencies 1, 2, 1, 0. Then F₁ = -20-log_e(2), F₂ = -20+log_e(2), F₃ =40. The unobserved top category has a very high parameter estimate, but the relationship between other estimates is unchanged, and their overall sum remains zero.

Incidental zeroes: a curve-fitting approach
The Guttman-component technique (D. Andrich & G. Luo, 2003, Conditional pairwise..., Journal of Applied Measurement 4:3, 205-221) is one technique that can bridge over unobserved categories by modeling all categories to be part of a smooth process. This is particularly powerful for long rating scales, such as percentages, with many incidental-zero categories .
John Michael Linacre

Unobserved categories: Estimating and anchoring Rasch Measures. Linacre J.M. … 17:2 p. 924-925

Rasch Publications
Rasch Measurement Transactions (free, online)	Rasch Measurement research papers (free, online)	Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch	Applying the Rasch Model 3rd. Ed., Bond & Fox	Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters	Introduction to Rasch Measurement, E. Smith & R. Smith	Introduction to Many-Facet Rasch Measurement, Thomas Eckes	Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr.	Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar	Journal of Applied Measurement	Rasch models for measurement, David Andrich	Constructing Measures, Mark Wilson	Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish:	Análisis de Rasch para todos, Agustín Tristán		Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez

Forum

Rasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

The URL of this page is www.rasch.org/rmt/rmt172e.htm

Website: www.rasch.org/rmt/contents.htm