One of the persistent problems in psychometrics is the determination of the goodnessoffit between observed and expected values. The problem is particularly tricky with discrete multivariate data that form the basis for measurement in the social, behavioral, and health sciences.
Early work in statistics led to Pearson's chisquare statistic (Pearson, 1900). The chisquare statistic has been quite robust and useful in a variety of applications. Several researchers have proposed adaptations and improvements of chisquare statistics that have ranged from adjustments in the degrees of freedom (Fisher, 1924) to the development of the closely related log_{e} likelihood ratio statistic (Wilks, 1935). Unfortunately, the assumptions of the Pearson chisquare statistic are not always met, and therefore the χ^{2} sampling distribution is not necessarily a useful guide for judgments regarding modeldata fit.
The purpose of this note is to describe a family of tuneable goodnessoffit statistics based on the Power Divergence (PD) Statistics (Cressie & Read, 1988). Tuneable goodnessoffit statistics offer a useful approach for examining both person and item fit that has not been explored with Rasch measurement models.
The basic equation for tuneable statistics, τ^{2}, is
where Oi is the observed frequency in a cell i, Ei is the expected frequency for cell i based on the model, and k is the number of cells. Tuneable goodnessoffit statistics can be obtained by inserting the appropriate λ value. The λ values can range from ∞ to +∞.
In order to illustrate the use of tuneable statistics, data from Stouffer and Toby (1951) are presented in Table 1. These data are used to illustrate the obtained estimates of τ^{2} for several λ values. See Engelhard (2008) for additional details regarding the StoufferToby scale, as well as the calculation of conditional probabilities and expected frequencies based on Rasch analyses.
Table 1. StoufferToby (1951) Data 
Note. Rasch item difficulties are 1.89, .20, .10, and 2.20 logits for items A to D respectively.
Conditional probabilities and expected frequencies are based on the Rasch model.
Table 2. Values of the Tuneable Goodnessoffit statistics

Table 2 presents the values for various goodnessoffit statistics with λ values reported at various points between 3.00 and 3.00. Some of these λ values correspond to other goodnessoffit statistics, and these are labeled in the Table 2. The 95th percentile of the chisquared distribution with 17 degrees of freedom is χ^{2} (17, p=.05) = 27.59. Based on this value, we conclude that the goodnessoffit is quite good between the observed and expected frequencies based on the Rasch model. Only one of the estimated values suggests rejecting the null hypothesis (λ value = 2).
This note describes a potentially useful set of tuneable goodnessoffit statistics. It is important to recognize that explorations of goodnessoffit should not involve a simple decision (e.g., reject the null hypothesis), but also require judgments and "cognitive evaluations of propositions" (Rozeboom, 1960, p. 427).
Additional research is needed on the utility of these tuneable statistics for making judgments regarding overall goodnessoffit, item and person fit, and various approaches for defining and conducting residual analyses within the framework of Rasch measurement. This research should include research on the sampling distributions for various tuneable statistics applied to different aspects of goodnessoffit, research on appropriate degrees of freedom, and research on the versions of the τ^{2} statistic that yield the most relevant substantive interpretations within the context of Rasch measurement theory and the construct being measured.
George Engelhard, Jr.
Emory University
Engelhard, G. (2008). Historical perspectives on invariant measurement: Guttman, Rasch, and Mokken. Measurement: Interdisciplinary Research and Perspectives (6), 135.
Fisher, R.A. (1924). The conditions under which χ^{2} measures the discrepancy between observations and hypothesis. Journal of the Royal Statistical Society, 87, 442450.
Freeman, D.H., & Tukey, J.W. (1950). Transformations related to the angular and the square root. Annals of Mathematical Statistics, 21, 607611.
Kullback, S. (1959). Information theory and statistics. New York: John Wiley.
Neyman, J. (1949). Contribution to the theory of the χ^{2} test. Proceedings of the First Berkeley Symposium on Mathematical statistics and Probability, 239273.
Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Philosophy Magazine, 50, 157172.
Read, T.R.C., & Cressie, N.A.C. (1988). Goodnessoffit for discrete multivariate data. New York: SpringerVerlag.
Rozeboom, W.W. (1960). The fallacy of the nullhypothesis significance test. Psychological Bulletin, 57, 416428.
Stouffer, S.A. & Toby, J. (1951). Role conflict and personality. The American Journal of Sociology, 56, 395406.
Wilks, S.S. (1935). The likelihood test of independence in contingency tables. Annals of Mathematical Statistics, 6, 190196.
Tuneable GoodnessofFit Statistics. Engelhard, G. Jr. … Rasch Measurement Transactions, 2008, 22:1 p. 11589
Rasch Publications  

Rasch Measurement Transactions (free, online)  Rasch Measurement research papers (free, online)  Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch  Applying the Rasch Model 3rd. Ed., Bond & Fox  Best Test Design, Wright & Stone 
Rating Scale Analysis, Wright & Masters  Introduction to Rasch Measurement, E. Smith & R. Smith  Introduction to ManyFacet Rasch Measurement, Thomas Eckes  Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr.  Statistical Analyses for Language Testers, Rita Green 
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar  Journal of Applied Measurement  Rasch models for measurement, David Andrich  Constructing Measures, Mark Wilson  Rasch Analysis in the Human Sciences, Boone, Stave, Yale 
in Spanish:  Análisis de Rasch para todos, Agustín Tristán  Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez 
Forum  Rasch Measurement Forum to discuss any Raschrelated topic 
Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement
Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.
Coming Raschrelated Events  

June 23  July 21, 2023, Fri.Fri.  Online workshop: Practical Rasch Measurement  Further Topics (E. Smith, Winsteps), www.statistics.com 
Aug. 11  Sept. 8, 2023, Fri.Fri.  Online workshop: ManyFacet Rasch Measurement (E. Smith, Facets), www.statistics.com 
The URL of this page is www.rasch.org/rmt/rmt221l.htm
Website: www.rasch.org/rmt/contents.htm