Treatment Effects: Fixed-Effects Chi-Square of Homogeneity

Treatment effects, structural item properties, experimental conditions and time points can be modelled as constituents of item difficulty according to Gerhard Fischer and Ivo Ponocny (Annual Meeting of the Psychometric Society, Columbus, Ohio, July 1992).

This paper discusses a self-rating inventory of 24 items (symptoms of psychosomatic disorders) with 4 response categories, administered to 87 patients prior to and after treatment. 37 patients received medication and relaxation therapy, 50 only medication. Each patient is modelled at a constant level of functioning. New constituents are introduced into the item difficulties to mark changes in patient functioning after treatment. The model can be:

loge(Pnimrj/Pnimrj-1) = Bn - Di - Mim - Rir - Fij

Bn is the functional level of person n
Di is the difficulty of item i
Mim is medication improvement
Mi1=0 prior to treatment
Mi2 is post-treatment benefit on item i
Rir is relaxation improvement
Ri1=0 prior to treatment, or for no-relaxation
Ri2 is post-treatment benefit on item i
Fij is the partial credit scale structure for item i.

The authors ask: Is the effect of medication, Mi2, (or relaxation, Ri2) the same for all items? Does it have a statistically significant effect on patient functioning? They base their answers on Conditional Maximum Likelihood estimation CMLE, because "likelihood ratio tests can be established rigorously and can easily be carried out". Likelihood ratio tests are valid when the hypothesis is true, the sample size is large and normality holds. Then differences between log-likelihoods become Chi^2- distributed. To test the hypothesis that Mi2 (or Ri2) are the same for all items, two analyses are performed. First the log-likelihood of the data under the full model is estimated, lambda1. Then Mi2 is reparameterized as M2, modelling a common medication effect across items, and a new log-likelihood estimated, lambda2. The test statistic is

Chi^2 = -2(lambda2 -lambda1)

with 23 degrees of freedom (24 item effects - 1 common effect). The hypothesis that M2=0 (or R2=0) can be tested in a similar way.

However, for any method producing a measure and standard error for Mi2 (or Ri2), the equivalence of treatment effects across items can be investigated with just one analysis, a "fixed effects" chi-square of homogeneity [see Box].

A graphical version of this is to plot the estimates of Mi2 (or Ri2) with 2 standard-error confidence bands. When all confidence bands overlap the common effect, then the estimates are statistically equivalent. An estimate of this common effect is the information-weighted mean of all item-level effect estimates [see Box]. The arithmetic mean is good enough when standard errors of item-level effects are similar.

A simple way to obtain the common effect of Mi2 (or Ri2) is to reanalyze the data parameterizing those common values as M2 (or R2). A t-test of the hypothesis that there is no M2 (or R2) effect is obtained by dividing M2 (or R2) by its standard error.

These approaches to estimating the size and statistical significance of effects yield slightly different numerical results, some more approximate than others, none absolutely precise. This reminds us that it is the meaning of numbers, not their values when compared to arbitrary criteria, that is paramount.

The "Fixed Effects" Chi-Square of Homogeneity
For L measures, Di, with standard errors, SEi,
the information in each measure is:
ωi = 1/SEi²
The mean (common information-weighted) effect is:
Σ(ωiDi)/Σωi with S.E.: √ 1/Σωi
A test of the hypothesis that all L measures are
statistically equivalent to one common
"fixed effect", apart from measurement error, is:
χ² = Σ(ωiDi²) - (ΣωiDi)²/Σωi
a chi-square statistic with L-1 d.f.
p>.05 (or p>.01) indicates
statistically equivalent estimates

Treatment Effects

Treatment Effects: Fixed-Effects Chi-Square of Homogeneity, J Linacre … Rasch Measurement Transactions, 1992, 6:2 p. 218-9

Please help with Standard Dataset 4: Andrich Rating Scale Model

Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr. Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez

To be emailed about new material on
please enter your email address here:

I want to Subscribe: & click below
I want to Unsubscribe: & click below

Please set your SPAM filter to accept emails from welcomes your comments:

Your email address (if you want us to reply):


ForumRasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website,

Coming Rasch-related Events
April 26-30, 2017, Wed.-Sun. NCME, San Antonio, TX, - April 29: Ben Wright book
April 27 - May 1, 2017, Thur.-Mon. AERA, San Antonio, TX,
April 29, 2017, Sat., 16:35 to 18:05. NCME Presidents Invitational Symposium: a new book commemorating Ben Wright's life and career, 16:35 to 18:05, San Antonio, TX,
May 26 - June 23, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps),
June 30 - July 29, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps),
July 31 - Aug. 3, 2017, Mon.-Thurs. Joint IMEKO TC1-TC7-TC13 Symposium 2017: Measurement Science challenges in Natural and Social Sciences, Rio de Janeiro, Brazil,
Aug. 7-9, 2017, Mon-Wed. In-person workshop and research coloquium: Effect size of family and school indexes in writing competence using TERCE data (C. Pardo, A. Atorressi, Winsteps), Bariloche Argentina. Carlos Pardo, Universidad Catòlica de Colombia
Aug. 7-9, 2017, Mon-Wed. PROMS 2017: Pacific Rim Objective Measurement Symposium, Sabah, Borneo, Malaysia,
Aug. 10, 2017, Thurs. In-person Winsteps Training Workshop (M. Linacre, Winsteps), Sydney, Australia.
Aug. 11 - Sept. 8, 2017, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets),
Aug. 18-21, 2017, Fri.-Mon. IACAT 2017: International Association for Computerized Adaptive Testing, Niigata, Japan,
Sept. 15-16, 2017, Fri.-Sat. IOMC 2017: International Outcome Measurement Conference, Chicago,
Oct. 13 - Nov. 10, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps),
Jan. 5 - Feb. 2, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps),
Jan. 10-16, 2018, Wed.-Tues. In-person workshop: Advanced Course in Rasch Measurement Theory and the application of RUMM2030, Perth, Australia (D. Andrich), Announcement
Jan. 17-19, 2018, Wed.-Fri. Rasch Conference: Seventh International Conference on Probabilistic Models for Measurement, Matilda Bay Club, Perth, Australia, Website
May 25 - June 22, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps),
June 29 - July 27, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps),
Aug. 10 - Sept. 7, 2018, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets),
Oct. 12 - Nov. 9, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps),
The HTML to add "Coming Rasch-related Events" to your webpage is:
<script type="text/javascript" src=""></script>


The URL of this page is