Lessons from Statewide Assessment: Equating and Linking

Several measurement problems and practical issues have been solved in the 1991 Oregon Statewide Assessment through the use of Rasch methodology. The Oregon Statewide Assessment is administered to all students in grades 3, 5, 8 and 11 in Oregon's public schools. The emphasis is on the assessment of skill levels and curriculum evaluation, but the tests are not strictly "competency tests" and no passing scores are implemented. Through the help of extensive Rasch- calibrated, computerized item banks (NWEA, 1990) and the expertise of many people, solutions were obtained to several common testing problems.

Efficient Linking. Multiple test forms are needed in large scale assessment so that broad curriculum coverage can be obtained with brief, time-efficient tests. All item response theory models allow for pre-equating multiple test forms. The advantage of the Rasch model is that stable calibrations can be obtained from relatively small field-test samples for the new items added each year. With changes in curriculum emphasis and the need for test security, new items are always needed. Research by Fred Forster and George Ingebo has shown that excellent calibration stability can be obtained with samples of 200+. Our experience shows that linking should be done, however, with approximately 20 previously-calibrated items embedded in each field-test form (in addition to the new items). Field testing is completed several months in advance of the construction of the final test forms. The Oregon Statewide Assessment has used Fred Forster's "fixed parameter" model (which fixes the calibrations of the item-bank items during the iterative scaling of new items) to complete the linking, with excellent results.

Include "core" items. Limited testing time requires matrix sampling of items in order to keep test forms brief. Our experience, and that of others such as Richard Hill, is that a "core test" embedded in each multiple form is a wise idea. The core test allays concerns that statistical "wizardry" must be used to equate results. The core test can also be used to conduct final checks on the stability of calibrations. Occasionally, items are effected by historical events, such as a Dr. Martin Luther King Jr. reading-comprehension item which showed a calibration shift after national attention was given to the anniversary of his death.

Tie the scale to the curriculum. Longitudinal comparisons are enabled by Rasch scaling. A "curriculum-referenced" scale can be used to track the proportion of students at various skill levels at various times. Oregon uses calibrated items to define the types of skills students have mastered (e.g., with 80% probability of passing) at each of three levels (e.g., Basic, Proficient, and Advanced). Educators find such curriculum-referencing meaningful--perhaps more than the scale scores themselves. With Rasch scales, the proportion of students at each level can be compared across time. The proportions of students in the lower, middle and upper ranges, for example, can be useful for judging educational equity.

Cooperate! Linking state and local tests is essential because school districts often invest greatly in their own testing systems, Portland (OR) Public Schools have developed an important series of functional- levels tests. These tests allow emphasis to be placed on local curricula as well as state-mandated goals. With technical help from Ron Houser and Gage Kingsbury, a series of Rasch-based linkings were established between these Portland and State-wide testing systems. A core of 20 calibrated items at the functional center of each grade level were duplicated in State and Local tests to provide a "hard link" as well at the linkage formed from the scaling of the item banks.

"Often devising a measurement instrument is as important in what it teaches us about the variable as are the subsequent acts of measurement."
Andrich, D. Rasch Models for Measurement, 1988, Sage, p. 10

Lessons from Statewide Assessment, G Roid … Rasch Measurement Transactions, 1991, 5:1 p. 133

Rasch-Related Resources: Rasch Measurement YouTube Channel
Rasch Measurement Transactions & Rasch Measurement research papers - free An Introduction to the Rasch Model with Examples in R (eRm, etc.), Debelak, Strobl, Zeigenfuse Rasch Measurement Theory Analysis in R, Wind, Hua Applying the Rasch Model in Social Sciences Using R, Lamprianou El modelo métrico de Rasch: Fundamentación, implementación e interpretación de la medida en ciencias sociales (Spanish Edition), Manuel González-Montesinos M.
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Rasch Models for Measurement, David Andrich Constructing Measures, Mark Wilson Best Test Design - free, Wright & Stone
Rating Scale Analysis - free, Wright & Masters
Virtual Standard Setting: Setting Cut Scores, Charalambos Kollias Diseño de Mejores Pruebas - free, Spanish Best Test Design A Course in Rasch Measurement Theory, Andrich, Marais Rasch Models in Health, Christensen, Kreiner, Mesba Multivariate and Mixture Distribution Rasch Models, von Davier, Carstensen
Rasch Books and Publications: Winsteps and Facets
Applying the Rasch Model (Winsteps, Facets) 4th Ed., Bond, Yan, Heene Advances in Rasch Analyses in the Human Sciences (Winsteps, Facets) 1st Ed., Boone, Staver Advances in Applications of Rasch Measurement in Science Education, X. Liu & W. J. Boone Rasch Analysis in the Human Sciences (Winsteps) Boone, Staver, Yale Appliquer le modèle de Rasch: Défis et pistes de solution (Winsteps) E. Dionne, S. Béland
Introduction to Many-Facet Rasch Measurement (Facets), Thomas Eckes Rasch Models for Solving Measurement Problems (Facets), George Engelhard, Jr. & Jue Wang Statistical Analyses for Language Testers (Facets), Rita Green Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments (Facets), George Engelhard, Jr. & Stefanie Wind Aplicação do Modelo de Rasch (Português), de Bond, Trevor G., Fox, Christine M
Exploring Rating Scale Functioning for Survey Research (R, Facets), Stefanie Wind Rasch Measurement: Applications, Khine Winsteps Tutorials - free
Facets Tutorials - free
Many-Facet Rasch Measurement (Facets) - free, J.M. Linacre Fairness, Justice and Language Assessment (Winsteps, Facets), McNamara, Knoch, Fan

To be emailed about new material on www.rasch.org
please enter your email address here:

I want to Subscribe: & click below
I want to Unsubscribe: & click below

Please set your SPAM filter to accept emails from Rasch.org

www.rasch.org welcomes your comments:

Your email address (if you want us to reply):


ForumRasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

Coming Rasch-related Events
May 17 - June 21, 2024, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 12 - 14, 2024, Wed.-Fri. 1st Scandinavian Applied Measurement Conference, Kristianstad University, Kristianstad, Sweden http://www.hkr.se/samc2024
June 21 - July 19, 2024, Fri.-Fri. On-line workshop: Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
Aug. 5 - Aug. 6, 2024, Fri.-Fri. 2024 Inaugural Conference of the Society for the Study of Measurement (Berkeley, CA), Call for Proposals
Aug. 9 - Sept. 6, 2024, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Oct. 4 - Nov. 8, 2024, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 17 - Feb. 21, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
May 16 - June 20, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 20 - July 18, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Further Topics (E. Smith, Facets), www.statistics.com
Oct. 3 - Nov. 7, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com


The URL of this page is www.rasch.org/rmt/rmt51h.htm

Website: www.rasch.org/rmt/contents.htm