Lessons from Statewide Assessment: Equating and Linking

Several measurement problems and practical issues have been solved in the 1991 Oregon Statewide Assessment through the use of Rasch methodology. The Oregon Statewide Assessment is administered to all students in grades 3, 5, 8 and 11 in Oregon's public schools. The emphasis is on the assessment of skill levels and curriculum evaluation, but the tests are not strictly "competency tests" and no passing scores are implemented. Through the help of extensive Rasch- calibrated, computerized item banks (NWEA, 1990) and the expertise of many people, solutions were obtained to several common testing problems.

Efficient Linking. Multiple test forms are needed in large scale assessment so that broad curriculum coverage can be obtained with brief, time-efficient tests. All item response theory models allow for pre-equating multiple test forms. The advantage of the Rasch model is that stable calibrations can be obtained from relatively small field-test samples for the new items added each year. With changes in curriculum emphasis and the need for test security, new items are always needed. Research by Fred Forster and George Ingebo has shown that excellent calibration stability can be obtained with samples of 200+. Our experience shows that linking should be done, however, with approximately 20 previously-calibrated items embedded in each field-test form (in addition to the new items). Field testing is completed several months in advance of the construction of the final test forms. The Oregon Statewide Assessment has used Fred Forster's "fixed parameter" model (which fixes the calibrations of the item-bank items during the iterative scaling of new items) to complete the linking, with excellent results.

Include "core" items. Limited testing time requires matrix sampling of items in order to keep test forms brief. Our experience, and that of others such as Richard Hill, is that a "core test" embedded in each multiple form is a wise idea. The core test allays concerns that statistical "wizardry" must be used to equate results. The core test can also be used to conduct final checks on the stability of calibrations. Occasionally, items are effected by historical events, such as a Dr. Martin Luther King Jr. reading-comprehension item which showed a calibration shift after national attention was given to the anniversary of his death.

Tie the scale to the curriculum. Longitudinal comparisons are enabled by Rasch scaling. A "curriculum-referenced" scale can be used to track the proportion of students at various skill levels at various times. Oregon uses calibrated items to define the types of skills students have mastered (e.g., with 80% probability of passing) at each of three levels (e.g., Basic, Proficient, and Advanced). Educators find such curriculum-referencing meaningful--perhaps more than the scale scores themselves. With Rasch scales, the proportion of students at each level can be compared across time. The proportions of students in the lower, middle and upper ranges, for example, can be useful for judging educational equity.

Cooperate! Linking state and local tests is essential because school districts often invest greatly in their own testing systems, Portland (OR) Public Schools have developed an important series of functional- levels tests. These tests allow emphasis to be placed on local curricula as well as state-mandated goals. With technical help from Ron Houser and Gage Kingsbury, a series of Rasch-based linkings were established between these Portland and State-wide testing systems. A core of 20 calibrated items at the functional center of each grade level were duplicated in State and Local tests to provide a "hard link" as well at the linkage formed from the scaling of the item banks.


"Often devising a measurement instrument is as important in what it teaches us about the variable as are the subsequent acts of measurement."
Andrich, D. Rasch Models for Measurement, 1988, Sage, p. 10



Lessons from Statewide Assessment, G Roid … Rasch Measurement Transactions, 1991, 5:1 p. 133


Please help with Standard Dataset 4: Andrich Rating Scale Model



Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr. Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez

To be emailed about new material on www.rasch.org
please enter your email address here:

I want to Subscribe: & click below
I want to Unsubscribe: & click below

Please set your SPAM filter to accept emails from Rasch.org

www.rasch.org welcomes your comments:

Your email address (if you want us to reply):

 

ForumRasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

Coming Rasch-related Events
July 31 - Aug. 3, 2017, Mon.-Thurs. Joint IMEKO TC1-TC7-TC13 Symposium 2017: Measurement Science challenges in Natural and Social Sciences, Rio de Janeiro, Brazil, imeko-tc7-rio.org.br
Aug. 7-9, 2017, Mon-Wed. In-person workshop and research coloquium: Effect size of family and school indexes in writing competence using TERCE data (C. Pardo, A. Atorressi, Winsteps), Bariloche Argentina. Carlos Pardo, Universidad Catòlica de Colombia
Aug. 7-9, 2017, Mon-Wed. PROMS 2017: Pacific Rim Objective Measurement Symposium, Sabah, Borneo, Malaysia, proms.promsociety.org/2017/
Aug. 10, 2017, Thurs. In-person Winsteps Training Workshop (M. Linacre, Winsteps), Sydney, Australia. www.winsteps.com/sydneyws.htm
Aug. 11 - Sept. 8, 2017, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Aug. 18-21, 2017, Fri.-Mon. IACAT 2017: International Association for Computerized Adaptive Testing, Niigata, Japan, iacat.org
Sept. 15-16, 2017, Fri.-Sat. IOMC 2017: International Outcome Measurement Conference, Chicago, jampress.org/iomc2017.htm
Oct. 13 - Nov. 10, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Oct. 25-27, 2017, Wed.-Fri. In-person workshop: Applying the Rasch Model hands-on introductory workshop, Melbourne, Australia (T. Bond, B&FSteps), Announcement
Jan. 5 - Feb. 2, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 10-16, 2018, Wed.-Tues. In-person workshop: Advanced Course in Rasch Measurement Theory and the application of RUMM2030, Perth, Australia (D. Andrich), Announcement
Jan. 17-19, 2018, Wed.-Fri. Rasch Conference: Seventh International Conference on Probabilistic Models for Measurement, Matilda Bay Club, Perth, Australia, Website
April 13-17, 2018, Fri.-Tues. AERA, New York, NY, www.aera.net
May 25 - June 22, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 29 - July 27, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
Aug. 10 - Sept. 7, 2018, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Oct. 12 - Nov. 9, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
The HTML to add "Coming Rasch-related Events" to your webpage is:
<script type="text/javascript" src="http://www.rasch.org/events.txt"></script>

 

The URL of this page is www.rasch.org/rmt/rmt51h.htm

Website: www.rasch.org/rmt/contents.htm