Rasch Measurement With WINSTEPS

 

Department of Applied Research

Chicago Board of Education

 

OTS Training Center

 

 15th Floor, 125 S. Clark Street, Chicago, IL

 

September 24-25, 2007

 

Objective: Learn Rasch measurement with WINSTEPS software.  This training emphasizes measurement concepts and WINSTEPS rating scale analysis in education.  A terminal objective is mastery of conceptual foundations and empirical interpretation of software results. 

 

Target audience:  Statistical workers analyzing questionnaire responses.  Training accommodates three competency goals: a) measurement foundations without software, b) foundations with active software participation, and c) independent Rasch measurement applications. 

  

Materials and equipment:  Pre-training reading materials include Applying Rasch Model, Bond and Fox, 2nd edition, 2007; Rating Scale Analysis, Wright and Masters, 1979; and Rasch Measurement in Health Sciences, Bezruczko, 2005.  A training binder and table-top computer will support participants during guided software applications.  Participants with computer programming expertise and elementary statistical background may bring an item by person data file in .txt format to analyze during training.

 

Enrollment:  For information contact Nikolaus Bezruczko at Nbezruczko ~ at ~ msn.com

    

         Day 1                Introduction:  Concepts and procedures

 

         8:45 - 9:00        Coffee and introductions

 

-         Review workshop themes

-         Discuss raw data assumptions in contemporary survey research.

-         Introduce concept of linear unit.  Why are linear units important?

-         Demonstrate practical difference between linear and nonlinear units

-         Discuss relations to mathematical axioms.

 

         9:00 - 10:00      Rasch Measurement Overview (concepts, purpose, and goals)

Prerequisites:  Read ARM Ch. 1-5, BTD Ch. 1.

        

-         What is it?  Why do we need it?  When do we use it?   

-         What does it look like?  Key properties?  Assumptions and requirements?  Costs and benefits?

-         How does it differ from conventional psychometrics?  What is a logarithm?  Why do we use it? 

 

         10:00 -11:00     Set-up WINSTEPS: The Science Questionnaire

Prerequisites:  Read RSA Ch. 1-2, ARM Ch. 5, RMHS Ch. 2.

 

-         Present the measurement problem

-         Review alternative approaches

-         Organize a measurement solution

-         Review WINSTEPS control file

 

         11:00 -12:00     Interpretation of results:  The Science Questionnaire

Prerequisites:  Read RSA p. 48-49.

 

-     Item/person parameter estimation:  Properties, procedures, and results

-     Construct definition:  Interpret item hierarchy

-     Measurement properties:  Targeting, density, separate./reliability, SEs

-     Parameter invariance

-     Fit: Purpose, computation, and evaluation

-     Measurement quality: Item and person fit, residual analyses

-     Differential item function (DIF):  Compare gender DIF

 

Noon               Lunch

 

            1:00 - 2:00     Analyze My School My Voice CPS parent questionnaire

 

-         Review questionnaire form

-         Review control files (basic structure and adaptations)

-         Analyze fit statistics, residuals, person and item plots

 

            2:00 – 3:00    Analyze Preschool Mini-Assessment interview protocol

 

-         Review instrument

-         Review construct

-         Review control file

-         Summarize results

 

            Day 2             Participant-initiated Applications

 

            9:00 – 12:00  Participant data

 

-         Break into smaller groups

-         Prepare control file

-         Analyze data

-         Interpret results

 

Noon               Lunch

 

            1:00 – 4:00    Present results to workshop audience with following outline.

 

1.  Participant presentation outline

            -  Present survey instrument

            -  Critique items

            -  Discuss target audience

            -  Summarize purpose

 

2.  Interpretation of results

            -  Discuss construct

            -  Review item ordering

            -  Review summary table

         -  Discuss measurement properties

         -  Summarize item and person fit results

         -  Examine rating scale functioning

         -  Present residuals analysis results

                                   

3.  Practical implications:  Formulate recommendation

            -  What does it mean?

            -  What are limitations?

      -  Recommendations for improvement?

 

Workshop Themes

 

This training introduces and develops several conceptual themes, which underlie implementation of Rasch models and interpretation of results.  Some prominent themes follow below. 

 

 

  1. Complexity reduction.  Rasch measurement is a practical effort to manage complexity without sacrificing integrity.

-         multi-dimensional reality

-         unidimensional construction

-         mathematical mediation between reality and perfection

 

  1. Competency strands.  Rasch model facility is mediated by several “skill complexes”.

-         procedures and steps

-         conceptual continuity

-         reoccurring issues

 

  1. Measurement levels.  A key distinction between Rasch measurement and virtually all other social research methods is axiom level.   

-         axiomatic foundations

-         assumptions and implications

-         goals and aspirations

 

  1. Quality control.  Rasch models supervise data quality with several statistics/analyzes.

                                    -  When are responses reasonable?

                                    -  When is measurement unidimensional?

                                    -  Models that supervise analysis.

 

  1. Saving the person.  Persons and items are separated during parameter estimation.

                                    -  Item and person parameters do not depend on samples.

                                    -  Aggregation obscures persons.

                                    -  Rasch measurement preserves individual differences.

 

  1. Structural inference.  Rasch models do not estimate population parameters.

                              -  Items and persons define an absolute scale.

                              -  Invariant parameters are fundamental to valid inference.

                              -  Distribution-free parameters establish objective measures.

 

  1. Conceptual entities.  Measurement is an assertion that observations represent a hypothetical construct.

-         abstraction of a continuum

-         imaginary construction

-         empirical implications

 
 
 


Preparation for Training

 

This training is an intensive commitment to establishing practical Rasch measurement skills in a short time.  Participants are encouraged to enhance their capacity to benefit from training by reading following material prior to training.  

 

1.  (ARM) Applying the Rasch Model, 2nd edition, Bond and Fox, 2007.  This revised edition of a best-seller now includes a WINSTEPS training CD.  All participants are strongly recommended to review chapters 1 through 6.  Participants comfortable with software and statistics should install the CD and become familiar with WINSTEPS. 

 

2.  (BTD) Best Test Design, Wright and Stone, 1979.  Review chapter 1.  A classic handbook to Rasch ideas and methods, it remains authoritative and relevant.  The opening chapter is a wonderful introduction to Rasch measurement.  The mathematical treatment can be postponed until training.

 

3.  (RSA) Rating Scale Analysis, Wright and Masters, 1982.  “The Science Questionnaire”, which will be presented in this training, is published here.  Review chapters 1 and 2.  Chapter 3 is relevant but optional. 

 

4.  (RMHS) Rasch Measurement in Health Sciences, Bezruczko, 2005.  Review Chapter 2.  This chapter presents same material as BTD but with less mathematics. 

 

After Training

 

After training, consolidate and expand your new skills, first, by active applications.   Participants should: 

·   form user group at your worksite to compare analyses.

·   seek opportunities to present your advances at conferences and meetings.

·   access Rasch Measurement Transactions at http://www.rasch.org/rmt/contents.htm.

 

Then steep yourself in published Rasch literature such as following:

 

1.  Applications of Rasch Measurement, Smith, 1992 JAM Press.  A compendium of published articles.

2.  “Fundamental measurement for psychologists”, Wright, 1999.  In Embretson and Hershberger (Eds.).  The new rules of measurement (pp. 65-104), Erlbaum. 

3.  “A new construct for Functional Caregiving”, Journal of Applied Measurement, Bezruczko and Chen, 2005.  Summarizes development of a caregiving construct in health sciences.

4.  Advanced applications:  Rasch measurement, Smith and Smith, 2006, JAM Press.  Describes multi-level applications.

5.   “Nonequivalent Survey Consolidation: An Example from Functional Caregiving”, Journal of Applied Measurement, (in press). 

6.  “Which is more valid:  ECERS or ELLCO?”manuscript submitted to Early Childhood Research Quarterly.

7.  “A construct map for developmental change”, submitted manuscript, Journal of Educational Psychology.

8.  “A Rasch Analysis for Classification of Systemic Lupus Erythematosus and Mixed Connective Tissue Disease”.  Journal of Applied Measurement, Perkins et al., in press.  Describes Rasch model application to medical diagnosis. 

 

Models Govern Our Expectations For Reality.