Multidimensional Scaling vs. Rasch PCA Residual Analysis

"Multidimensional-scaling (MDS) models are based on the idea that a set of ordinal data can be converted into a smaller amount of cardinal information" (Nijkamp & Voogd, 1984, p. 201). Rasch measurement is based on the same idea. But do these methods find the same information?

30 schools districts in the Chicago suburbs have been ranked on their performance on each of 13 state-mandated tests. These data have been analyzed with MDS and also with a principal components analysis (PCA) of Rasch residuals (RRP).

Analysis of "Items"
Figure 1 shows an MDS plot, produced using the default options of SYSTAT, with the 13 tests treated as "variables". W is Writing, R is Reading, M is Math, S is Science, C is Social Studies. The number is the grade level, e.g., W3 is Writing - 3rd Grade. On the plot, W3, W6 and W8 are all outliers, but in different directions. There is little natural clustering by either subject area or grade level.

The identical data were subjected to Rasch analysis. Since the schools are rank ordered, all 13 tests (treated as items) have the same raw score and the same measure. The Rasch analysis confirmed the MDS finding that W3, W6 and W8 are the most misfitting tests. Figure 2 plots the first two principal components in the Rasch residuals. Clusters and patterns are now evident both by subject area and grade level. The Rasch Figure is easier to interpret and suggestive. The fact that W3, W6 and W8 form a cluster on this plot suggests that their commonalities are greater than their differences. This example suggests that Rasch analysis is a powerful investigative tool even when measurement construction is not the primary objective.

Analysis of "persons"
The MDS and Rasch analyses were continued focusing on the schools as "cases" or "persons". Figure 3 shows an MDS plot of the 30 schools, identified by District number. It has the appearance of a ring. This plot suggests that there are two processes at work, but gives little guidance to what they are.

The RRP plot for the schools is Figure 4. Since the schools were rank ordered on the 13 Tests, the schools have different sums of ranks, and so different measures. Since measures are available, they are plotted along the x-axis. School 181 is the highest ranked overall, school 33 the lowest. Now the y-axis in the MDS plot, Figure 3, is seen to be a faint shadow of overall school performance. This is depicted by the main diagonal in Figure 5, but sub-diagonals compound the difficulty of interpreting the first MDS dimension.

The y-axis in Figure 4, the RRP plot, is the first principal component in the Rasch residuals. By inspection of the raw data, it is seen that the schools at the positive pole of this component are strong in Writing and 3rd Grade performance relative to schools at the negative pole. On this plot, clusters of similarly performing schools are apparent. This axis corresponds to the x-axis in Figure 3, the MDS plot, as shown in Figure 6.

These plots are remarkable. According to Figure 6, the misfit from the Rasch model has been captured by the MDS analysis. But, according to Figure 5, the level of overall school performance, a typical focus of studies such as this, has been almost lost in the MDS results. MDS has not been as successful as Rasch in extracting cardinal information from ordinal data.
John Michael Linacre

Nijkamp P., Voogd H. (1984) Multidimensional and homogenous scaling in spatial analysis. Chapter 12 in Bahrenburg G., Fischer M.M., Nijkamp P. (Eds.) Recent Developments in Spatial Data Analysis: Methodology, Measurement, Models. Brookfield, Vermont: Gower Publishing Co.


Rasch PCA residual analysis Linacre, J.M. Multidimensional scaling vs. … Rasch Measurement Transactions, 1999, 13:1 p. 684


Please help with Standard Dataset 4: Andrich Rating Scale Model



Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr. Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez

To be emailed about new material on www.rasch.org
please enter your email address here:

I want to Subscribe: & click below
I want to Unsubscribe: & click below

Please set your SPAM filter to accept emails from Rasch.org

www.rasch.org welcomes your comments:

Your email address (if you want us to reply):

 

ForumRasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

Coming Rasch-related Events
June 30 - July 29, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
July 31 - Aug. 3, 2017, Mon.-Thurs. Joint IMEKO TC1-TC7-TC13 Symposium 2017: Measurement Science challenges in Natural and Social Sciences, Rio de Janeiro, Brazil, imeko-tc7-rio.org.br
Aug. 7-9, 2017, Mon-Wed. In-person workshop and research coloquium: Effect size of family and school indexes in writing competence using TERCE data (C. Pardo, A. Atorressi, Winsteps), Bariloche Argentina. Carlos Pardo, Universidad Catòlica de Colombia
Aug. 7-9, 2017, Mon-Wed. PROMS 2017: Pacific Rim Objective Measurement Symposium, Sabah, Borneo, Malaysia, proms.promsociety.org/2017/
Aug. 10, 2017, Thurs. In-person Winsteps Training Workshop (M. Linacre, Winsteps), Sydney, Australia. www.winsteps.com/sydneyws.htm
Aug. 11 - Sept. 8, 2017, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Aug. 18-21, 2017, Fri.-Mon. IACAT 2017: International Association for Computerized Adaptive Testing, Niigata, Japan, iacat.org
Sept. 15-16, 2017, Fri.-Sat. IOMC 2017: International Outcome Measurement Conference, Chicago, jampress.org/iomc2017.htm
Oct. 13 - Nov. 10, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 5 - Feb. 2, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 10-16, 2018, Wed.-Tues. In-person workshop: Advanced Course in Rasch Measurement Theory and the application of RUMM2030, Perth, Australia (D. Andrich), Announcement
Jan. 17-19, 2018, Wed.-Fri. Rasch Conference: Seventh International Conference on Probabilistic Models for Measurement, Matilda Bay Club, Perth, Australia, Website
April 13-17, 2018, Fri.-Tues. AERA, New York, NY, www.aera.net
May 25 - June 22, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 29 - July 27, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
Aug. 10 - Sept. 7, 2018, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Oct. 12 - Nov. 9, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
The HTML to add "Coming Rasch-related Events" to your webpage is:
<script type="text/javascript" src="http://www.rasch.org/events.txt"></script>

 

The URL of this page is www.rasch.org/rmt/rmt131e.htm

Website: www.rasch.org/rmt/contents.htm