Too Many Factors in Factor Analysis?

Guilford (April 1941) reports that "a factor analysis of the ten sub-tests of the Seashore test of pitch discrimination [of 0.5 to 30 cycles per second differences] revealed that more than one ability is involved. One factor, which accounted for the greater share of the variances, had loadings that decreased systematically with increasing difficulty [from 30 to 0.5 cps differences]. A second factor had strongest loadings among the more difficult items with frequency differences of 2 to 5 cps. A third had strongest loadings at differences of 5 to 12 cps. No explanation for the three factors is apparent, but the hypothesis is accepted that they represent distinct abilities. In tests so homogeneous as to content and form, where a single common factor might well have been expected, the appearance of additional common factors emphasizes the importance of considering the difficulty level of test items, both in the attempt to interpret new factors and in the practice of testing. The same kind of item may measure different abilities according as it is easy or difficult for the individuals to whom it is applied" (p.67).

Can there really be three distinct hearing discrimination abilities within such a narrow range of frequency differences? Guilford, an expert on psychophysics, cannot explain how that could be. He allows his data analysis to control his thinking. Yet he gives us a clue: "the importance of considering the difficulty level of test items.. in the attempt to interpret new factors."

Let us follow this clue to Ferguson (October 1941), a paper with no direct reference to Guilford: "In general, the greater the number of degrees of difficulty among the items in a test or among the tests in a battery, the higher the rank of the matrix of inter-correlations; that is differences in difficulty are represented in the factorial configuration as additional factors" (p. 323).

Is this the reality underlying Guilford's three factors? Not three abilities, but three item difficulty levels pertaining to one ability? In attempting to discover the meaning of the three factors of pitch discrimination, appeal to Ferguson's explanation of general statistical effects is more convincing than to Guilford's hypothesis of an inexplicable local psychophysical manifestation.

Ferguson reports the predicament that factor analysis identifies difficulty levels with different factors. Rasch analysis avoids this predicament by constructing one latent variable that spans all difficulty levels. Since the analyst's motivation is to span the data set with one meaning, rather than to stratify it, Rasch analysis is the method of choice.

Trevor G Bond 1994 RMT 8:1 p. 347
School of Education
James Cook University of North Queensland
Australia

Ferguson GA. 1941. The factorial interpretation of test difficulty. Psychometrika 6:5 323-329.

Guilford JP. 1941. The difficulty of a test and its factor composition. Psychometrika 6:2 67-77.

Note: A further confusing aspect of Factor Analysis is the analyst's choice of diagonalization, maximization, rotation and obliqueness. These can make highly-correlated content strands, which from the Rasch perspective are part of the dimension, appear to the casual reader to be orthogonal dimensions. Also small perturbations in the data can be made to appear to be large factors. If the factor loadings are plotted against item difficulty, it can be seen which factors may be due to nodes in the item distributions. To identify which factors are distribution-related, simulate data with the estimated Rasch measures as generators, and then apply EFA in order to obtain a baseline against which to interpret the empirically-based EFA. Principal Components Analysis of Residuals avoids many of the problems associated with Common Factor Analysis.


Too Many Factors?

"Therefore, one might expect the emergence of only one factor when a factor analysis would be performed on all newly defined subsets [of unidimensional items]. However, factor analysis of the newly defined subsets yielded two factors. Further inspection of the factor plot showed that the emergence of a second factor could be considered as an artefact due to the skewness of the subset scores."
Van der Ven, A.H.G.S., & Ellis, J.L. (2000). A Rasch Analysis of Raven's Standard Progressive Matrices. Personality and Individual Differences, 29 (1), 45-64.

reported in RMT 16:1


Duncan in his 1984 book "Notes on Social Measurement" has a good discussion on the limitations of factor analysis. In particular how inter-item correlations and item loadings are affected by item difficulty. Thus if a pool of items containing both easy and difficult items is factor analyzed, it will produce 2 factors, even if all the items are on exactly the same topic.
Scott Bayley

Duncan O.D. (1984). Notes on social measurement: Historical and critical. New York: Russell Sage Foundation.

For more information,
The Impact of Rasch Item Difficulty on Confirmatory Factor Analysis , S.V. Aryadoust … Rasch Measurement Transactions, 2009, 23:2 p. 1207
Confirmatory factor analysis vs. Rasch approaches: Differences and Measurement Implications, M.T. Ewing, T. Salzberger, R.R. Sinkovics … Rasch Measurement Transactions, 2009, 23:1 p. 1194-5
Conventional factor analysis vs. Rasch residual factor analysis, Wright, B.D. … 2000, 14:2 p. 753.
Rasch Analysis First or Factor Analysis First? Linacre J.M. … 1998, 11:4 p. 603.
Factor analysis and Rasch analysis, Schumacker RE, Linacre JM. … 1996, 9:4 p.470
Too many factors in Factor Analysis? Bond TG. … 1994, 8:1 p.347
Comparing factor analysis and Rasch measurement, Wright BD. … 1994, 8:1 p.350
Factor analysis vs. Rasch analysis of items, Wright BD. … 5:1 p.134


Too many factors in Factor Analysis? Bond TG. … Rasch Measurement Transactions, 1994, 8:1 p.347



Rasch-Related Resources: Rasch Measurement YouTube Channel
Rasch Measurement Transactions & Rasch Measurement research papers - free An Introduction to the Rasch Model with Examples in R (eRm, etc.), Debelak, Strobl, Zeigenfuse Rasch Measurement Theory Analysis in R, Wind, Hua Applying the Rasch Model in Social Sciences Using R, Lamprianou El modelo métrico de Rasch: Fundamentación, implementación e interpretación de la medida en ciencias sociales (Spanish Edition), Manuel González-Montesinos M.
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Rasch Models for Measurement, David Andrich Constructing Measures, Mark Wilson Best Test Design - free, Wright & Stone
Rating Scale Analysis - free, Wright & Masters
Virtual Standard Setting: Setting Cut Scores, Charalambos Kollias Diseño de Mejores Pruebas - free, Spanish Best Test Design A Course in Rasch Measurement Theory, Andrich, Marais Rasch Models in Health, Christensen, Kreiner, Mesba Multivariate and Mixture Distribution Rasch Models, von Davier, Carstensen
Rasch Books and Publications: Winsteps and Facets
Applying the Rasch Model (Winsteps, Facets) 4th Ed., Bond, Yan, Heene Advances in Rasch Analyses in the Human Sciences (Winsteps, Facets) 1st Ed., Boone, Staver Advances in Applications of Rasch Measurement in Science Education, X. Liu & W. J. Boone Rasch Analysis in the Human Sciences (Winsteps) Boone, Staver, Yale Appliquer le modèle de Rasch: Défis et pistes de solution (Winsteps) E. Dionne, S. Béland
Introduction to Many-Facet Rasch Measurement (Facets), Thomas Eckes Rasch Models for Solving Measurement Problems (Facets), George Engelhard, Jr. & Jue Wang Statistical Analyses for Language Testers (Facets), Rita Green Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments (Facets), George Engelhard, Jr. & Stefanie Wind Aplicação do Modelo de Rasch (Português), de Bond, Trevor G., Fox, Christine M
Exploring Rating Scale Functioning for Survey Research (R, Facets), Stefanie Wind Rasch Measurement: Applications, Khine Winsteps Tutorials - free
Facets Tutorials - free
Many-Facet Rasch Measurement (Facets) - free, J.M. Linacre Fairness, Justice and Language Assessment (Winsteps, Facets), McNamara, Knoch, Fan

To be emailed about new material on www.rasch.org
please enter your email address here:

I want to Subscribe: & click below
I want to Unsubscribe: & click below

Please set your SPAM filter to accept emails from Rasch.org

www.rasch.org welcomes your comments:

Your email address (if you want us to reply):

 

ForumRasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

Coming Rasch-related Events
Oct. 4 - Nov. 8, 2024, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 17 - Feb. 21, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
May 16 - June 20, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 20 - July 18, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Further Topics (E. Smith, Facets), www.statistics.com
Oct. 3 - Nov. 7, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com

 

The URL of this page is www.rasch.org/rmt/rmt81p.htm

Website: www.rasch.org/rmt/contents.htm