Review of Reviews of Bond & Fox (2001)

Trevor Bond and Christine Fox have accomplished a remarkable feat: writing an academic best-seller about an obscure area of statistical measurement. Sales figures assert that "Applying The Rasch Model: Fundamental Measurement in the Human Sciences" (Mahwah NJ: Lawrence Erlbaum Assoc.) has succeeded in reaching its goal of communicating highly technical material in a non-technical way.

For almost 20 years, Ben Wright toyed with writing a non-technical introductory Rasch text. Ben had a model of what he wanted to do, his "Conversational Statistics in Education & Psychology" (Wright & Mayers, McGraw-Hill, 1984). That book focuses on one dataset and each chapter uses different statistical tools to analyze it. Ben perceived that the "spiral of conversation" develops into the "arrow of knowledge". But Ben's introductory Rasch text too quickly became clogged with mathematical and philosophical minutia, and so no progress was made.

Bond & Fox are relatively new to the Rasch field. They remember what they needed to know, and what they didn't need to know; what they needed to understand, and what they didn't need to understand. They follow the "spiral of conversation", reiterating ideas and examples, but from different perspectives, so enabling the reader to accumulate knowledge and experience. The book is a pleasant read. It is nicely laid out and typeset. It has a comprehensive glossary and a suitably non-technical reference list. It can be used for personal study or as a classroom text.

Of course, the book runs into some trouble with expert reviewers, such as Wim van der Linden (2001), Mark Wilson (2002), Ed Wolfe (2002) and myself. I say "of course" because that is always the fate of introductory texts, such as school books. "Not one of the [middle-school science] books we reviewed reached a level that we could call scientifically accurate as far as the physical science contained therein." (John Hubisz, 2001). Sweeping generalizations, substantive short-cuts, and imprecise use of technical terms are features of introductory texts. One of my own pedanticisms is to distinguish between "precision" and "accuracy". Sure enough, the very first time "precision" appears (on p. xvii), "accuracy" is meant. Our reviewers point out more examples.

What is to be done? Hopefully, as with the first 100 years of the "King James" Bible, the most blatant inaccuracies and omissions will be corrected in the next edition. Instructors would be wise to read ahead and provide students with a sheet of annotations for each chapter, focusing on those features relevant to their students. As van der Linden writes, "a statement that is wrong can never be understood." Though van der Linden and I would undoubtedly disagree as to exactly what is right! However we, along with Wilson, definitely agree that European research is conspicuous for its absence. The next edition must include Fischer & Molenaar's (1995) "Rasch Models" in its list of "Classic Reference Texts". [Apologies to Bond & Fox! It was listed in the 1st Edition, and also in the 2nd Edition (2007).]

Wolfe correctly says that "there is insufficient information … to allow readers to apply the Rasch model to their own data sets." This flaw I find in many statistics texts, cookery books and guidebooks. So long as you follow closely along with the author, all is well, but branch out on your own, and you are quickly lost. Again, it requires an instructor or coach to help the neophyte, particularly in the operation of ever-changing software.

A fundamental question about "… Fundamental Measurement …" is what is measurement? We have three answers. The Bond & Fox Glossary answer is "The location of objects along a single dimension on the basis of observations which add together." An amazingly succinct statement from which the Rasch model can be deduced in two ways. If the "observations" are scored qualitative indicators, then the "add together" specifies that the raw score is a sufficient statistic, and the Rasch model follows. On the other hand, if the "observations" are differences between person abilities and item difficulties, then the "add together" is Campbell concatenation of those differences, and the Rasch model follows. Thus Bond & Fox assert, in my words, "if you want to measure, you've got to use a Rasch model!"

Van der Linden answers: "I have difficulty recommending this book as an introductory text to modern measurement. Readers will be much better off with a balanced, elementary text [such] as Hambleton, Swaminathan and Rogers (1991)." But Hambleton's book is actually about IRT, i.e., data description in a quasi-linear framework, which is only measurement in a Stevens sense, as Michell (1999) "Measurement in psychology …", referenced in Bond & Fox, explains. This suggests that the relationship between Rasch and IRT needs more than one page in Bond & Fox, particularly because more advanced books in the field, such as Embretson & Hershberger's (1999) "New Rules" and van der Linden & Hambleton's (1997), "Handbook of Modern Item Response Theory" unabashedly classify Rasch under IRT. Wilson aptly summarizes the problem: "one person's oversimplification is another person's strong measurement philosophy."

Van der Linden's final recommendation is difficult to comprehend, "better still, the introductory chapter and chapters 5 and 6 in the original text by Rasch (1960)." Certainly read them: Chapter 1 ends up with a discussion of binomial trials using mice in a maze. Chapter 5 shows how to draw empirical ICCs. Chapter 6 straightens them out with a logistic transformation. But Wright & Masters (1982) "Rating Scale Analysis" does much, much more. For "non-dichotomous" readers, it is the obvious next step after Bond & Fox.

Wolfe's answer to "what is measurement" restricts its use: "there is no discussion of sample size requirements … the book could lead a practitioner to erroneously conclude that the Rasch model can be utilized with virtually any data set." Certainly, we need to toss a coin more than once to check that it is fair. But how many times? 3, 5, 20, 1000? After 3 or 4 tosses we have a good idea. By the time we get to 10 we are convinced. Wright & Stone's (1979) "Best Test Design" (mentioned favorably by van der Linden) is based on the analysis of a data set comprising 35 children encountering 18 items. Only one Bond & Fox data set has fewer observations.

But this raises another fundamental question: when is there too little data for Rasch measurement to be informative? Is there a better alternative to Rasch for the analysis of small data sets? In the early days, test developers would make remarks such as "Rasch analysis ruined my perfectly good test!" In fact, Rasch analysis did not change their tests at all. It merely pointed out the flaws that were there all along. Ben Wright advised students to start Rasch analysis of their data just as soon as they started collecting it. Don't wait till you have 1000, 100 or even 10 cases to discover that a typographical error is making the answers to Question 3 unintelligible. Even when precision is lacking due to small sample size, a concern of Wolfe, data-quality-control and construct validity must still be there. If the data can possibly be Rasch analyzed, do it! With computers it takes almost no time, and what you learn may save you weeks of agony later. To take a statement by Fred Lord somewhat out of context, "Small N justifies the Rasch model."

Finally, along with Wolfe, "I applaud the authors …", and, along with Wilson, "I judge that Bond and Fox have largely succeeded …" Purchase this first edition, in anticipation of an even better second edition! [which has indeed emerged.]

John M. Linacre

Official authors' website: Bond & Fox

Hubisz, J. (2001) Review of Middle School Physical Science Texts. Final Report. The David and Lucile Packard Foundation. Grant #1998-4248.

van der Linden, W.J. (2001) Book Review - Applying the Rasch Model. International Journal Of Testing, 1(3&4), 319-326.

Wilson M. (2002) Book Review - Applying the Rasch Model: ... Applied Psychological Measurement, 26, 2.

Wolfe E.W. (2002) Book Review - Applying the Rasch Model: ... Journal of Applied Measurement, 3, 4.

Bond & Fox Figure 3.1
Bond & Fox (2001) Figure 3.1, p. 22

Review of reviews of Bond & Fox (2001). Van der Linden, WJ, Wilson, M, Wolfe, E, Linacre, JM. … 16:2 p.871-2


Review of reviews of Bond & Fox (2001). Van der Linden, WJ, Wilson, M, Wolfe, E, Linacre, JM. … Rasch Measurement Transactions, 2002, 16:2 p.871-2

Please help with Standard Dataset 4: Andrich Rating Scale Model



Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr. Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez

To be emailed about new material on www.rasch.org
please enter your email address here:

I want to Subscribe: & click below
I want to Unsubscribe: & click below

Please set your SPAM filter to accept emails from Rasch.org

www.rasch.org welcomes your comments:

Your email address (if you want us to reply):

 

ForumRasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

Coming Rasch-related Events
April 26-30, 2017, Wed.-Sun. NCME, San Antonio, TX, www.ncme.org - April 29: Ben Wright book
April 27 - May 1, 2017, Thur.-Mon. AERA, San Antonio, TX, www.aera.net
April 29, 2017, Sat., 16:35 to 18:05. NCME Presidents Invitational Symposium: a new book commemorating Ben Wright's life and career, 16:35 to 18:05, San Antonio, TX, www.ncme.org
May 26 - June 23, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 30 - July 29, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
July 31 - Aug. 3, 2017, Mon.-Thurs. Joint IMEKO TC1-TC7-TC13 Symposium 2017: Measurement Science challenges in Natural and Social Sciences, Rio de Janeiro, Brazil, imeko-tc7-rio.org.br
Aug. 7-9, 2017, Mon-Wed. In-person workshop and research coloquium: Effect size of family and school indexes in writing competence using TERCE data (C. Pardo, A. Atorressi, Winsteps), Bariloche Argentina. Carlos Pardo, Universidad Catòlica de Colombia
Aug. 7-9, 2017, Mon-Wed. PROMS 2017: Pacific Rim Objective Measurement Symposium, Sabah, Borneo, Malaysia, proms.promsociety.org/2017/
Aug. 10, 2017, Thurs. In-person Winsteps Training Workshop (M. Linacre, Winsteps), Sydney, Australia. www.winsteps.com/sydneyws.htm
Aug. 11 - Sept. 8, 2017, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Aug. 18-21, 2017, Fri.-Mon. IACAT 2017: International Association for Computerized Adaptive Testing, Niigata, Japan, iacat.org
Sept. 15-16, 2017, Fri.-Sat. IOMC 2017: International Outcome Measurement Conference, Chicago, jampress.org/iomc2017.htm
Oct. 13 - Nov. 10, 2017, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 5 - Feb. 2, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 10-16, 2018, Wed.-Tues. In-person workshop: Advanced Course in Rasch Measurement Theory and the application of RUMM2030, Perth, Australia (D. Andrich), Announcement
Jan. 17-19, 2018, Wed.-Fri. Rasch Conference: Seventh International Conference on Probabilistic Models for Measurement, Matilda Bay Club, Perth, Australia, Website
May 25 - June 22, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 29 - July 27, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
Aug. 10 - Sept. 7, 2018, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Oct. 12 - Nov. 9, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
The HTML to add "Coming Rasch-related Events" to your webpage is:
<script type="text/javascript" src="http://www.rasch.org/events.txt"></script>

 

The URL of this page is www.rasch.org/rmt/rmt162a.htm

Website: www.rasch.org/rmt/contents.htm