Rasch Benchmarking

When a global 500 company was asked to assess whether they were a "Great Place to Work," they went through a lengthy vendor selection process to find a research organization that could conduct a benchmark study. The winning bid would take six months to complete at a cost of just north of $100,000. Then the economy went south and the project was canceled, but the need for the project persisted so we conducted a stealth benchmark study. Here's what we did.

A top-down approach: We put Business Week, Vault, and Consulting Magazine's ranking of company prestige into a matrix, converted the rankings into a five-point scale, ran them through Rasched them, and converted them to percentiles.

A bottom-up approach: We went to Vault.com, where employees give the inside scoop on the companies they work for and coded who is good to work for and why. For example, "[Company X's] promotion policy is among the best, if not the best in the industry. This is a major draw to X for prospective hires, and is truly one of the best aspects of the firm." This received a 1 under "Career Path," while "The company is certainly the opposite of a meritocracy, with advancement more dependent on who you know rather than what you know" received a 0. We ended up with half a dozen categories that were consistent across all companies: Compensation, Career Path, Culture, Diversity, Work-life Balance, Necessity of Face Time, and Training. As before, we put the data in a matrix, Rasched, and converted into percentiles.

A both approach: We then graphed the results (see Figure below) and were surprised to find A) that we stank and B) that a factor that distinguished whether a company was a great place to work was whether they offered tuition reimbursement. The latter (at least) was surprising, surprising because economists consistently argue that companies should not offer tuition reimbursement, because it increases employees' portable skills. "Thanks for the degree. Bye." The great companies, however, are not thinking economics. They are thinking game theory. You win the war for talent by outflanking your opponent. "Sure, you can steal my highly-educated people, but I can steal yours and we can all take from those who don't educate their people."

Three interesting things came from this study. First, it became part of a business case for us to create our own legal and accredited graduate program, which puts our competitors in a pickle, because it is too expensive to offer tuition reimbursement and their own degree and once you've offered reimbursement, employees will howl if you take it away. Second, it raises a philosophical issue; all of the data is subjective-prestige ratings and employee comments-but since anybody can replicate the study to get the same result, the study is objective. Third, we completed the study with no non-payroll cost in a week.

Tad Waddington

Waddington T. (2009) Rasch Benchmarking, Rasch Measurement Transactions, 2009, 23:1, 1200

Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr. Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez

To be emailed about new material on www.rasch.org
please enter your email address here:

I want to Subscribe: & click below
I want to Unsubscribe: & click below

Please set your SPAM filter to accept emails from Rasch.org

www.rasch.org welcomes your comments:

Your email address (if you want us to reply):


ForumRasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

Coming Rasch-related Events
Oct. 6 - Nov. 3, 2023, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Facets), www.statistics.com
Oct. 12, 2023, Thursday 5 to 7 pm Colombian timeOn-line workshop: Deconstruyendo el concepto de validez y Discusiones sobre estimaciones de confiabilidad SICAPSI (J. Escobar, C.Pardo) www.colpsic.org.co
June 12 - 14, 2024, Wed.-Fri. 1st Scandinavian Applied Measurement Conference, Kristianstad University, Kristianstad, Sweden http://www.hkr.se/samc2024
Aug. 9 - Sept. 6, 2024, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com


The URL of this page is www.rasch.org/rmt/rmt231k.htm

Website: www.rasch.org/rmt/contents.htm