| Rating Scale Instrument Quality Criteria | |||||
|---|---|---|---|---|---|
| Criterion | Poor | Fair | Good | Very Good | Excellent |
| Targeting * | > 2 errors | 1-2 errors | < 1 error | < .5 error | < .25 error |
| Item Model Fit Mean-Square Range Extremes | < .33 - >3.0 | .34 - 2.9 | .5 - 2.0 | .71 - 1.4 | .77 - 1.3 |
| Person and Item Measurement Reliability | <.67 | .67-.80 | .81-.90 | .91-.94 | >.94 |
| Person and Item Strata Separated | 2 or less | 2-3 | 3-4 | 4-5 | >5 |
| Ceiling effect: % maximum extreme scores | >5% | 2-5% | 1-2% | .5-1% | <.5% |
| Floor effect: % minimum extreme scores | >5% | 2-5% | 1-2% | .5-1% | <.5% |
| Variance in data explained by measures | <50% | 50-60% | 60-70% | 70-80% | >80% |
| Unexplained variance in contrasts 1-5 of PCA of residuals | >15% | 10-15% | 5-10% | 3-5% | <3% |
| This Table has been developed by William P. Fisher, Jr. based on the Rasch literature and his many years of experience conducting Rasch analyses in different settings. | |||||
* How far the average or modal measure is from the center of the item calibrations. Just as a rough indication, if the average measure is less than one error of measurement from the center of the item hierarchy, then the instrument is on target. The average individual error of measurement made more sense to me to use than the SE of the mean, as this latter can be quite small. Of course, an instrument might not have items that go as far up or down the scale as the measures go, so some indication of how well the measure and calibration distributions match would also be in order.
Rating Scale Instrument Quality Criteria. Fisher, W.P. Jr. … Rasch Measurement Transactions, 2007, 21:1 p. 1095
The URL of this page is www.rasch.org/rmt/rmt211m.htm
Website: www.rasch.org/rmt/contents.htm