The Ability of an Item

At a 1990 AERA Rasch SIG session, the history of some key ideas underlying fundamental measurement was reviewed by both the presenters and the discussants. It was suggested that measurement in the physical sciences is "direct", e.g. a ruler measuring the height (length) of a person itself has a length. On the other hand, measurement in the social sciences is "indirect", e.g. an item measuring the ability of a person does not itself have an ability. In this analysis I will show that items do have abilities, and that it is only because we call their abilities "difficulties", that we lose sight of the similarity between physical and social measurement.

When one thinks of a ruler and its application, one thinks of equally spaced marks on a continuum with the feature that, when the ruler covers the appropriate range, any person can be located between a pair of adjacent marks. Latent trait theory and Rasch measurement specify that the items of a test can be located on a continuum onto which persons can also be mapped. The response of any person to any item is characterized by a probabilistic model and we expect that, from a set of relevant responses, each person can be located on the continuum, and therefore, in general, be located above some and below other items on that continuum. The locations of the items are counterparts to the marks on a ruler. So far, then, the analogy between physical and social measurement is maintained. There are two differences, but they are not critical to the case to be made: one is that items are seldom equally spaced like the markings on a ruler, and the other is that the location of a person is probabilistic and not deterministic as is mistakenly presumed in the case of measurement by a ruler. When the markings on the ruler are very close, a probabilistic model, usually the normal, is applied to determine whether the object to be measured is located above or below any mark.

The locations of the items on the continuum are generally called "difficulties" and the locations of the persons are generally called "abilities". However, because both the items and the persons are located on the same continuum, it follows that either the items and the persons both have difficulties or the items and the persons both have abilities. It is only a matter of convention that different words are used for the locations of persons and items. Thus it may be said that, by definition, the items have abilities.

In order to provide a substantive, in addition to semantic, case that an item has an ability, consider the history of assessment and the intention behind the construction and use of items. It is only recently that emphasis has been placed on the written item. Even now much testing is carried out without formal written items. The main form of assessment, conducted by teachers across the whole range of education, is in oral interchange between teacher and student. At one end of the education continuum, there is the Ph.D. oral examination; at the other is the oral assessment of students by teachers in schools as they supervise their students' studies. The written item has become so widespread because of the development of educational assessment technology. Literacy, including literacy in particular subject disciplines, can only be assessed by including a written component in the assessment. In addition, it is necessary to retain records of the tasks and performances, sometimes for legal reasons. The science of educational measurement has shown the potential for unreliability in oral assessments. The need to increase reliability and validity by controlling disturbing factors has enhanced the establishment of written forms of assessment. The assessment of abilities through written items, then, can be seen as an evolution from oral assessment, but a technical evolution that has retained the essential intention and process of oral assessment.

In an oral assessment, the examiner may pose a question, and the student may answer it. By focusing on oral assessment, it becomes clear that the examiner, the instrument of assessment, must have an ability of the same kind as is being assessed in the student, (and hopefully more of it, just as a ruler should have a greater length than the person whose height is being measured). The examiner may choose to manifest his or her ability in posing a question at a particular level. If the student's answer manifests an ability that the examiner is satisfied is above that level, then the examiner may manifest a higher level of ability by posing a harder question, and thus check whether the student can also manifest a correspondingly higher level of ability. This process continues, as it does in individually administered intelligence test such as the Binet, until the student cannot answer further questions requiring yet greater levels of ability.

This kind of interaction between examiner and student can be compared to aligning an object with a ruler to see at what point the object is beyond one mark but before the next one. Here we can invoke another analogy with physical measurement - the measurement of the mass of an object by adding a series of calibrated masses, "weights", to one side of a beam balance until it balances the other side.

Thus it can be seen that the written item is a substitute, a formal and written substitute, for an interaction between two persons, the examiner and the student. In writing the item, the examiner has to manifest the same ability that is to be assessed in the student. Clearly, the construction of the items can fall short of assessing only the intended ability. This is why developments in educational measurement have centered on concepts like reliability and validity. However, the key point remains. When items are to be constructed in a particular field, it is those with a level of knowledge, at least up to the required level in the field, and not those with less than the required level of knowledge, who are asked to construct the items. In constructing the items, they are supposed to manifest their knowledge in the items, In short, the items are expressions of the ability of the person constructing the items. When students respond to the items, they are engaging in an interaction with the manifested ability of another person: the conclusion is unavoidable, therefore, that items do have abilities, and that either physical measurement is equally "indirect" or social measurement is equally "direct".

A ruler manifests its length through its elongated shape and its sequence of markings. Both sides of a beam balance manifest their masses through the effects of gravity on them. Likewise, examiners, who write test items against which students are to pit their abilities, manifest their own abilities through the items they construct. Moreover, the ruler, the beam balance, and test items, all must be administered and used according to prescribed procedures before their intended property is manifested well enough to lead to measurement. The ruler and the beam balance can be used for all kinds of other purposes - as weapons, toys, souvenirs, and so on. Without correct administration, they do not manifest their property of length or mass in any special way. Indeed, every object has some length or mass, and yet every object is not used for measuring length or weight. An engagement between examiners and students is an engagement of abilities on both sides. To ensure that the examiners' and the students' abilities are manifested successfully for measurement, the abilities of the examiners are administered as written test items, and the abilities of the students are recorded as written responses.



The Ability of an Item, D Andrich … Rasch Measurement Transactions, 1990, 4:2 p. 101-102




Rasch-Related Resources: Rasch Measurement YouTube Channel
Rasch Measurement Transactions & Rasch Measurement research papers - free An Introduction to the Rasch Model with Examples in R (eRm, etc.), Debelak, Strobl, Zeigenfuse Rasch Measurement Theory Analysis in R, Wind, Hua Applying the Rasch Model in Social Sciences Using R, Lamprianou El modelo métrico de Rasch: Fundamentación, implementación e interpretación de la medida en ciencias sociales (Spanish Edition), Manuel González-Montesinos M.
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Rasch Models for Measurement, David Andrich Constructing Measures, Mark Wilson Best Test Design - free, Wright & Stone
Rating Scale Analysis - free, Wright & Masters
Virtual Standard Setting: Setting Cut Scores, Charalambos Kollias Diseño de Mejores Pruebas - free, Spanish Best Test Design A Course in Rasch Measurement Theory, Andrich, Marais Rasch Models in Health, Christensen, Kreiner, Mesba Multivariate and Mixture Distribution Rasch Models, von Davier, Carstensen
Rasch Books and Publications: Winsteps and Facets
Applying the Rasch Model (Winsteps, Facets) 4th Ed., Bond, Yan, Heene Advances in Rasch Analyses in the Human Sciences (Winsteps, Facets) 1st Ed., Boone, Staver Advances in Applications of Rasch Measurement in Science Education, X. Liu & W. J. Boone Rasch Analysis in the Human Sciences (Winsteps) Boone, Staver, Yale Appliquer le modèle de Rasch: Défis et pistes de solution (Winsteps) E. Dionne, S. Béland
Introduction to Many-Facet Rasch Measurement (Facets), Thomas Eckes Rasch Models for Solving Measurement Problems (Facets), George Engelhard, Jr. & Jue Wang Statistical Analyses for Language Testers (Facets), Rita Green Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments (Facets), George Engelhard, Jr. & Stefanie Wind Aplicação do Modelo de Rasch (Português), de Bond, Trevor G., Fox, Christine M
Exploring Rating Scale Functioning for Survey Research (R, Facets), Stefanie Wind Rasch Measurement: Applications, Khine Winsteps Tutorials - free
Facets Tutorials - free
Many-Facet Rasch Measurement (Facets) - free, J.M. Linacre Fairness, Justice and Language Assessment (Winsteps, Facets), McNamara, Knoch, Fan

To be emailed about new material on www.rasch.org
please enter your email address here:

I want to Subscribe: & click below
I want to Unsubscribe: & click below

Please set your SPAM filter to accept emails from Rasch.org

www.rasch.org welcomes your comments:

Your email address (if you want us to reply):

 

ForumRasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

Coming Rasch-related Events
Oct. 4 - Nov. 8, 2024, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 17 - Feb. 21, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
May 16 - June 20, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 20 - July 18, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Further Topics (E. Smith, Facets), www.statistics.com
Oct. 3 - Nov. 7, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com

 

The URL of this page is www.rasch.org/rmt/rmt42a.htm

Website: www.rasch.org/rmt/contents.htm