Institute for Objective Measurement

Certification Process

Data Analysis Component

 

 

#                   For qualified candidates, the IOM will make available the three following types of data sets:

 

"                   A multiple-choice test

"                    An attitude survey

"                   A rated performance-assessment

 

Candidates will then select two of these three.   The IOM will follow by supplying the two requested data sets and documentation (in ASCII format) for standard objective-measurement situations.

 

 

#                   To complete the practical component of the certification process, candidates must prepare two succinct and clear reports, and provide the appropriate supporting evidence, in the form of tables and graphs.   Candidates should treat the projects as pilot-tests of an instrument and include a discussion of at least the following:

 

"                   Construct validity (item and/or person measure hierarchy)

"                   Measure precision (reliability)

"                   Measure accuracy (quality-control fit)

"                   Rating-scale structure or distractor functioning

 

Along with the discussions of the above, candidates must also include their recommendations.   These recommendations should contain, but not be limited to, the strengths and weaknesses of the instrument and outcomes.

 

 

#                   Candidates may perform their analyses using any software they would like or, if they wish, they may perform them by hand.   They must use the report forms (to see an example of this form, see page three) for providing a one-page summary.

 

#                   In preparing their supporting documentation, candidates may not exceed twenty pages.   In addition, they should include in their reports only the key statistics, plots, and tables.

 

Note:     Since there are alternative software and methodological approaches within Rasch measurement, candidates should describe the general labeling that they used in the programming.

Data Analysis Component, page 2

 

 

#                   For each section (listed above) of the candidates’ two reports, three IOM reviewers will provide ratings for the following:

 

"                   The quality of the analysis

 

"                   The completeness of the analysis

 

"                   The appropriate use of objective-measurement analysis

 

 

#                   In providing these ratings, the reviewers will use the four-category scale below:

 

4     Mastery :          The candidate exhibits excellent understanding of the concepts             of objective-measurement

 

3     Adequate :        The candidate can produce results that are correct and                   sufficiently detailed to be meaningful

 

2     Deficient :        The candidate can produce reports correctly, but shows some misunderstanding in interpretation of the results

 

1     Unacceptable : The candidate produces the reports incorrectly in some, or most, instances. He/she does not interpret the information as expected for the results.

 

 

#                   To become certified, candidates must obtain an overall performance-level (adjusted for reviewer severity and project difficulty) equivalent to Adequate.   Should the reviewers doubt the ability of a candidate, for any reason, the IOM will ask the candidate to complete a third dataset analysis.

 

 

The IOM will offer candidates a period of three months to submit their completed data analyses.   When doing so, candidates should transmit their documents as an e-mail attachment (a zipped file, if necessary).   If electronic format is not possible, then candidates must send four copies of each report by regular mail.

 

Once candidates receive Diplomate status, their certification will remain valid for ten years.   During this period, Diplomates will be afforded the privilege of using the designation of Diplomate in Measurement (DM).   However, they must be certain not to refer to the certification status using any other terminology!   At the end of the ten-year period, Diplomates must once again qualify.

[Example]

Institute for Objective Measurement

Certification Process

Candidates Report Form

 

Candidates must label their supporting tables and other documentation, reference them on the cover sheet, and attach them in the order in which they are referenced.  If they wish to expand upon their comments about each outcome, candidates may use additional pages, as needed.   The entire report for each analysis must be less than 20 pages.

 

Report Contents

Construct Validity (item and/or person-measure hierarchy)

 

 

 

 

 

Measurement precision and reliability

 

 

 

 

 

 

Measure accuracy (quality-control fit)

 

 

 

 

 

Discussion of rating-scale or distractor functioning

 

 

 

 

 

Recommendations, including the strengths and weakness of the instrument and outcomes