Item Analysis Issues

TCRM Workshop Sessions 2
Session ID#: 
007-TC2A

Discussants: Isaac Washburn and Arthur Greil
Presider: Sarah Shoppe-Sullivan

Date: 
Tuesday, November 15, 2011
Time: 
1:15 pm - 2:45 pm
Session Location: 
Salon 13
Session Type: Paper, TCRM

About the Session

  • Strategies We Now Know for Analyzing Data from Respondents who "Don't Know"
    Presented by: Rebekah L. Young, David R. Johnson

  • Evaluating the Functioning of Response Categories Using Rasch Analysis
    Presented by:
    Keitaro Yoshida, Dean M. Busby

Strategies We Now Know for Analyzing Data from Respondents who "Don't Know"
Presented by:
Rebekah L. Young, David R. Johnson

Many surveys include "don't know" or "no opinion" options to ensure answers to all questions. Unfortunately, no guidelines exist for how to handle these responses in data analysis. Researchers make varying decisions about handling "don't know" responses while little attention is given to the possibility that research findings may be conditional on the meaning attributed to and the handling of these responses. This paper first develops a conceptual framework for evaluating the meaning of the "don't know" responses for a survey items. The second part of the paper uses a simulation approach to tests several analytic strategies for handling "don't know" responses in the analysis. It concludes with recommendations as to the best approach for the different situations.

Evaluating the Functioning of Response Categories Using Rasch Analysis
Presented by:
Keitaro Yoshida, Dean M. Busby

The subscales of the RELATionship Evaluation (RELATE; Busby, Holman, & Taniguchi, 2001) were evaluated using Rasch analysis to understand the common patterns in the malfunctioning response categories and the impact of the modification of the response categories. The authors found that infrequently used extreme categories tend to misfit or be disordered. On the other hand, the middle categories representing a neutral state tend to be statistically indistinguishable from the adjacent categories. The modification of the response categories for a subscale resulting in an improvement of its functioning supports the usefulness of Rasch analysis in evaluating the functioning of response categories.