Jungsun Go

Date of Award

Summer 2023

Document Type

Open Access Dissertation

Department

Psychology

First Advisor

Christine DiStefano

Abstract

The purpose of this study was to investigate the effectiveness of four different models (bifactor, CTC(M-1), CTCU and unidimensional) as to optimal model selection when the wording effect associated with negatively worded items was present. A Monte Carlo simulation study was conducted to compare model-data fit and accuracy in parameter estimates of the four models. Design factors include: two true models (CTC(M-1), CTCU models) × three sample sizes (small N=250, moderate N=500, large N=1,000) × two item ratios (positive items:negative items = 2:1 or 1:1) × three distributions (distribution of negative items; symmetric, moderately asymmetric, and extremely asymmetric). The generated data were analyzed with two different estimators (MLMV, WLSMV) under four different models (bifactor, CTC(M-1), CTCU, and unidimensional). Outcomes examined included fit index values (chi-square, RMSEA, SRMR, CFI, TLI, AIC, BIC), power, Type I error, and relative bias in factor loadings on general trait factor estimated from the four tested models. As a second study, the four tested models were applied to empirical data set from the RSE scale to evaluate whether the simulation study results would be observed in practice.

Results showed that three tested models (bifactor, CTC(M-1), and CTCU) fit the data equivalently well regardless of study conditions. Misspecified unidimensional models would erroneously be selected as an optimal model when certain study conditions (small sample, asymmetrically distributed negative items, unbalanced number of positive and negative items, WLSMV estimator) were present. Among tested models, the CTC(M-1) tended to produce most accurate factor loadings for both positive and negative items. According to the study's findings, there could be overlaps in the fit index values under several alternative solutions. Researchers should not select one model over the other solutions only because it produced the best fit index values. Instead, a model selection should be determined based on the study's objectives or on substantive considerations. Additionally, researchers should carefully examine the data's features when misspecified unidimensional models yield goof-fit. Unidimensional models ignoring negative wording effects could be recognized as a good-fitting model in some circumstances because of different performance of fit indices. Several practical advice and guidelines are provided for researchers using unidimensional scales with mixed item wording.

Rights

© 2023, Jungsun Go

Download

Share

COinS