Evaluation of careless or insufficient effort responses given to scales

Abstract

Scales are commonly administered to participants in various studies, including those in psychology, sociology, and medicine; however, obtaining accurate results from these applied scales is often challenging due to participants typically providing careless or insufficient effort responses. In this study, Cattell's sabotage index and the modified fixed individualized chance (MFIC) score will be used to assess the effectiveness of these indices in identifying careless or insufficient effort responses, using data obtained through simulation. The purpose of the simulation study is to compare the performance of these indices based on metrics such as root mean square error (RMSE) and bias (BIAS). These comparisons will be conducted under varying conditions, taking into account factors such as scale type, the number of items, the number of reversed items, and sample size. The study aims to determine how these factors influence the occurrence of careless or insufficient effort responding across different scenarios. Cattell's sabotage index outperforms the modified fixed individualized chance (MFIC) score in identifying careless or insufficient effort responses. For Cattell's sabotage index, high performance is observed in large samples at a careless and insufficient effort response rate of 0.30.