A Two-step latent profile method for identifying invalid respondents in self-reported survey data

Abstract

Insincere respondents can have an adverse impact on the validity of substantive inferences arising from self-administered questionnaires (SAQs). The current study introduces a new method for identifying potentially invalid respondents from their atypical response patterns. The two-step procedure involves generating a response inconsistency (RI) score for each participant and scale on the SAQ and subjecting the resulting scores to latent profile analysis to identify classes of atypical RI respondent profiles. The procedure can be implemented post–data collection and is illustrated through a survey of school climate that was administered to N = 52,102 high school students. Results of this screening procedure revealed high levels of specificity and expected levels of concordance when contrasted with the results of traditionally used methods of screening items and response time. Contrasts between valid and invalid respondents revealed similar patterns across the three screening procedures when compared across external measures of academics and risk behaviors.