Researchers at HumRRO have produced one of the first-ever, practitioner-oriented guides on developing situational judgment tests (SJTs). Drawing on scientific literature and their own extensive research and real-world experience developing and implementing SJTs in high-stakes assessment contexts for public and private sector clients, Deborah L. Whetzel, Ph.D., Taylor S. Sullivan, Ph.D., and Rodney A McCloy, Ph.D., wrote “Situational Judgment Tests: An Overview of Development Practices and Psychometric Characteristics,” published in the journal, Personnel Assessment and Decisions. According to [email protected], the article has already been downloaded in the United States and internationally nearly 400 times since its publication in March.
SJTs assess individual judgment by presenting examinees with problems to solve via scenarios and a list of plausible response options. Examinees then evaluate each response option for addressing the problem described in the scenario.
The paper discusses a variety of issues that affect SJTs, including reliability, validity, group differences, presentation modes, faking, and coaching, and provides best-practice guidance for practitioners.
“Consistent with HumRRO’s mission to give back to the profession, we are sharing experience- and evidence-based conclusions and suggestions for improving the development of SJTs,” said Sullivan.
It is clear from both psychometric properties and examinee response behavior that not all SJT designs are equally effective, and not all designs may be appropriate for all intended uses and assessment goals. To help practitioners and researchers alike, the authors provide best practices for developing SJTs: