HumRRO Vice President, David Dorsey, Ph.D., and HumRRO Director of Education, Hillary Michaels, Ph.D., leveraged their unique expertise and experience to edit the Fall 2022 Special Issue of the Journal of Educational Measurement.
The issue, Validity Arguments Meet Artificial Intelligence in Innovative Educational Assessment, discusses artificial intelligence (AI)-enabled assessments—tools used in organizational and workforce development and, increasingly, in education that have intrigued scientists and practitioners. While AI presents the lure of efficiency and effectiveness, Dorsey and Michaels caution that users must maintain high standards for validity and fairness, which can be challenging in the complicated world of AI-enabled assessment.
With that in mind, Dorsey and Michaels curated a series of original research articles from practitioners and researchers that tackle the challenges of AI validity arguments in educational assessment domains. Specifically, the articles explore how AI impacts Evidence-Centered Design (ECD) and development from assessment concept and coding to scoring and reporting. They also present information on ways to ensure that bias is not built into these systems. Lastly, they discuss future horizons, many that are almost here, for maximizing what AI offers while minimizing negative effects on test takers and programs.
It was an honor to work with some of the top educational measurement specialists in the field to wrestle with tough issues around AI-enabled assessments and validity arguments. This topic will surely remain important for the field going forward, so our hope was to begin a conversation and start an exploration of related topics.
— Hillary Michaels, Ph.D.
The JEM Special Issue includes the following articles:
“For those interested in AI-enabled assessments—practitioners and researchers alike—it is critical that we keep validity arguments front and center,” said Dorsey. “For this special issue, we sought to find contributions that showed a way forward for ensuring that our ability to produce robust validity arguments keeps pace with the ever-increasing power of AI.”