Three personnel assessment themes emerged from the formal presentations and informal conversations among other testing professionals at the conference: the impact of artificial intelligence (AI); increased use of skill-based assessments; and addressing diversity, equity, inclusion, and accessibility (DEIA) concerns. At HumRRO, we’ve conducted a great deal of research relevant to these themes that help guide our assessment decisions and solutions for clients:
Artificial intelligence (AI) will continue to have an important impact on assessment development and use. While generative AI appears to be effective at developing test items, concerns about legal and ethical issues associated with its use persist. At HumRRO, we’ve been conducting foundational research for several years on ways to leverage AI to improve efficiency for clients while navigating relevant legal and ethical issues. Some workplace examples include using AI methods to streamline job analyses and to develop a more intuitive way to identify related occupations, and in educational assessment, exploring how AI presents new challenges to building validity arguments.
Like the private sector, federal government agencies are developing and using more skills-based assessments. This is largely in response to the Trump Era Executive Order Modernizing and Reforming the Assessment and Hiring of Federal Job Candidates, which requires agencies to “assess candidates in a manner that does not rely solely on educational attainment to determine the extent to which candidates possess relevant knowledge, skills, competencies, and abilities.” The result has been an increased focus on both the development of custom assessment instruments and use of commercial off-the shelf testing. HumRRO has played a leading role in conducting research and developing applications of skill-based assessments, from our industry leading work on rich media assessments to our ongoing research on ways to improve situational judgment tests (SJTs) in high-stakes assessment contexts.
Organizations continue to search for better ways to ensure that their policies and procedures address diversity, equity, inclusion, and accessibility (DEIA) concerns. Conference presenters identified selection practices that improve fairness and equity, including increased use of skills-based hiring and removing coachable elements of assessments. Practitioners also discussed the difficulties competing with the private sector in luring top candidates from various underrepresented groups. For many years, HumRRO has taken proactive steps to promote assessment fairness. Recently, we conducted work on DEIA pulse checks, implicit bias training, and barrier analyses.
As always, testing professionals are highly engaged in tactical methods to improve the conduct of job analyses, test development efforts, and test administration, with several panels and presentations on these topics at IPAC 2023. Much of the conversation at the conference focused on new technologies to facilitate these efforts, including in such areas as virtual meeting and workshop platforms, online and remote testing, and the development of computer adaptive tests.