Automated test assembly (ATA) couples computing power with algorithm-based approaches to manage the complexities and competing goals of test assembly. At HumRRO, our industrial-organizational (I-O) psychologists use an ATA approach that structures the testing program requirements as a “problem” that is then “solved” by an integer programming algorithm, as Willem J. Van Der Linden described in his 2005 book, Linear Models for Optimal Test Design.
The test blueprint essentially creates boundary conditions—for example, each test must include 100 assigned items—within which all viable solutions must exist. The ATA algorithm then identifies the most favorable solution within the pre-specified boundary conditions to meet program goals, such as maximizing test-level reliability. This approach to ATA allows a systematic search of the viable solutions, iteratively improving upon the stated goals.
Over the years, we have designed and deployed custom ATA programs for a range of high-stakes assessments sponsored by organizations such as the Society for Human Resource Management (SHRM), the Federation of State Boards of Physical Therapy (FSBPT), the U.S. Department of State, and the U.S. Department of Defense. This experience allows us to set up and customize ATA processes for new clients efficiently and cost-effectively.