Optimal Designs in Computerized Adaptive Testing

Abstract

Computerized adaptive testing (CAT) using information-based item selection rules (ISR) (e.g., maximum Fisher information) shows an improvement in test efficiency over linear testing but also leads to the undesirable result of unbalanced item usage. To achieve the optimal CAT design, the concept of global adaptiveness (GA) (Chen, Chao, & Chen, 2020) is introduced. While traditional CATs consider how to maximize information at a single-item administration, GA-based CATs additionally consider how to maximize test information for all examinees. The GA is further applied to develop ISRs, including the dynamic Stratification method based on Dominance Curves (SDC) (heuristic approach) and Integer Linear Programing Approach based on Real-time Test Data (IPRD) (0-1 programming approach). Specifically, these ISRs utilize information regarding real-time test data to optimize each item administration from a comprehensive perspective. According to simulation studies, SDC and IPRD can efficiently improve both trait estimation precision and item pool usage while satisfying all test requirements in most test scenarios.