scholarly journals Uniform Test Assembly: Concepts, Problems, Solvers, and Applications for Adaptive Testing

2017 ◽  
Vol 5 (1) ◽  
pp. 1-21 ◽  
Author(s):  
Dmitry Belov
Keyword(s):  
Adaptive Testing ◽  
Test Assembly

Benefits from Computerized Adaptive Testing as Seen in Simulation Studies

1999 ◽  
Vol 15 (2) ◽  
pp. 91-98 ◽  
Author(s):  
Lutz F. Hornke
Keyword(s):  
Measurement Error ◽  
Computerized Adaptive Testing ◽  
Test Procedure ◽  
Adaptive Testing ◽  
Parameter Estimates ◽  
Simulation Studies ◽  
Computerized Adaptive Test ◽  
Item Banks ◽  
Item Parameters ◽  
General Reliability

Summary: Item parameters for several hundreds of items were estimated based on empirical data from several thousands of subjects. The logistic one-parameter (1PL) and two-parameter (2PL) model estimates were evaluated. However, model fit showed that only a subset of items complied sufficiently, so that the remaining ones were assembled in well-fitting item banks. In several simulation studies 5000 simulated responses were generated in accordance with a computerized adaptive test procedure along with person parameters. A general reliability of .80 or a standard error of measurement of .44 was used as a stopping rule to end CAT testing. We also recorded how often each item was used by all simulees. Person-parameter estimates based on CAT correlated higher than .90 with true values simulated. For all 1PL fitting item banks most simulees used more than 20 items but less than 30 items to reach the pre-set level of measurement error. However, testing based on item banks that complied to the 2PL revealed that, on average, only 10 items were sufficient to end testing at the same measurement error level. Both clearly demonstrate the precision and economy of computerized adaptive testing. Empirical evaluations from everyday uses will show whether these trends will hold up in practice. If so, CAT will become possible and reasonable with some 150 well-calibrated 2PL items.

Methods for Restricting Maximum Exposure Rate in Computerized Adaptative Testing

Methodology ◽  
2007 ◽  
Vol 3 (1) ◽  
pp. 14-23 ◽  
Author(s):  
Juan Ramon Barrada ◽  
Julio Olea ◽  
Vicente Ponsoda
Keyword(s):  
Measurement Accuracy ◽  
Computerized Adaptive Testing ◽  
Computation Time ◽  
Adaptive Testing ◽  
Exposure Rate ◽  
Control Parameters ◽  
The Impact ◽  
Two Alternatives ◽  
Selection Of ◽  
Maximum Exposure

Abstract. The Sympson-Hetter (1985) method provides a means of controlling maximum exposure rate of items in Computerized Adaptive Testing. Through a series of simulations, control parameters are set that mark the probability of administration of an item on being selected. This method presents two main problems: it requires a long computation time for calculating the parameters and the maximum exposure rate is slightly above the fixed limit. Van der Linden (2003) presented two alternatives which appear to solve both of the problems. The impact of these methods in the measurement accuracy has not been tested yet. We show how these methods over-restrict the exposure of some highly discriminating items and, thus, the accuracy is decreased. It also shown that, when the desired maximum exposure rate is near the minimum possible value, these methods offer an empirical maximum exposure rate clearly above the goal. A new method, based on the initial estimation of the probability of administration and the probability of selection of the items with the restricted method ( Revuelta & Ponsoda, 1998 ), is presented in this paper. It can be used with the Sympson-Hetter method and with the two van der Linden's methods. This option, when used with Sympson-Hetter, speeds the convergence of the control parameters without decreasing the accuracy.

Optimal Test Assembly in Practice

2013 ◽  
Vol 221 (3) ◽  
pp. 190-200 ◽  
Author(s):  
Jörg-Tobias Kuhn ◽  
Thomas Kiefer
Keyword(s):  
Particle Physics ◽  
Large Scale ◽  
Block Design ◽  
Linear Optimization ◽  
Mixed Integer ◽  
Optimal Test ◽  
Automated Test Assembly ◽  
Test Assembly ◽  
Block Level ◽  
Mixed Integer Linear Optimization

Several techniques have been developed in recent years to generate optimal large-scale assessments (LSAs) of student achievement. These techniques often represent a blend of procedures from such diverse fields as experimental design, combinatorial optimization, particle physics, or neural networks. However, despite the theoretical advances in the field, there still exists a surprising scarcity of well-documented test designs in which all factors that have guided design decisions are explicitly and clearly communicated. This paper therefore has two goals. First, a brief summary of relevant key terms, as well as experimental designs and automated test assembly routines in LSA, is given. Second, conceptual and methodological steps in designing the assessment of the Austrian educational standards in mathematics are described in detail. The test design was generated using a two-step procedure, starting at the item block level and continuing at the item level. Initially, a partially balanced incomplete item block design was generated using simulated annealing, whereas in a second step, items were assigned to the item blocks using mixed-integer linear optimization in combination with a shadow-test approach.

MARKOV MATHEMATICAL MODEL OF DYNAMIC ADAPTIVE TESTING OF AN ACTIVE AGENT

2018 ◽  
pp. 29-35
Author(s):  
N. V. Brovka ◽  
P. P. Dyachuk ◽  
M. V. Noskov ◽  
I. P. Peregudova
Keyword(s):  
Mathematical Model ◽  
Finite State Machine ◽  
Active Agent ◽  
Adaptive Testing ◽  
State Machine ◽  
Learning Activities ◽  
Independent Learning ◽  
Complex Object ◽  
Total Reward ◽  
Finite State

The problem and the goal.The urgency of the problem of mathematical description of dynamic adaptive testing is due to the need to diagnose the cognitive abilities of students for independent learning activities. The goal of the article is to develop a Markov mathematical model of the interaction of an active agent (AA) with the Liquidator state machine, canceling incorrect actions, which will allow mathematically describe dynamic adaptive testing with an estimated feedback.The research methodologyconsists of an analysis of the results of research by domestic and foreign scientists on dynamic adaptive testing in education, namely: an activity approach that implements AA developmental problem-solving training; organizational and technological approach to managing the actions of AA in terms of evaluative feedback; Markow’s theory of cement and reinforcement learning.Results.On the basis of the theory of Markov processes, a Markov mathematical model of the interaction of an active agent with a finite state machine, canceling incorrect actions, was developed. This allows you to develop a model for diagnosing the procedural characteristics of students ‘learning activities, including: building axiograms of total reward for students’ actions; probability distribution of states of the solution of the problem of identifying elements of the structure of a complex object calculate the number of AA actions required to achieve the target state depending on the number of elements that need to be identified; construct a scatter plot of active agents by target states in space (R, k), where R is the total reward AA, k is the number of actions performed.Conclusion.Markov’s mathematical model of the interaction of an active agent with a finite state machine, canceling wrong actions allows you to design dynamic adaptive tests and diagnostics of changes in the procedural characteristics of educational activities. The results and conclusions allow to formulate the principles of dynamic adaptive testing based on the estimated feedback.

Sign in / Sign up

Export Citation Format

Share Document