The use of item scores and response times to detect examinees who may have benefited from item preknowledge

2019 ◽  
Vol 73 (3) ◽  
pp. 397-419 ◽  
Author(s):  
Sandip Sinharay ◽  
Matthew S. Johnson
Keyword(s):  
Response Times ◽  
Item Scores ◽  
Item Preknowledge

scholarly journals Detection of Item Preknowledge Using Response Times

2020 ◽  
Vol 44 (5) ◽  
pp. 376-392
Author(s):  
Sandip Sinharay
Keyword(s):  
Type I Error ◽  
Response Times ◽  
Real Data ◽  
Standard Normal Distribution ◽  
Major Type ◽  
Type I ◽  
Standard Normal ◽  
The Difference ◽  
Item Scores ◽  
Item Preknowledge

Benefiting from item preknowledge is a major type of fraudulent behavior during educational assessments. This article suggests a new statistic that can be used for detecting the examinees who may have benefited from item preknowledge using their response times. The statistic quantifies the difference in speed between the compromised items and the non-compromised items of the examinees. The distribution of the statistic under the null hypothesis of no preknowledge is proved to be the standard normal distribution. A simulation study is used to evaluate the Type I error rate and power of the suggested statistic. A real data example demonstrates the usefulness of the new statistic that is found to provide information that is not provided by statistics based only on item scores.

scholarly journals Detecting Examinees With Item Preknowledge in Large-Scale Testing Using Extreme Gradient Boosting (XGBoost)

2019 ◽  
Vol 79 (5) ◽  
pp. 931-961 ◽  
Author(s):  
Cengiz Zopluoglu
Keyword(s):  
Large Scale ◽  
Response Times ◽  
Characteristic Curve ◽  
Gradient Boosting ◽  
Data Set ◽  
Extreme Gradient Boosting ◽  
Positive Rate ◽  
Item Responses ◽  
Dichotomous Item ◽  
Item Preknowledge

Researchers frequently use machine-learning methods in many fields. In the area of detecting fraud in testing, there have been relatively few studies that have used these methods to identify potential testing fraud. In this study, a technical review of a recently developed state-of-the-art algorithm, Extreme Gradient Boosting (XGBoost), is provided and the utility of XGBoost in detecting examinees with potential item preknowledge is investigated using a real data set that includes examinees who engaged in fraudulent testing behavior, such as illegally obtaining live test content before the exam. Four different XGBoost models were trained using different sets of input features based on (a) only dichotomous item responses, (b) only nominal item responses, (c) both dichotomous item responses and response times, and (d) both nominal item responses and response times. The predictive performance of each model was evaluated using the area under the receiving operating characteristic curve and several classification measures such as the false-positive rate, true-positive rate, and precision. For comparison purposes, the results from two person-fit statistics on the same data set were also provided. The results indicated that XGBoost successfully classified the honest test takers and fraudulent test takers with item preknowledge. Particularly, the classification performance of XGBoost was reasonably good when the response time information and item responses were both taken into account.

scholarly journals Latent-variable Approaches Utilizing Both Item Scores and Response Times To Detect Test Fraud

2021 ◽  
Vol 3 (1) ◽  
pp. 1-16
Author(s):  
Sandip Sinharay
Keyword(s):  
Latent Variable ◽  
Response Times ◽  
Real Data ◽  
Latent Variable Models ◽  
Detailed Review ◽  
Item Scores ◽  
Educational Tests

Abstract There is a growing interest in approaches based on latent-variable models for detecting fraudulent behavior on educational tests. Wollack and Schoenig (2018) noted the presence of five types of statistical/psychometric approaches to detect the three broad types of test fraud that occur in educational tests. This paper includes a brief review of the five types of statistical/psychometric approaches mentioned by Wollack and Schoenig (2018). This paper then includes a more detailed review of the recent approaches for detecting test fraud using both item scores and response times—all of these approaches are based on latent-variable models. A real data example demonstrates the use of two of the approaches.

Nanostructure of semiconductor quantum dots in a borosilicate glass matrix by complementary use of HREM and AEM

1990 ◽  
Vol 48 (4) ◽  
pp. 728-729
Author(s):  
M.J. Kim ◽  
L.C. Liu ◽  
S.H. Risbud ◽  
R.W. Carpenter
Keyword(s):  
Quantum Dots ◽  
Borosilicate Glass ◽  
Glass Matrix ◽  
Optical Switching ◽  
Response Times ◽  
Fast Response ◽  
Processing Technique ◽  
Internal Stresses ◽  
Electron Hole ◽  
Spatial Dimensions

When the size of a semiconductor is reduced by an appropriate materials processing technique to a dimension less than about twice the radius of an exciton in the bulk crystal, the band like structure of the semiconductor gives way to discrete molecular orbital electronic states. Clusters of semiconductors in a size regime lower than 2R {where R is the exciton Bohr radius; e.g. 3 nm for CdS and 7.3 nm for CdTe) are called Quantum Dots (QD) because they confine optically excited electron- hole pairs (excitons) in all three spatial dimensions. Structures based on QD are of great interest because of fast response times and non-linearity in optical switching applications.In this paper we report the first HREM analysis of the size and structure of CdTe and CdS QD formed by precipitation from a modified borosilicate glass matrix. The glass melts were quenched by pouring on brass plates, and then annealed to relieve internal stresses. QD precipitate particles were formed during subsequent "striking" heat treatments above the glass crystallization temperature, which was determined by differential thermal analysis.

Compact and efficient gas diffusion electrodes based on nanoporous alumina membranes for microfuel cells and gas sensors

The Analyst ◽  
2020 ◽  
Vol 145 (1) ◽  
pp. 122-131 ◽  
Author(s):  
Wanda V. Fernandez ◽  
Rocío T. Tosello ◽  
José L. Fernández
Keyword(s):  
Response Times ◽  
Hydrogen Oxidation ◽  
Fast Response ◽  
Gas Diffusion ◽  
Limiting Current ◽  
Gas Diffusion Electrodes ◽  
Nanoporous Alumina ◽  
Alumina Membranes ◽  
Current Densities ◽  
Microfuel Cells

Gas diffusion electrodes based on nanoporous alumina membranes electrocatalyze hydrogen oxidation at high diffusion-limiting current densities with fast response times.

The S-SH Confusion Test and the Effects of Frequency Lowering

2019 ◽  
Vol 62 (5) ◽  
pp. 1486-1505
Author(s):  
Joshua M. Alexander
Keyword(s):  
Cognitive Processing ◽  
Hearing Aids ◽  
Response Times ◽  
Nonlinear Frequency ◽  
Low Frequencies ◽  
Frequency Compression ◽  
Negative Side ◽  
Minimal Word ◽  
Low Pass ◽  
Negative Side Effects

PurposeFrequency lowering in hearing aids can cause listeners to perceive [s] as [ʃ]. The S-SH Confusion Test, which consists of 66 minimal word pairs spoken by 6 female talkers, was designed to help clinicians and researchers document these negative side effects. This study's purpose was to use this new test to evaluate the hypothesis that these confusions will increase to the extent that low frequencies are altered.MethodTwenty-one listeners with normal hearing were each tested on 7 conditions. Three were control conditions that were low-pass filtered at 3.3, 5.0, and 9.1 kHz. Four conditions were processed with nonlinear frequency compression (NFC): 2 had a 3.3-kHz maximum audible output frequency (MAOF), with a start frequency (SF) of 1.6 or 2.2 kHz; 2 had a 5.0-kHz MAOF, with an SF of 1.6 or 4.0 kHz. Listeners' responses were analyzed using concepts from signal detection theory. Response times were also collected as a measure of cognitive processing.ResultsOverall, [s] for [ʃ] confusions were minimal. As predicted, [ʃ] for [s] confusions increased for NFC conditions with a lower versus higher MAOF and with a lower versus higher SF. Response times for trials with correct [s] responses were shortest for the 9.1-kHz control and increased for the 5.0- and 3.3-kHz controls. NFC response times were also significantly longer as MAOF and SF decreased. The NFC condition with the highest MAOF and SF had statistically shorter response times than its control condition, indicating that, under some circumstances, NFC may ease cognitive processing.ConclusionsLarge differences in the S-SH Confusion Test across frequency-lowering conditions show that it can be used to document a major negative side effect associated with frequency lowering. Smaller but significant differences in response times for correct [s] trials indicate that NFC can help or hinder cognitive processing, depending on its settings.

Causal attribution and counterfactual thinking - when does performing one facilitate performance of the other

2003 ◽  
Vol 62 (4) ◽  
pp. 209-218
Author(s):  
A. N’gbala ◽  
N. R. Branscombe
Keyword(s):  
Response Times ◽  
Causal Attribution ◽  
Counterfactual Thinking ◽  
The Other

When do causal attribution and counterfactual thinking facilitate one another, and when do the two responses overlap? Undergraduates (N = 78) both explained and undid, in each of two orders, events that were described either with their potential causes or not. The time to perform either response was recorded. Overall, mutation response times were shorter when performed after an attribution was made than before, while attribution response times did not vary as a consequence of sequence. Depending on whether the causes of the target events were described in the scenario or not, respondents undid the actor and assigned causality to another antecedent, or pointed to the actor for both responses. These findings suggest that counterfactual mutation is most likely to be facilitated by attribution, and that mutation and attribution responses are most likely to overlap when no information about potential causes of the event is provided.

Psychophysiological Responses to Stress Related to Anxiety in Healthy Aging

2019 ◽  
Vol 33 (3) ◽  
pp. 188-197 ◽  
Author(s):  
Roberta Adorni ◽  
Agostino Brugnera ◽  
Alessia Gatti ◽  
Giorgio A. Tasca ◽  
Kaoru Sakatani ◽  
...  
Keyword(s):  
Healthy Aging ◽  
Near Infrared ◽  
Response Times ◽  
Cognitive Resources ◽  
Stressful Condition ◽  
Healthy Elderly ◽  
Situational Stress ◽  
Anxiety Response ◽  
Nirs Signal ◽  
Psychophysiological Correlates

Abstract. The aim of the study was to explore the effects of situational stress and anxiety in a group of healthy elderly, both in terms of psychophysiological correlates and cognitive performance. Eighteen participants ( Mage = 70 ± 6.3; range 60–85) were assessed for anxiety and were instructed to perform a computerized math task, under both a stressful and a control condition, while near-infrared spectroscopy (NIRS) signal and electrocardiography (ECG) were recorded. NIRS results evidenced an increased activation of right PFC during the entire procedure, even if effect sizes between left and right channels were larger during the experimental condition. The amount of right activation during the stressful condition was positively correlated with anxiety. Response times (RTs) were slower in more anxious than in less anxious individuals, both during the control and stressful conditions. Accuracy was lower in more anxious than in less anxious individuals, only during the stressful condition. Moreover, heart rate (HR) was not modulated by situational stress, nor by anxiety. Overall, the present study suggests that in healthy elderly, anxiety level has a significant impact on cerebral responses, and both on the amount of cognitive resources and the quality of performance in stressful situations.

The Latin Square Task as a Measure of Relational Reasoning

2020 ◽  
Vol 36 (2) ◽  
pp. 296-302 ◽  
Author(s):  
Luke J. Hearne ◽  
Damian P. Birney ◽  
Luca Cocchi ◽  
Jason B. Mattingley
Keyword(s):  
Brain Imaging ◽  
Response Times ◽  
Latin Square ◽  
Error Rates ◽  
Functional Brain Imaging ◽  
Inspection Time ◽  
Relational Reasoning ◽  
Validity And Reliability ◽  
The Stability ◽  
Test Retest Reliability

Abstract. The Latin Square Task (LST) is a relational reasoning paradigm developed by Birney, Halford, and Andrews (2006) . Previous work has shown that the LST elicits typical reasoning complexity effects, such that increases in complexity are associated with decrements in task accuracy and increases in response times. Here we modified the LST for use in functional brain imaging experiments, in which presentation durations must be strictly controlled, and assessed its validity and reliability. Modifications included presenting the components within each trial serially, such that the reasoning and response periods were separated. In addition, the inspection time for each LST problem was constrained to five seconds. We replicated previous findings of higher error rates and slower response times with increasing relational complexity and observed relatively large effect sizes (η2p > 0.70, r > .50). Moreover, measures of internal consistency and test-retest reliability confirmed the stability of the LST within and across separate testing sessions. Interestingly, we found that limiting the inspection time for individual problems in the LST had little effect on accuracy relative to the unconstrained times used in previous work, a finding that is important for future brain imaging experiments aimed at investigating the neural correlates of relational reasoning.

Disfluent, But Fast

2019 ◽  
Vol 66 (5) ◽  
pp. 346-354
Author(s):  
Štěpán Bahník
Keyword(s):  
Cognitive Processing ◽  
Response Times ◽  
Processing Fluency ◽  
Retrieval Fluency ◽  
Decision Times

Abstract. Processing fluency, a metacognitive feeling of ease of cognitive processing, serves as a cue in various types of judgments. Processing fluency is sometimes evaluated by response times, with shorter response times indicating higher fluency. The present study examined existence of the opposite association; that is, it tested whether disfluency may lead to faster decision times when it serves as a strong cue in judgment. Retrieval fluency was manipulated in an experiment using previous presentation and phonological fluency by varying pronounceability of pseudowords. Participants liked easy-to-pronounce and previously presented words more. Importantly, their decisions were faster for hard-to-pronounce and easy-to-pronounce pseudowords than for pseudowords moderate in pronounceability. The results thus showed an inverted-U shaped relationship between fluency and decision times. The findings suggest that disfluency can lead to faster decision times and thus demonstrate the importance of separating different processes comprising judgment when response times are used as a measure of processing fluency.

Sign in / Sign up

Export Citation Format

Share Document