Does the choice of response time threshold procedure substantially affect inferences concerning the identification and exclusion of rapid guessing responses? A meta-analysis

Background In testing contexts that are predominately concerned with power, rapid guessing (RG) has the potential to undermine the validity of inferences made from educational assessments, as such responses are unreflective of the knowledge, skills, and abilities assessed. Given this concern, practitioners/researchers have utilized a multitude of response time threshold procedures that classify RG responses in these contexts based on either the use of no empirical data (e.g., an arbitrary time limit), response time distributions, and the combination of response time and accuracy information. As there is little understanding of how these procedures compare to each other, this meta-analysis sought to investigate whether threshold typology is related to differences in descriptive, measurement property, and performance outcomes in these contexts. Methods Studies were sampled that: (a) employed two or more response time (RT) threshold procedures to identify and exclude RG responses on the same computer-administered low-stakes power test; and (b) evaluated differences between procedures on the proportion of RG responses and responders, measurement properties, and test performance. Results Based on as many as 86 effect sizes, our findings indicated non-negligible differences between RT threshold procedures in the proportion of RG responses and responders. The largest differences for these outcomes were observed between procedures using no empirical data and those relying on response time and accuracy information. However, these differences were not related to variability in aggregate-level measurement properties and test performance. Conclusions When filtering RG responses to improve inferences concerning item properties and group score outcomes, the actual threshold procedure chosen may be of less importance than the act of identifying such deleterious responses. However, given the conservative nature of RT thresholds that use no empirical data, practitioners may look to avoid the use of these procedures when making inferences at the individual-level, given their potential for underclassifying RG.

Selective reinforcement of conflict processing in the Stroop task

Motivation signals have been shown to influence the engagement of cognitive control processes. However, most studies focus on the invigorating effect of reward prospect, rather than the reinforcing effect of reward feedback. The present study aimed to test whether people strategically adapt conflict processing when confronted with condition-specific congruency-reward contingencies in a manual Stroop task. Results show that the size of the Stroop effect can be affected by selectively rewarding responses following incongruent versus congruent trials. However, our findings also suggest important boundary conditions. Our first two experiments only show a modulation of the Stroop effect in the first half of the experimental blocks, possibly due to our adaptive threshold procedure demotivating adaptive behavior over time. The third experiment showed an overall modulation of the Stroop effect, but did not find evidence for a similar modulation on test items, leaving open whether this effect generalizes to the congruency conditions, or is stimulus-specific. More generally, our results are consistent with computational models of cognitive control and support contemporary learning perspectives on cognitive control. The findings also offer new guidelines and directions for future investigations on the selective reinforcement of cognitive control processes.

Selective reinforcement of conflict processing in the Stroop task

Motivation signals have been shown to influence the engagement of cognitive control processes. However, most studies focus on the invigorating effect of reward prospect, rather than the reinforcing effect of reward feedback. The present study aimed to test whether people strategically adapt conflict processing when confronted with condition-specific congruency-reward contingencies in a manual Stroop task. Results show that the size of the Stroop effect can be affected by selectively rewarding responses following incongruent versus congruent trials. However, our findings also suggest important boundary conditions. Our first two experiments only show a modulation of the Stroop effect in the first half of the experimental blocks, possibly due to our adaptive threshold procedure demotivating adaptive behavior over time. The third experiment showed an overall modulation of the Stroop effect, but did not find evidence for a similar modulation on test items, leaving open whether this effect generalizes to the congruency conditions, or is stimulus-specific. More generally, our results are consistent with computational models of cognitive control and support contemporary learning perspectives on cognitive control. The findings also offer new guidelines and directions for future investigations on the selective reinforcement of cognitive control processes.

A Fuzzy Logic System for Seizure Onset Detection in Intracranial EEG

We present a multistage fuzzy rule-based algorithm for epileptic seizure onset detection. Amplitude, frequency, and entropy-based features were extracted from intracranial electroencephalogram (iEEG) recordings and considered as the inputs for a fuzzy system. These features extracted from multichannel iEEG signals were combined using fuzzy algorithms both in feature domain and in spatial domain. Fuzzy rules were derived based on experts' knowledge and reasoning. An adaptive fuzzy subsystem was used for combining characteristics features extracted from iEEG. For the spatial combination, three channels from epileptogenic zone and one from remote zone were considered into another fuzzy subsystem. Finally, a threshold procedure was applied to the fuzzy output derived from the final fuzzy subsystem. The method was evaluated on iEEG datasets selected from Freiburg Seizure Prediction EEG (FSPEEG) database. A total of 112.45 hours of intracranial EEG recordings was selected from 20 patients having 56 seizures was used for the system performance evaluation. The overall sensitivity of 95.8% with false detection rate of 0.26 per hour and average detection latency of 15.8 seconds was achieved.

Reliability and Validity of Infant Speech-Sound Discrimination-in-Noise Thresholds

Infants were tested on a speech-sound discrimination-in-noise task using the visual reinforcement infant speech discrimination (VRISD) procedure with an adaptive (up-down) threshold protocol. An adult control group was tested using the same stimuli and apparatus. The speech sounds were synthetic /ba/ and /ga/. The masker was band-passed noise presented continuously at 48 dB SPL. Test-retest reliability was good for both groups, although test-retest differences were smaller for adults. For infants the mean of the absolute values of the differences between tests was only 5.2 dB, and there was less than a 10-dB difference between the two tests of 14 (87.5%) of the 16 infants completing the study. The infant-adult difference in discrimination threshold in noise was 6.9 dB, which agrees well with detection-in-noise thresholds from earlier studies and with discrimination-in-noise thresholds obtained on a subset of subjects in our earlier work. Advantages of the adaptive threshold procedure and its possible applications both in research studies and in the clinic are discussed.

Infant Thresholds with Enhanced Attention to the Signal in Visual Reinforcement Audiometry

Experimental data show a discrepancy between adult auditory thresholds and best estimates of infant thresholds. The source of the discrepancy has not been determined. Attentional differences between infants and adults and procedural differences to accommodate infant attentional behavior may account at least in part for the discrepancy. A standard operant procedure, Visual Reinforcement Audiometry (VRA), was modified to enhance infant attention to impending auditory signals. Results on 16 infants evaluated with conventional VRA and modified VRA revealed an average 5.5 dB improvement in threshold with the modified technique. Correction for adult performance in similar tasks indicated a 3.3 dB attentional effect between infant and adult thresholds. The shift in threshold reflects on the adequacy Of VRA as a threshold procedure.

Use of a Monosyllabic Adaptive Speech Test (MAST) with Young Children

Sixty children aged 3, 5, and 7 years were tested using a simple up-down adaptive speech threshold procedure. The test stimuli were familiar monosyllabic words presented as a closed set with a picture-pointing response. The results indicate that monosyllabic adaptive speech test (MAST) procedures can be used reliably with children as young as 3 years of age. Thirty of the children also received a different randomization of the same speech stimuli presented at a constant level, equal to their MAST threshold. The results confirmed the accuracy of the MAST estimate of the children's 50% speech threshold. Further support for the validity of the MAST threshold procedure with young children was obtained using a group of 10 children with conductive hearing loss. Their results show a significant correlation between the MAST threshold and pure-tone loss. The data also indicated significant improvement in MAST thresholds over the three age groups investigated. These developmental changes are discussed in terms of a word frequency effect.

Measurement of Speech Thresholds of Spanish-Speaking Children by Non-Spanish Speaking Clinicians

This study investigated the feasibility of a prerecorded speech threshold procedure that was used in a picture-pointing format, and was administered to Spanish-speaking children by non-Spanish-speaking clinicians. The derived Spanish word list was compared for equivalency to English spondees on a group of bilingual adults. The test, administered to 16 children ages three to six years, resulted in good agreement between SRT and pure-tone average. The test was found to be feasible, rapid, and reliable.