scholarly journals Beyond test accuracy: Benefits of measuring response time in computerised testing

2007 ◽  
Vol 23 (4) ◽  
Author(s):  
Eugene Gvozdenko ◽  
Dianne Chambers
Keyword(s):  
Response Time ◽  
Teaching And Learning ◽  
Response Times ◽  
Response Time Distribution ◽  
Test Accuracy ◽  
Individual Response ◽  
Time Data ◽  
Cognitive Demands ◽  
Mathematics Skills ◽  
The Impact

<span>This paper investigates how monitoring the time spent on a question in a test of basic mathematics skills can provide insights into learning processes, the quality of test takers' knowledge, and cognitive demands and performance of test items that otherwise would remain undiscovered if the usual test outcome of accuracy only format ("correct/incorrect") was used. Data from three tests of basic mathematics skills taken by tertiary students in 2004-2006 were analysed. Means and distributions of individual response times on parallel test questions were examined and differences were further investigated.</span><p>Analysis of response time data revealed a number of surprising findings in regard to the impact of variables on preferences for written and mental calculation methods and regarding additional cognitive demands of a question. The study examined how simple statistical analysis of response time distribution can be used to investigate abnormalities of the item functioning. These findings may be of value to educators and to test producers by informing them about the potential of utilising response time measurements as a diagnostic facility in computerised tests, for the purposes of improving teaching and learning.</p>

scholarly journals Response Time Distribution Analysis of Semantic and Response Interference in a Manual Response Stroop Task

2019 ◽  
Vol 66 (3) ◽  
pp. 231-238 ◽  
Author(s):  
Nabil Hasshim ◽  
Michelle Downes ◽  
Sarah Bate ◽  
Benjamin A. Parris
Keyword(s):  
Response Time ◽  
Stroop Task ◽  
Time Distribution ◽  
Response Times ◽  
Response Time Distribution ◽  
Stroop Interference ◽  
Response Conflict ◽  
Distribution Analysis ◽  
Manual Response ◽  
Response Interference

Abstract. Previous analyses of response time distributions have shown that the Stroop effect is observed in the mode (μ) and standard deviation (σ) of the normal part of the distribution, as well as its tail (τ). Specifically, interference related to semantic and response processes has been suggested to specifically affect the mode and tail, respectively. However, only one study in the literature has directly manipulated semantic interference, and none manipulating response interference. The present research aims to address this gap by manipulating both semantic and response interference in a manual response Stroop task, and examining how these components of Stroop interference affect the response time distribution. Ex-Gaussian analysis showed both semantic and response conflict to only affect τ. Analyzing the distribution by rank-ordered response times (Vincentizing) showed converging results as the magnitude of both semantic and response conflict increased with slower response times. Additionally, response conflict appeared earlier on the distribution compared to semantic conflict. These findings further highlight the difficulty in attributing specific psychological processes to different parameters (i.e., μ, σ, and τ). The effect of different response modalities on the makeup of Stroop interference is also discussed.

Unravelling the relationship between response time and user experience in mobile applications

2020 ◽  
Vol 30 (5) ◽  
pp. 1353-1382
Author(s):  
Mengli Yu ◽  
Ronggang Zhou ◽  
Zhao Cai ◽  
Chee-Wee Tan ◽  
Huiwen Wang
Keyword(s):  
Response Time ◽  
User Experience ◽  
Mobile Applications ◽  
Response Times ◽  
Negative Influence ◽  
Three Dimensions ◽  
Network Environment ◽  
Content Type ◽  
Varied Response ◽  
The Impact

PurposeThis study examines the impact of response time on user experience for mobile applications and considers the moderating influence of gender and network environment on this relationship.Design/methodology/approachAn experiment was conducted with 50 young adults to evaluate their user experience of a mobile application that simulates variations in network environment and response time. User experience was evaluated based on the three constituent dimensions of tolerance, acceptance, and satisfaction.FindingsAnalytical results demonstrate that response time not only adversely affects user experience of mobile applications, but that this effect is not homogeneous across the three dimensions of tolerance, acceptance and satisfaction. The findings also illustrate that gender moderates the effect of response time on user experience, however, the negative influence is more salient for males than females, which is opposite to our hypothesis. The joint moderating influence of gender and network environment turned out to be partly significant.Practical implicationsBy illuminating users' tolerance, acceptance, and satisfaction with varied response times, findings from this study can inform the design of mobile applications such that desired levels of user experience can be assured with minimum resources.Originality/valueAlthough response time has been hailed as a key determinant of user experience for desktop applications, there is a paucity of studies that have investigated the impact of response time on user experience for mobile applications. Furthermore, prior research on response time neglects the multi-dimensional nature of user experience. This study bridges the above mentioned knowledge gaps by delineating user experience into its constituent dimensions and clarifying the effects of response time on each of these dimensions.

scholarly journals Effects of Visual and Auditory Response Times in Males and Females by using Tactile and Mouse on Web Based Environment

2019 ◽  
Vol 8 (3) ◽  
pp. 6458-6464
Keyword(s):  
Response Time ◽  
Web Application ◽  
Response Times ◽  
Research Work ◽  
Visual Response ◽  
Time Data ◽  
Web Based ◽  
Male Response ◽  
Scientific Disciplines ◽  
Auditory Response

The purpose of this research work is to find out whether one of visual response time or auditory response time is faster than another on web based environment. Simple reaction time can be strong when an individual is asked to press a button as soon as a light or sound appears. A tool for capturing response times is created. This tool is a web application www.responsetime.muhammadsuhaib.com that runs on server and can be accessed through internet. R is commonly used in countless scientific disciplines for statistical analysis. Collected data will be analyzed by using R. it is concluded that on web based environment visual response times are not different than auditory response time. The data are also classified by gender to compare male response time and female response time, data analyzed based on gender, on visual stimuli, male react faster than female. However, on auditory stimuli, there is no different between male and female

Measures of phonological typicality

2010 ◽  
Vol 5 (3) ◽  
pp. 281-299 ◽  
Author(s):  
Padraic Monaghan ◽  
Morten H. Christiansen ◽  
Thomas A. Farmer ◽  
Stanka A. Fitneva
Keyword(s):  
Lexical Decision ◽  
Response Time ◽  
Word Processing ◽  
Response Times ◽  
Parameter Variation ◽  
Response Time Data ◽  
Speech Sounds ◽  
Time Data ◽  
Monosyllabic Word ◽  
Lexical Categories

Phonological Typicality (PT) is a measure of the extent to which a word’s phonology is typical of other words in the lexical category to which it belongs. There is a general coherence among words from the same category in terms of speech sounds, and we have found that words that are phonologically typical of their category tend to be processed more quickly and accurately than words that are less typical. In this paper we describe in greater detail the operationalisation of measures of a word’s PT, and report validations of different parameterisations of the measure. For each variant of PT, we report the extent to which it reflects the coherence of the lexical categories of words in terms of their sound, as well as the extent to which the measure predicts naming and lexical decision response times from a database of monosyllabic word processing. We show that PT is robust to parameter variation, but that measures based on PT of uninflected words (lemmas) best predict response time data for naming and lexical decision of single words.

Open source online behavioural experimentation using Collector: Proof of principle & sample size considerations

2020 ◽  
Author(s):  
Anthony Haffey ◽  
Kristian Tempelmans Plat ◽  
Prasannajeet Mane ◽  
Adam Blake ◽  
Bhismadev Chakrabarti
Keyword(s):  
Open Source ◽  
Task Analysis ◽  
Response Times ◽  
Online Research ◽  
Time Data ◽  
Future Studies ◽  
Home Based ◽  
The Impact ◽  
Accuracy Data ◽  
Proof Of Principle

Online research is becoming more common worldwide, offering rapid and remote data collection. However, systematic comparisons of data metrics in lab and home-based studies within the same individuals are lacking. Collector is an open source platform for collecting behavioural and survey data online. The current study compared the response times and accuracy on a simple inhibition task in the same group of participants, when run in the lab on Collector and E-Prime, as well as in home settings on Collector. The aim was to estimate the impact of differences in mode of administration (online vs. offline) as well as setting (lab vs. home) on performance in this task. Analysis of the compatibility effect found that performance in each setting reliably predicted performance in the other settings. These results also provide an estimate of the number of participants needed for testing in home settings to get a similarly significant result as in lab settings. While required sample sizes for lab and home settings were found to be similar for response time data, these were significantly different for accuracy data on this task. These considerations are vital in designing future studies that investigate smaller effects.

scholarly journals Analysis of speed-accuracy trade-offs using the Wiener diffusion model for choice reaction times

2021 ◽  
Author(s):  
Annalise Aleta LaPlume
Keyword(s):  
Response Time ◽  
Diffusion Model ◽  
Response Times ◽  
Reaction Times ◽  
Response Time Data ◽  
Choice Response ◽  
Time Data ◽  
Trade Offs ◽  
R Programming ◽  
Speed Accuracy

A methodology review paper on the utility and challenges of modelling speed-accuracy trade-offs in response time data. The paper reviews the importance of accounting for speed-accuracy trade-offs when measuring response times, and provides background on diffusion models for response time data. It then describes a practical software implementation of the EZ-diffusion model to model speed-accuracy trade-offs in choice response time data using the R programming language.

scholarly journals Why is hitting A&E time targets so hard?: using Nudge theory and modelling to improve response times

BJPsych Open ◽  
2021 ◽  
Vol 7 (S1) ◽  
pp. S223-S223
Author(s):  
Kaj Svedberg
Keyword(s):  
Response Time ◽  
Statistical Difference ◽  
Distribution Curve ◽  
Response Times ◽  
University Hospital ◽  
Time Data ◽  
Performance Targets ◽  
Significant Statistical Difference ◽  
Large Display ◽  
The One

AimsTo improve the one hour response times to referrals made to psychiatric Liaison in A&E without adding or changing available resources.MethodResponse time data of referrals made to the Homerton University Hospital psychiatric liaison service was collected dating back from August 2016 to October 2019 (n = 10225).A nudge was introduced in the form of a large display showing referrals arriving in real time in the staff office.Data was then collected over a period of 5 weeks (n = 436) to measure if any change had occurred in response times.ResultResponse times appear to follow a Poisson like distribution curve. The average referral was responded to within 6 minutes (n = 1577) prior to the nudge, and 6 minutes (n = 88) after. Prior to the nudge the 95% referral envelope fell within 134 minutes (n = 9728) and was 122 minutes (n = 414) after the intervention. Significant statistical difference is observed upon considering response in the first 240 minutes.ConclusionNudge interventions could be a useful resource-sparing method to improve services. The average referral to the HUH liaison team was quickly responded to within 6 minutes and yet hitting the 1 hour 95% target appears ever-elusive. Hitting targets of 95% responses within 1 hour may prove very difficult if we are not considering natural distributions, such as Poisson, occuring in the backgroung which ultimately may require a change in approaches to how we set performance targets.

scholarly journals Testing the Within-State Distribution in Mixture Models for Responses and Response Times

2020 ◽  
pp. 107699862095724
Author(s):  
Renske E. Kuijpers ◽  
Ingmar Visser ◽  
Dylan Molenaar
Keyword(s):  
Response Time ◽  
Mixture Models ◽  
Time Distribution ◽  
Response Times ◽  
Response Time Distribution ◽  
Mixture Modeling ◽  
State Distribution ◽  
Distributional Assumption ◽  
Modeling Framework ◽  
State Response

Mixture models have been developed to enable detection of within-subject differences in responses and response times to psychometric test items. To enable mixture modeling of both responses and response times, a distributional assumption is needed for the within-state response time distribution. Since violations of the assumed response time distribution may bias the modeling results, choosing an appropriate within-state distribution is important. However, testing this distributional assumption is challenging as the latent within-state response time distribution is by definition different from the observed distribution. Therefore, existing tests on the observed distribution cannot be used. In this article, we propose statistical tests on the within-state response time distribution in a mixture modeling framework for responses and response times. We investigate the viability of the newly proposed tests in a simulation study, and we apply the test to a real data set.

scholarly journals Variable Speed Across Dimensions of Ability in the Joint Model for Responses and Response Times

2021 ◽  
Vol 12 ◽  
Author(s):  
Peida Zhan ◽  
Hong Jiao ◽  
Kaiwen Man ◽  
Wen-Chung Wang ◽  
Keren He
Keyword(s):  
Response Time ◽  
Latent Variable ◽  
Response Times ◽  
Complex Structure ◽  
Joint Model ◽  
Time Data ◽  
Common Assumption ◽  
Test Items ◽  
Proposed Model ◽  
Specific Skill

Working speed as a latent variable reflects a respondent’s efficiency to apply a specific skill, or a piece of knowledge to solve a problem. In this study, the common assumption of many response time models is relaxed in which respondents work with a constant speed across all test items. It is more likely that respondents work with different speed levels across items, in specific when these items measure different dimensions of ability in a multidimensional test. Multiple speed factors are used to model the speed process by allowing speed to vary across different domains of ability. A joint model for multidimensional abilities and multifactor speed is proposed. Real response time data are analyzed with an exploratory factor analysis as an example to uncover the complex structure of working speed. The feasibility of the proposed model is examined using simulation data. An empirical example with responses and response times is presented to illustrate the proposed model’s applicability and rationality.

Sign in / Sign up

Export Citation Format

Share Document