Modality Effects in Free Recall: A Retrieved-Context Account

The modality effect refers to the robust finding that memory performance differs for items presented aurally, as compared with visually. Whereas auditory presentation leads to stronger recency performance in immediate recall, visual presentation often produces better primacy performance (the inverse modality effect). To investigate and model these differences, we conducted two large-scale web-based immediate free recall experiments. In both experiments, participants studied visual and auditory word lists of varying lengths and rates of presentation. We observed typical modality and inverse modality effects, while also discovering that participants were more likely to initiate recall from recent items on auditory trials than on visual trials. However, modality effects persisted regardless of the first item recalled. Meanwhile, an analysis of intrusion errors revealed that participants were more likely on visual trials than on auditory trials to erroneously recall words from one list prior. Furthermore, words presented in the same modality as the present list intruded more often than those presented in a different modality. We next developed a retrieved-context account of the modality effect by fitting the Context Maintenance and Retrieval model to data across multiple list lengths. Through our simulations, we demonstrate that the modality effect can be explained by faster contextual drift and stronger context-to-item association formation during auditory presentation, relative to visual. Our modeling shows that modality effects can arise without hypothesizing distinct memory stores for recent and remote information. Finally, we propose that modality effects may derive primarily from the temporal dynamics of stimuli, rather than their modality.

Sustained Auditory Attentional Load Decreases Audiovisual Integration in Older and Younger Adults

The modulation of attentional load on the perception of auditory and visual information has been widely reported; however, whether attentional load alters audiovisual integration (AVI) has seldom been investigated. Here, to explore the effect of sustained auditory attentional load on AVI and the effects of aging, nineteen older and 20 younger adults performed an AV discrimination task with a rapid serial auditory presentation task competing for attentional resources. The results showed that responses to audiovisual stimuli were significantly faster than those to auditory and visual stimuli ( AV > V ≥ A , all p < 0.001 ), and the younger adults were significantly faster than the older adults under all attentional load conditions (all p < 0.001 ). The analysis of the race model showed that AVI was decreased and delayed with the addition of auditory sustained attention ( no _ load > load _ 1 > load _ 2 > load _ 3 > load _ 4 ) for both older and younger adults. In addition, AVI was lower and more delayed in older adults than in younger adults in all attentional load conditions. These results suggested that auditory sustained attentional load decreased AVI and that AVI was reduced in older adults.

EXPRESS: Long-term written language experience affects grammaticality judgments and usage but not priming of spoken sentences

‘Book language’ offers a richer linguistic experience than conversational speech in terms of its syntactic properties. Here, we investigated the role of long-term syntactic experience on syntactic knowledge and processing. In a pre-registered study with 161 adult native Dutch speakers with varying levels of literacy, we assessed the contribution of individual differences in written language experience to offline and online syntactic processes. Offline syntactic knowledge was assessed as accuracy in an auditory grammaticality judgment task in which we tested violations of four Dutch grammatical norms. Online syntactic processing was indexed by syntactic priming of the Dutch dative alternation, using a comprehension-to-production priming paradigm with auditory presentation. Controlling for the contribution of non-verbal IQ, verbal working memory, and processing speed, we observed a robust effect of literacy experience on the detection of grammatical norm violations in spoken sentences, suggesting that exposure to the syntactic complexity and diversity of written language has specific benefits for general (modality-independent) syntactic knowledge. We replicated previous results by finding robust comprehension-to-production structural priming, both with and without lexical overlap between prime and target. Although literacy experience affected the usage of syntactic alternates in our large sample, it did not modulate their priming. We conclude that amount of experience with written language increases explicit awareness of grammatical norm violations and changes the usage of (PO vs. DO) dative spoken sentences but has no detectable effect on their implicit syntactic priming in proficient language users. These findings constrain theories about the effect of long-term experience on syntactic processing.

Joint entrainment to the speech envelope during speaking and listening cannot completely explain brain-to-brain synchronisation

Speakers and listeners coordinate brain activity during conversation. How this is done is less well understood. It could be due only to hearing the same speech—i.e., self–produced speech when speaking, and another’s speech when listening. Or, it could be due to coordination beyond that which is explained by the physical acoustic characteristics of the signal alone. This study investigates the dynamics of speech envelope tracking and inter–brain coupling during speaking and listening. After time–locking EEG data collection and auditory recording and playback, we used a Gaussian copula mutual information measure to estimate related information content between the EEG and auditory signals, as well as between two different EEG signals from the same participant. We refer to this design as auto–pseudo–hyperscanning. In the 2–10 Hz frequency range, we identified different latencies for maximal speech envelope tracking during speech production and speech perception. Maximal speech tracking takes place after auditory presentation during perception and before vocalisation during speech production. We also identified time–lags showing a significant increase in time–locked EEG responses during speaking and self–listening: The EEG signal in the listener both preceded and followed the EEG signal of the speaker at latencies beyond those that could be explained by auditory entrainment during speech perception or production. The results indicate that brain–to–brain synchronisation during speaking and listening cannot be explained only by joint entrainment to the speech envelope.

Coherence formation during narrative text processing: a comparison between auditory and audiovisual text presentation in 9- to 12-year-old children

In an experiment with 114 children aged 9–12 years, we compared the ability to establish local and global coherence of narrative texts between auditory and audiovisual (auditory text and pictures) presentation. The participants listened to a series of short narrative texts, in each of which a protagonist pursued a goal. Following each text, we collected the response time to a query word that was either associated with a near or a distant causal antecedent of the final sentence. Analysis of these response times indicated that audiovisual presentation has advantages over auditory presentation for accessing information relevant for establishing both local and global coherence, but there are indications that this effect may be slightly more pronounced for global coherence.

An investigation of novel carrying experience on infant rhythm perception

Prior research suggests infants who are regularly carried in an infant sling are more accurate at moving in time with an auditory rhythm. Moreover, the rate of walking cadence during infant carrying was found to impact infant spontaneous motor tempo. In the current study we ask if part of the advantage of locomotive experience on sensorimotor synchronization results from greater sensorimotor neural activation, utilized to predict the beat, during the perception of rhythmic auditory stimuli. Five-month-old infants were carried at either a Fast or Slow rate. EEG was then used to measure infants’ recruitment of a sensorimotor program, indexed by sensorimotor alpha suppression, during the auditory presentation of the Fast and Slow rhythms. Our results suggest that infants do not seem to preferentially exploit a sensorimotor representation when hearing an auditory tempo that matched the rate at which they were carried, compared to when hearing an incongruent tempo.

Solo piano compositions by Hsiao Tyzen in the aspect of performing problems

Background. The article is dedicated to the piano work of Hsiao Tyzen (1938–2015) – one of Taiwan’s most famous composers. The solo piano compositions of the musician are considered, which make up a significant part of his compositional heritage: three cycles, united under the general name “Poetic Response” – op. 37 (1974), op. 38 (1975) and op. 40 (1977); “The Amazing Grace” (1984), cycle “Memories of Home” op. 49 (1987), consisting of six plays – “Prelude”, “Memory”, “Playground”, “Ancient Taiwanese Melody”, “Elegy”, “Frolicking”; “Farewell Etude”, Op. 55 (1993), “Toccata”, op. 57 (1995), “Dragon Boat Festival”, op. 58 (1996), “Spirit of Taiwan” (1998), “Nana Oh’s Meditation” (1999), “The Angel from Formosa” (1999). This huge layer of music is not sufficiently studied in the performing aspect. The characteristic of the pianistic level of complexity of the compositions under consideration is given, technical and artistic difficulties are revealed. Objectives. The purpose of the study is to identify the main performing tasks in the solo piano compositions of Hsiao Tyzen. Methods of research are based on a set of scientific approaches necessary for the disclosure of its theme. The complex approach, combining the principle of musical-theoretical, musical-historical and performing analysis, is taken as the basis of the methodology. Results. Piano works of Hsiao Tyzen of an early period of art op. 37, 38 and 40, united in the general cycle “Poetic Response”, are devoted to religious themes and include melodies of religious hymns. Combining three diverse piano cycles, the composer builds a kind of complex form of cycles in the cycle. The influence of romanticism in the music of Hsiao Tyzen is felt in his interpretation of melody, harmony, rhythm, tempo and texture. Composers of the twentieth century, especially C. Debussy, influenced the piano compositions of the late period. Since the works created by Hsiao Tyzen have varying degrees of pianistic complexity, it seems important to determine the pedagogical significance of the uncomplicated piano repertoire and the performing tasks that the interpreter of concert compositions faces. The cycle “Memories of Home” op. 49, consisting of six miniatures, is dedicated to the composer’s childhood memories. The main tasks of the pianist in the “Ancient Taiwanese melody” from the cycle “Memories of Home”, op. 49 will be the auditory implementation and development of a touch of legato, the performance of melismatics, the observance of sound balance between hands, the ability to draw a long melodic line, cleverly using a finger swap and moving from one position to another. In the Prelude, the tasks of the performer’s main technical and artistic problems are to accurately reflect dynamic contrasts, the agility of transitions in various textured combinations, the sound realization of polyphony, precise articulation and coordination of small notes in passages. “Memory” requires the performer of the highly professional possession of legato, manifested in the combination of the upper voice of the chord musical fabric, flexible movement skills and an accurate sense of polyrhythmia. In the “Playground” you need to show imaginative imagination, while reflecting the variety of strokes and dynamics specified by the author. “Elegy” requires a deep soulful feeling from a performer, high-quality sounding of a melody, and the formation of long phrases. “Farewell Etude” op. 55 and “Toccata” op. 57 – detailed compositions saturated with romantic technique and imagery. The intonational filling of the plays reflects the national Taiwanese flavor. “Farewell sketch” was the last work of Hsiao Tyzen, written in the tradition of romanticism. The piece is based on the famous Taiwanese folk song “Four Seasons”. The composer places the melody in the middle register, framing on both sides with a luxurious romantic texture. “Toccata” is full of numerous techniques that are difficult enough not only to execute, but even to remember. Frequent change of textured formulas is especially difficult for a pianist, because in addition to the clever execution of a virtuoso texture, you need to keep an accurate rhythmic pulsation. This repertoire is intended for concert performance and requires a pianist of a high professional pianistic level and bright artistry. It requires scale of performance, absolute technical and sound knowledge of the texture, knowledge and auditory presentation of the characteristics of Taiwanese musical culture, and mastery of pedalization. Conclusions. Hsiao Tyzen’s solo piano compositions provide a better understanding of the work of contemporary Taiwanese composers. They are rightfully one of the most striking pages of Taiwanese musical culture and deserve further introduction into a wide international music audience. These works, in our opinion, have high artistic merits and are intended for a different contingent of performers. Their value as a pedagogical and concert repertoire is a vivid imagery, a reflection of the national principle, interesting compositional and sound solutions.

Effect of Text-to-Speech Rate on Reading Comprehension by Adults With Aphasia

Purpose Accessing auditory and written material simultaneously benefits people with aphasia; however, the extent of benefit as well as people's preferences and experiences may vary given different auditory presentation rates. This study's purpose was to determine how 3 text-to-speech rates affect comprehension when adults with aphasia access newspaper articles through combined modalities. Secondary aims included exploring time spent reviewing written texts after speech output cessation, rate preference, preference consistency, and participant rationales for preferences. Method Twenty-five adults with aphasia read and listened to passages presented at slow (113 words per minute [wpm]), medium (154 wpm), and fast (200 wpm) rates. Participants answered comprehension questions, selected most and least preferred rates following the 1st and 3rd experimental sessions and after receiving performance feedback, and explained rate preferences and reading and listening strategies. Results Comprehension accuracy did not vary significantly across presentation rates, but reviewing time after cessation of auditory content did. Visual data inspection revealed that, in particular, participants with substantial extra reviewing time took longer given fast than medium or slow presentation. Regardless of exposure amount or receipt of performance feedback, participants most preferred the medium rate and least preferred the fast rate; rationales centered on reading and listening synchronization, benefits to comprehension, and perceived normality of speaking rate. Conclusion As a group, people with aphasia most preferred and were most efficient given a text-to-speech rate around 150 wpm when processing dual modality content; individual differences existed, however, and mandate attention to personal preferences and processing strengths.

Surprise-induced Deafness: Unexpected Auditory Stimuli Capture Attention to the Detriment of Subsequent Detection

Our attention is often captured by unexpected or unusual sounds. Such stimulus- driven control of attention can be adaptive, as potentially relevant events need to be quickly evaluated and acted upon. Attentional capture, however, comes with a cost: Ongoing tasks may be disrupted. In a series of seven experiments (n=773), we investigated the effects of task-irrelevant, rare, and relatively unexpected sounds (“surprise stimuli”) on probe detection in rapid auditory presentation (RAP) streams. Surprise stimuli caused “Surprise-induced Deafness” (SiD), a severe detection deficit that lasted for under one second within each trial and gradually habituated across several trials. SiD was sensitive to informational “surprise”, with larger deficits following stimuli that were infrequent or varied across trials. The effect also generalized: Natural sounds or constructed stimuli could disrupt detection of either spoken letters or simple tones. We also compared SiD to the auditory attentional blink (AAB), a similar paradigm in which goal-directed target processing disrupts probe detection. We found that the two deficits were weakly correlated. We conclude that SiD is a novel perceptual deficit that primarily reflects stimulus-driven attentional capture. It may involve other forms of attentional control as well, thereby reflecting multiple attentional influences on awareness.