A representational asymmetry for composition in the human left-middle temporal gyrus

Natural language is notable amongst representational systems for the rich internal structure of phrase and sentence-level expressions. Here, we provide evidence from two fMRI studies that a region of the left Middle Temporal Gyrus (MTG) exhibits a surprising representational asymmetry: verbs and patients (to whom was it done?) are bound to form a representation, but verbs and agents (who did it?) are not. Within MTG, BOLD signal to novel combinations of familiar components can be modeled by combining learned verb-patient conjunctive representations with more general agent representations, but not by the converse (verb-agent + patient). This asymmetry is not predicted by an abstract propositional representation of the event (e.g., chased(dog,cat), nor by a theory which derives conjunctions from the experienced statistical co-occurences between verbs and nouns. However, this asymmetry is predicted by various linguistic accounts of the internal structure of event descriptions (e.g., Williams, 1981; Marantz,1984; Grimshaw, 1990; Kratzer, 1996). These results provide evidence for the time-varying instantiation of re-usable representations of structure in MTG, consistent with the principle of compositionality, as well as accounts of verb-argument structure.

Why Maps are Not Propositional

Many philosophers and logicians assume an exhaustive and exclusive dichotomy between “imagistic”, iconic, or pictorial representations and “discursive”, logical, or propositional ones. Maps seem to fall somewhere in between, with different theorists assimilating them to one or the other side of the divide. Given this assumption, philosophers and logicians interested in defending the logical tractability of maps have typically analyzed them as being predicative, where this is understood as a species of logical, propositional representation. This chapter argues that the best way to interpret the debate about propositionality is as concerning a representational system’s operative functional structure. Propositional structure is claimed to exhibit several distinctive properties: it is digital, asymmetrical, general, recursive, and hierarchical. However, there is little positive evidence that cartographic structure exhibits these features in the relevant sense.

Semantics for a Module

Fodor’s The Modularity of Mind (1983) and subsequent work propose a principled distinction between perceptual computations and background knowledge. The chapter argues that language input analyzers produce a minimally—and highly constrained—context-sensitive propositional representation of the sentence, built up from sentence constituents. Compatible with the original Modularity story, it thus takes the output of sentence perception to be a “shallow” representation—though a semantic one. The empirical data discussed bear on alleged cases of sentence indeterminacy and how such cases might be assigned (shallow) semantic representations, interact with context in highly regulated ways, and whether and how they can be enriched. The chapter proposes a semantic level of representation that serves as output of the module and as input to other systems of interpretation, arguing for a form of modularity or encapsulation that is minimally context-sensitive provided that the information from context is itself determined by linguistic principles.

(Non-)Sense in note-taking for consecutive interpreting

The paper applies cognitive theories of text and language processing, and in particular relevance theory, to the analysis of notes in consecutive interpreting. In contrast to the pre-cognitive view, in which note-taking is seen mainly as a memory-supporting technique, the process of note-taking is described as the reception and production of a notation text. Adding the relevance-theoretical constructs of explicature and implicature to the general account of cognitive text processing as coherence building and the construction of a mental representation at local and global levels, this approach allows for the comparison of source, notation and target texts with respect to the underlying propositional representation, and shows how the sense of highly fragmentary notation texts is recovered in consecutive interpreting. The paper is based on an empirical study involving consecutive interpretations (English–German) by five trainee interpreters. The analysis shows that the interpreters operate relatively closely along micropropositional lines when processing the source, notation and target texts, with the explicature regularly having the same propositional form as the corresponding proposition in the source text.

MULTIMODAL COMMUNICATION FROM MULTIMODAL THINKING — TOWARDS AN INTEGRATED MODEL OF SPEECH AND GESTURE PRODUCTION

A computational model for the automatic production of combined speech and iconic gesture is presented. The generation of multimodal behavior is grounded in processes of multimodal thinking, in which a propositional representation interacts and interfaces with an imagistic representation of visuo-spatial imagery. An integrated architecture for this is described, in which the planning of content and the planning of form across both modalities proceed in an interactive manner. Results from an empirical study are reported that inform the on-the-spot formation of gestures.

Discourse Representation in the Two Cerebral Hemispheres

Readers construct at least two interrelated representations when they comprehend a text: (a) a propositional representation containing the individual ideas that are derived from each sentence and the relations among them and (b) a discourse model, a representation of the context or situation to which the text refers. We used a paradigm called “item priming in recognition” to examine how these representations are stored in the cerebral hemispheres. In Experiment 1, the priming paradigm was used in combination with a lateralized visual field (VF) procedure. We found evidence that readers' representations were structured according to propositional relations, but only in the left hemisphere. Item recognition was facilitated when a concept was preceded by another concept from the same proposition when targets were presented to the left, but not to the right, hemisphere. We found priming in both hemispheres, however, when targets were context-appropriate senses of ambiguous words or topics of passages. In Experiment 2, we replicated the priming effects in three callosotomy patients. We argue that the distinction between a propositional representation and a discourse model is important with respect to how discourse is represented in the brain. The propositional representation appears to reside in the left hemisphere, whereas aspects of the discourse model appear to be represented in both hemispheres.

The functional role of representations cannot explain basic implicit memory phenomena

The propositional account of explicit and implicit knowledge interprets cognitive differences between direct and indirect test performance as emerging from the elements in different hierarchical levels of the propositional representation that have been made explicit. The hierarchical nature of explicitness is challenged, however, on the basis of neuropsychological dissociations between direct and indirect tests of memory, as well as the stochastic independence that has been observed between these two types of tests. Furthermore, format specificity on indirect test of memory challenges the basic notion of a propositional theory of implicit and explicit knowledge.