Ԫ Computer Science and Information Science Seminars, University of Otago, New Zealand
University of Otago logo. Computer and Information Science Seminars

Seminar Homepage


Dr Alistair Knott, Department of Compouter Science


Language, vision and action


Archway 2


This talk is about the relationship between models of human language and models of human sensorimotor cognition. Language and sensorimotor systems have traditionally been studied quite separately in the cognitive sciences, but recently there has been some interest in looking for commonalities between the models developed in these two areas. The new interest is partly motivated from a desire to explore the interface between language and sensorimotor experience. (There obviously is such an interface, because we can talk about what we see, and we can carry out verbal instructions to perform actions.) It is also motivated from a renewed interest in how human language evolved. (On the evidence of our closest primate relatives, our ancestors had well-developed sensorimotor systems---these could conceivably have functioned as a preadaptive platform for the language faculty.) The more our theoretical model of natural language has in common with our model of sensorimotor cognition, the easier it is to tell a story about how language evolved from sensorimotor machinery, or about how modern humans convert sensorimotor representations into linguistic expressions.

The theory I will outline is that the syntactic structure of a sentence describing a particular state or event can be understood as an encoding of the sensorimotor processes which occur in an agent who directly witnesses it, by observing it or participating in it. I will illustrate the idea using a simple concrete transitive sentence: "The man grabbed a cup". I will first give a syntactic analysis for this sentence, within the tradition of Chomsky's Minimalist programme (Chomsky, 1997). I will then present some psychological findings about the sensorimotor processes which underlie the observation (or execution) of a grabbing action, focussing on findings about the time course of the various sub-processes which are assumed to be involved. There are some interesting correspondences between the syntactic model and the sensorimotor model; I will conclude by suggesting a way in which the syntactic structure of the sentence can be understood as a description or 'trace' of processing within the sensorimotor system.

Last modified: Tuesday, 17-May-2005 09:15:11 NZST

This page is maintained by Caroline Wills.