To appear in : Agency and self awareness : Issues in philosophy and psychology, J. Roessler and N. Eilan (Eds). Oxford, Oxford University Press.

Marc Jeannerod
Institut des Sciences Cognitives
67 Boulevard Pinel,
69675 Bron, France
jeannerod@isc.cnrs.fr
============================================================

Introduction.
The mutual relationships of action to consciousness are complex and far from unequivocal. We execute many of our daily actions unconsciously ; conversely, we can consciously simulate or imagine actions we dot not execute. This vast domain of research, motor cognition, is central to the study, not only of action itself (how it is planned, prepared and finally executed), but also of how action contributes to the representations we build from objects and from other selves. In this essay, I will use the broad term of representation to include the various internal states in relation to action. There could be better terms, with the advantage of greater precision, but with the disadvantage of loosing continuity between different levels of functioning. Indeed, the term representation of an action can be used in its strong sense, to designate a mental state in relation to goals and desires, as well as in its weak sense, to indicate the ensemble of mechanisms that precede execution of a movement. Finally, it can also be accepted by biologists to designate the state of the neural network during a mental state related to action.
 The various levels of representation of an action will be described with regard to the subjective experience felt by an agent during interactions with objects in space or with other individuals. Three main levels will be identified whether consciousness of objects, consciousness of goals or consciousness of the self are concerned.

Object oriented movements. The automatic level.
Most of our movements directed at objects are prepared and executed automatically. Once started, they are performed accurately and rapidly (in less than one second), which leaves only little time for feedback regulations. Models accounting for such characteristics have used the notion of optimization principles, that is, principles that are represented within the motor system and operate during execution itself. In the motor literature, such principles have been described in the kinematic domain (e.g., the minimum jerk model, Hogan and Flash, 1987), in the domain of biomechanics (optimized trajectories, Desmurget et al, 1997 ; endpoint comfort, Rosenbaum et al, 1990) and in the time domain (Hoff and Arbib, 1993). Besides optimization of execution, other features of object oriented movements strongly suggest that they are organized, or represented, prior to execution. A widely studied example is the action of grasping. Changes in finger position appropriate for a stable grasp occur during the reaching component that transport the hand at the object location, that is, far ahead of contact with the object. This process of grip formation has been epitomized by the parameter of maximum grip amplitude : At about 70% of movement time, the distance between fingers that compose the grip (e.g., thumb and index finger) reaches a maximum value which exceeds object size. The important point, however, is that maximum grip amplitude has been shown to be exactly correlated to object size and is therefore likely to reflect a representation of the object (see Jeannerod, 1986).
 The representation that could account for such anticipatory adjustments must therefore encode, not only the properties of the central and peripheral motor system used for optimization of the movement execution. It must also encode those properties of the object that are relevant to potential interactions with the agent, according to his intentions or needs: object's shape and size are relevant to grip formation (maximum grip size, number of fingers involved), its texture and estimated weight are relevant to anticipatory computation of grip and load forces, etc. In addition, the processing of object properties must take into account the location and orientation of the object with respect to the body. In other words, the representation of object properties interact with motor factors, such as the biomechanics of the arm, for defining the final pattern of the grasp. To demonstrate this point, Paulignan et al (1996) systematically studied the orientation of the opposition axis (the axis defined by the positions of the thumb and the index finger on the object surface) during grasping the same object placed at different positions in the working space. The object they used (an upright cylinder) was visually "simple" in that it afforded an infinite number of possible positions of the fingers on its surface. In this condition, the orientation of the opposition axis was only guided by the biomechanics of the arm. They found that the orientation of the opposition axis remained invariant with respect to the body axis of the subject, which is a clear indication that computation of the grasp by the visual system was effected within a body centered (egocentric) frame of reference (Figure 1a). In most everyday situations, however, the interaction between visual and motor factors is dominated by visual constraints. Indeed, if the shape of the object affords only few positions for the opposition axis, the arm biomechanics have to adapt to the object. As shown by Figure 1b, a slight change in the orientation of such an object may cause a major reconfiguration of the grasping movement directed at that object (Stelmach et al, 1994).

The anatomical theory of a duality of visual pathways.
The term of pragmatic representation has been proposed (see Jeannerod, 1994) for qualifying this mode of representing objects as goals for action. The most striking characteristic of pragmatic representation is its implicit functioning and correlatively, its unconscious nature. Although further qualification will be needed to better define the term pragmatic (see below), this mode of representation opposes another one, not directly related to action (the semantic representation) whereby the same objects can be processed for identification, naming, etc. This distinction stresses the fact that objects are multiply represented, according to the task in which they are involved. For the purpose of the present paper, which is to discuss the degree of consciousness attached to different forms of action, we have to concentrate on two controversial and related issues : i), whether pragmatic processing is associated to a specific neural system. and ii), whether the unconscious nature of pragmatic processing is a consequence of the structure of this system, or a consequence of constraints imposed by pragmatic processing itself.
 The first issue, which will be discussed only briefly here (see Jeannerod, 1997), relies on an anatomical theory for the processing of visual information. This theory, which was developed in the last twenty years or so, proposes a duality of cortico-cortical visual pathways (Ungerleider and Mishkin, 1982) : an occipito-parietal route (the dorsal visual pathway) specialized for visuospatial processing and, by extension, for processing object-oriented movements ; and an occipito-temporal route (the ventral pathway) for object recognition. Subjects with lesions located in specific areas of the parietal lobe exhibit a typical impairment in object oriented behavior with their contralesional arm. They misreach object location and are unable to form the proper grip : grip size is improperly calibrated, orientation of opposition axis is inappropriate, etc. (Jeannerod et al, 1994). This impairment, however, appears to be limited to processing those visual properties of an object that are relevant to interacting with it. Of course, in the same subjects, semantic identification of the objects is preserved. These observations stress the fact that pragmatic processing might take place in the parietal lobe (i.e., in the dorsal pathway of visual cortico-cortical connections), whereas semantic processing would take place somewhere else (indeed, there are examples of subjects with large lesions in the ventral part of the visual system, who fail to recognize objects, whereas they can correctly handle them ; Goodale et al, 1991). The fact that, in parietal patients, the grasping deficit is limited to the arm contralateral to the lesion supports the notion that pragmatic processing functions locally and, by way of consequence, should remain unconscious
This raises the second issue : are automatic movements unconscious because they pertain to a neural system (the dorsal pathway) that, by design, does not operate on the conscious mode ? This attractive possibility is not entirely supported by experimental data in normal subjects. In a recent study using PET Faillenot et al (1997) have compared the patterns of cortical activation during two different tasks, an action task (grasping objects of different sizes and shapes by hand), and a perceptual task (perceptually matching these objects with each other). In the first task, the main activation foci were in the motor areas and the inferior parietal lobule contralateral to the hand used for the task, but also in the right posterior part of the intraparietal sulcus. In the second task, two foci were found, one in the left inferotemporal cortex, and one in the right posterior parietal cortex : this latter focus clearly overlapped with the homologous focus activated during grasping. This result means that perceptual analysis, even when no action is to occur, uses resources that pertain to the dorsal pathway and are also used during object-oriented action. Following up this result, Faillenot et al (1999) tested further the degree of involvement of parietal cortex during perceptual discrimination. Subjects had to discriminate shapes which were presented with different degrees of slant in the frontal plane (a 2-D orientation task) or in the sagittal plane (a 3-D orientation task). These tasks, where no action was ever required, did produce activation in areas located in the posterior part of the intraparietal sulcus (the dorsal pathway), as well as at the occipito-temporal junction and in the inferior temporal gyrus (the ventral pathway).
These results in normal subjects illustrating perceptual functions of the parietal lobe are to be compared with the effects of lesions of the same areas. Patients with such lesions, besides their problems in visuomotor transformation already described, may also exhibit typical perceptual deficits. They may be unable to copy objects by drawing (the so-called constructive apraxia) ; they may have great difficulties recognizing on a photograph objects displayed in a non canonical orientation (Warrington and James, 1986). They cannot resolve, either motorically or perceptually, the 3-D orientation of objects or their spatial relations with respect to other objects. One could speculate that these perceptual difficulties represent one and the same deficit as the visuomotor ones : only those aspects of perception which relate to action would be affected by the lesion, whereas other aspects, those related to identification and semantic processing would be left intact. Obviously, this would require a radical redefinition of what has been called pragmatic processing, as it would now include representation of the same visual stimuli both in an allocentric and an egocentric reference frame, where that part of the processing coded egocentrically (the visuomotor transformation) would be devoid of conscious counterpart, whereas the perceptual part would be conscious.
Another interpretation for automaticity of object-oriented movements would be that these movements are unconscious because this is a prerequisite for accuracy. The main argument would be that the life span of the working memory used for a goal directed movement must not exceed the duration of the movement itself : if corrections have to be generated they must be effected on the basis of the goal of each movement, and the representation of that goal must be erased before another segment of the action starts. Experiments where the goal is modified during the movement support this view : if a target briskly changes its location during the ocular saccade that precedes a pointing movement toward that target, subjects usually remain unaware of the displacement (they see only one, stationary, target) ; yet, they correctly point at the final target location (e.g., Bridgeman et al, 1981). Goodale et al (1986) reported a pointing experiment where the target occasionally made jumps of several degrees, unnoticed by the subjects. They found that the subjects were nonetheless able to adjust the trajectory of their moving hand to the target position : Interestingly, no additional time was needed for producing the correction, and no secondary movement was observed, suggesting that the visual signals related to the target shift were used without delay for adjusting the trajectory. According to this view, generating a motor response to a stimulus and building a perceptual experience of that same stimulus would activate different mechanisms (Castiello et al, 1991).

A model for automatic control of movements.
The system for automatic control of movements has been described using a simple feedforward model. Roughly speaking, such a model includes a representation of the goal of the forthcoming movement (the desired state), which can be used as a reference for completion of the movement, and which generates output signals for executing it. During execution, the reafferent signals  (i.e. signals arising as a consequence of the movement itself) are checked against the desired state. Any mismatch resulting from this online comparison process is in itself a signal for immediate corrections. The reafferent signals used for the comparison are of several types. Visual signals arise from limb displacements and from concomitant displacements of visual objects ; proprioceptive signals arise from joint or skin receptors, etc. Similar models were invented about 50 years ago by physiologists (e.g., von Holst, 1954) for accounting for the distinction between self produced and externally produced changes in the external world. According to these authors, if perceived changes were correlated with self-generated output signals, they were registered as consequences of one's own action. If not, by contrast, they were registered as originating from an external source. The correlation between outgoing signals and the resultant incoming signals is thus an unambiguous feature of self-generated changes.
 The goal of the desired action must be kept in memory for a sufficient duration so as to allow the comparison to operate. As stressed above, if the comparison reveals a mismatch with respect to the desired state, an immediate correction may be generated. If, on the other hand, no mismatch occurs and the desired state is reached, the memory can be erased. The model offers the possibility that the comparison process could also take place ahead of execution itself : a representation of the desired state as it would occur if it were executed and of the reafferences that it would generate is built and matched with the initial internal model. If this comparison does not anticipate any mismatch, then the system would proceed to the next step (e.g., Wolpert et al, 1995). Such a view, based on engineering models, opens new possibilities for understanding the functions of motor representations.

Evidence for lack of awareness of automatic  actions.
These mechanisms are supposed to operate automatically, that is, they should remain outside subjective experience from the agent. This prediction was tested in an experiment specifically designed to investigate the degree of awareness to which subjects can access about the details of their own movements. In this experiment (Fourneret and Jeannerod, 1998), subjects were instructed to draw lines in the sagittal direction with a stylus on a digital tablet. The output of the stylus was shown to them on a computer screen seen in a mirror, itself placed so as to mask the subject’s hand (Figure 2a). On some trials, a bias was introduced in the output of the digital tablet, such that the line seen in the mirror appeared to deviate from the sagittal direction (to the right or to the left) and by a given angle. The subject therefore had to deviate his tracing in the opposite direction and by the same angle in order to fulfil the instruction of drawing in the sagittal direction. At the end of each trial, the subject indicated verbally (by selecting a line on a test card) in which direction he thought his hand had actually moved. The results were twofold : first, the subjects were consistently able to trace lines that appeared sagittal, that is, they accurately corrected for the bias. Second, they gave verbal responses indicating that they thought their hand had moved sagitally, hence ignoring the actual movements they had performed (Figure 2b). Thus, normal subjects appear to be unable to consciously monitor the signals generated by their own movements. Instead, the all set of mechanisms outlined above (e.g., desired state, memory storage, comparator, etc.) operate on an implicit mode, without conscious counterpart. Subjects tend to adhere to the goal, not to the way it has been achieved.
 A different situation arises in case of failure of the movement, that is, when the goal cannot be reached. In this case, according to the above speculations, because the mismatch between the desired state and the expected result is not corrected, the representation of the desired state would not be erased. It has been previously suggested, on the basis of introspective evidence, that in such a situation one should shift to a conscious strategy whereby one becomes aware of the goal of the movement and of the cause of its failure (Jeannerod, 1994). This suggestion seems to be supported by the results of an unpublished experiment of Slachevsky, Pillon, Fourneret, Pradat-Diehl, Jeannerod and Dubois, using the same design as above. Although Fourneret and Jeannerod (1998) had limited the amplitude of the bias to 10 degrees and randomized the trials with different amplitudes and directions of bias, Slachevsky and her colleagues used a bias in one direction only (e.g., right) with increasing amplitude from trial to trial, up to 42 degrees. At an average amplitude between 14 and 24 degrees, normal subjects became clearly aware of the bias and declared that the movement of their hand erred in a direction different from that seen on the screen. Interestingly, beyond this value of 24 degrees, their motor performance changed: although inaccuracy in compensating for the bias and variability kept increasing up to 24 degrees,  it suddenly improved beyond this value. This result tends to confirm the introspective evidence that a movement becomes conscious when it fails (e.g., Pacherie, 1997). When large discrepancies appear and corrections are beyond the capacities of the automatic level, another level of the action system is activated. This hypothesis will be further developed below.
 Only little is known about the neural basis of this mechanism. Slachevski and her colleagues examined further a group of patients with frontal lobe lesions. Unlike normal subjects, frontal patients did not shift from the automatic to the conscious mode when the bias increased. Instead, they kept using the same automatic strategy, making larger and larger, unnoticed errors. Indeed, patients with frontal lobe lesions exhibit typical impairments demonstrating their inability to consciously monitor their performance and to override their automatic responses. Some of these patients present with the so-called  “imitation behavior” whereby they compulsively reproduce actions performed by other people in front of them or compulsively use objects displayed in front of them (Lhermitte, 1983).

What is consciously represented in actions.
The next problem, therefore, is to identify the level at which the agent becomes aware of his goals, intentions and desires or, more simply, is able to produce a conscious judgement about his action. According to a widespread conception in psychology, the conscious level of cognitive activities should pertain to a unitary symbolic representation, common to many modalities (e.g., linguistic, perceptual, motor, etc). Requin (1992), for example, considered that, at the highest level, motor actions would be represented in a non-motoric, holistic and symbolic mode and could give rise to a conscious experience. This formulation departs from the concept of a motor representation used here, for several reasons. The first reason is that motor cognitive activity seems to be endowed with modular properties. Studies on motor memory show that it relies on mechanisms distinct from those of semantic memory, for example. This can be verified by comparing retention in subjects instructed to learn a list of action sentences (“open a book”) either after mere verbal presentation by the experimenter, or after pantomimed performance of each action by the subjects. It was found that performing the actions remarkably improves recall and recognition of these actions. This enactment  effect, however, allows for a good recall only of action-relevant information, not contextual information associated with the actions (e.g. in sentences like “smoke a pipe in the garden”). Conversely, if instructed to mentally visualize the action during the learning phase, subjects retain contextual information better than action-related information (see Engelkamp, 1995). This is clearly in contradiction with the idea that consciousness of action could be an attribute of a purely speculative “symbolic level”. The empirical evidence is more in favor of the idea that mental representations consist of modality-specific structures, than it is of a propositional-computational mode of functioning (e.g., Paivio, 1986).
The second set of reasons for rejecting a single holistic system for conscious representations is that it contradicts many data obtained with cognitive neuroscience methods, especially neuroimagery, showing that there is no discontinuity between the automatic level and the conscious level. Our working hypothesis here will be that the motor representation is a recursive structure, in the sense that goals which account for automatic execution of individual movements are embedded into a broader goal, which accounts for the unfolding of the whole action. The memory system in which this ultimate goal is stored for building a representation of the whole action must have characteristics different from those of the short term storage for performance of individual movements. The problem in this section, therefore, will be to determine at which level of integration of individual movements into the whole action these characteristics begin to change.
Before we can address this point, however, we have to introduce a more complete definition of the term action, a definition which did not appear necessary as long as only execution was considered. Action as it is understood here should not be limited to its overt appearance (the production of a set of observable movements). It also includes the covert aspect which corresponds to the internal representation of its goal. Indeed, we postulate that the two aspects, covert and overt, of an action are closely related with each other, such that they are parts of the same continuum. This postulate bears important logical consequences, namely that an overt action necessarily involves a covert counterpart, but that a covert action does not necessarily involve an overt counterpart. This asymmetry has raised one of the most critical issues in classical psychology, about how to infer the properties of the covert part from its behavioral counterpart. Although the measure of reaction times (the so-called mental chronometry) has long been the only available solution, the introduction in recent years of objective methods for measuring brain activity now provides a more direct access to purely internal states, even in the absence of any behavioral manifestation (see Jeannerod, 1999).
The problem at this point is to determine the content of this conscious level. It is not expected to be a mere “reading” of the content of the automatic level, where the optimization rules, the biomechanical constraints and the pragmatic properties of objects are represented. Indeed, as already shown, the content of the conscious level does not include the complete set of details of what has been actually performed and how it has been performed. Introspectively, the agent seems to have only access to the general context of the action, its ultimate goal, its consequences, and the possible alternatives to it. In order to examine this point, however, it is necessary to design experimental situations where the consciously available content of an action could be objectively tested. To this aim, we will focus on motor imagery, the conscious simulation of actions, a condition that has been proposed several years ago as a plausible model for studying aspects of motor representations, including their conscious content (Jeannerod, 1994). Somewhat paradoxically, as the following paragraphs will show, the subjective content of a mentally imagined action will appear to be limited, whereas its actual content will be found to extend far beyond what is consciously available. Two sets of experimental arguments will be used, those derived from chronometric analysis, and those obtained with neuroimaging techniques.

Chronometric analysis of motor images.
Mental chronometry relies on subjects responses, based on their subjective experience. Analyzing the pattern of responses allows making inferences about the implicit cues that subjects potentially use to give their responses. To take a typical example, consider the task of mentally simulating hitting alternatively two targets of a given size and placed at a given distance from each other (the Fitts' task). The rate of hitting is paced by a metronome, beating at an increasing frequency. The subject is instructed to warn the experimenter whenever he or she feels unable to keep hitting the targets as the metronome frequency increases. Sirigu et al (1996) found that the critical frequency at which subjects failed to hit the imaginary targets was the same as for hitting real targets. All that the subjects (consciously) know is that, when metronome frequency increases, they experience an increasing difficulty to perform the mental task of hitting the targets. What they do not know, is that their responses follow the same regularity as if they were actually hitting visual targets.
Another example of the existence of an implicit content in motor imagery is provided by a series of experiments based on conscious judgements of feasibility of an action. In an unpublished experiment of Frak, Paulignan and Jeannerod, subjects were shown a glass of water from above with an indication of where the thumb and the index finger should contact it. Subjects, without performing the action, had to judge (by pressing on different keys) whether the action of raising the glass and pouring the water into another container would be easy, difficult, or impossible for each set of finger positions. Their pattern of responses followed the limitations that the geometry of the upper limb would have imposed on real motor performance, which implies, although they received no instruction to do so, that they unconsciously simulated the movement before giving the response. Their response times (within 1500-2000 ms) increased with the estimated difficulty of the task. This result is in line with other observations showing that recognition of the handedness of a visually presented hand depends on similar covert sensorimotor processes (Parsons, 1994). As a rule, these experiments tend to converge on a common finding, namely that the time to give the response reflects the degree of mental rotation needed to bring one’s hand in a position adequate for achieving the task. This implicit process is the motor counterpart of the classical mental rotations or displacements used for giving responses about visual objects (Shepard and Metzler, 1971). However, unlike 3-D shapes, which can be rotated at the same rate in any direction, rotation of one's hand is limited by biomechanics of the arm joints. According to Parsons et al (1995), response times thus reflect biomechanically compatible trajectories, at the same rate as for executed movements. Similar effects on response times have been observed for making judgements on how to use hand held objects or tools. Our implicit knowledge about actions influences the way we cognitively process the visual world. In other words low level constraints, which normally pertain to the level of automatic execution, are also represented at the level of simulated actions, but they remain implicit.

Functional anatomy of motor representations.
A second set of information about the organization of motor representations can be drawn from neuroimaging experiments. Studying the pattern of brain activity during the process of generating an action, either limited to its covert part as in intending or mentally simulating, for example, or also including overt motor performance, reveals that activated areas partly overlap during different modalities of representation. During mental simulation of movement of the right hand, activity increases in several areas directly concerned with motor behavior. At the cortical level, the primary motor area, area 6 in the inferior part of the frontal gyrus and area 40 in the inferior parietal lobule are activated on the left side. Subcortically, the caudate nucleus is activated on both sides and the cerebellum on the left side only. Another focus of activity is observed in left prefrontal areas, extending to the dorsolateral frontal cortex (areas 9 and 46) (Decety et al, 1994). Finally, the anterior cingulate cortex (areas 24 and 32) is bilaterally activated, as is SMA (Stephan et al., 1995). Besides mental simulation, there are other modalities of consciously represented actions, like intentionally selecting a motor pattern among several possible alternatives. Brain activation in this condition also involves the left dorsolateral prefrontal cortex, the anterior cingulate region (Frith et al, 1991), as well as premotor and parietal cortices (Spence et al, 1997). In addition, a similar pattern of activation can be found in the case of completely implicit motor representations, like those generated by observing tools (Perani et al, 1995) or even hearing action verbs.
 The implicitness of covert actions like mentally simulated ones opens interesting perspectives on other modalities of motor cognition. Indeed, another form of implicit mental operation has been recently added to the realm of motor imagery: this is the cognitive process related to recognizing and understanding actions observed from other individuals. A series of PET experiments were performed for exploring brain activity during observation of actions performed by others. In the Grafton et al.'s (1996) experiment, subjects were instructed to carefully observe simple meaningful actions (prehension movements performed by the experimenter's right hand). Brain activation in this condition was compared with a control condition where the subjects merely looked at visual objects. The activated areas were mostly located in the frontal lobe, where the SMA and the lateral area 6 in the precentral gyrus were involved, as well as area 45 in the inferior frontal gyrus, and in the parietal lobe, where area 40 was involved. The Grafton et al's study, however, did not allow making a clear distinction between different modalities of acquiring information related to the observed action. It is likely that the subject's cognitive strategy during observation influences the distribution of cerebral activation, whether the observed action has an overt content, or can only be described as a spatiotemporal pattern, or is a motoric pattern to be replicated later on. In order to evaluate this possibility, Decety et al. (1997) manipulated the cognitive strategy of the subjects by giving them specific instructions during the observation and by presenting them with different types of actions. When the subjects had received the instruction to memorize the actions with the purpose of later imitation, the SMA and the ventral premotor cortex were activated. A bilateral involvement of dorsolateral prefrontal cortex was also found in this condition, in agreement with previous studies concerning the planning of voluntary actions (Frith et al., 1991) and the mental simulation of actions. When the instruction was to observe the actions with the purpose of recognising them, by contrast, only the parahippocampal gyrus in the temporal lobe was activated.
 This pattern of activation supports the idea that actions performed by others can be understood by an observer to the extent that they can be “simulated” by that observer (the so-called simulation theory, see Gallese and Goldman,  1998). This strategy of putting oneself in the shoes of the agent would represent the basis for a broad spectrum of cognitive functions, starting from understanding others' actions, up to learning by observation and imitation. The simulation theory is in line with many of the results described earlier in this paper, like, for example, the implicit simulation of action during handedness recognition. Thus, observing an action would activate within the observer’s brain the same mechanisms that would be activated were that action intended or imagined by that observer. In turn, this implicit representation in the brain of how movements are generated influences interpretation of actions observed from other people (see Jeannerod, 1999).
A hypothesis accounting for how a representation of an action performed by someone else could be built has been put forward by the Rizzolatti group, based on monkey experiments. In these animals, a population of neurons located in the premotor cortex can be activated both when the animal performs a certain type of movements and when it observes that same movement performed by a conspecific (mirror neurons, Rizzolatti et al, 1996). According to the above simulation theory, these neurons might fulfil the role of a representational system encoding observed actions within a format compatible with their execution by the observer. A number of behavioral phenomena could be explained in this way. Among those are motor facilitation phenomena, whereby an action can become contagious among a group of people (yawning and laughing are typical examples). Whether this rather primitive mechanism would also hold for extracting the cognitive significance of an observed action is an open possibility.
The main conclusion to be drawn from this section is that motor cognition relies on cortical patterns of activity specific for each function (e.g., mentally simulating, intending, understanding) but partly overlapping between functions. Motor representations, although they can be built or activated by several different ways (e.g., mental simulation of an action originates from a certain type of motor memory ; action understanding relies on a special type of perception for biological motion, etc) share common mechanisms. The automatic level and the “ conscious ” level are not independent from each other: they are different effects of a common process. As stressed by Jeannerod (1994) and Rizzolatti (1994), there is a complete functional equivalence between motor imagery and the events leading nonconsciously to action: conscious and nonconscious motor images are the basic elements of which motor actions are constructed.

Attribution of an action to its author. The agency level
This section will emphasize a paradox in motor cognition. Whereas subjects execute actions, the content of which they remain essentially unaware, they seem to have no problem in correctly attributing these actions to themselves or to an external agent. Besides having explicit, albeit limited, knowledge of the content of their own actions, normal individuals are good at judging whether an action originated from them or not. Although such a judgement may often be trivial, this is not always the case. Not to mention pathological conditions (see below), there are in everyday life ambiguous situations which obviously require a subtle mechanism for signaling the origin of an action. Among those are situations created by interactions between two or more individuals (e.g., joint attention, matched actions, or mutual imitation, for example). There are also situations, usually pertaining to the domain of man-machine interactions, where the cues for an agency judgement about an action can be degraded (e.g., telemanipulation, virtual reality systems, etc).
Schematically, a basic distinction can be made between situations involving actions observed from other individuals and situations involving self-generated actions (executed or not). In the first case, the incoming information is exclusively based on visual perception. It has been shown that visual information arising from “ biological ” movements (e.g., those produced by living individuals as opposed to mechanical devices) contain kinematic features that render them “intentional” and make them easily recognizable, even by very young infants (Dasser et al, 1989 , Johansson, 1977). This visual information, according to the mirror neuron concept, would ultimately activate motor neurons, with the consequence that the observed action would be understood as a potential action of the self. Now consider the second case, i.e., when the action is imagined or intended. Here, the available information is entirely internal to the subject, based on activation of those structures that will represent a desired state. The processing of this self-generated information also ultimately reaches the motor areas : as stressed earlier, the pattern of cortical activation in the two cases resemble each other and may create confusion. It has to be assumed, however, that the creation of a desired state, although it may activate neurons which are also activated during observation of action (e.g., the mirror neurons) will activate many other populations which will not participate in the situation of mere observation. Finally, it is only when the self-generated action comes to execution that the distinction from an observed action becomes unambiguous. A set of signals, which are very likely to be the same as those described in the first section, become involved, with the possibility of comparing the self-generated signals (which are transformed into command signals) with coordinated reafferent signals, visual or proprioceptive. This comparison, in principle, should not be possible during action observation, where no reafferent signals should be produced. However, pathological conditions such as compulsory imitation in frontal patients, where the observed action is readily transformed into execution (Lhermitte, 1983), clearly represent an exception to this rule. This behavior suggests that an observer monitoring an action performed by someone else is never far from being also the agent of that action.

An experimental study of agency.
Let us consider here the situation where an action is executed, but its origin is rendered ambiguous. The question will be to determine how an agent can disentangle this ambiguity to produce an agency judgement. To this aim, an experimental situation was designed by Daprati et al (1997). In this experiment, the subject’s hand and the experimenter’s hand were filmed by two different cameras. By changing the position of a switch, one or the other hand could be briefly displayed on the video screen seen by the subject. The two hands looked alike as they were covered with a similar glove. In each trial, both the experimenter and the subject had to perform a given hand movement on command (e.g., stretch thumb, stretch fingers 1 and 2, etc) : on some trials, however, the experimenter’s movement departed from the instruction. As a result of this experimental arrangement, the subject was randomly shown either his own hand, or the experimenter’s hand performing the same movement as him, or a different movement. At the end of each trial, a verbal agency judgement was recorded : the subject had to say whether the hand he had seen was his hand or another hand. Normal subjects were able to unambiguously determine whether the moving hand seen on the screen was theirs or not, in the two "easy" conditions : First, when they saw their own hand, they correctly attributed the movement to themselves ; second, when they saw the experimenter's hand performing a movement which departed from the instruction they had received, they correctly denied seeing their own hand. By contrast, their performance degraded in the "difficult" trials where they saw the experimenter's hand performing the same movement as required by the instruction : in this condition, they misjudged the alien hand as theirs in about 30% of cases (Figure 3a).
 One possibility is that the subjects in this experiment based their judgement on internal signals arising from their own action, and on monitoring the degree of correlation between these internal signals and other types of signals (e.g., visual). However, if one considers that these action-related signals are not consciously monitorable, the problem remains to understand how they can influence the state of the conscious level, where the agency judgement is generated. Another possibility is that subjects built their own representation of the action from available information and that conscious distinction between a representation corresponding to a self-produced action and an action produced by another agent relied on the pattern of cortical activation generated for each of these modalities. Internal, self-generated, signals contributed to the correct agency judgement by reinforcing those representations pertaining to a self-produced action (see Jeannerod, 1999).

Pathological dysfunction of agency.
One possibility for substantiating the above hypothesis is to examine pathological conditions which affect consciousness of action and self-consciousness. One class of symptoms displayed by schizophrenic patients seem to be closely related to dysfunction of the above mechanisms. These symptoms include insertion of thought, auditory-verbal hallucinations, delusion of reference, delusion of alien control. Those are false beliefs which lead to a feeling of depersonalisation by impairing the distinction between the self and the external world (e.g., Schneider, 1959). The impairment of such patients in correctly attributing an action to the proper agent is demonstrated by the frequent occurrence of auditory verbal hallucinations. Early results of neuroimaging studies in hallucinating schizophrenics, showing activation of Broca's area, had first suggested that the patients were in fact misinterpreting their own inner speech (e.g., David, 1994, Silbersweig et al, 1995). More recent results rather suggest that hallucinations might rely on activation of primary auditory cortical areas. A more reasonable approach, however, is to determine first the pattern of cortical activity in normal people in different situations like hearing one's own voice, producing inner speech, or imagining someone's voice, and to compare the results with those of hallucinating patients. Available data suggest that, in normal subjects, the temporal area is more activated during hearing the voice of someone else than during hearing one's own voice. In addition, cingular and frontal areas are activated only during hearing the stranger's voice (McGuire et al, 1996). It is thus possible that this mechanism might not operate correctly in schizophrenics.
Spence et al (1997) also examined the same problem during generation of spontaneous arm movements. Cortical activity was monitored with PET in schizophrenic patients with experience of delusional control. During the scan, the patients were required to voluntarily move a joystick and to freely select the direction of the movement. Most of them reported vivid experiences of alien control when performing the motor task. Brain activation was found to be increased in a cortical network including the left premotor cortex and the right inferior parietal lobule and angular gyrus, at the level of areas 40 and 39. This right parietal hyperactivity in deluded subjects is particularly interesting : it is noteworthy that lesions at this level frequently result in altered awareness (neglect) for the contralateral limbs and space, and denial of the disease (anosognosia) ; conversely, transient hyperactivity (during epileptic fits for example) may produce the impression of an alien phantom limb (see Spence et al, 1997). This result, together with  those obtained for verbal hallucinations might be interpreted as a deficit in cortico-cortical inhibition (possibly of prefrontal origin, Frith, 1996), normally suppressing activity in critical areas during self-produced action. Increased activity in these regions would therefore lead to incorrect agency judgements with a tendency to misattribute actions to an external agent.
The pattern of misattributions in schizophrenic patients is twofold. First, hallucinating schizophrenic patients may show a tendency to incorporate external events in their own experience, or to interpret environmental cues as specifically directed to themselves. Accordingly, they may misattribute their own intentions or actions to external agents. During auditory hallucinations, the patient will hear voices that are typically experienced as coming from a powerful entity trying to monitor and control his/her own behavior. In cases of delusion of alien control, the patient may declare that he or she is being acted upon by an alien force, as if his/her thoughts or acts were controlled by an external agent. The reverse pattern of misattribution can also be observed. In this case, patients are convinced that their intentions or actions can affect external events, for example, that they can influence the thought and the actions of other people. As a consequence, they tend to misattribute the occurrence of external events to themselves. Daprati et al (1997), using the same paradigm as above in groups of schizophrenic patients, found a dramatic increase in the rate of incorrect responses in the "difficult" trials. The error rate was 80% in a group of schizophrenics with delusional experiences, whereas in a non-hallucinating group, it was only 50% (Figure 3b). The fact that all patients gave nearly correct responses in the other two conditions (the error rate remained within 1%-7%) excludes the possibility that the effect observed in the “difficult” trials could be due to factors unrelated to the task, such as lack of attention. In this experiment, schizophrenic patients thus tended to overattribute to themselves actions produced by others. This behavior might correspond to a dysfunction of a comparison process (as already suggested by Feinberg, 1978 and Gray et al, 1991), similar to that described for the control of automatic actions, such that the effects of actions of others would be interpreted through the intentions of the self. The consequence of this misinterpretation would be that external events are seen as the result expected from one’s own actions. This type of errors by overattribution is an exaggeration of what is observed in normal subjects who may also attribute to themselves actions performed by others when they are presented in ambiguous conditions.

Conclusion.
Several levels of representation of actions have been identified. The automatic level essentially corresponds to nonconscious interaction with objects in the external world. It relies on pragmatic processing of these objects, in order to extract those properties (extrinsic and intrinsic) that are relevant to action. Brain areas responsible for visuomotor transformation, like parietal areas surrounding the intraparietal sulcus, and premotor areas, are likely to be involved in this mechanism.
The content of motor representations may become conscious under a variety of conditions. One condition is the failure of the automatic level. Other examples are mental simulation of an action, where the representation is voluntarily activated from within, or intentionally selecting an action from several possible alternatives. Brain areas located in the left dorsolateral prefrontal cortex and in the anterior cingular region might be critical for this function, as suggested by neuroimaging and also by behavioral observations in patients with frontal lobe lesions : these patients consistently cannot shift from the non-conscious to the conscious mode in case of failure of the automatic level. Whatever the way it is accessed, however, the conscious content remains remarkably limited.
 Finally, attribution of an action to its proper agent represents the ultimate aspect of consciousness of action. A possible hypothesis for the neural mechanism of agency is that internal self-generated signals would exert an inhibitory influence on those cortical sites which are responsible for analyzing the effects of an action. Lack of inhibition and correlative hyperactivity in those sites (due to disruption of these self-generated signals, for example) would automatically refer the origin of the action to an external agent. Conversely, lack of disinhibition would lead to overattribution to the self. The specific impairment of this network in schizophrenia would explain the variety of agency problems in these patients. This could represent the core of the disease, if one thinks that the ability to produce agency judgements is one of the foundations of self-consciousness and, by extension, represents the first step for communication between individuals and social interactions. Those questions of how can the self become aware of its own productions, how does it distinguish itself from other selves, in other words, how can the "Who ?" of an action be determined, are critical questions inherent to the social nature of human beings (Georgieff and Jeannerod, 1999).
 
 
 

References

Bridgeman, B., Kirch, M., & Sperling, A. (1981). Segregation of cognitive and motor aspect of visual function using induced motion. Perception and Psychophysics, 29, 336-342.

 Castiello, U., Paulignan, Y. & Jeannerod, M. (1991) Temporal dissociation of motor responses and subjective awareness. A study in normal subjects. Brain, 114: 2639-2655..

Daprati, E., Franck, N., Georgieff, N., Proust, J. Pacherie, E., Dalery, J. & Jeannerod, M. (1997) Looking for the agent. An investigation into consciousness of action and self-consciousness in schizophrenic patients. Cognition, 65, 71-86.

  Dasser, V., Ulbaek, I., & Premack, D. (1989). The perception of intention. Science, 243, 365-367.

  David, A.S. (1994). The neuropsychological origin of auditory hallucinations. In  David AS & Cutting JC (Eds) : The neuropsychology of schizophrenia. Lawrence Erlbaum, Hove, pp. 269-313.

Decety, J., Perani, D., Jeannerod, M., Bettinardi, V., Tadary, B., Woods, R., Mazziotta, J. C., & Fazio, F. (1994). Mapping motor representations with PET. Nature, 371, 600-602.

Decety, J., Grezes, J., Costes, N., Perani, D., Jeannerod, M., Procyk, E., Grassi, F. & Fazio, F. (1997). Brain activity during observation of action. Influence of action content and subject's strategy. Brain, 120, 1763-1777.

Desmurget, M., Jordan, M.A., Prablanc, C. & Jeannerod, M. (1997) Constrained and unconstrained movements involve different control strategies. Journal of Neurophysiology, XX: 1644-1650.

Engelkamp, J. (1995) Visual imagery and enactment of actions in memory. British Journal of Psychology, 86: 227-240.

Faillenot, I., Toni, I., Decety, J., Gregoire, M. C. & Jeannerod, M. (1997). Visual pathways for object-oriented action and object recognition: Functional anatomy with PET. Cerebral Cortex, 7,  77-85.

Faillenot, I., Decety, J. & Jeannerod, M. (1999) Human brain activity related to the perception of spatial features of objects. Neuroimage, 10: 114-124.

Feinberg, I. (1978) Efference copy and corollary discharge. Implications for thinking and its disorders. Schizophrenia Bulletin, 4, 636-640.

Fourneret, P. & Jeannerod, M. (1998) Limited conscious monitoring of motor performance in normal subjects. Neuropsychologia, 36: 1133-1140

Frith, C. (1996) The role of the prefronal cortex in self-consciousness : the case of auditory hallucinations. Philosophical Transactions of the Royal Society, London B, 351, 1505-1512.

Frith, C. D., Friston, K. Liddle, P. F. & Frackowiak, R. S. J. (1991). Willed action and the prefrontal cortex in man: A study with PET.  Proceedings of the Royal Society, 244,  241-246.

Gallese, V. & Goldman, A. (1998) Mirror neurons and the simulation theory of mind-reading. Trends in Cognitive Science, 2: 493-501

Georgieff, N. & Jeannerod, M. (1998) Beyond consciousness of external reality. A "Who" system for consciousness of action and self-consciousness. Consciousness and Cognition, 7: 465-477.

 Goodale, M.A., Pélisson, D. & Prablanc, C. (1986) Large adjustments in visually guided reaching do not depend on vision of the hand or perception of target displacement. Nature, 320:748-750.

 Goodale, M.A., Milner, A.D., Jakobson, L.S. & Carey, D.P. (1991) Perceiving the world and grasping it. A neurological dissociation. Nature, 349:154-156.

 Grafton, S. T., Arbib, M. A., Fadiga, L. & Rizzolatti, G. (1996). Localization of grasp representations in humans by positron emission tomography. 2. Observation compared with imagination. Experimental Brain Research, 112, 103-111.

 Gray, J.A., Feldon, J.N., Rawlins, J.N.P., Hemsley, D.R., & Smith, A.D. (1991) The neuropsychology of schizophrenia. Behavioral and Brain Sciences, 14, 1-84.

 Hoff, B.  & Arbib, M.A. (1993) Models of trajectory formation and temporal interaction of reach and grasp, Journal of Motor Behavior, 25:175-192.

 Hogan, N. & Flash, T. (1987) Moving gracefully: quantitative theories of motor coordination. Trends in Neuroscience, 10: 170-174.

 Holst, E. von (1954) Relations between the central nervous system and the peripheral organs. British Journal of Animal Behavior 2: 89-94.

 Jeannerod, M. (1986) The neural and behavioural organisation of goal-directed movements. Oxford, Oxford University Press.

 Jeannerod, M. (1994). The representing brain. Neural correlates of motor intention and imagery. Behavioral and Brain Sciences, 17, 187-245.

 Jeannerod, M. (1997). The cognitive neuroscience of action. Oxford, Blackwell.

 Jeannerod, M. (1999). To act or not to act. Perspectives on the representation of actions. Quarterly Journal of Experimental Psychology, 52A: 1-29.

 Jeannerod, M., Decety, J., & Michel, F. (1994). Impairment of grasping movements following a bilateral posterior parietal lesion.  Neuropsychologia, 32: 369-380.

Johansson, G. (1977). Studies on visual perception of locomotion. Perception, 6, 365-376.

 Lhermitte, F. (1983) “Utilisation behaviour” and its relation to lesions of the frontal lobes. Brain,106, 237-255.

 McGuire, P. K., Silbersweig, D. A., Murray, R. M., David, A. S., Frackowiak, R. S. J. & Frith, C. D. (1996). Functional anatomy of inner speech and auditory verbal imagery. Psychological Medicine, 26, 29-38.

Pacherie, E. (1997) Motor images, self-consciousness and autism. In Autism as an executive disorder, J. Russell (Ed). Oxford, Oxford University Press, pp 215-255.

Paivio, A. (1986) Mental representations. A dual coding approach. Clarendon Press, Oxford.

 Parsons, L. M. (1994). Temporal and kinematic properties of motor behavior reflected in mentally simulated action. Journal of Experimental Psychology. Human Perception and Performance, 20, 709-730.

 Parsons, L. M., Fox, P. T., Downs, J. H., Glass, T., Hirsch, T. B., Martin, C. C., Jerabek, P. A. & Lancaster, J. L. (1995). Use of implicit motor imagery for visual shape discrimination as revealed by PET. Nature, 375, 54-58.

 Paulignan, Y., Frak, V.G., Toni, I. & Jeannerod, M. (1996) Influence of object position and size on human prehension movements. Experimental Brain Research, 114: 226-234.

 Perani, D., Cappa, S. F., Bettinardi, V., Bressi, S., Gorno-Tempini, M., Matarrese, M. & Fazio, F. (1995). Different neural systems for the recognition of animals and man-made tools. NeuroReport, 6, 1637-1641.

 Requin, J. (1992) From action representation to motor control. In Tutorials in motot behavior, G.E. Stelmach and J. Requin (Eds). Amsterdam, Elsevier.

 Rizzolatti, G. (1994) Nonconscious motor images. Behavioral and Brain Sciences, 17 : 220.

 Rizzolatti, G., Fadiga, L., Gallese, V. & Fogassi, L. (1996). Premotor cortex and the recognition of motor actions. Cognitive Brain Research, 3, 131-141.

 Rosenbaum, D.A., Marchak, F., Barnes, H.J., Vaughan, J., Slotta, J.D. & Jorgensen, M.J. (1990) Constraints for action selection. Overhand versus underhand grips. In: Motor representation and control, Attention and Performance XIII, M. Jeannerod, Ed, Erlbaum, Hillsdale pp. 321-342.

 Schneider, K. (1959) Clinical psychopathology. New-York, Grunne and Stratton.
 Shepard, R.N. & Metzler, J. (1971)Mental rotation of three dimensional objects. Science, 171 : 701-703.

 Silbersweig, D.A., Stern, E., Frith, C.D., Cahill, C., Holmes, A., Grootoonk, S., Seeward, J., McKenna, P., Chua, S.E., Schnoor, L., Jones, T. & Frackowiak, R.S.J. (1995) A functional neuroanatomy of hallucinations in schizophrenia. Nature, 378, 176-179.

 Sirigu, A., Duhamel, J.-R., Cohen, L., Pillon, B., Dubois, B. & Agid, Y. (1996) The mental representation of hand movements after parietal cortex damage. Science, 273, 1564-156.

 Spence, S.A., Brooks, D.J., Hirsch, S.R., Liddle, P.F., Meehan, J. & Grasby, P.M. (1997) A PET study of voluntary movement in schizophrenic patients, Brain

 Stelmach, G.E., Castiello, U. & Jeannerod, M. (1994) Orienting the finger opposition space during prehension movements. Journal of Motor Behavior, 26: 178-186.

 Stephan, K. M., Fink, G. R., Passingham, R. E., Silbersweig, D., Ceballos-Baumann, A. O., Frith, C. D. & Frackowiak, R. S. J. (1995). Functional anatomy of the mental representation of upper extremity movements in healthy subjects. Journal of Neurophysiology, 73, 373-386.

 Ungerleider, L. & Mishkin, M. (1982) Two cortical visual systems. In: Analysis of visual behavior, D.J. Ingle, M.A. Goodale and R.J.W. Mansfield (Eds), MIT Press, Cambridge, pp. 549-586.

Warrington, E.K. & James, M. (1986) Visual object recognition in patients with right hemisphere lesions. Axis or features. Perception, 15, 355-366.

 Wolpert, D. M., Ghahramani, Z. & Jordan, M. I. (1995). An internal model for sensorimotor integration. Science, 269, 1880-1882.
 

Figure Legends

Figure 1

Orientation of opposition axis during grasping movements.
The diagram on the left illustrates averaged XY trajectories of the hand grasping the same object (a 6cm diameter upright cylinder) placed at different locations (from 40° right to 10° left with respect to body axis). I, T, W, trajectories of Index finger, thumb and wrist, respectively. Note change in orientation of the axis between final position of index finger and thumb (opposition axis), as a function of object position..
Curves on the left are plots of orientation of opposition axis as a function of object position computed within two different frames of reference. Upper right : object centered frame of reference. The angle of the opposition axis with respect to the reference axis changes for each position of the object. Lower right : head-centered frame of reference : the angle of the opposition axis remains more or less invariant. In each diagram, the three curves refer to cylinders of different diameters (circles, 3cm, squares, 6cm, diamond, 9cm). (From Paulignan et al, 1997).
Photographs on the lower part illustrate critical changes of hand configuration when varying the orientation of an object (Data from Stelmach et al, 1994).

Figure 2.

Poor conscious monitoring of components of one’s own actions.
An experiment using the apparatus displayed on the upper left part of the figure was used to assess subjective awareness of actions. Subjects drew on a graphic tablet straight lines in the sagittal direction between two targets appearing on a computer seen in a mirror. Subjects saw the line tracing appearing in the mirror as a red line. As shown in the right upper part of the figure, a bias could be introduced (dashed line) such that the subjects, in order to keep a straight line, had to orient their movement in the opposite direction. At the end of the trial, subjects were asked to indicate verbally on a chart (lower right) in which direction they thought their hand had moved during the trial. Results appear on the diagram on the left where the verbal responses were plotted as a function of direction and angle of the bias (-10° to –2°, to the left, 10° to 2°, to the right). As the figure shows, subjects consistently indicated a direction corresponding to about 2°, even when their movements erred by a larger angle. Black crosses indicate the correct responses if subjects had perceived the direction of their movements. From Fourneret and Jeannerod, 1998.

Figure 3.

Misattribution of action in schizophrenic patients.
Normal and schizophrenic subjects were shown for five seconds their hand or an alien hand in a mirror (insert upper left). The hand seen in the mirror could be that of the subject (subject), or that of an experimenter performing an action different from that of the subject (exp. Different), or the same as that of the subject (exp. Same). After each trial, subjects had to respond whether the hand they saw was theirs or not. In the most difficult situation (exp. Same), normal controls misattributed the alien hand as theirs in about 25% of cases. This proportion raised up to more than 80% in schizophrenic patients with delusion of control. Non deluded patients were significantly better than the deluded ones. From Daprati et al, 1998.