Abstract
The single-cell recording technique has long been used in attempts to dissociate two regions of the temporal cortex involved in face recognition and facial expression processing. As a classical example, the results obtained by Hasselmo et al. (1989) are frequently quoted in support of this functional dissociation. However a critical analysis of these data and a review of behavioral, neurophysiological, and neuropsychological findings on this topic show that we cannot purely and simply accept the conclusion that the superior temporal sulcus is associated with expression processing, and the inferior temporal cortex, with face recognition.
 
 

A very popular research strategy in cognitive science consists of trying to show that a given cognitive process can be precisely mapped to a localized brain activity. But doing so poses some tricky methodological problems that are often kept in the dark. We would like to discuss one such problem in the field of face recognition. A large body of behavioral data has been brought to bear in attempts to validate the hypothesis that the cognitive mechanisms responsible for face recognition are separate from the ones underlying the processing of emotional expressions (Young et al., 1986; Haxby et al., 2000). An obvious argument that would now be considered decisive in this matter would be a demonstration showing that these two types of processing are performed by distinct brain structures. This was precisely the conclusion drawn by Hasselmo et al. (1989), who used a single-neuron recording technique to study the temporal cortex of the macaque monkey. They observed what they deemed to be a substantial proportion of neurons that responded selectively to either the identity or the expression of photographed faces of congeners. The superior temporal sulcus (STS) appeared to be mainly involved in the processing of facial expressions, and the inferior temporal cortex (ITC), in the processing of identity. The results of this study have been widely quoted in the field, even very recently (Haxby et al., 2000; Allison et al., 2000).

In the Hasselmo et al. (1989) investigation, 25 neurons were shown to respond either specifically to a face's expression, or specifically to its identity: 12 in the STS and 13 in the ITC. Among the 12 STS neurons, 10 (83%) responded to expression and 2 to identity (17%); among the 13 ITC neurons, all 13 (100%) responded to identity and none (0%) responded to expression. Obviously, the probability of obtaining such a distribution by chance is extremely low (?2 (1) = 14.876, p = .0001). Based on these figures, one would feel very confident about the conclusion that there is a functional dissociation between the STS and the ITC. But such a conclusion is premature at best, illusory at worst.
The problem is that Hasselmo et al. (1989) did not analyze the responses of only 25 neurons. They in fact recorded the activity of a total of 45 neurons, 21 in the STS and 24 in the ITC. Out of the 21 STS neurons, 10 responded to facial expression only, 2 responded to identity only, 1 to both expression and identity, 2 to the interaction between expression and identity, and 6 to neither. Out of the 24 neurons recorded in the ITC, none responded to expression, 13 responded to identity, 3 to both expression and identity, 0 to the interaction between expression and identity and 8 to neither. In other words, for the STS, 10 neurons (48%) exhibited an expression-specific response and 11 (52%) exhibited a non-expression-specific response. This distribution is not significantly different from chance (?2 (1)= 0.000) and clearly does not authorize the conclusion that STS neurons are specialized in the processing of expression alone. The same holds true for the ITC, where 13 neurons (54%) responded specifically to identity while 11 (46%) did not (?2 (1)= 0.000). On these empirical grounds, then, one cannot accept the hypothesized difference in the specificity of these two regions, nor the resulting functional dissociation between the STS and the ITC.

Of course, one objection might be that our calculations include numerous neurons that responded to neither expression nor identity. But in order to legitimately eliminate them from the analysis, one would have to justify the statistical validity of such a sampling: 45 neurons recorded ... out of how many in this part of the temporal cortex? Although a precise answer to this question cannot be given, anyone would agree that n=45 is a very small sample size and that its statistical representativeness relative to the whole parent population of neurons is indeed questionable. If, in spite of this, we decide to discard from the analysis the 14 neurons that responded to neither expression nor identity (that is, 29% of the total!), the STS distribution is still not statistically different from chance (?2 (1)= 1.067) and the ITC distribution is only marginally so (?2(1) = 5.063, p = .05).

This case study illustrates the risk of methodological error and theoretical over-interpretation of conclusions drawn from data obtained using the single-cell recording technique. In this respect, Hasselmo et al.'s experiment is highly representative of the practices in this field, so our re-analysis could have been applied to other experimental data (see for example: Leinonen and Nyman,1979 ; Perrett et al., 1987, 1992 ; Wang et al., 1998). Although the number of neurons studied is always very small, that is not our main concern. What we are objecting to is the lack of a random sampling of cells and the selection bias that necessarily results from this technique. If one wishes to attribute a property observed in a limited sample of neurons to the entire parent population of neurons in the brain region from which the sample was drawn, the following two conditions must be rigorously satisfied: (a) the sample size must be large enough (statistical power), and (b) the neurons must be chosen at random, i.e., without a priori or a posteriori selection (random sample). We are concerned here with the second condition. Indeed, the 45 neurons studied by Hasselmo et al. only represent 11% of the temporal cortex neurons they tested that responded significantly more to faces than to other objects (Hasselmo et al., p. 207). One could even contend, then, that this particular set of neurons does not respond preferentially to faces and thus, that its face-specificity is not evident (for a convergent critique, see Desimone, 1991; Farah et al., 1998; Gauthier and Logothétis, 2000; Gauthier et al., 2000). What is more, these 45 neurons were recorded in two different monkeys (20 neurons in monkey QQ and 25 in monkey NN). The merging of the two separate sets of neurons into the same analysis in fact amounts to creating a single "fictitious" brain. Strictly speaking, the conclusion sought by Hasselmo et al. should be reached individually for NN and for QQ, which the authors did not do (Hasselmo et al., 1989, p. 213).

Clearly, one could always argue that, despite this serious methodological shortcoming, the conclusion of two types of processing performed by distinct brain structures drawn on the basis of the single-cell recording technique have been confirmed by other studies using different investigation methods. However, a consensus on this matter is still far from being reached, as the following brief review of the available data shows (see also Gross, 1992):
a) Neuropsychological and neurophysiological data. While certain prosopagnosic patients exhibit a recognition deficit without expression-processing impairment (Bruyer et al., 1983), others are deficient in both identity and expression recognition (Schweich and Bruyer, 1993). Similarly, some patients have intact recognition abilities accompanied by deficient expression processing (Bornstein, 1963; Parry et al., 1991), whereas a lesion of the amygdala suffices to provoke an interaction between expression processing and recognition (Young et al., 1996).

Furthermore, in a selective attention paradigm with a categorization task, expression processing and recognition were found to be independent in patients with left brain damage, but not in those with lesions on the right (Etcoff, 1984). In fact, in this type of paradigm, categorization of facial expressions is always identity-dependent (Schweinberger and Soukup, 1998 ; Schweinberger et al., 1999 ; Baudouin et al., 2002 ).

Finally, a correlation between a recognition deficit and an impaired ability to analyze facial expressions has been observed in right brain-damaged patients (Wedell, 1989), right hemispheric lobotomized patients (Braun et al., 1994), schizophrenics (Baudouin et al., 2002 ; Salem et al., 1996), and patients with a Capgras syndrome (Sansone et al., 1998). Moreover, neuroimaging data has also shown that certain cerebral regions are activated during both recognition and expression processing (Sergent et al., 1994).

b) Behavioral data. RTs on identity matching tasks are known to be shorter for familiar faces than for unfamiliar ones, whereas RTs on expression matching do not vary with face familiarity (Young et al., 1986). In addition, a change of expression appears to hamper episodic recognition of unfamiliar faces but not of familiar ones (Bruce, 1982). In the same vein, Bruce (1986) was unable to find evidence of a face familiarity effect on expression judgment RTs. All of these results have been used in support of the independence hypothesis. But other investigators have shown, on the contrary, that familiar smiling faces were recognized better than familiar faces with a neutral expression (Kottor, 1989; Endo et al., 1992). And a change of expression has been found to disrupt the episodic recognition of famous or unfamiliar faces (Davies and Milne, 1982; Sansone and Tiberghien, 1994). Finally, a smile increases the feeling of familiarity for both known and unknown faces (Baudouin et al., 2000b) and familiarity promotes the recognition of facial expressions of emotion (Baudouin et al., 2000a; Guillaume and Tiberghien, 2001).

In conclusion, one cannot legitimately say that the neurophysiological and behavioral data undeniably validate the hypothesized functional independence between the STS and the ITC. Stating that a consensus has been reached on this matter can only be the result of a biased selection of the empirical data. It is therefore not surprising either to find little agreement on how this hypothetical brain specialization should be interpreted from the functional standpoint. For example, for Allison et al. (2000), the STS is not implicated in expression processing but in "social perception", and as such, it is obviously related not only to the processing of facial expressions, but also to face identification. Moreover, Haxby et al. (2000) suggest that in humans, the STS is not specifically involved in expression processing but handles all "variable" aspects of the face, whereas the lateral fusiform gyrus is involved in a face's "invariant" aspects; these two sub-systems are assumed to be highly interactive. Note that the variable features of a face are no doubt more context-sensitive than its invariant features, the latter being sometimes also regarded as the product of expertise not limited to faces alone (Gauthier et al., 2000). The specialization at stake here could therefore even pertain to different kinds of processing, i.e., dependent or independent of context and based or not based on expertise. In the same vein Desimone (1997) suggests that he interactions between memory and attention in ITC result in the selection of objects that are foveated .

We have attempted to show from this case study that any statistical inference requires a random observation, and if this condition is not met, the inferred theoretical conclusions remain problematic. Showing that a "selected" neuron responds more often to a face or to one of its features, say its expression, in no way demonstrates that this particular neuron does not respond to a more specific or more general property of the face, or even to a given processing operation.
The present methodological analysis was aimed solely to defend a much more interactive and integrated approach to face information processing. The data obtained using the analysis technique described by Hasselmo et al. therefore does not authorize the conclusion that two distinct brain regions exist, one specialized in facial-expression processing and the other in face-identity processing. This conclusion would be obvious if these two regions were assumed to be mutually exclusive sub-systems -- a hypothesis which no one defends.

But the data obtained by Hasselmo et al. could also be interpreted by assuming the stochastic independence of two specialized, partially overlapping regions. But why then consider only two functional regions, when more than a third of the neurons selected in Hasselmo et al.’s study responded to neither expression nor identity, but probably to something else? The greater the number of stochastically independent systems, the more they overlap and the more the functional architecture necessarily tends to be distributed.

However, our second analysis of Hasselmo et al.’s data (supra, p. 4) could support the hypothesis of redundancy between identity processing and expression processing: identity-responding neurons would be widely spread across both the ITC and the STS whereas expression-responding neurons would be localized, or better yet distributed, in a redundant fashion principally in the STS. This would amount to contending, in a sense, that all face characteristics (and all neurons that respond specifically to them) could in principle contribute to recognizing identity, but not necessarily expression. On the other hand, a smaller number of features (and the neurons they activate) could respond to expression while enabling face identification. After all, let's not confuse Occam's razor with a hatchet.
 

References

ALLISON  T, PUCE  A, and MCCARTHY  G. Social perception from visual cues: role of the STS region. Trends in Cognitive Sciences, 4: 267-278, 2000.

BAUDOUIN JY , MARTIN F , TIBERGHIEN G , VERLUT N , and FRANCK N. Selective attention to facial emotion and identity in schizophrenia. Neuropsychologia, 40: 503-511, 2002.

BAUDOUIN JY, SANSONE S , and TIBERGHIEN G. Recognizing expression from familiar and unfamiliar faces. Pragmatics & Cognition, 8: 123-146, 2000a.

BAUDOUIN JY , GILIBERT D , SANSONE S, and TIBERGHIEN G. When the smile is a cue to familiarity. Memory, 8: 285-292, 2000b.

BORNSTEIN B.  Prosopagnosia. In HALPERN L (Ed), Problems of Dynamic Neurology. Jerusalem: Hadassah Medical School, 1963, pp. 283-318.

BRAUN CMJ, DENAULT C, COHEN H , and ROULEAU I. Emotional facial expressive in discrimination performance and lateralization in normal young adults. Brain and Cognition, 24: 198-212, 1994.

BRUCE V.  Changing faces: Visual and non visual coding processes in face recognition. British Journal of Psychology, 73: 369-384, 1982

BRUCE V. Influences of familiarity on the processing of faces. Perception, 15: 387-397, 1986.

BRUYER R, LATERRE C, SERON X , FEYEREISEN P, STRYPSTEIN E, PIERRARD E, and RECTEM D. A case of prosopagnosia with some preserved covert remembrance of familiar faces. Brain and Cognition, 2: 257-284, 1983.

DAVIES GM  and MILNE A. Recognising faces in and out of context. Current Psychological Research, 2: 235-246, 1982.

DESIMONE R.  Face-selective cells in the temporal cortex of monkeys. Journal of Cognitive Neuroscience, 3: 1-8, 1991.

DESIMONE R. Attention control of visual perception. Electroencephalography and Clinical Neurophysiology, 102: 4P, 1997.

ENDO N, ENDO M, KIRITA T, and MARUYAMA K. The effects of expression on face recognition, Tohoku Psychologia Folia, 51: 37-44, 1992.

ETCOFF NL. Selective attention to facial identity and facial emotion. Neuropsychologia, 22: 281-295, 1984.

FARAH MJ, WILSON KD, DRAIN M, and TANAKA JN. What is "special" about face perception? Psychological Review, 105(3): 482-498, 1998.

GAUTHIER I, SKUDLARSKI P, GORE JC, and ANDERSON AW. Expertise for cars and birds recruits brain areas involved in face recognition. Nature Neuroscience, 3(2): 191-197, 2000.

GAUTHIER I  and LOGOTHETIS NK. Is face recognition not so unique after all? Cognitive Neuropsychology, 17: 125-142, 2000.

GROSS CG. Visual stimuli and the inferior temporal cortex. Philosophical Transactions of the Royal Society, London, B335, 3-10, 1992.

GUILLAUME F  and TIBERGHIEN G. An event-related potential study of contextual modifications in a face recognition task. NeuroReport, 12(6): 1209-1216, 2001.

HASSELMO ME, ROLLS DI, and BAYLIS GC. The role of expression and identity in the face-selective responses of neurons in the temporal visual cortex of the monkey. Behavioral Brain Research, 32: 203-218, 1989.

HAXBY JV, HOFFMAN EA, and GOBBINI MI. The distributed human neural system for face perception. Trends in Cognitive Science, 4: 223-233, 2000.

KOTTOR TM. Recognition of faces by adults. Psychological Studies. 34: 102-105, 1989.

LEINONEN L  and NYMAN G. Functional properties of cells in antero part of aera 7 associative face area of awake monkeys. Experimental Brain Research, 34: 321-333, 1979.

PARRY FM, YOUNG AW, SAUL JSM, and MOSS A. Dissociable face processing impairments after brain injury. Journal of Clinical and Experimental Neuropsychology, 13: 545-558, 1991.

PERRETT DI, MISTLIN, AJ, and CHITTY AJ. Visual neurons responsive to faces. Trends in Neurosciences, 10(9): 358-364, 1987.

PERRETT DI, HIETANEN JK, ORAM MW, and BENSON PJ. Organization and functions of cells responsive to faces in the temporal cortex. Philosophical Transactions of the Royal Society, London, B335, 23-30, 1992.

SALEM JE, KRING A, and KERR SL. More evidence for generalized poor performance in facial emotion perception in schizophrenia. Journal of Abnormal Psychology, 105: 480-483, 1996.

SANSONE S  and TIBERGHIEN G.  Traitement de l'expression faciale et reconnaissance des visages: Indépendance ou interaction? [Facial expression processing and face recognition: Independence or interaction?]. Psychologie Française, 39: 327-344, 1994.

 SANSONE S, LUAUTE J P, BIDAULT E, and TIBERGHIEN G. Une conception neuropsychologique unitaire du syndrome de Capgras.[An unitary neuropsychological conception for the syndrome of Capgras]. Annales Médico-Psychologiques, 156: 433-444, 1998.

SCHWEICH M and BRUYER R. Heterogeneity in the cognitive manifestations of prosopagnosia: The study of a group of single cases. Cognitive Neuropsychology, 10: 529-547, 1993.

SCHWEINBERGER SR  and SOUKUP GR. Asymmetric relationship among perceptions of facial identity, emotion, and facial speech. Journal of Experimental Psychology: Human Perception and Performance, 24: 1748-1765, 1998.

SCHWEINBERGER SR, BURTON AM, and KELLY SW. Asymmetric relationship between identity and emotion perception: Experiments with morphed faces. Perception & Psychophysics, 61: 1102-1115, 1999.

SERGENT J, SHINSUKE O, MACDONALD B, and ZUCK E.  Segregated processing of facial identity and emotion in the human brain: A PET-scan study. Visual Cognition, 1 (2/3): 349-369, 1994.

WANG G, TANIFUJI M, TANAKA K. Functional architecture in monkey inferotemporal cortex revealed by in vivo optical imaging. Neuroscience Research, 32: 33-46, 1998.

WEDELL RA. Recognition memory for emotional facial expressions in patients with focal cerebral lesions. Brain and Cognition, 11: 1-17, 1989.

YOUNG AW, MCWEENY KH, HAY DC, and ELLIS, AW. Matching familiar and unfamiliar faces on identity and expression. Psychological Research, 48: 63-68, 1986.

YOUNG AW, HELLAWELL D, VAN DE WAL C, and JOHNSON M. Facial expression processing after amygdalotomy. Neuropsychologia, 34: 31-39, 1996.
.