The Inner and Outer Meanings of Facial Expressions

Joseph C. Hager
University of California-San Francisco

Paul Ekman
University of California-San Francisco

Chapter 10
From: J. T. Cacioppo & R. E. Petty (Eds.)
Social Psychophysiology: A Sourcebook
New York: The Guilford Press, 1983
Copyright © 1983 The Guilford Press
Reprinted by permission.

INTRODUCTION

Investigators from a number of fields of psychology have been interested in facial expressions of emotion. Social psychologists studying person perception have often focused on the face. Recent research is examining the relative weight given to the face as compared to other sources of information, the relationship between encoding and decoding, and individual differences. Developmental psychologists are examining the age at which infants first show what can be considered an emotion, whether this age precedes or follows an infant's ability to recognize emotions, and the sequencing of expressions between caregiver and infant. Physiological psychologists have been concerned with the role of the right hemisphere in the recognition and, more recently, in the production of facial expression, and in the relationship between facial and autonomic measures of arousal. Many different investigators are studying the face in order to help answer the question of how we know how we feel.

These are but a few examples of the many divergent questions that involve consideration of facial expression. Most of these questions are not new. They were subject to considerable research a few decades ago, although sometimes the questions were phrased differently. Unfortunately, little progress was made. The most basic questions were not answered, and methods for measuring facial expression were not well developed. In the last decade, progress has been made both on methods and on a set of fundamental questions.

This chapter begins by reviewing the answers that have emerged to three basic questions about the face and emotion: Is there any relationship at all? Are facial expressions culturally bound or universal? And, are any universals in expression biologically based? Then we describe a new tool for measuring facial movement which has allowed more precise study of the face. We will report new findings that begin to clarify the nature of facial signals, and the degree to which they relate to different types of feelings.

Darwin (1872) argued that certain emotional expressions are innate and the same for all people. His evidence and arguments were largely ignored by scientists in the subsequent century. Instead, the view that facial expressions are not valid indicators of emotion was widely accepted even though the evidence was contradictory (Bruner & Tagiuri, 1954). Ekman, Friesen, and Ellsworth (1972, 1982) resolved this issue definitively by pointing out methodological problems that had confused other researchers. They showed that observers could agree on how to label both posed and spontaneous facial expressions in terms of either emotional categories or emotional dimensions. Much evidence, including reanalysis of negative studies, indicated that facial expressions can provide accurate information about emotion. The labels judges assigned to posed expressions tended to agree with the poser's intended message. For spontaneous expressions, judges selected labels consistent with emotions appropriate in the situations that elicited the expressions. These studies of spontaneous expression indicated that observers could distinguish pleasant from unpleasant emotions, but evidence was weak that observers could make finer distinctions about more specific categories of emotion, such as fear from anger. Also, there was no evidence about whether the face provided graded information about the intensity of any specific emotion (e.g., annoyance, anger, fury).

Unlike Darwin, anthropologists who endorsed cultural relativity argued that the meanings of expressions were arbitrary and specific to each culture, like symbols in a language (e.g., Birdwhistell, 1970; LaBarre, 1947). Recent evidence (e.g., Ekman, Sorenson, & Friesen, 1969; Izard, 1971) has indisputably shown that there are constants across cultures in the emotional meanings of certain facial expressions (for a detailed review of all the evidence, including studies of infants, the blind, and other primates, see Ekman, 1973). Ekman (1972) used a "neurocultural" theory to explain how cultural as well as biological influences could contribute to the meaning and use of facial expressions. A central concept in this theory is "display rules," which are an informal, nonverbal etiquette about socially acceptable ways to use and control expressions. Previous researchers had probably confused these culture specific modifications of emotional behaviors with the universals of expression. For example, Samurai women were reported to smile rather than to cry when hearing that their loved ones had died in battle (LaBarre, 1947). Although such observations were taken as evidence of cultural variability in the meaning of smiles, these smiles may not have been signs of grief, but rather could have been culturally required masks implementing the display rule to show joy and hide distress in this public situation (Ekman, 1973).

Evidence of universals in facial expression does not prove that they are innate, as Darwin believed. Universal connections between expressions and emotions could arise from learning which has a high probability of occurring in all cultures (Allport, 1924) or from a functional role of the movements in the emotional situation (Ekman, 1979). However, other evidence also supports the hypothesis that innate, biological factors mold some facial expressions. Oster (1978; Oster & Ekrnan, 1978) examined the expressions of neonates and found that certain spontaneous facial actions were not random, but rather were organized and temporally patterned. Since there was no opportunity to learn these patterns, some hardwired instructions underlie this organization. The relation of infant expressions to emotional behaviors has yet to be established, although some expressions, such as disgust, distress, and enjoyment, correspond to situations that can elicit these emotions in adults. Anencephalic neonates also have patterned facial responses to certain stimuli, such as disgust expressions to bitter-tasting substances and smiles to sweet tastes (Steiner, 1973). Studies of blind infants and children generally support the position that many facial expressions result from innate factors rather than depending on visual learning (Charlesworth & Kreutzer, 1973).

The evidence shows that facial expressions are related to emotion both biologically and culturally, but many other important questions remain. Until recently, all the evidence was based on observers' judgments of the face, which presumably reflect the expressions and messages the face provides. Few studies have tried to measure how the face conveys this information or precisely what the cues are for each emotion. Even fewer researchers have tried to measure every possible facial expression. Is it possible to describe and quantify every action the face can perform? If so, facial measurement can tell us about the universe of facial signals, and answer such questions as: How many different expressions are possible? Which of these expressions have emotional meanings and which have some other meanings? What muscles are involved in each emotion? Are there different muscles for each emotion, or do the same muscles play a role in more than one emotion? Can an emotion be shown by one muscle action, or does the expression of emotion require the combination of actions which are not meaningful singly?

THE FACIAL ACTION CODING SYSTEM

Ekman and Friesen's Facial Action Coding System (FACS) (1976, 1978) measures all visible facial movements. Ideally, FACS would differentiate every change in muscular action, but it is limited to what a user can reliably discriminate when movements are inspected repeatedly, in stopped and slowed motion. It does not measure invisible changes (e.g., certain changes in muscle tonus) or vascular and glandular changes produced by the autonomic nervous system. Limiting FACS measurement to visible movements was consistent with an interest in those behaviors which may be social signals, usually detected during social interactions. FACS can be applied to any reasonably detailed visual record of facial behavior. If the technique were to measure invisible or autonomic nervous system (ANS) activity, it would be limited to situations were sensors were attached (e.g., EMG electrodes) or special sensing and recording methods were used (e.g., thermography).

The primary goal in developing FACS was comprehensiveness, a technique that could measure all possible, visible discriminable facial actions. Comprehensiveness was important because many of the fundamental questions about the universe and nature of facial expressions cannot be answered if just a subset of behaviors is measurable. FACS was derived from an analysis of the anatomical basis for facial movement. A comprehensive system was obtained by discovering how each muscle of the face acts to change visible appearances. With this knowledge it is possible to analyze any facial movement into anatomically based, minimal action units.

Another consideration that guided the development of FACS was the need to separate description from inferences about the meanings of behaviors. Scoring is less likely to be biased if the observer does not have to evaluate or attach meanings to behaviors. Almost all the previous descriptive systems have included some inferential scores, such as "aggressive frown" (Grant, 1969), "lower lip pout" (Blurton Jones, 1971), and "smile tight-loose o" (Birdwhistell, 1970). Each of these actions could be described in noninferential terms.

By emphasizing measurement of the face in terms of muscle actions, inferences about meanings are minimized. The user of FACS learns the mechanics or muscular basis of facial movement, not simply the consequences of actions or a description of static landmarks. FACS emphasizes patterns of movement: the movements of the skin, the temporary changes in size and location of the features, and the gathering, pouching, bulging, and wrinkling of the skin. As time passes, FACS users increasingly focus on behavioral description and are rarely aware of "meanings."

FACS's emphasis on movement and muscular action also helps overcome problems due to physiognomic differences between people. Individuals differ in the size, shape, and location of their features and in permanent wrinkles, bulges, or pouches. The particular shape of a landmark may vary from one person to another; for example, when the lip corner goes up, all people may not have the same angle, shape, or wrinkle pattern. If only the end result of movement is described, scoring may be confused by physiognomic variation. Knowledge of the muscular basis for actions helps deal with these differences.

FACS measures facial behaviors with "action units" (AUs), which indicate what muscles have contracted to produce the expression. Figure 10-1 illustrates the three AUs in the brow area and their combinations. Ekman and Friesen learned to contract each muscle separately and determined each AU based on the discriminability of their actions. In a few cases, more than one muscle was combined into one AU or more than one AU was derived from what most anatomists have described as one muscle.

After determining the single AUs, between 4000 and 5000 AU combinations were performed and examined. This total includes all the possible combinations of AUs in the upper regions of the face, all two-AU and three-AU combinations in the lower face, plus some of the four-, five-, six-, seven-, and eight-AU combinations in the lower face. Study of these combinations showed that most of the appearance changes were additive (i.e., each AU was clearly recognizable and virtually unchanged). There were a few AU combinations which were not additive, but instead showed new appearances. All of these distinctive combinations are described in FACS in the same detail as the single AUs.

FACS is a very elaborate system, much more comprehensive than any previous technique. There is no facial action described by other systems which cannot be described by FACS, and there are many behaviors described by FACS not previously distinguished. FACS allows for scoring asymmetries, either in terms of different AUs or different intensities. A means for measuring the intensity and the timing of actions is also detailed.

FIGURE 10-1. The three FACS action units in the brow area and their combinations are illustrated. AU 1 (action of inner frontalis) raises the inner corners of the eyebrows, forming wrinkles in the medial part of the brow. AU 2 (action of the outer frontalis) raises the outer portion of the eyebrows, forming wrinkles In the lateral part of the brow. AU 4 (action of procerus, corrugators, and depressor supercilii) pulls the eyebrows down and together, forming vertical wrinkles between them and horizontal wrinkles near the nasion. The combinations of AUS show how these AUs can act together to form composites of the appearances each produces separately.
upper face components of emotional expressions

Reliability was a major concern in the development of FACS. Ekman and Friesen (1978) assessed several aspects of reliability, including describing the behavior verses locating it in time. Their studies have repeatedly shown good reliability even when the learner uses only the self-instructional FACS manual without direct guidance from FACS's authors. The evidence shows that FACS can successfully measure the visibly distinctive facial actions as its authors intended.

Besides being reliable, FACS has revealed the answer to many basic questions about expressions. From the single AUs and their combinations, Ekman and Friesen have estimated that there are several hundred thousand possible visibly distinguishable facial expressions, most of which are never seen on people's faces in everyday life. FACS has been used to score pictures of faces which observers have judged to express emotion and to score faces of people in emotionally arousing situations. Based on evidence from such scoring, the expressions produced by different combinations of AUs which convey emotional meanings appear to number in the hundreds, if not thousands. If the strength of muscular contraction and the timing or sequence of muscular recruitment were included, this number would be substantially increased. Of course, people do not have a different emotion name for each of these expressions. Instead, many emotional expressions are synonyms or convey different connotations of particular emotions. Further research is needed to determine the extent to which synonymous expressions with different AUs, strengths of contraction, or timing can be distinguished by naive observers and whether 'Such distinctions are accompanied by additional messages. For example, different expressions which are judged as one emotion may be perceived as genuine, as an attempt to deceive, as artificial, or as a word-like symbol for an emotion. Observers also perceive differences in the intensity of emotion expressions which may be based on strength of muscular contraction, number of muscles recruited, or area of the face in which contractions occur. The number of expressions conveying emotional meanings is much greater than researchers have typically acknowledged, but it is much smaller than the number of possible expressions.

Every facial muscle can be involved in one or more emotional expressions, so there is no distinction between emotional and nonemotional muscles. Some muscles always signal a particular emotion, such as zygomatic major which produces a smile and is characteristic of happiness. It is never involved in a negative emotional expression without blending its own message. Other muscles, such as the corrugator, are involved in expressions which convey many different emotional messages and nonemotional messages. Some emotions, such as happiness and disgust, can be signaled by the action of only one muscle, but other emotions, such as sadness, need the action of more than one muscle to be signaled unambiguously.

Findings like those above cannot be obtained without the comprehensiveness of FACS. Consider other measurement systems which are not comprehensive, such as the Facial Affect Scoring Technique (Ekman, Friesen, & Tomkins, 1971) and Affex (Izard, 1980). Such noncomprehensive techniques consist of a limited number of expressions which the authors thought on a priori grounds were relevant to emotion. The problem with these techniques is that only the expressions included in the system can be studied; other possible expressions are ignored if they cannot be coded. These techniques can say little about the universe of expressions. Conclusions based on them are limited to only the expressions measured. Hypotheses about those expressions can be tested, but the possibility that other actions signal emotion cannot be discovered.

Researchers using electromyography (EMG) have measured facial activity without relying on an observer's ability to distinguish visible actions, but their efforts have not resulted in a comprehensive measurement system. It would be possible to construct a comprehensive measurement system by specifying the electrode placements which would measure all distinctive facial expressions. However, a surface electrode measures any muscular activity in its general area (see Basmajian, 1978, p. 26), eliminating distinctions that may be made visually. Thus, the researcher would have to use a pattern of surface electrode placements carefully configured to discriminate the activities of nearby muscles. An alternative solution might be to use a needle electrode which measures only the activity of the muscle in which it is inserted. If measuring all muscular activity in the face were the goal, either of these procedures might involve an impractical number of electrodes since each muscle or portions of muscles on both sides of the face would have to be monitored.

EMG has both advantages and disadvantages in respect to visible measurement of facial behavior. EMG units for measuring the activity of muscles are smaller and enable more precise measurement of the degree of activity than units of intensity based on visible distinctions. The often used procedure of averaging EMG over seconds loses information about the visually distinguishable timing of contractions. Also, the relation between EMG measurements and visible movement is ambiguous because EMG may measure inhibition of movement as well as actual movement. A distinct advantage of EMG is that it can measure invisible activity, but a distinct disadvantage is that the electrodes alert subjects to the observation of their faces, which may alter normal behavior (see the section on asymmetry of expressions). Although attaching electrodes to other parts of the body may or may not confuse the subject about the investigator's interest in the face, there is no doubt that EMG electrodes are intrusive and make it impossible to hide the very fact of observation and recording. (See Ekman, 1981, for a more detailed comparison and results on the relationship between EMG and the coding of visible facial movement.)

THE RELATION BETWEEN FACIAL EXPRESSIONS AND THE FEELINGS OF EMOTION

Facial measurement has answered questions about the universe of facial expression. It has also helped to discern the functions of facial expressions. Social psychologists have usually been interested in expressions as social signals, although they rarely measured the signal itself. Another function of facial expression may be as a signal to the self about one's own emotional state. The next several paragraphs discuss how some important theories of emotion view the relation between facial expressions and the feelings of emotion. This relation is only one of many which comprise the construct of emotion, but discussing these other relationships is beyond the scope of this chapter. Later, however, a brief look at the evidence for these theories emphasizes the mistake of assuming that these different relationships, such as those between facial expression and feelings versus those between physiological arousal and feelings, are interchangeable rather than possibly quite different and even apparently contradictory.

William James (1884) promoted the idea that the feelings of emotions arise from the perception of characteristic bodily changes. In an elaborate theory of emotion, Tomkins (1962, 1982) postulated that there are nine fundamental affects and affect auxiliaries, each having a characteristic and innate facial, vocal, and physiological expression. These expressions provide feedback which when consciously perceived gives rise to the feelings of affect. Each innate expression has inherently different feedback which underlies each emotional feeling. In recent statements, Tomkins (1982) regarded the feedback from the skin of the face, altered by blood flow and muscle movements, as most important for the feelings of affect. Although the number of affects is limited, the variety of feelings experienced is great because affects can blend and because other inputs to consciousness influence emotional experience. According to Tomkins, emotional feelings can also arise in the absence of facial expressions if there are memories that can substitute for them.

In contrast to the theories that emphasize peripheral changes in the body are theories that specify cognitive or inferential decisions as the source of emotional feelings. Schachter and Singer (1962) promulgated one of the first cognitive social theories of emotion in a widely cited experiment. In their theory, the important determinants of the quality of emotional feelings are cognitions about physiological arousal. Arousal that has no apparent explanation creates a need to label the feeling it produces in emotional terms. Situational and social cues provide a basis for inferring an appropriate category of emotion, and this decision underlies the qualitative differences in emotional feelings. Arousal itself is probably undifferentiated (i.e., the same across all emotions), but if different patterns exist, they are unimportant factors. This theory was intensely criticized, in part, because it did not explain how arousal itself was elicited, the results of the experiment were not as predicted and were rationalized with post hoc explanations, and the experiment has not replicated (e.g., Izard, 1971; Leventhal, 1974; Marshall & Zimbardo, 1979; Maslach, 1979). Nevertheless, this theory became the dominant model of emotion for social psychologists, in part because of its emphasis on cognition.

Mandler (1975) also emphasized autonomic arousal and cognitive interpretation as the important factors in determining the feelings of emotion. Arousal is probably undifferentiated and determines only the visceral character and intensity of emotion. Interpretation determines the quality and category of the experience. Mandler discussed the role of facial expression in this kind of model. Facial expressions may be biologically tied to certain events or situations which, in turn, have a high probability of eliciting particular cognitions about emotion. Also, expressions may generate automatic cognitions which contribute to the interpretative process. Rather than an inherent, direct link to feelings, these automatic cognitions depend on cognitive interpretation to influence emotional feelings. The role of cognition in emotion is so important that Mandler considers the belief in fundamental emotions "a human vanity."

Bem's (1972) self-perception theory also links emotional feelings to inferences based on behavioral cues. People may observe their own facial behaviors and use them as cues about emotion (Laird, 1974). The ability to use such cues depends on learning from the verbal community to make the discriminations. Unlike Mandler, Bem's theory does not explain why the same connections between particular facial expressions and emotions are found universally across widely differing cultures.

The dispute over the source of emotional feelings is difficult to resolve empirically, in part because conceptions like Mandler's "automatic cognitions" and Tomkins's "consciousness of facial feedback" generate many similar predictions. One approach is to examine how closely distinct facial expressions are associated with different emotional feelings. If the differences are merely between positive and negative emotions, they cannot be the substrates for emotional feelings as hypothesized by facial feedback models. Conversely, subtle, intimate relations between expressions and feelings would challenge the cognitive theories, especially if these relations are too subtle for an ordinary observer to detect or to teach to expressors.

None of the past studies which have shown that the face can provide accurate information about emotion measured felt experience. Some did provide indirect evidence of a relation between expressions and feelings. For example, in many studies of posed expressions, subjects were asked to make their face look like they felt an emotion, and these expressions were distinctive enough for judges to identify the intended emotion. Only recently have studies directly assessed the relation between feelings and expression.

In his review of this issue, Buck (1980) distinguished two versions of the hypothesis that feedback from facial expression underlies the feelings of emotion. The "between-subjects version" specifies individual differences in emotion (e.g., less expressive people have less intense emotions), and the "within-subjects version" specifies that for any given person, the degree of expressiveness is positively related to intensity of emotion. Looking at the evidence for each version separately, Buck rejected the between-subjects version based on evidence showing an inverse, rather than a positive relation between facial expression and physiological arousal (e.g., Buck, Miller, & Caul, 1974; Notarius & Levenson, 1979). However, a recent study by Levenson and Mades (1980) suggested that this inverse relationship may arise only between "true-low-anxious" versus "repressor" subjects. None of these studies directly addressed the central issues of the facial feedback model proposed by Tomkins (1962) and reiterated by Izard(1971). They did not examine the evidence crucial to this model - the relation between facial expression and the subjective experience of emotion. Instead, they examined the relation between expression and autonomic activity. There is no reason to presuppose that ANS activity and facial expression are related the same way as subjective experience and facial expression. Still another problem with these studies is that they lumped together quite diverse expressions, failed to measure expressions directly, and only counted activities, not the type of activity (e.g., whether the expressions were emotional or not).

Two substantive issues concern a relation between facial expression and emotional feelings: (1) distinctive expressions correspond to different feelings, and (2) intensities of expressions and feelings correlate. Two methodological approaches are possible. One is to manipulate facial expression experimentally and - measure feelings to show that changes in expression alter feelings. This approach can provide a direct test of the facial-feedback models of emotional feelings, but it is difficult to design such a study without introducing artifacts.

Several studies have manipulated the degree of spontaneous smiling to films and measured the effect on evaluation of the films (e.g., Fuller & Sheehy-Skeffington, 1974; Leventhal & Mace, 1970). Generally, conditions that produced greater smiling also produced evaluations that the content of the films was more humorous, although this relation may not hold for men (Leventhal, 1974). Regrettably, these studies did not assess whether there was a greater experience of humor. The implications of such findings for the facial-feedback models is unclear because the relationship between evaluations of films and felt emotion is not straightforward (Leventhal, 1974). Evaluating the humor in a film may depend, for example, not only on evaluating the emotional feelings one has in response to it, but also on evaluating how funny it was in respect to other films, how easily one responds to humor in respect to other people, how tasteful the humorous devices in the film were, and so on. In other words, evaluating films is more cognitively complex than evaluating how one feels. Lanzetta, Cartwright-Smith, and Kleck (1976) manipulated the intensity of pain expressions and assessed the effect on self-reports of pain in response to electric shock. Conditions that affected intensity of pain expressions altered self-reports of shock painfulness. Although this finding fits with the facial-feedback hypothesis, most of its proponents (e.g., Izard, 1971; Tomkins, 1962) do not think that pain is an emotion.

Several studies have tried to show that manipulating the face into simulacrums of emotion expressions produces emotion. Laird (1974) had subjects frown or smile and concluded that these movements altered selfreports of aggression and elation. He theorized that expressions influence feelings by a process of tacit inference, but he did not explain why subjects inferred that the expressions were relevant to emotion if they were clearly prescribed by the experimenter. Rhodewalt and Comer (1979; Comer& Rhodewalt, 1979) and McArthur, Solomon, and Jaffe (1980) used a procedure similar to Laird's and obtained similar results concerning the relations between manipulated frowns or smiles and self-reports of feelings. Their results indicated that individual differences (e.g., weight and field dependence) influence this relation. A problem for interpreting these studies is that in the McArthur et al. study, only the frown appeared to alter reports of feelings. The other two studies did not permit assessing which expression was effective.

Tourangeau and Ellsworth (1979) conducted a similar but more elaborate study than those described above. Like the others, they claimed to have manipulated facial expressions without alerting subjects to their emotional meanings. Subjects made either fear or sad expressions, or a grimace unrelated to emotion. They watched either a fear, sad, or neutral film and reported their emotional feelings. The experiment revealed no effect of facial expression on self-report, either for producing a corresponding feeling or for inhibiting other feelings aroused by the film. Nor did the investigators find a correlation between observers' ratings of the intensity of expression and self-reports of feelings.

Three articles criticized the Tourangeau and Ellsworth experiment, but many of the criticisms also apply to other studies which have experimentally manipulated facial expressions whether their results confirmed or disconfirmed a relation between expressions and feelings. Hager and Ekman (1981) argued that the experiment was an inadequate test of the hypothesis, partly because of inadequate facial measurement. Tomkins (1981) explained that the experiment had nothing to do with his theory of emotion because artificial expressions are not related to emotional feelings, as are spontaneous expressions. In his comments on methodological and conceptual problems, Izard (1981) referred to his own studies of this issue (Kotsch, Izard, & Walker, 1979). They gave little evidence that such artificial expressions can produce emotional feelings. Izard argued that making extreme voluntary facial movements can increase self-reports of anger, a view that suggests a reinterpretation of the studies using Laird's procedure. In general, the studies of manipulated expressions have not provided much evidence for an association between facial expression and emotion.

The other approach to demonstrating a relation between expressions and feelings is to create conditions which elicit different emotions, measure the feelings and expressions, and determine the relations between them. This approach also has problems. For example, retrospective reports may be distorted by memory. The alternative procedure of interrupting the emotional experience to obtain self-reports may create artifacts, especially if done repeatedly.

Several recent studies have examined how facial expressions are related to the experience of emotion, but for the most part, they have only replicated established findings. Schwartz, Fair, Salt, Mandel, and Klerman (1976), for example, found that EMG activity was different when subjects imagined different emotions. The contribution of this study was using EMG to measure low or invisible levels of facial activity. It is not clear, however, just what processes are involved in imagining emotion. They may be little different than posing, and many studies from 1930 to 1960 (reviewed by Ekman et al., 1972) have shown that facial expressions differ when people pose different emotions. The question remains whether expressions differ among more spontaneous emotions. Buck and his colleagues (e.g., Buck et al., 1974) showed observers videotapes of subjects who viewed pleasant or unpleasant slides. They found that observers' ratings of subjects' pleasant-unpleasant feelings were correlated with the subjects' own ratings. Again, many studies decades ago showed that facial expressions differed for this simple pleasant-unpleasant distinction. What is needed is to go beyond this distinction and determine whether spontaneous facial expressions vary with more specific aspects of emotional experience.

Ekman, Friesen, and Ancoli (1980) conducted such a study. The faces of 35 women were videotaped without their knowledge. The women viewed a positive film with three segments, of which two had elicited mostly happiness and one, mostly relaxation in previous studies. They also viewed a negative film which showed two industrial accidents. Subjects reported their emotional reactions on a questionnaire which had separate unipolar scales for interest, anger, disgust, fear, happy, pain, sadness, surprise, and arousal.

The experiment was conducted individually for each subject. After a baseline period in which subjects relaxed, they reported their feelings on the questionnaire. They then saw the positive and the negative films in a counterbalanced order. Between the two films was another baseline period and a questionnaire for it. After the positive film, subjects filled out the questionnaire once for each of the three segments. They also reported their emotions for each accident after the negative film.

FACS was used to measure the activity of the face during the films. The investigators looked for signs of positive affect in the two happy segments of the positive film. It may seem obvious that smiles are the signs of positive affect, but some observers (e.g., Birdwhistell, 1970) claimed that the smile can be a sign of negative affect. The term "smile" may be too imprecise to distinguish behaviors with different meanings. Other researchers (e.g., Brannigan & Humphries, 1972; Grant, 1969) distinguished more than one type of smile (e.g., upper smile, broad smile, tight smile), but they rarely specified the same number of smiles or which ones, if any, are signs of positive affect.

FACS distinguishes many more types of smiles than other measurement systems have. A smiling appearance in which the lip corners are pulled upward and laterally can be produced by the action of zygomatic major, zygomatic minor, buccinator, risorius, and caninus muscles. FACS can score each of these actions, their combinations, and their combinations with other facial actions. Ekman, Friesen, and Ancoli (1980) found, as they predicted, that the specific smiling action of zygomatic major was related to subjects' self-reports of happiness, but other smiling actions were not. Second, the relation between this one action and felt experience was so precise that its activity accurately reflected during which film segment each subject felt happier. Third, measures of the extent of this muscular activity were related to the intensity of felt happiness. The authors also reported that other facial actions correlated as predicted with the intensity of felt negative emotions. Finally, actions predicted to be signs of disgust (levator labii superioris) correlated with reports of disgust feelings but not with the reports of other negative emotions.

ASYMMETRY OF FACIAL ACTIONS

Facial actions are not limited to spontaneous emotional expressions. In addition to posed expressions, there are false expressions which are put on to convince others that an emotion not actually felt is being experienced. There are also many facial actions which bear little relationship to any type of emotion: facial gestures such as the wink, and facial emphasis and question marks (see Ekman, 1978, 1979, for a description of various facial signals). Our analyses of recent studies of facial asymmetry, and a new study we then conducted, suggest that symmetry of facial action may be informative about whether a facial action is an expression of a felt emotion or is not felt but purposefully made.

Most researchers have used observers' judgments to assess facial asymmetry. Thus, evidence that observers rate one side of the face as happier, angrier, and so on, has been interpreted to show that this side expresses the emotion more intensely. This logic assumes that judgments are based on facial cues that express emotion, but they may not be. The face provides many cues which are irrelevant to the expression of emotion, but which observers sometimes confuse with emotional cues. For example, people who have thick eyebrows set low in relation to their eyes may appear to be frowning perpetually and give the impression of anger to observers, 'regardless of the true* emotional state. Such physiognomic features and other features which change slowly over time (e.g., wrinkles) are known to be asymmetrical (e.g., Gorney & Harries, 1974), and these asymmetries may be lateralized. For example, Burke (1971) found that in a group of children, the maxillary skin surface area tended to be greater on the left side. Artificially created cues may also be asymmetrical, as in hair combing and applying some cosmetics (e.g., to emphasize moles or hide blemishes).

Using observers' judgments to index asymmetry in the intensity of emotional expression creates a significant problem for interpretation. The influence on judgments of the muscle movement cues which signal emotion cannot be separated from physiognomic or artificial cues which are irrelevant to emotion. If intensity judgments of the sides of the face differ due to asymmetries in cues irrelevant to emotion, it would be a mistake to conclude that emotions are expressed more intensely on one side. Only if judgments are based on cues relevant to emotion expression would such a conclusion be warranted. The problem is increased when still photographs are used as stimuli because there are fewer cues for judges to distinguish static features from muscle actions.

An example of this problem is a study by Sackeim, Gur, and Saucy (1978). To obtain stimulus faces, they printed one photograph normally and one mirror-reversed photograph by turning the negative over. They cut these photographs down the midline and rearranged the halves to produce faces in which each side was an image of the other (i.e., either a right or a left composite). Observers judged on rating scales how intensely the composite faces expressed emotion. The left composite pictures were judged as more intense than the right composites in five of the six emotion categories. The authors stated that "emotions are expressed more intensely on the left side of the face."

Nelson and Horowitz (1980) showed that there were asymmetries in the sizes of the faces used by Sackeim et al. and argued that this variable might have influenced their findings. Spinrad (1980) noted that artificial cues produced by different lighting on the two sides might have been responsible. Such alternative explanations can be eliminated by measuring the facial movement cues of emotion directly.

Whether observers' judgments or direct measurement are used to assess asymmetry, the researcher must determine whether the expressions are emotional or not, or more generally, the type of facial movement must be distinguished. Ekman (1980) criticized the Sackeim et al. study because they did not distinguish carefully enough the type of facial movements they studied. He explained that they failed to recognize that they had studied at least two types of facial movements and that there was a difference in judged asymmetry between these two types. Ekman noted that the expressions which were judged as more intense on the left were not genuine emotional expressions, but rather were deliberately produced movements, carefully directed by the photographer, who gave instructions to move particular muscles, such as "raise your upper lip." Sackeim et al. did not find left composites more intense for the one expression (happy) which spontaneously occurred during the photographic session.

Other studies of asymmetry have studied ambiguous types of facial movements. Some have recorded conversations (e.g., Moscovitch & Olds, 1979), but the types of movements during conversation are especially varied. For example, although some expressions are spontaneous in the sense that they have not been requested, the speaker can initiate deliberate movements. Movements that signal emotion may be spontaneous emotional expressions or emblems that refer to emotion but do not involve emotional experience (Ekman, 1978).

Some studies have examined the movements of subjects who posed emotion (e.g., Borod & Caron, 1980), but subjects can adopt a variety of performance strategies to produce different types of movements. Ekman, Roper, and Hager (1980) noted that when people are asked to pose an emotion or to imitate an expression, they could use at least two methods to solve the problem. Subjects could self-induce the emotion and allow the expression to emerge, as in method acting. Alternatively, subjects could deliberately produce movements without emotion. Situations' like conversation and posing an emotion are not conducive for observing one specific type of movement.

Another circumstance that creates ambiguity about the type of movement produced is when subjects know or suspect that their face is being scrutinized. Once aware of observation, subjects may become self-conscious and alter their facial behaviors (Ekman, 1972; Kleck, Vaughan, Cartwright-Smith, Vaughan, Colby, & Lanzetta, 1976). In studies of natural, spontaneous movements, self-consciousness can be minimized by recording behaviors in a manner that does not draw attention to the observation.

Ekman, Hager, and Friesen (1981) showed that, indeed, the pattern of asymmetry depends on the type of movement examined. They reduced ambiguity about the type of movement by carefully choosing circumstances that would elicit particular types. The two sets of records chosen for asymmetry scoring were collected in other studies and had been scored by FACS previously. To score asymmetry, each action was again viewed repeatedly in slowed and real time. At the apex of each action, the movements were scored as symmetrical or asymmetrical, with greater intensity on the left or the right side.

One set of records was from a study of the development of the ability to imitate facial movements (Ekman, Roper, & Hager, 1980). Boys and girls (n = 36) imitated a series of facial actions shown to them one at a time on a television monitor. Their performances of six actions were selected for symmetry scoring. In addition, spontaneous movements of zygomatic major were located and scored on most children's faces. These smiles occurred in response to the experimenter's jokes and praise.

A contrast between the deliberate and spontaneous emotional use of zygomatic major showed different patterns of asymmetry. Asymmetrical movements were significantly more frequent in deliberate imitations than in spontaneous smiles. Deliberate asymmetrical movements were more frequently greater on the left than the right side of the face. This laterality of movement was not apparent for spontaneous smiles. The deliberate imitations of the five other muscles scored were as frequently asymmetrical as deliberate smiles, and these asymmetrical movements were more often greater on the left than the right.

The other set of records (from Ekman, Friesen, & Ancoli, 1980) showed women spontaneously expressing both positive and negative emotions. Like children's spontaneous smiles, the smiles of women during a humorous film were rarely asymmetrical and did not manifest laterality. Asymmetries of negative emotional movements during a stress film also were not lateralized, but they were often asymmetrical. The small number of these negative movements made these findings tentative.

Taken together, these results indicate that asymmetry of facial actions is a function of the type of movement. Spontaneous movements that occurred in an emotional context were rarely asymmetrical and were not stronger on one side more frequently than on the other. Lynn and Lynn (1943) reported results for spontaneous happy expressions entirely consistent with this finding. Deliberate imitative actions were more often asymmetrical and these asymmetries were lateralized, with the left side stronger. Campbell (1978) and Chaurasia and Goswami (1975) reported similar results for the deliberate movements they studied.

The evidence that symmetry differs between spontaneous, emotional movements and deliberate nonemotional movements suggests that the symmetry of facial movements may be related to the felt experience of emotion. Since the nonemotional movements in the studies discussed above were more frequently asymmetrical than the emotional movements, one prediction is that the more symmetrical the expression, the more likely it is that the person actually experiences the emotion. Our attempts to verify such hypotheses with post hoc analyses have been inconsistent. Contrary to the prediction, women with greater asymmetry of zygomatic major reported more happiness during the second humorous film segment, although there was an insignificant trend in the predicted direction during the first segment. Consistent with the prediction, zygomatic major smiles that occurred during pleasant films were more symmetrical than such smiles which occurred during unpleasant films, but additional. aspects of the smiles differed between film conditions, such as the actions that co-occurred with the zygomatic action. Our preliminary observations of conversational facial movements which do not involve emotion indicate that they are often asymmetrical.

DISCUSSION AND SUMMARY

The evidence on the universe of facial expression indicates that it is a large and complex set. The relation of spontaneous expressions to emotion is precise and refined with different expressions corresponding to distinct emotions. Even subtle differences in one expression (i.e., intensity, duration, frequency) correspond to differences in the feelings of the corresponding emotion. The symmetry of expression may reveal whether it is spontaneous and emotional or more deliberate and cortically mediated. What are the implications of these findings for the face as a signal system and for theories of emotion?

Whether the information revealed by careful facial measurement can be detected by untrained observers has yet to be determined. It is likely that the characteristic spontaneous expressions of different emotions can be seen and understood by the naive observer, as indicated by studies using posed expressions. More research on spontaneous expression is needed to verify this point. On the other hand, some aspects of expression, such as differences in intensity and asymmetry, typically appear too subtle for untrained observers to detect or too insignificant for attaching meaning.

Our findings do not allow us to pick the one correct theory of emotion, but they are more consistent with some theories than others. First, the evidence of a dose association of several aspects of facial expression to the experience of emotions creates difficulties for theories, like Schachter and Singer's, which view the bodily changes during emotions as undifferentiated. Also, the subtlety of some of these relations argues against theories, such as Bem's, which state that culture teaches the distinctions among emotions, because it is unlikely that people learn or teach such relations. It is also unlikely that people are aware of such subtle qualities in their own expressions, so that if the inferential processes proposed by cognitive theories are based on such cues, they must be made outside of awareness. Of course, there are obvious differences among emotions in facial expressions which people could be aware of and use to make conscious inferences about their own feelings. On the other hand, there may be even more subtle differences in facial activity among emotions that cannot be scored with visible measurement. Simple associations cannot prove that facial expressions provide a basis for emotional feelings, but the close association between feelings and subtle, varied aspects of expression as well as gross differences in expression among emotions show that such a relation is possible. Whether this relation is mediated by inferences such as Mandler's automatic cognitions or more directly as proposed by Tomkins is an issue as yet unresolved.

REFERENCES

Allport, F. H. Social psychology. Boston: Houghton Mifflin, 1924.

Basmajian, J. V. Muscles alive. Their functions as revealed by electromyography (4th ed.). Baltimore: Williams & Wilkins, 1978.

Bem, D.J. Self-perception theory. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 6). New York: Academic, 1972.

Birdwhistell, R. L. Kinesics and context. Philadelphia: University of Pennsylvania Press, 1970.

Blurton Jones, N. G. Criteria for use in describing facial expressions of children. Human Biology, 1971, 41, 365-413.

Borod, J. C., & Caron, H. S. Facedness and emotion related to lateral dominance, sex, and expression type. Neuropsychologia, 1980, 18, 237-241.

Brannigan, C. R., &Humphries, D.A. Human non-verbal behavior, a means of communication. In N. Blurton Jones (Ed.), Ethological studies of child behavior. Cambridge: Cambridge University Press, 1972.

Bruner, J. S., & Tagiuri, R. The perception of people. In G. Lindzey (Ed.), Handbook of social psychology. Reading, Mass.: Addison-Wesley, 1954.

Buck, R. Nonverbal behavior and the theory of emotion: The facial feedback hypothesis. Journal of Personality and Social Psychology, 1980, 38, 811-824.

Buck, R., Miller, R. E., & Caul, W. F. Sex, personality, and physiological variables in the communication of affect via facial expression. Journal of Personality and Social Psychology, 1974, 30,587-596.

Burke, P. H. Stereophotogrammetric measurement of normal facial &symmetry in children. Human Biology, 1971, 43, 536-548.

Campbell, R. Asymmetries in interpreting and expressing a posed facial expression. Cortex, 1978, 14, 327-342.

Charlesworth, W. R., & Kreutzer, M. A. Facial expression of infants and children. In P. Ekman (Ed.), Darwin and facial expression: A century of research in review. New York: Academic, 1973.

Chaurasia, B. D., & Goswami, H. K. Functional asymmetry in the face. Acta Anatomy, 91,154-160.

Comer, R., & Rhodewalt, F. Cue utilization in the self-attribution of emotions and attitudes. Personality and Social Psychology Bulletin, 1979, 5, 320-324.

Darwin, C. The expression of the emotions in man and animals. London: Murray,1872.

Ekman, P. Universals and cultural differences in facial expressions of emotion. (Ed.), Nebraska Symposium on Motivation, 1971. Lincoln: University of Nebraska Press.

Ekman, P. Cross-cultural studies of facial expression. In P. Ekman (Ed.), Darwin and facial expression: A century of research in review. New York: Academic, 1973.

Ekman, P. Facial signs: Facts, fantasies, and possibilities. In T. A. Sebeok (Ed.), Sight, sound, and sense. Bloomington: Indiana University Press, 1978.

Ekman, P. About brows: Emotional and conversational signals. In M. von Cranach, W. Lepenies, & D. Ploog (Eds.), Human ethology. Cambridge: Cambridge University Press, 1979.

Ekman, P. Asymmetry in facial expression. Science, 1980, 209, 833-834.

Ekman. P. Methods for measuring facial action. In K. Scherer & P. Ekman (Eds.), Recent methods in nonverbal behavior research. New York: Cambridge University Press, 19

Ekman, P., & Friesen, W. V. Measuring facial movement. Environmental Psychology and Nonverbal Behavior, 1976, 1, 56-75.

Ekman, P., & Friesen, W. V. The facial action coding system. Palo Alto, Calif.: Consulting Psychologists Press, 1978.

Ekman, P., Friesen, W. V., & Ancoli, S. Facial signs of emotional experience Journal of Personality and Social Psychology, 1980, 39, 1125-1134.

Ekman, P., Friesen, W. V., & Ellsworth, P. Emotion in the human face. Elmsf Pergamon, 1972.

Ekman, P., Friesen, W. V., & Ellsworth, P. Research foundations. In P. Ekman (Ed.) Emotion in the human face (2nd ed.). Cambridge: Cambridge University Press, 1982.

Ekman, P., Friesen, W. V., & Tomkins, S. S. Facial Affect Scoring Technique: A field study. Semiotica, 1971, 3(1), 37-58.

Ekman, P., Hager, J. C., &Friesen, W.V. The symmetry of emotional and deliberate actions. Psychophysiology, 1981, 18, 101-106.

Ekman, P., Roper, G., & Hager, J. C. Deliberate facial movement. Child Development, 51, 886-891.

Ekman, P., Sorenson, E. R., & Friesen, W. V. Pan-cultural elements in facial emotion, Science, 1969, 164, 86-88.

Fuller, R.G.C, & Sheehy-Skeffington, A. Effects of group laughter on rehumorous material, a replication and extension. Psychological Reports, 1974, 35,

Gorney, M., & Harries, T. The preoperative and postoperative consideration facial asymmetry. Plastic and Reconstructive Surgery, 1974, 54, 187-191.

Grant, E. C. Human facial expression. Man, 1969, 4, 525-536.

Hager, J. C., & Ekman, P. A methodological criticism of Tourangeau and Ellsworth of facial expression and emotion. Journal of Personality and Social Psychology, 1981,4

Izard, C. E. The face of emotion. New York: Appleton-Century-Crofts, 1971.

Izard, C. E. Differential emotions theory and the facial feedback hypothesis activation: Comments on Tourangeau and Ellsworth's 'The role of facial response in experience of emotion." Journal of Personality and Social Psychology. 1981, 40, 350

Izard, C. E, & Dougherty, L. M. A system for identifying affect expressions by holistic (Affex). Newark: University of Delaware, Instructional Resources Center, 19xx

James, W. What is emotion? Mind, 1834, 9, 188-204.

Kleck, R. E., Vaughan, R. C, Cartwright-Smith, I., Vaughan, K. B., Colby Lanzetta, J. T. Effects of being observed on expressive, subjective, and private responses to painful stimuli. Journal of Personality and Social Psychology, 1976, 34,

Kotsch, W. E., Izard, C. E., & Walker, S. Experimenter manipulated facial patterns and emotion experience. Unpublished manuscript, 1979.

LaBarre, W. The cultural basis of emotions and gestures. Journal of Personality, 1947,16, 1968.

Laird, J.D. Self-attribution of emotion: The effects of expressive behavior on the quality of emotional experience. Journal of Personality and Social Psychology, 1974, 29, 475-486.

Lanzetta, J. T., Cartwright-Smith, J., & Kleck, R. E. Effects of nonverbal dissimulation on emotional experience and autonomic arousal. Journal of Personality and Social Psychology, 1976, 33, 354-370.

Levenson, R. W., & Mades, L.L. Physiological response, facial expression, and trail anxiety: Two methods for improving consistency. Paper presented at the Society for Psychophysical Research, Vancouver, British Columbia, 1980.

Leventhal, H. Emotions: A basic problem for social psychology. In C. Nemeth (Ed.), Social psychology: Classic and contemporary integrations. Chicago: Rand McNally, 1974.

Leventhal, H.,& Mace, W. The effect of laughter on evaluation of a slapstick movie. Journal of Personality, 1970, 30, 16-30.

Lynn, J.G., &Lynn, D.R. Smile and hand dominance in relation to basic modes of adaptation. Journal of Abnormal and Social Psychology, 1943, 38, 250-276.

Mandler, G. Mind and emotion. New York: Wiley, 1975.

Marshall, G. D., & Zimbardo, P. G. Affective consequences of inadequately explained arousal. Journal of Personality and Social Psychology, 1979, 37, 970-988.

Maslach,C. Negative and emotional biasing of unexplained arousal. Journal of Personality and Social Psychology, 1979, 3 7, 953-969.

McArthur, L. Z., Solomon, M. R., & Jaffe, R. H. Weight differences in emotional responsiveness to proprioceptive and pictorial stimuli. Journal of Personality and Social Psychology, 1980, 39, 308-319.

Moscovitch, M., & Olds, J. Asymmetries in spontaneous, facial expressions and their possible relation to hemispheric specialization. Paper presented at the International Neuropsychology Society, The Netherlands, June 1979.

Nelson, C. A. & Horowitz, F. D. Asymmetry in facial expression. Science, 1980, 209, 834.

Notarius, C. I., & Levenson, R. W. Expressive tendencies and physiological response to stress. Journal of Personality and Social Psychology, 1979, 3 7, 1204-1210.

Oster, H. Facial expression and affect development. In M. Lewis & L. Rosenblum (Eds.), The development of affect. New York: Plenum, 1978.

Oster, H., & Ekman, P. Facial behavior in child development. Minnesota Symposium on Child Psychology, 1978, 11, 231-276.

Rhodewalt, F., & Comer, R. Induced-compliance attitude change: Once more with feeling. Journal of Experimental Social Psychology, 1979, 15, 35-47.

Sackeim, H. A., Gur, R. C., & Saucy, M. C. Emotions are expressed more intensely on the left side of the face. Science, 1978, 202, 434-436.

Schachter, S., & Singer, J. Cognitive, social and physiological determinants of emotional state. Psychological Review, 1962, 69, 379-399.

Schwartz, G. E., Fair, P. L., Salt, P., Mandel, M. R., & Klerman, G. L. Facial muscle patterning to affective imagery in depressed and non-depressed subjects. Science, 1976, 192, 489-491.

Spinrad, S. I. Asymmetry in facial expression. Science, 1980, 209, 834.

Steiner, J. E. The gustofacial response: Observation on normal and anencephalic newborn infants. In J. F. Bosma (Ed.), Fourth symposium an oral sensation and perception. Bethesda, Md.: U.S. Department of Health, Education and Welfare, 1973.

Tomkins, S.S. Affect, imagery, consciousness (Vol. 1). The positive affects. New York: Springer, 1962.

Tomkins, S. S. The role of facial response in the experience of emotion. Journal of Personality and Social Psychology, 1981, 40, 35-1-357.

Tomkins, S.S. Affect theory. In P. Ekman (Ed.), Emotion in the human face (2nd ed.). Cambridge: Cambridge University Press, 1982.

Tourangeau, R., & Ellsworth, P.C. The role of facial response in the experience of emotion. Journal of Personality and Social Psychology, 1979, 3 7, 1519-1531.