Uncanny valley

• 1. Introduction

  ›

  The uncanny valley is a hypothesis about unease, and in 2020 it helped decide the release date of the movie Sonic the Hedgehog. The first trailer drew an extremely negative audience reaction. The film was then delayed for three months so that its title character could look less human and more cartoonish.

  That episode raises an odd question. Why would a character become less appealing as it grows more lifelike? Who first described this reversal, and what exactly did he claim? Scientists have proposed explanations ranging from fear of death to the avoidance of disease. Some of them doubt that a single phenomenon exists at all. The evidence involves a robotics professor, brain scanners, monkeys shown computer-generated faces, and a computer-animated baby.

  8112

• 2. The Curve Mori Drew

  ›

  In 1970, the robotics professor Masahiro Mori introduced the concept in a book titled Bukimi No Tani. He phrased it as bukimi no tani genshō. His claim concerned how observers feel about robots as the machines grow more humanlike.

  At first, each step toward a human look makes some observers more positive and more empathetic. Then the robot becomes almost human, and the response abruptly turns into strong revulsion. If the robot's appearance keeps improving until it is hard to tell from a person, the feeling recovers. It climbs back toward the empathy people feel for one another.

  Drawn on a graph, the reaction falls steeply and then rises steeply, in the zone where anthropomorphism comes closest to reality. That dip is the valley. It marks robots whose look and motion sit between somewhat human and fully human. To some people such a machine seems overly strange. The strangeness blocks the empathy that productive interaction between humans and robots would need.

  The English name arrived later. In 1978, Jasia Reichardt's book Robots: Fact, Fiction, and Prediction translated bukimi no tani literally as uncanny valley. That wording had an unintended effect. It tied Mori's idea to Ernst Jentsch's psychoanalytic concept of the uncanny, set out in his 1906 essay "On the Psychology of the Uncanny". The German title was Zur Psychologie des Unheimlichen.

  Sigmund Freud later critiqued and extended Jentsch's concept in his 1919 essay "The Uncanny", which carried the German title Das Unheimliche. Today the argument has moved well beyond robots. Virtual reality, augmented reality and photorealistic computer animation keep growing more convincing, and each advance reopens the debate over the valley.

  12357

• 3. Theories of Unease

  ›

  Researchers have proposed many explanations for the valley, and the source stresses that support for each one is inconsistent. The first concerns anomalies. An entity that looks plainly nonhuman makes its few human features stand out, and that produces empathy. An almost human entity instead calls up our detailed model of a person, so its nonhuman features stand out as wrong.

  Under this view, a robot in the valley is not judged as a machine pretending well. It is judged as an abnormal human doing a bad job of seeming normal. The effect appears when something that looks mechanical moves like a person. It also appears in the reverse case, when something that looks human moves mechanically. This idea has been linked to perceptual uncertainty and to the theory of predictive coding.

  A second explanation is category error. When an entity is ambiguous, a person may sort it slowly or wrongly. Names for this include category failure, category conflict and categorization difficulty. The label category confusion also circulates, though research backing for it is weaker. Some scholars connect the idea to the Sorites paradox. Stimuli with both human and nonhuman traits link two very different categories through a single measure, the degree of human likeness.

  A third explanation points to conflicting perceptual cues. Two clashing mental representations are activated together, as when a humanoid figure moves like a robot. The tension is felt as discomfort, much like the discomfort of cognitive dissonance. Burleigh and colleagues found that faces midway between human and nonhuman produced more eeriness than a straight-line model would predict.

  Yamada and colleagues linked cognitive difficulty to negative feeling at the midpoint of a morph between a cartoon dog and a real dog. Ferrey and colleagues found the strongest negative feeling at the midpoint between two categories, for human and nonhuman entities alike. Schoenherr and Burleigh offered a cultural parallel, the aversion to genetically modified organisms nicknamed "Frankenfoods". Moore built a Bayesian mathematical model of the conflict. Debate continues over whether categorization difficulty, configural processing, perceptual mismatch, frequency-based sensitization or inhibitory devaluation does the real work.

  Other theories reach for fear. One holds that a disassembled android reminds viewers of death. Its argument has four parts. A human façade over a mechanical interior hints that we may all be soulless machines. Androids in states of mutilation, decapitation or disassembly resemble a battlefield after a conflict. Because most androids copy real people, they are doppelgängers, and they may stir a fear of being replaced at work or in a relationship. Finally, jerky android movement may evoke the fear of losing control of one's own body.

  Biology supplies two more. The mate selection theory says that uncanny stimuli trigger an evolved mechanism for avoiding partners with low fertility, poor hormonal health or weak immune systems. Such traits can show in the face and body. The pathogen avoidance theory says that disgust evolved to keep us away from sources of infection. Its authors argue that the more human an organism looks, the stronger the aversion to its defects. Their reasoning has three parts. Defects signal disease. More human-looking organisms are genetically closer to us. And the chance of catching a disease rises with genetic similarity. On this account, the oddities of androids provoke alarm and revulsion much as corpses and visibly diseased people do.

  Identity offers a final cluster. Kaplan argued that very humanlike machines challenge human uniqueness and push for a redefinition of humanness. Ferrari, Paladino and Jetten found that a more anthropomorphic robot raises the sense of threat to human distinctiveness. The more a robot resembles a real person, the more it challenges our social identity as humans.

  The psychiatrist Irvin Yalom supplies a related framework. Humans, he explains, build defenses against the anxiety that death causes. One defense is specialness, the irrational belief that aging and death apply to everyone except oneself. A compelling, humanlike robot may shake that belief and bring the anxiety back. Folklore carries a similar warning. In Judaism the golem, a humanlike creature without a soul, shows how good intentions can still end in disaster.

  Cognitive scientists have since proposed an uncanny valley of mind. Rapid advances in artificial intelligence and affective computing mean people may feel strong aversion toward highly advanced, emotion-sensitive technology. Contemporary research weighs a perceived loss of human uniqueness against an expectation of immediate physical harm.

  916212729

• 4. Faces on Trial

  ›

  Experiments have tried to test these ideas with real faces. Mathur and Reichling used two sets of stimuli running from very mechanical to very human-like. One was a sample of 80 robot face images chosen objectively from Internet searches. The other was a controlled series of six faces built with careful morphing and graphics work.

  Subjects rated how likable each face was. To gauge trust indirectly, they also played an investment game in which they wagered money on a robot's trustworthiness. Both sets of faces produced a robust uncanny valley in explicit likability ratings. The effect on implicit trust was more dependent on context. Perceptual confusion at the category boundary did occur inside the valley, but it did not account for the social and emotional reactions.

  Eyes have emerged as a prime suspect. In 2015, Schein and Gray found that human faces without eyes felt more uncanny than faces without a nose, or faces left unchanged. They also noted that people with autism, who often show abnormalities in eye contact and other social skills, may be less susceptible to the valley.

  Animals have been tested too. A study from 2009 showed five monkeys three images: a real monkey photo, a realistic 3D monkey face and an unrealistic 3D monkey face. Eye gaze served as a measure of preference. The monkeys looked least at the realistic 3D face, which the researchers read as aversion. They concluded that neither human-specific thinking nor human culture can explain the valley, and that its origin is evolutionary.

  A later study complicates that picture. Carp and colleagues, in 2022, found little to support the valley in rhesus monkeys. The monkeys did prefer to look at the eyes over the mouth in realistic stimuli. When a stronger signal such as bared teeth was available, however, they did not mainly focus on the eyes.

  By 2011, researchers at the University of California, San Diego were measuring brain activity linked to the effect. The California Institute for Telecommunications and Information Technology was also involved. An fMRI study by cognitive scientists and roboticists found the largest differences for uncanny robots in the parietal cortex, on both sides of the brain. These areas connect the visual region that processes bodily movement with the part of the motor cortex thought to hold mirror neurons. The researchers described evidence of mismatch, with the brain lighting up when an android's human look and robotic motion did not compute.

  Ayşe Pınar Saygın, an assistant professor at UCSD, stated: "The brain doesn't seem selectively tuned to either biological appearance or biological motion per se." She continued: "What it seems to be doing is looking for its expectations to be met , for appearance and motion to be congruent."

  1418343653

• 5. Screen Verdicts

  ›

  Roboticist Dario Floreano points to a baby named Billy as the start of it all. Billy starred in Pixar's 1988 animated short Tin Toy. Negative audience reactions to him first made the movie industry take the uncanny valley seriously.

  The 2001 movie Final Fantasy: The Spirits Within, one of the first photorealistic computer-animated features, drew similar reactions. Peter Bradshaw of The Guardian called the animation brilliant. Yet the "solemnly realist human faces look shriekingly phoney precisely because they're almost there but not quite", he wrote. Peter Travers of Rolling Stone found the characters fun at first. Then, he wrote, you notice "a coldness in the eyes, a mechanical quality in the movements".

  Reviewers of the 2004 film The Polar Express reached for the word eerie, including Kurt Loder and Manohla Dargis. Paul Clinton of CNN said the human characters come across as downright creepy. John Anderson of Newsday called them dead-eyed and wrote that "The Polar Express is a zombie train". The animation director Ward Jenkins published an online analysis. He argued that changes to the characters' eyes and eyebrows could have avoided the deadness he saw.

  Critics split over the 2011 film The Adventures of Tintin: The Secret of the Unicorn. Daniel D. Snyder of The Atlantic complained that motion capture gave Tintin an alien visage with pores and wrinkles, and that Spielberg had made the characters dead. N.B. of The Economist called parts of it grotesque and likened it to The Invasion of the Body Snatchers. Dana Stevens of Slate felt the cast narrowly escaped the valley. Kevin Kelly of Wired went further, saying we had passed beyond it into the plains of hyperreality.

  The 2019 remake of The Lion King, with photo-realistic digital animals in place of the 1994 film's traditional animation, divided critics in the same way. Ann Hornaday of The Washington Post suggested that 2019 might be remembered as the summer we left the Uncanny Valley for good. Other critics found the singing and dancing scenes disturbing and weird because the animals looked so real. Multiple commentators cited the CGI half-human, half-cat characters of the 2019 movie Cats, first after its trailer and again after its release.

  Deepfake technology has carried the effect into broader popular culture, producing the eerie discomfort that Mori described.

  Virtual actors are a related practice. Filmmakers use CGI likenesses of real actors who look too old for a part or have died. Sometimes the original actor contributes motion capture or audio, and sometimes the actor has no involvement. Reviewers often criticize the results as eerie. The examples include Arnold Schwarzenegger in Terminator Salvation in 2009 and Terminator Genisys in 2015, and Jeff Bridges in Tron: Legacy in 2010. Others are Peter Cushing and Carrie Fisher in Rogue One in 2016, and Will Smith in Gemini Man in 2019.

  6768697478

• 6. Designing Around the Valley

  ›

  Voice offers the simplest lesson in matching. A robot with a synthetic voice is less eerie than one with a human voice. A human with a human voice is less eerie than one with a synthetic voice. Mixed signals are the problem. The same rule applies to behavior, since an animated character that looks more human than it moves leaves a negative impression. Neuroimaging studies point the same way, showing that matching appearance and motion matters.

  Expectation is a second lever. If a robot looks like an appliance, people expect little from it. If it looks very human, they expect too much. A humanlike appearance implies humanlike motion dynamics, and the expectation may work below conscious awareness. Neuroscientists note that when expectations fail, the brain generates a prediction error. They add that perceptual systems may retune as artificial agents become common. Or, in their words, we may decide "it is not a good idea to make robots so clearly in our image after all".

  For computer-generated characters, texture and proportion must agree. A photorealistic human skin demands human facial proportions. Departures from them, including the larger eyes artists often use to enhance attractiveness, can look eerie under photorealistic texture.

  Angela Tinwell and colleagues have studied this in video games and movies. Their 2011 work examined how viewers perceive facial expression and speech in realistic human-like characters. A 2010 paper considered how the uncanny might be exaggerated for antipathetic characters in survival horror games. Using 3D characters built in a real-time gaming engine, the team aimed to map the valley. They also tested how facial expression and speech together can intensify it. Tinwell's team also proposed an unscalable uncanny wall. Viewers' ability to spot imperfections, the idea goes, will keep pace with every new technology for simulating realism. Her book The Uncanny Valley in Games and Animation, published by CRC Press, summarizes the work and advice for designers.

  David Hanson reports a way out. He showed that the effect could be eliminated by adding neotenous, cartoonish features to entities that had previously caused it. The method draws on the appeal of features that recall the young of our own species and of many others, as cartoons have long exploited.

  414243506

• 7. Doubts About the Valley

  ›

  Not everyone accepts that the valley is a single thing. One criticism says the effect is really a heterogeneous group of phenomena. They can involve different senses and have several overlapping causes. Cultural heritage may also shape how people perceive androids.

  A second criticism is generational. Younger people, raised on computer-generated imagery and robots, may be less affected by the hypothesized problem.

  Cheetham and colleagues offer a third view, that the valley is an ordinary product of categorization, with a category boundary defining the valley. Burleigh and Schoenherr went further. They split the effects into those due to the category boundary and those due to how often individual examples are seen, much like the mere-exposure effect. By varying how often training items appeared, they separated cognitive uncertainty from affective uncertainty. In a follow-up, Schoenherr and Burleigh found that instructions changed categorization accuracy but not ratings of negative feeling. Generational and cultural effects, they argue, can be explained by basic information processing. The valley may therefore reflect greater familiarity with human categories and nothing unique.

  A fourth criticism comes from Hanson, who says uncanny entities can appear anywhere on the human-likeness spectrum. His examples run from MIT's abstract robot Lazlo to the perfectly human, such as cosmetically atypical people. Capgras delusion supports the broader point. Patients with this rare condition believe that people, or sometimes things, have been replaced by duplicates. They accept that the duplicates are physically identical but hold that the true entity is gone. Some say the duplicate is a robot.

  Ellis and Lewis attribute the delusion to an intact system for overt recognition combined with a damaged system for covert recognition. The result is a person who is identifiable but not emotionally familiar. That pattern supports the view that the valley may come from how the brain handles categorical perception.

  The last criticism targets method. It holds that the valley appears only because researchers carefully select their images. No single physical dimension of an image produces the curve. Instead, different categories of images each have a different straight-line relationship between human likeness and uncanniness.

  1923552022

• 8. Echoes Beyond Robots

  ›

  In 1839, Charles Darwin described a snake he judged hideous and fierce, with a vertical slit pupil in a mottled and coppery iris. He wrote that he could think of nothing uglier except perhaps some vampire bats. His explanation was that the features sit in positions roughly proportional to those on a human face, giving "a scale of hideousness". If the valley comes from general cognitive processes, observations like this one would be expected in evolutionary history.

  The ethical-futurist writer Jamais Cascio sees a possible valley ahead in transhuman enhancement. Such enhancements might improve eyesight, muscle strength or cognition beyond normal limits. While they stay within a perceived norm, a negative reaction is unlikely. Once individuals go beyond normal human variety, he suggests, revulsion can be expected. Further along, the transhuman would be judged as separate entities altogether, a point called posthuman, and acceptance would rise again.

  Retouched pageant photographs offer a more everyday example. Pictures of children smoothed in this way strike some people as disturbingly doll-like.

  In 2022, Diel and Lewis applied the idea to liminal spaces, also called the backrooms. This internet sensation describes unease at strange, eerie or anomalous built environments. Their study found that atypical features describe the uncanniness of such places better than category confusion from ambiguity. They concluded that deviating architectural features and social presence help determine how uncanny a structure feels.

  585960616