The Science of AI Pareidolia: Can Machines See Faces in Objects Like We Do?

In 1994, Florida jewelry designer Diana Duyser claimed to have seen the Virgin Mary’s face on a grilled cheese sandwich—a discovery she famously auctioned off for $28,000. While the incident made headlines, it also brought attention to an ancient psychological phenomenon known as pareidolia, where people perceive faces or patterns in inanimate objects. But what exactly triggers this tendency, and can machines, with their advancing capabilities, ever experience pareidolia in the same way humans do?
A groundbreaking new study from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) aims to answer these questions, introducing an extensive dataset of 5,000 human-labeled images of illusory faces. This dataset has allowed researchers to dive into the nuances of how humans and artificial intelligence (AI) systems process these phantom faces, uncovering unexpected insights along the way.

The Origins of Face Pareidolia
The study, led by Mark Hamilton, a PhD student at MIT, explores the deeper origins of pareidolia. “Face pareidolia has long fascinated psychologists, but it’s been largely unexplored in the computer vision community,” explains Hamilton. The team’s objective was to bridge that gap and investigate how humans and AI detect faces in places they don’t technically exist, such as clouds, electrical sockets, or even grilled cheese sandwiches.

AI's Struggle to Recognize Pareidolic Faces
One of the key findings was that AI models, at first, didn’t recognize pareidolic faces in the same way humans do. While humans can instantly “see” a face in an everyday object, like a car’s headlights or a pattern on the floor, machines require more complex training. The research showed that it wasn’t until the algorithms were fine-tuned to detect animal faces that they became significantly better at identifying pareidolic faces.
This surprising link between recognizing animal faces and perceiving illusory faces points to a possible evolutionary origin of pareidolia. Spotting faces, particularly those of predators or prey, may have been a survival mechanism for our distant ancestors. "This result suggests that pareidolia might not arise from human social behavior, but from something deeper: like quickly spotting a lurking tiger or identifying which way a deer is looking so our primordial ancestors could hunt,” says Hamilton.

The Goldilocks Zone of Pareidolia
Another significant discovery from the MIT study is the identification of the “Goldilocks Zone of Pareidolia.” This term refers to a specific range of visual complexity where both humans and machines are most likely to perceive faces in non-face objects. According to William T. Freeman, MIT professor and principal investigator of the project, if an image is too simple, there’s not enough detail to form a face, but if it’s too complex, it becomes visual noise.
To model this phenomenon, the researchers developed a formula that predicts where pareidolia is most likely to occur. They found that the likelihood of detecting faces peaks within a particular range of complexity, which they validated through tests with both human subjects and AI systems. This “pareidolic peak” highlights the delicate balance between simplicity and complexity required for our brains—and now machines—to perceive illusory faces.

Building the Largest Pareidolic Dataset
A key component of the study was the creation of a dataset that dwarfs previous collections of pareidolic stimuli. The CSAIL team curated around 20,000 candidate images from the LAION-5B dataset, meticulously labeling and judging them based on human perceptions of faces. Each image was evaluated on various factors, such as the emotion the face evoked, its age, and whether the face seemed accidental or intentional. This process, which involved hours of human annotation, was essential to building a resource that could advance research in both AI and human cognition.
Hamilton humorously acknowledges the personal effort that went into this monumental task: “Much of the dataset owes its existence to my mom,” he shares, referring to his mother, a retired banker, who helped label the images.

Applications Beyond Pareidolia: From Face Detection to Product Design
While the study focuses on the peculiar phenomenon of pareidolia, its implications reach far beyond this. The insights gleaned from the research could significantly improve AI-based face detection systems, reducing false positives. This has practical applications in areas like self-driving cars, where false face detection could lead to dangerous mistakes, and in fields like robotics and human-computer interaction, where recognizing or avoiding pareidolia could make interactions with machines smoother.
Moreover, the findings have potential applications in product design. Understanding and controlling pareidolia could help designers create products that appear friendlier and less intimidating. “Imagine being able to automatically tweak the design of a car or a child’s toy so it looks friendlier, or ensuring a medical device doesn’t inadvertently appear threatening,” says Hamilton.

The Human-Machine Perception Gap
The study also raises intriguing questions about the difference between human and machine perception. Humans instinctively interpret inanimate objects with human-like traits, while AI algorithms do not. For instance, people might perceive an electrical socket as “singing,” and even imagine its “moving lips,” but AI doesn’t “see” these cartoonish faces.
“What accounts for this difference between human perception and algorithmic interpretation? Is pareidolia beneficial or detrimental?” asks Hamilton. These questions, along with many others, form the basis of the team's ongoing investigations into this classic psychological phenomenon.

Looking Ahead: Toward Human-Like AI Systems
As the CSAIL team prepares to share their dataset with the broader scientific community, they are already looking to the future. Potential next steps include training AI systems to not only detect pareidolic faces but to understand and describe them in more human-like ways. Vision-language models, for example, could be designed to engage with visual stimuli in ways that align more closely with human experiences.
“This is a delightful paper! It is fun to read and it makes me think,” comments Pietro Perona, a professor at Caltech, who was not involved in the study. He adds, “Hamilton et al. propose a tantalizing question: Why do we see faces in things?”
Ultimately, the study may not only enhance our understanding of pareidolia but also shed light on the broader mechanisms of human and machine perception. Supported by the National Science Foundation, the United States Air Force Research Laboratory, and the United States Air Force Artificial Intelligence Accelerator, this research paves the way for creating AI systems that better grasp human-like experiences and interactions.
As our world continues to be shaped by AI, understanding the gap between human intuition and algorithmic logic is crucial. By examining a phenomenon as whimsical as pareidolia, MIT researchers have opened the door to a deeper understanding of how machines can better “see” the world—and possibly one day share in the wonder of a face in a grilled cheese sandwich.