Scientists Are Translating Babies' Cries with Artificial Intelligence

By: Julia Wilde
August 01, 2019

Raising a baby — or even babysitting one — is daunting. They're so small and delicate, yet so hard to understand. Why are they crying? Why are they making that face? What's that smell? For Pete's sake, why are they crying? For anyone who's fretted over an infant's mysterious needs, we have good news: Scientists are getting closer to figuring out what's going on in that little head. Why isn't there an app for that?

Look Who's Talking Now

For babies, crying is the best way to communicate with the outside world. Crying expresses their needs, helps to build the parent-child relationship, and, importantly, communicates information about their health. Understandably, then, every caregiver of a young infant wishes at some point that they could understand those cries, and a team of researchers might be one step closer to creating technology that can do just that. In a recent study published in IEEE/CAA Journal of Automatica Sinica, researchers describe how they used AI to translate baby's cries.

Of course, translating any foreign language requires the help of fluent speakers, so the team enlisted the help of some experienced translators: neonatal nurses. The researchers brought a microphone into a hospital's neonatal intensive care unit, or NICU, and made 48 cry recordings from babies ranging in age from 3 days to 6 months. Nurses and caregivers used context clues and years of experience to identify the reason for each cry.

While all babies sound a little bit different, there were still some reliable patterns. For instance, when a baby is hungry, the suck reflex is involved in the cry, so their tongue is on the roof of their mouth, making the cry sound like a "neh." An "owh" is a half yawn, half cry sound, suggesting that the baby is sleepy. "Heh" is an all-purpose "need" cry; maybe they're cold, uncomfortable, or need a diaper change. Those cries are normal, but when a baby is in pain — from gas, for example — they might make an "eair" sound, which comes from deeper in the belly.

With help from their expert nurses and caregivers, the team categorized cries into five different categories: attention, diaper change needed, hungry, sleepy, and discomfort. The researchers considered the first four categories as normal cries, meaning there's no danger to the infant. The last category, discomfort, was considered abnormal since a cry like this could mean the baby was in pain from an injury, gas, or other more serious problems.

Baby Ex Machina

Then came the algorithm, which used automatic speech recognition to detect specific features and patterns in each of the 48 recordings. It was clear from examining the waveforms of the cries that each category had a specific pattern. When a baby wants their diaper changed, for example, the pattern starts with crying, then silence, then a high-pitched whistling sound as they inhale. When a baby is hungry, their cry is louder and longer, then abruptly stops, followed by a longer period of silence. When a baby is sleepy, their cry starts low and slow, gains momentum, then drops slowly. A distress cry, like when a baby is in pain from a shot, is sudden and loud followed by a period of silence when they hold their breath.

Photo by: Lichuan Liu, Wei Li, Xianwen Wu, Benjamin X. Zhou / Northern Illinois University / The College of New Jersey

Lichuan Liu, Wei Li, Xianwen Wu, Benjamin X. Zhou / Northern Illinois University / The College of New Jersey

Photo by: Lichuan Liu, Wei Li, Xianwen Wu, Benjamin X. Zhou / Northern Illinois University / The College of New Jersey

Lichuan Liu, Wei Li, Xianwen Wu, Benjamin X. Zhou / Northern Illinois University / The College of New Jersey

With their waveforms analyzed and identified, the researchers fed them to an AI to teach it to recognize the difference between normal cries and abnormal ones. After trying several approaches, they eventually trained the system to identify the cries with up to a 76 percent success rate.

They hope that this preliminary research could lead to software that can help medical professionals and caregivers treat infants. While the current research only looked at five cry categories, the researchers hope to add more features that can identify when a baby is cold, hot, or just needs a burp.

While some caregivers get a knack for speaking baby over time, sometimes you just can't figure out what the heck a kid wants. If this technology bears fruit, who knows what the future could hold: The next generation of parents might be using an app to translate their infant's wails. For now, you'll just have to wait — the system is still in its infancy, after all.

This article first appeared on Curiosity.com.

Next Up

Scientists in China Discover Rare Moon Crystal that Could Power Earth

A rare lunar crystal found on the near side of the moon is giving scientists hope of providing limitless power for the world – forever.

How 3D Print Building is Changing the Future

Building with 3D printing technology is sparking widespread interest in the construction industry. Besides reducing waste and our impact on the environment, it can speed up construction from weeks, or months, to days. Projects that use simple raw materials like soil, straw, and even salt, can be built in a fraction of the time and cost of traditional construction.

How to Save Humanity from Extinction

Here are some goals we need to achieve if we want to reach our 500,000th birthday as a species.

Quiz: Test Your Space Exploration Knowledge

Ahead of the historic May 27th NASA and SpaceX crewed space launch, test your space exploration knowledge!

This Country is Transforming an Abandoned Airport into a Green Oasis

Most old, disused airports are torn down to make way for shiny new developments, although at a huge cost and via a lengthy process.Not Tegel airport, in Berlin, Germany.

Extreme Weather Tests the Durability of Solar and Wind Power

As category four Hurricane Ian swept across the Caribbean into south west Florida on 28 September 2022, knocking out Cuba’s electricity grid along the way, hundreds of thousands of homes were hit by flooding and power loss. In contrast, the solar-powered community of Babcock Ranch 24 miles to the north of coastal town Fort Myers survived intact.

Miss America Can be a Scientist

Miss Virginia Camille Schrier, a 24-year-old scientist, was crowned Miss America 2020 and inspired young women across the country that Miss America can be a scientist too.

5 LGBT Scientists Who Changed the World

Without their scientific accomplishments, the sciences would be very different today.

Scientists Have Discovered Enormous Balloon-Like Structures in the Center of Our Galaxy

There's something really, really big in the middle of our Milky Way galaxy — one of the largest structures ever observed in the region, in fact.

Scientists Have Decoded the Universal Language of Honey Bees

Learn more about the secret language of bees on Discovery.com.