Scientists Are Translating Babies' Cries with Artificial Intelligence

By: Julia Wilde
August 01, 2019

Raising a baby — or even babysitting one — is daunting. They're so small and delicate, yet so hard to understand. Why are they crying? Why are they making that face? What's that smell? For Pete's sake, why are they crying? For anyone who's fretted over an infant's mysterious needs, we have good news: Scientists are getting closer to figuring out what's going on in that little head. Why isn't there an app for that?

Look Who's Talking Now

For babies, crying is the best way to communicate with the outside world. Crying expresses their needs, helps to build the parent-child relationship, and, importantly, communicates information about their health. Understandably, then, every caregiver of a young infant wishes at some point that they could understand those cries, and a team of researchers might be one step closer to creating technology that can do just that. In a recent study published in IEEE/CAA Journal of Automatica Sinica, researchers describe how they used AI to translate baby's cries.

Of course, translating any foreign language requires the help of fluent speakers, so the team enlisted the help of some experienced translators: neonatal nurses. The researchers brought a microphone into a hospital's neonatal intensive care unit, or NICU, and made 48 cry recordings from babies ranging in age from 3 days to 6 months. Nurses and caregivers used context clues and years of experience to identify the reason for each cry.

While all babies sound a little bit different, there were still some reliable patterns. For instance, when a baby is hungry, the suck reflex is involved in the cry, so their tongue is on the roof of their mouth, making the cry sound like a "neh." An "owh" is a half yawn, half cry sound, suggesting that the baby is sleepy. "Heh" is an all-purpose "need" cry; maybe they're cold, uncomfortable, or need a diaper change. Those cries are normal, but when a baby is in pain — from gas, for example — they might make an "eair" sound, which comes from deeper in the belly.

With help from their expert nurses and caregivers, the team categorized cries into five different categories: attention, diaper change needed, hungry, sleepy, and discomfort. The researchers considered the first four categories as normal cries, meaning there's no danger to the infant. The last category, discomfort, was considered abnormal since a cry like this could mean the baby was in pain from an injury, gas, or other more serious problems.

Baby Ex Machina

Then came the algorithm, which used automatic speech recognition to detect specific features and patterns in each of the 48 recordings. It was clear from examining the waveforms of the cries that each category had a specific pattern. When a baby wants their diaper changed, for example, the pattern starts with crying, then silence, then a high-pitched whistling sound as they inhale. When a baby is hungry, their cry is louder and longer, then abruptly stops, followed by a longer period of silence. When a baby is sleepy, their cry starts low and slow, gains momentum, then drops slowly. A distress cry, like when a baby is in pain from a shot, is sudden and loud followed by a period of silence when they hold their breath.

Photo by: Lichuan Liu, Wei Li, Xianwen Wu, Benjamin X. Zhou / Northern Illinois University / The College of New Jersey

Lichuan Liu, Wei Li, Xianwen Wu, Benjamin X. Zhou / Northern Illinois University / The College of New Jersey

Photo by: Lichuan Liu, Wei Li, Xianwen Wu, Benjamin X. Zhou / Northern Illinois University / The College of New Jersey

Lichuan Liu, Wei Li, Xianwen Wu, Benjamin X. Zhou / Northern Illinois University / The College of New Jersey

With their waveforms analyzed and identified, the researchers fed them to an AI to teach it to recognize the difference between normal cries and abnormal ones. After trying several approaches, they eventually trained the system to identify the cries with up to a 76 percent success rate.

They hope that this preliminary research could lead to software that can help medical professionals and caregivers treat infants. While the current research only looked at five cry categories, the researchers hope to add more features that can identify when a baby is cold, hot, or just needs a burp.

While some caregivers get a knack for speaking baby over time, sometimes you just can't figure out what the heck a kid wants. If this technology bears fruit, who knows what the future could hold: The next generation of parents might be using an app to translate their infant's wails. For now, you'll just have to wait — the system is still in its infancy, after all.

This article first appeared on Curiosity.com.

Next Up

5 LGBT Scientists Who Changed the World

Without their scientific accomplishments, the sciences would be very different today.

Scientists Have Discovered Enormous Balloon-Like Structures in the Center of Our Galaxy

There's something really, really big in the middle of our Milky Way galaxy — one of the largest structures ever observed in the region, in fact.

Supercomputers and Artificial Intelligence Create Future Green Industries

Supercomputers and artificial intelligence (AI) are indispensable tools for cooking up the next generation of advanced materials. Advanced computers allow scientists to rapidly design better alloys, chemical catalysts, and plastics using millions of potential candidates. Tomorrow’s high-tech materials are being road-tested this way to cut down human trial and error.

Scientists Have Mapped Where People Feel Emotions in Their Bodies

It turns out that most of us feel our emotions in similar places.

A Russian Scientist Injected Himself With 3.5-Million-Year-Old Bacteria

Did this scientist find the key to eternity frozen in permafrost?

Scientists in China Discover Rare Moon Crystal that Could Power Earth

A rare lunar crystal found on the near side of the moon is giving scientists hope of providing limitless power for the world – forever.

The Perseid Meteor Shower Reaches its Peak

Stargazers rejoice! The annual Perseid meteor shower is upon us. Here's what you need to know...(updated August 11, 2022)

Astronomers May Have Found a Rare “Free-Floating” Black Hole

How do you see a perfectly black object in the middle of a pitch-dark night? It sounds like the start of an annoying riddle, but it’s really the question faced by astronomers when they want to search for black holes.

Pluto and Neptune Swap Places Every 248 Years

The farthest planet in our solar system varies.

Food Coma? Here's Why You Get Sleepy After You Eat

You can reduce the need for nodding off after dinner with a few simple steps.