Scientists Are Translating Babies' Cries with Artificial Intelligence

By: Discovery
August 01, 2019

Raising a baby — or even babysitting one — is daunting. They're so small and delicate, yet so hard to understand. Why are they crying? Why are they making that face? What's that smell? For Pete's sake, why are they crying? For anyone who's fretted over an infant's mysterious needs, we have good news: Scientists are getting closer to figuring out what's going on in that little head. Why isn't there an app for that?

Look Who's Talking Now

For babies, crying is the best way to communicate with the outside world. Crying expresses their needs, helps to build the parent-child relationship, and, importantly, communicates information about their health. Understandably, then, every caregiver of a young infant wishes at some point that they could understand those cries, and a team of researchers might be one step closer to creating technology that can do just that. In a recent study published in IEEE/CAA Journal of Automatica Sinica, researchers describe how they used AI to translate baby's cries.

Of course, translating any foreign language requires the help of fluent speakers, so the team enlisted the help of some experienced translators: neonatal nurses. The researchers brought a microphone into a hospital's neonatal intensive care unit, or NICU, and made 48 cry recordings from babies ranging in age from 3 days to 6 months. Nurses and caregivers used context clues and years of experience to identify the reason for each cry.

While all babies sound a little bit different, there were still some reliable patterns. For instance, when a baby is hungry, the suck reflex is involved in the cry, so their tongue is on the roof of their mouth, making the cry sound like a "neh." An "owh" is a half yawn, half cry sound, suggesting that the baby is sleepy. "Heh" is an all-purpose "need" cry; maybe they're cold, uncomfortable, or need a diaper change. Those cries are normal, but when a baby is in pain — from gas, for example — they might make an "eair" sound, which comes from deeper in the belly.

With help from their expert nurses and caregivers, the team categorized cries into five different categories: attention, diaper change needed, hungry, sleepy, and discomfort. The researchers considered the first four categories as normal cries, meaning there's no danger to the infant. The last category, discomfort, was considered abnormal since a cry like this could mean the baby was in pain from an injury, gas, or other more serious problems.

Baby Ex Machina

Then came the algorithm, which used automatic speech recognition to detect specific features and patterns in each of the 48 recordings. It was clear from examining the waveforms of the cries that each category had a specific pattern. When a baby wants their diaper changed, for example, the pattern starts with crying, then silence, then a high-pitched whistling sound as they inhale. When a baby is hungry, their cry is louder and longer, then abruptly stops, followed by a longer period of silence. When a baby is sleepy, their cry starts low and slow, gains momentum, then drops slowly. A distress cry, like when a baby is in pain from a shot, is sudden and loud followed by a period of silence when they hold their breath.

Photo by: Lichuan Liu, Wei Li, Xianwen Wu, Benjamin X. Zhou / Northern Illinois University / The College of New Jersey

Lichuan Liu, Wei Li, Xianwen Wu, Benjamin X. Zhou / Northern Illinois University / The College of New Jersey

Photo by: Lichuan Liu, Wei Li, Xianwen Wu, Benjamin X. Zhou / Northern Illinois University / The College of New Jersey

Lichuan Liu, Wei Li, Xianwen Wu, Benjamin X. Zhou / Northern Illinois University / The College of New Jersey

With their waveforms analyzed and identified, the researchers fed them to an AI to teach it to recognize the difference between normal cries and abnormal ones. After trying several approaches, they eventually trained the system to identify the cries with up to a 76 percent success rate.

They hope that this preliminary research could lead to software that can help medical professionals and caregivers treat infants. While the current research only looked at five cry categories, the researchers hope to add more features that can identify when a baby is cold, hot, or just needs a burp.

While some caregivers get a knack for speaking baby over time, sometimes you just can't figure out what the heck a kid wants. If this technology bears fruit, who knows what the future could hold: The next generation of parents might be using an app to translate their infant's wails. For now, you'll just have to wait — the system is still in its infancy, after all.

This article first appeared on Curiosity.com.

Next Up

Artificial Intelligence in Medicine: the Rise of the Digital Doctors

As AI technology becomes more sophisticated, we can expect them to be used more often in the world of human medicine and healthcare. But it is possible to create medical AIs that rapidly outperform doctors in certain tasks? Find out all the ways AI is helping the healthcare world.

Miss America Can be a Scientist

Miss Virginia Camille Schrier, a 24-year-old scientist, was crowned Miss America 2020 and inspired young women across the country that Miss America can be a scientist too.

Scientists Have Discovered Enormous Balloon-Like Structures in the Center of Our Galaxy

There's something really, really big in the middle of our Milky Way galaxy — one of the largest structures ever observed in the region, in fact.

5 LGBT Scientists Who Changed the World

Without their scientific accomplishments, the sciences would be very different today.

Scientists Have Decoded the Universal Language of Honey Bees

Learn more about the secret language of bees on Discovery.com.

Scientists Have Pinpointed the Energy Limit of the Human Body

Scientists have pinpointed the energy limit fo the human body--basically, the maximum mileage we can get out of the tank. The takeaway? Pregnancy isn't that different from an ultra marathon. Read on to learn more about how your body burns energy.

Scientists Determined Why Room Temperature is so Important for Sleep

Find out what the optimal sleeping temperatures are on Discovery.com.

Scientists Have Mapped Where People Feel Emotions in Their Bodies

It turns out that most of us feel our emotions in similar places.

When Was There Life on Venus?

What we have is a cosmic whodunit. Venus, the second planet from the sun and considered by the more romantic types as "Earth's twin" and the avatar of love, is dead.

DNA's Building Blocks May Have Their Origins in Outer Space

One of life's building blocks could have originated in outer space. But if this experiment shows how these building blocks actually formed, how exactly did they get to Earth?