Researchers from Dartmouth College developed MoodCapture, a smartphone app that uses AI to analyze facial features for early detection of depression.
The stigma associated with mental health is a big reason why people put on a brave face, even when they’re feeling down. This is also why we make more of an effort to smile when we interact with people or take a selfie we plan to post on social media.
When we look down to use our phones, we are mostly unaware that there’s a camera looking back at us. MoodCapture uses a smartphone’s front-facing camera to regularly capture authentic, unguarded facial expressions when a user unlocks their phone. It then uses an AI algorithm to continuously monitor for early signs of depression.
The researchers trained their algorithm using 125,000 photos from 177 participants diagnosed with major depressive disorder.
The photos were captured over 90 days as participants used an app on their phones to answer a brief Patient Health Questionnaire (PHQ-8) three times a day.
When the participant responded to the “I have felt down, depressed, or hopeless,” item, the app captured an image of the person’s face. The patients consented to have their photo taken but didn’t know exactly when it happened.
In addition to the facial features, the AI model was also trained on the phone angle, dominant color, lighting condition, photo location, and background elements present in the photos.
Using machine learning and deep learning, the researchers created a model that learned which nuanced facial expressions and contextual background elements correlated with depression.
The prototype MoodCapture app can reliably detect the onset of depression even before the user knows that they’re at risk.
Millions of people suffer from depression worldwide, with severe cases accounting for 800,000 suicides each year. Early detection of depression could lead to more people accessing appropriate and timely care.
The app functionality is impressive, but the researchers note in their paper that mental health monitoring apps like MoodCapture come with a host of sticky ethical issues.
If my phone sensed that I was feeling down, it might be nice for an AI assistant to give me a pep talk or recommend a healthcare option. I’d want to keep that info private though.
The prospect of being the target of mood-related advertising or Big Brother mental health monitoring by authorities isn’t that far-fetched if the app’s user data was exposed. As on-device AI becomes more achievable, these privacy issues will become easier to manage.
Improved cameras and integrated AI may soon enable our phones to know whether we’re having a good day or not, even if our friends can’t always tell.
