Software Could Help Monitor Mental Health through "Selfie" Videos

Feb 9, 2015

Luo's rendering of what an app with this software could potentially look like.
Credit Jiebo Luo / University of Rochester

New software could help you track your mental health using your smartphone.

Researchers at the University of Rochester are finding ways to use video and front-facing cameras to monitor tiny facial cues that may indicate a person's current mood or mental health.

Jiebo Luo is a professor of Computer Science. He heads up a team that developed software designed to pick up on these little facial cues.

"We can read heart rate from the forehead. We can track people's eye movement, get a sense of how fast their eyes are blinking. We can actually look into their pupil to see how dilated it is."

Luo says he's "trained" his software to recognize changing facial movements and interpret them as indicators of one of six different moods.

Psychiatrist Vincent Silenzio is also involved in the project. He's excited about the clinical applications of this research.

"It certainly could augment the capacity of people who are even seasoned clinicians to recognize cues that they may not be picking up."

For example, Silenzio says nurses working in psychiatric intake could use an app as a supplementary tool to help them make assessments in a sometimes chaotic environment. Silenzio says it could also help advance the "telehealth" field, by making up for some of what's lost when doctors and their patients are separated physically.

Luo describes the software as quiet, and unobtrusive, and both researchers stressed their commitment to users' privacy.

Luo says they still have a long way to go before you can expect anything like this to show up in the App Store.