Video Chat Algorithms Can Now Detect Your Heart Rate and Stress Levels

It’s been about a month since the 2020 Las Vegas Consumer Electronics Show (CES) wrapped up. But the time passed hasn’t stopped experts from still talking about some seriously cool products introduced at the event.

At this year’s convention, flying taxi rides, foldable cell phones, and new-age headphones were some of the main highlights. But one particular product stole the spotlight from others: Medical app development company Binah.ai’s video chat AI product is shaking up the world of telehealth and wearables.

Binah.ai set up their booth at CES to calculate the stress levels of passersby using video to measure vital signals like heart rate, respiration, and more. Binah.ai’s ultimate goal is to help doctors perform better telehealth by measuring their patients’ vitals remotely.

How Binah.ai’s Technology Works

The video chat currently measures heart rate, respiration, stress level, heart rate variability, and oxygen saturation. Eventually, the company says, the user’s blood pressure will also be measured through video.

How in the world is it possible to measure this suite of vitals through a video chat? You might be wondering if Binah.ai offers a wearable or physical product to accompany its AI tool. In reality, the company uses a technology called plethysmography, which monitors the user’s facial coloring to find slight changes that it can calculate pulse from.

The other vitals are measured with proprietary technology which is in patent review, so Binah.ai has declined to comment on those. But experts do know that plethysmography can be used to measure breathing and oxygen levels.

Plethysmography isn’t a new technology or theory; it’s been studied by researchers for the past decade, and other AI development companies have been using it for a relatively new field called “affective computing”. This field uses AI to measure a person’s emotional state for diagnosis and for computers to interact accordingly.

Although Binah.ai’s technology has no published studies investigating its accuracy or efficacy, the company says clients have tested it against other measurement tools, and the technology performs with accuracy. In the past, studies have shown that when there is controlled light and faces are clear of hair, piercings, and glasses, the best results are achieved.

Binah.ai’s Applications

Sompo, a Japanese insurance company, is using Binah.ai’s technology in an app that drivers employ to measure their stress levels while in transit. Although this application has seen a lot of success, Binah.ai’s main goal is to become a staple in healthcare.

Mona Popilian-Yona is a spokesperson for Binah.ai. She says the company imagines a world where a patient lines up at a kiosk while they’re in the waiting room to have vitals measured “without being touched at all.” This can obviously save hospitals countless hours of vital-taking, and it can bridge the gap in telehealth where a doctor doesn’t feel as close to the patient through the video because they can’t touch the patient when needed.

The U.S.’s Food and Drug Administration is currently reviewing Binah.ai’s technology, and the company is waiting for approval to launch their products and technology for use by other companies. Binah.ai says other healthcare organizations are currently testing their technology’s algorithms.

In March, the company plans to launch a mobile app developed for consumers to test their own stress levels and vital signs. The app and its features won’t be free, though; Popilian-Yona says that the company hasn’t figured out their pricing model yet. As far as data privacy goes, Binah.ai said customers will have the option to disallow the app from storing any personal data. If the user wants to see their data analyzed and receive insights from the app, the mobile app will only store the vitals data but not any image or video data.

Another major application that Binah.ai could be used for in the future is law enforcement. Police believe that this technology can be invaluable during interrogations. But this is a major cause for concern for many consumers because police in recent times have been shown to abuse their power and tools as a means to their desired end.

Currently, there are no laws regarding the use of this type of technology in law enforcement. Like polygraph tests, police must ask their subject for permission to use facial scanning software on the subject.

Still, says Popilian-Yona, Binah.ai’s end-game is in assisting providers and healthcare organizations, not police officers and law enforcement agencies: “We want to put this in the hands of the healthcare providers.”

Another Key Player In The Game

Other companies have used similar technology to develop their products. For example, last year, Moscow-based Neurodata Lab released its heart-rate detection video technology online as a demo. Within the next several months, the company wants to make the technology publicly available through a medical mobile app.

A spokesperson from Neurodata Lab said that the pulse detection technology from the company is as accurate as the pulse measurements from fitness trackers and smartwatches on the current market. She said that using pulse and other physiological data, AI can deduce if the person is anxious, stressed, or experiencing other emotions.

A Murky Future

There is no precedent for this type of technology. It generates HIPAA-level data outside of healthcare institutions, making the laws and regulations unclear and vague. We don’t know how this data should be protected or handled. For Binah.ai and Neurodata Lab, liabilities like this are transferred to their clients and customers.

It’s not certain when the FDA will approve Binah.ai’s technology, or if consumers will enjoy their mobile app, but the growing interest in this technology points to a strong need to begin discussing regulations and laws around use by law enforcement and governments, HIPAA protection, and patient privacy.

What do you make of Binah.ai’s innovation? Would you download this if it was available in the App Store? Let us know your thoughts in the comments below!