Intelligent image-based physiological information measurement technology with face recognition and emotion recognition functions
Dr. Chuan-Yu Chang is a Distinguished Professor of YunTech, he is currently the Deputy General Director of Service Systems Technology Center, ITRI. His research interests include computational intelligence, medical image processing, AOI, and infant crying recognition. He is an IET Fellow and the President of the Chinese Image Processing and Pattern Recognition Society.
Academy/University/OrganizationIndustrial Technology Research Institute
Share this article
You are free to share this article under the Attribution 4.0 International license
With the change in global population structure and increased frequency of epidemic and pandemic outbreaks, the application of digital technology is crucial to epidemic-prevention measures. In the COVID-19 outbreak, the number of confirmed cases has exceeded 106 million around the world, with the death toll reaching 2.31 million. However, due to the government of Taiwan preparing in advance, only 920 confirmed cases and 9 deaths had been reported in Taiwan as of 7th February 2021. This result has been affirmed by various countries. Frontline medical personnel stick to their positions, face the COVID-19 and significant stress every day, and are also exposed to high risk environments in the long term. For all the frontline medical personnel, Industrial Technology Research Institute (ITRI), and Taipei Medical University Hospital (TMUH) are working together on some epidemic-prevention measures. ITRI and TMUH jointly developed a contactless remote monitoring system－The Contactless Healthcare Connected IoT Platform. The IoT Gateway is used to integrate many technologies, providing patients with contactless, round-the-clock, real-time monitoring of their vital signs to achieve the goal of “Contactless and Considerate.” Pandemic prevention by technology can reduce infection risks during patient care.
With the global increased frequency of epidemic and pandemic outbreaks, medical personnel check on their patients 12-15 times a day, and so are exposed to a high risk environment over the long term, and endure high levels of stress. Also, it takes lot of time to put on and take off the protective gear, so entering an infectious patient’s room takes about 40 minutes. Therefore, the application of digital technology to achieve improvement in efficiency is crucial for medical system epidemic prevention. Industrial Technology Research Institute (ITRI) and Taipei Medical University Hospital (TMUH) are working together on some epidemic-prevention measures. ITRI and TMUH jointly developed a contactless remote monitoring system－The Contactless Healthcare Connected IoT Platform (see Figure 1). Using the IoT Gateway to integrate many technologies, it provides patients with contactless, round-the-clock, real-time monitoring of their vital signs to achieve the goal of “Contactless and Considerate.” Pandemic prevention by technology can reduce infection risks during patient care.
The Contactless Healthcare Connected IoT Platform consists of an all-in-one (AIO) vital signs detector, IoT Gateway and Nurse Station Dashboard. There are three benefits. First, using the contactless platform reduces infection risks during patient care. Second, 24-hour distant interactive treatment improves efficiency. Third, the treatment integrated situation room facilitates knowing the patients’ vital signs, such as heart rate, respiration rate, facial skin temperature, and bed-exit detection. It may reduce the burden of medical personnel and become a helper during the pandemic prevention period.
The AIO vital signs detector combines many kinds of sensors, has embedded facial recognition detection by AI algorithm, and remotely collects a patient's facial skin temperature, heart rate, and respiration rate (see Figure 2). The IoT Gateway follows HL7 FHIR standards, and receives transmissions from the AIO vital signs detector (see Figure 3). The device also supports MQTT protocol which is in charge of dispatching all messages between the devices and backend software. The IoT Gateway transform by internet gathers and transfers the patient’s vital signs to the Nurse Station Dashboard (see Figure 4), and allows for faster interpretation and treatment. The Nurse Station Dashboard can display real-time patient physiological readings with their summarized EMR data, including lab test results, images and reports, medication history, and more. With the Nurse Station Dashboard, quarantined patients can easily consult with their doctors and medical care team using video-conferencing during the course of treatment. If the system detects any emergency situation, the nurse station can also use this system to interact with and provide consultation to patients immediately. Medical personnel can know the patient’s situation well without entering the room, dramatically reducing the medical resources costs and the risk of infection. All medical teams can also use video conferencing to share the cases. Using technology to break the limited amount of space and time also shortens the distance between medical personnel and patients, exploring and accelerating the nursing ability.
In the future, contactless medical equipment will play a key role in preventing highly contagious diseases, as the demand to keep monitoring and collecting patients’ vital signs will be increasingly urgent. The Contactless Healthcare Connected IoT Platform won the National Industrial Innovation Award in 2020 as a simply embedded system which can measure every vital sign by AI. Not only can it improve the medical data of Taiwan, but it also reduces the risk of infection of medical personnel. To get to the realm of “Contactless and Considerate,” the technology can also be applied to negative pressure isolation ward, distance patient nursing, day care centers, long-term care centers, and home or remote districts with a lack of resources. With the power of technology, we can balance pandemic prevention and safety, and also protect the frontline medical personnel.
Figure 1: The vital signs data gathered by the IoT Gateway
Figure 2: The all-in-one vital signs detector equipment collects a patient's facial skin temperature, heart rate, and respiration rate in Taipei Medical University Hospital
Figure 3: The system framework of the Generic IoT Gateway
Figure 4: Patient’s vital signs fully presented on the Nurse Station Dashboard
STAY CONNECTED. SUBSCRIBE TO OUR NEWSLETTER.
Add your information below to receive daily updates.