Multimodal Non-Contact Sensing of Neonatal Vital Signs Using Radar and Video

Abstract

Preterm neonates represent a vulnerable population which traditional contact-based monitoring devices are not optimized for their small size and complicated physiology. Adhesive sensors and wires can cause infections, discomfort, and impair the delivery of clinical care. Therefore, these most fragile patients could significantly benefit from remote health monitoring. This thesis establishes the foundation for a multimodal device designed for noncontact monitoring of neonates in the Neonatal Intensive Care Unit (NICU) that integrates a video camera and a radar. The device is used to estimate vital signs such as respiratory rate (RR), using both unimodal (solely video or radar) and multimodal fusion approaches that combine data from both sensors. Preliminary testing was conducted on neonatal simulator mannequins, followed by a clinical study at Tufts Medical Center NICU which collected data from 16 neonates so far (with the goal of reaching 20). The collected data was processed, labeled, and organized using image processing techniques and manual review, and then analyzed using a Video Vision Transformer (ViViT) architecture, incorporating early, intermediate, and late fusion strategies. Initial analysis was conducted on the mannequin data and the first neonatal subject. The results show that for estimating RR in neonates, the early fusion approach outperformed the unimodal methods. In movement detection, compared to human labeling, the fusion techniques achieved high accuracy and precision. To conclude, this study demonstrates that multimodal analysis has the potential to outperform unimodal approaches by improving accuracy against gold standard monitoring, particularly in challenging real-life conditions, including motion artifacts and poor lighting. This work represents a step toward more robust, non-invasive monitoring solutions for neonatal care, with implications for broader applications in remote health monitoring.