Real-Time Non-Line-of-Sight Imaging Using Single-Photon LiDAR

Abstract

Robust real-time imaging systems have allowed for many advances in robotics and autonomous navigation, though limited visibility in many real-world settings remains a significant challenge. Non-Line-of-Sight (NLOS) sensing allows for imaging systems to “see around corners", expanding their range of perception, providing access information for realtime decision-making. A promising approach to NLOS sensing is through single-photon LiDAR, which is commonly used for range-finding in many imaging systems. In addition to range-finding, single-photon LiDAR systems can provide a deeply rich data source in the form of photon count histograms after reflecting off scene geometry, capturing detailed information from multiple bounces. NLOS imaging can be achieved by parsing third-bounce light from such single-photon LiDAR sensors, which can be used for a variety of detection and localization tasks, and recent work has demonstrated capabilities in a wide range of applications. This work aims to further develop the NLOS imaging system by demonstrating a fully functional NLOS system using low-cost, consumer-grade SPAD hardware for real-time NLOS imaging, detection, and localization. We lay the ground work for NLOS imaging systems by developing infrastructure for NLOS processing in real-time, and we examine the potential for NLOS systems to operate on cheap hardware using data-driven approaches. Our work implements and demonstrates full end-to-end capacity for these NLOS imaging systems in a number of applications including person detection and localization, facilitating future research in this field and paving the way for NLOS integration into consumer devices.