Our reconstruction of a large crane structure, based on images collected by a Skydio drone. Thermal imaging with 3D reconstruction can be used for building and infrastructure inspection, among many other applications ranging from agriculture to search and rescue.
Abstract
Thermal imaging has a variety of applications, from agricultural monitoring to building inspection to imaging under poor visibility, such as in low light, fog, and rain. However, reconstructing thermal scenes in 3D presents several challenges due to the comparatively lower resolution and limited features present in long-wave infrared (LWIR) images. To overcome these challenges, we propose a unified framework for scene reconstruction from a set of LWIR and RGB images, using a multispectral radiance field to represent a scene viewed by both visible and infrared cameras, thus leveraging information across both spectra. We calibrate the RGB and infrared cameras with respect to each other, as a preprocessing step using a simple calibration target. We demonstrate our method on real-world sets of RGB and LWIR photographs captured from a handheld thermal camera, showing the effectiveness of our method at scene representation across the visible and infrared spectra. We show that our method is capable of thermal super-resolution, as well as visually removing obstacles to reveal objects that are occluded in either the RGB or thermal channels.
Broad-Spectrum Radiance Fields
When we begin to consider radiance fields across a wider spectrum, including our setting of RGB and LWIR thermal radiance field modeling, we find that more materials exhibit differing absorption behavior. We model this behavior by explicitly endowing each spatial location with separate densities (absorption coefficients) for each wavelength, while introducing regularization to encourage these wavelength-specific densities to remain similar for most materials.
Optimization and Regularization
![](./img/eq_radiance-field.png)
![](./img/eq_loss.png)
|
Additional Results
Revealing Hidden Objects
![](./img/eq_removal.png)
|
|
|
|
Citation
Acknowledgements
This material is based upon work supported by the National Science Foundation under award number 2303178 to SFK. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
The website template was borrowed from Michaƫl Gharbi.