Remote Sensing Neural Radiance Fields for Multi-View Satellite Photogrammetry

• Published in: Remote sensing (Basel, Switzerland) Vol. 15; no. 15; p. 3808
• Main Authors: Xie, Songlin, Zhang, Lei, Jeon, Gwanggil, Yang, Xiaomin
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.08.2023
• Subjects: Aerial photogrammetry; Aerial surveying; Artificial intelligence; Cameras; Comparative analysis; digital surface models; Earth observations (from space); Elevation; Estimates; geometry; hash table; Image processing; Image reconstruction; Image resolution; Localization; Machine learning; Methods; multi-view stereo; Neural networks; neural radiance fields; Photogrammetry; Radiance; Remote observing; Remote sensing; Satellite imagery; Satellite imaging; Satellite observation; satellite photogrammetry; Satellites; Self-supervised learning; Software; Synthesis; Technology application; China; United States > US; San Francisco California
• ISSN: 2072-4292; 2072-4292
• DOI: 10.3390/rs15153808

Neural radiance fields (NeRFs) combining machine learning with differentiable rendering have arisen as one of the most promising approaches for novel view synthesis and depth estimates. However, NeRFs only applies to close-range static imagery and it takes several hours to train the model. The satellites are hundreds of kilometers from the earth. Satellite multi-view images are usually captured over several years, and the scene of images is dynamic in the wild. Therefore, multi-view satellite photogrammetry is far beyond the capabilities of NeRFs. In this paper, we present a new method for multi-view satellite photogrammetry of Earth observation called remote sensing neural radiance fields (RS-NeRFs). It aims to generate novel view images and accurate elevation predictions quickly. For each scene, we train an RS-NeRF using high-resolution optical images without labels or geometric priors and apply image reconstruction losses for self-supervised learning. Multi-date images exhibit significant changes in appearance, mainly due to cars and varying shadows, which brings challenges to satellite photogrammetry. Robustness to these changes is achieved by the input of solar ray direction and the vehicle removal method. NeRFs make it intolerable by requiring a very long time to train an easy scene. In order to significantly reduce the training time of RS-NeRFs, we build a tiny network with HashEncoder and adopted a new sampling technique with our custom CUDA kernels. Compared with previous work, our method performs better on novel view synthesis and elevation estimates, taking several minutes.