Multipatch Feature Pyramid Network for Weakly Supervised Object Detection in Optical Remote Sensing Images

• Published in: IEEE transactions on geoscience and remote sensing Vol. 60; pp. 1 - 13
• Main Authors: Shamsolmoali, Pourya, Chanussot, Jocelyn, Zareapoor, Masoumeh, Zhou, Huiyu, Yang, Jie
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.01.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE); Institute of Electrical and Electronics Engineers
• Subjects: Computer Science; Detection; Detectors; Feature extraction; Feature fusion; Methods; multiple patch learning (MPL); Multiprotocol label switching; multiscale object detection; Object detection; Object recognition; Patches (structures); Remote sensing; remote sensing images (RSIs); Semantics; Signal and Image Processing; Training; Transition layers; Training; Multiprotocol label switching; Semantics; Object detection; Detectors; Feature extraction; Standards
• ISSN: 0196-2892; 1558-0644; 1558-0644
• DOI: 10.1109/TGRS.2021.3106442

Object detection is a challenging task in remote sensing because objects only occupy a few pixels in the images, and the models are required to simultaneously learn object locations and detection. Even though the established approaches well perform for the objects of regular sizes, they achieve weak performance when analyzing small ones or getting stuck in the local minima (e.g., false object parts). Two possible issues stand in their way. First, the existing methods struggle to perform stably on the detection of small objects because of the complicated background. Second, most of the standard methods used handcrafted features and do not work well on the detection of objects parts that are missing. We here address the above issues and propose a new architecture with a multipatch feature pyramid network (MPFP-Net). Different from the current models that, during training, only pursue the most discriminative patches, in MPFP-Net, the patches are divided into class-affiliated subsets, in which the patches are related, and based on the primary loss function, a sequence of smooth loss functions is determined for the subsets to improve the model for collecting small object parts. To enhance the feature representation for patch selection, we introduce an effective method to regularize the residual values and make the fusion transition layers strictly norm-preserving. The network contains bottom-up and crosswise connections to fuse the features of different scales to achieve better accuracy compared to several state-of-the-art object detection models. Also, the developed architecture is more efficient than the baselines.