Message-Passing-Driven Triplet Representation for Geo-Object Relational Inference in HRSI

• Published in: IEEE geoscience and remote sensing letters Vol. 19; pp. 1 - 5
• Main Authors: Chen, Jie, Zhou, Xing, Zhang, Yi, Sun, Geng, Deng, Min, Li, Haifeng
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Buildings; Data exchange; Geospatial analysis; Geospatial relation; high-resolution remote sensing image (HRSI); Image resolution; image understanding; Message passing; Object recognition; Remote sensing; Representations; Semantics; triplet; Visualization
• ISSN: 1545-598X; 1558-0571
• DOI: 10.1109/LGRS.2020.3038569

A high-resolution remote sensing image (HRSI) scene typically contains multiple geo-objects, and geospatial relations among these geo-objects are obvious. As the important information conveyed by HRSI, the intelligent expression of geospatial relation is helpful in understanding HRSI scenes. Previous HRSI semantic understanding was mainly based on image captions that only generate one sentence to describe image content, thereby resulting in insufficient understanding of the scene. Thus, the present letter proposes an approach to represent geospatial relations in an HRSI scene with structured form of <inline-formula> <tex-math notation="LaTeX">\langle </tex-math></inline-formula>subject, geospatial relation, object<inline-formula> <tex-math notation="LaTeX">\rangle </tex-math></inline-formula>. A geospatial relation triplet representation data set that contains visual and semantic information, such as category, location, and geospatial relations of the geo-objects, is constructed first. An "object-relation" message-passing mechanism is adopted to enhance the information exchange between the geo-objects and geospatial relations to predict triplets accurately. The experimental results show that the proposed method can effectively predict the geospatial relation in a HRSI scene.