Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders

• Published in: ISPRS international journal of geo-information Vol. 7; no. 4; p. 129
• Main Authors: Rußwurm, Marc, Körner, Marco
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.04.2018
• Subjects: Atmospheric models; Classification; Coders; crop classification; Data; deep learning; Domains; Earth; Feature recognition; Filtration; Land cover; Land use; land use and land cover classification; land use and land cover maps; Machine translation; multi-temporal classification; phenology; Preprocessing; recurrent networks; Reflectance; Sensors; Sentinel 2; sequence encoder; sequence-to-sequence; Sequencing; speech; Speech recognition; vegetation
• ISSN: 2220-9964; 2220-9964
• DOI: 10.3390/ijgi7040129

Earth observation (EO) sensors deliver data at daily or weekly intervals. Most land use and land cover classification (LULC) approaches, however, are designed for cloud-free and mono-temporal observations. The increasing temporal capabilities of today’s sensors enable the use of temporal, along with spectral and spatial features.Domains such as speech recognition or neural machine translation, work with inherently temporal data and, today, achieve impressive results by using sequential encoder-decoder structures. Inspired by these sequence-to-sequence models, we adapt an encoder structure with convolutional recurrent layers in order to approximate a phenological model for vegetation classes based on a temporal sequence of Sentinel 2 (S2) images. In our experiments, we visualize internal activations over a sequence of cloudy and non-cloudy images and find several recurrent cells that reduce the input activity for cloudy observations. Hence, we assume that our network has learned cloud-filtering schemes solely from input data, which could alleviate the need for tedious cloud-filtering as a preprocessing step for many EO approaches. Moreover, using unfiltered temporal series of top-of-atmosphere (TOA) reflectance data, our experiments achieved state-of-the-art classification accuracies on a large number of crop classes with minimal preprocessing, compared to other classification approaches.