A dual-path model merging CNN and RNN with attention mechanism for crop classification

• Published in: European journal of agronomy Vol. 159; p. 127273
• Main Authors: Zhang, Fuyao, Yin, Jielin, Wu, Nan, Hu, Xinyu, Sun, Shikun, Wang, Yubao
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.09.2024
• Subjects: agronomy; Attention mechanism; China; Crop classification; cropland; Deep learning; Google Earth Engine; Internet; irrigation; neural networks; remote sensing; time series analysis; Time-series data; China; Deep learning; Crop classification; Attention mechanism; Google Earth Engine; Time-series data
• ISSN: 1161-0301
• DOI: 10.1016/j.eja.2024.127273

Rapid and accurate crop classification is essential for estimating crop information and improving cropland management. The application of deep learning models for crop classification using time-series data has become the most promising method. However, most approaches rely on single models for data processing result in lower classification accuracy and poor stability. Therefore, this study proposes a dual-path approach with attention mechanisms (DPACR) to promote the performance of this model architecture in crop classification using time series data. Specifically, the model comprises two branches, the Recurrent neural network (RNN) branch with bidirectional gated recurrent units (GRU) with a self-attention mechanism, and the convolutional neural network (CNN) branch based on SE-ResNet. Crop classification is accomplished by a main classifier, supported by auxiliary classifiers from the two branches. Using the Google Earth Engine and the Sentinel-2 satellite data, DPACR was tested in the Hetao irrigation district in Inner Mongolia, China. The comparison experiment demonstrated that the DPACR achieved the highest overall accuracy (OA = 0.959) and Kappa coefficient (Kappa = 0.941) compared to other five models (MLP, SE-ResNet, Bi-At-GRU, SVM, and RF). DPACR excelled in classifying six crops, maintaining high accuracy across multiple classes. Compared to attention mechanisms, auxiliary classifiers can significantly improve classification performance. This study highlights the effective combination of cloud computing and deep learning for large-scale crop classification, providing a practical method for agricultural monitoring and management. [Display omitted] •Deep learning model with dual-path is proposed for crop classification (DPACR).•The classification performance of DPACR outperforms comparison models.•The classification performance of CNN branch is more robust than RNN branch.•The auxiliary classifiers and attention mechanisms can improve performance.•The learning process of DPACR is explained by t-SNE.