|
A query-aware CNN-transformer framework for multi-temporal crop classification and phenological stage estimation using Sentinel-2 time series
|
Authors: Yusuf Oztop, Berrin Bilgin, Sudenaz Senbay, Ozan Celik, Cavit Karakus
Status: Final
Date of publication: 30 September 2026
Published in: ITU Journal on Future and Evolving Technologies, Volume 7 (2026), Issue 3, Pages 372-386
Article DOI : https://doi.org/10.52953/LSBT6505
|
Abstract:
This study presents a high performance deep learning framework for multi-temporal agricultural monitoring using satellite image time series. A hybrid query-aware CNN-transformer architecture (C03) is proposed to jointly perform crop type classification and rice phenology stage prediction within a unified multitask learning setting. The model leverages spatial-temporal feature extraction by combining convolutional neural networks for local spatial context and transformer-based attention mechanisms for long-range temporal dependencies.
Initial baseline experiments with a CNN-TCN-BiLSTM-attention architecture revealed significant performance limitations, including early metric stagnation and severe mode collapse, where the model overfits to dominant classes while failing to generalize across minority crop types. To address these challenges, the proposed C03 model introduces explicit query conditioning based on day-of-year (DOY), enabling precise temporal alignment between input sequences and phenological stages.
The final C03 + P6 system achieved a Macro-F1Crop of 0.9941, a Macro-F1RicePheno of 0.9382, and a weighted total score of 0.9605. Ablation studies confirm that query-awareness is a critical component, as removing the queried day-of-year (DOY) information results in a dramatic performance degradation. A complementary input ablation shows that crop classification strongly depends on spectral inputs, whereas rice phenology performance is largely preserved when imagery is removed.
Furthermore, an optimized inference pipeline (P6) significantly reduces computational overhead on the ground-based evaluation platform, indicating suitability for resource-constrained deployment settings; on-orbit use would additionally require validation on flight-representative hardware. The findings demonstrate that integrating spatial context, temporal attention, and query-aware conditioning yields a robust and scalable solution for agricultural monitoring. |
Keywords: Agricultural monitoring, CNN-transformer framework, crop classification, phenological stage estimation, Sentinel-2 time series
Rights: © International Telecommunication Union, available under the CC BY-NC-ND 3.0 IGO license.
|
|
| Detalle del artículo | Artículo | Precio | |
|---|
| 0
| Gratuito | Descargar |
|
|
| |