Automatic Mixed Precision and Distributed Data-Parallel Training for Satellite Image Classification using CNN

Yohanes Nuwara*, Wong W. Kitt, Filbert H. Juwono, Gregory Ollivierre

*Corresponding author for this work

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review


Deep learning models for computer vision in remote sensing such as Convolutional Neural Network (CNN) has benefitted acceleration from the usage of multiple CPUs and GPUs. There are several ways to make the training stage more effective in terms of utilizing multiple cores at the same time by processing different image mini-batches with a duplicated model called Distributed Data Parallelization (DDP) and computing the parameters in a lower precision floating-point number called Automatic Mixed Precision (AMP). We would like to investigate the impact of DDP and AMP training modes on the overall utilization and memory consumption of CPU and GPU, as well as the accuracy of a CNN model. The study is performed on the EuroSAT dataset, a Sentinel-2-based benchmark satellite image dataset for image classification of land covers. We compare training using 1 CPU, using DDP, and using both DDP and AMP over 100 epochs using ResNet-18 architecture. The hardware that we used are Intel Xeon Silver 4116 CPU with 24 cores and an NVIDIA v100 GPU. We find that although parallelization of CPUs or DDP takes less time to train on the images, it can take 50 MB more memory than using only a single CPU. The combination of DDP and AMP can release memory up to 160 MB and reduce computation time by 20 seconds. The test accuracy is slightly higher for both DDP and DDP-AMP at 90.61% and 90.77% respectively than without DDP and AMP at 89.84%. Hence, training using Distributed Data Parallelization (DDP) and Automatic Mixed Precision (AMP) has more benefits in terms of lower GPU memory consumption, faster training execution time, faster convergence towards solutions, and finally, higher accuracy.

Original languageEnglish
Title of host publicationFifteenth International Conference on Machine Vision, ICMV 2022
EditorsWolfgang Osten, Dmitry Nikolaev, Jianhong Zhou
ISBN (Electronic)9781510666184
Publication statusPublished - 2023
Externally publishedYes
Event15th International Conference on Machine Vision, ICMV 2022 - Rome, Italy
Duration: 18 Nov 202220 Nov 2022

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X


Conference15th International Conference on Machine Vision, ICMV 2022


  • Automatic Mixed Precision
  • Convolutional Neural Network
  • Distributed Data-Parallel
  • Graphics Processing Unit
  • Remote Sensing


Dive into the research topics of 'Automatic Mixed Precision and Distributed Data-Parallel Training for Satellite Image Classification using CNN'. Together they form a unique fingerprint.

Cite this