Scientific Journal

Herald of Advanced Information Technology

Climate change is one of the most important challenges that humanity faces now. The essential part of climate models is the movement of clouds, which affects climate parameters dramatically. Shallow clouds play a huge role in determining the Earth's climate. They're also difficult to understand and to represent in climate models. Unfortunately, the exact modeling of clouds movement is notoriously tricky and requires perfect knowledge of underling physical processes and initial states. Boundaries between different types of clouds are usually blurry and difficult to define with rule-based decision systems. Simplification of the segmentation step is crucial and can help researchers to develop better climate models. Convolutional neural networks have been successfully applied in many similar areas, and for cloud segmentation itself, too. However, there is a high cost of good, pixel-level labeled datasets, so the industry often uses coarse-labeled datasets with the either region or image-level labels. In this paper, we propose an end-to-end deep-learning-based method for classification and segmentation of different types of clouds from a single colored satellite image. Here, we propose the multi-task learning approach to cloud segmentation. Additionally to the segmentation model, we introduce a separate classifier that uses features from the middle layer of the segmentation model. The presented method can use coarse, uneven and overlapping masks for clouds. From the experimental results, the proposed method demonstrates stable results and learns good general features from noisy data. As we observed during the experiments, our model finds types of clouds, which are not annotated on the images but seem to be correctly defined. It is ranked in top three percent competing methods on Understanding Clouds from Satellite Images Dataset
1. Rasp S., Schulz H., Bony S. & Stevens B. (2019) “Combining crowd-sourcing and deep learning to explore the mesoscale organization of shallow convection”, eprint arXiv:1906.01906, USA. 
2. Zhe Z., Shi Q., Binbin H. & Chengbing D. (2018). “Cloud and Cloud Shadow Detection for Landsat Images: The Fundamental Basis for Analyzing Landsat Time Series”, Remote Sensing Time Series Image Processing, USA. 
3. Harb M., Gamba P. & Dell'Acqua F. (2016). “Automatic Delineation of Clouds and Their Shadows in Landsat and CBERS (HRCC) Data”, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, USA. DOI:10.1109/JSTARS.2016.2514274. 
4. Hu X., Wang Y. & Shan J. (2015). “Automatic Recognition of Cloud Images by Using Visual Saliency Features”, IEEE Geoscience and Remote Sensing Letters, Vol. 12, No. 8, pp. 1760- 1764. DOI: 10.1109/LGRS.2015.2424531. 
5. Zhang, Jinglin Z., Liu P., Feng Z. & Qianqian S. (2018). “CloudNet: Ground-Based Cloud Classification With Deep Convolutional Neural Network” Geophysical Research Letters, USA. DOI: 10.1029/2018GL077787. 
6. (2020). “Understanding Clouds from Satellite Images | Kaggle”. [Digital resource] – Available at: anization. – Active link – 15.03.2020. 
7. (2020). “Sugar, Flower Fish, or Gravel?” [Digital resource]/ – Available at: ar-flower-fish-or-gravel. – Active link – 15.03.2020. 
8. (2020). “Albumentations: fast image augmentation library and easy to use wrapper around other libraries”. [Digital resource] – Available at: – Active link – 15.03.2020. 
9. Iglovikov V. & Shvets A. (2018). “TernausNet: U-Net with VGG11 Encoder PreTrained on ImageNet for Image Segmentation”, eprint arXiv: 1801.05746, USA. 
10. Ronneberger O., Fischer P. & Brox T. (2015). “U-Net: Convolutional Networks for Biomedical Image Segmentation”, eprint arXiv: 1505.04597, USA. 
11. Lin T., Dollár P., Girshick R., He K., Hariharan B. & Belongie S. (2016). “Feature Pyramid Networks for Object Detection”, eprint arXiv: 1612.03144, USA. 
12. He K., Zhang X., Ren S. & Sun J. (2015). “Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification”, eprint arXiv: 1502.01852, USA. 
13. Rich C., Lawrence S. & Giles C. (2000). “Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping”, Advances in Neural Information Processing Systems. 13. pp. 402-408, USA. 
14. Liu L., Jiang H., He P., Chen W., Liu X., Gao J. & Han J. (2019). “On the Variance of the Adaptive Learning Rate and Beyond”, eprint arXiv: 1908.03265, USA. 
15. You Y., Gitman I. & Ginsburg B. (2017). “Large Batch Training of Convolutional Networks”, eprint arXiv: 1708.03888, USA. 
16. Zhang M., Lucas J., Hinton G., Ba J. & Han J. (2019). “Lookahead Optimizer: k steps forward, 1 step back”, eprint arXiv: 1907.08610, USA. 
17. (2020). “Mgrankin/over9000: Over9000 optimizer” [Digital resource]. – Available at: – Active link – 15.03.2020. 
18. Loshchilov I. & Hutter F. (2016). “SGDR: Stochastic Gradient Descent with Warm Restarts”, eprint arXiv: 1608.03983, USA. 
19. Jeroen Bertels, Tom Eelbode, Maxim Berman, Dirk Vandermeulen, Frederik Maes, Raf Bisschops & Matthew Blaschko. (2019). “Optimizing the Dice Score and Jaccard Index for Medical Image Segmentation: Theory & Practice”, eprints arXiv: 1911.01685, USA. 
20. Dong-Hyun L. (2013). “Pseudo-Label: The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks”, ICML 2013 Workshop: Challenges in Representation Learning (WREPL), USA. 
21. Arazo E., Ortego D., Albert P., O'Connor N. & McGuinness K. (2020). “Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning”, eprints arXiv: 1908.02983, USA. 
22. Moshkov N., Mathe B., Kertesz-Farkas A., Hollandi R. & Horvath P. (2020). “Test-time augmentation for deep learning-based cell segmentation on microscopy images”, bioRxiv 814962, USA, DOI: 10.1101/814962. 
23. He K., Zhang X., Ren S. & Sun J. (2015). “Deep Residual Learning for Image Recognition", eprints arXiv: 1512.03385, USA. 
24. Huang G., Liu Z., van der Maaten L. & Weinberger K. (2016). “Densely Connected Convolutional Networks”, eprints arXiv: 1608.06993, USA. 
25. Tan M. & Le Q. (2019). “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks”, eprints arXiv: 1905.11946, USA. 
26. (2018). “Accelerated DL R&D”. [Digital resource]. – Available at: – Active link – 15.03.2020. 
27. (2017). “PyTorch”. [Digital resource] – Available at: – Active link – 15.03.2020. 28. Tymchenko B., Hramatik A., Tulchiy H. & Antoshchuk S. (2019). “Classifying Mixed Patterns of Proteins in Microscopic Images with Deep Neural Networks”, Herald of advanced information technology, Vol. 2, No. 1, pp. 29-36. DOI: 10.15276/hait.02.2019.3.

Received 29.01.2020
Received after revision 12.02.2020
Accepted 15.02.2020
Last download:
16 Oct 2021


[ © KarelWintersky ] [ All articles ] [ All authors ]
[ © Odessa National Polytechnic University, 2018.]