Scientific Journal

Herald of Advanced Information Technology

Skin cancer is the most prevalent type of cancer disease. The most of skin cancer deaths are caused by melanoma, despite being the least common skin cancer. Early and accurate detection and treatment is the best healing, however detection of this type of malignancy in the early stages is not obvious. Data-driven solutions for malignant melanomas detection can make treatment more effective. Convolutional neural networks have been successfully applied in different areas of computer vision, also in the classification of cancer types and stages. But in most cases, images are not enough to reach robust and accurate classification. Such metadata as sex, age, nationality, etc. could also be applied inside the models. In this paper, we propose an end-to-end method for the classification of melanoma stage using convolutional neural networks from an RGB photo and persons' metadata. Also, we provide a method of semi-supervised segmentation of the region of melanoma appearance. From the experimental results, the proposed method demonstrates stable results and learns good general features. The main advantage of this method is that it increases generalization and reduces variance by using an ensemble of the networks, pretrained on a large dataset, and fine-tuned on the target dataset. This method reaches ROC-AUC of 0.93 on 10982 unique unseen images.
1. “Melanoma: Statistics”. Available from: [Accessed 29th September 2019].
2. Castilla, R., Rangel-Cortes, J., García-Lamon, F., Adrian, T. “CNN and Metadata for Classification of Benign and Malignant Melanomas”. Intelligent Computing Theories and Application. 2019. p. 569–579. DOI: 10.1007/978-3-030-26969-2_54. 
3. “Melanoma Warning Signs”. Available from: [Accessed 15th August 2020].
4. Nasiri, S., Helsper, J., Jung, M. & Fathi, M. “DePicT Melanoma Deep-CLASS: a deep convolutional neural networks approach to classify skin lesion images”. BMC Bioinformatics. 2020; Vol. 21(2):84. DOI:10.1186/s12859-020-3351-y.
5. Song, L., Lin J., Wang Z. & Wang H. “An End-to-end Multi-task Deep Learning Framework for Skin Lesion Analysis”. IEEE J Biomed Health Inform. Epub ahead of print. PMID: 32071016. (13 Feb. 2020). DOI: 10.1109/JBHI.2020.2973614.
6. Chen, E., Dong, X., Li, X., Jiang, H., Rong, R. & Wu, J., “Lesion Attributes Segmentation for Melanoma Detection with Multi-Task U-Net”. IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy: 2019. p. 485-488. DOI: 10.1109/ISBI.2019.8759483.
7. Mustafa, S. & Kimura, A., “A SVM-based diagnosis of melanoma using only useful image features”, 2018 International Workshop on Advanced Image Technology (IWAIT), Chiang Mai, 2018. p. 1-4. DOI:10.1109/IWAIT.2018.8369646.
8. Nasiri, S., Jung, M., Helsper, J. & Fathi M. “Detect and Predict Melanoma Utilizing TCBR and Classification of Skin Lesions in a Learning Assistant System”. Bioinformatics and Biomedical Engineering, Lecture Notes in Computer Science. Springer International Publishing. 2018; Vol. 10813: 531–542.
9. Brinker, T., Hekler, A. & Alexander, H. “Deep neural networks are superior to dermatologists in melanoma image classification”. European Journal of Cancer. 2019; Vol.119. DOI:10.1016/j.ejca.2019.05.023.
10. Codella, N., Nguyen, Q., Pankanti, S. & Gutman, D. “Learning Deep Ensembles for Melanoma Recognition in Dermoscopy Images”, eprint arXiv:1610.04662. USA. 2016.
11. Yang, X., Zeng, Z., Yeo, S., Tan, C., Tey, H. & Su, Y. “A Novel Multi-task Deep Learning Model for Skin Lesion Segmentation and Classification”, eprint arXiv:1703.01025. USA. 2017.
12. “SIIM-ISIC Melanoma Classification | Data”. Available from: [Accessed 19th August 2020].
13. “What is “diagnosis=unknown” in the CSV train file?”. Available from: [Accessed 15th August 2020].
14. “Triple Stratified Leak-Free KFold CV”. Available from: [Accessed 15th August 2020].
15. Bragley, A. “The use of the area under the ROC curve in the evaluation of machine learning algorithms”. Pattern Recognition. 1997; Vol. 30 Issue 7. DOI: 10.1016/S0031-3203(96)00142-2.
16. Caruana, R. “Multitask Learning”. Machine Learning 28. 1997. p. 41–75. DOI:10.1023/A:1007379606734.
17. Iglovikov, V. & Shvets, A. “TernausNet: U-Net with VGG11 Encoder Pre-Trained on ImageNet for Image Segmentation”, eprint arXiv: 1801.05746. USA. 2018.
18. Schlempe,r J., Oktay, O., Schaap, M., Heinrich, M., Kainz, B., Glocker, B. & Rueckert D. “Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images”, eprint arXiv:1808.08114. USA. 2018.
19. Li, K., Wu, Z., Peng, K., Ernst, J. & Fu, Y. “Tell Me Where to Look: Guided Attention Inference Network”, eprint arXiv:1802.10171. USA. 2018.
20. Tan, M. & Le, Q. “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks”, eprints arXiv: 1905.11946. USA. 2019.
21. Zuiderveld, K. “Contrast Limited Adaptive Histogram Equalization”. Graphics Gems IV. Academic Press. 1994.
22. “Albumentations: fast image augmentation library and easy to use wrapper around other libraries”. Available from: [Accessed 15th August 2020].
23. “Microscope augmentation”. Available from: [Accessed 15th August 2020].
24. DeVries, T. & Taylor, G. “Improved Regularization of Convolutional Neural Networks with Cutout”, eprint arXiv:1708.04552. USA. 2017.
25. Xie, Q., Luong, M., Hovy, E. & Le, Q. “Self-training with Noisy Student improves ImageNet classification”, eprint arXiv:1911.04252. USA. 2019.
26. Rich, C., Lawrence, S. & Giles, C. “Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping”. Advances in Neural Information Processing Systems. USA. 2001. p. 402–408.
27. Kingma, D. & Ba, J. “Adam: A Method for Stochastic Optimization”, eprint arXiv:1412.6980. USA. 2014.
28. Robbins, H. “A Stochastic Approximation Method”. Annals of Mathematical Statistics Volume 22, Number 3 (1951): 400-407. DOI: 10.1214/aoms/1177729586.
29. Loshchilov, I. & Hutter, F. “SGDR: Stochastic Gradient Descent with Warm Restarts”, eprint arXiv:1608.03983. USA. 2016.
30. Tarvainen, A. & Valpol, H. “Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results”, eprint arXiv: 1703.01780. USA. 2017.
31. Lin, T., Goyal, P., Girshick, R., He, K. & Dollár, P. “Focal Loss for Dense Object Detection”, eprint arXiv: 1708.02002. USA. 2017.
32. Shrivastava, A., Gupta, A. & Girshick, R. “Training Region-based Object Detectors with Online Hard Example Mining”, eprint arXiv: 1604.03540, USA. 2016.
33. Ishida, T., Yamane, I., Sakai, T., Niu, G. & Sugiyama M. “Do We Need Zero Training Loss After Achieving Zero Training Error?”, eprint arXiv: 2002.08709. USA. 2020.
34. Otsu, N. “A Threshold Selection Method from Gray-Level Histograms”. IEEE Transactions on Systems, Man, and Cybernetics. Vol. 9 , Issue 1, Jan. 1979. p. 62-66. DOI: 10.1109/TSMC.1979.4310076.
35. Moshkov, N., Mathe, B., Kertesz-Farkas, A., Hollandi, R. & Horvath, P. “Test-time augmentation for deep learning-based cell segmentation on microscopy images”. Eprint bioRxiv 814962. USA. 2020. DOI:10.1101/814962.
36. “Image Test Time Augmentation with PyTorch”. Available from: [Accessed 15th September 2020].
37. Fort, S., Hu, H. & Lakshminarayanan, B. “Deep Ensembles: A Loss Landscape Perspective”, eprint arXiv: 1912.02757. USA. 2019.
38. Hu, J., Shen, L., Albanie, S., Sun, G. & Wu, E. “Squeeze-and-Excitation Networks”, eprints arXiv: 1709.01507. USA. 2017.
39. “Accelerated DL R&D”. Available from: [Accessed 15th September 2020].
40. “PyTorch”. Available from: [Accessed 15th August 2020].

Received 07.08.2020
Received after revision 17.09.2020
Accepted 21.09.2020
Last download:
24 Oct 2021


[ © KarelWintersky ] [ All articles ] [ All authors ]
[ © Odessa National Polytechnic University, 2018.]