A MOBILE APPLICATION FOR PERSONALIZED SELECTION OF SKINCARE PRODUCTS BASED ON CONVOLUTIONAL NEURAL NETWORKS AND VISION-LANGUAGE MODELS

Abstract

This paper presents the architecture and software implementation of a mobile application for skin analysis and personalized skincare product selection driven by artificial intelligence techniques. The solution primarily targets individuals whose skin undergoes increased physiological stress – professional athletes and active fitness enthusiasts – while remaining applicable to a broader audience. The analysis module utilizes a two-stage pipeline: a ResNet-family convolutional neural network (CNN) trained via transfer learning on open-source selfie image datasets with dermatological labeling, and the Gemini Flash 2.5 vision-language model (VLM), which validates CNN outputs for complex classes. The recommendation algorithm is detailed, featuring stringent allergen filters, product scoring based on skin type, an active ingredient incompatibility graph, and selection criteria that account for price tiers and brand diversity. Quantitative performance metrics demonstrate an accuracy of 88.65% for skin type classification and 91.35% for skin concerns; the integration of the VLM validator improves accuracy on challenging class pairs from 79.0% to 92.4%. These results confirm the practical viability of the hybrid CNN + VLM approach for mobile facial analysis in cosmetic applications.