Researchers Nursultan Amanzhol and Jurn-Gyu Park, affiliated with the University of California, Berkeley, have conducted a study focused on optimizing Vision Transformers (ViTs) for energy-efficient deployment on edge devices. Their work aims to address the growing need for efficient AI models in energy-constrained environments, a critical concern for the energy sector as it increasingly integrates AI for monitoring, predictive maintenance, and automation.
In their research, Amanzhol and Park present a two-stage pipeline designed to evaluate the energy efficiency of ViTs. The first stage is device-agnostic, using the NetScore metric to screen models based on their general efficiency. The second stage is device-specific, employing the Sustainable Accuracy Metric (SAM) to rank models based on their performance on particular hardware. The researchers benchmarked 13 ViT models on two datasets, ImageNet-1K and CIFAR-10, running inference on an NVIDIA Jetson TX2 (an edge device) and an NVIDIA RTX 3050 (a mobile GPU).
The results of their study highlight significant energy savings. Hybrid models, such as LeViT_Conv_192, demonstrated up to 53% energy reduction on the Jetson TX2 compared to a baseline ViT model. Specifically, the LeViT_Conv_192 model achieved a SAM5 score of 1.44 on the TX2 with the CIFAR-10 dataset. On the other hand, distilled models like TinyViT-11M_Distilled showed exceptional performance on the mobile GPU, achieving a SAM5 score of 1.72 on the RTX 3050 with CIFAR-10 and 0.76 with ImageNet-1K.
For the energy sector, these findings are particularly relevant as they enable more efficient deployment of AI models on edge devices, which are often used in remote or energy-sensitive locations. By optimizing the energy efficiency of AI models, the sector can reduce operational costs and environmental impact while maintaining high levels of accuracy and performance. The research was published in the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, a reputable venue for cutting-edge research in computer vision and machine learning.
This study underscores the importance of considering both accuracy and energy efficiency when deploying AI models in the energy sector. By adopting the two-stage pipeline proposed by Amanzhol and Park, energy companies can select the most suitable models for their specific hardware, ensuring optimal performance and energy savings.
This article is based on research available at arXiv.