Researchers Jonas Christoffer Villumsen and Yusuke Sugita from the University of Copenhagen have been exploring ways to make artificial intelligence (AI) more energy-efficient, particularly for use in mobile devices and edge machines. Their work focuses on binary neural networks (BNNs), which are known for their energy and memory efficiency but are challenging to train due to their discrete nature.
In their recent study, Villumsen and Sugita extend existing models for training BNNs using a method called quadratic unconstrained binary optimisation (QUBO). This approach is particularly relevant to the energy sector as it could lead to more efficient AI models that require less power to operate, which is crucial for deploying AI in energy-constrained environments such as remote sensors or edge devices.
The researchers propose two novel regularisation methods to improve the training of BNNs. The first method, called neuron margin maximisation, biases the training process towards configurations that yield larger pre-activation magnitudes. The second method uses a dropout-inspired iterative scheme where reduced subnetworks are trained and used to adjust linear penalties on network parameters. These methods aim to enhance the performance of BNNs, making them more accurate and reliable for practical applications.
The study applied the proposed QUBO formulation to a small binary image classification problem and conducted computational experiments on a GPU-based Ising machine. The results indicated that the proposed regularisation terms modified the training behaviour and improved classification accuracy on data not present in the training set. This suggests that the methods could be useful for training more accurate and efficient AI models.
The research was published in the journal Nature Communications, a reputable source for scientific research. The findings could have significant implications for the energy sector, particularly in areas where energy-efficient AI models are needed for tasks such as predictive maintenance, energy management, and remote monitoring. By reducing the energy consumption of AI models, these methods could contribute to more sustainable and efficient energy systems.
In summary, Villumsen and Sugita’s work on QUBO-based training and regularisation of BNNs offers a promising approach to making AI more energy-efficient and practical for use in resource-constrained environments. The proposed methods could have wide-ranging applications in the energy sector, helping to advance the deployment of AI in critical energy infrastructure.
This article is based on research available at arXiv.