Approximating Deep Neural Networks via Custom Arithmetic: a Tradeoff Between Accuracy and Power Consumption

Neural networks (NNs) have become a ubiquitous and essential element of modern computing, driving advancements in fields ranging from computer vision and natural language processing to autonomous vehicles and medical diagnostics. Despite their widespread and undisputed utility, however, NNs are inherently extremely resource-intensive, and training large models, especially for deep neural networks (DNNs), requires enormous computational power, with recent studies suggesting that training large-scale models is contributing to significant environmental impact, and that training a single state-of-the-art language model can emit as much carbon dioxide as five cars over their entire lifetimes~\cite{StrubellJan19, StrubellFeb20}. This project aims at exploring the efficient approximation of DNNs, specifically though the use of custom arithmetic components. Exploiting the knowhow and experience gained by my research in the field of Approximate Computing, and in particular in the design of Custom Arithmetic Components, this project will explore novel approximation avenues for deploying neural networks at scale that carefully balance the tradeoff between a limited loss in accuracy for a substantial energy saving.