Artificial intelligence (AI) has enabled a new paradigm of smart applications
-- changing our way of living entirely. Many of these AI-enabled applications
have very stringent latency requirements, especially for applications on mobile
devices (e.g., smartphones, wearable devices, and vehicles). Hence, smaller and
quantized deep neural network (DNN) models are developed for mobile devices,
which provide faster and more energy-efficient computation for mobile AI
applications. However, how AI models consume energy in a mobile device is still
unexplored. Predicting the energy consumption of these models, along with their
different applications, such as vision and non-vision, requires a thorough
investigation of their behavior using various processing sources. In this
paper, we introduce a comprehensive study of mobile AI applications considering
different DNN models and processing sources, focusing on computational resource
utilization, delay, and energy consumption. We measure the latency, energy
consumption, and memory usage of all the models using four processing sources
through extensive experiments. We explain the challenges in such investigations
and how we propose to overcome them. Our study highlights important insights,
such as how mobile AI behaves in different applications (vision and non-vision)
using CPU, GPU, and NNAPI. Finally, we propose a novel Gaussian process
regression-based general predictive energy model based on DNN structures,
computation resources, and processors, which can predict the energy for each
complete application cycle irrespective of device configuration and
application. This study provides crucial facts and an energy prediction
mechanism to the AI research community to help bring energy efficiency to
mobile AI applications.