Benchmarking of CNNs for Low-Cost, Low-Power Robotics Applications

Introduction

Available benchmarks for object classification and detection using Convolutional Neural Networks (CNNs) focus on evaluating accuracy only. This is reflected in the state-of-the-art CNNs as they provide high accuracy at the cost of increasing the number of parameters which translates into a large number of operations. However constrained environments like low-cost and low-power robots running on batteries commonly used by designers and hobbyists are required to obtain good performance at the minimal power consumption. Based on this we focus our benchmark on the importance of time and power consumption for inference in embedded platforms.

Hardware setup

Three embedded platforms are used for performing inference: Movidius NCS, Raspberry Pi 3, Intel Joule The power consumption is measured by sampling on the power lines using the INA219 power monitor setup at a sampling rate of ~500Hz. For the Raspberry Pi and Joule, Caffe and TensorFlow frameworks are used to perform inference.

Benchmarked networks:

Results

Inference Time:

Power Consumption

Conclusion:

The obtained inference times and power consumption show that by using the NCS some networks are up to 4x faster than running on other embedded platforms while keeping a low-power consumption. This is critical for battery powered low-cost robots. Future work includes the addition of more networks for the benchmark on more platforms and the creation of a public database for the obtained results.