SmartPartNet: Part-Informed Person Detection for Body-Worn Smartphones

Abstract

We are interested in the development of image-based person detection algorithms for wearable computing using commodity smartphones. We focus on the use of smartphones as a wearable device because it is a practical means of augmenting human sensing for applications such as navigation for the blind or assisting social interaction. We identify two unique features of developing a vision-based person detector for body-worn smartphones: (1) the detector must take into account the strong bias in the size of people in the images taken with a wearable device and (2) the detector must consider the low image quality due to dim lighting and rapid ego-motion which leads to motion blur. In order to account for the unique distribution over the visibility of body parts when using a wearable camera, we propose a part-based person detector specialized for chestmounted smartphones. We perform extensive ablative analysis on the usefulness of part information, providing several insights regarding the design of the optimal person detector across different application domains. To account for the frequent occurrence of motion blur in our target domain, we introduce a data augmentation technique to generate synthetic motion-blurred images during training. In addition to addressing the aforementioned features, the final detector must also run in real-time using only smartphone resources. We leverage recent progress in deep neural networks for mobile devices and show that our proposed person detector, SmartPartNet, obtains performance similar to state-of-the-art pedestrian detection networks, while being 3X smaller and 5X faster.