Hardware Implementation of a Smart Camera with Keypoint Detection and Description

Feature detection and description constitute important steps of many computer vision applications such as object detection and panorama stitching. Since those steps are computationally heavy, they might occupy significant portion of the full operation. Although fast feature detection algorithms and resource-efficient binary description methods have been proposed and implemented, resource limited embedded devices and distributed camera systems still require more effective solutions. In this paper, we propose a novel smart camera architecture which finds the FAST keypoints and computes their FREAK descriptions by processing pixel stream. Thus, this smart camera system provides useful metadata associated with the pixel stream at the same time with no latency. Moreover, performance of this hardware reaches very high frame rates with power and area efficiency. With this approach, this costly operation is locally solved in the smart camera node, and this leads to meet timing and power constraints of the large camera networks.