Friday, June 8, 2012

Finger Tracking + Android 4.0 + Tegra 3 + Kinect + OpenNI + OpenCV

This tutorial we will show you how to implement the 'well-known' finger tracking algorithm on Kinect + Android + OpenNI + Tegra3 + OpenCV. The algorithm is indeed very simple and computationally efficient. We can achieve real-time performance easily on Tegra 3 without any s

We will be using the range camera (depth map) to create a contour of our hand. Then, we will perform a simple polynomial approximate on the contour, and extract the convex hull (the peaks) as our finger tips. All of these operations can happen in real-time on the Tegra 3 @ 25fps! That's pretty impressive.

The requirements are as follows:
  1. Tegra 3 or Tegra 2 Android device.
  2. Microsoft Kinect or Asus Xtion (not tested)
  3. NVIDIA NDK + OpenNI binary (see my previous tutorials)
  4. OpenCV binary (will be provided below)
  5. and 10 mins of your time :)


First of all, the method provided here are simplified for real-time usage, and we can get better results if we have a better way of extracting the hand location and filtering the depth map (See Limitation section at the end).  Right now, we are thresholding on a range, i.e., we assume that the hand of a person will be positioned closer to the range camera.



Contour Extraction.


Area and Bounding Boxes.


Polynomial Approximations and Convex Hulls 


Snapshot:

My Setup. Tegra 3 Tablet + Kinect; powered by a 12V battery.
Coming up Soon...