Tuesday, May 8, 2012

Tutorial: Using OpenNI on Android + Tegra 3 + Kinect

OpenNI C++ Wrapper for Android:


OpenNI Binary (Kinect only):
http://www.eyetap.org/~raymondlo84/multitouch/openni_binary.tgz

Performance: 

CPU usage and FPS:
  • PID PR CPU% S  #THR     VSS     RSS PCY UID      Name
  • 12539  2  60% R    14 532164K  65048K  fg app_39   com.nvidia.devtech.multi
  • I/Render Loop:(12539): Display loop 0.014692 (s) per frame --> over 60fps for rendering, the capture loop is threaded.

Source Code Preview: :) 
#ifndef __INCLUDED_OPENNI_WRAPPER_H
#define __INCLUDED_OPENNI_WRAPPER_H
#include <xnopenni.h>
#include <xnlog.h>
#include <xncppwrapper.h>
#include <xnfpscalculator.h>
#include <xnos.h>
#include <math.h>
#include "common.h"

//---------------------------------------------------------------------------
// Defines
//---------------------------------------------------------------------------
#define SAMPLE_XML_PATH "/data/ni/SamplesConfig.xml"

using namespace xn;
using namespace std;

class OpenniWrapper {

public:
 OpenniWrapper();
 ~OpenniWrapper();
 int start();
 void release();
 bool isReady();
 //the raw rgb
 void getRGB(unsigned char *rgb_buffer);
 //the raw depth image
 void getDepthDataShort(unsigned short *depth_buffer);
 //the rgbd (D = depth value and it is compressed and shifted). See documentation
 void getRGBD(unsigned char *rgbd_buffer);

private:
 //memory for the OpenNI data
 Context g_context;
 ScriptNode g_scriptNode;
 //the depth map and color image
 DepthGenerator g_depth;
 ImageGenerator g_image;
 DepthMetaData g_depthMD;
 ImageMetaData g_imageMD;
 int width, height;
 //local storage for the depth and rgb
 unsigned char *rgbImage;
 unsigned short *pDepth;

 float depthLookUpFloat[2048];
 float depthLookUpInt[2048];
 char buf[1024];

 //private/helper functions
 XnBool fileExists(const char *fn);
 float rawDepthToMeters(int input);
};
#endif


Demo Videos:



4 comments:

  1. Please write a full Tutorial :'(
    I need it asap

    ReplyDelete
  2. x) wish granted... see the link! I've posted the material to the PCL developer blog!

    XD YOU ARE WELCOME!

    ReplyDelete
  3. Hi Raymond, I am trying to build multitouch project on Android 4.0 device. Whenever I try to import either yours multitouch project or the one placed at openvidia sourceforge repository it is not building properly because the libs project in Java Build Path is missing. Please let me know where can I find that libs project.

    ReplyDelete
  4. hi ramond, it is amazing work you guys have done. I am quite interested in the project you are working on: Tegra 3 + Android 4.0 + OpenNI + OpenCV. Based on your experience, is it usable for the apps which require higher performance? i.e. a 3D game, a 3D virtual fitting room(with real time rgb campera video feeding), etc? Look forward to hearing from you and I want to contact with you further to discuss any opportunity, i.e. contract work or any other format.

    Thanks

    ReplyDelete