Friday, April 27, 2012

Tegra 3 + Android + Kinect?

Looks like we will be getting very little performance gain from the Tegra 3? Maybe I need to upgrade the OS to 4.0?



  • I/Render Loop:( 2655): Display loop 0.022757 (s)
  • I/Render Loop:( 2655): Display loop 0.023210 (s)
  • I/Render Loop:( 2655): Display loop 0.023210 (s)


What... and 75% CPU usage? x\

Thursday, April 19, 2012

OpenNI + Kinect + Android 4.0 + Tegra 2!?

So close, but yet so far away. I believe I got the openni library setup properly now. At least, the shared libraries from OpenNI are placed properly in the /system/lib and it will not be erased after a reboot. I can now compile the NiSimpleRead using NDK and so.

Now I have a new problem...

D/OPENNI  (  921): Open failed: USB interface is not supported!

Somehow the Kinect isn't supported? What is going on? I have compiled the source tree and have it running on the Mac OSX 10.6...


Update # 1:

After more code reading and digging... It seems that such error arises from the USE Interface flag...


#Macintosh-259:SensorKinect raymond$ grep -ir "USBinterface" *
CHANGES:  * Default UsbInterface is now BULK on Arm (for performance reasons) and ISO on all other platforms.
CHANGES:  * Updated the value of the UsbInterface property once endpoints are opened.
Data/GlobalDefaultsKinect.ini:UsbInterface=2

I'm not too sure but I believe that could be a fix to such problem!

Here is what I have so far.

  • Recompiled OpenNI and SensorKinect and created the shared libraries (.so) files from the git repositories.
  • Inserted the header to the Tegra Kinect Project and successfully ported the NiSimpleViewer to Android. Spent hours to solve all linker problems!
  • Committed the shared libraries to /system/lib and figured out all dependencies. 
  • Successfully ran niReg on the Tegra 2 dev-board! ;)
==================================================================
#adb shell
#run these before the push
#mount -o remount rw /system
adb push system/lib/libOpenNI.so /system/lib
adb push system/lib/libOpenNI.jni.so /system/lib
adb push system/lib/libXnCore.so /system/lib
adb push system/lib/libXnDDK.so /system/lib
adb push system/lib/libXnDeviceFile.so /system/lib
adb push system/lib/libXnDeviceSensorV2.so /system/lib
adb push system/lib/libXnFormats.so /system/lib
adb push system/lib/libusb.so /system/lib
adb push system/lib/libnimCodecs.so /system/lib
adb push system/lib/libnimRecorder.so /system/lib
adb push system/lib/libnimMockNodes.so /system/lib
adb push data/in/modules.xml /data/ni/
#mount -o devmode=0666 -t usbfs none /proc/bus/usb
===================================================================
root@android:/ # niReg -l
3186 INFO New log started on 2012-04-20 03:02:30
4704 INFO OpenNI version is 1.5.2 (Build 23)-Android-Arm (Apr 19 2012 22:33:09)
4763 INFO --- Filter Info --- Minimum Severity: NONE
OpenNI version is 1.5.2.23.
Registered modules:
(compiled with OpenNI 1.5.2.23):
Script: OpenNI/OpenNI/1.5.2.23
/system/lib/libnimMockNodes.so (compiled with OpenNI 1.5.2.23):
ProductionNode: OpenNI/Mock/1.5.2.23
Generator: OpenNI/Mock/1.5.2.23
MapGenerator: OpenNI/Mock/1.5.2.23
Device: OpenNI/Mock/1.5.2.23
Depth: OpenNI/Mock/1.5.2.23
IR: OpenNI/Mock/1.5.2.23
Image: OpenNI/Mock/1.5.2.23
Audio: OpenNI/Mock/1.5.2.23
/system/lib/libnimCodecs.so (compiled with OpenNI 1.5.2.23):
Codec: OpenNI/16zP/1.5.2.23
Codec: OpenNI/16zT/1.5.2.23
Codec: OpenNI/Im8z/1.5.2.23
Codec: OpenNI/JPEG/1.5.2.23
Codec: OpenNI/NONE/1.5.2.23
/system/lib/libnimRecorder.so (compiled with OpenNI 1.5.2.23):
Recorder: OpenNI/Recorder/1.5.2.23
Player: OpenNI/Player/1.5.2.23
/system/lib/libXnDeviceSensorV2.so (compiled with OpenNI 1.5.2.23):
Device: PrimeSense/SensorKinect/5.1.0.25
Depth: PrimeSense/SensorKinect/5.1.0.25
Image: PrimeSense/SensorKinect/5.1.0.25
IR: PrimeSense/SensorKinect/5.1.0.25
Audio: PrimeSense/SensorKinect/5.1.0.25
/system/lib/libXnDeviceFile.so (compiled with OpenNI 1.5.2.23):
Player: PrimeSense/File/5.1.0.25
/system/lib/libXnCore.so (compiled with OpenNI 1.5.2.23):

:) That's it for the day.


Update # 2:

Solved the puzzle today! It was the GlobalDefaultsKinect.ini!!

Basically, we need to use 1 instead 2! whatever that is...

; USB interface to be used. 0 - FW Default, 1 - ISO endpoints, 2 - BULK endpoints. Default: Arm - 2, other platforms - 1


UsbInterface=1

Here is the result we get on Tegra 2 + Kinect + OpenNI
depth.GetMetaData(depthMD);const XnDepthPixel* pDepthMap = depthMD.Data();sprintf(buf, "Frame %d Middle point is: %u. FPS: %f\n", depthMD.FrameID(), depthMD(depthMD.XRes() / 2, depthMD.YRes() / 2), xnFPSCalc(&xnFPS));    __android_log_print(ANDROID_LOG_DEBUG, "OPENNI",  buf);
 From adb logcat
D/OPENNI  (  888): Frame 3764 Middle point is: 2993. FPS: 30.526403....


:) It is reading from the device and extracting a value from the depth map! :)

Horray! Now I will write up an official tutorial that show you how to port OpenNI to your own Android project using NDK! Looking forward for more hardware optimization as well!

Wednesday, April 18, 2012

OpenNI as static libraries?

Apparently, the OpenNI is a pain to port over. I somehow got everything compiled as static libraries... A great start!

However, that's only the beginning of the whole story! It seems that it refused to load the modules (Kinect Drivers)!... I hope I'm not mistaken but the OpenNI indeed checks for shared libraries in runtime and I end up stuck at this step today...



"One or more of the following nodes could not be enumerated:"

Oh great... That basically means I would need to go back to step one!?