Thursday, April 19, 2012

OpenNI + Kinect + Android 4.0 + Tegra 2!?

So close, but yet so far away. I believe I got the openni library setup properly now. At least, the shared libraries from OpenNI are placed properly in the /system/lib and it will not be erased after a reboot. I can now compile the NiSimpleRead using NDK and so.

Now I have a new problem...

D/OPENNI  (  921): Open failed: USB interface is not supported!

Somehow the Kinect isn't supported? What is going on? I have compiled the source tree and have it running on the Mac OSX 10.6...


Update # 1:

After more code reading and digging... It seems that such error arises from the USE Interface flag...


#Macintosh-259:SensorKinect raymond$ grep -ir "USBinterface" *
CHANGES:  * Default UsbInterface is now BULK on Arm (for performance reasons) and ISO on all other platforms.
CHANGES:  * Updated the value of the UsbInterface property once endpoints are opened.
Data/GlobalDefaultsKinect.ini:UsbInterface=2

I'm not too sure but I believe that could be a fix to such problem!

Here is what I have so far.

  • Recompiled OpenNI and SensorKinect and created the shared libraries (.so) files from the git repositories.
  • Inserted the header to the Tegra Kinect Project and successfully ported the NiSimpleViewer to Android. Spent hours to solve all linker problems!
  • Committed the shared libraries to /system/lib and figured out all dependencies. 
  • Successfully ran niReg on the Tegra 2 dev-board! ;)
==================================================================
#adb shell
#run these before the push
#mount -o remount rw /system
adb push system/lib/libOpenNI.so /system/lib
adb push system/lib/libOpenNI.jni.so /system/lib
adb push system/lib/libXnCore.so /system/lib
adb push system/lib/libXnDDK.so /system/lib
adb push system/lib/libXnDeviceFile.so /system/lib
adb push system/lib/libXnDeviceSensorV2.so /system/lib
adb push system/lib/libXnFormats.so /system/lib
adb push system/lib/libusb.so /system/lib
adb push system/lib/libnimCodecs.so /system/lib
adb push system/lib/libnimRecorder.so /system/lib
adb push system/lib/libnimMockNodes.so /system/lib
adb push data/in/modules.xml /data/ni/
#mount -o devmode=0666 -t usbfs none /proc/bus/usb
===================================================================
root@android:/ # niReg -l
3186 INFO New log started on 2012-04-20 03:02:30
4704 INFO OpenNI version is 1.5.2 (Build 23)-Android-Arm (Apr 19 2012 22:33:09)
4763 INFO --- Filter Info --- Minimum Severity: NONE
OpenNI version is 1.5.2.23.
Registered modules:
(compiled with OpenNI 1.5.2.23):
Script: OpenNI/OpenNI/1.5.2.23
/system/lib/libnimMockNodes.so (compiled with OpenNI 1.5.2.23):
ProductionNode: OpenNI/Mock/1.5.2.23
Generator: OpenNI/Mock/1.5.2.23
MapGenerator: OpenNI/Mock/1.5.2.23
Device: OpenNI/Mock/1.5.2.23
Depth: OpenNI/Mock/1.5.2.23
IR: OpenNI/Mock/1.5.2.23
Image: OpenNI/Mock/1.5.2.23
Audio: OpenNI/Mock/1.5.2.23
/system/lib/libnimCodecs.so (compiled with OpenNI 1.5.2.23):
Codec: OpenNI/16zP/1.5.2.23
Codec: OpenNI/16zT/1.5.2.23
Codec: OpenNI/Im8z/1.5.2.23
Codec: OpenNI/JPEG/1.5.2.23
Codec: OpenNI/NONE/1.5.2.23
/system/lib/libnimRecorder.so (compiled with OpenNI 1.5.2.23):
Recorder: OpenNI/Recorder/1.5.2.23
Player: OpenNI/Player/1.5.2.23
/system/lib/libXnDeviceSensorV2.so (compiled with OpenNI 1.5.2.23):
Device: PrimeSense/SensorKinect/5.1.0.25
Depth: PrimeSense/SensorKinect/5.1.0.25
Image: PrimeSense/SensorKinect/5.1.0.25
IR: PrimeSense/SensorKinect/5.1.0.25
Audio: PrimeSense/SensorKinect/5.1.0.25
/system/lib/libXnDeviceFile.so (compiled with OpenNI 1.5.2.23):
Player: PrimeSense/File/5.1.0.25
/system/lib/libXnCore.so (compiled with OpenNI 1.5.2.23):

:) That's it for the day.


Update # 2:

Solved the puzzle today! It was the GlobalDefaultsKinect.ini!!

Basically, we need to use 1 instead 2! whatever that is...

; USB interface to be used. 0 - FW Default, 1 - ISO endpoints, 2 - BULK endpoints. Default: Arm - 2, other platforms - 1


UsbInterface=1

Here is the result we get on Tegra 2 + Kinect + OpenNI
depth.GetMetaData(depthMD);const XnDepthPixel* pDepthMap = depthMD.Data();sprintf(buf, "Frame %d Middle point is: %u. FPS: %f\n", depthMD.FrameID(), depthMD(depthMD.XRes() / 2, depthMD.YRes() / 2), xnFPSCalc(&xnFPS));    __android_log_print(ANDROID_LOG_DEBUG, "OPENNI",  buf);
 From adb logcat
D/OPENNI  (  888): Frame 3764 Middle point is: 2993. FPS: 30.526403....


:) It is reading from the device and extracting a value from the depth map! :)

Horray! Now I will write up an official tutorial that show you how to port OpenNI to your own Android project using NDK! Looking forward for more hardware optimization as well!

11 comments:

  1. Hi

    I get the same error if i try to run a sample like NiSimpleRead.
    This is the first post i found about the error (Open Failed: USB interface is not supported!)
    I use a pandaboard with ubuntu 11.04.

    Perhaps you have any advice to fix this error at my system. That would be really nice. I don't found anything else beside your blog about this error.

    ReplyDelete
  2. You have to make sure UsbInterface=1 in the UseGlobalDefaultsKinect.ini. Of course, I assume the modules.xml and the path settings are correct :) I will be posting the tutorial on http://www.pointclouds.org/blog/nvcs/ next. :)!

    ReplyDelete
  3. Awesome! Can't wait for the full tutorial!!

    ReplyDelete
    Replies
    1. More documentation here.

      http://www.pointclouds.org/blog/nvcs/raymondlo84/index.php

      Tutorial is coming up! It takes a while to write and verify the results x)

      Delete
  4. Great work! Could you please tell me what is the frame rate I can expect? Thanks!

    ReplyDelete
  5. Oh I miss the point: "D/OPENNI ( 888): Frame 3764 Middle point is: 2993. FPS: 30.526403...."

    Sorry, I have no experience. Is it the processing fps?

    ReplyDelete
  6. I can get 30fps with depth rgb calibration + display on Tegra 3 and libfreenect driver. I believe OpenNI won't be that much slower? I haven't confirmed that yet, but I would give that a try once I get back from a conference next week.

    ReplyDelete
  7. I can get over 30fps easily using Tegra 3... I have updated the blog and you can check out the source code as well

    ReplyDelete
  8. Hi Raymond,

    I tried running the command :

    mount -o devmode=0666 -t usbfs none /proc/bus/usb in adb shell.

    I get the following error message :

    root@android:/ # mount -o devmode=0666 -t usbfs none /proc/bus/usb
    mount -o devmode=0666 -t usbfs none /proc/bus/usb
    mount: No such file or directory

    I cd-ed into the bus folder and found that there is not usb folder there.

    Do you have any ideas on how to resolve this?


    ReplyDelete
    Replies
    1. hi blue wolverine! having the same prob here. can you remeber if and how you solved it . iam looking for a solution for hours now, cheers.

      Delete
  9. Is there a way to connect Ubi to my ceiling speakers?

    ReplyDelete