unstable behaviour using Kinect

[adadawuhehe]

I am doing a project that is using the Kinect, however I always failed.. currently i m using ubuntu precise and install everythings already.. but the crazyflie does not act as the same way that show in the youtube video, before it reaching the green target it already failed, as in my case it will fly in front instead of going up reaching the green target..
http://www.youtube.com/watch?v=PNdEcJ1bxXI

Code: Select all

da@ubuntu:~/b/bin$ sudo ./cfkinect
Info: Using config path:  /home/da/b/lib/../conf
Info: sys.path= ['/home/da/b/lib', '/home/da/b/lib/../conf', '/home/da/b/bin', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PIL', '/usr/lib/python2.7/dist-packages/gst-0.10', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client', '/usr/lib/python2.7/dist-packages/ubuntuone-client', '/usr/lib/python2.7/dist-packages/ubuntuone-control-panel', '/usr/lib/python2.7/dist-packages/ubuntuone-couch', '/usr/lib/python2.7/dist-packages/ubuntuone-installer', '/usr/lib/python2.7/dist-packages/ubuntuone-storage-protocol']
init done 
opengl support available 
INFO:cflib.crazyflie:Callback->Connection initialized[radio://0/10/250K]
INFO:cflib.crazyflie:We are connected[radio://0/10/250K], request connection setup
WARNING:cflib.crazyflie:ExpectAnswer: ERROR! Older timer whas running while scheduling new one on [5]
WARNING:cflib.crazyflie:ExpectAnswer: ERROR! Older timer whas running while scheduling new one on [5]
WARNING:cflib.crazyflie:ExpectAnswer: ERROR! Older timer whas running while scheduling new one on [5]
INFO:cflib.crazyflie:Callback->Connected to [radio://0/10/250K]
ERROR:cflib.crazyflie.toc:Got packet that was not on TOC channel, TOC fetch will probably not succeed
INFO:cflib.crazyflie.toc:TOC for port [5] found in cache
INFO:cflib.crazyflie:Log TOC finished updating
INFO:cflib.crazyflie.toc:TOC for port [2] found in cache
INFO:cflib.crazyflie:Param TOC finished updating
INFO:cflib.crazyflie:Callback->Connection setup finished [radio://0/10/250K]
[Stream 70] Expected max 1748 data bytes, but got 1908. Dropping...
[Stream 70] Expected max 1748 data bytes, but got 1908. Dropping...
154.0
154.0

or it is because of my pole attached with the ball too short, ball too small or i assemble it wrongly or because of centre mass or kinect angle problem?
kindly help me please? Thanks.

[adadawuhehe]

Anyone has tried crazyflie with kinect, kindly help me please

[marcus]

Hi,

Nice to see that you have it running, even though it's not working very well. The image-processing seems to work, so the problem is probably the regulation. This was just done as a proof-of-concept and has at least two big problems:

• The position (and the error in position) is calculated directly from the pixels in the image and not mapped in any space. This means that there's no distinction made between moving close to the camera or further away. But the response of the system is very different.
• The regulation would require a better model. We are controlling the acceleration but the input to the control loop on the PC is position.

The quick way to get it working is to change the thrust offset (set on line 105 in lib/cfkinect.py). This has a fixed offset (38000) that enables the regulation to get the Crazyflie into the center by starting at an offset for the thrust. Increasing this should make it work, but it's not a solution since this changes over time.

[adadawuhehe]

Thanks for your reply, finally the crazyflie is able to hover stably . Keeping in mind that RGB image is used for the X/Y position (right or left) whereas the depth image is used for the pitch (distance between the Kinect and crazyflie).
Three things to take note of are shown as below:
1)​ ​Orientation of the crazyflie - eg. the motor 3 of the crazyflie should face the kinect whereas motor 1 should face the wall. Failed to do so, may end up with different directions of pitch or roll.

2) Filtering out red color, cv.InRangeS(imgHSV, cv.Scalar(140, 150, 120), cv.Scalar(180, 255, 255), img_th);
- modification of the value(eg, 30 to 120) may be done to fit with your environment lighting so that kinect will detect the red color and decline the light red/pink color pixels.

3) Filtering out the crazyflie object - cv.InRangeS(depth_image, 10, 150, img_th);
- The intention for this API being invoked in the get_depth() function is to filter out the crazyflie object only. Modification of the value (eg. 200 to 150) has been done to avoid the white wall being picked up by the Kinect. It depends on your flying environment( eg. the kinect distance between the wall and kinet).This is the main reason why my crazyflie initially fly off from the screen because the Kinect detect/pick up the fixed white wall, consequently causing the depth value and also the pitch value outdated. Moreover, even the box where the crazyflie is placed during the taking off should be avoided appearing on the Kinect image.

[tobias]

Cool! Making it work isn't that easy so congrats!

Oliver has done a pretty nice integration with ROS which is available here. The result is better then what we did but requires some more dependencies and also some knowledge about ROS.

[nurinuri]

Hi adadawuhehe,
could you please provide more details how did you get your crazyflie fly using kinect (hover stably)? I have that same problem that you faced.
thank you.
Nuradeen.