High level automatic piloting project

[erget]

Hi all,

A few friends and I have started a project to write high level automatic flight software for drones. We're using Crazyflie as a test platform, but the software is designed to be non-hardware specific. It will be using OpenCV as its main real-world interface, both for positioning, object recognition, etc. We're just getting started so thus far there isn't a whole lot going on with it, but I wanted to announce it in case anyone else is working on a similar project or wanting to do so - I value collaboration much more than elaborate, similar wheel redesigns

If you want to take a look at what's going on, check us out on Github:
http://erget.github.com/apisElectronica/

I'd be happy over any possibilities for cooperation.

Daniel

[TheFrog4u]

You mean somthing like this; http://www.bitcraze.se/2013/01/crazyfli ... cking-test ?
I read that you are going to use the raspberry PI, are you also going to use the ps3 eye?

Another alternative would be to use Android. Since every smart phone also has a camera and a lot more people have an android phone than a PI and a ps3 eye, this would be the much more attraktive solution..IMHO..

Some usefull links:

http://opencv.org/platforms/android.html

Regarding tracking the cf I would recommend using cvblob:
http://code.google.com/p/cvblob/
http://code.google.com/p/cvblob-for-android
[youtube]0dMGDzFYXVY[/youtube]

Technicaly I wonder how you plan to do 3D control from a 2D image.. The depth might me aproximated from the copter size.. however its would be better if you have 2 cameras.
A challenge will be to get the vertical orientation of the copter.. I assume you will have to get the magnetometer working. This would be another advantage of using android, where some smatphones also have a magnetometer.. in this case you can compare both and easily get the orientation of the copter
Or you have the 3rd camera from top tracking the little leds.. not a preferred solution

I will follow this nteresting project, good luck

[tobias]

We at bitcraze are also very interrested in this project but we are currently occupied with to much administration. We will do our best to support you and answer your questions as they pop up.

I like the idea from TheFrog4u using android. That would open up to some cool possibilities and it is very portable and wide spread.

[erget]

Hi there,

Wow, thanks for the replies!

I think I need to update the project description on Github a bit so that it's a little bit more understandable. The aim of the project is to write a library that allows drones to be controlled either from a ground control station, such as a laptop, or from the drone itself. That's the reason that we're making the programs portable for Raspberry Pi - eventually, a drone could carry its own ground control station, eliminating the need for a GCP in the traditional sense. Of course, that would be using a larger drone, not Crazyflie. We are planning on using Crazyflie as our primary development platform, but the software is designed to operate on a more obstract level, with all the direct interaction with the physical drone taking place at the end. The drone, and all algorithms that work on it, is handled abstractly.

@TheFrog4u: Those are some cool ideas. What are the advantages of cvblob compared with OpenCV?

We're not planning on doing 3d control from a 2d image, the plan is more to use FPV data from a keychain camera combined with the accelerometer and gyrometer data that gets sent back to the ground station. OpenCV has some pretty sweet modules for doing that

In general: An Android client would be very cool, but I think the manual remote control using Android is even cooler, and some smarter people are working on that When the library's working we could try to port our autopilot code to Android. I have no experience with Python on Android, but it's supposed to work, at least to a certain degree

@tobias: No problem that you can't work on it too, I'm really excited that you answered The most helpful thing for us is if you would post a detailed manual on FPV with Crazyflie. I'm looking forward to reporting back on our first experiments once the Crazyflies get here!

[TheFrog4u]

Ok, its getting a bit clearer now. CVblob is not an alternative to openCV, but rather an addition. It mainly adds fast and easy blob detection, which can be used for tracking. But if you want to to FPV auto pilot thats a different story.

Putting a FPV camera on the cf is already a difficult task. I assume you have looked into this:
http://www.bitcraze.se/2012/06/camera-a ... er-thesis/
and this:
http://www.bitcraze.se/2013/03/first-fpv-flight-test/
and this:
http://forum.bitcraze.se/viewtopic.php?f=6&t=5
already?

I understand that you want to have a fpv camera, but do the image processing and controlling on a pc, but can you explan a bit more, how your auto-pilot algorithm will work, and what exactly your goal is? Is it just to not crash into walls, or even path-finding? Neverless, it sounds like a very though job to me - do you have some experience in robot control?

[erget]

Hi there,

Some of those posts are indeed new to me, the one from last month sounded pretty promising though to us.

Our entire set of goals with the library is hard to describe, because we've got a very broad range of goals. The principle, of course, is to do everything as modularly as possible. The principle is to do everything as modularly as possible, so that other people can take our functions and use them to build their own applications.

As always, I'm open for tips and happy to hear any that you have. I realize that the project is quite challenging. Right now I'm waiting to get the drone so that I can start hooking up the hardware, as well as reading up on computer vision and similar topics.

Our goals are, in order:
* Get familiar with the drone itself and its programming interface
* Attach the camera and be able to analyze the video feed
* Create depth maps using the camera (although I'd love to do this with optical flow, I think we're going to end up using simulated stereo view for this by moving the drone back and forth)
* Allow the drone to avoid obstacles

I have no experience with autopiloting algorithms, nor with physical robotics, but I do have experience with efficient algorithms in a variety of fields. The two other people that are planning on working on the project are full-time software developers, whereas I'm more of a scientist who programs to get things done and enjoys programming them well if he can. I'm aware that it's a very challenging project, and I'm not being overconfident about it - but I am confident that if we fail, we'll at least fail while having a lot of fun The project will also move somewhat slowly, since we're all doing it in our free time, but that doesn't mean that we're incapable or lazy, just that we all have other jobs as well

Do you have experience with robot control? You're welcome to join us if you like.

[tobias]

Great to hear that your planning is moving forward!

From the beginning I though you would use a ground camera to control the drone and not a FPV. A FPV control will make things very challenging. I see a hurdle to pass: latency. I'm note sure what would be an acceptable latency between the image is taken and the drone getting the control command back but probably not more then 0.5s, if the drone flies slowly. Quicker manoeuvres would require lower latency. Analog FPV system to have low latency but must be digital converted at the computer side which takes time. Add to that the image processing and sending the data back and the latency grows. I would look at this first to get an feel of how big to expect it to be.

You found the FPV information and the master thesis, great. I have not had time to investigate the analog FPV on the Crazyflie any further but on our production version of the boards, Rev.F, we added mosfet gate resistors which decrease the electrical noise. We are hoping that will make the FPV work better. I hope I will be able to test that during next week.

Cheers!
/Tobias