[VIDEO] Fully Autonomous Flight - Working Proof of Concept

[omwdunkley]

Hey guys!

In recent posts I have mentioned I am working on a fully autonomous mode using an on-board camera. Here is a video of my first test:


Fully Autonomous Hover Flight [Video Link]


The controller is not used through out the whole video. The flie should hold the position it is dropped from, even if you move it or blow it away.
Ive had it follow some waypoints too:)

Ill document everything in my master thesis in a few months, but if you have questions now, please ask here!

The SLAM system runs in one thread @ 20-30hz, detects and ignores bad images (red frames = bad ones), does deinterlacing (removes scan lines), uses the IMU to speed up/simplify computations (for pose estimation and to pre-rotate descriptors for matching), uses opencv for detecting features (green circles), matching features between frames and keyframes (green lines) and estimates a pose using the inlier matches (via ransac, purple lines).

Here is a picture of initialisation:



Of the flie and camera system used for the experiment:



And a screen shot from the VO system:



Cheers

[mgooch]

This is really cool. Did you use Python to develop the openCV client? Did you make those mounts to hold the camera?

[omwdunkley]

Yo!

This is really cool. Did you use Python to develop the openCV client?

Used python for the control stuff and for the client, but all the SLAM stuff is written in c++ using eigen for the linear algebra stuff. Speed is really important here, it only just runs at full speed in c++. Note the code is not really optimized and not parallelised at all.

Did you make those mounts to hold the camera?

Yup, in blender
Here is a 3d render:

and a link to the model on shapways.

[mgooch]

Yooooooo bro!

Used python for the control stuff and for the client, but all the SLAM stuff is written in c++ using eigen for the linear algebra stuff. Speed is really important here, it only just runs at full speed in c++. Note the code is not really optimized and not parallelised at all.

Nice. We primarly used Python for all the robotic simulation stuff. Keeps the project cost low when you break stuff in simulation . I'm currently freshening up on my Image Processing knowledge and quad copter kinematics. There is always time to optimize it, but parallelising things sometimes is pure hell . I only got into mobile robotics my last semester as an undergrad, after taking a class, but I loved every second of it.The mounts are pro (Didn't know you had a 3d blender printer ). I will probably order a couple this afternoon. What are you getting your master's in?

Did you get the camera system from FPVhobby?

[omwdunkley]

Yeah I find python much mroe maintainable...and especially useful if you need to make quick code changes in the field while running experiments
But for time critical tasks c++ just gives you more options
Exactly, optimisation was not a priority. Once the system is robost and happy, etc etc I would focus on the efficiency. Too many quick hacks and fixes at this point anyway..I would recode from scratch

Glad youre enjoying mobile robotics! I got into it pretty late too. The ardrone and crazyflie are really great and affordable ways to get into it though! Enjoy

Studying informatics at TUM with the Computer Vision Group.

Yup, camera, lens and transmitter from FPV hobby. Had probs with the 2.4ghz system, so I went with the rather bulky 5.8ghz. Id love to get the 2.4er to work - but its giving me lots of transmission/connection issues.
Where are you studying at?

[mgooch]

Well you have impressive modifications to the Crazyflie. A kludge of a quad-copter, but still very legit. I saw some of your other youtube videos and they spark a lot of interest. Informatics is a cool area of study and to be doing it with a computer vision group is a unique opportunity. Thanks for the recommendation on the FPV camera system. I hope to get one soon and implement it. I completed my BS in CS from SDSM&T. I'm getting my hands dirty in industry before jumping back into academia for my masters.

[tobias]

You're the man Oliver! Really impressive work. Looking forward to read your master thesis when it is ready!

[shahrukh480]

Hi all,

Its really good work.
I just want to know which is the best radio transmitter receiver and camera set to make this quadrotor autonomous. can't the transmitter of crazyflie be used to send the video to the base station, so that we can see the video on the crazyflie client.

[bobby_d]

Nudge on the above. Is there any way to process video without using an external transmitter (i.e. with the Crazyflie's onboard transmitter or else just onboard the Crazyflie)?

[arnaud]

It would not be possible to do this kind of video processing with the Crazyflie MCU, there is not enough computing power and no fast IO. Though it would be possible to add a camera with a powerful CPU as a deck, we are thinking about it but it is a lot of work to design such a board.

As for transmitting the video with the Crazyflie radio this has been attempted in a master thesis and the bandwidth is not high enough to get usable videos for processing (https://www.bitcraze.io/2012/06/camera- ... er-thesis/). For doing real time image processing on the ground analog transmitter seems to be the best we have anyway (virtually no latency).