[VIDEO] Fully Autonomous Flight - Working Proof of Concept

Firmware/software/electronics/mechanics
Post Reply
omwdunkley
Expert
Posts: 162
Joined: Thu Jun 06, 2013 9:56 pm
Location: Munich

[VIDEO] Fully Autonomous Flight - Working Proof of Concept

Post by omwdunkley » Sat Jun 21, 2014 11:27 am

Hey guys!

In recent posts I have mentioned I am working on a fully autonomous mode using an on-board camera. Here is a video of my first test:


Fully Autonomous Hover Flight [Video Link]


The controller is not used through out the whole video. The flie should hold the position it is dropped from, even if you move it or blow it away.
Ive had it follow some waypoints too:)

Ill document everything in my master thesis in a few months, but if you have questions now, please ask here!

The SLAM system runs in one thread @ 20-30hz, detects and ignores bad images (red frames = bad ones), does deinterlacing (removes scan lines), uses the IMU to speed up/simplify computations (for pose estimation and to pre-rotate descriptors for matching), uses opencv for detecting features (green circles), matching features between frames and keyframes (green lines) and estimates a pose using the inlier matches (via ransac, purple lines).

Here is a picture of initialisation:
Image


Of the flie and camera system used for the experiment:
Image


And a screen shot from the VO system:
Image


Cheers :)

mgooch
Beginner
Posts: 11
Joined: Fri Jun 20, 2014 2:53 pm

Re: [VIDEO] Fully Autonomous Flight - Working Proof of Conce

Post by mgooch » Sat Jun 21, 2014 4:04 pm

This is really cool. Did you use Python to develop the openCV client? Did you make those mounts to hold the camera?

omwdunkley
Expert
Posts: 162
Joined: Thu Jun 06, 2013 9:56 pm
Location: Munich

Re: [VIDEO] Fully Autonomous Flight - Working Proof of Conce

Post by omwdunkley » Sun Jun 22, 2014 10:12 am

Yo!
mgooch wrote:This is really cool. Did you use Python to develop the openCV client?
Used python for the control stuff and for the client, but all the SLAM stuff is written in c++ using eigen for the linear algebra stuff. Speed is really important here, it only just runs at full speed in c++. Note the code is not really optimized and not parallelised at all.
mgooch wrote:Did you make those mounts to hold the camera?
Yup, in blender ;)
Here is a 3d render:
Image
and a link to the model on shapways.

mgooch
Beginner
Posts: 11
Joined: Fri Jun 20, 2014 2:53 pm

Re: [VIDEO] Fully Autonomous Flight - Working Proof of Conce

Post by mgooch » Sun Jun 22, 2014 1:19 pm

Yooooooo bro!
omwdunkley wrote:Used python for the control stuff and for the client, but all the SLAM stuff is written in c++ using eigen for the linear algebra stuff. Speed is really important here, it only just runs at full speed in c++. Note the code is not really optimized and not parallelised at all.
Nice. We primarly used Python for all the robotic simulation stuff. Keeps the project cost low when you break stuff in simulation :D . I'm currently freshening up on my Image Processing knowledge and quad copter kinematics. There is always time to optimize it, but parallelising things sometimes is pure hell :evil: . I only got into mobile robotics my last semester as an undergrad, after taking a class, but I loved every second of it.The mounts are pro (Didn't know you had a 3d blender printer :lol: ). I will probably order a couple this afternoon. What are you getting your master's in?

Did you get the camera system from FPVhobby?

omwdunkley
Expert
Posts: 162
Joined: Thu Jun 06, 2013 9:56 pm
Location: Munich

Re: [VIDEO] Fully Autonomous Flight - Working Proof of Conce

Post by omwdunkley » Sun Jun 22, 2014 7:32 pm

Yeah I find python much mroe maintainable...and especially useful if you need to make quick code changes in the field while running experiments :)
But for time critical tasks c++ just gives you more options ;)
Exactly, optimisation was not a priority. Once the system is robost and happy, etc etc I would focus on the efficiency. Too many quick hacks and fixes at this point anyway..I would recode from scratch :)

Glad youre enjoying mobile robotics! I got into it pretty late too. The ardrone and crazyflie are really great and affordable ways to get into it though! Enjoy :)

Studying informatics at TUM with the Computer Vision Group.

Yup, camera, lens and transmitter from FPV hobby. Had probs with the 2.4ghz system, so I went with the rather bulky 5.8ghz. Id love to get the 2.4er to work - but its giving me lots of transmission/connection issues.
Where are you studying at?

mgooch
Beginner
Posts: 11
Joined: Fri Jun 20, 2014 2:53 pm

Re: [VIDEO] Fully Autonomous Flight - Working Proof of Conce

Post by mgooch » Sun Jun 22, 2014 10:04 pm

Well you have impressive modifications to the Crazyflie. A kludge of a quad-copter, but still very legit. I saw some of your other youtube videos and they spark a lot of interest. Informatics is a cool area of study and to be doing it with a computer vision group is a unique opportunity. Thanks for the recommendation on the FPV camera system. I hope to get one soon and implement it. I completed my BS in CS from SDSM&T. I'm getting my hands dirty in industry before jumping back into academia for my masters.

tobias
Bitcraze
Posts: 1736
Joined: Mon Jan 28, 2013 7:17 pm
Location: Sweden

Re: [VIDEO] Fully Autonomous Flight - Working Proof of Conce

Post by tobias » Wed Jun 25, 2014 6:53 am

You're the man Oliver! Really impressive work. Looking forward to read your master thesis when it is ready!

shahrukh480
Beginner
Posts: 1
Joined: Tue Oct 21, 2014 6:52 pm

Re: [VIDEO] Fully Autonomous Flight - Working Proof of Conce

Post by shahrukh480 » Tue Oct 21, 2014 7:07 pm

Hi all,

Its really good work.
I just want to know which is the best radio transmitter receiver and camera set to make this quadrotor autonomous. can't the transmitter of crazyflie be used to send the video to the base station, so that we can see the video on the crazyflie client.

bobby_d
Beginner
Posts: 5
Joined: Fri Feb 13, 2015 6:33 pm

Re: [VIDEO] Fully Autonomous Flight - Working Proof of Concept

Post by bobby_d » Wed Oct 14, 2015 9:11 pm

Nudge on the above. Is there any way to process video without using an external transmitter (i.e. with the Crazyflie's onboard transmitter or else just onboard the Crazyflie)?

arnaud
Bitcraze
Posts: 1826
Joined: Tue Feb 06, 2007 12:36 pm

Re: [VIDEO] Fully Autonomous Flight - Working Proof of Concept

Post by arnaud » Wed Oct 21, 2015 9:05 am

It would not be possible to do this kind of video processing with the Crazyflie MCU, there is not enough computing power and no fast IO. Though it would be possible to add a camera with a powerful CPU as a deck, we are thinking about it but it is a lot of work to design such a board.

As for transmitting the video with the Crazyflie radio this has been attempted in a master thesis and the bandwidth is not high enough to get usable videos for processing (https://www.bitcraze.io/2012/06/camera- ... er-thesis/). For doing real time image processing on the ground analog transmitter seems to be the best we have anyway (virtually no latency).

Post Reply