Working FPV Camera setup with realtime HUD

[omwdunkley]

Hey guys! After lots of trial and error I finally got it working!


Photo of in flight (very much pre alpha) HUD while max throttle applied (maximise browser to view full image..):

The image shows the hud node, which displays the camera image, optionally rectifying it and doing some really cheap "deinterlacing" - dropping every 2nd row. The image is updated at 25hz. The hud is drawn up to 100hz, the maximum speed at which I can grab the crazyflie IMU, etc data. The hud shows control input data, link/battery status, G, heading, initial heading and direction/distance from it, charge mode/status, flight mode, barometric data, actual thrust data for each motor, an artificial horizon, an altitude indicator (still in progress) and how fast it is receiving which data.
You can actually see that the front motor has to do lots more work - as the camera shifts the center of mass waaay forward!
The hud colours can be changed, screenshots made, etc etc. Still tons of work to do but basic functionality is there.



Photo of flie with FPV parts:

Detail: https://lh3.googleusercontent.com/-1LWK ... se_fpv.jpg
Underside: https://lh4.googleusercontent.com/-A3IG ... om_fpv.jpg - switch is visable and used to turn off fpv system while charging.

720p video of it in action:
http://www.youtube.com/watch?v=AWSUMGJK ... e=youtu.be


Battery life: 2:30 (then noise/cutouts start to appear)
Weight: 24g

Hardware:

• 5V step up voltage regulator: http://www.pololu.com/catalog/product/2115
• Transmitter: http://www.fpvhobby.com/transmitter/21- ... itter.html
• Camera: http://www.fpvhobby.com/63-1-gram-nano- ... 80tvl.html
• Lens: http://www.fpvhobby.com/138-wide-angle- ... amera.html
• Receiver: http://www.fpvhobby.com/20-4-ghz-receiver-module.html
• Digitiser: http://www.amazon.com/Hauppauge-610-USB ... B0036VO2BI
• Some random shift switch

Connections:

• Flie Pin 17 DGNG wired to
  • Regulator GND
  • TX GND
• Flie Pin 16 VCOM wired to a slide switch wired to
  • Regulator VIN
• Regulator VOUT wired to
  • Cam VIN
  • TX VIN
• TX GND_cam and TX VID wired to Cam GND and Cam VID respectively

Software:

• Flie driver node
  • written in python. Communicates with the flie. Advertises crazyflie logs as ROS messages,c hanges params on flie and forwards control commands.
• Dynamic reconfigure
  • interface to change flie params (http://ros.org/wiki/dynamic_reconfigure)
• Flie control node
  • Listens to joystick messages and converts them into flie commands; has different settings (eg sensitivity, etc)
• C++ GUI node
  • listens to logs and camera data, uses QGLWidget to draw everything
• usb_cam node
  • outputs digitized image (http://www.ros.org/wiki/usb_cam)
• image_proc
  • used to rectify the distorted image (http://www.ros.org/wiki/image_proc)
• ps3joy
  • bluetooth driver for sony gamepad (http://www.ros.org/wiki/ps3joy)

Problems:

• Some video dropouts after 2-3 mins when applying high throttle
  • Solution -> Not sure yet, need to experiment
• Weight too far forward
  • Solution -> Design a proper holder for the camera. Work in progress: Id like to print a 3d model that clips under the flie using the 4 1mm holes in each corner. Ive ordered some prints and will need to experiment with different pins to plug into the holes

For now the cam is attached with cheap tape, just to make the proof of concept vid.

The key here was the pololu 5V step up voltage regulator.
I actually tried a 3.3V voltage regulator, but that didnt work very well and there was still lots of noise.
a 22uH inductor helped with most of the noise, but not the drop outs. I was very happy to not have to use an inductor at all, the v regulator seems to drop all the noise without the need of an inductor! If some one is interested in the progress I have recordings of image/flight data of all previous test flights.

Any ideas on extending battery life would be cool. Id still like to see if I could get that 3.3v regulator working. And Id like to use a smaller transmitter - Ive seen some of you use those 2.4ghz nano ones.

Thanks to MShelley for helping out with the soldering
Thanks CVPR @ TU Munich
And thanks to all of you for all our fpv related discussions on this forum

EDIT: links updated
EDIT2: fixing more dead links..advice: dont use dropbox!

[danhamilt1]

That is awesome!!!!

[DesTinY]

Really cool! Great work!
Have you tried to fly in X-mode and place the cam between the motors. Result: No motor in the picture and not only the front motor must carry the most.

BTW: Please check your links.
Thanks for posting your great work. An inspiring all of us.

[omwdunkley]

Just updated the links! Dropbox was acting up.

I had no quick and dirty way of attaching the cam in x-mode;) And I woiuld have to apply the rotation to all the incoming IMU data which I didnt feel like doing last night...

If I were to just yaw the cam 45 degrees you would be right, two motors would be holding it. However, the view is not better. The corner of the flie is relatively wide pretty close to the camera so actually you see less. Wide angle lens..
Ultimately, I would like to have the mass of the cam perfectly centered under the flie.

[tobias]

Your HUD is just amazingly cool! I especially like your PWM representation in the bottom right corner!
Any chance you will share it later?

[omwdunkley]

Thanks!

Yeah I plan to release the code one day. Ill probably re-write the whole thing once I get it working how I like. Still playing with the visual elements and want to make it scale to different resolutions better. Ill also need to try and abstract it enough to fit into your existing framework. Right now it runs as a seperate C++ ROS node. We can discuss integrating it into the client code once Im happy with the overlay drawing code. For now its more of a proof of concept and really hacked together in the least efficient way possible Its possible to cache lots of the drawing too. Gotta read through those Qt guides again.

Realistic is towards the end of the year. If youre interested in seeing unmainted code I could send you a link!

Cheers,
-Oliver

[hsanjuan]

This is amazing Wondering... How easy is it to fly just looking at the image?

[omwdunkley]

This is amazing Wondering... How easy is it to fly just looking at the image?

Thanks!
Its hard You can hover..if you have enough space. I need more practise. But with the short flight times its a little annoying. I just ordered a smaller less power hungry transmitter, that might help.

I printed a 3d model to hold the camera. The first iteration pegs worked out really well, but the camera box was too tight (the cam is 9x9x11 mm³, but that excludes glue - so it did not fit. I would like to save a little more weight too. The material is ridiculously strong with 3x3 mm² crossbeams. Ill rework the design for 2x2 mm² crossbeams and a new camera cage. Something more minimalist..

Old:


Next:

[SuperRoach]

Curved circular camera holder? Genius!

I may need to consult my local hackerspace to get one printed in the future

Thinner the better though really, Is there some way to measure the weight of what you 3d print?

[omwdunkley]

Curved circular camera holder? Genius!
I may need to consult my local hackerspace to get one printed in the future
Thinner the better though really, Is there some way to measure the weight of what you 3d print?

Haha thanks. It will only be genius if it actually works; not sure how to set the tolerances. Trial and error with 3 weeks between each iteration. The idea is that you can use the lens to hold everything in place. File down the protruding prong on the front so that when you screw the lens on tightly it is the right distance from the imaging chip. This should pull the square part of the case against the back of the holder..and in theory hold everything tightly together;)

Regarding printing, I recommend using some 3D selective laser sintering printers like those at shapeways. My university here has some 3d printers but the pins that they produce that click into the flie break off easily. Using 1mm screws solved this issue though. But as they are more fragile than the shapeways models, the print needs to be thicker...and heavier. Nothing is perfect

Weight for the rather thick frame above must be around a gram. My scales are not that accurate;) Putting 3 on the scales together results in a 3g measurement. Nothing I would count on though!