Autonomous Selfie Drone by Crazyflie using Deep Learning Models

Discussions and questions about the Crazyflie Nano Quadcopter
Post Reply
ildoonet
Beginner
Posts: 1
Joined: Thu Aug 10, 2017 4:34 am

Autonomous Selfie Drone by Crazyflie using Deep Learning Models

Post by ildoonet »

Hi Bitcraze folks,

Our team has been working with Crazyflie platform to demonstrate how deep learning algorithms can be applied to robotics especially a nano aerial vehicles; thanks to Bitcraze development team~ :) To verify the reliability of our approach, our team has also applied the same approach to Petrone, made by Byrobot Co., one of the drone manufacturing companies in Korea

As rapid growth of deep learning algorithms has been made, we sincerely believe that autonomous drone tracking system using deep learning techniques will be generally designed and implemented in robotics community in short period of time. We hope that our project can be thought of as a first step towards the Robotics applied with AI

Here are some of the video clips of our demos; please enjoy it.

1) Demo with Crazyflie
https://www.youtube.com/watch?v=DZCguy0uvhU
2) Demo with Petrone (on-going)
https://www.youtube.com/watch?v=nQ6OiCneJdg

Image

Hardware Specifications
[*] Crazyflie 2.0: https://store.bitcraze.io/collections/k ... zyflie-2-0
[*] 3g FPV Camera with transmitter, 5.8GHz 25mW 48CH: https://www.amazon.com/gp/product/B06XB ... UTF8&psc=1
[*] FPV Camera receiver 5.8GHz 40ch RX for Computers: http://laserbgc.com/index.php?route=pro ... duct_id=85
[*] 1g LiPo battery pack, 1s, 100mAh (for FPV camera)

Software Specifications
[*] Embedded Ubuntu 14.04
[*] ROS indigo ver.
[*] Crazyflie ROS repo: https://github.com/whoenig/crazyflie_ros
[*] Petrone ROS repo (our version): https://github.com/ildoonet/PyByrobotPetrone
[*] Openpose ROS repo (our version): https://github.com/ildoonet/ros-openpose
*** Please refer to the README for installation ***
*** For more details about Openpose, please refer to the link below ***
https://github.com/CMU-Perceptual-Compu ... b/openpose
[*] ROS video recorder (our version): https://github.com/ildoonet/ros-video-recorder

Descriptions of Key Features

[*] Face Detection
Using pre-trained data set using deep learning, it detects human face and calculates out the 3d vectors with unit of meter
For the initial flight triggering, the drone must detects human face for 3 seconds.
Be careful if you are wearing a photo necklace, it will detect your photo and rotates motors :)
[*] Human Body Detection
Using Openpose libraries from CMU (please refer to the link above), a drone is able to detect human body structures such as right/left shoulder, wrist, ankle, elbow, neck, etc
Using this technique, the drone can recognize human body and behave accordingly based on the defined flight motion
[*] Special Flight Movements
When a drone detects certain patterns of human body, it delivers special flight motion listed below:
- Fade-away: when a person waves either left/right hand,
- Landing / Flip Motion: when a person raises both hands up

Thank you,

P.S. Our team is eagerly looking for passionate drone engineers and AI researchers who would like to enjoy flying a smart-S drone, For more details regarding to the job description, please refer to the link below.

Job Posting Link: http://robotics-worldwide.1046236.n5.na ... 17224.html
Last edited by ildoonet on Tue Oct 02, 2018 4:52 am, edited 1 time in total.
tobias
Bitcraze
Posts: 2339
Joined: Mon Jan 28, 2013 7:17 pm
Location: Sweden

Re: Autonomous Selfie Drone by Crazyflie using Deep Learning Models

Post by tobias »

Wow, very nicely done!

What kind of processing power do you need to run the algorithms? I guess most of it is pre-trained? Would it be possible to run on a RaspPi? I'm thinking of e.g. testing on this :-).

By the way, would you be interested in writing a guest blog post about your work. Could be based of what your wrote here with a little bit more detail and embedded videos :-). Send us a mail to contact(at)bitcraze.io

Also please add video and if you have a paper to the research portal as well. Can be done with a pull request ;-).
Kavyanjali579
Beginner
Posts: 1
Joined: Thu Jun 06, 2019 8:46 am
Location: Pune
Contact:

Re: Autonomous Selfie Drone by Crazyflie using Deep Learning Models

Post by Kavyanjali579 »

Those were amazing video clips.
Post Reply