Would crazyflie work for reinforcement learning?

Discussions and questions about the Crazyflie Nano Quadcopter
Post Reply
rmp1
Beginner
Posts: 1
Joined: Tue Mar 27, 2018 9:55 pm

Would crazyflie work for reinforcement learning?

Post by rmp1 » Tue Mar 27, 2018 10:31 pm

I am considering getting a crazyflie w/ flowdeck, primarily to test reinforcement learning. I have a few questions about the system:

It looks like the standard crazyflie python controller (e.g. motioncommander) allows for giving commands in terms of either throttle/yaw/pitch/roll or move/rotate commands. There is no way to directly control the 4 motor outputs though correct? In which case, would it be possible to implement something that does this by editing the firmware?

Also, what sort of feedback is available on the python level? I don't see any real feedback in motioncommander. I would ideally need at least x/y/z/theta1/theta2/theta3 feedback on a timescale similar to the control signal. Is there a way to get these from the Kalman filter from the drone?

The goal of the project would be to autonomously learn a flight controller (rather than using the inbuilt firmware one) - so the more data available, the better!

arnaud
Bitcraze
Posts: 1650
Joined: Tue Feb 06, 2007 12:36 pm

Re: Would crazyflie work for reinforcement learning?

Post by arnaud » Wed Apr 04, 2018 7:07 am

Hi and welcome!

The firmware is open-source so everything is possible :). There is parameter variables available to set individual motors raw values, though it is mostly intended for debugging and so it requires to send 1 radio packet per motor, if you need better efficiency/latency you could modify the stabilization loop so that you can send all raw motors PWM values as a single commander packet.

All the feedback is implemented in the Crazyflie by default which makes things easier to control. One of the use-case for the Crazyflie is to fly swarms and for that purpose the more you do onboard the better (currently we are merging a high level commander form Crazyswarm so even the trajectories will not have to be controlled from the ground :). But it is possible to control the Crazyflie at as low level as you want and the log subsystem allows you to pull sensor values at up to 100Hz by default. The only hard limitation is the latency and bandwidth of the radio link: I expect about 1 to 2ms of round-trip delay in the radio link and up to about 1000 packet per seconds. For this reason we usually only go as low as controlling the attitude, maybe attitude rate, from the radio. Controlling the motors PWM directly might be a challenge. Though there was someone on the forum that ran an LQR controller entirely off-board so it looks possible.

The project sounds very interesting. If you need a lot of data with low latency you could consider running your code in the Crazyflie firmware, if possible.

Post Reply