Random Behavior while hovering/flying autonoumously

Discussions about autonomous flight in general, regardless of positioning method
aligriv
Beginner
Posts: 5
Joined: Thu Dec 19, 2019 5:36 pm

Random Behavior while hovering/flying autonoumously

Post by aligriv »

Hello all,

Our research lab have decided to use Crazyflies as our platform for controlling autonomous UAVs. We have Optitrack motion capture system in our lab and I'm trying to merge our existing approach with crazyflies. In brief, the problem is that I do not get a consistent result each time I try to run my code and fly the drones. Some times the flight (e.g. hovering) is successful, but most of the time I see random behavior, in particular, the drone takes off and reach its hovering point, stays there for a couple of seconds, and suddenly it drifts off, or reduces its height significantly, and then it becomes unstable and therefore, I have to stop the experiment. In other words, to me it seems that the Crazyflie decides not to follow the control commands at some point, for some reasons that I don't understand.

I am not using the Crazyswarm, and I personally prefer not to do (I am not good with ROS :D). Instead, the architecture for my approach can be briefly explained as following :

1. Thread1: Receives data from motive and computes the outer loop control commands (roll, pitch, yaw, thrust). The rate is almost 120 Hz.
2. Thread2: Sends commands to the crazyflie at a specified rate (I have tried values between 40-100 Hz, no difference)
3. Main Thread which initializes the variables and other classes, and initiates Thread1 and Thread2.
(I am using cflib and everything is implemented in Python)

The main body of my code can be found in https://github.com/AliGriv/Crazyflie_te ... in/Main.py

I know that sometimes the data received from Motive could be flawed, especially since the markers arrangements are a little bit close to each other and they form a relatively symmetric rigid-body. But I am not convinced this could be the source of my problem. The reason is that I do not observe problems such as "axis flipping" or even the plots generated from Motive data seems to be fine.
Also as I mentioned before there is a problem that I have some successful experiments now and then, and sometimes the result is astonishing in terms of position errors (it happens rarely, bit it happens!). Moreover, by monitoring the outer-loop control commands, it seems that my controller structure is also fine (which makes sense because we had used it on some other drones using SPRacingF3/Cleanflight controllers on hand-made drones).

I could not find a similar topic regarding this issue, and I would greatly appreciate if anyone can give me some hints about this problem or re-direct me to the right thread of discussions.

Thank you everyone.
whoenig
Expert
Posts: 377
Joined: Mon Oct 27, 2014 2:55 am

Re: Random Behavior while hovering/flying autonoumously

Post by whoenig »

Do you see the same problem when flying a single CF only?

I had a brief look at your code and I noticed that this is not threadsafe. In order to exchange data, you should either use queues (https://docs.python.org/3/library/queue.html) or semaphores (https://docs.python.org/3/library/threa ... re-objects). Using just events does not suffice, because it doesn't protect the data that you exchange. In principle your design is OK, but you could also just use the single main thread, assuming that your controller computation is fast enough. This would be especially interesting if you do see the problem for a single CF, as it simplifies debugging.
aligriv
Beginner
Posts: 5
Joined: Thu Dec 19, 2019 5:36 pm

Re: Random Behavior while hovering/flying autonoumously

Post by aligriv »

whoenig wrote: Mon Apr 05, 2021 10:37 am Do you see the same problem when flying a single CF only?

I had a brief look at your code and I noticed that this is not threadsafe. In order to exchange data, you should either use queues (https://docs.python.org/3/library/queue.html) or semaphores (https://docs.python.org/3/library/threa ... re-objects). Using just events does not suffice, because it doesn't protect the data that you exchange. In principle your design is OK, but you could also just use the single main thread, assuming that your controller computation is fast enough. This would be especially interesting if you do see the problem for a single CF, as it simplifies debugging.
Thank you for your comment.
Yes, I see the problem when flying a single CF. In order to follow your suggestions, I tried to implement the multi-thread version with Queues, however, I did not get a single stable flight as a result. A confession:I am not a professional programmer and I thought maybe I'm not using queue correctly :D . Hence, I decided to step back, and as you had suggested, use a single thread. My single thread version can be found here: https://github.com/AliGriv/Crazyflie_te ... eThread.py

Unfortunately, the result was the same. I had both successful hovering and unsuccessful flights!

For the sake of the record, here is an example of a successful flight:
The actual position: https://github.com/AliGriv/Crazyflie_te ... opter0.png
The desired position: https://github.com/AliGriv/Crazyflie_te ... sition.png

On the other hand, the same code resulted in an unsuccessful flight like this:
The actual position: https://github.com/AliGriv/Crazyflie_te ... opter0.png
The desired position: https://github.com/AliGriv/Crazyflie_te ... sition.png

Both trajectories were the same, and the code was also the same. I still don't know what could be the source of these random results. I am deeply grateful for your help and will be looking forward to hearing your further suggestions.

P.S: I should've also mentioned that based on my observations, this sort of problem usually happens with (relatively) longer trajectories. For example, for a trajectory where it is supposed to do only ~5 seconds of hovering, this problem is rarely observed.

Regards,
Ali
whoenig
Expert
Posts: 377
Joined: Mon Oct 27, 2014 2:55 am

Re: Random Behavior while hovering/flying autonoumously

Post by whoenig »

From your plots, what is the problem with the unsuccessful flight? It looks like the hover height is not as high as it should be, but otherwise "normal"? It would also be interesting to plot the commands you send (roll, pitch, yaw, thrust), as sudden spikes there could hint to an issue with the mocap or controller.

Your code looks better now, but there is still a potential race between the optitrack callback function and your main control loop. I think in your case it is pretty save to fully ignore this race for now and remove the event portion as well (using the event the way you do can cause strange starvation effects, especially if you mocap frequency is low).
aligriv
Beginner
Posts: 5
Joined: Thu Dec 19, 2019 5:36 pm

Re: Random Behavior while hovering/flying autonoumously

Post by aligriv »

For more clarification, please see this figure (scroll to the right side of the plot if the image is too large):
https://github.com/AliGriv/Crazyflie_te ... values.png
In this plot, the values of the actual and desired position in z-axis are compared. The problem with this flight is after t=25s where the drone starts to descend strangely, while the desired z-position is still at hovering-state. The experiment was stopped manually after the error got large.
I have also the plots for thrust and attitude commands:

Pitch/Roll/Yaw Commands: https://github.com/AliGriv/Crazyflie_te ... mmands.png
Thrust Commands: https://github.com/AliGriv/Crazyflie_te ... thrust.png

I also tried removing the event portion, and the result did not get any better. The same problem was observed. And since now and then I see a normal (stable) flight, I'm still confused about the source of this problem.
kristoffer
Bitcraze
Posts: 472
Joined: Tue Jun 30, 2015 7:47 am

Re: Random Behavior while hovering/flying autonoumously

Post by kristoffer »

It looks like your controller is increasing the thrust when the CF starts to descend (after t=25s) but that it does not help and the CF is continuing down anyway.
Are you running the stock firmware or do you have any modifications/decks that might change the normal behaviour?

I had a quick look at your code and it looks like you (possibly) are lowering the update rate to 2Hz at some point. There is a safety feature in the CF that turns off the motors if setpoints are not received regularly (I think at >2Hz), maybe your problem is related to this? I would expect the CF to fall from the sky in this case though which does not sound like your behaviour.

you could also log the log variable motor.m1 - motor.m4 to see what the the thrust of the motors actually are. It seems as your thrust commands are lost somewhere between your controller and the motors.
aligriv
Beginner
Posts: 5
Joined: Thu Dec 19, 2019 5:36 pm

Re: Random Behavior while hovering/flying autonoumously

Post by aligriv »

kristoffer wrote: Thu Apr 08, 2021 9:32 am It looks like your controller is increasing the thrust when the CF starts to descend (after t=25s) but that it does not help and the CF is continuing down anyway.
Are you running the stock firmware or do you have any modifications/decks that might change the normal behaviour?

I had a quick look at your code and it looks like you (possibly) are lowering the update rate to 2Hz at some point. There is a safety feature in the CF that turns off the motors if setpoints are not received regularly (I think at >2Hz), maybe your problem is related to this? I would expect the CF to fall from the sky in this case though which does not sound like your behaviour.

you could also log the log variable motor.m1 - motor.m4 to see what the the thrust of the motors actually are. It seems as your thrust commands are lost somewhere between your controller and the motors.
Thank you for your reply Sir.

For the first question: Yes. I did not make any modifications to the firmware except changing the gains of the PID controller a little bit. It did not affect the behavior of the CF that much.

Lowering the update rate to 2Hz: That part of the code was a test for something else, I had forgotten to update the code. Please, ignore that part.
Yes, you are right. If I activated that part of the code, the CF would fall suddenly.

Thank you for your suggestion. I ran a new experiment (hovering) and logged motor.m variables. Here are the plots:

Position:
https://github.com/AliGriv/Crazyflie_te ... sition.png

Thrust Command:
https://github.com/AliGriv/Crazyflie_te ... ommand.png

Attitude Commands:
https://github.com/AliGriv/Crazyflie_te ... mmands.png

And finally, motor.m variables:
https://github.com/AliGriv/Crazyflie_te ... iables.png

I'm not sure how to interpret the values of the last plot (motor.m variables). Does it seem that there might be a problem with the way that I control yaw? The commands sent to m2 and m4 keep decreasing while the other two keep going toward the saturation limit.
kristoffer
Bitcraze
Posts: 472
Joined: Tue Jun 30, 2015 7:47 am

Re: Random Behavior while hovering/flying autonoumously

Post by kristoffer »

I can see in your code (https://github.com/AliGriv/Crazyflie_te ... ain.py#L92) that you log the yaw-rate while the Crazyflie set point function taks an absolute yaw. Maybe there is a mixup between rates and absolute angles?
aligriv
Beginner
Posts: 5
Joined: Thu Dec 19, 2019 5:36 pm

Re: Random Behavior while hovering/flying autonoumously

Post by aligriv »

Originally, I thought that CF is taking yawRate as input, and my control input was kp*(measured_yaw-desired_yaw), which in my case the desired value for yaw angle was zero. However, after you mentioned that I should send yaw (not its rate), I still think this method should work if I send a control value proportional to the error in yaw.

For example, one thing I tried was to set yaw_Kp = 2.0, and send yaw_kp*(yaw_error) as a corrective command to the CF.
I also tried to activate EKF on CF and send position/orientation data (received from mocap) to make sure there is nothing wrong with yaw measurements in the CF. However, the result was not different, and random behavior still shows up!. One thing I noticed lately is that apparently, this problem only occurs with hovering, and not when I give it a trajectory to follow (like a straight line or zig-zag).

For example, here, is the position plots of a test that I got today (with EKF enabled and outer-loop yaw_kp = 2.0). You can see at two different times (t1 = 18s , t2 = 62s), CF starts coming down suddenly, but after a few moments, it manages to compensate and comes back to its hovering position. I also have the plots for Yaw values (measured and commanded), motor.m variables and the thrust command.
kristoffer
Bitcraze
Posts: 472
Joined: Tue Jun 30, 2015 7:47 am

Re: Random Behavior while hovering/flying autonoumously

Post by kristoffer »

I'm not completely sure about the details, but in the graph for measured yaw VS command (https://github.com/AliGriv/Crazyflie_te ... ommand.png), it looks like the command increase the error. Should it not be reversed?
Post Reply