Frequent fly aways, toiletbowling and flips with Lighthouse positioning

Discussions about autonomous flight in general, regardless of positioning method
hanz
Beginner
Posts: 10
Joined: Wed Nov 13, 2019 10:02 pm

Frequent fly aways, toiletbowling and flips with Lighthouse positioning

Post by hanz » Mon Nov 25, 2019 11:21 pm

Hello,

We have been working with the Crazyflie Lighthouse positioning system for the last couple of months and these problems has been haunting us.
There seems to be no real pattern to these failures, the takeoff->goto->land sequence works great sometimes, others fails miserably.
We have 6 functioning crazyflies with the exact same hardware - this does not seem like a hardware-related issue.

setup steps:
- Basestation positions are written and verified
- "stabilizer.controller" is set to 2 and verified
- "commander.enHighLevel" is set to 1 and verified
- "kalman.resetEstimation" is set to 1 and then 0 100ms later
- program waits for kalman variance to trend under 0.006

The basestations are at the "top right" and "bottom left" of the lighthouse calibration screen, large space.
Calibration done with Bitcraze latest script and the headset in the middle pointed to x.

sequence:
- takeoff(1.0, 1)
- goto(0.6, 0, 0, 0, 2, relative=False) example, changed for each drone
- land(0, 1) negative height tested to reduce bounce causing unwanted rotation

Our code works fine simulated, but we barley get a single good flight on the actual drones.
videos of some recent failures for reference: https://imgur.com/a/7BErlq6

failures observed:
- YAW twitching/drifting - slight movements which seems to come from the drone itself (ie. not outside forces) thows off the kalman filter and causes a crash/drift
- instant flip of death - either directly on the ground or ~10-20cm in the air
- tolietbowling - the drone circles around a somewhat-constant point for a couple of seconds before flying away
- fly aways - the most common issue. everything goes ok till the drone start to veer off course, slow and smooth movements until its outside the lighthouse area where it throws itself to the nearest wall.

Is this something experienced by the Bitcraze team or any other Lighthouse users?
If not, any recommendations for debugging this issue?



arnaud
Bitcraze
Posts: 2275
Joined: Tue Feb 06, 2007 12:36 pm

Re: Frequent fly aways, toiletbowling and flips with Lighthouse positioning

Post by arnaud » Tue Nov 26, 2019 3:17 pm

Hi,

No we have not observed such results: lighthouse works almost all the time for us and for example when running out fair demo we have had Crazyflies running for a full day without ever crashing (accumulating 1h of flight time in 8h) and we normally have up to about 1 or 2 random crash per hours for our 9 Crazyflies. This is the kind of reliability we have been observing.

What version of the firmware are you using? The lighthouse implementation has been in flux lately so it might be good to use the latest master.

The first thing I would check is if the geometry is well setup in the Crazylfie. One way to check it is to observe the value of the log variable "lighthouse.delta". This variables contains the distance between the two lighthouse beams, in a properly setup system it should be of a couple of centimeters (we see less than 10cm in our system, less than 2cm with a fix that we will push today).

hanz
Beginner
Posts: 10
Joined: Wed Nov 13, 2019 10:02 pm

Re: Frequent fly aways, toiletbowling and flips with Lighthouse positioning

Post by hanz » Wed Dec 11, 2019 11:47 pm

Hey!

So we flashed all our drones with the latest master at the time (5bcbb5e950b9c681c6d7ea8bbd228b9d33b54620) and everything worked fine for 2-3 sessions.
Today when we setup the system our issues returned though.

Our new (and likely old) problem seems to be related to the initial yaw position. The estimated yaw position is consistently off by 60~70 degrees from the x-axis.
The odd thing though is that we get great X, Y and Z estimated positions that perfectly matches 0, 0, 0 with the headsets position at the time of calibration.
Worth nothing is that the yaw rotation estimate matched perfectly with the x-axis when the system was working as expected.

CFlib is at latest master and tested with 0.1.8 pip release.

We wouldn't mind giving up some functionality and accuracy for stability at this point.
Is there a firmware relase/commit that you guys consider 'stable'?

Maybe related to firmware issues #517 or #522?

arnaud
Bitcraze
Posts: 2275
Joined: Tue Feb 06, 2007 12:36 pm

Re: Frequent fly aways, toiletbowling and flips with Lighthouse positioning

Post by arnaud » Thu Dec 12, 2019 6:47 am

At the time of your initial message I beleive there was no yaw measurement taken from the lighthouse system, this meant that initial YAW was indeed very important to get a stable flight.

With newer firmware we do measure the YAW from the system and so the initial YAW should not be a problem anymore.

There is another post that might look like your problem, a video has been posted there, does your problems look the same? viewtopic.php?f=18&t=4042&p=18416#p18395

The key here is to look at the YAW: if the Crazyflie has an incorrect YAW estimate, X/Y control become compromised (there is cross-talk in the x/y axis...). So if you fly all your Crazyflies with a YAW setpoint of 0 and they are not flying heading the same direction, it indicates a similar problem.

Issue #517 is likely unrelated. Issue #522 is what we think is the cause of the problem in the other thread.

hanz
Beginner
Posts: 10
Joined: Wed Nov 13, 2019 10:02 pm

Re: Frequent fly aways, toiletbowling and flips with Lighthouse positioning

Post by hanz » Thu Dec 12, 2019 11:13 am

Yeah the yaw offset video in the linked thread is identical to our issue.

We do have a bunch of testing that we'd like to get done in the near future, is #522 nearly solved or should we revert to an older commit?
If we need to revert, what commit do you recommend that we use?

arnaud
Bitcraze
Posts: 2275
Joined: Tue Feb 06, 2007 12:36 pm

Re: Frequent fly aways, toiletbowling and flips with Lighthouse positioning

Post by arnaud » Fri Dec 13, 2019 7:18 am

We have not had time to look at the issue #522 yet.

If the yaw estimation is the problem, you can comment this line to disable it: https://github.com/bitcraze/crazyflie-f ... use.c#L306. In that case you must of course make sure you start and reset the kalman filter with the Crazyflies facing X.

arnaud
Bitcraze
Posts: 2275
Joined: Tue Feb 06, 2007 12:36 pm

Re: Frequent fly aways, toiletbowling and flips with Lighthouse positioning

Post by arnaud » Mon Dec 16, 2019 6:49 am

A quick update, I ran some test and if there is any sensor that do not output data on the deck, the yaw code will indeed cause problem.

The easiest way to disable it (without compiler warning) is to change this line: https://github.com/bitcraze/crazyflie-f ... use.c#L304 into:

Code: Select all

if (false && estimateYawDeltaOneBaseStation(baseStation, angles, lighthouseBaseStationsGeometry, cfPos, n, &RR, &yawDelta)) {
This way the lighthouse YAW measurement will be disabled.

hanz
Beginner
Posts: 10
Joined: Wed Nov 13, 2019 10:02 pm

Re: Frequent fly aways, toiletbowling and flips with Lighthouse positioning

Post by hanz » Fri Dec 20, 2019 2:56 pm

Thanks,
We tried this today but without any luck.

Just to make sure our code wasn't to blame i tested the CFlib "autonomous_sequence_high_level.py" with lighthouse basestation positions hard-coded in the lighthouse deck driver.
Everything flew great and we experienced no issues, even with yaw estimation (ie. #522).

So after all #517 seems to be the culprit of our issues.
The euler angles and inversions of rotation matrixes are not calculated so the lighthouse data is of no help.

We are trying to solve this from our side but obviously it would be great if this could be fixed officially in the firmware.
Was this overlooked or could the current code actually work fine if the deck was initialized/ordered in another way?

kimberly
Bitcraze
Posts: 220
Joined: Fri Jul 06, 2018 11:13 am

Re: Frequent fly aways, toiletbowling and flips with Lighthouse positioning

Post by kimberly » Mon Dec 23, 2019 10:47 am

Hi!

Arnaud is on his christmas holiday now. The rest of us will go ourselves soon but let's see if I will be able to help to shed a different light on it.

I checked your videos, and there might be something else going on. How far are your base stations apart and how high are they mounted? Are you sure the crazy flies are detecting both base stations?

It might not be it but at least we can investigate other probably causes.

arnaud
Bitcraze
Posts: 2275
Joined: Tue Feb 06, 2007 12:36 pm

Re: Frequent fly aways, toiletbowling and flips with Lighthouse positioning

Post by arnaud » Fri Dec 27, 2019 11:09 am

I did not understand that you sent lighthouse geometry via the radio. If you do so in your script #517 indeed breaks this functionality and is the issue. In that case hardcoding the lighthouse geometry in the Crazyflies is the quickest workaround for now.

Post Reply