Crazyflie sometimes can't boot after flashing the firmware

Firmware/software/electronics/mechanics
Post Reply
MadeInPierre
Beginner
Posts: 7
Joined: Thu Sep 05, 2019 2:24 pm

Crazyflie sometimes can't boot after flashing the firmware

Post by MadeInPierre » Thu Oct 10, 2019 1:27 pm

Hi all,

I am currently modifying the crazyflie firmware for my own use case, and I use the regular `make cload` command to flash the crazyflie.

The problem I am experiencing is that the drone sometimes fails to boot after flashing the firmware, even though the same code runs on another drone or on the same drone after a few tries. I sometimes have to flash the firmware up to 10 times until the drone successfuly boots, which gets pretty tedious to do...

Does someone have an explanation on why this happens? Could it be a packet loss while transfering the firmware through the crazyradio, or some other cause? Would you have advices on how to reduce the failure rate?
Thanks!

P.S: As a side question, I couldn't find a way to flash the drone through a regular USB cable but this could maybe help to solve the issue?



crazypie
Beginner
Posts: 1
Joined: Wed Oct 16, 2019 5:55 pm

Re: Crazyflie sometimes can't boot after flashing the firmware

Post by crazypie » Wed Oct 16, 2019 6:02 pm

I have experienced the same issue and don't have a clear explanation either.
Do you also see two red LEDs in this case?

arnaud
Bitcraze
Posts: 2275
Joined: Tue Feb 06, 2007 12:36 pm

Re: Crazyflie sometimes can't boot after flashing the firmware

Post by arnaud » Thu Oct 17, 2019 8:29 am

Hi,

We have experienced some problems flashing when we flash demos, but usually the fail rate is more like 1 failed flashing every ~0 times. But I can confirm that there is a reliability problem in the bootloader that should be addressed. The first step would be to get working on this ticket: https://github.com/bitcraze/crazyflie-c ... issues/346. I will have a look at it.

One very important thing is to make sure that Crazyflies are flashed one by one and not together: when starting the bootloader mode manually all Crazyflie listen to the same address/channel and if more than one is in bootloader mode they will affect each-other and flashing will most likely fail for both.

There could also be another device in the neighborhood transmitting packet on this channel and address (we use the nRF24 default address).

One way to make flashing more reliable is to use the warm-boot functionality: in this mode the PC ask the Crazyflie to restart in bootloader mode and the bootloader is using a random address. This removes the interference between crazyflies. This is the command that will flash in warm boot mode:

Code: Select all

python3 -m cfloader flash cf2.bin stm32-fw -w radio://0/10/2M/E7E7E7E701
This is the kind of script we have been using to flash multiple Crazyfles in our demos: https://github.com/bitcraze/crazyflie-f ... oad-all.sh

MadeInPierre
Beginner
Posts: 7
Joined: Thu Sep 05, 2019 2:24 pm

Re: Crazyflie sometimes can't boot after flashing the firmware

Post by MadeInPierre » Tue Oct 22, 2019 2:15 pm

crazypie wrote:
Wed Oct 16, 2019 6:02 pm
I have experienced the same issue and don't have a clear explanation either.
Do you also see two red LEDs in this case?
I have two possible outcomes after a flash:
- Either only the NRF51 LED only turns on (bottom left blue LED) and all the others are turned off, I assume the STM32 can't even boot. This happens most of the time.
- Sometimes I also have the two red LEDs lit up (just after a LED sequence where the two green ones light up and then the red ones for a moment) and the Crazyflie isn't responsive at all either.

This usually doesn't happen very often when I flash the latest official Crazyflie firmware, but it is now being really present when I develop my code in top of the current firmware. Do you maybe have some clue on why this could happen and do you have any solution to avoid (part of) it ?
Thank you!

MadeInPierre
Beginner
Posts: 7
Joined: Thu Sep 05, 2019 2:24 pm

Re: Crazyflie sometimes can't boot after flashing the firmware

Post by MadeInPierre » Tue Oct 22, 2019 2:47 pm

arnaud wrote:
Thu Oct 17, 2019 8:29 am
We have experienced some problems flashing when we flash demos, but usually the fail rate is more like 1 failed flashing every ~0 times. But I can confirm that there is a reliability problem in the bootloader that should be addressed. The first step would be to get working on this ticket: https://github.com/bitcraze/crazyflie-c ... issues/346. I will have a look at it.
This would really be useful. Just to understand what is currently done, does the flashing have any protection or way to verify that the firmware has been correctly flashed ? Does it have a checksum or some other method while its flashing? The 'Verify' option in the cfclient isn't enabled so I wonder if there's a way to use it.

I'm indeed only flashing one CF at a time. And good point for the other devices around, also there's a lot of devices using 2.4GHz (bluetooth, wifi...) so that probably doesn't help...

Another question about flashing, maybe you have some experience on this, would the crazyflies maybe only be made for a certain number of flashes ? Could it be a hardware problem and that they start wearing out ? I regularly have to change the drone I use because one sometimes just doesn't want to start up even after 10 tries. Maybe you have a clue on why this phenomenon could happen?
Thank you for your help.

arnaud
Bitcraze
Posts: 2275
Joined: Tue Feb 06, 2007 12:36 pm

Re: Crazyflie sometimes can't boot after flashing the firmware

Post by arnaud » Wed Oct 23, 2019 6:05 am

Hi,

One quick note about LEDs: the only LED handled by the nRF51 is the back right blue LED, every other LED are handled by the STM32. The LED state is indeed a very good indicator of what is booting or not. The back blue LED is switched ON early during the STM32 boot so what you are describing looks like a corrupted STM32 firmware that has just enough healthy code to light-up the LED.

In the current implementation I did not use any checksum since it was deemed 'safe by design' (famous last word ....). The idea is that the firmware is transfered using the radio and the radio has a CRC attached to each packet: if a packet does not make it intact to the Crazyflie it will not be acked and it will then be resent by the PC. This is why I have usually considered uplink (PC to Crazyflie) safe. The bootloader can read back the flash and that is what the verify button should do (I think it broke at some point and was never re-enabled). Unfortunatly the current bootloader does not have the capabilities to checksum the flash.

I do not think you are hitting hardware limit. The STM32F405 datasheet gives the flash for minimum 10k erase cycle.

One thing I have discovered recently though is that it works much worst on Mac than on Linux: as I said earlier I very rarely have the problem and I am on Linux, we where working with a Mac yesterday and we had very similar problem to you, having to recompile and re-flash many time before succeeding. What OS are you using on your computer?

MadeInPierre
Beginner
Posts: 7
Joined: Thu Sep 05, 2019 2:24 pm

Re: Crazyflie sometimes can't boot after flashing the firmware

Post by MadeInPierre » Wed Oct 23, 2019 9:06 am

Hi, thank you for your help and quick reply!

Yes the LEDs are an excellent design choice! I see, unfortunately I get this quite often now, maybe I'm around too many 2.4GHz devices. Do you know what the two red LEDs mean on the other hand? Is it a fail in the same way but just happens later in the code?

I see, it indeed seems sufficient for sending correct packets, I wonder why this isn't enough then. Does the bootloader notify the user when too many packets have not received an ACK ? If it does, how is it possible that cfloader completes the flashing without noticing something went wrong (e.g. no ack received)?

I'm using Linux too but thanks for sharing, good to know just in case. One thing I noticed is that the official firmware usually flashes fine, but my modified one is the one that's causing trouble (between 1 to 10 tries before it boots up, sometimes one CF just doesn't want to be flashed for the whole day while the same code runs on another). My modified firmware is currently at 319 pages (the official one is 311 pages), can this small increase affect the flashing success rate that much? Do you maybe have other clues on why this could happen?

MadeInPierre
Beginner
Posts: 7
Joined: Thu Sep 05, 2019 2:24 pm

Re: Crazyflie sometimes can't boot after flashing the firmware

Post by MadeInPierre » Mon Oct 28, 2019 4:33 pm

Hi,
Just to update this post, I just received the JTAG Adapter & STLink V2 Debugger and flashing through this works like a charm! I have not had a single flash error until now.
I guess something is happening when flashing the CF with the radio, i hope the reason will be found out some day... if I can do some tests I'd gladly try to help.

For now I'll continue working using the JTAG method. Thank you for your help!

Post Reply