[SOLVED] Flashing custom bitstream

Topics related to the Lighthouse positioning system, configuration and use
marvin
Beginner
Posts: 7
Joined: Wed Jan 26, 2022 10:02 am

[SOLVED] Flashing custom bitstream

Post by marvin »

Hi,

currently I'm struggling to flash the ICE40 of the Lighthouse system with a custom bitstream. First tried to to use the

Code: Select all

<paht-to-lighthouse-bootloader>/scripts/uart_bootloader.py /dev/ttyUSB0 lighthouse.bin
command. Is my assumption correct that the

Code: Select all

/dev/ttyUSB0
shall be the Crazyradio PA dongle? But since this device doesn't show up in /dev/ (but the cf-client can connect to crazyflie), I tried it using the cf-client and a zip file with my bitstream. However after the programing, booting the FPGA fails with the message from https://github.com/bitcraze/crazyflie-f ... sher.c#L68.
I'm fairly new to this topic so maybe I'm also missing out on something trivial here.
Last edited by marvin on Thu Feb 10, 2022 3:09 pm, edited 1 time in total.
kristoffer
Bitcraze
Posts: 630
Joined: Tue Jun 30, 2015 7:47 am

Re: Flashing custom bitstream

Post by kristoffer »

Hi!

No that is unfortunately not the case, the documentation was written a long time ago and it is maybe a bit confusing.
This is slightly outside my knowledge, but I think that the uart_bootloader.py script only can be used if the lighthouse deck is connected with an FTDI cable.

There might be an easier way, but you can always flash it from the python client using a zip file.
1. Start from an existing release file, download from https://github.com/bitcraze/crazyflie-release/releases
2. Unpack the zip
3. Replace the lighthouse.bin with your fpga bin
4. Modify the manifest.json file and remove everything in the "files" section, except the lighthouse. I'm unsure if you have to modify the version, maybe
5. Pack to a zip again. Make sure it has the same structure as the original zip, error information is very limited when using the file.
6. Open the client and connect the Crazyflie with the lighthouse deck
7. Open the Bootloader dialog in the Connect/bootloader menu
8. Click Browse and pick you zip
9. Click Program
marvin
Beginner
Posts: 7
Joined: Wed Jan 26, 2022 10:02 am

Re: Flashing custom bitstream

Post by marvin »

Hi,

thanks for you response. But I don't think this worked, since
1. the status of the bootloader stays IDEL and the connection is immedeatly lost when I press program.
2. the LEDs on the LH stay and work as they were with the release file, even though my test bitstream has no logic controlling them.
3. the CRC passes when the drone boots. But with my knowledge of CRC the CRC of my test-bitstream and the one deposited one the drone (CRC of the LH bitstream) should differ.
4. the message "LHFL: Firmware version 6 verified, booting deck!" appaears, even though I have changed version to 2 and realese number to 7. Other combinations have the same result.
kristoffer
Bitcraze
Posts: 630
Joined: Tue Jun 30, 2015 7:47 am

Re: Flashing custom bitstream

Post by kristoffer »

Hi!

I had similar problems at first before I got it to work and as mentioned earlier, the error reporting is non-existing so it pretty much just fails silently.
My mistakes were that the manifest.json was not correct (I had an extra comma). Here is an example of a working json for a bin file called my-new-lighthouse.bin

Code: Select all

{
    "version": 1,
    "subversion": 2,
    "fw_platform": "cf2",
    "release": "2022.01",
    "files": {
        "my-new-lighthouse.bin": {
            "platform": "deck",
            "target": "bcLighthouse4",
            "type": "fw",
            "release": "V6",
            "repository": "lighthouse-fpga"
        }
    }
}
Secondly, I had one level too much in the zip when I packed it. To verify that it your zip is OK you can unpack it again and make sure you get the same structure as with the release zip from github
kristoffer
Bitcraze
Posts: 630
Joined: Tue Jun 30, 2015 7:47 am

Re: Flashing custom bitstream

Post by kristoffer »

Hi again!

A better way is to use cfloader. I was not sure how to do it but this works

Code: Select all

cfloader flash my-new-lighthouse.bin deck-bcLighthouse4-fw -w radio://0/30/2M
marvin
Beginner
Posts: 7
Joined: Wed Jan 26, 2022 10:02 am

Re: Flashing custom bitstream

Post by marvin »

Hi,

thanks for your response. I've tested your suggestions, but I'm still quite certain that it doesn't work. Here are the reason therefore (the test bitstream is unrelated to the LH project):

1. Using the cfloader generates the following output:

Code: Select all

Reset to bootloader mode ...
Could not save cache, no writable directory
Could not save cache, no writable directory
Deck bcLighthouse4 firmware up to date, skipping
2. Using the zip file method has the following stati:
Idle
Restarting firmware to update decks
Detecting deck to be updated
Deck updated. Restarting firmware.
(0/0) Flashing done. Deck update complete.
Idle

I think both outputs indicate that the flashing failed, because the loader thinks the version of the currently flashed bitstream is higher than the one of mine.

3. For both approaches the output of the consol contains:

Code: Select all

LHFL: Bitstream CRC32: 112BC794 [PASS]
LHFL: Firmware version 6 verified, booting deck!
Both statements cant be true in my opinion, since they are generated from the bitstream. Since my test bitstream is unrelated to the LH-bitstream, the CRC cant be the same like the one of the LH ( 112BC794), and I have no verion number in mine.

I'm not sure if it matters, but I'm using iCEcube2 instead of ICEstrom, since ICEstrom cant synthesise a reference design of Lattice I'm using.
kristoffer
Bitcraze
Posts: 630
Joined: Tue Jun 30, 2015 7:47 am

Re: Flashing custom bitstream

Post by kristoffer »

The details of flashing the LH deck are slowly coming back to me (after some digging in the code).

You are right, but there is a solution :-)

So it works like this:
When the Crazyflie boots, the firmware checks the lighthouse binary to understand if it is the correct one or not. If the binary is good, the lighthouse deck is booted with the binary and the deck goes on to do its business. If the binary is not the right one, the deck is started in bootloader mode waiting for a new binary to be flashed. If I remember correctly, there is no way to go from "running deck" to bootloader mode so this has to be done while booting.

The CF firmware contains the checksum of the accepted lighthouse binary and you must change this checksum to force the deck into bootloader mode at startup. You can find the checksum here https://github.com/bitcraze/crazyflie-f ... ghthouse.h.

The full procedure is:
1. Change the checksum to 0
2. Build the CF firmware and flash
3. Boot the CF
4. Flash the lighthouse deck (cfloader flash my-new-lighthouse.bin deck-bcLighthouse4-fw -w radio://0/30/2M)
5. Reboot the CF and check the console log. It should report the checksum as invalid, but more importantly it should give you the cecksum.
6. Change the checksum in lighthouse.h again, to the new checksum
7. build and flash
8. Reboot the Crazyflie
9. The deck should now start and use your binary

It is not obvious, I hope it works :-)
marvin
Beginner
Posts: 7
Joined: Wed Jan 26, 2022 10:02 am

Re: Flashing custom bitstream

Post by marvin »

Hi Kristoffer,

thanks for your effort and respone, but unfortunally it still wont work. It fails, when I try to flash the bitstream with the following message:

Code: Select all

Reset to bootloader mode ...
Could not save cache, no writable directory
Could not save cache, no writable directory
Exception while doing callback on port [4]

Traceback (most recent call last):
  File "/home/bitcraze/.local/lib/python3.8/site-packages/cflib/crazyflie/__init__.py", line 403, in run
    cb.callback(pk)
  File "/home/bitcraze/.local/lib/python3.8/site-packages/cflib/crazyflie/mem/__init__.py", line 540, in _new_packet_cb
    if wreq.write_done(addr):
  File "/home/bitcraze/.local/lib/python3.8/site-packages/cflib/crazyflie/mem/__init__.py", line 220, in write_done
    self._progress_cb(self._get_progress_message(), self._progress)
  File "/home/bitcraze/.local/lib/python3.8/site-packages/cflib/bootloader/__init__.py", line 446, in progress_cb
    print('{} {}% {}'.format(frame, percent, msg))
UnicodeEncodeError: 'latin-1' codec can't encode character '\u25e2' in position 0: ordinal not in range(256)
My assumption: I think I've read somewhere that the bootloader extracts a version number form the bitstream and compares it the the current one (but I don't remember anymore where I read this). Since my bitstream doesn't have such a number and I dont know how to add one, it just tries to decode a part of the bitstream...

Another question I have regarding step 6 (eventhough I didn't get there so far): Am I correct that the LIGHTHOUSE_BITSTREAM_SIZE has also set to the size of the new Bitstream?

Thanks in advance.
kristoffer
Bitcraze
Posts: 630
Joined: Tue Jun 30, 2015 7:47 am

Re: Flashing custom bitstream

Post by kristoffer »

Hi!

I think the flashing part actually is working but there seems to be a problem with character encodings on a python level, I don't know why. Let's try to create a workaround.

Install the python lib from source (if you have not done that yet). You can read here https://github.com/bitcraze/crazyflie-l ... install.md, that is something like

Code: Select all

git clone https://github.com/bitcraze/crazyflie-lib-python.git
pip install -e crazyflie-lib-python
go to the file in your cloned lib that corresponds to this file
https://github.com/bitcraze/crazyflie-l ... #L443-L446

Change the part around line 443 to something like this:

Code: Select all

                    def progress_cb(msg: str, percent: int):
                        # frames = ['◢', '◣', '◤', '◥']
                        # frame = frames[int(percent) % 4]
                        # print('{} {}% {}'.format(frame, percent, msg))
                        print('{}%'.format(percent))
kristoffer
Bitcraze
Posts: 630
Joined: Tue Jun 30, 2015 7:47 am

Re: Flashing custom bitstream

Post by kristoffer »

This has been fixted by Jonas (super fast!) and it should work if you use the latest version of master of crazyflie-lib-python
Locked