downloading a log file from the SD

Firmware/software/electronics/mechanics
Post Reply
wydmynd
Member
Posts: 87
Joined: Thu Dec 06, 2018 9:25 am

downloading a log file from the SD

Post by wydmynd »

Hello. I noticed that in crazyflie_cpp (equivalent of cflib, the base for crazyflie_ros) there is a function
that downloads the latest log file from the SD card over radio / usb.
I was looking for the equivalent method in cfLib , if there is any. if not, where should I start (interfaces) when trying to develop such method?
thanks!
whoenig
Expert
Posts: 395
Joined: Mon Oct 27, 2014 2:55 am

Re: downloading a log file from the SD

Post by whoenig »

This is not implemented in cflib, yet. In the firmware, the memory type is defined here: https://github.com/bitcraze/crazyflie-f ... /mem.h#L41. On the cflib side, this memory type is not yet included (see https://github.com/bitcraze/crazyflie-l ... py#L30-L38 for a supported list). It should be possible to add it by following the logic of the memory tester type. The resulting data that you read from this memory type, needs to be written to a binary file.
wydmynd
Member
Posts: 87
Joined: Thu Dec 06, 2018 9:25 am

Re: downloading a log file from the SD

Post by wydmynd »

retrying if anyone has insights..
I dug a bit deeper to find -

Code: Select all

  enum MemoryType {
    MemoryTypeEEPROM  = 0x00,
    MemoryTypeOneWire = 0x01,
    MemoryTypeLED12   = 0x10,
    MemoryTypeLOCO    = 0x11,
    MemoryTypeTRAJ    = 0x12,
    MemoryTypeLOCO2   = 0x13,
    MemoryTypeLH      = 0x14,
    MemoryTypeTester  = 0x15,
    MemoryTypeUSD     = 0x16, // Crazyswarm experimental
  };
so I think I can use the "memory" class in cflib which currently has these types -

Code: Select all

class MemoryElement(object):
    """A memory """

    TYPE_I2C = 0
    TYPE_1W = 1
    TYPE_DRIVER_LED = 0x10
    TYPE_LOCO = 0x11
    TYPE_TRAJ = 0x12
    TYPE_LOCO2 = 0x13
    TYPE_LH = 0x14
    TYPE_MEMORY_TESTER = 0x15
    TYPE_DRIVER_LEDTIMING = 0x17
I am trying to add a new type and then follow one of the read examples and modify it in some way to support the SD card memory type

is this the right direction?
thanks
wydmynd
Member
Posts: 87
Joined: Thu Dec 06, 2018 9:25 am

Re: downloading a log file from the SD

Post by wydmynd »

Thanks for wyour reply whoenig! I wrote the post and found out about your answer only later.. :oops:

I have a few specific questions after studying your answer-

first - I understand the usddeckRead method used in the firmware is relevant only for an active logging session which was stopped but the drone was not rebooted since. This because the file name referenced is usdLogConfig.filename correct?

second - this also means it will open the file, read 24 bytes, close the file, over and over again. I assume this is why the method is so slow? If so , I assume it is possible to write a method able to read larger chunks using the large buffer already allocated usdLogBuffer.

Thanks in advance
whoenig
Expert
Posts: 395
Joined: Mon Oct 27, 2014 2:55 am

Re: downloading a log file from the SD

Post by whoenig »

1. This is correct and can be an annoying limitation depending on your application. Improving this situation would require additional firmware changes.

2. The limit to 24 bytes is because of the CRTP + memory protocol format (and a physical limitation of the Crazyradio PA to only allow payloads of up to 32 Bytes). I actually never did any profiling, but my guess has been that opening/closing the file is not the limitation here over the radio bottleneck. It would be interesting to run some targeted tests for that, though. (As you said, transferring a file over the radio is very slow.) I do agree that using the usdLogBuffer as a cache could help in that case (assuming the client requests the data in the file sequentially.)
wydmynd
Member
Posts: 87
Joined: Thu Dec 06, 2018 9:25 am

Re: downloading a log file from the SD

Post by wydmynd »

just a quick update, after some tests with crazyflie_tools, downloading using radio or USB does not matter much in terms of speed, on USB I get 4-9KBps and on radio around 7KBps. I started writing the python functions to allow downloading and will update again.
wydmynd
Member
Posts: 87
Joined: Thu Dec 06, 2018 9:25 am

Re: downloading a log file from the SD

Post by wydmynd »

update 2: it works! the only thing is that the rate is ~1.5KB/sec (both radio and USB). in the CPP implementation by whoenig it was ~3 times faster.
anyway I am considering making some improvements for performance, any suggestions welcome.
the modified code is posted as a PR in the cflib github
whoenig
Expert
Posts: 395
Joined: Mon Oct 27, 2014 2:55 am

Re: downloading a log file from the SD

Post by whoenig »

Thanks for the PR - we'll take a look (it will take a few days).

For the speed we are aware that Python does not seem to be able to use the radio as efficiently as the CPP version. One option we are currently working on is a native C++ backend for the lowest-level communication with the Crazyradio over USB. I think your new function will be an excellent test case to see if this new backend will improve the overall performance for high-level functions. There are a few other differences between the C++ code and the Python code (e.g., the C++ code does not use the safelink feature), which might have an impact, too.

Finally, the other developers confirmed yesterday that opening/closing the file has a high impact at least on the SPI bus utilization. Thus, your earlier idea of using the buffer to avoid opening/closing the file might help significantly.
Post Reply