CF2.1 firmwire query for machine learning (ML) applications.
-
- Member
- Posts: 36
- Joined: Tue Mar 17, 2020 7:01 pm
CF2.1 firmwire query for machine learning (ML) applications.
Hello,
I was planning to do some machine learning code in the CF2.1 firmwire. I have following query regarding this:
1. How much memory in RAM, and ROM is available to implement ML approach?
2. Please suggest me a ML package which will be suitable for CF2.1 and be compatible with its core? I want to have a solution which will need minimal code change in existing CF2.1 firmwire, and it should also be cheap in terms of memory and computation overhead.
Thanks a lot!
I was planning to do some machine learning code in the CF2.1 firmwire. I have following query regarding this:
1. How much memory in RAM, and ROM is available to implement ML approach?
2. Please suggest me a ML package which will be suitable for CF2.1 and be compatible with its core? I want to have a solution which will need minimal code change in existing CF2.1 firmwire, and it should also be cheap in terms of memory and computation overhead.
Thanks a lot!
Re: CF2.1 firmwire query for machine learning (ML) applications.
I think this blogpost has all the answers you are looking for: https://www.bitcraze.io/2019/10/learnin ... crazyflie/. It explains implementing a small neural network onboard of the stm processor of the crazyflie with TFlite. It also includes a overview of the remaining ram, but you do need to know that that is probably less these days since there is a lot of adde d functionality.
Also consider using the app layer for implementing your NN if you do not want to change too much to the standard firmware.
Also consider using the app layer for implementing your NN if you do not want to change too much to the standard firmware.
-
- Member
- Posts: 36
- Joined: Tue Mar 17, 2020 7:01 pm
Re: CF2.1 firmwire query for machine learning (ML) applications.
Thank you for your reply. How much RAM is available in the current CF2.1 firmwire? And, also how can I find this out myself?
Additional question:
1. I was searching the manual for STM32F405 (https://www.st.com/resource/en/referenc ... ronics.pdf) and found that there are 112KB SRMA1 and 16KB SRAM2 (total: 128KB). But CF2.1 page at https://store.bitcraze.io/products/crazyflie-2-1 says it has 192KB. Which one is correct? How to understand that myself?
2. I was able to verify that the CF2.1 firwire is utilizing SRAM1 only in its current version. What do I have to do to use SRAM2? Can I just augment my code and the additional resource will automatically go SRAM2? Please advise how to use SRAM2.
I have ran arm-none-eabi-size --format=SysV -x cf2.elf in command promt and got the following output (I thought this output may help you answer with more information):
Additional question:
1. I was searching the manual for STM32F405 (https://www.st.com/resource/en/referenc ... ronics.pdf) and found that there are 112KB SRMA1 and 16KB SRAM2 (total: 128KB). But CF2.1 page at https://store.bitcraze.io/products/crazyflie-2-1 says it has 192KB. Which one is correct? How to understand that myself?
2. I was able to verify that the CF2.1 firwire is utilizing SRAM1 only in its current version. What do I have to do to use SRAM2? Can I just augment my code and the additional resource will automatically go SRAM2? Please advise how to use SRAM2.
I have ran arm-none-eabi-size --format=SysV -x cf2.elf in command promt and got the following output (I thought this output may help you answer with more information):
Code: Select all
cf2.elf :
section size addr
.isr_vector 0x188 0x8004000
.flashtext 0x0 0x8004188
.text 0x4d828 0x80041c0
.libc 0x8 0x80519e8
.flashpatch 0x0 0x0
.endflash 0x0 0x0
.data 0x125c 0x20000000
.bss 0x19e48 0x20001260
.nzds 0x28 0x2001b0a8
._usrstack 0x100 0x2001b0d0
.comment 0x31 0x0
.ARM.attributes 0x32 0x0
.debug_aranges 0x5f28 0x0
.debug_info 0x122dee 0x0
.debug_abbrev 0x2a4e2 0x0
.debug_line 0x56168 0x0
.debug_frame 0xfaa8 0x0
.debug_str 0xbe4ab 0x0
.debug_loc 0x4a9ee 0x0
.debug_ranges 0x9238 0x0
.debug_macro 0x3c2fb 0x0
Total 0x36fdbb
Re: CF2.1 firmwire query for machine learning (ML) applications.
Currently I'm unsure how much RAM is still left exactly, but we do know it is getting less and less. This issue on the github repo is also discussing this current problem.
About your questions:
1. The additional 64 kb is called CCRAM, which is used for uploading trajectories for the highlevel commander. There is a separated issue about that on the github repo here to discuss easier accessibility.
2. I think so yes, but I will need to double check this. I'm not sure how the SRAM1 and 2 are handled in the crazyflie.
I believe that the output you showed is more about the flash size than the amount of RAM used in runtime. If you go to the cfclient > Console tab, you can press the button 'task dump', which shows the CPU load and stack left. The latter indicates the load on RAM and might also be an good indication where we can lower the total stack if we assigned too much to a certain module.
I will try to come back with some info, but do also keep an eye on those discussions on the ticket.
Btw, are you aware of the AIdeck coming out soon (https://www.bitcraze.io/2019/10/sneak-p ... e-ai-deck/)
About your questions:
1. The additional 64 kb is called CCRAM, which is used for uploading trajectories for the highlevel commander. There is a separated issue about that on the github repo here to discuss easier accessibility.
2. I think so yes, but I will need to double check this. I'm not sure how the SRAM1 and 2 are handled in the crazyflie.
I believe that the output you showed is more about the flash size than the amount of RAM used in runtime. If you go to the cfclient > Console tab, you can press the button 'task dump', which shows the CPU load and stack left. The latter indicates the load on RAM and might also be an good indication where we can lower the total stack if we assigned too much to a certain module.
I will try to come back with some info, but do also keep an eye on those discussions on the ticket.
Btw, are you aware of the AIdeck coming out soon (https://www.bitcraze.io/2019/10/sneak-p ... e-ai-deck/)
Re: CF2.1 firmwire query for machine learning (ML) applications.
here is some more info with some help of @kristoffer!
When the firmware is build, you see this information:
the bss stands for the ram used by variables. So these are allocated with static task macros (also queues and timers)
There are also task that are allocated dynamically like deck driver tasks which are not showed in the bss. They will be allocated on the heap. This can be seen in the taskdump that I explained before which indicates the stack usage per task.
Hope this is a bit clear?
When the firmware is build, you see this information:
Code: Select all
text data bss dec hex filename
213344 5148 110236 328728 50418 cf2.elf
There are also task that are allocated dynamically like deck driver tasks which are not showed in the bss. They will be allocated on the heap. This can be seen in the taskdump that I explained before which indicates the stack usage per task.
Hope this is a bit clear?
-
- Member
- Posts: 36
- Joined: Tue Mar 17, 2020 7:01 pm
Re: CF2.1 firmwire query for machine learning (ML) applications.
Hello,
I ran the 'task dump' and obtained the following:
Please help me understand the meaning of this. I see different task at the right, and there are stack left for each of the tasks. Does this mean this much stack is left in each task? Which one is the stack available to use? And, also what is the unit of 'stack left'? Is this in bits or bytes or something else?
Followup about previous questios:
1. Can I use CCRAM for my ML codes? If yes, I would like to be able to use this, and I am ok with quick and dirty way to do this. Please let me know what needs to be done to be able to solve this.
2. Could you confirm me how to use SRAM2?
Additional questions:
3. I am thinking about all ways to free up spaces. Could you give me a list of the files which are not used in the CF2.1 but are in the codebase anyway and taking up space? I am not sure if there is any but would like to confirm with you. I am using on regular PID controller (So, I think I can get rid of melinger controller code from firmwire, and so on and so forth). Am I thinking correctly? If yes, then can I curb extra code for other part (for example, I see there are different functions in Kalman filter which are defined for different sensors. But I am using only lighthouse and flow deck. So, I can remove the other parts, and save space right? This is not very convenient but would work for now.)
Thanks.
I ran the 'task dump' and obtained the following:
Code: Select all
SYSLOAD: Task dump
SYSLOAD: Load Stack left Name
SYSLOAD: 2.98 175 Tmr Svc
SYSLOAD: 69.16 127 IDLE
SYSLOAD: 2.53 207 FLOW
SYSLOAD: 0.0 75 PWRMGNT
SYSLOAD: 1.01 214 CRTP-RX
SYSLOAD: 5.94 169 SENSORS
SYSLOAD: 2.15 77 ZRANGER2
SYSLOAD: 0.0 220 MEM
SYSLOAD: 0.0 109 LOG
SYSLOAD: 0.0 91 PARAM
SYSLOAD: 0.0 177 LH
SYSLOAD: 0.05 158 SYSTEM
SYSLOAD: 0.04 62 CRTP-TX
SYSLOAD: 2.96 20 SYSLINK
SYSLOAD: 4.64 311 STABILIZE
SYSLOAD: 8.47 258 KALMAN
SYSLOAD: 0.0 113 USBLINK
SYSLOAD: 0.0 204 CMDHL
Followup about previous questios:
1. Can I use CCRAM for my ML codes? If yes, I would like to be able to use this, and I am ok with quick and dirty way to do this. Please let me know what needs to be done to be able to solve this.
2. Could you confirm me how to use SRAM2?
Additional questions:
3. I am thinking about all ways to free up spaces. Could you give me a list of the files which are not used in the CF2.1 but are in the codebase anyway and taking up space? I am not sure if there is any but would like to confirm with you. I am using on regular PID controller (So, I think I can get rid of melinger controller code from firmwire, and so on and so forth). Am I thinking correctly? If yes, then can I curb extra code for other part (for example, I see there are different functions in Kalman filter which are defined for different sensors. But I am using only lighthouse and flow deck. So, I can remove the other parts, and save space right? This is not very convenient but would work for now.)
Thanks.
Re: CF2.1 firmwire query for machine learning (ML) applications.
Hi,
I think using the CCM is the way to go. It can not be used for storing instructions, only data, but should not be a problem? I know we played a bit with this before but I can not find any traces of it in the code or linker files so it should be all free to use. The linker scripts needs to be updated and the variables using it needs to be attributed but should not be hard to do. Found two good resources here and here.
I think using the CCM is the way to go. It can not be used for storing instructions, only data, but should not be a problem? I know we played a bit with this before but I can not find any traces of it in the code or linker files so it should be all free to use. The linker scripts needs to be updated and the variables using it needs to be attributed but should not be hard to do. Found two good resources here and here.
Re: CF2.1 firmwire query for machine learning (ML) applications.
I was wrong about not being able to execute from the CCM. Found this AN4296 app note that explains the CCM in detail.
"The CCM SRAM is tightly coupled with the Arm® Cortex® core, to execute the code at the maximum system clock
frequency without any wait-state penalty. This also brings a significant decrease of the critical task execution time,
compared to code execution from Flash memory."
"The CCM SRAM is tightly coupled with the Arm® Cortex® core, to execute the code at the maximum system clock
frequency without any wait-state penalty. This also brings a significant decrease of the critical task execution time,
compared to code execution from Flash memory."
-
- Member
- Posts: 36
- Joined: Tue Mar 17, 2020 7:01 pm
Re: CF2.1 firmwire query for machine learning (ML) applications.
Thank you. I will go through the material you shared, and will comeback if I have any queries. In the meantime, I have following query:
Where is the linker file located in the firmwire? I found several files (having .ld extension) in the whole directory.
Additionally, do I have update any startup file? If yes, where is that file in the firmware?
Also I asked following queries before but did not receive your answer.
Where is the linker file located in the firmwire? I found several files (having .ld extension) in the whole directory.
Additionally, do I have update any startup file? If yes, where is that file in the firmware?
Also I asked following queries before but did not receive your answer.
Please help me understand the meaning of this. I see different task at the right, and there are stack left for each of the tasks. Does this mean this much stack is left in each task? Which one is the stack available to use? And, also what is the unit of 'stack left'? Is this in bits or bytes or something else?
2. Could you confirm me how to use SRAM2?
Additional questions:
3. I am thinking about all ways to free up spaces. Could you give me a list of the files which are not used in the CF2.1 but are in the codebase anyway and taking up space? I am not sure if there is any but would like to confirm with you. I am using on regular PID controller (So, I think I can get rid of melinger controller code from firmwire, and so on and so forth). Am I thinking correctly? If yes, then can I curb extra code for other part (for example, I see there are different functions in Kalman filter which are defined for different sensors. But I am using only lighthouse and flow deck. So, I can remove the other parts, and save space right? This is not very convenient but would work for now.)
Re: CF2.1 firmwire query for machine learning (ML) applications.
The linker files in use for CF2.X are located here and the startup script here. If startup file might have to be changed to zero fill the CCM, this was described in one of the links.
2. I'm pretty sure SRAM2 (16k) is already in use. As I've understood it the reason for having this is so that the bus matrix could be utilized better e.g. when using DMA but it also requires designing the code so it has an effect. Not interesting for you I think.
3. It would be a quite big work to compile a list like that. There is plenty of flash so no reason to exclude code for that but could free static mem. Is the 64k CCM not enough?
2. I'm pretty sure SRAM2 (16k) is already in use. As I've understood it the reason for having this is so that the bus matrix could be utilized better e.g. when using DMA but it also requires designing the code so it has an effect. Not interesting for you I think.
3. It would be a quite big work to compile a list like that. There is plenty of flash so no reason to exclude code for that but could free static mem. Is the 64k CCM not enough?