HITL and SITL for crazyflie

Discussions about all things Bitcraze
burakhocaoglu
Beginner
Posts: 5
Joined: Wed Jun 06, 2018 1:34 pm

Re: HITL and SITL for crazyflie

Post by burakhocaoglu »

wuwushrek wrote: Wed Jun 06, 2018 5:21 pm Hi,
Interesting what you are saying ! Before testing that path (and Yes it is possible to implement something like that ), I'm currently working on another alternative. Maybe I will combine the two at the end (?).
I will keep you updated when things will be better with multiple cf.
Hi again,

Sorry for late update. In the meantime, I tried to implement a router layer that will transmit CRTP packets from Crazyflie(s) to SITL. I have used simple python SocketServer API; however, the ACK packets sent from SITL to Crazyflie(s) have faced timeout issues in my every experimental run. Should I try socket API of C (I am not sure it will increase the speed of networking in local computer)? Should I send you my source code for Router (or the whole package with my modifications explained)?

Before implementing Router layer, I tried to connect 2 Crazyflies to the same SITL by setting the destination ports the same, but I could not get RESET KALMAN FILTERS DONE and therefore, could not make any of Crazyflies fly. Also, I tried to open another socket in SITL and tried to connect 2 Crazyflies to those sockets accordingly. This requires a new task to be opened, though. However, I think there is a risk of miscommunication as the task management in SITL is set to process 1 packet at a time (socketlinkTask calls socketlinkReceiveCRTPPacket and socketlinkSendPacket). Even though it somewhat queues the packets, SITL does not actually know whose packet it deals with, right? At least that's what I understood from it.

I'd be glad if you tell us what you are working on SITL to improve the performance.

Burak
wuwushrek
Beginner
Posts: 15
Joined: Mon Apr 16, 2018 1:05 pm

Re: HITL and SITL for crazyflie

Post by wuwushrek »

Hi,

Sorry I was busy recently on my main project. However if you check my github you will see on the multi-cf-test branch the last modifications I made that work pretty well with at least 4 crazyflies (I included a launch file and a python script in order to test it).Please Tell me how it behaves on your computer !!! I haven't try more than 4 quads yet ! I will have some time for it tomorrow.
The thing is, with already 4 Cfs my FPS goes at 40 and the real time factor in gazebo is not steady one. Actually just launching 4 cf2 instance (without gazebo ) uses more than one 1 CPU cores. I don't think I can do better on that part with the constraints to be as close as possible from the real cf2 behavior). That's the main goal ! Anyway I still have some work to do on this tomorrow.

No using the socket API of C won't change anything ! Yes please send me your code source and some explanation I will understand it better.
Why are you trying to connect 2 cfs on the same port ? It shouldn't be possible ! One of them won't start !
Yes the gazebo plugin does minimalist things ! Just sending sensors information and receiving motors data ! He redirects everything else rob the crazyflieROS that works in a parallel thread.
wuwushrek
Beginner
Posts: 15
Joined: Mon Apr 16, 2018 1:05 pm

Re: HITL and SITL for crazyflie

Post by wuwushrek »

OK it seems the head of my submodules are detached for my multi-uav-fix branch. You need to go inside crazyflie-firmware/ and crazyflie_gazebo/external/crazyflie_comm/ and make this submodules to point to their master branch in order to be able to test the multi-uav-fix branch.
I didn't put any information about it on the github because I'm actually refactoring few things here based on what we talked about here and some benchmark I made. I may have some good results by friday :)
tugayalperen
Beginner
Posts: 19
Joined: Mon May 14, 2018 8:33 am

Re: HITL and SITL for crazyflie

Post by tugayalperen »

Hi,

We manage to work multi-uav-fix branch and in our case, it is working perfectly with 3 cfs but for 4 cfs, it sometimes crashes at some point. Yet multi cf performance is developed significantly, thank you again for this :D Apart from this, everything seems fine.

Hope to hear from you again
wuwushrek
Beginner
Posts: 15
Joined: Mon Apr 16, 2018 1:05 pm

Re: HITL and SITL for crazyflie

Post by wuwushrek »

Hi,
I have recently push some modifications on the multi-uav-final branch. You will have to clone/pull the project and go to that branch for sim_cf. Be sure that the submodules external/crazyflie_comm and crazyflie-firmware also points to multi-uav-final branch. This branch breaks most of the things implemented in the master branch that's why I wanted to be fully tested before making it the master branch. We managed to fly 7 crazyflie with this new version of SITL. Check the branch readme.md of crazyflie_gazebo for testing procedure. Message me or leave a message here for implementation details or erratic behavior.
7cfs.gif
Franck
burakhocaoglu
Beginner
Posts: 5
Joined: Wed Jun 06, 2018 1:34 pm

Re: HITL and SITL for crazyflie

Post by burakhocaoglu »

Hi there,

Thanks for the update for multiple Crazyflies. I'd like to know the specs of your computer and how you experiment with 7 Crazyflies. My laptop is not quite remarkable in terms of performance (i7-3632QM 2.2 GHz 4 Cores, RAM 8GB, Ubuntu 16.04 LTS, 5 years old and has overheating issues, without a cooler it can reach 88 °C and if exceeds 90, the laptop shuts down itself for the safety of circuitry). As you pointed out, SITL requires lots of resources. We tried to run 3 SITLs in 1 computer and 2 in another (we tried different combinations as well), but this did not result in as how we wanted (the crazyflies communicating with the external SITLs crashed and the others were shaking a bit). I observed that one of my 8 CPU cores get clogged (~96-98% of usage while others are not even half busy) with 2 SITLs running simultaneously (I guess Intel hyperthreading causes a 4 core CPU to be seen as if it has 8 cores by Ubuntu).

How are your experiments running in this context? Did you use Desktop computers? Is only one enough or did you use multiple? Did you use a smart tool that shares the workload between threads/cores? It may not be a good idea because passing resources from 1 core to the other has risks proportional to the number of resources.

Sorry for long post, thanks in advance...

Burak
wuwushrek
Beginner
Posts: 15
Joined: Mon Apr 16, 2018 1:05 pm

Re: HITL and SITL for crazyflie

Post by wuwushrek »

Hi Burak,
If you had a look at the docu on github you will see my computer specs : Intel® Core™ i7-8650U CPU @ 1.90GHz × 8 and 15.6 Gb of RAM and it is a LAPTOP computer : a DELL Latitude 7490.
No I didn't use any tools to share the workload between threads/cores. However I have not tested simulation accross many computers as I assumed the behavior should be the same as on a single computer. I will test that as soon as possible to debug the problem you got.
With only 2 SITLs instances, I have one my CPU at 100%, another one is 60% and the rest are less than 25%. However the more crazyflie I put, the more CPU it uses. What is your FPS in gazebo and Real time Factor value in that configuration ?
burakhocaoglu
Beginner
Posts: 5
Joined: Wed Jun 06, 2018 1:34 pm

Re: HITL and SITL for crazyflie

Post by burakhocaoglu »

Hi,

With 4 Crazyflies, I can get around 30 - 40 FPS, but only 1 of them actually worked properly; 2 crashed after takeoff, the last one endured a little bit, but then crashed as well. My Real time factor is 1 and the real time update rate is 500 for all of world configurations.

EDIT: the basic world has an update rate of 1000, when I dropped it to 500, the motions became much smoother. The crazyflies were a bit off from the locations they were supposed to be; however, there were not any crashes.
wuwushrek
Beginner
Posts: 15
Joined: Mon Apr 16, 2018 1:05 pm

Re: HITL and SITL for crazyflie

Post by wuwushrek »

Oups it looks like your are not using the correct branch. Which branch are you using ? (I know the names sucks xD) But you should point on multi-uav-final (NOT multi-uav-fix) and be sure that crazyflie-firmware point to that same branch and the same for external/crazyflie_comm.
If you look at that branch you should see that the real time update rate is indeed 500 AND the max_step_size should be 0.002.
tugayalperen
Beginner
Posts: 19
Joined: Mon May 14, 2018 8:33 am

Re: HITL and SITL for crazyflie

Post by tugayalperen »

Hi,

The multi-uav-final branch works fine for me with 5-6 cfs, but not 7. I compared my CPU (Intel® Core™ i7-6700HQ CPU @ 2.60GHz × 8 ) with yours and its an expected result I guess (I have same RAM size).

When it comes to HITL, I wonder did you ever tried to fly multi cfs in HITL mode, or a single one maybe? If you do try and succeeded, what feedback do you use (Vicon or LPS)? We have LPS system, about to be installed. Yet I am not sure if we can integrate LPS into the current directory of yours to try with HITL mode (since there should be another package must be added for LPS system and I doubt that messaging protocols will agree for position feedback). Thank you again.

Edit: By changing real_time_update_rate to 300 and max_step_size to 0.0033, I have good results with 7cfs.

Kind regards,
Post Reply