Hello guys,
I would like to implement a streaming video using the existing framework. I don't really need too high framerate and I was wondering if, using the existing log/parameter framework, this could be possible.
I know that log should be used when high data rate is required, but I suppose that somehow I would need to send some information from the client to the Crazyflie firmware in order to tell him that the client have successfully read the data. For the latter, I would require a parameter as suggested here (https://wiki.bitcraze.io/doc:crazyflie:api:python:index).
Basically I need a method to synchronize the read and write of the data (subframes).
Do you have any suggestions on how to implement this?
Requirements:
Resolution: 80x60
Min frame rate: 1fps
Preferred language: C/C++
Let me know your opinions.
Best regards,
Jassy
Streaming video
Streaming video
Last edited by jassi0709 on Mon Jan 30, 2017 8:58 am, edited 2 times in total.
Re: Streaming video
Hi Jassy,
The existing log and param framework have not been designed to carry big chunk of synchronous data like an image but for asynchronous individual scalar. So you cannot use them. The amount of data you want to transfer should be possible to carry via the radio so that will not be a problem.
I have had some success recently just sending the data on the console using a loop with consolePrintf("0x02x", data) and a new line at the end. This has proven to be great for quick development since I could then copy-paste the resulting line from the console or a python software can receive the information using the API.
A more proper way would be to either use the memory subsystem or use your own port.
The memory subsystem is used to access configuration memories on the Crazyflie but we also had a though of using it to access any data buffer. I am not sure how to handle the synchronization though, maybe an A/B buffers would work (the api allows to read and write so the idea was that software could implement registers).
Another more straightforward way (and I think this might be better for images actually) would be to just stream the data using a new CRTP port. You would just have to call the CRTPSendPacketBlock function with the data of the image and using a new port (there is a list of existing port there: https://github.com/bitcraze/crazyflie-f ... crtp.h#L39). The drawback is that you will have to implement this port in the crazyflie-lib-python on the other side, but this will not be that hard.
I have been thinking of implementing generic deck CRTP ports that would be easily acquired by a deck driver and would allow for a deck to have a channel of communication with some code on the ground, I guess it would have been useful in this case?
Best regards,
/Arnaud
The existing log and param framework have not been designed to carry big chunk of synchronous data like an image but for asynchronous individual scalar. So you cannot use them. The amount of data you want to transfer should be possible to carry via the radio so that will not be a problem.
I have had some success recently just sending the data on the console using a loop with consolePrintf("0x02x", data) and a new line at the end. This has proven to be great for quick development since I could then copy-paste the resulting line from the console or a python software can receive the information using the API.
A more proper way would be to either use the memory subsystem or use your own port.
The memory subsystem is used to access configuration memories on the Crazyflie but we also had a though of using it to access any data buffer. I am not sure how to handle the synchronization though, maybe an A/B buffers would work (the api allows to read and write so the idea was that software could implement registers).
Another more straightforward way (and I think this might be better for images actually) would be to just stream the data using a new CRTP port. You would just have to call the CRTPSendPacketBlock function with the data of the image and using a new port (there is a list of existing port there: https://github.com/bitcraze/crazyflie-f ... crtp.h#L39). The drawback is that you will have to implement this port in the crazyflie-lib-python on the other side, but this will not be that hard.
I have been thinking of implementing generic deck CRTP ports that would be easily acquired by a deck driver and would allow for a deck to have a channel of communication with some code on the ground, I guess it would have been useful in this case?
Best regards,
/Arnaud
Re: Streaming video
Hi Arnaud,
Thank you very much for you inputs. They were really useful. I was already transferring the image with the consolePrintf but it was not fast enough.
Therefore, as you suggested, I've implemented the streaming using a new CRTP port and it actually worked quite well because I'm already able to show my picture on the PC using libcflie (https://github.com/fairlight1337/libcflie) and openCV.
I will share everything as soon as I will be ready to do that (anyway by April 2017 which is my thesis deadline).
I still have some issue related to the SPI speed setting and a general slow frame rate problem.
Here some pictures.
Thank you very much for you inputs. They were really useful. I was already transferring the image with the consolePrintf but it was not fast enough.
Therefore, as you suggested, I've implemented the streaming using a new CRTP port and it actually worked quite well because I'm already able to show my picture on the PC using libcflie (https://github.com/fairlight1337/libcflie) and openCV.
I will share everything as soon as I will be ready to do that (anyway by April 2017 which is my thesis deadline).
I still have some issue related to the SPI speed setting and a general slow frame rate problem.
Here some pictures.
- ArduCAM connected to Crazyflie
- Streaming screenshot
Re: [Solved] Streaming video
Hi Jassy,
Thanks for the pictures! The camera setup looks ... interesting
. Does it fly?
/Arnaud
Thanks for the pictures! The camera setup looks ... interesting
. Does it fly?/Arnaud
Re: [Solved] Streaming video
Hi Arnaud.
Hahahah no the mechanical part is not ready yet. I have to find a proper way to fix it and understand how to minimize the weight.
Will do that in few days.
Jassy
Hahahah no the mechanical part is not ready yet. I have to find a proper way to fix it and understand how to minimize the weight.
Will do that in few days.
Jassy
Re: [Solved] Streaming video
By the way did you read the master thesis we had a while ago?
Streaming video
Yes I did but I was searching some camera which was easy to integrate with the Crazyflie.tobias wrote:By the way did you read the master thesis we had a while ago?
I have actually another question related to the streaming video. So far I've done the streaming using this library libcflie (https://github.com/fairlight1337/libcflie). The drawback is that I can either use that library for streaming or use the official crazyflie client. Therefore I cannot stream and fly at the same time.
Do you have any suggestion on how I can do those things simultaneously? Should I write something in python (which I'm not good at all) or is there a way to use different channels with the Crazyradio and CRTP?
Please let me know.
Best regards,
Jassy
Re: Streaming video
Hi Jassy,
There is no way to open the Crazyradio from 2 different programs so that is going to be the blocker.
Bletooth LE works at the same time as CRTP/Crazyradio so if all you want is to fly manually this will work: if you do not send any commander packets from your client, a BLE phone can fly the Crazyflie.
If you want to minimize the python code you write, you could use ZMQ to send the image data to your C client.
Another solution might be to use SDL in your C client to get the gamepad input and send the Commander packet.
Best regards,
/Arnaud
There is no way to open the Crazyradio from 2 different programs so that is going to be the blocker.
Bletooth LE works at the same time as CRTP/Crazyradio so if all you want is to fly manually this will work: if you do not send any commander packets from your client, a BLE phone can fly the Crazyflie.
If you want to minimize the python code you write, you could use ZMQ to send the image data to your C client.
Another solution might be to use SDL in your C client to get the gamepad input and send the Commander packet.
Best regards,
/Arnaud