2-way communication for HoloLens 2 + CrazyFlie 2.1 (with Lighthouse v2)

Discussions and questions about the Crazyflie Nano Quadcopter
Post Reply
oxygenwater
Beginner
Posts: 4
Joined: Sun Apr 24, 2022 7:10 pm

2-way communication for HoloLens 2 + CrazyFlie 2.1 (with Lighthouse v2)

Post by oxygenwater »

Hello all,
I am doing a mixed reality project. The expected outcome, in a nutshell, should be a scenario like this: The user, wearing a HoloLens 2 (HL2), has a CrazyFlie 2.1 drone hovering in front of her. But through HL2, the drone is overplayed with a helicopter model. As she reaches out her hand, the drone (seen as a virtual helicopter in HL2) flies closer to her hand, then hovers around her hand.

Therefore, this project needs two-way communications between a HoloLens 2 and a CrazyFlie 2.1 drone. And I am using Lighthouse Base Station v2 for positioning.

These are the methods I find for implementation, along with questions:

><Method 1>>
This tutorial allows a HL1 to control a CrazyFlie 2.0. But having taken a quick look at the tutorial’s projects file, my questions are:

(a) Is this two-way communication, i.e., can the drone update its current status (position, rotation, batter life, etc.) to the HL1? (Doesn’t look like so, but I would like to make sure.)

(b) Will this work well for HL2?

(c) Will Bluetooth be a reliable communication? (The tutorial mentions that Bluetooth has some latency. Plus, my HL2 cannot discover my CrazyFlie during in Bluetooth system setting. So I don’t even know if it will work in Unity.)

<<Method 2>>
Using a computer as the “middle broker”. Like this:

Drone ---> Computer ---> HL2 (Drone sends position etc. to HL2)
Drone <--- Computer <--- HL2 (HL2 sends commands to Drone)

My questions:

(a) If I go with Method 2, it’s going to be: Drone <---> Computer communication run on CrazyRadio (and positioned by Lighthouse); Computer <---> HL2 run on Wifi. Will this structure likely to run better than Method 1’s Bluetooth? (in terms of coverage, latency, stability, et.)

(b) So it looks like I will need to establish a Python-supported server, or basically a cross-device data reading/writing mechanism, on the middle computer. Could you share your suggestions on establishing such mechanism? REST API? ROS API? Or just read/write a local text file?

Thank you!
oxygenwater
Beginner
Posts: 4
Joined: Sun Apr 24, 2022 7:10 pm

Re: 2-way communication for HoloLens 2 + CrazyFlie 2.1 (with Lighthouse v2)

Post by oxygenwater »

My original post may have been too wordy. Allow me to simplify and update my questions in this post. I plan to use this structure to make two-way communication between HoloLens 2 and a CrazyFlie (positioned by Lighthouse v2). It will be:

DRONE <---CrazyRadio --> COMPUTER <---Wifi---> HL2

Having done some research, it looks like this structure is reasonable. And looks like ROS API is more used. However, ROS only runs on Unix-based computer (e.g., virtual machines), and it needs to install a lot of additional things to carry out such communication. Will things be easier if use REST API? (e.g., Can it run on the computer's Windows environment? Will it be more stable for such cross-device communication? etc.)

Thank you. (I am new to web connections and CrazyFlie, please shed some light on this question.)
kimberly
Bitcraze
Posts: 1027
Joined: Fri Jul 06, 2018 11:13 am

Re: 2-way communication for HoloLens 2 + CrazyFlie 2.1 (with Lighthouse v2)

Post by kimberly »

Hi!

We all haven't worked with the hololens 2 before... although I would love to get my hands on this! I have a Oculus Quest 2 and I managed to control the crazyflie with the Android crazyflie app with a crazyradio with it.

I don't know if the Hololens has a USB connection somewhere, but you could try to connect a crazyradio to it? I don't think that connecting with bluetooth with the Crazyflie from a windows machine is possible, but as I understand, a hololens is practically a windows computer right? I am still able to develop for python on windows quite easily, but you will need to install the crazyradio driver with Zadig.

If you want to make it work via another Windows computer, you should probably try out the crazyflie python lib instead of the ROS api for now.. https://www.bitcraze.io/documentation/r ... on/master/. If you are able to receive the commands to wifi on python, then you can relay those to the Crazyflie. ROS is not yet ideal on Windows and WSL2 doesn't have stable USB support yet (at least the third party package I tried didn't work for the crazyradio).
oxygenwater
Beginner
Posts: 4
Joined: Sun Apr 24, 2022 7:10 pm

Re: 2-way communication for HoloLens 2 + CrazyFlie 2.1 (with Lighthouse v2)

Post by oxygenwater »

Hi Kimberly,

Thank you for sharing your Oculus Quest2 experience. HL2 does have a USB-C port (mainly for charging). I am sure with an USB adapter, the CrazyRaido dongle can be connected to a HL2 headset. However, I am not sure if/how doing Windows-based things (Zadig driver, running CFClient to config Lighthouse geometry, etc) will be smooth in HL2 -- Although HL2 is UWP-based, for now I don't know how "Windows-alike" HL2's UWP can function as a Windows Computer.

And thank you for the heads-up about ROS vs. CrazyFlie's Python lib!
kimberly
Bitcraze
Posts: 1027
Joined: Fri Jul 06, 2018 11:13 am

Re: 2-way communication for HoloLens 2 + CrazyFlie 2.1 (with Lighthouse v2)

Post by kimberly »

yeah unfortunately we wouldn't know about those technicalities about the Hololens2... We did had a blogpost of Sean Kelly controlling a Crazyflie with the Hololens 1 which would be interesting for you to read?

Please let us know on your progress! And ofcourse if you have questions on the crazyflie side of things :) Good luck!
oxygenwater
Beginner
Posts: 4
Joined: Sun Apr 24, 2022 7:10 pm

Re: 2-way communication for HoloLens 2 + CrazyFlie 2.1 (with Lighthouse v2)

Post by oxygenwater »

Hi Kimberly,
Thank you for sharing the blogpost from Sean Kelly. Yes, I'd read it, and in the first post of this thread, I included the webpage (linked from the blogpost) in which he described more technical details.

His project was based on CRTP to let HoloLens 1 control CrazyFlie. Could you tell me if CRTP can be implemented as a 2-way communication? If so, maybe the 2-way communication between a CrazyFlie and a HoloLens doesn't really need to a middle-man computer to realize (assuming external positioning data is not needed).

(It would be like: HoloLens gesture-control pitch/roll/yaw/thrust, etc of a drone, the drone sends back latest pitch/roll/yaw/thrust, battery level, etc.)

Thank you.
Post Reply