Bitcraze Forums

After some more experimenting and understanding the differences between older and newer versions of the various libraries, here is some example code that takes a picture from a webcam (you could presumably have more than one as video0, video1 etc), and then scans this for "blobs", and prints out the coordinates of the blobs.

So this could be used a a simple tracker for CrazyFlie if it has a suitably high-contrast "blob" stuck to it. Since this runs in Python 3, you could now add mc.commander code etc to control the CrazyFlie from the same script and run it from IDLE 3, etc.

I am just putting this code snippet here for the record to show the libraries and syntax needed for some of the commands. For instance the pillow (PIL) library is needed, even though, as far as I can see, no PIL commands are used... I don't understand this either! I stumbled upon this when trying to use PIL to display an image captured by cv2, just to prove that cv2 was even working.
So, you will need to do:
pip3 install pillow
pip3 install opencv-python
pip3 install pygame
first.


# Standard imports
import cv2
from PIL import Image
import numpy as np
import pygame
import pygame.camera
import time

pygame.camera.init()
pygame.camera.list_cameras()
cam = pygame.camera.Camera("/dev/video0", (640, 480))
cam.start()
time.sleep(0.1) # You might need something higher in the beginning
img = cam.get_image()
pygame.image.save(img, "pygamex.jpg")
cam.stop()

# imports the image you just captured
im = cv2.imread('pygamex.jpg',cv2.IMREAD_GRAYSCALE)
print (im)

# writes a copy of the image if you want to keep track of things
cv2.imwrite('messigray.png',im)

# Setup SimpleBlobDetector parameters.
params = cv2.SimpleBlobDetector_Params()

# Change thresholds
params.minThreshold = 10;
params.maxThreshold = 200;

# Filter by Area.
params.filterByArea = True
params.minArea = 1500

# Filter by Circularity
params.filterByCircularity = True
params.minCircularity = 0.1

# Filter by Convexity
params.filterByConvexity = True
params.minConvexity = 0.87

# Filter by Inertia
params.filterByInertia = True
params.minInertiaRatio = 0.01

# Create a detector with the parameters
ver = (cv2.__version__).split('.')
if int(ver[0]) < 3 :
detector = cv2.SimpleBlobDetector(params)
else :
detector = cv2.SimpleBlobDetector_create(params)


# Detect blobs.
keypoints = detector.detect(im)

# Draw detected blobs as red circles.
# cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS ensures the size of the circle corresponds to the size of blob
im_with_keypoints = cv2.drawKeypoints(im, keypoints, np.array([]), (0,0,255), cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)


# Show keypoints
cv2.imshow("Keypoints", im_with_keypoints)
cv2.imwrite("with keypoints.jpg", im_with_keypoints)

# displays coordinates of keypoints
kl=len(keypoints)
print (kl)

x=0

while x<kl:
kl1=0
kl1=(keypoints[x].pt)
klx=kl1[0]
kly=kl1[1]
print (keypoints[x].pt)
print (klx)
print (kly)

x+=1

print ("Finished")
cv2.waitKey(0)

Wow, I completely overlooked the detail of this one, sorry.
I was experimenting with high contrast dark "blobs" with OpenCV, but a bright illuminated beacon on the CrazyFlie might be easier (many black objects and tones look surprisingly grey or reflective when viewed on a webcam... might need to find a way to buy some vantablack at this rate! *).
The only problem is that, using the flow deck, the LED ring cannot be flown.
Would this work just as well with LEDs mounted elsewhere? Perhaps one on each arm... probably yes, since the ledring-detector code as written detects LEDs in the pattern of a square?
Spacing between the LEDs presumably doesn't matter- since the flow deck can now provide Z anyway.


* https://www.surreynanosystems.com/purchasing

arnaud wrote: ↑Wed May 03, 2017 7:51 am Hi,

Just an update, I have worked a bit further and have had very good result when using the LED ring as marker and putting a webcam under the Crazyflie. I made a detector for it: https://github.com/ataffanel/crazyflie-ledring-detector. I haven't had time to document it yet but I still plan to make a short video showing the performances .

If you want to use the ledring detector, protip: as soon as the Crazylie enter the frame the kalman filter will freak out and output NaN, you need to reset it using the reset parameter. This is true for any detector apparently (I think it happens when the detector suddenly send a position that is far from x,y,z = 0,0,0), so this is good to keep in mind.

I am preparing an ext-pos tab that hopefully contains useful info and control for the task, do you see anything I should add?
extpos_prototype.png

You can put the LED in the pattern you want. The led detector contains the physical position of the points detected on the object: https://github.com/ataffanel/crazyflie- ... py#L62-L65

The solvePnP function from opencv will try to fit this physical solution with the position of the point in the image, so you are not even forced to put the point in a plan. I think there was some limitation in number of point and placement of points depending of the algorithm used, you can look at the OpenCV doc for more info: https://docs.opencv.org/2.4/modules/cal ... .html#bool solvePnP(InputArray objectPoints, InputArray imagePoints, InputArray cameraMatrix, InputArray distCoeffs, OutputArray rvec, OutputArray tvec, bool useExtrinsicGuess, int flags)

Many thanks for this. I checked the GitHub code quickly, but could I just ask: Why is the variable "W" set the way it is?
I am assuming this is the X,Y coordinates of your LED square to detect.
So why w=0.026/2.0 ?
Were you tracking a particular altitude?
Apologies if I have got the wrong end of the stick.

The coordinate are the X,Y,Z coordinate of each LEDs, this is setup to have the 0,0,0 point in the middle of the LED ring. W is set so that the coordinate matches my lit-up LEDs, I do not have a LED ring to measure right now but looking at the code I am setting the LEDs 2.6cm appart. This is setup so that I am tracking a 45 degree rotated square, which is alright since I do not care about the orientation.