Medical Augmented Reality - Chair for Computer Aided Medical

Technische Universität München
Fakultät für Informatik
Prof. Nassir Navab
Summer Semester 2015
Exercise Sheet 3
PRACTICAL
Medical Augmented Reality
Exercise 1
Setup marker tracking
In this exercise we will implement part 1 of 2 of the marker tracking algorithm. It is advised
to create a MarkerTracker class for this exercise to avoid huge clutter in the main.cpp.
The find function we want to implement takes a reference to an OpenCV image cv::Mat&
and will later return a std::vector<> of found markers.
Place the call to your find method directly before the display() method of the main loop.
For this exercise we will only use the OpenCV API. Please use the online documentation
for help and samples on cv:: functions.
(a) Print the six markers found in Marker-Cube.pdf on the course web site. You might
want to glue them onto cardbord and maybe extend the border, otherwise your fingers
will occlude the pattern when holding the markers.
(b) At first convert our colored webcam image to back and white using cv::cvtColor()
and cv::threshold(). Experiment with the threshold (a value of 100 should work).
You can always view resulting images using cv::imshow() to check your progress.
(c) Find the contours in the b&w image using cv::findContours() with CV_RETR_TREE
and CV_CHAIN_APPROX_SIMPLE. For this you have to create a copy of the image since it
will be modified by the method. Finally use cv::approxPolyDP() to find a polygons
from the contours. See last few lines of https://github.com/Itseez/opencv/blob/
master/samples/cpp/contours2.cpp for details.
(d) Since our markers are squares, you can skip polygons that do not have 4 corners. It is
also beneficial the skip too small or too large polygons. (use cv::boundingRect())
(e) Remember to check your work with cv::imshow. You can also draw lines cv::line
and circles cv::circle.
points on each edge e (with sampleSubPix() from the
(f) Find six equal distant h = ||e||
7
website). Also find the points left and right (same distance, perpendicular direction)
from the six previous points and sample 6 3px wide stripes with length h. (see slides
for example).
(g) Apply the y-Sobel operator on each stripe (http://en.wikipedia.org/wiki/Sobel_
operator) to enhance the edge.
(h) Fit a parabola y = ax2 + bx + c to the stripe’s maximum and its two neighbor pixels.
Then compute the extremum (gradient is zero) 2ax + b = 0 of the parabola to find
the sub-pixel accurate location of the edge.
–2–
(i) Now you have 6 points on the edge with sub-pixel accuracy. Fit a line through those
six points to get the best approximation of the edge. You can use cv::fitLine() for
this.
(j) The final step now is intersecting the four lines to find accurate corners for the
marker. cv::fitLine() return a point p and a direction vector d for the resulting
line. Intersect two lines by solving the linear system equations p1 + sd1 = p2 + td2 for
s and t. The intersection point c can then be computed as c = p1 + sd1 .
(k) Show your final lines and points on the image using cv::line and cv::circle. It
might be also a good idea to show the stripes images and locations during testing.