I have been working on integrating the gpu accelerated tracking into tbeta–a new tracker that is being developed by Seth Sandler.  I’ve managed to add all the gpu based pre processing to the current OpenCV based filter pipeline and added some new glsl filters along the way.  For now I am just reading back the result from the preprocessing using glReadPixels (which is really slow) and feeding it to the OpenCV contourFilter and tracker.  Still I seem to be getting a speedup from 170fps to 270fps (although I am not sure if those numbers are accurate).

I’ve got some bugs to fix when using video instead of the camera, and also still need to hook up the sliders to the gpu filters.  The parameters are still in the xml format I had them before.  And of course I still need to fix the rest of the pipeline to avoid the costly readback.

I have been playing around with a couple ideas of how to cope with this.  Some of which use alternative algorithm to label the individual components  Here is a screenshot of a really naive approach that actually works better than i thought it would.  It marks the horizontal and vertical middle pixel red and green respectively for each row/column.  The center point becomes yellow.  The idea is that if i have an image with just single pixels representing the center of each blob I can find them quickly on the gpu by processing e.g. each 10×10 block by a separate geometry shader thread.  Then I could easily produce variable length output to store the coordinates and only transfer back to the cpu what is really needed.

I’m pretty convinced that this approach is too naiive to work well, but it kind of looked cool.  We’ll see what else we can come up with.

Anyways, enough complaining. I just submitted some more changes to svn and so the project now compiles on Windows as well.

Now on to some more shader programming.  I started working on shaders for dilating and eroding the image.  I can use those for a highpass filter, but I have also been trying to write a shader that converts the image into the frequency domain (ala Fourier transform) that would make a highpass filter really easy and also allows one to combine certain filters more efficiently.

Next week I also get to meet with a graphics prof at my university.  I was talking to him about the project and he is going to help me work out the best (we can come up with) way of detecting and tracking the blobs.  The usual approaches to connected component analysis dont work well on the GPU, as they rely sequential scans of the image array and information from pixels visited before the current one.  On the GPU the computations are perfomed in parallel instead…but I’m pretty sure we can work some GPU magic with the geometry shaders that are supported by newer graphics cards

I’ll add some documentation in the days to come. I need to also make the user interface for filter chain configuration better and get the blob detection and tracking working.

I have started coding and am working on adding camera support for a couple of different camera types. USB and ieee1394 work great on windows through DSLib and the CMU driver. I have gotten firewire support to work on OSX and Linux as well through libdc1394 but I still have quite some work to do with the cameras. I will see how far I can get this weekend. I am getting eager to write some glsl code for the image processing.

I will upload some code to svn after playing a bit more with the various camera interfaces so that people can actually use it with the hardware they have.