Edge Detection Project Uploaded

Another project that I did last semester was an investigation of six different edge detectors to determine which ones performed better on different images and if there was any way to come up with a generalized scheme of determining which edge detector to apply to a given image depending on certain traits of the image (such as brightness, intensity contrast, color contrast, etc.). This project was done with a partner, Mr. Dave Stears.

The edge detectors considered were:

• The Classical Canny Edge Detector
• A Canny Edge Detector applied to the three color channels rather than simply the intensity.
• The Marr-Hildreth Edge Detector (I lost the code for this one, but you should be able to re-implement it very easily from the description)
• A edge detection routine based on boolean images and boolean functions
• A Distance and Vector Angle Edge Detector designed specifically for color channel processing
• An edge detector based on determining depth discontinuities from multiple images taken with different flashes.

All code is in MATLAB. Included in zip file is a document with our final results, a document with our final discussion, and a short Powerpoint presentation describing our work. Test images are also provided.

Final Report Uploaded!

Ok, the Final Report for my project on Occlusion Detection and Ordinal Depth Determination is now officially up. Sorry it took so long, I had more to talk about than I had originally imagined.

Expect to see my other major project go up soon as well, just as soon as I get motivated enough to organize it into some sort of deliverable form. I may also post up some of my other projects through the semester, but I'd hate to give away answers to future students and make them miss all the pain I had to go through. :-P

Ordinal Depth Project Uploaded

Ok, I've gone ahead and uploaded my code and final presentation for my Ordinal Depth Detection project (ECE 598). They are available under the heading "ECE598 Project: Ordinal Depth Detection" in the links on the right-hand side of the page.

Hopefully I'll have the final report uploaded by the end of the day. If not, It probably won't be up until Thursday or Friday.

I'm also planning to upload some of my other computer vision projects from throughout the semester as well, when I get the chance.

The Joys of Recursion

So, in order to detect occlusions in a scene from a few images, first you've got to break up the image into regions. Then you simply determine which regions are not visible in different images, and this tells you what is in front of what.

The problem with this is that it requires you to break up your image into regions... The algorithm I'm using involves the image flow. So I calculate all the flows in the image and then segment by determining which part of the image belongs to which flow.

The basic algorithm is this:
-Get the image flows
-For each image flow, find the regions that match this image flow
-Repeat until all image flows have been run through the image and the whole image is broken into regions.

The problem with this is in finding regions that match a particular image flow. All the image flows were calculated between two frames, so you take the first frame, apply the image flow to it, and then compare it with the second frame. You then record good intensity matches. With this map of intensity matches you try to grow regions.

Well, how do you grow a region out of a bunch of pixels? Clustering. And the most simple clustering algorithm is simply to connect together adjacent pixels. But of course to search out these adjacent pixels, you have to use a recursive algorithm, much like searching a tree.

Well, it turns out MATLAB dies very quickly when performing recursion. When I try to find a region that's more than, say, 500 pixels (which is very likely) MATLAB will simply crash.

So it looks like I'll have to go back and use a different clustering method. Maybe K-means will work.

Image Flow

So, to detect occlusions, pretty much the first step in any case is to calculate the image flow (aka optical flow, free flow, etc). This is very similar to feature tracking, except with feature tracking the goal is to follow certain things from frame to frame, and with image flow you want to see how EVERYTHING moves from frame to frame.

Why is this important to occlusions? If you calculate the image flow, you will notice certain regions will disappear from the image flow from frame to frame. This gives an idea of what regions are in front of other regions.

The problem? Image Flow is actually pretty difficult to calculate accurately. However, I think I've found some MATLAB code by Temu Gautama that seems to do a decent job. I added a link to it on the side bar.

The next step is to take the image flows calculated between various images and use them to find all the occlusions. We'll see how long it takes me to get stuck this time. :-)

Ideas

As a part of my research into occlusions I read a paper on "Motion Segmentation Using Occlusions" by Abhijit S. Ogale, Cornelia Fermuller, and Yiannis Aloimonos. I've added a link to the paper in the sidebar under Papers and Downloads if you're interested in reading.

This paper the authors strive to create a general method for segmenting a scene into various different objects, but the section which I found most interesting was section V, in which they discussed their algorithm for determining ordinal depths (meaning like object 1 in front of object 2 in front of object 3) from three video frames. Their technique seems to me elegant in it's simplicity, although obviously the math behind it is somewhat complex.

While considering what to do for a project, I have found myself stuck on the question "What useful applications of occlusion detection are there?" The problem here is questioning when occlusion detection, which is a relatively long-winded process, is needed over a more simple solution. For example, if you want to detect whether a door is open or not, a simple laser beam will do the trick much more efficiently than a costly occlusion detection system. And if you merely want to detect when your view of a certain object is being blocked, simply looking for changes in intensity makes a lot more sense than trying to actually detect an occluding object.

So I've been able to brainstorm a small number of examples where occlusion detection would be a better solution than something computationally simple:

1. When there might be multiple occlusions occuring in a scene, and only certain ones are important (Example: a door opening may not be a problem, but someone walking through it might be).

2. When the surface being occluded is too complexly textured for a simple intensity algorithm to work (Example: Suppose you want to detect if someone is in front of your TV screen, but the screen itself may be constantly changing).

3. You care about the structure of the occluding object (Example: you have a machine which puts labels on cans. You want it to stick a label when it has a can in front of it, but not someone's hand)

These are some interesting ideas to think about, and ultimately I will rest on one of them, or a similar one, for my project. However, initially I want to get some sort of occlusion detection algorithm working. I think it will give me a better idea of when exactly occlusion can be a useful tool.

Getting Started...

Hello out there.

My name is Matt Young, and I set up this blog to serve as a journal of my computer vision projects here at the University of Illinois. I hope that any visitors to the site find my work both interesting and useful.

My main project right now is for Professor Yi Ma, and it's just getting underway. My initial idea was to design an algorithm which would be capable of taking several pictures of a scene and performing the following operations:

1. Recover the 3D geometry of the scene
2. Find occlusions in the scene and determine what regions of the scene are not visible
3. Determine an optimal set of movements that will reveal all the hidden areas of the scene

However, in a meeting with Prof. Ma earlier today I was easily convinced that this sort of project is FAR too difficult for the time constraint that I have. Prof. Ma encouraged me to focus simply on the occlusion problem and try to find an interesting subtopic to consider. He gave me a paper to read on the subject to jump start my work.

I'm going to have to give this some serious thought over the next few days...