Harris corner detector

It is important to detect corners in images because corner points are easy to find if an image is deformed - corners are good detectors. Corners can be found by moving a window over the image and storing locations where the contents of the window change a lot when it is shifted. Change in a window can be expressed as follows:

E(u,v) = Σ(x,y) w(x,y)*(I(x+u, y+v) - I(x,y))²

Here w(x,y) is the result of the window function at location (x,y). To common window functions are a square window (1 inside, 0 outside) and a gaussian. I(x,y) means the intensity of the image at location (x,y). E(u,v) represents the total change in intensity when moving the window u pixels horizontally and v pixels vertically from its starting location.

Quadratic approximation

To find in-between-pixel results we can use a Taylor expansion, an approximation of this can be given by:

E(u,v) ≈ [u v] M [u; v]

Where M, the second moment matrix, is defined _a by:

M = Σ(x,y) w(x,y)*[Iₓ², IₓI_y; IₓI_y, I_y²]

Now with some linear algebra magic we can express M with two orthogonal eigenvalues:

M -> [λ₁ 0; 0 λ₂]

As can be seen in the following image, these two eigenvalues form an ellipse together that describes the rate in which E changes when moving the window in a particular direction.

ellipse

Interpreting eigenvalues

For a location in the image to be classified as being a corner, its E needs to change a lot when moving in any direction. This means we want our ellipse to be much like a circle, and as large as possible. In short: λ₁ and λ₂ both need to be large and λ₁ ≈ λ₂.

cornerness