MATLAB Programming/Advanced Topics/Toolboxes and Extensions/Image Processing Toolbox

Introduction to Image Processing Toolbox

The core MATLAB package comes with several rudimentary functions (to be described later) that can be used to load, save, and perform custom functions on images. However, it is often necessary to perform more complicated operations on images. The image processing toolbox allows such manipulations as:

Direct visualization of images in MATLAB
Color space conversions (e.g. between RGB, HSV, L*a*b*, and so on)
Object grouping and data collection
Filtering and fast convolution
Fourier analysis of images
Image arithmetic
Morphological operations

and many others.

To do:
This is a rather incomplete introduction, expand it to include the other capabilities of the toolbox

MATLAB built-in functions

To load an image into MATLAB, you can use the "import data" GUI (the same way as you would for a text file or similar), or you can use the "imread" function. To use "imread", you must have the directory where the image is located in your list of directories. Then use the syntax:

>> myimage = imread('myimage.extension');

Use of a semicolon is recommended especially here since images contain large amounts of data.

If the image is a color image, MATLAB will (for most data formats that are compatible with it) convert the image data to the RGB color space by default. The separate channels are represented by the third dimension of the image. The following code separates the channels of the image and indicates the color of each channel.

>> redchannel = myimage(:,:,1);
>> greenchannel = myimage(:,:,2);
>> bluechannel = myimage(:,:,3);

Note:
Care must be taken when doing calculations with images because most image formats are imported as class UINT8, not as class double, to save space. There are many operations which cannot be performed on UINT8 arrays (or, for that matter, on arrays with more than two dimensions). Some of these can be circumvented by using MATLAB's image arithmetic functions but sometimes it will be necessary to convert the image or a portion of it to class double to manipulate it.

Another default function in MATLAB can be used to save images to a new file. The function, "imwrite", accepts two different syntaxes:

>> imwrite(myimage, 'nameoffile.extension')

or

>> imwrite(myimage, 'nameoffile', 'extension')

When using the second syntax, do not use a period before the extension. See the MATLAB documentation for supported file types for import and export.

MATLAB's image arithmetic functions

MATLAB by default imports most images as class uint8, which is not supported by the native addition, subtraction, multiplication, or division functions. In order to overcome this, one option is to convert the images to class double. However, this will cause memory issues if a large number of images are to be used, or if the images are large. It is also very slow. Therefore, MATLAB provides a better option in its arithmetic functions. MATLAB can perform the following operations using its special arithmetic functions for integers (they also work if either A or B is a constant value):

imadd(A, B): add two images A and B (equivalent to A + B, due to an operator overload, but this syntax will not work without the image analysis toolbox so it is not recommended, since it'll give a more difficult error to track down in its absence than "function not found")
imsubtract(A, B): Subtract two images A and B (equivalent to A - B, but again not recommended since it won't work without the toolbox)
immultiply(A, B): Elementwise multiplication of two images A and B. Equivalent to A.*B.
imdivide(A, B): Elementwise division of two images A and B. Equivalent to A./B
imlincomb(K1, A1, K2, A2, ..., C): Linear combinations of images A1, A2, etc. This means you end up with $K1*A1+K2*A2+...+C$ . This is more efficient than using e.g. $immultiply(K1,A1)+immultiply(K2,A2)$ because the latter rounds after every operation, leading to sometimes significant errors, while the former only rounds at the end.

Visualization of images

To visualize an image or a section of an image, use the imshow function:

>> imshow(myimage);

This will show a figure (often scaled down) of the image. You should not attempt to copy this and paste into another program, the RGB values resulting will be different. Use imwrite instead.

Color space changes and cforms

To do:
Describe the 'makecform' function, the form of the cform struct, and how to perform the color conversions provided in this function

A note on color space conversions

Note that when converting from device-specific color spaces like RGB to non-device-specific spaces like LAB or XYZ, some inaccuracy will inevitably arise. One reason for this is that RGB values as given by MATLAB are often encoded with a gamma from the camera (and given another gamma by the monitor before displayed on the screen), which makes the relationship between the RGB and LAB values dependent on the internals of the camera and also nonlinear (and difficult to determine accurately). As an approximation, one can often use a power-law function to arrive at the un-modified RGB values, which in theory should be linearly correlated with the LAB values of each pixel:

$R'=(R/255)^{1/\gamma }$

$G'=(G/255)^{1/\gamma }$

$B'=(B/255)^{1/\gamma }$

Note that these equations assume R, G, and B are on the range from 0 to 255, and the computation requires that R, G, and B be converted to class double from the default class uint8.

Image filtering

MATLAB uses one of two methods, correlation or convolution, to filter various images. The two operations are identical except convolution rotates the filter matrix 180o before filtering, while correlation keeps it the same.

Convolution

2D Convolution Animation

To do:
Show the principle of the calculation

Direct convolution is implemented using both the conv2 method and the imfilter function. For images, since the filter is much smaller than the image, the direct method implemented in imfilter (which uses a MEX function to increase speed) is usually faster.

FFT method

It can be shown that convolution in the space domain (as described earlier) is equivalent to multiplication in the Fourier domain:

$A{\mbox{ o }}B=F^{-}1(F(A)*F(B))$

where F is the Fourier transform operator. If A and B are nearly the same size, performing the Fourier transform of both A and B, multiplying, and inverting the result is often faster than performing the computation directly.

In order to use this method, it is necessary to pad the smaller of A and B with zeros so that both A and B are the same size. The method is actually fastest if both A and B are padded so that they have lengths of a power of 2, since the FFT algorithm is optimized for these sizes. After forming the padded matrices A' and B', compute the FFT of both matrices (using the fft2 function), multiply them, and invert them to find the full convolution (padded with many zeros). Following is some sample code implementing this method, and returning the convolution equivalent to the 'same' parameter in MATLAB:

To do:
Put in the code to do this

Using the image analysis toolbox

Filters can be used to blur images in different ways, or to sharpen them. The function fspecial contains a number of pre-defined filters that you can use for these purposes. The syntax for this function is:

>> filter = fspecial(typeoffilter, parameters);

If no parameters are provided, the default size of the filter is different depending on the type of filter; see the MATLAB documentation page for details. Some other types of filter also have other options, such as the standard deviation in a Gaussian filter.

To do:
Make some example images to show the effects of each of these filters

You can also use your own filters by simply defining them as a matrix.

To do:
Talk about filter design a little

To filter an image, MATLAB replaces each pixel of the image with a weighted average of the surrounding pixels. The weights are determined by the values of the filter, and the number of surrounding pixels is determined by the size of the filter used. MATLAB includes a function to do the filtering: imfilter

>> newimage = imfilter(myimage, filter, parameters);

There are several possible parameters that determine how MATLAB deals with pixels on the edges of an image (since the weighted average at the edges needs something to weigh beyond the image borders). These include padding it with a constant (give imfilter a number in the parameters spot), reflecting the image across the border (parameters = 'symmetric'),assuming the image is periodic (parameters = 'circular'), or duplicating border pixel values (parameters = 'replicate'). The best method to use depends on the application, so it is best to experiment with them.

Another parameter you can pass is the choice of correlation vs. convolution. Correlation is used by default; pass 'conv' to imfilter to use convolution. Be aware that correlation will rotate your filter, be prepared for this if it is intended to have effects in a certain direction on the image.