Cris’ Image Analysis Blog

There are a few things in DIPimage that I’m annoyed with but can’t solve. Some of these are caused by the limitations of the MATLAB interpreter, and some are caused by my own poor design choices. DIPimage evolved over time to do things that I hadn’t at first even thought about, such as processing color images. Tensor and color images were “tagged on”, if you will, on top of the dip_image object, introducing some oddities and inconsistencies. To fix these up we would need to change existing behavior, which we don’t want to do because it would break too much existing code. This blog entry is about the few weird things with the dip_image object and how to work around them.

The dip_image object was born to contain only a single grey-value object. Pretty soon it was extended to contain multiple, unrelated images (mirroring DIPlib’s dip_ImageArray structure). It seemed logical to me to distinguish a single image and an array of images, even though to MATLAB they are the same object, so I overloaded the isa and class methods to make this distinction artificially. When there is more than one image in the dip_image object, it reports itself as an object of class dip_image_array. This caused a few problems when we introduced tensor (and color) images. As they say, hindsight is always 20-20. Now I wish I had implemented image arrays as a separate class, or even simply as a cell array with dip_image objects.

Indexing into a dip_image or a dip_image_array

Indexing into a dip_image_array is done using braces: A{1}; indexing into a dip_image is done using brackets: A(20). Because of a limitation of the MATLAB parser, it is only possible to combine these two indices by putting the braces before the brackets: A{1}(20). This is not so bad if you consider the images in the array to be unrelated: first you select image number 1, then you select pixel 20 in that image. But when we introduced the concept of tensor images, this limitation became frustrating. In a tensor image each pixel is a matrix. This is represented as a dip_image_array where each image has the same size, and contains the data for one of the tensor elements. For example, an RGB image is a tensor image with a 3-element vector at each pixel. The first image in the dip_image_array is the red channel, the second is green, the third is blue. In such an image, A(20) is a tensor (in this example an RGB triplet). It would sometimes make more sense to index a tensor element by A(20){1} rather than A{1}(20). But it generates an error when you type it in, and there’s nothing I can do about that.

Methods overloaded for dip_image and dip_image_array objects

Even worse is the way that the functions size, length, ndims, numel and end work. When the object is of type dip_image, they work on the image, and when the object is of type dip_image_array they work on the array, not the images inside. This made a little bit of sense initially, since it was assumed that the images inside the array were unrelated. But with a color image A:

size(A)
B = colorspace(A,'grey');
size(B)

ans =
     [3,1]

ans =
     [256,256]

In the same way, size(A{1}) would return [256,256] and size(A{1:2}) would return [2,1]. Needless to say, this has lead to some hard-to-find bugs.

It is possible to get around this issue by using only the functions imsize and imarsize for the image size and the image array size, respectively. length(A) should be written as max(imsize(A)), and ndims(A) as length(imsize(A)). numel always returns 1. This was implemented to circumvent a bug in MATLAB 6 that broke the indexing (I don’t know if this was fixed in later versions). numel(A) should be written as prod(imsize(A)).

Finally, the end operator is the worst of the bunch, because I keep using it without thinking about it, and it always takes me a while to find the cause of the bug. Also, the workaround is rather ugly. Like size, end uses the image size if there is only one image in the object, but uses the array size if there is more than one image. Thus, A(0:2:end,:) works perfectly well when A is a grey-value image, but doesn’t do what you’d expect if A is a tensor image. The workaround again is using imsize: A(0:2:imsize(A,1)-1,:). Ugly, no? The reason for this issue is that MATLAB does not report to the overloaded end method whether it is invoked in brackets or braces. The end method existed before cell arrays and braces indexing were introduced in MATLAB. It seems that MATLAB itself also has problems caused by evolution of the software and the desire to keep backwards compatibility! Because the end method does not know the context in which it is called, it has to guess what it is that the user wants, just like size and length have to do. The decision to have all these functions make the same assumptions is sound, but if we would have thought about tensor and color images before implementing these, they might have always looked only at the image dimensions and never at the array dimensions.

Concatenating dip_image and dip_image_array objects

The last issue I have is with concatenation of images. Again, this was implemented before we even thought of the possibility of using color images, and image arrays usually contained unrelated images. When concatenating an image array, it is treated like one wishes to concatenate the images in the array. Therefore, [A,B], with A an image array, is equivalent to [A{1},A{2},...,A{end},B]. This is very frustrating when you have two color images that you wish to concatenate together. The workaround is, as usual with color images, to use iterate.

iterate repeats one operation on every image in an image array. This is often the only simple way of applying a filter to an image. Bit by bit we are improving functions so that they work on tensor images. For example, gaussf(A) now works as expected on a color image. But many functions do not work yet, and many will never work (for example, mathematical morphology on tensor images can not be defined properly, because there is no universally correct way to decide which of a set of tensors is the largest). For these functions it is possible to use iterate to apply them to each of the components of a color image: iterate('dilation',A,25,'rectangular'). Note that the result contains colors not in the input image: this operation is not a dilation!

iterate is clever enough that, when it is called with multiple image arrays as input, it applies the function with corresponding images from each of the arrays. Thus, to concatenate two color images, one can do iterate('cat',1,A,B).

A better design

If we had thought about color images from the start, we would have created a dip_image object that can hold a grey-value, tensor or color image, but never unrelated images. An image array would be either a cell array with images or a completely separate object. In that way it would be possible to have an array with one image, which now is impossible. The size and related functions would know whether the input is an image (whatever the type) or an image array. Concatenation would be able to do different things on an image array and on a tensor image. However, the end operator would still be incomplete, only usable within brackets but not braces, and the limitation to the order of brackets and braces would still stand. We would also have to introduce methods to return the size, length and number of dimensions of the tensor.

But it is too late for this type of change, I feel. I’m hoping this post will help people understand the design choices made, and help them get around the limitations and oddities of the toolbox as it stands now.