Principles

Template-matching uses pre-defined images (hence ‘templates’) to execute a search in a separate image. It involves convolving the template image over the target image, and in effect can be understood as a form of cross-correlation: both images are signals in space, and if they are strongly correlated at a few specific points, we can interpret this as a ‘match’.

In our use case, we are looking at images to count yeast cells. Since we expect yeast cells to generally have little variance in shape, template matching seemed like a good start. It would also be possible to account for variations in size by scaling the template images.

Results

Since we performed template matching much earlier in the project cycle, tests performed with more complete input images are unavailable. As such, the results presented below are for tests images taken with a phone camera using a different biological microscope as a proof of concept. Hence, the images acquired were high quality with good contrast and sharp focus. Hence, not much preprocessing was required, aside from a median blur to remove camera noise, along with a low pass filter to remove the background.

Bounding boxes are drawn for a cell count of 256. We can estimate the accuracy to be above 90%.

As we can see, the model seems to draw bounding boxes correctly but seems to miss out on smaller cells that did not match as well with the template. A threshold of 0.5 was used.

It also runs into other limitations, which is that of thresholding, and that of clustering. As discussed earlier, template matching is in essence a cross-correlation operation; if the signals are well-correlated then the two signals are likely to match. However, there has to be a threshold in order to accept or to reject the match. We found that we had to keep adjusting the threshold in order to accommodate poorer matches that were still cells.

One alternative would be to have a large bank of template cells under different conditions. However, it would be challenging to merge the outputs from different templates, as there is no a priori way to decide which template is correct.

The other issue had to do with cell clustering. Since we are working in 2 dimensions, the cross-correlation would give a signal in 2 dimensions. Furthermore, the objects of interest (cells) are not point objects, and occupy area. Hence, our thresholding operation would give a region that surpasses the threshold, but does not tell us how many cells are together. 

We could perform a clustering operation, by defining a minimum and maximum area. However, this would easily be defeated by an unknown use-case involving unusually large or small cells.

Due to these limitations, we did not end up using this for our live model. At the same time, we were also experimenting with a machine learning model. Nevertheless, this was a surprisingly powerful and easy method to apply, and future work would look at applying multiple methods at the same time to cross-validate the cell counts.