Image Recognition Overview

For this subsystem, our team decided to perform the image recognition on a PC instead of on the RPi. The purpose of this subsystem is to:

  1. Receive images taken by the camera onboard the RPi
  2. Detect the target images by plotting a bounding rectangle around it
  3. Recognise the signs detected by assigning its corresponding image number
  4. Send those image numbers to the Algorithm subsystem, which processes them. The Algorithm subsystem then finally sends the image number together with their corresponding coordinates to the Android subsystem which displays them.
  5. Display the raw images with the bounding rectangle at the end

Problem Specification

Raw images captured by the camera onboard the RPi may or may not contain the target images that we are tasked to recognise (localise by plotting bounding rectangle and classify by identifying its image number). A raw image that contains one of out target images is shown below. Our task is to plot a bounding rectangle around it and classify the area specified by the bounding rectangle as class label “a“, image number “11“.

                  

We have 15 target images that we have to recognise should our RPi manage to capture them. Each target images has its own unique image number. The 15 target images are shown below.

Source Code Files

  • main.py: Runs the HTTP server to communicate with other subsystems, calls on relevant functions for the image recognition task, and does the final processing (for coordinates) before sending the predictions to Algorithms subsystem
  • imgReg.py: Does the image processing before image classification. It also does the image localization (plotting bounding rectangles)
  • cnn.py: Defines the CNN object and methods to interact with its Convolutional Neural Network for the image classification task