main.py

The main.py runs a HTTP server to interact with other subsystems. Upon receiving captured images sent from the RPi, it passes those images to the run function of imgReg.py.

Variables Defined
  • img_count: Used to assign a unique id to name each raw image and its respective processed image
  • cnn: CNN object used for the image classification task
  • graph: Tensorflow graph and it is required to tackle thread issues that may arise
  • app: Required for our imported Flask module to run the HTTP server
  • predictions: List of predictions recognised in this particular maze run
  • images: List of filenames of the processed images in this particular maze run. A processed image is a copy of its original raw image, and a bounding rectangle is plotted around a target image
  • areas: Python dictionary, the key is the image number, and the item is a list which contains image count number (img_count), area of bounding rectangle (area), position of bounding rectangle (pos)
  • uniquePreds: This contains the set of unique image number predictions of this particular maze run, and it is initialised to contain -1 (-1 is to represent no prediction)
Functions Defined
receiveImg():

Function to handle HTTP POST request received at the specified endpoint. The key steps of this function is described as follows.

  1. Reads in the data sent from RPi and deserialize it to obtain the image data
  2. Saves a copy of the raw image received
  3. Image passed to the run function from imgReg.py. The run function returns 4 variables, pred is the image number, file is the filename of the processed image, area is the area size of the bounding rectangle, and pos is the position of the bounding rectangle. If the raw image does not contain our target images, -1 is returned for the image number this function ends after incrementing the img_count.
  4. If a valid and new image number have been recognised,
    1. Store the filename of the processed image into the variable images
    2. Add the image number into the variable uniquePreds
    3. Store the corresponding area of the bounding rectangle (area), the position of the bounding rectangle (pos) and its img_count number
  5. If a valid image number is recognised but this image number had already been recognised previously in this particular maze run (this can occur as the robot passes through an obstacle with our target image),
    1. It compares the area of this bounding rectangle detected with the previously detected one
    2. Updates the area, pos and image_count if the area of this bounding rectangle is larger
finished():

Function to handle HTTP GET request received at the specified endpoint. This occurs when the robot finishes the maze exploration, and the Algorithms subsystem sends a HTTP GET request to the HTTP server running the image recognition. Its purpose is the send the image numbers detected and recognised during the exploration in JSON format. As a target image may be recognised more than once, to handle this issue we ensure that there is only 1 instance of any image number (except -1) in the list of predictions sent to Algorithms subsystem. The steps of this function is described as follows.

  1. Initialise a list of length the same as our prediction list, filled with -1
  2. Update the list such that the image numbers are unique and its index corresponds to when the image was taken (also corresponds to img_count) with the largest bounding rectangle size
  3. Call the function to plot the raw images captured together with the plotted bounding rectangles
  4. The final prediction list is converted to JSON format and appended with the position list before sending to the Algorithms subsystem
plotImages(images):

Function to plot the processed images in this particular maze run. The function iterates through the images list, then reads in the processed images and displays them.

forDebug():

Function to simulate or re-run a maze run, enabling us to develop this Image Recognition subsystem without the need of the other subsystems. It first reads in image filenames that were located in a certain directory (collected from previous runs etc) and iterates through each of them using Step 3 to 5 in receiveImg function as specified above.

debugEnd(images):

Function to print out the JSON string that would be sent out to the Algorithms subsystem should it be a real maze run and to display the processed images. The steps of this function is similar to finished and plotImages as specified above, except that this function is not triggered by a HTTP GET request.