Phase 1

During Phase 1, we were focused on implementing our barcode scanning and inventory management.

Upon consultation with our mentor and course instructors, we decided to go ahead with a Raspberry Pi 4 Model B (RPi), with 8GB RAM, as we needed it to have decent computational abilities to run our future codes. We also purchased a 7″ Touchscreen Display that we would need to display our GUI and for the user to interact with our system.

 

Additionally, we also decided to get a default RPi Camera Module v2 instead of a normal barcode scanner. This was because we were looking to perform text recognition from images in Phase 2, and having the camera module gave us the flexibility to implement that more easily.

Phase 2

In order to bring our project one step further, we wanted to make the scanning of items even more convenient and hands-free. Hence, we decided to embark on a multi-camera setup which would be able to do a complete scan of the item. We hoped that this would allow our system to get all the information off the product in just one scan, regardless of where the barcode and expiry date are located on the object.

This seemed easy at first but that proved not to be the case…

There were some major considerations that we needed to address. Firstly, the size of the scanning platform. The field of view of the webcams had to fully capture the whole of any regular products that were placed in fridges. This was particularly difficult as we had to consider the size of the enclosure, extremely large variance in shapes and sizes of products, as well as the field of view of the webcams. After much consideration, we proceeded with a 30cm x 30cm enclosure with 6 webcams as it could comfortably contain a standard size milk carton. This is a common, and one of the larger sized perishable products to be kept in fridges. Secondly, we had to allow users for easy placement of the object into our scanning platform which will be enclosed/surrounded with cameras.  Therefore, we adopted a hinge and magnetic catch system for one of our extended arms which could be raised to access the platform. Lastly, the RPi was unable to support 6 RPi camera modules, even with extensions. We then had to change our scanning devices to USB webcams instead of the RPi camera modules, and also purchase a USB Hub to accommodate all the 6 webcams. During this phase, we managed to finalise the design of the multi camera set up, as well as try out the webcam mounts.

Upon deciding on using USB webcams, we realised that the quality of images from the webcams was an integral part of making our system work well. While testing on our laptop webcams, which offered images of 1280×720p, we realised that such images lacked clarity for accurate text recognition to take place. Thus, we resorted to purchasing six 2k quality USB webcams for the final product, rather than 1080p quality webcams.

• 

• 

Phase 3

The first issue we aimed to resolve were the webcam mounts as there were a few issues with image quality due to the design of the mounts. Firstly, they were too thick which caused the webcams to be too near to the product, thus obtaining blurry and out-of-focus images. Secondly, the prototype webcam mounts were designed based on the 1080p webcams which we originally aimed to purchase. However, we made the switch to 2k quality webcams instead so the design had to be modified to fit the new webcams.

After we completed the second renditions, the new webcams arrived and they seemed to fit perfectly to the dimensions of the mounts. However, when trying to link up all six webcams to the USB hub, it was apparent that we did not consider the position of the USB cable of each webcam and how they would interfere with the mounts. Finally, we redesigned the webcam mounts with an accurate measurement to include a notch for the cable to run through the mount. We also centralised the mount to be in the middle of the webcam instead of being off-centred. The webcam mounts were all locked in place with cable ties. Each webcam was manually adjusted to obtain the best focal length and range of photographs to capture maximum information. We tried to do this with a few kinds of products with varying shapes and sizes.

Upon fixing the multi camera set up and trying out the image quality, we found out that the bottom camera was too close to the product and will obtain a blurry image. Furthermore, the lighting was insufficient to obtain meaningful photographs. We then abandoned the bottom webcam. To make use of the spare aluminium bars, we extended the top view webcam to obtain a larger range for the top view photo. Since we also had a spare webcam, we decided to resolve an issue present with most milk carton bottles – the cap of the bottle frequently covers the expiry date when looking at it from the front, which was where our webcam was. Thus, we decided to implement an angled webcam at the top of the frame to capture the expiry date of milk carton from an angle so it would not be blocked. Our group felt that this will be a more efficient use of the webcam as compared to having a poor quality photo taking from the bottom of the product.

The final segment of our hardware component is to combine the RPi and the multi camera set up. For the RPi, we managed to design a case to enclose the screen. Making use of the 4 screws on the back of the RPi for mounting, we designed a set of bracket connectors with holes for screwing and as well as slots for the brackets to slide into the top bar of our aluminium profile. Thankfully, this proved to be a very secure connection with no issues. 

After we finalised the designs and placements for all the webcam mounts and RPi mounts, we began testing the 6 webcams in our multi-camera setup to perform the barcode scanning and text recognition for some perishable goods such as milk and yoghurt. Apart from which, we were mainly left with minor adjustments such as cable management and placement of the USB hub which is a relatively bulky item. Currently, we are considering to tape the USB hub below the platform since the bottom webcam is already removed.