Progress of project for week # 1-2

Our team had an online zoom session for our group’s risk assessment. Risk assessment is extremely important for us to ensure that we are aware of the possible hazards around us as we are carrying out our activities! Due to the current COVID situation, we could only meet online for this exercise, but we all sure did learn more on how to conduct our own personal risk assessment using the system provided by NTU! #safety

Progress of project for week # 3

In this week, our team met up to discuss and prepare for our 1st draft presentation! Fun fact, some of us learned how to properly read resistors from their bands today!

While waiting for the print, the group found out that for the camera module to be in focus using the raw camera module, the focal length would be very small. In order to solve this problem, our group considered many possibilities like switching the camera module to a better one with an inbuilt zoom function or macro capabilities, but as the discussion went on, we realized that we could then make use of those clip-on macro lenses that are cheap and readily available! This simple yet easy solution helped to solve the problem of focal length is too small, and also helped to save on quite a bit of money! #savemoneyforarainyday

Progress of project week #7

Building on the macro lens, we found out that the lighting had a very significant influence on the computer vision image captured for the model to process and calculate. The lighting affects the shadows and also due to resistors having relatively reflective surfaces, a diffused and yet strong and equal light source will then have to be made. As a trial test, we made use of the classic office standard white printing paper as a diffusing filter for a single led.

With this, the group then sought out to design a case that can encase the camera module together with the diffused light source. To design the camera module, we made use of the Fusion360 designing software to quickly design a casing like shown below.

This week, our group also set out to conduct rapid prototyping to have a sense of what we want the product to look like. We made use of the spare materials left around in the M&T Lab to create the prototype within a couple of hours! Here is a photo of our prototype with Chester posing together with it. #reducereuserecycle #savetheplanet.

Progress of project week #8

In this week, the group sought to design the hardware components needed for the physical setup of our product. These components would be responsible for holding up our camera module that is connected to the RPi. In the meantime, as a proof of concept, we made use of some of the spare 3D printed parts to build a temporary structure for conducting our tests of our CV model. These unwanted spare 3D printed parts were found in a cardboard box in the M&T lab, which our group has nicknamed, “The Graveyard”. Below are some photos of how we made use of the spare parts as a temporary structure!

While we were testing the software side of things, we also began work on the design of the parts we wanted to get done up by the week. . Some of our design considerations include using a slide-in component over the traditional screws due to the line of action of force being parallel to the screws for the top bracket, and also choosing to create a slimmer camera housing to reduce the bulk and weight of the camera module. 

The above images are snippets of our group’s progress of converting our sketch into Fusion360, and then actually printing it out and trying it out on our product itself. Here’s how it looks like when assembled together!! #0.2mmgap #fitslikeaglove

As for the camera housing, to avoid extra bulk and weight, we thought of making use of a snap-in fit for the camera housing. For this case, we had to make multiple prints in order to find the optimum gap that would allow a snug and tight fit befitting of its purpose. While we did have a lot of failures (AKA broken pieces because we wanted a thin casing), we also learned a lot. For this snap-in and also the sliding component, it allowed us to make use of some of the theoretical concepts that our CBE friends learned in their “Intro to Multidisciplinary Engineering”, which included theory on how fits (interference/clearance and transition fits) are important in the design process for choosing the right components and parts for assembly. The process of designing these few parts definitely was enlightening. Below are some photos of the design of our snap-on casing! #applyingtheoryinreality #trialanderror

LIGHTING WOES

Lighting has proven to be problematic for our group. We explored making use of ambient light, a single white LED, using our own smartphone torchlights, 2 Neopixel strips, 4 LED bulbs from each corner, and now are making use of a Neopixel ring strip which has 12 LED bulbs. Lighting is extremely important for our project as it affects the shadows and also reflection off the resistor surface. With a strong bright and diffused light source, our group can then shift all of our focus onto the software model and adjust the necessary segments of the code for detection of the resistors. Examples of some of the light sources we tried using are shown below.

Progress of project week #9

With the hardware side of our project starting to come alive, for this week, we shifted our focus onto the testing of the computer vision model, in hopes of being able to have some successful resistor detection and make the necessary changes and updates to the code prior to the progress update later that week. With our new setup, we then shifted our usual project meetings into the physics lab where we could test out the effectiveness of a ‘black box’ concept for our detection.

However, during the testing, we found out that the ambient lighting (or lack thereof) had no real significant impact on the resistor detection due to the short distance between the resistor and the camera. At this moment, we were rather stumped and hence looked into how we could improve the setup to achieve successful detection using the computer vision model. During this process, we then revisited the one time we had a successful detection several weeks back when we made use of a cylindrical paper cone as a light diffuser, as shown in the image below.

We noticed that the amount of reflection coming off the resistor was significantly reduced and hence led us to look more into using translucent materials as a temporary diffuser to not only block and prevent direct illumination of the resistor with the Neopixel ring LED strip, but also to provide consistent lighting that we need for testing. For this matter, we tried using plastic cups by cutting them up, and also paper strips to make cones and cylinders for testing. From our testing, we then arrived at the conclusion that using a paper cone (as of now) would still be the best solution for providing the decently bright diffused light source which we so very dearly needed.

With a consistent diffused light source, we then worked on the code by adjusting the HSV color bounds ranges for the color bands that can be found on resistors. Initially, we approached the color detection problem broadly, by looking to adjust the ranges for all of the colors at once. However, we realized that this more often than not led to incorrect yet inconsistent color detection. The inconsistency of the color detection then led us to wonder if the lighting still is not good enough for the purpose of resistor color band detection. This definitely was a frustrating point for our group and led to a kind of ‘programmer’s block’ situation.

But thankfully, after taking a break and chatting with other people from other groups, our group then realized that maybe we should change our approach toward this problem and narrow down our scope. #take1stepbacktotake2stepsforward

From that, we focused our color detection on the primary colors (red, blue, green). Turns out that by approaching the problem this way, we managed to achieve consistent resistor color band detections for the 3 primary colors! It was an extremely exciting moment for all of us when we saw the correct resistor value printed out on the screen. Below shows the screenshot we took when we managed to detect this resistor value rather consistently! #eurekamoment

Additionally, we also printed a box and a lid for containing our wiring and microprocessors in a secure location and to keep the prototype compact. This print took us the longest time so far (about a day to print), which was why our group decided on printing it over the weekend. Junyoung also began to work on the circuit diagram for all of the components which our project uses. #circuitdesign

Progress of project week #10

LCD screen, box, circuit arrangement and hardware 90 percent complete, printed bracket to contain the joystick and butt

This week, the group began working on hooking up the 7-inch LCD display to the RPi, together with the assembly of the parts that we 3D printed over the weekend. With the box, we finally could begin arranging and thinking of how we want our prototype to look like and how we should place the joysticks for fine-tuning of the focus, and also the button to control the lighting. After our discussion and team brainstorming, we eventually came up with the sketch for a bracket that would be used to hold the joysticks and also a mini housing that will contain the button. The rough sketch looked as shown below.

Given that the dimensions of the button housing being rather small, we decided that it would be good to first test out the size of the housing by printing that particular small piece separately, to prevent wastage of not only material but also PRECIOUS TIME. So that’s what we ended up doing. The piece came out to be a little bit too small, which was an extremely valuable finding because it tells us that we should edit the STL file and adjust the sizing accordingly.

smol print button housin fusion360

After adjusting the sizing, we then printed out the following STL file, and it turned out that the joysticks and button.

L bracket fusion360

With our freshly printed bracket the group then proceeded to work with the wiring also made use of the hot glue gun available in the M&T lab to glue this bracket to our setup and it ended up looking like this!

Progress of project week #11-12

In these few weeks, our group focused solely on improving the color detection of our model. With our model able to consistently detect red, green and blue, we then set our sights toward obtaining some of the intermediate colors. To correctly identify a suitable HSV range for specific colors, we firstly made use of many images (which we took using our phones and cameras) of resistors in different lighting conditions, and color-picked several pixels representing different shades of the same color! We then compiled the RGB values from color picking and converted them into the HSV color space for use in our computer vision code! Example of color picking is shown in the image below!

After we did the color picking, we also found an existing repository of a big range of colors from an online color detection repository, which we made use of to compare the commonly available colors that are found on resistors with the color range that we obtained from color picking prior. #crossreferencing 

Using this process we were able to successfully improve the color detection from just R/G/B, to being able to identify black and purple as well! (we also did get some brown color detection, however, it had problems with consistency due to some overlap with red and even black). Below shows some of the screenshots we took of the successful detections of purple, black and brown!

Screenshot of first successful purple band detection post color-picking and cross-referencing!

Successful detection of both purple and black bands (3 resistors all at once on a typical circuit board)!

Successful detection of brown band (2 resistors in the frame)!