Progress of project for week 1

(10/06/24 – 14/06/24)

We’re finally all back in Singapore! On paper, this was actually supposed to be week 3 or 4? However, we all had different overseas trips and commitments that led to us only being able to finally meet and get started this week. Most of our time was spent sketching some initial designs and ideas and deciding on how we wanted to achieve the desired 3-DOF motion while remaining unique from existing designs online.

We also did research on and decided on a few different cameras and microcontrollers to test out to see what works best for us! Currently, we’re going to test 1) A Raspberry to control the camera and signal an Arduino to control the motors and 2) A single Arduino to control both the camera and motors. Different cameras are needed for each option so we’ll see what works best in the coming week!

Lastly, we also finally decided on our group name! It all happened when our group leader, Mitchel, was eating his Yong Tau Foo and couldn’t find his fork and spoon because they were the same colour as his bowl. In the heat of the moment, he exclaimed “WHERE’S MY FORK AND SPOON” as we all stared at his green utensils floating in his soup in his green bowl. In that instant, we knew that his words had to be our group name.

Progress of project for week 2

(17/06/24 – 21/06/24)

This was a quieter week for our project given all 5 of us were unavailable for 3/5 days this week due to our CNYSC Freshmen Orientation Camp Dry Run! Most of our time on the project this week was spent discussing our progress update slides and ordering components. Our Raspberry Pi and Arducam arrived over the weekend on the 22nd so coming next week we’ll kick start the electronics experimenting side!

We also managed to 3D print and test out our Rack-and-Pinion Design, some adjustments are needed but it was nonetheless a surprising success!

Progress of project for week 3

(24/06/24 – 28/06/24)

Most of what we ordered arrived this week, so we began testing out the arducam with the raspberry pi and we managed to get it to take pictures! We wanted to test out the other camera with our Arduino but we realized that we need a micro-SD card expansion to immediately store the images taken rather than sending them over the Arduino’s USB interface, so more ordering to do!

On the mechanical side of things, we tested out a few more iterations of our designs and found some aluminium brackets to try out some ideas for the base that holds everything together and supports the weight of the whole device.

Next week we plan to go full force in some initial integration and motor testing designs!

Progress of project for week 4

(01/07/24 – 05/07/24)

The focus of this week was to get all our stepper motors spinning in unison and roughly putting everything together – which we did! Just look at the 2 pictures below. We also redesigned the motor brackets (Bits that hold the motors onto the vertical pillars).

The logic of our design follows the flowchart below, TL;DR is that the Raspberry Pi is the main brain of the operations that will tell the Arduino what motor to spin and when before taking each picture. We’re hoping to find some software or Python library that will let us actually generate the 3D models on the RPi to really make this an all-in-one without the need for your own laptop where all you need is an SDCard to store the final generated model.

We also changed the design of the camera mount to be more modular as one of the ideas we have for this project is to actually allow the user to swap out the sensor for whatever best suits their needs, where the skeleton provides them with 3 DOF in a hemisphere should they not use this for 3D Photogrammetry.

Progress of project for week 5

(08/07/24 – 13/07/24)

After last week, we realized we needed to do some serious redesigns and mockups of the whole setup together. Until this point, we had only modelled each part separately, so with us wanting to move on to integration, we had to finally do a full mockup to scale.

Expertly done by our in-house 3D modeller, Mr Timothy. From here we can see the three motors that control each DOF. The blue boxes hold the motor that controls the Rack-and-Pinion mechanism. The 2 outermost motors control the rotation of the whole arm. The last motor in the centre facing upwards controls the turntable that the object will sit on. A close-up of the first two motors is shown in the picture below.

Some of our print attempts failed, and there were a few sleepless nights to reach this point, but it is all in the name of progress! You can also see one of our initial iterations of the modular camera in these models.

Progress of project for week 6

(15/07/24 – 19/07/24)

Week 6. The week of flaws. After we integrated everything in its initial basic form, we were able to detect a couple bugs here and there that we need to resolve to move forward. On the hardware side:

1. Firstly, we had issues with the strength of the motors that rotate the arm. Essentially, they were too weak, so we need to order stronger ones.
2. Secondly, we found that the cam arm needed motors on both sides of the build (similar to the 3D mockup from last week), else the arms would get stuck due to friction and start to bend.
3. Excessive torque on the arms potentially due to:
   1. Camera arm frame is not rigid as the magnets we used were not fully set in.
   2. Torque is not distributed and it’s just one gear driving a single rack which moves the entire arm (relating back to point 2).

On the software side:

1. Software used is too intense for Raspberry Pi to generate the 3D models as the Pi doesn’t have enough horsepower to run it.

The solutions we came up with after some back and forth was

1. Smaller pinion to reduce friction with the shell
2. White backdrop attached to racks. This provides a flat background to the object and could also acts as counterweight to reduce the torque on the motor
3. Shift the arm rotation motor to the opposite side of the setup (Left image below). So we’ll have one on each side like in the models in week 5. This way for the rotation of the arm, one side is a motor, and one side is a free axle and similar for the rotation of the arm.
4. Use planetary gearboxes as this provides more torque and does not take up much space
5. Rpi will relay the images to the computer to do the stitching via API calls, and will be readily available on the computer directly through a web app.

For point 5, we’ll need to try out API calls from different stitching software such as Agisoft, Recap Pro and Meshroom.

Essentially, we originally wanted EVERYTHING to be done on the Pi, but it’s way too weak to do the stitching and model generation. So instead, we’re planning to make a Web App that syncs with the Rpi to take pictures and upload them to existing stitching and model generation software. This way, we are still able to keep to our goal of having an all-in-one system where the user has minimal work to do to get their 3D models.

An interesting problem we had was that the original design for how magnets would hold the camera module together caused the magnets to rip themselves out of their holder. We’ll need to redesign these slots to prevent this.

Progress of project for week 7

(22/07/24 – 26/07/24)

This was a slow week for our group as 4/5 of us were involved in this year’s CNYSP Freshmen Orientation Programme and 1/5 of us was serving the nation through his first Highkey ICT. Just look at the image of our MnT group chat below, we didn’t meet or talk about MnT for the whole week because we were all too busy.

The drawing above is a schematic of one of the stepper motors that we are using. At the end of the week once FOP was over, we finally continued our discussions, now focusing on the platform that holds the object. We wanted to find a better way to control the right of the object while also being able to rotate it as desired. One of the ideas we floated around was to get a lift platform and jerry-rig it to start spinning as well.

In addressing the issues we found last week, we plan to shift the arm motor (translational motion of the arm) to the same side as the arm rotation motor, but this leads to the added complication of needing a form of rotational feedback to correct for movements.

Progress of project for week 8-9

(29/07/24 – 09/08/24)

A fun picture we took here shows the evolution of our camera mount, from a solid brick of plastic to what it is today as a more lightweight and modular design. The biggest development thus far is finding out that the Rpi is too weak to handle everything in-house. Originally, we wanted to generate the 3D models on the Pi, but it has proved far too weak for this. We can still have it as the main brain to control the camera and the motors through the Arduino, but we’ll have to offload model generation. This is where our Web App comes in.

Essentially the web app will interface the Pi, Arduino, Laptop and photogrammetry software all in 1 to make the user experience easier.

We’re also thinking of adding a rotary encoder of some sorts to measure the exact angles of motion, maybe this will help with precision! Since we have no space for a physical knob-style rotary encoder, we’re looking at using the AS5600 Rotary Encoder that relies on magnets to detect rotation. Just nice the back of each motor has the shaft exposed so we can plop a small magnet there and lay the sensor over it!

We also finalized our design for the camera holder to be a more modular one so the user can eventually swap out the camera for any other sensors they’d like!

Progress of project for week 10

(12/08/24 – 16/08/24)

Busy week for all of us as classes finally start and a lot of our Research Attachments begin to get into motion. Biggest thing this week was finally deciding to scrap the rotation sensors and trust the math. The whole idea of the rotation sensor in the first place was to prevent a problem that MIGHT occur, but we actually don’t know if it will occur. We’re gonna focus on getting to that stage to see if it actually is a problem. At least we know that it works and we know how to incorporate it should we need it in the end.

Progress of project for week 11

(19/08/24 – 23/08/24)

One of the main issues so far that we’ve just been putting off is that fact that the axes of rotation of both sides of the camera arm are not aligned. Currently, the side that rotates the arm has its axis through the geometric centre of the arm holders while the side that moves the arm linearly has its axis of rotation slightly higher (See the picture on the right). We need to redesign the side that translates the arm to align the axis of the arm movement motor and the arm rotation motor. Our first idea right now is to try out 2 smaller gears as one large gear would not fit in our current gear holders. It was absolutely annoying to figure out the right size of the gears (As evidenced by Timothy’s defeat in the picture below).

Progress of project for week 12

(26/08/24 – 30/08/24)

Having 2 small gears proved to NOT work as the torque exerted by them is not enough to drive the linear motion of the arm. We could either use a stronger motor or one with a planetary gearbox or use a larger gear. But to use a larger gear, we’ll need to make the gear holder larger and remove the teeth from one of the racks as the gear would be large enough to reach both of them. We’ll likely try out the gear solution first as it’s easy enough to reprint parts so we’ll see how it goes next week! Image on the right shoes one of the two gears used.

We also did up the platform that holds the object and incorporated the motor into it. However, the current design is too large and has some balance issues due to the weight of the additional parts so again, we’ll have to touch it up to make things fit nicely.

Progress of project for week 13

(02/09/24 – 06/09/24)

Continuing from last week where we found that our original design for the new platform was a little too large as the arms would extend out and collide with some parts of the platform. Just a few modifications were necessary to solve this. Another issue was the weight of the motor causing the platform to bend. To solve this we just added some mobile supports that can be adjusted and tightened. So once the user has adjusted the platform to the desired height, they need only tighten the supporting screws at the side!

Also continuing from last week where we found that using 2 small gears was too weak to drive the linear motion of the arm, so instead we’re trying 1 large gear. This would require us to enlarge the gear holder (Pill-shaped things) and one of the racks to remove all the teeth from it as the gear would be able to reach both racks.

Progress of project for week 14

(09/09/24 – 13/09/24)

We had a lovely little heart attack to start the week

Problem was solved pretty quickly, lab managers shifting things around in an attempt to tidy up the lab.

Pretty chill week because we’re all super busy but the new pill prints work but need some touching up (additional friction so they don’t fall under their own weight!). We came up with some ideas, mainly simple ways to increase the friction between the racks and pill shells (see our lovely sketches below!)

Progress of project for week 15

(16/09/24 – 20/09/24)

This week was our first taste of issues with our camera. For some, at the time, unknown reasons, we would have issues with our RPi not detecting our camera. A lot of arguing and back and forth and swapping components and reflashing the software of the Pi and factory resets later, we eventually found that we managed to screw up one of our camera’s cables! Easy enough fix, throw the cable away, use a new one and order more for backup.

Seems like we accidentally crimped the cable too hard with a bend that caused some of the connections to be a little messed up. Lesson learned!

This week we also started ideating how to solve the issue of backdriving/misalignment with the radial motion. Essentially sometimes when run, as it homes back to the origin position, the arm connected to the motor that drives radial motion would go off and ‘backdrive’ sort of. We were thinking of 3 solutions:
1. Brute force math it with gear ratios and whatnot

No picture here, we gave up matching it out

2. Sensor to detect the exact angle of rotation of the gear

3. Limit switch for feedback loop

Option 1 would be too imprecise and require a little too much brain power from us, Option 2 we did try for a while but found it to be slightly overcomplicated a bit too over-engineered. So we settled on Option 3, simple to implement and elegant!

Essentially, the limit switch would tell the Arduino when the arm has homed back to the max radial distance then switch off the arm motor. If the limit switch is not pressed when Home is activated, then it’ll tell the Arduino to turn on the arm motor!

Progress of project for week 16

(23/09/24 – 27/09/24)

Funnily enough, after deciding to use the limit switch, we figured out that option 1 is actually REDUNDENT regardless!

We also started to cook up how we can use an accelerometer to get precise angles for our entire project as up till now we’ve been mathing the angles.

We did indeed get Liyang’s magic touch. Limit switch hooked up and everything sort of put together, ready for testing!

Progress of project for week 17-19

(30/09/24 – 18/10/24)

So, the camera stopped working for seemingly no reason! We keep getting the same error and even with a lot of googling and forum diving and even trying to contact the developers/manufacturers, we couldn’t solve it!

This derailed our progress for like more than 2 weeks. We kept flashing new versions of the RPi OS, reverting to old versions, changing cameras, buying new cameras, changing cables, praying and screaming for diving inspiration. We even tried to BUILD THE SOFTWARE OURSELVES ON THE PI??? Like, that’s crazy. I don’t think you’ll ever hear of anyone at a hobbyist level BUILDING entire packages from random Github links on their RPi.

Eventually, after a LOT of reading through all the error messages and deciphering that programmer lingo, we figured out that we had to manually install each library from older versions. Problem solved! Wow so simple! Besides this, we’re still having some issues with the radial motion as both arms go out of alignment every now and then. We can only think of convoluted and over-engineered ways to bring the motion to the other arm so they both move together. Belt, 90-degree gears, axles…

In the end, we decided to drop our focus on radial. It still is there but it’s just not going to work as well as we intend it to. It is what it is!

Progress of project for week 20

(21/10/24 – 25/10/24)

Finally after all the trials and tribulations, we actually got to test it… AND IT WORKED LET’S GOOOO. Not the prettiest as the lighting was inconsistent so we’re gonna install a ring light over the camera to try to address this. We’re also gonna keep messing around with the camera’s settings until we find something we like! BUT FOR NOW LOOK AT IT!!!!

Ok, It’s a little blurry but here you can see the shape of the model!!  Artefacts in the background can be minimized with better masking mages and yeah the colour is quite off and inconsistent (it should be all white)

Progress of project for week 21

(28/10/24 – 01/11/24)

On top of the ring light, we cut out a nice ring of cloth to act as a diffuse so that the light isn’t too sharp on the models, and we added a lightbox over the top to provide more global illumination. The skirt you see is something we played around with to create a consistent white background, bt I think we’re gonna remove it!

Yeah, we removed the skirt. When taking the images now for an ideal setup, we prop up an additional white background for consistency. That’s what we are doing for our next data set which should hopefully be the last test…

Progress of project for week 22, We’re done!

(04/10/24 – 08/11/24)

SHUT THE FRONT DOOR WE COOKED SO HARD! It works, WONDERFULLY. We took our refined data set (With more images, better lighting), ran everything from the webapp, got the 3D model, cleaned it up and sent it straight to be 3D printed and WOW it works great! We’re done, MnT is over! Time to present.

The end.

Presentation went well and we ran one last run before our presentation started. Timed it just nice to show it off and wow this was the best yet. A lot better than we expected! MnT is NOW over. Team Fork n’ Spoon signing off~