Achieving Autonomy

 

Testing The Capability Of The AI Algorithm

 

Object Detection

The Jetson Nano equipped with an Arduino and a camera shows how a combination of hardware and software can achieve object recognition functionality. In the above video, the system’s algorithm has been trained on a comprehensive dataset comprising more than 1000 images containing representations of both tables and people. This training enables the algorithm to learn distinctive features associated with these objects. When the camera captures a new visual scene, the algorithm processes the incoming data, identifies relevant patterns, and compares them to its learned knowledge. In the context of the video, the algorithm successfully detects and labels the presence of a table and a person by matching the extracted features with its pre-existing understanding.

 

Movement Tracking

Apart from identifying an object (person), the algorithm also produces a position output in an XY coordinate system and updates the position through a loop, enabling it to perform tracking in realtime. This functionality will allow Ball-E to determine how far the object is and reposition itself closer to the object.

 

Detailed Implementation of AI into Ball-E

Data collection:
1. Platform: Roboflow. Enables fast annotation of videos.
2. Augmentation: Beside the data collected, we use some data augmentation methods to increase the number of training images to 2000+. Methods include: tilting, crop, modify colour intensity, etc. This enhances the model robustness to environmental noise while providing more data.

Training:
1. The model used to train is Single Shot Detector (SSD) for object detection. This is a task that involves identifying objects in an image and locating them by bounding box.
2. Despite this model not being the state-of-the-art (SotA), we decided to proceed with it because there is a library using it that is compatible to the Jetson Nano. The current SotA model is called YOLO (You Only Look Once), which is faster and actively maintained on Github. In the future, we can incorporate this into computer vision and enable faster and more accurate object detection.
3. We passed the whole dataset through the model 30 times to allow the model to learn from the dataset. The way the model learns is through gradient descent – the accuracy loss computed from training a small batch (subset) of the dataset will be used to compute the direction of greatest descent to minimise the loss. Over time, this will help the model lower the loss on prediction.
4. However, this does not mean we can keep training the model indefinitely. Firstly, it is time consuming to train a model; a lot of GPU resource is needed. Secondly, if the model is too simple, reiterating training will not significantly improve model performance because the model does not have enough parameters to generalise the dateset. If the model is more sophisticated (more layers and more parameters), over-training will lead to overfitting; this means the model will memorise the dataset (leading to lower loss in training) but not actually generalise on other images.

Deployment:
1. Training a model will result in a PyTorch file. However, for faster deployment, especially on edge-AI devices which has significantly less computing power than our personal computers, GPU utilisation is essential. For example, at first, when we deploy it as a PyTorch file and utilise only the CPU of the Jetson Nano, we managed to achieve only 1.2FPS.
2. To utilise the GPU, we employed TensorRT, an NVIDIA library which allows model to be exponentially more optimised to run on GPU. This involves fusing some of the layers and combining prediction heads in order to perform faster on the GPU. As a result, we are able to significantly improve inference to 30FPS.

Tracking:
1. This tasks faces significant challenges because it is hard to estimate distance to an object using only a monocular camera. The adopted methodology was to compare the predicted size of the object on the image with the size of the object at a pre-determined distance, allowing for some calibration to be done.
2. The control mechanism was Proportional Control – the command movement for the bot is proportional to the difference between the target and the current status. However, this has the critical weakness of not being able to estimate how fast we are approaching the target, making overshooting a frequent problem.
3. In the future, a more completed PID control (Proportional, Integrative and Derivative) should be implemented in order to allow the bot to better follow a target.

 

Optimising Tilt

 

 

We improved the weight design for the stepper motor to make use of any additional space and incorporate additional weights. This will improve the stability of Ball-E in the upright position and allow a larger angle of tilt, hence reducing the turn radius.

New Outer Casing

After analyzing properties of different materials, our group chose polycarbonate as the new material as it possesses the following suitable properties:

1. Strength and Durability: Polycarbonate is known for its high impact resistance and toughness. This property ensures that the casing can withstand accidental impacts, scratches, and rough handling during Ball-E’s operation in confined spaces and challenging environments such as drains.
2. Lightweight: Polycarbonate is relatively lightweight compared to many other engineering plastics and metals. This is advantageous for Ball-E’s casing because it helps maintain the overall weight of Ball-E at a manageable level, which is important for mobility and ease of operation.
3. Transparency: Polycarbonate is transparent, allowing for visibility of Ball-E case such that the internal camera can capture Ball-E’s surrounding clearly. This transparency can be useful for monitoring Ball-E’s operation and inspecting its surroundings for defects.
4. Chemical Resistance: Polycarbonate has good resistance to a variety of chemicals, oils, and greases. This is essential when Ball-E operates in potentially hazardous or contaminated environments.
5. Recyclability: Polycarbonate is recyclable, contributing to sustainability and environmental considerations

We obtained custom made hemispheres from GreenCast, and incorporated the new casing into our design.