SOFTWARE

ZERO TO ARMADILLO*

*Armadillo cos at the end we ended up nocturnal and with bad posture. Also our eyesight worsened 😛

Into the Unknown

The journey was a steep climb, as none of the group members have experience in Web Development. As we began our initial deep dive into the deep end of Web Dev, we realised that we had not 1 mountain to climb, but 4: they are programming languages in their own right, and each has their own “rules” and syntax, as well as their own role to fill, in the puzzle that is Web Development:

HTML markup language, for structure;
CSS, for aesthetics;
JavaScript, for interactivity behaviours;
Flask, which is a subfield of Python, which facilitates the setup of a Web Server

Flask acts like a publisher – an excellent webpage made from HTML CSS and Javascript needs to be “published” to the wider Internet for others to access. Otherwise, it would just exist on one’s computer – and be rendered essentially useless. It is a library in Python, but requires a wider understanding of networks and is so inextricably linked to HTML and Javascript, that it could be considered part of Web Dev.

These are all steep hills to climb, and it would probably take years for an individual to be confidently proficient in them, and many more years to be a full Web Developer. However, we are short on time (relatively) and we are in the immensely fortunate position to be guided by experienced and wonderful mentors of the MnT lab – namely Justin and Qi Jie, whose experience and advice came in invaluable in our development and learning process. They “supercharged” our learning, and we owe them a deep debt of gratitude.

After gaining some insight into HTML, CSS and JavaScript languages, we broke down the deliverables to achieve, and we tackled them one by one:

firstly, we must learn how to create a webpage. from scratch, with minimal aesthetics first – that comes later
secondly, we need to learn how to handle video feeds, how could we embed them into our webpages?
thirdly, we need to be familiar with our own video streams, and need to find a way to pipe in a *live* feed into our webpages, from our cameras (spoiler alert: it is *much* harder than we thought, and easily one of the hardest parts of the software development)
we need to learn how to control water pumps (physical modules connected to the Raspberry Pi), using code
we need to learn how to set up controls in our webpage, and link it to the code that we established earlier in part 4. this endeavor could be split into several parts
1. firstly, we need a steering system for our pumps, so that our pumps could dynamically adjust depending on the steering commands that the user wants
2. secondly, we need a system to change the speed/power directed to the pumps as needed, an analogy would be an “accelerator” to our “steering wheel” in 5-1
3. the challenge, as we quickly found out, would be that these controls are done on a continuous scale, instead of a discrete system whereby there are a finite number of “buttons to press” eg up down left right. Instead of using that system (which, admittedly, might be easier to program for), we opted for a more intuitive and user friendly system of a round slider (Image 1) which mimics a steering wheel – the most intuitive/easy-to-use/hopefully-idiot-proof system of steering!
we need to learn how to remotely access our webpage/controls, even when we are on a different internet connection. This would facilitate the “remote control” aspect of our craft, even at large distances away
and finally, we would need to make it look nice and professional 🙂

Now, let’s take a deeper dive into each of the seven components:

Image 1: An image of the round slider mentioned in the paragraphs earlier – which we found to be deceptively simple

Part 1: Order, Chaos, and Everything in Between

As we first embark on the journey of Web Development, we are almost completely at a loss. We have no idea how to make a webpage, much less how to deploy and design them. At the first few days, the software team was in a complete state of loss and confusion. Many new terms were thrown around, and we had to make sense of them.

We eventually gained our footing as we discovered the main entrypoint into the world of WebDev – the HTML (HyperText Markup Language) and CSS languages. These languages allow us to create the structure and astecthics of our webpage first, and allowed us to begin our journey.

To learn these new languages, we googled and relied on many excellent web tutorials. One of the primary methods we used to learn Web Dev was Harvard University’s CS50 Course: a free MOOC made available to the world under the OpenCourseWare system. The course could be found here: https://cs50.harvard.edu/, and we greatly encourage anyone that’s interested to pursue Computer Science, or coding, to give it a look. A word of caution though: the pace of the course is extremely fast!

After a week of intense study and coupled with the guidance of our mentors, we are able to read, write, and work with HTML and CSS code rather confidently. However, as we would quickly find out, our journey has only just begun..

Part 2+3: Let There Be Video

The next part of our progress would be to add and embed videos into our webpages. This is a realm of confusing terminologies as well as many different paths to take to get to the same result.

We first experimented with embedding a youtube video onto our webpage using an <iframe> tag. This was simple and worked well without any hiccups, but what we want is a live feed transmitted from our cameras and not a pre-existing video. From there on, we did some research online and compiled a few methods of streaming live videos and attempted integrating them with our system:

Streaming using the Amcrest Cloud Server

It made sense to first look at Amcrest’s own web services to see if they provided a way to broadcast their live feed onto the internet. We found out that the Cloud Server was able to intermittently backup recordings from our feed onto an online server from which we could retrieve them. We tried implementing this with the OneDrive server, and it was successful! However, this was not yet near what we had hoped to achieve – that is, to display a live feed on our webpage. With Amcrest’s Cloud Server, our feed would simply be a recorded playback of what our cameras see 🙁

Streaming using external video management software

The Internet provided many other supposedly brilliant solutions to our problem, of which we sifted through and handpicked the note-worthy ones from. Unfortunately, these platforms did not live up to their marketing and after a few days spent at toggling between Blue Iris (another video management software), OBS and VLC, we decided to move on…

< insert subtitles>

Broadcasting live feed on Youtube

We then attempted borrowing some ideas from the multimillion dollar platform, YouTube. We referenced a tutorial that guided us through setting up a DDNS (dynamic Domain Name System) that maps internet domain names to IP addresses. We then port forwarded our connection and plugged in our livestream link into our <iframe>. Unfortunately, we were not sure what went wrong but our webpage was unable to establish a connection to retrieve any video from YouTube.

https://support.amcrest.com/hc/en-us/articles/360050181252-How-to-Setup-DDNS-on-your-Device

RTSP (Real Time Streaming Protocol)

We then chanced upon this fascinating concept that ALSO seemed to be just what we wanted (again)! RTSP allowed us to pull an IP camera’s stream using a specifically formatted URL (rtsp://[username]:password@IPaddress:554/cam/realmonitor?channel=1&subtype=0) that could be plugged into our <iframe>. However, we soon realised that because our IP camera was private, it required a username and password to access. This username and password could not be sent over our Internet browser (Google Chrome, Safari etc).

THE one:

We (kudos to Justin!) found a GitHub repo that worked (https://github.com/jantman/python-amrest-noauth-proxy). This repo allows us to set up a Python/Flask reverse proxy server that removes the HTTP Digest auth from our Amcrest IP cameras. This means that the reverse proxy can handle the HTTP Digest authentication on our behalf – our live video feed can still remain secure while removing the need to insert the camera ID and password directly in the URL.

We then needed to be able to import this repository from the web into our Raspberry Pi system. This involved using the command line from within the Pi (which we had SSHed into, or ‘remote desktop’) to:

navigate the file directory — purple files refer to directories which cannot be removed with a simple ‘rm’ (safety feature) while green files refer to executable files
clone the git repo (git clone https://github.com/jantman/python-amcrest-noauth-proxy.git)
creating a virtual environment (python3 -m venv (name) )
activating the virtual environment (source (name)/bin/activate )
installing the files needed for the repo to run successfully into the virtual environment by running the ‘requirements.txt’ file from the imported folder (pip3 install -r requirements.txt )
creating and running (./(filename).sh) a shell script file that contain variables that the Pi would need to set up the reverse-proxy

Since we have multiple (2) cameras, we needed to run 2 instances of this proxy on 2 different ports. With this, we were able to access our camera’s live feed (with about a 1s latency)…

However, if we were too stick by this current system, we would have to execute (./(filename).sh) the shell script files for each camera feed every single time we wanted to step up our camera system. We anticipate that this would prove to be a hassle in the long run — therefore, instead of having the executable file in a separate shell script file, we decided to shift the code contained in the file directly into our Python code. After ensuring no variables conflicted with one another, we were finally done with our live feed! The 2 cameras hosted on different static IPs were able to be run concurrently with a slight latency, without the need for a virtual environment and executing a file every time.

Part 4+5: controlling water pumps??

In order to control the water pumps, we had to make use of the GPIO library(??) that the Raspberry Pi uses. We searched up a simple ‘blinking lights at fixed intervals’ tutorial and assembled the circuitry in order to familiarise ourselves. A short video on the final assemblage can be viewed here (Blinking Lights). This demonstrated helped build up our confidence to tackle our actual goal — replacing the blinking LED light with 8 of our water pumps.

We drafted up some checkpoints or mini-goals-t0-meet:

having a main on-off switch that when ‘off’, renders all other pump controls useless, and only when ‘on’ will the other pump controls work (Tony likens this to turning the key in the car??)
having an accelerator that would be able to vary the power delivered to all pumps, uniformly (0% to 100%)
having a steering-wheel-of-sorts to vary the power distributed between the left group of 4 pumps and the right group of 4 pumps

Main on-off switch

It turns out this was pretty simple to code — we just needed an over-arching indicator tagged to our switch — if the indicator indicates ‘on’, the value assigned to the pumps (and hence the power delivered to the pump) will be updated; if the indicator reads ‘off’, the values will not update (remain at 0).

Accelerator

In order to vary power, we researched online and found that the Raspberry Pi system does not read the power in terms of Watts (nor does it read it as current or as a percentage out of 100). It wasn’t as user-friendly as we hoped (cries). We had to make use of the Adafruit PCS9685 library — the library enabled us to assign values to each of the 16 channels of our motorway indicating its number in the following manner: pca.channels[1].duty_cycle = value; pca.channels[2].duty_cycle = value and so on and so forth. The catch was that the value that is understood by the computer had to be in terms of hexadecimals — for example, assigning a value of 0xFFFF and ??? tells us the Pi to deliver 100% and 50% of the power it can to the pump respectively. (?? pls fact check this)

Steering wheel

The concept behind our steering wheel is that it would be the element determining (on user input) the distribution of power between the left group of pumps and the right group of pumps. In other words, if the slider were positioned in the left half, the left group of pumps would receive a smaller percentage of the power than the right group of pumps, meaning as a whole, the boat should turn to the left. Conversely, if the slider were positioned in the right half, the right groups of pumps would receive a higher power than the left group of pumps, causing the overall movement of the boat to be to the right. Regarding the aesthetics of our steering wheel, we decided that by adapting a normal slider and making it rounded, we hoped this simple design would make our controls more intuitive to use.

The Challenge

The challenge came in translating the different quantifiers (accelerator value, steering wheel distributions) from HTML (the face of our website, where the user will adjust the controls) to our Python program (which is the brain that controls the power delivered to the pumps).

Part 6: Achieving Network Indepedence

The final barrier to be overcome was to publish the webpage to the wider internet.

By allowing any device with an internet connection to any Wifi/4G network to access the web app

This is because currently at this point in time our web application could only be accessed by other devices on the same wifi network, it is localised in that sense of network accessibility.

This proved to be one of the main challenges we faced, as it is a whole field of network programming, which we had not delved into, as it is rather tangentially related to web development as a whole

With the invaluable help of our mentors, we were pointed to a form of solution known as the Fast Reverse Proxy (FRP) Setup

Essentially, it consists of an external server being hosted and connected to the wider internet, and our web application connected to it.
For our web application, the DigitalOcean server hosted by one of the mentors was used for the FRP
The server as well as our web application would each contain a “file” that would “talk” to each other,
The server would hold the frps (s for server) file, while our web application would hold the client (frpc) file.

A client who’s connected to the internet can access our publicly available link, which will send a request to the frps file
An open source program known as Apache will handle the incoming request, and connect it with the frps file in the DigitalOcean server

And finally the data will be tunneled to our web application using the frpc file

One of the benefits of this solution is that it is widely scalable, multiple clients could be connected to the website at once, and the Apache software will handle the multiple lines of traffic.

The tunneling process as illustrated between the raspberry pi and the VPS is encrypted, and encryption tokens would have to be exchanged with the server for the data to be transferred accordingly.

These are hardcoded in an initialization file for the frpc, which is shown here.

Taken in totality, all these allow us to step away from our craft, and still retain the ability to control and observe the feeds from it. Furthermore, this could be extended to long distance situations, such as Work from home fr eg.

Concluding Thoughts

As it could be seen, our code ecosystem comprises a lot of overlapping parts, HTML intertwined with Javascript which enacts changes on more HTML elements which triggers even more changes. And these are all capped off with a CSS layer of aesthetic design.

In summary, the software development process was an illuminating and insightful one, with its ups and downs, and here are some quick statistics for our efforts.

We would like to express our deepest thanks and gratitude to the Mentors of the MnT Lab – Justin, Qijie, Tony, Limzy, as well as the CNYang Scholars Programme for the resources and the funding. And of course, Dr Ho, Dr Alicia and Hanyang as the course coordinators for this module!

This has been an unforgettable experience full of ups and downs, and a fruitful learning experience, that we would forever cherish.

Project B.O.A.T. Software Team

Signing Off