3.1 Machine programming
Earlier generations of artificial intelligence used the rules-based approach to program computers to learn and reproduce the human language. However, this has human limits as computers can only respond based on the algorithms they have been programmed with, or as they say, “Garbage in, garbage out”. Some language software use this rules-based approach to program the computer to compose sentences and semantically-correct phrases.
One example is LuaJIT, which is a scripting language designed and created by Tecgraf, a team at the Pontifical Catholic University of Rio de Janeiro (PUC-Rio). LuaJIT is able to support procedural, object-oriented, functional and data-driven programming as well as data description.9
The script for the video above was written using LuaJIT, using data in the form of previous scripts written by the host himself. The program processes all the data that has been inputted before creating its own script. However, while the software is able to construct grammatical sentences, it is unable to construct meaningful sentences that combine logical propositions using the rules-based approach. The end result is a sentence that is grammatically correct and logical in its parts, but ungrammatical nor illogical as a whole, complete sentence.
The way to progress beyond such limits is to develop the ability for machines to learn, in the same way that humans are able to learn, deduce patterns and infer possibilities.
3.2 Machine learning
Technological advances have made machine learning a reality. Machine learning refers to computer software that can learn autonomously beyond the algorithm that it has been programmed with.10 Through the use of pattern recognition, computers can learn from and make predictions based on available data. For example, the quality of language translations improved dramatically in 2007 when Google transited from a rules-based approach to a statistics-based one to train its translation engines.11
The dark side of machine learning is that programmers cannot control what the machine learns. In 2016, Microsoft experimented with Tay, an artificial intelligence chatbot that was designed to interact with millennials via Twitter and messaging apps Kik and GroupMe, so as to learn the way millennials spoke. Unfortunately, Tay ended up learning racist and Nazi phrases from the community it interacted with.12
3.3 Deep learning / Digital neural networks
The next bound is deep learning with digital neural networks. Modelled after brain neural networks, digital neural networks broadly comprise (a) neurons arranged in multiple layers, (b) connections between the processing elements (artificial neurons), and (c) the adaptive weights of each of these connections. Each weight indicates the relative importance of a particular connection. If the total of all the weighted inputs received by a particular neuron surpasses a certain threshold value, the neuron will send a signal to each neuron to which it is connected in the next layer. The network then learns through exposure to various situations, by adjusting the weight of the connections between the communicating neurons.13
Scientists from the University of Sassari and University of Plymouth developed a cognitive model composed of two million interconnected artificial neurons. The system is known as ANNABELL (Artificial Neural Network with Adaptive Behavior Exploited for Language Learning). From a “blank slate” starting point without any pre-coded language knowledge, ANNABELL has already demonstrated an ability to learn 500 new words – the language capacity of a four year-old.14
3.4 Genetic algorithms
Inspired by natural selection, genetic algorithms mimic the evolution of potential candidate solutions towards the optimal solution. Each candidate solution has a set of properties (its chromosomes or genotype, or bits) that are expressed in terms of a genetic representation which can be mutated and altered, and a fitness function. As the candidate solutions undergo iterative processes or generations of evolution, their genotype may be mutated or recombined, and the resulting fitness is evaluated.15
Genetic algorithms are used in natural language processing, including syntactic and semantic analysis, grammar induction, summaries and text generation, document clustering and machine translation.16 It has been found to be useful in the generation of dialogue phrases.
The development of artificial intelligence has been rapidly progressing as a result of these technological advances in machine learning, particularly in the domain of language processing. Computers are now exhibiting a heightened ability to convey and understand utterances in the natural languages. What could this mean for the big, bright future of AI? For now, let’s take a look at some applications that have already been invented and which play important roles in bettering our lives. Indeed, when machines understand language, humans are the ones who benefit most!