Introduced in 2015, Skype’s auto-translate function serves to improve communication between people of different native languages by translating video calls in real-time.¹⁷ Imagine speaking with an online friend from halfway across the world, and as you see him on your computer screen, Skype rolls out sentence-by-sentence translations for whatever he is saying, like real-time subtitles.

It is a technology that breaks down language barriers between people. This has huge implications not just in a person’s social life, but also in making other areas of life such as work and travel much easier.

Granted, Skype’s auto-translations are not quite there yet and often fails to give accurate translations of utterances. But what is illuminating is how, just a decade ago, no one would have predicted that such technology would be made possible. Its mere inception is a reminder of how much room there is for development, and even though there are faults in the app’s language processing ability, we can be sure that with more research and improvisation, such faults might be ironed out in the future.

It has already been done with devices that are closer to us. Look at your mobile phone, or even your computer. Within it lies your own personal assistant. Take Apple’s Siri, for example. Its inception in 2011 was not quite successful. People were intrigued by its ability to chat, to answer questions, and to provide route directions. Yet, more often than not, Siri invoked a lot of frustration by misinterpreting commands.¹⁸

But that was in 2011. Now, because of rapidly evolving new methods of machine learning (such as deep learning and genetic algorithms, as we’ve discussed earlier), Siri is more intelligent. The difference is that Siri has been supercharged with artificial neural networks. And, according to Eddy Cue (below; inset), Apple’s Vice-President of internet software and services, it has “impacted all of Siri in hugely significant ways” such as its speech recognition, natural language understanding, execution of requests, and responses to the user. Because Siri now understands and uses language so much better than before, she has become an efficient digital personal assistant to whoever owns an Apple device, many of whom find that they cannot do without her in their day-to-day lives.

The repercussions associated with successful language processing abilities in machines are huge. When machines are able to grasp a language and use it well, they have the capacity to make our lives easier. The best thing is that the area of language learning in Artificial Intelligence is continuously evolving and expanding. In fact, one of the big predictions for AI in the year of 2017, according to MIT Review’s Will Knight, is that we can “expect further advances” in the area of language learning.¹⁹ It will be more than just voice recognition and obeying commands.

It may prove to be a formidable challenge, but how exciting it would be to finally have machines understand us in ways we never once thought possible.

3.1 Machine programming

Earlier generations of artificial intelligence used the rules-based approach to program computers to learn and reproduce the human language. However, this has human limits as computers can only respond based on the algorithms they have been programmed with, or as they say, “Garbage in, garbage out”.  Some language software use this rules-based approach to program the computer to compose sentences and semantically-correct phrases.

One example is LuaJIT, which is a scripting language designed and created by Tecgraf, a team at the Pontifical Catholic University of Rio de Janeiro (PUC-Rio). LuaJIT is able to support procedural, object-oriented, functional and data-driven programming as well as data description.⁹

 The script for the video above was written using LuaJIT,  using data in the form of previous scripts written by the host himself. The program processes all the data that has been inputted before creating its own script. However, while the software is able to construct grammatical sentences, it is unable to construct meaningful sentences that combine logical propositions using the rules-based approach. The end result is a sentence that is grammatically correct and logical in its parts, but ungrammatical nor illogical as a whole, complete sentence.

The way to progress beyond such limits is to develop the ability for machines to learn, in the same way that humans are able to learn, deduce patterns and infer possibilities.

 

3.2 Machine learning

Technological advances have made machine learning a reality. Machine learning refers to computer software that can learn autonomously beyond the algorithm that it has been programmed with.¹⁰ Through the use of pattern recognition, computers can learn from and make predictions based on available data. For example, the quality of language translations improved dramatically in 2007 when Google transited from a rules-based approach to a statistics-based one to train its translation engines.¹¹

The dark side of machine learning is that programmers cannot control what the machine learns. In 2016, Microsoft experimented with Tay, an artificial intelligence chatbot that was designed to interact with millennials via Twitter and messaging apps Kik and GroupMe, so as to learn the way millennials spoke. Unfortunately, Tay ended up learning racist and Nazi phrases from the community it interacted with.¹²

 

3.3 Deep learning / Digital neural networks

The next bound is deep learning with digital neural networks. Modelled after brain neural networks, digital neural networks broadly comprise (a) neurons arranged in multiple layers, (b) connections between the processing elements (artificial neurons), and (c) the adaptive weights of each of these connections. Each weight indicates the relative importance of a particular connection. If the total of all the weighted inputs received by a particular neuron surpasses a certain threshold value, the neuron will send a signal to each neuron to which it is connected in the next layer. The network then learns through exposure to various situations, by adjusting the weight of the connections between the communicating neurons.¹³

Scientists from the University of Sassari and University of Plymouth developed a cognitive model composed of two million interconnected artificial neurons. The system is known as ANNABELL (Artificial Neural Network with Adaptive Behavior Exploited for Language Learning). From a “blank slate” starting point without any pre-coded language knowledge, ANNABELL has already demonstrated an ability to learn 500 new words – the language capacity of a four year-old.¹⁴

 

3.4 Genetic algorithms

Inspired by natural selection, genetic algorithms mimic the evolution of potential candidate solutions towards the optimal solution. Each candidate solution has a set of properties (its chromosomes or genotype, or bits) that are expressed in terms of a genetic representation which can be mutated and altered, and a fitness function. As the candidate solutions undergo iterative processes or generations of evolution, their genotype may be mutated or recombined, and the resulting fitness is evaluated.¹⁵

Genetic algorithms are used in natural language processing, including syntactic and semantic analysis, grammar induction, summaries and text generation, document clustering and machine translation.¹⁶ It has been found to be useful in the generation of dialogue phrases.

The development of artificial intelligence has been rapidly progressing as a result of these technological advances in machine learning, particularly in the domain of language processing. Computers are now exhibiting a heightened ability to convey and understand utterances in the natural languages. What could this mean for the big, bright future of AI? For now, let’s take a look at some applications that have already been invented and which play important roles in bettering our lives. Indeed, when machines understand language, humans are the ones who benefit most!

2.1 It needs to understand the differences between utterances, sentences and propositions.

The spoken language can be broken down into three significant layers – utterance, sentence and proposition. The most concrete of the three is utterance which refers to the act of speaking. It involves a specific person, time and place but does not encode any special form of content. Utterances are identified as any length of speaking by a single person with distinct silence before and after. It need not be grammatically correct and can be meaningful or meaningless.

A sentence refers to the ‘abstract grammatical elements obtained from utterances’ .⁵ Unlike utterances, sentences cannot be defined by a specific time or place. They are grammatically complete and correct strings of words that express a complete idea or thought. An example of this would be if someone asked “Would you like a cup of tea?”. The reply could be “Yes, I would like a cup of tea.” or “No, I would not like a cup of tea.”. These two utterances can be considered sentences as they are both grammatically complete and correct and express a complete thought. However, if the reply was either “No, thank you.” or “That would be nice.”, they would be considered utterances but not sentences as the first is not grammatically complete and the second does not express a complete thought. The person who had asked the question, however, would still be able to perfectly understand the reply.

These non-sentence utterances play a large role in our day-to-day communication, and while they cannot be considered sentences, they still contain the abstract idea of a sentence. This brings us to our third layer of language, propositions. Propositions are concerned with the meanings behind the non-sentence utterances as well as whole sentences. It can be defined as the meaning behind an utterance and are claims or ideas about the world that may or may not be true.

As suggested by the Chinese Room Argument, computers are able to recognise utterances and sentences through a series of commands and are even able to respond accordingly. However, computers seem to face difficulty understanding and constructing propositions.

 

2.2 It needs to understand that the spoken language is able to encode meaning in different ways, such as patterns and unspoken meanings.

Speech is also able to encode meaning in ways that the written language cannot such as through tone, context, shared knowledge etc. It allows things to be left unsaid and indirectly implied. An example to demonstrate this would be if someone asks, “Have you finished writing your essay?” Your response could be “I started writing it but…” and end it there. While in the written language, a ‘but’ would indicate that there is more to come in the sentence, in the spoken language, the person would understand that the sentence is finished and fully comprehend the reply in association with the facial expressions conveyed with that utterance. True language comprehension requires computers to identify hidden and context-dependent relationships between words and clauses in texts.

 

2.3 It needs to have the ability to understand concepts, mental representations and abstract relations.

The semiotic triangle shows the relationship between concepts, symbols and real world objects. Together, they form the building blocks of language. Words contain representations of the world, as well as abstract, relational concepts embodied by the words. Concepts are the abstract ideas that these words represent. As such, they possess perceptual qualities as well as sensations through association. The word “dog” can possess the following representations: mammal, furry, barks, has a tail, man’s best friend, canine. The language processor must be able to know what each word represents, and be able to put it together to know what it is referring to even in the absence of immediate stimuli.

Insofar as a computer programmer can programme for the meaning of each word to be the multiple representations for each concept, similar to a dictionary, as well as the various grammatical and syntactic rules of language, then we can say that the computer would be able to learn the basic building blocks of the human language. This, however, limits the ability of the computer to the data and codes it has been programmed with.

 

2.4 It needs to have the ability to understand that others have different perspectives from itself.

As we have seen, language is very much influenced by thought. We can express words, sentences, utterances, but only if we are able to conceptualize them first. Do machines possess this mental aspect of language – the faculty of imagination and rational thought? It seems like the language that they know and use to communicate with one another, or with humans, is based upon only an algorithm.

This leads straight into the Theory of Mind (ToM), which is the ability to attribute mental states to oneself and others, and to understand that others have different perspectives from one’s own.⁶ The ToM has been found in non-human primates and humans, and is closely connected to the way we empathize with others. It explains our emotional intelligence as well as our intuitive desires to understand other humans and why they think the way that they do.

Our minds have sometimes been likened to machines (based on its abilities to process information, solve mathematical problems, and store important data), yet can we switch the nouns around to say that the machine functions like a mind? If we are on the side that is compelled to say that computers understand language, then we must say that they have a mind, since understanding language is a trilateral feat in which the mind plays an important role.

However, even though a computer may seem to understand our basic utterances that have their meanings entrenched in the structures of the world, it seems to meet measurable difficulty when it comes to understanding nuances. Aspects of speech like sarcasm, dry humour, and puns may not be understood by the computer unless they have been programmed to, and even this programming might require a technology far reaching beyond our times. But this reveals an important fact – sans programming, sans deliberate action in creation, the computer just cannot adapt to these aspects of human language.

 

2.5 It needs to have the ability to understand when different languages are being used.

Computer programmes can already identify what national language is being used through profiling algorithms. These pick out the words used in the text, and match them with the most commonly used words of a particular language, to identify what particular language is being used. It becomes trickier, however, when code-switching or hybrid languages such as Singlish or Spanglish are used. These mixed languages combine vocabulary, grammatical and syntactic rules from at least two different languages. In the case of Singlish, a relatively simple sentence could combine the lexicon and grammar from 6 different Singaporean languages and dialects. Decoding this would require a highly sophisticated computer, if at all possible.

 

2.6 It needs to adapt to the constantly changing and evolving spoken language.

Some linguists consider that the nuances contained in language make it impossible for computers to ever learn how to interpret. In addition, these nuances are not static but constantly evolving – application of these nuances rely not only on syntactic, phonetic and semantic rules, but also on social convention and current events.⁷ The word “bitch” could mean different things when spoken by different racial groups, in different social situations, and with different tones.

 

2.7 It needs to process speech, which has added complexity beyond written language.

It is difficult for computers to achieve good speech recognition. Spoken language differs widely from written language and there is wide variation in spoken language between individuals, such as differences in dialects.⁸ Researchers are looking into identifying what language is being spoken based on phonetics of the sounds of human speech. However, it is not as easy to distinguish phonemes and individual words when spoken, particularly when different speakers have differing accents, tonality, timbre and pitch of voice, pauses and pronunciations. One way to overcome this is through machine learning by providing massive amounts of data for the computer to discern patterns.

In the following section, we look at some of these methods of machine learning that have been developed to enhance the language comprehension skills of a computer. 

Imagine that you are locked up in a room with no windows, save for two slits cut out in one of the walls. One slit is marked “INPUT”, and another is marked “OUTPUT”. There are countless books all around you and an instruction sheet on the wall. It tells you that you are about to receive some notes through the INPUT slit, and that once you do, you are to flip through the books deposited all around you to locate a correct response to what is written on the note. As an analogy, imagine that written on an INPUT note is the string of symbols “^%$@&&”. Your job is to flip through the books to locate this specific string of symbols and its indicated response, which, for example, might be “&#$!++”, and then proceed to write them on a separate slip of paper which you will pass through the OUTPUT slit.

Such was The Chinese Room Argument, a thought experiment* posited by John Searle in 1980, except that what the participant would be dealing with are Chinese characters instead of keyboard symbols.¹ Searle’s person-in-the-room would be answering questions written in Chinese posed by real Chinese people outside the room. Even though this person-in-the-room doesn’t understand a word of Chinese, he would be able to deceive the people outside the room into thinking that he does by delivering accurate responses derived from the books and manuals around him.

In the above picture, the note going into the INPUT slit has the Chinese characters meaning “Where are you from?” written on it. The response, which comes out through the OUTPUT slit, has the Chinese characters “I am from China.” written on it. 

Searle’s thought experiment serves as a springboard to answering the questions: Can machines think? Can they understand the meaning of words? Evidently, he thought not, for like the person-in-the-room, computers operate the same way. They are programmed to recognize a sequence of commands, and to respond accordingly with what their encoded programmes and algorithms instruct. Surely they don’t use words in the same way that a normal human would!

But tackling those questions means that we would need to delve into what language is all about. What makes a language, a language? What do we mean when we make certain utterances? Or what is meaning in the first place? Gottlob Frege thought that meaning was a dual composition of sense and reference.² Hilary Putnam advocated against a psychological theory of meaning, that “meanings just ain’t in the head” and are mostly determined by one’s external environment.³

(from left to right: Gottlob Frege, Hilary Putnam, Ludwig Wittgenstein)

And then there’s Ludwig Wittgenstein, whose theory of language is encapsulated in three words: Meaning is Use. That is, we know the meanings of words insofar as we know how to use them.

“For a large class of cases – though not for all – in which we employ the word “meaning”, it can be defined thus: the meaning of a word is its use in the language.” ⁴

Much of his theory is concerned with how we come to know the meanings of words. We know them through being taught how to use them, through playing what he calls language games, and not by being recited dictionary definitions of a particular word. Take the word “five”, for example. How does one teach a child the meaning of “five”? Surely it is not to tell her that it is an abstract numerical term. Rather, we engage in language games, using things like wooden blocks and fingers, to illustrate what the word “five” represents – five fingers, five blocks, five apples. We teach it to her through demonstrating its use.

Is Wittgenstein correct? If it is true that Meaning is Use, then it seems that it is not the case that computers do not understand the meaning of words. They are engaging in the use of language by recognising how certain letters, symbols, or characters are used in response to others. We acknowledge that even though these machines demonstrate an understanding based on mere programming and computation, it is an understanding of meaning nonetheless. Such rule-governed activity is not very much different from how humans have come to know language and meaning too.

If it is true that computers can successfully master the human language, then what exactly do computers need in order to learn and understand the meaning in language?

 

*A thought experiment is carried out using the faculty of imagination – to imagine how things would be if they occurred in a certain way or manner. Although it is not a proper means of formal scientific investigation, engaging in thought experiments enables one to consider and deliberate about the repercussions that might result from certain hypothetical situations. As you have seen, the Chinese Room Argument is one such example. If you are interested, you may visit this page for other interesting examples of thought experiments.

How does a computer learn the human language? Could a computer really understand the meaning of words? What does this mean for us humans and our future? In this WikiChapter, we explore the idea of whether computers are capable of learning and understanding the human language.

To start, let’s take a look at the history of the development of artificial language processing capabilities. The timeline below tracks the history of artificial language processing, from the invention of the very first computer in 1822 to the present moment where computers are able to successfully use language. 

To explore the timeline, click on the right arrow to move forward. You may click the left arrow to return to any previous phase of computer development.

Computers are getting more sophisticated in learning the human language. But does this mean that computers are getting better at understanding the meaning of human language? The rest of the Wikichapter explores the answers to this question in the following sections:

 

1. Searle, J. (1980). Minds, brains, and programs. Behavioral And Brain Sciences, 3(03), 417. http://dx.doi.org/10.1017/s0140525x00005756
2. Frege, G. (1948). Sense and Reference. The Philosophical Review, 57(3), 209. http://dx.doi.org/10.2307/2181485
3. Putnam, H. (1973). Meaning and Reference. The Journal Of Philosophy, 70(19), 699. http://dx.doi.org/10.2307/2025079
4. Wittgenstein, L. (1992). Philosophical investigations (1st ed., p. S43). Oxford: Blackwell.
5. Saeed, J. I. (2016). Semantics. Hoboken, NJ: Wiley-Blackwell.
6. Theory of Mind. (2016, July 17). Retrieved April 18, 2017, from https://psychcentral.com/encyclopedia/theory-of-mind/
7. Arbesu, D. (2016, March 28). Computers may be able to translate language but will they convey its meaning. PBS.Org. Retrieved from http://www.pbs.org/wgbh/nova/next/tech/computers-may-be-able-to-translate-language-but-will-they-convey-its-meaning/
8. The Research Council of Norway. (2012, August 21). Computer program recognizes any language. ScienceDaily. Retrieved March 14, 2017 from www.sciencedaily.com/releases/2012/08/120821094125.htm
9. About. (n.d.). Retrieved April 18, 2017, from https://www.lua.org/about.html
10. Machine learning. Encyclopaedia Britannica. Retrieved on 28 Mar 2017 from https://global.britannica.com/technology/machine-learning
11. The Economist. (2017, January 5). Language:Finding a Voice. Technology Quarterly. Retrieved from http://www.economist.com/technology-quarterly/2017-05-01/language
12. West, J, (2016, April 2). Microsoft’s disastrous Tay experiment shows the hidden dangers of AI. QZ. Retrieved from https://qz.com/653084/microsofts-disastrous-tay-experiment-shows-the-hidden-dangers-of-ai/
13. Neural network. Encyclopaedia Britannica. Retrieved on 28 Mar 2017 from https://global.britannica.com/technology/neural-network
14. Libunao, J. (2015, November 13). Artificial brain learns to use human language. Futurism. Retrieved from https://futurism.com/artificial-brain-learns-to-use-human-language-3/
15. Genetic algorithm. Wikipedia. Retrieved on 28 Mar 2017 from https://en.m.wikipedia.org/wiki/Genetic_algorithm
16. Araujo, L. (2007). How evolutionary algorithms are applied to statistical natural language processing. Artificial Intelligence Review, 28(4), 275–303.
17. Pavlus, J. (2015). Skype Translator: Impressive, but Imperfect. MIT Technology Review. Retrieved 18 April 2017, from https://www.technologyreview.com/s/534101/something-lost-in-skype-translation/
18. Levy, S. (2016). An Exclusive Look at How AI and Machine Learning Work at Apple. Backchannel. Retrieved 18 April 2017, from https://backchannel.com/an-exclusive-look-at-how-ai-and-machine-learning-work-at-apple-8dbfb131932b
19. Knight, W. (2017). What to expect of artificial intelligence in 2017. MIT Technology Review. Retrieved 18 April 2017, from https://www.technologyreview.com/s/603216/5-big-predictions-for-artificial-intelligence-in-2017/

Timeline

https://cdn.knightlab.com/libs/timeline3/latest/embed/index.html?source=1qPyAavyy8TMLxaqy0AbD2XjOuON9TZ5w53VHxFsoQrY&font=Default&lang=en&initial_zoom=0&height=650