Given the mostly positive effects that both visual and audio learning have on learning a nonnative language, there are now questions to how either form of learning may influence the other, if both are combined.

Based on numerous studies which have found adults’ nonnative language comprehension to improve (though not to native level) through auditory training, Hirata & Kelly (2010) investigated the effect of visual information on auditory ability.

In their study, 60 participants, all monolingual speakers of English, first sat for a prettest and were then given 4 different types of training in Japanese- audio-only, audio-mouth, audio-hands, and a combination of audio, mouth and hand gestures. The results of the posttest proved insightful- it seems that lipreading was more effective in facilitating phoneme learning rather than beat gestures.

While gestures are semantically significant in second language acquisition, they do not contribute effectively when functioning in a phonological aspect. That is, only congruent gestures would aid in one’s understanding of a nonnative word because they carry the meaning of the word, but beat gestures on the other hand carry no meaning in themselves but are used for rhythmic emphasis.

It is interesting to note as well that two channels (audio-mouth, audio-hands) of information during the training were more effective than when three channels were used. Though this may be surprising, it is likely due to an “overload” in participants’ working memory in which they are unable to encode the audio information properly because their attention is shared between the visual and audio learning cues in addition to the audio provided.

The findings from Hirata & Kelly’s study, coupled with previous studies on auditory and visual learning that have been conducted over the past few decades, have implications on possible methods that will be developed to aid in second language acquisition.