Translation Technology and Hallucinating Computers

AI is taking on an increasingly important role in modern society and so, it is critical that algorithms can understand and utilize many different languages. In lieu of this, researchers continue exploring innovative methods of improving AI's capabilities.

A combined team of researchers from MIT, IBM, and the University of California San Diego have developed a unique AI translation algorithm that replicates the capabilities of the human imagination to help visualize words and phrases in different languages. This new model is one of many vital steps toward making AI and its benefits accessible to everyone, without language barriers. 

The research team at MIT, IBM, and UC San Diego call their AI model “VALHALLA”. The algorithm takes an innovative approach to learning and translating languages, inspired by the way humans learn and process new languages. 

Rameswar Panda, one of the researchers on the VALHALLA team, explained in an interview, “In real-world scenarios, you might not have an image with respect to the source sentence. So, our motivation was basically: Instead of using an external image during inference as input, can we use visual hallucination — the ability to imagine visual scenes — to improve machine translation systems?”

When humans learn new languages, we often use pictures to make the connection between new words and known words, such as flashcards with images on them. Humans are capable of imagining an object, place, or other items without a real photo, though. The idea with VALHALLA is to train a computer to do the same thing, to essentially “hallucinate” or imagine objects in order to improve language translation. 

The model was trained by using dual sets of data. For each sentence VALHALLA translated, it was given the source sentence along with a “ground truth” image that visually represented what was in the sentence. The image that the AI generated or “hallucinated” for the source sentence was compared to the corresponding “ground truth” image to test the accuracy of each translation. Once the model was trained, the ground truth images were removed so the model completely relied on its hallucination mechanism. 

In testing, the VALHALLA research team found that their model was able to perform better than many text-only translation algorithms. While VALHALLA still has room for improvement, it does indicate that multimodal AI models may be the future of translation algorithms. 

When many people think of AI translation, programs like Google Translate are often the first thing that comes to mind. They may wonder why more extensive research into AI translation is even necessary with programs like this so easily accessible. 

However, many of today’s most widely available language translation algorithms leave much to be desired. In fact, Google Translate’s inaccurate and unusual translations have inspired parody covers of popular songs all over YouTube.

So, there is certainly room for improvement, as optimized AI capabilities can greatly improve service delivery in the following areas:

On a more technical level, AI is becoming more widely used in things like customer service. For these models to provide the best service possible, they should be able to interact with people who don’t speak English. 

Multilingual AI models could be incredibly valuable for businesses in many different ways. For example, more and more businesses are launching multilingual websites, with the express goal of helping them communicate better with customers from all language backgrounds. Improved translation algorithms could help make website translations and AI  customer service much more accessible and affordable for businesses. 

Multilingual AI models could also help people on the job. For example, improved translation algorithms could be used in customer-facing roles to help employees interact with customers who do not speak the same language. Similarly, businesses that employ people who do not speak English could use AI translation for training and on-the-job communication. Even in the executive suite, effective translation algorithms could help business leaders to communicate with international partners. 

There are also ethical considerations that are central to the need for improved AI language translation. AI is opening the door to all sorts of benefits in everything from education to healthcare. It is important to make sure these benefits aren’t locked behind language barriers. 

Numerous projects are working to make AI more accessible. For example, the AI program Sanas received a major grant in June 2022 to continue the development of its breakthrough language algorithm. Sanas bridges the gap between languages and accents by subtly changing a user’s voice to change their accent. It is designed to help people with certain accents communicate without facing discrimination because of their accents. 

Similarly, a group of 1,000 academic researchers united to create an open-source multilingual AI model known as BLOOM, which is designed to compete with some of the biggest AI models around today, such as GPT-3. The BLOOM development team notably included people with expertise in fields like ethics, law, and philosophy, in addition to computer scientists. The goal of the project was to create a large-scale AI model that was accurately multilingual. BLOOM is designed to be free of underlying racial or gender biases and capable of giving authentic translations for all types of input. 

Projects like this ensure that the world’s most powerful AI models are equally accessible and effective for people who speak any language. As AI technology becomes more powerful, the gap between those who can use it and those who can’t will only grow. Language is the most pressing barrier to inclusion. MIT’s advancements in AI language translation with VALHALLA are a step toward ensuring that computer translations are both fast and accurate, so no one gets left behind. 

Research has found that only 20% of Americans are bilingual but some of those who are, particularly those who speak Spanish as a first language, face discrimination because of their language skills or accent. AI translation algorithms could help improve communication, empathy, and understanding between people from all language backgrounds. 

MIT’s VALHALLA model is just one of a growing number of efforts to advance the capabilities of AI language processing and translation. Improving AI’s understanding of different languages is imperative to ensure that the countless benefits that come with AI usage are available to everyone.