Sector · Languages

The word stays next to the thing it names

Learning a language on glasses is not translation. It is putting the word where the object is, and the laboratory result that matters was measured four days later, not on the day.

A flashcard asks you to remember a word with nothing around it. A pair of glasses can put the word on the object, in the room, at the moment you are looking at it, and the difference shows up in the only test that counts, the one taken later.

At UC Santa Barbara, 52 people learned vocabulary in a language they did not know, half through AR labels on real objects with a HoloLens, half with flashcards. The AR group scored 7 percent better on same-day productive recall and 21 percent better after four days. The gap grew with time, which is what you want from a memory result and what a same-day test would have hidden.

A Taiwanese study took students through a museum's dinosaur exhibition in English, one group on glasses and one on tablets. Learning effectiveness and motivation were both significantly higher with glasses, and among the glasses group the visual learning style benefited most. A Singapore study at MobileHCI 2025 found that presenting a sentence progressively, subject then verb then object, kept retention up while the learner was walking, and that pauses between words helped further.

The design finding that cost somebody a prototype

German researchers tested three places to put the translation: on top of the foreign word, to its right, or below it. Putting the translation over the word significantly reduced comprehension and raised the reported task load. It is the most useful single sentence in this file for anyone about to build one of these apps, and it is the kind of thing you only learn by measuring.

What is shipping, and what is only announced

Duolingo shipped a Google Glass version in 2014 with voice-answered flashcards and spaced repetition. Today the working code lives in the open: LinkLingo glosses rare words heard in live speech and runs its processing on the glasses themselves, and MemCards puts FSRS spaced repetition on the Even Realities G1. Rokid and NetEase Youdao put an English tutor called Hi Echo into an AR bundle, and the CEO gave a quote about changing study habits, but nobody outside the company has published an adoption or learning figure, so that is where the claim stops. The same goes for the Even Realities user studying Japanese: it is a customer story published by the manufacturer, and we label it as one.

Cited cases

Who
Rokid, in partnership with NetEase Youdao (Chinese education group), integrating the Hi Echo English app
What they did
Anyone wearing the Rokid AR Lite (the Max 2 + Station 2 bundle) can open Youdao's Hi Echo app and enter an augmented reality setting with a virtual English teacher: she starts the conversation, adjusts the difficulty to the user's answers, and explains the meaning and examples of an unfamiliar word when it is tapped on screen.
What came out of it
Zhu Mingming (Misa Zhu), Rokid's CEO, described the feature in an interview (18/11/2024) saying it simulates a scene and allows dialogue with a human, which will change many language-learning habits (free translation of a Chinese quote, checked against the original article); the partnership launched in April 2024 and the Rokid AR Lite bundle went on official sale in China in July 2024 (additional source: qbitai.com/2024/04/136995.html), but no independent source has published a Hi Echo adoption rate or a measured learning outcome.
news.qq.com
Who
Hyungmin Lee, Chen-Chun Hsia, Aleksandr Tsoy, Sungmin Choi, Hanchao Hou and Shiguang Ni, published at the 15th biennial conference of the Italian SIGCHI chapter (CHItaly 2023)
What they did
They built VisionARy, a system that runs on AR glasses and combines ChatGPT with object detection and speech recognition: the glasses recognise what is around the user and generate conversation topics for speaking practice in context, with correction during the dialogue.
What came out of it
In the within-subject study, the 25 participants rated VisionARy as well accepted and with potential to improve spoken learning; qualitative interviews showed different needs by level: beginners wanted immediate encouraging feedback, intermediates wanted a personalised topic and advanced learners wanted correction on demand for a specific use, such as a job interview or an exchange programme. The paper does not identify the commercial model of glasses used in the prototype.
dl.acm.org
Who
Researchers in Taiwan, study published in IEEE Transactions on Learning Technologies
What they did
They compared students exploring a museum's dinosaur exhibition in English: one group used smart glasses and the other used tablets, to measure the effect of each strategy on situated English learning and on motivation.
What came out of it
Learning effectiveness and motivation were significantly higher in the smart glasses group than in the tablet group; among the students with glasses, the visual learning style showed a more marked effect than the auditory or kinaesthetic ones. Published in 2023 (volume 16, issue 5, pages 820 to 830); the paper does not identify the commercial model of glasses used.
ieeexplore.ieee.org
Who
Adam Ibrahim, Brandon Huynh, Jonathan Downey, Tobias Hollerer, Dorothy Chun and John O'Donovan, University of California, Santa Barbara (UCSB)
What they did
They built ARbis Pictus, which labels real objects in augmented reality with their names in a foreign language unknown to the participant, using a Microsoft HoloLens headset, and compared vocabulary retention against the traditional flashcard method.
What came out of it
Participants who learned through AR scored 7 percent better on a same-day productive recall test and 21 percent better on a recall test after a 4 day delay, compared with the group that studied with flashcards. Published at IEEE ISMAR 2018.
sites.cs.ucsb.edu
Who
Mentra-Community, maintainer of the open MentraOS operating system
What they did
The LinkLingo mini-app runs locally on the glasses (with no dependence on a server session) and glosses rare words heard in live speech, shows real-time captions with the translation overlaid, and has a mode that suggests one new vocabulary word every five seconds.
What came out of it
Published as a working mini-app in the MentraOS ecosystem; the repository history shows a recent rewrite to run processing locally on the glasses themselves rather than depending on the cloud. The maintainer does not disclose install or user counts. The app manifest (miniapp.json) requires DISPLAY and MICROPHONE hardware, so it runs only on MentraOS glasses with a HUD; the repository does not name a specific commercial model.
github.com
Who
Tomas Tauber, independent development studio Tomtau
What they did
He published MemCards, a spaced repetition flashcard app (FSRS algorithm, the same one used by apps like Anki) that shows the cards on the Even Realities G1 display, so vocabulary can be studied without looking at a phone.
What came out of it
Repository published in October 2025 (author's blog post of 22/10/2025) and listed on the MentraOS store package page; the code's own README describes the app as "designed for smart glasses, such as Even Realities G1, in MentraOS". The developer does not disclose install or user counts, nor confirm whether the app already has an active public listing in the store (the general Mentra Miniapp Store shopfront showed "Coming Soon" on the date of this check).
github.com
Who
Rufat Rzayev, Sabrina Hartl, Vera Wittmann, Valentin Schwind and Niels Henze, human-computer interaction researchers
What they did
They tested three positions for an overlaid translation on AR glasses (on top of the foreign word, to its right, or below it) in a vocabulary learning task where the participant saw foreign words scattered around the surrounding environment.
What came out of it
Overlaying the translation on top of the foreign word significantly reduced comprehension and increased the task load reported by participants, compared with showing the translation beside or below the word. The paper does not identify the commercial model of glasses used in the experiment.
dl.gi.de
Who
Levi Michalski, software developer, in a testimonial published on manufacturer Even Realities' blog
What they did
He uses the Even Realities G1 to learn Japanese: he listens to 30 minutes of Japanese conversation lessons a day and, when he hits a difficult word, writes it down with the glasses' QuickNote feature to review the vocabulary later without needing his phone, including seeing the note whenever he checks the time on the glasses themselves.
What came out of it
According to the testimonial, published by the manufacturer itself, the feature became the best tool he did not know he needed for studying vocabulary; it is a customer story published by the maker, with no independent measurement of a learning outcome.
www.evenrealities.com
Who
Duolingo (founded by Luis von Ahn and Severin Hacker)
What they did
It shipped a version of its own app for Google Glass (Explorer Edition): by voice command the user opens a sequence of vocabulary flashcards in Spanish, French, German, Portuguese or Italian, answers by speaking the English translation, and the glasses use speech recognition to check the answer and apply spaced repetition, all without touching a screen or a phone.
What came out of it
According to Duolingo's own official release (25/06/2014), the app presents voice-activated flashcard sequences, tuned by a spaced repetition algorithm that prioritises each user's weakest words; the user answers by speaking the English translation and speech recognition checks it, rewarding a correct answer with points. It is one of the first language education apps for Google Glass; the press of the time (Engadget, TheNextWeb) reported the launch without publishing any usage or retention figure specific to this version.
duolingo-data.s3.amazonaws.com
Who
Nuwan Janaka, Shengdong Zhao, Ashwin Ram, Ruoxin Sun, Sherisse Tan Jing Wen, Danae Li and David Hsu, Synteraction Lab, National University of Singapore (NUS)
What they did
On lightweight consumer AR glasses they tested a way of presenting second-language vocabulary in progressive sentences (subject, verb and object appear in sequence, keeping what has already appeared on screen), designed for someone walking or doing another task at the same time, and compared it against the traditional method of showing the word on its own.
What came out of it
The progressive presentation improved vocabulary retention in the walking scenario, and inserting strategic pauses between words raised learning further during multitasking. Published in the MobileHCI 2025 adjunct proceedings; the paper does not identify the commercial model of glasses used in the study.
arxiv.org

Devices used

Talk to us

Back to the use cases