Dana Slabbekoorn

Eyes on the Mind: How VR and AI Could Revolutionize Dementia Screening

Reference: Xu, Y., Zhang, C., Pan, B. et al. (2024). A portable and efficient dementia screening tool using eye tracking machine learning and virtual reality. npj Digit. Med. 7, 219.

Someone in the world develops dementia every three seconds. Dementia gradually wears away memory, reasoning, and independence. Alzheimer’s disease, the most well-known form of dementia, can cause people to forget how to do everyday tasks, such as operating the microwave or brushing their teeth. With no cure and an aging global population, the number of people living with dementia is expected to triple by 2050. Yet, dementia doesn’t appear overnight. It usually starts with mild cognitive impairment, which is characterized by subtle memory or thinking problems that don’t yet disrupt daily life; but about half will progress to dementia within a few years. Catching this stage early can make a big difference, allowing for preventative measures and finding care. Unfortunately, early detection is difficult. Brain scans and spinal fluid tests are expensive and invasive. Paper-and-pencil assessments like the Montreal Cognitive Assessment (MoCA) are more practical but time-consuming, subjective, and dependent on a tester’s skill. To address these challenges, a team of scientists in Shenzhen, China, decided to try a new approach – building a software program to assess cognitive health through eye movements.

Eyes are a window into cognition

The eyes don’t just reveal what we see; they reveal what we think. Subtle patterns in our eye movements divulge information about how we pay attention, remember, and make decisions. Scientists have long known that mild cognitive impairment and dementia affect these patterns. For example, people with Alzheimer’s disease often scan scenes differently than those without; they look less at new objects, or struggle to fixate on the right targets during visual tasks. Eye-tracking technology captures and quantifies these movements which underlie cognitive processes. In recent years, researchers have used eye-tracking tasks to detect early memory problems or assess visual attention, sometimes even before traditional tests flag an issue. But despite its promise, eye-tracking research has remained largely confined to labs. The challenge has been making it fast, portable, and affordable enough for real-world screening.

This is where virtual reality (VR) and artificial intelligence (AI) come into the picture. The Shenzhen researchers combined these two technologies with eye tracking to create a single, five-minute experience they call VECA, the VR Eye-tracking Cognitive Assessment.

Inside the VECA experience

Imagine putting on a lightweight VR headset that transports you into a calm, virtual room. On a screen before you, pictures or numbers appear. Sometimes you’re asked to spot the correct shape, recall a pattern, follow a moving object, or solve a simple calculation. The entire experience feels more like an interactive game than a medical exam. While you complete these short tasks (each lasting just a few seconds) the headset’s built-in eye tracker captures where you look and for how long. Your gaze data, along with basic demographic information like age, gender, and education, are analyzed by an AI software trained to predict MoCA scores to determine who has mild cognitive impairment. Since mild cognitive impairment precedes dementia, the Shenzhen team hope that the VECA experience can be used as an early detection device.

The VECA setup in this study used a VR headset with an integrated eye tracker to test multiple aspects of thinking, memory, and attention in just five minutes.
Source: Xu et al., npj Digital Medicine (2024). Licensed under CC BY-NC-ND 4.0.

In order to develop the AI software, they recruited a sample of 201 adults aged 55–75 at community centers in Shenzhen to participate in this VECA assessment. Each person also took the standard MoCA test for comparison. The researchers then split the 201 participants into two groups: one group was used to train the AI, and the other was used to test it. They tested different models to train the AI software and found one that accurately predicted MoCA scores from eye-movement features (such as the proportion of fixation time on the correct answer). For every 10 people that MoCA identified as having mild cognitive impairment, the AI software identified 9 of those correctly. For every 10 people whose MoCA scores detected no cognitive impairment, the AI software correctly identified 8 of those, showing that the AI software’s performance is on par with most existing early screening tools. In other words, a five-minute VR-eye test matched the precision of a full paper-and-pencil exam—without the paper, the pencil, or the examiner.

What the eyes revealed

The researchers found that the features of the test most helpful in identifying people with cognitive impairment were education level and cognitive tasks involving calculation, memory, and executive function (cognitive skills that enable us to manage our thoughts or actions to achieve goal-directed behaviors). This is important because education can skew results on traditional tests. For example, people with fewer years of schooling often score lower even when they do not have cognitive impairment. VECA adjusted for this by dividing participants into three education groups and setting different cut-off points for each.

The findings echo a growing consensus: eye movement patterns, when interpreted with AI, can serve as rich digital biomarkers of brain function. In VECA, these patterns became reliable indicators of real-world cognitive function.

Beyond its accuracy, VECA offers practical advantages that could transform the early detection of dementia.

  • Portable: It runs on commercial VR headsets that are small and increasingly affordable.
  • Fast: The test takes only five minutes.
  • Objective: No examiner bias. Everything is recorded automatically.
  • Standardized: Every participant experiences the same controlled virtual environment.
  • Cost-effective: As VR hardware spreads, large-scale community screening becomes feasible.

Together, these qualities make VECA not just a research tool but a glimpse into what digital medicine could look like.

Challenges and next steps

The researchers are quick to point out that their study is just a first step.
All participants were from a single city in China, so broader testing is needed to confirm the results across different cultures and health backgrounds. They also didn’t measure how older adults felt about using VR—some may find it unfamiliar or tiring. And since the AI software learned from MoCA scores, any bias or error in that traditional test could influence VECA’s detection.

This technology also raises some tough ethical questions. Who owns your gaze data? How will data privacy be protected? Would such tools empower individuals, or burden them with anxiety over algorithmic detections? And how might doctors integrate AI-driven digital tools to improve, rather than replace, human care? Still, when simply considering proof of concept, VECA represents a major leap forward. It shows that meaningful insights about brain health can be gleaned from a few minutes of eye movement data.

The future of “digital eyes”

The implications of VECA go far beyond cognitive impairment screening. Eye movements are shaped by complex neural networks involved in broader cognitive operations, such as attention, memory, and even emotion. As such, similar tools could someday detect signs of Parkinson’s disease, depression, or traumatic brain injury. Moreover, as consumer VR devices like Meta Quest and Apple Vision Pro become more mainstream, the boundary between entertainment and health monitoring could blur. Imagine a future where your headset periodically runs a quick eye-tracking game, silently flagging early cognitive changes long before symptoms arise.

For now, VECA is a glimpse of what’s possible when technology is highlighted in neuroscience research. By merging eye-tracking, virtual reality, and machine learning, the Shenzhen team has shown that cognitive screening can be fast, portable, and engaging. If further studies confirm its reliability, early dementia testing might one day be as simple as putting on a headset and playing a few games.

By harnessing the power of eye movements, we may uncover some much deeper insights into our own minds.

Learn more about eye tracking and cognitive impairment here: