What is Eye Tracking? Definition and Applications
There are various ways to collect user data and assess engagement, health, or attention within our industry. These include surveys and on-site experiments. In the technology sector, a highly effective method is gaze estimation, or eye tracking.
In this article, we’ll discuss eye tracking, or gaze estimation, technology, its potential to revolutionize different industries, and what real-world applications we can already find. To best understand how eye tracking is used and what technology is involved in it, we need to start with the basics.
1. What is Eye Tracking Technology?
Eye tracking, also known as gaze estimation, is a branch of computer vision and machine learning technologies that use AI to track a user’s eye movement (gaze point) across the screen. It’s a process of determining where a person looks at a given moment. Eye tracking can tell us a lot about human behaviour and provide crucial information on where their attention goes and sometimes it can help us determine a person’s health.
The core of gaze estimation technology involves tracking the position and movement of the eyes. You can do it using specialized hardware, such as infrared eye trackers or cameras. Sometimes, it might require extra hardware such as eye-tracking glasses. Modern gaze-tracking technologies, however, aim to perform the task without additional equipment.
2. How Does Eye Tracking Work?
Gaze estimation works in different ways depending on the technology used.
Dedicated hardware:
- Devices placed under the monitor track eye movements.
- Advanced systems use multiple LEDs and cameras to determine the exact point the user is looking at on the screen.
Wearable devices:
- Head-mounted sets have built-in cameras directed at the eyes.
- These cameras track pupil movement to accurately determine the gaze direction.
- Traditional eye tracking methods might rely on geometric models (e.g., PCCR—Pupil Center Corneal Reflection).
Gaze estimation using a built-in camera and deep learning (“in the wild”):
- Uses a standard camera in a laptop, tablet, or phone.
- Software analyzes the camera image to estimate what the user is looking at.
- An example is the solution from DAC.digital and eye square, which transmits the camera feed to the cloud.
- In the cloud, a neural networks processes the video and determines the gaze point.
- Convolutional Neural Networks (CNNs) are particularly effective for gaze estimation because they can recognize and extract complex patterns from eye images, such as the shape of the pupil, iris, and reflections.e complex patterns found in eye images.
- Such technology is especially useful in the so-called “in the wild” conditions, where lighting, head movement, and camera angles are unpredictable
Many gaze estimation systems require a calibration process in which the user looks at known points on the screen. It allows the system to map eye position to screen coordinates for higher accuracy and more reliable measurements.
5 Steps of an Eye-tracking Process
Cameras capture images or video of the eyes (or even whole faces when the environment needs to be more natural). These cameras can be specialised infrared cameras or standard webcams. If used, infrared light illuminates the eye for more explicit images, while standard cameras record eye movements. This eye-tracking data makes the basis for the next steps in the process.
Algorithms find important eye features, such as the pupil and corneal reflections from infrared light on the eye surface. Eye contours are also identified.
In this step, geometric models (e.g., PCCR – Pupil Center Corneal Reflection) or deep learning (e.g., CNNs) determine the user’s gaze direction.
The user looks at known points on a screen to help the system map eye positions to screen coordinates.
The system updates the direction multiple times per second, showing where the user looks on a screen or in a 3D space.
3. What is Eye Tracking Used for?
Now that you understand gaze tracking let’s examine the trends and opportunities in different industries.
Example #1: Eye Tracking in Marketing and Market Research
Eye-tracking methods give insights into human behaviour and are highly helpful in marketing and market research. Let’s examine the most valuable applications of gaze estimation in these sectors.
Ad Effectiveness Measurement
Eye tracking can help you generate heat maps of user engagement. These visual representations show users’ eyes focusing most frequently on an advertisement. They help identify which elements (e.g., headlines, images) attract the most attention. It also measures how long users look at specific aspects of an ad, indicating their level of interest and engagement.
Website Usability Check
Gaze estimation allows us to track how users navigate through a website, identifying which sections or links attract attention and which don’t. These insights are also helpful when planning content placement. Gaze estimation helps optimise text placement, images, buttons, and ads to improve user experience and increase conversions.
Packaging Design Tests for Best User Experience
An eye-tracking system can be an excellent solution for design choices. For example, you can test packaging ideas with your users with their eye movements directing you across the elements that catch their attention and those that are ignored or only glanced over for a fraction of a second. It also helps you determine the best placement for important information, such as a brand or product name.
Example #2: Eye Tracking in Healthcare and MedTech
Gaze estimation applications in medicine can make a powerful tool for improved diagnostics, rehabilitation, and assistance for patients and doctors. Let’s go through some of them.
Assistive Technology
The eye-controlled environment enables those with severe physical disabilities (e.g., ALS, spinal cord injuries) to communicate and interact with computers using their eyes.
Gaze-based interfaces improve accessibility for patients with mobility impairments, allowing them to independently control various devices and systems.
Robotic Surgery Assistance
When integrated into robotic surgery systems, eye-tracking technology offers significant enhancements in precision and control, allowing surgeons to perform operations more effectively. Eye-tracking data helps to assess surgical skills and identify areas for improvement. This data is valuable for training programs, helping to ensure that surgeons can perform at their best.
Diagnostics Support
Gaze tracking helps identify characteristic eye movement abnormalities in Parkinson’s disease patients. Abnormal eye movements, such as reduced saccade velocity (the speed of rapid eye movements) and difficulties with smooth pursuit (tracking moving objects), indicate Parkinson’s and can be monitored using gaze estimation technology.
Eye-tracking technology is increasingly used to diagnose and monitor multiple sclerosis (MS). This technology helps detect subtle changes in eye movements indicative of the disease’s progression and severity, which might not be detectable through traditional imaging techniques.
Rehabilitation
Gaze-based exercises and games can aid stroke patients’ cognitive and motor recovery, encouraging them to interact and engage more actively in rehabilitation. Tracking eye movements also helps assess and rehabilitate patients with a Traumatic Brain Injury (TBI) by providing insights into their visual and cognitive impairments.
Example #3: Eye Tracking in the Automotive Industry
Eye-tracking software can effectively improve driver safety on the road and overall driving experience. Here are some practical applications of this technology in the automotive field.
Driver Monitoring Systems (DMS)
Eye tracking can monitor eyelid movements, blink rates, and eye closure duration to detect signs of drowsiness and alert the driver to take a break. Additionally, by tracking where the driver is looking, the system can identify if the driver is distracted (e.g., looking at a mobile device or not focusing on the road) and issue warnings, increasing awareness and safety.
Augmented Reality (AR) Displays
Gaze tracking offers a range of uses with augmented reality. It can enhance interaction in heads-up displays (HUDs). For example, it ensures that critical information is displayed where the driver looks, improving readability and reducing distraction. The system can overlay navigation instructions directly onto the road in the driver’s line of sight, making it easier to follow directions without taking eyes off the road.
Vehicle Personalisation and Comfort
Eye tracker technology may also help increase comfort and personalise the driving experience. For example, it can adjust the seat automatically and mirror the driver’s optimal eye position. Drivers can control infotainment systems using their gaze, reducing the need to use physical buttons or touchscreens and thus minimising distraction.
Example #4: Eye Tracking in Manufacturing and Warehousing
Gaze estimation is becoming increasingly valuable for manufacturing and warehousing. It provides numerous productivity, safety, and training benefits. Here are a few gaze-tracking applications in this industry.
Skill Development and Assessment
Thanks to its ability to measure eye movements and visual attention, gaze tracking is a perfect tool for developing training programs that provide real-time feedback to workers. Providing additional instructions is more accessible when the trainers can determine gaze direction and map focus areas. Similarly, measuring eye positions during tasks can show whether workers follow the procedures correctly and assess their skills.
Process Optimisation
Eye-tracking data provides insights into how workers interact with their environment, helping identify process inefficiencies. For example, it can reveal whether workers spend too much time searching for tools or navigating complex interfaces. By studying eye movement patterns, companies can redesign workstations to reduce unnecessary movements and improve ergonomics, leading to faster and more comfortable task completion.
Augmented Reality (AR) Assistance
In manufacturing and warehousing, you can integrate gaze-tracking systems with AR systems to provide real-time information overlays. For instance, workers can receive step-by-step assembly instructions or guidance to the correct storage location for items, all based on where they are looking.
Example #5: Eye Tracking in Research and Development (R&D)
Eye-tracking research methods provide an array of valuable ideas on how to improve lives with gaze estimation. Here are some examples of popular topics for eye-tracking research.
Human-Computer Interaction (HCI) Studies
Researchers use gaze estimation to evaluate how users interact with software interfaces, websites, and applications. By tracking where users look and how long they spend on different elements, designers can optimise layouts for a better user experience. Gaze tracking data helps develop more intuitive and user-friendly interfaces by identifying which design elements capture attention and which don’t.
Cognitive and Behavioural Research
Gaze estimation allows you to study how people perceive and attend to different stimuli. It can be critical in understanding cognitive load, visual search patterns, and visual information processing. By tracking eye movements, you can infer how individuals learn and remember information, which is valuable in educational research and the development of learning tools.
Psychological and Medical Research
Gaze estimation aids in studying the visual and cognitive deficits associated with neurological disorders such as autism, ADHD, and schizophrenia. It helps in developing diagnostic tools and therapeutic interventions. You can also apply it in everyday research to measure mental workload in various tasks. It is crucial in designing better ergonomic solutions and safety systems, particularly in high-stress environments like air traffic control and surgery.
4. Common Challenges with Custom Eye Tracking Development
As with every advanced technology, gaze estimation isn’t free of challenges. Here are the three most prominent ones and ideas for overcoming them when you pursue eye-tracking development projects.
Achieving Desired Accuracy and Robustness
One critical struggle in developing an eye-tracking device or adopting it as part of existing technologies is achieving a satisfying level of accuracy. Diverse lighting conditions and head movements caused by facial occlusions like hair or glasses often make the results inconsistent and cause prediction errors.
Here are a few ways to solve this issue:
- Personalised calibration: Implement personalised calibration procedures that adapt the gaze estimation model to individual users. It can involve short calibration sessions where users look at specific points on a screen, allowing the system to fine-tune its predictions based on the unique characteristics of each user’s eyes.
- Advanced machine learning models: Deep learning techniques, such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), enable the development of more sophisticated models that can learn to account for variations in lighting, head poses, and occlusions. Training these models on large and diverse datasets can improve their generalisation ability across different conditions.
- Multimodal data integration: Combining data from different sensors, for example, visual and thermal, enhances gaze estimation’s robustness. This approach can help compensate for any single sensor’s limitations by leveraging complementary information.
Real-Time Processing
For many applications, gaze estimation needs to be performed in real-time, which requires processing high-resolution images or video frames at high speeds. It demands significant computational resources and efficient algorithms, which can be challenging to implement, especially on mobile or embedded devices.
Here’s how you can overcome challenges associated with real-time processing demand:
- Edge computing: Employ edge computing strategies where data processing is done on local devices (e.g., smartphones, AR/VR headsets) rather than relying on cloud-based servers. It helps reduce latency and ensures eye tracking operates in real-time, even in environments with limited internet connectivity.
- Developing optimized algorithms: Techniques such as model pruning, quantisation, and lightweight neural network architectures (e.g., MobileNet, SqueezeNet) help build algorithms that can reduce computational load.
- Hardware acceleration: Hardware accelerators such as GPUs, TPUs, and FPGAs speed up the processing of gaze estimation algorithms. These specialised hardware components can perform parallel computations more efficiently than traditional CPUs.
Gathering Quality Training Data
Machine learning models rely heavily on data, as it’s the core of its training and functionality. That’s why collecting quality datasets is critical to achieving the optimal outcome. However, gathering reliable samples is a challenge of its own, and here are some ways to overcome it:
- Use a diverse dataset: Use diverse recruitment strategies or synthetic data generation and augmentation techniques to create diverse training data. It can include partnerships with organisations with access to diverse populations, collecting data in various locations, or outsourcing your data.
- Standardised collection protocols: Develop and adhere to standardised data collection protocols to minimise variability, including consistent camera setups, calibration procedures, and environmental controls.
- Calibration techniques: Apply advanced calibration techniques such as multipoint or dynamic calibration to warrant high accuracy and precision. Calibration methods that adapt to individual differences can help improve data quality.
5. A Real-World Example of Eye Tracking in the Wild
Above, we listed several trends in eye tracking for different industries. We would like to give you a practical example of using gaze-tracking technology. The solution we built in partnership with eye square involves adopting eye tracking for smartphones for market research purposes. Specifically, it tracks users’ eyes while they scroll through their everyday tasks to discover more about what keeps them engaged. It can be helpful for:
- Researching ad usefulness
- Testing whether interface is intuitive to use
- Learning more about user behaviour and which elements of an application or website attract their attention
Learn more about the solutions and challenges of applying eye-tracking technology in the wild.
6. Join Forces with Our Experts and Create a Custom Gaze Estimation AI
Now that you know what eye tracking is and how it works, it’s time to take action. Whether you need eye-tracking software or a complete solution built from scratch, you can consult with our experts to do it for you. Don’t hesitate to reach out.
Contact us!
Send us an email: [email protected]