Virtual reality (VR) is reshaping how we perceive the digital and physical worlds. VR technology immerses users in computer-generated environments, allowing them to interact with artificial three-dimensional spaces. This is achieved through Head-Mounted Displays (HMDs), commonly known as VR headsets or VR glasses. These devices, widely recognized as fundamental components of VR, are worn on the user’s head and project images directly to their eyes.¹ As VR technology evolves and its applications expand, the market size has shown significant growth, with the global VR market valued at USD 59.96 billion in 2022, USD 227.34 billion by 2029, and a projected growth at an impressive compound annual growth rate of 27.5% from 2023 to 2030.² This growth is driven by VR’s transformative role in multiple sectors such as the automotive industry, retail industry, aviation industry, military, etc.

Authentication in VR is crucial for protecting sensitive user data and maintaining the integrity of virtual environments due to their immersive nature. The research projects on authentication in VR environments have unveiled numerous research gaps, one of which is the adoption of direct authentication such as personal identification number (PIN) codes, two-dimensional graphical passwords³ and drawing graphics that are more vulnerable to observation attacks and can easily be compromised by shoulder-surfing attacks.⁴ Another significant research gap in this field is the inconsideration of continuous authentication mechanisms with the users frequently interacting with the virtual world over long periods of time, it is essential to monitor the user’s identity continuously during the entirely of the VR experience; otherwise, it leaves the system vulnerable to unauthorized access.⁵ In addition to these two vital research gaps, integration of long-term dependency has gone unaddressed by any work proposed till now, such as VRCAuth,⁵ Doodle-Based Authentication,⁶ MoveAR.⁷ These models do not take into account the change in user behavior and the change in environmental factors.

Although several papers have made strides in addressing these challenges, the notable long-term dependency gap remains. To overcome this research gap, we have built a model using the long-short-term memory (LSTM) network that not only takes care of the continuous authentication part but also solves the long-term dependency issue. LSTM is designed to adapt to shifts in user behavior over time, contributing towards a more robust continuous authentication system and enhancing its reliability in virtual-reality environments.

To further address the issue of data and user privacy,⁸ this research uses deterministic tokenisation in sensor’s data. Deterministic tokenisation converts sensitive user keys, providing the same cipher text even after encrypting it multiple times and is known to minimize transparency and maintain data privacy. It complements the characteristic of LSTM learning from sequential data by making the data into structured sequential tokens.

The following are the contributions of this work.

For further elaboration on our approach, this paper is organized as follows: Section “Related Works” discusses the related work, existing solutions, and research gaps. In Section “Dataset Information”, we discuss the dataset we have used, followed by section “Proposed Work”, where we describe our proposed solution. Section “Result and Comparison” talks about the result and comparison, following Section “Conclusion and Future Research Directions”, which highlights the conclusion and the future research direction.

Traditional approaches like PINs and passwords are fast being either supplemented or replaced by those more advanced ones, like graphical passwords, gesture-based recognition, and biometric ways.^9,10,11,12 These try to overcome some of the inadequacies found in traditional approaches to authentication, as they provide alternative solutions that are more secure, user-friendly, and engaging. This work demonstrates a constant investigation into finding new ways to enhance security without sacrificing the ease with which users interact.

Sivasamy et al.⁵ have introduced VRCAuth, a continuous authentication system that verifies users according to their head movement patterns within VR environments. As the system depends on reliable patterns of movements, long time interaction in the VR may create a drop in accuracy that causes possible false acceptance or rejection, making VRCAuth less trustworthy for continuous authentication in long times of use for VR. A challenge to these long times depends on the reliability of user behavior.

Wazir et al.⁶ introduced a doodle-based authentication system using augmented reality for enhanced security and ease of use. In this approach, users draw doodles in a 3D space through touch-gesture recognition on smartphones, which are then matched for authentication. However, user fatigue and inconsistencies in gesture recognition may impact the system’s reliability over time. Variations in touch pressure and hand gestures can alter behavioral patterns, affecting authentication accuracy. If drawing patterns become irregular, the system’s accuracy may decline, making it unsuitable for long-term authentication. Bhalla et al.⁷ have explained how continuous user authentication using head movements and gestures may be developed for AR environments. This provides for increased security while enhancing user experience, with the significant disadvantage of long-term dependency on sensor data, resulting in performance degradation over time. Moreover, continuous authentication systems tend to have difficulties with accuracy maintenance in dynamic environments and are thus less reliable in the long run. These problems reflect the requirement for improvements in sensor stability and long-term consistency of authentication accuracy.

Zhu et al.¹³ have proposed the BlinKey system. BlinKey is a two-factor authentication scheme, using eye movement and blink rhythm for augmented security on VR devices. While robust against shoulder surfing attacks, such a long-term dependency of eye movement pattern in the system can decrease its accuracy, as there will be potential losses due to fatigue or other environmental factors; even users are more likely to not maintain uniform movements of eyes throughout the extended usage. As the users get fatigued, the effectiveness of the system might decrease, thus posing a potential security risk. Variations in eye conditions might also affect the accuracy over time, especially when authentication is required continuously.

Kim et al.¹⁴ proposed eye-writing pattern authentication using EOG for user authentication in VR, offering a low-cost, hands-free, and convenient solution. It achieved 97.74% accuracy, but long-term usage faced challenges like fatigue, distraction, and variations in eye movement patterns. Over time, user adaptation may impact authentication consistency and system performance.

Olade et al.¹⁵ introduced BioMove, a biometric system that identifies users in VR through head and hand movements. The system achieved high accuracy but depends on consistent movement patterns. Over time, user fatigue may alter movements, reducing authentication reliability. This makes continuous authentication less effective during extended VR sessions. The system’s reliance on physical movement poses a challenge for long-term accuracy. George et al.¹⁶ have investigated the possibility of adopting traditional forms of authentication in VR environments which include PIN and patterns. According to their studies, these types of authentication schemes work well short term in the aspects of usability and security especially due to a private visual channel where observers face difficulties for their attacks. However, the users will eventually lose their situational awareness as they become completely immersed in the VR environment, thus increasing the risk of observation attacks. This method has the drawbacks of decreased reliability in continuous authentication as immersion increases, and long-term dependency could lead to usability issues in extended VR interaction.

Hu et al.¹⁷ have presented a multi-modal identity authentication system based on gait and face recognition that aims to overcome the problems associated with long-distance identity authentication. It will use a combination of gait data, less sensitive to changes in distance, and face recognition for the purpose of strong and reliable authentication at different distances. However, the main drawback of this system is based on the principle of gait pattern and facial feature consistency that could change in time due to fatigue or even changes in posture. Environmental changes, such as lighting or even changes in the location of the user, can also degrade performance over time.

Miller et al.¹⁸ have focused on the application of Siamese neural networks in cross-system behavioral authentication in VR. The motion trajectories from various VR systems are utilized for the purpose of verification of identity. A learned distance metric is used in the system to account for differences in data between various VR systems, thus achieving better cross-system authentication. However, the problem in long-term dependency is user behavior change through time, in that users will possibly change the ways they move or sit while increasing the dependence error of the system in following sessions. It is also more vulnerable to continuous authentication with less reliability, where it does not learn gradual changes, and thereby becomes inaccurate in the long run for the whole process.

Lohr et al. paper¹⁹ is a good baseline for how well the authentication using gaze has performed on the very large database of 9202 participants operating under VR scenarios. Both the monocular and the binocular gaze datasets were compared concerning the efficiency with which the optical and the visual axes are put to work toward better authentications in terms of the accuracy levels achieved during authentications. Improved results were demonstrated when the two axes were taken together. However, this method might face a challenge for the long run since the patterns of eye movements keep changing. Continuous authentication might not be reliable since user fatigue or environmental changes can easily affect a user’s gaze behavior thereby reducing the effectiveness of long-term use. A larger number of users may impact the performance of the system witnessing a challenge in scalability during the long-term continuous authentication process.

Jonathan Lieber and Stefan Schneegass²⁰ proposed a gaze-based continuous authentication framework for VR applications, utilizing behavioral gaze biometrics for enhanced security. This method offers high resistance to attacks as eye movements are hidden within HMDs and difficult to mimic. It is also cost-effective, leveraging built-in eye-tracking in modern VR headsets like HTC Vive Pro Eye and Pico Neo 2 Eye. However, the approach lacks experimental validation and does not address long-term dependency issues. Future work could focus on real-world testing and improving long-term authentication reliability.

Tianfang Zhang et al.²¹ proposed SAFARI, a spoof-resistant, text-independent speech authentication system for AR/VR, addressing vulnerabilities to voice spoofing and phonetic variations. It derives facial biometrics from viseme-associated vibrations captured by AR/VR motion sensors, making it resistant to audio-based spoofing. Key advantages include cost-effectiveness (utilizing existing motion sensors) and robustness against body motion using a diffusion model. However, SAFARI relies on session-based authentication rather than continuous authentication and does not account for long-term data dependencies. Future research could enhance continuous authentication and adaptive learning for improved security in AR/VR environments.

Arman Bhalla et al.²² and Tianfang Zhang et al.²¹ introduced SAFARI, a spoof-resistant speech authentication system for AR/VR. It extracts facial biometrics from viseme-associated vibrations using motion sensors, making it resistant to phonetic variations and voice spoofing attacks. SAFARI is cost-effective as it utilizes pre-existing motion sensors and compensates for body motion using a diffusion model. However, it relies on session-based authentication rather than continuous authentication and does not account for long-term data dependencies. Future improvements could focus on continuous authentication and adaptive learning for enhanced security in AR/VR environments.

Table 1 is a summary of the related works section discussing the dataset, setups, accuracy and limitations of each work.

We have used two different datasets in this project. The VR Driving Simulator dataset²³ contains information regarding driver behavior in controlled environment. HTC Vive was the hardware used. Data from 25 users were collected with greater flexibility. It contains data from project cars, steering wheel, pedals and VR headset. The required Head Movement Data is taken from this dataset. The second 360° Video Viewing Dataset in Head-Mounted VR²⁴ encompasses content data like image saliency maps and motion maps from 360° videos, along with sensor data such as head positions and orientations recorded by HMD sensors. Oculus Ri DK2 was the HMD used and the data was collected from 50 viewers between ages of 20 and 48. The required sensor data was taken into consideration. We have used two different datasets in this project.

Head movement is a key indicator of a user’s unique motion pattern, visual focus, and preferences in VR applications.⁵ The proposal was based on continuous authentication in the security system by using a LSTM network to deal with both long-and short-term d-ependencies in user behavior. This method aims at elevating the ability of the system to monitor and recognize patterns over time, hence more accurate and reliable authentication. In addition to that, the study faces the problem of mimicry attacks in which unauthorized users try to replicate the behavior of real people.⁶ Hence, based on the property that LSTM does to remember what occurred before and adapt changes in user behavior. This approach of analyzing temporal dependencies along with the detection of attacks essentially creates a stable and adaptable authentication system. LSTM can remember the long-term dependencies, which helps to increase the accuracy in user classification.

Compared to traditional machine learning models, LSTM does not rely on human-domain knowledge and manual feature engineering. Unlike machine learning models, LSTM automatically learns and extracts complex features from sequential data, eliminating the need for manual interference. Moreover, machine learning models face problems in effectively capturing temporal dependencies in sequential data, which adds the requirement of additional mechanisms for processing time-series information, whereas LSTM can capture both long-term and short-term dependencies in sequential data which makes it much more appropriate for analyzing time-series information. In addition, LSTM can model complex data structures and capture intricate patterns that enable identification of variations in user behavior for more accurate authentication.

Moreover to ensure privacy, the system secures gyroscope and accelerometer sensor data by encrypting it prior to transmission, minimizing the risk of sensitive information being inferred or exposed during server communication. It utilizes deterministic tokenisation for the head motion data, guaranteeing a consistent conversion of sensor data into secure tokens for transmission. This method not only safeguards sensitive details but also preserves high classification accuracy by leveraging LSTM networks.

We are training a deep learning model in order to classify the data into authorized and unauthorized users using a LSTM Network. The processed authorized and unauthorized datasets are being combined and features are being separated from the target label, indicating whether the user is authorized (1) or unauthorized (0).

The features are being normalized using MinMaxScaler to scale values between 0 and 1 for uniformity, and the target labels use one-hot-encoding to represent the two classes in categorical format.

The data is reshaped into a 3D structure (samples, time steps, features) required by LSTM networks, treating each row as a single time step with all columns as features. After splitting the data into training and testing sets, an LSTM-based neural network is constructed.

The model has an LSTM layer with 16 units, followed by a dropout layer²⁷ to reduce overfitting and a dense layer with sigmoid activation for binary classification. A single-layer LSTM with 16 units was employed, achieving improved accuracy while maintaining computational efficiency. This lightweight configuration enables faster inference without compromising performance, as increasing the number of units did not yield significant additional gains.

It is being compiled using the Adam optimizer²⁷ with a low learning rate, categorical cross-entropy as the loss function, and accuracy, precision, recall, and F1-score as the evaluation metrics. The Adam optimizer is highly effective for training LSTM networks because, with its adaptive learning rates, it handles the variation in gradients across timesteps effectively. Combining the properties of both momentum and adaptive scaling accelerates the convergence by making oscillations smoother; therefore, the efficiency of the training increases. Its bias-correction terms add stability in early training phases, which is crucial for complex models.

In this work, every row of the dataset corresponds to one interaction instance and not a multi-step temporal sequence. Therefore, the temporal dimension for LSTM input is defined with a sequence length of one. Those data are reshaped into format (samples, 1, features) where the single time step corresponds to the features recorded at this instant. No sliding window or step-size mechanism has been applied because there are no consecutive time-linked samples in the data. Consequently, LSTM learns no long-range temporal dependencies. In fact, it works as a gate nonlinear transformation of the feature vector. Using LSTM in such a setup enables the model to take advantage of the gating mechanisms-i.e., input, forget, and output gates-to model complex features’ interactions in spite of explicit temporal sequences being absent.

The data from the VR headset is transferred to a server through a secure communication channel. In this server, the data are used to train and test the LSTM model for continuous authentication. The trained model can be transferred to resource-constrained devices like VR headsets, which has been left for future work.

Figure 1 illustrates the proposed user authorization system using motion sensor data from a HMD. The process begins with collecting motion data, which is pre-processed to prepare it for analysis. A single-layer LSTM model, trained on this data, performs binary classification to determine if the user is authorized or unauthorized. Authorized users proceed to gain access, while unauthorized users are denied.

This work presents a novel continuous authentication system for VR environments that uses head movement as a biometric feature, effectively addressing the limitations of traditional PIN and password-based mechanisms. Using LSTM networks, the proposed solution achieves an impressive accuracy of 99% in distinguishing between authorized and unauthorized users in the two raw datasets without encryption and around 97% using encrypted motion sensor data for user authentication, ensuring robust and reliable user authentication. To mitigate privacy concerns associated with outsourcing motion sensor data for training purposes, the system incorporates data encryption, safeguarding sensitive user information such as PINs and passwords.

In order to cover the limitations of deterministic tokenization with AES the future research direction includes:

Future versions of the system will be designed and bench-marked with holistic security protections to mitigate the threats of replay, injection, and linkage in a practical setting. In particular, we intend to adopt challenge–response authentication schemes, nonce or per-record IV creation, verification of sequence-number or timestamp, and rate limiting on servers for replay and injection resilience. Furthermore, on the privacy-preserving side, we intend to explore salted hashing, randomized encryption, and linkage-resilient identifiers for preventing cross-session correlation. As soon as the hardware or communication module is present, these protections will be integrated into the end-to-end pipeline and evaluated using adversarial testing and security benchmarks. Only then will a fully secure prototype that is resistant to practical attacks be developed.

The seamless integration with existing VR hardware without requiring OS-level modifications underscores the practicality and scalability of the solution. Future research can focus on exploring lightweight encryption methods to further improve performance and investigating additional biometric features to strengthen authentication mechanisms incomplex VR scenarios.