Perceptual Models and Processes For XR Design

Week7

With major tech companies recently unveiling AR glasses powered by AI-driven operating systems, the principles of designing for AR and VR have become increasingly relevant. Since both technologies align closely with human perception, this chapter explores perceptual models and processes that are essential for designing extended reality (XR) experiences, encompassing both AR and VR. By briefly introducing key perceptual process models, I will discuss how these concepts can be applied in XR design.

Chapter 7: Perceptual Models and Processes

This chapter introduces several perceptual models and processes that help build a foundational understanding of human perception, which is essential for user testing in UX research.

7.1 Distal and Proximal Stimuli

The distinction between distal and proximal stimuli highlights the potential discrepancies between the objective world and an individual’s perception of it. As the chapter states, “a proximal stimulus is not always an ideal source of information about the distal stimulus.” Various factors — such as context, prior knowledge, expectations, and additional sensory input — can influence how distal stimuli are characterized. Understanding these concepts allows designers to analyze and fine-tune stimuli to create intended experiences in XR.

7.2 Sensation vs. Perception

When working with proximal stimuli, distinguishing between sensation and perception is crucial.

• Sensation refers to the elementary processes that allow low-level recognition of distal stimuli through proximal stimuli.

• Perception is the high-level process of interpreting sensations and assigning meaning to them.

Because perception is subjective, people who receive the same sensory information can interpret it differently. Even a single person’s perception of the same stimulus can vary depending on their current context. In XR design, where experiences are crafted with specific goals and target audiences in mind, understanding this concept is fundamental. By carefully selecting sensory inputs, designers can guide users toward intended sensations and perceptions.

7.2.1 Binding

Binding is the brain’s process of integrating multiple sensory inputs into a unified perception of objects or events. This concept is directly related to motion sickness, which should be avoided for optimal immersion. In XR design, ensuring spatial and temporal alignment of sensory inputs enhances immersion, supports intuitive interactions, and minimizes discomfort.

7.3 Bottom-Up and Top-Down Processing

Bottom-up and top-down processing are useful frameworks for determining an experience design strategy.

• Bottom-up processing is sensory-driven, beginning with environmental stimuli that are processed by the nervous system to form perceptions. This approach helps create a sense of presence, counteracting the top-down awareness that comes from knowing one is wearing a headset.

• Top-down processing relies on prior knowledge, experiences, and expectations to shape perception. XR designers can leverage this by integrating familiar real-world or virtual-world elements, enhancing engagement and immersion as content complexity increases.

By balancing these two approaches, XR creators can make experiences more intuitive and immersive.

7.4 Afference and Efference

• Afference refers to sensory signals traveling from the body to the brain (e.g., feeling a touch).

• Efference refers to motor signals traveling from the brain to the body (e.g., moving a hand).

The brain also generates an efference copy, a prediction of the expected sensory feedback from an action. If the actual sensory feedback matches this prediction (re-afference), the brain recognizes the movement as self-initiated. If there is a mismatch, the brain interprets it as an external force.

This concept is important for XR design because matching predicted and actual sensory feedback reinforces a sense of agency and immersion. Conversely, mismatches can lead to disorientation and motion sickness. In XR hardware, eye tracking and hand-tracking technologies must align with visual outputs to ensure that users’ sensory predictions match their actual experiences.

7.5 Iterative Perceptual Processing

Perception is a continual process that is dynamic and continually changing. Purple arrows represent stimuli, blue arrows represent sensation, and orange arrows represent perception.

This shows the simplified process of human perception. The map provides a clear sense of where the VR system is and how it will be processed in this iterative perceptual loop.

7.6 The Subconscious and Conscious

Distinguishing between subconscious and conscious processing is crucial for establishing an effective experience design strategy. This understanding serves as the foundation for core UX concepts such as affordance, signifiers, discoverability, and mental models.

In XR design, this concept plays a more significant role, as XR interacts with the entire surrounding environment and the user’s full-body movements — including eye and head tracking, gestures, and posture — unlike mobile or desktop applications, which involve fewer interactive elements.

7.7 Visceral, Behavioral, Reflective, and Emotional Processes

Visceral and much of behavioral processing occur subconsciously, shaping a user’s first impressions automatically. The visceral process, in particular, influences whether an experience feels good or bad at an instinctive level.

Beyond functionality, XR design should prioritize evoking positive emotions and strong user engagement from the start. This can be achieved through sensory elements such as aesthetics, sound, and tactile feedback, creating an immersive and emotionally compelling experience.

Behavioral processes involve learned skills and stored neural patterns, operating largely unconsciously, so users may not actively notice each action they take. When users feel a sense of control over their environment, they are more motivated to learn and adapt to new behaviors.

In XR design, providing immediate feedback — whether for success or failure — is crucial. Without clear responses to their actions, users may experience frustration or a loss of control, which can lead to disengagement and ultimately abandoning the experience.

Reflective and emotional processing both stem from conscious thought, but they serve distinct roles in cognition. Emotional processing is faster and more intuitive, assigning subjective value to judgments and enabling quick assessments of situations and goals. It plays a significant role in motivation and decision-making, and once a decision is made emotionally, it is difficult to overturn with rational thought — unless new, compelling insights emerge. In XR experiences, the emotional connection at the end of an interaction often shapes the user’s overall perception. If the final moments are positive, the memory of the experience is likely to be favorable — but only if the user remains engaged long enough to reach that point.

Reflective processing, on the other hand, is the deepest level of cognition, involving deliberate evaluation of past events, often linking thoughts and emotions through storytelling. Interestingly, reflective memories can outweigh actual events in shaping future predictions and plans. In XR environments, designers can leverage reflective processing for training and education by integrating interactive tutorials or guided challenges that help users understand cause-and-effect relationships.

Because not all users are naturally adept at navigating XR interfaces, reflection-based designs — such as step-by-step guidance that encourages users to pause and think — can make the experience more accessible and inclusive. Additionally, feedback systems, review features, narrative-driven elements, and goal-tracking mechanisms can foster deeper engagement, ensuring that users not only absorb information but also retain and apply it meaningfully.

7.8 Mental Models

Mental models are subjective, simplified representations of how the world — or specific aspects of it — function. Their primary role is to help people predict outcomes and take effective actions. These models are shaped through top-down processes, drawing from past experiences, training, and instruction. Because they are based on individual perspectives, mental models can vary from person to person, sometimes leading to different or even conflicting understandings of the same concept or system.

Models don’t need to be fully accurate, but they must be useful. They should strike a balance between simplicity and complexity — intuitive enough for users to understand quickly, yet detailed enough to prevent errors.

Mental models play a crucial role in guiding understanding, prediction, and interaction with the world. They are essential for problem-solving and adapting to unexpected situations. In VR environments, signifying cues, feedback, and constraints help users develop effective mental models, ensuring smoother interactions.

Conversely, poor interface design can lead to learned helplessness, where users feel a lack of control and ultimately disengage. A lack of feedback or poorly designed tasks can erode confidence, causing frustration and abandonment of the VR experience.

7.9 Neuro-Linguistic Programming

Although some aspects of neuro-linguistic programming (NLP) have been debated, the model remains useful for understanding how humans process stimuli. Certain high-level concepts from NLP can inform how users interpret stimuli in a VR application.

NLP, developed by Richard Bandler and John Grinder, connects neurological processes (“neuro”) to language (“linguistic”), suggesting that behaviors result from psychological “programs” shaped by experiences. They propose that expert skills can be acquired by modeling the neuro-linguistic patterns of those experts. When designing an AR or VR app for vocational training, this concept could be valuable for enhancing the learning process. However, creating accurate models of neuro-linguistic patterns would require extensive qualitative data collection, including in-depth interviews and observations. Additionally, integrating these patterns into the structure and UX elements of an XR application requires careful consideration.

In VR, NLP can help personalize stimuli and sensory elements based on user behaviors and preferences. The model suggests that individuals have dominant sensory modalities — people tend to process external stimuli more effectively when aligned with their preferred mode (visual, auditory, kinesthetic, etc.). This principle can enhance XR-based education by tailoring the experience to a user’s preferred sensory modality. In AR/XR applications, this personalization could extend beyond sight and sound to include smell, touch, and even taste.

To determine a user’s dominant sensory modality at the start of an XR experience, three potential methods could be explored:

1. Pre-session user survey — A questionnaire could assess learning preferences before beginning the training.

2. Test sessions — The system could analyze user behavior during interactive tasks and adjust the primary modality accordingly.

3. Personal history and data access on the device — With user permission, the app could analyze past data stored on the device to automatically tailor the learning environment to the user’s preferences.

The VR Book by Jason Jerald, Ph.D. (Page.81)

Filtering External Stimuli in XR

When processing external stimuli, the human mind applies three fundamental filters — deletion, distortion, and generalization — to manage information overload. These filters operate through six layers, increasing in awareness:

1. Meta programs (unconscious patterns)

2. Values (personal priorities)

3. Beliefs (deep-seated convictions)

4. Attitudes (emotional predispositions)

5. Memories (stored experiences)

6. Decisions (conscious choices)

Although XR designers cannot directly control how individuals filter information, these elements serve as a useful framework. Understanding how users form impressions, beliefs, and decisions about XR can help designers create experiences that leave a lasting positive impact, shaping perceptions of the technology in the long run.

Why XR Product Designers Should Understand Perceptual Models and Processes

Instant video replay captures events from multiple angles using high-resolution, high-frame-rate cameras. By slowing down, pausing, and zooming in on key moments, sports teams can analyze strengths and weaknesses at a micro level to optimize athletes’ performance.

Similarly, understanding perceptual models and processes is crucial in UX design for VR. While human perception is highly complex and no single model is absolute, these simplified frameworks still offer valuable insights into how users interpret and interact with digital environments. By leveraging these models, designers can craft more intuitive, engaging, and user-friendly experiences.