This project was presented at the ACM Symposium on Applied Perception 2026 (SAP '26) in Rennes, France.
In the real world we walk through crowds without colliding because we can read where other people intend to go. Navigating among autonomous virtual humans in Virtual Reality is much harder, because they rarely provide anticipatory cues that communicate turning intention.
We investigate which body-part kinematic cues carry that intention, combining a VR perceptual experiment with a Reinforcement Learning (RL) analysis built on the same task formulation. Participants predicted the avoidance direction of an approaching virtual pedestrian animated from full-body motion capture and gaze tracking of a real person, while their own gaze was recorded. In parallel, RL agents were trained on the identical task and their input features systematically ablated to quantify the contribution of each cue.
Participants attended predominantly to the head and upper body, while lower-body fixations were associated with prediction errors. The RL ablation identified hip position as the dominant cue, with head rotation as a significant secondary cue for early decisions. Together they reveal a two-phase cue structure: an early orientation phase driven by head rotation, followed by a displacement phase driven by lateral hip movement.
This suggests that avoidance maneuvers in virtual humans should be animated as a head–torso–hip rotation sequence preceding the trajectory deviation, rather than the sudden root rotation used by current models—making virtual humans behaviorally predictable as well as visually plausible.
In the real world we walk through crowds without colliding because we can read where other people intend to go. Navigating among autonomous virtual humans in Virtual Reality is much harder, because they rarely provide anticipatory cues that communicate turning intention.
We investigate which body-part kinematic cues carry that intention, combining a VR perceptual experiment with a Reinforcement Learning (RL) analysis built on the same task formulation. Participants predicted the avoidance direction of an approaching virtual pedestrian animated from full-body motion capture and gaze tracking of a real person, while their own gaze was recorded. In parallel, RL agents were trained on the identical task and their input features systematically ablated to quantify the contribution of each cue.
Participants attended predominantly to the head and upper body, while lower-body fixations were associated with prediction errors. The RL ablation identified hip position as the dominant cue, with head rotation as a significant secondary cue for early decisions. Together they reveal a two-phase cue structure: an early orientation phase driven by head rotation, followed by a displacement phase driven by lateral hip movement.
This suggests that avoidance maneuvers in virtual humans should be animated as a head–torso–hip rotation sequence preceding the trajectory deviation, rather than the sudden root rotation used by current models—making virtual humans behaviorally predictable as well as visually plausible.