How Does Virtual Reality Work? Headsets, Tracking and Displays
Virtual reality works by showing each eye a slightly different image through lenses, while sensors track how your head moves and redraw the scene to match, many times per second. When the picture follows your motion closely enough, your brain accepts the scene as a space you are inside rather than a screen you are watching.
Three parts do the work: the display and lenses, the tracking system and the computer that renders the world. Each has trade-offs that explain why headsets differ.
Displays and lenses: creating depth
Your eyes sit a few centimeters apart, so each sees the world from a slightly different angle. The brain merges the two views and reads the difference as depth. A headset copies this by showing a separate image to each eye. The images come from one screen split in two, or from two small screens.
The screen is too close to focus on, so lenses sit between the eye and the display. They bend the light so the image appears farther away and fills a wide field of view. Because lenses distort the picture, the software pre-distorts the image in the opposite way so that it looks correct through the glass.
Two other specifications shape comfort and quality:
- Resolution. More pixels per eye reduce the “screen door” effect, where the gaps between pixels are visible.
- Refresh rate. Headsets commonly run between 72 and 120 frames per second. A higher rate makes motion smoother and helps reduce discomfort.
The distance between the centers of the pupils, called interpupillary distance, differs from person to person. Many headsets let you adjust the lens spacing to match.
Tracking: knowing where you are
For the world to feel stable, the headset has to know how your head is oriented and where it is in the room.
Degrees of freedom
Rotation only (looking up, down and around) is called 3 degrees of freedom, or 3DoF. Adding forward-back, left-right and up-down movement gives 6DoF. Basic viewers and many 360-degree videos use 3DoF, while room-scale headsets use 6DoF.
How movement is measured
- Inertial sensors. Gyroscopes and accelerometers inside the headset report rotation and acceleration at very high speed.
- Inside-out tracking. Cameras on the headset watch the room and work out the headset’s position from features such as edges and corners.
- Outside-in tracking. External base stations or cameras observe markers or sensors on the headset and controllers.
Controllers add hand tracking. Some systems also track the hands directly with cameras, with no controllers at all.
Rendering: building the world
A computer, a game console, a phone or the headset’s own chip generates the 3D scene. For each frame, it takes the latest position and orientation, renders the view for each eye and sends it to the display. That is twice the work of a normal screen, and it has to happen on time, every time.
Delay matters. The gap between moving your head and seeing the picture update is often called motion-to-photon latency. Engineers commonly aim for a figure around 20 milliseconds or lower, because longer delays make the world feel as if it is lagging behind you. Headsets use prediction and techniques that adjust the last frame just before display to hide small delays.
Why some people feel sick
VR sickness comes from a mismatch between what your eyes see and what your inner ear senses. If the image says you are moving but your body is still, some people feel dizzy or nauseous. Designers reduce the problem with smooth frame rates, comfortable motion such as teleport movement, and fixed reference points like a cockpit. Short sessions and breaks help new users. Anyone who feels unwell should stop and rest.
Types of headsets
| Type | Where the computing happens | Typical use |
|---|---|---|
| Standalone | Inside the headset | Games, apps and 360 content without cables |
| PC-connected | On a nearby computer | Detailed simulations and high-end games |
| Console-connected | On a game console | Games |
Where VR is used
Gaming and entertainment are the best-known uses, but the same technology supports training, design review and virtual property tours. For market figures, see the virtual reality statistics post, and for lighter reading the fun facts about VR.
Frequently asked questions
Is virtual reality the same as augmented reality?
No. Virtual reality replaces your view with a computer-generated world. Augmented reality places digital content on top of what you see in the real world.
Do I need a powerful computer for VR?
Not always. Standalone headsets have their own processors. PC-connected headsets rely on the computer’s graphics hardware, so requirements vary by model.
Can VR damage your eyes?
Evidence of lasting eye damage in healthy adults from normal use is limited, but eye strain and fatigue can occur. Take breaks, and ask an eye-care professional about specific concerns. Age guidance from headset makers should be followed for children.
The home page collects the site’s guides on 360-degree and VR tours.
For a general overview of the field, the Wikipedia article on virtual reality is a reasonable starting point.