Three.js AR Depth Occlusion (Passthrough Depth Sensing)
Depth-based occlusion for passthrough AR in Three.js, where a virtual robot walks behind real furniture as every virtual fragment is tested against a real-world depth map, with soft edges, adjustable depth resolution and noise to show real-sensor artefacts.
Key topics: ar occlusion, depth sensing, webxr depth, passthrough ar, three.js shader, mixed reality
Game Over
What this code does
- Inputs: a passthrough device provides colour plus a depth map of the real world (here a synthetic room rendered twice, colour and linear depth in metres).
- Per-Fragment Test: onBeforeCompile injects a lookup of real depth at gl_FragCoord and compares it with the virtual fragment's view depth.
- Soft Edges: smoothstep(−s, s, realZ − virtualZ) fades instead of cutting, hiding sensor noise at boundaries.
- Sensor Realism: low resolution (160×120) and distance-squared noise reproduce the halos and shimmering edges of real depth sensors.
- Without Occlusion: virtual content always draws over reality, the main AR immersion breaker, which is easy to see by toggling.
- Controls: occlusion, depth resolution, soft edge, depth noise and depth-map view; Enter AR requests the 'depth-sensing' feature.
Related Demos
Continue Learning
JavaScript Implementation
// Depth-based occlusion: hide virtual fragments that are behind real surfaces
material.onBeforeCompile = (shader) => {
shader.uniforms.tRealDepth = { value: depthTexture } // metres, from the device's depth sensor
shader.fragmentShader = shader.fragmentShader.replace('#include <dithering_fragment>', `#include <dithering_fragment>
float realZ = texture2D(tRealDepth, gl_FragCoord.xy / resolution).r;
gl_FragColor.a *= smoothstep(-0.04, 0.04, realZ - vViewZ); // soft edge hides sensor noise`)
}
// WebXR: requestSession('immersive-ar', { requiredFeatures: ['depth-sensing'], depthSensing: { usagePreference: ['cpu-optimized'], dataFormatPreference: ['luminance-alpha'] } })