Three.js AR Marker Tracking (Fiducial Detection & Pose)
Marker-based AR from scratch in Three.js, where a synthetic phone-camera feed is thresholded, the marker quad is found from the dark blob's convex hull, a 5×5 fiducial ID is decoded through a homography, and the 6-DoF pose recovered from K⁻¹H anchors 3D content, checked against ground truth.
Key topics: ar marker tracking, fiducial marker, homography, pose estimation, augmented reality javascript, three.js ar
Game Over
What this code does
- Camera Feed: the "real" desk is rendered by a hand-held phone camera into a 480×360 target and read back to the CPU, a stand-in for getUserMedia frames.
- Quad Detection: global threshold, largest dark connected component, convex hull, then the farthest pair plus the farthest point on each side of it as the four corners.
- Homography: a 4-point DLT (8×8 linear solve with h₃₃ = 1) maps the unit square onto the image quad.
- Decoding: a 7×7 grid sampled through H must have an all-black border, and the 5×5 interior is matched against a small dictionary in all four rotations, which also fixes orientation.
- Pose: [r₁ r₂ t] = λ K⁻¹ H S, then r₃ = r₁ × r₂, re-orthonormalised and converted to three.js axes; content renders with a camera matching the phone's intrinsics.
- Controls: printed marker, AR content, marker motion, hand shake and debug overlays (outline, corners, binary inset).
Related Demos
Continue Learning
JavaScript Implementation
// Fiducial tracking pipeline on camera pixels
const dark = threshold(gray, mean * 0.6)
const blob = largestComponent(dark)
const quad = quadFromHull(convexHull(blob)) // farthest pair + farthest points either side
const H = homography(unitSquare, quad) // 8×8 DLT solve with h33 = 1
const code = sample7x7(gray, H) // border must be black, 5×5 interior = id bits
// Pose: [r1 r2 t] = λ K⁻¹ H S (S maps marker metres → unit square), r3 = r1 × r2
anchor.matrix.makeBasis(r1, r2, r3).setPosition(t) // then render AR content with the same intrinsics