Three.js Neural Network Training Visualization (Backprop in 3D)

Watch a small neural network learn in real time in Three.js, as a 2-8-8-1 multilayer perceptron trains with backpropagation on spiral, XOR, circle or moons data, with a live 3D decision surface and neurons and weights coloured by activation and sign.

Key topics: neural network visualization, backpropagation, three.js machine learning, decision boundary, mlp, javascript neural network

What this code does

- Network: two inputs (x, y), two tanh hidden layers and a sigmoid output giving P(orange), trained with binary cross-entropy.
- Backpropagation: errors flow backwards layer by layer (dL/dz = output − label, then through tanh′ = 1 − a²), and mini-batch SGD updates every weight.
- Decision Surface: the left terrain is the network output over the input plane, with height and colour as probability and white tracing the 0.5 boundary.
- Network View: neurons glow with their activation for a moving probe input; edges are red for positive weights, blue for negative, brighter when larger.
- Experiments: two hidden units can't solve the spiral, and a learning rate that's too high makes the loss bounce.
- Controls: dataset, hidden units, learning rate, steps per frame, pause and re-initialise.

Continue Learning

JavaScript Implementation

ES6+
// Backpropagation for one hidden layer (tanh) and a sigmoid output with cross-entropy loss

function trainStep (x, y, lr) {
  const h = W1.map((row, j) => Math.tanh(row[0] * x[0] + row[1] * x[1] + b1[j]))
  const out = 1 / (1 + Math.exp(-(W2.reduce((s, w, j) => s + w * h[j], b2))))
  const dOut = out - y                                   // dLoss/dz at the output
  W2.forEach((w, j) => {
    const dH = dOut * w * (1 - h[j] * h[j])               // back through tanh
    W2[j] -= lr * dOut * h[j]
    W1[j][0] -= lr * dH * x[0]
    W1[j][1] -= lr * dH * x[1]
    b1[j] -= lr * dH
  })
  b2 -= lr * dOut
}
// Visualise: neurons = spheres coloured by activation, weights = lines coloured by sign