topos

The guide

The neural tier

Layers, losses, attention, dropout, optimizers: every facade is a spelling over the public operations, and the dump proves it.

The rule#

Every facade — every layer, loss, optimizer, initializer — composes through the public operation surface alone. No privileged engine access, no hidden opcode. Two things follow. A facade's internal spelling can change without breaking anyone, because nobody could depend on it. And a hand-rolled equivalent behaves identically, so each facade stands as proof that the primitives suffice.

The dump is the proof. Linear::express records a product, an explicit broadcast of the bias over the batch axis, and an add:

   0  Parameter                         [3, 2]
   1  Parameter                         [2]
   2  Input                             [4, 3]
   3  MatMul         2, 0               [4, 2]
   4  BroadcastAlong 1  axis=0 extent=4 [4, 2]
   5  Add            3, 4               [4, 2]
tape: 6 nodes, 2 parameters, 1 input
Plate 1. Linear::express records four nodes over the public surface: a product, an explicit broadcast of the bias over the batch axis, and an add. There is no privileged engine access to hide behind.
The code that produced it
use topos::{Linear, Module, Shape, Tape, Tensor, init};

let tape: Tape<f32> = Tape::new();
let layer = Linear::new(
    &tape,
    init::xavier(7)(&Shape::new([3, 2])),
    Tensor::filled([2], 0.0_f32),
);
let batch = tape.input(Tensor::filled([4, 3], 0.0_f32));
let _output = layer.express(batch);
show::spec(tape.describe())

Write the same four lines by hand and the spec is the same text, asserted:

   0  Parameter                         [3, 2]
   1  Parameter                         [2]
   2  Input                             [4, 3]
   3  MatMul         2, 0               [4, 2]
   4  BroadcastAlong 1  axis=0 extent=4 [4, 2]
   5  Add            3, 4               [4, 2]
tape: 6 nodes, 2 parameters, 1 input
Plate 2. The facade and a hand-rolled affine layer record the same spec, line for line, asserted equal. A facade is packaging, not different math.
The code that produced it
use topos::{Linear, Module, Shape, Tape, Tensor, init};

let weights: Tensor<f32> = init::xavier(7)(&Shape::new([3, 2]));
let bias = Tensor::filled([2], 0.0_f32);

// The facade.
let facade: Tape<f32> = Tape::new();
let layer = Linear::new(&facade, weights.clone(), bias.clone());
let _output = layer.express(facade.input(Tensor::filled([4, 3], 0.0_f32)));

// By hand, through the public operations alone.
let by_hand: Tape<f32> = Tape::new();
let weights = by_hand.parameter(weights);
let bias = by_hand.parameter(bias);
let batch = by_hand.input(Tensor::filled([4, 3], 0.0_f32));
let _output = batch.matmul(weights) + bias.broadcast_along(0, 4);

assert_eq!(facade.describe(), by_hand.describe());
show::panels([
    ("Linear::express", show::spec(facade.describe())),
    ("by hand", show::spec(by_hand.describe())),
])

The shapes a facade takes#

Pick the facade's shape by what it is:

  • a stateless closed set of alternatives is a plain Copy enum with an express method — Activation::{Tanh, Relu};
  • a stateful or user-extensible strategy is an open, object-safe trait with the built-ins as ordinary implementations — Optimizer, so a custom optimizer has the same standing as Adam;
  • an operand-asymmetric formula with no natural self is a free function — cross_entropy, conv2d, max_pool, scaled_dot_product;
  • a factory returns impl FnMut closures with explicit seeds — init::{uniform, normal, xavier, kaiming, dropout}.

Layers with parameters are Modules: a named recording function whose express(input) records on the input's tape, holding Symbols and never payloads. Sequential chains modules; visit walks parameters under structured paths for checkpoints.

Layers#

Mlp::new(&tape, &[2, 4, 1], Activation::Tanh, init::xavier(7)) allocates two Linear stages and records them with the activation between. The picture is the graph the facade recorded, and nothing more.

0 Parameter[2, 4]0 Parameter [2, 4]1 Parameter[4]1 Parameter [4]2 Parameter[4, 1]2 Parameter [4, 1]3 Parameter[1]3 Parameter [1]4 Input[8, 2]4 Input [8, 2]5 MatMul[8, 4]5 MatMul [8, 4]6 BroadcastAlong[8, 4] axis=0 extent=86 BroadcastAlong [8, 4] axis=0 extent=87 Add[8, 4]7 Add [8, 4]8 Tanh[8, 4]8 Tanh [8, 4]9 MatMul[8, 1]9 MatMul [8, 1]10 BroadcastAlong[8, 1] axis=0 extent=810 BroadcastAlong [8, 1] axis=0 extent=811 Add[8, 1]11 Add [8, 1]
Plate 3. A [2, 4, 1] perceptron with a tanh hidden stage, as the graph the facade recorded.
The code that produced it
use topos::{Activation, Mlp, Module, Tape, Tensor, init};

let tape: Tape<f32> = Tape::new();
let mlp = Mlp::new(&tape, &[2, 4, 1], Activation::Tanh, init::xavier(7));
let _output = mlp.express(tape.input(Tensor::filled([8, 2], 0.0_f32)));
show::graph(Graph::from_nodes(tape.nodes()))

Initialization and hyperparameters are caller-owned and visible at the call site. A facade never chooses a distribution, a learning rate, or a float constant; its internal constants are integer counted ratios only.

Losses#

A loss is a composition that records a rank-0 objective. cross_entropy is the mean negative log-likelihood in the expanded, stable spelling: the fused LogSumExp is finite for every finite logit, and no term ever multiplies a zero target by an infinite log-probability. Targets travel as a one-hot Tensor::selection, so one recorded graph serves any batch of labels.

   0  Input                             [4, 3]
   1  Input                             [4, 3]
   2  SumAlong       1  axis=1          [4]
   3  LogSumExp      0  axis=1          [4]
   4  Mul            2, 3               [4]
   5  Sum            4                  []
   6  Mul            1, 0               [4, 3]
   7  Sum            6                  []
   8  Sub            5, 7               []
   9  Sum            1                  []
  10  Div            8, 9               []
tape: 11 nodes, 0 parameters, 2 inputs
Plate 4. Cross-entropy is a composition: the fused LogSumExp is the only primitive that earned a seat, for stability; everything around it is plain sums and products.
The code that produced it
use topos::{Tape, Tensor, cross_entropy};

let tape: Tape<f32> = Tape::new();
let logits = tape.input(Tensor::filled([4, 3], 0.0_f32));
// One-hot targets travel as a selection: the class index per row.
let targets = tape.input(Tensor::selection([0, 2, 1, 2], 3, 1.0_f32));
let _loss = cross_entropy(logits, targets);
show::spec(tape.describe())

Dropout is a feed#

Dropout looks like it needs a random generator in the graph and a switch that turns it off at inference. Neither is required. The layer multiplies by a mask input whose default is all ones, so an unfed run is the identity: that is inference. Training draws the mask on the host from a seeded factory and feeds it like a batch. Seeded replay stays exact, gradients stay gradient * mask, and an emitted training step gains one dynamic argument instead of a mode.

unfed  [[1, 2, 3, 4, 5, 6]]
mask   [[2, 2, 0, 0, 2, 2]]   (0 drops, 1 / keep rescales)
fed    [[2, 4, 0, 0, 10, 12]]
Plate 5. Dropout is a multiply by a mask input whose default is all ones. Unfed, the expression is the identity; that is inference. Fed a seeded mask, it drops and rescales.
The code that produced it
use topos::{Dropout, Module, Shape, Tape, Tensor, init};

let tape: Tape<f32> = Tape::new();
let dropout = Dropout::new(&tape, [1, 6]);
let activations = tape.input(Tensor::new([1, 6], [1.0_f32, 2.0, 3.0, 4.0, 5.0, 6.0]));
let masked = dropout.express(activations).symbol();
let network = tape.into_network();
let parameters = network.parameters();

// Unfed, the mask is its all-ones default: inference.
let inference = network.forward(&parameters, []);
// Training draws a seeded mask on the host and feeds it like a batch.
let mask = init::dropout(7, 0.5)(&Shape::new([1, 6]));
let training = network.forward(&parameters, [(dropout.mask(), mask.clone())]);
show::lines([
    format!("unfed  {}", inference.of(masked)),
    format!("mask   {mask}   (0 drops, 1 / keep rescales)"),
    format!("fed    {}", training.of(masked)),
])

Attention is a formula#

scaled_dot_product(query, key, value, mask, scale) records one head, rank 2: softmax((q @ kᵀ) * scale + mask) @ v. The mask and the scale are caller-supplied values — causal_mask(extent, fill) mints the standard triangular payload with a fill you choose, because a custom element may not have an infinity. Head loops, projections, rotary embeddings, grouped queries, and caches are architecture and stay with the caller, since fused-QKV and separate-projection checkpoints split heads differently.

   0  Input                             [4, 8]
   1  Input                             [4, 8]
   2  Input                             [4, 8]
   3  Leaf                              [4, 4]
   4  Leaf                              []
   5  Permute        1  order=[1, 0]    [8, 4]
   6  MatMul         0, 5               [4, 4]
   7  Broadcast      4  shape=[4, 4]    [4, 4]
   8  Mul            6, 7               [4, 4]
   9  Add            8, 3               [4, 4]
  10  LogSoftmax     9  axis=1          [4, 4]
  11  Exp            10                 [4, 4]
  12  MatMul         11, 2              [4, 8]
tape: 13 nodes, 0 parameters, 3 inputs
Plate 6. One head of scaled dot-product attention as the spec: a product, a scale, an additive causal mask, a softmax composed from the stable log-softmax, and a product with the values.
The code that produced it
use topos::{Shape, Tape, Tensor, causal_mask, init, scaled_dot_product};

let tape: Tape<f32> = Tape::new();
let query = tape.input(init::normal(1, 1.0)(&Shape::new([4, 8])));
let key = tape.input(init::normal(2, 1.0)(&Shape::new([4, 8])));
let value = tape.input(init::normal(3, 1.0)(&Shape::new([4, 8])));
// The mask and the scale are caller-supplied values: a facade
// never chooses a float constant.
let mask = tape.leaf(causal_mask(4, f32::NEG_INFINITY));
let scale = tape.leaf(Tensor::from(1.0 / 8f32.sqrt()));
let _head = scaled_dot_product(query, key, value, mask, scale);
show::spec(tape.describe())

Convolution is a formula#

conv2d is pad, unfold along each spatial axis, permute, reshape, one matrix product, and the bias spread over every position. There is no Convolution instruction: the composition is exact, and a fused kernel earns its place only through the pattern catalog, which finds this idiom whether a facade or a hand wrote it.

   0  Input                             [1, 1, 6, 6]
   1  Parameter                         [2, 1, 3, 3]
   2  Parameter                         [2]
   3  Pad            0  axis=2 start=1 full_extent=8 [1, 1, 8, 6]
   4  Pad            3  axis=3 start=1 full_extent=8 [1, 1, 8, 8]
   5  Unfold         4  axis=2 size=3 step=1 dilation=1 [1, 1, 6, 3, 8]
   6  Unfold         5  axis=4 size=3 step=1 dilation=1 [1, 1, 6, 3, 6, 3]
   7  Permute        6  order=[0, 2, 4, 1, 3, 5] [1, 6, 6, 1, 3, 3]
   8  Reshape        7  shape=[36, 9]   [36, 9]
   9  Permute        1  order=[1, 2, 3, 0] [1, 3, 3, 2]
  10  Reshape        9  shape=[9, 2]    [9, 2]
  11  MatMul         8, 10              [36, 2]
  12  BroadcastAlong 2  axis=0 extent=36 [36, 2]
  13  Add            11, 12             [36, 2]
  14  Reshape        13  shape=[1, 6, 6, 2] [1, 6, 6, 2]
  15  Permute        14  order=[0, 3, 1, 2] [1, 2, 6, 6]
tape: 16 nodes, 2 parameters, 1 input
Plate 7. A convolution is a formula: pad, unfold along each spatial axis, permute, reshape, one matrix product, and the bias spread over every position. No Convolution instruction exists.
The code that produced it
use topos::{Shape, Tape, Tensor, conv2d, init};

let tape: Tape<f32> = Tape::new();
let image = tape.input(init::normal(1, 1.0)(&Shape::new([1, 1, 6, 6])));
let filters = tape.parameter(init::normal(2, 0.3)(&Shape::new([2, 1, 3, 3])));
let bias = tape.parameter(Tensor::filled([2], 0.0_f32));
let _convolved = conv2d(image, filters, bias, 1, 1);
show::spec(tape.describe())

Optimizers and the rest#

Sgd, Adam, and AdamW implement Optimizer; moments are parameter-aligned tables that initialize from the first step's gradients. BatchNorm records the training normalization by batch statistics and, through express_with, the inference twin normalized by running estimates fed per run — the estimates live in the caller's loop as plain payloads. LayerNorm and RmsNorm are the transformer norms; Embedding is a Module holding one [vocab, dim] table whose express is the gather over a one-hot selection. Training compares two optimizers on one graph.

Where next#

  • The element seam — the other kind of extension: a number, not a layer.
  • The gallery — the makemore series, where each act adds one facade and trains identically to the hand-rolled act before it.