topos
The guide
Entries and plans
Declare what a run may read. A plan is the schedule that declaration licenses: liveness, skipped nodes, elected patterns, exact and fast.
Say what you want to read#
A sealed spec is a compilation unit, and it may export several functions that share weights: train and sample, full context and decode. An Entry is one such export, declared: the roots a run must compute, the extra interior values it will also observe, the memory posture, and the numerics.
let entry = network.entry([loss]).observe([hidden]).numerics(Numerics::Exact);
let run = entry.interpret(¶meters, []); // the oracle over just those ancestors
let plan = entry.lower(); // the compiled scheduleSay it once. Every executor honors that keep-set: the interpreter evaluates exactly the ancestors of the declared results; a plan is a derived schedule of the same closure. A read outside the declaration panics — an interior that liveness happened to retain is still unreadable, because the contract does not depend on the optimizer's choices. Network::forward, the whole-spec oracle, is the debug road for when you really do want every node.
Lowering#
lower derives a Plan: what to run, what to keep, what to free, which patterns to elect. It freezes a prefix of the spec and takes Parameters per call, so it holds no state, survives every training step, and outlives a reopen. describe prints one line per scheduled node with its liveness, then the summary.
0 Parameter [2, 2] freed after 2 1 Input [1, 2] freed after 2 2 MatMul 1, 0 [1, 2] freed after 3 3 Tanh 2 [1, 2] freed after 4 4 Mul 3, 3 [1, 2] freed after 5 5 Sum 4 [] kept plan: forward; 6 of 6 nodes evaluated, 1 readable live volume: peak 8 elements at node 2, retain-all 13
The code that produced it
use topos::{Tape, Tensor};
let tape: Tape<f32> = Tape::new();
let weights = tape.parameter(Tensor::new([2, 2], [0.5_f32, -0.25, 0.25, 0.4]));
let input = tape.input(Tensor::new([1, 2], [1.0_f32, 2.0]));
let hidden = input.matmul(weights).tanh();
let loss = (hidden * hidden).sum().symbol();
let network = tape.into_network();
show::spec(network.entry([loss]).lower().describe())Observability is a license#
The declaration is what every optimization runs on. The observed must survive any derived artifact; the unobserved is fair game to skip, free, or fuse away. Add one name to the declaration and one line changes.
0 Parameter [2, 2] freed after 2 1 Input [1, 2] freed after 2 2 MatMul 1, 0 [1, 2] freed after 3 3 Tanh 2 [1, 2] freed after 4 4 Mul 3, 3 [1, 2] freed after 5 5 Sum 4 [] kept plan: forward; 6 of 6 nodes evaluated, 1 readable live volume: peak 8 elements at node 2, retain-all 13
0 Parameter [2, 2] freed after 2 1 Input [1, 2] freed after 2 2 MatMul 1, 0 [1, 2] freed after 3 3 Tanh 2 [1, 2] kept 4 Mul 3, 3 [1, 2] freed after 5 5 Sum 4 [] kept plan: forward; 6 of 6 nodes evaluated, 2 readable live volume: peak 8 elements at node 2, retain-all 13
hidden observed turns one line from freed to kept; nothing else changes.The code that produced it
use topos::{Tape, Tensor};
let tape: Tape<f32> = Tape::new();
let weights = tape.parameter(Tensor::new([2, 2], [0.5_f32, -0.25, 0.25, 0.4]));
let input = tape.input(Tensor::new([1, 2], [1.0_f32, 2.0]));
let hidden = input.matmul(weights).tanh();
let loss = (hidden * hidden).sum();
let (hidden, loss) = (hidden.symbol(), loss.symbol());
let network = tape.into_network();
let roots_only = network.entry([loss]).lower();
let observed = network.entry([loss]).observe([hidden]).lower();
show::panels([
("roots only", show::spec(roots_only.describe())),
("observe hidden", show::spec_marked(observed.describe(), [hidden.index()])),
])Nodes an entry does not reach are not scheduled at all. When the gradient of the walkthrough is recorded with respect to the weights, the rules also produce the gradient with respect to the input — four nodes nobody declared, so a run never computes them and the emitted text never mentions them.
Live volume#
Every freed after N in a plan is a buffer release, and the summary's peak is the most elements alive at once. Plan::live_series is that accounting as a curve. A training plan's curve climbs while the forward values are still needed by the chain rule and falls as the backward half consumes them; an inference plan over the same graph stays flat and low.
The code that produced it
use malevich::{Line, LineStyle, Plot};
use topos::{Activation, Mlp, Module, Shape, Tape, Tensor, init};
let tape: Tape<f32> = Tape::new();
let mlp = Mlp::new(&tape, &[8, 32, 32, 4], Activation::Tanh, init::xavier(3));
let features: Tensor<f32> = init::normal(5, 1.0)(&Shape::new([64, 8]));
let predicted = mlp.express(tape.input(features));
let loss = (predicted * predicted).sum();
let adjoints = tape.differentiate(loss, mlp.parameters());
let loss = loss.symbol();
let network = tape.into_network();
let training = network.entry(adjoints.roots()).lower();
let inference = network.entry([loss]).lower();
let series = |plan: &topos::Plan<f32>| -> Vec<f64> {
plan.live_series().iter().map(|&elements| elements as f64).collect()
};
show::plot(
Plot::new()
.layer(Line::y(series(&training)).label("loss + gradients").style(LineStyle::Corners))
.layer(Line::y(series(&inference)).label("loss only").style(LineStyle::Corners))
.title("live volume along two plans of one [8, 32, 32, 4] mlp")
.x_label("scheduled node")
.y_label("elements"),
)The numbers are the plan's own accounting in elements — constants and placeholders count as zero — not allocator truth. They are what the analysis licenses, which is exactly the number a design decision needs. The notebook card for a plan draws this curve under the schedule.
The backward posture#
Entry::backward() declares runs that answer Run::backward: buffers retain what the engine reverse scan reads. It is a memory posture — the retain-all one, which the graded consumers preferred over freeing or rematerializing mid-run — not a second compiler. The column then says what the analysis could release, and the summary reports the retain-all total a run actually holds beside the release floor.
0 Parameter [2, 2] retained 1 Input [1, 2] retained 2 MatMul 1, 0 [1, 2] releasable after 3 3 Tanh 2 [1, 2] retained 4 Mul 3, 3 [1, 2] releasable after 5 5 Sum 4 [] kept plan: retain; 6 of 6 nodes evaluated, 1 readable live volume: retain-all 13, release floor 11 at node 5 d loss / d weights through the plan's run: [[0.6397, 0.75025934], [1.2794, 1.5005187]]
Run::backward can still scan them. The column says what the analysis *could* release, and the summary reports the retain-all total a run actually holds.The code that produced it
use topos::{Tape, Tensor};
let tape: Tape<f32> = Tape::new();
let weights = tape.parameter(Tensor::new([2, 2], [0.5_f32, -0.25, 0.25, 0.4]));
let input = tape.input(Tensor::new([1, 2], [1.0_f32, 2.0]));
let hidden = input.matmul(weights).tanh();
let loss = (hidden * hidden).sum().symbol();
let weights = weights.symbol();
let network = tape.into_network();
// `backward()` is a memory posture: runs keep what the engine scan
// will reread, so `Run::backward` still works on a plan's run.
let plan = network.entry([loss]).backward().lower();
let run = plan.forward(&network.parameters(), []);
let gradients = run.backward(loss);
let mut text = plan.describe();
text.push_str(&format!("\nd loss / d weights through the plan's run: {}\n", gradients.of(weights)));
show::spec(text)For compiled training, prefer recording the derivative and compiling a forward-only plan over adjoints.roots(): fusion and liveness then apply to the chain rule itself. backward's place is the oracle, and quick procedural use.
Patterns are offers#
The catalog matches structures over the frozen columns — a windowed product, a window reduction, the two batch-norm formulas — and a plan elects the matches it can use. A match never rewrites the tape: at home it calls a fused kernel, and a region nobody elects just runs the original operations. A value you declared readable inside a match blocks it, unless the match names that value as a result and writes it back — which is how observing batch-norm's statistics survives the fusion.
0 Input [1, 1, 6, 6] freed after 9 1 Parameter [2, 1, 3, 3] freed after 7 2 Parameter [2] freed after 10 3 Unfold 0 axis=2 size=3 step=1 dilation=1 [1, 1, 4, 3, 6] fused 4 Unfold 3 axis=4 size=3 step=1 dilation=1 [1, 1, 4, 3, 4, 3] fused 5 Permute 4 order=[0, 2, 4, 1, 3, 5] [1, 4, 4, 1, 3, 3] fused 6 Reshape 5 shape=[16, 9] [16, 9] fused 7 Permute 1 order=[1, 2, 3, 0] [1, 3, 3, 2] freed after 8 8 Reshape 7 shape=[9, 2] [9, 2] freed after 9 9 MatMul 6, 8 [16, 2] freed after 11 10 BroadcastAlong 2 axis=0 extent=16 [16, 2] freed after 11 11 Add 9, 10 [16, 2] freed after 12 12 Reshape 11 shape=[1, 4, 4, 2] [1, 4, 4, 2] freed after 13 13 Permute 12 order=[0, 3, 1, 2] [1, 2, 4, 4] freed after 15 14 Leaf [1, 2, 4, 4] freed after 15 15 Maximum 13, 14 [1, 2, 4, 4] freed after 27 16 Unfold 15 axis=2 size=2 step=2 dilation=1 [1, 2, 2, 2, 4] fused 17 Unfold 16 axis=4 size=2 step=2 dilation=1 [1, 2, 2, 2, 2, 2] fused 18 Permute 17 order=[0, 1, 2, 4, 3, 5] [1, 2, 2, 2, 2, 2] fused 19 Reshape 18 shape=[1, 2, 2, 2, 4] [1, 2, 2, 2, 4] fused 20 Narrow 19 axis=4 start=0 len=1 [1, 2, 2, 2, 1] fused 21 Narrow 19 axis=4 start=1 len=1 [1, 2, 2, 2, 1] fused 22 Maximum 20, 21 [1, 2, 2, 2, 1] fused 23 Narrow 19 axis=4 start=2 len=1 [1, 2, 2, 2, 1] fused 24 Maximum 22, 23 [1, 2, 2, 2, 1] fused 25 Narrow 19 axis=4 start=3 len=1 [1, 2, 2, 2, 1] fused 26 Maximum 24, 25 [1, 2, 2, 2, 1] fused 27 Reshape 26 shape=[1, 2, 2, 2] [1, 2, 2, 2] freed after 28 28 Sum 27 [] kept plan: forward; 29 of 29 nodes evaluated, 1 readable fused 2 groups, 15 nodes replaced live volume: peak 96 elements at node 11, retain-all 325
2 patterns elected: WindowProduct rooted at node 9, covering 5 nodes, served as WindowProduct ReduceWindow rooted at node 27, covering 12 nodes, served as ReduceWindow
fused.The code that produced it
use topos::{Shape, Tape, Tensor, conv2d, init, max_pool};
let tape: Tape<f32> = Tape::new();
let image = tape.input(init::normal(1, 1.0)(&Shape::new([1, 1, 6, 6])));
let filters = tape.parameter(init::normal(2, 0.3)(&Shape::new([2, 1, 3, 3])));
let bias = tape.parameter(Tensor::filled([2], 0.0_f32));
let convolved = conv2d(image, filters, bias, 1, 0).relu();
let pooled = max_pool(convolved, 2, 2);
let loss = pooled.sum().symbol();
let network = tape.into_network();
let plan = network.entry([loss]).lower();
let mut elected: Vec<String> = plan
.patterns()
.iter()
.map(|pattern| {
format!(
"{:?} rooted at node {}, covering {} nodes, served as {:?}",
pattern.kind(),
pattern.root().index(),
pattern.nodes().len(),
pattern.formula()
)
})
.collect();
elected.insert(0, format!("{} patterns elected:", elected.len()));
show::panels([
("plan", show::spec(plan.describe())),
("elected", show::text(elected.join("\n"))),
])Plan::patterns is the elections as data. The same match has a second job abroad: when the plan is emitted, it becomes the other compiler's named op. Fusion raises tells that story; emission shows the text.
Exact and Fast#
Results are a function of the payloads, the compiled features, the numerics posture, and the machine. Network::forward is exact by construction: its bits are the same in every build, on every platform. A plan under Numerics::Exact computes those same bits in the same process, whatever backend features are compiled in. Under Fast — an entry's default — a backend may serve large products with sums in its own order; that is a labeled choice on the entry, never a silent effect of a feature flag.
interpreter 15.916065 exact plan 15.916065 fast plan 15.916065 exact == interpreter: asserted, in every build fast == exact: yes, no backend is on in this build
Exact plan, and the default Fast plan. The first two are asserted equal in every build; the third differs only where a backend served and reordered a sum.The code that produced it
use topos::{Numerics, Shape, Tape, Tensor, init};
let tape: Tape<f32> = Tape::new();
let weights = tape.parameter(init::xavier(3)(&Shape::new([64, 64])));
let batch = tape.input(init::normal(5, 1.0)(&Shape::new([32, 64])));
let loss = batch.matmul(weights).tanh().sum().symbol();
let network = tape.into_network();
let parameters = network.parameters();
let interpreter = network.forward(¶meters, []);
let exact = network.entry([loss]).numerics(Numerics::Exact).lower().forward(¶meters, []);
let fast = network.entry([loss]).lower().forward(¶meters, []);
let bits = |tensor: &Tensor<f32>| tensor.scalar().to_bits();
assert_eq!(bits(interpreter.of(loss)), bits(exact.of(loss)));
show::lines([
format!("interpreter {}", interpreter.of(loss)),
format!("exact plan {}", exact.of(loss)),
format!("fast plan {}", fast.of(loss)),
String::new(),
"exact == interpreter: asserted, in every build".to_string(),
format!(
"fast == exact: {}",
if bits(fast.of(loss)) == bits(exact.of(loss)) { "yes, no backend is on in this build" } else { "no: a backend summed in its own order" }
),
])Which backend actually served is run-time data: wrap any region in Backend::tallied and read the per-formula tally. Acceleration has the ladder and the numbers.
Where next#
- Emission — the plan as text.
- Observability is a license — the rule, and several exports, one spec, the twins it makes real.