Changelog

All notable changes to this project are documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased#

Added#

• The documentation site, https://shergin.github.io/topos/, generated by the topos-site workspace member in site/ and deployed by .github/workflows/pages.yml. Every plate on it is program output: a block of Rust in site/figures/ records a graph and asks the crate for one reading of it -- the spec dump with every operand reference linked, the derivative shaded below a divider, a plan with its liveness, the StableHLO, a run's values, a training curve, a notebook card, or the graph drawn as a picture -- and the code shown beside the plate is the code that ran. The site-authored chapters (getting started, the walkthrough, recording, differentiation, training, entries and plans, emission, the neural tier, the element seam) live beside the repository's own docs, which the site renders with plates inserted under their headings. The gallery shows every example with what it printed into a pipe, recorded by site/record.sh into site/gallery/. The playground compiles the crate to WebAssembly (site/wasm/) and reads a small tape language six ways, live. site/plates/ holds the pieces the generator and the playground share.

0.13.1 - 2026-09-07#

Added#

• examples/walkthrough.rs, the stack read end to end on one graph. A matrix product, a tanh, and a scalar loss are recorded once, then printed as the spec, differentiated into more spec (with a plain-language key to the appended nodes), lowered into a plan whose liveness and skipped nodes are explained, emitted as StableHLO, and run three ways: the interpreter, an Exact plan asserted bit-identical to it, and the default Fast plan, with the recorded gradient asserted equal to the engine scan. Five steps of gradient descent close the loop, and a closing section replays the spec by hand with Opcode::express to show the printed IR is executable. The README lists it first, and the terminology and crate docs point at it.

Changed#

• docs/vision.md ranks its audiences. The third commitment now reads "built for learning and research", and a "Who it is for" section says the rest plainly: learners and teachers first, systems researchers second, and production as an export of that discipline, offered where it fits and never chased. The five rules are unchanged.

• Rustdoc caught up with batched matmul and the spec/state split. Value::matmul and Tensor::matmul describe the batch prefix instead of claiming rank 2. The parameter node, the slot table, the run posture, the notebook payload card, and the dropout, MLP, and batch-norm facades no longer speak of generations, Network::update, or rematerialization recipes, and the entry, plan, adjoints, emission, and numerics docs say "entry" where they still said "request": the names 0.12 retired.

• The crate declares its minimum supported Rust version, 1.88: the release that stabilized let chains in edition 2024, the newest language feature the crate uses. rust-version makes Cargo refuse an older toolchain at resolve time instead of mid-build, a new msrv CI job runs the default test gate on exactly that toolchain, and the README says the floor in one line.

0.13.0 - 2026-08-30#

Added#

• A dual number as a research element (examples/dual.rs): forward mode by reinterpreting the payloads, where the forward-mode example got it by transforming the graph. Dual implements Differentiable and Elementary from outside the crate — every map the chain rule on the primal's own libm kernel, the accumulator carrying product-rule terms through inner sums — and Tape<Dual> then records the unchanged primal graph while the unchanged interpreter computes a directional derivative in every payload slot. Graded as a triangle at dyadic values: the dual tangent equals the engine reverse scan equals the recorded gradient, bit for bit (a zero-error point is excluded on record: the two modes then differ in the sign of zero). tests/dual.rs gates the same asserts plus the accumulator check through the example's own number, and element_seam.rs points here as the seam's research consumer.

• Attention and embedding join the facade tier, in the shapes the three in-repo transformers dictate. scaled_dot_product is the operand-asymmetric free function — one head, rank 2, mask and scale as caller-supplied values, recording exactly the node sequence every hand-roll already recorded, so migration is bit-identical; head loops, projections, RoPE, grouped queries, and caches stay with the caller, because fused-QKV and separate-projection checkpoints split heads differently. causal_mask(extent, fill) is the host-side payload factory — the caller supplies the fill, since a facade never chooses float constants and a custom element may not have an infinity. Embedding is a Module holding one [vocab, dim] table whose express is the gather; it visits its parameter as weights, so migrating the language models' tables preserves their checkpoint paths, and tying reads the exposed table symbol. The makemore transformer is the consumer of record: migrated, its 5000-step trajectory and sampled names are byte-identical to the hand-rolled form. The GPT-2 and Llama attention loops migrated too — both full-context paths and the GPT-2 decode step — generating byte-identical text; Llama's group-hoisted key transpose moved inside the call, which re-records the O(1) permute view per query head (616 more nodes at TinyLlama scale, no value changed).

• docs/openings/, a short series of architectural decisions and what they paid for — not principles: AD as a named reading, recurrence as feeds, several exports of one spec, fusion that raises, the shipped oracle, and dropout as a feed.

• Forward mode, as the first out-of-tree AD mode (examples/forward_mode.rs): a Dual payload implementing Recordable — forward-mode knowledge is dual arithmetic, a payload, never a second rule body — replayed through the spec with Opcode::express. Over Dual<Tensor> the walk is an eager JVP; over Dual<Trace> it records the tangent computation as ordinary spec beside resolved sources. Graded bitwise: eager equals recorded, the directional derivative equals the reverse gradient contracted with the seed on a rational spec, and forward-over-reverse equals reverse-over-reverse on a Hessian-vector product — the recorded gradient is just more spec, so forward mode walks straight over it. Terminology gains Replay and JVP entries.

• The spec is executable IR, not only printable. Opcode::express computes or records one operation over any Recordable payload, and Opcode::vjp is the public name of the one derivative-rule body — so walking Network::nodes and expressing each opcode replays a frozen spec (over Tensor it is the interpreter, over Trace it re-records), and a reverse scan can be written from outside the crate against the crate's own rules, with the engine scan as its oracle. Sources panic ("supplied, not expressed"); the opcode set stays closed — the surface applies rules, it does not define them. tests/spec_surface.rs welds all three readings: replay-over-tensors equals Network::forward bitwise, replay-over-traces re-records a spec whose describe matches byte for byte, and a public reverse scan matches Run::backward bitwise.

• BatchNormTask::new is public, for the same reason as GemmTask::new and MapTask::new: an out-of-tree element implementing Elementary::batch_norm builds the same validated tasks its differential tests need, closing the one hole in the reference module's promise that every hook grades exactly the way the in-crate ones are graded.

• The hand-maintained op tables are welded by test. op_tests constructs one instance of every Op variant — so a new variant breaks the list at compile time — and asserts the engine's arity agrees with Opcode::arity, that is_source names exactly the arity-0 variants, and that every read contract fits Reads::operands' width: the day an op outgrows the liveness array, the failure is a named test, not an index panic inside a caller's training-plan compile.

• The run-time half of dispatch is visible: Backend::tallied(body) runs any region — a plan run, a backward, a direct payload call, a whole training step — with a dispatch tally open and answers the collected Service rows: formula, precision, who served (with None naming the reference paths), and how many tasks landed there. Coverage declares may; the tally reports what did. Shaped like Numerics::exactly — a scoped closure, per thread, nested scopes capturing innermost — and free when no scope is open. The throughput example prints one step's tally (six products and a map, and which backend took them), the exact pinned-bits test now also asserts its roads tally only reference rows, and service_tests pins the row semantics in every build.

• The declared reading is public data. Entry's four fields — roots, observe, backward, numerics — are public (the builder methods stay the construction road), and BoundEntry::entry borrows the declaration as the twin of into_entry. Symbol::index answers the position describe prints for nodes and operands, previously unreadable from the public IR view. Parameters::payloads is the slot-ordered table read, the parameter-grain twin of Field::payloads. Tensor::as_constant answers the fill value of a constant payload — a representation fact for O(1) displays and norms. The notebook tier composes through these reads alone: its last privileged accesses are gone, and every card is now hand-rollable from outside the crate. The new tests/notebook_surface.rs welds that claim — it compiles as an external consumer and rebuilds each card's data from the public readers.

Changed#

• Recordable now spans the whole recordable operation set: logsumexp and log_softmax join the vocabulary. The two log-domain opcodes earned their seats on max-shifted bits, but no derivative rule happens to call them, so the trait — whose own doc says every member corresponds to something a tape can record — was the opcode set minus two, and no payload-generic algorithm could replay a spec containing them. Their stable forward bodies moved from the op files to inherent Tensor methods, where every other forward lives; Trace records the nodes. Out-of-tree Recordable impls gain two members.

• Kinship has one spelling. The origin-plus-coverage check a carrier makes when it meets a Symbol was written out six times — on Tape, Network, Run, Field, Parameters, and Plan — the same one-job-many-spellings shape the identity protocol was. A crate-internal Kinship now owns the check: one locate with the shared family message and each carrier's own coverage wording, and a family half for the slot-grained Parameters, whose coverage is the slot lookup. Every panic message is verbatim what it was; purely internal. The carrier-vs-carrier agreements (a parameter table against a network's slots) are a different check and keep their own spellings.

• Fidelity gained a third value, Composed, and the word stopped meaning two things. BitIdentical is now the absolute claim — the reference bits in every build, under either posture — and only the fused reduce-window walk makes it. The fused window product and batch-norm cells are Composed: exactly as faithful as the composition they replace, because their interiors consult the same chain, which is why they honor an Exact demand — the interior chain declines along with everything else. Admission is unchanged and still one comparison (Composed meets both demands, and demands remain the two postures' fidelities), so no election, plan, or bit changes anywhere.

• Network::forward — whole-spec evaluation, the proving road — now runs under Numerics::Exact by construction: the backend chain declines every task, so its bits are the reference bits, the same in every build and on every platform, and Run::backward on such a run differentiates exactly. BoundEntry::interpret now honors its entry's declared numerics posture (it previously ran under the ambient default whatever the entry declared), and every interpreter run records the posture it executed under, so backward re-enters exactly what the forward ran under. Compiled speed is untouched: plans keep the Fast default and the backend chain, and the throughput example's training phase moved to an engine-backward plan accordingly. Vision rule 3 and docs/acceleration.md now state the anchor precisely, and the new numerics_tests welds it: pinned digests of a product, a map, and a log-softmax loss that every feature build and platform must reproduce bit for bit on the exact roads.

Fixed#

• The vocabulary's identity constructors agree between their two interpretations for every element. Trace::zero_like/one_like recorded counted leaves while Tensor's hold zero/one — the same from_count divergence as the scan seeds, one level down and worse: five derivative rules call one_like, so for an element where from_count(1) is not one the recorded gradients themselves would drift from the engine's. Trace now records filled leaves of the engine's zero and one.

• The two reverse scans plant the same seed expressions for every element. Tape::differentiate seeded with counted(shape, 1) and minted non-ancestor zeros as counted(shape, 0) — the size-derived constants — while the engine scan plants one_like and zero_like; identical for the built-in elements, silently divergent for any element where from_count is not one/zero. Both recorded spellings now use the engine's one and zero, and the closure contract is pinned off f64 by a Bf16 case in the closure suite.

• Plan::describe and the live-volume accounting now tell the truth about a fused group's named results. An observed batch-norm mean or variance is written back by the group's action — materialized and readable — so its line reads kept, no longer fused; and live_story/live_series count the written-back statistics from the group's root onward instead of omitting them. Observing a named result costs no live volume (the write-back happens either way), which a new build-adaptive test pins along with the labels. The Plan::patterns doc no longer claims roots print as fused, and a stale intra-doc link in the window pattern points at Tensor::windowed_product again.

0.12.0 - 2026-08-25#

Added#

• The gpt2 example generates through a one-token decode plan with per-layer KV caches carried by the caller (a second expression of the same parameters on the same tape; scatter appends the new rows over a position one-hot). Measured on an M1 Pro with accelerate: 18 ms/token f32 and 31 bf16 against the full-context loop's 193 and 341, token-identical text; the retained full-context loop is the full engine, and a plan test pins the two graphs bit-for-bit at toy scale.

• Numerics, the two-valued numerics posture on compile requests: Request::numerics(Numerics::Exact) makes the backend chain decline every task, so those runs compute on the built-in reference paths — the same bits as the default build, in every build — while Fast (the default) keeps the chain with its thresholds as cost heuristics. Runs carry the posture into backward, and an exact and a fast result are comparable in one process.

• StableHLO emission raises the recorded BatchNorm formulas to stablehlo.batch_norm_training / batch_norm_inference. The training raise returns the batch statistics as its own results, so observing the mean and variance no longer bars recognition; the primitive statistic reductions never cross the boundary. Raise-only: forward runs execute the recorded formulas unchanged.

• Differentiable::is_counted, the recognizer half of counted: a provided method certifying that a payload is exactly the size-derived constant counted mints. Pattern matchers use it to verify a recorded mean's divisor before raising; the conservative default answers false and only forgoes recognitions.

• StableHLO emission raises the canonical max_pool window fold to stablehlo.reduce_window: the unfolded lanes and the recorded maximum fold no longer cross the boundary as a static gather. The pattern is raise-only — forward runs execute the recorded fold unchanged, on every memory posture.

Changed#

• The Keep trait is Detach: [w, x, loss].detach(), the associated type Detached, and Tape::record bounding its closure on Detach. Construction's "names that leave the tape" and a plan's keep-set are different declarations, and one word for both was the observability muddle -- the execution keep-set keeps the name, and what construction does is detach. use topos::Keep becomes use topos::Detach.

• Tensorial is Recordable. The docs always called it the recordable vocabulary -- the operations a derivative rule may call, and nothing else -- while the name suggested a third payload kind beside Tensor and Trace, which is exactly the confusion the element seam forbids. The trait's members are unchanged; every E: Tensorial bound and use topos::Tensorial becomes Recordable.

• The pattern layer split into discovery and election: compilation pools every closed candidate once, posture-blind, and each consumer elects the entries its repertoire supports, so a partial-support consumer never claims a region it cannot use. Emission's repertoire is total, and engine-backward plans now raise window-GEMM groups to stablehlo.convolution exactly like forward plans — the old posture gate wrongly kept them primitive. Home fusion is unchanged on every posture.

• The window-GEMM match moved from ad-hoc Plan columns into a pattern catalog (src/engine/pattern/): one compiled column any number of patterns share, with home fusion and StableHLO raising as two actions on the same match. Same matches, same skips, same describe wording, same emitted modules, same bits.

• The spec and the state are now separate types, and the whole identity protocol is gone. Recording happens on the new public Tape (the construction phase: leaf, parameter, input, resolve, differentiate, and the Value operators); tape.into_network() consumes it and seals the recording into an immutable Network (structure, shapes, parameter initials, input defaults — no lock, no live state, shareable by &/Arc, not Clone); network.parameters() materializes the record-site initials into the caller-owned Parameters state. Training is pure data — parameters = parameters.step(&gradients, rule) (with step_each passing the parameter's Symbol) — replacing Network::update/update_each, and mints no new network, so generations no longer exist. Runs and plans take the state per call: network.forward(&parameters, feeds) (replacing forward/forward_with), network.forward_for(&parameters, targets, feeds), and plan.forward(&parameters, feeds). network.into_tape() consumes the network to reopen recording — linear by ownership, so divergent histories are unrepresentable — and parameters.carried(&network) moves state across the round trip; parameters.of(symbol) reads a payload, parameters.with_payloads(...) installs checkpoints. What-ifs are parameters.clone(): one spec, any number of states.

• The module tree now mirrors the stack, in four tiers under src/: core — the payload seam, the closed instruction set (op), the graph world (tape, network, parameters, fields, handles, the recording surface), and the engine that reads them (Run, Plan, and the forward entry points); derived — the backend chain and StableHLO emission, faster or foreign readings of the same spec; facade — the neural and notebook conveniences, which compose through the public surface alone; and the published reference kernels. The engine enum is Op, not Function: it is the instruction set, not an ML function and not a Rust fn. Purely internal: the crate root re-exports are unchanged.

• The compile request is Request, and its posture flag is backward(). Compile was the register's one verb-named type and stuttered at 57 of its 60 call sites (network.compile(Compile::roots(...))); the request now reads as English — network.compile(Request::roots([loss]).backward()) — and the flag mirrors Plan::can_backward instead of leaning on the internal engine-versus-recorded vocabulary. (training() was rejected as the simplification because it would lie: the recorded route trains through forward-only plans.)

• Post-recording APIs speak Symbol only: Run::of, Run::backward, Field::of, forward_for targets, and recorded_gradients pairs take symbols, Compile is non-generic with roots and observes converting through Into<Symbol>, and Tape::differentiate (moved from Network) accepts values or symbols. A Value is construction-phase only — the borrow checker rejects one crossing the seal, and .symbol() is the bridge.

• The facades record on the tape: Module::express(&tape, input), the constructors of Linear, Mlp, Neuron, BatchNorm, LayerNorm, RmsNorm, Conv2d, and Dropout take &Tape, Optimizer::step(&parameters, &gradients, &rate) -> Parameters (with AdamW::step_where's policy now FnMut(Symbol, &Data) -> bool), and the checkpoint pairs are plain Parameters transforms (snapshot(&parameters, &module), restore(..) -> Parameters, and the named twins) — the internal restore hack that forged a generation through a zero-gradient update is gone.

Removed#

• The identity protocol and everything it existed to arbitrate: branches, the tip claim, witnesses, chain agreement, Misbinding, Network::compacted, generation vocabulary, Network::clone forks, the ValueRef trait (no API accepts both a Value and a Symbol anymore), and the notebook leak apparatus (Network::leaked/leak) — owned Tape/Network/Parameters values move cell to cell and symbols are the cross-cell currency. What remains of graph identity at runtime is origin equality plus coverage: two integer compares.

• The consumer-less quarter of the neural tier, retired by the post-split audit under the consumers-before-machinery rule: Neuron (the scalar teaching block no example used), Residual (whose target consumers hand-roll pre-norm skips it cannot express), BatchNormInference, the module wrappers MaxPool, AveragePool, Flatten, and Reshape (their Sequential-composed convnet consumer never appeared; the max_pool free function and Value::reshape stay), the average_pool free function, Linear::from_symbols (tying goes through exposed symbols, as GPT-2's tied head does), Activation's composed Identity, Sigmoid, LeakyRelu, and Elu variants (the enum keeps the dedicated Tanh/Relu pair; other activations are caller-side compositions, like the GELU example), Value::broadcast_pair, and Tape::try_resolve.

0.11.0 - 2026-08-17#

Changed#

• Renamed the crate from poorgrad to topos. The repository moved to https://github.com/shergin/topos, the POORGRAD_* environment variables the examples, tools, and CI honor are now TOPOS_*, and emitted StableHLO modules open with module @topos.

• One compile verb: Network::compile takes an explicit Compile request — roots (no root is special; recorded gradient symbols compile as ordinary roots), observe (extra readable interiors), and an optional engine_backward() posture that retains what backward reads — replacing compile(targets, keep) and compile_training(loss, keep, Retention). The internal per-op contract is Reads (ending the name collision it shared with the public policy), Plan::can_backward is the only posture a plan exposes, and describe prints the posture as forward/retain. Every in-repo consumer migrated in the same change and trains bit-identically — the gradings' loss bit patterns are unchanged across the break.

• Evaluation is now Run, and it no longer borrows the network: a run owns its frozen structure columns and an identity witness, so it can be stashed, moved across threads, and differentiated after the generation that produced it is gone. Network::forward* and Plan::forward return Run<Data> (no lifetime parameter); of, backward, and recorded_gradients are unchanged. Documented under Run in TERMINOLOGY.md.

Removed#

• Rematerialization. The remat posture traded backward time for memory and once won on MNIST (9% less peak RSS than retain-all for 22% more step time); with recorded gradients landed, the gradings measured the recorded route smaller and faster than both engine postures on both consumers — CIFAR ~337 ms/step and ~1.05 GiB against remat's ~414/1.31, MNIST ~78 and ~270 MiB against remat's ~98/~330 — so the machinery retired: the drop set, the backward rematerializer and its memo, the fused-patch rebuild, and the size threshold. Engine-backward plans now always retain what backward reads; the recorded route is the measured choice when training memory matters. Every posture remains bit-exact.

Added#

• Dropout and init::dropout — mask-fed dropout: the module multiplies its input by a declared mask input whose default payload is all ones, so an unfed run is the identity and inference is the absence of a feed, not a mode; the seeded factory draws inverted-dropout masks (0 or 1 / keep) host-side, fed per training step like any other run state. No RNG enters the graph, so seeded replay stays bitwise — held by a masked-training replay test — and an emitted training step gains one dynamic argument. The transformer act trains with dropout on both residual writes (keep 0.9 on the page) and samples through the unfed default.

• The makemore_attention_grading example: the transformer training joint step recorded twice — rank-2 head loop and batched attention, identical payloads and arguments — both emitted to StableHLO and timed through one resident XLA server, with the in-crate oracle as the envelope. On XLA-CPU the batched recording serves ~3% faster (42 dot_generals against 48); on XLA-Metal (jax-metal) both modules return the same wrong loss — red rows the harness reports instead of hiding (TOPOS_ENVELOPE=report) — reproducing the tier-1 finding on training modules. The serving script's device placement now falls back when a PJRT plugin exposes but does not implement buffer_from_pyval.

• Batched matmul: operands of rank above two multiply batched — the trailing two axes contract as the plain product, and every leading axis is a batch axis, required identical on both operands (no broadcast batching; rank-2 behavior is unchanged). The forward loops the rank-2 gemm seam over the batch prefix, so the accelerated backends and the bf16 accumulator contract are inherited and each batch slice is bitwise the rank-2 product of that slice — held by an op-level test against a narrowed head loop, forward and gradients both. The adjoint closes inside the existing op set through permute, differentiate covers the batched case in the bitwise closure suite, and emission lowers to dot_general with batching dimensions, conformance-run through the XLA evaluator.

• The makemore_mlp_emitted example — E2, emitted training: the makemore_mlp_compiled model's loss and recorded gradients compiled as one forward-only plan and emitted as a single StableHLO function (parameters, batch) -> (loss, gradients...), executed by XLA per training step while the host keeps the update loop as plain payload arithmetic (parameters never re-enter the tape). The emitted result order is pinned by recording one same-shape reshape alias per gradient in caller order — results are the readable set in recording order — and every argument stages dynamic, since parameters change each generation. Graded against the in-crate oracle trajectory in the same binary: step-0 relative drift 6.8e-8, every 500-step window mean equal to four decimals over 5000 steps, final window 2.2450 exactly; 0.82 ms/step through XLA-CPU against the in-crate plan's 0.39 at this scale.

• The makemore_mlp_adam example — the optimizer act: the same model, seeds, and batches as makemore_mlp_compiled, trained once with Sgd and once with Adam, both curves on one chart; the SGD run reproduces that example's losses bit for bit, and Adam converges below it (last-500 mean 2.2287 vs 2.2450) at one flat learning rate. TOPOS_GRADIENTS=engine flips the gradient source to the interpreter's backward — losses identical, but the moment fields densify: ~16.8 MB peak RSS against the recorded route's ~15.0 at makemore scale, confirming that recorded gradients (O(1)-constant non-parameter slots) are the natural optimizer partner.

• The cifar10_grading and mnist_grading examples: the same convnets trained through the gradient routes — engine backward over an engine-backward plan, and recorded gradients (differentiate plus one forward-only plan over [loss, gradients...]) — bit-identically under matched seeds, one route per process so an external monitor attributes peak RSS cleanly. On the measured 300-step runs the recorded route wins both axes on both consumers: CIFAR ~337 ms/step and ~1.05 GiB against ~379/1.35 (retain-all) and ~414/1.31 (remat); MNIST ~78 and ~270 MiB against ~89/~365 (retain-all) and ~98/~330 (remat).

• Network::compacted: rebuilds the structure columns into private arenas that hold only this network's live nodes. A plain clone is still O(1) and shares the append-only arena — right for train-only forks, wrong when siblings record after the fork and pin sibling garbage for the lineage. Compaction is O(live nodes), stays in the same lineage, and is the explicit unwind for that trade. Documented under Arena / Fork / Compaction in TERMINOLOGY.md.

Fixed#

• Docs: the train-loop memory contract is stated accurately — parameter payloads reclaimed per generation since the parameter store (0.7 era); residual arena cost is sibling structure after forked recording, not weights per step.

0.10.0 - 2026-08-10#

Added#

• The evcxr feature: rich cell output for Evcxr notebooks and the Evcxr REPL, and Network::leaked/Network::leak, which return a &'static Network so recorded proxies outlive a notebook cell. The feature adds inherent methods to existing types and no new vocabulary — a notebook drives the ordinary API — and every display is a pure to_html(Theme) string covered by cargo test, with a text/plain alternative for the terminal REPL. Value, Tensor, Network, Plan, Field, Evaluation, and Symbol all draw; charts come from malevich, and Plan plots its live volume beside the schedule describe prints. A companion evcxr-pixel feature upgrades terminal charts to sixel/kitty images. The idiom, what leaking costs, and the rough edges: NOTEBOOKS.md.

• The Module composition tier: Module, a named, parameterized recording function — express(&network, input) records through the public op surface, parameters held as detached Symbols, the cost never reaching a run — with Sequential (heterogeneous stages behind the sanctioned record-time dyn, appended with the boxing then), the path-transparent Residual, the shape adapters Flatten and Reshape, module forms of pooling (MaxPool, AveragePool), and implementations for Activation, Conv2d, LayerNorm, RmsNorm, and Mlp. BatchNorm gains the explicit inference-mode adapter (inference(mean, variance)); training mode deliberately stays a plain method, because it returns the batch statistics and a module must not hide values its caller needs. Parameter traversal is visit over structured Path/Segment paths (static-literal leaves, integer indices), with parameters and named_parameters derived; programmatic access — tying, freezing — uses typed accessors (weights(), bias(), Linear::from_symbols), never names.

• Module checkpoints in two identities, with zero engine changes: positional checkpoint::snapshot/restore match by visit order — sufficient for resuming the same code — and checkpoint::named_snapshot/named_restore match by structured path, which survives code evolution and maps to foreign name-to-tensor checkpoints; missing and unexpected paths are loud errors. Restoring builds a new network generation through update_each, so shape mismatches panic through the existing validation and nothing mutates; weight tying round-trips.

• Bf16, the brain-float payload: a u16 newtype implementing Differentiable, Elementary, and the scalar-identity Tensorial, where every operation converts to f32, computes, and rounds back to nearest-even — the standard bf16 semantic, deterministic on every platform. Half the memory of f32; integers exact up to 256, per the documented counted contract. Tensor<Bf16> and Network<Bf16> run the engine unchanged, autodiff included — the payload contract holding beyond the IEEE singles, with no engine changes at all. Matmul is the one documented exception to the per-op semantic: the Elementary::gemm hook accumulates in f32 and rounds once per output element — the convention bf16 hardware and every mixed-precision recipe follow — expanding the operands exactly and riding the accelerated f32 backend chain, with the composed f32 kernel as the deterministic fallback. The everyday numeric traits come along: Display and PartialOrd through the exact f32 expansion with float semantics, Default as the additive identity, From<f64> (rounding once — double rounding through f32 is exact at bf16's precision), and exact widening From<Bf16> for f64.

• ValueRef, the unified value reference: Evaluation::of, Evaluation::backward, Field::of (so Gradients::of), plan targets (compile, compile_training, compile_training_compact, forward_for), and differentiate accept either a generation-bound Value or a detached Symbol, so the evaluation.of(network.resolve(symbol)) chain collapses to evaluation.of(symbol) and compile([loss.symbol()], []) to compile([loss], []). A sealed trait with monomorphized dispatch; each form keeps its full validation and panic messages — a symbol read on a Field checks lineage, branch, and position against the field's own chain, the detachment fields were built for. Feed pairs and keep lists stay Symbol-typed so empty list literals keep inferring; existing Value call sites compile unchanged.

• Tensor::convert: storage-preserving element conversion through the target's From — a constant stays a constant, a selection stays a selection, and a dense view keeps its layout, so a broadcast converts only its distinct buffer elements. The precision boundary for mixed-precision work: loading an f32 checkpoint into a Tensor<Bf16> model, or widening bf16 results back, priced at one conversion per stored element. Held by an end-to-end test recording the same model in both precisions and bounding the gradient divergence by bf16 epsilon.

• StableHLO emission for Bf16: an Emittable implementation (bf16 element type, literals through the exact f32 expansion, bit-pattern hex for the non-finite values), and dtype-aware conformance tooling — the evaluator scripts read each argument's element type from the module's own @main signature, feeding the reference interpreter through parsed dense literals and XLA through ml_dtypes arrays. The execution envelope is per-case, scaled to the element type's epsilon, since bf16's 2^-8 cannot live under an f32-shaped fixed tolerance. Accumulation is IR semantics, never an implementation's private choice: Emittable gains ACCUMULATION (bf16 declares f32), and matmuls and fused convolutions of such an element emit the wider result type with an explicit stablehlo.convert back — exactly what the home gemm seam computes.

Changed#

• Breaking: Layer is gone, replaced by the unfused Linear — the affine transform alone, with activation as its own composition stage, unlocking the orderings a bundled activation forbids (pre-norm blocks, activation-before-projection). Mlp keeps its constructor and its bit-identical recordings as the convenience over Linear.
• Breaking: the compile facade loses its fork — compile_training_compact is gone, and compile_training takes an explicit Retention policy (All or Compact) as its third parameter. The policy is a closed set of alternatives, so per the facade rules it is a plain Copy enum chosen at the call site, with each variant's measured trade documented on the variant instead of split across two method docs. Symbol::from(value) / From<Value> for Symbol also lands as the conversion form of Value::symbol, for lists that must be homogeneous in Symbol.
• Breaking: Differentiable gains the accumulation contract — type Accumulator with promote/demote. Matmul inner products (every path, including the composed fallback a constant operand takes), the sum reductions, fold, and scatter promote each term, accumulate there, and round back once. The IEEE singles accumulate in themselves (Accumulator = Self, bit-identical and bench-checked); Bf16 accumulates in f32, which closes the representation dependence a gemm-hook-only contract left open. Emission follows the contract: add-reduces, fold, scatter, and the reduces inside the fused log_sum_exp and log_softmax decompositions emit the declared accumulation type with explicit converts whenever the element names one.
• Breaking: Tensor::iter yields owned elements instead of references (Item = Element, under the Clone bound every real element type already satisfies), and PartialEq for Tensor gains the same Clone bound. The reference bought nothing: for the numeric payloads both spellings compile to the same load, and a storage representation that computes its elements has nothing to lend a reference to. Callers migrate by deleting .cloned() or .copied() after iter().
• malevich moved from a dev-dependency to an optional dependency behind the evcxr feature, and both entries moved from 1.12 to 1.15.0, whose public evcxr module supplies the stdout protocol and the card background topos's own cards are drawn on — the same two a Plot paints itself with, so a tensor table and a chart in one notebook cell cannot disagree. A default build's dependency tree is unchanged.
• The gpt2 example reads JSON through serde_json instead of a hand-written parser: the safetensors header is a derived struct and the vocabulary a HashMap<String, usize>, retiring 248 lines of parser that taught nothing about autodiff. GPT-2's byte-level BPE and the safetensors layout stay hand-rolled and in view — the dependency reads the syntax, never the algorithm. A dev-dependency only, and no new crate in the tree: criterion already brought serde_json.
• The gpt2 example rebuilds on the module tier, closing the design's proof milestone: the model is a model.rs module tree — blocks as structs of Linears and LayerNorms around a custom attention module, stacked in a Sequential, the tied head read through a typed accessor — and the bespoke construction-time loader is gone, replaced by one checkpoint::named_restore over the paths the tree announces itself: visit mirrors the checkpoint's own layout (h.{i}.attn.c_attn, ln_f), so the adapter shrinks to the leaf spellings. The tree is generic over the element type, which is the new bf16 engine — the identical modules over Tensor<Bf16>, the checkpoint converted at the precision boundary, 341 ms/token against the f32 tape's 195. The XLA engine's static arguments now come from the positional checkpoint::snapshot, whose visit order is recorded to match the emitted argument order.

Fixed#

• The sealed-trait private_interfaces warnings on cargo check. They were harmless to a normal build and fatal to a notebook: Evcxr determines a cell's variable types by parsing rustc's output and treats a dependency's warnings as a compilation failure, so any warning here broke every cell that bound a variable of a topos type.

0.9.0 - 2026-08-08#

Added#

• The Optimizer trait with Sgd, Adam, and AdamW: a training-step strategy is a uniform, object-safe slot the loop can hand any implementation — deliberately an open trait, not a closed enum, so custom optimizers have the same standing as the built-in ones. Hyperparameters are single-value payloads written at the call site; Adam carries its moments as Fields and its bias-correction powers as payloads (exact, no powf); AdamW applies decoupled decay under a structural default policy (rank two and above decays; biases and norm gains are spared) with a step_where predicate override. Optimizer steps are pure field algebra: identical runs are bit-identical, and fields from recorded_gradients drive the same trajectory as the engine's backward, held by tests.
• Network::update_each: the identity-aware update — the rule receives the parameter's Value besides its payloads, so per-parameter policy (selective decay, clipping, logging) reads the parameter's symbol, shape, or rank at the call site. update now accepts FnMut and delegates to it.

Changed#

• Field::scale takes a single-value factor and spreads it to each entry's shape through broadcast_to-style broadcasting — the scalar arithmetic optimizer state needs. Scalar fields scale exactly as before; tensor fields gain the case that previously panicked. The factor is now passed by reference, and the method requires the Tensorial payload contract.

• Network::differentiate(loss, wrt): reverse-mode differentiation as a tape-to-tape transform. Gradients record as ordinary computed nodes — compilable, emittable, readable, and differentiable again (higher-order derivatives work by re-application; relu Hessians are exact zeros). The transform runs the engine's own derivative rules over a recording payload, so derivative knowledge cannot fork, and it mirrors the engine scan's seed and accumulation order: a compiled plan over [loss, gradients...] reproduces Evaluation::backward bitwise, held by per-variant closure tests.

• Evaluation::recorded_gradients: assembles the update direction from recorded gradient values — the bridge from differentiate to Network::update, so a training step is one forward run of a compiled [loss, gradients...] plan with no backward pass. The makemore_mlp_compiled example is that loop: bit-identical to makemore_mlp under matched seeds (the closure suite pins the routes bitwise), at speed parity and a measurably lower memory peak, because forward-only liveness frees what the gradient computation no longer needs.

• Activation::gain and init::scaled: the principled link between a layer's nonlinearity and its initialization. Each activation states the standard factor by which it shrinks a unit-variance signal, and the gain-parameterized fan initializer compensates it — init::scaled(seed, activation.gain()) is the general form behind the named classics, which stay frozen (kaiming is the relu gain; seeded outputs never change).

• Activation::Sigmoid, Activation::LeakyRelu, and Activation::Elu, with the public Activation::express that records each variant's expression: the new three are short compositions with stable spellings — sigmoid through the fused tanh, leaky relu and ELU through maximum with correct subgradients at zero and no overflow at finite extremes — and their gradients are the chain rule, closed under differentiate like every composition.

• Value::step, Value::fold, and Value::scatter: the three adjoints that close the op set under differentiation (the maximum family's locally constant mask, unfold's adjoint, and gather's), each with its StableHLO lowering — step as compare plus select, scatter and fold as contractions — and emission conformance coverage, including a differentiated module in the E2 shape verified against the reference interpreter.

• Value::logsumexp as a fused operation: the max-shifted reduction is finite for every finite operand, with the softmax as its gradient, replacing the composition over log_softmax that returned inf once finite logits differed by more than the representable range. It lowers to StableHLO as its shift-form decomposition and joins the emission conformance suite.

Changed#

• cross_entropy composes the expanded form ((targets.sum_along(1) * logsumexp(logits)).sum() - (targets * logits).sum()) / targets.sum(): exact mathematics, and no term can evaluate 0 * -inf into NaN for finite extreme logits. The targets' domain (finite, nonnegative, positive total mass) is now documented. Loss values may differ from 0.8.0 in the last bits, as any re-associated float expression may.

Fixed#

• The 0.7.0 deep-audit invariant batch: plans take one snapshot for validation and execution and reject a shorter sibling that does not contain their graph prefix; scalar payloads reject recorded shapes they cannot carry, backward checks the recorded target shape besides the payload's, and debug runs assert every rule's output shape against the recorded column; counted, selection, and the private constructors prove the tensor invariant at construction; scatter validates the adjoint contract instead of silently discarding gradient rows; a single-window unfold no longer overflows its unused stride in debug builds; the backend seams check the length of every Elementary::map/gemm answer; and the CUDA pool loads cudaFree, frees above-cap buffers on return, returns buffers through an RAII loan on every error path, and caps its parked bytes.

0.8.0 - 2026-08-05#

Added#

• StableHLO emission, the crate's first exit to the XLA world: Plan::emit_stablehlo serializes a forward plan as a textual StableHLO module — parameters then inputs as @main's arguments, the readable set as the result list, leaves as dense constants. Lowering is near-1:1 over the whole op set; the fused log_softmax decomposes into its stable shift form, the one-hot gather becomes a dot_general against the selection (which crosses the boundary as its dense matrix), and unfold lowers to a static gather as a documented completeness fallback. Matched window-GEMM fusion groups raise to stablehlo.convolution — the pattern library earning twice, fused executor at home and the richer op abroad. A typed builder owns every fragment of MLIR syntax; nothing heavier than string building enters the crate.
• Emission conformance, two tiers riding external toolchains the crate never links: TOPOS_STABLEHLO_VALIDATOR names a parser and TOPOS_STABLEHLO_EVALUATOR an executor (scripts under tools/ serve both from any Python with jax), and the suite's round-trip and execution tests check every emitted module against the plan's own results, passing vacuously without a toolchain. Verified beyond the reference interpreter on real backends: compiled XLA-CPU runs the emitted batch-8 CNN probe eleven times faster than the plan (0.24 against 2.6 ms), and Apple's experimental jax-metal plugin runs all five conformance modules on the GPU within the oracle envelope. Numbers and readings in ACCELERATION.md.
• broadcast_to and broadcast_pair: explicit broadcasting under the right-aligned NumPy rule as composites over the named expansions — the target shape is always written, never inferred by an operator, and the gradient is the chain rule over the existing adjoints.
• concat and stack, the designed route: concat sums each value zero-padded to the combined extent at its offset (each operand's gradient is its own narrow window back), stack lifts through unsqueeze. Consumer-shaped tests close the transformer rung's other gaps by composition: masked axis-aware softmax is a broadcast additive mask before the existing axis softmax, and multi-head attention is a loop of rank-2 heads joined by concat — no batched matmul.
• The makemore_transformer example — the attention act: a one-block pre-norm transformer over eight characters of context. The batch packs its samples into one token row so each head's attention is a single rank-2 matmul pair under a block-diagonal causal mask (the sequence-packing idiom); heads join through concat, prediction rows come back through a one-hot gather, and RmsNorm feeds both residual branches. Mean minibatch loss 2.205 against the MLP act's 2.2450, on a 179-node tape, 5000 steps in 12 s.
• Elementwise map kernels on the metal backend: exp, ln, sqrt, and tanh as one-thread-per-element GPU kernels in the same compiled library, pooled buffers, and poison contract as the GEMM path. Measured on the M1 Pro, the GPU passes the scalar path near 128k elements and vForce near 512k (2.7 against 1.2 Gelem/s at 8M), so the map chain runs Metal first — the reverse of the GEMM order — with a size gate that adapts to whether accelerate is compiled behind it.
• The broadcast bench group, measuring elementwise operations over broadcast views against their materialized twins.

Changed#

• Broadcast views compute at slice speed: binary elementwise operations walk same-shape dense operands by innermost runs (unit stride as a slice, zero stride held for the run), and elementwise maps over a broadcast view transform only the distinct elements and keep the layout — a view in, a view out, and the backend seam reads the contiguous window. Bias-style adds over 2M elements went from 0.17 to 7.6 Gelem/s; a transcendental over a broadcast row computes its 1k distinct elements instead of 2M.
• Reshapes that only insert or remove extent-1 axes keep strided views as views, so a multi-axis broadcast_to records no intermediate copy: squeeze and unsqueeze of a broadcast view are layout edits, not materializations.

Fixed#

• A transposed view's elementwise map could reach the backend seam through the new window path (its window is exactly as wide as its volume), silently replacing the documented bitwise scalar fallback for non-contiguous views under accelerate. The window path now requires a strictly narrower window: only broadcast views, which compute fewer elements, earn staying views.

0.7.0 - 2026-08-04#

Added#

• The makemore_mlp_batchnorm example — makemore's third act: the character MLP with its hidden preactivation batch-normalized before the tanh, the hidden bias retired in favor of the learned shift, running statistics maintained in the loop from the batch statistics the training plan's keep-set exposes, and the single-row sampling twin fed those estimates per draw. Final loss matches the plain MLP at this shallow depth, as the lecture it follows predicts: the norm buys robustness, not loss.

• Window-GEMM fusion, the plan tier's first pattern: plans recognize the canonical im2col chain feeding a matmul and execute it as one Tensorial::windowed_product call, never materializing the chain. Matching is structural and provenance-blind, keep-set nodes are fusion barriers, and fusion follows the plan's memory posture — forward-only plans always fuse, compact training plans fuse (backward rebuilds patches with one windowed_patches fast fill, bit-identically), and the default retain-all training plan stays unfused, because per-step patch re-allocation in backward measured as a peak-RSS regression on the deeper consumer. Profile-driven: the CIFAR-10 step was ~50% strided-view iteration and materialization, under 2% elementwise arithmetic — which also retired the planned elementwise-chain and MaxAlong fusions as worthless. The MNIST example's compact training dropped from 114.6 to 106.8 ms/step at unchanged memory, with byte-identical output.

• Tensorial::windowed_product and windowed_patches: the im2col product and its patch-matrix half as payload calls, with composed defaults that are the bitwise references and a Tensor fast path that fills patches in contiguous runs instead of the general odometer walk. The descriptors are the method arguments, so payloads and backends never see graph structure.

• Rematerialization, opt-in via compile_training_compact: the plan drops its large intermediates (im2col patches, padded copies, pooling lanes — the allocator's page-returning size class) right after their last forward consumer, and backward recomputes them on demand, memoized with prompt eviction and bit-identical gradients. The trade is explicit because it does not always win — measured at 9% less peak RSS for 22% more step time on the MNIST example, but negative on the deeper CIFAR-10 stack, where gradient cotangent buffers dominate; the default compile_training stays retain-all. describe reports the drop set and the remat peak either way.

• The retention contract: every operation now declares which payload values its derivative rule reads (both operands for mul and matmul, its own output for tanh and log_softmax, the selection for gather, nothing for the view family — whose backwards read shapes that placeholders answer). Training plans use it to compute and report their memory floor — on the MNIST convnet, 3.3M of 12.3M elements (3.75x) is releasable with gradients still bit-identical, which tests prove by forcing the releases. Training runs do not execute the releases by default: A/B measurement showed per-step mid-run freeing regresses peak RSS under the system allocator (fragmentation), so the floor awaits rematerialization or arena reuse — while forward-only plans keep executing theirs, where the win is measured.

• Plan, Network::compile, and Network::compile_training: the first lowering tier. A plan is a compiled execution schedule — dead-node elimination against declared targets, a keep-set that alone answers reads, and (for forward-only plans) buffer liveness that frees every intermediate after its last consumer. Plan runs are bit-identical to the interpreter's, survive every update generation (compile once, train forever), and refuse backward unless compiled for training, so freed buffers can never leak into gradients. Plan::describe renders the schedule: per-node liveness spans and the static peak-live-volume estimate. The MNIST example runs on plans — compile-once training plus a forward-only probe whose liveness cuts its live volume 6.8x (28M of 191M elements) and the process peak RSS by 31%, with byte-identical output.

• The cifar10 example: a three-stage VGG-style convnet on real 32x32 color images, the plan tier's pressure consumer. One training plan serves all 2000 generations, and the forward-only probe plan holds the 500-image accuracy probe's live volume 8.8x below retain-all. Reaches 65.2% test accuracy (chance is 10%) in about 13 CPU minutes at 392 ms/step; downloads and caches the binary archive on first run.

• Tensorial::unfold and Tensorial::fold: single-axis sliding windows (torch semantics, with a dilation parameter) as a strided view over the shared buffer, and their adjoint — each source position sums its own window contributions in window order, so folding is deterministic under any evaluation strategy. The substrate for convolution and pooling. Breaking for custom payload implementations, which must add both methods.

• Value::pad and Value::unfold: the corresponding recorded operations. pad places a value inside zeros along one axis and is narrow's adjoint (each is the other's gradient rule); unfold records the sliding-window view with fold as its gradient, so overlapping windows accumulate correctly.

• conv2d and the Conv2d layer: 2-D convolution as a composed formula — padding, two unfolds, and an im2col reshape feeding one rank-2 matmul on the accelerated GEMM path — with stride and symmetric zero padding, torch-shaped weights, and the gradient from the chain rule alone.

• max_pool and average_pool: spatial pooling over the same window view; the maximum folds with the left-biased binary maximum, so ties route deterministically to the earliest window position.

• The mnist example: a LeNet-style convolutional network trained on MNIST through the composed convolution and pooling formulas — the convolution rung's first consumer. It downloads and caches the IDX files on first run and reports test accuracy, per-step time, and the loss chart.

• Network::forward_for: the target-sliced run — it evaluates only the ancestors of the declared targets, leaving every skipped slot an O(1) shape-correct placeholder that of and backward refuse to answer with, so skipped reads fail loudly. Sliced gradients drive update soundly (a parameter outside the closure receives its true gradient, zero), and results are bit-identical to full runs. With the training and evaluation expressions sharing one tape, the MNIST example dropped from 517 to 95 ms per step (5.4x) with an unchanged 98.22% test accuracy. Every example loop now slices its runs the same way; the makemore family reproduced byte-identical output after the switch.

• BatchNorm: batch normalization at tensor granularity over [batch, features] values. express records the training mode — normalization by the batch's own mean and biased variance — and returns a Normalization carrying the output and the statistic values; express_with records the inference mode over statistics supplied as values, fed per run, so running estimates live with the training loop rather than on the tape.

• LayerNorm and RmsNorm: the stateless normalization siblings, taking per-sample statistics along the feature axis — full standardization with a per-feature affine, and root-mean-square re-scaling with a per-feature scale, respectively. No running estimates and no training/inference split: one recorded expression serves both. All three norms share one epsilon contract: a single-value constant broadcast in-graph to the variance's shape.

• Value::mean_along: the mean-reduction composite, sum_along divided by the reduced axis's extent.

• Differentiable::counted: the shape-derived constant constructor — a payload of a given shape holding an integer count — that lets composed formulas mint axis extents as payloads. Breaking for custom payload implementations, which must add the method.

• The cuda feature: large dense f32/f64 products through cuBLAS on an NVIDIA GPU, Linux only. The libraries (libcudart/libcublas) are bound at run time by dlopen, so the build never links them and a machine without the toolkit or a device declines at run time; typed setup errors make the GPU tests skip only in those two environments and fail loudly on any other defect. Backend::Cuda joins the diagnostics enum between Metal and Simd. Built blind against the documented APIs and not yet validated on NVIDIA hardware: treat it as experimental until the first measured run, which will also tune its provisional flop threshold.

0.6.0 - 2026-08-02#

Added#

• The simd feature: a portable CPU acceleration backend over the matrixmultiply crate's tuned, single-threaded microkernels with runtime instruction-set dispatch (AVX-512F, AVX2+FMA, AVX, NEON). It accelerates dense f32 and f64 products on every platform — the acceleration story for Linux — and sits last in the chain on macOS. Backend::Simd joins the diagnostics enum, and the ubuntu CI job now executes the backend grid it used to only compile.

0.5.4 - 2026-08-02#

Fixed#

• backward no longer runs the derivative rules of operands used only as shape or index data (a broadcast's reference, a gather's selection): a None cotangent no longer marks its operand as an ancestor, so a singular expression behind such a reference cannot leak NaN into unrelated gradients (audit finding PG-001).
• narrow rejects zero-length windows at both the recording and payload boundaries instead of manufacturing the empty tensors the payload forbids by construction (audit finding PG-002).
• narrow and pad compute their window ends with checked arithmetic, so an overflowing start + len fails identically in debug and release builds instead of wrapping past the range check in release (audit finding PG-005).
• The Metal test grid and the backend status test skip only when the machine has no Metal device; every other setup failure — a shader that does not compile, a missing kernel, a rejected pipeline — is now a hard test failure instead of a silent skip (audit finding PG-006).
• cargo test no longer executes the Criterion benchmarks: the bench targets set test = false and CI names its test targets instead of --all-targets (which implies --benches and forces them regardless), with doctests run explicitly (audit finding PG-007).
• Documentation drift: the README's unsafe-code claim now names the default build, the Tensor storage list includes the one-hot selection, operand links are attributed to the tape's operand column, and the accelerate module no longer claims to be the only unsafe code in every build.

Added#

• Add dense-payload twins of the tensor-regression run benches: the existing cases build their payloads with Tensor::filled, which is constant storage and bypasses the dense matmul and slice paths, so the new cases are the ones that price the accelerated tiers.

Changed#

• Shrink the metal kernel's staging depth (BK 16 to 8) with a banded epilogue: six threadgroups stay resident per core instead of three, measured worth a few percent (~1.45 TFLOP/s at 2048-square); wider 64x128 tiles measured no better and were not kept.

0.5.3 - 2026-08-02#

Changed#

• Give the elementwise paths slice fast lanes: map and zip over contiguous dense buffers (and dense-with-constant pairs) run straight over slices instead of the per-element iterator dispatch, which measured 40x below memory speed. Dense multiplies went from 235 Melem/s to 5.7 Gelem/s and the gradient seed's constant-plus-dense add to 12.8 Gelem/s on an M1 Pro; a wide accelerated training step dropped from 112 ms to 19 ms. Every lane hands the combiner the same pairs in the same order, so results stay bit-identical across lanes, pinned by a test.

0.5.2 - 2026-08-02#

Documentation#

• Add ACCELERATION.md: what each build supports, how routing and determinism work, the safety layering, the seam for payload authors, and every measured number; the README's acceleration section moves up beside the design bet and shrinks to the claim, the one command, and a pointer.

Fixed#

• Skip the metal GPU tests on machines without a Metal device (the virtualized CI runners), reporting the skip instead of failing: the backend already declines cleanly there, and the tests now honor the same contract.

0.5.1 - 2026-08-02#

Added#

• Add the elementwise acceleration seam: Elementary::map offers a whole-buffer transcendental (MapOperation: exp, ln, sqrt, tanh) to the backend chain, and the tensor's elementwise operations consult it for contiguous dense buffers before the scalar path. The accelerate feature answers through vForce's vectorized transcendentals; measured on an M1 Pro, a wide training step dropped from 145 ms to 112 ms — the scalar tanh wall — with strided views and small buffers keeping the scalar path bit-for-bit.

Changed#

• Specialize the metal backend's pipelines per shape: the tiled kernel's dimensions and strides bake as Metal function constants, one cached pipeline per recurring shape (record-once training replays a handful), with the generic params-driven pipeline as the fallback past the cache cap.
• Raise the metal kernel's occupancy threefold: a GPU-counter trace showed the dedicated output-staging tile capping compute occupancy at one resident threadgroup per core, so the epilogue now reuses the operand staging area as a half-tile buffer in two coalesced passes, cutting the threadgroup footprint from 26.9 KB to 9.5 KB. Together with the per-shape pipelines, measured on an M1 Pro at 2048-square: 534 to about 1400 GFLOP/s.

0.5.0 - 2026-08-02#

Added#

• Add the metal feature: large dense f32 products (Metal has no f64) run on the GPU through hand-written simdgroup-matrix kernels — no MPS, no vendor library — compiled from source at first use, with shared-mode buffers from a size-classed pool on unified memory. The kernels read operands through the task's strides, so transposed, narrowed, and broadcast views pass through without copies. Accelerate leads the chain where both features are compiled (it measured ahead at every size), so Metal serves the stride patterns BLAS declines and everything large in metal-only builds — about twenty times the built-in slice path. A failed setup or runtime error poisons the backend into declining forever, degrading to slow, never to wrong; Backend::Metal.status() reports readiness, doubling as warmup for the one-time kernel compilation.
• Add the throughput example: the acceleration ladder measured on a wide dense model — the raw 2048-square product and whole training steps — with the dimensions shrinking eightfold when no backend is compiled in so the run still terminates.
• Add init::Sample and make the initializer factories element-generic: uniform, normal, xavier, and kaiming now produce Tensor<Element> for any element implementing Sample, with the element inferred from the network the closure feeds. The generator pipeline stays in f64 and converts once at the end, so the f64 path is bit-identical to every previous release (pinned by a golden-bits test) and the f32 path is the same stream rounded once per element. Context-free factory calls bound to nothing now need a type annotation.

Changed#

• Move the tensor examples (mlp_xor and the makemore family) to Tensor<f32>: the field's training dtype, and the one every acceleration rung favors. The scalar examples and the crate-root doctest stay f64, and f64 tensors remain fully supported and tested. The facade example still trains bit-identically to its hand-rolled twin from matching seeds.

0.4.0 - 2026-08-02#

Added#

• Add the accelerate feature, the backend chain's first resident: dense f32/f64 matrix products above a small flop threshold route to Apple's Accelerate framework (cblas_sgemm/cblas_dgemm — the AMX/SME matrix units on Apple Silicon, AVX kernels on Intel Macs), with transposed and narrowed views mapping to BLAS transpose flags and leading dimensions without copies; stride patterns BLAS cannot express and small tasks decline to the built-in paths. macOS only, zero dependencies, and a safe stub elsewhere. The default build is untouched and keeps #![forbid(unsafe_code)]; with the feature on, unsafe is confined to the backend module under a crate-wide deny.
• Add Backend and BackendUnavailable: the backend diagnostics surface, present in every build so no user code ever needs a cfg — Backend::ALL lists the defined backends in chain order and Backend::status reports Ok, NotCompiled, PlatformUnsupported, or a setup/poison reason.

0.3.1 - 2026-08-02#

Added#

• Add the acceleration seam: GemmTask describes one dense matrix-multiplication job (spanning slices plus per-axis strides, so transposed and narrowed views pass through unmaterialized), and the provided Elementary::gemm offers each task to the compiled backend chain before the built-in paths compute. The chain is empty until the first backend feature lands, so behavior and results are unchanged; custom payload implementations keep the default.

0.3.0 - 2026-08-02#

Added#

• Back Tensor with a strided layout over an extensible Storage representation (a shared dense buffer or a non-allocating constant), so transpose and the broadcasts are O(1) views instead of copies and the backward gradient seed no longer allocates a zeroed buffer per node.
• Add the view operations Value::reshape and Value::permute (with the reshape-based conveniences squeeze and unsqueeze), each a differentiable graph node whose gradient routes back by the inverse view. permute generalizes transpose to any rank.
• Add Value::narrow (a slice window along one axis): the forward is an O(1) view and the gradient scatters back into the excluded positions as zeros.
• Add Value::gather and Tensor::selection: an embedding-style row lookup, table.gather(selection), where selection is a one-hot [count, vocab] input stored as its usize indices. The gradient scatter-adds into the table only (repeated rows accumulate); the selection is data and takes no gradient.
• Add Value::log_softmax, a fused, numerically stable log-softmax along a named axis (the max-shifted forward cannot be composed from recorded operations), and cross_entropy, the classification loss composed on top of it, normalizing by the targets' total mass — the batch size for one-hot targets.
• Add the elementwise operations Value::sqrt, Value::powf, Value::maximum, and Value::relu, and Activation::Relu for layers and neurons. The Elementary payload contract gains sqrt, maximum, and the 0/1 indicator step; Tensorial gains the max_along reduction.
• Add the composite expressions Value::abs, Value::softmax, and Value::logsumexp — formulas recorded as several primitive nodes, with the softmax pair composed stably on top of the fused log-softmax core — collected in a dedicated composition tier beside the single-node opcode methods.
• Add the init module: deterministic initializer factories (uniform, normal, and the fan-aware xavier and kaiming, which scale rank-2 weights from the requested shape and zero rank-1 biases) matching the shape-to-payload closures Layer and Mlp take. Every factory is seeded explicitly and owns its generator state, so initialization is reproducible without a rand dependency.
• Add the makemore_bigram example: a character-level bigram language model over names — a [vocab, vocab] logit table read by gather, scored by cross_entropy on per-run one-hot minibatches, and sampled through the composite softmax.
• Add the makemore_mlp example: the Bengio-style character-level MLP — a three-character context embedded by gather, flattened by reshape, and squashed through a hand-rolled tanh hidden layer, with a single-row twin expression of the same parameters recorded for sampling since input shapes are baked in at recording time.
• Add the makemore_mlp_facade example: the same model on the Mlp facade, training bit-identically to makemore_mlp from matching seeds. The makemore examples live in examples/makemore/ (declared as explicit example targets) and share their corpus machinery and dataset there.
• Add the makemore_mlp_parallel example: the same model trained data parallel — every step fans shard-shaped forward and backward runs across rayon's threads against the shared network, sums the gradient fields in a deterministic pairwise tree, and averages, computing the full-batch gradient exactly while cutting the wall clock several-fold.
• Add the makemore_embedding_map example: the MLP with a two-dimensional character embedding, rendered in the terminal before and after training by a small reusable labeled scatter chart (examples/makemore/chart.rs) whose marks are the letters themselves.
• Add the gemm benchmark group: the dense matmul path measured across sizes, element types, and transposed operands, reported in elements per second — one element per floating-point operation.

Changed#

• Accept impl Into<Shape> in Tensor::new, Tensor::filled, and Value::reshape: axis literals keep working unchanged, and a Shape or its reference now passes directly instead of being decomposed into an axis iterator. Other iterator sources go through Shape::new.
• Use plain verbs consistently for operations: rename Field::scaled to Field::scale, Network::updated to Network::update, Tensorial::{transposed, permuted, narrowed, padded} to Tensorial::{transpose, permute, narrow, pad} and Value::{transposed, permuted, squeezed, unsqueezed} to Value::{transpose, permute, squeeze, unsqueeze}; align the internal tape, layout, storage, and test helpers with the same rule.
• Make Gradients an alias for Field rather than a wrapper around it. Evaluation::backward still returns Gradients, but the result is a field directly, so Network::update and the field algebra take it without a conversion.
• Read tensor elements through Tensor::iter (logical row-major order), Tensor::as_slice (a borrowed slice when contiguous), or Tensor::to_vec, and compare tensors by logical value across storage representations.
• Multiply dense matrices on a slice path: matmul now reads dense rank-2 operands — including transposed, narrowed, and broadcast views — through their layout strides instead of per-element logical access, in loops shaped for the compiler's auto-vectorizer. The per-element accumulation order is unchanged (seeded from the first term), so results are bit-identical to the logical path, which non-dense storages keep. Measured on an Apple M1 Pro: 26 GFLOP/s f32 and 13 GFLOP/s f64 for square products, from 0.41 before.

Removed#

• Remove Gradients::as_field and Gradients::into_field. Pass the gradients themselves instead: network.update(&gradients, ..).
• Remove Tensor::elements; use iter, as_slice, or to_vec instead.

0.2.0 - 2026-07-27#

Added#

• Add declared inputs and per-run payload binding through Network::input and Network::forward_with, allowing one recorded graph to evaluate different samples concurrently.
• Add Value::sum_along and Value::broadcast_along for explicit axis-wise tensor reductions and broadcasting.
• Add the tensor-native Mlp facade and an end-to-end XOR training example.
• Add Criterion benchmarks for recording, execution, training, scaling, and memory behavior.
• Add continuous integration checks and finite-difference gradient tests.

Changed#

• Rebuild Layer at tensor granularity. Layer::new now accepts weight and bias payloads, and Layer::express accepts and returns one batched tensor value instead of slices and vectors of scalar values.
• Store parameter payloads per network generation so updates take O(parameters) work while preserving older generations.
• Track fork ancestry in symbols and fields so divergent network branches are rejected reliably.
• Restrict backward passes to scalar targets and to the target's ancestors.
• Reorganize internals into engine, neural, and payload modules while retaining the crate-root public exports.

Fixed#

• Restore operand shapes when differentiating broadcasts.
• Reject parameter updates whose payload shape changes.
• Reject empty tensors and detect tensor-volume overflow.

Documentation#

• Expand the README and crate documentation around concurrency, inputs, generations, tensor operations, layers, and MLPs.
• Add the project logo and refresh the terminology guide.

0.1.0 - 2026-07-26#

• Initial release.