Before Mapping KV Caches, Make Them Measurable

summary

i started this work with a tempting next step: take a KV cache from one model, learn a small linear map, and test whether another model can continue from it without repeating the full prefix.

i did not build the mapper.

instead, i spent the day making KV state into something Ember can export, verify, import, replay, compare, and perturb under explicit contracts. the result is a first-class three-file snapshot, exact same-model replay validation, per-layer and per-head K/V metrics, attention-output and logit diagnostics, and a controlled in-memory altered-state path.

that is less exciting than a transfer demo. it is also the work that makes a later transfer result interpretable.

why the mapper had to wait

a mapped cache can fail for several unrelated reasons. the snapshot may be malformed. the token prefix may not match. the cache cursor may be off by one. RoPE or Q/K normalization may be interpreted in the wrong coordinate system. import may change f16 bits. the model may take a different execution path. or the learned map may simply be bad.

if all of those failures collapse into one final generated string, a negative result says almost nothing. even a positive-looking string is weak evidence: it can hide numerical drift, a lucky top-1 tie, or a replay path that never used the intended state.

so the first question became narrower: can Ember freeze native KV state and prove that importing it changes nothing under a strict same-model contract?

only after that comes the second question: when state is deliberately changed, where does the difference first appear—in K/V reconstruction, the target attention output, the logits, or the greedy trajectory?

what became first-class

the new artifact is independently versioned as ember.kv-snapshot.v1. it does not modify Ember's frozen execution-plan or experiment-bundle schemas.

snapshot/
├── manifest.json
├── keys.f16le
└── values.f16le

K and V are stored as compact f16 bit patterns in [layer][kv_head][position][dimension] order. unused cache capacity is not serialized. the manifest records the model and tokenizer hashes, cache geometry, RoPE layout and theta, Q/K normalization semantics, value state, execution mode and fingerprint, prefix-token hash, resume token, and payload hashes.

import is intentionally strict. there is no “close enough” switch. a mismatch in model identity, tokenizer, geometry, position semantics, representation state, or numerical execution provenance is rejected before decode.

ember kv export --model MODEL.gguf --tokenizer TOKENIZER.json   --arch qwen3 --prompt 'prefix' --output runs/kv/prefix
ember kv verify runs/kv/prefix
ember kv replay --snapshot runs/kv/prefix   --model MODEL.gguf --tokenizer TOKENIZER.json --arch qwen3

the off-by-one that has to stay visible

a snapshot covers the completed prefix, but the token selected from the prefix-boundary logits is not yet in the cache. if the prefix length is P, the stored resume token belongs at absolute position P.

the first replay forward evaluates that token, appends its K/V row, and produces logits for position P+1. feeding the last prompt token again would duplicate it. pretending that the snapshot contains the original boundary logits would also be wrong.

this sounds like bookkeeping. it is exactly the kind of bookkeeping that can make a transfer experiment look broken when the actual bug is sequence alignment.

how exact replay was checked

the process-level validation did not compare decoded text or use an allclose tolerance. it saved full f32 selection logits from uninterrupted generation, the export boundary, and replay:

native[N,V] == concatenate(export_boundary[1,V], replay[N-1,V])

equality here means exact f32 bit equality. the completed matrix covered Llama-3.2-1B and Qwen2.5-1.5B, Q8_0/Q6_K/Q4_K_M, and fixed English and Arabic prompts.

this establishes the tested snapshot/replay boundary. it does not establish reference correctness for every model, and the recorded phase timings are observational rather than a speedup claim.

measurement before mapping

exact replay is the control condition. the next addition was ember kv compare, which measures two snapshots only when their target and prefix coordinates are proven to align.

for every layer and KV head, it reports K and V:

• cosine similarity,
• mean squared error,
• optional directional R²,
• maximum absolute error,
• and f16 bit-mismatch counts.

optional thresholds produce a deterministic failure list and first exceedance. the machine-readable report contains no timing, host, process ID, or unordered map fields.

ember kv compare LEFT RIGHT --json --r2   --max-abs 0.001 --min-cosine 0.999

with the exact target model loaded, the same command can feed a shared reference-greedy token path into both caches. it then compares the semantic attention O-projection output at every layer, the full logits, top-1 agreement, and the first prediction disagreement.

a separate phase re-imports two clean caches and lets each branch follow its own greedy predictions. that gives sequence agreement and first divergence without treating post-divergence activations as if they still had the same input.

a causal control without fake mapper provenance

i also needed to know that the diagnostic path could detect a real cache change before giving it mapper output. the narrow control zeroes or scales one selected K/V head across the initialized prefix in memory.

the operation has a typed receipt naming the native source snapshot, layer, head, K/V selection, exact scale bits, and affected element counts. it cannot be saved as ember.kv-snapshot.v1 and it does not fill the reserved mapper hash with a placeholder. ordinary replay still accepts native snapshots only.

on a Qwen2.5-1.5B Q6_K planned smoke, comparing a snapshot with itself produced zero paired differences, top-1 agreement, and the same short greedy sequence. zeroing K and V for layer 0, head 0 produced a layer-0 attention-output cosine of about 0.886 and a final-logit cosine of about 0.9953. top-1 still agreed over the very short horizon.

what the current evidence establishes

• the tested native snapshot boundary preserves active f16 KV bits and the restored cursor,
• same-model continuation logits are bit-exact across the completed process matrix,
• comparison failures can be localized to a specific layer, head, and K/V component,
• and a known in-memory cache edit can be measured through attention output and logits, with short-horizon behavior reported separately.

those are instrument-validity results. they make a later mapper experiment easier to diagnose; they are not mapper evidence.

what this work does not prove

• no ridge or neural mapper was fitted,
• no cross-model cache was authored or replayed,
• no learned transform improved held-out reconstruction,
• no attention or generation retention claim was established,
• no latency benefit was measured under a controlled benchmark,
• and Gemma's heterogeneous cache semantics remain out of scope.

attention-output cosine is useful, but it is not an attention probability and not a causal explanation by itself. K/V R² is a reconstruction statistic, not proof that the target model can use the reconstructed cache. greedy agreement over a short horizon is behavior, but it is still a censored and prompt-dependent measure.

the deliberately small next experiment

a ridge pilot can happen later, and it should stay smaller than the infrastructure built around it.

the current proposed scope is one fixed Qwen-to-Qwen pair with the same tokenizer and matched KV geometry. keys would be mapped in the declared post-normalization, pre-RoPE content space; values would be mapped in their declared value state. each target layer/head would get its own fixed closed-form ridge map, with a frozen calibration split, one lambda, and no layer search.

the pilot should compare identity and ridge reconstruction first, then use the existing same-input attention/logit and independent greedy diagnostics. it should not generalize architectures, tune a serving path, optimize kernels, or create a production transformed snapshot format.

if the local files remain the only available Qwen pair, Q4_K_M to Q8_0 would be a quantization-boundary pilot on the same base model, not evidence for cross-size model transfer. that distinction needs to stay in the title, artifact metadata, and conclusion.

bottom line

today did not produce a mapper. it produced the conditions under which a mapper can fail honestly.

native state now has a strict identity. replay has a bit-exact control. reconstruction has layer/head metrics. downstream effects have same-input attention and logit measurements. behavioral drift has an independent greedy trajectory. altered state has explicit provenance without pretending to be a learned artifact.

that is the boring boundary i want before asking whether one Qwen cache can stand in for another.