hook contract

hooks describe semantic stages, not interchangeable implementation steps. ember invokes them in a fixed order, but each model family reaches that boundary through its own numerically explicit path.

lifecycle

a successful ordinary generation run has one model-load callback, one prefill callback, one hooked prefill evaluation, zero or more hooked decode evaluations, and one completion callback.

the six tensor-bearing hooks repeat first with phase=Prefill, then with phase=Decode for each single-token evaluation. layer indices are zero-based and increase in model execution order.

trait surface

only name is required. every lifecycle method defaults to a successful no-op. a built-in experiment implements only the stages it needs.

pub trait Experiment: Send {
    fn name(&self) -> &'static str;

    fn on_model_loaded(
        &mut self,
        ctx: &ModelContext<'_>,
    ) -> Result<(), ExperimentError> { Ok(()) }

    fn before_prefill(
        &mut self,
        ctx: &ExecutionContext<'_>,
    ) -> Result<(), ExperimentError> { Ok(()) }

    fn before_layer(
        &mut self,
        ctx: &LayerContext<'_>,
        hidden: &mut TensorAccess<'_>,
    ) -> Result<(), ExperimentError> { Ok(()) }

    fn after_attention(
        &mut self,
        ctx: &LayerContext<'_>,
        attention_output: &mut TensorAccess<'_>,
    ) -> Result<(), ExperimentError> { Ok(()) }

    fn after_mlp(
        &mut self,
        ctx: &LayerContext<'_>,
        mlp_output: &mut TensorAccess<'_>,
    ) -> Result<(), ExperimentError> { Ok(()) }

    fn after_layer(
        &mut self,
        ctx: &LayerContext<'_>,
        hidden: &mut TensorAccess<'_>,
    ) -> Result<(), ExperimentError> { Ok(()) }

    fn before_logits(
        &mut self,
        ctx: &ExecutionContext<'_>,
        hidden: &mut TensorAccess<'_>,
    ) -> Result<(), ExperimentError> { Ok(()) }

    fn after_logits(
        &mut self,
        ctx: &ExecutionContext<'_>,
        logits: &mut TensorAccess<'_>,
    ) -> Result<(), ExperimentError> { Ok(()) }

    fn on_generation_complete(
        &mut self,
        ctx: &GenerationContext<'_>,
    ) -> Result<(), ExperimentError> { Ok(()) }
}

context types

contexts are lightweight immutable values. model strings are borrowed; token buffers, weight handles, full GGUF metadata, caches, and backend internals are not exposed.

execution position

sequence_length is start_position + input_token_count. token_position() returns an absolute position only when the evaluation contains one token, which is the ordinary decode case.

small enums

ModelFamily

Llama, Qwen3, Gemma4

ExecutionPhase

Prefill, Decode

TracingState

Disabled, Enabled

TensorDType

F32 in v0.1

TensorAccess

TensorAccess is a narrow view over an existing contiguous two-dimensional f32 activation. it stores a fixed [rows, columns] shape and borrows the model-owned value slice.

let shape: &[usize; 2] = tensor.shape();
let dtype: TensorDType = tensor.dtype();
let observed: &[f32] = tensor.values();

// explicit intervention on the same allocation:
for value in tensor.values_mut() {
    *value *= 0.5;
}

// or replace existing values with zero:
tensor.zero();

ember does not create an activation copy for a hook. mutation is possible only because these points already own a mutable intermediate. observation is a convention enforced by the experiment implementation: reading values() preserves execution; calling values_mut() or zero() does not.

model-family semantics

the shared names identify genuinely equivalent semantic stages. they do not make the underlying blocks identical.

LLaMA's active single-token hooks operate directly on its preallocated decode workspace. they do not force the generic tensor path. Gemma's packed versus generic MLP dispatch is also unchanged.

disabled path

normal generation never builds an ExperimentRunner. model methods instantiate the zero-sized DisabledHooks adapter, whose methods are #[inline(always)] no-ops. hooked model functions are generic over the adapter type.

pub(crate) struct DisabledHooks;

impl<T, E> LayerHooks<T, E> for DisabledHooks {
    #[inline(always)]
    fn after_attention(
        &mut self,
        _layer_index: usize,
        _tensor: &mut T,
    ) -> Result<(), E> {
        Ok(())
    }
    // the other disabled methods have the same shape.
}

the v0.1 validation inspected optimized code for a representative LLaMA layer and found no remaining disabled experiment dispatch. warmed allocation tests found no per-layer allocation from the absent path, and controlled prefill/decode A/B runs stayed inside measurement noise.

failure propagation

hook implementations return ExperimentError. ExperimentRunner wraps it in ExperimentFailure before ember's normal error path sees it.

experiment 'my-experiment' failed in after_attention
(phase=decode, layer=7): expected a finite activation

hooks are synchronous. a failure stops the current generation; completion is not reported as successful and later hooks do not run.

inspection-surface boundary

normal no-experiment tracing, hidden-state extraction, layer dumps, logits dumps, benchmark reporting, and packed kernel dispatch remain unchanged. structured tracing can be active during an experiment, and its state is reported in the immutable execution context, but hooks do not replace or redefine the trace stream.

source map