research notes

why Ember froze KV state into a verified artifact, proved exact same-model replay, and added reconstruction and causal diagnostics before fitting a Qwen-to-Qwen ridge mapper.

a frozen baseline showing how Ember's current K-quant loader expands smaller Q6_K and Q4_K_M files into resident f32 weights, making them use more memory and decode more slowly than Q8_0.

a thread-scaling experiment across three model sizes and a synthetic matmul sweep. the crossover lives around 25 million FLOPs per operation.

a frozen-feature control before targeted LoRA: development-ranked K=4 layer groups carry stronger held-out Arabic POS signal than five non-overlapping random groups in Qwen3-0.6B and Llama-3.2-1B.

why benchmark reproducibility has to include item manifests, prompt templates, rendered prompts, parser regimes, and scoring artifacts, not only model names and final scores.

from the first regex prototype to Atlas 1.0.0: why the extraction layer stays deterministic, traceable, and deliberately narrower than generated research-gap claims.

a small ArabicMMLU robustness audit showing that prompt formatting and option-label position can affect measured multiple-choice accuracy in small decoder-only models.

this week Ember's Arabic morphology probing shifted from hidden-state extraction into leakage-aware measurement. the result is narrower: POS survives stricter heldout evaluation in Qwen3-0.6B and Llama-3.2-1B, while root, lemma, and pattern need a different evaluation framework.

preliminary layerwise probe results across LLaMA, Qwen, and Gemma on Arabic nonce root-pattern stimuli. root labels vary across final layers; pattern labels saturate and should not be used for scale or family claims in this run.

probing LLaMA 3.2 1B/3B/8B with linear classifiers, CCA, and RSA. the representations are there — structured, disentangled, scaling non-monotonically — but every model outputs "The". findings, charts, and next steps.

what i learned reading arabic nlp papers for a week. the 2026 paper that broke the assumption, and why the real question is about internal representations, not tokenization.

research plan: use ember for activation probing to find where and how models learn Arabic root-pattern morphology internally. builds on the open question from Alakeel et al. (2026).

alakeel, qwaider, aldarmaki, alqahtani · LREC 2026. token-morpheme alignment doesn't predict morphological generation in Arabic LLMs. GPT-4o scores 97% on nonce roots with terrible tokenizer alignment.

attia · ~2007. a finite-state, modular Arabic tokenizer with clitic guesser, morphological analyzer, and cashida-based disambiguation. the guess-and-filter architecture.

alkaoud & syed · WANLP 2020. morphology-aware tokenization at the embedding layer: 60% smaller vocab, better OOV handling, SOTA without retraining. worked at Word2Vec/mBERT scale.