← voidwest research notes

A Local Signal for Geometry-Guided Arabic Morphology Adaptation

frozen-feature controls before targeted LoRA
mohammed al-thobaiti · 2026-07-19
Arabic NLP morphology probing LoRA Qwen LLaMA

summary

Development-selected layer groups carried stronger held-out POS signal than the random controls. At K=4, macro-F1 improved by 4.70 points for Qwen and 3.15 for Llama, beating all five random groups in each model. This tests frozen features; LoRA adaptation remains untested.

We fitted deterministic linear classifiers to concatenated frozen states from Qwen3-0.6B and Llama-3.2-1B. The replicated K=4 result motivates a small targeted-LoRA test. Adaptation and causal localization remain open questions.

status

Scope: frozen-feature classification. LoRA adaptation and causal localization of Arabic morphology remain untested.

motivation

Related Arabic forms share roots, lemmas, and surface patterns. Random splits can reward that overlap; grouped splits test transfer across lexical groups.

random layer adaptation is also a weak control unless it is matched. changing the number of layers, target modules, adapter rank, data, or training budget changes capacity or compute along with layer choice.

the contrast i care about is narrower: geometry-selected layers versus non-overlapping random layers, with everything else held fixed.

the local control cannot tell me whether targeted LoRA will work. it can tell me whether the proposed intervention has enough signal to be worth running before i spend money on real training.

experimental setup

models: Qwen3-0.6B and Llama-3.2-1B.
task: POS classification from frozen prompt-final hidden states.
data: 1,416 examples; root-heldout split with seed 17.
split sizes: 991 train, 213 development, 212 held-out test.
selection: linear probes fit on train; layers ranked using development evidence only.
test: held-out reporting only; test performance never entered layer selection.
layer counts: K=1, K=2, and K=4.
random controls: five deterministic random draws per K, sampled after removing the geometry-selected layers.

dataset paths, SHA-256 checksums, split identifiers, and selection provenance were held fixed. future-LoRA manifests also declare the same q_proj/v_proj target group, rank 8, alpha 16, and matched expected adapter counts within each K.

The module metadata is reserved for future manifest parity. sklearn concatenates selected layers’ frozen hidden states and fits a linear classifier; q_proj and v_proj remain untouched.

main results

model	K	geometry test	random mean ± SD	difference	wins	percentile
Qwen3-0.6B	1	83.49%	84.91% ± 1.16	−1.42 pp	1/5	20th
Qwen3-0.6B	2	83.96%	82.83% ± 2.61	+1.13 pp	4/5	80th
Qwen3-0.6B	4	85.85%	81.79% ± 2.32	+4.06 pp	5/5	100th
Llama-3.2-1B	1	87.74%	85.09% ± 2.10	+2.64 pp	5/5	100th
Llama-3.2-1B	2	85.38%	85.28% ± 2.22	+0.09 pp	3/5	60th
Llama-3.2-1B	4	87.26%	84.53% ± 2.03	+2.74 pp	5/5	100th

Grouped bar chart showing K=4 geometry-selected held-out POS accuracy above the mean of five non-overlapping random layer controls for Qwen3-0.6B and Llama-3.2-1B — K=4 held-out POS accuracy. random bars show mean ± population SD across five deterministic, non-overlapping layer draws.

K=4 is positive for both models and beats all five non-overlapping random controls in both cases. macro-F1 improves by 4.70 points for Qwen and 3.15 points for Llama.

the K=1 and K=2 rows matter too: Qwen K=1 fails this control, and Llama K=2 is effectively neutral. the result is not “selected layers always win.” it is a replicated local signal at K=4.

class-level K=4 result

the table reports geometry minus the random-control mean on the held-out test split.

model	class	precision Δ	recall Δ	F1 Δ
Qwen	ADJ	+8.17 pp	+3.81 pp	+6.00 pp
Qwen	NOUN	−0.19 pp	+5.37 pp	+2.81 pp
Qwen	VERB	+8.71 pp	−0.56 pp	+5.29 pp
Llama	ADJ	+0.57 pp	+5.71 pp	+3.02 pp
Llama	NOUN	+1.76 pp	+2.54 pp	+2.21 pp
Llama	VERB	+6.37 pp	0.00 pp	+4.21 pp

Horizontal bars showing positive K=4 F1 differences for ADJ, NOUN, and VERB in both Qwen3-0.6B and Llama-3.2-1B — every POS class improves in F1 at K=4. the aggregate result is not only a majority-class NOUN effect.

interpretation

what the current evidence supports

development-ranked morphology-sensitive layer groups can carry more held-out POS signal than matched random layer groups;
the K=4 direction repeats in Qwen3-0.6B and Llama-3.2-1B;
the K=4 gain is not confined to the majority NOUN class;
the result is strong enough to justify a small targeted-LoRA pilot.

what it does not support yet

that targeted LoRA works;
that these layers causally localize Arabic morphology;
that the effect generalizes to gender, number, root, lemma, pattern, nonce forms, larger models, or other architectures;
that geometry is stable across architectures in a strong theoretical sense.

predeclared next step

the first intervention is already frozen: Qwen3-0.6B, POS, K=4, geometry layers 7, 8, 18, 20, and five fixed non-overlapping random controls. every run targets q_proj/v_proj with rank 8, alpha 16, and exactly 163,840 trainable adapter parameters. the budget is 200 optimizer steps.

the primary metric is held-out POS macro-F1. success requires the geometry condition to exceed the arithmetic mean of the five random controls and strictly beat at least three of them. accuracy and per-class F1 are secondary diagnostics.

before execution, the model and tokenizer revisions, backend, maximum sequence length, batch and accumulation settings, learning rate, optimizer, scheduler, warmup, weight decay, dropout, precision, evaluation cadence, and checkpoint policy must be frozen. early stopping is disabled for the pilot.

limitations

this is frozen-feature evidence only, not a LoRA result;
the task is POS only;
the models are small;
five random draws provide a useful local control distribution, not full inferential uncertainty;
K=4 is a predeclared intervention candidate based on replicated local signal, not proof that K=4 is globally optimal;
the sklearn control is deterministic and does not represent training variance;
real LoRA optimization may not follow the frozen-feature pattern.

conclusion

this is a local proof-of-life. development-ranked K=4 layer groups carry stronger held-out POS signal than five non-overlapping random groups in two small decoder models, and the gain appears across all three POS classes.

that justifies the intervention experiment. it does not yet claim adaptation success.