Research Papers

Original research from our own hardware lab. Every paper below is archived at Zenodo with a citable DOI under a CC BY 4.0 license, and the full experimental data and analysis code are public.

Preprint · September 2026

Semantic Drift in Iterated LLM Paraphrase Chains

Model Architecture, Scale, Quantization, Serving Infrastructure, and Prompt Engineering

James H. Smith · DOI 10.5281/zenodo.22646929

A telephone-game protocol — one text paraphrased 30 times by the same model, with cosine similarity measured at every step — applied to 17 models from 8B to 123B parameters and to the full Qwen3.5 family across 7 sizes, four 4-bit formats, three serving stacks, five temperatures and 20 system prompts. 2,627 completed chains and roughly 79,000 scored inferences reveal a stability hierarchy across families, a capacity threshold near 4B parameters, and a discrete runaway-thinking failure mode that silently returns nothing on 15–54% of chains.

Read Paper Download PDF (2.4 MB) View on Zenodo

Preprint · September 2026

Semantic Drift Across Qwen Generations

Reproducibility, Thinking Mode, Serving Stack, and What Cosine Similarity Misses in Iterated Paraphrase Chains

James H. Smith · DOI 10.5281/zenodo.22646931

The companion study, extended to the Qwen3.6 and Qwen3.8 generations across 2,717 chains and 87,110 model calls on vLLM, SGLang and llama.cpp. Its central negative result: cosine similarity — the drift measure this whole line of work depends on — does not track the information a document actually carries. It is only weakly related to the survival of checkable facts (ρ = 0.32), and on construction statements of work it is negatively related (ρ = −0.42). Fixed sampling seeds do not reproduce a chain on any stack tested.

Read Paper Download PDF (1.8 MB) View on Zenodo

Data & Code

Dataset · Version 1.2

Full Chain Texts, Prompts, and Analysis Code

The complete supplementary record behind both papers

James H. Smith · DOI 10.5281/zenodo.22646927

Every per-iteration input and output text for every paraphrase chain reported in both papers, together with the prompt files and their fact anchors, the experiment runners, and the analysis code that produces every figure and table. Released so that drift metrics other than cosine similarity can be applied to the same chains. Distributed as compressed archives with published SHA-256 checksums — roughly 80 MB in total, so it is hosted at Zenodo and GitHub rather than here.

Download from Zenodo View on GitHub

Related reading on this blog: the posts that preceded these papers — the LLM telephone game, KV cache precision, and system prompt engineering.