Research · Preprint · September 2026
Semantic Drift in Iterated LLM Paraphrase Chains
Model Architecture, Scale, Quantization, Serving Infrastructure, and Prompt Engineering
Download PDF (2.4 MB) View on Zenodo Data & Code on GitHub
Abstract
When large language models (LLMs) operate in sequential pipelines, each agent paraphrasing the output of the previous one, the cumulative signal diverges from the original in ways that are poorly understood. We study this semantic drift with a telephone-game protocol: a single text is paraphrased 30 times by the same model and cosine similarity to the original is measured at every step. Two complementary datasets are analysed. A cross-family sweep covers 17 models from 8B to 123B parameters with ablations of KV-cache precision (fp16 vs. q8_0 on 10 Qwen3 variants), temperature, and 20 system prompts. A within-family sweep holds the architecture fixed and covers the Qwen3.5 family at 7 sizes (0.8B–122B) and 3 GGUF precision levels, four 4-bit formats of the same 35B mixture-of-experts checkpoint (GGUF, AWQ, NVFP4, GPTQ-Int4) on three serving stacks, five temperatures, and the same 20 system prompts on six configurations plus an out-of-family control. Together the datasets comprise 2,627 completed 30-step chains and roughly 79,000 scored inferences.
We find:
- a stability hierarchy across families — three occupied tiers on a four-band scale, with no model in the Degraded band — with a 0.67 gap in final similarity between the best and worst models and catastrophic tipping-point failures in the weakest;
- within one family, a capacity threshold near 4B parameters below which no configuration is reliable, and above which the ordering by size is irregular;
- a discrete runaway-thinking failure mode in which a model with reasoning enabled spends its entire 16,384-token budget inside the thinking block and returns nothing — absent wherever reasoning was disabled but affecting 15–54% of chains on the two stacks that ran Qwen3.5 with its default reasoning on, concentrated in 4-bit builds, small models, the short stimulus, and constraining system prompts;
- 4-bit quantization formats of identical weights that differ by up to 0.26 on the short stimulus among the chains that survive runaway (two to five per arm, with format and serving stack covarying), while precision level is non-monotonic throughout;
- KV-cache precision and temperature effects that are directionally mixed and small for stable models, while below the threshold low temperature raises the runaway rate to 100% of chains; and
- system-prompt effects that range from invisible (spread < 0.01 on the most stable cross-family model) to dominant (spreads of 0.3–0.7 within the Qwen3.5 family), with prompt rankings that transfer between some configurations (ρ up to 0.83) and not others (ρ ≈ 0 against the control model).
Fixing the sampling seed does not reproduce a chain on any of these stacks, so all statistics are reported over independent samples with bootstrap intervals. The similarity metric is validated against a second embedding model on both datasets (r = 0.96 and 0.83).
Read the Paper
Citation
@misc{smith2026drift,
author = {Smith, James H.},
title = {Semantic Drift in Iterated {LLM} Paraphrase Chains:
Model Architecture, Scale, Quantization,
Serving Infrastructure, and Prompt Engineering},
year = {2026},
howpublished = {Preprint, Zenodo},
doi = {10.5281/zenodo.22646929},
url = {https://doi.org/10.5281/zenodo.22646929}
}Companion paper: Semantic Drift Across Qwen Generations. The full chain texts and analysis code are archived as a Zenodo dataset.