ctxprune: context compression for AI agents

A small token classifier deletes the least useful words from tool outputs, logs, code and documents before an LLM reads them. It is a successor to Microsoft's LLMLingua-2, trained on agent text, and it never splits or re-spaces identifiers.

pip install ctxprune GitHub Model card @imdariotoo

51.5 vs 36.5exact-match QA at 50% of tokens kept, ctxprune vs headroom's Kompress-v2 (SQuAD v2 + HotpotQA, 400 questions, CI +10.5..+19.5; reader Qwen3.8-27B, also our labeling teacher)

68.8% vs 53.4%questions an LLM still answers at 50% kept, ctxprune vs LLMLingua-2-large (our 378-question agent set, independent reader; 91.8% uncompressed)

~6× fasteron CPU than LLMLingua-2-large; int8 ONNX, no torch needed

Try it on your own text

Runs entirely in your browser with the int8 ONNX model. The first run downloads about 180 MB from Hugging Face (cached after that); nothing you paste is sent anywhere.

keep 50% never drop IDs, numbers or paths

ctxprune-small, in your browser

What the LLM receives

Side by side with LLMLingua-2

Precomputed outputs of both models on the same inputs.

ctxprune: never drop IDs, numbers or paths (LLMLingua-2 is then run at the same output size)

struck = deleted · red = identifier deleted or mangled (what the LLM no longer sees intact)

Real outputs of darioooooo0o/ctxprune-small (int8 ONNX) and microsoft/llmlingua-2-bert-base-multilingual-cased-meetingbank, computed offline with each library's own rate. LLMLingua-2 measures rate in its own tokens and often keeps more than asked. QA numbers: 378 held-out questions answered by an independent LLM, details on the model card. The in-browser demo runs the same int8 model in WebAssembly; its scores are as close to the exact fp32 model as the Python int8 build's are, but a few borderline words can differ from pip install ctxprune output.