Headroom vs Microsoft LLMLingua: Das richtige Kontextkompressions-Framework für LLM-Agenten

Microsoft LLMLingua (EMNLP 2023, LongLLMLingua ACL 2024) ist der akademische Goldstandard für perplexitätsgetriebene Prompt-Kompression—bis 20×. Headroom (chopratejas/headroom) ist die Engineering-Spur: lokaler Proxy 8787, reversibles CCR, ContentRouter, OpenClaw + Headroom Runbook.

Always-on-LLM-Agent-Teams wählen: PromptCompressor in Python oder ANTHROPIC_BASE_URL auf Headroom. Entscheidungsmatrix für Produktion: Perplexitäts-Pruning vs Proxy-CCR, headroom proxy --llmlingua, 8-Schritte-Eval.

Kompressionsmodelle im Vergleich

LLMLingua family (Microsoft Research)
  Small LM (GPT-2-small / LLaMA-7B class)
    → token perplexity p(token | context)
    → drop low-perplexity tokens (budget controller + iterative passes)
    → optional distribution alignment to target LLM
  LongLLMLingua adds:
    → contrastive perplexity p(question | document)
    → document reorder ("lost in the middle" mitigation)
    → coarse-to-fine compression for RAG stacks

Headroom (engineering stack)
  Incoming messages (tools, logs, JSON, code, chat)
    → CacheAligner (KV-cache-friendly prefixes)
    → ContentRouter
         ├─ SmartCrusher (JSON arrays/objects)
         ├─ CodeCompressor (AST: Py/JS/Go/Rust/…)
         └─ Kompress-base (agentic prose, HF model)
    → CCR stores originals locally; model calls headroom_retrieve
    → Proxy forwards /v1/messages to Anthropic/OpenAI/Bedrock

Zitat: LLMLingua entfernt per SLM-Perplexität; Headroom routet nach Inhaltstyp mit reversiblem CCR.

Entscheidungsmatrix: akademisch vs Engineering

Szenario A: RAG über lange Dokumenthaufen

Profile: Legal, support, or internal wiki QA—10–50 PDFs chunked into a single prompt, user question appended.

LLMLingua fit: LongLLMLingua was built for this. Use condition_in_question="after_condition", reorder_context="sort", rate=0.55 per Microsoft's examples. Contrastive perplexity beats vanilla when documents are noisy.

Headroom fit: Strong when chunks mix JSON metadata + prose (ticket exports, CI logs in KB). Proxy mode compresses without rewriting LangChain/LlamaIndex glue.

If X, do Y: If bottleneck is multi-document ordering and lost-in-the-middle, do prototype LongLLMLingua first. If bottleneck is heterogeneous tool+json context in an agent loop, do prototype Headroom proxy first.

Szenario B: Always-on Code/Ops-Agenten (OpenClaw-Klasse)

Profile: Nightly repo audits, MCP stdio tools, megabyte linter JSON—context grows every turn.

LLMLingua fit: Works as a pre-step if you batch-compress static prompts offline. Per-request compress_prompt() adds SLM inference latency on every gateway call unless cached.

Headroom fit: Designed for this shape—documented OpenClaw plugin, /stats Prometheus metrics, headroom mcp install. See OpenClaw + Headroom Runbook for LaunchAgent wiring.

If X, do Y: If you need drop-in proxy on macOS launchd gateways, do Headroom. If you publish research pipelines with frozen prompts, do LLMLingua in the ingest stage.

Szenario C: Hybrid-Stack

Headroom supports headroom proxy --llmlingua—Microsoft's perplexity compressor as an optional deeper pass after structural crushers. Trade-off: ~2 GB extra dependencies, 10–30s cold start per Headroom proxy docs.

If X, do Y: If eval shows SmartCrusher leaves >30% fat JSON, do enable --llmlingua on a 24 GB Mac mini M4 only. If latency SLO < 2s p95, do stay on structural crushers + CCR without ML pass.

Empfohlener Pfad

• If you optimize ACL-style RAG benchmarks, do start with LongLLMLingua PromptCompressor and Microsoft rate sweeps.
• If you operate OpenClaw / Claude Code / Cursor fleets, do start with Headroom proxy and measure /stats-history for seven nights.
• If compliance requires verbatim audit trails, do prefer Headroom CCR over lossy perplexity-only pipelines.
• If you need KV-cache compression research, do evaluate LLMLingua-2 and Microsoft's cache line per Microsoft Research.
• If neither hits 40% savings on your traces, do fix prompt design first—compression cannot rescue redundant tool round-trips.

Acht-Schritte-Evaluations-Runbook

1. Golden-Prompt-Set einfrieren

Capture N≥20 real agent turns: tool JSON, stack traces, instructions. Store SHA-256 per fixture under ~/compression-eval/fixtures/.

2. Baseline-Token zählen (unkomprimiert)

Record input tokens from provider dashboard or tiktoken for each fixture.

3. LLMLingua / LongLLMLingua-Arm ausführen

pip install llmlingua

from llmlingua import PromptCompressor
pc = PromptCompressor(model_name="microsoft/llmlingua-2-xlm-roberta-large-meetingbank")
out = pc.compress_prompt(prompt_list, question=question, rate=0.55,
    condition_in_question="after_condition", reorder_context="sort",
    rank_method="longllmlingua")
compressed = out["compressed_prompt"]

Log origin_tokens, compressed_tokens, wall-clock ms.

4. Headroom-Proxy-Arm ausführen

pip install "headroom-ai[proxy]"
headroom proxy --port 8787 --log-file ~/.headroom/eval.jsonl

POST fixtures through /v1/compress or route live agent traffic; read tokens_saved from /stats.

5. Optionaler Hybrid-Arm

headroom proxy --port 8788 --llmlingua --llmlingua-rate 0.3

Compare p95 latency vs savings uplift.

6. Qualitätsgate (gleiches Downstream-LLM)

Re-run each compressed fixture through your production model with identical temperature. Score: exact-match for structured fields, LLM-judge for summaries, human spot-check 5%.

7. Agent-Regressionssuite

For OpenClaw operators: replay nightly audit job with each arm; compare finding counts and false-negative rate on known seeded bugs.

8. Gewinner pro Workload wählen

Document: RAG ingest → LongLLMLingua, live gateway → Headroom proxy, max compression lab → hybrid—publish internally with token $/month math.

Fehlerbehebung

LLMLingua collapsed instruction-following

Symptom: Compressed prompt drops negation or JSON keys.

Fix: Lower rate (0.55 → 0.75). Use budget controller to exempt instruction block. Compare LLMLingua-2 per Microsoft Research.

Headroom proxy saves tokens but agent misses line numbers

Symptom: Audit agent cites wrong file:line.

Fix: Instruct model to headroom_retrieve before closing findings; set x-headroom-bypass: true on one repro. Narrow SmartCrusher if schema keys stripped.

Both arms slower than uncompressed

Symptom: p95 latency > 3× baseline.

Fix: LLMLingua—cache SLM on GPU/MPS, batch offline. Headroom—disable --llmlingua, keep structural crushers only; co-locate proxy on same host as gateway.

FAQ

Ist Headroom ein Fork von LLMLingua?

Nein. Headroom ist ein unabhängiges Apache-2.0-Projekt mit optionalem --llmlingua. Standardpfad: SmartCrusher, CodeCompressor, Kompress-base—nicht nur Perplexitäts-Pruning.

Wann schlägt Perplexitäts-Pruning typbasierte Crusher?

Bei homogenem Fließtext und LongLLMLingua mit bekannter Frage-Anker. Heterogene Agent-Tool-Ausgaben begünstigen Headrooms Router.

LongLLMLingua in OpenClaw ohne Headroom?

Ja—statischen Kontext in Skills mit PromptCompressor vor-komprimieren. Proxy-Transparenz und CCR entfallen ohne eigenes Retrieval.

LLMLingua-2 vs LongLLMLingua?

LLMLingua-2: Token-Klassifikation mit BERT-Encoder—laut Microsoft 3–6× schneller. Headroom stapelt via --llmlingua; Geschwindigkeit vs SLO separat prüfen.

Was sollte Finance für 20-Repo-Nacht-Audit-Fleet freigeben?

Acht-Schritte-Eval in Woche 1 auf einem Repo. Tool-JSON dominiert → Headroom-Proxy + OpenClaw; statisches Doc-RAG → LongLLMLingua kann qualität/pro Dollar gewinnen.