Headroom vs Microsoft LLMLingua: LLM Agent를 위한 컨텍스트 압축 프레임워크 선택

Microsoft LLMLingua(EMNLP 2023, LongLLMLingua ACL 2024)는 혼란도 기반 프롬프트 압축 학술 표준—소형 LM이 토큰을 점수화해 최대 20× 압축.Headroom(chopratejas/headroom)은 엔지니어링 레인—로컬 프록시 8787, CCR 가역, 콘텐츠 라우터, OpenClaw + Headroom 런북 등 agent 통합.

상시 가동 LLM agent 팀은 갈림길: Python PromptCompressor(연구) vs ANTHROPIC_BASE_URL Headroom(운영). 상호 보완. 본문은 프로덕션 의사결정 매트릭스—혼란도 가지치기 vs 프록시 CCR, headroom proxy --llmlingua, 8단계 평가.

압축 모델 나란히

LLMLingua family (Microsoft Research)
  Small LM (GPT-2-small / LLaMA-7B class)
    → token perplexity p(token | context)
    → drop low-perplexity tokens (budget controller + iterative passes)
    → optional distribution alignment to target LLM
  LongLLMLingua adds:
    → contrastive perplexity p(question | document)
    → document reorder ("lost in the middle" mitigation)
    → coarse-to-fine compression for RAG stacks

Headroom (engineering stack)
  Incoming messages (tools, logs, JSON, code, chat)
    → CacheAligner (KV-cache-friendly prefixes)
    → ContentRouter
         ├─ SmartCrusher (JSON arrays/objects)
         ├─ CodeCompressor (AST: Py/JS/Go/Rust/…)
         └─ Kompress-base (agentic prose, HF model)
    → CCR stores originals locally; model calls headroom_retrieve
    → Proxy forwards /v1/messages to Anthropic/OpenAI/Bedrock

인용: LLMLingua는 SLM 혼란도로 토큰 제거; Headroom은 콘텐츠 유형 라우팅과 가역 CCR.

의사결정 매트릭스: 학술 vs 엔지니어링

시나리오 A: 긴 문서 더미 RAG

Profile: Legal, support, or internal wiki QA—10–50 PDFs chunked into a single prompt, user question appended.

LLMLingua fit: LongLLMLingua was built for this. Use condition_in_question="after_condition", reorder_context="sort", rate=0.55 per Microsoft's examples. Contrastive perplexity beats vanilla when documents are noisy.

Headroom fit: Strong when chunks mix JSON metadata + prose (ticket exports, CI logs in KB). Proxy mode compresses without rewriting LangChain/LlamaIndex glue.

If X, do Y: If bottleneck is multi-document ordering and lost-in-the-middle, do prototype LongLLMLingua first. If bottleneck is heterogeneous tool+json context in an agent loop, do prototype Headroom proxy first.

시나리오 B: 상시 코딩/운영 agent(OpenClaw급)

Profile: Nightly repo audits, MCP stdio tools, megabyte linter JSON—context grows every turn.

LLMLingua fit: Works as a pre-step if you batch-compress static prompts offline. Per-request compress_prompt() adds SLM inference latency on every gateway call unless cached.

Headroom fit: Designed for this shape—documented OpenClaw plugin, /stats Prometheus metrics, headroom mcp install. See OpenClaw + Headroom 런북 for LaunchAgent wiring.

If X, do Y: If you need drop-in proxy on macOS launchd gateways, do Headroom. If you publish research pipelines with frozen prompts, do LLMLingua in the ingest stage.

시나리오 C: 하이브리드 스택

Headroom supports headroom proxy --llmlingua—Microsoft's perplexity compressor as an optional deeper pass after structural crushers. Trade-off: ~2 GB extra dependencies, 10–30s cold start per Headroom proxy docs.

If X, do Y: If eval shows SmartCrusher leaves >30% fat JSON, do enable --llmlingua on a 24 GB Mac mini M4 only. If latency SLO < 2s p95, do stay on structural crushers + CCR without ML pass.

권장 경로

• If you optimize ACL-style RAG benchmarks, do start with LongLLMLingua PromptCompressor and Microsoft rate sweeps.
• If you operate OpenClaw / Claude Code / Cursor fleets, do start with Headroom proxy and measure /stats-history for seven nights.
• If compliance requires verbatim audit trails, do prefer Headroom CCR over lossy perplexity-only pipelines.
• If you need KV-cache compression research, do evaluate LLMLingua-2 and Microsoft's cache line per Microsoft Research.
• If neither hits 40% savings on your traces, do fix prompt design first—compression cannot rescue redundant tool round-trips.

8단계 평가 런북

1. 골든 프롬프트 세트 고정

Capture N≥20 real agent turns: tool JSON, stack traces, instructions. Store SHA-256 per fixture under ~/compression-eval/fixtures/.

2. 베이스라인 토큰 수(비압축)

Record input tokens from provider dashboard or tiktoken for each fixture.

3. LLMLingua / LongLLMLingua 암 실행

pip install llmlingua

from llmlingua import PromptCompressor
pc = PromptCompressor(model_name="microsoft/llmlingua-2-xlm-roberta-large-meetingbank")
out = pc.compress_prompt(prompt_list, question=question, rate=0.55,
    condition_in_question="after_condition", reorder_context="sort",
    rank_method="longllmlingua")
compressed = out["compressed_prompt"]

Log origin_tokens, compressed_tokens, wall-clock ms.

4. Headroom 프록시 암 실행

pip install "headroom-ai[proxy]"
headroom proxy --port 8787 --log-file ~/.headroom/eval.jsonl

POST fixtures through /v1/compress or route live agent traffic; read tokens_saved from /stats.

5. 선택 하이브리드 암

headroom proxy --port 8788 --llmlingua --llmlingua-rate 0.3

Compare p95 latency vs savings uplift.

6. 품질 게이트(동일 하류 LLM)

Re-run each compressed fixture through your production model with identical temperature. Score: exact-match for structured fields, LLM-judge for summaries, human spot-check 5%.

7. Agent 회귀 스위트

For OpenClaw operators: replay nightly audit job with each arm; compare finding counts and false-negative rate on known seeded bugs.

8. 워크로드별 승자 선택

Document: RAG ingest → LongLLMLingua, live gateway → Headroom proxy, max compression lab → hybrid—publish internally with token $/month math.

문제 해결

LLMLingua collapsed instruction-following

증상: Compressed prompt drops negation or JSON keys.

조치: Lower rate (0.55 → 0.75). Use budget controller to exempt instruction block. Compare LLMLingua-2 per Microsoft Research.

Headroom proxy saves tokens but agent misses line numbers

증상: Audit agent cites wrong file:line.

조치: Instruct model to headroom_retrieve before closing findings; set x-headroom-bypass: true on one repro. Narrow SmartCrusher if schema keys stripped.

Both arms slower than uncompressed

증상: p95 latency > 3× baseline.

조치: LLMLingua—cache SLM on GPU/MPS, batch offline. Headroom—disable --llmlingua, keep structural crushers only; co-locate proxy on same host as gateway.

FAQ

Headroom은 LLMLingua 포크인가요?

아닙니다. Headroom은 독립 Apache-2.0 프로젝트이며 --llmlingua로 선택 호출 가능합니다. 기본은 SmartCrusher, CodeCompressor, Kompress-base—혼란도 가지치기만이 아닙니다.

혼란도 가지치기가 콘텐츠 유형 크러셔보다 나은 때는?

프롬프트가 동질 자연어(장문, JSON 섬 적음)이고 알려진 질문 앵커로 LongLLMLingua를 튜닝할 때. 이종 agent 도구 출력은 Headroom 라우터가 유리합니다.

Headroom 없이 OpenClaw에서 LongLLMLingua만 쓸 수 있나요?

가능—스킬에서 PromptCompressor로 정적 컨텍스트 사전 압축. 요청별 프록시 투명성과 CCR은 잃습니다.

LLMLingua-2 vs LongLLMLingua?

LLMLingua-2는 BERT 규모 인코더 토큰 분류—Microsoft 보고서 기준 반복 혼란도 대비 3–6배 빠름. Headroom은 --llmlingua로 계층화; SLO는 별도 평가.

20개 저장소 야간 감사 함대에 재무는 무엇을 승인해야 하나요?

1주차에 1개 저장소로 8단계 평가. 도구 JSON 지배 시 Headroom 프록시 + OpenClaw가 운영 통합 빠름; 정적 RAG 지배 시 LongLLMLingua가 품질/달러에서 이길 수 있음.