chiguyong
|
027f7909aa
|
docs(solutions): CJK token estimation undercount fix
Test / backend-test (pull_request) Has been cancelled
Details
Test / frontend-unit (pull_request) Has been cancelled
Details
Test / api-e2e (pull_request) Has been cancelled
Details
Test / frontend-e2e (pull_request) Has been cancelled
Details
Document the ContextCompressor CJK 4x underestimation bug and fix:
- estimate_text_tokens() CJK 1:1 / ASCII 4:1 heuristic
- _summarize() max_chars budget fix (P1: was * 4, allowed 4x CJK budget)
- Linear compress flow + structured logging
- Prevention: charset-aware heuristics, audit dependent truncation points
|
2026-07-03 09:40:09 +08:00 |