Context pre-filtering as a token optimization layer — benchmark data #982
nicolalessi
started this conversation in
General
Replies: 1 comment
-
|
Maintainer note: thanks for sharing the benchmark data. We are using this to inform ECC 2.0 context-filtering work and will follow up with our own measurements and implementation notes. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hey @affaan-m, love the work on ECC. The token optimization and memory persistence sections address a real problem.
I've been working on a complementary approach: instead of optimizing at the prompt/session level (which ECC does well), pre-filtering what goes into the context window at the codebase level before the agent even starts.
Ran a controlled benchmark on FastAPI (~800 files):
Tool calls: 23 → 2.3 per task (-90%)
Cost: $0.78 → $0.33 per task (-58%)
Output tokens: 504 → 189 (-63%)
The approach uses tree-sitter AST parsing + dependency graph + SQLite, served via MCP. Everything local.
Curious if you've explored integrating a context pre-filtering layer into ECC's architecture. The two approaches (ECC for agent behavior + context engine for input optimization) could stack well.
Benchmark data: vexp.dev/benchmark
FastAPI writeup: reddit.com/r/ClaudeCode/comments/1rjra2w
Beta Was this translation helpful? Give feedback.
All reactions