feat: support memory compression - #3079
undertaker86001 wants to merge 20 commits into
Conversation
|
kaori-seasons seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
undertaker86001
changed the title
| @@ -0,0 +1,161 @@ | |||
| # AI Context Compressor 插件集成指南 | |||
|
|
|||
There was a problem hiding this comment.
这个需求应该不需要一个独立的压缩插件，第一版本只需要实现工具输出的压缩即可，统一在ai-proxy插件中实现
| contentChan := make(chan string, 1) | ||
| errChan := make(chan error, 1) | ||
|
|
||
| go func() { |
There was a problem hiding this comment.
好的，这里只是想提升读取的性能所以做的优化.
| saveContextTool := tool{ | ||
| Type: "function", | ||
| Function: function{ | ||
| Name: MemoryToolSaveContext, |
There was a problem hiding this comment.
这个tool似乎没有用到？而且应该不需要通过工具的方式触发让llm输出要存储到内存到内容？
| 4. 长度控制在500字符以内 | ||
|
|
||
| 工具输出内容： | ||
| %s |
There was a problem hiding this comment.
是否应该同时给出工具输出对应的输入，这样LLM在发现某次工具调用的输出被压缩掉后，根据这里的信息可以再查找到。
|
|
||
| // TransformResponseBody 处理DeepSeek API响应，确保token计算符合DeepSeek标准 | ||
| // DeepSeek使用OpenAI兼容的API格式，但需要特别处理reasoning tokens | ||
| func (m *deepseekProvider) TransformResponseBody(ctx wrapper.HttpContext, apiName ApiName, body []byte) ([]byte, error) { |
There was a problem hiding this comment.
这个为什么改在deepseek provider里？
| } | ||
|
|
||
| // Step 1: Check if there are compressed references that need to be restored | ||
| needRetrievalIds := c.extractCompressedContextIds(request.Messages) |
There was a problem hiding this comment.
这个逻辑有点问题，不是把每个工具输出分别压缩替换，而是将指定数量的历史（根据配置的保留最近多少工具数）的工具输出一次性做压缩替换
This branch has not been deployed
4 participants
Ⅰ. Describe what this PR did
Related to issue-3077
Ⅱ. Does this pull request fix one issue?
Ⅲ. Why don't you add test cases (unit test/integration test)?
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews