feat: 8 framework adapters + CI/CD + Demo + Docker - #37
Conversation
- A3MLangChainAdapter: Drop-in replacement for LangChain's ChatOpenAI
- A3MLlamaIndexAdapter: Drop-in replacement for LlamaIndex's BaseLLM
- A3MConfig: Configuration management for adapters
Features:
- Lazy initialization (no backend needed to instantiate)
- Framework-agnostic design
- Full compatibility with existing A3M Router backend
Usage:
from adapters import A3MLangChainAdapter, A3MLlamaIndexAdapter
llm = A3MLangChainAdapter(model='auto', temperature=0.7)
response = llm.invoke('What is the capital of France?')
|
Warning Review limit reached
Next review available in: 58 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (12)
📝 WalkthroughWalkthroughThe change adds LangChain and LlamaIndex adapters, configuration support, packaging, CI and release automation, deployment services, demonstrations, and updated product documentation. ChangesA3M Router adapter and deployment update
Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant Adapter
participant Executor
participant A3MRouter
Client->>Adapter: Submit messages or prompt
Adapter->>Executor: Submit converted routing request
Executor->>A3MRouter: Execute generation
A3MRouter-->>Executor: Return routed result
Executor-->>Adapter: Return response data
Adapter-->>Client: Return framework response
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 12
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@adapters/a3m_adapter/adapter/config.py`:
- Around line 10-14: Update the usage example to import the public A3MConfig
symbol from the correct package and instantiate the exported A3MLangChainAdapter
instead of A3MChatModel, while preserving the existing from_file and to_dict
usage.
- Around line 94-100: Update update_budget_limits to handle a zero
daily_budget_usd before calculating remaining_pct, treating it as an exhausted
budget and reducing parallel_ensemble to 1; also ensure configuration validation
rejects negative budget limits while preserving zero as the valid
spending-disabled value.
- Around line 64-73: Update A3MConfig.from_file to choose the parser based on
the file extension: use JSON loading only for .json paths, and require PyYAML
for YAML files by preserving or surfacing the ImportError instead of falling
back to from_json. Keep YAML parsing through yaml.safe_load when PyYAML is
available.
In `@adapters/a3m_adapter/adapter/langchain.py`:
- Around line 15-23: Preserve the LANGCHAIN_AVAILABLE flag in
adapters/a3m_adapter/adapter/langchain.py (lines 15-23), and add an early
documented ImportError check before message conversion or LangChain output
construction in the relevant adapter flow (lines 103-106). Preserve
LLAMAINDEX_AVAILABLE in adapters/a3m_adapter/adapter/llamaindex.py (lines
15-28), and add the same availability guard before using LlamaIndex response
types (lines 99-103), preventing NameError when either optional dependency is
absent.
- Around line 144-146: Update A3MLangChainAdapter.bind_tools to either persist
the provided tool definitions and ensure A3MRouter.route receives them for
subsequent requests, or raise NotImplementedError when routing cannot support
tools; do not return self while silently discarding tools.
- Around line 109-124: Resolve the future returned by run_in_executor before
accessing the routed response, or call _a3m_router.route synchronously in the
affected synchronous methods. Apply this to the route-result handling in
adapters/a3m_adapter/adapter/langchain.py lines 109-124 and both affected sites
in adapters/a3m_adapter/adapter/llamaindex.py lines 105-117 and 125-137; each
must use the actual response before reading content and constructing the result.
- Around line 33-39: The A3MLangChainAdapter class must implement LangChain’s
BaseChatModel contract rather than remain a plain class. Make
A3MLangChainAdapter inherit BaseChatModel, provide a compatible constructor and
_generate() implementation that resolves the asynchronous route() result before
constructing ChatResult, and wire callbacks through run_manager; implement
bind_tools() correctly or explicitly reject unsupported tool binding instead of
returning self.
- Around line 41-66: Update A3MChatModel initialization in
adapters/a3m_adapter/adapter/langchain.py: consume kwargs from
A3MConfig.to_dict(), map supported fields such as top_p, fallback_enabled,
cost_threshold, and provider lists to router or request parameters, and reject
unsupported fields instead of discarding them. Apply the same full-configuration
handling to the adapter in adapters/a3m_adapter/adapter/llamaindex.py. In
adapters/a3m_adapter/adapter/config.py, update its usage only after both
adapters consume the complete serialized configuration.
In `@adapters/a3m_adapter/adapter/llamaindex.py`:
- Around line 38-44: Update A3MLlamaIndexAdapter to fully conform to LlamaIndex
BaseLLM: inherit from BaseLLM, implement all required abstract methods, and
change both metadata implementations to return a version-compatible LLMMetadata
instance rather than Dict[str, Any]. Preserve the existing CompletionResponse
and ChatMessage behavior while satisfying BaseLLM’s metadata contract.
In `@adapters/a3m_adapter/tests/test_adapters.py`:
- Line 26: Update adapters/a3m_adapter/tests/test_adapters.py at lines 26-26,
47-47, and 67-67 to use the published package API: import A3MLangChainAdapter,
A3MLlamaIndexAdapter, and A3MConfig respectively from a3m_adapter, replacing the
legacy module and class imports.
- Around line 54-55: Update the metadata handling in the test to invoke the
adapter’s metadata() method and use the returned dictionary values instead of
treating metadata as an object with model_name and num_output attributes;
preserve the existing diagnostic output using the dictionary keys.
- Around line 38-40: Remove the return False failure paths from
test_langchain_adapter, test_llamaindex_adapter, and test_config in
adapters/a3m_adapter/tests/test_adapters.py at lines 38-40, 58-60, and 85-87;
let their assertions or raised exceptions propagate so pytest correctly marks
failures.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 5df206d8-ea1d-4b1d-bd91-da77d2f883c1
📒 Files selected for processing (10)
adapters/README.mdadapters/__init__.pyadapters/a3m_adapter/__init__.pyadapters/a3m_adapter/adapter/__init__.pyadapters/a3m_adapter/adapter/config.pyadapters/a3m_adapter/adapter/langchain.pyadapters/a3m_adapter/adapter/llamaindex.pyadapters/a3m_adapter/tests/__init__.pyadapters/a3m_adapter/tests/test_adapters.pyadapters/setup.py
| Usage: | ||
| from a3m_adapter_config import A3MConfig | ||
|
|
||
| config = A3MConfig.from_file("a3m_config.yaml") | ||
| llm = A3MChatModel(**config.to_dict()) |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Correct the package names in the usage example.
a3m_adapter_config and A3MChatModel are not exported by this package. The public API exports A3MConfig and A3MLangChainAdapter.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@adapters/a3m_adapter/adapter/config.py` around lines 10 - 14, Update the
usage example to import the public A3MConfig symbol from the correct package and
instantiate the exported A3MLangChainAdapter instead of A3MChatModel, while
preserving the existing from_file and to_dict usage.
| def from_file(cls, path: str) -> "A3MConfig": | ||
| """Load configuration from YAML file.""" | ||
| try: | ||
| import yaml | ||
| with open(path, 'r') as f: | ||
| data = yaml.safe_load(f) | ||
| return cls(**data) | ||
| except ImportError: | ||
| logger.warning("PyYAML not installed, using JSON") | ||
| return cls.from_json(path) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Do not parse a documented YAML file as JSON.
When PyYAML is absent, from_file("a3m_config.yaml") calls json.load() for YAML content. This fails with JSONDecodeError.
Require PyYAML for YAML files. Alternatively, select JSON loading only for .json paths.
🧰 Tools
🪛 ast-grep (0.45.0)
[warning] 67-67: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(path, 'r')
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@adapters/a3m_adapter/adapter/config.py` around lines 64 - 73, Update
A3MConfig.from_file to choose the parser based on the file extension: use JSON
loading only for .json paths, and require PyYAML for YAML files by preserving or
surfacing the ImportError instead of falling back to from_json. Keep YAML
parsing through yaml.safe_load when PyYAML is available.
| def update_budget_limits(self, remaining_usd: float) -> None: | ||
| """Update budget limits based on remaining funds.""" | ||
| if self.daily_budget_usd is not None: | ||
| remaining_pct = remaining_usd / self.daily_budget_usd | ||
| if remaining_pct < 0.1: | ||
| logger.warning("Low daily budget: %s remaining", remaining_usd) | ||
| self.parallel_ensemble = 1 # Reduce to single-provider |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Handle a zero daily budget before division.
When daily_budget_usd is 0, Line 97 raises ZeroDivisionError. A zero budget is a valid way to disable spending.
Reject non-positive budget limits during configuration validation. Alternatively, handle zero as an exhausted budget before calculating remaining_pct.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@adapters/a3m_adapter/adapter/config.py` around lines 94 - 100, Update
update_budget_limits to handle a zero daily_budget_usd before calculating
remaining_pct, treating it as an exhausted budget and reducing parallel_ensemble
to 1; also ensure configuration validation rejects negative budget limits while
preserving zero as the valid spending-disabled value.
| # Check availability | ||
| LANGCHAIN_AVAILABLE = False | ||
| try: | ||
| from langchain_core.language_models import BaseChatModel | ||
| from langchain_core.messages import AIMessage, BaseMessage, HumanMessage, SystemMessage, ToolMessage | ||
| from langchain_core.outputs import ChatGeneration, ChatResult, LLMResult | ||
| LANGCHAIN_AVAILABLE = True | ||
| except ImportError: | ||
| logger.warning("LangChain not installed. Install with: pip install langchain langchain-core") |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Raise a clear error when the framework dependency is unavailable.
Both modules set an availability flag but never use it. If the framework package is absent, later references to imported message or response classes can raise NameError instead of the documented ImportError.
adapters/a3m_adapter/adapter/langchain.py#L15-L23: Retain the availability state for the optional LangChain dependency.adapters/a3m_adapter/adapter/langchain.py#L103-L106: CheckLANGCHAIN_AVAILABLEbefore converting messages or constructing LangChain outputs.adapters/a3m_adapter/adapter/llamaindex.py#L15-L28: Retain the availability state for the optional LlamaIndex dependency.adapters/a3m_adapter/adapter/llamaindex.py#L99-L103: CheckLLAMAINDEX_AVAILABLEbefore using LlamaIndex response types.
📍 Affects 2 files
adapters/a3m_adapter/adapter/langchain.py#L15-L23(this comment)adapters/a3m_adapter/adapter/langchain.py#L103-L106adapters/a3m_adapter/adapter/llamaindex.py#L15-L28adapters/a3m_adapter/adapter/llamaindex.py#L99-L103
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@adapters/a3m_adapter/adapter/langchain.py` around lines 15 - 23, Preserve the
LANGCHAIN_AVAILABLE flag in adapters/a3m_adapter/adapter/langchain.py (lines
15-23), and add an early documented ImportError check before message conversion
or LangChain output construction in the relevant adapter flow (lines 103-106).
Preserve LLAMAINDEX_AVAILABLE in adapters/a3m_adapter/adapter/llamaindex.py
(lines 15-28), and add the same availability guard before using LlamaIndex
response types (lines 99-103), preventing NameError when either optional
dependency is absent.
| class A3MLangChainAdapter: | ||
| """ | ||
| A3M Router adapter for LangChain's ChatOpenAI interface. | ||
|
|
||
| Routes prompts through A3M Router to automatically select the cheapest | ||
| capable model across 47+ LLM providers. | ||
| """ |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Locate langchain adapter =="
fd -a 'langchain\.py$' . || true
echo "== File outline =="
ast-grep outline adapters/a3m_adapter/adapter/langchain.py --view expanded || true
echo "== Relevant file contents =="
cat -n adapters/a3m_adapter/adapter/langchain.py
echo "== pyproject langchain dependencies =="
fd -a 'pyproject.toml|requirements.*|setup.py|setup.cfg' . -d 3 | sort | while read -r f; do
echo "--- $f ---"
sed -n '1,220p' "$f"
done
echo "== Search LangChain usage =="
rg -n "BaseChatModel|GENERATIONAL|invoke\\(|run_in_executor|_generate|bind_tools|ChatMessage|AIMessage|LANGCHAIN_AVAILABLE" adapters/a3m_adapter/adapter/langchain.py .Repository: Das-rebel/a3m-router
Length of output: 21711
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Runtime availability check =="
python3 - <<'PY'
import ast
from pathlib import Path
p = Path("adapters/a3m_adapter/adapter/langchain.py")
mod = ast.parse(p.read_text())
classes = [n for n in mod.body if isinstance(n, ast.ClassDef)]
assert len(classes) == 1 and classes[0].name == "A3MLangChainAdapter"
cls = classes[0]
bases = [ast.unparse(b) for b in cls.bases]
print("classes=", [n.name for n in classes])
print("_generate_is_async=", next((c.name == "_generate" and isinstance(c.value, ast.AsyncFunctionDef) for c in cls.body), False))
print("class_bases=", bases)
print("methods=", [c.name for c in cls.body if isinstance(c, (ast.FunctionDef, ast.AsyncFunctionDef))])
print("imports=", ', '.join([ast.unparse(i) for i in mod.body if isinstance(i, ast.Import)]))
print("imports_from=", ', '.join(['langchain_core:' + ast.unparse(n.name)+':'+','.join(a.name for a in n.names) for i in mod.body if isinstance(i, ast.ImportFrom) and hasattr(i, 'module') and i.module]))
PY
echo "== Static compatibility with LangChain BaseChatModel signature/source if installed =="
python3 - <<'PY'
try:
import inspect
import langchain_core.language_models.chat_models as chat_models
except Exception as e:
print(f"langchain_core unavailable: {e}")
raise SystemExit(0)
sig = inspect.signature(chat_models.BaseChatModel.invoke)
print("BaseChatModel.invoke signature =", *sig.parameters.keys())
gen = chat_models.BaseChatModel._generate
print("BaseChatModel._generate signature =", *inspect.signature(gen).parameters.keys())
print("", inspect.getmro(chat_models.BaseChatModel))
PYRepository: Das-rebel/a3m-router
Length of output: 392
🌐 Web query:
langchain langchain-core BaseChatModel invoke _generate implementation language models chat models
💡 Result:
In LangChain, BaseChatModel serves as the foundational abstract base class for all chat models within langchain-core [1][2]. Its architecture defines a clear separation between the public interface used by developers and the internal implementation required by custom model providers [2]. Key Architectural Concepts: 1. Public Invoke Interface: The invoke method is part of the public Runnable interface [3]. When called, it handles the input, manages execution configuration, and orchestrates the call to the underlying model, ultimately returning an AIMessage [4][1]. It is designed to be user-friendly, accepting various input formats like strings, prompt values, or lists of messages [1][3]. 2. The _generate Implementation Requirement: For developers creating custom chat models, BaseChatModel mandates the implementation of the protected _generate method [1][2]. This is the core imperative method where the provider-specific logic resides [2][3]. The signature for _generate is: def _generate( self, messages: list[BaseMessage], stop: list[str] | None = None, run_manager: CallbackManagerForLLMRun | None = None, kwargs: Any,) -> ChatResult: - Purpose: It generates a ChatResult from the provided input messages [2][5]. - Integration: It receives an optional list of stop sequences, a run manager for handling callbacks, and arbitrary keyword arguments to pass to the underlying model's API [2][5]. - Mandatory Nature: It is explicitly defined as an abstract method that must be implemented by any concrete subclass, ensuring that all chat models maintain a unified behavior when invoked [2][5]. Relationship between invoke and _generate: When a user calls invoke, the BaseChatModel class manages the setup (such as processing input and handling callbacks) and internally calls generate (or its asynchronous equivalent), which in turn relies on the provider's specific _generate implementation to perform the actual request to the language model [6][2]. This abstraction allows users to interact with different models through a consistent API while forcing developers to adhere to a standard structural contract [1][2].
Citations:
- 1: https://reference.langchain.com/python/langchain-core/language_models/chat_models/BaseChatModel
- 2: https://github.com/langchain-ai/langchain/blob/50febb79/libs/core/langchain_core/language_models/chat_models.py
- 3: https://github.com/langchain-ai/langchain/blob/master/libs/core/langchain_core/language_models/chat_models.py
- 4: https://reference.langchain.com/python/langchain-core/language_models/chat_models/BaseChatModel/invoke
- 5: https://fossies.org/dox/langchain-langchain-core-1.4.0/classlangchain__core_1_1language__models_1_1chat__models_1_1BaseChatModel.html
- 6: https://reference.langchain.com/python/langchain-core/language_models/chat_models/BaseChatModel/generate
Implement the LangChain BaseChatModel contract.
A3MLangChainAdapter is declared as a drop-in ChatOpenAI replacement, but it does not inherit BaseChatModel. That disables the public LangChain flow: invoke() is not provided by this class, callbacks from run_manager are not wired, and serialization/copying behavior is left to a plain class.
Subclass langchain_core.language_models.BaseChatModel with a compatible constructor and _generate() implementation. Resolve the async route() call correctly before building ChatResult, and handle bind_tools() or reject it instead of returning self.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@adapters/a3m_adapter/adapter/langchain.py` around lines 33 - 39, The
A3MLangChainAdapter class must implement LangChain’s BaseChatModel contract
rather than remain a plain class. Make A3MLangChainAdapter inherit
BaseChatModel, provide a compatible constructor and _generate() implementation
that resolves the asynchronous route() result before constructing ChatResult,
and wire callbacks through run_manager; implement bind_tools() correctly or
explicitly reject unsupported tool binding instead of returning self.
| def bind_tools(self, tools: List[Dict[str, Any]], **kwargs: Any) -> "A3MLangChainAdapter": | ||
| """Bind tools for function calling.""" | ||
| return self |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Do not silently discard bound tools.
bind_tools() returns self, but it does not store tools or pass them to A3MRouter.route(). Tool calling therefore appears enabled but never reaches the provider.
Persist the tool definitions and include them in routed requests. If the router does not support tools, raise NotImplementedError instead.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@adapters/a3m_adapter/adapter/langchain.py` around lines 144 - 146, Update
A3MLangChainAdapter.bind_tools to either persist the provided tool definitions
and ensure A3MRouter.route receives them for subsequent requests, or raise
NotImplementedError when routing cannot support tools; do not return self while
silently discarding tools.
| class A3MLlamaIndexAdapter: | ||
| """ | ||
| A3M Router adapter for LlamaIndex's BaseLLM interface. | ||
|
|
||
| Routes prompts through A3M Router to automatically select the cheapest | ||
| capable model across 47+ LLM providers. | ||
| """ |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== locate file =='
fd -a 'llamaindex.py' . || true
echo "== relevant file =="
if [ -f adapters/a3m_adapter/adapter/llamaindex.py ]; then
wc -l adapters/a3m_adapter/adapter/llamaindex.py
sed -n '1,170p' adapters/a3m_adapter/adapter/llamaindex.py | cat -n
fi
echo "== search BaseLLM usage in repo =="
rg -n "A3MLlamaIndexAdapter|LLMInterface|BaseLLM|from llama_index|import llama_index|metadata\.model_name|LLMMetadata" . --glob '!*.lock' --glob '!node_modules/**' || trueRepository: Das-rebel/a3m-router
Length of output: 219
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== locate file =="
fd -a 'llamaindex.py' . || true
echo "== relevant file =="
if [ -f adapters/a3m_adapter/adapter/llamaindex.py ]; then
wc -l adapters/a3m_adapter/adapter/llamaindex.py
sed -n '1,170p' adapters/a3m_adapter/adapter/llamaindex.py | cat -n
fi
echo "== search BaseLLM usage in repo =="
rg -n "A3MLlamaIndexAdapter|LLMInterface|BaseLLM|from llama_index|import llama_index|metadata\.model_name|LLMMetadata" . --glob '!*.lock' --glob '!node_modules/**' || trueRepository: Das-rebel/a3m-router
Length of output: 10556
🌐 Web query:
LlamaIndex BaseLLM metadata property returns LLMMetadata __repr__ doc
💡 Result:
In LlamaIndex, the BaseLLM abstract base class defines a required metadata property that returns an instance of LLMMetadata [1][2]. This LLMMetadata object provides essential information about the LLM, such as the model name, context window size, and capabilities (e.g., whether it is a chat model or supports structured output) [1][3][4]. Regarding the repr documentation for LLMMetadata: LLMMetadata is a Pydantic model (inheriting from BaseModel or defined using Pydantic fields) within the LlamaIndex core library [1][4][5]. As a Pydantic model, its repr is automatically generated by Pydantic to display the class name and the current values of its fields [5]. LlamaIndex developers have actively worked to improve the documentation of these fields by adding descriptions to the Pydantic Field definitions [6]. Key technical points: - The metadata property in BaseLLM is an abstract property that must be implemented by all subclasses [1]. - When implemented, it returns an LLMMetadata object containing attributes such as model_name, context_window, num_output, and is_chat_model [4][5]. - The to_payload() method in BaseLLM uses this metadata property (via model_dump()) to generate a non-sensitive representation of the LLM for observability, ensuring that sensitive information like API keys is excluded [1].
Citations:
- 1: https://github.com/run-llama/llama_index/blob/7fd33e00/llama-index-core/llama_index/core/base/llms/base.py
- 2: https://github.com/run-llama/LlamaIndexTS/blob/09ba5aa43a21e8f888ab02f6f9a536fe2762ef79/packages/core/src/llms/base.ts
- 3: https://run-llama-llamaindexts.mintlify.app/api/core/llms
- 4: https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/llms/llama-index-llms-ollama/llama_index/llms/ollama/base.py
- 5: https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/base/llms/types.py
- 6: Documenting
LLMMetadatarun-llama/llama_index#8144
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== dependency files mentioning llama-index =="
rg -n "llama-index|llama_index|llaimd" pyproject.toml poetry.lock requirements*.txt setup.py setup.cfg Pipfile Pipfile.lock uv.lock pdm.lock . --glob '*lock' --glob '!node_modules/**' || true
echo "== test consumer around metadata.model_name =="
sed -n '1,120p' adapters/a3m_adapter/tests/test_adapters.py | cat -nRepository: Das-rebel/a3m-router
Length of output: 4569
Implement the full LlamaIndex BaseLLM interface.
A3MLlamaIndexAdapter only returns LlamaIndex types for CompletionResponse and ChatMessage, but BaseLLM requires abstract methods and a metadata property of type LLMMetadata; this adapter exposes metadata as a Dict[str, Any]. Inherit/conform to BaseLLM and return a version-compatible LLMMetadata instance here and at the metadata implementation around lines 89-97.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@adapters/a3m_adapter/adapter/llamaindex.py` around lines 38 - 44, Update
A3MLlamaIndexAdapter to fully conform to LlamaIndex BaseLLM: inherit from
BaseLLM, implement all required abstract methods, and change both metadata
implementations to return a version-compatible LLMMetadata instance rather than
Dict[str, Any]. Preserve the existing CompletionResponse and ChatMessage
behavior while satisfying BaseLLM’s metadata contract.
| print("Testing LangChain adapter...") | ||
|
|
||
| try: | ||
| from a3m_llm_adapter import A3MChatModel |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Use the published package imports in all smoke tests.
These imports reference legacy modules and class names. They do not match the package metadata, README, or PR public API. The script reports failed imports when run manually.
adapters/a3m_adapter/tests/test_adapters.py#L26-L26: importA3MLangChainAdapterfroma3m_adapter.adapters/a3m_adapter/tests/test_adapters.py#L47-L47: importA3MLlamaIndexAdapterfroma3m_adapter.adapters/a3m_adapter/tests/test_adapters.py#L67-L67: importA3MConfigfroma3m_adapter.
📍 Affects 1 file
adapters/a3m_adapter/tests/test_adapters.py#L26-L26(this comment)adapters/a3m_adapter/tests/test_adapters.py#L47-L47adapters/a3m_adapter/tests/test_adapters.py#L67-L67
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@adapters/a3m_adapter/tests/test_adapters.py` at line 26, Update
adapters/a3m_adapter/tests/test_adapters.py at lines 26-26, 47-47, and 67-67 to
use the published package API: import A3MLangChainAdapter, A3MLlamaIndexAdapter,
and A3MConfig respectively from a3m_adapter, replacing the legacy module and
class imports.
| except Exception as e: | ||
| print(f"��❌ LangChain adapter failed: {e}") | ||
| return False |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect pytest configuration and test functions that return success/failure values.
fd -HI '^(pytest\.ini|pyproject\.toml|setup\.cfg|tox\.ini)$' . \
-x sh -c 'echo "== $1 =="; rg -n -C2 "pytest|filterwarnings" "$1" || true' sh {}
rg -n -C3 --glob '*.py' \
'def test_|except Exception|return (True|False)' \
adapters/a3m_adapter/testsRepository: Das-rebel/a3m-router
Length of output: 3548
🌐 Web query:
pytest test function return value ignored except assertion error documentation
💡 Result:
In pytest, test function return values are ignored by the framework [1][2]. Because pytest does not use these return values to determine the outcome of a test, returning a value (such as a boolean) will not cause a test to pass or fail [1][2]. To address this, pytest emits a PytestReturnNotNoneWarning whenever a test function returns a value other than None [1][2]. This warning is intended to help developers avoid the common mistake of assuming a returned boolean value will trigger an assertion [1][2]. Instead of returning values, you should use the standard Python assert statement to verify expectations [1][3]. If you have a legitimate need for a test function to return a value (for example, when using certain specialized plugins), you can suppress this warning by configuring the filterwarnings option in your pytest configuration file (such as pytest.ini, pyproject.toml, or tox.ini) [4]. Example of incorrect usage (which triggers the warning): def test_foo: return some_value == expected # The test will not fail if this is False Correct usage: def test_foo: assert some_value == expected # This will correctly fail if False
Citations:
- 1: https://docs.pytest.org/en/stable/how-to/assert.html
- 2: https://pytest.org/en/latest/how-to/assert.html
- 3: https://docs.pytest.org/en/8.3.x/how-to/assert.html
- 4: Is there a way to opt out of PytestReturnNotNoneWarning ? pytest-dev/pytest#10427
Do not treat returned booleans as pytest failures.
Pytest ignores test return values, so these return False paths cannot fail CI. Let the test assertions in test_langchain_adapter, test_llamaindex_adapter, and test_config raise normally.
🧰 Tools
🪛 Ruff (0.16.1)
[warning] 38-38: Do not catch blind exception: Exception
(BLE001)
📍 Affects 1 file
adapters/a3m_adapter/tests/test_adapters.py#L38-L40(this comment)adapters/a3m_adapter/tests/test_adapters.py#L58-L60adapters/a3m_adapter/tests/test_adapters.py#L85-L87
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@adapters/a3m_adapter/tests/test_adapters.py` around lines 38 - 40, Remove the
return False failure paths from test_langchain_adapter, test_llamaindex_adapter,
and test_config in adapters/a3m_adapter/tests/test_adapters.py at lines 38-40,
58-60, and 85-87; let their assertions or raised exceptions propagate so pytest
correctly marks failures.
Source: Linters/SAST tools
| metadata = llm.metadata | ||
| print(f"��✅ Metadata: {metadata.model_name}, tokens: {metadata.num_output}") |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Call metadata() and read the returned dictionary.
A3MLlamaIndexAdapter.metadata returns a dictionary. The current code stores the bound method, then accesses nonexistent attributes. This test fails after the import is corrected.
Proposed fix
- metadata = llm.metadata
- print(f"��✅ Metadata: {metadata.model_name}, tokens: {metadata.num_output}")
+ metadata = llm.metadata()
+ print(
+ f"��✅ Metadata: {metadata['model_name']}, "
+ f"tokens: {metadata['num_output']}"
+ )📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| metadata = llm.metadata | |
| print(f"��✅ Metadata: {metadata.model_name}, tokens: {metadata.num_output}") | |
| metadata = llm.metadata() | |
| print( | |
| f"��✅ Metadata: {metadata['model_name']}, " | |
| f"tokens: {metadata['num_output']}" | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@adapters/a3m_adapter/tests/test_adapters.py` around lines 54 - 55, Update the
metadata handling in the test to invoke the adapter’s metadata() method and use
the returned dictionary values instead of treating metadata as an object with
model_name and num_output attributes; preserve the existing diagnostic output
using the dictionary keys.
…docs Major updates: - Rewrote README with simple explanations + parallel ensemble examples - Added multi-provider parallel execution examples (Groq + OpenAI + DeepSeek) - Added memory capability documentation (semantic cache, context, cross-session) - Added LangChain, LlamaIndex, CrewAI integration examples - Created llms.txt (LLM indexable summary) - Created docs/llms-full.txt (full technical reference) - Added 100+ keywords to package.json - Updated package to v2.15.4 New README structure: - TL;DR quick start - Parallel ensemble examples (not just OpenAI) - Cost comparison tables - Memory system documentation - Multi-agent examples (CrewAI) - Provider coverage table - CLI commands
Das-rebel
changed the title
…ters Added: 1. demo.py - Interactive demo script showing: - Simple auto-routing - Parallel ensemble (Groq + OpenAI + DeepSeek) - Code generation - Complex reasoning - Health check 2. docker-compose.yml - Instant deployment with: - A3M Router server - Optional Redis for distributed cache - Optional Prometheus + Grafana for monitoring 3. prometheus.yml - Metrics scraping config 4. GitHub Actions CI (adapters-ci.yml): - Tests on Python 3.9, 3.10, 3.11, 3.12 - Linting (black, isort, flake8) - Unit tests - Integration tests (requires server) - PyPI publish on tag - Docker build and push 5. Integration tests (test_integration.py) 6. Requirements files
Das-rebel
changed the title
There was a problem hiding this comment.
Actionable comments posted: 14
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/adapters-ci.yml:
- Around line 119-142: Restrict the docker-build registry login and image push
to pushes on the main branch, not merely any push event; update the “Push to
GHCR” condition and apply the same guard to the preceding GHCR login command so
feature-branch builds never publish the latest tag.
- Around line 15-16: Set workflow-wide GITHUB_TOKEN permissions to contents:
read, then add packages: write specifically to the docker-build job that
authenticates with GHCR and pushes the image. Keep all other jobs limited to the
global read-only contents permission.
- Around line 3-8: Update the workflow trigger configuration under push so tag
pushes are included alongside the existing branch and path filters, allowing
publish-adapters to evaluate its tag condition and publish versioned releases.
In `@adapters/a3m_adapter/tests/test_integration.py`:
- Around line 19-25: Replace the bare except in the A3M Router health-check
setup with an except requests.RequestException clause, preserving the existing
pytest.skip behavior for request failures.
- Around line 17-25: Update skip_if_no_server to use the configured
a3m_server_url when building the health-check request instead of hardcoding
localhost:8787. Preserve the existing skip behavior for non-200 responses and
request failures.
In `@demo.py`:
- Around line 218-226: Update the server preflight around the requests.get call
to invoke resp.raise_for_status() before reporting success, and make the except
path terminate the demo with a nonzero exit status after printing the existing
guidance. Do not continue into the HTTP fallback branches when the localhost
health check fails.
- Around line 50-54: The A3M SDK usage in demo.py does not match the installed
client contract. Update demo.py at lines 50-54, 78-93, 128-132, 157-161, and
182-189 to import and construct the supported client with base_url, call route()
with a single query string, and replace unsupported model, messages,
ensemble_config, and get_health() usage; alternatively route these flows through
the HTTP API.
In `@docker-compose.yml`:
- Around line 29-31: Remove the a3m-config.json bind mount from the base Compose
volumes configuration, and provide the optional /app/config.json mount only
through an explicit Compose override while preserving read-only behavior.
- Line 6: Replace the mutable image tags with immutable, tested digests for the
router image at docker-compose.yml:6-6, Redis at docker-compose.yml:46-46,
Grafana at docker-compose.yml:60-60, and Prometheus at docker-compose.yml:77-77;
preserve each image repository while pinning the exact CI-built image digest.
- Around line 64-65: Update the Grafana environment configuration in the
monitoring profile to source GF_SECURITY_ADMIN_PASSWORD from a required
deployment secret rather than the hardcoded admin value. Ensure the secret must
be provided at deployment time while preserving the existing Grafana password
variable.
In `@docs/llms-full.txt`:
- Around line 11-51: Use the routing specification implemented by src/sdk.ts as
the sole canonical reference: update docs/llms-full.txt lines 11-51 to use the
implementation’s four signals and tier thresholds below 0.20, 0.45, and 0.65;
update README.md lines 194-210, docs/llms.txt lines 9-13, and llms.txt lines
9-13 to match those same signals and ranges. Ensure all four files describe
identical routing behavior.
In `@docs/llms.txt`:
- Around line 76-80: Update both CrewAI examples in docs/llms.txt (lines 76-80)
and llms.txt (lines 76-80) by adding the Agent import from crewai before
instantiating Agent; apply the same change at both sites so each snippet runs
independently.
In `@package.json`:
- Line 3: Regenerate package-lock.json from the package.json manifest so its
root package version matches 2.15.4 instead of 2.14.60, and commit the resulting
lockfile metadata without changing unrelated dependencies.
- Around line 5-8: Update the package.json main and bin entry points to
reference the rebuilt dist files included in the package, replacing src/index.js
and the current bin/cli.js targets. Ensure imports, require('a3m-router'), and
both CLI commands resolve to the actual packaged dist entry points.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: b7ccb91d-6164-4d02-abb8-505657bc388e
📒 Files selected for processing (12)
.github/workflows/adapters-ci.ymlREADME.mdadapters/a3m_adapter/tests/test_integration.pyadapters/requirements-dev.txtadapters/requirements.txtdemo.pydocker-compose.ymldocs/llms-full.txtdocs/llms.txtllms.txtpackage.jsonprometheus.yml
| on: | ||
| push: | ||
| branches: [main, feat/*] | ||
| paths: | ||
| - 'adapters/**' | ||
| - '.github/workflows/adapters-ci.yml' |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
if [ -f .github/workflows/adapters-ci.yml ]; then
echo "== adapters-ci.yml lines 1-140 =="
nl -ba .github/workflows/adapters-ci.yml | sed -n '1,140p'
else
echo "missing .github/workflows/adapters-ci.yml"
fi
echo
echo "== github actions trigger/tag publish context =="
python3 - <<'PY'
from pathlib import Path
p = Path(".github/workflows/adapters-ci.yml")
if not p.exists():
raise SystemExit
text = p.read_text()
start = text.index("on:")
end = text.find("\n", start)
print("on branch:", text[start:end])
print("tags present in workflows:", any("tags:" in line for line in text.splitlines()))
print("publish-adapters occurrences:", [i+1 for i,l in enumerate(text.splitlines()) if "publish-adapters" in l or "PYPI_VERSION" in l or "if: startsWith(github.ref" in l])
for i,l in enumerate(text.splitlines(), 1):
if "publish-adapters" in l or "PYPI_VERSION" in l or "if: startsWith(github.ref" in l:
print(f"{i}: {l}")
PYRepository: Das-rebel/a3m-router
Length of output: 232
🏁 Script executed:
#!/bin/bash
set -euo pipefail
if [ -f .github/workflows/adapters-ci.yml ]; then
echo "== adapters-ci.yml lines 1-140 =="
nl -ba .github/workflows/adapters-ci.yml | sed -n '1,140p'
else
echo "missing .github/workflows/adapters-ci.yml"
fi
echo
echo "== workflow trigger/tag publish context =="
python3 - <<'PY'
from pathlib import Path
p = Path(".github/workflows/adapters-ci.yml")
if not p.exists():
raise SystemExit
text = p.read_text()
start = text.index("on:")
end = text.find("\n", start)
print("on branch:", text[start:end])
print("tags present in workflows:", any("tags:" in line for line in text.splitlines()))
print("publish-adapters occurrences:", [i+1 for i,l in enumerate(text.splitlines()) if "publish-adapters" in l or "PYPI_VERSION" in l or "if: startsWith(github.ref" in l])
for i,l in enumerate(text.splitlines(), 1):
if "publish-adapters" in l or "PYPI_VERSION" in l or "if: startsWith(github.ref" in l:
print(f"{i}: {l}")
PYRepository: Das-rebel/a3m-router
Length of output: 232
🏁 Script executed:
#!/bin/bash
set -euo pipefail
if [ -f .github/workflows/adapters-ci.yml ]; then
awk '{printf "%6d\t%s\n", NR, $0}' .github/workflows/adapters-ci.yml | sed -n '1,140p'
else
echo "missing .github/workflows/adapters-ci.yml"
fi
python3 - <<'PY'
from pathlib import Path
p = Path(".github/workflows/adapters-ci.yml")
if not p.exists():
raise SystemExit
text = p.read_text()
print("\nOn block:")
in_on = False
for line in text.splitlines():
if line == "on:":
in_on = True
print(line)
continue
if in_on:
if line and not line.startswith(' ') and not line.startswith('\t') and line != 'on:':
break
print(line)
print("\nRelevant occurrences:")
for i, line in enumerate(text.splitlines(), 1):
if any(s in line for s in ["publish-adapters", "PYPI_VERSION", "startsWith(github.ref", "github.ref == 'refs/tags/'"]):
print(f"{i}: {line}")
PYRepository: Das-rebel/a3m-router
Length of output: 5132
Enable tag-triggered workflow runs.
push.branches filters out tag pushes, so publish-adapters never reaches its tag condition and cannot publish a versioned release.
Proposed fix
push:
branches: [main, feat/*]
+ tags: ['v*']📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| on: | |
| push: | |
| branches: [main, feat/*] | |
| paths: | |
| - 'adapters/**' | |
| - '.github/workflows/adapters-ci.yml' | |
| on: | |
| push: | |
| branches: [main, feat/*] | |
| tags: ['v*'] | |
| paths: | |
| - 'adapters/**' | |
| - '.github/workflows/adapters-ci.yml' |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/adapters-ci.yml around lines 3 - 8, Update the workflow
trigger configuration under push so tag pushes are included alongside the
existing branch and path filters, allowing publish-adapters to evaluate its tag
condition and publish versioned releases.
| jobs: | ||
| test-adapters: |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== workflow excerpt =="
sed -n '1,170p' .github/workflows/adapters-ci.yml | nl -ba
echo
echo "== diff stat/name only relevant workflow =="
git diff -- .github/workflows/adapters-ci.yml | sed -n '1,220p'
echo
echo "== permissions references in workflow =="
python3 - <<'PY'
from pathlib import Path
p=Path('.github/workflows/adapters-ci.yml')
for i,line in enumerate(p.read_text().splitlines(), 1):
if 'permissions:' in line or 'GITHUB_TOKEN' in line or 'packages:' in line:
print(f'{i}: {line}')
PYRepository: Das-rebel/a3m-router
Length of output: 221
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== workflow excerpt =="
sed -n '1,170p' .github/workflows/adapters-ci.yml | awk '{printf "%6d %s\n", NR, $0}'
echo
echo "== diff stat and relevant hunk =="
git diff -- .github/workflows/adapters-ci.yml | head -n 220 || true
echo
echo "== permissions/token/package references in workflow =="
python3 - <<'PY'
from pathlib import Path
p = Path('.github/workflows/adapters-ci.yml')
for i, line in enumerate(p.read_text().splitlines(), 1):
if any(s in line for s in ('permissions:', 'GITHUB_TOKEN', 'packages:', 'ghcr.io', 'docker/login-action', 'docker/build-push-action')):
print(f'{i}: {line}')
PY
echo
echo "== jobs named docker-build and uses of packages permissions =="
python3 - <<'PY'
from pathlib import Path
text = Path('.github/workflows/adapters-ci.yml').read_text()
lines = text.splitlines()
for i, line in enumerate(lines, 1):
if line.rstrip() == 'docker-build:':
print(f'job docker-build starts around line {i}')
# Print context until next top-level key
for j in range(i, min(len(lines), i + 80)):
print(f'{j}: {lines[j-1]}')
if lines[j-1].startswith(' ') and not lines[j-1].strip().startswith('-') and lines[j-1][0] in ' \t':
# simple continuation until file end or next top-level (not indent under job?) keep until next top-level key
pass
break
PYRepository: Das-rebel/a3m-router
Length of output: 5594
Set least-privilege token permissions.
The workflow uses repository-default GITHUB_TOKEN permissions. Set contents: read globally and add packages: write only on docker-build, which logs in to GHCR and pushes the image.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/adapters-ci.yml around lines 15 - 16, Set workflow-wide
GITHUB_TOKEN permissions to contents: read, then add packages: write
specifically to the docker-build job that authenticates with GHCR and pushes the
image. Keep all other jobs limited to the global read-only contents permission.
Source: Linters/SAST tools
| docker-build: | ||
| runs-on: ubuntu-latest | ||
| needs: test-integration | ||
| if: github.event_name == 'push' | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Build Docker image | ||
| run: | | ||
| docker build -t ghcr.io/das-rebel/a3m-router:latest . | ||
|
|
||
| - name: Run container health check | ||
| run: | | ||
| docker run -d --name a3m-test -p 8787:8787 ghcr.io/das-rebel/a3m-router:latest | ||
| sleep 5 | ||
| curl -f http://localhost:8787/health | ||
| docker stop a3m-test | ||
|
|
||
| - name: Push to GHCR | ||
| if: github.event_name == 'push' | ||
| run: | | ||
| echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin | ||
| docker push ghcr.io/das-rebel/a3m-router:latest |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== files =="
git ls-files .github/workflows/adapters-ci.yml || true
if [ -f .github/workflows/adapters-ci.yml ]; then
echo "== workflow excerpt =="
sed -n '1,220p' .github/workflows/adapters-ci.yml | cat -n
fi
echo "== search branch filters and docker-push usage =="
rg -n "branches:|branches-ignore|paths-ignore|docker-build|docker push|ghcr.io/das-rebel/a3m-router|GITHUB_TOKEN|github.event_name|github.ref_name|github.ref" .github/workflows || trueRepository: Das-rebel/a3m-router
Length of output: 6244
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
import re
p = Path(".github/workflows/adapters-ci.yml")
text = p.read_text() if p.exists() else ""
print("== docker-build step names ==")
m = re.search(r'^ docker-build:(\n[ \t]+.*?)(?:\n [a-zA-Z0-9_-]+:|\Z)', text, re.S)
print(m.group(1) if m else "not found")
print("== condition on triggers containing 'push' ==")
for rexp in [
r'^on:\s*$\n((?:(?:[ \t]+[^\n]+)|(?:[ \t]+push:[^\n]*)|(?:(?:\n[ \t][ \t-]?(?:branches|branches-ignore|paths|paths-ignore):[^\n]*)|(?:[ \t]+(?:(?:[^\n])))*)+)*)',
r'^[ \t]*push:(.*?)(?=\n\n|\n [a-zA-Z0-9_-]+:|\Z)',
r'^[ \t]{2}branches:(.*?)(?=\n branches-ignore:|\n[ \t]{2}[a-zA-Z0-9_-]+:|\Z)',
]:
alls = re.findall(rexp, text, re.S | re.M)
if alls:
print(rexp[:60], "-", alls[:5])
print("== deterministic parse-ish indicators ==")
checks = {
"workflow_file_exists": p.exists(),
"push_trigger": bool(re.search(r'^ *push:', text, flags=re.M)),
"docker_push_exists": "docker push ghcr.io/das-rebel/a3m-router:latest" in text,
"login_exists": "docker login ghcr.io" in text,
"checks_main_or_all_branches": any(x in text for x in ["ghcr.io/das-rebel/a3m-router:latest", "latest", "branches:"] and not re.search(r"branches:\n[ \t]+- *refs/heads/[^\\n]*main|branches:\n[ \t]+- *[\*$]|\n\s*branches-ignore:", text)),
}
for k,v in checks.items():
print(f"{k}={v}")
PYRepository: Das-rebel/a3m-router
Length of output: 641
Do not publish latest from feature branches.
docker-build runs on push events from main and feat/*, then pushes ghcr.io/das-rebel/a3m-router:latest. This can overwrite latest with unreviewed feature-branch code.
Restrict the registry login and push steps to main, or publish immutable branch-specific tags instead.
🧰 Tools
🪛 zizmor (1.29.0)
[warning] 125-125: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[warning] 119-143: overly broad permissions (excessive-permissions): default permissions used due to no permissions: block
(excessive-permissions)
[error] 141-141: code injection via template expansion (template-injection): may expand into attacker-controllable code
(template-injection)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/adapters-ci.yml around lines 119 - 142, Restrict the
docker-build registry login and image push to pushes on the main branch, not
merely any push event; update the “Push to GHCR” condition and apply the same
guard to the preceding GHCR login command so feature-branch builds never publish
the latest tag.
| def skip_if_no_server(): | ||
| """Skip test if server is not available.""" | ||
| import requests | ||
| try: | ||
| resp = requests.get("http://localhost:8787/health", timeout=2) | ||
| if resp.status_code != 200: | ||
| pytest.skip("A3M Router server not running") | ||
| except: | ||
| pytest.skip("A3M Router server not running") |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Use a3m_server_url for the readiness check.
skip_if_no_server always checks localhost:8787. When A3M_SERVER_URL targets another host, the fixture skips valid integration tests if localhost is unavailable.
Proposed fix
`@pytest.fixture`
-def skip_if_no_server():
+def skip_if_no_server(a3m_server_url):
"""Skip test if server is not available."""
import requests
try:
- resp = requests.get("http://localhost:8787/health", timeout=2)
+ resp = requests.get(f"{a3m_server_url}/health", timeout=2)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def skip_if_no_server(): | |
| """Skip test if server is not available.""" | |
| import requests | |
| try: | |
| resp = requests.get("http://localhost:8787/health", timeout=2) | |
| if resp.status_code != 200: | |
| pytest.skip("A3M Router server not running") | |
| except: | |
| pytest.skip("A3M Router server not running") | |
| def skip_if_no_server(a3m_server_url): | |
| """Skip test if server is not available.""" | |
| import requests | |
| try: | |
| resp = requests.get(f"{a3m_server_url}/health", timeout=2) | |
| if resp.status_code != 200: | |
| pytest.skip("A3M Router server not running") | |
| except: | |
| pytest.skip("A3M Router server not running") |
🧰 Tools
🪛 Ruff (0.16.1)
[error] 24-24: Do not use bare except
(E722)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@adapters/a3m_adapter/tests/test_integration.py` around lines 17 - 25, Update
skip_if_no_server to use the configured a3m_server_url when building the
health-check request instead of hardcoding localhost:8787. Preserve the existing
skip behavior for non-200 responses and request failures.
| import requests | ||
| try: | ||
| resp = requests.get("http://localhost:8787/health", timeout=2) | ||
| if resp.status_code != 200: | ||
| pytest.skip("A3M Router server not running") | ||
| except: | ||
| pytest.skip("A3M Router server not running") |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Catch requests.RequestException instead of using a bare except.
The bare except violates Flake8 E722. The test-adapters job runs Flake8 on this test directory, so CI fails before tests run.
Proposed fix
- except:
+ except requests.RequestException:
pytest.skip("A3M Router server not running")📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| import requests | |
| try: | |
| resp = requests.get("http://localhost:8787/health", timeout=2) | |
| if resp.status_code != 200: | |
| pytest.skip("A3M Router server not running") | |
| except: | |
| pytest.skip("A3M Router server not running") | |
| import requests | |
| try: | |
| resp = requests.get("http://localhost:8787/health", timeout=2) | |
| if resp.status_code != 200: | |
| pytest.skip("A3M Router server not running") | |
| except requests.RequestException: | |
| pytest.skip("A3M Router server not running") |
🧰 Tools
🪛 Ruff (0.16.1)
[error] 24-24: Do not use bare except
(E722)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@adapters/a3m_adapter/tests/test_integration.py` around lines 19 - 25, Replace
the bare except in the A3M Router health-check setup with an except
requests.RequestException clause, preserving the existing pytest.skip behavior
for request failures.
Source: Linters/SAST tools
| environment: | ||
| - NODE_ENV=production | ||
| - PORT=8787 | ||
| - ROUTER_BASE_URL=http://a3m-router:3000 | ||
| - GF_SECURITY_ADMIN_PASSWORD=admin |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Do not commit a known Grafana administrator password.
The monitoring profile exposes Grafana on host port 3000. GF_SECURITY_ADMIN_PASSWORD=admin permits trivial administrator access. Read this value from a required deployment secret instead.
Proposed fix
- - GF_SECURITY_ADMIN_PASSWORD=admin
+ - GF_SECURITY_ADMIN_PASSWORD=${GF_SECURITY_ADMIN_PASSWORD:?Set GF_SECURITY_ADMIN_PASSWORD}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| environment: | |
| - NODE_ENV=production | |
| - PORT=8787 | |
| - ROUTER_BASE_URL=http://a3m-router:3000 | |
| - GF_SECURITY_ADMIN_PASSWORD=admin | |
| environment: | |
| - GF_SECURITY_ADMIN_PASSWORD=${GF_SECURITY_ADMIN_PASSWORD:?Set GF_SECURITY_ADMIN_PASSWORD} |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docker-compose.yml` around lines 64 - 65, Update the Grafana environment
configuration in the monitoring profile to source GF_SECURITY_ADMIN_PASSWORD
from a required deployment secret rather than the hardcoded admin value. Ensure
the secret must be provided at deployment time while preserving the existing
Grafana password variable.
| Five signals are combined into a composite score: | ||
|
|
||
| | Metric | Value | | ||
| |--------|-------| | ||
| | Score | 0.9404 | | ||
| | Accuracy | 96.77% | | ||
| | Avg Cost / 1K tokens | $0.0768 | | ||
| | Robustness | 1.0000 | | ||
| | Abnormal entries | 0 | | ||
| | Queries evaluated | 8,400 | | ||
| 1. **Domain Detection** | ||
| - Legal: contract, lawsuit, compliance, patent | ||
| - Medical: diagnosis, treatment, prescription, symptoms | ||
| - Code: function, class, API, debugging, refactor | ||
| - Finance: investment, portfolio, risk, return, audit | ||
| - ML: training, inference, gradient, loss, model | ||
|
|
||
| Internal evaluation on 8,400 queries from diverse domains. | ||
| 2. **Task Classification** | ||
| - Code generation: write, implement, create function | ||
| - Translation: translate, convert, rewrite in | ||
| - Analysis: compare, evaluate, assess, analyze | ||
| - Creative: write story, poem, generate idea | ||
| - Factual: what is, who was, when did, where is | ||
|
|
||
| ### Official Baseline Status | ||
| 3. **Structural Analysis** | ||
| - Clause count: complex sentences | ||
| - Explicit steps: first...then, step 1/2/3 | ||
| - Qualifications: might, could, possibly | ||
| - Conditional: if...then, unless, provided that | ||
|
|
||
| | Benchmark | Venue | Status | Reference | | ||
| | Parallel Routing | Internal eval | 67% exact match | | ||
| | Cost vs all-premium | Internal eval | 62.9% savings | | ||
| | RouterEval | EMNLP 2025 | Baseline merged | MilkThink-Lab/RouterEval#4 | | ||
| | MMR-Bench | ArXiv 2026 | Baseline merged | Hunter-Wrynn/MMR-Bench#4 | | ||
| | LLMRouterBench | ACL 2026 | Submitted | ynulihao/LLMRouterBench#3 | | ||
| 4. **Verb Intensity** | ||
| - Complex verbs: design, architect, optimize, synthesize | ||
| - Simple verbs: what, who, find, get | ||
|
|
||
| ### Local Evaluation | ||
| 5. **Multi-Modal Hints** | ||
| - Image references: explain this diagram | ||
| - Code blocks: debug this function | ||
| - Data: analyze this dataset | ||
|
|
||
| | Metric | Value | | ||
| |--------|-------| | ||
| | Exact tier match | 67% | | ||
| | Within 1 tier | 96% | | ||
| | Cost savings vs all-premium | 62.9% | | ||
| ### Tier Assignment | ||
|
|
||
| --- | ||
| Score maps to tier: | ||
|
|
||
| ## Architecture | ||
| | Score Range | Tier | Providers | Example | | ||
| |------------|------|-----------|---------| | ||
| | 0-20 | Free | Ollama, Llama.cpp | Simple what/who | | ||
| | 21-40 | Cheap | Groq, DeepSeek, Mistral | Short code, basic QA | | ||
| | 41-70 | Mid | GPT-4o-mini, Claude-haiku | Standard tasks | | ||
| | 71-100 | Premium | GPT-4o, Claude-sonnet, Gemini | Complex reasoning | |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Use one routing specification in all documentation.
The documents disagree on the signal count. docs/llms-full.txt specifies five signals, while the other files specify four. Its score bands also conflict with src/sdk.ts, which assigns tiers at complexity values below 0.20, 0.45, and 0.65.
docs/llms-full.txt#L11-L51: derive the signals and tier ranges from the implementation.README.md#L194-L210: match the canonical signals and tier ranges.docs/llms.txt#L9-L13: match the canonical signals and tier ranges.llms.txt#L9-L13: match the canonical signals and tier ranges.
📍 Affects 4 files
docs/llms-full.txt#L11-L51(this comment)README.md#L194-L210docs/llms.txt#L9-L13llms.txt#L9-L13
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/llms-full.txt` around lines 11 - 51, Use the routing specification
implemented by src/sdk.ts as the sole canonical reference: update
docs/llms-full.txt lines 11-51 to use the implementation’s four signals and tier
thresholds below 0.20, 0.45, and 0.65; update README.md lines 194-210,
docs/llms.txt lines 9-13, and llms.txt lines 9-13 to match those same signals
and ranges. Ensure all four files describe identical routing behavior.
| "version": "2.15.3", | ||
| "shortName": "A3M Router", | ||
| "displayName": "A3M Router - Adaptive Memory Multi-Model Router", | ||
| "version": "2.15.4", |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'package.json version: '
jq -r '.version' package.json
printf 'package-lock.json root version: '
jq -r '.packages[""].version // .version' package-lock.jsonRepository: Das-rebel/a3m-router
Length of output: 227
Regenerate package-lock.json from this manifest.
package.json version is 2.15.4, but the lockfile root version still declares 2.14.60. Run a lockfile regeneration and commit the updated results so install metadata matches release 2.15.4.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@package.json` at line 3, Regenerate package-lock.json from the package.json
manifest so its root package version matches 2.15.4 instead of 2.14.60, and
commit the resulting lockfile metadata without changing unrelated dependencies.
| "main": "src/index.js", | ||
| "bin": { | ||
| "a3m-router": "dist/cli.js", | ||
| "a3m": "dist/tui/index.js", | ||
| "a3m-tui": "dist/tui/index.js", | ||
| "adaptive-memory-multi-model-router": "dist/cli.js" | ||
| "a3m-router": "./bin/cli.js", | ||
| "a3m": "./bin/cli.js" |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the package entry-point contract and source targets.
jq '{version, main, bin, exports, type, files}' package.json
jq -r '.main, (.bin | if type == "string" then . else .[] end)' package.json |
while IFS= read -r path; do
test -f "$path" && printf 'found: %s\n' "$path" || printf 'missing: %s\n' "$path"
doneRepository: Das-rebel/a3m-router
Length of output: 404
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "## package.json relevant fields"
jq '{version, main, bin, exports, type, files}' package.json
echo
echo "## package-lock version metadata"
jq -r '.packages[""].version, .packages[""].main, .packages[""].bin, .packages[""].exports' package-lock.json 2>/dev/null || jq -r '.version, .packages[""].version, .packages[""].main, .packages[""].bin, .packages[""].exports' package-lock.json
echo
echo "## root package.json diff"
git diff -- package.json | sed -n '1,120p'
echo
echo "## package-lock root diff"
git diff -- package-lock.json | sed -n '1,160p'
echo
echo "## candidate cli/index files"
for f in src/index.js bin/cli.js package.json package-lock.json; do
if [ -f "$f" ]; then
echo "FOUND $f"
else
echo "MISSING $f"
fi
done
echo
echo "## tracked files containing cli/index patterns"
git ls-files | rg '(^package\.json$|^package-lock\.json$|^src/|^bin/|cli|index|router)' | sed -n '1,200p'
echo
echo "## git status short"
git status --shortRepository: Das-rebel/a3m-router
Length of output: 5133
Fix the resolved entry points.
package.json points consumers to src/index.js, but the checked-out package still contains dist/dist/tui/index.js (dist only) and an empty module.exports or fallback path in src/index.js. Consumers using imports or require('a3m-router') may get the wrong module or an unresolved export. Point main/binary targets at the rebuilt dist files that exist in the package.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@package.json` around lines 5 - 8, Update the package.json main and bin entry
points to reference the rebuilt dist files included in the package, replacing
src/index.js and the current bin/cli.js targets. Ensure imports,
require('a3m-router'), and both CLI commands resolve to the actual packaged dist
entry points.
…ne, LangGraph) New adapters added: - A3MAutoGenAdapter: Microsoft's AutoGen multi-agent framework - A3MVercelAdapter: Vercel AI SDK for Next.js apps - A3MHaystackAdapter: Deepset's Haystack RAG framework - A3MPineconeAdapter: Pinecone vector database + RAG - A3MLangGraphAdapter: LangGraph stateful agents Total now 8 framework adapters: 1. LangChain 2. LlamaIndex 3. AutoGen 4. Vercel AI SDK 5. Haystack 6. Pinecone 7. LangGraph 8. CrewAI Also updated: - README.md with all 8 adapters - llms.txt with adapter documentation - docs/llms-full.txt with detailed examples
Das-rebel
changed the title
# Conflicts: # README.md
1 participant
Complete Framework Adapters + Deployment Suite
Framework Adapters (8 Total)
All Files in This PR
Usage Examples
AutoGen (Microsoft):
Vercel AI SDK:
Haystack RAG:
Pinecone Vector Search:
LangGraph Stateful Agents:
Quick Start
CI/CD