Understanding of project requirements and feature suggestions

[itsbharatj]

Hey @stex2005, thanks for creating this project! The ROS-MCP client has tremendous potential for enhancing integration between the MCP-SERVER and various LLM providers.

As a new contributor, I'd love to understand the broader scope and work on expanding the project's capabilities. Here are some of my suggestions for some key enhancements to improve usability:

---

1. Multi-Provider LLM Support

Proposed: Unified interface supporting multiple providers

Widely Used Providers:

• OpenAI GPT-4/GPT-4o
• Anthropic Claude
• Ollama (local LLMs)
• Cerebras (fast inference)
• Azure OpenAI

Architecture:

clients/
├── base/              # Abstract base + factory pattern
├── openai/            # OpenAI implementation
├── anthropic/         # Claude implementation  
├── ollama/            # Local LLM support
└── gemini_live/       # Existing
.env                             ## API keys of all clients providers

---

2. Orchestration & MCP Server Integration

Multi-MCP server coordination for agentic applications

Key Features:

• Connect to multiple MCP servers simultaneously (ros-mcp-server + others)
• Cross-server tool orchestration and workflow management
• Unified session management across different MCP servers
• Agent coordination for complex multi-domain tasks

Example MCP Server Integrations:

• News-MCP: A specific action is triggered when a certain event occurs
• ros-mcp-server: Robot control and sensor data
• filesystem-mcp: File operations and data management
• web-mcp: Web browsing and API interactions
• database-mcp: Data storage and retrieval
• vision-mcp: Image processing and computer vision

Would like to know your thoughts on the suggested features and overall vision for the project. Thanks

[stex2005]

Hey @stex2005, thanks for creating this project! The ROS-MCP client has tremendous potential for enhancing integration between the MCP-SERVER and various LLM providers.
As a new contributor, I'd love to understand the broader scope and work on expanding the project's capabilities. Here are some of my suggestions for some key enhancements to improve usability:

Hey @itsbharatj,

Thanks a lot for your message, and welcome!
Really appreciate the time you took to outline these ideas; they’re very much aligned with the long-term vision of the ROS-MCP client project.

Overall Vision

The goal of this client is to make the Model Context Protocol (MCP) a universal bridge between robotic systems (ROS/ROS 2) and large-language-model (LLM) backends — enabling LLMs to access robot state, control interfaces, and external data sources through standardized tool schemas.

While the core functionality currently resides in the ros-mcp-server, we want to improve compatibility and usability by adding tutorials and example integrations demonstrating how different LLM/MCP clients can interact with it.

1. Multi-Provider LLM Support

Proposed: Unified interface supporting multiple providers

Absolutely agreed! Abstracting the provider layer via a base LLM client interface is the right next step.
The current implementation focuses on gemini_live (not for a specific reason, just the simplest to set up), but the plan is to refactor toward a structure like:

clients/
├── base/          # Abstract base 
├── openai/        # GPT-4/GPT-4o
├── anthropic/     # Claude
├── ollama/        # Local LLMs
├── gemini/        # Google
└── ...

This would make it far easier to plug in new providers without touching the core ros-mcp sever/client logic.

The idea is to define a shared interface — something like send_message(), stream_events(), and tool_call() — so that provider selection becomes configuration-based rather than code-based.

In other words, a single ros-mcp-client could switch between OpenAI, Gemini, Anthropic, etc., just by passing an argument.

If you’re interested, we could start a multi-provider-interface issue or branch and define the minimal common contract first.

2. Orchestration & Multi-MCP Integration

Proposed: Multi-MCP server coordination for agentic applications

This is another key growth area. The idea of cross-MCP orchestration (e.g., ros-mcp-server + filesystem-mcp + web-mcp) fits perfectly into the agentic architecture of the ROS-MCP.

This would enable scenarios such as:

• Retrieving live sensor data from ros-mcp-server while logging to filesystem-mcp
• Using web-mcp or database-mcp for external information retrieval
• Combining perception and control via vision-mcp and ros-mcp-server

However, for now, I think it’s best to focus on ROS-MCP integration to ensure stability and feature completeness before expanding outward.

3. Next Steps

[ ] Opening a “Multi-Provider-Client-Architecture” issue to track design and interface discussion.
[ ] We can then break them into smaller PRs — starting with a BaseClient class, followed by one provider implementation (OpenAI or Anthropic).
[ ] Continuiing the “Multi-MCP Orchestration” discussion to collect use-cases and propose API features.

@itsbharatj Would love to have you contribute!

Thanks
@stex2005

[itsbharatj]

Hey @stex2005,
Thank you for your message, I now have good understanding of the expectations from the project.

How I see it is that the base class will have all the higher level functions like finding-tools, processing-user-query and running the main loop.

The argument to the base-class will be the {LLM_Provider}_class. It will contain all the low level features for the support in the base class. It will include initializing the LLM, process_message etc.

I would recommend using langgraph for our integration. It comes with prebuilt LLM integrations and support for MCP client.

cross-MCP orchestration (e.g., ros-mcp-server + filesystem-mcp + web-mcp)

This sounds like a great idea, would love to work on more such examples and use cases once we have the base setup completed.