[REF-746] feat: support parallel execution for tools#2036
Conversation
📝 WalkthroughWalkthroughAdds a new config property Changes
Sequence Diagram(s)sequenceDiagram
participant LLM as LLM/Agent
participant ToolNode as Enhanced Tool Node
participant ToolA as Tool A
participant ToolB as Tool B
participant ToolN as Tool N
LLM->>ToolNode: invoke with tool_calls array
ToolNode->>ToolNode: validate calls, map indices
par Parallel execution (rgba(60,179,113,0.5))
ToolNode->>ToolA: execute call (index 0)
ToolNode->>ToolB: execute call (index 1)
ToolNode->>ToolN: execute call (index N)
end
ToolA-->>ToolNode: result {index, ToolMessage} or error
ToolB-->>ToolNode: result {index, ToolMessage} or error
ToolN-->>ToolNode: result {index, ToolMessage} or error
ToolNode->>ToolNode: collect results, sort by index
ToolNode-->>LLM: ordered ToolMessage array for LLM context
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
packages/skill-template/src/skills/agent.ts (1)
285-287: Minor: Line length exceeds 100 characters.Line 286 is 113 characters, exceeding the project's 100-character limit. Consider breaking it into multiple lines.
♻️ Suggested formatting
- this.engine.logger.info( - `Executing ${toolCalls.length} tool calls in parallel: [${toolCalls.map((tc) => tc?.name).join(', ')}]`, - ); + const toolNames = toolCalls.map((tc) => tc?.name).join(', '); + this.engine.logger.info( + `Executing ${toolCalls.length} tool calls in parallel: [${toolNames}]`, + );As per coding guidelines, the maximum line length is 100 characters.
📜 Review details
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
apps/api/src/modules/config/app.config.tsapps/api/src/modules/drive/drive.service.tspackages/skill-template/src/skills/agent.ts
🧰 Additional context used
📓 Path-based instructions (15)
**/*.{js,ts,jsx,tsx}
📄 CodeRabbit inference engine (.cursorrules)
**/*.{js,ts,jsx,tsx}: Always use optional chaining (?.) when accessing object properties
Always use nullish coalescing (??) or default values for potentially undefined values
Always check array existence before using array methods
Always validate object properties before destructuring
Always use single quotes for string literals in JavaScript/TypeScript code
**/*.{js,ts,jsx,tsx}: Use semicolons at the end of statements
Include spaces around operators (e.g.,a + binstead ofa+b)
Always use curly braces for control statements
Place opening braces on the same line as their statement
**/*.{js,ts,jsx,tsx}: Group import statements in order: React/framework libraries, third-party libraries, internal modules, relative path imports, type imports, style imports
Sort imports alphabetically within each import group
Leave a blank line between import groups
Extract complex logic into custom hooks
Use functional updates for state (e.g.,setCount(prev => prev + 1))
Split complex state into multiple state variables rather than single large objects
Use useReducer for complex state logic instead of multiple useState calls
Files:
apps/api/src/modules/config/app.config.tspackages/skill-template/src/skills/agent.tsapps/api/src/modules/drive/drive.service.ts
**/*.{js,ts,tsx,jsx,py,java,cpp,c,cs,rb,go,rs,php,swift,kt,scala,r,m,mm,sql}
📄 CodeRabbit inference engine (.cursor/rules/00-language-priority.mdc)
**/*.{js,ts,tsx,jsx,py,java,cpp,c,cs,rb,go,rs,php,swift,kt,scala,r,m,mm,sql}: All code comments MUST be written in English
All variable names, function names, class names, and other identifiers MUST use English words
Comments should be concise and explain 'why' rather than 'what'
Use proper grammar and punctuation in comments
Keep comments up-to-date when code changes
Document complex logic, edge cases, and important implementation details
Use clear, descriptive names that indicate purpose
Avoid abbreviations unless they are universally understood
Files:
apps/api/src/modules/config/app.config.tspackages/skill-template/src/skills/agent.tsapps/api/src/modules/drive/drive.service.ts
**/*.{js,ts,tsx,jsx}
📄 CodeRabbit inference engine (.cursor/rules/00-language-priority.mdc)
Use JSDoc style comments for functions and classes in JavaScript/TypeScript
Files:
apps/api/src/modules/config/app.config.tspackages/skill-template/src/skills/agent.tsapps/api/src/modules/drive/drive.service.ts
**/*.{js,jsx,ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/01-code-style.mdc)
**/*.{js,jsx,ts,tsx}: Use single quotes for string literals in TypeScript/JavaScript
Always use optional chaining (?.) when accessing object properties in TypeScript/JavaScript
Always use nullish coalescing (??) or default values for potentially undefined values in TypeScript/JavaScript
Always check array existence before using array methods in TypeScript/JavaScript
Validate object properties before destructuring in TypeScript/JavaScript
Use ES6+ features like arrow functions, destructuring, and spread operators in TypeScript/JavaScript
Avoid magic numbers and strings - use named constants in TypeScript/JavaScript
Use async/await instead of raw promises for asynchronous code in TypeScript/JavaScript
Files:
apps/api/src/modules/config/app.config.tspackages/skill-template/src/skills/agent.tsapps/api/src/modules/drive/drive.service.ts
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/03-typescript-guidelines.mdc)
**/*.{ts,tsx}: Avoid usinganytype whenever possible - useunknowntype instead with proper type guards
Always define explicit return types for functions, especially for public APIs
Prefer extending existing types over creating entirely new types
Use TypeScript utility types (Partial<T>,Pick<T, K>,Omit<T, K>,Readonly<T>,Record<K, T>) to derive new types
Use union types and intersection types to combine existing types
Always import types explicitly using theimport typesyntax
Group type imports separately from value imports
Minimize creating local type aliases for imported types
Files:
apps/api/src/modules/config/app.config.tspackages/skill-template/src/skills/agent.tsapps/api/src/modules/drive/drive.service.ts
**/*.{js,ts,jsx,tsx,css,json}
📄 CodeRabbit inference engine (.cursor/rules/04-code-formatting.mdc)
Maximum line length of 100 characters
Files:
apps/api/src/modules/config/app.config.tspackages/skill-template/src/skills/agent.tsapps/api/src/modules/drive/drive.service.ts
**/*.{js,ts,jsx,tsx,css,json,yml,yaml}
📄 CodeRabbit inference engine (.cursor/rules/04-code-formatting.mdc)
Use 2 spaces for indentation, no tabs
Files:
apps/api/src/modules/config/app.config.tspackages/skill-template/src/skills/agent.tsapps/api/src/modules/drive/drive.service.ts
**/*.{js,ts,jsx,tsx,css,json,yml,yaml,md}
📄 CodeRabbit inference engine (.cursor/rules/04-code-formatting.mdc)
No trailing whitespace at the end of lines
Files:
apps/api/src/modules/config/app.config.tspackages/skill-template/src/skills/agent.tsapps/api/src/modules/drive/drive.service.ts
**/*.{css,scss,sass,less,js,jsx,ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/09-design-system.mdc)
**/*.{css,scss,sass,less,js,jsx,ts,tsx}: Primary color (#155EEF) should be used for main brand color in buttons, links, and accents
Error color (#F04438) should be used for error states and destructive actions
Success color (#12B76A) should be used for success states and confirmations
Warning color (#F79009) should be used for warnings and important notifications
Info color (#0BA5EC) should be used for informational elements
Files:
apps/api/src/modules/config/app.config.tspackages/skill-template/src/skills/agent.tsapps/api/src/modules/drive/drive.service.ts
**/*.{tsx,ts}
📄 CodeRabbit inference engine (.cursor/rules/09-i18n-guidelines.mdc)
**/*.{tsx,ts}: Use the translation wrapper component and useTranslation hook in components
Ensure all user-facing text is translatable
Files:
apps/api/src/modules/config/app.config.tspackages/skill-template/src/skills/agent.tsapps/api/src/modules/drive/drive.service.ts
**/*.{tsx,ts,json}
📄 CodeRabbit inference engine (.cursor/rules/09-i18n-guidelines.mdc)
Support dynamic content with placeholders in translations
Files:
apps/api/src/modules/config/app.config.tspackages/skill-template/src/skills/agent.tsapps/api/src/modules/drive/drive.service.ts
**/*.{tsx,ts,jsx,js,vue,css,scss,less}
📄 CodeRabbit inference engine (.cursor/rules/11-ui-design-patterns.mdc)
**/*.{tsx,ts,jsx,js,vue,css,scss,less}: Use the primary blue (#155EEF) for main UI elements, CTAs, and active states
Use red (#F04438) only for errors, warnings, and destructive actions
Use green (#12B76A) for success states and confirmations
Use orange (#F79009) for warning states and important notifications
Use blue (#0BA5EC) for informational elements
Primary buttons should be solid with the primary color
Secondary buttons should have a border with transparent or light background
Danger buttons should use the error color
Use consistent padding, border radius, and hover states for all buttons
Follow fixed button sizes based on their importance and context
Use consistent border radius (rounded-lg) for all cards
Apply light shadows (shadow-sm) for card elevation
Maintain consistent padding inside cards (p-4orp-6)
Use subtle borders for card separation
Ensure proper spacing between card elements
Apply consistent styling to all form inputs
Use clear visual indicators for focus, hover, and error states in form elements
Apply proper spacing between elements using 8px, 16px, 24px increments
Ensure proper alignment of elements (left, center, or right)
Use responsive layouts that work across different device sizes
Maintain a minimum contrast ratio of 4.5:1 for text
Files:
apps/api/src/modules/config/app.config.tspackages/skill-template/src/skills/agent.tsapps/api/src/modules/drive/drive.service.ts
**/*.{tsx,ts,jsx,js,vue}
📄 CodeRabbit inference engine (.cursor/rules/11-ui-design-patterns.mdc)
**/*.{tsx,ts,jsx,js,vue}: Include appropriate loading states for async actions in buttons
Group related form elements with appropriate spacing
Provide clear validation feedback for forms
Ensure proper labeling and accessibility for form elements
Ensure all interactive elements are keyboard accessible
Include appropriate ARIA attributes for complex components
Provide alternative text for images and icons
Support screen readers with semantic HTML elements
Files:
apps/api/src/modules/config/app.config.tspackages/skill-template/src/skills/agent.tsapps/api/src/modules/drive/drive.service.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.cursor/rules/08-contributing-guidelines.mdc)
**/*.{ts,tsx,js,jsx}: Follow the TypeScript/JavaScript style guidelines
Ensure code is well-tested and documented
Files:
apps/api/src/modules/config/app.config.tspackages/skill-template/src/skills/agent.tsapps/api/src/modules/drive/drive.service.ts
apps/api/src/**/*.{controller,service}.ts
📄 CodeRabbit inference engine (.cursor/rules/06-api-structure.mdc)
Implement proper error handling in API modules
Files:
apps/api/src/modules/drive/drive.service.ts
🧠 Learnings (1)
📚 Learning: 2025-12-25T06:26:57.096Z
Learnt from: nettee
Repo: refly-ai/refly PR: 1942
File: packages/skill-template/src/skills/agent.ts:159-169
Timestamp: 2025-12-25T06:26:57.096Z
Learning: In packages/skill-template/src/skills/agent.ts, the supportToolChoice capability check uses `!== false` rather than `=== true` by design to maintain backward compatibility. Models without the supportToolChoice field defined (undefined) should still receive tool_choice: 'auto', and only models explicitly marked as supportToolChoice: false should skip the parameter.
Applied to files:
packages/skill-template/src/skills/agent.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: build / Build
🔇 Additional comments (5)
packages/skill-template/src/skills/agent.ts (2)
273-384: Excellent implementation of parallel tool execution!The enhanced tool node successfully executes multiple tool calls concurrently while maintaining order and providing comprehensive error handling. Key strengths:
- Uses
Promise.allfor concurrent execution (line 353)- Preserves original order by tracking indices and sorting results (lines 356-358)
- Returns structured
ToolMessageobjects for all cases (success, missing tool, empty name, execution failure)- Adds detailed logging for observability
The approach will significantly improve performance when multiple tools are invoked simultaneously.
290-350: Well-structured error handling for parallel execution.Each tool call scenario is properly handled:
- Empty tool names → Returns structured error ToolMessage
- Missing tools → Returns "tool not available" ToolMessage
- Execution failures → Returns error ToolMessage with details
- Successful execution → Returns content ToolMessage
The consistent return structure
{index, message}enables proper ordering after parallel execution completes.apps/api/src/modules/config/app.config.ts (1)
306-307: LGTM! Clean configuration addition.The new
lambda.s3.inputBucketproperty follows the established pattern with a sensible fallback chain (LAMBDA_INPUT_S3_BUCKET → MINIO_INTERNAL_BUCKET → default). This standardizes S3 bucket configuration and aligns with the usage indrive.service.ts.apps/api/src/modules/drive/drive.service.ts (2)
676-680: LGTM! Consistent use of new S3 bucket configuration.The change correctly separates input and output bucket configuration using:
lambda.s3.inputBucketfor source fileslambda.s3.bucketfor Lambda outputBoth properties have defaults defined in
app.config.ts, ensuring the values are always available.
1394-1395: Consistent S3 bucket configuration across all Lambda dispatches.All Lambda dispatch methods (
triggerAsyncParse,dispatchLambdaImageTransform,dispatchLambdaDocumentRender,startExportJob) consistently use the new configuration keys for input and output buckets. This standardization improves maintainability and aligns with the configuration changes inapp.config.ts.Also applies to: 2313-2314, 2369-2370, 2477-2478
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (5)
apps/api/src/modules/drive/drive.service.ts (5)
687-701: Missing null safety for bucket configuration values.The
config.get<string>()method returnsundefinedif the configuration key is not set. Without fallback values or validation,inputBucketandoutputBucketcould beundefined, potentially causing runtime errors in the Lambda service.As per coding guidelines, always use nullish coalescing or default values for potentially undefined values.
🛠️ Suggested fix: Add validation or fallbacks
Option 1: Add validation (recommended)
// S3 input bucket (where source files are stored) const inputBucket = this.config.get<string>('lambda.s3.inputBucket'); + if (!inputBucket) { + throw new ParamsError('Lambda S3 input bucket is not configured'); + } // Output bucket for Lambda results const outputBucket = this.config.get<string>('lambda.s3.bucket'); + if (!outputBucket) { + throw new ParamsError('Lambda S3 output bucket is not configured'); + }Option 2: Add fallbacks if defaults exist
- const inputBucket = this.config.get<string>('lambda.s3.inputBucket'); + const inputBucket = this.config.get<string>('lambda.s3.inputBucket') ?? this.config.get<string>('objectStorage.minio.internal.bucket'); - const outputBucket = this.config.get<string>('lambda.s3.bucket'); + const outputBucket = this.config.get<string>('lambda.s3.bucket') ?? this.config.get<string>('objectStorage.minio.internal.bucket');
1480-1491: Same null safety issue asparseViaLambda.Apply the same validation pattern here. While this method catches exceptions (lines 1518-1523), a clear error message for missing bucket configuration would aid debugging.
🛠️ Suggested fix
const inputBucket = this.config.get<string>('lambda.s3.inputBucket'); + if (!inputBucket) { + this.logger.warn('Lambda S3 input bucket not configured, skipping async parse'); + return; + } const outputBucket = this.config.get<string>('lambda.s3.bucket'); + if (!outputBucket) { + this.logger.warn('Lambda S3 output bucket not configured, skipping async parse'); + return; + }
2427-2440: Same null safety issue for bucket configuration.Apply consistent validation across all Lambda dispatch methods.
🛠️ Suggested fix
const inputBucket = this.config.get<string>('lambda.s3.inputBucket'); + if (!inputBucket) { + throw new ParamsError('Lambda S3 input bucket is not configured'); + } const outputBucket = this.config.get<string>('lambda.s3.bucket'); + if (!outputBucket) { + throw new ParamsError('Lambda S3 output bucket is not configured'); + }
2483-2497: Same null safety issue for bucket configuration.Apply consistent validation as suggested for
dispatchLambdaImageTransform.
2593-2608: Same null safety issue for bucket configuration.Apply consistent validation as suggested for other Lambda dispatch methods.
🧹 Nitpick comments (1)
apps/api/src/modules/drive/drive.service.ts (1)
687-691: Consider extracting Lambda bucket configuration into a helper method.The same bucket retrieval pattern is repeated 5 times across this file. Extracting this into a reusable helper would centralize validation and reduce code duplication.
♻️ Suggested refactor
/** * Get Lambda S3 bucket configuration with validation * `@throws` ParamsError if buckets are not configured */ private getLambdaS3Buckets(): { inputBucket: string; outputBucket: string } { const inputBucket = this.config.get<string>('lambda.s3.inputBucket'); const outputBucket = this.config.get<string>('lambda.s3.bucket'); if (!inputBucket || !outputBucket) { throw new ParamsError( 'Lambda S3 buckets not configured. Set lambda.s3.inputBucket and lambda.s3.bucket.' ); } return { inputBucket, outputBucket }; }Then use it in all dispatch methods:
const { inputBucket, outputBucket } = this.getLambdaS3Buckets();
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@packages/skill-template/package.json`:
- Line 30: The project is CommonJS but package.json pins "p-limit" to "^5.0.0"
(ESM-only) causing ERR_REQUIRE_ESM; change the dependency in package.json to a
CommonJS-compatible version (e.g., "p-limit": "^3.1.0"), run your package
manager to reinstall, and confirm the import in src/skills/agent.ts uses a
CommonJS-compatible form (or adjust to require if necessary) so pLimit is loaded
at runtime without ESM errors.
🧹 Nitpick comments (2)
packages/skill-template/src/skills/agent.ts (2)
48-62: Consider adding an upper bound for the concurrency limit.The function validates that the value is positive, but extremely high values (e.g., 1000) could cause resource exhaustion. Consider adding a maximum cap.
♻️ Suggested improvement
+const MAX_TOOL_PARALLEL_CONCURRENCY = 50; + function getToolParallelConcurrency(): number { const envConcurrency = process.env.WORKFLOW_TOOL_PARALLEL_CONCURRENCY; if (envConcurrency) { const parsed = Number.parseInt(envConcurrency, 10); if (!Number.isNaN(parsed) && parsed > 0) { - return parsed; + return Math.min(parsed, MAX_TOOL_PARALLEL_CONCURRENCY); } } return DEFAULT_TOOL_PARALLEL_CONCURRENCY; }
346-346: Prefer nullish coalescing over logical OR.Per coding guidelines, use
??instead of||for fallback values. While functionally equivalent here (empty arrays are truthy),??is the preferred pattern.♻️ Suggested fix
- const matchedTool = (availableToolsForNode || []).find((t) => t?.name === toolName); + const matchedTool = (availableToolsForNode ?? []).find((t) => t?.name === toolName);
lefarcen
changed the title
mrcfps
changed the title
3 participants
This pull request makes two main improvements: it updates the configuration and usage of S3 input/output buckets throughout the backend, and it enhances the agent skill template to execute tool calls in parallel rather than sequentially. The configuration changes ensure consistency and flexibility for specifying S3 buckets, while the agent update improves performance by running tool calls concurrently.
Configuration and S3 Bucket Handling:
lambda.s3.inputBucketthroughout the codebase, replacing older references toobjectStorage.minio.internal.bucketand hardcoded defaults. This affects both the app configuration (app.config.ts) and all Lambda dispatch logic indrive.service.ts. [1] [2] [3] [4] [5] [6]Agent Skill Tool Call Execution:
Summary by CodeRabbit
New Features
Chores
Dependencies
✏️ Tip: You can customize this high-level summary in your review settings.