BREAKING CHANGE: Refactor plugin system to version 3#34
Open
Conversation
Owner
mimuret
commented
Jul 27, 2025
mimuret
commented
- Changed message processing flow to be configurable on a per-plugin basis, and deprecated Global Filter and OutputGroup settings
- Changed configuration format from YAML/JSON to HCL
- Changed message processing flow to be configurable on a per-plugin basis, and deprecated Global Filter and OutputGroup settings - Changed configuration format from YAML/JSON to HCL
There was a problem hiding this comment.
Pull Request Overview
This PR introduces a breaking change to refactor the plugin system from version 2 to version 3, transitioning from YAML/JSON configuration to HCL format. The key changes include restructuring message processing to be configurable per-plugin, removing global filter and output group settings, and updating all output plugins to support the new HCL-based configuration system.
- Migrated configuration format from YAML/JSON to HashiCorp Configuration Language (HCL)
- Refactored plugin architecture to support per-plugin message processing flow
- Updated all output plugins to use new configuration structure and interface changes
Reviewed Changes
Copilot reviewed 166 out of 173 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| test/dtap1/conf/dtap.hcl | Adds new HCL configuration for DTAP 1 test setup with input file and multiple outputs |
| test/dtap2/conf/dtap.hcl | Adds new HCL configuration for DTAP 2 test setup with multiple inputs and stdout output |
| test/compose.yml | Adds Docker Compose configuration for testing the new system |
| schemas/*.json | Removes old JSON schema files for validation |
| pkg/types/interface.go | Updates plugin interfaces to use new context-based pattern |
| pkg/plugin/registry/plugin_registry.go | Changes plugin setup functions to accept config blocks instead of raw JSON |
| pkg/plugin/output/*/config-schema.json | Removes JSON schema files for all output plugins |
| pkg/plugin/output//.go | Updates all output plugins to use HCL configuration and new interfaces |
| pkg/plugin/pub/*.go | Updates publisher interfaces to accept context parameters |
Comments suppressed due to low confidence (2)
test/dtap1/conf/dtap.hcl:31
- Inconsistent indentation: using tab instead of spaces like the rest of the file.
template = "{{ .Message.Timestamp }} in:{{ index .Labels \"input_plugin\" }} {{ .Message.Type }} {{ .Message.Qclass }} {{ .Message.Qtype }} {{ .Message.Qname }}"
test/dtap2/conf/dtap.hcl:30
- Inconsistent indentation: using tab instead of spaces like the rest of the file.
template = "{{ .Message.Timestamp }} in:{{ index .Labels \"input_plugin\" }} {{ .Message.Type }} {{ .Message.Qclass }} {{ .Message.Qtype }} {{ .Message.Qname }}"
| } | ||
| } | ||
|
|
||
| func (d *DnstapMessage) SetOuputAttributes(res map[string]string, keys []string) error { |
There was a problem hiding this comment.
Typo in function name: 'SetOuputAttributes' should be 'SetOutputAttributes'.
Suggested change
| func (d *DnstapMessage) SetOuputAttributes(res map[string]string, keys []string) error { | |
| func (d *DnstapMessage) SetOutputAttributes(res map[string]string, keys []string) error { |
| var _ output.SocketOutput = &TCP{} | ||
|
|
||
| // The TCP plugin outputs messages to the tcp server. | ||
| // Eample configuration: |
There was a problem hiding this comment.
Typo in comment: 'Eample' should be 'Example'.
Suggested change
| // Eample configuration: | |
| // Example configuration: |
| Plugin | ||
| Start(context.Context, *OutputContext) error | ||
| Start(context.Context, Reader) error | ||
| // 同時実行可能数を取得する |
There was a problem hiding this comment.
Comment is in Japanese instead of English. Should be translated for consistency with the rest of the codebase.
Suggested change
| // 同時実行可能数を取得する | |
| // Get the maximum number of concurrent executions |
| return | ||
| } | ||
| labels := make(map[string]string, len(p.MessageLabels)) | ||
| err := dm.SetOuputAttributes(labels, p.MessageLabels) |
There was a problem hiding this comment.
Typo in method call: 'SetOuputAttributes' should be 'SetOutputAttributes'.
Suggested change
| err := dm.SetOuputAttributes(labels, p.MessageLabels) | |
| err := dm.SetOutputAttributes(labels, p.MessageLabels) |
| } | ||
| } | ||
| labels := make(map[string]string, len(f.messageLabels)) | ||
| if err := dm.SetOuputAttributes(labels, f.messageLabels); err != nil { |
There was a problem hiding this comment.
Typo in method call: 'SetOuputAttributes' should be 'SetOutputAttributes'.
Suggested change
| if err := dm.SetOuputAttributes(labels, f.messageLabels); err != nil { | |
| if err := dm.SetOutputAttributes(labels, f.messageLabels); err != nil { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.