Skip to content

Commit da31004

Browse files
karimrakarimra
authored
Merge pull request #774 from nokia/collector-cmd
Add collector cmd
2 parents c55b298 + 2905192 commit da31004

File tree

109 files changed

+16417
-1951
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

109 files changed

+16417
-1951
lines changed

‎.gitignore‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,12 @@ _test/
22
tests/clab-*
33
tests/srl-*
44
tests/.*clab.yaml
5+
tests/collector/suite/*/clab-*
56
builds/
67
dist
78
*.log
89
gnmic
910
*.tmp
1011
*.work*
11-
.idea
12+
.idea
13+
tests/collector

‎Dockerfile‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,18 @@
77
# SPDX-License-Identifier: Apache-2.0
88

99
FROM golang:1.24.7 AS builder
10-
ADD . /build
10+
1111
WORKDIR /build
12+
13+
COPY go.mod go.sum /build/
14+
COPY pkg/api/go.mod pkg/api/go.sum /build/pkg/api/
15+
COPY pkg/cache/go.mod pkg/cache/go.sum /build/pkg/cache/
16+
RUN go mod download
17+
18+
ADD . /build
19+
1220
#RUN CGO_ENABLED=0 go build -ldflags="-s -w" -o gnmic .
13-
RUN CGO_ENABLED=0 go build -o gnmic .
21+
RUN CGO_ENABLED=0 go build -ldflags="-s -w" -o gnmic .
1422

1523
FROM alpine
1624
LABEL org.opencontainers.image.source=https://github.com/openconfig/gnmic

‎docs/cmd/collector.md‎

Lines changed: 210 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,210 @@
1+
### Description
2+
3+
The `[collect | collector | coll | c]` command starts gNMIc as a long-running telemetry collector service. Unlike the `subscribe` command which is designed for interactive use, the collector command is optimized for production deployments with dynamic configuration capabilities via REST API.
4+
5+
The collector provides:
6+
7+
- **Dynamic configuration** - Add, modify, or remove targets, subscriptions, outputs, inputs, and processors at runtime via REST API
8+
- **Clustering support** - Multiple collector instances can form a cluster with automatic target distribution and failover
9+
- **Embedded gNMI server** - Expose collected telemetry to downstream gNMI clients
10+
- **Tunnel target support** - Accept connections from gNMI tunnel targets
11+
12+
### Usage
13+
14+
`gnmic [global-flags] collect [local-flags]`
15+
16+
### Local Flags
17+
18+
#### pyroscope-server-address
19+
20+
The `[--pyroscope-server-address]` flag sets the Pyroscope server address for continuous profiling. When set, the collector will send profiling data to the specified Pyroscope server.
21+
22+
#### pyroscope-application-name
23+
24+
The `[--pyroscope-application-name]` flag sets the application name used in Pyroscope. Defaults to `gnmic-collector`.
25+
26+
### Subcommands
27+
28+
The collector command provides subcommands to interact with a running collector instance via its REST API:
29+
30+
| Subcommand | Aliases | Description |
31+
|------------|---------|-------------|
32+
| `targets` | `target`, `tg` | Manage targets |
33+
| `subscriptions` | `subscription`, `sub` | Manage subscriptions |
34+
| `outputs` | `output`, `out` | Manage outputs |
35+
| `inputs` | `input`, `in` | Manage inputs |
36+
| `processors` | `processor`, `proc` | Manage processors |
37+
38+
Each subcommand supports the following operations:
39+
40+
| Operation | Aliases | Description |
41+
|-----------|---------|-------------|
42+
| `list` | `ls` | List all resources |
43+
| `get` | `g`, `show`, `sh` | Get a specific resource |
44+
| `set` | `create`, `cr` | Create or update a resource |
45+
| `delete` | `d`, `del`, `rm` | Delete a resource |
46+
47+
### Configuration
48+
49+
The collector is configured using the standard gNMIc configuration file. The key sections are:
50+
51+
```yaml
52+
# API server configuration (required for collector)
53+
api-server:
54+
address: :7890
55+
timeout: 10s
56+
tls:
57+
ca-file:
58+
cert-file:
59+
key-file:
60+
enable-metrics: false
61+
debug: false
62+
63+
# Targets to collect from
64+
targets:
65+
router1:
66+
address: 10.0.0.1:57400
67+
username: admin
68+
password: admin
69+
skip-verify: true
70+
71+
# Subscriptions define what data to collect
72+
subscriptions:
73+
interfaces:
74+
paths:
75+
- /interfaces/interface/state/counters
76+
mode: stream
77+
stream-mode: sample
78+
sample-interval: 10s
79+
80+
# Outputs define where to send collected data
81+
outputs:
82+
prometheus:
83+
type: prometheus
84+
listen: :9804
85+
path: /metrics
86+
87+
# Inputs for receiving data from message queues
88+
inputs:
89+
nats-input:
90+
type: nats
91+
address: nats://localhost:4222
92+
subject: telemetry.>
93+
94+
# Event processors for data transformation
95+
processors:
96+
add-hostname:
97+
event-add-tag:
98+
tags:
99+
- tag-name: hostname
100+
value: ${HOST}
101+
102+
# Clustering configuration (optional)
103+
clustering:
104+
cluster-name: gnmic-cluster
105+
instance-name: gnmic-1
106+
locker:
107+
type: consul
108+
address: consul:8500
109+
110+
# gNMI server configuration (optional)
111+
gnmi-server:
112+
address: :57401
113+
skip-verify: true
114+
```
115+
116+
### Examples
117+
118+
#### 1. Start a basic collector
119+
120+
```bash
121+
gnmic --config collector.yaml collect
122+
```
123+
124+
#### 2. Start with Pyroscope profiling
125+
126+
```bash
127+
gnmic --config collector.yaml collect \
128+
--pyroscope-server-address http://pyroscope:4040 \
129+
--pyroscope-application-name my-collector
130+
```
131+
132+
#### 3. List targets from a running collector
133+
134+
```bash
135+
gnmic --config collector.yaml collect targets list
136+
```
137+
138+
Output:
139+
```
140+
NAME ADDRESS USERNAME STATE SUBSCRIPTIONS OUTPUTS INSECURE SKIP VERIFY
141+
router1 10.0.0.1:57400 admin running 2 1 false true
142+
router2 10.0.0.2:57400 admin running 2 1 false true
143+
```
144+
145+
#### 4. Get details of a specific target
146+
147+
```bash
148+
gnmic --config collector.yaml collect targets get --name router1
149+
```
150+
151+
#### 5. Create a new target
152+
153+
```bash
154+
gnmic --config collector.yaml collect targets set --input target.yaml
155+
```
156+
157+
Where `target.yaml` contains:
158+
```yaml
159+
name: router3
160+
address: 10.0.0.3:57400
161+
username: admin
162+
password: admin
163+
skip-verify: true
164+
subscriptions:
165+
- interfaces
166+
outputs:
167+
- prometheus
168+
```
169+
170+
#### 6. Delete a target
171+
172+
```bash
173+
gnmic --config collector.yaml collect targets delete --name router3
174+
```
175+
176+
#### 7. List subscriptions
177+
178+
```bash
179+
gnmic --config collector.yaml collect subscriptions list
180+
```
181+
182+
Output:
183+
```
184+
NAME PREFIX PATHS ENCODING MODE SAMPLE INTERVAL TARGETS OUTPUTS
185+
interfaces - /interfaces/interface/state/counters json stream/sample 10s 2/2 1
186+
```
187+
188+
#### 8. List outputs
189+
190+
```bash
191+
gnmic --config collector.yaml collect outputs list
192+
```
193+
194+
Output:
195+
```
196+
NAME TYPE FORMAT EVENT PROCESSORS
197+
prometheus prometheus - 1
198+
```
199+
200+
#### 9. List processors with details
201+
202+
```bash
203+
gnmic --config collector.yaml collect processors list --details
204+
```
205+
206+
### See Also
207+
208+
- [Collector Introduction](../user_guide/collector/collector_intro.md) - Overview and architecture
209+
- [Collector Configuration](../user_guide/collector/collector_configuration.md) - Detailed configuration reference
210+
- [Collector REST API](../user_guide/collector/collector_api.md) - API endpoints reference

0 commit comments

Comments
 (0)