Skip to content

[pkg/stanza] file read status observability #30473

@newly12

Description

@newly12

Component(s)

pkg/stanza

Is your feature request related to a problem? Please describe.

We are adopting file log receiver largely for collecting logs and would like to see apart from checking registry file manually for the last read offset, trying to figure out the possibility to build this feature as part of stanza package or the receiver itself.

Describe the solution you'd like

If we are adding this, I think it could be metrics use case, for example, something like following with the file path as the metrics label(as long as the file is still a valid input source)

  1. file size
  2. read offset

After checking the code, the best place for this is the Manager struct from the fileconsumer package as it has the info of file path and latest offset. One alternative I can think of is build something around the storage extension so when any offset/metadata is persisted via it, it can decode the data to get the offset, but this also means overhead on the decoding work.

I appreciate all suggestions or comments.

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestnever staleIssues marked with this label will be never staled and automatically removedpkg/stanzareceiver/filelog

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions