Live check dog-fooding: doc, code-gen and live-check#1232
Live check dog-fooding: doc, code-gen and live-check#1232jerbly wants to merge 52 commits intoopen-telemetry:mainfrom
Conversation
…live-check-doc
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1232 +/- ##
=======================================
- Coverage 80.4% 80.3% -0.2%
=======================================
Files 110 113 +3
Lines 9025 9083 +58
=======================================
+ Hits 7260 7297 +37
- Misses 1765 1786 +21 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…live-check-doc
…live-check-doc
…live-check-doc
…live-check-doc
…live-check-doc
| type: event | ||
| name: weaver.live_check.finding | ||
| # yaml-language-server: $schema=../../../schemas/semconv-group.json | ||
| file_format: definition/2 |
There was a problem hiding this comment.
WOOT! v2!
I love the idea of dogfooding here, just looking through all the CI/CD changes.
One question - should we have per-crate weaver, or "global project" weaver?
There was a problem hiding this comment.
I'm not sure what you mean exactly. Feels like the code gen is for this crate so the def needs to be here?
There was a problem hiding this comment.
I'm asking if we should have a crate for codegen, or run weaver so it generates all code across all crates or if we plan to run it per-create, i.e separately.
There was a problem hiding this comment.
I guess that depends on whether we think other crates will produce telemetry in the future. As it stands today, only emit and live-check produce telemetry. emit spoofs telemetry from other models so it's not really a candidate for codegen.
Would it be useful to have other commands, like check produce Findings as OTLP?
…live-check-doc
…live-check-doc
| note: > | ||
| Information findings provide helpful context for understanding the telemetry | ||
| or the validation results. For example, noting that a template attribute | ||
| was used. |
There was a problem hiding this comment.
Not blocking: I still don't understand why is it important to know that template attribute was used. It's the same as 'string' attribute was used or value of this attribute was 5.
There was a problem hiding this comment.
Not quite the same as string etc. It's useful to catch unintended use of a defined template-type. e.g. http.request.header.my.mistake is valid but might be unintentional.
There was a problem hiding this comment.
but there is no difference between http.request.header.all-good and http.request.header.my.mistake from schema / semantic-conventions.
My argument was that template[string] is a type in definition, string is also a type in a definition.
We see string attribute over the wire and it matches template[string] definition type.
So we fire info advise when we see that attribute conforms to the conventions - is that useful?
I'm trying to create some mental model on when to fire which level and in my mind "everything is according to spec" is either not fired at all and only appears in metrics or has some trace/debug level, but consistently.
There was a problem hiding this comment.
I understand, I feel that template-types fall into an awkward category where you're providing keys and values. And therefore there's a different class of mistake that can be made. This is why I set this at information level.
Here's another example. Let's say there's an error in my code. I wanted to use: http.request.method_original but I accidentally wrote http.request.header.method_original. These are both valid according to the definition.
I realize these examples are somewhat tenuous - so if we think there's no value having this information level finding we can remove it. Should we also look at other information level findings? opt_in_attribute_not_present, conditionally_required_attribute_not_present, undefined_enum_variant, extends_namespace
#1256 allows you to exclude all information level findings or specific findings if you so wish.
Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
This PR makes live-check "eat its own dog food" — the OTLP log events emitted by
--emit-otlp-logsare now defined as a semantic convention model. Weaver is used to generate docs and code and to run live-check against tests that emit logs.crates/weaver_live_check/model/live_check.yaml): Defines theweaver.findingevent, its attributes (finding_id,signal_type,sample_type, etc.), and enumerations as a v2 semconv registry.crates/weaver_live_check/templates/): Jinja templates generate Rust structs and enums withstrumderives and a Markdown reference doc — all derived generically from the model, no hardcoded names.crates/weaver_live_check/src/generated/):attributes.rs,events.rs,entities.rs— type-safe enums forFindingId,SignalType,SampleType, etc., replacing hand-written string constants.crates/weaver_live_check/docs/finding.md): Auto-generated reference documentation for the finding event schema.crates/weaver_live_check/docs/dog-fooding.md): Explains the model, how to regenerate, and the design.crates/weaver_live_check/tests/livecheck_emit.rs): End-to-end test that emits diverse findings viaOtlpEmitterinto a live-check instance using the live_check model as its registry, then asserts zero violations and 100% registry coverage — confirming the emitted OTLP log records conform to the model.cargo xtask check-generated): Cross-platform xtask that regenerates code/docs and fails if there's any drift from what's committed. Runs on ubuntu in the test job.