Skip to content

Live check dog-fooding: doc, code-gen and live-check#1232

Open
jerbly wants to merge 52 commits intoopen-telemetry:mainfrom
jerbly:live-check-doc
Open

Live check dog-fooding: doc, code-gen and live-check#1232
jerbly wants to merge 52 commits intoopen-telemetry:mainfrom
jerbly:live-check-doc

Conversation

@jerbly
Copy link
Contributor

@jerbly jerbly commented Feb 22, 2026

This PR makes live-check "eat its own dog food" — the OTLP log events emitted by --emit-otlp-logs are now defined as a semantic convention model. Weaver is used to generate docs and code and to run live-check against tests that emit logs.

  • Model (crates/weaver_live_check/model/live_check.yaml): Defines the weaver.finding event, its attributes (finding_id, signal_type, sample_type, etc.), and enumerations as a v2 semconv registry.
  • Templates (crates/weaver_live_check/templates/): Jinja templates generate Rust structs and enums with strum derives and a Markdown reference doc — all derived generically from the model, no hardcoded names.
  • Generated code (crates/weaver_live_check/src/generated/): attributes.rs, events.rs, entities.rs — type-safe enums for FindingId, SignalType, SampleType, etc., replacing hand-written string constants.
  • Generated docs (crates/weaver_live_check/docs/finding.md): Auto-generated reference documentation for the finding event schema.
  • Guide (crates/weaver_live_check/docs/dog-fooding.md): Explains the model, how to regenerate, and the design.
  • Integration test (crates/weaver_live_check/tests/livecheck_emit.rs): End-to-end test that emits diverse findings via OtlpEmitter into a live-check instance using the live_check model as its registry, then asserts zero violations and 100% registry coverage — confirming the emitted OTLP log records conform to the model.
  • CI freshness check (cargo xtask check-generated): Cross-platform xtask that regenerates code/docs and fails if there's any drift from what's committed. Runs on ubuntu in the test job.

@jerbly jerbly requested a review from a team as a code owner February 22, 2026 20:29
@codecov
Copy link

codecov bot commented Feb 22, 2026

Codecov Report

❌ Patch coverage is 82.80255% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.3%. Comparing base (151b1e0) to head (ec0b338).

Files with missing lines Patch % Lines
crates/weaver_live_check/src/generated/entities.rs 50.0% 8 Missing ⚠️
crates/weaver_live_check/src/generated/events.rs 81.8% 6 Missing ⚠️
crates/weaver_live_check/src/otlp_logger.rs 88.4% 6 Missing ⚠️
crates/weaver_live_check/src/lib.rs 68.7% 5 Missing ⚠️
...weaver_live_check/src/advice/deprecated_advisor.rs 66.6% 1 Missing ⚠️
...rates/weaver_live_check/src/advice/type_advisor.rs 93.7% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            main   #1232     +/-   ##
=======================================
- Coverage   80.4%   80.3%   -0.2%     
=======================================
  Files        110     113      +3     
  Lines       9025    9083     +58     
=======================================
+ Hits        7260    7297     +37     
- Misses      1765    1786     +21     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

type: event
name: weaver.live_check.finding
# yaml-language-server: $schema=../../../schemas/semconv-group.json
file_format: definition/2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WOOT! v2!

I love the idea of dogfooding here, just looking through all the CI/CD changes.

One question - should we have per-crate weaver, or "global project" weaver?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you mean exactly. Feels like the code gen is for this crate so the def needs to be here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm asking if we should have a crate for codegen, or run weaver so it generates all code across all crates or if we plan to run it per-create, i.e separately.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess that depends on whether we think other crates will produce telemetry in the future. As it stands today, only emit and live-check produce telemetry. emit spoofs telemetry from other models so it's not really a candidate for codegen.

Would it be useful to have other commands, like check produce Findings as OTLP?

Copy link
Member

@lmolkova lmolkova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is so cool!!!

note: >
Information findings provide helpful context for understanding the telemetry
or the validation results. For example, noting that a template attribute
was used.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not blocking: I still don't understand why is it important to know that template attribute was used. It's the same as 'string' attribute was used or value of this attribute was 5.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite the same as string etc. It's useful to catch unintended use of a defined template-type. e.g. http.request.header.my.mistake is valid but might be unintentional.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but there is no difference between http.request.header.all-good and http.request.header.my.mistake from schema / semantic-conventions.

My argument was that template[string] is a type in definition, string is also a type in a definition.
We see string attribute over the wire and it matches template[string] definition type.
So we fire info advise when we see that attribute conforms to the conventions - is that useful?

I'm trying to create some mental model on when to fire which level and in my mind "everything is according to spec" is either not fired at all and only appears in metrics or has some trace/debug level, but consistently.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand, I feel that template-types fall into an awkward category where you're providing keys and values. And therefore there's a different class of mistake that can be made. This is why I set this at information level.

Here's another example. Let's say there's an error in my code. I wanted to use: http.request.method_original but I accidentally wrote http.request.header.method_original. These are both valid according to the definition.

I realize these examples are somewhat tenuous - so if we think there's no value having this information level finding we can remove it. Should we also look at other information level findings? opt_in_attribute_not_present, conditionally_required_attribute_not_present, undefined_enum_variant, extends_namespace

#1256 allows you to exclude all information level findings or specific findings if you so wish.

jerbly and others added 7 commits March 1, 2026 06:49
Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants