Skip to content

Weaver live check test util#5088

Open
lmolkova wants to merge 10 commits intoopen-telemetry:mainfrom
lmolkova:weaver-live-check
Open

Weaver live check test util#5088
lmolkova wants to merge 10 commits intoopen-telemetry:mainfrom
lmolkova:weaver-live-check

Conversation

@lmolkova
Copy link
Copy Markdown
Member

@lmolkova lmolkova commented Apr 14, 2026

Description

Adds WeaverLiveCheck helper that allows to validate instrumentation output against otel semconv.

Checklist:

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

@lmolkova lmolkova requested a review from a team as a code owner April 14, 2026 06:08
Copy link
Copy Markdown
Contributor

@xrmx xrmx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love this, thanks a lot!

if exit_code == 0:
return report

violations = _extract_violations(report)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if we can provide an helper that returns the report if one don't want to write policies for tests. I understand policies are natural for weaver but in tests I think plain assertions looks more natural. Also since policies works from the filesystem it may be a bit complicated to only match against a subset of the policies contained in a directory.
IOW providing an end method that will always return a report (and exit_code?) that can be called there as well.

Copy link
Copy Markdown
Member Author

@lmolkova lmolkova Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I've done this:

  1. you can call report = check.end() and it won't raise. You can explore report (which contains a list of violations and also full weaver report as dict from json) - party on
        with WeaverLiveCheck() as weaver:
            # ... configure provider, emit telemetry ...
            provider.force_flush()
            report = weaver.end()

        seen_metrics = report["statistics"]["seen_registry_metrics"]
        self.assertEqual(seen_metrics.get("test.metric"), 1)
  1. you can call report = check.end_and_check() and it'd raise LiveCheckError (extends AssertionError) if any violations are found. You still get full report as a property on the error if you want to assert on more
        with WeaverLiveCheck() as weaver:
            # ... configure provider, emit telemetry ...
            provider.force_flush()

            with self.assertRaises(LiveCheckError) as cm:
                weaver.end_and_check()

        seen_metrics = cm.exception.report["statistics"]["seen_registry_metrics"]
        self.assertEqual(seen_metrics.get("test.metric"), 1)

wdyt?

@lmolkova lmolkova force-pushed the weaver-live-check branch from 80a4f86 to 284dece Compare April 14, 2026 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants