feat(llmobs): distributed experiments SDK preview#16570
Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 19 commits intomainfrom Feb 21, 2026
Merged
feat(llmobs): distributed experiments SDK preview#16570gh-worker-dd-mergequeue-cf854d[bot] merged 19 commits intomainfrom
gh-worker-dd-mergequeue-cf854d[bot] merged 19 commits intomainfrom
Conversation
…wal/add-tags-to-pull-datasets
…wal/add-tags-to-pull-datasets
Codeowners resolved as |
…wal/add-tags-to-pull-datasets
gary-huang
force-pushed
the
gary/mlob-4860-v2
branch
3 times, most recently
from
ca99469 to
4d7e924
Compare
…s-to-pull-datasets # Conflicts: # ddtrace/llmobs/_writer.py
gary-huang
force-pushed
the
gary/mlob-4860-v2
branch
from
4d7e924 to
9fd11e0
Compare
gary-huang
changed the base branch from
main
to
mehul.sonowal/add-tags-to-pull-datasets
gary-huang
changed the title
gary-huang
added
the
changelog/no-changelog
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9fd11e02c6
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Base automatically changed from
mehul.sonowal/add-tags-to-pull-datasets
to
main
February 20, 2026 21:44
gh-worker-dd-mergequeue-cf854d
bot
requested a review
from a team
as a code owner
gh-worker-dd-mergequeue-cf854d
bot
requested review from
ncybul and
r1viollet
Kyle-Verhoog
approved these changes
Feb 20, 2026
Contributor
Author
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
adds a set of APIs to support running experiments across different hosts
LLMObs._distributed_experimentto create the experimentLLMObs._run_for_experimentto run the experimentexperiment._submit_eval_metricto submit evalsthis set of APIs is not publicly supported, internal only for the timebeing, subject to change
Testing
ran internal tests with instrumented mortar jobs, as well as local testing
Risks
this is unsupported to the public and blast radius should be limited to internal users
Additional Notes