Skip to content

Query: Fix panic in ZLabelsToPromLabels due to unsafe pointer cast incompatible with stringlabels #8709

Open
coleenquadros wants to merge 5 commits intothanos-io:mainfrom
coleenquadros:fix-label-conversion
Open

Query: Fix panic in ZLabelsToPromLabels due to unsafe pointer cast incompatible with stringlabels #8709
coleenquadros wants to merge 5 commits intothanos-io:mainfrom
coleenquadros:fix-label-conversion

Conversation

@coleenquadros
Copy link
Copy Markdown
Contributor

@coleenquadros coleenquadros commented Mar 9, 2026

  • The old implementation reinterpreted memory between []ZLabel (slice of structs) and labels.Labels via unsafe.Pointer, which broke when Prometheus v0.308.0 changed labels.Labels from a slice-of-structs to a compact string encoding (stringlabels)
  • This caused a panic in the Query component during endpoint discovery: panic: runtime error: index out of range [108] with length 1 at labels_stringlabels.go:decodeSize
  • I added CHANGELOG entry for this change.
  • Change is not relevant to the end user.

Changes

Verification

@coleenquadros coleenquadros changed the title fix label conversion Query: Fix panic in ZLabelsToPromLabels due to unsafe pointer cast incompatible with stringlabels Mar 9, 2026
@pull-request-size pull-request-size bot added size/M and removed size/S labels Mar 9, 2026
Signed-off-by: Coleen Iona Quadros <coleen.quadros27@gmail.com>
Signed-off-by: Coleen Iona Quadros <coleen.quadros27@gmail.com>
Signed-off-by: Coleen Iona Quadros <coleen.quadros27@gmail.com>
Signed-off-by: Coleen Iona Quadros <coleen.quadros27@gmail.com>
Signed-off-by: Coleen Iona Quadros <coleen.quadros27@gmail.com>
Copy link
Copy Markdown
Member

@GiedriusS GiedriusS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit more involved. For starters, you can read https://giedrius.blog/2022/02/04/things-learned-from-trying-to-migrate-to-protobuf-v2-api-from-gogoprotobuf-so-far/ or look through my previous PRs on this repo. I personally think we need to handwrite the marshaling/unmarshaling functions that construct labels.Labels directly, and then return the used memory to a pool on query.Close(). Feel free to jump onto the CNCF slack to discuss further.

@coleenquadros
Copy link
Copy Markdown
Contributor Author

@GiedriusS however is this failure/panic specific to a set up? I was using v0.41.0 version and the query, query frontend pods were crashlooping on bring up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants