Skip to content

add device data in resourceclaim#128

Open
guptaNswati wants to merge 7 commits intokubernetes-sigs:mainfrom
guptaNswati:resourceclaim-status
Open

add device data in resourceclaim#128
guptaNswati wants to merge 7 commits intokubernetes-sigs:mainfrom
guptaNswati:resourceclaim-status

Conversation

@guptaNswati
Copy link

Addressing #101 to add device attributes to demonstrate how to do resourceclaim status update.

Test run

    devices:
    - conditions:
      - lastTransitionTime: "2025-11-13T22:22:56Z"
        message: ""
        reason: GPUDeviceReady
        status: "True"
        type: Ready
      data:
        driverVersion:
          version: 1.0.0
        uuid:
          string: gpu-18db0e85-99e9-c746-8531-ffeb86328b39
      device: gpu-0

  devices:
    - conditions:
      - lastTransitionTime: "2025-11-13T22:22:56Z"
        message: ""
        reason: GPUDeviceReady
        status: "True"
        type: Ready
      data:
        driverVersion:
          version: 1.0.0
        uuid:
          string: gpu-93d37703-997c-c46f-a531-755e3e0dc2ac
      device: gpu-1

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Nov 13, 2025
@k8s-ci-robot
Copy link
Contributor

Welcome @guptaNswati!

It looks like this is your first PR to kubernetes-sigs/dra-example-driver 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/dra-example-driver has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Nov 13, 2025
@guptaNswati
Copy link
Author

cc @nojnhuh for review.

@nojnhuh nojnhuh moved this from 🆕 New to 👀 In review in Dynamic Resource Allocation Nov 14, 2025
@nojnhuh
Copy link
Contributor

nojnhuh commented Dec 3, 2025

FYI @guptaNswati I was hoping to get this PR merged before #129 to keep you from having to deal with those changes here. There are other changes in the works that are based on that PR though, so I may elect to merge that one first. If #129 merges first, I'm happy to help resolve conflicts with your PR here!

@guptaNswati
Copy link
Author

FYI @guptaNswati I was hoping to get this PR merged before #129 to keep you from having to deal with those changes here. There are other changes in the works that are based on that PR though, so I may elect to merge that one first. If #129 merges first, I'm happy to help resolve conflicts with your PR here!

Oh i see. thank you for the headsup. i will see if i can update this tomorrow or else you can go ahead and merge #129. Main blocker is the e2e, right? we can't merge this without the tests?

@nojnhuh
Copy link
Contributor

nojnhuh commented Dec 4, 2025

@guptaNswati Yes, I would like to include something in the tests for this. #128 (comment) is the other comment I would like to resolve before merging.

@nojnhuh
Copy link
Contributor

nojnhuh commented Dec 5, 2025

@guptaNswati Have you pushed your recent changes? The most recent commit I see here is from when the PR was first opened.

@guptaNswati
Copy link
Author

@guptaNswati Have you pushed your recent changes? The most recent commit I see here is from when the PR was first opened.

sorry pushing them rn.

@guptaNswati
Copy link
Author

guptaNswati commented Dec 5, 2025

this is the quick test with updated changes. fixing e2e test

I1205 21:38:59.352921       1 state.go:229] Adding device attribute to claim gpu-test1/pod0-gpu-zfncn
  devices:
  - conditions:
    - lastTransitionTime: "2025-12-05T21:38:59Z"
      message: GPU Device Allocated for this request
      reason: GPUDeviceAllocated
      status: "True"
      type: Allocated
    data:
      driverVersion:
        version: 1.0.0
      model:
        string: LATEST-GPU-MODEL
      uuid:
        string: gpu-657bd2e7-f5c2-a7f2-fbaa-0d1cdc32f81b

@guptaNswati
Copy link
Author

e2e test

 ./test/e2e/e2e.sh 
./test/e2e/e2e.sh: line 1: !/usr/bin/env: No such file or directory
dra-example-driver-cluster
kind-dra-1
NAME                                       STATUS   ROLES           AGE    VERSION
dra-example-driver-cluster-control-plane   Ready    control-plane   104s   v1.34.0
dra-example-driver-cluster-worker          Ready    <none>          89s    v1.34.0
node/dra-example-driver-cluster-worker condition met
Waiting for webhook to be available
resourceclaim.resource.k8s.io/webhook-test created (server dry run)
Webhook is available
namespace/gpu-test1 created
resourceclaimtemplate.resource.k8s.io/single-gpu created
pod/pod0 created
pod/pod1 created
namespace/gpu-test2 created
resourceclaimtemplate.resource.k8s.io/multiple-gpus created
pod/pod0 created
namespace/gpu-test3 created
resourceclaimtemplate.resource.k8s.io/single-gpu created
pod/pod0 created
namespace/gpu-test4 created
resourceclaim.resource.k8s.io/single-gpu created
pod/pod0 created
pod/pod1 created
namespace/gpu-test5 created
resourceclaimtemplate.resource.k8s.io/multiple-gpus created
pod/pod0 created
namespace/gpu-test6 created
resourceclaimtemplate.resource.k8s.io/single-gpu created
pod/pod0 created
pod/pod0 condition met
pod/pod1 condition met
=== Verifying ResourceClaim device data in namespace gpu-test1 ===
Found ResourceClaim gpu-test1/pod0-gpu-pqv5n, checking status.devices[0].data ...
OK: ResourceClaim gpu-test1/pod0-gpu-pqv5n has device data (uuid=gpu-e7b42cb1-4fd8-91b2-bc77-352a0c1f5747, driverVersion=1.0.0)
Pod gpu-test1/pod0, container ctr0 claimed gpu-4
Pod gpu-test1/pod1, container ctr0 claimed gpu-5
pod/pod0 condition met
Pod gpu-test2/pod0, container ctr0 claimed gpu-0
Pod gpu-test2/pod0, container ctr0 claimed gpu-6
pod/pod0 condition met
Pod gpu-test3/pod0, container ctr0 claimed gpu-1
Pod gpu-test3/pod0, container ctr1 claimed gpu-1
pod/pod0 condition met
pod/pod1 condition met
Pod gpu-test4/pod0, container ctr0 claimed gpu-2
Pod gpu-test4/pod1, container ctr0 claimed gpu-2
pod/pod0 condition met
Pod gpu-test5/pod0, container ts-ctr0 claimed gpu-3
Pod gpu-test5/pod0, container ts-ctr1 claimed gpu-3
Pod gpu-test5/pod0, container sp-ctr0 claimed gpu-7
Pod gpu-test5/pod0, container sp-ctr1 claimed gpu-7
pod/pod0 condition met
Pod gpu-test6/pod0, container init0 claimed gpu-8
Pod gpu-test6/pod0, container ctr0 claimed gpu-8
namespace "gpu-test1" deleted
resourceclaimtemplate.resource.k8s.io "single-gpu" deleted
pod "pod0" deleted
pod "pod1" deleted
namespace "gpu-test2" deleted
resourceclaimtemplate.resource.k8s.io "multiple-gpus" deleted
pod "pod0" deleted
namespace "gpu-test3" deleted
resourceclaimtemplate.resource.k8s.io "single-gpu" deleted
pod "pod0" deleted
namespace "gpu-test4" deleted
resourceclaim.resource.k8s.io "single-gpu" deleted
pod "pod0" deleted
pod "pod1" deleted
namespace "gpu-test5" deleted
resourceclaimtemplate.resource.k8s.io "multiple-gpus" deleted
pod "pod0" deleted
namespace "gpu-test6" deleted
resourceclaimtemplate.resource.k8s.io "single-gpu" deleted
pod "pod0" deleted
test ran successfully

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 5, 2025
@guptaNswati guptaNswati force-pushed the resourceclaim-status branch 2 times, most recently from a03108c to b77e105 Compare December 5, 2025 22:18
@guptaNswati guptaNswati requested a review from nojnhuh December 5, 2025 22:19
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to address this now, but we should think about how to handle errors here besides only logging them. Retries will help in some cases (HTTP 409), but we should prevent blocking subsequent NodePrepareResources calls on retries that may never succeed and I'm not sure what an appropriate amount of time would be to spend on that. I wonder if a workqueue running async would be the most appropriate way to handle status updates?

@guptaNswati Could you please open an issue to track improvements to error handling here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done #133

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You also need this to de-couple the allocation times from apiserver update latency. May be as a hacky first implementation you can do this in a goroutine?

I guess the main question to ask is, is this update required for the driver to declare allocation as complete or is the information in this update auxiliary to allocation?

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 10, 2026
@guptaNswati
Copy link
Author

sorry was out for holidays. looking at it now.

@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 14, 2026
@guptaNswati
Copy link
Author

Quick test after the rebase:

$ kubectl get resourceclaim pod1-gpu-ksml7 -n gpu-test1 -o yaml
  devices:
  - conditions: null
    data:
      driverVersion:
        version: 1.0.0
      model:
        string: LATEST-GPU-MODEL
      uuid:
        string: gpu-93d37703-997c-c46f-a531-755e3e0dc2ac
    device: gpu-1
    driver: gpu.example.com

@guptaNswati
Copy link
Author

@nojnhuh i made the suggested changes. Its ready to review again.

@guptaNswati guptaNswati requested a review from nojnhuh January 20, 2026 22:38
@pohly
Copy link
Contributor

pohly commented Jan 22, 2026

/assign @nojnhuh

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we call this something like gpu-device-status instead to avoid confusing with #125?

return resultConfigs, nil
}

func (s *DeviceState) buildDeviceStatus(res resourceapi.DeviceRequestAllocationResult) resourceapi.AllocatedDeviceStatus {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move this logic into the gpu profile so we can implement this differently for different kinds of devices. See #129 for more about the "profile" concept.

Ideally we shouldn't have to implement this for every kind of device.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay. looking to it. need sometime to understand the refractor. will be back at it.

checkpointManager: checkpointManager,
configDecoder: decoder,
configHandler: configHandler,
config: config,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep NewDeviceState where we pull everything out of the Config that we need vs. storing the whole Config on the DeviceState. Config contains user-facing config like command line flags, so let's not plumb that too far down.

@guptaNswati
Copy link
Author

Ack. I will work on it next week.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 14, 2026
Signed-off-by: Swati Gupta <swatig@nvidia.com>

address review comment: fix pointer ref

Signed-off-by: Swati Gupta <swatig@nvidia.com>
Signed-off-by: Swati Gupta <swatig@nvidia.com>
Signed-off-by: Swati Gupta <swatig@nvidia.com>
Signed-off-by: Swati Gupta <swatig@nvidia.com>
Signed-off-by: Swati Gupta <swatig@nvidia.com>
Signed-off-by: Swati Gupta <swatig@nvidia.com>
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 17, 2026
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: guptaNswati
Once this PR has been reviewed and has the lgtm label, please ask for approval from nojnhuh. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Signed-off-by: Swati Gupta <swatig@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

Comments