Skip to content

"Container does not exist" error when getting PIDs for containerd metadata collector. #498

@tobz

Description

@tobz

I noticed an error log in staging complaining about a container not existing when attempting to get the PIDs for it:

yyyy-MM-dd HH:mm:ss UTC | DATAPLANE | ERROR | (lib/saluki-env/src/workload/collectors/containerd.rs:157) | namespace:k8s.io,container_id:*,error:failed to make gRPC request: status: Unknown, message: "runc did not terminate successfully: exit status 1: container does not exist : unknown", details: [], metadata: MetadataMap { headers: {"content-type": "application/grpc"} } | Error getting PIDs for container.

Thinking about it, it seems perfectly reasonable that we're notified about a new container, but it exits fast enough that we can't get the PIDs for it first... but we should maybe look at the code path involved to make sure that we're not missing some other non-obvious failure mode.

We might also consider hiding these error messages (why complain about not being able to get PIDs for a container that indeed exited?) although it's also not clear if there's a good status field to key off of instead of having to manually parse the error message, etc.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/coreCore functionality, event model, etc.type/investigationA potential issue that requires further investigation to quantify and categorize.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions