-
Notifications
You must be signed in to change notification settings - Fork 8
Open
Labels
area/coreCore functionality, event model, etc.Core functionality, event model, etc.type/investigationA potential issue that requires further investigation to quantify and categorize.A potential issue that requires further investigation to quantify and categorize.
Description
I noticed an error log in staging complaining about a container not existing when attempting to get the PIDs for it:
yyyy-MM-dd HH:mm:ss UTC | DATAPLANE | ERROR | (lib/saluki-env/src/workload/collectors/containerd.rs:157) | namespace:k8s.io,container_id:*,error:failed to make gRPC request: status: Unknown, message: "runc did not terminate successfully: exit status 1: container does not exist : unknown", details: [], metadata: MetadataMap { headers: {"content-type": "application/grpc"} } | Error getting PIDs for container.
Thinking about it, it seems perfectly reasonable that we're notified about a new container, but it exits fast enough that we can't get the PIDs for it first... but we should maybe look at the code path involved to make sure that we're not missing some other non-obvious failure mode.
We might also consider hiding these error messages (why complain about not being able to get PIDs for a container that indeed exited?) although it's also not clear if there's a good status field to key off of instead of having to manually parse the error message, etc.
Metadata
Metadata
Assignees
Labels
area/coreCore functionality, event model, etc.Core functionality, event model, etc.type/investigationA potential issue that requires further investigation to quantify and categorize.A potential issue that requires further investigation to quantify and categorize.