[Bug]: Get endpointslice - reconcile loop storm

[Vormillion]

Version

4.0.1

What Kubernetes platforms are you running on?

EKS Amazon

Steps to reproduce

I have ingress with 57 hosts and each host has 2 paths.
Almost during whole day ~54 hosts does not have any active endpoints as environments are scaled down.

When I'm restarting ingress controller pods, it takes ages to reach ready status, because controller is checking each endpointslice multiple time, just for single domain I've counted it 347 times in logs before configuration is fully loaded.
It's adding delay between 10-60 seconds for readiness and of course sometimes pod is restarting due to readiness failures.

I suspect issue is there:

kubernetes-ingress/internal/k8s/controller.go

Line 2330 in 2b2f329

podEndps, external, err = lbc.getEndpointsForIngressBackend(&path.Backend, svc)

NOTE: It's working fine without any pefromance issues with Ingress-Nginx.

[github-actions]

Hi @Vormillion thanks for reporting!

Be sure to check out the docs and the Contributing Guidelines while you wait for a human to take a look at this 🙂

Cheers!

[vepatel]

thanks @Vormillion for the issue, do you think this is something similar to #7778?

[Vormillion]

I'm not so sure that it's linked. CPU seems to be stable for me during this time, it just takes crazy amount of time to reach ready status.

During this time I can also see such log 401 times (which probably triggers 401 times endpointslice check)
I20260312 15:06:09.201832 1 main.go:103] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"xxx-f5-ingress", UID:"8bc196fa-ddb4-4be1-8118-0850c0e718bb", APIVersion:"networking.k8s.io/v1", ResourceVersion:"606801351", FieldPath:""}): type: 'Normal' reason: 'AddedOrUpdated' Configuration for default/xxx-f5-ingress was added or updated

[vepatel]

thanks I'll suggest upgrade to 5.3.x and check if you see similar behaviour and we'll do some investigation on this as well 👍🏼

[matei]

The issue is present in the latest version. A description of the issue and proposed fix here