Add DC/DR failover policy and per-DC roles to PlacementPolicy#49
Merged
Conversation
![kodiakhq[bot]](https://avatars.githubusercontent.com/in/29196?s=60&v=4){kind=small}
tamalsaha
force-pushed
the
dc-dr-failover-policy
branch
from
189b7f2 to
b66f18a
Compare
Introduce a FailoverPolicy on ClusterSpreadConstraint and a per-rule DCRole (Member/Arbiter/Witness) to model cross data center (DC/DR) deployments. FailoverTrigger selects which primary-dc Lease a workload follows (Global or per Group). Adds a Validate() helper for the PlacementPolicy webhook and regenerates deepcopy + CRD manifests. Signed-off-by: Tamal Saha <tamal@appscode.com>
tamalsaha
force-pushed
the
dc-dr-failover-policy
branch
from
b66f18a to
0577749
Compare
…ation ClusterSpreadConstraint.Validate() (the DC/DR failover-policy and per-DC role checks: Member/Arbiter/Witness data-bearing rules, member count, mode vs role counts, scope vs group) was defined but never called, so an invalid DC/DR policy passed validation. Call it from validatePlacementPolicy after the constraint nil guard. It is a no-op for a non-DC/DR policy (FailoverPolicy unset), so existing PlacementPolicies are unaffected. Signed-off-by: Tamal Saha <tamal@appscode.com>
tamalsaha
changed the title
…lector subset match Two PetSets of one application can have subset-overlapping selectors (a DC-DR data PetSet and its per-DC arbiter share the offshoot + cluster-name labels). ManifestWork->PetSet resolution and the virtual-pod listing matched by label selector alone, so each set attributed the other's ManifestWorks to itself: observed live as 'more than one PetSet is selecting manifestwork ... Sets: [pg-dcdr-dc-a pg-dcdr-dc-a-arbiter]' and the arbiter's pod ManifestWork never being created (the controller took the update path for a pod that did not exist). - Stamp apps.k8s.appscode.com/petset-name on every pod and claim ManifestWork at creation. - GetManifestWorkPetSets resolves by the stamp when present (selector match kept as a consistency check); unlabeled pre-upgrade ManifestWorks keep the selector fallback. - ListPodsManifestWork additionally requires the virtual pod's name to parse as <set.Name>-<ordinal>, so unlabeled sibling ManifestWorks can never be adopted across sets. Signed-off-by: Tamal Saha <tamal@appscode.com>
…anifestWork too The sync path gathers a distributed set's pods through the package-level ListPodsFromManifestWork, not the object manager's ListPodsManifestWork, so the ownership guards added for the latter never ran during reconcile: the arbiter set still listed the data set's ManifestWorks (selector subset match), identity-rewrote the ordinal-0 data pod's name to <arbiter>-0 (update of a ManifestWork that does not exist, looping), and condemned + DELETED the ordinal-1 data pod. Observed live repeatedly. Apply the same owner-stamp + name-parses-as-<set>-<ordinal> guards here. Signed-off-by: Tamal Saha <tamal@appscode.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The placement substrate for KubeDB cross data center disaster recovery (DC-DR). Step 2 of the staging order (after apimachinery, before the dependent operators).
What
FailoverPolicyonClusterSpreadConstraint: marks a PlacementPolicy as a DC/DR deployment and selects how the common DC failover service drives it (FailoverTriggerscope Global/Group;ModeTwoDC/ThreeDC).nilmeans the placement is not DC/DR-managed.DCRole(Member,Arbiter,Witness) onDistributionRule: how each data center participates. Member is data-bearing and primary-eligible; Arbiter votes only and holds no data; Witness is data-bearing but never primary (for engines whose witness must carry data). Defaults to Member.ClusterSpreadConstraint.Validate(): rejects invalid policies (role vs replicaIndices, member count ≥ 2, mode vs role counts, scope vs group), now wired into the PetSet validating webhook so an invalid DC/DR policy is actually rejected. No-op for non-DC/DR policies.ManifestWorkClusterNameLabel = open-cluster-management.io/cluster-name, the single DC-name label used across the chain (agent--dc-name, Lease holder, markeractiveDC, pod label,distributionRule.clusterName).Deepcopy and the regenerated CRD (
roleenum,failoverPolicysubtree) are included. Additive and optional — existing PlacementPolicies are unaffected.Verification
go build ./...,go vet ./...,go test ./...green.