Skip to content

Commit ecd29c8

Browse files
gCyrillegCyrille
authored
Merge pull request #43 from vatesfr/fix-leader-election
fix: remove leader election when 1 replica and add useDaemonSet for the CCM
2 parents 95cf40e + 0568353 commit ecd29c8

File tree

10 files changed

+268
-13
lines changed

10 files changed

+268
-13
lines changed

‎Makefile‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,10 @@ docs:
118118
helm template -n kube-system xenorchestra-cloud-controller-manager \
119119
--set-string image.tag=$(TAG) \
120120
charts/xenorchestra-cloud-controller-manager > docs/deploy/cloud-controller-manager.yml
121+
helm template -n kube-system xenorchestra-cloud-controller-manager \
122+
--set-string image.tag=$(TAG) \
123+
--set useDaemonSet=true \
124+
charts/xenorchestra-cloud-controller-manager > docs/deploy/cloud-controller-manager-daemonset.yml
121125
helm-docs --sort-values-order=file charts/xenorchestra-cloud-controller-manager
122126

123127
release-update:

‎charts/xenorchestra-cloud-controller-manager/Chart.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,5 +10,5 @@ keywords:
1010
- ccm
1111
- xenorchestra
1212
- kubernetes
13-
version: 1.0.0-rc.1
13+
version: 1.0.0-rc.2
1414
appVersion: v1.0.0-rc.1

‎charts/xenorchestra-cloud-controller-manager/README.md‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# xenorchestra-cloud-controller-manager
22

3-
![Version: 1.0.0-rc.1](https://img.shields.io/badge/Version-1.0.0--rc.1-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v1.0.0-rc.1](https://img.shields.io/badge/AppVersion-v1.0.0--rc.1-informational?style=flat-square)
3+
![ChartVersion: 1.0.0-rc.2](https://img.shields.io/badge/ChartVersion-1.0.0--rc.2-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v1.0.0-rc.1](https://img.shields.io/badge/AppVersion-v1.0.0--rc.1-informational?style=flat-square)
44

55
Cloud Controller Manager plugin for Xen Orchestra
66

@@ -69,6 +69,16 @@ tolerations:
6969
effect: NoSchedule
7070
```
7171
72+
Using a daemonset:
73+
```yaml
74+
75+
useDaemonSet: true
76+
77+
# Set nodeSelector in daemonset mode is required
78+
nodeSelector:
79+
node-role.kubernetes.io/control-plane: "true"
80+
```
81+
7282
Deploy chart:
7383
7484
```shell
@@ -102,10 +112,11 @@ helm upgrade -i --namespace=kube-system -f xo-ccm.yaml \
102112
| podSecurityContext | object | `{"fsGroup":10258,"fsGroupChangePolicy":"OnRootMismatch","runAsGroup":10258,"runAsNonRoot":true,"runAsUser":10258}` | Pods Security Context. ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod |
103113
| securityContext | object | `{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"seccompProfile":{"type":"RuntimeDefault"}}` | Container Security Context. ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod |
104114
| resources | object | `{"requests":{"cpu":"10m","memory":"32Mi"}}` | Resource requests and limits. ref: https://kubernetes.io/docs/user-guide/compute-resources/ |
115+
| useDaemonSet | bool | `false` | Deploy CCM in Daemonset mode. |
105116
| updateStrategy | object | `{"rollingUpdate":{"maxUnavailable":1},"type":"RollingUpdate"}` | Deployment update strategy type. ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment |
106117
| nodeSelector | object | `{}` | Node labels for data pods assignment. ref: https://kubernetes.io/docs/user-guide/node-selection/ |
107118
| tolerations | list | `[{"effect":"NoSchedule","key":"node-role.kubernetes.io/control-plane","operator":"Exists"},{"effect":"NoSchedule","key":"node.cloudprovider.kubernetes.io/uninitialized","operator":"Exists"}]` | Tolerations for data pods assignment. ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ |
108119
| affinity | object | `{}` | Affinity for data pods assignment. ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity |
109120
| extraVolumes | list | `[]` | Additional volumes for Pods |
110121
| extraVolumeMounts | list | `[]` | Additional volume mounts for Pods |
111-
| useHostNetwork | bool | `false` | Host networking requested for the CCM Pod |
122+
| useHostNetwork | bool | `false` | Host networking requested for the CCM Pod. CCM will use hostNetwork. It allows to use CCM without CNI plugins. |

‎charts/xenorchestra-cloud-controller-manager/README.md.gotmpl‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,16 @@ tolerations:
7474
effect: NoSchedule
7575
```
7676

77+
Using a daemonset:
78+
```yaml
79+
80+
useDaemonSet: true
81+
82+
# Set nodeSelector in daemonset mode is required
83+
nodeSelector:
84+
node-role.kubernetes.io/control-plane: "true"
85+
```
86+
7787
Deploy chart:
7888

7989
```shell

‎charts/xenorchestra-cloud-controller-manager/templates/deployment.yaml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,23 @@
11
apiVersion: apps/v1
2+
{{- if .Values.useDaemonSet }}
3+
kind: DaemonSet
4+
{{- else }}
25
kind: Deployment
6+
{{- end }}
37
metadata:
48
name: {{ include "xenorchestra-cloud-controller-manager.fullname" . }}
59
labels:
610
{{- include "xenorchestra-cloud-controller-manager.labels" . | nindent 4 }}
711
namespace: {{ .Release.Namespace }}
812
spec:
13+
{{- if not .Values.useDaemonSet }}
914
replicas: {{ .Values.replicaCount }}
1015
strategy:
1116
type: {{ .Values.updateStrategy.type }}
17+
{{- else }}
18+
updateStrategy:
19+
type: {{ .Values.updateStrategy.type }}
20+
{{- end }}
1221
selector:
1322
matchLabels:
1423
{{- include "xenorchestra-cloud-controller-manager.selectorLabels" . | nindent 6 }}
@@ -59,6 +68,9 @@ spec:
5968
- --use-service-account-credentials
6069
- --secure-port=10258
6170
- --authorization-always-allow-paths=/healthz,/livez,/readyz,/metrics
71+
{{- if and (eq (int .Values.replicaCount) 1) (not .Values.useDaemonSet) }}
72+
- --leader-elect=false
73+
{{- end }}
6274
{{- with .Values.extraArgs }}
6375
{{- toYaml . | nindent 12 }}
6476
{{- end }}
@@ -108,13 +120,20 @@ spec:
108120
{{- with .Values.tolerations }}
109121
{{- toYaml . | nindent 8 }}
110122
{{- end }}
123+
{{- if .Values.useDaemonSet }}
124+
- effect: NoSchedule
125+
key: node.kubernetes.io/not-ready
126+
operator: Exists
127+
{{- end }}
128+
{{- if not .Values.useDaemonSet }}
111129
topologySpreadConstraints:
112130
- maxSkew: 1
113131
topologyKey: kubernetes.io/hostname
114132
whenUnsatisfiable: DoNotSchedule
115133
labelSelector:
116134
matchLabels:
117135
{{- include "xenorchestra-cloud-controller-manager.selectorLabels" . | nindent 14 }}
136+
{{- end }}
118137
volumes:
119138
{{- if .Values.existingConfigSecret }}
120139
- name: cloud-config

‎charts/xenorchestra-cloud-controller-manager/values.yaml‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,9 @@ resources:
111111
cpu: 10m
112112
memory: 32Mi
113113

114+
# -- Deploy CCM in Daemonset mode.
115+
useDaemonSet: false
116+
114117
# -- Deployment update strategy type.
115118
# ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment
116119
updateStrategy:
@@ -149,5 +152,7 @@ extraVolumes: []
149152
# -- Additional volume mounts for Pods
150153
extraVolumeMounts: []
151154

152-
# -- Host networking requested for the CCM Pod
155+
# -- Host networking requested for the CCM Pod.
156+
# CCM will use hostNetwork.
157+
# It allows to use CCM without CNI plugins.
153158
useHostNetwork: false

‎docs/deploy/cloud-controller-manager-daemonset.yml‎

Lines changed: 203 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,203 @@
1+
---
2+
# Source: xenorchestra-cloud-controller-manager/templates/serviceaccount.yaml
3+
apiVersion: v1
4+
kind: ServiceAccount
5+
metadata:
6+
name: xenorchestra-cloud-controller-manager
7+
labels:
8+
helm.sh/chart: xenorchestra-cloud-controller-manager-1.0.0-rc.2
9+
app.kubernetes.io/name: xenorchestra-cloud-controller-manager
10+
app.kubernetes.io/instance: xenorchestra-cloud-controller-manager
11+
app.kubernetes.io/version: "v1.0.0-rc.1"
12+
app.kubernetes.io/managed-by: Helm
13+
namespace: kube-system
14+
---
15+
# Source: xenorchestra-cloud-controller-manager/templates/role.yaml
16+
apiVersion: rbac.authorization.k8s.io/v1
17+
kind: ClusterRole
18+
metadata:
19+
name: system:xenorchestra-cloud-controller-manager
20+
labels:
21+
helm.sh/chart: xenorchestra-cloud-controller-manager-1.0.0-rc.2
22+
app.kubernetes.io/name: xenorchestra-cloud-controller-manager
23+
app.kubernetes.io/instance: xenorchestra-cloud-controller-manager
24+
app.kubernetes.io/version: "v1.0.0-rc.1"
25+
app.kubernetes.io/managed-by: Helm
26+
rules:
27+
- apiGroups:
28+
- coordination.k8s.io
29+
resources:
30+
- leases
31+
verbs:
32+
- get
33+
- create
34+
- update
35+
- apiGroups:
36+
- ""
37+
resources:
38+
- events
39+
verbs:
40+
- create
41+
- patch
42+
- update
43+
- apiGroups:
44+
- ""
45+
resources:
46+
- nodes
47+
verbs:
48+
- get
49+
- list
50+
- watch
51+
- update
52+
- patch
53+
- delete
54+
- apiGroups:
55+
- ""
56+
resources:
57+
- nodes/status
58+
verbs:
59+
- patch
60+
- apiGroups:
61+
- ""
62+
resources:
63+
- serviceaccounts
64+
verbs:
65+
- get
66+
- apiGroups:
67+
- ""
68+
resources:
69+
- serviceaccounts/token
70+
verbs:
71+
- create
72+
---
73+
# Source: xenorchestra-cloud-controller-manager/templates/rolebinding.yaml
74+
kind: ClusterRoleBinding
75+
apiVersion: rbac.authorization.k8s.io/v1
76+
metadata:
77+
name: system:xenorchestra-cloud-controller-manager
78+
roleRef:
79+
apiGroup: rbac.authorization.k8s.io
80+
kind: ClusterRole
81+
name: system:xenorchestra-cloud-controller-manager
82+
subjects:
83+
- kind: ServiceAccount
84+
name: xenorchestra-cloud-controller-manager
85+
namespace: kube-system
86+
---
87+
# Source: xenorchestra-cloud-controller-manager/templates/rolebinding.yaml
88+
apiVersion: rbac.authorization.k8s.io/v1
89+
kind: RoleBinding
90+
metadata:
91+
name: system:xenorchestra-cloud-controller-manager:extension-apiserver-authentication-reader
92+
namespace: kube-system
93+
roleRef:
94+
apiGroup: rbac.authorization.k8s.io
95+
kind: Role
96+
name: extension-apiserver-authentication-reader
97+
subjects:
98+
- kind: ServiceAccount
99+
name: xenorchestra-cloud-controller-manager
100+
namespace: kube-system
101+
---
102+
# Source: xenorchestra-cloud-controller-manager/templates/deployment.yaml
103+
apiVersion: apps/v1
104+
kind: DaemonSet
105+
metadata:
106+
name: xenorchestra-cloud-controller-manager
107+
labels:
108+
helm.sh/chart: xenorchestra-cloud-controller-manager-1.0.0-rc.2
109+
app.kubernetes.io/name: xenorchestra-cloud-controller-manager
110+
app.kubernetes.io/instance: xenorchestra-cloud-controller-manager
111+
app.kubernetes.io/version: "v1.0.0-rc.1"
112+
app.kubernetes.io/managed-by: Helm
113+
namespace: kube-system
114+
spec:
115+
updateStrategy:
116+
type: RollingUpdate
117+
selector:
118+
matchLabels:
119+
app.kubernetes.io/name: xenorchestra-cloud-controller-manager
120+
app.kubernetes.io/instance: xenorchestra-cloud-controller-manager
121+
template:
122+
metadata:
123+
annotations:
124+
labels:
125+
app.kubernetes.io/name: xenorchestra-cloud-controller-manager
126+
app.kubernetes.io/instance: xenorchestra-cloud-controller-manager
127+
spec:
128+
enableServiceLinks: false
129+
priorityClassName: system-cluster-critical
130+
serviceAccountName: xenorchestra-cloud-controller-manager
131+
securityContext:
132+
fsGroup: 10258
133+
fsGroupChangePolicy: OnRootMismatch
134+
runAsGroup: 10258
135+
runAsNonRoot: true
136+
runAsUser: 10258
137+
dnsPolicy: Default
138+
initContainers: []
139+
containers:
140+
- name: xenorchestra-cloud-controller-manager
141+
securityContext:
142+
allowPrivilegeEscalation: false
143+
capabilities:
144+
drop:
145+
- ALL
146+
seccompProfile:
147+
type: RuntimeDefault
148+
image: "ghcr.io/vatesfr/xenorchestra-cloud-controller-manager:v1.0.0-rc.1"
149+
imagePullPolicy: IfNotPresent
150+
args:
151+
- --v=2
152+
- --cloud-provider=xenorchestra
153+
- --cloud-config=/etc/xenorchestra/config.yaml
154+
- --controllers=cloud-node,cloud-node-lifecycle,cloud-node-label-sync
155+
- --leader-elect-resource-name=cloud-controller-manager-xenorchestra
156+
- --use-service-account-credentials
157+
- --secure-port=10258
158+
- --authorization-always-allow-paths=/healthz,/livez,/readyz,/metrics
159+
ports:
160+
- name: metrics
161+
containerPort: 10258
162+
protocol: TCP
163+
livenessProbe:
164+
httpGet:
165+
path: /healthz
166+
port: metrics
167+
scheme: HTTPS
168+
initialDelaySeconds: 20
169+
periodSeconds: 30
170+
timeoutSeconds: 5
171+
resources:
172+
requests:
173+
cpu: 10m
174+
memory: 32Mi
175+
volumeMounts:
176+
- name: cloud-config
177+
mountPath: /etc/xenorchestra
178+
readOnly: true
179+
affinity:
180+
podAntiAffinity:
181+
preferredDuringSchedulingIgnoredDuringExecution:
182+
- podAffinityTerm:
183+
labelSelector:
184+
matchLabels:
185+
app.kubernetes.io/name: xenorchestra-cloud-controller-manager
186+
app.kubernetes.io/instance: xenorchestra-cloud-controller-manager
187+
topologyKey: topology.kubernetes.io/zone
188+
weight: 1
189+
tolerations:
190+
- effect: NoSchedule
191+
key: node-role.kubernetes.io/control-plane
192+
operator: Exists
193+
- effect: NoSchedule
194+
key: node.cloudprovider.kubernetes.io/uninitialized
195+
operator: Exists
196+
- effect: NoSchedule
197+
key: node.kubernetes.io/not-ready
198+
operator: Exists
199+
volumes:
200+
- name: cloud-config
201+
secret:
202+
secretName: xenorchestra-cloud-controller-manager
203+
defaultMode: 416

‎docs/deploy/cloud-controller-manager-edge.yml‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ kind: ServiceAccount
55
metadata:
66
name: xenorchestra-cloud-controller-manager
77
labels:
8-
helm.sh/chart: xenorchestra-cloud-controller-manager-1.0.0-rc.1
8+
helm.sh/chart: xenorchestra-cloud-controller-manager-1.0.0-rc.2
99
app.kubernetes.io/name: xenorchestra-cloud-controller-manager
1010
app.kubernetes.io/instance: xenorchestra-cloud-controller-manager
1111
app.kubernetes.io/version: "v1.0.0-rc.1"
@@ -18,7 +18,7 @@ kind: ClusterRole
1818
metadata:
1919
name: system:xenorchestra-cloud-controller-manager
2020
labels:
21-
helm.sh/chart: xenorchestra-cloud-controller-manager-1.0.0-rc.1
21+
helm.sh/chart: xenorchestra-cloud-controller-manager-1.0.0-rc.2
2222
app.kubernetes.io/name: xenorchestra-cloud-controller-manager
2323
app.kubernetes.io/instance: xenorchestra-cloud-controller-manager
2424
app.kubernetes.io/version: "v1.0.0-rc.1"
@@ -105,7 +105,7 @@ kind: Deployment
105105
metadata:
106106
name: xenorchestra-cloud-controller-manager
107107
labels:
108-
helm.sh/chart: xenorchestra-cloud-controller-manager-1.0.0-rc.1
108+
helm.sh/chart: xenorchestra-cloud-controller-manager-1.0.0-rc.2
109109
app.kubernetes.io/name: xenorchestra-cloud-controller-manager
110110
app.kubernetes.io/instance: xenorchestra-cloud-controller-manager
111111
app.kubernetes.io/version: "v1.0.0-rc.1"
@@ -157,6 +157,7 @@ spec:
157157
- --use-service-account-credentials
158158
- --secure-port=10258
159159
- --authorization-always-allow-paths=/healthz,/livez,/readyz,/metrics
160+
- --leader-elect=false
160161
ports:
161162
- name: metrics
162163
containerPort: 10258

0 commit comments

Comments
 (0)