Cannot create Team on OpenShift 4.11.59

[bastien-reinhardt]

Expected Behavior

Follow installation guide for Openshift and Multi-Juicer should work.

Current Behavior

Follow installation guide.
"Step: Creating a Team" does not work.

JuiceBalancer Pod Logs:
time="2024-05-31T07:47:47.425Z" level="info" msg="JuiceBalancer listening on port 3000!"
time="2024-05-31T07:50:20.649Z" level="info" msg="Team test doesn't have a JuiceShop deployment yet"
time="2024-05-31T07:50:20.741Z" level="info" msg="Reached 0/10 instances"
time="2024-05-31T07:50:23.533Z" level="info" msg="Creating JuiceShop Deployment for team 'test'"
time="2024-05-31T07:50:23.723Z" level="error" msg="Error while creating deployment or service for team test"
time="2024-05-31T07:50:23.723Z" level="error" msg="deployments.apps "t-test-juiceshop" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , "

Possible Solution

This solution works for me.
As per RedHat Solution add the "update" verb to the "deployments/finalizers" ressources on the "juice-balancer" namespace role.
The role should look like this:

rules:
  - verbs:
      - get
      - create
      - list
      - delete
      - patch
      - update
    apiGroups:
      - apps
    resources:
      - deployments
      - deployments/finalizers
  - verbs:
      - get
      - create
      - delete
    apiGroups:
      - ''
    resources:
      - services
  - verbs:
      - get
      - list
      - delete
    apiGroups:
      - ''
    resources:
      - pods

I hope this helps others :)

Cheers

[J12934]

Hi @bastien-reinhardt

Thanks for the pointer. Could you maybe open a or adding this role to the chart?

This should not impact normal Kubernetes users and would make it easier for openshift users to install MJ 👍

[GaryGSC]

Sorry if I'm hijacking this issue to generalize it. My org had a great time running a CTF using Multi-Juicer on OpenShift the other day. I have limited bandwidth to contribute a PR while on-the-clock, but here are some snags we ran into and how we overcame them.

---

We had to create a namespace. The docs already instruct us to create a Project, but we were trying to do everything using infrastructure-as-code, so I missed that step and got some unclear errors as a result.

---

We got the same finalizers issue listed above and added the following role:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: finalizers-update
rules:
  - apiGroups:
      - apps
    resources:
      - deployments/finalizers
    verbs:
      - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: balancer-finalizers-update
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: finalizers-update
subjects:
  - kind: ServiceAccount
    name: balancer
    namespace: multi-juicer-dev # The namespace/project we created

The cleaner solution is probably to adjust the existing Role's permissions.

---

We ran into file permissions issues on Juice Shop instance startup.

I resolved that by specifying a UID that matches the user from the Juice Shop Dockerfile in our values:

config:
  juiceShop:
    containerSecurityContext:
      runAsUser: 65532

(This could be useful for anyone.)

And then I had to create a RoleBinding so the ServiceAccount creating the instances could use the nonroot-v2 SCC, which allows it to run containers as user 65532:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: use-nonroot-v2-for-default
  namespace: multi-juicer-dev
subjects:
  - kind: ServiceAccount
    name: default
    namespace: multi-juicer-dev
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:openshift:scc:nonroot-v2

(This is specific to OpenShift. By default, OpenShift uses really high randomized UIDs. This SCC already exists to facilitate using specific UIDs that aren't root.)

---

We ran into a couple issues with OpenShift Routes, but those were possibly due to user error on my part.

Most notably, I needed to use the named port web rather than http:

apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: multi-juicer-dev
  annotations:
    haproxy.router.openshift.io/balance: roundrobin
    haproxy.router.openshift.io/disable_cookies: "true"
spec:
  host: example.com # Redacted
  to:
    kind: Service
    name: balancer
    weight: 100
  port:
    targetPort: web
  tls:
    termination: edge
    insecureEdgeTerminationPolicy: Redirect

I vaguely remember also having to adjust the Metrics targets, but I don't remember what was wrong or how I fixed things there.

---

Our Grafana instance didn't automatically pick up the dashboard, but it was straightforward to manually import the Dashboard JSON and then find-and-replace to adjust datasources to point to Prometheus:

"datasource": {
  "type": "prometheus",
  "uid": "e20d99d2-1ccc-4d32-940b-123abc456789"
},

I obtained a valid uid by copying it from another dashboard.

---

We used an entirely-different logging solution, so I can't speak to how Loki went.

[J12934]

Hi 🙋‍♂️ thanks for the report

Adding finalized thingy as a PR would be nice to get the that solved.

But he service port name thing is likely a error in the OC guide too, would be cool to get this fixed.

The startup issues with juiceshop are also interesting. Did you save the logs? It used to work with the usual random UIDs from OC back in the days but I guess something gras changed with the JS image which makes it not work anymore

[GaryGSC]

Right after these logs to stdout:

Port 3000 is available (OK)
Chatbot training data botDefaultTrainingData.json validated (OK)
Domain https://www.alchemy.com/ is reachable (OK)

We got this on stderr:

node:fs:2426

Error: EACCES: permission denied, open '.well-known/csaf/provider-metadata.json'
    at Object.writeFileSync (node:fs:2426:20)
    at replacizeFileSync (/juice-shop/node_modules/replace/replace.js:156:22)
    at module.exports (/juice-shop/node_modules/replace/replace.js:80:17)
    at prepareFilesystem (/juice-shop/build/data/datacreator.js:675:27)
    at exports.default (/juice-shop/build/data/datacreator.js:82:15)
    at async Object.start (/juice-shop/build/server.js:664:5)
    at async app (/juice-shop/build/app.js:33:5)

[barvhaim]

+1