The VCF Management Services introduced with 9.1 and newer versions consolidates existing and new components into a Kubernetes cluster, and while this does bring some benefits to the table, like significantly easier scaling and availability, it has brought the unfortunately side effect of massively increased resource usage
For production environments, this is less of an issue, but if you are running labs, or even a POC, these requirements cripple the environment, and in some cases, mean you cant run VCF 9.1
Now, this is also just Kubernetes, which means, unlike normal VMware appliances, we can downsize this manually without too much effort
The control plane node, the 4vCPU/10GB VM, I couldnt reduce the size of sadly, but the worker nodes, the 12vCPU/24GB VMs, can be, and the target size for those will be 8vCPU/16GB
The one thing worth noting, with the guide, to fit in 3 smaller worker nodes, you will have to chose between having the Identity Broker, or Log Management, I opted for Log Management, so this article will cover the uninstallation of the Identity Broker
I have not covered Real-Time metrics, yet, as my lab currently has resource constraints, but I will expand some information to cover that, but it will require a 4th worker node, the good news is that you should be able get Identity Broker in with it
It should go without saying, but this is NOT even remotely supported by Broadcom, and should, under no circumstances, be applied to a production environment
I have also not tested upgrades from this, so there is no guarantee, when 9.1.1 is released, that it can be easily upgraded, in theory it should be fine, but as its not released, I dont know
Important – By continuing you are agreeing to the disclaimer here
1 – System Overview
1.1 – System Architecture
Under the hood, this is a K8S cluster running on an auto scaler, the more resources that are needed, the more workers that are deployed, so you dont need to worry about sizing the cluster, it will automatically handle it
The control plane size is 4vCPU/10GB
For the workers, the sizes I have seen are
- Small – 12vCPU/24GB
- Medium – 24vCPU/48GB
When you reach ~7 workers, the cluster will replace them with the next size up
Key services running on this are
- VCF Service Runtime
- Fleet LCM
- SDDC LCM
- Depot
- Identity Broker
- Salt RaaS
- Salt Master
- Telemetry
Additional services are
- Log Management
- Real-Time Metrics
1.2 – Accessing The Cluster With SSH
To access the cluster you’ll need to SSH into it, you will need the password you set during the upgrade/deployment, the [email protected] password
To then access it, in vSphere, find the control plane node, the 4vCPU/10GB VM, SSH to its IP address as the vmware-system-user, and enter the password prompted
Then switch to root with
sudo -i
1.3 – Pod List
Based on my cluster running the Log Management service only as an extra, the following pods were running and using these resources
Bare in mind this is a point in time snapshot, and doesnt factor in requests or limits by the pods, just their current usage
| Namespace | Name | CPU (Milicores) | Memory(MiB) |
|---|---|---|---|
| kube-system | antrea-agent-7968f | 25m | 239Mi |
| kube-system | antrea-agent-7bxpd | 49m | 240Mi |
| kube-system | antrea-agent-bs5zw | 34m | 147Mi |
| kube-system | antrea-agent-sd8hw | 58m | 242Mi |
| kube-system | antrea-controller-78975945b4-95rl5 | 11m | 65Mi |
| kube-system | coredns-5496b74c4-p9lrh | 8m | 22Mi |
| kube-system | coredns-5496b74c4-rnqgt | 6m | 19Mi |
| kube-system | etcd-aerosphere-94pj5 | 119m | 346Mi |
| kube-system | kube-apiserver-aerosphere-94pj5 | 277m | 2311Mi |
| kube-system | kube-controller-manager-aerosphere-94pj5 | 69m | 192Mi |
| kube-system | kube-proxy-27vx6 | 10m | 31Mi |
| kube-system | kube-proxy-bg4lk | 32m | 29Mi |
| kube-system | kube-proxy-q29qp | 24m | 32Mi |
| kube-system | kube-proxy-rsjp9 | 13m | 31Mi |
| kube-system | kube-scheduler-aerosphere-94pj5 | 11m | 38Mi |
| kube-system | kube-vip-aerosphere-94pj5 | 4m | 18Mi |
| kube-system | vsphere-cpi-mxnnd | 4m | 22Mi |
| kube-system | vsphere-csi-controller-c8679dbc5-shbv9 | 8m | 93Mi |
| kube-system | vsphere-csi-node-48qf7 | 8m | 51Mi |
| kube-system | vsphere-csi-node-lmlxv | 2m | 60Mi |
| kube-system | vsphere-csi-node-sl8f6 | 8m | 62Mi |
| kube-system | vsphere-csi-node-v9j5b | 3m | 60Mi |
| ops-logs | log-processor-0 | 74m | 3459Mi |
| ops-logs | log-store-0 | 139m | 6643Mi |
| salt | salt-master-8674dd877b-2xbkx | 31m | 960Mi |
| salt | salt-minion-6c58844558-dvwj2 | 57m | 127Mi |
| salt-raas | pgdatabase-0 | 20m | 594Mi |
| salt-raas | raas-5857f54b4f-n6vc4 | 36m | 698Mi |
| salt-raas | redis-6cccfb46f4-q56cb | 4m | 7Mi |
| telemetry | telemetry-acceptor-79746d45c6-78s9q | 2m | 445Mi |
| vcf-fleet-depot | depot-service-77cb99b56b-jpfc8 | 23m | 505Mi |
| vcf-fleet-depot | distribution-service-84574b5cd-vfs8h | 1m | 7Mi |
| vcf-fleet-lcm | vcf-fleet-build-service-fleetbuild-7b8464ffb8-mvc2z | 37m | 1005Mi |
| vcf-fleet-lcm | vcf-fleet-lcm-db-0 | 18m | 291Mi |
| vcf-fleet-lcm | vcf-fleet-lcm-db-1 | 27m | 698Mi |
| vcf-fleet-lcm | vcf-fleet-lcm-db-2 | 6m | 235Mi |
| vcf-fleet-lcm | vcf-fleet-upgrade-service-fleetupgrade-799c99f9bc-lfwbf | 23m | 1190Mi |
| vcf-sddc-lcm | vcf-sddc-build-service-sddcbuild-79886b96bb-l8hnv | 124m | 1394Mi |
| vcf-sddc-lcm | vcf-sddc-lcm-db-0 | 67m | 535Mi |
| vcf-sddc-lcm | vcf-sddc-lcm-db-1 | 10m | 159Mi |
| vcf-sddc-lcm | vcf-sddc-lcm-db-2 | 6m | 219Mi |
| vcf-sddc-lcm | vcf-sddc-upgrade-service-sddcupgrade-79d4584486-8v2sg | 73m | 1043Mi |
| vidb-external | vidb-postgres-instance-0 | 28m | 1495Mi |
| vidb-external | vidb-service-6759897d7c-vptr9 | 82m | 1306Mi |
| vmsp-platform | alertmanager-kube-prometheus-stack-alertmanager-0 | 23m | 47Mi |
| vmsp-platform | argo-workflows-workflow-controller-57bbc66676-m7nz7 | 7m | 66Mi |
| vmsp-platform | capi-controller-manager-d55798b89-fp9c5 | 2m | 35Mi |
| vmsp-platform | capi-controller-manager-d55798b89-t26vr | 5m | 59Mi |
| vmsp-platform | capi-ipam-in-cluster-controller-manager-5dc6c6b847-k5zfp | 1m | 14Mi |
| vmsp-platform | capi-ipam-in-cluster-controller-manager-5dc6c6b847-lq8dw | 3m | 16Mi |
| vmsp-platform | capi-kubeadm-bootstrap-controller-manager-5fb45864db-cb8gn | 3m | 35Mi |
| vmsp-platform | capi-kubeadm-bootstrap-controller-manager-5fb45864db-gjdlw | 2m | 28Mi |
| vmsp-platform | capi-kubeadm-control-plane-controller-manager-ffd8fc9f-85s2b | 4m | 45Mi |
| vmsp-platform | capi-kubeadm-control-plane-controller-manager-ffd8fc9f-r4s5h | 2m | 31Mi |
| vmsp-platform | capv-controller-manager-7cd6655bff-qvbfx | 3m | 30Mi |
| vmsp-platform | capv-controller-manager-7cd6655bff-wk5q4 | 4m | 72Mi |
| vmsp-platform | cert-manager-58d87667b-dzh6l | 1m | 35Mi |
| vmsp-platform | cert-manager-cainjector-967cc9898-5l68x | 1m | 94Mi |
| vmsp-platform | cert-manager-webhook-68987786cf-hl6j2 | 1m | 19Mi |
| vmsp-platform | cluster-autoscaler-clusterapi-cluster-autoscaler-7c7b8d4c9qx466 | 2m | 51Mi |
| vmsp-platform | envoy-gateway-5bb66999-jk8jq | 3m | 132Mi |
| vmsp-platform | event-tailer-event-tailer-0 | 1m | 17Mi |
| vmsp-platform | helm-controller-577d87cc94-jf6xt | 23m | 77Mi |
| vmsp-platform | hooks-server-synthetic-checker-854d6b5747-tbf9l | 2m | 42Mi |
| vmsp-platform | kube-prometheus-stack-kube-state-metrics-f87d55467-wnlm4 | 9m | 102Mi |
| vmsp-platform | kube-prometheus-stack-operator-776b689c68-fkzdf | 1m | 48Mi |
| vmsp-platform | kube-prometheus-stack-prometheus-node-exporter-lgh5b | 2m | 15Mi |
| vmsp-platform | kube-prometheus-stack-prometheus-node-exporter-skbrc | 6m | 43Mi |
| vmsp-platform | kube-prometheus-stack-prometheus-node-exporter-xjfmn | 7m | 16Mi |
| vmsp-platform | kube-prometheus-stack-prometheus-node-exporter-z8p8k | 5m | 15Mi |
| vmsp-platform | kube-vip-dr69g | 1m | 17Mi |
| vmsp-platform | kube-vip-nfj58 | 1m | 16Mi |
| vmsp-platform | kube-vip-xlsj2 | 7m | 21Mi |
| vmsp-platform | logging-operator-cc6f9f696-j4qc7 | 2m | 79Mi |
| vmsp-platform | logging-operator-filelogs-fluentbit-7fh7v | 1m | 6Mi |
| vmsp-platform | logging-operator-filelogs-fluentbit-nccbk | 1m | 4Mi |
| vmsp-platform | logging-operator-filelogs-fluentbit-sx7tw | 1m | 4Mi |
| vmsp-platform | logging-operator-filelogs-fluentbit-xf2tn | 2m | 7Mi |
| vmsp-platform | logging-operator-fluentbit-2g495 | 5m | 19Mi |
| vmsp-platform | logging-operator-fluentbit-5s4dj | 4m | 13Mi |
| vmsp-platform | logging-operator-fluentbit-7jnwl | 11m | 18Mi |
| vmsp-platform | logging-operator-fluentbit-pxrfq | 5m | 21Mi |
| vmsp-platform | logging-operator-fluentd-0 | 56m | 630Mi |
| vmsp-platform | logging-operator-systemlogs-fluentbit-96l5v | 1m | 5Mi |
| vmsp-platform | logging-operator-systemlogs-fluentbit-fg4xm | 20m | 10Mi |
| vmsp-platform | logging-operator-systemlogs-fluentbit-jk7q2 | 1m | 5Mi |
| vmsp-platform | logging-operator-systemlogs-fluentbit-wsnv2 | 2m | 5Mi |
| vmsp-platform | logs-tailer-host-tailer-d6bxn | 1m | 4Mi |
| vmsp-platform | logs-tailer-host-tailer-dflj6 | 1m | 3Mi |
| vmsp-platform | logs-tailer-host-tailer-jtz2f | 2m | 3Mi |
| vmsp-platform | logs-tailer-host-tailer-znbhw | 2m | 4Mi |
| vmsp-platform | metadata-service-gdnvc | 1m | 21Mi |
| vmsp-platform | metadata-service-jf5tx | 1m | 21Mi |
| vmsp-platform | metadata-service-jxx9m | 2m | 22Mi |
| vmsp-platform | metrics-server-c77f6c7f8-lk9d8 | 3m | 30Mi |
| vmsp-platform | ndc-controller-manager-7547fdf4bf-jbcvb | 1m | 10Mi |
| vmsp-platform | ndc-controller-manager-7547fdf4bf-x9njg | 2m | 21Mi |
| vmsp-platform | node-problem-detector-cbmt9 | 1m | 16Mi |
| vmsp-platform | node-problem-detector-n85vs | 1m | 12Mi |
| vmsp-platform | node-problem-detector-nktz7 | 1m | 12Mi |
| vmsp-platform | node-problem-detector-qwmx9 | 6m | 13Mi |
| vmsp-platform | ops-logs-gateway-589859bd7-zp2k9 | 29m | 68Mi |
| vmsp-platform | prometheus-kube-prometheus-stack-prometheus-0 | 26m | 643Mi |
| vmsp-platform | proxy-service-778f9cb5f9-dlkrv | 1m | 18Mi |
| vmsp-platform | seaweedfs-cleanup-54787b956c-zkvnm | 0m | 17Mi |
| vmsp-platform | seaweedfs-filer-0 | 2m | 160Mi |
| vmsp-platform | seaweedfs-master-0 | 3m | 69Mi |
| vmsp-platform | seaweedfs-volume-0 | 1m | 41Mi |
| vmsp-platform | seaweedfs-volume-1 | 1m | 60Mi |
| vmsp-platform | seaweedfs-volume-2 | 1m | 71Mi |
| vmsp-platform | source-controller-755f68b474-dhhns | 5m | 51Mi |
| vmsp-platform | strimzi-cluster-operator-f7bf56558-s227j | 4m | 243Mi |
| vmsp-platform | support-bundle-84989d847f-kbbbz | 2m | 21Mi |
| vmsp-platform | support-bundle-cleanup-nodes-2dvnt | 0m | 4Mi |
| vmsp-platform | support-bundle-cleanup-nodes-6ht9p | 0m | 4Mi |
| vmsp-platform | support-bundle-cleanup-nodes-dcq2g | 0m | 4Mi |
| vmsp-platform | support-bundle-cleanup-nodes-qs4fj | 0m | 3Mi |
| vmsp-platform | support-bundle-logcopier-2klqn | 18m | 20Mi |
| vmsp-platform | support-bundle-logcopier-cdlmt | 12m | 21Mi |
| vmsp-platform | support-bundle-logcopier-dzz4t | 5m | 22Mi |
| vmsp-platform | support-bundle-logcopier-event-tailer-597448947b-bpq4r | 1m | 6Mi |
| vmsp-platform | support-bundle-logcopier-m2nnr | 132m | 26Mi |
| vmsp-platform | support-bundle-logoffloader-7v7jn | 4m | 65Mi |
| vmsp-platform | support-bundle-logoffloader-gn6c7 | 0m | 62Mi |
| vmsp-platform | support-bundle-logoffloader-p7chj | 11m | 57Mi |
| vmsp-platform | support-bundle-logoffloader-xj5mv | 284m | 62Mi |
| vmsp-platform | synthetic-checker-6df68966df-xkfjx | 24m | 75Mi |
| vmsp-platform | trust-manager-5476545759-4w7fw | 3m | 42Mi |
| vmsp-platform | velero-6b9655b487-r5mzl | 72m | 60Mi |
| vmsp-platform | vmsp-agent-5458874df9-5r8rs | 5m | 191Mi |
| vmsp-platform | vmsp-agent-configure-node-6nhfg | 0m | 2Mi |
| vmsp-platform | vmsp-agent-configure-node-lq5nb | 0m | 2Mi |
| vmsp-platform | vmsp-agent-configure-node-wsw5q | 0m | 2Mi |
| vmsp-platform | vmsp-etcd-0 | 11m | 187Mi |
| vmsp-platform | vmsp-gateway-6f4f75cd8-pprrc | 12m | 103Mi |
| vmsp-platform | vmsp-identity-564649bfbb-c5wmq | 4m | 42Mi |
| vmsp-platform | vmsp-identity-564649bfbb-sjr54 | 1m | 27Mi |
| vmsp-platform | vmsp-operator-784cf768d8-m8wdp | 13m | 217Mi |
| vmsp-platform | zalando-postgres-operator-588894b94-8kw9v | 2m | 53Mi |
| vmsp-platform | zot-1-0 | 10m | 61Mi |
| vmsp-platform | zot-1-configure-node-4jnz6 | 0m | 8Mi |
| vmsp-platform | zot-1-configure-node-kprhg | 0m | 8Mi |
| vmsp-platform | zot-1-configure-node-x8lfk | 0m | 8Mi |
| vmsp-platform | zot-1-configure-node-zst54 | 0m | 7Mi |
| vmsp-policies | kyverno-admission-controller-558cd4c545-px9s5 | 4m | 79Mi |
| vmsp-policies | kyverno-admission-controller-558cd4c545-sllq5 | 8m | 70Mi |
| vmsp-policies | kyverno-admission-controller-558cd4c545-sm2l6 | 18m | 81Mi |
| vmsp-policies | kyverno-background-controller-67966cc6d8-s9thw | 15m | 107Mi |
| vmsp-policies | kyverno-cleanup-controller-6c946746c7-s9g4w | 3m | 49Mi |
1.4 – Pod Architecture Diagram
Under the hood of my small non HA cluster, this is what pods we have, what namespaces they are in, and where replcias sit

2 – Log Management
2.1 – Log Management Optimisation
For this, we are aiming at setting the resources to 4vCPU/8GB in line with the old lab/POC deployment size
The default YAMLs call for
Log Processor – 3vCPU/6GB with no limit on the CPU
Log Store – 5vCPU/10GB
The target we want to update these too, factoring in the JVM heap on the log store is
Log Processor – 1vCPU/3GB with no limit on the CPU allowing it to burst
Log Store – 1vCPU/6GB with a 2vCPU limit for burst – Any lower and KVM would get OOM killed
While this is a lot of memory removed on the store, this will largely be the cache data that the old log appliance used to use so the performance should be fine for a small lab environment
We can edit the log processor to this by patching the YAML with
kubectl patch statefulset log-processor -n ops-logs --type='json' -p='[
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/cpu", "value": "1"},
{"op": "add", "path": "/spec/template/spec/containers/0/resources/limits/cpu", "value": "2"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/memory", "value": "2Gi"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/limits/memory", "value": "3Gi"}
]'
And the log store with this
O opted for 3 cores as a limit for the store, I did have this at 2, but queries were slow, and this over provisioning should be fine since the cluster is mostly idle
kubectl patch statefulset log-store -n ops-logs --type='json' -p='[
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/cpu", "value": "1"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/limits/cpu", "value": "3"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/memory", "value": "4.5Gi"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/limits/memory", "value": "5Gi"},
{"op": "replace", "path": "/spec/template/spec/containers/0/env/5/value", "value": "-Xms3584m -Xmx3584m -Djavax.net.ssl.trustStore=/usr/share/opensearch/config/truststore/bundle.jks -Djavax.net.ssl.trustStorePassword=changeit -Djava.util.logging.config.file=/usr/share/opensearch/config/bc.logging.properties"}
]'
This should start a restart which we can check on with
kubectl rollout status statefulset/log-processor -n ops-logs
kubectl rollout status statefulset/log-store -n ops-logs
We can also view the pods with, this will take a while to restart
kubectl top pods -n ops-logs
I did get to this point where the log processor was idle and couldnt get resources from the log-store like this, and it didnt seem to recover

We can recreate it with
kubectl delete pod log-processor-0 -n ops-logs
Which kicked it back into life now the log store is up and running, if you left it long enough the system may recover it
2.2 Original YAML Backups
2.2.1 – log-processor.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
annotations:
meta.helm.sh/release-name: vcf-ops-logs
meta.helm.sh/release-namespace: ops-logs
creationTimestamp: "2026-05-12T16:04:45Z"
generation: 1
labels:
app.kubernetes.io/instance: vcf-ops-logs
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: vcf-ops-logs
app.kubernetes.io/version: 0.0.1574
helm.sh/chart: vcf-ops-logs-0.0.1574
helm.toolkit.fluxcd.io/name: vcf-ops-logs
helm.toolkit.fluxcd.io/namespace: ops-logs
monitoring: vcf-ops-logs-app
ops.vmsp.vmware.com/type: log-processor-service
name: log-processor
namespace: ops-logs
resourceVersion: "13212589"
uid: 0e479d18-21a5-4753-92e6-c868482dfdc8
spec:
persistentVolumeClaimRetentionPolicy:
whenDeleted: Retain
whenScaled: Retain
podManagementPolicy: Parallel
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/instance: vcf-ops-logs
app.kubernetes.io/name: vcf-ops-logs
serviceName: log-processor-svc
template:
metadata:
annotations:
identity.vmsp.vmware.com/serviceaccount: opslogs-client
labels:
app.kubernetes.io/instance: vcf-ops-logs
app.kubernetes.io/name: vcf-ops-logs
monitoring: vcf-ops-logs-app
ops.vmsp.vmware.com/type: log-processor-resource
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchLabels:
app.kubernetes.io/instance: vcf-ops-logs
app.kubernetes.io/name: vcf-ops-logs
topologyKey: kubernetes.io/hostname
weight: 100
containers:
- envFrom:
- configMapRef:
name: ops-logs-config
image: registry.vmsp-platform.svc.cluster.local:5000/images/ops-logs/9.1.0.0.25346055/ops-logs/vcf-ops-logs:0.0.1574
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 15
httpGet:
path: /healthz
port: web-nonssl
scheme: HTTP
initialDelaySeconds: 45
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 1
name: vcf-ops-logs
ports:
- containerPort: 9000
name: cfapi-nonssl
protocol: TCP
- containerPort: 514
name: syslog-nonssl
protocol: TCP
- containerPort: 9543
name: cfapi-ssl
protocol: TCP
- containerPort: 1514
name: syslog-ssl
protocol: TCP
- containerPort: 8080
name: web-nonssl
protocol: TCP
readinessProbe:
failureThreshold: 15
httpGet:
path: /readyz
port: web-nonssl
scheme: HTTP
initialDelaySeconds: 45
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
ephemeral-storage: 2Gi
memory: 6Gi
requests:
cpu: "3"
ephemeral-storage: 2Gi
memory: 6Gi
securityContext:
capabilities:
drop:
- CAP_CHOWN
- CAP_DAC_OVERRIDE
- CAP_FOWNER
- CAP_NET_RAW
- CAP_MKNOD
startupProbe:
failureThreshold: 60
httpGet:
path: /healthz
port: web-nonssl
scheme: HTTP
initialDelaySeconds: 45
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /opt/vmware/ops-logs/data
name: opslogs-data
- mountPath: /opt/vmware/ops-logs/debug
name: opslogs-debug
- mountPath: /opt/vmware/certs/opensearch
name: opensearch-certs
- mountPath: /opt/vmware/certs/ops-logs
name: opslogs-certs
- mountPath: /opt/vmware/certs/ingress
name: ingress-certs
- mountPath: /opt/vmware/certs/auth
name: opslogs-auth
- mountPath: /opt/vmware/secrets
name: logs-secrets
readOnly: true
- mountPath: /opt/vmware/security/platform-trust
name: vmsp-certs
readOnly: true
- image: registry.vmsp-platform.svc.cluster.local:5000/images/ops-logs/9.1.0.0.25346055/ops-logs/vcf-ops-logs-ui:0.0.1574
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /nginx-health
port: http-ui
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
name: ui-server
ports:
- containerPort: 8081
name: http-ui
protocol: TCP
- containerPort: 8444
name: https-ui
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /nginx-health
port: http-ui
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
cpu: 200m
memory: 256Mi
requests:
cpu: 100m
memory: 128Mi
securityContext:
capabilities:
drop:
- CAP_CHOWN
- CAP_DAC_OVERRIDE
- CAP_FOWNER
- CAP_NET_RAW
- CAP_MKNOD
runAsNonRoot: true
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/nginx/conf.d
name: ui-nginx-config
readOnly: true
- mountPath: /etc/nginx/certs
name: ui-certs
readOnly: true
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 3000
runAsGroup: 3000
runAsUser: 2000
serviceAccount: log-processor
serviceAccountName: log-processor
terminationGracePeriodSeconds: 240
volumes:
- name: opensearch-certs
secret:
defaultMode: 384
secretName: tls-for-opensearch-key-pair
- name: opensearch-admin-certs
secret:
defaultMode: 384
secretName: tls-for-opensearch-admin-key-pair
- name: opslogs-certs
secret:
defaultMode: 384
secretName: tls-for-opslogs-key-pair
- name: ingress-certs
secret:
defaultMode: 384
secretName: ingress-service-cert
- name: opslogs-auth
secret:
defaultMode: 384
secretName: auth-keypair-secret
- name: logs-secrets
secret:
defaultMode: 384
secretName: ops-logs-secrets
- configMap:
defaultMode: 384
items:
- key: bundle.pem
path: bundle.pem
name: platform-trust
name: vmsp-certs
- configMap:
defaultMode: 384
name: log-processor-ui-nginx-config
name: ui-nginx-config
- name: ui-certs
secret:
defaultMode: 384
secretName: tls-for-opslogs-key-pair
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
volumeClaimTemplates:
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
labels:
velero.io/exclude-from-backup: "true"
volume-groups.vmsp.vmware.com/resizable: "false"
name: opslogs-debug
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 15Gi
volumeMode: Filesystem
status:
phase: Pending
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
labels:
velero.io/exclude-from-backup: "true"
volume-groups.vmsp.vmware.com/resizable: "false"
name: opslogs-data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 60Gi
volumeMode: Filesystem
status:
phase: Pending
status:
availableReplicas: 1
collisionCount: 0
currentReplicas: 1
currentRevision: log-processor-774bcc4967
observedGeneration: 1
readyReplicas: 1
replicas: 1
updateRevision: log-processor-774bcc4967
updatedReplicas: 1
2.2.2 – log-store.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
annotations:
majorVersion: "3"
meta.helm.sh/release-name: vcf-ops-logs
meta.helm.sh/release-namespace: ops-logs
creationTimestamp: "2026-05-12T16:04:45Z"
generation: 2
labels:
app.kubernetes.io/component: log-store
app.kubernetes.io/instance: vcf-ops-logs
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: opensearch
app.kubernetes.io/version: 3.0.0
helm.sh/chart: opensearch-3.0.0
helm.toolkit.fluxcd.io/name: vcf-ops-logs
helm.toolkit.fluxcd.io/namespace: ops-logs
ops.vmsp.vmware.com/type: log-store-service
name: log-store
namespace: ops-logs
resourceVersion: "13211289"
uid: 5859f867-be7b-42cc-9cf1-0a33fa15c58d
spec:
persistentVolumeClaimRetentionPolicy:
whenDeleted: Retain
whenScaled: Retain
podManagementPolicy: Parallel
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/instance: vcf-ops-logs
app.kubernetes.io/name: opensearch
serviceName: log-store-headless
template:
metadata:
annotations:
configchecksum: 88acb1d6fcb84e84077a20e3544fdf5c07fee8c04a7fcd3e5d35d28f6d76294
labels:
app.kubernetes.io/component: log-store
app.kubernetes.io/instance: vcf-ops-logs
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: opensearch
app.kubernetes.io/version: 3.0.0
helm.sh/chart: opensearch-3.0.0
ops.vmsp.vmware.com/type: log-store-resource
name: log-store
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchExpressions:
- key: app.kubernetes.io/instance
operator: In
values:
- vcf-ops-logs
- key: app.kubernetes.io/name
operator: In
values:
- opensearch
topologyKey: kubernetes.io/hostname
weight: 1
automountServiceAccountToken: false
containers:
- env:
- name: node.name
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: cluster.initial_master_nodes
value: log-store-0,
- name: discovery.seed_hosts
value: log-store-headless
- name: cluster.name
value: log-store
- name: network.host
value: 0.0.0.0
- name: OPENSEARCH_JAVA_OPTS
value: -Xms5120m -Xmx5120m -Djavax.net.ssl.trustStore=/usr/share/opensearch/config/truststore/bundle.jks
-Djavax.net.ssl.trustStorePassword=changeit -Djava.util.logging.config.file=/usr/share/opensearch/config/bc.logging.properties
- name: node.roles
value: master,ingest,data,remote_cluster_client,warm,
- name: DISABLE_INSTALL_DEMO_CONFIG
value: "true"
- name: LOG_PROCESSOR_USER
value: log-processor-user
- name: LOG_PROCESSOR_ROLE
value: log-processor-role
- name: KEYSTORE_UTIL_SECRET
valueFrom:
configMapKeyRef:
key: LOGS_OPENSEARCH_KEYSTORE_UTIL
name: ops-logs-config
- name: CLIENT_NAME
valueFrom:
configMapKeyRef:
key: CLIENT_NAME
name: ops-logs-config
image: registry.vmsp-platform.svc.cluster.local:5000/images/ops-logs/9.1.0.0.25346055/ops-logs/opensearchproject/log-store:3.0.0-fips-1.8
imagePullPolicy: IfNotPresent
name: opensearch
ports:
- containerPort: 9200
name: http
protocol: TCP
- containerPort: 9300
name: transport
protocol: TCP
- containerPort: 9600
name: metrics
protocol: TCP
readinessProbe:
failureThreshold: 3
periodSeconds: 5
successThreshold: 1
tcpSocket:
port: 9200
timeoutSeconds: 3
resources:
limits:
cpu: "5"
ephemeral-storage: 2Gi
memory: 10Gi
requests:
cpu: "5"
ephemeral-storage: 2Gi
memory: 10Gi
securityContext:
capabilities:
drop:
- ALL
runAsNonRoot: true
runAsUser: 1000
startupProbe:
failureThreshold: 30
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: 9200
timeoutSeconds: 3
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /usr/share/opensearch/data
name: log-store
- mountPath: /usr/share/opensearch/config/certs
name: tls-for-opensearch-key-pair
- mountPath: /usr/share/opensearch/config/certs/admin
name: tls-for-opensearch-admin-key-pair
- mountPath: /usr/share/opensearch/config/certs/log-processor
name: tls-for-opslogs-key-pair
- mountPath: /usr/share/opensearch/config/ops-logs-secrets
name: ops-logs-secrets-mount
- mountPath: /usr/share/opensearch/config/s3-secrets
name: s3-secret-mount
- mountPath: /usr/share/opensearch/config/s3-backup-restore-secrets
name: s3-backup-restore-secret-mount
- mountPath: /usr/share/opensearch/config/opensearch.yml
name: config-emptydir
subPath: opensearch.yml
- mountPath: /usr/share/opensearch/cayman-opensearch-util.sh
name: cayman-opensearch-util
readOnly: true
subPath: cayman-opensearch-util.sh
- mountPath: /usr/share/opensearch/config/bc.logging.properties
name: cayman-opensearch-bc-logging
readOnly: true
subPath: bc.logging.properties
- mountPath: /usr/share/opensearch/config/truststore
name: truststore-volume
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
initContainers:
- args:
- chown -R 1000:1000 /usr/share/opensearch/data
command:
- sh
- -c
image: registry.vmsp-platform.svc.cluster.local:5000/images/ops-logs/9.1.0.0.25346055/ops-logs/busybox:latest
imagePullPolicy: IfNotPresent
name: fsgroup-volume
resources:
limits:
cpu: 100m
memory: 128Mi
requests:
cpu: 100m
memory: 128Mi
securityContext:
runAsUser: 0
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /usr/share/opensearch/data
name: log-store
- command:
- sh
- -c
- |
#!/usr/bin/env bash
cp -r /tmp/configfolder/* /tmp/config/
image: registry.vmsp-platform.svc.cluster.local:5000/images/ops-logs/9.1.0.0.25346055/ops-logs/opensearchproject/log-store:3.0.0-fips-1.8
imagePullPolicy: IfNotPresent
name: configfile
resources:
limits:
cpu: 100m
memory: 128Mi
requests:
cpu: 100m
memory: 128Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /tmp/config/
name: config-emptydir
- mountPath: /tmp/configfolder/opensearch.yml
name: config
subPath: opensearch.yml
priorityClassName: high
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 1000
runAsUser: 1000
serviceAccount: log-store-sa
serviceAccountName: log-store-sa
terminationGracePeriodSeconds: 240
volumes:
- configMap:
defaultMode: 420
name: log-store-config
name: config
- emptyDir: {}
name: config-emptydir
- name: tls-for-opensearch-key-pair
secret:
defaultMode: 384
secretName: tls-for-opensearch-key-pair
- name: tls-for-opensearch-admin-key-pair
secret:
defaultMode: 384
secretName: tls-for-opensearch-admin-key-pair
- name: tls-for-opslogs-key-pair
secret:
defaultMode: 384
secretName: tls-for-opslogs-key-pair
- name: ops-logs-secrets-mount
secret:
defaultMode: 384
secretName: ops-logs-secrets
- name: s3-secret-mount
secret:
defaultMode: 384
secretName: s3-secret
- name: s3-backup-restore-secret-mount
secret:
defaultMode: 384
secretName: ops-logs-bucket-credentials
- configMap:
defaultMode: 384
name: cayman-opensearch-util-script
name: cayman-opensearch-util
- configMap:
defaultMode: 384
name: cayman-opensearch-bc-logging
name: cayman-opensearch-bc-logging
- configMap:
defaultMode: 384
items:
- key: bundle.jks
path: bundle.jks
name: platform-trust-no-default
name: truststore-volume
updateStrategy:
type: RollingUpdate
volumeClaimTemplates:
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
labels:
app.kubernetes.io/component: log-store
app.kubernetes.io/instance: vcf-ops-logs
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: opensearch
app.kubernetes.io/version: 3.0.0
helm.sh/chart: opensearch-3.0.0
vcf.vmware.com/keep-during-restore: "true"
velero.io/exclude-from-backup: "true"
name: log-store
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 500Gi
volumeMode: Filesystem
status:
phase: Pending
status:
availableReplicas: 1
collisionCount: 0
currentReplicas: 1
currentRevision: log-store-95968949d
observedGeneration: 2
readyReplicas: 1
replicas: 1
updateRevision: log-store-95968949d
updatedReplicas: 1
3 – Ops Log Gateway
3.1 – Ops Log Gateway Optimisation
This service was the best place to start reducing vCPU, its got 4 cores assigned but is actively using less than 100m, memory wise it was under 100MiB with an assigned 2GiB
The log gateway is managed by the envoy gateway, so we need to edit it there
These resources are managed by a helm package, so while this change will effect the system, it will revert on an update
I opted for 400m CPU as I tried 250m, but it sat at that for a while, I assume doing background processing, which may make sense that it can use a lot given it had 4000m assigned before
kubectl patch envoyproxy ops-logs-gateway-config -n vmsp-platform --type='json' -p='[
{"op": "replace", "path": "/spec/provider/kubernetes/envoyDeployment/patch/value/spec/template/spec/containers/0/resources/requests/cpu", "value": "400m"},
{"op": "replace", "path": "/spec/provider/kubernetes/envoyDeployment/patch/value/spec/template/spec/containers/0/resources/requests/memory", "value": "256Mi"},
{"op": "replace", "path": "/spec/provider/kubernetes/envoyDeployment/patch/value/spec/template/spec/containers/0/resources/limits/memory", "value": "512Mi"}
]'
The pods will be auto recreated
3.2 – Original YAML Backups
3.2.1 – envoy-proxy.yaml
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
annotations:
meta.helm.sh/release-name: ops-logs-gateway
meta.helm.sh/release-namespace: vmsp-platform
creationTimestamp: "2026-05-12T16:04:50Z"
generation: 1
labels:
app.kubernetes.io/managed-by: Helm
helm.toolkit.fluxcd.io/name: ops-logs-gateway
helm.toolkit.fluxcd.io/namespace: vmsp-platform
name: ops-logs-gateway-config
namespace: vmsp-platform
resourceVersion: "176946"
uid: 1f3c50f1-4e24-4b87-9578-076a35b941a8
spec:
logging:
level:
default: warn
provider:
kubernetes:
envoyDeployment:
container:
volumeMounts:
- mountPath: /tmp
name: shutdown-manager
name: ops-logs-gateway
patch:
type: StrategicMerge
value:
metadata:
labels:
ops.vmsp.vmware.com/type: envoygatewayproxy-service
spec:
template:
metadata:
labels:
ops.vmsp.vmware.com/type: envoygatewayproxy-service
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchLabels:
gateway.envoyproxy.io/owning-gateway-name: ops-logs-gateway
topologyKey: kubernetes.io/hostname
weight: 100
containers:
- image: registry.vmsp-platform.svc.cluster.local:5000/images/envoy:v1.34.1
name: envoy
resources:
limits:
memory: 2Gi
requests:
cpu: 4000m
memory: 2Gi
volumeMounts:
- mountPath: /etc/ssl/certs
name: config-volume
- image: registry.vmsp-platform.svc.cluster.local:5000/images/envoyproxy-gateway:v1.5.0
name: shutdown-manager
resources:
limits:
cpu: 50m
memory: 256Mi
requests:
cpu: 5m
memory: 32Mi
volumeMounts:
- mountPath: /tmp
name: shutdown-manager
priorityClassName: vmsp-high
pod:
volumes:
- emptyDir:
medium: Memory
name: shutdown-manager
- configMap:
items:
- key: bundle.pem
path: ca-certificates.crt
name: platform-trust
name: config-volume
replicas: 1
envoyService:
allocateLoadBalancerNodePorts: false
externalTrafficPolicy: Local
loadBalancerClass: envoy
name: ops-logs-gateway
type: LoadBalancer
type: Kubernetes
telemetry:
accessLog:
disable: true
4 – Identity Broker Uninstallation
Unfortunately, we cant get the resources low enough with the log management server installed to downsize the worker nodes, so if you want to drop them, you’ll need to uninstall the Identity Broker, or Log Management
Broadcom have a python script that can be used to cleanup components which lists the Identity Broker, so we will be using that for this, which can be found here
Download the script that the bottom

I am running this on Ubuntu, you’ll need to be in the directory of the script and run
chmod +x ./cleanup_component.py
Then to check components, we can run
python3 cleanup_component.py list vsp-component --fleet-fqdn <fleet-lcm-fqdn> --vcf-services-runtime-fqdn <vcf-services-runtime-fqdn> --vcf-services-runtime-username <vcf-services-runtime-username>
For my system, which has the service runtime of aerosphere.leaha.co.uk, and a fleet service fqdn of, skyveil.leaha.co.uk, I ran the following
python3 cleanup_component.py delete vsp-component --component-id aceb5c33-d3ab-49cc-b4c4-9aecfbfe15d9 --fleet-fqdn skyveil.leaha.co.uk --vcf-services-runtime-fqdn aerosphere.leaha.co.uk --vcf-services-runtime-username admin
You’ll be asked for a password, which from the deployment/upgrade should be the same as the vmware-system-user, and you’ll see an output like this
Enter password for user 'admin':
Authenticating with VSP platform (aerosphere.leaha.co.uk)...
Authentication successful.
Fetching VSP components from Fleet...
Available VSP Components:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ID | Type | Deployment | Version | FQDN | VCF Services Runtime FQDN
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
aceb5c33-d3ab-49cc-b4c4-9aecfbfe15d9 | Identity broker | VSP | 9.1.0.0.25368698 | nimbus.leaha.co.uk | aerosphere.leaha.co.uk
95e53a26-3f09-4284-a53f-3c64ca7c48fe | Log management | VSP | 9.1.0.0.25346055 | mistral.leaha.co.uk | aerosphere.leaha.co.uk
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Here we can see the Identity Broker component
The generic command to remove this is
python3 cleanup_component.py delete vsp-component --component-id <vsp-component-id> --fleet-fqdn <fleet-lcm-fqdn> --vcf-services-runtime-fqdn <vcf-services-runtime-fqdn> --vcf-services-runtime-username <vcf-services-runtime-username>
For my environment it was
python3 cleanup_component.py delete vsp-component --component-id aceb5c33-d3ab-49cc-b4c4-9aecfbfe15d9 --fleet-fqdn skyveil.leaha.co.uk --vcf-services-runtime-fqdn aerosphere.leaha.co.uk --vcf-services-runtime-username admin
You’ll be asked to press Y to confirm
5 – LCM Postgres
5.1 – LCM Postgres Optimisation
There is both the Fleet and SDDC lifecycle that have a three node HA DB on them, now the memory seems fine on them, but the CPU is 500m for both, with three instances each totaling 3000m, meanwhile the usage is in the single or double digits, so aiming for 200m brings the total down to 1200m
We can set the CPU requests to 200m for both SDDC lifecycle
kubectl patch postgresql vcf-sddc-lcm-db -n vcf-sddc-lcm --type='json' -p='[
{"op": "replace", "path": "/spec/resources/requests/cpu", "value": "200m"}
]'
And fleet lifecycle
kubectl patch postgresql vcf-fleet-lcm-db -n vcf-fleet-lcm --type='json' -p='[
{"op": "replace", "path": "/spec/resources/requests/cpu", "value": "200m"}
]'
5.2 – Original YAML Backups
5.2.1 – vcf-fleet-lcm-db.yaml
apiVersion: acid.zalan.do/v1
kind: postgresql
metadata:
annotations:
vmsp.vmware.com/delete-protection: "true"
creationTimestamp: "2026-05-12T13:50:10Z"
generation: 2
labels:
ops.vmsp.vmware.com/type: pgdatabase
velero.io/exclude-from-backup: "true"
name: vcf-fleet-lcm-db
namespace: vcf-fleet-lcm
resourceVersion: "13213812"
uid: b1cae9df-2a4d-4ddd-9f42-e5846ca2565a
spec:
additionalVolumes:
- mountPath: /usr/local/share/ca-certificates/client/
name: seaweedfs-client-cert
targetContainers:
- postgres
- walg
volumeSource:
secret:
optional: false
secretName: seaweedfs-client-cert
- mountPath: /usr/local/share/ca-certificates/tls/
name: pgbackup-sidecar-tls-cert
targetContainers:
- walg
volumeSource:
secret:
optional: false
secretName: vcf-fleet-lcm-db-postgres-cert-secret
allowedSourceRanges: null
enableConnectionPooler: false
enableReplicaConnectionPooler: false
env:
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
key: access_key
name: vcf-fleet-lcm.db.vcf-fleet-lcm-db.wal-backup-credentials
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
key: secret_key
name: vcf-fleet-lcm.db.vcf-fleet-lcm-db.wal-backup-credentials
- name: AWS_ENDPOINT
value: https://seaweedfs-s3.vmsp-platform.svc.cluster.local:8443
- name: WALG_S3_CA_CERT_FILE
value: /usr/local/share/ca-certificates/client/ca.crt
- name: USE_WALG_BACKUP
value: "true"
- name: USE_WALG_RESTORE
value: "true"
- name: CLONE_USE_WALG_RESTORE
value: "true"
- name: CLONE_WAL_BUCKET_SCOPE_PREFIX
- name: BACKUP_NUM_TO_RETAIN
value: "1"
- name: WAL_S3_BUCKET
value: vcf-fleet-lcm.db.vcf-fleet-lcm-db.wal-backup
- name: AWS_S3_FORCE_PATH_STYLE
value: "true"
- name: BACKUP_SCHEDULE
value: '#0 25 31 2 * '
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- preference:
matchExpressions:
- key: node.kubernetes.io/instance-type
operator: In
values:
- vsphere-vm.cpu-12.mem-24gb.os-photon
weight: 1
numberOfInstances: 3
patroni:
failsafe_mode: true
retry_timeout: 20
synchronous_mode: true
synchronous_mode_strict: true
synchronous_node_count: 1
ttl: 50
postgresql:
parameters:
archive_cleanup_command: curl -sf --connect-timeout 1 --max-time 5 http://127.0.0.1:8008/replica
> /dev/null && envdir "/run/etc/wal-e.d/env" pg_archivecleanup "/home/postgres/pgdata/pgroot/data/pg_wal"
"%r" || exit 0
archive_command: (curl -sf --connect-timeout 1 --max-time 5 http://127.0.0.1:8008/primary
> /dev/null || exit 0) && envdir "/run/etc/wal-e.d/env" wal-g wal-push "%p"
archive_timeout: 900s
client_min_messages: error
log_destination: stderr
log_line_prefix: '%m %c %x %d %u %r %p %l'
log_parameter_max_length: "0"
log_parameter_max_length_on_error: "0"
logging_collector: "off"
max_connections: "500"
password_encryption: scram-sha-256
pgaudit.log: none
pgaudit.log_catalog: "off"
pgaudit.log_parameter: "off"
restore_command: 'envdir "/run/etc/wal-e.d/env" /bin/bash -c "echo $(date -u
+%%Y-%%m-%%d_%%H:%%M:%%S_UTC) rc: start %f >&2; if ! timeout 1s bash -c ''exec
3<>/dev/tcp/seaweedfs-s3.vmsp-platform.svc.cluster.local/8443''; then echo
$(date -u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC) rc: preflight-fail https://seaweedfs-s3.vmsp-platform.svc.cluster.local:8443
>&2; exit 1; fi; if timeout 30s /scripts/restore_command.sh \"%f\" \"%p\";
then if [ -s \"%p\" ]; then echo $(date -u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC) rc:
success %f >&2; exit 0; else echo $(date -u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC)
rc: empty-file %f >&2; exit 1; fi; else rc=$?; if [ \"${rc:-0}\" -eq 124 ];
then echo $(date -u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC) rc: fetch-timeout %f >&2;
else echo $(date -u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC) rc: fetch-fail %f rc=${rc:-?}
>&2; fi; exit 1; fi"'
shared_preload_libraries: bg_mon,pg_stat_statements,pgextwlist,pg_auth_mon,set_user,timescaledb,pg_cron,pg_stat_kcache
statement_timeout: "30000"
synchronous_commit: "on"
version: "16"
preparedDatabases:
vcffleetlcmdb:
defaultUsers: true
schemas:
public: {}
resources:
limits:
cpu: "1"
memory: 3048Mi
requests:
cpu: 500m
memory: 512Mi
sidecars:
- env:
- name: DATA_SOURCE_URI
value: localhost:5432/postgres?sslmode=disable
- name: DATA_SOURCE_USER
value: $(POSTGRES_USER)
- name: DATA_SOURCE_PASS
value: $(POSTGRES_PASSWORD)
image: registry.vmsp-platform.svc.cluster.local:5000/images/prometheuscommunity-postgres-exporter:v0.18.1
name: metrics-exporter
ports:
- containerPort: 9187
name: http-metrics
protocol: TCP
- env:
- name: PGPORT
value: "5432"
- name: PGHOST
value: vcf-fleet-lcm-db.vcf-fleet-lcm.svc.cluster.local
- name: WALE_S3_PREFIX
value: s3://vcf-fleet-lcm.db.vcf-fleet-lcm-db.wal-backup/spilo/vcf-fleet-lcm-vcf-fleet-lcm-db/wal/16/
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
key: access_key
name: vcf-fleet-lcm.db.vcf-fleet-lcm-db.wal-backup-credentials
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
key: secret_key
name: vcf-fleet-lcm.db.vcf-fleet-lcm-db.wal-backup-credentials
- name: AWS_ENDPOINT
value: https://seaweedfs-s3.vmsp-platform.svc.cluster.local:8443
- name: WALG_S3_CA_CERT_FILE
value: /usr/local/share/ca-certificates/client/ca.crt
- name: USE_WALG_BACKUP
value: "true"
- name: USE_WALG_RESTORE
value: "true"
- name: CLONE_USE_WALG_RESTORE
value: "true"
- name: CLONE_WAL_BUCKET_SCOPE_PREFIX
- name: BACKUP_NUM_TO_RETAIN
value: "1"
- name: WAL_S3_BUCKET
value: vcf-fleet-lcm.db.vcf-fleet-lcm-db.wal-backup
- name: AWS_S3_FORCE_PATH_STYLE
value: "true"
- name: BACKUP_SCHEDULE
value: '#0 25 31 2 * '
image: registry.vmsp-platform.svc.cluster.local:5000/images/pgbackup:v1.0.0
name: walg
ports:
- containerPort: 9100
name: walg
protocol: TCP
resources:
limits:
cpu: 1000m
memory: 768Mi
requests:
cpu: 100m
memory: 100Mi
teamId: vcf-fleet-lcm-db
tls:
caFile: ca.crt
secretName: vcf-fleet-lcm-db-postgres-cert-secret
users:
postgres:
- superuser
- createdb
volume:
size: 30Gi
status:
PostgresClusterStatus: Running
5.2.2 – vcf-sddc-lcm-db.yaml
apiVersion: acid.zalan.do/v1
kind: postgresql
metadata:
annotations:
vmsp.vmware.com/delete-protection: "true"
creationTimestamp: "2026-05-12T13:51:09Z"
generation: 2
labels:
ops.vmsp.vmware.com/type: pgdatabase
velero.io/exclude-from-backup: "true"
name: vcf-sddc-lcm-db
namespace: vcf-sddc-lcm
resourceVersion: "13213769"
uid: 107f3f96-c4eb-47b8-8f34-9ab0b4d18e8b
spec:
additionalVolumes:
- mountPath: /usr/local/share/ca-certificates/client/
name: seaweedfs-client-cert
targetContainers:
- postgres
- walg
volumeSource:
secret:
optional: false
secretName: seaweedfs-client-cert
- mountPath: /usr/local/share/ca-certificates/tls/
name: pgbackup-sidecar-tls-cert
targetContainers:
- walg
volumeSource:
secret:
optional: false
secretName: vcf-sddc-lcm-db-postgres-cert-secret
allowedSourceRanges: null
enableConnectionPooler: false
enableReplicaConnectionPooler: false
env:
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
key: access_key
name: vcf-sddc-lcm.db.vcf-sddc-lcm-db.wal-backup-credentials
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
key: secret_key
name: vcf-sddc-lcm.db.vcf-sddc-lcm-db.wal-backup-credentials
- name: AWS_ENDPOINT
value: https://seaweedfs-s3.vmsp-platform.svc.cluster.local:8443
- name: WALG_S3_CA_CERT_FILE
value: /usr/local/share/ca-certificates/client/ca.crt
- name: USE_WALG_BACKUP
value: "true"
- name: USE_WALG_RESTORE
value: "true"
- name: CLONE_USE_WALG_RESTORE
value: "true"
- name: CLONE_WAL_BUCKET_SCOPE_PREFIX
- name: BACKUP_NUM_TO_RETAIN
value: "1"
- name: WAL_S3_BUCKET
value: vcf-sddc-lcm.db.vcf-sddc-lcm-db.wal-backup
- name: AWS_S3_FORCE_PATH_STYLE
value: "true"
- name: BACKUP_SCHEDULE
value: '#0 25 31 2 * '
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- preference:
matchExpressions:
- key: node.kubernetes.io/instance-type
operator: In
values:
- vsphere-vm.cpu-12.mem-24gb.os-photon
weight: 1
numberOfInstances: 3
patroni:
failsafe_mode: true
retry_timeout: 20
synchronous_mode: true
synchronous_mode_strict: true
synchronous_node_count: 1
ttl: 50
postgresql:
parameters:
archive_cleanup_command: curl -sf --connect-timeout 1 --max-time 5 http://127.0.0.1:8008/replica
> /dev/null && envdir "/run/etc/wal-e.d/env" pg_archivecleanup "/home/postgres/pgdata/pgroot/data/pg_wal"
"%r" || exit 0
archive_command: (curl -sf --connect-timeout 1 --max-time 5 http://127.0.0.1:8008/primary
> /dev/null || exit 0) && envdir "/run/etc/wal-e.d/env" wal-g wal-push "%p"
archive_timeout: 900s
client_min_messages: error
log_destination: stderr
log_line_prefix: '%m %c %x %d %u %r %p %l'
log_parameter_max_length: "0"
log_parameter_max_length_on_error: "0"
logging_collector: "off"
max_connections: "500"
password_encryption: scram-sha-256
pgaudit.log: none
pgaudit.log_catalog: "off"
pgaudit.log_parameter: "off"
restore_command: 'envdir "/run/etc/wal-e.d/env" /bin/bash -c "echo $(date -u
+%%Y-%%m-%%d_%%H:%%M:%%S_UTC) rc: start %f >&2; if ! timeout 1s bash -c ''exec
3<>/dev/tcp/seaweedfs-s3.vmsp-platform.svc.cluster.local/8443''; then echo
$(date -u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC) rc: preflight-fail https://seaweedfs-s3.vmsp-platform.svc.cluster.local:8443
>&2; exit 1; fi; if timeout 30s /scripts/restore_command.sh \"%f\" \"%p\";
then if [ -s \"%p\" ]; then echo $(date -u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC) rc:
success %f >&2; exit 0; else echo $(date -u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC)
rc: empty-file %f >&2; exit 1; fi; else rc=$?; if [ \"${rc:-0}\" -eq 124 ];
then echo $(date -u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC) rc: fetch-timeout %f >&2;
else echo $(date -u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC) rc: fetch-fail %f rc=${rc:-?}
>&2; fi; exit 1; fi"'
shared_preload_libraries: bg_mon,pg_stat_statements,pgextwlist,pg_auth_mon,set_user,timescaledb,pg_cron,pg_stat_kcache
statement_timeout: "30000"
synchronous_commit: "on"
version: "16"
preparedDatabases:
vcfsddclcmdb:
defaultUsers: true
schemas:
public: {}
resources:
limits:
cpu: "1"
memory: 3048Mi
requests:
cpu: 500m
memory: 512Mi
sidecars:
- env:
- name: DATA_SOURCE_URI
value: localhost:5432/postgres?sslmode=disable
- name: DATA_SOURCE_USER
value: $(POSTGRES_USER)
- name: DATA_SOURCE_PASS
value: $(POSTGRES_PASSWORD)
image: registry.vmsp-platform.svc.cluster.local:5000/images/prometheuscommunity-postgres-exporter:v0.18.1
name: metrics-exporter
ports:
- containerPort: 9187
name: http-metrics
protocol: TCP
- env:
- name: PGPORT
value: "5432"
- name: PGHOST
value: vcf-sddc-lcm-db.vcf-sddc-lcm.svc.cluster.local
- name: WALE_S3_PREFIX
value: s3://vcf-sddc-lcm.db.vcf-sddc-lcm-db.wal-backup/spilo/vcf-sddc-lcm-vcf-sddc-lcm-db/wal/16/
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
key: access_key
name: vcf-sddc-lcm.db.vcf-sddc-lcm-db.wal-backup-credentials
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
key: secret_key
name: vcf-sddc-lcm.db.vcf-sddc-lcm-db.wal-backup-credentials
- name: AWS_ENDPOINT
value: https://seaweedfs-s3.vmsp-platform.svc.cluster.local:8443
- name: WALG_S3_CA_CERT_FILE
value: /usr/local/share/ca-certificates/client/ca.crt
- name: USE_WALG_BACKUP
value: "true"
- name: USE_WALG_RESTORE
value: "true"
- name: CLONE_USE_WALG_RESTORE
value: "true"
- name: CLONE_WAL_BUCKET_SCOPE_PREFIX
- name: BACKUP_NUM_TO_RETAIN
value: "1"
- name: WAL_S3_BUCKET
value: vcf-sddc-lcm.db.vcf-sddc-lcm-db.wal-backup
- name: AWS_S3_FORCE_PATH_STYLE
value: "true"
- name: BACKUP_SCHEDULE
value: '#0 25 31 2 * '
image: registry.vmsp-platform.svc.cluster.local:5000/images/pgbackup:v1.0.0
name: walg
ports:
- containerPort: 9100
name: walg
protocol: TCP
resources:
limits:
cpu: 1000m
memory: 768Mi
requests:
cpu: 100m
memory: 100Mi
teamId: vcf-sddc-lcm-db
tls:
caFile: ca.crt
secretName: vcf-sddc-lcm-db-postgres-cert-secret
users:
postgres:
- superuser
- createdb
volume:
size: 30Gi
status:
PostgresClusterStatus: Running
6 – Salt
6.1 – Salt RaaS
6.1.1 – Salt RaaS Optimisations
The RaaS instance is quite over provisioned
An example of what mine was running at
NAME CPU(cores) MEMORY(bytes)
pgdatabase-0 43m 790Mi
raas-5857f54b4f-l4zb4 20m 716Mi
redis-6cccfb46f4-gp7p6 6m 7Mi
The current requests are
RaaS – 1000m/1536MiB
Redis – 500m/500MiB
Postgres – 1200m/2248MiB
So these are very over provisioned, we can set them to the following, Request and Limits as
Raas – 250m/1024MiB
Redis – 100m/128MiB
Postgres – 250m/768MiB – 500m/1536MiB
Raas
kubectl patch deployment raas -n salt-raas --type='json' -p='[
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/cpu", "value": "250m"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/memory", "value": "1Gi"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/limits/memory", "value": "1Gi"}
]'
Redis
kubectl patch deployment redis -n salt-raas --type='json' -p='[
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/cpu", "value": "100m"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/memory", "value": "128Mi"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/limits/memory", "value": "128Mi"}
]'
Postgres
kubectl patch postgresinstance pgdatabase -n salt-raas --type='json' -p='[
{"op": "replace", "path": "/spec/resources/requests/cpu", "value": "250m"},
{"op": "replace", "path": "/spec/resources/requests/memory", "value": "768Mi"},
{"op": "replace", "path": "/spec/resources/limits/cpu", "value": "500m"},
{"op": "replace", "path": "/spec/resources/limits/memory", "value": "1536Mi"}
]'
We then need to delete the pod to force the changes to be picked up with
kubectl delete pod pgdatabase-0 -n salt-raas
6.1.2 – Original YAML Backups
6.1.2.1 – raas.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
meta.helm.sh/release-name: raas
meta.helm.sh/release-namespace: salt-raas
reloader.vmsp.vmware.com/auto: "true"
creationTimestamp: "2026-05-12T15:01:21Z"
generation: 1
labels:
app: raas
app.kubernetes.io/managed-by: Helm
helm.toolkit.fluxcd.io/name: raas
helm.toolkit.fluxcd.io/namespace: salt-raas
ops.vmsp.vmware.com/type: salt-service-fleet
name: raas
namespace: salt-raas
resourceVersion: "13208867"
uid: efbe38c7-0ef5-47bd-943c-746cf3c6fff2
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: raas
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
labels:
app: raas
app.kubernetes.io/name: raas
ops.vmsp.vmware.com/type: salt-resource-fleet
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector: {}
namespaces:
- salt-raas
- salt
topologyKey: kubernetes.io/hostname
weight: 100
containers:
- env:
- name: RAAS_LICENSES
valueFrom:
secretKeyRef:
key: license
name: raas-secret
- name: DATABASE_URL
valueFrom:
secretKeyRef:
key: uri
name: raasdb-owner-user
- name: REDIS_URL
value: rediss://redis-service:6379
- name: SSE_LOG_LEVEL
value: error
- name: ENABLE_METRICS_SERVER
value: "false"
- name: RAAS_UI_INDEX
value: raas-index.html
- name: RAAS_UI_ENABLED
value: "false"
- name: RAAS_PATH_PREFIX
value: /raas
- name: RAAS_HEALTH_PORT
value: "8444"
- name: DISABLE_SECOPS_TASKS
value: "1"
- name: DISABLE_VMAN_TASKS
value: "1"
- name: DISABLE_LDAP_TASKS
value: "1"
- name: DISABLE_CELERY_TASK_ROUTES
value: "1"
image: registry.vmsp-platform.svc.cluster.local:5000/images/raas:9.1.0-25346036
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 5
httpGet:
path: /livez
port: 8444
scheme: HTTPS
initialDelaySeconds: 20
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 55
name: raas
ports:
- containerPort: 8443
protocol: TCP
readinessProbe:
failureThreshold: 4
httpGet:
path: /readyz
port: 8444
scheme: HTTPS
initialDelaySeconds: 10
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 30
resources:
limits:
cpu: "1"
memory: 1536Mi
requests:
cpu: "1"
memory: 1536Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: false
runAsGroup: 999
runAsNonRoot: true
runAsUser: 999
seccompProfile:
type: RuntimeDefault
startupProbe:
failureThreshold: 30
httpGet:
path: /readyz
port: 8444
scheme: HTTPS
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 40
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /config
name: raas-config-vol
- mountPath: /etc/raas/pki
name: raas-keys-vol
- mountPath: /tls
name: raas-tls
- mountPath: /postgresql-tls
name: raas-postgresql-tls
- mountPath: /redis-tls
name: raas-redis-tls
dnsPolicy: ClusterFirst
hostAliases:
- hostnames:
- skyveil.leaha.co.uk
ip: 192.168.1.91
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 999
runAsGroup: 999
runAsNonRoot: true
runAsUser: 999
serviceAccount: raas-account
serviceAccountName: raas-account
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 420
items:
- key: raas_config
path: raas
name: raas-configmap
name: raas-config-vol
- name: raas-keys-vol
secret:
defaultMode: 420
items:
- key: db_key
path: .raas.key
- key: instance_id
path: .instance_id
secretName: raas-secret
- name: raas-tls
secret:
defaultMode: 420
items:
- key: tls.crt
path: tls.crt
- key: tls.key
path: tls.key
- key: ca.crt
path: ca.crt
secretName: raas-instance-cert-secret
- name: raas-postgresql-tls
secret:
defaultMode: 420
items:
- key: tls.crt
path: tls.crt
- key: tls.key
path: tls.key
- key: ca.crt
path: ca.crt
secretName: pgdatabase-postgres-cert-secret
- name: raas-redis-tls
secret:
defaultMode: 420
items:
- key: tls.crt
path: tls.crt
- key: tls.key
path: tls.key
- key: ca.crt
path: ca.crt
secretName: redis-cert-secret
status:
availableReplicas: 1
conditions:
- lastTransitionTime: "2026-05-12T15:01:21Z"
lastUpdateTime: "2026-05-12T15:03:03Z"
message: ReplicaSet "raas-5857f54b4f" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
- lastTransitionTime: "2026-05-27T06:12:01Z"
lastUpdateTime: "2026-05-27T06:12:01Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
observedGeneration: 1
readyReplicas: 1
replicas: 1
updatedReplicas: 1
6.1.2.2 – redis.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: “1”
meta.helm.sh/release-name: raas
meta.helm.sh/release-namespace: salt-raas
creationTimestamp: “2026-05-12T15:01:21Z”
generation: 1
labels:
app: redis
app.kubernetes.io/managed-by: Helm
helm.toolkit.fluxcd.io/name: raas
helm.toolkit.fluxcd.io/namespace: salt-raas
ops.vmsp.vmware.com/type: salt-service-fleet
name: redis
namespace: salt-raas
resourceVersion: “13190086”
uid: a9757a2a-6741-40c0-a6cd-08eb49d3eb9d
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: redis
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
labels:
app: redis
app.kubernetes.io/name: redis
ops.vmsp.vmware.com/type: salt-resource-fleet
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
– podAffinityTerm:
labelSelector: {}
namespaces:
– salt-raas
– salt
topologyKey: kubernetes.io/hostname
weight: 100
containers:
– args:
– –port 0
– –tls-port 6379
– –tls-cert-file /redis-tls/tls.crt
– –tls-key-file /redis-tls/tls.key
– –tls-ca-cert-file /redis-tls/ca.crt
– –tls-auth-clients yes
– –protected-mode no
command:
– redis-server
image: registry.vmsp-platform.svc.cluster.local:5000/images/redis:9.1.0-25346036
imagePullPolicy: IfNotPresent
name: redis
ports:
– containerPort: 6379
protocol: TCP
resources:
limits:
cpu: 500m
memory: 500Mi
requests:
cpu: 500m
memory: 500Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
– mountPath: /redis-tls
name: raas-redis-tls
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
volumes:
– name: raas-redis-tls
secret:
defaultMode: 420
items:
– key: tls.crt
path: tls.crt
– key: tls.key
path: tls.key
– key: ca.crt
path: ca.crt
secretName: redis-cert-secret
status:
availableReplicas: 1
conditions:
- lastTransitionTime: “2026-05-12T15:01:21Z”
lastUpdateTime: “2026-05-12T15:01:28Z”
message: ReplicaSet “redis-6cccfb46f4” has successfully progressed.
reason: NewReplicaSetAvailable
status: “True”
type: Progressing - lastTransitionTime: “2026-05-26T07:06:38Z”
lastUpdateTime: “2026-05-26T07:06:38Z”
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: “True”
type: Available
observedGeneration: 1
readyReplicas: 1
replicas: 1
updatedReplicas: 1
6.2.2.3 – raas-postgres.yaml
apiVersion: v1
items:
- apiVersion: acid.zalan.do/v1
kind: postgresql
metadata:
annotations:
vmsp.vmware.com/delete-protection: "true"
creationTimestamp: "2026-05-12T15:01:28Z"
generation: 2
labels:
ops.vmsp.vmware.com/type: pgdatabase
velero.io/exclude-from-backup: "true"
name: pgdatabase
namespace: salt-raas
resourceVersion: "13213752"
uid: d71adbfd-9c4f-4ece-9e9f-d98ea6e92fca
spec:
additionalVolumes:
- mountPath: /usr/local/share/ca-certificates/client/
name: seaweedfs-client-cert
targetContainers:
- postgres
- walg
volumeSource:
secret:
optional: false
secretName: seaweedfs-client-cert
- mountPath: /usr/local/share/ca-certificates/tls/
name: pgbackup-sidecar-tls-cert
targetContainers:
- walg
volumeSource:
secret:
optional: false
secretName: pgdatabase-postgres-cert-secret
allowedSourceRanges: null
enableConnectionPooler: false
enableReplicaConnectionPooler: false
env:
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
key: access_key
name: salt-raas.db.pgdatabase.wal-backup-credentials
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
key: secret_key
name: salt-raas.db.pgdatabase.wal-backup-credentials
- name: AWS_ENDPOINT
value: https://seaweedfs-s3.vmsp-platform.svc.cluster.local:8443
- name: WALG_S3_CA_CERT_FILE
value: /usr/local/share/ca-certificates/client/ca.crt
- name: USE_WALG_BACKUP
value: "true"
- name: USE_WALG_RESTORE
value: "true"
- name: CLONE_USE_WALG_RESTORE
value: "true"
- name: CLONE_WAL_BUCKET_SCOPE_PREFIX
- name: BACKUP_NUM_TO_RETAIN
value: "1"
- name: WAL_S3_BUCKET
value: salt-raas.db.pgdatabase.wal-backup
- name: AWS_S3_FORCE_PATH_STYLE
value: "true"
- name: BACKUP_SCHEDULE
value: '#0 25 31 2 * '
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- preference:
matchExpressions:
- key: node.kubernetes.io/instance-type
operator: In
values:
- vsphere-vm.cpu-12.mem-24gb.os-photon
weight: 1
numberOfInstances: 1
patroni:
failsafe_mode: true
podAnnotations:
database.vmsp.vmware.com/affinity: required
postgresql:
parameters:
archive_cleanup_command: curl -sf --connect-timeout 1 --max-time 5 http://127.0.0.1:8008/replica
> /dev/null && envdir "/run/etc/wal-e.d/env" pg_archivecleanup "/home/postgres/pgdata/pgroot/data/pg_wal"
"%r" || exit 0
archive_command: (curl -sf --connect-timeout 1 --max-time 5 http://127.0.0.1:8008/primary
> /dev/null || exit 0) && envdir "/run/etc/wal-e.d/env" wal-g wal-push "%p"
archive_timeout: 900s
client_min_messages: error
log_destination: stderr
log_line_prefix: '%m %c %x %d %u %r %p %l'
logging_collector: "off"
max_connections: "500"
password_encryption: scram-sha-256
pgaudit.log: none
pgaudit.log_catalog: "off"
pgaudit.log_parameter: "off"
restore_command: 'envdir "/run/etc/wal-e.d/env" /bin/bash -c "echo $(date
-u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC) rc: start %f >&2; if ! timeout 1s bash
-c ''exec 3<>/dev/tcp/seaweedfs-s3.vmsp-platform.svc.cluster.local/8443'';
then echo $(date -u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC) rc: preflight-fail https://seaweedfs-s3.vmsp-platform.svc.cluster.local:8443
>&2; exit 1; fi; if timeout 30s /scripts/restore_command.sh \"%f\" \"%p\";
then if [ -s \"%p\" ]; then echo $(date -u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC)
rc: success %f >&2; exit 0; else echo $(date -u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC)
rc: empty-file %f >&2; exit 1; fi; else rc=$?; if [ \"${rc:-0}\" -eq 124
]; then echo $(date -u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC) rc: fetch-timeout %f
>&2; else echo $(date -u +%%Y-%%m-%%d_%%H:%%M:%%S_UTC) rc: fetch-fail %f
rc=${rc:-?} >&2; fi; exit 1; fi"'
shared_preload_libraries: bg_mon,pg_stat_statements,pgextwlist,pg_auth_mon,set_user,timescaledb,pg_cron,pg_stat_kcache
statement_timeout: "30000"
synchronous_commit: "on"
version: "15"
preparedDatabases:
raasdb:
defaultUsers: true
schemas:
public: {}
resources:
limits:
cpu: "1"
memory: 2Gi
requests:
cpu: "1"
memory: 2Gi
sidecars:
- env:
- name: DATA_SOURCE_URI
value: localhost:5432/postgres?sslmode=disable
- name: DATA_SOURCE_USER
value: $(POSTGRES_USER)
- name: DATA_SOURCE_PASS
value: $(POSTGRES_PASSWORD)
image: registry.vmsp-platform.svc.cluster.local:5000/images/prometheuscommunity-postgres-exporter:v0.18.1
name: metrics-exporter
ports:
- containerPort: 9187
name: http-metrics
protocol: TCP
- env:
- name: PGPORT
value: "5432"
- name: PGHOST
value: pgdatabase.salt-raas.svc.cluster.local
- name: WALE_S3_PREFIX
value: s3://salt-raas.db.pgdatabase.wal-backup/spilo/salt-raas-pgdatabase/wal/15/
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
key: access_key
name: salt-raas.db.pgdatabase.wal-backup-credentials
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
key: secret_key
name: salt-raas.db.pgdatabase.wal-backup-credentials
- name: AWS_ENDPOINT
value: https://seaweedfs-s3.vmsp-platform.svc.cluster.local:8443
- name: WALG_S3_CA_CERT_FILE
value: /usr/local/share/ca-certificates/client/ca.crt
- name: USE_WALG_BACKUP
value: "true"
- name: USE_WALG_RESTORE
value: "true"
- name: CLONE_USE_WALG_RESTORE
value: "true"
- name: CLONE_WAL_BUCKET_SCOPE_PREFIX
- name: BACKUP_NUM_TO_RETAIN
value: "1"
- name: WAL_S3_BUCKET
value: salt-raas.db.pgdatabase.wal-backup
- name: AWS_S3_FORCE_PATH_STYLE
value: "true"
- name: BACKUP_SCHEDULE
value: '#0 25 31 2 * '
image: registry.vmsp-platform.svc.cluster.local:5000/images/pgbackup:v1.0.0
name: walg
ports:
- containerPort: 9100
name: walg
protocol: TCP
resources:
limits:
cpu: 1000m
memory: 768Mi
requests:
cpu: 100m
memory: 100Mi
teamId: pgdatabase
tls:
caFile: ca.crt
secretName: pgdatabase-postgres-cert-secret
users:
postgres:
- superuser
- createdb
volume:
size: 100Gi
status:
PostgresClusterStatus: Running
kind: List
metadata:
resourceVersion: ""
6.2 – Salt Master
6.2.1 – Salt Master Optimisations
The Master instance is quite over provisioned
An example of what mine was running at
NAME CPU(cores) MEMORY(bytes)
salt-master-8674dd877b-8j6q2 16m 912Mi
salt-minion-6c58844558-zkxqf 3m 91Mi
The current requests are
Master – 500m/1024MiB
Minion – 500m/500MiB
So these are very over provisioned, we can set them to the following, Request and Limits as
Master – 250m/1024MiB – 500m
Minion – 100m/256MiB
Master
kubectl patch deployment salt-master -n salt --type='json' -p='[
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/cpu", "value": "250m"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/limits/cpu", "value": "500m"}
]'
Minion
kubectl patch deployment salt-minion -n salt --type='json' -p='[
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/cpu", "value": "100m"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/memory", "value": "256Mi"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/limits/memory", "value": "256Mi"}
]'
6.2.2 – Original YAML Backups
6.2.2.1 – master.yaml
cat salt-master-original.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "2"
meta.helm.sh/release-name: salt-master
meta.helm.sh/release-namespace: salt
reloader.vmsp.vmware.com/auto: "true"
creationTimestamp: "2026-05-12T15:16:45Z"
generation: 3
labels:
app: salt-master
app.kubernetes.io/managed-by: Helm
helm.toolkit.fluxcd.io/name: salt-master
helm.toolkit.fluxcd.io/namespace: salt
ops.vmsp.vmware.com/type: salt-service
name: salt-master
namespace: salt
resourceVersion: "13208502"
uid: f35451ab-5d66-4313-8099-2fd5ccbab367
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: salt-master
strategy:
type: Recreate
template:
metadata:
annotations:
vmsp.vmware.com/triggerrestart: "138727"
labels:
app: salt-master
app.kubernetes.io/name: salt-master
ops.vmsp.vmware.com/type: salt-resource
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector: {}
namespaces:
- salt
- salt-raas
topologyKey: kubernetes.io/hostname
weight: 100
containers:
- env:
- name: OPS_IDENTIFIER
value: f4f174ed-d5d9-4990-8cb8-aaf9e5fb913e
- name: MASTER_FQDN
value: windspire.leaha.co.uk
- name: SSL_CERT_FILE
value: /etc/pki/tls/certs/ca-bundle.crt
image: registry.vmsp-platform.svc.cluster.local:5000/images/salt-master:9.1.0-25346036
imagePullPolicy: IfNotPresent
name: salt-master
ports:
- containerPort: 4505
protocol: TCP
- containerPort: 4506
protocol: TCP
resources:
limits:
cpu: 500m
memory: 1Gi
requests:
cpu: 500m
memory: 1Gi
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: false
runAsGroup: 999
runAsNonRoot: true
runAsUser: 999
seccompProfile:
type: RuntimeDefault
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/salt/master.d
name: salt-master-config-vol
- mountPath: /var/cache/salt
name: salt-master-volume
subPath: salt
- mountPath: /etc/salt/pki/master/minions
name: salt-master-volume
subPath: minions
- mountPath: /etc/pki/tls/certs/
name: ca-bundle
- mountPath: /etc/salt/pki/master/master.pem
name: salt-master-keys-vol
subPath: master.pem
- mountPath: /etc/salt/pki/master/master.pub
name: salt-master-keys-vol
subPath: master.pub
- mountPath: /etc/salt/pki/master/sseapi_key.pem
name: salt-master-keys-vol
subPath: sseapi_key.pem
- mountPath: /etc/salt/pki/master/sseapi_key.pub
name: salt-master-keys-vol
subPath: sseapi_key.pub
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 999
runAsGroup: 999
runAsNonRoot: true
runAsUser: 999
serviceAccount: salt-master-account
serviceAccountName: salt-master-account
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 420
items:
- key: master_conf
path: master.conf
- key: raas_conf
path: raas.conf
- key: raas_server_conf
path: server.conf
name: salt-master-configmap
name: salt-master-config-vol
- configMap:
defaultMode: 420
items:
- key: bundle.pem
path: ca-bundle.crt
name: platform-trust
name: ca-bundle
- name: salt-master-volume
persistentVolumeClaim:
claimName: salt-master-volume
- name: salt-master-keys-vol
secret:
defaultMode: 420
items:
- key: master_private_key
path: master.pem
- key: master_public_key
path: master.pub
- key: raas_private_key
path: sseapi_key.pem
- key: raas_public_key
path: sseapi_key.pub
secretName: salt-master-secret
status:
availableReplicas: 1
conditions:
- lastTransitionTime: "2026-05-12T15:16:45Z"
lastUpdateTime: "2026-05-12T15:33:26Z"
message: ReplicaSet "salt-master-8674dd877b" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
- lastTransitionTime: "2026-05-27T06:11:57Z"
lastUpdateTime: "2026-05-27T06:11:57Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
observedGeneration: 3
readyReplicas: 1
replicas: 1
updatedReplicas: 1
6.2.2.2 – minion.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "3"
identity.vmsp.vmware.com/serviceaccount: salt-minion
meta.helm.sh/release-name: salt-minion
meta.helm.sh/release-namespace: salt
reloader.vmsp.vmware.com/auto: "true"
creationTimestamp: "2026-05-12T15:16:44Z"
generation: 3
labels:
app: salt-minion
app.kubernetes.io/managed-by: Helm
helm.toolkit.fluxcd.io/name: salt-minion
helm.toolkit.fluxcd.io/namespace: salt
ops.vmsp.vmware.com/type: salt-service
name: salt-minion
namespace: salt
resourceVersion: "13190067"
uid: 48d403a6-49ad-4480-9fa0-bd4d6411fa75
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: salt-minion
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
annotations:
identity.vmsp.vmware.com/serviceaccount: salt-minion
vmsp.vmware.com/triggerrestart: "138702"
labels:
app: salt-minion
app.kubernetes.io/name: salt-minion
ops.vmsp.vmware.com/type: salt-resource
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector: {}
namespaces:
- salt
- salt-raas
topologyKey: kubernetes.io/hostname
weight: 100
containers:
- env:
- name: IMDS_FQDN
value: 169.254.100.200
- name: VIP_FQDN
value: aerosphere.leaha.co.uk
image: registry.vmsp-platform.svc.cluster.local:5000/images/salt-minion:9.1.0-25346036
imagePullPolicy: IfNotPresent
name: salt-minion
resources:
limits:
cpu: 500m
memory: 500Mi
requests:
cpu: 500m
memory: 500Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: false
runAsGroup: 999
runAsNonRoot: true
runAsUser: 999
seccompProfile:
type: RuntimeDefault
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/salt/minion.d
name: salt-minion-config-vol
- mountPath: /etc/salt/pki/minion/minion.pem
name: salt-minion-keys-vol
subPath: minion.pem
- mountPath: /etc/salt/pki/minion/minion.pub
name: salt-minion-keys-vol
subPath: minion.pub
- mountPath: /etc/salt/pki/minion/minion_master.pub
name: salt-minion-keys-vol
subPath: minion_master.pub
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 999
runAsGroup: 999
runAsNonRoot: true
runAsUser: 999
serviceAccount: salt-minion-account
serviceAccountName: salt-minion-account
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 420
items:
- key: minion_conf
path: minion.conf
name: salt-minion-configmap
name: salt-minion-config-vol
- name: salt-minion-keys-vol
secret:
defaultMode: 420
items:
- key: minion_private_key
path: minion.pem
- key: minion_public_key
path: minion.pub
- key: master_public_key
path: minion_master.pub
secretName: salt-minion-secret
status:
availableReplicas: 1
conditions:
- lastTransitionTime: "2026-05-12T15:16:44Z"
lastUpdateTime: "2026-05-12T16:14:34Z"
message: ReplicaSet "salt-minion-6c58844558" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
- lastTransitionTime: "2026-05-26T07:06:38Z"
lastUpdateTime: "2026-05-26T07:06:38Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
observedGeneration: 3
readyReplicas: 1
replicas: 1
updatedReplicas: 1
7 – Zot-1
7.1 – Zot-1 Optimisations
The Zot container registry is a little over provisioned, as its mostly caching container images, it should be fine to tweak a little without breaking functionaility
The YAML doesnt have a limit, so that covers any burt requirements, but the memory is sat very low vs the default 350m/600MiB with a 700MiB limit
We can run the following to give it 180m default CPU and 400MiB memory with up to 512MiB total
kubectl patch statefulset zot-1 -n vmsp-platform --type='json' -p='[
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/cpu", "value": "180m"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/memory", "value": "400Mi"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/limits/memory", "value": "512Mi"}
]'
7.2 – Original YAML Backups
7.2.1 – zot-1.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
annotations:
meta.helm.sh/release-name: zot-1
meta.helm.sh/release-namespace: vmsp-platform
reloader.vmsp.vmware.com/auto: "true"
creationTimestamp: "2026-05-12T12:46:29Z"
generation: 2
labels:
app.kubernetes.io/instance: zot-1
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: zot
app.kubernetes.io/version: v2.1.11
helm.sh/chart: zot-0.1.68-34
helm.toolkit.fluxcd.io/name: zot-1
helm.toolkit.fluxcd.io/namespace: vmsp-platform
ops.vmsp.vmware.com/type: registry-service
name: zot-1
namespace: vmsp-platform
resourceVersion: "13209559"
uid: afa14b9d-52f7-4098-bbf0-f458d3cc51df
spec:
minReadySeconds: 10
persistentVolumeClaimRetentionPolicy:
whenDeleted: Retain
whenScaled: Retain
podManagementPolicy: OrderedReady
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/instance: zot-1
app.kubernetes.io/name: zot
serviceName: ""
template:
metadata:
annotations:
checksum/config: 0a98204231fad36afb0c28360a385ad1090e9094feff5d18e517fe5411a4042f
vmsp.vmware.com/triggerrestart: "13967"
labels:
app.kubernetes.io/component: zot
app.kubernetes.io/instance: zot-1
app.kubernetes.io/name: zot
ops.vmsp.vmware.com/type: registry-service
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchExpressions:
- key: app.kubernetes.io/name
operator: In
values:
- zot
topologyKey: kubernetes.io/hostname
weight: 100
containers:
- image: registry.vmsp-platform.svc.cluster.local:5000/images/zot:v2.1.11
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /livez
port: 5000
scheme: HTTPS
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: zot
ports:
- containerPort: 5000
name: zot
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /readyz
port: 5000
scheme: HTTPS
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
limits:
ephemeral-storage: 1Gi
memory: 700Mi
requests:
cpu: 350m
ephemeral-storage: 512Mi
memory: 600Mi
securityContext:
runAsNonRoot: true
startupProbe:
failureThreshold: 3
httpGet:
path: /startupz
port: 5000
scheme: HTTPS
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/lib/registry
name: zot-1-pvc
- mountPath: /etc/zot
name: zot-1-config
- mountPath: /secret
name: zot-htpasswd-secret
- mountPath: /certs
name: certificate
readOnly: true
- mountPath: /creds
name: sync-creds
dnsPolicy: ClusterFirst
priorityClassName: system-cluster-critical
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 65534
runAsGroup: 65534
runAsNonRoot: true
runAsUser: 65534
serviceAccount: zot-1
serviceAccountName: zot-1
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 420
name: zot-1-config
name: zot-1-config
- name: zot-htpasswd-secret
secret:
defaultMode: 420
secretName: zot-htpasswd-secret
- name: certificate
secret:
defaultMode: 420
secretName: registry-certificate
- configMap:
defaultMode: 420
name: zot-sync-credentials
name: sync-creds
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
volumeClaimTemplates:
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: zot-1-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 75Gi
storageClassName: vmsp-default
volumeMode: Filesystem
status:
phase: Pending
status:
availableReplicas: 1
collisionCount: 0
currentReplicas: 1
currentRevision: zot-1-64cf7c8845
observedGeneration: 2
readyReplicas: 1
replicas: 1
updateRevision: zot-1-64cf7c8845
updatedReplicas: 1
8 – Depot
8.1 – Depot Optimisations
The depot and distribution service has a pretty high request on CPU but has no limits, so we can lower this a fair bit without compromising performance when its needed
Memory on the depot we wont touch, while it could be lowered, we can see the pod usage, but its has three containers so I dont want to over reduce one of them
For the depot we can run
kubectl patch deployment depot-service -n vcf-fleet-depot --type='json' -p='[
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/cpu", "value": "100m"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/memory", "value": "384Mi"},
{"op": "replace", "path": "/spec/template/spec/containers/1/resources/requests/cpu", "value": "100m"},
{"op": "replace", "path": "/spec/template/spec/containers/1/resources/requests/memory", "value": "384Mi"},
{"op": "replace", "path": "/spec/template/spec/containers/2/resources/requests/cpu", "value": "100m"},
{"op": "replace", "path": "/spec/template/spec/containers/2/resources/requests/memory", "value": "192Mi"}
]'
And for the distribution service we can run
# distribution-service
kubectl patch deployment distribution-service -n vcf-fleet-depot --type='json' -p='[
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/cpu", "value": "250m"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/memory", "value": "350Mi"}
]'
8.2 – Original YAML Backups
8.2.1 – depot.service.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "2"
meta.helm.sh/release-name: depot-service
meta.helm.sh/release-namespace: vcf-fleet-depot
reloader.vmsp.vmware.com/auto: "true"
reloader.vmsp.vmware.com/exclusion.secret: depot-service-secret
creationTimestamp: "2026-05-12T13:39:13Z"
generation: 2
labels:
app: depot-service
app.kubernetes.io/instance: depot-service
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: depot-service
app.kubernetes.io/version: 9.1.0
component.vmsp.vmware.com/type: vcf-fleet-depot
helm.sh/chart: depot-service-9.1.0-0_25371105
helm.toolkit.fluxcd.io/name: depot-service
helm.toolkit.fluxcd.io/namespace: vcf-fleet-depot
ops.vmsp.vmware.com/type: vcf-depot-service
name: depot-service
namespace: vcf-fleet-depot
resourceVersion: "13212166"
uid: 8b151d0c-0877-4355-99da-e73673a5cfb6
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 5
selector:
matchLabels:
app: depot-service
app.kubernetes.io/instance: depot-service
app.kubernetes.io/name: depot-service
component.vmsp.vmware.com/type: vcf-fleet-depot
strategy:
type: Recreate
template:
metadata:
annotations:
vmsp.vmware.com/triggerrestart: "50227"
labels:
app: depot-service
app.kubernetes.io/instance: depot-service
app.kubernetes.io/name: depot-service
component.vmsp.vmware.com/type: vcf-fleet-depot
ops.vmsp.vmware.com/type: vcf-depot-service-instance
spec:
containers:
- env:
- name: DEPOT_TYPE
valueFrom:
secretKeyRef:
key: depotType
name: depot-service-secret
- name: OFFLINE_DEPOT_ORIGIN
valueFrom:
secretKeyRef:
key: offlineOrigin
name: depot-service-secret
- name: OFFLINE_DEPOT_BASE_PATH
valueFrom:
secretKeyRef:
key: offlineBasePath
name: depot-service-secret
- name: OFFLINE_DEPOT_USERNAME
valueFrom:
secretKeyRef:
key: offlineUsername
name: depot-service-secret
- name: OFFLINE_DEPOT_PASSWORD
valueFrom:
secretKeyRef:
key: offlinePassword
name: depot-service-secret
- name: DEPOT_SERVICE_SERVICE_FQDN
value: depot-service.vcf-fleet-depot.svc.cluster.local
- name: DISTRIBUTION_SERVICE_HOST
value: distribution-service.vcf-fleet-depot.svc.cluster.local
- name: PROXY_FORWARDER_HOST
value: 127.0.0.1
- name: SERVER_HTTPS_PORT
value: "7443"
- name: DNS_RESOLVER
value: kube-dns.kube-system.svc.cluster.local
- name: SKIP_PROXY_FORWARDER
value: "false"
- name: PUBLIC_DEPOT_HOST
value: dl.broadcom.com
- name: PUBLIC_VVS_HOST
value: vvs.broadcom.com
- name: PUBLIC_VSAN_HCL_HOST
value: eapi.broadcom.com
- name: PUBLIC_PACKAGES_HOST
value: projects.packages.broadcom.com
- name: OFFLINE_OCI_WRITE_ENABLED
value: "false"
- name: OCI_REPO_PATH_MAPPINGS_OVERRIDE
value: '[]'
- name: PUBLIC_VVS_VCF_LCM_BUNDLE_PATH
value: /v1/products/bundles/type/vcf-lcm-v2-bundle
- name: PUBLIC_VVS_VCF_INTEROP_BUNDLE_PATH
value: /v1/products/bundles/type/vcf-interop-bundle
- name: PUBLIC_VVS_VLCM_INTEROP_VCG_BUNDLE_PATH
value: /v1/products/bundles/vlcm-compatibility-data/v1/vlcm-interop-vcg-bundle
- name: CONFIG_WORKER_CONNECTIONS
value: "1024"
- name: CONFIG_WORKER_RLIMIT_NOFILE
value: "4096"
- name: CONFIG_SEND_TIMEOUT
value: 300s
- name: CONFIG_OUTPUT_BUFFERS
value: 2 512k
- name: CONFIG_RATE_LIMIT_ZONE
value: downloads:10m rate=100r/s
- name: CONFIG_RATE_LIMIT_BURST
value: "100"
- name: DISABLE_CACHE
value: "false"
- name: CONFIG_CACHE_INACTIVE
value: 365d
- name: CONFIG_CACHE_KEYS_ZONE_SIZE
value: 100m
- name: CONFIG_CACHE_LOCK_AGE
value: 3600s
- name: CONFIG_CACHE_LOCK_TIMEOUT
value: 60s
- name: CONFIG_CACHE_MAX_SIZE
value: 950g
- name: CONFIG_CACHE_VALID_BINARIES
value: 200 365d
- name: CONFIG_CACHE_REVALIDATE_BINARIES
value: "off"
- name: CONFIG_CACHE_REVALIDATE_METADATA
value: "on"
- name: CONFIG_CACHE_EXCLUDE_FILE_EXTENSIONS
value: json,yaml,xml,yml,xml.sha256,json.sha256,xml.sign,json.sign,xml.sig
- name: CONFIG_PROXY_MAX_TEMP_FILE_SIZE
value: "0"
- name: CONFIG_SSL_SESSION_CACHE
value: shared:SSL:50m
- name: CONFIG_SSL_SESSION_TIMEOUT
value: 1d
- name: CONFIG_PROXY_SSL_SESSION_REUSE
value: "on"
- name: CONFIG_PROXY_READ_TIMEOUT
value: "300"
image: registry.vmsp-platform.svc.cluster.local:5000/images/file-server:9.1.0-0_25371105
imagePullPolicy: Always
livenessProbe:
failureThreshold: 3
httpGet:
path: /internal/api/content-gateway/v1/health
port: 7443
scheme: HTTPS
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: file-server
ports:
- containerPort: 7443
name: fs-https
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /internal/api/content-gateway/v1/health
port: 7443
scheme: HTTPS
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
limits:
memory: 1Gi
requests:
cpu: 300m
memory: 512Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
- CHOWN
- SETGID
- SETUID
drop:
- ALL
privileged: false
startupProbe:
failureThreshold: 60
httpGet:
path: /internal/api/content-gateway/v1/health
port: 7443
scheme: HTTPS
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /data
name: data-volume
subPath: data
- mountPath: /etc/service/trust
name: vcf-fleet-depot-certificate
readOnly: true
- mountPath: /etc/platform/trust
name: platform-trust
readOnly: true
- mountPath: /etc/service/secret/depot-service-secret
name: depot-service-secret
readOnly: true
- env:
- name: PLATFORM_TRUST
value: /etc/platform/trust
- name: BCF_TRUSTSTORE_PATH
value: /etc/platform/java/fips/bundle.bcfks
- name: JAVA_TRUSTSTORE_FILE_NAME
value: bundle.jks
- name: SERVER_CERT_DIR
value: /etc/service/trust
- name: SERVER_CERT_FILE
value: tls.crt
- name: SERVER_KEY_FILE
value: tls.key
- name: BCF_SERVER_KEYSTORE_PATH
value: /etc/platform/java/fips/tls.bcfks
- name: PEER_PROXY_CERT_DIR
value: /etc/platform/peer-proxy-certs
- name: PEER_PROXY_CERT_FILENAME
value: https_proxy.pem
- name: SPRING_PROFILES_ACTIVE
value: depot
- name: SPRING_PROFILES_INCLUDE
value: override
- name: VCF_USE_RESOURCE_BUNDLE_LOCALIZATION
value: "true"
- name: KUBERNETES_CLUSTER_DOMAIN
value: svc.cluster.local
- name: KUBERNETES_SERVICE_ACCOUNT_PATH
value: /etc/service/secret/depot-service-secret
- name: DEPOT_SERVICE_NAME
value: depot-service
- name: DEPOT_SERVICE_SECRET_NAME
value: depot-service-secret
- name: DEPOT_SERVICE_SERVICE_FQDN
value: depot-service.vcf-fleet-depot.svc.cluster.local
envFrom:
- configMapRef:
name: download-service-configs
image: registry.vmsp-platform.svc.cluster.local:5000/images/download-service:9.1.0-0_25371105
imagePullPolicy: Always
livenessProbe:
failureThreshold: 3
httpGet:
path: /depot-service/api/depot/v1/health/livenessState
port: 8443
scheme: HTTPS
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: download-service
ports:
- containerPort: 8080
name: ds-http
protocol: TCP
- containerPort: 8443
name: ds-https
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /depot-service/api/depot/v1/health/readinessState
port: 8443
scheme: HTTPS
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
limits:
memory: 1Gi
requests:
cpu: 300m
memory: 512Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
- CHOWN
- SETGID
- SETUID
drop:
- ALL
privileged: false
startupProbe:
failureThreshold: 60
httpGet:
path: /depot-service/api/depot/v1/health/ping
port: 8443
scheme: HTTPS
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /data
name: data-volume
subPath: data
- mountPath: /etc/config
name: config-volume
- mountPath: /etc/platform/peer-proxy-service-config
name: peer-proxy-config
readOnly: true
- mountPath: /etc/platform/peer-proxy-certs
name: proxy-peer-certificate
readOnly: true
- mountPath: /etc/platform/peer-proxy-credentials
name: proxy-peer-credentials
readOnly: true
- mountPath: /etc/service/trust
name: vcf-fleet-depot-certificate
readOnly: true
- mountPath: /etc/platform/trust
name: platform-trust
readOnly: true
- mountPath: /etc/service/secret/depot-service-secret
name: depot-service-secret
readOnly: true
- mountPath: /etc/vmware/feature-state-switches
name: feature-state-switches-volume
readOnly: true
- env:
- name: LISTEN_PORT
value: "9080"
- name: PROXY_HOST
- name: PROXY_PORT
- name: PROXY_SCHEME
value: http
- name: PROXY_CREDENTIALS_ENABLED
value: "false"
- name: PROXY_USERNAME
valueFrom:
secretKeyRef:
key: username
name: proxy-service-credentials
optional: true
- name: PROXY_PASSWORD
valueFrom:
secretKeyRef:
key: password
name: proxy-service-credentials
optional: true
- name: PROXY_CERTIFICATE
valueFrom:
secretKeyRef:
key: https_proxy.pem
name: proxy-service-peer-certificate
optional: true
- name: PLATFORM_TRUST_BUNDLE
value: /etc/platform/trust/bundle.pem
- name: OFFLINE_DEPOT_CERTIFICATES_PATH
value: /etc/depot/trust/certificates.pem
- name: DEPOT_SERVICE_SERVICE_FQDN
value: depot-service.vcf-fleet-depot.svc.cluster.local
- name: DISABLE_PROXY
value: "false"
image: registry.vmsp-platform.svc.cluster.local:5000/images/proxy-forwarder:9.1.0-0_25371105
imagePullPolicy: Always
livenessProbe:
failureThreshold: 3
httpGet:
path: /internal/api/proxy-forwarder/v1/health
port: 9080
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: proxy-forwarder
ports:
- containerPort: 9080
name: pf-http
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /internal/api/proxy-forwarder/v1/health
port: 9080
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
limits:
memory: 320Mi
requests:
cpu: 300m
memory: 256Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
- CHOWN
- SETGID
- SETUID
drop:
- ALL
privileged: false
startupProbe:
failureThreshold: 60
httpGet:
path: /internal/api/proxy-forwarder/v1/health
port: 9080
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/platform/trust
name: platform-trust
readOnly: true
- mountPath: /etc/depot/trust
name: offline-depot-certificates
readOnly: true
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: depot-service-service-account
serviceAccountName: depot-service-service-account
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 420
name: peer-proxy-config
optional: true
name: peer-proxy-config
- name: proxy-peer-certificate
secret:
defaultMode: 420
optional: true
secretName: proxy-service-peer-certificate
- name: proxy-peer-credentials
secret:
defaultMode: 420
optional: true
secretName: proxy-service-credentials
- name: data-volume
persistentVolumeClaim:
claimName: depotservice-data-pvc
- configMap:
defaultMode: 420
name: download-service-configs
name: config-volume
- name: vcf-fleet-depot-certificate
secret:
defaultMode: 420
secretName: depot-service-intra-cert
- configMap:
defaultMode: 420
name: platform-trust
name: platform-trust
- name: depot-service-secret
secret:
defaultMode: 420
secretName: depot-service-secret
- name: offline-depot-certificates
secret:
defaultMode: 420
items:
- key: offlineCertificate
path: certificates.pem
optional: true
secretName: depot-service-secret
- configMap:
defaultMode: 420
name: feature-state-switches
name: feature-state-switches-volume
status:
availableReplicas: 1
conditions:
- lastTransitionTime: "2026-05-12T13:39:13Z"
lastUpdateTime: "2026-05-12T14:03:31Z"
message: ReplicaSet "depot-service-77cb99b56b" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
- lastTransitionTime: "2026-05-27T06:15:00Z"
lastUpdateTime: "2026-05-27T06:15:00Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
observedGeneration: 2
readyReplicas: 1
replicas: 1
updatedReplicas: 1
8.2.2 – distribution.service.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "2"
meta.helm.sh/release-name: distribution-service
meta.helm.sh/release-namespace: vcf-fleet-depot
reloader.vmsp.vmware.com/auto: "true"
creationTimestamp: "2026-05-12T13:38:43Z"
generation: 2
labels:
app: distribution-service
app.kubernetes.io/component: distribution-service
app.kubernetes.io/instance: distribution-service
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: distribution-service
app.kubernetes.io/version: 9.1.0
helm.sh/chart: distribution-service-9.1.0-0_25371105
helm.toolkit.fluxcd.io/name: distribution-service
helm.toolkit.fluxcd.io/namespace: vcf-fleet-depot
ops.vmsp.vmware.com/type: vcf-distribution-service
name: distribution-service
namespace: vcf-fleet-depot
resourceVersion: "13209944"
uid: 9b0c4088-29fa-44fc-b577-0af811ec60eb
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 5
selector:
matchLabels:
app.kubernetes.io/component: distribution-service
app.kubernetes.io/instance: distribution-service
app.kubernetes.io/name: distribution-service
strategy:
type: Recreate
template:
metadata:
annotations:
checksum/init-config: 9035ec91c56485b6d0bb0367988735b7afd40a05d63b49b752241482381daaca
vmsp.vmware.com/triggerrestart: "50227"
labels:
app: distribution-service
app.kubernetes.io/component: distribution-service
app.kubernetes.io/instance: distribution-service
app.kubernetes.io/name: distribution-service
ops.vmsp.vmware.com/type: vcf-distribution-service-instance
spec:
containers:
- image: registry.vmsp-platform.svc.cluster.local:5000/images/distribution-service:9.1.0-0_25371105
imagePullPolicy: Always
livenessProbe:
failureThreshold: 3
httpGet:
path: /
port: 5443
scheme: HTTPS
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 5
name: distribution-service
ports:
- containerPort: 18080
name: distr-http
protocol: TCP
- containerPort: 18443
name: distr-https
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /
port: 5443
scheme: HTTPS
periodSeconds: 120
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
memory: 1Gi
requests:
cpu: 500m
memory: 512Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
- CHOWN
- SETGID
- SETUID
drop:
- ALL
privileged: false
startupProbe:
failureThreshold: 30
httpGet:
path: /
port: 5443
scheme: HTTPS
periodSeconds: 15
successThreshold: 1
timeoutSeconds: 5
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/lib/registry
name: distributionservice-data-pvc
- mountPath: /etc/docker/registry
name: config-dir
- mountPath: /certs
name: distribution-certificate
readOnly: true
- mountPath: /etc/platform/trust
name: platform-trust
readOnly: true
dnsPolicy: ClusterFirst
initContainers:
- args:
- /etc/scripts/init-config.sh
command:
- /bin/sh
image: registry.vmsp-platform.svc.cluster.local:5000/images/distribution-service:9.1.0-0_25371105
imagePullPolicy: Always
name: distribution-service-init
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/docker/registry
name: config-dir
- mountPath: /etc/scripts
name: init-config
- mountPath: /certs
name: distribution-certificate
readOnly: true
- mountPath: /etc/platform/trust
name: platform-trust
readOnly: true
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
volumes:
- name: distributionservice-data-pvc
persistentVolumeClaim:
claimName: distributionservice-data-pvc
- emptyDir: {}
name: config-dir
- configMap:
defaultMode: 493
name: distribution-service-init-config
name: init-config
- emptyDir: {}
name: registry-workdir
- name: distribution-certificate
secret:
defaultMode: 420
secretName: distribution-intra-cert
- configMap:
defaultMode: 420
name: platform-trust
name: platform-trust
status:
availableReplicas: 1
conditions:
- lastTransitionTime: "2026-05-12T13:38:43Z"
lastUpdateTime: "2026-05-12T14:00:57Z"
message: ReplicaSet "distribution-service-84574b5cd" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
- lastTransitionTime: "2026-05-27T06:12:21Z"
lastUpdateTime: "2026-05-27T06:12:21Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
observedGeneration: 2
readyReplicas: 1
replicas: 1
updatedReplicas: 1
9 – Telemetry
9.1 – Telemetry Optimisations
The telemetry service has 500m/512MiB, the RAM its using, but the CPU not so much, this I am going to drop to 100m as this is just a phone home service and is well within the current usage
NAME CPU(cores) MEMORY(bytes)
telemetry-acceptor-79746d45c6-7cs9g 2m 441Mi
We can change this with
kubectl patch deployment telemetry-acceptor -n telemetry --type='json' -p='[
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/cpu", "value": "100m"},
{"op": "replace", "path": "/spec/template/spec/containers/0/resources/limits/cpu", "value": "250m"}
]'
9.2 Original YAML Backups
9.2.1 – telemetry-acceptor.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
meta.helm.sh/release-name: telemetry-acceptor
meta.helm.sh/release-namespace: telemetry
reloader.vmsp.vmware.com/auto: "true"
reloader.vmsp.vmware.com/exclusion.configmap: ceip
reloader.vmsp.vmware.com/exclusion.secret: telemetry-config
creationTimestamp: "2026-05-12T14:30:06Z"
generation: 1
labels:
app.kubernetes.io/instance: telemetry-acceptor
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: telemetry-acceptor
app.kubernetes.io/version: 1.16.0
helm.sh/chart: telemetry-acceptor-9.1.0-25181946
helm.toolkit.fluxcd.io/name: telemetry-acceptor
helm.toolkit.fluxcd.io/namespace: telemetry
ops.vmsp.vmware.com/type: telemetry-acceptor-service
name: telemetry-acceptor
namespace: telemetry
resourceVersion: "13208066"
uid: f65038fe-8e4f-467b-b7f1-090efb8069b6
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/instance: telemetry-acceptor
app.kubernetes.io/name: telemetry-acceptor
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 50%
type: RollingUpdate
template:
metadata:
labels:
app.kubernetes.io/instance: telemetry-acceptor
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: telemetry-acceptor
app.kubernetes.io/version: 1.16.0
helm.sh/chart: telemetry-acceptor-9.1.0-25181946
ops.vmsp.vmware.com/type: telemetry-acceptor-service-resource
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- telemetry-acceptor
topologyKey: kubernetes.io/hostname
weight: 100
containers:
- env:
- name: LOG_LEVEL
value: INFO
- name: SERVER_PORT
value: "1081"
- name: MANAGEMENT_SERVER_PORT
value: "9091"
- name: JAVA_TOOL_OPTIONS
value: |
-XX:+UseContainerSupport -XX:MaxRAMPercentage="75" -XX:InitialRAMPercentage="50"
- name: COMPONENT_ID
value: da545315-0aa9-49c3-b136-f17294358bbc
- name: COMPONENT_TYPE
value: telemetry-acceptor
- name: COMPONENT_VERSION
value: 9.1.0.0.25181946
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: VSP_PROXY_HOST
valueFrom:
secretKeyRef:
key: host
name: vmsp-proxy-service-secret
optional: true
- name: VSP_PROXY_PORT
valueFrom:
secretKeyRef:
key: port
name: vmsp-proxy-service-secret
optional: true
- name: VSP_PROXY_USERNAME
valueFrom:
secretKeyRef:
key: username
name: vmsp-proxy-service-secret
optional: true
- name: VSP_PROXY_PASSWORD
valueFrom:
secretKeyRef:
key: password
name: vmsp-proxy-service-secret
optional: true
- name: VSP_PROXY_SCHEME
valueFrom:
secretKeyRef:
key: scheme
name: vmsp-proxy-service-secret
optional: true
- name: SERVER_SSL_ENABLED
value: "true"
- name: SERVER_SSL_CERTIFICATE
value: file:/etc/tls/tls.crt
- name: SERVER_SSL_CERTIFICATE_PRIVATE_KEY
value: file:/etc/tls/tls.key
- name: SERVER_HTTP_ENABLED
value: "true"
- name: SERVER_HTTP_PORT
value: "8080"
image: registry.vmsp-platform.svc.cluster.local:5000/images/telemetry-acceptor/9.1.0.0.25181946/telemetry-acceptor/telemetry-acceptor:9.1.0.0.25181946
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 30
httpGet:
path: /debug/health/liveness
port: 9091
scheme: HTTP
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 1
name: telemetry-acceptor
ports:
- containerPort: 1081
name: https
protocol: TCP
- containerPort: 8080
name: http
protocol: TCP
- containerPort: 9091
name: management
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /debug/health/readiness
port: 9091
scheme: HTTP
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 1
resources:
limits:
cpu: "1"
memory: 1Gi
requests:
cpu: 500m
memory: 512Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 65532
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/telemetry-config
name: telemetry-config-volume
readOnly: true
- mountPath: /etc/ceip
name: ceip-volume
readOnly: true
- mountPath: /etc/tls
name: tls-cert
readOnly: true
- mountPath: /tmp
name: tmp-volume
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 65532
runAsGroup: 65532
runAsNonRoot: true
runAsUser: 65532
terminationGracePeriodSeconds: 30
volumes:
- name: telemetry-config-volume
secret:
defaultMode: 420
secretName: telemetry-config
- configMap:
defaultMode: 420
name: ceip
name: ceip-volume
- name: tls-cert
secret:
defaultMode: 420
secretName: telemetry-acceptor-tls
- emptyDir: {}
name: tmp-volume
status:
availableReplicas: 1
conditions:
- lastTransitionTime: "2026-05-12T14:30:06Z"
lastUpdateTime: "2026-05-12T14:30:28Z"
message: ReplicaSet "telemetry-acceptor-79746d45c6" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
- lastTransitionTime: "2026-05-27T06:11:51Z"
lastUpdateTime: "2026-05-27T06:11:51Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
observedGeneration: 1
readyReplicas: 1
replicas: 1
updatedReplicas: 1
10 – Node Resize
10.1 – Node Resize Optimisations
Now we have tweaked the pods and reduce the CPU/Memory requests we can tweak the node sizes now without causing as many issues
There is a few places this is defined, and we must edit it at the top level, else it will be reconciled back to the original configuration
The VMSP Platform is where this is stored, and we can get the YAML file defining node size, amoung other things, with
kubectl get packagedeployment vmsp-platform -n vmsp-platform -o yaml
In that YAML we have this spec field, which labels the node sizes as large and management.large
values:
backups:
destination: sftp
full:
enable: true
schedule: 10 0 * * MON,TUE,WED,THU,FRI,SAT,SUN
incremental:
enable: false
schedule: '*/15 * * * *'
retention:
maxAgeDays: 7
maxBackups: -1
storage:
sftp:
fingerprint: SHA256:LjIZbgd6sU5G1d4jIvimEJdRJpF4wrkT0xOGm8IJRDU
ceip: {}
cluster:
generation: 0
ha: "false"
id: ae05f3e0-1b68-4a70-87c8-5a27bb70f9bc
name: aerosphere
nodePrefix: aerosphere
type: management
vmspVMMode: bootstrap
worker:
enabled: true
machineType: management.large
minReplicas: 3
size: large
We can then patch this to the medium node size, 8vCPU/16GB with
kubectl patch packagedeployment vmsp-platform -n vmsp-platform --type='json' -p='[
{"op": "replace", "path": "/spec/values/cluster/worker/machineType", "value": "management.medium"},
{"op": "replace", "path": "/spec/values/cluster/worker/size", "value": "medium"}
]'
We first need to get the machine deployment
kubectl get machinedeployment -A
This returns something like this, I have removed the end bits so its fits here better
NAMESPACE NAME CLUSTER AVAILABLE DESIRED CURRENT READY AVAILABLE
vmsp-platform aerosphere-default-szs5k aerosphere True 3 3 3 3
Whats we need here is the name, aerosphere-default-szs5k
Now we know the machine deployment name, we can get the full YAML file with
kubectl get machinedeployment <machine-deployment> -n vmsp-platform -o yaml
For my cluster, I ran
kubectl get machinedeployment aerosphere-default-szs5k -n vmsp-platform -o yaml
In that yaml, we are looking for, is the infastructureRed section with the kind: VSphereMachineTemplate, here its the name value we need, I have aerosphere-default-j72pl
In mine I have
infrastructureRef:
apiGroup: infrastructure.cluster.x-k8s.io
kind: VSphereMachineTemplate
name: aerosphere-default-j72pl
version: v1.34.2
Now we know what the name of the vSphere Machine Template, we can get its yaml config with
kubectl get vspheremachinetemplate <VSphereMachineTemplate-name> -n vmsp-platform -o yaml
For my environment I ran
kubectl get vspheremachinetemplate aerosphere-default-j72pl -n vmsp-platform -o yaml
What we are looking for is the ownerReferences part, and we want the name
ownerReferences:
- apiVersion: cluster.x-k8s.io/v1beta2
kind: Cluster
name: aerosphere
uid: f2eca7f5-de56-4500-8a7c-bf295ef842d3
The correct place to edit this will be the cluster level, to check that yaml, we need to run
kubectl get cluster <name> -n vmsp-platform -o yaml
I ran
kubectl get cluster aerosphere -n vmsp-platform -o yaml
And its here we can find the name of the helm chart, we need the helm flux name, helm.toolkit.fluxcd.io/name, which for me is vmsp-cluster-aerosphere
labels:
app.kubernetes.io/instance: vmsp-cluster-aerosphere
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: vmsp-cluster
app.kubernetes.io/version: 9.1.1865
cluster.x-k8s.io/cluster-name: aerosphere
controlplane.vmsp.vmware.com/cloud-provider: vsphere
helm.sh/chart: vmsp-cluster-9.1.1865
helm.toolkit.fluxcd.io/name: vmsp-cluster-aerosphere
helm.toolkit.fluxcd.io/namespace: vmsp-platform
topology.cluster.x-k8s.io/owned: ""
We can patch this to the correct size, the reason we patch the helm release is flux will reconcile its yaml, so while we can manually change the size of the worker nodes, they will get overidden
Then next size down is medium, the default is large, and we can set this with
kubectl patch helmrelease <helm.toolkit.fluxcd.io/name> -n vmsp-platform --type='json' -p='[
{"op": "replace", "path": "/spec/values/worker/machineType", "value": "management.medium"}
]'
Which for me was
kubectl patch helmrelease vmsp-cluster-aerosphere -n vmsp-platform --type='json' -p='[
{"op": "replace", "path": "/spec/values/worker/machineType", "value": "management.medium"}
]'
CAPV will begin automatically rolling out new nodes of the correct size to replace the old ones
Once the roll out was completed I did have to go back and re apply the identity broker config as the DB has reset
10.2 – Original YAML Backups
10.2.1 – cluster-autoscaler.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
meta.helm.sh/release-name: cluster-autoscaler
meta.helm.sh/release-namespace: vmsp-platform
creationTimestamp: "2026-05-12T12:59:50Z"
generation: 1
labels:
app.kubernetes.io/instance: cluster-autoscaler
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: clusterapi-cluster-autoscaler
helm.sh/chart: cluster-autoscaler-9.5.1
helm.toolkit.fluxcd.io/name: cluster-autoscaler
helm.toolkit.fluxcd.io/namespace: vmsp-platform
name: cluster-autoscaler-clusterapi-cluster-autoscaler
namespace: vmsp-platform
resourceVersion: "13190340"
uid: 9cc9caab-a879-44ea-b3a4-95ee66480edb
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/instance: cluster-autoscaler
app.kubernetes.io/name: clusterapi-cluster-autoscaler
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
labels:
app.kubernetes.io/instance: cluster-autoscaler
app.kubernetes.io/name: clusterapi-cluster-autoscaler
spec:
containers:
- command:
- ./cluster-autoscaler
- --cloud-provider=clusterapi
- --namespace=vmsp-platform
- --node-group-auto-discovery=clusterapi:clusterName=aerosphere
- --enforce-node-group-min-size=true
- --leader-elect=false
- --logtostderr=true
- --scan-interval=60s
- --stderrthreshold=info
- --v=4
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: SERVICE_ACCOUNT
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.serviceAccountName
image: registry.vmsp-platform.svc.cluster.local:5000/images/cluster-autoscaler:v1.34.0
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /health-check
port: 8085
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: clusterapi-cluster-autoscaler
ports:
- containerPort: 8085
protocol: TCP
resources:
limits:
memory: 1Gi
requests:
cpu: 10m
memory: 100Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
priorityClassName: system-cluster-critical
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: cluster-autoscaler-clusterapi-cluster-autoscaler
serviceAccountName: cluster-autoscaler-clusterapi-cluster-autoscaler
terminationGracePeriodSeconds: 30
status:
availableReplicas: 1
conditions:
- lastTransitionTime: "2026-05-12T12:59:50Z"
lastUpdateTime: "2026-05-12T12:59:53Z"
message: ReplicaSet "cluster-autoscaler-clusterapi-cluster-autoscaler-7c7b8d4c99"
has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
- lastTransitionTime: "2026-05-26T07:06:44Z"
lastUpdateTime: "2026-05-26T07:06:44Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
observedGeneration: 1
readyReplicas: 1
replicas: 1
updatedReplicas: 1
10.2.2 – worker-machinedeployment.yaml
apiVersion: cluster.x-k8s.io/v1beta2
kind: MachineDeployment
metadata:
annotations:
cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: "20"
cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: "3"
machinedeployment.clusters.x-k8s.io/revision: "1"
creationTimestamp: "2026-05-12T12:51:25Z"
finalizers:
- cluster.x-k8s.io/machinedeployment
- machinedeployment.topology.cluster.x-k8s.io
generation: 17
labels:
cluster.x-k8s.io/cluster-name: aerosphere
topology.cluster.x-k8s.io/deployment-name: default
topology.cluster.x-k8s.io/owned: ""
name: aerosphere-default-szs5k
namespace: vmsp-platform
ownerReferences:
- apiVersion: cluster.x-k8s.io/v1beta2
kind: Cluster
name: aerosphere
uid: f2eca7f5-de56-4500-8a7c-bf295ef842d3
resourceVersion: "13209883"
uid: a37dc85c-bbfe-472a-acd2-1555fc6722de
spec:
clusterName: aerosphere
replicas: 3
rollout:
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
type: RollingUpdate
selector:
matchLabels:
cluster.x-k8s.io/cluster-name: aerosphere
topology.cluster.x-k8s.io/deployment-name: default
topology.cluster.x-k8s.io/owned: ""
template:
metadata:
annotations:
cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: "20"
cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: "3"
labels:
cluster.x-k8s.io/cluster-name: aerosphere
topology.cluster.x-k8s.io/deployment-name: default
topology.cluster.x-k8s.io/owned: ""
spec:
bootstrap:
configRef:
apiGroup: bootstrap.cluster.x-k8s.io
kind: KubeadmConfigTemplate
name: aerosphere-default-jg76p
clusterName: aerosphere
deletion:
nodeDeletionTimeoutSeconds: 1800
nodeDrainTimeoutSeconds: 1800
nodeVolumeDetachTimeoutSeconds: 1800
infrastructureRef:
apiGroup: infrastructure.cluster.x-k8s.io
kind: VSphereMachineTemplate
name: aerosphere-default-j72pl
version: v1.34.2
status:
availableReplicas: 3
conditions:
- lastTransitionTime: "2026-05-25T13:19:28Z"
message: ""
observedGeneration: 17
reason: Available
status: "True"
type: Available
- lastTransitionTime: "2026-05-12T12:51:31Z"
message: ""
observedGeneration: 17
reason: NotRollingOut
status: "False"
type: RollingOut
- lastTransitionTime: "2026-05-13T06:29:16Z"
message: ""
observedGeneration: 17
reason: NotRemediating
status: "False"
type: Remediating
- lastTransitionTime: "2026-05-26T07:09:22Z"
message: ""
observedGeneration: 17
reason: NotScalingDown
status: "False"
type: ScalingDown
- lastTransitionTime: "2026-05-24T22:17:55Z"
message: ""
observedGeneration: 17
reason: NotScalingUp
status: "False"
type: ScalingUp
- lastTransitionTime: "2026-05-26T07:09:22Z"
message: ""
observedGeneration: 17
reason: Ready
status: "True"
type: MachinesReady
- lastTransitionTime: "2026-05-12T12:51:31Z"
message: ""
observedGeneration: 17
reason: UpToDate
status: "True"
type: MachinesUpToDate
- lastTransitionTime: "2026-05-12T12:51:31Z"
message: ""
observedGeneration: 17
reason: NotPaused
status: "False"
type: Paused
- lastTransitionTime: "2026-05-12T12:51:31Z"
message: ""
observedGeneration: 17
reason: NotDeleting
status: "False"
type: Deleting
deprecated:
v1beta1:
availableReplicas: 3
conditions:
- lastTransitionTime: "2026-05-27T06:12:18Z"
status: "True"
type: Ready
- lastTransitionTime: "2026-05-27T06:12:18Z"
status: "True"
type: Available
- lastTransitionTime: "2026-05-26T07:09:22Z"
status: "True"
type: MachineSetReady
readyReplicas: 3
unavailableReplicas: 0
updatedReplicas: 3
observedGeneration: 17
phase: Running
readyReplicas: 3
replicas: 3
selector: cluster.x-k8s.io/cluster-name=aerosphere,topology.cluster.x-k8s.io/deployment-name=default,topology.cluster.x-k8s.io/owned=
upToDateReplicas: 3
10.2.3 – vspheremachinetemplate.yaml
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: VSphereMachineTemplate
metadata:
annotations:
cluster.x-k8s.io/cloned-from-groupkind: VSphereMachineTemplate.infrastructure.cluster.x-k8s.io
cluster.x-k8s.io/cloned-from-name: vsphere-9.1.1865-worker-machinetemplate
helm.sh/resource-policy: keep
meta.helm.sh/release-name: vmsp-configs
meta.helm.sh/release-namespace: vmsp-platform
creationTimestamp: "2026-05-12T12:51:25Z"
generation: 1
labels:
app.kubernetes.io/instance: vmsp-configs
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: vmsp-configs
app.kubernetes.io/version: 9.1.1865
cluster.x-k8s.io/cluster-name: aerosphere
helm.sh/chart: vmsp-configs-9.1.1865
helm.toolkit.fluxcd.io/name: vmsp-configs
helm.toolkit.fluxcd.io/namespace: vmsp-platform
topology.cluster.x-k8s.io/deployment-name: default
topology.cluster.x-k8s.io/owned: ""
name: aerosphere-default-j72pl
namespace: vmsp-platform
ownerReferences:
- apiVersion: cluster.x-k8s.io/v1beta2
kind: Cluster
name: aerosphere
uid: f2eca7f5-de56-4500-8a7c-bf295ef842d3
resourceVersion: "23039"
uid: 4bccfe6d-a9bb-46c2-aa73-a861cc6fd709
spec:
template:
spec:
cloneMode: fullClone
datacenter: Datacenter:datacenter-1001
datastore: Datastore:datastore-672582
diskGiB: 100
folder: Folder:group-v683669
memoryMiB: 24576
namingStrategy:
template: aerosphere-{{ trunc -5 .machine.name }}
network:
devices:
- addressesFromPools:
- apiGroup: ipam.cluster.x-k8s.io
kind: InClusterIPPool
name: aerosphere-global-ip-pool
dhcp4: false
dhcp6: false
nameservers:
- 192.168.1.5
- 192.168.1.27
networkName: DistributedVirtualPortgroup:dvportgroup-1069
numCPUs: 12
os: Linux
powerOffMode: trySoft
resourcePool: ResourcePool:resgroup-672691
server: borealis.leaha.co.uk
storagePolicyName: ""
template: VirtualMachine:vm-683670
10.2.4 – <cluster-name>-cluster.yaml
apiVersion: cluster.x-k8s.io/v1beta2
kind: Cluster
metadata:
creationTimestamp: "2026-05-12T12:51:24Z"
finalizers:
- cluster.cluster.x-k8s.io
generation: 110
labels:
app.kubernetes.io/instance: vmsp-cluster-aerosphere
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: vmsp-cluster
app.kubernetes.io/version: 9.1.1865
cluster.x-k8s.io/cluster-name: aerosphere
controlplane.vmsp.vmware.com/cloud-provider: vsphere
helm.sh/chart: vmsp-cluster-9.1.1865
helm.toolkit.fluxcd.io/name: vmsp-cluster-aerosphere
helm.toolkit.fluxcd.io/namespace: vmsp-platform
topology.cluster.x-k8s.io/owned: ""
name: aerosphere
namespace: vmsp-platform
resourceVersion: "14160176"
uid: f2eca7f5-de56-4500-8a7c-bf295ef842d3
spec:
clusterNetwork:
pods:
cidrBlocks:
- 198.18.0.0/17
services:
cidrBlocks:
- 198.18.128.0/17
controlPlaneEndpoint:
host: 192.168.1.130
port: 6443
controlPlaneRef:
apiGroup: controlplane.cluster.x-k8s.io
kind: KubeadmControlPlane
name: aerosphere-25ndd
infrastructureRef:
apiGroup: infrastructure.cluster.x-k8s.io
kind: VSphereCluster
name: aerosphere-9dwkk
topology:
classRef:
name: vsphere-9.1.1865
controlPlane:
replicas: 1
variables:
- name: network
value: DistributedVirtualPortgroup:dvportgroup-1069
- name: credsSecretName
value: aerosphere-vsphere-secret
- name: dhcp6Enabled
value: false
- name: infraServerThumbprint
value: 1A:65:C2:2A:F1:64:75:0C:74:BF:00:6A:A4:F1:E3:40:16:7A:78:61
- name: datacenter
value: Datacenter:datacenter-1001
- name: dhcp4Enabled
value: false
- name: datastore
value: Datastore:datastore-672582
- name: storagePolicy
value: ""
- name: nodePrefix
value: aerosphere
- name: ipPool
value:
- apiGroup: ipam.cluster.x-k8s.io
kind: InClusterIPPool
name: aerosphere-global-ip-pool
- name: nameservers
value:
- 192.168.1.5
- 192.168.1.27
- name: ntpServers
value:
- time.cloudflare.com
- name: infraServerURL
value: borealis.leaha.co.uk
- name: resourcePool
value: ResourcePool:resgroup-672691
- name: folder
value: Folder:group-v683669
- name: addressFamily
value: ipv4
- name: vmTemplate
value: VirtualMachine:vm-683670
- name: dnsImageTag
value: v1.12.4
- name: etcdImageTag
value: 3.6.5-0
- name: imageRepository
value: registry.vmsp-platform.svc.cluster.local:5000/images
- name: kubeVipPodManifest
value: |
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
name: kube-vip
namespace: kube-system
spec:
containers:
- args:
- manager
env:
- name: vip_arp
value: "true"
- name: port
value: "6443"
- name: vip_interface
value: ""
- name: vip_cidr
value: "32"
- name: cp_enable
value: "true"
- name: cp_namespace
value: kube-system
- name: vip_ddns
value: "false"
- name: svc_enable
value: "false"
- name: svc_leasename
value: plndr-svcs-lock
- name: svc_election
value: "true"
- name: vip_leaderelection
value: "false"
- name: vip_leasename
value: plndr-cp-lock
- name: vip_leaseduration
value: "15"
- name: vip_renewdeadline
value: "10"
- name: vip_retryperiod
value: "2"
- name: vip_preserve_on_leadership_loss
value: "false"
- name: address
value: ${CONTROL_PLANE_ENDPOINT_IP}
- name: prometheus_server
value: :2112
image: registry.vmsp-platform.svc.cluster.local:5000/images/kube-vip:v1.0.2
imagePullPolicy: IfNotPresent
name: kube-vip
resources:
requests:
cpu: 50m
memory: 100Mi
limits:
cpu: 200m
memory: 256Mi
securityContext:
capabilities:
add:
- NET_ADMIN
- NET_RAW
volumeMounts:
- mountPath: /etc/kubernetes/admin.conf
name: kubeconfig
- mountPath: /etc/hosts
name: etchosts
hostNetwork: true
volumes:
- hostPath:
path: /etc/kubernetes/admin.conf
type: FileOrCreate
name: kubeconfig
- hostPath:
path: /etc/kube-vip.hosts
type: File
name: etchosts
status: {}
- name: kubeApiServer
value:
cpuRequest: 250m
defaultWatchCacheSize: "150"
gogc: "85"
gomemlimit: 3200MiB
memoryLimit: 3500Mi
memoryRequest: 512Mi
watchCacheSizes: secrets#300,pods#300,deployments#150,statefulsets#75,daemonsets#75,replicasets#75,configmaps#150,endpoints#150,services#150,events#0,customresourcedefinitions#150,apiservices#75,leases#300,helmreleases#300,machines#200,machinedeployments#150,clusters#150,updaterequests#300,selfsubjectaccessreviews#500
- name: controlPlaneIpAddr
value: 192.168.1.130
- name: numCPUs
value: 4
- name: memoryMiB
value: 10240
- name: diskGiB
value: 100
- name: taintControlPlane
value: true
- name: user
value:
passwordSecret:
key: sshPassword
name: aerosphere-ssh-password-secret
version: v1.34.2
workers:
machineDeployments:
- class: vsphere-9.1.1865-worker
metadata:
annotations:
cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: "20"
cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: "3"
name: default
variables:
overrides:
- name: numCPUs
value: 12
- name: memoryMiB
value: 24576
- name: diskGiB
value: 100
status:
conditions:
- lastTransitionTime: "2026-05-28T02:54:24Z"
message: ""
observedGeneration: 110
reason: Available
status: "True"
type: Available
- lastTransitionTime: "2026-05-14T15:20:51Z"
message: ""
observedGeneration: 110
reason: ProbeSucceeded
status: "True"
type: RemoteConnectionProbe
- lastTransitionTime: "2026-05-25T08:30:51Z"
message: ""
observedGeneration: 110
reason: Ready
status: "True"
type: InfrastructureReady
- lastTransitionTime: "2026-05-12T12:51:36Z"
message: ""
observedGeneration: 2
reason: Initialized
status: "True"
type: ControlPlaneInitialized
- lastTransitionTime: "2026-05-28T02:54:24Z"
message: ""
observedGeneration: 110
reason: Available
status: "True"
type: ControlPlaneAvailable
- lastTransitionTime: "2026-05-25T13:19:28Z"
message: ""
observedGeneration: 110
reason: Available
status: "True"
type: WorkersAvailable
- lastTransitionTime: "2026-05-27T06:12:19Z"
message: ""
observedGeneration: 110
reason: ReconcileSucceeded
status: "True"
type: TopologyReconciled
- lastTransitionTime: "2026-05-12T12:51:31Z"
message: ""
observedGeneration: 110
reason: NotRollingOut
status: "False"
type: RollingOut
- lastTransitionTime: "2026-05-13T06:29:16Z"
message: ""
observedGeneration: 110
reason: NotRemediating
status: "False"
type: Remediating
- lastTransitionTime: "2026-05-26T07:09:22Z"
message: ""
observedGeneration: 110
reason: NotScalingDown
status: "False"
type: ScalingDown
- lastTransitionTime: "2026-05-24T22:17:55Z"
message: ""
observedGeneration: 110
reason: NotScalingUp
status: "False"
type: ScalingUp
- lastTransitionTime: "2026-05-28T02:54:24Z"
message: ""
observedGeneration: 110
reason: Ready
status: "True"
type: ControlPlaneMachinesReady
- lastTransitionTime: "2026-05-26T07:09:22Z"
message: ""
observedGeneration: 110
reason: Ready
status: "True"
type: WorkerMachinesReady
- lastTransitionTime: "2026-05-12T12:51:31Z"
message: ""
observedGeneration: 110
reason: UpToDate
status: "True"
type: ControlPlaneMachinesUpToDate
- lastTransitionTime: "2026-05-12T12:51:31Z"
message: ""
observedGeneration: 110
reason: UpToDate
status: "True"
type: WorkerMachinesUpToDate
- lastTransitionTime: "2026-05-12T12:51:31Z"
message: ""
observedGeneration: 110
reason: NotPaused
status: "False"
type: Paused
- lastTransitionTime: "2026-05-12T12:51:31Z"
message: ""
observedGeneration: 110
reason: NotDeleting
status: "False"
type: Deleting
controlPlane:
availableReplicas: 1
desiredReplicas: 1
readyReplicas: 1
replicas: 1
upToDateReplicas: 1
deprecated:
v1beta1:
conditions:
- lastTransitionTime: "2026-05-25T08:30:51Z"
status: "True"
type: Ready
- lastTransitionTime: "2026-05-12T12:51:36Z"
status: "True"
type: ControlPlaneInitialized
- lastTransitionTime: "2026-05-12T12:51:36Z"
status: "True"
type: ControlPlaneReady
- lastTransitionTime: "2026-05-25T08:30:51Z"
status: "True"
type: InfrastructureReady
- lastTransitionTime: "2026-05-27T06:12:19Z"
status: "True"
type: TopologyReconciled
initialization:
controlPlaneInitialized: true
infrastructureProvisioned: true
observedGeneration: 110
phase: Provisioned
workers:
availableReplicas: 3
desiredReplicas: 3
readyReplicas: 3
replicas: 3
upToDateReplicas: 3
10.2.5 – vmsp-platform.yaml
apiVersion: releases.vmsp.vmware.com/v1alpha1
kind: PackageDeployment
metadata:
creationTimestamp: "2026-05-12T12:51:38Z"
finalizers:
- releases.vmsp.vmware.com/finalizer
generation: 11
labels:
backup.vmsp.vmware.com/platform: "true"
backup.vmsp.vmware.com/skip-restore: "true"
component.vmsp.vmware.com/id: ae05f3e0-1b68-4a70-87c8-5a27bb70f9bc
component.vmsp.vmware.com/name: vsp
component.vmsp.vmware.com/type: vsp
component.vmsp.vmware.com/version: 9.1.0.0.25370367
packages.vcf.vmware.com/component: vsp
packages.vcf.vmware.com/name: vmsp-platform
packages.vcf.vmware.com/version: 9.1.0.0.25370367
name: vmsp-platform
namespace: vmsp-platform
resourceVersion: "17906959"
uid: ae3406ee-b815-4aa4-b89c-140aa56d7543
spec:
packages:
- name: envoyproxy-gateway
version: v1.5.0-4
- name: vmsp-global-config
version: 9.1.1865
- name: logging-operator
version: 4.11.4-6
- name: support-bundle
version: 9.1.1865
- name: reloader
version: v1.4.12
- name: cluster-api-installer
version: v1.11.1
- name: vsphere-cpi
version: 1.34.0
- name: gateway-api
version: v1.3.0-7
- name: antrea
version: 2.4.2-1
- name: coredns
version: v1.12.4
- name: vmsp-api
version: 9.1.1865
- name: trust-manager
version: v0.20.3
- name: vmsp-scripts
version: v9.1.1865
- name: seaweedfs
version: 3.96.0-6
- name: node-problem-detector
version: v0.8.25
- name: zalando-pg-operator
version: v1.14.4-2
- name: debug
version: 1.0.8
- name: etcd-defrag
version: 0.37.0
- name: flux2
version: 2.17.1-1
- name: cpa
version: 1.9.1-1
- name: argo-workflows
version: 0.45.27-5
- name: metrics-server
version: 3.13.0-1
- name: velero
version: v1.16.2-9
- name: vmsp-hooks
version: 9.1.1865
- name: kube-prometheus-stack
version: 51.10.7-13
- name: vmsp-backup
version: 9.1.1865
- name: namespace-config
version: 0.1.10
- name: kube-rbac-proxy
version: v0.20.0
- name: kyverno-policies
version: 3.5.2-26
- name: busybox
version: 1.36.2-1
- name: kyverno
version: 1.15.2-4
- name: ndc
version: v9.1.1865
- name: cluster-autoscaler
version: 9.5.1-3
- name: vsphere-csi
version: 3.6.1-3
- name: vmsp-configs
version: 9.1.1865
- name: kubernetes
version: 1.0.8-2
- name: vmsp-identity
version: v9.1.1865
- name: proxy-service
version: 9.1.1865
- name: metadata-service
version: 9.1.1865
- name: vmsp-etcd
version: 3.6.5-4
- name: vmsp-certs
version: 9.1.1865
- name: descheduler
version: v0.33.0-4
- name: vmsp-cluster
version: 9.1.1865
- name: zot
version: v2.1.11-3
- name: kube-vip
version: v1.0.2-2
- name: strimzi-kafka-operator
version: v0.48.0-1
- name: vmsp-agent
version: v9.1.1865
- name: synthetic-checker
version: v9.1.1865
- name: vmsp-operator
version: v9.1.1865
- name: cert-manager
version: v1.18.2-5
- name: argo-templates
version: v9.1.1865
- name: gatewayconfiguration
version: 9.1.1865
suspend: {}
templateSelector: {}
values:
backups:
destination: sftp
full:
enable: true
schedule: 10 0 * * MON,TUE,WED,THU,FRI,SAT,SUN
incremental:
enable: false
schedule: '*/15 * * * *'
retention:
maxAgeDays: 7
maxBackups: -1
storage:
sftp:
fingerprint: SHA256:LjIZbgd6sU5G1d4jIvimEJdRJpF4wrkT0xOGm8IJRDU
ceip: {}
cluster:
generation: 0
ha: "false"
id: ae05f3e0-1b68-4a70-87c8-5a27bb70f9bc
name: aerosphere
nodePrefix: aerosphere
type: management
vmspVMMode: bootstrap
worker:
enabled: true
machineType: management.large
minReplicas: 3
size: large
depot:
url: https://skyveil.leaha.co.uk/depot-service/content-gateway
flags:
assignNodesToVMGroup: true
pdbs: "true"
helmRepository:
certSecretRef: registry-certificate
insecure: false
ingress:
fleet:
fqdn: skyveil.leaha.co.uk
vips:
ipv4:
- 192.168.1.91
ipv6: []
instance:
fqdn: windspire.leaha.co.uk
vips:
ipv4:
- 192.168.1.92
ipv6: []
platform:
fqdn: aerosphere.leaha.co.uk
vips:
ipv4:
- 192.168.1.90
ipv6: []
logs:
loginsight:
host: mistral.leaha.co.uk
port: 9543
scheme: https
sslVerify: false
syslog:
host: ""
insecure: true
transport: tcp
type: loginsight
network:
addressFamily: ipv4
dhcp: false
dhcp6: false
dnsServers:
- 192.168.1.5
- 192.168.1.27
gateway:
type: envoy
ingress:
vips:
controlplane: 192.168.1.130
primary: ""
internal:
cidr: 198.18.0.0/15
cidrv6: ""
clusterCIDR: 198.18.0.0/16
clusterCIDRv6: ""
serviceCIDR: 198.18.128.0/17
serviceCIDRv6: ""
ipPool:
addresses:
- 192.168.1.128/28
excludedAddresses:
- 192.168.1.90
- 192.168.1.91
- 192.168.1.92
- 192.168.1.130
gateway: 192.168.1.1
prefix: "24"
ipPoolv6:
addresses: []
excludedAddresses: []
gateway: ""
prefix: ""
ntpServers:
- time.cloudflare.com
objectStorage:
filerBacking: etcd
profiles:
name: small
provider:
vsphere:
cluster: /Homelab/host/Cryosphere
clusterId: ClusterComputeResource:domain-c671572
datacenter: /Homelab
datacenterId: Datacenter:datacenter-1001
datastore: /Homelab/datastore/Triton
datastoreId: Datastore:datastore-672582
datastoreURL: ds:///vmfs/volumes/vsan:119e14b026f911f1-9185005056b9f29b/
enriched: "true"
folder: /Homelab/vm/vcf-management-services
folderId: Folder:group-v683669
host: borealis.leaha.co.uk
insecureTLS: false
network: /Homelab/network/vSphere-MGMT-1
networkId: DistributedVirtualPortgroup:dvportgroup-1069
port: "443"
resourcePool: /Homelab/host/Cryosphere/Resources/VCF Appliances
resourcePoolId: ResourcePool:resgroup-672691
server: borealis.leaha.co.uk
templateFolder: /Homelab/vm/vcf-management-services
templateFolderId: Folder:group-v683669
templateId: VirtualMachine:vm-683670
thumbprint: 1A:65:C2:2A:F1:64:75:0C:74:BF:00:6A:A4:F1:E3:40:16:7A:78:61
tlsCertificate: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUZBekNDQTJ1Z0F3SUJBZ0lKQU1kUWRtcWhSUTVXTUEwR0NTcUdTSWIzRFFFQkN3VUFNSUdiTVFzd0NRWUQKVlFRRERBSkRRVEVYTUJVR0NnbVNKb21UOGl4a0FSa1dCM1p6Y0dobGNtVXhGVEFUQmdvSmtpYUprL0lzWkFFWgpGZ1ZzYjJOaGJERUxNQWtHQTFVRUJoTUNWVk14RXpBUkJnTlZCQWdNQ2tOaGJHbG1iM0p1YVdFeEhUQWJCZ05WCkJBb01GR0p2Y21WaGJHbHpMbXhsWVdoaExtTnZMblZyTVJzd0dRWURWUVFMREJKV1RYZGhjbVVnUlc1bmFXNWwKWlhKcGJtY3dIaGNOTWpVeE1qQTFNVFUxTnpNeFdoY05NamN4TWpBMk1ETTFOek14V2pBc01SMHdHd1lEVlFRRApEQlJpYjNKbFlXeHBjeTVzWldGb1lTNWpieTUxYXpFTE1Ba0dBMVVFQmhNQ1ZWTXdnZ0dpTUEwR0NTcUdTSWIzCkRRRUJBUVVBQTRJQmp3QXdnZ0dLQW9JQmdRRGM4WmwxczVvUmZyL2h4SGVmZzlVWHFBWGR6b0UvY2pzUnpWSHkKZlEwZUt1cWdiOEdhZkE3WE83dG1FMDJCclBCelFmRUU1dzhPd1k3ekNTVEVPclErVlJ6a3FBRmdmbFA0V09WZApVZTYzY0NaaXFYRXBoN1h5b0JGK256ZklBbzF3eTZPL0Q0WEsydHBGVkQxWFo4TEdtZXgxei9Pd2FPVERSSjJsCi94U1kyMzB6ZE5iRjJveWJxL3FDVkFFSWtYMmdSRzF1ejY2dURlZlVoUThFYlpZbWUyNVFCekVya2xRWTFjVkkKZU42MlRPeWhUUEl1ZzcyVHptQk9qV1pkK0FtSzJLMHN0YnliVHpqSXRZczRjRC9LeUs1N0dJYWpGZjJNa1BiSQp3YjJIeUFMMGY4bmlLbXJqYUZZMUlLZk9sMmU1TVNDeGFuVzZNQ0FON1J4WkRpZG1OQmdCMnR3dlB0ZFI1bDFQClVtODJTYVRXNHZacXV6VTJWdkZHRnJKVXVVeHVhUEUyUVFnTENKc3NlODJtZFBoMkJGTk5NNS8yUnlTaDJ0WEYKWFpXWnp0dXBVZmdNVWdTc0xhQnAxUldvb1U5NEFiU29MSmVTbnZaK3d4Yy9aV2dZQndNU21sYjFDSmkwdGZtTgpkcld5dnJJNU9YMDM2QUlNQzRudmZOVTFmU01DQXdFQUFhT0J0ekNCdERBTEJnTlZIUThFQkFNQ0JlQXdId1lEClZSMFJCQmd3Rm9JVVltOXlaV0ZzYVhNdWJHVmhhR0V1WTI4dWRXc3dIUVlEVlIwT0JCWUVGTXNxRE0yNjlucFcKUWQ5RVZmSnQwSTYyZEtkb01COEdBMVVkSXdRWU1CYUFGRHdOTEtVMitKTGNKR01BLzBRYWFXOHlZdVgvTUVRRwpDQ3NHQVFVRkJ3RUJCRGd3TmpBMEJnZ3JCZ0VGQlFjd0FvWW9hSFIwY0hNNkx5OWliM0psWVd4cGN5NXNaV0ZvCllTNWpieTUxYXk5aFptUXZkbVZqY3k5allUQU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FZRUF3SUNPTzE2YS9rU1cKd0w5UnZyeHhldUxvWFI2TytNNUxoUm4wanZwbXo3SEROK1EydlFCSFlMYWhEQWQrVHc5dFRMRURyUUU0NWc0YQpzN1JNdDd3MlhMUEswQ3oxQWpPYUV2SkhZMGt6YzdyMVBKOXQvYzFvdTZOMTNVUklwWkZrSW9wUG5ZK2k1OVhQCnc3WDd5aGJiK21BRzdLVFRuY3BWRUxpcWgzN3VCeUhIdnBxYkhPQ1docXRvaFFvdmYvdUw1cm0wQm43ZW9UU3YKRXVJdG1OK3RnbVVyeXJ1K3liLzlaRGZ0SDRsa1JwbGprcDhhV0N1ZS8vRFZDQkZwUkdvcnJHS2tseklMRk4xSgpka3ZOaERlZUFrU3dwdVVsazAyK2tkV1UwVGNkZzdTZnpTa1FrS0NQYXlzU2llcWdQMzA3VVJOSmRiVitWdUhHCmN4TW4yZDlrc2ZLVmhWS0lJcE5RMnU3TUJYR0l3ZzV0UzEyaEJxVkVMTEYrRDg3QkpNSEovMmdOSzdFb2pQZ3YKMXZ0MHV2Tkl1LzBxYllwMitlOEJFVTI1MDIvTTU1c3NQelZqc1dBLzZ3VXYwZmZXWW1oUGVNV3JqajBtR0dMNwpJaS9sWngzelU0V043cDZvdkJIU3ZWVWxrNjFpQXJibUxad1FtTlAzYmgrOWFvSVBub2JrCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
vmGroup: Cryosphere_primary-az-vmgroup
registry:
server: registry.vmsp-platform.svc.cluster.local:5000
service:
enabled: true
nodePort: 30000
port: 5000
type: ClusterIP
storageType: pvc
security:
enforcePasswordPolicy: "true"
passwordPolicy:
complexityConstraint:
minLength: 15
minLowercase: 1
minNumeric: 1
minSpecial: 1
minUppercase: 1
passwordHistory: 5
expirationConstraint:
passwordExpirationDays: 365
lockoutConstraint:
lockoutEvaluationPeriod: 900
lockoutMaxAuthFailures: 3
lockoutPeriod: 900
policyId: default
policyInfo:
description: VSP Default Password Policy
policyName: VSP Default Password Policy
sftp:
directory: /mnt/Sol/General/Backups/VCF/
host: 192.168.1.4
port: "22"
username: leaha
ssh:
publicKey: ""
vcf:
opsMgmt:
host: ""
username: ""
zot:
config:
auth:
enable: false
status:
builtInsDigest: 84e657f5f4e5a4d87a2b37464b2f7347b12d376d21f31877aba3202e88f50fb1
conditions:
- lastTransitionTime: "2026-05-12T12:51:38Z"
message: found templates for provided package
observedGeneration: 1
reason: MatchedTemplates
status: "True"
type: TemplatesIdentified
- lastTransitionTime: "2026-05-31T20:31:46Z"
message: 'failed to create or patch helmrelease vmsp-platform.node-problem-detector:
Internal error occurred: failed calling webhook "validate.kyverno.svc-fail":
failed to call webhook: Post "https://kyverno-svc.vmsp-policies.svc:443/validate/fail?timeout=10s":
dial tcp 198.18.219.19:443: connect: connection refused'
observedGeneration: 11
reason: DeploymentFailed
status: "False"
type: Ready
observedGeneration: 11
packageDeployments:
- name: antrea
releases:
- name: antrea
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: argo-templates
releases:
- message: 'Could not determine release state: failed to retrieve last release
from storage: query: failed to query with labels: Get "https://192.168.1.130:6443/api/v1/namespaces/vmsp-platform/secrets?labelSelector=name%3Dargo-templates%2Cowner%3Dhelm":
context deadline exceeded (Client.Timeout exceeded while awaiting headers)'
name: argo-templates
namespace: vmsp-platform
ready: "False"
reason: StateError
- name: argo-workflows
releases:
- name: argo-workflows
namespace: vmsp-platform
ready: "True"
reason: InstallSucceeded
- name: cert-manager
releases:
- name: cert-manager
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: cluster-api-installer
releases:
- name: cluster-api-installer
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: cluster-autoscaler
releases:
- name: cluster-autoscaler
namespace: vmsp-platform
ready: "True"
reason: InstallSucceeded
- name: descheduler
releases:
- name: descheduler
namespace: vmsp-platform
ready: "True"
reason: InstallSucceeded
- name: envoyproxy-gateway
releases:
- name: envoyproxy-gateway
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: flux2
releases:
- name: fluxcd
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: gateway-api
releases:
- name: gateway-api
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: gatewayconfiguration
releases:
- name: gatewayconfiguration
namespace: vmsp-platform
ready: "True"
reason: InstallSucceeded
- name: kube-prometheus-stack
releases:
- name: kube-prometheus-stack
namespace: vmsp-platform
ready: "True"
reason: InstallSucceeded
- name: kube-vip
releases:
- name: kube-vip
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: kyverno
releases:
- name: kyverno
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: kyverno-policies
releases:
- name: kyverno-policies
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- message: 'Could not determine release state: failed to retrieve last release
from storage: query: failed to query with labels: Get "https://192.168.1.130:6443/api/v1/namespaces/vmsp-platform/secrets?labelSelector=name%3Dvmsp-kyverno-policies%2Cowner%3Dhelm":
context deadline exceeded (Client.Timeout exceeded while awaiting headers)'
name: vmsp-kyverno-policies
namespace: vmsp-platform
ready: "False"
reason: StateError
- name: logging-operator
releases:
- name: logging-operator
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: metadata-service
releases:
- name: metadata-service
namespace: vmsp-platform
ready: "True"
reason: InstallSucceeded
- name: metrics-server
releases:
- message: 'Could not determine release state: failed to retrieve last release
from storage: query: failed to query with labels: Get "https://192.168.1.130:6443/api/v1/namespaces/vmsp-platform/secrets?labelSelector=name%3Dmetrics-server%2Cowner%3Dhelm":
context deadline exceeded (Client.Timeout exceeded while awaiting headers)'
name: metrics-server
namespace: vmsp-platform
ready: "False"
reason: StateError
- name: ndc
releases:
- name: ndc
namespace: vmsp-platform
ready: "True"
reason: InstallSucceeded
- name: node-problem-detector
releases:
- name: node-problem-detector
namespace: vmsp-platform
ready: "True"
reason: InstallSucceeded
- name: proxy-service
releases:
- name: proxy-service
namespace: vmsp-platform
ready: "True"
reason: InstallSucceeded
- name: reloader
releases:
- name: reloader
namespace: vmsp-platform
ready: "True"
reason: InstallSucceeded
- name: seaweedfs
releases:
- name: seaweedfs
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: strimzi-kafka-operator
releases:
- name: strimzi-kafka-operator
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: support-bundle
releases:
- name: support-bundle
namespace: vmsp-platform
ready: "True"
reason: InstallSucceeded
- name: synthetic-checker
releases:
- name: hooks-server
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: synthetic-checker
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: trust-manager
releases:
- name: trust-manager
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: velero
releases:
- name: velero
namespace: vmsp-platform
ready: Unknown
reason: DependencyNotReady
- name: vmsp-agent
releases:
- message: 'Could not determine release state: failed to retrieve last release
from storage: query: failed to query with labels: Get "https://192.168.1.130:6443/api/v1/namespaces/vmsp-platform/secrets?labelSelector=name%3Dvmsp-agent%2Cowner%3Dhelm":
context deadline exceeded - error from a previous attempt: http2: client connection
lost'
name: vmsp-agent
namespace: vmsp-platform
ready: "False"
reason: StateError
- name: vmsp-api
releases:
- name: vmsp-api
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: vmsp-backup
releases:
- name: vmsp-backup
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: vmsp-certs
releases:
- name: vmsp-certs
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: vmsp-cluster
releases:
- name: vmsp-cluster-aerosphere
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: vmsp-configs
releases:
- name: vmsp-configs
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: vmsp-etcd
releases:
- name: vmsp-etcd
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: vmsp-global-config
releases:
- name: vmsp-global-config
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: vmsp-hooks
releases:
- name: vmsp-hooks
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: vmsp-identity
releases:
- message: 'Could not determine release state: unable to determine cluster state:
[failed to determine if ServiceAccount is namespace scoped: failed to get
restmapping: failed to get server groups: unexpected error when reading response
body. Please retry. Original error: context deadline exceeded (Client.Timeout
or context cancellation while reading body), failed to determine if ClusterRole
is namespace scoped: failed to get restmapping: failed to get server groups:
Get "https://192.168.1.130:6443/api?timeout=30s": context deadline exceeded
- error from a previous attempt: http2: client connection lost, failed to
determine if ClusterRole is namespace scoped: failed to get restmapping: failed
to get server groups: Get "https://192.168.1.130:6443/api?timeout=30s": context
deadline exceeded, Get "https://192.168.1.130:6443/apis/policy/v1/namespaces/vmsp-platform/poddisruptionbudgets/vmsp-identity?timeout=30s":
context deadline exceeded, an empty namespace may not be set when a resource
name is provided, Get "https://192.168.1.130:6443/apis/rbac.authorization.k8s.io/v1/clusterroles/vmsp-identity-manager-role?timeout=30s":
context deadline exceeded - error from a previous attempt: http2: client connection
lost, Get "https://192.168.1.130:6443/apis/rbac.authorization.k8s.io/v1/clusterroles/vmsp-identity-metrics-reader?timeout=30s":
net/http: request canceled (Client.Timeout exceeded while awaiting headers),
client rate limiter Wait returned an error: context deadline exceeded - error
from a previous attempt: http2: client connection lost, Get "https://192.168.1.130:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/vmsp-identity-manager-rolebinding?timeout=30s":
context deadline exceeded (Client.Timeout exceeded while awaiting headers),
Get "https://192.168.1.130:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/vmsp-identity-proxy-rolebinding?timeout=30s":
dial tcp 192.168.1.130:6443: connect: no route to host, Get "https://192.168.1.130:6443/apis/rbac.authorization.k8s.io/v1/namespaces/vmsp-platform/roles/vmsp-identity-leader-election-role?timeout=30s":
dial tcp 192.168.1.130:6443: connect: no route to host, Get "https://192.168.1.130:6443/apis/rbac.authorization.k8s.io/v1/namespaces/vmsp-platform/rolebindings/vmsp-identity-leader-election-rolebinding?timeout=30s":
dial tcp 192.168.1.130:6443: connect: no route to host, Get "https://192.168.1.130:6443/api/v1/namespaces/vmsp-platform/services/vmsp-identity-auth-service?timeout=30s":
dial tcp 192.168.1.130:6443: connect: no route to host, Get "https://192.168.1.130:6443/api/v1/namespaces/vmsp-platform/services/vmsp-identity-webhook-service?timeout=30s":
dial tcp 192.168.1.130:6443: connect: no route to host, Get "https://192.168.1.130:6443/apis/apps/v1/namespaces/vmsp-platform/deployments/vmsp-identity?timeout=30s":
dial tcp 192.168.1.130:6443: connect: no route to host, Get "https://192.168.1.130:6443/apis/identity.vmsp.vmware.com/v1alpha1/namespaces/vmsp-platform/accessgrants/fleetbuilder-grants?timeout=30s":
dial tcp 192.168.1.130:6443: connect: no route to host, Get "https://192.168.1.130:6443/apis/identity.vmsp.vmware.com/v1alpha1/namespaces/vmsp-platform/accessgrants/ops-logs-grants?timeout=30s":
dial tcp 192.168.1.130:6443: connect: no route to host]'
name: vmsp-identity
namespace: vmsp-platform
ready: "False"
reason: StateError
- name: vmsp-operator
releases:
- name: vmsp-operator
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: vmsp-scripts
releases:
- name: vmsp-scripts
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: vsphere-cpi
releases:
- name: vsphere-cpi
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: vsphere-csi
releases:
- name: vsphere-csi
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: zalando-pg-operator
releases:
- name: zalando-postgres-operator
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: zot
releases:
- name: zot-2
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
- name: zot-1
namespace: vmsp-platform
ready: "True"
reason: UpgradeSucceeded
phase: Failed