Mount this to the server, by using rufus to create a bootable USB, or by mounting it to your servers virtual CD ROM in the IPMI, iDRAC for Dell and iLO for HPE

Once the server has booted ESX, you’ll have this screen, select enter to continue

Accept the EULA with F11

It will then scan for bootable devices, for a production system this should be something in RAID 1
Examples are Dell’s BOSS card
For HPE G11 you should have the NS204i-U, or for G10 systems the NS204i-P, which is a PCIe card

As this is a lab, I have a virtual disk, and will be using the 400GB one by making sure its highlighted in Yellow and clicking Enter to Continue

Select your keyboard layout and hit Enter

Set a root password, use something easy to use, we can set a secure random one later

You may get a CPU warning depending on your hardware
Press Enter

Then click F11 to install

Once thats done, reboot the server when prompted and unmount your media

Once thats done, reboot the server when prompted and unmount your media

When the host boots, it should look like this, press F2 to login

Enter the root credentials and press enter

Scroll to Configure Management Network and press enter

Press Enter on Network Adapters and ensure that a connected Nic is selected, these should all be configured the same on the switch

In my case VMNIC0 is connected, and I will be using this for management, so I will press Escape and leave it as it is

Press enter on VLAN

And enter your ESXi management VLAN, I am using VLAN 1023
This is only needed if you have your VLANs trunked down, if your management VLAN is the native VLAN you can ignore this, as all my VLANs are trunked down, I am entering mine

On IPv4

Use the space bar to select the third option to set a static IP and add your management IP details in and press enter to Save

For IPv6, select disable on the first option, unless you are specifically using it, and press Enter

Add your DNS servers and the hostname for this server and press Enter

Add your domain under DNS Suffixes and press Enter

Now press Escape and enter Y to apply changes and reboot the host

Then, login on the WebUI at
https://fqdn
And login with the root credentials

Navigate to Host/Configure/Security & Users/Users, and click the root account and click Edit User to change the root password to something more secure

Then add the password
This needs to be 15 characters with the only allowed special characters being !@#$%^&*
Then click Save

Then we need to enable SSH, click Host/Configure/System/Services select SSH and click Start

Now we need to SSH into the host with the root credentials, using something like Putty and run the following to set the hostname/FQDN correctly for the certificate and renew it for the VCF deployment wizard

For my host, lab-vcf91-esx01, lets set the hostname with
esxcli system hostname set -H=<hostname>
So for my host this is

esxcli system hostname set -H=lab-vcf91-esx01

Then set the FQDN with

esxcli system hostname set -f=<fqdn>

Which for my host is

esxcli system hostname set -f=lab-vcf91-esx01.leaha.co.uk

Now renew the certificates with

/sbin/generate-certificates

And reboot the host with

reboot

When it comes back up, you will need to restart SSH for the cloud builder

Lastly, we need to setup NTP on all servers, you can use a windows App, DC or a docker container, for this

Head to Host/Comfigure/System/Time Configuration and click Edit under Network Time Protocol

Check the box to enable it, enter your NTP server and click ok

Then, under services, click NTP Daemon and click Edit Startup Policy

Check the radio button for Start And Stop With Host and click ok

We will need to repeat this on the remaining hosts

Before we proceed we need to check we have enough space on the first host to deploy the VCF installer appliance, your boot device will typically be 512GB or larger so there should be a good size local datastore created we can use for this as we cant use our vSAN disks

We can check this under Storage/Datastores
Mine is only 271GB but should be enough

This only seems to happen for virtual ESX hosts used in labs, when the build option tries to migrate the management vmk to a VDS it will fail

This does not need to be done on physical hardware

We need SSH and the ESX Shell, to enable this, click Host/Configure/System/Services, select ESXi Shell and click Start

Then open up the ESX console and press Alt + F1 to access the shell and login

Then run

esxcli network ip interface list | less

We should see vmk0, the management interface

We need to note the portgroup, which should be ‘Management Network’
We can press ‘q’ to exit this

Now remove the interface with

esxcli network ip interface remove –-interface-name=vmk0

Then recreate it with

esxcli network ip interface add -–interface-name=vmk0 -p "Management Network"

We can press Alt + F2 to switch back to the DCUI, press F2 and login as root

Press Enter on Configure Management Network

Press Enter on IPv4 Configuration

Use the third option to set a static IP and enter the details setup at the ESX deployment stage and press Enter

Then press Escape

And press Y here

Now back in the host UI, click Host/Configure/System/Services, select ESXi Shell and click Disable

Then repeat for the remaining hosts
 

Log into the first host and click Networking
If you set a VLAN for the management VLAN and the management components are going on the same VLAN, which I recommend, we will need to edit the VM Network to set this VLAN, click Networking, then click the three dots on the VM Network entry and click Edit Settings

Set the VLAN tag and click ok

Now click Host/Actions/Deploy OVF Template

Give the VM a name and click Upload Files

Double click the SDDC Manager OVA

Then click Next

Click Next

Accept the EULA

Select the default datastore and make sure Thin Provision is enabled then click Next

Make sure the VM Network is selected and click Next

Under Application enter a root and local user password, these need to be 15 characters with the only special characters being !@#$%^&*
For the hostname enter the FQDN and for NTP add your NTP server

For the networking section, select the IP version, likely IPv4, enter the SDDC Manager IP address, subnet mask, gateway, DNS domain and search domain path and DNS servers, comma separated, then click Next

And click Finish
Do not refresh your page while this is deploying

When thats uploaded, click Virtual Machines, right click the VM and click Power/Power On

Now we have the appliance deployed we need to download all the software binaries, log into the VCF Installer on
https://fqdn

Then click Depot Settings And Binary Management

On the Connect To The Online Depot widget, click Configure

Copy the service ID and then head to vcf.broadcom.com

Then click Software Depot Registration and click New Registration

Paste the Service ID from the VCF Installer and give the depot a name, then click Register

Copy the activation code and click Finish

Paste this into the Activation Code section in the VCF Installer and click Authenticate

At the bottom, select your release version, I am doing 9.1.0.0, and select everything apart from the SDDC Manager and click Download

When its all done, it should look like this

We can then click Return Home at the top left to get back to the main menu

As we proceed through we will need to add various appliances, ensure all are DNS registered as you go through it before you finish the deployment

On the Deploy Widget, click Deployment Wizard/VMware Cloud Foundatio

Click Next

As this is a brand new deployment from scratch we want to make sure we have Deploy A New VCF Fleet selected and click Next

Review the config and click Next when you are happy

This will kick off the validation

If anything fails it must be addressed, the only warning I got was over capacity, it assumes a 1:1 ratio on pCPU to vCPU which is a little overkill, but you are unlikely to hit this in production and my storage is a little lower, with thin provisioning it should be fine, but ensure you have enough storage on a production system

When you are happy, click Deploy

We can then watch it go through all of the stages
We can also click Review Passwords for all the generated credentials, ensure you save these

You can copy this as a JSON or CSV

When its done it will look like this and we can proceed to the VCF Operations UI

The one system we really do want HA on is the NSX manager cluster, we need to expand this with the SDDC Manager API, as of 9.0.x

Thankfully we dont need to do anything complex with this, as APIs can be very confusing if you are new, Operations has an API explorer with a nice template we can use to easily do this

Log into VCF Operations and click Build/Developer Center/APIs & SDKs then click API Explorer on the SDDC Manager API widget

Search for NSX and the bit we want to expand is the GET request for getting our clusters, this will give us our cluster ID which we need for the scale out operation

Expand the request and click Execute, you dont need to fill anything out

We can see the cluster object, you can click the link to expand it

Now we can see our single node, and we have the ID we can copy for later
In my case my ID is 8f4739c3-f002-493b-ae7c-731c1247344a

Now we need the POST request to scale out the cluster

We have two main parameters we need, the cluster ID and the body, for the cluster ID, enter your ID

We then need to add the body, the template for it is this
Remove the IPv6 entries, like mine below, if you arent using it

{
  "nsxManagerSpecs": [
    {
      "name": "",
      "networkDetailsSpec": {
        "dnsName": "",
        "gateway": "",
        "ipAddress": "",
        "ipv6Gateway": "",
        "ipv6PrefixLength": 0,
        "subnetMask": ""
      }
    },
    {
      "name": "",
      "networkDetailsSpec": {
        "dnsName": "",
        "gateway": "",
        "ipAddress": "",
        "ipv6Gateway": "",
        "ipv6PrefixLength": 0,
        "subnetMask": ""
      }
    }
  ]
}

We then need to fill our the variables like below, for each manager

• name – Hostname
• dnsName – FQDN
• ipAddress – IPv4 Addres
• gateway – Network Gateway
• subnetMask – Subnet Mask

This is what I did for my managers

{
  "nsxManagerSpecs": [
    {
      "name": "lab-vcf91-nsx02",
      "networkDetailsSpec": {
        "dnsName": "lab-vcf91-nsx02.leaha.co.uk",
        "gateway": "10.1.23.1",
        "ipAddress": "10.1.23.162",
        "subnetMask": "255.255.255.0"
      }
    },
    {
      "name": "lab-vcf91-nsx03",
      "networkDetailsSpec": {
        "dnsName": "lab-vcf91-nsx03.leaha.co.uk",
        "gateway": "10.1.23.1",
        "ipAddress": "10.1.23.163",
        "subnetMask": "255.255.255.0"
      }
    }
  ]
}

Then click Execute

We’ll see it showing as in progress now

We can view the status from Build/Tasks and then by clicking our VCF instance

Now we have the extra appliances deployed we need some Edge VMs to do the network transport
We need two DNS registered FQDNs for this

Its here that we need our Uplink VLANs for BGP
I would recommend having ToR 1 owning Uplink 1 as the BGP neighbor on this subnet, and ToR 2 owning Uplink 2
In my lab, I only have 1 OPNsense router, so it my case it will own both Uplink VLANs

In vCenter, click the vCenter its self then Networks/Transit Gateways and click Setup Network Connectivity

Select the Span to be the default, we can use spans to limit connectivity between certain vCenters within an NSX instance, but the default will set it for all, then set the Connection to Centralized Connection and click Next

Check the Select All Box, reviewing the prerequisites, and click Continue

Give the Edge cluster a name, select the Large form factor, this will be needed for the Supervisor, and click Add

Enter the node FQDN, select the cluster, optionally add a resource pool, leave host affinity on No, we can configure this later, select a datastore
Then for the management IP select IPv4 only and click Static for the assignment, enter the management IP in CIDR address, add the gateway, then add the VM Management port group we set during the deployment, if you need to check its the port group vCenter is on

For the uplinks, uncheck the box to use the host overlay network the active/standby pNICs should alternate like this and will be populated by default

Enter the Edge TEP VLAN, for the IP Type, select IPv4, select IP pool, then click the three dots and click Create New

Give it a name and click Set under Subnets

Click Add Subnet/IP Ranges

Add an IP range for TEPs, then add the network in CIDR notification, gateway, DNS servers and DNS suffix then click Add

Then click Apply

Add a description and click Save

Now click Run Check to check the VLAN MTU

Then click Apply

And repeat for the second Edge Node, we wont need a new IP pool as we can select the new one from the drop down

Once thats done it should look like this

At the bottom we can remove the toggle to set our own passwords if we want to, when you are happy, click Next

We then need a name for the gateway, enter a name for the T0 gateway, keep HA on Active/Standby, this is very difficult to change later, and Active/Active isnt supported for the supervisor with VPC, routing needs to be BGP, and we then need a local AS number, this must be unique on your network

My lab router has ASN 65535 and thats all I have
But you might want something like ToR1 on 65534 and ToR2 on 65535
I used 65532 for the edge cluster

For gateway uplinks click Set

Now we need to enter the details for the Uplink 1 VLAN
Enter the VLAN ID, interface CIDR, this is the UP the Edge will have and much be unique, gateway IP the ToR has, and enter the ASN number configured on ToR1, then click Next

Repeat for Uplink 2 and click Apply

The same config should be applied to the other Edge node
Uplink 1

Uplink 2

When its done it should look like this

We then need our VPC connectivity, we need two large subnets, I recommend /16s, these must not overlap anywhere else on your datacenter, that can be split out as needed within VPCs, these blocks should not overlap anywhere else on your network
For VPC External IP Blocks, click the three dots and click Create New

Enter a name and add the CIDR, I opted for 10.102.0.0/16 and click Save

Repeat for the transit gateway blocks clicking the three dots and click Create New

Add a name and the IP block, I opted for 10.103.0.0/16, then click Save

Then click Next

Review the config and when you are happy, click Deploy

You will need to update your BGP config on the ToRs with the addresses the Edges have on each uplink VLAN so BGP is then communicating properly

We can click View Details to see the deployment

This takes us to Configure/Networking/Edge Clusters

This K8S cluster hosting a large sum of components can be scaled if you did a simple deployment when building the VCF instance, the key difference is the K8S control plane consists of a single server, 4vCPU/10GB, and can be scaled to a HA 3 node cluster

I would recommend scaling this to HA on the control plane in a production environment

Its also worth noting, during this deployment I did notice one additional worker node, the 12vCPU/24GB VMs, get deployed so scaling this can have larger requirements than just the control plane cluster

Also, after scaling this to medium, the worker nodes were slowly replaced

I would recommend scaling this if you are planning to scale components running on this platform beyond the minimum, this includes

• Log Management
• Real Time Analytics
• Identity Broker
• Salt
• Software Depot

To begin, log into VCF Operations and head to Build/Lifecycle/VCF Management and click VCF Services Runtime

Click Actions/Scale

Select the Medium Size, unless your VCF Fleet is massive Large is very overkill, check the box to acknowledge potential service interruption and click Next

We can add additional IPs if needed, as we deployed this with an IP pool of 30, meant for the larger scales, we can just click Next as we have plenty of IPs

And click Finish

Now that we have the environment and Log Management deployed, we need to onboard our domains so they log to the Log Management cluster

Head to Operate/Administration/Integrations, expand VMware Cloud Foundation, and click the three dots on your VCF instance and click Edit

Click Domains, here we will then see all our domains, you will be on the vCenter by default, under Log Operations, click Activate Log Collection, and select the radio button to bring them directly into the cluster

On vSAN, if you have it, check the box for Enable SMART Data Collection

And for NSX do the same as vCenter and click Save

Now that we have the environment and Log Management deployed, we need to onboard our domains so they log to the VCF Operations For Networks cluster

Head to Operate/Administration/Integrations, expand VMware Cloud Foundation, and click the three dots on your VCF instance and click Edit

Click Domains, here we will then see all our domains, you will be on the vCenter by default, under Log Operations, click Activate Network And Flow Collection, select our collector, then click the check box to Enable NetFlow on the vCenter

Click Enable

Repeat for the Antrea IPFIX and check the box

And click Enable

Click NSX at the top, and check the box for Activate Network And Flow Collection, select the collector and check the box to enable IPFIX

Then check the bot for latency collection and click enable

Then click Save

This section was done on another environment due to licensing constraints, FQDNs will not match the rest of the guide

Its worth noting, vDefend licensing is no longer done by a key in NSX, it requires a different licensing server deployed by the Security Services Platform, requiring 6vCPU and 24GB RAM
It has been excluded from this section due to it being a paid add on and will be included in the vDefend configuration guide, though there is no ETA on this

To license our environment we need to log into VCF Operations and click Manage/Licensing/Licenses & Registration then click Continue

Then click Start

Click Connected and click Continue

Now click Start again under Registration

Log into the Broadcom portal and click Start
The screenshot below may automatically appear

We can use the VCF Operations name and click Save

Click Start on Generate Activation Code

Click Copy to copy the code, then click Finish

In VCF Operations click Start on Enter Activation Code

Paste the code in and click Activate

Now under Add Licenses To License Server, click Start

In the VCF portal, under Add Licenses, click Start

Select your licences and click Confirm

In VCF Operations, click Download

You should get a pop up, when thats come through, click Close

And click Finish

In the VCF Portal, click Mark As Completed

In VCF Operations, scroll down, select your vCenter and click Assign Primary License

Select the license and click Assign

Repeat click Assign Addon License/VMware vSAN for vSAN

Select the license and click Assign

Open the management domain vCenter and click the three lines in the top left, then click Global Inventory Lists

On the left, click Hosts

We first need a network pool, if you are expanding a cluster thats already been created, there will already be a pool that can be used, in that case you can skip this part, but if you are adding a workload domain a new pool will be required, for a new cluster you may or may not need a new pool
Pools must not have overlapping IP ranges
Click the Network Pools section and click Create Network Pool

This part will depend on what you are deploying storage wise, but you’ll pool for vMotion and one for your storage, for this cluster I am using vSAN, it also needs a name
Hosts will also need to be setup like in the ESX section

When you have your networks enter the VLAN, MTU, which likely is 9000, but this will match what you did in the deployment, then add the gateway in CIDR notation, its worth noting here, neither of my networks actually have a gateway, then enter an IP range for hosts
When you have the address ranges, you’ll need to click Add

It should then look like this with the network range added

We then need to repeat for vMotion and click Save

Then we need to commission a host, in vSphere click the three lines in the top left and click Global Inventory Lists

Click Hosts/Unassigned Hosts/ click Commission Host

Check and prerequisites, you will need to select all before continuing
When you are happy click Proceed

Add your host FQDN, select the storage type, select the network pool we created earlier, enter the root credentials and click Add
If your hostname is over 15 characters click Acknowledge, this can be ignored as hosts shouldnt be domain joined

Repeat for all hosts to commission, a non vSAN cluster requires at least two hosts, while vSAN requires at least three hosts, though I recommend four

At the bottom, click the toggle to confirm the fingerprint and click Validate All

Then click Validate All

Once thats validated, click Next

Then click Commission

When its done it should look like this

This will need to be done via VCF Operations, click Operate/Inventory, by default you will be on the simplified view, so click Details View

Expand VCF Instances and select your VCF instance, then click Add Workload Domain/Create New

Check the prerequisites and click Proceed

Give the workload domain a name and select Full Deployment With Cluster

We can disable the supervisor for now, it can be manually configured later like in section 7, credentials will be automatically generated and can be extracted after the domain has been built from the Operations UI, click Next

Add the vCenter FQDN, this will need to be on the same network as the management domain vCenter, enter an SSO domain fort he vCenter, the default vsphere.lcoal wil be fine then click Next
This will default to a large vCenter with 8vCPU and 30GB RAM

Give the cluster a name and click Next

Select our new cluster image and click Next

We then need to setup our NSX instance, I would always recommend HA in production

We get a couple of options, in VCF 9.1 we no longer require a dedicated NSX instance for the first workload domain and we now have the option of joining this to the management domain NSX instance

There are a few of things to note here

Firstly, you likely deployed the medium size for NSX during the deployment, this only supports two vCenters and I cant find the correct way, if possible, to scale this up

Secondly, everything in NSX will generally be available across domains, some people prefer separation, in which case we would want a dedicated NSX instance, though we can use the new Spans features to localize VPCs to particular domains

Thirdly, if you reuse a given NSX instance, the hosts will need access to the TEP VLAN the NSX is configured to use

If you are only deploying a single workload domain, this easily fits into the management domain NSX instance, so I would use this, it will save a lot of resources, if you are planning many workload domains, deploying a dedicated NSX instance here makes more sense, and then using the large size to accommodate up to 16 vCenters 

If you deploy a new NSX instance you will need a VIP FQDN plus three more, one for each node and these will need to be on the same VLAN as the management domain vCenter

I will be adding my instance to my existing NSX manager as I have a single workload domain and this is more resource efficient, if I need to isolate VPCs I can use the Spans feature

Select the Join Existing NSX Manager Instance and select the NSX instance, then click Next

We then need to choose our storage type, I have planned for vSAN ESA, so I select that and clicked Next

For vSAN, the default vSAN HCI is what you likely want, if you are unsure use this option, if you know you need a vSAN storage cluster select that and click Next

We then need to select our hosts, I am using all four I added earlier, then click Next

We then need to select our networking topology, I recommend 6 NICs using the storage and NSX traffic separation, but if you have only 4 NICs, what I would consider to be the minimum, use storage separation

Now, while we have the defaults, this doesnt actually let you edit the settings, and the issue there is the NSX TEP addresses will be using DHCP when we want an IP pool like the management domain during the deployment, so click Create Custom Switch Configuration at the bottom, and we will manually set up the topology

Click Create Distributed Switch

Add a VDS name and set the MTU, this should be 9000, but in line with what was set for the management domain, for Type, I would use VDS Uplinks, we need 2 uplinks, select the vmnics you want to use then click Configure Network Traffic Type/Management
One of these uplinks should be bound to the vSwitch on the default ESX install

Give the port group a name and select the load balancing policy of route based on physical NIC load then click Save Configuration

Click Configure Network Traffic/vMotion

Give the port group a name and select the load balancing policy of route based on physical NIC load then click Save Configuration

Scroll to the bottom and click Create Distributed Switch

Click Create Distributed Switch

Give the VDS a name, set the MTU the same, which should be 9000, for the Type I recommend VDS Uplink, select the two uplinks you want to bind for storage and click Configure Network Traffic/vSAN
If you chose NFS configure that here

Give the port group a name and set the same load balancing option then click Save Configuration

Then scroll to the bottom and click Create Distributed Switch

Click Create Distributed Switch one last time

Give the VDS a name, set the MTU to the same, at 9000, for the Type I recommend VDS Uplinks, then add the remaining uplinks, then click Configure Network Traffic/NSX

Leave the default boxes checked, all three should be, give the overlay transport zone a name, add the host TEP VLAN as the Transport VLAN, set the IP Allocation to Static IP Pool and create a new pool
Then give the pool a name, set the CIDR, IP range and gateway

Give the VLAN transport zone a name, the uplink number should match the number on the VDS, in our case, 2, and set the NSX and VDS uplinks to match

Give the NSX uplink profile a name and set the Teaming Policy to Load Balance Source and click Save Configuration

Scroll down and click Create Distributed Switch

Then click Next

Review all the info and when you are happy click Finish

Once the validation has passed click Finish

We can click View SDDC Manager Tasks on the pop up to track the progress

Ideally, you want one collector/cloud proxy per domain, this isnt a hard rule, and it can be more efficient to reuse existing cloud proxies for data collection rather than deploying more increasing resource requirements

Here we will go through deploying a new proxy, if you wanted one, and in the next sub section, setting up the workload domain to use it

NSX and vCenter appliances are deployed into the management domain, and thats the bulk of the data we are collecting, and as such, the default workflow will also deploy the collector into the management domain, on the same network, so its as close to the data sources as possible#

Log into VCF Operations and click Build/Lifecycle/VCF Management/Components and click VCF Operations

Click Actions/Add Cloud Proxy

Enter an FQDN, this must be registered in DNS, select the size, Small will be fine for most scenarios, enter the VCF Operations password and click Re-Generate

You can use the eye to see the password and save it, you wont be shown it again

Select our VCF instance and click Add

By Default, the workload domain should get integrated after about an hour or so, but lets properly configure this with our new collector, Ops For Logs and Networks

In VCF Operations click Operate/Administration/Integrations, expand VMware Cloud Foundation, then click the three dots on your VCF Instance and click Edit

Make sure System Managed Credentials is checked, select our new workload domain collector, and make sure Operational Actions is enabled, then check Activate Log Collection and click the vSAN tab

Ensure its enabled and SMART data is being collected then click the NSX tab

Ensure NSX is activated, if you have Operations For Networks, also enable that, I didnt redeploy it when I rebuilt my lab, so I left it, then click Save in the bottom left

Before we can start, we need a new Edge cluster in the workload domain to keep all its networking local

We will also need two FQDNs registering on DNS, for the network, I will be using the same network as my ESX management, in a similar style to the management domain, but you can choose any port group in the workload domain, my ESX management VLAN in the workload domain is different from that of my management domain

In the workload domain vCenter, click the vCenter object on the left, and click Configure/Networking/Edge Clusters/Add Cluster

Give the cluster a name and set the form factor to Large then click Add

Enter the node FQDN, select the cluster, optionally add a resource pool, leave host affinity on No, we can configure this later, select a datastore

Then for the management IP select IPv4 only and click Static for the assignment, enter the management IP in CIDR address, add the gateway, then select the port group you want for management, you can use the ESX management one, but I recommend a dedicated port group on for VM Management, this should be ephemeral

For the uplinks, uncheck the box to use the host overlay network the active/standby pNICs should alternate like this and will be populated by default

Enter the Edge TEP VLAN, for the IP Type, select IPv4, select IP pool, then click the three dots and click Create New

Give it a name and click Set under Subnets

Click Add Subnet/IP Ranges

Add an IP range for TEPs, then add the network in CIDR notification, gateway, DNS servers and DNS suffix then click Add

Then click Apply

Add a description and click Save

Now click Run Check to check the VLAN MTU

Then click Apply

And repeat for the second Edge Node, we wont need a new IP pool as we can select the new one from the drop down

Once thats done it should look like this

At the bottom we can remove the toggle to set our own passwords if we want to, when you are happy, click Save

When its done it should look like this

Before we can get our T0 gateway configured for external connectivity at the vCenter we need the uplink segments, and we need 4 to match the existing ones but for the uplink VLANs for our new workload domain 

Open the NSX GUI at the VIP address and log in with admin account

Head to Networking/Connectivity/Segment Connectivity/Segments and click Add Segment

Give it a name, and select the same transport zone as the other segments, the nsx-system-vlan-transport-zone, add the VLAN tag for uplink 1 and select the pre created teaming-1 Uplink Teaming Policy and click Save

Click No here

Repeat for three additional segments, 1 per edge per uplink VLAN
For the second VLAN, in my case 1047, use Teaming-2 like this

When your done it should look like this

Now we have our cluster, we need a new T0 gateway, if we reuse the gateway from our initial deployment, then all traffic leaving NSX will go via our other domain, which we dont want

To do this, open the NSX GUI, on the VIP FQDN and log in with the admin account

Click Networking/Connectivity/Tier-0 Gateways/Add Gateway/Tier-0

Enter a name, select the HA mode as Active/Standby, else NAT wont work causing issues with the Supervisor and K8S, select our new edge cluster and click Save

When you see this popup, click Yes, as we need to edit a few things

Expand Interfaces & GRE Tunnels and click Set

Click Add Interface

The first interface will be for the first edge on the first uplink VLAN, give it a name, add an IP in CIDR notation for that VLAN, in my case 1046, and select the segment for teaming-1 on that VLAN for edge01, then select the Edge Node, and click Save

We then need to repeat for all Edge nodes so that there is 1 interface per segment, one per VLAN per Edge

You should get something like this, click Close

Now for BGP, like before, this depends on how you have it setup, but we will be using the same concept as the management domain with a local AS number unique to this cluster, going over two BGP VLANs, which each of your ToR switches would own one each

This will not cover the BGP side outside of NSX

Expand BGP and set the local AS number, and by BGP neighbors on the right, click Set

Click Add BGP Neighbor

Add the BGP peer IP and remote AS number, then add the source addresses from the drop down that match the subnet and click Save

Then repeat for the other VLAN, your remote AS will likely be different for these peers, as mine is a single device for my lab, they ahev the same AS number

When you are done it should look like this, click Close

Now expand Route Redistribution and click Set

Click Add Route Redistribution

Click Set

Check all boxes and click Apply

Then click Add

Click Apply

Click Save

Then click Close Editing

Now we have an edge cluster, we can create a new transit gateway and connectivity profile to utilize at our workload domain

On the vSphere networking tab, right click Virtual private Clouds and click New Transit Gateway

Give the gateway a new name, leave the span on default, select Centralized Connection, for the HA mode, use Active Standby, else the Supervisor and K8S workloads will not work properly with NAT, for the Edge cluster, select our newly deployed cluster, and click Next

Select the radio button for Create New, then give the external connection a name, then select our new T0 gateway, then for VPC external blocks click the three dots and click Create New

Add a /16 block of IPs to be globally accessible over your datacenter and click Save

For Private Transit Gateway blocks click the three dots and click Create New

Add another /16 that doesnt overlap anywhere and click Save

Enable Default Outbound NAT and click Save

By default all VPCs land in the default transit gateway and connectivity profile, this is fine for the initial domain, but not so much for our workload domain

To create a new VPC click the networking tab in vSphere and right click Virtual private Clouds/New VPC

Give the VPC a name, and assign a private IP space, this wants to be another /16 that doesnt overlap anywhere else on the datacenter, this /16 can, be reused over all VPCs, which I typically recommend

Expand Advanced Settings and make sure for the connectivity profile we select the one for our new transit gateway and click Save

Now when we create subnets they will be attached to the correct Edge cluster with the local IP ranges and we dont end up with lots of cross domain routing

We’ll cover more on creating networking in the NSX configuration guide

We need to log into the SDDC Manager UI on
https://fqdn
We will then get redirected to login with the management domain vSphere SSO accounts

Click Inventory/Workload Domains and select the workload domain which is having its cluster expanded, I will be doing this on the vcf9-wld01 domain

Click Actions/Add Cluster

Enter a cluster name, and select the existing datacenter then click Next

Select our image, I will be using the same as my first workload domain cluster, then click Next

Select your storage type

For vSAN select the vSAN type, this will typically be vSAN HCI unless you know you need a vSAN storage/Compute cluster, then click Next

Select our hosts and click Next

We then need to select our networking topology, I recommend 6 NICs using the storage and NSX traffic separation, but if you have only 4 NICs, what I would consider to be the minimum, use storage separation

Now, while we have the defaults, this doesnt actually let you edit the settings, and the issue there is the NSX TEP addresses will be using DHCP when we want an IP pool like the management domain during the deployment, so click Create Custom Switch Configuration at the bottom, and we will manually set up the topology

Click Create Distributed Switch

Add a VDS name and set the MTU, this should be 9000, but in line with what was set for the management domain, for Type, I would use VDS Uplinks, we need 2 uplinks, select the vmnics you want to use then click Configure Network Traffic Type/Management
One of these uplinks should be bound to the vSwitch on the default ESX install

Give the port group a name and select the load balancing policy of route based on physical NIC load then click Save Configuration

Click Configure Network Traffic/vMotion

Give the port group a name and select the load balancing policy of route based on physical NIC load then click Save Configuration

Scroll to the bottom and click Create Distributed Switch

Click Create Distributed Switch

Give the VDS a name, set the MTU the same, which should be 9000, for the Type I recommend VDS Uplink, select the two uplinks you want to bind for storage and click Configure Network Traffic/vSAN
If you chose NFS configure that here

Give the port group a name and set the same load balancing option then click Save Configuration

Then scroll to the bottom and click Create Distributed Switch

Click Create Distributed Switch one last time

Give the VDS a name, set the MTU to the same, at 9000, for the Type I recommend VDS Uplinks, then add the remaining uplinks, then click Configure Network Traffic/NSX

Leave the default boxes checked, all three should be set the transport VLAN for NSX, and for IP Allocation, select Static IP Pool, then click Re-Use An Existing Pool and select the pool used for the first cluster in our workload domain

You can optionally create a new pool with a different VLAN if required

Give the VLAN transport zone a name, the uplink number should match the number on the VDS, in our case, 2, and set the NSX and VDS uplinks to match

Give the NSX uplink profile a name and set the Teaming Policy to Load Balance Source and click Save Configuration

Scroll down and click Create Distributed Switch

Then click Next

Review all the info and when you are happy click Next

Once the validation has passed click Finish

To scale VCF Automation out from a single node deployment to a three node medium cluster, in VCF Operations, click Build/Lifecycle/VCF Management/VCF Automation

Click Actions/Scale

Select the target size, Medium, check the box to acknowledge a potential service outage and click Scale

We can scale the Identity Broker out to a three node cluster from VCF Operations

To change this, click Build/Lifecycle/VCF Management/Identity Broker

Click Actions/Scale

Select the size as Medium, check the box to acknowledge a potential service interruption and click Scale

What I have added below is not an exhaustive guide of everything that can be added, for a full list, look at Veeam’s documentation on their API, various bits on information have been gathered from it, can be found here

Its also worth noting, section 1.1.1 covers the base and single request, other sections in 1.1 add additional requests, any time you make any changes in the management pack builder and want to re install the pack, you must uninstall the pack, which removes all data and will break any dashboards, and re install it
So I would recommend adding all requests off the bat, having too many metrics isnt really an issue anyway

1.1.1 – Setup And Licensing Request

In this section we will cover setting up the base and adding a single request, additional requests will be added later

1.1.1.1 – Adding The Source

Details on here were only possible due to Daniel Krieger’s guide here, so be sure to check it out

I would recommend adding an API user to VBR for this if you use the backup viewer role this can generally read most things, but some bits like licensing it cant

To start building the pack, for 9.0.x, we need to head to Developer Center/Management Pack Builder and click Create New Design

Give it a name and click Create

We then need to click Add Source

We then need a few bits

• Our collector, dont use the Ops built in collector, use the dedicated cloud proxy appliance collector if its available
• The VBR server FQDN
• The port, which for VBR, the API is accessible on 9419
• TLS needs to be no verify
• We need Custom authentication type here
• And add a variable called password, and enter the password for the user this pack will use, in my case, a dedicated API user with administration permissions – some values cant be read with read only

Make sure you then enable the toggle for Use Session Authentication and click Next

Now we need the HTTP method, this needs to be POST, and add an API path of
api/oauth2/token

Now expand the Advanced Settings, we’ll need a few things here

For the header, we need to set Content-Type needs to be
application/x-www-form-urlencoded

We need to add a new header called, x-api-version, and set its value to
1.3-rev1

And a third header called, accept, with a value of
application/json

Lastly we need a body which should look like this
username=<username>&password=${authentication.credentials.password}&grant_type=password&refresh_token=&code=&use_short_term_refresh=&vbr_token=

As I am using the Windows user, api, I changed the username to api like this
username=api&password=${authentication.credentials.password}&grant_type=password&refresh_token=&code=&use_short_term_refresh=&vbr_token=

Back at the top we can now click Make Session

And with a bit of magic, we should see the test response with a new bearer token and it marked as a success, when thats done, click Next

For global Settings, we need to change the Content-Type to the same as we has previously, which is
application/x-www-form-urlencoded
We also need to add a header called x-api-version, and the value needs to be
1.3-rev1
Lastly, click Add Variable in the Session Fields part at the top, the default value will have our access token pre populated
Then click Next

We then need to test the release, for this, we need to change the HTTP method to POST, and the API path is
apt/oauth2/logout

Now expand Advanced Settings and Add another HTTP header called Authorization with the value
Bearer ${authentication.session.access_token}

Now click Release Session to test, this should say success, then click Next

We can now test the API, so to start with lets get our VBR license details, keep the HTTP method on GET and use this API path
api/v1/license
If you get error 403 this is usually caused by not having permissions to view what the API is requesting

We need to expand Advanced Settings and add the same header from before, called Authorization, with the value
Bearer ${authentication.session.access_token}

When we click Test Connection, we should see our licensing information in the output, when you are happy, click Save Source

1.1.1.2 – Requests

Now we have the source configured, click the red exclamation mark on Requests

Click Add Request

Lets start with one for our license
For the API path fill in
api/v1/license
And give the request a name

Then, under Advanced Settings we need to add a header called Authorization again with the value
Bearer ${authentication.session.access_token}

The click Make Request

This should run, and give us the same results as our test, click Next

On the Pagination section, we can configure this, but I just disabled it, I am not 100% if/where its needed
When you are happy, click Save Request

We can add additional requests, but for now I am leaving this blank to come back to later for more data

1.1.1.3 – Objects

Click the red exclamation mark on Objects

Click Add new Object

Give the object a name, I called this License Spread, you can change the icon which I did, then click Next

How we need to define this object attributes is going to depend on what you are requesting, in my case, I am doing licensing spread, so I want everything on instanceLicenseSummary.objects*
When you are happy, click Next

I left this on default, then click Next

I only had Type available here, so I selected those and clicked Save Object

We then want another for our licensing info, since we cannot overlap both on the same object click Add New

Give this a different name, I went for License Info, we can also change the icon if needed, and click Next

This time we can add several of the metrics at the bottom like so and click Next

Click Next here

Here I added the instance name as the Support ID, as this should be unique if you have multiple VBR servers, this may need tweaking, and for the Object Identifier add everything and click Save Object

1.1.1.4 – Verify And Install

I am skipping over events, for now I want to get the data into VCF Ops and create a dashboard, so lets click Verify and click Perform Collection

We can now see thats run successfully and is pulling some data in, we can click the number links to see the data
When you are happy, click Install

Then click Review Configuration

We set the Port to 9419 and the SSL Configuration to No verify then click Save Configuration
Its worth noting, if you add other requests this will need setting each time you try to install it

Then click Install

Once thats done we can click View Management Pack

1.1.1.5 – Adding Our VBR Server

Now the pack is built and setup, we need to connect our VBR instance

Click our new Veeam type

Give it a name and enter the FQDN of the VBR server

Click the + icon and add the credential for accessing the API, in this case, given we defined the API user in our management pack, we need the password for this account, then click ok

Then select our collector, and click Validate Connection

This should be successful, click ok

Then click Add at the bottom

We will then need to click the three dots on our new object and click Start Collecting

Then give it ~15 mins for data to come in

1.1.2 – Adding More Requests

To get back to the menu to add more requests, click Developer Center/Management Pack Builder then click our Veeam design

Then click Requests on the top timeline bar, then click Add New

1.1.3 – Get Job States

Make sure the HTTP Method is GET, the API path is
api/v1/jobs

Add a name for the request

We need the variable Authorization into the header with this value adding
Bearer ${authentication.session.access_token}

Then click Make Request

This will return an output and report as successful, then click Next

Disable pagination and click Save Request here

Click Objects in the timeline bar at the top and click Add New

Give it a name and optionally change the default Icon and click Next in the bottom right

Then expand the Get Job States request, if you chose a different name, yours will be different

And select the below, then click Next

Make sure each label has the correct datatype, this will control if its a property or metric in most cases, here is what I chose, then click Next

We need some identifiers, for the Object Instance Name, set this to Name, and for the Object Identifiers, use ID and Name then click Save Object

1.1.4 – Repository States

Make sure the HTTP Method is GET, the API path is
api/v1/backupInfrastructure/repositories/states

Add a name for the request

We need the variable Authorization into the header with this value adding
Bearer ${authentication.session.access_token}

Then click Make Request

When thats run ok and returns data, click Next

Disable pagination and click Save Request here

Click Objects in the timeline bar at the top and click Add New

Give it a name and optionally change the default Icon and click Next in the bottom right

Then expand the Get All Repository States request, if you chose a different name, yours will be different
And select the below, then click Next

Make sure each label has the correct datatype, this will control if its a property or metric in most cases, here is what I chose, then click Next

We need some identifiers, for the Object Instance Name, set this to Name, and for the Object Identifiers, use ID and Name then click Save Object

1.1.5 – Backup Server Info

Make sure the HTTP Method is GET, the API path is
api/v1/serverInfo

Add a name for the request

We need the variable Authorization into the header with this value adding
Bearer ${authentication.session.access_token}

Then click Make Request

When thats returned data, click Save Request

Click Objects in the timeline bar at the top and click Add New

Give it a name and optionally change the default Icon and click Next in the bottom right

Then expand the Get Backup Server Info request, if you chose a different name, yours will be different
And select the below, then click Next

Make sure each label has the correct datatype, this will control if its a property or metric in most cases, here is what I chose, then click Next

We need some identifiers, for the Object Instance Name, set this to Name, and for the Object Identifiers, use Name then click Save Object

1.1.6 – Config Backup Status

Make sure the HTTP Method is GET, the API path is
api/v1/configBackup

Add a name for the request

We need the variable Authorization into the header with this value adding
Bearer ${authentication.session.access_token}

Then click Make Request

When thats run ok and returns data, click Next

Disable pagination and click Save Request here

Click Objects in the timeline bar at the top and click Add New

Give it a name and optionally change the default Icon and click Next in the bottom right

Then expand the Config Backup Status request, if you chose a different name, yours will be different
And select the below, then click Next

We also want to expand Get Backup Server Info and select name, this will give us a name identifier for later

Make sure each label has the correct datatype, this will control if its a property or metric in most cases, here is what I chose, then click Next

We need some identifiers, for the Object Instance Name, set this to Name, and for the Object Identifiers, use Name then click Save Object

1.1.7 – Get All Restore Points

Make sure the HTTP Method is GET, the API path is
api/v1/backupObjects

Add a name for the request

We need the variable Authorization into the header with this value adding
Bearer ${authentication.session.access_token}

We also need a Query parameter to limit these for our platform, our options are

“VMware” “HyperV” “CloudDirector” “WindowsPhysical” “LinuxPhysical” “Tape” “CustomPlatform” “EntraID” “UnstructuredData” “Test” “MongoDb” “AWSEC2” “AzureCompute” “GCE”

I am doing this for VMware, so I added the parameter name platformNameFilter with the value VMware

Then click Make Request

When thats successful click Next

Disable pagination and click Save Request here

Click Objects in the timeline bar at the top and click Add New

Give it a name and optionally change the default Icon

Then expand the Get All VMware Restore Points request, if you chose a different name, yours will be different
And select the below, then click Next

Make sure each label has the correct datatype, this will control if its a property or metric in most cases, here is what I chose, then click Next

We need some identifiers, for the Object Instance Name, set this to Name, and for the Object Identifiers, use ID and Name then click Save Object

I repeated again, but using the filter UnstructuredData, for my NAS backups for a new request

We then need another object, give it a name and click Next in the bottom right

Then expand the Get All File Restore Points request, if you chose a different name, yours will be different
And select the below, then click Next

Make sure each label has the correct datatype, this will control if its a property or metric in most cases, here is what I chose, then click Next

We need some identifiers, for the Object Instance Name, set this to Name, and for the Object Identifiers, use ID and Name then click Save Object

You can repeat for other types like physical hosts

What you can also do, as this for vSphere VMs, as they exist within the VCF Operations Inventory is define a new Relationship to match the Veeam VM entries to VMs, click Relationships/Add New

On the right hand side enable the toggle to use All Objects, then add vCenter Virtual Machine, and for the property look for Configuration|Name and click that, at the bottom give the relationship a name

Then click Add

For the Child Object, look for the VMware Backup Restore Points object and for the property click Name

Then click Add

Now click Save Relationship

While the above was tested and working properly in VCF Operations 9.0.x, I did notice it had broken after upgrading to Ops 9.1, when checking back in on the management pack, now located in Build/Developer Center/Management Pack Builder, the Objects section has a vCenter Virtual Machine option, not sure if I missed it before

If you click into that

Then click Next

From the Get All VMware Restore Points API request the below, and click Next

Leave this at the default and click Next

Then set the Object Binding to be the name from the API request on the left and the Configuration|Name for the vCenter VM on the right, and click Save Object

This will fix the issue and the relation works as intended, it is work noting, if you verify the management pack and go to the Install section, you now have the option to upgrade the management pack in Ops 9.1

1.1.8 – Get All Managed Servers

Make sure the HTTP Method is GET, the API path is
api/v1/backupInfrastructure/managedServers

Add a name for the request

We need the variable Authorization into the header with this value adding
Bearer ${authentication.session.access_token}

Then click Make Request

When thats run ok and returns data, click Next

Disable pagination and click Save Request here

Click Objects in the timeline bar at the top and click Add New

Give it a name and optionally change the default Icon and click Next in the bottom right

Then expand the Get All Managed Servers request, if you chose a different name, yours will be different
And select the below, then click Next

Make sure each label has the correct datatype, this will control if its a property or metric in most cases, here is what I chose, then click Next

We need some identifiers, for the Object Instance Name, set this to Name, and for the Object Identifiers, use ID and Name then click Save Object

1.1.9 – Get Email Settings

Make sure the HTTP Method is GET, the API path is
api/v1/generalOptions/emailSettings

Add a name for the request

We need the variable Authorization into the header with this value adding
Bearer ${authentication.session.access_token}

Then click Make Request

When thats run ok and returns data, click Next

Click Objects in the timeline bar at the top and click Add New

Give it a name and optionally change the default Icon and click Next in the bottom right

Then expand the Get Email Settings request, if you chose a different name, yours will be different
And select the below, then click Next

Also expand Get Backup Server Info and select name as this will give us an identifier we need later

Make sure each label has the correct datatype, this will control if its a property or metric in most cases, here is what I chose, then click Next

We need some identifiers, for the Object Instance Name, set this to Name, and for the Object Identifiers, use Name then click Save Object

1.1.10 – Get All Proxies

Make sure the HTTP Method is GET, the API path is
api/v1/backupInfrastructure/proxies

Add a name for the request

We need the variable Authorization into the header with this value adding
Bearer ${authentication.session.access_token}

Then click Make Request

When thats run ok and returns data, click Next

Disable pagination and click Save Request here

Click Objects in the timeline bar at the top and click Add New and click Next in the bottom right

Give it a name and optionally change the default Icon

Then expand the Get All Proxies request, if you chose a different name, yours will be different
And select the below, then click Next

Make sure each label has the correct datatype, this will control if its a property or metric in most cases, here is what I chose, then click Next

We need some identifiers, for the Object Instance Name, set this to Name, and for the Object Identifiers, use ID then click Save Object

1.1.11 – Get Security Best Practices Analyzer Results

Make sure the HTTP Method is GET, the API path is
api/v1/securityAnalyzer/bestPractices

Add a name for the request

We need the variable Authorization into the header with this value adding
Bearer ${authentication.session.access_token}

Then click Make Request

When thats run ok and returns data, click Next

Disable pagination and click Save Request here

Now this gets us the results, but not when it ran, create another request, we need the same HTTP header with our Bearer token, but the API path should be
api/v1/securityAnalyzer/lastRun

Then click Make Request

Here we can see its run time was, for me, yesterday at 2200, click Save Request

Click Objects in the timeline bar at the top and click Add New

Give it a name and optionally change the default Icon and click Next in the bottom right

Then expand the Get Security Analyzer Results request, if you chose a different name, yours will be different
And select the below, then click Next

Make sure each label has the correct datatype, this will control if its a property or metric in most cases, here is what I chose, then click Next

We need some identifiers, for the Object Instance Name, set this to Best Practice, and for the Object Identifiers, use ID then click Save Object

We cant combine the last scan run time we got in our other request, into the one with the results, but we can create another object so the data is in Operations

Click Objects in the timeline bar at the top and click Add New

Give it a name and optionally change the default Icon and click Next in the bottom right

Then expand the Get Security Analyzer Last Run request, if you chose a different name, yours will be different
And select the below, then click Next

Make sure each label has the correct datatype, this will control if its a property or metric in most cases, here is what I chose, then click Next

We need some identifiers, for the Object Instance Name, set this to Name, and for the Object Identifiers, use ID then click Save Object

This dashboard does assume you have a single VBR instance, if you have multiple added some parts may show data differently, or need additional filtering

Here we will create the widgets to form this dashboard, the widgets are in no order, and can be re arranged in any way you like

The other thing that will be created is a widget for my VM dashboard which shows the restore points of a selected VM, if its backed up by Veeam, if its not, its empty

2.1.1 – Creating The Dashboard And The Licensing Widget

We first need to create a new dashboard, in VCF Operations click Infrastructure Operations then click Dashboards & Reports then click Create

Give it a Name and click Save

Then click Actions/Edit Dashboard

We now need to drag a Score Board onto the cavas

Click the pencil to edit it

Give it a name, then ensure Refresh Content and Self Provider are set to On, Box Columns is set to 4, visual theme you can change, I opted for Pastel
Then click Input Data at the bottom

Click the + icon

Search for License, and we should see the License Info name from the License Info Object Type, click that and expand Metrics and Properties and click all of the ones shown below

Click the Plus again

Then click Virtual Machines and click Metrics|Used Instance Number

And click the Plus one more time

And repeat for File Shares, if you have those, then click ok

Double click the first object, give the Box Label a name, set the Color Method to None and click Update

Repeat for the rest, you can drag them into any order, when you are happy, click Save

And when you are done you should get something like this

2.1.2 – Repository Capacity

We first need a Super Metric, head to Infrastructure Operations/Configurations and click Super Metrics

Then click Add

Give it a name and click Next

Search for repository states and click it

Then click Next

Now we need our formula, the Veeam API doesnt return a % free on the metric list, but we have the total capacity and the used capacity so we can calculate the used %

Make sure the toggle is set for unformatted text, then enter this in the box

100 - (${this, metric=free_gb} / ${this, metric=capacity_gb} * 100) 

Then click the unformatted toggle to reformat it, we can also click Preview to see what it will look like if you need, then add the % unit at the bottom, when you are happy, click Next

Select the default policy, if you miss this, the super metric wont be created
And click Create

This metric will take 5-10 mins to show through when creating the dashboard

We can then head back to our Dashboard editing

For a nice widget for our repository free capacity, drag a Top-N onto the canvas

Click the pencil to edit it

Give it a name, set Refresh Content and Self provider to On, then set the Bars Count to 10 or 15, Round Decimals to 2, Filter Old Metrics to On, the Top-N Options should Metric Analysis and select Top Highest Utilization
Then click Input Data

Make sure this is set to all and click Output Data

Click the +

Use the drop down to set our adapter to the Veeam adapter then click the Repository States object and click ok

Then we need to select our metric, find and double click our new super metric
If its not there, give it 10-15 mins, the metric needs to be populated on a data collection cycle

Add a label, ensure the unit is %, and add a custom Color Method, I went for 80/88/92% for my thresholds but you can change this, then click Additional Columns

Click the +

Use the adapter drop down to select our Veeam adapter, then search for and click Repository States, on the right, expand properties and click Type, then click ok

Now click Save

When you are done, you will end up with a widget like this

2.1.3 – VBR Info

Drag a scoreboard onto the canvas

Click the pencil to edit it

Give it a name, then set the following

• Refresh Content to On
• Self Provider to On
• Box Columns to 3
• Visual Theme to Pastel

Then click Output Data

Click the +

Change the Adapter Type to Veeam and click Backup Server Info, then click properties/Build Version, Name and Platform, then click ok

Click the + again

Set the Adapter Type to Veeam like before, then click Config Backup Status, then click Metrics/Encryption Is Enabled, Properties/ Is Enabled and Last Successful Backup Last Successful Time, then click ok

We can double click an object under the Box label to edit its display name and colour method, when its set, click Update

You can also drag them around to change the order, this is what I ended up with, when you are happy click Save

And you should end up with something that looks a little like this

2.1.4 – Security Analyzer Results

Drag a Distribution View onto the canvas

Then click the Pencil

Give it a name and set Refresh Content and Self Provider to On, then click Input Data

Click the +

Expand Veeam/Veeam World and click Veeam World then click ok

Then click Output Data

Click the + to create a new View

Give it a name, set the visualization to Donut Chart, set the Distribution Type to Discrete Distribution and click Next

For the Subject, search for and add, Veeam/Security Analyzer Results

Click Security Analyzer Results on the Selected Subject, then drag Properties/Status over to the middle section like so and select the Preview Source as Veeam World, we should see the chart populate

Now also drag over Note then click Create

Then hit Save

The widget will then look like this

If you click any section of the donut, we get extra information about the status, and here we can see the best practice and note, as part of the additional information we added, I made the widget longer and made the details box on the right take up about two thirds, when its done you’ll get something like this

And if we need to close the extra information, we get this

2.1.5 – Job Results

For this widget we will be configuring a pair, ideally I wanted a bar chart of the last 7 days with the number of failures and successes like in Veeam ONE, you can also configure that in Grafana so its a shame I couldnt find a way to do this in VCF Operations

This will have two widgets, failures in red, nice and clear so you can see if anything needs immediate attention, and another for successes in green

Drag a Distribution View onto the canvas

Click on the pencil to edit the widget

Give it a name, set Refresh Content to On and Self Provider to On and click Input Data

Click the +

Click Veeam/Veeam World and click ok

Then click Output Data

Click the +

Click Distribution

Give it a name, set the visualization to Donut Chart, set the colour to green, set the Distribution type to Discrete Distribution and click Next

For the subject, use the drop down to select Veeam/Backup Job States

Click Backup Job States, drag Properties/Last Result over to the middle box, if we wait or refresh the data we can see this is shown, and you can see the colour issue as failed jobs are populated, ensure the Preview Source int he top right is Veeam World, I have an entry for none, this is caused by a job in progress and will clear
Click Next

Click Next again

For the filter, on the Backup Job State Filter, use Property, Last Result, Current, Is and Success, this only includes succeeded jobs, then click Create

Then click Save

The widget will look like this, the only odd thing is the colour on the preview is correct, but on the actual widget, it uses the default, which is a pain

We can repeat again with another Distribution View, when you are back on Output Data, click the + again

Give it a name, set the Visualization to Donut Chart, and select the red colour, hopefully the bug with the colour not showing will be fixed in a later release, set the Distribution type to Discrete Distribution and click Next

Add a subject and from the drop down select Veeam/Backup Job States

Click Backup Job States as the selected subject, then drag Properties/Last Result to the middle section and set the Preview Source as Veeam World and the preview should look like this, then click Next

Click Next again

On the Backup Job State Filter, enter Properties, Last Result, Current, Is and Failed then click Create

Then click Save

The widget will then look like this

You can also click on the donut for more info, you get info on that

So if I click the failed donut, I can see what job failed

2.1.6 – Backup Job Stats

2.1.6.1 – Super Metrics

Before we start we will need a few super metrics as Veeam reports the data values in B, when I want GB, you can also add an extra /1024 to all super metric definitions for TB, but as my data values arent that large I didnt

To add a super metric, click Configurations/Super Metrics

Then click Add

We first need one for the read data, called Veeam – Session Read Size, add that title and click Next

We then need an Object Type, for this click Veeam/Backup Job States

Then click Next

Enable the toggle to change the data to unformatted

Then enter this value

${this, metric=session_progress_read_size}/1024/1024/1024

Then click the toggle again to format it

Select the unit to be GB, or TB if you opted for that, and click Next

Select the default policy and click Create

We then need to add another called Veeam – Transferred Size

With the same Object Type

Using this unformatted code for

${this, metric=session_progress_transferred_size}/1024/1024/1024

Like this, when re formatted, dont forget to add the Unit at the bottom of GB and click Next

And defining it in our default policy and clicking Create

We then need one more called Veeam – Processed Size

With the same Object Type

Using the following unformatted code

${this, metric=session_progress_processed_size}/1024/1024/1024

When formatted will look like this

2.1.6.2 – Object List

Drag an Object List onto the canvas

Then click the pencil icon to edit it

Give it a name and click Output Filter

Expand Object Types, search for and click Backup Job States then Additional Columns

Click the +

Change the Adapter Type to Veeam, select Backup Job States and click Properties/Type, then click ok

Now click Save

We should then end up with a list of all our jobs gathered by the adapter

2.1.6.3 – Job Data Metrics

Now we have the object list we can create some widgets that use it as input giving us a job selector and info on the dashboard, drag a Metric Chart onto the canvas

Click the pencil to edit it

We’ll create a pair of widgets like this, so lets start with data read from the backup job, give it a name, like Read Data, set Refresh Content to On, leave Self Provider as Off and click Output Data

Click the +

Set the Adapter Type to Veeam, then click Backup Job States/Super Metrics and click on the one we create for the Veeam Session Read GB, then click ok

Now hit Save

Now to use the selected job from our job list as the input for this widget, click Show Interactions at the top of the dashboard configuration page

Drag the output from the job list into the new widget’s input like so

Then click Hide Interactions

Now whenever you click on a job for this dashboard, the Read Data is pulled for that job

We can do another for transferred data, drag another Metric Chart onto the canvas

Click the pencil to edit it

Use the same settings on the configuration giving it a useful name, then click Output Data

Click the +

We are then going to do the same and change the Adapter Type to Veeam, click Backup Job States/Super Metrics and our other super metric we created, then click ok

And click Save

At the top click Show Interactions

And drag the output from the Veeam Job List onto the new widget, mine is called Transferred Data

Then click Hide Interactions

It should look like this

2.1.6.4 – Job Stats

Lastly we need a widget for the jobs info for some of the remaining metrics, drag a Scoreboard onto the canvas

Click the pencil to edit it

We then need to configure a few bits

• Add a widget name
• Set Refresh Content to On
• Box Columns should be set to 1
• Visual Theme should be set to Pastel

Then click Output Data

Click the +

Search for and click Backup Job States, then expand Properties, from there click Next Run, Repository Name, Session progress Bottleneck and Session Progress Duration and click ok

Double click the Box Label and set a label, change Color Method to None and click Update, then repeat on the rest

I ended up with something like this, then click Save

Now at the top, click Show Integrations

And connect the output from the Veeam Job List to the input on the Job Stats widget

Then click Hide Integrations

Now when we select a job we can see additional information about it
The only thing of note, is the next run doesnt get a value if the job runs only after another

2.1.7 – Managed Server Status

We can define a new view to show what the status of our managed server is, available or not, drag a new Distribution View onto the canvas

Give it a name and set both Refresh Content and Self Provider to On, then click Input Data

Click the +

Then click Veeam/Veeam World/Veeam World and click ok

Then click Output Data

Click the +

Give the view a name, set the Visualization as Donut Chart, set the Distribution Type to Discrete Distribution and click Next

Use the Add Subject drop down to click Veeam/All Managed Servers

On the Selected Subject, click All Managed Servers, then drag Properties/Status over to the middle column and change the Preview Source on the right to Veeam World, we will then see a visual representation of our server status, click Create

Then click Save

The widget should look like this

2.1.8 – Managed Server Info

For some information on our managed servers, drag a List View onto the canvas

Click the pencil to edit it

Give it a name, set Refresh Content and Self Provider to On and click Input Data

Click the +

Click Veeam/Veeam World/Veeam World and click ok

Then click Output Data

Now click the + to create a new view

Give it a name and click Next

On the Add Subject drop down, select Veeam/All Managed Servers

Then click All managed Servers for the Selected Subject, and from the left, drag Type, Is VBR Linux Appliance, Is Backup Server and Credential Storage into the middle box, you can re arrange them in the order you want

When you select a data point in the middle you can change the label on the right, eg I changed Is VBR Linux Appliance to Is VIA, Veeam Infrastructure Appliance

On the far right, if you change the Review Source to Veeam World, we can see the data, there is however, an entry for Veeam World, which we can leave for now, click Next at the bottom

Click Next here

If we look at the Veeam World Filter, and use Properties, VCF Operations Generated Properties/Object Type Classification, Current and Empty, and we refresh the preview, the Veeam World entry is now removed, click Create at the bottom

Then click Save

And we should get a widget that looks like this

2.1.9 – Proxy Info

Drag a List View onto the canvas

Click the pencil to edit it

Give it a name, set Refresh Content and Self Provider to On and click Input Data

Click the +

Click Veeam/Veeam World/Veeam World and click ok

Then click Output Data

Now click the + to create a new view

Give it a name and click Next

On the Add Subject drop down, select Veeam/All Proxies

For the Selected Subject, click All Proxies, then drag Metrics/Server Max Tasks Count and Properties/Type onto the middle section, we can drag to re arrange their order, then if we select an object from the middle area, on the right we can edit the label if needed

On the far right, change the Preview Source to Veeam World and we can see our populated values, we can ignore the Veeam World entry for now and click Next at the bottom

Click Next here

If we look at the Veeam World Filter, and use Properties, VCF Operations Generated Properties/Object Type Classification, Current and Empty, and we refresh the preview, the Veeam World entry is now removed, click Create at the bottom

Then click Save

Our widget should then look like this

2.1.10 – SMTP Info

For a status widget to show the SMTP configuration drag a Scoreboard onto the canvas, click the pencil to edit the widget

Give it a name, then set Refresh Content and Self provider to On, Box Columns to 1 and Visual theme to Pastel then click Output Data

Click the +

Change the Adapter Type drop down to Veeam, then click Email settings and under Properties click Email Settings From, Email Settings To, Notifications Enabled, SMTP Server Name and click ok

Double click the Box Label to change the name and set the Color Method to None and click Update

You can change the order by dragging them around, when you are happy click Save

And you should get something like this

2.1.11 – Restore Points

I added this widget to my VM dashboard, which will have a configuration guide with the main VCF Operations 9 article, but thats not been fully finished as of April 2026

Much like the job list, we have a list of all VMs in vSphere, as we have the relationship in section 1, the below widget can show the Veeam restore points for a selected vSphere VM, if the VM isnt backed up by Veeam, no data is given

Drag a Metric Chart onto the canvas

Click the pencil to edit it

Give it a name and set the Refresh Content to On, Self Provider should be Off, then click Input Transformation

Click Children, the default Depth of 1 will be fine, then click Output Data

Click the +

At the top, change the Adapter Type to Veeam, then click VMware Backup Restore Points and on the left click Metrics/Restore Points Count and click ok

Then click Save

At the top click, Show Interactions

Connect the output from the Powered On VMs list over to the Restore Points widget

Then click Hide Interactions

Now if we select a VM that Veeam is backing up, we can see the number of restore points, if the VM isnt being backed up by Veeam no data is displayed

The dashboard looks something like this

What widgets you have here will depend entirely on what you are bringing in metric wise

For me, this will be based on my Home Assistant management pack bringing data in as a general idea of a dashboard you could create

2.2.1 – Creating The Dashboard

Before we can add any widgets, we need to create the dashboard, to do this, open VCF Operations and click Infrastructure Operations/Dashboard And Reports then click Create

Give it a name and click Save

Now we have the dashboard, click Actions/Edit Dashboard and we can begin adding Widgets

2.2.2 – Smart Plug Power Monitoring

This section will walk you through creating a metric chart for power data being received from my smart plug, I have 5 of these, so we will go through the first and you can create this for any other devices like this you may have

Drag a Metric Chart onto the canvas

Click the pencil to edit the widget

Give the widget a name, set Refresh Content and Self Provider to On and click Output Data

Click the +

Change the Adapter Type to Home Assistant, and find the entry for the endpoint you want to monitor, in my case, the Server Stack Current Consumption, and my power data is under Metrics/State, then click ok

Double click the Unit and add w for watts, then click Update

Then click Save

And you should see a widget like this

One thing I did, was take the three servers making up a cluster, and rather than adding just a single output, the state object, I added the ones for those servers, all into one metric chart

What you can then do is click the eye on the widget and click the first icon on the left, as shown, to combine them into a single chart

You can use the little tab at the bottom to extend it

And if you click any of the entries, they are removed from the chart

I created these widgets for my power stats

2.2.3 – Octopus Power Rates

In this section we will have a look through creating some metric charts for power rates in Octopus

How you put this into your setup will depend on what data you are pulling in

Drag a Metric Chart onto the canvas

Click the pencil to edit the widget

Give the widget a name, set Refresh Content and Self Provider to On and click Output Data

Click the +

Change the Adapter Type to Home Assistant, find the entry for the current rate in Octopus, my rate data is in Metrics/State, click ok

Click Save

Now, this value only changes once per day, click the eye in the top right of the widget, then click the calendar icon, third in on the right, then set the range to Last 30 Days, then click Go

We can hide the bar with the Eye icon, and we can see a fixed 30 days of the rate

We can also use the far left icon to disable the Split Charts option to make it look like this

I then created this set of three widgets

2.2.4 – Octopus Electric Properties

This widget will focus on giving us a properties box with data like my electric rate for tomorrow, tariff type

Add a Scoreboard onto the canvas

Click the pencil to edit the widget

Give the widget a name, then set the following and click Output Data

• Visual Theme – Pastel
• Refresh Content – On
• Self Provider – On
• Box Columns – 2

Click the +

Set the Adapter Type to Home Assistant, I added my Octopus Tomorrows Rate/Metrics/State and clicked ok

I added more properties under Octopus Current Rate/ Properties and selected the Attributes Tariff option

Double click the entry and set the Color Method to None, you could optionally set one which I did for my rate tomorrow, then click Update

Then click Save

The widget should look like this

First shut down the VEM VM and take a snapshot, then power it back up

We then need to download the VBR ISO which we can find here, and its this link we need

Double click to mount the ISO

And double click the Setup.exe

Click Upgrade

Click ok here for VBR to install the required .NET files

Click Upgrade Veeam Backup Enterprise Manager

Accept the EULA

Your license should be automatically populated, then click Next

Wait for the configuration check to pass

Once thats run, click Upgrade

When its done click Finsh

And then Yes to reboot

To upgrade the vCenter plugin, log into the vEM web UI and click Configuration on the right

Click vCenter Servers, then select your install vCenter, like mine, Borealis, and click Remove

Click Yes

You should see this banner in vCenter

Then in VEM, click Install on the vCenter

And in vCenter you should see this banner

First we need to download the patch for LCM so we can get VCF Operations upgraded to version 9, you can find this here

And download the patch

We then need to upload the patch to Aria LCM
Connect to it with the root account using WinSCP

Double click the bar here

And navigate to /data and click ok

On the right panel head to where ever you have the patch downloaded and drag it over to the root of /data
It should look like this, the folders in /data may be different, the important thing is that the upgrade file is under /data

Head to the Aria LCM WebUI on
https://fqdn
And log in with the vcfadmin@local credentials

We will want to ensure we have a valid vCenter SSO admin account added in here, eg a service account, I have had issues using the pre generated ones, so we will be adding a custom one

Click Locker

Click Passwords/Add

And add the credentials of a vCenter admin, this can be the administrator account, or a service account you added in vCenter and click Add
You may already have done this, and it can be skipped

Click Aria Suite Lifecycle in the top left to return to the main menu

Then head to Lifecycle Operations

First lets ensure we have enough storage space, we will want ~30GB free
Click Settings/System Details

And wait for the to populate, I have plenty of storage, but you can extend it here if needed on the right

Now click into Settings/Binary Mapping

Click Patch Binaries/Add Patch Binary

Enter /data for the base location and click Discover

Select the patch and click Add

We can also track the request from the link here

You’ll need to reload the web page, and it should appear when the request is done

Click into Settings/System Patches

Click Create Snapshot

Now we should have vCenter credentials in here for the account we just added, enter the vCenter FQDN then click Select vCenter Credential

Then select the account we added earlier

And click Submit

We can click the link to check the progress

If we check the VM in vSphere we can see thats been applied too

Now in Aria LCM click New Patch

Select the patch and click Next

Then click Install

The WebUI will show this as services restart, this is expected

When thats applied log back in

If you have a cloud proxy in Aria Operations you will need to make sure SSH is prepped first else the product inventory will fail
This is only relevant if you have a cloud proxy, if you dont then this can be skipped

We need to make sure SSH is enabled on the cloud proxy
To activate it, open the VM console and login as root, you dont set a password on deployment so it will prompt you to set one if this is your first time logging in

Then run

systemctl start sshd

Now we need to add the password into Aria LCM

From the main menu, which you can get back to by clicking Aria Suit Lifecycle at the top, and click Locker

Click Passwords/Add

Enter the root details and click Add

We first need the upgrade pak file from the Broadcom portal
Click My Downloads/VMware Cloud Foundation

Expand VCF 9 and click the latest release

Find Cloud Foundations Operations and click View Group

And download the upgrade pak file download here

Head back up one level and click View Group on VMware Cloud Foundation Operations Fleet Management

And download that OVA

Heading back to our WinSCP session on Aria LCM in the /data directory, we need to drag those two downloaded files over, it should look like this

Now, the fleet management is a separate appliance, so we need an FQDN assigning in your DNS server, this needs to resolve histname to IP and IP to hostname, an IP address on the same network as the VCF Operations appliance, and 15 character or longer password for the root and admin@local passwords

Logging back into Aria LCM, click Locker

Under passwords click Add

Add the details for the admin@local credential for the fleet management appliance, making sure the username is admin@local, then click Add
This needs to be 15 characters long, the only allows special characters are !@#$%^&*

And repeating for the root account
This needs to be 15 characters long, the only allows special characters are !@#$%^&*

Then click VMware Aria Suite Lifecycle in the top left to head back to the main menu

Click Lifecycle Operations

Then click Settings/Binary Mapping

And click Add Binaries

Fill /data in for the base location and click Discover

Check both of the new binaries and click Add

We can check the progress from this link

When thats done you will need to refresh the page on the Product Binaries and it should look like this

Now we can head to Environments and click View Details on the environment with Aria Operations listed

Click the Operations tab then click Upgrade

Now click trigger Inventory Sync

Then submit and wait for the request to finish

Then head back and click Upgrade again, bit this time then click Proceed

The version should be automatically populated, click Next

We are using VCF, so we will select that licensing type and click Next

Click Run Assessment

Click View Report

Six of my dashboards are impacted and 5 management packs

For the dashboards, none of these are my custom one, so thats fine

Using the Dashboards drop down selector I can change this to management packs

Which seems to be these, I am going to continue with the upgrade, with the exception of the Dell OpenManage pack the rest are VMware ones so they should be fine, the Dell one hasnt been configured so I am not bothered if it breaks

Now I am happy, I will check the box and click next back in Aria LCM

We will check both boxes to take and retain the product snapshots in case we need to roll back and click Next

Now we need to setup the infrastructure for the fleet management appliance
Select your vCenter, cluster, optionally a folder and resource pool, then select the network, this should match the Aira operations appliance you already have, the data store, this is my vSAN datastore, and thin provisioning mode

Add the VM name, FQDN and IP address then click Select Admin Password

And select the admin@local credential we setup earlier

And repeat for the root account

It should look like this, then click Next

Enter our domain and search domain, and click Edit Selection to select our DNS servers

Select both and click Next

And then Finish

It should look like this

For NTP I have an NTP server, so I will select that over hoist time and click Edit Selection

Select the NTP server and click Next

Then Finish

It should look like this

And lastly add the gateway for the network the appliance is on and the subnet mask and click Next

Now click Run Precheck

I only got one warning about the VCF Operations node size, but the 4vCPU and 16GB it has is plenty, so I will ignore this

Once you are happy click Next

And then click Submit

Now we have VCF Operations upgraded to version 9 and the fleet management appliance deployed we can import the old Aria Automation 8.18 appliance into the fleet management, removing it from Aria LCM and then do the upgrade process

Log into VCF Operations and head to Fleet Management/Lifecycle and on the VCF Management tab we will get options for product components, on the Automation tab, click Add

Click Import From legacy Fleet Management and click Next

Add the FQDN of the Aria LCM appliance, and the vcfadmin@local username, on the Admin password field add the password for the vcfadmin@local account, and add the root password as well, then click Next

Select the component, it has picked my appliance up from the environment, then click Next

Now we want to run an inventory sync from Aria LCM, log in and click Lifecycle Operations

Click Environments and then click View Details on the environment which has the Aria Automation Appliance

Make sure the Automation tab is selected, click the three dots then click Trigger Inventory Sync

Then click Submit and wait for the request to finish

Back in VCF Operations, review the warnings, we have done the inventory sync, but its important to understand this is a one way process and cannot be undone, when you are happy, check the box to acknowledge this and click Submit

Then wait for the task to complete

From Fleet Management/Lifecycle/VCF Management/Components click Plan Upgrade in the top right

Under Automation select the target version as 9, and do the same for operations as all components need to be filled out and click Create Plan

Automation now switches to Pending Upgrade

Now we need to set the depot up, click the Depot Configuration tab and click Configure under Online Depot

Click the + to add an account to login to the Broadcom portal

Then add an alias, and put the token in the password fields and click Add

Now click Select Download Token

And select the token alias

Accept the certificate and click ok

The depot will then show as connected

While we are here, lets set the default DNS/NTP servers, click the Settings tab then Networking Settings/DNS and click Add DNS Server

Enter the server name and IP address and click Add

And repeat for any additional DNS servers, it should look like this

Then click Network Settings/NTP Servers and click Add NTP Server

And add your NTP server, I was using a local one on the same server as my primary DNS, but after a few issues, I switched everything over to Cloudflare
When thats filled out click Add

It should look like this

Now we can download the binary we need for this, head to Binary Management, click Automation and click Download

Then wait while its in progress, until it says Downloaded

Then, back under the Overview tab we can now click Upgrade on the Automation tab

And click Run Trigger Sync

Before proceeding, we need to ensure NSX isnt added into Aria Automation in Manager mode, it must be policy mode, if you log into Aria Automation as ad admin and head to the Assembler

Head to infrastructure/Connections/Cloud Accounts and click on your cloud account

And if we scroll down we can see it is using Manager mode

This cannot be changed and must be addressed

Unfortunately for me, this is a full VCF account, not just NSX, so I will have to delete the entire account and start again, and this will likely remove a lot of config so ensure you have all your policies, templates and mappings backed up and noted down 

If you are like me and have a full VCF account in the entire thing must be removed, and this included all associated projects so this is basically a full rebuild, if you cannot do this, do not start the upgrade
I had to remove everything and start from scratch basically

It may also be worth powering the appliance off and snapshotting it in case you want to roll back, but I am going to just click Delete again

I then left the account removed and will reconfigure it all after the upgrade to ensure there are no compatibility issues

And remain on this window without closing it

Do check the linked KB before proceeding, this has the NSX Manager mode listed for example, and when you are happy check the box to acknowledge this and click Proceed

This should then be auto populated correctly, click Next

Ensure the Domain and search domain and DNS are populated

For NTP we can either click Edit Server Selection to use the one we setup earlier

Like so

Or we can use host time, as they all point to the same thing, I am going to keep it on host time, as the NTP server is external
Then make sure the default gateway and subnet mask is correct and click Next

Now for the certificate and password it should have pulled these from Aria LCM, click the certificate drop down and select the right one
Oddly I have two, I suspect this is due to the Ops/Automation being imported into the fleet management server has caused the two with the GUID, so I will use that

Which should look like this

Click Add password at the top

Enter an alias and a complex 15 character password for the admin/root accounts, then click Add
Its worth noting we need the same special character requirements so we can only use !@#$%^&*

Click the cross on the password under the certificate

Then click Select Component Password

And select the one we just created

It should look like this

I then left this at the default setting

Add a node prefix, this upgrade is more of a migration to a new appliance, prefix wise as I have one node I just used the server name as the prefix, the primary VIP should be auto populated so we will use that, its the old FQDN in my case as I have a single node deployment, and leave the internal cluster CIDR
Additional VIPs are optional so I will leave them as is

Now we need a cluster node IP Pool, click Add Cluster Node IP Pool

Broadcoms upgrade guide here, shows the sizing recommendations, but the idea is if you have a single node medium Aria Automation 8.x appliance, the 12 vCPU 54GB node, add two IPs here, for everything else, add four IP Addresses

I opted for two additional IP addresses on the same subnet that arent in use, like so, when you are done click Add

Then click Next in the bottom right

Then click Run Precheck

When thats come back all clear click Next

When you are ready, click submit, this will take ~60-90 mins

Now we have VCF Automation added and upgraded we need to do the same for Aria Operations For networks

In VCF Operations head to Fleet Management/Lifecycle/VCF Management/Overview, then on the operations-Networks tab, click Add

Select Import from Legacy Fleet Management and click Next

Add the Aria LCM FQDN, for the username use vcfadmin@local, add the password for thata ccount and the root account, and click Next

Select the component, it should automatically detect it and list the environment, then click Next

Now log into Aria LCM and click Lifecycle Operations

Click View Details on the environment you selected before

Click the Networks tab, then the three dots and click trigger Inventory Sync

And click Submit

When thats done head back to VCF Operations, click the check box and then click Submit, this is one way action

When thats imported successfully we should see it listed under Components

Now click Binary Management, check the Operations Networks package and click Download

When thats done the status will change to Downloaded

Now head to the components tab and click Plan Upgrade

Select the target version to 9.0.0.0 for Operations-Networks then click Create Plan

It should look like this, then click Upgrade on the right for the operations-Networks component

Click Trigger Inventory Sync

Then Submit

This will bring you too the tasks tab, when thats run head back to Components and click Upgrade again, this time click Proceed

This field should be auto populated, I did have it bug where it couldnt find the repository URL, if you get that  return and go back through clicking Upgrade to reload it
Then click Next

Check both boxes to take and retain a snapshot then click Next

Click Run Pre Check

When thats passed click Next, if anything flags up it will need addressing

When you are happy click Submit

This will take ~90-120 mins to complete, though took two and a half hours so this can be a long one

When its done the Components section will look like this

Now thats done, lets clean up our binaries, head to the Binary Management tab and click the Bin icon on Operations Networks and Automation to remove the binaries

Give it a minute and when you refresh the screen they will not show as downloaded anymore

Log into VCF Operations and head to Fleet Management/Lifecycle, click the VCF Instance, and under SDDC Manager Updates, click Download on Available Updates
In my case I am going from 9.0.0.0 to 9.0.1.0 as my upgrade got halted from 5.2 to 9 at NSX due to NSX 4.2.2 being incompatible with NSX 9 as an upgrade path, however the process is the same

Its also worth noting a depot connection for the SDDC Manager is already configured and you have done the depot change with download tokens for VCF outlined here

This will queue the download

Then, ensure you have a backup taken in the last 24 hours should anything go wrong, head to Administration/SDDC Manager/Backup Settings and ensure its successful

When thats done it will validate it

Then click run Pre Check back under Fleet Management/Lifecycle/VCF Instances from your selected SDDC Manager

Click View Details once its done, I have a few warnings and an error I need to check on

My root password seems to have expired

Unfortunately you can only change this from the console, log in as root and set a new password, it must have 15 characters, and the only special characters you can use are !@#$%^&*
Then re run the pre check

The error has disappeared, and my warnings are just for licensing so I can ignore those as my licensing is all sorted

When you are ready click Update Now

And click Start Upgrade

We will then see the upgrade progress

We then want to check if their are any available configuration updates under Fleet Management/Lifecycle, select your workload domain, mine is istlab-vcf

If there are any available configuration updates click Apply All

Give it a few mins and refresh the page, it should now show no more updates

Next up is NSX, log into VCF Operations and head to Fleet Management/Lifecycle, expand VCF Instances, then expand the SDDC Manager, and then select your workload domain, starting with the management domain

Scroll down to Available Updates, this should be part way through a patching plan, and click Download now, as the NSX upgrade will be already loaded as the next step

Then click Configure Upgrade

Click Next

Click Next again, it will upgrade all Edge clusters

Click Next again, the default of cluster in parallel, but hosts within a cluster sequentially is fine

Then click Run Pre Check

If you have any warnings click View Details

These I can ignore, I dont use the API, NSX-V migration is irrelevant and the Operations version validation I can manually check

NSX 9.0.1.0 is interoperable with my VCF operations version, 9.0.0.0

Now click Schedule Update

Click Next

Select your maintenance window, you can schedule it for overnight, or apply now,. I will be opting for now, and check the box to say you have reviewed the pre check findings and click Finish
This will take ~2 hours depending on your setup, hosts are not remediated

If we click View Status we can see its progress over the upgrade

And we can expand the components

When its done it will look like this

As this will cause a small outage windows for vCenter, you should make sure no backup jobs using it are scheduled to run for a couple hours while the upgrade completes

Back at the management domain upgrade section, click Download Now to get the vCenter upgrade files

Before continuing, log into vSphere, click the cluster and click Configure/Services/vSphere DRS and then click Edit on the right

Then change the automation level to Partially Automated

When thats downloaded, click Configure Update

Click Next, reduced downtime upgrade is required for this

Log into the vCenter VAMI on
https://fqdn:5480
And confirm under Backup that one has run in the last 24 hours, if one hasnt, manually run one

Then check the radio button to say you have backed up vCenter and click Next

For Network Configuration, click static, enter a free IP on the subnet for the new vCenter to use, then add the subnet mask and gateway in and click Next

We can the select our schedule option, to deploy the vCenter at a time, and the switch over automatically or at a set time, I want to do this now so I used Immediate and Automatic
Then click Next

Check you are happy with everything and click Next

We can click View Status under In-Progress Updates to see where it is

This will take ~1 hour, but the total downtime in services was 8 mins during the switchover and will look like this

And when you are done, log back into vSphere, click the cluster then Configure/Services/vSphere DRS and click Edit on the right again

Then change the automation level back to Fully Automated and click ok

For ESX we again need to click Download Now under Available Updates for the management domain

While thats downloading we need to create a cluster image, in vSphere, click the three lines in the top left and click LifeCycle Manager

Under Image Library, click Create Image

Give it a name, I name it in the format ESX-<Version>-<Vendor> and select the target ESX build from the upgrade plan, in my case, 9.0.1.0
Then click Select under Vendor Addon

And select the addon that matches your vendor, in my case Dell, then click Select

Optionally, we can add the async VMware tools, to add this click Show Details on Components, then click Add Components

The latest 13.0.5 is included in ESX 9.0.1.0 so we dont need to do this for our upgrade path, when you are happy with your image click Validate

If the image is valid, click Save

Now we need to import the image through VCF Operations, click Fleet Management/Lifecycle and click your VCF Instance, mine is Capella, then click Image Management/Import Image

I only have one workload domain, the management domain, so the vCenter is automatically selected, but we should see our new image, click that and click Import

We can see the progress from Fleet Management/Tasks under our VCF instance if needed

When thats done we can head back to Fleet Management/Lifecycle, select our management Domain and click Configure Update

Click Next

We will want all cluster selected, then click Next

Select your cluster and click Assign Image

Select our image and click Assign Image

Then click Next

As ESX 9.0.1.0 is not Live Patch eligible, ensure Enforce Live Patch is not checked, and click Next
For upgrades, even if the version is live patch eligible I dont recommend using them for 8 –> 9

Then click Run Precheck

Click View Details to see any warnings, I got quite a lot

The first two were for hardware compatibility issues, I am running 3xR640 and 1xR740 in this cluster which doesnt officially support VCF 9 so that explains that, and another for firmware, I dont have vendor firmware integration licenses for Dell OME so thats fine, I can manually check this

All 4 hosts then generated the same 4 errors

I have a CPU support error, 1st Gen Xeon Scalable isnt officially supported, only 2nd gen and later, it will work so for my lab I can ignore this

Not too sure what the baseOS warning is for, this is expected and can be ignored

Same for the vendor addon, as you should upgrade this, so this can also be ignored

The last one is more of an issue, the H330 does have drivers in VCF 9 and when testing with the ESX 9 ISO my vSAN disks are detected so for my lab I am ignoring this, for production you should not upgrade

Click Exit Details in the top left when you are happy

Now that I have checked these and am happy to proceed, I am going to click Schedule Update

Click Next

Then select to run the upgrade now or schedule it, and in my case as I have warnings I need to accept the box to acknowledge them and click Finish

We can view the status on the In-Progress section, you may need to refresh the page, by clicking View Status

When its done it should look like this

This part is only needed if you didnt have an Operations Collector orginally, which I didnt, its worth noting this you do need one, the Supervisor can only collect metrics through it and I would recommend having one for everything else

In the Broadcom portal, search and click VMware Cloud Foundation

Expand VCF 9 and select the release you are upgrading to, I am upgrading to 9.0.1 so I selected that

On VCF Operations Collector, click View Group

And download the appliance

Right click the cluster and click Deploy OVF Template

Click Local File and then Upload Files

Double click the OVF file

And click Next

Give the VM a name, optionally select a folder and click Next

Select the cluster for compute and check the box to automatically power on the VM, then click Next

Click Next here

Accept the EULA and click Next

We will want to select a Small Unified Proxy and click Next
The large proxy can do up to 80000 VMs for larger environments

Select a datastore and click Next, I am using my vSAN datastore

Select the same network as VCF Operations, as this is the domain that contains VCF Operations, and click Next

We need a root password, this must be 15 characters and the only allowed special characters are !@#$%^&*

We then need to log into VCF Operations and head to Administration/Cloud Proxies and click Add

We then need to copy this large registration key

Enter the registration key into the OVF deployment page, enter a friendly name, I am using the VM name, and add an NTP server

This can be left blank unless you explicitly need it

Enter the domain name, search domain and comma separated DNS servers

Set the IPv4 type to static and add the VM IP, gateway and subnet mask, we can leave IPv6 blank unless we explicitly need it and click Next

When you are happy, click Finish

After ~15 mins the cloud proxy should appear in VCF Operations and be in the Going Online state

We then need to move our integrations to the new collector, head to Administration/Integrations and expand the VCF instance, click the three dots and click Edit

Change the collector to the new cloud proxy we deployed and click Validate Connection

At the bottom under Advanced Settings click Manage Integration

Enable the toggle so that we can license the vCenter and click Save

Then click Save

We need to repeat this for all other integrations, but the important one is the supervisor as this requires it as well
Expand the supervisor, click the three dots and click Edit

Change the collector to the proxy VM and click Validate Connection
Then Save in the bottom left

We have a couple of options here, we can deploy a new appliance with a new IP and migrate the data, or we can do the option I will be using since I am not worries about loosing the log data, and power off the old appliance and rename it, then re use the IP/FQDN as everything is already pointing to it

From VCF Ops heat to Fleet Management/Lifecycle/VCF Management/Binary Management/Install Binaries, select the operations-logs binary, on 9.0.1, then click Download

When its done it will say Downloaded

Click Overview, and under operations-logs, click Add

Click New Install, and then for the deployment, unless you have a very large environment, Standard will be fine, then click Next

We then need a certificate, its important we have the hostname registered in DNS
Click the + then Generate certificate

Enter an Alias, this is a friendly name, the CN, O, OU and C dont matter for self signed certificates, what is important is the server FQDN and IP which need to match this server, then click Generate

Select the certificate from the drop down and click Next

Select the vCenter server, cluster, optionally and folder and resource pool, then the network, datastore and disk mode which wants to be thick, then click Next

Enter the domain and search domain name then click Edit Server Selection

Select the DNS servers and click Next

Then click Finish

For NTP you can use host time, or an NTP server, I select the latter, if so, click Edit Server Selection

Select the server and click Next

And click Finish

Then enter the gateway and subnet mask and click Next

Click Add Password

And add the admin/root password, it should be 15 characters, the only allowed special characters are !@#$%^&*
Then click Add

Select the node size, small is ok for me, but medium is a good production fit, I disabled FIPS, the certificate should be auto populated, we can then leave the affinity rules, as we are not doing a cluster select No for a VIP, upgrading VM compatibility isnt needed but can be checked, check the box to always use English, and add an admin email

Click Select Component Password, this will be both the admin and root password

And select the password we just added

NTP should also be pre populated

Then at the bottom, enter the VM name, FQDN and IP address and click Next

Then click Run Precheck

When thats passed click Next

When you are happy, click Submit

When thats done it should look like this

The last little bit is updating the agents if you have any
Click Management/Agents

Then enable the toggle for auto updating all agents

This doesnt seem to update Windows agents, older 8.18.x agents will work with Logs 9, but you’ll need to manually run the new agent on those machines to upgrade them

This doesnt register the vCenter directly into VCF Ops For Logs as a source, this means while all logs are coming in, it will be stuck in evaluation mode

In the VCF Ops For Logs GUI click Integrations/vSphere and click Add vCenter Server

Enter your hostname for the vCenter and an administrator account, I recommend a service account, but I am using the default administrator account, I changed my SSO domain from vsphere.local to leaha.co.uk, so thats why its different
Select the target from the drop down, which is the Log server, make sure both boxes are checked and click Test Connection

Click Accept on the SSL certificate

Then click Save

This removes the evaluation licensing mode from the server

Back in VCF Operations head back to Administration/Integrations, expand the VCF integration and on your VCF instance click the three dots and click Edit

Select the domain and check the box to enable log collection, repeat for additional workload domains and click Save

First we need to download the patch for LCM so we can get VCF Operations upgraded to version 9, we need Patch 5 for VCF 9.0.1, we can find this from the Broadcom portal downloads and then search and select Aria Suite

Click Solutions at the top, expand Enterprise and click the 2019 release

We then need to find and click the Patch5 link

And download the patch

We then need to upload the patch to Aria LCM
Connect to it with the root account using WinSCP

Double click the bar here

And navigate to /data and click ok

On the right panel head to where ever you have the patch downloaded and drag it over to the root of /data
It should look like this, the folders in /data may be different, the important thing is that the upgrade file is under /data