The Second Cluster, As Built

VCF 9.1.1 lab rebuild. Part of the follow-along guide, published live as the build happens.

📘 Official documentation. This is a lab walkthrough, not a replacement for the vendor docs. For the authoritative, supported procedures, follow Broadcom’s VMware Cloud Foundation 9.1 documentation. Where this guide and the official docs differ, the official docs win.

Where we are: after the VNA cluster. The management cluster, pgvcf1-cl01, is up on vSAN ESA with distributed VPC connectivity. This page adds the four Intel hosts as a second cluster in the same vCenter, on NFS.

Status: built and healthy, 2026-09-19. Every figure on this page was read from vCenter after the build, not carried over from a plan.

What is built

Hosts

EVC

The cluster mixes 11th-generation (Rocket Lake) and 12th-generation (Alder Lake) parts, so it needs an EVC baseline. The baseline here is derived, not chosen. vCenter reports each host’s ceiling as maxEVCModeKey, its supportedEVCMode list is ordered oldest to newest, and the cluster baseline is the lowest of the hosts’ ceilings by that order. All four hosts report intel-haswell. It was set while the cluster held no VMs.

Networking

Where: vCenter › Networking.

One distributed switch, pgvcf1-cl02-vds01, set up to match the switch SDDC Manager built for pgvcf1-cl01: version 9.1.0, MTU 9000, Network I/O Control on, LLDP advertise and listen. It defines uplink1 and uplink2. Only uplink1 is populated, with vmnic1 on every host.

VMkernel adapters

Every adapter is MTU 9000 with a static address. The vMotion and NFS addresses reuse the host’s management last octet. That keeps them outside .100–.199, the range the SDDC Manager network pool pgvcf1-np02 owns.

• vMotion is on the dedicated vmotion TCP/IP stack, with its own default gateway, 10.129.5.1. pgvcf1-cl01’s vMotion network is a different subnet — 10.129.2.0/24 on VLAN 1102 — so vMotion between the clusters is routed, and the gateway on the vMotion stack is what gives it a route.
• NFS is on the default stack. The storage server is 10.129.4.10, in the same subnet, so storage traffic is one layer-2 hop and never uses the management VMkernel.

Storage

Six NFS v3 datastores, all mounted read-write from the NAS’s storage-VLAN address, 10.129.4.10.

One server address per share, on every host. vCenter identifies an NFS datastore by server and path. The NAS also answers on the management subnet, at 10.128.1.110. The same share mounted through that address on one host and through 10.129.4.10 on another becomes two datastores, and vMotion and HA do not work between them. Each of the five shared datastores here is a single vCenter object, mounted on all eight hosts.

pgvcf1-cl01 reaches the same storage VLAN through a port group, pgvcf1-cl01-vds01-pg-nfs on VLAN 1104, and one VMkernel per host: 10.129.4.220, .222, .224 and .226.

Tags and folder

The six datastores sit in the datastore folder tenancy/pgnet-core and carry seven tags. Each category is single-cardinality and can be attached to datastores only.

These tags are descriptive. Workload placement uses a separate storage-use category, which tag-based storage policies read.

How it was built

Three scripts, each idempotent — a second run reports nothing to do — and each a dry run unless --apply is given. They are in the build repo and not published yet.

uv run --group vsphere scripts/cl02_vcenter_build.py --apply   # cluster, switch, hosts, uplink move, VMkernels, storage, EVC, HA/DRS
uv run --group vsphere scripts/nfs_storage.py --apply          # NFS port group and VMkernel where missing, and the shared mounts, both clusters
uv run --group vsphere scripts/datastore_tags.py --apply       # tag categories, tags, datastore folder

cl02_vcenter_build.py runs in stages: cluster, vds, hosts, migrate, vmk, storage, evc, ha.

The migrate stage moves a host’s only uplink. vmnic1 and vmk0 move to the distributed switch in a single UpdateNetworkConfig call made through vCenter, so vCenter’s network rollback covers the whole move: if the host loses contact, the change is reverted. It was run on one host, checked, then run on the other three. Management connectivity was not interrupted on any host. The empty vSwitch0 was removed afterwards.

nfs_storage.py sends an 8972-byte ping with the don’t-fragment bit set to the storage server before it mounts anything, and stops if the ping fails. A mount over a VLAN that does not carry MTU 9000 succeeds and then performs badly. The ping fails immediately.

Evidence

Lab-only conditions

Limitations of this cluster

It was built in vCenter and is not managed by SDDC Manager. The four hosts are not commissioned there and the cluster is not in its inventory. In practice:

• No VCF lifecycle for the cluster. Upgrades are driven from vCenter’s Lifecycle Manager, not from SDDC Manager.
• No SDDC Manager credential rotation for these hosts.
• SDDC Manager did not prepare the hosts for NSX. In 9.1 that is started from vCenter, and it was done by hand on 2026-09-19: all four hosts are NSX transport nodes with tunnel endpoints on VLAN 1305. The inputs, the result and the evidence are on the next page. Anything SDDC Manager would re-apply or reconcile on a cluster it prepared, it does not do here.
• A day-2 change to a switch SDDC Manager owns. The NFS port group and VMkernels added to pgvcf1-cl01’s switch are not known to SDDC Manager.

Why SDDC Manager could not build it

SDDC Manager’s add-cluster workflow validates vMotion and NFS connectivity before it creates anything. Each validator builds a temporary standard switch on the host, and needs a free physical NIC taken from the NICs named in the cluster specification. On these hosts the only NIC that can be named is vmnic1, and it carries management:

No physical NIC could be attached to vSwitch 'pgvcf1-cl02-vds01' on host pgesxi1.pgnet.io. All candidate PNICs
[vmnic1] are already in use by other virtual switches. Verify that the active/standby uplink NICs for the 'NFS'
network are not exclusively consumed by the management vSwitch (vSwitch0).

A second NIC cannot simply be added to the specification, because the specification validator requires:

• The same vmnic names mapped to the same switch and uplink on every host. Otherwise: “Vmnic to vds association is not same across given set of hosts”.
• Every NIC at least 10 Gb/s, or at least 1 Gb/s if it connects only to a management network.

These hosts have one cabled 10 Gb/s port each, and their spare ports differ from host to host in name and speed. pgesxi1 and pgesxi4 have a second 10 Gb/s port named vmnic0. pgesxi2’s second 10 Gb/s port is vmnic2, because its onboard 1 Gb/s NIC holds vmnic0. pgesxi3 has a single-port 10 Gb/s card.

The documentation states that a host with a single physical NIC is supported:

Hosts can have a single physical NIC or multiple physical NICs. For hosts with multiple physical NICs, only one can be connected to a standard switch.

Commissioning accepts a single-NIC host. The add-cluster validators on 9.1.1 do not run on one.

📖 Reference — VMware Cloud Foundation 9.1 documentation. Building Cloud Infrastructure › Managing ESX Hosts in VMware Cloud Foundation, pp. 2665–2672: the single-NIC statement above. Create a New Workload Domain, pp. 2674–2681: “Each host must have at least one physical NIC, connected to a standard switch.” Add an SDDC Cluster to a VCF Domain, pp. 2700–2708: the Skip failed hosts during cluster creation option. The validator messages are quoted from the product.

Plan to rectify

1. Uniform dual 10 Gb/s on every host — the same two vmnic names on each, both cabled and trunked with VLANs 1001, 1104, 1105, 1305 and 4000 at MTU 9000. Optics and converters were ordered on 2026-09-19. pgesxi2 needs its 10 Gb/s ports to enumerate as vmnic0 and vmnic1, and pgesxi3 needs a second 10 Gb/s port.
2. Prove each new link, with a check that tests a VLAN through one named NIC on a throwaway switch.
3. Re-add the hosts through SDDC Manager. Evacuate the cluster and remove it from vCenter, commission the hosts against the network pool pgvcf1-np02 — vMotion on VLAN 1105, NFS on VLAN 1104, kept for this — and create the cluster with both NICs in the specification and skipFailedHosts set to false.

The 9.1 documentation describes importing a whole vCenter as a workload domain. It does not describe adopting one cluster, built by hand, into a vCenter that SDDC Manager already manages. So step 3 rebuilds the cluster’s membership rather than converting it in place. The names, addressing, switch layout and storage on this page are the ones SDDC Manager’s own build would produce, so workloads and tenancy configuration that refer to them carry over.

📖 Reference — VMware Cloud Foundation 9.1 documentation, Building Cloud Infrastructure › Import an Existing vCenter to Create a Workload Domain, pp. 2688–2695, Table 977 Supported Configurations: a vSphere Distributed Switch 8.0 or later, statically assigned VMkernel addresses, a dedicated network for vMotion, a datastore shared across and writable by all hosts in the cluster, and clusters using vSphere Lifecycle Manager images.

About the references

Page numbers are from the September 2026 build of the VMware Cloud Foundation 9.1 PDF. Broadcom rebuilds the PDF from time to time and the page numbers move, so each reference also gives the section name.