vcf vcf911 hardware esx vsan memory-tiering lab

The Hardware

The eight ESX hosts the lab is built on, before any software goes on them — and what that hardware rules in and out.

VCF 9.1.1 lab rebuild. Part of the follow-along guide, published live as the build happens.

📘 Official documentation. This is a lab walkthrough, not a replacement for the vendor docs. For the authoritative, supported procedures, follow Broadcom’s VMware Cloud Foundation 9.1 documentation. Where this guide and the official docs differ, the official docs win.

Where we are: nothing is installed yet. This page is the physical inventory — eight machines in two clusters — and what that hardware allows and rules out. Every later page builds on it.

Four identical AMD mini-workstations carry the management cluster on vSAN. Four Intel machines, each a different board and CPU, carry workloads over NFS. Every host uses an NVMe drive as a second tier of memory, set to 300 percent, so each one presents four times its installed DRAM.

warning
Lab only — this is not supported hardwareThese are consumer CPUs, boards and NVMe drives. None of it is on the Broadcom Compatibility Guide, and several parts need community drivers. It is a lab built to learn and demonstrate VCF, not a design to copy into production. Each lab-only choice is called out where it appears below.

Where the numbers come from

The figures on this page come from labman, the lab’s estate source-of-truth site. It is not part of VCF. It holds the record of every host and VM in the lab, and the build reads from it. The hardware was checked live against vCenter on 2026-08-20.

labman is behind on one thing: it still lists ESX 9.1.0 builds. All eight hosts have run ESX 9.1.1.0 build 25714478 since 2026-09-13, and that is the build this guide is written against.

Serial numbers, MAC addresses and full device identifiers are left out on purpose.

Fleet totals

Hosts84 AMD in cl01 · 4 Intel in cl02
CPU92 cores184 threads · 64 AMD, 28 Intel
Installed DRAM918 GiB439 GiB cl01 · 479 GiB cl02
Presented memory3,672 GiBexactly 4.0× on all eight hosts
vSAN raw capacity8 TBcl01 only, 4 × 2 TB, before FTT=1
10 GbE uplinks in use122 per cl01 host · 1 per cl02 host

Memory per host

Installed DRAM and the NVMe tier stack up to what ESX presents to the scheduler. One management host has half the DRAM of the others. Because tiering multiplies it, the scheduler sees that host as 256 GiB short, not 64.

Installed DRAMNVMe memory tier
cl01 · management · AMD
pgesxa1
pgesxa2
pgesxa3
pgesxa4
cl02 · workload · Intel
pgesxi1
pgesxi2
pgesxi3
pgesxi4
warning
Lab only — memory tiering at 300 percentThe capacity model for memory tiering in VCF Operations works on a 1:1 ratio of DRAM to NVMe. The lab runs 3:1, on consumer flash, some of it DRAM-less. That is a choice to fit more into a small lab. It is not a sizing recommendation.

📖 Reference — VMware Cloud Foundation 9.1 documentation, Cost and Capacity Management › Using Memory Tiering for Resource Efficiency, pp. 6561–6563. The tier size is set as a percentage of DRAM and accepts 1 to 400: esxcli memtier enable --help on ESX 9.1.1.0 build 25714478.

pgvcf1-cl01 · management · vSAN ESA

Four Minisforum MS-A2 units: AMD Ryzen 9 9955HX, 16 cores and 32 threads each, BIOS 1.03 on all four. Each has three NVMe drives with one job apiece — boot, memory tier and vSAN — and two Intel X710 10 GbE SFP+ ports as its distributed switch uplinks. The onboard 2.5 GbE port is not used.

🔗 Community — for VCF on this machine, William Lam’s VMware Cloud Foundation (VCF) on Minisforum MS-A2 is the write-up to start from. Community work for lab hardware, not Broadcom guidance.

HostBoardCPUCores / threadsDRAM fittedPresentedUplinks in useBootMemory tiervSANBIOS
pgesxa1Minisforum MS-A2Ryzen 9 9955HX16 / 32128 GiB502.7 GiBvmnic1 + vmnic2 · X710 10G2.5G onboard unusedCrucial P1 1 TBCrucial T500 1 TBCrucial T500 2 TB1.03
pgesxa2Minisforum MS-A2Ryzen 9 9955HX16 / 32128 GiB502.7 GiBvmnic1 + vmnic2 · X710 10G2.5G onboard unusedSamsung 970 EVO 500 GBCrucial T500 1 TBCrucial T500 2 TB1.03
pgesxa3Minisforum MS-A2Ryzen 9 9955HX16 / 32128 GiB502.7 GiBvmnic1 + vmnic2 · X710 10G2.5G onboard unusedSPCC 500 GB, DRAM-lessCrucial T500 1 TBCrucial T500 2 TB1.03
pgesxa4Minisforum MS-A2Ryzen 9 9955HX16 / 3264 GiB246.7 GiBvmnic1 + vmnic2 · X710 10G2.5G onboard unusedSPCC 256 GB, DRAM-lessCrucial T500 1 TB · slot reversedCrucial T500 2 TB · slot reversed1.03

pgvcf1-cl02 · workload · NFS

Four Intel hosts on four different boards. None has a vSAN drive, so all storage comes from the NAS over NFS. The NAS is the lab’s TrueNAS storage server. It is not part of VCF. Each host has one 10 GbE uplink in use.

HostBoardCPUCores / threadsDRAM fittedPresentedUplinks in useBootMemory tierBIOS
pgesxi1Minisforum MS-01Core i9-12900H6 / 1296 GiB383.0 GiBvmnic1 · X710 10G1 × X710 10G port idleKingston NV3 500 GB, DRAM-lessSPCC 2 TB, DRAM-less1.27
pgesxi2ASRock Z590M Pro4Core i7-11700KF8 / 16128 GiB511.5 GiBvmnic1 · 82599 10G1 × 82599 10G port idleCrucial BX500 240 GB SATASPCC 500 GB, DRAM-lessL2.22
pgesxi3ASRock Z790M PG Lightning/D4Core i7-12700K8 / 16128 GiB511.2 GiBvmnic1 · X520 10G single-portno spare 10G portPatriot P210 256 GB SATACrucial T500 1 TB13.01
pgesxi4Gigabyte B760 Gaming X AX DDR4Core i5-124006 / 12128 GiB511.1 GiBvmnic1 · 82599 10G1 × 82599 10G port idleSPCC 256 GB, DRAM-lessSamsung 960 PRO 512 GBF16

What the hardware decides

Only cl01 can run vSAN

cl02 has no capacity drives at all. Its storage is NFS from the NAS. A second vSAN cluster would mean buying hardware.

The cl01 hosts meet the vSAN ESA floor of 16 cores and 128 GB per node, with one exception on memory, which is the next point.

warning
Lab only — consumer NVMe for vSAN ESAThe vSAN drives are Crucial T500 consumer NVMe. They are not on the vSAN compatibility list. From VCF 9.0.1 the docs allow non-certified ESA disks for proof-of-concept only (Broadcom KB 408300), and that is the basis this lab runs on.

📖 Reference — VCF-VSAN-ESA-REQD-CFG-003 (16 cores or higher per vSAN node) and VCF-VSAN-ESA-REQD-CFG-004 (128 GB or higher per node), both requirements, VMware Cloud Foundation 9.1 documentation, p. 863. Non-certified ESA disks: Deployment › Preparing ESX Hosts for VCF or vSphere Foundation, pp. 2419–2425.

cl01 is not uniform

pgesxa4 has 64 GiB fitted where the other three have 128 GiB, so it sits below the 128 GB ESA requirement above. Because tiering multiplies DRAM by four, the scheduler sees it as 256 GiB short, not 64. Admission control and vSAN rebuild headroom sized for four equal hosts are wrong by about a quarter.

pgesxa4 also has its NVMe slots the other way round. Its 2 TB vSAN drive is at PCI 09:00.0 and the 1 TB tier drive at 0a:00.0, the opposite of the other three. The kickstart page shows why this does no harm: the install binds drives by identity, not by slot.

A single cable failure isolates a host. It has happened. pgesxi1, pgesxi2 and pgesxi4 each have an idle 10 GbE port. pgesxi3 has a single-port card and no spare.

A uniform first cluster, a mixed management domain

The docs set two rules here. The lab is close on the first and does not meet the second.

The first cluster must be uniform. VCF does not support mixed hardware in the cluster the VCF Installer builds, because vCenter is not there yet to manage it. cl01 is four MS-A2 units with the same board, CPU and network cards, which is why the AMD machines are the management cluster and the mixed Intel machines are not. The one difference inside cl01 is the memory in pgesxa4, covered above.

Every host in the management domain must come from one hardware vendor. cl02 sits in the management domain’s vCenter. It was built there directly and is not yet managed by SDDC Manager; the plan is to add it through SDDC Manager, at which point the domain holds boards from Minisforum, ASRock and Gigabyte and the lab does not meet this rule. The rule is about the hardware vendor, not the CPU vendor.

warning
Lab only — more than one hardware vendor in the management domainCross-vendor hardware in the management domain is not supported. The lab does it because this is the hardware the lab has.

📖 Reference — VMware Cloud Foundation 9.1 documentation, Deployment › Preparing ESX Hosts for VCF or vSphere Foundation, pp. 2419–2425.

No remote management on any host

None of the eight has a BMC. BIOS updates and the ESX install are hands-on, with a USB stick. That is why the next page is about kickstart.

About the references

Page numbers are from the September 2026 build of the VMware Cloud Foundation 9.1 PDF. Broadcom rebuilds the PDF from time to time and the page numbers move, so each reference also gives the section name.