VCF on VxRail: 5.2.1.1 to 5.2.2 Upgrade (Management WLD)

In this post I will provide an example walk-thru upgrade of a VCF Management Workload Domain with vLCM enabled.:

VCF 5.2.1.1 (VxRail 8.0.322) -> 5.2.2 upgrade (VxRail 8.0.361)

One of the great advantages of a VCF On VxRail solution is the ability to LCM your entire SDDC stack in an automated fashion driven through a single user interface (SDDC Manager). The following post details how you step through this LCM process.

By expanding the Lifecycle Management tab within SDDC manager it is possible to view the full list of components deployed as part of VCF and their respective current running code versions. The following is a very useful VMware kb article detailing the BOM for each VCF release:

Correlating VMware Cloud Foundation version with the versions of its constituent products

It is also advisable to read the release notes of the target version prior to performing the upgrade.

Release Notes: the bill-of-materials below can be officially referenced in the following Release Notes links :

DELL: VCF 5.2.2 on VxRail Release Notes

VMware: VCF 5.2.2 on VxRail Release Notes

In relation to this particular example upgrade of VCF 5.2.1.1 -> 5.2.2 we can again view from the ‘Release Versions’ in SDDC Manager the components being upgraded and their code versions & build numbers.

You can perform a sequential or skip-level upgrade to VMware Cloud Foundation 5.2.2 on Dell VxRail from VMware Cloud Foundation 4.5.2 or later. If your environment is at a version earlier than 4.5.2, you must upgrade the management domain and all VI workload domains to VMware Cloud Foundation 4.5.2 or later and then upgrade to VMware Cloud Foundation 5.2.2.

The sequence of the upgrade is defined within SDDC Manager itself. SDDC Manager presents the next available upgrade bundle throughout the procedure which makes the overall procedure quite easy and seamless to the end user performing the upgrade. In an effort to incrementally speed up the process of upgrading it is advisable to download all the required bundles in advance (bundles marked as available/future apply), as some of the bundles can be quite large in size for example the NSX bundle is 12GB. After each of the bundles have been downloaded click View Details, which details the version changes that will be applied as part of the bundle upgrade. The update bundles can be applied manually or scheduled for a preferred time window.

Ensure to run the upgrade Precheck tasks from within SDDC Manager in order to validate VCF is in a healthy state before proceeding with each of the upgrades and view the health status of each component by expanding their respective results. If the precheck has failed, fix the issue and then retry the precheck task to repeat the validation process. (More details on context level prechecks here). It is also advisable to run VxVerify – Pre-upgrade Health Check Tool.

If your VCF instance has been upgrade using the AP tool ensure to update SDDC-M inventory with accurate product versions by using “–sync” option from VCF Async Patch Tool Options.

SDDC MANAGER PRECHECK

Important: Before proceeding with the upgrade ensure to take snapshots of your management virtual machines. Do not perform any SDDC workflows such as VI WLD creation or WLD expansions during the upgrade and confirm no failed workflows or errors are displayed in SDDC manager.

While proceeding with the updates, click on View update activity to monitor progress and track progress via the lcm log:

tail -f /var/log/vmware/vcf/lcm/lcm.log

lcm-debug log file contains debug level logging information.

It is also possible to confirm a successful update via the SDDC manager ssh console:

cat /var/log/vmware/vcf/lcm/lcm.log | grep "reached final state COMPLETED_WITH_SUCCESS“

Radar log used for any VxRail precheck validations:

/var/log/msytic/radar.log

All of the above steps/guidelines are displayed as part of the walk-thru series detailed for each of the upgrades below.

NOTE: In advance of completing the VxRail upgrade users are advised to follow the steps outlined in this KB article before initiating the upgrade process to ensure a smooth transition from VCF on VxRail 5.x to 5.y, and to avoid encountering the described error:

https://knowledge.broadcom.com/external/article?legacyId=94747

Stepping through each of the upgrade bundles:

1. VMware Cloud Foundation 5.2.2 (5.2.2-24936865)

In order to check the availability of Update/Patches from within the SDDC manager console navigate to Inventory->WLD->Updates and begin the ‘PLAN UPGRADE’ process. In this example we see that VCF 5.2.2.0 update is the first available bundle to be applied:

2. VMware NSX-T Data Center 4.2.3 (24866349)

Broadcom Knowledge Base article 396719 details a critical issue affecting certain versions of VMware NSX due to a Java Development Kit (JDK) bug (JDK-8330017). This bug causes the ForkJoinPool’s internal thread management to malfunction, leading to blocked thread creation and unresponsive NSX services. Symptoms include degraded Edge and ESXi Host Transport Nodes, VM network failures, vMotion issues, and NSX Manager crashes. The root cause is an overflow in the ForkJoinPool’s Release Count (RC) field, which disrupts task execution across services like the Controller, Upgrade Coordinator, and Corfu. The issue is resolved in NSX versions 4.2.1.4, 4.2.2.0, and 9.0.1.0 and later. Broadcom recommends a monthly rolling reboot of NSX Managers as a preventative measure and provides steps to restore functionality if symptoms occur before upgrading.

https://knowledge.broadcom.com/external/article?articleNumber=396719

3. VMware vCenter Server Appliance 8.0 Update 3g (24853646)

4. VXRAIL UPGRADE BUNDLE VERSION 8.0.361

NOTE: In advance of completing the VxRail upgrade users are advised to follow the steps outlined in this KB article before initiating the upgrade process to ensure a smooth transition from VCF on VxRail 5.x to 5.y and to avoid encountering the described error:

https://knowledge.broadcom.com/external/article?legacyId=94747

Example Steps for KB: