Operational Defect Database
BugZero updated this defect 54 days ago.
Data sources
All data on this page is proprietary to BugZero® or gathered from public sources
VMware | 96830
VCSA patching to 8.0U2 or Higher version fails with "Internal error occurs during execution of update process. Pre-Install failed for vpxd:Expand"
Last update date:
3/26/2024
Affected products:
vCenter Server
Affected releases:
8.0
Fixed releases:
No fixed releases provided.
Description:
Symptoms
vCenter Server Appliance (VCSA) patching to 8.0 U2 or Higher version fails with "Internal error occurs during execution of update process. Pre-Install failed for vpxd:Expand"/var/log/vmware/applmgmt/PatchRunner.log 2024-03-02T15:21:00.627Z vpxd:Expand INFO vmware_b2b.patching.utils.reporting_utils Setting global ReportingFactory with identifier - vpxd:Expand2024-03-02T15:21:00.912Z vpxd:Expand INFO extensions Found patch hook <module 'vpxd' from '/storage/seat/software-update39az1wwa/stage/scripts/patches/payload/components-script/vpxd/__init__.py'>:Expand'2024-03-02T15:21:00.912Z vpxd:Expand INFO extensions Executing patch hook '<module 'vpxd' from '/storage/seat/software-update39az1wwa/stage/scripts/patches/payload/components-script/vpxd/__init__.py'>:Expand' with context <patch_specs.PatchContext object at 0x7fb9a9382710>.2024-03-02T15:21:00.912Z vpxd:Expand INFO vpxd Vpxd expand phase started2024-03-02T15:21:00.912Z vpxd:Expand INFO vpxd Placed B2B marker2024-03-02T15:21:00.912Z vpxd:Expand INFO vpxd Placed temp NDU marker2024-03-02T15:21:00.913Z vpxd:Expand INFO vpxd Copied vpxd cfg files to tmp2024-03-02T15:21:00.922Z vpxd:Expand INFO vpxd Completed vpxd.cfg.tmp modification2024-03-02T15:21:00.947Z vpxd:Expand ERROR vmware_b2b.patching.executor.hook_executor Patch hook 'vpxd:Expand' failed.Traceback (most recent call last): File "/storage/seat/software-update39az1wwa/stage/scripts/patches/py/vmware_b2b/patching/executor/hook_executor.py", line 74, in executeHook executionResult = systemExtension(args) File "/storage/seat/software-update39az1wwa/stage/scripts/patches/libs/sdk/extensions.py", line 106, in __call__ result = self.extension(*args) File "/storage/seat/software-update39az1wwa/stage/scripts/patches/libs/sdk/extensions.py", line 123, in _func return func(*args) File "/storage/seat/software-update39az1wwa/stage/scripts/patches/payload/components-script/vpxd/__init__.py", line 153, in expand if role_update_for_sioc(): File "/storage/seat/software-update39az1wwa/stage/scripts/patches/payload/components-script/vpxd/sdrs_utils.py", line 119, in role_update_for_sioc isUpdated = role_update(authz_service) File "/storage/seat/software-update39az1wwa/stage/scripts/patches/payload/components-script/vpxd/sdrs_utils.py", line 52, in role_update privileges = authz_service.GetPrivileges() File "/usr/lib/vmware/site-packages/pyVmomi/VmomiSupport.py", line 598, in <lambda> self.f(*(self.args + (obj,) + args), **kwargs) File "/usr/lib/vmware/site-packages/pyVmomi/VmomiSupport.py", line 388, in _InvokeMethod return self._stub.InvokeMethod(self, info, args) File "/usr/lib/vmware/site-packages/pyVmomi/SoapAdapter.py", line 1733, in InvokeMethod self._CallLoginMethod() File "/usr/lib/vmware/site-packages/pyVmomi/SoapAdapter.py", line 1805, in _CallLoginMethod self.loginMethod(self.soapStub) File "/usr/lib/vmware/site-packages/cis/cisreglib.py", line 589, in _doLogin with sso_client.securityctx_modifier(soapStub): File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/usr/lib/vmware/site-packages/cis/cisreglib.py", line 252, in securityctx_modifier self._update_saml_token() File "/usr/lib/vmware/site-packages/cis/cisreglib.py", line 233, in _update_saml_token self._saml_token = sts_auth.get_hok_saml_assertion( File "/usr/lib/vmware/site-packages/pyVim/sso.py", line 613, in get_hok_saml_assertion hok_token = self.perform_request(soap_message, public_key, private_key, File "/usr/lib/vmware/site-packages/pyVim/sso.py", line 301, in perform_request raise Exception("Got response %s: %s\n%s" %Exception: Got response 503: content-length: 1 Reference Screenshot Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
Cause
This error is seen if the start operation using service-control enters a rare race condition during the patch workflow
Resolution
Broadcom VMware Engineering is aware of this issue and working to resolve this in a future release
Workaround
In order to resolve the issue, perform one of the below Log in to the VCSA using sshStart the services manually using the below commands service-control --start lookupsvc sts Access the VAMI (vCenter Appliance Management Interface) and click Resume vCenter update If Resume vCenter update is not available, select Rollback to the previous version and then restart the Patching via VAMI.Note: In case if the above steps is not recovering from the failed state, proceed with restore of the vCenter server. In case if vCenter is in ELM (enhanced linked mode), then all the nodes must be restored from backup. If the failed vCenter server is part of VCF instance, engage technical support for assistance with recovery.
