Operational Defect Database
BugZero found this defect 1060 days ago.
Data sources
All data on this page is proprietary to BugZero® or gathered from public sources
Hewlett Packard Enterprise | a00115963en_us
Advisory: HPE BladeSystem - c-Class Server Blades May Power Off After an Onboard Administrator Reboot, and Will Not Power On Without a Reset
Last update date:
3/4/2024
Affected products:
HPE BladeSystem c7000 Carrier-grade Enclosures
HPE BladeSystem c7000 Enclosures
HPE Integrity BladeSystem c7000 Enclosures
HPE Onboard Administrator
Affected releases:
No affected releases provided.
Fixed releases:
No fixed releases provided.
Description:
Info
After an HPE c-Class BladeSystem Onboard Administrator (OA) is rebooted, which will also occur after updating OA firmware, some server blades may power off, reporting the following in the Integrated Management Log (IML): Critical,1041,78,0x0014,System Error,,,09/05/2020 04:38:39,194: Server Critical Fault (Service Information: Runtime Fault System Board AUX/Main EFUSE Regulator 1 (01h) P12V Main/AUX Regulator 1 (04h)) In addition, the OA reports this in its syslog: Sep 4 14:50:05 OA: Blade in bay #x status changed to Failed (Note: x can be any valid bay) When this occurs, affected server blades will be off, and will have a red "X" mark next to them in the Onboard Administrator GUI, and those blades must either be reseated or have an e-fuse reset via telnet or SSH session to the OA Command Line Interface (CLI) in order for them to power back on and operate normally. This occurs because Dynamic Power Savings is temporarily enabled while the OA Tray reboots. Disabling Dynamic Power Savings will not prevent this issue from occurring during an OA tray reboot, until the resolution is performed.
Scope
Any HPE Bladesystem c7000 or c3000 enclosure.
Resolution
An updated version of OA tray firmware is included in OA firmware version 4.97. Upgrade to OA firmware 4.97, and then the OA Tray firmware (also called Power PIC firmware) may be updated. Updated OA tray firmware (Power PIC firmware) is included with OA 4.97, but is not visible by default. OA tray firmware must be updated to the new versions as noted below to prevent this issue from occurring. Note: OA tray firmware version numbering is not identical between c7000 enclosures and c3000 enclosures. The version of tray firmware on a c7000 prior to this update is version 1.7, and once updated will be version 1.8. For c3000 enclosures, OA tray firmware prior to this update will be version 2.10, and after this update, will be version 2.11. In order to update the OA Tray firmware it is necessary to perform the following: 1. Telnet or SSH into the OA CLI, log in, and issue the following commands: show update tray If this is done immediately after updating OA firmware to 4.97 for the first time, it is expected that the affected firmware versions will be listed both in the "Version" and "NewVersion" fields. upgrade powerpic force 2. After upgrading the OA tray firmware, issue the following command: show update tray The new version of OA tray (version is dependent upon model of enclosure, see above) will be listed in the "Version" field, but the previous version will still be listed in the "NewVersion" field. The OA syslog will record the following log entries for each operation (note that the versions reported will be as noted above for c3000; log entries for c7000 are displayed below): upgrade powerpic force OA: TRAY microcode updated from version 1.7 to version 1.8 downgrade powerpic OA: TRAY microcode updated from version 1.8 to version 1.7 If at any time the OA tray firmware needs to be downgraded back to the older version, issue this command from the SSH or telnet console: downgrade powerpic Enter the "show power" command to verify (Dynamic Power) is disabled. If needed, enter command "set power savings off" to disable the Dynamic Power. FLASH NOTES: Once completed, the Power PIC FW update takes effect immediately, no rebooting or power-cycling is required of any component of the enclosure. The Power PIC update will not disrupt any enclosure operations. RECEIVE PROACTIVE UPDATES : Receive support alerts (such as Customer Advisories), as well as updates on drivers, software, firmware, and customer replaceable components, proactively in your e-mail through HPE Support Alerts. Sign up for Support Alerts at the following URL: Proactive Updates Subscription Form. NAVIGATION TIP: For hints on navigating HPE.com to locate the latest drivers, patches and other support software downloads, refer to the Navigation Tips document. SEARCH TIP: For hints on locating similar documents on HPE.com, refer to the Search Tips document.
