Operational Defect Database
BugZero found this defect 2833 days ago.
Data sources
All data on this page is proprietary to BugZero® or gathered from public sources
Hewlett Packard Enterprise | c05239518
Advisory: HPE B-Series Switches - Switches Or Control Processor Blade May Hang If Real Time Clock Access Is Aborted
Last update date:
2/28/2024
Affected products:
HPE 8/80 SAN Switch
HPE B-series SN6500B Fibre Channel Switch
HPE Encryption SAN Switch
HPE Storage SAN Director Switch
Affected releases:
No affected releases provided.
Fixed releases:
No fixed releases provided.
Description:
Info
An HPE B-series switch or Control Processor (CP) blade hang may be encountered during a power-on or reboot, on any switch or director listed in the Scope section. This issue occurs when access to the Real Time Clock (RTC) is aborted, followed by a restart of the switch or CP blade which leads to the hang condition of the switch or CP blade.
Scope
Brocade FOS versions 8.0.1a, 8.0.1, 7.4.0 through 7.4.1c, 7.3.1 through 7.3.2a, and 7.2.1e and later are impacted. The issue is a corner case scenario that has an increased risk of occurring on switches and CP blades that have Network Time Protocol (NTP) service enabled and are running a release with the fix for DEFECT000501658, which introduced an abort of the Real Time Clock (RTC) access. This change was first introduced in Fabric OS (FOS) versions 7.2.1e, 7.3.1, 7.4.0, and 8.0.1. The affected units will depict the following symptoms: /sbin/hwclock command execution will fail with the following response: RTC_RD_TIME: Bad address ioctl() to /dev/rtc to read the time failed. If the unit depicting the above failure is rebooted, it will remain hung in the ensuing boot up phase with the serial console displaying only the following messages: "The system is coming up, please wait..." "The system is coming up, please wait..."
Resolution
Perform the Workaround, then upgrade to FOS version 7.3.2b (September 2016), 7.4.1d, 8.0.1b (September 2016), or later to obtain a fix for DEFECT000594896. Workaround For Director class switches Prior to upgrading or high availability (HA) failover execution, check all CPs (active and standby) to verify the hardware clock command is functioning properly by logging in as root and running the following command: /sbin/hwclock It should respond as expected and return the time as in the example below: DCX:root> /sbin/hwclock Thu Mar 31 16:58:29 2016 -0.000354 seconds If it encounters a failure and responds as shown below, then any subsequent reboot of that CP blade will most likely result in a hang of the corresponding CP blade during the boot up: DCX:root> /sbin/hwclock RTC_RD_TIME: Bad address ioctl() to /dev/rtc to read the time failed. NOTE: Log out of the root account after running the /sbin/hwclock command to verify hardware clock functionality. NOTE: If both CPs are affected, correct the standby CP prior to correcting the active CP. For those CPs exhibiting this RTC access issue, the following steps are recommended: If it is encountered on a standby CP: Reseat the standby CP blade. After HA is back in sync, verify the RTC issue is corrected by running the /sbin/hwclock command on both CPs. If it returns time as expected, then proceed with the FOS upgrade, if the active CP blade is not affected. If this is encountered on the Active CP and HA is in sync: Perform a HA failover. Upon completion, this (previously active) CP will probably hang during a power up. If it does, reseat this CP blade. After HA is back in sync, verify the RTC issue is corrected by running the /sbin/hwclock command on both CPs. If it returns time as expected, proceed with the FOS upgrade. If it is encountered on the Active CP and HA is not in sync: Obtain a maintenance window and reseat the active CP blade. After the blade powers back up, verify the RTC issue is corrected by running the /sbin/hwclock command on both CPs. If it returns time as expected, proceed with the FOS upgrade. For Non-Director class switches Prior to upgrading FOS or an HA reboot, log into the switch as root , and run the following command: /sbin/hwclock It should respond as expected and return the time as in the example below: SN6500B:root> /sbin/hwclock Thu Mar 31 16:58:29 2016 -0.000354 seconds If it returns the following, then a reboot will most likely hang the switch during the boot up process: SN6500B:root> /sbin/hwclock RTC_RD_TIME: Bad address ioctl() to /dev/rtc to read the time failed. NOTE: Log out of the root account after running the /sbin/hwclock command If the switch exhibits this issue, schedule a maintenance window and power cycle the switch. Then proceed with the FOS upgrade. RECEIVE PROACTIVE UPDATES : Receive support alerts (such as Customer Advisories), as well as updates on drivers, software, firmware, and customer replaceable components, proactively in your e-mail through HPE Subscriber's Choice. Sign up for Subscriber's Choice at the following URL: Proactive Updates Subscription Form
