Operational Defect Database
BugZero found this defect 45 days ago.
Data sources
All data on this page is proprietary to BugZero® or gathered from public sources
Hewlett Packard Enterprise | a00138650en_us
Advisory: HPE M-Series ONYX Switches - Link Partner Forward Error Correction (FEC) Configurations May Cause Packet, Performance, And Connectivity Issues On 25GbE Links
Last update date:
4/5/2024
Affected products:
HPE M-series SN2010M Switch
HPE M-series SN2100M Switch
HPE M-series SN2410M Switch
HPE M-series SN2700M Switch
HPE M-series SN3700M Switch
HPE Storage Switch M-series SN4600M
Affected releases:
No affected releases provided.
Fixed releases:
No fixed releases provided.
Description:
Info
Devices connected to HPE M-series ONYX switches via 25GbE, 50GbE, 100GbE, or 200GbE links are required to operate with forward error correction (FEC). Links with speeds of 10G and 40G do not require operational FEC. Forward Error Correction (FEC) is a technique used to detect and correct a certain number of errors in a bit stream by appending redundant bits and error-checking code to the message block before transmission. The addition contains sufficient information on the actual data to enable the FEC decoder at the receiver end to reconstruct the original message; the FEC decoder can identify the bits received in error and correct them. It then removes the redundant bits before passing the message to the upper layers of the network. Because the FEC decoder uses only the redundant bits to detect and correct errors, it does not request retransmission of the entire errored frame, saving bandwidth that would otherwise be used for retransmission. The IEEE 802.3 standard defines KR-FEC used for NRZ signals, such as 25GbE and 4x25G=100GbE. Reed-Solomon forward error correction (RS-FEC) is recommended and qualified for 25GbE optical and copper links with HPE M-series switches. By default, M-series switches auto-negotiate operational RS-FEC with a link partner. With use of short copper direct-access cables (DACs), RS-FEC may be negotiated; however, operational RS-FEC should continue to be configured at both ends. If the link partner has FEC disabled or is unable to negotiate matching FEC type due to copper DAC cables or special optical transceiver or AOC configurations with built in FEC, packet errors (e.g. Rx fcs and symbol errors) followed by performance and connectivity issues will occur. HPE M-series switch ports are configured with "auto fec" by default, which is negotiated along with the link speed. Device link partners should support "auto fec" which is sometimes displayed as "CL74 and CL91 – Both" or have the configuration option to enable "RS-FEC or fc-fec", so that the link operational fec is RS-FEC or fc-fec. The M-series ONYX switch CLI enables each port to support Auto, RS-FEC, fc-fec, or no-fec, as shown below: (config) # interface ethernet 1/1 fec-override ? no-fec Do not use FEC fc-fec FireCode FEC rs-fec Reed Solomon FEC (config) # interface ethernet 1/1 fec-override rs-fec force To revert the configuration to auto-fec, use the CLI no fec-override force command: (config) # interface ethernet 1/1 no fec-override force Some multi-speed adapters have auto-fec enabled by default while others (e.g. Broadcom 1/10/25G adapters) have by default fec disabled. If "auto fec" is allowed in the device “Link FEC” option, choose "Auto" for 25GbE link speed. If "Auto" is not allowed in the "Link FEC" option, choose "CL91 – Reed Solomon" or "RS-FEC"> for 25GbE link speed. For example 1, on the figures below, in the iLO-displayed Device Configure Menu of a Broadcom adapter, change the "Link FEC" option from " Disabled " to the " CL91 - Reed Solomon " option. This change may also require changing the HPE M-series port auto-fec to a fec-override rs-fec configuration. Also for most Broadcom 25G adapter iSCSI applications, disable the "DCB Protocol" if it is not disabled by default: The DCB Protocol is shown as "enabled": For example 2, on the figure below, in the iLO-displayed Device Configure Menu of a Broadcom adapter, change the 25G "Link FEC" option from " Disabled " to the " Reed-Solomon code " option: Once the Fec settings are configured on the 25GbE links, verify that "rs-fec" is shown on the "Operational Fec" switch port. (config) # show interfaces ethernet 1/n Eth1/n: Admin state : Enabled Operational state : Up Last change in operational status: 1w 0d and 0:00:59 ago (5 oper change) Boot delay time : 0 sec Description : N/A Mac address : 1c:34:da:f1:df:be MTU : 9216 bytes (Maximum packet size 9238 bytes) Fec : auto Operational Fec : rs-fec Flow-control : receive on send on Supported speeds : 1G 10G 25G Advertised speeds : 1G 10G 25G Actual speed : 25G (auto) Auto-negotiation : Enabled Width reduction mode : Unknown Switchport mode : access MAC learning mode : Enabled Forwarding mode : inherited cut-through Telemetry sampling: Disabled TCs: N/A Telemetry threshold: Disabled TCs: N/A Telemetry threshold level: N/A Last clearing of "show interface" counters: Never 60 seconds ingress rate : 64 bits/sec, 8 bytes/sec, 1 packets/sec 60 seconds egress rate : 312 bits/sec, 39 bytes/sec, 1 packets/sec Rx: 103046 packets 0 unicast packets 103046 multicast packets 0 broadcast packets 19266888 bytes 0 discard packets 0 error packets 0 fcs errors 0 undersize packets 0 oversize packets 0 pause packets 0 unknown control opcode 0 symbol errors 0 discard packets by storm control Tx: 1641207 packets 0 unicast packets 1641207 multicast packets 0 broadcast packets 117297580 bytes 0 discard packets 0 error packets 0 hoq discard packets 0 ECN marked packets Then, clear the port counters and monitor for “fcs error” or “symbol error” counts by running the “show interface ethernet 1/n (n is port #). Monitor for some time and verify that Rx: fcs and symbol error counts are zero. (config) # clear counters interface ethernet 1/n If the display shows Operational Fec: no-fec, then the server adapter did not negotiate FEC with the M-series switch port. Eth1/n: Admin state : Enabled Operational state : Up Last change in operational status: 1w 0d and 0:12:29 ago (3 oper change) Boot delay time : 0 sec Description : 25G-link Mac address : 1c:34:da:e9:10:ae MTU : 9216 bytes (Maximum packet size 9238 bytes) Fec : auto Operational Fec : no-fec Flow-control : receive on send on Supported speeds : 1G 10G 25G Advertised speeds : 1G 10G 25G Actual speed : 25G (auto) Auto-negotiation : Enabled Width reduction mode : Unknown Switchport mode : access MAC learning mode : Enabled Forwarding mode : inherited store-and-forward Example of a 25GbE smart AOC Q9S67A which supports fc-fec: Eth1/n: Admin state : Enabled Operational state : Up Last change in operational status: 1w 0d and 0:12:29 ago (3 oper change) Boot delay time : 0 sec Description : N/A Mac address : 1c:34:da:e9:10:ae MTU : 9216 bytes (Maximum packet size 9238 bytes) Fec : auto Operational Fec : fc-fec Flow-control : receive on send on Supported speeds : 1G 10G 25G Advertised speeds : 1G 10G 25G Actual speed : 25G (auto) Auto-negotiation : Enabled Width reduction mode : Unknown Switchport mode : access MAC learning mode : Enabled Forwarding mode : inherited store-and-forward
Scope
The following HPE M-Series ONYX switches with configured speed 25GbE native or breakout ports are affected by this issue: HPE SN2100M 100GbE 16QSFP28 Half Width Switch (Q2F23A) HPE SN2100M 100GbE 8QSFP28 Half Width Switch (Q2F24A) HPE SN2100M 100GbE 16QSFP28 Power to Connector Airflow Half TAA-compliant (R0P75A) HPE SN2100M 100GbE 16QSFP28 Connector to Power Airflow Half TAA-compliant (R0P76A) HPE SN2010M 25GbE 18SFP28 4QSFP28 Switch [Q9E63A) HPE SN2010M 25GbE 18SFP28 4QSFP28 Power to Connector Airflow TAA-compliant (R0P77A) HPE SN2010M 25GbE 18SFP28 4QSFP28 Connector to Power Airflow TAA-compliant (R0P78A) HPE SN2700M 100GbE 32QSFP28 Switch (Q2F21A) HPE SN2700M 100GbE 32QSFP28 Power to Connector Airflow TAA-compliant (R0P71A) HPE SN2700M 100GbE 32QSFP28 Connector to Power Airflow TAA-compliant (R0P72A) HPE SN2700M 100GbE 16QSFP28 Switch (Q6M26A) HPE SN2410M 25GbE 48SFP28 8QSFP28 Switch (Q2F22A) HPE SN2410M 25GbE 48SFP28 8QSFP28 Power to Connector Airflow TAA-compliant (R0P73A) HPE SN2410M 25GbE 48SFP28 8QSFP28 Connector to Power Airflow TAA-compliant (R0P74A) HPE SN2410bM 10GbE 48SFP+ 8QSFP28 Switch (Q6M28A) HPE SN2410M 25GbE 24SFP28 4QSFP28 Switch (Q6M27A) HPE SN2410bM 10GbE 24SFP+ 4QSFP28 Switch (Q6M29A) HPE SN3700cM 100GbE 32QSFP28 Power to Connector (P2C) Airflow Switch (R3B14A) HPE SN4600cM 100GbE 64QSFP28 Power to Connector (P2C) Airflow Switch (R6R24A) HPE SN3700M 200GbE 32QSFP28 Power to Connector (P2C) Airflow Switch (R5Z74A)
Resolution
The proper operational FEC setting is required for 25GbE M-series switch ports. The correct FEC for a link shows operational "rs-fec or fc-fec", indicating the switch and link partner have negotiated the correct version. If this is not the case, correct the FEC setting on all adapter ports and, if necessary, the switch port using the procedure described above. Open a case to the HPE Support team if further assistance is needed. RECEIVE PROACTIVE UPDATES : Receive support alerts (such as Customer Advisories), as well as updates on drivers, software, firmware, and customer replaceable components, proactively in your e-mail through HPE Support Alerts. Sign up for Support Alerts at the following URL: HPE Email Preference Center
