Unable to get link lights on Chelsio T520-SO using SFP+

0

I'm new to working with fiber so please bear with me.

I've got a Dell R520 that I've just installed a Chelsio T520-SO fiber adapter into. The server is running Ubuntu 18. I've got a compatible Chelsio SM10G-SR:10G transceiver on the adapter side and the switch (HP5406) which is using a HP short-range 10G transceiver. Between the transceivers, I have a 3-meter aqua fiber cable. I have another server with the exact same transceivers, T520-SO Chelsio card, and the same exact cable that the previous administrator installed, which is running great. I've even tried swapping everything between the existing server and the new server, but have been unable to establish an active link or get link lights while in Ubuntu.

Ubuntu sees the card and interfaces just fine. When I was in the Dell UEFI BIOS and viewed the network adapters, the switch link light lit up. Not sure if this meant the link was good or not.

I've installed the Chelsio Unified Wire drivers, configured link speeds to 10G, disabled auto-negotiation, and disabled FEC per documentation found here, I.3 and I.4 sections: https://service.chelsio.com/store2/T5/Unified%20Wire/Linux/ChelsioUwire-3.13.0.1/Chelsio-UnifiedWire-Linux-UserGuide.pdf

Here's some additional information:

:/lib/firmware/cxgb4# lshw -short | grep network
/0/100/1/0           eno1        network    NetXtreme II BCM5716 Gigabit Ethernet
/0/100/1/0.1         eno2        network    NetXtreme II BCM5716 Gigabit Ethernet
/0/100/a/0                       network    T520-SO Unified Wire Ethernet Controller
/0/100/a/0.1                     network    T520-SO Unified Wire Ethernet Controller
/0/100/a/0.2                     network    T520-SO Unified Wire Ethernet Controller
/0/100/a/0.3                     network    T520-SO Unified Wire Ethernet Controller
/0/100/a/0.4         enp5s0f4    network    T520-SO Unified Wire Ethernet Controller
/1                   bond0       network    Ethernet interface
/2                   enp5s0f4d1  network    Ethernet interface



ethtool enp5s0f4d1
Settings for enp5s0f4d1:
        Supported ports: [ FIBRE ]
        Supported link modes:   1000baseT/Full
                                10000baseT/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: None
        Advertised link modes:  1000baseT/Full
                                10000baseT/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Advertised FEC modes: None
        Link partner advertised link modes:  Not reported
        Link partner advertised pause frame use: Symmetric
        Link partner advertised auto-negotiation: No
        Link partner advertised FEC modes: None
        Speed: Unknown!
        Duplex: Full
        Port: Other
        PHYAD: 255
        Transceiver: internal
        Auto-negotiation: on
        Current message level: 0x000000ff (255)
                               drv probe link timer ifdown ifup rx_err tx_err
        Link detected: no

/etc/network/interfaces
auto lo
iface lo inet loopback

auto enp5s0f4d1
iface enp5s0f4d1 inet static
address 10.1.5.4
subnet 255.255.255.0
gateway 10.1.5.1

Any clue where to go next? No network activity and it seems to assign it's IP just fine.

linux
ubuntu
linux-networking
switch
fiber
asked on Server Fault Aug 20, 2020 by emike09

0 Answers

Nobody has answered this question yet.


User contributions licensed under CC BY-SA 3.0