RHEL7 NIC is UP but not RUNNING

Programmer that has to look at a server for some issues, in need of help.

Our configuration is clients connected to a Dell S5000 switch, 2 identical servers (minus recently replaced NIC in one server) connected to each other via Cat5e and connected to another Dell S5000 via fiber lines, and the 2 Dell S5000's connected to each other via a single fiber line. Attempted diagram:

                   Clients 
                      |
                      F
                      |
              Dell S5000 switch
                      |
                      F
                      |
              Dell S5000 switch
                 /         \
                F           F
               /             \
           Server ----E---- Server

We had a NIC go bad (crashes on boot if in the system) so we replaced it with another (not same model). Grabbed the docs for it which led me to this guide from Broadcom which states it is supposed to use the tg3 driver.

An ifconfig -a shows that the NIC is p1p1:

# ifconfig p1p1
p1p1 : flags=4099<UP,BROADCAST,MULTICAST> mut 1500
       inet 192.168.0.41 netmask 255.255.255.0 broadcast 192.168.0.255
       inet6 fe80::eecd:6dff:febb:dc67 prefixlen 64 scopeid 0x20<link>
       ether ee:cd::6d:bb:dc:67 txqueuelen 1000 (Ethernet)
       # bunch of packet data here with all 0's
       device interrupt 48

From the client machines I can do an arp -n and ip neigh to see that they in the past talked to 192.168.0.41 but it is now unreachable, so I know this IP must be correct, the NIC is not up and talking to the switch. The inet6 and ether entries also contain a hex address that is talked about below.

I checked that it was using tg3 with ethtool -i p1p1 and it was:

# ethool -i p1p1
driver: tg3
version: 3.137
firmware-version: 578-v1.58
bus-info: 0000:0b:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no

# ethtool p1p1
Settings for p1p1
    Supported ports: [FIBRE]
    Supported link modes: 1000 baseT/Half 1000 baseT/Full
    Supported pause frame use: No
    Advertised link modes: 1000 baseT/Half 1000 baseT/Full
    Advertised pause frame use: Symmetric
    Advertised auto-negotiation: Yes
    Speed: Unknown!
    Duplex: Unknown!
    Port: FIBRE
    PHYAD: 8
    Transciever: Internal
    Auto-negotiation: on
    Supports wake-on: d
    Wake-on: d
    Current message level: 0x000000ff (255)
                           drv probe link timer ifdown ifup rx_err tx_err
    Link detected: no

Whats interesting here is that the other server (of identical configuration; but with original NIC) is running:

# ethtool -i p1p1
driver: bnx2x
...
supports-priv-flags: yes

# ethtool p1p1
...
Current message level: 0x00000000 (0)
Link detected: yes

Not sure where the IP listed in the ifconfig came from originally, but the only time I found it was in /etc/sysconfig/networking-scripts/ifcgf-p1p1. Here the following information was found:

TYPE=Ethernet
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT_yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME=p1p1
UUID=32038ebe-554e-4e6a-904b-5fcccf2a23be2
ONBOOT=yes
IPV6_PEERDNS=yes
IPV6_PERRROUTES=yes
HWADDR=EC:CD:6D:BB:DC:67
IPADDR0=192.168.0.41
PREFIX0=24
GATEWAY0=192.168.0.41

Here, HWADDR refers to the MAC listed on the back of the NIC device for the new card. I put it here when I pulled the old card which had its MAC for the port the fiber line was connected to. In the same file for the other server, the BOOTPROTO is set to none; tried that but same result.

I can activate and deactivate the card using nmtui but the operational state still doesnt show.

I've changed out the fiber and still no change. Not sure what else to try.

Updates

• This page describes the problem well, but I am not a RHEL subscriber so I cannot see the solution. The dmesg | grep -i -e tg3 -e p1p1 shows IPV6 ADDRCONF(NETDEV_UP): p1p1: link is not ready.