Here is the issue description:
After linux bootup, running command "lspci -v", we can see that pci devices can be found, but no address is assigned to devices. By further checking the linux booting message, many "bridge window mem ..." like messages are found there. Not sure those messages are just normal or actually a sign of the address assignment failure happening.
The processor I'm running is Freescale p2010, which is powerpc architecture. The bootloader is designed by ourselves and did not touch anything related with pci. So we totally rely on Linux kernel to do the complete pci enumeration by its own.
The only thing we did before Linux kernel pci enumeration starts is to configure a LAW window (specifically LAW4) for pci space usage. And besides, of course, appropriate modification to the dts file according to our hardware config. We thought with all those preparation done, Linux kernel should be able to bring all the pci stuff up to work, however, it did not.
So, does anyone know how to resolve this issues or ever ran into similar issues and can shed some light on it? Thank you very much!
More info attached below:
========Kernel boot args===============
root=/dev/ram rw ip=off console=ttyS0,9600
========PCI section setup in dts=======
pci0: pcie@fbf0a000 {
interrupt-map-mask = <0xf800 0x0 0x0 0x7>;
interrupt-map = <
00000 0x0 0x0 0x1 &mpic 0x0 0x1
00000 0x0 0x0 0x2 &mpic 0x1 0x1
00000 0x0 0x0 0x3 &mpic 0x2 0x1
00000 0x0 0x0 0x4 &mpic 0x3 0x1>;
interrupt-parent = <&mpic>;
interrupts = <26 2>;
bus-range = <0 255>;
ranges = <0x2000000 0x0 0x80000000 0x80000000 0x0 0x40000000
0x1000000 0x0 0x0 0xd3000000 0x0 0x100000>;
clock-frequency = <33333333>;
#interrupt-cells = <1>;
#size-cells = <2>;
#address-cells = <3>;
reg = <0xfbf0a000 0x1000>;
compatible = "fsl,mpc8548-pcie";
device_type = "pci";
fsl,pcie-access-big-endian;
pcie@0 {
reg = <0x0 0x0 0x0 0x0 0x0>;
#size-cells = <2>;
#address-cells = <3>;
device_type = "pci";
ranges = <0x2000000 0x0 0x80000000
0x2000000 0x0 0x80000000
0x0 0x40000000
0x1000000 0x0 0x0
0x1000000 0x0 0x0
0x0 0x100000>;
};
};
==============result of "lspci -v"===============
# lspci -v
0000:00:00.0 Class 0b20: 1957:007b (rev 21)
!!! Invalid class 0b20 for header type 01
Flags: bus master, fast devsel, latency 0
Memory at <ignored> (32-bit, non-prefetchable)
Bus: primary=00, secondary=01, subordinate=07, sec-latency=0
I/O behind bridge: 00000000-00000fff
Memory behind bridge: 00000000-000fffff
Prefetchable memory behind bridge: 0000000000000000-0000000000000000
Capabilities: [44] Power Management version 2
Capabilities: [4c] #10 [0041]
0000:01:00.0 Class 0604: 10b5:8606 (rev ba)
Flags: bus master, fast devsel, latency 0
Bus: primary=01, secondary=02, subordinate=07, sec-latency=0
Capabilities: [40] Power Management version 3
Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/2 Enable+
Capabilities: [68] #10 [0052]
Capabilities: [a4] #0d [0000]
0000:02:01.0 Class 0604: 10b5:8606 (rev ba)
Flags: bus master, fast devsel, latency 0
Bus: primary=02, secondary=03, subordinate=03, sec-latency=0
Capabilities: [40] Power Management version 3
Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/2 Enable+
Capabilities: [68] #10 [0162]
Capabilities: [a4] #0d [0000]
0000:02:04.0 Class 0604: 10b5:8606 (rev ba)
Flags: bus master, fast devsel, latency 0
Bus: primary=02, secondary=04, subordinate=04, sec-latency=0
Capabilities: [40] Power Management version 3
Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/2 Enable+
Capabilities: [68] #10 [0162]
Capabilities: [a4] #0d [0000]
0000:02:05.0 Class 0604: 10b5:8606 (rev ba)
Flags: bus master, fast devsel, latency 0
Bus: primary=02, secondary=05, subordinate=05, sec-latency=0
Capabilities: [40] Power Management version 3
Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/2 Enable+
Capabilities: [68] #10 [0162]
Capabilities: [a4] #0d [0000]
0000:02:07.0 Class 0604: 10b5:8606 (rev ba)
Flags: bus master, fast devsel, latency 0
Bus: primary=02, secondary=06, subordinate=06, sec-latency=0
Capabilities: [40] Power Management version 3
Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/2 Enable+
Capabilities: [68] #10 [0162]
Capabilities: [a4] #0d [0000]
0000:02:09.0 Class 0604: 10b5:8606 (rev ba)
Flags: bus master, fast devsel, latency 0
Bus: primary=02, secondary=07, subordinate=07, sec-latency=0
Capabilities: [40] Power Management version 3
Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/2 Enable+
Capabilities: [68] #10 [0162]
Capabilities: [a4] #0d [0000]
0000:03:00.0 Class 0580: 10ee:7011
Subsystem: 10ee:0007
Flags: fast devsel, IRQ 17
Capabilities: [40] Power Management version 3
Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
Capabilities: [60] #10 [0002]
0000:04:00.0 Class 0580: 10ee:7011
Subsystem: 10ee:0007
Flags: fast devsel, IRQ 16
Capabilities: [40] Power Management version 3
Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
Capabilities: [60] #10 [0002]
0000:05:00.0 Class 0200: 14e4:8372 (rev 13)
Subsystem: 14e4:8372
Flags: fast devsel, IRQ 17
Memory at <ignored> (64-bit, non-prefetchable)
Capabilities: [48] Power Management version 3
Capabilities: [50] Vital Product Data
Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-
Capabilities: [ac] #10 [0002]
0000:06:00.0 Class 0200: 14e4:8372 (rev 13)
Subsystem: 14e4:8372
Flags: fast devsel, IRQ 18
Memory at <ignored> (64-bit, non-prefetchable)
Capabilities: [48] Power Management version 3
Capabilities: [50] Vital Product Data
Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-
Capabilities: [ac] #10 [0002]
#
=======================Linux kernel booting message====================
irq: irq 47 on host /soc@fbf00000/pic@40000 mapped to virtual irq 47
PCI: Probing PCI hardware
pci 0000:00:00.0: ignoring class b20 (doesn't match header type 01)
pci 0000:00:00.0: supports D1 D2
pci 0000:00:00.0: PME# supported from D0 D1 D2 D3hot D3cold
pci 0000:00:00.0: PME# disabled
pci 0000:01:00.0: reg 10: [mem 0x00000000-0x0001ffff]
pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
pci 0000:01:00.0: PME# disabled
pci 0000:00:00.0: PCI bridge to [bus 01-ff]
pci 0000:00:00.0: bridge window [io 0x0000-0x0000] (disabled)
pci 0000:00:00.0: bridge window [mem 0x00000000-0x000fffff] (disabled)
pci 0000:00:00.0: bridge window [mem 0x00000000-0x000fffff pref] (disabled)
irq: irq 0 on host /soc@fbf00000/pic@40000 mapped to virtual irq 16
pci 0000:02:01.0: PME# supported from D0 D3hot D3cold
pci 0000:02:01.0: PME# disabled
pci 0000:02:04.0: PME# supported from D0 D3hot D3cold
pci 0000:02:04.0: PME# disabled
pci 0000:02:05.0: PME# supported from D0 D3hot D3cold
pci 0000:02:05.0: PME# disabled
pci 0000:02:07.0: PME# supported from D0 D3hot D3cold
pci 0000:02:07.0: PME# disabled
pci 0000:02:09.0: PME# supported from D0 D3hot D3cold
pci 0000:02:09.0: PME# disabled
pci 0000:01:00.0: PCI bridge to [bus 02-ff]
pci 0000:01:00.0: bridge window [io 0xf000-0x0000] (disabled)
pci 0000:01:00.0: bridge window [mem 0xfff00000-0x000fffff] (disabled)
pci 0000:01:00.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
irq: irq 1 on host /soc@fbf00000/pic@40000 mapped to virtual irq 17
irq: irq 3 on host /soc@fbf00000/pic@40000 mapped to virtual irq 18
pci 0000:03:00.0: reg 10: [mem 0x00000000-0x00003fff]
pci 0000:03:00.0: PME# supported from D0 D1 D2 D3hot
pci 0000:03:00.0: PME# disabled
pci 0000:02:01.0: PCI bridge to [bus 03-ff]
pci 0000:02:01.0: bridge window [io 0xf000-0x0000] (disabled)
pci 0000:02:01.0: bridge window [mem 0xfff00000-0x000fffff] (disabled)
pci 0000:02:01.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:04:00.0: reg 10: [mem 0x00000000-0x00003fff]
pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot
pci 0000:04:00.0: PME# disabled
pci 0000:02:04.0: PCI bridge to [bus 04-ff]
pci 0000:02:04.0: bridge window [io 0xf000-0x0000] (disabled)
pci 0000:02:04.0: bridge window [mem 0xfff00000-0x000fffff] (disabled)
pci 0000:02:04.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:05:00.0: reg 10: [mem 0x80400000-0x8043ffff 64bit]
pci 0000:05:00.0: PME# supported from D0 D3hot D3cold
pci 0000:05:00.0: PME# disabled
pci 0000:02:05.0: PCI bridge to [bus 05-ff]
pci 0000:02:05.0: bridge window [io 0xf000-0x0000] (disabled)
pci 0000:02:05.0: bridge window [mem 0xfff00000-0x000fffff] (disabled)
pci 0000:02:05.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:06:00.0: reg 10: [mem 0x80500000-0x8053ffff 64bit]
pci 0000:06:00.0: PME# supported from D0 D3hot D3cold
pci 0000:06:00.0: PME# disabled
pci 0000:02:07.0: PCI bridge to [bus 06-ff]
pci 0000:02:07.0: bridge window [io 0xf000-0x0000] (disabled)
pci 0000:02:07.0: bridge window [mem 0xfff00000-0x000fffff] (disabled)
pci 0000:02:07.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:02:09.0: PCI bridge to [bus 07-ff]
pci 0000:02:09.0: bridge window [io 0xf000-0x0000] (disabled)
pci 0000:02:09.0: bridge window [mem 0xfff00000-0x000fffff] (disabled)
pci 0000:02:09.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
PCI: Cannot allocate resource region 0 of device 0000:05:00.0, will remap
PCI: Cannot allocate resource region 0 of device 0000:06:00.0, will remap
pci 0000:01:00.0: BAR 8: can't assign mem (size 0x400000)
pci 0000:01:00.0: BAR 0: can't assign mem (size 0x20000)
pci 0000:02:01.0: BAR 8: can't assign mem (size 0x100000)
pci 0000:02:04.0: BAR 8: can't assign mem (size 0x100000)
pci 0000:02:05.0: BAR 8: can't assign mem (size 0x100000)
pci 0000:02:07.0: BAR 8: can't assign mem (size 0x100000)
pci 0000:03:00.0: BAR 0: can't assign mem (size 0x4000)
pci 0000:02:01.0: PCI bridge to [bus 03-03]
pci 0000:02:01.0: bridge window [io disabled]
pci 0000:02:01.0: bridge window [mem disabled]
pci 0000:02:01.0: bridge window [mem pref disabled]
pci 0000:04:00.0: BAR 0: can't assign mem (size 0x4000)
pci 0000:02:04.0: PCI bridge to [bus 04-04]
pci 0000:02:04.0: bridge window [io disabled]
pci 0000:02:04.0: bridge window [mem disabled]
pci 0000:02:04.0: bridge window [mem pref disabled]
pci 0000:05:00.0: BAR 0: can't assign mem (size 0x40000)
pci 0000:02:05.0: PCI bridge to [bus 05-05]
pci 0000:02:05.0: bridge window [io disabled]
pci 0000:02:05.0: bridge window [mem disabled]
pci 0000:02:05.0: bridge window [mem pref disabled]
pci 0000:06:00.0: BAR 0: can't assign mem (size 0x40000)
pci 0000:02:07.0: PCI bridge to [bus 06-06]
pci 0000:02:07.0: bridge window [io disabled]
pci 0000:02:07.0: bridge window [mem disabled]
pci 0000:02:07.0: bridge window [mem pref disabled]
pci 0000:02:09.0: PCI bridge to [bus 07-07]
pci 0000:02:09.0: bridge window [io disabled]
pci 0000:02:09.0: bridge window [mem disabled]
pci 0000:02:09.0: bridge window [mem pref disabled]
pci 0000:01:00.0: PCI bridge to [bus 02-07]
pci 0000:01:00.0: bridge window [io disabled]
pci 0000:01:00.0: bridge window [mem disabled]
pci 0000:01:00.0: bridge window [mem pref disabled]
pci 0000:00:00.0: not setting up bridge for bus 0000:01
pci_bus 0000:00: resource 0 [io 0xffafd000-0xffbfcfff]
pci_bus 0000:00: resource 1 [mem 0x80000000-0xbfffffff]
Registering qe_ic with sysfs...
bio: create slab <bio-0> at 0
vgaarb: loaded
Generic PHY: Registered new driver
On one of our SuperMicro: enable following BIOS options under:
Advanced -> PCIe/PCI/PnP Configuration:
PCI PERR/SERR Support - Enabled - event log pci errors.
Above 4G Decoding - Enabled - 64bit O/S and many pcie devices.
SR-IOV Support - Enabled - Linux gets dmesg "no space for" errs.
The first is a suggestion ...
User contributions licensed under CC BY-SA 3.0