Kernel Newbies archive on lore.kernel.org
 help / color / Atom feed
* pcie: kernel log - BAR 15: no space for... BAR 15: failed to assign..
@ 2020-02-09  2:29 Muni Sekhar
  2020-02-10 22:28 ` Bjorn Helgaas
  0 siblings, 1 reply; 7+ messages in thread
From: Muni Sekhar @ 2020-02-09  2:29 UTC (permalink / raw)
  To: linux-pci, kernelnewbies

Hi all,

After rebooting the system following messages are seen in dmesg.
Not sure if these indicate a problem. Can some one look at these and
confirm if this is problem or can be ignored ?

Also any suggestions as to what would cause this?

[    1.084728] pci 0000:00:1c.0: BAR 15: no space for [mem size
0x00200000 64bit pref]
[    1.084813] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
0x00200000 64bit pref]
[    1.084890] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
[    1.084949] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
[    1.085037] pci 0000:00:1c.2: BAR 15: no space for [mem size
0x00200000 64bit pref]
[    1.085108] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
0x00200000 64bit pref]
[    1.085199] pci 0000:00:1c.3: BAR 15: no space for [mem size
0x00200000 64bit pref]
[    1.085270] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
0x00200000 64bit pref]
[    1.085343] pci 0000:00:1c.0: BAR 13: assigned [io  0x1000-0x1fff]
[    1.085403] pci 0000:00:1c.2: BAR 13: assigned [io  0x2000-0x2fff]
[    1.085470] pci 0000:00:1c.3: BAR 15: no space for [mem size
0x00200000 64bit pref]
[    1.085540] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
0x00200000 64bit pref]
[    1.085613] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
[    1.085672] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
[    1.085738] pci 0000:00:1c.2: BAR 15: no space for [mem size
0x00200000 64bit pref]
[    1.085808] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
0x00200000 64bit pref]
[    1.085884] pci 0000:00:1c.0: BAR 15: no space for [mem size
0x00200000 64bit pref]
[    1.085954] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
0x00200000 64bit pref]
[    1.086026] pci 0000:00:1c.0: PCI bridge to [bus 01]
[    1.086083] pci 0000:00:1c.0:   bridge window [io  0x1000-0x1fff]
[    1.086144] pci 0000:00:1c.0:   bridge window [mem 0xd0400000-0xd07fffff]


-- 
Thanks,
Sekhar

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: pcie: kernel log - BAR 15: no space for... BAR 15: failed to assign..
  2020-02-09  2:29 pcie: kernel log - BAR 15: no space for... BAR 15: failed to assign Muni Sekhar
@ 2020-02-10 22:28 ` Bjorn Helgaas
  2020-02-11 14:06   ` Muni Sekhar
  0 siblings, 1 reply; 7+ messages in thread
From: Bjorn Helgaas @ 2020-02-10 22:28 UTC (permalink / raw)
  To: Muni Sekhar; +Cc: linux-pci, kernelnewbies

On Sun, Feb 09, 2020 at 07:59:41AM +0530, Muni Sekhar wrote:
> Hi all,
> 
> After rebooting the system following messages are seen in dmesg.
> Not sure if these indicate a problem. Can some one look at these and
> confirm if this is problem or can be ignored ?
> 
> Also any suggestions as to what would cause this?
> 
> [    1.084728] pci 0000:00:1c.0: BAR 15: no space for [mem size
> 0x00200000 64bit pref]
> [    1.084813] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
> 0x00200000 64bit pref]
> [    1.084890] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
> [    1.084949] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
> [    1.085037] pci 0000:00:1c.2: BAR 15: no space for [mem size
> 0x00200000 64bit pref]
> [    1.085108] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
> 0x00200000 64bit pref]
> [    1.085199] pci 0000:00:1c.3: BAR 15: no space for [mem size
> 0x00200000 64bit pref]
> [    1.085270] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
> 0x00200000 64bit pref]
> [    1.085343] pci 0000:00:1c.0: BAR 13: assigned [io  0x1000-0x1fff]
> [    1.085403] pci 0000:00:1c.2: BAR 13: assigned [io  0x2000-0x2fff]
> [    1.085470] pci 0000:00:1c.3: BAR 15: no space for [mem size
> 0x00200000 64bit pref]
> [    1.085540] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
> 0x00200000 64bit pref]
> [    1.085613] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
> [    1.085672] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
> [    1.085738] pci 0000:00:1c.2: BAR 15: no space for [mem size
> 0x00200000 64bit pref]
> [    1.085808] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
> 0x00200000 64bit pref]
> [    1.085884] pci 0000:00:1c.0: BAR 15: no space for [mem size
> 0x00200000 64bit pref]
> [    1.085954] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
> 0x00200000 64bit pref]
> [    1.086026] pci 0000:00:1c.0: PCI bridge to [bus 01]
> [    1.086083] pci 0000:00:1c.0:   bridge window [io  0x1000-0x1fff]
> [    1.086144] pci 0000:00:1c.0:   bridge window [mem 0xd0400000-0xd07fffff]

The "no space" and "failed to assign" messages are all for bridge
windows (13 is the I/O window, 14 is the MMIO window, 15 is the MMIO
pref window).  I can't tell if you have any devices below these
bridges (lspci would show them).  If you don't have any devices below
these bridges, you can ignore the messages.

Bjorn

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: pcie: kernel log - BAR 15: no space for... BAR 15: failed to assign..
  2020-02-10 22:28 ` Bjorn Helgaas
@ 2020-02-11 14:06   ` Muni Sekhar
  2020-02-11 14:40     ` Bjorn Helgaas
  0 siblings, 1 reply; 7+ messages in thread
From: Muni Sekhar @ 2020-02-11 14:06 UTC (permalink / raw)
  To: Bjorn Helgaas; +Cc: linux-pci, kernelnewbies

On Tue, Feb 11, 2020 at 3:58 AM Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> On Sun, Feb 09, 2020 at 07:59:41AM +0530, Muni Sekhar wrote:
> > Hi all,
> >
> > After rebooting the system following messages are seen in dmesg.
> > Not sure if these indicate a problem. Can some one look at these and
> > confirm if this is problem or can be ignored ?
> >
> > Also any suggestions as to what would cause this?
> >
> > [    1.084728] pci 0000:00:1c.0: BAR 15: no space for [mem size
> > 0x00200000 64bit pref]
> > [    1.084813] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
> > 0x00200000 64bit pref]
> > [    1.084890] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
> > [    1.084949] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
> > [    1.085037] pci 0000:00:1c.2: BAR 15: no space for [mem size
> > 0x00200000 64bit pref]
> > [    1.085108] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
> > 0x00200000 64bit pref]
> > [    1.085199] pci 0000:00:1c.3: BAR 15: no space for [mem size
> > 0x00200000 64bit pref]
> > [    1.085270] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
> > 0x00200000 64bit pref]
> > [    1.085343] pci 0000:00:1c.0: BAR 13: assigned [io  0x1000-0x1fff]
> > [    1.085403] pci 0000:00:1c.2: BAR 13: assigned [io  0x2000-0x2fff]
> > [    1.085470] pci 0000:00:1c.3: BAR 15: no space for [mem size
> > 0x00200000 64bit pref]
> > [    1.085540] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
> > 0x00200000 64bit pref]
> > [    1.085613] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
> > [    1.085672] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
> > [    1.085738] pci 0000:00:1c.2: BAR 15: no space for [mem size
> > 0x00200000 64bit pref]
> > [    1.085808] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
> > 0x00200000 64bit pref]
> > [    1.085884] pci 0000:00:1c.0: BAR 15: no space for [mem size
> > 0x00200000 64bit pref]
> > [    1.085954] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
> > 0x00200000 64bit pref]
> > [    1.086026] pci 0000:00:1c.0: PCI bridge to [bus 01]
> > [    1.086083] pci 0000:00:1c.0:   bridge window [io  0x1000-0x1fff]
> > [    1.086144] pci 0000:00:1c.0:   bridge window [mem 0xd0400000-0xd07fffff]
>
> The "no space" and "failed to assign" messages are all for bridge
> windows (13 is the I/O window, 14 is the MMIO window, 15 is the MMIO
> pref window).  I can't tell if you have any devices below these
> bridges (lspci would show them).  If you don't have any devices below
> these bridges, you can ignore the messages.
I have the devices below these bridges. FPGA endpoint is connected to
'00:1c.0 PCI bridge' and Ethernet controller is connected to '00:1c.3
PCI bridge'.
Does these messages impact the functionality of the connected devices?
If so what kind of impact and is there any solution for this?
Also, I'd like to know why "no space" and "failed to assign" messages displayed?


>
> Bjorn



-- 
Thanks,
Sekhar

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: pcie: kernel log - BAR 15: no space for... BAR 15: failed to assign..
  2020-02-11 14:06   ` Muni Sekhar
@ 2020-02-11 14:40     ` Bjorn Helgaas
  2020-02-11 16:32       ` Muni Sekhar
  0 siblings, 1 reply; 7+ messages in thread
From: Bjorn Helgaas @ 2020-02-11 14:40 UTC (permalink / raw)
  To: Muni Sekhar; +Cc: linux-pci, kernelnewbies

On Tue, Feb 11, 2020 at 07:36:00PM +0530, Muni Sekhar wrote:
> On Tue, Feb 11, 2020 at 3:58 AM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > On Sun, Feb 09, 2020 at 07:59:41AM +0530, Muni Sekhar wrote:
> > > Hi all,
> > >
> > > After rebooting the system following messages are seen in dmesg.
> > > Not sure if these indicate a problem. Can some one look at these and
> > > confirm if this is problem or can be ignored ?
> > >
> > > Also any suggestions as to what would cause this?
> > >
> > > [    1.084728] pci 0000:00:1c.0: BAR 15: no space for [mem size
> > > 0x00200000 64bit pref]
> > > [    1.084813] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
> > > 0x00200000 64bit pref]
> > > [    1.084890] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
> > > [    1.084949] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
> > > [    1.085037] pci 0000:00:1c.2: BAR 15: no space for [mem size
> > > 0x00200000 64bit pref]
> > > [    1.085108] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
> > > 0x00200000 64bit pref]
> > > [    1.085199] pci 0000:00:1c.3: BAR 15: no space for [mem size
> > > 0x00200000 64bit pref]
> > > [    1.085270] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
> > > 0x00200000 64bit pref]
> > > [    1.085343] pci 0000:00:1c.0: BAR 13: assigned [io  0x1000-0x1fff]
> > > [    1.085403] pci 0000:00:1c.2: BAR 13: assigned [io  0x2000-0x2fff]
> > > [    1.085470] pci 0000:00:1c.3: BAR 15: no space for [mem size
> > > 0x00200000 64bit pref]
> > > [    1.085540] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
> > > 0x00200000 64bit pref]
> > > [    1.085613] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
> > > [    1.085672] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
> > > [    1.085738] pci 0000:00:1c.2: BAR 15: no space for [mem size
> > > 0x00200000 64bit pref]
> > > [    1.085808] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
> > > 0x00200000 64bit pref]
> > > [    1.085884] pci 0000:00:1c.0: BAR 15: no space for [mem size
> > > 0x00200000 64bit pref]
> > > [    1.085954] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
> > > 0x00200000 64bit pref]
> > > [    1.086026] pci 0000:00:1c.0: PCI bridge to [bus 01]
> > > [    1.086083] pci 0000:00:1c.0:   bridge window [io  0x1000-0x1fff]
> > > [    1.086144] pci 0000:00:1c.0:   bridge window [mem 0xd0400000-0xd07fffff]
> >
> > The "no space" and "failed to assign" messages are all for bridge
> > windows (13 is the I/O window, 14 is the MMIO window, 15 is the MMIO
> > pref window).  I can't tell if you have any devices below these
> > bridges (lspci would show them).  If you don't have any devices below
> > these bridges, you can ignore the messages.
>
> I have the devices below these bridges. FPGA endpoint is connected to
> '00:1c.0 PCI bridge' and Ethernet controller is connected to '00:1c.3
> PCI bridge'.
> Does these messages impact the functionality of the connected devices?

Yes.  We tried to allocate 0x00200000 of prefetchable MMIO to 00:1c.0
for use by the FPGA endpoint.  But this failed, so there is no
prefetchable MMIO available for the FPGA.  The 0xd0400000-0xd07fffff
non-prefetachable MMIO space *is* available for it.

Similarly, we were unable to allocate 0x00200000 of prefetchable MMIO
for the 00:1c.3 bridge for use by the ethernet controller.  I don't
know what non-prefetchable MMIO space was allocated or what the NIC
needs.

"lspci -v" will show you what the FPGA and the NIC need.

> If so what kind of impact and is there any solution for this?

> Also, I'd like to know why "no space" and "failed to assign"
> messages displayed?

These messages mean we tried to allocate space for the bridges but we
unable to do so.  This is because either the host bridge didn't have
big enough apertures (on x86, these usually come from ACPI _CRS
methods), or there's a bug in the Linux allocation algorithms.

I can't tell anything more without seeing the complete dmesg log,
which contains the host bridge aperture information and the BAR sizes
of the FPGA and NIC.

Bjorn

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: pcie: kernel log - BAR 15: no space for... BAR 15: failed to assign..
  2020-02-11 14:40     ` Bjorn Helgaas
@ 2020-02-11 16:32       ` Muni Sekhar
  2020-02-11 20:16         ` Bjorn Helgaas
  0 siblings, 1 reply; 7+ messages in thread
From: Muni Sekhar @ 2020-02-11 16:32 UTC (permalink / raw)
  To: Bjorn Helgaas; +Cc: linux-pci, kernelnewbies

[-- Attachment #1: Type: text/plain, Size: 4478 bytes --]

On Tue, Feb 11, 2020 at 8:10 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> On Tue, Feb 11, 2020 at 07:36:00PM +0530, Muni Sekhar wrote:
> > On Tue, Feb 11, 2020 at 3:58 AM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > > On Sun, Feb 09, 2020 at 07:59:41AM +0530, Muni Sekhar wrote:
> > > > Hi all,
> > > >
> > > > After rebooting the system following messages are seen in dmesg.
> > > > Not sure if these indicate a problem. Can some one look at these and
> > > > confirm if this is problem or can be ignored ?
> > > >
> > > > Also any suggestions as to what would cause this?
> > > >
> > > > [    1.084728] pci 0000:00:1c.0: BAR 15: no space for [mem size
> > > > 0x00200000 64bit pref]
> > > > [    1.084813] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
> > > > 0x00200000 64bit pref]
> > > > [    1.084890] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
> > > > [    1.084949] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
> > > > [    1.085037] pci 0000:00:1c.2: BAR 15: no space for [mem size
> > > > 0x00200000 64bit pref]
> > > > [    1.085108] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
> > > > 0x00200000 64bit pref]
> > > > [    1.085199] pci 0000:00:1c.3: BAR 15: no space for [mem size
> > > > 0x00200000 64bit pref]
> > > > [    1.085270] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
> > > > 0x00200000 64bit pref]
> > > > [    1.085343] pci 0000:00:1c.0: BAR 13: assigned [io  0x1000-0x1fff]
> > > > [    1.085403] pci 0000:00:1c.2: BAR 13: assigned [io  0x2000-0x2fff]
> > > > [    1.085470] pci 0000:00:1c.3: BAR 15: no space for [mem size
> > > > 0x00200000 64bit pref]
> > > > [    1.085540] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
> > > > 0x00200000 64bit pref]
> > > > [    1.085613] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
> > > > [    1.085672] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
> > > > [    1.085738] pci 0000:00:1c.2: BAR 15: no space for [mem size
> > > > 0x00200000 64bit pref]
> > > > [    1.085808] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
> > > > 0x00200000 64bit pref]
> > > > [    1.085884] pci 0000:00:1c.0: BAR 15: no space for [mem size
> > > > 0x00200000 64bit pref]
> > > > [    1.085954] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
> > > > 0x00200000 64bit pref]
> > > > [    1.086026] pci 0000:00:1c.0: PCI bridge to [bus 01]
> > > > [    1.086083] pci 0000:00:1c.0:   bridge window [io  0x1000-0x1fff]
> > > > [    1.086144] pci 0000:00:1c.0:   bridge window [mem 0xd0400000-0xd07fffff]
> > >
> > > The "no space" and "failed to assign" messages are all for bridge
> > > windows (13 is the I/O window, 14 is the MMIO window, 15 is the MMIO
> > > pref window).  I can't tell if you have any devices below these
> > > bridges (lspci would show them).  If you don't have any devices below
> > > these bridges, you can ignore the messages.
> >
> > I have the devices below these bridges. FPGA endpoint is connected to
> > '00:1c.0 PCI bridge' and Ethernet controller is connected to '00:1c.3
> > PCI bridge'.
> > Does these messages impact the functionality of the connected devices?
>
> Yes.  We tried to allocate 0x00200000 of prefetchable MMIO to 00:1c.0
> for use by the FPGA endpoint.  But this failed, so there is no
> prefetchable MMIO available for the FPGA.  The 0xd0400000-0xd07fffff
> non-prefetachable MMIO space *is* available for it.
>
> Similarly, we were unable to allocate 0x00200000 of prefetchable MMIO
> for the 00:1c.3 bridge for use by the ethernet controller.  I don't
> know what non-prefetchable MMIO space was allocated or what the NIC
> needs.
>
> "lspci -v" will show you what the FPGA and the NIC need.
>
> > If so what kind of impact and is there any solution for this?
>
> > Also, I'd like to know why "no space" and "failed to assign"
> > messages displayed?
>
> These messages mean we tried to allocate space for the bridges but we
> unable to do so.  This is because either the host bridge didn't have
> big enough apertures (on x86, these usually come from ACPI _CRS
> methods), or there's a bug in the Linux allocation algorithms.
>
> I can't tell anything more without seeing the complete dmesg log,
> which contains the host bridge aperture information and the BAR sizes
> of the FPGA and NIC.
Thank you for the clarification. dmesg log is attached.
If PCI bridge aperture is small, What is the most sensible way to proceed?

>
> Bjorn



-- 
Thanks,
Sekhar

[-- Attachment #2: dmesg_log --]
[-- Type: application/octet-stream, Size: 74210 bytes --]

[-- Attachment #3: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: pcie: kernel log - BAR 15: no space for... BAR 15: failed to assign..
  2020-02-11 20:16         ` Bjorn Helgaas
@ 2020-02-12  8:12           ` Muni Sekhar
  0 siblings, 0 replies; 7+ messages in thread
From: Muni Sekhar @ 2020-02-12  8:12 UTC (permalink / raw)
  To: Bjorn Helgaas; +Cc: linux-pci, kernelnewbies

On Wed, Feb 12, 2020 at 1:46 AM Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> On Tue, Feb 11, 2020 at 10:02:13PM +0530, Muni Sekhar wrote:
> > On Tue, Feb 11, 2020 at 8:10 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > > On Tue, Feb 11, 2020 at 07:36:00PM +0530, Muni Sekhar wrote:
> > > > On Tue, Feb 11, 2020 at 3:58 AM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > > > > On Sun, Feb 09, 2020 at 07:59:41AM +0530, Muni Sekhar wrote:
> > > > > > Hi all,
> > > > > >
> > > > > > After rebooting the system following messages are seen in dmesg.
> > > > > > Not sure if these indicate a problem. Can some one look at these and
> > > > > > confirm if this is problem or can be ignored ?
> > > > > >
> > > > > > Also any suggestions as to what would cause this?
> > > > > >
> > > > > > [    1.084728] pci 0000:00:1c.0: BAR 15: no space for [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.084813] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.084890] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
> > > > > > [    1.084949] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
> > > > > > [    1.085037] pci 0000:00:1c.2: BAR 15: no space for [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085108] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085199] pci 0000:00:1c.3: BAR 15: no space for [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085270] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085343] pci 0000:00:1c.0: BAR 13: assigned [io  0x1000-0x1fff]
> > > > > > [    1.085403] pci 0000:00:1c.2: BAR 13: assigned [io  0x2000-0x2fff]
> > > > > > [    1.085470] pci 0000:00:1c.3: BAR 15: no space for [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085540] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085613] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
> > > > > > [    1.085672] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
> > > > > > [    1.085738] pci 0000:00:1c.2: BAR 15: no space for [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085808] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085884] pci 0000:00:1c.0: BAR 15: no space for [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085954] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.086026] pci 0000:00:1c.0: PCI bridge to [bus 01]
> > > > > > [    1.086083] pci 0000:00:1c.0:   bridge window [io  0x1000-0x1fff]
> > > > > > [    1.086144] pci 0000:00:1c.0:   bridge window [mem 0xd0400000-0xd07fffff]
> > > > >
> > > > > The "no space" and "failed to assign" messages are all for bridge
> > > > > windows (13 is the I/O window, 14 is the MMIO window, 15 is the MMIO
> > > > > pref window).  I can't tell if you have any devices below these
> > > > > bridges (lspci would show them).  If you don't have any devices below
> > > > > these bridges, you can ignore the messages.
> > > >
> > > > I have the devices below these bridges. FPGA endpoint is connected to
> > > > '00:1c.0 PCI bridge' and Ethernet controller is connected to '00:1c.3
> > > > PCI bridge'.
> > > > Does these messages impact the functionality of the connected devices?
> > >
> > > Yes.  We tried to allocate 0x00200000 of prefetchable MMIO to 00:1c.0
> > > for use by the FPGA endpoint.  But this failed, so there is no
> > > prefetchable MMIO available for the FPGA.  The 0xd0400000-0xd07fffff
> > > non-prefetachable MMIO space *is* available for it.
> > >
> > > Similarly, we were unable to allocate 0x00200000 of prefetchable MMIO
> > > for the 00:1c.3 bridge for use by the ethernet controller.  I don't
> > > know what non-prefetchable MMIO space was allocated or what the NIC
> > > needs.
> > >
> > > "lspci -v" will show you what the FPGA and the NIC need.
> > >
> > > > If so what kind of impact and is there any solution for this?
> > >
> > > > Also, I'd like to know why "no space" and "failed to assign"
> > > > messages displayed?
> > >
> > > These messages mean we tried to allocate space for the bridges but we
> > > unable to do so.  This is because either the host bridge didn't have
> > > big enough apertures (on x86, these usually come from ACPI _CRS
> > > methods), or there's a bug in the Linux allocation algorithms.
> > >
> > > I can't tell anything more without seeing the complete dmesg log,
> > > which contains the host bridge aperture information and the BAR sizes
> > > of the FPGA and NIC.
> > Thank you for the clarification. dmesg log is attached.
> > If PCI bridge aperture is small, What is the most sensible way to proceed?
>
> Here's the relevant info from your dmesg log:
>
>   PCI host bridge to bus 0000:00
>   pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
>   pci_bus 0000:00: root bus resource [mem 0xc0000000-0xd0c18ffe window]
>   pci_bus 0000:00: root bus resource [bus 00-ff]
>   pci 0000:00:1c.0: PCI bridge to [bus 01]
>   pci 0000:00:1c.0:   bridge window [mem 0xd0400000-0xd07fffff]
>   pci 0000:01:00.0: [1556:5555] type 00 class 0x050000 # FPGA
>   pci 0000:01:00.0: reg 0x10: [mem 0xd0400000-0xd07fffff]
>   pci 0000:00:1c.3: PCI bridge to [bus 03]
>   pci 0000:00:1c.3:   bridge window [io  0xd000-0xdfff]
>   pci 0000:00:1c.3:   bridge window [mem 0xd0a00000-0xd0bfffff]
>   pci 0000:03:00.0: [8086:1533] type 00 class 0x020000 # I210 NIC
>   pci 0000:03:00.0: reg 0x10: [mem 0xd0a00000-0xd0afffff]
>   pci 0000:03:00.0: reg 0x18: [io  0xd000-0xd01f]
>   pci 0000:03:00.0: reg 0x1c: [mem 0xd0b00000-0xd0b03fff]
>
> The FPGA at 01:00.0 has one non-prefetchable memory BAR and is
> assigned [mem 0xd0400000-0xd07fffff].  That range is inside the host
> bridge aperture ([mem 0xc0000000-0xd0c18ffe window]) and is routed
> through the 00:1c.0 bridge (bridge window [mem 0xd0400000-0xd07fffff]).
> So the FPGA resources are fine.
>
> The fact that Linux tried and failed to allocate prefetchable memory
> space for the bridge is immaterial because the FPGA can't use
> prefetchable memory anyway.
>
> The NIC at 03:00.0 is similar: it has two non-prefetchable memory BARs
> and one I/O BAR.  They're all assigned valid space, and the fact that
> Linux tried and failed to allocate prefetchable memory space for the
> bridge makes no difference because the NIC can't use it.
>
> So these messages do not indicate a problem.  Maybe there's something
> we can do to make this more clear to the user.
Thank you for the detailed explanation.
>
> Bjorn



-- 
Thanks,
Sekhar

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: pcie: kernel log - BAR 15: no space for... BAR 15: failed to assign..
  2020-02-11 16:32       ` Muni Sekhar
@ 2020-02-11 20:16         ` Bjorn Helgaas
  2020-02-12  8:12           ` Muni Sekhar
  0 siblings, 1 reply; 7+ messages in thread
From: Bjorn Helgaas @ 2020-02-11 20:16 UTC (permalink / raw)
  To: Muni Sekhar; +Cc: linux-pci, kernelnewbies

On Tue, Feb 11, 2020 at 10:02:13PM +0530, Muni Sekhar wrote:
> On Tue, Feb 11, 2020 at 8:10 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > On Tue, Feb 11, 2020 at 07:36:00PM +0530, Muni Sekhar wrote:
> > > On Tue, Feb 11, 2020 at 3:58 AM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > > > On Sun, Feb 09, 2020 at 07:59:41AM +0530, Muni Sekhar wrote:
> > > > > Hi all,
> > > > >
> > > > > After rebooting the system following messages are seen in dmesg.
> > > > > Not sure if these indicate a problem. Can some one look at these and
> > > > > confirm if this is problem or can be ignored ?
> > > > >
> > > > > Also any suggestions as to what would cause this?
> > > > >
> > > > > [    1.084728] pci 0000:00:1c.0: BAR 15: no space for [mem size
> > > > > 0x00200000 64bit pref]
> > > > > [    1.084813] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
> > > > > 0x00200000 64bit pref]
> > > > > [    1.084890] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
> > > > > [    1.084949] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
> > > > > [    1.085037] pci 0000:00:1c.2: BAR 15: no space for [mem size
> > > > > 0x00200000 64bit pref]
> > > > > [    1.085108] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
> > > > > 0x00200000 64bit pref]
> > > > > [    1.085199] pci 0000:00:1c.3: BAR 15: no space for [mem size
> > > > > 0x00200000 64bit pref]
> > > > > [    1.085270] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
> > > > > 0x00200000 64bit pref]
> > > > > [    1.085343] pci 0000:00:1c.0: BAR 13: assigned [io  0x1000-0x1fff]
> > > > > [    1.085403] pci 0000:00:1c.2: BAR 13: assigned [io  0x2000-0x2fff]
> > > > > [    1.085470] pci 0000:00:1c.3: BAR 15: no space for [mem size
> > > > > 0x00200000 64bit pref]
> > > > > [    1.085540] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
> > > > > 0x00200000 64bit pref]
> > > > > [    1.085613] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
> > > > > [    1.085672] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
> > > > > [    1.085738] pci 0000:00:1c.2: BAR 15: no space for [mem size
> > > > > 0x00200000 64bit pref]
> > > > > [    1.085808] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
> > > > > 0x00200000 64bit pref]
> > > > > [    1.085884] pci 0000:00:1c.0: BAR 15: no space for [mem size
> > > > > 0x00200000 64bit pref]
> > > > > [    1.085954] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
> > > > > 0x00200000 64bit pref]
> > > > > [    1.086026] pci 0000:00:1c.0: PCI bridge to [bus 01]
> > > > > [    1.086083] pci 0000:00:1c.0:   bridge window [io  0x1000-0x1fff]
> > > > > [    1.086144] pci 0000:00:1c.0:   bridge window [mem 0xd0400000-0xd07fffff]
> > > >
> > > > The "no space" and "failed to assign" messages are all for bridge
> > > > windows (13 is the I/O window, 14 is the MMIO window, 15 is the MMIO
> > > > pref window).  I can't tell if you have any devices below these
> > > > bridges (lspci would show them).  If you don't have any devices below
> > > > these bridges, you can ignore the messages.
> > >
> > > I have the devices below these bridges. FPGA endpoint is connected to
> > > '00:1c.0 PCI bridge' and Ethernet controller is connected to '00:1c.3
> > > PCI bridge'.
> > > Does these messages impact the functionality of the connected devices?
> >
> > Yes.  We tried to allocate 0x00200000 of prefetchable MMIO to 00:1c.0
> > for use by the FPGA endpoint.  But this failed, so there is no
> > prefetchable MMIO available for the FPGA.  The 0xd0400000-0xd07fffff
> > non-prefetachable MMIO space *is* available for it.
> >
> > Similarly, we were unable to allocate 0x00200000 of prefetchable MMIO
> > for the 00:1c.3 bridge for use by the ethernet controller.  I don't
> > know what non-prefetchable MMIO space was allocated or what the NIC
> > needs.
> >
> > "lspci -v" will show you what the FPGA and the NIC need.
> >
> > > If so what kind of impact and is there any solution for this?
> >
> > > Also, I'd like to know why "no space" and "failed to assign"
> > > messages displayed?
> >
> > These messages mean we tried to allocate space for the bridges but we
> > unable to do so.  This is because either the host bridge didn't have
> > big enough apertures (on x86, these usually come from ACPI _CRS
> > methods), or there's a bug in the Linux allocation algorithms.
> >
> > I can't tell anything more without seeing the complete dmesg log,
> > which contains the host bridge aperture information and the BAR sizes
> > of the FPGA and NIC.
> Thank you for the clarification. dmesg log is attached.
> If PCI bridge aperture is small, What is the most sensible way to proceed?

Here's the relevant info from your dmesg log:

  PCI host bridge to bus 0000:00
  pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
  pci_bus 0000:00: root bus resource [mem 0xc0000000-0xd0c18ffe window]
  pci_bus 0000:00: root bus resource [bus 00-ff]
  pci 0000:00:1c.0: PCI bridge to [bus 01]
  pci 0000:00:1c.0:   bridge window [mem 0xd0400000-0xd07fffff]
  pci 0000:01:00.0: [1556:5555] type 00 class 0x050000 # FPGA
  pci 0000:01:00.0: reg 0x10: [mem 0xd0400000-0xd07fffff]
  pci 0000:00:1c.3: PCI bridge to [bus 03]
  pci 0000:00:1c.3:   bridge window [io  0xd000-0xdfff]
  pci 0000:00:1c.3:   bridge window [mem 0xd0a00000-0xd0bfffff]
  pci 0000:03:00.0: [8086:1533] type 00 class 0x020000 # I210 NIC
  pci 0000:03:00.0: reg 0x10: [mem 0xd0a00000-0xd0afffff]
  pci 0000:03:00.0: reg 0x18: [io  0xd000-0xd01f]
  pci 0000:03:00.0: reg 0x1c: [mem 0xd0b00000-0xd0b03fff]

The FPGA at 01:00.0 has one non-prefetchable memory BAR and is
assigned [mem 0xd0400000-0xd07fffff].  That range is inside the host
bridge aperture ([mem 0xc0000000-0xd0c18ffe window]) and is routed
through the 00:1c.0 bridge (bridge window [mem 0xd0400000-0xd07fffff]).
So the FPGA resources are fine.

The fact that Linux tried and failed to allocate prefetchable memory
space for the bridge is immaterial because the FPGA can't use
prefetchable memory anyway.

The NIC at 03:00.0 is similar: it has two non-prefetchable memory BARs
and one I/O BAR.  They're all assigned valid space, and the fact that
Linux tried and failed to allocate prefetchable memory space for the
bridge makes no difference because the NIC can't use it.

So these messages do not indicate a problem.  Maybe there's something
we can do to make this more clear to the user.

Bjorn

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, back to index

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-09  2:29 pcie: kernel log - BAR 15: no space for... BAR 15: failed to assign Muni Sekhar
2020-02-10 22:28 ` Bjorn Helgaas
2020-02-11 14:06   ` Muni Sekhar
2020-02-11 14:40     ` Bjorn Helgaas
2020-02-11 16:32       ` Muni Sekhar
2020-02-11 20:16         ` Bjorn Helgaas
2020-02-12  8:12           ` Muni Sekhar

Kernel Newbies archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/kernelnewbies/0 kernelnewbies/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 kernelnewbies kernelnewbies/ https://lore.kernel.org/kernelnewbies \
		kernelnewbies@kernelnewbies.org
	public-inbox-index kernelnewbies

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernelnewbies.kernelnewbies


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git