From: Punit Agrawal <punitagrawal@gmail.com>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: Vidya Sagar <vidyas@nvidia.com>,
robh+dt@kernel.org, linux-rockchip@lists.infradead.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org,
alexandru.elisei@arm.com, wqu@suse.com, robin.murphy@arm.com,
pgwipeout@gmail.com, ardb@kernel.org, briannorris@chromium.org,
shawn.lin@rock-chips.com
Subject: Re: [PATCH v3 2/4] PCI: of: Relax the condition for warning about non-prefetchable memory aperture size
Date: Thu, 10 Jun 2021 23:11:10 +0900 [thread overview]
Message-ID: <87y2bhkdxd.fsf@stealth> (raw)
In-Reply-To: <20210610040427.GA2696540@bjorn-Precision-5520> (Bjorn Helgaas's message of "Wed, 9 Jun 2021 23:04:27 -0500")
Hi Bjorn,
Bjorn Helgaas <helgaas@kernel.org> writes:
> On Wed, Jun 09, 2021 at 12:36:08AM +0530, Vidya Sagar wrote:
>> On 6/7/2021 4:58 PM, Punit Agrawal wrote:
>> >
>> > Commit fede8526cc48 ("PCI: of: Warn if non-prefetchable memory
>> > aperture size is > 32-bit") introduced a warning for non-prefetchable
>> > resources that need more than 32bits to resolve. It turns out that the
>> > check is too restrictive and should be applicable to only resources
>> > that are limited to host bridge windows that don't have the ability to
>> > map 64-bit address space.
>>
>> I think the host bridge windows having the ability to map 64-bit address
>> space is different from restricting the non-prefetchable memory aperture
>> size to 32-bit.
>
>> Whether the host bridge uses internal translations or not to map the
>> non-prefetchable resources to 64-bit space, the size needs to be programmed
>> in the host bridge's 'Memory Limit Register (Offset 22h)' which can
>> represent sizes only fit into 32-bits.
>
>> Host bridges having the ability to map 64-bit address spaces gives
>> flexibility to utilize the vast 64-bit space for the (restrictive)
>> non-prefetchable memory (i.e. mapping non-prefetchable BARs of endpoints to
>> the 64-bit space in CPU's view) and get it translated internally and put a
>> 32-bit address on the PCIe bus finally.
>
> The vastness of the 64-bit space in the CPU view only helps with
> non-prefetchable memory if you have multiple host bridges with
> different CPU-to-PCI translations. Each root bus can only carve up
> 4GB of PCI memory space for use by its non-prefetchable memory
> windows.
>
> Of course, if we're willing to give up the performance, there's
> nothing to prevent us from using non-prefetchable space for
> *prefetchable* resources, as in my example below.
>
> I think the fede8526cc48 commit log is incorrect, or at least
> incomplete:
>
> As per PCIe spec r5.0, sec 7.5.1.3.8 only 32-bit BAR registers are defined
> for non-prefetchable memory and hence a warning should be reported when
> the size of them go beyond 32-bits.
>
> 7.5.1.3.8 is talking about non-prefetchable PCI-to-PCI bridge windows,
> not BARs. AFAIK, 64-bit BARs may be non-prefetchable. The warning is
> in pci_parse_request_of_pci_ranges(), which isn't looking at
> PCI-to-PCI bridge windows; it's looking at PCI host bridge windows.
> It's legal for a host bridge to have only non-prefetchable windows,
> and prefetchable PCI BARs can be placed in them.
>
> For example, we could have the following:
>
> pci_bus 0000:00: root bus resource [mem 0x80000000-0x1_ffffffff] (6GB)
> pci 0000:00:00.0: PCI bridge to [bus 01-7f]
> pci 0000:00:00.0: bridge window [mem 0x80000000-0xbfffffff] (1GB)
> pci 0000:00:00.0: bridge window [mem 0x1_00000000-0x1_7fffffff 64bit pref] (2GB)
> pci 0000:00:00.1: PCI bridge to [bus 80-ff]
> pci 0000:00:00.1: bridge window [mem 0xc0000000-0xffffffff] (1GB)
> pci 0000:00:00.1: bridge window [mem 0x1_80000000-0x1_ffffffff 64bit pref] (2GB)
>
> Here the host bridge window is 6GB and is not prefetchable. The
> PCI-to-PCI bridge non-prefetchable windows are 1GB each and the bases
> and limits fit in 32 bits. The prefetchable windows are 2GB each, and
> we're allowed but not required to put these in prefetchable host
> bridge windows.
>
> So I'm not convinced this warning is valid to begin with. It may be
> that this host bridge configuration isn't optimal, and we might want
> an informational message, but I think it's *legal*.
By "optimal" - are you referring to the use of non-prefetchable space
for prefetchable window?
Also, if the warning doesn't apply to PCI host bridge windows, should I
drop it in the next update? Or leave out this and the next patch to be
dealt with separately.
Thanks,
Punit
[...]
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-06-10 14:13 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-07 11:28 [PATCH v3 0/4] PCI: of: Improvements to handle 64-bit attribute for non-prefetchable ranges Punit Agrawal
2021-06-07 11:28 ` [PATCH v3 1/4] PCI: of: Clear 64-bit flag for non-prefetchable memory below 4GB Punit Agrawal
2021-06-10 0:22 ` Bjorn Helgaas
2021-06-10 13:34 ` Punit Agrawal
2021-06-10 18:28 ` Bjorn Helgaas
2021-06-07 11:28 ` [PATCH v3 2/4] PCI: of: Relax the condition for warning about non-prefetchable memory aperture size Punit Agrawal
2021-06-08 19:06 ` Vidya Sagar
2021-06-10 4:04 ` Bjorn Helgaas
2021-06-10 14:11 ` Punit Agrawal [this message]
2021-06-10 19:58 ` Bjorn Helgaas
2021-06-07 11:28 ` [PATCH v3 3/4] PCI: of: Refactor the check for non-prefetchable 32-bit window Punit Agrawal
2021-06-07 11:28 ` [PATCH v3 4/4] arm64: dts: rockchip: Update PCI host bridge window to 32-bit address memory Punit Agrawal
2021-06-10 21:50 ` Heiko Stübner
2021-06-11 14:38 ` Punit Agrawal
2021-06-15 21:29 ` Rob Herring
2021-06-15 21:49 ` Heiko Stübner
2021-06-16 13:00 ` Punit Agrawal
2021-06-09 16:08 ` [PATCH v3 0/4] PCI: of: Improvements to handle 64-bit attribute for non-prefetchable ranges Marc Zyngier
2021-06-10 14:17 ` Punit Agrawal
2021-06-10 9:05 ` Anand Moon
2021-06-10 14:25 ` Punit Agrawal
2021-06-10 18:36 ` Anand Moon
2021-06-11 22:15 ` (subset) " Heiko Stuebner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87y2bhkdxd.fsf@stealth \
--to=punitagrawal@gmail.com \
--cc=alexandru.elisei@arm.com \
--cc=ardb@kernel.org \
--cc=briannorris@chromium.org \
--cc=helgaas@kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-rockchip@lists.infradead.org \
--cc=pgwipeout@gmail.com \
--cc=robh+dt@kernel.org \
--cc=robin.murphy@arm.com \
--cc=shawn.lin@rock-chips.com \
--cc=vidyas@nvidia.com \
--cc=wqu@suse.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).