From: Bjorn Helgaas <bhelgaas@google.com>
To: Yinghai Lu <yinghai@kernel.org>
Cc: "H. Peter Anvin" <hpa@zytor.com>,
Linus Torvalds <torvalds@linux-foundation.org>,
Steven Newbury <steve@snewbury.org.uk>,
Andrew Morton <akpm@linux-foundation.org>,
linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 02/11] PCI: Try to allocate mem64 above 4G at first
Date: Tue, 29 May 2012 17:24:44 -0600 [thread overview]
Message-ID: <CAErSpo7+7fReDEzzgjonz76FD5MZFC5trPAiKZ8hLq5Wi2j13Q@mail.gmail.com> (raw)
In-Reply-To: <CAE9FiQWORRjxEQwr+933ReNAAKAEoJ5B6AWte4gr33_BO+4heQ@mail.gmail.com>
On Tue, May 29, 2012 at 2:40 PM, Yinghai Lu <yinghai@kernel.org> wrote:
> On Tue, May 29, 2012 at 12:23 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
>> On Tue, May 29, 2012 at 12:17 PM, Yinghai Lu <yinghai@kernel.org> wrote:
>>> On Tue, May 29, 2012 at 10:57 AM, H. Peter Anvin <hpa@zytor.com> wrote:
>>>> On 05/29/2012 10:55 AM, Yinghai Lu wrote:
>>>>>
>>>>> x86 are using 16bits.
>>>>>
>>>>> some others use 32 bits.
>>>>> #define IO_SPACE_LIMIT 0xffffffff
>>>>>
>>>>> ia64 and sparc are using 64bits.
>>>>> #define IO_SPACE_LIMIT 0xffffffffffffffffUL
>>>>>
>>>>> but pci only support 16bits and 32bits.
>>>>>
>>>>> maybe later we can add
>>>>> PCI_MAX_RESOURCE_16
>>>>>
>>>>> to handle 16bits and 32bit io ports.
>>>>>
>>>>
>>>> Shouldn't this be dealt by root port apertures?
>>>>
>>>
>>> pci bridge could support 16bits and 32bits io port.
>>> but we did not record if 32bits is supported.
>>>
>>> so during allocating, could have allocated above 64k address to non
>>> 32bit bridge.
>>>
>>> but x86 is ok, because ioport.end always set to 0xffff.
>>> other arches with IO_SPACE_LIMIT with 0xffffffff or
>>> 0xffffffffffffffffUL may have problem.
>>
>> I think current IO_SPACE_LIMIT usage is a little confused. The
>> "ioport_resource.end = IO_SPACE_LIMIT" in kernel/resource.c refers to
>> a CPU-side address, not a bus address. Other uses, e.g., in
>> __pci_read_base(), apply it to bus addresses from BARs, which is
>> wrong. Host bridges apply I/O port offsets just like they apply
>> memory offsets. The ia64 IO_SPACE_LIMIT of 0xffffffffffffffffUL means
>> there's no restriction on CPU-side I/O port addresses, but any given
>> host bridge will translate its I/O port aperture to bus addresses that
>> fit in 32 bits.
>>
>> None of this is really relevant to the question I asked, namely, "why
>> Yinghai's patch doesn't limit I/O BAR values to 32 bits?" That
>> constraint is clearly a requirement because I/O BARs are only 32 bits
>> wide, but I don't think it needs to be enforced in the code here. The
>> host bridge or upstream P2P bridge apertures should already take care
>> of that automatically. I don't think the 16- or 32-bitness of P2P
>> bridge apertures is relevant here, because the I/O resources available
>> on the secondary bus already reflect that.
>>
>> After all that discussion, I think my objection here boils down to
>> "you shouldn't change the I/O BAR constraints in a patch that claims
>> to allocate 64-bit *memory* BARs above 4GB."
>>
>> I think the code below is still the clearest way to set the constraints:
>>
>> if (res->flags & IORESOURCE_MEM_64) {
>> start = (resource_size_t) (1ULL << 32);
>> end = PCI_MAX_RESOURCE;
>> } else {
>> start = 0;
>> end = PCI_MAX_RESOURCE_32;
>> }
>>
>> It's not strictly necessary to limit I/O BARs to PCI_MAX_RESOURCE_32
>> because host bridge apertures should already enforce that, but I think
>> the code above just makes it clearer.
>
>
> ok, please check the version, that put back PCI_MAX_RESOURCE_32 for io ports.
I like the fact that this patch no longer changes anything for I/O
resources. I assume this is part of fixing some bug (Steven's?) I'd
like to have a pointer in the changelog to a bugzilla or discussion
about the bug.
The effect of this patch is similar to what I did earlier with
b126b4703afa4 and e7f8567db9a7 (allocate space from top down), though
this one is more limited and it won't change quite as much. We ran
into problems (BIOS defects, I think) and had to revert my patches, so
it's quite possible that we'll run into similar problems here.
I'm a little nervous because this is a fundamental area that explores
new areas of the address space minefield. I think we're generally
safer if we follow a path similar to where Windows has been. I think
Windows also prefers space above 4GB for 64-bit BARs, but I suspect
that's just a natural consequence of allocating from the top down. So
we'll place things just above 4GB, and Windows will place things as
high as possible.
I don't know the best solution here. This patch ("bottom-up above
4GB") is one possibility. Another is to allocate only 64-bit BARs
top-down. Or maybe allocate everything top-down on machines newer
than some date. They all seem ugly. What makes me uneasy is that
your patch strikes out on a new path that is different from what we've
done before *and* different from what Windows does.
next prev parent reply other threads:[~2012-05-29 23:24 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-05-23 6:34 [PATCH 00/11] PCI: resource allocation related Yinghai Lu
2012-05-23 6:34 ` [PATCH 01/11] PCI: Should add children device res to fail list Yinghai Lu
2012-05-23 6:34 ` [PATCH 02/11] PCI: Try to allocate mem64 above 4G at first Yinghai Lu
2012-05-23 15:57 ` Linus Torvalds
2012-05-23 17:30 ` Yinghai Lu
2012-05-23 18:40 ` Yinghai Lu
2012-05-25 4:36 ` Bjorn Helgaas
2012-05-25 17:53 ` Yinghai Lu
2012-05-25 18:39 ` Yinghai Lu
2012-05-25 19:37 ` Bjorn Helgaas
2012-05-25 20:18 ` H. Peter Anvin
2012-05-25 20:19 ` Yinghai Lu
2012-05-25 21:55 ` Bjorn Helgaas
2012-05-25 21:58 ` H. Peter Anvin
2012-05-25 22:14 ` Bjorn Helgaas
2012-05-25 23:10 ` Yinghai Lu
2012-05-26 0:12 ` Bjorn Helgaas
2012-05-26 15:01 ` Bjorn Helgaas
2012-05-29 17:56 ` Yinghai Lu
2012-05-29 17:55 ` Yinghai Lu
2012-05-29 17:57 ` H. Peter Anvin
2012-05-29 18:17 ` Yinghai Lu
2012-05-29 19:03 ` H. Peter Anvin
2012-05-29 20:46 ` Yinghai Lu
2012-05-29 20:50 ` H. Peter Anvin
2012-06-01 23:30 ` Yinghai Lu
2012-06-04 1:05 ` Bjorn Helgaas
2012-06-05 2:37 ` Yinghai Lu
2012-06-05 4:50 ` Bjorn Helgaas
2012-06-05 5:04 ` Yinghai Lu
2012-06-06 9:44 ` Steven Newbury
2012-06-06 16:18 ` Bjorn Helgaas
[not found] ` <CAGLnvc_ejMWiiubVMo7DLz5ZVn1iMbf67FB4H7crRCCTRRqt2A@mail.gmail.com>
2012-07-04 3:00 ` joeyli
2012-05-29 20:53 ` David Miller
2012-05-29 19:23 ` Bjorn Helgaas
2012-05-29 20:40 ` Yinghai Lu
2012-05-29 23:24 ` Bjorn Helgaas [this message]
2012-05-29 23:27 ` Bjorn Helgaas
2012-05-29 23:33 ` Yinghai Lu
2012-05-29 23:47 ` Bjorn Helgaas
2012-05-30 7:40 ` Steven Newbury
2012-05-30 16:27 ` Bjorn Helgaas
2012-05-30 16:30 ` H. Peter Anvin
2012-05-30 16:33 ` Linus Torvalds
2012-05-23 6:34 ` [PATCH 03/11] intel-gtt: Read 64bit for gmar_bus_addr Yinghai Lu
2012-05-23 7:21 ` Dave Airlie
2012-05-23 7:44 ` Daniel Vetter
2012-05-23 6:34 ` [PATCH 04/11] PCI: Make sure assign same align with large size resource at first Yinghai Lu
2012-05-23 6:34 ` [PATCH 05/11] resources: Split out __allocate_resource() Yinghai Lu
2012-05-23 6:34 ` [PATCH 06/11] resource: make find_resource could return just fit resource Yinghai Lu
2012-05-23 6:34 ` [PATCH 07/11] PCI: Don't allocate small resource in big empty space Yinghai Lu
2012-05-23 6:34 ` [PATCH 08/11] resource: only return range with needed align Yinghai Lu
2012-05-23 6:34 ` [PATCH 09/11] PCI: Add is_pci_iov_resource_idx() Yinghai Lu
2012-05-23 6:34 ` [PATCH 10/11] PCI: Sort unassigned resources with correct alignment Yinghai Lu
2012-05-23 6:34 ` [PATCH 11/11] PCI: Treat ROM resource as optional during assigning Yinghai Lu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAErSpo7+7fReDEzzgjonz76FD5MZFC5trPAiKZ8hLq5Wi2j13Q@mail.gmail.com \
--to=bhelgaas@google.com \
--cc=akpm@linux-foundation.org \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=steve@snewbury.org.uk \
--cc=torvalds@linux-foundation.org \
--cc=yinghai@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).