Re: [RFC PATCH v1 1/4] arm/pci: PCI setup and PCI host bridge discovery within XEN on ARM.

From: "Roger Pau Monné" <roger.pau@citrix.com>
To: Julien Grall <julien.grall.oss@gmail.com>
Cc: nd <nd@arm.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Bertrand Marquis <Bertrand.Marquis@arm.com>,
	Jan Beulich <jbeulich@suse.com>,
	xen-devel <xen-devel@lists.xenproject.org>,
	Rahul Singh <rahul.singh@arm.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [RFC PATCH v1 1/4] arm/pci: PCI setup and PCI host bridge discovery within XEN on ARM.
Date: Mon, 27 Jul 2020 13:06:48 +0200	[thread overview]
Message-ID: <20200727110648.GQ7191@Air-de-Roger> (raw)
In-Reply-To: <CAJ=z9a1RWXq3EN5DC=_279yzdsq3M0nw6+CZtKD00yBzKomcaw@mail.gmail.com>

On Sat, Jul 25, 2020 at 10:59:50AM +0100, Julien Grall wrote:
> On Sat, 25 Jul 2020 at 00:46, Stefano Stabellini <sstabellini@kernel.org> wrote:
> >
> > On Fri, 24 Jul 2020, Julien Grall wrote:
> > > On Fri, 24 Jul 2020 at 19:32, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > > > > If they are not equal, then I fail to see why it would be useful to have this
> > > > > value in Xen.
> > > >
> > > > I think that's because the domain is actually more convenient to use
> > > > because a segment can span multiple PCI host bridges. So my
> > > > understanding is that a segment alone is not sufficient to identify a
> > > > host bridge. From a software implementation point of view it would be
> > > > better to use domains.
> > >
> > > AFAICT, this would be a matter of one check vs two checks in Xen :).
> > > But... looking at Linux, they will also use domain == segment for ACPI
> > > (see [1]). So, I think, they still have to use (domain, bus) to do the lookup.

You have to use the (segment, bus) tuple when doing a lookup because
MMCFG regions on ACPI are defined for a segment and a bus range, you
can have a MMCFG region that covers segment 0 bus [0, 20) and another
MMCFG region that covers segment 0 bus [20, 255], and those will use
different addresses in the MMIO space.

> > > > > In which case, we need to use PHYSDEVOP_pci_mmcfg_reserved so
> > > > > Dom0 and Xen can synchronize on the segment number.
> > > >
> > > > I was hoping we could write down the assumption somewhere that for the
> > > > cases we care about domain == segment, and error out if it is not the
> > > > case.
> > >
> > > Given that we have only the domain in hand, how would you enforce that?
> > >
> > > >From this discussion, it also looks like there is a mismatch between the
> > > implementation and the understanding on QEMU devel. So I am a bit
> > > concerned that this is not stable and may change in future Linux version.
> > >
> > > IOW, we are know tying Xen to Linux. So could we implement
> > > PHYSDEVOP_pci_mmcfg_reserved *or* introduce a new property that
> > > really represent the segment?
> >
> > I don't think we are tying Xen to Linux. Rob has already said that
> > linux,pci-domain is basically a generic device tree property.
> 
> My concern is not so much the name of the property, but the definition of it.
> 
> AFAICT, from this thread there can be two interpretation:
>       - domain == segment
>       - domain == (segment, bus)

I think domain is just an alias for segment, the difference seems to
be that when using DT all bridges get a different segment (or domain)
number, and thus you will always end up starting numbering at bus 0
for each bridge?

Ideally you would need a way to specify the segment and start/end bus
numbers of each bridge, if not you cannot match what ACPI does. Albeit
it might be fine as long as the OS and Xen agree on the segments and
bus numbers that belong to each bridge (and thus each ECAM region).

Roger.