From: Stefano Stabellini <sstabellini@kernel.org>
To: Julien Grall <julien@xen.org>
Cc: "'Kevin Tian'" <kevin.tian@intel.com>,
"Stefano Stabellini" <sstabellini@kernel.org>,
"'Jun Nakajima'" <jun.nakajima@intel.com>,
"'Wei Liu'" <wl@xen.org>,
paul@xen.org, "'Andrew Cooper'" <andrew.cooper3@citrix.com>,
"'Ian Jackson'" <ian.jackson@eu.citrix.com>,
"'George Dunlap'" <george.dunlap@citrix.com>,
"'Tim Deegan'" <tim@xen.org>, Oleksandr <olekstysh@gmail.com>,
"'Oleksandr Tyshchenko'" <oleksandr_tyshchenko@epam.com>,
"'Julien Grall'" <julien.grall@arm.com>,
"'Jan Beulich'" <jbeulich@suse.com>,
xen-devel@lists.xenproject.org,
"'Roger Pau Monné'" <roger.pau@citrix.com>
Subject: Re: [RFC PATCH V1 01/12] hvm/ioreq: Make x86's IOREQ feature common
Date: Mon, 10 Aug 2020 16:34:42 -0700 (PDT) [thread overview]
Message-ID: <alpine.DEB.2.21.2008101433050.16004@sstabellini-ThinkPad-T480s> (raw)
In-Reply-To: <598e2f35-e70e-36a7-1e5d-259ebb2e3cde@xen.org>
[-- Attachment #1: Type: text/plain, Size: 6917 bytes --]
On Mon, 10 Aug 2020, Julien Grall wrote:
> On 07/08/2020 00:48, Stefano Stabellini wrote:
> > On Thu, 6 Aug 2020, Julien Grall wrote:
> > > On 06/08/2020 01:37, Stefano Stabellini wrote:
> > > > On Wed, 5 Aug 2020, Julien Grall wrote:
> > > > > On 04/08/2020 20:11, Stefano Stabellini wrote:
> > > > > > On Tue, 4 Aug 2020, Julien Grall wrote:
> > > > > > > On 04/08/2020 12:10, Oleksandr wrote:
> > > > > > > > On 04.08.20 10:45, Paul Durrant wrote:
> > > > > > > > > > +static inline bool hvm_ioreq_needs_completion(const ioreq_t
> > > > > > > > > > *ioreq)
> > > > > > > > > > +{
> > > > > > > > > > + return ioreq->state == STATE_IOREQ_READY &&
> > > > > > > > > > + !ioreq->data_is_ptr &&
> > > > > > > > > > + (ioreq->type != IOREQ_TYPE_PIO || ioreq->dir !=
> > > > > > > > > > IOREQ_WRITE);
> > > > > > > > > > +}
> > > > > > > > > I don't think having this in common code is correct. The
> > > > > > > > > short-cut
> > > > > > > > > of
> > > > > > > > > not
> > > > > > > > > completing PIO reads seems somewhat x86 specific.
> > > > > > >
> > > > > > > Hmmm, looking at the code, I think it doesn't wait for PIO writes
> > > > > > > to
> > > > > > > complete
> > > > > > > (not read). Did I miss anything?
> > > > > > >
> > > > > > > > Does ARM even
> > > > > > > > > have the concept of PIO?
> > > > > > > >
> > > > > > > > I am not 100% sure here, but it seems that doesn't have.
> > > > > > >
> > > > > > > Technically, the PIOs exist on Arm, however they are accessed the
> > > > > > > same
> > > > > > > way
> > > > > > > as
> > > > > > > MMIO and will have a dedicated area defined by the HW.
> > > > > > >
> > > > > > > AFAICT, on Arm64, they are only used for PCI IO Bar.
> > > > > > >
> > > > > > > Now the question is whether we want to expose them to the Device
> > > > > > > Emulator
> > > > > > > as
> > > > > > > PIO or MMIO access. From a generic PoV, a DM shouldn't have to
> > > > > > > care
> > > > > > > about
> > > > > > > the
> > > > > > > architecture used. It should just be able to request a given
> > > > > > > IOport
> > > > > > > region.
> > > > > > >
> > > > > > > So it may make sense to differentiate them in the common ioreq
> > > > > > > code as
> > > > > > > well.
> > > > > > >
> > > > > > > I had a quick look at QEMU and wasn't able to tell if PIOs and
> > > > > > > MMIOs
> > > > > > > address
> > > > > > > space are different on Arm as well. Paul, Stefano, do you know
> > > > > > > what
> > > > > > > they
> > > > > > > are
> > > > > > > doing?
> > > > > >
> > > > > > On the QEMU side, it looks like PIO (address_space_io) is used in
> > > > > > connection with the emulation of the "in" or "out" instructions, see
> > > > > > ioport.c:cpu_inb for instance. Some parts of PCI on QEMU emulate PIO
> > > > > > space regardless of the architecture, such as
> > > > > > hw/pci/pci_bridge.c:pci_bridge_initfn.
> > > > > >
> > > > > > However, because there is no "in" and "out" on ARM, I don't think
> > > > > > address_space_io can be accessed. Specifically, there is no
> > > > > > equivalent
> > > > > > for target/i386/misc_helper.c:helper_inb on ARM.
> > > > >
> > > > > So how PCI I/O BAR are accessed? Surely, they could be used on Arm,
> > > > > right?
> > > >
> > > > PIO is also memory mapped on ARM and it seems to have its own MMIO
> > > > address window.
> > > This part is already well-understood :). However, this only tell us how an
> > > OS
> > > is accessing a PIO.
> > >
> > > What I am trying to figure out is how the hardware (or QEMU) is meant to
> > > work.
> > >
> > > From my understanding, the MMIO access will be received by the hostbridge
> > > and
> > > then forwarded to the appropriate PCI device. The two questions I am
> > > trying to
> > > answer is: How the I/O BARs are configured? Will it contain an MMIO
> > > address or
> > > an offset?
> > >
> > > If the answer is the latter, then we will need PIO because a DM will never
> > > see
> > > the MMIO address (the hostbridge will be emulated in Xen).
> >
> > Now I understand the question :-)
> >
> > This is the way I understand it works. Let's say that the PIO aperture
> > is 0x1000-0x2000 which is aliased to 0x3eff0000-0x3eff1000.
> > 0x1000-0x2000 are addresses that cannot be accessed directly.
> > 0x3eff0000-0x3eff1000 is the range that works.
> >
> > A PCI device PIO BAR will have an address in the 0x1000-0x2000 range,
> > for instance 0x1100.
> >
> > However, when the operating system access 0x1100, it will issue a read
> > to 0x3eff0100.
> >
> > Xen will trap the read to 0x3eff0100 and send it to QEMU.
> >
> > QEMU has to know that 0x3eff0000-0x3eff1000 is the alias to the PIO
> > aperture and that 0x3eff0100 correspond to PCI device foobar. Similarly,
> > QEMU has also to know the address range of the MMIO aperture and its
> > remappings, if any (it is possible to have address remapping for MMIO
> > addresses too.)
> >
> > I think today this information is "built-in" QEMU, not configurable. It
> > works fine because *I think* the PCI aperture is pretty much the same on
> > x86 boards, at least the one supported by QEMU for Xen.
>
> Well on x86, the OS will access PIO using inb/outb. So the address received by
> Xen is 0x1000-0x2000 and then forwarded to the DM using the PIO type.
>
> > On ARM, I think we should explicitly declare the PCI MMIO aperture and
> > its alias/address-remapping. When we do that, we can also declare the
> > PIO aperture and its alias/address-remapping.
>
> Well yes, we need to define PCI MMIO and PCI I/O region because the guest OS
> needs to know them.
[1]
(see below)
> However, I am unsure how this would help us to solve the question whether
> access to the PCI I/O aperture should be sent as a PIO or MMIO.
>
> Per what you wrote, the PCI I/O Bar would be configured with the range
> 0x1000-0x2000. So a device emulator (this may not be QEMU and only emulate one
> PCI device!!) will only see that range.
>
> How does the device-emulator then know that it needs to watch the region
> 0x3eff0000-0x3eff1000?
It would know because the PCI PIO aperture, together with the alias, are
specified [1].
> It feels to me that it would be easier/make more sense if the DM only say "I
> want to watch the PIO range 0x1000-0x2000". So Xen would be in charge to do
> the translation between the OS view and the DM view.
>
> This also means a DM would be completely arch-agnostic. This would follow the
> HW where you can plug your PCI card on any HW.
As you know, PIO access is actually not modelled by QEMU for ARM
targets. I worry about the long term stability of it, given that it is
untested. I.e. qemu-system-aarch64 could have a broken PIO emulation
and nobody would find out except for us when we send ioreqs to it.
Thinking from a Xen/Emulator interface on ARM, is it wise to rely on an
access-type that doesn't exist on the architecture?
next prev parent reply other threads:[~2020-08-10 23:34 UTC|newest]
Thread overview: 140+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-03 18:21 [RFC PATCH V1 00/12] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
2020-08-03 18:21 ` [RFC PATCH V1 01/12] hvm/ioreq: Make x86's IOREQ feature common Oleksandr Tyshchenko
2020-08-04 7:45 ` Paul Durrant
2020-08-04 11:10 ` Oleksandr
2020-08-04 11:23 ` Paul Durrant
2020-08-04 11:51 ` Oleksandr
2020-08-04 13:18 ` Paul Durrant
2020-08-04 13:52 ` Julien Grall
2020-08-04 15:41 ` Jan Beulich
2020-08-04 19:11 ` Stefano Stabellini
2020-08-05 7:01 ` Jan Beulich
2020-08-06 0:37 ` Stefano Stabellini
2020-08-06 6:59 ` Jan Beulich
2020-08-06 20:32 ` Stefano Stabellini
2020-08-07 13:19 ` Oleksandr
2020-08-07 16:45 ` Oleksandr
2020-08-07 21:50 ` Stefano Stabellini
2020-08-07 22:19 ` Oleksandr
2020-08-10 13:41 ` Oleksandr
2020-08-10 23:34 ` Stefano Stabellini
2020-08-11 9:19 ` Julien Grall
2020-08-11 10:10 ` Oleksandr
2020-08-11 22:47 ` Stefano Stabellini
2020-08-12 14:35 ` Oleksandr
2020-08-12 23:08 ` Stefano Stabellini
2020-08-13 20:16 ` Julien Grall
2020-08-07 23:45 ` Oleksandr
2020-08-10 23:34 ` Stefano Stabellini
2020-08-05 8:33 ` Julien Grall
2020-08-06 0:37 ` Stefano Stabellini
2020-08-06 9:45 ` Julien Grall
2020-08-06 23:48 ` Stefano Stabellini
2020-08-10 19:20 ` Julien Grall
2020-08-10 23:34 ` Stefano Stabellini [this message]
2020-08-11 11:28 ` Julien Grall
2020-08-11 22:48 ` Stefano Stabellini
2020-08-12 8:19 ` Julien Grall
2020-08-20 19:14 ` Oleksandr
2020-08-21 0:53 ` Stefano Stabellini
2020-08-21 18:54 ` Julien Grall
2020-08-05 13:30 ` Julien Grall
2020-08-06 11:37 ` Oleksandr
2020-08-10 16:29 ` Julien Grall
2020-08-10 17:28 ` Oleksandr
2020-08-05 16:15 ` Andrew Cooper
2020-08-06 8:20 ` Oleksandr
2020-08-15 17:30 ` Julien Grall
2020-08-16 19:37 ` Oleksandr
2020-08-03 18:21 ` [RFC PATCH V1 02/12] hvm/dm: Make x86's DM " Oleksandr Tyshchenko
2020-08-03 18:21 ` [RFC PATCH V1 03/12] xen/mm: Make x86's XENMEM_resource_ioreq_server handling common Oleksandr Tyshchenko
2020-08-03 18:21 ` [RFC PATCH V1 04/12] xen/arm: Introduce arch specific bits for IOREQ/DM features Oleksandr Tyshchenko
2020-08-04 7:49 ` Paul Durrant
2020-08-04 14:01 ` Julien Grall
2020-08-04 23:22 ` Stefano Stabellini
2020-08-15 17:56 ` Julien Grall
2020-08-17 14:36 ` Oleksandr
2020-08-04 23:22 ` Stefano Stabellini
2020-08-05 7:05 ` Jan Beulich
2020-08-05 16:41 ` Stefano Stabellini
2020-08-05 19:45 ` Oleksandr
2020-08-05 9:32 ` Julien Grall
2020-08-05 15:41 ` Oleksandr
2020-08-06 10:19 ` Julien Grall
2020-08-10 18:09 ` Oleksandr
2020-08-10 18:21 ` Oleksandr
2020-08-10 19:00 ` Julien Grall
2020-08-10 20:29 ` Oleksandr
2020-08-10 22:37 ` Julien Grall
2020-08-11 6:13 ` Oleksandr
2020-08-12 15:08 ` Oleksandr
2020-08-11 17:09 ` Oleksandr
2020-08-11 17:50 ` Julien Grall
2020-08-13 18:41 ` Oleksandr
2020-08-13 20:36 ` Julien Grall
2020-08-13 21:49 ` Oleksandr
2020-08-13 20:39 ` Oleksandr Tyshchenko
2020-08-13 22:14 ` Julien Grall
2020-08-14 12:08 ` Oleksandr
2020-08-05 14:12 ` Julien Grall
2020-08-05 14:45 ` Jan Beulich
2020-08-05 19:30 ` Oleksandr
2020-08-06 11:08 ` Julien Grall
2020-08-06 11:29 ` Jan Beulich
2020-08-20 18:30 ` Oleksandr
2020-08-21 6:16 ` Jan Beulich
2020-08-21 11:13 ` Oleksandr
2020-08-06 13:27 ` Oleksandr
2020-08-10 18:25 ` Julien Grall
2020-08-10 19:58 ` Oleksandr
2020-08-05 16:13 ` Jan Beulich
2020-08-05 19:47 ` Oleksandr
2020-08-03 18:21 ` [RFC PATCH V1 05/12] hvm/dm: Introduce xendevicemodel_set_irq_level DM op Oleksandr Tyshchenko
2020-08-04 23:22 ` Stefano Stabellini
2020-08-05 9:39 ` Julien Grall
2020-08-06 0:37 ` Stefano Stabellini
2020-08-06 11:32 ` Julien Grall
2020-08-06 23:49 ` Stefano Stabellini
2020-08-07 8:43 ` Jan Beulich
2020-08-07 21:50 ` Stefano Stabellini
2020-08-08 9:27 ` Julien Grall
2020-08-08 9:28 ` Julien Grall
2020-08-10 23:34 ` Stefano Stabellini
2020-08-11 13:04 ` Julien Grall
2020-08-11 22:48 ` Stefano Stabellini
2020-08-18 9:31 ` Julien Grall
2020-08-21 0:53 ` Stefano Stabellini
2020-08-17 15:23 ` Jan Beulich
2020-08-17 22:56 ` Stefano Stabellini
2020-08-18 8:03 ` Jan Beulich
2020-08-05 16:15 ` Jan Beulich
2020-08-05 22:12 ` Oleksandr
2020-08-03 18:21 ` [RFC PATCH V1 06/12] libxl: Introduce basic virtio-mmio support on Arm Oleksandr Tyshchenko
2020-08-03 18:21 ` [RFC PATCH V1 07/12] A collection of tweaks to be able to run emulator in driver domain Oleksandr Tyshchenko
2020-08-05 16:19 ` Jan Beulich
2020-08-05 16:40 ` Paul Durrant
2020-08-06 9:22 ` Oleksandr
2020-08-06 9:27 ` Jan Beulich
2020-08-14 16:30 ` Oleksandr
2020-08-16 15:36 ` Julien Grall
2020-08-17 15:07 ` Oleksandr
2020-08-03 18:21 ` [RFC PATCH V1 08/12] xen/arm: Invalidate qemu mapcache on XENMEM_decrease_reservation Oleksandr Tyshchenko
2020-08-05 16:21 ` Jan Beulich
2020-08-06 11:35 ` Julien Grall
2020-08-06 11:50 ` Jan Beulich
2020-08-06 14:28 ` Oleksandr
2020-08-06 16:33 ` Jan Beulich
2020-08-06 16:57 ` Oleksandr
2020-08-03 18:21 ` [RFC PATCH V1 09/12] libxl: Handle virtio-mmio irq in more correct way Oleksandr Tyshchenko
2020-08-04 23:22 ` Stefano Stabellini
2020-08-05 20:51 ` Oleksandr
2020-08-03 18:21 ` [RFC PATCH V1 10/12] libxl: Add support for virtio-disk configuration Oleksandr Tyshchenko
2020-08-04 23:23 ` Stefano Stabellini
2020-08-05 21:12 ` Oleksandr
2020-08-06 0:37 ` Stefano Stabellini
2020-08-03 18:21 ` [RFC PATCH V1 11/12] libxl: Insert "dma-coherent" property into virtio-mmio device node Oleksandr Tyshchenko
2020-08-04 23:23 ` Stefano Stabellini
2020-08-05 20:35 ` Oleksandr
2020-08-03 18:21 ` [RFC PATCH V1 12/12] libxl: Fix duplicate memory node in DT Oleksandr Tyshchenko
2020-08-15 17:24 ` [RFC PATCH V1 00/12] IOREQ feature (+ virtio-mmio) on Arm Julien Grall
2020-08-16 19:34 ` Oleksandr
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.DEB.2.21.2008101433050.16004@sstabellini-ThinkPad-T480s \
--to=sstabellini@kernel.org \
--cc=andrew.cooper3@citrix.com \
--cc=george.dunlap@citrix.com \
--cc=ian.jackson@eu.citrix.com \
--cc=jbeulich@suse.com \
--cc=julien.grall@arm.com \
--cc=julien@xen.org \
--cc=jun.nakajima@intel.com \
--cc=kevin.tian@intel.com \
--cc=oleksandr_tyshchenko@epam.com \
--cc=olekstysh@gmail.com \
--cc=paul@xen.org \
--cc=roger.pau@citrix.com \
--cc=tim@xen.org \
--cc=wl@xen.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).