From: "Roger Pau Monné" <roger.pau@citrix.com> To: Jason Andryuk <jandryuk@gmail.com> Cc: Stefano Stabellini <sstabellini@kernel.org>, Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <pdurrant@amazon.com>, "marmarek@invisiblethingslab.com" <marmarek@invisiblethingslab.com>, "qemu-devel@nongnu.org" <qemu-devel@nongnu.org>, Simon Gaiser <simon@invisiblethingslab.com>, Anthony Perard <anthony.perard@citrix.com>, "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org> Subject: Re: [Xen-devel] [PATCH 6/6] xen-pt: Round pci regions sizes to XEN_PAGE_SIZE Date: Tue, 14 Jan 2020 19:04:57 +0100 [thread overview] Message-ID: <20200114180457.GG11756@Air-de-Roger> (raw) In-Reply-To: <CAKf6xpuDo=FGTOJ9ipmk7nY+qs+pKBWESqY0ah6B1vS3kZsT=A@mail.gmail.com> On Tue, Jan 14, 2020 at 09:41:46AM -0500, Jason Andryuk wrote: > On Tue, Jan 14, 2020 at 5:04 AM Roger Pau Monné <roger.pau@citrix.com> wrote: > > > > On Mon, Jan 13, 2020 at 02:01:47PM -0500, Jason Andryuk wrote: > > > On Fri, Mar 22, 2019 at 3:43 PM Jason Andryuk <jandryuk@gmail.com> wrote: > > > > > > > > On Thu, Mar 21, 2019 at 11:09 PM Roger Pau Monné <roger.pau@citrix.com> wrote: > > > > > > > > > > The patch below should prevent hvmloader from placing multiple BARs on > > > > > the same page, could you give it a try? > > > > > > > > > > Note that this is not going to prevent the guest from moving those > > > > > BARs around and place them in the same page, thus breaking the initial > > > > > placement done by hvmloader. > > > > > > > > > > Thanks, Roger. > > > > > > > > Hi, Roger. > > > > > > > > I've minimally tested this. Yes, this patch seems to place small BARs > > > > into separate pages. The linux stubdom and QEMU then use the spacing > > > > as provided by hvmloader. > > > > > > Roger, > > > > > > Would you mind submitting this patch to Xen? > > > > Hm, I'm half minded regarding this patch. It feels more like a bandaid > > than a proper solution. Mapping BARs not multiple of page-sizes is > > dangerous because AFAIK there's no entity that asserts there isn't any > > other BAR from a different device on the same page, and hence you > > might end up mapping some MMIO region from another device > > inadvertently. > > We have the guest, linux stubdom with qemu, & dom0. Are you concerned > that all of them need a minimum of page alignment? No, not really. The hardware domain (dom0 in normal deployments) should be the one that makes sure there are no BARs sharing physical pages. > Linux PCI subsytem has an option resource_alignment that can be > applied to either a single device or all devices. Booting with > pci=resource_aligment=4096 will align each device to a page. Do you > think pciback should force resource_alignment=4096 for dom0? Ideally Xen should keep track of the BARs position and size and refuse to passthrough devices that have BARs sharing a page with other devices BARs. > Are > there other MMIO ranges to be concerned about adjacent to BARs? IIRC you can have two BARs of different devices in the same 4K page, BARs are only aligned to it's size, so BARs smaller than 4K are not required to be page aligned. > On my one test machine with a BAR smaller than 4096, the firmware > already sets an alignment of 4096. Linux dom0 seems to keep the > firmware BAR alignment by default. The PCI spec recommend BARs to be sized to a multiple of a page size, but sadly that's not a mandatory requirement. Will submit the patch now, thanks for the ping, I completely forgot about this TBH. Roger.
WARNING: multiple messages have this Message-ID (diff)
From: "Roger Pau Monné" <roger.pau@citrix.com> To: Jason Andryuk <jandryuk@gmail.com> Cc: Stefano Stabellini <sstabellini@kernel.org>, Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <pdurrant@amazon.com>, "marmarek@invisiblethingslab.com" <marmarek@invisiblethingslab.com>, "qemu-devel@nongnu.org" <qemu-devel@nongnu.org>, Simon Gaiser <simon@invisiblethingslab.com>, Anthony Perard <anthony.perard@citrix.com>, "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org> Subject: Re: [Xen-devel] [PATCH 6/6] xen-pt: Round pci regions sizes to XEN_PAGE_SIZE Date: Tue, 14 Jan 2020 19:04:57 +0100 [thread overview] Message-ID: <20200114180457.GG11756@Air-de-Roger> (raw) In-Reply-To: <CAKf6xpuDo=FGTOJ9ipmk7nY+qs+pKBWESqY0ah6B1vS3kZsT=A@mail.gmail.com> On Tue, Jan 14, 2020 at 09:41:46AM -0500, Jason Andryuk wrote: > On Tue, Jan 14, 2020 at 5:04 AM Roger Pau Monné <roger.pau@citrix.com> wrote: > > > > On Mon, Jan 13, 2020 at 02:01:47PM -0500, Jason Andryuk wrote: > > > On Fri, Mar 22, 2019 at 3:43 PM Jason Andryuk <jandryuk@gmail.com> wrote: > > > > > > > > On Thu, Mar 21, 2019 at 11:09 PM Roger Pau Monné <roger.pau@citrix.com> wrote: > > > > > > > > > > The patch below should prevent hvmloader from placing multiple BARs on > > > > > the same page, could you give it a try? > > > > > > > > > > Note that this is not going to prevent the guest from moving those > > > > > BARs around and place them in the same page, thus breaking the initial > > > > > placement done by hvmloader. > > > > > > > > > > Thanks, Roger. > > > > > > > > Hi, Roger. > > > > > > > > I've minimally tested this. Yes, this patch seems to place small BARs > > > > into separate pages. The linux stubdom and QEMU then use the spacing > > > > as provided by hvmloader. > > > > > > Roger, > > > > > > Would you mind submitting this patch to Xen? > > > > Hm, I'm half minded regarding this patch. It feels more like a bandaid > > than a proper solution. Mapping BARs not multiple of page-sizes is > > dangerous because AFAIK there's no entity that asserts there isn't any > > other BAR from a different device on the same page, and hence you > > might end up mapping some MMIO region from another device > > inadvertently. > > We have the guest, linux stubdom with qemu, & dom0. Are you concerned > that all of them need a minimum of page alignment? No, not really. The hardware domain (dom0 in normal deployments) should be the one that makes sure there are no BARs sharing physical pages. > Linux PCI subsytem has an option resource_alignment that can be > applied to either a single device or all devices. Booting with > pci=resource_aligment=4096 will align each device to a page. Do you > think pciback should force resource_alignment=4096 for dom0? Ideally Xen should keep track of the BARs position and size and refuse to passthrough devices that have BARs sharing a page with other devices BARs. > Are > there other MMIO ranges to be concerned about adjacent to BARs? IIRC you can have two BARs of different devices in the same 4K page, BARs are only aligned to it's size, so BARs smaller than 4K are not required to be page aligned. > On my one test machine with a BAR smaller than 4096, the firmware > already sets an alignment of 4096. Linux dom0 seems to keep the > firmware BAR alignment by default. The PCI spec recommend BARs to be sized to a multiple of a page size, but sadly that's not a mandatory requirement. Will submit the patch now, thanks for the ping, I completely forgot about this TBH. Roger. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
next prev parent reply other threads:[~2020-01-14 18:06 UTC|newest] Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-03-11 18:02 [Qemu-devel] [PATCH 0/6] Xen stubdom support Jason Andryuk 2019-03-11 18:02 ` Jason Andryuk 2019-03-11 18:02 ` [Qemu-devel] [PATCH 1/6] xen: Introduce -xen-stubdom option Jason Andryuk 2019-03-11 18:02 ` Jason Andryuk 2019-03-11 18:06 ` [Qemu-devel] " Paolo Bonzini 2019-03-11 18:06 ` Paolo Bonzini 2019-03-11 19:46 ` [Qemu-devel] " Jason Andryuk 2019-03-11 19:46 ` Jason Andryuk 2019-03-11 18:02 ` [Qemu-devel] [PATCH 2/6] xen: Move xenstore initialization to common location Jason Andryuk 2019-03-11 18:02 ` Jason Andryuk 2019-03-13 15:01 ` Paul Durrant 2019-03-13 18:11 ` Jason Andryuk 2019-03-14 14:00 ` Paul Durrant 2019-03-11 18:02 ` [Qemu-devel] [PATCH 3/6] xen: Skip backend initialization for stubdom Jason Andryuk 2019-03-11 18:02 ` Jason Andryuk 2019-03-13 15:04 ` Paul Durrant 2019-03-11 18:02 ` [Qemu-devel] [PATCH 4/6] xen: Set HVM_PARAM_DM_DOMAIN for stubdom on older Xen Jason Andryuk 2019-03-11 18:02 ` Jason Andryuk 2019-03-11 18:02 ` [Qemu-devel] [PATCH 5/6] xen-pt: Hide MSI-X from xen stubdoms Jason Andryuk 2019-03-11 18:02 ` Jason Andryuk 2019-03-12 12:04 ` Roger Pau Monné 2019-03-12 12:38 ` Marek Marczykowski-Górecki 2019-03-12 13:58 ` Jason Andryuk 2019-03-12 14:13 ` Roger Pau Monné 2019-03-12 15:15 ` Jason Andryuk 2019-03-13 2:15 ` Jason Andryuk 2019-03-12 14:29 ` Marek Marczykowski-Górecki 2019-03-11 18:02 ` [Qemu-devel] [PATCH 6/6] xen-pt: Round pci regions sizes to XEN_PAGE_SIZE Jason Andryuk 2019-03-11 18:02 ` Jason Andryuk 2019-03-13 15:09 ` Paul Durrant 2019-03-14 18:15 ` Jason Andryuk 2019-03-14 19:22 ` Simon Gaiser 2019-03-14 19:37 ` Andrew Cooper 2019-03-15 9:12 ` Paul Durrant 2019-03-14 20:45 ` Simon Gaiser 2019-03-15 9:17 ` Paul Durrant 2019-03-15 16:28 ` Andrew Cooper 2019-03-20 17:28 ` Jason Andryuk 2019-03-22 3:09 ` Roger Pau Monné 2019-03-22 19:43 ` Jason Andryuk 2020-01-13 19:01 ` [Xen-devel] " Jason Andryuk 2020-01-13 19:01 ` Jason Andryuk 2020-01-14 10:04 ` Roger Pau Monné 2020-01-14 10:04 ` Roger Pau Monné 2020-01-14 14:41 ` Jason Andryuk 2020-01-14 14:41 ` Jason Andryuk 2020-01-14 18:04 ` Roger Pau Monné [this message] 2020-01-14 18:04 ` Roger Pau Monné 2020-01-15 8:33 ` Durrant, Paul 2020-01-15 8:33 ` Durrant, Paul
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200114180457.GG11756@Air-de-Roger \ --to=roger.pau@citrix.com \ --cc=andrew.cooper3@citrix.com \ --cc=anthony.perard@citrix.com \ --cc=jandryuk@gmail.com \ --cc=marmarek@invisiblethingslab.com \ --cc=pdurrant@amazon.com \ --cc=qemu-devel@nongnu.org \ --cc=simon@invisiblethingslab.com \ --cc=sstabellini@kernel.org \ --cc=xen-devel@lists.xenproject.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.