From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org, sstabellini@kernel.org,
oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com,
Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com,
andrew.cooper3@citrix.com, george.dunlap@citrix.com,
paul@xen.org, bertrand.marquis@arm.com, rahul.singh@arm.com,
Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v4 10/11] vpci: add initial support for virtual PCI bus topology
Date: Fri, 5 Nov 2021 08:56:28 +0200 [thread overview]
Message-ID: <20211105065629.940943-11-andr2000@gmail.com> (raw)
In-Reply-To: <20211105065629.940943-1-andr2000@gmail.com>
From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Assign SBDF to the PCI devices being passed through with bus 0.
The resulting topology is where PCIe devices reside on the bus 0 of the
root complex itself (embedded endpoints).
This implementation is limited to 32 devices which are allowed on
a single PCI bus.
Please note, that at the moment only function 0 of a multifunction
device can be passed through.
Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
Since v3:
- make use of VPCI_INIT
- moved all new code to vpci.c which belongs to it
- changed open-coded 31 to PCI_SLOT(~0)
- revisited locking: add dedicated vdev list's lock
- added comments and code to reject multifunction devices with
functions other than 0
- updated comment about vpci_dev_next and made it unsigned int
- implement roll back in case of error while assigning/deassigning devices
- s/dom%pd/%pd
Since v2:
- remove casts that are (a) malformed and (b) unnecessary
- add new line for better readability
- remove CONFIG_HAS_VPCI_GUEST_SUPPORT ifdef's as the relevant vPCI
functions are now completely gated with this config
- gate common code with CONFIG_HAS_VPCI_GUEST_SUPPORT
New in v2
---
xen/drivers/vpci/vpci.c | 52 +++++++++++++++++++++++++++++++++++++++++
xen/include/xen/sched.h | 8 +++++++
xen/include/xen/vpci.h | 4 ++++
3 files changed, 64 insertions(+)
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 45733300f00b..6657d236dc1a 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -101,6 +101,9 @@ int vpci_add_handlers(struct pci_dev *pdev)
INIT_LIST_HEAD(&pdev->vpci->handlers);
spin_lock_init(&pdev->vpci->lock);
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+ pdev->vpci->guest_sbdf.sbdf = ~0;
+#endif
header = &pdev->vpci->header;
for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
@@ -125,6 +128,54 @@ int vpci_add_handlers(struct pci_dev *pdev)
}
#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+int vpci_add_virtual_device(struct pci_dev *pdev)
+{
+ struct domain *d = pdev->domain;
+ pci_sbdf_t sbdf;
+ unsigned long new_dev_number;
+
+ /*
+ * Each PCI bus supports 32 devices/slots at max or up to 256 when
+ * there are multi-function ones which are not yet supported.
+ */
+ if ( pdev->info.is_extfn )
+ {
+ gdprintk(XENLOG_ERR, "%pp: only function 0 passthrough supported\n",
+ &pdev->sbdf);
+ return -EOPNOTSUPP;
+ }
+
+ new_dev_number = find_first_zero_bit(&d->vpci_dev_assigned_map,
+ PCI_SLOT(~0) + 1);
+ if ( new_dev_number > PCI_SLOT(~0) )
+ return -ENOSPC;
+
+ set_bit(new_dev_number, &d->vpci_dev_assigned_map);
+
+ /*
+ * Both segment and bus number are 0:
+ * - we emulate a single host bridge for the guest, e.g. segment 0
+ * - with bus 0 the virtual devices are seen as embedded
+ * endpoints behind the root complex
+ *
+ * TODO: add support for multi-function devices.
+ */
+ sbdf.sbdf = 0;
+ sbdf.devfn = PCI_DEVFN(new_dev_number, 0);
+ pdev->vpci->guest_sbdf = sbdf;
+
+ return 0;
+
+}
+REGISTER_VPCI_INIT(vpci_add_virtual_device, VPCI_PRIORITY_MIDDLE);
+
+static void vpci_remove_virtual_device(struct domain *d,
+ const struct pci_dev *pdev)
+{
+ clear_bit(pdev->vpci->guest_sbdf.dev, &d->vpci_dev_assigned_map);
+ pdev->vpci->guest_sbdf.sbdf = ~0;
+}
+
/* Notify vPCI that device is assigned to guest. */
int vpci_assign_device(struct domain *d, struct pci_dev *pdev)
{
@@ -150,6 +201,7 @@ int vpci_deassign_device(struct domain *d, struct pci_dev *pdev)
if ( is_system_domain(d) || !has_vpci(d) )
return 0;
+ vpci_remove_virtual_device(d, pdev);
vpci_remove_device_handlers(pdev);
return 0;
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 28146ee404e6..10bff103317c 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -444,6 +444,14 @@ struct domain
#ifdef CONFIG_HAS_PCI
struct list_head pdev_list;
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+ /*
+ * The bitmap which shows which device numbers are already used by the
+ * virtual PCI bus topology and is used to assign a unique SBDF to the
+ * next passed through virtual PCI device.
+ */
+ unsigned long vpci_dev_assigned_map;
+#endif
#endif
#ifdef CONFIG_HAS_PASSTHROUGH
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 143f3166a730..9cc7071bc0af 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -145,6 +145,10 @@ struct vpci {
struct vpci_arch_msix_entry arch;
} entries[];
} *msix;
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+ /* Virtual SBDF of the device. */
+ pci_sbdf_t guest_sbdf;
+#endif
#endif
};
--
2.25.1
next prev parent reply other threads:[~2021-11-05 6:57 UTC|newest]
Thread overview: 101+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-05 6:56 [PATCH v4 00/11] PCI devices passthrough on Arm, part 3 Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 01/11] vpci: fix function attributes for vpci_process_pending Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 02/11] vpci: cancel pending map/unmap on vpci removal Oleksandr Andrushchenko
2021-11-15 16:56 ` Jan Beulich
2021-11-16 7:32 ` Oleksandr Andrushchenko
2021-11-16 8:01 ` Jan Beulich
2021-11-16 8:23 ` Oleksandr Andrushchenko
2021-11-16 11:38 ` Jan Beulich
2021-11-16 13:27 ` Oleksandr Andrushchenko
2021-11-16 14:11 ` Jan Beulich
2021-11-16 13:41 ` Oleksandr Andrushchenko
2021-11-16 14:12 ` Jan Beulich
2021-11-16 14:24 ` Oleksandr Andrushchenko
2021-11-16 14:37 ` Oleksandr Andrushchenko
2021-11-16 16:09 ` Jan Beulich
2021-11-16 18:02 ` Julien Grall
2021-11-18 12:57 ` Oleksandr Andrushchenko
2021-11-17 8:28 ` Jan Beulich
2021-11-18 7:49 ` Oleksandr Andrushchenko
2021-11-18 8:36 ` Jan Beulich
2021-11-18 8:54 ` Oleksandr Andrushchenko
2021-11-18 9:15 ` Jan Beulich
2021-11-18 9:32 ` Oleksandr Andrushchenko
2021-11-18 13:25 ` Jan Beulich
2021-11-18 13:48 ` Oleksandr Andrushchenko
2021-11-18 14:04 ` Roger Pau Monné
2021-11-18 14:14 ` Oleksandr Andrushchenko
2021-11-18 14:35 ` Jan Beulich
2021-11-18 15:11 ` Oleksandr Andrushchenko
2021-11-18 15:16 ` Jan Beulich
2021-11-18 15:21 ` Oleksandr Andrushchenko
2021-11-18 15:41 ` Jan Beulich
2021-11-18 15:46 ` Oleksandr Andrushchenko
2021-11-18 15:53 ` Jan Beulich
2021-11-19 12:34 ` Oleksandr Andrushchenko
2021-11-19 13:00 ` Jan Beulich
2021-11-19 13:16 ` Oleksandr Andrushchenko
2021-11-19 13:25 ` Jan Beulich
2021-11-19 13:34 ` Oleksandr Andrushchenko
2021-11-22 14:21 ` Oleksandr Andrushchenko
2021-11-22 14:37 ` Jan Beulich
2021-11-22 14:45 ` Oleksandr Andrushchenko
2021-11-22 14:57 ` Jan Beulich
2021-11-22 15:02 ` Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 03/11] vpci: make vpci registers removal a dedicated function Oleksandr Andrushchenko
2021-11-15 16:57 ` Jan Beulich
2021-11-16 8:02 ` Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 04/11] vpci: add hooks for PCI device assign/de-assign Oleksandr Andrushchenko
2021-11-15 17:06 ` Jan Beulich
2021-11-16 9:38 ` Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 05/11] vpci/header: implement guest BAR register handlers Oleksandr Andrushchenko
2021-11-19 11:58 ` Jan Beulich
2021-11-19 12:10 ` Oleksandr Andrushchenko
2021-11-19 12:37 ` Jan Beulich
2021-11-19 12:46 ` Oleksandr Andrushchenko
2021-11-19 12:49 ` Jan Beulich
2021-11-19 12:54 ` Oleksandr Andrushchenko
2021-11-19 13:02 ` Jan Beulich
2021-11-19 13:17 ` Oleksandr Andrushchenko
2021-11-23 15:14 ` Oleksandr Andrushchenko
2021-11-24 12:32 ` Roger Pau Monné
2021-11-24 12:36 ` Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 06/11] vpci/header: handle p2m range sets per BAR Oleksandr Andrushchenko
2021-11-19 12:05 ` Jan Beulich
2021-11-19 12:13 ` Oleksandr Andrushchenko
2021-11-19 12:45 ` Jan Beulich
2021-11-19 12:50 ` Oleksandr Andrushchenko
2021-11-19 13:06 ` Jan Beulich
2021-11-19 13:19 ` Oleksandr Andrushchenko
2021-11-19 13:29 ` Jan Beulich
2021-11-19 13:38 ` Oleksandr Andrushchenko
2021-11-19 13:16 ` Jan Beulich
2021-11-19 13:41 ` Oleksandr Andrushchenko
2021-11-19 13:57 ` Jan Beulich
2021-11-19 14:09 ` Oleksandr Andrushchenko
2021-11-22 8:24 ` Jan Beulich
2021-11-22 8:31 ` Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 07/11] vpci/header: program p2m with guest BAR view Oleksandr Andrushchenko
2021-11-19 12:33 ` Jan Beulich
2021-11-19 12:44 ` Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 08/11] vpci/header: emulate PCI_COMMAND register for guests Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 09/11] vpci/header: reset the command register when adding devices Oleksandr Andrushchenko
2021-11-05 6:56 ` Oleksandr Andrushchenko [this message]
2021-11-18 16:45 ` [PATCH v4 10/11] vpci: add initial support for virtual PCI bus topology Jan Beulich
2021-11-24 11:28 ` Oleksandr Andrushchenko
2021-11-24 12:36 ` Roger Pau Monné
2021-11-24 12:43 ` Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 11/11] xen/arm: translate virtual PCI bus topology for guests Oleksandr Andrushchenko
2021-11-08 11:10 ` Jan Beulich
2021-11-08 11:16 ` Oleksandr Andrushchenko
2021-11-08 14:23 ` Roger Pau Monné
2021-11-08 15:28 ` Oleksandr Andrushchenko
2021-11-24 11:31 ` Oleksandr Andrushchenko
2021-11-19 13:56 ` [PATCH v4 00/11] PCI devices passthrough on Arm, part 3 Jan Beulich
2021-11-19 14:06 ` Oleksandr Andrushchenko
2021-11-19 14:23 ` Roger Pau Monné
2021-11-19 14:26 ` Oleksandr Andrushchenko
2021-11-20 9:47 ` Roger Pau Monné
2021-11-22 8:22 ` Jan Beulich
2021-11-22 8:34 ` Oleksandr Andrushchenko
2021-11-22 8:44 ` Jan Beulich
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211105065629.940943-11-andr2000@gmail.com \
--to=andr2000@gmail.com \
--cc=Artem_Mygaiev@epam.com \
--cc=andrew.cooper3@citrix.com \
--cc=bertrand.marquis@arm.com \
--cc=george.dunlap@citrix.com \
--cc=jbeulich@suse.com \
--cc=julien@xen.org \
--cc=oleksandr_andrushchenko@epam.com \
--cc=oleksandr_tyshchenko@epam.com \
--cc=paul@xen.org \
--cc=rahul.singh@arm.com \
--cc=roger.pau@citrix.com \
--cc=sstabellini@kernel.org \
--cc=volodymyr_babchuk@epam.com \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).