All of lore.kernel.org
 help / color / mirror / Atom feed
* PCI passthrough with stubdomain
@ 2016-09-23  8:48 Marek Marczykowski-Górecki
  2016-09-23  9:22 ` Jan Beulich
  2016-09-23 13:27 ` Samuel Thibault
  0 siblings, 2 replies; 10+ messages in thread
From: Marek Marczykowski-Górecki @ 2016-09-23  8:48 UTC (permalink / raw)
  To: xen-devel; +Cc: HW42


[-- Attachment #1.1: Type: text/plain, Size: 1242 bytes --]

Hi,

I'm still trying to get PCI passthrough working with stubdomain and
without qemu in dom0 (even for just vfb/vkbd backends). How is this
supposed to work?

1. Should xen-pcifront in the target domain be used (and consequently,
should xen-pciback be set for it)? Currently xen-pciback is set for both
stubdomain and target domain, which presents a race condition and
xen-pciback refuses to setup one of them.
(I have a patch for this, but not sure how it really should work)

1a. How does it look in case of qemu in dom0 (no stubdomain)?

2. What operations are done through stubdomain and what are handled
directly by Xen (itself, or with help of IOMMU)? I'd guess config space
accesses are done through device model. Anything else?

3. What changes (if any) are required in qemu-xen to have it working in
stubdomain in regards to PCI passthrough? Should it just work (assuming
Linux-based stubdomain with xen-pcifront driver)?

PS If anyone interested, here are more details:
https://github.com/QubesOS/qubes-issues/issues/1659

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: PCI passthrough with stubdomain
  2016-09-23  8:48 PCI passthrough with stubdomain Marek Marczykowski-Górecki
@ 2016-09-23  9:22 ` Jan Beulich
  2016-09-23 13:27 ` Samuel Thibault
  1 sibling, 0 replies; 10+ messages in thread
From: Jan Beulich @ 2016-09-23  9:22 UTC (permalink / raw)
  To: Marek Marczykowski-Górecki; +Cc: HW42, xen-devel

>>> On 23.09.16 at 10:48, <marmarek@invisiblethingslab.com> wrote:
> 2. What operations are done through stubdomain and what are handled
> directly by Xen (itself, or with help of IOMMU)? I'd guess config space
> accesses are done through device model. Anything else?

MSI-X table handling I think (guest accesses really mainly get
snooped by Xen).

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: PCI passthrough with stubdomain
  2016-09-23  8:48 PCI passthrough with stubdomain Marek Marczykowski-Górecki
  2016-09-23  9:22 ` Jan Beulich
@ 2016-09-23 13:27 ` Samuel Thibault
  2016-09-23 14:25   ` Marek Marczykowski-Górecki
  1 sibling, 1 reply; 10+ messages in thread
From: Samuel Thibault @ 2016-09-23 13:27 UTC (permalink / raw)
  To: Marek Marczykowski-Górecki; +Cc: HW42, xen-devel

Marek Marczykowski-Górecki, on Fri 23 Sep 2016 10:48:14 +0200, wrote:
> I'm still trying to get PCI passthrough working with stubdomain and
> without qemu in dom0 (even for just vfb/vkbd backends). How is this
> supposed to work?

Just as I recall from my memory:

> 1. Should xen-pcifront in the target domain be used (and consequently,
> should xen-pciback be set for it)?

I guess that could work.

> Currently xen-pciback is set for both
> stubdomain and target domain, which presents a race condition and
> xen-pciback refuses to setup one of them.

Yes, that's expected, for the reason you say.


For each question, I'll answer for two cases: either plain PCI drivers
in the guest, or xen-pvPCI drivers.


* Using plain PCI drivers.
**************************
I.e. no PV from the point of view of the guest, it directly pokes I/O
ports emulated by qemu.

> 1a. How does it look in case of qemu in dom0 (no stubdomain)?

qemu uses libpci (from pciutils) to access the board and pass through
requests coming from the guest. no pciback should thus be set since qemu
pokes directly from dom0.

> 2. What operations are done through stubdomain and what are handled
> directly by Xen (itself, or with help of IOMMU)? I'd guess config space
> accesses are done through device model. Anything else?

When using a qemu stubdomain, qemu still uses libpci to access the
board. The stubdom/ directory contains a patch to make the stubdom
libpci use the minios PV frontend. Thus, the pciback should be set for
the stubdom, since that's the one which will poke it through PV.  I
don't remember how the guest iomemory access are handled. At worse
they're trapped into qemu that does the access from the stubdom. At best
qemu maps the pages into the guest memory, and thus the guest directly
accesses it through Xen (potentially made safe by IOMMU). I guess I
implemented the latter, but that's far away, so my memory might be wrong
:)

> 3. What changes (if any) are required in qemu-xen to have it working in
> stubdomain in regards to PCI passthrough? Should it just work (assuming
> Linux-based stubdomain with xen-pcifront driver)?

IIRC it should just work, just need to set a PV pcifront on the stubdom
when using a stubdom. In the dom0 case qemu will access the PCI board
directly.


* Using PV drivers from the guest.
**********************************
So the PV is running its own PV drivers.
The stubdom thus doesn't have to know about it.

> 1a. How does it look in case of qemu in dom0 (no stubdomain)?

qemu will not manage it, the guest will talk directly with the pciback,
to be set on the guest.

> 2. What operations are done through stubdomain and what are handled
> directly by Xen (itself, or with help of IOMMU)? I'd guess config space
> accesses are done through device model. Anything else?

Everything goes through Xen, potentially with the use of IOMMU. config
space goes through PV too, values are read from the xenstore. See
minios' pcifront_physical_to_virtual for an instance how it's
implemented. The iomemory accesses are done by the guest PV driver by
just mapping the right physical pages.

> 3. What changes (if any) are required in qemu-xen to have it working in
> stubdomain in regards to PCI passthrough? Should it just work (assuming
> Linux-based stubdomain with xen-pcifront driver)?

qemu is not involved, so it doesn't have to be changed :)


* to summarize
**************

If running PV drivers in the guest, you set the pciback on the guest, be
it run with stubdom or not. 
If running plain drivers in the guest,
  * when not using a stubdom you don't need to set a pciback.
  * when using a stubdom you need to set a pciback on the stubdom.

So the unfortunate thing is that when using stubdom, you'd have to set
the pciback either on the guest (to run a PV driver in it), or on the
stubdom (to run a plain driver in the guest, and let mini-os use PV to
let qemu pass the board through)

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: PCI passthrough with stubdomain
  2016-09-23 13:27 ` Samuel Thibault
@ 2016-09-23 14:25   ` Marek Marczykowski-Górecki
  2016-09-23 14:51     ` Samuel Thibault
  2016-09-23 15:35     ` Konrad Rzeszutek Wilk
  0 siblings, 2 replies; 10+ messages in thread
From: Marek Marczykowski-Górecki @ 2016-09-23 14:25 UTC (permalink / raw)
  To: Samuel Thibault, xen-devel, HW42


[-- Attachment #1.1.1: Type: text/plain, Size: 3183 bytes --]

On Fri, Sep 23, 2016 at 03:27:07PM +0200, Samuel Thibault wrote:
> Marek Marczykowski-Górecki, on Fri 23 Sep 2016 10:48:14 +0200, wrote:
> > I'm still trying to get PCI passthrough working with stubdomain and
> > without qemu in dom0 (even for just vfb/vkbd backends). How is this
> > supposed to work?
> 
> Just as I recall from my memory:
> 
> > 1. Should xen-pcifront in the target domain be used (and consequently,
> > should xen-pciback be set for it)?
> 
> I guess that could work.

Could, or should? ;)

In the meantime, I've found this in xen-pcifront driver:

	static int __init pcifront_init(void)
	{
		if (!xen_pv_domain() || xen_initial_domain())
			return -ENODEV;

So, it looks like pcifront is not used in PV domain.

> > Currently xen-pciback is set for both
> > stubdomain and target domain, which presents a race condition and
> > xen-pciback refuses to setup one of them.
> 
> Yes, that's expected, for the reason you say.

In the meantime I've tried to modify libxl to not setup pciback for
target domain. This, along with other modifications (see below) improved
the situation.

> * to summarize
> **************
>
> If running PV drivers in the guest, you set the pciback on the guest, be
> it run with stubdom or not. 
> If running plain drivers in the guest,
>   * when not using a stubdom you don't need to set a pciback.
>   * when using a stubdom you need to set a pciback on the stubdom.

Thanks for the explanation!

> So the unfortunate thing is that when using stubdom, you'd have to set
> the pciback either on the guest (to run a PV driver in it), or on the
> stubdom (to run a plain driver in the guest, and let mini-os use PV to
> let qemu pass the board through)

I've tried only on Linux HVM guest and as noted above xen-pcifront
doesn't look to be involved. Is it any different on other OSes?

As for getting PCI passthrough working with stubdomain, for now I hit
those problems:
1. getdomaininfo not allowed from stubdomain (already fixed in master),
2. double setup of pciback (explained above) - for now I've disabled one
of them,
3. race condition between pcifront in mini-os and qemu.

Regarding the last one:
Toolstack during (stub)domain startup setup two things, mostly
asynchronously:
1. pciback/pcifront (through standard xenstore entries)
2. instruct qemu to attach device (by writing "pci-ins" to
device-model/xxx/command)

The later fails if pcifront is not initialized yet. For now I've moved
the second step to be really after the first one, including
synchronization through libxl__wait_for_backend call. 
Is it ok? Or maybe it should be done on stubdomain side (handling
"pci-ins" should wait on pcifront)? I guess the same will apply to
qemu-xen case, or any other stubdomain, so probably better have it in
toolstack, right?

Work-in-progress patches attached. The result is that lspci inside the
guest finally list the device. No idea if it's really working yet.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[-- Attachment #1.1.2: 0001-libxl-attach-xen-pciback-only-to-stubdomain.patch --]
[-- Type: text/plain, Size: 1604 bytes --]

From b8a34904411cc6e7a7abdc6296657ff5f3eb92e9 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
Date: Thu, 22 Sep 2016 16:07:14 +0200
Subject: [PATCH 1/2] libxl: attach xen-pciback only to stubdomain
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Organization: Invisible Things Lab
Cc: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 tools/libxl/libxl_create.c | 2 +-
 tools/libxl/libxl_pci.c    | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index c6862b8..5625ef2 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -1429,7 +1429,7 @@ static void domcreate_attach_pci(libxl__egc *egc, libxl__multidev *multidev,
         }
     }
 
-    if (d_config->num_pcidevs > 0) {
+    if (d_config->num_pcidevs > 0 && d_config->c_info.type == LIBXL_DOMAIN_TYPE_PV) {
         ret = libxl__create_pci_backend(gc, domid, d_config->pcidevs,
             d_config->num_pcidevs);
         if (ret < 0) {
diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c
index 19c597e..2942772 100644
--- a/tools/libxl/libxl_pci.c
+++ b/tools/libxl/libxl_pci.c
@@ -1001,7 +1001,7 @@ out:
         }
     }
 
-    if (!starting)
+    if (!starting && !hvm)
         rc = libxl__device_pci_add_xenstore(gc, domid, pcidev, starting);
     else
         rc = 0;
-- 
2.5.5


[-- Attachment #1.1.3: 0002-libxl-attach-PCI-device-to-qemu-only-after-setting-p.patch --]
[-- Type: text/plain, Size: 3805 bytes --]

From 930398583d5a590c511e58057d000e1ed7defb82 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
Date: Fri, 23 Sep 2016 16:13:37 +0200
Subject: [PATCH 2/2] libxl: attach PCI device to qemu only after setting
 pciback/pcifront
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Organization: Invisible Things Lab
Cc: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

When qemu is running in stubdomain, handling "pci-ins" command will fail
if pcifront is not initialized already. Fix this by sending such command
only after confirming that pciback/front is running.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 tools/libxl/libxl_pci.c | 50 ++++++++++++++++++++++++++-----------------------
 1 file changed, 27 insertions(+), 23 deletions(-)

diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c
index 2942772..88f8f5b 100644
--- a/tools/libxl/libxl_pci.c
+++ b/tools/libxl/libxl_pci.c
@@ -891,34 +891,15 @@ static int do_pci_add(libxl__gc *gc, uint32_t domid, libxl_device_pci *pcidev, i
     libxl_ctx *ctx = libxl__gc_owner(gc);
     libxl_domain_type type = libxl__domain_type(gc, domid);
     char *sysfs_path;
+    char *be_path;
     FILE *f;
     unsigned long long start, end, flags, size;
-    int irq, i, rc, hvm = 0;
+    int irq, i, rc, hvm = (type == LIBXL_DOMAIN_TYPE_HVM);
     uint32_t flag = XEN_DOMCTL_DEV_RDM_RELAXED;
 
     if (type == LIBXL_DOMAIN_TYPE_INVALID)
         return ERROR_FAIL;
 
-    if (type == LIBXL_DOMAIN_TYPE_HVM) {
-        hvm = 1;
-        if (libxl__wait_for_device_model_deprecated(gc, domid, "running",
-                                         NULL, NULL, NULL) < 0) {
-            return ERROR_FAIL;
-        }
-        switch (libxl__device_model_version_running(gc, domid)) {
-            case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL:
-                rc = qemu_pci_add_xenstore(gc, domid, pcidev);
-                break;
-            case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
-                rc = libxl__qmp_pci_add(gc, domid, pcidev);
-                break;
-            default:
-                return ERROR_INVAL;
-        }
-        if ( rc )
-            return ERROR_FAIL;
-    }
-
     sysfs_path = libxl__sprintf(gc, SYSFS_PCI_DEV"/"PCI_BDF"/resource", pcidev->domain,
                                 pcidev->bus, pcidev->dev, pcidev->func);
     f = fopen(sysfs_path, "r");
@@ -1001,10 +982,33 @@ out:
         }
     }
 
-    if (!starting && !hvm)
+    if (!starting && !hvm) {
         rc = libxl__device_pci_add_xenstore(gc, domid, pcidev, starting);
-    else
+        be_path = libxl__sprintf(gc, "%s/backend/pci/%d/0", libxl__xs_get_dompath(gc, 0), domid);
+        if (libxl__wait_for_backend(gc, be_path, GCSPRINTF("%d", XenbusStateConnected)) < 0)
+            return ERROR_FAIL;
+    } else
         rc = 0;
+
+    if (type == LIBXL_DOMAIN_TYPE_HVM) {
+        if (libxl__wait_for_device_model_deprecated(gc, domid, "running",
+                                         NULL, NULL, NULL) < 0) {
+            return ERROR_FAIL;
+        }
+        switch (libxl__device_model_version_running(gc, domid)) {
+            case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL:
+                rc = qemu_pci_add_xenstore(gc, domid, pcidev);
+                break;
+            case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
+                rc = libxl__qmp_pci_add(gc, domid, pcidev);
+                break;
+            default:
+                return ERROR_INVAL;
+        }
+        if ( rc )
+            return ERROR_FAIL;
+    }
+
     return rc;
 }
 
-- 
2.5.5


[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: PCI passthrough with stubdomain
  2016-09-23 14:25   ` Marek Marczykowski-Górecki
@ 2016-09-23 14:51     ` Samuel Thibault
  2016-09-23 18:56       ` Marek Marczykowski-Górecki
  2016-09-23 15:35     ` Konrad Rzeszutek Wilk
  1 sibling, 1 reply; 10+ messages in thread
From: Samuel Thibault @ 2016-09-23 14:51 UTC (permalink / raw)
  To: Marek Marczykowski-Górecki; +Cc: HW42, xen-devel

Marek Marczykowski-Górecki, on Fri 23 Sep 2016 16:25:41 +0200, wrote:
> On Fri, Sep 23, 2016 at 03:27:07PM +0200, Samuel Thibault wrote:
> > Marek Marczykowski-Górecki, on Fri 23 Sep 2016 10:48:14 +0200, wrote:
> > > I'm still trying to get PCI passthrough working with stubdomain and
> > > without qemu in dom0 (even for just vfb/vkbd backends). How is this
> > > supposed to work?
> > 
> > Just as I recall from my memory:
> > 
> > > 1. Should xen-pcifront in the target domain be used (and consequently,
> > > should xen-pciback be set for it)?
> > 
> > I guess that could work.
> 
> Could, or should? ;)

I don't really remember, actually. I do remember doing some passthrough
tests, but I don't remember the exact conditions.

> In the meantime, I've found this in xen-pcifront driver:
> 
> 	static int __init pcifront_init(void)
> 	{
> 		if (!xen_pv_domain() || xen_initial_domain())
> 			return -ENODEV;
> 
> So, it looks like pcifront is not used in PV domain.

? I read it as disabling pcifront when used in an HVM or native domain,
i.e. precisely used in PV domains.

> > So the unfortunate thing is that when using stubdom, you'd have to set
> > the pciback either on the guest (to run a PV driver in it), or on the
> > stubdom (to run a plain driver in the guest, and let mini-os use PV to
> > let qemu pass the board through)
> 
> I've tried only on Linux HVM guest and as noted above xen-pcifront
> doesn't look to be involved. Is it any different on other OSes?

I don't know, perhaps it's different on windows. Or perhaps for windows
we just rely on the qemu pass-through.

> Toolstack during (stub)domain startup setup two things, mostly
> asynchronously:
> 1. pciback/pcifront (through standard xenstore entries)
> 2. instruct qemu to attach device (by writing "pci-ins" to
> device-model/xxx/command)

Ah, since pcifront_watches runs in a separate thread, it may not have
called init_pcifront before qemu calls libpci, i.e. pcifront_scan.

I'd say that's where the fix should be: make pcifront_scan wait for
init_pcifront to be done.  It's however not so simple: we want to make
pcifront_scan do nothing if no pciback is to be launched. My memory is
rusty: is there something we are sure will show up in the xenstore when
a pciback is to be launched?  If so, pcifront_scan could do this: wait
for pcifront_watches to have checked for the potential presence of
pciback. If pciback is not to be started, just do nothing; if pciback is
to be started, wait for init_pcifront to have been called by
pcifront_watches. Then it will find the devices.

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: PCI passthrough with stubdomain
  2016-09-23 14:25   ` Marek Marczykowski-Górecki
  2016-09-23 14:51     ` Samuel Thibault
@ 2016-09-23 15:35     ` Konrad Rzeszutek Wilk
  2016-09-23 18:47       ` Marek Marczykowski-Górecki
  1 sibling, 1 reply; 10+ messages in thread
From: Konrad Rzeszutek Wilk @ 2016-09-23 15:35 UTC (permalink / raw)
  To: Marek Marczykowski-Górecki; +Cc: Samuel Thibault, HW42, xen-devel

On Fri, Sep 23, 2016 at 04:25:41PM +0200, Marek Marczykowski-Górecki wrote:
> On Fri, Sep 23, 2016 at 03:27:07PM +0200, Samuel Thibault wrote:
> > Marek Marczykowski-Górecki, on Fri 23 Sep 2016 10:48:14 +0200, wrote:
> > > I'm still trying to get PCI passthrough working with stubdomain and
> > > without qemu in dom0 (even for just vfb/vkbd backends). How is this
> > > supposed to work?
> > 
> > Just as I recall from my memory:
> > 
> > > 1. Should xen-pcifront in the target domain be used (and consequently,
> > > should xen-pciback be set for it)?
> > 
> > I guess that could work.
> 
> Could, or should? ;)
> 
> In the meantime, I've found this in xen-pcifront driver:
> 
> 	static int __init pcifront_init(void)
> 	{
> 		if (!xen_pv_domain() || xen_initial_domain())
> 			return -ENODEV;
> 
> So, it looks like pcifront is not used in PV domain.

You mean in HVM domains.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: PCI passthrough with stubdomain
  2016-09-23 15:35     ` Konrad Rzeszutek Wilk
@ 2016-09-23 18:47       ` Marek Marczykowski-Górecki
  0 siblings, 0 replies; 10+ messages in thread
From: Marek Marczykowski-Górecki @ 2016-09-23 18:47 UTC (permalink / raw)
  To: Konrad Rzeszutek Wilk; +Cc: Samuel Thibault, HW42, xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 1231 bytes --]

On Fri, Sep 23, 2016 at 11:35:27AM -0400, Konrad Rzeszutek Wilk wrote:
> On Fri, Sep 23, 2016 at 04:25:41PM +0200, Marek Marczykowski-Górecki wrote:
> > On Fri, Sep 23, 2016 at 03:27:07PM +0200, Samuel Thibault wrote:
> > > Marek Marczykowski-Górecki, on Fri 23 Sep 2016 10:48:14 +0200, wrote:
> > > > I'm still trying to get PCI passthrough working with stubdomain and
> > > > without qemu in dom0 (even for just vfb/vkbd backends). How is this
> > > > supposed to work?
> > > 
> > > Just as I recall from my memory:
> > > 
> > > > 1. Should xen-pcifront in the target domain be used (and consequently,
> > > > should xen-pciback be set for it)?
> > > 
> > > I guess that could work.
> > 
> > Could, or should? ;)
> > 
> > In the meantime, I've found this in xen-pcifront driver:
> > 
> > 	static int __init pcifront_init(void)
> > 	{
> > 		if (!xen_pv_domain() || xen_initial_domain())
> > 			return -ENODEV;
> > 
> > So, it looks like pcifront is not used in PV domain.
> 
> You mean in HVM domains.

Of course.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: PCI passthrough with stubdomain
  2016-09-23 14:51     ` Samuel Thibault
@ 2016-09-23 18:56       ` Marek Marczykowski-Górecki
  2016-09-23 21:00         ` Samuel Thibault
  0 siblings, 1 reply; 10+ messages in thread
From: Marek Marczykowski-Górecki @ 2016-09-23 18:56 UTC (permalink / raw)
  To: Samuel Thibault, xen-devel, HW42


[-- Attachment #1.1: Type: text/plain, Size: 1719 bytes --]

On Fri, Sep 23, 2016 at 04:51:33PM +0200, Samuel Thibault wrote:
> Marek Marczykowski-Górecki, on Fri 23 Sep 2016 16:25:41 +0200, wrote:
> > Toolstack during (stub)domain startup setup two things, mostly
> > asynchronously:
> > 1. pciback/pcifront (through standard xenstore entries)
> > 2. instruct qemu to attach device (by writing "pci-ins" to
> > device-model/xxx/command)
> 
> Ah, since pcifront_watches runs in a separate thread, it may not have
> called init_pcifront before qemu calls libpci, i.e. pcifront_scan.

Yes, exactly.

> I'd say that's where the fix should be: make pcifront_scan wait for
> init_pcifront to be done.  It's however not so simple: we want to make
> pcifront_scan do nothing if no pciback is to be launched. My memory is
> rusty: is there something we are sure will show up in the xenstore when
> a pciback is to be launched?  If so, pcifront_scan could do this: wait
> for pcifront_watches to have checked for the potential presence of
> pciback. If pciback is not to be started, just do nothing; if pciback is
> to be started, wait for init_pcifront to have been called by
> pcifront_watches. Then it will find the devices.

Ok. So, two questions:
1. How to do this? ;) I.e. what synchronization primitives are available
in mini-os? Just pthread_mutex_lock/unlock?

2. Wouldn't the same problem be with other stubdomain implementations?
Like Linux + qemu-xen or rumprun + qemu-xen? At least in Linux case
pcifront driver will also needs some time for initialization...

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: PCI passthrough with stubdomain
  2016-09-23 18:56       ` Marek Marczykowski-Górecki
@ 2016-09-23 21:00         ` Samuel Thibault
  2016-09-23 21:04           ` Marek Marczykowski-Górecki
  0 siblings, 1 reply; 10+ messages in thread
From: Samuel Thibault @ 2016-09-23 21:00 UTC (permalink / raw)
  To: Marek Marczykowski-Górecki; +Cc: HW42, xen-devel

Marek Marczykowski-Górecki, on Fri 23 Sep 2016 20:56:43 +0200, wrote:
> 1. How to do this? ;) I.e. what synchronization primitives are available
> in mini-os? Just pthread_mutex_lock/unlock?

pthread_mutex_lock are nops :o) because we don't have pthread_create.
But for mini-os itself there are synchronization primitives, yes:
there are also semaphores (./include/semaphore.h) and waitqueues
(include/wait.h).

> 2. Wouldn't the same problem be with other stubdomain implementations?
> Like Linux + qemu-xen or rumprun + qemu-xen? At least in Linux case
> pcifront driver will also needs some time for initialization...

Possibly, I don't know about them, they didn't exist at the time ;)

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: PCI passthrough with stubdomain
  2016-09-23 21:00         ` Samuel Thibault
@ 2016-09-23 21:04           ` Marek Marczykowski-Górecki
  0 siblings, 0 replies; 10+ messages in thread
From: Marek Marczykowski-Górecki @ 2016-09-23 21:04 UTC (permalink / raw)
  To: Samuel Thibault, xen-devel, HW42


[-- Attachment #1.1: Type: text/plain, Size: 1107 bytes --]

On Fri, Sep 23, 2016 at 11:00:42PM +0200, Samuel Thibault wrote:
> Marek Marczykowski-Górecki, on Fri 23 Sep 2016 20:56:43 +0200, wrote:
> > 1. How to do this? ;) I.e. what synchronization primitives are available
> > in mini-os? Just pthread_mutex_lock/unlock?
> 
> pthread_mutex_lock are nops :o) because we don't have pthread_create.
> But for mini-os itself there are synchronization primitives, yes:
> there are also semaphores (./include/semaphore.h) and waitqueues
> (include/wait.h).

Ok, will take a look at this. Thanks.

> > 2. Wouldn't the same problem be with other stubdomain implementations?
> > Like Linux + qemu-xen or rumprun + qemu-xen? At least in Linux case
> > pcifront driver will also needs some time for initialization...
> 
> Possibly, I don't know about them, they didn't exist at the time ;)

Actually they don't exist even now ;) But hopefully will, in the near
future...

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2016-09-23 21:04 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-09-23  8:48 PCI passthrough with stubdomain Marek Marczykowski-Górecki
2016-09-23  9:22 ` Jan Beulich
2016-09-23 13:27 ` Samuel Thibault
2016-09-23 14:25   ` Marek Marczykowski-Górecki
2016-09-23 14:51     ` Samuel Thibault
2016-09-23 18:56       ` Marek Marczykowski-Górecki
2016-09-23 21:00         ` Samuel Thibault
2016-09-23 21:04           ` Marek Marczykowski-Górecki
2016-09-23 15:35     ` Konrad Rzeszutek Wilk
2016-09-23 18:47       ` Marek Marczykowski-Górecki

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.