All of lore.kernel.org
 help / color / mirror / Atom feed
* libxl - avoid calling block script
@ 2018-02-09  1:02 Marek Marczykowski-Górecki
  2018-02-09 10:55 ` Roger Pau Monné
  0 siblings, 1 reply; 8+ messages in thread
From: Marek Marczykowski-Górecki @ 2018-02-09  1:02 UTC (permalink / raw)
  To: xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 3173 bytes --]

Hi,

I'd like to avoid calling block script to speed up domain startup a
little (there may be multiple disks, all already being block devices).
Right now I have restored setting physical-device xenstore entry in
libxl (by reverting [1]), then applying the patch below (it's on 4.8).
This works well for my case, but maybe there is some option to have it
in vanilla Xen? Right now, this require explicit "script=block" to call
the script (for example to setup loop device).

Alternative idea I have is setting disk->script="block" early
(in libxl_device_disk_init()?), so default do not change, but it's still
possible to change it to NULL and avoid calling the script. The problem
is libxl_device_disk_init() is a generated and I don't see how it could
be modified... Any hints?

Yet another idea is having some specific value for disk->script,
that would avoid calling it, but I find this much less elegant solution.

[1] http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=e885362

From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Subject: [PATCH] libxl: do not call default block script

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 tools/libxl/libxl.c       | 8 +++++---
 tools/libxl/libxl_linux.c | 5 ++---
 2 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 11d94ff..74a2421 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -2093,9 +2093,11 @@ static void device_disk_add(libxl__egc *egc, uint32_t domid,
                 flexarray_append(back, "params");
                 flexarray_append(back, dev);
 
-                script = libxl__abs_path(gc, disk->script?: "block",
-                                         libxl__xen_script_dir_path());
-                flexarray_append_pair(back, "script", script);
+                if (disk->script || disk->backend_domid != LIBXL_TOOLSTACK_DOMID) {
+                    script = libxl__abs_path(gc, disk->script?: "block",
+                            libxl__xen_script_dir_path());
+                    flexarray_append_pair(back, "script", script);
+                }
 
                 /* If the user did not supply a block script then we
                  * write the physical-device node ourselves.
diff --git a/tools/libxl/libxl_linux.c b/tools/libxl/libxl_linux.c
index 115332a..923a1d0 100644
--- a/tools/libxl/libxl_linux.c
+++ b/tools/libxl/libxl_linux.c
@@ -334,9 +334,8 @@ static int libxl__hotplug_disk(libxl__gc *gc, libxl__device *dev,
     script = libxl__xs_read(gc, XBT_NULL,
                             GCSPRINTF("%s/%s", be_path, "script"));
     if (!script) {
-        LOGEV(ERROR, errno, "unable to read script from %s", be_path);
-        rc = ERROR_FAIL;
-        goto error;
+        LOG(INFO, "no script for %s", be_path);
+        return 0;
     }
 
     *env = get_hotplug_env(gc, script, dev);
-- 
1.8.1.4


-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: libxl - avoid calling block script
  2018-02-09  1:02 libxl - avoid calling block script Marek Marczykowski-Górecki
@ 2018-02-09 10:55 ` Roger Pau Monné
  2018-02-09 11:03   ` Roger Pau Monné
  0 siblings, 1 reply; 8+ messages in thread
From: Roger Pau Monné @ 2018-02-09 10:55 UTC (permalink / raw)
  To: Marek Marczykowski-Górecki; +Cc: xen-devel

On Fri, Feb 09, 2018 at 02:02:42AM +0100, Marek Marczykowski-Górecki wrote:
> Hi,
> 
> I'd like to avoid calling block script to speed up domain startup a
> little (there may be multiple disks, all already being block devices).
> Right now I have restored setting physical-device xenstore entry in
> libxl (by reverting [1]),

This will work for Linux, but will break for FreeBSD, which expects
physical-device-path instead of physical-device as the output of
hotplug scripts.

> then applying the patch below (it's on 4.8).
> This works well for my case, but maybe there is some option to have it
> in vanilla Xen? Right now, this require explicit "script=block" to call
> the script (for example to setup loop device).
> 
> Alternative idea I have is setting disk->script="block" early
> (in libxl_device_disk_init()?), so default do not change, but it's still
> possible to change it to NULL and avoid calling the script. The problem
> is libxl_device_disk_init() is a generated and I don't see how it could
> be modified... Any hints?
> 
> Yet another idea is having some specific value for disk->script,
> that would avoid calling it, but I find this much less elegant solution.

So the problem is creation time for domains that have quite a lot of
disks attached. Adding Ian and Wei who know more about the async
dispatch system, but I think (at least from a technical PoV) it
should be possible to parallelize device attachment and thus hotplug
script execution. Devices are independent from each other.

Also the Linux hotplug scripts in general seem extremely convoluted,
I'm not sure whether we could gain some speed there just by
simplification.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: libxl - avoid calling block script
  2018-02-09 10:55 ` Roger Pau Monné
@ 2018-02-09 11:03   ` Roger Pau Monné
  2018-02-09 11:35     ` Marek Marczykowski-Górecki
  0 siblings, 1 reply; 8+ messages in thread
From: Roger Pau Monné @ 2018-02-09 11:03 UTC (permalink / raw)
  To: Marek Marczykowski-Górecki; +Cc: Ian Jackson, Wei Liu, xen-devel

Really adding Ian and Wei.

On Fri, Feb 09, 2018 at 10:55:24AM +0000, Roger Pau Monné wrote:
> On Fri, Feb 09, 2018 at 02:02:42AM +0100, Marek Marczykowski-Górecki wrote:
> > Hi,
> > 
> > I'd like to avoid calling block script to speed up domain startup a
> > little (there may be multiple disks, all already being block devices).
> > Right now I have restored setting physical-device xenstore entry in
> > libxl (by reverting [1]),
> 
> This will work for Linux, but will break for FreeBSD, which expects
> physical-device-path instead of physical-device as the output of
> hotplug scripts.
> 
> > then applying the patch below (it's on 4.8).
> > This works well for my case, but maybe there is some option to have it
> > in vanilla Xen? Right now, this require explicit "script=block" to call
> > the script (for example to setup loop device).
> > 
> > Alternative idea I have is setting disk->script="block" early
> > (in libxl_device_disk_init()?), so default do not change, but it's still
> > possible to change it to NULL and avoid calling the script. The problem
> > is libxl_device_disk_init() is a generated and I don't see how it could
> > be modified... Any hints?
> > 
> > Yet another idea is having some specific value for disk->script,
> > that would avoid calling it, but I find this much less elegant solution.
> 
> So the problem is creation time for domains that have quite a lot of
> disks attached. Adding Ian and Wei who know more about the async
> dispatch system, but I think (at least from a technical PoV) it
> should be possible to parallelize device attachment and thus hotplug
> script execution. Devices are independent from each other.
> 
> Also the Linux hotplug scripts in general seem extremely convoluted,
> I'm not sure whether we could gain some speed there just by
> simplification.
> 
> Thanks, Roger.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: libxl - avoid calling block script
  2018-02-09 11:03   ` Roger Pau Monné
@ 2018-02-09 11:35     ` Marek Marczykowski-Górecki
  2018-02-23 18:28       ` Wei Liu
  2018-02-26 11:58       ` Ian Jackson
  0 siblings, 2 replies; 8+ messages in thread
From: Marek Marczykowski-Górecki @ 2018-02-09 11:35 UTC (permalink / raw)
  To: Roger Pau Monné; +Cc: Ian Jackson, Wei Liu, xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 1337 bytes --]

On Fri, Feb 09, 2018 at 11:03:55AM +0000, Roger Pau Monné wrote:
> Really adding Ian and Wei.
> 
> On Fri, Feb 09, 2018 at 10:55:24AM +0000, Roger Pau Monné wrote:
> > So the problem is creation time for domains that have quite a lot of
> > disks attached. Adding Ian and Wei who know more about the async
> > dispatch system, but I think (at least from a technical PoV) it
> > should be possible to parallelize device attachment and thus hotplug
> > script execution. Devices are independent from each other.

In theory yes, but in practice block script (at least on Linux) takes a
lock and serialize execution...

> > Also the Linux hotplug scripts in general seem extremely convoluted,
> > I'm not sure whether we could gain some speed there just by
> > simplification.

Well, we're comparing a bunch of fork+exec(), including starting bash
(default /bin/sh on most systems), with just a single stat() call...
Handling scripts in libxl itself also takes some time (in my case libxl
live in libvirt, which may or may not have an impact). For a domU with
4 disks, getting rid of hotplug scripts saved about 2s of startup time.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: libxl - avoid calling block script
  2018-02-09 11:35     ` Marek Marczykowski-Górecki
@ 2018-02-23 18:28       ` Wei Liu
  2018-02-23 20:14         ` Marek Marczykowski-Górecki
  2018-02-26 11:58       ` Ian Jackson
  1 sibling, 1 reply; 8+ messages in thread
From: Wei Liu @ 2018-02-23 18:28 UTC (permalink / raw)
  To: Marek Marczykowski-Górecki
  Cc: Ian Jackson, xen-devel, Wei Liu, Roger Pau Monné

On Fri, Feb 09, 2018 at 12:35:13PM +0100, Marek Marczykowski-Górecki wrote:
> On Fri, Feb 09, 2018 at 11:03:55AM +0000, Roger Pau Monné wrote:
> > Really adding Ian and Wei.
> > 
> > On Fri, Feb 09, 2018 at 10:55:24AM +0000, Roger Pau Monné wrote:
> > > So the problem is creation time for domains that have quite a lot of
> > > disks attached. Adding Ian and Wei who know more about the async
> > > dispatch system, but I think (at least from a technical PoV) it
> > > should be possible to parallelize device attachment and thus hotplug
> > > script execution. Devices are independent from each other.
> 
> In theory yes, but in practice block script (at least on Linux) takes a
> lock and serialize execution...
> 
> > > Also the Linux hotplug scripts in general seem extremely convoluted,
> > > I'm not sure whether we could gain some speed there just by
> > > simplification.
> 
> Well, we're comparing a bunch of fork+exec(), including starting bash
> (default /bin/sh on most systems), with just a single stat() call...
> Handling scripts in libxl itself also takes some time (in my case libxl
> live in libvirt, which may or may not have an impact). For a domU with
> 4 disks, getting rid of hotplug scripts saved about 2s of startup time.
> 

Sorry for the late reply.

If you really don't want block scripts, can you not specify a script
that only does "exit 0"? That seems to be easier than modifying libxl
and it is also useable in older versions of Xen.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: libxl - avoid calling block script
  2018-02-23 18:28       ` Wei Liu
@ 2018-02-23 20:14         ` Marek Marczykowski-Górecki
  2018-02-26 10:11           ` Roger Pau Monné
  0 siblings, 1 reply; 8+ messages in thread
From: Marek Marczykowski-Górecki @ 2018-02-23 20:14 UTC (permalink / raw)
  To: Wei Liu; +Cc: xen-devel, Ian Jackson, Roger Pau Monné


[-- Attachment #1.1: Type: text/plain, Size: 2242 bytes --]

On Fri, Feb 23, 2018 at 06:28:56PM +0000, Wei Liu wrote:
> On Fri, Feb 09, 2018 at 12:35:13PM +0100, Marek Marczykowski-Górecki wrote:
> > On Fri, Feb 09, 2018 at 11:03:55AM +0000, Roger Pau Monné wrote:
> > > Really adding Ian and Wei.
> > > 
> > > On Fri, Feb 09, 2018 at 10:55:24AM +0000, Roger Pau Monné wrote:
> > > > So the problem is creation time for domains that have quite a lot of
> > > > disks attached. Adding Ian and Wei who know more about the async
> > > > dispatch system, but I think (at least from a technical PoV) it
> > > > should be possible to parallelize device attachment and thus hotplug
> > > > script execution. Devices are independent from each other.
> > 
> > In theory yes, but in practice block script (at least on Linux) takes a
> > lock and serialize execution...
> > 
> > > > Also the Linux hotplug scripts in general seem extremely convoluted,
> > > > I'm not sure whether we could gain some speed there just by
> > > > simplification.
> > 
> > Well, we're comparing a bunch of fork+exec(), including starting bash
> > (default /bin/sh on most systems), with just a single stat() call...
> > Handling scripts in libxl itself also takes some time (in my case libxl
> > live in libvirt, which may or may not have an impact). For a domU with
> > 4 disks, getting rid of hotplug scripts saved about 2s of startup time.
> > 
> 
> Sorry for the late reply.
> 
> If you really don't want block scripts, can you not specify a script
> that only does "exit 0"? That seems to be easier than modifying libxl
> and it is also useable in older versions of Xen.

But this is only one part of the picture. Something needs to set
physical-device xenstore entry. Libxl did that before, but it was
removed (see original message in this thread). I may write alternative
simplified block script for such case and measure performance of it, but
this feels overly complex, especially when libxl already have everything
it needs to quickly fill that xenstore entry.

Anyway, I'll go that way for now.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: libxl - avoid calling block script
  2018-02-23 20:14         ` Marek Marczykowski-Górecki
@ 2018-02-26 10:11           ` Roger Pau Monné
  0 siblings, 0 replies; 8+ messages in thread
From: Roger Pau Monné @ 2018-02-26 10:11 UTC (permalink / raw)
  To: Marek Marczykowski-Górecki; +Cc: Ian Jackson, Wei Liu, xen-devel

On Fri, Feb 23, 2018 at 09:14:03PM +0100, Marek Marczykowski-Górecki wrote:
> On Fri, Feb 23, 2018 at 06:28:56PM +0000, Wei Liu wrote:
> > On Fri, Feb 09, 2018 at 12:35:13PM +0100, Marek Marczykowski-Górecki wrote:
> > > On Fri, Feb 09, 2018 at 11:03:55AM +0000, Roger Pau Monné wrote:
> > > > Really adding Ian and Wei.
> > > > 
> > > > On Fri, Feb 09, 2018 at 10:55:24AM +0000, Roger Pau Monné wrote:
> > > > > So the problem is creation time for domains that have quite a lot of
> > > > > disks attached. Adding Ian and Wei who know more about the async
> > > > > dispatch system, but I think (at least from a technical PoV) it
> > > > > should be possible to parallelize device attachment and thus hotplug
> > > > > script execution. Devices are independent from each other.
> > > 
> > > In theory yes, but in practice block script (at least on Linux) takes a
> > > lock and serialize execution...
> > > 
> > > > > Also the Linux hotplug scripts in general seem extremely convoluted,
> > > > > I'm not sure whether we could gain some speed there just by
> > > > > simplification.
> > > 
> > > Well, we're comparing a bunch of fork+exec(), including starting bash
> > > (default /bin/sh on most systems), with just a single stat() call...
> > > Handling scripts in libxl itself also takes some time (in my case libxl
> > > live in libvirt, which may or may not have an impact). For a domU with
> > > 4 disks, getting rid of hotplug scripts saved about 2s of startup time.
> > > 
> > 
> > Sorry for the late reply.
> > 
> > If you really don't want block scripts, can you not specify a script
> > that only does "exit 0"? That seems to be easier than modifying libxl
> > and it is also useable in older versions of Xen.
> 
> But this is only one part of the picture. Something needs to set
> physical-device xenstore entry. Libxl did that before, but it was
> removed (see original message in this thread). I may write alternative
> simplified block script for such case and measure performance of it, but
> this feels overly complex, especially when libxl already have everything
> it needs to quickly fill that xenstore entry.

Keep in mind hotplug scripts can be written in any language, they
don't need to be shell scripts. They are written in shell ATM because
it's easier to modify, but I think it would be perfectly normal to
instead have binary executables (ie: written in C) for performance
reasons.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: libxl - avoid calling block script
  2018-02-09 11:35     ` Marek Marczykowski-Górecki
  2018-02-23 18:28       ` Wei Liu
@ 2018-02-26 11:58       ` Ian Jackson
  1 sibling, 0 replies; 8+ messages in thread
From: Ian Jackson @ 2018-02-26 11:58 UTC (permalink / raw)
  To: Marek Marczykowski-Górecki; +Cc: xen-devel, Wei Liu, Roger Pau Monné

Marek Marczykowski-Górecki writes ("Re: [Xen-devel] libxl - avoid calling block script"):
> On Fri, Feb 09, 2018 at 11:03:55AM +0000, Roger Pau Monné wrote:
> > Really adding Ian and Wei.
> > 
> > On Fri, Feb 09, 2018 at 10:55:24AM +0000, Roger Pau Monné wrote:
> > > So the problem is creation time for domains that have quite a lot of
> > > disks attached. Adding Ian and Wei who know more about the async
> > > dispatch system, but I think (at least from a technical PoV) it
> > > should be possible to parallelize device attachment and thus hotplug
> > > script execution. Devices are independent from each other.
> 
> In theory yes, but in practice block script (at least on Linux) takes a
> lock and serialize execution...

Indeed.

> > > Also the Linux hotplug scripts in general seem extremely convoluted,
> > > I'm not sure whether we could gain some speed there just by
> > > simplification.
> 
> Well, we're comparing a bunch of fork+exec(), including starting bash
> (default /bin/sh on most systems), with just a single stat() call...
> Handling scripts in libxl itself also takes some time (in my case libxl
> live in libvirt, which may or may not have an impact). For a domU with
> 4 disks, getting rid of hotplug scripts saved about 2s of startup time.

The scripts themselves are terribly terribly slow.  They are as Roger
says incredibly convoluted.  I'm sure your 2s is right, but almost all
of that will be actual script execution.

I am not opposed to moving the functionality for very simplest case
into libxl.

But I think from your pov it would be worth trying a simple shell
script which doesn't take a lock, but just provides the physical
device information.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2018-02-26 11:58 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-02-09  1:02 libxl - avoid calling block script Marek Marczykowski-Górecki
2018-02-09 10:55 ` Roger Pau Monné
2018-02-09 11:03   ` Roger Pau Monné
2018-02-09 11:35     ` Marek Marczykowski-Górecki
2018-02-23 18:28       ` Wei Liu
2018-02-23 20:14         ` Marek Marczykowski-Górecki
2018-02-26 10:11           ` Roger Pau Monné
2018-02-26 11:58       ` Ian Jackson

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.