[Xen-devel] Relevance of global lock (/var/lock/xl) in xl tool

[Xen-devel] Relevance of global lock (/var/lock/xl) in xl tool

[Michał Leszczyński]

Hello,

when either "xl restore" or "xl create" command is invoked, a global lock is acquired here:

https://github.com/xen-project/xen/blob/master/tools/xl/xl_vmcontrol.c#L876

I'm trying to figure out what is the exact importance of this lock? Is it really critical for XL operation? I have a pretty powerful machine on which I want to run a few dozen of short-lived VMs. This lock is seriously slowing me down, not letting to restore more than one domain at once. Turns out that everything still works fine (and much faster) when I comment-out the lock in the abovementioned place, but I'm not sure if it's a correct impression.

Does anyone know if there is a situation in which Xen would screw up without this lock?


Best regards,
Michał Leszczyński

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Relevance of global lock (/var/lock/xl) in xl tool

[Anthony PERARD]

On Fri, Jun 07, 2019 at 02:06:30PM +0200, Michał Leszczyński wrote:
> Hello,

Hi,

> when either "xl restore" or "xl create" command is invoked, a global lock is acquired here:
> 
> https://github.com/xen-project/xen/blob/master/tools/xl/xl_vmcontrol.c#L876
> 
> I'm trying to figure out what is the exact importance of this lock? Is it really critical for XL operation? I have a pretty powerful machine on which I want to run a few dozen of short-lived VMs. This lock is seriously slowing me down, not letting to restore more than one domain at once. Turns out that everything still works fine (and much faster) when I comment-out the lock in the abovementioned place, but I'm not sure if it's a correct impression.
> 
> Does anyone know if there is a situation in which Xen would screw up without this lock?

Here is the reason for the lock:
https://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=ea4dce89d478d62341cd2f9d8944e215f7086144

  xl: free memory before building a domain
  Free the needed amount of memory before proceeding with the domain
  build.
  Use a filelock to prevent other xl instances from conflicting during
  this operation.

So there are probably configurations where the lock isn't useful, or
there are better ways to reserve memory for domain creation.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Relevance of global lock (/var/lock/xl) in xl tool

[Michał Leszczyński]

[-- Attachment #1.1: Type: text/plain, Size: 2095 bytes --]

Hello,

I've tested the scenario with restoring too many domains having lock
commented out. It turns out that in case when there is no memory left
for XEN to allocate, all pending /xl restore/ commands will simply fail
with an internal error.

As far as I understand, it's not going to do any damage unless I request
too much memory?

If so, then there is some place for improvement, i.e. to make xl acquire
the lock, allocate memory, release the lock and then start restoring the
domain state. Maybe I could provide some pull request if I would manage
to implement such behavior.


Best regards,
Michał Leszczyński


On 10.06.2019 11:41, Anthony PERARD wrote:
> On Fri, Jun 07, 2019 at 02:06:30PM +0200, Michał Leszczyński wrote:
>> Hello,
> Hi,
>
>> when either "xl restore" or "xl create" command is invoked, a global lock is acquired here:
>>
>> https://github.com/xen-project/xen/blob/master/tools/xl/xl_vmcontrol.c#L876
>>
>> I'm trying to figure out what is the exact importance of this lock? Is it really critical for XL operation? I have a pretty powerful machine on which I want to run a few dozen of short-lived VMs. This lock is seriously slowing me down, not letting to restore more than one domain at once. Turns out that everything still works fine (and much faster) when I comment-out the lock in the abovementioned place, but I'm not sure if it's a correct impression.
>>
>> Does anyone know if there is a situation in which Xen would screw up without this lock?
> Here is the reason for the lock:
> https://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=ea4dce89d478d62341cd2f9d8944e215f7086144
>
>   xl: free memory before building a domain
>   Free the needed amount of memory before proceeding with the domain
>   build.
>   Use a filelock to prevent other xl instances from conflicting during
>   this operation.
>
> So there are probably configurations where the lock isn't useful, or
> there are better ways to reserve memory for domain creation.
>
-- 
Pozdrawiam
Michał Leszczyński
CERT Polska/NASK
+48 532 461 124


[-- Attachment #1.2: Type: text/html, Size: 3199 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Relevance of global lock (/var/lock/xl) in xl tool

[Fanny Dwargee]

[-- Attachment #1.1: Type: text/plain, Size: 2648 bytes --]

I'd like to point out that a patch for getting ride of the lock was
provided by Dmitry Isaykin on March but unfortunately seems that nobody
noticed.

https://lists.xenproject.org/archives/html/xen-devel/2019-03/msg00567.html

El lun., 10 jun. 2019 a las 22:00, Michał Leszczyński (<
michal.leszczynski@cert.pl>) escribió:

> Hello,
>
> I've tested the scenario with restoring too many domains having lock
> commented out. It turns out that in case when there is no memory left for
> XEN to allocate, all pending *xl restore* commands will simply fail with
> an internal error.
>
> As far as I understand, it's not going to do any damage unless I request
> too much memory?
>
> If so, then there is some place for improvement, i.e. to make xl acquire
> the lock, allocate memory, release the lock and then start restoring the
> domain state. Maybe I could provide some pull request if I would manage to
> implement such behavior.
>
>
> Best regards,
> Michał Leszczyński
>
>
> On 10.06.2019 11:41, Anthony PERARD wrote:
>
> On Fri, Jun 07, 2019 at 02:06:30PM +0200, Michał Leszczyński wrote:
>
> Hello,
>
> Hi,
>
>
> when either "xl restore" or "xl create" command is invoked, a global lock is acquired here:
> https://github.com/xen-project/xen/blob/master/tools/xl/xl_vmcontrol.c#L876
>
> I'm trying to figure out what is the exact importance of this lock? Is it really critical for XL operation? I have a pretty powerful machine on which I want to run a few dozen of short-lived VMs. This lock is seriously slowing me down, not letting to restore more than one domain at once. Turns out that everything still works fine (and much faster) when I comment-out the lock in the abovementioned place, but I'm not sure if it's a correct impression.
>
> Does anyone know if there is a situation in which Xen would screw up without this lock?
>
> Here is the reason for the lock:https://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=ea4dce89d478d62341cd2f9d8944e215f7086144
>
>   xl: free memory before building a domain
>   Free the needed amount of memory before proceeding with the domain
>   build.
>   Use a filelock to prevent other xl instances from conflicting during
>   this operation.
>
> So there are probably configurations where the lock isn't useful, or
> there are better ways to reserve memory for domain creation.
>
>
> --
> Pozdrawiam
> Michał Leszczyński
> CERT Polska/NASK
> +48 532 461 124
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel

[-- Attachment #1.2: Type: text/html, Size: 4378 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Relevance of global lock (/var/lock/xl) in xl tool

[Roger Pau Monné]

Please try to avoid top posting, it breaks the flow of the
conversation.

+ Ian and Wei (tools maintainers)

On Mon, Jun 10, 2019 at 09:57:34PM +0200, Michał Leszczyński wrote:
> Hello,
> 
> I've tested the scenario with restoring too many domains having lock
> commented out. It turns out that in case when there is no memory left
> for XEN to allocate, all pending /xl restore/ commands will simply fail
> with an internal error.
> 
> As far as I understand, it's not going to do any damage unless I request
> too much memory?

Iff the lock is solely used to automatically adjust the memory balloon
for dom0 in order to allocate memory to use for other domains then as
long as you don't have autoballoon enabled it should be fine.

Failures during domain creation are always harmless, the only damage
is the domain not being actually created.

> If so, then there is some place for improvement, i.e. to make xl acquire
> the lock, allocate memory, release the lock and then start restoring the
> domain state. Maybe I could provide some pull request if I would manage
> to implement such behavior.

FWIW, there's a 'claim' hypercall which is already implemented and
used by libxl that should reserve memory for a domain beforehand.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Relevance of global lock (/var/lock/xl) in xl tool

[Roger Pau Monné]

Please try to avoid top posting, it breaks the flow of the
conversation.

+ Ian and Wei.

On Tue, Jun 11, 2019 at 08:30:47AM +0200, Fanny Dwargee wrote:
> I'd like to point out that a patch for getting ride of the lock was
> provided by Dmitry Isaykin on March but unfortunately seems that nobody
> noticed.
> 
> https://lists.xenproject.org/archives/html/xen-devel/2019-03/msg00567.html

That seems like a sensible patch, iff indeed the lock is only needed
for autoballooning memory.

Ian, Wei, do you think this is a sensible approach? Are there other
parts of xl/libxl that rely on the lock?

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel