* [PATCH RFC] docs: document file-posix locking protocol
@ 2021-03-22 18:27 Vladimir Sementsov-Ogievskiy
2021-05-24 14:40 ` Vladimir Sementsov-Ogievskiy
2021-06-22 12:00 ` Daniel P. Berrangé
0 siblings, 2 replies; 4+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-03-22 18:27 UTC (permalink / raw)
To: qemu-block; +Cc: qemu-devel, kwolf, mreitz, eblake, vsementsov, den
Let's document how we use file locks in file-posix driver, to allow
external programs to "communicate" in this way with Qemu.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
Hi all!
We need to access disk images from non-Qemu code and coordinate with
Qemu utilities which may use same image. So, we want to support Qemu
file locking in the external code.
So, here is a patch to document how Qemu locking works, and make this
thing "public".
This is an RFC, because I'm unsure how should we actually document
different operations we have.
For example greaph-mod is a strange thing, I think we should get rid
of it at all.. And at least, no sense in locking corresponding byte in a
raw file.
The other thing is write-unchanged.. What it means when we consider a
raw file opened in several processes? Probably we don't need it too..
docs/system/qemu-block-drivers.rst.inc | 55 ++++++++++++++++++++++++++
1 file changed, 55 insertions(+)
diff --git a/docs/system/qemu-block-drivers.rst.inc b/docs/system/qemu-block-drivers.rst.inc
index b052a6d14e..3cd708b3dc 100644
--- a/docs/system/qemu-block-drivers.rst.inc
+++ b/docs/system/qemu-block-drivers.rst.inc
@@ -952,3 +952,58 @@ on host and see if there are locks held by the QEMU process on the image file.
More than one byte could be locked by the QEMU instance, each byte of which
reflects a particular permission that is acquired or protected by the running
block driver.
+
+Image locking protocol
+~~~~~~~~~~~~~~~~~~~~~~
+
+QEMU holds rd locks and never rw locks. Instead, GETLK fcntl is used with F_WRLCK
+to handle permissions as described below.
+QEMU process may rd-lock the following bytes of the image with corresponding
+meaning:
+
+Permission bytes. If permission byte is rd-locked, it means that some process
+uses corresponding permission on that file.
+
+Byte Operation
+100 read
+ Lock holder can read
+101 write
+ Lock holder can write
+102 write-unchanged
+ Lock holder can write same data
+103 resize
+ Lock holder can resize the file
+104 graph-mod
+ Undefined. QEMU sometimes locks this byte, but external programs
+ should not. QEMU will stop locking this byte in future
+
+Unshare bytes. If permission byte is rd-locked, it means that some process
+does not allow the others use corresponding options on that file.
+
+Byte Operation
+200 read
+ Lock holder don't allow read operation to other processes.
+201 write
+ Lock holder don't allow write operation to other processes.
+202 write-unchanged
+ Lock holder don't allow write-unchanged operation to other processes.
+203 resize
+ Lock holder don't allow resizing the file by other processes.
+204 graph-mod
+ Undefined. QEMU sometimes locks this byte, but external programs
+ should not. QEMU will stop locking this byte in future
+
+Handling the permissions works as follows: assume we want to open the file to do
+some operations and in the same time want to disallow some operation to other
+processes. So, we want to lock some of the bytes described above. We operate as
+follows:
+
+1. rd-lock all needed bytes, both "permission" bytes and "unshare" bytes.
+
+2. For each "unshare" byte we rd-locked, do GETLK that "tries" to wr-lock
+corresponding "permission" byte. So, we check is there any other process that
+uses the permission we want to unshare. If it exists we fail.
+
+3. For each "permission" byte we rd-locked, do GETLK that "tries" to wr-lock
+corresponding "unshare" byte. So, we check is there any other process that
+unshares the permission we want to have. If it exists we fail.
--
2.29.2
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH RFC] docs: document file-posix locking protocol
2021-03-22 18:27 [PATCH RFC] docs: document file-posix locking protocol Vladimir Sementsov-Ogievskiy
@ 2021-05-24 14:40 ` Vladimir Sementsov-Ogievskiy
2021-06-22 11:14 ` Vladimir Sementsov-Ogievskiy
2021-06-22 12:00 ` Daniel P. Berrangé
1 sibling, 1 reply; 4+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-24 14:40 UTC (permalink / raw)
To: qemu-block; +Cc: qemu-devel, kwolf, mreitz, eblake, den, Kirill Tkhai
Ping.
We need to synchronize access to qcow2 images between Qemu and our internal program. For this solution to be reliable, the locking protocol should be documented at least..
22.03.2021 21:27, Vladimir Sementsov-Ogievskiy wrote:
> Let's document how we use file locks in file-posix driver, to allow
> external programs to "communicate" in this way with Qemu.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>
> Hi all!
>
> We need to access disk images from non-Qemu code and coordinate with
> Qemu utilities which may use same image. So, we want to support Qemu
> file locking in the external code.
>
> So, here is a patch to document how Qemu locking works, and make this
> thing "public".
>
> This is an RFC, because I'm unsure how should we actually document
> different operations we have.
>
> For example greaph-mod is a strange thing, I think we should get rid
> of it at all.. And at least, no sense in locking corresponding byte in a
> raw file.
>
> The other thing is write-unchanged.. What it means when we consider a
> raw file opened in several processes? Probably we don't need it too..
>
> docs/system/qemu-block-drivers.rst.inc | 55 ++++++++++++++++++++++++++
> 1 file changed, 55 insertions(+)
>
> diff --git a/docs/system/qemu-block-drivers.rst.inc b/docs/system/qemu-block-drivers.rst.inc
> index b052a6d14e..3cd708b3dc 100644
> --- a/docs/system/qemu-block-drivers.rst.inc
> +++ b/docs/system/qemu-block-drivers.rst.inc
> @@ -952,3 +952,58 @@ on host and see if there are locks held by the QEMU process on the image file.
> More than one byte could be locked by the QEMU instance, each byte of which
> reflects a particular permission that is acquired or protected by the running
> block driver.
> +
> +Image locking protocol
> +~~~~~~~~~~~~~~~~~~~~~~
> +
> +QEMU holds rd locks and never rw locks. Instead, GETLK fcntl is used with F_WRLCK
> +to handle permissions as described below.
> +QEMU process may rd-lock the following bytes of the image with corresponding
> +meaning:
> +
> +Permission bytes. If permission byte is rd-locked, it means that some process
> +uses corresponding permission on that file.
> +
> +Byte Operation
> +100 read
> + Lock holder can read
> +101 write
> + Lock holder can write
> +102 write-unchanged
> + Lock holder can write same data
> +103 resize
> + Lock holder can resize the file
> +104 graph-mod
> + Undefined. QEMU sometimes locks this byte, but external programs
> + should not. QEMU will stop locking this byte in future
> +
> +Unshare bytes. If permission byte is rd-locked, it means that some process
> +does not allow the others use corresponding options on that file.
> +
> +Byte Operation
> +200 read
> + Lock holder don't allow read operation to other processes.
> +201 write
> + Lock holder don't allow write operation to other processes.
> +202 write-unchanged
> + Lock holder don't allow write-unchanged operation to other processes.
> +203 resize
> + Lock holder don't allow resizing the file by other processes.
> +204 graph-mod
> + Undefined. QEMU sometimes locks this byte, but external programs
> + should not. QEMU will stop locking this byte in future
> +
> +Handling the permissions works as follows: assume we want to open the file to do
> +some operations and in the same time want to disallow some operation to other
> +processes. So, we want to lock some of the bytes described above. We operate as
> +follows:
> +
> +1. rd-lock all needed bytes, both "permission" bytes and "unshare" bytes.
> +
> +2. For each "unshare" byte we rd-locked, do GETLK that "tries" to wr-lock
> +corresponding "permission" byte. So, we check is there any other process that
> +uses the permission we want to unshare. If it exists we fail.
> +
> +3. For each "permission" byte we rd-locked, do GETLK that "tries" to wr-lock
> +corresponding "unshare" byte. So, we check is there any other process that
> +unshares the permission we want to have. If it exists we fail.
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH RFC] docs: document file-posix locking protocol
2021-05-24 14:40 ` Vladimir Sementsov-Ogievskiy
@ 2021-06-22 11:14 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 4+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-06-22 11:14 UTC (permalink / raw)
To: qemu-block; +Cc: qemu-devel, kwolf, mreitz, eblake, den, Kirill Tkhai
Ping. Any thoughts?
24.05.2021 17:40, Vladimir Sementsov-Ogievskiy wrote:
> Ping.
>
> We need to synchronize access to qcow2 images between Qemu and our internal program. For this solution to be reliable, the locking protocol should be documented at least..
>
> 22.03.2021 21:27, Vladimir Sementsov-Ogievskiy wrote:
>> Let's document how we use file locks in file-posix driver, to allow
>> external programs to "communicate" in this way with Qemu.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>
>> Hi all!
>>
>> We need to access disk images from non-Qemu code and coordinate with
>> Qemu utilities which may use same image. So, we want to support Qemu
>> file locking in the external code.
>>
>> So, here is a patch to document how Qemu locking works, and make this
>> thing "public".
>>
>> This is an RFC, because I'm unsure how should we actually document
>> different operations we have.
>>
>> For example greaph-mod is a strange thing, I think we should get rid
>> of it at all.. And at least, no sense in locking corresponding byte in a
>> raw file.
>>
>> The other thing is write-unchanged.. What it means when we consider a
>> raw file opened in several processes? Probably we don't need it too..
>>
>> docs/system/qemu-block-drivers.rst.inc | 55 ++++++++++++++++++++++++++
>> 1 file changed, 55 insertions(+)
>>
>> diff --git a/docs/system/qemu-block-drivers.rst.inc b/docs/system/qemu-block-drivers.rst.inc
>> index b052a6d14e..3cd708b3dc 100644
>> --- a/docs/system/qemu-block-drivers.rst.inc
>> +++ b/docs/system/qemu-block-drivers.rst.inc
>> @@ -952,3 +952,58 @@ on host and see if there are locks held by the QEMU process on the image file.
>> More than one byte could be locked by the QEMU instance, each byte of which
>> reflects a particular permission that is acquired or protected by the running
>> block driver.
>> +
>> +Image locking protocol
>> +~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +QEMU holds rd locks and never rw locks. Instead, GETLK fcntl is used with F_WRLCK
>> +to handle permissions as described below.
>> +QEMU process may rd-lock the following bytes of the image with corresponding
>> +meaning:
>> +
>> +Permission bytes. If permission byte is rd-locked, it means that some process
>> +uses corresponding permission on that file.
>> +
>> +Byte Operation
>> +100 read
>> + Lock holder can read
>> +101 write
>> + Lock holder can write
>> +102 write-unchanged
>> + Lock holder can write same data
>> +103 resize
>> + Lock holder can resize the file
>> +104 graph-mod
>> + Undefined. QEMU sometimes locks this byte, but external programs
>> + should not. QEMU will stop locking this byte in future
>> +
>> +Unshare bytes. If permission byte is rd-locked, it means that some process
>> +does not allow the others use corresponding options on that file.
>> +
>> +Byte Operation
>> +200 read
>> + Lock holder don't allow read operation to other processes.
>> +201 write
>> + Lock holder don't allow write operation to other processes.
>> +202 write-unchanged
>> + Lock holder don't allow write-unchanged operation to other processes.
>> +203 resize
>> + Lock holder don't allow resizing the file by other processes.
>> +204 graph-mod
>> + Undefined. QEMU sometimes locks this byte, but external programs
>> + should not. QEMU will stop locking this byte in future
>> +
>> +Handling the permissions works as follows: assume we want to open the file to do
>> +some operations and in the same time want to disallow some operation to other
>> +processes. So, we want to lock some of the bytes described above. We operate as
>> +follows:
>> +
>> +1. rd-lock all needed bytes, both "permission" bytes and "unshare" bytes.
>> +
>> +2. For each "unshare" byte we rd-locked, do GETLK that "tries" to wr-lock
>> +corresponding "permission" byte. So, we check is there any other process that
>> +uses the permission we want to unshare. If it exists we fail.
>> +
>> +3. For each "permission" byte we rd-locked, do GETLK that "tries" to wr-lock
>> +corresponding "unshare" byte. So, we check is there any other process that
>> +unshares the permission we want to have. If it exists we fail.
>>
>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH RFC] docs: document file-posix locking protocol
2021-03-22 18:27 [PATCH RFC] docs: document file-posix locking protocol Vladimir Sementsov-Ogievskiy
2021-05-24 14:40 ` Vladimir Sementsov-Ogievskiy
@ 2021-06-22 12:00 ` Daniel P. Berrangé
1 sibling, 0 replies; 4+ messages in thread
From: Daniel P. Berrangé @ 2021-06-22 12:00 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: kwolf, qemu-block, qemu-devel, mreitz, den, eblake
On Mon, Mar 22, 2021 at 09:27:38PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> Let's document how we use file locks in file-posix driver, to allow
> external programs to "communicate" in this way with Qemu.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>
> Hi all!
>
> We need to access disk images from non-Qemu code and coordinate with
> Qemu utilities which may use same image. So, we want to support Qemu
> file locking in the external code.
>
> So, here is a patch to document how Qemu locking works, and make this
> thing "public".
>
> This is an RFC, because I'm unsure how should we actually document
> different operations we have.
>
> For example greaph-mod is a strange thing, I think we should get rid
> of it at all.. And at least, no sense in locking corresponding byte in a
> raw file.
>
> The other thing is write-unchanged.. What it means when we consider a
> raw file opened in several processes? Probably we don't need it too..
>
> docs/system/qemu-block-drivers.rst.inc | 55 ++++++++++++++++++++++++++
> 1 file changed, 55 insertions(+)
>
> diff --git a/docs/system/qemu-block-drivers.rst.inc b/docs/system/qemu-block-drivers.rst.inc
> index b052a6d14e..3cd708b3dc 100644
> --- a/docs/system/qemu-block-drivers.rst.inc
> +++ b/docs/system/qemu-block-drivers.rst.inc
> @@ -952,3 +952,58 @@ on host and see if there are locks held by the QEMU process on the image file.
> More than one byte could be locked by the QEMU instance, each byte of which
> reflects a particular permission that is acquired or protected by the running
> block driver.
> +
> +Image locking protocol
> +~~~~~~~~~~~~~~~~~~~~~~
> +
> +QEMU holds rd locks and never rw locks. Instead, GETLK fcntl is used with F_WRLCK
> +to handle permissions as described below.
> +QEMU process may rd-lock the following bytes of the image with corresponding
> +meaning:
> +
> +Permission bytes. If permission byte is rd-locked, it means that some process
> +uses corresponding permission on that file.
> +
> +Byte Operation
> +100 read
> + Lock holder can read
> +101 write
> + Lock holder can write
> +102 write-unchanged
> + Lock holder can write same data
> +103 resize
> + Lock holder can resize the file
> +104 graph-mod
> + Undefined. QEMU sometimes locks this byte, but external programs
> + should not. QEMU will stop locking this byte in future
> +
> +Unshare bytes. If permission byte is rd-locked, it means that some process
> +does not allow the others use corresponding options on that file.
> +
> +Byte Operation
> +200 read
> + Lock holder don't allow read operation to other processes.
> +201 write
> + Lock holder don't allow write operation to other processes.
> +202 write-unchanged
> + Lock holder don't allow write-unchanged operation to other processes.
> +203 resize
> + Lock holder don't allow resizing the file by other processes.
> +204 graph-mod
> + Undefined. QEMU sometimes locks this byte, but external programs
> + should not. QEMU will stop locking this byte in future
> +
> +Handling the permissions works as follows: assume we want to open the file to do
> +some operations and in the same time want to disallow some operation to other
> +processes. So, we want to lock some of the bytes described above. We operate as
> +follows:
> +
> +1. rd-lock all needed bytes, both "permission" bytes and "unshare" bytes.
> +
> +2. For each "unshare" byte we rd-locked, do GETLK that "tries" to wr-lock
> +corresponding "permission" byte. So, we check is there any other process that
> +uses the permission we want to unshare. If it exists we fail.
> +
> +3. For each "permission" byte we rd-locked, do GETLK that "tries" to wr-lock
> +corresponding "unshare" byte. So, we check is there any other process that
> +unshares the permission we want to have. If it exists we fail.
I'd perhaps illustrate the few common scenarios - read-only, shared access,
read-write, exclusive and read-write share access, using C-psuedo-code, from
opening image, locking, doing operations, unlocking and closing image.
This would make it explicit that these locks need to be held open
for the duration of the i/o operations.
I'd also probably warn about the dangers of traditional fcntl locks in
threaded programs, compared to the saner OFD locks.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-06-22 12:03 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-22 18:27 [PATCH RFC] docs: document file-posix locking protocol Vladimir Sementsov-Ogievskiy
2021-05-24 14:40 ` Vladimir Sementsov-Ogievskiy
2021-06-22 11:14 ` Vladimir Sementsov-Ogievskiy
2021-06-22 12:00 ` Daniel P. Berrangé
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).