* [Qemu-devel] [PATCH v3 01/19] maint: Allow for EXAMPLES in texi2pod
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
@ 2019-01-12 17:57 ` Eric Blake
2019-01-15 17:51 ` Richard W.M. Jones
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 02/19] qemu-nbd: Enhance man page Eric Blake
` (20 subsequent siblings)
21 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:57 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block
The next commit will add an EXAMPLES section to qemu-nbd.8;
for that to work, we need to recognize EXAMPLES in texi2pod.
We also need to add a dependency from all man pages against
the generator script, since a change to the generator may
cause the resulting man page to differ.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: add generic dependency for all man pages in $(DOCS) instead of
per-line editing [Vladimir]
---
Makefile | 2 ++
scripts/texi2pod.pl | 2 +-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/Makefile b/Makefile
index a9ac16d94e8..e2d3ace190a 100644
--- a/Makefile
+++ b/Makefile
@@ -857,6 +857,8 @@ docs/interop/qemu-qmp-ref.dvi docs/interop/qemu-qmp-ref.html \
docs/interop/qemu-qmp-ref.txt docs/interop/qemu-qmp-ref.7: \
docs/interop/qemu-qmp-ref.texi docs/interop/qemu-qmp-qapi.texi
+$(filter %.1 %.7 %.8,$(DOCS)): scripts/texi2pod.pl
+
# Reports/Analysis
%/coverage-report.html:
diff --git a/scripts/texi2pod.pl b/scripts/texi2pod.pl
index 39ce584a322..839b7917cf7 100755
--- a/scripts/texi2pod.pl
+++ b/scripts/texi2pod.pl
@@ -398,7 +398,7 @@ $sects{NAME} = "$fn \- $tl\n";
$sects{FOOTNOTES} .= "=back\n" if exists $sects{FOOTNOTES};
for $sect (qw(NAME SYNOPSIS DESCRIPTION OPTIONS ENVIRONMENT FILES
- BUGS NOTES FOOTNOTES SEEALSO AUTHOR COPYRIGHT)) {
+ BUGS NOTES FOOTNOTES EXAMPLES SEEALSO AUTHOR COPYRIGHT)) {
if(exists $sects{$sect}) {
$head = $sect;
$head =~ s/SEEALSO/SEE ALSO/;
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 01/19] maint: Allow for EXAMPLES in texi2pod
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 01/19] maint: Allow for EXAMPLES in texi2pod Eric Blake
@ 2019-01-15 17:51 ` Richard W.M. Jones
0 siblings, 0 replies; 71+ messages in thread
From: Richard W.M. Jones @ 2019-01-15 17:51 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On Sat, Jan 12, 2019 at 11:57:54AM -0600, Eric Blake wrote:
> The next commit will add an EXAMPLES section to qemu-nbd.8;
> for that to work, we need to recognize EXAMPLES in texi2pod.
> We also need to add a dependency from all man pages against
> the generator script, since a change to the generator may
> cause the resulting man page to differ.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> v3: add generic dependency for all man pages in $(DOCS) instead of
> per-line editing [Vladimir]
> ---
> Makefile | 2 ++
> scripts/texi2pod.pl | 2 +-
> 2 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/Makefile b/Makefile
> index a9ac16d94e8..e2d3ace190a 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -857,6 +857,8 @@ docs/interop/qemu-qmp-ref.dvi docs/interop/qemu-qmp-ref.html \
> docs/interop/qemu-qmp-ref.txt docs/interop/qemu-qmp-ref.7: \
> docs/interop/qemu-qmp-ref.texi docs/interop/qemu-qmp-qapi.texi
>
> +$(filter %.1 %.7 %.8,$(DOCS)): scripts/texi2pod.pl
Much simpler, and adds the dependencies for all applicable $(DOCS)
targets as was suggested by Vladimir previously, so:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
> # Reports/Analysis
>
> %/coverage-report.html:
> diff --git a/scripts/texi2pod.pl b/scripts/texi2pod.pl
> index 39ce584a322..839b7917cf7 100755
> --- a/scripts/texi2pod.pl
> +++ b/scripts/texi2pod.pl
> @@ -398,7 +398,7 @@ $sects{NAME} = "$fn \- $tl\n";
> $sects{FOOTNOTES} .= "=back\n" if exists $sects{FOOTNOTES};
>
> for $sect (qw(NAME SYNOPSIS DESCRIPTION OPTIONS ENVIRONMENT FILES
> - BUGS NOTES FOOTNOTES SEEALSO AUTHOR COPYRIGHT)) {
> + BUGS NOTES FOOTNOTES EXAMPLES SEEALSO AUTHOR COPYRIGHT)) {
> if(exists $sects{$sect}) {
> $head = $sect;
> $head =~ s/SEEALSO/SEE ALSO/;
> --
> 2.20.1
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
^ permalink raw reply [flat|nested] 71+ messages in thread
* [Qemu-devel] [PATCH v3 02/19] qemu-nbd: Enhance man page
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 01/19] maint: Allow for EXAMPLES in texi2pod Eric Blake
@ 2019-01-12 17:57 ` Eric Blake
2019-01-15 17:53 ` Richard W.M. Jones
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 03/19] qemu-nbd: Sanity check partition bounds Eric Blake
` (19 subsequent siblings)
21 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:57 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block
Document some useful qemu-nbd command lines. Mention some restrictions
on particular options, like -p being only for MBR images, or -c/-d
being Linux-only. Update some text given the recent change to no
longer serve oldstyle protocol (missed in commit 7f7dfe2a). Also,
consistently use trailing '.' in describing options.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: wording improvements, use -t in more examples [Rich]
---
qemu-nbd.texi | 91 ++++++++++++++++++++++++++++++++++++++++-----------
1 file changed, 72 insertions(+), 19 deletions(-)
diff --git a/qemu-nbd.texi b/qemu-nbd.texi
index 96b1546006a..3f22559beb4 100644
--- a/qemu-nbd.texi
+++ b/qemu-nbd.texi
@@ -10,11 +10,17 @@
Export a QEMU disk image using the NBD protocol.
+Other uses:
+@itemize
+@item
+Bind a /dev/nbdX block device to a QEMU server (on Linux).
+@end itemize
+
@c man end
@c man begin OPTIONS
@var{filename} is a disk image filename, or a set of block
-driver options if @var{--image-opts} is specified.
+driver options if @option{--image-opts} is specified.
@var{dev} is an NBD device.
@@ -27,24 +33,25 @@ supported. The common object types that it makes sense to define are the
keys, and the @code{tls-creds} object, which is used to supply TLS
credentials for the qemu-nbd server.
@item -p, --port=@var{port}
-The TCP port to listen on (default @samp{10809})
+The TCP port to listen on (default @samp{10809}).
@item -o, --offset=@var{offset}
-The offset into the image
+The offset into the image.
@item -b, --bind=@var{iface}
-The interface to bind to (default @samp{0.0.0.0})
+The interface to bind to (default @samp{0.0.0.0}).
@item -k, --socket=@var{path}
-Use a unix socket with path @var{path}
+Use a unix socket with path @var{path}.
@item --image-opts
Treat @var{filename} as a set of image options, instead of a plain
filename. If this flag is specified, the @var{-f} flag should
not be used, instead the '@code{format=}' option should be set.
@item -f, --format=@var{fmt}
Force the use of the block driver for format @var{fmt} instead of
-auto-detecting
+auto-detecting.
@item -r, --read-only
-Export the disk as read-only
+Export the disk as read-only.
@item -P, --partition=@var{num}
-Only expose partition @var{num}
+Only expose MBR partition @var{num}. Understands physical partitions
+1-4 and logical partitions 5-8.
@item -B, --bitmap=@var{name}
If @var{filename} has a qcow2 persistent bitmap @var{name}, expose
that bitmap via the ``qemu:dirty-bitmap:@var{name}'' context
@@ -52,7 +59,7 @@ accessible through NBD_OPT_SET_META_CONTEXT.
@item -s, --snapshot
Use @var{filename} as an external snapshot, create a temporary
file with backing_file=@var{filename}, redirect the write to
-the temporary one
+the temporary one.
@item -l, --load-snapshot=@var{snapshot_param}
Load an internal snapshot inside @var{filename} and export it
as an read-only device, @var{snapshot_param} format is
@@ -76,19 +83,20 @@ driver-specific optimized zero write commands. @var{detect-zeroes} is one of
converts a zero write to an unmap operation and can only be used if
@var{discard} is set to @samp{unmap}. The default is @samp{off}.
@item -c, --connect=@var{dev}
-Connect @var{filename} to NBD device @var{dev}
+Connect @var{filename} to NBD device @var{dev} (Linux only).
@item -d, --disconnect
-Disconnect the device @var{dev}
+Disconnect the device @var{dev} (Linux only).
@item -e, --shared=@var{num}
-Allow up to @var{num} clients to share the device (default @samp{1})
+Allow up to @var{num} clients to share the device (default
+@samp{1}). Safe for readers, but for now, consistency is not
+guaranteed between multiple writers.
@item -t, --persistent
-Don't exit on the last connection
+Don't exit on the last connection.
@item -x, --export-name=@var{name}
-Set the NBD volume export name. This switches the server to use
-the new style NBD protocol negotiation
+Set the NBD volume export name (default of a zero-length string).
@item -D, --description=@var{description}
Set the NBD volume export description, as a human-readable
-string. Requires the use of @option{-x}
+string.
@item --tls-creds=ID
Enable mandatory TLS encryption for the server by setting the ID
of the TLS credentials object previously created with the --object
@@ -96,11 +104,11 @@ option.
@item --fork
Fork off the server process and exit the parent once the server is running.
@item -v, --verbose
-Display extra debugging information
+Display extra debugging information.
@item -h, --help
-Display this help and exit
+Display this help and exit.
@item -V, --version
-Display version information and exit
+Display version information and exit.
@item -T, --trace [[enable=]@var{pattern}][,events=@var{file}][,file=@var{file}]
@findex --trace
@include qemu-option-trace.texi
@@ -108,6 +116,51 @@ Display version information and exit
@c man end
+@c man begin EXAMPLES
+Start a server listening on port 10809 that exposes only the
+guest-visible contents of a qcow2 file, with no TLS encryption, and
+with the default export name (an empty string). The command is
+one-shot, and will block until the first successful client
+disconnects:
+
+@example
+qemu-nbd -f qcow2 file.qcow2
+@end example
+
+Start a long-running server listening with encryption on port 10810,
+and require clients to have a correct X.509 certificate to connect to
+a 1 megabyte subset of a raw file, using the export name 'subset':
+
+@example
+qemu-nbd \
+ --object tls-creds-x509,id=tls0,endpoint=server,dir=/path/to/qemutls \
+ --tls-creds tls0 -t -x subset -p 10810 \
+ --image-opts driver=raw,offset=1M,length=1M,file.driver=file,file.filename=file.raw
+@end example
+
+Serve a read-only copy of just the first MBR partition of a guest
+image over a Unix socket with as many as 5 simultaneous readers, with
+a persistent process forked as a daemon:
+
+@example
+qemu-nbd --fork -t -e 5 -s /path/to/sock -p 1 -r -f qcow2 file.qcow2
+@end example
+
+Expose the guest-visible contents of a qcow2 file via a block device
+/dev/nbd0 (and possibly creating /dev/nbd0p1 and friends for
+partitions found within), then disconnect the device when done.
+@emph{CAUTION}: Do not use this method to mount filesystems from an
+untrusted guest image - a malicious guest may have prepared the image
+to attempt to trigger kernel bugs in partition probing or file system
+mounting.
+
+@example
+qemu-nbd -c /dev/nbd0 -f qcow2 file.qcow2
+qemu-nbd -d /dev/nbd0
+@end example
+
+@c man end
+
@ignore
@setfilename qemu-nbd
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 02/19] qemu-nbd: Enhance man page
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 02/19] qemu-nbd: Enhance man page Eric Blake
@ 2019-01-15 17:53 ` Richard W.M. Jones
0 siblings, 0 replies; 71+ messages in thread
From: Richard W.M. Jones @ 2019-01-15 17:53 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On Sat, Jan 12, 2019 at 11:57:55AM -0600, Eric Blake wrote:
> Document some useful qemu-nbd command lines. Mention some restrictions
> on particular options, like -p being only for MBR images, or -c/-d
> being Linux-only. Update some text given the recent change to no
> longer serve oldstyle protocol (missed in commit 7f7dfe2a). Also,
> consistently use trailing '.' in describing options.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v3: wording improvements, use -t in more examples [Rich]
> ---
> qemu-nbd.texi | 91 ++++++++++++++++++++++++++++++++++++++++-----------
> 1 file changed, 72 insertions(+), 19 deletions(-)
>
> diff --git a/qemu-nbd.texi b/qemu-nbd.texi
> index 96b1546006a..3f22559beb4 100644
> --- a/qemu-nbd.texi
> +++ b/qemu-nbd.texi
> @@ -10,11 +10,17 @@
>
> Export a QEMU disk image using the NBD protocol.
>
> +Other uses:
> +@itemize
> +@item
> +Bind a /dev/nbdX block device to a QEMU server (on Linux).
> +@end itemize
> +
> @c man end
>
> @c man begin OPTIONS
> @var{filename} is a disk image filename, or a set of block
> -driver options if @var{--image-opts} is specified.
> +driver options if @option{--image-opts} is specified.
>
> @var{dev} is an NBD device.
>
> @@ -27,24 +33,25 @@ supported. The common object types that it makes sense to define are the
> keys, and the @code{tls-creds} object, which is used to supply TLS
> credentials for the qemu-nbd server.
> @item -p, --port=@var{port}
> -The TCP port to listen on (default @samp{10809})
> +The TCP port to listen on (default @samp{10809}).
> @item -o, --offset=@var{offset}
> -The offset into the image
> +The offset into the image.
> @item -b, --bind=@var{iface}
> -The interface to bind to (default @samp{0.0.0.0})
> +The interface to bind to (default @samp{0.0.0.0}).
> @item -k, --socket=@var{path}
> -Use a unix socket with path @var{path}
> +Use a unix socket with path @var{path}.
> @item --image-opts
> Treat @var{filename} as a set of image options, instead of a plain
> filename. If this flag is specified, the @var{-f} flag should
> not be used, instead the '@code{format=}' option should be set.
> @item -f, --format=@var{fmt}
> Force the use of the block driver for format @var{fmt} instead of
> -auto-detecting
> +auto-detecting.
> @item -r, --read-only
> -Export the disk as read-only
> +Export the disk as read-only.
> @item -P, --partition=@var{num}
> -Only expose partition @var{num}
> +Only expose MBR partition @var{num}. Understands physical partitions
> +1-4 and logical partitions 5-8.
> @item -B, --bitmap=@var{name}
> If @var{filename} has a qcow2 persistent bitmap @var{name}, expose
> that bitmap via the ``qemu:dirty-bitmap:@var{name}'' context
> @@ -52,7 +59,7 @@ accessible through NBD_OPT_SET_META_CONTEXT.
> @item -s, --snapshot
> Use @var{filename} as an external snapshot, create a temporary
> file with backing_file=@var{filename}, redirect the write to
> -the temporary one
> +the temporary one.
> @item -l, --load-snapshot=@var{snapshot_param}
> Load an internal snapshot inside @var{filename} and export it
> as an read-only device, @var{snapshot_param} format is
> @@ -76,19 +83,20 @@ driver-specific optimized zero write commands. @var{detect-zeroes} is one of
> converts a zero write to an unmap operation and can only be used if
> @var{discard} is set to @samp{unmap}. The default is @samp{off}.
> @item -c, --connect=@var{dev}
> -Connect @var{filename} to NBD device @var{dev}
> +Connect @var{filename} to NBD device @var{dev} (Linux only).
> @item -d, --disconnect
> -Disconnect the device @var{dev}
> +Disconnect the device @var{dev} (Linux only).
> @item -e, --shared=@var{num}
> -Allow up to @var{num} clients to share the device (default @samp{1})
> +Allow up to @var{num} clients to share the device (default
> +@samp{1}). Safe for readers, but for now, consistency is not
> +guaranteed between multiple writers.
> @item -t, --persistent
> -Don't exit on the last connection
> +Don't exit on the last connection.
> @item -x, --export-name=@var{name}
> -Set the NBD volume export name. This switches the server to use
> -the new style NBD protocol negotiation
> +Set the NBD volume export name (default of a zero-length string).
> @item -D, --description=@var{description}
> Set the NBD volume export description, as a human-readable
> -string. Requires the use of @option{-x}
> +string.
> @item --tls-creds=ID
> Enable mandatory TLS encryption for the server by setting the ID
> of the TLS credentials object previously created with the --object
> @@ -96,11 +104,11 @@ option.
> @item --fork
> Fork off the server process and exit the parent once the server is running.
> @item -v, --verbose
> -Display extra debugging information
> +Display extra debugging information.
> @item -h, --help
> -Display this help and exit
> +Display this help and exit.
> @item -V, --version
> -Display version information and exit
> +Display version information and exit.
> @item -T, --trace [[enable=]@var{pattern}][,events=@var{file}][,file=@var{file}]
> @findex --trace
> @include qemu-option-trace.texi
> @@ -108,6 +116,51 @@ Display version information and exit
>
> @c man end
>
> +@c man begin EXAMPLES
> +Start a server listening on port 10809 that exposes only the
> +guest-visible contents of a qcow2 file, with no TLS encryption, and
> +with the default export name (an empty string). The command is
> +one-shot, and will block until the first successful client
> +disconnects:
> +
> +@example
> +qemu-nbd -f qcow2 file.qcow2
> +@end example
> +
> +Start a long-running server listening with encryption on port 10810,
> +and require clients to have a correct X.509 certificate to connect to
> +a 1 megabyte subset of a raw file, using the export name 'subset':
> +
> +@example
> +qemu-nbd \
> + --object tls-creds-x509,id=tls0,endpoint=server,dir=/path/to/qemutls \
> + --tls-creds tls0 -t -x subset -p 10810 \
> + --image-opts driver=raw,offset=1M,length=1M,file.driver=file,file.filename=file.raw
> +@end example
> +
> +Serve a read-only copy of just the first MBR partition of a guest
> +image over a Unix socket with as many as 5 simultaneous readers, with
> +a persistent process forked as a daemon:
> +
> +@example
> +qemu-nbd --fork -t -e 5 -s /path/to/sock -p 1 -r -f qcow2 file.qcow2
> +@end example
> +
> +Expose the guest-visible contents of a qcow2 file via a block device
> +/dev/nbd0 (and possibly creating /dev/nbd0p1 and friends for
> +partitions found within), then disconnect the device when done.
> +@emph{CAUTION}: Do not use this method to mount filesystems from an
> +untrusted guest image - a malicious guest may have prepared the image
> +to attempt to trigger kernel bugs in partition probing or file system
> +mounting.
> +
> +@example
> +qemu-nbd -c /dev/nbd0 -f qcow2 file.qcow2
> +qemu-nbd -d /dev/nbd0
> +@end example
> +
> +@c man end
> +
> @ignore
>
> @setfilename qemu-nbd
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
^ permalink raw reply [flat|nested] 71+ messages in thread
* [Qemu-devel] [PATCH v3 03/19] qemu-nbd: Sanity check partition bounds
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 01/19] maint: Allow for EXAMPLES in texi2pod Eric Blake
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 02/19] qemu-nbd: Enhance man page Eric Blake
@ 2019-01-12 17:57 ` Eric Blake
2019-01-15 16:20 ` Vladimir Sementsov-Ogievskiy
` (2 more replies)
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 04/19] nbd/server: Hoist length check to qemp_nbd_server_add Eric Blake
` (18 subsequent siblings)
21 siblings, 3 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:57 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block
When the user requests a partition, we were using data read
from the disk as disk offsets without a bounds check. We got
lucky that even when computed offsets are out-of-bounds,
blk_pread() will gracefully catch the error later (so I don't
think a malicious image can crash or exploit qemu-nbd, and am
not treating this as a security flaw), but it's better to
flag the problem up front than to risk permanent EIO death of
the block device down the road. Also, note that the
partition code blindly overwrites any offset passed in by the
user; so make the -o/-P combo an error for less confusion.
This can be tested with nbdkit:
$ echo hi > file
$ nbdkit -fv --filter=truncate partitioning file truncate=64k
Pre-patch:
$ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 &
$ qemu-io -f raw nbd://localhost:10810
qemu-io> r -v 0 1
Disconnect client, due to: Failed to send reply: reading from file failed: Input/output error
Connection closed
read failed: Input/output error
qemu-io> q
[1]+ Done qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809
Post-patch:
$ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809
qemu-nbd: Discovered partition 1 at offset 1048576 size 512, but size exceeds file length 65536
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: new patch
---
qemu-nbd.c | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/qemu-nbd.c b/qemu-nbd.c
index 51b55f2e066..ff4adb9b3eb 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -1013,12 +1013,28 @@ int main(int argc, char **argv)
fd_size -= dev_offset;
if (partition != -1) {
- ret = find_partition(blk, partition, &dev_offset, &fd_size);
+ off_t limit;
+
+ if (dev_offset) {
+ error_report("Cannot request partition and offset together");
+ exit(EXIT_FAILURE);
+ }
+ ret = find_partition(blk, partition, &dev_offset, &limit);
if (ret < 0) {
error_report("Could not find partition %d: %s", partition,
strerror(-ret));
exit(EXIT_FAILURE);
}
+ /* partition limits are (32-bit << 9); can't overflow 64 bits */
+ assert(dev_offset >= 0 && dev_offset + limit >= dev_offset);
+ if (dev_offset + limit > fd_size) {
+ error_report("Discovered partition %d at offset %lld size %lld, "
+ "but size exceeds file length %lld", partition,
+ (long long int) dev_offset, (long long int) limit,
+ (long long int) fd_size);
+ exit(EXIT_FAILURE);
+ }
+ fd_size = limit;
}
export = nbd_export_new(bs, dev_offset, fd_size, export_name,
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 03/19] qemu-nbd: Sanity check partition bounds
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 03/19] qemu-nbd: Sanity check partition bounds Eric Blake
@ 2019-01-15 16:20 ` Vladimir Sementsov-Ogievskiy
2019-01-15 16:53 ` Eric Blake
2019-01-15 18:00 ` Richard W.M. Jones
2019-01-16 7:46 ` Vladimir Sementsov-Ogievskiy
2 siblings, 1 reply; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-15 16:20 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
12.01.2019 20:57, Eric Blake wrote:
> When the user requests a partition, we were using data read
> from the disk as disk offsets without a bounds check. We got
> lucky that even when computed offsets are out-of-bounds,
> blk_pread() will gracefully catch the error later (so I don't
> think a malicious image can crash or exploit qemu-nbd, and am
> not treating this as a security flaw), but it's better to
> flag the problem up front than to risk permanent EIO death of
> the block device down the road. Also, note that the
> partition code blindly overwrites any offset passed in by the
> user; so make the -o/-P combo an error for less confusion.
>
> This can be tested with nbdkit:
> $ echo hi > file
> $ nbdkit -fv --filter=truncate partitioning file truncate=64k
>
> Pre-patch:
> $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 &
> $ qemu-io -f raw nbd://localhost:10810
> qemu-io> r -v 0 1
> Disconnect client, due to: Failed to send reply: reading from file failed: Input/output error
> Connection closed
> read failed: Input/output error
> qemu-io> q
> [1]+ Done qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809
>
> Post-patch:
> $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809
> qemu-nbd: Discovered partition 1 at offset 1048576 size 512, but size exceeds file length 65536
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> v3: new patch
> ---
> qemu-nbd.c | 18 +++++++++++++++++-
> 1 file changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index 51b55f2e066..ff4adb9b3eb 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -1013,12 +1013,28 @@ int main(int argc, char **argv)
> fd_size -= dev_offset;
>
> if (partition != -1) {
> - ret = find_partition(blk, partition, &dev_offset, &fd_size);
> + off_t limit;
> +
> + if (dev_offset) {
> + error_report("Cannot request partition and offset together");
hmm, but you still allow to request partition and set -o 0, I think it's better
to forbid it too.
> + exit(EXIT_FAILURE);
> + }
> + ret = find_partition(blk, partition, &dev_offset, &limit);
> if (ret < 0) {
> error_report("Could not find partition %d: %s", partition,
> strerror(-ret));
> exit(EXIT_FAILURE);
> }
> + /* partition limits are (32-bit << 9); can't overflow 64 bits */
> + assert(dev_offset >= 0 && dev_offset + limit >= dev_offset);
hmm, so these values are read from file and may be whatsoever. Why to assert instead
of error_report and exit() ?
> + if (dev_offset + limit > fd_size) {
> + error_report("Discovered partition %d at offset %lld size %lld, "
> + "but size exceeds file length %lld", partition,
> + (long long int) dev_offset, (long long int) limit,
> + (long long int) fd_size);
> + exit(EXIT_FAILURE);
> + }
> + fd_size = limit;
> }
>
> export = nbd_export_new(bs, dev_offset, fd_size, export_name,
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 03/19] qemu-nbd: Sanity check partition bounds
2019-01-15 16:20 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-15 16:53 ` Eric Blake
0 siblings, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-15 16:53 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block
[-- Attachment #1: Type: text/plain, Size: 4142 bytes --]
On 1/15/19 10:20 AM, Vladimir Sementsov-Ogievskiy wrote:
> 12.01.2019 20:57, Eric Blake wrote:
>> When the user requests a partition, we were using data read
>> from the disk as disk offsets without a bounds check. We got
>> lucky that even when computed offsets are out-of-bounds,
>> blk_pread() will gracefully catch the error later (so I don't
>> think a malicious image can crash or exploit qemu-nbd, and am
>> not treating this as a security flaw), but it's better to
>> flag the problem up front than to risk permanent EIO death of
>> the block device down the road. Also, note that the
>> partition code blindly overwrites any offset passed in by the
>> user; so make the -o/-P combo an error for less confusion.
>>
>> This can be tested with nbdkit:
>> $ echo hi > file
>> $ nbdkit -fv --filter=truncate partitioning file truncate=64k
>>
>> Pre-patch:
>> $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 &
>> $ qemu-io -f raw nbd://localhost:10810
>> qemu-io> r -v 0 1
>> Disconnect client, due to: Failed to send reply: reading from file failed: Input/output error
>> Connection closed
>> read failed: Input/output error
>> qemu-io> q
>> [1]+ Done qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809
>>
>> Post-patch:
>> $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809
>> qemu-nbd: Discovered partition 1 at offset 1048576 size 512, but size exceeds file length 65536
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>> v3: new patch
>> ---
>> qemu-nbd.c | 18 +++++++++++++++++-
>> 1 file changed, 17 insertions(+), 1 deletion(-)
>>
>> diff --git a/qemu-nbd.c b/qemu-nbd.c
>> index 51b55f2e066..ff4adb9b3eb 100644
>> --- a/qemu-nbd.c
>> +++ b/qemu-nbd.c
>> @@ -1013,12 +1013,28 @@ int main(int argc, char **argv)
>> fd_size -= dev_offset;
>>
>> if (partition != -1) {
>> - ret = find_partition(blk, partition, &dev_offset, &fd_size);
>> + off_t limit;
>> +
>> + if (dev_offset) {
>> + error_report("Cannot request partition and offset together");
>
> hmm, but you still allow to request partition and set -o 0, I think it's better
> to forbid it too.
Not worth the bother. Someday, we may want to permit -o and -P together,
where we first calculate the bounds of the partition from -P, and then
use -o to further restrict an even smaller subset of the image exposed
to the user. Under that interpretation, the added error makes sense as
a short-term stop-gap that prevents us from doing what we want (since we
did not actually treat a non-zero offset as an offset within the
partition), but an offset of 0 is the same as omitting the offset. So I
didn't see the point to complicate the code just to check whether -o had
been explicitly supplied with its already-default value.
>
>> + exit(EXIT_FAILURE);
>> + }
>> + ret = find_partition(blk, partition, &dev_offset, &limit);
>> if (ret < 0) {
>> error_report("Could not find partition %d: %s", partition,
>> strerror(-ret));
>> exit(EXIT_FAILURE);
>> }
>> + /* partition limits are (32-bit << 9); can't overflow 64 bits */
>> + assert(dev_offset >= 0 && dev_offset + limit >= dev_offset);
>
> hmm, so these values are read from file and may be whatsoever. Why to assert instead
> of error_report and exit() ?
The assertion is letting the compiler know that we can't get a negative
value. (off_t)(uint32_t << 9)*2 is always a positive value for 64-bit
off_t, but the compiler doesn't necessarily know that ((off_t)a +
(off_t)b >= a) is going to be true if it doesn't track the range
limitations imposed by find_partition(). You are right that we are
doing an assertion based on data obtained by reading from potentially
malicious data, but the assertion is safe because it will NEVER fail,
but is rather augmenting the compiler's data-type analysis.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [Qemu-devel] [PATCH v3 03/19] qemu-nbd: Sanity check partition bounds
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 03/19] qemu-nbd: Sanity check partition bounds Eric Blake
2019-01-15 16:20 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-15 18:00 ` Richard W.M. Jones
2019-01-15 18:08 ` Eric Blake
2019-01-16 7:46 ` Vladimir Sementsov-Ogievskiy
2 siblings, 1 reply; 71+ messages in thread
From: Richard W.M. Jones @ 2019-01-15 18:00 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On Sat, Jan 12, 2019 at 11:57:56AM -0600, Eric Blake wrote:
> When the user requests a partition, we were using data read
> from the disk as disk offsets without a bounds check. We got
> lucky that even when computed offsets are out-of-bounds,
> blk_pread() will gracefully catch the error later (so I don't
> think a malicious image can crash or exploit qemu-nbd, and am
> not treating this as a security flaw), but it's better to
> flag the problem up front than to risk permanent EIO death of
> the block device down the road. Also, note that the
> partition code blindly overwrites any offset passed in by the
> user; so make the -o/-P combo an error for less confusion.
>
> This can be tested with nbdkit:
> $ echo hi > file
> $ nbdkit -fv --filter=truncate partitioning file truncate=64k
>
> Pre-patch:
> $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 &
> $ qemu-io -f raw nbd://localhost:10810
> qemu-io> r -v 0 1
> Disconnect client, due to: Failed to send reply: reading from file failed: Input/output error
> Connection closed
> read failed: Input/output error
> qemu-io> q
> [1]+ Done qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809
>
> Post-patch:
> $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809
> qemu-nbd: Discovered partition 1 at offset 1048576 size 512, but size exceeds file length 65536
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> v3: new patch
> ---
> qemu-nbd.c | 18 +++++++++++++++++-
> 1 file changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index 51b55f2e066..ff4adb9b3eb 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -1013,12 +1013,28 @@ int main(int argc, char **argv)
> fd_size -= dev_offset;
>
> if (partition != -1) {
> - ret = find_partition(blk, partition, &dev_offset, &fd_size);
> + off_t limit;
I was only vaguely following the other review comments, but off_t does
seem odd here. Even though we can assume that _FILE_OFFSET_BITS=64
maybe we should just use {u,}int64_t? Does this represent an offset
in a host file? Only in the case where qemu-nbd is serving a raw
format file. In other cases (say, qcow2) the partition size exists in
an abstract virtual space.
> + if (dev_offset) {
> + error_report("Cannot request partition and offset together");
> + exit(EXIT_FAILURE);
> + }
> + ret = find_partition(blk, partition, &dev_offset, &limit);
> if (ret < 0) {
> error_report("Could not find partition %d: %s", partition,
> strerror(-ret));
> exit(EXIT_FAILURE);
> }
> + /* partition limits are (32-bit << 9); can't overflow 64 bits */
> + assert(dev_offset >= 0 && dev_offset + limit >= dev_offset);
> + if (dev_offset + limit > fd_size) {
> + error_report("Discovered partition %d at offset %lld size %lld, "
> + "but size exceeds file length %lld", partition,
> + (long long int) dev_offset, (long long int) limit,
> + (long long int) fd_size);
> + exit(EXIT_FAILURE);
> + }
> + fd_size = limit;
> }
>
> export = nbd_export_new(bs, dev_offset, fd_size, export_name,
But it's not a big deal, so:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 03/19] qemu-nbd: Sanity check partition bounds
2019-01-15 18:00 ` Richard W.M. Jones
@ 2019-01-15 18:08 ` Eric Blake
0 siblings, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-15 18:08 UTC (permalink / raw)
To: Richard W.M. Jones; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
[-- Attachment #1: Type: text/plain, Size: 3193 bytes --]
On 1/15/19 12:00 PM, Richard W.M. Jones wrote:
> On Sat, Jan 12, 2019 at 11:57:56AM -0600, Eric Blake wrote:
>> When the user requests a partition, we were using data read
>> from the disk as disk offsets without a bounds check. We got
>> lucky that even when computed offsets are out-of-bounds,
>> blk_pread() will gracefully catch the error later (so I don't
>> think a malicious image can crash or exploit qemu-nbd, and am
>> not treating this as a security flaw), but it's better to
>> flag the problem up front than to risk permanent EIO death of
>> the block device down the road. Also, note that the
>> partition code blindly overwrites any offset passed in by the
>> user; so make the -o/-P combo an error for less confusion.
>>
>> This can be tested with nbdkit:
>> $ echo hi > file
>> $ nbdkit -fv --filter=truncate partitioning file truncate=64k
>>
>> Pre-patch:
>> $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 &
>> $ qemu-io -f raw nbd://localhost:10810
>> qemu-io> r -v 0 1
>> Disconnect client, due to: Failed to send reply: reading from file failed: Input/output error
>> Connection closed
>> read failed: Input/output error
>> qemu-io> q
>> [1]+ Done qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809
>>
>> Post-patch:
>> $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809
>> qemu-nbd: Discovered partition 1 at offset 1048576 size 512, but size exceeds file length 65536
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>> v3: new patch
>> ---
>> qemu-nbd.c | 18 +++++++++++++++++-
>> 1 file changed, 17 insertions(+), 1 deletion(-)
>>
>> diff --git a/qemu-nbd.c b/qemu-nbd.c
>> index 51b55f2e066..ff4adb9b3eb 100644
>> --- a/qemu-nbd.c
>> +++ b/qemu-nbd.c
>> @@ -1013,12 +1013,28 @@ int main(int argc, char **argv)
>> fd_size -= dev_offset;
>>
>> if (partition != -1) {
>> - ret = find_partition(blk, partition, &dev_offset, &fd_size);
>> + off_t limit;
>
> I was only vaguely following the other review comments, but off_t does
> seem odd here. Even though we can assume that _FILE_OFFSET_BITS=64
> maybe we should just use {u,}int64_t? Does this represent an offset
> in a host file? Only in the case where qemu-nbd is serving a raw
> format file. In other cases (say, qcow2) the partition size exists in
> an abstract virtual space.
Yes, later patches switch it to int64_t. Here, it remains off_t because
find_partition(&limit) still expects off_t. I suppose in my later
patches, I could use uint64_t limit in spite of keeping int64_t fd_size,
and change the signature of find_partition() accordingly, since I've
decoupled the MBR partition lookup (which is a 41-bit value, always
positive) from the file size checks.
>
> But it's not a big deal, so:
Yeah, no need to rewrite this patch, since later patches improve the
type anyway (whether or not I stick to int64_t or uint64_t for the
find_partition() code).
>
> Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
>
> Rich.
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [Qemu-devel] [PATCH v3 03/19] qemu-nbd: Sanity check partition bounds
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 03/19] qemu-nbd: Sanity check partition bounds Eric Blake
2019-01-15 16:20 ` Vladimir Sementsov-Ogievskiy
2019-01-15 18:00 ` Richard W.M. Jones
@ 2019-01-16 7:46 ` Vladimir Sementsov-Ogievskiy
2 siblings, 0 replies; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-16 7:46 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
12.01.2019 20:57, Eric Blake wrote:
> When the user requests a partition, we were using data read
> from the disk as disk offsets without a bounds check. We got
> lucky that even when computed offsets are out-of-bounds,
> blk_pread() will gracefully catch the error later (so I don't
> think a malicious image can crash or exploit qemu-nbd, and am
> not treating this as a security flaw), but it's better to
> flag the problem up front than to risk permanent EIO death of
> the block device down the road. Also, note that the
> partition code blindly overwrites any offset passed in by the
> user; so make the -o/-P combo an error for less confusion.
>
> This can be tested with nbdkit:
> $ echo hi > file
> $ nbdkit -fv --filter=truncate partitioning file truncate=64k
>
> Pre-patch:
> $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 &
> $ qemu-io -f raw nbd://localhost:10810
> qemu-io> r -v 0 1
> Disconnect client, due to: Failed to send reply: reading from file failed: Input/output error
> Connection closed
> read failed: Input/output error
> qemu-io> q
> [1]+ Done qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809
>
> Post-patch:
> $ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809
> qemu-nbd: Discovered partition 1 at offset 1048576 size 512, but size exceeds file length 65536
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> v3: new patch
> ---
> qemu-nbd.c | 18 +++++++++++++++++-
> 1 file changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index 51b55f2e066..ff4adb9b3eb 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -1013,12 +1013,28 @@ int main(int argc, char **argv)
> fd_size -= dev_offset;
This reuse of file-size variable as export-size is a source of errors, like your
assert in the following part.
>
> if (partition != -1) {
> - ret = find_partition(blk, partition, &dev_offset, &fd_size);
> + off_t limit;
> +
> + if (dev_offset) {
> + error_report("Cannot request partition and offset together");
> + exit(EXIT_FAILURE);
> + }
> + ret = find_partition(blk, partition, &dev_offset, &limit);
> if (ret < 0) {
> error_report("Could not find partition %d: %s", partition,
> strerror(-ret));
> exit(EXIT_FAILURE);
> }
> + /* partition limits are (32-bit << 9); can't overflow 64 bits */
> + assert(dev_offset >= 0 && dev_offset + limit >= dev_offset);
> + if (dev_offset + limit > fd_size) {
> + error_report("Discovered partition %d at offset %lld size %lld, "
> + "but size exceeds file length %lld", partition,
> + (long long int) dev_offset, (long long int) limit,
> + (long long int) fd_size);
> + exit(EXIT_FAILURE);
> + }
> + fd_size = limit;
> }
>
> export = nbd_export_new(bs, dev_offset, fd_size, export_name,
>
Ok, anyway, finally I understand the point, thank you for detailed explanation:
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread
* [Qemu-devel] [PATCH v3 04/19] nbd/server: Hoist length check to qemp_nbd_server_add
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (2 preceding siblings ...)
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 03/19] qemu-nbd: Sanity check partition bounds Eric Blake
@ 2019-01-12 17:57 ` Eric Blake
2019-01-15 9:44 ` Vladimir Sementsov-Ogievskiy
2019-01-16 18:05 ` Eric Blake
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 05/19] nbd/server: Favor [u]int64_t over off_t Eric Blake
` (17 subsequent siblings)
21 siblings, 2 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:57 UTC (permalink / raw)
To: qemu-devel
Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block, Kevin Wolf, Max Reitz
We only had two callers to nbd_export_new; qemu-nbd.c always
passed a valid offset/length pair (because it already checked
the file length, to ensure that offset was in bounds), while
blockdev-nbd always passed 0/-1. Then nbd_export_new reduces
the size to a multiple of BDRV_SECTOR_SIZE (can only happen
when offset is not sector-aligned, since bdrv_getlength()
currently rounds up), which can result in offset being greater
than the enforced length, but that's not fatal (the server
rejects client requests that exceed the advertised length).
However, I'm finding it easier to work with the code if we are
consistent on having both callers pass in a valid length, and
just assert that things are sane in nbd_export_new.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: new patch
---
blockdev-nbd.c | 10 +++++++++-
nbd/server.c | 9 ++-------
2 files changed, 11 insertions(+), 8 deletions(-)
diff --git a/blockdev-nbd.c b/blockdev-nbd.c
index c76d5416b90..d73ac1b026a 100644
--- a/blockdev-nbd.c
+++ b/blockdev-nbd.c
@@ -146,6 +146,7 @@ void qmp_nbd_server_add(const char *device, bool has_name, const char *name,
BlockDriverState *bs = NULL;
BlockBackend *on_eject_blk;
NBDExport *exp;
+ int64_t len;
if (!nbd_server) {
error_setg(errp, "NBD server not running");
@@ -168,6 +169,13 @@ void qmp_nbd_server_add(const char *device, bool has_name, const char *name,
return;
}
+ len = bdrv_getlength(bs);
+ if (len < 0) {
+ error_setg_errno(errp, -len,
+ "Failed to determine the NBD export's length");
+ return;
+ }
+
if (!has_writable) {
writable = false;
}
@@ -175,7 +183,7 @@ void qmp_nbd_server_add(const char *device, bool has_name, const char *name,
writable = false;
}
- exp = nbd_export_new(bs, 0, -1, name, NULL, bitmap,
+ exp = nbd_export_new(bs, 0, len, name, NULL, bitmap,
writable ? 0 : NBD_FLAG_READ_ONLY,
NULL, false, on_eject_blk, errp);
if (!exp) {
diff --git a/nbd/server.c b/nbd/server.c
index e8c56607eff..c9937ccdc2a 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -1499,13 +1499,8 @@ NBDExport *nbd_export_new(BlockDriverState *bs, off_t dev_offset, off_t size,
exp->name = g_strdup(name);
exp->description = g_strdup(description);
exp->nbdflags = nbdflags;
- exp->size = size < 0 ? blk_getlength(blk) : size;
- if (exp->size < 0) {
- error_setg_errno(errp, -exp->size,
- "Failed to determine the NBD export's length");
- goto fail;
- }
- exp->size -= exp->size % BDRV_SECTOR_SIZE;
+ assert(dev_offset <= size);
+ exp->size = QEMU_ALIGN_DOWN(size, BDRV_SECTOR_SIZE);
if (bitmap) {
BdrvDirtyBitmap *bm = NULL;
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 04/19] nbd/server: Hoist length check to qemp_nbd_server_add
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 04/19] nbd/server: Hoist length check to qemp_nbd_server_add Eric Blake
@ 2019-01-15 9:44 ` Vladimir Sementsov-Ogievskiy
2019-01-15 15:25 ` Eric Blake
2019-01-16 18:05 ` Eric Blake
1 sibling, 1 reply; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-15 9:44 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block, Kevin Wolf, Max Reitz
12.01.2019 20:57, Eric Blake wrote:
> We only had two callers to nbd_export_new; qemu-nbd.c always
> passed a valid offset/length pair (because it already checked
> the file length, to ensure that offset was in bounds), while
> blockdev-nbd always passed 0/-1. Then nbd_export_new reduces
> the size to a multiple of BDRV_SECTOR_SIZE (can only happen
> when offset is not sector-aligned, since bdrv_getlength()
> currently rounds up), which can result in offset being greater
> than the enforced length, but that's not fatal (the server
> rejects client requests that exceed the advertised length).
>
> However, I'm finding it easier to work with the code if we are
> consistent on having both callers pass in a valid length, and
> just assert that things are sane in nbd_export_new.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v3: new patch
> ---
> blockdev-nbd.c | 10 +++++++++-
> nbd/server.c | 9 ++-------
> 2 files changed, 11 insertions(+), 8 deletions(-)
>
> diff --git a/blockdev-nbd.c b/blockdev-nbd.c
> index c76d5416b90..d73ac1b026a 100644
> --- a/blockdev-nbd.c
> +++ b/blockdev-nbd.c
> @@ -146,6 +146,7 @@ void qmp_nbd_server_add(const char *device, bool has_name, const char *name,
> BlockDriverState *bs = NULL;
> BlockBackend *on_eject_blk;
> NBDExport *exp;
> + int64_t len;
>
> if (!nbd_server) {
> error_setg(errp, "NBD server not running");
> @@ -168,6 +169,13 @@ void qmp_nbd_server_add(const char *device, bool has_name, const char *name,
> return;
> }
>
> + len = bdrv_getlength(bs);
> + if (len < 0) {
> + error_setg_errno(errp, -len,
> + "Failed to determine the NBD export's length");
> + return;
> + }
> +
> if (!has_writable) {
> writable = false;
> }
> @@ -175,7 +183,7 @@ void qmp_nbd_server_add(const char *device, bool has_name, const char *name,
> writable = false;
> }
>
> - exp = nbd_export_new(bs, 0, -1, name, NULL, bitmap,
> + exp = nbd_export_new(bs, 0, len, name, NULL, bitmap,
> writable ? 0 : NBD_FLAG_READ_ONLY,
> NULL, false, on_eject_blk, errp);
> if (!exp) {
> diff --git a/nbd/server.c b/nbd/server.c
> index e8c56607eff..c9937ccdc2a 100644
> --- a/nbd/server.c
> +++ b/nbd/server.c
> @@ -1499,13 +1499,8 @@ NBDExport *nbd_export_new(BlockDriverState *bs, off_t dev_offset, off_t size,
> exp->name = g_strdup(name);
> exp->description = g_strdup(description);
> exp->nbdflags = nbdflags;
> - exp->size = size < 0 ? blk_getlength(blk) : size;
> - if (exp->size < 0) {
> - error_setg_errno(errp, -exp->size,
> - "Failed to determine the NBD export's length");
> - goto fail;
> - }
> - exp->size -= exp->size % BDRV_SECTOR_SIZE;
> + assert(dev_offset <= size);
@size is not size of the image, but size of the export, so it may be less than dev_offset
(qemu-nbd.c do "fd_size -= dev_offset" before "nbd_export_new(bs, dev_offset, fd_size, "
> + exp->size = QEMU_ALIGN_DOWN(size, BDRV_SECTOR_SIZE);
>
> if (bitmap) {
> BdrvDirtyBitmap *bm = NULL;
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 04/19] nbd/server: Hoist length check to qemp_nbd_server_add
2019-01-15 9:44 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-15 15:25 ` Eric Blake
2019-01-15 16:26 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-15 15:25 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block, Kevin Wolf, Max Reitz
[-- Attachment #1: Type: text/plain, Size: 2127 bytes --]
On 1/15/19 3:44 AM, Vladimir Sementsov-Ogievskiy wrote:
> 12.01.2019 20:57, Eric Blake wrote:
>> We only had two callers to nbd_export_new; qemu-nbd.c always
>> passed a valid offset/length pair (because it already checked
>> the file length, to ensure that offset was in bounds), while
>> blockdev-nbd always passed 0/-1. Then nbd_export_new reduces
>> the size to a multiple of BDRV_SECTOR_SIZE (can only happen
>> when offset is not sector-aligned, since bdrv_getlength()
>> currently rounds up), which can result in offset being greater
>> than the enforced length, but that's not fatal (the server
>> rejects client requests that exceed the advertised length).
>>
>> However, I'm finding it easier to work with the code if we are
>> consistent on having both callers pass in a valid length, and
>> just assert that things are sane in nbd_export_new.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> +++ b/nbd/server.c
>> @@ -1499,13 +1499,8 @@ NBDExport *nbd_export_new(BlockDriverState *bs, off_t dev_offset, off_t size,
>> exp->name = g_strdup(name);
>> exp->description = g_strdup(description);
>> exp->nbdflags = nbdflags;
>> - exp->size = size < 0 ? blk_getlength(blk) : size;
>> - if (exp->size < 0) {
>> - error_setg_errno(errp, -exp->size,
>> - "Failed to determine the NBD export's length");
>> - goto fail;
>> - }
>> - exp->size -= exp->size % BDRV_SECTOR_SIZE;
>> + assert(dev_offset <= size);
>
> @size is not size of the image, but size of the export, so it may be less than dev_offset
> (qemu-nbd.c do "fd_size -= dev_offset" before "nbd_export_new(bs, dev_offset, fd_size, "
But the assert is fine because patch 3/19 fixed qemu-nbd.c to never pass
in dev_offset larger than size (it fails up front if dev_offset is out
of bounds, whether from the -o command line option or from what it read
from the partition header with the -P command line option).
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 04/19] nbd/server: Hoist length check to qemp_nbd_server_add
2019-01-15 15:25 ` Eric Blake
@ 2019-01-15 16:26 ` Vladimir Sementsov-Ogievskiy
2019-01-15 16:58 ` Eric Blake
0 siblings, 1 reply; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-15 16:26 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block, Kevin Wolf, Max Reitz
15.01.2019 18:25, Eric Blake wrote:
> On 1/15/19 3:44 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 12.01.2019 20:57, Eric Blake wrote:
>>> We only had two callers to nbd_export_new; qemu-nbd.c always
>>> passed a valid offset/length pair (because it already checked
>>> the file length, to ensure that offset was in bounds), while
>>> blockdev-nbd always passed 0/-1. Then nbd_export_new reduces
>>> the size to a multiple of BDRV_SECTOR_SIZE (can only happen
>>> when offset is not sector-aligned, since bdrv_getlength()
>>> currently rounds up), which can result in offset being greater
>>> than the enforced length, but that's not fatal (the server
>>> rejects client requests that exceed the advertised length).
>>>
>>> However, I'm finding it easier to work with the code if we are
>>> consistent on having both callers pass in a valid length, and
>>> just assert that things are sane in nbd_export_new.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>>
>
>>> +++ b/nbd/server.c
>>> @@ -1499,13 +1499,8 @@ NBDExport *nbd_export_new(BlockDriverState *bs, off_t dev_offset, off_t size,
>>> exp->name = g_strdup(name);
>>> exp->description = g_strdup(description);
>>> exp->nbdflags = nbdflags;
>>> - exp->size = size < 0 ? blk_getlength(blk) : size;
>>> - if (exp->size < 0) {
>>> - error_setg_errno(errp, -exp->size,
>>> - "Failed to determine the NBD export's length");
>>> - goto fail;
>>> - }
>>> - exp->size -= exp->size % BDRV_SECTOR_SIZE;
>>> + assert(dev_offset <= size);
>>
>> @size is not size of the image, but size of the export, so it may be less than dev_offset
>> (qemu-nbd.c do "fd_size -= dev_offset" before "nbd_export_new(bs, dev_offset, fd_size, "
>
> But the assert is fine because patch 3/19 fixed qemu-nbd.c to never pass
> in dev_offset larger than size (it fails up front if dev_offset is out
> of bounds, whether from the -o command line option or from what it read
> from the partition header with the -P command line option).
>
Don't follow =(
Assume, image size 3M, and we have offset 2M, i.e. -o 2M.
than in qemu-nbd.c, we have
fd_size = blk_getlength(blk); # 3M
...
fd_size -= dev_offset; # 1M
...
export = nbd_export_new(bs, dev_offset, fd_size # bs, 2M, 1M
in nbd_export_new:
assert(dev_offset <= size); # 2M <= 1M
fail.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 04/19] nbd/server: Hoist length check to qemp_nbd_server_add
2019-01-15 16:26 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-15 16:58 ` Eric Blake
2019-01-16 18:03 ` Eric Blake
0 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-15 16:58 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block, Kevin Wolf, Max Reitz
[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]
On 1/15/19 10:26 AM, Vladimir Sementsov-Ogievskiy wrote:
>>> @size is not size of the image, but size of the export, so it may be less than dev_offset
>>> (qemu-nbd.c do "fd_size -= dev_offset" before "nbd_export_new(bs, dev_offset, fd_size, "
>>
>> But the assert is fine because patch 3/19 fixed qemu-nbd.c to never pass
>> in dev_offset larger than size (it fails up front if dev_offset is out
>> of bounds, whether from the -o command line option or from what it read
>> from the partition header with the -P command line option).
>>
>
> Don't follow =(
>
> Assume, image size 3M, and we have offset 2M, i.e. -o 2M.
>
> than in qemu-nbd.c, we have
>
> fd_size = blk_getlength(blk); # 3M
> ...
> fd_size -= dev_offset; # 1M
> ...
> export = nbd_export_new(bs, dev_offset, fd_size # bs, 2M, 1M
>
> in nbd_export_new:
>
> assert(dev_offset <= size); # 2M <= 1M
>
> fail.
Ouch, you are right. I don't need the assertion in server.c at all;
because all callers pass in a validated size, but the validated size has
no comparable relation to dev_offset.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [Qemu-devel] [PATCH v3 04/19] nbd/server: Hoist length check to qemp_nbd_server_add
2019-01-15 16:58 ` Eric Blake
@ 2019-01-16 18:03 ` Eric Blake
0 siblings, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-16 18:03 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: Kevin Wolf, qemu-block, rjones, Max Reitz, nsoffer, jsnow
[-- Attachment #1: Type: text/plain, Size: 2113 bytes --]
On 1/15/19 10:58 AM, Eric Blake wrote:
> On 1/15/19 10:26 AM, Vladimir Sementsov-Ogievskiy wrote:
>
>>>> @size is not size of the image, but size of the export, so it may be less than dev_offset
>>>> (qemu-nbd.c do "fd_size -= dev_offset" before "nbd_export_new(bs, dev_offset, fd_size, "
>>>
>>> But the assert is fine because patch 3/19 fixed qemu-nbd.c to never pass
>>> in dev_offset larger than size (it fails up front if dev_offset is out
>>> of bounds, whether from the -o command line option or from what it read
>>> from the partition header with the -P command line option).
>>>
>>
>> Don't follow =(
>>
>> Assume, image size 3M, and we have offset 2M, i.e. -o 2M.
>>
>> than in qemu-nbd.c, we have
>>
>> fd_size = blk_getlength(blk); # 3M
>> ...
>> fd_size -= dev_offset; # 1M
>> ...
>> export = nbd_export_new(bs, dev_offset, fd_size # bs, 2M, 1M
>>
>> in nbd_export_new:
>>
>> assert(dev_offset <= size); # 2M <= 1M
>>
>> fail.
>
> Ouch, you are right. I don't need the assertion in server.c at all;
> because all callers pass in a validated size, but the validated size has
> no comparable relation to dev_offset.
Here's what I'm considering using instead, merely asserting that the
inputs are non-negative and do not overflow 63 bits:
diff --git i/nbd/server.c w/nbd/server.c
index c9937ccdc2a..3ebcbddaa2c 100644
--- i/nbd/server.c
+++ w/nbd/server.c
@@ -1495,11 +1495,12 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
off_t dev_offset, off_t size,
exp->refcount = 1;
QTAILQ_INIT(&exp->clients);
exp->blk = blk;
+ assert(dev_offset >= 0 && dev_offset <= INT64_MAX);
exp->dev_offset = dev_offset;
exp->name = g_strdup(name);
exp->description = g_strdup(description);
exp->nbdflags = nbdflags;
- assert(dev_offset <= size);
+ assert(size >= 0 && size <= INT64_MAX - dev_offset);
exp->size = QEMU_ALIGN_DOWN(size, BDRV_SECTOR_SIZE);
if (bitmap) {
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply related [flat|nested] 71+ messages in thread
* Re: [Qemu-devel] [PATCH v3 04/19] nbd/server: Hoist length check to qemp_nbd_server_add
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 04/19] nbd/server: Hoist length check to qemp_nbd_server_add Eric Blake
2019-01-15 9:44 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-16 18:05 ` Eric Blake
1 sibling, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-16 18:05 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, qemu-block, rjones, Max Reitz, nsoffer, jsnow
[-- Attachment #1: Type: text/plain, Size: 974 bytes --]
On 1/12/19 11:57 AM, Eric Blake wrote:
s/qemp/qmp/ in the subject line
> We only had two callers to nbd_export_new; qemu-nbd.c always
> passed a valid offset/length pair (because it already checked
> the file length, to ensure that offset was in bounds), while
> blockdev-nbd always passed 0/-1. Then nbd_export_new reduces
> the size to a multiple of BDRV_SECTOR_SIZE (can only happen
> when offset is not sector-aligned, since bdrv_getlength()
> currently rounds up), which can result in offset being greater
> than the enforced length, but that's not fatal (the server
> rejects client requests that exceed the advertised length).
>
> However, I'm finding it easier to work with the code if we are
> consistent on having both callers pass in a valid length, and
> just assert that things are sane in nbd_export_new.
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* [Qemu-devel] [PATCH v3 05/19] nbd/server: Favor [u]int64_t over off_t
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (3 preceding siblings ...)
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 04/19] nbd/server: Hoist length check to qemp_nbd_server_add Eric Blake
@ 2019-01-12 17:57 ` Eric Blake
2019-01-15 10:19 ` Vladimir Sementsov-Ogievskiy
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 06/19] qemu-nbd: Avoid strtol open-coding Eric Blake
` (16 subsequent siblings)
21 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:57 UTC (permalink / raw)
To: qemu-devel
Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block, Kevin Wolf, Max Reitz
Although our compile-time environment is set up so that we always
support long files with 64-bit off_t, we have no guarantee whether
off_t is the same type as int64_t. This requires casts when
printing values, and prevents us from directly using qemu_strtoi64().
Let's just flip to [u]int64_t (signed for length, because we have to
detect failure of blk_getlength() and because off_t was signed;
unsigned for offset because it lets us simplify some math without
having to worry about signed overflow).
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: new patch
---
include/block/nbd.h | 4 ++--
nbd/server.c | 14 +++++++-------
qemu-nbd.c | 26 ++++++++++----------------
3 files changed, 19 insertions(+), 25 deletions(-)
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 1971b557896..0f252829376 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -294,8 +294,8 @@ int nbd_errno_to_system_errno(int err);
typedef struct NBDExport NBDExport;
typedef struct NBDClient NBDClient;
-NBDExport *nbd_export_new(BlockDriverState *bs, off_t dev_offset, off_t size,
- const char *name, const char *description,
+NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
+ int64_t size, const char *name, const char *desc,
const char *bitmap, uint16_t nbdflags,
void (*close)(NBDExport *), bool writethrough,
BlockBackend *on_eject_blk, Error **errp);
diff --git a/nbd/server.c b/nbd/server.c
index c9937ccdc2a..15357d40fd7 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -77,8 +77,8 @@ struct NBDExport {
BlockBackend *blk;
char *name;
char *description;
- off_t dev_offset;
- off_t size;
+ uint64_t dev_offset;
+ int64_t size;
uint16_t nbdflags;
QTAILQ_HEAD(, NBDClient) clients;
QTAILQ_ENTRY(NBDExport) next;
@@ -1455,8 +1455,8 @@ static void nbd_eject_notifier(Notifier *n, void *data)
nbd_export_close(exp);
}
-NBDExport *nbd_export_new(BlockDriverState *bs, off_t dev_offset, off_t size,
- const char *name, const char *description,
+NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
+ int64_t size, const char *name, const char *desc,
const char *bitmap, uint16_t nbdflags,
void (*close)(NBDExport *), bool writethrough,
BlockBackend *on_eject_blk, Error **errp)
@@ -1497,7 +1497,7 @@ NBDExport *nbd_export_new(BlockDriverState *bs, off_t dev_offset, off_t size,
exp->blk = blk;
exp->dev_offset = dev_offset;
exp->name = g_strdup(name);
- exp->description = g_strdup(description);
+ exp->description = g_strdup(desc);
exp->nbdflags = nbdflags;
assert(dev_offset <= size);
exp->size = QEMU_ALIGN_DOWN(size, BDRV_SECTOR_SIZE);
@@ -2131,8 +2131,8 @@ static int nbd_co_receive_request(NBDRequestData *req, NBDRequest *request,
if (request->from > client->exp->size ||
request->from + request->len > client->exp->size) {
error_setg(errp, "operation past EOF; From: %" PRIu64 ", Len: %" PRIu32
- ", Size: %" PRIu64, request->from, request->len,
- (uint64_t)client->exp->size);
+ ", Size: %" PRId64, request->from, request->len,
+ client->exp->size);
return (request->type == NBD_CMD_WRITE ||
request->type == NBD_CMD_WRITE_ZEROES) ? -ENOSPC : -EINVAL;
}
diff --git a/qemu-nbd.c b/qemu-nbd.c
index ff4adb9b3eb..96c0829970c 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -176,7 +176,7 @@ static void read_partition(uint8_t *p, struct partition_record *r)
}
static int find_partition(BlockBackend *blk, int partition,
- off_t *offset, off_t *size)
+ uint64_t *offset, int64_t *size)
{
struct partition_record mbr[4];
uint8_t data[MBR_SIZE];
@@ -500,14 +500,14 @@ int main(int argc, char **argv)
{
BlockBackend *blk;
BlockDriverState *bs;
- off_t dev_offset = 0;
+ uint64_t dev_offset = 0;
uint16_t nbdflags = 0;
bool disconnect = false;
const char *bindto = NULL;
const char *port = NULL;
char *sockpath = NULL;
char *device = NULL;
- off_t fd_size;
+ int64_t fd_size;
QemuOpts *sn_opts = NULL;
const char *sn_id_or_name = NULL;
const char *sopt = "hVb:o:p:rsnP:c:dvk:e:f:tl:x:T:D:B:";
@@ -665,10 +665,6 @@ int main(int argc, char **argv)
error_report("Invalid offset `%s'", optarg);
exit(EXIT_FAILURE);
}
- if (dev_offset < 0) {
- error_report("Offset must be positive `%s'", optarg);
- exit(EXIT_FAILURE);
- }
break;
case 'l':
if (strstart(optarg, SNAPSHOT_OPT_BASE, NULL)) {
@@ -1005,15 +1001,14 @@ int main(int argc, char **argv)
}
if (dev_offset >= fd_size) {
- error_report("Offset (%lld) has to be smaller than the image size "
- "(%lld)",
- (long long int)dev_offset, (long long int)fd_size);
+ error_report("Offset (%" PRIu64 ") has to be smaller than the image "
+ "size (%" PRIu64 ")", dev_offset, fd_size);
exit(EXIT_FAILURE);
}
fd_size -= dev_offset;
if (partition != -1) {
- off_t limit;
+ int64_t limit;
if (dev_offset) {
error_report("Cannot request partition and offset together");
@@ -1026,12 +1021,11 @@ int main(int argc, char **argv)
exit(EXIT_FAILURE);
}
/* partition limits are (32-bit << 9); can't overflow 64 bits */
- assert(dev_offset >= 0 && dev_offset + limit >= dev_offset);
+ assert(dev_offset + limit >= dev_offset);
if (dev_offset + limit > fd_size) {
- error_report("Discovered partition %d at offset %lld size %lld, "
- "but size exceeds file length %lld", partition,
- (long long int) dev_offset, (long long int) limit,
- (long long int) fd_size);
+ error_report("Discovered partition %d at offset %" PRIu64
+ " size %" PRId64 ", but size exceeds file length %"
+ PRId64, partition, dev_offset, limit, fd_size);
exit(EXIT_FAILURE);
}
fd_size = limit;
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 05/19] nbd/server: Favor [u]int64_t over off_t
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 05/19] nbd/server: Favor [u]int64_t over off_t Eric Blake
@ 2019-01-15 10:19 ` Vladimir Sementsov-Ogievskiy
2019-01-15 15:33 ` Eric Blake
0 siblings, 1 reply; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-15 10:19 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block, Kevin Wolf, Max Reitz
12.01.2019 20:57, Eric Blake wrote:
> Although our compile-time environment is set up so that we always
> support long files with 64-bit off_t, we have no guarantee whether
> off_t is the same type as int64_t. This requires casts when
> printing values, and prevents us from directly using qemu_strtoi64().
> Let's just flip to [u]int64_t (signed for length, because we have to
> detect failure of blk_getlength()
we have not, after previous patch
and because off_t was signed;
> unsigned for offset because it lets us simplify some math without
> having to worry about signed overflow).
>
> Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v3: new patch
> ---
> include/block/nbd.h | 4 ++--
> nbd/server.c | 14 +++++++-------
> qemu-nbd.c | 26 ++++++++++----------------
> 3 files changed, 19 insertions(+), 25 deletions(-)
>
> diff --git a/include/block/nbd.h b/include/block/nbd.h
> index 1971b557896..0f252829376 100644
> --- a/include/block/nbd.h
> +++ b/include/block/nbd.h
> @@ -294,8 +294,8 @@ int nbd_errno_to_system_errno(int err);
> typedef struct NBDExport NBDExport;
> typedef struct NBDClient NBDClient;
>
> -NBDExport *nbd_export_new(BlockDriverState *bs, off_t dev_offset, off_t size,
> - const char *name, const char *description,
> +NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
> + int64_t size, const char *name, const char *desc,
in previous patch you drop use of negative @size parameter, so it looks better
to use unsigned for size like for offset
> const char *bitmap, uint16_t nbdflags,
> void (*close)(NBDExport *), bool writethrough,
> BlockBackend *on_eject_blk, Error **errp);
> diff --git a/nbd/server.c b/nbd/server.c
> index c9937ccdc2a..15357d40fd7 100644
> --- a/nbd/server.c
> +++ b/nbd/server.c
> @@ -77,8 +77,8 @@ struct NBDExport {
> BlockBackend *blk;
> char *name;
> char *description;
> - off_t dev_offset;
> - off_t size;
> + uint64_t dev_offset;
> + int64_t size;
> uint16_t nbdflags;
> QTAILQ_HEAD(, NBDClient) clients;
> QTAILQ_ENTRY(NBDExport) next;
> @@ -1455,8 +1455,8 @@ static void nbd_eject_notifier(Notifier *n, void *data)
> nbd_export_close(exp);
> }
>
> -NBDExport *nbd_export_new(BlockDriverState *bs, off_t dev_offset, off_t size,
> - const char *name, const char *description,
> +NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
> + int64_t size, const char *name, const char *desc,
> const char *bitmap, uint16_t nbdflags,
> void (*close)(NBDExport *), bool writethrough,
> BlockBackend *on_eject_blk, Error **errp)
> @@ -1497,7 +1497,7 @@ NBDExport *nbd_export_new(BlockDriverState *bs, off_t dev_offset, off_t size,
> exp->blk = blk;
> exp->dev_offset = dev_offset;
> exp->name = g_strdup(name);
> - exp->description = g_strdup(description);
> + exp->description = g_strdup(desc);
unrelated but at least obvious, OK. However tiny note in commit message won't hurt.
> exp->nbdflags = nbdflags;
> assert(dev_offset <= size);
> exp->size = QEMU_ALIGN_DOWN(size, BDRV_SECTOR_SIZE);
> @@ -2131,8 +2131,8 @@ static int nbd_co_receive_request(NBDRequestData *req, NBDRequest *request,
> if (request->from > client->exp->size ||
> request->from + request->len > client->exp->size) {
> error_setg(errp, "operation past EOF; From: %" PRIu64 ", Len: %" PRIu32
> - ", Size: %" PRIu64, request->from, request->len,
> - (uint64_t)client->exp->size);
> + ", Size: %" PRId64, request->from, request->len,
> + client->exp->size);
> return (request->type == NBD_CMD_WRITE ||
> request->type == NBD_CMD_WRITE_ZEROES) ? -ENOSPC : -EINVAL;
> }
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index ff4adb9b3eb..96c0829970c 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -176,7 +176,7 @@ static void read_partition(uint8_t *p, struct partition_record *r)
> }
>
> static int find_partition(BlockBackend *blk, int partition,
> - off_t *offset, off_t *size)
> + uint64_t *offset, int64_t *size)
function never return negative @size, so what is the reason to keep it signed?
Also, type conversion (uint64_t) should be dropped from the function code I think.
> {
> struct partition_record mbr[4];
> uint8_t data[MBR_SIZE];
> @@ -500,14 +500,14 @@ int main(int argc, char **argv)
> {
> BlockBackend *blk;
> BlockDriverState *bs;
> - off_t dev_offset = 0;
> + uint64_t dev_offset = 0;
> uint16_t nbdflags = 0;
> bool disconnect = false;
> const char *bindto = NULL;
> const char *port = NULL;
> char *sockpath = NULL;
> char *device = NULL;
> - off_t fd_size;
> + int64_t fd_size;
and here signed type is reasonable
> QemuOpts *sn_opts = NULL;
> const char *sn_id_or_name = NULL;
> const char *sopt = "hVb:o:p:rsnP:c:dvk:e:f:tl:x:T:D:B:";
> @@ -665,10 +665,6 @@ int main(int argc, char **argv)
> error_report("Invalid offset `%s'", optarg);
> exit(EXIT_FAILURE);
> }
> - if (dev_offset < 0) {
> - error_report("Offset must be positive `%s'", optarg);
> - exit(EXIT_FAILURE);
> - }
hm, then, may be, s/strtoll/strtoull before this?
> break;
> case 'l':
> if (strstart(optarg, SNAPSHOT_OPT_BASE, NULL)) {
> @@ -1005,15 +1001,14 @@ int main(int argc, char **argv)
> }
>
> if (dev_offset >= fd_size) {
> - error_report("Offset (%lld) has to be smaller than the image size "
> - "(%lld)",
> - (long long int)dev_offset, (long long int)fd_size);
> + error_report("Offset (%" PRIu64 ") has to be smaller than the image "
> + "size (%" PRIu64 ")", dev_offset, fd_size);
PRId64 for fd_size
> exit(EXIT_FAILURE);
> }
> fd_size -= dev_offset;
>
> if (partition != -1) {
> - off_t limit;
> + int64_t limit;
>
> if (dev_offset) {
> error_report("Cannot request partition and offset together");
> @@ -1026,12 +1021,11 @@ int main(int argc, char **argv)
> exit(EXIT_FAILURE);
> }
> /* partition limits are (32-bit << 9); can't overflow 64 bits */
> - assert(dev_offset >= 0 && dev_offset + limit >= dev_offset);
> + assert(dev_offset + limit >= dev_offset);
> if (dev_offset + limit > fd_size) {
> - error_report("Discovered partition %d at offset %lld size %lld, "
> - "but size exceeds file length %lld", partition,
> - (long long int) dev_offset, (long long int) limit,
> - (long long int) fd_size);
> + error_report("Discovered partition %d at offset %" PRIu64
> + " size %" PRId64 ", but size exceeds file length %"
> + PRId64, partition, dev_offset, limit, fd_size);
> exit(EXIT_FAILURE);
hmm, it may be better to place this patch before [03], to squash this chunk into [03]
> }
> fd_size = limit;
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 05/19] nbd/server: Favor [u]int64_t over off_t
2019-01-15 10:19 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-15 15:33 ` Eric Blake
2019-01-15 15:41 ` Vladimir Sementsov-Ogievskiy
2019-01-16 8:23 ` Vladimir Sementsov-Ogievskiy
0 siblings, 2 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-15 15:33 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block, Kevin Wolf, Max Reitz
[-- Attachment #1: Type: text/plain, Size: 6414 bytes --]
On 1/15/19 4:19 AM, Vladimir Sementsov-Ogievskiy wrote:
> 12.01.2019 20:57, Eric Blake wrote:
>> Although our compile-time environment is set up so that we always
>> support long files with 64-bit off_t, we have no guarantee whether
>> off_t is the same type as int64_t. This requires casts when
>> printing values, and prevents us from directly using qemu_strtoi64().
>> Let's just flip to [u]int64_t (signed for length, because we have to
>> detect failure of blk_getlength()
>
> we have not, after previous patch
nbd/server.c no longer has to check for blk_getlength() failures, but
blockdev-nbd.c and qemu-nbd.c still do. Since the callers have to use
an int64_t type for the length as part of their error checking, it's
easier to accept an int64_t length to nbd_export_new(), even if
nbd_export_new() could also use an unsigned type.
>
> and because off_t was signed;
>> unsigned for offset because it lets us simplify some math without
>> having to worry about signed overflow).
>>
>> Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> ---
>> v3: new patch
>> ---
>> include/block/nbd.h | 4 ++--
>> nbd/server.c | 14 +++++++-------
>> qemu-nbd.c | 26 ++++++++++----------------
>> 3 files changed, 19 insertions(+), 25 deletions(-)
>>
>> diff --git a/include/block/nbd.h b/include/block/nbd.h
>> index 1971b557896..0f252829376 100644
>> --- a/include/block/nbd.h
>> +++ b/include/block/nbd.h
>> @@ -294,8 +294,8 @@ int nbd_errno_to_system_errno(int err);
>> typedef struct NBDExport NBDExport;
>> typedef struct NBDClient NBDClient;
>>
>> -NBDExport *nbd_export_new(BlockDriverState *bs, off_t dev_offset, off_t size,
>> - const char *name, const char *description,
>> +NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
>> + int64_t size, const char *name, const char *desc,
>
> in previous patch you drop use of negative @size parameter, so it looks better
> to use unsigned for size like for offset
You can't have a size larger than 2^63; but an unsigned type holds
nearly 2^64. I prefer a signed size, for the same reason that off_t is
signed, in that checking for a negative size is easier to rule out sizes
that are too large.
>>
>> static int find_partition(BlockBackend *blk, int partition,
>> - off_t *offset, off_t *size)
>> + uint64_t *offset, int64_t *size)
>
> function never return negative @size, so what is the reason to keep it signed?
Because the C compiler does NOT like:
int64_t len;
find_partition(..., &len);
with a uint64_t* parameter type - you HAVE to match the signed-ness of
your caller's parameter with your pointer type. Since the caller already
has to use a signed type (to check for blk_getlength() failure AND
because sizes really cannot exceed 2^63), it's easier to keep it signed
here.
>
> Also, type conversion (uint64_t) should be dropped from the function code I think.
Are you talking about this part:
if ((ext_partnum + j + 1) == partition) {
*offset = (uint64_t)ext[j].start_sector_abs << 9;
*size = (uint64_t)ext[j].nb_sectors_abs << 9;
return 0;
}
}
ext_partnum += 4;
} else if ((i + 1) == partition) {
*offset = (uint64_t)mbr[i].start_sector_abs << 9;
*size = (uint64_t)mbr[i].nb_sectors_abs << 9;
return 0;
No - that has to keep the cast, because .start_sector_abs and
.nb_sectors_abs are uint32_t values, but we want to shift into 64-bit
results. You need the cast to force the correct arithmetic rather than
truncating into a 32-bit value that then gets widened into 64-bit storage.
>> @@ -665,10 +665,6 @@ int main(int argc, char **argv)
>> error_report("Invalid offset `%s'", optarg);
>> exit(EXIT_FAILURE);
>> }
>> - if (dev_offset < 0) {
>> - error_report("Offset must be positive `%s'", optarg);
>> - exit(EXIT_FAILURE);
>> - }
>
> hm, then, may be, s/strtoll/strtoull before this?
I clean that up in patch 6/19.
>
>> break;
>> case 'l':
>> if (strstart(optarg, SNAPSHOT_OPT_BASE, NULL)) {
>> @@ -1005,15 +1001,14 @@ int main(int argc, char **argv)
>> }
>>
>> if (dev_offset >= fd_size) {
>> - error_report("Offset (%lld) has to be smaller than the image size "
>> - "(%lld)",
>> - (long long int)dev_offset, (long long int)fd_size);
>> + error_report("Offset (%" PRIu64 ") has to be smaller than the image "
>> + "size (%" PRIu64 ")", dev_offset, fd_size);
>
> PRId64 for fd_size
Sure.
>> @@ -1026,12 +1021,11 @@ int main(int argc, char **argv)
>> exit(EXIT_FAILURE);
>> }
>> /* partition limits are (32-bit << 9); can't overflow 64 bits */
>> - assert(dev_offset >= 0 && dev_offset + limit >= dev_offset);
>> + assert(dev_offset + limit >= dev_offset);
>> if (dev_offset + limit > fd_size) {
>> - error_report("Discovered partition %d at offset %lld size %lld, "
>> - "but size exceeds file length %lld", partition,
>> - (long long int) dev_offset, (long long int) limit,
>> - (long long int) fd_size);
>> + error_report("Discovered partition %d at offset %" PRIu64
>> + " size %" PRId64 ", but size exceeds file length %"
>> + PRId64, partition, dev_offset, limit, fd_size);
>> exit(EXIT_FAILURE);
>
> hmm, it may be better to place this patch before [03], to squash this chunk into [03]
I didn't mind the churn; also, I prefer patch 3 first, because it's more
likely to get backported as a bug fix than the rest of the series (and
the earlier you stick backport candidates in a series, the easier it is
to backport).
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 484 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 05/19] nbd/server: Favor [u]int64_t over off_t
2019-01-15 15:33 ` Eric Blake
@ 2019-01-15 15:41 ` Vladimir Sementsov-Ogievskiy
2019-01-16 8:23 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-15 15:41 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block, Kevin Wolf, Max Reitz
15.01.2019 18:33, Eric Blake wrote:
> On 1/15/19 4:19 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 12.01.2019 20:57, Eric Blake wrote:
>>> Although our compile-time environment is set up so that we always
>>> support long files with 64-bit off_t, we have no guarantee whether
>>> off_t is the same type as int64_t. This requires casts when
>>> printing values, and prevents us from directly using qemu_strtoi64().
>>> Let's just flip to [u]int64_t (signed for length, because we have to
>>> detect failure of blk_getlength()
>>
>> we have not, after previous patch
>
> nbd/server.c no longer has to check for blk_getlength() failures, but
> blockdev-nbd.c and qemu-nbd.c still do. Since the callers have to use
> an int64_t type for the length as part of their error checking, it's
> easier to accept an int64_t length to nbd_export_new(), even if
> nbd_export_new() could also use an unsigned type.
>
>>
>> and because off_t was signed;
>>> unsigned for offset because it lets us simplify some math without
>>> having to worry about signed overflow).
>>>
>>> Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>>
>>> ---
>>> v3: new patch
>>> ---
>>> include/block/nbd.h | 4 ++--
>>> nbd/server.c | 14 +++++++-------
>>> qemu-nbd.c | 26 ++++++++++----------------
>>> 3 files changed, 19 insertions(+), 25 deletions(-)
>>>
>>> diff --git a/include/block/nbd.h b/include/block/nbd.h
>>> index 1971b557896..0f252829376 100644
>>> --- a/include/block/nbd.h
>>> +++ b/include/block/nbd.h
>>> @@ -294,8 +294,8 @@ int nbd_errno_to_system_errno(int err);
>>> typedef struct NBDExport NBDExport;
>>> typedef struct NBDClient NBDClient;
>>>
>>> -NBDExport *nbd_export_new(BlockDriverState *bs, off_t dev_offset, off_t size,
>>> - const char *name, const char *description,
>>> +NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
>>> + int64_t size, const char *name, const char *desc,
>>
>> in previous patch you drop use of negative @size parameter, so it looks better
>> to use unsigned for size like for offset
>
> You can't have a size larger than 2^63; but an unsigned type holds
> nearly 2^64. I prefer a signed size, for the same reason that off_t is
> signed, in that checking for a negative size is easier to rule out sizes
> that are too large.
>
>
>>>
>>> static int find_partition(BlockBackend *blk, int partition,
>>> - off_t *offset, off_t *size)
>>> + uint64_t *offset, int64_t *size)
>>
>> function never return negative @size, so what is the reason to keep it signed?
>
> Because the C compiler does NOT like:
>
> int64_t len;
> find_partition(..., &len);
>
> with a uint64_t* parameter type - you HAVE to match the signed-ness of
> your caller's parameter with your pointer type. Since the caller already
> has to use a signed type (to check for blk_getlength() failure AND
> because sizes really cannot exceed 2^63), it's easier to keep it signed
> here.
>
>>
>> Also, type conversion (uint64_t) should be dropped from the function code I think.
>
> Are you talking about this part:
>
> if ((ext_partnum + j + 1) == partition) {
> *offset = (uint64_t)ext[j].start_sector_abs << 9;
> *size = (uint64_t)ext[j].nb_sectors_abs << 9;
> return 0;
> }
> }
> ext_partnum += 4;
> } else if ((i + 1) == partition) {
> *offset = (uint64_t)mbr[i].start_sector_abs << 9;
> *size = (uint64_t)mbr[i].nb_sectors_abs << 9;
> return 0;
>
> No - that has to keep the cast, because .start_sector_abs and
> .nb_sectors_abs are uint32_t values, but we want to shift into 64-bit
> results. You need the cast to force the correct arithmetic rather than
> truncating into a 32-bit value that then gets widened into 64-bit storage.
Oops, I'm stupid)
I thought about something like (uint64_t)<variable that was off_t, but now it is uint64_t>,
but pointed to <variable that was off_t, but now it is uint64_t> = (uint64_t)<something other>
>
>>> @@ -665,10 +665,6 @@ int main(int argc, char **argv)
>>> error_report("Invalid offset `%s'", optarg);
>>> exit(EXIT_FAILURE);
>>> }
>>> - if (dev_offset < 0) {
>>> - error_report("Offset must be positive `%s'", optarg);
>>> - exit(EXIT_FAILURE);
>>> - }
>>
>> hm, then, may be, s/strtoll/strtoull before this?
>
> I clean that up in patch 6/19.
>
>>
>>> break;
>>> case 'l':
>>> if (strstart(optarg, SNAPSHOT_OPT_BASE, NULL)) {
>>> @@ -1005,15 +1001,14 @@ int main(int argc, char **argv)
>>> }
>>>
>>> if (dev_offset >= fd_size) {
>>> - error_report("Offset (%lld) has to be smaller than the image size "
>>> - "(%lld)",
>>> - (long long int)dev_offset, (long long int)fd_size);
>>> + error_report("Offset (%" PRIu64 ") has to be smaller than the image "
>>> + "size (%" PRIu64 ")", dev_offset, fd_size);
>>
>> PRId64 for fd_size
>
> Sure.
>
>
>>> @@ -1026,12 +1021,11 @@ int main(int argc, char **argv)
>>> exit(EXIT_FAILURE);
>>> }
>>> /* partition limits are (32-bit << 9); can't overflow 64 bits */
>>> - assert(dev_offset >= 0 && dev_offset + limit >= dev_offset);
>>> + assert(dev_offset + limit >= dev_offset);
>>> if (dev_offset + limit > fd_size) {
>>> - error_report("Discovered partition %d at offset %lld size %lld, "
>>> - "but size exceeds file length %lld", partition,
>>> - (long long int) dev_offset, (long long int) limit,
>>> - (long long int) fd_size);
>>> + error_report("Discovered partition %d at offset %" PRIu64
>>> + " size %" PRId64 ", but size exceeds file length %"
>>> + PRId64, partition, dev_offset, limit, fd_size);
>>> exit(EXIT_FAILURE);
>>
>> hmm, it may be better to place this patch before [03], to squash this chunk into [03]
>
> I didn't mind the churn; also, I prefer patch 3 first, because it's more
> likely to get backported as a bug fix than the rest of the series (and
> the earlier you stick backport candidates in a series, the easier it is
> to backport).
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 05/19] nbd/server: Favor [u]int64_t over off_t
2019-01-15 15:33 ` Eric Blake
2019-01-15 15:41 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-16 8:23 ` Vladimir Sementsov-Ogievskiy
2019-01-16 14:23 ` Eric Blake
1 sibling, 1 reply; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-16 8:23 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block, Kevin Wolf, Max Reitz
15.01.2019 18:33, Eric Blake wrote:
> On 1/15/19 4:19 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 12.01.2019 20:57, Eric Blake wrote:
>>> Although our compile-time environment is set up so that we always
>>> support long files with 64-bit off_t, we have no guarantee whether
>>> off_t is the same type as int64_t. This requires casts when
>>> printing values, and prevents us from directly using qemu_strtoi64().
>>> Let's just flip to [u]int64_t (signed for length, because we have to
>>> detect failure of blk_getlength()
>>
>> we have not, after previous patch
>
> nbd/server.c no longer has to check for blk_getlength() failures, but
> blockdev-nbd.c and qemu-nbd.c still do. Since the callers have to use
> an int64_t type for the length as part of their error checking, it's
> easier to accept an int64_t length to nbd_export_new(), even if
> nbd_export_new() could also use an unsigned type.
>
>>
>> and because off_t was signed;
>>> unsigned for offset because it lets us simplify some math without
>>> having to worry about signed overflow).
>>>
>>> Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>>
>>> ---
>>> v3: new patch
>>> ---
>>> include/block/nbd.h | 4 ++--
>>> nbd/server.c | 14 +++++++-------
>>> qemu-nbd.c | 26 ++++++++++----------------
>>> 3 files changed, 19 insertions(+), 25 deletions(-)
>>>
>>> diff --git a/include/block/nbd.h b/include/block/nbd.h
>>> index 1971b557896..0f252829376 100644
>>> --- a/include/block/nbd.h
>>> +++ b/include/block/nbd.h
>>> @@ -294,8 +294,8 @@ int nbd_errno_to_system_errno(int err);
>>> typedef struct NBDExport NBDExport;
>>> typedef struct NBDClient NBDClient;
>>>
>>> -NBDExport *nbd_export_new(BlockDriverState *bs, off_t dev_offset, off_t size,
>>> - const char *name, const char *description,
>>> +NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
>>> + int64_t size, const char *name, const char *desc,
>>
>> in previous patch you drop use of negative @size parameter, so it looks better
>> to use unsigned for size like for offset
>
> You can't have a size larger than 2^63; but an unsigned type holds
> nearly 2^64. I prefer a signed size, for the same reason that off_t is
> signed, in that checking for a negative size is easier to rule out sizes
> that are too large.
>
>
>>>
>>> static int find_partition(BlockBackend *blk, int partition,
>>> - off_t *offset, off_t *size)
>>> + uint64_t *offset, int64_t *size)
>>
>> function never return negative @size, so what is the reason to keep it signed?
>
> Because the C compiler does NOT like:
>
> int64_t len;
> find_partition(..., &len);
>
> with a uint64_t* parameter type - you HAVE to match the signed-ness of
> your caller's parameter with your pointer type. Since the caller already
> has to use a signed type (to check for blk_getlength() failure AND
> because sizes really cannot exceed 2^63), it's easier to keep it signed
> here.
>
>>
>> Also, type conversion (uint64_t) should be dropped from the function code I think.
>
> Are you talking about this part:
>
> if ((ext_partnum + j + 1) == partition) {
> *offset = (uint64_t)ext[j].start_sector_abs << 9;
> *size = (uint64_t)ext[j].nb_sectors_abs << 9;
> return 0;
> }
> }
> ext_partnum += 4;
> } else if ((i + 1) == partition) {
> *offset = (uint64_t)mbr[i].start_sector_abs << 9;
> *size = (uint64_t)mbr[i].nb_sectors_abs << 9;
> return 0;
>
> No - that has to keep the cast, because .start_sector_abs and
> .nb_sectors_abs are uint32_t values, but we want to shift into 64-bit
> results. You need the cast to force the correct arithmetic rather than
> truncating into a 32-bit value that then gets widened into 64-bit storage.
>
>>> @@ -665,10 +665,6 @@ int main(int argc, char **argv)
>>> error_report("Invalid offset `%s'", optarg);
>>> exit(EXIT_FAILURE);
>>> }
>>> - if (dev_offset < 0) {
>>> - error_report("Offset must be positive `%s'", optarg);
>>> - exit(EXIT_FAILURE);
>>> - }
>>
>> hm, then, may be, s/strtoll/strtoull before this?
>
> I clean that up in patch 6/19.
>
>>
>>> break;
>>> case 'l':
>>> if (strstart(optarg, SNAPSHOT_OPT_BASE, NULL)) {
>>> @@ -1005,15 +1001,14 @@ int main(int argc, char **argv)
>>> }
>>>
>>> if (dev_offset >= fd_size) {
>>> - error_report("Offset (%lld) has to be smaller than the image size "
>>> - "(%lld)",
>>> - (long long int)dev_offset, (long long int)fd_size);
>>> + error_report("Offset (%" PRIu64 ") has to be smaller than the image "
>>> + "size (%" PRIu64 ")", dev_offset, fd_size);
>>
>> PRId64 for fd_size
>
> Sure.
With this one fixed:
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Interesting, decided to search a bit, about don't we have an overflow,
when adding request.offset to exp.dev_offset and found in nbd_co_receive_request:
if (request->from > client->exp->size ||
request->from + request->len > client->exp->size) {
shouldn't it be
if (request->from > client->exp->size ||
request->len > client->exp->size - request->from) {
to avoid overflow?
>
>
>>> @@ -1026,12 +1021,11 @@ int main(int argc, char **argv)
>>> exit(EXIT_FAILURE);
>>> }
>>> /* partition limits are (32-bit << 9); can't overflow 64 bits */
>>> - assert(dev_offset >= 0 && dev_offset + limit >= dev_offset);
>>> + assert(dev_offset + limit >= dev_offset);
>>> if (dev_offset + limit > fd_size) {
>>> - error_report("Discovered partition %d at offset %lld size %lld, "
>>> - "but size exceeds file length %lld", partition,
>>> - (long long int) dev_offset, (long long int) limit,
>>> - (long long int) fd_size);
>>> + error_report("Discovered partition %d at offset %" PRIu64
>>> + " size %" PRId64 ", but size exceeds file length %"
>>> + PRId64, partition, dev_offset, limit, fd_size);
>>> exit(EXIT_FAILURE);
>>
>> hmm, it may be better to place this patch before [03], to squash this chunk into [03]
>
> I didn't mind the churn; also, I prefer patch 3 first, because it's more
> likely to get backported as a bug fix than the rest of the series (and
> the earlier you stick backport candidates in a series, the easier it is
> to backport).
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 05/19] nbd/server: Favor [u]int64_t over off_t
2019-01-16 8:23 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-16 14:23 ` Eric Blake
0 siblings, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-16 14:23 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block, Kevin Wolf, Max Reitz
[-- Attachment #1: Type: text/plain, Size: 1171 bytes --]
On 1/16/19 2:23 AM, Vladimir Sementsov-Ogievskiy wrote:
>
> Interesting, decided to search a bit, about don't we have an overflow,
> when adding request.offset to exp.dev_offset and found in nbd_co_receive_request:
>
> if (request->from > client->exp->size ||
> request->from + request->len > client->exp->size) {
uint64_t + uint32_t > uint64_t
>
> shouldn't it be
>
> if (request->from > client->exp->size ||
> request->len > client->exp->size - request->from) {
>
> to avoid overflow?
You are correct that the canonical way to check for overflow is to
compare against subtraction, rather than do addition before compare.
But we got lucky: even though client->exp->size is uint64_t, in practice
it can only represent at most off_t (remember, off_t is signed), so the
first leg of the branch proves that request->from fits in 63 bits, and
thus the second is performing 63-bit + 32-bit which can be at most 64
bits, and therefore does not overflow. Still, I might rewrite it.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* [Qemu-devel] [PATCH v3 06/19] qemu-nbd: Avoid strtol open-coding
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (4 preceding siblings ...)
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 05/19] nbd/server: Favor [u]int64_t over off_t Eric Blake
@ 2019-01-12 17:57 ` Eric Blake
2019-01-15 12:31 ` Vladimir Sementsov-Ogievskiy
2019-01-15 18:09 ` Richard W.M. Jones
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 07/19] nbd/client: Refactor nbd_receive_list() Eric Blake
` (15 subsequent siblings)
21 siblings, 2 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:57 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block
Our copy-and-pasted open-coding of strtol handling forgot to
handle overflow conditions. Use qemu_strto*() instead.
In the case of --partition, since we insist on a user-supplied
partition to be non-zero, we can use 0 rather than -1 for our
initial value to distinguish when a partition is not being
served, for slightly more optimal code.
The error messages for out-of-bounds values are less specific,
but should not be a terrible loss in quality.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20181215135324.152629-8-eblake@redhat.com>
---
v3: rebase to use int64_t rather than off_t [Vladimir]
---
qemu-nbd.c | 28 +++++++++-------------------
1 file changed, 9 insertions(+), 19 deletions(-)
diff --git a/qemu-nbd.c b/qemu-nbd.c
index 96c0829970c..4670b659167 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -546,9 +546,8 @@ int main(int argc, char **argv)
};
int ch;
int opt_ind = 0;
- char *end;
int flags = BDRV_O_RDWR;
- int partition = -1;
+ int partition = 0;
int ret = 0;
bool seen_cache = false;
bool seen_discard = false;
@@ -660,9 +659,8 @@ int main(int argc, char **argv)
port = optarg;
break;
case 'o':
- dev_offset = strtoll (optarg, &end, 0);
- if (*end) {
- error_report("Invalid offset `%s'", optarg);
+ if (qemu_strtou64(optarg, NULL, 0, &dev_offset) < 0) {
+ error_report("Invalid offset '%s'", optarg);
exit(EXIT_FAILURE);
}
break;
@@ -684,13 +682,9 @@ int main(int argc, char **argv)
flags &= ~BDRV_O_RDWR;
break;
case 'P':
- partition = strtol(optarg, &end, 0);
- if (*end) {
- error_report("Invalid partition `%s'", optarg);
- exit(EXIT_FAILURE);
- }
- if (partition < 1 || partition > 8) {
- error_report("Invalid partition %d", partition);
+ if (qemu_strtoi(optarg, NULL, 0, &partition) < 0 ||
+ partition < 1 || partition > 8) {
+ error_report("Invalid partition '%s'", optarg);
exit(EXIT_FAILURE);
}
break;
@@ -711,15 +705,11 @@ int main(int argc, char **argv)
device = optarg;
break;
case 'e':
- shared = strtol(optarg, &end, 0);
- if (*end) {
+ if (qemu_strtoi(optarg, NULL, 0, &shared) < 0 ||
+ shared < 1) {
error_report("Invalid shared device number '%s'", optarg);
exit(EXIT_FAILURE);
}
- if (shared < 1) {
- error_report("Shared device number must be greater than 0");
- exit(EXIT_FAILURE);
- }
break;
case 'f':
fmt = optarg;
@@ -1007,7 +997,7 @@ int main(int argc, char **argv)
}
fd_size -= dev_offset;
- if (partition != -1) {
+ if (partition) {
int64_t limit;
if (dev_offset) {
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 06/19] qemu-nbd: Avoid strtol open-coding
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 06/19] qemu-nbd: Avoid strtol open-coding Eric Blake
@ 2019-01-15 12:31 ` Vladimir Sementsov-Ogievskiy
2019-01-15 15:35 ` Eric Blake
2019-01-15 18:09 ` Richard W.M. Jones
1 sibling, 1 reply; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-15 12:31 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
12.01.2019 20:57, Eric Blake wrote:
> Our copy-and-pasted open-coding of strtol handling forgot to
> handle overflow conditions. Use qemu_strto*() instead.
>
> In the case of --partition, since we insist on a user-supplied
> partition to be non-zero, we can use 0 rather than -1 for our
> initial value to distinguish when a partition is not being
> served, for slightly more optimal code.
>
> The error messages for out-of-bounds values are less specific,
> but should not be a terrible loss in quality.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Message-Id: <20181215135324.152629-8-eblake@redhat.com>
>
> ---
> v3: rebase to use int64_t rather than off_t [Vladimir]
> ---
> qemu-nbd.c | 28 +++++++++-------------------
> 1 file changed, 9 insertions(+), 19 deletions(-)
>
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index 96c0829970c..4670b659167 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -546,9 +546,8 @@ int main(int argc, char **argv)
> };
> int ch;
> int opt_ind = 0;
> - char *end;
> int flags = BDRV_O_RDWR;
> - int partition = -1;
> + int partition = 0;
> int ret = 0;
> bool seen_cache = false;
> bool seen_discard = false;
> @@ -660,9 +659,8 @@ int main(int argc, char **argv)
> port = optarg;
> break;
> case 'o':
> - dev_offset = strtoll (optarg, &end, 0);
> - if (*end) {
> - error_report("Invalid offset `%s'", optarg);
> + if (qemu_strtou64(optarg, NULL, 0, &dev_offset) < 0) {
> + error_report("Invalid offset '%s'", optarg);
> exit(EXIT_FAILURE);
> }
> break;
> @@ -684,13 +682,9 @@ int main(int argc, char **argv)
> flags &= ~BDRV_O_RDWR;
> break;
> case 'P':
> - partition = strtol(optarg, &end, 0);
> - if (*end) {
> - error_report("Invalid partition `%s'", optarg);
> - exit(EXIT_FAILURE);
> - }
> - if (partition < 1 || partition > 8) {
> - error_report("Invalid partition %d", partition);
> + if (qemu_strtoi(optarg, NULL, 0, &partition) < 0 ||
we can use unsigned conversion like for offset (and unsigned type for partition), but this doesn't really matter.
> + partition < 1 || partition > 8) {
> + error_report("Invalid partition '%s'", optarg);
> exit(EXIT_FAILURE);
> }
> break;
> @@ -711,15 +705,11 @@ int main(int argc, char **argv)
> device = optarg;
> break;
> case 'e':
> - shared = strtol(optarg, &end, 0);
> - if (*end) {
> + if (qemu_strtoi(optarg, NULL, 0, &shared) < 0 ||
and here
> + shared < 1) {
> error_report("Invalid shared device number '%s'", optarg);
> exit(EXIT_FAILURE);
> }
> - if (shared < 1) {
> - error_report("Shared device number must be greater than 0");
> - exit(EXIT_FAILURE);
> - }
> break;
> case 'f':
> fmt = optarg;
> @@ -1007,7 +997,7 @@ int main(int argc, char **argv)
> }
> fd_size -= dev_offset;
>
> - if (partition != -1) {
> + if (partition) {
> int64_t limit;
>
> if (dev_offset) {
>
anyway,
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 06/19] qemu-nbd: Avoid strtol open-coding
2019-01-15 12:31 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-15 15:35 ` Eric Blake
0 siblings, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-15 15:35 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block
[-- Attachment #1: Type: text/plain, Size: 1328 bytes --]
On 1/15/19 6:31 AM, Vladimir Sementsov-Ogievskiy wrote:
> 12.01.2019 20:57, Eric Blake wrote:
>> Our copy-and-pasted open-coding of strtol handling forgot to
>> handle overflow conditions. Use qemu_strto*() instead.
>>
>> In the case of --partition, since we insist on a user-supplied
>> partition to be non-zero, we can use 0 rather than -1 for our
>> initial value to distinguish when a partition is not being
>> served, for slightly more optimal code.
>>
>> The error messages for out-of-bounds values are less specific,
>> but should not be a terrible loss in quality.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> Message-Id: <20181215135324.152629-8-eblake@redhat.com>
>>
>> - if (partition < 1 || partition > 8) {
>> - error_report("Invalid partition %d", partition);
>> + if (qemu_strtoi(optarg, NULL, 0, &partition) < 0 ||
>
> we can use unsigned conversion like for offset (and unsigned type for partition), but this doesn't really matter.
Yes, but I didn't see the point in changing the variable types in this
patch.
>
> anyway,
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [Qemu-devel] [PATCH v3 06/19] qemu-nbd: Avoid strtol open-coding
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 06/19] qemu-nbd: Avoid strtol open-coding Eric Blake
2019-01-15 12:31 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-15 18:09 ` Richard W.M. Jones
1 sibling, 0 replies; 71+ messages in thread
From: Richard W.M. Jones @ 2019-01-15 18:09 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On Sat, Jan 12, 2019 at 11:57:59AM -0600, Eric Blake wrote:
> Our copy-and-pasted open-coding of strtol handling forgot to
> handle overflow conditions. Use qemu_strto*() instead.
>
> In the case of --partition, since we insist on a user-supplied
> partition to be non-zero, we can use 0 rather than -1 for our
> initial value to distinguish when a partition is not being
> served, for slightly more optimal code.
>
> The error messages for out-of-bounds values are less specific,
> but should not be a terrible loss in quality.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Message-Id: <20181215135324.152629-8-eblake@redhat.com>
>
> ---
> v3: rebase to use int64_t rather than off_t [Vladimir]
> ---
> qemu-nbd.c | 28 +++++++++-------------------
> 1 file changed, 9 insertions(+), 19 deletions(-)
>
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index 96c0829970c..4670b659167 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -546,9 +546,8 @@ int main(int argc, char **argv)
> };
> int ch;
> int opt_ind = 0;
> - char *end;
> int flags = BDRV_O_RDWR;
> - int partition = -1;
> + int partition = 0;
> int ret = 0;
> bool seen_cache = false;
> bool seen_discard = false;
> @@ -660,9 +659,8 @@ int main(int argc, char **argv)
> port = optarg;
> break;
> case 'o':
> - dev_offset = strtoll (optarg, &end, 0);
> - if (*end) {
> - error_report("Invalid offset `%s'", optarg);
> + if (qemu_strtou64(optarg, NULL, 0, &dev_offset) < 0) {
> + error_report("Invalid offset '%s'", optarg);
> exit(EXIT_FAILURE);
> }
> break;
> @@ -684,13 +682,9 @@ int main(int argc, char **argv)
> flags &= ~BDRV_O_RDWR;
> break;
> case 'P':
> - partition = strtol(optarg, &end, 0);
> - if (*end) {
> - error_report("Invalid partition `%s'", optarg);
> - exit(EXIT_FAILURE);
> - }
> - if (partition < 1 || partition > 8) {
> - error_report("Invalid partition %d", partition);
> + if (qemu_strtoi(optarg, NULL, 0, &partition) < 0 ||
> + partition < 1 || partition > 8) {
> + error_report("Invalid partition '%s'", optarg);
> exit(EXIT_FAILURE);
> }
> break;
> @@ -711,15 +705,11 @@ int main(int argc, char **argv)
> device = optarg;
> break;
> case 'e':
> - shared = strtol(optarg, &end, 0);
> - if (*end) {
> + if (qemu_strtoi(optarg, NULL, 0, &shared) < 0 ||
> + shared < 1) {
> error_report("Invalid shared device number '%s'", optarg);
> exit(EXIT_FAILURE);
> }
> - if (shared < 1) {
> - error_report("Shared device number must be greater than 0");
> - exit(EXIT_FAILURE);
> - }
> break;
> case 'f':
> fmt = optarg;
> @@ -1007,7 +997,7 @@ int main(int argc, char **argv)
> }
> fd_size -= dev_offset;
>
> - if (partition != -1) {
> + if (partition) {
> int64_t limit;
>
> if (dev_offset) {
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
^ permalink raw reply [flat|nested] 71+ messages in thread
* [Qemu-devel] [PATCH v3 07/19] nbd/client: Refactor nbd_receive_list()
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (5 preceding siblings ...)
2019-01-12 17:57 ` [Qemu-devel] [PATCH v3 06/19] qemu-nbd: Avoid strtol open-coding Eric Blake
@ 2019-01-12 17:58 ` Eric Blake
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 08/19] nbd/client: Move export name into NBDExportInfo Eric Blake
` (14 subsequent siblings)
21 siblings, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:58 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block
Right now, nbd_receive_list() is only called by
nbd_receive_query_exports(), which in turn is only called if the
server lacks NBD_OPT_GO but has working option negotiation, and is
merely used as a quality-of-implementation trick since servers
can't give decent errors for NBD_OPT_EXPORT_NAME. However, servers
that lack NBD_OPT_GO are becoming increasingly rare (nbdkit was a
latecomer, in Aug 2018, but qemu has been such a server since commit
f37708f6 in July 2017 and released in 2.10), so it no longer makes
sense to micro-optimize that function for performance.
Furthermore, when debugging a server's implementation, tracing the
full reply (both names and descriptions) is useful, not to mention
that upcoming patches adding 'qemu-nbd --list' will want to collect
that data. And when you consider that a server can send an export
name up to the NBD protocol length limit of 4k; but our current
NBD_MAX_NAME_SIZE is only 256, we can't trace all valid server
names without more storage, but 4k is large enough that the heap
is better than the stack for long names.
Thus, I'm changing the division of labor, with nbd_receive_list()
now always malloc'ing a result on success (the malloc is bounded
by the fact that we reject servers with a reply length larger
than 32M), and moving the comparison to 'wantname' to the caller.
There is a minor change in behavior where a server with 0 exports
(an immediate NBD_REP_ACK reply) is now no longer distinguished
from a server without LIST support (NBD_REP_ERR_UNSUP); this
information could be preserved with a complication to the calling
contract to provide a bit more information, but I didn't see the
point. After all, the worst that can happen if our guess at a
match is wrong is that the caller will get a cryptic disconnect
when NBD_OPT_EXPORT_NAME fails (which is no different from what
would happen if we had not tried LIST), while treating an empty
list as immediate failure would prevent connecting to really old
servers that really did lack LIST. Besides, NBD servers with 0
exports are rare (qemu can do it when using QMP nbd-server-start
without nbd-server-add - but qemu understands NBD_OPT_GO and
thus won't tickle this change in behavior).
Fix the spelling of foundExport to match coding standards while
in the area.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20181215135324.152629-10-eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
v3: comment tweak, s/listEmpty/list_empty/,
s/foundExport/found_export/ [Vladimir]
---
nbd/client.c | 91 ++++++++++++++++++++++++++++++------------------
nbd/trace-events | 1 +
2 files changed, 59 insertions(+), 33 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index f625c207c54..fd4ba8dec37 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -234,18 +234,24 @@ static int nbd_handle_reply_err(QIOChannel *ioc, NBDOptionReply *reply,
return result;
}
-/* Process another portion of the NBD_OPT_LIST reply. Set *@match if
- * the current reply matches @want or if the server does not support
- * NBD_OPT_LIST, otherwise leave @match alone. Return 0 if iteration
- * is complete, positive if more replies are expected, or negative
- * with @errp set if an unrecoverable error occurred. */
-static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
+/* nbd_receive_list:
+ * Process another portion of the NBD_OPT_LIST reply, populating any
+ * name received into *@name. If @description is non-NULL, and the
+ * server provided a description, that is also populated. The caller
+ * must eventually call g_free() on success.
+ * Returns 1 if name and description were set and iteration must continue,
+ * 0 if iteration is complete (including if OPT_LIST unsupported),
+ * -1 with @errp set if an unrecoverable error occurred.
+ */
+static int nbd_receive_list(QIOChannel *ioc, char **name, char **description,
Error **errp)
{
+ int ret = -1;
NBDOptionReply reply;
uint32_t len;
uint32_t namelen;
- char name[NBD_MAX_NAME_SIZE + 1];
+ char *local_name = NULL;
+ char *local_desc = NULL;
int error;
if (nbd_receive_option_reply(ioc, NBD_OPT_LIST, &reply, errp) < 0) {
@@ -253,9 +259,6 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
}
error = nbd_handle_reply_err(ioc, &reply, errp);
if (error <= 0) {
- /* The server did not support NBD_OPT_LIST, so set *match on
- * the assumption that any name will be accepted. */
- *match = true;
return error;
}
len = reply.length;
@@ -292,33 +295,38 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
nbd_send_opt_abort(ioc);
return -1;
}
- if (namelen != strlen(want)) {
- if (nbd_drop(ioc, len, errp) < 0) {
- error_prepend(errp,
- "failed to skip export name with wrong length: ");
- nbd_send_opt_abort(ioc);
- return -1;
- }
- return 1;
- }
- assert(namelen < sizeof(name));
- if (nbd_read(ioc, name, namelen, errp) < 0) {
+ local_name = g_malloc(namelen + 1);
+ if (nbd_read(ioc, local_name, namelen, errp) < 0) {
error_prepend(errp, "failed to read export name: ");
nbd_send_opt_abort(ioc);
- return -1;
+ goto out;
}
- name[namelen] = '\0';
+ local_name[namelen] = '\0';
len -= namelen;
- if (nbd_drop(ioc, len, errp) < 0) {
- error_prepend(errp, "failed to read export description: ");
- nbd_send_opt_abort(ioc);
- return -1;
+ if (len) {
+ local_desc = g_malloc(len + 1);
+ if (nbd_read(ioc, local_desc, len, errp) < 0) {
+ error_prepend(errp, "failed to read export description: ");
+ nbd_send_opt_abort(ioc);
+ goto out;
+ }
+ local_desc[len] = '\0';
}
- if (!strcmp(name, want)) {
- *match = true;
+
+ trace_nbd_receive_list(local_name, local_desc ?: "");
+ *name = local_name;
+ local_name = NULL;
+ if (description) {
+ *description = local_desc;
+ local_desc = NULL;
}
- return 1;
+ ret = 1;
+
+ out:
+ g_free(local_name);
+ g_free(local_desc);
+ return ret;
}
@@ -493,7 +501,8 @@ static int nbd_receive_query_exports(QIOChannel *ioc,
const char *wantname,
Error **errp)
{
- bool foundExport = false;
+ bool list_empty = true;
+ bool found_export = false;
trace_nbd_receive_query_exports_start(wantname);
if (nbd_send_option_request(ioc, NBD_OPT_LIST, 0, NULL, errp) < 0) {
@@ -501,14 +510,25 @@ static int nbd_receive_query_exports(QIOChannel *ioc,
}
while (1) {
- int ret = nbd_receive_list(ioc, wantname, &foundExport, errp);
+ char *name;
+ int ret = nbd_receive_list(ioc, &name, NULL, errp);
if (ret < 0) {
/* Server gave unexpected reply */
return -1;
} else if (ret == 0) {
/* Done iterating. */
- if (!foundExport) {
+ if (list_empty) {
+ /*
+ * We don't have enough context to tell a server that
+ * sent an empty list apart from a server that does
+ * not support the list command; but as this function
+ * is just used to trigger a nicer error message
+ * before trying NBD_OPT_EXPORT_NAME, assume the
+ * export is available.
+ */
+ return 0;
+ } else if (!found_export) {
error_setg(errp, "No export with name '%s' available",
wantname);
nbd_send_opt_abort(ioc);
@@ -517,6 +537,11 @@ static int nbd_receive_query_exports(QIOChannel *ioc,
trace_nbd_receive_query_exports_success(wantname);
return 0;
}
+ list_empty = false;
+ if (!strcmp(name, wantname)) {
+ found_export = true;
+ }
+ g_free(name);
}
}
diff --git a/nbd/trace-events b/nbd/trace-events
index 5492042acbf..d1e1ca64ee5 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -3,6 +3,7 @@ nbd_send_option_request(uint32_t opt, const char *name, uint32_t len) "Sending o
nbd_receive_option_reply(uint32_t option, const char *optname, uint32_t type, const char *typename, uint32_t length) "Received option reply %" PRIu32" (%s), type %" PRIu32" (%s), len %" PRIu32
nbd_server_error_msg(uint32_t err, const char *type, const char *msg) "server reported error 0x%" PRIx32 " (%s) with additional message: %s"
nbd_reply_err_unsup(uint32_t option, const char *name) "server doesn't understand request %" PRIu32 " (%s), attempting fallback"
+nbd_receive_list(const char *name, const char *desc) "export list includes '%s', description '%s'"
nbd_opt_go_start(const char *name) "Attempting NBD_OPT_GO for export '%s'"
nbd_opt_go_success(void) "Export is good to go"
nbd_opt_go_info_unknown(int info, const char *name) "Ignoring unknown info %d (%s)"
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* [Qemu-devel] [PATCH v3 08/19] nbd/client: Move export name into NBDExportInfo
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (6 preceding siblings ...)
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 07/19] nbd/client: Refactor nbd_receive_list() Eric Blake
@ 2019-01-12 17:58 ` Eric Blake
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 09/19] nbd/client: Change signature of nbd_negotiate_simple_meta_context() Eric Blake
` (13 subsequent siblings)
21 siblings, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:58 UTC (permalink / raw)
To: qemu-devel
Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block, Kevin Wolf, Max Reitz
Refactor the 'name' parameter of nbd_receive_negotiate() from
being a separate parameter into being part of the in-out 'info'.
This also spills over to a simplification of nbd_opt_go().
The main driver for this refactoring is that an upcoming patch
would like to add support to qemu-nbd to list information about
all exports available on a server, where the name(s) will be
provided by the server instead of the client. But another benefit
is that we can now allow the client to explicitly specify the
empty export name "" even when connecting to an oldstyle server
(even if qemu is no longer such a server after commit 7f7dfe2a).
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181215135324.152629-11-eblake@redhat.com>
---
include/block/nbd.h | 8 ++++----
block/nbd-client.c | 5 +++--
nbd/client.c | 39 ++++++++++++++++++---------------------
qemu-nbd.c | 6 ++++--
nbd/trace-events | 2 +-
5 files changed, 30 insertions(+), 30 deletions(-)
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 0f252829376..6d76d3dc221 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -262,6 +262,7 @@ struct NBDExportInfo {
/* Set by client before nbd_receive_negotiate() */
bool request_sizes;
char *x_dirty_bitmap;
+ char *name; /* must be non-NULL */
/* In-out fields, set by client before nbd_receive_negotiate() and
* updated by server results during nbd_receive_negotiate() */
@@ -279,10 +280,9 @@ struct NBDExportInfo {
};
typedef struct NBDExportInfo NBDExportInfo;
-int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
- QCryptoTLSCreds *tlscreds, const char *hostname,
- QIOChannel **outioc, NBDExportInfo *info,
- Error **errp);
+int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, QIOChannel **outioc,
+ NBDExportInfo *info, Error **errp);
int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
Error **errp);
int nbd_send_request(QIOChannel *ioc, NBDRequest *request);
diff --git a/block/nbd-client.c b/block/nbd-client.c
index ef320759716..3309376bc16 100644
--- a/block/nbd-client.c
+++ b/block/nbd-client.c
@@ -999,10 +999,11 @@ int nbd_client_init(BlockDriverState *bs,
client->info.structured_reply = true;
client->info.base_allocation = true;
client->info.x_dirty_bitmap = g_strdup(x_dirty_bitmap);
- ret = nbd_receive_negotiate(QIO_CHANNEL(sioc), export,
- tlscreds, hostname,
+ client->info.name = g_strdup(export ?: "");
+ ret = nbd_receive_negotiate(QIO_CHANNEL(sioc), tlscreds, hostname,
&client->ioc, &client->info, errp);
g_free(client->info.x_dirty_bitmap);
+ g_free(client->info.name);
if (ret < 0) {
logout("Failed to negotiate with the NBD server\n");
return ret;
diff --git a/nbd/client.c b/nbd/client.c
index fd4ba8dec37..8227e69478a 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -330,15 +330,14 @@ static int nbd_receive_list(QIOChannel *ioc, char **name, char **description,
}
-/* Returns -1 if NBD_OPT_GO proves the export @wantname cannot be
+/* Returns -1 if NBD_OPT_GO proves the export @info->name cannot be
* used, 0 if NBD_OPT_GO is unsupported (fall back to NBD_OPT_LIST and
* NBD_OPT_EXPORT_NAME in that case), and > 0 if the export is good to
- * go (with @info populated). */
-static int nbd_opt_go(QIOChannel *ioc, const char *wantname,
- NBDExportInfo *info, Error **errp)
+ * go (with the rest of @info populated). */
+static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
{
NBDOptionReply reply;
- uint32_t len = strlen(wantname);
+ uint32_t len = strlen(info->name);
uint16_t type;
int error;
char *buf;
@@ -348,10 +347,10 @@ static int nbd_opt_go(QIOChannel *ioc, const char *wantname,
* flags still 0 is a witness of a broken server. */
info->flags = 0;
- trace_nbd_opt_go_start(wantname);
+ trace_nbd_opt_go_start(info->name);
buf = g_malloc(4 + len + 2 + 2 * info->request_sizes + 1);
stl_be_p(buf, len);
- memcpy(buf + 4, wantname, len);
+ memcpy(buf + 4, info->name, len);
/* At most one request, everything else up to server */
stw_be_p(buf + 4 + len, info->request_sizes);
if (info->request_sizes) {
@@ -753,10 +752,9 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return 0;
}
-int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
- QCryptoTLSCreds *tlscreds, const char *hostname,
- QIOChannel **outioc, NBDExportInfo *info,
- Error **errp)
+int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, QIOChannel **outioc,
+ NBDExportInfo *info, Error **errp)
{
uint64_t magic;
int rc;
@@ -766,6 +764,8 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
trace_nbd_receive_negotiate(tlscreds, hostname ? hostname : "<null>");
+ assert(info->name);
+ trace_nbd_receive_negotiate_name(info->name);
info->structured_reply = false;
info->base_allocation = false;
rc = -EINVAL;
@@ -834,10 +834,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
goto fail;
}
}
- if (!name) {
- trace_nbd_receive_negotiate_default_name();
- name = "";
- }
if (fixedNewStyle) {
int result;
@@ -853,7 +849,8 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
if (info->structured_reply && base_allocation) {
result = nbd_negotiate_simple_meta_context(
- ioc, name, info->x_dirty_bitmap ?: "base:allocation",
+ ioc, info->name,
+ info->x_dirty_bitmap ?: "base:allocation",
&info->meta_base_allocation_id, errp);
if (result < 0) {
goto fail;
@@ -866,7 +863,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
* TLS). If it is not available, fall back to
* NBD_OPT_LIST for nicer error messages about a missing
* export, then use NBD_OPT_EXPORT_NAME. */
- result = nbd_opt_go(ioc, name, info, errp);
+ result = nbd_opt_go(ioc, info, errp);
if (result < 0) {
goto fail;
}
@@ -879,12 +876,12 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
* query gives us better error reporting if the
* export name is not available.
*/
- if (nbd_receive_query_exports(ioc, name, errp) < 0) {
+ if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
goto fail;
}
}
/* write the export name request */
- if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, name,
+ if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, info->name,
errp) < 0) {
goto fail;
}
@@ -904,8 +901,8 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
} else if (magic == NBD_CLIENT_MAGIC) {
uint32_t oldflags;
- if (name) {
- error_setg(errp, "Server does not support export names");
+ if (*info->name) {
+ error_setg(errp, "Server does not support non-empty export names");
goto fail;
}
if (tlscreds) {
diff --git a/qemu-nbd.c b/qemu-nbd.c
index 4670b659167..f1c24683129 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -264,7 +264,7 @@ static void *show_parts(void *arg)
static void *nbd_client_thread(void *arg)
{
char *device = arg;
- NBDExportInfo info = { .request_sizes = false, };
+ NBDExportInfo info = { .request_sizes = false, .name = g_strdup("") };
QIOChannelSocket *sioc;
int fd;
int ret;
@@ -279,7 +279,7 @@ static void *nbd_client_thread(void *arg)
goto out;
}
- ret = nbd_receive_negotiate(QIO_CHANNEL(sioc), NULL,
+ ret = nbd_receive_negotiate(QIO_CHANNEL(sioc),
NULL, NULL, NULL, &info, &local_error);
if (ret < 0) {
if (local_error) {
@@ -318,6 +318,7 @@ static void *nbd_client_thread(void *arg)
}
close(fd);
object_unref(OBJECT(sioc));
+ g_free(info.name);
kill(getpid(), SIGTERM);
return (void *) EXIT_SUCCESS;
@@ -326,6 +327,7 @@ out_fd:
out_socket:
object_unref(OBJECT(sioc));
out:
+ g_free(info.name);
kill(getpid(), SIGTERM);
return (void *) EXIT_FAILURE;
}
diff --git a/nbd/trace-events b/nbd/trace-events
index d1e1ca64ee5..c3966d2b653 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -17,7 +17,7 @@ nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of contex
nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are 0x%" PRIx32
-nbd_receive_negotiate_default_name(void) "Using default NBD export name \"\""
+nbd_receive_negotiate_name(const char *name) "Requesting NBD export name '%s'"
nbd_receive_negotiate_size_flags(uint64_t size, uint16_t flags) "Size is %" PRIu64 ", export flags 0x%" PRIx16
nbd_init_set_socket(void) "Setting NBD socket"
nbd_init_set_block_size(unsigned long block_size) "Setting block size to %lu"
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* [Qemu-devel] [PATCH v3 09/19] nbd/client: Change signature of nbd_negotiate_simple_meta_context()
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (7 preceding siblings ...)
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 08/19] nbd/client: Move export name into NBDExportInfo Eric Blake
@ 2019-01-12 17:58 ` Eric Blake
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 10/19] nbd/client: Split out nbd_send_one_meta_context() Eric Blake
` (12 subsequent siblings)
21 siblings, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:58 UTC (permalink / raw)
To: qemu-devel
Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block, Kevin Wolf, Max Reitz
Pass 'info' instead of three separate parameters related to info,
when requesting the server to set the meta context. Update the
NBDExportInfo struct to rename the received id field to match the
fact that we are currently overloading the field to match whatever
context the user supplied through the x-dirty-bitmap hack, as well
as adding a TODO comment to remind future patches about a desire
to request two contexts at once.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20181215135324.152629-12-eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
include/block/nbd.h | 2 +-
block/nbd-client.c | 4 ++--
nbd/client.c | 53 +++++++++++++++++++++------------------------
3 files changed, 28 insertions(+), 31 deletions(-)
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 6d76d3dc221..bdc7ec7195a 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -276,7 +276,7 @@ struct NBDExportInfo {
uint32_t opt_block;
uint32_t max_block;
- uint32_t meta_base_allocation_id;
+ uint32_t context_id;
};
typedef struct NBDExportInfo NBDExportInfo;
diff --git a/block/nbd-client.c b/block/nbd-client.c
index 3309376bc16..813539676d2 100644
--- a/block/nbd-client.c
+++ b/block/nbd-client.c
@@ -249,11 +249,11 @@ static int nbd_parse_blockstatus_payload(NBDClientSession *client,
}
context_id = payload_advance32(&payload);
- if (client->info.meta_base_allocation_id != context_id) {
+ if (client->info.context_id != context_id) {
error_setg(errp, "Protocol error: unexpected context id %d for "
"NBD_REPLY_TYPE_BLOCK_STATUS, when negotiated context "
"id is %d", context_id,
- client->info.meta_base_allocation_id);
+ client->info.context_id);
return -EINVAL;
}
diff --git a/nbd/client.c b/nbd/client.c
index 8227e69478a..77993890f04 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -630,26 +630,30 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
}
/* nbd_negotiate_simple_meta_context:
- * Set one meta context. Simple means that reply must contain zero (not
- * negotiated) or one (negotiated) contexts. More contexts would be considered
- * as a protocol error. It's also implied that meta-data query equals queried
- * context name, so, if server replies with something different than @context,
- * it is considered an error too.
- * return 1 for successful negotiation, context_id is set
+ * Request the server to set the meta context for export @info->name
+ * using @info->x_dirty_bitmap with a fallback to "base:allocation",
+ * setting @info->context_id to the resulting id. Fail if the server
+ * responds with more than one context or with a context different
+ * than the query.
+ * return 1 for successful negotiation,
* 0 if operation is unsupported,
* -1 with errp set for any other error
*/
static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
- const char *export,
- const char *context,
- uint32_t *context_id,
+ NBDExportInfo *info,
Error **errp)
{
+ /*
+ * TODO: Removing the x_dirty_bitmap hack will mean refactoring
+ * this function to request and store ids for multiple contexts
+ * (both base:allocation and a dirty bitmap), at which point this
+ * function should lose the term _simple.
+ */
int ret;
NBDOptionReply reply;
- uint32_t received_id = 0;
+ const char *context = info->x_dirty_bitmap ?: "base:allocation";
bool received = false;
- uint32_t export_len = strlen(export);
+ uint32_t export_len = strlen(info->name);
uint32_t context_len = strlen(context);
uint32_t data_len = sizeof(export_len) + export_len +
sizeof(uint32_t) + /* number of queries */
@@ -657,9 +661,9 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
char *data = g_malloc(data_len);
char *p = data;
- trace_nbd_opt_meta_request(context, export);
+ trace_nbd_opt_meta_request(context, info->name);
stl_be_p(p, export_len);
- memcpy(p += sizeof(export_len), export, export_len);
+ memcpy(p += sizeof(export_len), info->name, export_len);
stl_be_p(p += export_len, 1);
stl_be_p(p += sizeof(uint32_t), context_len);
memcpy(p += sizeof(context_len), context, context_len);
@@ -685,7 +689,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
if (reply.type == NBD_REP_META_CONTEXT) {
char *name;
- if (reply.length != sizeof(received_id) + context_len) {
+ if (reply.length != sizeof(info->context_id) + context_len) {
error_setg(errp, "Failed to negotiate meta context '%s', server "
"answered with unexpected length %" PRIu32, context,
reply.length);
@@ -693,12 +697,13 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return -1;
}
- if (nbd_read(ioc, &received_id, sizeof(received_id), errp) < 0) {
+ if (nbd_read(ioc, &info->context_id, sizeof(info->context_id),
+ errp) < 0) {
return -1;
}
- received_id = be32_to_cpu(received_id);
+ info->context_id = be32_to_cpu(info->context_id);
- reply.length -= sizeof(received_id);
+ reply.length -= sizeof(info->context_id);
name = g_malloc(reply.length + 1);
if (nbd_read(ioc, name, reply.length, errp) < 0) {
g_free(name);
@@ -715,7 +720,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
}
g_free(name);
- trace_nbd_opt_meta_reply(context, received_id);
+ trace_nbd_opt_meta_reply(context, info->context_id);
received = true;
/* receive NBD_REP_ACK */
@@ -744,12 +749,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return -1;
}
- if (received) {
- *context_id = received_id;
- return 1;
- }
-
- return 0;
+ return received;
}
int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
@@ -848,10 +848,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
}
if (info->structured_reply && base_allocation) {
- result = nbd_negotiate_simple_meta_context(
- ioc, info->name,
- info->x_dirty_bitmap ?: "base:allocation",
- &info->meta_base_allocation_id, errp);
+ result = nbd_negotiate_simple_meta_context(ioc, info, errp);
if (result < 0) {
goto fail;
}
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* [Qemu-devel] [PATCH v3 10/19] nbd/client: Split out nbd_send_one_meta_context()
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (8 preceding siblings ...)
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 09/19] nbd/client: Change signature of nbd_negotiate_simple_meta_context() Eric Blake
@ 2019-01-12 17:58 ` Eric Blake
2019-01-15 13:18 ` Vladimir Sementsov-Ogievskiy
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 11/19] nbd/client: Split out nbd_receive_one_meta_context() Eric Blake
` (11 subsequent siblings)
21 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:58 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block
Refactor nbd_negotiate_simple_meta_context() to pull out the
code that can be reused to send a LIST request for 0 or 1 query.
No semantic change. The old comment about 'sizeof(uint32_t)'
being equivalent to '/* number of queries */' is no longer
needed, now that we are computing 'sizeof(queries)' instead.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20181215135324.152629-14-eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
---
v3: Improve commit message [Rich], formatting tweak [checkpatch],
rebase to dropped patch
---
nbd/client.c | 67 +++++++++++++++++++++++++++++++++---------------
nbd/trace-events | 2 +-
2 files changed, 48 insertions(+), 21 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index 77993890f04..3c716be2719 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -629,6 +629,49 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
return QIO_CHANNEL(tioc);
}
+/*
+ * nbd_send_one_meta_context:
+ * Send 0 or 1 set/list meta context queries.
+ * Return 0 on success, -1 with errp set for any error
+ */
+static int nbd_send_one_meta_context(QIOChannel *ioc,
+ uint32_t opt,
+ const char *export,
+ const char *query,
+ Error **errp)
+{
+ int ret;
+ uint32_t export_len = strlen(export);
+ uint32_t queries = !!query;
+ uint32_t context_len = 0;
+ uint32_t data_len;
+ char *data;
+ char *p;
+
+ data_len = sizeof(export_len) + export_len + sizeof(queries);
+ if (query) {
+ context_len = strlen(query);
+ data_len += sizeof(context_len) + context_len;
+ } else {
+ assert(opt == NBD_OPT_LIST_META_CONTEXT);
+ }
+ data = g_malloc(data_len);
+ p = data;
+
+ trace_nbd_opt_meta_request(nbd_opt_lookup(opt), query ?: "(all)", export);
+ stl_be_p(p, export_len);
+ memcpy(p += sizeof(export_len), export, export_len);
+ stl_be_p(p += export_len, queries);
+ if (query) {
+ stl_be_p(p += sizeof(uint32_t), context_len);
+ memcpy(p += sizeof(context_len), query, context_len);
+ }
+
+ ret = nbd_send_option_request(ioc, opt, data_len, data, errp);
+ g_free(data);
+ return ret;
+}
+
/* nbd_negotiate_simple_meta_context:
* Request the server to set the meta context for export @info->name
* using @info->x_dirty_bitmap with a fallback to "base:allocation",
@@ -653,26 +696,10 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
NBDOptionReply reply;
const char *context = info->x_dirty_bitmap ?: "base:allocation";
bool received = false;
- uint32_t export_len = strlen(info->name);
- uint32_t context_len = strlen(context);
- uint32_t data_len = sizeof(export_len) + export_len +
- sizeof(uint32_t) + /* number of queries */
- sizeof(context_len) + context_len;
- char *data = g_malloc(data_len);
- char *p = data;
- trace_nbd_opt_meta_request(context, info->name);
- stl_be_p(p, export_len);
- memcpy(p += sizeof(export_len), info->name, export_len);
- stl_be_p(p += export_len, 1);
- stl_be_p(p += sizeof(uint32_t), context_len);
- memcpy(p += sizeof(context_len), context, context_len);
-
- ret = nbd_send_option_request(ioc, NBD_OPT_SET_META_CONTEXT, data_len, data,
- errp);
- g_free(data);
- if (ret < 0) {
- return ret;
+ if (nbd_send_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
+ info->name, context, errp) < 0) {
+ return -1;
}
if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
@@ -689,7 +716,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
if (reply.type == NBD_REP_META_CONTEXT) {
char *name;
- if (reply.length != sizeof(info->context_id) + context_len) {
+ if (reply.length != sizeof(info->context_id) + strlen(context)) {
error_setg(errp, "Failed to negotiate meta context '%s', server "
"answered with unexpected length %" PRIu32, context,
reply.length);
diff --git a/nbd/trace-events b/nbd/trace-events
index c3966d2b653..59521e47a3d 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -12,7 +12,7 @@ nbd_receive_query_exports_start(const char *wantname) "Querying export list for
nbd_receive_query_exports_success(const char *wantname) "Found desired export name '%s'"
nbd_receive_starttls_new_client(void) "Setting up TLS"
nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
-nbd_opt_meta_request(const char *context, const char *export) "Requesting to set meta context %s for export %s"
+nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/19] nbd/client: Split out nbd_send_one_meta_context()
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 10/19] nbd/client: Split out nbd_send_one_meta_context() Eric Blake
@ 2019-01-15 13:18 ` Vladimir Sementsov-Ogievskiy
2019-01-15 15:44 ` Eric Blake
0 siblings, 1 reply; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-15 13:18 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
12.01.2019 20:58, Eric Blake wrote:
> Refactor nbd_negotiate_simple_meta_context() to pull out the
> code that can be reused to send a LIST request for 0 or 1 query.
> No semantic change. The old comment about 'sizeof(uint32_t)'
> being equivalent to '/* number of queries */' is no longer
> needed, now that we are computing 'sizeof(queries)' instead.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Message-Id: <20181215135324.152629-14-eblake@redhat.com>
> Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
>
> ---
> v3: Improve commit message [Rich], formatting tweak [checkpatch],
> rebase to dropped patch
> ---
> nbd/client.c | 67 +++++++++++++++++++++++++++++++++---------------
> nbd/trace-events | 2 +-
> 2 files changed, 48 insertions(+), 21 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 77993890f04..3c716be2719 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -629,6 +629,49 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
> return QIO_CHANNEL(tioc);
> }
>
> +/*
> + * nbd_send_one_meta_context:
> + * Send 0 or 1 set/list meta context queries.
> + * Return 0 on success, -1 with errp set for any error
> + */
> +static int nbd_send_one_meta_context(QIOChannel *ioc,
> + uint32_t opt,
> + const char *export,
> + const char *query,
> + Error **errp)
> +{
> + int ret;
> + uint32_t export_len = strlen(export);
> + uint32_t queries = !!query;
n_ or nb_ prefix may make it more clear
> + uint32_t context_len = 0;
> + uint32_t data_len;
> + char *data;
> + char *p;
> +
> + data_len = sizeof(export_len) + export_len + sizeof(queries);
> + if (query) {
> + context_len = strlen(query);
looks like it then should be query_len
> + data_len += sizeof(context_len) + context_len;
> + } else {
> + assert(opt == NBD_OPT_LIST_META_CONTEXT);
> + }
> + data = g_malloc(data_len);
> + p = data;
may use p = data = g_malloc
> +
> + trace_nbd_opt_meta_request(nbd_opt_lookup(opt), query ?: "(all)", export);
> + stl_be_p(p, export_len);
> + memcpy(p += sizeof(export_len), export, export_len);
> + stl_be_p(p += export_len, queries);
> + if (query) {
> + stl_be_p(p += sizeof(uint32_t), context_len);
:), aha, please, s/uint32_t/queries, as you promised
Hmm, its my code. It's hard to read and not very comfortable to maintain..
In block/nbd-client.c we have
payload_advance* functions, to read such formatted data, I think, it should be
good to make something like this for server-part. Not about these series, of course.
Interesting, troubles around "don't use be64_to_cpuS, use only be64_to_cpu",
do they apply somehow to *_be_p functions family?
> + memcpy(p += sizeof(context_len), query, context_len);
> + }
> +
> + ret = nbd_send_option_request(ioc, opt, data_len, data, errp);
> + g_free(data);
> + return ret;
> +}
> +
> /* nbd_negotiate_simple_meta_context:
> * Request the server to set the meta context for export @info->name
> * using @info->x_dirty_bitmap with a fallback to "base:allocation",
> @@ -653,26 +696,10 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> NBDOptionReply reply;
> const char *context = info->x_dirty_bitmap ?: "base:allocation";
> bool received = false;
> - uint32_t export_len = strlen(info->name);
> - uint32_t context_len = strlen(context);
> - uint32_t data_len = sizeof(export_len) + export_len +
> - sizeof(uint32_t) + /* number of queries */
> - sizeof(context_len) + context_len;
> - char *data = g_malloc(data_len);
> - char *p = data;
>
> - trace_nbd_opt_meta_request(context, info->name);
> - stl_be_p(p, export_len);
> - memcpy(p += sizeof(export_len), info->name, export_len);
> - stl_be_p(p += export_len, 1);
> - stl_be_p(p += sizeof(uint32_t), context_len);
> - memcpy(p += sizeof(context_len), context, context_len);
> -
> - ret = nbd_send_option_request(ioc, NBD_OPT_SET_META_CONTEXT, data_len, data,
> - errp);
> - g_free(data);
> - if (ret < 0) {
> - return ret;
> + if (nbd_send_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
> + info->name, context, errp) < 0) {
> + return -1;
> }
>
> if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
> @@ -689,7 +716,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> if (reply.type == NBD_REP_META_CONTEXT) {
> char *name;
>
> - if (reply.length != sizeof(info->context_id) + context_len) {
> + if (reply.length != sizeof(info->context_id) + strlen(context)) {
> error_setg(errp, "Failed to negotiate meta context '%s', server "
> "answered with unexpected length %" PRIu32, context,
> reply.length);
> diff --git a/nbd/trace-events b/nbd/trace-events
> index c3966d2b653..59521e47a3d 100644
> --- a/nbd/trace-events
> +++ b/nbd/trace-events
> @@ -12,7 +12,7 @@ nbd_receive_query_exports_start(const char *wantname) "Querying export list for
> nbd_receive_query_exports_success(const char *wantname) "Found desired export name '%s'"
> nbd_receive_starttls_new_client(void) "Setting up TLS"
> nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
> -nbd_opt_meta_request(const char *context, const char *export) "Requesting to set meta context %s for export %s"
> +nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
Hmm, you forget nbd_opt_lookup()
With this and s/uint32_t/queries (other tiny things up to you):
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
> nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
> nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/19] nbd/client: Split out nbd_send_one_meta_context()
2019-01-15 13:18 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-15 15:44 ` Eric Blake
2019-01-15 15:52 ` Vladimir Sementsov-Ogievskiy
2019-01-16 10:40 ` Vladimir Sementsov-Ogievskiy
0 siblings, 2 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-15 15:44 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block
[-- Attachment #1: Type: text/plain, Size: 7458 bytes --]
On 1/15/19 7:18 AM, Vladimir Sementsov-Ogievskiy wrote:
> 12.01.2019 20:58, Eric Blake wrote:
>> Refactor nbd_negotiate_simple_meta_context() to pull out the
>> code that can be reused to send a LIST request for 0 or 1 query.
>> No semantic change. The old comment about 'sizeof(uint32_t)'
>> being equivalent to '/* number of queries */' is no longer
>> needed, now that we are computing 'sizeof(queries)' instead.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> Message-Id: <20181215135324.152629-14-eblake@redhat.com>
>> Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
>>
>> ---
>> v3: Improve commit message [Rich], formatting tweak [checkpatch],
>> rebase to dropped patch
>> ---
>> +++ b/nbd/client.c
>> @@ -629,6 +629,49 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
>> return QIO_CHANNEL(tioc);
>> }
>>
>> +/*
>> + * nbd_send_one_meta_context:
>> + * Send 0 or 1 set/list meta context queries.
>> + * Return 0 on success, -1 with errp set for any error
>> + */
>> +static int nbd_send_one_meta_context(QIOChannel *ioc,
>> + uint32_t opt,
>> + const char *export,
>> + const char *query,
>> + Error **errp)
>> +{
>> + int ret;
>> + uint32_t export_len = strlen(export);
>> + uint32_t queries = !!query;
>
> n_ or nb_ prefix may make it more clear
>
>> + uint32_t context_len = 0;
>> + uint32_t data_len;
>> + char *data;
>> + char *p;
>> +
>> + data_len = sizeof(export_len) + export_len + sizeof(queries);
[1]
>> + if (query) {
>> + context_len = strlen(query);
>
> looks like it then should be query_len
Sure, an alternative name may make things easier to read (I think this
is somewhat fallout from my rebase churn, where earlier versions of the
patch shared as much code with NBD_OPT_SET_META_CONTEXT, and that code
used the name 'context' rather than 'query'; but now that I've split
things to add a new function, it doesn't have to maintain the old naming).
>
>> + data_len += sizeof(context_len) + context_len;
>> + } else {
>> + assert(opt == NBD_OPT_LIST_META_CONTEXT);
>> + }
>> + data = g_malloc(data_len);
>> + p = data;
>
> may use p = data = g_malloc
Will do.
>
>> +
>> + trace_nbd_opt_meta_request(nbd_opt_lookup(opt), query ?: "(all)", export);
[2]
>> + stl_be_p(p, export_len);
>> + memcpy(p += sizeof(export_len), export, export_len);
>> + stl_be_p(p += export_len, queries);
>> + if (query) {
>> + stl_be_p(p += sizeof(uint32_t), context_len);
>
> :), aha, please, s/uint32_t/queries, as you promised
I did up at [1], but should indeed do so again here.
>
> Hmm, its my code. It's hard to read and not very comfortable to maintain..
>
> In block/nbd-client.c we have
> payload_advance* functions, to read such formatted data, I think, it should be
> good to make something like this for server-part. Not about these series, of course.
Yes, I wouldn't object to even more cleanups to make the code easier to
maintain, but not as part of this series.
>
> Interesting, troubles around "don't use be64_to_cpuS, use only be64_to_cpu",
> do they apply somehow to *_be_p functions family?
The problem was in the in-place conversion routines where the
destination type was strongly typed to something wider than char*. This
is not an inplace conversion, because st*_p takes a raw pointer
interpreted as char* as its destination. So no, clang does not have
problems with this construct.
>
>> + memcpy(p += sizeof(context_len), query, context_len);
>> + }
>> +
>> + ret = nbd_send_option_request(ioc, opt, data_len, data, errp);
>> + g_free(data);
>> + return ret;
>> +}
>> +
>> /* nbd_negotiate_simple_meta_context:
>> * Request the server to set the meta context for export @info->name
>> * using @info->x_dirty_bitmap with a fallback to "base:allocation",
>> @@ -653,26 +696,10 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
>> NBDOptionReply reply;
>> const char *context = info->x_dirty_bitmap ?: "base:allocation";
>> bool received = false;
>> - uint32_t export_len = strlen(info->name);
>> - uint32_t context_len = strlen(context);
>> - uint32_t data_len = sizeof(export_len) + export_len +
>> - sizeof(uint32_t) + /* number of queries */
>> - sizeof(context_len) + context_len;
>> - char *data = g_malloc(data_len);
>> - char *p = data;
>>
>> - trace_nbd_opt_meta_request(context, info->name);
>> - stl_be_p(p, export_len);
>> - memcpy(p += sizeof(export_len), info->name, export_len);
>> - stl_be_p(p += export_len, 1);
>> - stl_be_p(p += sizeof(uint32_t), context_len);
>> - memcpy(p += sizeof(context_len), context, context_len);
>> -
>> - ret = nbd_send_option_request(ioc, NBD_OPT_SET_META_CONTEXT, data_len, data,
>> - errp);
>> - g_free(data);
>> - if (ret < 0) {
>> - return ret;
>> + if (nbd_send_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
>> + info->name, context, errp) < 0) {
>> + return -1;
>> }
>>
>> if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
>> @@ -689,7 +716,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
>> if (reply.type == NBD_REP_META_CONTEXT) {
>> char *name;
>>
>> - if (reply.length != sizeof(info->context_id) + context_len) {
>> + if (reply.length != sizeof(info->context_id) + strlen(context)) {
>> error_setg(errp, "Failed to negotiate meta context '%s', server "
>> "answered with unexpected length %" PRIu32, context,
>> reply.length);
>> diff --git a/nbd/trace-events b/nbd/trace-events
>> index c3966d2b653..59521e47a3d 100644
>> --- a/nbd/trace-events
>> +++ b/nbd/trace-events
>> @@ -12,7 +12,7 @@ nbd_receive_query_exports_start(const char *wantname) "Querying export list for
>> nbd_receive_query_exports_success(const char *wantname) "Found desired export name '%s'"
>> nbd_receive_starttls_new_client(void) "Setting up TLS"
>> nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
>> -nbd_opt_meta_request(const char *context, const char *export) "Requesting to set meta context %s for export %s"
>> +nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
>
> Hmm, you forget nbd_opt_lookup()
Where? The updated trace point at [2] has nbd_opt_lookup() for
determining optname.
>
> With this and s/uint32_t/queries (other tiny things up to you):
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>
>> nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
>> nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
>> nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
>>
>
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/19] nbd/client: Split out nbd_send_one_meta_context()
2019-01-15 15:44 ` Eric Blake
@ 2019-01-15 15:52 ` Vladimir Sementsov-Ogievskiy
2019-01-15 15:55 ` Eric Blake
2019-01-16 10:40 ` Vladimir Sementsov-Ogievskiy
1 sibling, 1 reply; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-15 15:52 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
15.01.2019 18:44, Eric Blake wrote:
> On 1/15/19 7:18 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 12.01.2019 20:58, Eric Blake wrote:
>>> Refactor nbd_negotiate_simple_meta_context() to pull out the
>>> code that can be reused to send a LIST request for 0 or 1 query.
>>> No semantic change. The old comment about 'sizeof(uint32_t)'
>>> being equivalent to '/* number of queries */' is no longer
>>> needed, now that we are computing 'sizeof(queries)' instead.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>> Message-Id: <20181215135324.152629-14-eblake@redhat.com>
>>> Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
>>>
>>> ---
>>> v3: Improve commit message [Rich], formatting tweak [checkpatch],
>>> rebase to dropped patch
>>> ---
>
>>> +++ b/nbd/client.c
>>> @@ -629,6 +629,49 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
>>> return QIO_CHANNEL(tioc);
>>> }
>>>
>>> +/*
>>> + * nbd_send_one_meta_context:
>>> + * Send 0 or 1 set/list meta context queries.
>>> + * Return 0 on success, -1 with errp set for any error
>>> + */
>>> +static int nbd_send_one_meta_context(QIOChannel *ioc,
>>> + uint32_t opt,
>>> + const char *export,
>>> + const char *query,
>>> + Error **errp)
>>> +{
>>> + int ret;
>>> + uint32_t export_len = strlen(export);
>>> + uint32_t queries = !!query;
>>
>> n_ or nb_ prefix may make it more clear
>>
>>> + uint32_t context_len = 0;
>>> + uint32_t data_len;
>>> + char *data;
>>> + char *p;
>>> +
>>> + data_len = sizeof(export_len) + export_len + sizeof(queries);
>
> [1]
>
>>> + if (query) {
>>> + context_len = strlen(query);
>>
>> looks like it then should be query_len
>
> Sure, an alternative name may make things easier to read (I think this
> is somewhat fallout from my rebase churn, where earlier versions of the
> patch shared as much code with NBD_OPT_SET_META_CONTEXT, and that code
> used the name 'context' rather than 'query'; but now that I've split
> things to add a new function, it doesn't have to maintain the old naming).
>
>>
>>> + data_len += sizeof(context_len) + context_len;
>>> + } else {
>>> + assert(opt == NBD_OPT_LIST_META_CONTEXT);
>>> + }
>>> + data = g_malloc(data_len);
>>> + p = data;
>>
>> may use p = data = g_malloc
>
> Will do.
>
>>
>>> +
>>> + trace_nbd_opt_meta_request(nbd_opt_lookup(opt), query ?: "(all)", export);
>
> [2]
>
>>> + stl_be_p(p, export_len);
>>> + memcpy(p += sizeof(export_len), export, export_len);
>>> + stl_be_p(p += export_len, queries);
>>> + if (query) {
>>> + stl_be_p(p += sizeof(uint32_t), context_len);
>>
>> :), aha, please, s/uint32_t/queries, as you promised
>
> I did up at [1], but should indeed do so again here.
>
>>
>> Hmm, its my code. It's hard to read and not very comfortable to maintain..
>>
>> In block/nbd-client.c we have
>> payload_advance* functions, to read such formatted data, I think, it should be
>> good to make something like this for server-part. Not about these series, of course.
>
> Yes, I wouldn't object to even more cleanups to make the code easier to
> maintain, but not as part of this series.
>
>>
>> Interesting, troubles around "don't use be64_to_cpuS, use only be64_to_cpu",
>> do they apply somehow to *_be_p functions family?
>
> The problem was in the in-place conversion routines where the
> destination type was strongly typed to something wider than char*. This
> is not an inplace conversion, because st*_p takes a raw pointer
> interpreted as char* as its destination. So no, clang does not have
> problems with this construct.
>
>>
>>> + memcpy(p += sizeof(context_len), query, context_len);
>>> + }
>>> +
>>> + ret = nbd_send_option_request(ioc, opt, data_len, data, errp);
>>> + g_free(data);
>>> + return ret;
>>> +}
>>> +
>>> /* nbd_negotiate_simple_meta_context:
>>> * Request the server to set the meta context for export @info->name
>>> * using @info->x_dirty_bitmap with a fallback to "base:allocation",
>>> @@ -653,26 +696,10 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
>>> NBDOptionReply reply;
>>> const char *context = info->x_dirty_bitmap ?: "base:allocation";
>>> bool received = false;
>>> - uint32_t export_len = strlen(info->name);
>>> - uint32_t context_len = strlen(context);
>>> - uint32_t data_len = sizeof(export_len) + export_len +
>>> - sizeof(uint32_t) + /* number of queries */
>>> - sizeof(context_len) + context_len;
>>> - char *data = g_malloc(data_len);
>>> - char *p = data;
>>>
>>> - trace_nbd_opt_meta_request(context, info->name);
>>> - stl_be_p(p, export_len);
>>> - memcpy(p += sizeof(export_len), info->name, export_len);
>>> - stl_be_p(p += export_len, 1);
>>> - stl_be_p(p += sizeof(uint32_t), context_len);
>>> - memcpy(p += sizeof(context_len), context, context_len);
>>> -
>>> - ret = nbd_send_option_request(ioc, NBD_OPT_SET_META_CONTEXT, data_len, data,
>>> - errp);
>>> - g_free(data);
>>> - if (ret < 0) {
>>> - return ret;
>>> + if (nbd_send_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
>>> + info->name, context, errp) < 0) {
>>> + return -1;
>>> }
>>>
>>> if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
>>> @@ -689,7 +716,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
>>> if (reply.type == NBD_REP_META_CONTEXT) {
>>> char *name;
>>>
>>> - if (reply.length != sizeof(info->context_id) + context_len) {
>>> + if (reply.length != sizeof(info->context_id) + strlen(context)) {
>>> error_setg(errp, "Failed to negotiate meta context '%s', server "
>>> "answered with unexpected length %" PRIu32, context,
>>> reply.length);
>>> diff --git a/nbd/trace-events b/nbd/trace-events
>>> index c3966d2b653..59521e47a3d 100644
>>> --- a/nbd/trace-events
>>> +++ b/nbd/trace-events
>>> @@ -12,7 +12,7 @@ nbd_receive_query_exports_start(const char *wantname) "Querying export list for
>>> nbd_receive_query_exports_success(const char *wantname) "Found desired export name '%s'"
>>> nbd_receive_starttls_new_client(void) "Setting up TLS"
>>> nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
>>> -nbd_opt_meta_request(const char *context, const char *export) "Requesting to set meta context %s for export %s"
>>> +nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
>>
>> Hmm, you forget nbd_opt_lookup()
>
> Where? The updated trace point at [2] has nbd_opt_lookup() for
> determining optname.
Your morning is my evening)
Yes, it's ok, and in the next patch too. But you usually trace both number + lookup-string,
and here only string. Is there a reason to?
>
>>
>> With this and s/uint32_t/queries (other tiny things up to you):
>> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>
>>> nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
>>> nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
>>> nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
>>>
>>
>>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/19] nbd/client: Split out nbd_send_one_meta_context()
2019-01-15 15:52 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-15 15:55 ` Eric Blake
2019-01-15 15:59 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-15 15:55 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block
[-- Attachment #1: Type: text/plain, Size: 1015 bytes --]
On 1/15/19 9:52 AM, Vladimir Sementsov-Ogievskiy wrote:
>>>> -nbd_opt_meta_request(const char *context, const char *export) "Requesting to set meta context %s for export %s"
>>>> +nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
>>>
>>> Hmm, you forget nbd_opt_lookup()
>>
>> Where? The updated trace point at [2] has nbd_opt_lookup() for
>> determining optname.
>
> Your morning is my evening)
>
> Yes, it's ok, and in the next patch too. But you usually trace both number + lookup-string,
> and here only string. Is there a reason to?
I didn't see any added value in tracing the number, since the trace
point can only happen on one of two opt values (the number is more
important when the opt value can be anything, including one that does
not have a corresponding string name compiled in).
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [Qemu-devel] [PATCH v3 10/19] nbd/client: Split out nbd_send_one_meta_context()
2019-01-15 15:55 ` Eric Blake
@ 2019-01-15 15:59 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-15 15:59 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
15.01.2019 18:55, Eric Blake wrote:
> On 1/15/19 9:52 AM, Vladimir Sementsov-Ogievskiy wrote:
>
>>>>> -nbd_opt_meta_request(const char *context, const char *export) "Requesting to set meta context %s for export %s"
>>>>> +nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
>>>>
>>>> Hmm, you forget nbd_opt_lookup()
>>>
>>> Where? The updated trace point at [2] has nbd_opt_lookup() for
>>> determining optname.
>>
>> Your morning is my evening)
>>
>> Yes, it's ok, and in the next patch too. But you usually trace both number + lookup-string,
>> and here only string. Is there a reason to?
>
> I didn't see any added value in tracing the number, since the trace
> point can only happen on one of two opt values (the number is more
> important when the opt value can be anything, including one that does
> not have a corresponding string name compiled in).
>
OK, agreed.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [Qemu-devel] [PATCH v3 10/19] nbd/client: Split out nbd_send_one_meta_context()
2019-01-15 15:44 ` Eric Blake
2019-01-15 15:52 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-16 10:40 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-16 10:40 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
15.01.2019 18:44, Eric Blake wrote:
> On 1/15/19 7:18 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 12.01.2019 20:58, Eric Blake wrote:
>>> Refactor nbd_negotiate_simple_meta_context() to pull out the
>>> code that can be reused to send a LIST request for 0 or 1 query.
>>> No semantic change. The old comment about 'sizeof(uint32_t)'
>>> being equivalent to '/* number of queries */' is no longer
>>> needed, now that we are computing 'sizeof(queries)' instead.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>> Message-Id: <20181215135324.152629-14-eblake@redhat.com>
>>> Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
>>>
>>> ---
>>> v3: Improve commit message [Rich], formatting tweak [checkpatch],
>>> rebase to dropped patch
>>> ---
>
>>> +++ b/nbd/client.c
>>> @@ -629,6 +629,49 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
>>> return QIO_CHANNEL(tioc);
>>> }
>>>
>>> +/*
>>> + * nbd_send_one_meta_context:
>>> + * Send 0 or 1 set/list meta context queries.
>>> + * Return 0 on success, -1 with errp set for any error
>>> + */
>>> +static int nbd_send_one_meta_context(QIOChannel *ioc,
>>> + uint32_t opt,
>>> + const char *export,
>>> + const char *query,
>>> + Error **errp)
>>> +{
>>> + int ret;
>>> + uint32_t export_len = strlen(export);
>>> + uint32_t queries = !!query;
>>
>> n_ or nb_ prefix may make it more clear
>>
>>> + uint32_t context_len = 0;
>>> + uint32_t data_len;
>>> + char *data;
>>> + char *p;
>>> +
>>> + data_len = sizeof(export_len) + export_len + sizeof(queries);
>
> [1]
>
>>> + if (query) {
>>> + context_len = strlen(query);
>>
>> looks like it then should be query_len
>
> Sure, an alternative name may make things easier to read (I think this
> is somewhat fallout from my rebase churn, where earlier versions of the
> patch shared as much code with NBD_OPT_SET_META_CONTEXT, and that code
> used the name 'context' rather than 'query'; but now that I've split
> things to add a new function, it doesn't have to maintain the old naming).
and if you are going to fix it, note, that function name may be fixed too, as we
don't send context, we send query. Especially in the case when query=NULL,
we send empty query, but what is empty context?
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread
* [Qemu-devel] [PATCH v3 11/19] nbd/client: Split out nbd_receive_one_meta_context()
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (9 preceding siblings ...)
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 10/19] nbd/client: Split out nbd_send_one_meta_context() Eric Blake
@ 2019-01-12 17:58 ` Eric Blake
2019-01-15 15:05 ` Vladimir Sementsov-Ogievskiy
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 12/19] nbd/client: Refactor return of nbd_receive_negotiate() Eric Blake
` (10 subsequent siblings)
21 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:58 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block
Extract portions of nbd_negotiate_simple_meta_context() to
a new function nbd_receive_one_meta_context() that copies the
pattern of nbd_receive_list() for performing the argument
validation of one reply. The error message when the server
replies with more than one context changes slightly, but
that shouldn't happen in the common case.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20181215135324.152629-15-eblake@redhat.com>
---
v3: rebase, without changing into a loop
---
nbd/client.c | 148 +++++++++++++++++++++++++++++------------------
nbd/trace-events | 2 +-
2 files changed, 92 insertions(+), 58 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index 3c716be2719..22505199d3b 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -672,7 +672,86 @@ static int nbd_send_one_meta_context(QIOChannel *ioc,
return ret;
}
-/* nbd_negotiate_simple_meta_context:
+/*
+ * nbd_receive_one_meta_context:
+ * Called in a loop to receive and trace one set/list meta context reply.
+ * Pass non-NULL @name or @id to collect results back to the caller, which
+ * must eventually call g_free().
+ * return 1 if name is set and iteration must continue,
+ * 0 if iteration is complete (including if option is unsupported),
+ * -1 with errp set for any error
+ */
+static int nbd_receive_one_meta_context(QIOChannel *ioc,
+ uint32_t opt,
+ char **name,
+ uint32_t *id,
+ Error **errp)
+{
+ int ret;
+ NBDOptionReply reply;
+ char *local_name = NULL;
+ uint32_t local_id;
+
+ if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0) {
+ return -1;
+ }
+
+ ret = nbd_handle_reply_err(ioc, &reply, errp);
+ if (ret <= 0) {
+ return ret;
+ }
+
+ if (reply.type == NBD_REP_ACK) {
+ if (reply.length != 0) {
+ error_setg(errp, "Unexpected length to ACK response");
+ nbd_send_opt_abort(ioc);
+ return -1;
+ }
+ return 0;
+ } else if (reply.type != NBD_REP_META_CONTEXT) {
+ error_setg(errp, "Unexpected reply type %u (%s), expected %u (%s)",
+ reply.type, nbd_rep_lookup(reply.type),
+ NBD_REP_META_CONTEXT, nbd_rep_lookup(NBD_REP_META_CONTEXT));
+ nbd_send_opt_abort(ioc);
+ return -1;
+ }
+
+ if (reply.length <= sizeof(local_id) ||
+ reply.length > NBD_MAX_BUFFER_SIZE) {
+ error_setg(errp, "Failed to negotiate meta context, server "
+ "answered with unexpected length %" PRIu32,
+ reply.length);
+ nbd_send_opt_abort(ioc);
+ return -1;
+ }
+
+ if (nbd_read(ioc, &local_id, sizeof(local_id), errp) < 0) {
+ return -1;
+ }
+ local_id = be32_to_cpu(local_id);
+
+ reply.length -= sizeof(local_id);
+ local_name = g_malloc(reply.length + 1);
+ if (nbd_read(ioc, local_name, reply.length, errp) < 0) {
+ g_free(local_name);
+ return -1;
+ }
+ local_name[reply.length] = '\0';
+ trace_nbd_opt_meta_reply(nbd_opt_lookup(opt), local_name, local_id);
+
+ if (name) {
+ *name = local_name;
+ } else {
+ g_free(local_name);
+ }
+ if (id) {
+ *id = local_id;
+ }
+ return 1;
+}
+
+/*
+ * nbd_negotiate_simple_meta_context:
* Request the server to set the meta context for export @info->name
* using @info->x_dirty_bitmap with a fallback to "base:allocation",
* setting @info->context_id to the resulting id. Fail if the server
@@ -693,50 +772,21 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
* function should lose the term _simple.
*/
int ret;
- NBDOptionReply reply;
const char *context = info->x_dirty_bitmap ?: "base:allocation";
bool received = false;
+ char *name = NULL;
if (nbd_send_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
info->name, context, errp) < 0) {
return -1;
}
- if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
- errp) < 0)
- {
+ ret = nbd_receive_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
+ &name, &info->context_id, errp);
+ if (ret < 0) {
return -1;
}
-
- ret = nbd_handle_reply_err(ioc, &reply, errp);
- if (ret <= 0) {
- return ret;
- }
-
- if (reply.type == NBD_REP_META_CONTEXT) {
- char *name;
-
- if (reply.length != sizeof(info->context_id) + strlen(context)) {
- error_setg(errp, "Failed to negotiate meta context '%s', server "
- "answered with unexpected length %" PRIu32, context,
- reply.length);
- nbd_send_opt_abort(ioc);
- return -1;
- }
-
- if (nbd_read(ioc, &info->context_id, sizeof(info->context_id),
- errp) < 0) {
- return -1;
- }
- info->context_id = be32_to_cpu(info->context_id);
-
- reply.length -= sizeof(info->context_id);
- name = g_malloc(reply.length + 1);
- if (nbd_read(ioc, name, reply.length, errp) < 0) {
- g_free(name);
- return -1;
- }
- name[reply.length] = '\0';
+ if (ret == 1) {
if (strcmp(context, name)) {
error_setg(errp, "Failed to negotiate meta context '%s', server "
"answered with different context '%s'", context,
@@ -746,36 +796,20 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return -1;
}
g_free(name);
-
- trace_nbd_opt_meta_reply(context, info->context_id);
received = true;
- /* receive NBD_REP_ACK */
- if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
- errp) < 0)
- {
+ ret = nbd_receive_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
+ &name, &info->context_id, errp);
+ if (ret < 0) {
return -1;
}
-
- ret = nbd_handle_reply_err(ioc, &reply, errp);
- if (ret <= 0) {
- return ret;
- }
}
-
- if (reply.type != NBD_REP_ACK) {
- error_setg(errp, "Unexpected reply type %u (%s), expected %u (%s)",
- reply.type, nbd_rep_lookup(reply.type),
- NBD_REP_ACK, nbd_rep_lookup(NBD_REP_ACK));
+ if (ret != 0) {
+ error_setg(errp, "Server answered with more than one context");
+ g_free(name);
nbd_send_opt_abort(ioc);
return -1;
}
- if (reply.length) {
- error_setg(errp, "Unexpected length to ACK response");
- nbd_send_opt_abort(ioc);
- return -1;
- }
-
return received;
}
diff --git a/nbd/trace-events b/nbd/trace-events
index 59521e47a3d..b4802c1570e 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -13,7 +13,7 @@ nbd_receive_query_exports_success(const char *wantname) "Found desired export na
nbd_receive_starttls_new_client(void) "Setting up TLS"
nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
-nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
+nbd_opt_meta_reply(const char *optname, const char *context, uint32_t id) "Received %s mapping of %s to id %" PRIu32
nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are 0x%" PRIx32
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 11/19] nbd/client: Split out nbd_receive_one_meta_context()
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 11/19] nbd/client: Split out nbd_receive_one_meta_context() Eric Blake
@ 2019-01-15 15:05 ` Vladimir Sementsov-Ogievskiy
2019-01-15 15:50 ` Eric Blake
0 siblings, 1 reply; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-15 15:05 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
12.01.2019 20:58, Eric Blake wrote:
> Extract portions of nbd_negotiate_simple_meta_context() to
> a new function nbd_receive_one_meta_context() that copies the
> pattern of nbd_receive_list() for performing the argument
> validation of one reply. The error message when the server
> replies with more than one context changes slightly, but
> that shouldn't happen in the common case.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Message-Id: <20181215135324.152629-15-eblake@redhat.com>
>
> ---
> v3: rebase, without changing into a loop
> ---
> nbd/client.c | 148 +++++++++++++++++++++++++++++------------------
> nbd/trace-events | 2 +-
> 2 files changed, 92 insertions(+), 58 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 3c716be2719..22505199d3b 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -672,7 +672,86 @@ static int nbd_send_one_meta_context(QIOChannel *ioc,
> return ret;
> }
>
> -/* nbd_negotiate_simple_meta_context:
> +/*
> + * nbd_receive_one_meta_context:
> + * Called in a loop to receive and trace one set/list meta context reply.
> + * Pass non-NULL @name or @id to collect results back to the caller, which
> + * must eventually call g_free().
> + * return 1 if name is set and iteration must continue,
> + * 0 if iteration is complete (including if option is unsupported),
> + * -1 with errp set for any error
> + */
> +static int nbd_receive_one_meta_context(QIOChannel *ioc,
> + uint32_t opt,
> + char **name,
> + uint32_t *id,
> + Error **errp)
> +{
> + int ret;
> + NBDOptionReply reply;
> + char *local_name = NULL;
> + uint32_t local_id;
> +
> + if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0) {
> + return -1;
> + }
> +
> + ret = nbd_handle_reply_err(ioc, &reply, errp);
> + if (ret <= 0) {
> + return ret;
> + }
> +
> + if (reply.type == NBD_REP_ACK) {
> + if (reply.length != 0) {
> + error_setg(errp, "Unexpected length to ACK response");
> + nbd_send_opt_abort(ioc);
> + return -1;
> + }
> + return 0;
> + } else if (reply.type != NBD_REP_META_CONTEXT) {
> + error_setg(errp, "Unexpected reply type %u (%s), expected %u (%s)",
> + reply.type, nbd_rep_lookup(reply.type),
> + NBD_REP_META_CONTEXT, nbd_rep_lookup(NBD_REP_META_CONTEXT));
> + nbd_send_opt_abort(ioc);
> + return -1;
> + }
> +
> + if (reply.length <= sizeof(local_id) ||
> + reply.length > NBD_MAX_BUFFER_SIZE) {
> + error_setg(errp, "Failed to negotiate meta context, server "
> + "answered with unexpected length %" PRIu32,
> + reply.length);
> + nbd_send_opt_abort(ioc);
> + return -1;
> + }
> +
> + if (nbd_read(ioc, &local_id, sizeof(local_id), errp) < 0) {
> + return -1;
> + }
> + local_id = be32_to_cpu(local_id);
> +
> + reply.length -= sizeof(local_id);
> + local_name = g_malloc(reply.length + 1);
> + if (nbd_read(ioc, local_name, reply.length, errp) < 0) {
> + g_free(local_name);
> + return -1;
> + }
> + local_name[reply.length] = '\0';
> + trace_nbd_opt_meta_reply(nbd_opt_lookup(opt), local_name, local_id);
> +
> + if (name) {
> + *name = local_name;
> + } else {
> + g_free(local_name);
> + }
> + if (id) {
> + *id = local_id;
> + }
> + return 1;
> +}
> +
> +/*
> + * nbd_negotiate_simple_meta_context:
> * Request the server to set the meta context for export @info->name
> * using @info->x_dirty_bitmap with a fallback to "base:allocation",
> * setting @info->context_id to the resulting id. Fail if the server
> @@ -693,50 +772,21 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> * function should lose the term _simple.
> */
> int ret;
> - NBDOptionReply reply;
> const char *context = info->x_dirty_bitmap ?: "base:allocation";
> bool received = false;
> + char *name = NULL;
>
> if (nbd_send_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
> info->name, context, errp) < 0) {
> return -1;
> }
>
> - if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
> - errp) < 0)
> - {
> + ret = nbd_receive_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
> + &name, &info->context_id, errp);
> + if (ret < 0) {
> return -1;
> }
> -
> - ret = nbd_handle_reply_err(ioc, &reply, errp);
> - if (ret <= 0) {
> - return ret;
> - }
> -
> - if (reply.type == NBD_REP_META_CONTEXT) {
> - char *name;
> -
> - if (reply.length != sizeof(info->context_id) + strlen(context)) {
> - error_setg(errp, "Failed to negotiate meta context '%s', server "
> - "answered with unexpected length %" PRIu32, context,
> - reply.length);
> - nbd_send_opt_abort(ioc);
> - return -1;
> - }
> -
> - if (nbd_read(ioc, &info->context_id, sizeof(info->context_id),
> - errp) < 0) {
> - return -1;
> - }
> - info->context_id = be32_to_cpu(info->context_id);
> -
> - reply.length -= sizeof(info->context_id);
> - name = g_malloc(reply.length + 1);
> - if (nbd_read(ioc, name, reply.length, errp) < 0) {
> - g_free(name);
> - return -1;
> - }
> - name[reply.length] = '\0';
> + if (ret == 1) {
> if (strcmp(context, name)) {
> error_setg(errp, "Failed to negotiate meta context '%s', server "
> "answered with different context '%s'", context,
> @@ -746,36 +796,20 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return -1;
> }
> g_free(name);
> -
> - trace_nbd_opt_meta_reply(context, info->context_id);
> received = true;
>
> - /* receive NBD_REP_ACK */
> - if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
> - errp) < 0)
> - {
> + ret = nbd_receive_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
> + &name, &info->context_id, errp);
indent, and, no reasons to use variables instead of NULL, NULL, as success here is an error
path anyway
> + if (ret < 0) {
> return -1;
> }
> -
> - ret = nbd_handle_reply_err(ioc, &reply, errp);
> - if (ret <= 0) {
> - return ret;
> - }
> }
> -
> - if (reply.type != NBD_REP_ACK) {
> - error_setg(errp, "Unexpected reply type %u (%s), expected %u (%s)",
> - reply.type, nbd_rep_lookup(reply.type),
> - NBD_REP_ACK, nbd_rep_lookup(NBD_REP_ACK));
> + if (ret != 0) {
> + error_setg(errp, "Server answered with more than one context");
> + g_free(name);
and then, this g_free may be dropped too.
> nbd_send_opt_abort(ioc);
> return -1;
> }
> - if (reply.length) {
> - error_setg(errp, "Unexpected length to ACK response");
> - nbd_send_opt_abort(ioc);
> - return -1;
> - }
> -
> return received;
> }
>
> diff --git a/nbd/trace-events b/nbd/trace-events
> index 59521e47a3d..b4802c1570e 100644
> --- a/nbd/trace-events
> +++ b/nbd/trace-events
> @@ -13,7 +13,7 @@ nbd_receive_query_exports_success(const char *wantname) "Found desired export na
> nbd_receive_starttls_new_client(void) "Setting up TLS"
> nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
> nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
> -nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
> +nbd_opt_meta_reply(const char *optname, const char *context, uint32_t id) "Received %s mapping of %s to id %" PRIu32
nbd_opt_lookup
> nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
> nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
> nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are 0x%" PRIx32
>
With at least nbd_opt_lookup:
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 11/19] nbd/client: Split out nbd_receive_one_meta_context()
2019-01-15 15:05 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-15 15:50 ` Eric Blake
2019-01-15 15:53 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-15 15:50 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block
[-- Attachment #1: Type: text/plain, Size: 3395 bytes --]
On 1/15/19 9:05 AM, Vladimir Sementsov-Ogievskiy wrote:
> 12.01.2019 20:58, Eric Blake wrote:
>> Extract portions of nbd_negotiate_simple_meta_context() to
>> a new function nbd_receive_one_meta_context() that copies the
>> pattern of nbd_receive_list() for performing the argument
>> validation of one reply. The error message when the server
>> replies with more than one context changes slightly, but
>> that shouldn't happen in the common case.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> Message-Id: <20181215135324.152629-15-eblake@redhat.com>
>>
>> ---
>> v3: rebase, without changing into a loop
>> ---
>>
>> +static int nbd_receive_one_meta_context(QIOChannel *ioc,
>> + uint32_t opt,
>> + char **name,
>> + uint32_t *id,
>> + Error **errp)
>> +{
>> + reply.length -= sizeof(local_id);
>> + local_name = g_malloc(reply.length + 1);
>> + if (nbd_read(ioc, local_name, reply.length, errp) < 0) {
>> + g_free(local_name);
>> + return -1;
>> + }
>> + local_name[reply.length] = '\0';
>> + trace_nbd_opt_meta_reply(nbd_opt_lookup(opt), local_name, local_id);
>> +
[1]
>> @@ -746,36 +796,20 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
>> return -1;
>> }
>> g_free(name);
>> -
>> - trace_nbd_opt_meta_reply(context, info->context_id);
>> received = true;
>>
>> - /* receive NBD_REP_ACK */
>> - if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
>> - errp) < 0)
>> - {
>> + ret = nbd_receive_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
>> + &name, &info->context_id, errp);
>
> indent, and, no reasons to use variables instead of NULL, NULL, as success here is an error
> path anyway
Not sure how I missed the indent, and good point about not needing &name
or &info->context_id on this second call.
>> +++ b/nbd/trace-events
>> @@ -13,7 +13,7 @@ nbd_receive_query_exports_success(const char *wantname) "Found desired export na
>> nbd_receive_starttls_new_client(void) "Setting up TLS"
>> nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
>> nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
>> -nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
>> +nbd_opt_meta_reply(const char *optname, const char *context, uint32_t id) "Received %s mapping of %s to id %" PRIu32
>
> nbd_opt_lookup
Added at [1], so I'm not sure what you are still wanting here.
>
>> nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
>> nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
>> nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are 0x%" PRIx32
>>
>
> With at least nbd_opt_lookup:
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 11/19] nbd/client: Split out nbd_receive_one_meta_context()
2019-01-15 15:50 ` Eric Blake
@ 2019-01-15 15:53 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-15 15:53 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
15.01.2019 18:50, Eric Blake wrote:
> On 1/15/19 9:05 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 12.01.2019 20:58, Eric Blake wrote:
>>> Extract portions of nbd_negotiate_simple_meta_context() to
>>> a new function nbd_receive_one_meta_context() that copies the
>>> pattern of nbd_receive_list() for performing the argument
>>> validation of one reply. The error message when the server
>>> replies with more than one context changes slightly, but
>>> that shouldn't happen in the common case.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>> Message-Id: <20181215135324.152629-15-eblake@redhat.com>
>>>
>>> ---
>>> v3: rebase, without changing into a loop
>>> ---
>
>>>
>>> +static int nbd_receive_one_meta_context(QIOChannel *ioc,
>>> + uint32_t opt,
>>> + char **name,
>>> + uint32_t *id,
>>> + Error **errp)
>>> +{
>
>>> + reply.length -= sizeof(local_id);
>>> + local_name = g_malloc(reply.length + 1);
>>> + if (nbd_read(ioc, local_name, reply.length, errp) < 0) {
>>> + g_free(local_name);
>>> + return -1;
>>> + }
>>> + local_name[reply.length] = '\0';
>>> + trace_nbd_opt_meta_reply(nbd_opt_lookup(opt), local_name, local_id);
>>> +
>
> [1]
>
>>> @@ -746,36 +796,20 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
>>> return -1;
>>> }
>>> g_free(name);
>>> -
>>> - trace_nbd_opt_meta_reply(context, info->context_id);
>>> received = true;
>>>
>>> - /* receive NBD_REP_ACK */
>>> - if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
>>> - errp) < 0)
>>> - {
>>> + ret = nbd_receive_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
>>> + &name, &info->context_id, errp);
>>
>> indent, and, no reasons to use variables instead of NULL, NULL, as success here is an error
>> path anyway
>
> Not sure how I missed the indent, and good point about not needing &name
> or &info->context_id on this second call.
>
>>> +++ b/nbd/trace-events
>>> @@ -13,7 +13,7 @@ nbd_receive_query_exports_success(const char *wantname) "Found desired export na
>>> nbd_receive_starttls_new_client(void) "Setting up TLS"
>>> nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
>>> nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
>>> -nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
>>> +nbd_opt_meta_reply(const char *optname, const char *context, uint32_t id) "Received %s mapping of %s to id %" PRIu32
>>
>> nbd_opt_lookup
>
> Added at [1], so I'm not sure what you are still wanting here.
Yes, my fault, it's OK.
>
>>
>>> nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
>>> nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
>>> nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are 0x%" PRIx32
>>>
>>
>> With at least nbd_opt_lookup:
>> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread
* [Qemu-devel] [PATCH v3 12/19] nbd/client: Refactor return of nbd_receive_negotiate()
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (10 preceding siblings ...)
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 11/19] nbd/client: Split out nbd_receive_one_meta_context() Eric Blake
@ 2019-01-12 17:58 ` Eric Blake
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 13/19] nbd/client: Split handshake into two functions Eric Blake
` (9 subsequent siblings)
21 siblings, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:58 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block
The function could only ever return 0 or -EINVAL; make this
clearer by dropping a useless 'fail:' label.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181215135324.152629-16-eblake@redhat.com>
---
nbd/client.c | 51 +++++++++++++++++++++++----------------------------
1 file changed, 23 insertions(+), 28 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index 22505199d3b..fa931fd8e5d 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -818,7 +818,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
NBDExportInfo *info, Error **errp)
{
uint64_t magic;
- int rc;
bool zeroes = true;
bool structured_reply = info->structured_reply;
bool base_allocation = info->base_allocation;
@@ -829,31 +828,30 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
trace_nbd_receive_negotiate_name(info->name);
info->structured_reply = false;
info->base_allocation = false;
- rc = -EINVAL;
if (outioc) {
*outioc = NULL;
}
if (tlscreds && !outioc) {
error_setg(errp, "Output I/O channel required for TLS");
- goto fail;
+ return -EINVAL;
}
if (nbd_read(ioc, &magic, sizeof(magic), errp) < 0) {
error_prepend(errp, "Failed to read initial magic: ");
- goto fail;
+ return -EINVAL;
}
magic = be64_to_cpu(magic);
trace_nbd_receive_negotiate_magic(magic);
if (magic != NBD_INIT_MAGIC) {
error_setg(errp, "Bad initial magic received: 0x%" PRIx64, magic);
- goto fail;
+ return -EINVAL;
}
if (nbd_read(ioc, &magic, sizeof(magic), errp) < 0) {
error_prepend(errp, "Failed to read server magic: ");
- goto fail;
+ return -EINVAL;
}
magic = be64_to_cpu(magic);
trace_nbd_receive_negotiate_magic(magic);
@@ -865,7 +863,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
if (nbd_read(ioc, &globalflags, sizeof(globalflags), errp) < 0) {
error_prepend(errp, "Failed to read server flags: ");
- goto fail;
+ return -EINVAL;
}
globalflags = be16_to_cpu(globalflags);
trace_nbd_receive_negotiate_server_flags(globalflags);
@@ -881,18 +879,18 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
clientflags = cpu_to_be32(clientflags);
if (nbd_write(ioc, &clientflags, sizeof(clientflags), errp) < 0) {
error_prepend(errp, "Failed to send clientflags field: ");
- goto fail;
+ return -EINVAL;
}
if (tlscreds) {
if (fixedNewStyle) {
*outioc = nbd_receive_starttls(ioc, tlscreds, hostname, errp);
if (!*outioc) {
- goto fail;
+ return -EINVAL;
}
ioc = *outioc;
} else {
error_setg(errp, "Server does not support STARTTLS");
- goto fail;
+ return -EINVAL;
}
}
if (fixedNewStyle) {
@@ -903,7 +901,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
NBD_OPT_STRUCTURED_REPLY,
errp);
if (result < 0) {
- goto fail;
+ return -EINVAL;
}
info->structured_reply = result == 1;
}
@@ -911,7 +909,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
if (info->structured_reply && base_allocation) {
result = nbd_negotiate_simple_meta_context(ioc, info, errp);
if (result < 0) {
- goto fail;
+ return -EINVAL;
}
info->base_allocation = result == 1;
}
@@ -923,7 +921,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
* export, then use NBD_OPT_EXPORT_NAME. */
result = nbd_opt_go(ioc, info, errp);
if (result < 0) {
- goto fail;
+ return -EINVAL;
}
if (result > 0) {
return 0;
@@ -935,25 +933,25 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
* export name is not available.
*/
if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
- goto fail;
+ return -EINVAL;
}
}
/* write the export name request */
if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, info->name,
errp) < 0) {
- goto fail;
+ return -EINVAL;
}
/* Read the response */
if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
error_prepend(errp, "Failed to read export length: ");
- goto fail;
+ return -EINVAL;
}
info->size = be64_to_cpu(info->size);
if (nbd_read(ioc, &info->flags, sizeof(info->flags), errp) < 0) {
error_prepend(errp, "Failed to read export flags: ");
- goto fail;
+ return -EINVAL;
}
info->flags = be16_to_cpu(info->flags);
} else if (magic == NBD_CLIENT_MAGIC) {
@@ -961,43 +959,40 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
if (*info->name) {
error_setg(errp, "Server does not support non-empty export names");
- goto fail;
+ return -EINVAL;
}
if (tlscreds) {
error_setg(errp, "Server does not support STARTTLS");
- goto fail;
+ return -EINVAL;
}
if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
error_prepend(errp, "Failed to read export length: ");
- goto fail;
+ return -EINVAL;
}
info->size = be64_to_cpu(info->size);
if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
error_prepend(errp, "Failed to read export flags: ");
- goto fail;
+ return -EINVAL;
}
oldflags = be32_to_cpu(oldflags);
if (oldflags & ~0xffff) {
error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
- goto fail;
+ return -EINVAL;
}
info->flags = oldflags;
} else {
error_setg(errp, "Bad server magic received: 0x%" PRIx64, magic);
- goto fail;
+ return -EINVAL;
}
trace_nbd_receive_negotiate_size_flags(info->size, info->flags);
if (zeroes && nbd_drop(ioc, 124, errp) < 0) {
error_prepend(errp, "Failed to read reserved block: ");
- goto fail;
+ return -EINVAL;
}
- rc = 0;
-
-fail:
- return rc;
+ return 0;
}
#ifdef __linux__
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* [Qemu-devel] [PATCH v3 13/19] nbd/client: Split handshake into two functions
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (11 preceding siblings ...)
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 12/19] nbd/client: Refactor return of nbd_receive_negotiate() Eric Blake
@ 2019-01-12 17:58 ` Eric Blake
2019-01-15 15:34 ` Vladimir Sementsov-Ogievskiy
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 14/19] nbd/client: Pull out oldstyle size determination Eric Blake
` (8 subsequent siblings)
21 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:58 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block
An upcoming patch will add the ability for qemu-nbd to list
the services provided by an NBD server. Share the common
code of the TLS handshake by splitting the initial exchange
into a separate function, leaving only the export handling
in the original function. Functionally, there should be no
change in behavior in this patch, although some of the code
motion may be difficult to follow due to indentation changes
(view with 'git diff -w' for a smaller changeset).
I considered an enum for the return code coordinating state
between the two functions, but in the end just settled with
ample comments.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181215135324.152629-17-eblake@redhat.com>
---
nbd/client.c | 144 +++++++++++++++++++++++++++++++----------------
nbd/trace-events | 2 +-
2 files changed, 95 insertions(+), 51 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index fa931fd8e5d..5053433ea5e 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -813,21 +813,24 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return received;
}
-int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
- const char *hostname, QIOChannel **outioc,
- NBDExportInfo *info, Error **errp)
+/*
+ * nbd_start_negotiate:
+ * Start the handshake to the server. After a positive return, the server
+ * is ready to accept additional NBD_OPT requests.
+ * Returns: negative errno: failure talking to server
+ * 0: server is oldstyle, client must still parse export size
+ * 1: server is newstyle, but can only accept EXPORT_NAME
+ * 2: server is newstyle, but lacks structured replies
+ * 3: server is newstyle and set up for structured replies
+ */
+static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, QIOChannel **outioc,
+ bool structured_reply, bool *zeroes,
+ Error **errp)
{
uint64_t magic;
- bool zeroes = true;
- bool structured_reply = info->structured_reply;
- bool base_allocation = info->base_allocation;
- trace_nbd_receive_negotiate(tlscreds, hostname ? hostname : "<null>");
-
- assert(info->name);
- trace_nbd_receive_negotiate_name(info->name);
- info->structured_reply = false;
- info->base_allocation = false;
+ trace_nbd_start_negotiate(tlscreds, hostname ? hostname : "<null>");
if (outioc) {
*outioc = NULL;
@@ -872,7 +875,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
clientflags |= NBD_FLAG_C_FIXED_NEWSTYLE;
}
if (globalflags & NBD_FLAG_NO_ZEROES) {
- zeroes = false;
+ *zeroes = false;
clientflags |= NBD_FLAG_C_NO_ZEROES;
}
/* client requested flags */
@@ -894,7 +897,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
}
}
if (fixedNewStyle) {
- int result;
+ int result = 0;
if (structured_reply) {
result = nbd_request_simple_option(ioc,
@@ -903,39 +906,85 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
if (result < 0) {
return -EINVAL;
}
- info->structured_reply = result == 1;
}
+ return 2 + result;
+ } else {
+ return 1;
+ }
+ } else if (magic == NBD_CLIENT_MAGIC) {
+ if (tlscreds) {
+ error_setg(errp, "Server does not support STARTTLS");
+ return -EINVAL;
+ }
+ return 0;
+ } else {
+ error_setg(errp, "Bad server magic received: 0x%" PRIx64, magic);
+ return -EINVAL;
+ }
+}
- if (info->structured_reply && base_allocation) {
- result = nbd_negotiate_simple_meta_context(ioc, info, errp);
- if (result < 0) {
- return -EINVAL;
- }
- info->base_allocation = result == 1;
- }
+/*
+ * nbd_receive_negotiate:
+ * Connect to server, complete negotiation, and move into transmission phase.
+ * Returns: negative errno: failure talking to server
+ * 0: server is connected
+ */
+int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, QIOChannel **outioc,
+ NBDExportInfo *info, Error **errp)
+{
+ int result;
+ bool zeroes = true;
+ bool base_allocation = info->base_allocation;
+ uint32_t oldflags;
+
+ assert(info->name);
+ trace_nbd_receive_negotiate_name(info->name);
+
+ result = nbd_start_negotiate(ioc, tlscreds, hostname, outioc,
+ info->structured_reply, &zeroes, errp);
+
+ info->structured_reply = false;
+ info->base_allocation = false;
+ if (tlscreds && *outioc) {
+ ioc = *outioc;
+ }
- /* Try NBD_OPT_GO first - if it works, we are done (it
- * also gives us a good message if the server requires
- * TLS). If it is not available, fall back to
- * NBD_OPT_LIST for nicer error messages about a missing
- * export, then use NBD_OPT_EXPORT_NAME. */
- result = nbd_opt_go(ioc, info, errp);
+ switch (result) {
+ case 3: /* newstyle, with structured replies */
+ info->structured_reply = true;
+ if (base_allocation) {
+ result = nbd_negotiate_simple_meta_context(ioc, info, errp);
if (result < 0) {
return -EINVAL;
}
- if (result > 0) {
- return 0;
- }
- /* Check our desired export is present in the
- * server export list. Since NBD_OPT_EXPORT_NAME
- * cannot return an error message, running this
- * query gives us better error reporting if the
- * export name is not available.
- */
- if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
- return -EINVAL;
- }
+ info->base_allocation = result == 1;
}
+ /* fall through */
+ case 2: /* newstyle, try OPT_GO */
+ /* Try NBD_OPT_GO first - if it works, we are done (it
+ * also gives us a good message if the server requires
+ * TLS). If it is not available, fall back to
+ * NBD_OPT_LIST for nicer error messages about a missing
+ * export, then use NBD_OPT_EXPORT_NAME. */
+ result = nbd_opt_go(ioc, info, errp);
+ if (result < 0) {
+ return -EINVAL;
+ }
+ if (result > 0) {
+ return 0;
+ }
+ /* Check our desired export is present in the
+ * server export list. Since NBD_OPT_EXPORT_NAME
+ * cannot return an error message, running this
+ * query gives us better error reporting if the
+ * export name is not available.
+ */
+ if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
+ return -EINVAL;
+ }
+ /* fall through */
+ case 1: /* newstyle, but limited to EXPORT_NAME */
/* write the export name request */
if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, info->name,
errp) < 0) {
@@ -954,17 +1003,12 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
return -EINVAL;
}
info->flags = be16_to_cpu(info->flags);
- } else if (magic == NBD_CLIENT_MAGIC) {
- uint32_t oldflags;
-
+ break;
+ case 0: /* oldstyle, parse length and flags */
if (*info->name) {
error_setg(errp, "Server does not support non-empty export names");
return -EINVAL;
}
- if (tlscreds) {
- error_setg(errp, "Server does not support STARTTLS");
- return -EINVAL;
- }
if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
error_prepend(errp, "Failed to read export length: ");
@@ -982,9 +1026,9 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
return -EINVAL;
}
info->flags = oldflags;
- } else {
- error_setg(errp, "Bad server magic received: 0x%" PRIx64, magic);
- return -EINVAL;
+ break;
+ default:
+ return result;
}
trace_nbd_receive_negotiate_size_flags(info->size, info->flags);
diff --git a/nbd/trace-events b/nbd/trace-events
index b4802c1570e..663d11687ab 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -14,7 +14,7 @@ nbd_receive_starttls_new_client(void) "Setting up TLS"
nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
nbd_opt_meta_reply(const char *optname, const char *context, uint32_t id) "Received %s mapping of %s to id %" PRIu32
-nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
+nbd_start_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are 0x%" PRIx32
nbd_receive_negotiate_name(const char *name) "Requesting NBD export name '%s'"
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 13/19] nbd/client: Split handshake into two functions
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 13/19] nbd/client: Split handshake into two functions Eric Blake
@ 2019-01-15 15:34 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-15 15:34 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
12.01.2019 20:58, Eric Blake wrote:
> An upcoming patch will add the ability for qemu-nbd to list
> the services provided by an NBD server. Share the common
> code of the TLS handshake by splitting the initial exchange
> into a separate function, leaving only the export handling
> in the original function. Functionally, there should be no
> change in behavior in this patch, although some of the code
> motion may be difficult to follow due to indentation changes
> (view with 'git diff -w' for a smaller changeset).
>
> I considered an enum for the return code coordinating state
> between the two functions, but in the end just settled with
> ample comments.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Message-Id: <20181215135324.152629-17-eblake@redhat.com>
> ---
> nbd/client.c | 144 +++++++++++++++++++++++++++++++----------------
> nbd/trace-events | 2 +-
> 2 files changed, 95 insertions(+), 51 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index fa931fd8e5d..5053433ea5e 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -813,21 +813,24 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return received;
> }
>
> -int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> - const char *hostname, QIOChannel **outioc,
> - NBDExportInfo *info, Error **errp)
> +/*
> + * nbd_start_negotiate:
> + * Start the handshake to the server. After a positive return, the server
> + * is ready to accept additional NBD_OPT requests.
+ * @zeroes must be set to true before call !!!
> + * Returns: negative errno: failure talking to server
> + * 0: server is oldstyle, client must still parse export size
> + * 1: server is newstyle, but can only accept EXPORT_NAME
> + * 2: server is newstyle, but lacks structured replies
> + * 3: server is newstyle and set up for structured replies
> + */
> +static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> + const char *hostname, QIOChannel **outioc,
> + bool structured_reply, bool *zeroes,
> + Error **errp)
> {
> uint64_t magic;
> - bool zeroes = true;
> - bool structured_reply = info->structured_reply;
> - bool base_allocation = info->base_allocation;
>
> - trace_nbd_receive_negotiate(tlscreds, hostname ? hostname : "<null>");
> -
> - assert(info->name);
> - trace_nbd_receive_negotiate_name(info->name);
> - info->structured_reply = false;
> - info->base_allocation = false;
> + trace_nbd_start_negotiate(tlscreds, hostname ? hostname : "<null>");
or, a lot better:
+ *zeroes = true
>
> if (outioc) {
> *outioc = NULL;
> @@ -872,7 +875,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> clientflags |= NBD_FLAG_C_FIXED_NEWSTYLE;
> }
> if (globalflags & NBD_FLAG_NO_ZEROES) {
> - zeroes = false;
> + *zeroes = false;
> clientflags |= NBD_FLAG_C_NO_ZEROES;
> }
> /* client requested flags */
[..]
> +/*
> + * nbd_receive_negotiate:
> + * Connect to server, complete negotiation, and move into transmission phase.
> + * Returns: negative errno: failure talking to server
> + * 0: server is connected
> + */
> +int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> + const char *hostname, QIOChannel **outioc,
> + NBDExportInfo *info, Error **errp)
> +{
> + int result;
> + bool zeroes = true;
and then, this initialization may be dropped (or left as is, if gcc like it)
> + bool base_allocation = info->base_allocation;
> + uint32_t oldflags;
> +
> + assert(info->name);
> + trace_nbd_receive_negotiate_name(info->name);
> +
> + result = nbd_start_negotiate(ioc, tlscreds, hostname, outioc,
> + info->structured_reply, &zeroes, errp);
> +
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread
* [Qemu-devel] [PATCH v3 14/19] nbd/client: Pull out oldstyle size determination
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (12 preceding siblings ...)
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 13/19] nbd/client: Split handshake into two functions Eric Blake
@ 2019-01-12 17:58 ` Eric Blake
2019-01-15 15:35 ` Vladimir Sementsov-Ogievskiy
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 15/19] nbd/client: Add nbd_receive_export_list() Eric Blake
` (7 subsequent siblings)
21 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:58 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block
Another refactoring creating nbd_negotiate_finish_oldstyle()
for further reuse during 'qemu-nbd --list'.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20181215135324.152629-18-eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
---
nbd/client.c | 49 ++++++++++++++++++++++++++++++++-----------------
1 file changed, 32 insertions(+), 17 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index 5053433ea5e..620fbb5ef01 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -818,7 +818,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
* Start the handshake to the server. After a positive return, the server
* is ready to accept additional NBD_OPT requests.
* Returns: negative errno: failure talking to server
- * 0: server is oldstyle, client must still parse export size
+ * 0: server is oldstyle, must call nbd_negotiate_finish_oldstyle
* 1: server is newstyle, but can only accept EXPORT_NAME
* 2: server is newstyle, but lacks structured replies
* 3: server is newstyle and set up for structured replies
@@ -923,6 +923,36 @@ static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
}
}
+/*
+ * nbd_negotiate_finish_oldstyle:
+ * Populate @info with the size and export flags from an oldstyle server,
+ * but does not consume 124 bytes of reserved zero padding.
+ * Returns 0 on success, -1 with @errp set on failure
+ */
+static int nbd_negotiate_finish_oldstyle(QIOChannel *ioc, NBDExportInfo *info,
+ Error **errp)
+{
+ uint32_t oldflags;
+
+ if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
+ error_prepend(errp, "Failed to read export length: ");
+ return -EINVAL;
+ }
+ info->size = be64_to_cpu(info->size);
+
+ if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
+ error_prepend(errp, "Failed to read export flags: ");
+ return -EINVAL;
+ }
+ oldflags = be32_to_cpu(oldflags);
+ if (oldflags & ~0xffff) {
+ error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
+ return -EINVAL;
+ }
+ info->flags = oldflags;
+ return 0;
+}
+
/*
* nbd_receive_negotiate:
* Connect to server, complete negotiation, and move into transmission phase.
@@ -936,7 +966,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
int result;
bool zeroes = true;
bool base_allocation = info->base_allocation;
- uint32_t oldflags;
assert(info->name);
trace_nbd_receive_negotiate_name(info->name);
@@ -1009,23 +1038,9 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
error_setg(errp, "Server does not support non-empty export names");
return -EINVAL;
}
-
- if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
- error_prepend(errp, "Failed to read export length: ");
+ if (nbd_negotiate_finish_oldstyle(ioc, info, errp) < 0) {
return -EINVAL;
}
- info->size = be64_to_cpu(info->size);
-
- if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
- error_prepend(errp, "Failed to read export flags: ");
- return -EINVAL;
- }
- oldflags = be32_to_cpu(oldflags);
- if (oldflags & ~0xffff) {
- error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
- return -EINVAL;
- }
- info->flags = oldflags;
break;
default:
return result;
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 14/19] nbd/client: Pull out oldstyle size determination
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 14/19] nbd/client: Pull out oldstyle size determination Eric Blake
@ 2019-01-15 15:35 ` Vladimir Sementsov-Ogievskiy
2019-01-15 15:45 ` Vladimir Sementsov-Ogievskiy
2019-01-16 19:47 ` Eric Blake
0 siblings, 2 replies; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-15 15:35 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
12.01.2019 20:58, Eric Blake wrote:
> Another refactoring creating nbd_negotiate_finish_oldstyle()
> for further reuse during 'qemu-nbd --list'.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Message-Id: <20181215135324.152629-18-eblake@redhat.com>
> Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Hmm, looking at this, I found that nbd_start_negotiate() has very unusual semantics
for @zeroes field, without any comment about it:
It must be set to true before calling, and nbd_start_negotiate may set it to true.
I think better is not to rely on caller initialization and set zeroes to true at the
beginning of nbd_start_negotiate().
> ---
> nbd/client.c | 49 ++++++++++++++++++++++++++++++++-----------------
> 1 file changed, 32 insertions(+), 17 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 5053433ea5e..620fbb5ef01 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -818,7 +818,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> * Start the handshake to the server. After a positive return, the server
> * is ready to accept additional NBD_OPT requests.
> * Returns: negative errno: failure talking to server
> - * 0: server is oldstyle, client must still parse export size
> + * 0: server is oldstyle, must call nbd_negotiate_finish_oldstyle
> * 1: server is newstyle, but can only accept EXPORT_NAME
> * 2: server is newstyle, but lacks structured replies
> * 3: server is newstyle and set up for structured replies
> @@ -923,6 +923,36 @@ static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> }
> }
>
> +/*
> + * nbd_negotiate_finish_oldstyle:
> + * Populate @info with the size and export flags from an oldstyle server,
> + * but does not consume 124 bytes of reserved zero padding.
> + * Returns 0 on success, -1 with @errp set on failure
> + */
> +static int nbd_negotiate_finish_oldstyle(QIOChannel *ioc, NBDExportInfo *info,
> + Error **errp)
> +{
> + uint32_t oldflags;
> +
> + if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
> + error_prepend(errp, "Failed to read export length: ");
> + return -EINVAL;
> + }
> + info->size = be64_to_cpu(info->size);
> +
> + if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
> + error_prepend(errp, "Failed to read export flags: ");
> + return -EINVAL;
> + }
> + oldflags = be32_to_cpu(oldflags);
> + if (oldflags & ~0xffff) {
> + error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
> + return -EINVAL;
> + }
> + info->flags = oldflags;
> + return 0;
> +}
> +
> /*
> * nbd_receive_negotiate:
> * Connect to server, complete negotiation, and move into transmission phase.
> @@ -936,7 +966,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> int result;
> bool zeroes = true;
> bool base_allocation = info->base_allocation;
> - uint32_t oldflags;
>
> assert(info->name);
> trace_nbd_receive_negotiate_name(info->name);
> @@ -1009,23 +1038,9 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> error_setg(errp, "Server does not support non-empty export names");
> return -EINVAL;
> }
> -
> - if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
> - error_prepend(errp, "Failed to read export length: ");
> + if (nbd_negotiate_finish_oldstyle(ioc, info, errp) < 0) {
> return -EINVAL;
> }
> - info->size = be64_to_cpu(info->size);
> -
> - if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
> - error_prepend(errp, "Failed to read export flags: ");
> - return -EINVAL;
> - }
> - oldflags = be32_to_cpu(oldflags);
> - if (oldflags & ~0xffff) {
> - error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
> - return -EINVAL;
> - }
> - info->flags = oldflags;
> break;
> default:
> return result;
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 14/19] nbd/client: Pull out oldstyle size determination
2019-01-15 15:35 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-15 15:45 ` Vladimir Sementsov-Ogievskiy
2019-01-16 19:47 ` Eric Blake
1 sibling, 0 replies; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-15 15:45 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
15.01.2019 18:35, Vladimir Sementsov-Ogievskiy wrote:
> It must be set to true before calling, and nbd_start_negotiate may set it to true.
set it to false, I wanted to say
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [Qemu-devel] [PATCH v3 14/19] nbd/client: Pull out oldstyle size determination
2019-01-15 15:35 ` Vladimir Sementsov-Ogievskiy
2019-01-15 15:45 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-16 19:47 ` Eric Blake
1 sibling, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-16 19:47 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block
[-- Attachment #1: Type: text/plain, Size: 991 bytes --]
On 1/15/19 9:35 AM, Vladimir Sementsov-Ogievskiy wrote:
> 12.01.2019 20:58, Eric Blake wrote:
>> Another refactoring creating nbd_negotiate_finish_oldstyle()
>> for further reuse during 'qemu-nbd --list'.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> Message-Id: <20181215135324.152629-18-eblake@redhat.com>
>> Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
>
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>
> Hmm, looking at this, I found that nbd_start_negotiate() has very unusual semantics
> for @zeroes field, without any comment about it:
>
> It must be set to true before calling, and nbd_start_negotiate may set it to true.
>
> I think better is not to rely on caller initialization and set zeroes to true at the
> beginning of nbd_start_negotiate().
Makes sense; done in my local tree.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* [Qemu-devel] [PATCH v3 15/19] nbd/client: Add nbd_receive_export_list()
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (13 preceding siblings ...)
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 14/19] nbd/client: Pull out oldstyle size determination Eric Blake
@ 2019-01-12 17:58 ` Eric Blake
2019-01-16 10:15 ` Vladimir Sementsov-Ogievskiy
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 16/19] nbd/client: Add meta contexts to nbd_receive_export_list() Eric Blake
` (6 subsequent siblings)
21 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:58 UTC (permalink / raw)
To: qemu-devel
Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block, Kevin Wolf, Max Reitz
We want to be able to detect whether a given qemu NBD server is
exposing the right export(s) and dirty bitmaps, at least for
regression testing. We could use 'nbd-client -l' from the upstream
NBD project to list exports, but it's annoying to rely on
out-of-tree binaries; furthermore, nbd-client doesn't necessarily
know about all of the qemu NBD extensions. Thus, we plan on adding
a new mode to qemu-nbd that merely sniffs all possible information
from the server during handshake phase, then disconnects and dumps
the information.
This patch adds the low-level client code for grabbing the list
of exports. It benefits from the recent refactoring patches, as
well as a minor tweak of changing nbd_opt_go() to nbd_opt_info_or_go(),
in order to share as much code as possible when it comes to doing
validation of server replies. The resulting information is stored
in an array of NBDExportInfo which has been expanded to any
description string, along with a convenience function for freeing
the list.
Note: a malicious server could exhaust memory of a client by feeding
an unending loop of exports; perhaps we should place a limit on how
many we are willing to receive. But note that a server could
reasonably be serving an export for every file in a large directory,
where an arbitrary limit in the client means we can't list anything
from such a server; the same happens if we just run until the client
fails to malloc() and thus dies by an abort(), where the limit is
no longer arbitrary but determined by available memory. Since the
client is already planning on being short-lived, it's hard to call
this a denial of service attack that would starve off other uses,
so it does not appear to be a security issue.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20181215135324.152629-19-eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
---
v3: mention security (non-)issue in commit message [Rich], formatting
tweaks
---
include/block/nbd.h | 15 ++++-
nbd/client.c | 144 +++++++++++++++++++++++++++++++++++++++++---
nbd/trace-events | 2 +-
3 files changed, 150 insertions(+), 11 deletions(-)
diff --git a/include/block/nbd.h b/include/block/nbd.h
index bdc7ec7195a..e9a442ce7e9 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2016-2017 Red Hat, Inc.
+ * Copyright (C) 2016-2019 Red Hat, Inc.
* Copyright (C) 2005 Anthony Liguori <anthony@codemonkey.ws>
*
* Network Block Device
@@ -262,6 +262,9 @@ struct NBDExportInfo {
/* Set by client before nbd_receive_negotiate() */
bool request_sizes;
char *x_dirty_bitmap;
+
+ /* Set by client before nbd_receive_negotiate(), or by server results
+ * during nbd_receive_export_list() */
char *name; /* must be non-NULL */
/* In-out fields, set by client before nbd_receive_negotiate() and
@@ -269,7 +272,8 @@ struct NBDExportInfo {
bool structured_reply;
bool base_allocation; /* base:allocation context for NBD_CMD_BLOCK_STATUS */
- /* Set by server results during nbd_receive_negotiate() */
+ /* Set by server results during nbd_receive_negotiate() and
+ * nbd_receive_export_list() */
uint64_t size;
uint16_t flags;
uint32_t min_block;
@@ -277,12 +281,19 @@ struct NBDExportInfo {
uint32_t max_block;
uint32_t context_id;
+
+ /* Set by server results during nbd_receive_export_list() */
+ char *description;
};
typedef struct NBDExportInfo NBDExportInfo;
int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
const char *hostname, QIOChannel **outioc,
NBDExportInfo *info, Error **errp);
+void nbd_free_export_list(NBDExportInfo *info, int count);
+int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, NBDExportInfo **info,
+ Error **errp);
int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
Error **errp);
int nbd_send_request(QIOChannel *ioc, NBDRequest *request);
diff --git a/nbd/client.c b/nbd/client.c
index 620fbb5ef01..a5a705a67f7 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -330,11 +330,14 @@ static int nbd_receive_list(QIOChannel *ioc, char **name, char **description,
}
-/* Returns -1 if NBD_OPT_GO proves the export @info->name cannot be
+/*
+ * Returns -1 if NBD_OPT_GO proves the export @info->name cannot be
* used, 0 if NBD_OPT_GO is unsupported (fall back to NBD_OPT_LIST and
* NBD_OPT_EXPORT_NAME in that case), and > 0 if the export is good to
- * go (with the rest of @info populated). */
-static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
+ * go (with the rest of @info populated).
+ */
+static int nbd_opt_info_or_go(QIOChannel *ioc, uint32_t opt,
+ NBDExportInfo *info, Error **errp)
{
NBDOptionReply reply;
uint32_t len = strlen(info->name);
@@ -347,7 +350,8 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
* flags still 0 is a witness of a broken server. */
info->flags = 0;
- trace_nbd_opt_go_start(info->name);
+ assert(opt == NBD_OPT_GO || opt == NBD_OPT_INFO);
+ trace_nbd_opt_go_start(nbd_opt_lookup(opt), info->name);
buf = g_malloc(4 + len + 2 + 2 * info->request_sizes + 1);
stl_be_p(buf, len);
memcpy(buf + 4, info->name, len);
@@ -356,7 +360,7 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
if (info->request_sizes) {
stw_be_p(buf + 4 + len + 2, NBD_INFO_BLOCK_SIZE);
}
- error = nbd_send_option_request(ioc, NBD_OPT_GO,
+ error = nbd_send_option_request(ioc, opt,
4 + len + 2 + 2 * info->request_sizes,
buf, errp);
g_free(buf);
@@ -365,7 +369,7 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
}
while (1) {
- if (nbd_receive_option_reply(ioc, NBD_OPT_GO, &reply, errp) < 0) {
+ if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0) {
return -1;
}
error = nbd_handle_reply_err(ioc, &reply, errp);
@@ -875,7 +879,9 @@ static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
clientflags |= NBD_FLAG_C_FIXED_NEWSTYLE;
}
if (globalflags & NBD_FLAG_NO_ZEROES) {
- *zeroes = false;
+ if (zeroes) {
+ *zeroes = false;
+ }
clientflags |= NBD_FLAG_C_NO_ZEROES;
}
/* client requested flags */
@@ -996,7 +1002,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
* TLS). If it is not available, fall back to
* NBD_OPT_LIST for nicer error messages about a missing
* export, then use NBD_OPT_EXPORT_NAME. */
- result = nbd_opt_go(ioc, info, errp);
+ result = nbd_opt_info_or_go(ioc, NBD_OPT_GO, info, errp);
if (result < 0) {
return -EINVAL;
}
@@ -1054,6 +1060,128 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
return 0;
}
+/* Clean up result of nbd_receive_export_list */
+void nbd_free_export_list(NBDExportInfo *info, int count)
+{
+ int i;
+
+ if (!info) {
+ return;
+ }
+
+ for (i = 0; i < count; i++) {
+ g_free(info[i].name);
+ g_free(info[i].description);
+ }
+ g_free(info);
+}
+
+/*
+ * nbd_receive_export_list:
+ * Query details about a server's exports, then disconnect without
+ * going into transmission phase. Return a count of the exports listed
+ * in @info by the server, or -1 on error. Caller must free @info using
+ * nbd_free_export_list().
+ */
+int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, NBDExportInfo **info,
+ Error **errp)
+{
+ int result;
+ int count = 0;
+ int i;
+ int rc;
+ int ret = -1;
+ NBDExportInfo *array = NULL;
+ QIOChannel *sioc = NULL;
+
+ *info = NULL;
+ result = nbd_start_negotiate(ioc, tlscreds, hostname, &sioc, true, NULL,
+ errp);
+ if (tlscreds && sioc) {
+ ioc = sioc;
+ }
+
+ switch (result) {
+ case 2:
+ case 3:
+ /* newstyle - use NBD_OPT_LIST to populate array, then try
+ * NBD_OPT_INFO on each array member. If structured replies
+ * are enabled, also try NBD_OPT_LIST_META_CONTEXT. */
+ if (nbd_send_option_request(ioc, NBD_OPT_LIST, 0, NULL, errp) < 0) {
+ goto out;
+ }
+ while (1) {
+ char *name;
+ char *desc;
+
+ rc = nbd_receive_list(ioc, &name, &desc, errp);
+ if (rc < 0) {
+ goto out;
+ } else if (rc == 0) {
+ break;
+ }
+ array = g_renew(NBDExportInfo, array, ++count);
+ memset(&array[count - 1], 0, sizeof(*array));
+ array[count - 1].name = name;
+ array[count - 1].description = desc;
+ array[count - 1].structured_reply = result == 3;
+ }
+
+ for (i = 0; i < count; i++) {
+ array[i].request_sizes = true;
+ rc = nbd_opt_info_or_go(ioc, NBD_OPT_INFO, &array[i], errp);
+ if (rc < 0) {
+ goto out;
+ } else if (rc == 0) {
+ /* Pointless to try rest of loop. If OPT_INFO doesn't work,
+ * it's unlikely that meta contexts work either */
+ break;
+ }
+
+ /* TODO: Grab meta contexts */
+ }
+
+ /* Send NBD_OPT_ABORT as a courtesy before hanging up */
+ nbd_send_opt_abort(ioc);
+ break;
+ case 1: /* newstyle, but limited to EXPORT_NAME */
+ error_setg(errp, "Server does not support export lists");
+ /* We can't even send NBD_OPT_ABORT, so merely hang up */
+ goto out;
+ case 0: /* oldstyle, parse length and flags */
+ array = g_new0(NBDExportInfo, 1);
+ array->name = g_strdup("");
+ count = 1;
+
+ if (nbd_negotiate_finish_oldstyle(ioc, array, errp) < 0) {
+ return -EINVAL;
+ }
+
+ /* Send NBD_CMD_DISC as a courtesy to the server, but ignore all
+ * errors now that we have the information we wanted. */
+ if (nbd_drop(ioc, 124, NULL) == 0) {
+ NBDRequest request = { .type = NBD_CMD_DISC };
+
+ nbd_send_request(ioc, &request);
+ }
+ break;
+ default:
+ goto out;
+ }
+
+ *info = array;
+ array = NULL;
+ ret = count;
+
+ out:
+ qio_channel_shutdown(ioc, QIO_CHANNEL_SHUTDOWN_BOTH, NULL);
+ qio_channel_close(ioc, NULL);
+ object_unref(OBJECT(sioc));
+ nbd_free_export_list(array, count);
+ return ret;
+}
+
#ifdef __linux__
int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
Error **errp)
diff --git a/nbd/trace-events b/nbd/trace-events
index 663d11687ab..cbe03da24e9 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -4,7 +4,7 @@ nbd_receive_option_reply(uint32_t option, const char *optname, uint32_t type, co
nbd_server_error_msg(uint32_t err, const char *type, const char *msg) "server reported error 0x%" PRIx32 " (%s) with additional message: %s"
nbd_reply_err_unsup(uint32_t option, const char *name) "server doesn't understand request %" PRIu32 " (%s), attempting fallback"
nbd_receive_list(const char *name, const char *desc) "export list includes '%s', description '%s'"
-nbd_opt_go_start(const char *name) "Attempting NBD_OPT_GO for export '%s'"
+nbd_opt_go_start(const char *opt, const char *name) "Attempting %s for export '%s'"
nbd_opt_go_success(void) "Export is good to go"
nbd_opt_go_info_unknown(int info, const char *name) "Ignoring unknown info %d (%s)"
nbd_opt_go_info_block_size(uint32_t minimum, uint32_t preferred, uint32_t maximum) "Block sizes are 0x%" PRIx32 ", 0x%" PRIx32 ", 0x%" PRIx32
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 15/19] nbd/client: Add nbd_receive_export_list()
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 15/19] nbd/client: Add nbd_receive_export_list() Eric Blake
@ 2019-01-16 10:15 ` Vladimir Sementsov-Ogievskiy
2019-01-16 14:33 ` Eric Blake
2019-01-16 20:01 ` Eric Blake
0 siblings, 2 replies; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-16 10:15 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block, Kevin Wolf, Max Reitz
12.01.2019 20:58, Eric Blake wrote:
> We want to be able to detect whether a given qemu NBD server is
> exposing the right export(s) and dirty bitmaps, at least for
> regression testing. We could use 'nbd-client -l' from the upstream
> NBD project to list exports, but it's annoying to rely on
> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
> know about all of the qemu NBD extensions. Thus, we plan on adding
> a new mode to qemu-nbd that merely sniffs all possible information
> from the server during handshake phase, then disconnects and dumps
> the information.
>
> This patch adds the low-level client code for grabbing the list
> of exports. It benefits from the recent refactoring patches, as
> well as a minor tweak of changing nbd_opt_go() to nbd_opt_info_or_go(),
> in order to share as much code as possible when it comes to doing
> validation of server replies. The resulting information is stored
> in an array of NBDExportInfo which has been expanded to any
> description string, along with a convenience function for freeing
> the list.
>
> Note: a malicious server could exhaust memory of a client by feeding
> an unending loop of exports; perhaps we should place a limit on how
> many we are willing to receive. But note that a server could
> reasonably be serving an export for every file in a large directory,
> where an arbitrary limit in the client means we can't list anything
> from such a server; the same happens if we just run until the client
> fails to malloc() and thus dies by an abort(), where the limit is
> no longer arbitrary but determined by available memory. Since the
> client is already planning on being short-lived, it's hard to call
> this a denial of service attack that would starve off other uses,
> so it does not appear to be a security issue.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Message-Id: <20181215135324.152629-19-eblake@redhat.com>
> Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
>
> ---
> v3: mention security (non-)issue in commit message [Rich], formatting
> tweaks
> ---
[..]
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -330,11 +330,14 @@ static int nbd_receive_list(QIOChannel *ioc, char **name, char **description,
> }
>
>
> -/* Returns -1 if NBD_OPT_GO proves the export @info->name cannot be
> +/*
> + * Returns -1 if NBD_OPT_GO proves the export @info->name cannot be
> * used, 0 if NBD_OPT_GO is unsupported (fall back to NBD_OPT_LIST and
> * NBD_OPT_EXPORT_NAME in that case), and > 0 if the export is good to
> - * go (with the rest of @info populated). */
> -static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
> + * go (with the rest of @info populated).
> + */
Don't you want to upgrade a comment a little bit, to reflect support for OPT_INFO?
> +static int nbd_opt_info_or_go(QIOChannel *ioc, uint32_t opt,
> + NBDExportInfo *info, Error **errp)
> {
> NBDOptionReply reply;
> uint32_t len = strlen(info->name);
> @@ -347,7 +350,8 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
> * flags still 0 is a witness of a broken server. */
> info->flags = 0;
>
> - trace_nbd_opt_go_start(info->name);
> + assert(opt == NBD_OPT_GO || opt == NBD_OPT_INFO);
> + trace_nbd_opt_go_start(nbd_opt_lookup(opt), info->name);
I'd prefer to upgrade trace-point name too, as well as other several trace_nbd_opt_go_* trace
points in the function.
> buf = g_malloc(4 + len + 2 + 2 * info->request_sizes + 1);
> stl_be_p(buf, len);
> memcpy(buf + 4, info->name, len);
[..]
> +/*
> + * nbd_receive_export_list:
> + * Query details about a server's exports, then disconnect without
> + * going into transmission phase. Return a count of the exports listed
> + * in @info by the server, or -1 on error. Caller must free @info using
> + * nbd_free_export_list().
> + */
> +int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> + const char *hostname, NBDExportInfo **info,
> + Error **errp)
> +{
> + int result;
> + int count = 0;
> + int i;
> + int rc;
> + int ret = -1;
> + NBDExportInfo *array = NULL;
> + QIOChannel *sioc = NULL;
> +
> + *info = NULL;
> + result = nbd_start_negotiate(ioc, tlscreds, hostname, &sioc, true, NULL,
> + errp);
> + if (tlscreds && sioc) {
> + ioc = sioc;
> + }
> +
> + switch (result) {
> + case 2:
> + case 3:
> + /* newstyle - use NBD_OPT_LIST to populate array, then try
> + * NBD_OPT_INFO on each array member. If structured replies
> + * are enabled, also try NBD_OPT_LIST_META_CONTEXT. */
> + if (nbd_send_option_request(ioc, NBD_OPT_LIST, 0, NULL, errp) < 0) {
> + goto out;
> + }
> + while (1) {
> + char *name;
> + char *desc;
> +
> + rc = nbd_receive_list(ioc, &name, &desc, errp);
> + if (rc < 0) {
> + goto out;
> + } else if (rc == 0) {
> + break;
> + }
> + array = g_renew(NBDExportInfo, array, ++count);
> + memset(&array[count - 1], 0, sizeof(*array));
> + array[count - 1].name = name;
> + array[count - 1].description = desc;
> + array[count - 1].structured_reply = result == 3;
> + }
> +
> + for (i = 0; i < count; i++) {
> + array[i].request_sizes = true;
> + rc = nbd_opt_info_or_go(ioc, NBD_OPT_INFO, &array[i], errp);
> + if (rc < 0) {
> + goto out;
> + } else if (rc == 0) {
> + /* Pointless to try rest of loop. If OPT_INFO doesn't work,
> + * it's unlikely that meta contexts work either */
> + break;
> + }
> +
> + /* TODO: Grab meta contexts */
> + }
> +
> + /* Send NBD_OPT_ABORT as a courtesy before hanging up */
> + nbd_send_opt_abort(ioc);
> + break;
> + case 1: /* newstyle, but limited to EXPORT_NAME */
> + error_setg(errp, "Server does not support export lists");
> + /* We can't even send NBD_OPT_ABORT, so merely hang up */
But, on the other hand, why not to send it? It will be unknown for the server,
so, it will have to close the connection accordingly to the protocol, isn't it
better a bit?
> + goto out;
> + case 0: /* oldstyle, parse length and flags */
> + array = g_new0(NBDExportInfo, 1);
> + array->name = g_strdup("");
> + count = 1;
> +
> + if (nbd_negotiate_finish_oldstyle(ioc, array, errp) < 0) {
> + return -EINVAL;
goto out, you mean.
And with at least this one fixed:
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 15/19] nbd/client: Add nbd_receive_export_list()
2019-01-16 10:15 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-16 14:33 ` Eric Blake
2019-01-16 20:01 ` Eric Blake
1 sibling, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-16 14:33 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block, Kevin Wolf, Max Reitz
[-- Attachment #1: Type: text/plain, Size: 1956 bytes --]
On 1/16/19 4:15 AM, Vladimir Sementsov-Ogievskiy wrote:
>> @@ -347,7 +350,8 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
>> * flags still 0 is a witness of a broken server. */
>> info->flags = 0;
>>
>> - trace_nbd_opt_go_start(info->name);
>> + assert(opt == NBD_OPT_GO || opt == NBD_OPT_INFO);
>> + trace_nbd_opt_go_start(nbd_opt_lookup(opt), info->name);
>
> I'd prefer to upgrade trace-point name too, as well as other several trace_nbd_opt_go_* trace
> points in the function.
>
Can do.
>> + /* Send NBD_OPT_ABORT as a courtesy before hanging up */
>> + nbd_send_opt_abort(ioc);
>> + break;
>> + case 1: /* newstyle, but limited to EXPORT_NAME */
>> + error_setg(errp, "Server does not support export lists");
>> + /* We can't even send NBD_OPT_ABORT, so merely hang up */
>
> But, on the other hand, why not to send it? It will be unknown for the server,
> so, it will have to close the connection accordingly to the protocol, isn't it
> better a bit?
The NBD spec says that if the server did not advertise fixed newstyle,
then the client must not send any other NBD_OPT. And servers that don't
support fixed newstyle are rather rare, so it doesn't really hurt if we
aren't courteous to them.
>
>> + goto out;
>> + case 0: /* oldstyle, parse length and flags */
>> + array = g_new0(NBDExportInfo, 1);
>> + array->name = g_strdup("");
>> + count = 1;
>> +
>> + if (nbd_negotiate_finish_oldstyle(ioc, array, errp) < 0) {
>> + return -EINVAL;
>
> goto out, you mean.
Indeed. Thanks for spotting it.
> And with at least this one fixed:
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>
>
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 15/19] nbd/client: Add nbd_receive_export_list()
2019-01-16 10:15 ` Vladimir Sementsov-Ogievskiy
2019-01-16 14:33 ` Eric Blake
@ 2019-01-16 20:01 ` Eric Blake
1 sibling, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-16 20:01 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block, Kevin Wolf, Max Reitz
[-- Attachment #1: Type: text/plain, Size: 1431 bytes --]
On 1/16/19 4:15 AM, Vladimir Sementsov-Ogievskiy wrote:
>> This patch adds the low-level client code for grabbing the list
>> of exports. It benefits from the recent refactoring patches, as
>> well as a minor tweak of changing nbd_opt_go() to nbd_opt_info_or_go(),
>> -/* Returns -1 if NBD_OPT_GO proves the export @info->name cannot be
>> +/*
>> + * Returns -1 if NBD_OPT_GO proves the export @info->name cannot be
>> * used, 0 if NBD_OPT_GO is unsupported (fall back to NBD_OPT_LIST and
>> * NBD_OPT_EXPORT_NAME in that case), and > 0 if the export is good to
>> - * go (with the rest of @info populated). */
>> -static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
>> + * go (with the rest of @info populated).
>> + */
>
> Don't you want to upgrade a comment a little bit, to reflect support for OPT_INFO?
Yeah, but then the tweaks start to build up. I'll split the function
rename to a separate patch,
>>
>> - trace_nbd_opt_go_start(info->name);
>> + assert(opt == NBD_OPT_GO || opt == NBD_OPT_INFO);
>> + trace_nbd_opt_go_start(nbd_opt_lookup(opt), info->name);
>
> I'd prefer to upgrade trace-point name too, as well as other several trace_nbd_opt_go_* trace
> points in the function.
and improve the traces there too.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* [Qemu-devel] [PATCH v3 16/19] nbd/client: Add meta contexts to nbd_receive_export_list()
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (14 preceding siblings ...)
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 15/19] nbd/client: Add nbd_receive_export_list() Eric Blake
@ 2019-01-12 17:58 ` Eric Blake
2019-01-16 10:54 ` Vladimir Sementsov-Ogievskiy
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 17/19] qemu-nbd: Add --list option Eric Blake
` (5 subsequent siblings)
21 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:58 UTC (permalink / raw)
To: qemu-devel
Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block, Kevin Wolf, Max Reitz
We want to be able to detect whether a given qemu NBD server is
exposing the right export(s) and dirty bitmaps, at least for
regression testing. We could use 'nbd-client -l' from the upstream
NBD project to list exports, but it's annoying to rely on
out-of-tree binaries; furthermore, nbd-client doesn't necessarily
know about all of the qemu NBD extensions. Thus, we plan on adding
a new mode to qemu-nbd that merely sniffs all possible information
from the server during handshake phase, then disconnects and dumps
the information.
This patch continues the work of the previous patch, by adding the
ability to track the list of available meta contexts into
NBDExportInfo. It benefits from the recent refactoring patches
with a new nbd_list_meta_contexts() that reuses much of the same
framework as setting a meta context.
Note: a malicious server could exhaust memory of a client by feeding
an unending loop of contexts; perhaps we could place a limit on how
many we are willing to receive. But this is no different from our
earlier analysis on a server sending an unending list of exports,
and the death of a client due to memory exhaustion when the client
was going to exit soon anyways is not really a denial of service
attack.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20181215135324.152629-20-eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
---
v3: mention security (non-)issue in commit message [Rich]
---
include/block/nbd.h | 2 ++
nbd/client.c | 41 +++++++++++++++++++++++++++++++++++++++--
2 files changed, 41 insertions(+), 2 deletions(-)
diff --git a/include/block/nbd.h b/include/block/nbd.h
index e9a442ce7e9..80ee3cc997e 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -284,6 +284,8 @@ struct NBDExportInfo {
/* Set by server results during nbd_receive_export_list() */
char *description;
+ int n_contexts;
+ char **contexts;
};
typedef struct NBDExportInfo NBDExportInfo;
diff --git a/nbd/client.c b/nbd/client.c
index a5a705a67f7..2001e6e8160 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -817,6 +817,36 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return received;
}
+/*
+ * nbd_list_meta_contexts:
+ * Request the server to list all meta contexts for export @info->name.
+ * return 0 if list is complete (even if empty),
+ * -1 with errp set for any other error
+ */
+static int nbd_list_meta_contexts(QIOChannel *ioc,
+ NBDExportInfo *info,
+ Error **errp)
+{
+ int ret;
+
+ if (nbd_send_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
+ info->name, NULL, errp) < 0) {
+ return -1;
+ }
+
+ while (1) {
+ char *context;
+
+ ret = nbd_receive_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
+ &context, NULL, errp);
+ if (ret <= 0) {
+ return ret;
+ }
+ info->contexts = g_renew(char *, info->contexts, ++info->n_contexts);
+ info->contexts[info->n_contexts - 1] = context;
+ }
+}
+
/*
* nbd_start_negotiate:
* Start the handshake to the server. After a positive return, the server
@@ -1063,7 +1093,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
/* Clean up result of nbd_receive_export_list */
void nbd_free_export_list(NBDExportInfo *info, int count)
{
- int i;
+ int i, j;
if (!info) {
return;
@@ -1072,6 +1102,10 @@ void nbd_free_export_list(NBDExportInfo *info, int count)
for (i = 0; i < count; i++) {
g_free(info[i].name);
g_free(info[i].description);
+ for (j = 0; j < info[i].n_contexts; j++) {
+ g_free(info[i].contexts[j]);
+ }
+ g_free(info[i].contexts);
}
g_free(info);
}
@@ -1139,7 +1173,10 @@ int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
break;
}
- /* TODO: Grab meta contexts */
+ if (result == 3 &&
+ nbd_list_meta_contexts(ioc, &array[i], errp) < 0) {
+ goto out;
+ }
}
/* Send NBD_OPT_ABORT as a courtesy before hanging up */
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 16/19] nbd/client: Add meta contexts to nbd_receive_export_list()
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 16/19] nbd/client: Add meta contexts to nbd_receive_export_list() Eric Blake
@ 2019-01-16 10:54 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-16 10:54 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block, Kevin Wolf, Max Reitz
12.01.2019 20:58, Eric Blake wrote:
> We want to be able to detect whether a given qemu NBD server is
> exposing the right export(s) and dirty bitmaps, at least for
> regression testing. We could use 'nbd-client -l' from the upstream
> NBD project to list exports, but it's annoying to rely on
> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
> know about all of the qemu NBD extensions. Thus, we plan on adding
> a new mode to qemu-nbd that merely sniffs all possible information
> from the server during handshake phase, then disconnects and dumps
> the information.
>
> This patch continues the work of the previous patch, by adding the
> ability to track the list of available meta contexts into
> NBDExportInfo. It benefits from the recent refactoring patches
> with a new nbd_list_meta_contexts() that reuses much of the same
> framework as setting a meta context.
>
> Note: a malicious server could exhaust memory of a client by feeding
> an unending loop of contexts; perhaps we could place a limit on how
> many we are willing to receive. But this is no different from our
> earlier analysis on a server sending an unending list of exports,
> and the death of a client due to memory exhaustion when the client
> was going to exit soon anyways is not really a denial of service
> attack.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Message-Id: <20181215135324.152629-20-eblake@redhat.com>
> Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
>
> ---
> v3: mention security (non-)issue in commit message [Rich]
> ---
> include/block/nbd.h | 2 ++
> nbd/client.c | 41 +++++++++++++++++++++++++++++++++++++++--
> 2 files changed, 41 insertions(+), 2 deletions(-)
>
> diff --git a/include/block/nbd.h b/include/block/nbd.h
> index e9a442ce7e9..80ee3cc997e 100644
> --- a/include/block/nbd.h
> +++ b/include/block/nbd.h
> @@ -284,6 +284,8 @@ struct NBDExportInfo {
>
> /* Set by server results during nbd_receive_export_list() */
> char *description;
> + int n_contexts;
> + char **contexts;
> };
> typedef struct NBDExportInfo NBDExportInfo;
>
> diff --git a/nbd/client.c b/nbd/client.c
> index a5a705a67f7..2001e6e8160 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -817,6 +817,36 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return received;
> }
>
> +/*
> + * nbd_list_meta_contexts:
> + * Request the server to list all meta contexts for export @info->name.
> + * return 0 if list is complete (even if empty),
> + * -1 with errp set for any other error
Not sure about need of "other" word in this context.
> + */
> +static int nbd_list_meta_contexts(QIOChannel *ioc,
> + NBDExportInfo *info,
> + Error **errp)
> +{
> + int ret;
> +
> + if (nbd_send_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
> + info->name, NULL, errp) < 0) {
Hmm, this made me to add one more comment to previous patch about
nbd_send_one_meta_context, the it don't send context but query.
> + return -1;
> + }
> +
> + while (1) {
> + char *context;
> +
> + ret = nbd_receive_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
> + &context, NULL, errp);
> + if (ret <= 0) {
> + return ret;
> + }
> + info->contexts = g_renew(char *, info->contexts, ++info->n_contexts);
> + info->contexts[info->n_contexts - 1] = context;
> + }
A bit odd for me, that function leaves allocated memory on error path, but it will
be freed anyway in nbd_free_export_list, so, don't mind.
> +}
> +
> /*
> * nbd_start_negotiate:
> * Start the handshake to the server. After a positive return, the server
> @@ -1063,7 +1093,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> /* Clean up result of nbd_receive_export_list */
> void nbd_free_export_list(NBDExportInfo *info, int count)
> {
> - int i;
> + int i, j;
>
> if (!info) {
> return;
> @@ -1072,6 +1102,10 @@ void nbd_free_export_list(NBDExportInfo *info, int count)
> for (i = 0; i < count; i++) {
> g_free(info[i].name);
> g_free(info[i].description);
> + for (j = 0; j < info[i].n_contexts; j++) {
> + g_free(info[i].contexts[j]);
> + }
> + g_free(info[i].contexts);
> }
> g_free(info);
> }
> @@ -1139,7 +1173,10 @@ int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> break;
> }
>
> - /* TODO: Grab meta contexts */
> + if (result == 3 &&
> + nbd_list_meta_contexts(ioc, &array[i], errp) < 0) {
> + goto out;
> + }
> }
>
> /* Send NBD_OPT_ABORT as a courtesy before hanging up */
>
anyway,
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread
* [Qemu-devel] [PATCH v3 17/19] qemu-nbd: Add --list option
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (15 preceding siblings ...)
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 16/19] nbd/client: Add meta contexts to nbd_receive_export_list() Eric Blake
@ 2019-01-12 17:58 ` Eric Blake
2019-01-17 10:05 ` Vladimir Sementsov-Ogievskiy
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 18/19] nbd/client: Work around 3.0 bug for listing meta contexts Eric Blake
` (4 subsequent siblings)
21 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:58 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block
We want to be able to detect whether a given qemu NBD server is
exposing the right export(s) and dirty bitmaps, at least for
regression testing. We could use 'nbd-client -l' from the upstream
NBD project to list exports, but it's annoying to rely on
out-of-tree binaries; furthermore, nbd-client doesn't necessarily
know about all of the qemu NBD extensions. Thus, it is time to add
a new mode to qemu-nbd that merely sniffs all possible information
from the server during handshake phase, then disconnects and dumps
the information.
This patch actually implements --list/-L, while reusing other
options such as --tls-creds for now designating how to connect
as the client (rather than their non-list usage of how to operate
as the server).
I debated about adding this functionality to something akin to
'qemu-img info' - but that tool does not readily lend itself
to connecting to an arbitrary NBD server without also tying to
a specific export (I may, however, still add ImageInfoSpecificNBD
for reporting the bitmaps available when connecting to a single
export). And, while it may feel a bit odd that normally
qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not
really making the qemu-nbd binary that much larger, because
'qemu-nbd -c' has to operate as both server and client
simultaneously across two threads when feeding the kernel module
for /dev/nbdN access.
Sample output:
$ qemu-nbd -L
exports available: 1
export: ''
size: 65536
flags: 0x4ed ( flush fua trim zeroes df cache )
min block: 512
opt block: 4096
max block: 33554432
available meta contexts: 1
base:allocation
Note that the output only lists sizes if the server sent
NBD_FLAG_HAS_FLAGS, because a newstyle server does not give
the size otherwise. It has the side effect that for really
old servers that did not send any flags, the size is not
output even though it was available. However, I'm not too
concerned about that - oldstyle servers are (rightfully)
getting less common to encounter (qemu 3.0 was the last
version where we even serve it), and most existing servers
that still even offer oldstyle negotiation (such as nbdkit)
still send flags (since that was added to the NBD protocol
in 2007 to permit read-only connections).
Not done here, but maybe worth future experiments: capture
the meat of NBDExportInfo into a QAPI struct, and use the
generated QAPI pretty-printers instead of hand-rolling our
output loop. It would also permit us to add a JSON output
mode for machine parsing.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20181215135324.152629-21-eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
---
v3: comment tweak [Rich], rebase to earlier changes
---
qemu-nbd.texi | 27 +++++++--
qemu-nbd.c | 155 +++++++++++++++++++++++++++++++++++++++++++++-----
2 files changed, 165 insertions(+), 17 deletions(-)
diff --git a/qemu-nbd.texi b/qemu-nbd.texi
index 3f22559beb4..65caeb7874a 100644
--- a/qemu-nbd.texi
+++ b/qemu-nbd.texi
@@ -2,6 +2,8 @@
@c man begin SYNOPSIS
@command{qemu-nbd} [OPTION]... @var{filename}
+@command{qemu-nbd} @option{-L} [OPTION]...
+
@command{qemu-nbd} @option{-d} @var{dev}
@c man end
@end example
@@ -14,6 +16,8 @@ Other uses:
@itemize
@item
Bind a /dev/nbdX block device to a QEMU server (on Linux).
+@item
+As a client to query exports of a remote NBD server.
@end itemize
@c man end
@@ -31,13 +35,15 @@ See the @code{qemu(1)} manual page for full details of the properties
supported. The common object types that it makes sense to define are the
@code{secret} object, which is used to supply passwords and/or encryption
keys, and the @code{tls-creds} object, which is used to supply TLS
-credentials for the qemu-nbd server.
+credentials for the qemu-nbd server or client.
@item -p, --port=@var{port}
-The TCP port to listen on (default @samp{10809}).
+The TCP port to listen on as a server, or connect to as a client
+(default @samp{10809}).
@item -o, --offset=@var{offset}
The offset into the image.
@item -b, --bind=@var{iface}
-The interface to bind to (default @samp{0.0.0.0}).
+The interface to bind to as a server, or connect to as a client
+(default @samp{0.0.0.0}).
@item -k, --socket=@var{path}
Use a unix socket with path @var{path}.
@item --image-opts
@@ -97,10 +103,14 @@ Set the NBD volume export name (default of a zero-length string).
@item -D, --description=@var{description}
Set the NBD volume export description, as a human-readable
string.
+@item -L, --list
+Connect as a client and list all details about the exports exposed by
+a remote NBD server.
@item --tls-creds=ID
Enable mandatory TLS encryption for the server by setting the ID
of the TLS credentials object previously created with the --object
-option.
+option; or provide the credentials needed for connecting as a client
+in list mode.
@item --fork
Fork off the server process and exit the parent once the server is running.
@item -v, --verbose
@@ -159,6 +169,15 @@ qemu-nbd -c /dev/nbd0 -f qcow2 file.qcow2
qemu-nbd -d /dev/nbd0
@end example
+Query a remote server to see details about what export(s) it is
+serving on port 10809, and authenticating via PSK:
+
+@example
+qemu-nbd \
+ --object tls-creds-psk,id=tls0,dir=/tmp/keys,username=eblake,endpoint=client \
+ --tls-creds tls0 -L -b remote.example.com
+@end example
+
@c man end
@ignore
diff --git a/qemu-nbd.c b/qemu-nbd.c
index f1c24683129..daccb86d0d7 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -76,7 +76,8 @@ static void usage(const char *name)
{
(printf) (
"Usage: %s [OPTIONS] FILE\n"
-"QEMU Disk Network Block Device Server\n"
+" or: %s -L [OPTIONS]\n"
+"QEMU Disk Network Block Device Utility\n"
"\n"
" -h, --help display this help and exit\n"
" -V, --version output version information and exit\n"
@@ -98,6 +99,7 @@ static void usage(const char *name)
" -B, --bitmap=NAME expose a persistent dirty bitmap\n"
"\n"
"General purpose options:\n"
+" -L, --list list exports available from another NBD server\n"
" --object type,id=ID,... define an object such as 'secret' for providing\n"
" passwords and/or encryption keys\n"
" --tls-creds=ID use id of an earlier --object to provide TLS\n"
@@ -131,7 +133,7 @@ static void usage(const char *name)
" --image-opts treat FILE as a full set of image options\n"
"\n"
QEMU_HELP_BOTTOM "\n"
- , name, NBD_DEFAULT_PORT, "DEVICE");
+ , name, name, NBD_DEFAULT_PORT, "DEVICE");
}
static void version(const char *name)
@@ -243,6 +245,92 @@ static void termsig_handler(int signum)
}
+static int qemu_nbd_client_list(SocketAddress *saddr, QCryptoTLSCreds *tls,
+ const char *hostname)
+{
+ int ret = EXIT_FAILURE;
+ int rc;
+ Error *err = NULL;
+ QIOChannelSocket *sioc;
+ NBDExportInfo *list;
+ int i, j;
+
+ sioc = qio_channel_socket_new();
+ if (qio_channel_socket_connect_sync(sioc, saddr, &err) < 0) {
+ error_report_err(err);
+ goto out;
+ }
+ rc = nbd_receive_export_list(QIO_CHANNEL(sioc), tls, hostname, &list,
+ &err);
+ if (rc < 0) {
+ if (err) {
+ error_report_err(err);
+ }
+ goto out_socket;
+ }
+ printf("exports available: %d\n", rc);
+ for (i = 0; i < rc; i++) {
+ printf(" export: '%s'\n", list[i].name);
+ if (list[i].description && *list[i].description) {
+ printf(" description: %s\n", list[i].description);
+ }
+ if (list[i].flags & NBD_FLAG_HAS_FLAGS) {
+ printf(" size: %" PRIu64 "\n", list[i].size);
+ printf(" flags: 0x%x (", list[i].flags);
+ if (list[i].flags & NBD_FLAG_READ_ONLY) {
+ printf(" readonly");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_FLUSH) {
+ printf(" flush");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_FUA) {
+ printf(" fua");
+ }
+ if (list[i].flags & NBD_FLAG_ROTATIONAL) {
+ printf(" rotational");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_TRIM) {
+ printf(" trim");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_WRITE_ZEROES) {
+ printf(" zeroes");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_DF) {
+ printf(" df");
+ }
+ if (list[i].flags & NBD_FLAG_CAN_MULTI_CONN) {
+ printf(" multi");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_RESIZE) {
+ printf(" resize");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_CACHE) {
+ printf(" cache");
+ }
+ printf(" )\n");
+ }
+ if (list[i].min_block) {
+ printf(" min block: %u\n", list[i].min_block);
+ printf(" opt block: %u\n", list[i].opt_block);
+ printf(" max block: %u\n", list[i].max_block);
+ }
+ if (list[i].n_contexts) {
+ printf(" available meta contexts: %d\n", list[i].n_contexts);
+ for (j = 0; j < list[i].n_contexts; j++) {
+ printf(" %s\n", list[i].contexts[j]);
+ }
+ }
+ }
+ nbd_free_export_list(list, rc);
+
+ ret = EXIT_SUCCESS;
+ out_socket:
+ object_unref(OBJECT(sioc));
+ out:
+ return ret;
+}
+
+
#if HAVE_NBD_DEVICE
static void *show_parts(void *arg)
{
@@ -425,7 +513,8 @@ static QemuOptsList qemu_object_opts = {
-static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, Error **errp)
+static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, bool list,
+ Error **errp)
{
Object *obj;
QCryptoTLSCreds *creds;
@@ -445,10 +534,18 @@ static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, Error **errp)
return NULL;
}
- if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_SERVER) {
- error_setg(errp,
- "Expecting TLS credentials with a server endpoint");
- return NULL;
+ if (list) {
+ if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT) {
+ error_setg(errp,
+ "Expecting TLS credentials with a client endpoint");
+ return NULL;
+ }
+ } else {
+ if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_SERVER) {
+ error_setg(errp,
+ "Expecting TLS credentials with a server endpoint");
+ return NULL;
+ }
}
object_ref(obj);
return creds;
@@ -471,7 +568,8 @@ static void setup_address_and_port(const char **address, const char **port)
static const char *socket_activation_validate_opts(const char *device,
const char *sockpath,
const char *address,
- const char *port)
+ const char *port,
+ bool list)
{
if (device != NULL) {
return "NBD device can't be set when using socket activation";
@@ -489,6 +587,10 @@ static const char *socket_activation_validate_opts(const char *device,
return "TCP port number can't be set when using socket activation";
}
+ if (list) {
+ return "List mode is incompatible with socket activation";
+ }
+
return NULL;
}
@@ -512,7 +614,7 @@ int main(int argc, char **argv)
int64_t fd_size;
QemuOpts *sn_opts = NULL;
const char *sn_id_or_name = NULL;
- const char *sopt = "hVb:o:p:rsnP:c:dvk:e:f:tl:x:T:D:B:";
+ const char *sopt = "hVb:o:p:rsnP:c:dvk:e:f:tl:x:T:D:B:L";
struct option lopt[] = {
{ "help", no_argument, NULL, 'h' },
{ "version", no_argument, NULL, 'V' },
@@ -525,6 +627,7 @@ int main(int argc, char **argv)
{ "bitmap", required_argument, NULL, 'B' },
{ "connect", required_argument, NULL, 'c' },
{ "disconnect", no_argument, NULL, 'd' },
+ { "list", no_argument, NULL, 'L' },
{ "snapshot", no_argument, NULL, 's' },
{ "load-snapshot", required_argument, NULL, 'l' },
{ "nocache", no_argument, NULL, 'n' },
@@ -559,7 +662,7 @@ int main(int argc, char **argv)
Error *local_err = NULL;
BlockdevDetectZeroesOptions detect_zeroes = BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF;
QDict *options = NULL;
- const char *export_name = ""; /* Default export name */
+ const char *export_name = NULL; /* defaults to "" later for server mode */
const char *export_description = NULL;
const char *bitmap = NULL;
const char *tlscredsid = NULL;
@@ -567,6 +670,7 @@ int main(int argc, char **argv)
bool writethrough = true;
char *trace_file = NULL;
bool fork_process = false;
+ bool list = false;
int old_stderr = -1;
unsigned socket_activation;
@@ -760,13 +864,32 @@ int main(int argc, char **argv)
case QEMU_NBD_OPT_FORK:
fork_process = true;
break;
+ case 'L':
+ list = true;
+ break;
}
}
- if ((argc - optind) != 1) {
+ if (list) {
+ if (argc != optind) {
+ error_report("List mode is incompatible with a file name");
+ exit(EXIT_FAILURE);
+ }
+ if (export_name || export_description || dev_offset || partition ||
+ device || disconnect || fmt || sn_id_or_name || bitmap) {
+ error_report("List mode is incompatible with per-device settings");
+ exit(EXIT_FAILURE);
+ }
+ if (fork_process) {
+ error_report("List mode is incompatible with forking");
+ exit(EXIT_FAILURE);
+ }
+ } else if ((argc - optind) != 1) {
error_report("Invalid number of arguments");
error_printf("Try `%s --help' for more information.\n", argv[0]);
exit(EXIT_FAILURE);
+ } else if (!export_name) {
+ export_name = "";
}
qemu_opts_foreach(&qemu_object_opts,
@@ -785,7 +908,8 @@ int main(int argc, char **argv)
} else {
/* Using socket activation - check user didn't use -p etc. */
const char *err_msg = socket_activation_validate_opts(device, sockpath,
- bindto, port);
+ bindto, port,
+ list);
if (err_msg != NULL) {
error_report("%s", err_msg);
exit(EXIT_FAILURE);
@@ -808,7 +932,7 @@ int main(int argc, char **argv)
error_report("TLS is not supported with a host device");
exit(EXIT_FAILURE);
}
- tlscreds = nbd_get_tls_creds(tlscredsid, &local_err);
+ tlscreds = nbd_get_tls_creds(tlscredsid, list, &local_err);
if (local_err) {
error_report("Failed to get TLS creds %s",
error_get_pretty(local_err));
@@ -816,6 +940,11 @@ int main(int argc, char **argv)
}
}
+ if (list) {
+ saddr = nbd_build_socket_address(sockpath, bindto, port);
+ return qemu_nbd_client_list(saddr, tlscreds, bindto);
+ }
+
#if !HAVE_NBD_DEVICE
if (disconnect || device) {
error_report("Kernel /dev/nbdN support not available");
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 17/19] qemu-nbd: Add --list option
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 17/19] qemu-nbd: Add --list option Eric Blake
@ 2019-01-17 10:05 ` Vladimir Sementsov-Ogievskiy
2019-01-17 16:58 ` Eric Blake
0 siblings, 1 reply; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-17 10:05 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
12.01.2019 20:58, Eric Blake wrote:
> We want to be able to detect whether a given qemu NBD server is
> exposing the right export(s) and dirty bitmaps, at least for
> regression testing. We could use 'nbd-client -l' from the upstream
> NBD project to list exports, but it's annoying to rely on
> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
> know about all of the qemu NBD extensions. Thus, it is time to add
> a new mode to qemu-nbd that merely sniffs all possible information
> from the server during handshake phase, then disconnects and dumps
> the information.
>
> This patch actually implements --list/-L, while reusing other
> options such as --tls-creds for now designating how to connect
> as the client (rather than their non-list usage of how to operate
> as the server).
>
> I debated about adding this functionality to something akin to
> 'qemu-img info' - but that tool does not readily lend itself
> to connecting to an arbitrary NBD server without also tying to
> a specific export (I may, however, still add ImageInfoSpecificNBD
> for reporting the bitmaps available when connecting to a single
> export). And, while it may feel a bit odd that normally
> qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not
> really making the qemu-nbd binary that much larger, because
> 'qemu-nbd -c' has to operate as both server and client
> simultaneously across two threads when feeding the kernel module
> for /dev/nbdN access.
>
> Sample output:
> $ qemu-nbd -L
> exports available: 1
> export: ''
> size: 65536
> flags: 0x4ed ( flush fua trim zeroes df cache )
> min block: 512
> opt block: 4096
> max block: 33554432
> available meta contexts: 1
> base:allocation
>
> Note that the output only lists sizes if the server sent
> NBD_FLAG_HAS_FLAGS, because a newstyle server does not give
> the size otherwise. It has the side effect that for really
> old servers that did not send any flags, the size is not
> output even though it was available. However, I'm not too
> concerned about that - oldstyle servers are (rightfully)
> getting less common to encounter (qemu 3.0 was the last
> version where we even serve it), and most existing servers
> that still even offer oldstyle negotiation (such as nbdkit)
> still send flags (since that was added to the NBD protocol
> in 2007 to permit read-only connections).
>
> Not done here, but maybe worth future experiments: capture
> the meat of NBDExportInfo into a QAPI struct, and use the
> generated QAPI pretty-printers instead of hand-rolling our
> output loop. It would also permit us to add a JSON output
> mode for machine parsing.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Message-Id: <20181215135324.152629-21-eblake@redhat.com>
> Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
>
> ---
> v3: comment tweak [Rich], rebase to earlier changes
> ---
> qemu-nbd.texi | 27 +++++++--
> qemu-nbd.c | 155 +++++++++++++++++++++++++++++++++++++++++++++-----
> 2 files changed, 165 insertions(+), 17 deletions(-)
>
> diff --git a/qemu-nbd.texi b/qemu-nbd.texi
> index 3f22559beb4..65caeb7874a 100644
> --- a/qemu-nbd.texi
> +++ b/qemu-nbd.texi
> @@ -2,6 +2,8 @@
> @c man begin SYNOPSIS
> @command{qemu-nbd} [OPTION]... @var{filename}
>
> +@command{qemu-nbd} @option{-L} [OPTION]...
> +
> @command{qemu-nbd} @option{-d} @var{dev}
> @c man end
> @end example
> @@ -14,6 +16,8 @@ Other uses:
> @itemize
> @item
> Bind a /dev/nbdX block device to a QEMU server (on Linux).
> +@item
> +As a client to query exports of a remote NBD server.
> @end itemize
>
> @c man end
> @@ -31,13 +35,15 @@ See the @code{qemu(1)} manual page for full details of the properties
> supported. The common object types that it makes sense to define are the
> @code{secret} object, which is used to supply passwords and/or encryption
> keys, and the @code{tls-creds} object, which is used to supply TLS
> -credentials for the qemu-nbd server.
> +credentials for the qemu-nbd server or client.
> @item -p, --port=@var{port}
> -The TCP port to listen on (default @samp{10809}).
> +The TCP port to listen on as a server, or connect to as a client
> +(default @samp{10809}).
> @item -o, --offset=@var{offset}
> The offset into the image.
> @item -b, --bind=@var{iface}
> -The interface to bind to (default @samp{0.0.0.0}).
> +The interface to bind to as a server, or connect to as a client
> +(default @samp{0.0.0.0}).
> @item -k, --socket=@var{path}
> Use a unix socket with path @var{path}.
> @item --image-opts
> @@ -97,10 +103,14 @@ Set the NBD volume export name (default of a zero-length string).
> @item -D, --description=@var{description}
> Set the NBD volume export description, as a human-readable
> string.
> +@item -L, --list
> +Connect as a client and list all details about the exports exposed by
> +a remote NBD server.
> @item --tls-creds=ID
> Enable mandatory TLS encryption for the server by setting the ID
> of the TLS credentials object previously created with the --object
> -option.
> +option; or provide the credentials needed for connecting as a client
> +in list mode.
may be "list mode (--list)", as "list mode" is not directly defined. On the other hand,
list option is extremely close to tls-creds, so it is obvious anyway.
> @item --fork
> Fork off the server process and exit the parent once the server is running.
> @item -v, --verbose
> @@ -159,6 +169,15 @@ qemu-nbd -c /dev/nbd0 -f qcow2 file.qcow2
> qemu-nbd -d /dev/nbd0
> @end example
>
> +Query a remote server to see details about what export(s) it is
> +serving on port 10809, and authenticating via PSK:
> +
> +@example
> +qemu-nbd \
> + --object tls-creds-psk,id=tls0,dir=/tmp/keys,username=eblake,endpoint=client \
> + --tls-creds tls0 -L -b remote.example.com
> +@end example
> +
> @c man end
>
> @ignore
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index f1c24683129..daccb86d0d7 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -76,7 +76,8 @@ static void usage(const char *name)
> {
> (printf) (
> "Usage: %s [OPTIONS] FILE\n"
> -"QEMU Disk Network Block Device Server\n"
> +" or: %s -L [OPTIONS]\n"
> +"QEMU Disk Network Block Device Utility\n"
> "\n"
> " -h, --help display this help and exit\n"
> " -V, --version output version information and exit\n"
> @@ -98,6 +99,7 @@ static void usage(const char *name)
> " -B, --bitmap=NAME expose a persistent dirty bitmap\n"
> "\n"
> "General purpose options:\n"
> +" -L, --list list exports available from another NBD server\n"
> " --object type,id=ID,... define an object such as 'secret' for providing\n"
> " passwords and/or encryption keys\n"
> " --tls-creds=ID use id of an earlier --object to provide TLS\n"
> @@ -131,7 +133,7 @@ static void usage(const char *name)
> " --image-opts treat FILE as a full set of image options\n"
> "\n"
> QEMU_HELP_BOTTOM "\n"
> - , name, NBD_DEFAULT_PORT, "DEVICE");
> + , name, name, NBD_DEFAULT_PORT, "DEVICE");
> }
>
> static void version(const char *name)
> @@ -243,6 +245,92 @@ static void termsig_handler(int signum)
> }
>
>
> +static int qemu_nbd_client_list(SocketAddress *saddr, QCryptoTLSCreds *tls,
> + const char *hostname)
> +{
> + int ret = EXIT_FAILURE;
> + int rc;
> + Error *err = NULL;
> + QIOChannelSocket *sioc;
> + NBDExportInfo *list;
> + int i, j;
> +
> + sioc = qio_channel_socket_new();
> + if (qio_channel_socket_connect_sync(sioc, saddr, &err) < 0) {
> + error_report_err(err);
> + goto out;
May be just return EXIT_FAUILURE here;
remove out label;
s/out_socket/out
> + }
> + rc = nbd_receive_export_list(QIO_CHANNEL(sioc), tls, hostname, &list,
> + &err);
> + if (rc < 0) {
> + if (err) {
> + error_report_err(err);
> + }
> + goto out_socket;
> + }
> + printf("exports available: %d\n", rc);
> + for (i = 0; i < rc; i++) {
> + printf(" export: '%s'\n", list[i].name);
> + if (list[i].description && *list[i].description) {
> + printf(" description: %s\n", list[i].description);
> + }
> + if (list[i].flags & NBD_FLAG_HAS_FLAGS) {
actually this is
if (server not have a bug of not setting NBD_FLAG_HAS_FLAGS) {
...
}
Why not to print @size for example, if @flags field has a bug?
Or, then, why to print flags, if @size has a bug?
> + printf(" size: %" PRIu64 "\n", list[i].size);
> + printf(" flags: 0x%x (", list[i].flags);
> + if (list[i].flags & NBD_FLAG_READ_ONLY) {
> + printf(" readonly");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_FLUSH) {
> + printf(" flush");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_FUA) {
> + printf(" fua");
> + }
> + if (list[i].flags & NBD_FLAG_ROTATIONAL) {
> + printf(" rotational");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_TRIM) {
> + printf(" trim");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_WRITE_ZEROES) {
> + printf(" zeroes");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_DF) {
> + printf(" df");
> + }
> + if (list[i].flags & NBD_FLAG_CAN_MULTI_CONN) {
> + printf(" multi");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_RESIZE) {
> + printf(" resize");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_CACHE) {
> + printf(" cache");
> + }
> + printf(" )\n");
> + }
> + if (list[i].min_block) {
> + printf(" min block: %u\n", list[i].min_block);
> + printf(" opt block: %u\n", list[i].opt_block);
> + printf(" max block: %u\n", list[i].max_block);
> + }
> + if (list[i].n_contexts) {
> + printf(" available meta contexts: %d\n", list[i].n_contexts);
> + for (j = 0; j < list[i].n_contexts; j++) {
> + printf(" %s\n", list[i].contexts[j]);
> + }
> + }
> + }
> + nbd_free_export_list(list, rc);
> +
> + ret = EXIT_SUCCESS;
> + out_socket:
> + object_unref(OBJECT(sioc));
> + out:
> + return ret;
> +}
> +
> +
> #if HAVE_NBD_DEVICE
> static void *show_parts(void *arg)
> {
> @@ -425,7 +513,8 @@ static QemuOptsList qemu_object_opts = {
>
>
>
> -static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, Error **errp)
> +static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, bool list,
> + Error **errp)
> {
> Object *obj;
> QCryptoTLSCreds *creds;
> @@ -445,10 +534,18 @@ static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, Error **errp)
> return NULL;
> }
>
> - if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_SERVER) {
> - error_setg(errp,
> - "Expecting TLS credentials with a server endpoint");
> - return NULL;
> + if (list) {
> + if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT) {
> + error_setg(errp,
> + "Expecting TLS credentials with a client endpoint");
> + return NULL;
> + }
> + } else {
> + if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_SERVER) {
> + error_setg(errp,
> + "Expecting TLS credentials with a server endpoint");
> + return NULL;
> + }
> }
> object_ref(obj);
> return creds;
> @@ -471,7 +568,8 @@ static void setup_address_and_port(const char **address, const char **port)
> static const char *socket_activation_validate_opts(const char *device,
> const char *sockpath,
> const char *address,
> - const char *port)
> + const char *port,
> + bool list)
> {
> if (device != NULL) {
> return "NBD device can't be set when using socket activation";
> @@ -489,6 +587,10 @@ static const char *socket_activation_validate_opts(const char *device,
> return "TCP port number can't be set when using socket activation";
> }
>
> + if (list) {
> + return "List mode is incompatible with socket activation";
> + }
> +
> return NULL;
> }
>
> @@ -512,7 +614,7 @@ int main(int argc, char **argv)
> int64_t fd_size;
> QemuOpts *sn_opts = NULL;
> const char *sn_id_or_name = NULL;
> - const char *sopt = "hVb:o:p:rsnP:c:dvk:e:f:tl:x:T:D:B:";
> + const char *sopt = "hVb:o:p:rsnP:c:dvk:e:f:tl:x:T:D:B:L";
> struct option lopt[] = {
> { "help", no_argument, NULL, 'h' },
> { "version", no_argument, NULL, 'V' },
> @@ -525,6 +627,7 @@ int main(int argc, char **argv)
> { "bitmap", required_argument, NULL, 'B' },
> { "connect", required_argument, NULL, 'c' },
> { "disconnect", no_argument, NULL, 'd' },
> + { "list", no_argument, NULL, 'L' },
> { "snapshot", no_argument, NULL, 's' },
> { "load-snapshot", required_argument, NULL, 'l' },
> { "nocache", no_argument, NULL, 'n' },
> @@ -559,7 +662,7 @@ int main(int argc, char **argv)
> Error *local_err = NULL;
> BlockdevDetectZeroesOptions detect_zeroes = BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF;
> QDict *options = NULL;
> - const char *export_name = ""; /* Default export name */
> + const char *export_name = NULL; /* defaults to "" later for server mode */
> const char *export_description = NULL;
> const char *bitmap = NULL;
> const char *tlscredsid = NULL;
> @@ -567,6 +670,7 @@ int main(int argc, char **argv)
> bool writethrough = true;
> char *trace_file = NULL;
> bool fork_process = false;
> + bool list = false;
> int old_stderr = -1;
> unsigned socket_activation;
>
> @@ -760,13 +864,32 @@ int main(int argc, char **argv)
> case QEMU_NBD_OPT_FORK:
> fork_process = true;
> break;
> + case 'L':
> + list = true;
> + break;
> }
> }
>
> - if ((argc - optind) != 1) {
> + if (list) {
> + if (argc != optind) {
> + error_report("List mode is incompatible with a file name");
> + exit(EXIT_FAILURE);
> + }
> + if (export_name || export_description || dev_offset || partition ||
> + device || disconnect || fmt || sn_id_or_name || bitmap) {
> + error_report("List mode is incompatible with per-device settings");
> + exit(EXIT_FAILURE);
and what about aio, discard, etc? Also, I think, it would be good to specify in Usage
(or in man), which options are available for list mode.
Hm, note, --help has grouping of options and man don't.
> + }
> + if (fork_process) {
> + error_report("List mode is incompatible with forking");
> + exit(EXIT_FAILURE);
> + }
> + } else if ((argc - optind) != 1) {
> error_report("Invalid number of arguments");
> error_printf("Try `%s --help' for more information.\n", argv[0]);
> exit(EXIT_FAILURE);
> + } else if (!export_name) {
> + export_name = "";
> }
>
> qemu_opts_foreach(&qemu_object_opts,
> @@ -785,7 +908,8 @@ int main(int argc, char **argv)
> } else {
> /* Using socket activation - check user didn't use -p etc. */
> const char *err_msg = socket_activation_validate_opts(device, sockpath,
> - bindto, port);
> + bindto, port,
> + list);
> if (err_msg != NULL) {
> error_report("%s", err_msg);
> exit(EXIT_FAILURE);
> @@ -808,7 +932,7 @@ int main(int argc, char **argv)
> error_report("TLS is not supported with a host device");
> exit(EXIT_FAILURE);
> }
> - tlscreds = nbd_get_tls_creds(tlscredsid, &local_err);
> + tlscreds = nbd_get_tls_creds(tlscredsid, list, &local_err);
> if (local_err) {
> error_report("Failed to get TLS creds %s",
> error_get_pretty(local_err));
> @@ -816,6 +940,11 @@ int main(int argc, char **argv)
> }
> }
>
> + if (list) {
> + saddr = nbd_build_socket_address(sockpath, bindto, port);
> + return qemu_nbd_client_list(saddr, tlscreds, bindto);
> + }
> +
> #if !HAVE_NBD_DEVICE
> if (disconnect || device) {
> error_report("Kernel /dev/nbdN support not available");
>
I don't have good understanding of tls related things, the rest looks OK,
my suggestions are optional, so, if you don't want to improve docs and
option conflict checking now:
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 17/19] qemu-nbd: Add --list option
2019-01-17 10:05 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-17 16:58 ` Eric Blake
2019-01-17 17:11 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-17 16:58 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block
[-- Attachment #1: Type: text/plain, Size: 5839 bytes --]
On 1/17/19 4:05 AM, Vladimir Sementsov-Ogievskiy wrote:
> 12.01.2019 20:58, Eric Blake wrote:
>> We want to be able to detect whether a given qemu NBD server is
>> exposing the right export(s) and dirty bitmaps, at least for
>> regression testing. We could use 'nbd-client -l' from the upstream
>> NBD project to list exports, but it's annoying to rely on
>> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
>> know about all of the qemu NBD extensions. Thus, it is time to add
>> a new mode to qemu-nbd that merely sniffs all possible information
>> from the server during handshake phase, then disconnects and dumps
>> the information.
>>
>> This patch actually implements --list/-L, while reusing other
>> options such as --tls-creds for now designating how to connect
>> as the client (rather than their non-list usage of how to operate
>> as the server).
>>
>>
>> Not done here, but maybe worth future experiments: capture
>> the meat of NBDExportInfo into a QAPI struct, and use the
>> generated QAPI pretty-printers instead of hand-rolling our
>> output loop. It would also permit us to add a JSON output
>> mode for machine parsing.
A start of that experiment has now been posted:
https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg04196.html
>> @item --tls-creds=ID
>> Enable mandatory TLS encryption for the server by setting the ID
>> of the TLS credentials object previously created with the --object
>> -option.
>> +option; or provide the credentials needed for connecting as a client
>> +in list mode.
>
> may be "list mode (--list)", as "list mode" is not directly defined. On the other hand,
> list option is extremely close to tls-creds, so it is obvious anyway.
I'm thinking of adding this (and see conversation below that mentions [1]):
diff --git i/qemu-nbd.texi w/qemu-nbd.texi
index 65caeb7874a..0d297eed6db 100644
--- i/qemu-nbd.texi
+++ w/qemu-nbd.texi
@@ -105,7 +105,9 @@ Set the NBD volume export description, as a
human-readable
string.
@item -L, --list
Connect as a client and list all details about the exports exposed by
-a remote NBD server.
+a remote NBD server. This enables list mode, and is incompatible
+with options that change behavior related to a specific export (such as
+@option{--export-name}, @option{--offset}, ...).
@item --tls-creds=ID
Enable mandatory TLS encryption for the server by setting the ID
of the TLS credentials object previously created with the --object
>> +static int qemu_nbd_client_list(SocketAddress *saddr, QCryptoTLSCreds *tls,
>> + const char *hostname)
>> +{
>> + int ret = EXIT_FAILURE;
>> + int rc;
>> + Error *err = NULL;
>> + QIOChannelSocket *sioc;
>> + NBDExportInfo *list;
>> + int i, j;
>> +
>> + sioc = qio_channel_socket_new();
>> + if (qio_channel_socket_connect_sync(sioc, saddr, &err) < 0) {
>> + error_report_err(err);
>> + goto out;
>
> May be just return EXIT_FAUILURE here;
> remove out label;
> s/out_socket/out
Will do. Probably leftovers from earlier attempts as I changed my
approach over time.
>> + printf("exports available: %d\n", rc);
>> + for (i = 0; i < rc; i++) {
>> + printf(" export: '%s'\n", list[i].name);
>> + if (list[i].description && *list[i].description) {
>> + printf(" description: %s\n", list[i].description);
>> + }
>> + if (list[i].flags & NBD_FLAG_HAS_FLAGS) {
>
> actually this is
>
> if (server not have a bug of not setting NBD_FLAG_HAS_FLAGS) {
Which, as the commit message mentions, is for servers so old and rare
that it really doesn't matter.
> ...
> }
>
> Why not to print @size for example, if @flags field has a bug?
>
> Or, then, why to print flags, if @size has a bug?
Because we don't have to worry about those servers being mainstream.
>>
>> - if ((argc - optind) != 1) {
>> + if (list) {
>> + if (argc != optind) {
>> + error_report("List mode is incompatible with a file name");
>> + exit(EXIT_FAILURE);
>> + }
>> + if (export_name || export_description || dev_offset || partition ||
>> + device || disconnect || fmt || sn_id_or_name || bitmap) {
>> + error_report("List mode is incompatible with per-device settings");
>> + exit(EXIT_FAILURE);
>
> and what about aio, discard, etc?
I don't mind adding in any more options that you think are useful to
flag the user on. Looks like I missed seen_aio, seen_cache,
seen_discard. Catching '-s' is harder, as it merely sets a bit within
flags rather than a witness variable.
> Also, I think, it would be good to specify in Usage
> (or in man), which options are available for list mode.
I worry that keeping an exact list may be a maintenance nightmare (the
two are likely to get out of sync); does my proposed wording above at
[1] satisfy the problem by at least making the user aware that not all
combinations will work?
Another alternative would be to just silently ignore all per-export
options, instead of warning the user that they are incompatible. I
don't know if that's any friendlier, but it is less code.
>
> Hm, note, --help has grouping of options and man don't.
That could be a separate patch, if it is desired (or squashed into 2/19)
>
>
> I don't have good understanding of tls related things, the rest looks OK,
> my suggestions are optional, so, if you don't want to improve docs and
> option conflict checking now:
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 17/19] qemu-nbd: Add --list option
2019-01-17 16:58 ` Eric Blake
@ 2019-01-17 17:11 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-17 17:11 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
17.01.2019 19:58, Eric Blake wrote:
> On 1/17/19 4:05 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 12.01.2019 20:58, Eric Blake wrote:
>>> We want to be able to detect whether a given qemu NBD server is
>>> exposing the right export(s) and dirty bitmaps, at least for
>>> regression testing. We could use 'nbd-client -l' from the upstream
>>> NBD project to list exports, but it's annoying to rely on
>>> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
>>> know about all of the qemu NBD extensions. Thus, it is time to add
>>> a new mode to qemu-nbd that merely sniffs all possible information
>>> from the server during handshake phase, then disconnects and dumps
>>> the information.
>>>
>>> This patch actually implements --list/-L, while reusing other
>>> options such as --tls-creds for now designating how to connect
>>> as the client (rather than their non-list usage of how to operate
>>> as the server).
>>>
>
>>>
>>> Not done here, but maybe worth future experiments: capture
>>> the meat of NBDExportInfo into a QAPI struct, and use the
>>> generated QAPI pretty-printers instead of hand-rolling our
>>> output loop. It would also permit us to add a JSON output
>>> mode for machine parsing.
>
> A start of that experiment has now been posted:
> https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg04196.html
>
>
>>> @item --tls-creds=ID
>>> Enable mandatory TLS encryption for the server by setting the ID
>>> of the TLS credentials object previously created with the --object
>>> -option.
>>> +option; or provide the credentials needed for connecting as a client
>>> +in list mode.
>>
>> may be "list mode (--list)", as "list mode" is not directly defined. On the other hand,
>> list option is extremely close to tls-creds, so it is obvious anyway.
>
> I'm thinking of adding this (and see conversation below that mentions [1]):
>
> diff --git i/qemu-nbd.texi w/qemu-nbd.texi
> index 65caeb7874a..0d297eed6db 100644
> --- i/qemu-nbd.texi
> +++ w/qemu-nbd.texi
> @@ -105,7 +105,9 @@ Set the NBD volume export description, as a
> human-readable
> string.
> @item -L, --list
> Connect as a client and list all details about the exports exposed by
> -a remote NBD server.
> +a remote NBD server. This enables list mode, and is incompatible
> +with options that change behavior related to a specific export (such as
> +@option{--export-name}, @option{--offset}, ...).
> @item --tls-creds=ID
> Enable mandatory TLS encryption for the server by setting the ID
> of the TLS credentials object previously created with the --object
>
>
>>> +static int qemu_nbd_client_list(SocketAddress *saddr, QCryptoTLSCreds *tls,
>>> + const char *hostname)
>>> +{
>>> + int ret = EXIT_FAILURE;
>>> + int rc;
>>> + Error *err = NULL;
>>> + QIOChannelSocket *sioc;
>>> + NBDExportInfo *list;
>>> + int i, j;
>>> +
>>> + sioc = qio_channel_socket_new();
>>> + if (qio_channel_socket_connect_sync(sioc, saddr, &err) < 0) {
>>> + error_report_err(err);
>>> + goto out;
>>
>> May be just return EXIT_FAUILURE here;
>> remove out label;
>> s/out_socket/out
>
> Will do. Probably leftovers from earlier attempts as I changed my
> approach over time.
>
>
>>> + printf("exports available: %d\n", rc);
>>> + for (i = 0; i < rc; i++) {
>>> + printf(" export: '%s'\n", list[i].name);
>>> + if (list[i].description && *list[i].description) {
>>> + printf(" description: %s\n", list[i].description);
>>> + }
>>> + if (list[i].flags & NBD_FLAG_HAS_FLAGS) {
>>
>> actually this is
>>
>> if (server not have a bug of not setting NBD_FLAG_HAS_FLAGS) {
>
> Which, as the commit message mentions, is for servers so old and rare
> that it really doesn't matter.
>
>> ...
>> }
>>
>> Why not to print @size for example, if @flags field has a bug?
>>
>> Or, then, why to print flags, if @size has a bug?
>
> Because we don't have to worry about those servers being mainstream.
>
>>>
>>> - if ((argc - optind) != 1) {
>>> + if (list) {
>>> + if (argc != optind) {
>>> + error_report("List mode is incompatible with a file name");
>>> + exit(EXIT_FAILURE);
>>> + }
>>> + if (export_name || export_description || dev_offset || partition ||
>>> + device || disconnect || fmt || sn_id_or_name || bitmap) {
>>> + error_report("List mode is incompatible with per-device settings");
>>> + exit(EXIT_FAILURE);
>>
>> and what about aio, discard, etc?
>
> I don't mind adding in any more options that you think are useful to
> flag the user on. Looks like I missed seen_aio, seen_cache,
> seen_discard. Catching '-s' is harder, as it merely sets a bit within
> flags rather than a witness variable.
>
>> Also, I think, it would be good to specify in Usage
>> (or in man), which options are available for list mode.
>
> I worry that keeping an exact list may be a maintenance nightmare (the
> two are likely to get out of sync); does my proposed wording above at
> [1] satisfy the problem by at least making the user aware that not all
> combinations will work?
I don't really care of it, current version is OK too. Of course, an addition
sounds better than nothing)
>
> Another alternative would be to just silently ignore all per-export
> options, instead of warning the user that they are incompatible. I
> don't know if that's any friendlier, but it is less code.
>
>>
>> Hm, note, --help has grouping of options and man don't.
>
> That could be a separate patch, if it is desired (or squashed into 2/19)
>
>>
>>
>> I don't have good understanding of tls related things, the rest looks OK,
>> my suggestions are optional, so, if you don't want to improve docs and
>> option conflict checking now:
>> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>
>>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread
* [Qemu-devel] [PATCH v3 18/19] nbd/client: Work around 3.0 bug for listing meta contexts
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (16 preceding siblings ...)
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 17/19] qemu-nbd: Add --list option Eric Blake
@ 2019-01-12 17:58 ` Eric Blake
2019-01-16 15:43 ` Vladimir Sementsov-Ogievskiy
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 19/19] iotests: Enhance 223, 233 to cover 'qemu-nbd --list' Eric Blake
` (3 subsequent siblings)
21 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:58 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block
Commit 3d068aff forgot to advertise available qemu: contexts
when the client requests a list with 0 queries. Furthermore,
3.0 shipped with a qemu-img hack of x-dirty-bitmap (commit
216ee365) that _silently_ acts as though the entire image is
clean if a requested bitmap is not present. Both bugs have
been recently fixed, so that a modern qemu server gives full
context output right away, and the client refuses a
connection if a requested x-dirty-bitmap was not found.
Still, it is likely that there will be users that have to
work with a mix of old and new qemu versions, depending on
which features get backported where, at which point being
able to rely on 'qemu-img --list' output to know for sure
whether a given NBD export has the desired dirty bitmap is
much nicer than blindly connecting and risking that the
entire image may appear clean. We can make our --list code
smart enough to work around buggy servers by tracking
whether we've seen any qemu: replies in the original 0-query
list; if not, repeat with a single query on "qemu:" (which
may still have no replies, but then we know for sure we
didn't trip up on the server bug).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20181215135324.152629-22-eblake@redhat.com>
---
nbd/client.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/nbd/client.c b/nbd/client.c
index 2001e6e8160..64f3e45edd4 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -21,6 +21,7 @@
#include "qapi/error.h"
#include "trace.h"
#include "nbd-internal.h"
+#include "qemu/cutils.h"
/* Definitions for opaque data types */
@@ -828,6 +829,8 @@ static int nbd_list_meta_contexts(QIOChannel *ioc,
Error **errp)
{
int ret;
+ int seen_any = false;
+ int seen_qemu = false;
if (nbd_send_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
info->name, NULL, errp) < 0) {
@@ -839,9 +842,25 @@ static int nbd_list_meta_contexts(QIOChannel *ioc,
ret = nbd_receive_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
&context, NULL, errp);
+ if (ret == 0 && seen_any && !seen_qemu) {
+ /*
+ * Work around qemu 3.0 bug: the server forgot to send
+ * "qemu:" replies to 0 queries. If we saw at least one
+ * reply (probably base:allocation), but none of them were
+ * qemu:, then run a more specific query to make sure.
+ */
+ seen_qemu = true;
+ if (nbd_send_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
+ info->name, "qemu:", errp) < 0) {
+ return -1;
+ }
+ continue;
+ }
if (ret <= 0) {
return ret;
}
+ seen_any = true;
+ seen_qemu |= strstart(context, "qemu:", NULL);
info->contexts = g_renew(char *, info->contexts, ++info->n_contexts);
info->contexts[info->n_contexts - 1] = context;
}
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 18/19] nbd/client: Work around 3.0 bug for listing meta contexts
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 18/19] nbd/client: Work around 3.0 bug for listing meta contexts Eric Blake
@ 2019-01-16 15:43 ` Vladimir Sementsov-Ogievskiy
2019-01-17 3:21 ` Eric Blake
0 siblings, 1 reply; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-16 15:43 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
12.01.2019 20:58, Eric Blake wrote:
> Commit 3d068aff forgot to advertise available qemu: contexts
> when the client requests a list with 0 queries. Furthermore,
> 3.0 shipped with a qemu-img hack of x-dirty-bitmap (commit
> 216ee365) that _silently_ acts as though the entire image is
> clean if a requested bitmap is not present. Both bugs have
> been recently fixed, so that a modern qemu server gives full
> context output right away, and the client refuses a
> connection if a requested x-dirty-bitmap was not found.
>
> Still, it is likely that there will be users that have to
> work with a mix of old and new qemu versions, depending on
> which features get backported where, at which point being
> able to rely on 'qemu-img --list' output to know for sure
> whether a given NBD export has the desired dirty bitmap is
> much nicer than blindly connecting and risking that the
> entire image may appear clean. We can make our --list code
> smart enough to work around buggy servers by tracking
> whether we've seen any qemu: replies in the original 0-query
> list; if not, repeat with a single query on "qemu:" (which
> may still have no replies, but then we know for sure we
> didn't trip up on the server bug).
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Message-Id: <20181215135324.152629-22-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> nbd/client.c | 19 +++++++++++++++++++
> 1 file changed, 19 insertions(+)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 2001e6e8160..64f3e45edd4 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -21,6 +21,7 @@
> #include "qapi/error.h"
> #include "trace.h"
> #include "nbd-internal.h"
> +#include "qemu/cutils.h"
>
> /* Definitions for opaque data types */
>
> @@ -828,6 +829,8 @@ static int nbd_list_meta_contexts(QIOChannel *ioc,
> Error **errp)
> {
> int ret;
> + int seen_any = false;
> + int seen_qemu = false;
>
> if (nbd_send_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
> info->name, NULL, errp) < 0) {
> @@ -839,9 +842,25 @@ static int nbd_list_meta_contexts(QIOChannel *ioc,
>
> ret = nbd_receive_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
> &context, NULL, errp);
> + if (ret == 0 && seen_any && !seen_qemu) {
> + /*
> + * Work around qemu 3.0 bug: the server forgot to send
> + * "qemu:" replies to 0 queries. If we saw at least one
> + * reply (probably base:allocation), but none of them were
if we are saying about 3.0, it is base:allocation for sure, isn't it?
> + * qemu:, then run a more specific query to make sure.
> + */
> + seen_qemu = true;
> + if (nbd_send_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
> + info->name, "qemu:", errp) < 0) {
> + return -1;
> + }
> + continue;
> + }
> if (ret <= 0) {
> return ret;
> }
> + seen_any = true;
> + seen_qemu |= strstart(context, "qemu:", NULL);
> info->contexts = g_renew(char *, info->contexts, ++info->n_contexts);
> info->contexts[info->n_contexts - 1] = context;
> }
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 18/19] nbd/client: Work around 3.0 bug for listing meta contexts
2019-01-16 15:43 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-17 3:21 ` Eric Blake
2019-01-17 8:07 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-17 3:21 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block
[-- Attachment #1: Type: text/plain, Size: 1041 bytes --]
On 1/16/19 9:43 AM, Vladimir Sementsov-Ogievskiy wrote:
>> @@ -839,9 +842,25 @@ static int nbd_list_meta_contexts(QIOChannel *ioc,
>>
>> ret = nbd_receive_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
>> &context, NULL, errp);
>> + if (ret == 0 && seen_any && !seen_qemu) {
>> + /*
>> + * Work around qemu 3.0 bug: the server forgot to send
>> + * "qemu:" replies to 0 queries. If we saw at least one
>> + * reply (probably base:allocation), but none of them were
>
> if we are saying about 3.0, it is base:allocation for sure, isn't it?
>
>> + * qemu:, then run a more specific query to make sure.
If the server is qemu 3.0, then yes, it is base:allocation. But it could
be some other server that has its own custom return without implementing
base:allocation.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 18/19] nbd/client: Work around 3.0 bug for listing meta contexts
2019-01-17 3:21 ` Eric Blake
@ 2019-01-17 8:07 ` Vladimir Sementsov-Ogievskiy
2019-01-17 14:20 ` Eric Blake
0 siblings, 1 reply; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-17 8:07 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
17.01.2019 6:21, Eric Blake wrote:
> On 1/16/19 9:43 AM, Vladimir Sementsov-Ogievskiy wrote:
>
>>> @@ -839,9 +842,25 @@ static int nbd_list_meta_contexts(QIOChannel *ioc,
>>>
>>> ret = nbd_receive_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
>>> &context, NULL, errp);
>>> + if (ret == 0 && seen_any && !seen_qemu) {
>>> + /*
>>> + * Work around qemu 3.0 bug: the server forgot to send
>>> + * "qemu:" replies to 0 queries. If we saw at least one
>>> + * reply (probably base:allocation), but none of them were
>>
>> if we are saying about 3.0, it is base:allocation for sure, isn't it?
>>
>>> + * qemu:, then run a more specific query to make sure.
>
> If the server is qemu 3.0, then yes, it is base:allocation. But it could
> be some other server that has its own custom return without implementing
> base:allocation.
Indeed) And in this context, heuristic about that server should have at least one
context listed with no query seems not generic. Why not query 'qemu:' even if empty
query returns nothing? So, at least, "probably" is imbalanced with this not described
in comment heuristic which seems bound to 3.0.
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 18/19] nbd/client: Work around 3.0 bug for listing meta contexts
2019-01-17 8:07 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-17 14:20 ` Eric Blake
0 siblings, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-17 14:20 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block
[-- Attachment #1: Type: text/plain, Size: 1872 bytes --]
On 1/17/19 2:07 AM, Vladimir Sementsov-Ogievskiy wrote:
> 17.01.2019 6:21, Eric Blake wrote:
>> On 1/16/19 9:43 AM, Vladimir Sementsov-Ogievskiy wrote:
>>
>>>> @@ -839,9 +842,25 @@ static int nbd_list_meta_contexts(QIOChannel *ioc,
>>>>
>>>> ret = nbd_receive_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
>>>> &context, NULL, errp);
>>>> + if (ret == 0 && seen_any && !seen_qemu) {
>>>> + /*
>>>> + * Work around qemu 3.0 bug: the server forgot to send
>>>> + * "qemu:" replies to 0 queries. If we saw at least one
>>>> + * reply (probably base:allocation), but none of them were
>>>
>>> if we are saying about 3.0, it is base:allocation for sure, isn't it?
>>>
>>>> + * qemu:, then run a more specific query to make sure.
>>
>> If the server is qemu 3.0, then yes, it is base:allocation. But it could
>> be some other server that has its own custom return without implementing
>> base:allocation.
>
> Indeed) And in this context, heuristic about that server should have at least one
> context listed with no query seems not generic. Why not query 'qemu:' even if empty
> query returns nothing?
Because it is highly unlikely that we will ever encounter a server that
knows how to serve "qemu:" contexts but not "base:allocation" (qemu is
not such a server, and why would any other server bother with qemu:
specific information?).
> So, at least, "probably" is imbalanced with this not described
> in comment heuristic which seems bound to 3.0.
qemu 3.0 is the only server where the heuristic will make a difference,
but not the only server where the heuristic may trigger.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* [Qemu-devel] [PATCH v3 19/19] iotests: Enhance 223, 233 to cover 'qemu-nbd --list'
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (17 preceding siblings ...)
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 18/19] nbd/client: Work around 3.0 bug for listing meta contexts Eric Blake
@ 2019-01-12 17:58 ` Eric Blake
2019-01-17 13:34 ` Eric Blake
2019-01-14 12:22 ` [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Vladimir Sementsov-Ogievskiy
` (2 subsequent siblings)
21 siblings, 1 reply; 71+ messages in thread
From: Eric Blake @ 2019-01-12 17:58 UTC (permalink / raw)
To: qemu-devel
Cc: nsoffer, rjones, jsnow, vsementsov, qemu-block, Kevin Wolf, Max Reitz
Any good new feature deserves some regression testing :)
Coverage includes:
- 223: what happens when there are 0 or more than 1 export,
proof that we can see multiple contexts including qemu:dirty-bitmap
- 233: proof that we can list over TLS, and that mix-and-match of
plain/TLS listings will behave sanely
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
v3: Rebase to earlier changes
---
tests/qemu-iotests/223 | 2 ++
tests/qemu-iotests/223.out | 20 ++++++++++++++++++++
tests/qemu-iotests/233 | 19 +++++++++++++------
tests/qemu-iotests/233.out | 15 +++++++++++++++
4 files changed, 50 insertions(+), 6 deletions(-)
diff --git a/tests/qemu-iotests/223 b/tests/qemu-iotests/223
index 773892dbe60..f120a016460 100755
--- a/tests/qemu-iotests/223
+++ b/tests/qemu-iotests/223
@@ -127,6 +127,7 @@ _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-start",
_send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-start",
"arguments":{"addr":{"type":"unix",
"data":{"path":"'"$TEST_DIR/nbd"1'"}}}}' "error" # Attempt second server
+$QEMU_NBD_PROG -L -k "$TEST_DIR/nbd"
_send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add",
"arguments":{"device":"n", "bitmap":"b"}}' "return"
_send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add",
@@ -142,6 +143,7 @@ _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add",
_send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add",
"arguments":{"device":"n", "name":"n2", "writable":true,
"bitmap":"b2"}}' "return"
+$QEMU_NBD_PROG -L -k "$TEST_DIR/nbd"
echo
echo "=== Contrast normal status to large granularity dirty-bitmap ==="
diff --git a/tests/qemu-iotests/223.out b/tests/qemu-iotests/223.out
index 0de5240a75e..6476b77ba20 100644
--- a/tests/qemu-iotests/223.out
+++ b/tests/qemu-iotests/223.out
@@ -30,12 +30,32 @@ wrote 2097152/2097152 bytes at offset 2097152
{"error": {"class": "GenericError", "desc": "NBD server not running"}}
{"return": {}}
{"error": {"class": "GenericError", "desc": "NBD server already running"}}
+exports available: 0
{"return": {}}
{"error": {"class": "GenericError", "desc": "Cannot find device=nosuch nor node_name=nosuch"}}
{"error": {"class": "GenericError", "desc": "NBD server already has export named 'n'"}}
{"error": {"class": "GenericError", "desc": "Enabled bitmap 'b2' incompatible with readonly export"}}
{"error": {"class": "GenericError", "desc": "Bitmap 'b3' is not found"}}
{"return": {}}
+exports available: 2
+ export: 'n'
+ size: 4194304
+ flags: 0x4ef ( readonly flush fua trim zeroes df cache )
+ min block: 512
+ opt block: 4096
+ max block: 33554432
+ available meta contexts: 2
+ base:allocation
+ qemu:dirty-bitmap:b
+ export: 'n2'
+ size: 4194304
+ flags: 0x4ed ( flush fua trim zeroes df cache )
+ min block: 512
+ opt block: 4096
+ max block: 33554432
+ available meta contexts: 2
+ base:allocation
+ qemu:dirty-bitmap:b2
=== Contrast normal status to large granularity dirty-bitmap ===
diff --git a/tests/qemu-iotests/233 b/tests/qemu-iotests/233
index 1814efe3333..a6ef7b20fb4 100755
--- a/tests/qemu-iotests/233
+++ b/tests/qemu-iotests/233
@@ -68,10 +68,12 @@ echo
echo "== check TLS client to plain server fails =="
nbd_server_start_tcp_socket -f $IMGFMT "$TEST_IMG"
-$QEMU_IMG info --image-opts \
- --object tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0 \
+obj=tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0
+$QEMU_IMG info --image-opts --object $obj \
driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port --object $obj \
+ --tls-creds=tls0
nbd_server_stop
@@ -84,20 +86,25 @@ nbd_server_start_tcp_socket \
-f $IMGFMT "$TEST_IMG"
$QEMU_IMG info nbd://localhost:$nbd_tcp_port 2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port
echo
echo "== check TLS works =="
-$QEMU_IMG info --image-opts \
- --object tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0 \
+obj=tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0
+$QEMU_IMG info --image-opts --object $obj \
driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port --object $obj \
+ --tls-creds=tls0
echo
echo "== check TLS with different CA fails =="
-$QEMU_IMG info --image-opts \
- --object tls-creds-x509,dir=${tls_dir}/client2,endpoint=client,id=tls0 \
+obj=tls-creds-x509,dir=${tls_dir}/client2,endpoint=client,id=tls0
+$QEMU_IMG info --image-opts --object $obj \
driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port --object $obj \
+ --tls-creds=tls0
echo
echo "== perform I/O over TLS =="
diff --git a/tests/qemu-iotests/233.out b/tests/qemu-iotests/233.out
index 5f416721b03..ab669488669 100644
--- a/tests/qemu-iotests/233.out
+++ b/tests/qemu-iotests/233.out
@@ -15,20 +15,35 @@ wrote 1048576/1048576 bytes at offset 1048576
== check TLS client to plain server fails ==
qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': Denied by server for option 5 (starttls)
server reported: TLS not configured
+qemu-nbd: Denied by server for option 5 (starttls)
+server reported: TLS not configured
== check plain client to TLS server fails ==
qemu-img: Could not open 'nbd://localhost:PORT': TLS negotiation required before option 8 (structured reply)
server reported: Option 0x8 not permitted before TLS
+qemu-nbd: TLS negotiation required before option 8 (structured reply)
+server reported: Option 0x8 not permitted before TLS
== check TLS works ==
image: nbd://127.0.0.1:PORT
file format: nbd
virtual size: 64M (67108864 bytes)
disk size: unavailable
+exports available: 1
+ export: ''
+ size: 67108864
+ flags: 0x4ed ( flush fua trim zeroes df cache )
+ min block: 512
+ opt block: 4096
+ max block: 33554432
+ available meta contexts: 1
+ base:allocation
== check TLS with different CA fails ==
qemu-nbd: option negotiation failed: Verify failed: No certificate was found.
qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': The certificate hasn't got a known issuer
+qemu-nbd: option negotiation failed: Verify failed: No certificate was found.
+qemu-nbd: The certificate hasn't got a known issuer
== perform I/O over TLS ==
read 1048576/1048576 bytes at offset 1048576
--
2.20.1
^ permalink raw reply related [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 19/19] iotests: Enhance 223, 233 to cover 'qemu-nbd --list'
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 19/19] iotests: Enhance 223, 233 to cover 'qemu-nbd --list' Eric Blake
@ 2019-01-17 13:34 ` Eric Blake
0 siblings, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-17 13:34 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, qemu-block, rjones, Max Reitz, nsoffer, jsnow
[-- Attachment #1: Type: text/plain, Size: 2026 bytes --]
On 1/12/19 11:58 AM, Eric Blake wrote:
> Any good new feature deserves some regression testing :)
> Coverage includes:
> - 223: what happens when there are 0 or more than 1 export,
> proof that we can see multiple contexts including qemu:dirty-bitmap
> - 233: proof that we can list over TLS, and that mix-and-match of
> plain/TLS listings will behave sanely
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
> Tested-by: Richard W.M. Jones <rjones@redhat.com>
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>
> ---
> v3: Rebase to earlier changes
> ---
> tests/qemu-iotests/223 | 2 ++
> tests/qemu-iotests/223.out | 20 ++++++++++++++++++++
> tests/qemu-iotests/233 | 19 +++++++++++++------
> tests/qemu-iotests/233.out | 15 +++++++++++++++
> 4 files changed, 50 insertions(+), 6 deletions(-)
Test 233 has a pre-existing race, but this patch made it more likely to
hit. I got:
+++ /home/eblake/qemu/tests/qemu-iotests/233.out.bad 2019-01-16
22:16:17.482019114 -0600
@@ -42,8 +42,8 @@
== check TLS with different CA fails ==
qemu-nbd: option negotiation failed: Verify failed: No certificate was
found.
qemu-img: Could not open
'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': The certificate
hasn't got a known issuer
-qemu-nbd: option negotiation failed: Verify failed: No certificate was
found.
qemu-nbd: The certificate hasn't got a known issuer
+qemu-nbd: option negotiation failed: Verify failed: No certificate was
found.
The problem? We have output from two separate qemu-nbd processes (the
server, and the --list client), which are getting interleaved in an
arbitrary fashion according to who wins the scheduler race. I'm working
on a fix; probably by capturing the server's output to a file and then
cat'ing that file at the end.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (18 preceding siblings ...)
2019-01-12 17:58 ` [Qemu-devel] [PATCH v3 19/19] iotests: Enhance 223, 233 to cover 'qemu-nbd --list' Eric Blake
@ 2019-01-14 12:22 ` Vladimir Sementsov-Ogievskiy
2019-01-14 16:46 ` Eric Blake
2019-01-17 11:38 ` Vladimir Sementsov-Ogievskiy
2019-01-23 12:36 ` no-reply
21 siblings, 1 reply; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-14 12:22 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
12.01.2019 20:57, Eric Blake wrote:
> I got tired of debugging whether a server was advertising the
> correct things during negotiation by inspecting the trace
> logs of qemu-io as client - not to mention that without SOME
> sort of client tracing particular commands, we can't easily
> regression test the server for correct behavior. The final
> straw was at KVM Forum, when Nir asked me to make sure there
> was a way to easily determine if an NBD server is exposing what
> we really want (and fixing x-dirty-bitmap to behave saner fell
> out as a result of answering that question).
>
> I note that upstream NBD has 'nbd-client -l $host' for querying
> just export names (with no quoting, so you have to know that
> a blank line means the default export), but it wasn't powerful
> enough, so I implemented 'qemu-nbd -L' to document everything.
> Upstream NBD has separate 'nbd-client' and 'nbd-server' binaries,
> while we only have 'qemu-nbd' (which is normally just a server,
> but 'qemu-nbd -c' also operates a second thread as a client).
> Our other uses of qemu as NBD client are for consuming a block
> device (as in qemu-io, qemu-img, or a drive to qemu) - but those
> binaries are less suited to something so specific to the NBD
> protocol.
>
> Bonus: As a result of my work on this series, nbdkit now supports
> NBD_OPT_INFO (my interoperability testing between server
> implementations has been paying off, both at fixing server bugs,
> and at making this code more reliable across difference in valid
> servers).
>
> Also available at:
> https://repo.or.cz/qemu/ericb.git qemu-nbd-list-v2
hmm, not -v2. But not -v3 too, as I've tried to apply these patches on
-v3 base (commit under "maint: Allow for EXAMPLES in texi2pod") and it
failed.
Could you please fix tag?
>
> Based-on: <20181221093529.23855-1-jsnow@redhat.com>
> [jsnow: 0/11 bitmaps: remove x- prefix from QMP api]
> Based-on: <20190111163519.11457-1-philmd@redhat.com>
> [philmd: qemu-nbd: Rename 'exp' variable clashing with math::exp() symbol]
> Based-on: <20190111194720.15671-1-eblake@redhat.com>
> [eblake: 0/8 Promote x-nbd-server-add-bitmap to stable]
>
> Since v2:
> - Several patches merged already
> - 3 new patches based on audit of off_t vs. strtol
> - rebase patches on top of other changes, such as qemu-nbd --bitmap
> - address various review comments [Vladimir, Rich]
> - drop patch 12/22
>
> 001/19:[0020] [FC] 'maint: Allow for EXAMPLES in texi2pod'
> 002/19:[0030] [FC] 'qemu-nbd: Enhance man page'
> 003/19:[down] 'qemu-nbd: Sanity check partition bounds'
> 004/19:[down] 'nbd/server: Hoist length check to qemp_nbd_server_add'
> 005/19:[down] 'nbd/server: Favor [u]int64_t over off_t'
> 006/19:[0007] [FC] 'qemu-nbd: Avoid strtol open-coding'
> 007/19:[0016] [FC] 'nbd/client: Refactor nbd_receive_list()'
> 008/19:[----] [--] 'nbd/client: Move export name into NBDExportInfo'
> 009/19:[----] [--] 'nbd/client: Change signature of nbd_negotiate_simple_meta_context()'
> 010/19:[0007] [FC] 'nbd/client: Split out nbd_send_one_meta_context()'
> 011/19:[0048] [FC] 'nbd/client: Split out nbd_receive_one_meta_context()'
> 012/19:[----] [--] 'nbd/client: Refactor return of nbd_receive_negotiate()'
> 013/19:[----] [-C] 'nbd/client: Split handshake into two functions'
> 014/19:[----] [--] 'nbd/client: Pull out oldstyle size determination'
> 015/19:[0008] [FC] 'nbd/client: Add nbd_receive_export_list()'
> 016/19:[----] [-C] 'nbd/client: Add meta contexts to nbd_receive_export_list()'
> 017/19:[0015] [FC] 'qemu-nbd: Add --list option'
> 018/19:[----] [--] 'nbd/client: Work around 3.0 bug for listing meta contexts'
> 019/19:[0002] [FC] 'iotests: Enhance 223, 233 to cover 'qemu-nbd --list''
>
> Eric Blake (19):
> maint: Allow for EXAMPLES in texi2pod
> qemu-nbd: Enhance man page
> qemu-nbd: Sanity check partition bounds
> nbd/server: Hoist length check to qemp_nbd_server_add
> nbd/server: Favor [u]int64_t over off_t
> qemu-nbd: Avoid strtol open-coding
> nbd/client: Refactor nbd_receive_list()
> nbd/client: Move export name into NBDExportInfo
> nbd/client: Change signature of nbd_negotiate_simple_meta_context()
> nbd/client: Split out nbd_send_one_meta_context()
> nbd/client: Split out nbd_receive_one_meta_context()
> nbd/client: Refactor return of nbd_receive_negotiate()
> nbd/client: Split handshake into two functions
> nbd/client: Pull out oldstyle size determination
> nbd/client: Add nbd_receive_export_list()
> nbd/client: Add meta contexts to nbd_receive_export_list()
> qemu-nbd: Add --list option
> nbd/client: Work around 3.0 bug for listing meta contexts
> iotests: Enhance 223, 233 to cover 'qemu-nbd --list'
>
> qemu-nbd.texi | 114 ++++--
> Makefile | 2 +
> include/block/nbd.h | 31 +-
> block/nbd-client.c | 9 +-
> blockdev-nbd.c | 10 +-
> nbd/client.c | 756 ++++++++++++++++++++++++++-----------
> nbd/server.c | 23 +-
> qemu-nbd.c | 221 ++++++++---
> nbd/trace-events | 11 +-
> scripts/texi2pod.pl | 2 +-
> tests/qemu-iotests/223 | 2 +
> tests/qemu-iotests/223.out | 20 +
> tests/qemu-iotests/233 | 19 +-
> tests/qemu-iotests/233.out | 15 +
> 14 files changed, 910 insertions(+), 325 deletions(-)
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list
2019-01-14 12:22 ` [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Vladimir Sementsov-Ogievskiy
@ 2019-01-14 16:46 ` Eric Blake
0 siblings, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-14 16:46 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block
[-- Attachment #1: Type: text/plain, Size: 758 bytes --]
On 1/14/19 6:22 AM, Vladimir Sementsov-Ogievskiy wrote:
>>
>> Also available at:
>> https://repo.or.cz/qemu/ericb.git qemu-nbd-list-v2
>
> hmm, not -v2. But not -v3 too, as I've tried to apply these patches on
> -v3 base (commit under "maint: Allow for EXAMPLES in texi2pod") and it
> failed.
>
> Could you please fix tag?
Yes, I typo'd my intent to advertise a v3 tag. To avoid confusion with
anyone that may have already pulled the old tag, I've now pushed a
qemu-nbd-list-v3a tag, on top of my most recent pull request.
Based-on:
https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg02960.html
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (19 preceding siblings ...)
2019-01-14 12:22 ` [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Vladimir Sementsov-Ogievskiy
@ 2019-01-17 11:38 ` Vladimir Sementsov-Ogievskiy
2019-01-17 14:20 ` Eric Blake
2019-01-23 12:36 ` no-reply
21 siblings, 1 reply; 71+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-01-17 11:38 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, rjones, jsnow, qemu-block
12.01.2019 20:57, Eric Blake wrote:
> I got tired of debugging whether a server was advertising the
> correct things during negotiation by inspecting the trace
> logs of qemu-io as client - not to mention that without SOME
> sort of client tracing particular commands, we can't easily
> regression test the server for correct behavior. The final
> straw was at KVM Forum, when Nir asked me to make sure there
> was a way to easily determine if an NBD server is exposing what
> we really want (and fixing x-dirty-bitmap to behave saner fell
> out as a result of answering that question).
>
> I note that upstream NBD has 'nbd-client -l $host' for querying
> just export names (with no quoting, so you have to know that
> a blank line means the default export), but it wasn't powerful
> enough, so I implemented 'qemu-nbd -L' to document everything.
> Upstream NBD has separate 'nbd-client' and 'nbd-server' binaries,
> while we only have 'qemu-nbd' (which is normally just a server,
> but 'qemu-nbd -c' also operates a second thread as a client).
> Our other uses of qemu as NBD client are for consuming a block
> device (as in qemu-io, qemu-img, or a drive to qemu) - but those
> binaries are less suited to something so specific to the NBD
> protocol.
>
> Bonus: As a result of my work on this series, nbdkit now supports
> NBD_OPT_INFO (my interoperability testing between server
> implementations has been paying off, both at fixing server bugs,
> and at making this code more reliable across difference in valid
> servers).
>
> Also available at:
> https://repo.or.cz/qemu/ericb.git qemu-nbd-list-v2
>
> Based-on: <20181221093529.23855-1-jsnow@redhat.com>
> [jsnow: 0/11 bitmaps: remove x- prefix from QMP api]
> Based-on: <20190111163519.11457-1-philmd@redhat.com>
> [philmd: qemu-nbd: Rename 'exp' variable clashing with math::exp() symbol]
> Based-on: <20190111194720.15671-1-eblake@redhat.com>
> [eblake: 0/8 Promote x-nbd-server-add-bitmap to stable]
>
> Since v2:
> - Several patches merged already
> - 3 new patches based on audit of off_t vs. strtol
> - rebase patches on top of other changes, such as qemu-nbd --bitmap
> - address various review comments [Vladimir, Rich]
> - drop patch 12/22
>
> 001/19:[0020] [FC] 'maint: Allow for EXAMPLES in texi2pod'
> 002/19:[0030] [FC] 'qemu-nbd: Enhance man page'
Strange, but I don't have 02 in my mailbox..
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 71+ messages in thread* Re: [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list
2019-01-17 11:38 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-17 14:20 ` Eric Blake
0 siblings, 0 replies; 71+ messages in thread
From: Eric Blake @ 2019-01-17 14:20 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, rjones, jsnow, qemu-block
[-- Attachment #1: Type: text/plain, Size: 650 bytes --]
On 1/17/19 5:38 AM, Vladimir Sementsov-Ogievskiy wrote:
>> Since v2:
>> - Several patches merged already
>> - 3 new patches based on audit of off_t vs. strtol
>> - rebase patches on top of other changes, such as qemu-nbd --bitmap
>> - address various review comments [Vladimir, Rich]
>> - drop patch 12/22
>>
>> 001/19:[0020] [FC] 'maint: Allow for EXAMPLES in texi2pod'
>> 002/19:[0030] [FC] 'qemu-nbd: Enhance man page'
>
> Strange, but I don't have 02 in my mailbox..
I'll bounce a copy to you...
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list
2019-01-12 17:57 [Qemu-devel] [PATCH v3 00/19] nbd: add qemu-nbd --list Eric Blake
` (20 preceding siblings ...)
2019-01-17 11:38 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-23 12:36 ` no-reply
21 siblings, 0 replies; 71+ messages in thread
From: no-reply @ 2019-01-23 12:36 UTC (permalink / raw)
To: eblake; +Cc: fam, qemu-devel, nsoffer, vsementsov, jsnow, rjones, qemu-block
Patchew URL: https://patchew.org/QEMU/20190112175812.27068-1-eblake@redhat.com/
Hi,
This series failed the docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-mingw@fedora SHOW_ENV=1 J=14
=== TEST SCRIPT END ===
CC qom/container.o
CC qom/qom-qobject.o
/tmp/qemu-test/src/block/sheepdog.c: In function 'find_vdi_name':
/tmp/qemu-test/src/block/sheepdog.c:1239:5: error: 'strncpy' specified bound 256 equals destination size [-Werror=stringop-truncation]
strncpy(buf + SD_MAX_VDI_LEN, tag, SD_MAX_VDI_TAG_LEN);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
The full log is available at
http://patchew.org/logs/20190112175812.27068-1-eblake@redhat.com/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 71+ messages in thread