* [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list
@ 2018-12-15 13:53 Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 01/22] qemu-nbd: Use program name in error messages Eric Blake
` (21 more replies)
0 siblings, 22 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
I got tired of debugging whether a server was advertising the
correct things during negotiation by inspecting the trace
logs of qemu-io as client - not to mention that without SOME
sort of client tracing particular commands, we can't easily
regression test the server for correct behavior. The final
straw was at KVM Forum, when Nir asked me to make sure there
was a way to easily determine if an NBD server is exposing what
we really want (and fixing x-dirty-bitmap to behave saner fell
out as a result of answering that question).
I note that upstream NBD has 'nbd-client -l $host' for querying
just export names (with no quoting, so you have to know that
a blank line means the default export), but it wasn't powerful
enough, so I implemented 'qemu-nbd -L' to document everything.
Upstream NBD has separate 'nbd-client' and 'nbd-server' binaries,
while we only have 'qemu-nbd' (which is normally just a server,
but 'qemu-nbd -c' also operates a second thread as a client).
Our other uses of qemu as NBD client are for consuming a block
device (as in qemu-io, qemu-img, or a drive to qemu) - but those
binaries are less suited to something so specific to the NBD
protocol.
Bonus: As a result of my work on this series, nbdkit now supports
NBD_OPT_INFO (my interoperability testing between server
implementations has been paying off, both at fixing server bugs,
and at making this code more reliable across difference in valid
servers).
Also available at:
https://repo.or.cz/qemu/ericb.git qemu-nbd-list-v2
v1->v2:
Lots of rework: several new patches added, several split into more
managable pieces, a couple reordered. Use g_free consistently. Add
R-b where the patch didn't change too much beyond the review.
001/22:[----] [--] 'qemu-nbd: Use program name in error messages'
002/22:[down] 'nbd: Document timeline of various features'
003/22:[down] 'maint: Allow for EXAMPLES in texi2pod'
004/22:[down] 'qemu-nbd: Enhance man page'
005/22:[0006] [FC] 'nbd/client: More consistent error messages'
006/22:[0009] [FC] 'qemu-nbd: Fail earlier for -c/-d on non-linux'
007/22:[down] 'qemu-nbd: Avoid strtol open-coding'
008/22:[0012] [FC] 'nbd/client: Drop pointless buf variable'
009/22:[0114] [FC] 'nbd/client: Refactor nbd_receive_list()'
010/22:[0006] [FC] 'nbd/client: Move export name into NBDExportInfo'
011/22:[down] 'nbd/client: Change signature of nbd_negotiate_simple_meta_context()'
012/22:[down] 'nbd/client: Improve error handling in nbd_negotiate_simple_meta_context()'
013/22:[down] 'nbd/client: Split out nbd_send_one_meta_context()'
014/22:[down] 'nbd/client: Split out nbd_receive_one_meta_context()'
015/22:[----] [-C] 'nbd/client: Refactor return of nbd_receive_negotiate()'
016/22:[0036] [FC] 'nbd/client: Split handshake into two functions'
017/22:[down] 'nbd/client: Pull out oldstyle size determination'
018/22:[0078] [FC] 'nbd/client: Add nbd_receive_export_list()'
019/22:[down] 'nbd/client: Add meta contexts to nbd_receive_export_list()'
020/22:[0034] [FC] 'qemu-nbd: Add --list option'
021/22:[0028] [FC] 'nbd/client: Work around 3.0 bug for listing meta contexts'
022/22:[0021] [FC] 'iotests: Enhance 223, 233 to cover 'qemu-nbd --list''
Eric Blake (22):
qemu-nbd: Use program name in error messages
nbd: Document timeline of various features
maint: Allow for EXAMPLES in texi2pod
qemu-nbd: Enhance man page
nbd/client: More consistent error messages
qemu-nbd: Fail earlier for -c/-d on non-linux
qemu-nbd: Avoid strtol open-coding
nbd/client: Drop pointless buf variable
nbd/client: Refactor nbd_receive_list()
nbd/client: Move export name into NBDExportInfo
nbd/client: Change signature of nbd_negotiate_simple_meta_context()
nbd/client: Improve error handling in
nbd_negotiate_simple_meta_context()
nbd/client: Split out nbd_send_one_meta_context()
nbd/client: Split out nbd_receive_one_meta_context()
nbd/client: Refactor return of nbd_receive_negotiate()
nbd/client: Split handshake into two functions
nbd/client: Pull out oldstyle size determination
nbd/client: Add nbd_receive_export_list()
nbd/client: Add meta contexts to nbd_receive_export_list()
qemu-nbd: Add --list option
nbd/client: Work around 3.0 bug for listing meta contexts
iotests: Enhance 223, 233 to cover 'qemu-nbd --list'
docs/interop/nbd.txt | 19 +-
qemu-nbd.texi | 107 ++++-
Makefile | 18 +-
include/block/nbd.h | 27 +-
nbd/nbd-internal.h | 3 +-
block/nbd-client.c | 9 +-
nbd/client.c | 796 +++++++++++++++++++++++++------------
qemu-nbd.c | 216 ++++++++--
nbd/trace-events | 11 +-
scripts/texi2pod.pl | 2 +-
tests/qemu-iotests/223 | 2 +
tests/qemu-iotests/223.out | 20 +
tests/qemu-iotests/233 | 19 +-
tests/qemu-iotests/233.out | 17 +-
14 files changed, 918 insertions(+), 348 deletions(-)
--
2.17.2
^ permalink raw reply [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 01/22] qemu-nbd: Use program name in error messages
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 02/22] nbd: Document timeline of various features Eric Blake
` (20 subsequent siblings)
21 siblings, 0 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel
Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block, Kevin Wolf, Max Reitz
This changes output from:
$ qemu-nbd nosuch
Failed to blk_new_open 'nosuch': Could not open 'nosuch': No such file or directory
to something more consistent with qemu-img and qemu:
$ qemu-nbd nosuch
qemu-nbd: Failed to blk_new_open 'nosuch': Could not open 'nosuch': No such file or directory
Update the lone affected test to match. (Hmm - is it sad that we don't
do much testing of expected failures?)
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
qemu-nbd.c | 1 +
tests/qemu-iotests/233.out | 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/qemu-nbd.c b/qemu-nbd.c
index ca7109652e5..e169b839ece 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -571,6 +571,7 @@ int main(int argc, char **argv)
#endif
module_call_init(MODULE_INIT_TRACE);
+ error_set_progname(argv[0]);
qcrypto_init(&error_fatal);
module_call_init(MODULE_INIT_QOM);
diff --git a/tests/qemu-iotests/233.out b/tests/qemu-iotests/233.out
index 94acd9b9479..5f416721b03 100644
--- a/tests/qemu-iotests/233.out
+++ b/tests/qemu-iotests/233.out
@@ -27,7 +27,7 @@ virtual size: 64M (67108864 bytes)
disk size: unavailable
== check TLS with different CA fails ==
-option negotiation failed: Verify failed: No certificate was found.
+qemu-nbd: option negotiation failed: Verify failed: No certificate was found.
qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': The certificate hasn't got a known issuer
== perform I/O over TLS ==
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 02/22] nbd: Document timeline of various features
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 01/22] qemu-nbd: Use program name in error messages Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 14:02 ` Richard W.M. Jones
2018-12-18 13:03 ` Vladimir Sementsov-Ogievskiy
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 03/22] maint: Allow for EXAMPLES in texi2pod Eric Blake
` (19 subsequent siblings)
21 siblings, 2 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
It can be useful to figure out which NBD protocol features are
exposed by a server, as well as what features a client will
take advantage of if available, for a given qemu release. It's
not always precise to base features on version numbers (thanks
to downstream backports), but any documentation is better than
making users search through git logs themselves.
This patch originally stemmed from a request to document that
pristine 3.0 has a known bug where NBD_OPT_LIST_META_CONTEXT
with 0 queries forgot to advertise an available
"qemu:dirty-bitmap" context, but documenting bugs like this (or
the fact that 3.0 also botched NBD_CMD_CACHE) gets to be too
much details, especially since buggy releases will be less
likely connection targets over time. Instead, I chose to just
remind users to check stable release branches.
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v2: new patch
---
docs/interop/nbd.txt | 19 ++++++++++++++++++-
1 file changed, 18 insertions(+), 1 deletion(-)
diff --git a/docs/interop/nbd.txt b/docs/interop/nbd.txt
index 77b5f459111..2b25f871e7c 100644
--- a/docs/interop/nbd.txt
+++ b/docs/interop/nbd.txt
@@ -15,7 +15,6 @@ Qemu supports the "base:allocation" metadata context as defined in the
NBD protocol specification, and also defines an additional metadata
namespace "qemu".
-
== "qemu" namespace ==
The "qemu" namespace currently contains only one type of context,
@@ -36,3 +35,21 @@ in addition to "qemu:dirty-bitmap:<dirty-bitmap-export-name>":
namespace.
* "qemu:dirty-bitmap:" - returns list of all available dirty-bitmap
metadata contexts.
+
+= Features by version =
+
+The following list documents which qemu version first implemented
+various features (both as a server exposing the feature, and as a
+client taking advantage of the feature when present), to make it
+easier to plan for cross-version interoperability. Note that in
+several cases, the initial release containing a feature may require
+additional patches from the corresponding stable branch to fix bugs in
+the operation of that feature.
+
+* 2.6: NBD_OPT_STARTTLS with TLS X.509 Certificates
+* 2.8: NBD_CMD_WRITE_ZEROES
+* 2.10: NBD_OPT_GO, NBD_INFO_BLOCK
+* 2.11: NBD_OPT_STRUCTURED_READ
+* 2.12: NBD_CMD_BLOCK_STATUS for "base:allocation"
+* 3.0: NBD_OPT_STARTTLS with TLS Pre-Shared Keys (PSK),
+NBD_CMD_BLOCK_STATUS for "qemu:dirty-bitmap:", NBD_CMD_CACHE
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 03/22] maint: Allow for EXAMPLES in texi2pod
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 01/22] qemu-nbd: Use program name in error messages Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 02/22] nbd: Document timeline of various features Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 14:04 ` Richard W.M. Jones
2018-12-18 13:46 ` Vladimir Sementsov-Ogievskiy
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 04/22] qemu-nbd: Enhance man page Eric Blake
` (18 subsequent siblings)
21 siblings, 2 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
The next commit will add an EXAMPLES section to qemu-nbd.8;
for that to work, we need to recognize EXAMPLES in texi2pod,
and we need to make all man pages be regenerated since the
output of texi2pod can be different.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v2: new patch
---
Makefile | 18 ++++++++++--------
scripts/texi2pod.pl | 2 +-
2 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/Makefile b/Makefile
index c8b9efdad4b..0bd204eff8a 100644
--- a/Makefile
+++ b/Makefile
@@ -824,14 +824,16 @@ docs/interop/qemu-qmp-qapi.texi: qapi/qapi-doc.texi
docs/interop/qemu-ga-qapi.texi: qga/qapi-generated/qga-qapi-doc.texi
@cp -p $< $@
-qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi qemu-monitor-info.texi
-qemu.1: qemu-option-trace.texi
-qemu-img.1: qemu-img.texi qemu-option-trace.texi qemu-img-cmds.texi
-fsdev/virtfs-proxy-helper.1: fsdev/virtfs-proxy-helper.texi
-qemu-nbd.8: qemu-nbd.texi qemu-option-trace.texi
-qemu-ga.8: qemu-ga.texi
-docs/qemu-block-drivers.7: docs/qemu-block-drivers.texi
-docs/qemu-cpu-models.7: docs/qemu-cpu-models.texi
+qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi \
+ qemu-monitor-info.texi scripts/texi2pod.pl
+qemu.1: qemu-option-trace.texi scripts/texi2pod.pl
+qemu-img.1: qemu-img.texi qemu-option-trace.texi qemu-img-cmds.texi \
+ scripts/texi2pod.pl
+fsdev/virtfs-proxy-helper.1: fsdev/virtfs-proxy-helper.texi scripts/texi2pod.pl
+qemu-nbd.8: qemu-nbd.texi qemu-option-trace.texi scripts/texi2pod.pl
+qemu-ga.8: qemu-ga.texi scripts/texi2pod.pl
+docs/qemu-block-drivers.7: docs/qemu-block-drivers.texi scripts/texi2pod.pl
+docs/qemu-cpu-models.7: docs/qemu-cpu-models.texi scripts/texi2pod.pl
html: qemu-doc.html docs/interop/qemu-qmp-ref.html docs/interop/qemu-ga-ref.html
info: qemu-doc.info docs/interop/qemu-qmp-ref.info docs/interop/qemu-ga-ref.info
diff --git a/scripts/texi2pod.pl b/scripts/texi2pod.pl
index 39ce584a322..839b7917cf7 100755
--- a/scripts/texi2pod.pl
+++ b/scripts/texi2pod.pl
@@ -398,7 +398,7 @@ $sects{NAME} = "$fn \- $tl\n";
$sects{FOOTNOTES} .= "=back\n" if exists $sects{FOOTNOTES};
for $sect (qw(NAME SYNOPSIS DESCRIPTION OPTIONS ENVIRONMENT FILES
- BUGS NOTES FOOTNOTES SEEALSO AUTHOR COPYRIGHT)) {
+ BUGS NOTES FOOTNOTES EXAMPLES SEEALSO AUTHOR COPYRIGHT)) {
if(exists $sects{$sect}) {
$head = $sect;
$head =~ s/SEEALSO/SEE ALSO/;
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 04/22] qemu-nbd: Enhance man page
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (2 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 03/22] maint: Allow for EXAMPLES in texi2pod Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 14:13 ` Richard W.M. Jones
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 05/22] nbd/client: More consistent error messages Eric Blake
` (17 subsequent siblings)
21 siblings, 1 reply; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
Document some useful qemu-nbd command lines. Mention some restrictions
on particular options, like -p being only for MBR images, or -c/-d
being Linux-only. Update some text given the recent change to no
longer serve oldstyle protocol (missed in commit 7f7dfe2a). Also,
consistently use trailing '.' in describing options.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v2: new patch
---
qemu-nbd.texi | 85 +++++++++++++++++++++++++++++++++++++++------------
1 file changed, 66 insertions(+), 19 deletions(-)
diff --git a/qemu-nbd.texi b/qemu-nbd.texi
index 9a84e81eed9..0e24c2801ee 100644
--- a/qemu-nbd.texi
+++ b/qemu-nbd.texi
@@ -8,7 +8,10 @@
@c man begin DESCRIPTION
-Export a QEMU disk image using the NBD protocol.
+Provide access to various QEMU NBD features. Most commonly used to
+export a QEMU disk image using the NBD protocol as a server, but can
+also be used (on Linux) to manage kernel bindings of a /dev/nbdX
+block device to a QEMU server.
@c man end
@@ -27,28 +30,29 @@ supported. The common object types that it makes sense to define are the
keys, and the @code{tls-creds} object, which is used to supply TLS
credentials for the qemu-nbd server.
@item -p, --port=@var{port}
-The TCP port to listen on (default @samp{10809})
+The TCP port to listen on (default @samp{10809}).
@item -o, --offset=@var{offset}
-The offset into the image
+The offset into the image.
@item -b, --bind=@var{iface}
-The interface to bind to (default @samp{0.0.0.0})
+The interface to bind to (default @samp{0.0.0.0}).
@item -k, --socket=@var{path}
-Use a unix socket with path @var{path}
+Use a unix socket with path @var{path}.
@item --image-opts
Treat @var{filename} as a set of image options, instead of a plain
filename. If this flag is specified, the @var{-f} flag should
not be used, instead the '@code{format=}' option should be set.
@item -f, --format=@var{fmt}
Force the use of the block driver for format @var{fmt} instead of
-auto-detecting
+auto-detecting.
@item -r, --read-only
-Export the disk as read-only
+Export the disk as read-only.
@item -P, --partition=@var{num}
-Only expose partition @var{num}
+Only expose MBR partition @var{num}. Understands physical partitions
+1-4 and logical partitions 5-8.
@item -s, --snapshot
Use @var{filename} as an external snapshot, create a temporary
file with backing_file=@var{filename}, redirect the write to
-the temporary one
+the temporary one.
@item -l, --load-snapshot=@var{snapshot_param}
Load an internal snapshot inside @var{filename} and export it
as an read-only device, @var{snapshot_param} format is
@@ -72,19 +76,18 @@ driver-specific optimized zero write commands. @var{detect-zeroes} is one of
converts a zero write to an unmap operation and can only be used if
@var{discard} is set to @samp{unmap}. The default is @samp{off}.
@item -c, --connect=@var{dev}
-Connect @var{filename} to NBD device @var{dev}
+Connect @var{filename} to NBD device @var{dev} (Linux only).
@item -d, --disconnect
-Disconnect the device @var{dev}
+Disconnect the device @var{dev} (Linux only).
@item -e, --shared=@var{num}
-Allow up to @var{num} clients to share the device (default @samp{1})
+Allow up to @var{num} clients to share the device (default @samp{1}).
@item -t, --persistent
-Don't exit on the last connection
+Don't exit on the last connection.
@item -x, --export-name=@var{name}
-Set the NBD volume export name. This switches the server to use
-the new style NBD protocol negotiation
+Set the NBD volume export name (default of a zero-length string).
@item -D, --description=@var{description}
Set the NBD volume export description, as a human-readable
-string. Requires the use of @option{-x}
+string.
@item --tls-creds=ID
Enable mandatory TLS encryption for the server by setting the ID
of the TLS credentials object previously created with the --object
@@ -92,11 +95,11 @@ option.
@item --fork
Fork off the server process and exit the parent once the server is running.
@item -v, --verbose
-Display extra debugging information
+Display extra debugging information.
@item -h, --help
-Display this help and exit
+Display this help and exit.
@item -V, --version
-Display version information and exit
+Display version information and exit.
@item -T, --trace [[enable=]@var{pattern}][,events=@var{file}][,file=@var{file}]
@findex --trace
@include qemu-option-trace.texi
@@ -104,6 +107,50 @@ Display version information and exit
@c man end
+@c man begin EXAMPLES
+Start a server listening on port 10809 that exposes only the
+guest-visible contents of a qcow2 file, with no TLS encryption, and
+with the default export name (an empty string). The command will block
+until the first successful client disconnects:
+
+@example
+qemu-nbd -f qcow2 file.qcow2
+@end example
+
+Start a server listening with encryption on port 10810, and require
+the client to have a correct X.509 certificate to connect to a 1
+megabyte subset of a raw file, using the export name 'subset':
+
+@example
+qemu-nbd \
+ --object tls-creds-x509,id=tls0,endpoint=server,dir=/path/to/qemutls \
+ --tls-creds tls0 -x subset -p 10810 \
+ --image-opts driver=raw,offset=1M,length=1M,file.driver=file,file.filename=file.raw
+@end example
+
+Serve a read-only copy of just the first MBR partition of a guest
+image over a Unix socket with as many as 5 simultaneous readers, with
+a persistent process forked as a daemon:
+
+@example
+qemu-nbd --fork -t -e 5 -s /path/to/sock -p 1 -r -f qcow2 file.qcow2
+@end example
+
+Expose the guest-visible contents of a qcow2 file via a block device
+/dev/nbd0 (and possibly creating /dev/nbd0p1 and friends for
+partitions found within), then disconnect the device when done.
+@emph{CAUTION}: Do not use this method to mount filesystems from an
+untrusted guest image - a malicious guest may have prepared the image
+to attempt to trigger kernel bugs in partition probing or file system
+mounting.
+
+@example
+qemu-nbd -c /dev/nbd0 -f qcow2 file.qcow2
+qemu-nbd -d /dev/nbd0
+@end example
+
+@c man end
+
@ignore
@setfilename qemu-nbd
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 05/22] nbd/client: More consistent error messages
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (3 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 04/22] qemu-nbd: Enhance man page Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 06/22] qemu-nbd: Fail earlier for -c/-d on non-linux Eric Blake
` (16 subsequent siblings)
21 siblings, 0 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
Consolidate on using decimal (not hex), on outputting the
option reply name (not just value), and a consistent comma between
clauses, when the client reports protocol discrepancies from the
server. While it won't affect normal operation, it makes
debugging additions easier.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
v2: consistent use of comma [Vladimir]
---
nbd/client.c | 21 ++++++++++++---------
1 file changed, 12 insertions(+), 9 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index b4d457a19ad..5d59d5ba78a 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -132,8 +132,9 @@ static int nbd_receive_option_reply(QIOChannel *ioc, uint32_t opt,
return -1;
}
if (reply->option != opt) {
- error_setg(errp, "Unexpected option type %x expected %x",
- reply->option, opt);
+ error_setg(errp, "Unexpected option type %u (%s), expected %u (%s)",
+ reply->option, nbd_opt_lookup(reply->option),
+ opt, nbd_opt_lookup(opt));
nbd_send_opt_abort(ioc);
return -1;
}
@@ -265,8 +266,9 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
}
return 0;
} else if (reply.type != NBD_REP_SERVER) {
- error_setg(errp, "Unexpected reply type %" PRIx32 " expected %x",
- reply.type, NBD_REP_SERVER);
+ error_setg(errp, "Unexpected reply type %u (%s), expected %u (%s)",
+ reply.type, nbd_rep_lookup(reply.type),
+ NBD_REP_SERVER, nbd_rep_lookup(NBD_REP_SERVER));
nbd_send_opt_abort(ioc);
return -1;
}
@@ -378,9 +380,9 @@ static int nbd_opt_go(QIOChannel *ioc, const char *wantname,
return 1;
}
if (reply.type != NBD_REP_INFO) {
- error_setg(errp, "unexpected reply type %" PRIu32
- " (%s), expected %u",
- reply.type, nbd_rep_lookup(reply.type), NBD_REP_INFO);
+ error_setg(errp, "unexpected reply type %u (%s), expected %u (%s)",
+ reply.type, nbd_rep_lookup(reply.type),
+ NBD_REP_INFO, nbd_rep_lookup(NBD_REP_INFO));
nbd_send_opt_abort(ioc);
return -1;
}
@@ -704,8 +706,9 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
}
if (reply.type != NBD_REP_ACK) {
- error_setg(errp, "Unexpected reply type %" PRIx32 " expected %x",
- reply.type, NBD_REP_ACK);
+ error_setg(errp, "Unexpected reply type %u (%s), expected %u (%s)",
+ reply.type, nbd_rep_lookup(reply.type),
+ NBD_REP_ACK, nbd_rep_lookup(NBD_REP_ACK));
nbd_send_opt_abort(ioc);
return -1;
}
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 06/22] qemu-nbd: Fail earlier for -c/-d on non-linux
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (4 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 05/22] nbd/client: More consistent error messages Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 14:15 ` Richard W.M. Jones
2018-12-18 14:26 ` Vladimir Sementsov-Ogievskiy
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 07/22] qemu-nbd: Avoid strtol open-coding Eric Blake
` (15 subsequent siblings)
21 siblings, 2 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
Connecting to a /dev/nbdN device is a Linux-specific action.
We were already masking -c and -d from 'qemu-nbd --help' on
non-linux. However, while -d fails with a sensible error
message, it took hunting through a couple of files to prove
that. What's more, the code for -c doesn't fail until after
it has created a pthread and tried to open a device - possibly
even printing an error message with %m on a non-Linux platform
in spite of the comment that %m is glibc-specific. Make the
failure happen sooner, then get rid of stubs that are no
longer needed because of the early exits.
While at it: tweak the blank newlines in --help output to be
consistent, whether or not built on Linux.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v2: Hoist -c error message to share with -d message [Vladimir]
Bonus: gets rid of a stray TAB in nbd/client.c
---
nbd/client.c | 18 +-----------------
qemu-nbd.c | 21 +++++++++++++++++++--
2 files changed, 20 insertions(+), 19 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index 5d59d5ba78a..3d9086af39d 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -1029,23 +1029,7 @@ int nbd_disconnect(int fd)
return 0;
}
-#else
-int nbd_init(int fd, QIOChannelSocket *ioc, NBDExportInfo *info,
- Error **errp)
-{
- error_setg(errp, "nbd_init is only supported on Linux");
- return -ENOTSUP;
-}
-
-int nbd_client(int fd)
-{
- return -ENOTSUP;
-}
-int nbd_disconnect(int fd)
-{
- return -ENOTSUP;
-}
-#endif
+#endif /* __linux__ */
int nbd_send_request(QIOChannel *ioc, NBDRequest *request)
{
diff --git a/qemu-nbd.c b/qemu-nbd.c
index e169b839ece..2807e132396 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -43,6 +43,12 @@
#include "trace/control.h"
#include "qemu-version.h"
+#ifdef __linux__
+#define HAVE_NBD_DEVICE 1
+#else
+#define HAVE_NBD_DEVICE 0
+#endif
+
#define SOCKET_PATH "/var/lock/qemu-nbd-%s"
#define QEMU_NBD_OPT_CACHE 256
#define QEMU_NBD_OPT_AIO 257
@@ -98,11 +104,11 @@ static void usage(const char *name)
" specify tracing options\n"
" --fork fork off the server process and exit the parent\n"
" once the server is running\n"
-#ifdef __linux__
+#if HAVE_NBD_DEVICE
+"\n"
"Kernel NBD client support:\n"
" -c, --connect=DEV connect FILE to the local NBD device DEV\n"
" -d, --disconnect disconnect the specified device\n"
-"\n"
#endif
"\n"
"Block device options:\n"
@@ -236,6 +242,7 @@ static void termsig_handler(int signum)
}
+#if HAVE_NBD_DEVICE
static void *show_parts(void *arg)
{
char *device = arg;
@@ -321,6 +328,7 @@ out:
kill(getpid(), SIGTERM);
return (void *) EXIT_FAILURE;
}
+#endif /* HAVE_NBD_DEVICE */
static int nbd_can_accept(void)
{
@@ -814,6 +822,12 @@ int main(int argc, char **argv)
}
}
+#if !HAVE_NBD_DEVICE
+ if (disconnect || device) {
+ error_report("Kernel /dev/nbdN support not available");
+ exit(EXIT_FAILURE);
+ }
+#else /* HAVE_NBD_DEVICE */
if (disconnect) {
int nbdfd = open(argv[optind], O_RDWR);
if (nbdfd < 0) {
@@ -829,6 +843,7 @@ int main(int argc, char **argv)
return 0;
}
+#endif
if ((device && !verbose) || fork_process) {
int stderr_fd[2];
@@ -1006,6 +1021,7 @@ int main(int argc, char **argv)
nbd_export_set_description(exp, export_description);
if (device) {
+#if HAVE_NBD_DEVICE
int ret;
ret = pthread_create(&client_thread, NULL, nbd_client_thread, device);
@@ -1013,6 +1029,7 @@ int main(int argc, char **argv)
error_report("Failed to create client thread: %s", strerror(ret));
exit(EXIT_FAILURE);
}
+#endif
} else {
/* Shut up GCC warnings. */
memset(&client_thread, 0, sizeof(client_thread));
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 07/22] qemu-nbd: Avoid strtol open-coding
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (5 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 06/22] qemu-nbd: Fail earlier for -c/-d on non-linux Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 14:17 ` Richard W.M. Jones
2018-12-18 15:11 ` Vladimir Sementsov-Ogievskiy
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 08/22] nbd/client: Drop pointless buf variable Eric Blake
` (14 subsequent siblings)
21 siblings, 2 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
Our copy-and-pasted open-coding of strtol handling forgot to
handle overflow conditions. Use qemu_strto*() instead.
In the case of --partition, since we insist on a user-supplied
partition to be non-zero, we can use 0 rather than -1 for our
initial value to distinguish when a partition is not being
served, for slightly more optimal code.
The error messages for out-of-bounds values are less specific,
but should not be a terrible loss in quality.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v2: Retitle, catch more uses of strtol
[Hmm - this depends on int64_t and off_t being compatible; if they
aren't that way on all platforms, I'll need a temporary variable]
---
qemu-nbd.c | 33 ++++++++++-----------------------
1 file changed, 10 insertions(+), 23 deletions(-)
diff --git a/qemu-nbd.c b/qemu-nbd.c
index 2807e132396..e3f739671b5 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -544,9 +544,8 @@ int main(int argc, char **argv)
};
int ch;
int opt_ind = 0;
- char *end;
int flags = BDRV_O_RDWR;
- int partition = -1;
+ int partition = 0;
int ret = 0;
bool seen_cache = false;
bool seen_discard = false;
@@ -657,13 +656,9 @@ int main(int argc, char **argv)
port = optarg;
break;
case 'o':
- dev_offset = strtoll (optarg, &end, 0);
- if (*end) {
- error_report("Invalid offset `%s'", optarg);
- exit(EXIT_FAILURE);
- }
- if (dev_offset < 0) {
- error_report("Offset must be positive `%s'", optarg);
+ if (qemu_strtoi64(optarg, NULL, 0, &dev_offset) < 0 ||
+ dev_offset < 0) {
+ error_report("Invalid offset '%s'", optarg);
exit(EXIT_FAILURE);
}
break;
@@ -685,13 +680,9 @@ int main(int argc, char **argv)
flags &= ~BDRV_O_RDWR;
break;
case 'P':
- partition = strtol(optarg, &end, 0);
- if (*end) {
- error_report("Invalid partition `%s'", optarg);
- exit(EXIT_FAILURE);
- }
- if (partition < 1 || partition > 8) {
- error_report("Invalid partition %d", partition);
+ if (qemu_strtoi(optarg, NULL, 0, &partition) < 0 ||
+ partition < 1 || partition > 8) {
+ error_report("Invalid partition '%s'", optarg);
exit(EXIT_FAILURE);
}
break;
@@ -709,15 +700,11 @@ int main(int argc, char **argv)
device = optarg;
break;
case 'e':
- shared = strtol(optarg, &end, 0);
- if (*end) {
+ if (qemu_strtoi(optarg, NULL, 0, &shared) < 0 ||
+ shared < 1) {
error_report("Invalid shared device number '%s'", optarg);
exit(EXIT_FAILURE);
}
- if (shared < 1) {
- error_report("Shared device number must be greater than 0");
- exit(EXIT_FAILURE);
- }
break;
case 'f':
fmt = optarg;
@@ -1006,7 +993,7 @@ int main(int argc, char **argv)
}
fd_size -= dev_offset;
- if (partition != -1) {
+ if (partition) {
ret = find_partition(blk, partition, &dev_offset, &fd_size);
if (ret < 0) {
error_report("Could not find partition %d: %s", partition,
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 08/22] nbd/client: Drop pointless buf variable
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (6 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 07/22] qemu-nbd: Avoid strtol open-coding Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2019-01-05 13:54 ` Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 09/22] nbd/client: Refactor nbd_receive_list() Eric Blake
` (13 subsequent siblings)
21 siblings, 1 reply; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
There's no need to read into a temporary buffer (oversized
since commit 7d3123e1) followed by a byteswap into a uint64_t
to check for a magic number via memcmp(), when the code
immediately below demonstrates reading into the uint64_t then
byteswapping in place and checking for a magic number via
integer math. What's more, having a different error message
when the server's first reply byte is 0 is unusual - it's no
different from any other wrong magic number, and we already
detected short reads. That whole strlen() issue has been
present and useless since commit 1d45f8b5 in 2010; perhaps it
was leftover debugging (since the correct magic number happens
to be ASCII)? Make the error messages more consistent and
detailed while touching things.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
v2: improve commit message based on git archaeology [Rich]
added strategic comments, and improve error messages [Vladimir]
---
nbd/nbd-internal.h | 3 ++-
nbd/client.c | 22 +++++++---------------
2 files changed, 9 insertions(+), 16 deletions(-)
diff --git a/nbd/nbd-internal.h b/nbd/nbd-internal.h
index eeff78d3c98..443e177d44a 100644
--- a/nbd/nbd-internal.h
+++ b/nbd/nbd-internal.h
@@ -46,8 +46,9 @@
/* Size of oldstyle negotiation */
#define NBD_OLDSTYLE_NEGOTIATE_SIZE (8 + 8 + 8 + 4 + 124)
+#define NBD_INIT_MAGIC 0x4e42444d41474943LL /* ASCII "NBDMAGIC" */
#define NBD_REQUEST_MAGIC 0x25609513
-#define NBD_OPTS_MAGIC 0x49484156454F5054LL
+#define NBD_OPTS_MAGIC 0x49484156454F5054LL /* ASCII "IHAVEOPT" */
#define NBD_CLIENT_MAGIC 0x0000420281861253LL
#define NBD_REP_MAGIC 0x0003e889045565a9LL
diff --git a/nbd/client.c b/nbd/client.c
index 3d9086af39d..1a3a620fb6d 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -731,7 +731,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
QIOChannel **outioc, NBDExportInfo *info,
Error **errp)
{
- char buf[256];
uint64_t magic;
int rc;
bool zeroes = true;
@@ -752,27 +751,20 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
goto fail;
}
- if (nbd_read(ioc, buf, 8, errp) < 0) {
- error_prepend(errp, "Failed to read data: ");
+ if (nbd_read(ioc, &magic, sizeof(magic), errp) < 0) {
+ error_prepend(errp, "Failed to read initial magic: ");
goto fail;
}
-
- buf[8] = '\0';
- if (strlen(buf) == 0) {
- error_setg(errp, "Server connection closed unexpectedly");
- goto fail;
- }
-
- magic = ldq_be_p(buf);
+ magic = be64_to_cpu(magic);
trace_nbd_receive_negotiate_magic(magic);
- if (memcmp(buf, "NBDMAGIC", 8) != 0) {
- error_setg(errp, "Invalid magic received");
+ if (magic != NBD_INIT_MAGIC) {
+ error_setg(errp, "Bad initial magic received: 0x%" PRIx64, magic);
goto fail;
}
if (nbd_read(ioc, &magic, sizeof(magic), errp) < 0) {
- error_prepend(errp, "Failed to read magic: ");
+ error_prepend(errp, "Failed to read server magic: ");
goto fail;
}
magic = be64_to_cpu(magic);
@@ -911,7 +903,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
}
info->flags = oldflags;
} else {
- error_setg(errp, "Bad magic received");
+ error_setg(errp, "Bad server magic received: 0x%" PRIx64, magic);
goto fail;
}
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 09/22] nbd/client: Refactor nbd_receive_list()
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (7 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 08/22] nbd/client: Drop pointless buf variable Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 15:07 ` Richard W.M. Jones
2018-12-19 13:11 ` Vladimir Sementsov-Ogievskiy
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 10/22] nbd/client: Move export name into NBDExportInfo Eric Blake
` (12 subsequent siblings)
21 siblings, 2 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
Right now, nbd_receive_list() is only called by
nbd_receive_query_exports(), which in turn is only called if the
server lacks NBD_OPT_GO but has working option negotiation, and is
merely used as a quality-of-implementation trick since servers
can't give decent errors for NBD_OPT_EXPORT_NAME. However, servers
that lack NBD_OPT_GO are becoming increasingly rare (nbdkit was a
latecomer, in Aug 2018, but qemu has been such a server since commit
f37708f6 in July 2017 and released in 2.10), so it no longer makes
sense to micro-optimize that function for performance.
Furthermore, when debugging a server's implementation, tracing the
full reply (both names and descriptions) is useful, not to mention
that upcoming patches adding 'qemu-nbd --list' will want to collect
that data. And when you consider that a server can send an export
name up to the NBD protocol length limit of 4k; but our current
NBD_MAX_NAME_SIZE is only 256, we can't trace all valid server
names without more storage, but 4k is large enough that the heap
is better than the stack for long names.
Thus, I'm changing the division of labor, with nbd_receive_list()
now always malloc'ing a result on success (the malloc is bounded
by the fact that we reject servers with a reply length larger
than 32M), and moving the comparison to 'wantname' to the caller.
There is a minor change in behavior where a server with 0 exports
(an immediate NBD_REP_ACK reply) is now no longer distinguished
from a server without LIST support (NBD_REP_ERR_UNSUP); this
information could be preserved with a complication to the calling
contract to provide a bit more information, but I didn't see the
point. After all, the worst that can happen if our guess at a
match is wrong is that the caller will get a cryptic disconnect
when NBD_OPT_EXPORT_NAME fails (which is no different from what
would happen if we had not tried LIST), while treating an empty
list as immediate failure would prevent connecting to really old
servers that really did lack LIST. Besides, NBD servers with 0
exports are rare (qemu can do it when using QMP nbd-server-start
without nbd-server-add - but qemu understands NBD_OPT_GO and
thus won't tickle this change in behavior).
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v2: Rewrite in a different manner (move the comparison to the
caller)
Fix free to g_free [Vladimir]
---
nbd/client.c | 89 +++++++++++++++++++++++++++++++-----------------
nbd/trace-events | 1 +
2 files changed, 58 insertions(+), 32 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index 1a3a620fb6d..28f5a286cba 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -232,18 +232,24 @@ static int nbd_handle_reply_err(QIOChannel *ioc, NBDOptionReply *reply,
return result;
}
-/* Process another portion of the NBD_OPT_LIST reply. Set *@match if
- * the current reply matches @want or if the server does not support
- * NBD_OPT_LIST, otherwise leave @match alone. Return 0 if iteration
- * is complete, positive if more replies are expected, or negative
- * with @errp set if an unrecoverable error occurred. */
-static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
+/* nbd_receive_list:
+ * Process another portion of the NBD_OPT_LIST reply, populating any
+ * name received into *@name. If @description is non-NULL, and the
+ * server provided a description, that is also populated. The caller
+ * must eventually call g_free() on success.
+ * Returns 1 if a name was received and iteration must continue,
+ * 0 if iteration is complete,
+ * -1 with @errp set if an unrecoverable error occurred.
+ */
+static int nbd_receive_list(QIOChannel *ioc, char **name, char **description,
Error **errp)
{
+ int ret = -1;
NBDOptionReply reply;
uint32_t len;
uint32_t namelen;
- char name[NBD_MAX_NAME_SIZE + 1];
+ char *local_name = NULL;
+ char *local_desc = NULL;
int error;
if (nbd_receive_option_reply(ioc, NBD_OPT_LIST, &reply, errp) < 0) {
@@ -251,9 +257,6 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
}
error = nbd_handle_reply_err(ioc, &reply, errp);
if (error <= 0) {
- /* The server did not support NBD_OPT_LIST, so set *match on
- * the assumption that any name will be accepted. */
- *match = true;
return error;
}
len = reply.length;
@@ -290,33 +293,38 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
nbd_send_opt_abort(ioc);
return -1;
}
- if (namelen != strlen(want)) {
- if (nbd_drop(ioc, len, errp) < 0) {
- error_prepend(errp,
- "failed to skip export name with wrong length: ");
- nbd_send_opt_abort(ioc);
- return -1;
- }
- return 1;
- }
- assert(namelen < sizeof(name));
- if (nbd_read(ioc, name, namelen, errp) < 0) {
+ local_name = g_malloc(namelen + 1);
+ if (nbd_read(ioc, local_name, namelen, errp) < 0) {
error_prepend(errp, "failed to read export name: ");
nbd_send_opt_abort(ioc);
- return -1;
+ goto out;
}
- name[namelen] = '\0';
+ local_name[namelen] = '\0';
len -= namelen;
- if (nbd_drop(ioc, len, errp) < 0) {
- error_prepend(errp, "failed to read export description: ");
- nbd_send_opt_abort(ioc);
- return -1;
+ if (len) {
+ local_desc = g_malloc(len + 1);
+ if (nbd_read(ioc, local_desc, len, errp) < 0) {
+ error_prepend(errp, "failed to read export description: ");
+ nbd_send_opt_abort(ioc);
+ goto out;
+ }
+ local_desc[len] = '\0';
}
- if (!strcmp(name, want)) {
- *match = true;
+
+ trace_nbd_receive_list(local_name, local_desc ?: "");
+ *name = local_name;
+ local_name = NULL;
+ if (description) {
+ *description = local_desc;
+ local_desc = NULL;
}
- return 1;
+ ret = 1;
+
+ out:
+ g_free(local_name);
+ g_free(local_desc);
+ return ret;
}
@@ -491,6 +499,7 @@ static int nbd_receive_query_exports(QIOChannel *ioc,
const char *wantname,
Error **errp)
{
+ bool listEmpty = true;
bool foundExport = false;
trace_nbd_receive_query_exports_start(wantname);
@@ -499,14 +508,25 @@ static int nbd_receive_query_exports(QIOChannel *ioc,
}
while (1) {
- int ret = nbd_receive_list(ioc, wantname, &foundExport, errp);
+ char *name;
+ int ret = nbd_receive_list(ioc, &name, NULL, errp);
if (ret < 0) {
/* Server gave unexpected reply */
return -1;
} else if (ret == 0) {
/* Done iterating. */
- if (!foundExport) {
+ if (listEmpty) {
+ /*
+ * We don't have enough context to tell a server that
+ * sent an empty list apart from a server that does
+ * not support the list command; but as this function
+ * is just used to trigger a nicer error message
+ * before trying NBD_OPT_EXPORT_NAME, assume the
+ * export is available.
+ */
+ return 0;
+ } else if (!foundExport) {
error_setg(errp, "No export with name '%s' available",
wantname);
nbd_send_opt_abort(ioc);
@@ -515,6 +535,11 @@ static int nbd_receive_query_exports(QIOChannel *ioc,
trace_nbd_receive_query_exports_success(wantname);
return 0;
}
+ listEmpty = false;
+ if (!strcmp(name, wantname)) {
+ foundExport = true;
+ }
+ g_free(name);
}
}
diff --git a/nbd/trace-events b/nbd/trace-events
index 5e1d4afe8e6..8e9fc024c28 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -2,6 +2,7 @@
nbd_send_option_request(uint32_t opt, const char *name, uint32_t len) "Sending option request %" PRIu32" (%s), len %" PRIu32
nbd_receive_option_reply(uint32_t option, const char *optname, uint32_t type, const char *typename, uint32_t length) "Received option reply %" PRIu32" (%s), type %" PRIu32" (%s), len %" PRIu32
nbd_reply_err_unsup(uint32_t option, const char *name) "server doesn't understand request %" PRIu32 " (%s), attempting fallback"
+nbd_receive_list(const char *name, const char *desc) "export list includes '%s', description '%s'"
nbd_opt_go_start(const char *name) "Attempting NBD_OPT_GO for export '%s'"
nbd_opt_go_success(void) "Export is good to go"
nbd_opt_go_info_unknown(int info, const char *name) "Ignoring unknown info %d (%s)"
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 10/22] nbd/client: Move export name into NBDExportInfo
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (8 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 09/22] nbd/client: Refactor nbd_receive_list() Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 11/22] nbd/client: Change signature of nbd_negotiate_simple_meta_context() Eric Blake
` (11 subsequent siblings)
21 siblings, 0 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel
Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block, Kevin Wolf, Max Reitz
Refactor the 'name' parameter of nbd_receive_negotiate() from
being a separate parameter into being part of the in-out 'info'.
This also spills over to a simplification of nbd_opt_go().
The main driver for this refactoring is that an upcoming patch
would like to add support to qemu-nbd to list information about
all exports available on a server, where the name(s) will be
provided by the server instead of the client. But another benefit
is that we can now allow the client to explicitly specify the
empty export name "" even when connecting to an oldstyle server
(even if qemu is no longer such a server after commit 7f7dfe2a).
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
v2: Fix g_free usage [Vladimir]
---
include/block/nbd.h | 8 ++++----
block/nbd-client.c | 5 +++--
nbd/client.c | 39 ++++++++++++++++++---------------------
qemu-nbd.c | 6 ++++--
nbd/trace-events | 2 +-
5 files changed, 30 insertions(+), 30 deletions(-)
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 6a5bfe5d559..65feff8ba96 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -262,6 +262,7 @@ struct NBDExportInfo {
/* Set by client before nbd_receive_negotiate() */
bool request_sizes;
char *x_dirty_bitmap;
+ char *name; /* must be non-NULL */
/* In-out fields, set by client before nbd_receive_negotiate() and
* updated by server results during nbd_receive_negotiate() */
@@ -279,10 +280,9 @@ struct NBDExportInfo {
};
typedef struct NBDExportInfo NBDExportInfo;
-int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
- QCryptoTLSCreds *tlscreds, const char *hostname,
- QIOChannel **outioc, NBDExportInfo *info,
- Error **errp);
+int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, QIOChannel **outioc,
+ NBDExportInfo *info, Error **errp);
int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
Error **errp);
int nbd_send_request(QIOChannel *ioc, NBDRequest *request);
diff --git a/block/nbd-client.c b/block/nbd-client.c
index fc5b7eda8ee..417971d8b05 100644
--- a/block/nbd-client.c
+++ b/block/nbd-client.c
@@ -984,10 +984,11 @@ int nbd_client_init(BlockDriverState *bs,
client->info.structured_reply = true;
client->info.base_allocation = true;
client->info.x_dirty_bitmap = g_strdup(x_dirty_bitmap);
- ret = nbd_receive_negotiate(QIO_CHANNEL(sioc), export,
- tlscreds, hostname,
+ client->info.name = g_strdup(export ?: "");
+ ret = nbd_receive_negotiate(QIO_CHANNEL(sioc), tlscreds, hostname,
&client->ioc, &client->info, errp);
g_free(client->info.x_dirty_bitmap);
+ g_free(client->info.name);
if (ret < 0) {
logout("Failed to negotiate with the NBD server\n");
return ret;
diff --git a/nbd/client.c b/nbd/client.c
index 28f5a286cba..7462fa5ae0e 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -328,15 +328,14 @@ static int nbd_receive_list(QIOChannel *ioc, char **name, char **description,
}
-/* Returns -1 if NBD_OPT_GO proves the export @wantname cannot be
+/* Returns -1 if NBD_OPT_GO proves the export @info->name cannot be
* used, 0 if NBD_OPT_GO is unsupported (fall back to NBD_OPT_LIST and
* NBD_OPT_EXPORT_NAME in that case), and > 0 if the export is good to
- * go (with @info populated). */
-static int nbd_opt_go(QIOChannel *ioc, const char *wantname,
- NBDExportInfo *info, Error **errp)
+ * go (with the rest of @info populated). */
+static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
{
NBDOptionReply reply;
- uint32_t len = strlen(wantname);
+ uint32_t len = strlen(info->name);
uint16_t type;
int error;
char *buf;
@@ -346,10 +345,10 @@ static int nbd_opt_go(QIOChannel *ioc, const char *wantname,
* flags still 0 is a witness of a broken server. */
info->flags = 0;
- trace_nbd_opt_go_start(wantname);
+ trace_nbd_opt_go_start(info->name);
buf = g_malloc(4 + len + 2 + 2 * info->request_sizes + 1);
stl_be_p(buf, len);
- memcpy(buf + 4, wantname, len);
+ memcpy(buf + 4, info->name, len);
/* At most one request, everything else up to server */
stw_be_p(buf + 4 + len, info->request_sizes);
if (info->request_sizes) {
@@ -751,10 +750,9 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return 0;
}
-int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
- QCryptoTLSCreds *tlscreds, const char *hostname,
- QIOChannel **outioc, NBDExportInfo *info,
- Error **errp)
+int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, QIOChannel **outioc,
+ NBDExportInfo *info, Error **errp)
{
uint64_t magic;
int rc;
@@ -764,6 +762,8 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
trace_nbd_receive_negotiate(tlscreds, hostname ? hostname : "<null>");
+ assert(info->name);
+ trace_nbd_receive_negotiate_name(info->name);
info->structured_reply = false;
info->base_allocation = false;
rc = -EINVAL;
@@ -832,10 +832,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
goto fail;
}
}
- if (!name) {
- trace_nbd_receive_negotiate_default_name();
- name = "";
- }
if (fixedNewStyle) {
int result;
@@ -851,7 +847,8 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
if (info->structured_reply && base_allocation) {
result = nbd_negotiate_simple_meta_context(
- ioc, name, info->x_dirty_bitmap ?: "base:allocation",
+ ioc, info->name,
+ info->x_dirty_bitmap ?: "base:allocation",
&info->meta_base_allocation_id, errp);
if (result < 0) {
goto fail;
@@ -864,7 +861,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
* TLS). If it is not available, fall back to
* NBD_OPT_LIST for nicer error messages about a missing
* export, then use NBD_OPT_EXPORT_NAME. */
- result = nbd_opt_go(ioc, name, info, errp);
+ result = nbd_opt_go(ioc, info, errp);
if (result < 0) {
goto fail;
}
@@ -877,12 +874,12 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
* query gives us better error reporting if the
* export name is not available.
*/
- if (nbd_receive_query_exports(ioc, name, errp) < 0) {
+ if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
goto fail;
}
}
/* write the export name request */
- if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, name,
+ if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, info->name,
errp) < 0) {
goto fail;
}
@@ -902,8 +899,8 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
} else if (magic == NBD_CLIENT_MAGIC) {
uint32_t oldflags;
- if (name) {
- error_setg(errp, "Server does not support export names");
+ if (*info->name) {
+ error_setg(errp, "Server does not support non-empty export names");
goto fail;
}
if (tlscreds) {
diff --git a/qemu-nbd.c b/qemu-nbd.c
index e3f739671b5..545a6dfbbc7 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -263,7 +263,7 @@ static void *show_parts(void *arg)
static void *nbd_client_thread(void *arg)
{
char *device = arg;
- NBDExportInfo info = { .request_sizes = false, };
+ NBDExportInfo info = { .request_sizes = false, .name = g_strdup("") };
QIOChannelSocket *sioc;
int fd;
int ret;
@@ -278,7 +278,7 @@ static void *nbd_client_thread(void *arg)
goto out;
}
- ret = nbd_receive_negotiate(QIO_CHANNEL(sioc), NULL,
+ ret = nbd_receive_negotiate(QIO_CHANNEL(sioc),
NULL, NULL, NULL, &info, &local_error);
if (ret < 0) {
if (local_error) {
@@ -317,6 +317,7 @@ static void *nbd_client_thread(void *arg)
}
close(fd);
object_unref(OBJECT(sioc));
+ g_free(info.name);
kill(getpid(), SIGTERM);
return (void *) EXIT_SUCCESS;
@@ -325,6 +326,7 @@ out_fd:
out_socket:
object_unref(OBJECT(sioc));
out:
+ g_free(info.name);
kill(getpid(), SIGTERM);
return (void *) EXIT_FAILURE;
}
diff --git a/nbd/trace-events b/nbd/trace-events
index 8e9fc024c28..446d10b8603 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -16,7 +16,7 @@ nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of contex
nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are 0x%" PRIx32
-nbd_receive_negotiate_default_name(void) "Using default NBD export name \"\""
+nbd_receive_negotiate_name(const char *name) "Requesting NBD export name '%s'"
nbd_receive_negotiate_size_flags(uint64_t size, uint16_t flags) "Size is %" PRIu64 ", export flags 0x%" PRIx16
nbd_init_set_socket(void) "Setting NBD socket"
nbd_init_set_block_size(unsigned long block_size) "Setting block size to %lu"
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 11/22] nbd/client: Change signature of nbd_negotiate_simple_meta_context()
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (9 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 10/22] nbd/client: Move export name into NBDExportInfo Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 15:12 ` Richard W.M. Jones
2018-12-20 13:37 ` Vladimir Sementsov-Ogievskiy
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 12/22] nbd/client: Improve error handling in nbd_negotiate_simple_meta_context() Eric Blake
` (10 subsequent siblings)
21 siblings, 2 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel
Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block, Kevin Wolf, Max Reitz
Pass 'info' instead of three separate parameters related to info,
when requesting the server to set the meta context. Update the
NBDExportInfo struct to rename the received id field to match the
fact that we are currently overloading the field to match whatever
context the user supplied through the x-dirty-bitmap hack, as well
as adding a TODO comment to remind future patches about a desire
to request two contexts at once.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v2: split patch into easier-to-review pieces [Rich, Vladimir]
rename NBDExportInfo meta_base_allocation_id [Vladimir]
---
include/block/nbd.h | 2 +-
block/nbd-client.c | 4 ++--
nbd/client.c | 53 +++++++++++++++++++++------------------------
3 files changed, 28 insertions(+), 31 deletions(-)
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 65feff8ba96..ae5fe28f486 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -276,7 +276,7 @@ struct NBDExportInfo {
uint32_t opt_block;
uint32_t max_block;
- uint32_t meta_base_allocation_id;
+ uint32_t context_id;
};
typedef struct NBDExportInfo NBDExportInfo;
diff --git a/block/nbd-client.c b/block/nbd-client.c
index 417971d8b05..608b578e1d3 100644
--- a/block/nbd-client.c
+++ b/block/nbd-client.c
@@ -246,11 +246,11 @@ static int nbd_parse_blockstatus_payload(NBDClientSession *client,
}
context_id = payload_advance32(&payload);
- if (client->info.meta_base_allocation_id != context_id) {
+ if (client->info.context_id != context_id) {
error_setg(errp, "Protocol error: unexpected context id %d for "
"NBD_REPLY_TYPE_BLOCK_STATUS, when negotiated context "
"id is %d", context_id,
- client->info.meta_base_allocation_id);
+ client->info.context_id);
return -EINVAL;
}
diff --git a/nbd/client.c b/nbd/client.c
index 7462fa5ae0e..bcccd5f555e 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -628,26 +628,30 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
}
/* nbd_negotiate_simple_meta_context:
- * Set one meta context. Simple means that reply must contain zero (not
- * negotiated) or one (negotiated) contexts. More contexts would be considered
- * as a protocol error. It's also implied that meta-data query equals queried
- * context name, so, if server replies with something different than @context,
- * it is considered an error too.
- * return 1 for successful negotiation, context_id is set
+ * Request the server to set the meta context for export @info->name
+ * using @info->x_dirty_bitmap with a fallback to "base:allocation",
+ * setting @info->context_id to the resulting id. Fail if the server
+ * responds with more than one context or with a context different
+ * than the query.
+ * return 1 for successful negotiation,
* 0 if operation is unsupported,
* -1 with errp set for any other error
*/
static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
- const char *export,
- const char *context,
- uint32_t *context_id,
+ NBDExportInfo *info,
Error **errp)
{
+ /*
+ * TODO: Removing the x_dirty_bitmap hack will mean refactoring
+ * this function to request and store ids for multiple contexts
+ * (both base:allocation and a dirty bitmap), at which point this
+ * function should lose the term _simple.
+ */
int ret;
NBDOptionReply reply;
- uint32_t received_id = 0;
+ const char *context = info->x_dirty_bitmap ?: "base:allocation";
bool received = false;
- uint32_t export_len = strlen(export);
+ uint32_t export_len = strlen(info->name);
uint32_t context_len = strlen(context);
uint32_t data_len = sizeof(export_len) + export_len +
sizeof(uint32_t) + /* number of queries */
@@ -655,9 +659,9 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
char *data = g_malloc(data_len);
char *p = data;
- trace_nbd_opt_meta_request(context, export);
+ trace_nbd_opt_meta_request(context, info->name);
stl_be_p(p, export_len);
- memcpy(p += sizeof(export_len), export, export_len);
+ memcpy(p += sizeof(export_len), info->name, export_len);
stl_be_p(p += export_len, 1);
stl_be_p(p += sizeof(uint32_t), context_len);
memcpy(p += sizeof(context_len), context, context_len);
@@ -683,7 +687,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
if (reply.type == NBD_REP_META_CONTEXT) {
char *name;
- if (reply.length != sizeof(received_id) + context_len) {
+ if (reply.length != sizeof(info->context_id) + context_len) {
error_setg(errp, "Failed to negotiate meta context '%s', server "
"answered with unexpected length %" PRIu32, context,
reply.length);
@@ -691,12 +695,13 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return -1;
}
- if (nbd_read(ioc, &received_id, sizeof(received_id), errp) < 0) {
+ if (nbd_read(ioc, &info->context_id, sizeof(info->context_id),
+ errp) < 0) {
return -1;
}
- received_id = be32_to_cpu(received_id);
+ info->context_id = be32_to_cpu(info->context_id);
- reply.length -= sizeof(received_id);
+ reply.length -= sizeof(info->context_id);
name = g_malloc(reply.length + 1);
if (nbd_read(ioc, name, reply.length, errp) < 0) {
g_free(name);
@@ -713,7 +718,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
}
g_free(name);
- trace_nbd_opt_meta_reply(context, received_id);
+ trace_nbd_opt_meta_reply(context, info->context_id);
received = true;
/* receive NBD_REP_ACK */
@@ -742,12 +747,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return -1;
}
- if (received) {
- *context_id = received_id;
- return 1;
- }
-
- return 0;
+ return received;
}
int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
@@ -846,10 +846,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
}
if (info->structured_reply && base_allocation) {
- result = nbd_negotiate_simple_meta_context(
- ioc, info->name,
- info->x_dirty_bitmap ?: "base:allocation",
- &info->meta_base_allocation_id, errp);
+ result = nbd_negotiate_simple_meta_context(ioc, info, errp);
if (result < 0) {
goto fail;
}
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 12/22] nbd/client: Improve error handling in nbd_negotiate_simple_meta_context()
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (10 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 11/22] nbd/client: Change signature of nbd_negotiate_simple_meta_context() Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 15:19 ` Richard W.M. Jones
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 13/22] nbd/client: Split out nbd_send_one_meta_context() Eric Blake
` (9 subsequent siblings)
21 siblings, 1 reply; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
Always allocate space for the reply returned by the server and
hoist the trace earlier, as it is more interesting to trace the
server's reply (even if it is unexpected) than parroting our
request only on success. After all, skipping the allocation
for a wrong size was merely a micro-optimization that only
benefitted a broken server, rather than the common case of a
compliant server that meets our expectations.
Then turn the reply handling into a loop (even though we still
never iterate more than once), to make this code easier to use
when later patches do support multiple server replies. This
changes the error message for a server with two replies (a
corner case we are unlikely to hit in practice) from:
Unexpected reply type 4 (meta context), expected 0 (ack)
to:
Server replied with more than one context
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v2: split patch into easier-to-review pieces [Rich, Vladimir]
---
nbd/client.c | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index bcccd5f555e..b6a85fc3ef8 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -684,10 +684,11 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return ret;
}
- if (reply.type == NBD_REP_META_CONTEXT) {
+ while (reply.type == NBD_REP_META_CONTEXT) {
char *name;
- if (reply.length != sizeof(info->context_id) + context_len) {
+ if (reply.length <= sizeof(info->context_id) ||
+ reply.length > NBD_MAX_BUFFER_SIZE) {
error_setg(errp, "Failed to negotiate meta context '%s', server "
"answered with unexpected length %" PRIu32, context,
reply.length);
@@ -708,6 +709,15 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return -1;
}
name[reply.length] = '\0';
+ trace_nbd_opt_meta_reply(name, info->context_id);
+
+ if (received) {
+ error_setg(errp, "Server replied with more than one context");
+ g_free(name);
+ nbd_send_opt_abort(ioc);
+ return -1;
+ }
+
if (strcmp(context, name)) {
error_setg(errp, "Failed to negotiate meta context '%s', server "
"answered with different context '%s'", context,
@@ -717,8 +727,6 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return -1;
}
g_free(name);
-
- trace_nbd_opt_meta_reply(context, info->context_id);
received = true;
/* receive NBD_REP_ACK */
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 13/22] nbd/client: Split out nbd_send_one_meta_context()
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (11 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 12/22] nbd/client: Improve error handling in nbd_negotiate_simple_meta_context() Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 15:22 ` Richard W.M. Jones
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 14/22] nbd/client: Split out nbd_receive_one_meta_context() Eric Blake
` (8 subsequent siblings)
21 siblings, 1 reply; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
Refactor nbd_negotiate_simple_meta_context() to pull out the
code that can be reused to send a LIST request for 0 or 1 query.
No semantic change.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v2: split patch into easier-to-review pieces [Rich, Vladimir]
---
nbd/client.c | 64 ++++++++++++++++++++++++++++++++++--------------
nbd/trace-events | 2 +-
2 files changed, 46 insertions(+), 20 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index b6a85fc3ef8..5b6b9964097 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -627,6 +627,48 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
return QIO_CHANNEL(tioc);
}
+/* nbd_send_one_meta_context:
+ * Send 0 or 1 set/list meta context queries.
+ * return 0 on success, -1 with errp set for any error
+ */
+static int nbd_send_one_meta_context(QIOChannel *ioc,
+ uint32_t opt,
+ const char *export,
+ const char *query,
+ Error **errp)
+{
+ int ret;
+ uint32_t export_len = strlen(export);
+ uint32_t queries = !!query;
+ uint32_t context_len = 0;
+ uint32_t data_len;
+ char *data;
+ char *p;
+
+ data_len = sizeof(export_len) + export_len + sizeof(queries);
+ if (query) {
+ context_len = strlen(query);
+ data_len += sizeof(context_len) + context_len;
+ } else {
+ assert(opt == NBD_OPT_LIST_META_CONTEXT);
+ }
+ data = g_malloc(data_len);
+ p = data;
+
+ trace_nbd_opt_meta_request(nbd_opt_lookup(opt), query ?: "(all)", export);
+ stl_be_p(p, export_len);
+ memcpy(p += sizeof(export_len), export, export_len);
+ stl_be_p(p += export_len, queries);
+ if (query) {
+ stl_be_p(p += sizeof(uint32_t), context_len);
+ memcpy(p += sizeof(context_len), query, context_len);
+ }
+
+ ret = nbd_send_option_request(ioc, opt, data_len, data, errp);
+ g_free(data);
+ return ret;
+}
+
/* nbd_negotiate_simple_meta_context:
* Request the server to set the meta context for export @info->name
* using @info->x_dirty_bitmap with a fallback to "base:allocation",
@@ -651,26 +693,10 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
NBDOptionReply reply;
const char *context = info->x_dirty_bitmap ?: "base:allocation";
bool received = false;
- uint32_t export_len = strlen(info->name);
- uint32_t context_len = strlen(context);
- uint32_t data_len = sizeof(export_len) + export_len +
- sizeof(uint32_t) + /* number of queries */
- sizeof(context_len) + context_len;
- char *data = g_malloc(data_len);
- char *p = data;
- trace_nbd_opt_meta_request(context, info->name);
- stl_be_p(p, export_len);
- memcpy(p += sizeof(export_len), info->name, export_len);
- stl_be_p(p += export_len, 1);
- stl_be_p(p += sizeof(uint32_t), context_len);
- memcpy(p += sizeof(context_len), context, context_len);
-
- ret = nbd_send_option_request(ioc, NBD_OPT_SET_META_CONTEXT, data_len, data,
- errp);
- g_free(data);
- if (ret < 0) {
- return ret;
+ if (nbd_send_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
+ info->name, context, errp) < 0) {
+ return -1;
}
if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
diff --git a/nbd/trace-events b/nbd/trace-events
index 446d10b8603..00872a6f9d4 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -11,7 +11,7 @@ nbd_receive_query_exports_start(const char *wantname) "Querying export list for
nbd_receive_query_exports_success(const char *wantname) "Found desired export name '%s'"
nbd_receive_starttls_new_client(void) "Setting up TLS"
nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
-nbd_opt_meta_request(const char *context, const char *export) "Requesting to set meta context %s for export %s"
+nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 14/22] nbd/client: Split out nbd_receive_one_meta_context()
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (12 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 13/22] nbd/client: Split out nbd_send_one_meta_context() Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 15:30 ` Richard W.M. Jones
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 15/22] nbd/client: Refactor return of nbd_receive_negotiate() Eric Blake
` (7 subsequent siblings)
21 siblings, 1 reply; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
Refactor nbd_negotiate_simple_meta_context() to more closely
resemble the pattern of nbd_receive_list(), separating the
argument validation for one pass from the caller making a loop
over passes. No major semantic change (although one error
message loses the original query). The diff may be a bit hard
to follow due to indentation changes and handling ACK first
rather than last.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v2: split patch into easier-to-review pieces [Rich, Vladimir]
---
nbd/client.c | 144 +++++++++++++++++++++++++++--------------------
nbd/trace-events | 2 +-
2 files changed, 84 insertions(+), 62 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index 5b6b9964097..0e5a9d59dbd 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -669,6 +669,83 @@ static int nbd_send_one_meta_context(QIOChannel *ioc,
return ret;
}
+/* nbd_receive_one_meta_context:
+ * Called in a loop to receive and trace one set/list meta context reply.
+ * Pass non-NULL @name or @id to collect results back to the caller, which
+ * must eventually call g_free().
+ * return 1 if more contexts are expected,
+ * 0 if operation is complete,
+ * -1 with errp set for any error
+ */
+static int nbd_receive_one_meta_context(QIOChannel *ioc,
+ uint32_t opt,
+ char **name,
+ uint32_t *id,
+ Error **errp)
+{
+ int ret;
+ NBDOptionReply reply;
+ char *local_name = NULL;
+ uint32_t local_id;
+
+ if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0) {
+ return -1;
+ }
+
+ ret = nbd_handle_reply_err(ioc, &reply, errp);
+ if (ret <= 0) {
+ return ret;
+ }
+
+ if (reply.type == NBD_REP_ACK) {
+ if (reply.length != 0) {
+ error_setg(errp, "Unexpected length to ACK response");
+ nbd_send_opt_abort(ioc);
+ return -1;
+ }
+ return 0;
+ } else if (reply.type != NBD_REP_META_CONTEXT) {
+ error_setg(errp, "Unexpected reply type %u (%s), expected %u (%s)",
+ reply.type, nbd_rep_lookup(reply.type),
+ NBD_REP_META_CONTEXT, nbd_rep_lookup(NBD_REP_META_CONTEXT));
+ nbd_send_opt_abort(ioc);
+ return -1;
+ }
+
+ if (reply.length <= sizeof(local_id) ||
+ reply.length > NBD_MAX_BUFFER_SIZE) {
+ error_setg(errp, "Failed to negotiate meta context, server "
+ "answered with unexpected length %" PRIu32,
+ reply.length);
+ nbd_send_opt_abort(ioc);
+ return -1;
+ }
+
+ if (nbd_read(ioc, &local_id, sizeof(local_id), errp) < 0) {
+ return -1;
+ }
+ local_id = be32_to_cpu(local_id);
+
+ reply.length -= sizeof(local_id);
+ local_name = g_malloc(reply.length + 1);
+ if (nbd_read(ioc, local_name, reply.length, errp) < 0) {
+ g_free(local_name);
+ return -1;
+ }
+ local_name[reply.length] = '\0';
+ trace_nbd_opt_meta_reply(nbd_opt_lookup(opt), local_name, local_id);
+
+ if (name) {
+ *name = local_name;
+ } else {
+ g_free(local_name);
+ }
+ if (id) {
+ *id = local_id;
+ }
+ return 1;
+}
+
/* nbd_negotiate_simple_meta_context:
* Request the server to set the meta context for export @info->name
* using @info->x_dirty_bitmap with a fallback to "base:allocation",
@@ -690,7 +767,6 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
* function should lose the term _simple.
*/
int ret;
- NBDOptionReply reply;
const char *context = info->x_dirty_bitmap ?: "base:allocation";
bool received = false;
@@ -699,44 +775,17 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return -1;
}
- if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
- errp) < 0)
- {
- return -1;
- }
-
- ret = nbd_handle_reply_err(ioc, &reply, errp);
- if (ret <= 0) {
- return ret;
- }
-
- while (reply.type == NBD_REP_META_CONTEXT) {
+ while (1) {
char *name;
- if (reply.length <= sizeof(info->context_id) ||
- reply.length > NBD_MAX_BUFFER_SIZE) {
- error_setg(errp, "Failed to negotiate meta context '%s', server "
- "answered with unexpected length %" PRIu32, context,
- reply.length);
- nbd_send_opt_abort(ioc);
+ ret = nbd_receive_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
+ &name, &info->context_id, errp);
+ if (ret < 0) {
return -1;
+ } else if (ret == 0) {
+ return received;
}
- if (nbd_read(ioc, &info->context_id, sizeof(info->context_id),
- errp) < 0) {
- return -1;
- }
- info->context_id = be32_to_cpu(info->context_id);
-
- reply.length -= sizeof(info->context_id);
- name = g_malloc(reply.length + 1);
- if (nbd_read(ioc, name, reply.length, errp) < 0) {
- g_free(name);
- return -1;
- }
- name[reply.length] = '\0';
- trace_nbd_opt_meta_reply(name, info->context_id);
-
if (received) {
error_setg(errp, "Server replied with more than one context");
g_free(name);
@@ -754,34 +803,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
}
g_free(name);
received = true;
-
- /* receive NBD_REP_ACK */
- if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
- errp) < 0)
- {
- return -1;
- }
-
- ret = nbd_handle_reply_err(ioc, &reply, errp);
- if (ret <= 0) {
- return ret;
- }
}
-
- if (reply.type != NBD_REP_ACK) {
- error_setg(errp, "Unexpected reply type %u (%s), expected %u (%s)",
- reply.type, nbd_rep_lookup(reply.type),
- NBD_REP_ACK, nbd_rep_lookup(NBD_REP_ACK));
- nbd_send_opt_abort(ioc);
- return -1;
- }
- if (reply.length) {
- error_setg(errp, "Unexpected length to ACK response");
- nbd_send_opt_abort(ioc);
- return -1;
- }
-
- return received;
}
int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
diff --git a/nbd/trace-events b/nbd/trace-events
index 00872a6f9d4..02956c96042 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -12,7 +12,7 @@ nbd_receive_query_exports_success(const char *wantname) "Found desired export na
nbd_receive_starttls_new_client(void) "Setting up TLS"
nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
-nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
+nbd_opt_meta_reply(const char *optname, const char *context, uint32_t id) "Received %s mapping of %s to id %" PRIu32
nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are 0x%" PRIx32
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 15/22] nbd/client: Refactor return of nbd_receive_negotiate()
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (13 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 14/22] nbd/client: Split out nbd_receive_one_meta_context() Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 16/22] nbd/client: Split handshake into two functions Eric Blake
` (6 subsequent siblings)
21 siblings, 0 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
The function could only ever return 0 or -EINVAL; make this
clearer by dropping a useless 'fail:' label.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
nbd/client.c | 51 +++++++++++++++++++++++----------------------------
1 file changed, 23 insertions(+), 28 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index 0e5a9d59dbd..64a0e5760c3 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -811,7 +811,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
NBDExportInfo *info, Error **errp)
{
uint64_t magic;
- int rc;
bool zeroes = true;
bool structured_reply = info->structured_reply;
bool base_allocation = info->base_allocation;
@@ -822,31 +821,30 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
trace_nbd_receive_negotiate_name(info->name);
info->structured_reply = false;
info->base_allocation = false;
- rc = -EINVAL;
if (outioc) {
*outioc = NULL;
}
if (tlscreds && !outioc) {
error_setg(errp, "Output I/O channel required for TLS");
- goto fail;
+ return -EINVAL;
}
if (nbd_read(ioc, &magic, sizeof(magic), errp) < 0) {
error_prepend(errp, "Failed to read initial magic: ");
- goto fail;
+ return -EINVAL;
}
magic = be64_to_cpu(magic);
trace_nbd_receive_negotiate_magic(magic);
if (magic != NBD_INIT_MAGIC) {
error_setg(errp, "Bad initial magic received: 0x%" PRIx64, magic);
- goto fail;
+ return -EINVAL;
}
if (nbd_read(ioc, &magic, sizeof(magic), errp) < 0) {
error_prepend(errp, "Failed to read server magic: ");
- goto fail;
+ return -EINVAL;
}
magic = be64_to_cpu(magic);
trace_nbd_receive_negotiate_magic(magic);
@@ -858,7 +856,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
if (nbd_read(ioc, &globalflags, sizeof(globalflags), errp) < 0) {
error_prepend(errp, "Failed to read server flags: ");
- goto fail;
+ return -EINVAL;
}
globalflags = be16_to_cpu(globalflags);
trace_nbd_receive_negotiate_server_flags(globalflags);
@@ -874,18 +872,18 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
clientflags = cpu_to_be32(clientflags);
if (nbd_write(ioc, &clientflags, sizeof(clientflags), errp) < 0) {
error_prepend(errp, "Failed to send clientflags field: ");
- goto fail;
+ return -EINVAL;
}
if (tlscreds) {
if (fixedNewStyle) {
*outioc = nbd_receive_starttls(ioc, tlscreds, hostname, errp);
if (!*outioc) {
- goto fail;
+ return -EINVAL;
}
ioc = *outioc;
} else {
error_setg(errp, "Server does not support STARTTLS");
- goto fail;
+ return -EINVAL;
}
}
if (fixedNewStyle) {
@@ -896,7 +894,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
NBD_OPT_STRUCTURED_REPLY,
errp);
if (result < 0) {
- goto fail;
+ return -EINVAL;
}
info->structured_reply = result == 1;
}
@@ -904,7 +902,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
if (info->structured_reply && base_allocation) {
result = nbd_negotiate_simple_meta_context(ioc, info, errp);
if (result < 0) {
- goto fail;
+ return -EINVAL;
}
info->base_allocation = result == 1;
}
@@ -916,7 +914,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
* export, then use NBD_OPT_EXPORT_NAME. */
result = nbd_opt_go(ioc, info, errp);
if (result < 0) {
- goto fail;
+ return -EINVAL;
}
if (result > 0) {
return 0;
@@ -928,25 +926,25 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
* export name is not available.
*/
if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
- goto fail;
+ return -EINVAL;
}
}
/* write the export name request */
if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, info->name,
errp) < 0) {
- goto fail;
+ return -EINVAL;
}
/* Read the response */
if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
error_prepend(errp, "Failed to read export length: ");
- goto fail;
+ return -EINVAL;
}
info->size = be64_to_cpu(info->size);
if (nbd_read(ioc, &info->flags, sizeof(info->flags), errp) < 0) {
error_prepend(errp, "Failed to read export flags: ");
- goto fail;
+ return -EINVAL;
}
info->flags = be16_to_cpu(info->flags);
} else if (magic == NBD_CLIENT_MAGIC) {
@@ -954,43 +952,40 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
if (*info->name) {
error_setg(errp, "Server does not support non-empty export names");
- goto fail;
+ return -EINVAL;
}
if (tlscreds) {
error_setg(errp, "Server does not support STARTTLS");
- goto fail;
+ return -EINVAL;
}
if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
error_prepend(errp, "Failed to read export length: ");
- goto fail;
+ return -EINVAL;
}
info->size = be64_to_cpu(info->size);
if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
error_prepend(errp, "Failed to read export flags: ");
- goto fail;
+ return -EINVAL;
}
oldflags = be32_to_cpu(oldflags);
if (oldflags & ~0xffff) {
error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
- goto fail;
+ return -EINVAL;
}
info->flags = oldflags;
} else {
error_setg(errp, "Bad server magic received: 0x%" PRIx64, magic);
- goto fail;
+ return -EINVAL;
}
trace_nbd_receive_negotiate_size_flags(info->size, info->flags);
if (zeroes && nbd_drop(ioc, 124, errp) < 0) {
error_prepend(errp, "Failed to read reserved block: ");
- goto fail;
+ return -EINVAL;
}
- rc = 0;
-
-fail:
- return rc;
+ return 0;
}
#ifdef __linux__
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 16/22] nbd/client: Split handshake into two functions
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (14 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 15/22] nbd/client: Refactor return of nbd_receive_negotiate() Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 17/22] nbd/client: Pull out oldstyle size determination Eric Blake
` (5 subsequent siblings)
21 siblings, 0 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
An upcoming patch will add the ability for qemu-nbd to list
the services provided by an NBD server. Share the common
code of the TLS handshake by splitting the initial exchange
into a separate function, leaving only the export handling
in the original function. Functionally, there should be no
change in behavior in this patch, although some of the code
motion may be difficult to follow due to indentation changes
(view with 'git diff -w' for a smaller changeset).
I considered an enum for the return code coordinating state
between the two functions, but in the end just settled with
ample comments.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
v2: improve commit message, tweak code comment formatting
---
nbd/client.c | 144 +++++++++++++++++++++++++++++++----------------
nbd/trace-events | 2 +-
2 files changed, 95 insertions(+), 51 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index 64a0e5760c3..8b0ae20fae8 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -806,21 +806,24 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
}
}
-int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
- const char *hostname, QIOChannel **outioc,
- NBDExportInfo *info, Error **errp)
+/*
+ * nbd_start_negotiate:
+ * Start the handshake to the server. After a positive return, the server
+ * is ready to accept additional NBD_OPT requests.
+ * Returns: negative errno: failure talking to server
+ * 0: server is oldstyle, client must still parse export size
+ * 1: server is newstyle, but can only accept EXPORT_NAME
+ * 2: server is newstyle, but lacks structured replies
+ * 3: server is newstyle and set up for structured replies
+ */
+static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, QIOChannel **outioc,
+ bool structured_reply, bool *zeroes,
+ Error **errp)
{
uint64_t magic;
- bool zeroes = true;
- bool structured_reply = info->structured_reply;
- bool base_allocation = info->base_allocation;
- trace_nbd_receive_negotiate(tlscreds, hostname ? hostname : "<null>");
-
- assert(info->name);
- trace_nbd_receive_negotiate_name(info->name);
- info->structured_reply = false;
- info->base_allocation = false;
+ trace_nbd_start_negotiate(tlscreds, hostname ? hostname : "<null>");
if (outioc) {
*outioc = NULL;
@@ -865,7 +868,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
clientflags |= NBD_FLAG_C_FIXED_NEWSTYLE;
}
if (globalflags & NBD_FLAG_NO_ZEROES) {
- zeroes = false;
+ *zeroes = false;
clientflags |= NBD_FLAG_C_NO_ZEROES;
}
/* client requested flags */
@@ -887,7 +890,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
}
}
if (fixedNewStyle) {
- int result;
+ int result = 0;
if (structured_reply) {
result = nbd_request_simple_option(ioc,
@@ -896,39 +899,85 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
if (result < 0) {
return -EINVAL;
}
- info->structured_reply = result == 1;
}
+ return 2 + result;
+ } else {
+ return 1;
+ }
+ } else if (magic == NBD_CLIENT_MAGIC) {
+ if (tlscreds) {
+ error_setg(errp, "Server does not support STARTTLS");
+ return -EINVAL;
+ }
+ return 0;
+ } else {
+ error_setg(errp, "Bad server magic received: 0x%" PRIx64, magic);
+ return -EINVAL;
+ }
+}
- if (info->structured_reply && base_allocation) {
- result = nbd_negotiate_simple_meta_context(ioc, info, errp);
- if (result < 0) {
- return -EINVAL;
- }
- info->base_allocation = result == 1;
- }
+/*
+ * nbd_receive_negotiate:
+ * Connect to server, complete negotiation, and move into transmission phase.
+ * Returns: negative errno: failure talking to server
+ * 0: server is connected
+ */
+int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, QIOChannel **outioc,
+ NBDExportInfo *info, Error **errp)
+{
+ int result;
+ bool zeroes = true;
+ bool base_allocation = info->base_allocation;
+ uint32_t oldflags;
+
+ assert(info->name);
+ trace_nbd_receive_negotiate_name(info->name);
+
+ result = nbd_start_negotiate(ioc, tlscreds, hostname, outioc,
+ info->structured_reply, &zeroes, errp);
+
+ info->structured_reply = false;
+ info->base_allocation = false;
+ if (tlscreds && *outioc) {
+ ioc = *outioc;
+ }
- /* Try NBD_OPT_GO first - if it works, we are done (it
- * also gives us a good message if the server requires
- * TLS). If it is not available, fall back to
- * NBD_OPT_LIST for nicer error messages about a missing
- * export, then use NBD_OPT_EXPORT_NAME. */
- result = nbd_opt_go(ioc, info, errp);
+ switch (result) {
+ case 3: /* newstyle, with structured replies */
+ info->structured_reply = true;
+ if (base_allocation) {
+ result = nbd_negotiate_simple_meta_context(ioc, info, errp);
if (result < 0) {
return -EINVAL;
}
- if (result > 0) {
- return 0;
- }
- /* Check our desired export is present in the
- * server export list. Since NBD_OPT_EXPORT_NAME
- * cannot return an error message, running this
- * query gives us better error reporting if the
- * export name is not available.
- */
- if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
- return -EINVAL;
- }
+ info->base_allocation = result == 1;
}
+ /* fall through */
+ case 2: /* newstyle, try OPT_GO */
+ /* Try NBD_OPT_GO first - if it works, we are done (it
+ * also gives us a good message if the server requires
+ * TLS). If it is not available, fall back to
+ * NBD_OPT_LIST for nicer error messages about a missing
+ * export, then use NBD_OPT_EXPORT_NAME. */
+ result = nbd_opt_go(ioc, info, errp);
+ if (result < 0) {
+ return -EINVAL;
+ }
+ if (result > 0) {
+ return 0;
+ }
+ /* Check our desired export is present in the
+ * server export list. Since NBD_OPT_EXPORT_NAME
+ * cannot return an error message, running this
+ * query gives us better error reporting if the
+ * export name is not available.
+ */
+ if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
+ return -EINVAL;
+ }
+ /* fall through */
+ case 1: /* newstyle, but limited to EXPORT_NAME */
/* write the export name request */
if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, info->name,
errp) < 0) {
@@ -947,17 +996,12 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
return -EINVAL;
}
info->flags = be16_to_cpu(info->flags);
- } else if (magic == NBD_CLIENT_MAGIC) {
- uint32_t oldflags;
-
+ break;
+ case 0: /* oldstyle, parse length and flags */
if (*info->name) {
error_setg(errp, "Server does not support non-empty export names");
return -EINVAL;
}
- if (tlscreds) {
- error_setg(errp, "Server does not support STARTTLS");
- return -EINVAL;
- }
if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
error_prepend(errp, "Failed to read export length: ");
@@ -975,9 +1019,9 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
return -EINVAL;
}
info->flags = oldflags;
- } else {
- error_setg(errp, "Bad server magic received: 0x%" PRIx64, magic);
- return -EINVAL;
+ break;
+ default:
+ return result;
}
trace_nbd_receive_negotiate_size_flags(info->size, info->flags);
diff --git a/nbd/trace-events b/nbd/trace-events
index 02956c96042..922b36d1843 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -13,7 +13,7 @@ nbd_receive_starttls_new_client(void) "Setting up TLS"
nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
nbd_opt_meta_reply(const char *optname, const char *context, uint32_t id) "Received %s mapping of %s to id %" PRIu32
-nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
+nbd_start_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are 0x%" PRIx32
nbd_receive_negotiate_name(const char *name) "Requesting NBD export name '%s'"
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 17/22] nbd/client: Pull out oldstyle size determination
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (15 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 16/22] nbd/client: Split handshake into two functions Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 15:31 ` Richard W.M. Jones
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 18/22] nbd/client: Add nbd_receive_export_list() Eric Blake
` (4 subsequent siblings)
21 siblings, 1 reply; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
Another refactoring creating nbd_negotiate_finish_oldstyle()
for further reuse during 'qemu-nbd --list'.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v2: new patch [Vladimir]
---
nbd/client.c | 49 ++++++++++++++++++++++++++++++++-----------------
1 file changed, 32 insertions(+), 17 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index 8b0ae20fae8..4bdfba43068 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -811,7 +811,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
* Start the handshake to the server. After a positive return, the server
* is ready to accept additional NBD_OPT requests.
* Returns: negative errno: failure talking to server
- * 0: server is oldstyle, client must still parse export size
+ * 0: server is oldstyle, must call nbd_negotiate_finish_oldstyle
* 1: server is newstyle, but can only accept EXPORT_NAME
* 2: server is newstyle, but lacks structured replies
* 3: server is newstyle and set up for structured replies
@@ -916,6 +916,36 @@ static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
}
}
+/*
+ * nbd_negotiate_finish_oldstyle:
+ * Populate @info with the size and export flags from an oldstyle server,
+ * but does not consume 124 bytes of reserved zero padding.
+ * Returns 0 on success, -1 with @errp set on failure
+ */
+static int nbd_negotiate_finish_oldstyle(QIOChannel *ioc, NBDExportInfo *info,
+ Error **errp)
+{
+ uint32_t oldflags;
+
+ if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
+ error_prepend(errp, "Failed to read export length: ");
+ return -EINVAL;
+ }
+ info->size = be64_to_cpu(info->size);
+
+ if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
+ error_prepend(errp, "Failed to read export flags: ");
+ return -EINVAL;
+ }
+ oldflags = be32_to_cpu(oldflags);
+ if (oldflags & ~0xffff) {
+ error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
+ return -EINVAL;
+ }
+ info->flags = oldflags;
+ return 0;
+}
+
/*
* nbd_receive_negotiate:
* Connect to server, complete negotiation, and move into transmission phase.
@@ -929,7 +959,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
int result;
bool zeroes = true;
bool base_allocation = info->base_allocation;
- uint32_t oldflags;
assert(info->name);
trace_nbd_receive_negotiate_name(info->name);
@@ -1002,23 +1031,9 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
error_setg(errp, "Server does not support non-empty export names");
return -EINVAL;
}
-
- if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
- error_prepend(errp, "Failed to read export length: ");
+ if (nbd_negotiate_finish_oldstyle(ioc, info, errp) < 0) {
return -EINVAL;
}
- info->size = be64_to_cpu(info->size);
-
- if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
- error_prepend(errp, "Failed to read export flags: ");
- return -EINVAL;
- }
- oldflags = be32_to_cpu(oldflags);
- if (oldflags & ~0xffff) {
- error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
- return -EINVAL;
- }
- info->flags = oldflags;
break;
default:
return result;
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 18/22] nbd/client: Add nbd_receive_export_list()
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (16 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 17/22] nbd/client: Pull out oldstyle size determination Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 15:42 ` Richard W.M. Jones
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 19/22] nbd/client: Add meta contexts to nbd_receive_export_list() Eric Blake
` (3 subsequent siblings)
21 siblings, 1 reply; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel
Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block, Kevin Wolf, Max Reitz
We want to be able to detect whether a given qemu NBD server is
exposing the right export(s) and dirty bitmaps, at least for
regression testing. We could use 'nbd-client -l' from the upstream
NBD project to list exports, but it's annoying to rely on
out-of-tree binaries; furthermore, nbd-client doesn't necessarily
know about all of the qemu NBD extensions. Thus, we plan on adding
a new mode to qemu-nbd that merely sniffs all possible information
from the server during handshake phase, then disconnects and dumps
the information.
This patch adds the low-level client code for grabbing the list
of exports. It benefits from the recent refactoring patches, as
well as a minor tweak of changing nbd_opt_go() to nbd_opt_info_or_go(),
in order to share as much code as possible when it comes to doing
validation of server replies. The resulting information is stored
in an array of NBDExportInfo which has been expanded to any
description string, along with a convenience function for freeing
the list.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v2: split out oldstyle size computation into earlier patch [Vladimir]
rename nbd_opt_info_or_go [Rich]
split out collection of meta context collection into later patch
---
include/block/nbd.h | 15 ++++-
nbd/client.c | 138 ++++++++++++++++++++++++++++++++++++++++++--
nbd/trace-events | 2 +-
3 files changed, 146 insertions(+), 9 deletions(-)
diff --git a/include/block/nbd.h b/include/block/nbd.h
index ae5fe28f486..09d2157efe0 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2016-2017 Red Hat, Inc.
+ * Copyright (C) 2016-2018 Red Hat, Inc.
* Copyright (C) 2005 Anthony Liguori <anthony@codemonkey.ws>
*
* Network Block Device
@@ -262,6 +262,9 @@ struct NBDExportInfo {
/* Set by client before nbd_receive_negotiate() */
bool request_sizes;
char *x_dirty_bitmap;
+
+ /* Set by client before nbd_receive_negotiate(), or by server results
+ * during nbd_receive_export_list() */
char *name; /* must be non-NULL */
/* In-out fields, set by client before nbd_receive_negotiate() and
@@ -269,7 +272,8 @@ struct NBDExportInfo {
bool structured_reply;
bool base_allocation; /* base:allocation context for NBD_CMD_BLOCK_STATUS */
- /* Set by server results during nbd_receive_negotiate() */
+ /* Set by server results during nbd_receive_negotiate() and
+ * nbd_receive_export_list() */
uint64_t size;
uint16_t flags;
uint32_t min_block;
@@ -277,12 +281,19 @@ struct NBDExportInfo {
uint32_t max_block;
uint32_t context_id;
+
+ /* Set by server results during nbd_receive_export_list() */
+ char *description;
};
typedef struct NBDExportInfo NBDExportInfo;
int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
const char *hostname, QIOChannel **outioc,
NBDExportInfo *info, Error **errp);
+void nbd_free_export_list(NBDExportInfo *info, int count);
+int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, NBDExportInfo **info,
+ Error **errp);
int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
Error **errp);
int nbd_send_request(QIOChannel *ioc, NBDRequest *request);
diff --git a/nbd/client.c b/nbd/client.c
index 4bdfba43068..0e6c575ccad 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -332,7 +332,8 @@ static int nbd_receive_list(QIOChannel *ioc, char **name, char **description,
* used, 0 if NBD_OPT_GO is unsupported (fall back to NBD_OPT_LIST and
* NBD_OPT_EXPORT_NAME in that case), and > 0 if the export is good to
* go (with the rest of @info populated). */
-static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
+static int nbd_opt_info_or_go(QIOChannel *ioc, uint32_t opt,
+ NBDExportInfo *info, Error **errp)
{
NBDOptionReply reply;
uint32_t len = strlen(info->name);
@@ -345,7 +346,8 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
* flags still 0 is a witness of a broken server. */
info->flags = 0;
- trace_nbd_opt_go_start(info->name);
+ assert(opt == NBD_OPT_GO || opt == NBD_OPT_INFO);
+ trace_nbd_opt_go_start(nbd_opt_lookup(opt), info->name);
buf = g_malloc(4 + len + 2 + 2 * info->request_sizes + 1);
stl_be_p(buf, len);
memcpy(buf + 4, info->name, len);
@@ -354,7 +356,7 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
if (info->request_sizes) {
stw_be_p(buf + 4 + len + 2, NBD_INFO_BLOCK_SIZE);
}
- error = nbd_send_option_request(ioc, NBD_OPT_GO,
+ error = nbd_send_option_request(ioc, opt,
4 + len + 2 + 2 * info->request_sizes,
buf, errp);
g_free(buf);
@@ -363,7 +365,7 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
}
while (1) {
- if (nbd_receive_option_reply(ioc, NBD_OPT_GO, &reply, errp) < 0) {
+ if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0) {
return -1;
}
error = nbd_handle_reply_err(ioc, &reply, errp);
@@ -868,7 +870,9 @@ static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
clientflags |= NBD_FLAG_C_FIXED_NEWSTYLE;
}
if (globalflags & NBD_FLAG_NO_ZEROES) {
- *zeroes = false;
+ if (zeroes) {
+ *zeroes = false;
+ }
clientflags |= NBD_FLAG_C_NO_ZEROES;
}
/* client requested flags */
@@ -989,7 +993,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
* TLS). If it is not available, fall back to
* NBD_OPT_LIST for nicer error messages about a missing
* export, then use NBD_OPT_EXPORT_NAME. */
- result = nbd_opt_go(ioc, info, errp);
+ result = nbd_opt_info_or_go(ioc, NBD_OPT_GO, info, errp);
if (result < 0) {
return -EINVAL;
}
@@ -1047,6 +1051,128 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
return 0;
}
+/* Clean up result of nbd_receive_export_list */
+void nbd_free_export_list(NBDExportInfo *info, int count)
+{
+ int i;
+
+ if (!info) {
+ return;
+ }
+
+ for (i = 0; i < count; i++) {
+ g_free(info[i].name);
+ g_free(info[i].description);
+ }
+ g_free(info);
+}
+
+/*
+ * nbd_receive_export_list:
+ * Query details about a server's exports, then disconnect without
+ * going into transmission phase. Return a count of the exports listed
+ * in @info by the server, or -1 on error. Caller must free @info using
+ * nbd_free_export_list().
+ */
+int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, NBDExportInfo **info,
+ Error **errp)
+{
+ int result;
+ int count = 0;
+ int i;
+ int rc;
+ int ret = -1;
+ NBDExportInfo *array = NULL;
+ QIOChannel *sioc = NULL;
+
+ *info = NULL;
+ result = nbd_start_negotiate(ioc, tlscreds, hostname, &sioc, true, NULL,
+ errp);
+ if (tlscreds && sioc) {
+ ioc = sioc;
+ }
+
+ switch (result) {
+ case 2:
+ case 3:
+ /* newstyle - use NBD_OPT_LIST to populate array, then try
+ * NBD_OPT_INFO on each array member. If structured replies
+ * are enabled, also try NBD_OPT_LIST_META_CONTEXT. */
+ if (nbd_send_option_request(ioc, NBD_OPT_LIST, 0, NULL, errp) < 0) {
+ goto out;
+ }
+ while (1) {
+ char *name;
+ char *desc;
+
+ rc = nbd_receive_list(ioc, &name, &desc, errp);
+ if (rc < 0) {
+ goto out;
+ } else if (rc == 0) {
+ break;
+ }
+ array = g_renew(NBDExportInfo, array, ++count);
+ memset(&array[count - 1], 0, sizeof(*array));
+ array[count - 1].name = name;
+ array[count - 1].description = desc;
+ array[count - 1].structured_reply = result == 3;
+ }
+
+ for (i = 0; i < count; i++) {
+ array[i].request_sizes = true;
+ rc = nbd_opt_info_or_go(ioc, NBD_OPT_INFO, &array[i], errp);
+ if (rc < 0) {
+ goto out;
+ } else if (rc == 0) {
+ /* Pointless to try rest of loop. If OPT_INFO doesn't work,
+ * it's unlikely that meta contexts work either */
+ break;
+ }
+
+ /* TODO: Grab meta contexts */
+ }
+
+ /* Send NBD_OPT_ABORT as a courtesy before hanging up */
+ nbd_send_opt_abort(ioc);
+ break;
+ case 1: /* newstyle, but limited to EXPORT_NAME */
+ error_setg(errp, "Server does not support export lists");
+ /* We can't even send NBD_OPT_ABORT, so merely hang up */
+ goto out;
+ case 0: /* oldstyle, parse length and flags */
+ array = g_new0(NBDExportInfo, 1);
+ array->name = g_strdup("");
+ count = 1;
+
+ if (nbd_negotiate_finish_oldstyle(ioc, array, errp) < 0) {
+ return -EINVAL;
+ }
+
+ /* Send NBD_CMD_DISC as a courtesy to the server, but ignore all
+ * errors now that we have the information we wanted. */
+ if (nbd_drop(ioc, 124, NULL) == 0) {
+ NBDRequest request = { .type = NBD_CMD_DISC };
+
+ nbd_send_request(ioc, &request);
+ }
+ break;
+ default:
+ goto out;
+ }
+
+ *info = array;
+ array = NULL;
+ ret = count;
+
+ out:
+ qio_channel_shutdown(ioc, QIO_CHANNEL_SHUTDOWN_BOTH, NULL);
+ qio_channel_close(ioc, NULL);
+ object_unref(OBJECT(sioc));
+ nbd_free_export_list(array, count);
+ return ret;
+}
+
#ifdef __linux__
int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
Error **errp)
diff --git a/nbd/trace-events b/nbd/trace-events
index 922b36d1843..a66bf891cc9 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -3,7 +3,7 @@ nbd_send_option_request(uint32_t opt, const char *name, uint32_t len) "Sending o
nbd_receive_option_reply(uint32_t option, const char *optname, uint32_t type, const char *typename, uint32_t length) "Received option reply %" PRIu32" (%s), type %" PRIu32" (%s), len %" PRIu32
nbd_reply_err_unsup(uint32_t option, const char *name) "server doesn't understand request %" PRIu32 " (%s), attempting fallback"
nbd_receive_list(const char *name, const char *desc) "export list includes '%s', description '%s'"
-nbd_opt_go_start(const char *name) "Attempting NBD_OPT_GO for export '%s'"
+nbd_opt_go_start(const char *opt, const char *name) "Attempting %s for export '%s'"
nbd_opt_go_success(void) "Export is good to go"
nbd_opt_go_info_unknown(int info, const char *name) "Ignoring unknown info %d (%s)"
nbd_opt_go_info_block_size(uint32_t minimum, uint32_t preferred, uint32_t maximum) "Block sizes are 0x%" PRIx32 ", 0x%" PRIx32 ", 0x%" PRIx32
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 19/22] nbd/client: Add meta contexts to nbd_receive_export_list()
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (17 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 18/22] nbd/client: Add nbd_receive_export_list() Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 15:59 ` Richard W.M. Jones
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 20/22] qemu-nbd: Add --list option Eric Blake
` (2 subsequent siblings)
21 siblings, 1 reply; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel
Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block, Kevin Wolf, Max Reitz
We want to be able to detect whether a given qemu NBD server is
exposing the right export(s) and dirty bitmaps, at least for
regression testing. We could use 'nbd-client -l' from the upstream
NBD project to list exports, but it's annoying to rely on
out-of-tree binaries; furthermore, nbd-client doesn't necessarily
know about all of the qemu NBD extensions. Thus, we plan on adding
a new mode to qemu-nbd that merely sniffs all possible information
from the server during handshake phase, then disconnects and dumps
the information.
This patch continues the work of the previous patch, by adding the
ability to track the list of available meta contexts into
NBDExportInfo. It benefits from the recent refactoring patches
with a new nbd_list_meta_contexts() that reuses much of the same
framework as setting a meta context.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v2: new patch to split out collection of meta context collection
s/free/g_free/ [Vladimir]
---
include/block/nbd.h | 2 ++
nbd/client.c | 41 +++++++++++++++++++++++++++++++++++++++--
2 files changed, 41 insertions(+), 2 deletions(-)
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 09d2157efe0..6d9fbb941d7 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -284,6 +284,8 @@ struct NBDExportInfo {
/* Set by server results during nbd_receive_export_list() */
char *description;
+ int n_contexts;
+ char **contexts;
};
typedef struct NBDExportInfo NBDExportInfo;
diff --git a/nbd/client.c b/nbd/client.c
index 0e6c575ccad..d392b5e8bee 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -808,6 +808,36 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
}
}
+/*
+ * nbd_list_meta_contexts:
+ * Request the server to list all meta contexts for export @info->name.
+ * return 0 if list is complete (even if empty),
+ * -1 with errp set for any other error
+ */
+static int nbd_list_meta_contexts(QIOChannel *ioc,
+ NBDExportInfo *info,
+ Error **errp)
+{
+ int ret;
+
+ if (nbd_send_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
+ info->name, NULL, errp) < 0) {
+ return -1;
+ }
+
+ while (1) {
+ char *context;
+
+ ret = nbd_receive_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
+ &context, NULL, errp);
+ if (ret <= 0) {
+ return ret;
+ }
+ info->contexts = g_renew(char *, info->contexts, ++info->n_contexts);
+ info->contexts[info->n_contexts - 1] = context;
+ }
+}
+
/*
* nbd_start_negotiate:
* Start the handshake to the server. After a positive return, the server
@@ -1054,7 +1084,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
/* Clean up result of nbd_receive_export_list */
void nbd_free_export_list(NBDExportInfo *info, int count)
{
- int i;
+ int i, j;
if (!info) {
return;
@@ -1063,6 +1093,10 @@ void nbd_free_export_list(NBDExportInfo *info, int count)
for (i = 0; i < count; i++) {
g_free(info[i].name);
g_free(info[i].description);
+ for (j = 0; j < info[i].n_contexts; j++) {
+ g_free(info[i].contexts[j]);
+ }
+ g_free(info[i].contexts);
}
g_free(info);
}
@@ -1130,7 +1164,10 @@ int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
break;
}
- /* TODO: Grab meta contexts */
+ if (result == 3 &&
+ nbd_list_meta_contexts(ioc, &array[i], errp) < 0) {
+ goto out;
+ }
}
/* Send NBD_OPT_ABORT as a courtesy before hanging up */
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 20/22] qemu-nbd: Add --list option
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (18 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 19/22] nbd/client: Add meta contexts to nbd_receive_export_list() Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 16:02 ` Richard W.M. Jones
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 21/22] nbd/client: Work around 3.0 bug for listing meta contexts Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 22/22] iotests: Enhance 223, 233 to cover 'qemu-nbd --list' Eric Blake
21 siblings, 1 reply; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
We want to be able to detect whether a given qemu NBD server is
exposing the right export(s) and dirty bitmaps, at least for
regression testing. We could use 'nbd-client -l' from the upstream
NBD project to list exports, but it's annoying to rely on
out-of-tree binaries; furthermore, nbd-client doesn't necessarily
know about all of the qemu NBD extensions. Thus, it is time to add
a new mode to qemu-nbd that merely sniffs all possible information
from the server during handshake phase, then disconnects and dumps
the information.
This patch actually implements --list/-L, while reusing other
options such as --tls-creds for now designating how to connect
as the client (rather than their non-list usage of how to operate
as the server).
I debated about adding this functionality to something akin to
'qemu-img info' - but that tool does not readily lend itself
to connecting to an arbitrary NBD server without also tying to
a specific export (I may, however, still add ImageInfoSpecificNBD
for reporting the bitmaps available when connecting to a single
export). And, while it may feel a bit odd that normally
qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not
really making the qemu-nbd binary that much larger, because
'qemu-nbd -c' has to operate as both server and client
simultaneously across two threads when feeding the kernel module
for /dev/nbdN access.
Sample output:
$ qemu-nbd -L
exports available: 1
export: ''
size: 65536
flags: 0x4ed ( flush fua trim zeroes df cache )
min block: 512
opt block: 4096
max block: 33554432
available meta contexts: 1
base:allocation
Note that the output only lists sizes if the server sent
NBD_FLAG_HAS_FLAGS, because a newstyle server does not give
the size otherwise. It has the side effect that for really
old servers that did not send any flags, the size is not
output even though it was available. However, I'm not too
concerned about that - oldstyle servers are (rightfully)
getting less common to encounter (qemu 3.0 was the last
version where we even serve it), and most existing servers
that still even offer oldstyle negotiation (such as nbdkit)
still send flags (since that was added to the NBD protocol
in 2007 to permit read-only connections).
Not done here, but maybe worth future experiments: capture
the meat of NBDExportInfo into a QAPI struct, and use the
generated QAPI pretty-printers instead of hand-rolling our
output loop. It would also permit us to add a JSON output
mode for machine parsing.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v2: commit message improvements [Vladimir]
wording tweak to --help output [Vladimir]
update man page documentation
---
qemu-nbd.texi | 30 ++++++++--
qemu-nbd.c | 155 +++++++++++++++++++++++++++++++++++++++++++++-----
2 files changed, 166 insertions(+), 19 deletions(-)
diff --git a/qemu-nbd.texi b/qemu-nbd.texi
index 0e24c2801ee..1e168c10850 100644
--- a/qemu-nbd.texi
+++ b/qemu-nbd.texi
@@ -2,6 +2,8 @@
@c man begin SYNOPSIS
@command{qemu-nbd} [OPTION]... @var{filename}
+@command{qemu-nbd} @option{-L} [OPTION]...
+
@command{qemu-nbd} @option{-d} @var{dev}
@c man end
@end example
@@ -10,8 +12,9 @@
Provide access to various QEMU NBD features. Most commonly used to
export a QEMU disk image using the NBD protocol as a server, but can
-also be used (on Linux) to manage kernel bindings of a /dev/nbdX
-block device to a QEMU server.
+also be used as a client to query properties of a remote server, or
+(on Linux) to manage kernel bindings of a /dev/nbdX block device to a
+QEMU server.
@c man end
@@ -28,13 +31,15 @@ See the @code{qemu(1)} manual page for full details of the properties
supported. The common object types that it makes sense to define are the
@code{secret} object, which is used to supply passwords and/or encryption
keys, and the @code{tls-creds} object, which is used to supply TLS
-credentials for the qemu-nbd server.
+credentials for the qemu-nbd server or client.
@item -p, --port=@var{port}
-The TCP port to listen on (default @samp{10809}).
+The TCP port to listen on as a server, or connect to as a client
+(default @samp{10809}).
@item -o, --offset=@var{offset}
The offset into the image.
@item -b, --bind=@var{iface}
-The interface to bind to (default @samp{0.0.0.0}).
+The interface to bind to as a server, or connect to as a client
+(default @samp{0.0.0.0}).
@item -k, --socket=@var{path}
Use a unix socket with path @var{path}.
@item --image-opts
@@ -88,10 +93,14 @@ Set the NBD volume export name (default of a zero-length string).
@item -D, --description=@var{description}
Set the NBD volume export description, as a human-readable
string.
+@item -L, --list
+Connect as a client and list all details about the exports exposed by
+a remote NBD server.
@item --tls-creds=ID
Enable mandatory TLS encryption for the server by setting the ID
of the TLS credentials object previously created with the --object
-option.
+option; or provide the credentials needed for connecting as a client
+in list mode.
@item --fork
Fork off the server process and exit the parent once the server is running.
@item -v, --verbose
@@ -149,6 +158,15 @@ qemu-nbd -c /dev/nbd0 -f qcow2 file.qcow2
qemu-nbd -d /dev/nbd0
@end example
+Query a remote server to see details about what export(s) it is
+serving on port 10809, and authenticating via PSK:
+
+@example
+qemu-nbd \
+ --object tls-creds-psk,id=tls0,dir=/tmp/keys,username=eblake,endpoint=client \
+ --tls-creds tls0 -L -b remote.example.com
+@end example
+
@c man end
@ignore
diff --git a/qemu-nbd.c b/qemu-nbd.c
index 545a6dfbbc7..4541bbd07d3 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -76,7 +76,8 @@ static void usage(const char *name)
{
(printf) (
"Usage: %s [OPTIONS] FILE\n"
-"QEMU Disk Network Block Device Server\n"
+" or: %s -L [OPTIONS]\n"
+"QEMU Disk Network Block Device Utility\n"
"\n"
" -h, --help display this help and exit\n"
" -V, --version output version information and exit\n"
@@ -97,6 +98,7 @@ static void usage(const char *name)
" -P, --partition=NUM only expose partition NUM\n"
"\n"
"General purpose options:\n"
+" -L, --list list exports available from another NBD server\n"
" --object type,id=ID,... define an object such as 'secret' for providing\n"
" passwords and/or encryption keys\n"
" --tls-creds=ID use id of an earlier --object to provide TLS\n"
@@ -130,7 +132,7 @@ static void usage(const char *name)
" --image-opts treat FILE as a full set of image options\n"
"\n"
QEMU_HELP_BOTTOM "\n"
- , name, NBD_DEFAULT_PORT, "DEVICE");
+ , name, name, NBD_DEFAULT_PORT, "DEVICE");
}
static void version(const char *name)
@@ -242,6 +244,92 @@ static void termsig_handler(int signum)
}
+static int qemu_nbd_client_list(SocketAddress *saddr, QCryptoTLSCreds *tls,
+ const char *hostname)
+{
+ int ret = EXIT_FAILURE;
+ int rc;
+ Error *err = NULL;
+ QIOChannelSocket *sioc;
+ NBDExportInfo *list;
+ int i, j;
+
+ sioc = qio_channel_socket_new();
+ if (qio_channel_socket_connect_sync(sioc, saddr, &err) < 0) {
+ error_report_err(err);
+ goto out;
+ }
+ rc = nbd_receive_export_list(QIO_CHANNEL(sioc), tls, hostname, &list,
+ &err);
+ if (rc < 0) {
+ if (err) {
+ error_report_err(err);
+ }
+ goto out_socket;
+ }
+ printf("exports available: %d\n", rc);
+ for (i = 0; i < rc; i++) {
+ printf(" export: '%s'\n", list[i].name);
+ if (list[i].description && *list[i].description) {
+ printf(" description: %s\n", list[i].description);
+ }
+ if (list[i].flags & NBD_FLAG_HAS_FLAGS) {
+ printf(" size: %" PRIu64 "\n", list[i].size);
+ printf(" flags: 0x%x (", list[i].flags);
+ if (list[i].flags & NBD_FLAG_READ_ONLY) {
+ printf(" readonly");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_FLUSH) {
+ printf(" flush");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_FUA) {
+ printf(" fua");
+ }
+ if (list[i].flags & NBD_FLAG_ROTATIONAL) {
+ printf(" rotational");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_TRIM) {
+ printf(" trim");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_WRITE_ZEROES) {
+ printf(" zeroes");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_DF) {
+ printf(" df");
+ }
+ if (list[i].flags & NBD_FLAG_CAN_MULTI_CONN) {
+ printf(" multi");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_RESIZE) {
+ printf(" resize");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_CACHE) {
+ printf(" cache");
+ }
+ printf(" )\n");
+ }
+ if (list[i].min_block) {
+ printf(" min block: %u\n", list[i].min_block);
+ printf(" opt block: %u\n", list[i].opt_block);
+ printf(" max block: %u\n", list[i].max_block);
+ }
+ if (list[i].n_contexts) {
+ printf(" available meta contexts: %d\n", list[i].n_contexts);
+ for (j = 0; j < list[i].n_contexts; j++) {
+ printf(" %s\n", list[i].contexts[j]);
+ }
+ }
+ }
+ nbd_free_export_list(list, rc);
+
+ ret = EXIT_SUCCESS;
+ out_socket:
+ object_unref(OBJECT(sioc));
+ out:
+ return ret;
+}
+
+
#if HAVE_NBD_DEVICE
static void *show_parts(void *arg)
{
@@ -424,7 +512,8 @@ static QemuOptsList qemu_object_opts = {
-static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, Error **errp)
+static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, bool list,
+ Error **errp)
{
Object *obj;
QCryptoTLSCreds *creds;
@@ -444,10 +533,18 @@ static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, Error **errp)
return NULL;
}
- if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_SERVER) {
- error_setg(errp,
- "Expecting TLS credentials with a server endpoint");
- return NULL;
+ if (list) {
+ if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT) {
+ error_setg(errp,
+ "Expecting TLS credentials with a client endpoint");
+ return NULL;
+ }
+ } else {
+ if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_SERVER) {
+ error_setg(errp,
+ "Expecting TLS credentials with a server endpoint");
+ return NULL;
+ }
}
object_ref(obj);
return creds;
@@ -470,7 +567,8 @@ static void setup_address_and_port(const char **address, const char **port)
static const char *socket_activation_validate_opts(const char *device,
const char *sockpath,
const char *address,
- const char *port)
+ const char *port,
+ bool list)
{
if (device != NULL) {
return "NBD device can't be set when using socket activation";
@@ -488,6 +586,10 @@ static const char *socket_activation_validate_opts(const char *device,
return "TCP port number can't be set when using socket activation";
}
+ if (list) {
+ return "List mode is incompatible with socket activation";
+ }
+
return NULL;
}
@@ -511,7 +613,7 @@ int main(int argc, char **argv)
off_t fd_size;
QemuOpts *sn_opts = NULL;
const char *sn_id_or_name = NULL;
- const char *sopt = "hVb:o:p:rsnP:c:dvk:e:f:tl:x:T:D:";
+ const char *sopt = "hVb:o:p:rsnP:c:dvk:e:f:tl:x:T:D:L";
struct option lopt[] = {
{ "help", no_argument, NULL, 'h' },
{ "version", no_argument, NULL, 'V' },
@@ -523,6 +625,7 @@ int main(int argc, char **argv)
{ "partition", required_argument, NULL, 'P' },
{ "connect", required_argument, NULL, 'c' },
{ "disconnect", no_argument, NULL, 'd' },
+ { "list", no_argument, NULL, 'L' },
{ "snapshot", no_argument, NULL, 's' },
{ "load-snapshot", required_argument, NULL, 'l' },
{ "nocache", no_argument, NULL, 'n' },
@@ -557,13 +660,14 @@ int main(int argc, char **argv)
Error *local_err = NULL;
BlockdevDetectZeroesOptions detect_zeroes = BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF;
QDict *options = NULL;
- const char *export_name = ""; /* Default export name */
+ const char *export_name = NULL;
const char *export_description = NULL;
const char *tlscredsid = NULL;
bool imageOpts = false;
bool writethrough = true;
char *trace_file = NULL;
bool fork_process = false;
+ bool list = false;
int old_stderr = -1;
unsigned socket_activation;
@@ -755,13 +859,32 @@ int main(int argc, char **argv)
case QEMU_NBD_OPT_FORK:
fork_process = true;
break;
+ case 'L':
+ list = true;
+ break;
}
}
- if ((argc - optind) != 1) {
+ if (list) {
+ if (argc != optind) {
+ error_report("List mode is incompatible with a file name");
+ exit(EXIT_FAILURE);
+ }
+ if (export_name || export_description || dev_offset || partition ||
+ device || disconnect || fmt || sn_id_or_name) {
+ error_report("List mode is incompatible with per-device settings");
+ exit(EXIT_FAILURE);
+ }
+ if (fork_process) {
+ error_report("List mode is incompatible with forking");
+ exit(EXIT_FAILURE);
+ }
+ } else if ((argc - optind) != 1) {
error_report("Invalid number of arguments");
error_printf("Try `%s --help' for more information.\n", argv[0]);
exit(EXIT_FAILURE);
+ } else if (!export_name) {
+ export_name = "";
}
qemu_opts_foreach(&qemu_object_opts,
@@ -780,7 +903,8 @@ int main(int argc, char **argv)
} else {
/* Using socket activation - check user didn't use -p etc. */
const char *err_msg = socket_activation_validate_opts(device, sockpath,
- bindto, port);
+ bindto, port,
+ list);
if (err_msg != NULL) {
error_report("%s", err_msg);
exit(EXIT_FAILURE);
@@ -803,7 +927,7 @@ int main(int argc, char **argv)
error_report("TLS is not supported with a host device");
exit(EXIT_FAILURE);
}
- tlscreds = nbd_get_tls_creds(tlscredsid, &local_err);
+ tlscreds = nbd_get_tls_creds(tlscredsid, list, &local_err);
if (local_err) {
error_report("Failed to get TLS creds %s",
error_get_pretty(local_err));
@@ -811,6 +935,11 @@ int main(int argc, char **argv)
}
}
+ if (list) {
+ saddr = nbd_build_socket_address(sockpath, bindto, port);
+ return qemu_nbd_client_list(saddr, tlscreds, bindto);
+ }
+
#if !HAVE_NBD_DEVICE
if (disconnect || device) {
error_report("Kernel /dev/nbdN support not available");
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 21/22] nbd/client: Work around 3.0 bug for listing meta contexts
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (19 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 20/22] qemu-nbd: Add --list option Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 22/22] iotests: Enhance 223, 233 to cover 'qemu-nbd --list' Eric Blake
21 siblings, 0 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block
Commit 3d068aff forgot to advertise available qemu: contexts
when the client requests a list with 0 queries. Furthermore,
3.0 shipped with a qemu-img hack of x-dirty-bitmap (commit
216ee365) that _silently_ acts as though the entire image is
clean if a requested bitmap is not present. Both bugs have
been recently fixed to give full output from the start, and
to refuse a connection if a requested x-dirty-bitmap was not
found.
Still, it is likely that there will be users that have to
work with a mix of old and new qemu versions, depending on
which features get backported where, at which point being
able to rely on 'qemu-img --list' output to know for sure
whether a given NBD export has the desired dirty bitmap is
much nicer than blindly connecting and risking that the
entire image may appear clean. We can make our --list code
smart enough to work around buggy servers by tracking
whether we've seen any qemu: replies in the original 0-query
list; if not, recurse to a single query on "qemu:" (which
may still have no replies, but then we know for sure we
didn't trip up on the server bug).
Signed-off-by: Eric Blake <eblake@redhat.com>
---
Done as a separate patch to make it easier to revert when we no
longer care about 3.0 servers
v2: rebase on top of earlier patch splits
---
nbd/client.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/nbd/client.c b/nbd/client.c
index d392b5e8bee..48fa6e10b92 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -21,6 +21,7 @@
#include "qapi/error.h"
#include "trace.h"
#include "nbd-internal.h"
+#include "qemu/cutils.h"
/* Definitions for opaque data types */
@@ -819,6 +820,8 @@ static int nbd_list_meta_contexts(QIOChannel *ioc,
Error **errp)
{
int ret;
+ int seen_any = false;
+ int seen_qemu = false;
if (nbd_send_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
info->name, NULL, errp) < 0) {
@@ -830,9 +833,25 @@ static int nbd_list_meta_contexts(QIOChannel *ioc,
ret = nbd_receive_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
&context, NULL, errp);
+ if (ret == 0 && seen_any && !seen_qemu) {
+ /*
+ * Work around qemu 3.0 bug: the server forgot to send
+ * "qemu:" replies to 0 queries. If we saw at least one
+ * reply (probably base:allocation), but none of them were
+ * qemu:, then run a more specific query to make sure.
+ */
+ seen_qemu = true;
+ if (nbd_send_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
+ info->name, "qemu:", errp) < 0) {
+ return -1;
+ }
+ continue;
+ }
if (ret <= 0) {
return ret;
}
+ seen_any = true;
+ seen_qemu |= strstart(context, "qemu:", NULL);
info->contexts = g_renew(char *, info->contexts, ++info->n_contexts);
info->contexts[info->n_contexts - 1] = context;
}
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Qemu-devel] [PATCH v2 22/22] iotests: Enhance 223, 233 to cover 'qemu-nbd --list'
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
` (20 preceding siblings ...)
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 21/22] nbd/client: Work around 3.0 bug for listing meta contexts Eric Blake
@ 2018-12-15 13:53 ` Eric Blake
21 siblings, 0 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-15 13:53 UTC (permalink / raw)
To: qemu-devel
Cc: nsoffer, jsnow, rjones, vsementsov, qemu-block, Kevin Wolf, Max Reitz
Any good new feature deserves some regression testing :)
Coverage includes:
- 223: what happens when there are 0 or more than 1 export,
proof that we can see multiple contexts including qemu:dirty-bitmap
- 233: proof that we can list over TLS, and that mix-and-match of
plain/TLS listings will behave sanely
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
v2: Factor out repeated --object text [Vladimir]
---
tests/qemu-iotests/223 | 2 ++
tests/qemu-iotests/223.out | 20 ++++++++++++++++++++
tests/qemu-iotests/233 | 19 +++++++++++++------
tests/qemu-iotests/233.out | 15 +++++++++++++++
4 files changed, 50 insertions(+), 6 deletions(-)
diff --git a/tests/qemu-iotests/223 b/tests/qemu-iotests/223
index 397b865d347..e64747a9a61 100755
--- a/tests/qemu-iotests/223
+++ b/tests/qemu-iotests/223
@@ -119,6 +119,7 @@ _send_qemu_cmd $QEMU_HANDLE '{"execute":"x-block-dirty-bitmap-disable",
_send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-start",
"arguments":{"addr":{"type":"unix",
"data":{"path":"'"$TEST_DIR/nbd"'"}}}}' "return"
+$QEMU_NBD_PROG -L -k "$TEST_DIR/nbd"
_send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add",
"arguments":{"device":"n"}}' "return"
_send_qemu_cmd $QEMU_HANDLE '{"execute":"x-nbd-server-add-bitmap",
@@ -127,6 +128,7 @@ _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add",
"arguments":{"device":"n", "name":"n2"}}' "return"
_send_qemu_cmd $QEMU_HANDLE '{"execute":"x-nbd-server-add-bitmap",
"arguments":{"name":"n2", "bitmap":"b2"}}' "return"
+$QEMU_NBD_PROG -L -k "$TEST_DIR/nbd"
echo
echo "=== Contrast normal status to large granularity dirty-bitmap ==="
diff --git a/tests/qemu-iotests/223.out b/tests/qemu-iotests/223.out
index de417477de0..3342bff3447 100644
--- a/tests/qemu-iotests/223.out
+++ b/tests/qemu-iotests/223.out
@@ -29,10 +29,30 @@ wrote 2097152/2097152 bytes at offset 2097152
{"return": {}}
{"return": {}}
{"return": {}}
+exports available: 0
{"return": {}}
{"return": {}}
{"return": {}}
{"return": {}}
+exports available: 2
+ export: 'n'
+ size: 4194304
+ flags: 0x4ef ( readonly flush fua trim zeroes df cache )
+ min block: 512
+ opt block: 4096
+ max block: 33554432
+ available meta contexts: 2
+ base:allocation
+ qemu:dirty-bitmap:b
+ export: 'n2'
+ size: 4194304
+ flags: 0x4ef ( readonly flush fua trim zeroes df cache )
+ min block: 512
+ opt block: 4096
+ max block: 33554432
+ available meta contexts: 2
+ base:allocation
+ qemu:dirty-bitmap:b2
=== Contrast normal status to large granularity dirty-bitmap ===
diff --git a/tests/qemu-iotests/233 b/tests/qemu-iotests/233
index 1814efe3333..a6ef7b20fb4 100755
--- a/tests/qemu-iotests/233
+++ b/tests/qemu-iotests/233
@@ -68,10 +68,12 @@ echo
echo "== check TLS client to plain server fails =="
nbd_server_start_tcp_socket -f $IMGFMT "$TEST_IMG"
-$QEMU_IMG info --image-opts \
- --object tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0 \
+obj=tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0
+$QEMU_IMG info --image-opts --object $obj \
driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port --object $obj \
+ --tls-creds=tls0
nbd_server_stop
@@ -84,20 +86,25 @@ nbd_server_start_tcp_socket \
-f $IMGFMT "$TEST_IMG"
$QEMU_IMG info nbd://localhost:$nbd_tcp_port 2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port
echo
echo "== check TLS works =="
-$QEMU_IMG info --image-opts \
- --object tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0 \
+obj=tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0
+$QEMU_IMG info --image-opts --object $obj \
driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port --object $obj \
+ --tls-creds=tls0
echo
echo "== check TLS with different CA fails =="
-$QEMU_IMG info --image-opts \
- --object tls-creds-x509,dir=${tls_dir}/client2,endpoint=client,id=tls0 \
+obj=tls-creds-x509,dir=${tls_dir}/client2,endpoint=client,id=tls0
+$QEMU_IMG info --image-opts --object $obj \
driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port --object $obj \
+ --tls-creds=tls0
echo
echo "== perform I/O over TLS =="
diff --git a/tests/qemu-iotests/233.out b/tests/qemu-iotests/233.out
index 5f416721b03..ab669488669 100644
--- a/tests/qemu-iotests/233.out
+++ b/tests/qemu-iotests/233.out
@@ -15,20 +15,35 @@ wrote 1048576/1048576 bytes at offset 1048576
== check TLS client to plain server fails ==
qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': Denied by server for option 5 (starttls)
server reported: TLS not configured
+qemu-nbd: Denied by server for option 5 (starttls)
+server reported: TLS not configured
== check plain client to TLS server fails ==
qemu-img: Could not open 'nbd://localhost:PORT': TLS negotiation required before option 8 (structured reply)
server reported: Option 0x8 not permitted before TLS
+qemu-nbd: TLS negotiation required before option 8 (structured reply)
+server reported: Option 0x8 not permitted before TLS
== check TLS works ==
image: nbd://127.0.0.1:PORT
file format: nbd
virtual size: 64M (67108864 bytes)
disk size: unavailable
+exports available: 1
+ export: ''
+ size: 67108864
+ flags: 0x4ed ( flush fua trim zeroes df cache )
+ min block: 512
+ opt block: 4096
+ max block: 33554432
+ available meta contexts: 1
+ base:allocation
== check TLS with different CA fails ==
qemu-nbd: option negotiation failed: Verify failed: No certificate was found.
qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': The certificate hasn't got a known issuer
+qemu-nbd: option negotiation failed: Verify failed: No certificate was found.
+qemu-nbd: The certificate hasn't got a known issuer
== perform I/O over TLS ==
read 1048576/1048576 bytes at offset 1048576
--
2.17.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 02/22] nbd: Document timeline of various features
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 02/22] nbd: Document timeline of various features Eric Blake
@ 2018-12-15 14:02 ` Richard W.M. Jones
2018-12-18 13:03 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 55+ messages in thread
From: Richard W.M. Jones @ 2018-12-15 14:02 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On Sat, Dec 15, 2018 at 07:53:04AM -0600, Eric Blake wrote:
> It can be useful to figure out which NBD protocol features are
> exposed by a server, as well as what features a client will
> take advantage of if available, for a given qemu release. It's
> not always precise to base features on version numbers (thanks
> to downstream backports), but any documentation is better than
> making users search through git logs themselves.
>
> This patch originally stemmed from a request to document that
> pristine 3.0 has a known bug where NBD_OPT_LIST_META_CONTEXT
> with 0 queries forgot to advertise an available
> "qemu:dirty-bitmap" context, but documenting bugs like this (or
> the fact that 3.0 also botched NBD_CMD_CACHE) gets to be too
> much details, especially since buggy releases will be less
> likely connection targets over time. Instead, I chose to just
> remind users to check stable release branches.
>
> Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: new patch
> ---
> docs/interop/nbd.txt | 19 ++++++++++++++++++-
> 1 file changed, 18 insertions(+), 1 deletion(-)
>
> diff --git a/docs/interop/nbd.txt b/docs/interop/nbd.txt
> index 77b5f459111..2b25f871e7c 100644
> --- a/docs/interop/nbd.txt
> +++ b/docs/interop/nbd.txt
> @@ -15,7 +15,6 @@ Qemu supports the "base:allocation" metadata context as defined in the
> NBD protocol specification, and also defines an additional metadata
> namespace "qemu".
>
> -
> == "qemu" namespace ==
>
> The "qemu" namespace currently contains only one type of context,
> @@ -36,3 +35,21 @@ in addition to "qemu:dirty-bitmap:<dirty-bitmap-export-name>":
> namespace.
> * "qemu:dirty-bitmap:" - returns list of all available dirty-bitmap
> metadata contexts.
> +
> += Features by version =
> +
> +The following list documents which qemu version first implemented
> +various features (both as a server exposing the feature, and as a
> +client taking advantage of the feature when present), to make it
> +easier to plan for cross-version interoperability. Note that in
> +several cases, the initial release containing a feature may require
> +additional patches from the corresponding stable branch to fix bugs in
> +the operation of that feature.
> +
> +* 2.6: NBD_OPT_STARTTLS with TLS X.509 Certificates
> +* 2.8: NBD_CMD_WRITE_ZEROES
> +* 2.10: NBD_OPT_GO, NBD_INFO_BLOCK
> +* 2.11: NBD_OPT_STRUCTURED_READ
> +* 2.12: NBD_CMD_BLOCK_STATUS for "base:allocation"
> +* 3.0: NBD_OPT_STARTTLS with TLS Pre-Shared Keys (PSK),
> +NBD_CMD_BLOCK_STATUS for "qemu:dirty-bitmap:", NBD_CMD_CACHE
Sensible documentation change.
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 03/22] maint: Allow for EXAMPLES in texi2pod
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 03/22] maint: Allow for EXAMPLES in texi2pod Eric Blake
@ 2018-12-15 14:04 ` Richard W.M. Jones
2018-12-18 13:46 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 55+ messages in thread
From: Richard W.M. Jones @ 2018-12-15 14:04 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On Sat, Dec 15, 2018 at 07:53:05AM -0600, Eric Blake wrote:
> The next commit will add an EXAMPLES section to qemu-nbd.8;
> for that to work, we need to recognize EXAMPLES in texi2pod,
> and we need to make all man pages be regenerated since the
> output of texi2pod can be different.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: new patch
> ---
> Makefile | 18 ++++++++++--------
> scripts/texi2pod.pl | 2 +-
> 2 files changed, 11 insertions(+), 9 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index c8b9efdad4b..0bd204eff8a 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -824,14 +824,16 @@ docs/interop/qemu-qmp-qapi.texi: qapi/qapi-doc.texi
> docs/interop/qemu-ga-qapi.texi: qga/qapi-generated/qga-qapi-doc.texi
> @cp -p $< $@
>
> -qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi qemu-monitor-info.texi
> -qemu.1: qemu-option-trace.texi
> -qemu-img.1: qemu-img.texi qemu-option-trace.texi qemu-img-cmds.texi
> -fsdev/virtfs-proxy-helper.1: fsdev/virtfs-proxy-helper.texi
> -qemu-nbd.8: qemu-nbd.texi qemu-option-trace.texi
> -qemu-ga.8: qemu-ga.texi
> -docs/qemu-block-drivers.7: docs/qemu-block-drivers.texi
> -docs/qemu-cpu-models.7: docs/qemu-cpu-models.texi
> +qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi \
> + qemu-monitor-info.texi scripts/texi2pod.pl
> +qemu.1: qemu-option-trace.texi scripts/texi2pod.pl
> +qemu-img.1: qemu-img.texi qemu-option-trace.texi qemu-img-cmds.texi \
> + scripts/texi2pod.pl
> +fsdev/virtfs-proxy-helper.1: fsdev/virtfs-proxy-helper.texi scripts/texi2pod.pl
> +qemu-nbd.8: qemu-nbd.texi qemu-option-trace.texi scripts/texi2pod.pl
> +qemu-ga.8: qemu-ga.texi scripts/texi2pod.pl
> +docs/qemu-block-drivers.7: docs/qemu-block-drivers.texi scripts/texi2pod.pl
> +docs/qemu-cpu-models.7: docs/qemu-cpu-models.texi scripts/texi2pod.pl
>
> html: qemu-doc.html docs/interop/qemu-qmp-ref.html docs/interop/qemu-ga-ref.html
> info: qemu-doc.info docs/interop/qemu-qmp-ref.info docs/interop/qemu-ga-ref.info
> diff --git a/scripts/texi2pod.pl b/scripts/texi2pod.pl
> index 39ce584a322..839b7917cf7 100755
> --- a/scripts/texi2pod.pl
> +++ b/scripts/texi2pod.pl
> @@ -398,7 +398,7 @@ $sects{NAME} = "$fn \- $tl\n";
> $sects{FOOTNOTES} .= "=back\n" if exists $sects{FOOTNOTES};
>
> for $sect (qw(NAME SYNOPSIS DESCRIPTION OPTIONS ENVIRONMENT FILES
> - BUGS NOTES FOOTNOTES SEEALSO AUTHOR COPYRIGHT)) {
> + BUGS NOTES FOOTNOTES EXAMPLES SEEALSO AUTHOR COPYRIGHT)) {
> if(exists $sects{$sect}) {
> $head = $sect;
> $head =~ s/SEEALSO/SEE ALSO/;
> --
> 2.17.2
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 04/22] qemu-nbd: Enhance man page
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 04/22] qemu-nbd: Enhance man page Eric Blake
@ 2018-12-15 14:13 ` Richard W.M. Jones
2018-12-17 15:19 ` Eric Blake
0 siblings, 1 reply; 55+ messages in thread
From: Richard W.M. Jones @ 2018-12-15 14:13 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On Sat, Dec 15, 2018 at 07:53:06AM -0600, Eric Blake wrote:
> Document some useful qemu-nbd command lines. Mention some restrictions
> on particular options, like -p being only for MBR images, or -c/-d
> being Linux-only. Update some text given the recent change to no
> longer serve oldstyle protocol (missed in commit 7f7dfe2a). Also,
> consistently use trailing '.' in describing options.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: new patch
> ---
> qemu-nbd.texi | 85 +++++++++++++++++++++++++++++++++++++++------------
> 1 file changed, 66 insertions(+), 19 deletions(-)
>
> diff --git a/qemu-nbd.texi b/qemu-nbd.texi
> index 9a84e81eed9..0e24c2801ee 100644
> --- a/qemu-nbd.texi
> +++ b/qemu-nbd.texi
> @@ -8,7 +8,10 @@
>
> @c man begin DESCRIPTION
>
> -Export a QEMU disk image using the NBD protocol.
> +Provide access to various QEMU NBD features. Most commonly used to
> +export a QEMU disk image using the NBD protocol as a server, but can
> +also be used (on Linux) to manage kernel bindings of a /dev/nbdX
> +block device to a QEMU server.
This is only a minor quibble, but I thought the original text was a
good summary, and only needs additional paragraphs describing the more
minor use cases. Thus the description would become by the end of the
patch series:
DESCRIPTION
Export a QEMU disk image using the NBD protocol.
Other uses:
* (On Linux) bind /dev/nbdX block device to a QEMU server.
* As a client to query exports of a remote NBD server.
> +@c man begin EXAMPLES
> +Start a server listening on port 10809 that exposes only the
> +guest-visible contents of a qcow2 file, with no TLS encryption, and
> +with the default export name (an empty string). The command will block
> +until the first successful client disconnects:
TBH I'd always include the -t option in every example. I don't
understand (except for backwards compatibility) why it isn't the
default since it's something I always trip over when using qemu-nbd.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 06/22] qemu-nbd: Fail earlier for -c/-d on non-linux
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 06/22] qemu-nbd: Fail earlier for -c/-d on non-linux Eric Blake
@ 2018-12-15 14:15 ` Richard W.M. Jones
2018-12-18 14:26 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 55+ messages in thread
From: Richard W.M. Jones @ 2018-12-15 14:15 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On Sat, Dec 15, 2018 at 07:53:08AM -0600, Eric Blake wrote:
> Connecting to a /dev/nbdN device is a Linux-specific action.
> We were already masking -c and -d from 'qemu-nbd --help' on
> non-linux. However, while -d fails with a sensible error
> message, it took hunting through a couple of files to prove
> that. What's more, the code for -c doesn't fail until after
> it has created a pthread and tried to open a device - possibly
> even printing an error message with %m on a non-Linux platform
> in spite of the comment that %m is glibc-specific. Make the
> failure happen sooner, then get rid of stubs that are no
> longer needed because of the early exits.
>
> While at it: tweak the blank newlines in --help output to be
> consistent, whether or not built on Linux.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: Hoist -c error message to share with -d message [Vladimir]
> Bonus: gets rid of a stray TAB in nbd/client.c
> ---
> nbd/client.c | 18 +-----------------
> qemu-nbd.c | 21 +++++++++++++++++++--
> 2 files changed, 20 insertions(+), 19 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 5d59d5ba78a..3d9086af39d 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -1029,23 +1029,7 @@ int nbd_disconnect(int fd)
> return 0;
> }
>
> -#else
> -int nbd_init(int fd, QIOChannelSocket *ioc, NBDExportInfo *info,
> - Error **errp)
> -{
> - error_setg(errp, "nbd_init is only supported on Linux");
> - return -ENOTSUP;
> -}
> -
> -int nbd_client(int fd)
> -{
> - return -ENOTSUP;
> -}
> -int nbd_disconnect(int fd)
> -{
> - return -ENOTSUP;
> -}
> -#endif
> +#endif /* __linux__ */
>
> int nbd_send_request(QIOChannel *ioc, NBDRequest *request)
> {
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index e169b839ece..2807e132396 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -43,6 +43,12 @@
> #include "trace/control.h"
> #include "qemu-version.h"
>
> +#ifdef __linux__
> +#define HAVE_NBD_DEVICE 1
> +#else
> +#define HAVE_NBD_DEVICE 0
> +#endif
> +
> #define SOCKET_PATH "/var/lock/qemu-nbd-%s"
> #define QEMU_NBD_OPT_CACHE 256
> #define QEMU_NBD_OPT_AIO 257
> @@ -98,11 +104,11 @@ static void usage(const char *name)
> " specify tracing options\n"
> " --fork fork off the server process and exit the parent\n"
> " once the server is running\n"
> -#ifdef __linux__
> +#if HAVE_NBD_DEVICE
> +"\n"
> "Kernel NBD client support:\n"
> " -c, --connect=DEV connect FILE to the local NBD device DEV\n"
> " -d, --disconnect disconnect the specified device\n"
> -"\n"
> #endif
> "\n"
> "Block device options:\n"
> @@ -236,6 +242,7 @@ static void termsig_handler(int signum)
> }
>
>
> +#if HAVE_NBD_DEVICE
> static void *show_parts(void *arg)
> {
> char *device = arg;
> @@ -321,6 +328,7 @@ out:
> kill(getpid(), SIGTERM);
> return (void *) EXIT_FAILURE;
> }
> +#endif /* HAVE_NBD_DEVICE */
>
> static int nbd_can_accept(void)
> {
> @@ -814,6 +822,12 @@ int main(int argc, char **argv)
> }
> }
>
> +#if !HAVE_NBD_DEVICE
> + if (disconnect || device) {
> + error_report("Kernel /dev/nbdN support not available");
> + exit(EXIT_FAILURE);
> + }
> +#else /* HAVE_NBD_DEVICE */
> if (disconnect) {
> int nbdfd = open(argv[optind], O_RDWR);
> if (nbdfd < 0) {
> @@ -829,6 +843,7 @@ int main(int argc, char **argv)
>
> return 0;
> }
> +#endif
>
> if ((device && !verbose) || fork_process) {
> int stderr_fd[2];
> @@ -1006,6 +1021,7 @@ int main(int argc, char **argv)
> nbd_export_set_description(exp, export_description);
>
> if (device) {
> +#if HAVE_NBD_DEVICE
> int ret;
>
> ret = pthread_create(&client_thread, NULL, nbd_client_thread, device);
> @@ -1013,6 +1029,7 @@ int main(int argc, char **argv)
> error_report("Failed to create client thread: %s", strerror(ret));
> exit(EXIT_FAILURE);
> }
> +#endif
> } else {
> /* Shut up GCC warnings. */
> memset(&client_thread, 0, sizeof(client_thread));
> --
An obvious simplification of the last version, so:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 07/22] qemu-nbd: Avoid strtol open-coding
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 07/22] qemu-nbd: Avoid strtol open-coding Eric Blake
@ 2018-12-15 14:17 ` Richard W.M. Jones
2018-12-18 15:11 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 55+ messages in thread
From: Richard W.M. Jones @ 2018-12-15 14:17 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On Sat, Dec 15, 2018 at 07:53:09AM -0600, Eric Blake wrote:
> Our copy-and-pasted open-coding of strtol handling forgot to
> handle overflow conditions. Use qemu_strto*() instead.
>
> In the case of --partition, since we insist on a user-supplied
> partition to be non-zero, we can use 0 rather than -1 for our
> initial value to distinguish when a partition is not being
> served, for slightly more optimal code.
>
> The error messages for out-of-bounds values are less specific,
> but should not be a terrible loss in quality.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: Retitle, catch more uses of strtol
> [Hmm - this depends on int64_t and off_t being compatible; if they
> aren't that way on all platforms, I'll need a temporary variable]
> ---
> qemu-nbd.c | 33 ++++++++++-----------------------
> 1 file changed, 10 insertions(+), 23 deletions(-)
>
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index 2807e132396..e3f739671b5 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -544,9 +544,8 @@ int main(int argc, char **argv)
> };
> int ch;
> int opt_ind = 0;
> - char *end;
> int flags = BDRV_O_RDWR;
> - int partition = -1;
> + int partition = 0;
> int ret = 0;
> bool seen_cache = false;
> bool seen_discard = false;
> @@ -657,13 +656,9 @@ int main(int argc, char **argv)
> port = optarg;
> break;
> case 'o':
> - dev_offset = strtoll (optarg, &end, 0);
> - if (*end) {
> - error_report("Invalid offset `%s'", optarg);
> - exit(EXIT_FAILURE);
> - }
> - if (dev_offset < 0) {
> - error_report("Offset must be positive `%s'", optarg);
> + if (qemu_strtoi64(optarg, NULL, 0, &dev_offset) < 0 ||
> + dev_offset < 0) {
> + error_report("Invalid offset '%s'", optarg);
> exit(EXIT_FAILURE);
> }
> break;
> @@ -685,13 +680,9 @@ int main(int argc, char **argv)
> flags &= ~BDRV_O_RDWR;
> break;
> case 'P':
> - partition = strtol(optarg, &end, 0);
> - if (*end) {
> - error_report("Invalid partition `%s'", optarg);
> - exit(EXIT_FAILURE);
> - }
> - if (partition < 1 || partition > 8) {
> - error_report("Invalid partition %d", partition);
> + if (qemu_strtoi(optarg, NULL, 0, &partition) < 0 ||
> + partition < 1 || partition > 8) {
> + error_report("Invalid partition '%s'", optarg);
> exit(EXIT_FAILURE);
> }
> break;
> @@ -709,15 +700,11 @@ int main(int argc, char **argv)
> device = optarg;
> break;
> case 'e':
> - shared = strtol(optarg, &end, 0);
> - if (*end) {
> + if (qemu_strtoi(optarg, NULL, 0, &shared) < 0 ||
> + shared < 1) {
> error_report("Invalid shared device number '%s'", optarg);
> exit(EXIT_FAILURE);
> }
> - if (shared < 1) {
> - error_report("Shared device number must be greater than 0");
> - exit(EXIT_FAILURE);
> - }
> break;
> case 'f':
> fmt = optarg;
> @@ -1006,7 +993,7 @@ int main(int argc, char **argv)
> }
> fd_size -= dev_offset;
>
> - if (partition != -1) {
> + if (partition) {
> ret = find_partition(blk, partition, &dev_offset, &fd_size);
> if (ret < 0) {
> error_report("Could not find partition %d: %s", partition,
Using qemu_strtoi has made the patch slightly bigger that last time
but is still a simplification, so:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 09/22] nbd/client: Refactor nbd_receive_list()
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 09/22] nbd/client: Refactor nbd_receive_list() Eric Blake
@ 2018-12-15 15:07 ` Richard W.M. Jones
2018-12-19 13:11 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 55+ messages in thread
From: Richard W.M. Jones @ 2018-12-15 15:07 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On Sat, Dec 15, 2018 at 07:53:11AM -0600, Eric Blake wrote:
> Right now, nbd_receive_list() is only called by
> nbd_receive_query_exports(), which in turn is only called if the
> server lacks NBD_OPT_GO but has working option negotiation, and is
> merely used as a quality-of-implementation trick since servers
> can't give decent errors for NBD_OPT_EXPORT_NAME. However, servers
> that lack NBD_OPT_GO are becoming increasingly rare (nbdkit was a
> latecomer, in Aug 2018, but qemu has been such a server since commit
> f37708f6 in July 2017 and released in 2.10), so it no longer makes
> sense to micro-optimize that function for performance.
>
> Furthermore, when debugging a server's implementation, tracing the
> full reply (both names and descriptions) is useful, not to mention
> that upcoming patches adding 'qemu-nbd --list' will want to collect
> that data. And when you consider that a server can send an export
> name up to the NBD protocol length limit of 4k; but our current
> NBD_MAX_NAME_SIZE is only 256, we can't trace all valid server
> names without more storage, but 4k is large enough that the heap
> is better than the stack for long names.
>
> Thus, I'm changing the division of labor, with nbd_receive_list()
> now always malloc'ing a result on success (the malloc is bounded
> by the fact that we reject servers with a reply length larger
> than 32M), and moving the comparison to 'wantname' to the caller.
>
> There is a minor change in behavior where a server with 0 exports
> (an immediate NBD_REP_ACK reply) is now no longer distinguished
> from a server without LIST support (NBD_REP_ERR_UNSUP); this
> information could be preserved with a complication to the calling
> contract to provide a bit more information, but I didn't see the
> point. After all, the worst that can happen if our guess at a
> match is wrong is that the caller will get a cryptic disconnect
> when NBD_OPT_EXPORT_NAME fails (which is no different from what
> would happen if we had not tried LIST), while treating an empty
> list as immediate failure would prevent connecting to really old
> servers that really did lack LIST. Besides, NBD servers with 0
> exports are rare (qemu can do it when using QMP nbd-server-start
> without nbd-server-add - but qemu understands NBD_OPT_GO and
> thus won't tickle this change in behavior).
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: Rewrite in a different manner (move the comparison to the
> caller)
> Fix free to g_free [Vladimir]
> ---
> nbd/client.c | 89 +++++++++++++++++++++++++++++++-----------------
> nbd/trace-events | 1 +
> 2 files changed, 58 insertions(+), 32 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 1a3a620fb6d..28f5a286cba 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -232,18 +232,24 @@ static int nbd_handle_reply_err(QIOChannel *ioc, NBDOptionReply *reply,
> return result;
> }
>
> -/* Process another portion of the NBD_OPT_LIST reply. Set *@match if
> - * the current reply matches @want or if the server does not support
> - * NBD_OPT_LIST, otherwise leave @match alone. Return 0 if iteration
> - * is complete, positive if more replies are expected, or negative
> - * with @errp set if an unrecoverable error occurred. */
> -static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> +/* nbd_receive_list:
> + * Process another portion of the NBD_OPT_LIST reply, populating any
> + * name received into *@name. If @description is non-NULL, and the
> + * server provided a description, that is also populated. The caller
> + * must eventually call g_free() on success.
> + * Returns 1 if a name was received and iteration must continue,
> + * 0 if iteration is complete,
> + * -1 with @errp set if an unrecoverable error occurred.
> + */
> +static int nbd_receive_list(QIOChannel *ioc, char **name, char **description,
> Error **errp)
> {
> + int ret = -1;
> NBDOptionReply reply;
> uint32_t len;
> uint32_t namelen;
> - char name[NBD_MAX_NAME_SIZE + 1];
> + char *local_name = NULL;
> + char *local_desc = NULL;
> int error;
>
> if (nbd_receive_option_reply(ioc, NBD_OPT_LIST, &reply, errp) < 0) {
> @@ -251,9 +257,6 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> }
> error = nbd_handle_reply_err(ioc, &reply, errp);
> if (error <= 0) {
> - /* The server did not support NBD_OPT_LIST, so set *match on
> - * the assumption that any name will be accepted. */
> - *match = true;
> return error;
> }
> len = reply.length;
> @@ -290,33 +293,38 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> nbd_send_opt_abort(ioc);
> return -1;
> }
> - if (namelen != strlen(want)) {
> - if (nbd_drop(ioc, len, errp) < 0) {
> - error_prepend(errp,
> - "failed to skip export name with wrong length: ");
> - nbd_send_opt_abort(ioc);
> - return -1;
> - }
> - return 1;
> - }
>
> - assert(namelen < sizeof(name));
> - if (nbd_read(ioc, name, namelen, errp) < 0) {
> + local_name = g_malloc(namelen + 1);
> + if (nbd_read(ioc, local_name, namelen, errp) < 0) {
> error_prepend(errp, "failed to read export name: ");
> nbd_send_opt_abort(ioc);
> - return -1;
> + goto out;
> }
> - name[namelen] = '\0';
> + local_name[namelen] = '\0';
> len -= namelen;
> - if (nbd_drop(ioc, len, errp) < 0) {
> - error_prepend(errp, "failed to read export description: ");
> - nbd_send_opt_abort(ioc);
> - return -1;
> + if (len) {
> + local_desc = g_malloc(len + 1);
> + if (nbd_read(ioc, local_desc, len, errp) < 0) {
> + error_prepend(errp, "failed to read export description: ");
> + nbd_send_opt_abort(ioc);
> + goto out;
> + }
> + local_desc[len] = '\0';
> }
> - if (!strcmp(name, want)) {
> - *match = true;
> +
> + trace_nbd_receive_list(local_name, local_desc ?: "");
> + *name = local_name;
> + local_name = NULL;
> + if (description) {
> + *description = local_desc;
> + local_desc = NULL;
> }
> - return 1;
> + ret = 1;
> +
> + out:
> + g_free(local_name);
> + g_free(local_desc);
> + return ret;
> }
>
>
> @@ -491,6 +499,7 @@ static int nbd_receive_query_exports(QIOChannel *ioc,
> const char *wantname,
> Error **errp)
> {
> + bool listEmpty = true;
> bool foundExport = false;
>
> trace_nbd_receive_query_exports_start(wantname);
> @@ -499,14 +508,25 @@ static int nbd_receive_query_exports(QIOChannel *ioc,
> }
>
> while (1) {
> - int ret = nbd_receive_list(ioc, wantname, &foundExport, errp);
> + char *name;
> + int ret = nbd_receive_list(ioc, &name, NULL, errp);
>
> if (ret < 0) {
> /* Server gave unexpected reply */
> return -1;
> } else if (ret == 0) {
> /* Done iterating. */
> - if (!foundExport) {
> + if (listEmpty) {
> + /*
> + * We don't have enough context to tell a server that
> + * sent an empty list apart from a server that does
> + * not support the list command; but as this function
> + * is just used to trigger a nicer error message
> + * before trying NBD_OPT_EXPORT_NAME, assume the
> + * export is available.
> + */
> + return 0;
> + } else if (!foundExport) {
> error_setg(errp, "No export with name '%s' available",
> wantname);
> nbd_send_opt_abort(ioc);
> @@ -515,6 +535,11 @@ static int nbd_receive_query_exports(QIOChannel *ioc,
> trace_nbd_receive_query_exports_success(wantname);
> return 0;
> }
> + listEmpty = false;
> + if (!strcmp(name, wantname)) {
> + foundExport = true;
> + }
> + g_free(name);
> }
> }
>
> diff --git a/nbd/trace-events b/nbd/trace-events
> index 5e1d4afe8e6..8e9fc024c28 100644
> --- a/nbd/trace-events
> +++ b/nbd/trace-events
> @@ -2,6 +2,7 @@
> nbd_send_option_request(uint32_t opt, const char *name, uint32_t len) "Sending option request %" PRIu32" (%s), len %" PRIu32
> nbd_receive_option_reply(uint32_t option, const char *optname, uint32_t type, const char *typename, uint32_t length) "Received option reply %" PRIu32" (%s), type %" PRIu32" (%s), len %" PRIu32
> nbd_reply_err_unsup(uint32_t option, const char *name) "server doesn't understand request %" PRIu32 " (%s), attempting fallback"
> +nbd_receive_list(const char *name, const char *desc) "export list includes '%s', description '%s'"
> nbd_opt_go_start(const char *name) "Attempting NBD_OPT_GO for export '%s'"
> nbd_opt_go_success(void) "Export is good to go"
> nbd_opt_go_info_unknown(int info, const char *name) "Ignoring unknown info %d (%s)"
> --
> 2.17.2
I agree that moving the wantname functionality into the parent greatly
simplifies this refactoring.
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 11/22] nbd/client: Change signature of nbd_negotiate_simple_meta_context()
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 11/22] nbd/client: Change signature of nbd_negotiate_simple_meta_context() Eric Blake
@ 2018-12-15 15:12 ` Richard W.M. Jones
2018-12-20 13:37 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 55+ messages in thread
From: Richard W.M. Jones @ 2018-12-15 15:12 UTC (permalink / raw)
To: Eric Blake
Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block, Kevin Wolf,
Max Reitz
On Sat, Dec 15, 2018 at 07:53:13AM -0600, Eric Blake wrote:
> Pass 'info' instead of three separate parameters related to info,
> when requesting the server to set the meta context. Update the
> NBDExportInfo struct to rename the received id field to match the
> fact that we are currently overloading the field to match whatever
> context the user supplied through the x-dirty-bitmap hack, as well
> as adding a TODO comment to remind future patches about a desire
> to request two contexts at once.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> v2: split patch into easier-to-review pieces [Rich, Vladimir]
> rename NBDExportInfo meta_base_allocation_id [Vladimir]
> ---
> include/block/nbd.h | 2 +-
> block/nbd-client.c | 4 ++--
> nbd/client.c | 53 +++++++++++++++++++++------------------------
> 3 files changed, 28 insertions(+), 31 deletions(-)
>
> diff --git a/include/block/nbd.h b/include/block/nbd.h
> index 65feff8ba96..ae5fe28f486 100644
> --- a/include/block/nbd.h
> +++ b/include/block/nbd.h
> @@ -276,7 +276,7 @@ struct NBDExportInfo {
> uint32_t opt_block;
> uint32_t max_block;
>
> - uint32_t meta_base_allocation_id;
> + uint32_t context_id;
> };
> typedef struct NBDExportInfo NBDExportInfo;
>
> diff --git a/block/nbd-client.c b/block/nbd-client.c
> index 417971d8b05..608b578e1d3 100644
> --- a/block/nbd-client.c
> +++ b/block/nbd-client.c
> @@ -246,11 +246,11 @@ static int nbd_parse_blockstatus_payload(NBDClientSession *client,
> }
>
> context_id = payload_advance32(&payload);
> - if (client->info.meta_base_allocation_id != context_id) {
> + if (client->info.context_id != context_id) {
> error_setg(errp, "Protocol error: unexpected context id %d for "
> "NBD_REPLY_TYPE_BLOCK_STATUS, when negotiated context "
> "id is %d", context_id,
> - client->info.meta_base_allocation_id);
> + client->info.context_id);
> return -EINVAL;
> }
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 7462fa5ae0e..bcccd5f555e 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -628,26 +628,30 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
> }
>
> /* nbd_negotiate_simple_meta_context:
> - * Set one meta context. Simple means that reply must contain zero (not
> - * negotiated) or one (negotiated) contexts. More contexts would be considered
> - * as a protocol error. It's also implied that meta-data query equals queried
> - * context name, so, if server replies with something different than @context,
> - * it is considered an error too.
> - * return 1 for successful negotiation, context_id is set
> + * Request the server to set the meta context for export @info->name
> + * using @info->x_dirty_bitmap with a fallback to "base:allocation",
> + * setting @info->context_id to the resulting id. Fail if the server
> + * responds with more than one context or with a context different
> + * than the query.
> + * return 1 for successful negotiation,
> * 0 if operation is unsupported,
> * -1 with errp set for any other error
> */
> static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> - const char *export,
> - const char *context,
> - uint32_t *context_id,
> + NBDExportInfo *info,
> Error **errp)
> {
> + /*
> + * TODO: Removing the x_dirty_bitmap hack will mean refactoring
> + * this function to request and store ids for multiple contexts
> + * (both base:allocation and a dirty bitmap), at which point this
> + * function should lose the term _simple.
> + */
> int ret;
> NBDOptionReply reply;
> - uint32_t received_id = 0;
> + const char *context = info->x_dirty_bitmap ?: "base:allocation";
> bool received = false;
> - uint32_t export_len = strlen(export);
> + uint32_t export_len = strlen(info->name);
> uint32_t context_len = strlen(context);
> uint32_t data_len = sizeof(export_len) + export_len +
> sizeof(uint32_t) + /* number of queries */
> @@ -655,9 +659,9 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> char *data = g_malloc(data_len);
> char *p = data;
>
> - trace_nbd_opt_meta_request(context, export);
> + trace_nbd_opt_meta_request(context, info->name);
> stl_be_p(p, export_len);
> - memcpy(p += sizeof(export_len), export, export_len);
> + memcpy(p += sizeof(export_len), info->name, export_len);
> stl_be_p(p += export_len, 1);
> stl_be_p(p += sizeof(uint32_t), context_len);
> memcpy(p += sizeof(context_len), context, context_len);
> @@ -683,7 +687,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> if (reply.type == NBD_REP_META_CONTEXT) {
> char *name;
>
> - if (reply.length != sizeof(received_id) + context_len) {
> + if (reply.length != sizeof(info->context_id) + context_len) {
> error_setg(errp, "Failed to negotiate meta context '%s', server "
> "answered with unexpected length %" PRIu32, context,
> reply.length);
> @@ -691,12 +695,13 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return -1;
> }
>
> - if (nbd_read(ioc, &received_id, sizeof(received_id), errp) < 0) {
> + if (nbd_read(ioc, &info->context_id, sizeof(info->context_id),
> + errp) < 0) {
> return -1;
> }
> - received_id = be32_to_cpu(received_id);
> + info->context_id = be32_to_cpu(info->context_id);
>
> - reply.length -= sizeof(received_id);
> + reply.length -= sizeof(info->context_id);
> name = g_malloc(reply.length + 1);
> if (nbd_read(ioc, name, reply.length, errp) < 0) {
> g_free(name);
> @@ -713,7 +718,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> }
> g_free(name);
>
> - trace_nbd_opt_meta_reply(context, received_id);
> + trace_nbd_opt_meta_reply(context, info->context_id);
> received = true;
>
> /* receive NBD_REP_ACK */
> @@ -742,12 +747,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return -1;
> }
>
> - if (received) {
> - *context_id = received_id;
> - return 1;
> - }
> -
> - return 0;
> + return received;
> }
>
> int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> @@ -846,10 +846,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> }
>
> if (info->structured_reply && base_allocation) {
> - result = nbd_negotiate_simple_meta_context(
> - ioc, info->name,
> - info->x_dirty_bitmap ?: "base:allocation",
> - &info->meta_base_allocation_id, errp);
> + result = nbd_negotiate_simple_meta_context(ioc, info, errp);
> if (result < 0) {
> goto fail;
> }
This is much clearer and smaller refactoring so:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 12/22] nbd/client: Improve error handling in nbd_negotiate_simple_meta_context()
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 12/22] nbd/client: Improve error handling in nbd_negotiate_simple_meta_context() Eric Blake
@ 2018-12-15 15:19 ` Richard W.M. Jones
2018-12-17 15:26 ` Eric Blake
2018-12-17 15:30 ` Eric Blake
0 siblings, 2 replies; 55+ messages in thread
From: Richard W.M. Jones @ 2018-12-15 15:19 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On Sat, Dec 15, 2018 at 07:53:14AM -0600, Eric Blake wrote:
> Always allocate space for the reply returned by the server and
> hoist the trace earlier, as it is more interesting to trace the
> server's reply (even if it is unexpected) than parroting our
> request only on success. After all, skipping the allocation
> for a wrong size was merely a micro-optimization that only
> benefitted a broken server, rather than the common case of a
> compliant server that meets our expectations.
>
> Then turn the reply handling into a loop (even though we still
> never iterate more than once), to make this code easier to use
> when later patches do support multiple server replies. This
> changes the error message for a server with two replies (a
> corner case we are unlikely to hit in practice) from:
>
> Unexpected reply type 4 (meta context), expected 0 (ack)
>
> to:
>
> Server replied with more than one context
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: split patch into easier-to-review pieces [Rich, Vladimir]
> ---
> nbd/client.c | 16 ++++++++++++----
> 1 file changed, 12 insertions(+), 4 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index bcccd5f555e..b6a85fc3ef8 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -684,10 +684,11 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return ret;
> }
>
> - if (reply.type == NBD_REP_META_CONTEXT) {
> + while (reply.type == NBD_REP_META_CONTEXT) {
I'm not sure I understand why this change is safe.
As far as I can see reply.type is only updated in the loop by
nbd_receive_option_reply, and that reads from the server, and so the
server might keep sending NBD_REP_META_CONTEXT packets (instead of the
expected NBD_REP_ACK), so it could now loop forever against a
malicious server? (This is not taking into account any later patches)
Rich.
> char *name;
>
> - if (reply.length != sizeof(info->context_id) + context_len) {
> + if (reply.length <= sizeof(info->context_id) ||
> + reply.length > NBD_MAX_BUFFER_SIZE) {
> error_setg(errp, "Failed to negotiate meta context '%s', server "
> "answered with unexpected length %" PRIu32, context,
> reply.length);
> @@ -708,6 +709,15 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return -1;
> }
> name[reply.length] = '\0';
> + trace_nbd_opt_meta_reply(name, info->context_id);
> +
> + if (received) {
> + error_setg(errp, "Server replied with more than one context");
> + g_free(name);
> + nbd_send_opt_abort(ioc);
> + return -1;
> + }
> +
> if (strcmp(context, name)) {
> error_setg(errp, "Failed to negotiate meta context '%s', server "
> "answered with different context '%s'", context,
> @@ -717,8 +727,6 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return -1;
> }
> g_free(name);
> -
> - trace_nbd_opt_meta_reply(context, info->context_id);
> received = true;
>
> /* receive NBD_REP_ACK */
> --
> 2.17.2
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 13/22] nbd/client: Split out nbd_send_one_meta_context()
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 13/22] nbd/client: Split out nbd_send_one_meta_context() Eric Blake
@ 2018-12-15 15:22 ` Richard W.M. Jones
2018-12-17 15:34 ` Eric Blake
0 siblings, 1 reply; 55+ messages in thread
From: Richard W.M. Jones @ 2018-12-15 15:22 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On Sat, Dec 15, 2018 at 07:53:15AM -0600, Eric Blake wrote:
> Refactor nbd_negotiate_simple_meta_context() to pull out the
> code that can be reused to send a LIST request for 0 or 1 query.
> No semantic change.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: split patch into easier-to-review pieces [Rich, Vladimir]
> ---
> nbd/client.c | 64 ++++++++++++++++++++++++++++++++++--------------
> nbd/trace-events | 2 +-
> 2 files changed, 46 insertions(+), 20 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index b6a85fc3ef8..5b6b9964097 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -627,6 +627,48 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
> return QIO_CHANNEL(tioc);
> }
>
> +/* nbd_send_one_meta_context:
> + * Send 0 or 1 set/list meta context queries.
> + * return 0 on success, -1 with errp set for any error
> + */
> +static int nbd_send_one_meta_context(QIOChannel *ioc,
> + uint32_t opt,
> + const char *export,
> + const char *query,
> + Error **errp)
> +{
> + int ret;
> + uint32_t export_len = strlen(export);
> + uint32_t queries = !!query;
> + uint32_t context_len = 0;
> + uint32_t data_len;
> + char *data;
> + char *p;
> +
> + data_len = sizeof(export_len) + export_len + sizeof(queries);
> + if (query) {
> + context_len = strlen(query);
> + data_len += sizeof(context_len) + context_len;
> + } else {
> + assert(opt == NBD_OPT_LIST_META_CONTEXT);
> + }
> + data = g_malloc(data_len);
> + p = data;
> +
> + trace_nbd_opt_meta_request(nbd_opt_lookup(opt), query ?: "(all)", export);
> + stl_be_p(p, export_len);
> + memcpy(p += sizeof(export_len), export, export_len);
> + stl_be_p(p += export_len, queries);
> + if (query) {
> + stl_be_p(p += sizeof(uint32_t), context_len);
> + memcpy(p += sizeof(context_len), query, context_len);
> + }
> +
> + ret = nbd_send_option_request(ioc, opt, data_len, data, errp);
> + g_free(data);
> + return ret;
> +}
> +
> /* nbd_negotiate_simple_meta_context:
> * Request the server to set the meta context for export @info->name
> * using @info->x_dirty_bitmap with a fallback to "base:allocation",
> @@ -651,26 +693,10 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> NBDOptionReply reply;
> const char *context = info->x_dirty_bitmap ?: "base:allocation";
> bool received = false;
> - uint32_t export_len = strlen(info->name);
> - uint32_t context_len = strlen(context);
> - uint32_t data_len = sizeof(export_len) + export_len +
> - sizeof(uint32_t) + /* number of queries */
> - sizeof(context_len) + context_len;
> - char *data = g_malloc(data_len);
> - char *p = data;
>
> - trace_nbd_opt_meta_request(context, info->name);
> - stl_be_p(p, export_len);
> - memcpy(p += sizeof(export_len), info->name, export_len);
> - stl_be_p(p += export_len, 1);
> - stl_be_p(p += sizeof(uint32_t), context_len);
> - memcpy(p += sizeof(context_len), context, context_len);
> -
> - ret = nbd_send_option_request(ioc, NBD_OPT_SET_META_CONTEXT, data_len, data,
> - errp);
> - g_free(data);
> - if (ret < 0) {
> - return ret;
> + if (nbd_send_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
> + info->name, context, errp) < 0) {
> + return -1;
> }
>
> if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
> diff --git a/nbd/trace-events b/nbd/trace-events
> index 446d10b8603..00872a6f9d4 100644
> --- a/nbd/trace-events
> +++ b/nbd/trace-events
> @@ -11,7 +11,7 @@ nbd_receive_query_exports_start(const char *wantname) "Querying export list for
> nbd_receive_query_exports_success(const char *wantname) "Found desired export name '%s'"
> nbd_receive_starttls_new_client(void) "Setting up TLS"
> nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
> -nbd_opt_meta_request(const char *context, const char *export) "Requesting to set meta context %s for export %s"
> +nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
> nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
> nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
> nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
Although a straightforward refactoring, we still lost the comment
/* number of queries */. I'd still perhaps like to see a bit more
explanation of the layout and reasoning behind the data buffer.
But anyway ..
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 14/22] nbd/client: Split out nbd_receive_one_meta_context()
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 14/22] nbd/client: Split out nbd_receive_one_meta_context() Eric Blake
@ 2018-12-15 15:30 ` Richard W.M. Jones
0 siblings, 0 replies; 55+ messages in thread
From: Richard W.M. Jones @ 2018-12-15 15:30 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On Sat, Dec 15, 2018 at 07:53:16AM -0600, Eric Blake wrote:
> Refactor nbd_negotiate_simple_meta_context() to more closely
> resemble the pattern of nbd_receive_list(), separating the
> argument validation for one pass from the caller making a loop
> over passes. No major semantic change (although one error
> message loses the original query). The diff may be a bit hard
> to follow due to indentation changes and handling ACK first
> rather than last.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: split patch into easier-to-review pieces [Rich, Vladimir]
> ---
> nbd/client.c | 144 +++++++++++++++++++++++++++--------------------
> nbd/trace-events | 2 +-
> 2 files changed, 84 insertions(+), 62 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 5b6b9964097..0e5a9d59dbd 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -669,6 +669,83 @@ static int nbd_send_one_meta_context(QIOChannel *ioc,
> return ret;
> }
>
> +/* nbd_receive_one_meta_context:
> + * Called in a loop to receive and trace one set/list meta context reply.
> + * Pass non-NULL @name or @id to collect results back to the caller, which
> + * must eventually call g_free().
> + * return 1 if more contexts are expected,
> + * 0 if operation is complete,
> + * -1 with errp set for any error
> + */
> +static int nbd_receive_one_meta_context(QIOChannel *ioc,
> + uint32_t opt,
> + char **name,
> + uint32_t *id,
> + Error **errp)
> +{
> + int ret;
> + NBDOptionReply reply;
> + char *local_name = NULL;
> + uint32_t local_id;
> +
> + if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0) {
> + return -1;
> + }
> +
> + ret = nbd_handle_reply_err(ioc, &reply, errp);
> + if (ret <= 0) {
> + return ret;
> + }
> +
> + if (reply.type == NBD_REP_ACK) {
> + if (reply.length != 0) {
> + error_setg(errp, "Unexpected length to ACK response");
> + nbd_send_opt_abort(ioc);
> + return -1;
> + }
> + return 0;
> + } else if (reply.type != NBD_REP_META_CONTEXT) {
> + error_setg(errp, "Unexpected reply type %u (%s), expected %u (%s)",
> + reply.type, nbd_rep_lookup(reply.type),
> + NBD_REP_META_CONTEXT, nbd_rep_lookup(NBD_REP_META_CONTEXT));
> + nbd_send_opt_abort(ioc);
> + return -1;
> + }
> +
> + if (reply.length <= sizeof(local_id) ||
> + reply.length > NBD_MAX_BUFFER_SIZE) {
> + error_setg(errp, "Failed to negotiate meta context, server "
> + "answered with unexpected length %" PRIu32,
> + reply.length);
> + nbd_send_opt_abort(ioc);
> + return -1;
> + }
> +
> + if (nbd_read(ioc, &local_id, sizeof(local_id), errp) < 0) {
> + return -1;
> + }
> + local_id = be32_to_cpu(local_id);
> +
> + reply.length -= sizeof(local_id);
> + local_name = g_malloc(reply.length + 1);
> + if (nbd_read(ioc, local_name, reply.length, errp) < 0) {
> + g_free(local_name);
> + return -1;
> + }
> + local_name[reply.length] = '\0';
> + trace_nbd_opt_meta_reply(nbd_opt_lookup(opt), local_name, local_id);
> +
> + if (name) {
> + *name = local_name;
> + } else {
> + g_free(local_name);
> + }
> + if (id) {
> + *id = local_id;
> + }
> + return 1;
> +}
> +
> /* nbd_negotiate_simple_meta_context:
> * Request the server to set the meta context for export @info->name
> * using @info->x_dirty_bitmap with a fallback to "base:allocation",
> @@ -690,7 +767,6 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> * function should lose the term _simple.
> */
> int ret;
> - NBDOptionReply reply;
> const char *context = info->x_dirty_bitmap ?: "base:allocation";
> bool received = false;
>
> @@ -699,44 +775,17 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return -1;
> }
>
> - if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
> - errp) < 0)
> - {
> - return -1;
> - }
> -
> - ret = nbd_handle_reply_err(ioc, &reply, errp);
> - if (ret <= 0) {
> - return ret;
> - }
> -
> - while (reply.type == NBD_REP_META_CONTEXT) {
> + while (1) {
> char *name;
>
> - if (reply.length <= sizeof(info->context_id) ||
> - reply.length > NBD_MAX_BUFFER_SIZE) {
> - error_setg(errp, "Failed to negotiate meta context '%s', server "
> - "answered with unexpected length %" PRIu32, context,
> - reply.length);
> - nbd_send_opt_abort(ioc);
> + ret = nbd_receive_one_meta_context(ioc, NBD_OPT_SET_META_CONTEXT,
> + &name, &info->context_id, errp);
> + if (ret < 0) {
> return -1;
> + } else if (ret == 0) {
> + return received;
> }
>
> - if (nbd_read(ioc, &info->context_id, sizeof(info->context_id),
> - errp) < 0) {
> - return -1;
> - }
> - info->context_id = be32_to_cpu(info->context_id);
> -
> - reply.length -= sizeof(info->context_id);
> - name = g_malloc(reply.length + 1);
> - if (nbd_read(ioc, name, reply.length, errp) < 0) {
> - g_free(name);
> - return -1;
> - }
> - name[reply.length] = '\0';
> - trace_nbd_opt_meta_reply(name, info->context_id);
> -
> if (received) {
> error_setg(errp, "Server replied with more than one context");
> g_free(name);
> @@ -754,34 +803,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> }
> g_free(name);
> received = true;
> -
> - /* receive NBD_REP_ACK */
> - if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
> - errp) < 0)
> - {
> - return -1;
> - }
> -
> - ret = nbd_handle_reply_err(ioc, &reply, errp);
> - if (ret <= 0) {
> - return ret;
> - }
> }
> -
> - if (reply.type != NBD_REP_ACK) {
> - error_setg(errp, "Unexpected reply type %u (%s), expected %u (%s)",
> - reply.type, nbd_rep_lookup(reply.type),
> - NBD_REP_ACK, nbd_rep_lookup(NBD_REP_ACK));
> - nbd_send_opt_abort(ioc);
> - return -1;
> - }
> - if (reply.length) {
> - error_setg(errp, "Unexpected length to ACK response");
> - nbd_send_opt_abort(ioc);
> - return -1;
> - }
> -
> - return received;
> }
>
> int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> diff --git a/nbd/trace-events b/nbd/trace-events
> index 00872a6f9d4..02956c96042 100644
> --- a/nbd/trace-events
> +++ b/nbd/trace-events
> @@ -12,7 +12,7 @@ nbd_receive_query_exports_success(const char *wantname) "Found desired export na
> nbd_receive_starttls_new_client(void) "Setting up TLS"
> nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
> nbd_opt_meta_request(const char *optname, const char *context, const char *export) "Requesting %s %s for export %s"
> -nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
> +nbd_opt_meta_reply(const char *optname, const char *context, uint32_t id) "Received %s mapping of %s to id %" PRIu32
> nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
> nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
> nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are 0x%" PRIx32
So I looked at both the changes you've made and the overall effect on
the code.
I think the individual changes are correct, albeit quite convoluted to
follow because of (as you say in the commit) moving the test for
NBD_REP_ACK first and also a few conditionals which are inverted.
However importantly the overall effect on the code after the patch is
applied makes it much easier to understand than before.
Therefore:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 17/22] nbd/client: Pull out oldstyle size determination
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 17/22] nbd/client: Pull out oldstyle size determination Eric Blake
@ 2018-12-15 15:31 ` Richard W.M. Jones
0 siblings, 0 replies; 55+ messages in thread
From: Richard W.M. Jones @ 2018-12-15 15:31 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On Sat, Dec 15, 2018 at 07:53:19AM -0600, Eric Blake wrote:
> Another refactoring creating nbd_negotiate_finish_oldstyle()
> for further reuse during 'qemu-nbd --list'.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> v2: new patch [Vladimir]
> ---
> nbd/client.c | 49 ++++++++++++++++++++++++++++++++-----------------
> 1 file changed, 32 insertions(+), 17 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 8b0ae20fae8..4bdfba43068 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -811,7 +811,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> * Start the handshake to the server. After a positive return, the server
> * is ready to accept additional NBD_OPT requests.
> * Returns: negative errno: failure talking to server
> - * 0: server is oldstyle, client must still parse export size
> + * 0: server is oldstyle, must call nbd_negotiate_finish_oldstyle
> * 1: server is newstyle, but can only accept EXPORT_NAME
> * 2: server is newstyle, but lacks structured replies
> * 3: server is newstyle and set up for structured replies
> @@ -916,6 +916,36 @@ static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> }
> }
>
> +/*
> + * nbd_negotiate_finish_oldstyle:
> + * Populate @info with the size and export flags from an oldstyle server,
> + * but does not consume 124 bytes of reserved zero padding.
> + * Returns 0 on success, -1 with @errp set on failure
> + */
> +static int nbd_negotiate_finish_oldstyle(QIOChannel *ioc, NBDExportInfo *info,
> + Error **errp)
> +{
> + uint32_t oldflags;
> +
> + if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
> + error_prepend(errp, "Failed to read export length: ");
> + return -EINVAL;
> + }
> + info->size = be64_to_cpu(info->size);
> +
> + if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
> + error_prepend(errp, "Failed to read export flags: ");
> + return -EINVAL;
> + }
> + oldflags = be32_to_cpu(oldflags);
> + if (oldflags & ~0xffff) {
> + error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
> + return -EINVAL;
> + }
> + info->flags = oldflags;
> + return 0;
> +}
> +
> /*
> * nbd_receive_negotiate:
> * Connect to server, complete negotiation, and move into transmission phase.
> @@ -929,7 +959,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> int result;
> bool zeroes = true;
> bool base_allocation = info->base_allocation;
> - uint32_t oldflags;
>
> assert(info->name);
> trace_nbd_receive_negotiate_name(info->name);
> @@ -1002,23 +1031,9 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> error_setg(errp, "Server does not support non-empty export names");
> return -EINVAL;
> }
> -
> - if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
> - error_prepend(errp, "Failed to read export length: ");
> + if (nbd_negotiate_finish_oldstyle(ioc, info, errp) < 0) {
> return -EINVAL;
> }
> - info->size = be64_to_cpu(info->size);
> -
> - if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
> - error_prepend(errp, "Failed to read export flags: ");
> - return -EINVAL;
> - }
> - oldflags = be32_to_cpu(oldflags);
> - if (oldflags & ~0xffff) {
> - error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
> - return -EINVAL;
> - }
> - info->flags = oldflags;
> break;
> default:
> return result;
Straightforward refactoring, so:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 18/22] nbd/client: Add nbd_receive_export_list()
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 18/22] nbd/client: Add nbd_receive_export_list() Eric Blake
@ 2018-12-15 15:42 ` Richard W.M. Jones
2018-12-17 15:43 ` Eric Blake
0 siblings, 1 reply; 55+ messages in thread
From: Richard W.M. Jones @ 2018-12-15 15:42 UTC (permalink / raw)
To: Eric Blake
Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block, Kevin Wolf,
Max Reitz
On Sat, Dec 15, 2018 at 07:53:20AM -0600, Eric Blake wrote:
> We want to be able to detect whether a given qemu NBD server is
> exposing the right export(s) and dirty bitmaps, at least for
> regression testing. We could use 'nbd-client -l' from the upstream
> NBD project to list exports, but it's annoying to rely on
> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
> know about all of the qemu NBD extensions. Thus, we plan on adding
> a new mode to qemu-nbd that merely sniffs all possible information
> from the server during handshake phase, then disconnects and dumps
> the information.
>
> This patch adds the low-level client code for grabbing the list
> of exports. It benefits from the recent refactoring patches, as
> well as a minor tweak of changing nbd_opt_go() to nbd_opt_info_or_go(),
> in order to share as much code as possible when it comes to doing
> validation of server replies. The resulting information is stored
> in an array of NBDExportInfo which has been expanded to any
> description string, along with a convenience function for freeing
> the list.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: split out oldstyle size computation into earlier patch [Vladimir]
> rename nbd_opt_info_or_go [Rich]
> split out collection of meta context collection into later patch
> ---
> include/block/nbd.h | 15 ++++-
> nbd/client.c | 138 ++++++++++++++++++++++++++++++++++++++++++--
> nbd/trace-events | 2 +-
> 3 files changed, 146 insertions(+), 9 deletions(-)
>
> diff --git a/include/block/nbd.h b/include/block/nbd.h
> index ae5fe28f486..09d2157efe0 100644
> --- a/include/block/nbd.h
> +++ b/include/block/nbd.h
> @@ -1,5 +1,5 @@
> /*
> - * Copyright (C) 2016-2017 Red Hat, Inc.
> + * Copyright (C) 2016-2018 Red Hat, Inc.
> * Copyright (C) 2005 Anthony Liguori <anthony@codemonkey.ws>
> *
> * Network Block Device
> @@ -262,6 +262,9 @@ struct NBDExportInfo {
> /* Set by client before nbd_receive_negotiate() */
> bool request_sizes;
> char *x_dirty_bitmap;
> +
> + /* Set by client before nbd_receive_negotiate(), or by server results
> + * during nbd_receive_export_list() */
> char *name; /* must be non-NULL */
>
> /* In-out fields, set by client before nbd_receive_negotiate() and
> @@ -269,7 +272,8 @@ struct NBDExportInfo {
> bool structured_reply;
> bool base_allocation; /* base:allocation context for NBD_CMD_BLOCK_STATUS */
>
> - /* Set by server results during nbd_receive_negotiate() */
> + /* Set by server results during nbd_receive_negotiate() and
> + * nbd_receive_export_list() */
> uint64_t size;
> uint16_t flags;
> uint32_t min_block;
> @@ -277,12 +281,19 @@ struct NBDExportInfo {
> uint32_t max_block;
>
> uint32_t context_id;
> +
> + /* Set by server results during nbd_receive_export_list() */
> + char *description;
> };
> typedef struct NBDExportInfo NBDExportInfo;
>
> int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> const char *hostname, QIOChannel **outioc,
> NBDExportInfo *info, Error **errp);
> +void nbd_free_export_list(NBDExportInfo *info, int count);
> +int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> + const char *hostname, NBDExportInfo **info,
> + Error **errp);
> int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
> Error **errp);
> int nbd_send_request(QIOChannel *ioc, NBDRequest *request);
> diff --git a/nbd/client.c b/nbd/client.c
> index 4bdfba43068..0e6c575ccad 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -332,7 +332,8 @@ static int nbd_receive_list(QIOChannel *ioc, char **name, char **description,
> * used, 0 if NBD_OPT_GO is unsupported (fall back to NBD_OPT_LIST and
> * NBD_OPT_EXPORT_NAME in that case), and > 0 if the export is good to
> * go (with the rest of @info populated). */
> -static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
> +static int nbd_opt_info_or_go(QIOChannel *ioc, uint32_t opt,
> + NBDExportInfo *info, Error **errp)
> {
> NBDOptionReply reply;
> uint32_t len = strlen(info->name);
> @@ -345,7 +346,8 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
> * flags still 0 is a witness of a broken server. */
> info->flags = 0;
>
> - trace_nbd_opt_go_start(info->name);
> + assert(opt == NBD_OPT_GO || opt == NBD_OPT_INFO);
> + trace_nbd_opt_go_start(nbd_opt_lookup(opt), info->name);
> buf = g_malloc(4 + len + 2 + 2 * info->request_sizes + 1);
> stl_be_p(buf, len);
> memcpy(buf + 4, info->name, len);
> @@ -354,7 +356,7 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
> if (info->request_sizes) {
> stw_be_p(buf + 4 + len + 2, NBD_INFO_BLOCK_SIZE);
> }
> - error = nbd_send_option_request(ioc, NBD_OPT_GO,
> + error = nbd_send_option_request(ioc, opt,
> 4 + len + 2 + 2 * info->request_sizes,
> buf, errp);
> g_free(buf);
> @@ -363,7 +365,7 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
> }
>
> while (1) {
> - if (nbd_receive_option_reply(ioc, NBD_OPT_GO, &reply, errp) < 0) {
> + if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0) {
> return -1;
> }
> error = nbd_handle_reply_err(ioc, &reply, errp);
> @@ -868,7 +870,9 @@ static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> clientflags |= NBD_FLAG_C_FIXED_NEWSTYLE;
> }
> if (globalflags & NBD_FLAG_NO_ZEROES) {
> - *zeroes = false;
> + if (zeroes) {
> + *zeroes = false;
> + }
> clientflags |= NBD_FLAG_C_NO_ZEROES;
> }
> /* client requested flags */
> @@ -989,7 +993,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> * TLS). If it is not available, fall back to
> * NBD_OPT_LIST for nicer error messages about a missing
> * export, then use NBD_OPT_EXPORT_NAME. */
> - result = nbd_opt_go(ioc, info, errp);
> + result = nbd_opt_info_or_go(ioc, NBD_OPT_GO, info, errp);
> if (result < 0) {
> return -EINVAL;
> }
> @@ -1047,6 +1051,128 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> return 0;
> }
>
> +/* Clean up result of nbd_receive_export_list */
> +void nbd_free_export_list(NBDExportInfo *info, int count)
> +{
> + int i;
> +
> + if (!info) {
> + return;
> + }
> +
> + for (i = 0; i < count; i++) {
> + g_free(info[i].name);
> + g_free(info[i].description);
> + }
> + g_free(info);
> +}
> +
> +/*
> + * nbd_receive_export_list:
> + * Query details about a server's exports, then disconnect without
> + * going into transmission phase. Return a count of the exports listed
> + * in @info by the server, or -1 on error. Caller must free @info using
> + * nbd_free_export_list().
> + */
> +int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> + const char *hostname, NBDExportInfo **info,
> + Error **errp)
> +{
> + int result;
> + int count = 0;
> + int i;
> + int rc;
> + int ret = -1;
> + NBDExportInfo *array = NULL;
> + QIOChannel *sioc = NULL;
> +
> + *info = NULL;
> + result = nbd_start_negotiate(ioc, tlscreds, hostname, &sioc, true, NULL,
> + errp);
> + if (tlscreds && sioc) {
> + ioc = sioc;
> + }
> +
> + switch (result) {
> + case 2:
> + case 3:
> + /* newstyle - use NBD_OPT_LIST to populate array, then try
> + * NBD_OPT_INFO on each array member. If structured replies
> + * are enabled, also try NBD_OPT_LIST_META_CONTEXT. */
> + if (nbd_send_option_request(ioc, NBD_OPT_LIST, 0, NULL, errp) < 0) {
> + goto out;
> + }
> + while (1) {
> + char *name;
> + char *desc;
> +
> + rc = nbd_receive_list(ioc, &name, &desc, errp);
> + if (rc < 0) {
> + goto out;
> + } else if (rc == 0) {
> + break;
> + }
> + array = g_renew(NBDExportInfo, array, ++count);
> + memset(&array[count - 1], 0, sizeof(*array));
> + array[count - 1].name = name;
> + array[count - 1].description = desc;
> + array[count - 1].structured_reply = result == 3;
> + }
Do we care about limiting ‘count’ to some reasonable value here?
I tried to look at the protocol document to see if there's a limit on
the number of exports that a server can have, but if there is I cannot
find it. I don't know how much we care about malicious NBD servers --
mostly I'm interested in malicious NBD clients :-)
> + for (i = 0; i < count; i++) {
> + array[i].request_sizes = true;
> + rc = nbd_opt_info_or_go(ioc, NBD_OPT_INFO, &array[i], errp);
> + if (rc < 0) {
> + goto out;
> + } else if (rc == 0) {
> + /* Pointless to try rest of loop. If OPT_INFO doesn't work,
> + * it's unlikely that meta contexts work either */
> + break;
> + }
> +
> + /* TODO: Grab meta contexts */
> + }
> +
> + /* Send NBD_OPT_ABORT as a courtesy before hanging up */
> + nbd_send_opt_abort(ioc);
> + break;
> + case 1: /* newstyle, but limited to EXPORT_NAME */
> + error_setg(errp, "Server does not support export lists");
> + /* We can't even send NBD_OPT_ABORT, so merely hang up */
> + goto out;
> + case 0: /* oldstyle, parse length and flags */
> + array = g_new0(NBDExportInfo, 1);
> + array->name = g_strdup("");
> + count = 1;
> +
> + if (nbd_negotiate_finish_oldstyle(ioc, array, errp) < 0) {
> + return -EINVAL;
> + }
> +
> + /* Send NBD_CMD_DISC as a courtesy to the server, but ignore all
> + * errors now that we have the information we wanted. */
> + if (nbd_drop(ioc, 124, NULL) == 0) {
> + NBDRequest request = { .type = NBD_CMD_DISC };
> +
> + nbd_send_request(ioc, &request);
> + }
> + break;
> + default:
> + goto out;
> + }
> +
> + *info = array;
> + array = NULL;
> + ret = count;
> +
> + out:
> + qio_channel_shutdown(ioc, QIO_CHANNEL_SHUTDOWN_BOTH, NULL);
> + qio_channel_close(ioc, NULL);
> + object_unref(OBJECT(sioc));
> + nbd_free_export_list(array, count);
> + return ret;
> +}
> +
> #ifdef __linux__
> int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
> Error **errp)
> diff --git a/nbd/trace-events b/nbd/trace-events
> index 922b36d1843..a66bf891cc9 100644
> --- a/nbd/trace-events
> +++ b/nbd/trace-events
> @@ -3,7 +3,7 @@ nbd_send_option_request(uint32_t opt, const char *name, uint32_t len) "Sending o
> nbd_receive_option_reply(uint32_t option, const char *optname, uint32_t type, const char *typename, uint32_t length) "Received option reply %" PRIu32" (%s), type %" PRIu32" (%s), len %" PRIu32
> nbd_reply_err_unsup(uint32_t option, const char *name) "server doesn't understand request %" PRIu32 " (%s), attempting fallback"
> nbd_receive_list(const char *name, const char *desc) "export list includes '%s', description '%s'"
> -nbd_opt_go_start(const char *name) "Attempting NBD_OPT_GO for export '%s'"
> +nbd_opt_go_start(const char *opt, const char *name) "Attempting %s for export '%s'"
> nbd_opt_go_success(void) "Export is good to go"
> nbd_opt_go_info_unknown(int info, const char *name) "Ignoring unknown info %d (%s)"
> nbd_opt_go_info_block_size(uint32_t minimum, uint32_t preferred, uint32_t maximum) "Block sizes are 0x%" PRIx32 ", 0x%" PRIx32 ", 0x%" PRIx32
Anyway looks good, so:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 19/22] nbd/client: Add meta contexts to nbd_receive_export_list()
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 19/22] nbd/client: Add meta contexts to nbd_receive_export_list() Eric Blake
@ 2018-12-15 15:59 ` Richard W.M. Jones
0 siblings, 0 replies; 55+ messages in thread
From: Richard W.M. Jones @ 2018-12-15 15:59 UTC (permalink / raw)
To: Eric Blake
Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block, Kevin Wolf,
Max Reitz
On Sat, Dec 15, 2018 at 07:53:21AM -0600, Eric Blake wrote:
> We want to be able to detect whether a given qemu NBD server is
> exposing the right export(s) and dirty bitmaps, at least for
> regression testing. We could use 'nbd-client -l' from the upstream
> NBD project to list exports, but it's annoying to rely on
> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
> know about all of the qemu NBD extensions. Thus, we plan on adding
> a new mode to qemu-nbd that merely sniffs all possible information
> from the server during handshake phase, then disconnects and dumps
> the information.
>
> This patch continues the work of the previous patch, by adding the
> ability to track the list of available meta contexts into
> NBDExportInfo. It benefits from the recent refactoring patches
> with a new nbd_list_meta_contexts() that reuses much of the same
> framework as setting a meta context.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: new patch to split out collection of meta context collection
> s/free/g_free/ [Vladimir]
> ---
> include/block/nbd.h | 2 ++
> nbd/client.c | 41 +++++++++++++++++++++++++++++++++++++++--
> 2 files changed, 41 insertions(+), 2 deletions(-)
>
> diff --git a/include/block/nbd.h b/include/block/nbd.h
> index 09d2157efe0..6d9fbb941d7 100644
> --- a/include/block/nbd.h
> +++ b/include/block/nbd.h
> @@ -284,6 +284,8 @@ struct NBDExportInfo {
>
> /* Set by server results during nbd_receive_export_list() */
> char *description;
> + int n_contexts;
> + char **contexts;
> };
> typedef struct NBDExportInfo NBDExportInfo;
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 0e6c575ccad..d392b5e8bee 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -808,6 +808,36 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> }
> }
>
> +/*
> + * nbd_list_meta_contexts:
> + * Request the server to list all meta contexts for export @info->name.
> + * return 0 if list is complete (even if empty),
> + * -1 with errp set for any other error
> + */
> +static int nbd_list_meta_contexts(QIOChannel *ioc,
> + NBDExportInfo *info,
> + Error **errp)
> +{
> + int ret;
> +
> + if (nbd_send_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
> + info->name, NULL, errp) < 0) {
> + return -1;
> + }
> +
> + while (1) {
> + char *context;
> +
> + ret = nbd_receive_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT,
> + &context, NULL, errp);
> + if (ret <= 0) {
> + return ret;
> + }
> + info->contexts = g_renew(char *, info->contexts, ++info->n_contexts);
> + info->contexts[info->n_contexts - 1] = context;
> + }
> +}
> +
> /*
> * nbd_start_negotiate:
> * Start the handshake to the server. After a positive return, the server
> @@ -1054,7 +1084,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> /* Clean up result of nbd_receive_export_list */
> void nbd_free_export_list(NBDExportInfo *info, int count)
> {
> - int i;
> + int i, j;
>
> if (!info) {
> return;
> @@ -1063,6 +1093,10 @@ void nbd_free_export_list(NBDExportInfo *info, int count)
> for (i = 0; i < count; i++) {
> g_free(info[i].name);
> g_free(info[i].description);
> + for (j = 0; j < info[i].n_contexts; j++) {
> + g_free(info[i].contexts[j]);
> + }
> + g_free(info[i].contexts);
> }
> g_free(info);
> }
> @@ -1130,7 +1164,10 @@ int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> break;
> }
>
> - /* TODO: Grab meta contexts */
> + if (result == 3 &&
> + nbd_list_meta_contexts(ioc, &array[i], errp) < 0) {
> + goto out;
> + }
> }
>
> /* Send NBD_OPT_ABORT as a courtesy before hanging up */
> --
> 2.17.2
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 20/22] qemu-nbd: Add --list option
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 20/22] qemu-nbd: Add --list option Eric Blake
@ 2018-12-15 16:02 ` Richard W.M. Jones
0 siblings, 0 replies; 55+ messages in thread
From: Richard W.M. Jones @ 2018-12-15 16:02 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On Sat, Dec 15, 2018 at 07:53:22AM -0600, Eric Blake wrote:
> @@ -557,13 +660,14 @@ int main(int argc, char **argv)
> Error *local_err = NULL;
> BlockdevDetectZeroesOptions detect_zeroes = BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF;
> QDict *options = NULL;
> - const char *export_name = ""; /* Default export name */
> + const char *export_name = NULL;
You might want to comment that export_name is now a tri-state value,
with NULL meaning that it's not set by the user.
Nevertheless the patch looks good, and the addition of the example
(including host name setting) resolves my previous problem with the
new functionality. Therefore:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 04/22] qemu-nbd: Enhance man page
2018-12-15 14:13 ` Richard W.M. Jones
@ 2018-12-17 15:19 ` Eric Blake
2019-01-04 23:47 ` Eric Blake
0 siblings, 1 reply; 55+ messages in thread
From: Eric Blake @ 2018-12-17 15:19 UTC (permalink / raw)
To: Richard W.M. Jones; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On 12/15/18 8:13 AM, Richard W.M. Jones wrote:
> On Sat, Dec 15, 2018 at 07:53:06AM -0600, Eric Blake wrote:
>> Document some useful qemu-nbd command lines. Mention some restrictions
>> on particular options, like -p being only for MBR images, or -c/-d
>> being Linux-only. Update some text given the recent change to no
>> longer serve oldstyle protocol (missed in commit 7f7dfe2a). Also,
>> consistently use trailing '.' in describing options.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> ---
>> v2: new patch
>> ---
>> qemu-nbd.texi | 85 +++++++++++++++++++++++++++++++++++++++------------
>> 1 file changed, 66 insertions(+), 19 deletions(-)
>>
>> diff --git a/qemu-nbd.texi b/qemu-nbd.texi
>> index 9a84e81eed9..0e24c2801ee 100644
>> --- a/qemu-nbd.texi
>> +++ b/qemu-nbd.texi
>> @@ -8,7 +8,10 @@
>>
>> @c man begin DESCRIPTION
>>
>> -Export a QEMU disk image using the NBD protocol.
>> +Provide access to various QEMU NBD features. Most commonly used to
>> +export a QEMU disk image using the NBD protocol as a server, but can
>> +also be used (on Linux) to manage kernel bindings of a /dev/nbdX
>> +block device to a QEMU server.
>
> This is only a minor quibble, but I thought the original text was a
> good summary, and only needs additional paragraphs describing the more
> minor use cases. Thus the description would become by the end of the
> patch series:
>
> DESCRIPTION
>
> Export a QEMU disk image using the NBD protocol.
>
> Other uses:
> * (On Linux) bind /dev/nbdX block device to a QEMU server.
> * As a client to query exports of a remote NBD server.
Seems reasonable.
>
>> +@c man begin EXAMPLES
>> +Start a server listening on port 10809 that exposes only the
>> +guest-visible contents of a qcow2 file, with no TLS encryption, and
>> +with the default export name (an empty string). The command will block
>> +until the first successful client disconnects:
>
> TBH I'd always include the -t option in every example. I don't
> understand (except for backwards compatibility) why it isn't the
> default since it's something I always trip over when using qemu-nbd.
I'd still like one example without -t, to call out specifically that it
creates a one-shot server that goes away after the first client, but
don't mind fixing the rest of the examples to use -t.
Using -e for read-only connections makes sense, using -e for writable
exports is a bit more questionable - we _don't_ advertise the
NBD_FLAG_CAN_MULTI_CONN which states that caches are kept consistent
between simultaneous write connections, although maybe we should see if
qemu-nbd can start promising multi-write consistency in future patches?
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 12/22] nbd/client: Improve error handling in nbd_negotiate_simple_meta_context()
2018-12-15 15:19 ` Richard W.M. Jones
@ 2018-12-17 15:26 ` Eric Blake
2018-12-17 15:30 ` Eric Blake
1 sibling, 0 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-17 15:26 UTC (permalink / raw)
To: Richard W.M. Jones; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On 12/15/18 9:19 AM, Richard W.M. Jones wrote:
> On Sat, Dec 15, 2018 at 07:53:14AM -0600, Eric Blake wrote:
>> Always allocate space for the reply returned by the server and
>> hoist the trace earlier, as it is more interesting to trace the
>> server's reply (even if it is unexpected) than parroting our
>> request only on success. After all, skipping the allocation
>> for a wrong size was merely a micro-optimization that only
>> benefitted a broken server, rather than the common case of a
>> compliant server that meets our expectations.
>>
>> Then turn the reply handling into a loop (even though we still
>> never iterate more than once), to make this code easier to use
>> when later patches do support multiple server replies. This
>> changes the error message for a server with two replies (a
>> corner case we are unlikely to hit in practice) from:
>>
>> Unexpected reply type 4 (meta context), expected 0 (ack)
>>
>> to:
>>
>> Server replied with more than one context
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> ---
>> v2: split patch into easier-to-review pieces [Rich, Vladimir]
>> ---
>> nbd/client.c | 16 ++++++++++++----
>> 1 file changed, 12 insertions(+), 4 deletions(-)
>>
>> diff --git a/nbd/client.c b/nbd/client.c
>> index bcccd5f555e..b6a85fc3ef8 100644
>> --- a/nbd/client.c
>> +++ b/nbd/client.c
>> @@ -684,10 +684,11 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
>> return ret;
>> }
>>
>> - if (reply.type == NBD_REP_META_CONTEXT) {
>> + while (reply.type == NBD_REP_META_CONTEXT) {
>
> I'm not sure I understand why this change is safe.
>
> As far as I can see reply.type is only updated in the loop by
> nbd_receive_option_reply, and that reads from the server, and so the
> server might keep sending NBD_REP_META_CONTEXT packets (instead of the
> expected NBD_REP_ACK), so it could now loop forever against a
> malicious server? (This is not taking into account any later patches)
The loop can execute at most twice:
>
> Rich.
>
>> char *name;
>>
>> - if (reply.length != sizeof(info->context_id) + context_len) {
>> + if (reply.length <= sizeof(info->context_id) ||
>> + reply.length > NBD_MAX_BUFFER_SIZE) {
>> error_setg(errp, "Failed to negotiate meta context '%s', server "
>> "answered with unexpected length %" PRIu32, context,
>> reply.length);
>> @@ -708,6 +709,15 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
>> return -1;
>> }
>> name[reply.length] = '\0';
>> + trace_nbd_opt_meta_reply(name, info->context_id);
>> +
>> + if (received) {
>> + error_setg(errp, "Server replied with more than one context");
>> + g_free(name);
>> + nbd_send_opt_abort(ioc);
>> + return -1;
>> + }
If the server replies with a second context, we break the loop by
complaining.
The old code accepted at most one context, by complaining if the
server's second reply was not ACK; the new code accepts at most one
context, by complaining if the server sent more than one context, so the
net effect of killing the connection for a misbehaving server response
to SET is unchanged.
However, your point about a misbehaving server providing an infinite
stream of responses to NBD_OPT_LIST or NBD_OPT_LIST_META_CONTEXT is an
interesting question, and may be worth asking upstream to see if the NBD
protocol should be tweaked to document any boundaries at how many
listings a server might send before a client should worry about the
server being malicious. (Does not affect this patch, but pre-existing
when we call nbd_receive_list() for servers that lack NBD_OPT_GO, and
does impact the later patches in this series that call
NBD_OPT_LIST_META_CONTEXT for 'qemu-nbd --list').
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 12/22] nbd/client: Improve error handling in nbd_negotiate_simple_meta_context()
2018-12-15 15:19 ` Richard W.M. Jones
2018-12-17 15:26 ` Eric Blake
@ 2018-12-17 15:30 ` Eric Blake
2018-12-20 14:13 ` Vladimir Sementsov-Ogievskiy
1 sibling, 1 reply; 55+ messages in thread
From: Eric Blake @ 2018-12-17 15:30 UTC (permalink / raw)
To: Richard W.M. Jones; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On 12/15/18 9:19 AM, Richard W.M. Jones wrote:
> On Sat, Dec 15, 2018 at 07:53:14AM -0600, Eric Blake wrote:
>> Always allocate space for the reply returned by the server and
>> hoist the trace earlier, as it is more interesting to trace the
>> server's reply (even if it is unexpected) than parroting our
>> request only on success. After all, skipping the allocation
>> for a wrong size was merely a micro-optimization that only
>> benefitted a broken server, rather than the common case of a
>> compliant server that meets our expectations.
>>
>> Then turn the reply handling into a loop (even though we still
>> never iterate more than once), to make this code easier to use
>> when later patches do support multiple server replies. This
>> changes the error message for a server with two replies (a
>> corner case we are unlikely to hit in practice) from:
>>
>> Unexpected reply type 4 (meta context), expected 0 (ack)
>>
>> to:
>>
>> Server replied with more than one context
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> ---
>> v2: split patch into easier-to-review pieces [Rich, Vladimir]
>> ---
>> nbd/client.c | 16 ++++++++++++----
>> 1 file changed, 12 insertions(+), 4 deletions(-)
>>
>> diff --git a/nbd/client.c b/nbd/client.c
>> index bcccd5f555e..b6a85fc3ef8 100644
>> --- a/nbd/client.c
>> +++ b/nbd/client.c
>> @@ -684,10 +684,11 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
>> return ret;
>> }
>>
>> - if (reply.type == NBD_REP_META_CONTEXT) {
>> + while (reply.type == NBD_REP_META_CONTEXT) {
>
> I'm not sure I understand why this change is safe.
>
> As far as I can see reply.type is only updated in the loop by
> nbd_receive_option_reply, and that reads from the server, and so the
> server might keep sending NBD_REP_META_CONTEXT packets (instead of the
> expected NBD_REP_ACK), so it could now loop forever against a
> malicious server? (This is not taking into account any later patches)
Hmm - now that I've already responded to why the conversion to a loop
does not change this code, I'm now wondering if I even need this patch.
In v1 of the series, both SET and LIST shared a common function, and
since LIST needs the loop, converting SET to use a loop that exits early
if it executes more than once was needed to make the two actions share a
common entry point. But since v2 uses different entry points (because
it separated the common code into separate helper functions, leaving the
SET entry point unchanged and adding a new LIST entry point), where only
the LIST entry point actually has to loop, I might be able to just drop
this patch entirely and still achieve the same effect.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 13/22] nbd/client: Split out nbd_send_one_meta_context()
2018-12-15 15:22 ` Richard W.M. Jones
@ 2018-12-17 15:34 ` Eric Blake
0 siblings, 0 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-17 15:34 UTC (permalink / raw)
To: Richard W.M. Jones; +Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block
On 12/15/18 9:22 AM, Richard W.M. Jones wrote:
> On Sat, Dec 15, 2018 at 07:53:15AM -0600, Eric Blake wrote:
>> Refactor nbd_negotiate_simple_meta_context() to pull out the
>> code that can be reused to send a LIST request for 0 or 1 query.
>> No semantic change.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> ---
>> v2: split patch into easier-to-review pieces [Rich, Vladimir]
>> ---
>> nbd/client.c | 64 ++++++++++++++++++++++++++++++++++--------------
>> nbd/trace-events | 2 +-
>> 2 files changed, 46 insertions(+), 20 deletions(-)
>> + uint32_t queries = !!query;
This initialization...
>> + uint32_t context_len = 0;
>> + uint32_t data_len;
>> + char *data;
>> + char *p;
>> +
>> + data_len = sizeof(export_len) + export_len + sizeof(queries);
...plus the fact that it is now sizeof(variable) instead of sizeof(type)...
>> @@ -651,26 +693,10 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
>> NBDOptionReply reply;
>> const char *context = info->x_dirty_bitmap ?: "base:allocation";
>> bool received = false;
>> - uint32_t export_len = strlen(info->name);
>> - uint32_t context_len = strlen(context);
>> - uint32_t data_len = sizeof(export_len) + export_len +
>> - sizeof(uint32_t) + /* number of queries */
...was the reason that I dropped the comment here. The comment made
sense for explaining why a sizeof(type) was being injected into data_len,
>> - sizeof(context_len) + context_len;
>> - char *data = g_malloc(data_len);
>> - char *p = data;
>>
>> - trace_nbd_opt_meta_request(context, info->name);
>> - stl_be_p(p, export_len);
>> - memcpy(p += sizeof(export_len), info->name, export_len);
>> - stl_be_p(p += export_len, 1);
...because the old code was hard-coding 1 query, while the new code uses
the variable (whose value is the number of queries) rather than a
hard-coding.
>
> Although a straightforward refactoring, we still lost the comment
> /* number of queries */. I'd still perhaps like to see a bit more
> explanation of the layout and reasoning behind the data buffer.
Perhaps a possible improvement would be to introduce a packed struct
that matches the protocol layout, instead of piece-meal constructing the
struct. But I'm not sure how much effort to spend on this code.
> But anyway ..
>
> Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Thanks. I'm glad you forced me to split the v1 patch into more
manageable pieces; I like how it turned out.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 18/22] nbd/client: Add nbd_receive_export_list()
2018-12-15 15:42 ` Richard W.M. Jones
@ 2018-12-17 15:43 ` Eric Blake
0 siblings, 0 replies; 55+ messages in thread
From: Eric Blake @ 2018-12-17 15:43 UTC (permalink / raw)
To: Richard W.M. Jones
Cc: qemu-devel, nsoffer, jsnow, vsementsov, qemu-block, Kevin Wolf,
Max Reitz
On 12/15/18 9:42 AM, Richard W.M. Jones wrote:
> On Sat, Dec 15, 2018 at 07:53:20AM -0600, Eric Blake wrote:
>> We want to be able to detect whether a given qemu NBD server is
>> exposing the right export(s) and dirty bitmaps, at least for
>> regression testing. We could use 'nbd-client -l' from the upstream
>> NBD project to list exports, but it's annoying to rely on
>> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
>> know about all of the qemu NBD extensions. Thus, we plan on adding
>> a new mode to qemu-nbd that merely sniffs all possible information
>> from the server during handshake phase, then disconnects and dumps
>> the information.
>>
>> This patch adds the low-level client code for grabbing the list
>> of exports. It benefits from the recent refactoring patches, as
>> well as a minor tweak of changing nbd_opt_go() to nbd_opt_info_or_go(),
>> in order to share as much code as possible when it comes to doing
>> validation of server replies. The resulting information is stored
>> in an array of NBDExportInfo which has been expanded to any
>> description string, along with a convenience function for freeing
>> the list.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> ---
>> + while (1) {
>> + char *name;
>> + char *desc;
>> +
>> + rc = nbd_receive_list(ioc, &name, &desc, errp);
>> + if (rc < 0) {
>> + goto out;
>> + } else if (rc == 0) {
>> + break;
>> + }
>> + array = g_renew(NBDExportInfo, array, ++count);
>> + memset(&array[count - 1], 0, sizeof(*array));
>> + array[count - 1].name = name;
>> + array[count - 1].description = desc;
>> + array[count - 1].structured_reply = result == 3;
>> + }
>
> Do we care about limiting ‘count’ to some reasonable value here?
Maybe.
>
> I tried to look at the protocol document to see if there's a limit on
> the number of exports that a server can have, but if there is I cannot
> find it. I don't know how much we care about malicious NBD servers --
> mostly I'm interested in malicious NBD clients :-)
You brought up the same question earlier in the series, and we already
have a pre-existing repeat of the situation with NBD_OPT_LIST for
servers that lack NBD_OPT_GO.
A client can always hang up on a server that it doesn't like; so maybe
we could set our own limit - but what is a good limit? An NBD server
set up to export every file in a given directory, where that directory
has over 1k files, doesn't seem too far-fetched. Having 1000 contexts
available in NBD_OPT_LIST_META_CONTEXT, on the other hand, is less likely.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 02/22] nbd: Document timeline of various features
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 02/22] nbd: Document timeline of various features Eric Blake
2018-12-15 14:02 ` Richard W.M. Jones
@ 2018-12-18 13:03 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 55+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-18 13:03 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, jsnow, rjones, qemu-block
15.12.2018 16:53, Eric Blake wrote:
> It can be useful to figure out which NBD protocol features are
> exposed by a server, as well as what features a client will
> take advantage of if available, for a given qemu release. It's
> not always precise to base features on version numbers (thanks
> to downstream backports), but any documentation is better than
> making users search through git logs themselves.
>
> This patch originally stemmed from a request to document that
> pristine 3.0 has a known bug where NBD_OPT_LIST_META_CONTEXT
> with 0 queries forgot to advertise an available
> "qemu:dirty-bitmap" context, but documenting bugs like this (or
> the fact that 3.0 also botched NBD_CMD_CACHE) gets to be too
> much details, especially since buggy releases will be less
> likely connection targets over time. Instead, I chose to just
> remind users to check stable release branches.
>
> Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: new patch
> ---
> docs/interop/nbd.txt | 19 ++++++++++++++++++-
> 1 file changed, 18 insertions(+), 1 deletion(-)
>
> diff --git a/docs/interop/nbd.txt b/docs/interop/nbd.txt
> index 77b5f459111..2b25f871e7c 100644
> --- a/docs/interop/nbd.txt
> +++ b/docs/interop/nbd.txt
> @@ -15,7 +15,6 @@ Qemu supports the "base:allocation" metadata context as defined in the
> NBD protocol specification, and also defines an additional metadata
> namespace "qemu".
>
> -
> == "qemu" namespace ==
>
> The "qemu" namespace currently contains only one type of context,
> @@ -36,3 +35,21 @@ in addition to "qemu:dirty-bitmap:<dirty-bitmap-export-name>":
> namespace.
> * "qemu:dirty-bitmap:" - returns list of all available dirty-bitmap
> metadata contexts.
> +
> += Features by version =
> +
> +The following list documents which qemu version first implemented
> +various features (both as a server exposing the feature, and as a
> +client taking advantage of the feature when present), to make it
> +easier to plan for cross-version interoperability. Note that in
> +several cases, the initial release containing a feature may require
> +additional patches from the corresponding stable branch to fix bugs in
> +the operation of that feature.
> +
> +* 2.6: NBD_OPT_STARTTLS with TLS X.509 Certificates
> +* 2.8: NBD_CMD_WRITE_ZEROES
> +* 2.10: NBD_OPT_GO, NBD_INFO_BLOCK
> +* 2.11: NBD_OPT_STRUCTURED_READ
NBD_OPT_STRUCTURED_REPLY you mean
> +* 2.12: NBD_CMD_BLOCK_STATUS for "base:allocation"
> +* 3.0: NBD_OPT_STARTTLS with TLS Pre-Shared Keys (PSK),
> +NBD_CMD_BLOCK_STATUS for "qemu:dirty-bitmap:", NBD_CMD_CACHE
>
Hm, trying to check by:
features=$(echo NBD_OPT_{STARTTLS,GO,STRUCTURED_REPLY} NBD_CMD_{WRITE_ZEROES,BLOCK_STATUS,CACHE} NBD_INFO_BLOCK PSK)
versions=$(echo v2.{5..12}.0 v3.0.0)
for f in $features; do echo -------------; echo $f; for v in $versions; do echo $v:; git grep $f $v; done; done | less
looks correct.
Checked base:allocation/qemu:dirty-bitmap appearance.
Don't want to dig in TLS version checking which is I'm not familiar with, so, I'm OK and with s/READ/REPLY:
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 03/22] maint: Allow for EXAMPLES in texi2pod
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 03/22] maint: Allow for EXAMPLES in texi2pod Eric Blake
2018-12-15 14:04 ` Richard W.M. Jones
@ 2018-12-18 13:46 ` Vladimir Sementsov-Ogievskiy
2019-01-04 23:45 ` Eric Blake
1 sibling, 1 reply; 55+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-18 13:46 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, jsnow, rjones, qemu-block
15.12.2018 16:53, Eric Blake wrote:
> The next commit will add an EXAMPLES section to qemu-nbd.8;
> for that to work, we need to recognize EXAMPLES in texi2pod,
> and we need to make all man pages be regenerated since the
> output of texi2pod can be different.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: new patch
> ---
> Makefile | 18 ++++++++++--------
> scripts/texi2pod.pl | 2 +-
> 2 files changed, 11 insertions(+), 9 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index c8b9efdad4b..0bd204eff8a 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -824,14 +824,16 @@ docs/interop/qemu-qmp-qapi.texi: qapi/qapi-doc.texi
> docs/interop/qemu-ga-qapi.texi: qga/qapi-generated/qga-qapi-doc.texi
> @cp -p $< $@
>
> -qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi qemu-monitor-info.texi
> -qemu.1: qemu-option-trace.texi
> -qemu-img.1: qemu-img.texi qemu-option-trace.texi qemu-img-cmds.texi
> -fsdev/virtfs-proxy-helper.1: fsdev/virtfs-proxy-helper.texi
> -qemu-nbd.8: qemu-nbd.texi qemu-option-trace.texi
> -qemu-ga.8: qemu-ga.texi
> -docs/qemu-block-drivers.7: docs/qemu-block-drivers.texi
> -docs/qemu-cpu-models.7: docs/qemu-cpu-models.texi
> +qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi \
> + qemu-monitor-info.texi scripts/texi2pod.pl
> +qemu.1: qemu-option-trace.texi scripts/texi2pod.pl
I think, no needs to add scripts/texi2pod.pl to both qemu.1: lines, as they should be
merged anyway. And if you want to add line breaking anyway, it looks better to use one qemu.1:
dependency line.
On the other hand, it may be better to add MANS variable as subset of DOCS to combine all man page targets,
and than do like $(MANS): scripts/texi2pod.pl.. (and recombine DOCS, using MANS ofcourse)
Hm, and I don't see here docs/interop/qemu-ga-ref.7 and docs/interop/qemu-qmp-ref.7,
which exist in DOCS.
On the third hand, I'm unsure about how all this work, where are rules for these targets,
and should not we add this dependency for all DOCS targets :)
> +qemu-img.1: qemu-img.texi qemu-option-trace.texi qemu-img-cmds.texi \
> + scripts/texi2pod.pl
> +fsdev/virtfs-proxy-helper.1: fsdev/virtfs-proxy-helper.texi scripts/texi2pod.pl
> +qemu-nbd.8: qemu-nbd.texi qemu-option-trace.texi scripts/texi2pod.pl
> +qemu-ga.8: qemu-ga.texi scripts/texi2pod.pl
> +docs/qemu-block-drivers.7: docs/qemu-block-drivers.texi scripts/texi2pod.pl
> +docs/qemu-cpu-models.7: docs/qemu-cpu-models.texi scripts/texi2pod.pl
>
> html: qemu-doc.html docs/interop/qemu-qmp-ref.html docs/interop/qemu-ga-ref.html
> info: qemu-doc.info docs/interop/qemu-qmp-ref.info docs/interop/qemu-ga-ref.info
> diff --git a/scripts/texi2pod.pl b/scripts/texi2pod.pl
> index 39ce584a322..839b7917cf7 100755
> --- a/scripts/texi2pod.pl
> +++ b/scripts/texi2pod.pl
> @@ -398,7 +398,7 @@ $sects{NAME} = "$fn \- $tl\n";
> $sects{FOOTNOTES} .= "=back\n" if exists $sects{FOOTNOTES};
>
> for $sect (qw(NAME SYNOPSIS DESCRIPTION OPTIONS ENVIRONMENT FILES
> - BUGS NOTES FOOTNOTES SEEALSO AUTHOR COPYRIGHT)) {
> + BUGS NOTES FOOTNOTES EXAMPLES SEEALSO AUTHOR COPYRIGHT)) {
> if(exists $sects{$sect}) {
> $head = $sect;
> $head =~ s/SEEALSO/SEE ALSO/;
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 06/22] qemu-nbd: Fail earlier for -c/-d on non-linux
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 06/22] qemu-nbd: Fail earlier for -c/-d on non-linux Eric Blake
2018-12-15 14:15 ` Richard W.M. Jones
@ 2018-12-18 14:26 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 55+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-18 14:26 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, jsnow, rjones, qemu-block
15.12.2018 16:53, Eric Blake wrote:
> Connecting to a /dev/nbdN device is a Linux-specific action.
> We were already masking -c and -d from 'qemu-nbd --help' on
> non-linux. However, while -d fails with a sensible error
> message, it took hunting through a couple of files to prove
> that. What's more, the code for -c doesn't fail until after
> it has created a pthread and tried to open a device - possibly
> even printing an error message with %m on a non-Linux platform
> in spite of the comment that %m is glibc-specific. Make the
> failure happen sooner, then get rid of stubs that are no
> longer needed because of the early exits.
>
> While at it: tweak the blank newlines in --help output to be
> consistent, whether or not built on Linux.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: Hoist -c error message to share with -d message [Vladimir]
> Bonus: gets rid of a stray TAB in nbd/client.c
> ---
> nbd/client.c | 18 +-----------------
> qemu-nbd.c | 21 +++++++++++++++++++--
> 2 files changed, 20 insertions(+), 19 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 5d59d5ba78a..3d9086af39d 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -1029,23 +1029,7 @@ int nbd_disconnect(int fd)
> return 0;
> }
>
> -#else
> -int nbd_init(int fd, QIOChannelSocket *ioc, NBDExportInfo *info,
> - Error **errp)
> -{
> - error_setg(errp, "nbd_init is only supported on Linux");
> - return -ENOTSUP;
> -}
> -
> -int nbd_client(int fd)
> -{
> - return -ENOTSUP;
> -}
> -int nbd_disconnect(int fd)
> -{
> - return -ENOTSUP;
> -}
> -#endif
> +#endif /* __linux__ */
>
> int nbd_send_request(QIOChannel *ioc, NBDRequest *request)
> {
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index e169b839ece..2807e132396 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -43,6 +43,12 @@
> #include "trace/control.h"
> #include "qemu-version.h"
>
> +#ifdef __linux__
> +#define HAVE_NBD_DEVICE 1
> +#else
> +#define HAVE_NBD_DEVICE 0
> +#endif
> +
> #define SOCKET_PATH "/var/lock/qemu-nbd-%s"
> #define QEMU_NBD_OPT_CACHE 256
> #define QEMU_NBD_OPT_AIO 257
> @@ -98,11 +104,11 @@ static void usage(const char *name)
> " specify tracing options\n"
> " --fork fork off the server process and exit the parent\n"
> " once the server is running\n"
> -#ifdef __linux__
> +#if HAVE_NBD_DEVICE
> +"\n"
> "Kernel NBD client support:\n"
> " -c, --connect=DEV connect FILE to the local NBD device DEV\n"
> " -d, --disconnect disconnect the specified device\n"
> -"\n"
> #endif
> "\n"
> "Block device options:\n"
> @@ -236,6 +242,7 @@ static void termsig_handler(int signum)
> }
>
>
> +#if HAVE_NBD_DEVICE
> static void *show_parts(void *arg)
> {
> char *device = arg;
> @@ -321,6 +328,7 @@ out:
> kill(getpid(), SIGTERM);
> return (void *) EXIT_FAILURE;
> }
> +#endif /* HAVE_NBD_DEVICE */
>
> static int nbd_can_accept(void)
> {
> @@ -814,6 +822,12 @@ int main(int argc, char **argv)
> }
> }
>
> +#if !HAVE_NBD_DEVICE
> + if (disconnect || device) {
> + error_report("Kernel /dev/nbdN support not available");
> + exit(EXIT_FAILURE);
> + }
> +#else /* HAVE_NBD_DEVICE */
hm, still there are two places (did I miss it during previous review?),
where device variable is used, so it may be better to move this hunk even upper.
On the other hand, one of these places is unreachable under windows and second
is error path anyway, so, with, or without moving up:
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> if (disconnect) {
> int nbdfd = open(argv[optind], O_RDWR);
> if (nbdfd < 0) {
> @@ -829,6 +843,7 @@ int main(int argc, char **argv)
>
> return 0;
> }
> +#endif
>
> if ((device && !verbose) || fork_process) {
> int stderr_fd[2];
> @@ -1006,6 +1021,7 @@ int main(int argc, char **argv)
> nbd_export_set_description(exp, export_description);
>
> if (device) {
aha, HAVE_NBD_DEVICE is definitely enabled here, we need #if only to mask symbol which
is undefined if HAVE_NBD_DEVICE is 0.
it may look funny, but why not add
assert(HAVE_NBD_DEVICE);
here ? :)
> +#if HAVE_NBD_DEVICE > int ret;
>
> ret = pthread_create(&client_thread, NULL, nbd_client_thread, device);
> @@ -1013,6 +1029,7 @@ int main(int argc, char **argv)
> error_report("Failed to create client thread: %s", strerror(ret));
> exit(EXIT_FAILURE);
> }
> +#endif
> } else {
> /* Shut up GCC warnings. */
> memset(&client_thread, 0, sizeof(client_thread));
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 07/22] qemu-nbd: Avoid strtol open-coding
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 07/22] qemu-nbd: Avoid strtol open-coding Eric Blake
2018-12-15 14:17 ` Richard W.M. Jones
@ 2018-12-18 15:11 ` Vladimir Sementsov-Ogievskiy
2019-01-04 23:50 ` Eric Blake
1 sibling, 1 reply; 55+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-18 15:11 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, jsnow, rjones, qemu-block
15.12.2018 16:53, Eric Blake wrote:
> Our copy-and-pasted open-coding of strtol handling forgot to
> handle overflow conditions. Use qemu_strto*() instead.
>
> In the case of --partition, since we insist on a user-supplied
> partition to be non-zero, we can use 0 rather than -1 for our
> initial value to distinguish when a partition is not being
> served, for slightly more optimal code.
>
> The error messages for out-of-bounds values are less specific,
> but should not be a terrible loss in quality.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: Retitle, catch more uses of strtol
> [Hmm - this depends on int64_t and off_t being compatible; if they
> aren't that way on all platforms, I'll need a temporary variable]
hmm, as I understand, even if this compatibility exists, it's not a part
of standard and nothing about off_t size in POSIX..
Moreover: what is the reason for using off_t in NBD code? We don't have it
in NBD protocol, we don't have it in generic block layer interface. Isn't it
always casted to int64_t or like this?
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 09/22] nbd/client: Refactor nbd_receive_list()
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 09/22] nbd/client: Refactor nbd_receive_list() Eric Blake
2018-12-15 15:07 ` Richard W.M. Jones
@ 2018-12-19 13:11 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 55+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-19 13:11 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: nsoffer, jsnow, rjones, qemu-block
15.12.2018 16:53, Eric Blake wrote:
> Right now, nbd_receive_list() is only called by
> nbd_receive_query_exports(), which in turn is only called if the
> server lacks NBD_OPT_GO but has working option negotiation, and is
> merely used as a quality-of-implementation trick since servers
> can't give decent errors for NBD_OPT_EXPORT_NAME. However, servers
> that lack NBD_OPT_GO are becoming increasingly rare (nbdkit was a
> latecomer, in Aug 2018, but qemu has been such a server since commit
> f37708f6 in July 2017 and released in 2.10), so it no longer makes
> sense to micro-optimize that function for performance.
>
> Furthermore, when debugging a server's implementation, tracing the
> full reply (both names and descriptions) is useful, not to mention
> that upcoming patches adding 'qemu-nbd --list' will want to collect
> that data. And when you consider that a server can send an export
> name up to the NBD protocol length limit of 4k; but our current
> NBD_MAX_NAME_SIZE is only 256, we can't trace all valid server
> names without more storage, but 4k is large enough that the heap
> is better than the stack for long names.
>
> Thus, I'm changing the division of labor, with nbd_receive_list()
> now always malloc'ing a result on success (the malloc is bounded
> by the fact that we reject servers with a reply length larger
> than 32M), and moving the comparison to 'wantname' to the caller.
>
> There is a minor change in behavior where a server with 0 exports
> (an immediate NBD_REP_ACK reply) is now no longer distinguished
> from a server without LIST support (NBD_REP_ERR_UNSUP); this
> information could be preserved with a complication to the calling
> contract to provide a bit more information, but I didn't see the
> point. After all, the worst that can happen if our guess at a
> match is wrong is that the caller will get a cryptic disconnect
> when NBD_OPT_EXPORT_NAME fails (which is no different from what
> would happen if we had not tried LIST), while treating an empty
> list as immediate failure would prevent connecting to really old
> servers that really did lack LIST. Besides, NBD servers with 0
> exports are rare (qemu can do it when using QMP nbd-server-start
> without nbd-server-add - but qemu understands NBD_OPT_GO and
> thus won't tickle this change in behavior).
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: Rewrite in a different manner (move the comparison to the
> caller)
> Fix free to g_free [Vladimir]
> ---
> nbd/client.c | 89 +++++++++++++++++++++++++++++++-----------------
> nbd/trace-events | 1 +
> 2 files changed, 58 insertions(+), 32 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 1a3a620fb6d..28f5a286cba 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -232,18 +232,24 @@ static int nbd_handle_reply_err(QIOChannel *ioc, NBDOptionReply *reply,
> return result;
> }
>
> -/* Process another portion of the NBD_OPT_LIST reply. Set *@match if
> - * the current reply matches @want or if the server does not support
> - * NBD_OPT_LIST, otherwise leave @match alone. Return 0 if iteration
> - * is complete, positive if more replies are expected, or negative
> - * with @errp set if an unrecoverable error occurred. */
> -static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> +/* nbd_receive_list:
> + * Process another portion of the NBD_OPT_LIST reply, populating any
> + * name received into *@name. If @description is non-NULL, and the
> + * server provided a description, that is also populated. The caller
> + * must eventually call g_free() on success.
> + * Returns 1 if a name was received and iteration must continue,
> + * 0 if iteration is complete,
maybe, something like: if iteration is complete, or NBD_OPT_LIST is unsupported. @name and
@description are not set.
(it's not obvious thing, that iteration complete assumes no new name received, and that 0
serves unsupported too)
> + * -1 with @errp set if an unrecoverable error occurred.
> + */
> +static int nbd_receive_list(QIOChannel *ioc, char **name, char **description,
> Error **errp)
> {
> + int ret = -1;
> NBDOptionReply reply;
> uint32_t len;
> uint32_t namelen;
> - char name[NBD_MAX_NAME_SIZE + 1];
> + char *local_name = NULL;
> + char *local_desc = NULL;
> int error;
>
> if (nbd_receive_option_reply(ioc, NBD_OPT_LIST, &reply, errp) < 0) {
> @@ -251,9 +257,6 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> }
> error = nbd_handle_reply_err(ioc, &reply, errp);
> if (error <= 0) {
> - /* The server did not support NBD_OPT_LIST, so set *match on
> - * the assumption that any name will be accepted. */
> - *match = true;
hm, actually, match was wrongly set to true on negative error, but it doesn't matter.
> return error;
> }
> len = reply.length;
> @@ -290,33 +293,38 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> nbd_send_opt_abort(ioc);
> return -1;
> }
> - if (namelen != strlen(want)) {
> - if (nbd_drop(ioc, len, errp) < 0) {
> - error_prepend(errp,
> - "failed to skip export name with wrong length: ");
> - nbd_send_opt_abort(ioc);
> - return -1;
> - }
> - return 1;
> - }
>
> - assert(namelen < sizeof(name));
> - if (nbd_read(ioc, name, namelen, errp) < 0) {
> + local_name = g_malloc(namelen + 1);
> + if (nbd_read(ioc, local_name, namelen, errp) < 0) {
> error_prepend(errp, "failed to read export name: ");
> nbd_send_opt_abort(ioc);
> - return -1;
> + goto out;
> }
> - name[namelen] = '\0';
> + local_name[namelen] = '\0';
> len -= namelen;
> - if (nbd_drop(ioc, len, errp) < 0) {
> - error_prepend(errp, "failed to read export description: ");
> - nbd_send_opt_abort(ioc);
> - return -1;
> + if (len) {
> + local_desc = g_malloc(len + 1);
> + if (nbd_read(ioc, local_desc, len, errp) < 0) {
> + error_prepend(errp, "failed to read export description: ");
> + nbd_send_opt_abort(ioc);
> + goto out;
> + }
> + local_desc[len] = '\0';
> }
> - if (!strcmp(name, want)) {
> - *match = true;
> +
> + trace_nbd_receive_list(local_name, local_desc ?: "");
> + *name = local_name;
> + local_name = NULL;
> + if (description) {
> + *description = local_desc;
> + local_desc = NULL;
> }
> - return 1;
> + ret = 1;
> +
> + out:
> + g_free(local_name);
> + g_free(local_desc);
> + return ret;
> }
>
>
> @@ -491,6 +499,7 @@ static int nbd_receive_query_exports(QIOChannel *ioc,
> const char *wantname,
> Error **errp)
> {
> + bool listEmpty = true;
> bool foundExport = false;
being here, better is to fix naming to Qemu coding style, list_empty and found_export
>
> trace_nbd_receive_query_exports_start(wantname);
> @@ -499,14 +508,25 @@ static int nbd_receive_query_exports(QIOChannel *ioc,
> }
>
> while (1) {
> - int ret = nbd_receive_list(ioc, wantname, &foundExport, errp);
> + char *name;
> + int ret = nbd_receive_list(ioc, &name, NULL, errp);
>
> if (ret < 0) {
> /* Server gave unexpected reply */
> return -1;
> } else if (ret == 0) {
> /* Done iterating. */
> - if (!foundExport) {
> + if (listEmpty) {
> + /*
> + * We don't have enough context to tell a server that
> + * sent an empty list apart from a server that does
> + * not support the list command; but as this function
> + * is just used to trigger a nicer error message
> + * before trying NBD_OPT_EXPORT_NAME, assume the
> + * export is available.
> + */
> + return 0;
> + } else if (!foundExport) {
> error_setg(errp, "No export with name '%s' available",
> wantname);
> nbd_send_opt_abort(ioc);
> @@ -515,6 +535,11 @@ static int nbd_receive_query_exports(QIOChannel *ioc,
> trace_nbd_receive_query_exports_success(wantname);
> return 0;
> }
> + listEmpty = false;
> + if (!strcmp(name, wantname)) {
> + foundExport = true;
> + }
> + g_free(name);
> }
> }
>
> diff --git a/nbd/trace-events b/nbd/trace-events
> index 5e1d4afe8e6..8e9fc024c28 100644
> --- a/nbd/trace-events
> +++ b/nbd/trace-events
> @@ -2,6 +2,7 @@
> nbd_send_option_request(uint32_t opt, const char *name, uint32_t len) "Sending option request %" PRIu32" (%s), len %" PRIu32
> nbd_receive_option_reply(uint32_t option, const char *optname, uint32_t type, const char *typename, uint32_t length) "Received option reply %" PRIu32" (%s), type %" PRIu32" (%s), len %" PRIu32
> nbd_reply_err_unsup(uint32_t option, const char *name) "server doesn't understand request %" PRIu32 " (%s), attempting fallback"
> +nbd_receive_list(const char *name, const char *desc) "export list includes '%s', description '%s'"
> nbd_opt_go_start(const char *name) "Attempting NBD_OPT_GO for export '%s'"
> nbd_opt_go_success(void) "Export is good to go"
> nbd_opt_go_info_unknown(int info, const char *name) "Ignoring unknown info %d (%s)"
>
New semantics of nbd_receive_list is much more simple, thank you! With or without my tiny suggestions:
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 11/22] nbd/client: Change signature of nbd_negotiate_simple_meta_context()
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 11/22] nbd/client: Change signature of nbd_negotiate_simple_meta_context() Eric Blake
2018-12-15 15:12 ` Richard W.M. Jones
@ 2018-12-20 13:37 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 55+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-20 13:37 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: nsoffer, jsnow, rjones, qemu-block, Kevin Wolf, Max Reitz
15.12.2018 16:53, Eric Blake wrote:
> Pass 'info' instead of three separate parameters related to info,
> when requesting the server to set the meta context. Update the
> NBDExportInfo struct to rename the received id field to match the
> fact that we are currently overloading the field to match whatever
> context the user supplied through the x-dirty-bitmap hack, as well
> as adding a TODO comment to remind future patches about a desire
> to request two contexts at once.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> v2: split patch into easier-to-review pieces [Rich, Vladimir]
> rename NBDExportInfo meta_base_allocation_id [Vladimir]
> ---
[..]
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -628,26 +628,30 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
> }
>
> /* nbd_negotiate_simple_meta_context:
> - * Set one meta context. Simple means that reply must contain zero (not
> - * negotiated) or one (negotiated) contexts. More contexts would be considered
> - * as a protocol error. It's also implied that meta-data query equals queried
> - * context name, so, if server replies with something different than @context,
> - * it is considered an error too.
> - * return 1 for successful negotiation, context_id is set
> + * Request the server to set the meta context for export @info->name
> + * using @info->x_dirty_bitmap with a fallback to "base:allocation",
> + * setting @info->context_id to the resulting id. Fail if the server
> + * responds with more than one context or with a context different
> + * than the query.
> + * return 1 for successful negotiation,
> * 0 if operation is unsupported,
> * -1 with errp set for any other error
> */
> static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> - const char *export,
> - const char *context,
> - uint32_t *context_id,
> + NBDExportInfo *info,
> Error **errp)
> {
> + /*
> + * TODO: Removing the x_dirty_bitmap hack will mean refactoring
> + * this function to request and store ids for multiple contexts
> + * (both base:allocation and a dirty bitmap), at which point this
> + * function should lose the term _simple.
> + */
> int ret;
> NBDOptionReply reply;
> - uint32_t received_id = 0;
> + const char *context = info->x_dirty_bitmap ?: "base:allocation";
> bool received = false;
> - uint32_t export_len = strlen(export);
> + uint32_t export_len = strlen(info->name);
> uint32_t context_len = strlen(context);
> uint32_t data_len = sizeof(export_len) + export_len +
> sizeof(uint32_t) + /* number of queries */
> @@ -655,9 +659,9 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> char *data = g_malloc(data_len);
> char *p = data;
>
> - trace_nbd_opt_meta_request(context, export);
> + trace_nbd_opt_meta_request(context, info->name);
> stl_be_p(p, export_len);
> - memcpy(p += sizeof(export_len), export, export_len);
> + memcpy(p += sizeof(export_len), info->name, export_len);
> stl_be_p(p += export_len, 1);
> stl_be_p(p += sizeof(uint32_t), context_len);
> memcpy(p += sizeof(context_len), context, context_len);
> @@ -683,7 +687,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> if (reply.type == NBD_REP_META_CONTEXT) {
> char *name;
>
> - if (reply.length != sizeof(received_id) + context_len) {
> + if (reply.length != sizeof(info->context_id) + context_len) {
> error_setg(errp, "Failed to negotiate meta context '%s', server "
> "answered with unexpected length %" PRIu32, context,
> reply.length);
> @@ -691,12 +695,13 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return -1;
> }
>
> - if (nbd_read(ioc, &received_id, sizeof(received_id), errp) < 0) {
here you may set info->context_id on failure path. Looks like no harm in it (we use
info->base_allocation and on the other hand, zero is possible id), so I'm not against.
I just try to keep "no change on fail" semantics whenever possible, and don't care,
is it really needed or not. Do you have an opinion on this topic, or may be some
general agreements exists on it?
ok, in this case it looks like a kind of overlogic, and, anyway, agree with Richard that
now the refactoring is very clear:
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 12/22] nbd/client: Improve error handling in nbd_negotiate_simple_meta_context()
2018-12-17 15:30 ` Eric Blake
@ 2018-12-20 14:13 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 55+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-20 14:13 UTC (permalink / raw)
To: Eric Blake, Richard W.M. Jones; +Cc: qemu-devel, nsoffer, jsnow, qemu-block
17.12.2018 18:30, Eric Blake wrote:
> On 12/15/18 9:19 AM, Richard W.M. Jones wrote:
>> On Sat, Dec 15, 2018 at 07:53:14AM -0600, Eric Blake wrote:
>>> Always allocate space for the reply returned by the server and
>>> hoist the trace earlier, as it is more interesting to trace the
>>> server's reply (even if it is unexpected) than parroting our
>>> request only on success. After all, skipping the allocation
>>> for a wrong size was merely a micro-optimization that only
>>> benefitted a broken server, rather than the common case of a
>>> compliant server that meets our expectations.
>>>
>>> Then turn the reply handling into a loop (even though we still
>>> never iterate more than once), to make this code easier to use
>>> when later patches do support multiple server replies. This
>>> changes the error message for a server with two replies (a
>>> corner case we are unlikely to hit in practice) from:
>>>
>>> Unexpected reply type 4 (meta context), expected 0 (ack)
>>>
>>> to:
>>>
>>> Server replied with more than one context
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>>
>>> ---
>>> v2: split patch into easier-to-review pieces [Rich, Vladimir]
>>> ---
>>> nbd/client.c | 16 ++++++++++++----
>>> 1 file changed, 12 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/nbd/client.c b/nbd/client.c
>>> index bcccd5f555e..b6a85fc3ef8 100644
>>> --- a/nbd/client.c
>>> +++ b/nbd/client.c
>>> @@ -684,10 +684,11 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
>>> return ret;
>>> }
>>>
>>> - if (reply.type == NBD_REP_META_CONTEXT) {
>>> + while (reply.type == NBD_REP_META_CONTEXT) {
>>
>> I'm not sure I understand why this change is safe.
>>
>> As far as I can see reply.type is only updated in the loop by
>> nbd_receive_option_reply, and that reads from the server, and so the
>> server might keep sending NBD_REP_META_CONTEXT packets (instead of the
>> expected NBD_REP_ACK), so it could now loop forever against a
>> malicious server? (This is not taking into account any later patches)
>
> Hmm - now that I've already responded to why the conversion to a loop does not change this code, I'm now wondering if I even need this patch. In v1 of the series, both SET and LIST shared a common function, and since LIST needs the loop, converting SET to use a loop that exits early if it executes more than once was needed to make the two actions share a common entry point. But since v2 uses different entry points (because it separated the common code into separate helper functions, leaving the SET entry point unchanged and adding a new LIST entry point), where only the LIST entry point actually has to loop, I might be able to just drop this patch entirely and still achieve the same effect.
>
Are you going to resend series without this patch? In this case I'd prefer to continue review on already rebased patches. Or it doesn't make much difference?
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 03/22] maint: Allow for EXAMPLES in texi2pod
2018-12-18 13:46 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-04 23:45 ` Eric Blake
2019-01-11 2:56 ` Eric Blake
0 siblings, 1 reply; 55+ messages in thread
From: Eric Blake @ 2019-01-04 23:45 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, jsnow, rjones, qemu-block
[-- Attachment #1: Type: text/plain, Size: 1456 bytes --]
On 12/18/18 7:46 AM, Vladimir Sementsov-Ogievskiy wrote:
> 15.12.2018 16:53, Eric Blake wrote:
>> The next commit will add an EXAMPLES section to qemu-nbd.8;
>> for that to work, we need to recognize EXAMPLES in texi2pod,
>> and we need to make all man pages be regenerated since the
>> output of texi2pod can be different.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> +qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi \
>> + qemu-monitor-info.texi scripts/texi2pod.pl
>> +qemu.1: qemu-option-trace.texi scripts/texi2pod.pl
>
> I think, no needs to add scripts/texi2pod.pl to both qemu.1: lines, as they should be
> merged anyway. And if you want to add line breaking anyway, it looks better to use one qemu.1:
> dependency line.
Easy to fix.
>
> On the other hand, it may be better to add MANS variable as subset of DOCS to combine all man page targets,
> and than do like $(MANS): scripts/texi2pod.pl.. (and recombine DOCS, using MANS ofcourse)
> Hm, and I don't see here docs/interop/qemu-ga-ref.7 and docs/interop/qemu-qmp-ref.7,
> which exist in DOCS.
Oh, good point. The .txt files should also have the dependency - in
general, ALL generated documentation should depend on the generator, not
just the man pages. I'll have to think about respinning this patch.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 04/22] qemu-nbd: Enhance man page
2018-12-17 15:19 ` Eric Blake
@ 2019-01-04 23:47 ` Eric Blake
0 siblings, 0 replies; 55+ messages in thread
From: Eric Blake @ 2019-01-04 23:47 UTC (permalink / raw)
To: Richard W.M. Jones; +Cc: nsoffer, vsementsov, jsnow, qemu-devel, qemu-block
[-- Attachment #1: Type: text/plain, Size: 1456 bytes --]
On 12/17/18 9:19 AM, Eric Blake wrote:
>>
>>> +@c man begin EXAMPLES
>>> +Start a server listening on port 10809 that exposes only the
>>> +guest-visible contents of a qcow2 file, with no TLS encryption, and
>>> +with the default export name (an empty string). The command will block
>>> +until the first successful client disconnects:
>>
>> TBH I'd always include the -t option in every example. I don't
>> understand (except for backwards compatibility) why it isn't the
>> default since it's something I always trip over when using qemu-nbd.
>
> I'd still like one example without -t, to call out specifically that it
> creates a one-shot server that goes away after the first client, but
> don't mind fixing the rest of the examples to use -t.
>
> Using -e for read-only connections makes sense, using -e for writable
> exports is a bit more questionable - we _don't_ advertise the
> NBD_FLAG_CAN_MULTI_CONN which states that caches are kept consistent
> between simultaneous write connections, although maybe we should see if
> qemu-nbd can start promising multi-write consistency in future patches?
And I see you've just posted patches for nbdkit to start advertising
CAN_MULTI_CONN - so I really need to spend some time figuring out when
it makes sense for qemu to advertise the flag.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 07/22] qemu-nbd: Avoid strtol open-coding
2018-12-18 15:11 ` Vladimir Sementsov-Ogievskiy
@ 2019-01-04 23:50 ` Eric Blake
2019-01-11 22:47 ` Eric Blake
0 siblings, 1 reply; 55+ messages in thread
From: Eric Blake @ 2019-01-04 23:50 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, jsnow, rjones, qemu-block
[-- Attachment #1: Type: text/plain, Size: 1660 bytes --]
On 12/18/18 9:11 AM, Vladimir Sementsov-Ogievskiy wrote:
> 15.12.2018 16:53, Eric Blake wrote:
>> Our copy-and-pasted open-coding of strtol handling forgot to
>> handle overflow conditions. Use qemu_strto*() instead.
>>
>> In the case of --partition, since we insist on a user-supplied
>> partition to be non-zero, we can use 0 rather than -1 for our
>> initial value to distinguish when a partition is not being
>> served, for slightly more optimal code.
>>
>> The error messages for out-of-bounds values are less specific,
>> but should not be a terrible loss in quality.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> ---
>> v2: Retitle, catch more uses of strtol
>> [Hmm - this depends on int64_t and off_t being compatible; if they
>> aren't that way on all platforms, I'll need a temporary variable]
>
> hmm, as I understand, even if this compatibility exists, it's not a part
> of standard and nothing about off_t size in POSIX..
off_t allows you to run on older systems with 32-bit offsets and newer
systems with 64-bit offsets; but these days, even on 32-bit systems, we
compile qemu to always ask for 64-bit off_t. Using off_t instead of
int64_t is probably a separate cleanup, but one that may be worth making
prior to this patch, so I'll defer this one to my v3.
>
> Moreover: what is the reason for using off_t in NBD code? We don't have it
> in NBD protocol, we don't have it in generic block layer interface. Isn't it
> always casted to int64_t or like this?
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 08/22] nbd/client: Drop pointless buf variable
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 08/22] nbd/client: Drop pointless buf variable Eric Blake
@ 2019-01-05 13:54 ` Eric Blake
0 siblings, 0 replies; 55+ messages in thread
From: Eric Blake @ 2019-01-05 13:54 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, vsementsov, jsnow, rjones, qemu-block
[-- Attachment #1: Type: text/plain, Size: 1278 bytes --]
On 12/15/18 7:53 AM, Eric Blake wrote:
> There's no need to read into a temporary buffer (oversized
> since commit 7d3123e1) followed by a byteswap into a uint64_t
> to check for a magic number via memcmp(), when the code
> immediately below demonstrates reading into the uint64_t then
> byteswapping in place and checking for a magic number via
> integer math. What's more, having a different error message
> when the server's first reply byte is 0 is unusual - it's no
> different from any other wrong magic number, and we already
> detected short reads. That whole strlen() issue has been
> present and useless since commit 1d45f8b5 in 2010; perhaps it
> was leftover debugging (since the correct magic number happens
> to be ASCII)? Make the error messages more consistent and
> detailed while touching things.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
I'm queuing 1, 2, 5, 6, and 8 for my next NBD pull request, and will
respin the rest of the series to incorporate review comments.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 03/22] maint: Allow for EXAMPLES in texi2pod
2019-01-04 23:45 ` Eric Blake
@ 2019-01-11 2:56 ` Eric Blake
0 siblings, 0 replies; 55+ messages in thread
From: Eric Blake @ 2019-01-11 2:56 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel; +Cc: qemu-block
[-- Attachment #1: Type: text/plain, Size: 2148 bytes --]
On 1/4/19 5:45 PM, Eric Blake wrote:
> On 12/18/18 7:46 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 15.12.2018 16:53, Eric Blake wrote:
>>> The next commit will add an EXAMPLES section to qemu-nbd.8;
>>> for that to work, we need to recognize EXAMPLES in texi2pod,
>>> and we need to make all man pages be regenerated since the
>>> output of texi2pod can be different.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>>
>
>>> +qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi \
>>> + qemu-monitor-info.texi scripts/texi2pod.pl
>>> +qemu.1: qemu-option-trace.texi scripts/texi2pod.pl
>>
>> I think, no needs to add scripts/texi2pod.pl to both qemu.1: lines, as they should be
>> merged anyway. And if you want to add line breaking anyway, it looks better to use one qemu.1:
>> dependency line.
>
> Easy to fix.
>
>>
>> On the other hand, it may be better to add MANS variable as subset of DOCS to combine all man page targets,
>> and than do like $(MANS): scripts/texi2pod.pl.. (and recombine DOCS, using MANS ofcourse)
Even easier (although it took me a while digging through 'info man' to
learn it):
$(filter %.1 %.7 %.8,$(DOCS)): scripts/texi2pod.pl
although the dependency must be listed AFTER any more specific rule
(since rules.mak has a %.1: pattern rule which expands $< to the first
dependency, and we do not want scripts/texi2pod.pl to be treated as our
first dependency).
>> Hm, and I don't see here docs/interop/qemu-ga-ref.7 and docs/interop/qemu-qmp-ref.7,
>> which exist in DOCS.
They are there, just harder to see (hence why my v2 missed them). Of
course, doing the generic dependency is better than editing every single
man file listing.
>
> Oh, good point. The .txt files should also have the dependency
Or not. The .txt output files depend on the .texi source, not on the
generated .1 page; only the files that used scripts/texi2pod.pl need to
depend on that, and based on rules.mk, that is only the 3 man page
categories.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Qemu-devel] [PATCH v2 07/22] qemu-nbd: Avoid strtol open-coding
2019-01-04 23:50 ` Eric Blake
@ 2019-01-11 22:47 ` Eric Blake
0 siblings, 0 replies; 55+ messages in thread
From: Eric Blake @ 2019-01-11 22:47 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, jsnow, rjones, qemu-block
[-- Attachment #1: Type: text/plain, Size: 1560 bytes --]
On 1/4/19 5:50 PM, Eric Blake wrote:
>> hmm, as I understand, even if this compatibility exists, it's not a part
>> of standard and nothing about off_t size in POSIX..
>
> off_t allows you to run on older systems with 32-bit offsets and newer
> systems with 64-bit offsets; but these days, even on 32-bit systems, we
> compile qemu to always ask for 64-bit off_t. Using off_t instead of
> int64_t is probably a separate cleanup, but one that may be worth making
> prior to this patch, so I'll defer this one to my v3.
>
>>
>> Moreover: what is the reason for using off_t in NBD code? We don't have it
>> in NBD protocol, we don't have it in generic block layer interface. Isn't it
>> always casted to int64_t or like this?
Thanks again for asking this question. In auditing the use of off_t, I
found that 'qemu-nbd -P 1' happily tries to read beyond the bounds of
the BDS. Thankfully, I can't find an exploit (escaped the CVE bullet -
no DoS assertion, overlarge malloc, information leak, or other nasty
problem), merely a permanent EIO down the road once the client tries to
access advertised available bytes; but I'm also adding a patch to my v3
series that does sanity checking (as we should NEVER blindly trust
values in a potentially-malicious image as being in bounds, so that
clients can't even connect to such images in the first place). [I also
think that qemu-nbd -P is seldom-used...]
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 55+ messages in thread
end of thread, other threads:[~2019-01-11 22:47 UTC | newest]
Thread overview: 55+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-12-15 13:53 [Qemu-devel] [PATCH v2 00/22] nbd: add qemu-nbd --list Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 01/22] qemu-nbd: Use program name in error messages Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 02/22] nbd: Document timeline of various features Eric Blake
2018-12-15 14:02 ` Richard W.M. Jones
2018-12-18 13:03 ` Vladimir Sementsov-Ogievskiy
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 03/22] maint: Allow for EXAMPLES in texi2pod Eric Blake
2018-12-15 14:04 ` Richard W.M. Jones
2018-12-18 13:46 ` Vladimir Sementsov-Ogievskiy
2019-01-04 23:45 ` Eric Blake
2019-01-11 2:56 ` Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 04/22] qemu-nbd: Enhance man page Eric Blake
2018-12-15 14:13 ` Richard W.M. Jones
2018-12-17 15:19 ` Eric Blake
2019-01-04 23:47 ` Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 05/22] nbd/client: More consistent error messages Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 06/22] qemu-nbd: Fail earlier for -c/-d on non-linux Eric Blake
2018-12-15 14:15 ` Richard W.M. Jones
2018-12-18 14:26 ` Vladimir Sementsov-Ogievskiy
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 07/22] qemu-nbd: Avoid strtol open-coding Eric Blake
2018-12-15 14:17 ` Richard W.M. Jones
2018-12-18 15:11 ` Vladimir Sementsov-Ogievskiy
2019-01-04 23:50 ` Eric Blake
2019-01-11 22:47 ` Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 08/22] nbd/client: Drop pointless buf variable Eric Blake
2019-01-05 13:54 ` Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 09/22] nbd/client: Refactor nbd_receive_list() Eric Blake
2018-12-15 15:07 ` Richard W.M. Jones
2018-12-19 13:11 ` Vladimir Sementsov-Ogievskiy
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 10/22] nbd/client: Move export name into NBDExportInfo Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 11/22] nbd/client: Change signature of nbd_negotiate_simple_meta_context() Eric Blake
2018-12-15 15:12 ` Richard W.M. Jones
2018-12-20 13:37 ` Vladimir Sementsov-Ogievskiy
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 12/22] nbd/client: Improve error handling in nbd_negotiate_simple_meta_context() Eric Blake
2018-12-15 15:19 ` Richard W.M. Jones
2018-12-17 15:26 ` Eric Blake
2018-12-17 15:30 ` Eric Blake
2018-12-20 14:13 ` Vladimir Sementsov-Ogievskiy
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 13/22] nbd/client: Split out nbd_send_one_meta_context() Eric Blake
2018-12-15 15:22 ` Richard W.M. Jones
2018-12-17 15:34 ` Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 14/22] nbd/client: Split out nbd_receive_one_meta_context() Eric Blake
2018-12-15 15:30 ` Richard W.M. Jones
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 15/22] nbd/client: Refactor return of nbd_receive_negotiate() Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 16/22] nbd/client: Split handshake into two functions Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 17/22] nbd/client: Pull out oldstyle size determination Eric Blake
2018-12-15 15:31 ` Richard W.M. Jones
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 18/22] nbd/client: Add nbd_receive_export_list() Eric Blake
2018-12-15 15:42 ` Richard W.M. Jones
2018-12-17 15:43 ` Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 19/22] nbd/client: Add meta contexts to nbd_receive_export_list() Eric Blake
2018-12-15 15:59 ` Richard W.M. Jones
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 20/22] qemu-nbd: Add --list option Eric Blake
2018-12-15 16:02 ` Richard W.M. Jones
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 21/22] nbd/client: Work around 3.0 bug for listing meta contexts Eric Blake
2018-12-15 13:53 ` [Qemu-devel] [PATCH v2 22/22] iotests: Enhance 223, 233 to cover 'qemu-nbd --list' Eric Blake
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.