* [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list
@ 2018-11-30 22:03 Eric Blake
2018-11-30 22:03 ` [Qemu-devel] [PATCH 01/14] qemu-nbd: Use program name in error messages Eric Blake
` (14 more replies)
0 siblings, 15 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:03 UTC (permalink / raw)
To: qemu-devel; +Cc: jsnow, nsoffer, rjones, vsementsov, qemu-block
I got tired of debugging whether a server was advertising the
correct things during negotiation by inspecting the trace
logs of qemu-io as client - not to mention that without SOME
sort of client tracing particular commands, we can't easily
regression test the server for correct behavior. The final
straw was at KVM Forum, when Nir asked me to make sure there
was a way to easily determine if an NBD server is exposing what
we really want (and fixing x-dirty-bitmap to behave saner fell
out as a result of answering that question).
I note that upstream NBD has 'nbd-client -l $host' for querying
just export names (with no quoting, so you have to know that
a blank line means the default export), but it wasn't powerful
enough, so I implemented 'qemu-nbd -L' to document everything.
Upstream NBD has separate 'nbd-client' and 'nbd-server' binaries,
while we only have 'qemu-nbd' (which is normally just a server,
but 'qemu-nbd -c' also operates a second thread as a client).
Our other uses of qemu as NBD client are for consuming a block
device (as in qemu-io, qemu-img, or a drive to qemu) - but those
binaries are less suited to something so specific to the NBD
protocol.
Bonus: As a result of my work on this series, nbdkit now supports
NBD_OPT_INFO (my interoperability testing between server
implementations has been paying off, both at fixing server bugs,
and at making this code more reliable across difference in valid
servers).
Based-on: <20181130023232.3079982-1-eblake@redhat.com>
[0/3] NBD dirty bitmap cleanups
(well, only weakly based on it - the two series can be applied
in either order, where if you apply this series but revert 12/14,
then iotest 223 in 14/14 will expose the bug fixed by patch 1/3
in the other series)
Also available at:
https://repo.or.cz/qemu/ericb.git qemu-nbd-list-v1
Eric Blake (14):
qemu-nbd: Use program name in error messages
nbd/client: More consistent error messages
qemu-nbd: Fail earlier for -c/-d on non-linux
qemu-nbd: Simplify --partition handling
nbd/client: Drop pointless buf variable
nbd/client: Move export name into NBDExportInfo
nbd/client: Refactor nbd_negotiate_simple_meta_context()
nbd/client: Refactor nbd_receive_list()
nbd/client: Refactor return of nbd_receive_negotiate()
nbd/client: Split handshake into two functions
nbd/client: Add nbd_receive_export_list()
nbd/client: Work around 3.0 bug for listing meta contexts
qemu-nbd: Add --list option
iotests: Enhance 223, 233 to cover 'qemu-nbd --list'
include/block/nbd.h | 23 +-
nbd/nbd-internal.h | 1 +
block/nbd-client.c | 5 +-
nbd/client.c | 595 +++++++++++++++++++++++++------------
qemu-nbd.c | 196 ++++++++++--
nbd/trace-events | 8 +-
tests/qemu-iotests/223 | 2 +
tests/qemu-iotests/223.out | 20 ++
tests/qemu-iotests/233 | 10 +
tests/qemu-iotests/233.out | 17 +-
10 files changed, 650 insertions(+), 227 deletions(-)
--
2.17.2
^ permalink raw reply [flat|nested] 65+ messages in thread
* [Qemu-devel] [PATCH 01/14] qemu-nbd: Use program name in error messages
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
@ 2018-11-30 22:03 ` Eric Blake
2018-11-30 22:17 ` Richard W.M. Jones
2018-12-05 14:55 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 02/14] nbd/client: More consistent " Eric Blake
` (13 subsequent siblings)
14 siblings, 2 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:03 UTC (permalink / raw)
To: qemu-devel
Cc: jsnow, nsoffer, rjones, vsementsov, qemu-block, Kevin Wolf, Max Reitz
This changes output from:
$ qemu-nbd nosuch
Failed to blk_new_open 'nosuch': Could not open 'nosuch': No such file or directory
to something more consistent with qemu-img and qemu:
$ qemu-nbd nosuch
qemu-nbd: Failed to blk_new_open 'nosuch': Could not open 'nosuch': No such file or directory
Update the lone affected test to match. (Hmm - is it sad that we don't
do much testing of expected failures?)
Signed-off-by: Eric Blake <eblake@redhat.com>
---
qemu-nbd.c | 1 +
tests/qemu-iotests/233.out | 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/qemu-nbd.c b/qemu-nbd.c
index ca7109652e5..e169b839ece 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -571,6 +571,7 @@ int main(int argc, char **argv)
#endif
module_call_init(MODULE_INIT_TRACE);
+ error_set_progname(argv[0]);
qcrypto_init(&error_fatal);
module_call_init(MODULE_INIT_QOM);
diff --git a/tests/qemu-iotests/233.out b/tests/qemu-iotests/233.out
index 94acd9b9479..5f416721b03 100644
--- a/tests/qemu-iotests/233.out
+++ b/tests/qemu-iotests/233.out
@@ -27,7 +27,7 @@ virtual size: 64M (67108864 bytes)
disk size: unavailable
== check TLS with different CA fails ==
-option negotiation failed: Verify failed: No certificate was found.
+qemu-nbd: option negotiation failed: Verify failed: No certificate was found.
qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': The certificate hasn't got a known issuer
== perform I/O over TLS ==
--
2.17.2
^ permalink raw reply related [flat|nested] 65+ messages in thread
* [Qemu-devel] [PATCH 02/14] nbd/client: More consistent error messages
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
2018-11-30 22:03 ` [Qemu-devel] [PATCH 01/14] qemu-nbd: Use program name in error messages Eric Blake
@ 2018-11-30 22:03 ` Eric Blake
2018-11-30 22:20 ` Richard W.M. Jones
2018-12-05 15:03 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 03/14] qemu-nbd: Fail earlier for -c/-d on non-linux Eric Blake
` (12 subsequent siblings)
14 siblings, 2 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:03 UTC (permalink / raw)
To: qemu-devel; +Cc: jsnow, nsoffer, rjones, vsementsov, qemu-block
Consolidate on using decimal (not hex) and on outputting the
option reply name (not just value) when the client reports
protocol discrepancies from the server. While it won't affect
normal operation, it makes debugging additions easier.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
nbd/client.c | 21 ++++++++++++---------
1 file changed, 12 insertions(+), 9 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index b4d457a19ad..b667a1b56fd 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -132,8 +132,9 @@ static int nbd_receive_option_reply(QIOChannel *ioc, uint32_t opt,
return -1;
}
if (reply->option != opt) {
- error_setg(errp, "Unexpected option type %x expected %x",
- reply->option, opt);
+ error_setg(errp, "Unexpected option type %u (%s) expected %u (%s)",
+ reply->option, nbd_opt_lookup(reply->option),
+ opt, nbd_opt_lookup(opt));
nbd_send_opt_abort(ioc);
return -1;
}
@@ -265,8 +266,9 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
}
return 0;
} else if (reply.type != NBD_REP_SERVER) {
- error_setg(errp, "Unexpected reply type %" PRIx32 " expected %x",
- reply.type, NBD_REP_SERVER);
+ error_setg(errp, "Unexpected reply type %u (%s) expected %u (%s)",
+ reply.type, nbd_rep_lookup(reply.type),
+ NBD_REP_SERVER, nbd_rep_lookup(NBD_REP_SERVER));
nbd_send_opt_abort(ioc);
return -1;
}
@@ -378,9 +380,9 @@ static int nbd_opt_go(QIOChannel *ioc, const char *wantname,
return 1;
}
if (reply.type != NBD_REP_INFO) {
- error_setg(errp, "unexpected reply type %" PRIu32
- " (%s), expected %u",
- reply.type, nbd_rep_lookup(reply.type), NBD_REP_INFO);
+ error_setg(errp, "unexpected reply type %u (%s), expected %u (%s)",
+ reply.type, nbd_rep_lookup(reply.type),
+ NBD_REP_INFO, nbd_rep_lookup(NBD_REP_INFO));
nbd_send_opt_abort(ioc);
return -1;
}
@@ -704,8 +706,9 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
}
if (reply.type != NBD_REP_ACK) {
- error_setg(errp, "Unexpected reply type %" PRIx32 " expected %x",
- reply.type, NBD_REP_ACK);
+ error_setg(errp, "Unexpected reply type %u (%s) expected %u (%s)",
+ reply.type, nbd_rep_lookup(reply.type),
+ NBD_REP_ACK, nbd_rep_lookup(NBD_REP_ACK));
nbd_send_opt_abort(ioc);
return -1;
}
--
2.17.2
^ permalink raw reply related [flat|nested] 65+ messages in thread
* [Qemu-devel] [PATCH 03/14] qemu-nbd: Fail earlier for -c/-d on non-linux
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
2018-11-30 22:03 ` [Qemu-devel] [PATCH 01/14] qemu-nbd: Use program name in error messages Eric Blake
2018-11-30 22:03 ` [Qemu-devel] [PATCH 02/14] nbd/client: More consistent " Eric Blake
@ 2018-11-30 22:03 ` Eric Blake
2018-11-30 22:23 ` Richard W.M. Jones
2018-12-05 15:20 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 04/14] qemu-nbd: Simplify --partition handling Eric Blake
` (11 subsequent siblings)
14 siblings, 2 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:03 UTC (permalink / raw)
To: qemu-devel; +Cc: jsnow, nsoffer, rjones, vsementsov, qemu-block
Connecting to a /dev/nbdN device is a Linux-specific action.
We were already masking -c and -d from 'qemu-nbd --help' on
non-linux. However, while -d fails with a sensible error
message, it took hunting through a couple of files to prove
that. What's more, the code for -c doesn't fail until after
it has created a pthread and tried to open a device - possibly
even printing an error message with %m on a non-Linux platform
in spite of the comment that %m is glibc-specific. Make the
failure happen sooner, then get rid of stubs that are no
longer needed because of the early exits.
While at it: tweak the blank newlines in --help output to be
consistent, whether or not built on Linux.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
nbd/client.c | 18 +-----------------
qemu-nbd.c | 22 ++++++++++++++++++++--
2 files changed, 21 insertions(+), 19 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index b667a1b56fd..0be89f9e641 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -1029,23 +1029,7 @@ int nbd_disconnect(int fd)
return 0;
}
-#else
-int nbd_init(int fd, QIOChannelSocket *ioc, NBDExportInfo *info,
- Error **errp)
-{
- error_setg(errp, "nbd_init is only supported on Linux");
- return -ENOTSUP;
-}
-
-int nbd_client(int fd)
-{
- return -ENOTSUP;
-}
-int nbd_disconnect(int fd)
-{
- return -ENOTSUP;
-}
-#endif
+#endif /* __linux__ */
int nbd_send_request(QIOChannel *ioc, NBDRequest *request)
{
diff --git a/qemu-nbd.c b/qemu-nbd.c
index e169b839ece..55e29bd9a7e 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -43,6 +43,12 @@
#include "trace/control.h"
#include "qemu-version.h"
+#ifdef __linux__
+#define HAVE_NBD_DEVICE 1
+#else
+#define HAVE_NBD_DEVICE 0
+#endif
+
#define SOCKET_PATH "/var/lock/qemu-nbd-%s"
#define QEMU_NBD_OPT_CACHE 256
#define QEMU_NBD_OPT_AIO 257
@@ -98,11 +104,11 @@ static void usage(const char *name)
" specify tracing options\n"
" --fork fork off the server process and exit the parent\n"
" once the server is running\n"
-#ifdef __linux__
+#if HAVE_NBD_DEVICE
+"\n"
"Kernel NBD client support:\n"
" -c, --connect=DEV connect FILE to the local NBD device DEV\n"
" -d, --disconnect disconnect the specified device\n"
-"\n"
#endif
"\n"
"Block device options:\n"
@@ -236,6 +242,7 @@ static void termsig_handler(int signum)
}
+#if HAVE_NBD_DEVICE
static void *show_parts(void *arg)
{
char *device = arg;
@@ -321,6 +328,7 @@ out:
kill(getpid(), SIGTERM);
return (void *) EXIT_FAILURE;
}
+#endif /* HAVE_NBD_DEVICE */
static int nbd_can_accept(void)
{
@@ -815,6 +823,7 @@ int main(int argc, char **argv)
}
if (disconnect) {
+#if HAVE_NBD_DEVICE
int nbdfd = open(argv[optind], O_RDWR);
if (nbdfd < 0) {
error_report("Cannot open %s: %s", argv[optind],
@@ -828,6 +837,10 @@ int main(int argc, char **argv)
printf("%s disconnected\n", argv[optind]);
return 0;
+#else
+ error_report("Kernel /dev/nbdN support not available");
+ exit(EXIT_FAILURE);
+#endif
}
if ((device && !verbose) || fork_process) {
@@ -1006,6 +1019,7 @@ int main(int argc, char **argv)
nbd_export_set_description(exp, export_description);
if (device) {
+#if HAVE_NBD_DEVICE
int ret;
ret = pthread_create(&client_thread, NULL, nbd_client_thread, device);
@@ -1013,6 +1027,10 @@ int main(int argc, char **argv)
error_report("Failed to create client thread: %s", strerror(ret));
exit(EXIT_FAILURE);
}
+#else
+ error_report("Kernel /dev/nbdN support not available");
+ exit(EXIT_FAILURE);
+#endif
} else {
/* Shut up GCC warnings. */
memset(&client_thread, 0, sizeof(client_thread));
--
2.17.2
^ permalink raw reply related [flat|nested] 65+ messages in thread
* [Qemu-devel] [PATCH 04/14] qemu-nbd: Simplify --partition handling
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
` (2 preceding siblings ...)
2018-11-30 22:03 ` [Qemu-devel] [PATCH 03/14] qemu-nbd: Fail earlier for -c/-d on non-linux Eric Blake
@ 2018-11-30 22:03 ` Eric Blake
2018-11-30 22:26 ` Richard W.M. Jones
` (2 more replies)
2018-11-30 22:03 ` [Qemu-devel] [PATCH 05/14] nbd/client: Drop pointless buf variable Eric Blake
` (10 subsequent siblings)
14 siblings, 3 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:03 UTC (permalink / raw)
To: qemu-devel; +Cc: jsnow, nsoffer, rjones, vsementsov, qemu-block
Our open-coding of strtol handling forgot to handle overflow
conditions. What's more, since we insiste on a user-supplied
partition to be non-zero, we can use 0 rather than -1 for our
initial value to distinguish when a partition is not being
served, for slightly more optimal code.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
qemu-nbd.c | 14 +++++---------
1 file changed, 5 insertions(+), 9 deletions(-)
diff --git a/qemu-nbd.c b/qemu-nbd.c
index 55e29bd9a7e..866e64779f1 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -546,7 +546,7 @@ int main(int argc, char **argv)
int opt_ind = 0;
char *end;
int flags = BDRV_O_RDWR;
- int partition = -1;
+ int partition = 0;
int ret = 0;
bool seen_cache = false;
bool seen_discard = false;
@@ -685,13 +685,9 @@ int main(int argc, char **argv)
flags &= ~BDRV_O_RDWR;
break;
case 'P':
- partition = strtol(optarg, &end, 0);
- if (*end) {
- error_report("Invalid partition `%s'", optarg);
- exit(EXIT_FAILURE);
- }
- if (partition < 1 || partition > 8) {
- error_report("Invalid partition %d", partition);
+ if (qemu_strtoi(optarg, NULL, 0, &partition) < 0 ||
+ partition < 1 || partition > 8) {
+ error_report("Invalid partition %s", optarg);
exit(EXIT_FAILURE);
}
break;
@@ -1004,7 +1000,7 @@ int main(int argc, char **argv)
}
fd_size -= dev_offset;
- if (partition != -1) {
+ if (partition) {
ret = find_partition(blk, partition, &dev_offset, &fd_size);
if (ret < 0) {
error_report("Could not find partition %d: %s", partition,
--
2.17.2
^ permalink raw reply related [flat|nested] 65+ messages in thread
* [Qemu-devel] [PATCH 05/14] nbd/client: Drop pointless buf variable
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
` (3 preceding siblings ...)
2018-11-30 22:03 ` [Qemu-devel] [PATCH 04/14] qemu-nbd: Simplify --partition handling Eric Blake
@ 2018-11-30 22:03 ` Eric Blake
2018-11-30 22:30 ` Richard W.M. Jones
2018-12-05 15:59 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 06/14] nbd/client: Move export name into NBDExportInfo Eric Blake
` (9 subsequent siblings)
14 siblings, 2 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:03 UTC (permalink / raw)
To: qemu-devel; +Cc: jsnow, nsoffer, rjones, vsementsov, qemu-block
There's no need to read into a temporary buffer (oversized
since commit 7d3123e1) followed by a byteswap into a uint64_t
to check for a magic number via memcmp(), when the code
immediately below demonstrates reading into the uint64_t then
byteswapping in place and checking for a magic number via
integer math. What's more, having a different error message
when the server's first reply byte is 0 is unusual - it's no
different from any other wrong magic number, and we already
detected short reads.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
nbd/nbd-internal.h | 1 +
nbd/client.c | 14 +++-----------
2 files changed, 4 insertions(+), 11 deletions(-)
diff --git a/nbd/nbd-internal.h b/nbd/nbd-internal.h
index eeff78d3c98..306a533dcd1 100644
--- a/nbd/nbd-internal.h
+++ b/nbd/nbd-internal.h
@@ -46,6 +46,7 @@
/* Size of oldstyle negotiation */
#define NBD_OLDSTYLE_NEGOTIATE_SIZE (8 + 8 + 8 + 4 + 124)
+#define NBD_INIT_MAGIC 0x4e42444d41474943LL
#define NBD_REQUEST_MAGIC 0x25609513
#define NBD_OPTS_MAGIC 0x49484156454F5054LL
#define NBD_CLIENT_MAGIC 0x0000420281861253LL
diff --git a/nbd/client.c b/nbd/client.c
index 0be89f9e641..17ee24492a4 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -731,7 +731,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
QIOChannel **outioc, NBDExportInfo *info,
Error **errp)
{
- char buf[256];
uint64_t magic;
int rc;
bool zeroes = true;
@@ -752,21 +751,14 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
goto fail;
}
- if (nbd_read(ioc, buf, 8, errp) < 0) {
+ if (nbd_read(ioc, &magic, sizeof(magic), errp) < 0) {
error_prepend(errp, "Failed to read data: ");
goto fail;
}
-
- buf[8] = '\0';
- if (strlen(buf) == 0) {
- error_setg(errp, "Server connection closed unexpectedly");
- goto fail;
- }
-
- magic = ldq_be_p(buf);
+ magic = be64_to_cpu(magic);
trace_nbd_receive_negotiate_magic(magic);
- if (memcmp(buf, "NBDMAGIC", 8) != 0) {
+ if (magic != NBD_INIT_MAGIC) {
error_setg(errp, "Invalid magic received");
goto fail;
}
--
2.17.2
^ permalink raw reply related [flat|nested] 65+ messages in thread
* [Qemu-devel] [PATCH 06/14] nbd/client: Move export name into NBDExportInfo
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
` (4 preceding siblings ...)
2018-11-30 22:03 ` [Qemu-devel] [PATCH 05/14] nbd/client: Drop pointless buf variable Eric Blake
@ 2018-11-30 22:03 ` Eric Blake
2018-11-30 22:34 ` Richard W.M. Jones
2018-12-05 17:26 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 07/14] nbd/client: Refactor nbd_negotiate_simple_meta_context() Eric Blake
` (8 subsequent siblings)
14 siblings, 2 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:03 UTC (permalink / raw)
To: qemu-devel
Cc: jsnow, nsoffer, rjones, vsementsov, qemu-block, Kevin Wolf, Max Reitz
Refactor the 'name' parameter of nbd_receive_negotiate() from
being a separate parameter into being part of the in-out 'info'.
This also spills over to a simplification of nbd_opt_go().
The main driver for this refactoring is that an upcoming patch
would like to add support to qemu-nbd to list information about
all exports available on a server, where the name(s) will be
provided by the server instead of the client. But another benefit
is that we can now allow the client to explicitly specify the
empty export name "" even when connecting to an oldstyle server
(even if qemu is no longer such a server after commit 7f7dfe2a).
Signed-off-by: Eric Blake <eblake@redhat.com>
---
include/block/nbd.h | 8 ++++----
block/nbd-client.c | 5 +++--
nbd/client.c | 39 ++++++++++++++++++---------------------
qemu-nbd.c | 6 ++++--
nbd/trace-events | 2 +-
5 files changed, 30 insertions(+), 30 deletions(-)
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 6a5bfe5d559..65feff8ba96 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -262,6 +262,7 @@ struct NBDExportInfo {
/* Set by client before nbd_receive_negotiate() */
bool request_sizes;
char *x_dirty_bitmap;
+ char *name; /* must be non-NULL */
/* In-out fields, set by client before nbd_receive_negotiate() and
* updated by server results during nbd_receive_negotiate() */
@@ -279,10 +280,9 @@ struct NBDExportInfo {
};
typedef struct NBDExportInfo NBDExportInfo;
-int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
- QCryptoTLSCreds *tlscreds, const char *hostname,
- QIOChannel **outioc, NBDExportInfo *info,
- Error **errp);
+int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, QIOChannel **outioc,
+ NBDExportInfo *info, Error **errp);
int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
Error **errp);
int nbd_send_request(QIOChannel *ioc, NBDRequest *request);
diff --git a/block/nbd-client.c b/block/nbd-client.c
index fc5b7eda8ee..417971d8b05 100644
--- a/block/nbd-client.c
+++ b/block/nbd-client.c
@@ -984,10 +984,11 @@ int nbd_client_init(BlockDriverState *bs,
client->info.structured_reply = true;
client->info.base_allocation = true;
client->info.x_dirty_bitmap = g_strdup(x_dirty_bitmap);
- ret = nbd_receive_negotiate(QIO_CHANNEL(sioc), export,
- tlscreds, hostname,
+ client->info.name = g_strdup(export ?: "");
+ ret = nbd_receive_negotiate(QIO_CHANNEL(sioc), tlscreds, hostname,
&client->ioc, &client->info, errp);
g_free(client->info.x_dirty_bitmap);
+ g_free(client->info.name);
if (ret < 0) {
logout("Failed to negotiate with the NBD server\n");
return ret;
diff --git a/nbd/client.c b/nbd/client.c
index 17ee24492a4..b5818a99d21 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -320,15 +320,14 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
}
-/* Returns -1 if NBD_OPT_GO proves the export @wantname cannot be
+/* Returns -1 if NBD_OPT_GO proves the export @info->name cannot be
* used, 0 if NBD_OPT_GO is unsupported (fall back to NBD_OPT_LIST and
* NBD_OPT_EXPORT_NAME in that case), and > 0 if the export is good to
- * go (with @info populated). */
-static int nbd_opt_go(QIOChannel *ioc, const char *wantname,
- NBDExportInfo *info, Error **errp)
+ * go (with the rest of @info populated). */
+static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
{
NBDOptionReply reply;
- uint32_t len = strlen(wantname);
+ uint32_t len = strlen(info->name);
uint16_t type;
int error;
char *buf;
@@ -338,10 +337,10 @@ static int nbd_opt_go(QIOChannel *ioc, const char *wantname,
* flags still 0 is a witness of a broken server. */
info->flags = 0;
- trace_nbd_opt_go_start(wantname);
+ trace_nbd_opt_go_start(info->name);
buf = g_malloc(4 + len + 2 + 2 * info->request_sizes + 1);
stl_be_p(buf, len);
- memcpy(buf + 4, wantname, len);
+ memcpy(buf + 4, info->name, len);
/* At most one request, everything else up to server */
stw_be_p(buf + 4 + len, info->request_sizes);
if (info->request_sizes) {
@@ -726,10 +725,9 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return 0;
}
-int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
- QCryptoTLSCreds *tlscreds, const char *hostname,
- QIOChannel **outioc, NBDExportInfo *info,
- Error **errp)
+int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, QIOChannel **outioc,
+ NBDExportInfo *info, Error **errp)
{
uint64_t magic;
int rc;
@@ -739,6 +737,8 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
trace_nbd_receive_negotiate(tlscreds, hostname ? hostname : "<null>");
+ assert(info->name);
+ trace_nbd_receive_negotiate_name(info->name);
info->structured_reply = false;
info->base_allocation = false;
rc = -EINVAL;
@@ -807,10 +807,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
goto fail;
}
}
- if (!name) {
- trace_nbd_receive_negotiate_default_name();
- name = "";
- }
if (fixedNewStyle) {
int result;
@@ -826,7 +822,8 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
if (info->structured_reply && base_allocation) {
result = nbd_negotiate_simple_meta_context(
- ioc, name, info->x_dirty_bitmap ?: "base:allocation",
+ ioc, info->name,
+ info->x_dirty_bitmap ?: "base:allocation",
&info->meta_base_allocation_id, errp);
if (result < 0) {
goto fail;
@@ -839,7 +836,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
* TLS). If it is not available, fall back to
* NBD_OPT_LIST for nicer error messages about a missing
* export, then use NBD_OPT_EXPORT_NAME. */
- result = nbd_opt_go(ioc, name, info, errp);
+ result = nbd_opt_go(ioc, info, errp);
if (result < 0) {
goto fail;
}
@@ -852,12 +849,12 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
* query gives us better error reporting if the
* export name is not available.
*/
- if (nbd_receive_query_exports(ioc, name, errp) < 0) {
+ if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
goto fail;
}
}
/* write the export name request */
- if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, name,
+ if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, info->name,
errp) < 0) {
goto fail;
}
@@ -877,8 +874,8 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
} else if (magic == NBD_CLIENT_MAGIC) {
uint32_t oldflags;
- if (name) {
- error_setg(errp, "Server does not support export names");
+ if (*info->name) {
+ error_setg(errp, "Server does not support non-empty export names");
goto fail;
}
if (tlscreds) {
diff --git a/qemu-nbd.c b/qemu-nbd.c
index 866e64779f1..c57053a0795 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -263,7 +263,7 @@ static void *show_parts(void *arg)
static void *nbd_client_thread(void *arg)
{
char *device = arg;
- NBDExportInfo info = { .request_sizes = false, };
+ NBDExportInfo info = { .request_sizes = false, .name = g_strdup("") };
QIOChannelSocket *sioc;
int fd;
int ret;
@@ -278,7 +278,7 @@ static void *nbd_client_thread(void *arg)
goto out;
}
- ret = nbd_receive_negotiate(QIO_CHANNEL(sioc), NULL,
+ ret = nbd_receive_negotiate(QIO_CHANNEL(sioc),
NULL, NULL, NULL, &info, &local_error);
if (ret < 0) {
if (local_error) {
@@ -317,6 +317,7 @@ static void *nbd_client_thread(void *arg)
}
close(fd);
object_unref(OBJECT(sioc));
+ free(info.name);
kill(getpid(), SIGTERM);
return (void *) EXIT_SUCCESS;
@@ -325,6 +326,7 @@ out_fd:
out_socket:
object_unref(OBJECT(sioc));
out:
+ free(info.name);
kill(getpid(), SIGTERM);
return (void *) EXIT_FAILURE;
}
diff --git a/nbd/trace-events b/nbd/trace-events
index 5e1d4afe8e6..289337d0dc3 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -15,7 +15,7 @@ nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of contex
nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are 0x%" PRIx32
-nbd_receive_negotiate_default_name(void) "Using default NBD export name \"\""
+nbd_receive_negotiate_name(const char *name) "Requesting NBD export name \"%s\""
nbd_receive_negotiate_size_flags(uint64_t size, uint16_t flags) "Size is %" PRIu64 ", export flags 0x%" PRIx16
nbd_init_set_socket(void) "Setting NBD socket"
nbd_init_set_block_size(unsigned long block_size) "Setting block size to %lu"
--
2.17.2
^ permalink raw reply related [flat|nested] 65+ messages in thread
* [Qemu-devel] [PATCH 07/14] nbd/client: Refactor nbd_negotiate_simple_meta_context()
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
` (5 preceding siblings ...)
2018-11-30 22:03 ` [Qemu-devel] [PATCH 06/14] nbd/client: Move export name into NBDExportInfo Eric Blake
@ 2018-11-30 22:03 ` Eric Blake
2018-12-01 10:30 ` Richard W.M. Jones
2018-12-06 13:20 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 08/14] nbd/client: Refactor nbd_receive_list() Eric Blake
` (7 subsequent siblings)
14 siblings, 2 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:03 UTC (permalink / raw)
To: qemu-devel; +Cc: jsnow, nsoffer, rjones, vsementsov, qemu-block
Change the signature to make it easier for a future patch to
reuse this function for calling NBD_OPT_LIST_META_CONTEXT with
0 or 1 queries. Also, always allocate space for the received
name, even if it doesn't match expected lengths (no point
trying to optimize the unlikely error case, and tracing the
received rather than expected name can help debug a server
implementation).
While there are now slightly different traces, and the error
message for a server replying with too many contexts is
different, there are no runtime-observable changes in behavior
for the more common case of the lone caller interacting with a
compliant server.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
nbd/client.c | 105 +++++++++++++++++++++++++++--------------------
nbd/trace-events | 2 +-
2 files changed, 61 insertions(+), 46 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index b5818a99d21..1dc8f83e19a 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -603,49 +603,57 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
}
/* nbd_negotiate_simple_meta_context:
- * Set one meta context. Simple means that reply must contain zero (not
- * negotiated) or one (negotiated) contexts. More contexts would be considered
- * as a protocol error. It's also implied that meta-data query equals queried
- * context name, so, if server replies with something different than @context,
- * it is considered an error too.
- * return 1 for successful negotiation, context_id is set
- * 0 if operation is unsupported,
+ * List or set meta context data for export @info->name, based on @opt.
+ * For list, leave @context NULL for 0 queries, or supplied for a single
+ * query; all replies are ignored, and the call merely traces server behavior.
+ * For set, @context must result in at most one matching server reply, in
+ * which case @info->meta_base_allocation_id is set to the resulting id.
+ * return 1 for successful negotiation,
+ * 0 if operation is unsupported or context unavailable,
* -1 with errp set for any other error
*/
static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
- const char *export,
+ int32_t opt,
const char *context,
- uint32_t *context_id,
+ NBDExportInfo *info,
Error **errp)
{
int ret;
NBDOptionReply reply;
uint32_t received_id = 0;
bool received = false;
- uint32_t export_len = strlen(export);
- uint32_t context_len = strlen(context);
- uint32_t data_len = sizeof(export_len) + export_len +
- sizeof(uint32_t) + /* number of queries */
- sizeof(context_len) + context_len;
- char *data = g_malloc(data_len);
- char *p = data;
+ uint32_t export_len = strlen(info->name);
+ uint32_t context_len;
+ uint32_t data_len = sizeof(export_len) + export_len + sizeof(uint32_t);
+ char *data;
+ char *p;
- trace_nbd_opt_meta_request(context, export);
+ if (!context) {
+ assert(opt == NBD_OPT_LIST_META_CONTEXT);
+ } else {
+ context_len = strlen(context);
+ data_len += sizeof(context_len) + context_len;
+ }
+ data = g_malloc(data_len);
+ p = data;
+
+ trace_nbd_opt_meta_request(nbd_opt_lookup(opt), context ?: "(all)",
+ info->name);
stl_be_p(p, export_len);
- memcpy(p += sizeof(export_len), export, export_len);
- stl_be_p(p += export_len, 1);
- stl_be_p(p += sizeof(uint32_t), context_len);
- memcpy(p += sizeof(context_len), context, context_len);
+ memcpy(p += sizeof(export_len), info->name, export_len);
+ stl_be_p(p += export_len, !!context);
+ if (context) {
+ stl_be_p(p += sizeof(uint32_t), context_len);
+ memcpy(p += sizeof(context_len), context, context_len);
+ }
- ret = nbd_send_option_request(ioc, NBD_OPT_SET_META_CONTEXT, data_len, data,
- errp);
+ ret = nbd_send_option_request(ioc, opt, data_len, data, errp);
g_free(data);
if (ret < 0) {
return ret;
}
- if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
- errp) < 0)
+ if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0)
{
return -1;
}
@@ -655,10 +663,10 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return ret;
}
- if (reply.type == NBD_REP_META_CONTEXT) {
+ while (reply.type == NBD_REP_META_CONTEXT) {
char *name;
- if (reply.length != sizeof(received_id) + context_len) {
+ if (reply.length <= sizeof(received_id)) {
error_setg(errp, "Failed to negotiate meta context '%s', server "
"answered with unexpected length %" PRIu32, context,
reply.length);
@@ -678,23 +686,31 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return -1;
}
name[reply.length] = '\0';
- if (strcmp(context, name)) {
- error_setg(errp, "Failed to negotiate meta context '%s', server "
- "answered with different context '%s'", context,
- name);
- g_free(name);
- nbd_send_opt_abort(ioc);
- return -1;
+
+ trace_nbd_opt_meta_reply(name, received_id);
+ if (opt == NBD_OPT_SET_META_CONTEXT) {
+ if (received) {
+ error_setg(errp, "Server replied with more than one context");
+ free(name);
+ nbd_send_opt_abort(ioc);
+ return -1;
+ }
+
+ if (strcmp(context, name)) {
+ error_setg(errp,
+ "Failed to negotiate meta context '%s', server "
+ "answered with different context '%s'", context,
+ name);
+ g_free(name);
+ nbd_send_opt_abort(ioc);
+ return -1;
+ }
}
g_free(name);
-
- trace_nbd_opt_meta_reply(context, received_id);
received = true;
/* receive NBD_REP_ACK */
- if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
- errp) < 0)
- {
+ if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0) {
return -1;
}
@@ -717,12 +733,11 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return -1;
}
- if (received) {
- *context_id = received_id;
- return 1;
+ if (received && opt == NBD_OPT_SET_META_CONTEXT) {
+ info->meta_base_allocation_id = received_id;
}
- return 0;
+ return received || opt == NBD_OPT_LIST_META_CONTEXT;
}
int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
@@ -822,9 +837,9 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
if (info->structured_reply && base_allocation) {
result = nbd_negotiate_simple_meta_context(
- ioc, info->name,
+ ioc, NBD_OPT_SET_META_CONTEXT,
info->x_dirty_bitmap ?: "base:allocation",
- &info->meta_base_allocation_id, errp);
+ info, errp);
if (result < 0) {
goto fail;
}
diff --git a/nbd/trace-events b/nbd/trace-events
index 289337d0dc3..5d0d202fad2 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -10,7 +10,7 @@ nbd_receive_query_exports_start(const char *wantname) "Querying export list for
nbd_receive_query_exports_success(const char *wantname) "Found desired export name '%s'"
nbd_receive_starttls_new_client(void) "Setting up TLS"
nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
-nbd_opt_meta_request(const char *context, const char *export) "Requesting to set meta context %s for export %s"
+nbd_opt_meta_request(const char *opt, const char *context, const char *export) "Requesting to %s %s for export %s"
nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
--
2.17.2
^ permalink raw reply related [flat|nested] 65+ messages in thread
* [Qemu-devel] [PATCH 08/14] nbd/client: Refactor nbd_receive_list()
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
` (6 preceding siblings ...)
2018-11-30 22:03 ` [Qemu-devel] [PATCH 07/14] nbd/client: Refactor nbd_negotiate_simple_meta_context() Eric Blake
@ 2018-11-30 22:03 ` Eric Blake
2018-12-01 10:37 ` Richard W.M. Jones
2018-12-06 14:18 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 09/14] nbd/client: Refactor return of nbd_receive_negotiate() Eric Blake
` (6 subsequent siblings)
14 siblings, 2 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:03 UTC (permalink / raw)
To: qemu-devel; +Cc: jsnow, nsoffer, rjones, vsementsov, qemu-block
Add some parameters to make this function reusable in upcoming
export listing, where we will want to capture the name and
description rather than compare against a user-supplied name.
No change in semantics to the existing caller.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
nbd/client.c | 66 +++++++++++++++++++++++++++++++++++++---------------
1 file changed, 47 insertions(+), 19 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index 1dc8f83e19a..27785c55d0a 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -232,18 +232,21 @@ static int nbd_handle_reply_err(QIOChannel *ioc, NBDOptionReply *reply,
return result;
}
-/* Process another portion of the NBD_OPT_LIST reply. Set *@match if
- * the current reply matches @want or if the server does not support
- * NBD_OPT_LIST, otherwise leave @match alone. Return 0 if iteration
- * is complete, positive if more replies are expected, or negative
- * with @errp set if an unrecoverable error occurred. */
+/* Process another portion of the NBD_OPT_LIST reply. If @want, then
+ * set *@match if the current reply matches @want or if the server
+ * does not support NBD_OPT_LIST, otherwise leave @match alone.
+ * Otherwise, @nameout and @description are malloc'd to contain
+ * NUL-terminated copies of the reply. Return 0 if iteration is
+ * complete, positive if more replies are expected, or negative with
+ * @errp set if an unrecoverable error occurred. */
static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
- Error **errp)
+ char **nameout, char **description, Error **errp)
{
NBDOptionReply reply;
uint32_t len;
uint32_t namelen;
- char name[NBD_MAX_NAME_SIZE + 1];
+ char array[NBD_MAX_NAME_SIZE + 1];
+ char *name = array;
int error;
if (nbd_receive_option_reply(ioc, NBD_OPT_LIST, &reply, errp) < 0) {
@@ -253,7 +256,12 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
if (error <= 0) {
/* The server did not support NBD_OPT_LIST, so set *match on
* the assumption that any name will be accepted. */
- *match = true;
+ if (want) {
+ *match = true;
+ } else if (!error) {
+ error_setg(errp, "Server does not support export lists");
+ error = -1;
+ }
return error;
}
len = reply.length;
@@ -290,30 +298,49 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
nbd_send_opt_abort(ioc);
return -1;
}
- if (namelen != strlen(want)) {
- if (nbd_drop(ioc, len, errp) < 0) {
- error_prepend(errp,
- "failed to skip export name with wrong length: ");
- nbd_send_opt_abort(ioc);
- return -1;
+ if (want) {
+ if (namelen != strlen(want)) {
+ if (nbd_drop(ioc, len, errp) < 0) {
+ error_prepend(errp,
+ "failed to skip export name with wrong length: ");
+ nbd_send_opt_abort(ioc);
+ return -1;
+ }
+ return 1;
}
- return 1;
+ assert(namelen < sizeof(array));
+ } else {
+ assert(nameout);
+ *nameout = name = g_new(char, namelen + 1);
}
- assert(namelen < sizeof(name));
if (nbd_read(ioc, name, namelen, errp) < 0) {
error_prepend(errp, "failed to read export name: ");
nbd_send_opt_abort(ioc);
+ if (!want) {
+ free(name);
+ }
return -1;
}
name[namelen] = '\0';
len -= namelen;
- if (nbd_drop(ioc, len, errp) < 0) {
+ if (!want) {
+ assert(description);
+ *description = g_new(char, len + 1);
+ if (nbd_read(ioc, *description, len, errp) < 0) {
+ error_prepend(errp, "failed to read export description: ");
+ nbd_send_opt_abort(ioc);
+ free(name);
+ free(*description);
+ return -1;
+ }
+ (*description)[len] = '\0';
+ } else if (nbd_drop(ioc, len, errp) < 0) {
error_prepend(errp, "failed to read export description: ");
nbd_send_opt_abort(ioc);
return -1;
}
- if (!strcmp(name, want)) {
+ if (want && !strcmp(name, want)) {
*match = true;
}
return 1;
@@ -498,7 +525,8 @@ static int nbd_receive_query_exports(QIOChannel *ioc,
}
while (1) {
- int ret = nbd_receive_list(ioc, wantname, &foundExport, errp);
+ int ret = nbd_receive_list(ioc, wantname, &foundExport,
+ NULL, NULL, errp);
if (ret < 0) {
/* Server gave unexpected reply */
--
2.17.2
^ permalink raw reply related [flat|nested] 65+ messages in thread
* [Qemu-devel] [PATCH 09/14] nbd/client: Refactor return of nbd_receive_negotiate()
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
` (7 preceding siblings ...)
2018-11-30 22:03 ` [Qemu-devel] [PATCH 08/14] nbd/client: Refactor nbd_receive_list() Eric Blake
@ 2018-11-30 22:03 ` Eric Blake
2018-11-30 22:41 ` Richard W.M. Jones
2018-12-06 14:24 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 10/14] nbd/client: Split handshake into two functions Eric Blake
` (5 subsequent siblings)
14 siblings, 2 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:03 UTC (permalink / raw)
To: qemu-devel; +Cc: jsnow, nsoffer, rjones, vsementsov, qemu-block
The function could only ever return 0 or -EINVAL; make this
clearer by dropping a useless 'fail:' label.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
nbd/client.c | 51 +++++++++++++++++++++++----------------------------
1 file changed, 23 insertions(+), 28 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index 27785c55d0a..1ed5009642e 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -773,7 +773,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
NBDExportInfo *info, Error **errp)
{
uint64_t magic;
- int rc;
bool zeroes = true;
bool structured_reply = info->structured_reply;
bool base_allocation = info->base_allocation;
@@ -784,31 +783,30 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
trace_nbd_receive_negotiate_name(info->name);
info->structured_reply = false;
info->base_allocation = false;
- rc = -EINVAL;
if (outioc) {
*outioc = NULL;
}
if (tlscreds && !outioc) {
error_setg(errp, "Output I/O channel required for TLS");
- goto fail;
+ return -EINVAL;
}
if (nbd_read(ioc, &magic, sizeof(magic), errp) < 0) {
error_prepend(errp, "Failed to read data: ");
- goto fail;
+ return -EINVAL;
}
magic = be64_to_cpu(magic);
trace_nbd_receive_negotiate_magic(magic);
if (magic != NBD_INIT_MAGIC) {
error_setg(errp, "Invalid magic received");
- goto fail;
+ return -EINVAL;
}
if (nbd_read(ioc, &magic, sizeof(magic), errp) < 0) {
error_prepend(errp, "Failed to read magic: ");
- goto fail;
+ return -EINVAL;
}
magic = be64_to_cpu(magic);
trace_nbd_receive_negotiate_magic(magic);
@@ -820,7 +818,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
if (nbd_read(ioc, &globalflags, sizeof(globalflags), errp) < 0) {
error_prepend(errp, "Failed to read server flags: ");
- goto fail;
+ return -EINVAL;
}
globalflags = be16_to_cpu(globalflags);
trace_nbd_receive_negotiate_server_flags(globalflags);
@@ -836,18 +834,18 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
clientflags = cpu_to_be32(clientflags);
if (nbd_write(ioc, &clientflags, sizeof(clientflags), errp) < 0) {
error_prepend(errp, "Failed to send clientflags field: ");
- goto fail;
+ return -EINVAL;
}
if (tlscreds) {
if (fixedNewStyle) {
*outioc = nbd_receive_starttls(ioc, tlscreds, hostname, errp);
if (!*outioc) {
- goto fail;
+ return -EINVAL;
}
ioc = *outioc;
} else {
error_setg(errp, "Server does not support STARTTLS");
- goto fail;
+ return -EINVAL;
}
}
if (fixedNewStyle) {
@@ -858,7 +856,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
NBD_OPT_STRUCTURED_REPLY,
errp);
if (result < 0) {
- goto fail;
+ return -EINVAL;
}
info->structured_reply = result == 1;
}
@@ -869,7 +867,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
info->x_dirty_bitmap ?: "base:allocation",
info, errp);
if (result < 0) {
- goto fail;
+ return -EINVAL;
}
info->base_allocation = result == 1;
}
@@ -881,7 +879,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
* export, then use NBD_OPT_EXPORT_NAME. */
result = nbd_opt_go(ioc, info, errp);
if (result < 0) {
- goto fail;
+ return -EINVAL;
}
if (result > 0) {
return 0;
@@ -893,25 +891,25 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
* export name is not available.
*/
if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
- goto fail;
+ return -EINVAL;
}
}
/* write the export name request */
if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, info->name,
errp) < 0) {
- goto fail;
+ return -EINVAL;
}
/* Read the response */
if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
error_prepend(errp, "Failed to read export length: ");
- goto fail;
+ return -EINVAL;
}
info->size = be64_to_cpu(info->size);
if (nbd_read(ioc, &info->flags, sizeof(info->flags), errp) < 0) {
error_prepend(errp, "Failed to read export flags: ");
- goto fail;
+ return -EINVAL;
}
info->flags = be16_to_cpu(info->flags);
} else if (magic == NBD_CLIENT_MAGIC) {
@@ -919,43 +917,40 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
if (*info->name) {
error_setg(errp, "Server does not support non-empty export names");
- goto fail;
+ return -EINVAL;
}
if (tlscreds) {
error_setg(errp, "Server does not support STARTTLS");
- goto fail;
+ return -EINVAL;
}
if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
error_prepend(errp, "Failed to read export length: ");
- goto fail;
+ return -EINVAL;
}
info->size = be64_to_cpu(info->size);
if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
error_prepend(errp, "Failed to read export flags: ");
- goto fail;
+ return -EINVAL;
}
oldflags = be32_to_cpu(oldflags);
if (oldflags & ~0xffff) {
error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
- goto fail;
+ return -EINVAL;
}
info->flags = oldflags;
} else {
error_setg(errp, "Bad magic received");
- goto fail;
+ return -EINVAL;
}
trace_nbd_receive_negotiate_size_flags(info->size, info->flags);
if (zeroes && nbd_drop(ioc, 124, errp) < 0) {
error_prepend(errp, "Failed to read reserved block: ");
- goto fail;
+ return -EINVAL;
}
- rc = 0;
-
-fail:
- return rc;
+ return 0;
}
#ifdef __linux__
--
2.17.2
^ permalink raw reply related [flat|nested] 65+ messages in thread
* [Qemu-devel] [PATCH 10/14] nbd/client: Split handshake into two functions
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
` (8 preceding siblings ...)
2018-11-30 22:03 ` [Qemu-devel] [PATCH 09/14] nbd/client: Refactor return of nbd_receive_negotiate() Eric Blake
@ 2018-11-30 22:03 ` Eric Blake
2018-12-01 10:41 ` Richard W.M. Jones
2018-12-06 15:16 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 11/14] nbd/client: Add nbd_receive_export_list() Eric Blake
` (4 subsequent siblings)
14 siblings, 2 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:03 UTC (permalink / raw)
To: qemu-devel; +Cc: jsnow, nsoffer, rjones, vsementsov, qemu-block
An upcoming patch will add the ability for qemu-nbd to list
the services provided by an NBD server. Share the common
code of the TLS handshake by splitting the initial exchange
into a separate function, leaving only the export handling
in the original function. Functionally, there should be no
change in behavior in this patch, although some of the code
motion may be difficult to follow due to indentation changes
(view with 'git diff -w' for a smaller changeset).
Signed-off-by: Eric Blake <eblake@redhat.com>
---
nbd/client.c | 142 ++++++++++++++++++++++++++++++-----------------
nbd/trace-events | 2 +-
2 files changed, 92 insertions(+), 52 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index 1ed5009642e..a282712724d 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -768,21 +768,22 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return received || opt == NBD_OPT_LIST_META_CONTEXT;
}
-int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
- const char *hostname, QIOChannel **outioc,
- NBDExportInfo *info, Error **errp)
+/* Start the handshake to the server. After a positive return, the server
+ * is ready to accept additional NBD_OPT requests.
+ * Returns: negative errno: failure talking to server
+ * 0: server is oldstyle, client must still parse export size
+ * 1: server is newstyle, but can only accept EXPORT_NAME
+ * 2: server is newstyle, but lacks structured replies
+ * 3: server is newstyle and set up for structured replies
+ */
+static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, QIOChannel **outioc,
+ bool structured_reply, bool *zeroes,
+ Error **errp)
{
uint64_t magic;
- bool zeroes = true;
- bool structured_reply = info->structured_reply;
- bool base_allocation = info->base_allocation;
- trace_nbd_receive_negotiate(tlscreds, hostname ? hostname : "<null>");
-
- assert(info->name);
- trace_nbd_receive_negotiate_name(info->name);
- info->structured_reply = false;
- info->base_allocation = false;
+ trace_nbd_start_negotiate(tlscreds, hostname ? hostname : "<null>");
if (outioc) {
*outioc = NULL;
@@ -827,7 +828,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
clientflags |= NBD_FLAG_C_FIXED_NEWSTYLE;
}
if (globalflags & NBD_FLAG_NO_ZEROES) {
- zeroes = false;
+ *zeroes = false;
clientflags |= NBD_FLAG_C_NO_ZEROES;
}
/* client requested flags */
@@ -849,7 +850,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
}
}
if (fixedNewStyle) {
- int result;
+ int result = 0;
if (structured_reply) {
result = nbd_request_simple_option(ioc,
@@ -858,42 +859,86 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
if (result < 0) {
return -EINVAL;
}
- info->structured_reply = result == 1;
}
+ return 2 + result;
+ } else {
+ return 1;
+ }
+ } else if (magic == NBD_CLIENT_MAGIC) {
+ if (tlscreds) {
+ error_setg(errp, "Server does not support STARTTLS");
+ return -EINVAL;
+ }
+ return 0;
+ } else {
+ error_setg(errp, "Bad magic received");
+ return -EINVAL;
+ }
+}
- if (info->structured_reply && base_allocation) {
- result = nbd_negotiate_simple_meta_context(
+/* Connect to server, complete negotiation, and move into transmission phase.
+ * Returns: negative errno: failure talking to server
+ * 0: server is connected
+ */
+int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, QIOChannel **outioc,
+ NBDExportInfo *info, Error **errp)
+{
+ int result;
+ bool zeroes = true;
+ bool base_allocation = info->base_allocation;
+ uint32_t oldflags;
+
+ assert(info->name);
+ trace_nbd_receive_negotiate_name(info->name);
+
+ result = nbd_start_negotiate(ioc, tlscreds, hostname, outioc,
+ info->structured_reply, &zeroes, errp);
+
+ info->structured_reply = false;
+ info->base_allocation = false;
+ if (tlscreds && *outioc) {
+ ioc = *outioc;
+ }
+
+ switch (result) {
+ case 3: /* newstyle, with structured replies */
+ info->structured_reply = true;
+ if (base_allocation) {
+ result = nbd_negotiate_simple_meta_context(
ioc, NBD_OPT_SET_META_CONTEXT,
info->x_dirty_bitmap ?: "base:allocation",
info, errp);
- if (result < 0) {
- return -EINVAL;
- }
- info->base_allocation = result == 1;
- }
-
- /* Try NBD_OPT_GO first - if it works, we are done (it
- * also gives us a good message if the server requires
- * TLS). If it is not available, fall back to
- * NBD_OPT_LIST for nicer error messages about a missing
- * export, then use NBD_OPT_EXPORT_NAME. */
- result = nbd_opt_go(ioc, info, errp);
if (result < 0) {
return -EINVAL;
}
- if (result > 0) {
- return 0;
- }
- /* Check our desired export is present in the
- * server export list. Since NBD_OPT_EXPORT_NAME
- * cannot return an error message, running this
- * query gives us better error reporting if the
- * export name is not available.
- */
- if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
- return -EINVAL;
- }
+ info->base_allocation = result == 1;
}
+ /* fall through */
+ case 2: /* newstyle, try OPT_GO */
+ /* Try NBD_OPT_GO first - if it works, we are done (it
+ * also gives us a good message if the server requires
+ * TLS). If it is not available, fall back to
+ * NBD_OPT_LIST for nicer error messages about a missing
+ * export, then use NBD_OPT_EXPORT_NAME. */
+ result = nbd_opt_go(ioc, info, errp);
+ if (result < 0) {
+ return -EINVAL;
+ }
+ if (result > 0) {
+ return 0;
+ }
+ /* Check our desired export is present in the
+ * server export list. Since NBD_OPT_EXPORT_NAME
+ * cannot return an error message, running this
+ * query gives us better error reporting if the
+ * export name is not available.
+ */
+ if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
+ return -EINVAL;
+ }
+ /* fall through */
+ case 1: /* newstyle, but limited to EXPORT_NAME */
/* write the export name request */
if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, info->name,
errp) < 0) {
@@ -912,17 +957,12 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
return -EINVAL;
}
info->flags = be16_to_cpu(info->flags);
- } else if (magic == NBD_CLIENT_MAGIC) {
- uint32_t oldflags;
-
+ break;
+ case 0: /* oldstyle, parse length and flags */
if (*info->name) {
error_setg(errp, "Server does not support non-empty export names");
return -EINVAL;
}
- if (tlscreds) {
- error_setg(errp, "Server does not support STARTTLS");
- return -EINVAL;
- }
if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
error_prepend(errp, "Failed to read export length: ");
@@ -940,9 +980,9 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
return -EINVAL;
}
info->flags = oldflags;
- } else {
- error_setg(errp, "Bad magic received");
- return -EINVAL;
+ break;
+ default:
+ return result;
}
trace_nbd_receive_negotiate_size_flags(info->size, info->flags);
diff --git a/nbd/trace-events b/nbd/trace-events
index 5d0d202fad2..570b04997ff 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -12,7 +12,7 @@ nbd_receive_starttls_new_client(void) "Setting up TLS"
nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
nbd_opt_meta_request(const char *opt, const char *context, const char *export) "Requesting to %s %s for export %s"
nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
-nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
+nbd_start_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are 0x%" PRIx32
nbd_receive_negotiate_name(const char *name) "Requesting NBD export name \"%s\""
--
2.17.2
^ permalink raw reply related [flat|nested] 65+ messages in thread
* [Qemu-devel] [PATCH 11/14] nbd/client: Add nbd_receive_export_list()
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
` (9 preceding siblings ...)
2018-11-30 22:03 ` [Qemu-devel] [PATCH 10/14] nbd/client: Split handshake into two functions Eric Blake
@ 2018-11-30 22:03 ` Eric Blake
2018-12-01 10:45 ` Richard W.M. Jones
` (2 more replies)
2018-11-30 22:03 ` [Qemu-devel] [PATCH 12/14] nbd/client: Work around 3.0 bug for listing meta contexts Eric Blake
` (3 subsequent siblings)
14 siblings, 3 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:03 UTC (permalink / raw)
To: qemu-devel
Cc: jsnow, nsoffer, rjones, vsementsov, qemu-block, Kevin Wolf, Max Reitz
We want to be able to detect whether a given qemu NBD server is
exposing the right export(s) and dirty bitmaps, at least for
regression testing. We could use 'nbd-client -l' from the upstream
NBD project to list exports, but it's annoying to rely on
out-of-tree binaries; furthermore, nbd-client doesn't necessarily
know about all of the qemu NBD extensions. Thus, we plan on adding
a new mode to qemu-nbd that merely sniffs all possible information
from the server during handshake phase, then disconnects and dumps
the information.
This patch adds the low-level client code for grabbing the list
of exports. It benefits from the recent refactoring patches, as
well as a minor tweak of changing nbd_opt_go() to nbd_opt_info_go(),
in order to share as much code as possible when it comes to doing
validation of server replies. The resulting information is stored
in an array of NBDExportInfo which has been expanded to hold more
members, along with a convenience function for freeing the list.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
include/block/nbd.h | 15 +++-
nbd/client.c | 166 ++++++++++++++++++++++++++++++++++++++++++--
nbd/trace-events | 2 +-
3 files changed, 174 insertions(+), 9 deletions(-)
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 65feff8ba96..74d006b8d62 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -262,6 +262,9 @@ struct NBDExportInfo {
/* Set by client before nbd_receive_negotiate() */
bool request_sizes;
char *x_dirty_bitmap;
+
+ /* Set by client before nbd_receive_negotiate(), or by server results
+ * during nbd_receive_export_list() */
char *name; /* must be non-NULL */
/* In-out fields, set by client before nbd_receive_negotiate() and
@@ -269,7 +272,8 @@ struct NBDExportInfo {
bool structured_reply;
bool base_allocation; /* base:allocation context for NBD_CMD_BLOCK_STATUS */
- /* Set by server results during nbd_receive_negotiate() */
+ /* Set by server results during nbd_receive_negotiate() and
+ * nbd_receive_export_list() */
uint64_t size;
uint16_t flags;
uint32_t min_block;
@@ -277,12 +281,21 @@ struct NBDExportInfo {
uint32_t max_block;
uint32_t meta_base_allocation_id;
+
+ /* Set by server results during nbd_receive_export_list() */
+ char *description;
+ int n_contexts;
+ char **contexts;
};
typedef struct NBDExportInfo NBDExportInfo;
int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
const char *hostname, QIOChannel **outioc,
NBDExportInfo *info, Error **errp);
+void nbd_free_export_list(NBDExportInfo *info, int count);
+int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, NBDExportInfo **info,
+ Error **errp);
int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
Error **errp);
int nbd_send_request(QIOChannel *ioc, NBDRequest *request);
diff --git a/nbd/client.c b/nbd/client.c
index a282712724d..6292de560ee 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -351,7 +351,8 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
* used, 0 if NBD_OPT_GO is unsupported (fall back to NBD_OPT_LIST and
* NBD_OPT_EXPORT_NAME in that case), and > 0 if the export is good to
* go (with the rest of @info populated). */
-static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
+static int nbd_opt_info_go(QIOChannel *ioc, uint32_t opt,
+ NBDExportInfo *info, Error **errp)
{
NBDOptionReply reply;
uint32_t len = strlen(info->name);
@@ -364,7 +365,8 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
* flags still 0 is a witness of a broken server. */
info->flags = 0;
- trace_nbd_opt_go_start(info->name);
+ assert(opt == NBD_OPT_GO || opt == NBD_OPT_INFO);
+ trace_nbd_opt_go_start(nbd_opt_lookup(opt), info->name);
buf = g_malloc(4 + len + 2 + 2 * info->request_sizes + 1);
stl_be_p(buf, len);
memcpy(buf + 4, info->name, len);
@@ -373,7 +375,7 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
if (info->request_sizes) {
stw_be_p(buf + 4 + len + 2, NBD_INFO_BLOCK_SIZE);
}
- error = nbd_send_option_request(ioc, NBD_OPT_GO,
+ error = nbd_send_option_request(ioc, opt,
4 + len + 2 + 2 * info->request_sizes,
buf, errp);
g_free(buf);
@@ -382,7 +384,7 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
}
while (1) {
- if (nbd_receive_option_reply(ioc, NBD_OPT_GO, &reply, errp) < 0) {
+ if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0) {
return -1;
}
error = nbd_handle_reply_err(ioc, &reply, errp);
@@ -733,8 +735,12 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
nbd_send_opt_abort(ioc);
return -1;
}
+ g_free(name);
+ } else {
+ info->contexts = g_renew(char *, info->contexts,
+ ++info->n_contexts);
+ info->contexts[info->n_contexts - 1] = name;
}
- g_free(name);
received = true;
/* receive NBD_REP_ACK */
@@ -828,7 +834,9 @@ static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
clientflags |= NBD_FLAG_C_FIXED_NEWSTYLE;
}
if (globalflags & NBD_FLAG_NO_ZEROES) {
- *zeroes = false;
+ if (zeroes) {
+ *zeroes = false;
+ }
clientflags |= NBD_FLAG_C_NO_ZEROES;
}
/* client requested flags */
@@ -921,7 +929,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
* TLS). If it is not available, fall back to
* NBD_OPT_LIST for nicer error messages about a missing
* export, then use NBD_OPT_EXPORT_NAME. */
- result = nbd_opt_go(ioc, info, errp);
+ result = nbd_opt_info_go(ioc, NBD_OPT_GO, info, errp);
if (result < 0) {
return -EINVAL;
}
@@ -993,6 +1001,150 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
return 0;
}
+/* Clean up result of nbd_receive_export_list */
+void nbd_free_export_list(NBDExportInfo *info, int count)
+{
+ int i, j;
+
+ for (i = 0; info && i < count; i++) {
+ free(info[i].name);
+ free(info[i].description);
+ for (j = 0; j < info[i].n_contexts; j++) {
+ free(info[i].contexts[j]);
+ }
+ free(info[i].contexts);
+ }
+ free(info);
+}
+
+/* Query details about a server's exports, then disconnect without
+ * going into transmission phase. Return a count of the exports listed
+ * in @info by the server, or -1 on error. Caller must free @info. */
+int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
+ const char *hostname, NBDExportInfo **info,
+ Error **errp)
+{
+ int result;
+ int count = 0;
+ int i;
+ int rc;
+ int ret = -1;
+ NBDExportInfo *array = NULL;
+ uint32_t oldflags;
+ QIOChannel *sioc = NULL;
+ bool try_context = true;
+
+ *info = NULL;
+ result = nbd_start_negotiate(ioc, tlscreds, hostname, &sioc, true, NULL,
+ errp);
+ if (tlscreds && sioc) {
+ ioc = sioc;
+ }
+
+ switch (result) {
+ case 2:
+ /* meta contexts are only useful with structured reply */
+ try_context = false;
+ /* fall through */
+ case 3:
+ /* newstyle - use NBD_OPT_LIST to populate array, then try
+ * NBD_OPT_INFO on each array member. If structured replies
+ * are enabled, also try NBD_OPT_LIST_META_CONTEXT. */
+ if (nbd_send_option_request(ioc, NBD_OPT_LIST, 0, NULL, errp) < 0) {
+ goto out;
+ }
+ while (1) {
+ char *name;
+ char *desc;
+
+ rc = nbd_receive_list(ioc, NULL, NULL, &name, &desc, errp);
+ if (rc < 0) {
+ goto out;
+ } else if (rc == 0) {
+ break;
+ }
+ array = g_renew(NBDExportInfo, array, ++count);
+ memset(&array[count - 1], 0, sizeof(*array));
+ array[count - 1].name = name;
+ array[count - 1].description = desc;
+ array[count - 1].structured_reply = result == 3;
+ }
+
+ for (i = 0; i < count; i++) {
+ array[i].request_sizes = true;
+ rc = nbd_opt_info_go(ioc, NBD_OPT_INFO, &array[i], errp);
+ if (rc < 0) {
+ goto out;
+ } else if (rc == 0) {
+ /* Pointless to try rest of loop. If OPT_INFO doesn't work,
+ * it's unlikely that meta contexts work either */
+ break;
+ }
+
+ if (try_context) {
+ rc = nbd_negotiate_simple_meta_context(
+ ioc, NBD_OPT_LIST_META_CONTEXT, NULL, &array[i], errp);
+ if (rc < 0) {
+ goto out;
+ } else if (rc == 0) {
+ try_context = false;
+ }
+ }
+ }
+
+ /* Send NBD_OPT_ABORT as a courtesy before hanging up */
+ nbd_send_opt_abort(ioc);
+ break;
+ case 1: /* newstyle, but limited to EXPORT_NAME */
+ error_setg(errp, "Server does not support export lists");
+ /* We can't even send NBD_OPT_ABORT, so merely hang up */
+ goto out;
+ case 0: /* oldstyle, parse length and flags */
+ array = g_new0(NBDExportInfo, 1);
+ array->name = g_strdup("");
+ count = 1;
+
+ if (nbd_read(ioc, &array->size, sizeof(array->size), errp) < 0) {
+ error_prepend(errp, "Failed to read export length: ");
+ goto out;
+ }
+ array->size = be64_to_cpu(array->size);
+
+ if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
+ error_prepend(errp, "Failed to read export flags: ");
+ goto out;
+ }
+ oldflags = be32_to_cpu(oldflags);
+ if (oldflags & ~0xffff) {
+ error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
+ goto out;
+ }
+ array->flags = oldflags;
+
+ /* Send NBD_CMD_DISC as a courtesy to the server, but ignore all
+ * errors now that we have the information we wanted. */
+ if (nbd_drop(ioc, 124, NULL) == 0) {
+ NBDRequest request = { .type = NBD_CMD_DISC };
+
+ nbd_send_request(ioc, &request);
+ }
+ break;
+ default:
+ goto out;
+ }
+
+ *info = array;
+ array = NULL;
+ ret = count;
+
+ out:
+ qio_channel_shutdown(ioc, QIO_CHANNEL_SHUTDOWN_BOTH, NULL);
+ qio_channel_close(ioc, NULL);
+ object_unref(OBJECT(sioc));
+ nbd_free_export_list(array, count);
+ return ret;
+}
+
#ifdef __linux__
int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
Error **errp)
diff --git a/nbd/trace-events b/nbd/trace-events
index 570b04997ff..2cf83ebed15 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -2,7 +2,7 @@
nbd_send_option_request(uint32_t opt, const char *name, uint32_t len) "Sending option request %" PRIu32" (%s), len %" PRIu32
nbd_receive_option_reply(uint32_t option, const char *optname, uint32_t type, const char *typename, uint32_t length) "Received option reply %" PRIu32" (%s), type %" PRIu32" (%s), len %" PRIu32
nbd_reply_err_unsup(uint32_t option, const char *name) "server doesn't understand request %" PRIu32 " (%s), attempting fallback"
-nbd_opt_go_start(const char *name) "Attempting NBD_OPT_GO for export '%s'"
+nbd_opt_go_start(const char *opt, const char *name) "Attempting %s for export '%s'"
nbd_opt_go_success(void) "Export is good to go"
nbd_opt_go_info_unknown(int info, const char *name) "Ignoring unknown info %d (%s)"
nbd_opt_go_info_block_size(uint32_t minimum, uint32_t preferred, uint32_t maximum) "Block sizes are 0x%" PRIx32 ", 0x%" PRIx32 ", 0x%" PRIx32
--
2.17.2
^ permalink raw reply related [flat|nested] 65+ messages in thread
* [Qemu-devel] [PATCH 12/14] nbd/client: Work around 3.0 bug for listing meta contexts
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
` (10 preceding siblings ...)
2018-11-30 22:03 ` [Qemu-devel] [PATCH 11/14] nbd/client: Add nbd_receive_export_list() Eric Blake
@ 2018-11-30 22:03 ` Eric Blake
2018-12-07 11:21 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 13/14] qemu-nbd: Add --list option Eric Blake
` (2 subsequent siblings)
14 siblings, 1 reply; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:03 UTC (permalink / raw)
To: qemu-devel; +Cc: jsnow, nsoffer, rjones, vsementsov, qemu-block
Commit 3d068aff forgot to advertise available qemu: contexts
when the client requests a list with 0 queries. Furthermore,
3.0 shipped with a qemu-img hack of x-dirty-bitmap (commit
216ee365) that _silently_ acts as though the entire image is
clean if a requested bitmap is not present. Both bugs have
been recently fixed to give full output from the start, and
to refuse a connection if a requested x-dirty-bitmap was not
found.
Still, it is likely that there will be users that have to
work with a mix of old and new qemu versions, depending on
which features get backported where, at which point being
able to rely on 'qemu-img --list' output to know for sure
whether a given NBD export has the desired dirty bitmap is
much nicer than blindly connecting and risking that the
entire image may appear clean. We can make our --list code
smart enough to work around buggy servers by tracking
whether we've seen any qemu: replies in the original 0-query
list; if not, recurse to a single query on "qemu:" (which
may still have no replies, but then we know for sure we
didn't trip up on the server bug).
Signed-off-by: Eric Blake <eblake@redhat.com>
---
Done as a separate patch to make it easier to revert when we no
longer care about 3.0 servers
---
nbd/client.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/nbd/client.c b/nbd/client.c
index 6292de560ee..928ecabd420 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -21,6 +21,7 @@
#include "qapi/error.h"
#include "trace.h"
#include "nbd-internal.h"
+#include "qemu/cutils.h"
/* Definitions for opaque data types */
@@ -736,12 +737,13 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
return -1;
}
g_free(name);
+ received = true;
} else {
info->contexts = g_renew(char *, info->contexts,
++info->n_contexts);
info->contexts[info->n_contexts - 1] = name;
+ received |= strstart(name, "qemu:", NULL);
}
- received = true;
/* receive NBD_REP_ACK */
if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0) {
@@ -771,6 +773,13 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
info->meta_base_allocation_id = received_id;
}
+ /* Recurse to work around qemu 3.0 bug - the server forgot to send
+ * "qemu:" replies to 0 queries. */
+ if (!context && !received) {
+ return nbd_negotiate_simple_meta_context(ioc, opt, "qemu:", info,
+ errp);
+ }
+
return received || opt == NBD_OPT_LIST_META_CONTEXT;
}
--
2.17.2
^ permalink raw reply related [flat|nested] 65+ messages in thread
* [Qemu-devel] [PATCH 13/14] qemu-nbd: Add --list option
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
` (11 preceding siblings ...)
2018-11-30 22:03 ` [Qemu-devel] [PATCH 12/14] nbd/client: Work around 3.0 bug for listing meta contexts Eric Blake
@ 2018-11-30 22:03 ` Eric Blake
2018-12-01 10:58 ` Richard W.M. Jones
2018-12-07 12:48 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 14/14] iotests: Enhance 223, 233 to cover 'qemu-nbd --list' Eric Blake
2018-12-01 7:42 ` [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Richard W.M. Jones
14 siblings, 2 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:03 UTC (permalink / raw)
To: qemu-devel; +Cc: jsnow, nsoffer, rjones, vsementsov, qemu-block
We want to be able to detect whether a given qemu NBD server is
exposing the right export(s) and dirty bitmaps, at least for
regression testing. We could use 'nbd-client -l' from the upstream
NBD project to list exports, but it's annoying to rely on
out-of-tree binaries; furthermore, nbd-client doesn't necessarily
know about all of the qemu NBD extensions. Thus, we plan on adding
a new mode to qemu-nbd that merely sniffs all possible information
from the server during handshake phase, then disconnects and dumps
the information.
This patch actually implements --list/-L, while reusing other
options such as --tls-creds for now designating how to connect
as the client (rather than their non-list usage of how to operate
as the server).
I debated about adding this functionality to something akin to
'qemu-img info' - but that tool does not readily lend itself
to connecting to an arbitrary NBD server without also tying to
a specific export (I may, however, still add ImageInfoSpecificNBD
for reporting the bitmaps available when connecting to a single
export). And, while it may feel a bit odd that normally
qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not
really making the qemu-nbd binary that much larger, because
'qemu-nbd -c' has to operate as both server and client
simultaneously across two threads when feeding the kernel module
for /dev/nbdN access.
Sample output:
$ qemu-nbd -L
exports available: 1
export: ''
size: 65536
flags: 0x4ed ( flush fua trim zeroes df cache )
min block: 512
opt block: 4096
max block: 33554432
available meta contexts: 1
base:allocation
Signed-off-by: Eric Blake <eblake@redhat.com>
---
qemu-nbd.c | 153 ++++++++++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 141 insertions(+), 12 deletions(-)
diff --git a/qemu-nbd.c b/qemu-nbd.c
index c57053a0795..e19a841b869 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -76,6 +76,7 @@ static void usage(const char *name)
{
(printf) (
"Usage: %s [OPTIONS] FILE\n"
+" or: %s -L [OPTIONS]\n"
"QEMU Disk Network Block Device Server\n"
"\n"
" -h, --help display this help and exit\n"
@@ -97,6 +98,7 @@ static void usage(const char *name)
" -P, --partition=NUM only expose partition NUM\n"
"\n"
"General purpose options:\n"
+" -L, --list list NBD exports visible to client\n"
" --object type,id=ID,... define an object such as 'secret' for providing\n"
" passwords and/or encryption keys\n"
" --tls-creds=ID use id of an earlier --object to provide TLS\n"
@@ -130,7 +132,7 @@ static void usage(const char *name)
" --image-opts treat FILE as a full set of image options\n"
"\n"
QEMU_HELP_BOTTOM "\n"
- , name, NBD_DEFAULT_PORT, "DEVICE");
+ , name, name, NBD_DEFAULT_PORT, "DEVICE");
}
static void version(const char *name)
@@ -242,6 +244,92 @@ static void termsig_handler(int signum)
}
+static int qemu_nbd_client_list(SocketAddress *saddr, QCryptoTLSCreds *tls,
+ const char *hostname)
+{
+ int ret = EXIT_FAILURE;
+ int rc;
+ Error *err = NULL;
+ QIOChannelSocket *sioc;
+ NBDExportInfo *list;
+ int i, j;
+
+ sioc = qio_channel_socket_new();
+ if (qio_channel_socket_connect_sync(sioc, saddr, &err) < 0) {
+ error_report_err(err);
+ goto out;
+ }
+ rc = nbd_receive_export_list(QIO_CHANNEL(sioc), tls, hostname, &list,
+ &err);
+ if (rc < 0) {
+ if (err) {
+ error_report_err(err);
+ }
+ goto out_socket;
+ }
+ printf("exports available: %d\n", rc);
+ for (i = 0; i < rc; i++) {
+ printf(" export: '%s'\n", list[i].name);
+ if (list[i].description && *list[i].description) {
+ printf(" description: %s\n", list[i].description);
+ }
+ if (list[i].flags & NBD_FLAG_HAS_FLAGS) {
+ printf(" size: %" PRIu64 "\n", list[i].size);
+ printf(" flags: 0x%x (", list[i].flags);
+ if (list[i].flags & NBD_FLAG_READ_ONLY) {
+ printf(" readonly");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_FLUSH) {
+ printf(" flush");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_FUA) {
+ printf(" fua");
+ }
+ if (list[i].flags & NBD_FLAG_ROTATIONAL) {
+ printf(" rotational");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_TRIM) {
+ printf(" trim");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_WRITE_ZEROES) {
+ printf(" zeroes");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_DF) {
+ printf(" df");
+ }
+ if (list[i].flags & NBD_FLAG_CAN_MULTI_CONN) {
+ printf(" multi");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_RESIZE) {
+ printf(" resize");
+ }
+ if (list[i].flags & NBD_FLAG_SEND_CACHE) {
+ printf(" cache");
+ }
+ printf(" )\n");
+ }
+ if (list[i].min_block) {
+ printf(" min block: %u\n", list[i].min_block);
+ printf(" opt block: %u\n", list[i].opt_block);
+ printf(" max block: %u\n", list[i].max_block);
+ }
+ if (list[i].n_contexts) {
+ printf(" available meta contexts: %d\n", list[i].n_contexts);
+ for (j = 0; j < list[i].n_contexts; j++) {
+ printf(" %s\n", list[i].contexts[j]);
+ }
+ }
+ }
+ nbd_free_export_list(list, rc);
+
+ ret = EXIT_SUCCESS;
+ out_socket:
+ object_unref(OBJECT(sioc));
+ out:
+ return ret;
+}
+
+
#if HAVE_NBD_DEVICE
static void *show_parts(void *arg)
{
@@ -424,7 +512,8 @@ static QemuOptsList qemu_object_opts = {
-static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, Error **errp)
+static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, bool list,
+ Error **errp)
{
Object *obj;
QCryptoTLSCreds *creds;
@@ -444,10 +533,18 @@ static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, Error **errp)
return NULL;
}
- if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_SERVER) {
- error_setg(errp,
- "Expecting TLS credentials with a server endpoint");
- return NULL;
+ if (list) {
+ if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT) {
+ error_setg(errp,
+ "Expecting TLS credentials with a client endpoint");
+ return NULL;
+ }
+ } else {
+ if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_SERVER) {
+ error_setg(errp,
+ "Expecting TLS credentials with a server endpoint");
+ return NULL;
+ }
}
object_ref(obj);
return creds;
@@ -470,7 +567,8 @@ static void setup_address_and_port(const char **address, const char **port)
static const char *socket_activation_validate_opts(const char *device,
const char *sockpath,
const char *address,
- const char *port)
+ const char *port,
+ bool list)
{
if (device != NULL) {
return "NBD device can't be set when using socket activation";
@@ -488,6 +586,10 @@ static const char *socket_activation_validate_opts(const char *device,
return "TCP port number can't be set when using socket activation";
}
+ if (list) {
+ return "List mode is incompatible with socket activation";
+ }
+
return NULL;
}
@@ -511,7 +613,7 @@ int main(int argc, char **argv)
off_t fd_size;
QemuOpts *sn_opts = NULL;
const char *sn_id_or_name = NULL;
- const char *sopt = "hVb:o:p:rsnP:c:dvk:e:f:tl:x:T:D:";
+ const char *sopt = "hVb:o:p:rsnP:c:dvk:e:f:tl:x:T:D:L";
struct option lopt[] = {
{ "help", no_argument, NULL, 'h' },
{ "version", no_argument, NULL, 'V' },
@@ -523,6 +625,7 @@ int main(int argc, char **argv)
{ "partition", required_argument, NULL, 'P' },
{ "connect", required_argument, NULL, 'c' },
{ "disconnect", no_argument, NULL, 'd' },
+ { "list", no_argument, NULL, 'L' },
{ "snapshot", no_argument, NULL, 's' },
{ "load-snapshot", required_argument, NULL, 'l' },
{ "nocache", no_argument, NULL, 'n' },
@@ -558,13 +661,14 @@ int main(int argc, char **argv)
Error *local_err = NULL;
BlockdevDetectZeroesOptions detect_zeroes = BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF;
QDict *options = NULL;
- const char *export_name = ""; /* Default export name */
+ const char *export_name = NULL;
const char *export_description = NULL;
const char *tlscredsid = NULL;
bool imageOpts = false;
bool writethrough = true;
char *trace_file = NULL;
bool fork_process = false;
+ bool list = false;
int old_stderr = -1;
unsigned socket_activation;
@@ -764,13 +868,32 @@ int main(int argc, char **argv)
case QEMU_NBD_OPT_FORK:
fork_process = true;
break;
+ case 'L':
+ list = true;
+ break;
}
}
- if ((argc - optind) != 1) {
+ if (list) {
+ if (argc != optind) {
+ error_report("List mode is incompatible with a file name");
+ exit(EXIT_FAILURE);
+ }
+ if (export_name || export_description || dev_offset || partition ||
+ device || disconnect || fmt || sn_id_or_name) {
+ error_report("List mode is incompatible with per-device settings");
+ exit(EXIT_FAILURE);
+ }
+ if (fork_process) {
+ error_report("List mode is incompatible with forking");
+ exit(EXIT_FAILURE);
+ }
+ } else if ((argc - optind) != 1) {
error_report("Invalid number of arguments");
error_printf("Try `%s --help' for more information.\n", argv[0]);
exit(EXIT_FAILURE);
+ } else if (!export_name) {
+ export_name = "";
}
qemu_opts_foreach(&qemu_object_opts,
@@ -789,7 +912,8 @@ int main(int argc, char **argv)
} else {
/* Using socket activation - check user didn't use -p etc. */
const char *err_msg = socket_activation_validate_opts(device, sockpath,
- bindto, port);
+ bindto, port,
+ list);
if (err_msg != NULL) {
error_report("%s", err_msg);
exit(EXIT_FAILURE);
@@ -812,7 +936,7 @@ int main(int argc, char **argv)
error_report("TLS is not supported with a host device");
exit(EXIT_FAILURE);
}
- tlscreds = nbd_get_tls_creds(tlscredsid, &local_err);
+ tlscreds = nbd_get_tls_creds(tlscredsid, list, &local_err);
if (local_err) {
error_report("Failed to get TLS creds %s",
error_get_pretty(local_err));
@@ -820,6 +944,11 @@ int main(int argc, char **argv)
}
}
+ if (list) {
+ saddr = nbd_build_socket_address(sockpath, bindto, port);
+ return qemu_nbd_client_list(saddr, tlscreds, bindto);
+ }
+
if (disconnect) {
#if HAVE_NBD_DEVICE
int nbdfd = open(argv[optind], O_RDWR);
--
2.17.2
^ permalink raw reply related [flat|nested] 65+ messages in thread
* [Qemu-devel] [PATCH 14/14] iotests: Enhance 223, 233 to cover 'qemu-nbd --list'
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
` (12 preceding siblings ...)
2018-11-30 22:03 ` [Qemu-devel] [PATCH 13/14] qemu-nbd: Add --list option Eric Blake
@ 2018-11-30 22:03 ` Eric Blake
2018-12-01 11:04 ` Richard W.M. Jones
2018-12-07 13:08 ` Vladimir Sementsov-Ogievskiy
2018-12-01 7:42 ` [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Richard W.M. Jones
14 siblings, 2 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:03 UTC (permalink / raw)
To: qemu-devel
Cc: jsnow, nsoffer, rjones, vsementsov, qemu-block, Kevin Wolf, Max Reitz
Any good new feature deserves some regression testing :)
Coverage includes:
- 223: what happens when there are 0 or more than 1 export,
proof that we can see multiple contexts including qemu:dirty-bitmap
- 233: proof that we can list over TLS, and that mix-and-match of
plain/TLS listings sanely
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/qemu-iotests/223 | 2 ++
tests/qemu-iotests/223.out | 20 ++++++++++++++++++++
tests/qemu-iotests/233 | 10 ++++++++++
tests/qemu-iotests/233.out | 15 +++++++++++++++
4 files changed, 47 insertions(+)
diff --git a/tests/qemu-iotests/223 b/tests/qemu-iotests/223
index 397b865d347..e64747a9a61 100755
--- a/tests/qemu-iotests/223
+++ b/tests/qemu-iotests/223
@@ -119,6 +119,7 @@ _send_qemu_cmd $QEMU_HANDLE '{"execute":"x-block-dirty-bitmap-disable",
_send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-start",
"arguments":{"addr":{"type":"unix",
"data":{"path":"'"$TEST_DIR/nbd"'"}}}}' "return"
+$QEMU_NBD_PROG -L -k "$TEST_DIR/nbd"
_send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add",
"arguments":{"device":"n"}}' "return"
_send_qemu_cmd $QEMU_HANDLE '{"execute":"x-nbd-server-add-bitmap",
@@ -127,6 +128,7 @@ _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add",
"arguments":{"device":"n", "name":"n2"}}' "return"
_send_qemu_cmd $QEMU_HANDLE '{"execute":"x-nbd-server-add-bitmap",
"arguments":{"name":"n2", "bitmap":"b2"}}' "return"
+$QEMU_NBD_PROG -L -k "$TEST_DIR/nbd"
echo
echo "=== Contrast normal status to large granularity dirty-bitmap ==="
diff --git a/tests/qemu-iotests/223.out b/tests/qemu-iotests/223.out
index de417477de0..3342bff3447 100644
--- a/tests/qemu-iotests/223.out
+++ b/tests/qemu-iotests/223.out
@@ -29,10 +29,30 @@ wrote 2097152/2097152 bytes at offset 2097152
{"return": {}}
{"return": {}}
{"return": {}}
+exports available: 0
{"return": {}}
{"return": {}}
{"return": {}}
{"return": {}}
+exports available: 2
+ export: 'n'
+ size: 4194304
+ flags: 0x4ef ( readonly flush fua trim zeroes df cache )
+ min block: 512
+ opt block: 4096
+ max block: 33554432
+ available meta contexts: 2
+ base:allocation
+ qemu:dirty-bitmap:b
+ export: 'n2'
+ size: 4194304
+ flags: 0x4ef ( readonly flush fua trim zeroes df cache )
+ min block: 512
+ opt block: 4096
+ max block: 33554432
+ available meta contexts: 2
+ base:allocation
+ qemu:dirty-bitmap:b2
=== Contrast normal status to large granularity dirty-bitmap ===
diff --git a/tests/qemu-iotests/233 b/tests/qemu-iotests/233
index 1814efe3333..5d694d9d242 100755
--- a/tests/qemu-iotests/233
+++ b/tests/qemu-iotests/233
@@ -72,6 +72,9 @@ $QEMU_IMG info --image-opts \
--object tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0 \
driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port \
+ --object tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0 \
+ --tls-creds=tls0
nbd_server_stop
@@ -84,6 +87,7 @@ nbd_server_start_tcp_socket \
-f $IMGFMT "$TEST_IMG"
$QEMU_IMG info nbd://localhost:$nbd_tcp_port 2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port
echo
echo "== check TLS works =="
@@ -91,6 +95,9 @@ $QEMU_IMG info --image-opts \
--object tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0 \
driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port \
+ --object tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0 \
+ --tls-creds=tls0
echo
echo "== check TLS with different CA fails =="
@@ -98,6 +105,9 @@ $QEMU_IMG info --image-opts \
--object tls-creds-x509,dir=${tls_dir}/client2,endpoint=client,id=tls0 \
driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port \
+ --object tls-creds-x509,dir=${tls_dir}/client2,endpoint=client,id=tls0 \
+ --tls-creds=tls0
echo
echo "== perform I/O over TLS =="
diff --git a/tests/qemu-iotests/233.out b/tests/qemu-iotests/233.out
index 5f416721b03..ab669488669 100644
--- a/tests/qemu-iotests/233.out
+++ b/tests/qemu-iotests/233.out
@@ -15,20 +15,35 @@ wrote 1048576/1048576 bytes at offset 1048576
== check TLS client to plain server fails ==
qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': Denied by server for option 5 (starttls)
server reported: TLS not configured
+qemu-nbd: Denied by server for option 5 (starttls)
+server reported: TLS not configured
== check plain client to TLS server fails ==
qemu-img: Could not open 'nbd://localhost:PORT': TLS negotiation required before option 8 (structured reply)
server reported: Option 0x8 not permitted before TLS
+qemu-nbd: TLS negotiation required before option 8 (structured reply)
+server reported: Option 0x8 not permitted before TLS
== check TLS works ==
image: nbd://127.0.0.1:PORT
file format: nbd
virtual size: 64M (67108864 bytes)
disk size: unavailable
+exports available: 1
+ export: ''
+ size: 67108864
+ flags: 0x4ed ( flush fua trim zeroes df cache )
+ min block: 512
+ opt block: 4096
+ max block: 33554432
+ available meta contexts: 1
+ base:allocation
== check TLS with different CA fails ==
qemu-nbd: option negotiation failed: Verify failed: No certificate was found.
qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': The certificate hasn't got a known issuer
+qemu-nbd: option negotiation failed: Verify failed: No certificate was found.
+qemu-nbd: The certificate hasn't got a known issuer
== perform I/O over TLS ==
read 1048576/1048576 bytes at offset 1048576
--
2.17.2
^ permalink raw reply related [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 01/14] qemu-nbd: Use program name in error messages
2018-11-30 22:03 ` [Qemu-devel] [PATCH 01/14] qemu-nbd: Use program name in error messages Eric Blake
@ 2018-11-30 22:17 ` Richard W.M. Jones
2018-12-05 14:55 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Richard W.M. Jones @ 2018-11-30 22:17 UTC (permalink / raw)
To: Eric Blake
Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block, Kevin Wolf,
Max Reitz
On Fri, Nov 30, 2018 at 04:03:30PM -0600, Eric Blake wrote:
> This changes output from:
>
> $ qemu-nbd nosuch
> Failed to blk_new_open 'nosuch': Could not open 'nosuch': No such file or directory
>
> to something more consistent with qemu-img and qemu:
>
> $ qemu-nbd nosuch
> qemu-nbd: Failed to blk_new_open 'nosuch': Could not open 'nosuch': No such file or directory
>
> Update the lone affected test to match. (Hmm - is it sad that we don't
> do much testing of expected failures?)
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> qemu-nbd.c | 1 +
> tests/qemu-iotests/233.out | 2 +-
> 2 files changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index ca7109652e5..e169b839ece 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -571,6 +571,7 @@ int main(int argc, char **argv)
> #endif
>
> module_call_init(MODULE_INIT_TRACE);
> + error_set_progname(argv[0]);
> qcrypto_init(&error_fatal);
>
> module_call_init(MODULE_INIT_QOM);
> diff --git a/tests/qemu-iotests/233.out b/tests/qemu-iotests/233.out
> index 94acd9b9479..5f416721b03 100644
> --- a/tests/qemu-iotests/233.out
> +++ b/tests/qemu-iotests/233.out
> @@ -27,7 +27,7 @@ virtual size: 64M (67108864 bytes)
> disk size: unavailable
>
> == check TLS with different CA fails ==
> -option negotiation failed: Verify failed: No certificate was found.
> +qemu-nbd: option negotiation failed: Verify failed: No certificate was found.
> qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': The certificate hasn't got a known issuer
>
> == perform I/O over TLS ==
Same as done in qemu (vl.c) and qemu-img.c, so:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 02/14] nbd/client: More consistent error messages
2018-11-30 22:03 ` [Qemu-devel] [PATCH 02/14] nbd/client: More consistent " Eric Blake
@ 2018-11-30 22:20 ` Richard W.M. Jones
2018-12-05 15:03 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Richard W.M. Jones @ 2018-11-30 22:20 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block
On Fri, Nov 30, 2018 at 04:03:31PM -0600, Eric Blake wrote:
> Consolidate on using decimal (not hex) and on outputting the
> option reply name (not just value) when the client reports
> protocol discrepancies from the server. While it won't affect
> normal operation, it makes debugging additions easier.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> nbd/client.c | 21 ++++++++++++---------
> 1 file changed, 12 insertions(+), 9 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index b4d457a19ad..b667a1b56fd 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -132,8 +132,9 @@ static int nbd_receive_option_reply(QIOChannel *ioc, uint32_t opt,
> return -1;
> }
> if (reply->option != opt) {
> - error_setg(errp, "Unexpected option type %x expected %x",
> - reply->option, opt);
> + error_setg(errp, "Unexpected option type %u (%s) expected %u (%s)",
> + reply->option, nbd_opt_lookup(reply->option),
> + opt, nbd_opt_lookup(opt));
> nbd_send_opt_abort(ioc);
> return -1;
> }
> @@ -265,8 +266,9 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> }
> return 0;
> } else if (reply.type != NBD_REP_SERVER) {
> - error_setg(errp, "Unexpected reply type %" PRIx32 " expected %x",
> - reply.type, NBD_REP_SERVER);
> + error_setg(errp, "Unexpected reply type %u (%s) expected %u (%s)",
> + reply.type, nbd_rep_lookup(reply.type),
> + NBD_REP_SERVER, nbd_rep_lookup(NBD_REP_SERVER));
> nbd_send_opt_abort(ioc);
> return -1;
> }
> @@ -378,9 +380,9 @@ static int nbd_opt_go(QIOChannel *ioc, const char *wantname,
> return 1;
> }
> if (reply.type != NBD_REP_INFO) {
> - error_setg(errp, "unexpected reply type %" PRIu32
> - " (%s), expected %u",
> - reply.type, nbd_rep_lookup(reply.type), NBD_REP_INFO);
> + error_setg(errp, "unexpected reply type %u (%s), expected %u (%s)",
> + reply.type, nbd_rep_lookup(reply.type),
> + NBD_REP_INFO, nbd_rep_lookup(NBD_REP_INFO));
> nbd_send_opt_abort(ioc);
> return -1;
> }
> @@ -704,8 +706,9 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> }
>
> if (reply.type != NBD_REP_ACK) {
> - error_setg(errp, "Unexpected reply type %" PRIx32 " expected %x",
> - reply.type, NBD_REP_ACK);
> + error_setg(errp, "Unexpected reply type %u (%s) expected %u (%s)",
> + reply.type, nbd_rep_lookup(reply.type),
> + NBD_REP_ACK, nbd_rep_lookup(NBD_REP_ACK));
> nbd_send_opt_abort(ioc);
> return -1;
> }
The NBD protocol doc seems to use integers pretty consistently (and
certainly not "bare" hex numbers). Obviously having the mnemonic name
too is helpful. So:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 03/14] qemu-nbd: Fail earlier for -c/-d on non-linux
2018-11-30 22:03 ` [Qemu-devel] [PATCH 03/14] qemu-nbd: Fail earlier for -c/-d on non-linux Eric Blake
@ 2018-11-30 22:23 ` Richard W.M. Jones
2018-12-05 15:20 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Richard W.M. Jones @ 2018-11-30 22:23 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block
On Fri, Nov 30, 2018 at 04:03:32PM -0600, Eric Blake wrote:
> Connecting to a /dev/nbdN device is a Linux-specific action.
> We were already masking -c and -d from 'qemu-nbd --help' on
> non-linux. However, while -d fails with a sensible error
> message, it took hunting through a couple of files to prove
> that. What's more, the code for -c doesn't fail until after
> it has created a pthread and tried to open a device - possibly
> even printing an error message with %m on a non-Linux platform
> in spite of the comment that %m is glibc-specific. Make the
> failure happen sooner, then get rid of stubs that are no
> longer needed because of the early exits.
>
> While at it: tweak the blank newlines in --help output to be
> consistent, whether or not built on Linux.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> nbd/client.c | 18 +-----------------
> qemu-nbd.c | 22 ++++++++++++++++++++--
> 2 files changed, 21 insertions(+), 19 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index b667a1b56fd..0be89f9e641 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -1029,23 +1029,7 @@ int nbd_disconnect(int fd)
> return 0;
> }
>
> -#else
> -int nbd_init(int fd, QIOChannelSocket *ioc, NBDExportInfo *info,
> - Error **errp)
> -{
> - error_setg(errp, "nbd_init is only supported on Linux");
> - return -ENOTSUP;
> -}
> -
> -int nbd_client(int fd)
> -{
> - return -ENOTSUP;
> -}
> -int nbd_disconnect(int fd)
> -{
> - return -ENOTSUP;
> -}
> -#endif
> +#endif /* __linux__ */
>
> int nbd_send_request(QIOChannel *ioc, NBDRequest *request)
> {
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index e169b839ece..55e29bd9a7e 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -43,6 +43,12 @@
> #include "trace/control.h"
> #include "qemu-version.h"
>
> +#ifdef __linux__
> +#define HAVE_NBD_DEVICE 1
> +#else
> +#define HAVE_NBD_DEVICE 0
> +#endif
> +
> #define SOCKET_PATH "/var/lock/qemu-nbd-%s"
> #define QEMU_NBD_OPT_CACHE 256
> #define QEMU_NBD_OPT_AIO 257
> @@ -98,11 +104,11 @@ static void usage(const char *name)
> " specify tracing options\n"
> " --fork fork off the server process and exit the parent\n"
> " once the server is running\n"
> -#ifdef __linux__
> +#if HAVE_NBD_DEVICE
> +"\n"
> "Kernel NBD client support:\n"
> " -c, --connect=DEV connect FILE to the local NBD device DEV\n"
> " -d, --disconnect disconnect the specified device\n"
> -"\n"
> #endif
> "\n"
> "Block device options:\n"
> @@ -236,6 +242,7 @@ static void termsig_handler(int signum)
> }
>
>
> +#if HAVE_NBD_DEVICE
> static void *show_parts(void *arg)
> {
> char *device = arg;
> @@ -321,6 +328,7 @@ out:
> kill(getpid(), SIGTERM);
> return (void *) EXIT_FAILURE;
> }
> +#endif /* HAVE_NBD_DEVICE */
>
> static int nbd_can_accept(void)
> {
> @@ -815,6 +823,7 @@ int main(int argc, char **argv)
> }
>
> if (disconnect) {
> +#if HAVE_NBD_DEVICE
> int nbdfd = open(argv[optind], O_RDWR);
> if (nbdfd < 0) {
> error_report("Cannot open %s: %s", argv[optind],
> @@ -828,6 +837,10 @@ int main(int argc, char **argv)
> printf("%s disconnected\n", argv[optind]);
>
> return 0;
> +#else
> + error_report("Kernel /dev/nbdN support not available");
> + exit(EXIT_FAILURE);
> +#endif
> }
>
> if ((device && !verbose) || fork_process) {
> @@ -1006,6 +1019,7 @@ int main(int argc, char **argv)
> nbd_export_set_description(exp, export_description);
>
> if (device) {
> +#if HAVE_NBD_DEVICE
> int ret;
>
> ret = pthread_create(&client_thread, NULL, nbd_client_thread, device);
> @@ -1013,6 +1027,10 @@ int main(int argc, char **argv)
> error_report("Failed to create client thread: %s", strerror(ret));
> exit(EXIT_FAILURE);
> }
> +#else
> + error_report("Kernel /dev/nbdN support not available");
> + exit(EXIT_FAILURE);
> +#endif
> } else {
> /* Shut up GCC warnings. */
> memset(&client_thread, 0, sizeof(client_thread));
Looks like a sensible code refactoring/simplification to me, so:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 04/14] qemu-nbd: Simplify --partition handling
2018-11-30 22:03 ` [Qemu-devel] [PATCH 04/14] qemu-nbd: Simplify --partition handling Eric Blake
@ 2018-11-30 22:26 ` Richard W.M. Jones
2018-11-30 22:41 ` Eric Blake
2018-12-05 15:40 ` Vladimir Sementsov-Ogievskiy
2018-12-10 22:28 ` Eric Blake
2 siblings, 1 reply; 65+ messages in thread
From: Richard W.M. Jones @ 2018-11-30 22:26 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block
On Fri, Nov 30, 2018 at 04:03:33PM -0600, Eric Blake wrote:
> Our open-coding of strtol handling forgot to handle overflow
> conditions. What's more, since we insiste on a user-supplied
"insist"
> partition to be non-zero, we can use 0 rather than -1 for our
> initial value to distinguish when a partition is not being
> served, for slightly more optimal code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> qemu-nbd.c | 14 +++++---------
> 1 file changed, 5 insertions(+), 9 deletions(-)
>
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index 55e29bd9a7e..866e64779f1 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -546,7 +546,7 @@ int main(int argc, char **argv)
> int opt_ind = 0;
> char *end;
> int flags = BDRV_O_RDWR;
> - int partition = -1;
> + int partition = 0;
> int ret = 0;
> bool seen_cache = false;
> bool seen_discard = false;
> @@ -685,13 +685,9 @@ int main(int argc, char **argv)
> flags &= ~BDRV_O_RDWR;
> break;
> case 'P':
> - partition = strtol(optarg, &end, 0);
> - if (*end) {
> - error_report("Invalid partition `%s'", optarg);
> - exit(EXIT_FAILURE);
> - }
> - if (partition < 1 || partition > 8) {
> - error_report("Invalid partition %d", partition);
> + if (qemu_strtoi(optarg, NULL, 0, &partition) < 0 ||
> + partition < 1 || partition > 8) {
> + error_report("Invalid partition %s", optarg);
Pffft only supporting a mere 8 partitions :-? I raised the limits in
nbdkit recently so it can handle an infinite number of partitions :-)
Anyway, it's a problem with the existing code, so doesn't affect the
review.
> exit(EXIT_FAILURE);
> }
> break;
> @@ -1004,7 +1000,7 @@ int main(int argc, char **argv)
> }
> fd_size -= dev_offset;
>
> - if (partition != -1) {
> + if (partition) {
> ret = find_partition(blk, partition, &dev_offset, &fd_size);
> if (ret < 0) {
> error_report("Could not find partition %d: %s", partition,
> --
> 2.17.2
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 05/14] nbd/client: Drop pointless buf variable
2018-11-30 22:03 ` [Qemu-devel] [PATCH 05/14] nbd/client: Drop pointless buf variable Eric Blake
@ 2018-11-30 22:30 ` Richard W.M. Jones
2018-11-30 22:54 ` Eric Blake
2018-12-05 15:59 ` Vladimir Sementsov-Ogievskiy
1 sibling, 1 reply; 65+ messages in thread
From: Richard W.M. Jones @ 2018-11-30 22:30 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block
On Fri, Nov 30, 2018 at 04:03:34PM -0600, Eric Blake wrote:
> There's no need to read into a temporary buffer (oversized
> since commit 7d3123e1) followed by a byteswap into a uint64_t
> to check for a magic number via memcmp(), when the code
> immediately below demonstrates reading into the uint64_t then
> byteswapping in place and checking for a magic number via
> integer math. What's more, having a different error message
> when the server's first reply byte is 0 is unusual - it's no
> different from any other wrong magic number, and we already
> detected short reads.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> nbd/nbd-internal.h | 1 +
> nbd/client.c | 14 +++-----------
> 2 files changed, 4 insertions(+), 11 deletions(-)
>
> diff --git a/nbd/nbd-internal.h b/nbd/nbd-internal.h
> index eeff78d3c98..306a533dcd1 100644
> --- a/nbd/nbd-internal.h
> +++ b/nbd/nbd-internal.h
> @@ -46,6 +46,7 @@
> /* Size of oldstyle negotiation */
> #define NBD_OLDSTYLE_NEGOTIATE_SIZE (8 + 8 + 8 + 4 + 124)
>
> +#define NBD_INIT_MAGIC 0x4e42444d41474943LL
> #define NBD_REQUEST_MAGIC 0x25609513
> #define NBD_OPTS_MAGIC 0x49484156454F5054LL
> #define NBD_CLIENT_MAGIC 0x0000420281861253LL
> diff --git a/nbd/client.c b/nbd/client.c
> index 0be89f9e641..17ee24492a4 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -731,7 +731,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
> QIOChannel **outioc, NBDExportInfo *info,
> Error **errp)
> {
> - char buf[256];
> uint64_t magic;
> int rc;
> bool zeroes = true;
> @@ -752,21 +751,14 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
> goto fail;
> }
>
> - if (nbd_read(ioc, buf, 8, errp) < 0) {
> + if (nbd_read(ioc, &magic, sizeof(magic), errp) < 0) {
> error_prepend(errp, "Failed to read data: ");
> goto fail;
> }
> -
> - buf[8] = '\0';
> - if (strlen(buf) == 0) {
> - error_setg(errp, "Server connection closed unexpectedly");
> - goto fail;
> - }
> -
> - magic = ldq_be_p(buf);
> + magic = be64_to_cpu(magic);
> trace_nbd_receive_negotiate_magic(magic);
>
> - if (memcmp(buf, "NBDMAGIC", 8) != 0) {
> + if (magic != NBD_INIT_MAGIC) {
> error_setg(errp, "Invalid magic received");
> goto fail;
> }
The original code is really odd. What's the whole strlen about?
Anyway this is an obvious improvement so:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 06/14] nbd/client: Move export name into NBDExportInfo
2018-11-30 22:03 ` [Qemu-devel] [PATCH 06/14] nbd/client: Move export name into NBDExportInfo Eric Blake
@ 2018-11-30 22:34 ` Richard W.M. Jones
2018-12-05 17:26 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Richard W.M. Jones @ 2018-11-30 22:34 UTC (permalink / raw)
To: Eric Blake
Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block, Kevin Wolf,
Max Reitz
On Fri, Nov 30, 2018 at 04:03:35PM -0600, Eric Blake wrote:
> Refactor the 'name' parameter of nbd_receive_negotiate() from
> being a separate parameter into being part of the in-out 'info'.
> This also spills over to a simplification of nbd_opt_go().
>
> The main driver for this refactoring is that an upcoming patch
> would like to add support to qemu-nbd to list information about
> all exports available on a server, where the name(s) will be
> provided by the server instead of the client. But another benefit
> is that we can now allow the client to explicitly specify the
> empty export name "" even when connecting to an oldstyle server
> (even if qemu is no longer such a server after commit 7f7dfe2a).
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> include/block/nbd.h | 8 ++++----
> block/nbd-client.c | 5 +++--
> nbd/client.c | 39 ++++++++++++++++++---------------------
> qemu-nbd.c | 6 ++++--
> nbd/trace-events | 2 +-
> 5 files changed, 30 insertions(+), 30 deletions(-)
>
> diff --git a/include/block/nbd.h b/include/block/nbd.h
> index 6a5bfe5d559..65feff8ba96 100644
> --- a/include/block/nbd.h
> +++ b/include/block/nbd.h
> @@ -262,6 +262,7 @@ struct NBDExportInfo {
> /* Set by client before nbd_receive_negotiate() */
> bool request_sizes;
> char *x_dirty_bitmap;
> + char *name; /* must be non-NULL */
>
> /* In-out fields, set by client before nbd_receive_negotiate() and
> * updated by server results during nbd_receive_negotiate() */
> @@ -279,10 +280,9 @@ struct NBDExportInfo {
> };
> typedef struct NBDExportInfo NBDExportInfo;
>
> -int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
> - QCryptoTLSCreds *tlscreds, const char *hostname,
> - QIOChannel **outioc, NBDExportInfo *info,
> - Error **errp);
> +int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> + const char *hostname, QIOChannel **outioc,
> + NBDExportInfo *info, Error **errp);
> int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
> Error **errp);
> int nbd_send_request(QIOChannel *ioc, NBDRequest *request);
> diff --git a/block/nbd-client.c b/block/nbd-client.c
> index fc5b7eda8ee..417971d8b05 100644
> --- a/block/nbd-client.c
> +++ b/block/nbd-client.c
> @@ -984,10 +984,11 @@ int nbd_client_init(BlockDriverState *bs,
> client->info.structured_reply = true;
> client->info.base_allocation = true;
> client->info.x_dirty_bitmap = g_strdup(x_dirty_bitmap);
> - ret = nbd_receive_negotiate(QIO_CHANNEL(sioc), export,
> - tlscreds, hostname,
> + client->info.name = g_strdup(export ?: "");
> + ret = nbd_receive_negotiate(QIO_CHANNEL(sioc), tlscreds, hostname,
> &client->ioc, &client->info, errp);
> g_free(client->info.x_dirty_bitmap);
> + g_free(client->info.name);
> if (ret < 0) {
> logout("Failed to negotiate with the NBD server\n");
> return ret;
> diff --git a/nbd/client.c b/nbd/client.c
> index 17ee24492a4..b5818a99d21 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -320,15 +320,14 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> }
>
>
> -/* Returns -1 if NBD_OPT_GO proves the export @wantname cannot be
> +/* Returns -1 if NBD_OPT_GO proves the export @info->name cannot be
> * used, 0 if NBD_OPT_GO is unsupported (fall back to NBD_OPT_LIST and
> * NBD_OPT_EXPORT_NAME in that case), and > 0 if the export is good to
> - * go (with @info populated). */
> -static int nbd_opt_go(QIOChannel *ioc, const char *wantname,
> - NBDExportInfo *info, Error **errp)
> + * go (with the rest of @info populated). */
> +static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
> {
> NBDOptionReply reply;
> - uint32_t len = strlen(wantname);
> + uint32_t len = strlen(info->name);
> uint16_t type;
> int error;
> char *buf;
> @@ -338,10 +337,10 @@ static int nbd_opt_go(QIOChannel *ioc, const char *wantname,
> * flags still 0 is a witness of a broken server. */
> info->flags = 0;
>
> - trace_nbd_opt_go_start(wantname);
> + trace_nbd_opt_go_start(info->name);
> buf = g_malloc(4 + len + 2 + 2 * info->request_sizes + 1);
> stl_be_p(buf, len);
> - memcpy(buf + 4, wantname, len);
> + memcpy(buf + 4, info->name, len);
> /* At most one request, everything else up to server */
> stw_be_p(buf + 4 + len, info->request_sizes);
> if (info->request_sizes) {
> @@ -726,10 +725,9 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return 0;
> }
>
> -int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
> - QCryptoTLSCreds *tlscreds, const char *hostname,
> - QIOChannel **outioc, NBDExportInfo *info,
> - Error **errp)
> +int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> + const char *hostname, QIOChannel **outioc,
> + NBDExportInfo *info, Error **errp)
> {
> uint64_t magic;
> int rc;
> @@ -739,6 +737,8 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
>
> trace_nbd_receive_negotiate(tlscreds, hostname ? hostname : "<null>");
>
> + assert(info->name);
> + trace_nbd_receive_negotiate_name(info->name);
> info->structured_reply = false;
> info->base_allocation = false;
> rc = -EINVAL;
> @@ -807,10 +807,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
> goto fail;
> }
> }
> - if (!name) {
> - trace_nbd_receive_negotiate_default_name();
> - name = "";
> - }
> if (fixedNewStyle) {
> int result;
>
> @@ -826,7 +822,8 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
>
> if (info->structured_reply && base_allocation) {
> result = nbd_negotiate_simple_meta_context(
> - ioc, name, info->x_dirty_bitmap ?: "base:allocation",
> + ioc, info->name,
> + info->x_dirty_bitmap ?: "base:allocation",
> &info->meta_base_allocation_id, errp);
> if (result < 0) {
> goto fail;
> @@ -839,7 +836,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
> * TLS). If it is not available, fall back to
> * NBD_OPT_LIST for nicer error messages about a missing
> * export, then use NBD_OPT_EXPORT_NAME. */
> - result = nbd_opt_go(ioc, name, info, errp);
> + result = nbd_opt_go(ioc, info, errp);
> if (result < 0) {
> goto fail;
> }
> @@ -852,12 +849,12 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
> * query gives us better error reporting if the
> * export name is not available.
> */
> - if (nbd_receive_query_exports(ioc, name, errp) < 0) {
> + if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
> goto fail;
> }
> }
> /* write the export name request */
> - if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, name,
> + if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, info->name,
> errp) < 0) {
> goto fail;
> }
> @@ -877,8 +874,8 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
> } else if (magic == NBD_CLIENT_MAGIC) {
> uint32_t oldflags;
>
> - if (name) {
> - error_setg(errp, "Server does not support export names");
> + if (*info->name) {
> + error_setg(errp, "Server does not support non-empty export names");
> goto fail;
> }
> if (tlscreds) {
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index 866e64779f1..c57053a0795 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -263,7 +263,7 @@ static void *show_parts(void *arg)
> static void *nbd_client_thread(void *arg)
> {
> char *device = arg;
> - NBDExportInfo info = { .request_sizes = false, };
> + NBDExportInfo info = { .request_sizes = false, .name = g_strdup("") };
> QIOChannelSocket *sioc;
> int fd;
> int ret;
> @@ -278,7 +278,7 @@ static void *nbd_client_thread(void *arg)
> goto out;
> }
>
> - ret = nbd_receive_negotiate(QIO_CHANNEL(sioc), NULL,
> + ret = nbd_receive_negotiate(QIO_CHANNEL(sioc),
> NULL, NULL, NULL, &info, &local_error);
> if (ret < 0) {
> if (local_error) {
> @@ -317,6 +317,7 @@ static void *nbd_client_thread(void *arg)
> }
> close(fd);
> object_unref(OBJECT(sioc));
> + free(info.name);
> kill(getpid(), SIGTERM);
> return (void *) EXIT_SUCCESS;
>
> @@ -325,6 +326,7 @@ out_fd:
> out_socket:
> object_unref(OBJECT(sioc));
> out:
> + free(info.name);
> kill(getpid(), SIGTERM);
> return (void *) EXIT_FAILURE;
> }
> diff --git a/nbd/trace-events b/nbd/trace-events
> index 5e1d4afe8e6..289337d0dc3 100644
> --- a/nbd/trace-events
> +++ b/nbd/trace-events
> @@ -15,7 +15,7 @@ nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of contex
> nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
> nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
> nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are 0x%" PRIx32
> -nbd_receive_negotiate_default_name(void) "Using default NBD export name \"\""
> +nbd_receive_negotiate_name(const char *name) "Requesting NBD export name \"%s\""
> nbd_receive_negotiate_size_flags(uint64_t size, uint16_t flags) "Size is %" PRIu64 ", export flags 0x%" PRIx16
> nbd_init_set_socket(void) "Setting NBD socket"
> nbd_init_set_block_size(unsigned long block_size) "Setting block size to %lu"
Simple parameter refactoring, so:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 09/14] nbd/client: Refactor return of nbd_receive_negotiate()
2018-11-30 22:03 ` [Qemu-devel] [PATCH 09/14] nbd/client: Refactor return of nbd_receive_negotiate() Eric Blake
@ 2018-11-30 22:41 ` Richard W.M. Jones
2018-12-06 14:24 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Richard W.M. Jones @ 2018-11-30 22:41 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block
On Fri, Nov 30, 2018 at 04:03:38PM -0600, Eric Blake wrote:
> The function could only ever return 0 or -EINVAL; make this
> clearer by dropping a useless 'fail:' label.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> nbd/client.c | 51 +++++++++++++++++++++++----------------------------
> 1 file changed, 23 insertions(+), 28 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 27785c55d0a..1ed5009642e 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -773,7 +773,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> NBDExportInfo *info, Error **errp)
> {
> uint64_t magic;
> - int rc;
> bool zeroes = true;
> bool structured_reply = info->structured_reply;
> bool base_allocation = info->base_allocation;
> @@ -784,31 +783,30 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> trace_nbd_receive_negotiate_name(info->name);
> info->structured_reply = false;
> info->base_allocation = false;
> - rc = -EINVAL;
>
> if (outioc) {
> *outioc = NULL;
> }
> if (tlscreds && !outioc) {
> error_setg(errp, "Output I/O channel required for TLS");
> - goto fail;
> + return -EINVAL;
> }
>
> if (nbd_read(ioc, &magic, sizeof(magic), errp) < 0) {
> error_prepend(errp, "Failed to read data: ");
> - goto fail;
> + return -EINVAL;
> }
> magic = be64_to_cpu(magic);
> trace_nbd_receive_negotiate_magic(magic);
>
> if (magic != NBD_INIT_MAGIC) {
> error_setg(errp, "Invalid magic received");
> - goto fail;
> + return -EINVAL;
> }
>
> if (nbd_read(ioc, &magic, sizeof(magic), errp) < 0) {
> error_prepend(errp, "Failed to read magic: ");
> - goto fail;
> + return -EINVAL;
> }
> magic = be64_to_cpu(magic);
> trace_nbd_receive_negotiate_magic(magic);
> @@ -820,7 +818,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
>
> if (nbd_read(ioc, &globalflags, sizeof(globalflags), errp) < 0) {
> error_prepend(errp, "Failed to read server flags: ");
> - goto fail;
> + return -EINVAL;
> }
> globalflags = be16_to_cpu(globalflags);
> trace_nbd_receive_negotiate_server_flags(globalflags);
> @@ -836,18 +834,18 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> clientflags = cpu_to_be32(clientflags);
> if (nbd_write(ioc, &clientflags, sizeof(clientflags), errp) < 0) {
> error_prepend(errp, "Failed to send clientflags field: ");
> - goto fail;
> + return -EINVAL;
> }
> if (tlscreds) {
> if (fixedNewStyle) {
> *outioc = nbd_receive_starttls(ioc, tlscreds, hostname, errp);
> if (!*outioc) {
> - goto fail;
> + return -EINVAL;
> }
> ioc = *outioc;
> } else {
> error_setg(errp, "Server does not support STARTTLS");
> - goto fail;
> + return -EINVAL;
> }
> }
> if (fixedNewStyle) {
> @@ -858,7 +856,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> NBD_OPT_STRUCTURED_REPLY,
> errp);
> if (result < 0) {
> - goto fail;
> + return -EINVAL;
> }
> info->structured_reply = result == 1;
> }
> @@ -869,7 +867,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> info->x_dirty_bitmap ?: "base:allocation",
> info, errp);
> if (result < 0) {
> - goto fail;
> + return -EINVAL;
> }
> info->base_allocation = result == 1;
> }
> @@ -881,7 +879,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> * export, then use NBD_OPT_EXPORT_NAME. */
> result = nbd_opt_go(ioc, info, errp);
> if (result < 0) {
> - goto fail;
> + return -EINVAL;
> }
> if (result > 0) {
> return 0;
> @@ -893,25 +891,25 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> * export name is not available.
> */
> if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
> - goto fail;
> + return -EINVAL;
> }
> }
> /* write the export name request */
> if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, info->name,
> errp) < 0) {
> - goto fail;
> + return -EINVAL;
> }
>
> /* Read the response */
> if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
> error_prepend(errp, "Failed to read export length: ");
> - goto fail;
> + return -EINVAL;
> }
> info->size = be64_to_cpu(info->size);
>
> if (nbd_read(ioc, &info->flags, sizeof(info->flags), errp) < 0) {
> error_prepend(errp, "Failed to read export flags: ");
> - goto fail;
> + return -EINVAL;
> }
> info->flags = be16_to_cpu(info->flags);
> } else if (magic == NBD_CLIENT_MAGIC) {
> @@ -919,43 +917,40 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
>
> if (*info->name) {
> error_setg(errp, "Server does not support non-empty export names");
> - goto fail;
> + return -EINVAL;
> }
> if (tlscreds) {
> error_setg(errp, "Server does not support STARTTLS");
> - goto fail;
> + return -EINVAL;
> }
>
> if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
> error_prepend(errp, "Failed to read export length: ");
> - goto fail;
> + return -EINVAL;
> }
> info->size = be64_to_cpu(info->size);
>
> if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
> error_prepend(errp, "Failed to read export flags: ");
> - goto fail;
> + return -EINVAL;
> }
> oldflags = be32_to_cpu(oldflags);
> if (oldflags & ~0xffff) {
> error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
> - goto fail;
> + return -EINVAL;
> }
> info->flags = oldflags;
> } else {
> error_setg(errp, "Bad magic received");
> - goto fail;
> + return -EINVAL;
> }
>
> trace_nbd_receive_negotiate_size_flags(info->size, info->flags);
> if (zeroes && nbd_drop(ioc, 124, errp) < 0) {
> error_prepend(errp, "Failed to read reserved block: ");
> - goto fail;
> + return -EINVAL;
> }
> - rc = 0;
> -
> -fail:
> - return rc;
> + return 0;
> }
>
Simple refactoring:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 04/14] qemu-nbd: Simplify --partition handling
2018-11-30 22:26 ` Richard W.M. Jones
@ 2018-11-30 22:41 ` Eric Blake
0 siblings, 0 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:41 UTC (permalink / raw)
To: Richard W.M. Jones; +Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block
On 11/30/18 4:26 PM, Richard W.M. Jones wrote:
> On Fri, Nov 30, 2018 at 04:03:33PM -0600, Eric Blake wrote:
>> Our open-coding of strtol handling forgot to handle overflow
>> conditions. What's more, since we insiste on a user-supplied
>
> "insist"
(Ever wonder if I stick in a typo on purpose, just to see who reviews
closely?)
>
>> partition to be non-zero, we can use 0 rather than -1 for our
>> initial value to distinguish when a partition is not being
>> served, for slightly more optimal code.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>> qemu-nbd.c | 14 +++++---------
>> 1 file changed, 5 insertions(+), 9 deletions(-)
>>
>> + if (qemu_strtoi(optarg, NULL, 0, &partition) < 0 ||
>> + partition < 1 || partition > 8) {
>> + error_report("Invalid partition %s", optarg);
>
> Pffft only supporting a mere 8 partitions :-? I raised the limits in
> nbdkit recently so it can handle an infinite number of partitions :-)
nbdkit also handles GPT partitions. qemu-nbd is still stuck on MBR:
static int find_partition(BlockBackend *blk, int partition,
off_t *offset, off_t *size)
{
struct partition_record mbr[4];
uint8_t data[MBR_SIZE];
and if I read git blame correctly, the partition code in qemu-nbd is
mostly untouched since Fabrice committed Anthony's initial
implementation in 2008.
Also, reading the code, I see that qemu-nbd allows partitions 5-8
because it reads through the extended partition descriptor in 4 to
expose the logical partitions within; which nbdkit can't do yet :)
>
> Anyway, it's a problem with the existing code, so doesn't affect the
> review.
Indeed, and with nbdkit around, I'm wondering if I should deprecate
'qemu-nbd --partition' and just point people to nbdkit instead
(provided, of course, that nbdkit learns logical partitions).
Even if you have a qcow2 image with MBR partitions, and where we don't
want to write a qcow2 plugin in nbdkit, you can still always rewrite:
client -> qemu-nbd --partition=1
into
client -> nbdkit --filter=partition nbd partition=1 -> qemu-nbd
(although I don't know what the performance penalty would be)
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 05/14] nbd/client: Drop pointless buf variable
2018-11-30 22:30 ` Richard W.M. Jones
@ 2018-11-30 22:54 ` Eric Blake
0 siblings, 0 replies; 65+ messages in thread
From: Eric Blake @ 2018-11-30 22:54 UTC (permalink / raw)
To: Richard W.M. Jones; +Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block
On 11/30/18 4:30 PM, Richard W.M. Jones wrote:
> On Fri, Nov 30, 2018 at 04:03:34PM -0600, Eric Blake wrote:
>> There's no need to read into a temporary buffer (oversized
>> since commit 7d3123e1) followed by a byteswap into a uint64_t
>> to check for a magic number via memcmp(), when the code
>> immediately below demonstrates reading into the uint64_t then
>> byteswapping in place and checking for a magic number via
>> integer math. What's more, having a different error message
>> when the server's first reply byte is 0 is unusual - it's no
>> different from any other wrong magic number, and we already
>> detected short reads.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>> -
>> - buf[8] = '\0';
>> - if (strlen(buf) == 0) {
>> - error_setg(errp, "Server connection closed unexpectedly");
>> - goto fail;
>> - }
>> -
>> - magic = ldq_be_p(buf);
>> + magic = be64_to_cpu(magic);
>> trace_nbd_receive_negotiate_magic(magic);
>>
>> - if (memcmp(buf, "NBDMAGIC", 8) != 0) {
>> + if (magic != NBD_INIT_MAGIC) {
>> error_setg(errp, "Invalid magic received");
>> goto fail;
>> }
>
> The original code is really odd. What's the whole strlen about?
Looks like it was added in commit 1d45f8b4 in 2010 in "nbd: Introduce
NBD named exports." but with no good explanation why it was added in the
context of the larger patch. Leftover debugging code that should have
been nuked years ago?
> Anyway this is an obvious improvement so:
>
> Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
>
> Rich.
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
` (13 preceding siblings ...)
2018-11-30 22:03 ` [Qemu-devel] [PATCH 14/14] iotests: Enhance 223, 233 to cover 'qemu-nbd --list' Eric Blake
@ 2018-12-01 7:42 ` Richard W.M. Jones
2018-12-01 13:57 ` Eric Blake
14 siblings, 1 reply; 65+ messages in thread
From: Richard W.M. Jones @ 2018-12-01 7:42 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block
On Fri, Nov 30, 2018 at 04:03:29PM -0600, Eric Blake wrote:
> I note that upstream NBD has 'nbd-client -l $host' for querying
> just export names (with no quoting, so you have to know that
> a blank line means the default export), but it wasn't powerful
> enough, so I implemented 'qemu-nbd -L' to document everything.
> Upstream NBD has separate 'nbd-client' and 'nbd-server' binaries,
> while we only have 'qemu-nbd' (which is normally just a server,
> but 'qemu-nbd -c' also operates a second thread as a client).
> Our other uses of qemu as NBD client are for consuming a block
> device (as in qemu-io, qemu-img, or a drive to qemu) - but those
> binaries are less suited to something so specific to the NBD
> protocol.
I tried it against nbdkit and it works (obviously):
$ ./qemu-nbd -L
exports available: 1
export: ''
size: 67108864
flags: 0x61 ( trim zeroes )
What I couldn't work out is how to connect to other hosts. It's
possible to add -p or -k to change the localhost port number or to use
a Unix domain socket (and I checked both work). However can we
connect to remote hosts?
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 07/14] nbd/client: Refactor nbd_negotiate_simple_meta_context()
2018-11-30 22:03 ` [Qemu-devel] [PATCH 07/14] nbd/client: Refactor nbd_negotiate_simple_meta_context() Eric Blake
@ 2018-12-01 10:30 ` Richard W.M. Jones
2018-12-06 13:20 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Richard W.M. Jones @ 2018-12-01 10:30 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block
On Fri, Nov 30, 2018 at 04:03:36PM -0600, Eric Blake wrote:
> Change the signature to make it easier for a future patch to
> reuse this function for calling NBD_OPT_LIST_META_CONTEXT with
> 0 or 1 queries. Also, always allocate space for the received
> name, even if it doesn't match expected lengths (no point
> trying to optimize the unlikely error case, and tracing the
> received rather than expected name can help debug a server
> implementation).
>
> While there are now slightly different traces, and the error
> message for a server replying with too many contexts is
> different, there are no runtime-observable changes in behavior
> for the more common case of the lone caller interacting with a
> compliant server.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> nbd/client.c | 105 +++++++++++++++++++++++++++--------------------
> nbd/trace-events | 2 +-
> 2 files changed, 61 insertions(+), 46 deletions(-)
I've stared at this patch over a 12 hour period now and while I do
_not_ think it's wrong at all, and I've tested it against nbdkit, it's
not very easy to follow. Also my unfamiliarity with the qemu block
layer / NBD client code doesn't help.
> diff --git a/nbd/client.c b/nbd/client.c
> index b5818a99d21..1dc8f83e19a 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -603,49 +603,57 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
> }
>
> /* nbd_negotiate_simple_meta_context:
> - * Set one meta context. Simple means that reply must contain zero (not
> - * negotiated) or one (negotiated) contexts. More contexts would be considered
> - * as a protocol error. It's also implied that meta-data query equals queried
> - * context name, so, if server replies with something different than @context,
> - * it is considered an error too.
> - * return 1 for successful negotiation, context_id is set
> - * 0 if operation is unsupported,
> + * List or set meta context data for export @info->name, based on @opt.
> + * For list, leave @context NULL for 0 queries, or supplied for a single
> + * query; all replies are ignored, and the call merely traces server behavior.
> + * For set, @context must result in at most one matching server reply, in
> + * which case @info->meta_base_allocation_id is set to the resulting id.
> + * return 1 for successful negotiation,
> + * 0 if operation is unsupported or context unavailable,
> * -1 with errp set for any other error
> */
> static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> - const char *export,
> + int32_t opt,
> const char *context,
> - uint32_t *context_id,
> + NBDExportInfo *info,
> Error **errp)
> {
> int ret;
> NBDOptionReply reply;
> uint32_t received_id = 0;
> bool received = false;
> - uint32_t export_len = strlen(export);
> - uint32_t context_len = strlen(context);
> - uint32_t data_len = sizeof(export_len) + export_len +
> - sizeof(uint32_t) + /* number of queries */
> - sizeof(context_len) + context_len;
> - char *data = g_malloc(data_len);
> - char *p = data;
> + uint32_t export_len = strlen(info->name);
> + uint32_t context_len;
> + uint32_t data_len = sizeof(export_len) + export_len + sizeof(uint32_t);
> + char *data;
> + char *p;
I think you could add a comment about how data_len is calculated here.
The calculation is different from the previous code (in the fragment
above, but the fragment below makes the calculation the same). The
old code at least had a brief comment.
> - trace_nbd_opt_meta_request(context, export);
> + if (!context) {
> + assert(opt == NBD_OPT_LIST_META_CONTEXT);
> + } else {
> + context_len = strlen(context);
> + data_len += sizeof(context_len) + context_len;
> + }
> + data = g_malloc(data_len);
> + p = data;
> +
> + trace_nbd_opt_meta_request(nbd_opt_lookup(opt), context ?: "(all)",
> + info->name);
> stl_be_p(p, export_len);
> - memcpy(p += sizeof(export_len), export, export_len);
> - stl_be_p(p += export_len, 1);
> - stl_be_p(p += sizeof(uint32_t), context_len);
> - memcpy(p += sizeof(context_len), context, context_len);
> + memcpy(p += sizeof(export_len), info->name, export_len);
> + stl_be_p(p += export_len, !!context);
> + if (context) {
> + stl_be_p(p += sizeof(uint32_t), context_len);
> + memcpy(p += sizeof(context_len), context, context_len);
> + }
Again even though the old code wasn't commented, a brief comment
describing the data layout wouldn't go amiss.
> - ret = nbd_send_option_request(ioc, NBD_OPT_SET_META_CONTEXT, data_len, data,
> - errp);
> + ret = nbd_send_option_request(ioc, opt, data_len, data, errp);
> g_free(data);
> if (ret < 0) {
> return ret;
> }
>
> - if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
> - errp) < 0)
> + if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0)
> {
> return -1;
> }
> @@ -655,10 +663,10 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return ret;
> }
>
> - if (reply.type == NBD_REP_META_CONTEXT) {
> + while (reply.type == NBD_REP_META_CONTEXT) {
> char *name;
>
> - if (reply.length != sizeof(received_id) + context_len) {
> + if (reply.length <= sizeof(received_id)) {
> error_setg(errp, "Failed to negotiate meta context '%s', server "
> "answered with unexpected length %" PRIu32, context,
> reply.length);
> @@ -678,23 +686,31 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return -1;
> }
> name[reply.length] = '\0';
> - if (strcmp(context, name)) {
> - error_setg(errp, "Failed to negotiate meta context '%s', server "
> - "answered with different context '%s'", context,
> - name);
> - g_free(name);
> - nbd_send_opt_abort(ioc);
> - return -1;
> +
> + trace_nbd_opt_meta_reply(name, received_id);
> + if (opt == NBD_OPT_SET_META_CONTEXT) {
> + if (received) {
> + error_setg(errp, "Server replied with more than one context");
> + free(name);
> + nbd_send_opt_abort(ioc);
> + return -1;
> + }
> +
> + if (strcmp(context, name)) {
> + error_setg(errp,
> + "Failed to negotiate meta context '%s', server "
> + "answered with different context '%s'", context,
> + name);
> + g_free(name);
> + nbd_send_opt_abort(ioc);
> + return -1;
> + }
> }
> g_free(name);
> -
> - trace_nbd_opt_meta_reply(context, received_id);
> received = true;
>
> /* receive NBD_REP_ACK */
> - if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
> - errp) < 0)
> - {
> + if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0) {
> return -1;
> }
>
> @@ -717,12 +733,11 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return -1;
> }
>
> - if (received) {
> - *context_id = received_id;
> - return 1;
> + if (received && opt == NBD_OPT_SET_META_CONTEXT) {
> + info->meta_base_allocation_id = received_id;
> }
>
> - return 0;
> + return received || opt == NBD_OPT_LIST_META_CONTEXT;
> }
>
> int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> @@ -822,9 +837,9 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
>
> if (info->structured_reply && base_allocation) {
> result = nbd_negotiate_simple_meta_context(
> - ioc, info->name,
> + ioc, NBD_OPT_SET_META_CONTEXT,
> info->x_dirty_bitmap ?: "base:allocation",
> - &info->meta_base_allocation_id, errp);
> + info, errp);
> if (result < 0) {
> goto fail;
> }
> diff --git a/nbd/trace-events b/nbd/trace-events
> index 289337d0dc3..5d0d202fad2 100644
> --- a/nbd/trace-events
> +++ b/nbd/trace-events
> @@ -10,7 +10,7 @@ nbd_receive_query_exports_start(const char *wantname) "Querying export list for
> nbd_receive_query_exports_success(const char *wantname) "Found desired export name '%s'"
> nbd_receive_starttls_new_client(void) "Setting up TLS"
> nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
> -nbd_opt_meta_request(const char *context, const char *export) "Requesting to set meta context %s for export %s"
> +nbd_opt_meta_request(const char *opt, const char *context, const char *export) "Requesting to %s %s for export %s"
> nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
> nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
> nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
> --
> 2.17.2
Tested-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 08/14] nbd/client: Refactor nbd_receive_list()
2018-11-30 22:03 ` [Qemu-devel] [PATCH 08/14] nbd/client: Refactor nbd_receive_list() Eric Blake
@ 2018-12-01 10:37 ` Richard W.M. Jones
2018-12-06 14:18 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Richard W.M. Jones @ 2018-12-01 10:37 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block
On Fri, Nov 30, 2018 at 04:03:37PM -0600, Eric Blake wrote:
> Add some parameters to make this function reusable in upcoming
> export listing, where we will want to capture the name and
> description rather than compare against a user-supplied name.
> No change in semantics to the existing caller.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> nbd/client.c | 66 +++++++++++++++++++++++++++++++++++++---------------
> 1 file changed, 47 insertions(+), 19 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 1dc8f83e19a..27785c55d0a 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -232,18 +232,21 @@ static int nbd_handle_reply_err(QIOChannel *ioc, NBDOptionReply *reply,
> return result;
> }
>
> -/* Process another portion of the NBD_OPT_LIST reply. Set *@match if
> - * the current reply matches @want or if the server does not support
> - * NBD_OPT_LIST, otherwise leave @match alone. Return 0 if iteration
> - * is complete, positive if more replies are expected, or negative
> - * with @errp set if an unrecoverable error occurred. */
> +/* Process another portion of the NBD_OPT_LIST reply. If @want, then
> + * set *@match if the current reply matches @want or if the server
> + * does not support NBD_OPT_LIST, otherwise leave @match alone.
> + * Otherwise, @nameout and @description are malloc'd to contain
> + * NUL-terminated copies of the reply. Return 0 if iteration is
> + * complete, positive if more replies are expected, or negative with
> + * @errp set if an unrecoverable error occurred. */
> static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> - Error **errp)
> + char **nameout, char **description, Error **errp)
> {
> NBDOptionReply reply;
> uint32_t len;
> uint32_t namelen;
> - char name[NBD_MAX_NAME_SIZE + 1];
> + char array[NBD_MAX_NAME_SIZE + 1];
> + char *name = array;
> int error;
>
> if (nbd_receive_option_reply(ioc, NBD_OPT_LIST, &reply, errp) < 0) {
> @@ -253,7 +256,12 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> if (error <= 0) {
> /* The server did not support NBD_OPT_LIST, so set *match on
> * the assumption that any name will be accepted. */
> - *match = true;
> + if (want) {
> + *match = true;
> + } else if (!error) {
> + error_setg(errp, "Server does not support export lists");
> + error = -1;
> + }
> return error;
> }
> len = reply.length;
> @@ -290,30 +298,49 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> nbd_send_opt_abort(ioc);
> return -1;
> }
> - if (namelen != strlen(want)) {
> - if (nbd_drop(ioc, len, errp) < 0) {
> - error_prepend(errp,
> - "failed to skip export name with wrong length: ");
> - nbd_send_opt_abort(ioc);
> - return -1;
> + if (want) {
> + if (namelen != strlen(want)) {
> + if (nbd_drop(ioc, len, errp) < 0) {
> + error_prepend(errp,
> + "failed to skip export name with wrong length: ");
> + nbd_send_opt_abort(ioc);
> + return -1;
> + }
> + return 1;
> }
> - return 1;
> + assert(namelen < sizeof(array));
> + } else {
> + assert(nameout);
> + *nameout = name = g_new(char, namelen + 1);
> }
>
> - assert(namelen < sizeof(name));
> if (nbd_read(ioc, name, namelen, errp) < 0) {
> error_prepend(errp, "failed to read export name: ");
> nbd_send_opt_abort(ioc);
> + if (!want) {
> + free(name);
> + }
> return -1;
> }
> name[namelen] = '\0';
> len -= namelen;
> - if (nbd_drop(ioc, len, errp) < 0) {
> + if (!want) {
> + assert(description);
> + *description = g_new(char, len + 1);
> + if (nbd_read(ioc, *description, len, errp) < 0) {
> + error_prepend(errp, "failed to read export description: ");
> + nbd_send_opt_abort(ioc);
> + free(name);
> + free(*description);
> + return -1;
> + }
> + (*description)[len] = '\0';
> + } else if (nbd_drop(ioc, len, errp) < 0) {
> error_prepend(errp, "failed to read export description: ");
> nbd_send_opt_abort(ioc);
> return -1;
> }
> - if (!strcmp(name, want)) {
> + if (want && !strcmp(name, want)) {
> *match = true;
> }
> return 1;
> @@ -498,7 +525,8 @@ static int nbd_receive_query_exports(QIOChannel *ioc,
> }
>
> while (1) {
> - int ret = nbd_receive_list(ioc, wantname, &foundExport, errp);
> + int ret = nbd_receive_list(ioc, wantname, &foundExport,
> + NULL, NULL, errp);
>
> if (ret < 0) {
> /* Server gave unexpected reply */
I found this patch a lot easier to review once I started to use the
‘git show -w’ option. The changes look like a reasonable adaptation
of the function, and the only consumer of the function (at this point)
is unaffected, so:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 10/14] nbd/client: Split handshake into two functions
2018-11-30 22:03 ` [Qemu-devel] [PATCH 10/14] nbd/client: Split handshake into two functions Eric Blake
@ 2018-12-01 10:41 ` Richard W.M. Jones
2018-12-06 15:16 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Richard W.M. Jones @ 2018-12-01 10:41 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block
On Fri, Nov 30, 2018 at 04:03:39PM -0600, Eric Blake wrote:
> An upcoming patch will add the ability for qemu-nbd to list
> the services provided by an NBD server. Share the common
> code of the TLS handshake by splitting the initial exchange
> into a separate function, leaving only the export handling
> in the original function. Functionally, there should be no
> change in behavior in this patch, although some of the code
> motion may be difficult to follow due to indentation changes
> (view with 'git diff -w' for a smaller changeset).
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> nbd/client.c | 142 ++++++++++++++++++++++++++++++-----------------
> nbd/trace-events | 2 +-
> 2 files changed, 92 insertions(+), 52 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 1ed5009642e..a282712724d 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -768,21 +768,22 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return received || opt == NBD_OPT_LIST_META_CONTEXT;
> }
>
> -int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> - const char *hostname, QIOChannel **outioc,
> - NBDExportInfo *info, Error **errp)
> +/* Start the handshake to the server. After a positive return, the server
> + * is ready to accept additional NBD_OPT requests.
> + * Returns: negative errno: failure talking to server
> + * 0: server is oldstyle, client must still parse export size
> + * 1: server is newstyle, but can only accept EXPORT_NAME
> + * 2: server is newstyle, but lacks structured replies
> + * 3: server is newstyle and set up for structured replies
> + */
> +static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> + const char *hostname, QIOChannel **outioc,
> + bool structured_reply, bool *zeroes,
> + Error **errp)
> {
> uint64_t magic;
> - bool zeroes = true;
> - bool structured_reply = info->structured_reply;
> - bool base_allocation = info->base_allocation;
>
> - trace_nbd_receive_negotiate(tlscreds, hostname ? hostname : "<null>");
> -
> - assert(info->name);
> - trace_nbd_receive_negotiate_name(info->name);
> - info->structured_reply = false;
> - info->base_allocation = false;
> + trace_nbd_start_negotiate(tlscreds, hostname ? hostname : "<null>");
>
> if (outioc) {
> *outioc = NULL;
> @@ -827,7 +828,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> clientflags |= NBD_FLAG_C_FIXED_NEWSTYLE;
> }
> if (globalflags & NBD_FLAG_NO_ZEROES) {
> - zeroes = false;
> + *zeroes = false;
> clientflags |= NBD_FLAG_C_NO_ZEROES;
> }
> /* client requested flags */
> @@ -849,7 +850,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> }
> }
> if (fixedNewStyle) {
> - int result;
> + int result = 0;
>
> if (structured_reply) {
> result = nbd_request_simple_option(ioc,
> @@ -858,42 +859,86 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> if (result < 0) {
> return -EINVAL;
> }
> - info->structured_reply = result == 1;
> }
> + return 2 + result;
> + } else {
> + return 1;
> + }
> + } else if (magic == NBD_CLIENT_MAGIC) {
> + if (tlscreds) {
> + error_setg(errp, "Server does not support STARTTLS");
> + return -EINVAL;
> + }
> + return 0;
> + } else {
> + error_setg(errp, "Bad magic received");
> + return -EINVAL;
> + }
> +}
>
> - if (info->structured_reply && base_allocation) {
> - result = nbd_negotiate_simple_meta_context(
> +/* Connect to server, complete negotiation, and move into transmission phase.
> + * Returns: negative errno: failure talking to server
> + * 0: server is connected
> + */
> +int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> + const char *hostname, QIOChannel **outioc,
> + NBDExportInfo *info, Error **errp)
> +{
> + int result;
> + bool zeroes = true;
> + bool base_allocation = info->base_allocation;
> + uint32_t oldflags;
> +
> + assert(info->name);
> + trace_nbd_receive_negotiate_name(info->name);
> +
> + result = nbd_start_negotiate(ioc, tlscreds, hostname, outioc,
> + info->structured_reply, &zeroes, errp);
> +
> + info->structured_reply = false;
> + info->base_allocation = false;
> + if (tlscreds && *outioc) {
> + ioc = *outioc;
> + }
> +
> + switch (result) {
> + case 3: /* newstyle, with structured replies */
> + info->structured_reply = true;
> + if (base_allocation) {
> + result = nbd_negotiate_simple_meta_context(
> ioc, NBD_OPT_SET_META_CONTEXT,
> info->x_dirty_bitmap ?: "base:allocation",
> info, errp);
> - if (result < 0) {
> - return -EINVAL;
> - }
> - info->base_allocation = result == 1;
> - }
> -
> - /* Try NBD_OPT_GO first - if it works, we are done (it
> - * also gives us a good message if the server requires
> - * TLS). If it is not available, fall back to
> - * NBD_OPT_LIST for nicer error messages about a missing
> - * export, then use NBD_OPT_EXPORT_NAME. */
> - result = nbd_opt_go(ioc, info, errp);
> if (result < 0) {
> return -EINVAL;
> }
> - if (result > 0) {
> - return 0;
> - }
> - /* Check our desired export is present in the
> - * server export list. Since NBD_OPT_EXPORT_NAME
> - * cannot return an error message, running this
> - * query gives us better error reporting if the
> - * export name is not available.
> - */
> - if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
> - return -EINVAL;
> - }
> + info->base_allocation = result == 1;
> }
> + /* fall through */
> + case 2: /* newstyle, try OPT_GO */
> + /* Try NBD_OPT_GO first - if it works, we are done (it
> + * also gives us a good message if the server requires
> + * TLS). If it is not available, fall back to
> + * NBD_OPT_LIST for nicer error messages about a missing
> + * export, then use NBD_OPT_EXPORT_NAME. */
> + result = nbd_opt_go(ioc, info, errp);
> + if (result < 0) {
> + return -EINVAL;
> + }
> + if (result > 0) {
> + return 0;
> + }
> + /* Check our desired export is present in the
> + * server export list. Since NBD_OPT_EXPORT_NAME
> + * cannot return an error message, running this
> + * query gives us better error reporting if the
> + * export name is not available.
> + */
> + if (nbd_receive_query_exports(ioc, info->name, errp) < 0) {
> + return -EINVAL;
> + }
> + /* fall through */
> + case 1: /* newstyle, but limited to EXPORT_NAME */
> /* write the export name request */
> if (nbd_send_option_request(ioc, NBD_OPT_EXPORT_NAME, -1, info->name,
> errp) < 0) {
> @@ -912,17 +957,12 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> return -EINVAL;
> }
> info->flags = be16_to_cpu(info->flags);
> - } else if (magic == NBD_CLIENT_MAGIC) {
> - uint32_t oldflags;
> -
> + break;
> + case 0: /* oldstyle, parse length and flags */
> if (*info->name) {
> error_setg(errp, "Server does not support non-empty export names");
> return -EINVAL;
> }
> - if (tlscreds) {
> - error_setg(errp, "Server does not support STARTTLS");
> - return -EINVAL;
> - }
>
> if (nbd_read(ioc, &info->size, sizeof(info->size), errp) < 0) {
> error_prepend(errp, "Failed to read export length: ");
> @@ -940,9 +980,9 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> return -EINVAL;
> }
> info->flags = oldflags;
> - } else {
> - error_setg(errp, "Bad magic received");
> - return -EINVAL;
> + break;
> + default:
> + return result;
> }
>
> trace_nbd_receive_negotiate_size_flags(info->size, info->flags);
> diff --git a/nbd/trace-events b/nbd/trace-events
> index 5d0d202fad2..570b04997ff 100644
> --- a/nbd/trace-events
> +++ b/nbd/trace-events
> @@ -12,7 +12,7 @@ nbd_receive_starttls_new_client(void) "Setting up TLS"
> nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
> nbd_opt_meta_request(const char *opt, const char *context, const char *export) "Requesting to %s %s for export %s"
> nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
> -nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
> +nbd_start_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
> nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
> nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are 0x%" PRIx32
> nbd_receive_negotiate_name(const char *name) "Requesting NBD export name \"%s\""
Pretty much a straight splitting out of the nbd_start_negotiate
feature into a separate function. The only tricky bit is the return
code between the two functions, but the codes are amply documented.
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 11/14] nbd/client: Add nbd_receive_export_list()
2018-11-30 22:03 ` [Qemu-devel] [PATCH 11/14] nbd/client: Add nbd_receive_export_list() Eric Blake
@ 2018-12-01 10:45 ` Richard W.M. Jones
2018-12-07 10:04 ` Vladimir Sementsov-Ogievskiy
2018-12-07 10:07 ` Vladimir Sementsov-Ogievskiy
2 siblings, 0 replies; 65+ messages in thread
From: Richard W.M. Jones @ 2018-12-01 10:45 UTC (permalink / raw)
To: Eric Blake
Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block, Kevin Wolf,
Max Reitz
On Fri, Nov 30, 2018 at 04:03:40PM -0600, Eric Blake wrote:
> We want to be able to detect whether a given qemu NBD server is
> exposing the right export(s) and dirty bitmaps, at least for
> regression testing. We could use 'nbd-client -l' from the upstream
> NBD project to list exports, but it's annoying to rely on
> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
> know about all of the qemu NBD extensions. Thus, we plan on adding
> a new mode to qemu-nbd that merely sniffs all possible information
> from the server during handshake phase, then disconnects and dumps
> the information.
>
> This patch adds the low-level client code for grabbing the list
> of exports. It benefits from the recent refactoring patches, as
> well as a minor tweak of changing nbd_opt_go() to nbd_opt_info_go(),
I would probably have called the new function
‘nbd_opt_info_or_opt_go’, but then I prefer longer descriptive names.
> in order to share as much code as possible when it comes to doing
> validation of server replies. The resulting information is stored
> in an array of NBDExportInfo which has been expanded to hold more
> members, along with a convenience function for freeing the list.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> include/block/nbd.h | 15 +++-
> nbd/client.c | 166 ++++++++++++++++++++++++++++++++++++++++++--
> nbd/trace-events | 2 +-
> 3 files changed, 174 insertions(+), 9 deletions(-)
>
> diff --git a/include/block/nbd.h b/include/block/nbd.h
> index 65feff8ba96..74d006b8d62 100644
> --- a/include/block/nbd.h
> +++ b/include/block/nbd.h
> @@ -262,6 +262,9 @@ struct NBDExportInfo {
> /* Set by client before nbd_receive_negotiate() */
> bool request_sizes;
> char *x_dirty_bitmap;
> +
> + /* Set by client before nbd_receive_negotiate(), or by server results
> + * during nbd_receive_export_list() */
> char *name; /* must be non-NULL */
>
> /* In-out fields, set by client before nbd_receive_negotiate() and
> @@ -269,7 +272,8 @@ struct NBDExportInfo {
> bool structured_reply;
> bool base_allocation; /* base:allocation context for NBD_CMD_BLOCK_STATUS */
>
> - /* Set by server results during nbd_receive_negotiate() */
> + /* Set by server results during nbd_receive_negotiate() and
> + * nbd_receive_export_list() */
> uint64_t size;
> uint16_t flags;
> uint32_t min_block;
> @@ -277,12 +281,21 @@ struct NBDExportInfo {
> uint32_t max_block;
>
> uint32_t meta_base_allocation_id;
> +
> + /* Set by server results during nbd_receive_export_list() */
> + char *description;
> + int n_contexts;
> + char **contexts;
> };
> typedef struct NBDExportInfo NBDExportInfo;
>
> int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> const char *hostname, QIOChannel **outioc,
> NBDExportInfo *info, Error **errp);
> +void nbd_free_export_list(NBDExportInfo *info, int count);
> +int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> + const char *hostname, NBDExportInfo **info,
> + Error **errp);
> int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
> Error **errp);
> int nbd_send_request(QIOChannel *ioc, NBDRequest *request);
> diff --git a/nbd/client.c b/nbd/client.c
> index a282712724d..6292de560ee 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -351,7 +351,8 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> * used, 0 if NBD_OPT_GO is unsupported (fall back to NBD_OPT_LIST and
> * NBD_OPT_EXPORT_NAME in that case), and > 0 if the export is good to
> * go (with the rest of @info populated). */
> -static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
> +static int nbd_opt_info_go(QIOChannel *ioc, uint32_t opt,
> + NBDExportInfo *info, Error **errp)
> {
> NBDOptionReply reply;
> uint32_t len = strlen(info->name);
> @@ -364,7 +365,8 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
> * flags still 0 is a witness of a broken server. */
> info->flags = 0;
>
> - trace_nbd_opt_go_start(info->name);
> + assert(opt == NBD_OPT_GO || opt == NBD_OPT_INFO);
> + trace_nbd_opt_go_start(nbd_opt_lookup(opt), info->name);
> buf = g_malloc(4 + len + 2 + 2 * info->request_sizes + 1);
> stl_be_p(buf, len);
> memcpy(buf + 4, info->name, len);
> @@ -373,7 +375,7 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
> if (info->request_sizes) {
> stw_be_p(buf + 4 + len + 2, NBD_INFO_BLOCK_SIZE);
> }
> - error = nbd_send_option_request(ioc, NBD_OPT_GO,
> + error = nbd_send_option_request(ioc, opt,
> 4 + len + 2 + 2 * info->request_sizes,
> buf, errp);
> g_free(buf);
> @@ -382,7 +384,7 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
> }
>
> while (1) {
> - if (nbd_receive_option_reply(ioc, NBD_OPT_GO, &reply, errp) < 0) {
> + if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0) {
> return -1;
> }
> error = nbd_handle_reply_err(ioc, &reply, errp);
> @@ -733,8 +735,12 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> nbd_send_opt_abort(ioc);
> return -1;
> }
> + g_free(name);
> + } else {
> + info->contexts = g_renew(char *, info->contexts,
> + ++info->n_contexts);
> + info->contexts[info->n_contexts - 1] = name;
> }
> - g_free(name);
> received = true;
>
> /* receive NBD_REP_ACK */
> @@ -828,7 +834,9 @@ static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> clientflags |= NBD_FLAG_C_FIXED_NEWSTYLE;
> }
> if (globalflags & NBD_FLAG_NO_ZEROES) {
> - *zeroes = false;
> + if (zeroes) {
> + *zeroes = false;
> + }
> clientflags |= NBD_FLAG_C_NO_ZEROES;
> }
> /* client requested flags */
> @@ -921,7 +929,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> * TLS). If it is not available, fall back to
> * NBD_OPT_LIST for nicer error messages about a missing
> * export, then use NBD_OPT_EXPORT_NAME. */
> - result = nbd_opt_go(ioc, info, errp);
> + result = nbd_opt_info_go(ioc, NBD_OPT_GO, info, errp);
> if (result < 0) {
> return -EINVAL;
> }
> @@ -993,6 +1001,150 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> return 0;
> }
>
> +/* Clean up result of nbd_receive_export_list */
> +void nbd_free_export_list(NBDExportInfo *info, int count)
> +{
> + int i, j;
> +
> + for (i = 0; info && i < count; i++) {
> + free(info[i].name);
> + free(info[i].description);
> + for (j = 0; j < info[i].n_contexts; j++) {
> + free(info[i].contexts[j]);
> + }
> + free(info[i].contexts);
> + }
> + free(info);
> +}
> +
> +/* Query details about a server's exports, then disconnect without
> + * going into transmission phase. Return a count of the exports listed
> + * in @info by the server, or -1 on error. Caller must free @info. */
> +int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> + const char *hostname, NBDExportInfo **info,
> + Error **errp)
> +{
> + int result;
> + int count = 0;
> + int i;
> + int rc;
> + int ret = -1;
> + NBDExportInfo *array = NULL;
> + uint32_t oldflags;
> + QIOChannel *sioc = NULL;
> + bool try_context = true;
> +
> + *info = NULL;
> + result = nbd_start_negotiate(ioc, tlscreds, hostname, &sioc, true, NULL,
> + errp);
> + if (tlscreds && sioc) {
> + ioc = sioc;
> + }
> +
> + switch (result) {
> + case 2:
> + /* meta contexts are only useful with structured reply */
> + try_context = false;
> + /* fall through */
> + case 3:
> + /* newstyle - use NBD_OPT_LIST to populate array, then try
> + * NBD_OPT_INFO on each array member. If structured replies
> + * are enabled, also try NBD_OPT_LIST_META_CONTEXT. */
> + if (nbd_send_option_request(ioc, NBD_OPT_LIST, 0, NULL, errp) < 0) {
> + goto out;
> + }
> + while (1) {
> + char *name;
> + char *desc;
> +
> + rc = nbd_receive_list(ioc, NULL, NULL, &name, &desc, errp);
> + if (rc < 0) {
> + goto out;
> + } else if (rc == 0) {
> + break;
> + }
> + array = g_renew(NBDExportInfo, array, ++count);
> + memset(&array[count - 1], 0, sizeof(*array));
> + array[count - 1].name = name;
> + array[count - 1].description = desc;
> + array[count - 1].structured_reply = result == 3;
> + }
> +
> + for (i = 0; i < count; i++) {
> + array[i].request_sizes = true;
> + rc = nbd_opt_info_go(ioc, NBD_OPT_INFO, &array[i], errp);
> + if (rc < 0) {
> + goto out;
> + } else if (rc == 0) {
> + /* Pointless to try rest of loop. If OPT_INFO doesn't work,
> + * it's unlikely that meta contexts work either */
> + break;
> + }
> +
> + if (try_context) {
> + rc = nbd_negotiate_simple_meta_context(
> + ioc, NBD_OPT_LIST_META_CONTEXT, NULL, &array[i], errp);
> + if (rc < 0) {
> + goto out;
> + } else if (rc == 0) {
> + try_context = false;
> + }
> + }
> + }
> +
> + /* Send NBD_OPT_ABORT as a courtesy before hanging up */
> + nbd_send_opt_abort(ioc);
> + break;
> + case 1: /* newstyle, but limited to EXPORT_NAME */
> + error_setg(errp, "Server does not support export lists");
> + /* We can't even send NBD_OPT_ABORT, so merely hang up */
> + goto out;
> + case 0: /* oldstyle, parse length and flags */
> + array = g_new0(NBDExportInfo, 1);
> + array->name = g_strdup("");
> + count = 1;
> +
> + if (nbd_read(ioc, &array->size, sizeof(array->size), errp) < 0) {
> + error_prepend(errp, "Failed to read export length: ");
> + goto out;
> + }
> + array->size = be64_to_cpu(array->size);
> +
> + if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
> + error_prepend(errp, "Failed to read export flags: ");
> + goto out;
> + }
> + oldflags = be32_to_cpu(oldflags);
> + if (oldflags & ~0xffff) {
> + error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
> + goto out;
> + }
> + array->flags = oldflags;
> +
> + /* Send NBD_CMD_DISC as a courtesy to the server, but ignore all
> + * errors now that we have the information we wanted. */
> + if (nbd_drop(ioc, 124, NULL) == 0) {
> + NBDRequest request = { .type = NBD_CMD_DISC };
> +
> + nbd_send_request(ioc, &request);
> + }
> + break;
> + default:
> + goto out;
> + }
> +
> + *info = array;
> + array = NULL;
> + ret = count;
> +
> + out:
> + qio_channel_shutdown(ioc, QIO_CHANNEL_SHUTDOWN_BOTH, NULL);
> + qio_channel_close(ioc, NULL);
> + object_unref(OBJECT(sioc));
> + nbd_free_export_list(array, count);
> + return ret;
> +}
> +
> #ifdef __linux__
> int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
> Error **errp)
> diff --git a/nbd/trace-events b/nbd/trace-events
> index 570b04997ff..2cf83ebed15 100644
> --- a/nbd/trace-events
> +++ b/nbd/trace-events
> @@ -2,7 +2,7 @@
> nbd_send_option_request(uint32_t opt, const char *name, uint32_t len) "Sending option request %" PRIu32" (%s), len %" PRIu32
> nbd_receive_option_reply(uint32_t option, const char *optname, uint32_t type, const char *typename, uint32_t length) "Received option reply %" PRIu32" (%s), type %" PRIu32" (%s), len %" PRIu32
> nbd_reply_err_unsup(uint32_t option, const char *name) "server doesn't understand request %" PRIu32 " (%s), attempting fallback"
> -nbd_opt_go_start(const char *name) "Attempting NBD_OPT_GO for export '%s'"
> +nbd_opt_go_start(const char *opt, const char *name) "Attempting %s for export '%s'"
> nbd_opt_go_success(void) "Export is good to go"
> nbd_opt_go_info_unknown(int info, const char *name) "Ignoring unknown info %d (%s)"
> nbd_opt_go_info_block_size(uint32_t minimum, uint32_t preferred, uint32_t maximum) "Block sizes are 0x%" PRIx32 ", 0x%" PRIx32 ", 0x%" PRIx32
> --
> 2.17.2
This is all straightforward, so:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 13/14] qemu-nbd: Add --list option
2018-11-30 22:03 ` [Qemu-devel] [PATCH 13/14] qemu-nbd: Add --list option Eric Blake
@ 2018-12-01 10:58 ` Richard W.M. Jones
2018-12-07 12:48 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Richard W.M. Jones @ 2018-12-01 10:58 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block
On Fri, Nov 30, 2018 at 04:03:42PM -0600, Eric Blake wrote:
> We want to be able to detect whether a given qemu NBD server is
> exposing the right export(s) and dirty bitmaps, at least for
> regression testing. We could use 'nbd-client -l' from the upstream
> NBD project to list exports, but it's annoying to rely on
> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
> know about all of the qemu NBD extensions. Thus, we plan on adding
> a new mode to qemu-nbd that merely sniffs all possible information
> from the server during handshake phase, then disconnects and dumps
> the information.
>
> This patch actually implements --list/-L, while reusing other
> options such as --tls-creds for now designating how to connect
> as the client (rather than their non-list usage of how to operate
> as the server).
>
> I debated about adding this functionality to something akin to
> 'qemu-img info' - but that tool does not readily lend itself
> to connecting to an arbitrary NBD server without also tying to
> a specific export (I may, however, still add ImageInfoSpecificNBD
> for reporting the bitmaps available when connecting to a single
> export). And, while it may feel a bit odd that normally
> qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not
> really making the qemu-nbd binary that much larger, because
> 'qemu-nbd -c' has to operate as both server and client
> simultaneously across two threads when feeding the kernel module
> for /dev/nbdN access.
>
> Sample output:
> $ qemu-nbd -L
> exports available: 1
> export: ''
> size: 65536
> flags: 0x4ed ( flush fua trim zeroes df cache )
> min block: 512
> opt block: 4096
> max block: 33554432
> available meta contexts: 1
> base:allocation
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> qemu-nbd.c | 153 ++++++++++++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 141 insertions(+), 12 deletions(-)
>
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index c57053a0795..e19a841b869 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -76,6 +76,7 @@ static void usage(const char *name)
> {
> (printf) (
> "Usage: %s [OPTIONS] FILE\n"
> +" or: %s -L [OPTIONS]\n"
> "QEMU Disk Network Block Device Server\n"
> "\n"
> " -h, --help display this help and exit\n"
> @@ -97,6 +98,7 @@ static void usage(const char *name)
> " -P, --partition=NUM only expose partition NUM\n"
> "\n"
> "General purpose options:\n"
> +" -L, --list list NBD exports visible to client\n"
> " --object type,id=ID,... define an object such as 'secret' for providing\n"
> " passwords and/or encryption keys\n"
> " --tls-creds=ID use id of an earlier --object to provide TLS\n"
> @@ -130,7 +132,7 @@ static void usage(const char *name)
> " --image-opts treat FILE as a full set of image options\n"
> "\n"
> QEMU_HELP_BOTTOM "\n"
> - , name, NBD_DEFAULT_PORT, "DEVICE");
> + , name, name, NBD_DEFAULT_PORT, "DEVICE");
> }
>
> static void version(const char *name)
> @@ -242,6 +244,92 @@ static void termsig_handler(int signum)
> }
>
>
> +static int qemu_nbd_client_list(SocketAddress *saddr, QCryptoTLSCreds *tls,
> + const char *hostname)
> +{
> + int ret = EXIT_FAILURE;
> + int rc;
> + Error *err = NULL;
> + QIOChannelSocket *sioc;
> + NBDExportInfo *list;
> + int i, j;
> +
> + sioc = qio_channel_socket_new();
> + if (qio_channel_socket_connect_sync(sioc, saddr, &err) < 0) {
> + error_report_err(err);
> + goto out;
> + }
> + rc = nbd_receive_export_list(QIO_CHANNEL(sioc), tls, hostname, &list,
> + &err);
> + if (rc < 0) {
> + if (err) {
> + error_report_err(err);
> + }
> + goto out_socket;
> + }
> + printf("exports available: %d\n", rc);
> + for (i = 0; i < rc; i++) {
> + printf(" export: '%s'\n", list[i].name);
> + if (list[i].description && *list[i].description) {
> + printf(" description: %s\n", list[i].description);
> + }
> + if (list[i].flags & NBD_FLAG_HAS_FLAGS) {
> + printf(" size: %" PRIu64 "\n", list[i].size);
> + printf(" flags: 0x%x (", list[i].flags);
> + if (list[i].flags & NBD_FLAG_READ_ONLY) {
> + printf(" readonly");
> + }
This adds an extra space in the output unconditionally, if that's
a problem, ie. the output looks like:
flags: 0x61 ( trim zeroes )
instead of:
flags: 0x61 (trim zeroes)
if that's a problem.
I might have used a loop with a static array of flag names, like:
const char *nbd_flag_name[] = {
...
.5 = "trim",
.6 = "zeroes",
};
although it's a bit awkward given the current definitions in
include/block/nbd.h.
Anyway not a big issue.
> + if (list[i].flags & NBD_FLAG_SEND_FLUSH) {
> + printf(" flush");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_FUA) {
> + printf(" fua");
> + }
> + if (list[i].flags & NBD_FLAG_ROTATIONAL) {
> + printf(" rotational");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_TRIM) {
> + printf(" trim");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_WRITE_ZEROES) {
> + printf(" zeroes");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_DF) {
> + printf(" df");
> + }
> + if (list[i].flags & NBD_FLAG_CAN_MULTI_CONN) {
> + printf(" multi");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_RESIZE) {
> + printf(" resize");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_CACHE) {
> + printf(" cache");
> + }
> + printf(" )\n");
> + }
> + if (list[i].min_block) {
> + printf(" min block: %u\n", list[i].min_block);
> + printf(" opt block: %u\n", list[i].opt_block);
> + printf(" max block: %u\n", list[i].max_block);
> + }
> + if (list[i].n_contexts) {
> + printf(" available meta contexts: %d\n", list[i].n_contexts);
> + for (j = 0; j < list[i].n_contexts; j++) {
> + printf(" %s\n", list[i].contexts[j]);
> + }
> + }
> + }
> + nbd_free_export_list(list, rc);
> +
> + ret = EXIT_SUCCESS;
> + out_socket:
> + object_unref(OBJECT(sioc));
> + out:
> + return ret;
> +}
> +
> +
> #if HAVE_NBD_DEVICE
> static void *show_parts(void *arg)
> {
> @@ -424,7 +512,8 @@ static QemuOptsList qemu_object_opts = {
>
>
>
> -static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, Error **errp)
> +static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, bool list,
> + Error **errp)
> {
> Object *obj;
> QCryptoTLSCreds *creds;
> @@ -444,10 +533,18 @@ static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, Error **errp)
> return NULL;
> }
>
> - if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_SERVER) {
> - error_setg(errp,
> - "Expecting TLS credentials with a server endpoint");
> - return NULL;
> + if (list) {
> + if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT) {
> + error_setg(errp,
> + "Expecting TLS credentials with a client endpoint");
> + return NULL;
> + }
> + } else {
> + if (creds->endpoint != QCRYPTO_TLS_CREDS_ENDPOINT_SERVER) {
> + error_setg(errp,
> + "Expecting TLS credentials with a server endpoint");
> + return NULL;
> + }
> }
> object_ref(obj);
> return creds;
> @@ -470,7 +567,8 @@ static void setup_address_and_port(const char **address, const char **port)
> static const char *socket_activation_validate_opts(const char *device,
> const char *sockpath,
> const char *address,
> - const char *port)
> + const char *port,
> + bool list)
> {
> if (device != NULL) {
> return "NBD device can't be set when using socket activation";
> @@ -488,6 +586,10 @@ static const char *socket_activation_validate_opts(const char *device,
> return "TCP port number can't be set when using socket activation";
> }
>
> + if (list) {
> + return "List mode is incompatible with socket activation";
> + }
> +
> return NULL;
> }
>
> @@ -511,7 +613,7 @@ int main(int argc, char **argv)
> off_t fd_size;
> QemuOpts *sn_opts = NULL;
> const char *sn_id_or_name = NULL;
> - const char *sopt = "hVb:o:p:rsnP:c:dvk:e:f:tl:x:T:D:";
> + const char *sopt = "hVb:o:p:rsnP:c:dvk:e:f:tl:x:T:D:L";
> struct option lopt[] = {
> { "help", no_argument, NULL, 'h' },
> { "version", no_argument, NULL, 'V' },
> @@ -523,6 +625,7 @@ int main(int argc, char **argv)
> { "partition", required_argument, NULL, 'P' },
> { "connect", required_argument, NULL, 'c' },
> { "disconnect", no_argument, NULL, 'd' },
> + { "list", no_argument, NULL, 'L' },
> { "snapshot", no_argument, NULL, 's' },
> { "load-snapshot", required_argument, NULL, 'l' },
> { "nocache", no_argument, NULL, 'n' },
> @@ -558,13 +661,14 @@ int main(int argc, char **argv)
> Error *local_err = NULL;
> BlockdevDetectZeroesOptions detect_zeroes = BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF;
> QDict *options = NULL;
> - const char *export_name = ""; /* Default export name */
> + const char *export_name = NULL;
> const char *export_description = NULL;
> const char *tlscredsid = NULL;
> bool imageOpts = false;
> bool writethrough = true;
> char *trace_file = NULL;
> bool fork_process = false;
> + bool list = false;
> int old_stderr = -1;
> unsigned socket_activation;
>
> @@ -764,13 +868,32 @@ int main(int argc, char **argv)
> case QEMU_NBD_OPT_FORK:
> fork_process = true;
> break;
> + case 'L':
> + list = true;
> + break;
> }
> }
>
> - if ((argc - optind) != 1) {
> + if (list) {
> + if (argc != optind) {
> + error_report("List mode is incompatible with a file name");
> + exit(EXIT_FAILURE);
> + }
> + if (export_name || export_description || dev_offset || partition ||
> + device || disconnect || fmt || sn_id_or_name) {
> + error_report("List mode is incompatible with per-device settings");
> + exit(EXIT_FAILURE);
> + }
> + if (fork_process) {
> + error_report("List mode is incompatible with forking");
> + exit(EXIT_FAILURE);
> + }
> + } else if ((argc - optind) != 1) {
> error_report("Invalid number of arguments");
> error_printf("Try `%s --help' for more information.\n", argv[0]);
> exit(EXIT_FAILURE);
> + } else if (!export_name) {
> + export_name = "";
> }
>
> qemu_opts_foreach(&qemu_object_opts,
> @@ -789,7 +912,8 @@ int main(int argc, char **argv)
> } else {
> /* Using socket activation - check user didn't use -p etc. */
> const char *err_msg = socket_activation_validate_opts(device, sockpath,
> - bindto, port);
> + bindto, port,
> + list);
> if (err_msg != NULL) {
> error_report("%s", err_msg);
> exit(EXIT_FAILURE);
> @@ -812,7 +936,7 @@ int main(int argc, char **argv)
> error_report("TLS is not supported with a host device");
> exit(EXIT_FAILURE);
> }
> - tlscreds = nbd_get_tls_creds(tlscredsid, &local_err);
> + tlscreds = nbd_get_tls_creds(tlscredsid, list, &local_err);
> if (local_err) {
> error_report("Failed to get TLS creds %s",
> error_get_pretty(local_err));
> @@ -820,6 +944,11 @@ int main(int argc, char **argv)
> }
> }
>
> + if (list) {
> + saddr = nbd_build_socket_address(sockpath, bindto, port);
> + return qemu_nbd_client_list(saddr, tlscreds, bindto);
> + }
> +
> if (disconnect) {
> #if HAVE_NBD_DEVICE
> int nbdfd = open(argv[optind], O_RDWR);
The code is a bit awkward since we've now effectively added a new mode
to qemu-nbd.
I can foresee in future that someone will add some code before the
‘if (list) / if (disconnect)’ statements which will apply only to
"server mode", and they won't necessarily know that they have to add a
mode check to that code, resulting in the list option doing something
weird that might not show up as broken in unit tests.
Still, I don't have any idea how to make it better, for all the
reasons you outlined in your commit message. I'll leave it to others
who know qemu code in greater detail to comment.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 14/14] iotests: Enhance 223, 233 to cover 'qemu-nbd --list'
2018-11-30 22:03 ` [Qemu-devel] [PATCH 14/14] iotests: Enhance 223, 233 to cover 'qemu-nbd --list' Eric Blake
@ 2018-12-01 11:04 ` Richard W.M. Jones
2018-12-07 13:08 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Richard W.M. Jones @ 2018-12-01 11:04 UTC (permalink / raw)
To: Eric Blake
Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block, Kevin Wolf,
Max Reitz
On Fri, Nov 30, 2018 at 04:03:43PM -0600, Eric Blake wrote:
> Any good new feature deserves some regression testing :)
> Coverage includes:
> - 223: what happens when there are 0 or more than 1 export,
> proof that we can see multiple contexts including qemu:dirty-bitmap
> - 233: proof that we can list over TLS, and that mix-and-match of
> plain/TLS listings sanely
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> tests/qemu-iotests/223 | 2 ++
> tests/qemu-iotests/223.out | 20 ++++++++++++++++++++
> tests/qemu-iotests/233 | 10 ++++++++++
> tests/qemu-iotests/233.out | 15 +++++++++++++++
> 4 files changed, 47 insertions(+)
>
> diff --git a/tests/qemu-iotests/223 b/tests/qemu-iotests/223
> index 397b865d347..e64747a9a61 100755
> --- a/tests/qemu-iotests/223
> +++ b/tests/qemu-iotests/223
> @@ -119,6 +119,7 @@ _send_qemu_cmd $QEMU_HANDLE '{"execute":"x-block-dirty-bitmap-disable",
> _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-start",
> "arguments":{"addr":{"type":"unix",
> "data":{"path":"'"$TEST_DIR/nbd"'"}}}}' "return"
> +$QEMU_NBD_PROG -L -k "$TEST_DIR/nbd"
> _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add",
> "arguments":{"device":"n"}}' "return"
> _send_qemu_cmd $QEMU_HANDLE '{"execute":"x-nbd-server-add-bitmap",
> @@ -127,6 +128,7 @@ _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add",
> "arguments":{"device":"n", "name":"n2"}}' "return"
> _send_qemu_cmd $QEMU_HANDLE '{"execute":"x-nbd-server-add-bitmap",
> "arguments":{"name":"n2", "bitmap":"b2"}}' "return"
> +$QEMU_NBD_PROG -L -k "$TEST_DIR/nbd"
>
> echo
> echo "=== Contrast normal status to large granularity dirty-bitmap ==="
> diff --git a/tests/qemu-iotests/223.out b/tests/qemu-iotests/223.out
> index de417477de0..3342bff3447 100644
> --- a/tests/qemu-iotests/223.out
> +++ b/tests/qemu-iotests/223.out
> @@ -29,10 +29,30 @@ wrote 2097152/2097152 bytes at offset 2097152
> {"return": {}}
> {"return": {}}
> {"return": {}}
> +exports available: 0
> {"return": {}}
> {"return": {}}
> {"return": {}}
> {"return": {}}
> +exports available: 2
> + export: 'n'
> + size: 4194304
> + flags: 0x4ef ( readonly flush fua trim zeroes df cache )
> + min block: 512
> + opt block: 4096
> + max block: 33554432
> + available meta contexts: 2
> + base:allocation
> + qemu:dirty-bitmap:b
> + export: 'n2'
> + size: 4194304
> + flags: 0x4ef ( readonly flush fua trim zeroes df cache )
> + min block: 512
> + opt block: 4096
> + max block: 33554432
> + available meta contexts: 2
> + base:allocation
> + qemu:dirty-bitmap:b2
>
> === Contrast normal status to large granularity dirty-bitmap ===
>
> diff --git a/tests/qemu-iotests/233 b/tests/qemu-iotests/233
> index 1814efe3333..5d694d9d242 100755
> --- a/tests/qemu-iotests/233
> +++ b/tests/qemu-iotests/233
> @@ -72,6 +72,9 @@ $QEMU_IMG info --image-opts \
> --object tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0 \
> driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
> 2>&1 | sed "s/$nbd_tcp_port/PORT/g"
> +$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port \
> + --object tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0 \
> + --tls-creds=tls0
>
> nbd_server_stop
>
> @@ -84,6 +87,7 @@ nbd_server_start_tcp_socket \
> -f $IMGFMT "$TEST_IMG"
>
> $QEMU_IMG info nbd://localhost:$nbd_tcp_port 2>&1 | sed "s/$nbd_tcp_port/PORT/g"
> +$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port
>
> echo
> echo "== check TLS works =="
> @@ -91,6 +95,9 @@ $QEMU_IMG info --image-opts \
> --object tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0 \
> driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
> 2>&1 | sed "s/$nbd_tcp_port/PORT/g"
> +$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port \
> + --object tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0 \
> + --tls-creds=tls0
>
> echo
> echo "== check TLS with different CA fails =="
> @@ -98,6 +105,9 @@ $QEMU_IMG info --image-opts \
> --object tls-creds-x509,dir=${tls_dir}/client2,endpoint=client,id=tls0 \
> driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
> 2>&1 | sed "s/$nbd_tcp_port/PORT/g"
> +$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port \
> + --object tls-creds-x509,dir=${tls_dir}/client2,endpoint=client,id=tls0 \
> + --tls-creds=tls0
>
> echo
> echo "== perform I/O over TLS =="
> diff --git a/tests/qemu-iotests/233.out b/tests/qemu-iotests/233.out
> index 5f416721b03..ab669488669 100644
> --- a/tests/qemu-iotests/233.out
> +++ b/tests/qemu-iotests/233.out
> @@ -15,20 +15,35 @@ wrote 1048576/1048576 bytes at offset 1048576
> == check TLS client to plain server fails ==
> qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': Denied by server for option 5 (starttls)
> server reported: TLS not configured
> +qemu-nbd: Denied by server for option 5 (starttls)
> +server reported: TLS not configured
>
> == check plain client to TLS server fails ==
> qemu-img: Could not open 'nbd://localhost:PORT': TLS negotiation required before option 8 (structured reply)
> server reported: Option 0x8 not permitted before TLS
> +qemu-nbd: TLS negotiation required before option 8 (structured reply)
> +server reported: Option 0x8 not permitted before TLS
>
> == check TLS works ==
> image: nbd://127.0.0.1:PORT
> file format: nbd
> virtual size: 64M (67108864 bytes)
> disk size: unavailable
> +exports available: 1
> + export: ''
> + size: 67108864
> + flags: 0x4ed ( flush fua trim zeroes df cache )
> + min block: 512
> + opt block: 4096
> + max block: 33554432
> + available meta contexts: 1
> + base:allocation
>
> == check TLS with different CA fails ==
> qemu-nbd: option negotiation failed: Verify failed: No certificate was found.
> qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': The certificate hasn't got a known issuer
> +qemu-nbd: option negotiation failed: Verify failed: No certificate was found.
> +qemu-nbd: The certificate hasn't got a known issuer
>
> == perform I/O over TLS ==
> read 1048576/1048576 bytes at offset 1048576
> --
> 2.17.2
Tests look good, and I ran them too:
QEMU -- "/home/rjones/d/qemu/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64" -nodefaults -machine accel=qtest
QEMU_IMG -- "/home/rjones/d/qemu/tests/qemu-iotests/../../qemu-img"
QEMU_IO -- "/home/rjones/d/qemu/tests/qemu-iotests/../../qemu-io" --cache writeback -f qcow2
QEMU_NBD -- "/home/rjones/d/qemu/tests/qemu-iotests/../../qemu-nbd"
IMGFMT -- qcow2 (compat=1.1)
IMGPROTO -- file
PLATFORM -- Linux/x86_64 moo 4.18.18-300.fc29.x86_64
TEST_DIR -- /home/rjones/d/qemu/tests/qemu-iotests/scratch
SOCKET_SCM_HELPER -- /home/rjones/d/qemu/tests/qemu-iotests/socket_scm_helper
223
233
Passed all 2 tests
So:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list
2018-12-01 7:42 ` [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Richard W.M. Jones
@ 2018-12-01 13:57 ` Eric Blake
2018-12-01 15:00 ` Richard W.M. Jones
0 siblings, 1 reply; 65+ messages in thread
From: Eric Blake @ 2018-12-01 13:57 UTC (permalink / raw)
To: Richard W.M. Jones; +Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block
On 12/1/18 1:42 AM, Richard W.M. Jones wrote:
> On Fri, Nov 30, 2018 at 04:03:29PM -0600, Eric Blake wrote:
>> I note that upstream NBD has 'nbd-client -l $host' for querying
>> just export names (with no quoting, so you have to know that
>> a blank line means the default export), but it wasn't powerful
>> enough, so I implemented 'qemu-nbd -L' to document everything.
>> Upstream NBD has separate 'nbd-client' and 'nbd-server' binaries,
>> while we only have 'qemu-nbd' (which is normally just a server,
>> but 'qemu-nbd -c' also operates a second thread as a client).
>> Our other uses of qemu as NBD client are for consuming a block
>> device (as in qemu-io, qemu-img, or a drive to qemu) - but those
>> binaries are less suited to something so specific to the NBD
>> protocol.
>
> I tried it against nbdkit and it works (obviously):
>
> $ ./qemu-nbd -L
> exports available: 1
> export: ''
> size: 67108864
> flags: 0x61 ( trim zeroes )
>
> What I couldn't work out is how to connect to other hosts. It's
> possible to add -p or -k to change the localhost port number or to use
> a Unix domain socket (and I checked both work). However can we
> connect to remote hosts?
Should work by adding '-b host' (if -b is omitted, it defaults to
0.0.0.0). However, I admit that so far I have only tested '-b
localhost' or '-b 127.0.0.1' (per the iotest additions at the end of the
series), so it could well be something that I missed in setting up the
client socket in nbd_build_socket_address().
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list
2018-12-01 13:57 ` Eric Blake
@ 2018-12-01 15:00 ` Richard W.M. Jones
0 siblings, 0 replies; 65+ messages in thread
From: Richard W.M. Jones @ 2018-12-01 15:00 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, jsnow, nsoffer, vsementsov, qemu-block
On Sat, Dec 01, 2018 at 07:57:51AM -0600, Eric Blake wrote:
> On 12/1/18 1:42 AM, Richard W.M. Jones wrote:
> >On Fri, Nov 30, 2018 at 04:03:29PM -0600, Eric Blake wrote:
> >>I note that upstream NBD has 'nbd-client -l $host' for querying
> >>just export names (with no quoting, so you have to know that
> >>a blank line means the default export), but it wasn't powerful
> >>enough, so I implemented 'qemu-nbd -L' to document everything.
> >>Upstream NBD has separate 'nbd-client' and 'nbd-server' binaries,
> >>while we only have 'qemu-nbd' (which is normally just a server,
> >>but 'qemu-nbd -c' also operates a second thread as a client).
> >>Our other uses of qemu as NBD client are for consuming a block
> >>device (as in qemu-io, qemu-img, or a drive to qemu) - but those
> >>binaries are less suited to something so specific to the NBD
> >>protocol.
> >
> >I tried it against nbdkit and it works (obviously):
> >
> >$ ./qemu-nbd -L
> >exports available: 1
> > export: ''
> > size: 67108864
> > flags: 0x61 ( trim zeroes )
> >
> >What I couldn't work out is how to connect to other hosts. It's
> >possible to add -p or -k to change the localhost port number or to use
> >a Unix domain socket (and I checked both work). However can we
> >connect to remote hosts?
>
> Should work by adding '-b host' (if -b is omitted, it defaults to
> 0.0.0.0). However, I admit that so far I have only tested '-b
> localhost' or '-b 127.0.0.1' (per the iotest additions at the end of
> the series), so it could well be something that I missed in setting
> up the client socket in nbd_build_socket_address().
Must have missed that option. It does work, thanks.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 01/14] qemu-nbd: Use program name in error messages
2018-11-30 22:03 ` [Qemu-devel] [PATCH 01/14] qemu-nbd: Use program name in error messages Eric Blake
2018-11-30 22:17 ` Richard W.M. Jones
@ 2018-12-05 14:55 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-05 14:55 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: jsnow, nsoffer, rjones, qemu-block, Kevin Wolf, Max Reitz
01.12.2018 1:03, Eric Blake wrote:
> This changes output from:
>
> $ qemu-nbd nosuch
> Failed to blk_new_open 'nosuch': Could not open 'nosuch': No such file or directory
>
> to something more consistent with qemu-img and qemu:
>
> $ qemu-nbd nosuch
> qemu-nbd: Failed to blk_new_open 'nosuch': Could not open 'nosuch': No such file or directory
>
> Update the lone affected test to match. (Hmm - is it sad that we don't
> do much testing of expected failures?)
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 02/14] nbd/client: More consistent error messages
2018-11-30 22:03 ` [Qemu-devel] [PATCH 02/14] nbd/client: More consistent " Eric Blake
2018-11-30 22:20 ` Richard W.M. Jones
@ 2018-12-05 15:03 ` Vladimir Sementsov-Ogievskiy
2018-12-10 22:03 ` Eric Blake
1 sibling, 1 reply; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-05 15:03 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: jsnow, nsoffer, rjones, qemu-block
01.12.2018 1:03, Eric Blake wrote:
> Consolidate on using decimal (not hex) and on outputting the
> option reply name (not just value) when the client reports
> protocol discrepancies from the server. While it won't affect
> normal operation, it makes debugging additions easier.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> nbd/client.c | 21 ++++++++++++---------
> 1 file changed, 12 insertions(+), 9 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index b4d457a19ad..b667a1b56fd 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -132,8 +132,9 @@ static int nbd_receive_option_reply(QIOChannel *ioc, uint32_t opt,
> return -1;
> }
> if (reply->option != opt) {
> - error_setg(errp, "Unexpected option type %x expected %x",
> - reply->option, opt);
> + error_setg(errp, "Unexpected option type %u (%s) expected %u (%s)",
> + reply->option, nbd_opt_lookup(reply->option),
> + opt, nbd_opt_lookup(opt));
> nbd_send_opt_abort(ioc);
> return -1;
> }
> @@ -265,8 +266,9 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> }
> return 0;
> } else if (reply.type != NBD_REP_SERVER) {
> - error_setg(errp, "Unexpected reply type %" PRIx32 " expected %x",
> - reply.type, NBD_REP_SERVER);
> + error_setg(errp, "Unexpected reply type %u (%s) expected %u (%s)",
> + reply.type, nbd_rep_lookup(reply.type),
> + NBD_REP_SERVER, nbd_rep_lookup(NBD_REP_SERVER));
> nbd_send_opt_abort(ioc);
> return -1;
> }
> @@ -378,9 +380,9 @@ static int nbd_opt_go(QIOChannel *ioc, const char *wantname,
> return 1;
> }
> if (reply.type != NBD_REP_INFO) {
> - error_setg(errp, "unexpected reply type %" PRIu32
> - " (%s), expected %u",
> - reply.type, nbd_rep_lookup(reply.type), NBD_REP_INFO);
> + error_setg(errp, "unexpected reply type %u (%s), expected %u (%s)",
hmm, we are definitely inconsistent about having comma before "expected" word...
anyway,
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> + reply.type, nbd_rep_lookup(reply.type),
> + NBD_REP_INFO, nbd_rep_lookup(NBD_REP_INFO));
> nbd_send_opt_abort(ioc);
> return -1;
> }
> @@ -704,8 +706,9 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> }
>
> if (reply.type != NBD_REP_ACK) {
> - error_setg(errp, "Unexpected reply type %" PRIx32 " expected %x",
> - reply.type, NBD_REP_ACK);
> + error_setg(errp, "Unexpected reply type %u (%s) expected %u (%s)",
> + reply.type, nbd_rep_lookup(reply.type),
> + NBD_REP_ACK, nbd_rep_lookup(NBD_REP_ACK));
> nbd_send_opt_abort(ioc);
> return -1;
> }
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 03/14] qemu-nbd: Fail earlier for -c/-d on non-linux
2018-11-30 22:03 ` [Qemu-devel] [PATCH 03/14] qemu-nbd: Fail earlier for -c/-d on non-linux Eric Blake
2018-11-30 22:23 ` Richard W.M. Jones
@ 2018-12-05 15:20 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-05 15:20 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: jsnow, nsoffer, rjones, qemu-block
01.12.2018 1:03, Eric Blake wrote:
> Connecting to a /dev/nbdN device is a Linux-specific action.
> We were already masking -c and -d from 'qemu-nbd --help' on
> non-linux. However, while -d fails with a sensible error
> message, it took hunting through a couple of files to prove
> that. What's more, the code for -c doesn't fail until after
> it has created a pthread and tried to open a device - possibly
> even printing an error message with %m on a non-Linux platform
> in spite of the comment that %m is glibc-specific. Make the
> failure happen sooner, then get rid of stubs that are no
> longer needed because of the early exits.
>
> While at it: tweak the blank newlines in --help output to be
> consistent, whether or not built on Linux.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
[...]
> @@ -815,6 +823,7 @@ int main(int argc, char **argv)
> }
>
> if (disconnect) {
> +#if HAVE_NBD_DEVICE
> int nbdfd = open(argv[optind], O_RDWR);
> if (nbdfd < 0) {
> error_report("Cannot open %s: %s", argv[optind],
> @@ -828,6 +837,10 @@ int main(int argc, char **argv)
> printf("%s disconnected\n", argv[optind]);
>
> return 0;
> +#else
> + error_report("Kernel /dev/nbdN support not available");
> + exit(EXIT_FAILURE);
> +#endif
> }
>
> if ((device && !verbose) || fork_process) {
> @@ -1006,6 +1019,7 @@ int main(int argc, char **argv)
> nbd_export_set_description(exp, export_description);
>
> if (device) {
> +#if HAVE_NBD_DEVICE
> int ret;
>
> ret = pthread_create(&client_thread, NULL, nbd_client_thread, device);
> @@ -1013,6 +1027,10 @@ int main(int argc, char **argv)
> error_report("Failed to create client thread: %s", strerror(ret));
> exit(EXIT_FAILURE);
> }
> +#else
> + error_report("Kernel /dev/nbdN support not available");
> + exit(EXIT_FAILURE);
hmm, we have some if (device ...) conditions with extra logic above this point.
I think it's better to fail earlier in this case. For example, exactly after
disconnect hunk
> +#endif
> } else {
> /* Shut up GCC warnings. */
> memset(&client_thread, 0, sizeof(client_thread));
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 04/14] qemu-nbd: Simplify --partition handling
2018-11-30 22:03 ` [Qemu-devel] [PATCH 04/14] qemu-nbd: Simplify --partition handling Eric Blake
2018-11-30 22:26 ` Richard W.M. Jones
@ 2018-12-05 15:40 ` Vladimir Sementsov-Ogievskiy
2018-12-05 16:26 ` Eric Blake
2018-12-10 22:28 ` Eric Blake
2 siblings, 1 reply; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-05 15:40 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: jsnow, nsoffer, rjones, qemu-block
01.12.2018 1:03, Eric Blake wrote:
> Our open-coding of strtol handling forgot to handle overflow
> conditions. What's more, since we insiste on a user-supplied
> partition to be non-zero, we can use 0 rather than -1 for our
> initial value to distinguish when a partition is not being
> served, for slightly more optimal code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> qemu-nbd.c | 14 +++++---------
> 1 file changed, 5 insertions(+), 9 deletions(-)
>
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index 55e29bd9a7e..866e64779f1 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -546,7 +546,7 @@ int main(int argc, char **argv)
> int opt_ind = 0;
> char *end;
> int flags = BDRV_O_RDWR;
> - int partition = -1;
> + int partition = 0;
> int ret = 0;
> bool seen_cache = false;
> bool seen_discard = false;
> @@ -685,13 +685,9 @@ int main(int argc, char **argv)
> flags &= ~BDRV_O_RDWR;
> break;
> case 'P':
> - partition = strtol(optarg, &end, 0);
> - if (*end) {
> - error_report("Invalid partition `%s'", optarg);
> - exit(EXIT_FAILURE);
> - }
> - if (partition < 1 || partition > 8) {
> - error_report("Invalid partition %d", partition);
> + if (qemu_strtoi(optarg, NULL, 0, &partition) < 0 ||
I decided to look into qemu_strtoi, hmm.
is it possible, that "char *ep" remains uninitialized, and than we access
it in check_strtox_error? I don's see in strtol spec a guarantee of setting
endptr on failure path.
> + partition < 1 || partition > 8) {
don't you like brace on separate line after multi-line conditions?
> + error_report("Invalid partition %s", optarg);
> exit(EXIT_FAILURE);
> }
> break;
> @@ -1004,7 +1000,7 @@ int main(int argc, char **argv)
> }
> fd_size -= dev_offset;
>
> - if (partition != -1) {
> + if (partition) {
> ret = find_partition(blk, partition, &dev_offset, &fd_size);
> if (ret < 0) {
> error_report("Could not find partition %d: %s", partition,
>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 05/14] nbd/client: Drop pointless buf variable
2018-11-30 22:03 ` [Qemu-devel] [PATCH 05/14] nbd/client: Drop pointless buf variable Eric Blake
2018-11-30 22:30 ` Richard W.M. Jones
@ 2018-12-05 15:59 ` Vladimir Sementsov-Ogievskiy
2018-12-05 16:29 ` Eric Blake
1 sibling, 1 reply; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-05 15:59 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: jsnow, nsoffer, rjones, qemu-block
01.12.2018 1:03, Eric Blake wrote:
> There's no need to read into a temporary buffer (oversized
> since commit 7d3123e1) followed by a byteswap into a uint64_t
> to check for a magic number via memcmp(), when the code
> immediately below demonstrates reading into the uint64_t then
> byteswapping in place and checking for a magic number via
> integer math. What's more, having a different error message
> when the server's first reply byte is 0 is unusual - it's no
> different from any other wrong magic number, and we already
> detected short reads.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> nbd/nbd-internal.h | 1 +
> nbd/client.c | 14 +++-----------
> 2 files changed, 4 insertions(+), 11 deletions(-)
>
> diff --git a/nbd/nbd-internal.h b/nbd/nbd-internal.h
> index eeff78d3c98..306a533dcd1 100644
> --- a/nbd/nbd-internal.h
> +++ b/nbd/nbd-internal.h
> @@ -46,6 +46,7 @@
> /* Size of oldstyle negotiation */
> #define NBD_OLDSTYLE_NEGOTIATE_SIZE (8 + 8 + 8 + 4 + 124)
>
> +#define NBD_INIT_MAGIC 0x4e42444d41474943LL
Worth add comment /* "NBDMAGIC" */
> #define NBD_REQUEST_MAGIC 0x25609513
> #define NBD_OPTS_MAGIC 0x49484156454F5054LL
> #define NBD_CLIENT_MAGIC 0x0000420281861253LL
> diff --git a/nbd/client.c b/nbd/client.c
> index 0be89f9e641..17ee24492a4 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -731,7 +731,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
> QIOChannel **outioc, NBDExportInfo *info,
> Error **errp)
> {
> - char buf[256];
> uint64_t magic;
> int rc;
> bool zeroes = true;
> @@ -752,21 +751,14 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
> goto fail;
> }
>
> - if (nbd_read(ioc, buf, 8, errp) < 0) {
> + if (nbd_read(ioc, &magic, sizeof(magic), errp) < 0) {
> error_prepend(errp, "Failed to read data: ");
may be, change to "Failed to read magic: ", as in code below
> goto fail;
> }
> -
> - buf[8] = '\0';
> - if (strlen(buf) == 0) {
> - error_setg(errp, "Server connection closed unexpectedly");
> - goto fail;
> - }
> -
> - magic = ldq_be_p(buf);
> + magic = be64_to_cpu(magic);
Isn't it better to use be64_to_cpus?
> trace_nbd_receive_negotiate_magic(magic);
>
> - if (memcmp(buf, "NBDMAGIC", 8) != 0) {
> + if (magic != NBD_INIT_MAGIC) {
> error_setg(errp, "Invalid magic received");
> goto fail;
> }
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 04/14] qemu-nbd: Simplify --partition handling
2018-12-05 15:40 ` Vladimir Sementsov-Ogievskiy
@ 2018-12-05 16:26 ` Eric Blake
2018-12-05 16:32 ` Eric Blake
0 siblings, 1 reply; 65+ messages in thread
From: Eric Blake @ 2018-12-05 16:26 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: jsnow, nsoffer, rjones, qemu-block
On 12/5/18 9:40 AM, Vladimir Sementsov-Ogievskiy wrote:
> 01.12.2018 1:03, Eric Blake wrote:
>> Our open-coding of strtol handling forgot to handle overflow
>> conditions. What's more, since we insiste on a user-supplied
>> partition to be non-zero, we can use 0 rather than -1 for our
>> initial value to distinguish when a partition is not being
>> served, for slightly more optimal code.
>>
>> - if (partition < 1 || partition > 8) {
>> - error_report("Invalid partition %d", partition);
>> + if (qemu_strtoi(optarg, NULL, 0, &partition) < 0 ||
>
> I decided to look into qemu_strtoi, hmm.
>
> is it possible, that "char *ep" remains uninitialized, and than we access
> it in check_strtox_error? I don's see in strtol spec a guarantee of setting
> endptr on failure path.
C99 7.10.1.4 P5-7 requires strtoll() and friends to assign through
'endptr' if it is non-NULL, for both success and ERANGE failure cases.
POSIX then further requires 'endptr' to be set for EINVAL failures due
to out-of-range 'base' (not that we have any such callers), and permits
(but does not require) the empty string to cause an EINVAL failure (but
whether or not EINVAL happened, 'endptr' is still set). There are no
other possible failures, so no, we are not dereferencing an
uninitialized variable in check_strtox_error.
>
>
>> + partition < 1 || partition > 8) {
>
> don't you like brace on separate line after multi-line conditions?
CODING_STYLE is silent on the matter; checkpatch.pl allows either style
for multi-line, while requesting the same line as the condition for
one-line (see commit a97ceca57). But since I'm already in the habit of
putting { right after the condition because of checkpatch, I don't go
out of my way to put it on its own line after multi-line conditionals.
I don't think it matters too much.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 05/14] nbd/client: Drop pointless buf variable
2018-12-05 15:59 ` Vladimir Sementsov-Ogievskiy
@ 2018-12-05 16:29 ` Eric Blake
2018-12-05 16:38 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 65+ messages in thread
From: Eric Blake @ 2018-12-05 16:29 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: jsnow, nsoffer, rjones, qemu-block
On 12/5/18 9:59 AM, Vladimir Sementsov-Ogievskiy wrote:
> 01.12.2018 1:03, Eric Blake wrote:
>> There's no need to read into a temporary buffer (oversized
>> since commit 7d3123e1) followed by a byteswap into a uint64_t
>> to check for a magic number via memcmp(), when the code
>> immediately below demonstrates reading into the uint64_t then
>> byteswapping in place and checking for a magic number via
>> integer math. What's more, having a different error message
>> when the server's first reply byte is 0 is unusual - it's no
>> different from any other wrong magic number, and we already
>> detected short reads.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>> nbd/nbd-internal.h | 1 +
>> nbd/client.c | 14 +++-----------
>> 2 files changed, 4 insertions(+), 11 deletions(-)
>>
>> diff --git a/nbd/nbd-internal.h b/nbd/nbd-internal.h
>> index eeff78d3c98..306a533dcd1 100644
>> --- a/nbd/nbd-internal.h
>> +++ b/nbd/nbd-internal.h
>> @@ -46,6 +46,7 @@
>> /* Size of oldstyle negotiation */
>> #define NBD_OLDSTYLE_NEGOTIATE_SIZE (8 + 8 + 8 + 4 + 124)
>>
>> +#define NBD_INIT_MAGIC 0x4e42444d41474943LL
>
> Worth add comment /* "NBDMAGIC" */
Maybe. But if so,
>
>> #define NBD_REQUEST_MAGIC 0x25609513
>> #define NBD_OPTS_MAGIC 0x49484156454F5054LL
this should also get a comment "IHAVEOPT".
>> #define NBD_CLIENT_MAGIC 0x0000420281861253LL
>> diff --git a/nbd/client.c b/nbd/client.c
>> index 0be89f9e641..17ee24492a4 100644
>> --- a/nbd/client.c
>> +++ b/nbd/client.c
>> @@ -731,7 +731,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
>> QIOChannel **outioc, NBDExportInfo *info,
>> Error **errp)
>> {
>> - char buf[256];
>> uint64_t magic;
>> int rc;
>> bool zeroes = true;
>> @@ -752,21 +751,14 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
>> goto fail;
>> }
>>
>> - if (nbd_read(ioc, buf, 8, errp) < 0) {
>> + if (nbd_read(ioc, &magic, sizeof(magic), errp) < 0) {
>> error_prepend(errp, "Failed to read data: ");
>
> may be, change to "Failed to read magic: ", as in code below
Argument for: consistency sake. Argument against: having distinct
messages lets you debug which of the two magic strings was wrong. I'm
not sure I have a strong preference.
>
>> goto fail;
>> }
>> -
>> - buf[8] = '\0';
>> - if (strlen(buf) == 0) {
>> - error_setg(errp, "Server connection closed unexpectedly");
>> - goto fail;
>> - }
>> -
>> - magic = ldq_be_p(buf);
>> + magic = be64_to_cpu(magic);
>
> Isn't it better to use be64_to_cpus?
No. We're intentionally getting rid of that because of clang; see commit
80c7c2b0.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 04/14] qemu-nbd: Simplify --partition handling
2018-12-05 16:26 ` Eric Blake
@ 2018-12-05 16:32 ` Eric Blake
0 siblings, 0 replies; 65+ messages in thread
From: Eric Blake @ 2018-12-05 16:32 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: nsoffer, jsnow, rjones, qemu-block
On 12/5/18 10:26 AM, Eric Blake wrote:
>> is it possible, that "char *ep" remains uninitialized, and than we access
>> it in check_strtox_error? I don's see in strtol spec a guarantee of
>> setting
>> endptr on failure path.
>
> C99 7.10.1.4 P5-7 requires strtoll() and friends to assign through
> 'endptr' if it is non-NULL, for both success and ERANGE failure cases.
> POSIX then further requires 'endptr' to be set for EINVAL failures due
> to out-of-range 'base' (not that we have any such callers), and permits
> (but does not require) the empty string to cause an EINVAL failure (but
> whether or not EINVAL happened, 'endptr' is still set). There are no
> other possible failures, so no, we are not dereferencing an
> uninitialized variable in check_strtox_error.
Correction, quoting POSIX:
http://pubs.opengroup.org/onlinepubs/9699919799/functions/strtol.html
APPLICATION USAGE
Since the value of *endptr is unspecified if the value of base is
not supported, applications should either ensure that base has a
supported value (0 or between 2 and 36) before the call, or check for an
[EINVAL] error before examining *endptr.
So yes, we CAN end up transferring an uninitialized variable into the
caller's non-NULL endpointer if the caller passes an out-of-range base
(this particular caller passes NULL for an endpointer, and an in-range
base, so it's not an issue). Might be worth a separate patch to assert
that base is in range for all of the qemu_strto* helpers, if we are
worried (I'm not).
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 05/14] nbd/client: Drop pointless buf variable
2018-12-05 16:29 ` Eric Blake
@ 2018-12-05 16:38 ` Vladimir Sementsov-Ogievskiy
2018-12-05 16:49 ` Eric Blake
0 siblings, 1 reply; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-05 16:38 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: jsnow, nsoffer, rjones, qemu-block
05.12.2018 19:29, Eric Blake wrote:
> On 12/5/18 9:59 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 01.12.2018 1:03, Eric Blake wrote:
>>> There's no need to read into a temporary buffer (oversized
>>> since commit 7d3123e1) followed by a byteswap into a uint64_t
>>> to check for a magic number via memcmp(), when the code
>>> immediately below demonstrates reading into the uint64_t then
>>> byteswapping in place and checking for a magic number via
>>> integer math. What's more, having a different error message
>>> when the server's first reply byte is 0 is unusual - it's no
>>> different from any other wrong magic number, and we already
>>> detected short reads.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>> ---
>>> nbd/nbd-internal.h | 1 +
>>> nbd/client.c | 14 +++-----------
>>> 2 files changed, 4 insertions(+), 11 deletions(-)
>>>
>>> diff --git a/nbd/nbd-internal.h b/nbd/nbd-internal.h
>>> index eeff78d3c98..306a533dcd1 100644
>>> --- a/nbd/nbd-internal.h
>>> +++ b/nbd/nbd-internal.h
>>> @@ -46,6 +46,7 @@
>>> /* Size of oldstyle negotiation */
>>> #define NBD_OLDSTYLE_NEGOTIATE_SIZE (8 + 8 + 8 + 4 + 124)
>>>
>>> +#define NBD_INIT_MAGIC 0x4e42444d41474943LL
>>
>> Worth add comment /* "NBDMAGIC" */
>
> Maybe. But if so,
>
>>
>>> #define NBD_REQUEST_MAGIC 0x25609513
>>> #define NBD_OPTS_MAGIC 0x49484156454F5054LL
>
> this should also get a comment "IHAVEOPT".
>
>>> #define NBD_CLIENT_MAGIC 0x0000420281861253LL
>>> diff --git a/nbd/client.c b/nbd/client.c
>>> index 0be89f9e641..17ee24492a4 100644
>>> --- a/nbd/client.c
>>> +++ b/nbd/client.c
>>> @@ -731,7 +731,6 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
>>> QIOChannel **outioc, NBDExportInfo *info,
>>> Error **errp)
>>> {
>>> - char buf[256];
>>> uint64_t magic;
>>> int rc;
>>> bool zeroes = true;
>>> @@ -752,21 +751,14 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
>>> goto fail;
>>> }
>>>
>>> - if (nbd_read(ioc, buf, 8, errp) < 0) {
>>> + if (nbd_read(ioc, &magic, sizeof(magic), errp) < 0) {
>>> error_prepend(errp, "Failed to read data: ");
>>
>> may be, change to "Failed to read magic: ", as in code below
>
> Argument for: consistency sake. Argument against: having distinct messages lets you debug which of the two magic strings was wrong. I'm not sure I have a strong preference.
>
>>
>>> goto fail;
>>> }
>>> -
>>> - buf[8] = '\0';
>>> - if (strlen(buf) == 0) {
>>> - error_setg(errp, "Server connection closed unexpectedly");
>>> - goto fail;
>>> - }
>>> -
>>> - magic = ldq_be_p(buf);
>>> + magic = be64_to_cpu(magic);
>>
>> Isn't it better to use be64_to_cpus?
>
> No. We're intentionally getting rid of that because of clang; see commit 80c7c2b0.
>
>
Ok, thanks. In this case it should be safe, but if we decided to avoid these functions in general than OK.
Hmm, not in general, but only in nbd.. Strange, why not in qcow2 for ex?, anyway, with or without other tiny things:
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 05/14] nbd/client: Drop pointless buf variable
2018-12-05 16:38 ` Vladimir Sementsov-Ogievskiy
@ 2018-12-05 16:49 ` Eric Blake
0 siblings, 0 replies; 65+ messages in thread
From: Eric Blake @ 2018-12-05 16:49 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: jsnow, nsoffer, rjones, qemu-block
On 12/5/18 10:38 AM, Vladimir Sementsov-Ogievskiy wrote:
>>>> - magic = ldq_be_p(buf);
>>>> + magic = be64_to_cpu(magic);
>>>
>>> Isn't it better to use be64_to_cpus?
>>
>> No. We're intentionally getting rid of that because of clang; see commit 80c7c2b0.
>>
>>
>
> Ok, thanks. In this case it should be safe, but if we decided to avoid these functions in general than OK.
> Hmm, not in general, but only in nbd.. Strange, why not in qcow2 for ex?,
Peter is working on that - it's a slow process, because he's sending
separate patch series per maintainer, so they are not all getting
checked in at the same time. But the idea is that once everything is
converted, we nuke the *_to_*s variants as unused, and in the meantime,
we don't add more uses of it.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 06/14] nbd/client: Move export name into NBDExportInfo
2018-11-30 22:03 ` [Qemu-devel] [PATCH 06/14] nbd/client: Move export name into NBDExportInfo Eric Blake
2018-11-30 22:34 ` Richard W.M. Jones
@ 2018-12-05 17:26 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-05 17:26 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: jsnow, nsoffer, rjones, qemu-block, Kevin Wolf, Max Reitz
01.12.2018 1:03, Eric Blake wrote:
> Refactor the 'name' parameter of nbd_receive_negotiate() from
> being a separate parameter into being part of the in-out 'info'.
> This also spills over to a simplification of nbd_opt_go().
>
> The main driver for this refactoring is that an upcoming patch
> would like to add support to qemu-nbd to list information about
> all exports available on a server, where the name(s) will be
> provided by the server instead of the client. But another benefit
> is that we can now allow the client to explicitly specify the
> empty export name "" even when connecting to an oldstyle server
> (even if qemu is no longer such a server after commit 7f7dfe2a).
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
[...]
> @@ -877,8 +874,8 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name,
> } else if (magic == NBD_CLIENT_MAGIC) {
> uint32_t oldflags;
>
> - if (name) {
> - error_setg(errp, "Server does not support export names");
> + if (*info->name) {
> + error_setg(errp, "Server does not support non-empty export names");
hm, old message is a bit nearer to nbd spec, the new may be a bit more
understandable in context of current qemu-nbd help:
"-x, --export-name=NAME expose export by name (default is empty string)"
so, I'm OK with either one.
> goto fail;
> }
> if (tlscreds) {
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index 866e64779f1..c57053a0795 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -263,7 +263,7 @@ static void *show_parts(void *arg)
> static void *nbd_client_thread(void *arg)
> {
> char *device = arg;
> - NBDExportInfo info = { .request_sizes = false, };
> + NBDExportInfo info = { .request_sizes = false, .name = g_strdup("") };
> QIOChannelSocket *sioc;
> int fd;
> int ret;
> @@ -278,7 +278,7 @@ static void *nbd_client_thread(void *arg)
> goto out;
> }
>
> - ret = nbd_receive_negotiate(QIO_CHANNEL(sioc), NULL,
> + ret = nbd_receive_negotiate(QIO_CHANNEL(sioc),
> NULL, NULL, NULL, &info, &local_error);
> if (ret < 0) {
> if (local_error) {
> @@ -317,6 +317,7 @@ static void *nbd_client_thread(void *arg)
> }
> close(fd);
> object_unref(OBJECT(sioc));
> + free(info.name);
g_free
> kill(getpid(), SIGTERM);
> return (void *) EXIT_SUCCESS;
>
> @@ -325,6 +326,7 @@ out_fd:
> out_socket:
> object_unref(OBJECT(sioc));
> out:
> + free(info.name);
and here
> kill(getpid(), SIGTERM);
> return (void *) EXIT_FAILURE;
> }
> diff --git a/nbd/trace-events b/nbd/trace-events
> index 5e1d4afe8e6..289337d0dc3 100644
> --- a/nbd/trace-events
> +++ b/nbd/trace-events
> @@ -15,7 +15,7 @@ nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of contex
> nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
> nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
> nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are 0x%" PRIx32
> -nbd_receive_negotiate_default_name(void) "Using default NBD export name \"\""
> +nbd_receive_negotiate_name(const char *name) "Requesting NBD export name \"%s\""
Other trace points prefers to use single quotes with export name (or without quotes),
this may be changed too. Oh, too deep nit-picking o_0
> nbd_receive_negotiate_size_flags(uint64_t size, uint16_t flags) "Size is %" PRIu64 ", export flags 0x%" PRIx16
> nbd_init_set_socket(void) "Setting NBD socket"
> nbd_init_set_block_size(unsigned long block_size) "Setting block size to %lu"
>
with fixed s/free/g_free:
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 07/14] nbd/client: Refactor nbd_negotiate_simple_meta_context()
2018-11-30 22:03 ` [Qemu-devel] [PATCH 07/14] nbd/client: Refactor nbd_negotiate_simple_meta_context() Eric Blake
2018-12-01 10:30 ` Richard W.M. Jones
@ 2018-12-06 13:20 ` Vladimir Sementsov-Ogievskiy
2018-12-06 16:20 ` Eric Blake
1 sibling, 1 reply; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-06 13:20 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: jsnow, nsoffer, rjones, qemu-block
01.12.2018 1:03, Eric Blake wrote:
> Change the signature to make it easier for a future patch to
> reuse this function for calling NBD_OPT_LIST_META_CONTEXT with
> 0 or 1 queries. Also, always allocate space for the received
> name, even if it doesn't match expected lengths (no point
> trying to optimize the unlikely error case, and tracing the
> received rather than expected name can help debug a server
> implementation).
>
> While there are now slightly different traces, and the error
> message for a server replying with too many contexts is
> different, there are no runtime-observable changes in behavior
> for the more common case of the lone caller interacting with a
> compliant server.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> nbd/client.c | 105 +++++++++++++++++++++++++++--------------------
> nbd/trace-events | 2 +-
> 2 files changed, 61 insertions(+), 46 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index b5818a99d21..1dc8f83e19a 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -603,49 +603,57 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
> }
>
> /* nbd_negotiate_simple_meta_context:
> - * Set one meta context. Simple means that reply must contain zero (not
> - * negotiated) or one (negotiated) contexts. More contexts would be considered
> - * as a protocol error. It's also implied that meta-data query equals queried
> - * context name, so, if server replies with something different than @context,
> - * it is considered an error too.
> - * return 1 for successful negotiation, context_id is set
> - * 0 if operation is unsupported,
> + * List or set meta context data for export @info->name, based on @opt.
hm, just list or set meta context? What is "data" about?
> + * For list, leave @context NULL for 0 queries, or supplied for a single
> + * query; all replies are ignored, and the call merely traces server behavior.
> + * For set, @context must result in at most one matching server reply, in
> + * which case @info->meta_base_allocation_id is set to the resulting id.
Hmm, looks too cheating. Then it should be renamed to
nbd_negotiate_base_allocation
and parameter @context should be renamed to @x_dirty_bitmap,
and if it is unset, we'll use "base:allocation" here.
but in this case, it still weird about opt=list case.. So, it should be named
like nbd_negotiation_helper, as it is doing several different things, which
I can't describe in one word.
> + * return 1 for successful negotiation,
> + * 0 if operation is unsupported or context unavailable,
> * -1 with errp set for any other error
this return value description looks not very related to opt=list case
> */
> static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> - const char *export,
> + int32_t opt,
> const char *context,
> - uint32_t *context_id,
> + NBDExportInfo *info,
> Error **errp)
> {
> int ret;
> NBDOptionReply reply;
> uint32_t received_id = 0;
> bool received = false;
> - uint32_t export_len = strlen(export);
> - uint32_t context_len = strlen(context);
> - uint32_t data_len = sizeof(export_len) + export_len +
> - sizeof(uint32_t) + /* number of queries */
> - sizeof(context_len) + context_len;
> - char *data = g_malloc(data_len);
> - char *p = data;
> + uint32_t export_len = strlen(info->name);
> + uint32_t context_len;
> + uint32_t data_len = sizeof(export_len) + export_len + sizeof(uint32_t);
I'd prefer to leave the comment /* number of queries */
> + char *data;
> + char *p;
>
> - trace_nbd_opt_meta_request(context, export);
> + if (!context) {
> + assert(opt == NBD_OPT_LIST_META_CONTEXT);
> + } else {
> + context_len = strlen(context);
> + data_len += sizeof(context_len) + context_len;
> + }
> + data = g_malloc(data_len);
> + p = data;
> +
> + trace_nbd_opt_meta_request(nbd_opt_lookup(opt), context ?: "(all)",
> + info->name);
> stl_be_p(p, export_len);
> - memcpy(p += sizeof(export_len), export, export_len);
> - stl_be_p(p += export_len, 1);
> - stl_be_p(p += sizeof(uint32_t), context_len);
> - memcpy(p += sizeof(context_len), context, context_len);
> + memcpy(p += sizeof(export_len), info->name, export_len);
> + stl_be_p(p += export_len, !!context);
> + if (context) {
> + stl_be_p(p += sizeof(uint32_t), context_len);
> + memcpy(p += sizeof(context_len), context, context_len);
> + }
>
> - ret = nbd_send_option_request(ioc, NBD_OPT_SET_META_CONTEXT, data_len, data,
> - errp);
> + ret = nbd_send_option_request(ioc, opt, data_len, data, errp);
> g_free(data);
> if (ret < 0) {
> return ret;
> }
>
> - if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
> - errp) < 0)
> + if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0)
> {
> return -1;
> }
> @@ -655,10 +663,10 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return ret;
> }
>
> - if (reply.type == NBD_REP_META_CONTEXT) {
> + while (reply.type == NBD_REP_META_CONTEXT) {
> char *name;
>
> - if (reply.length != sizeof(received_id) + context_len) {
> + if (reply.length <= sizeof(received_id)) {
> error_setg(errp, "Failed to negotiate meta context '%s', server "
> "answered with unexpected length %" PRIu32, context,
> reply.length);
> @@ -678,23 +686,31 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return -1;
> }
> name[reply.length] = '\0';
> - if (strcmp(context, name)) {
> - error_setg(errp, "Failed to negotiate meta context '%s', server "
> - "answered with different context '%s'", context,
> - name);
> - g_free(name);
> - nbd_send_opt_abort(ioc);
> - return -1;
> +
> + trace_nbd_opt_meta_reply(name, received_id);
> + if (opt == NBD_OPT_SET_META_CONTEXT) {
> + if (received) {
> + error_setg(errp, "Server replied with more than one context");
> + free(name);
g_free
> + nbd_send_opt_abort(ioc);
> + return -1;
> + }
> +
> + if (strcmp(context, name)) {
> + error_setg(errp,
> + "Failed to negotiate meta context '%s', server "
> + "answered with different context '%s'", context,
> + name);
> + g_free(name);
> + nbd_send_opt_abort(ioc);
> + return -1;
> + }
> }
> g_free(name);
> -
> - trace_nbd_opt_meta_reply(context, received_id);
> received = true;
>
> /* receive NBD_REP_ACK */
> - if (nbd_receive_option_reply(ioc, NBD_OPT_SET_META_CONTEXT, &reply,
> - errp) < 0)
> - {
> + if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0) {
> return -1;
> }
>
> @@ -717,12 +733,11 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return -1;
> }
>
> - if (received) {
> - *context_id = received_id;
> - return 1;
> + if (received && opt == NBD_OPT_SET_META_CONTEXT) {
> + info->meta_base_allocation_id = received_id;
> }
>
> - return 0;
> + return received || opt == NBD_OPT_LIST_META_CONTEXT;
> }
the changes looks correct, but new semantics seems too weird for me.
>
> int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> @@ -822,9 +837,9 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
>
> if (info->structured_reply && base_allocation) {
> result = nbd_negotiate_simple_meta_context(
> - ioc, info->name,
> + ioc, NBD_OPT_SET_META_CONTEXT,
> info->x_dirty_bitmap ?: "base:allocation",
> - &info->meta_base_allocation_id, errp);
> + info, errp);
> if (result < 0) {
> goto fail;
> }
> diff --git a/nbd/trace-events b/nbd/trace-events
> index 289337d0dc3..5d0d202fad2 100644
> --- a/nbd/trace-events
> +++ b/nbd/trace-events
> @@ -10,7 +10,7 @@ nbd_receive_query_exports_start(const char *wantname) "Querying export list for
> nbd_receive_query_exports_success(const char *wantname) "Found desired export name '%s'"
> nbd_receive_starttls_new_client(void) "Setting up TLS"
> nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
> -nbd_opt_meta_request(const char *context, const char *export) "Requesting to set meta context %s for export %s"
> +nbd_opt_meta_request(const char *opt, const char *context, const char *export) "Requesting to %s %s for export %s"
> nbd_opt_meta_reply(const char *context, uint32_t id) "Received mapping of context %s to id %" PRIu32
> nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation tlscreds=%p hostname=%s"
> nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 08/14] nbd/client: Refactor nbd_receive_list()
2018-11-30 22:03 ` [Qemu-devel] [PATCH 08/14] nbd/client: Refactor nbd_receive_list() Eric Blake
2018-12-01 10:37 ` Richard W.M. Jones
@ 2018-12-06 14:18 ` Vladimir Sementsov-Ogievskiy
2018-12-06 16:31 ` Eric Blake
1 sibling, 1 reply; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-06 14:18 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: jsnow, nsoffer, rjones, qemu-block
01.12.2018 1:03, Eric Blake wrote:
> Add some parameters to make this function reusable in upcoming
> export listing, where we will want to capture the name and
> description rather than compare against a user-supplied name.
> No change in semantics to the existing caller.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> nbd/client.c | 66 +++++++++++++++++++++++++++++++++++++---------------
> 1 file changed, 47 insertions(+), 19 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 1dc8f83e19a..27785c55d0a 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -232,18 +232,21 @@ static int nbd_handle_reply_err(QIOChannel *ioc, NBDOptionReply *reply,
> return result;
> }
>
> -/* Process another portion of the NBD_OPT_LIST reply. Set *@match if
> - * the current reply matches @want or if the server does not support
> - * NBD_OPT_LIST, otherwise leave @match alone. Return 0 if iteration
> - * is complete, positive if more replies are expected, or negative
> - * with @errp set if an unrecoverable error occurred. */
> +/* Process another portion of the NBD_OPT_LIST reply. If @want, then
> + * set *@match if the current reply matches @want or if the server
> + * does not support NBD_OPT_LIST, otherwise leave @match alone.
> + * Otherwise, @nameout and @description are malloc'd to contain
what this "otherwise" referred to?
upd: aha, after rereading, I understood that it relates to the very first
If, and the whole thing became clear. Hmm, I'm okay with this now, but it
may be still worth some simplifying.
> + * NUL-terminated copies of the reply.
You didn't say anything about @match in "Otherwise" branch
Return 0 if iteration is
however, "iterations is complete" may differ for @want/!@want cases
> + * complete, positive if more replies are expected, or negative with
> + * @errp set if an unrecoverable error occurred. */
> static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> - Error **errp)
> + char **nameout, char **description, Error **errp)
> {
> NBDOptionReply reply;
> uint32_t len;
> uint32_t namelen;
> - char name[NBD_MAX_NAME_SIZE + 1];
> + char array[NBD_MAX_NAME_SIZE + 1];
> + char *name = array;
> int error;
>
> if (nbd_receive_option_reply(ioc, NBD_OPT_LIST, &reply, errp) < 0) {
> @@ -253,7 +256,12 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> if (error <= 0) {
> /* The server did not support NBD_OPT_LIST, so set *match on
> * the assumption that any name will be accepted. */
> - *match = true;
> + if (want) {
> + *match = true;
> + } else if (!error) {
> + error_setg(errp, "Server does not support export lists");
> + error = -1;
> + }
> return error;
> }
> len = reply.length;
> @@ -290,30 +298,49 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> nbd_send_opt_abort(ioc);
> return -1;
> }
> - if (namelen != strlen(want)) {
> - if (nbd_drop(ioc, len, errp) < 0) {
> - error_prepend(errp,
> - "failed to skip export name with wrong length: ");
> - nbd_send_opt_abort(ioc);
> - return -1;
> + if (want) {
> + if (namelen != strlen(want)) {
> + if (nbd_drop(ioc, len, errp) < 0) {
> + error_prepend(errp,
> + "failed to skip export name with wrong length: ");
> + nbd_send_opt_abort(ioc);
> + return -1;
> + }
> + return 1;
> }
> - return 1;
> + assert(namelen < sizeof(array));
> + } else {
> + assert(nameout);
this assert looks a bit redundant, if nameout is 0, next line will abort not less clearly
> + *nameout = name = g_new(char, namelen + 1);
We should check namelen <= NBD_MAX_NAME_SIZE before it, I think.
> }
>
> - assert(namelen < sizeof(name));
> if (nbd_read(ioc, name, namelen, errp) < 0) {
> error_prepend(errp, "failed to read export name: ");
> nbd_send_opt_abort(ioc);
> + if (!want) {
> + free(name);
g_free
> + }
> return -1;
> }
> name[namelen] = '\0';
> len -= namelen;
> - if (nbd_drop(ioc, len, errp) < 0) {
> + if (!want) {
> + assert(description);
NBD_MAX_NAME_SIZE
> + *description = g_new(char, len + 1);
> + if (nbd_read(ioc, *description, len, errp) < 0) {
> + error_prepend(errp, "failed to read export description: ");
> + nbd_send_opt_abort(ioc);
> + free(name);
> + free(*description);
g_free
> + return -1;
> + }
> + (*description)[len] = '\0';
> + } else if (nbd_drop(ioc, len, errp) < 0) {
> error_prepend(errp, "failed to read export description: ");
> nbd_send_opt_abort(ioc);
> return -1;
> }
> - if (!strcmp(name, want)) {
> + if (want && !strcmp(name, want)) {
> *match = true;
> }
> return 1;
one more thing: on fail path, you finally fill output name and description
with freed pointers. I'd prefer to keep them unchanged in this case, however,
it's a matter of taste.
> @@ -498,7 +525,8 @@ static int nbd_receive_query_exports(QIOChannel *ioc,
> }
>
> while (1) {
> - int ret = nbd_receive_list(ioc, wantname, &foundExport, errp);
> + int ret = nbd_receive_list(ioc, wantname, &foundExport,
> + NULL, NULL, errp);
>
> if (ret < 0) {
> /* Server gave unexpected reply */
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 09/14] nbd/client: Refactor return of nbd_receive_negotiate()
2018-11-30 22:03 ` [Qemu-devel] [PATCH 09/14] nbd/client: Refactor return of nbd_receive_negotiate() Eric Blake
2018-11-30 22:41 ` Richard W.M. Jones
@ 2018-12-06 14:24 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-06 14:24 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: jsnow, nsoffer, rjones, qemu-block
01.12.2018 1:03, Eric Blake wrote:
> The function could only ever return 0 or -EINVAL; make this
> clearer by dropping a useless 'fail:' label.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> nbd/client.c | 51 +++++++++++++++++++++++----------------------------
> 1 file changed, 23 insertions(+), 28 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 27785c55d0a..1ed5009642e 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
[...]
> trace_nbd_receive_negotiate_size_flags(info->size, info->flags);
> if (zeroes && nbd_drop(ioc, 124, errp) < 0) {
> error_prepend(errp, "Failed to read reserved block: ");
> - goto fail;
> + return -EINVAL;
> }
> - rc = 0;
> -
hmm, personally I like this empty line
> -fail:
> - return rc;
> + return 0;
> }
>
> #ifdef __linux__
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 10/14] nbd/client: Split handshake into two functions
2018-11-30 22:03 ` [Qemu-devel] [PATCH 10/14] nbd/client: Split handshake into two functions Eric Blake
2018-12-01 10:41 ` Richard W.M. Jones
@ 2018-12-06 15:16 ` Vladimir Sementsov-Ogievskiy
2018-12-06 17:06 ` Vladimir Sementsov-Ogievskiy
1 sibling, 1 reply; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-06 15:16 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: jsnow, nsoffer, rjones, qemu-block
01.12.2018 1:03, Eric Blake wrote:
> An upcoming patch will add the ability for qemu-nbd to list
> the services provided by an NBD server. Share the common
> code of the TLS handshake by splitting the initial exchange
> into a separate function, leaving only the export handling
> in the original function. Functionally, there should be no
> change in behavior in this patch, although some of the code
> motion may be difficult to follow due to indentation changes
> (view with 'git diff -w' for a smaller changeset).
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> nbd/client.c | 142 ++++++++++++++++++++++++++++++-----------------
> nbd/trace-events | 2 +-
> 2 files changed, 92 insertions(+), 52 deletions(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 1ed5009642e..a282712724d 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -768,21 +768,22 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return received || opt == NBD_OPT_LIST_META_CONTEXT;
> }
>
> -int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> - const char *hostname, QIOChannel **outioc,
> - NBDExportInfo *info, Error **errp)
> +/* Start the handshake to the server. After a positive return, the server
> + * is ready to accept additional NBD_OPT requests.
> + * Returns: negative errno: failure talking to server
> + * 0: server is oldstyle, client must still parse export size
> + * 1: server is newstyle, but can only accept EXPORT_NAME
> + * 2: server is newstyle, but lacks structured replies
> + * 3: server is newstyle and set up for structured replies
> + */
hmm, may be, introduce enum of named constants, instead of raw numbers?
NBD_STARTED_OLD_STYLE
NBD_STARTED_NEW_STYLE
NBD_STARTED_NEW_STYLE_FIXED
NBD_STARTED_STRUCTURED_REPLIES
or, at least, add short comment after each return statement, to not scroll up
every time (like you gracefully do after each case: statement).
However, I'm okay with either way.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 07/14] nbd/client: Refactor nbd_negotiate_simple_meta_context()
2018-12-06 13:20 ` Vladimir Sementsov-Ogievskiy
@ 2018-12-06 16:20 ` Eric Blake
0 siblings, 0 replies; 65+ messages in thread
From: Eric Blake @ 2018-12-06 16:20 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: jsnow, nsoffer, rjones, qemu-block
On 12/6/18 7:20 AM, Vladimir Sementsov-Ogievskiy wrote:
> 01.12.2018 1:03, Eric Blake wrote:
>> Change the signature to make it easier for a future patch to
>> reuse this function for calling NBD_OPT_LIST_META_CONTEXT with
>> 0 or 1 queries. Also, always allocate space for the received
>> name, even if it doesn't match expected lengths (no point
>> trying to optimize the unlikely error case, and tracing the
>> received rather than expected name can help debug a server
>> implementation).
>>
>> While there are now slightly different traces, and the error
>> message for a server replying with too many contexts is
>> different, there are no runtime-observable changes in behavior
>> for the more common case of the lone caller interacting with a
>> compliant server.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>> nbd/client.c | 105 +++++++++++++++++++++++++++--------------------
>> nbd/trace-events | 2 +-
>> 2 files changed, 61 insertions(+), 46 deletions(-)
>>
>> diff --git a/nbd/client.c b/nbd/client.c
>> index b5818a99d21..1dc8f83e19a 100644
>> --- a/nbd/client.c
>> +++ b/nbd/client.c
>> @@ -603,49 +603,57 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
>> }
>>
>> /* nbd_negotiate_simple_meta_context:
>> - * Set one meta context. Simple means that reply must contain zero (not
>> - * negotiated) or one (negotiated) contexts. More contexts would be considered
>> - * as a protocol error. It's also implied that meta-data query equals queried
>> - * context name, so, if server replies with something different than @context,
>> - * it is considered an error too.
>> - * return 1 for successful negotiation, context_id is set
>> - * 0 if operation is unsupported,
>> + * List or set meta context data for export @info->name, based on @opt.
>
> hm, just list or set meta context? What is "data" about?
Okay, that's two different reviewers complaining. I'll do a bit more
refactoring and comments for v2 to make it more obvious that I'm writing
a common routine that can handle the bulk of the commonality between
list and set; but maybe keep two wrappers so that the existing callers
don't have to change their calls as drastically.
>
>> + * For list, leave @context NULL for 0 queries, or supplied for a single
>> + * query; all replies are ignored, and the call merely traces server behavior.
>> + * For set, @context must result in at most one matching server reply, in
>> + * which case @info->meta_base_allocation_id is set to the resulting id.
>
> Hmm, looks too cheating. Then it should be renamed to
> nbd_negotiate_base_allocation
>
> and parameter @context should be renamed to @x_dirty_bitmap,
>
> and if it is unset, we'll use "base:allocation" here.
No, the next patch uses "qemu:" as the context for LIST when recursing
to work around qemu-nbd 3.0 not advertising the dirty-bitmap context
under list with 0 queries.
>
> but in this case, it still weird about opt=list case.. So, it should be named
> like nbd_negotiation_helper, as it is doing several different things, which
> I can't describe in one word.
Yes, I think a function rename will help.
>
>> + * return 1 for successful negotiation,
>> + * 0 if operation is unsupported or context unavailable,
>> * -1 with errp set for any other error
>
> this return value description looks not very related to opt=list case
It's related - but list returns 1 for a lot more cases than set (a
successful negotiation for list meant that all server replies were
processed, while a successful negotiation for set requires that the
server replies with exactly the one context we requested).
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 08/14] nbd/client: Refactor nbd_receive_list()
2018-12-06 14:18 ` Vladimir Sementsov-Ogievskiy
@ 2018-12-06 16:31 ` Eric Blake
2018-12-06 17:03 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 65+ messages in thread
From: Eric Blake @ 2018-12-06 16:31 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: jsnow, nsoffer, rjones, qemu-block
On 12/6/18 8:18 AM, Vladimir Sementsov-Ogievskiy wrote:
> 01.12.2018 1:03, Eric Blake wrote:
>> Add some parameters to make this function reusable in upcoming
>> export listing, where we will want to capture the name and
>> description rather than compare against a user-supplied name.
>> No change in semantics to the existing caller.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>> nbd/client.c | 66 +++++++++++++++++++++++++++++++++++++---------------
>> 1 file changed, 47 insertions(+), 19 deletions(-)
>> @@ -290,30 +298,49 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
>> nbd_send_opt_abort(ioc);
>> return -1;
>> }
>> - if (namelen != strlen(want)) {
>> - if (nbd_drop(ioc, len, errp) < 0) {
>> - error_prepend(errp,
>> - "failed to skip export name with wrong length: ");
>> - nbd_send_opt_abort(ioc);
>> - return -1;
>> + if (want) {
>> + if (namelen != strlen(want)) {
>> + if (nbd_drop(ioc, len, errp) < 0) {
>> + error_prepend(errp,
>> + "failed to skip export name with wrong length: ");
>> + nbd_send_opt_abort(ioc);
>> + return -1;
>> + }
>> + return 1;
>> }
>> - return 1;
>> + assert(namelen < sizeof(array));
>> + } else {
>> + assert(nameout);
>
> this assert looks a bit redundant, if nameout is 0, next line will abort not less clearly
>
>> + *nameout = name = g_new(char, namelen + 1);
>
> We should check namelen <= NBD_MAX_NAME_SIZE before it, I think.
Why? We already validated that the overall option is not too large, and
even if the resulting name from the server is larger than
NBD_MAX_NAME_SIZE, it's still nice to report that name to the client for
'qemu-nbd --list'. And these days, we only call nbd_receive_list if a
server lacked NBD_OPT_GO, which is getting rarer and rarer, so
micro-optimizing a rare case to avoid a large malloc isn't going to make
a noticeable difference.
>
>> }
>>
>> - assert(namelen < sizeof(name));
>> if (nbd_read(ioc, name, namelen, errp) < 0) {
>> error_prepend(errp, "failed to read export name: ");
>> nbd_send_opt_abort(ioc);
>> + if (!want) {
>> + free(name);
>
> g_free
>
>> + }
>> return -1;
>> }
>> name[namelen] = '\0';
>> len -= namelen;
>> - if (nbd_drop(ioc, len, errp) < 0) {
>> + if (!want) {
>> + assert(description);
>
> NBD_MAX_NAME_SIZE
The description is not bound by NBD_MAX_NAME_SIZE. The NBD protocol
DOES document a maximum string size (4k), but we are (so far) not
actually honoring that limit (our choice for NBD_MAX_NAME_SIZE is 256,
which is smaller than the NBD protocol limit).
>
>> + *description = g_new(char, len + 1);
>> + if (nbd_read(ioc, *description, len, errp) < 0) {
>> + error_prepend(errp, "failed to read export description: ");
>> + nbd_send_opt_abort(ioc);
>> + free(name);
>> + free(*description);
>
> g_free
>
>> + return -1;
>> + }
>> + (*description)[len] = '\0';
>> + } else if (nbd_drop(ioc, len, errp) < 0) {
>> error_prepend(errp, "failed to read export description: ");
>> nbd_send_opt_abort(ioc);
>> return -1;
>> }
>> - if (!strcmp(name, want)) {
>> + if (want && !strcmp(name, want)) {
>> *match = true;
>> }
>> return 1;
>
> one more thing: on fail path, you finally fill output name and description
> with freed pointers. I'd prefer to keep them unchanged in this case, however,
> it's a matter of taste.
Okay, I'll try and be more careful in v2 about not altering the callers
pointers on failure.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 08/14] nbd/client: Refactor nbd_receive_list()
2018-12-06 16:31 ` Eric Blake
@ 2018-12-06 17:03 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-06 17:03 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: jsnow, nsoffer, rjones, qemu-block
06.12.2018 19:31, Eric Blake wrote:
> On 12/6/18 8:18 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 01.12.2018 1:03, Eric Blake wrote:
>>> Add some parameters to make this function reusable in upcoming
>>> export listing, where we will want to capture the name and
>>> description rather than compare against a user-supplied name.
>>> No change in semantics to the existing caller.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>> ---
>>> nbd/client.c | 66 +++++++++++++++++++++++++++++++++++++---------------
>>> 1 file changed, 47 insertions(+), 19 deletions(-)
>
>>> @@ -290,30 +298,49 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
>>> nbd_send_opt_abort(ioc);
>>> return -1;
>>> }
>>> - if (namelen != strlen(want)) {
>>> - if (nbd_drop(ioc, len, errp) < 0) {
>>> - error_prepend(errp,
>>> - "failed to skip export name with wrong length: ");
>>> - nbd_send_opt_abort(ioc);
>>> - return -1;
>>> + if (want) {
>>> + if (namelen != strlen(want)) {
>>> + if (nbd_drop(ioc, len, errp) < 0) {
>>> + error_prepend(errp,
>>> + "failed to skip export name with wrong length: ");
>>> + nbd_send_opt_abort(ioc);
>>> + return -1;
>>> + }
>>> + return 1;
>>> }
>>> - return 1;
>>> + assert(namelen < sizeof(array));
>>> + } else {
>>> + assert(nameout);
>>
>> this assert looks a bit redundant, if nameout is 0, next line will abort not less clearly
>>
>>> + *nameout = name = g_new(char, namelen + 1);
>>
>> We should check namelen <= NBD_MAX_NAME_SIZE before it, I think.
>
> Why? We already validated
Ah, than OK, the worst case is ~ NBD_MAX_BUFFER_SIZE (32M), not 4G. Missed that :(
that the overall option is not too large, and even if the resulting name from the server is larger than NBD_MAX_NAME_SIZE, it's still nice to report that name to the client for 'qemu-nbd --list'. And these days, we only call nbd_receive_list if a server lacked NBD_OPT_GO, which is getting rarer and rarer, so micro-optimizing a rare case to avoid a large malloc isn't going to make a noticeable difference.
>
>>
>>> }
>>>
>>> - assert(namelen < sizeof(name));
>>> if (nbd_read(ioc, name, namelen, errp) < 0) {
>>> error_prepend(errp, "failed to read export name: ");
>>> nbd_send_opt_abort(ioc);
>>> + if (!want) {
>>> + free(name);
>>
>> g_free
>>
>>> + }
>>> return -1;
>>> }
>>> name[namelen] = '\0';
>>> len -= namelen;
>>> - if (nbd_drop(ioc, len, errp) < 0) {
>>> + if (!want) {
>>> + assert(description);
>>
>> NBD_MAX_NAME_SIZE
>
> The description is not bound by NBD_MAX_NAME_SIZE. The NBD protocol DOES document a maximum string size (4k), but we are (so far) not actually honoring that limit (our choice for NBD_MAX_NAME_SIZE is 256, which is smaller than the NBD protocol limit).
>
Ohm, yes, you are right :(. Too much reviewing, I should stop and make some patches :)
>>
>>> + *description = g_new(char, len + 1);
>>> + if (nbd_read(ioc, *description, len, errp) < 0) {
>>> + error_prepend(errp, "failed to read export description: ");
>>> + nbd_send_opt_abort(ioc);
>>> + free(name);
>>> + free(*description);
>>
>> g_free
>>
>>> + return -1;
>>> + }
>>> + (*description)[len] = '\0';
>>> + } else if (nbd_drop(ioc, len, errp) < 0) {
>>> error_prepend(errp, "failed to read export description: ");
>>> nbd_send_opt_abort(ioc);
>>> return -1;
>>> }
>>> - if (!strcmp(name, want)) {
>>> + if (want && !strcmp(name, want)) {
>>> *match = true;
>>> }
>>> return 1;
>>
>> one more thing: on fail path, you finally fill output name and description
>> with freed pointers. I'd prefer to keep them unchanged in this case, however,
>> it's a matter of taste.
>
> Okay, I'll try and be more careful in v2 about not altering the callers pointers on failure.
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 10/14] nbd/client: Split handshake into two functions
2018-12-06 15:16 ` Vladimir Sementsov-Ogievskiy
@ 2018-12-06 17:06 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-06 17:06 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: jsnow, nsoffer, rjones, qemu-block
06.12.2018 18:16, Vladimir Sementsov-Ogievskiy wrote:
> 01.12.2018 1:03, Eric Blake wrote:
>> + * Returns: negative errno: failure talking to server
>> + * 0: server is oldstyle, client must still parse export size
>> + * 1: server is newstyle, but can only accept EXPORT_NAME
>> + * 2: server is newstyle, but lacks structured replies
>> + * 3: server is newstyle and set up for structured replies
>> + */
>
> hmm, may be, introduce enum of named constants, instead of raw numbers?
>
> NBD_STARTED_OLD_STYLE
> NBD_STARTED_NEW_STYLE
> NBD_STARTED_NEW_STYLE_FIXED
> NBD_STARTED_STRUCTURED_REPLIES
>
> or, at least, add short comment after each return statement, to not scroll up
> every time (like you gracefully do after each case: statement).
>
> However, I'm okay with either way.
>
I mean, including "as is".
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 11/14] nbd/client: Add nbd_receive_export_list()
2018-11-30 22:03 ` [Qemu-devel] [PATCH 11/14] nbd/client: Add nbd_receive_export_list() Eric Blake
2018-12-01 10:45 ` Richard W.M. Jones
@ 2018-12-07 10:04 ` Vladimir Sementsov-Ogievskiy
2018-12-07 15:19 ` Eric Blake
2018-12-07 10:07 ` Vladimir Sementsov-Ogievskiy
2 siblings, 1 reply; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-07 10:04 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: jsnow, nsoffer, rjones, qemu-block, Kevin Wolf, Max Reitz
01.12.2018 1:03, Eric Blake wrote:
> We want to be able to detect whether a given qemu NBD server is
> exposing the right export(s) and dirty bitmaps, at least for
> regression testing. We could use 'nbd-client -l' from the upstream
> NBD project to list exports, but it's annoying to rely on
> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
> know about all of the qemu NBD extensions. Thus, we plan on adding
> a new mode to qemu-nbd that merely sniffs all possible information
> from the server during handshake phase, then disconnects and dumps
> the information.
>
> This patch adds the low-level client code for grabbing the list
> of exports. It benefits from the recent refactoring patches, as
> well as a minor tweak of changing nbd_opt_go() to nbd_opt_info_go(),
> in order to share as much code as possible when it comes to doing
> validation of server replies. The resulting information is stored
> in an array of NBDExportInfo which has been expanded to hold more
> members, along with a convenience function for freeing the list.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> include/block/nbd.h | 15 +++-
> nbd/client.c | 166 ++++++++++++++++++++++++++++++++++++++++++--
> nbd/trace-events | 2 +-
> 3 files changed, 174 insertions(+), 9 deletions(-)
>
> diff --git a/include/block/nbd.h b/include/block/nbd.h
> index 65feff8ba96..74d006b8d62 100644
> --- a/include/block/nbd.h
> +++ b/include/block/nbd.h
> @@ -262,6 +262,9 @@ struct NBDExportInfo {
> /* Set by client before nbd_receive_negotiate() */
> bool request_sizes;
> char *x_dirty_bitmap;
> +
> + /* Set by client before nbd_receive_negotiate(), or by server results
> + * during nbd_receive_export_list() */
> char *name; /* must be non-NULL */
>
> /* In-out fields, set by client before nbd_receive_negotiate() and
> @@ -269,7 +272,8 @@ struct NBDExportInfo {
> bool structured_reply;
> bool base_allocation; /* base:allocation context for NBD_CMD_BLOCK_STATUS */
>
> - /* Set by server results during nbd_receive_negotiate() */
> + /* Set by server results during nbd_receive_negotiate() and
> + * nbd_receive_export_list() */
> uint64_t size;
> uint16_t flags;
> uint32_t min_block;
> @@ -277,12 +281,21 @@ struct NBDExportInfo {
> uint32_t max_block;
>
> uint32_t meta_base_allocation_id;
> +
> + /* Set by server results during nbd_receive_export_list() */
> + char *description;
> + int n_contexts;
> + char **contexts;
> };
> typedef struct NBDExportInfo NBDExportInfo;
>
> int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> const char *hostname, QIOChannel **outioc,
> NBDExportInfo *info, Error **errp);
> +void nbd_free_export_list(NBDExportInfo *info, int count);
> +int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> + const char *hostname, NBDExportInfo **info,
> + Error **errp);
> int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
> Error **errp);
> int nbd_send_request(QIOChannel *ioc, NBDRequest *request);
> diff --git a/nbd/client.c b/nbd/client.c
> index a282712724d..6292de560ee 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -351,7 +351,8 @@ static int nbd_receive_list(QIOChannel *ioc, const char *want, bool *match,
> * used, 0 if NBD_OPT_GO is unsupported (fall back to NBD_OPT_LIST and
> * NBD_OPT_EXPORT_NAME in that case), and > 0 if the export is good to
> * go (with the rest of @info populated). */
> -static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
> +static int nbd_opt_info_go(QIOChannel *ioc, uint32_t opt,
> + NBDExportInfo *info, Error **errp)
> {
> NBDOptionReply reply;
> uint32_t len = strlen(info->name);
> @@ -364,7 +365,8 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
> * flags still 0 is a witness of a broken server. */
> info->flags = 0;
>
> - trace_nbd_opt_go_start(info->name);
> + assert(opt == NBD_OPT_GO || opt == NBD_OPT_INFO);
> + trace_nbd_opt_go_start(nbd_opt_lookup(opt), info->name);
> buf = g_malloc(4 + len + 2 + 2 * info->request_sizes + 1);
> stl_be_p(buf, len);
> memcpy(buf + 4, info->name, len);
> @@ -373,7 +375,7 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
> if (info->request_sizes) {
> stw_be_p(buf + 4 + len + 2, NBD_INFO_BLOCK_SIZE);
> }
> - error = nbd_send_option_request(ioc, NBD_OPT_GO,
> + error = nbd_send_option_request(ioc, opt,
> 4 + len + 2 + 2 * info->request_sizes,
> buf, errp);
> g_free(buf);
> @@ -382,7 +384,7 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo *info, Error **errp)
> }
>
> while (1) {
> - if (nbd_receive_option_reply(ioc, NBD_OPT_GO, &reply, errp) < 0) {
> + if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0) {
> return -1;
> }
> error = nbd_handle_reply_err(ioc, &reply, errp);
> @@ -733,8 +735,12 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> nbd_send_opt_abort(ioc);
> return -1;
> }
> + g_free(name);
> + } else {
> + info->contexts = g_renew(char *, info->contexts,
> + ++info->n_contexts);
> + info->contexts[info->n_contexts - 1] = name;
> }
> - g_free(name);
> received = true;
>
> /* receive NBD_REP_ACK */
> @@ -828,7 +834,9 @@ static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> clientflags |= NBD_FLAG_C_FIXED_NEWSTYLE;
> }
> if (globalflags & NBD_FLAG_NO_ZEROES) {
> - *zeroes = false;
> + if (zeroes) {
> + *zeroes = false;
> + }
> clientflags |= NBD_FLAG_C_NO_ZEROES;
> }
> /* client requested flags */
> @@ -921,7 +929,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> * TLS). If it is not available, fall back to
> * NBD_OPT_LIST for nicer error messages about a missing
> * export, then use NBD_OPT_EXPORT_NAME. */
> - result = nbd_opt_go(ioc, info, errp);
> + result = nbd_opt_info_go(ioc, NBD_OPT_GO, info, errp);
> if (result < 0) {
> return -EINVAL;
> }
> @@ -993,6 +1001,150 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> return 0;
> }
>
> +/* Clean up result of nbd_receive_export_list */
> +void nbd_free_export_list(NBDExportInfo *info, int count)
> +{
> + int i, j;
> +
> + for (i = 0; info && i < count; i++) {
> + free(info[i].name);
> + free(info[i].description);
> + for (j = 0; j < info[i].n_contexts; j++) {
> + free(info[i].contexts[j]);
> + }
> + free(info[i].contexts);
> + }
> + free(info);
> +}
g_free
> +
> +/* Query details about a server's exports, then disconnect without
> + * going into transmission phase. Return a count of the exports listed
> + * in @info by the server, or -1 on error. Caller must free @info. */
> +int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
> + const char *hostname, NBDExportInfo **info,
> + Error **errp)
> +{
> + int result;
> + int count = 0;
> + int i;
> + int rc;
> + int ret = -1;
> + NBDExportInfo *array = NULL;
> + uint32_t oldflags;
> + QIOChannel *sioc = NULL;
it's a bit confusing that you use sioc name for the second produced ioc, when
in nbd_client_init it's visa-versa.
So, I assume that you mean Secure ioc, when in nbd_client_init it is Socket ioc.
> + bool try_context = true;
> +
> + *info = NULL;
> + result = nbd_start_negotiate(ioc, tlscreds, hostname, &sioc, true, NULL,
> + errp);
> + if (tlscreds && sioc) {
> + ioc = sioc;
> + }
> +
> + switch (result) {
> + case 2:
> + /* meta contexts are only useful with structured reply */
> + try_context = false;
> + /* fall through */
> + case 3:
> + /* newstyle - use NBD_OPT_LIST to populate array, then try
> + * NBD_OPT_INFO on each array member. If structured replies
> + * are enabled, also try NBD_OPT_LIST_META_CONTEXT. */
> + if (nbd_send_option_request(ioc, NBD_OPT_LIST, 0, NULL, errp) < 0) {
> + goto out;
> + }
> + while (1) {
> + char *name;
> + char *desc;
> +
> + rc = nbd_receive_list(ioc, NULL, NULL, &name, &desc, errp);
> + if (rc < 0) {
> + goto out;
> + } else if (rc == 0) {
> + break;
> + }
> + array = g_renew(NBDExportInfo, array, ++count);
> + memset(&array[count - 1], 0, sizeof(*array));
> + array[count - 1].name = name;
> + array[count - 1].description = desc;
> + array[count - 1].structured_reply = result == 3;
> + }
> +
> + for (i = 0; i < count; i++) {
> + array[i].request_sizes = true;
> + rc = nbd_opt_info_go(ioc, NBD_OPT_INFO, &array[i], errp);
> + if (rc < 0) {
> + goto out;
> + } else if (rc == 0) {
> + /* Pointless to try rest of loop. If OPT_INFO doesn't work,
> + * it's unlikely that meta contexts work either */
> + break;
> + }
> +
> + if (try_context) {
> + rc = nbd_negotiate_simple_meta_context(
> + ioc, NBD_OPT_LIST_META_CONTEXT, NULL, &array[i], errp);
> + if (rc < 0) {
> + goto out;
> + } else if (rc == 0) {
> + try_context = false;
> + }
> + }
> + }
> +
> + /* Send NBD_OPT_ABORT as a courtesy before hanging up */
> + nbd_send_opt_abort(ioc);
> + break;
> + case 1: /* newstyle, but limited to EXPORT_NAME */
> + error_setg(errp, "Server does not support export lists");
> + /* We can't even send NBD_OPT_ABORT, so merely hang up */
> + goto out;
> + case 0: /* oldstyle, parse length and flags */
> + array = g_new0(NBDExportInfo, 1);
> + array->name = g_strdup("");
> + count = 1;
> +
> + if (nbd_read(ioc, &array->size, sizeof(array->size), errp) < 0) {
> + error_prepend(errp, "Failed to read export length: ");
> + goto out;
> + }
> + array->size = be64_to_cpu(array->size);
> +
> + if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
> + error_prepend(errp, "Failed to read export flags: ");
> + goto out;
> + }
> + oldflags = be32_to_cpu(oldflags);
> + if (oldflags & ~0xffff) {
> + error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
> + goto out;
> + }
> + array->flags = oldflags;
^^^
this is a common part of nbd_receive_export_list and nbd_receive_negotiate,
can we move it to nbd_start_negotiate?
> +
> + /* Send NBD_CMD_DISC as a courtesy to the server, but ignore all
> + * errors now that we have the information we wanted. */
> + if (nbd_drop(ioc, 124, NULL) == 0) {
> + NBDRequest request = { .type = NBD_CMD_DISC };
> +
> + nbd_send_request(ioc, &request);
> + }
> + break;
> + default:
> + goto out;
> + }
> +
> + *info = array;
> + array = NULL;
> + ret = count;
tricky thing. shouldn't we set count to 0 too after it?
aha, no, go to nbd_free_export_list and see
info && i < count
so, it checks.
> +
> + out:
> + qio_channel_shutdown(ioc, QIO_CHANNEL_SHUTDOWN_BOTH, NULL);
> + qio_channel_close(ioc, NULL);
interesting, why we don't close it (only shutdown) in other nbd code..
> + object_unref(OBJECT(sioc));
> + nbd_free_export_list(array, count); > + return ret;
> +}
> +
> #ifdef __linux__
> int nbd_init(int fd, QIOChannelSocket *sioc, NBDExportInfo *info,
> Error **errp)
> diff --git a/nbd/trace-events b/nbd/trace-events
> index 570b04997ff..2cf83ebed15 100644
> --- a/nbd/trace-events
> +++ b/nbd/trace-events
> @@ -2,7 +2,7 @@
> nbd_send_option_request(uint32_t opt, const char *name, uint32_t len) "Sending option request %" PRIu32" (%s), len %" PRIu32
> nbd_receive_option_reply(uint32_t option, const char *optname, uint32_t type, const char *typename, uint32_t length) "Received option reply %" PRIu32" (%s), type %" PRIu32" (%s), len %" PRIu32
> nbd_reply_err_unsup(uint32_t option, const char *name) "server doesn't understand request %" PRIu32 " (%s), attempting fallback"
> -nbd_opt_go_start(const char *name) "Attempting NBD_OPT_GO for export '%s'"
> +nbd_opt_go_start(const char *opt, const char *name) "Attempting %s for export '%s'"
> nbd_opt_go_success(void) "Export is good to go"
> nbd_opt_go_info_unknown(int info, const char *name) "Ignoring unknown info %d (%s)"
> nbd_opt_go_info_block_size(uint32_t minimum, uint32_t preferred, uint32_t maximum) "Block sizes are 0x%" PRIx32 ", 0x%" PRIx32 ", 0x%" PRIx32
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 11/14] nbd/client: Add nbd_receive_export_list()
2018-11-30 22:03 ` [Qemu-devel] [PATCH 11/14] nbd/client: Add nbd_receive_export_list() Eric Blake
2018-12-01 10:45 ` Richard W.M. Jones
2018-12-07 10:04 ` Vladimir Sementsov-Ogievskiy
@ 2018-12-07 10:07 ` Vladimir Sementsov-Ogievskiy
2 siblings, 0 replies; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-07 10:07 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: jsnow, nsoffer, rjones, qemu-block, Kevin Wolf, Max Reitz
01.12.2018 1:03, Eric Blake wrote:
> +/* Clean up result of nbd_receive_export_list */
> +void nbd_free_export_list(NBDExportInfo *info, int count)
> +{
> + int i, j;
personally, I'd prefer explicit
if (!info) {
return;
}
here, it's more obvious, and info is unchanging, strange to check it in
a loop.
> +
> + for (i = 0; info && i < count; i++) {
> + free(info[i].name);
> + free(info[i].description);
> + for (j = 0; j < info[i].n_contexts; j++) {
> + free(info[i].contexts[j]);
> + }
> + free(info[i].contexts);
> + }
> + free(info);
> +}
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 12/14] nbd/client: Work around 3.0 bug for listing meta contexts
2018-11-30 22:03 ` [Qemu-devel] [PATCH 12/14] nbd/client: Work around 3.0 bug for listing meta contexts Eric Blake
@ 2018-12-07 11:21 ` Vladimir Sementsov-Ogievskiy
2018-12-07 15:21 ` Eric Blake
0 siblings, 1 reply; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-07 11:21 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: jsnow, nsoffer, rjones, qemu-block
01.12.2018 1:03, Eric Blake wrote:
> Commit 3d068aff forgot to advertise available qemu: contexts
> when the client requests a list with 0 queries. Furthermore,
> 3.0 shipped with a qemu-img hack of x-dirty-bitmap (commit
> 216ee365) that _silently_ acts as though the entire image is
> clean if a requested bitmap is not present. Both bugs have
> been recently fixed to give full output from the start, and
> to refuse a connection if a requested x-dirty-bitmap was not
> found.
>
> Still, it is likely that there will be users that have to
> work with a mix of old and new qemu versions, depending on
> which features get backported where, at which point being
> able to rely on 'qemu-img --list' output to know for sure
> whether a given NBD export has the desired dirty bitmap is
> much nicer than blindly connecting and risking that the
> entire image may appear clean. We can make our --list code
> smart enough to work around buggy servers by tracking
> whether we've seen any qemu: replies in the original 0-query
> list; if not, recurse to a single query on "qemu:" (which
> may still have no replies, but then we know for sure we
> didn't trip up on the server bug).
related thing:
should we document these bugs with corresponding version numbers in docs/interop/nbd.txt?
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> Done as a separate patch to make it easier to revert when we no
> longer care about 3.0 servers
> ---
> nbd/client.c | 11 ++++++++++-
> 1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/nbd/client.c b/nbd/client.c
> index 6292de560ee..928ecabd420 100644
> --- a/nbd/client.c
> +++ b/nbd/client.c
> @@ -21,6 +21,7 @@
> #include "qapi/error.h"
> #include "trace.h"
> #include "nbd-internal.h"
> +#include "qemu/cutils.h"
>
> /* Definitions for opaque data types */
>
> @@ -736,12 +737,13 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> return -1;
> }
> g_free(name);
> + received = true;
> } else {
> info->contexts = g_renew(char *, info->contexts,
> ++info->n_contexts);
> info->contexts[info->n_contexts - 1] = name;
> + received |= strstart(name, "qemu:", NULL);
so, for _LIST_, it is actually received_qemu var. It has taken some time for me
to understand that it's ok (turns out, that this variable actually isn't used for
_LIST_ case).. Personally I don't like it, for me it's too tricky, but anyway, you
said, you'll refactor this function somehow.
> }
> - received = true;
>
> /* receive NBD_REP_ACK */
> if (nbd_receive_option_reply(ioc, opt, &reply, errp) < 0) {
> @@ -771,6 +773,13 @@ static int nbd_negotiate_simple_meta_context(QIOChannel *ioc,
> info->meta_base_allocation_id = received_id;
> }
>
> + /* Recurse to work around qemu 3.0 bug - the server forgot to send
> + * "qemu:" replies to 0 queries. */
> + if (!context && !received) {
> + return nbd_negotiate_simple_meta_context(ioc, opt, "qemu:", info,
> + errp);
> + }
> +
> return received || opt == NBD_OPT_LIST_META_CONTEXT;
> }
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 13/14] qemu-nbd: Add --list option
2018-11-30 22:03 ` [Qemu-devel] [PATCH 13/14] qemu-nbd: Add --list option Eric Blake
2018-12-01 10:58 ` Richard W.M. Jones
@ 2018-12-07 12:48 ` Vladimir Sementsov-Ogievskiy
2018-12-07 15:36 ` Eric Blake
1 sibling, 1 reply; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-07 12:48 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: jsnow, nsoffer, rjones, qemu-block
01.12.2018 1:03, Eric Blake wrote:
> We want to be able to detect whether a given qemu NBD server is
> exposing the right export(s) and dirty bitmaps, at least for
> regression testing. We could use 'nbd-client -l' from the upstream
> NBD project to list exports, but it's annoying to rely on
> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
> know about all of the qemu NBD extensions. Thus, we plan on adding
ha, in this patch, not plan but add:)
> a new mode to qemu-nbd that merely sniffs all possible information
> from the server during handshake phase, then disconnects and dumps
> the information.
>
> This patch actually implements --list/-L, while reusing other
> options such as --tls-creds for now designating how to connect
> as the client (rather than their non-list usage of how to operate
> as the server).
>
> I debated about adding this functionality to something akin to
> 'qemu-img info' - but that tool does not readily lend itself
> to connecting to an arbitrary NBD server without also tying to
> a specific export (I may, however, still add ImageInfoSpecificNBD
> for reporting the bitmaps available when connecting to a single
> export). And, while it may feel a bit odd that normally
> qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not
> really making the qemu-nbd binary that much larger, because
> 'qemu-nbd -c' has to operate as both server and client
> simultaneously across two threads when feeding the kernel module
> for /dev/nbdN access.
>
> Sample output:
> $ qemu-nbd -L
> exports available: 1
> export: ''
> size: 65536
> flags: 0x4ed ( flush fua trim zeroes df cache )
> min block: 512
> opt block: 4096
> max block: 33554432
> available meta contexts: 1
> base:allocation
don't you plan to bind this all to QAPI and expose in json?
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> qemu-nbd.c | 153 ++++++++++++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 141 insertions(+), 12 deletions(-)
>
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index c57053a0795..e19a841b869 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -76,6 +76,7 @@ static void usage(const char *name)
> {
> (printf) (
> "Usage: %s [OPTIONS] FILE\n"
> +" or: %s -L [OPTIONS]\n"
> "QEMU Disk Network Block Device Server\n"
Do anyone know, why thunderbird add additional space to the lines started from space when quoting,
which breaks indentation in quoted patches? How to fix it? I use plain text.
> "\n"
> " -h, --help display this help and exit\n"
> @@ -97,6 +98,7 @@ static void usage(const char *name)
> " -P, --partition=NUM only expose partition NUM\n"
> "\n"
> "General purpose options:\n"
> +" -L, --list list NBD exports visible to client\n"
hm. I think, user who things that qemu-nbd is only a server, can understand this as
"dry run, don't actually start the server, but only list exports, which will be
available to clients, keeping in mind all other specified options".
so, may be, "list remote NBD server exports and options" or something like this?
> " --object type,id=ID,... define an object such as 'secret' for providing\n"
> " passwords and/or encryption keys\n"
> " --tls-creds=ID use id of an earlier --object to provide TLS\n"
> @@ -130,7 +132,7 @@ static void usage(const char *name)
> " --image-opts treat FILE as a full set of image options\n"
> "\n"
> QEMU_HELP_BOTTOM "\n"
> - , name, NBD_DEFAULT_PORT, "DEVICE");
> + , name, name, NBD_DEFAULT_PORT, "DEVICE");
> }
>
> static void version(const char *name)
> @@ -242,6 +244,92 @@ static void termsig_handler(int signum)
> }
>
>
> +static int qemu_nbd_client_list(SocketAddress *saddr, QCryptoTLSCreds *tls,
> + const char *hostname)
> +{
> + int ret = EXIT_FAILURE;
> + int rc;
> + Error *err = NULL;
> + QIOChannelSocket *sioc;
> + NBDExportInfo *list;
> + int i, j;
> +
> + sioc = qio_channel_socket_new();
> + if (qio_channel_socket_connect_sync(sioc, saddr, &err) < 0) {
> + error_report_err(err);
> + goto out;
> + }
> + rc = nbd_receive_export_list(QIO_CHANNEL(sioc), tls, hostname, &list,
> + &err);
> + if (rc < 0) {
> + if (err) {
> + error_report_err(err);
> + }
> + goto out_socket;
> + }
> + printf("exports available: %d\n", rc);
> + for (i = 0; i < rc; i++) {
> + printf(" export: '%s'\n", list[i].name);
> + if (list[i].description && *list[i].description) {
> + printf(" description: %s\n", list[i].description);
> + }
> + if (list[i].flags & NBD_FLAG_HAS_FLAGS) {
> + printf(" size: %" PRIu64 "\n", list[i].size);
size is available only if NBD_FLAG_HAS_FLAGS? at least, accordingly to code,
in case 0: size is unrelated to flags.
> + printf(" flags: 0x%x (", list[i].flags);
> + if (list[i].flags & NBD_FLAG_READ_ONLY) {
> + printf(" readonly");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_FLUSH) {
> + printf(" flush");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_FUA) {
> + printf(" fua");
> + }
> + if (list[i].flags & NBD_FLAG_ROTATIONAL) {
> + printf(" rotational");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_TRIM) {
> + printf(" trim");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_WRITE_ZEROES) {
> + printf(" zeroes");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_DF) {
> + printf(" df");
> + }
> + if (list[i].flags & NBD_FLAG_CAN_MULTI_CONN) {
> + printf(" multi");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_RESIZE) {
> + printf(" resize");
> + }
> + if (list[i].flags & NBD_FLAG_SEND_CACHE) {
> + printf(" cache");
> + }
> + printf(" )\n");
> + }
> + if (list[i].min_block) {
> + printf(" min block: %u\n", list[i].min_block);
> + printf(" opt block: %u\n", list[i].opt_block);
> + printf(" max block: %u\n", list[i].max_block);
> + }
> + if (list[i].n_contexts) {
> + printf(" available meta contexts: %d\n", list[i].n_contexts);
> + for (j = 0; j < list[i].n_contexts; j++) {
> + printf(" %s\n", list[i].contexts[j]);
> + }
> + }
> + }
> + nbd_free_export_list(list, rc);
> +
> + ret = EXIT_SUCCESS;
> + out_socket:
> + object_unref(OBJECT(sioc));
> + out:
> + return ret;
> +}
> +
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 14/14] iotests: Enhance 223, 233 to cover 'qemu-nbd --list'
2018-11-30 22:03 ` [Qemu-devel] [PATCH 14/14] iotests: Enhance 223, 233 to cover 'qemu-nbd --list' Eric Blake
2018-12-01 11:04 ` Richard W.M. Jones
@ 2018-12-07 13:08 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-07 13:08 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: jsnow, nsoffer, rjones, qemu-block, Kevin Wolf, Max Reitz
01.12.2018 1:03, Eric Blake wrote:
> Any good new feature deserves some regression testing :)
> Coverage includes:
> - 223: what happens when there are 0 or more than 1 export,
> proof that we can see multiple contexts including qemu:dirty-bitmap
> - 233: proof that we can list over TLS, and that mix-and-match of
> plain/TLS listings sanely
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
[..]
>
> === Contrast normal status to large granularity dirty-bitmap ===
>
> diff --git a/tests/qemu-iotests/233 b/tests/qemu-iotests/233
> index 1814efe3333..5d694d9d242 100755
> --- a/tests/qemu-iotests/233
> +++ b/tests/qemu-iotests/233
> @@ -72,6 +72,9 @@ $QEMU_IMG info --image-opts \
> --object tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0 \
> driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
> 2>&1 | sed "s/$nbd_tcp_port/PORT/g"
> +$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port \
> + --object tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0 \
> + --tls-creds=tls0
--object parameter may be stored to a variable before these two commands, to not duplicate,
and in following cases too.
with or without:
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 11/14] nbd/client: Add nbd_receive_export_list()
2018-12-07 10:04 ` Vladimir Sementsov-Ogievskiy
@ 2018-12-07 15:19 ` Eric Blake
0 siblings, 0 replies; 65+ messages in thread
From: Eric Blake @ 2018-12-07 15:19 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: jsnow, nsoffer, rjones, qemu-block, Kevin Wolf, Max Reitz
On 12/7/18 4:04 AM, Vladimir Sementsov-Ogievskiy wrote:
> 01.12.2018 1:03, Eric Blake wrote:
>> We want to be able to detect whether a given qemu NBD server is
>> exposing the right export(s) and dirty bitmaps, at least for
>> regression testing. We could use 'nbd-client -l' from the upstream
>> NBD project to list exports, but it's annoying to rely on
>> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
>> know about all of the qemu NBD extensions. Thus, we plan on adding
>> a new mode to qemu-nbd that merely sniffs all possible information
>> from the server during handshake phase, then disconnects and dumps
>> the information.
>>
>> This patch adds the low-level client code for grabbing the list
>> of exports. It benefits from the recent refactoring patches, as
>> well as a minor tweak of changing nbd_opt_go() to nbd_opt_info_go(),
>> in order to share as much code as possible when it comes to doing
>> validation of server replies. The resulting information is stored
>> in an array of NBDExportInfo which has been expanded to hold more
>> members, along with a convenience function for freeing the list.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>> +
>> +/* Query details about a server's exports, then disconnect without
>> + * going into transmission phase. Return a count of the exports listed
>> + * in @info by the server, or -1 on error. Caller must free @info. */
>> +int nbd_receive_export_list(QIOChannel *ioc, QCryptoTLSCreds *tlscreds,
>> + const char *hostname, NBDExportInfo **info,
>> + Error **errp)
>> +{
>> + int result;
>> + int count = 0;
>> + int i;
>> + int rc;
>> + int ret = -1;
>> + NBDExportInfo *array = NULL;
>> + uint32_t oldflags;
>> + QIOChannel *sioc = NULL;
>
> it's a bit confusing that you use sioc name for the second produced ioc, when
> in nbd_client_init it's visa-versa.
>
> So, I assume that you mean Secure ioc, when in nbd_client_init it is Socket ioc.
I can s/sioc/outioc/ if that helps. Basically, the output ioc is the
same as the original ioc for plaintext, and a secure wrapper socket for tls.
>> + case 0: /* oldstyle, parse length and flags */
>> + array = g_new0(NBDExportInfo, 1);
>> + array->name = g_strdup("");
>> + count = 1;
>> +
>> + if (nbd_read(ioc, &array->size, sizeof(array->size), errp) < 0) {
>> + error_prepend(errp, "Failed to read export length: ");
>> + goto out;
>> + }
>> + array->size = be64_to_cpu(array->size);
>> +
>> + if (nbd_read(ioc, &oldflags, sizeof(oldflags), errp) < 0) {
>> + error_prepend(errp, "Failed to read export flags: ");
>> + goto out;
>> + }
>> + oldflags = be32_to_cpu(oldflags);
>> + if (oldflags & ~0xffff) {
>> + error_setg(errp, "Unexpected export flags %0x" PRIx32, oldflags);
>> + goto out;
>> + }
>> + array->flags = oldflags;
>
>
> ^^^
> this is a common part of nbd_receive_export_list and nbd_receive_negotiate,
> can we move it to nbd_start_negotiate?
Not quite, but I could probably create yet another helper function.
>
>> +
>> + out:
>> + qio_channel_shutdown(ioc, QIO_CHANNEL_SHUTDOWN_BOTH, NULL);
>> + qio_channel_close(ioc, NULL);
>
> interesting, why we don't close it (only shutdown) in other nbd code..
I presume you mean block/nbd-client.c:nbd_teardown_connection, which
indeed oncly calls qio_channel_shutdown() followed by object_unref(). I
think that unref'ing a channel implies a close, but if not, then that
code is leaking an fd (shutdown ends the connection, but does not close
resources). I'll have to debug that, but it's independent of this patch.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 12/14] nbd/client: Work around 3.0 bug for listing meta contexts
2018-12-07 11:21 ` Vladimir Sementsov-Ogievskiy
@ 2018-12-07 15:21 ` Eric Blake
0 siblings, 0 replies; 65+ messages in thread
From: Eric Blake @ 2018-12-07 15:21 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: jsnow, nsoffer, rjones, qemu-block
On 12/7/18 5:21 AM, Vladimir Sementsov-Ogievskiy wrote:
> 01.12.2018 1:03, Eric Blake wrote:
>> Commit 3d068aff forgot to advertise available qemu: contexts
>> when the client requests a list with 0 queries. Furthermore,
>> 3.0 shipped with a qemu-img hack of x-dirty-bitmap (commit
>> 216ee365) that _silently_ acts as though the entire image is
>> clean if a requested bitmap is not present. Both bugs have
>> been recently fixed to give full output from the start, and
>> to refuse a connection if a requested x-dirty-bitmap was not
>> found.
>>
>> Still, it is likely that there will be users that have to
>> work with a mix of old and new qemu versions, depending on
>> which features get backported where, at which point being
>> able to rely on 'qemu-img --list' output to know for sure
>> whether a given NBD export has the desired dirty bitmap is
>> much nicer than blindly connecting and risking that the
>> entire image may appear clean. We can make our --list code
>> smart enough to work around buggy servers by tracking
>> whether we've seen any qemu: replies in the original 0-query
>> list; if not, recurse to a single query on "qemu:" (which
>> may still have no replies, but then we know for sure we
>> didn't trip up on the server bug).
>
> related thing:
> should we document these bugs with corresponding version numbers in docs/interop/nbd.txt?
Might be useful. I'll add that as another patch in my v2 series.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 13/14] qemu-nbd: Add --list option
2018-12-07 12:48 ` Vladimir Sementsov-Ogievskiy
@ 2018-12-07 15:36 ` Eric Blake
2018-12-07 16:49 ` Vladimir Sementsov-Ogievskiy
2018-12-07 16:49 ` Vladimir Sementsov-Ogievskiy
0 siblings, 2 replies; 65+ messages in thread
From: Eric Blake @ 2018-12-07 15:36 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: jsnow, nsoffer, rjones, qemu-block
On 12/7/18 6:48 AM, Vladimir Sementsov-Ogievskiy wrote:
> 01.12.2018 1:03, Eric Blake wrote:
>> We want to be able to detect whether a given qemu NBD server is
>> exposing the right export(s) and dirty bitmaps, at least for
>> regression testing. We could use 'nbd-client -l' from the upstream
>> NBD project to list exports, but it's annoying to rely on
>> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
>> know about all of the qemu NBD extensions. Thus, we plan on adding
>
> ha, in this patch, not plan but add:)
Yep, copy-and-paste of a common prefix across multiple patches. I'll
probably let the text diverge in v2 to be a bit more accurate per-patch,
at the loss of the nice copy-and-paste.
>> I debated about adding this functionality to something akin to
>> 'qemu-img info' - but that tool does not readily lend itself
>> to connecting to an arbitrary NBD server without also tying to
>> a specific export (I may, however, still add ImageInfoSpecificNBD
>> for reporting the bitmaps available when connecting to a single
>> export). And, while it may feel a bit odd that normally
>> qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not
>> really making the qemu-nbd binary that much larger, because
>> 'qemu-nbd -c' has to operate as both server and client
>> simultaneously across two threads when feeding the kernel module
>> for /dev/nbdN access.
>>
>> Sample output:
>> $ qemu-nbd -L
>> exports available: 1
>> export: ''
>> size: 65536
>> flags: 0x4ed ( flush fua trim zeroes df cache )
>> min block: 512
>> opt block: 4096
>> max block: 33554432
>> available meta contexts: 1
>> base:allocation
>
> don't you plan to bind this all to QAPI and expose in json?
No. As explained above, QAPI is very much centered on per-BDS actions,
while this action is a per-server action (where many servers have only
one export, but it is also possible to have a server with 0 exports or a
plurality of exports). I _can't_ fit this into query-block's ImageInfo
details about a block device, because we don't have a way of stating
'connect to this arbitrary server, create an unspecified number of block
devices, tell me about each of them, and then throw them all away
because we only wanted the info about what the server made available'.
The same is true for gluster or other remote access block devices - the
QMP commands pre-suppose you already know WHICH specific resource you
are accessing, rather than providing a way for you to query the remote
server about ALL resources available but without actually selecting
those resources.
I'm open to ideas about a new QMP command to do such a query, but who
would be the client? A management app that wants to hotplug a new NBD
device to a running guest can run 'qemu-nbd --list' just as easily as
they could run a new QMP command to learn what the server is offering.
Without a strong reason of a client that would need this in QMP, I don't
see the point in adding it to the qemu binary.
>> +++ b/qemu-nbd.c
>> @@ -76,6 +76,7 @@ static void usage(const char *name)
>> {
>> (printf) (
>> "Usage: %s [OPTIONS] FILE\n"
>> +" or: %s -L [OPTIONS]\n"
>> "QEMU Disk Network Block Device Server\n"
>
> Do anyone know, why thunderbird add additional space to the lines started from space when quoting,
> which breaks indentation in quoted patches? How to fix it? I use plain text.
It's a known thunderbird display bug (or more specifically, something
that thunderbird does to avoid even worse whitespace corruption later in
its pipeline). It shows up in your reply window, but NOT in the
recipients' view. I've been dealing with bad formatting in thunderbird
for years now, and this latest bug of displaying too many spaces on
quoted reply lines that begin with whitespace (which in turn makes lines
starting with - or + appear indented wrong) is certainly less annoying
than their previous bug of converting all leading whitespace in the
quoted reply to a single space.
>
>> "\n"
>> " -h, --help display this help and exit\n"
>> @@ -97,6 +98,7 @@ static void usage(const char *name)
>> " -P, --partition=NUM only expose partition NUM\n"
>> "\n"
>> "General purpose options:\n"
>> +" -L, --list list NBD exports visible to client\n"
>
> hm. I think, user who things that qemu-nbd is only a server, can understand this as
> "dry run, don't actually start the server, but only list exports, which will be
> available to clients, keeping in mind all other specified options".
>
> so, may be, "list remote NBD server exports and options" or something like this?
There's line lengths to worry about, but yes, I agree that adding
something to make it a bit more obvious that this option is special is
worth the word-smithing attempts.
>> + printf("exports available: %d\n", rc);
>> + for (i = 0; i < rc; i++) {
>> + printf(" export: '%s'\n", list[i].name);
>> + if (list[i].description && *list[i].description) {
>> + printf(" description: %s\n", list[i].description);
>> + }
>> + if (list[i].flags & NBD_FLAG_HAS_FLAGS) {
>> + printf(" size: %" PRIu64 "\n", list[i].size);
>
> size is available only if NBD_FLAG_HAS_FLAGS? at least, accordingly to code,
> in case 0: size is unrelated to flags.
case 0: code is for oldstyle servers. There are fewer and fewer of those
even worth worrying about (I used nbdkit -o to test that part of my
code, and we recently ripped oldstyle out of the qemu server); the ones
that remain happen to set NBD_FLAG_HAS_FLAGS even for oldstyle. I'm not
too worried if our --list output fails to list size for an oldstyle
server that did not set NBD_FLAG_HAS_FLAGS.
For all other servers, NBD_FLAG_HAS_FLAGS is a good witness of whether
NBD_OPT_INFO was recognized, and there, size is indeed not present
unless NBD_OPT_INFO worked.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 13/14] qemu-nbd: Add --list option
2018-12-07 15:36 ` Eric Blake
@ 2018-12-07 16:49 ` Vladimir Sementsov-Ogievskiy
2018-12-07 16:49 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-07 16:49 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: jsnow, nsoffer, rjones, qemu-block
07.12.2018 18:36, Eric Blake wrote:
> On 12/7/18 6:48 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 01.12.2018 1:03, Eric Blake wrote:
>>> We want to be able to detect whether a given qemu NBD server is
>>> exposing the right export(s) and dirty bitmaps, at least for
>>> regression testing. We could use 'nbd-client -l' from the upstream
>>> NBD project to list exports, but it's annoying to rely on
>>> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
>>> know about all of the qemu NBD extensions. Thus, we plan on adding
>>
>> ha, in this patch, not plan but add:)
>
> Yep, copy-and-paste of a common prefix across multiple patches. I'll probably let the text diverge in v2 to be a bit more accurate per-patch, at the loss of the nice copy-and-paste.
>
>>> I debated about adding this functionality to something akin to
>>> 'qemu-img info' - but that tool does not readily lend itself
>>> to connecting to an arbitrary NBD server without also tying to
>>> a specific export (I may, however, still add ImageInfoSpecificNBD
>>> for reporting the bitmaps available when connecting to a single
>>> export). And, while it may feel a bit odd that normally
>>> qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not
>>> really making the qemu-nbd binary that much larger, because
>>> 'qemu-nbd -c' has to operate as both server and client
>>> simultaneously across two threads when feeding the kernel module
>>> for /dev/nbdN access.
>>>
>>> Sample output:
>>> $ qemu-nbd -L
>>> exports available: 1
>>> export: ''
>>> size: 65536
>>> flags: 0x4ed ( flush fua trim zeroes df cache )
>>> min block: 512
>>> opt block: 4096
>>> max block: 33554432
>>> available meta contexts: 1
>>> base:allocation
>>
>> don't you plan to bind this all to QAPI and expose in json?
>
> No. As explained above, QAPI is very much centered on per-BDS actions, while this action is a per-server action (where many servers have only one export, but it is also possible to have a server with 0 exports or a plurality of exports). I _can't_ fit this into query-block's ImageInfo details about a block device, because we don't have a way of stating 'connect to this arbitrary server, create an unspecified number of block devices, tell me about each of them, and then throw them all away because we only wanted the info about what the server made available'. The same is true for gluster or other remote access block devices - the QMP commands pre-suppose you already know WHICH specific resource you are accessing, rather than providing a way for you to query the remote server about ALL resources available but without actually selecting those resources.
>
> I'm open to ideas about a new QMP command to do such a query, but who would be the client? A management app that wants to hotplug a new NBD device to a running guest can run 'qemu-nbd --list' just as easily as they could run a new QMP command to learn what the server is offering. Without a strong reason of a client that would need this in QMP, I don't see the point in adding it to the qemu binary.
I didn't mean QMP. For example, QAPI struct ImageCheck is used only in qemu-img, to format output.
Anyway, creating a struct in QAPI for something we want to export is a good thing, I think.
Also, if, as you said, some management app wants to query this information, again strictly
defined data + json output should be a good option. And, if there would be such users, we'll
need to track compatibility of exported structure between qemu versions and this is easier
with QAPI defined structure.
And then, defined structure may be then (at least partly) shared with ImageInfoSpecificNBD.
And if we will need at some point a qmp command like query-nbd-server, to get same information through
current qmp-connection, not running additional nbd-client, it would be a simple thing to do.
>
>
>>> +++ b/qemu-nbd.c
>>> @@ -76,6 +76,7 @@ static void usage(const char *name)
>>> {
>>> (printf) (
>>> "Usage: %s [OPTIONS] FILE\n"
>>> +" or: %s -L [OPTIONS]\n"
>>> "QEMU Disk Network Block Device Server\n"
>>
>> Do anyone know, why thunderbird add additional space to the lines started from space when quoting,
>> which breaks indentation in quoted patches? How to fix it? I use plain text.
>
> It's a known thunderbird display bug (or more specifically, something that thunderbird does to avoid even worse whitespace corruption later in its pipeline). It shows up in your reply window, but NOT in the recipients' view. I've been dealing with bad formatting in thunderbird for years now, and this latest bug of displaying too many spaces on quoted reply lines that begin with whitespace (which in turn makes lines starting with - or + appear indented wrong) is certainly less annoying than their previous bug of converting all leading whitespace in the quoted reply to a single space.
>
>>
>>> "\n"
>>> " -h, --help display this help and exit\n"
>>> @@ -97,6 +98,7 @@ static void usage(const char *name)
>>> " -P, --partition=NUM only expose partition NUM\n"
>>> "\n"
>>> "General purpose options:\n"
>>> +" -L, --list list NBD exports visible to client\n"
>>
>> hm. I think, user who things that qemu-nbd is only a server, can understand this as
>> "dry run, don't actually start the server, but only list exports, which will be
>> available to clients, keeping in mind all other specified options".
>>
>> so, may be, "list remote NBD server exports and options" or something like this?
>
> There's line lengths to worry about, but yes, I agree that adding something to make it a bit more obvious that this option is special is worth the word-smithing attempts.
>
>>> + printf("exports available: %d\n", rc);
>>> + for (i = 0; i < rc; i++) {
>>> + printf(" export: '%s'\n", list[i].name);
>>> + if (list[i].description && *list[i].description) {
>>> + printf(" description: %s\n", list[i].description);
>>> + }
>>> + if (list[i].flags & NBD_FLAG_HAS_FLAGS) {
>>> + printf(" size: %" PRIu64 "\n", list[i].size);
>>
>> size is available only if NBD_FLAG_HAS_FLAGS? at least, accordingly to code,
>> in case 0: size is unrelated to flags.
>
> case 0: code is for oldstyle servers. There are fewer and fewer of those even worth worrying about (I used nbdkit -o to test that part of my code, and we recently ripped oldstyle out of the qemu server); the ones that remain happen to set NBD_FLAG_HAS_FLAGS even for oldstyle. I'm not too worried if our --list output fails to list size for an oldstyle server that did not set NBD_FLAG_HAS_FLAGS.
>
> For all other servers, NBD_FLAG_HAS_FLAGS is a good witness of whether NBD_OPT_INFO was recognized, and there, size is indeed not present unless NBD_OPT_INFO worked.
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 13/14] qemu-nbd: Add --list option
2018-12-07 15:36 ` Eric Blake
2018-12-07 16:49 ` Vladimir Sementsov-Ogievskiy
@ 2018-12-07 16:49 ` Vladimir Sementsov-Ogievskiy
2018-12-07 16:59 ` Eric Blake
1 sibling, 1 reply; 65+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-12-07 16:49 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: jsnow, nsoffer, rjones, qemu-block
07.12.2018 18:36, Eric Blake wrote:
> On 12/7/18 6:48 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 01.12.2018 1:03, Eric Blake wrote:
>>> We want to be able to detect whether a given qemu NBD server is
>>> exposing the right export(s) and dirty bitmaps, at least for
>>> regression testing. We could use 'nbd-client -l' from the upstream
>>> NBD project to list exports, but it's annoying to rely on
>>> out-of-tree binaries; furthermore, nbd-client doesn't necessarily
>>> know about all of the qemu NBD extensions. Thus, we plan on adding
>>
>> ha, in this patch, not plan but add:)
>
> Yep, copy-and-paste of a common prefix across multiple patches. I'll probably let the text diverge in v2 to be a bit more accurate per-patch, at the loss of the nice copy-and-paste.
>
>>> I debated about adding this functionality to something akin to
>>> 'qemu-img info' - but that tool does not readily lend itself
>>> to connecting to an arbitrary NBD server without also tying to
>>> a specific export (I may, however, still add ImageInfoSpecificNBD
>>> for reporting the bitmaps available when connecting to a single
>>> export). And, while it may feel a bit odd that normally
>>> qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not
>>> really making the qemu-nbd binary that much larger, because
>>> 'qemu-nbd -c' has to operate as both server and client
>>> simultaneously across two threads when feeding the kernel module
>>> for /dev/nbdN access.
>>>
>>> Sample output:
>>> $ qemu-nbd -L
>>> exports available: 1
>>> export: ''
>>> size: 65536
>>> flags: 0x4ed ( flush fua trim zeroes df cache )
>>> min block: 512
>>> opt block: 4096
>>> max block: 33554432
>>> available meta contexts: 1
>>> base:allocation
>>
>> don't you plan to bind this all to QAPI and expose in json?
>
> No. As explained above, QAPI is very much centered on per-BDS actions, while this action is a per-server action (where many servers have only one export, but it is also possible to have a server with 0 exports or a plurality of exports). I _can't_ fit this into query-block's ImageInfo details about a block device, because we don't have a way of stating 'connect to this arbitrary server, create an unspecified number of block devices, tell me about each of them, and then throw them all away because we only wanted the info about what the server made available'. The same is true for gluster or other remote access block devices - the QMP commands pre-suppose you already know WHICH specific resource you are accessing, rather than providing a way for you to query the remote server about ALL resources available but without actually selecting those resources.
>
> I'm open to ideas about a new QMP command to do such a query, but who would be the client? A management app that wants to hotplug a new NBD device to a running guest can run 'qemu-nbd --list' just as easily as they could run a new QMP command to learn what the server is offering. Without a strong reason of a client that would need this in QMP, I don't see the point in adding it to the qemu binary.
I didn't mean QMP. For example, QAPI struct ImageCheck is used only in qemu-img, to format output.
Anyway, creating a struct in QAPI for something we want to export is a good thing, I think.
Also, if, as you said, some management app wants to query this information, again strictly
defined data + json output should be a good option. And, if there would be such users, we'll
need to track compatibility of exported structure between qemu versions and this is easier
with QAPI defined structure.
And then, defined structure may be then (at least partly) shared with ImageInfoSpecificNBD.
And if we will need at some point a qmp command like query-nbd-server, to get same information through
current qmp-connection, not running additional nbd-client, it would be a simple thing to do.
>
>
>>> +++ b/qemu-nbd.c
>>> @@ -76,6 +76,7 @@ static void usage(const char *name)
>>> {
>>> (printf) (
>>> "Usage: %s [OPTIONS] FILE\n"
>>> +" or: %s -L [OPTIONS]\n"
>>> "QEMU Disk Network Block Device Server\n"
>>
>> Do anyone know, why thunderbird add additional space to the lines started from space when quoting,
>> which breaks indentation in quoted patches? How to fix it? I use plain text.
>
> It's a known thunderbird display bug (or more specifically, something that thunderbird does to avoid even worse whitespace corruption later in its pipeline). It shows up in your reply window, but NOT in the recipients' view. I've been dealing with bad formatting in thunderbird for years now, and this latest bug of displaying too many spaces on quoted reply lines that begin with whitespace (which in turn makes lines starting with - or + appear indented wrong) is certainly less annoying than their previous bug of converting all leading whitespace in the quoted reply to a single space.
>
>>
>>> "\n"
>>> " -h, --help display this help and exit\n"
>>> @@ -97,6 +98,7 @@ static void usage(const char *name)
>>> " -P, --partition=NUM only expose partition NUM\n"
>>> "\n"
>>> "General purpose options:\n"
>>> +" -L, --list list NBD exports visible to client\n"
>>
>> hm. I think, user who things that qemu-nbd is only a server, can understand this as
>> "dry run, don't actually start the server, but only list exports, which will be
>> available to clients, keeping in mind all other specified options".
>>
>> so, may be, "list remote NBD server exports and options" or something like this?
>
> There's line lengths to worry about, but yes, I agree that adding something to make it a bit more obvious that this option is special is worth the word-smithing attempts.
>
>>> + printf("exports available: %d\n", rc);
>>> + for (i = 0; i < rc; i++) {
>>> + printf(" export: '%s'\n", list[i].name);
>>> + if (list[i].description && *list[i].description) {
>>> + printf(" description: %s\n", list[i].description);
>>> + }
>>> + if (list[i].flags & NBD_FLAG_HAS_FLAGS) {
>>> + printf(" size: %" PRIu64 "\n", list[i].size);
>>
>> size is available only if NBD_FLAG_HAS_FLAGS? at least, accordingly to code,
>> in case 0: size is unrelated to flags.
>
> case 0: code is for oldstyle servers. There are fewer and fewer of those even worth worrying about (I used nbdkit -o to test that part of my code, and we recently ripped oldstyle out of the qemu server); the ones that remain happen to set NBD_FLAG_HAS_FLAGS even for oldstyle. I'm not too worried if our --list output fails to list size for an oldstyle server that did not set NBD_FLAG_HAS_FLAGS.
>
> For all other servers, NBD_FLAG_HAS_FLAGS is a good witness of whether NBD_OPT_INFO was recognized, and there, size is indeed not present unless NBD_OPT_INFO worked.
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 13/14] qemu-nbd: Add --list option
2018-12-07 16:49 ` Vladimir Sementsov-Ogievskiy
@ 2018-12-07 16:59 ` Eric Blake
0 siblings, 0 replies; 65+ messages in thread
From: Eric Blake @ 2018-12-07 16:59 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: jsnow, nsoffer, rjones, qemu-block
On 12/7/18 10:49 AM, Vladimir Sementsov-Ogievskiy wrote:
>>>> $ qemu-nbd -L
>>>> exports available: 1
>>>> export: ''
>>>> size: 65536
>>>> flags: 0x4ed ( flush fua trim zeroes df cache )
>>>> min block: 512
>>>> opt block: 4096
>>>> max block: 33554432
>>>> available meta contexts: 1
>>>> base:allocation
>>>
>>> don't you plan to bind this all to QAPI and expose in json?
>>
>> No. As explained above, QAPI is very much centered on per-BDS actions, while this action is a per-server action (where many servers have only one export, but it is also possible to have a server with 0 exports or a plurality of exports). I _can't_ fit this into query-block's ImageInfo details about a block device, because we don't have a way of stating 'connect to this arbitrary server, create an unspecified number of block devices, tell me about each of them, and then throw them all away because we only wanted the info about what the server made available'. The same is true for gluster or other remote access block devices - the QMP commands pre-suppose you already know WHICH specific resource you are accessing, rather than providing a way for you to query the remote server about ALL resources available but without actually selecting those resources.
>>
>> I'm open to ideas about a new QMP command to do such a query, but who would be the client? A management app that wants to hotplug a new NBD device to a running guest can run 'qemu-nbd --list' just as easily as they could run a new QMP command to learn what the server is offering. Without a strong reason of a client that would need this in QMP, I don't see the point in adding it to the qemu binary.
>
> I didn't mean QMP. For example, QAPI struct ImageCheck is used only in qemu-img, to format output.
> Anyway, creating a struct in QAPI for something we want to export is a good thing, I think.
Oh, I see where you're going with this. Just as 'qemu-img info' has
routines to pretty-print a QAPI structure (and thus adding ImageInfo to
the .json files automatically gets output in HMP without any additional
work), you're suggesting that NBDExportInfo be converted into a QAPI
struct, even if it won't be tied to QMP, in order to make the output
more programmatic instead of manual effort. I'll have to play with
that, although it might be a separate series on top of this.
> Also, if, as you said, some management app wants to query this information, again strictly
> defined data + json output should be a good option. And, if there would be such users, we'll
> need to track compatibility of exported structure between qemu versions and this is easier
> with QAPI defined structure.
>
> And then, defined structure may be then (at least partly) shared with ImageInfoSpecificNBD.
>
> And if we will need at some point a qmp command like query-nbd-server, to get same information through
> current qmp-connection, not running additional nbd-client, it would be a simple thing to do.
If we ever need future extensions, we'll want to have QAPI structs in
place. But whether we implement the QAPI structs now, or at the time of
the future extension, is an engineering tradeoff (how much technical
debt are we incurring by not doing it now; and how likely are we to ever
want the future extension).
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 02/14] nbd/client: More consistent error messages
2018-12-05 15:03 ` Vladimir Sementsov-Ogievskiy
@ 2018-12-10 22:03 ` Eric Blake
0 siblings, 0 replies; 65+ messages in thread
From: Eric Blake @ 2018-12-10 22:03 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: jsnow, nsoffer, rjones, qemu-block
On 12/5/18 9:03 AM, Vladimir Sementsov-Ogievskiy wrote:
> 01.12.2018 1:03, Eric Blake wrote:
>> Consolidate on using decimal (not hex) and on outputting the
>> option reply name (not just value) when the client reports
>> protocol discrepancies from the server. While it won't affect
>> normal operation, it makes debugging additions easier.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>> + error_setg(errp, "Unexpected option type %u (%s) expected %u (%s)",
>> + reply->option, nbd_opt_lookup(reply->option),
>> + opt, nbd_opt_lookup(opt));
>> @@ -378,9 +380,9 @@ static int nbd_opt_go(QIOChannel *ioc, const char *wantname,
>> return 1;
>> }
>> if (reply.type != NBD_REP_INFO) {
>> - error_setg(errp, "unexpected reply type %" PRIu32
>> - " (%s), expected %u",
>> - reply.type, nbd_rep_lookup(reply.type), NBD_REP_INFO);
>> + error_setg(errp, "unexpected reply type %u (%s), expected %u (%s)",
>
> hmm, we are definitely inconsistent about having comma before "expected" word...
>
> anyway,
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
That's minor enough; I'll add commas to all instances, but keep your R-b.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [PATCH 04/14] qemu-nbd: Simplify --partition handling
2018-11-30 22:03 ` [Qemu-devel] [PATCH 04/14] qemu-nbd: Simplify --partition handling Eric Blake
2018-11-30 22:26 ` Richard W.M. Jones
2018-12-05 15:40 ` Vladimir Sementsov-Ogievskiy
@ 2018-12-10 22:28 ` Eric Blake
2 siblings, 0 replies; 65+ messages in thread
From: Eric Blake @ 2018-12-10 22:28 UTC (permalink / raw)
To: qemu-devel; +Cc: nsoffer, vsementsov, jsnow, rjones, qemu-block
On 11/30/18 4:03 PM, Eric Blake wrote:
> Our open-coding of strtol handling forgot to handle overflow
> conditions. What's more, since we insiste on a user-supplied
> partition to be non-zero, we can use 0 rather than -1 for our
> initial value to distinguish when a partition is not being
> served, for slightly more optimal code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> qemu-nbd.c | 14 +++++---------
> 1 file changed, 5 insertions(+), 9 deletions(-)
>
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index 55e29bd9a7e..866e64779f1 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -546,7 +546,7 @@ int main(int argc, char **argv)
> int opt_ind = 0;
> char *end;
> int flags = BDRV_O_RDWR;
> - int partition = -1;
> + int partition = 0;
> int ret = 0;
> bool seen_cache = false;
> bool seen_discard = false;
> @@ -685,13 +685,9 @@ int main(int argc, char **argv)
> flags &= ~BDRV_O_RDWR;
> break;
> case 'P':
> - partition = strtol(optarg, &end, 0);
> - if (*end) {
> - error_report("Invalid partition `%s'", optarg);
> - exit(EXIT_FAILURE);
> - }
> - if (partition < 1 || partition > 8) {
> - error_report("Invalid partition %d", partition);
> + if (qemu_strtoi(optarg, NULL, 0, &partition) < 0 ||
Hmm - the fact that 'char *end' is not a dead variable means there are
more uses of strtoll() that need fixing. I'll get those in v2.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 65+ messages in thread
end of thread, other threads:[~2018-12-10 22:28 UTC | newest]
Thread overview: 65+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-11-30 22:03 [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Eric Blake
2018-11-30 22:03 ` [Qemu-devel] [PATCH 01/14] qemu-nbd: Use program name in error messages Eric Blake
2018-11-30 22:17 ` Richard W.M. Jones
2018-12-05 14:55 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 02/14] nbd/client: More consistent " Eric Blake
2018-11-30 22:20 ` Richard W.M. Jones
2018-12-05 15:03 ` Vladimir Sementsov-Ogievskiy
2018-12-10 22:03 ` Eric Blake
2018-11-30 22:03 ` [Qemu-devel] [PATCH 03/14] qemu-nbd: Fail earlier for -c/-d on non-linux Eric Blake
2018-11-30 22:23 ` Richard W.M. Jones
2018-12-05 15:20 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 04/14] qemu-nbd: Simplify --partition handling Eric Blake
2018-11-30 22:26 ` Richard W.M. Jones
2018-11-30 22:41 ` Eric Blake
2018-12-05 15:40 ` Vladimir Sementsov-Ogievskiy
2018-12-05 16:26 ` Eric Blake
2018-12-05 16:32 ` Eric Blake
2018-12-10 22:28 ` Eric Blake
2018-11-30 22:03 ` [Qemu-devel] [PATCH 05/14] nbd/client: Drop pointless buf variable Eric Blake
2018-11-30 22:30 ` Richard W.M. Jones
2018-11-30 22:54 ` Eric Blake
2018-12-05 15:59 ` Vladimir Sementsov-Ogievskiy
2018-12-05 16:29 ` Eric Blake
2018-12-05 16:38 ` Vladimir Sementsov-Ogievskiy
2018-12-05 16:49 ` Eric Blake
2018-11-30 22:03 ` [Qemu-devel] [PATCH 06/14] nbd/client: Move export name into NBDExportInfo Eric Blake
2018-11-30 22:34 ` Richard W.M. Jones
2018-12-05 17:26 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 07/14] nbd/client: Refactor nbd_negotiate_simple_meta_context() Eric Blake
2018-12-01 10:30 ` Richard W.M. Jones
2018-12-06 13:20 ` Vladimir Sementsov-Ogievskiy
2018-12-06 16:20 ` Eric Blake
2018-11-30 22:03 ` [Qemu-devel] [PATCH 08/14] nbd/client: Refactor nbd_receive_list() Eric Blake
2018-12-01 10:37 ` Richard W.M. Jones
2018-12-06 14:18 ` Vladimir Sementsov-Ogievskiy
2018-12-06 16:31 ` Eric Blake
2018-12-06 17:03 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 09/14] nbd/client: Refactor return of nbd_receive_negotiate() Eric Blake
2018-11-30 22:41 ` Richard W.M. Jones
2018-12-06 14:24 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 10/14] nbd/client: Split handshake into two functions Eric Blake
2018-12-01 10:41 ` Richard W.M. Jones
2018-12-06 15:16 ` Vladimir Sementsov-Ogievskiy
2018-12-06 17:06 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 11/14] nbd/client: Add nbd_receive_export_list() Eric Blake
2018-12-01 10:45 ` Richard W.M. Jones
2018-12-07 10:04 ` Vladimir Sementsov-Ogievskiy
2018-12-07 15:19 ` Eric Blake
2018-12-07 10:07 ` Vladimir Sementsov-Ogievskiy
2018-11-30 22:03 ` [Qemu-devel] [PATCH 12/14] nbd/client: Work around 3.0 bug for listing meta contexts Eric Blake
2018-12-07 11:21 ` Vladimir Sementsov-Ogievskiy
2018-12-07 15:21 ` Eric Blake
2018-11-30 22:03 ` [Qemu-devel] [PATCH 13/14] qemu-nbd: Add --list option Eric Blake
2018-12-01 10:58 ` Richard W.M. Jones
2018-12-07 12:48 ` Vladimir Sementsov-Ogievskiy
2018-12-07 15:36 ` Eric Blake
2018-12-07 16:49 ` Vladimir Sementsov-Ogievskiy
2018-12-07 16:49 ` Vladimir Sementsov-Ogievskiy
2018-12-07 16:59 ` Eric Blake
2018-11-30 22:03 ` [Qemu-devel] [PATCH 14/14] iotests: Enhance 223, 233 to cover 'qemu-nbd --list' Eric Blake
2018-12-01 11:04 ` Richard W.M. Jones
2018-12-07 13:08 ` Vladimir Sementsov-Ogievskiy
2018-12-01 7:42 ` [Qemu-devel] [PATCH for-4.0 00/14] nbd: add qemu-nbd --list Richard W.M. Jones
2018-12-01 13:57 ` Eric Blake
2018-12-01 15:00 ` Richard W.M. Jones
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.