[Qemu-devel] [PATCH v5 0/5] Some fixes related to scsi-generic

[Qemu-devel] [PATCH v5 0/5] Some fixes related to scsi-generic

[Dimitris Aragiorgis]

Hi all,

These five patches make slight changes to the way QEMU handles SCSI
generic devices to fix a number of small problems.

I am sending them against the master branch, since I don't know if they
can be considered bugfixes.

Thanks,
dimara

v5 (rebased to current master):
* First use stat() and then ioctl() in bdrv_is_sg() (Eric's comment)
* Make bdrv_is_sg() return a bool and not an int (Eric's comment)

v4 (rebased to current master):
* Avoid errno clobbering with DPRINTF + strerror() (Eric's comment)
* Use {} in #define macro even if it is not necessary (single commands)

v3 (rebased to current master):
* Avoid bit-rot in DPRINTF (adopt Eric's suggestion)
* Address Kevin's comments (DEBUG_FLOPPY, line > 80 chars, SG device)
* Mention Kevin's comment wrt disk flush in the corresponding commit

v2:
* remove duplicate check for sg inside iscsi_co_flush()
* remove DEBUG_BLOCK_PRINT in block/raw-posix.c
* use DPRINTF for debugging in block/raw-posix.c

PS: Paolo suggested to use a tracepoint inside hdev_is_sg() but I chose DPRINTF
instead. It would make sense to add a tracepoint for bdrv_is_sg() (just like
most bdrv_* commands) but this is too much for now since it just returns the
bs->sg flag (and is not an actual driver function). If you insist I'll change
it in v3.

Dimitris Aragiorgis (5):
  block: Use bdrv_is_sg() everywhere
  Fix migration in case of scsi-generic
  raw-posix: DPRINTF instead of DEBUG_BLOCK_PRINT
  raw-posix: Use DPRINTF for DEBUG_FLOPPY
  raw-posix: Introduce hdev_is_sg()

 block.c           |    6 ++--
 block/io.c        |    3 +-
 block/iscsi.c     |    4 ---
 block/raw-posix.c |   91 +++++++++++++++++++++++++++++++----------------------
 4 files changed, 58 insertions(+), 46 deletions(-)

-- 
1.7.10.4

[Qemu-devel] [PATCH v5 1/5] block: Use bdrv_is_sg() everywhere

[Dimitris Aragiorgis]

Instead of checking bs->sg use bdrv_is_sg() consistently throughout
the code.

Signed-off-by: Dimitris Aragiorgis <dimara@arrikto.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block.c           |    6 +++---
 block/iscsi.c     |    2 +-
 block/raw-posix.c |    4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/block.c b/block.c
index dd4f58d..f04b2a1 100644
--- a/block.c
+++ b/block.c
@@ -583,7 +583,7 @@ static int find_image_format(BlockDriverState *bs, const char *filename,
     int ret = 0;
 
     /* Return the raw BlockDriver * to scsi-generic devices or empty drives */
-    if (bs->sg || !bdrv_is_inserted(bs) || bdrv_getlength(bs) == 0) {
+    if (bdrv_is_sg(bs) || !bdrv_is_inserted(bs) || bdrv_getlength(bs) == 0) {
         *pdrv = &bdrv_raw;
         return ret;
     }
@@ -615,7 +615,7 @@ static int refresh_total_sectors(BlockDriverState *bs, int64_t hint)
     BlockDriver *drv = bs->drv;
 
     /* Do not attempt drv->bdrv_getlength() on scsi-generic devices */
-    if (bs->sg)
+    if (bdrv_is_sg(bs))
         return 0;
 
     /* query actual device if possible, otherwise just trust the hint */
@@ -946,7 +946,7 @@ static int bdrv_open_common(BlockDriverState *bs, BlockDriverState *file,
 
     assert(bdrv_opt_mem_align(bs) != 0);
     assert(bdrv_min_mem_align(bs) != 0);
-    assert((bs->request_alignment != 0) || bs->sg);
+    assert((bs->request_alignment != 0) || bdrv_is_sg(bs));
 
     qemu_opts_del(opts);
     return 0;
diff --git a/block/iscsi.c b/block/iscsi.c
index 14e97a6..000528c 100644
--- a/block/iscsi.c
+++ b/block/iscsi.c
@@ -627,7 +627,7 @@ static int coroutine_fn iscsi_co_flush(BlockDriverState *bs)
     IscsiLun *iscsilun = bs->opaque;
     struct IscsiTask iTask;
 
-    if (bs->sg) {
+    if (bdrv_is_sg(bs)) {
         return 0;
     }
 
diff --git a/block/raw-posix.c b/block/raw-posix.c
index 44ade8c..3ee3bd6 100644
--- a/block/raw-posix.c
+++ b/block/raw-posix.c
@@ -303,9 +303,9 @@ static void raw_probe_alignment(BlockDriverState *bs, int fd, Error **errp)
     char *buf;
     size_t max_align = MAX(MAX_BLOCKSIZE, getpagesize());
 
-    /* For /dev/sg devices the alignment is not really used.
+    /* For SCSI generic devices the alignment is not really used.
        With buffered I/O, we don't have any restrictions. */
-    if (bs->sg || !s->needs_alignment) {
+    if (bdrv_is_sg(bs) || !s->needs_alignment) {
         bs->request_alignment = 1;
         s->buf_align = 1;
         return;
-- 
1.7.10.4

[Qemu-devel] [PATCH v5 2/5] Fix migration in case of scsi-generic

[Dimitris Aragiorgis]

During migration, QEMU uses fsync()/fdatasync() on the open file
descriptor for read-write block devices to flush data just before
stopping the VM.

However, fsync() on a scsi-generic device returns -EINVAL which
causes the migration to fail. This patch skips flushing data in case
of an SG device, since submitting SCSI commands directly via an SG
character device (e.g. /dev/sg0) bypasses the page cache completely,
anyway.

Note that fsync() not only flushes the page cache but also the disk
cache. The scsi-generic device never sends flushes, and for
migration it assumes that the same SCSI device is used by the
destination host, so it does not issue any SCSI SYNCHRONIZE CACHE
(10) command.

Finally, remove the bdrv_is_sg() test from iscsi_co_flush() since
this is now redundant (we flush the underlying protocol at the end
of bdrv_co_flush() which, with this patch, we never reach).

Signed-off-by: Dimitris Aragiorgis <dimara@arrikto.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/io.c    |    3 ++-
 block/iscsi.c |    4 ----
 2 files changed, 2 insertions(+), 5 deletions(-)

diff --git a/block/io.c b/block/io.c
index bb4f787..f45a3ad 100644
--- a/block/io.c
+++ b/block/io.c
@@ -2255,7 +2255,8 @@ int coroutine_fn bdrv_co_flush(BlockDriverState *bs)
 {
     int ret;
 
-    if (!bs || !bdrv_is_inserted(bs) || bdrv_is_read_only(bs)) {
+    if (!bs || !bdrv_is_inserted(bs) || bdrv_is_read_only(bs) ||
+        bdrv_is_sg(bs)) {
         return 0;
     }
 
diff --git a/block/iscsi.c b/block/iscsi.c
index 000528c..62e422e 100644
--- a/block/iscsi.c
+++ b/block/iscsi.c
@@ -627,10 +627,6 @@ static int coroutine_fn iscsi_co_flush(BlockDriverState *bs)
     IscsiLun *iscsilun = bs->opaque;
     struct IscsiTask iTask;
 
-    if (bdrv_is_sg(bs)) {
-        return 0;
-    }
-
     if (!iscsilun->force_next_flush) {
         return 0;
     }
-- 
1.7.10.4

[Qemu-devel] [PATCH v5 3/5] raw-posix: DPRINTF instead of DEBUG_BLOCK_PRINT

[Dimitris Aragiorgis]

Building the QEMU tools fails if we #define DEBUG_BLOCK inside
block/raw-posix.c. Here instead of adding qemu-log.o in block-obj-y
so that DEBUG_BLOCK_PRINT can be used, we substitute the latter with
a simple DPRINTF() (that does not cause bit-rot).

Signed-off-by: Dimitris Aragiorgis <dimara@arrikto.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/raw-posix.c |   26 ++++++++++++++++++--------
 1 file changed, 18 insertions(+), 8 deletions(-)

diff --git a/block/raw-posix.c b/block/raw-posix.c
index 3ee3bd6..60b5ef7 100644
--- a/block/raw-posix.c
+++ b/block/raw-posix.c
@@ -97,12 +97,18 @@
 //#define DEBUG_FLOPPY
 
 //#define DEBUG_BLOCK
-#if defined(DEBUG_BLOCK)
-#define DEBUG_BLOCK_PRINT(formatCstr, ...) do { if (qemu_log_enabled()) \
-    { qemu_log(formatCstr, ## __VA_ARGS__); qemu_log_flush(); } } while (0)
+
+#ifdef DEBUG_BLOCK
+# define DEBUG_BLOCK_PRINT 1
 #else
-#define DEBUG_BLOCK_PRINT(formatCstr, ...)
+# define DEBUG_BLOCK_PRINT 0
 #endif
+#define DPRINTF(fmt, ...) \
+do { \
+    if (DEBUG_BLOCK_PRINT) { \
+        printf(fmt, ## __VA_ARGS__); \
+    } \
+} while (0)
 
 /* OS X does not have O_DSYNC */
 #ifndef O_DSYNC
@@ -1018,6 +1024,7 @@ static ssize_t handle_aiocb_rw(RawPosixAIOData *aiocb)
 static int xfs_write_zeroes(BDRVRawState *s, int64_t offset, uint64_t bytes)
 {
     struct xfs_flock64 fl;
+    int err;
 
     memset(&fl, 0, sizeof(fl));
     fl.l_whence = SEEK_SET;
@@ -1025,8 +1032,9 @@ static int xfs_write_zeroes(BDRVRawState *s, int64_t offset, uint64_t bytes)
     fl.l_len = bytes;
 
     if (xfsctl(NULL, s->fd, XFS_IOC_ZERO_RANGE, &fl) < 0) {
-        DEBUG_BLOCK_PRINT("cannot write zero range (%s)\n", strerror(errno));
-        return -errno;
+        err = errno;
+        DPRINTF("cannot write zero range (%s)\n", strerror(errno));
+        return -err;
     }
 
     return 0;
@@ -1035,6 +1043,7 @@ static int xfs_write_zeroes(BDRVRawState *s, int64_t offset, uint64_t bytes)
 static int xfs_discard(BDRVRawState *s, int64_t offset, uint64_t bytes)
 {
     struct xfs_flock64 fl;
+    int err;
 
     memset(&fl, 0, sizeof(fl));
     fl.l_whence = SEEK_SET;
@@ -1042,8 +1051,9 @@ static int xfs_discard(BDRVRawState *s, int64_t offset, uint64_t bytes)
     fl.l_len = bytes;
 
     if (xfsctl(NULL, s->fd, XFS_IOC_UNRESVSP64, &fl) < 0) {
-        DEBUG_BLOCK_PRINT("cannot punch hole (%s)\n", strerror(errno));
-        return -errno;
+        err = errno;
+        DPRINTF("cannot punch hole (%s)\n", strerror(errno));
+        return -err;
     }
 
     return 0;
-- 
1.7.10.4

[Qemu-devel] [PATCH v5 4/5] raw-posix: Use DPRINTF for DEBUG_FLOPPY

[Dimitris Aragiorgis]

Get rid of several #ifdef DEBUG_FLOPPY and substitute them with
DPRINTF.

Signed-off-by: Dimitris Aragiorgis <dimara@arrikto.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/raw-posix.c |   22 +++++-----------------
 1 file changed, 5 insertions(+), 17 deletions(-)

diff --git a/block/raw-posix.c b/block/raw-posix.c
index 60b5ef7..69ae251 100644
--- a/block/raw-posix.c
+++ b/block/raw-posix.c
@@ -94,8 +94,6 @@
 #include <xfs/xfs.h>
 #endif
 
-//#define DEBUG_FLOPPY
-
 //#define DEBUG_BLOCK
 
 #ifdef DEBUG_BLOCK
@@ -2170,16 +2168,12 @@ static int fd_open(BlockDriverState *bs)
         (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - s->fd_open_time) >= FD_OPEN_TIMEOUT) {
         qemu_close(s->fd);
         s->fd = -1;
-#ifdef DEBUG_FLOPPY
-        printf("Floppy closed\n");
-#endif
+        DPRINTF("Floppy closed\n");
     }
     if (s->fd < 0) {
         if (s->fd_got_error &&
             (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - s->fd_error_time) < FD_OPEN_TIMEOUT) {
-#ifdef DEBUG_FLOPPY
-            printf("No floppy (open delayed)\n");
-#endif
+            DPRINTF("No floppy (open delayed)\n");
             return -EIO;
         }
         s->fd = qemu_open(bs->filename, s->open_flags & ~O_NONBLOCK);
@@ -2188,14 +2182,10 @@ static int fd_open(BlockDriverState *bs)
             s->fd_got_error = 1;
             if (last_media_present)
                 s->fd_media_changed = 1;
-#ifdef DEBUG_FLOPPY
-            printf("No floppy\n");
-#endif
+            DPRINTF("No floppy\n");
             return -EIO;
         }
-#ifdef DEBUG_FLOPPY
-        printf("Floppy opened\n");
-#endif
+        DPRINTF("Floppy opened\n");
     }
     if (!last_media_present)
         s->fd_media_changed = 1;
@@ -2463,9 +2453,7 @@ static int floppy_media_changed(BlockDriverState *bs)
     fd_open(bs);
     ret = s->fd_media_changed;
     s->fd_media_changed = 0;
-#ifdef DEBUG_FLOPPY
-    printf("Floppy changed=%d\n", ret);
-#endif
+    DPRINTF("Floppy changed=%d\n", ret);
     return ret;
 }
 
-- 
1.7.10.4

[Qemu-devel] [PATCH v5 5/5] raw-posix: Introduce hdev_is_sg()

[Dimitris Aragiorgis]

Until now, an SG device was identified only by checking if its path
started with "/dev/sg". Then, hdev_open() would set the bs->sg flag
accordingly. The patch relies on the actual properties of the device
instead of the specified file path.

To this end, test for an SG device (e.g. /dev/sg0) by ensuring that
all of the following holds:

 - The specified file name corresponds to a character device
 - The device supports the SG_GET_VERSION_NUM ioctl
 - The device supports the SG_GET_SCSI_ID ioctl

Signed-off-by: Dimitris Aragiorgis <dimara@arrikto.com>
---
 block/raw-posix.c |   39 ++++++++++++++++++++++++++++-----------
 1 file changed, 28 insertions(+), 11 deletions(-)

diff --git a/block/raw-posix.c b/block/raw-posix.c
index 69ae251..942cda3 100644
--- a/block/raw-posix.c
+++ b/block/raw-posix.c
@@ -57,6 +57,7 @@
 #include <linux/fd.h>
 #include <linux/fs.h>
 #include <linux/hdreg.h>
+#include <scsi/sg.h>
 #ifdef __s390__
 #include <asm/dasd.h>
 #endif
@@ -2084,15 +2085,38 @@ static void hdev_parse_filename(const char *filename, QDict *options,
     qdict_put_obj(options, "filename", QOBJECT(qstring_from_str(filename)));
 }
 
+static bool hdev_is_sg(BlockDriverState *bs)
+{
+
+#if defined(__linux__)
+
+    struct stat st;
+    struct sg_scsi_id scsiid;
+    int sg_version;
+
+    if (stat(bs->filename, &st) >= 0 && S_ISCHR(st.st_mode) &&
+        !bdrv_ioctl(bs, SG_GET_VERSION_NUM, &sg_version) &&
+        !bdrv_ioctl(bs, SG_GET_SCSI_ID, &scsiid)) {
+        DPRINTF("SG device found: type=%d, version=%d\n",
+            scsiid.scsi_type, sg_version);
+        return true;
+    }
+
+#endif
+
+    return false;
+}
+
 static int hdev_open(BlockDriverState *bs, QDict *options, int flags,
                      Error **errp)
 {
     BDRVRawState *s = bs->opaque;
     Error *local_err = NULL;
     int ret;
-    const char *filename = qdict_get_str(options, "filename");
 
 #if defined(__APPLE__) && defined(__MACH__)
+    const char *filename = qdict_get_str(options, "filename");
+
     if (strstart(filename, "/dev/cdrom", NULL)) {
         kern_return_t kernResult;
         io_iterator_t mediaIterator;
@@ -2121,16 +2145,6 @@ static int hdev_open(BlockDriverState *bs, QDict *options, int flags,
 #endif
 
     s->type = FTYPE_FILE;
-#if defined(__linux__)
-    {
-        char resolved_path[ MAXPATHLEN ], *temp;
-
-        temp = realpath(filename, resolved_path);
-        if (temp && strstart(temp, "/dev/sg", NULL)) {
-            bs->sg = 1;
-        }
-    }
-#endif
 
     ret = raw_open_common(bs, options, flags, 0, &local_err);
     if (ret < 0) {
@@ -2140,6 +2154,9 @@ static int hdev_open(BlockDriverState *bs, QDict *options, int flags,
         return ret;
     }
 
+    /* Since this does ioctl the device must be already opened */
+    bs->sg = hdev_is_sg(bs);
+
     if (flags & BDRV_O_RDWR) {
         ret = check_hdev_writable(s);
         if (ret < 0) {
-- 
1.7.10.4

Re: [Qemu-devel] [PATCH v5 5/5] raw-posix: Introduce hdev_is_sg()

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 911 bytes --]

On Tue, Jun 23, 2015 at 01:45:00PM +0300, Dimitris Aragiorgis wrote:
> +static bool hdev_is_sg(BlockDriverState *bs)
> +{
> +
> +#if defined(__linux__)
> +
> +    struct stat st;
> +    struct sg_scsi_id scsiid;
> +    int sg_version;
> +
> +    if (stat(bs->filename, &st) >= 0 && S_ISCHR(st.st_mode) &&
> +        !bdrv_ioctl(bs, SG_GET_VERSION_NUM, &sg_version) &&
> +        !bdrv_ioctl(bs, SG_GET_SCSI_ID, &scsiid)) {
> +        DPRINTF("SG device found: type=%d, version=%d\n",
> +            scsiid.scsi_type, sg_version);
> +        return true;
> +    }

If you respin this series, please use fstat() instead of stat() since we
already have the file descriptor open.  That ensures the stat is really
for the same file as the one we already have open (it avoids the race
condition).

I don't see a practical danger in using stat() for the time being:

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

[-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --]

Re: [Qemu-devel] [PATCH v5 0/5] Some fixes related to scsi-generic

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 2052 bytes --]

On Tue, Jun 23, 2015 at 01:44:55PM +0300, Dimitris Aragiorgis wrote:
> Hi all,
> 
> These five patches make slight changes to the way QEMU handles SCSI
> generic devices to fix a number of small problems.
> 
> I am sending them against the master branch, since I don't know if they
> can be considered bugfixes.
> 
> Thanks,
> dimara
> 
> v5 (rebased to current master):
> * First use stat() and then ioctl() in bdrv_is_sg() (Eric's comment)
> * Make bdrv_is_sg() return a bool and not an int (Eric's comment)
> 
> v4 (rebased to current master):
> * Avoid errno clobbering with DPRINTF + strerror() (Eric's comment)
> * Use {} in #define macro even if it is not necessary (single commands)
> 
> v3 (rebased to current master):
> * Avoid bit-rot in DPRINTF (adopt Eric's suggestion)
> * Address Kevin's comments (DEBUG_FLOPPY, line > 80 chars, SG device)
> * Mention Kevin's comment wrt disk flush in the corresponding commit
> 
> v2:
> * remove duplicate check for sg inside iscsi_co_flush()
> * remove DEBUG_BLOCK_PRINT in block/raw-posix.c
> * use DPRINTF for debugging in block/raw-posix.c
> 
> PS: Paolo suggested to use a tracepoint inside hdev_is_sg() but I chose DPRINTF
> instead. It would make sense to add a tracepoint for bdrv_is_sg() (just like
> most bdrv_* commands) but this is too much for now since it just returns the
> bs->sg flag (and is not an actual driver function). If you insist I'll change
> it in v3.
> 
> Dimitris Aragiorgis (5):
>   block: Use bdrv_is_sg() everywhere
>   Fix migration in case of scsi-generic
>   raw-posix: DPRINTF instead of DEBUG_BLOCK_PRINT
>   raw-posix: Use DPRINTF for DEBUG_FLOPPY
>   raw-posix: Introduce hdev_is_sg()
> 
>  block.c           |    6 ++--
>  block/io.c        |    3 +-
>  block/iscsi.c     |    4 ---
>  block/raw-posix.c |   91 +++++++++++++++++++++++++++++++----------------------
>  4 files changed, 58 insertions(+), 46 deletions(-)

Thanks, applied to my block tree:
https://github.com/stefanha/qemu/commits/block

Stefan

[-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --]