[PULL 0/2] virtiofs queue

[PULL 0/2] virtiofs queue

[Dr. David Alan Gilbert (git)]

From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

The following changes since commit 57b6f58c1d0df757c9311496c32d502925056894:

  Merge remote-tracking branch 'remotes/hreitz/tags/pull-block-2021-09-15' into staging (2021-09-15 18:55:59 +0100)

are available in the Git repository at:

  https://gitlab.com/dagrh/qemu.git tags/pull-virtiofs-20210916

for you to fetch changes up to 046d91c83caac29e2ba26c63fd7d685a57463f6d:

  virtiofsd: Reverse req_list before processing it (2021-09-16 14:50:48 +0100)

----------------------------------------------------------------
virtiofsd pull 2021-08-16

Two minor fixes; one for performance, the other seccomp
on s390x.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

----------------------------------------------------------------
Sergio Lopez (1):
      virtiofsd: Reverse req_list before processing it

Thomas Huth (1):
      tools/virtiofsd: Add fstatfs64 syscall to the seccomp allowlist

 tools/virtiofsd/fuse_virtio.c         | 1 +
 tools/virtiofsd/passthrough_seccomp.c | 1 +
 2 files changed, 2 insertions(+)

[PULL 1/2] tools/virtiofsd: Add fstatfs64 syscall to the seccomp allowlist

[Dr. David Alan Gilbert (git)]

From: Thomas Huth <thuth@redhat.com>

The virtiofsd currently crashes on s390x when doing something like
this in the guest:

 mkdir -p /mnt/myfs
 mount -t virtiofs myfs /mnt/myfs
 touch /mnt/myfs/foo.txt
 stat -f /mnt/myfs/foo.txt

The problem is that the fstatfs64 syscall is called in this case
from the virtiofsd. We have to put it on the seccomp allowlist to
avoid that the daemon gets killed in this case.

Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2001728
Suggested-by: Vivek Goyal <vgoyal@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210914123214.181885-1-thuth@redhat.com>
Reviewed-by: Vivek Goyal <vgoyal@redhat.com>
Reviewed-by: Sergio Lopez <slp@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 tools/virtiofsd/passthrough_seccomp.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/virtiofsd/passthrough_seccomp.c b/tools/virtiofsd/passthrough_seccomp.c
index f49ed94b5e..a3ce9f898d 100644
--- a/tools/virtiofsd/passthrough_seccomp.c
+++ b/tools/virtiofsd/passthrough_seccomp.c
@@ -51,6 +51,7 @@ static const int syscall_allowlist[] = {
     SCMP_SYS(fsetxattr),
     SCMP_SYS(fstat),
     SCMP_SYS(fstatfs),
+    SCMP_SYS(fstatfs64),
     SCMP_SYS(fsync),
     SCMP_SYS(ftruncate),
     SCMP_SYS(futex),
-- 
2.31.1

[PULL 2/2] virtiofsd: Reverse req_list before processing it

[Dr. David Alan Gilbert (git)]

From: Sergio Lopez <slp@redhat.com>

With the thread pool disabled, we add the requests in the queue to a
GList, processing by iterating over there afterwards.

For adding them, we're using "g_list_prepend()", which is more
efficient but causes the requests to be processed in reverse order,
breaking the read-ahead and request-merging optimizations in the host
for sequential operations.

According to the documentation, if you need to process the request
in-order, using "g_list_prepend()" and then reversing the list with
"g_list_reverse()" is more efficient than using "g_list_append()", so
let's do it that way.

Testing on a spinning disk (to boost the increase of read-ahead and
request-merging) shows a 4x improvement on sequential write fio test:

Test:
fio --directory=/mnt/virtio-fs --filename=fio-file1 --runtime=20
--iodepth=16 --size=4G --direct=1 --blocksize=4K --ioengine libaio
--rw write --name seqwrite-libaio

Without "g_list_reverse()":
...
Jobs: 1 (f=1): [W(1)][100.0%][w=22.4MiB/s][w=5735 IOPS][eta 00m:00s]
seqwrite-libaio: (groupid=0, jobs=1): err= 0: pid=710: Tue Aug 24 12:58:16 2021
  write: IOPS=5709, BW=22.3MiB/s (23.4MB/s)(446MiB/20002msec); 0 zone resets
...

With "g_list_reverse()":
...
Jobs: 1 (f=1): [W(1)][100.0%][w=84.0MiB/s][w=21.5k IOPS][eta 00m:00s]
seqwrite-libaio: (groupid=0, jobs=1): err= 0: pid=716: Tue Aug 24 13:00:15 2021
  write: IOPS=21.3k, BW=83.1MiB/s (87.2MB/s)(1663MiB/20001msec); 0 zone resets
...

Signed-off-by: Sergio Lopez <slp@redhat.com>
Message-Id: <20210824131158.39970-1-slp@redhat.com>
Reviewed-by: Vivek Goyal <vgoyal@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 tools/virtiofsd/fuse_virtio.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/virtiofsd/fuse_virtio.c b/tools/virtiofsd/fuse_virtio.c
index fc2564a603..8f4fd165b9 100644
--- a/tools/virtiofsd/fuse_virtio.c
+++ b/tools/virtiofsd/fuse_virtio.c
@@ -716,6 +716,7 @@ static void *fv_queue_thread(void *opaque)
 
         /* Process all the requests. */
         if (!se->thread_pool_size && req_list != NULL) {
+            req_list = g_list_reverse(req_list);
             g_list_foreach(req_list, fv_queue_worker, qi);
             g_list_free(req_list);
             req_list = NULL;
-- 
2.31.1

Re: [PULL 0/2] virtiofs queue

[Peter Maydell]

On Thu, 16 Sept 2021 at 14:58, Dr. David Alan Gilbert (git)
<dgilbert@redhat.com> wrote:
>
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>
> The following changes since commit 57b6f58c1d0df757c9311496c32d502925056894:
>
>   Merge remote-tracking branch 'remotes/hreitz/tags/pull-block-2021-09-15' into staging (2021-09-15 18:55:59 +0100)
>
> are available in the Git repository at:
>
>   https://gitlab.com/dagrh/qemu.git tags/pull-virtiofs-20210916
>
> for you to fetch changes up to 046d91c83caac29e2ba26c63fd7d685a57463f6d:
>
>   virtiofsd: Reverse req_list before processing it (2021-09-16 14:50:48 +0100)
>

gpg says your key has expired; what keyserver can I download an
updated key from?

thanks
-- PMM

Re: [PULL 0/2] virtiofs queue

[Dr. David Alan Gilbert]

* Peter Maydell (peter.maydell@linaro.org) wrote:
> On Thu, 16 Sept 2021 at 14:58, Dr. David Alan Gilbert (git)
> <dgilbert@redhat.com> wrote:
> >
> > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> >
> > The following changes since commit 57b6f58c1d0df757c9311496c32d502925056894:
> >
> >   Merge remote-tracking branch 'remotes/hreitz/tags/pull-block-2021-09-15' into staging (2021-09-15 18:55:59 +0100)
> >
> > are available in the Git repository at:
> >
> >   https://gitlab.com/dagrh/qemu.git tags/pull-virtiofs-20210916
> >
> > for you to fetch changes up to 046d91c83caac29e2ba26c63fd7d685a57463f6d:
> >
> >   virtiofsd: Reverse req_list before processing it (2021-09-16 14:50:48 +0100)
> >
> 
> gpg says your key has expired; what keyserver can I download an
> updated key from?

I pushed an updated one to pgp.mit.edu just before I sent the pull;
I can see it there (although it's a bit slow to respond).

Dave

> thanks
> -- PMM
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [PULL 0/2] virtiofs queue

[Peter Maydell]

On Thu, 16 Sept 2021 at 16:33, Dr. David Alan Gilbert
<dgilbert@redhat.com> wrote:
>
> * Peter Maydell (peter.maydell@linaro.org) wrote:
> > On Thu, 16 Sept 2021 at 14:58, Dr. David Alan Gilbert (git)
> > <dgilbert@redhat.com> wrote:
> > >
> > > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > >
> > > The following changes since commit 57b6f58c1d0df757c9311496c32d502925056894:
> > >
> > >   Merge remote-tracking branch 'remotes/hreitz/tags/pull-block-2021-09-15' into staging (2021-09-15 18:55:59 +0100)
> > >
> > > are available in the Git repository at:
> > >
> > >   https://gitlab.com/dagrh/qemu.git tags/pull-virtiofs-20210916
> > >
> > > for you to fetch changes up to 046d91c83caac29e2ba26c63fd7d685a57463f6d:
> > >
> > >   virtiofsd: Reverse req_list before processing it (2021-09-16 14:50:48 +0100)
> > >
> >
> > gpg says your key has expired; what keyserver can I download an
> > updated key from?
>
> I pushed an updated one to pgp.mit.edu just before I sent the pull;
> I can see it there (although it's a bit slow to respond).

It doesn't seem to respond for me; a recv-keys just hangs.
I recommend keys.openpgp.org or keyserver.ubuntu.com.

-- PMM

Re: [PULL 0/2] virtiofs queue

[Dr. David Alan Gilbert]

* Peter Maydell (peter.maydell@linaro.org) wrote:
> On Thu, 16 Sept 2021 at 16:33, Dr. David Alan Gilbert
> <dgilbert@redhat.com> wrote:
> >
> > * Peter Maydell (peter.maydell@linaro.org) wrote:
> > > On Thu, 16 Sept 2021 at 14:58, Dr. David Alan Gilbert (git)
> > > <dgilbert@redhat.com> wrote:
> > > >
> > > > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > > >
> > > > The following changes since commit 57b6f58c1d0df757c9311496c32d502925056894:
> > > >
> > > >   Merge remote-tracking branch 'remotes/hreitz/tags/pull-block-2021-09-15' into staging (2021-09-15 18:55:59 +0100)
> > > >
> > > > are available in the Git repository at:
> > > >
> > > >   https://gitlab.com/dagrh/qemu.git tags/pull-virtiofs-20210916
> > > >
> > > > for you to fetch changes up to 046d91c83caac29e2ba26c63fd7d685a57463f6d:
> > > >
> > > >   virtiofsd: Reverse req_list before processing it (2021-09-16 14:50:48 +0100)
> > > >
> > >
> > > gpg says your key has expired; what keyserver can I download an
> > > updated key from?
> >
> > I pushed an updated one to pgp.mit.edu just before I sent the pull;
> > I can see it there (although it's a bit slow to respond).
> 
> It doesn't seem to respond for me; a recv-keys just hangs.
> I recommend keys.openpgp.org or keyserver.ubuntu.com.

I've just pushed them to both of those.

Dave

> -- PMM
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [PULL 0/2] virtiofs queue

[Peter Maydell]

On Thu, 16 Sept 2021 at 14:58, Dr. David Alan Gilbert (git)
<dgilbert@redhat.com> wrote:
>
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>
> The following changes since commit 57b6f58c1d0df757c9311496c32d502925056894:
>
>   Merge remote-tracking branch 'remotes/hreitz/tags/pull-block-2021-09-15' into staging (2021-09-15 18:55:59 +0100)
>
> are available in the Git repository at:
>
>   https://gitlab.com/dagrh/qemu.git tags/pull-virtiofs-20210916
>
> for you to fetch changes up to 046d91c83caac29e2ba26c63fd7d685a57463f6d:
>
>   virtiofsd: Reverse req_list before processing it (2021-09-16 14:50:48 +0100)
>
> ----------------------------------------------------------------
> virtiofsd pull 2021-08-16
>
> Two minor fixes; one for performance, the other seccomp
> on s390x.
>
> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/6.2
for any user-visible changes.

-- PMM