fstests.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Andreas Grünbacher" <andreas.gruenbacher@gmail.com>
To: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: xfs <linux-xfs@vger.kernel.org>,
	Al Viro <viro@zeniv.linux.org.uk>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	fstests <fstests@vger.kernel.org>
Subject: Re: [RFC PATCH] generic: test splice() with pipes
Date: Fri, 22 Nov 2019 02:27:46 +0100	[thread overview]
Message-ID: <CAHpGcMK_pJA1KU0fbX28e41a8X9Fa7Kw12k8=P955LUW8yYEkw@mail.gmail.com> (raw)
In-Reply-To: <20191121191453.GN6211@magnolia>

Am Do., 21. Nov. 2019 um 20:14 Uhr schrieb Darrick J. Wong
<darrick.wong@oracle.com>:
> On Thu, Nov 21, 2019 at 07:48:54PM +0100, Andreas Grünbacher wrote:
> > Am Do., 21. Nov. 2019 um 18:01 Uhr schrieb Darrick J. Wong
> > <darrick.wong@oracle.com>:
> > > On Thu, Aug 29, 2019 at 05:44:07PM -0700, Darrick J. Wong wrote:
> > > > From: Darrick J. Wong <darrick.wong@oracle.com>
> > > >
> > > > Andreas Grünbacher reports that on the two filesystems that support
> > > > iomap directio, it's possible for splice() to return -EAGAIN (instead of
> > > > a short splice) if the pipe being written to has less space available in
> > > > its pipe buffers than the length supplied by the calling process.
> > > >
> > > > This is a regression test to check for correct operation.
> > > >
> > > > XXX Andreas: Since you wrote the C reproducer, can you send me the
> > > > proper copyright and author attribution statement for the C program?
> > >
> > > Ping?  Andreas, can I get the above info so I can get this moving again?
> >
> > Oops, sure, this is:
> >
> > Copyright (c) 2019 RedHat Inc.  All Rights Reserved.
> > Author: Andreas Gruenbacher <agruenba@redhat.com>
>
> Ok thanks.  It's appropriate to tag it as GPL v2 licensed, correct?

My prevous reply didn't quite make it, so again, yes, that's correct.

Thanks,
Andreas

> --D
>
> > > --D
> > >
> > > > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > > > ---
> > > >  .gitignore            |    1
> > > >  src/Makefile          |    2 -
> > > >  src/splice-test.c     |  173 +++++++++++++++++++++++++++++++++++++++++++++++++
> > > >  tests/generic/720     |   41 ++++++++++++
> > > >  tests/generic/720.out |    7 ++
> > > >  tests/generic/group   |    1
> > > >  6 files changed, 224 insertions(+), 1 deletion(-)
> > > >  create mode 100644 src/splice-test.c
> > > >  create mode 100755 tests/generic/720
> > > >  create mode 100644 tests/generic/720.out
> > > >
> > > > diff --git a/.gitignore b/.gitignore
> > > > index c8c815f9..26d4da11 100644
> > > > --- a/.gitignore
> > > > +++ b/.gitignore
> > > > @@ -112,6 +112,7 @@
> > > >  /src/runas
> > > >  /src/seek_copy_test
> > > >  /src/seek_sanity_test
> > > > +/src/splice-test
> > > >  /src/stale_handle
> > > >  /src/stat_test
> > > >  /src/swapon
> > > > diff --git a/src/Makefile b/src/Makefile
> > > > index c4fcf370..2920dfb1 100644
> > > > --- a/src/Makefile
> > > > +++ b/src/Makefile
> > > > @@ -28,7 +28,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
> > > >       attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type \
> > > >       dio-invalidate-cache stat_test t_encrypted_d_revalidate \
> > > >       attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles \
> > > > -     fscrypt-crypt-util bulkstat_null_ocount
> > > > +     fscrypt-crypt-util bulkstat_null_ocount splice-test
> > > >
> > > >  SUBDIRS = log-writes perf
> > > >
> > > > diff --git a/src/splice-test.c b/src/splice-test.c
> > > > new file mode 100644
> > > > index 00000000..d3c12075
> > > > --- /dev/null
> > > > +++ b/src/splice-test.c
> > > > @@ -0,0 +1,173 @@
> > > > +// SPDX-License-Identifier: GPL-2.0-or-later
> > > > +/*
> > > > + * Copyright (C) 2019 ????????????????????????????
> > > > + * Author:
> > > > + *
> > > > + * Make sure that reading and writing to a pipe via splice.
> > > > + */
> > > > +#include <sys/types.h>
> > > > +#include <sys/stat.h>
> > > > +#include <sys/wait.h>
> > > > +#include <unistd.h>
> > > > +#include <fcntl.h>
> > > > +#include <err.h>
> > > > +
> > > > +#include <stdlib.h>
> > > > +#include <stdio.h>
> > > > +#include <stdbool.h>
> > > > +#include <string.h>
> > > > +#include <errno.h>
> > > > +
> > > > +#define SECTOR_SIZE 512
> > > > +#define BUFFER_SIZE (150 * SECTOR_SIZE)
> > > > +
> > > > +void read_from_pipe(int fd, const char *filename, size_t size)
> > > > +{
> > > > +     char buffer[SECTOR_SIZE];
> > > > +     size_t sz;
> > > > +     ssize_t ret;
> > > > +
> > > > +     while (size) {
> > > > +             sz = size;
> > > > +             if (sz > sizeof buffer)
> > > > +                     sz = sizeof buffer;
> > > > +             ret = read(fd, buffer, sz);
> > > > +             if (ret < 0)
> > > > +                     err(1, "read: %s", filename);
> > > > +             if (ret == 0) {
> > > > +                     fprintf(stderr, "read: %s: unexpected EOF\n", filename);
> > > > +                     exit(1);
> > > > +             }
> > > > +             size -= sz;
> > > > +     }
> > > > +}
> > > > +
> > > > +void do_splice1(int fd, const char *filename, size_t size)
> > > > +{
> > > > +     bool retried = false;
> > > > +     int pipefd[2];
> > > > +
> > > > +     if (pipe(pipefd) == -1)
> > > > +             err(1, "pipe");
> > > > +     while (size) {
> > > > +             ssize_t spliced;
> > > > +
> > > > +             spliced = splice(fd, NULL, pipefd[1], NULL, size, SPLICE_F_MOVE);
> > > > +             if (spliced == -1) {
> > > > +                     if (errno == EAGAIN && !retried) {
> > > > +                             retried = true;
> > > > +                             fprintf(stderr, "retrying splice\n");
> > > > +                             sleep(1);
> > > > +                             continue;
> > > > +                     }
> > > > +                     err(1, "splice");
> > > > +             }
> > > > +             read_from_pipe(pipefd[0], filename, spliced);
> > > > +             size -= spliced;
> > > > +     }
> > > > +     close(pipefd[0]);
> > > > +     close(pipefd[1]);
> > > > +}
> > > > +
> > > > +void do_splice2(int fd, const char *filename, size_t size)
> > > > +{
> > > > +     bool retried = false;
> > > > +     int pipefd[2];
> > > > +     int pid;
> > > > +
> > > > +     if (pipe(pipefd) == -1)
> > > > +             err(1, "pipe");
> > > > +
> > > > +     pid = fork();
> > > > +     if (pid == 0) {
> > > > +             close(pipefd[1]);
> > > > +             read_from_pipe(pipefd[0], filename, size);
> > > > +             exit(0);
> > > > +     } else {
> > > > +             close(pipefd[0]);
> > > > +             while (size) {
> > > > +                     ssize_t spliced;
> > > > +
> > > > +                     spliced = splice(fd, NULL, pipefd[1], NULL, size, SPLICE_F_MOVE);
> > > > +                     if (spliced == -1) {
> > > > +                             if (errno == EAGAIN && !retried) {
> > > > +                                     retried = true;
> > > > +                                     fprintf(stderr, "retrying splice\n");
> > > > +                                     sleep(1);
> > > > +                                     continue;
> > > > +                             }
> > > > +                             err(1, "splice");
> > > > +                     }
> > > > +                     size -= spliced;
> > > > +             }
> > > > +             close(pipefd[1]);
> > > > +             waitpid(pid, NULL, 0);
> > > > +     }
> > > > +}
> > > > +
> > > > +void usage(const char *argv0)
> > > > +{
> > > > +     fprintf(stderr, "USAGE: %s [-rd] {filename}\n", basename(argv0));
> > > > +     exit(2);
> > > > +}
> > > > +
> > > > +int main(int argc, char *argv[])
> > > > +{
> > > > +     void (*do_splice)(int fd, const char *filename, size_t size);
> > > > +     const char *filename;
> > > > +     char *buffer;
> > > > +     int opt, open_flags, fd;
> > > > +     ssize_t ret;
> > > > +
> > > > +     do_splice = do_splice1;
> > > > +     open_flags = O_CREAT | O_TRUNC | O_RDWR | O_DIRECT;
> > > > +
> > > > +     while ((opt = getopt(argc, argv, "rd")) != -1) {
> > > > +             switch(opt) {
> > > > +             case 'r':
> > > > +                     do_splice = do_splice2;
> > > > +                     break;
> > > > +             case 'd':
> > > > +                     open_flags &= ~O_DIRECT;
> > > > +                     break;
> > > > +             default:  /* '?' */
> > > > +                     usage(argv[0]);
> > > > +             }
> > > > +     }
> > > > +
> > > > +     if (optind >= argc)
> > > > +             usage(argv[0]);
> > > > +     filename = argv[optind];
> > > > +
> > > > +     printf("%s reader %s O_DIRECT\n",
> > > > +                do_splice == do_splice1 ? "sequential" : "concurrent",
> > > > +                (open_flags & O_DIRECT) ? "with" : "without");
> > > > +
> > > > +     buffer = aligned_alloc(SECTOR_SIZE, BUFFER_SIZE);
> > > > +     if (buffer == NULL)
> > > > +             err(1, "aligned_alloc");
> > > > +
> > > > +     fd = open(filename, open_flags, 0666);
> > > > +     if (fd == -1)
> > > > +             err(1, "open: %s", filename);
> > > > +
> > > > +     memset(buffer, 'x', BUFFER_SIZE);
> > > > +     ret = write(fd, buffer, BUFFER_SIZE);
> > > > +     if (ret < 0)
> > > > +             err(1, "write: %s", filename);
> > > > +     if (ret != BUFFER_SIZE) {
> > > > +             fprintf(stderr, "%s: short write\n", filename);
> > > > +             exit(1);
> > > > +     }
> > > > +
> > > > +     ret = lseek(fd, 0, SEEK_SET);
> > > > +     if (ret != 0)
> > > > +             err(1, "lseek: %s", filename);
> > > > +
> > > > +     do_splice(fd, filename, BUFFER_SIZE);
> > > > +
> > > > +     if (unlink(filename) == -1)
> > > > +             err(1, "unlink: %s", filename);
> > > > +
> > > > +     return 0;
> > > > +}
> > > > diff --git a/tests/generic/720 b/tests/generic/720
> > > > new file mode 100755
> > > > index 00000000..b7f09c40
> > > > --- /dev/null
> > > > +++ b/tests/generic/720
> > > > @@ -0,0 +1,41 @@
> > > > +#! /bin/bash
> > > > +# SPDX-License-Identifier: GPL-2.0-or-later
> > > > +# Copyright (c) 2019, Oracle and/or its affiliates.  All Rights Reserved.
> > > > +#
> > > > +# FS QA Test No. 720
> > > > +#
> > > > +# Test using splice() to read from pipes.
> > > > +
> > > > +seq=`basename $0`
> > > > +seqres=$RESULT_DIR/$seq
> > > > +echo "QA output created by $seq"
> > > > +
> > > > +here=`pwd`
> > > > +tmp=/tmp/$$
> > > > +status=1    # failure is the default!
> > > > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > > > +
> > > > +_cleanup()
> > > > +{
> > > > +     cd /
> > > > +     rm -f $TEST_DIR/a
> > > > +}
> > > > +
> > > > +# get standard environment, filters and checks
> > > > +. ./common/rc
> > > > +
> > > > +# real QA test starts here
> > > > +_supported_os Linux
> > > > +_supported_fs generic
> > > > +_require_test
> > > > +
> > > > +rm -f $seqres.full
> > > > +
> > > > +src/splice-test -r $TEST_DIR/a
> > > > +src/splice-test -rd $TEST_DIR/a
> > > > +src/splice-test $TEST_DIR/a
> > > > +src/splice-test -d $TEST_DIR/a
> > > > +
> > > > +# success, all done
> > > > +status=0
> > > > +exit
> > > > diff --git a/tests/generic/720.out b/tests/generic/720.out
> > > > new file mode 100644
> > > > index 00000000..b0fc9935
> > > > --- /dev/null
> > > > +++ b/tests/generic/720.out
> > > > @@ -0,0 +1,7 @@
> > > > +QA output created by 720
> > > > +concurrent reader with O_DIRECT
> > > > +concurrent reader with O_DIRECT
> > > > +concurrent reader without O_DIRECT
> > > > +concurrent reader without O_DIRECT
> > > > +sequential reader with O_DIRECT
> > > > +sequential reader without O_DIRECT
> > > > diff --git a/tests/generic/group b/tests/generic/group
> > > > index cd418106..f75d4e60 100644
> > > > --- a/tests/generic/group
> > > > +++ b/tests/generic/group
> > > > @@ -569,3 +569,4 @@
> > > >  564 auto quick copy_range
> > > >  565 auto quick copy_range
> > > >  719 auto quick quota metadata
> > > > +720 auto quick rw pipe splice
> >
> > Thanks,
> > Andreas

      reply	other threads:[~2019-11-22  1:28 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20190829161155.GA5360@magnolia>
2019-08-30  0:44 ` [RFC PATCH] generic: test splice() with pipes Darrick J. Wong
2019-09-02  2:20   ` Zorro Lang
2019-09-02 16:56     ` Darrick J. Wong
2019-09-03  3:19       ` Zorro Lang
2019-11-21 17:01   ` Darrick J. Wong
2019-11-21 18:48     ` Andreas Grünbacher
2019-11-21 19:14       ` Darrick J. Wong
2019-11-22  1:27         ` Andreas Grünbacher [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAHpGcMK_pJA1KU0fbX28e41a8X9Fa7Kw12k8=P955LUW8yYEkw@mail.gmail.com' \
    --to=andreas.gruenbacher@gmail.com \
    --cc=darrick.wong@oracle.com \
    --cc=fstests@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-xfs@vger.kernel.org \
    --cc=viro@zeniv.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).