Linux-Fsdevel Archive on lore.kernel.org
 help / color / Atom feed
From: Miklos Szeredi <miklos@szeredi.hu>
To: Al Viro <viro@zeniv.linux.org.uk>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	Dave Chinner <david@fromorbit.com>, CAI Qian <caiqian@redhat.com>,
	linux-xfs <linux-xfs@vger.kernel.org>,
	xfs@oss.sgi.com, Jens Axboe <axboe@kernel.dk>,
	Nick Piggin <npiggin@gmail.com>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>
Subject: Re: [PATCH 10/12] new iov_iter flavour: pipe-backed
Date: Thu, 29 Sep 2016 22:53:55 +0200
Message-ID: <CAELBmZDpm635PcTPQfnpLGs2P4bfT6JU+DEuFu9pBut=uzOLHw@mail.gmail.com> (raw)
In-Reply-To: <20160924040117.GP2356@ZenIV.linux.org.uk>

On Sat, Sep 24, 2016 at 6:01 AM, Al Viro <viro@zeniv.linux.org.uk> wrote:
> iov_iter variant for passing data into pipe.  copy_to_iter()
> copies data into page(s) it has allocated and stuffs them into
> the pipe; copy_page_to_iter() stuffs there a reference to the
> page given to it.  Both will try to coalesce if possible.
> iov_iter_zero() is similar to copy_to_iter(); iov_iter_get_pages()
> and friends will do as copy_to_iter() would have and return the
> pages where the data would've been copied.  iov_iter_advance()
> will truncate everything past the spot it has advanced to.
>
> New primitive: iov_iter_pipe(), used for initializing those.
> pipe should be locked all along.
>
> Running out of space acts as fault would for iovec-backed ones;
> in other words, giving it to ->read_iter() may result in short
> read if the pipe overflows, or -EFAULT if it happens with nothing
> copied there.

This is the hardest part of the whole set.  I've been trying to
understand it, but the modular arithmetic makes it really tricky to
read.  Couldn't we have more small inline helpers like next_idx()?

Specific comments inline.

[...]

> +static size_t copy_page_to_iter_pipe(struct page *page, size_t offset, size_t bytes,
> +                        struct iov_iter *i)
> +{
> +       struct pipe_inode_info *pipe = i->pipe;
> +       struct pipe_buffer *buf;
> +       size_t off;
> +       int idx;
> +
> +       if (unlikely(bytes > i->count))
> +               bytes = i->count;
> +
> +       if (unlikely(!bytes))
> +               return 0;
> +
> +       if (!sanity(i))
> +               return 0;
> +
> +       off = i->iov_offset;
> +       idx = i->idx;
> +       buf = &pipe->bufs[idx];
> +       if (off) {
> +               if (offset == off && buf->page == page) {
> +                       /* merge with the last one */
> +                       buf->len += bytes;
> +                       i->iov_offset += bytes;
> +                       goto out;
> +               }
> +               idx = next_idx(idx, pipe);
> +               buf = &pipe->bufs[idx];
> +       }
> +       if (idx == pipe->curbuf && pipe->nrbufs)
> +               return 0;

The EFAULT logic seems to be missing across the board.  And callers
don't expect a zero return value.  Most will loop indefinitely.

[...]

> +static size_t push_pipe(struct iov_iter *i, size_t size,
> +                       int *idxp, size_t *offp)
> +{
> +       struct pipe_inode_info *pipe = i->pipe;
> +       size_t off;
> +       int idx;
> +       ssize_t left;
> +
> +       if (unlikely(size > i->count))
> +               size = i->count;
> +       if (unlikely(!size))
> +               return 0;
> +
> +       left = size;
> +       data_start(i, &idx, &off);
> +       *idxp = idx;
> +       *offp = off;
> +       if (off) {
> +               left -= PAGE_SIZE - off;
> +               if (left <= 0) {
> +                       pipe->bufs[idx].len += size;
> +                       return size;
> +               }
> +               pipe->bufs[idx].len = PAGE_SIZE;
> +               idx = next_idx(idx, pipe);
> +       }
> +       while (idx != pipe->curbuf || !pipe->nrbufs) {
> +               struct page *page = alloc_page(GFP_USER);
> +               if (!page)
> +                       break;

Again, unexpected zero return if this is the first page.  Should
return -ENOMEM?  Some callers only expect -EFAULT, though.

[...]

> +static void pipe_advance(struct iov_iter *i, size_t size)
> +{
> +       struct pipe_inode_info *pipe = i->pipe;
> +       struct pipe_buffer *buf;
> +       size_t off;
> +       int idx;
> +
> +       if (unlikely(i->count < size))
> +               size = i->count;
> +
> +       idx = i->idx;
> +       off = i->iov_offset;
> +       if (size || off) {
> +               /* take it relative to the beginning of buffer */
> +               size += off - pipe->bufs[idx].offset;
> +               while (1) {
> +                       buf = &pipe->bufs[idx];
> +                       if (size > buf->len) {
> +                               size -= buf->len;
> +                               idx = next_idx(idx, pipe);
> +                               off = 0;

off is unused and reassigned before breaking out of the loop.

[...]

> @@ -732,7 +1101,20 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages)
>         if (!size)
>                 return 0;
>
> -       iterate_all_kinds(i, size, v, ({
> +       if (unlikely(i->type & ITER_PIPE)) {
> +               struct pipe_inode_info *pipe = i->pipe;
> +               size_t off;
> +               int idx;
> +
> +               if (!sanity(i))
> +                       return 0;
> +
> +               data_start(i, &idx, &off);
> +               /* some of this one + all after this one */
> +               npages = ((pipe->curbuf - idx - 1) & (pipe->buffers - 1)) + 1;

It's supposed to take i->count into account, no?  And that calculation
will result in really funny things if the pipe is full.  And we can't
return -EFAULT here, since that's not expected by callers...

Thanks,
Miklos

  reply index

Thread overview: 101+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20160908235521.GL2356@ZenIV.linux.org.uk>
     [not found] ` <20160909015324.GD30056@dastard>
     [not found]   ` <CA+55aFzohsUXj_3BeFNr2t50Wm=G+7toRDEz=Tk7VJqP3n1hXQ@mail.gmail.com>
     [not found]     ` <CA+55aFxrqCng2Qxasc9pyMrKUGFjo==fEaFT1vkH9Lncte3RgQ@mail.gmail.com>
     [not found]       ` <20160909023452.GO2356@ZenIV.linux.org.uk>
     [not found]         ` <CA+55aFwHQMjO4-vtfB9-ytc=o+DRo-HXVGckvXLboUxgpwb7_g@mail.gmail.com>
     [not found]           ` <20160909221945.GQ2356@ZenIV.linux.org.uk>
     [not found]             ` <CA+55aFzTOOB6oEVaaGD0N7Uznk-W9+ULPwzsxS_L_oZqGVSeLA@mail.gmail.com>
     [not found]               ` <20160914031648.GB2356@ZenIV.linux.org.uk>
     [not found]                 ` <20160914133925.2fba4629@roar.ozlabs.ibm.com>
2016-09-18  5:33                   ` xfs_file_splice_read: possible circular locking dependency detected Al Viro
2016-09-19  3:08                     ` Nicholas Piggin
2016-09-19  6:11                       ` Al Viro
2016-09-19  7:26                         ` Nicholas Piggin
     [not found]                 ` <CA+55aFznQaOWoSMNphgGJJWZ=8-odrc0DAUMzfGPQe+_N4UgNA@mail.gmail.com>
     [not found]                   ` <20160914042559.GC2356@ZenIV.linux.org.uk>
     [not found]                     ` <20160917082007.GA6489@ZenIV.linux.org.uk>
     [not found]                       ` <20160917190023.GA8039@ZenIV.linux.org.uk>
2016-09-18 19:31                         ` skb_splice_bits() and large chunks in pipe (was " Al Viro
2016-09-18 20:12                           ` Linus Torvalds
2016-09-18 22:31                             ` Al Viro
2016-09-19  0:18                               ` Linus Torvalds
2016-09-23 19:00                         ` [RFC][CFT] splice_read reworked Al Viro
2016-09-23 19:01                           ` [PATCH 01/11] fix memory leaks in tracing_buffers_splice_read() Al Viro
2016-09-23 19:02                           ` [PATCH 02/11] splice_to_pipe(): don't open-code wakeup_pipe_readers() Al Viro
2016-09-23 19:02                           ` [PATCH 03/11] splice: switch get_iovec_page_array() to iov_iter Al Viro
2016-09-23 19:03                           ` [PATCH 04/11] splice: lift pipe_lock out of splice_to_pipe() Al Viro
2016-09-23 19:45                             ` Linus Torvalds
2016-09-23 20:10                               ` Al Viro
2016-09-23 20:36                                 ` Linus Torvalds
2016-09-24  3:59                                   ` Al Viro
2016-09-24 17:29                                     ` Al Viro
2016-09-27 15:38                                       ` Nicholas Piggin
2016-09-27 15:53                                       ` Chuck Lever
2016-09-24  3:59                                   ` [PATCH 04/12] " Al Viro
2016-09-26 13:35                                     ` Miklos Szeredi
2016-09-27  4:14                                       ` Al Viro
2016-12-17 19:54                                     ` Andreas Schwab
2016-12-18 19:28                                       ` Linus Torvalds
2016-12-18 19:57                                         ` Andreas Schwab
2016-12-18 20:12                                         ` Al Viro
2016-12-18 20:30                                           ` Al Viro
2016-12-18 22:10                                             ` Linus Torvalds
2016-12-18 22:18                                               ` Al Viro
2016-12-18 22:22                                                 ` Linus Torvalds
2016-12-18 22:49                                               ` Andreas Schwab
2016-12-21 18:56                                               ` Andreas Schwab
2016-12-21 19:12                                                 ` Linus Torvalds
2016-09-24  4:00                                   ` [PATCH 06/12] new helper: add_to_pipe() Al Viro
2016-09-26 13:49                                     ` Miklos Szeredi
2016-09-24  4:01                                   ` [PATCH 10/12] new iov_iter flavour: pipe-backed Al Viro
2016-09-29 20:53                                     ` Miklos Szeredi [this message]
2016-09-29 22:50                                       ` Al Viro
2016-09-30  7:30                                         ` Miklos Szeredi
2016-10-03  3:34                                           ` [RFC] O_DIRECT vs EFAULT (was Re: [PATCH 10/12] new iov_iter flavour: pipe-backed) Al Viro
2016-10-03 17:07                                             ` Linus Torvalds
2016-10-03 18:54                                               ` Al Viro
2016-09-24  4:01                                   ` [PATCH 11/12] switch generic_file_splice_read() to use of ->read_iter() Al Viro
2016-09-24  4:02                                   ` [PATCH 12/12] switch default_file_splice_read() to use of pipe-backed iov_iter Al Viro
2016-09-23 19:03                           ` [PATCH 05/11] skb_splice_bits(): get rid of callback Al Viro
2016-09-23 19:04                           ` [PATCH 06/11] new helper: add_to_pipe() Al Viro
2016-09-23 19:04                           ` [PATCH 07/11] fuse_dev_splice_read(): switch to add_to_pipe() Al Viro
2016-09-23 19:06                           ` [PATCH 08/11] cifs: don't use memcpy() to copy struct iov_iter Al Viro
2016-09-23 19:08                           ` [PATCH 09/11] fuse_ioctl_copy_user(): don't open-code copy_page_{to,from}_iter() Al Viro
2016-09-26  9:31                             ` Miklos Szeredi
2016-09-23 19:09                           ` [PATCH 10/11] new iov_iter flavour: pipe-backed Al Viro
2016-09-23 19:10                           ` [PATCH 11/11] switch generic_file_splice_read() to use of ->read_iter() Al Viro
2016-09-30 13:32                           ` [RFC][CFT] splice_read reworked CAI Qian
2016-09-30 17:42                             ` CAI Qian
2016-09-30 18:33                               ` CAI Qian
2016-10-03  1:37                                 ` Al Viro
2016-10-03 17:49                                   ` CAI Qian
2016-10-04 17:39                                     ` local DoS - systemd hang or timeout (WAS: Re: [RFC][CFT] splice_read reworked) CAI Qian
2016-10-04 21:42                                       ` tj
2016-10-05 14:09                                         ` CAI Qian
2016-10-05 15:30                                           ` tj
2016-10-05 15:54                                             ` CAI Qian
2016-10-05 18:57                                               ` CAI Qian
2016-10-05 20:05                                                 ` Al Viro
2016-10-06 12:20                                                   ` CAI Qian
2016-10-06 12:25                                                     ` CAI Qian
2016-10-06 16:11                                                       ` CAI Qian
2016-10-06 17:00                                                         ` Linus Torvalds
2016-10-06 18:12                                                           ` CAI Qian
2016-10-07  9:57                                                           ` Dave Chinner
2016-10-07 15:25                                                             ` Linus Torvalds
2016-10-07  7:08                                                       ` Jan Kara
2016-10-07 14:43                                                         ` CAI Qian
2016-10-07 15:27                                                           ` CAI Qian
2016-10-07 18:56                                                             ` CAI Qian
2016-10-09 21:54                                                               ` Dave Chinner
2016-10-10 14:10                                                                 ` CAI Qian
2016-10-10 20:14                                                                   ` CAI Qian
2016-10-10 21:57                                                                   ` Dave Chinner
2016-10-12 19:50                                                                     ` [bisected] " CAI Qian
2016-10-12 20:59                                                                       ` Dave Chinner
2016-10-13 16:25                                                                         ` CAI Qian
2016-10-13 20:49                                                                           ` Dave Chinner
2016-10-13 20:56                                                                             ` CAI Qian
2016-10-09 21:51                                                           ` Dave Chinner
2016-10-21 15:38                                                         ` [4.9-rc1+] overlayfs lockdep CAI Qian
2016-10-24 12:57                                                           ` Miklos Szeredi
2016-10-07  9:27                                                     ` local DoS - systemd hang or timeout (WAS: Re: [RFC][CFT] splice_read reworked) Dave Chinner
2016-10-03  1:42                               ` [RFC][CFT] splice_read reworked Al Viro
2016-10-03 14:06                                 ` CAI Qian
2016-10-03 15:20                                   ` CAI Qian
2016-10-03 21:12                                     ` Dave Chinner
2016-10-04 13:57                                       ` CAI Qian
2016-10-03 20:32                                   ` CAI Qian
2016-10-03 20:35                                     ` Al Viro
2016-10-04 13:29                                       ` CAI Qian
2016-10-04 14:28                                         ` Al Viro
2016-10-04 16:21                                           ` CAI Qian
2016-10-04 20:12                                             ` Al Viro
2016-10-05 14:30                                               ` CAI Qian
2016-10-05 16:07                                                 ` Al Viro
2016-09-19  0:22 skb_splice_bits() and large chunks in pipe (was Re: xfs_file_splice_read: possible circular locking dependency detected Al Viro
2016-09-20  9:51 ` Herbert Xu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAELBmZDpm635PcTPQfnpLGs2P4bfT6JU+DEuFu9pBut=uzOLHw@mail.gmail.com' \
    --to=miklos@szeredi.hu \
    --cc=axboe@kernel.dk \
    --cc=caiqian@redhat.com \
    --cc=david@fromorbit.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-xfs@vger.kernel.org \
    --cc=npiggin@gmail.com \
    --cc=torvalds@linux-foundation.org \
    --cc=viro@zeniv.linux.org.uk \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-Fsdevel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-fsdevel/0 linux-fsdevel/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-fsdevel linux-fsdevel/ https://lore.kernel.org/linux-fsdevel \
		linux-fsdevel@vger.kernel.org
	public-inbox-index linux-fsdevel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-fsdevel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git