From: Jens Axboe <axboe@kernel.dk>
To: Jann Horn <jannh@google.com>
Cc: linux-aio@kvack.org, linux-block@vger.kernel.org,
Linux API <linux-api@vger.kernel.org>,
hch@lst.de, jmoyer@redhat.com, Avi Kivity <avi@scylladb.com>
Subject: Re: [PATCH 12/18] io_uring: add support for pre-mapped user IO buffers
Date: Tue, 29 Jan 2019 15:56:29 -0700 [thread overview]
Message-ID: <dd366179-c003-752f-743d-2bea4f6b796c@kernel.dk> (raw)
In-Reply-To: <CAG48ez2VkeSEZA+9H4UAcg86uC77gU=UkTG8L1V58TuR9pDEqg@mail.gmail.com>
On 1/29/19 3:44 PM, Jann Horn wrote:
> On Tue, Jan 29, 2019 at 8:27 PM Jens Axboe <axboe@kernel.dk> wrote:
>> If we have fixed user buffers, we can map them into the kernel when we
>> setup the io_context. That avoids the need to do get_user_pages() for
>> each and every IO.
>>
>> To utilize this feature, the application must call io_uring_register()
>> after having setup an io_uring context, passing in
>> IORING_REGISTER_BUFFERS as the opcode. The argument must be a pointer
>> to an iovec array, and the nr_args should contain how many iovecs the
>> application wishes to map.
>>
>> If successful, these buffers are now mapped into the kernel, eligible
>> for IO. To use these fixed buffers, the application must use the
>> IORING_OP_READ_FIXED and IORING_OP_WRITE_FIXED opcodes, and then
>> set sqe->index to the desired buffer index. sqe->addr..sqe->addr+seq->len
>> must point to somewhere inside the indexed buffer.
>>
>> The application may register buffers throughout the lifetime of the
>> io_uring context. It can call io_uring_register() with
>> IORING_UNREGISTER_BUFFERS as the opcode to unregister the current set of
>> buffers, and then register a new set. The application need not
>> unregister buffers explicitly before shutting down the io_uring context.
> [...]
>> +static int io_import_fixed(struct io_ring_ctx *ctx, int rw,
>> + const struct io_uring_sqe *sqe,
>> + struct iov_iter *iter)
>> +{
>> + size_t len = READ_ONCE(sqe->len);
>> + struct io_mapped_ubuf *imu;
>> + int buf_index, index;
>> + size_t offset;
>> + u64 buf_addr;
>> +
>> + /* attempt to use fixed buffers without having provided iovecs */
>> + if (unlikely(!ctx->user_bufs))
>> + return -EFAULT;
>> +
>> + buf_index = READ_ONCE(sqe->buf_index);
>> + if (unlikely(buf_index >= ctx->nr_user_bufs))
>> + return -EFAULT;
>
> Nit: If you make the local copy of buf_index unsigned, it is slightly
> easier to see that this code is correct. (I know, it has to be
> positive anyway because the value in shared memory is a u16.)
I'll definitely fit, but I can make it unsigned.
>> + index = array_index_nospec(buf_index, ctx->sq_entries);
>
> This looks weird. Did you mean s/ctx->sq_entries/ctx->nr_user_bufs/?
I did, too much copy/paste for that one. Fixed.
>> + imu = &ctx->user_bufs[index];
>> + buf_addr = READ_ONCE(sqe->addr);
>> + if (buf_addr < imu->ubuf || buf_addr + len > imu->ubuf + imu->len)
>
> This can wrap around if `buf_addr` or `len` is very big, right? Then
> you e.g. get past the first check because `buf_addr` is sufficiently
> big, and get past the second check because `buf_addr + len` wraps
> around and becomes small.
Good point. I wonder if we have a verification helper for something like
this?
>> + return -EFAULT;
>> +
>> + /*
>> + * May not be a start of buffer, set size appropriately
>> + * and advance us to the beginning.
>> + */
>> + offset = buf_addr - imu->ubuf;
>> + iov_iter_bvec(iter, rw, imu->bvec, imu->nr_bvecs, offset + len);
>> + if (offset)
>> + iov_iter_advance(iter, offset);
>> + return 0;
>> +}
>> +
>> static int io_import_iovec(struct io_ring_ctx *ctx, int rw,
>> const struct io_uring_sqe *sqe, struct iovec **iovec,
>> struct iov_iter *iter)
>> {
>> void __user *buf = u64_to_user_ptr(READ_ONCE(sqe->addr));
>> size_t sqe_len = READ_ONCE(sqe->len);
>> + int opcode;
>> +
>> + opcode = READ_ONCE(sqe->opcode);
>> + if (opcode == IORING_OP_READ_FIXED ||
>> + opcode == IORING_OP_WRITE_FIXED) {
>> + ssize_t ret = io_import_fixed(ctx, rw, sqe, iter);
>> + *iovec = NULL;
>> + return ret;
>> + }
> [...]
>>
>> +static inline bool io_sqe_needs_user(const struct io_uring_sqe *sqe)
>> +{
>> + return !(sqe->opcode == IORING_OP_READ_FIXED ||
>> + sqe->opcode == IORING_OP_WRITE_FIXED);
>> +}
>
> This still looks racy to me?
I didn't change it because the below one you quoted below
(io_sq_wq_submit_work()) is using a local copy, but we do need it for
for the SQPOLL io_sq_thread() case. I'll get that one fixed up.
I suspect the easiest fix is to ensure that io_sq_thread() copies the
sqe.
>> +static int io_copy_iov(struct io_ring_ctx *ctx, struct iovec *dst,
>> + void __user *arg, unsigned index)
>> +{
>
> This function doesn't actually use the "ctx" parameter, right? You
> might want to remove it.
It should just use ctx->compat for this check, we're now only calling
in_compat_syscall() when we setup the ctx. This keeps it all in one
spot.
>> + struct iovec __user *src;
>> +
>> +#ifdef CONFIG_COMPAT
>> + if (in_compat_syscall()) {
>> + struct compat_iovec __user *ciovs;
>> + struct compat_iovec ciov;
>> +
>> + ciovs = (struct compat_iovec __user *) arg;
>> + if (copy_from_user(&ciov, &ciovs[index], sizeof(ciov)))
>> + return -EFAULT;
>> +
>> + dst->iov_base = (void __user *) (unsigned long) ciov.iov_base;
>> + dst->iov_len = ciov.iov_len;
>> + return 0;
>> + }
>> +#endif
>> + src = (struct iovec __user *) arg;
>> + if (copy_from_user(dst, &src[index], sizeof(*dst)))
>> + return -EFAULT;
>> + return 0;
>> +}
>> +
>> +static int io_sqe_buffer_register(struct io_ring_ctx *ctx, void __user *arg,
>> + unsigned nr_args)
>> +{
>> + struct vm_area_struct **vmas = NULL;
>> + struct page **pages = NULL;
>> + int i, j, got_pages = 0;
>> + int ret = -EINVAL;
>> +
>> + if (ctx->user_bufs)
>> + return -EBUSY;
>> + if (!nr_args || nr_args > UIO_MAXIOV)
>> + return -EINVAL;
>> +
>> + ctx->user_bufs = kcalloc(nr_args, sizeof(struct io_mapped_ubuf),
>> + GFP_KERNEL);
>> + if (!ctx->user_bufs)
>> + return -ENOMEM;
>> +
>> + if (!capable(CAP_IPC_LOCK))
>> + ctx->user = get_uid(current_user());
>> +
>> + for (i = 0; i < nr_args; i++) {
>> + struct io_mapped_ubuf *imu = &ctx->user_bufs[i];
>> + unsigned long off, start, end, ubuf;
>> + int pret, nr_pages;
>> + struct iovec iov;
>> + size_t size;
>> +
>> + ret = io_copy_iov(ctx, &iov, arg, i);
>> + if (ret)
>> + break;
>> +
>> + /*
>> + * Don't impose further limits on the size and buffer
>> + * constraints here, we'll -EINVAL later when IO is
>> + * submitted if they are wrong.
>> + */
>> + ret = -EFAULT;
>> + if (!iov.iov_base)
>> + goto err;
>> +
>> + /* arbitrary limit, but we need something */
>> + if (iov.iov_len > SZ_1G)
>> + goto err;
>
> You might also want to check for iov_len==0? Otherwise, if iov_base
> isn't page-aligned, the following code might grab a reference to one
> page even though the iov covers zero pages, that'd be kinda weird.
Good catch, will do.
>
>> + ubuf = (unsigned long) iov.iov_base;
>> + end = (ubuf + iov.iov_len + PAGE_SIZE - 1) >> PAGE_SHIFT;
>> + start = ubuf >> PAGE_SHIFT;
>> + nr_pages = end - start;
>> +
>> + ret = io_account_mem(ctx, nr_pages);
>> + if (ret)
>> + goto err;
>> +
>> + if (!pages || nr_pages > got_pages) {
>> + kfree(vmas);
>> + kfree(pages);
>> + pages = kmalloc_array(nr_pages, sizeof(struct page *),
>> + GFP_KERNEL);
>> + vmas = kmalloc_array(nr_pages,
>> + sizeof(struct vma_area_struct *),
>> + GFP_KERNEL);
>> + if (!pages || !vmas) {
>> + io_unaccount_mem(ctx, nr_pages);
>> + goto err;
>> + }
>> + got_pages = nr_pages;
>> + }
>> +
>> + imu->bvec = kmalloc_array(nr_pages, sizeof(struct bio_vec),
>> + GFP_KERNEL);
>> + if (!imu->bvec) {
>> + io_unaccount_mem(ctx, nr_pages);
>> + goto err;
>> + }
>> +
>> + down_write(¤t->mm->mmap_sem);
>
> Is there a reason why you're using down_write() and not down_read()?
> As far as I can tell, down_read() is all you need...
Looks like you are right, I'll change that.
>> + pret = get_user_pages_longterm(ubuf, nr_pages, FOLL_WRITE,
>> + pages, vmas);
>> + if (pret == nr_pages) {
>> + /* don't support file backed memory */
>> + for (j = 0; j < nr_pages; j++) {
>> + struct vm_area_struct *vma = vmas[j];
>> +
>> + if (vma->vm_file) {
>> + ret = -EOPNOTSUPP;
>> + break;
>> + }
>> + }
>
> Are you intentionally doing the check for vma->vm_file instead of
> calling GUP with FOLL_ANON, which would automatically verify
> vma->vm_ops==NULL for you using vma_is_anonymous()? FOLL_ANON is what
> procfs uses to avoid blocking on page faults when reading remote
> process memory via /proc/*/{cmdline,environ}. I don't entirely
> understand the motivation for this check, so I can't really tell
> whether FOLL_ANON would do the job.
I wasn't aware of FOLL_ANON, it looks exactly like what I need. All I
care about is the mapping being anon, and not file backed. If FOLL_ANON
ensures me that (or fails), then that'll kill this vma checking code.
Thanks!
--
Jens Axboe
next prev parent reply other threads:[~2019-01-29 22:56 UTC|newest]
Thread overview: 76+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-29 19:26 [PATCHSET v9] io_uring IO interface Jens Axboe
2019-01-29 19:26 ` [PATCH 01/18] fs: add an iopoll method to struct file_operations Jens Axboe
2019-01-29 19:26 ` [PATCH 02/18] block: wire up block device iopoll method Jens Axboe
2019-01-29 19:26 ` [PATCH 03/18] block: add bio_set_polled() helper Jens Axboe
2019-01-29 19:26 ` [PATCH 04/18] iomap: wire up the iopoll method Jens Axboe
2019-01-29 19:26 ` [PATCH 05/18] Add io_uring IO interface Jens Axboe
2019-01-29 19:26 ` [PATCH 06/18] io_uring: add fsync support Jens Axboe
2019-01-29 19:26 ` [PATCH 07/18] io_uring: support for IO polling Jens Axboe
2019-01-29 20:47 ` Jann Horn
2019-01-29 20:56 ` Jens Axboe
2019-01-29 21:10 ` Jann Horn
2019-01-29 21:33 ` Jens Axboe
2019-01-29 19:26 ` [PATCH 08/18] fs: add fget_many() and fput_many() Jens Axboe
2019-01-29 19:26 ` [PATCH 09/18] io_uring: use fget/fput_many() for file references Jens Axboe
2019-01-29 23:31 ` Jann Horn
2019-01-29 23:44 ` Jens Axboe
2019-01-30 15:33 ` Jens Axboe
2019-01-29 19:26 ` [PATCH 10/18] io_uring: batch io_kiocb allocation Jens Axboe
2019-01-29 19:26 ` [PATCH 11/18] block: implement bio helper to add iter bvec pages to bio Jens Axboe
2019-01-29 19:26 ` [PATCH 12/18] io_uring: add support for pre-mapped user IO buffers Jens Axboe
2019-01-29 22:44 ` Jann Horn
2019-01-29 22:56 ` Jens Axboe [this message]
2019-01-29 23:03 ` Jann Horn
2019-01-29 23:06 ` Jens Axboe
2019-01-29 23:08 ` Jann Horn
2019-01-29 23:14 ` Jens Axboe
2019-01-29 23:42 ` Jann Horn
2019-01-29 23:51 ` Jens Axboe
2019-01-29 19:26 ` [PATCH 13/18] io_uring: add file set registration Jens Axboe
2019-01-30 1:29 ` Jann Horn
2019-01-30 15:35 ` Jens Axboe
2019-02-04 2:56 ` Al Viro
2019-02-05 2:19 ` Jens Axboe
2019-02-05 17:57 ` Jens Axboe
2019-02-05 19:08 ` Jens Axboe
2019-02-06 0:27 ` Jens Axboe
2019-02-06 1:01 ` Al Viro
2019-02-06 17:56 ` Jens Axboe
2019-02-07 4:05 ` Al Viro
2019-02-07 16:14 ` Jens Axboe
2019-02-07 16:30 ` Al Viro
2019-02-07 16:35 ` Jens Axboe
2019-02-07 16:51 ` Al Viro
2019-02-06 0:56 ` Al Viro
2019-02-06 13:41 ` Jens Axboe
2019-02-07 4:00 ` Al Viro
2019-02-07 9:22 ` Miklos Szeredi
2019-02-07 13:31 ` Al Viro
2019-02-07 14:20 ` Miklos Szeredi
2019-02-07 15:20 ` Al Viro
2019-02-07 15:27 ` Miklos Szeredi
2019-02-07 16:26 ` Al Viro
2019-02-07 19:08 ` Miklos Szeredi
2019-02-07 18:45 ` Jens Axboe
2019-02-07 18:58 ` Jens Axboe
2019-02-11 15:55 ` Jonathan Corbet
2019-02-11 17:35 ` Al Viro
2019-02-11 20:33 ` Jonathan Corbet
2019-01-29 19:26 ` [PATCH 14/18] io_uring: add submission polling Jens Axboe
2019-01-29 19:26 ` [PATCH 15/18] io_uring: add io_kiocb ref count Jens Axboe
2019-01-29 19:27 ` [PATCH 16/18] io_uring: add support for IORING_OP_POLL Jens Axboe
2019-01-29 19:27 ` [PATCH 17/18] io_uring: allow workqueue item to handle multiple buffered requests Jens Axboe
2019-01-29 19:27 ` [PATCH 18/18] io_uring: add io_uring_event cache hit information Jens Axboe
-- strict thread matches above, loose matches on Subject: below --
2019-02-07 19:55 [PATCHSET v12] io_uring IO interface Jens Axboe
2019-02-07 19:55 ` [PATCH 12/18] io_uring: add support for pre-mapped user IO buffers Jens Axboe
2019-02-07 20:57 ` Jeff Moyer
2019-02-07 21:02 ` Jens Axboe
2019-02-07 22:38 ` Jeff Moyer
2019-02-07 22:47 ` Jens Axboe
2019-02-01 15:23 [PATCHSET v11] io_uring IO interface Jens Axboe
2019-02-01 15:24 ` [PATCH 12/18] io_uring: add support for pre-mapped user IO buffers Jens Axboe
2019-01-30 21:55 [PATCHSET v10] io_uring IO interface Jens Axboe
2019-01-30 21:55 ` [PATCH 12/18] io_uring: add support for pre-mapped user IO buffers Jens Axboe
2019-01-28 21:35 [PATCHSET v8] io_uring IO interface Jens Axboe
2019-01-28 21:35 ` [PATCH 12/18] io_uring: add support for pre-mapped user IO buffers Jens Axboe
2019-01-28 23:35 ` Jann Horn
2019-01-28 23:50 ` Jens Axboe
2019-01-29 0:36 ` Jann Horn
2019-01-29 1:25 ` Jens Axboe
2019-01-23 15:35 [PATCHSET v7] io_uring IO interface Jens Axboe
2019-01-23 15:35 ` [PATCH 12/18] io_uring: add support for pre-mapped user IO buffers Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=dd366179-c003-752f-743d-2bea4f6b796c@kernel.dk \
--to=axboe@kernel.dk \
--cc=avi@scylladb.com \
--cc=hch@lst.de \
--cc=jannh@google.com \
--cc=jmoyer@redhat.com \
--cc=linux-aio@kvack.org \
--cc=linux-api@vger.kernel.org \
--cc=linux-block@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).