From: Greg KH <gregkh@linuxfoundation.org>
To: Jan Ziak <0xe2.0x9a.0x9b@gmail.com>
Cc: Andreas Dilger <adilger@dilger.ca>,
Matthew Wilcox <willy@infradead.org>,
linux-api@vger.kernel.org, linux-fsdevel@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
linux-man@vger.kernel.org, mtk.manpages@gmail.com,
shuah@kernel.org, viro@zeniv.linux.org.uk
Subject: Re: [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster
Date: Sun, 5 Jul 2020 14:00:03 +0200 [thread overview]
Message-ID: <20200705120003.GC1227929@kroah.com> (raw)
In-Reply-To: <CAODFU0qT07ERWVH7F3rO1CK6CckmoF4p8ArHk09S9DCojD8M4w@mail.gmail.com>
On Sun, Jul 05, 2020 at 09:25:39AM +0200, Jan Ziak wrote:
> On Sun, Jul 5, 2020 at 8:32 AM Andreas Dilger <adilger@dilger.ca> wrote:
> >
> > On Jul 4, 2020, at 8:46 PM, Jan Ziak <0xe2.0x9a.0x9b@gmail.com> wrote:
> > >
> > > On Sun, Jul 5, 2020 at 4:16 AM Matthew Wilcox <willy@infradead.org> wrote:
> > >>
> > >> On Sun, Jul 05, 2020 at 04:06:22AM +0200, Jan Ziak wrote:
> > >>> Hello
> > >>>
> > >>> At first, I thought that the proposed system call is capable of
> > >>> reading *multiple* small files using a single system call - which
> > >>> would help increase HDD/SSD queue utilization and increase IOPS (I/O
> > >>> operations per second) - but that isn't the case and the proposed
> > >>> system call can read just a single file.
> > >>>
> > >>> Without the ability to read multiple small files using a single system
> > >>> call, it is impossible to increase IOPS (unless an application is
> > >>> using multiple reader threads or somehow instructs the kernel to
> > >>> prefetch multiple files into memory).
> > >>
> > >> What API would you use for this?
> > >>
> > >> ssize_t readfiles(int dfd, char **files, void **bufs, size_t *lens);
> > >>
> > >> I pretty much hate this interface, so I hope you have something better
> > >> in mind.
> > >
> > > I am proposing the following:
> > >
> > > struct readfile_t {
> > > int dirfd;
> > > const char *pathname;
> > > void *buf;
> > > size_t count;
> > > int flags;
> > > ssize_t retval; // set by kernel
> > > int reserved; // not used by kernel
> > > };
> >
> > If you are going to pass a struct from userspace to the kernel, it
> > should not mix int and pointer types (which may be 64-bit values,
> > so that there are not structure packing issues, like:
> >
> > struct readfile {
> > int dirfd;
> > int flags;
> > const char *pathname;
> > void *buf;
> > size_t count;
> > ssize_t retval;
> > };
> >
> > It would be better if "retval" was returned in "count", so that
> > the structure fits nicely into 32 bytes on a 64-bit system, instead
> > of being 40 bytes per entry, which adds up over many entries, like.
>
> I know what you mean and it is a valid point, but in my opinion it
> shouldn't (in most cases) be left to the programmer to decide what the
> binary layout of a data structure is - instead it should be left to an
> optimizing compiler to decide it.
We don't get that luxury when creating user/kernel apis in C, sorry.
I suggest using the pahole tool if you are interested in seeing the
"best" way a structure can be layed out, it can perform that
optimization for you so that you know how to fix your code.
thanks,
greg k-h
next prev parent reply other threads:[~2020-07-05 12:00 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-05 2:06 [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster Jan Ziak
2020-07-05 2:16 ` Matthew Wilcox
2020-07-05 2:46 ` Jan Ziak
2020-07-05 3:12 ` Matthew Wilcox
2020-07-05 3:18 ` Jan Ziak
2020-07-05 3:27 ` Matthew Wilcox
2020-07-05 4:09 ` Jan Ziak
2020-07-05 11:58 ` Greg KH
2020-07-06 6:07 ` Jan Ziak
2020-07-06 11:11 ` Matthew Wilcox
2020-07-06 11:18 ` Greg KH
2020-07-05 8:07 ` Vito Caputo
2020-07-05 11:44 ` Greg KH
2020-07-05 20:34 ` Vito Caputo
2020-07-05 6:32 ` Andreas Dilger
2020-07-05 7:25 ` Jan Ziak
2020-07-05 12:00 ` Greg KH [this message]
2020-07-05 11:50 ` Greg KH
2020-07-14 6:51 ` Pavel Machek
2020-07-14 8:07 ` Miklos Szeredi
2020-07-14 11:34 ` Pavel Begunkov
2020-07-14 11:55 ` Miklos Szeredi
2020-07-15 8:31 ` Pavel Begunkov
2020-07-15 8:41 ` Miklos Szeredi
2020-07-15 8:49 ` Pavel Begunkov
2020-07-15 9:00 ` Pavel Begunkov
2020-07-15 11:17 ` Miklos Szeredi
-- strict thread matches above, loose matches on Subject: below --
2020-07-04 14:02 Greg Kroah-Hartman
2020-07-04 19:30 ` Al Viro
2020-07-05 11:47 ` Greg Kroah-Hartman
2020-07-06 17:25 ` Dave Martin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200705120003.GC1227929@kroah.com \
--to=gregkh@linuxfoundation.org \
--cc=0xe2.0x9a.0x9b@gmail.com \
--cc=adilger@dilger.ca \
--cc=linux-api@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-man@vger.kernel.org \
--cc=mtk.manpages@gmail.com \
--cc=shuah@kernel.org \
--cc=viro@zeniv.linux.org.uk \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).