From: Amir Goldstein <amir73il@gmail.com>
To: Jan Kara <jack@suse.cz>
Cc: Shakeel Butt <shakeelb@google.com>,
Yang Shi <yang.s@alibaba-inc.com>,
Michal Hocko <mhocko@kernel.org>,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
Linux MM <linux-mm@kvack.org>,
LKML <linux-kernel@vger.kernel.org>,
linux-api@vger.kernel.org
Subject: Re: [PATCH v2] fs: fsnotify: account fsnotify metadata to kmemcg
Date: Tue, 20 Feb 2018 22:30:25 +0200 [thread overview]
Message-ID: <CAOQ4uxjAaRXPEwbqEMqL9Jr4-JhAscYcFtc01EMQbm5yEafq2Q@mail.gmail.com> (raw)
In-Reply-To: <20180220124354.6awua447q55lfduf@quack2.suse.cz>
On Tue, Feb 20, 2018 at 2:43 PM, Jan Kara <jack@suse.cz> wrote:
> On Mon 19-02-18 21:07:28, Amir Goldstein wrote:
[...]
>>
>> I just feel sorry about passing an opportunity to improve functionality.
>> The fact that fanotify does not have a way for defining the events queue
>> size is a deficiency IMO, one which I had to work around in the past.
>> I find that assigning group to memgc and configure memcg to desired
>> memory limit and getting Q_OVERFLOW on failure to allocate event
>> is going to be a proper way of addressing this deficiency.
>
> So if you don't pass FAN_Q_UNLIMITED, you will get queue with a fixed size
> and will get Q_OVERFLOW if that is exceeded. So is your concern that you'd
> like some other fixed limit? Larger one or smaller one and for what
> reason?
>
My use case was that with the default queue size, I would get Q_OVERFLOW
on bursty fs workloads, but using FAN_Q_UNLIMITED and allowing to
consume entire system memory with events was not a desired alternative.
The actual queue size was not important, only allowing admin to tune the
system to bursty workloads without overflowing the event queue.
Something like FAN_Q_BESTEFFORT (i.e. Q_OVERFLOW on ENOMEM)
+ allowing to restrict event allocation to memcg, would allow admin to tune
the system to bursty workloads.
>> But if you don't think we should bind these 2 things together,
>> I'll let Shakeel decide if he want to pursue the Q_OVERFLOW change
>> or not.
>
> So if there is still some uncovered use case for finer tuning of event
> queue length than setting or not setting FAN_Q_UNLIMITED (+ possibly
> putting the task to memcg to limit memory usage), we can talk about how to
> address that but at this point I don't see a strong reason to bind this to
> whether / how events are accounted to memcg...
Agreed.
>
> And we still need to make sure we properly do ENOMEM -> Q_OVERFLOW
> translation and use GFP_NOFAIL for FAN_Q_UNLIMITED groups before merging
Good. it wasn't clear to me from your summary if were going to require
ENOEM -> Q_OVERFLOW before merging this work. If you put it this way,
I think it makes sense to let user to choose between GFP_NOFAIL and
Q_OVERFLOW behavior when queue is not limited, for example by using new
fanotify_init flag FAN_Q_BESTEFFORT (or better name), but I have no problem
with postponing that for later.
Thanks,
Amir.
next prev parent reply other threads:[~2018-02-20 20:30 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-10-27 18:22 [PATCH v2] fs: fsnotify: account fsnotify metadata to kmemcg Yang Shi
2017-10-28 14:19 ` Amir Goldstein
2017-10-29 2:39 ` Matthew Wilcox
2017-10-30 12:43 ` Jan Kara
2017-10-30 16:39 ` Yang Shi
2017-10-31 10:12 ` Jan Kara
2017-10-31 16:44 ` Yang Shi
2017-11-01 15:15 ` Jan Kara
2017-11-09 13:54 ` Michal Hocko
2017-11-13 19:10 ` Yang Shi
2017-11-14 9:39 ` Michal Hocko
2017-11-14 17:32 ` Yang Shi
2017-11-15 9:31 ` Jan Kara
2018-01-19 15:02 ` Shakeel Butt
2018-01-22 20:31 ` Amir Goldstein
2018-01-24 10:34 ` Jan Kara
2018-01-24 11:12 ` Amir Goldstein
2018-01-25 1:08 ` Shakeel Butt
2018-01-25 1:54 ` Al Viro
2018-01-25 2:15 ` Shakeel Butt
2018-01-25 7:51 ` Amir Goldstein
2018-01-25 20:20 ` Shakeel Butt
2018-01-25 20:36 ` Amir Goldstein
2018-02-13 6:30 ` Amir Goldstein
2018-02-13 21:10 ` Shakeel Butt
2018-02-13 21:54 ` Amir Goldstein
2018-02-13 22:20 ` Shakeel Butt
2018-02-14 1:59 ` Shakeel Butt
2018-02-14 8:38 ` Amir Goldstein
2018-02-19 13:50 ` Jan Kara
2018-02-19 19:07 ` Amir Goldstein
2018-02-20 12:43 ` Jan Kara
2018-02-20 19:20 ` Shakeel Butt
2018-02-20 20:30 ` Amir Goldstein [this message]
2018-02-14 9:00 ` Amir Goldstein
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAOQ4uxjAaRXPEwbqEMqL9Jr4-JhAscYcFtc01EMQbm5yEafq2Q@mail.gmail.com \
--to=amir73il@gmail.com \
--cc=jack@suse.cz \
--cc=linux-api@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=shakeelb@google.com \
--cc=yang.s@alibaba-inc.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).