From: Konstantin Khlebnikov <khlebnikov@yandex-team.ru> To: Tejun Heo <tj@kernel.org> Cc: Greg Thelen <gthelen@google.com>, Johannes Weiner <hannes@cmpxchg.org>, Michal Hocko <mhocko@suse.cz>, Cgroups <cgroups@vger.kernel.org>, "linux-mm@kvack.org" <linux-mm@kvack.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Jan Kara <jack@suse.cz>, Dave Chinner <david@fromorbit.com>, Jens Axboe <axboe@kernel.dk>, Christoph Hellwig <hch@infradead.org>, Li Zefan <lizefan@huawei.com>, Hugh Dickins <hughd@google.com>, Roman Gushchin <klamm@yandex-team.ru> Subject: Re: [RFC] Making memcg track ownership per address_space or anon_vma Date: Wed, 04 Feb 2015 20:58:21 +0300 [thread overview] Message-ID: <54D25DBD.5080009@yandex-team.ru> (raw) In-Reply-To: <20150204171512.GB18858@htj.dyndns.org> On 04.02.2015 20:15, Tejun Heo wrote: > Hello, > > On Wed, Feb 04, 2015 at 01:49:08PM +0300, Konstantin Khlebnikov wrote: >> I think important shared data must be handled and protected explicitly. >> That 'catch-all' shared container could be separated into several > > I kinda disagree. That'd be a major pain in the ass to use and you > wouldn't know when you got something wrong unless it actually goes > wrong and you know enough about the innerworkings to look for that. > Doesn't sound like a sound design to me. > >> memory cgroups depending on importance of files: glibc protected >> with soft guarantee, less important stuff is placed into another >> cgroup and cannot push top-priority libraries out of ram. > > That sounds extremely painful. I mean this thing _could_ be controlled more precisely. Even if default policy works for 99% users manual override is still required for 1% or if something goes wrong. > >> If shared files are free for use then that 'shared' container must be >> ready to keep them in memory. Otherwise this need to be fixed at the >> container side: we could ignore mlock for shared inodes or amount of >> such vmas might be limited in per-container basis. >> >> But sharing responsibility for shared file is vague concept: memory >> usage and limit of container must depends only on its own behavior not >> on neighbors at the same machine. >> >> >> Generally incidental sharing could be handled as temporary sharing: >> default policy (if inode isn't pinned to memory cgroup) after some >> time should detect that inode is no longer shared and migrate it into >> original cgroup. Of course task could provide hit: O_NO_MOVEMEM or >> even while memory cgroup where it runs could be marked as "scanner" >> which shouldn't disturb memory classification. > > Ditto for annotating each file individually. Let's please try to stay > away from things like that. That's mostly a cop-out which is unlikely > to actually benefit the majority of users. Process which scans all files once isn't so rare use case. Linux still cannot handle this pattern sometimes. > >> I've missed obvious solution for controlling memory cgroup for files: >> project id. This persistent integer id stored in file system. For now >> it's implemented only for xfs and used for quota which is orthogonal >> to user/group quotas. We could map some of project id to memory cgroup. >> That is more flexible than per-superblock mark, has no conflicts like >> mark on bind-mount. > > Again, hell, no. > > Thanks. > -- Konstantin
WARNING: multiple messages have this Message-ID (diff)
From: Konstantin Khlebnikov <khlebnikov@yandex-team.ru> To: Tejun Heo <tj@kernel.org> Cc: Greg Thelen <gthelen@google.com>, Johannes Weiner <hannes@cmpxchg.org>, Michal Hocko <mhocko@suse.cz>, Cgroups <cgroups@vger.kernel.org>, "linux-mm@kvack.org" <linux-mm@kvack.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Jan Kara <jack@suse.cz>, Dave Chinner <david@fromorbit.com>, Jens Axboe <axboe@kernel.dk>, Christoph Hellwig <hch@infradead.org>, Li Zefan <lizefan@huawei.com>, Hugh Dickins <hughd@google.com>, Roman Gushchin <klamm@yandex-team.ru> Subject: Re: [RFC] Making memcg track ownership per address_space or anon_vma Date: Wed, 04 Feb 2015 20:58:21 +0300 [thread overview] Message-ID: <54D25DBD.5080009@yandex-team.ru> (raw) In-Reply-To: <20150204171512.GB18858@htj.dyndns.org> On 04.02.2015 20:15, Tejun Heo wrote: > Hello, > > On Wed, Feb 04, 2015 at 01:49:08PM +0300, Konstantin Khlebnikov wrote: >> I think important shared data must be handled and protected explicitly. >> That 'catch-all' shared container could be separated into several > > I kinda disagree. That'd be a major pain in the ass to use and you > wouldn't know when you got something wrong unless it actually goes > wrong and you know enough about the innerworkings to look for that. > Doesn't sound like a sound design to me. > >> memory cgroups depending on importance of files: glibc protected >> with soft guarantee, less important stuff is placed into another >> cgroup and cannot push top-priority libraries out of ram. > > That sounds extremely painful. I mean this thing _could_ be controlled more precisely. Even if default policy works for 99% users manual override is still required for 1% or if something goes wrong. > >> If shared files are free for use then that 'shared' container must be >> ready to keep them in memory. Otherwise this need to be fixed at the >> container side: we could ignore mlock for shared inodes or amount of >> such vmas might be limited in per-container basis. >> >> But sharing responsibility for shared file is vague concept: memory >> usage and limit of container must depends only on its own behavior not >> on neighbors at the same machine. >> >> >> Generally incidental sharing could be handled as temporary sharing: >> default policy (if inode isn't pinned to memory cgroup) after some >> time should detect that inode is no longer shared and migrate it into >> original cgroup. Of course task could provide hit: O_NO_MOVEMEM or >> even while memory cgroup where it runs could be marked as "scanner" >> which shouldn't disturb memory classification. > > Ditto for annotating each file individually. Let's please try to stay > away from things like that. That's mostly a cop-out which is unlikely > to actually benefit the majority of users. Process which scans all files once isn't so rare use case. Linux still cannot handle this pattern sometimes. > >> I've missed obvious solution for controlling memory cgroup for files: >> project id. This persistent integer id stored in file system. For now >> it's implemented only for xfs and used for quota which is orthogonal >> to user/group quotas. We could map some of project id to memory cgroup. >> That is more flexible than per-superblock mark, has no conflicts like >> mark on bind-mount. > > Again, hell, no. > > Thanks. > -- Konstantin -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2015-02-04 17:58 UTC|newest] Thread overview: 74+ messages / expand[flat|nested] mbox.gz Atom feed top 2015-01-30 4:43 [RFC] Making memcg track ownership per address_space or anon_vma Tejun Heo 2015-01-30 4:43 ` Tejun Heo 2015-01-30 5:55 ` Greg Thelen 2015-01-30 5:55 ` Greg Thelen 2015-01-30 6:27 ` Tejun Heo 2015-01-30 6:27 ` Tejun Heo 2015-01-30 16:07 ` Tejun Heo 2015-01-30 16:07 ` Tejun Heo 2015-01-30 16:07 ` Tejun Heo 2015-02-02 19:26 ` Konstantin Khlebnikov 2015-02-02 19:26 ` Konstantin Khlebnikov 2015-02-02 19:46 ` Tejun Heo 2015-02-02 19:46 ` Tejun Heo 2015-02-03 23:30 ` Greg Thelen 2015-02-03 23:30 ` Greg Thelen 2015-02-04 10:49 ` Konstantin Khlebnikov 2015-02-04 10:49 ` Konstantin Khlebnikov 2015-02-04 17:15 ` Tejun Heo 2015-02-04 17:15 ` Tejun Heo 2015-02-04 17:58 ` Konstantin Khlebnikov [this message] 2015-02-04 17:58 ` Konstantin Khlebnikov 2015-02-04 18:28 ` Tejun Heo 2015-02-04 18:28 ` Tejun Heo 2015-02-04 18:28 ` Tejun Heo 2015-02-04 17:06 ` Tejun Heo 2015-02-04 17:06 ` Tejun Heo 2015-02-04 23:51 ` Greg Thelen 2015-02-04 23:51 ` Greg Thelen 2015-02-04 23:51 ` Greg Thelen 2015-02-05 13:15 ` Tejun Heo 2015-02-05 13:15 ` Tejun Heo 2015-02-05 22:05 ` Greg Thelen 2015-02-05 22:05 ` Greg Thelen 2015-02-05 22:25 ` Tejun Heo 2015-02-05 22:25 ` Tejun Heo 2015-02-05 22:25 ` Tejun Heo 2015-02-06 0:03 ` Greg Thelen 2015-02-06 0:03 ` Greg Thelen 2015-02-06 14:17 ` Tejun Heo 2015-02-06 14:17 ` Tejun Heo 2015-02-06 23:43 ` Greg Thelen 2015-02-06 23:43 ` Greg Thelen 2015-02-07 14:38 ` Tejun Heo 2015-02-07 14:38 ` Tejun Heo 2015-02-07 14:38 ` Tejun Heo 2015-02-11 2:19 ` Tejun Heo 2015-02-11 2:19 ` Tejun Heo 2015-02-11 2:19 ` Tejun Heo 2015-02-11 7:32 ` Jan Kara 2015-02-11 7:32 ` Jan Kara 2015-02-11 7:32 ` Jan Kara 2015-02-11 18:28 ` Greg Thelen 2015-02-11 18:28 ` Greg Thelen 2015-02-11 18:28 ` Greg Thelen 2015-02-11 20:33 ` Tejun Heo 2015-02-11 20:33 ` Tejun Heo 2015-02-11 21:22 ` Konstantin Khlebnikov 2015-02-11 21:22 ` Konstantin Khlebnikov 2015-02-11 21:22 ` Konstantin Khlebnikov 2015-02-11 21:46 ` Tejun Heo 2015-02-11 21:46 ` Tejun Heo 2015-02-11 21:57 ` Konstantin Khlebnikov 2015-02-11 21:57 ` Konstantin Khlebnikov 2015-02-11 21:57 ` Konstantin Khlebnikov 2015-02-11 22:05 ` Tejun Heo 2015-02-11 22:05 ` Tejun Heo 2015-02-11 22:05 ` Tejun Heo 2015-02-11 22:15 ` Konstantin Khlebnikov 2015-02-11 22:15 ` Konstantin Khlebnikov 2015-02-11 22:15 ` Konstantin Khlebnikov 2015-02-11 22:30 ` Tejun Heo 2015-02-11 22:30 ` Tejun Heo 2015-02-12 2:10 ` Greg Thelen 2015-02-12 2:10 ` Greg Thelen
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=54D25DBD.5080009@yandex-team.ru \ --to=khlebnikov@yandex-team.ru \ --cc=axboe@kernel.dk \ --cc=cgroups@vger.kernel.org \ --cc=david@fromorbit.com \ --cc=gthelen@google.com \ --cc=hannes@cmpxchg.org \ --cc=hch@infradead.org \ --cc=hughd@google.com \ --cc=jack@suse.cz \ --cc=klamm@yandex-team.ru \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=lizefan@huawei.com \ --cc=mhocko@suse.cz \ --cc=tj@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.