From: Michal Hocko <mhocko@kernel.org> To: Johannes Weiner <hannes@cmpxchg.org> Cc: David Miller <davem@davemloft.net>, akpm@linux-foundation.org, vdavydov@virtuozzo.com, tj@kernel.org, netdev@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified hierarchy Date: Thu, 29 Oct 2015 16:25:46 +0100 [thread overview] Message-ID: <20151029152546.GG23598@dhcp22.suse.cz> (raw) In-Reply-To: <20151027164227.GB7749@cmpxchg.org> On Tue 27-10-15 09:42:27, Johannes Weiner wrote: > On Tue, Oct 27, 2015 at 05:15:54PM +0100, Michal Hocko wrote: > > On Tue 27-10-15 11:41:38, Johannes Weiner wrote: [...] > Or it could be exactly the other way around when you have a workload > that is heavy on filesystem metadata. I don't see why any scenario > would be more important than the other. Yes I definitely agree. No scenario is more important. We can only come up with a default that makes more sense for the majority and allow the minority to override. That was what I wanted to say basically. > I'm not saying that distinguishing between consumers is wrong, just > that "user memory vs kernel memory" is a false classification. Why do > you call page cache user memory but dentry cache kernel memory? It > doesn't make any sense. We are not talking about dcache vs. page cache alone here, though. We are talking about _all_ slab allocations vs. only user accessed memory. The slab consumption is directly under kernel control. A great pile of this logic is completly hidden from userspace. While user can estimate the user memory it is hard (if possible) to do that for the kernel memory footprint - not even mentioning this is variable and dependent on the particular kernel version. > > Also kmem accounting will make the load more non-deterministic because > > many of the resources are shared between tasks in separate cgroups > > unless they are explicitly configured. E.g. [id]cache will be shared > > and first to touch gets charged so you would end up with more false > > sharing. > > Exactly like page cache. This differentiation isn't based on reality. Yes false sharing is an existing and long term problem already. I just wanted to point out that the false sharing would be even a bigger problem because some kernel tracked resources are shared more naturally than file sharing. > > > IMO that's an implementation detail and a historical artifact that > > > should not be exposed to the user. And that's the thing I hate about > > > the current opt-out knob. > > You carefully skipped over this part. We can ignore it for socket > memory but it's something we need to figure out when it comes to slab > accounting and tracking. I am sorry, I didn't mean to skip this part, I though it would be clear from the previous text. I think kmem accounting falls into the same category. Have a sane default and a global boottime knob to override it for those that think differently - for whatever reason they might have. [...] > Having page cache accounting built in while presenting dentry+inode > cache as a configurable extension is completely random and doesn't > make sense. They are both first class memory consumers. They're not > separate categories. One isn't more "core" than the other. Again we are talking about all slab allocations not just the dcache. > > > For now, something like this as a boot commandline? > > > > > > cgroup.memory=nosocket > > > > That would work for me. > > Okay, then I'll go that route for the socket stuff. Thanks! -- Michal Hocko SUSE Labs
WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko@kernel.org> To: Johannes Weiner <hannes@cmpxchg.org> Cc: David Miller <davem@davemloft.net>, akpm@linux-foundation.org, vdavydov@virtuozzo.com, tj@kernel.org, netdev@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified hierarchy Date: Thu, 29 Oct 2015 16:25:46 +0100 [thread overview] Message-ID: <20151029152546.GG23598@dhcp22.suse.cz> (raw) In-Reply-To: <20151027164227.GB7749@cmpxchg.org> On Tue 27-10-15 09:42:27, Johannes Weiner wrote: > On Tue, Oct 27, 2015 at 05:15:54PM +0100, Michal Hocko wrote: > > On Tue 27-10-15 11:41:38, Johannes Weiner wrote: [...] > Or it could be exactly the other way around when you have a workload > that is heavy on filesystem metadata. I don't see why any scenario > would be more important than the other. Yes I definitely agree. No scenario is more important. We can only come up with a default that makes more sense for the majority and allow the minority to override. That was what I wanted to say basically. > I'm not saying that distinguishing between consumers is wrong, just > that "user memory vs kernel memory" is a false classification. Why do > you call page cache user memory but dentry cache kernel memory? It > doesn't make any sense. We are not talking about dcache vs. page cache alone here, though. We are talking about _all_ slab allocations vs. only user accessed memory. The slab consumption is directly under kernel control. A great pile of this logic is completly hidden from userspace. While user can estimate the user memory it is hard (if possible) to do that for the kernel memory footprint - not even mentioning this is variable and dependent on the particular kernel version. > > Also kmem accounting will make the load more non-deterministic because > > many of the resources are shared between tasks in separate cgroups > > unless they are explicitly configured. E.g. [id]cache will be shared > > and first to touch gets charged so you would end up with more false > > sharing. > > Exactly like page cache. This differentiation isn't based on reality. Yes false sharing is an existing and long term problem already. I just wanted to point out that the false sharing would be even a bigger problem because some kernel tracked resources are shared more naturally than file sharing. > > > IMO that's an implementation detail and a historical artifact that > > > should not be exposed to the user. And that's the thing I hate about > > > the current opt-out knob. > > You carefully skipped over this part. We can ignore it for socket > memory but it's something we need to figure out when it comes to slab > accounting and tracking. I am sorry, I didn't mean to skip this part, I though it would be clear from the previous text. I think kmem accounting falls into the same category. Have a sane default and a global boottime knob to override it for those that think differently - for whatever reason they might have. [...] > Having page cache accounting built in while presenting dentry+inode > cache as a configurable extension is completely random and doesn't > make sense. They are both first class memory consumers. They're not > separate categories. One isn't more "core" than the other. Again we are talking about all slab allocations not just the dcache. > > > For now, something like this as a boot commandline? > > > > > > cgroup.memory=nosocket > > > > That would work for me. > > Okay, then I'll go that route for the socket stuff. Thanks! -- Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2015-10-29 15:25 UTC|newest] Thread overview: 156+ messages / expand[flat|nested] mbox.gz Atom feed top 2015-10-22 4:21 [PATCH 0/8] mm: memcontrol: account socket memory in unified hierarchy Johannes Weiner 2015-10-22 4:21 ` Johannes Weiner 2015-10-22 4:21 ` [PATCH 1/8] mm: page_counter: let page_counter_try_charge() return bool Johannes Weiner 2015-10-22 4:21 ` Johannes Weiner 2015-10-23 11:31 ` Michal Hocko 2015-10-23 11:31 ` Michal Hocko 2015-10-22 4:21 ` [PATCH 2/8] mm: memcontrol: export root_mem_cgroup Johannes Weiner 2015-10-22 4:21 ` Johannes Weiner 2015-10-23 11:32 ` Michal Hocko 2015-10-23 11:32 ` Michal Hocko 2015-10-22 4:21 ` [PATCH 3/8] net: consolidate memcg socket buffer tracking and accounting Johannes Weiner 2015-10-22 4:21 ` Johannes Weiner 2015-10-22 18:46 ` Vladimir Davydov 2015-10-22 18:46 ` Vladimir Davydov 2015-10-22 18:46 ` Vladimir Davydov 2015-10-22 19:09 ` Johannes Weiner 2015-10-22 19:09 ` Johannes Weiner 2015-10-23 13:42 ` Vladimir Davydov 2015-10-23 13:42 ` Vladimir Davydov 2015-10-23 13:42 ` Vladimir Davydov 2015-10-23 12:38 ` Michal Hocko 2015-10-23 12:38 ` Michal Hocko 2015-10-22 4:21 ` [PATCH 4/8] mm: memcontrol: prepare for unified hierarchy socket accounting Johannes Weiner 2015-10-22 4:21 ` Johannes Weiner 2015-10-23 12:39 ` Michal Hocko 2015-10-23 12:39 ` Michal Hocko 2015-10-22 4:21 ` [PATCH 5/8] mm: memcontrol: account socket memory on unified hierarchy Johannes Weiner 2015-10-22 4:21 ` Johannes Weiner 2015-10-22 18:47 ` Vladimir Davydov 2015-10-22 18:47 ` Vladimir Davydov 2015-10-22 18:47 ` Vladimir Davydov 2015-10-23 13:19 ` Michal Hocko 2015-10-23 13:19 ` Michal Hocko 2015-10-23 13:19 ` Michal Hocko 2015-10-23 13:59 ` David Miller 2015-10-23 13:59 ` David Miller 2015-10-23 13:59 ` David Miller 2015-10-26 16:56 ` Johannes Weiner 2015-10-26 16:56 ` Johannes Weiner 2015-10-27 12:26 ` Michal Hocko 2015-10-27 12:26 ` Michal Hocko 2015-10-27 13:49 ` David Miller 2015-10-27 13:49 ` David Miller 2015-10-27 13:49 ` David Miller 2015-10-27 15:41 ` Johannes Weiner 2015-10-27 15:41 ` Johannes Weiner 2015-10-27 15:41 ` Johannes Weiner 2015-10-27 16:15 ` Michal Hocko 2015-10-27 16:15 ` Michal Hocko 2015-10-27 16:42 ` Johannes Weiner 2015-10-27 16:42 ` Johannes Weiner 2015-10-28 0:45 ` David Miller 2015-10-28 0:45 ` David Miller 2015-10-28 0:45 ` David Miller 2015-10-28 3:05 ` Johannes Weiner 2015-10-28 3:05 ` Johannes Weiner 2015-10-29 15:25 ` Michal Hocko [this message] 2015-10-29 15:25 ` Michal Hocko 2015-10-29 16:10 ` Johannes Weiner 2015-10-29 16:10 ` Johannes Weiner 2015-10-29 16:10 ` Johannes Weiner 2015-11-04 10:42 ` Michal Hocko 2015-11-04 10:42 ` Michal Hocko 2015-11-04 19:50 ` Johannes Weiner 2015-11-04 19:50 ` Johannes Weiner 2015-11-04 19:50 ` Johannes Weiner 2015-11-05 14:40 ` Michal Hocko 2015-11-05 14:40 ` Michal Hocko 2015-11-05 16:16 ` David Miller 2015-11-05 16:16 ` David Miller 2015-11-05 16:28 ` Michal Hocko 2015-11-05 16:28 ` Michal Hocko 2015-11-05 16:28 ` Michal Hocko 2015-11-05 16:30 ` David Miller 2015-11-05 16:30 ` David Miller 2015-11-05 22:32 ` Johannes Weiner 2015-11-05 22:32 ` Johannes Weiner 2015-11-05 22:32 ` Johannes Weiner 2015-11-06 12:51 ` Michal Hocko 2015-11-06 12:51 ` Michal Hocko 2015-11-05 20:55 ` Johannes Weiner 2015-11-05 20:55 ` Johannes Weiner 2015-11-05 22:52 ` Johannes Weiner 2015-11-05 22:52 ` Johannes Weiner 2015-11-05 22:52 ` Johannes Weiner 2015-11-06 10:57 ` Michal Hocko 2015-11-06 10:57 ` Michal Hocko 2015-11-06 16:19 ` Johannes Weiner 2015-11-06 16:19 ` Johannes Weiner 2015-11-06 16:46 ` Michal Hocko 2015-11-06 16:46 ` Michal Hocko 2015-11-06 16:46 ` Michal Hocko 2015-11-06 17:45 ` Johannes Weiner 2015-11-06 17:45 ` Johannes Weiner 2015-11-06 17:45 ` Johannes Weiner 2015-11-07 3:45 ` David Miller 2015-11-07 3:45 ` David Miller 2015-11-12 18:36 ` Mel Gorman 2015-11-12 18:36 ` Mel Gorman 2015-11-12 18:36 ` Mel Gorman 2015-11-12 19:12 ` Johannes Weiner 2015-11-12 19:12 ` Johannes Weiner 2015-11-06 9:05 ` Vladimir Davydov 2015-11-06 9:05 ` Vladimir Davydov 2015-11-06 9:05 ` Vladimir Davydov 2015-11-06 9:05 ` Vladimir Davydov 2015-11-06 13:29 ` Michal Hocko 2015-11-06 13:29 ` Michal Hocko 2015-11-06 16:35 ` Johannes Weiner 2015-11-06 16:35 ` Johannes Weiner 2015-11-06 13:21 ` Michal Hocko 2015-11-06 13:21 ` Michal Hocko 2015-10-22 4:21 ` [PATCH 6/8] mm: vmscan: simplify memcg vs. global shrinker invocation Johannes Weiner 2015-10-22 4:21 ` Johannes Weiner 2015-10-23 13:26 ` Michal Hocko 2015-10-23 13:26 ` Michal Hocko 2015-10-22 4:21 ` [PATCH 7/8] mm: vmscan: report vmpressure at the level of reclaim activity Johannes Weiner 2015-10-22 4:21 ` Johannes Weiner 2015-10-22 18:48 ` Vladimir Davydov 2015-10-22 18:48 ` Vladimir Davydov 2015-10-22 18:48 ` Vladimir Davydov 2015-10-22 18:48 ` Vladimir Davydov 2015-10-23 13:49 ` Michal Hocko 2015-10-23 13:49 ` Michal Hocko 2015-10-23 13:49 ` Michal Hocko 2015-10-22 4:21 ` [PATCH 8/8] mm: memcontrol: hook up vmpressure to socket pressure Johannes Weiner 2015-10-22 4:21 ` Johannes Weiner 2015-10-22 18:57 ` Vladimir Davydov 2015-10-22 18:57 ` Vladimir Davydov 2015-10-22 18:57 ` Vladimir Davydov 2015-10-22 18:45 ` [PATCH 0/8] mm: memcontrol: account socket memory in unified hierarchy Vladimir Davydov 2015-10-22 18:45 ` Vladimir Davydov 2015-10-22 18:45 ` Vladimir Davydov 2015-10-26 17:22 ` Johannes Weiner 2015-10-26 17:22 ` Johannes Weiner 2015-10-26 17:22 ` Johannes Weiner 2015-10-26 17:22 ` Johannes Weiner 2015-10-27 8:43 ` Vladimir Davydov 2015-10-27 8:43 ` Vladimir Davydov 2015-10-27 8:43 ` Vladimir Davydov 2015-10-27 16:01 ` Johannes Weiner 2015-10-27 16:01 ` Johannes Weiner 2015-10-28 8:20 ` Vladimir Davydov 2015-10-28 8:20 ` Vladimir Davydov 2015-10-28 8:20 ` Vladimir Davydov 2015-10-28 18:58 ` Johannes Weiner 2015-10-28 18:58 ` Johannes Weiner 2015-10-29 9:27 ` Vladimir Davydov 2015-10-29 9:27 ` Vladimir Davydov 2015-10-29 9:27 ` Vladimir Davydov 2015-10-29 17:52 ` Johannes Weiner 2015-10-29 17:52 ` Johannes Weiner 2015-10-29 17:52 ` Johannes Weiner 2015-11-02 14:47 ` Vladimir Davydov 2015-11-02 14:47 ` Vladimir Davydov 2015-11-02 14:47 ` Vladimir Davydov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20151029152546.GG23598@dhcp22.suse.cz \ --to=mhocko@kernel.org \ --cc=akpm@linux-foundation.org \ --cc=cgroups@vger.kernel.org \ --cc=davem@davemloft.net \ --cc=hannes@cmpxchg.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=netdev@vger.kernel.org \ --cc=tj@kernel.org \ --cc=vdavydov@virtuozzo.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.