From: Michal Hocko <mhocko@suse.com> To: Yosry Ahmed <yosryahmed@google.com> Cc: Wei Xu <weixugc@google.com>, Andrew Morton <akpm@linux-foundation.org>, Johannes Weiner <hannes@cmpxchg.org>, Shakeel Butt <shakeelb@google.com>, David Rientjes <rientjes@google.com>, Tejun Heo <tj@kernel.org>, Zefan Li <lizefan.x@bytedance.com>, Roman Gushchin <roman.gushchin@linux.dev>, cgroups@vger.kernel.org, linux-doc@vger.kernel.org, Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, Linux MM <linux-mm@kvack.org>, Jonathan Corbet <corbet@lwn.net>, Yu Zhao <yuzhao@google.com>, Dave Hansen <dave.hansen@linux.intel.com>, Greg Thelen <gthelen@google.com> Subject: Re: [PATCH resend] memcg: introduce per-memcg reclaim interface Date: Fri, 1 Apr 2022 15:03:09 +0200 [thread overview] Message-ID: <Ykb4DVt0eKrstDWv@dhcp22.suse.cz> (raw) In-Reply-To: <CAJD7tkarUYM6fjty8FXn9gtR=RVWgksa3LNwswi1Ug1LPnLt4g@mail.gmail.com> On Fri 01-04-22 02:17:28, Yosry Ahmed wrote: > On Thu, Mar 31, 2022 at 8:38 PM Wei Xu <weixugc@google.com> wrote: > > > > On Thu, Mar 31, 2022 at 5:33 PM Andrew Morton <akpm@linux-foundation.org> wrote: > > > > > > On Thu, 31 Mar 2022 08:41:51 +0000 Yosry Ahmed <yosryahmed@google.com> wrote: > > > > > > > --- a/mm/memcontrol.c > > > > +++ b/mm/memcontrol.c > > > > @@ -6355,6 +6355,38 @@ static ssize_t memory_oom_group_write(struct kernfs_open_file *of, > > > > return nbytes; > > > > } > > > > > > > > +static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, > > > > + size_t nbytes, loff_t off) > > > > +{ > > > > + struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); > > > > + unsigned int nr_retries = MAX_RECLAIM_RETRIES; > > > > + unsigned long nr_to_reclaim, nr_reclaimed = 0; > > > > + int err; > > > > + > > > > + buf = strstrip(buf); > > > > + err = page_counter_memparse(buf, "", &nr_to_reclaim); > > > > + if (err) > > > > + return err; > > > > + > > > > + while (nr_reclaimed < nr_to_reclaim) { > > > > + unsigned long reclaimed; > > > > + > > > > + if (signal_pending(current)) > > > > + break; > > > > + > > > > + reclaimed = try_to_free_mem_cgroup_pages(memcg, > > > > + nr_to_reclaim - nr_reclaimed, > > > > + GFP_KERNEL, true); > > > > + > > > > + if (!reclaimed && !nr_retries--) > > > > + break; > > > > + > > > > + nr_reclaimed += reclaimed; > > > > + } > > > > > > Is there any way in which this can be provoked into triggering the > > > softlockup detector? > > > > memory.reclaim is similar to memory.high w.r.t. reclaiming memory, > > except that memory.reclaim is stateless, while the kernel remembers > > the state set by memory.high. So memory.reclaim should not bring in > > any new risks of triggering soft lockup, if any. Memory reclaim already has cond_resched even if there is nothing reclaimable. See shrink_node_memcgs > > > Is it optimal to do the MAX_RECLAIM_RETRIES loop in the kernel? > > > Would additional flexibility be gained by letting userspace handle > > > retrying? > > > > I agree it is better to retry from the userspace. > > Thanks Andrew and Wei for looking at this. IIUC the > MAX_RECLAIM_RETRIES loop was modeled after the loop in memory.high as > well. Is there a reason why it should be different here? No, I would go with the same approach other interfaces use. I am not a great fan of MAX_RECLAIM_RETRIES - especially when we have a bail out on signals - but if we are to change this then let's do it consisently. -- Michal Hocko SUSE Labs
WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko-IBi9RG/b67k@public.gmane.org> To: Yosry Ahmed <yosryahmed-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> Cc: Wei Xu <weixugc-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>, Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>, Shakeel Butt <shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, David Rientjes <rientjes-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Tejun Heo <tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Zefan Li <lizefan.x-EC8Uxl6Npydl57MIdRCFDg@public.gmane.org>, Roman Gushchin <roman.gushchin-fxUVXftIFDnyG1zEObXtfA@public.gmane.org>, cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-doc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Linux Kernel Mailing List <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, Linux MM <linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org>, Jonathan Corbet <corbet-T1hC0tSOHrs@public.gmane.org>, Yu Zhao <yuzhao-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Dave Hansen <dave.hansen-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>, Greg Thelen <gthelen-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> Subject: Re: [PATCH resend] memcg: introduce per-memcg reclaim interface Date: Fri, 1 Apr 2022 15:03:09 +0200 [thread overview] Message-ID: <Ykb4DVt0eKrstDWv@dhcp22.suse.cz> (raw) In-Reply-To: <CAJD7tkarUYM6fjty8FXn9gtR=RVWgksa3LNwswi1Ug1LPnLt4g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> On Fri 01-04-22 02:17:28, Yosry Ahmed wrote: > On Thu, Mar 31, 2022 at 8:38 PM Wei Xu <weixugc-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote: > > > > On Thu, Mar 31, 2022 at 5:33 PM Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org> wrote: > > > > > > On Thu, 31 Mar 2022 08:41:51 +0000 Yosry Ahmed <yosryahmed-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote: > > > > > > > --- a/mm/memcontrol.c > > > > +++ b/mm/memcontrol.c > > > > @@ -6355,6 +6355,38 @@ static ssize_t memory_oom_group_write(struct kernfs_open_file *of, > > > > return nbytes; > > > > } > > > > > > > > +static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, > > > > + size_t nbytes, loff_t off) > > > > +{ > > > > + struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); > > > > + unsigned int nr_retries = MAX_RECLAIM_RETRIES; > > > > + unsigned long nr_to_reclaim, nr_reclaimed = 0; > > > > + int err; > > > > + > > > > + buf = strstrip(buf); > > > > + err = page_counter_memparse(buf, "", &nr_to_reclaim); > > > > + if (err) > > > > + return err; > > > > + > > > > + while (nr_reclaimed < nr_to_reclaim) { > > > > + unsigned long reclaimed; > > > > + > > > > + if (signal_pending(current)) > > > > + break; > > > > + > > > > + reclaimed = try_to_free_mem_cgroup_pages(memcg, > > > > + nr_to_reclaim - nr_reclaimed, > > > > + GFP_KERNEL, true); > > > > + > > > > + if (!reclaimed && !nr_retries--) > > > > + break; > > > > + > > > > + nr_reclaimed += reclaimed; > > > > + } > > > > > > Is there any way in which this can be provoked into triggering the > > > softlockup detector? > > > > memory.reclaim is similar to memory.high w.r.t. reclaiming memory, > > except that memory.reclaim is stateless, while the kernel remembers > > the state set by memory.high. So memory.reclaim should not bring in > > any new risks of triggering soft lockup, if any. Memory reclaim already has cond_resched even if there is nothing reclaimable. See shrink_node_memcgs > > > Is it optimal to do the MAX_RECLAIM_RETRIES loop in the kernel? > > > Would additional flexibility be gained by letting userspace handle > > > retrying? > > > > I agree it is better to retry from the userspace. > > Thanks Andrew and Wei for looking at this. IIUC the > MAX_RECLAIM_RETRIES loop was modeled after the loop in memory.high as > well. Is there a reason why it should be different here? No, I would go with the same approach other interfaces use. I am not a great fan of MAX_RECLAIM_RETRIES - especially when we have a bail out on signals - but if we are to change this then let's do it consisently. -- Michal Hocko SUSE Labs
next prev parent reply other threads:[~2022-04-01 13:03 UTC|newest] Thread overview: 81+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-03-31 8:41 [PATCH resend] memcg: introduce per-memcg reclaim interface Yosry Ahmed 2022-03-31 17:25 ` Roman Gushchin 2022-04-01 6:01 ` Wei Xu 2022-04-01 6:01 ` Wei Xu 2022-04-01 9:11 ` Yosry Ahmed 2022-04-01 9:11 ` Yosry Ahmed 2022-04-01 18:39 ` Roman Gushchin 2022-04-01 21:13 ` Johannes Weiner 2022-04-01 21:13 ` Johannes Weiner 2022-04-01 21:21 ` Roman Gushchin 2022-04-01 21:38 ` Wei Xu 2022-04-01 21:51 ` Johannes Weiner 2022-04-04 17:14 ` Shakeel Butt 2022-04-04 17:13 ` Yosry Ahmed 2022-04-04 17:55 ` Roman Gushchin 2022-04-01 9:15 ` Yosry Ahmed 2022-04-01 9:15 ` Yosry Ahmed 2022-04-01 15:41 ` Shakeel Butt 2022-04-01 13:49 ` Michal Hocko 2022-04-01 16:58 ` Roman Gushchin 2022-04-04 8:44 ` Michal Hocko 2022-04-04 18:25 ` Roman Gushchin 2022-03-31 19:25 ` Johannes Weiner 2022-04-01 0:33 ` Andrew Morton 2022-04-01 0:33 ` Andrew Morton 2022-04-01 3:38 ` Wei Xu 2022-04-01 9:17 ` Yosry Ahmed 2022-04-01 9:17 ` Yosry Ahmed 2022-04-01 13:03 ` Michal Hocko [this message] 2022-04-01 13:03 ` Michal Hocko 2022-04-01 3:05 ` Chen Wandun 2022-04-01 3:05 ` Chen Wandun 2022-04-01 9:20 ` Yosry Ahmed 2022-04-01 9:48 ` Chen Wandun 2022-04-01 9:48 ` Chen Wandun 2022-04-01 10:02 ` Yosry Ahmed 2022-04-01 4:05 ` Wei Xu 2022-04-01 4:05 ` Wei Xu 2022-04-01 9:22 ` Yosry Ahmed 2022-04-01 9:22 ` Yosry Ahmed 2022-04-01 15:22 ` Johannes Weiner 2022-04-01 20:14 ` Wei Xu 2022-04-01 21:07 ` Johannes Weiner 2022-04-01 21:07 ` Johannes Weiner 2022-04-04 17:08 ` Shakeel Butt 2022-04-05 2:30 ` Wei Xu 2022-04-05 10:09 ` Michal Koutný 2022-04-01 8:39 ` Vaibhav Jain 2022-04-01 9:23 ` Yosry Ahmed 2022-04-04 3:50 ` Vaibhav Jain 2022-04-04 17:18 ` Yosry Ahmed 2022-04-01 13:54 ` Michal Hocko 2022-04-01 16:56 ` Wei Xu 2022-04-01 16:56 ` Wei Xu 2022-04-02 8:13 ` Huang, Ying 2022-04-03 6:46 ` Wei Xu 2022-04-03 6:56 ` Wei Xu 2022-04-06 0:48 ` Huang, Ying 2022-04-06 1:07 ` Wei Xu 2022-04-06 1:07 ` Wei Xu 2022-04-06 2:49 ` Huang, Ying 2022-04-06 2:49 ` Huang, Ying 2022-04-06 5:02 ` Wei Xu 2022-04-06 6:32 ` Huang, Ying 2022-04-06 7:05 ` Wei Xu 2022-04-06 8:49 ` Huang, Ying 2022-04-06 8:49 ` Huang, Ying 2022-04-06 20:16 ` Wei Xu 2022-04-06 20:16 ` Wei Xu 2022-04-07 7:35 ` Michal Hocko 2022-04-07 21:26 ` Tim Chen 2022-04-07 22:07 ` Wei Xu 2022-04-07 22:12 ` Wei Xu 2022-04-07 22:12 ` Wei Xu 2022-04-07 23:11 ` Tim Chen 2022-04-08 2:10 ` Wei Xu 2022-04-08 2:10 ` Wei Xu 2022-04-08 3:08 ` Huang, Ying 2022-04-08 4:10 ` Wei Xu 2022-04-08 4:10 ` Wei Xu 2022-04-04 17:09 ` Yosry Ahmed
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=Ykb4DVt0eKrstDWv@dhcp22.suse.cz \ --to=mhocko@suse.com \ --cc=akpm@linux-foundation.org \ --cc=cgroups@vger.kernel.org \ --cc=corbet@lwn.net \ --cc=dave.hansen@linux.intel.com \ --cc=gthelen@google.com \ --cc=hannes@cmpxchg.org \ --cc=linux-doc@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=lizefan.x@bytedance.com \ --cc=rientjes@google.com \ --cc=roman.gushchin@linux.dev \ --cc=shakeelb@google.com \ --cc=tj@kernel.org \ --cc=weixugc@google.com \ --cc=yosryahmed@google.com \ --cc=yuzhao@google.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.