From: "Arkadiusz Miśkiewicz" <a.miskiewicz@gmail.com> To: cgroups@vger.kernel.org Cc: Aleksa Sarai <asarai@suse.de>, Jay Kamat <jgkamat@fb.com>, Roman Gushchin <guro@fb.com>, Michal Hocko <mhocko@suse.com>, Johannes Weiner <hannes@cmpxchg.org>, linux-kernel@vger.kernel.org Subject: Re: pids.current with invalid value for hours [5.0.0 rc3 git] Date: Fri, 25 Jan 2019 08:52:11 +0100 Message-ID: <1cdbef13-564d-61a6-95f4-579d2cad243d@gmail.com> (raw) In-Reply-To: <96ef6615-a5df-30af-b4dc-417a18ca63f1@gmail.com> On 24/01/2019 12:21, Arkadiusz Miśkiewicz wrote: > On 17/01/2019 14:17, Arkadiusz Miśkiewicz wrote: >> On 17/01/2019 13:25, Aleksa Sarai wrote: >>> On 2019-01-17, Arkadiusz Miśkiewicz <a.miskiewicz@gmail.com> wrote: >>>> Using kernel 4.19.13. >>>> >>>> For one cgroup I noticed weird behaviour: >>>> >>>> # cat pids.current >>>> 60 >>>> # cat cgroup.procs >>>> # >>> >>> Are there any zombies in the cgroup? pids.current is linked up directly >>> to __put_task_struct (so exit(2) won't decrease it, only the task_struct >>> actually being freed will decrease it). >>> >> >> There are no zombie processes. >> >> In mean time the problem shows on multiple servers and so far saw it >> only in cgroups that were OOMed. >> >> What has changed on these servers (yesterday) is turning on >> memory.oom.group=1 for all cgroups and changing memory.high from 1G to >> "max" (leaving memory.max=2G limit only). >> >> Previously there was no such problem. >> > > I'm attaching reproducer. This time tried on different distribution > kernel (arch linux). > > After 60s pids.current still shows 37 processes even if there are no > processes running (according to ps aux). The same test on 5.0.0-rc3-00104-gc04e2a780caf and it's easy to reproduce bug. No processes in cgroup but pids.current reports 91. memory.oom.group=0 - everything works fine, pids are counted properly memory.oom.group=1 - bug becomes visible [root@xps test]# python3 cg.py Created cgroup: /sys/fs/cgroup/test_5277 Start: pids.current: 0 Start: cgroup.procs: 0: pids.current: 103 0: cgroup.procs: 1: pids.current: 91 1: cgroup.procs: 2: pids.current: 91 2: cgroup.procs: 3: pids.current: 91 3: cgroup.procs: 4: pids.current: 91 4: cgroup.procs: 5: pids.current: 91 5: cgroup.procs: 6: pids.current: 91 6: cgroup.procs: 7: pids.current: 91 7: cgroup.procs: 8: pids.current: 91 8: cgroup.procs: 9: pids.current: 91 9: cgroup.procs: 10: pids.current: 91 10: cgroup.procs: 11: pids.current: 91 11: cgroup.procs: [root@xps test]# uname -a Linux xps 5.0.0-rc3-00104-gc04e2a780caf #288 SMP PREEMPT Thu Jan 24 19:00:32 CET 2019 x86_64 Intel(R)_Core(TM)_i9-8950HK_CPU_@_2.90GHz PLD Linux cc relevant people script is here: https://www.spinics.net/lists/cgroups/msg21330.html > > [root@warm ~]# uname -a > Linux warm 4.20.3-arch1-1-ARCH #1 SMP PREEMPT Wed Jan 16 22:38:58 UTC > 2019 x86_64 GNU/Linux > [root@warm ~]# python3 cg.py > Created cgroup: /sys/fs/cgroup/test_26207 > Start: pids.current: 0 > Start: cgroup.procs: > 0: pids.current: 62 > 0: cgroup.procs: > 1: pids.current: 37 > 1: cgroup.procs: > 2: pids.current: 37 > 2: cgroup.procs: > 3: pids.current: 37 > 3: cgroup.procs: > 4: pids.current: 37 > 4: cgroup.procs: > 5: pids.current: 37 > 5: cgroup.procs: > 6: pids.current: 37 > 6: cgroup.procs: > 7: pids.current: 37 > 7: cgroup.procs: > 8: pids.current: 37 > 8: cgroup.procs: > 9: pids.current: 37 > 9: cgroup.procs: > 10: pids.current: 37 > 10: cgroup.procs: > 11: pids.current: 37 > 11: cgroup.procs: > -- Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )
next parent reply index Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top [not found] <df806a77-3327-9db5-8be2-976fde1c84e5@gmail.com> [not found] ` <20190117122535.njcbqhlmzozdkncw@mikami> [not found] ` <1d36b181-cbaf-6694-1a31-2f7f55d15675@gmail.com> [not found] ` <96ef6615-a5df-30af-b4dc-417a18ca63f1@gmail.com> 2019-01-25 7:52 ` Arkadiusz Miśkiewicz [this message] 2019-01-25 16:37 ` Tejun Heo 2019-01-25 19:47 ` Arkadiusz Miśkiewicz 2019-01-26 1:27 ` Tetsuo Handa 2019-01-26 2:41 ` Arkadiusz Miśkiewicz 2019-01-26 6:10 ` Tetsuo Handa 2019-01-26 7:55 ` Tetsuo Handa 2019-01-26 11:09 ` Tetsuo Handa 2019-01-26 11:29 ` Arkadiusz Miśkiewicz 2019-01-26 13:10 ` [PATCH v2] oom, oom_reaper: do not enqueue same task twice Tetsuo Handa 2019-01-27 8:37 ` Michal Hocko 2019-01-27 10:56 ` Tetsuo Handa 2019-01-27 11:40 ` Michal Hocko 2019-01-27 14:57 ` [PATCH v3] " Tetsuo Handa 2019-01-27 16:58 ` Michal Hocko 2019-01-27 23:00 ` Roman Gushchin 2019-01-28 18:15 ` Andrew Morton 2019-01-28 18:42 ` Michal Hocko 2019-01-28 21:53 ` Johannes Weiner 2019-01-29 10:34 ` Tetsuo Handa 2019-01-26 1:41 ` pids.current with invalid value for hours [5.0.0 rc3 git] Roman Gushchin 2019-01-26 2:28 ` Arkadiusz Miśkiewicz
Reply instructions: You may reply publically to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1cdbef13-564d-61a6-95f4-579d2cad243d@gmail.com \ --to=a.miskiewicz@gmail.com \ --cc=asarai@suse.de \ --cc=cgroups@vger.kernel.org \ --cc=guro@fb.com \ --cc=hannes@cmpxchg.org \ --cc=jgkamat@fb.com \ --cc=linux-kernel@vger.kernel.org \ --cc=mhocko@suse.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
LKML Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \ linux-kernel@vger.kernel.org linux-kernel@archiver.kernel.org public-inbox-index lkml Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel AGPL code for this site: git clone https://public-inbox.org/ public-inbox