From: Shakeel Butt <shakeelb@google.com> To: Reinette Chatre <reinette.chatre@intel.com>, Fenghua Yu <fenghua.yu@intel.com>, Borislav Petkov <bp@alien8.de>, LKML <linux-kernel@vger.kernel.org>, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, x86@kernel.org Subject: [bug report] resctrl high memory comsumption Date: Wed, 8 Jan 2020 09:07:41 -0800 Message-ID: <CALvZod7E9zzHwenzf7objzGKsdBmVwTgEJ0nPgs0LUFU3SN5Pw@mail.gmail.com> (raw) Hi, Recently we had a bug in the system software writing the same pids to the tasks file of resctrl group multiple times. The resctrl code allocates "struct task_move_callback" for each such write and call task_work_add() for that task to handle it on return to user-space without checking if such request already exist for that particular task. The issue arises for long sleeping tasks which has thousands for such request queued to be handled. On our production, we notice thousands of tasks having thousands of such requests and taking GiBs of memory for "struct task_move_callback". I am not very familiar with the code to judge if task_work_cancel() is the right approach or just checking closid/rmid before doing task_work_add(). ==repro== # mkdir /sys/fs/resctrl/test # cat /proc/slabinfo | grep kmalloc-32 kmalloc-32 57219 57288 32 124 1 : tunables 120 60 8 : slabdata 462 462 0 # sleep 600& [1] 17611 # for i in {1..200000}; do echo 17611 > /sys/fs/resctrl/test/tasks ; done # cat /proc/slabinfo | grep kmalloc-32 kmalloc-32 257466 257548 32 124 1 : tunables 120 60 8 : slabdata 2077 2077 5 # kill 17611 [1]+ Terminated sleep 600 # cat /proc/slabinfo | grep kmalloc-32 kmalloc-32 57924 60636 32 124 1 : tunables 120 60 8 : slabdata 470 489 385 thanks, Shakeel
next reply index Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-01-08 17:07 Shakeel Butt [this message] 2020-01-08 20:23 ` Fenghua Yu 2020-01-08 20:42 ` Reinette Chatre 2020-01-08 21:42 ` Fenghua Yu 2020-01-08 21:54 ` Reinette Chatre 2020-01-13 18:38 ` Shakeel Butt 2020-01-08 21:20 ` Shakeel Butt
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=CALvZod7E9zzHwenzf7objzGKsdBmVwTgEJ0nPgs0LUFU3SN5Pw@mail.gmail.com \ --to=shakeelb@google.com \ --cc=bp@alien8.de \ --cc=fenghua.yu@intel.com \ --cc=linux-kernel@vger.kernel.org \ --cc=mingo@redhat.com \ --cc=reinette.chatre@intel.com \ --cc=tglx@linutronix.de \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
LKML Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \ linux-kernel@vger.kernel.org public-inbox-index lkml Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel AGPL code for this site: git clone https://public-inbox.org/public-inbox.git