From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753794AbdJSSxQ (ORCPT ); Thu, 19 Oct 2017 14:53:16 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:60606 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753656AbdJSSxK (ORCPT ); Thu, 19 Oct 2017 14:53:10 -0400 From: Roman Gushchin To: CC: Roman Gushchin , Michal Hocko , Vladimir Davydov , Johannes Weiner , Tetsuo Handa , David Rientjes , Andrew Morton , Tejun Heo , , , , Subject: [RESEND v12 0/6] cgroup-aware OOM killer Date: Thu, 19 Oct 2017 19:52:12 +0100 Message-ID: <20171019185218.12663-1-guro@fb.com> X-Mailer: git-send-email 2.13.6 MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [2620:10d:c092:200::1:8cb4] X-ClientProxiedBy: VI1PR0102CA0074.eurprd01.prod.exchangelabs.com (2603:10a6:803:15::15) To DM3PR15MB1081.namprd15.prod.outlook.com (2603:10b6:0:12::7) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2c69654c-e041-4311-84c8-08d5172298a0 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(2017030254172)(4534020)(4602075)(4627075)(201703031133081)(201702281549075)(2017052603199)(201703131423095);SRVR:DM3PR15MB1081; X-Microsoft-Exchange-Diagnostics: 1;DM3PR15MB1081;3:ltYOjz5qQHks9D/4tDbjADiXFbCRAlA75vxh0X/LzEXlkwSPQ83/15P/F86o8MYu4DfOFR1PimCIB8ky0wA7IE1WTjeJ6BxK1Nj63AW5umT+uqdnJiGrCW550kXgBw7laQuE516cEEcayQaKpZ8RQrflhbuAsXO8bTAttjhK96mDWyfzF6K4ogamDcvyk9GawljqW9/wBhJFKHfdBJT52OthkS+YOy/5JEJXJFmXgnxKEA0wH59M9r5f0gJzpLBV;25:B5hMu6XTxBhJASJWZIUQW2/VmcRb1SchsNtbQKp1xXlKlJ3oHAWqTXratUH/+lvTrH3feEQoUWVP1CnSeg5C2uERshXl7RWGHa7/WtAtkHdORigCwb6/w8mNnK2N+UQEoan6QTbFT/h/WG4n6kcouu+xdYArm9pRtv3s8BBMtvmVbqHKc/Z2aBmdTbyVX9OwKc3fydbVt0HMxKbELtYnYjUXO7FomQQSBFILcYYfel4f3E5nPLTxAn2eNseJ3IpznOesPi+7yl7XdtO2xYnr+0ZbzBRS3nqeilCtWwPDqqP6ZYXRVicXjkxsN8hTwoI1GhjW0a8SZ/sQ35ny3/KTEw==;31:/JmZ9s0s37mOwgLci2S9xMo8Xdt1JKaCGh6MLffw9N7fohGXnlnHnpmflCVX+dMyY4E0e9i9Mx+6sBfthqr/KcEV/Y1rrjelfY4esvcYWn6+I2zlEMJsMO/xODmv9+qZw26zHSok0O+EYlP2iWCYkr85tLhdONbbgVHL4/VbW0Am/aaAlnGpCtcX3upPj1uWPapEyrqH/USqJqO1HJinBq+l+E1/6NuYA8eXomztVm0= X-MS-TrafficTypeDiagnostic: DM3PR15MB1081: X-Microsoft-Exchange-Diagnostics: 1;DM3PR15MB1081;20:CWQc0DK8OaTEZus9NDHAwBh/jWK5xDsFjAfcYi+mWbEprbSGrp/4AX5R+rUs28Y1Of6I3+TLKQGnr/vnbfXqoAVowUvs3x0f3kRBgxCdsBAEYuAs/WltsMdFnjbjtKJR6TqLaxb9beuswOmBPDEIy3hE7psmqUXWnntC9JhpJMU7412lvNmT8Xbl7GheUoQsBrKaF8haUiAYm6OeSCKDiLZJOT5t6aWmKyZU0c4bAZLun8BHVAxnlKGKX6mb6yY9eHaYMOgnbpl5yjMIDdGBiQZOSaXu8Z2GrZZ4FZZY77NiLdYM8bwK0+MRwxtZ4YcCweoesKf/YmKVO3DFHTEkXsTeIumJjnm7t+VM1x/41FWPHjIPZlHae4z2j7PWS5fzw+8E06oQpK4wgQkK9tSYpC3VV8pwa0q7eZuEaj3xa7mFLr90kMrcvjSl4MelaVioEOwKKmXyNdMSnzoSRCEK46gPN1fc1AAdGN+OxJoCCDSZTay4IV1Hlbggyv+FVBx+;4:bFLW35EIrQ6Rgv7CsAkH8IsGqsCvTQJUGB7Ra7NZ5UxrKLz43pcqVJhfLhevg3ESSkPUmKghU1bhaNZhV0mC12VZUXZOYRvSWOrEVVPVv8Ey7Wv8yg0so4ooeIJ7C5voLlBAoh74BkLeNiZV5+tW3A+1SaP2kBJ8w4S44QDP1q0FrA79JOdMwT1zOBgmTfnGPIQF9CbRbwaaEz+F+Kf4PGAFKe9XehI9puLIOLPcvyaD/XBSd80lZkcBhpUAoNgxjSojI9dr02LYJMyJ+lnxNOmn9F1TvSM3ql/imZh6bmBQ3sceVaM9vtUsfWcQemIQ3vQuiV+Pe48nJFtXxfqNfTdwLS+AY6wEGqp0WjwxWGHtBTx4YqmLEdwkiet4GNAg8o3I7mqwXpdJXcslvwOMyg== X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(67672495146484)(211936372134217)(153496737603132); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(11241501159)(6040450)(2401047)(8121501046)(5005006)(93006095)(93001095)(3002001)(100000703101)(100105400095)(10201501046)(920507026)(6041248)(20161123560025)(20161123562025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123564025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:DM3PR15MB1081;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:DM3PR15MB1081; X-Forefront-PRVS: 0465429B7F X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(6009001)(346002)(376002)(189002)(199003)(53936002)(54906003)(48376002)(101416001)(6506006)(6486002)(316002)(6306002)(4326008)(69596002)(50466002)(5003940100001)(2906002)(25786009)(6916009)(6512007)(6666003)(39060400002)(97736004)(189998001)(86362001)(50226002)(6116002)(305945005)(7416002)(81156014)(53416004)(478600001)(34040400001)(50986999)(16586007)(7736002)(68736007)(33646002)(36756003)(966005)(47776003)(5660300001)(105586002)(2351001)(1076002)(106356001)(81166006)(8676002)(2361001)(8936002)(42262002);DIR:OUT;SFP:1102;SCL:1;SRVR:DM3PR15MB1081;H:castle.thefacebook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;DM3PR15MB1081;23:A7SejoSLZF5qOlu7jgepZF9uuU15selHdgq98Gwkm?= =?us-ascii?Q?soF/TRAfYBQBy59TVv04vk7XkamHHO5wtC7A6x0XshXvkPhrWDkGrGqhB/e1?= =?us-ascii?Q?vciKxlVLzuYUoMLnz3M6w14wLnRntEU71pbKJAVnaF7riBb4gsd5hYntrGsY?= =?us-ascii?Q?q70KLh4qMg6HIZ22pCcjgNrMk+yk5GNek9GkyogUBGY+l4EX6fYxNhMgqFfh?= =?us-ascii?Q?1eTXVnlbi/BbrAfa5HGo1GA8VTrtaW0c8Eco7MWZay39COceR83sIoUvFlIA?= =?us-ascii?Q?GqCJC108WQkb85glj021qjW14SaLnfa541S666s+P61aGt+LKIxMIlTJjQ2M?= =?us-ascii?Q?aCm242vhkiZfQaZmWTlJMkWLO8TYuWd3skCamK2aHnE8pzOsqQ+6jawZUV0B?= =?us-ascii?Q?MptE+Z6zGYTuKKKk1eq5rJtiCvKGPivPGjnjhq9FNAe7in6sC3it/60VnK0r?= =?us-ascii?Q?R3NzXQfeghlV2sJQ5eMCiK1tZDB3yhd2MbNGbCxxahAualjbsJXjbCw0wn33?= =?us-ascii?Q?QGKNmsDgq7wh79dsUqnPdtcJQwjGvoAqup2XP6T5c5lPY0/h+eyOlpHMeTot?= =?us-ascii?Q?MhcopGlzejBBB6RReb7JMU/nopiz16WdCm4ImQH91sAcg4xN2x0DYUavFQ7K?= =?us-ascii?Q?htrYrLiwvX+XA5zizl7S/1MhZsg9R7yh7t+I/0ZSFYXiSgkFe3NlG/l1mSqJ?= =?us-ascii?Q?oXnAUb11ZTw4dKBoXMzRDTEp2grUCdY6cXyVaEjPkxfxSLcUWeZj0CenP0Iu?= =?us-ascii?Q?lQVV2OxyT5K/2gD4KQtfIYOrCFPQfwm+ts8NNs9yyOMWJh8/GzzdBAEQT61n?= =?us-ascii?Q?BW7EDQwL0Njnc72k3PNY0XMxrVgufnZ1Ijz+JPEmwjMlehqQnutlo/tnhiCd?= =?us-ascii?Q?c/nRFZ36/iWqFuPvpIboDHOUe96D5VHnR9JD85zzw7eijCy4b272ckg2etAg?= =?us-ascii?Q?dvISqy8FQrIPYNM//v0fdKNnDJEpMmMJ062EDx6dQ+JYK+x6msdwIzIZw00k?= =?us-ascii?Q?O/TMQJs3kk2thEFBivetz2p9yi0Yjb1hTYvkzUIIirUOWf9B9ZTGyIY6hMy5?= =?us-ascii?Q?2ko7kppZflRlyWviWQfsXZi74W0Zz21nL9ZeiagkzgKjtkNjOBaet+7HPTNK?= =?us-ascii?Q?USQ0emDN6qq/BfBRY62RNI8196Ay9Rw97PP6TCp9oUy5IItqydvNJqCZ67CG?= =?us-ascii?Q?wi9hmNOPX07MUifHMo62+0cF38D10M10eXO?= X-Microsoft-Exchange-Diagnostics: 1;DM3PR15MB1081;6:2wQ9Nb9H1whjY/lPxA1P05xMTJHpPDvUis95h5ZRhF9xX3cHao9pd+HW1k9UpJkf5elM47Kg+MqDMOo67aBVCNs8ce+TPTlVvTJwZSXyrnnS4EuU3NWJaAIIo4GjhooTRc2SeXNTwwREOuSIaoi8XzQCflmL00FmWTHV4UmsL9bvjGzSwbTnGJymV3KRifDUyqi6ohPYfK+U9XOiEm2//tMOptLoiEwiGmUgi1dadd2GqGSgimpry6aia7aUacItEBUu4dIB8Ueq7QnY8c3G1uLLw78MuJtAt1GbHnX9H0WrR+FLj7YVFZqsXC4zHJWy/cwyEeCVBe5FugU0rvisHQ==;5:KEnz3KpjRO/ZWmfYnyUKXFzw1ep/vqJ+5lUnz4HXj0Ielciq5mdYlHxVCUZqkO2RcqMEopLQ3SxLEZC7nV7enL+Kz1OIKPfQ+uFlG/sQIkLE2o4HNJ6ldC8Hm0CIK2pBNprskC6GeKy83KQS5yrvmA==;24:nlzYQdAgCCRTG/3lbrj9RA+pYXZoqCpM+uU8mNVRycnnqDZpDV8Py0FOCwn3GGqVabsoL/Z8WtaFbSCcTl0Sl++PPHcpKkEaVdSEnZpUAMc=;7:marImcZ9vw7Gwch5cBp5s0YTCHvJKWCQVXSUA6ETVJijyRCHiLU6i9sHukdaFOGqaw1U0Hv3aDbSE92VRO9LLsJMTW9/PNyY5jzMZDDDLI50BsDMd+0NfmMcVjimwHvxWHxeJhD87MfNXFm1bwOCRRkkeoEmq2DN+UPtHJAc5pCNEyTJ88z+enzBZtMx5/hPmlPwqpyRIbk+nzuGpLkBLSq0SEI9pJ+PW9rQgT95eEc= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;DM3PR15MB1081;20:rpoJltItGvHkvLXb4jhLOoyH7ZuSicecpWw27Qc07ek+aRXkJv2b0tEFSLGrfvtYC6PHVlBK6RnRU0JWWUKYS+YxKBnhB5z3JFlAl+lrSgsR0mFJ7RGfM+IUqcFksdIzHG8OJkeyMROwFt51jvjk0GL7Ss6TO7cJwAaFuOpweWc= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Oct 2017 18:52:47.5401 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR15MB1081 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-10-19_09:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patchset makes the OOM killer cgroup-aware. v12: - Root memory cgroup is evaluated based on sum of the oom scores of belonging tasks - Do not fallback to the per-process behavior if there if it wasn't possbile to kill a memcg victim - Rebase on top of mm tree v11: - Fixed an issue with skipping the root mem cgroup (discovered by Shakeel Butt) - Moved a check in __oom_kill_process() to the memmory.oom_group patch, added corresponding comments - Added a note about ignoring tasks with oom_score_adj -1000 (proposed by Michal Hocko) - Rebase on top of mm tree v10: - Separate oom_group introduction into a standalone patch - Stop propagating oom_group - Make oom_group delegatable - Do not try to kill the biggest task in the first order, if the whole cgroup is going to be killed - Stop caching oom_score on struct memcg, optimize victim memcg selection - Drop dmesg printing (for further refining) - Small refactorings and comments added here and there - Rebase on top of mm tree v9: - Change siblings-to-siblings comparison to the tree-wide search, make related refactorings - Make oom_group implicitly propagated down by the tree - Fix an issue with task selection in root cgroup v8: - Do not kill tasks with OOM_SCORE_ADJ -1000 - Make the whole thing opt-in with cgroup mount option control - Drop oom_priority for further discussions - Kill the whole cgroup if oom_group is set and it's memory.max is reached - Update docs and commit messages v7: - __oom_kill_process() drops reference to the victim task - oom_score_adj -1000 is always respected - Renamed oom_kill_all to oom_group - Dropped oom_prio range, converted from short to int - Added a cgroup v2 mount option to disable cgroup-aware OOM killer - Docs updated - Rebased on top of mmotm v6: - Renamed oom_control.chosen to oom_control.chosen_task - Renamed oom_kill_all_tasks to oom_kill_all - Per-node NR_SLAB_UNRECLAIMABLE accounting - Several minor fixes and cleanups - Docs updated v5: - Rebased on top of Michal Hocko's patches, which have changed the way how OOM victims becoming an access to the memory reserves. Dropped corresponding part of this patchset - Separated the oom_kill_process() splitting into a standalone commit - Added debug output (suggested by David Rientjes) - Some minor fixes v4: - Reworked per-cgroup oom_score_adj into oom_priority (based on ideas by David Rientjes) - Tasks with oom_score_adj -1000 are never selected if oom_kill_all_tasks is not set - Memcg victim selection code is reworked, and synchronization is based on finding tasks with OOM victim marker, rather then on global counter - Debug output is dropped - Refactored TIF_MEMDIE usage v3: - Merged commits 1-4 into 6 - Separated oom_score_adj logic and debug output into separate commits - Fixed swap accounting v2: - Reworked victim selection based on feedback from Michal Hocko, Vladimir Davydov and Johannes Weiner - "Kill all tasks" is now an opt-in option, by default only one process will be killed - Added per-cgroup oom_score_adj - Refined oom score calculations, suggested by Vladimir Davydov - Converted to a patchset v1: https://lkml.org/lkml/2017/5/18/969 Cc: Michal Hocko Cc: Vladimir Davydov Cc: Johannes Weiner Cc: Tetsuo Handa Cc: David Rientjes Cc: Andrew Morton Cc: Tejun Heo Cc: kernel-team@fb.com Cc: cgroups@vger.kernel.org Cc: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org Roman Gushchin (6): mm, oom: refactor the oom_kill_process() function mm: implement mem_cgroup_scan_tasks() for the root memory cgroup mm, oom: cgroup-aware OOM killer mm, oom: introduce memory.oom_group mm, oom: add cgroup v2 mount option for cgroup-aware OOM killer mm, oom, docs: describe the cgroup-aware OOM killer Documentation/cgroup-v2.txt | 51 +++++++++ include/linux/cgroup-defs.h | 5 + include/linux/memcontrol.h | 34 ++++++ include/linux/oom.h | 12 ++- kernel/cgroup/cgroup.c | 10 ++ mm/memcontrol.c | 258 +++++++++++++++++++++++++++++++++++++++++++- mm/oom_kill.c | 212 ++++++++++++++++++++++++------------ 7 files changed, 506 insertions(+), 76 deletions(-) -- 2.13.6