linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
* [PATCH] mm, memcg: skip killing processes under memcg protection at first scan
@ 2019-08-18  4:24 Yafang Shao
  2019-08-18 14:08 ` kbuild test robot
                   ` (3 more replies)
  0 siblings, 4 replies; 9+ messages in thread
From: Yafang Shao @ 2019-08-18  4:24 UTC (permalink / raw)
  To: akpm
  Cc: linux-mm, Yafang Shao, Roman Gushchin, Randy Dunlap,
	Johannes Weiner, Michal Hocko, Vladimir Davydov, Tetsuo Handa

In the current memory.min design, the system is going to do OOM instead
of reclaiming the reclaimable pages protected by memory.min if the
system is lack of free memory. While under this condition, the OOM
killer may kill the processes in the memcg protected by memory.min.
This behavior is very weird.
In order to make it more reasonable, I make some changes in the OOM
killer. In this patch, the OOM killer will do two-round scan. It will
skip the processes under memcg protection at the first scan, and if it
can't kill any processes it will rescan all the processes.

Regarding the overhead this change may takes, I don't think it will be a
problem because this only happens under system  memory pressure and
the OOM killer can't find any proper victims which are not under memcg
protection.

Signed-off-by: Yafang Shao <laoar.shao@gmail.com>
Cc: Roman Gushchin <guro@fb.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
---
 include/linux/memcontrol.h |  6 ++++++
 mm/memcontrol.c            | 16 ++++++++++++++++
 mm/oom_kill.c              | 23 +++++++++++++++++++++--
 3 files changed, 43 insertions(+), 2 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 44c4146..58bd86b 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -337,6 +337,7 @@ static inline bool mem_cgroup_disabled(void)
 
 enum mem_cgroup_protection mem_cgroup_protected(struct mem_cgroup *root,
 						struct mem_cgroup *memcg);
+int task_under_memcg_protection(struct task_struct *p);
 
 int mem_cgroup_try_charge(struct page *page, struct mm_struct *mm,
 			  gfp_t gfp_mask, struct mem_cgroup **memcgp,
@@ -813,6 +814,11 @@ static inline enum mem_cgroup_protection mem_cgroup_protected(
 	return MEMCG_PROT_NONE;
 }
 
+int task_under_memcg_protection(struct task_struct *p)
+{
+	return 0;
+}
+
 static inline int mem_cgroup_try_charge(struct page *page, struct mm_struct *mm,
 					gfp_t gfp_mask,
 					struct mem_cgroup **memcgp,
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index cdbb7a8..c4d8e53 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -6030,6 +6030,22 @@ enum mem_cgroup_protection mem_cgroup_protected(struct mem_cgroup *root,
 		return MEMCG_PROT_NONE;
 }
 
+int task_under_memcg_protection(struct task_struct *p)
+{
+	struct mem_cgroup *memcg;
+	int protected;
+
+	rcu_read_lock();
+	memcg = mem_cgroup_from_task(p);
+	if (memcg != root_mem_cgroup && memcg->memory.min)
+		protected = 1;
+	else
+		protected = 0;
+	rcu_read_unlock();
+
+	return protected;
+}
+
 /**
  * mem_cgroup_try_charge - try charging a page
  * @page: page to charge
diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index eda2e2a..259dd2c 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -368,11 +368,30 @@ static void select_bad_process(struct oom_control *oc)
 		mem_cgroup_scan_tasks(oc->memcg, oom_evaluate_task, oc);
 	else {
 		struct task_struct *p;
+		int memcg_check = 0;
+		int memcg_skip = 0;
+		int selected = 0;
 
 		rcu_read_lock();
-		for_each_process(p)
-			if (oom_evaluate_task(p, oc))
+retry:
+		for_each_process(p) {
+			if (!memcg_check && task_under_memcg_protection(p)) {
+				memcg_skip = 1;
+				continue;
+			}
+			selected = oom_evaluate_task(p, oc);
+			if (selected)
 				break;
+		}
+
+		if (!selected) {
+			if (memcg_skip) {
+				if (!oc->chosen || oc->chosen == (void *)-1UL) {
+					memcg_check = 1;
+					goto retry;
+				}
+			}
+		}
 		rcu_read_unlock();
 	}
 }
-- 
1.8.3.1



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [PATCH] mm, memcg: skip killing processes under memcg protection at first scan
  2019-08-18  4:24 [PATCH] mm, memcg: skip killing processes under memcg protection at first scan Yafang Shao
@ 2019-08-18 14:08 ` kbuild test robot
  2019-08-18 14:08 ` kbuild test robot
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 9+ messages in thread
From: kbuild test robot @ 2019-08-18 14:08 UTC (permalink / raw)
  To: Yafang Shao
  Cc: kbuild-all, akpm, linux-mm, Yafang Shao, Roman Gushchin,
	Randy Dunlap, Johannes Weiner, Michal Hocko, Vladimir Davydov,
	Tetsuo Handa

[-- Attachment #1: Type: text/plain, Size: 10829 bytes --]

Hi Yafang,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linus/master]
[cannot apply to v5.3-rc4 next-20190816]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Yafang-Shao/mm-memcg-skip-killing-processes-under-memcg-protection-at-first-scan/20190818-205854
config: i386-randconfig-a002-201933 (attached as .config)
compiler: gcc-7 (Debian 7.4.0-10) 7.4.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

If you fix the issue, kindly add following tag
Reported-by: kbuild test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   ld: net/nfc/netlink.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/nfc/core.o:include/linux/memcontrol.h:819: first defined here
   ld: net/nfc/af_nfc.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/nfc/core.o:include/linux/memcontrol.h:819: first defined here
   ld: net/nfc/rawsock.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/nfc/core.o:include/linux/memcontrol.h:819: first defined here
   ld: net/nfc/llcp_core.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/nfc/core.o:include/linux/memcontrol.h:819: first defined here
   ld: net/nfc/llcp_commands.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/nfc/core.o:include/linux/memcontrol.h:819: first defined here
   ld: net/nfc/llcp_sock.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/nfc/core.o:include/linux/memcontrol.h:819: first defined here
--
   ld: net/nfc/nci/data.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/nfc/nci/core.o:include/linux/memcontrol.h:819: first defined here
   ld: net/nfc/nci/ntf.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/nfc/nci/core.o:include/linux/memcontrol.h:819: first defined here
   ld: net/nfc/nci/rsp.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/nfc/nci/core.o:include/linux/memcontrol.h:819: first defined here
   ld: net/nfc/nci/hci.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/nfc/nci/core.o:include/linux/memcontrol.h:819: first defined here
--
   ld: drivers/isdn/mISDN/dsp_cmx.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/isdn/mISDN/dsp_core.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/isdn/mISDN/dsp_tones.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/isdn/mISDN/dsp_core.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/isdn/mISDN/dsp_dtmf.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/isdn/mISDN/dsp_core.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/isdn/mISDN/dsp_audio.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/isdn/mISDN/dsp_core.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/isdn/mISDN/dsp_blowfish.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/isdn/mISDN/dsp_core.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/isdn/mISDN/dsp_pipeline.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/isdn/mISDN/dsp_core.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/isdn/mISDN/dsp_hwec.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/isdn/mISDN/dsp_core.o:include/linux/memcontrol.h:819: first defined here
--
   ld: drivers/isdn/mISDN/l1oip_codec.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/isdn/mISDN/l1oip_core.o:include/linux/memcontrol.h:819: first defined here
--
   ld: drivers/net/bonding/bond_alb.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/bonding/bond_main.o:include/linux/memcontrol.h:819: first defined here
--
   ld: fs/reiserfs/file.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; fs/reiserfs/inode.o:include/linux/memcontrol.h:819: first defined here
--
   ld: drivers/net/ipvlan/ipvlan_main.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ipvlan/ipvlan_core.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/ipvlan/ipvlan_l3s.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ipvlan/ipvlan_core.o:include/linux/memcontrol.h:819: first defined here
--
   ld: drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ethernet/broadcom/bnxt/bnxt.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ethernet/broadcom/bnxt/bnxt.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/ethernet/broadcom/bnxt/bnxt_dcb.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ethernet/broadcom/bnxt/bnxt.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ethernet/broadcom/bnxt/bnxt.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ethernet/broadcom/bnxt/bnxt.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/ethernet/broadcom/bnxt/bnxt_vfr.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ethernet/broadcom/bnxt/bnxt.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ethernet/broadcom/bnxt/bnxt.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/ethernet/broadcom/bnxt/bnxt_dim.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ethernet/broadcom/bnxt/bnxt.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/ethernet/broadcom/bnxt/bnxt_tc.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ethernet/broadcom/bnxt/bnxt.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/ethernet/broadcom/bnxt/bnxt_debugfs.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ethernet/broadcom/bnxt/bnxt.o:include/linux/memcontrol.h:819: first defined here
--
   ld: drivers/net/vmxnet3/vmxnet3_ethtool.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/vmxnet3/vmxnet3_drv.o:include/linux/memcontrol.h:819: first defined here
--
   ld: drivers/net/ethernet/chelsio/cxgb4/sge.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/ethernet/chelsio/cxgb4/clip_tbl.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/ethernet/chelsio/cxgb4/cxgb4_filter.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/ethernet/chelsio/cxgb4/cxgb4_ptp.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.o:include/linux/memcontrol.h:819: first defined here
--
   ld: drivers/net/ethernet/cisco/enic/enic_pp.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/ethernet/cisco/enic/enic_main.o:include/linux/memcontrol.h:819: first defined here
..

vim +819 include/linux/memcontrol.h

   817	
   818	int task_under_memcg_protection(struct task_struct *p)
 > 819	{
   820		return 0;
   821	}
   822	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 36392 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] mm, memcg: skip killing processes under memcg protection at first scan
  2019-08-18  4:24 [PATCH] mm, memcg: skip killing processes under memcg protection at first scan Yafang Shao
  2019-08-18 14:08 ` kbuild test robot
@ 2019-08-18 14:08 ` kbuild test robot
  2019-08-18 17:11 ` Souptick Joarder
  2019-08-19  7:31 ` Michal Hocko
  3 siblings, 0 replies; 9+ messages in thread
From: kbuild test robot @ 2019-08-18 14:08 UTC (permalink / raw)
  To: Yafang Shao
  Cc: kbuild-all, akpm, linux-mm, Yafang Shao, Roman Gushchin,
	Randy Dunlap, Johannes Weiner, Michal Hocko, Vladimir Davydov,
	Tetsuo Handa

[-- Attachment #1: Type: text/plain, Size: 11779 bytes --]

Hi Yafang,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linus/master]
[cannot apply to v5.3-rc4 next-20190816]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Yafang-Shao/mm-memcg-skip-killing-processes-under-memcg-protection-at-first-scan/20190818-205854
config: i386-randconfig-f004-201933 (attached as .config)
compiler: gcc-7 (Debian 7.4.0-10) 7.4.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

If you fix the issue, kindly add following tag
Reported-by: kbuild test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   ld: net/appletalk/ddp.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/appletalk/aarp.o:include/linux/memcontrol.h:819: first defined here
   ld: net/appletalk/atalk_proc.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/appletalk/aarp.o:include/linux/memcontrol.h:819: first defined here
   ld: net/appletalk/sysctl_net_atalk.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/appletalk/aarp.o:include/linux/memcontrol.h:819: first defined here
--
   ld: fs/dlm/lowcomms.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; fs/dlm/config.o:include/linux/memcontrol.h:819: first defined here
--
   ld: drivers/connector/connector.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/connector/cn_queue.o:include/linux/memcontrol.h:819: first defined here
--
   ld: net/netfilter/nft_chain_filter.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/netfilter/nf_tables_api.o:include/linux/memcontrol.h:819: first defined here
   ld: net/netfilter/nft_payload.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/netfilter/nf_tables_api.o:include/linux/memcontrol.h:819: first defined here
   ld: net/netfilter/nft_meta.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/netfilter/nf_tables_api.o:include/linux/memcontrol.h:819: first defined here
   ld: net/netfilter/nft_rt.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/netfilter/nf_tables_api.o:include/linux/memcontrol.h:819: first defined here
   ld: net/netfilter/nft_exthdr.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/netfilter/nf_tables_api.o:include/linux/memcontrol.h:819: first defined here
   ld: net/netfilter/nft_chain_route.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/netfilter/nf_tables_api.o:include/linux/memcontrol.h:819: first defined here
--
   ld: net/netfilter/nf_flow_table_ip.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/netfilter/nf_flow_table_core.o:include/linux/memcontrol.h:819: first defined here
--
   ld: net/openvswitch/datapath.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/openvswitch/actions.o:include/linux/memcontrol.h:819: first defined here
   ld: net/openvswitch/dp_notify.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/openvswitch/actions.o:include/linux/memcontrol.h:819: first defined here
   ld: net/openvswitch/flow.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/openvswitch/actions.o:include/linux/memcontrol.h:819: first defined here
   ld: net/openvswitch/flow_netlink.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/openvswitch/actions.o:include/linux/memcontrol.h:819: first defined here
   ld: net/openvswitch/flow_table.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/openvswitch/actions.o:include/linux/memcontrol.h:819: first defined here
   ld: net/openvswitch/meter.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/openvswitch/actions.o:include/linux/memcontrol.h:819: first defined here
   ld: net/openvswitch/vport.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/openvswitch/actions.o:include/linux/memcontrol.h:819: first defined here
   ld: net/openvswitch/vport-internal_dev.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/openvswitch/actions.o:include/linux/memcontrol.h:819: first defined here
   ld: net/openvswitch/vport-netdev.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/openvswitch/actions.o:include/linux/memcontrol.h:819: first defined here
   ld: net/openvswitch/conntrack.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; net/openvswitch/actions.o:include/linux/memcontrol.h:819: first defined here
--
   ld: drivers/mfd/cs47l15-tables.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/mfd/madera-core.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/mfd/cs47l92-tables.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/mfd/madera-core.o:include/linux/memcontrol.h:819: first defined here
--
   ld: drivers/net/wireless/ath/regd.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/ath/main.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/wireless/ath/hw.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/ath/main.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/wireless/ath/key.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/ath/main.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/wireless/ath/dfs_pattern_detector.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/ath/main.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/wireless/ath/dfs_pri_detector.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/ath/main.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/wireless/ath/debug.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/ath/main.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/wireless/ath/trace.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/ath/main.o:include/linux/memcontrol.h:819: first defined here
--
   ld: drivers/net/wireless/ath/wcn36xx/dxe.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/ath/wcn36xx/main.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/wireless/ath/wcn36xx/txrx.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/ath/wcn36xx/main.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/wireless/ath/wcn36xx/smd.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/ath/wcn36xx/main.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/wireless/ath/wcn36xx/pmc.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/ath/wcn36xx/main.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/wireless/ath/wcn36xx/debug.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/ath/wcn36xx/main.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/wireless/ath/wcn36xx/testmode.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/ath/wcn36xx/main.o:include/linux/memcontrol.h:819: first defined here
--
   ld: drivers/net/wireless/intersil/p54/fwio.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/intersil/p54/eeprom.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/wireless/intersil/p54/txrx.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/intersil/p54/eeprom.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/wireless/intersil/p54/main.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/intersil/p54/eeprom.o:include/linux/memcontrol.h:819: first defined here
   ld: drivers/net/wireless/intersil/p54/led.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/intersil/p54/eeprom.o:include/linux/memcontrol.h:819: first defined here
--
   ld: drivers/net/wireless/mediatek/mt76/mt76x0/usb_mcu.o: in function `task_under_memcg_protection':
>> include/linux/memcontrol.h:819: multiple definition of `task_under_memcg_protection'; drivers/net/wireless/mediatek/mt76/mt76x0/usb.o:include/linux/memcontrol.h:819: first defined here
..

vim +819 include/linux/memcontrol.h

   817	
   818	int task_under_memcg_protection(struct task_struct *p)
 > 819	{
   820		return 0;
   821	}
   822	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 31782 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] mm, memcg: skip killing processes under memcg protection at first scan
  2019-08-18  4:24 [PATCH] mm, memcg: skip killing processes under memcg protection at first scan Yafang Shao
  2019-08-18 14:08 ` kbuild test robot
  2019-08-18 14:08 ` kbuild test robot
@ 2019-08-18 17:11 ` Souptick Joarder
  2019-08-19  1:03   ` Yafang Shao
  2019-08-19  7:31 ` Michal Hocko
  3 siblings, 1 reply; 9+ messages in thread
From: Souptick Joarder @ 2019-08-18 17:11 UTC (permalink / raw)
  To: Yafang Shao
  Cc: Andrew Morton, Linux-MM, Roman Gushchin, Randy Dunlap,
	Johannes Weiner, Michal Hocko, Vladimir Davydov, Tetsuo Handa

On Sun, Aug 18, 2019 at 9:55 AM Yafang Shao <laoar.shao@gmail.com> wrote:
>
> In the current memory.min design, the system is going to do OOM instead
> of reclaiming the reclaimable pages protected by memory.min if the
> system is lack of free memory. While under this condition, the OOM
> killer may kill the processes in the memcg protected by memory.min.
> This behavior is very weird.
> In order to make it more reasonable, I make some changes in the OOM
> killer. In this patch, the OOM killer will do two-round scan. It will
> skip the processes under memcg protection at the first scan, and if it
> can't kill any processes it will rescan all the processes.
>
> Regarding the overhead this change may takes, I don't think it will be a
> problem because this only happens under system  memory pressure and
> the OOM killer can't find any proper victims which are not under memcg
> protection.
>
> Signed-off-by: Yafang Shao <laoar.shao@gmail.com>
> Cc: Roman Gushchin <guro@fb.com>
> Cc: Randy Dunlap <rdunlap@infradead.org>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Cc: Michal Hocko <mhocko@suse.com>
> Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
> Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
> ---
>  include/linux/memcontrol.h |  6 ++++++
>  mm/memcontrol.c            | 16 ++++++++++++++++
>  mm/oom_kill.c              | 23 +++++++++++++++++++++--
>  3 files changed, 43 insertions(+), 2 deletions(-)
>
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index 44c4146..58bd86b 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -337,6 +337,7 @@ static inline bool mem_cgroup_disabled(void)
>
>  enum mem_cgroup_protection mem_cgroup_protected(struct mem_cgroup *root,
>                                                 struct mem_cgroup *memcg);
> +int task_under_memcg_protection(struct task_struct *p);
>
>  int mem_cgroup_try_charge(struct page *page, struct mm_struct *mm,
>                           gfp_t gfp_mask, struct mem_cgroup **memcgp,
> @@ -813,6 +814,11 @@ static inline enum mem_cgroup_protection mem_cgroup_protected(
>         return MEMCG_PROT_NONE;
>  }
>
> +int task_under_memcg_protection(struct task_struct *p)
> +{
> +       return 0;
> +}
> +
>  static inline int mem_cgroup_try_charge(struct page *page, struct mm_struct *mm,
>                                         gfp_t gfp_mask,
>                                         struct mem_cgroup **memcgp,
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index cdbb7a8..c4d8e53 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -6030,6 +6030,22 @@ enum mem_cgroup_protection mem_cgroup_protected(struct mem_cgroup *root,
>                 return MEMCG_PROT_NONE;
>  }
>
> +int task_under_memcg_protection(struct task_struct *p)
> +{
> +       struct mem_cgroup *memcg;
> +       int protected;
> +
> +       rcu_read_lock();
> +       memcg = mem_cgroup_from_task(p);
> +       if (memcg != root_mem_cgroup && memcg->memory.min)
> +               protected = 1;
> +       else
> +               protected = 0;
> +       rcu_read_unlock();
> +
> +       return protected;

I think returning a bool type would be more appropriate.

> +}
> +
>  /**
>   * mem_cgroup_try_charge - try charging a page
>   * @page: page to charge
> diff --git a/mm/oom_kill.c b/mm/oom_kill.c
> index eda2e2a..259dd2c 100644
> --- a/mm/oom_kill.c
> +++ b/mm/oom_kill.c
> @@ -368,11 +368,30 @@ static void select_bad_process(struct oom_control *oc)
>                 mem_cgroup_scan_tasks(oc->memcg, oom_evaluate_task, oc);
>         else {
>                 struct task_struct *p;
> +               int memcg_check = 0;
> +               int memcg_skip = 0;
> +               int selected = 0;
>
>                 rcu_read_lock();
> -               for_each_process(p)
> -                       if (oom_evaluate_task(p, oc))
> +retry:
> +               for_each_process(p) {
> +                       if (!memcg_check && task_under_memcg_protection(p)) {
> +                               memcg_skip = 1;
> +                               continue;
> +                       }
> +                       selected = oom_evaluate_task(p, oc);
> +                       if (selected)
>                                 break;
> +               }
> +
> +               if (!selected) {
> +                       if (memcg_skip) {
> +                               if (!oc->chosen || oc->chosen == (void *)-1UL) {
> +                                       memcg_check = 1;
> +                                       goto retry;
> +                               }
> +                       }
> +               }
>                 rcu_read_unlock();
>         }
>  }
> --
> 1.8.3.1
>
>


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] mm, memcg: skip killing processes under memcg protection at first scan
  2019-08-18 17:11 ` Souptick Joarder
@ 2019-08-19  1:03   ` Yafang Shao
  0 siblings, 0 replies; 9+ messages in thread
From: Yafang Shao @ 2019-08-19  1:03 UTC (permalink / raw)
  To: Souptick Joarder
  Cc: Andrew Morton, Linux-MM, Roman Gushchin, Randy Dunlap,
	Johannes Weiner, Michal Hocko, Vladimir Davydov, Tetsuo Handa

On Mon, Aug 19, 2019 at 1:11 AM Souptick Joarder <jrdr.linux@gmail.com> wrote:
>
> On Sun, Aug 18, 2019 at 9:55 AM Yafang Shao <laoar.shao@gmail.com> wrote:
> >
> > In the current memory.min design, the system is going to do OOM instead
> > of reclaiming the reclaimable pages protected by memory.min if the
> > system is lack of free memory. While under this condition, the OOM
> > killer may kill the processes in the memcg protected by memory.min.
> > This behavior is very weird.
> > In order to make it more reasonable, I make some changes in the OOM
> > killer. In this patch, the OOM killer will do two-round scan. It will
> > skip the processes under memcg protection at the first scan, and if it
> > can't kill any processes it will rescan all the processes.
> >
> > Regarding the overhead this change may takes, I don't think it will be a
> > problem because this only happens under system  memory pressure and
> > the OOM killer can't find any proper victims which are not under memcg
> > protection.
> >
> > Signed-off-by: Yafang Shao <laoar.shao@gmail.com>
> > Cc: Roman Gushchin <guro@fb.com>
> > Cc: Randy Dunlap <rdunlap@infradead.org>
> > Cc: Johannes Weiner <hannes@cmpxchg.org>
> > Cc: Michal Hocko <mhocko@suse.com>
> > Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
> > Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
> > ---
> >  include/linux/memcontrol.h |  6 ++++++
> >  mm/memcontrol.c            | 16 ++++++++++++++++
> >  mm/oom_kill.c              | 23 +++++++++++++++++++++--
> >  3 files changed, 43 insertions(+), 2 deletions(-)
> >
> > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> > index 44c4146..58bd86b 100644
> > --- a/include/linux/memcontrol.h
> > +++ b/include/linux/memcontrol.h
> > @@ -337,6 +337,7 @@ static inline bool mem_cgroup_disabled(void)
> >
> >  enum mem_cgroup_protection mem_cgroup_protected(struct mem_cgroup *root,
> >                                                 struct mem_cgroup *memcg);
> > +int task_under_memcg_protection(struct task_struct *p);
> >
> >  int mem_cgroup_try_charge(struct page *page, struct mm_struct *mm,
> >                           gfp_t gfp_mask, struct mem_cgroup **memcgp,
> > @@ -813,6 +814,11 @@ static inline enum mem_cgroup_protection mem_cgroup_protected(
> >         return MEMCG_PROT_NONE;
> >  }
> >
> > +int task_under_memcg_protection(struct task_struct *p)
> > +{
> > +       return 0;
> > +}
> > +
> >  static inline int mem_cgroup_try_charge(struct page *page, struct mm_struct *mm,
> >                                         gfp_t gfp_mask,
> >                                         struct mem_cgroup **memcgp,
> > diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> > index cdbb7a8..c4d8e53 100644
> > --- a/mm/memcontrol.c
> > +++ b/mm/memcontrol.c
> > @@ -6030,6 +6030,22 @@ enum mem_cgroup_protection mem_cgroup_protected(struct mem_cgroup *root,
> >                 return MEMCG_PROT_NONE;
> >  }
> >
> > +int task_under_memcg_protection(struct task_struct *p)
> > +{
> > +       struct mem_cgroup *memcg;
> > +       int protected;
> > +
> > +       rcu_read_lock();
> > +       memcg = mem_cgroup_from_task(p);
> > +       if (memcg != root_mem_cgroup && memcg->memory.min)
> > +               protected = 1;
> > +       else
> > +               protected = 0;
> > +       rcu_read_unlock();
> > +
> > +       return protected;
>
> I think returning a bool type would be more appropriate.
>

Sure. Will change it.

> > +}
> > +
> >  /**
> >   * mem_cgroup_try_charge - try charging a page
> >   * @page: page to charge
> > diff --git a/mm/oom_kill.c b/mm/oom_kill.c
> > index eda2e2a..259dd2c 100644
> > --- a/mm/oom_kill.c
> > +++ b/mm/oom_kill.c
> > @@ -368,11 +368,30 @@ static void select_bad_process(struct oom_control *oc)
> >                 mem_cgroup_scan_tasks(oc->memcg, oom_evaluate_task, oc);
> >         else {
> >                 struct task_struct *p;
> > +               int memcg_check = 0;
> > +               int memcg_skip = 0;
> > +               int selected = 0;
> >
> >                 rcu_read_lock();
> > -               for_each_process(p)
> > -                       if (oom_evaluate_task(p, oc))
> > +retry:
> > +               for_each_process(p) {
> > +                       if (!memcg_check && task_under_memcg_protection(p)) {
> > +                               memcg_skip = 1;
> > +                               continue;
> > +                       }
> > +                       selected = oom_evaluate_task(p, oc);
> > +                       if (selected)
> >                                 break;
> > +               }
> > +
> > +               if (!selected) {
> > +                       if (memcg_skip) {
> > +                               if (!oc->chosen || oc->chosen == (void *)-1UL) {
> > +                                       memcg_check = 1;
> > +                                       goto retry;
> > +                               }
> > +                       }
> > +               }
> >                 rcu_read_unlock();
> >         }
> >  }
> > --
> > 1.8.3.1
> >
> >


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] mm, memcg: skip killing processes under memcg protection at first scan
  2019-08-18  4:24 [PATCH] mm, memcg: skip killing processes under memcg protection at first scan Yafang Shao
                   ` (2 preceding siblings ...)
  2019-08-18 17:11 ` Souptick Joarder
@ 2019-08-19  7:31 ` Michal Hocko
  2019-08-19  8:15   ` Yafang Shao
  3 siblings, 1 reply; 9+ messages in thread
From: Michal Hocko @ 2019-08-19  7:31 UTC (permalink / raw)
  To: Yafang Shao
  Cc: akpm, linux-mm, Roman Gushchin, Randy Dunlap, Johannes Weiner,
	Vladimir Davydov, Tetsuo Handa

On Sun 18-08-19 00:24:54, Yafang Shao wrote:
> In the current memory.min design, the system is going to do OOM instead
> of reclaiming the reclaimable pages protected by memory.min if the
> system is lack of free memory. While under this condition, the OOM
> killer may kill the processes in the memcg protected by memory.min.

Could you be more specific about the configuration that leads to this
situation?
-- 
Michal Hocko
SUSE Labs


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] mm, memcg: skip killing processes under memcg protection at first scan
  2019-08-19  7:31 ` Michal Hocko
@ 2019-08-19  8:15   ` Yafang Shao
  2019-08-20  6:31     ` Michal Hocko
  0 siblings, 1 reply; 9+ messages in thread
From: Yafang Shao @ 2019-08-19  8:15 UTC (permalink / raw)
  To: Michal Hocko
  Cc: Andrew Morton, Linux MM, Roman Gushchin, Randy Dunlap,
	Johannes Weiner, Vladimir Davydov, Tetsuo Handa

On Mon, Aug 19, 2019 at 3:31 PM Michal Hocko <mhocko@suse.com> wrote:
>
> On Sun 18-08-19 00:24:54, Yafang Shao wrote:
> > In the current memory.min design, the system is going to do OOM instead
> > of reclaiming the reclaimable pages protected by memory.min if the
> > system is lack of free memory. While under this condition, the OOM
> > killer may kill the processes in the memcg protected by memory.min.
>
> Could you be more specific about the configuration that leads to this
> situation?

When I did memory pressure test to verify memory.min I found that issue.
This issue can be produced as bellow,
    memcg setting,
        memory.max: 1G
        memory.min: 512M
        some processes are running is this memcg, with both serveral
hundreds MB  file mapping and serveral hundreds MB anon mapping.
    system setting,
         swap: off.
         some memory pressure test are running on the system.

When the memory usage of this memcg is bellow the memory.min, the
global reclaimers stop reclaiming pages in this memcg, and when
there's no available memory, the OOM killer will be invoked.
Unfortunately the OOM killer can chose the process running in the
protected memcg.

In order to produce it easy, you can incease the memroy.min and set
-1000 to the oom_socre_adj of the processes outside of the protected
memcg.

Is this setting proper ?

Thanks
Yafang


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] mm, memcg: skip killing processes under memcg protection at first scan
  2019-08-19  8:15   ` Yafang Shao
@ 2019-08-20  6:31     ` Michal Hocko
  2019-08-20  7:02       ` Yafang Shao
  0 siblings, 1 reply; 9+ messages in thread
From: Michal Hocko @ 2019-08-20  6:31 UTC (permalink / raw)
  To: Yafang Shao
  Cc: Andrew Morton, Linux MM, Roman Gushchin, Randy Dunlap,
	Johannes Weiner, Vladimir Davydov, Tetsuo Handa

[hmm the email got stuck on my send queue - sending again]

On Mon 19-08-19 16:15:08, Yafang Shao wrote:
> On Mon, Aug 19, 2019 at 3:31 PM Michal Hocko <mhocko@suse.com> wrote:
> >
> > On Sun 18-08-19 00:24:54, Yafang Shao wrote:
> > > In the current memory.min design, the system is going to do OOM instead
> > > of reclaiming the reclaimable pages protected by memory.min if the
> > > system is lack of free memory. While under this condition, the OOM
> > > killer may kill the processes in the memcg protected by memory.min.
> >
> > Could you be more specific about the configuration that leads to this
> > situation?
> 
> When I did memory pressure test to verify memory.min I found that issue.
> This issue can be produced as bellow,
>     memcg setting,
>         memory.max: 1G
>         memory.min: 512M
>         some processes are running is this memcg, with both serveral
> hundreds MB  file mapping and serveral hundreds MB anon mapping.
>     system setting,
>          swap: off.
>          some memory pressure test are running on the system.
> 
> When the memory usage of this memcg is bellow the memory.min, the
> global reclaimers stop reclaiming pages in this memcg, and when
> there's no available memory, the OOM killer will be invoked.
> Unfortunately the OOM killer can chose the process running in the
> protected memcg.

Well, the memcg protection was designed to prevent from regular
memory reclaim.  It was not aimed at acting as a group wide oom
protection. The global oom killer (but memcg as well) simply cares only
about oom_score_adj when selecting a victim.

Adding yet another oom protection is likely to complicate the oom
selection logic and make it more surprising. E.g. why should workload
fitting inside the min limit be so special? Do you have any real world
example?
 
> In order to produce it easy, you can incease the memroy.min and set
> -1000 to the oom_socre_adj of the processes outside of the protected
> memcg.

This sounds like a very dubious configuration to me. There is no other
option than chosing from the protected group.

> Is this setting proper ?
> 
> Thanks
> Yafang

-- 
Michal Hocko
SUSE Labs


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] mm, memcg: skip killing processes under memcg protection at first scan
  2019-08-20  6:31     ` Michal Hocko
@ 2019-08-20  7:02       ` Yafang Shao
  0 siblings, 0 replies; 9+ messages in thread
From: Yafang Shao @ 2019-08-20  7:02 UTC (permalink / raw)
  To: Michal Hocko
  Cc: Andrew Morton, Linux MM, Roman Gushchin, Randy Dunlap,
	Johannes Weiner, Vladimir Davydov, Tetsuo Handa

On Tue, Aug 20, 2019 at 2:31 PM Michal Hocko <mhocko@suse.com> wrote:
>
> [hmm the email got stuck on my send queue - sending again]
>
> On Mon 19-08-19 16:15:08, Yafang Shao wrote:
> > On Mon, Aug 19, 2019 at 3:31 PM Michal Hocko <mhocko@suse.com> wrote:
> > >
> > > On Sun 18-08-19 00:24:54, Yafang Shao wrote:
> > > > In the current memory.min design, the system is going to do OOM instead
> > > > of reclaiming the reclaimable pages protected by memory.min if the
> > > > system is lack of free memory. While under this condition, the OOM
> > > > killer may kill the processes in the memcg protected by memory.min.
> > >
> > > Could you be more specific about the configuration that leads to this
> > > situation?
> >
> > When I did memory pressure test to verify memory.min I found that issue.
> > This issue can be produced as bellow,
> >     memcg setting,
> >         memory.max: 1G
> >         memory.min: 512M
> >         some processes are running is this memcg, with both serveral
> > hundreds MB  file mapping and serveral hundreds MB anon mapping.
> >     system setting,
> >          swap: off.
> >          some memory pressure test are running on the system.
> >
> > When the memory usage of this memcg is bellow the memory.min, the
> > global reclaimers stop reclaiming pages in this memcg, and when
> > there's no available memory, the OOM killer will be invoked.
> > Unfortunately the OOM killer can chose the process running in the
> > protected memcg.
>
> Well, the memcg protection was designed to prevent from regular
> memory reclaim.  It was not aimed at acting as a group wide oom
> protection. The global oom killer (but memcg as well) simply cares only
> about oom_score_adj when selecting a victim.
>

OOM is a kind of memory reclaim, isn't it ?

> Adding yet another oom protection is likely to complicate the oom
> selection logic and make it more surprising. E.g. why should workload
> fitting inside the min limit be so special? Do you have any real world
> example?
>

The problem here is we want to use it ini the real world, but the
issuses we found  prevent us from using it in the real world.


> > In order to produce it easy, you can incease the memroy.min and set
> > -1000 to the oom_socre_adj of the processes outside of the protected
> > memcg.
>
> This sounds like a very dubious configuration to me. There is no other
> option than chosing from the protected group.
>

This is only an easy example to produce it.

> > Is this setting proper ?
> >
> > Thanks
> > Yafang
>
> --
> Michal Hocko
> SUSE Labs


^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2019-08-20  7:02 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-18  4:24 [PATCH] mm, memcg: skip killing processes under memcg protection at first scan Yafang Shao
2019-08-18 14:08 ` kbuild test robot
2019-08-18 14:08 ` kbuild test robot
2019-08-18 17:11 ` Souptick Joarder
2019-08-19  1:03   ` Yafang Shao
2019-08-19  7:31 ` Michal Hocko
2019-08-19  8:15   ` Yafang Shao
2019-08-20  6:31     ` Michal Hocko
2019-08-20  7:02       ` Yafang Shao

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).