[PATCH] mm: memcg: fix over reclaiming mem cgroup

[PATCH] mm: memcg: fix over reclaiming mem cgroup

[Hillf Danton]

In soft limit reclaim, overreclaim occurs when pages are reclaimed from mem
group that is under its soft limit, or when more pages are reclaimd than the
exceeding amount, then performance of reclaimee goes down accordingly.

A helper function is added to compute the number of pages that exceed the soft
limit of given mem cgroup, then the excess pages are used when every reclaimee
is reclaimed to avoid overreclaim.

Signed-off-by: Hillf Danton <dhillf@gmail.com>
---

--- a/mm/memcontrol.c	Tue Jan 17 20:41:36 2012
+++ b/mm/memcontrol.c	Sat Jan 21 21:18:46 2012
@@ -1662,6 +1662,21 @@ static int mem_cgroup_soft_reclaim(struc
 	return total;
 }

+unsigned long mem_cgroup_excess_pages(struct mem_cgroup *memcg)
+{
+	unsigned long pages;
+
+	if (mem_cgroup_disabled())
+		return 0;
+	if (!memcg)
+		return 0;
+	if (mem_cgroup_is_root(memcg))
+		return 0;
+
+	pages = res_counter_soft_limit_excess(&memcg->res) >> PAGE_SHIFT;
+	return pages;
+}
+
 /*
  * Check OOM-Killer is already running under our hierarchy.
  * If someone is running, return false.
--- a/mm/vmscan.c	Sat Jan 14 14:02:20 2012
+++ b/mm/vmscan.c	Sat Jan 21 21:30:06 2012
@@ -2150,8 +2150,34 @@ static void shrink_zone(int priority, st
 			.mem_cgroup = memcg,
 			.zone = zone,
 		};
+		unsigned long old;
+		bool clobbered = false;
+
+		if (memcg != NULL) {
+			unsigned long excess;
+
+			excess = mem_cgroup_excess_pages(memcg);
+			/*
+			 * No bother reclaiming pages from mem cgroup that
+			 * is under soft limit
+			 */
+			if (!excess)
+				goto next;
+			/*
+			 * And reclaim no more pages than excess
+			 */
+			if (excess < sc->nr_to_reclaim) {
+				old = sc->nr_to_reclaim;
+				sc->nr_to_reclaim = excess;
+				clobbered = true;
+			}
+		}

 		shrink_mem_cgroup_zone(priority, &mz, sc);
+
+		if (clobbered)
+			sc->nr_to_reclaim = old;
+next:
 		/*
 		 * Limit reclaim has historically picked one memcg and
 		 * scanned it with decreasing priority levels until
--- a/include/linux/memcontrol.h	Thu Jan 19 22:03:14 2012
+++ b/include/linux/memcontrol.h	Sat Jan 21 21:35:50 2012
@@ -161,6 +161,7 @@ unsigned long mem_cgroup_soft_limit_recl
 						gfp_t gfp_mask,
 						unsigned long *total_scanned);
 u64 mem_cgroup_get_limit(struct mem_cgroup *memcg);
+unsigned long mem_cgroup_excess_pages(struct mem_cgroup *memcg);

 void mem_cgroup_count_vm_event(struct mm_struct *mm, enum vm_event_item idx);
 #ifdef CONFIG_TRANSPARENT_HUGEPAGE
@@ -376,6 +377,11 @@ unsigned long mem_cgroup_soft_limit_recl

 static inline
 u64 mem_cgroup_get_limit(struct mem_cgroup *memcg)
+{
+	return 0;
+}
+
+static inline unsigned long mem_cgroup_excess_pages(struct mem_cgroup *memcg)
 {
 	return 0;
 }

Re: [PATCH] mm: memcg: fix over reclaiming mem cgroup

[Michal Hocko]

On Sat 21-01-12 22:49:23, Hillf Danton wrote:
> In soft limit reclaim, overreclaim occurs when pages are reclaimed from mem
> group that is under its soft limit, or when more pages are reclaimd than the
> exceeding amount, then performance of reclaimee goes down accordingly.

First of all soft reclaim is more a help for the global memory pressure
balancing rather than any guarantee about how much we reclaim for the
group.
We need to do more changes in order to make it a guarantee.
For example you implementation will cause severe problems when all
cgroups are soft unlimited (default conf.) or when nobody is above the
limit but the total consumption triggers the global reclaim. Therefore
nobody is in excess and you would skip all groups and only bang on the
root memcg.

Ying Han has a patch which basically skips all cgroups which are under
its limit until we reach a certain reclaim priority but even for this we
need some additional changes - e.g. reverse the current default setting
of the soft limit.

Anyway, I like the nr_to_reclaim reduction idea because we have to do
this in some way because the global reclaim starts with ULONG
nr_to_scan.

> A helper function is added to compute the number of pages that exceed the soft
> limit of given mem cgroup, then the excess pages are used when every reclaimee
> is reclaimed to avoid overreclaim.
> 
> Signed-off-by: Hillf Danton <dhillf@gmail.com>
> ---
> 
> --- a/mm/memcontrol.c	Tue Jan 17 20:41:36 2012
> +++ b/mm/memcontrol.c	Sat Jan 21 21:18:46 2012
> @@ -1662,6 +1662,21 @@ static int mem_cgroup_soft_reclaim(struc
>  	return total;
>  }
> 
> +unsigned long mem_cgroup_excess_pages(struct mem_cgroup *memcg)
> +{
> +	unsigned long pages;
> +
> +	if (mem_cgroup_disabled())
> +		return 0;
> +	if (!memcg)
> +		return 0;
> +	if (mem_cgroup_is_root(memcg))
> +		return 0;
> +
> +	pages = res_counter_soft_limit_excess(&memcg->res) >> PAGE_SHIFT;
> +	return pages;
> +}
> +
>  /*
>   * Check OOM-Killer is already running under our hierarchy.
>   * If someone is running, return false.
> --- a/mm/vmscan.c	Sat Jan 14 14:02:20 2012
> +++ b/mm/vmscan.c	Sat Jan 21 21:30:06 2012
> @@ -2150,8 +2150,34 @@ static void shrink_zone(int priority, st
>  			.mem_cgroup = memcg,
>  			.zone = zone,
>  		};
> +		unsigned long old;
> +		bool clobbered = false;
> +
> +		if (memcg != NULL) {
> +			unsigned long excess;
> +
> +			excess = mem_cgroup_excess_pages(memcg);
> +			/*
> +			 * No bother reclaiming pages from mem cgroup that
> +			 * is under soft limit
> +			 */
> +			if (!excess)
> +				goto next;
> +			/*
> +			 * And reclaim no more pages than excess
> +			 */
> +			if (excess < sc->nr_to_reclaim) {
> +				old = sc->nr_to_reclaim;
> +				sc->nr_to_reclaim = excess;
> +				clobbered = true;
> +			}
> +		}
> 
>  		shrink_mem_cgroup_zone(priority, &mz, sc);
> +
> +		if (clobbered)
> +			sc->nr_to_reclaim = old;
> +next:
>  		/*
>  		 * Limit reclaim has historically picked one memcg and
>  		 * scanned it with decreasing priority levels until
> --- a/include/linux/memcontrol.h	Thu Jan 19 22:03:14 2012
> +++ b/include/linux/memcontrol.h	Sat Jan 21 21:35:50 2012
> @@ -161,6 +161,7 @@ unsigned long mem_cgroup_soft_limit_recl
>  						gfp_t gfp_mask,
>  						unsigned long *total_scanned);
>  u64 mem_cgroup_get_limit(struct mem_cgroup *memcg);
> +unsigned long mem_cgroup_excess_pages(struct mem_cgroup *memcg);
> 
>  void mem_cgroup_count_vm_event(struct mm_struct *mm, enum vm_event_item idx);
>  #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> @@ -376,6 +377,11 @@ unsigned long mem_cgroup_soft_limit_recl
> 
>  static inline
>  u64 mem_cgroup_get_limit(struct mem_cgroup *memcg)
> +{
> +	return 0;
> +}
> +
> +static inline unsigned long mem_cgroup_excess_pages(struct mem_cgroup *memcg)
>  {
>  	return 0;
>  }
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
Michal Hocko
SUSE Labs
SUSE LINUX s.r.o.
Lihovarska 1060/12
190 00 Praha 9    
Czech Republic

Re: [PATCH] mm: memcg: fix over reclaiming mem cgroup

[Ying Han]

On Mon, Jan 23, 2012 at 5:02 AM, Michal Hocko <mhocko@suse.cz> wrote:
> On Sat 21-01-12 22:49:23, Hillf Danton wrote:
>> In soft limit reclaim, overreclaim occurs when pages are reclaimed from mem
>> group that is under its soft limit, or when more pages are reclaimd than the
>> exceeding amount, then performance of reclaimee goes down accordingly.
>
> First of all soft reclaim is more a help for the global memory pressure
> balancing rather than any guarantee about how much we reclaim for the
> group.
> We need to do more changes in order to make it a guarantee.
> For example you implementation will cause severe problems when all
> cgroups are soft unlimited (default conf.) or when nobody is above the
> limit but the total consumption triggers the global reclaim. Therefore
> nobody is in excess and you would skip all groups and only bang on the
> root memcg.
>
> Ying Han has a patch which basically skips all cgroups which are under
> its limit until we reach a certain reclaim priority but even for this we
> need some additional changes - e.g. reverse the current default setting
> of the soft limit.
>
> Anyway, I like the nr_to_reclaim reduction idea because we have to do
> this in some way because the global reclaim starts with ULONG
> nr_to_scan.

Agree with Michal where there are quite a lot changes we need to get
in for soft limit before any further optimization.

Hillf, please refer to the patch from Johannes
https://lkml.org/lkml/2012/1/13/99 which got quite a lot recent
discussions. I am expecting to get that in before further soft limit
changes.

Thanks

--Ying



>
>> A helper function is added to compute the number of pages that exceed the soft
>> limit of given mem cgroup, then the excess pages are used when every reclaimee
>> is reclaimed to avoid overreclaim.
>>
>> Signed-off-by: Hillf Danton <dhillf@gmail.com>
>> ---
>>
>> --- a/mm/memcontrol.c Tue Jan 17 20:41:36 2012
>> +++ b/mm/memcontrol.c Sat Jan 21 21:18:46 2012
>> @@ -1662,6 +1662,21 @@ static int mem_cgroup_soft_reclaim(struc
>>       return total;
>>  }
>>
>> +unsigned long mem_cgroup_excess_pages(struct mem_cgroup *memcg)
>> +{
>> +     unsigned long pages;
>> +
>> +     if (mem_cgroup_disabled())
>> +             return 0;
>> +     if (!memcg)
>> +             return 0;
>> +     if (mem_cgroup_is_root(memcg))
>> +             return 0;
>> +
>> +     pages = res_counter_soft_limit_excess(&memcg->res) >> PAGE_SHIFT;
>> +     return pages;
>> +}
>> +
>>  /*
>>   * Check OOM-Killer is already running under our hierarchy.
>>   * If someone is running, return false.
>> --- a/mm/vmscan.c     Sat Jan 14 14:02:20 2012
>> +++ b/mm/vmscan.c     Sat Jan 21 21:30:06 2012
>> @@ -2150,8 +2150,34 @@ static void shrink_zone(int priority, st
>>                       .mem_cgroup = memcg,
>>                       .zone = zone,
>>               };
>> +             unsigned long old;
>> +             bool clobbered = false;
>> +
>> +             if (memcg != NULL) {
>> +                     unsigned long excess;
>> +
>> +                     excess = mem_cgroup_excess_pages(memcg);
>> +                     /*
>> +                      * No bother reclaiming pages from mem cgroup that
>> +                      * is under soft limit
>> +                      */
>> +                     if (!excess)
>> +                             goto next;
>> +                     /*
>> +                      * And reclaim no more pages than excess
>> +                      */
>> +                     if (excess < sc->nr_to_reclaim) {
>> +                             old = sc->nr_to_reclaim;
>> +                             sc->nr_to_reclaim = excess;
>> +                             clobbered = true;
>> +                     }
>> +             }
>>
>>               shrink_mem_cgroup_zone(priority, &mz, sc);
>> +
>> +             if (clobbered)
>> +                     sc->nr_to_reclaim = old;
>> +next:
>>               /*
>>                * Limit reclaim has historically picked one memcg and
>>                * scanned it with decreasing priority levels until
>> --- a/include/linux/memcontrol.h      Thu Jan 19 22:03:14 2012
>> +++ b/include/linux/memcontrol.h      Sat Jan 21 21:35:50 2012
>> @@ -161,6 +161,7 @@ unsigned long mem_cgroup_soft_limit_recl
>>                                               gfp_t gfp_mask,
>>                                               unsigned long *total_scanned);
>>  u64 mem_cgroup_get_limit(struct mem_cgroup *memcg);
>> +unsigned long mem_cgroup_excess_pages(struct mem_cgroup *memcg);
>>
>>  void mem_cgroup_count_vm_event(struct mm_struct *mm, enum vm_event_item idx);
>>  #ifdef CONFIG_TRANSPARENT_HUGEPAGE
>> @@ -376,6 +377,11 @@ unsigned long mem_cgroup_soft_limit_recl
>>
>>  static inline
>>  u64 mem_cgroup_get_limit(struct mem_cgroup *memcg)
>> +{
>> +     return 0;
>> +}
>> +
>> +static inline unsigned long mem_cgroup_excess_pages(struct mem_cgroup *memcg)
>>  {
>>       return 0;
>>  }
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at  http://www.tux.org/lkml/
>
> --
> Michal Hocko
> SUSE Labs
> SUSE LINUX s.r.o.
> Lihovarska 1060/12
> 190 00 Praha 9
> Czech Republic

Re: [PATCH] mm: memcg: fix over reclaiming mem cgroup

[Hillf Danton]

Hi all

On Tue, Jan 24, 2012 at 3:14 AM, Ying Han <yinghan@google.com> wrote:
> On Mon, Jan 23, 2012 at 5:02 AM, Michal Hocko <mhocko@suse.cz> wrote:
>> On Sat 21-01-12 22:49:23, Hillf Danton wrote:
>>> In soft limit reclaim, overreclaim occurs when pages are reclaimed from mem
>>> group that is under its soft limit, or when more pages are reclaimd than the
>>> exceeding amount, then performance of reclaimee goes down accordingly.
>>
>> First of all soft reclaim is more a help for the global memory pressure
>> balancing rather than any guarantee about how much we reclaim for the
>> group.
>> We need to do more changes in order to make it a guarantee.
>> For example you implementation will cause severe problems when all
>> cgroups are soft unlimited (default conf.) or when nobody is above the
>> limit but the total consumption triggers the global reclaim. Therefore
>> nobody is in excess and you would skip all groups and only bang on the
>> root memcg.

If soft limits are set to be limited and there are no excessors,
who are consuming physical pages? The consumers maybe those with soft
unlimited. If so, they should be punished first, based on the assumption that
the unlimited is treated with no guarantee. Then soft limit guarantee could
be assured without changes in the current default setting of soft limit, no?

With soft limit available, victims are only selected from excessors, I think.

>>
>> Ying Han has a patch which basically skips all cgroups which are under
>> its limit until we reach a certain reclaim priority but even for this we
>> need some additional changes - e.g. reverse the current default setting
>> of the soft limit.
>>
>> Anyway, I like the nr_to_reclaim reduction idea because we have to do
>> this in some way because the global reclaim starts with ULONG
>> nr_to_scan.
>
> Agree with Michal where there are quite a lot changes we need to get
> in for soft limit before any further optimization.
>
> Hillf, please refer to the patch from Johannes
> https://lkml.org/lkml/2012/1/13/99 which got quite a lot recent
> discussions. I am expecting to get that in before further soft limit
> changes.
>

Johannes did great cleanup, why barriered?

Thanks
Hillf

Re: [PATCH] mm: memcg: fix over reclaiming mem cgroup

[Balbir Singh]

On Tue, Jan 24, 2012 at 12:44 AM, Ying Han <yinghan@google.com> wrote:
> On Mon, Jan 23, 2012 at 5:02 AM, Michal Hocko <mhocko@suse.cz> wrote:
>> On Sat 21-01-12 22:49:23, Hillf Danton wrote:
>>> In soft limit reclaim, overreclaim occurs when pages are reclaimed from mem
>>> group that is under its soft limit, or when more pages are reclaimd than the
>>> exceeding amount, then performance of reclaimee goes down accordingly.
>>
>> First of all soft reclaim is more a help for the global memory pressure
>> balancing rather than any guarantee about how much we reclaim for the
>> group.
>> We need to do more changes in order to make it a guarantee.
>> For example you implementation will cause severe problems when all
>> cgroups are soft unlimited (default conf.) or when nobody is above the
>> limit but the total consumption triggers the global reclaim. Therefore
>> nobody is in excess and you would skip all groups and only bang on the
>> root memcg.
>>

True, ideally soft reclaim should not turn on and allow global reclaim
to occur in the scenario mentioned.

>> Ying Han has a patch which basically skips all cgroups which are under
>> its limit until we reach a certain reclaim priority but even for this we
>> need some additional changes - e.g. reverse the current default setting
>> of the soft limit.
>>

I'd be wary of that approach, because it might be harder to explain
the working of soft limits,I'll look at the discussion thread
mentioned earlier for the benefits of that approach.

>> Anyway, I like the nr_to_reclaim reduction idea because we have to do
>> this in some way because the global reclaim starts with ULONG
>> nr_to_scan.
>
> Agree with Michal where there are quite a lot changes we need to get
> in for soft limit before any further optimization.
>
> Hillf, please refer to the patch from Johannes
> https://lkml.org/lkml/2012/1/13/99 which got quite a lot recent
> discussions. I am expecting to get that in before further soft limit
> changes.

Balbir

Re: [PATCH] mm: memcg: fix over reclaiming mem cgroup

[Michal Hocko]

On Tue 24-01-12 11:26:05, Hillf Danton wrote:
> Hi all
> 
> On Tue, Jan 24, 2012 at 3:14 AM, Ying Han <yinghan@google.com> wrote:
> > On Mon, Jan 23, 2012 at 5:02 AM, Michal Hocko <mhocko@suse.cz> wrote:
> >> On Sat 21-01-12 22:49:23, Hillf Danton wrote:
> >>> In soft limit reclaim, overreclaim occurs when pages are reclaimed from mem
> >>> group that is under its soft limit, or when more pages are reclaimd than the
> >>> exceeding amount, then performance of reclaimee goes down accordingly.
> >>
> >> First of all soft reclaim is more a help for the global memory pressure
> >> balancing rather than any guarantee about how much we reclaim for the
> >> group.
> >> We need to do more changes in order to make it a guarantee.
> >> For example you implementation will cause severe problems when all
> >> cgroups are soft unlimited (default conf.) or when nobody is above the
> >> limit but the total consumption triggers the global reclaim. Therefore
> >> nobody is in excess and you would skip all groups and only bang on the
> >> root memcg.
> 
> If soft limits are set to be limited and there are no excessors, who
> are consuming physical pages? The consumers maybe those with soft
> unlimited.

You might have many small groups which are all under their soft limit
but their total usage triggers global reclaim for example.

> If so, they should be punished first, based on the assumption that the
> unlimited is treated with no guarantee.
> Then soft limit guarantee could be assured without changes in the
> current default setting of soft limit, no?

How would you prioritize between over soft limit groups (which should
give at least some protection from over reclaim AFAIU from your
suggestion) from those that are unlimited?

Also, when we start talking about guarantees, why would somebody who
is soft unlimited be punished at all? Soft unlimited basically means
soft_limit >= hard_limit which says please do not reclaim from me unless
really really really necessary.

> With soft limit available, victims are only selected from excessors, I think.
> 
> >>
> >> Ying Han has a patch which basically skips all cgroups which are under
> >> its limit until we reach a certain reclaim priority but even for this we
> >> need some additional changes - e.g. reverse the current default setting
> >> of the soft limit.
> >>
> >> Anyway, I like the nr_to_reclaim reduction idea because we have to do
> >> this in some way because the global reclaim starts with ULONG
> >> nr_to_scan.
> >
> > Agree with Michal where there are quite a lot changes we need to get
> > in for soft limit before any further optimization.
> >
> > Hillf, please refer to the patch from Johannes
> > https://lkml.org/lkml/2012/1/13/99 which got quite a lot recent
> > discussions. I am expecting to get that in before further soft limit
> > changes.
> >
> 
> Johannes did great cleanup, why barriered?

Barriered?

-- 
Michal Hocko
SUSE Labs
SUSE LINUX s.r.o.
Lihovarska 1060/12
190 00 Praha 9    
Czech Republic

Re: [PATCH] mm: memcg: fix over reclaiming mem cgroup

[Michal Hocko]

On Tue 24-01-12 09:18:21, Balbir Singh wrote:
> On Tue, Jan 24, 2012 at 12:44 AM, Ying Han <yinghan@google.com> wrote:
> > On Mon, Jan 23, 2012 at 5:02 AM, Michal Hocko <mhocko@suse.cz> wrote:
> >> On Sat 21-01-12 22:49:23, Hillf Danton wrote:
> >>> In soft limit reclaim, overreclaim occurs when pages are reclaimed from mem
> >>> group that is under its soft limit, or when more pages are reclaimd than the
> >>> exceeding amount, then performance of reclaimee goes down accordingly.
> >>
> >> First of all soft reclaim is more a help for the global memory pressure
> >> balancing rather than any guarantee about how much we reclaim for the
> >> group.
> >> We need to do more changes in order to make it a guarantee.
> >> For example you implementation will cause severe problems when all
> >> cgroups are soft unlimited (default conf.) or when nobody is above the
> >> limit but the total consumption triggers the global reclaim. Therefore
> >> nobody is in excess and you would skip all groups and only bang on the
> >> root memcg.
> >>
> 
> True, ideally soft reclaim should not turn on and allow global reclaim
> to occur in the scenario mentioned.
> 
> >> Ying Han has a patch which basically skips all cgroups which are under
> >> its limit until we reach a certain reclaim priority but even for this we
> >> need some additional changes - e.g. reverse the current default setting
> >> of the soft limit.
> >>
> 
> I'd be wary of that approach, because it might be harder to explain
> the working of soft limits,

This is an attempt to turn the soft reclaim into a "guarantee". Changing
the default value from unlimited to 0 basically says that everybody will
be considered under memory pressure unless the soft limit setting says
otherwise.
This btw. has been the case with the double (global and per-cgroup) LRUs
as well. It was just hidden.

> I'll look at the discussion thread mentioned earlier for the benefits
> of that approach.
> 
> >> Anyway, I like the nr_to_reclaim reduction idea because we have to do
> >> this in some way because the global reclaim starts with ULONG
> >> nr_to_scan.
> >
> > Agree with Michal where there are quite a lot changes we need to get
> > in for soft limit before any further optimization.
> >
> > Hillf, please refer to the patch from Johannes
> > https://lkml.org/lkml/2012/1/13/99 which got quite a lot recent
> > discussions. I am expecting to get that in before further soft limit
> > changes.
> 
> Balbir
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
Michal Hocko
SUSE Labs
SUSE LINUX s.r.o.
Lihovarska 1060/12
190 00 Praha 9    
Czech Republic

Re: [PATCH] mm: memcg: fix over reclaiming mem cgroup

[Ying Han]

On Mon, Jan 23, 2012 at 7:26 PM, Hillf Danton <dhillf@gmail.com> wrote:
> Hi all
>
> On Tue, Jan 24, 2012 at 3:14 AM, Ying Han <yinghan@google.com> wrote:
>> On Mon, Jan 23, 2012 at 5:02 AM, Michal Hocko <mhocko@suse.cz> wrote:
>>> On Sat 21-01-12 22:49:23, Hillf Danton wrote:
>>>> In soft limit reclaim, overreclaim occurs when pages are reclaimed from mem
>>>> group that is under its soft limit, or when more pages are reclaimd than the
>>>> exceeding amount, then performance of reclaimee goes down accordingly.
>>>
>>> First of all soft reclaim is more a help for the global memory pressure
>>> balancing rather than any guarantee about how much we reclaim for the
>>> group.
>>> We need to do more changes in order to make it a guarantee.
>>> For example you implementation will cause severe problems when all
>>> cgroups are soft unlimited (default conf.) or when nobody is above the
>>> limit but the total consumption triggers the global reclaim. Therefore
>>> nobody is in excess and you would skip all groups and only bang on the
>>> root memcg.
>
> If soft limits are set to be limited and there are no excessors,
> who are consuming physical pages? The consumers maybe those with soft
> unlimited. If so, they should be punished first, based on the assumption that
> the unlimited is treated with no guarantee. Then soft limit guarantee could
> be assured without changes in the current default setting of soft limit, no?

The current code set the softlimit default to be RESOURCE_MAX, but we
do have plan to change it to 0. Then every cgroup will be eligible for
soft limit reclaim unless we set it otherwise. Not sure if that will
answer some of the questions?

>
> With soft limit available, victims are only selected from excessors, I think.
>
>>>
>>> Ying Han has a patch which basically skips all cgroups which are under
>>> its limit until we reach a certain reclaim priority but even for this we
>>> need some additional changes - e.g. reverse the current default setting
>>> of the soft limit.
>>>
>>> Anyway, I like the nr_to_reclaim reduction idea because we have to do
>>> this in some way because the global reclaim starts with ULONG
>>> nr_to_scan.
>>
>> Agree with Michal where there are quite a lot changes we need to get
>> in for soft limit before any further optimization.
>>
>> Hillf, please refer to the patch from Johannes
>> https://lkml.org/lkml/2012/1/13/99 which got quite a lot recent
>> discussions. I am expecting to get that in before further soft limit
>> changes.
>>
>
> Johannes did great cleanup, why barriered?

Not sure about the barriered here. As far as I can tell, some changes
we are talking about here would be easier to be applied after that
cleanup.

--Ying
>
> Thanks
> Hillf

Re: [PATCH] mm: memcg: fix over reclaiming mem cgroup

[Hillf Danton]

On Tue, Jan 24, 2012 at 4:23 PM, Michal Hocko <mhocko@suse.cz> wrote:
> Barriered?
>
pushed out for 3.3-rc2 last night?

Hillf

Re: [PATCH] mm: memcg: fix over reclaiming mem cgroup

[Johannes Weiner]

On Wed, Jan 25, 2012 at 09:55:01AM +0800, Hillf Danton wrote:
> On Tue, Jan 24, 2012 at 4:23 PM, Michal Hocko <mhocko@suse.cz> wrote:
> > Barriered?
> >
> pushed out for 3.3-rc2 last night?

New features are only merged during the merge window (between 3.2 and
3.3-rc1), from then on it's only bugfixes to stabilize for release.

My soft limit patch missed this merge window, so the earliest target
is 3.4.  And that's fine, there are still things that need to be
evaluated, like kswapd now reclaiming with priority 0 and
nr_to_reclaim at ULONG_MAX, which Michal pointed out.  Or KAME's
concerns regarding direct soft reclaim and numa setups.