All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] mm/vmscan: make do_shrink_slab more robust.
@ 2017-11-27  1:37 ` Jiang Biao
  0 siblings, 0 replies; 14+ messages in thread
From: Jiang Biao @ 2017-11-27  1:37 UTC (permalink / raw)
  To: akpm, mhocko, hannes, hillf.zj, minchan, ying.huang, mgorman
  Cc: linux-mm, linux-kernel, jiang.biao2, zhong.weidong

When running ltp stress test for 7*24 hours, the kernel occasionally
complains the following warning continuously,

mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
nr=-9222526086287711848
mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
nr=-9222420761333860545
mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
nr=-9222287677544280360
...

The tracing result shows the freeable(mb_cache_shrink_scan returns)
is -1, which causes the continuous accumulation and overflow of
total_scan.

This patch make do_shrink_slab more robust when
shrinker->count_objects return negative freeable.

Signed-off-by: Jiang Biao <jiang.biao2@zte.com.cn>
---
 mm/vmscan.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index eb2f031..3ea28f0 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -323,7 +323,7 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl,
 	long scanned = 0, next_deferred;
 
 	freeable = shrinker->count_objects(shrinker, shrinkctl);
-	if (freeable == 0)
+	if (freeable <= 0)
 		return 0;
 
 	/*
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH] mm/vmscan: make do_shrink_slab more robust.
@ 2017-11-27  1:37 ` Jiang Biao
  0 siblings, 0 replies; 14+ messages in thread
From: Jiang Biao @ 2017-11-27  1:37 UTC (permalink / raw)
  To: akpm, mhocko, hannes, hillf.zj, minchan, ying.huang, mgorman
  Cc: linux-mm, linux-kernel, jiang.biao2, zhong.weidong

When running ltp stress test for 7*24 hours, the kernel occasionally
complains the following warning continuously,

mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
nr=-9222526086287711848
mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
nr=-9222420761333860545
mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
nr=-9222287677544280360
...

The tracing result shows the freeable(mb_cache_shrink_scan returns)
is -1, which causes the continuous accumulation and overflow of
total_scan.

This patch make do_shrink_slab more robust when
shrinker->count_objects return negative freeable.

Signed-off-by: Jiang Biao <jiang.biao2@zte.com.cn>
---
 mm/vmscan.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index eb2f031..3ea28f0 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -323,7 +323,7 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl,
 	long scanned = 0, next_deferred;
 
 	freeable = shrinker->count_objects(shrinker, shrinkctl);
-	if (freeable == 0)
+	if (freeable <= 0)
 		return 0;
 
 	/*
-- 
2.7.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.
  2017-11-27  1:37 ` Jiang Biao
@ 2017-11-27  2:39   ` Minchan Kim
  -1 siblings, 0 replies; 14+ messages in thread
From: Minchan Kim @ 2017-11-27  2:39 UTC (permalink / raw)
  To: Jiang Biao
  Cc: akpm, mhocko, hannes, hillf.zj, ying.huang, mgorman, linux-mm,
	linux-kernel, zhong.weidong

Hello,

On Mon, Nov 27, 2017 at 09:37:30AM +0800, Jiang Biao wrote:
> When running ltp stress test for 7*24 hours, the kernel occasionally
> complains the following warning continuously,
> 
> mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
> nr=-9222526086287711848
> mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
> nr=-9222420761333860545
> mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
> nr=-9222287677544280360
> ...
> 
> The tracing result shows the freeable(mb_cache_shrink_scan returns)
> is -1, which causes the continuous accumulation and overflow of
> total_scan.

Good catch.

> 
> This patch make do_shrink_slab more robust when
> shrinker->count_objects return negative freeable.

Shrinker.h says count_objects should return 0 if there are no
freeable objects, not -1.

So if something returns -1, changing it with returning 0 would
be more proper fix.

Thanks.


> 
> Signed-off-by: Jiang Biao <jiang.biao2@zte.com.cn>
> ---
>  mm/vmscan.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index eb2f031..3ea28f0 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -323,7 +323,7 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl,
>  	long scanned = 0, next_deferred;
>  
>  	freeable = shrinker->count_objects(shrinker, shrinkctl);
> -	if (freeable == 0)
> +	if (freeable <= 0)
>  		return 0;
>  
>  	/*
> -- 
> 2.7.4
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.
@ 2017-11-27  2:39   ` Minchan Kim
  0 siblings, 0 replies; 14+ messages in thread
From: Minchan Kim @ 2017-11-27  2:39 UTC (permalink / raw)
  To: Jiang Biao
  Cc: akpm, mhocko, hannes, hillf.zj, ying.huang, mgorman, linux-mm,
	linux-kernel, zhong.weidong

Hello,

On Mon, Nov 27, 2017 at 09:37:30AM +0800, Jiang Biao wrote:
> When running ltp stress test for 7*24 hours, the kernel occasionally
> complains the following warning continuously,
> 
> mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
> nr=-9222526086287711848
> mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
> nr=-9222420761333860545
> mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
> nr=-9222287677544280360
> ...
> 
> The tracing result shows the freeable(mb_cache_shrink_scan returns)
> is -1, which causes the continuous accumulation and overflow of
> total_scan.

Good catch.

> 
> This patch make do_shrink_slab more robust when
> shrinker->count_objects return negative freeable.

Shrinker.h says count_objects should return 0 if there are no
freeable objects, not -1.

So if something returns -1, changing it with returning 0 would
be more proper fix.

Thanks.


> 
> Signed-off-by: Jiang Biao <jiang.biao2@zte.com.cn>
> ---
>  mm/vmscan.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index eb2f031..3ea28f0 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -323,7 +323,7 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl,
>  	long scanned = 0, next_deferred;
>  
>  	freeable = shrinker->count_objects(shrinker, shrinkctl);
> -	if (freeable == 0)
> +	if (freeable <= 0)
>  		return 0;
>  
>  	/*
> -- 
> 2.7.4
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.
  2017-11-27  2:39   ` Minchan Kim
  (?)
@ 2017-11-27  4:46   ` jiang.biao2
  2017-11-27  5:16       ` Minchan Kim
  -1 siblings, 1 reply; 14+ messages in thread
From: jiang.biao2 @ 2017-11-27  4:46 UTC (permalink / raw)
  To: minchan
  Cc: akpm, mhocko, hannes, hillf.zj, ying.huang, linux-mm, mgorman,
	linux-kernel, zhong.weidong


[-- Attachment #1.1: Type: text/plain, Size: 666 bytes --]

On Mon, Nov 27, 2017 at 09:37:30AM +0800, Jiang Biao wrote:> >
> > This patch make do_shrink_slab more robust when
> > shrinker->count_objects return negative freeable.
> 
> Shrinker.h says count_objects should return 0 if there are no
> freeable objects, not -1.
> 
> So if something returns -1, changing it with returning 0 would
> be more proper fix.
> 
Hi,
Indeed it's not a bug of vmscan, but there are many shrinkers
out there, which may return negative value unwillingly(in some 
rare cases, such as decreasing cocurrently). It's unlikely and 
should be avioded, but not impossible, this patch may make it 
more robust and could not hurt :).
Thanks.

Regards,

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.
  2017-11-27  4:46   ` jiang.biao2
@ 2017-11-27  5:16       ` Minchan Kim
  0 siblings, 0 replies; 14+ messages in thread
From: Minchan Kim @ 2017-11-27  5:16 UTC (permalink / raw)
  To: jiang.biao2
  Cc: akpm, mhocko, hannes, hillf.zj, ying.huang, linux-mm, mgorman,
	linux-kernel, zhong.weidong

On Mon, Nov 27, 2017 at 12:46:27PM +0800, jiang.biao2@zte.com.cn wrote:
> On Mon, Nov 27, 2017 at 09:37:30AM +0800, Jiang Biao wrote:> >
> > > This patch make do_shrink_slab more robust when
> > > shrinker->count_objects return negative freeable.
> > 
> > Shrinker.h says count_objects should return 0 if there are no
> > freeable objects, not -1.
> > 
> > So if something returns -1, changing it with returning 0 would
> > be more proper fix.
> > 
> Hi,
> Indeed it's not a bug of vmscan, but there are many shrinkers
> out there, which may return negative value unwillingly(in some 
> rare cases, such as decreasing cocurrently). It's unlikely and 
> should be avioded, but not impossible, this patch may make it 
> more robust and could not hurt :).

Yub, I'm not strong against of your claim. However, let's think
from different point of view.

API says it should return 0 unless shrinker cannot find freeable
object any more but with your change, implmentation handles
although a shrinker return minus value by mistake.

In future, MM guys might want to extend count_objects returning
-ERR_SOMETHING to propagate error, for example but we cannot.
Because some of shrinkers already rely on the implementation so
if we start to support minus value return, some of shrinker might
be broken.

Yes, it's the imaginary scenario but wanted why such changes
makes us trouble in future, API PoV.

Other way is you can change the description so that count_scan
API can return any value if it's less or equal to zero but not
sure it's worth. Anyway, maintainer will judge but my opinion
is not worth to do at this moment. We have been happy for a
long time without that.

Thanks.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.
@ 2017-11-27  5:16       ` Minchan Kim
  0 siblings, 0 replies; 14+ messages in thread
From: Minchan Kim @ 2017-11-27  5:16 UTC (permalink / raw)
  To: jiang.biao2
  Cc: akpm, mhocko, hannes, hillf.zj, ying.huang, linux-mm, mgorman,
	linux-kernel, zhong.weidong

On Mon, Nov 27, 2017 at 12:46:27PM +0800, jiang.biao2@zte.com.cn wrote:
> On Mon, Nov 27, 2017 at 09:37:30AM +0800, Jiang Biao wrote:> >
> > > This patch make do_shrink_slab more robust when
> > > shrinker->count_objects return negative freeable.
> > 
> > Shrinker.h says count_objects should return 0 if there are no
> > freeable objects, not -1.
> > 
> > So if something returns -1, changing it with returning 0 would
> > be more proper fix.
> > 
> Hi,
> Indeed it's not a bug of vmscan, but there are many shrinkers
> out there, which may return negative value unwillingly(in some 
> rare cases, such as decreasing cocurrently). It's unlikely and 
> should be avioded, but not impossible, this patch may make it 
> more robust and could not hurt :).

Yub, I'm not strong against of your claim. However, let's think
from different point of view.

API says it should return 0 unless shrinker cannot find freeable
object any more but with your change, implmentation handles
although a shrinker return minus value by mistake.

In future, MM guys might want to extend count_objects returning
-ERR_SOMETHING to propagate error, for example but we cannot.
Because some of shrinkers already rely on the implementation so
if we start to support minus value return, some of shrinker might
be broken.

Yes, it's the imaginary scenario but wanted why such changes
makes us trouble in future, API PoV.

Other way is you can change the description so that count_scan
API can return any value if it's less or equal to zero but not
sure it's worth. Anyway, maintainer will judge but my opinion
is not worth to do at this moment. We have been happy for a
long time without that.

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.
  2017-11-27  5:16       ` Minchan Kim
  (?)
@ 2017-11-27  6:27       ` jiang.biao2
  2017-11-27  6:38           ` Minchan Kim
  -1 siblings, 1 reply; 14+ messages in thread
From: jiang.biao2 @ 2017-11-27  6:27 UTC (permalink / raw)
  To: minchan
  Cc: akpm, mhocko, hannes, hillf.zj, ying.huang, linux-mm, mgorman,
	linux-kernel, zhong.weidong


[-- Attachment #1.1: Type: text/plain, Size: 2048 bytes --]

> On Mon, Nov 27, 2017 at 12:46:27PM +0800, jiang.biao2@zte.com.cn wrote:> > On Mon, Nov 27, 2017 at 09:37:30AM +0800, Jiang Biao wrote:> >
> > > > This patch make do_shrink_slab more robust when
> > > > shrinker->count_objects return negative freeable.
> > >
> > > Shrinker.h says count_objects should return 0 if there are no
> > > freeable objects, not -1.
> > >
> > > So if something returns -1, changing it with returning 0 would
> > be more proper fix.
> > >
> > Hi,
> > Indeed it's not a bug of vmscan, but there are many shrinkers
> > out there, which may return negative value unwillingly(in some
> > rare cases, such as decreasing cocurrently). It's unlikely and
> > should be avioded, but not impossible, this patch may make it
> > more robust and could not hurt :).
> 
> Yub, I'm not strong against of your claim. However, let's think
> from different point of view.
> 
> API says it should return 0 unless shrinker cannot find freeable
> object any more but with your change, implmentation handles
> although a shrinker return minus value by mistake.
> 
> In future, MM guys might want to extend count_objects returning
> -ERR_SOMETHING to propagate error, for example but we cannot.
> Because some of shrinkers already rely on the implementation so
> if we start to support minus value return, some of shrinker might
> be broken.
> 
> Yes, it's the imaginary scenario but wanted why such changes
> makes us trouble in future, API PoV.
I agree with your concern.  How about we take another way by 
adding some warning in such case? such as,
        freeable = shrinker->count_objects(shrinker, shrinkctl);
+       if (unlikely(freeable < 0)) {
+               pr_err("shrink_slab: %pF negative objects returned. freeable=%ld\n",
+                       shrinker->scan_objects, freeable);
+               freeable = 0;  //maybe not needed?
+       }
        if (freeable == 0)
                return 0;
In this way, we would not break the API, but could alert user exception 
with message, and make it more robust in such case.

Thanks.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.
  2017-11-27  6:27       ` jiang.biao2
@ 2017-11-27  6:38           ` Minchan Kim
  0 siblings, 0 replies; 14+ messages in thread
From: Minchan Kim @ 2017-11-27  6:38 UTC (permalink / raw)
  To: jiang.biao2
  Cc: akpm, mhocko, hannes, hillf.zj, ying.huang, linux-mm, mgorman,
	linux-kernel, zhong.weidong

On Mon, Nov 27, 2017 at 02:27:20PM +0800, jiang.biao2@zte.com.cn wrote:
> > On Mon, Nov 27, 2017 at 12:46:27PM +0800, jiang.biao2@zte.com.cn wrote:> > On Mon, Nov 27, 2017 at 09:37:30AM +0800, Jiang Biao wrote:> >
> > > > > This patch make do_shrink_slab more robust when
> > > > > shrinker->count_objects return negative freeable.
> > > >
> > > > Shrinker.h says count_objects should return 0 if there are no
> > > > freeable objects, not -1.
> > > >
> > > > So if something returns -1, changing it with returning 0 would
> > > be more proper fix.
> > > >
> > > Hi,
> > > Indeed it's not a bug of vmscan, but there are many shrinkers
> > > out there, which may return negative value unwillingly(in some
> > > rare cases, such as decreasing cocurrently). It's unlikely and
> > > should be avioded, but not impossible, this patch may make it
> > > more robust and could not hurt :).
> > 
> > Yub, I'm not strong against of your claim. However, let's think
> > from different point of view.
> > 
> > API says it should return 0 unless shrinker cannot find freeable
> > object any more but with your change, implmentation handles
> > although a shrinker return minus value by mistake.
> > 
> > In future, MM guys might want to extend count_objects returning
> > -ERR_SOMETHING to propagate error, for example but we cannot.
> > Because some of shrinkers already rely on the implementation so
> > if we start to support minus value return, some of shrinker might
> > be broken.
> > 
> > Yes, it's the imaginary scenario but wanted why such changes
> > makes us trouble in future, API PoV.
> I agree with your concern.  How about we take another way by 
> adding some warning in such case? such as,
>         freeable = shrinker->count_objects(shrinker, shrinkctl);
> +       if (unlikely(freeable < 0)) {
> +               pr_err("shrink_slab: %pF negative objects returned. freeable=%ld\n",
> +                       shrinker->scan_objects, freeable);
> +               freeable = 0;  //maybe not needed?
> +       }
>         if (freeable == 0)
>                 return 0;
> In this way, we would not break the API, but could alert user exception 
> with message, and make it more robust in such case.

True but it would be a problem robust vs. effectivess tradeoff.
Think about that everyone want to make thier code robust.
It means they start to dump lots of defensive code so code start
to look like complicated as well as binary bloating.
So, whenever we add some more, we should think how effective
the code I am putting?

In this case, I'm skeptical, Sorry. But others might have different
opinions. :)

Thanks.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.
@ 2017-11-27  6:38           ` Minchan Kim
  0 siblings, 0 replies; 14+ messages in thread
From: Minchan Kim @ 2017-11-27  6:38 UTC (permalink / raw)
  To: jiang.biao2
  Cc: akpm, mhocko, hannes, hillf.zj, ying.huang, linux-mm, mgorman,
	linux-kernel, zhong.weidong

On Mon, Nov 27, 2017 at 02:27:20PM +0800, jiang.biao2@zte.com.cn wrote:
> > On Mon, Nov 27, 2017 at 12:46:27PM +0800, jiang.biao2@zte.com.cn wrote:> > On Mon, Nov 27, 2017 at 09:37:30AM +0800, Jiang Biao wrote:> >
> > > > > This patch make do_shrink_slab more robust when
> > > > > shrinker->count_objects return negative freeable.
> > > >
> > > > Shrinker.h says count_objects should return 0 if there are no
> > > > freeable objects, not -1.
> > > >
> > > > So if something returns -1, changing it with returning 0 would
> > > be more proper fix.
> > > >
> > > Hi,
> > > Indeed it's not a bug of vmscan, but there are many shrinkers
> > > out there, which may return negative value unwillingly(in some
> > > rare cases, such as decreasing cocurrently). It's unlikely and
> > > should be avioded, but not impossible, this patch may make it
> > > more robust and could not hurt :).
> > 
> > Yub, I'm not strong against of your claim. However, let's think
> > from different point of view.
> > 
> > API says it should return 0 unless shrinker cannot find freeable
> > object any more but with your change, implmentation handles
> > although a shrinker return minus value by mistake.
> > 
> > In future, MM guys might want to extend count_objects returning
> > -ERR_SOMETHING to propagate error, for example but we cannot.
> > Because some of shrinkers already rely on the implementation so
> > if we start to support minus value return, some of shrinker might
> > be broken.
> > 
> > Yes, it's the imaginary scenario but wanted why such changes
> > makes us trouble in future, API PoV.
> I agree with your concern.  How about we take another way by 
> adding some warning in such case? such as,
>         freeable = shrinker->count_objects(shrinker, shrinkctl);
> +       if (unlikely(freeable < 0)) {
> +               pr_err("shrink_slab: %pF negative objects returned. freeable=%ld\n",
> +                       shrinker->scan_objects, freeable);
> +               freeable = 0;  //maybe not needed?
> +       }
>         if (freeable == 0)
>                 return 0;
> In this way, we would not break the API, but could alert user exception 
> with message, and make it more robust in such case.

True but it would be a problem robust vs. effectivess tradeoff.
Think about that everyone want to make thier code robust.
It means they start to dump lots of defensive code so code start
to look like complicated as well as binary bloating.
So, whenever we add some more, we should think how effective
the code I am putting?

In this case, I'm skeptical, Sorry. But others might have different
opinions. :)

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.
  2017-11-27  6:38           ` Minchan Kim
  (?)
@ 2017-11-27  7:26           ` jiang.biao2
  2017-11-27  8:27               ` Michal Hocko
  -1 siblings, 1 reply; 14+ messages in thread
From: jiang.biao2 @ 2017-11-27  7:26 UTC (permalink / raw)
  To: minchan
  Cc: akpm, mhocko, hannes, hillf.zj, ying.huang, linux-mm, mgorman,
	linux-kernel, zhong.weidong


[-- Attachment #1.1: Type: text/plain, Size: 1680 bytes --]

On Mon, Nov 27, 2017 at 02:27:20PM +0800, jiang.biao2@zte.com.cn wrote:> > I agree with your concern.  How about we take another way by
> > adding some warning in such case? such as,
> >         freeable = shrinker->count_objects(shrinker, shrinkctl);
> > +       if (unlikely(freeable < 0)) {
> > +               pr_err("shrink_slab: %pF negative objects returned. freeable=%ld\n",
> > +                       shrinker->scan_objects, freeable);
> > +               freeable = 0;  //maybe not needed?
> > +       }
> >         if (freeable == 0)
> >                 return 0;
> > In this way, we would not break the API, but could alert user exception
> > with message, and make it more robust in such case.
>
> True but it would be a problem robust vs. effectivess tradeoff.
> Think about that everyone want to make thier code robust.
> It means they start to dump lots of defensive code so code start
> to look like complicated as well as binary bloating.
> So, whenever we add some more, we should think how effective
> the code I am putting?
> 
> In this case, I'm skeptical, Sorry. But others might have different
> opinions. :)

With all due respect. I still think the robustness is more important than 
effectiveness in this case. :)
First, the minus value could break all the following procedure in 
do_shrink_slab which may lead to long time looping on 
*tight on memory* situations.
Second, there is *unlikely* macro to minimize the cost.
Third, there is already similar procedure for *total_scan < 0*.
Last, it did happen in my scenario and maybe happen in other 
unexpected situations.

Just my personal option, thanks a lot for you reply and explanation. :)

Regards,

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.
  2017-11-27  7:26           ` jiang.biao2
@ 2017-11-27  8:27               ` Michal Hocko
  0 siblings, 0 replies; 14+ messages in thread
From: Michal Hocko @ 2017-11-27  8:27 UTC (permalink / raw)
  To: jiang.biao2
  Cc: minchan, akpm, hannes, hillf.zj, ying.huang, linux-mm, mgorman,
	linux-kernel, zhong.weidong

On Mon 27-11-17 15:26:54, jiang.biao2@zte.com.cn wrote:
> On Mon, Nov 27, 2017 at 02:27:20PM +0800, jiang.biao2@zte.com.cn wrote:> > I agree with your concern.  How about we take another way by
> > > adding some warning in such case? such as,
> > >         freeable = shrinker->count_objects(shrinker, shrinkctl);
> > > +       if (unlikely(freeable < 0)) {
> > > +               pr_err("shrink_slab: %pF negative objects returned. freeable=%ld\n",
> > > +                       shrinker->scan_objects, freeable);
> > > +               freeable = 0;  //maybe not needed?
> > > +       }
> > >         if (freeable == 0)
> > >                 return 0;
> > > In this way, we would not break the API, but could alert user exception
> > > with message, and make it more robust in such case.
> >
> > True but it would be a problem robust vs. effectivess tradeoff.
> > Think about that everyone want to make thier code robust.
> > It means they start to dump lots of defensive code so code start
> > to look like complicated as well as binary bloating.
> > So, whenever we add some more, we should think how effective
> > the code I am putting?
> > 
> > In this case, I'm skeptical, Sorry. But others might have different
> > opinions. :)
> 
> With all due respect. I still think the robustness is more important than 
> effectiveness in this case. :)

This is a slow path so I wouldn't worry about the performance much. On
the other hand I agree that the API is well documented so adding a
warning is too defensive. We simply assume that the kernel running in
the kernel is reasonable. So I would say, fix your code.

-- 
Michal Hocko
SUSE Labs

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.
@ 2017-11-27  8:27               ` Michal Hocko
  0 siblings, 0 replies; 14+ messages in thread
From: Michal Hocko @ 2017-11-27  8:27 UTC (permalink / raw)
  To: jiang.biao2
  Cc: minchan, akpm, hannes, hillf.zj, ying.huang, linux-mm, mgorman,
	linux-kernel, zhong.weidong

On Mon 27-11-17 15:26:54, jiang.biao2@zte.com.cn wrote:
> On Mon, Nov 27, 2017 at 02:27:20PM +0800, jiang.biao2@zte.com.cn wrote:> > I agree with your concern.  How about we take another way by
> > > adding some warning in such case? such as,
> > >         freeable = shrinker->count_objects(shrinker, shrinkctl);
> > > +       if (unlikely(freeable < 0)) {
> > > +               pr_err("shrink_slab: %pF negative objects returned. freeable=%ld\n",
> > > +                       shrinker->scan_objects, freeable);
> > > +               freeable = 0;  //maybe not needed?
> > > +       }
> > >         if (freeable == 0)
> > >                 return 0;
> > > In this way, we would not break the API, but could alert user exception
> > > with message, and make it more robust in such case.
> >
> > True but it would be a problem robust vs. effectivess tradeoff.
> > Think about that everyone want to make thier code robust.
> > It means they start to dump lots of defensive code so code start
> > to look like complicated as well as binary bloating.
> > So, whenever we add some more, we should think how effective
> > the code I am putting?
> > 
> > In this case, I'm skeptical, Sorry. But others might have different
> > opinions. :)
> 
> With all due respect. I still think the robustness is more important than 
> effectiveness in this case. :)

This is a slow path so I wouldn't worry about the performance much. On
the other hand I agree that the API is well documented so adding a
warning is too defensive. We simply assume that the kernel running in
the kernel is reasonable. So I would say, fix your code.

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.
  2017-11-27  8:27               ` Michal Hocko
  (?)
@ 2017-11-27  8:41               ` jiang.biao2
  -1 siblings, 0 replies; 14+ messages in thread
From: jiang.biao2 @ 2017-11-27  8:41 UTC (permalink / raw)
  To: mhocko
  Cc: minchan, akpm, hannes, hillf.zj, ying.huang, linux-mm, mgorman,
	linux-kernel, zhong.weidong


[-- Attachment #1.1: Type: text/plain, Size: 565 bytes --]

On Mon 27-11-17 15:26:54, jiang.biao2@zte.com.cn wrote:
> >> > With all due respect. I still think the robustness is more important than
> > effectiveness in this case. :)
> 
> This is a slow path so I wouldn't worry about the performance much. On
> the other hand I agree that the API is well documented so adding a
> warning is too defensive. We simply assume that the kernel running in
> the kernel is reasonable. So I would say, fix your code.
Ok, already send another patch for mbache shrinker to ensure non-minus 
return.
Thank you for your reply :)

Regards,

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2017-11-27  8:42 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-11-27  1:37 [PATCH] mm/vmscan: make do_shrink_slab more robust Jiang Biao
2017-11-27  1:37 ` Jiang Biao
2017-11-27  2:39 ` Minchan Kim
2017-11-27  2:39   ` Minchan Kim
2017-11-27  4:46   ` jiang.biao2
2017-11-27  5:16     ` Minchan Kim
2017-11-27  5:16       ` Minchan Kim
2017-11-27  6:27       ` jiang.biao2
2017-11-27  6:38         ` Minchan Kim
2017-11-27  6:38           ` Minchan Kim
2017-11-27  7:26           ` jiang.biao2
2017-11-27  8:27             ` Michal Hocko
2017-11-27  8:27               ` Michal Hocko
2017-11-27  8:41               ` jiang.biao2

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.