All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH] rcu: reduce half heap memory size by malloc_trim()
@ 2017-11-17  6:23 Yang Zhong
  2017-11-17 13:54 ` Stefan Hajnoczi
  2017-11-20 14:28 ` Fam Zheng
  0 siblings, 2 replies; 11+ messages in thread
From: Yang Zhong @ 2017-11-17  6:23 UTC (permalink / raw)
  To: qemu-devel, pbonzini
  Cc: stefanha, anthony.xu, yang.zhong, stone.xulei, arei.gonglei,
	wangxinxin.wang, weidong.huang, zhang.zhanghailiang, liujunjie23

Since there are some issues in memory alloc/free machenism
in glibc for little chunk memory, if Qemu frequently
alloc/free little chunk memory, the glibc doesn't alloc
little chunk memory from free list of glibc and still
allocate from OS, which make the heap size bigger and bigger.

This patch introduce malloc_trim(), which will free heap memory.

Below are test results from smaps file.
55f0783e1000-55f07992a000 rw-p 00000000 00:00 0  [heap]
Size:              21796 kB
Rss:               14260 kB
Pss:               14260 kB

55cc5fadf000-55cc61008000 rw-p 00000000 00:00 0  [heap]
Size:              21668 kB
Rss:                6940 kB
Pss:                6940 kB

Signed-off-by: Yang Zhong <yang.zhong@intel.com>
---
 util/rcu.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/util/rcu.c b/util/rcu.c
index ca5a63e..8d491a6 100644
--- a/util/rcu.c
+++ b/util/rcu.c
@@ -26,6 +26,7 @@
  * IBM's contributions to this file may be relicensed under LGPLv2 or later.
  */
 
+#include <malloc.h>
 #include "qemu/osdep.h"
 #include "qemu-common.h"
 #include "qemu/rcu.h"
@@ -272,6 +273,9 @@ static void *call_rcu_thread(void *opaque)
             node->func(node);
         }
         qemu_mutex_unlock_iothread();
+#ifdef CONFIG_LINUX
+        malloc_trim(0);
+#endif
     }
     abort();
 }
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: [Qemu-devel] [PATCH] rcu: reduce half heap memory size by malloc_trim()
  2017-11-17  6:23 [Qemu-devel] [PATCH] rcu: reduce half heap memory size by malloc_trim() Yang Zhong
@ 2017-11-17 13:54 ` Stefan Hajnoczi
  2017-11-17 14:06   ` Daniel P. Berrange
  2017-11-20  8:41   ` Zhong Yang
  2017-11-20 14:28 ` Fam Zheng
  1 sibling, 2 replies; 11+ messages in thread
From: Stefan Hajnoczi @ 2017-11-17 13:54 UTC (permalink / raw)
  To: Yang Zhong
  Cc: qemu-devel, pbonzini, anthony.xu, stone.xulei, arei.gonglei,
	wangxinxin.wang, weidong.huang, zhang.zhanghailiang, liujunjie23

[-- Attachment #1: Type: text/plain, Size: 1025 bytes --]

On Fri, Nov 17, 2017 at 02:23:34PM +0800, Yang Zhong wrote:
> diff --git a/util/rcu.c b/util/rcu.c
> index ca5a63e..8d491a6 100644
> --- a/util/rcu.c
> +++ b/util/rcu.c
> @@ -26,6 +26,7 @@
>   * IBM's contributions to this file may be relicensed under LGPLv2 or later.
>   */
>  
> +#include <malloc.h>

This header file is not mentioned in the C99 standard or POSIX.  It is
probably not available on all host OSes that QEMU supports.  Please use
#ifdef CONFIG_LINUX.

>  #include "qemu/osdep.h"
>  #include "qemu-common.h"
>  #include "qemu/rcu.h"
> @@ -272,6 +273,9 @@ static void *call_rcu_thread(void *opaque)
>              node->func(node);
>          }
>          qemu_mutex_unlock_iothread();
> +#ifdef CONFIG_LINUX
> +        malloc_trim(0);
> +#endif

It is important that the rcu thread isn't overzealous in minimizing heap
size if that means ordinary malloc(3) calls will experience latency
spikes.  Please leave a few MB free so that malloc(3) doesn't take the
slow path.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [Qemu-devel] [PATCH] rcu: reduce half heap memory size by malloc_trim()
  2017-11-17 13:54 ` Stefan Hajnoczi
@ 2017-11-17 14:06   ` Daniel P. Berrange
  2017-11-20  8:54     ` Zhong Yang
  2017-11-20  8:41   ` Zhong Yang
  1 sibling, 1 reply; 11+ messages in thread
From: Daniel P. Berrange @ 2017-11-17 14:06 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: Yang Zhong, weidong.huang, zhang.zhanghailiang, liujunjie23,
	wangxinxin.wang, stone.xulei, qemu-devel, anthony.xu,
	arei.gonglei, pbonzini

On Fri, Nov 17, 2017 at 01:54:09PM +0000, Stefan Hajnoczi wrote:
> On Fri, Nov 17, 2017 at 02:23:34PM +0800, Yang Zhong wrote:
> > diff --git a/util/rcu.c b/util/rcu.c
> > index ca5a63e..8d491a6 100644
> > --- a/util/rcu.c
> > +++ b/util/rcu.c
> > @@ -26,6 +26,7 @@
> >   * IBM's contributions to this file may be relicensed under LGPLv2 or later.
> >   */
> >  
> > +#include <malloc.h>
> 
> This header file is not mentioned in the C99 standard or POSIX.  It is
> probably not available on all host OSes that QEMU supports.  Please use
> #ifdef CONFIG_LINUX.
> 
> >  #include "qemu/osdep.h"
> >  #include "qemu-common.h"
> >  #include "qemu/rcu.h"
> > @@ -272,6 +273,9 @@ static void *call_rcu_thread(void *opaque)
> >              node->func(node);
> >          }
> >          qemu_mutex_unlock_iothread();
> > +#ifdef CONFIG_LINUX
> > +        malloc_trim(0);
> > +#endif
> 
> It is important that the rcu thread isn't overzealous in minimizing heap
> size if that means ordinary malloc(3) calls will experience latency
> spikes.  Please leave a few MB free so that malloc(3) doesn't take the
> slow path.

If you pass '0' the docs say that the minimum amount is left in the
heap, per M_TOP_PAD, which is 128kb.

Strangely the mallopt(3) man page suggests, that free() should automatically
trim the heap when its size exceeds M_TOP_TRIM, which is again 128kb by
default.  So I'm puzzelled by malloc_trim() would be needed unless there
are scenarios in which free() won't trim, that aren't mentioned in the
manpage.

Also, how does malloc_trim interact with tcmalloc.so that people often
use in preference to glibc's built in malloc ?

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [Qemu-devel] [PATCH] rcu: reduce half heap memory size by malloc_trim()
  2017-11-17 13:54 ` Stefan Hajnoczi
  2017-11-17 14:06   ` Daniel P. Berrange
@ 2017-11-20  8:41   ` Zhong Yang
  2017-11-20 14:03     ` Stefan Hajnoczi
  1 sibling, 1 reply; 11+ messages in thread
From: Zhong Yang @ 2017-11-20  8:41 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: qemu-devel, pbonzini, anthony.xu, stone.xulei, arei.gonglei,
	wangxinxin.wang, weidong.huang, zhang.zhanghailiang, liujunjie23,
	yang.zhong

On Fri, Nov 17, 2017 at 01:54:09PM +0000, Stefan Hajnoczi wrote:
> On Fri, Nov 17, 2017 at 02:23:34PM +0800, Yang Zhong wrote:
> > diff --git a/util/rcu.c b/util/rcu.c
> > index ca5a63e..8d491a6 100644
> > --- a/util/rcu.c
> > +++ b/util/rcu.c
> > @@ -26,6 +26,7 @@
> >   * IBM's contributions to this file may be relicensed under LGPLv2 or later.
> >   */
> >  
> > +#include <malloc.h>
> 
> This header file is not mentioned in the C99 standard or POSIX.  It is
> probably not available on all host OSes that QEMU supports.  Please use
> #ifdef CONFIG_LINUX.
> 
  Hello Stefan,

  Thanks for your remind! 
  
  Regards,

  Yang 
> >  #include "qemu/osdep.h"
> >  #include "qemu-common.h"
> >  #include "qemu/rcu.h"
> > @@ -272,6 +273,9 @@ static void *call_rcu_thread(void *opaque)
> >              node->func(node);
> >          }
> >          qemu_mutex_unlock_iothread();
> > +#ifdef CONFIG_LINUX
> > +        malloc_trim(0);
> > +#endif
> 
> It is important that the rcu thread isn't overzealous in minimizing heap
> size if that means ordinary malloc(3) calls will experience latency
> spikes.  Please leave a few MB free so that malloc(3) doesn't take the
> slow path.
> 
  Hello Stefan,

  From the Qemu bootup procedure, the qemu malloc chunk memory from OS, not
  from glibc free list which is freed before by Qemu. Maybe there are some 
  issues in glibc memory mechanism. I will continue to fine this parameter
  to get better balance, Many thanks!

  Regards,

  Yang 

> Stefan

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [Qemu-devel] [PATCH] rcu: reduce half heap memory size by malloc_trim()
  2017-11-17 14:06   ` Daniel P. Berrange
@ 2017-11-20  8:54     ` Zhong Yang
  2017-11-20 14:14       ` Daniel P. Berrange
  0 siblings, 1 reply; 11+ messages in thread
From: Zhong Yang @ 2017-11-20  8:54 UTC (permalink / raw)
  To: Daniel P. Berrange
  Cc: qemu-devel, pbonzini, stefanha, anthony.xu, stone.xulei,
	arei.gonglei, wangxinxin.wang, weidong.huang,
	zhang.zhanghailiang, liujunjie23

On Fri, Nov 17, 2017 at 02:06:20PM +0000, Daniel P. Berrange wrote:
> On Fri, Nov 17, 2017 at 01:54:09PM +0000, Stefan Hajnoczi wrote:
> > On Fri, Nov 17, 2017 at 02:23:34PM +0800, Yang Zhong wrote:
> > > diff --git a/util/rcu.c b/util/rcu.c
> > > index ca5a63e..8d491a6 100644
> > > --- a/util/rcu.c
> > > +++ b/util/rcu.c
> > > @@ -26,6 +26,7 @@
> > >   * IBM's contributions to this file may be relicensed under LGPLv2 or later.
> > >   */
> > >  
> > > +#include <malloc.h>
> > 
> > This header file is not mentioned in the C99 standard or POSIX.  It is
> > probably not available on all host OSes that QEMU supports.  Please use
> > #ifdef CONFIG_LINUX.
> > 
> > >  #include "qemu/osdep.h"
> > >  #include "qemu-common.h"
> > >  #include "qemu/rcu.h"
> > > @@ -272,6 +273,9 @@ static void *call_rcu_thread(void *opaque)
> > >              node->func(node);
> > >          }
> > >          qemu_mutex_unlock_iothread();
> > > +#ifdef CONFIG_LINUX
> > > +        malloc_trim(0);
> > > +#endif
> > 
> > It is important that the rcu thread isn't overzealous in minimizing heap
> > size if that means ordinary malloc(3) calls will experience latency
> > spikes.  Please leave a few MB free so that malloc(3) doesn't take the
> > slow path.
> 
> If you pass '0' the docs say that the minimum amount is left in the
> heap, per M_TOP_PAD, which is 128kb.
> 
> Strangely the mallopt(3) man page suggests, that free() should automatically
> trim the heap when its size exceeds M_TOP_TRIM, which is again 128kb by
> default.  So I'm puzzelled by malloc_trim() would be needed unless there
> are scenarios in which free() won't trim, that aren't mentioned in the
> manpage.
  Hello Daniel,

  In fact, i firstly adopted mallopt() solution to optimize the heap memory,
  but i found this function is NOT useful, which are difference with MAN's
  description, so i had to swith to use malloc_trim().
  
  Regards,

  Yang

> Also, how does malloc_trim interact with tcmalloc.so that people often
> use in preference to glibc's built in malloc ?
  Thanks, you reminded me to consider tcmalloc or jemalloc.
 
  Whether below code is more suitable? thanks!
  #if defined(CONFIG_LINUX) && defined(__GLIBC__)
         malloc_trim(0);
  #endif

  Regards,

  Yang

> 
> Regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [Qemu-devel] [PATCH] rcu: reduce half heap memory size by malloc_trim()
  2017-11-20  8:41   ` Zhong Yang
@ 2017-11-20 14:03     ` Stefan Hajnoczi
  2017-11-21  3:20       ` Zhong Yang
  0 siblings, 1 reply; 11+ messages in thread
From: Stefan Hajnoczi @ 2017-11-20 14:03 UTC (permalink / raw)
  To: Zhong Yang
  Cc: Stefan Hajnoczi, weidong.huang, zhang.zhanghailiang, liujunjie23,
	wangxinxin.wang, stone.xulei, qemu-devel, anthony.xu,
	arei.gonglei, pbonzini

[-- Attachment #1: Type: text/plain, Size: 1094 bytes --]

On Mon, Nov 20, 2017 at 04:41:41PM +0800, Zhong Yang wrote:
> On Fri, Nov 17, 2017 at 01:54:09PM +0000, Stefan Hajnoczi wrote:
> > On Fri, Nov 17, 2017 at 02:23:34PM +0800, Yang Zhong wrote:
> > > @@ -272,6 +273,9 @@ static void *call_rcu_thread(void *opaque)
> > >              node->func(node);
> > >          }
> > >          qemu_mutex_unlock_iothread();
> > > +#ifdef CONFIG_LINUX
> > > +        malloc_trim(0);
> > > +#endif
> > 
> > It is important that the rcu thread isn't overzealous in minimizing heap
> > size if that means ordinary malloc(3) calls will experience latency
> > spikes.  Please leave a few MB free so that malloc(3) doesn't take the
> > slow path.
> > 
>   Hello Stefan,
> 
>   From the Qemu bootup procedure, the qemu malloc chunk memory from OS, not
>   from glibc free list which is freed before by Qemu. Maybe there are some 
>   issues in glibc memory mechanism. I will continue to fine this parameter
>   to get better balance, Many thanks!

I was suggesting malloc_trim(4 * 1024 * 1024) or similar instead of
malloc_trim(0).

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [Qemu-devel] [PATCH] rcu: reduce half heap memory size by malloc_trim()
  2017-11-20  8:54     ` Zhong Yang
@ 2017-11-20 14:14       ` Daniel P. Berrange
  2017-11-21  5:46         ` Zhong Yang
  0 siblings, 1 reply; 11+ messages in thread
From: Daniel P. Berrange @ 2017-11-20 14:14 UTC (permalink / raw)
  To: Zhong Yang
  Cc: qemu-devel, pbonzini, stefanha, anthony.xu, stone.xulei,
	arei.gonglei, wangxinxin.wang, weidong.huang,
	zhang.zhanghailiang, liujunjie23

On Mon, Nov 20, 2017 at 04:54:42PM +0800, Zhong Yang wrote:
> On Fri, Nov 17, 2017 at 02:06:20PM +0000, Daniel P. Berrange wrote:
> > On Fri, Nov 17, 2017 at 01:54:09PM +0000, Stefan Hajnoczi wrote:
> > > On Fri, Nov 17, 2017 at 02:23:34PM +0800, Yang Zhong wrote:
> > > > diff --git a/util/rcu.c b/util/rcu.c
> > > > index ca5a63e..8d491a6 100644
> > > > --- a/util/rcu.c
> > > > +++ b/util/rcu.c
> > > > @@ -26,6 +26,7 @@
> > > >   * IBM's contributions to this file may be relicensed under LGPLv2 or later.
> > > >   */
> > > >  
> > > > +#include <malloc.h>
> > > 
> > > This header file is not mentioned in the C99 standard or POSIX.  It is
> > > probably not available on all host OSes that QEMU supports.  Please use
> > > #ifdef CONFIG_LINUX.
> > > 
> > > >  #include "qemu/osdep.h"
> > > >  #include "qemu-common.h"
> > > >  #include "qemu/rcu.h"
> > > > @@ -272,6 +273,9 @@ static void *call_rcu_thread(void *opaque)
> > > >              node->func(node);
> > > >          }
> > > >          qemu_mutex_unlock_iothread();
> > > > +#ifdef CONFIG_LINUX
> > > > +        malloc_trim(0);
> > > > +#endif
> > > 
> > > It is important that the rcu thread isn't overzealous in minimizing heap
> > > size if that means ordinary malloc(3) calls will experience latency
> > > spikes.  Please leave a few MB free so that malloc(3) doesn't take the
> > > slow path.
> > 
> > If you pass '0' the docs say that the minimum amount is left in the
> > heap, per M_TOP_PAD, which is 128kb.
> > 
> > Strangely the mallopt(3) man page suggests, that free() should automatically
> > trim the heap when its size exceeds M_TOP_TRIM, which is again 128kb by
> > default.  So I'm puzzelled by malloc_trim() would be needed unless there
> > are scenarios in which free() won't trim, that aren't mentioned in the
> > manpage.
> 
>   In fact, i firstly adopted mallopt() solution to optimize the heap memory,
>   but i found this function is NOT useful, which are difference with MAN's
>   description, so i had to swith to use malloc_trim().


> > Also, how does malloc_trim interact with tcmalloc.so that people often
> > use in preference to glibc's built in malloc ?
>   Thanks, you reminded me to consider tcmalloc or jemalloc.
>  
>   Whether below code is more suitable? thanks!
>   #if defined(CONFIG_LINUX) && defined(__GLIBC__)
>          malloc_trim(0);
>   #endif

Both of those macro symbols will still be defined even when tcmalloc/jemalloc
are in use.

I wonder if tcmalloc/jemalloc even suffer from the same problem that libc's
builtin malloc has ?

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [Qemu-devel] [PATCH] rcu: reduce half heap memory size by malloc_trim()
  2017-11-17  6:23 [Qemu-devel] [PATCH] rcu: reduce half heap memory size by malloc_trim() Yang Zhong
  2017-11-17 13:54 ` Stefan Hajnoczi
@ 2017-11-20 14:28 ` Fam Zheng
  2017-11-21  3:12   ` Zhong Yang
  1 sibling, 1 reply; 11+ messages in thread
From: Fam Zheng @ 2017-11-20 14:28 UTC (permalink / raw)
  To: Yang Zhong
  Cc: qemu-devel, pbonzini, weidong.huang, arei.gonglei, liujunjie23,
	wangxinxin.wang, stone.xulei, anthony.xu, stefanha,
	zhang.zhanghailiang

On Fri, 11/17 14:23, Yang Zhong wrote:
> diff --git a/util/rcu.c b/util/rcu.c
> index ca5a63e..8d491a6 100644
> --- a/util/rcu.c
> +++ b/util/rcu.c
> @@ -26,6 +26,7 @@
>   * IBM's contributions to this file may be relicensed under LGPLv2 or later.
>   */
>  
> +#include <malloc.h>

BTW, if you respin, the "qemu/osdep.h" header should always be included first as
per HACKING file.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [Qemu-devel] [PATCH] rcu: reduce half heap memory size by malloc_trim()
  2017-11-20 14:28 ` Fam Zheng
@ 2017-11-21  3:12   ` Zhong Yang
  0 siblings, 0 replies; 11+ messages in thread
From: Zhong Yang @ 2017-11-21  3:12 UTC (permalink / raw)
  To: Fam Zheng
  Cc: qemu-devel, pbonzini, weidong.huang, arei.gonglei, liujunjie23,
	wangxinxin.wang, stone.xulei, anthony.xu, stefanha,
	zhang.zhanghailiang, yang.zhong

On Mon, Nov 20, 2017 at 10:28:28PM +0800, Fam Zheng wrote:
> On Fri, 11/17 14:23, Yang Zhong wrote:
> > diff --git a/util/rcu.c b/util/rcu.c
> > index ca5a63e..8d491a6 100644
> > --- a/util/rcu.c
> > +++ b/util/rcu.c
> > @@ -26,6 +26,7 @@
> >   * IBM's contributions to this file may be relicensed under LGPLv2 or later.
> >   */
> >  
> > +#include <malloc.h>
> 
> BTW, if you respin, the "qemu/osdep.h" header should always be included first as
> per HACKING file.

  Hello Fam,

  Thanks for your reminder!
  I ever thought the header file include in Qemu like linux kernel did, thanks again!

  Regards,

  Yang

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [Qemu-devel] [PATCH] rcu: reduce half heap memory size by malloc_trim()
  2017-11-20 14:03     ` Stefan Hajnoczi
@ 2017-11-21  3:20       ` Zhong Yang
  0 siblings, 0 replies; 11+ messages in thread
From: Zhong Yang @ 2017-11-21  3:20 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: qemu-devel, pbonzini, weidong.huang, arei.gonglei, liujunjie23,
	wangxinxin.wang, stone.xulei, anthony.xu, zhang.zhanghailiang,
	famz, berrange, yang.zhong

On Mon, Nov 20, 2017 at 02:03:44PM +0000, Stefan Hajnoczi wrote:
> On Mon, Nov 20, 2017 at 04:41:41PM +0800, Zhong Yang wrote:
> > On Fri, Nov 17, 2017 at 01:54:09PM +0000, Stefan Hajnoczi wrote:
> > > On Fri, Nov 17, 2017 at 02:23:34PM +0800, Yang Zhong wrote:
> > > > @@ -272,6 +273,9 @@ static void *call_rcu_thread(void *opaque)
> > > >              node->func(node);
> > > >          }
> > > >          qemu_mutex_unlock_iothread();
> > > > +#ifdef CONFIG_LINUX
> > > > +        malloc_trim(0);
> > > > +#endif
> > > 
> > > It is important that the rcu thread isn't overzealous in minimizing heap
> > > size if that means ordinary malloc(3) calls will experience latency
> > > spikes.  Please leave a few MB free so that malloc(3) doesn't take the
> > > slow path.
> > > 
> >   Hello Stefan,
> > 
> >   From the Qemu bootup procedure, the qemu malloc chunk memory from OS, not
> >   from glibc free list which is freed before by Qemu. Maybe there are some 
> >   issues in glibc memory mechanism. I will continue to fine this parameter
> >   to get better balance, Many thanks!
> 
> I was suggesting malloc_trim(4 * 1024 * 1024) or similar instead of
> malloc_trim(0).
>  
  Hello Stefan,

  Thanks for your suggestion!
  I tried your suggested changes, malloc_trim(4 * 1024 * 1024), the heap size with
  this change almost same with malloc_trim(0), thanks!

  Regards,

  Yang   
 
> Stefan

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [Qemu-devel] [PATCH] rcu: reduce half heap memory size by malloc_trim()
  2017-11-20 14:14       ` Daniel P. Berrange
@ 2017-11-21  5:46         ` Zhong Yang
  0 siblings, 0 replies; 11+ messages in thread
From: Zhong Yang @ 2017-11-21  5:46 UTC (permalink / raw)
  To: Daniel P. Berrange, stone.xulei
  Cc: qemu-devel, pbonzini, weidong.huang, arei.gonglei, liujunjie23,
	wangxinxin.wang, anthony.xu, stefanha, zhang.zhanghailiang, famz,
	yang.zhong

On Mon, Nov 20, 2017 at 02:14:50PM +0000, Daniel P. Berrange wrote:
> On Mon, Nov 20, 2017 at 04:54:42PM +0800, Zhong Yang wrote:
> > On Fri, Nov 17, 2017 at 02:06:20PM +0000, Daniel P. Berrange wrote:
> > > On Fri, Nov 17, 2017 at 01:54:09PM +0000, Stefan Hajnoczi wrote:
> > > > On Fri, Nov 17, 2017 at 02:23:34PM +0800, Yang Zhong wrote:
> > > > > diff --git a/util/rcu.c b/util/rcu.c
> > > > > index ca5a63e..8d491a6 100644
> > > > > --- a/util/rcu.c
> > > > > +++ b/util/rcu.c
> > > > > @@ -26,6 +26,7 @@
> > > > >   * IBM's contributions to this file may be relicensed under LGPLv2 or later.
> > > > >   */
> > > > >  
> > > > > +#include <malloc.h>
> > > > 
> > > > This header file is not mentioned in the C99 standard or POSIX.  It is
> > > > probably not available on all host OSes that QEMU supports.  Please use
> > > > #ifdef CONFIG_LINUX.
> > > > 
> > > > >  #include "qemu/osdep.h"
> > > > >  #include "qemu-common.h"
> > > > >  #include "qemu/rcu.h"
> > > > > @@ -272,6 +273,9 @@ static void *call_rcu_thread(void *opaque)
> > > > >              node->func(node);
> > > > >          }
> > > > >          qemu_mutex_unlock_iothread();
> > > > > +#ifdef CONFIG_LINUX
> > > > > +        malloc_trim(0);
> > > > > +#endif
> > > > 
> > > > It is important that the rcu thread isn't overzealous in minimizing heap
> > > > size if that means ordinary malloc(3) calls will experience latency
> > > > spikes.  Please leave a few MB free so that malloc(3) doesn't take the
> > > > slow path.
> > > 
> > > If you pass '0' the docs say that the minimum amount is left in the
> > > heap, per M_TOP_PAD, which is 128kb.
> > > 
> > > Strangely the mallopt(3) man page suggests, that free() should automatically
> > > trim the heap when its size exceeds M_TOP_TRIM, which is again 128kb by
> > > default.  So I'm puzzelled by malloc_trim() would be needed unless there
> > > are scenarios in which free() won't trim, that aren't mentioned in the
> > > manpage.
> > 
> >   In fact, i firstly adopted mallopt() solution to optimize the heap memory,
> >   but i found this function is NOT useful, which are difference with MAN's
> >   description, so i had to swith to use malloc_trim().
> 
> 
> > > Also, how does malloc_trim interact with tcmalloc.so that people often
> > > use in preference to glibc's built in malloc ?
> >   Thanks, you reminded me to consider tcmalloc or jemalloc.
> >  
> >   Whether below code is more suitable? thanks!
> >   #if defined(CONFIG_LINUX) && defined(__GLIBC__)
> >          malloc_trim(0);
> >   #endif
> 
> Both of those macro symbols will still be defined even when tcmalloc/jemalloc
> are in use.
  
  Hello Daniel,

  If those two macro are NOT useful for this scenario, how about below changes?

  In configure file, 

  if test "$tcmalloc" = "yes" ; then
      echo "CONFIG__TCMALLOC=y" >> $config_host_mak
  ........ 

  then use !CONFIG_TCMALLOC macro to check malloc_trim()? thanks!

  Regards,

  Yang

 
> I wonder if tcmalloc/jemalloc even suffer from the same problem that libc's
> builtin malloc has ?
 
  Hello Xulei,

  Did you compiled Qemu with tcmalloc or jemalloc ? If you did , would you 
  please share those datas with different malloc compile option?  Many thanks!

  Regards,

  Yang


> Regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2017-11-21  5:46 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-11-17  6:23 [Qemu-devel] [PATCH] rcu: reduce half heap memory size by malloc_trim() Yang Zhong
2017-11-17 13:54 ` Stefan Hajnoczi
2017-11-17 14:06   ` Daniel P. Berrange
2017-11-20  8:54     ` Zhong Yang
2017-11-20 14:14       ` Daniel P. Berrange
2017-11-21  5:46         ` Zhong Yang
2017-11-20  8:41   ` Zhong Yang
2017-11-20 14:03     ` Stefan Hajnoczi
2017-11-21  3:20       ` Zhong Yang
2017-11-20 14:28 ` Fam Zheng
2017-11-21  3:12   ` Zhong Yang

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.