From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36684) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eBxBG-0000TF-DE for qemu-devel@nongnu.org; Tue, 07 Nov 2017 01:16:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eBxBF-0005B1-9v for qemu-devel@nongnu.org; Tue, 07 Nov 2017 01:16:14 -0500 Received: from mail-oi0-x230.google.com ([2607:f8b0:4003:c06::230]:55678) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eBxBF-0005Ak-4F for qemu-devel@nongnu.org; Tue, 07 Nov 2017 01:16:13 -0500 Received: by mail-oi0-x230.google.com with SMTP id g125so9358565oib.12 for ; Mon, 06 Nov 2017 22:16:13 -0800 (PST) MIME-Version: 1.0 In-Reply-To: References: <80415da5-4eef-bf85-b617-fadc3dae4475@redhat.com> <20171106115931.GB3898@lemon> <78a404c5-fb2f-6fdf-c307-107ff1af4342@redhat.com> <5e0576f6-bb91-7e6e-7a1d-391d18717c0d@ozlabs.ru> From: Wanpeng Li Date: Tue, 7 Nov 2017 14:16:11 +0800 Message-ID: Content-Type: text/plain; charset="UTF-8" Subject: Re: [Qemu-devel] qemu core file size List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alexey Kardashevskiy Cc: Paolo Bonzini , Fam Zheng , "qemu-devel@nongnu.org Developers" 2017-11-07 14:12 GMT+08:00 Alexey Kardashevskiy : > On 07/11/17 17:02, Wanpeng Li wrote: >> Hi Alexey, >> 2017-11-07 13:46 GMT+08:00 Alexey Kardashevskiy : >>> On 07/11/17 01:08, Paolo Bonzini wrote: >>>> On 06/11/2017 13:18, Wanpeng Li wrote: >>>>> 2017-11-06 20:02 GMT+08:00 Paolo Bonzini : >>>>>> On 06/11/2017 12:59, Fam Zheng wrote: >>>>>>>>> Could you point out the patchset for the fix? >>>>>>>> Between 447b0d0b9ee8a0ac216c3186e0f3c427a1001f0c and >>>>>>>> 092aa2fc65b7a35121616aad8f39d47b8f921618. >>>>>>> Not sure how these relate to the core size, but I've tested upstream >>>>>>> (ec7a8bf0b8f7dc7288fe8745464ee8217528cc6c) and with dump-guest-core=off the core >>>>>>> file is 363M, still significantly larger than rss (~73M). >>>>>>> >>>>>>> What is bloating the core file? >>>>>> >>>>>> My guess would have been fragmented heap. The core file, unlike the >>>>>> RSS, includes all the mmaped memory (e.g. from shared libraries) that >>>>>> has never been used. >>>>>> >>>>>> For example, all the Ceph/Gluster/PulseAudio/SPICE/whatever libraries >>>>>> are included in the core file but likely are not in the RSS. >>>>> >>>>> Do you mean not use Memory API will avoid the fragmented heap? >>>> >>>> The high memory usage from the memory API causes excessive >>>> fragmentation. Alexey's work should help reducing memory usage and thus >>>> the fragmentation. >>> >>> Since centos6 does not have this issue and centos7 does, I'd suggest >>> MALLOC_ARENA_MAX&co >>> https://www.gnu.org/software/libc/manual/html_node/Memory-Allocation-Tunables.html >> >> Thanks for the inputs. What's the recommend glibc.malloc.arena_max >> value from you? :) > > I have no idea, you have to try different values as you are the one who is > unhappy about the sizes :) I do not normally mess with this as things > should just work with the defaults whatever they are. > > >> >>> >>> glibc changed some defaults between rhel/centos 6 and 7 if I recall >>> correctly, and these arenas now create big anon mappings per thread, these >>> are not really touched (so they do not use resident memory) but may appear >>> in these dumps, dunno. >>> >>> btw do you configure the machine to make "kill -11" produce qemu core dumps? >> >> Yeah, some tuning in /etc/libvirt/qemu.conf, max_core="unlimited", >> dump_guest_core=0 > > dump_guest_core=0 should mean "no dumps", no? > Do you know how to enable this without libvirt? Please add dump_guest_core=off in your qemu command-line, "dump_guest_core" mean not dump guest memory and just dump qemu itself. Regards, Wanpeng Li