From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0717FC433EF for ; Thu, 28 Oct 2021 21:52:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A0B9F60EFF for ; Thu, 28 Oct 2021 21:52:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A0B9F60EFF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 38D796B0071; Thu, 28 Oct 2021 17:52:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 33D846B0072; Thu, 28 Oct 2021 17:52:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 205166B0073; Thu, 28 Oct 2021 17:52:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0206.hostedemail.com [216.40.44.206]) by kanga.kvack.org (Postfix) with ESMTP id EDE8B6B0071 for ; Thu, 28 Oct 2021 17:52:37 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 6F8C58249980 for ; Thu, 28 Oct 2021 21:52:37 +0000 (UTC) X-FDA: 78747195954.16.ECF65D0 Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) by imf13.hostedemail.com (Postfix) with ESMTP id BC08D10496B9 for ; Thu, 28 Oct 2021 21:52:30 +0000 (UTC) Received: by mail-ed1-f43.google.com with SMTP id s1so30627444edd.3 for ; Thu, 28 Oct 2021 14:52:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=k6nlxuJLI7WRW8bBc1ndBHLLALciYBovB6sfrHDcCwE=; b=faM23Lo6wwteAgcUZybpAXmP1W/HbtnP9bYBOMwfPaugC8JBw4chVikwO19MVi5nmA ZXj6DxMxk67rlmn198qDvv8sj/oSKra3PDKD7lasuWFFW4KSgFEG72J/i/azKfxdfbdA ugB12wUCB1Z/EivsjVBxn+V5k6k1nmzzXVozPWmRK0mfCid0v03DpAQIMeVE4iV7WOmX ONeNvtL+VhzCG5GPQfqh8b4i/ea3I4Hx8m5BOgqwSmabETOqJQ/Zm7Siy1p52ZzGsfg7 Lc2cgnQ+i2rk22Pp7BtJlChlaE2j9e9tEAQc3sP5UUPfaAfMJv2K9pzkhugm3gf7sKKv tY6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=k6nlxuJLI7WRW8bBc1ndBHLLALciYBovB6sfrHDcCwE=; b=Of2G3l86hOLrzY3/O39asAbYFxmiLmPsNEDJxFV3kjtSxYb4t+bljXvq37h06SvOF5 BmJCutRWrZ9X8iEIuJTLYcvRCg6UIX+/bG8BP5VqbBqKR2Oz+TCqe5wlZHkqGNH8zQZC myDTC4HGeh4cjmKgT7CDZ0+pq330FqevPm0bG/r5qPE93NMMsDQS8Gio2gvCyMyCEXST vF3D4Hwtxmsc9waPSa4XPhWsWdcKHkRK6ZQbDwkBNynLeX3PQ5PUqMYCw7A2PkYfD8cR 2gKMV6hCcESwNXAIQ11nZ7MBKGaIcW2LarGHF21zhgMC2HCEoNeJ5e/Hz1X72nEeW7lW 7O+g== X-Gm-Message-State: AOAM530WN0f4hwdU4IoqBTiLqAMwOwpu7GA1SHUBQJslpvt8cb139ETt XLo+xLeLt8a0w4sVM4kbhJiDwWfyfMPL/Dj8vDs= X-Google-Smtp-Source: ABdhPJytgHqckj5KryDNzB+V5dD3RF97pcEUcep5l/WdvpvAvxCXn9tD6Jsroe8lfOl2E0jtWPP6RhVY2k5BoK34lac= X-Received: by 2002:a17:906:a94b:: with SMTP id hh11mr8762780ejb.85.1635457955739; Thu, 28 Oct 2021 14:52:35 -0700 (PDT) MIME-Version: 1.0 References: <71b597cb9a89c9f033861513da393501f04f4098.camel@infinera.com> <6e986b84ae325fc9e676a4a2026358573bbf7e24.camel@infinera.com> In-Reply-To: <6e986b84ae325fc9e676a4a2026358573bbf7e24.camel@infinera.com> From: Yang Shi Date: Thu, 28 Oct 2021 14:52:23 -0700 Message-ID: Subject: Re: kernel 5.10.x Reserves much more RAM than 5.11.0, 8 MB vs 20 MB To: Joakim Tjernlund Cc: "linux-mm@kvack.org" Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: BC08D10496B9 X-Stat-Signature: kyo4fpcru15ngri571w1ewg44p434ep3 Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=faM23Lo6; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf13.hostedemail.com: domain of shy828301@gmail.com designates 209.85.208.43 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-HE-Tag: 1635457950-458446 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Oct 28, 2021 at 2:55 AM Joakim Tjernlund wrote: > > On Wed, 2021-10-27 at 10:12 -0700, Yang Shi wrote: > > On Wed, Oct 27, 2021 at 5:48 AM Joakim Tjernlund > > wrote: > > > > > > Using a fairly minimal kernel I get a big difference in Reserved memory, using: > > > qemu-system-aarch64 -machine virt -kernel arch/arm64/boot/Image -cpu cortex-a53 -nographic -m 32M > > > > > > I get this: > > > Linux version 5.11.0 > > > Memory: 24052K/32768K available (3520K kernel code, 684K rwdata, 548K rodata, 320K init, 245K bss, 8716K reserved, 0K cma-reserved) > > > > > > Versus: > > > > > > Linux version 5.10.75 > > > Memory: 11836K/32768K available (3518K kernel code, 684K rwdata, 540K rodata, 320K init, 244K bss, 20932K reserved, 0K cma-reserved) > > > > > > Reserving 20 MB RAM on a 32 MB is way too much, even 8Mb is a bit much for us. > > > > This might be caused by the new lockless printk buffer patch > > introduced in 5.10. We noticed this spike on 5.10 kernel too. It may > > have much more memory reserved for meta data and the total amount may > > be as much as 3x of log_buf_len. The amount varies on different > > platforms with different configs. I didn't dig too much into the > > details to figure out how it gets the number. > > > > You could try to adjust CONFIG_LOG_BUF_SHIFT or set log_buf_len from > > kernel commandline to limit the printk buffer size. > > I did try this by changing LOG_BUF_SHIFT but the resulting change was just minor. Thanks for following up, then you may run into a different problem from us. > > To get some info about meminfo/zoneinfo I booted with an inird FS and got this: > Memory: 9576K/32768K available (3518K kernel code, 460K rwdata, 540K rodata, 320K init, 180K bss, 23192K reserved, 0K cma-reserved) > The initrd only added some 3 MB's > Here are meminfo/zoneinfo, not sure if they are useful though. They don't show who the reserved memory is consumed by. If possible you may run git bisect to figure out what commit caused this problem. > > cat /proc/meminfo > MemTotal: 12416 kB > MemFree: 4644 kB > MemAvailable: 4064 kB > Buffers: 0 kB > Cached: 4788 kB > SwapCached: 0 kB > Active: 2500 kB > Inactive: 2524 kB > Active(anon): 2500 kB > Inactive(anon): 2524 kB > Active(file): 0 kB > Inactive(file): 0 kB > Unevictable: 0 kB > Mlocked: 0 kB > SwapTotal: 0 kB > SwapFree: 0 kB > Dirty: 0 kB > Writeback: 0 kB > AnonPages: 248 kB > Mapped: 1844 kB > Shmem: 4788 kB > KReclaimable: 0 kB > Slab: 1712 kB > SReclaimable: 0 kB > SUnreclaim: 1712 kB > KernelStack: 384 kB > PageTables: 80 kB > NFS_Unstable: 0 kB > Bounce: 0 kB > WritebackTmp: 0 kB > CommitLimit: 6208 kB > Committed_AS: 5636 kB > VmallocTotal: 262930368 kB > VmallocUsed: 324 kB > VmallocChunk: 0 kB > Percpu: 76 kB > HardwareCorrupted: 0 kB > > cat /proc/zoneinfo > Node 0, zone Normal > per-node stats > nr_inactive_anon 631 > nr_active_anon 625 > nr_inactive_file 0 > nr_active_file 0 > nr_unevictable 0 > nr_slab_reclaimable 0 > nr_slab_unreclaimable 429 > nr_isolated_anon 0 > nr_isolated_file 0 > workingset_nodes 0 > workingset_refault_anon 0 > workingset_refault_file 0 > workingset_activate_anon 0 > workingset_activate_file 0 > workingset_restore_anon 0 > workingset_restore_file 0 > workingset_nodereclaim 0 > nr_anon_pages 62 > nr_mapped 461 > nr_file_pages 1197 > nr_dirty 0 > nr_writeback 0 > nr_writeback_temp 0 > nr_shmem 1197 > nr_shmem_hugepages 0 > nr_shmem_pmdmapped 0 > nr_file_hugepages 0 > nr_file_pmdmapped 0 > nr_anon_transparent_hugepages 0 > nr_vmscan_write 0 > nr_vmscan_immediate_reclaim 0 > nr_dirtied 0 > nr_written 0 > nr_kernel_misc_reclaimable 0 > nr_foll_pin_acquired 0 > nr_foll_pin_released 0 > nr_kernel_stack 384 > pages free 1162 > min 97 > low 121 > high 145 > spanned 8192 > present 8192 > managed 3104 > protection: (0, 0) > nr_free_pages 1162 > nr_zone_inactive_anon 631 > nr_zone_active_anon 625 > nr_zone_inactive_file 0 > nr_zone_active_file 0 > nr_zone_unevictable 0 > nr_zone_write_pending 0 > nr_mlock 0 > nr_page_table_pages 19 > nr_bounce 0 > nr_free_cma 0 > pagesets > cpu: 0 > count: 0 > high: 0 > batch: 1 > vm stats threshold: 2 > node_unreclaimable: 0 > start_pfn: 262144 > Node 0, zone Movable > pages free 0 > min 0 > low 0 > high 0 > spanned 0 > present 0 > managed 0 > protection: (0, 0) > >