All of lore.kernel.org
 help / color / mirror / Atom feed
From: Muchun Song <songmuchun@bytedance.com>
To: Cong Wang <xiyou.wangcong@gmail.com>
Cc: Eric Dumazet <eric.dumazet@gmail.com>,
	Eric Dumazet <edumazet@google.com>,
	Greg KH <gregkh@linuxfoundation.org>,
	rafael@kernel.org, "Michael S. Tsirkin" <mst@redhat.com>,
	Jason Wang <jasowang@redhat.com>,
	David Miller <davem@davemloft.net>,
	Jakub Kicinski <kuba@kernel.org>,
	Alexey Dobriyan <adobriyan@gmail.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>,
	Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>,
	Steffen Klassert <steffen.klassert@secunet.com>,
	Herbert Xu <herbert@gondor.apana.org.au>,
	Shakeel Butt <shakeelb@google.com>, Will Deacon <will@kernel.org>,
	Michal Hocko <mhocko@suse.com>, Roman Gushchin <guro@fb.com>,
	Neil Brown <neilb@suse.de>, Mike Rapoport <rppt@kernel.org>,
	Sami Tolvanen <samitolvanen@google.com>,
	"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
	Feng Tang <feng.tang@intel.com>, Paolo Abeni <pabeni@redhat.com>,
	Willem de Bruijn <willemb@google.com>,
	Randy Dunlap <rdunlap@infradead.org>,
	Florian Westphal <fw@strlen.de>,
	gustavoars@kernel.org, Pablo Neira Ayuso <pablo@netfilter.org>,
	Dexuan Cui <decui@microsoft.com>,
	Jakub Sitnicki <jakub@cloudflare.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Christian Brauner <christian.brauner@ubuntu.com>,
	"Eric W. Biederman" <ebiederm@xmission.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	dave@stgolabs.net, Michel Lespinasse <walken@google.com>,
	Jann Horn <jannh@google.com>,
	chenqiwu@xiaomi.com, christophe.leroy@c-s.fr,
	Minchan Kim <minchan@kernel.org>, Martin KaFai Lau <kafai@fb.com>,
	Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Miaohe Lin <linmiaohe@huawei.com>,
	Kees Cook <keescook@chromium.org>,
	LKML <linux-kernel@vger.kernel.org>,
	virtualization@lists.linux-foundation.org,
	Linux Kernel Network Developers <netdev@vger.kernel.org>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	linux-mm <linux-mm@kvack.org>
Subject: Re: [External] Re: [PATCH] mm: proc: add Sock to /proc/meminfo
Date: Tue, 13 Oct 2020 11:52:28 +0800	[thread overview]
Message-ID: <CAMZfGtUgX2JNVHW8F4cJ7rY2T-8DNkouQh8O0-j6CAfd4+TCew@mail.gmail.com> (raw)
In-Reply-To: <CAM_iQpUgy7MDka8A44U=pLGDOwqn8YhXMp8rgs8LCJBHb5DXYA@mail.gmail.com>

On Tue, Oct 13, 2020 at 6:12 AM Cong Wang <xiyou.wangcong@gmail.com> wrote:
>
> On Mon, Oct 12, 2020 at 2:53 AM Muchun Song <songmuchun@bytedance.com> wrote:
> > We are not complaining about TCP using too much memory, but how do
> > we know that TCP uses a lot of memory. When I firstly face this problem,
> > I do not know who uses the 25GB memory and it is not shown in the /proc/meminfo.
> > If we can know the amount memory of the socket buffer via /proc/meminfo, we
> > may not need to spend a lot of time troubleshooting this problem. Not everyone
> > knows that a lot of memory may be used here. But I believe many people
> > should know /proc/meminfo to confirm memory users.
>
> Well, I'd bet networking people know `ss -m` better than /proc/meminfo,

I agree with you. But if someone(not networking people) faces the same
problem. He may suspect that there is a memory leak or think that a certain
driver allocates memory but has no statistics. He only saw the memory
disappeared via /proc/meminfo.

> generally speaking.
>
> The practice here is that if you want some networking-specific counters,
> add it to where networking people know better, that is, `ss -m` or /proc/net/...
>
> Or maybe the problem you described is not specific to networking at all,
> there must be some other places where pages are allocated but not charged.

Yeah, it is not charged. The allocation path is as follows. This allocation
consumes 25GB memory on our server. And it belongs to the network core.

Thanks.

   __alloc_pages_nodemask+0x11d/0x290
   skb_page_frag_refill+0x68/0xf0
   sk_page_frag_refill+0x19/0x70
   tcp_sendmsg_locked+0x2f4/0xd10
   tcp_sendmsg+0x29/0xa0
   sock_sendmsg+0x30/0x40
   sock_write_iter+0x8f/0x100
   __vfs_write+0x10b/0x190
   vfs_write+0xb0/0x190
   ksys_write+0x5a/0xd0
   do_syscall_64+0x5d/0x110
   entry_SYSCALL_64_after_hwframe+0x44/0xa9

> If so, adding a general mm counter in /proc/meminfo makes sense, but
> it won't be specific to networking.
>
> Thanks.



-- 
Yours,
Muchun

  reply	other threads:[~2020-10-13  3:53 UTC|newest]

Thread overview: 50+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-10 10:38 [PATCH] mm: proc: add Sock to /proc/meminfo Muchun Song
2020-10-10 13:06 ` kernel test robot
2020-10-10 13:27 ` kernel test robot
2020-10-10 16:36 ` Randy Dunlap
2020-10-10 16:36   ` Randy Dunlap
2020-10-11  4:42   ` [External] " Muchun Song
2020-10-11  4:42     ` Muchun Song
2020-10-11 13:52 ` Mike Rapoport
2020-10-11 16:00   ` [External] " Muchun Song
2020-10-11 16:00     ` Muchun Song
2020-10-11 18:39 ` Cong Wang
2020-10-11 18:39   ` Cong Wang
2020-10-11 18:39   ` Cong Wang
2020-10-12  4:22   ` [External] " Muchun Song
2020-10-12  4:22     ` Muchun Song
2020-10-12  7:42     ` Eric Dumazet
2020-10-12  7:42       ` Eric Dumazet
2020-10-12  8:39       ` Muchun Song
2020-10-12  8:39         ` Muchun Song
2020-10-12  9:24         ` Eric Dumazet
2020-10-12  9:24           ` Eric Dumazet
2020-10-12  9:53           ` Muchun Song
2020-10-12  9:53             ` Muchun Song
2020-10-12 22:12             ` Cong Wang
2020-10-12 22:12               ` Cong Wang
2020-10-12 22:12               ` Cong Wang
2020-10-13  3:52               ` Muchun Song [this message]
2020-10-13  3:52                 ` Muchun Song
2020-10-13  6:55             ` Eric Dumazet
2020-10-13  6:55               ` Eric Dumazet
2020-10-13  8:09             ` Mike Rapoport
2020-10-13 14:43               ` Randy Dunlap
2020-10-13 14:43                 ` Randy Dunlap
2020-10-13 15:12                 ` Mike Rapoport
2020-10-13 15:21                   ` Randy Dunlap
2020-10-13 15:21                     ` Randy Dunlap
2020-10-14  5:34                     ` Mike Rapoport
2020-10-13 15:28               ` Muchun Song
2020-10-13 15:28                 ` Muchun Song
2020-10-16 15:38               ` Vlastimil Babka
2020-10-16 15:38                 ` Vlastimil Babka
2020-10-16 20:53                 ` Minchan Kim
2020-10-16 20:53                   ` Minchan Kim
2020-10-19 17:23                   ` Shakeel Butt
2020-10-19 17:23                     ` Shakeel Butt
2020-10-12 21:46     ` Cong Wang
2020-10-12 21:46       ` Cong Wang
2020-10-12 21:46       ` Cong Wang
2020-10-13  3:29       ` Muchun Song
2020-10-13  3:29         ` Muchun Song

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAMZfGtUgX2JNVHW8F4cJ7rY2T-8DNkouQh8O0-j6CAfd4+TCew@mail.gmail.com \
    --to=songmuchun@bytedance.com \
    --cc=adobriyan@gmail.com \
    --cc=akpm@linux-foundation.org \
    --cc=ast@kernel.org \
    --cc=chenqiwu@xiaomi.com \
    --cc=christian.brauner@ubuntu.com \
    --cc=christophe.leroy@c-s.fr \
    --cc=daniel@iogearbox.net \
    --cc=dave@stgolabs.net \
    --cc=davem@davemloft.net \
    --cc=decui@microsoft.com \
    --cc=ebiederm@xmission.com \
    --cc=edumazet@google.com \
    --cc=eric.dumazet@gmail.com \
    --cc=feng.tang@intel.com \
    --cc=fw@strlen.de \
    --cc=gregkh@linuxfoundation.org \
    --cc=guro@fb.com \
    --cc=gustavoars@kernel.org \
    --cc=herbert@gondor.apana.org.au \
    --cc=jakub@cloudflare.com \
    --cc=jannh@google.com \
    --cc=jasowang@redhat.com \
    --cc=kafai@fb.com \
    --cc=keescook@chromium.org \
    --cc=kirill.shutemov@linux.intel.com \
    --cc=kuba@kernel.org \
    --cc=kuznet@ms2.inr.ac.ru \
    --cc=linmiaohe@huawei.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@suse.com \
    --cc=minchan@kernel.org \
    --cc=mst@redhat.com \
    --cc=neilb@suse.de \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=pablo@netfilter.org \
    --cc=peterz@infradead.org \
    --cc=rafael@kernel.org \
    --cc=rdunlap@infradead.org \
    --cc=rppt@kernel.org \
    --cc=samitolvanen@google.com \
    --cc=shakeelb@google.com \
    --cc=steffen.klassert@secunet.com \
    --cc=tglx@linutronix.de \
    --cc=virtualization@lists.linux-foundation.org \
    --cc=walken@google.com \
    --cc=will@kernel.org \
    --cc=willemb@google.com \
    --cc=xiyou.wangcong@gmail.com \
    --cc=yoshfuji@linux-ipv6.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.