linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Eric Dumazet <eric.dumazet@gmail.com>
To: Muchun Song <songmuchun@bytedance.com>,
	Eric Dumazet <eric.dumazet@gmail.com>
Cc: Eric Dumazet <edumazet@google.com>,
	Cong Wang <xiyou.wangcong@gmail.com>,
	Greg KH <gregkh@linuxfoundation.org>,
	rafael@kernel.org, "Michael S. Tsirkin" <mst@redhat.com>,
	Jason Wang <jasowang@redhat.com>,
	David Miller <davem@davemloft.net>,
	Jakub Kicinski <kuba@kernel.org>,
	Alexey Dobriyan <adobriyan@gmail.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>,
	Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>,
	Steffen Klassert <steffen.klassert@secunet.com>,
	Herbert Xu <herbert@gondor.apana.org.au>,
	Shakeel Butt <shakeelb@google.com>, Will Deacon <will@kernel.org>,
	Michal Hocko <mhocko@suse.com>, Roman Gushchin <guro@fb.com>,
	Neil Brown <neilb@suse.de>,
	rppt@kernel.org, Sami Tolvanen <samitolvanen@google.com>,
	"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
	Feng Tang <feng.tang@intel.com>, Paolo Abeni <pabeni@redhat.com>,
	Willem de Bruijn <willemb@google.com>,
	Randy Dunlap <rdunlap@infradead.org>,
	Florian Westphal <fw@strlen.de>,
	gustavoars@kernel.org, Pablo Neira Ayuso <pablo@netfilter.org>,
	Dexuan Cui <decui@microsoft.com>,
	Jakub Sitnicki <jakub@cloudflare.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Christian Brauner <christian.brauner@ubuntu.com>,
	"Eric W. Biederman" <ebiederm@xmission.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	dave@stgolabs.net, Michel Lespinasse <walken@google.com>,
	Jann Horn <jannh@google.com>,
	chenqiwu@xiaomi.com, christophe.leroy@c-s.fr,
	Minchan Kim <minchan@kernel.org>, Martin KaFai Lau <kafai@fb.com>,
	Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Miaohe Lin <linmiaohe@huawei.com>,
	Kees Cook <keescook@chromium.org>,
	LKML <linux-kernel@vger.kernel.org>,
	virtualization@lists.linux-foundation.org,
	Linux Kernel Network Developers <netdev@vger.kernel.org>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	linux-mm <linux-mm@kvack.org>
Subject: Re: [External] Re: [PATCH] mm: proc: add Sock to /proc/meminfo
Date: Tue, 13 Oct 2020 08:55:29 +0200	[thread overview]
Message-ID: <93d5ad90-2bd5-07ad-618e-456ed2e6da87@gmail.com> (raw)
In-Reply-To: <CAMZfGtVF6OjNuJFUExRMY1k-EaDS744=nKy6_a2cYdrJRncTgQ@mail.gmail.com>



On 10/12/20 11:53 AM, Muchun Song wrote:

> We are not complaining about TCP using too much memory, but how do
> we know that TCP uses a lot of memory. When I firstly face this problem,
> I do not know who uses the 25GB memory and it is not shown in the /proc/meminfo.
> If we can know the amount memory of the socket buffer via /proc/meminfo, we
> may not need to spend a lot of time troubleshooting this problem. Not everyone
> knows that a lot of memory may be used here. But I believe many people
> should know /proc/meminfo to confirm memory users.

Adding yet another operations in networking fast path is a high cost to pay
just to add one extra line in /proc/meminfo, while /proc/net/sockstat
is already a good proxy, with per protocol details, instead of a single bucket.

I reiterate that zero copy would make this counter out of sync,
unless special support is added (adding yet another operations ?)

Also your patch does not address gazillions of page allocations from drivers
in RX path.

Here at Google the majority of networking mm usage when hosts are under stress
is in RX path, when out of order queues start to grow in TCP sockets.

Allocations in TX path were greatly reduced and optimally sized with the introduction
of /proc/sys/net/ipv4/tcp_notsent_lowat.

We have gazillions of put_page()/__free_page()/__free_pages()/alloc_page()/... all
over the places, adding yet another tracking of "this page is used by networking stacks"
is going to be quite a big project.

I thought memcg was a better goal in the long run, lets focus on it.




  parent reply	other threads:[~2020-10-13  6:55 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-10 10:38 [PATCH] mm: proc: add Sock to /proc/meminfo Muchun Song
2020-10-10 16:36 ` Randy Dunlap
2020-10-11  4:42   ` [External] " Muchun Song
2020-10-11 13:52 ` Mike Rapoport
2020-10-11 16:00   ` [External] " Muchun Song
2020-10-11 18:39 ` Cong Wang
2020-10-12  4:22   ` [External] " Muchun Song
2020-10-12  7:42     ` Eric Dumazet
2020-10-12  8:39       ` Muchun Song
2020-10-12  9:24         ` Eric Dumazet
2020-10-12  9:53           ` Muchun Song
2020-10-12 22:12             ` Cong Wang
2020-10-13  3:52               ` Muchun Song
2020-10-13  6:55             ` Eric Dumazet [this message]
2020-10-13  8:09             ` Mike Rapoport
2020-10-13 14:43               ` Randy Dunlap
2020-10-13 15:12                 ` Mike Rapoport
2020-10-13 15:21                   ` Randy Dunlap
2020-10-14  5:34                     ` Mike Rapoport
2020-10-13 15:28               ` Muchun Song
2020-10-16 15:38               ` Vlastimil Babka
2020-10-16 20:53                 ` Minchan Kim
2020-10-19 17:23                   ` Shakeel Butt
2020-10-12 21:46     ` Cong Wang
2020-10-13  3:29       ` Muchun Song

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=93d5ad90-2bd5-07ad-618e-456ed2e6da87@gmail.com \
    --to=eric.dumazet@gmail.com \
    --cc=adobriyan@gmail.com \
    --cc=akpm@linux-foundation.org \
    --cc=ast@kernel.org \
    --cc=chenqiwu@xiaomi.com \
    --cc=christian.brauner@ubuntu.com \
    --cc=christophe.leroy@c-s.fr \
    --cc=daniel@iogearbox.net \
    --cc=dave@stgolabs.net \
    --cc=davem@davemloft.net \
    --cc=decui@microsoft.com \
    --cc=ebiederm@xmission.com \
    --cc=edumazet@google.com \
    --cc=feng.tang@intel.com \
    --cc=fw@strlen.de \
    --cc=gregkh@linuxfoundation.org \
    --cc=guro@fb.com \
    --cc=gustavoars@kernel.org \
    --cc=herbert@gondor.apana.org.au \
    --cc=jakub@cloudflare.com \
    --cc=jannh@google.com \
    --cc=jasowang@redhat.com \
    --cc=kafai@fb.com \
    --cc=keescook@chromium.org \
    --cc=kirill.shutemov@linux.intel.com \
    --cc=kuba@kernel.org \
    --cc=kuznet@ms2.inr.ac.ru \
    --cc=linmiaohe@huawei.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@suse.com \
    --cc=minchan@kernel.org \
    --cc=mst@redhat.com \
    --cc=neilb@suse.de \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=pablo@netfilter.org \
    --cc=peterz@infradead.org \
    --cc=rafael@kernel.org \
    --cc=rdunlap@infradead.org \
    --cc=rppt@kernel.org \
    --cc=samitolvanen@google.com \
    --cc=shakeelb@google.com \
    --cc=songmuchun@bytedance.com \
    --cc=steffen.klassert@secunet.com \
    --cc=tglx@linutronix.de \
    --cc=virtualization@lists.linux-foundation.org \
    --cc=walken@google.com \
    --cc=will@kernel.org \
    --cc=willemb@google.com \
    --cc=xiyou.wangcong@gmail.com \
    --cc=yoshfuji@linux-ipv6.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).