All of lore.kernel.org
 help / color / mirror / Atom feed
From: Barry Song <21cnbao@gmail.com>
To: Shijie Huang <shijie@amperemail.onmicrosoft.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	viro@zeniv.linux.org.uk,
	Andrew Morton <akpm@linux-foundation.org>,
	linux-mm@kvack.org, Barry Song <song.bao.hua@hisilicon.com>,
	LKML <linux-kernel@vger.kernel.org>,
	Frank Wang <zwang@amperecomputing.com>
Subject: Re: Is it possible to implement the per-node page cache for programs/libraries?
Date: Wed, 1 Sep 2021 10:09:24 +0800	[thread overview]
Message-ID: <CAGsJ_4xtsaQL47D2D6PyW3vAr3+EwJqv-G-1V4-4GAYd2LnT3Q@mail.gmail.com> (raw)
In-Reply-To: <a2f423cf-9413-6bc8-e4d8-92374fc0449e@amperemail.onmicrosoft.com>

On Wed, Sep 1, 2021 at 11:09 AM Shijie Huang
<shijie@amperemail.onmicrosoft.com> wrote:
>
> Hi Everyone,
>
>      In the NUMA, we only have one page cache for each file. For the
> program/shared libraries, the
>
> remote-access delays longer then the  local-access.
>
> So, is it possible to implement the per-node page cache for
> programs/libraries?

as far as i know, this is an very interesting topic, we do have some
"solutions" on this.
MIPS kernel supports kernel TEXT replication:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/mips/sgi-ip27/Kconfig
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/mips/sgi-ip27/ip27-klnuma.c

config REPLICATE_KTEXT
bool "Kernel text replication support"
depends on SGI_IP27
select MAPPED_KERNEL
help
 Say Y here to enable replicating the kernel text across multiple
 nodes in a NUMA cluster.  This trades memory for speed.

for x86, RedHawk Linux(https://www.concurrent-rt.com/solutions/linux/)supports
kernel text replication.
here are some benchmark:
https://www.concurrent-rt.com/wp-content/uploads/2016/11/kernel-page-replication.pdf

For userspace, dplace from SGI can help replicate text:
https://www.spec.org/cpu2006/flags/SGI-platform.html

-r bl: specifies that text should be replicated on the NUMA node or
nodes where the process is running.
'b' indicates that binary (a.out) text should be replicated;
'l' indicates that library text should be replicated.

but all of the above except mips ktext replication are out of tree.

Please count me in if you have any solution and any pending patch.
I am interested in this topic.

>
>
>     We can do it like this:
>
>          1.) Add a new system call to control specific files to
> NUMA-aware, such as:
>
>                     set_numa_aware("/usr/lib/libc.so", enable);
>
>              After the system call, the page cache of libc.so has the
> flags "NUMA_ENABLED"
>
>
>          2.) When A new process tries to setup the MMU page table for
> libc.so, it will check
>
>               if NUMA_ENABLED is set. If it set, the kernel will give a
> page which is bind to the process's NUMA node.
>
>               By this way, we can eliminate the remote-access for
> programs/shared library.
>
>
> Is this proposal ok?  Or do you have a better idea?
>
>
> Thanks
>
> Huang Shijie

Thanks
barry

  reply	other threads:[~2021-09-01 10:09 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-01  3:07 Shijie Huang
2021-09-01  3:07 ` Shijie Huang
2021-09-01  2:09 ` Barry Song [this message]
2021-09-01  2:09   ` Barry Song
2021-09-01  3:25 ` Matthew Wilcox
2021-09-01 13:30   ` Huang Shijie
2021-09-01 14:25     ` Huang Shijie
2021-09-01 11:32       ` Matthew Wilcox
2021-09-01 23:58       ` Matthew Wilcox
2021-09-02  0:15         ` Barry Song
2021-09-02  0:15           ` Barry Song
2021-09-02  1:13           ` Linus Torvalds
2021-09-02  1:13             ` Linus Torvalds
2021-09-02 10:16         ` Huang Shijie
2021-09-02  3:25   ` Nicholas Piggin
2021-09-02 10:17     ` Matthew Wilcox
2021-09-03  7:10       ` Nicholas Piggin
2021-09-03 19:01         ` Matthew Wilcox
2021-09-03 19:08           ` Linus Torvalds
2021-09-03 19:08             ` Linus Torvalds
2021-09-06  9:56             ` Huang Shijie
2021-09-03 23:42           ` Nicholas Piggin
2021-09-01  4:55 ` Al Viro
2021-09-01 13:10   ` Huang Shijie
2021-09-01 17:24   ` Linus Torvalds
2021-09-01 17:24     ` Linus Torvalds
2021-09-01 17:29     ` Linus Torvalds
2021-09-01 17:29       ` Linus Torvalds
2021-09-01 22:56       ` Barry Song
2021-09-01 22:56         ` Barry Song
2021-09-02 10:12         ` Huang Shijie
2021-09-02 10:08       ` Huang Shijie

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAGsJ_4xtsaQL47D2D6PyW3vAr3+EwJqv-G-1V4-4GAYd2LnT3Q@mail.gmail.com \
    --to=21cnbao@gmail.com \
    --cc=akpm@linux-foundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=shijie@amperemail.onmicrosoft.com \
    --cc=song.bao.hua@hisilicon.com \
    --cc=torvalds@linux-foundation.org \
    --cc=viro@zeniv.linux.org.uk \
    --cc=zwang@amperecomputing.com \
    --subject='Re: Is it possible to implement the per-node page cache for programs/libraries?' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.