From: Jerome Glisse <jglisse@redhat.com>
To: chetan L <loke.chetan@gmail.com>
Cc: Bob Liu <lliubbo@gmail.com>, David Nellans <dnellans@nvidia.com>,
John Hubbard <jhubbard@nvidia.com>,
Balbir Singh <bsingharora@gmail.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
Michal Hocko <mhocko@kernel.org>, Linux MM <linux-mm@kvack.org>,
Dan Williams <dan.j.williams@intel.com>,
Andrew Morton <akpm@linux-foundation.org>,
linux-accelerators@lists.ozlabs.org
Subject: Re: [PATCH 0/6] Cache coherent device memory (CDM) with HMM v5
Date: Thu, 16 Nov 2017 18:11:55 -0500 [thread overview]
Message-ID: <20171116231155.GA5640@redhat.com> (raw)
In-Reply-To: <CAAsGZS7oCjHuUTUAUadb+F+drp3KgDARuaOaSBbW-8RWbJBDMA@mail.gmail.com>
On Thu, Nov 16, 2017 at 02:41:39PM -0800, chetan L wrote:
> On Thu, Nov 16, 2017 at 1:29 PM, Jerome Glisse <jglisse@redhat.com> wrote:
>
> >
> > For the NUMA discussion this is related to CPU less node ie not wanting
> > to add any more CPU less node (node with only memory) and they are other
> > aspect too. For instance you do not necessarily have good informations
> > from the device to know if a page is access a lot by the device (this
> > kind of information is often only accessible by the device driver). Thus
>
> @Jerome - one comment w.r.t 'do not necessarily have good info on
> device access'.
>
> So you could be assuming a few things here :). CCIX extends the CPU
> complex's coherency domain(it is now a single/unified coherency
> domain). The CCIX-EP (lets say an accelerator/XPU or a NIC or a combo)
> is now a true peer w.r.t the host-numa-node(s) (aka 1st class
> citizen). I don't know how much info was revealed at the latest ARM
> techcon where CCIX was presented. So I cannot divulge any further
> details until I see that slide deck. However, you can safely assume
> that the host will have *all* the info w.r.t the device-access and
> vice-versa.
I do have access to CCIX, last time i read the draft, few month ago,
my understanding was that there is no mechanism to differentiate between
device behind the root complex. So when you do autonuma you don't know
which of your CCIX device is the one faulting hence you can not keep
track of that inside struct page for autonuma (ignoring the issue with
the lack of CPUID for each device).
This is what i mean by NUMA is not a good fit as it is. Yes everything
is cache coherent and all, but that is just a small part of what is
needed to make autonuma as it is today work.
Jérôme
prev parent reply other threads:[~2017-11-16 23:12 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-13 21:15 [PATCH 0/6] Cache coherent device memory (CDM) with HMM v5 Jérôme Glisse
2017-07-13 21:15 ` [PATCH 1/6] mm/zone-device: rename DEVICE_PUBLIC to DEVICE_HOST Jérôme Glisse
2017-07-17 9:09 ` Balbir Singh
2017-07-13 21:15 ` [PATCH 2/6] mm/device-public-memory: device memory cache coherent with CPU v4 Jérôme Glisse
2017-07-13 23:01 ` Balbir Singh
2017-07-13 21:15 ` [PATCH 3/6] mm/hmm: add new helper to hotplug CDM memory region v3 Jérôme Glisse
2017-07-13 21:15 ` [PATCH 4/6] mm/memcontrol: allow to uncharge page without using page->lru field Jérôme Glisse
2017-07-17 9:10 ` Balbir Singh
2017-07-13 21:15 ` [PATCH 5/6] mm/memcontrol: support MEMORY_DEVICE_PRIVATE and MEMORY_DEVICE_PUBLIC v3 Jérôme Glisse
2017-07-17 9:15 ` Balbir Singh
2017-07-13 21:15 ` [PATCH 6/6] mm/hmm: documents how device memory is accounted in rss and memcg Jérôme Glisse
2017-07-14 13:26 ` Michal Hocko
2017-07-18 3:26 ` [PATCH 0/6] Cache coherent device memory (CDM) with HMM v5 Bob Liu
2017-07-18 15:38 ` Jerome Glisse
2017-07-19 1:46 ` Bob Liu
2017-07-19 2:25 ` Jerome Glisse
2017-07-19 9:09 ` Bob Liu
2017-07-20 15:03 ` Jerome Glisse
2017-07-21 1:15 ` Bob Liu
2017-07-21 1:41 ` Jerome Glisse
2017-07-21 2:10 ` Bob Liu
2017-07-21 12:01 ` Bob Liu
2017-07-21 15:21 ` Jerome Glisse
2017-07-21 3:48 ` Dan Williams
2017-07-21 15:22 ` Jerome Glisse
2017-09-05 19:36 ` Jerome Glisse
2017-09-09 23:22 ` Bob Liu
2017-09-11 23:36 ` Jerome Glisse
2017-09-12 1:02 ` Bob Liu
2017-09-12 16:17 ` Jerome Glisse
2017-09-26 9:56 ` Bob Liu
2017-09-26 16:16 ` Jerome Glisse
2017-09-30 2:57 ` Bob Liu
2017-09-30 22:49 ` Jerome Glisse
2017-10-11 13:15 ` Bob Liu
2017-10-12 15:37 ` Jerome Glisse
2017-11-16 2:10 ` chet l
2017-11-16 2:44 ` Jerome Glisse
2017-11-16 3:23 ` chetan L
2017-11-16 3:29 ` chetan L
2017-11-16 21:29 ` Jerome Glisse
2017-11-16 22:41 ` chetan L
2017-11-16 23:11 ` Jerome Glisse [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171116231155.GA5640@redhat.com \
--to=jglisse@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=bsingharora@gmail.com \
--cc=dan.j.williams@intel.com \
--cc=dnellans@nvidia.com \
--cc=jhubbard@nvidia.com \
--cc=linux-accelerators@lists.ozlabs.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=lliubbo@gmail.com \
--cc=loke.chetan@gmail.com \
--cc=mhocko@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).