From: Jerome Glisse <jglisse@redhat.com>
To: Logan Gunthorpe <logang@deltatee.com>
Cc: Dan Williams <dan.j.williams@intel.com>,
Andi Kleen <ak@linux.intel.com>, Linux MM <linux-mm@kvack.org>,
Andrew Morton <akpm@linux-foundation.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
"Rafael J. Wysocki" <rafael@kernel.org>,
Ross Zwisler <ross.zwisler@linux.intel.com>,
Dave Hansen <dave.hansen@intel.com>,
Haggai Eran <haggaie@mellanox.com>,
balbirs@au1.ibm.com,
"Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>,
Benjamin Herrenschmidt <benh@kernel.crashing.org>,
"Kuehling, Felix" <felix.kuehling@amd.com>,
Philip.Yang@amd.com, "Koenig,
Christian" <christian.koenig@amd.com>,
"Blinzer, Paul" <Paul.Blinzer@amd.com>,
John Hubbard <jhubbard@nvidia.com>,
rcampbell@nvidia.com
Subject: Re: [RFC PATCH 02/14] mm/hms: heterogenenous memory system (HMS) documentation
Date: Tue, 4 Dec 2018 14:22:21 -0500 [thread overview]
Message-ID: <20181204192221.GG2937@redhat.com> (raw)
In-Reply-To: <de7c1099-2717-6396-bf56-c4ab4085ee83@deltatee.com>
On Tue, Dec 04, 2018 at 12:11:42PM -0700, Logan Gunthorpe wrote:
>
>
> On 2018-12-04 11:57 a.m., Jerome Glisse wrote:
> >> That sounds needlessly restrictive. Let the kernel arbitrate what
> >> memory an application gets, don't design a system where applications
> >> are hard coded to a memory type. Applications can hint, or optionally
> >> specify an override and the kernel can react accordingly.
> >
> > You do not want to randomly use non cache coherent memory inside your
> > application :) This is not gonna go well with C++ or atomic :) Yes they
> > are legitimate use case where application can decide to give up cache
> > coherency temporarily for a range of virtual address. But the application
> > needs to understand what it is doing and opt in to do that knowing full
> > well that. The version thing allows for scenario like. You do not have
> > to define a new version with every new type of memory. If your new memory
> > has all the properties of v1 than you expose it as v1 and old application
> > on the new platform will use your new memory type being non the wiser.
>
> I agree with Dan and the general idea that this version thing is really
> ugly. Define some standard attributes so the application can say "I want
> cache-coherent, high bandwidth memory". If there's some future
> new-memory attribute, then the application needs to know about it to
> request it.
So version is a bad prefix, what about type, prefixing target with a
type id. So that application that are looking for a certain type of
memory (which has a set of define properties) can select them. Having
a type file inside the directory and hopping application will read
that sysfs file is a recipies for failure from my point of view. While
having it in the directory name is making sure that the application
has some idea of what it is doing.
>
> Also, in the same vein, I think it's wrong to have the API enumerate all
> the different memory available in the system. The API should simply
> allow userspace to say it wants memory that can be accessed by a set of
> initiators with a certain set of attributes and the bind call tries to
> fulfill that or fallback on system memory/hmm migration/whatever.
We have existing application that use topology today to partition their
workload and do load balancing. Those application leverage the fact that
they are only running on a small set of known platform with known topology
here i want to provide a common API so that topology can be queried in a
standard by application.
Yes basic application will not leverage all this information and will
be happy enough with give me memory that will be fast for initiator A
and B. That can easily be implemented inside userspace library which
dumbs down the topology on behalf of application.
I believe that proposing a new infrastructure should allow for maximum
expressiveness. The HMS API in this proposal allow to express any kind
of directed graph hence i do not see any limitation going forward. At
the same time userspace library can easily dumbs this down for average
Joe/Jane application.
Cheers,
Jérôme
next prev parent reply other threads:[~2018-12-04 19:22 UTC|newest]
Thread overview: 94+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-03 23:34 [RFC PATCH 00/14] Heterogeneous Memory System (HMS) and hbind() jglisse
2018-12-03 23:34 ` [RFC PATCH 01/14] mm/hms: heterogeneous memory system (sysfs infrastructure) jglisse
2018-12-03 23:34 ` [RFC PATCH 02/14] mm/hms: heterogenenous memory system (HMS) documentation jglisse
2018-12-04 17:06 ` Andi Kleen
2018-12-04 18:24 ` Jerome Glisse
2018-12-04 18:31 ` Dan Williams
2018-12-04 18:57 ` Jerome Glisse
2018-12-04 19:11 ` Logan Gunthorpe
2018-12-04 19:22 ` Jerome Glisse [this message]
2018-12-04 19:41 ` Logan Gunthorpe
2018-12-04 20:13 ` Jerome Glisse
2018-12-04 20:30 ` Logan Gunthorpe
2018-12-04 20:59 ` Jerome Glisse
2018-12-04 21:19 ` Logan Gunthorpe
2018-12-04 21:51 ` Jerome Glisse
2018-12-04 22:16 ` Logan Gunthorpe
2018-12-04 23:56 ` Jerome Glisse
2018-12-05 1:15 ` Logan Gunthorpe
2018-12-05 2:31 ` Jerome Glisse
2018-12-05 17:41 ` Logan Gunthorpe
2018-12-05 18:07 ` Jerome Glisse
2018-12-05 18:20 ` Logan Gunthorpe
2018-12-05 18:33 ` Jerome Glisse
2018-12-05 18:48 ` Logan Gunthorpe
2018-12-05 18:55 ` Jerome Glisse
2018-12-05 19:10 ` Logan Gunthorpe
2018-12-05 22:58 ` Jerome Glisse
2018-12-05 23:09 ` Logan Gunthorpe
2018-12-05 23:20 ` Jerome Glisse
2018-12-05 23:23 ` Logan Gunthorpe
2018-12-05 23:27 ` Jerome Glisse
2018-12-06 0:08 ` Dan Williams
2018-12-05 2:34 ` Dan Williams
2018-12-05 2:37 ` Jerome Glisse
2018-12-05 17:25 ` Logan Gunthorpe
2018-12-05 18:01 ` Jerome Glisse
2018-12-04 20:14 ` Andi Kleen
2018-12-04 20:47 ` Logan Gunthorpe
2018-12-04 21:15 ` Jerome Glisse
2018-12-04 19:19 ` Dan Williams
2018-12-04 19:32 ` Jerome Glisse
2018-12-04 20:12 ` Andi Kleen
2018-12-04 20:41 ` Jerome Glisse
2018-12-05 4:36 ` Aneesh Kumar K.V
2018-12-05 4:41 ` Jerome Glisse
2018-12-05 10:52 ` Mike Rapoport
2018-12-03 23:34 ` [RFC PATCH 03/14] mm/hms: add target memory to heterogeneous memory system infrastructure jglisse
2018-12-03 23:34 ` [RFC PATCH 04/14] mm/hms: add initiator " jglisse
2018-12-03 23:35 ` [RFC PATCH 05/14] mm/hms: add link " jglisse
2018-12-03 23:35 ` [RFC PATCH 06/14] mm/hms: add bridge " jglisse
2018-12-03 23:35 ` [RFC PATCH 07/14] mm/hms: register main memory with heterogenenous memory system jglisse
2018-12-03 23:35 ` [RFC PATCH 08/14] mm/hms: register main CPUs " jglisse
2018-12-03 23:35 ` [RFC PATCH 09/14] mm/hms: hbind() for heterogeneous memory system (aka mbind() for HMS) jglisse
2018-12-03 23:35 ` [RFC PATCH 10/14] mm/hbind: add heterogeneous memory policy tracking infrastructure jglisse
2018-12-03 23:35 ` [RFC PATCH 11/14] mm/hbind: add bind command to heterogeneous memory policy jglisse
2018-12-03 23:35 ` [RFC PATCH 12/14] mm/hbind: add migrate command to hbind() ioctl jglisse
2018-12-03 23:35 ` [RFC PATCH 13/14] drm/nouveau: register GPU under heterogeneous memory system jglisse
2018-12-03 23:35 ` [RFC PATCH 14/14] test/hms: tests for " jglisse
2018-12-04 7:44 ` [RFC PATCH 00/14] Heterogeneous Memory System (HMS) and hbind() Aneesh Kumar K.V
2018-12-04 14:44 ` Jerome Glisse
2018-12-04 18:02 ` Dave Hansen
2018-12-04 18:49 ` Jerome Glisse
2018-12-04 18:54 ` Dave Hansen
2018-12-04 19:11 ` Jerome Glisse
2018-12-04 21:37 ` Dave Hansen
2018-12-04 21:57 ` Jerome Glisse
2018-12-04 23:58 ` Dave Hansen
2018-12-05 0:29 ` Jerome Glisse
2018-12-05 1:22 ` Kuehling, Felix
2018-12-05 11:27 ` Aneesh Kumar K.V
2018-12-05 16:09 ` Jerome Glisse
2018-12-04 23:54 ` Dave Hansen
2018-12-05 0:15 ` Jerome Glisse
2018-12-05 1:06 ` Dave Hansen
2018-12-05 2:13 ` Jerome Glisse
2018-12-05 17:27 ` Dave Hansen
2018-12-05 17:53 ` Jerome Glisse
2018-12-06 18:25 ` Dave Hansen
2018-12-06 19:20 ` Jerome Glisse
2018-12-06 19:31 ` Dave Hansen
2018-12-06 20:11 ` Logan Gunthorpe
2018-12-06 22:04 ` Dave Hansen
2018-12-06 22:39 ` Jerome Glisse
2018-12-06 23:09 ` Dave Hansen
2018-12-06 23:28 ` Logan Gunthorpe
2018-12-06 23:34 ` Dave Hansen
2018-12-06 23:38 ` Dave Hansen
2018-12-06 23:48 ` Logan Gunthorpe
2018-12-07 0:20 ` Jerome Glisse
2018-12-07 15:06 ` Jonathan Cameron
2018-12-07 19:37 ` Jerome Glisse
2018-12-07 0:15 ` Jerome Glisse
2018-12-06 20:27 ` Jerome Glisse
2018-12-06 21:46 ` Jerome Glisse
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181204192221.GG2937@redhat.com \
--to=jglisse@redhat.com \
--cc=Paul.Blinzer@amd.com \
--cc=Philip.Yang@amd.com \
--cc=ak@linux.intel.com \
--cc=akpm@linux-foundation.org \
--cc=aneesh.kumar@linux.ibm.com \
--cc=balbirs@au1.ibm.com \
--cc=benh@kernel.crashing.org \
--cc=christian.koenig@amd.com \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@intel.com \
--cc=felix.kuehling@amd.com \
--cc=haggaie@mellanox.com \
--cc=jhubbard@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=logang@deltatee.com \
--cc=rafael@kernel.org \
--cc=rcampbell@nvidia.com \
--cc=ross.zwisler@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).