From: Michal Hocko <mhocko@kernel.org>
To: Jerome Glisse <jglisse@redhat.com>
Cc: Anshuman Khandual <khandual@linux.vnet.ibm.com>,
Mel Gorman <mgorman@suse.de>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org, vbabka@suse.cz,
minchan@kernel.org, aneesh.kumar@linux.vnet.ibm.com,
bsingharora@gmail.com, srikar@linux.vnet.ibm.com,
haren@linux.vnet.ibm.com, dave.hansen@intel.com,
dan.j.williams@intel.com
Subject: Re: [PATCH V3 0/4] Define coherent device memory node
Date: Wed, 22 Feb 2017 17:54:24 +0100 [thread overview]
Message-ID: <20170222165424.GA26472@dhcp22.suse.cz> (raw)
In-Reply-To: <20170222145915.GA4852@redhat.com>
On Wed 22-02-17 09:59:15, Jerome Glisse wrote:
> On Wed, Feb 22, 2017 at 10:29:21AM +0100, Michal Hocko wrote:
> > On Tue 21-02-17 18:39:17, Anshuman Khandual wrote:
> > > On 02/17/2017 07:02 PM, Mel Gorman wrote:
>
> [...]
>
> > [...]
> > > These are the reasons which prohibit the use of HMM for coherent
> > > addressable device memory purpose.
> > >
> > [...]
> > > (3) Application cannot directly allocate into device memory from user
> > > space using existing memory related system calls like mmap() and mbind()
> > > as the device memory hides away in ZONE_DEVICE.
> >
> > Why cannot the application simply use mmap on the device file?
>
> This has been said before but we want to share the address space this do
> imply that you can not rely on special allocator. For instance you can
> have an application that use a library and the library use the GPU but
> the application is un-aware and those any data provided by the application
> to the library will come from generic malloc (mmap anonymous or from
> regular file).
>
> Currently what happens is that the library reallocate memory through
> special allocator and copy thing. Not only does this waste memory (the
> new memory is often regular memory too) but you also have to paid the
> cost of copying GB of data.
>
> Last bullet to this, is complex data structure (list, tree, ...) having
> to go through special allocator means you have re-build the whole structure
> with the duplicated memory.
>
>
> Allowing to directly use memory allocated from malloc (mmap anonymous
> private or from a regular file) avoid the copy operation and the complex
> duplication of data structure. Moving the dataset to the GPU is then a
> simple memory migration from kernel point of view.
>
> This is share address space without special allocator is mandatory in new
> or future standard such as OpenCL, Cuda, C++, OpenMP, ... some other OS
> already have this and the industry want it. So the questions is do we
> want to support any of this, do we care about GPGPU ?
>
>
> I believe we want to support all this new standard but maybe i am the
> only one.
>
> In HMM case i have the extra painfull fact that the device memory is
> not accessible by the CPU. For CDM on contrary, CPU can access in a
> cache coherent way the device memory and all operation behave as regular
> memory (thing like atomic operation for instance).
>
>
> I hope this clearly explain why we can no longer rely on dedicated/
> specialized memory allocator.
Yes this clarifies this point. Thanks for the information which would be
really helpful in the initial description. Maybe I've just missed it,
though.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2017-02-22 16:54 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-02-15 12:07 [PATCH V3 0/4] Define coherent device memory node Anshuman Khandual
2017-02-15 12:07 ` [PATCH V3 1/4] mm: Define coherent device memory (CDM) node Anshuman Khandual
2017-02-17 14:05 ` Bob Liu
2017-02-21 10:20 ` Anshuman Khandual
2017-02-15 12:07 ` [PATCH V3 2/4] mm: Enable HugeTLB allocation isolation for CDM nodes Anshuman Khandual
2017-02-15 12:07 ` [PATCH V3 3/4] mm: Add new parameter to get_page_from_freelist() function Anshuman Khandual
2017-02-15 12:07 ` [PATCH V3 4/4] mm: Enable Buddy allocation isolation for CDM nodes Anshuman Khandual
2017-02-15 18:20 ` [PATCH V3 0/4] Define coherent device memory node Mel Gorman
2017-02-16 22:14 ` Balbir Singh
2017-02-17 9:33 ` Mel Gorman
2017-02-21 2:57 ` Balbir Singh
2017-03-01 2:42 ` Balbir Singh
2017-03-01 9:55 ` Mel Gorman
2017-03-01 10:59 ` Balbir Singh
2017-03-08 9:04 ` Anshuman Khandual
2017-03-08 9:21 ` [PATCH 1/2] mm: Change generic FALLBACK zonelist creation process Anshuman Khandual
2017-03-08 11:07 ` John Hubbard
2017-03-14 13:33 ` Anshuman Khandual
2017-03-15 4:10 ` John Hubbard
2017-03-08 9:21 ` [PATCH 2/2] mm: Change mbind(MPOL_BIND) implementation for CDM nodes Anshuman Khandual
2017-02-17 11:41 ` [PATCH V3 0/4] Define coherent device memory node Anshuman Khandual
2017-02-17 13:32 ` Mel Gorman
2017-02-21 13:09 ` Anshuman Khandual
2017-02-21 20:14 ` Jerome Glisse
2017-02-23 8:14 ` Anshuman Khandual
2017-02-23 15:27 ` Jerome Glisse
2017-02-22 9:29 ` Michal Hocko
2017-02-22 14:59 ` Jerome Glisse
2017-02-22 16:54 ` Michal Hocko [this message]
2017-03-06 5:48 ` Anshuman Khandual
2017-02-23 8:52 ` Anshuman Khandual
2017-02-23 15:57 ` Mel Gorman
2017-03-06 5:12 ` Anshuman Khandual
2017-02-21 11:11 ` Michal Hocko
2017-02-21 13:39 ` Anshuman Khandual
2017-02-22 9:50 ` Michal Hocko
2017-02-23 6:52 ` Anshuman Khandual
2017-03-05 12:39 ` Anshuman Khandual
2017-02-24 1:06 ` Bob Liu
2017-02-24 4:39 ` John Hubbard
2017-02-24 4:53 ` Jerome Glisse
2017-02-27 1:56 ` Bob Liu
2017-02-27 5:41 ` Anshuman Khandual
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170222165424.GA26472@dhcp22.suse.cz \
--to=mhocko@kernel.org \
--cc=aneesh.kumar@linux.vnet.ibm.com \
--cc=bsingharora@gmail.com \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@intel.com \
--cc=haren@linux.vnet.ibm.com \
--cc=jglisse@redhat.com \
--cc=khandual@linux.vnet.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=minchan@kernel.org \
--cc=srikar@linux.vnet.ibm.com \
--cc=vbabka@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).