From: John Hubbard <jhubbard@nvidia.com>
To: Balbir Singh <bsingharora@gmail.com>,
Andrew Morton <akpm@linux-foundation.org>
Cc: "Jérôme Glisse" <jglisse@redhat.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
linux-mm <linux-mm@kvack.org>,
"Naoya Horiguchi" <n-horiguchi@ah.jp.nec.com>,
"David Nellans" <dnellans@nvidia.com>,
"Aneesh Kumar KV" <aneesh.kumar@linux.vnet.ibm.com>,
"Reza Arbab" <arbab@linux.vnet.ibm.com>,
"Srikar Dronamraju" <srikar@linux.vnet.ibm.com>,
haren@linux.vnet.ibm.com,
"Evgeny Baskakov" <ebaskakov@nvidia.com>
Subject: Re: [HMM v17 00/14] HMM (Heterogeneous Memory Management) v17
Date: Wed, 22 Feb 2017 15:58:46 -0800 [thread overview]
Message-ID: <f748e40a-04ff-9907-f25d-dfc8d4e5e7b7@nvidia.com> (raw)
In-Reply-To: <CAKTCnzmA3B4r956GXv8UKxmCTqxdt=uoXr4KBbvzzfc=ciz03A@mail.gmail.com>
On 02/22/2017 12:27 AM, Balbir Singh wrote:
> On Wed, Feb 22, 2017 at 7:16 PM, Andrew Morton
> <akpm@linux-foundation.org> wrote:
>> On Wed, 22 Feb 2017 18:19:15 +1100 Balbir Singh <bsingharora@gmail.com> wrote:
>>>
>>> Andrew, do we expect to get this in 4.11/4.12? Just curious.
>>>
>>
>> I'll be taking a serious look after -rc1.
>>
>> The lack of reviewed-by, acked-by and tested-by is a concern. It's
>> rather odd for a patchset in the 17th revision! What's up with that?
>>
>> Have you reviewed or tested the patches?
>
> I reviewed v14/15 of the patches. Aneesh reviewed some versions as
> well. I know a few people who tested a small subset of the patches,
> I'll get them to report back as well. I think John Hubbard has been
> testing iterations as well. CC'ing other interested people as well
>
> Balbir
>
Yes, Evgeny Baskakov and I have been testing each of the posted versions. We are using both
migration and mirroring, and have a small set of multi-threaded and multi-device tests. I've been
procastinating about writing up a summary of the test results, partly because the patchset is still
changing (bug fixes, new features, API changes) and so we keep resetting our testing.
We (ahem, actually Evgeny has done most of the work) have been debugging and proposing fixes
directly to Jerome, and that email traffic with Jerome has not been CC-ing this list, so things have
looked a little quieter than they really were.
Anyway, a very rudimentary testing report:
1. What we are testing: Our latest testing (in the last few weeks) has been against Jerome's repo, here:
git://people.freedesktop.org/~glisse/linux (branch: hmm-next)
which has moved ahead from his hmm-v17 branch. hmm-next adds a few bug fixes, and a new feature
(populating CPU pages on a GPU fault). Here are the differences in summary:
$ git diff --stat hmm-v17 hmm-next
drivers/char/Kconfig | 10 +
drivers/char/Makefile | 1 +
drivers/char/hmm_dmirror.c | 1168 +++++++++++++++++++++++++++++++++++++++++++++++++++++
include/linux/migrate.h | 8 +-
include/uapi/linux/hmm_dmirror.h | 54 +++
mm/hmm.c | 6 +-
mm/migrate.c | 174 ++++++--
7 files changed, 1388 insertions(+), 33 deletions(-)
2. API: As for the driver-kernel API: this is looking OK, although of course the documentation can
be improved. As Jerome already explained, there are missing pieces functionality[1] that will be
added later, and this may change the API, but for now, OK. With this initial API, we can handle both
"device" and CPU page faults, and migrate pages around.
3. More testing plans: TODO: there are a lot of programs that can be easily modified, to use malloc
instead of a special device-centric allocator. On our list.
4. Stability: still a little shaky, as we have some pretty recent bug fixes to try out.
5. Performance: I'll send out another note for that at some point. There was a performance bug that
Jerome just recently fixed, and I want to see how it looks with that fix applied. No real surprises
though.
6. Code reviews: the large size of the patchset, plus the requirement for a complicated driver to
exercise it, makes it less likely for other people to review this patch series. It's a bit
chicken-and-eggy, too, because our UVM driver can't be checked in and shipped until the kernel API
stabilizes. heh.
-----
[1] For example, due to lacking file-backed memory support, some userspace program variables that
are file-backed (initialized globals, etc) have to be mapped (from the device) instead of migrated
to the device, on a device fault.
thanks,
john h
prev parent reply other threads:[~2017-02-22 23:59 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-01-27 22:52 [HMM v17 00/14] HMM (Heterogeneous Memory Management) v17 Jérôme Glisse
2017-01-27 22:52 ` [HMM v17 01/14] mm/memory/hotplug: convert device bool to int to allow for more flags v2 Jérôme Glisse
2017-01-27 22:52 ` [HMM v17 02/14] mm/ZONE_DEVICE/free-page: callback when page is freed v2 Jérôme Glisse
2017-01-27 22:52 ` [HMM v17 03/14] mm/ZONE_DEVICE/unaddressable: add support for un-addressable device memory v3 Jérôme Glisse
2017-01-27 22:52 ` [HMM v17 04/14] mm/ZONE_DEVICE/x86: add support for un-addressable device memory Jérôme Glisse
2017-01-27 22:52 ` [HMM v17 05/14] mm/migrate: add new boolean copy flag to migratepage() callback Jérôme Glisse
2017-01-27 22:52 ` [HMM v17 06/14] mm/migrate: new memory migration helper for use with device memory v3 Jérôme Glisse
2017-02-22 15:31 ` Reza Arbab
2017-01-27 22:52 ` [HMM v17 07/14] mm/migrate: migrate_vma() unmap page from vma while collecting pages Jérôme Glisse
2017-01-27 22:52 ` [HMM v17 08/14] mm/hmm: heterogeneous memory management (HMM for short) Jérôme Glisse
2017-01-27 22:52 ` [HMM v17 09/14] mm/hmm/mirror: mirror process address space on device with HMM helpers Jérôme Glisse
2017-03-13 22:10 ` Krishna Reddy
2017-01-27 22:52 ` [HMM v17 10/14] mm/hmm/mirror: helper to snapshot CPU page table Jérôme Glisse
2017-01-27 22:52 ` [HMM v17 11/14] mm/hmm/mirror: device page fault handler Jérôme Glisse
2017-01-27 22:52 ` [HMM v17 12/14] mm/hmm/migrate: support un-addressable ZONE_DEVICE page in migration Jérôme Glisse
2017-01-27 22:52 ` [HMM v17 13/14] mm/hmm/devmem: device memory hotplug using ZONE_DEVICE Jérôme Glisse
2017-01-27 22:52 ` [HMM v17 14/14] mm/hmm/devmem: dummy HMM device for ZONE_DEVICE memory v2 Jérôme Glisse
2017-01-28 5:39 ` [HMM v17 00/14] HMM (Heterogeneous Memory Management) v17 John Hubbard
2017-02-22 7:19 ` Balbir Singh
2017-02-22 8:16 ` Andrew Morton
2017-02-22 8:27 ` Balbir Singh
2017-02-22 15:49 ` Reza Arbab
2017-02-22 23:58 ` John Hubbard [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f748e40a-04ff-9907-f25d-dfc8d4e5e7b7@nvidia.com \
--to=jhubbard@nvidia.com \
--cc=akpm@linux-foundation.org \
--cc=aneesh.kumar@linux.vnet.ibm.com \
--cc=arbab@linux.vnet.ibm.com \
--cc=bsingharora@gmail.com \
--cc=dnellans@nvidia.com \
--cc=ebaskakov@nvidia.com \
--cc=haren@linux.vnet.ibm.com \
--cc=jglisse@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=n-horiguchi@ah.jp.nec.com \
--cc=srikar@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).