From: John Hubbard <jhubbard@nvidia.com> To: Tom Talpey <tom@talpey.com>, john.hubbard@gmail.com, linux-mm@kvack.org Cc: Andrew Morton <akpm@linux-foundation.org>, LKML <linux-kernel@vger.kernel.org>, linux-rdma <linux-rdma@vger.kernel.org>, linux-fsdevel@vger.kernel.org Subject: Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages Date: Tue, 27 Nov 2018 18:52:28 -0800 [thread overview] Message-ID: <313bf82d-cdeb-8c75-3772-7a124ecdfbd5@nvidia.com> (raw) In-Reply-To: <15e4a0c0-cadd-e549-962f-8d9aa9fc033a@talpey.com> On 11/27/18 5:21 PM, Tom Talpey wrote: > On 11/21/2018 5:06 PM, John Hubbard wrote: >> On 11/21/18 8:49 AM, Tom Talpey wrote: >>> On 11/21/2018 1:09 AM, John Hubbard wrote: >>>> On 11/19/18 10:57 AM, Tom Talpey wrote: [...] >>> >>> What I'd really like to see is to go back to the original fio parameters >>> (1 thread, 64 iodepth) and try to get a result that gets at least close >>> to the speced 200K IOPS of the NVMe device. There seems to be something >>> wrong with yours, currently. >> >> I'll dig into what has gone wrong with the test. I see fio putting data files >> in the right place, so the obvious "using the wrong drive" is (probably) >> not it. Even though it really feels like that sort of thing. We'll see. >> >>> >>> Then of course, the result with the patched get_user_pages, and >>> compare whichever of IOPS or CPU% changes, and how much. >>> >>> If these are within a few percent, I agree it's good to go. If it's >>> roughly 25% like the result just above, that's a rocky road. >>> >>> I can try this after the holiday on some basic hardware and might >>> be able to scrounge up better. Can you post that github link? >>> >> >> Here: >> >> git@github.com:johnhubbard/linux (branch: gup_dma_testing) > > I'm super-limited here this week hardware-wise and have not been able > to try testing with the patched kernel. > > I was able to compare my earlier quick test with a Bionic 4.15 kernel > (400K IOPS) against a similar 4.20rc3 kernel, and the rate dropped to > ~_375K_ IOPS. Which I found perhaps troubling. But it was only a quick > test, and without your change. > So just to double check (again): you are running fio with these parameters, right? [reader] direct=1 ioengine=libaio blocksize=4096 size=1g numjobs=1 rw=read iodepth=64 > Say, that branch reports it has not had a commit since June 30. Is that > the right one? What about gup_dma_for_lpc_2018? > That's the right branch, but the AuthorDate for the head commit (only) somehow got stuck in the past. I just now amended that patch with a new date and pushed it, so the head commit now shows Nov 27: https://github.com/johnhubbard/linux/commits/gup_dma_testing The actual code is the same, though. (It is still based on Nov 19th's f2ce1065e767 commit.) thanks, -- John Hubbard NVIDIA
WARNING: multiple messages have this Message-ID (diff)
From: John Hubbard <jhubbard@nvidia.com> To: Tom Talpey <tom@talpey.com>, <john.hubbard@gmail.com>, <linux-mm@kvack.org> Cc: Andrew Morton <akpm@linux-foundation.org>, LKML <linux-kernel@vger.kernel.org>, linux-rdma <linux-rdma@vger.kernel.org>, <linux-fsdevel@vger.kernel.org> Subject: Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages Date: Tue, 27 Nov 2018 18:52:28 -0800 [thread overview] Message-ID: <313bf82d-cdeb-8c75-3772-7a124ecdfbd5@nvidia.com> (raw) In-Reply-To: <15e4a0c0-cadd-e549-962f-8d9aa9fc033a@talpey.com> On 11/27/18 5:21 PM, Tom Talpey wrote: > On 11/21/2018 5:06 PM, John Hubbard wrote: >> On 11/21/18 8:49 AM, Tom Talpey wrote: >>> On 11/21/2018 1:09 AM, John Hubbard wrote: >>>> On 11/19/18 10:57 AM, Tom Talpey wrote: [...] >>> >>> What I'd really like to see is to go back to the original fio parameters >>> (1 thread, 64 iodepth) and try to get a result that gets at least close >>> to the speced 200K IOPS of the NVMe device. There seems to be something >>> wrong with yours, currently. >> >> I'll dig into what has gone wrong with the test. I see fio putting data files >> in the right place, so the obvious "using the wrong drive" is (probably) >> not it. Even though it really feels like that sort of thing. We'll see. >> >>> >>> Then of course, the result with the patched get_user_pages, and >>> compare whichever of IOPS or CPU% changes, and how much. >>> >>> If these are within a few percent, I agree it's good to go. If it's >>> roughly 25% like the result just above, that's a rocky road. >>> >>> I can try this after the holiday on some basic hardware and might >>> be able to scrounge up better. Can you post that github link? >>> >> >> Here: >> >> git@github.com:johnhubbard/linux (branch: gup_dma_testing) > > I'm super-limited here this week hardware-wise and have not been able > to try testing with the patched kernel. > > I was able to compare my earlier quick test with a Bionic 4.15 kernel > (400K IOPS) against a similar 4.20rc3 kernel, and the rate dropped to > ~_375K_ IOPS. Which I found perhaps troubling. But it was only a quick > test, and without your change. > So just to double check (again): you are running fio with these parameters, right? [reader] direct=1 ioengine=libaio blocksize=4096 size=1g numjobs=1 rw=read iodepth=64 > Say, that branch reports it has not had a commit since June 30. Is that > the right one? What about gup_dma_for_lpc_2018? > That's the right branch, but the AuthorDate for the head commit (only) somehow got stuck in the past. I just now amended that patch with a new date and pushed it, so the head commit now shows Nov 27: https://github.com/johnhubbard/linux/commits/gup_dma_testing The actual code is the same, though. (It is still based on Nov 19th's f2ce1065e767 commit.) thanks, -- John Hubbard NVIDIA
next prev parent reply other threads:[~2018-11-28 2:52 UTC|newest] Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-11-10 8:50 [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages john.hubbard 2018-11-10 8:50 ` [PATCH v2 1/6] mm/gup: finish consolidating error handling john.hubbard 2018-11-12 15:41 ` Keith Busch 2018-11-12 16:14 ` Dan Williams 2018-11-15 0:45 ` John Hubbard 2018-11-10 8:50 ` [PATCH v2 2/6] mm: introduce put_user_page*(), placeholder versions john.hubbard 2018-11-11 14:10 ` Mike Rapoport 2018-11-10 8:50 ` [PATCH v2 3/6] infiniband/mm: convert put_page() to put_user_page*() john.hubbard 2018-11-10 8:50 ` [PATCH v2 4/6] mm: introduce page->dma_pinned_flags, _count john.hubbard 2018-11-10 8:50 ` [PATCH v2 5/6] mm: introduce zone_gup_lock, for dma-pinned pages john.hubbard 2018-11-10 8:50 ` [PATCH v2 6/6] mm: track gup pages with page->dma_pinned_* fields john.hubbard 2018-11-12 13:58 ` Jan Kara 2018-11-15 6:28 ` [LKP] [mm] 0e9755bfa2: kernel_BUG_at_include/linux/mm.h kernel test robot 2018-11-15 6:28 ` kernel test robot 2018-11-19 18:57 ` [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages Tom Talpey 2018-11-21 6:09 ` John Hubbard 2018-11-21 6:09 ` John Hubbard 2018-11-21 16:49 ` Tom Talpey 2018-11-21 22:06 ` John Hubbard 2018-11-21 22:06 ` John Hubbard 2018-11-28 1:21 ` Tom Talpey 2018-11-28 2:52 ` John Hubbard [this message] 2018-11-28 2:52 ` John Hubbard 2018-11-28 13:59 ` Tom Talpey 2018-11-30 1:39 ` John Hubbard 2018-11-30 1:39 ` John Hubbard 2018-11-30 2:18 ` Tom Talpey 2018-11-30 2:21 ` John Hubbard 2018-11-30 2:21 ` John Hubbard 2018-11-30 2:30 ` Tom Talpey 2018-11-30 3:00 ` John Hubbard 2018-11-30 3:00 ` John Hubbard 2018-11-30 3:14 ` Tom Talpey
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=313bf82d-cdeb-8c75-3772-7a124ecdfbd5@nvidia.com \ --to=jhubbard@nvidia.com \ --cc=akpm@linux-foundation.org \ --cc=john.hubbard@gmail.com \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-rdma@vger.kernel.org \ --cc=tom@talpey.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.