From: Logan Gunthorpe <logang@deltatee.com> To: John Hubbard <jhubbard@nvidia.com>, Matthew Wilcox <willy@infradead.org>, linux-kernel@vger.kernel.org Cc: Christoph Hellwig <hch@lst.de>, Jason Gunthorpe <jgg@nvidia.com>, Joao Martins <joao.m.martins@oracle.com>, Ming Lei <ming.lei@redhat.com>, linux-block@vger.kernel.org, netdev@vger.kernel.org, linux-mm@kvack.org, linux-rdma@vger.kernel.org, dri-devel@lists.freedesktop.org, nvdimm@lists.linux.dev Subject: Re: Phyr Starter Date: Tue, 11 Jan 2022 10:31:42 -0700 [thread overview] Message-ID: <305b0b3b-e5d1-3dc2-a4a3-01c05dda6748@deltatee.com> (raw) In-Reply-To: <82989486-3780-f0aa-c13d-994e97d4ac89@nvidia.com> On 2022-01-11 1:17 a.m., John Hubbard wrote: > On 1/10/22 11:34, Matthew Wilcox wrote: >> TLDR: I want to introduce a new data type: >> >> struct phyr { >> phys_addr_t addr; >> size_t len; >> }; >> >> and use it to replace bio_vec as well as using it to replace the array >> of struct pages used by get_user_pages() and friends. >> >> --- > > This would certainly solve quite a few problems at once. Very compelling. I agree. > Zooming in on the pinning aspect for a moment: last time I attempted to > convert O_DIRECT callers from gup to pup, I recall wanting very much to > record, in each bio_vec, whether these pages were acquired via FOLL_PIN, > or some non-FOLL_PIN method. Because at the end of the IO, it is not > easy to disentangle which pages require put_page() and which require > unpin_user_page*(). > > And changing the bio_vec for *that* purpose was not really acceptable. > > But now that you're looking to change it in a big way (and with some > spare bits avaiable...oohh!), maybe I can go that direction after all. > > Or, are you looking at a design in which any phyr is implicitly FOLL_PIN'd > if it exists at all? I'd also second being able to store a handful of flags in each phyr. My userspace P2PDMA patchset needs to add a flag to each sgl to indicate whether it was mapped as a bus address or not (which would be necessary for the DMA mapped side dma_map_phyr). Though, it's not immediately obvious where to put the flags without increasing the size of the structure :( Logan
WARNING: multiple messages have this Message-ID (diff)
From: Logan Gunthorpe <logang@deltatee.com> To: John Hubbard <jhubbard@nvidia.com>, Matthew Wilcox <willy@infradead.org>, linux-kernel@vger.kernel.org Cc: nvdimm@lists.linux.dev, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, dri-devel@lists.freedesktop.org, Ming Lei <ming.lei@redhat.com>, linux-block@vger.kernel.org, linux-mm@kvack.org, Jason Gunthorpe <jgg@nvidia.com>, Joao Martins <joao.m.martins@oracle.com>, Christoph Hellwig <hch@lst.de> Subject: Re: Phyr Starter Date: Tue, 11 Jan 2022 10:31:42 -0700 [thread overview] Message-ID: <305b0b3b-e5d1-3dc2-a4a3-01c05dda6748@deltatee.com> (raw) In-Reply-To: <82989486-3780-f0aa-c13d-994e97d4ac89@nvidia.com> On 2022-01-11 1:17 a.m., John Hubbard wrote: > On 1/10/22 11:34, Matthew Wilcox wrote: >> TLDR: I want to introduce a new data type: >> >> struct phyr { >> phys_addr_t addr; >> size_t len; >> }; >> >> and use it to replace bio_vec as well as using it to replace the array >> of struct pages used by get_user_pages() and friends. >> >> --- > > This would certainly solve quite a few problems at once. Very compelling. I agree. > Zooming in on the pinning aspect for a moment: last time I attempted to > convert O_DIRECT callers from gup to pup, I recall wanting very much to > record, in each bio_vec, whether these pages were acquired via FOLL_PIN, > or some non-FOLL_PIN method. Because at the end of the IO, it is not > easy to disentangle which pages require put_page() and which require > unpin_user_page*(). > > And changing the bio_vec for *that* purpose was not really acceptable. > > But now that you're looking to change it in a big way (and with some > spare bits avaiable...oohh!), maybe I can go that direction after all. > > Or, are you looking at a design in which any phyr is implicitly FOLL_PIN'd > if it exists at all? I'd also second being able to store a handful of flags in each phyr. My userspace P2PDMA patchset needs to add a flag to each sgl to indicate whether it was mapped as a bus address or not (which would be necessary for the DMA mapped side dma_map_phyr). Though, it's not immediately obvious where to put the flags without increasing the size of the structure :( Logan
next prev parent reply other threads:[~2022-01-11 18:16 UTC|newest] Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-01-10 19:34 Phyr Starter Matthew Wilcox 2022-01-10 19:34 ` Matthew Wilcox 2022-01-11 0:41 ` Jason Gunthorpe 2022-01-11 0:41 ` Jason Gunthorpe 2022-01-11 4:32 ` Matthew Wilcox 2022-01-11 4:32 ` Matthew Wilcox 2022-01-11 15:01 ` Jason Gunthorpe 2022-01-11 15:01 ` Jason Gunthorpe 2022-01-11 18:33 ` Matthew Wilcox 2022-01-11 18:33 ` Matthew Wilcox 2022-01-11 20:21 ` Jason Gunthorpe 2022-01-11 20:21 ` Jason Gunthorpe 2022-01-11 21:25 ` Matthew Wilcox 2022-01-11 21:25 ` Matthew Wilcox 2022-01-11 22:09 ` Logan Gunthorpe 2022-01-11 22:09 ` Logan Gunthorpe 2022-01-11 22:57 ` Jason Gunthorpe 2022-01-11 22:57 ` Jason Gunthorpe 2022-01-11 23:02 ` Logan Gunthorpe 2022-01-11 23:02 ` Logan Gunthorpe 2022-01-11 22:53 ` Jason Gunthorpe 2022-01-11 22:53 ` Jason Gunthorpe 2022-01-11 22:57 ` Logan Gunthorpe 2022-01-11 22:57 ` Logan Gunthorpe 2022-01-11 23:02 ` Jason Gunthorpe 2022-01-11 23:02 ` Jason Gunthorpe 2022-01-11 23:08 ` Logan Gunthorpe 2022-01-11 23:08 ` Logan Gunthorpe 2022-01-12 18:37 ` Matthew Wilcox 2022-01-12 18:37 ` Matthew Wilcox 2022-01-12 19:08 ` Jason Gunthorpe 2022-01-12 19:08 ` Jason Gunthorpe 2022-01-20 14:03 ` Christoph Hellwig 2022-01-20 17:17 ` Jason Gunthorpe 2022-01-20 17:17 ` Jason Gunthorpe 2022-01-20 14:00 ` Christoph Hellwig 2022-01-11 9:05 ` Daniel Vetter 2022-01-11 9:05 ` Daniel Vetter 2022-01-11 20:26 ` Jason Gunthorpe 2022-01-11 20:26 ` Jason Gunthorpe 2022-01-20 14:09 ` Christoph Hellwig 2022-01-20 13:56 ` Christoph Hellwig 2022-01-20 15:27 ` Keith Busch 2022-01-20 15:27 ` Keith Busch 2022-01-20 15:28 ` Christoph Hellwig 2022-01-20 17:54 ` Robin Murphy 2022-01-11 8:17 ` John Hubbard 2022-01-11 8:17 ` John Hubbard 2022-01-11 14:01 ` Matthew Wilcox 2022-01-11 14:01 ` Matthew Wilcox 2022-01-11 15:02 ` Jason Gunthorpe 2022-01-11 15:02 ` Jason Gunthorpe 2022-01-11 17:31 ` Logan Gunthorpe [this message] 2022-01-11 17:31 ` Logan Gunthorpe 2022-01-20 14:12 ` Christoph Hellwig 2022-01-20 21:35 ` John Hubbard 2022-01-20 21:35 ` John Hubbard 2022-01-11 11:40 ` Thomas Zimmermann 2022-01-11 13:56 ` Matthew Wilcox 2022-01-11 13:56 ` Matthew Wilcox 2022-01-11 14:10 ` Thomas Zimmermann 2022-01-20 13:39 ` Christoph Hellwig
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=305b0b3b-e5d1-3dc2-a4a3-01c05dda6748@deltatee.com \ --to=logang@deltatee.com \ --cc=dri-devel@lists.freedesktop.org \ --cc=hch@lst.de \ --cc=jgg@nvidia.com \ --cc=jhubbard@nvidia.com \ --cc=joao.m.martins@oracle.com \ --cc=linux-block@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-rdma@vger.kernel.org \ --cc=ming.lei@redhat.com \ --cc=netdev@vger.kernel.org \ --cc=nvdimm@lists.linux.dev \ --cc=willy@infradead.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.