All of lore.kernel.org
 help / color / mirror / Atom feed
From: Christoph Hellwig <hch@lst.de>
To: Jason Gunthorpe <jgg@mellanox.com>
Cc: "Christoph Hellwig" <hch@lst.de>,
	"Jérôme Glisse" <jglisse@redhat.com>,
	"Ben Skeggs" <bskeggs@redhat.com>,
	"Felix Kuehling" <Felix.Kuehling@amd.com>,
	"Ralph Campbell" <rcampbell@nvidia.com>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>,
	"nouveau@lists.freedesktop.org" <nouveau@lists.freedesktop.org>,
	"dri-devel@lists.freedesktop.org"
	<dri-devel@lists.freedesktop.org>,
	"amd-gfx@lists.freedesktop.org" <amd-gfx@lists.freedesktop.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 07/13] mm: remove the page_shift member from struct hmm_range
Date: Thu, 1 Aug 2019 08:49:14 +0200	[thread overview]
Message-ID: <20190801064914.GA15404@lst.de> (raw)
In-Reply-To: <20190730175011.GL24038@mellanox.com>

On Tue, Jul 30, 2019 at 05:50:16PM +0000, Jason Gunthorpe wrote:
> The way ODP seems to work is once in hugetlb mode the dma addresses
> must give huge pages or the page fault will be failed. I think that is
> a terrible design, but this is how the driver is ..
> 
> So, from this HMM perspective if the caller asked for huge pages then
> the results have to be all huge pages or a hard failure.

Which isn't how the page_shift member works at moment.  It still
allows non-hugetlb mappings even with the member.

> It is not negotiated as an optimization like you are thinking.
> 
> [note, I haven't yet checked carefully how this works in ODP, every
>  time I look at parts of it the thing seems crazy]

This seems pretty crazy.  Especially as hugetlb use in applications
seems to fade in favour of THP, for which this ODP scheme does not seem
to work at all.

> > The best API for mlx4 would of course be to pass a biovec-style
> > variable length structure that hmm_fault could fill out, but that would
> > be a major restructure.
> 
> It would work, but the driver has to expand that into a page list
> right awayhow.
> 
> We can't even dma map the biovec with today's dma API as it needs the
> ability to remap on a page granularity.

We can do dma_map_page loops over each biovec entry pretty trivially,
and that won't be any worse than the current loop over each page in
the hmm dma helpers.  Once I get around the work to have a better
API for iommu mappings for bio_vecs we could coalesce it similar to
how we do it with scatterlist (but without all the mess of a new
structure).  That work is going to take a little longer through, as
it needs the amd and intell iommu drivers to be convered to dma-iommu
which isn't making progress as far as I hoped.

Let me know if you want to keep this code for now despite the issues,
or if we'd rather reimplement it once you've made sense of the ODP
code.

  reply	other threads:[~2019-08-01  6:49 UTC|newest]

Thread overview: 58+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-30  5:51 hmm_range_fault related fixes and legacy API removal v3 Christoph Hellwig
2019-07-30  5:51 ` [PATCH 01/13] amdgpu: remove -EAGAIN handling for hmm_range_fault Christoph Hellwig
2019-07-30 12:33   ` Jason Gunthorpe
2019-07-31 13:13   ` Kuehling, Felix
2019-07-30  5:51 ` [PATCH 02/13] amdgpu: don't initialize range->list in amdgpu_hmm_init_range Christoph Hellwig
2019-07-30  5:51   ` Christoph Hellwig
2019-07-30 12:33   ` Jason Gunthorpe
2019-07-30 12:33     ` Jason Gunthorpe
2019-07-31 13:25   ` Kuehling, Felix
2019-07-31 17:02     ` Jason Gunthorpe
2019-07-31 17:02       ` Jason Gunthorpe
2019-07-30  5:51 ` [PATCH 03/13] nouveau: pass struct nouveau_svmm to nouveau_range_fault Christoph Hellwig
2019-07-30  5:51   ` Christoph Hellwig
2019-07-30 12:35   ` Jason Gunthorpe
2019-07-30 13:10     ` Christoph Hellwig
2019-07-30 13:14       ` Jason Gunthorpe
2019-07-30 14:40         ` Christoph Hellwig
2019-07-30  5:51 ` [PATCH 04/13] mm: remove the pgmap field from struct hmm_vma_walk Christoph Hellwig
2019-07-30  5:51   ` Christoph Hellwig
2019-07-30  5:51 ` [PATCH 05/13] mm: remove the unused vma argument to hmm_range_dma_unmap Christoph Hellwig
2019-07-30  5:51   ` Christoph Hellwig
2019-07-30 12:45   ` Jason Gunthorpe
2019-07-30  5:51 ` [PATCH 06/13] mm: remove superflous arguments from hmm_range_register Christoph Hellwig
2019-07-30 17:51   ` Jason Gunthorpe
2019-07-31 13:31   ` Kuehling, Felix
2019-07-31 13:31     ` Kuehling, Felix
2019-07-30  5:51 ` [PATCH 07/13] mm: remove the page_shift member from struct hmm_range Christoph Hellwig
2019-07-30  5:51   ` Christoph Hellwig
2019-07-30 12:55   ` Jason Gunthorpe
2019-07-30 13:14     ` Christoph Hellwig
2019-07-30 17:50       ` Jason Gunthorpe
2019-08-01  6:49         ` Christoph Hellwig [this message]
2019-07-31 13:38   ` Kuehling, Felix
2019-07-30  5:51 ` [PATCH 08/13] mm: remove the mask variable in hmm_vma_walk_hugetlb_entry Christoph Hellwig
2019-07-30  5:51   ` Christoph Hellwig
2019-07-30 17:39   ` Jason Gunthorpe
2019-07-30 17:39     ` Jason Gunthorpe
2019-07-31  1:01   ` Ralph Campbell
2019-07-31  1:01     ` Ralph Campbell
2019-07-30  5:51 ` [PATCH 09/13] mm: don't abuse pte_index() in hmm_vma_handle_pmd Christoph Hellwig
2019-07-30  5:51   ` Christoph Hellwig
2019-07-30  5:52 ` [PATCH 10/13] mm: only define hmm_vma_walk_pud if needed Christoph Hellwig
2019-07-30  5:52   ` Christoph Hellwig
2019-07-30 18:02   ` Jason Gunthorpe
2019-07-30  5:52 ` [PATCH 11/13] mm: cleanup the hmm_vma_handle_pmd stub Christoph Hellwig
2019-07-30  5:52   ` Christoph Hellwig
2019-07-30 17:53   ` Jason Gunthorpe
2019-07-30 17:53     ` Jason Gunthorpe
2019-08-01  7:01     ` Christoph Hellwig
2019-07-30  5:52 ` [PATCH 12/13] mm: cleanup the hmm_vma_walk_hugetlb_entry stub Christoph Hellwig
2019-07-30  5:52   ` Christoph Hellwig
2019-07-30 18:02   ` Jason Gunthorpe
2019-07-30  5:52 ` [PATCH 13/13] mm: allow HMM_MIRROR on all architectures with MMU Christoph Hellwig
2019-07-30  5:52   ` Christoph Hellwig
2019-07-30 18:03   ` Jason Gunthorpe
2019-07-30 18:04     ` Jason Gunthorpe
2019-08-01  7:04       ` Christoph Hellwig
2019-08-01  7:04         ` Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190801064914.GA15404@lst.de \
    --to=hch@lst.de \
    --cc=Felix.Kuehling@amd.com \
    --cc=amd-gfx@lists.freedesktop.org \
    --cc=bskeggs@redhat.com \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=jgg@mellanox.com \
    --cc=jglisse@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=nouveau@lists.freedesktop.org \
    --cc=rcampbell@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.