From: Daniel Vetter <daniel.vetter@ffwll.ch>
To: Oded Gabbay <oded.gabbay@gmail.com>
Cc: "DRI Development" <dri-devel@lists.freedesktop.org>,
LKML <linux-kernel@vger.kernel.org>,
"KVM list" <kvm@vger.kernel.org>, linux-mm <linux-mm@kvack.org>,
"list@263.net:IOMMU DRIVERS <iommu@lists.linux-foundation.org>,
Joerg Roedel <joro@8bytes.org>,"
<linux-arm-kernel@lists.infradead.org>,
linux-samsung-soc <linux-samsung-soc@vger.kernel.org>,
"Linux Media Mailing List" <linux-media@vger.kernel.org>,
linux-s390 <linux-s390@vger.kernel.org>,
"Daniel Vetter" <daniel.vetter@intel.com>,
"Jason Gunthorpe" <jgg@ziepe.ca>,
"Andrew Morton" <akpm@linux-foundation.org>,
"John Hubbard" <jhubbard@nvidia.com>,
"Jérôme Glisse" <jglisse@redhat.com>, "Jan Kara" <jack@suse.cz>,
"Dan Williams" <dan.j.williams@intel.com>,
"Omer Shpigelman" <oshpigelman@habana.ai>,
"Ofir Bitton" <obitton@habana.ai>,
"Tomer Tayar" <ttayar@habana.ai>,
"Moti Haimovski" <mhaimovski@habana.ai>,
"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
"Pawel Piskorski" <ppiskorski@habana.ai>
Subject: Re: [PATCH v2 03/17] misc/habana: Stop using frame_vector helpers
Date: Sat, 10 Oct 2020 23:32:03 +0200 [thread overview]
Message-ID: <CAKMK7uG_kBpmuQDRgKdyh8SycFDhE7kuB2MEOsx+D5wRmerWKA@mail.gmail.com> (raw)
In-Reply-To: <CAFCwf1194Ce98y8tWxKzXT1rsdHDkzEcnERiaU=3-=t7hygmXg@mail.gmail.com>
On Sat, Oct 10, 2020 at 10:27 PM Oded Gabbay <oded.gabbay@gmail.com> wrote:
>
> On Fri, Oct 9, 2020 at 10:59 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> >
> > All we need are a pages array, pin_user_pages_fast can give us that
> > directly. Plus this avoids the entire raw pfn side of get_vaddr_frames.
> >
> Thanks for the patch Daniel.
>
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > Cc: Jason Gunthorpe <jgg@ziepe.ca>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: John Hubbard <jhubbard@nvidia.com>
> > Cc: Jérôme Glisse <jglisse@redhat.com>
> > Cc: Jan Kara <jack@suse.cz>
> > Cc: Dan Williams <dan.j.williams@intel.com>
> > Cc: linux-mm@kvack.org
> > Cc: linux-arm-kernel@lists.infradead.org
> > Cc: linux-samsung-soc@vger.kernel.org
> > Cc: linux-media@vger.kernel.org
> > Cc: Oded Gabbay <oded.gabbay@gmail.com>
> > Cc: Omer Shpigelman <oshpigelman@habana.ai>
> > Cc: Ofir Bitton <obitton@habana.ai>
> > Cc: Tomer Tayar <ttayar@habana.ai>
> > Cc: Moti Haimovski <mhaimovski@habana.ai>
> > Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Cc: Pawel Piskorski <ppiskorski@habana.ai>
> > --
> > v2: Use unpin_user_pages_dirty_lock (John)
> > ---
> > drivers/misc/habanalabs/Kconfig | 1 -
> > drivers/misc/habanalabs/common/habanalabs.h | 3 +-
> > drivers/misc/habanalabs/common/memory.c | 49 ++++++++-------------
> > 3 files changed, 20 insertions(+), 33 deletions(-)
> >
> > diff --git a/drivers/misc/habanalabs/Kconfig b/drivers/misc/habanalabs/Kconfig
> > index 8eb5d38c618e..2f04187f7167 100644
> > --- a/drivers/misc/habanalabs/Kconfig
> > +++ b/drivers/misc/habanalabs/Kconfig
> > @@ -6,7 +6,6 @@
> > config HABANA_AI
> > tristate "HabanaAI accelerators (habanalabs)"
> > depends on PCI && HAS_IOMEM
> > - select FRAME_VECTOR
> > select DMA_SHARED_BUFFER
> > select GENERIC_ALLOCATOR
> > select HWMON
> > diff --git a/drivers/misc/habanalabs/common/habanalabs.h b/drivers/misc/habanalabs/common/habanalabs.h
> > index edbd627b29d2..c1b3ad613b15 100644
> > --- a/drivers/misc/habanalabs/common/habanalabs.h
> > +++ b/drivers/misc/habanalabs/common/habanalabs.h
> > @@ -881,7 +881,8 @@ struct hl_ctx_mgr {
> > struct hl_userptr {
> > enum vm_type_t vm_type; /* must be first */
> > struct list_head job_node;
> > - struct frame_vector *vec;
> > + struct page **pages;
> > + unsigned int npages;
> Can you please update the kerneldoc comment section of this structure
> according to your changes ?
Apologies I missed the nice kerneldoc. I'll fix that in the next round.
> > struct sg_table *sgt;
> > enum dma_data_direction dir;
> > struct list_head debugfs_list;
> > diff --git a/drivers/misc/habanalabs/common/memory.c b/drivers/misc/habanalabs/common/memory.c
> > index 5ff4688683fd..327b64479f97 100644
> > --- a/drivers/misc/habanalabs/common/memory.c
> > +++ b/drivers/misc/habanalabs/common/memory.c
> > @@ -1281,45 +1281,41 @@ static int get_user_memory(struct hl_device *hdev, u64 addr, u64 size,
> > return -EFAULT;
> > }
> >
> > - userptr->vec = frame_vector_create(npages);
> > - if (!userptr->vec) {
> > + userptr->pages = kvmalloc_array(npages, sizeof(*userptr->pages),
> > + GFP_KERNEL);
> > + if (!userptr->pages) {
> > dev_err(hdev->dev, "Failed to create frame vector\n");
> > return -ENOMEM;
> > }
> >
> > - rc = get_vaddr_frames(start, npages, FOLL_FORCE | FOLL_WRITE,
> > - userptr->vec);
> > + rc = pin_user_pages_fast(start, npages, FOLL_FORCE | FOLL_WRITE,
> > + userptr->pages);
> >
> > if (rc != npages) {
> > dev_err(hdev->dev,
> > "Failed to map host memory, user ptr probably wrong\n");
> > if (rc < 0)
> > - goto destroy_framevec;
> > + goto destroy_pages;
> > + npages = rc;
> > rc = -EFAULT;
> > - goto put_framevec;
> > - }
> > -
> > - if (frame_vector_to_pages(userptr->vec) < 0) {
> > - dev_err(hdev->dev,
> > - "Failed to translate frame vector to pages\n");
> > - rc = -EFAULT;
> > - goto put_framevec;
> > + goto put_pages;
> > }
> > + userptr->npages = npages;
> >
> > rc = sg_alloc_table_from_pages(userptr->sgt,
> > - frame_vector_pages(userptr->vec),
> > - npages, offset, size, GFP_ATOMIC);
> > + userptr->pages,
> > + npages, offset, size, GFP_ATOMIC);
> I think that because the call to kvmalloc_array() is done with
> GFP_KERNEL, there is no point in using GFP_ATOMIC here.
> And actually, this path only needs to avoid yielding when using a
> special debug mode.
> So I suggest putting here GFP_KERNEL.
Huh, I didn't even notice the GFP_ATOMIC here. This looks indeed
strange and GFP_KERNEL should be perfectly fine in a function that
also calls pin_user_pages (since that one can allocate and do worse
stuff like userspace pagefaults).
But since that GFP_ATOMIC is there already I'll do that in a separate patch.
> In the meanwhile, I'll run this patch (coupled with the next patch) in
> our C/I to make sure there are no regressions.
Excellent. I'll wait with v3 until that's done, just in case you hit a
snag I need to fix.
Cheers, Daniel
> Thanks,
> Oded
>
> > if (rc < 0) {
> > dev_err(hdev->dev, "failed to create SG table from pages\n");
> > - goto put_framevec;
> > + goto put_pages;
> > }
> >
> > return 0;
> >
> > -put_framevec:
> > - put_vaddr_frames(userptr->vec);
> > -destroy_framevec:
> > - frame_vector_destroy(userptr->vec);
> > +put_pages:
> > + unpin_user_pages(userptr->pages, npages);
> > +destroy_pages:
> > + kvfree(userptr->pages);
> > return rc;
> > }
> >
> > @@ -1405,8 +1401,6 @@ int hl_pin_host_memory(struct hl_device *hdev, u64 addr, u64 size,
> > */
> > void hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr)
> > {
> > - struct page **pages;
> > -
> > hl_debugfs_remove_userptr(hdev, userptr);
> >
> > if (userptr->dma_mapped)
> > @@ -1414,15 +1408,8 @@ void hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr)
> > userptr->sgt->nents,
> > userptr->dir);
> >
> > - pages = frame_vector_pages(userptr->vec);
> > - if (!IS_ERR(pages)) {
> > - int i;
> > -
> > - for (i = 0; i < frame_vector_count(userptr->vec); i++)
> > - set_page_dirty_lock(pages[i]);
> > - }
> > - put_vaddr_frames(userptr->vec);
> > - frame_vector_destroy(userptr->vec);
> > + unpin_user_pages_dirty_lock(userptr->pages, userptr->npages, true);
> > + kvfree(userptr->pages);
> >
> > list_del(&userptr->job_node);
> >
> > --
> > 2.28.0
> >
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
WARNING: multiple messages have this Message-ID (diff)
From: Daniel Vetter <daniel.vetter@ffwll.ch>
To: Oded Gabbay <oded.gabbay@gmail.com>
Cc: linux-s390 <linux-s390@vger.kernel.org>,
linux-samsung-soc <linux-samsung-soc@vger.kernel.org>,
"Jan Kara" <jack@suse.cz>, "KVM list" <kvm@vger.kernel.org>,
"Jason Gunthorpe" <jgg@ziepe.ca>,
"Pawel Piskorski" <ppiskorski@habana.ai>,
"John Hubbard" <jhubbard@nvidia.com>,
LKML <linux-kernel@vger.kernel.org>,
"DRI Development" <dri-devel@lists.freedesktop.org>,
"Ofir Bitton" <obitton@habana.ai>, linux-mm <linux-mm@kvack.org>,
"Jérôme Glisse" <jglisse@redhat.com>,
"Tomer Tayar" <ttayar@habana.ai>,
"Omer Shpigelman" <oshpigelman@habana.ai>,
"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
"Daniel Vetter" <daniel.vetter@intel.com>,
"Andrew Morton" <akpm@linux-foundation.org>,
"Moti Haimovski" <mhaimovski@habana.ai>,
"Dan Williams" <dan.j.williams@intel.com>,
"list@263.net:IOMMU DRIVERS <iommu@lists.linux-foundation.org>,
Joerg Roedel <joro@8bytes.org>,
" <linux-arm-kernel@lists.infradead.org>,
"Linux Media Mailing List" <linux-media@vger.kernel.org>
Subject: Re: [PATCH v2 03/17] misc/habana: Stop using frame_vector helpers
Date: Sat, 10 Oct 2020 23:32:03 +0200 [thread overview]
Message-ID: <CAKMK7uG_kBpmuQDRgKdyh8SycFDhE7kuB2MEOsx+D5wRmerWKA@mail.gmail.com> (raw)
In-Reply-To: <CAFCwf1194Ce98y8tWxKzXT1rsdHDkzEcnERiaU=3-=t7hygmXg@mail.gmail.com>
On Sat, Oct 10, 2020 at 10:27 PM Oded Gabbay <oded.gabbay@gmail.com> wrote:
>
> On Fri, Oct 9, 2020 at 10:59 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> >
> > All we need are a pages array, pin_user_pages_fast can give us that
> > directly. Plus this avoids the entire raw pfn side of get_vaddr_frames.
> >
> Thanks for the patch Daniel.
>
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > Cc: Jason Gunthorpe <jgg@ziepe.ca>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: John Hubbard <jhubbard@nvidia.com>
> > Cc: Jérôme Glisse <jglisse@redhat.com>
> > Cc: Jan Kara <jack@suse.cz>
> > Cc: Dan Williams <dan.j.williams@intel.com>
> > Cc: linux-mm@kvack.org
> > Cc: linux-arm-kernel@lists.infradead.org
> > Cc: linux-samsung-soc@vger.kernel.org
> > Cc: linux-media@vger.kernel.org
> > Cc: Oded Gabbay <oded.gabbay@gmail.com>
> > Cc: Omer Shpigelman <oshpigelman@habana.ai>
> > Cc: Ofir Bitton <obitton@habana.ai>
> > Cc: Tomer Tayar <ttayar@habana.ai>
> > Cc: Moti Haimovski <mhaimovski@habana.ai>
> > Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Cc: Pawel Piskorski <ppiskorski@habana.ai>
> > --
> > v2: Use unpin_user_pages_dirty_lock (John)
> > ---
> > drivers/misc/habanalabs/Kconfig | 1 -
> > drivers/misc/habanalabs/common/habanalabs.h | 3 +-
> > drivers/misc/habanalabs/common/memory.c | 49 ++++++++-------------
> > 3 files changed, 20 insertions(+), 33 deletions(-)
> >
> > diff --git a/drivers/misc/habanalabs/Kconfig b/drivers/misc/habanalabs/Kconfig
> > index 8eb5d38c618e..2f04187f7167 100644
> > --- a/drivers/misc/habanalabs/Kconfig
> > +++ b/drivers/misc/habanalabs/Kconfig
> > @@ -6,7 +6,6 @@
> > config HABANA_AI
> > tristate "HabanaAI accelerators (habanalabs)"
> > depends on PCI && HAS_IOMEM
> > - select FRAME_VECTOR
> > select DMA_SHARED_BUFFER
> > select GENERIC_ALLOCATOR
> > select HWMON
> > diff --git a/drivers/misc/habanalabs/common/habanalabs.h b/drivers/misc/habanalabs/common/habanalabs.h
> > index edbd627b29d2..c1b3ad613b15 100644
> > --- a/drivers/misc/habanalabs/common/habanalabs.h
> > +++ b/drivers/misc/habanalabs/common/habanalabs.h
> > @@ -881,7 +881,8 @@ struct hl_ctx_mgr {
> > struct hl_userptr {
> > enum vm_type_t vm_type; /* must be first */
> > struct list_head job_node;
> > - struct frame_vector *vec;
> > + struct page **pages;
> > + unsigned int npages;
> Can you please update the kerneldoc comment section of this structure
> according to your changes ?
Apologies I missed the nice kerneldoc. I'll fix that in the next round.
> > struct sg_table *sgt;
> > enum dma_data_direction dir;
> > struct list_head debugfs_list;
> > diff --git a/drivers/misc/habanalabs/common/memory.c b/drivers/misc/habanalabs/common/memory.c
> > index 5ff4688683fd..327b64479f97 100644
> > --- a/drivers/misc/habanalabs/common/memory.c
> > +++ b/drivers/misc/habanalabs/common/memory.c
> > @@ -1281,45 +1281,41 @@ static int get_user_memory(struct hl_device *hdev, u64 addr, u64 size,
> > return -EFAULT;
> > }
> >
> > - userptr->vec = frame_vector_create(npages);
> > - if (!userptr->vec) {
> > + userptr->pages = kvmalloc_array(npages, sizeof(*userptr->pages),
> > + GFP_KERNEL);
> > + if (!userptr->pages) {
> > dev_err(hdev->dev, "Failed to create frame vector\n");
> > return -ENOMEM;
> > }
> >
> > - rc = get_vaddr_frames(start, npages, FOLL_FORCE | FOLL_WRITE,
> > - userptr->vec);
> > + rc = pin_user_pages_fast(start, npages, FOLL_FORCE | FOLL_WRITE,
> > + userptr->pages);
> >
> > if (rc != npages) {
> > dev_err(hdev->dev,
> > "Failed to map host memory, user ptr probably wrong\n");
> > if (rc < 0)
> > - goto destroy_framevec;
> > + goto destroy_pages;
> > + npages = rc;
> > rc = -EFAULT;
> > - goto put_framevec;
> > - }
> > -
> > - if (frame_vector_to_pages(userptr->vec) < 0) {
> > - dev_err(hdev->dev,
> > - "Failed to translate frame vector to pages\n");
> > - rc = -EFAULT;
> > - goto put_framevec;
> > + goto put_pages;
> > }
> > + userptr->npages = npages;
> >
> > rc = sg_alloc_table_from_pages(userptr->sgt,
> > - frame_vector_pages(userptr->vec),
> > - npages, offset, size, GFP_ATOMIC);
> > + userptr->pages,
> > + npages, offset, size, GFP_ATOMIC);
> I think that because the call to kvmalloc_array() is done with
> GFP_KERNEL, there is no point in using GFP_ATOMIC here.
> And actually, this path only needs to avoid yielding when using a
> special debug mode.
> So I suggest putting here GFP_KERNEL.
Huh, I didn't even notice the GFP_ATOMIC here. This looks indeed
strange and GFP_KERNEL should be perfectly fine in a function that
also calls pin_user_pages (since that one can allocate and do worse
stuff like userspace pagefaults).
But since that GFP_ATOMIC is there already I'll do that in a separate patch.
> In the meanwhile, I'll run this patch (coupled with the next patch) in
> our C/I to make sure there are no regressions.
Excellent. I'll wait with v3 until that's done, just in case you hit a
snag I need to fix.
Cheers, Daniel
> Thanks,
> Oded
>
> > if (rc < 0) {
> > dev_err(hdev->dev, "failed to create SG table from pages\n");
> > - goto put_framevec;
> > + goto put_pages;
> > }
> >
> > return 0;
> >
> > -put_framevec:
> > - put_vaddr_frames(userptr->vec);
> > -destroy_framevec:
> > - frame_vector_destroy(userptr->vec);
> > +put_pages:
> > + unpin_user_pages(userptr->pages, npages);
> > +destroy_pages:
> > + kvfree(userptr->pages);
> > return rc;
> > }
> >
> > @@ -1405,8 +1401,6 @@ int hl_pin_host_memory(struct hl_device *hdev, u64 addr, u64 size,
> > */
> > void hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr)
> > {
> > - struct page **pages;
> > -
> > hl_debugfs_remove_userptr(hdev, userptr);
> >
> > if (userptr->dma_mapped)
> > @@ -1414,15 +1408,8 @@ void hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr)
> > userptr->sgt->nents,
> > userptr->dir);
> >
> > - pages = frame_vector_pages(userptr->vec);
> > - if (!IS_ERR(pages)) {
> > - int i;
> > -
> > - for (i = 0; i < frame_vector_count(userptr->vec); i++)
> > - set_page_dirty_lock(pages[i]);
> > - }
> > - put_vaddr_frames(userptr->vec);
> > - frame_vector_destroy(userptr->vec);
> > + unpin_user_pages_dirty_lock(userptr->pages, userptr->npages, true);
> > + kvfree(userptr->pages);
> >
> > list_del(&userptr->job_node);
> >
> > --
> > 2.28.0
> >
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
WARNING: multiple messages have this Message-ID (diff)
From: Daniel Vetter <daniel.vetter@ffwll.ch>
To: Oded Gabbay <oded.gabbay@gmail.com>
Cc: linux-s390 <linux-s390@vger.kernel.org>,
linux-samsung-soc <linux-samsung-soc@vger.kernel.org>,
"Jan Kara" <jack@suse.cz>, "KVM list" <kvm@vger.kernel.org>,
"Jason Gunthorpe" <jgg@ziepe.ca>,
"Pawel Piskorski" <ppiskorski@habana.ai>,
"John Hubbard" <jhubbard@nvidia.com>,
LKML <linux-kernel@vger.kernel.org>,
"DRI Development" <dri-devel@lists.freedesktop.org>,
"Ofir Bitton" <obitton@habana.ai>, linux-mm <linux-mm@kvack.org>,
"Jérôme Glisse" <jglisse@redhat.com>,
"Tomer Tayar" <ttayar@habana.ai>,
"Omer Shpigelman" <oshpigelman@habana.ai>,
"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
"Daniel Vetter" <daniel.vetter@intel.com>,
"Andrew Morton" <akpm@linux-foundation.org>,
"Moti Haimovski" <mhaimovski@habana.ai>,
"Dan Williams" <dan.j.williams@intel.com>,
"list@263.net:IOMMU DRIVERS <iommu@lists.linux-foundation.org>,
Joerg Roedel <joro@8bytes.org>,
" <linux-arm-kernel@lists.infradead.org>,
"Linux Media Mailing List" <linux-media@vger.kernel.org>
Subject: Re: [PATCH v2 03/17] misc/habana: Stop using frame_vector helpers
Date: Sat, 10 Oct 2020 23:32:03 +0200 [thread overview]
Message-ID: <CAKMK7uG_kBpmuQDRgKdyh8SycFDhE7kuB2MEOsx+D5wRmerWKA@mail.gmail.com> (raw)
In-Reply-To: <CAFCwf1194Ce98y8tWxKzXT1rsdHDkzEcnERiaU=3-=t7hygmXg@mail.gmail.com>
On Sat, Oct 10, 2020 at 10:27 PM Oded Gabbay <oded.gabbay@gmail.com> wrote:
>
> On Fri, Oct 9, 2020 at 10:59 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> >
> > All we need are a pages array, pin_user_pages_fast can give us that
> > directly. Plus this avoids the entire raw pfn side of get_vaddr_frames.
> >
> Thanks for the patch Daniel.
>
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > Cc: Jason Gunthorpe <jgg@ziepe.ca>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: John Hubbard <jhubbard@nvidia.com>
> > Cc: Jérôme Glisse <jglisse@redhat.com>
> > Cc: Jan Kara <jack@suse.cz>
> > Cc: Dan Williams <dan.j.williams@intel.com>
> > Cc: linux-mm@kvack.org
> > Cc: linux-arm-kernel@lists.infradead.org
> > Cc: linux-samsung-soc@vger.kernel.org
> > Cc: linux-media@vger.kernel.org
> > Cc: Oded Gabbay <oded.gabbay@gmail.com>
> > Cc: Omer Shpigelman <oshpigelman@habana.ai>
> > Cc: Ofir Bitton <obitton@habana.ai>
> > Cc: Tomer Tayar <ttayar@habana.ai>
> > Cc: Moti Haimovski <mhaimovski@habana.ai>
> > Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Cc: Pawel Piskorski <ppiskorski@habana.ai>
> > --
> > v2: Use unpin_user_pages_dirty_lock (John)
> > ---
> > drivers/misc/habanalabs/Kconfig | 1 -
> > drivers/misc/habanalabs/common/habanalabs.h | 3 +-
> > drivers/misc/habanalabs/common/memory.c | 49 ++++++++-------------
> > 3 files changed, 20 insertions(+), 33 deletions(-)
> >
> > diff --git a/drivers/misc/habanalabs/Kconfig b/drivers/misc/habanalabs/Kconfig
> > index 8eb5d38c618e..2f04187f7167 100644
> > --- a/drivers/misc/habanalabs/Kconfig
> > +++ b/drivers/misc/habanalabs/Kconfig
> > @@ -6,7 +6,6 @@
> > config HABANA_AI
> > tristate "HabanaAI accelerators (habanalabs)"
> > depends on PCI && HAS_IOMEM
> > - select FRAME_VECTOR
> > select DMA_SHARED_BUFFER
> > select GENERIC_ALLOCATOR
> > select HWMON
> > diff --git a/drivers/misc/habanalabs/common/habanalabs.h b/drivers/misc/habanalabs/common/habanalabs.h
> > index edbd627b29d2..c1b3ad613b15 100644
> > --- a/drivers/misc/habanalabs/common/habanalabs.h
> > +++ b/drivers/misc/habanalabs/common/habanalabs.h
> > @@ -881,7 +881,8 @@ struct hl_ctx_mgr {
> > struct hl_userptr {
> > enum vm_type_t vm_type; /* must be first */
> > struct list_head job_node;
> > - struct frame_vector *vec;
> > + struct page **pages;
> > + unsigned int npages;
> Can you please update the kerneldoc comment section of this structure
> according to your changes ?
Apologies I missed the nice kerneldoc. I'll fix that in the next round.
> > struct sg_table *sgt;
> > enum dma_data_direction dir;
> > struct list_head debugfs_list;
> > diff --git a/drivers/misc/habanalabs/common/memory.c b/drivers/misc/habanalabs/common/memory.c
> > index 5ff4688683fd..327b64479f97 100644
> > --- a/drivers/misc/habanalabs/common/memory.c
> > +++ b/drivers/misc/habanalabs/common/memory.c
> > @@ -1281,45 +1281,41 @@ static int get_user_memory(struct hl_device *hdev, u64 addr, u64 size,
> > return -EFAULT;
> > }
> >
> > - userptr->vec = frame_vector_create(npages);
> > - if (!userptr->vec) {
> > + userptr->pages = kvmalloc_array(npages, sizeof(*userptr->pages),
> > + GFP_KERNEL);
> > + if (!userptr->pages) {
> > dev_err(hdev->dev, "Failed to create frame vector\n");
> > return -ENOMEM;
> > }
> >
> > - rc = get_vaddr_frames(start, npages, FOLL_FORCE | FOLL_WRITE,
> > - userptr->vec);
> > + rc = pin_user_pages_fast(start, npages, FOLL_FORCE | FOLL_WRITE,
> > + userptr->pages);
> >
> > if (rc != npages) {
> > dev_err(hdev->dev,
> > "Failed to map host memory, user ptr probably wrong\n");
> > if (rc < 0)
> > - goto destroy_framevec;
> > + goto destroy_pages;
> > + npages = rc;
> > rc = -EFAULT;
> > - goto put_framevec;
> > - }
> > -
> > - if (frame_vector_to_pages(userptr->vec) < 0) {
> > - dev_err(hdev->dev,
> > - "Failed to translate frame vector to pages\n");
> > - rc = -EFAULT;
> > - goto put_framevec;
> > + goto put_pages;
> > }
> > + userptr->npages = npages;
> >
> > rc = sg_alloc_table_from_pages(userptr->sgt,
> > - frame_vector_pages(userptr->vec),
> > - npages, offset, size, GFP_ATOMIC);
> > + userptr->pages,
> > + npages, offset, size, GFP_ATOMIC);
> I think that because the call to kvmalloc_array() is done with
> GFP_KERNEL, there is no point in using GFP_ATOMIC here.
> And actually, this path only needs to avoid yielding when using a
> special debug mode.
> So I suggest putting here GFP_KERNEL.
Huh, I didn't even notice the GFP_ATOMIC here. This looks indeed
strange and GFP_KERNEL should be perfectly fine in a function that
also calls pin_user_pages (since that one can allocate and do worse
stuff like userspace pagefaults).
But since that GFP_ATOMIC is there already I'll do that in a separate patch.
> In the meanwhile, I'll run this patch (coupled with the next patch) in
> our C/I to make sure there are no regressions.
Excellent. I'll wait with v3 until that's done, just in case you hit a
snag I need to fix.
Cheers, Daniel
> Thanks,
> Oded
>
> > if (rc < 0) {
> > dev_err(hdev->dev, "failed to create SG table from pages\n");
> > - goto put_framevec;
> > + goto put_pages;
> > }
> >
> > return 0;
> >
> > -put_framevec:
> > - put_vaddr_frames(userptr->vec);
> > -destroy_framevec:
> > - frame_vector_destroy(userptr->vec);
> > +put_pages:
> > + unpin_user_pages(userptr->pages, npages);
> > +destroy_pages:
> > + kvfree(userptr->pages);
> > return rc;
> > }
> >
> > @@ -1405,8 +1401,6 @@ int hl_pin_host_memory(struct hl_device *hdev, u64 addr, u64 size,
> > */
> > void hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr)
> > {
> > - struct page **pages;
> > -
> > hl_debugfs_remove_userptr(hdev, userptr);
> >
> > if (userptr->dma_mapped)
> > @@ -1414,15 +1408,8 @@ void hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr)
> > userptr->sgt->nents,
> > userptr->dir);
> >
> > - pages = frame_vector_pages(userptr->vec);
> > - if (!IS_ERR(pages)) {
> > - int i;
> > -
> > - for (i = 0; i < frame_vector_count(userptr->vec); i++)
> > - set_page_dirty_lock(pages[i]);
> > - }
> > - put_vaddr_frames(userptr->vec);
> > - frame_vector_destroy(userptr->vec);
> > + unpin_user_pages_dirty_lock(userptr->pages, userptr->npages, true);
> > + kvfree(userptr->pages);
> >
> > list_del(&userptr->job_node);
> >
> > --
> > 2.28.0
> >
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
next prev parent reply other threads:[~2020-10-10 23:01 UTC|newest]
Thread overview: 214+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-09 7:59 [PATCH v2 00/17] follow_pfn and other iomap races Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` [PATCH v2 01/17] drm/exynos: Stop using frame_vector helpers Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-16 7:42 ` John Hubbard
2020-10-16 7:42 ` John Hubbard
2020-10-16 7:42 ` John Hubbard
2020-10-09 7:59 ` [PATCH v2 02/17] drm/exynos: Use FOLL_LONGTERM for g2d cmdlists Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` [PATCH v2 03/17] misc/habana: Stop using frame_vector helpers Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-10 20:26 ` Oded Gabbay
2020-10-10 20:26 ` Oded Gabbay
2020-10-10 20:26 ` Oded Gabbay
2020-10-10 21:32 ` Daniel Vetter [this message]
2020-10-10 21:32 ` Daniel Vetter
2020-10-10 21:32 ` Daniel Vetter
2020-10-10 21:41 ` Daniel Vetter
2020-10-10 21:41 ` Daniel Vetter
2020-10-10 21:41 ` Daniel Vetter
2020-10-10 21:47 ` Oded Gabbay
2020-10-10 21:47 ` Oded Gabbay
2020-10-10 21:47 ` Oded Gabbay
2020-10-16 7:45 ` John Hubbard
2020-10-16 7:45 ` John Hubbard
2020-10-16 7:45 ` John Hubbard
2020-10-09 7:59 ` [PATCH v2 04/17] misc/habana: Use FOLL_LONGTERM for userptr Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` [PATCH v2 05/17] mm/frame-vector: Use FOLL_LONGTERM Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-16 7:54 ` John Hubbard
2020-10-16 7:54 ` John Hubbard
2020-10-16 7:54 ` John Hubbard
2020-10-16 8:03 ` Daniel Vetter
2020-10-16 8:03 ` Daniel Vetter
2020-10-16 8:03 ` Daniel Vetter
2020-10-09 7:59 ` [PATCH v2 06/17] media: videobuf2: Move frame_vector into media subsystem Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 10:14 ` Mauro Carvalho Chehab
2020-10-09 10:14 ` Mauro Carvalho Chehab
2020-10-09 10:14 ` Mauro Carvalho Chehab
2020-10-09 16:57 ` Daniel Vetter
2020-10-09 16:57 ` Daniel Vetter
2020-10-09 16:57 ` Daniel Vetter
2020-10-09 16:57 ` Daniel Vetter
2020-10-09 7:59 ` [PATCH v2 07/17] mm: Close race in generic_access_phys Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` [PATCH v2 08/17] s390/pci: Remove races against pte updates Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-12 14:03 ` Niklas Schnelle
2020-10-12 14:03 ` Niklas Schnelle
2020-10-12 14:03 ` Niklas Schnelle
2020-10-12 14:19 ` Daniel Vetter
2020-10-12 14:19 ` Daniel Vetter
2020-10-12 14:19 ` Daniel Vetter
2020-10-12 14:19 ` Daniel Vetter
2020-10-12 14:39 ` Niklas Schnelle
2020-10-12 14:39 ` Niklas Schnelle
2020-10-12 14:39 ` Niklas Schnelle
2020-10-21 7:55 ` Niklas Schnelle
2020-10-21 7:55 ` Niklas Schnelle
2020-10-21 7:55 ` Niklas Schnelle
2020-10-22 7:39 ` Daniel Vetter
2020-10-22 7:39 ` Daniel Vetter
2020-10-22 7:39 ` Daniel Vetter
2020-10-22 7:39 ` Daniel Vetter
2020-10-09 7:59 ` [PATCH v2 09/17] mm: Add unsafe_follow_pfn Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 10:34 ` Mauro Carvalho Chehab
2020-10-09 10:34 ` Mauro Carvalho Chehab
2020-10-09 10:34 ` Mauro Carvalho Chehab
2020-10-09 12:21 ` Jason Gunthorpe
2020-10-09 12:21 ` Jason Gunthorpe
2020-10-09 12:21 ` Jason Gunthorpe
2020-10-09 12:37 ` Mauro Carvalho Chehab
2020-10-09 12:37 ` Mauro Carvalho Chehab
2020-10-09 12:37 ` Mauro Carvalho Chehab
2020-10-09 12:39 ` Mauro Carvalho Chehab
2020-10-09 12:39 ` Mauro Carvalho Chehab
2020-10-09 12:39 ` Mauro Carvalho Chehab
2020-10-09 12:48 ` Jason Gunthorpe
2020-10-09 12:48 ` Jason Gunthorpe
2020-10-09 12:48 ` Jason Gunthorpe
2020-10-09 17:52 ` Daniel Vetter
2020-10-09 17:52 ` Daniel Vetter
2020-10-09 17:52 ` Daniel Vetter
2020-10-09 18:01 ` Jason Gunthorpe
2020-10-09 18:01 ` Jason Gunthorpe
2020-10-09 18:01 ` Jason Gunthorpe
2020-10-09 19:31 ` Daniel Vetter
2020-10-09 19:31 ` Daniel Vetter
2020-10-09 19:31 ` Daniel Vetter
2020-10-10 9:21 ` Mauro Carvalho Chehab
2020-10-10 9:21 ` Mauro Carvalho Chehab
2020-10-10 9:21 ` Mauro Carvalho Chehab
2020-10-10 10:53 ` Daniel Vetter
2020-10-10 10:53 ` Daniel Vetter
2020-10-10 10:53 ` Daniel Vetter
2020-10-10 11:39 ` Mauro Carvalho Chehab
2020-10-10 11:39 ` Mauro Carvalho Chehab
2020-10-10 11:39 ` Mauro Carvalho Chehab
2020-10-10 11:56 ` Daniel Vetter
2020-10-10 11:56 ` Daniel Vetter
2020-10-10 11:56 ` Daniel Vetter
2020-10-10 17:22 ` Tomasz Figa
2020-10-10 17:22 ` Tomasz Figa
2020-10-10 17:22 ` Tomasz Figa
2020-10-10 21:35 ` Laurent Pinchart
2020-10-10 21:35 ` Laurent Pinchart
2020-10-10 21:35 ` Laurent Pinchart
2020-10-10 21:50 ` Daniel Vetter
2020-10-10 21:50 ` Daniel Vetter
2020-10-10 21:50 ` Daniel Vetter
2020-10-11 6:27 ` Mauro Carvalho Chehab
2020-10-11 6:27 ` Mauro Carvalho Chehab
2020-10-11 6:27 ` Mauro Carvalho Chehab
2020-10-11 6:36 ` Mauro Carvalho Chehab
2020-10-11 6:36 ` Mauro Carvalho Chehab
2020-10-11 6:36 ` Mauro Carvalho Chehab
2020-10-10 21:11 ` Laurent Pinchart
2020-10-10 21:11 ` Laurent Pinchart
2020-10-10 21:11 ` Laurent Pinchart
2020-10-12 10:46 ` Marek Szyprowski
2020-10-12 10:46 ` Marek Szyprowski
2020-10-12 10:46 ` Marek Szyprowski
2020-10-12 13:49 ` Daniel Vetter
2020-10-12 13:49 ` Daniel Vetter
2020-10-12 13:49 ` Daniel Vetter
2020-10-10 17:30 ` Tomasz Figa
2020-10-10 17:30 ` Tomasz Figa
2020-10-10 17:30 ` Tomasz Figa
2020-10-09 7:59 ` [PATCH v2 10/17] media/videbuf1|2: Mark follow_pfn usage as unsafe Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-10 9:24 ` Mauro Carvalho Chehab
2020-10-10 9:24 ` Mauro Carvalho Chehab
2020-10-10 9:24 ` Mauro Carvalho Chehab
2020-10-09 7:59 ` [PATCH v2 11/17] vfio/type1: Mark follow_pfn " Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` [PATCH v2 12/17] PCI: Obey iomem restrictions for procfs mmap Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` [PATCH v2 13/17] /dev/mem: Only set filp->f_mapping Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` [PATCH v2 14/17] resource: Move devmem revoke code to resource framework Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 10:59 ` Greg Kroah-Hartman
2020-10-09 10:59 ` Greg Kroah-Hartman
2020-10-09 10:59 ` Greg Kroah-Hartman
2020-10-09 10:59 ` Greg Kroah-Hartman
2020-10-09 12:31 ` Jason Gunthorpe
2020-10-09 12:31 ` Jason Gunthorpe
2020-10-09 12:31 ` Jason Gunthorpe
2020-10-09 14:24 ` Daniel Vetter
2020-10-09 14:24 ` Daniel Vetter
2020-10-09 14:24 ` Daniel Vetter
2020-10-09 14:32 ` Jason Gunthorpe
2020-10-09 14:32 ` Jason Gunthorpe
2020-10-09 14:32 ` Jason Gunthorpe
2020-10-09 18:28 ` Dan Williams
2020-10-09 18:28 ` Dan Williams
2020-10-09 18:28 ` Dan Williams
2020-10-15 0:09 ` Jason Gunthorpe
2020-10-15 0:09 ` Jason Gunthorpe
2020-10-15 0:09 ` Jason Gunthorpe
2020-10-15 7:52 ` Daniel Vetter
2020-10-15 7:52 ` Daniel Vetter
2020-10-15 7:52 ` Daniel Vetter
2020-10-15 7:55 ` Daniel Vetter
2020-10-15 7:55 ` Daniel Vetter
2020-10-15 7:55 ` Daniel Vetter
2020-10-15 15:29 ` Daniel Vetter
2020-10-15 15:29 ` Daniel Vetter
2020-10-15 15:29 ` Daniel Vetter
2020-10-09 7:59 ` [PATCH v2 15/17] sysfs: Support zapping of binary attr mmaps Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 10:58 ` Greg Kroah-Hartman
2020-10-09 10:58 ` Greg Kroah-Hartman
2020-10-09 10:58 ` Greg Kroah-Hartman
2020-10-09 10:58 ` Greg Kroah-Hartman
2020-10-09 7:59 ` [PATCH v2 16/17] PCI: Revoke mappings like devmem Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` [PATCH v2 17/17] drm/i915: Properly request PCI BARs Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 7:59 ` Daniel Vetter
2020-10-09 9:47 ` Ville Syrjälä
2020-10-09 9:47 ` Ville Syrjälä
2020-10-09 9:47 ` Ville Syrjälä
2020-10-09 9:47 ` Ville Syrjälä
2020-10-09 10:01 ` Daniel Vetter
2020-10-09 10:01 ` Daniel Vetter
2020-10-09 10:01 ` Daniel Vetter
2020-10-09 10:41 ` Ville Syrjälä
2020-10-09 10:41 ` Ville Syrjälä
2020-10-09 10:41 ` Ville Syrjälä
2020-10-09 10:41 ` Ville Syrjälä
2020-10-09 14:18 ` Daniel Vetter
2020-10-09 14:18 ` Daniel Vetter
2020-10-09 14:18 ` Daniel Vetter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAKMK7uG_kBpmuQDRgKdyh8SycFDhE7kuB2MEOsx+D5wRmerWKA@mail.gmail.com \
--to=daniel.vetter@ffwll.ch \
--cc=akpm@linux-foundation.org \
--cc=dan.j.williams@intel.com \
--cc=daniel.vetter@intel.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=gregkh@linuxfoundation.org \
--cc=jack@suse.cz \
--cc=jgg@ziepe.ca \
--cc=jglisse@redhat.com \
--cc=jhubbard@nvidia.com \
--cc=kvm@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-s390@vger.kernel.org \
--cc=linux-samsung-soc@vger.kernel.org \
--cc=mhaimovski@habana.ai \
--cc=obitton@habana.ai \
--cc=oded.gabbay@gmail.com \
--cc=oshpigelman@habana.ai \
--cc=ppiskorski@habana.ai \
--cc=ttayar@habana.ai \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.