From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 388DDC43457 for ; Sat, 10 Oct 2020 20:27:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 83F572074D for ; Sat, 10 Oct 2020 20:27:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QdqGF5qe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 83F572074D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D5E876B005C; Sat, 10 Oct 2020 16:27:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CE949900002; Sat, 10 Oct 2020 16:27:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BAF6A6B0068; Sat, 10 Oct 2020 16:27:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0217.hostedemail.com [216.40.44.217]) by kanga.kvack.org (Postfix) with ESMTP id 8A49C6B005C for ; Sat, 10 Oct 2020 16:27:19 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 1C0A23628 for ; Sat, 10 Oct 2020 20:27:19 +0000 (UTC) X-FDA: 77357150598.07.art05_290ff4a271ec Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id F27CE1803FD61 for ; Sat, 10 Oct 2020 20:27:18 +0000 (UTC) X-HE-Tag: art05_290ff4a271ec X-Filterd-Recvd-Size: 10435 Received: from mail-oi1-f196.google.com (mail-oi1-f196.google.com [209.85.167.196]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Sat, 10 Oct 2020 20:27:18 +0000 (UTC) Received: by mail-oi1-f196.google.com with SMTP id z26so14122219oih.12 for ; Sat, 10 Oct 2020 13:27:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=oGo8IGgCZpTOoR3ZKZaP8e3qvIeXNb3yMrzl+Uani14=; b=QdqGF5qe16vB4wlkB20OoDG5GsZXhkiu+V7PIg7gevT+li9s2YZrjhJjrGKFlhR8f2 S2dMeFOHFe2Vy3+ViSPQkUcTH6WgZSgyFemNPe3jkRa1BRJLmCIpjdAjfS+Z+fMYuuFc J/zHg1WUv6YS6qXHyCbAbu2sfcz6kuW6fXvudSVSb/G2m0zjetF8vQarbaEqA1w5uj5C 1ytU+yCJQIoUFZyqLvf50g0rJYU5X+PHHN8XIpBwZ7Wgshsn/e8A5zDxqiYS9ZaCOgq0 +eTVZzOuHYBHFBmUTwz8gE0YQaPT+/D15xczuawlfzpKen+UVAP0g0nAO1GFs3+KLRMi 43mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=oGo8IGgCZpTOoR3ZKZaP8e3qvIeXNb3yMrzl+Uani14=; b=Js0l+2z7ivWkE2Hb+IwkBmINvl+SsAMBGCeFzSnPf7einTD7D+pQiR653kN1vj6swU VLDr66/lH1tDc7v+FhZMifIhH1asmtQ6x40sbARDU7JeBbgtvyFI4yoiuC0TNyLzBzcU xcHcKoiZCADNEIz8Zup3OExC3J831ULt8rEmbH4w12HCTelpR/YN6Eo3cEHQLFk+zJRV lGu4Jv1ba2sEN20Z7IIGq8Mo4Gf1+HM6SxQMwrplLI0O9bcKJc4prfYoo5MeKFwbZ//F 2KSksnNV5C872x7zv8X2qjKCCZ+KAlHJx4XDDVff+UUuhddn6+eg71C1z5BDs+nHjFsO OdNg== X-Gm-Message-State: AOAM533I852JUT/bPOnID9Pw/k7FsXpklpNHqRn1AU8+zUFCit+dp3x9 Oiwnx0IIfjlnQcWR09VDaYyDZbmvej0OwLY9Rvo= X-Google-Smtp-Source: ABdhPJwowo80GWbqkm6ZkqR7gSgknWZHhpMAZMgkL8s88FI9XTRLJCjdHnJ8k2W5KFXN71VQforAtfzOFMwHNRWQMIg= X-Received: by 2002:a05:6808:3bb:: with SMTP id n27mr6204755oie.130.1602361637761; Sat, 10 Oct 2020 13:27:17 -0700 (PDT) MIME-Version: 1.0 References: <20201009075934.3509076-1-daniel.vetter@ffwll.ch> <20201009075934.3509076-4-daniel.vetter@ffwll.ch> In-Reply-To: <20201009075934.3509076-4-daniel.vetter@ffwll.ch> From: Oded Gabbay Date: Sat, 10 Oct 2020 23:26:52 +0300 Message-ID: Subject: Re: [PATCH v2 03/17] misc/habana: Stop using frame_vector helpers To: Daniel Vetter Cc: DRI Development , LKML , kvm@vger.kernel.org, linux-mm , "list@263.net:IOMMU DRIVERS , Joerg Roedel ," , linux-samsung-soc , Linux Media Mailing List , linux-s390@vger.kernel.org, Daniel Vetter , Jason Gunthorpe , Andrew Morton , John Hubbard , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , Dan Williams , Omer Shpigelman , Ofir Bitton , Tomer Tayar , Moti Haimovski , Greg Kroah-Hartman , Pawel Piskorski Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Oct 9, 2020 at 10:59 AM Daniel Vetter wrot= e: > > All we need are a pages array, pin_user_pages_fast can give us that > directly. Plus this avoids the entire raw pfn side of get_vaddr_frames. > Thanks for the patch Daniel. > Signed-off-by: Daniel Vetter > Cc: Jason Gunthorpe > Cc: Andrew Morton > Cc: John Hubbard > Cc: J=C3=A9r=C3=B4me Glisse > Cc: Jan Kara > Cc: Dan Williams > Cc: linux-mm@kvack.org > Cc: linux-arm-kernel@lists.infradead.org > Cc: linux-samsung-soc@vger.kernel.org > Cc: linux-media@vger.kernel.org > Cc: Oded Gabbay > Cc: Omer Shpigelman > Cc: Ofir Bitton > Cc: Tomer Tayar > Cc: Moti Haimovski > Cc: Daniel Vetter > Cc: Greg Kroah-Hartman > Cc: Pawel Piskorski > -- > v2: Use unpin_user_pages_dirty_lock (John) > --- > drivers/misc/habanalabs/Kconfig | 1 - > drivers/misc/habanalabs/common/habanalabs.h | 3 +- > drivers/misc/habanalabs/common/memory.c | 49 ++++++++------------- > 3 files changed, 20 insertions(+), 33 deletions(-) > > diff --git a/drivers/misc/habanalabs/Kconfig b/drivers/misc/habanalabs/Kc= onfig > index 8eb5d38c618e..2f04187f7167 100644 > --- a/drivers/misc/habanalabs/Kconfig > +++ b/drivers/misc/habanalabs/Kconfig > @@ -6,7 +6,6 @@ > config HABANA_AI > tristate "HabanaAI accelerators (habanalabs)" > depends on PCI && HAS_IOMEM > - select FRAME_VECTOR > select DMA_SHARED_BUFFER > select GENERIC_ALLOCATOR > select HWMON > diff --git a/drivers/misc/habanalabs/common/habanalabs.h b/drivers/misc/h= abanalabs/common/habanalabs.h > index edbd627b29d2..c1b3ad613b15 100644 > --- a/drivers/misc/habanalabs/common/habanalabs.h > +++ b/drivers/misc/habanalabs/common/habanalabs.h > @@ -881,7 +881,8 @@ struct hl_ctx_mgr { > struct hl_userptr { > enum vm_type_t vm_type; /* must be first */ > struct list_head job_node; > - struct frame_vector *vec; > + struct page **pages; > + unsigned int npages; Can you please update the kerneldoc comment section of this structure according to your changes ? > struct sg_table *sgt; > enum dma_data_direction dir; > struct list_head debugfs_list; > diff --git a/drivers/misc/habanalabs/common/memory.c b/drivers/misc/haban= alabs/common/memory.c > index 5ff4688683fd..327b64479f97 100644 > --- a/drivers/misc/habanalabs/common/memory.c > +++ b/drivers/misc/habanalabs/common/memory.c > @@ -1281,45 +1281,41 @@ static int get_user_memory(struct hl_device *hdev= , u64 addr, u64 size, > return -EFAULT; > } > > - userptr->vec =3D frame_vector_create(npages); > - if (!userptr->vec) { > + userptr->pages =3D kvmalloc_array(npages, sizeof(*userptr->pages)= , > + GFP_KERNEL); > + if (!userptr->pages) { > dev_err(hdev->dev, "Failed to create frame vector\n"); > return -ENOMEM; > } > > - rc =3D get_vaddr_frames(start, npages, FOLL_FORCE | FOLL_WRITE, > - userptr->vec); > + rc =3D pin_user_pages_fast(start, npages, FOLL_FORCE | FOLL_WRITE= , > + userptr->pages); > > if (rc !=3D npages) { > dev_err(hdev->dev, > "Failed to map host memory, user ptr probably wro= ng\n"); > if (rc < 0) > - goto destroy_framevec; > + goto destroy_pages; > + npages =3D rc; > rc =3D -EFAULT; > - goto put_framevec; > - } > - > - if (frame_vector_to_pages(userptr->vec) < 0) { > - dev_err(hdev->dev, > - "Failed to translate frame vector to pages\n"); > - rc =3D -EFAULT; > - goto put_framevec; > + goto put_pages; > } > + userptr->npages =3D npages; > > rc =3D sg_alloc_table_from_pages(userptr->sgt, > - frame_vector_pages(userptr->vec), > - npages, offset, size, GFP_ATOMIC)= ; > + userptr->pages, > + npages, offset, size, GFP_ATOMIC); I think that because the call to kvmalloc_array() is done with GFP_KERNEL, there is no point in using GFP_ATOMIC here. And actually, this path only needs to avoid yielding when using a special debug mode. So I suggest putting here GFP_KERNEL. In the meanwhile, I'll run this patch (coupled with the next patch) in our C/I to make sure there are no regressions. Thanks, Oded > if (rc < 0) { > dev_err(hdev->dev, "failed to create SG table from pages\= n"); > - goto put_framevec; > + goto put_pages; > } > > return 0; > > -put_framevec: > - put_vaddr_frames(userptr->vec); > -destroy_framevec: > - frame_vector_destroy(userptr->vec); > +put_pages: > + unpin_user_pages(userptr->pages, npages); > +destroy_pages: > + kvfree(userptr->pages); > return rc; > } > > @@ -1405,8 +1401,6 @@ int hl_pin_host_memory(struct hl_device *hdev, u64 = addr, u64 size, > */ > void hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *use= rptr) > { > - struct page **pages; > - > hl_debugfs_remove_userptr(hdev, userptr); > > if (userptr->dma_mapped) > @@ -1414,15 +1408,8 @@ void hl_unpin_host_memory(struct hl_device *hdev, = struct hl_userptr *userptr) > userptr->sgt->nen= ts, > userptr->dir); > > - pages =3D frame_vector_pages(userptr->vec); > - if (!IS_ERR(pages)) { > - int i; > - > - for (i =3D 0; i < frame_vector_count(userptr->vec); i++) > - set_page_dirty_lock(pages[i]); > - } > - put_vaddr_frames(userptr->vec); > - frame_vector_destroy(userptr->vec); > + unpin_user_pages_dirty_lock(userptr->pages, userptr->npages, true= ); > + kvfree(userptr->pages); > > list_del(&userptr->job_node); > > -- > 2.28.0 >