From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.1 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 596ECC4332D for ; Thu, 19 Mar 2020 18:17:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 045C520724 for ; Thu, 19 Mar 2020 18:17:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="j8pnhR3Y" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 045C520724 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 743736B000A; Thu, 19 Mar 2020 14:17:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6F3E16B000C; Thu, 19 Mar 2020 14:17:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E2C16B000D; Thu, 19 Mar 2020 14:17:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0018.hostedemail.com [216.40.44.18]) by kanga.kvack.org (Postfix) with ESMTP id 4864B6B000A for ; Thu, 19 Mar 2020 14:17:19 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id F30651ADAA for ; Thu, 19 Mar 2020 18:17:18 +0000 (UTC) X-FDA: 76612918956.21.sign07_7a07331172f25 X-HE-Tag: sign07_7a07331172f25 X-Filterd-Recvd-Size: 12706 Received: from mail-qt1-f193.google.com (mail-qt1-f193.google.com [209.85.160.193]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Thu, 19 Mar 2020 18:17:18 +0000 (UTC) Received: by mail-qt1-f193.google.com with SMTP id d12so240784qtj.4 for ; Thu, 19 Mar 2020 11:17:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=k5kvdyWMY1o6QuKYjzO+Ges86hYeEHCBwSk9Em5z3RY=; b=j8pnhR3Ywal5Z1Ent5/7bTVo3Ir3yTvmYBTrp/3TgZ1En/8VgLPzeQNOQHocQ12Fbq +wlp/G9KV5LlvxtOL4lNeHPnUqx0JKyhI/wJx4Fe1/u5387bNzRW2EQpOBhAQpvf7O9d bHrajTsRgKVQHN74GEvPveGDOcgcu6xf8gjD8QKG8ib4/H8m/FI/jD2gu3QeYxi2GFVr odN7SjSmQJx6xMNo9Nc1TVDmMlbqZ/r5iNmpgCYw+cbn0V9fm1rrH/TBS/q4fVMGw3Q5 mmxcWC57wzoY3QOun7fHS9uZLTrCPfClXiPtMxLHjQb8pwbO3lCwGxSmikmrAshS9R4D HsZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=k5kvdyWMY1o6QuKYjzO+Ges86hYeEHCBwSk9Em5z3RY=; b=WEJ5lN2rCIweDitAE217kb2JVjRxYXfhRMlhuexo9R/NtVaR3amCnMVm7hMMpl4S4g OPHSXJFVJ88r5zNqrvPxUDdc79/jIAR6ZtsFCkPwcBTKgx1E3F7Yi94Cr65oIL1n9HgU WgkNf6tHkDou9LQlzLt+56s9b+oHiGESVz1oobcT+EsMPZbuo803ljNsMI/WfnYdo6wo p8R10pAglR4NPtq6HA/ZfoAljZHejYYZtVyCGHHFx7hAuI3jvy7sQwx26kIBylYeYHZs d76UU0UEZ7jQKc6OXRc841kv59RjzwW1mq0HnqIYquAymzMaJDag8cC1Bm6y61kUvQTV dMjg== X-Gm-Message-State: ANhLgQ0739usMVB630dd0nPzWsyQ+0TVvoM7ulCO2CDPmXObQS97MIUP 2IbfmIkcU+dCeDfCJaIFm3+xmJ7w6SHqVw== X-Google-Smtp-Source: ADFU+vs9n45VQ6cNCCfzK9jn/XCI43PqbzGXPuVaqhPNr6PSXuviZAOAXhocW0lnqyPDfm1FoSx9qg== X-Received: by 2002:ac8:6b54:: with SMTP id x20mr4299882qts.41.1584641837615; Thu, 19 Mar 2020 11:17:17 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id 199sm2067530qkm.7.2020.03.19.11.17.16 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Mar 2020 11:17:16 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jEzjI-0002Wc-6t; Thu, 19 Mar 2020 15:17:16 -0300 Date: Thu, 19 Mar 2020 15:17:16 -0300 From: Jason Gunthorpe To: Ralph Campbell Cc: Christoph Hellwig , Dan Williams , Bharata B Rao , Christian =?utf-8?B?S8O2bmln?= , Ben Skeggs , Jerome Glisse , kvm-ppc@vger.kernel.org, amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-mm@kvack.org Subject: Re: [PATCH 3/4] mm: simplify device private page handling in hmm_range_fault Message-ID: <20200319181716.GK20941@ziepe.ca> References: <20200316193216.920734-1-hch@lst.de> <20200316193216.920734-4-hch@lst.de> <7256f88d-809e-4aba-3c46-a223bd8cc521@nvidia.com> <20200317121536.GQ20941@ziepe.ca> <20200317122445.GA11662@lst.de> <20200317122813.GA11866@lst.de> <20200317124755.GR20941@ziepe.ca> <20200317125955.GA12847@lst.de> <24fca825-3b0f-188f-bcf2-fadcf3a9f05a@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <24fca825-3b0f-188f-bcf2-fadcf3a9f05a@nvidia.com> User-Agent: Mutt/1.9.4 (2018-02-28) Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Mar 17, 2020 at 04:14:31PM -0700, Ralph Campbell wrote: >=20 > On 3/17/20 5:59 AM, Christoph Hellwig wrote: > > On Tue, Mar 17, 2020 at 09:47:55AM -0300, Jason Gunthorpe wrote: > > > I've been using v7 of Ralph's tester and it is working well - it ha= s > > > DEVICE_PRIVATE support so I think it can test this flow too. Ralph = are > > > you able? > > >=20 > > > This hunk seems trivial enough to me, can we include it now? > >=20 > > I can send a separate patch for it once the tester covers it. I don'= t > > want to add it to the original patch as it is a significant behavior > > change compared to the existing code. > >=20 >=20 > Attached is an updated version of my HMM tests based on linux-5.6.0-rc6= . > I ran this OK with Jason's 8+1 HMM patches, Christoph's 1-5 misc HMM cl= ean ups, > and Christoph's 1-4 device private page changes applied. I'd like to get this to mergable, it looks pretty good now, but I have no idea about selftests - and I'm struggling to even compile the tools dir > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug > index 69def4a9df00..4d22ce7879a7 100644 > +++ b/lib/Kconfig.debug > @@ -2162,6 +2162,18 @@ config TEST_MEMINIT > =20 > If unsure, say N. > =20 > +config TEST_HMM > + tristate "Test HMM (Heterogeneous Memory Management)" > + depends on DEVICE_PRIVATE > + select HMM_MIRROR > + select MMU_NOTIFIER extra spaces In general I wonder if it even makes sense that DEVICE_PRIVATE is user selectable? > +static int dmirror_fops_open(struct inode *inode, struct file *filp) > +{ > + struct cdev *cdev =3D inode->i_cdev; > + struct dmirror *dmirror; > + int ret; > + > + /* Mirror this process address space */ > + dmirror =3D kzalloc(sizeof(*dmirror), GFP_KERNEL); > + if (dmirror =3D=3D NULL) > + return -ENOMEM; > + > + dmirror->mdevice =3D container_of(cdev, struct dmirror_device, cdevic= e); > + mutex_init(&dmirror->mutex); > + xa_init(&dmirror->pt); > + > + ret =3D mmu_interval_notifier_insert(&dmirror->notifier, current->mm, > + 0, ULONG_MAX & PAGE_MASK, &dmirror_min_ops); > + if (ret) { > + kfree(dmirror); > + return ret; > + } > + > + /* Pairs with the mmdrop() in dmirror_fops_release(). */ > + mmgrab(current->mm); > + dmirror->mm =3D current->mm; The notifier holds a mmgrab, no need for another one > + /* Only the first open registers the address space. */ > + filp->private_data =3D dmirror; Not sure what this comment means > +static inline struct dmirror_device *dmirror_page_to_device(struct pag= e *page) > + > +{ > + struct dmirror_chunk *devmem; > + > + devmem =3D container_of(page->pgmap, struct dmirror_chunk, pagemap); > + return devmem->mdevice; > +} extra devmem var is not really needed > + > +static bool dmirror_device_is_mine(struct dmirror_device *mdevice, > + struct page *page) > +{ > + if (!is_zone_device_page(page)) > + return false; > + return page->pgmap->ops =3D=3D &dmirror_devmem_ops && > + dmirror_page_to_device(page) =3D=3D mdevice; > +} Use new owner stuff, right? Actually this is redunant now, the check should be just WARN_ON pageowner !=3D self owner > +static int dmirror_do_fault(struct dmirror *dmirror, struct hmm_range = *range) > +{ > + uint64_t *pfns =3D range->pfns; > + unsigned long pfn; > + > + for (pfn =3D (range->start >> PAGE_SHIFT); > + pfn < (range->end >> PAGE_SHIFT); > + pfn++, pfns++) { > + struct page *page; > + void *entry; > + > + /* > + * HMM_PFN_ERROR is returned if it is accessing invalid memory > + * either because of memory error (hardware detected memory > + * corruption) or more likely because of truncate on mmap > + * file. > + */ > + if (*pfns =3D=3D range->values[HMM_PFN_ERROR]) > + return -EFAULT; Unless that snapshot is use hmm_range_fault() never returns success and sets PFN_ERROR, so this should be a WARN_ON > + if (!(*pfns & range->flags[HMM_PFN_VALID])) > + return -EFAULT; Same with valid. > + page =3D hmm_device_entry_to_page(range, *pfns); > + /* We asked for pages to be populated but check anyway. */ > + if (!page) > + return -EFAULT; WARN_ON > + if (is_zone_device_page(page)) { > + /* > + * TODO: need a way to ask HMM to fault foreign zone > + * device private pages. > + */ > + if (!dmirror_device_is_mine(dmirror->mdevice, page)) > + continue; Actually re > +static bool dmirror_interval_invalidate(struct mmu_interval_notifier *= mni, > + const struct mmu_notifier_range *range, > + unsigned long cur_seq) > +{ > + struct dmirror *dmirror =3D container_of(mni, struct dmirror, notifie= r); > + struct mm_struct *mm =3D dmirror->mm; > + > + /* > + * If the process doesn't exist, we don't need to invalidate the > + * device page table since the address space will be torn down. > + */ > + if (!mmget_not_zero(mm)) > + return true; Why? Don't the notifiers provide for this already.=20 mmget_not_zero() is required before calling hmm_range_fault() though > +static int dmirror_fault(struct dmirror *dmirror, unsigned long start, > + unsigned long end, bool write) > +{ > + struct mm_struct *mm =3D dmirror->mm; > + unsigned long addr; > + uint64_t pfns[64]; > + struct hmm_range range =3D { > + .notifier =3D &dmirror->notifier, > + .pfns =3D pfns, > + .flags =3D dmirror_hmm_flags, > + .values =3D dmirror_hmm_values, > + .pfn_shift =3D DPT_SHIFT, > + .pfn_flags_mask =3D ~(dmirror_hmm_flags[HMM_PFN_VALID] | > + dmirror_hmm_flags[HMM_PFN_WRITE]), > + .default_flags =3D dmirror_hmm_flags[HMM_PFN_VALID] | > + (write ? dmirror_hmm_flags[HMM_PFN_WRITE] : 0), > + .dev_private_owner =3D dmirror->mdevice, > + }; > + int ret =3D 0; > + > + /* Since the mm is for the mirrored process, get a reference first. *= / > + if (!mmget_not_zero(mm)) > + return 0; Right > + for (addr =3D start; addr < end; addr =3D range.end) { > + range.start =3D addr; > + range.end =3D min(addr + (ARRAY_SIZE(pfns) << PAGE_SHIFT), end); > + > + ret =3D dmirror_range_fault(dmirror, &range); > + if (ret) > + break; > + } > + > + mmput(mm); > + return ret; > +} > + > +static int dmirror_do_read(struct dmirror *dmirror, unsigned long star= t, > + unsigned long end, struct dmirror_bounce *bounce) > +{ > + unsigned long pfn; > + void *ptr; > + > + ptr =3D bounce->ptr + ((start - bounce->addr) & PAGE_MASK); > + > + for (pfn =3D start >> PAGE_SHIFT; pfn < (end >> PAGE_SHIFT); pfn++) { > + void *entry; > + struct page *page; > + void *tmp; > + > + entry =3D xa_load(&dmirror->pt, pfn); > + page =3D xa_untag_pointer(entry); > + if (!page) > + return -ENOENT; > + > + tmp =3D kmap(page); > + memcpy(ptr, tmp, PAGE_SIZE); > + kunmap(page); > + > + ptr +=3D PAGE_SIZE; > + bounce->cpages++; > + } > + > + return 0; > +} > + > +static int dmirror_read(struct dmirror *dmirror, struct hmm_dmirror_cm= d *cmd) > +{ > + struct dmirror_bounce bounce; > + unsigned long start, end; > + unsigned long size =3D cmd->npages << PAGE_SHIFT; > + int ret; > + > + start =3D cmd->addr; > + end =3D start + size; > + if (end < start) > + return -EINVAL; > + > + ret =3D dmirror_bounce_init(&bounce, start, size); > + if (ret) > + return ret; > + > +again: > + mutex_lock(&dmirror->mutex); > + ret =3D dmirror_do_read(dmirror, start, end, &bounce); > + mutex_unlock(&dmirror->mutex); > + if (ret =3D=3D 0) > + ret =3D copy_to_user((void __user *)cmd->ptr, bounce.ptr, > + bounce.size); Use u64_to_user_ptr() instead of the cast > +static int dmirror_write(struct dmirror *dmirror, struct hmm_dmirror_c= md *cmd) > +{ > + struct dmirror_bounce bounce; > + unsigned long start, end; > + unsigned long size =3D cmd->npages << PAGE_SHIFT; > + int ret; > + > + start =3D cmd->addr; > + end =3D start + size; > + if (end < start) > + return -EINVAL; > + > + ret =3D dmirror_bounce_init(&bounce, start, size); > + if (ret) > + return ret; > + ret =3D copy_from_user(bounce.ptr, (void __user *)cmd->ptr, > + bounce.size); ditto > + if (ret) > + return ret; > + > +again: > + mutex_lock(&dmirror->mutex); > + ret =3D dmirror_do_write(dmirror, start, end, &bounce); > + mutex_unlock(&dmirror->mutex); > + if (ret =3D=3D -ENOENT) { > + start =3D cmd->addr + (bounce.cpages << PAGE_SHIFT); > + ret =3D dmirror_fault(dmirror, start, end, true); > + if (ret =3D=3D 0) { > + cmd->faults++; > + goto again; Use a loop instead of goto? Also I get this: lib/test_hmm.c: In function =E2=80=98dmirror_devmem_fault_alloc_and_copy=E2= =80=99: lib/test_hmm.c:1041:25: warning: unused variable =E2=80=98vma=E2=80=99 [-= Wunused-variable] 1041 | struct vm_area_struct *vma =3D args->vma; But this is a kernel bug, due to alloc_page_vma being a #define not a static inline and me having CONFIG_NUMA off in this .config Jason From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3439EC4332D for ; Fri, 20 Mar 2020 08:01:59 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0865A20739 for ; Fri, 20 Mar 2020 08:01:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="j8pnhR3Y" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0865A20739 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id DC8E66EAC8; Fri, 20 Mar 2020 08:01:25 +0000 (UTC) Received: from mail-qt1-x844.google.com (mail-qt1-x844.google.com [IPv6:2607:f8b0:4864:20::844]) by gabe.freedesktop.org (Postfix) with ESMTPS id C7E836EA35 for ; Thu, 19 Mar 2020 18:17:18 +0000 (UTC) Received: by mail-qt1-x844.google.com with SMTP id i9so2050774qtw.7 for ; Thu, 19 Mar 2020 11:17:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=k5kvdyWMY1o6QuKYjzO+Ges86hYeEHCBwSk9Em5z3RY=; b=j8pnhR3Ywal5Z1Ent5/7bTVo3Ir3yTvmYBTrp/3TgZ1En/8VgLPzeQNOQHocQ12Fbq +wlp/G9KV5LlvxtOL4lNeHPnUqx0JKyhI/wJx4Fe1/u5387bNzRW2EQpOBhAQpvf7O9d bHrajTsRgKVQHN74GEvPveGDOcgcu6xf8gjD8QKG8ib4/H8m/FI/jD2gu3QeYxi2GFVr odN7SjSmQJx6xMNo9Nc1TVDmMlbqZ/r5iNmpgCYw+cbn0V9fm1rrH/TBS/q4fVMGw3Q5 mmxcWC57wzoY3QOun7fHS9uZLTrCPfClXiPtMxLHjQb8pwbO3lCwGxSmikmrAshS9R4D HsZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=k5kvdyWMY1o6QuKYjzO+Ges86hYeEHCBwSk9Em5z3RY=; b=mQaeuqYNneCt7lMSe5Z0JpXS4pK9s7b0CV/1IGP0q7dZQTgs/aoNZvOZTgrVq9o9Sh uaBQvsyfI89whktxVrw1jbcUodi4pPdk4gGpavdtp1ETWPUiNDyRZwvUovZ3awmepyxP rv8Fltj3lWFGNfc2L2bAcYDXvPnFcPXUqYdk2O9vV/qoclyVdgLqw2F/cIGC+Rb4CM2o qyffRZgrEnsygRpaIDfAoR/2OdRhpoj2igtbXxs1rWzddsLO+ox88Nbw4GVkN56+E9uN APmUhWs2TBRicny85evh6dQq+AyXPyIhM2+iKY4JvDwyMKwWentS+L0oXIr/Y1+QwyAq UgDg== X-Gm-Message-State: ANhLgQ33gI6OBJe5Lm4yxOteAY4Zo2NkvQheA/xutbtxcN6yEHrTVrtf 6oLDeeXaZ1ZTa49o3FLUcbfkog== X-Google-Smtp-Source: ADFU+vs9n45VQ6cNCCfzK9jn/XCI43PqbzGXPuVaqhPNr6PSXuviZAOAXhocW0lnqyPDfm1FoSx9qg== X-Received: by 2002:ac8:6b54:: with SMTP id x20mr4299882qts.41.1584641837615; Thu, 19 Mar 2020 11:17:17 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id 199sm2067530qkm.7.2020.03.19.11.17.16 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Mar 2020 11:17:16 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jEzjI-0002Wc-6t; Thu, 19 Mar 2020 15:17:16 -0300 Date: Thu, 19 Mar 2020 15:17:16 -0300 From: Jason Gunthorpe To: Ralph Campbell Subject: Re: [PATCH 3/4] mm: simplify device private page handling in hmm_range_fault Message-ID: <20200319181716.GK20941@ziepe.ca> References: <20200316193216.920734-1-hch@lst.de> <20200316193216.920734-4-hch@lst.de> <7256f88d-809e-4aba-3c46-a223bd8cc521@nvidia.com> <20200317121536.GQ20941@ziepe.ca> <20200317122445.GA11662@lst.de> <20200317122813.GA11866@lst.de> <20200317124755.GR20941@ziepe.ca> <20200317125955.GA12847@lst.de> <24fca825-3b0f-188f-bcf2-fadcf3a9f05a@nvidia.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <24fca825-3b0f-188f-bcf2-fadcf3a9f05a@nvidia.com> User-Agent: Mutt/1.9.4 (2018-02-28) X-Mailman-Approved-At: Fri, 20 Mar 2020 08:01:24 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: amd-gfx@lists.freedesktop.org, nouveau@lists.freedesktop.org, dri-devel@lists.freedesktop.org, kvm-ppc@vger.kernel.org, Christoph Hellwig , linux-mm@kvack.org, Jerome Glisse , Ben Skeggs , Dan Williams , Bharata B Rao , Christian =?utf-8?B?S8O2bmln?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" T24gVHVlLCBNYXIgMTcsIDIwMjAgYXQgMDQ6MTQ6MzFQTSAtMDcwMCwgUmFscGggQ2FtcGJlbGwg d3JvdGU6Cj4gCj4gT24gMy8xNy8yMCA1OjU5IEFNLCBDaHJpc3RvcGggSGVsbHdpZyB3cm90ZToK PiA+IE9uIFR1ZSwgTWFyIDE3LCAyMDIwIGF0IDA5OjQ3OjU1QU0gLTAzMDAsIEphc29uIEd1bnRo b3JwZSB3cm90ZToKPiA+ID4gSSd2ZSBiZWVuIHVzaW5nIHY3IG9mIFJhbHBoJ3MgdGVzdGVyIGFu ZCBpdCBpcyB3b3JraW5nIHdlbGwgLSBpdCBoYXMKPiA+ID4gREVWSUNFX1BSSVZBVEUgc3VwcG9y dCBzbyBJIHRoaW5rIGl0IGNhbiB0ZXN0IHRoaXMgZmxvdyB0b28uIFJhbHBoIGFyZQo+ID4gPiB5 b3UgYWJsZT8KPiA+ID4gCj4gPiA+IFRoaXMgaHVuayBzZWVtcyB0cml2aWFsIGVub3VnaCB0byBt ZSwgY2FuIHdlIGluY2x1ZGUgaXQgbm93Pwo+ID4gCj4gPiBJIGNhbiBzZW5kIGEgc2VwYXJhdGUg cGF0Y2ggZm9yIGl0IG9uY2UgdGhlIHRlc3RlciBjb3ZlcnMgaXQuICBJIGRvbid0Cj4gPiB3YW50 IHRvIGFkZCBpdCB0byB0aGUgb3JpZ2luYWwgcGF0Y2ggYXMgaXQgaXMgYSBzaWduaWZpY2FudCBi ZWhhdmlvcgo+ID4gY2hhbmdlIGNvbXBhcmVkIHRvIHRoZSBleGlzdGluZyBjb2RlLgo+ID4gCj4g Cj4gQXR0YWNoZWQgaXMgYW4gdXBkYXRlZCB2ZXJzaW9uIG9mIG15IEhNTSB0ZXN0cyBiYXNlZCBv biBsaW51eC01LjYuMC1yYzYuCj4gSSByYW4gdGhpcyBPSyB3aXRoIEphc29uJ3MgOCsxIEhNTSBw YXRjaGVzLCBDaHJpc3RvcGgncyAxLTUgbWlzYyBITU0gY2xlYW4gdXBzLAo+IGFuZCBDaHJpc3Rv cGgncyAxLTQgZGV2aWNlIHByaXZhdGUgcGFnZSBjaGFuZ2VzIGFwcGxpZWQuCgpJJ2QgbGlrZSB0 byBnZXQgdGhpcyB0byBtZXJnYWJsZSwgaXQgbG9va3MgcHJldHR5IGdvb2Qgbm93LCBidXQgSSBo YXZlCm5vIGlkZWEgYWJvdXQgc2VsZnRlc3RzIC0gYW5kIEknbSBzdHJ1Z2dsaW5nIHRvIGV2ZW4g Y29tcGlsZSB0aGUgdG9vbHMKZGlyCgo+IGRpZmYgLS1naXQgYS9saWIvS2NvbmZpZy5kZWJ1ZyBi L2xpYi9LY29uZmlnLmRlYnVnCj4gaW5kZXggNjlkZWY0YTlkZjAwLi40ZDIyY2U3ODc5YTcgMTAw NjQ0Cj4gKysrIGIvbGliL0tjb25maWcuZGVidWcKPiBAQCAtMjE2Miw2ICsyMTYyLDE4IEBAIGNv bmZpZyBURVNUX01FTUlOSVQKPiAgCj4gIAkgIElmIHVuc3VyZSwgc2F5IE4uCj4gIAo+ICtjb25m aWcgVEVTVF9ITU0KPiArCXRyaXN0YXRlICJUZXN0IEhNTSAoSGV0ZXJvZ2VuZW91cyBNZW1vcnkg TWFuYWdlbWVudCkiCj4gKwlkZXBlbmRzIG9uIERFVklDRV9QUklWQVRFCj4gKwlzZWxlY3QgSE1N X01JUlJPUgo+ICsgICAgICAgIHNlbGVjdCBNTVVfTk9USUZJRVIKCmV4dHJhIHNwYWNlcwoKSW4g Z2VuZXJhbCBJIHdvbmRlciBpZiBpdCBldmVuIG1ha2VzIHNlbnNlIHRoYXQgREVWSUNFX1BSSVZB VEUgaXMgdXNlcgpzZWxlY3RhYmxlPwoKPiArc3RhdGljIGludCBkbWlycm9yX2ZvcHNfb3Blbihz dHJ1Y3QgaW5vZGUgKmlub2RlLCBzdHJ1Y3QgZmlsZSAqZmlscCkKPiArewo+ICsJc3RydWN0IGNk ZXYgKmNkZXYgPSBpbm9kZS0+aV9jZGV2Owo+ICsJc3RydWN0IGRtaXJyb3IgKmRtaXJyb3I7Cj4g KwlpbnQgcmV0Owo+ICsKPiArCS8qIE1pcnJvciB0aGlzIHByb2Nlc3MgYWRkcmVzcyBzcGFjZSAq Lwo+ICsJZG1pcnJvciA9IGt6YWxsb2Moc2l6ZW9mKCpkbWlycm9yKSwgR0ZQX0tFUk5FTCk7Cj4g KwlpZiAoZG1pcnJvciA9PSBOVUxMKQo+ICsJCXJldHVybiAtRU5PTUVNOwo+ICsKPiArCWRtaXJy b3ItPm1kZXZpY2UgPSBjb250YWluZXJfb2YoY2Rldiwgc3RydWN0IGRtaXJyb3JfZGV2aWNlLCBj ZGV2aWNlKTsKPiArCW11dGV4X2luaXQoJmRtaXJyb3ItPm11dGV4KTsKPiArCXhhX2luaXQoJmRt aXJyb3ItPnB0KTsKPiArCj4gKwlyZXQgPSBtbXVfaW50ZXJ2YWxfbm90aWZpZXJfaW5zZXJ0KCZk bWlycm9yLT5ub3RpZmllciwgY3VycmVudC0+bW0sCj4gKwkJCQkwLCBVTE9OR19NQVggJiBQQUdF X01BU0ssICZkbWlycm9yX21pbl9vcHMpOwo+ICsJaWYgKHJldCkgewo+ICsJCWtmcmVlKGRtaXJy b3IpOwo+ICsJCXJldHVybiByZXQ7Cj4gKwl9Cj4gKwo+ICsJLyogUGFpcnMgd2l0aCB0aGUgbW1k cm9wKCkgaW4gZG1pcnJvcl9mb3BzX3JlbGVhc2UoKS4gKi8KPiArCW1tZ3JhYihjdXJyZW50LT5t bSk7Cj4gKwlkbWlycm9yLT5tbSA9IGN1cnJlbnQtPm1tOwoKVGhlIG5vdGlmaWVyIGhvbGRzIGEg bW1ncmFiLCBubyBuZWVkIGZvciBhbm90aGVyIG9uZQoKPiArCS8qIE9ubHkgdGhlIGZpcnN0IG9w ZW4gcmVnaXN0ZXJzIHRoZSBhZGRyZXNzIHNwYWNlLiAqLwo+ICsJZmlscC0+cHJpdmF0ZV9kYXRh ID0gZG1pcnJvcjsKCk5vdCBzdXJlIHdoYXQgdGhpcyBjb21tZW50IG1lYW5zCgo+ICtzdGF0aWMg aW5saW5lIHN0cnVjdCBkbWlycm9yX2RldmljZSAqZG1pcnJvcl9wYWdlX3RvX2RldmljZShzdHJ1 Y3QgcGFnZSAqcGFnZSkKPiArCj4gK3sKPiArCXN0cnVjdCBkbWlycm9yX2NodW5rICpkZXZtZW07 Cj4gKwo+ICsJZGV2bWVtID0gY29udGFpbmVyX29mKHBhZ2UtPnBnbWFwLCBzdHJ1Y3QgZG1pcnJv cl9jaHVuaywgcGFnZW1hcCk7Cj4gKwlyZXR1cm4gZGV2bWVtLT5tZGV2aWNlOwo+ICt9CgpleHRy YSBkZXZtZW0gdmFyIGlzIG5vdCByZWFsbHkgbmVlZGVkCgo+ICsKPiArc3RhdGljIGJvb2wgZG1p cnJvcl9kZXZpY2VfaXNfbWluZShzdHJ1Y3QgZG1pcnJvcl9kZXZpY2UgKm1kZXZpY2UsCj4gKwkJ CQkgICBzdHJ1Y3QgcGFnZSAqcGFnZSkKPiArewo+ICsJaWYgKCFpc196b25lX2RldmljZV9wYWdl KHBhZ2UpKQo+ICsJCXJldHVybiBmYWxzZTsKPiArCXJldHVybiBwYWdlLT5wZ21hcC0+b3BzID09 ICZkbWlycm9yX2Rldm1lbV9vcHMgJiYKPiArCQlkbWlycm9yX3BhZ2VfdG9fZGV2aWNlKHBhZ2Up ID09IG1kZXZpY2U7Cj4gK30KClVzZSBuZXcgb3duZXIgc3R1ZmYsIHJpZ2h0PyBBY3R1YWxseSB0 aGlzIGlzIHJlZHVuYW50IG5vdywgdGhlIGNoZWNrCnNob3VsZCBiZSBqdXN0IFdBUk5fT04gcGFn ZW93bmVyICE9IHNlbGYgb3duZXIKCj4gK3N0YXRpYyBpbnQgZG1pcnJvcl9kb19mYXVsdChzdHJ1 Y3QgZG1pcnJvciAqZG1pcnJvciwgc3RydWN0IGhtbV9yYW5nZSAqcmFuZ2UpCj4gK3sKPiArCXVp bnQ2NF90ICpwZm5zID0gcmFuZ2UtPnBmbnM7Cj4gKwl1bnNpZ25lZCBsb25nIHBmbjsKPiArCj4g Kwlmb3IgKHBmbiA9IChyYW5nZS0+c3RhcnQgPj4gUEFHRV9TSElGVCk7Cj4gKwkgICAgIHBmbiA8 IChyYW5nZS0+ZW5kID4+IFBBR0VfU0hJRlQpOwo+ICsJICAgICBwZm4rKywgcGZucysrKSB7Cj4g KwkJc3RydWN0IHBhZ2UgKnBhZ2U7Cj4gKwkJdm9pZCAqZW50cnk7Cj4gKwo+ICsJCS8qCj4gKwkJ ICogSE1NX1BGTl9FUlJPUiBpcyByZXR1cm5lZCBpZiBpdCBpcyBhY2Nlc3NpbmcgaW52YWxpZCBt ZW1vcnkKPiArCQkgKiBlaXRoZXIgYmVjYXVzZSBvZiBtZW1vcnkgZXJyb3IgKGhhcmR3YXJlIGRl dGVjdGVkIG1lbW9yeQo+ICsJCSAqIGNvcnJ1cHRpb24pIG9yIG1vcmUgbGlrZWx5IGJlY2F1c2Ug b2YgdHJ1bmNhdGUgb24gbW1hcAo+ICsJCSAqIGZpbGUuCj4gKwkJICovCj4gKwkJaWYgKCpwZm5z ID09IHJhbmdlLT52YWx1ZXNbSE1NX1BGTl9FUlJPUl0pCj4gKwkJCXJldHVybiAtRUZBVUxUOwoK VW5sZXNzIHRoYXQgc25hcHNob3QgaXMgdXNlIGhtbV9yYW5nZV9mYXVsdCgpIG5ldmVyIHJldHVy bnMgc3VjY2VzcwphbmQgc2V0cyBQRk5fRVJST1IsIHNvIHRoaXMgc2hvdWxkIGJlIGEgV0FSTl9P TgoKPiArCQlpZiAoISgqcGZucyAmIHJhbmdlLT5mbGFnc1tITU1fUEZOX1ZBTElEXSkpCj4gKwkJ CXJldHVybiAtRUZBVUxUOwoKU2FtZSB3aXRoIHZhbGlkLgoKPiArCQlwYWdlID0gaG1tX2Rldmlj ZV9lbnRyeV90b19wYWdlKHJhbmdlLCAqcGZucyk7Cj4gKwkJLyogV2UgYXNrZWQgZm9yIHBhZ2Vz IHRvIGJlIHBvcHVsYXRlZCBidXQgY2hlY2sgYW55d2F5LiAqLwo+ICsJCWlmICghcGFnZSkKPiAr CQkJcmV0dXJuIC1FRkFVTFQ7CgpXQVJOX09OCgo+ICsJCWlmIChpc196b25lX2RldmljZV9wYWdl KHBhZ2UpKSB7Cj4gKwkJCS8qCj4gKwkJCSAqIFRPRE86IG5lZWQgYSB3YXkgdG8gYXNrIEhNTSB0 byBmYXVsdCBmb3JlaWduIHpvbmUKPiArCQkJICogZGV2aWNlIHByaXZhdGUgcGFnZXMuCj4gKwkJ CSAqLwo+ICsJCQlpZiAoIWRtaXJyb3JfZGV2aWNlX2lzX21pbmUoZG1pcnJvci0+bWRldmljZSwg cGFnZSkpCj4gKwkJCQljb250aW51ZTsKCkFjdHVhbGx5IHJlCgo+ICtzdGF0aWMgYm9vbCBkbWly cm9yX2ludGVydmFsX2ludmFsaWRhdGUoc3RydWN0IG1tdV9pbnRlcnZhbF9ub3RpZmllciAqbW5p LAo+ICsJCQkJY29uc3Qgc3RydWN0IG1tdV9ub3RpZmllcl9yYW5nZSAqcmFuZ2UsCj4gKwkJCQl1 bnNpZ25lZCBsb25nIGN1cl9zZXEpCj4gK3sKPiArCXN0cnVjdCBkbWlycm9yICpkbWlycm9yID0g Y29udGFpbmVyX29mKG1uaSwgc3RydWN0IGRtaXJyb3IsIG5vdGlmaWVyKTsKPiArCXN0cnVjdCBt bV9zdHJ1Y3QgKm1tID0gZG1pcnJvci0+bW07Cj4gKwo+ICsJLyoKPiArCSAqIElmIHRoZSBwcm9j ZXNzIGRvZXNuJ3QgZXhpc3QsIHdlIGRvbid0IG5lZWQgdG8gaW52YWxpZGF0ZSB0aGUKPiArCSAq IGRldmljZSBwYWdlIHRhYmxlIHNpbmNlIHRoZSBhZGRyZXNzIHNwYWNlIHdpbGwgYmUgdG9ybiBk b3duLgo+ICsJICovCj4gKwlpZiAoIW1tZ2V0X25vdF96ZXJvKG1tKSkKPiArCQlyZXR1cm4gdHJ1 ZTsKCldoeT8gRG9uJ3QgdGhlIG5vdGlmaWVycyBwcm92aWRlIGZvciB0aGlzIGFscmVhZHkuIAoK bW1nZXRfbm90X3plcm8oKSBpcyByZXF1aXJlZCBiZWZvcmUgY2FsbGluZyBobW1fcmFuZ2VfZmF1 bHQoKSB0aG91Z2gKCj4gK3N0YXRpYyBpbnQgZG1pcnJvcl9mYXVsdChzdHJ1Y3QgZG1pcnJvciAq ZG1pcnJvciwgdW5zaWduZWQgbG9uZyBzdGFydCwKPiArCQkJIHVuc2lnbmVkIGxvbmcgZW5kLCBi b29sIHdyaXRlKQo+ICt7Cj4gKwlzdHJ1Y3QgbW1fc3RydWN0ICptbSA9IGRtaXJyb3ItPm1tOwo+ ICsJdW5zaWduZWQgbG9uZyBhZGRyOwo+ICsJdWludDY0X3QgcGZuc1s2NF07Cj4gKwlzdHJ1Y3Qg aG1tX3JhbmdlIHJhbmdlID0gewo+ICsJCS5ub3RpZmllciA9ICZkbWlycm9yLT5ub3RpZmllciwK PiArCQkucGZucyA9IHBmbnMsCj4gKwkJLmZsYWdzID0gZG1pcnJvcl9obW1fZmxhZ3MsCj4gKwkJ LnZhbHVlcyA9IGRtaXJyb3JfaG1tX3ZhbHVlcywKPiArCQkucGZuX3NoaWZ0ID0gRFBUX1NISUZU LAo+ICsJCS5wZm5fZmxhZ3NfbWFzayA9IH4oZG1pcnJvcl9obW1fZmxhZ3NbSE1NX1BGTl9WQUxJ RF0gfAo+ICsJCQkJICAgIGRtaXJyb3JfaG1tX2ZsYWdzW0hNTV9QRk5fV1JJVEVdKSwKPiArCQku ZGVmYXVsdF9mbGFncyA9IGRtaXJyb3JfaG1tX2ZsYWdzW0hNTV9QRk5fVkFMSURdIHwKPiArCQkJ CSh3cml0ZSA/IGRtaXJyb3JfaG1tX2ZsYWdzW0hNTV9QRk5fV1JJVEVdIDogMCksCj4gKwkJLmRl dl9wcml2YXRlX293bmVyID0gZG1pcnJvci0+bWRldmljZSwKPiArCX07Cj4gKwlpbnQgcmV0ID0g MDsKPiArCj4gKwkvKiBTaW5jZSB0aGUgbW0gaXMgZm9yIHRoZSBtaXJyb3JlZCBwcm9jZXNzLCBn ZXQgYSByZWZlcmVuY2UgZmlyc3QuICovCj4gKwlpZiAoIW1tZ2V0X25vdF96ZXJvKG1tKSkKPiAr CQlyZXR1cm4gMDsKClJpZ2h0Cgo+ICsJZm9yIChhZGRyID0gc3RhcnQ7IGFkZHIgPCBlbmQ7IGFk ZHIgPSByYW5nZS5lbmQpIHsKPiArCQlyYW5nZS5zdGFydCA9IGFkZHI7Cj4gKwkJcmFuZ2UuZW5k ID0gbWluKGFkZHIgKyAoQVJSQVlfU0laRShwZm5zKSA8PCBQQUdFX1NISUZUKSwgZW5kKTsKPiAr Cj4gKwkJcmV0ID0gZG1pcnJvcl9yYW5nZV9mYXVsdChkbWlycm9yLCAmcmFuZ2UpOwo+ICsJCWlm IChyZXQpCj4gKwkJCWJyZWFrOwo+ICsJfQo+ICsKPiArCW1tcHV0KG1tKTsKPiArCXJldHVybiBy ZXQ7Cj4gK30KPiArCj4gK3N0YXRpYyBpbnQgZG1pcnJvcl9kb19yZWFkKHN0cnVjdCBkbWlycm9y ICpkbWlycm9yLCB1bnNpZ25lZCBsb25nIHN0YXJ0LAo+ICsJCQkgICB1bnNpZ25lZCBsb25nIGVu ZCwgc3RydWN0IGRtaXJyb3JfYm91bmNlICpib3VuY2UpCj4gK3sKPiArCXVuc2lnbmVkIGxvbmcg cGZuOwo+ICsJdm9pZCAqcHRyOwo+ICsKPiArCXB0ciA9IGJvdW5jZS0+cHRyICsgKChzdGFydCAt IGJvdW5jZS0+YWRkcikgJiBQQUdFX01BU0spOwo+ICsKPiArCWZvciAocGZuID0gc3RhcnQgPj4g UEFHRV9TSElGVDsgcGZuIDwgKGVuZCA+PiBQQUdFX1NISUZUKTsgcGZuKyspIHsKPiArCQl2b2lk ICplbnRyeTsKPiArCQlzdHJ1Y3QgcGFnZSAqcGFnZTsKPiArCQl2b2lkICp0bXA7Cj4gKwo+ICsJ CWVudHJ5ID0geGFfbG9hZCgmZG1pcnJvci0+cHQsIHBmbik7Cj4gKwkJcGFnZSA9IHhhX3VudGFn X3BvaW50ZXIoZW50cnkpOwo+ICsJCWlmICghcGFnZSkKPiArCQkJcmV0dXJuIC1FTk9FTlQ7Cj4g Kwo+ICsJCXRtcCA9IGttYXAocGFnZSk7Cj4gKwkJbWVtY3B5KHB0ciwgdG1wLCBQQUdFX1NJWkUp Owo+ICsJCWt1bm1hcChwYWdlKTsKPiArCj4gKwkJcHRyICs9IFBBR0VfU0laRTsKPiArCQlib3Vu Y2UtPmNwYWdlcysrOwo+ICsJfQo+ICsKPiArCXJldHVybiAwOwo+ICt9Cj4gKwo+ICtzdGF0aWMg aW50IGRtaXJyb3JfcmVhZChzdHJ1Y3QgZG1pcnJvciAqZG1pcnJvciwgc3RydWN0IGhtbV9kbWly cm9yX2NtZCAqY21kKQo+ICt7Cj4gKwlzdHJ1Y3QgZG1pcnJvcl9ib3VuY2UgYm91bmNlOwo+ICsJ dW5zaWduZWQgbG9uZyBzdGFydCwgZW5kOwo+ICsJdW5zaWduZWQgbG9uZyBzaXplID0gY21kLT5u cGFnZXMgPDwgUEFHRV9TSElGVDsKPiArCWludCByZXQ7Cj4gKwo+ICsJc3RhcnQgPSBjbWQtPmFk ZHI7Cj4gKwllbmQgPSBzdGFydCArIHNpemU7Cj4gKwlpZiAoZW5kIDwgc3RhcnQpCj4gKwkJcmV0 dXJuIC1FSU5WQUw7Cj4gKwo+ICsJcmV0ID0gZG1pcnJvcl9ib3VuY2VfaW5pdCgmYm91bmNlLCBz dGFydCwgc2l6ZSk7Cj4gKwlpZiAocmV0KQo+ICsJCXJldHVybiByZXQ7Cj4gKwo+ICthZ2FpbjoK PiArCW11dGV4X2xvY2soJmRtaXJyb3ItPm11dGV4KTsKPiArCXJldCA9IGRtaXJyb3JfZG9fcmVh ZChkbWlycm9yLCBzdGFydCwgZW5kLCAmYm91bmNlKTsKPiArCW11dGV4X3VubG9jaygmZG1pcnJv ci0+bXV0ZXgpOwo+ICsJaWYgKHJldCA9PSAwKQo+ICsJCXJldCA9IGNvcHlfdG9fdXNlcigodm9p ZCBfX3VzZXIgKiljbWQtPnB0ciwgYm91bmNlLnB0ciwKPiArCQkJCQlib3VuY2Uuc2l6ZSk7CgpV c2UgdTY0X3RvX3VzZXJfcHRyKCkgaW5zdGVhZCBvZiB0aGUgY2FzdAoKPiArc3RhdGljIGludCBk bWlycm9yX3dyaXRlKHN0cnVjdCBkbWlycm9yICpkbWlycm9yLCBzdHJ1Y3QgaG1tX2RtaXJyb3Jf Y21kICpjbWQpCj4gK3sKPiArCXN0cnVjdCBkbWlycm9yX2JvdW5jZSBib3VuY2U7Cj4gKwl1bnNp Z25lZCBsb25nIHN0YXJ0LCBlbmQ7Cj4gKwl1bnNpZ25lZCBsb25nIHNpemUgPSBjbWQtPm5wYWdl cyA8PCBQQUdFX1NISUZUOwo+ICsJaW50IHJldDsKPiArCj4gKwlzdGFydCA9IGNtZC0+YWRkcjsK PiArCWVuZCA9IHN0YXJ0ICsgc2l6ZTsKPiArCWlmIChlbmQgPCBzdGFydCkKPiArCQlyZXR1cm4g LUVJTlZBTDsKPiArCj4gKwlyZXQgPSBkbWlycm9yX2JvdW5jZV9pbml0KCZib3VuY2UsIHN0YXJ0 LCBzaXplKTsKPiArCWlmIChyZXQpCj4gKwkJcmV0dXJuIHJldDsKPiArCXJldCA9IGNvcHlfZnJv bV91c2VyKGJvdW5jZS5wdHIsICh2b2lkIF9fdXNlciAqKWNtZC0+cHRyLAo+ICsJCQkJYm91bmNl LnNpemUpOwoKZGl0dG8KCj4gKwlpZiAocmV0KQo+ICsJCXJldHVybiByZXQ7Cj4gKwo+ICthZ2Fp bjoKPiArCW11dGV4X2xvY2soJmRtaXJyb3ItPm11dGV4KTsKPiArCXJldCA9IGRtaXJyb3JfZG9f d3JpdGUoZG1pcnJvciwgc3RhcnQsIGVuZCwgJmJvdW5jZSk7Cj4gKwltdXRleF91bmxvY2soJmRt aXJyb3ItPm11dGV4KTsKPiArCWlmIChyZXQgPT0gLUVOT0VOVCkgewo+ICsJCXN0YXJ0ID0gY21k LT5hZGRyICsgKGJvdW5jZS5jcGFnZXMgPDwgUEFHRV9TSElGVCk7Cj4gKwkJcmV0ID0gZG1pcnJv cl9mYXVsdChkbWlycm9yLCBzdGFydCwgZW5kLCB0cnVlKTsKPiArCQlpZiAocmV0ID09IDApIHsK PiArCQkJY21kLT5mYXVsdHMrKzsKPiArCQkJZ290byBhZ2FpbjsKClVzZSBhIGxvb3AgaW5zdGVh ZCBvZiBnb3RvPwoKQWxzbyBJIGdldCB0aGlzOgoKbGliL3Rlc3RfaG1tLmM6IEluIGZ1bmN0aW9u IOKAmGRtaXJyb3JfZGV2bWVtX2ZhdWx0X2FsbG9jX2FuZF9jb3B54oCZOgpsaWIvdGVzdF9obW0u YzoxMDQxOjI1OiB3YXJuaW5nOiB1bnVzZWQgdmFyaWFibGUg4oCYdm1h4oCZIFstV3VudXNlZC12 YXJpYWJsZV0KIDEwNDEgfCAgc3RydWN0IHZtX2FyZWFfc3RydWN0ICp2bWEgPSBhcmdzLT52bWE7 CgpCdXQgdGhpcyBpcyBhIGtlcm5lbCBidWcsIGR1ZSB0byBhbGxvY19wYWdlX3ZtYSBiZWluZyBh ICNkZWZpbmUgbm90IGEKc3RhdGljIGlubGluZSBhbmQgbWUgaGF2aW5nIENPTkZJR19OVU1BIG9m ZiBpbiB0aGlzIC5jb25maWcKCkphc29uCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fCmRyaS1kZXZlbCBtYWlsaW5nIGxpc3QKZHJpLWRldmVsQGxpc3RzLmZy ZWVkZXNrdG9wLm9yZwpodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3Rp bmZvL2RyaS1kZXZlbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.1 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19C7DC4332B for ; Thu, 19 Mar 2020 18:18:15 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E39EE20724 for ; Thu, 19 Mar 2020 18:18:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="j8pnhR3Y" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E39EE20724 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=amd-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 812F36EA35; Thu, 19 Mar 2020 18:18:14 +0000 (UTC) Received: from mail-qt1-x844.google.com (mail-qt1-x844.google.com [IPv6:2607:f8b0:4864:20::844]) by gabe.freedesktop.org (Postfix) with ESMTPS id C947C6EA48 for ; Thu, 19 Mar 2020 18:17:18 +0000 (UTC) Received: by mail-qt1-x844.google.com with SMTP id z8so2656229qto.12 for ; Thu, 19 Mar 2020 11:17:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=k5kvdyWMY1o6QuKYjzO+Ges86hYeEHCBwSk9Em5z3RY=; b=j8pnhR3Ywal5Z1Ent5/7bTVo3Ir3yTvmYBTrp/3TgZ1En/8VgLPzeQNOQHocQ12Fbq +wlp/G9KV5LlvxtOL4lNeHPnUqx0JKyhI/wJx4Fe1/u5387bNzRW2EQpOBhAQpvf7O9d bHrajTsRgKVQHN74GEvPveGDOcgcu6xf8gjD8QKG8ib4/H8m/FI/jD2gu3QeYxi2GFVr odN7SjSmQJx6xMNo9Nc1TVDmMlbqZ/r5iNmpgCYw+cbn0V9fm1rrH/TBS/q4fVMGw3Q5 mmxcWC57wzoY3QOun7fHS9uZLTrCPfClXiPtMxLHjQb8pwbO3lCwGxSmikmrAshS9R4D HsZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=k5kvdyWMY1o6QuKYjzO+Ges86hYeEHCBwSk9Em5z3RY=; b=XObveylBtQ1+c1ZRj0GV3fuom/2U1IR0afiZsM93prSqbpXeVHnott5tRHtB4Kaclc xH6fK1hTuaRsrjOWFKDcZu/XO4emukWFzj/iBm29c+febzsPXI4tGT7HnDB0BFUl1Kw4 dkcHvyzZRJplanWik25/1iObBkU3pTs7qusSQNQPxGN312MgD6dWRwwP0nIyOf10UlbI Hs7Mxm2ZPP7FuDPUAUNlZXss5EcqjZvSggvDyakHcQIJ1eOli51pEQEHINgaGimmSsOj QsYOqdcicZyxZYTq9H82s4fMUlnuCTI4OvGF5OXrrHESPuM9QBPiKqGPbFNjRDJarCNj lA/A== X-Gm-Message-State: ANhLgQ0t9qCyGXZUBToqcwQB4J/z2w0iprgpMx9mAkGoM6zMEyL33GK8 utKo7D/vUDtx5sWtoxiKWhKi5g== X-Google-Smtp-Source: ADFU+vs9n45VQ6cNCCfzK9jn/XCI43PqbzGXPuVaqhPNr6PSXuviZAOAXhocW0lnqyPDfm1FoSx9qg== X-Received: by 2002:ac8:6b54:: with SMTP id x20mr4299882qts.41.1584641837615; Thu, 19 Mar 2020 11:17:17 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id 199sm2067530qkm.7.2020.03.19.11.17.16 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Mar 2020 11:17:16 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jEzjI-0002Wc-6t; Thu, 19 Mar 2020 15:17:16 -0300 Date: Thu, 19 Mar 2020 15:17:16 -0300 From: Jason Gunthorpe To: Ralph Campbell Subject: Re: [PATCH 3/4] mm: simplify device private page handling in hmm_range_fault Message-ID: <20200319181716.GK20941@ziepe.ca> References: <20200316193216.920734-1-hch@lst.de> <20200316193216.920734-4-hch@lst.de> <7256f88d-809e-4aba-3c46-a223bd8cc521@nvidia.com> <20200317121536.GQ20941@ziepe.ca> <20200317122445.GA11662@lst.de> <20200317122813.GA11866@lst.de> <20200317124755.GR20941@ziepe.ca> <20200317125955.GA12847@lst.de> <24fca825-3b0f-188f-bcf2-fadcf3a9f05a@nvidia.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <24fca825-3b0f-188f-bcf2-fadcf3a9f05a@nvidia.com> User-Agent: Mutt/1.9.4 (2018-02-28) X-Mailman-Approved-At: Thu, 19 Mar 2020 18:18:14 +0000 X-BeenThere: amd-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion list for AMD gfx List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: amd-gfx@lists.freedesktop.org, nouveau@lists.freedesktop.org, dri-devel@lists.freedesktop.org, kvm-ppc@vger.kernel.org, Christoph Hellwig , linux-mm@kvack.org, Jerome Glisse , Ben Skeggs , Dan Williams , Bharata B Rao , Christian =?utf-8?B?S8O2bmln?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: amd-gfx-bounces@lists.freedesktop.org Sender: "amd-gfx" T24gVHVlLCBNYXIgMTcsIDIwMjAgYXQgMDQ6MTQ6MzFQTSAtMDcwMCwgUmFscGggQ2FtcGJlbGwg d3JvdGU6Cj4gCj4gT24gMy8xNy8yMCA1OjU5IEFNLCBDaHJpc3RvcGggSGVsbHdpZyB3cm90ZToK PiA+IE9uIFR1ZSwgTWFyIDE3LCAyMDIwIGF0IDA5OjQ3OjU1QU0gLTAzMDAsIEphc29uIEd1bnRo b3JwZSB3cm90ZToKPiA+ID4gSSd2ZSBiZWVuIHVzaW5nIHY3IG9mIFJhbHBoJ3MgdGVzdGVyIGFu ZCBpdCBpcyB3b3JraW5nIHdlbGwgLSBpdCBoYXMKPiA+ID4gREVWSUNFX1BSSVZBVEUgc3VwcG9y dCBzbyBJIHRoaW5rIGl0IGNhbiB0ZXN0IHRoaXMgZmxvdyB0b28uIFJhbHBoIGFyZQo+ID4gPiB5 b3UgYWJsZT8KPiA+ID4gCj4gPiA+IFRoaXMgaHVuayBzZWVtcyB0cml2aWFsIGVub3VnaCB0byBt ZSwgY2FuIHdlIGluY2x1ZGUgaXQgbm93Pwo+ID4gCj4gPiBJIGNhbiBzZW5kIGEgc2VwYXJhdGUg cGF0Y2ggZm9yIGl0IG9uY2UgdGhlIHRlc3RlciBjb3ZlcnMgaXQuICBJIGRvbid0Cj4gPiB3YW50 IHRvIGFkZCBpdCB0byB0aGUgb3JpZ2luYWwgcGF0Y2ggYXMgaXQgaXMgYSBzaWduaWZpY2FudCBi ZWhhdmlvcgo+ID4gY2hhbmdlIGNvbXBhcmVkIHRvIHRoZSBleGlzdGluZyBjb2RlLgo+ID4gCj4g Cj4gQXR0YWNoZWQgaXMgYW4gdXBkYXRlZCB2ZXJzaW9uIG9mIG15IEhNTSB0ZXN0cyBiYXNlZCBv biBsaW51eC01LjYuMC1yYzYuCj4gSSByYW4gdGhpcyBPSyB3aXRoIEphc29uJ3MgOCsxIEhNTSBw YXRjaGVzLCBDaHJpc3RvcGgncyAxLTUgbWlzYyBITU0gY2xlYW4gdXBzLAo+IGFuZCBDaHJpc3Rv cGgncyAxLTQgZGV2aWNlIHByaXZhdGUgcGFnZSBjaGFuZ2VzIGFwcGxpZWQuCgpJJ2QgbGlrZSB0 byBnZXQgdGhpcyB0byBtZXJnYWJsZSwgaXQgbG9va3MgcHJldHR5IGdvb2Qgbm93LCBidXQgSSBo YXZlCm5vIGlkZWEgYWJvdXQgc2VsZnRlc3RzIC0gYW5kIEknbSBzdHJ1Z2dsaW5nIHRvIGV2ZW4g Y29tcGlsZSB0aGUgdG9vbHMKZGlyCgo+IGRpZmYgLS1naXQgYS9saWIvS2NvbmZpZy5kZWJ1ZyBi L2xpYi9LY29uZmlnLmRlYnVnCj4gaW5kZXggNjlkZWY0YTlkZjAwLi40ZDIyY2U3ODc5YTcgMTAw NjQ0Cj4gKysrIGIvbGliL0tjb25maWcuZGVidWcKPiBAQCAtMjE2Miw2ICsyMTYyLDE4IEBAIGNv bmZpZyBURVNUX01FTUlOSVQKPiAgCj4gIAkgIElmIHVuc3VyZSwgc2F5IE4uCj4gIAo+ICtjb25m aWcgVEVTVF9ITU0KPiArCXRyaXN0YXRlICJUZXN0IEhNTSAoSGV0ZXJvZ2VuZW91cyBNZW1vcnkg TWFuYWdlbWVudCkiCj4gKwlkZXBlbmRzIG9uIERFVklDRV9QUklWQVRFCj4gKwlzZWxlY3QgSE1N X01JUlJPUgo+ICsgICAgICAgIHNlbGVjdCBNTVVfTk9USUZJRVIKCmV4dHJhIHNwYWNlcwoKSW4g Z2VuZXJhbCBJIHdvbmRlciBpZiBpdCBldmVuIG1ha2VzIHNlbnNlIHRoYXQgREVWSUNFX1BSSVZB VEUgaXMgdXNlcgpzZWxlY3RhYmxlPwoKPiArc3RhdGljIGludCBkbWlycm9yX2ZvcHNfb3Blbihz dHJ1Y3QgaW5vZGUgKmlub2RlLCBzdHJ1Y3QgZmlsZSAqZmlscCkKPiArewo+ICsJc3RydWN0IGNk ZXYgKmNkZXYgPSBpbm9kZS0+aV9jZGV2Owo+ICsJc3RydWN0IGRtaXJyb3IgKmRtaXJyb3I7Cj4g KwlpbnQgcmV0Owo+ICsKPiArCS8qIE1pcnJvciB0aGlzIHByb2Nlc3MgYWRkcmVzcyBzcGFjZSAq Lwo+ICsJZG1pcnJvciA9IGt6YWxsb2Moc2l6ZW9mKCpkbWlycm9yKSwgR0ZQX0tFUk5FTCk7Cj4g KwlpZiAoZG1pcnJvciA9PSBOVUxMKQo+ICsJCXJldHVybiAtRU5PTUVNOwo+ICsKPiArCWRtaXJy b3ItPm1kZXZpY2UgPSBjb250YWluZXJfb2YoY2Rldiwgc3RydWN0IGRtaXJyb3JfZGV2aWNlLCBj ZGV2aWNlKTsKPiArCW11dGV4X2luaXQoJmRtaXJyb3ItPm11dGV4KTsKPiArCXhhX2luaXQoJmRt aXJyb3ItPnB0KTsKPiArCj4gKwlyZXQgPSBtbXVfaW50ZXJ2YWxfbm90aWZpZXJfaW5zZXJ0KCZk bWlycm9yLT5ub3RpZmllciwgY3VycmVudC0+bW0sCj4gKwkJCQkwLCBVTE9OR19NQVggJiBQQUdF X01BU0ssICZkbWlycm9yX21pbl9vcHMpOwo+ICsJaWYgKHJldCkgewo+ICsJCWtmcmVlKGRtaXJy b3IpOwo+ICsJCXJldHVybiByZXQ7Cj4gKwl9Cj4gKwo+ICsJLyogUGFpcnMgd2l0aCB0aGUgbW1k cm9wKCkgaW4gZG1pcnJvcl9mb3BzX3JlbGVhc2UoKS4gKi8KPiArCW1tZ3JhYihjdXJyZW50LT5t bSk7Cj4gKwlkbWlycm9yLT5tbSA9IGN1cnJlbnQtPm1tOwoKVGhlIG5vdGlmaWVyIGhvbGRzIGEg bW1ncmFiLCBubyBuZWVkIGZvciBhbm90aGVyIG9uZQoKPiArCS8qIE9ubHkgdGhlIGZpcnN0IG9w ZW4gcmVnaXN0ZXJzIHRoZSBhZGRyZXNzIHNwYWNlLiAqLwo+ICsJZmlscC0+cHJpdmF0ZV9kYXRh ID0gZG1pcnJvcjsKCk5vdCBzdXJlIHdoYXQgdGhpcyBjb21tZW50IG1lYW5zCgo+ICtzdGF0aWMg aW5saW5lIHN0cnVjdCBkbWlycm9yX2RldmljZSAqZG1pcnJvcl9wYWdlX3RvX2RldmljZShzdHJ1 Y3QgcGFnZSAqcGFnZSkKPiArCj4gK3sKPiArCXN0cnVjdCBkbWlycm9yX2NodW5rICpkZXZtZW07 Cj4gKwo+ICsJZGV2bWVtID0gY29udGFpbmVyX29mKHBhZ2UtPnBnbWFwLCBzdHJ1Y3QgZG1pcnJv cl9jaHVuaywgcGFnZW1hcCk7Cj4gKwlyZXR1cm4gZGV2bWVtLT5tZGV2aWNlOwo+ICt9CgpleHRy YSBkZXZtZW0gdmFyIGlzIG5vdCByZWFsbHkgbmVlZGVkCgo+ICsKPiArc3RhdGljIGJvb2wgZG1p cnJvcl9kZXZpY2VfaXNfbWluZShzdHJ1Y3QgZG1pcnJvcl9kZXZpY2UgKm1kZXZpY2UsCj4gKwkJ CQkgICBzdHJ1Y3QgcGFnZSAqcGFnZSkKPiArewo+ICsJaWYgKCFpc196b25lX2RldmljZV9wYWdl KHBhZ2UpKQo+ICsJCXJldHVybiBmYWxzZTsKPiArCXJldHVybiBwYWdlLT5wZ21hcC0+b3BzID09 ICZkbWlycm9yX2Rldm1lbV9vcHMgJiYKPiArCQlkbWlycm9yX3BhZ2VfdG9fZGV2aWNlKHBhZ2Up ID09IG1kZXZpY2U7Cj4gK30KClVzZSBuZXcgb3duZXIgc3R1ZmYsIHJpZ2h0PyBBY3R1YWxseSB0 aGlzIGlzIHJlZHVuYW50IG5vdywgdGhlIGNoZWNrCnNob3VsZCBiZSBqdXN0IFdBUk5fT04gcGFn ZW93bmVyICE9IHNlbGYgb3duZXIKCj4gK3N0YXRpYyBpbnQgZG1pcnJvcl9kb19mYXVsdChzdHJ1 Y3QgZG1pcnJvciAqZG1pcnJvciwgc3RydWN0IGhtbV9yYW5nZSAqcmFuZ2UpCj4gK3sKPiArCXVp bnQ2NF90ICpwZm5zID0gcmFuZ2UtPnBmbnM7Cj4gKwl1bnNpZ25lZCBsb25nIHBmbjsKPiArCj4g Kwlmb3IgKHBmbiA9IChyYW5nZS0+c3RhcnQgPj4gUEFHRV9TSElGVCk7Cj4gKwkgICAgIHBmbiA8 IChyYW5nZS0+ZW5kID4+IFBBR0VfU0hJRlQpOwo+ICsJICAgICBwZm4rKywgcGZucysrKSB7Cj4g KwkJc3RydWN0IHBhZ2UgKnBhZ2U7Cj4gKwkJdm9pZCAqZW50cnk7Cj4gKwo+ICsJCS8qCj4gKwkJ ICogSE1NX1BGTl9FUlJPUiBpcyByZXR1cm5lZCBpZiBpdCBpcyBhY2Nlc3NpbmcgaW52YWxpZCBt ZW1vcnkKPiArCQkgKiBlaXRoZXIgYmVjYXVzZSBvZiBtZW1vcnkgZXJyb3IgKGhhcmR3YXJlIGRl dGVjdGVkIG1lbW9yeQo+ICsJCSAqIGNvcnJ1cHRpb24pIG9yIG1vcmUgbGlrZWx5IGJlY2F1c2Ug b2YgdHJ1bmNhdGUgb24gbW1hcAo+ICsJCSAqIGZpbGUuCj4gKwkJICovCj4gKwkJaWYgKCpwZm5z ID09IHJhbmdlLT52YWx1ZXNbSE1NX1BGTl9FUlJPUl0pCj4gKwkJCXJldHVybiAtRUZBVUxUOwoK VW5sZXNzIHRoYXQgc25hcHNob3QgaXMgdXNlIGhtbV9yYW5nZV9mYXVsdCgpIG5ldmVyIHJldHVy bnMgc3VjY2VzcwphbmQgc2V0cyBQRk5fRVJST1IsIHNvIHRoaXMgc2hvdWxkIGJlIGEgV0FSTl9P TgoKPiArCQlpZiAoISgqcGZucyAmIHJhbmdlLT5mbGFnc1tITU1fUEZOX1ZBTElEXSkpCj4gKwkJ CXJldHVybiAtRUZBVUxUOwoKU2FtZSB3aXRoIHZhbGlkLgoKPiArCQlwYWdlID0gaG1tX2Rldmlj ZV9lbnRyeV90b19wYWdlKHJhbmdlLCAqcGZucyk7Cj4gKwkJLyogV2UgYXNrZWQgZm9yIHBhZ2Vz IHRvIGJlIHBvcHVsYXRlZCBidXQgY2hlY2sgYW55d2F5LiAqLwo+ICsJCWlmICghcGFnZSkKPiAr CQkJcmV0dXJuIC1FRkFVTFQ7CgpXQVJOX09OCgo+ICsJCWlmIChpc196b25lX2RldmljZV9wYWdl KHBhZ2UpKSB7Cj4gKwkJCS8qCj4gKwkJCSAqIFRPRE86IG5lZWQgYSB3YXkgdG8gYXNrIEhNTSB0 byBmYXVsdCBmb3JlaWduIHpvbmUKPiArCQkJICogZGV2aWNlIHByaXZhdGUgcGFnZXMuCj4gKwkJ CSAqLwo+ICsJCQlpZiAoIWRtaXJyb3JfZGV2aWNlX2lzX21pbmUoZG1pcnJvci0+bWRldmljZSwg cGFnZSkpCj4gKwkJCQljb250aW51ZTsKCkFjdHVhbGx5IHJlCgo+ICtzdGF0aWMgYm9vbCBkbWly cm9yX2ludGVydmFsX2ludmFsaWRhdGUoc3RydWN0IG1tdV9pbnRlcnZhbF9ub3RpZmllciAqbW5p LAo+ICsJCQkJY29uc3Qgc3RydWN0IG1tdV9ub3RpZmllcl9yYW5nZSAqcmFuZ2UsCj4gKwkJCQl1 bnNpZ25lZCBsb25nIGN1cl9zZXEpCj4gK3sKPiArCXN0cnVjdCBkbWlycm9yICpkbWlycm9yID0g Y29udGFpbmVyX29mKG1uaSwgc3RydWN0IGRtaXJyb3IsIG5vdGlmaWVyKTsKPiArCXN0cnVjdCBt bV9zdHJ1Y3QgKm1tID0gZG1pcnJvci0+bW07Cj4gKwo+ICsJLyoKPiArCSAqIElmIHRoZSBwcm9j ZXNzIGRvZXNuJ3QgZXhpc3QsIHdlIGRvbid0IG5lZWQgdG8gaW52YWxpZGF0ZSB0aGUKPiArCSAq IGRldmljZSBwYWdlIHRhYmxlIHNpbmNlIHRoZSBhZGRyZXNzIHNwYWNlIHdpbGwgYmUgdG9ybiBk b3duLgo+ICsJICovCj4gKwlpZiAoIW1tZ2V0X25vdF96ZXJvKG1tKSkKPiArCQlyZXR1cm4gdHJ1 ZTsKCldoeT8gRG9uJ3QgdGhlIG5vdGlmaWVycyBwcm92aWRlIGZvciB0aGlzIGFscmVhZHkuIAoK bW1nZXRfbm90X3plcm8oKSBpcyByZXF1aXJlZCBiZWZvcmUgY2FsbGluZyBobW1fcmFuZ2VfZmF1 bHQoKSB0aG91Z2gKCj4gK3N0YXRpYyBpbnQgZG1pcnJvcl9mYXVsdChzdHJ1Y3QgZG1pcnJvciAq ZG1pcnJvciwgdW5zaWduZWQgbG9uZyBzdGFydCwKPiArCQkJIHVuc2lnbmVkIGxvbmcgZW5kLCBi b29sIHdyaXRlKQo+ICt7Cj4gKwlzdHJ1Y3QgbW1fc3RydWN0ICptbSA9IGRtaXJyb3ItPm1tOwo+ ICsJdW5zaWduZWQgbG9uZyBhZGRyOwo+ICsJdWludDY0X3QgcGZuc1s2NF07Cj4gKwlzdHJ1Y3Qg aG1tX3JhbmdlIHJhbmdlID0gewo+ICsJCS5ub3RpZmllciA9ICZkbWlycm9yLT5ub3RpZmllciwK PiArCQkucGZucyA9IHBmbnMsCj4gKwkJLmZsYWdzID0gZG1pcnJvcl9obW1fZmxhZ3MsCj4gKwkJ LnZhbHVlcyA9IGRtaXJyb3JfaG1tX3ZhbHVlcywKPiArCQkucGZuX3NoaWZ0ID0gRFBUX1NISUZU LAo+ICsJCS5wZm5fZmxhZ3NfbWFzayA9IH4oZG1pcnJvcl9obW1fZmxhZ3NbSE1NX1BGTl9WQUxJ RF0gfAo+ICsJCQkJICAgIGRtaXJyb3JfaG1tX2ZsYWdzW0hNTV9QRk5fV1JJVEVdKSwKPiArCQku ZGVmYXVsdF9mbGFncyA9IGRtaXJyb3JfaG1tX2ZsYWdzW0hNTV9QRk5fVkFMSURdIHwKPiArCQkJ CSh3cml0ZSA/IGRtaXJyb3JfaG1tX2ZsYWdzW0hNTV9QRk5fV1JJVEVdIDogMCksCj4gKwkJLmRl dl9wcml2YXRlX293bmVyID0gZG1pcnJvci0+bWRldmljZSwKPiArCX07Cj4gKwlpbnQgcmV0ID0g MDsKPiArCj4gKwkvKiBTaW5jZSB0aGUgbW0gaXMgZm9yIHRoZSBtaXJyb3JlZCBwcm9jZXNzLCBn ZXQgYSByZWZlcmVuY2UgZmlyc3QuICovCj4gKwlpZiAoIW1tZ2V0X25vdF96ZXJvKG1tKSkKPiAr CQlyZXR1cm4gMDsKClJpZ2h0Cgo+ICsJZm9yIChhZGRyID0gc3RhcnQ7IGFkZHIgPCBlbmQ7IGFk ZHIgPSByYW5nZS5lbmQpIHsKPiArCQlyYW5nZS5zdGFydCA9IGFkZHI7Cj4gKwkJcmFuZ2UuZW5k ID0gbWluKGFkZHIgKyAoQVJSQVlfU0laRShwZm5zKSA8PCBQQUdFX1NISUZUKSwgZW5kKTsKPiAr Cj4gKwkJcmV0ID0gZG1pcnJvcl9yYW5nZV9mYXVsdChkbWlycm9yLCAmcmFuZ2UpOwo+ICsJCWlm IChyZXQpCj4gKwkJCWJyZWFrOwo+ICsJfQo+ICsKPiArCW1tcHV0KG1tKTsKPiArCXJldHVybiBy ZXQ7Cj4gK30KPiArCj4gK3N0YXRpYyBpbnQgZG1pcnJvcl9kb19yZWFkKHN0cnVjdCBkbWlycm9y ICpkbWlycm9yLCB1bnNpZ25lZCBsb25nIHN0YXJ0LAo+ICsJCQkgICB1bnNpZ25lZCBsb25nIGVu ZCwgc3RydWN0IGRtaXJyb3JfYm91bmNlICpib3VuY2UpCj4gK3sKPiArCXVuc2lnbmVkIGxvbmcg cGZuOwo+ICsJdm9pZCAqcHRyOwo+ICsKPiArCXB0ciA9IGJvdW5jZS0+cHRyICsgKChzdGFydCAt IGJvdW5jZS0+YWRkcikgJiBQQUdFX01BU0spOwo+ICsKPiArCWZvciAocGZuID0gc3RhcnQgPj4g UEFHRV9TSElGVDsgcGZuIDwgKGVuZCA+PiBQQUdFX1NISUZUKTsgcGZuKyspIHsKPiArCQl2b2lk ICplbnRyeTsKPiArCQlzdHJ1Y3QgcGFnZSAqcGFnZTsKPiArCQl2b2lkICp0bXA7Cj4gKwo+ICsJ CWVudHJ5ID0geGFfbG9hZCgmZG1pcnJvci0+cHQsIHBmbik7Cj4gKwkJcGFnZSA9IHhhX3VudGFn X3BvaW50ZXIoZW50cnkpOwo+ICsJCWlmICghcGFnZSkKPiArCQkJcmV0dXJuIC1FTk9FTlQ7Cj4g Kwo+ICsJCXRtcCA9IGttYXAocGFnZSk7Cj4gKwkJbWVtY3B5KHB0ciwgdG1wLCBQQUdFX1NJWkUp Owo+ICsJCWt1bm1hcChwYWdlKTsKPiArCj4gKwkJcHRyICs9IFBBR0VfU0laRTsKPiArCQlib3Vu Y2UtPmNwYWdlcysrOwo+ICsJfQo+ICsKPiArCXJldHVybiAwOwo+ICt9Cj4gKwo+ICtzdGF0aWMg aW50IGRtaXJyb3JfcmVhZChzdHJ1Y3QgZG1pcnJvciAqZG1pcnJvciwgc3RydWN0IGhtbV9kbWly cm9yX2NtZCAqY21kKQo+ICt7Cj4gKwlzdHJ1Y3QgZG1pcnJvcl9ib3VuY2UgYm91bmNlOwo+ICsJ dW5zaWduZWQgbG9uZyBzdGFydCwgZW5kOwo+ICsJdW5zaWduZWQgbG9uZyBzaXplID0gY21kLT5u cGFnZXMgPDwgUEFHRV9TSElGVDsKPiArCWludCByZXQ7Cj4gKwo+ICsJc3RhcnQgPSBjbWQtPmFk ZHI7Cj4gKwllbmQgPSBzdGFydCArIHNpemU7Cj4gKwlpZiAoZW5kIDwgc3RhcnQpCj4gKwkJcmV0 dXJuIC1FSU5WQUw7Cj4gKwo+ICsJcmV0ID0gZG1pcnJvcl9ib3VuY2VfaW5pdCgmYm91bmNlLCBz dGFydCwgc2l6ZSk7Cj4gKwlpZiAocmV0KQo+ICsJCXJldHVybiByZXQ7Cj4gKwo+ICthZ2FpbjoK PiArCW11dGV4X2xvY2soJmRtaXJyb3ItPm11dGV4KTsKPiArCXJldCA9IGRtaXJyb3JfZG9fcmVh ZChkbWlycm9yLCBzdGFydCwgZW5kLCAmYm91bmNlKTsKPiArCW11dGV4X3VubG9jaygmZG1pcnJv ci0+bXV0ZXgpOwo+ICsJaWYgKHJldCA9PSAwKQo+ICsJCXJldCA9IGNvcHlfdG9fdXNlcigodm9p ZCBfX3VzZXIgKiljbWQtPnB0ciwgYm91bmNlLnB0ciwKPiArCQkJCQlib3VuY2Uuc2l6ZSk7CgpV c2UgdTY0X3RvX3VzZXJfcHRyKCkgaW5zdGVhZCBvZiB0aGUgY2FzdAoKPiArc3RhdGljIGludCBk bWlycm9yX3dyaXRlKHN0cnVjdCBkbWlycm9yICpkbWlycm9yLCBzdHJ1Y3QgaG1tX2RtaXJyb3Jf Y21kICpjbWQpCj4gK3sKPiArCXN0cnVjdCBkbWlycm9yX2JvdW5jZSBib3VuY2U7Cj4gKwl1bnNp Z25lZCBsb25nIHN0YXJ0LCBlbmQ7Cj4gKwl1bnNpZ25lZCBsb25nIHNpemUgPSBjbWQtPm5wYWdl cyA8PCBQQUdFX1NISUZUOwo+ICsJaW50IHJldDsKPiArCj4gKwlzdGFydCA9IGNtZC0+YWRkcjsK PiArCWVuZCA9IHN0YXJ0ICsgc2l6ZTsKPiArCWlmIChlbmQgPCBzdGFydCkKPiArCQlyZXR1cm4g LUVJTlZBTDsKPiArCj4gKwlyZXQgPSBkbWlycm9yX2JvdW5jZV9pbml0KCZib3VuY2UsIHN0YXJ0 LCBzaXplKTsKPiArCWlmIChyZXQpCj4gKwkJcmV0dXJuIHJldDsKPiArCXJldCA9IGNvcHlfZnJv bV91c2VyKGJvdW5jZS5wdHIsICh2b2lkIF9fdXNlciAqKWNtZC0+cHRyLAo+ICsJCQkJYm91bmNl LnNpemUpOwoKZGl0dG8KCj4gKwlpZiAocmV0KQo+ICsJCXJldHVybiByZXQ7Cj4gKwo+ICthZ2Fp bjoKPiArCW11dGV4X2xvY2soJmRtaXJyb3ItPm11dGV4KTsKPiArCXJldCA9IGRtaXJyb3JfZG9f d3JpdGUoZG1pcnJvciwgc3RhcnQsIGVuZCwgJmJvdW5jZSk7Cj4gKwltdXRleF91bmxvY2soJmRt aXJyb3ItPm11dGV4KTsKPiArCWlmIChyZXQgPT0gLUVOT0VOVCkgewo+ICsJCXN0YXJ0ID0gY21k LT5hZGRyICsgKGJvdW5jZS5jcGFnZXMgPDwgUEFHRV9TSElGVCk7Cj4gKwkJcmV0ID0gZG1pcnJv cl9mYXVsdChkbWlycm9yLCBzdGFydCwgZW5kLCB0cnVlKTsKPiArCQlpZiAocmV0ID09IDApIHsK PiArCQkJY21kLT5mYXVsdHMrKzsKPiArCQkJZ290byBhZ2FpbjsKClVzZSBhIGxvb3AgaW5zdGVh ZCBvZiBnb3RvPwoKQWxzbyBJIGdldCB0aGlzOgoKbGliL3Rlc3RfaG1tLmM6IEluIGZ1bmN0aW9u IOKAmGRtaXJyb3JfZGV2bWVtX2ZhdWx0X2FsbG9jX2FuZF9jb3B54oCZOgpsaWIvdGVzdF9obW0u YzoxMDQxOjI1OiB3YXJuaW5nOiB1bnVzZWQgdmFyaWFibGUg4oCYdm1h4oCZIFstV3VudXNlZC12 YXJpYWJsZV0KIDEwNDEgfCAgc3RydWN0IHZtX2FyZWFfc3RydWN0ICp2bWEgPSBhcmdzLT52bWE7 CgpCdXQgdGhpcyBpcyBhIGtlcm5lbCBidWcsIGR1ZSB0byBhbGxvY19wYWdlX3ZtYSBiZWluZyBh ICNkZWZpbmUgbm90IGEKc3RhdGljIGlubGluZSBhbmQgbWUgaGF2aW5nIENPTkZJR19OVU1BIG9m ZiBpbiB0aGlzIC5jb25maWcKCkphc29uCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fCmFtZC1nZnggbWFpbGluZyBsaXN0CmFtZC1nZnhAbGlzdHMuZnJlZWRl c2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8v YW1kLWdmeAo= From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jason Gunthorpe Date: Thu, 19 Mar 2020 18:17:16 +0000 Subject: Re: [PATCH 3/4] mm: simplify device private page handling in hmm_range_fault Message-Id: <20200319181716.GK20941@ziepe.ca> List-Id: References: <20200316193216.920734-1-hch@lst.de> <20200316193216.920734-4-hch@lst.de> <7256f88d-809e-4aba-3c46-a223bd8cc521@nvidia.com> <20200317121536.GQ20941@ziepe.ca> <20200317122445.GA11662@lst.de> <20200317122813.GA11866@lst.de> <20200317124755.GR20941@ziepe.ca> <20200317125955.GA12847@lst.de> <24fca825-3b0f-188f-bcf2-fadcf3a9f05a@nvidia.com> In-Reply-To: <24fca825-3b0f-188f-bcf2-fadcf3a9f05a@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit To: Ralph Campbell Cc: Christoph Hellwig , Dan Williams , Bharata B Rao , Christian =?utf-8?B?S8O2bmln?= , Ben Skeggs , Jerome Glisse , kvm-ppc@vger.kernel.org, amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-mm@kvack.org On Tue, Mar 17, 2020 at 04:14:31PM -0700, Ralph Campbell wrote: > > On 3/17/20 5:59 AM, Christoph Hellwig wrote: > > On Tue, Mar 17, 2020 at 09:47:55AM -0300, Jason Gunthorpe wrote: > > > I've been using v7 of Ralph's tester and it is working well - it has > > > DEVICE_PRIVATE support so I think it can test this flow too. Ralph are > > > you able? > > > > > > This hunk seems trivial enough to me, can we include it now? > > > > I can send a separate patch for it once the tester covers it. I don't > > want to add it to the original patch as it is a significant behavior > > change compared to the existing code. > > > > Attached is an updated version of my HMM tests based on linux-5.6.0-rc6. > I ran this OK with Jason's 8+1 HMM patches, Christoph's 1-5 misc HMM clean ups, > and Christoph's 1-4 device private page changes applied. I'd like to get this to mergable, it looks pretty good now, but I have no idea about selftests - and I'm struggling to even compile the tools dir > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug > index 69def4a9df00..4d22ce7879a7 100644 > +++ b/lib/Kconfig.debug > @@ -2162,6 +2162,18 @@ config TEST_MEMINIT > > If unsure, say N. > > +config TEST_HMM > + tristate "Test HMM (Heterogeneous Memory Management)" > + depends on DEVICE_PRIVATE > + select HMM_MIRROR > + select MMU_NOTIFIER extra spaces In general I wonder if it even makes sense that DEVICE_PRIVATE is user selectable? > +static int dmirror_fops_open(struct inode *inode, struct file *filp) > +{ > + struct cdev *cdev = inode->i_cdev; > + struct dmirror *dmirror; > + int ret; > + > + /* Mirror this process address space */ > + dmirror = kzalloc(sizeof(*dmirror), GFP_KERNEL); > + if (dmirror = NULL) > + return -ENOMEM; > + > + dmirror->mdevice = container_of(cdev, struct dmirror_device, cdevice); > + mutex_init(&dmirror->mutex); > + xa_init(&dmirror->pt); > + > + ret = mmu_interval_notifier_insert(&dmirror->notifier, current->mm, > + 0, ULONG_MAX & PAGE_MASK, &dmirror_min_ops); > + if (ret) { > + kfree(dmirror); > + return ret; > + } > + > + /* Pairs with the mmdrop() in dmirror_fops_release(). */ > + mmgrab(current->mm); > + dmirror->mm = current->mm; The notifier holds a mmgrab, no need for another one > + /* Only the first open registers the address space. */ > + filp->private_data = dmirror; Not sure what this comment means > +static inline struct dmirror_device *dmirror_page_to_device(struct page *page) > + > +{ > + struct dmirror_chunk *devmem; > + > + devmem = container_of(page->pgmap, struct dmirror_chunk, pagemap); > + return devmem->mdevice; > +} extra devmem var is not really needed > + > +static bool dmirror_device_is_mine(struct dmirror_device *mdevice, > + struct page *page) > +{ > + if (!is_zone_device_page(page)) > + return false; > + return page->pgmap->ops = &dmirror_devmem_ops && > + dmirror_page_to_device(page) = mdevice; > +} Use new owner stuff, right? Actually this is redunant now, the check should be just WARN_ON pageowner != self owner > +static int dmirror_do_fault(struct dmirror *dmirror, struct hmm_range *range) > +{ > + uint64_t *pfns = range->pfns; > + unsigned long pfn; > + > + for (pfn = (range->start >> PAGE_SHIFT); > + pfn < (range->end >> PAGE_SHIFT); > + pfn++, pfns++) { > + struct page *page; > + void *entry; > + > + /* > + * HMM_PFN_ERROR is returned if it is accessing invalid memory > + * either because of memory error (hardware detected memory > + * corruption) or more likely because of truncate on mmap > + * file. > + */ > + if (*pfns = range->values[HMM_PFN_ERROR]) > + return -EFAULT; Unless that snapshot is use hmm_range_fault() never returns success and sets PFN_ERROR, so this should be a WARN_ON > + if (!(*pfns & range->flags[HMM_PFN_VALID])) > + return -EFAULT; Same with valid. > + page = hmm_device_entry_to_page(range, *pfns); > + /* We asked for pages to be populated but check anyway. */ > + if (!page) > + return -EFAULT; WARN_ON > + if (is_zone_device_page(page)) { > + /* > + * TODO: need a way to ask HMM to fault foreign zone > + * device private pages. > + */ > + if (!dmirror_device_is_mine(dmirror->mdevice, page)) > + continue; Actually re > +static bool dmirror_interval_invalidate(struct mmu_interval_notifier *mni, > + const struct mmu_notifier_range *range, > + unsigned long cur_seq) > +{ > + struct dmirror *dmirror = container_of(mni, struct dmirror, notifier); > + struct mm_struct *mm = dmirror->mm; > + > + /* > + * If the process doesn't exist, we don't need to invalidate the > + * device page table since the address space will be torn down. > + */ > + if (!mmget_not_zero(mm)) > + return true; Why? Don't the notifiers provide for this already. mmget_not_zero() is required before calling hmm_range_fault() though > +static int dmirror_fault(struct dmirror *dmirror, unsigned long start, > + unsigned long end, bool write) > +{ > + struct mm_struct *mm = dmirror->mm; > + unsigned long addr; > + uint64_t pfns[64]; > + struct hmm_range range = { > + .notifier = &dmirror->notifier, > + .pfns = pfns, > + .flags = dmirror_hmm_flags, > + .values = dmirror_hmm_values, > + .pfn_shift = DPT_SHIFT, > + .pfn_flags_mask = ~(dmirror_hmm_flags[HMM_PFN_VALID] | > + dmirror_hmm_flags[HMM_PFN_WRITE]), > + .default_flags = dmirror_hmm_flags[HMM_PFN_VALID] | > + (write ? dmirror_hmm_flags[HMM_PFN_WRITE] : 0), > + .dev_private_owner = dmirror->mdevice, > + }; > + int ret = 0; > + > + /* Since the mm is for the mirrored process, get a reference first. */ > + if (!mmget_not_zero(mm)) > + return 0; Right > + for (addr = start; addr < end; addr = range.end) { > + range.start = addr; > + range.end = min(addr + (ARRAY_SIZE(pfns) << PAGE_SHIFT), end); > + > + ret = dmirror_range_fault(dmirror, &range); > + if (ret) > + break; > + } > + > + mmput(mm); > + return ret; > +} > + > +static int dmirror_do_read(struct dmirror *dmirror, unsigned long start, > + unsigned long end, struct dmirror_bounce *bounce) > +{ > + unsigned long pfn; > + void *ptr; > + > + ptr = bounce->ptr + ((start - bounce->addr) & PAGE_MASK); > + > + for (pfn = start >> PAGE_SHIFT; pfn < (end >> PAGE_SHIFT); pfn++) { > + void *entry; > + struct page *page; > + void *tmp; > + > + entry = xa_load(&dmirror->pt, pfn); > + page = xa_untag_pointer(entry); > + if (!page) > + return -ENOENT; > + > + tmp = kmap(page); > + memcpy(ptr, tmp, PAGE_SIZE); > + kunmap(page); > + > + ptr += PAGE_SIZE; > + bounce->cpages++; > + } > + > + return 0; > +} > + > +static int dmirror_read(struct dmirror *dmirror, struct hmm_dmirror_cmd *cmd) > +{ > + struct dmirror_bounce bounce; > + unsigned long start, end; > + unsigned long size = cmd->npages << PAGE_SHIFT; > + int ret; > + > + start = cmd->addr; > + end = start + size; > + if (end < start) > + return -EINVAL; > + > + ret = dmirror_bounce_init(&bounce, start, size); > + if (ret) > + return ret; > + > +again: > + mutex_lock(&dmirror->mutex); > + ret = dmirror_do_read(dmirror, start, end, &bounce); > + mutex_unlock(&dmirror->mutex); > + if (ret = 0) > + ret = copy_to_user((void __user *)cmd->ptr, bounce.ptr, > + bounce.size); Use u64_to_user_ptr() instead of the cast > +static int dmirror_write(struct dmirror *dmirror, struct hmm_dmirror_cmd *cmd) > +{ > + struct dmirror_bounce bounce; > + unsigned long start, end; > + unsigned long size = cmd->npages << PAGE_SHIFT; > + int ret; > + > + start = cmd->addr; > + end = start + size; > + if (end < start) > + return -EINVAL; > + > + ret = dmirror_bounce_init(&bounce, start, size); > + if (ret) > + return ret; > + ret = copy_from_user(bounce.ptr, (void __user *)cmd->ptr, > + bounce.size); ditto > + if (ret) > + return ret; > + > +again: > + mutex_lock(&dmirror->mutex); > + ret = dmirror_do_write(dmirror, start, end, &bounce); > + mutex_unlock(&dmirror->mutex); > + if (ret = -ENOENT) { > + start = cmd->addr + (bounce.cpages << PAGE_SHIFT); > + ret = dmirror_fault(dmirror, start, end, true); > + if (ret = 0) { > + cmd->faults++; > + goto again; Use a loop instead of goto? Also I get this: lib/test_hmm.c: In function ‘dmirror_devmem_fault_alloc_and_copy’: lib/test_hmm.c:1041:25: warning: unused variable ‘vma’ [-Wunused-variable] 1041 | struct vm_area_struct *vma = args->vma; But this is a kernel bug, due to alloc_page_vma being a #define not a static inline and me having CONFIG_NUMA off in this .config Jason