From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S642164AbdEAByQ (ORCPT ); Sun, 30 Apr 2017 21:54:16 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59072 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S638403AbdEAByH (ORCPT ); Sun, 30 Apr 2017 21:54:07 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com E4B6B46D096 Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=jglisse@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com E4B6B46D096 Date: Sun, 30 Apr 2017 21:54:03 -0400 From: Jerome Glisse To: Dan Williams Cc: "Kirill A. Shutemov" , Ingo Molnar , "linux-kernel@vger.kernel.org" , Linux MM , Ingo Molnar , Andrew Morton , Logan Gunthorpe , Kirill Shutemov Subject: Re: [PATCH v2] mm, zone_device: replace {get, put}_zone_device_page() with a single reference Message-ID: <20170501015403.GA16181@redhat.com> References: <1743017574.4309811.1493400875692.JavaMail.zimbra@redhat.com> <1579714997.4315035.1493402406629.JavaMail.zimbra@redhat.com> <1295710462.4327805.1493406971970.JavaMail.zimbra@redhat.com> <20170428193305.GA3912@redhat.com> <20170429101726.cdczojcjjupb7myy@node.shutemov.name> <20170430231421.GA15163@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.8.0 (2017-02-23) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Mon, 01 May 2017 01:54:07 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, Apr 30, 2017 at 06:42:02PM -0700, Dan Williams wrote: > On Sun, Apr 30, 2017 at 4:14 PM, Jerome Glisse wrote: > > On Sat, Apr 29, 2017 at 01:17:26PM +0300, Kirill A. Shutemov wrote: > >> On Fri, Apr 28, 2017 at 03:33:07PM -0400, Jerome Glisse wrote: > >> > On Fri, Apr 28, 2017 at 12:22:24PM -0700, Dan Williams wrote: > >> > > Are you sure about needing to hook the 2 -> 1 transition? Could we > >> > > change ZONE_DEVICE pages to not have an elevated reference count when > >> > > they are created so you can keep the HMM references out of the mm hot > >> > > path? > >> > > >> > 100% sure on that :) I need to callback into driver for 2->1 transition > >> > no way around that. If we change ZONE_DEVICE to not have an elevated > >> > reference count that you need to make a lot more change to mm so that > >> > ZONE_DEVICE is never use as fallback for memory allocation. Also need > >> > to make change to be sure that ZONE_DEVICE page never endup in one of > >> > the path that try to put them back on lru. There is a lot of place that > >> > would need to be updated and it would be highly intrusive and add a > >> > lot of special cases to other hot code path. > >> > >> Could you explain more on where the requirement comes from or point me to > >> where I can read about this. > >> > > > > HMM ZONE_DEVICE pages are use like other pages (anonymous or file back page) > > in _any_ vma. So i need to know when a page is freed ie either as result of > > unmap, exit or migration or anything that would free the memory. For zone > > device a page is free once its refcount reach 1 so i need to catch refcount > > transition from 2->1 > > > > This is the only way i can inform the device that the page is now free. See > > > > https://cgit.freedesktop.org/~glisse/linux/commit/?h=hmm-v21&id=52da8fe1a088b87b5321319add79e43b8372ed7d > > > > There is _no_ way around that. > > Ok, but I need to point out that this not a ZONE_DEVICE requirement. > This is an HMM-specific need. So, this extra reference counting should > be clearly delineated as part of the MEMORY_DEVICE_PRIVATE use case. And it already is delimited, i think you even gave your review by on the patch. > Can we hide the extra reference counting behind a static branch so > that the common case fast path doesn't get slower until a HMM device > shows up? Like i already did With likely()/unlikely() ? Or something else ? https://cgit.freedesktop.org/~glisse/linux/commit/?h=hmm-v21&id=e84778e9db0672e371eb6599dfcb812512118842 Cheers, Jérôme