From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B954C4361A for ; Sat, 5 Dec 2020 08:23:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AEFE522D6F for ; Sat, 5 Dec 2020 08:23:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AEFE522D6F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DCB2C6B0068; Sat, 5 Dec 2020 03:23:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D7ABD6B006C; Sat, 5 Dec 2020 03:23:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BCEC36B006E; Sat, 5 Dec 2020 03:23:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0098.hostedemail.com [216.40.44.98]) by kanga.kvack.org (Postfix) with ESMTP id A6A4E6B0068 for ; Sat, 5 Dec 2020 03:23:23 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 68A5A181AC9CC for ; Sat, 5 Dec 2020 08:23:23 +0000 (UTC) X-FDA: 77558539086.17.scent99_4d0b4eb273cb Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id 4EB91180D0180 for ; Sat, 5 Dec 2020 08:23:23 +0000 (UTC) X-HE-Tag: scent99_4d0b4eb273cb X-Filterd-Recvd-Size: 9769 Received: from esa5.hc3370-68.iphmx.com (esa5.hc3370-68.iphmx.com [216.71.155.168]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Sat, 5 Dec 2020 08:23:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1607156602; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=m1Fg+2fiuBESw+2cc2tVoewK2JbkdoEe4/+zgiw93hs=; b=UYMaeIu2GwOlQDfoW18KZhoQ/eR3WhrtJzFMpD+S9fqRP/TioGpk8Lbx o9rllbtmupr2dAJoTfolAeHq0wpaDvCttqJRJyzZ2zUr0vzAodNmYG6hf 8VMSACs7iom09uoQaZtEjurZy2VPl2f9f1biQiM3qWjGBatxCwvuXURdc g=; Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com IronPort-SDR: Dt7wjvltwcxpdH0voN6FPSBB/ul/o795btBvdlYX3ybfbPax6ufeIk/mQ7qoStw6cudU6Q7u67 a44X+P3mTYbUQS9b6NUI9OFHr4KttlnkgLLqozv5bviFIc/qwWgQ0JaKZiNcgB9quXX9uwcJGb KpuI6cyND9twmSXJ9Jwjf3gfqgQ6iWAfxRpGQ3uCFDy3FsOUBYBst1x46W7zrvBGHEeLSKVMvA 5x+GQ2aT5B5KlhYPP3w7sN4r9fXFyeiBwbUeV40Bl8J2NB2ohmQet9aef8DFANkd7+GzTo0hHV ofk= X-SBRS: 5.1 X-MesageID: 32603291 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.78,395,1599537600"; d="scan'208";a="32603291" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OaDhOWlAoAKuLCJgk+FPJ8iJZ8P6ZvNzlKxzoPYWpg1tF0C4y1LiXwY5aN7mSVIMf7svsKwBG/FahQFyUajIP1Mi4hO3HGEX/QT7FBQyiL2S7alT8A8c25xyqDQVzNpv+zPmy+RUAuSASnzqsQs8Dqs5TnEnO4q3AWnwGazbIXlhhYc0FEH5b80oawulTjXAiCXUn2iGdqRypZ69EzgYDjyWsXHLfCF8vhQRJmPANk5em4hVb8H6yEK2fdnpZxm0Ln5RJWixWcF/pzNydPfAjbIrltq68N9YPQm//R/1aQ8ieaknUcrb8oxAKRtbJNFS/oOjVNeJB1ybkBvUcee0Nw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wNTPicD1c6TUeogIRTmzHUxRmZNEBeOptjCqlBr/pYk=; b=LIOe9MlT615fWBmZ6QprFKWXVHPwhA2jLIfO4d2xR1Pi+EI9t9GknhanC9jnlcL7r7U9wzYfklIYsfvGGeeGBhyeyulkATPia9FbuVVcAuP/j3ys3onolSF6Y+7xV0Z1csoe4GErlRPgxiA7S9WXnptc+m+YEjC148jE+gi6eCzN9Jtw1VctnasPM8parh2jQtIY3p7MoopwobglZ8q2USeEH4nUumwiiw3/5LlVWq6p8IfE6WAK2JHq/+e2390MVTO7FflP3Y838/DyR9u12DFVbYPU/4I47NrpKKczVXiFGjmYibajNjBsc45NiuS2RcPjKYBdy2gev4HT23/GBA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wNTPicD1c6TUeogIRTmzHUxRmZNEBeOptjCqlBr/pYk=; b=onvq/Q4WShQJ4Uv6Zw11Y7LwSza7w+qIck+qGtuPc42KBWyQt64zW6XQiS0JvCug5b7MayeJ0kl/LD/LBGDosU7AOK9LUhLfGvtJVHvSGrytdWTmpsmNLbDlbicdUWZ10y31swTMK8rYzAqjJyPrBlU+bEb+gZphgEVjsb3ApDU= Date: Sat, 5 Dec 2020 09:22:59 +0100 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Christoph Hellwig CC: Jason Gunthorpe , Ralph Campbell , , , , , "Jerome Glisse" , John Hubbard , "Alistair Popple" , Bharata B Rao , Zi Yan , "Kirill A . Shutemov" , Yang Shi , Ben Skeggs , "Shuah Khan" , Andrew Morton Subject: Re: [PATCH v3 3/6] mm: support THP migration to device private memory Message-ID: <20201205082259.l7fdwfxdvi5xn3kz@Air-de-Roger> References: <20201106005147.20113-1-rcampbell@nvidia.com> <20201106005147.20113-4-rcampbell@nvidia.com> <20201106080322.GE31341@lst.de> <20201109091415.GC28918@lst.de> <20201120200133.GH917484@nvidia.com> <20201202100854.GB7597@lst.de> Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20201202100854.GB7597@lst.de> X-ClientProxiedBy: MR2P264CA0130.FRAP264.PROD.OUTLOOK.COM (2603:10a6:500:30::22) To DS7PR03MB5608.namprd03.prod.outlook.com (2603:10b6:5:2c9::18) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 607bbbe9-8aab-45d6-d92b-08d898f703cf X-MS-TrafficTypeDiagnostic: DM5PR03MB2844: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sYE5NZ0+t1ti5Fl5gELn1Zv+gMXxefWNrvyoRnMrSLXxDSfsMO2ILBWU2OtfHH1VB7whrU06ipX8bNKHX5SAIa64zadA4kzjYXSLu6Pwi03YQ3LulU3zozSbJSEmSJeo1BzKN1h0VyG3HnkhNWXhpK2IvIXlg3CkCNpHzjTQlDc1sojzQCnjFMLx3V9QQdbui8mK50vcXfgX+LGIGf15/tREYNBrOnA0YB5xuQh4PjD87VZTH13/GBdS8oxYXdbxwgtphFw/GGBh31MsuNqmcHTNRIEMCOyx4OA+2Zhgg2sFnby9Z3G5npzpBR0UEfjgQVke5/z1lLdYNxfr4Zf3xA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(7916004)(376002)(136003)(366004)(396003)(39860400002)(346002)(66556008)(33716001)(54906003)(6916009)(5660300002)(7416002)(83380400001)(86362001)(4326008)(8676002)(6496006)(956004)(85182001)(8936002)(9686003)(26005)(6666004)(6486002)(316002)(186003)(2906002)(66946007)(66476007)(478600001)(1076003)(16526019);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?Q0xtd2MxYnEwVEpzV0RtY1JPaTJ1TWJ3SzcwWHEzNXJDdjg1UjRNcVlQY0Jq?= =?utf-8?B?WDRTZXZNem9SRWpSYmJwb0JFU1M5a3p3ZHlmVUxIMlZRQlc2b1RMbVowUkZ4?= =?utf-8?B?UTA4ZThTWjRKMjJ3MzQ3cklaTkE1aFNwcUxqQ0lQdkxBR1JlZVFYNFBZd2ls?= =?utf-8?B?L09udUdPZCtEa2NNZXVENk5YZ3JWRGF3Z1BzK1JrTS9DV0VnY1hNZWNrYlh4?= =?utf-8?B?NUtJRGtNWWlaekkrUEhES2FUaTRBVUx1T3JuMXArS3lBdm5ybDJhVndqWjhy?= =?utf-8?B?Wnk0cDBSVWhXaHZMSGtNWE9yWDNUUVA2WFd6NlpSSGJxeW04UVJJNzlVa0Jh?= =?utf-8?B?aW53K2xWMUJQZWxJT1ZVWHN2blRvZXMxSFJGSS91QXEwTDh0OTRqeS9QV3pv?= =?utf-8?B?ek82cS9ZLy9Xbi95TTRPVS9pY3MrbjFGUFd5UHNBY3o2R1R2LzZ4VkdjbEw1?= =?utf-8?B?dlVTZmhJSVB0NmJyV0RhNFJJdlJwWGZrbXMrSjEvcU9vNEw1eTdaZS83RjNV?= =?utf-8?B?cVR3T0RjcHRSbEtWZW9PVUV2NDJaVnAzZ0VkR3hQNTc4SGtKRWR6ZStSSGFy?= =?utf-8?B?VFFMVURqUHdQb2E3TXcrWlY4NHN0bTFYK3BNWk85YXVMT1JKSU9HaUkxbFRh?= =?utf-8?B?ZzJqTGtpWVd5VEVpak1VWURHdzFnRjlXL2l6VkdIeE1NZjIwb1FFQXp2TEFv?= =?utf-8?B?ZmFBWnA5OHh0em1uVDN2S3ZQODFrdUdLMjhldHdJQ21aaTlrODV0bjZhOEN3?= =?utf-8?B?MS9ZQjRERXNTOFZvd3g5QUlpSzZXWGpFZk1IcVZaeHZmdXhxaGtJZmdYQzVo?= =?utf-8?B?anFCNHVQME1XT3pHQktZbndQelRuK1kzWWlQY092R3lRZDRoOTJWR1V6TGJG?= =?utf-8?B?ckhLeUpqdHVnV3lJSU1OSGp5VDdyazJseHdWbzh3bTJCOGc3R2JjaGZoc0k4?= =?utf-8?B?TUkyY2tIOXVpN2pjTjV2T2dzSFozQm1GWGI5VkhsMUJIc0k5UkF3ZngvUmlo?= =?utf-8?B?ZTdvdUo1aWFxcWg5VjI1UHh5dkxWK3RUYjN1c2N3clVTSnhSM0ZkWWxTM3VO?= =?utf-8?B?NXIrVCtVNWpaTVNncDdaYldPbUNKaXpkNWFNUloxbVA4T3RXQ3JDUjdzWmdr?= =?utf-8?B?RWo5WDRRandqN3hOWGx5TFdPUkhoV1ZyY2xOYWlFOGErQ2tHT1NUVTkzVVAy?= =?utf-8?B?OXRUeWNDME1EQ0VWTzdjdHMxRzhiLzdxM202WUp6VXR1SDRBaHNmZlNYdFF2?= =?utf-8?B?Wk1vS1FCb0pCNnlXK3NJdVRIbTQvMXZJR1Z4bFlIazg4aHh4QVJzRE41djlP?= =?utf-8?Q?FkOr3sCT6Ot/zdtFqy6eI6d0ff7Gxa73xq?= X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2020 08:23:15.9709 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b X-MS-Exchange-CrossTenant-Network-Message-Id: 607bbbe9-8aab-45d6-d92b-08d898f703cf X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: iUsR5KYomrrROUr8gqThA8AIeBmymlaxjmdwGBTKBizGD791XWobuUoubln+tVUD+o5I4h+4iEpHBu+dxYs0+Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB2844 X-OriginatorOrg: citrix.com X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Dec 02, 2020 at 11:08:54AM +0100, Christoph Hellwig wrote: > On Fri, Nov 20, 2020 at 04:01:33PM -0400, Jason Gunthorpe wrote: > > On Wed, Nov 11, 2020 at 03:38:42PM -0800, Ralph Campbell wrote: > > > > > MEMORY_DEVICE_GENERIC: > > > Struct pages are created in dev_dax_probe() and represent non-volatile memory. > > > The device can be mmap()'ed which calls dax_mmap() which sets > > > vma->vm_flags | VM_HUGEPAGE. > > > A CPU page fault will result in a PTE, PMD, or PUD sized page > > > (but not compound) to be inserted by vmf_insert_mixed() which will call either > > > insert_pfn() or insert_page(). > > > Neither insert_pfn() nor insert_page() increments the page reference > > > count. > > > > But why was this done? It seems very strange to put a pfn with a > > struct page into a VMA and then deliberately not take the refcount for > > the duration of that pfn being in the VMA? > > > > What prevents memunmap_pages() from progressing while VMAs still point > > at the memory? > > Agreed. Adding Roger who added MEMORY_DEVICE_GENERIC and the only > user. MEMORY_DEVICE_GENERIC is just a rename of the previous MEMORY_DEVICE_DEVDAX, and seems to be used by the DAX device apart from Xen? It's main purpose is to be able to allocate unused physical memory ranges and have a baking struct page for them, so they can be used to map foreign memory when running on Xen. I'm currently on leave and won't be back until the end of the month, could you please Cc the Xen maintainers if you modify the logic here in order to make sure it will work for Xen? Thanks, Roger.