From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934035AbcKWJcM (ORCPT ); Wed, 23 Nov 2016 04:32:12 -0500 Received: from mail-db5eur01on0057.outbound.protection.outlook.com ([104.47.2.57]:42566 "EHLO EUR01-DB5-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1755941AbcKWJbb (ORCPT ); Wed, 23 Nov 2016 04:31:31 -0500 Authentication-Results: spf=pass (sender IP is 193.47.165.134) smtp.mailfrom=mellanox.com; nvidia.com; dkim=none (message not signed) header.d=none;nvidia.com; dmarc=pass action=none header.from=mellanox.com; X-IncomingTopHeaderMarker: OriginalChecksum:;UpperCasedChecksum:;SizeAsReceived:1411;Count:17 Subject: Re: [HMM v13 00/18] HMM (Heterogeneous Memory Management) v13 To: =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , , , References: <1479493107-982-1-git-send-email-jglisse@redhat.com> CC: John Hubbard , Feras Daoud , Ilya Lesokhin , Liran Liss From: Haggai Eran Message-ID: <5ba45b16-8edf-d835-ac04-eca5f71212c9@mellanox.com> Date: Wed, 23 Nov 2016 11:16:04 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <1479493107-982-1-git-send-email-jglisse@redhat.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit X-Originating-IP: [10.0.52.254] X-IncomingHeaderCount: 17 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:193.47.165.134;IPV:NLI;CTRY:IL;EFV:NLI;SFV:NSPM;SFS:(10009020)(6009001)(7916002)(2980300002)(438002)(199003)(24454002)(377454003)(189002)(107886002)(4001350100001)(31686004)(64126003)(83506001)(2870700001)(7846002)(106466001)(7736002)(50466002)(77096005)(5001770100001)(626004)(305945005)(23676002)(2950100002)(65826007)(5660300001)(31696002)(2201001)(8676002)(86362001)(65956001)(229853002)(36756003)(47776003)(3846002)(4001430100002)(6116002)(4326007)(81166006)(81156014)(2906002)(50986999)(54356999)(189998001)(8936002)(76176999)(356003)(92566002)(33646002)(65806001)(38730400001)(3940600001)(2101003);DIR:OUT;SFP:1101;SCL:1;SRVR:AM2PR05MB1217;H:mtlcas13.mtl.com;FPR:;SPF:Pass;PTR:mail13.mellanox.com;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: 1;DB3FFO11FD008;1:FHyNkfXZr2I9bcby/GBWEQuKH2B13MCtbCZ/x4w0jNMOMwlkB7ITDMBaI6Zz2iKtqD/7v6W/Ce1xU4lkDh0nEogPh1sQw4wJZZ6b//sx0WefvKP0ntbdtXkd+AdcWDIW28PXzLIj1ME+6ok2AQOPWD4sIh8CNl/nXzmAl60MXOmCRRs3roHfPTmKb50YTZW5Yq8W3VIJ2q55bNln/rEH/00yy6SC7G2Wrn6xR1JZbmJh4MAU+GauzZETUH3kUua3MFcop7QklCu+GJASEMeOUQdOY8tEy6rpNjfwzec/vHU9lNDWllV3K/ANkCmatmRC39W9D8/208HNvti6jfjjI7SDrGrllcnJT9O/obY9e4AtX8UavwmMd8k8K7TbGb2ml2C9Muys+ORpJdRSVDSkR9N8DVvY43FQUHSPhxKICevnrXSaxJKbWr6KQDpjuO9yXH9sOf19TV9m1T9qMGXG33kIlyZHc/27jrc+xQQFotBoG7xqwLnI87R1Uy7wUPxNVIguanmqTImFGXrLf3gI4rK9e5WWxLY/ZQsZbX4mOCQ6z+OMcaecDBNwLi9xGCim X-Microsoft-Exchange-Diagnostics: 1;AM2PR05MB1217;2:6vxnhvWhIgtEyPsP/oNZqGBIuoTLkeRnfMM0dj7UmDytpXhgC3/blCUFj7LEi/IU/BLCSfelCFokMWOiVrYmmO/YhxtK3ptb484r0rJyy/gaOijSVCq2gq1uuzr/UIBCKXC6rsirazz9nR/J1K+MeoIRtcydtLQmyhRvLH1YiTs=;3:siDY1GUY+0XZHEUJnURFJcc8SpEhUKdNm0q8f4w8wHLtnyDNl3rUT/r8v1jgzJBkBgs66MUA9qlOzS/Uwft/pP/ZH33OE+0o+hCZZHhEiBb7IJOJ0uOUA0DRnvR+oPgCe3JtxDe71RaR+EsqOhMCFpg7iU370zUKDd4+254NM7+yxsw49UO1ULSfqbZ0Uv+cU/wF8Gz7DxuKVz/5yGN6epZlLNvUk9dwFyysMRMgknjKtU2bkSHGrOMngqfThkvlxK5BavDmbGVAl0zp1coDAYyOGQGIZPyBlSra7iAkpKE= X-MS-Office365-Filtering-Correlation-Id: 8c3157a0-e310-4d18-bfeb-08d413816b26 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(8251501002);SRVR:AM2PR05MB1217; X-Microsoft-Exchange-Diagnostics: 1;AM2PR05MB1217;25:VQQg/56WEbLwwiVpQYiCNIa7MEDmAKF5le0bPNY1SgfC7jT3TcRElpNERZpgAo4nGfia4qC1bSJeaHR3LwGLcOAP0izpsslYxt9y3so9SOvSyeHB0fcuPq6TzkyVyS5LkBAlraD53oDIlvCqfbRFufqo+qWczFw+auYTL7GjIY6D8qMYm+8l8FB7aUYiGPwvQALoRVFi6Ehw9iijPChZxlVnoYSGztw2SXfzlX6d2/Bw51y0IRiydqquWBq99KwgLjkinxgcIfIRlAep6qQJ09kYytFJtMm9aoJycxeD6ppX9xefj+OHyToburuODifqAxVJ9c9Su341o0sP/Etl6sABXHVMCGs+k5khIpQSAdjOEO0lRYOLeI+s4oh3jPXS4nB4bNcxxZlKP2PZAjqDBOHz3du9HHBLzhWtPzi1JltwPQDl9sx0EXEK16sSKsMaLWG5hVXhWPJp4HBQblEzWQ==;31:mWoYp/noG6pTE5ECbyRZkXP5CfQ+zaDESmxhgrvHRNbICp3OCjSq8XD6ibAs8hAb4IlV39aoJL3tFgNt3NHRFA1z9p5ee14QwD/8SE3fon2kBxrpuLPjJveWOZovTbvsy/X/3w1adXRZR6OuNa/nkMfue2qCz2XVhQ2wMgqIjnXS9zuLNHKg/N4ZEKis18qWUYyflpJJB2MAS38KC6zzlRSZZJck0f5PAQqN2Fp+zgf/dcseJzeW68SKtYjksRaQEgQKOAz+ILXfd0Ryg1UItQ== X-Microsoft-Exchange-Diagnostics: 1;AM2PR05MB1217;20:tr1NMoHD7Lsg+RXp3w7n0S2SxD6kG05o3ga1KPK1Aoz6dKVriKtSuEYVYfz0jbPBkTMC3DUP9MQz2CWZMyshcn9n+hoe+IaDKcQdqoLDvAqbXx7cWZZeXTygrSvhn4G/N7OSj7KbPcFnBZHa3REo8iLl4IgbT5mkfNPZ2SCeS5RkSpB3tFaapdyj+Ap8xuIMCExKkxjqI4PEjrm8aQ2awbH6/CQUZaolJpkPG5c5MBvoOGQR0BrCC3dUf4KB/qQqiV4e2jRjBNVJ3qUl+Kn1UDRSm7FwJiccCKcb44XoCmdAnuOAUpTF3+1ARgaxhZVPPcznfTeiLVKYppOv0tKpKXgYR3B4FWpyWr5XcN/dspnLC28xxItUty1rOmzWcoQtbI9CfHaTc80JnQ6KrS/3VbhhSFsocVg/OqJv8K/N99UAh3WcqGFBIuOOA1ictF/wclSWgU+fiZ885cxq7rVBreWbR0ZaJbFwAKZdeSt8bAF6lFVj8nMFuq75ExLb99CS X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(17755550239193); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6045199)(6040307)(6060326)(601004)(2401047)(13018025)(13016025)(8121501046)(5005006)(3002001)(10201501046)(6055026)(6041248)(6061324);SRVR:AM2PR05MB1217;BCL:0;PCL:0;RULEID:;SRVR:AM2PR05MB1217; X-Microsoft-Exchange-Diagnostics: 1;AM2PR05MB1217;4:gpYXX87P9cSYQqdcTyPl9qtur2Kru4HiQ1OdoozY+sX0wMtgigm0ZJHej1mW0lpJK7/T8jP9gALVPzzbP1Wu07DFCZrIkf+Z0KfZnkq2AZqQ0CDC4Qn6DxlSt6iNwLG6rPtyY4sS0RiXH/+7mZV/bnucSc/bvRzZ/AbzH+ijHXY+v9zs/Hfq4IYC+FyqAVhY4GANAjLQeiDy5HYQnRRhh1qBIKd95COfNsIB+9ax3HNom7JcVC2B1agj8zms1M4m3XFNeFWz5xfj9qK2QtSAXmHn50OMOfuaOPe2kyOSM141nowXSxiLzH+E+Hhj7wFvuaMRY5wZP+Pgn+iq3XrGMThOUzBk5YStA0crCs54+Pb7wsCkqtUoTArXguTf5QF5KZmVPdd6JELSpK6JXrQEgKwj36J4G6n2ntyyUgb7TNRWXbTvUKlYHMzpR2k8XcOVDlVNFeeqsC1pzFGN/KnLd8BoCbzU8v7osI2mupeZlVqnhZR0pgNxlC1raufHb8qDTNtiYwI+1L3GuB4FskFBq4RIWNqQCm8X6sfqxcwmxNFKOKBijbTEAS0UWZ1FP1jY3mOF4aC5HZSbAaky4M0TUg== X-Forefront-PRVS: 013568035E X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtBTTJQUjA1TUIxMjE3OzIzOlAzMDFhcXphUGJOSVorRlBGWnNWcHFqSDIx?= =?utf-8?B?UFZCRTVGazUrVTQ1V08zZlkrOGxadGRxYkZmZGtVQ3gvalZ6Tjd4WElkdmNX?= =?utf-8?B?NlJKQXVSM0ZIckNUK1ZBL2dKd01zZkJTdFJ1UytrRnU4dnpmZDFsOFdVelJN?= =?utf-8?B?c1M0Z3dpWWNDWTVtdVhOOE9ieSsvdW9qQ3J2QjR2Y1ZtMk9kQ2FQbHkwY2x4?= =?utf-8?B?UW50eXFoalB5OE1QdWtTWDI2L2M5eUJoOXE0UzFsTFNlR1I2THphUlJXRmtz?= =?utf-8?B?eVZTS01PeDZXN21JMVFWUkRCbTFMQ3o0bzF0ZnZDa0FMZmhOWnpjUFpxYzJM?= =?utf-8?B?eXc1MmI1N1BTZUlOL1BRVHVDZ0lVdFJCN2h1MFdwaTlSaDhNeEl6ckFoVnRZ?= =?utf-8?B?SzBuZHdUekk4SGprQzV4bERlajlUci9vQTB2MHJZMFBWdk4xMTZJVHA5QTRS?= =?utf-8?B?b2pHUjRlUVB0bjYzamwrVml3Q2YvejV2eFBOQ1BQL2trdkVSQnpQSFhFclJB?= =?utf-8?B?OXp3dU5Wa0w1TUxkQ242T1FrNFFaWDY4L0RZSENxV3JZdC9OckpqS0ZMYzJK?= =?utf-8?B?emY0M1NZRm9EYnpOZ245N2h1MVB4blUzQi9VOGtKRjBKVHp0MkV2OFZLczAy?= =?utf-8?B?aWtDTkwyN1RZOVFlUmQ0Y05tODdrNzI3QWMxRWNkTk4zcGFNVHc3b1lldTFy?= =?utf-8?B?NlFDTUtSR2k0bko2eWtPTG1uQm1sYndHcWEvZ0Vuc2ROREV6WUNwYnFHTEZy?= =?utf-8?B?aVRudEVwUTVRYlRsemNSaWlpMUhhYnplajlRbXZJODVmMS9EM05uWVg2YVdn?= =?utf-8?B?emJRbXMvRFhLTzNhS0FRVXlVcWVUQ1NqQlV3emNOU3NRYjZST2lQbTl4Q3Bz?= =?utf-8?B?bHpsTEhNb1hQQk1vTFZvaW1UWWdzZ2ZBdUZmNXMwcXpXelRQY0hQVFJLY2pq?= =?utf-8?B?YTQ1NHVRYW14Q1AwTkNSejJoUTNEMjE2ZHN3YnhwTWxmMFhWYnFjV3A0amRl?= =?utf-8?B?bElKQUpxdEF3Z1R0VVNJbjZLaTBGQWpWa2dObVRmVGdtQ0ZmTjJuYUthU1Ay?= =?utf-8?B?OThPOEx1dTNtTVR2QU1xNnc2NjQrN1dFUmJYakJteVhoVTZrd3A1WUYxMkdt?= =?utf-8?B?T2gyK1JTMFM3NlQ2SmVIdTE0Y2lXNnNySDA5YVJNOEZoQVY3ZkxWcHlsRUR4?= =?utf-8?B?NEpLeEZwV1FKMW43ZEp5RVF0WExMWTdUM0xzekFHOGRUN2ZRWXNJTWYxNmly?= =?utf-8?B?Unc0QjAwNVVZS0V5TUt3eklXSnlpbWtHRk95MFppdGhPU0NFY0I0WGhlaWxM?= =?utf-8?B?am5ZSkRBRm94RXhzZjJvbHgrSEdud2hObTNQbmRIQzhnRG9DNWkwWm1jSkJn?= =?utf-8?B?QTRKb0dZaHdrY1VoVmgrV0tXYWZnZ09Vb2RIaW1VUVBmeXNjR0xxZWFLa2FF?= =?utf-8?B?bXRwTjc0TWpCcEExZkY1bGdNMHZveDRaV3M5eTB1emNHV1IwS056bmhESk1T?= =?utf-8?B?eXdzcDA3N2NMcWQ0VDNydGxiZzhyRWJ6S3ZROEI5S0ZxODl1dWhHbkUvVHJJ?= =?utf-8?B?cmJ4aFZiVk1PMVNFQkZjL3BBL1lVS0l4aS8rd1BXQzVtV1pCbVNTbnY2L25O?= =?utf-8?B?QkRvenVpM0ZxWmJIWUZXWnJBM2J4clNwY29Cakx3K29XQm4rQWF2OGl4dVls?= =?utf-8?B?NlZkSWpPdVJ5b2t1UkI5d2tJQzNYVWROck8vLzQ5UWpkUFNkVlB0TTNYcUk1?= =?utf-8?Q?EGRspe1+WVlUslpUtbkMIAUzO7jIIh8O9ONfs=3D?= X-Microsoft-Exchange-Diagnostics: 1;AM2PR05MB1217;6:qb105rQs1/OR710XiF+XFstrmWXmtGo/i7bU2cV3xfTcd+k/SJWcJGj3JnneA8E3EmsWMJpP3A/daGTdeq0GMmyUAK+hfTgA8+FeDJ+DAlSHCKn1iyLdqu0UsUztgqRL8dpdjzBxTDbGs5G08dUZ7KDkmyho0AFWFTBQTNeS3R/4ahrFRvSczyDjVZZMjSVB2dexJZbnI7QPZPi4fP5iFjz0dumz2tKYVaWEZbW/UGe/IKcWtk42BhJdgI+HWH6RWkwGNmVwdEGVmEn7odr9fC1w8m842B15ukAjXAgz1LOAHqGm/2WdgQ3AQfGUV/zV5KoFaHhj60YBU9hrhgadQOOuiV/4jHJ4hjPJmyUyWhoaP1E5a9KAybUkzRmEfDaS;5:aSB7vyC6Ax9SztLvJFK1KqBPzCFL3+KUvjA1P6jHwXISCT0jE6sQ/+Ke6RzRSXKqFdpzNk2IUKYlMTq/bdVBBXhy73mk8Aw9Xxwtu9GY6LEA3lONn4J7w+xwj+WlNpejTev3Aq9XbPK/RBbTSqKDyw==;24:STJVLo/DYbeJcekiSV6ccr8QNbwUmWv20QtKhZDRNhLW11dwnkgVM/EKWUJUy2cnvNl90pFsp5LkWcm9VtlfalAdH+2gzC3VScby1q917b4= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;AM2PR05MB1217;7:ZhbGLoZg5rKW1sohxXjMeTzF/wlNApO7bZk94Iz9PHePcu1Lub8ev6d5+kJgu/U650rga+l4EDRHefrgcCcYo91MfcVlRU3UN6at7ZxmuqjJ5TE0S+F/s3AHP5txzgtUpB+Fhu4+JSzhXEt0N/cn9Y7RAqRZ+xwh2JnrR4UNIvZtmUbvh+/1TyNw7DOdzzKL2I9x0JOgivs9KbW95kgDhWXLtWXpjguJuUOQRLO8Ueoh5Tqmoz4v9xbmnByLs1Qx5CrgSQxDcWRaGHe+Qto8h8SPqIc3q0oqUXUaDRxATtPGjuhYRurB12y/PbT1489Fj/THjsSN0qa8arbn9Eqs2oAe/t/7gkv4dhalNY8UmEg= X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Nov 2016 09:16:33.3309 (UTC) X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a652971c-7d2e-4d9b-a6a4-d149256f461b;Ip=[193.47.165.134];Helo=[mtlcas13.mtl.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM2PR05MB1217 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 11/18/2016 8:18 PM, Jérôme Glisse wrote: > Cliff note: HMM offers 2 things (each standing on its own). First > it allows to use device memory transparently inside any process > without any modifications to process program code. Second it allows > to mirror process address space on a device. > > Change since v12 is the use of struct page for device memory even if > the device memory is not accessible by the CPU (because of limitation > impose by the bus between the CPU and the device). > > Using struct page means that their are minimal changes to core mm > code. HMM build on top of ZONE_DEVICE to provide struct page, it > adds new features to ZONE_DEVICE. The first 7 patches implement > those changes. > > Rest of patchset is divided into 3 features that can each be use > independently from one another. First is the process address space > mirroring (patch 9 to 13), this allow to snapshot CPU page table > and to keep the device page table synchronize with the CPU one. > > Second is a new memory migration helper which allow migration of > a range of virtual address of a process. This memory migration > also allow device to use their own DMA engine to perform the copy > between the source memory and destination memory. This can be > usefull even outside HMM context in many usecase. > > Third part of the patchset (patch 17-18) is a set of helper to > register a ZONE_DEVICE node and manage it. It is meant as a > convenient helper so that device drivers do not each have to > reimplement over and over the same boiler plate code. > > > I am hoping that this can now be consider for inclusion upstream. > Bottom line is that without HMM we can not support some of the new > hardware features on x86 PCIE. I do believe we need some solution > to support those features or we won't be able to use such hardware > in standard like C++17, OpenCL 3.0 and others. > > I have been working with NVidia to bring up this feature on their > Pascal GPU. There are real hardware that you can buy today that > could benefit from HMM. We also intend to leverage this inside the > open source nouveau driver. Hi, I think the way this new version of the patchset uses ZONE_DEVICE looks promising and makes the patchset a little simpler than the previous versions. The mirroring code seems like it could be used to simplify the on-demand paging code in the mlx5 driver and the RDMA subsystem. It currently uses mmu notifiers directly. I'm also curious whether it can be used to allow peer to peer access between devices. For instance, if one device calls hmm_vma_get_pfns on a process that has unaddressable memory mapped in, with some additional help from DMA-API, its driver can convert these pfns to bus addresses directed to another device's MMIO region and thus enable peer to peer access. Then by handling invalidations through HMM's mirroring callbacks it can safely handle cases where the peer migrates the page back to the CPU or frees it. Haggai From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-f197.google.com (mail-pf0-f197.google.com [209.85.192.197]) by kanga.kvack.org (Postfix) with ESMTP id 645946B026B for ; Wed, 23 Nov 2016 04:16:38 -0500 (EST) Received: by mail-pf0-f197.google.com with SMTP id 83so9471046pfx.1 for ; Wed, 23 Nov 2016 01:16:38 -0800 (PST) Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10048.outbound.protection.outlook.com. [40.107.1.48]) by mx.google.com with ESMTPS id 19si32795431pfr.164.2016.11.23.01.16.36 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 23 Nov 2016 01:16:37 -0800 (PST) Subject: Re: [HMM v13 00/18] HMM (Heterogeneous Memory Management) v13 References: <1479493107-982-1-git-send-email-jglisse@redhat.com> From: Haggai Eran Message-ID: <5ba45b16-8edf-d835-ac04-eca5f71212c9@mellanox.com> Date: Wed, 23 Nov 2016 11:16:04 +0200 MIME-Version: 1.0 In-Reply-To: <1479493107-982-1-git-send-email-jglisse@redhat.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Sender: owner-linux-mm@kvack.org List-ID: To: =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: John Hubbard , Feras Daoud , Ilya Lesokhin , Liran Liss On 11/18/2016 8:18 PM, JA(C)rA'me Glisse wrote: > Cliff note: HMM offers 2 things (each standing on its own). First > it allows to use device memory transparently inside any process > without any modifications to process program code. Second it allows > to mirror process address space on a device. > > Change since v12 is the use of struct page for device memory even if > the device memory is not accessible by the CPU (because of limitation > impose by the bus between the CPU and the device). > > Using struct page means that their are minimal changes to core mm > code. HMM build on top of ZONE_DEVICE to provide struct page, it > adds new features to ZONE_DEVICE. The first 7 patches implement > those changes. > > Rest of patchset is divided into 3 features that can each be use > independently from one another. First is the process address space > mirroring (patch 9 to 13), this allow to snapshot CPU page table > and to keep the device page table synchronize with the CPU one. > > Second is a new memory migration helper which allow migration of > a range of virtual address of a process. This memory migration > also allow device to use their own DMA engine to perform the copy > between the source memory and destination memory. This can be > usefull even outside HMM context in many usecase. > > Third part of the patchset (patch 17-18) is a set of helper to > register a ZONE_DEVICE node and manage it. It is meant as a > convenient helper so that device drivers do not each have to > reimplement over and over the same boiler plate code. > > > I am hoping that this can now be consider for inclusion upstream. > Bottom line is that without HMM we can not support some of the new > hardware features on x86 PCIE. I do believe we need some solution > to support those features or we won't be able to use such hardware > in standard like C++17, OpenCL 3.0 and others. > > I have been working with NVidia to bring up this feature on their > Pascal GPU. There are real hardware that you can buy today that > could benefit from HMM. We also intend to leverage this inside the > open source nouveau driver. Hi, I think the way this new version of the patchset uses ZONE_DEVICE looks promising and makes the patchset a little simpler than the previous versions. The mirroring code seems like it could be used to simplify the on-demand paging code in the mlx5 driver and the RDMA subsystem. It currently uses mmu notifiers directly. I'm also curious whether it can be used to allow peer to peer access between devices. For instance, if one device calls hmm_vma_get_pfns on a process that has unaddressable memory mapped in, with some additional help from DMA-API, its driver can convert these pfns to bus addresses directed to another device's MMIO region and thus enable peer to peer access. Then by handling invalidations through HMM's mirroring callbacks it can safely handle cases where the peer migrates the page back to the CPU or frees it. Haggai -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org