From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751901AbdF0AHz (ORCPT ); Mon, 26 Jun 2017 20:07:55 -0400 Received: from hqemgate15.nvidia.com ([216.228.121.64]:13656 "EHLO hqemgate15.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751723AbdF0AHt (ORCPT ); Mon, 26 Jun 2017 20:07:49 -0400 X-PGP-Universal: processed; by hqpgpgate101.nvidia.com on Mon, 26 Jun 2017 17:07:43 -0700 From: Evgeny Baskakov To: =?utf-8?B?SsOpcsO0bWUgR2xpc3Nl?= , "akpm@linux-foundation.org" , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" CC: John Hubbard , David Nellans , Mark Hairgrove , Sherry Cheung , Subhash Gutti Subject: RE: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4 Thread-Topic: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4 Thread-Index: AQHS0xvO6foAlf732UavDZdwSfUD6KI4BKmQ Date: Tue, 27 Jun 2017 00:07:42 +0000 Message-ID: References: <20170522165206.6284-1-jglisse@redhat.com> <20170522165206.6284-13-jglisse@redhat.com> In-Reply-To: <20170522165206.6284-13-jglisse@redhat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [172.17.162.71] Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id v5R07xMQ023590 On Monday, May 22, 2017 9:52 AM, Jérôme Glisse wrote: [...] + * The alloc_and_copy() callback happens once all source pages have +been locked, + * unmapped and checked (checked whether pinned or not). All pages that +can be + * migrated will have an entry in the src array set with the pfn value +of the + * page and with the MIGRATE_PFN_VALID and MIGRATE_PFN_MIGRATE flag set +(other + * flags might be set but should be ignored by the callback). + * + * The alloc_and_copy() callback can then allocate destination memory +and copy + * source memory to it for all those entries (ie with MIGRATE_PFN_VALID +and + * MIGRATE_PFN_MIGRATE flag set). Once these are allocated and copied, +the + * callback must update each corresponding entry in the dst array with +the pfn + * value of the destination page and with the MIGRATE_PFN_VALID and + * MIGRATE_PFN_LOCKED flags set (destination pages must have their +struct pages + * locked, via lock_page()). + * + * At this point the alloc_and_copy() callback is done and returns. + * + * Note that the callback does not have to migrate all the pages that +are + * marked with MIGRATE_PFN_MIGRATE flag in src array unless this is a +migration + * from device memory to system memory (ie the MIGRATE_PFN_DEVICE flag +is also + * set in the src array entry). If the device driver cannot migrate a +device + * page back to system memory, then it must set the corresponding dst +array + * entry to MIGRATE_PFN_ERROR. This will trigger a SIGBUS if CPU tries +to + * access any of the virtual addresses originally backed by this page. +Because + * a SIGBUS is such a severe result for the userspace process, the +device + * driver should avoid setting MIGRATE_PFN_ERROR unless it is really in +an + * unrecoverable state. + * + * THE alloc_and_copy() CALLBACK MUST NOT CHANGE ANY OF THE SRC ARRAY +ENTRIES + * OR BAD THINGS WILL HAPPEN ! + * Hi Jerome, The documentation shown above doesn't tell what the alloc_and_copy callback should do for source pages that have not been allocated yet. Instead, it unconditionally suggests checking if the MIGRATE_PFN_VALID and MIGRATE_PFN_MIGRATE flags are set. Based on my testing and looking in the source code, I see that for such pages the respective 'src' PFN entries are always set to 0 without any flags. The sample driver specifically handles that by checking if there's no page in the 'src' entry, and ignores any flags in such case: struct page *spage = migrate_pfn_to_page(*src_pfns); ... if (spage && !(*src_pfns & MIGRATE_PFN_MIGRATE)) continue; if (spage && (*src_pfns & MIGRATE_PFN_DEVICE)) { I would like to suggest reflecting that in the documentation. Or, which would be more logical, migrate_vma could keep the zero in the PFN entries for not allocated pages, but set the MIGRATE_PFN_MIGRATE flag anyway. Thanks! Evgeny Baskakov NVIDIA