From: Anshuman Khandual <khandual@linux.vnet.ibm.com> To: Jerome Glisse <jglisse@redhat.com>, Balbir Singh <bsingharora@gmail.com> Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, John Hubbard <jhubbard@nvidia.com>, Russell King <linux@armlinux.org.uk>, Benjamin Herrenschmidt <benh@kernel.crashing.org>, Paul Mackerras <paulus@samba.org>, Michael Ellerman <mpe@ellerman.id.au>, Martin Schwidefsky <schwidefsky@de.ibm.com>, Heiko Carstens <heiko.carstens@de.ibm.com>, Yoshinori Sato <ysato@users.sourceforge.jp>, Rich Felker <dalias@libc.org>, Chris Metcalf <cmetcalf@mellanox.com>, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com> Subject: Re: [HMM v13 01/18] mm/memory/hotplug: convert device parameter bool to set of flags Date: Mon, 21 Nov 2016 12:27:15 +0530 [thread overview] Message-ID: <58329ACB.6030700@linux.vnet.ibm.com> (raw) In-Reply-To: <20161121045352.GA7872@redhat.com> On 11/21/2016 10:23 AM, Jerome Glisse wrote: > On Mon, Nov 21, 2016 at 11:44:36AM +1100, Balbir Singh wrote: >> >> >> On 19/11/16 05:18, Jérôme Glisse wrote: >>> Only usefull for arch where we support ZONE_DEVICE and where we want to >>> also support un-addressable device memory. We need struct page for such >>> un-addressable memory. But we should avoid populating the kernel linear >>> mapping for the physical address range because there is no real memory >>> or anything behind those physical address. >>> >>> Hence we need more flags than just knowing if it is device memory or not. >>> >> >> >> Isn't it better to add a wrapper to arch_add/remove_memory and do those >> checks inside and then call arch_add/remove_memory to reduce the churn. >> If you need selectively enable MEMORY_UNADDRESSABLE that can be done with >> _ARCH_HAS_FEATURE > > The flag parameter can be use by other new features and thus i thought the > churn was fine. But i do not mind either way, whatever people like best. Right, once we get the device memory classification right, these flags can be used in more places. > > [...] > >>> -extern int arch_add_memory(int nid, u64 start, u64 size, bool for_device); >>> + >>> +/* >>> + * For device memory we want more informations than just knowing it is device >> information >>> + * memory. We want to know if we can migrate it (ie it is not storage memory >>> + * use by DAX). Is it addressable by the CPU ? Some device memory like GPU >>> + * memory can not be access by CPU but we still want struct page so that we >> accessed >>> + * can use it like regular memory. >> >> Can you please add some details on why -- migration needs them for example? > > I am not sure what you mean ? DAX ie persistent memory device is intended to be > use for filesystem or persistent storage. Hence memory migration does not apply > to it (it would go against its purpose). Why ? It can still be used for compaction, HW errors etc where we need to move between persistent storage areas. The source and destination can be persistent storage memory. > > So i want to extend ZONE_DEVICE to be more then just DAX/persistent memory. For > that i need to differentatiate between device memory that can be migrated and > should be more or less treated like regular memory (with struct page). This is > what the MEMORY_MOVABLE flag is for. ZONE_DEVICE right now also supports struct page for the addressable memory, (whether inside it's own range or in system RAM) with this we are extending it to cover un-addressable memory with struct pages. Yes the differentiation is required. > > Finaly in my case the device memory is not accessible by the CPU so i need yet > another flag. In the end i am extending ZONE_DEVICE to be use for 3 differents > type of memory. > > Is this the kind of explanation you are looking for ?
WARNING: multiple messages have this Message-ID (diff)
From: Anshuman Khandual <khandual@linux.vnet.ibm.com> To: Jerome Glisse <jglisse@redhat.com>, Balbir Singh <bsingharora@gmail.com> Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, John Hubbard <jhubbard@nvidia.com>, Russell King <linux@armlinux.org.uk>, Benjamin Herrenschmidt <benh@kernel.crashing.org>, Paul Mackerras <paulus@samba.org>, Michael Ellerman <mpe@ellerman.id.au>, Martin Schwidefsky <schwidefsky@de.ibm.com>, Heiko Carstens <heiko.carstens@de.ibm.com>, Yoshinori Sato <ysato@users.sourceforge.jp>, Rich Felker <dalias@libc.org>, Chris Metcalf <cmetcalf@mellanox.com>, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com> Subject: Re: [HMM v13 01/18] mm/memory/hotplug: convert device parameter bool to set of flags Date: Mon, 21 Nov 2016 12:27:15 +0530 [thread overview] Message-ID: <58329ACB.6030700@linux.vnet.ibm.com> (raw) In-Reply-To: <20161121045352.GA7872@redhat.com> On 11/21/2016 10:23 AM, Jerome Glisse wrote: > On Mon, Nov 21, 2016 at 11:44:36AM +1100, Balbir Singh wrote: >> >> >> On 19/11/16 05:18, Jerome Glisse wrote: >>> Only usefull for arch where we support ZONE_DEVICE and where we want to >>> also support un-addressable device memory. We need struct page for such >>> un-addressable memory. But we should avoid populating the kernel linear >>> mapping for the physical address range because there is no real memory >>> or anything behind those physical address. >>> >>> Hence we need more flags than just knowing if it is device memory or not. >>> >> >> >> Isn't it better to add a wrapper to arch_add/remove_memory and do those >> checks inside and then call arch_add/remove_memory to reduce the churn. >> If you need selectively enable MEMORY_UNADDRESSABLE that can be done with >> _ARCH_HAS_FEATURE > > The flag parameter can be use by other new features and thus i thought the > churn was fine. But i do not mind either way, whatever people like best. Right, once we get the device memory classification right, these flags can be used in more places. > > [...] > >>> -extern int arch_add_memory(int nid, u64 start, u64 size, bool for_device); >>> + >>> +/* >>> + * For device memory we want more informations than just knowing it is device >> information >>> + * memory. We want to know if we can migrate it (ie it is not storage memory >>> + * use by DAX). Is it addressable by the CPU ? Some device memory like GPU >>> + * memory can not be access by CPU but we still want struct page so that we >> accessed >>> + * can use it like regular memory. >> >> Can you please add some details on why -- migration needs them for example? > > I am not sure what you mean ? DAX ie persistent memory device is intended to be > use for filesystem or persistent storage. Hence memory migration does not apply > to it (it would go against its purpose). Why ? It can still be used for compaction, HW errors etc where we need to move between persistent storage areas. The source and destination can be persistent storage memory. > > So i want to extend ZONE_DEVICE to be more then just DAX/persistent memory. For > that i need to differentatiate between device memory that can be migrated and > should be more or less treated like regular memory (with struct page). This is > what the MEMORY_MOVABLE flag is for. ZONE_DEVICE right now also supports struct page for the addressable memory, (whether inside it's own range or in system RAM) with this we are extending it to cover un-addressable memory with struct pages. Yes the differentiation is required. > > Finaly in my case the device memory is not accessible by the CPU so i need yet > another flag. In the end i am extending ZONE_DEVICE to be use for 3 differents > type of memory. > > Is this the kind of explanation you are looking for ? -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2016-11-21 6:57 UTC|newest] Thread overview: 146+ messages / expand[flat|nested] mbox.gz Atom feed top 2016-11-18 18:18 [HMM v13 00/18] HMM (Heterogeneous Memory Management) v13 Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 18:18 ` [HMM v13 01/18] mm/memory/hotplug: convert device parameter bool to set of flags Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 0:44 ` Balbir Singh 2016-11-21 0:44 ` Balbir Singh 2016-11-21 4:53 ` Jerome Glisse 2016-11-21 4:53 ` Jerome Glisse 2016-11-21 6:57 ` Anshuman Khandual [this message] 2016-11-21 6:57 ` Anshuman Khandual 2016-11-21 12:19 ` Jerome Glisse 2016-11-21 12:19 ` Jerome Glisse 2016-11-21 6:41 ` Anshuman Khandual 2016-11-21 6:41 ` Anshuman Khandual 2016-11-21 12:27 ` Jerome Glisse 2016-11-21 12:27 ` Jerome Glisse 2016-11-22 5:35 ` Anshuman Khandual 2016-11-22 5:35 ` Anshuman Khandual 2016-11-22 14:08 ` Jerome Glisse 2016-11-22 14:08 ` Jerome Glisse 2016-11-18 18:18 ` [HMM v13 02/18] mm/ZONE_DEVICE/unaddressable: add support for un-addressable device memory Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 8:06 ` Anshuman Khandual 2016-11-21 8:06 ` Anshuman Khandual 2016-11-21 12:33 ` Jerome Glisse 2016-11-21 12:33 ` Jerome Glisse 2016-11-22 5:15 ` Anshuman Khandual 2016-11-22 5:15 ` Anshuman Khandual 2016-11-18 18:18 ` [HMM v13 03/18] mm/ZONE_DEVICE/free_hot_cold_page: catch ZONE_DEVICE pages Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 8:18 ` Anshuman Khandual 2016-11-21 8:18 ` Anshuman Khandual 2016-11-21 12:50 ` Jerome Glisse 2016-11-21 12:50 ` Jerome Glisse 2016-11-22 4:30 ` Anshuman Khandual 2016-11-22 4:30 ` Anshuman Khandual 2016-11-18 18:18 ` [HMM v13 04/18] mm/ZONE_DEVICE/free-page: callback when page is freed Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 1:49 ` Balbir Singh 2016-11-21 1:49 ` Balbir Singh 2016-11-21 4:57 ` Jerome Glisse 2016-11-21 4:57 ` Jerome Glisse 2016-11-21 8:26 ` Anshuman Khandual 2016-11-21 8:26 ` Anshuman Khandual 2016-11-21 12:34 ` Jerome Glisse 2016-11-21 12:34 ` Jerome Glisse 2016-11-22 5:02 ` Anshuman Khandual 2016-11-22 5:02 ` Anshuman Khandual 2016-11-18 18:18 ` [HMM v13 05/18] mm/ZONE_DEVICE/devmem_pages_remove: allow early removal of device memory Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 10:37 ` Anshuman Khandual 2016-11-21 10:37 ` Anshuman Khandual 2016-11-21 12:39 ` Jerome Glisse 2016-11-21 12:39 ` Jerome Glisse 2016-11-22 4:54 ` Anshuman Khandual 2016-11-22 4:54 ` Anshuman Khandual 2016-11-18 18:18 ` [HMM v13 06/18] mm/ZONE_DEVICE/unaddressable: add special swap for unaddressable Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 2:06 ` Balbir Singh 2016-11-21 2:06 ` Balbir Singh 2016-11-21 5:05 ` Jerome Glisse 2016-11-21 5:05 ` Jerome Glisse 2016-11-22 2:19 ` Balbir Singh 2016-11-22 2:19 ` Balbir Singh 2016-11-22 13:59 ` Jerome Glisse 2016-11-22 13:59 ` Jerome Glisse 2016-11-21 11:10 ` Anshuman Khandual 2016-11-21 11:10 ` Anshuman Khandual 2016-11-21 10:58 ` Anshuman Khandual 2016-11-21 10:58 ` Anshuman Khandual 2016-11-21 12:42 ` Jerome Glisse 2016-11-21 12:42 ` Jerome Glisse 2016-11-22 4:48 ` Anshuman Khandual 2016-11-22 4:48 ` Anshuman Khandual 2016-11-24 13:56 ` Jerome Glisse 2016-11-24 13:56 ` Jerome Glisse 2016-11-18 18:18 ` [HMM v13 07/18] mm/ZONE_DEVICE/x86: add support for un-addressable device memory Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 2:08 ` Balbir Singh 2016-11-21 2:08 ` Balbir Singh 2016-11-21 5:08 ` Jerome Glisse 2016-11-21 5:08 ` Jerome Glisse 2016-11-18 18:18 ` [HMM v13 08/18] mm/hmm: heterogeneous memory management (HMM for short) Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 2:29 ` Balbir Singh 2016-11-21 2:29 ` Balbir Singh 2016-11-21 5:14 ` Jerome Glisse 2016-11-21 5:14 ` Jerome Glisse 2016-11-23 4:03 ` Anshuman Khandual 2016-11-23 4:03 ` Anshuman Khandual 2016-11-27 13:10 ` Jerome Glisse 2016-11-27 13:10 ` Jerome Glisse 2016-11-28 2:58 ` Anshuman Khandual 2016-11-28 2:58 ` Anshuman Khandual 2016-11-28 9:41 ` Jerome Glisse 2016-11-28 9:41 ` Jerome Glisse 2016-11-18 18:18 ` [HMM v13 09/18] mm/hmm/mirror: mirror process address space on device with HMM helpers Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 2:42 ` Balbir Singh 2016-11-21 2:42 ` Balbir Singh 2016-11-21 5:18 ` Jerome Glisse 2016-11-21 5:18 ` Jerome Glisse 2016-11-18 18:18 ` [HMM v13 10/18] mm/hmm/mirror: add range lock helper, prevent CPU page table update for the range Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 18:18 ` [HMM v13 11/18] mm/hmm/mirror: add range monitor helper, to monitor CPU page table update Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 18:18 ` [HMM v13 12/18] mm/hmm/mirror: helper to snapshot CPU page table Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 18:18 ` [HMM v13 13/18] mm/hmm/mirror: device page fault handler Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 18:18 ` [HMM v13 14/18] mm/hmm/migrate: support un-addressable ZONE_DEVICE page in migration Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 18:18 ` [HMM v13 15/18] mm/hmm/migrate: add new boolean copy flag to migratepage() callback Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 18:18 ` [HMM v13 16/18] mm/hmm/migrate: new memory migration helper for use with device memory Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 19:57 ` Aneesh Kumar K.V 2016-11-18 19:57 ` Aneesh Kumar K.V 2016-11-18 20:15 ` Jerome Glisse 2016-11-18 20:15 ` Jerome Glisse 2016-11-19 14:32 ` Aneesh Kumar K.V 2016-11-19 14:32 ` Aneesh Kumar K.V 2016-11-19 17:17 ` Jerome Glisse 2016-11-19 17:17 ` Jerome Glisse 2016-11-20 18:21 ` Aneesh Kumar K.V 2016-11-20 18:21 ` Aneesh Kumar K.V 2016-11-20 20:06 ` Jerome Glisse 2016-11-20 20:06 ` Jerome Glisse 2016-11-21 3:30 ` Balbir Singh 2016-11-21 3:30 ` Balbir Singh 2016-11-21 5:31 ` Jerome Glisse 2016-11-21 5:31 ` Jerome Glisse 2016-11-18 18:18 ` [HMM v13 17/18] mm/hmm/devmem: device driver helper to hotplug ZONE_DEVICE memory Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 18:18 ` [HMM v13 18/18] mm/hmm/devmem: dummy HMM device as an helper for " Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-19 0:41 ` [HMM v13 00/18] HMM (Heterogeneous Memory Management) v13 John Hubbard 2016-11-19 0:41 ` John Hubbard 2016-11-19 14:50 ` Aneesh Kumar K.V 2016-11-19 14:50 ` Aneesh Kumar K.V 2016-11-23 9:16 ` Haggai Eran 2016-11-23 9:16 ` Haggai Eran 2016-11-25 16:16 ` Jerome Glisse 2016-11-25 16:16 ` Jerome Glisse 2016-11-27 13:27 ` Haggai Eran 2016-11-27 13:27 ` Haggai Eran
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=58329ACB.6030700@linux.vnet.ibm.com \ --to=khandual@linux.vnet.ibm.com \ --cc=akpm@linux-foundation.org \ --cc=benh@kernel.crashing.org \ --cc=bsingharora@gmail.com \ --cc=cmetcalf@mellanox.com \ --cc=dalias@libc.org \ --cc=heiko.carstens@de.ibm.com \ --cc=hpa@zytor.com \ --cc=jglisse@redhat.com \ --cc=jhubbard@nvidia.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux@armlinux.org.uk \ --cc=mingo@redhat.com \ --cc=mpe@ellerman.id.au \ --cc=paulus@samba.org \ --cc=schwidefsky@de.ibm.com \ --cc=tglx@linutronix.de \ --cc=ysato@users.sourceforge.jp \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.