Re: [PATCH v11 4/8] mm/demotion/dax/kmem: Set node's abstract distance to MEMTIER_ADISTANCE_PMEM

From: Aneesh Kumar K V <aneesh.kumar@linux.ibm.com>
To: "Huang, Ying" <ying.huang@intel.com>
Cc: linux-mm@kvack.org, akpm@linux-foundation.org,
	Wei Xu <weixugc@google.com>, Yang Shi <shy828301@gmail.com>,
	Davidlohr Bueso <dave@stgolabs.net>,
	Tim C Chen <tim.c.chen@intel.com>,
	Michal Hocko <mhocko@kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Hesham Almatary <hesham.almatary@huawei.com>,
	Dave Hansen <dave.hansen@intel.com>,
	Jonathan Cameron <Jonathan.Cameron@huawei.com>,
	Alistair Popple <apopple@nvidia.com>,
	Dan Williams <dan.j.williams@intel.com>,
	Johannes Weiner <hannes@cmpxchg.org>,
	jvgediya.oss@gmail.com
Subject: Re: [PATCH v11 4/8] mm/demotion/dax/kmem: Set node's abstract distance to MEMTIER_ADISTANCE_PMEM
Date: Mon, 1 Aug 2022 12:25:04 +0530	[thread overview]
Message-ID: <1aba0c44-b096-8c75-8086-62d3cffc08b3@linux.ibm.com> (raw)
In-Reply-To: <87les8jwpu.fsf@yhuang6-desk2.ccr.corp.intel.com>

On 8/1/22 12:07 PM, Huang, Ying wrote:
> Aneesh Kumar K V <aneesh.kumar@linux.ibm.com> writes:
> 
>> On 8/1/22 10:40 AM, Huang, Ying wrote:
>>> Aneesh Kumar K V <aneesh.kumar@linux.ibm.com> writes:
>>>
>>>> On 8/1/22 7:36 AM, Huang, Ying wrote:
>>>>> "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com> writes:
>>>>>
>>>>>> "Huang, Ying" <ying.huang@intel.com> writes:
>>>>>>
>>>>>>> "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com> writes:

....

>>>>
>>>> With the module unload, it is kind of force removing the usage of the specific memtype.
>>>> Considering module unload will remove the usage of specific memtype from other parts
>>>> of the kernel and we already do all the required reset in memory hot unplug, do we
>>>> need to do the clear_node_memory_type above? 
>>>
>>> Per my understanding, we need to call clear_node_memory_type() in
>>> dev_dax_kmem_remove().  After that, we have nothing to do in
>>> dax_kmem_exit().
>>>
>>
>> Ok, I guess you are suggesting to do the clear_node_memory_type even if we fail the memory remove. 
> 
> Can we use node_memory_types[] to indicate whether a node is managed by
> a driver?
> 
> Regardless being succeeded or failed, dev_dax_kmem_remove() will set
> node_memory_types[] = NULL.  But until node is offlined, we will still
> keep the node in the memory_dev_type (dax_pmem_type).
> 
> And we will prevent dax/kmem from unloading via try_module_get() and add
> "struct module *" to struct memory_dev_type.
> 

Current dax/kmem driver is not holding any module reference and allows the module to be unloaded
anytime. Even if the memory onlined by the driver fails to be unplugged. Addition of memory_dev_type
as suggested by you will be different than that. Page demotion can continue to work without the
support of dax_pmem_type as long as we keep the older demotion order. Any new demotion order
rebuild will remove the the memory node which was not hotunplugged  from the demotion order. Isn't that
a much simpler implementation? 

-aneesh