[PATCH] docs: reorder memory-hotplug documentation

[PATCH] docs: reorder memory-hotplug documentation

[Mike Rapoport]

The "Locking Internals" section of the memory-hotplug documentation is
duplicated in admin-guide and core-api. Drop the admin-guide copy as
locking internals does not belong there.

While on it, move the "Future Work" section to the core-api part.

Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
---
 Documentation/admin-guide/mm/memory-hotplug.rst | 51 -------------------------
 Documentation/core-api/memory-hotplug.rst       | 11 ++++++
 2 files changed, 11 insertions(+), 51 deletions(-)

diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst b/Documentation/admin-guide/mm/memory-hotplug.rst
index 5c4432c..72090ba 100644
--- a/Documentation/admin-guide/mm/memory-hotplug.rst
+++ b/Documentation/admin-guide/mm/memory-hotplug.rst
@@ -391,54 +391,3 @@ Physical memory remove
 Need more implementation yet....
  - Notification completion of remove works by OS to firmware.
  - Guard from remove if not yet.
-
-
-Locking Internals
-=================
-
-When adding/removing memory that uses memory block devices (i.e. ordinary RAM),
-the device_hotplug_lock should be held to:
-
-- synchronize against online/offline requests (e.g. via sysfs). This way, memory
-  block devices can only be accessed (.online/.state attributes) by user
-  space once memory has been fully added. And when removing memory, we
-  know nobody is in critical sections.
-- synchronize against CPU hotplug and similar (e.g. relevant for ACPI and PPC)
-
-Especially, there is a possible lock inversion that is avoided using
-device_hotplug_lock when adding memory and user space tries to online that
-memory faster than expected:
-
-- device_online() will first take the device_lock(), followed by
-  mem_hotplug_lock
-- add_memory_resource() will first take the mem_hotplug_lock, followed by
-  the device_lock() (while creating the devices, during bus_add_device()).
-
-As the device is visible to user space before taking the device_lock(), this
-can result in a lock inversion.
-
-onlining/offlining of memory should be done via device_online()/
-device_offline() - to make sure it is properly synchronized to actions
-via sysfs. Holding device_hotplug_lock is advised (to e.g. protect online_type)
-
-When adding/removing/onlining/offlining memory or adding/removing
-heterogeneous/device memory, we should always hold the mem_hotplug_lock in
-write mode to serialise memory hotplug (e.g. access to global/zone
-variables).
-
-In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read
-mode allows for a quite efficient get_online_mems/put_online_mems
-implementation, so code accessing memory can protect from that memory
-vanishing.
-
-
-Future Work
-===========
-
-  - allowing memory hot-add to ZONE_MOVABLE. maybe we need some switch like
-    sysctl or new control file.
-  - showing memory block and physical device relationship.
-  - test and make it better memory offlining.
-  - support HugeTLB page migration and offlining.
-  - memmap removing at memory offline.
-  - physical remove memory.
diff --git a/Documentation/core-api/memory-hotplug.rst b/Documentation/core-api/memory-hotplug.rst
index de7467e..e08be1c 100644
--- a/Documentation/core-api/memory-hotplug.rst
+++ b/Documentation/core-api/memory-hotplug.rst
@@ -123,3 +123,14 @@ In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read
 mode allows for a quite efficient get_online_mems/put_online_mems
 implementation, so code accessing memory can protect from that memory
 vanishing.
+
+Future Work
+===========
+
+  - allowing memory hot-add to ZONE_MOVABLE. maybe we need some switch like
+    sysctl or new control file.
+  - showing memory block and physical device relationship.
+  - test and make it better memory offlining.
+  - support HugeTLB page migration and offlining.
+  - memmap removing at memory offline.
+  - physical remove memory.
-- 
2.7.4

Re: [PATCH] docs: reorder memory-hotplug documentation

[David Hildenbrand]

On 14.05.19 10:23, Mike Rapoport wrote:
> The "Locking Internals" section of the memory-hotplug documentation is
> duplicated in admin-guide and core-api. Drop the admin-guide copy as
> locking internals does not belong there.
> 
> While on it, move the "Future Work" section to the core-api part.

Looks sane, but the future work part is really outdated, can we remove
this completely?

> 
> Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
> ---
>  Documentation/admin-guide/mm/memory-hotplug.rst | 51 -------------------------
>  Documentation/core-api/memory-hotplug.rst       | 11 ++++++
>  2 files changed, 11 insertions(+), 51 deletions(-)
> 
> diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst b/Documentation/admin-guide/mm/memory-hotplug.rst
> index 5c4432c..72090ba 100644
> --- a/Documentation/admin-guide/mm/memory-hotplug.rst
> +++ b/Documentation/admin-guide/mm/memory-hotplug.rst
> @@ -391,54 +391,3 @@ Physical memory remove
>  Need more implementation yet....
>   - Notification completion of remove works by OS to firmware.
>   - Guard from remove if not yet.
> -
> -
> -Locking Internals
> -=================
> -
> -When adding/removing memory that uses memory block devices (i.e. ordinary RAM),
> -the device_hotplug_lock should be held to:
> -
> -- synchronize against online/offline requests (e.g. via sysfs). This way, memory
> -  block devices can only be accessed (.online/.state attributes) by user
> -  space once memory has been fully added. And when removing memory, we
> -  know nobody is in critical sections.
> -- synchronize against CPU hotplug and similar (e.g. relevant for ACPI and PPC)
> -
> -Especially, there is a possible lock inversion that is avoided using
> -device_hotplug_lock when adding memory and user space tries to online that
> -memory faster than expected:
> -
> -- device_online() will first take the device_lock(), followed by
> -  mem_hotplug_lock
> -- add_memory_resource() will first take the mem_hotplug_lock, followed by
> -  the device_lock() (while creating the devices, during bus_add_device()).
> -
> -As the device is visible to user space before taking the device_lock(), this
> -can result in a lock inversion.
> -
> -onlining/offlining of memory should be done via device_online()/
> -device_offline() - to make sure it is properly synchronized to actions
> -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect online_type)
> -
> -When adding/removing/onlining/offlining memory or adding/removing
> -heterogeneous/device memory, we should always hold the mem_hotplug_lock in
> -write mode to serialise memory hotplug (e.g. access to global/zone
> -variables).
> -
> -In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read
> -mode allows for a quite efficient get_online_mems/put_online_mems
> -implementation, so code accessing memory can protect from that memory
> -vanishing.
> -
> -
> -Future Work
> -===========
> -
> -  - allowing memory hot-add to ZONE_MOVABLE. maybe we need some switch like
> -    sysctl or new control file.
> -  - showing memory block and physical device relationship.
> -  - test and make it better memory offlining.
> -  - support HugeTLB page migration and offlining.
> -  - memmap removing at memory offline.
> -  - physical remove memory.
> diff --git a/Documentation/core-api/memory-hotplug.rst b/Documentation/core-api/memory-hotplug.rst
> index de7467e..e08be1c 100644
> --- a/Documentation/core-api/memory-hotplug.rst
> +++ b/Documentation/core-api/memory-hotplug.rst
> @@ -123,3 +123,14 @@ In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read
>  mode allows for a quite efficient get_online_mems/put_online_mems
>  implementation, so code accessing memory can protect from that memory
>  vanishing.
> +
> +Future Work
> +===========
> +
> +  - allowing memory hot-add to ZONE_MOVABLE. maybe we need some switch like
> +    sysctl or new control file.

... that already works if I am not completely missing the point here

> +  - showing memory block and physical device relationship.

... that is available for s390x only AFAIK

> +  - test and make it better memory offlining.

... no big news ;)

> +  - support HugeTLB page migration and offlining.

... I remember that Oscar was doing something in that area, Oscar?

> +  - memmap removing at memory offline.

... no, we don't want this. However, we should properly clean up zone
information when offlining

> +  - physical remove memory.

... I don't even understand what that means.


I'd vote for removing the future work part, this is pretty outdated.


-- 

Thanks,

David / dhildenb

Re: [PATCH] docs: reorder memory-hotplug documentation

[Mike Rapoport]

On Tue, May 21, 2019 at 12:41:50PM +0200, David Hildenbrand wrote:
> On 14.05.19 10:23, Mike Rapoport wrote:
> > The "Locking Internals" section of the memory-hotplug documentation is
> > duplicated in admin-guide and core-api. Drop the admin-guide copy as
> > locking internals does not belong there.
> > 
> > While on it, move the "Future Work" section to the core-api part.
> 
> Looks sane, but the future work part is really outdated, can we remove
> this completely?
> 
> > 
> > Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
> > ---
> > +
> > +Future Work
> > +===========
> > +
> > +  - allowing memory hot-add to ZONE_MOVABLE. maybe we need some switch like
> > +    sysctl or new control file.
> 
> ... that already works if I am not completely missing the point here
> 
> > +  - showing memory block and physical device relationship.
> 
> ... that is available for s390x only AFAIK
> 
> > +  - test and make it better memory offlining.
> 
> ... no big news ;)
> 
> > +  - support HugeTLB page migration and offlining.
> 
> ... I remember that Oscar was doing something in that area, Oscar?
> 
> > +  - memmap removing at memory offline.
> 
> ... no, we don't want this. However, we should properly clean up zone
> information when offlining
> 
> > +  - physical remove memory.
> 
> ... I don't even understand what that means.
> 
> 
> I'd vote for removing the future work part, this is pretty outdated.
 
Frankly, I haven't looked at the details, just simply moved the text over.
I don't mind sending another mechanical patch that removes the future work
part.

But it would be far better if somebody who's actively working on memory
hotplug would replace it with a description how this actually works ;-)
 
> -- 
> 
> Thanks,
> 
> David / dhildenb
> 

-- 
Sincerely yours,
Mike.

Re: [PATCH] docs: reorder memory-hotplug documentation

[Oscar Salvador]

On Tue, May 21, 2019 at 12:41:50PM +0200, David Hildenbrand wrote:
> > +Future Work
> > +===========
> > +
> > +  - allowing memory hot-add to ZONE_MOVABLE. maybe we need some switch like
> > +    sysctl or new control file.
> 
> ... that already works if I am not completely missing the point here

It does.

> > +  - support HugeTLB page migration and offlining.
> 
> ... I remember that Oscar was doing something in that area, Oscar?

Yes, in general offlinining on hugetlb pages was already working, but we did not
allow to offline 1GB-hugetlb pages on x86_64.
I removed that limitation with
("commit: 10eeadf3045c mm,memory_hotplug: unlock 1GB-hugetlb on x86_64") , so now
offlining on hugetlb pages should be fully operative.

> I'd vote for removing the future work part, this is pretty outdated.

Instead of removing it, I would rather make it consistent with the present.
E.g:

- Move page handling from memory-hotremove to offline stage
- Enable a way to allocate vmemmap pages from hot-added memory
etc.


-- 
Oscar Salvador
SUSE L3

Re: [PATCH] docs: reorder memory-hotplug documentation

[David Hildenbrand]

On 21.05.19 18:11, Oscar Salvador wrote:
> On Tue, May 21, 2019 at 12:41:50PM +0200, David Hildenbrand wrote:
>>> +Future Work
>>> +===========
>>> +
>>> +  - allowing memory hot-add to ZONE_MOVABLE. maybe we need some switch like
>>> +    sysctl or new control file.
>>
>> ... that already works if I am not completely missing the point here
> 
> It does.
> 
>>> +  - support HugeTLB page migration and offlining.
>>
>> ... I remember that Oscar was doing something in that area, Oscar?
> 
> Yes, in general offlinining on hugetlb pages was already working, but we did not
> allow to offline 1GB-hugetlb pages on x86_64.
> I removed that limitation with
> ("commit: 10eeadf3045c mm,memory_hotplug: unlock 1GB-hugetlb on x86_64") , so now
> offlining on hugetlb pages should be fully operative.
> 
>> I'd vote for removing the future work part, this is pretty outdated.
> 
> Instead of removing it, I would rather make it consistent with the present.
> E.g:
> 
> - Move page handling from memory-hotremove to offline stage
> - Enable a way to allocate vmemmap pages from hot-added memory
> etc.
> 
> 

Fair enough, but at least the current content is absolutely useless and
confusing/misleading.

-- 

Thanks,

David / dhildenb