Re: slab tree for next

[Vlastimil Babka <vbabka@suse.cz>]

On 12/1/21 21:34, Vlastimil Babka wrote:
> On 12/1/21 19:39, Vlastimil Babka wrote:
>> On 12/1/21 19:14, Vlastimil Babka wrote:
>>> Folks from non-slab subsystems are Cc'd only to patches affecting them, and
>>> this cover letter.
>>>
>>> Series also available in git, based on 5.16-rc3:
>>> https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=slab-struct_slab-v2r2
>>>
>>> The plan: as my SLUB PREEMPT_RT series in 5.15, I would prefer to go again with
>>> the git pull request way of eventually merging this, as it's also not a small
>>> series. I will thus reply to this mail with asking to include my branch in
>>> linux-next.
>>>
>>> As stated in the v1/RFC cover letter, I wouldn't mind to then continue with
>>> maintaining a git tree for all slab patches in general. It was apparently
>>> already done that way before, by Pekka:
>>> https://lore.kernel.org/linux-mm/alpine.DEB.2.00.1107221108190.2996@tiger/
>> 
>> Hi Stephen,
>> 
>> Please include a new tree in linux-next:
> 
> OK, not yet, please:
> https://lore.kernel.org/all/70fbdc70-6838-0740-43e3-ed20caff47bf@arm.com/
> 
> I will reorder the patches so that changes needed in other subsystems
> and the dependent actual removal of slab fields from struct page will be
> ordered last, and only ask for the first, mm/ contained part to be in
> -next. The patches for other subsystems can then be reviewed and picked
> by their maintainers independently, while slab implementations are
> already using struct slab.

OK, I've did that and removed the iommu commit, and the dependent removal of
slab-specific struct page fields, until that is resolved. slab
implementatiosn is nevertheless converted  to struct slab, and zsmalloc and
bootmem stop using the slab's struct page fields, so the struct page cleanup
can be finished any time later when iommu is dealt with.

> 
>> git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git slab-next

So the 'slab-next' branch is now ready for linux-next, now identical to
branch slab-struct_slab-v3r1, head 	d395d823b3ae.

>> i.e.
>> https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=slab-next
>> 
>> Which now is identical slab-struct_slab-v2r2 branch [1]
>> 
>> When I tried to merge this to next-20211201, there were minor conflicts with
>> two patches from motm:
>> zsmalloc-move-huge-compressed-obj-from-page-to-zspage.patch
>> mm-memcg-relocate-mod_objcg_mlstate-get_obj_stock-and-put_obj_stock.patch
>> 
>> Both appear to be just a change in context.
>> 
>> Thanks,
>> Vlastimil
>> 
>> [1] https://lore.kernel.org/all/20211201181510.18784-1-vbabka@suse.cz/
>> 
>>> Changes from v1/RFC:
>>> https://lore.kernel.org/all/20211116001628.24216-1-vbabka@suse.cz/
>>> - Added virt_to_folio() and folio_address() in the new Patch 1.
>>> - Addressed feedback from Andrey Konovalov and Matthew Wilcox (Thanks!)
>>> - Added Tested-by: Marco Elver for the KFENCE parts (Thanks!)
>>>
>>> Previous version from Matthew Wilcox:
>>> https://lore.kernel.org/all/20211004134650.4031813-1-willy@infradead.org/
>>>
>>> LWN coverage of the above:
>>> https://lwn.net/Articles/871982/
>>>
>>> This is originally an offshoot of the folio work by Matthew. One of the more
>>> complex parts of the struct page definition are the parts used by the slab
>>> allocators. It would be good for the MM in general if struct slab were its own
>>> data type, and it also helps to prevent tail pages from slipping in anywhere.
>>> As Matthew requested in his proof of concept series, I have taken over the
>>> development of this series, so it's a mix of patches from him (often modified
>>> by me) and my own.
>>>
>>> One big difference is the use of coccinelle to perform the relatively trivial
>>> parts of the conversions automatically and at once, instead of a larger number
>>> of smaller incremental reviewable steps. Thanks to Julia Lawall and Luis
>>> Chamberlain for all their help!
>>>
>>> Another notable difference is (based also on review feedback) I don't represent
>>> with a struct slab the large kmalloc allocations which are not really a slab,
>>> but use page allocator directly. When going from an object address to a struct
>>> slab, the code tests first folio slab flag, and only if it's set it converts to
>>> struct slab. This makes the struct slab type stronger.
>>>
>>> Finally, although Matthew's version didn't use any of the folio work, the
>>> initial support has been merged meanwhile so my version builds on top of it
>>> where appropriate. This eliminates some of the redundant compound_head()
>>> being performed e.g. when testing the slab flag.
>>>
>>> To sum up, after this series, struct page fields used by slab allocators are
>>> moved from struct page to a new struct slab, that uses the same physical
>>> storage. The availability of the fields is further distinguished by the
>>> selected slab allocator implementation. The advantages include:
>>>
>>> - Similar to folios, if the slab is of order > 0, struct slab always is
>>>   guaranteed to be the head page. Additionally it's guaranteed to be an actual
>>>   slab page, not a large kmalloc. This removes uncertainty and potential for
>>>   bugs.
>>> - It's not possible to accidentally use fields of the slab implementation that's
>>>   not configured.
>>> - Other subsystems cannot use slab's fields in struct page anymore (some
>>>   existing non-slab usages had to be adjusted in this series), so slab
>>>   implementations have more freedom in rearranging them in the struct slab.
>>>
>>> Matthew Wilcox (Oracle) (16):
>>>   mm: Split slab into its own type
>>>   mm: Add account_slab() and unaccount_slab()
>>>   mm: Convert virt_to_cache() to use struct slab
>>>   mm: Convert __ksize() to struct slab
>>>   mm: Use struct slab in kmem_obj_info()
>>>   mm: Convert check_heap_object() to use struct slab
>>>   mm/slub: Convert detached_freelist to use a struct slab
>>>   mm/slub: Convert kfree() to use a struct slab
>>>   mm/slub: Convert print_page_info() to print_slab_info()
>>>   mm/slub: Convert pfmemalloc_match() to take a struct slab
>>>   mm/slob: Convert SLOB to use struct slab
>>>   mm/kasan: Convert to struct folio and struct slab
>>>   zsmalloc: Stop using slab fields in struct page
>>>   bootmem: Use page->index instead of page->freelist
>>>   iommu: Use put_pages_list
>>>   mm: Remove slab from struct page
>>>
>>> Vlastimil Babka (17):
>>>   mm: add virt_to_folio() and folio_address()
>>>   mm/slab: Dissolve slab_map_pages() in its caller
>>>   mm/slub: Make object_err() static
>>>   mm/slub: Convert __slab_lock() and __slab_unlock() to struct slab
>>>   mm/slub: Convert alloc_slab_page() to return a struct slab
>>>   mm/slub: Convert __free_slab() to use struct slab
>>>   mm/slub: Convert most struct page to struct slab by spatch
>>>   mm/slub: Finish struct page to struct slab conversion
>>>   mm/slab: Convert kmem_getpages() and kmem_freepages() to struct slab
>>>   mm/slab: Convert most struct page to struct slab by spatch
>>>   mm/slab: Finish struct page to struct slab conversion
>>>   mm: Convert struct page to struct slab in functions used by other
>>>     subsystems
>>>   mm/memcg: Convert slab objcgs from struct page to struct slab
>>>   mm/kfence: Convert kfence_guarded_alloc() to struct slab
>>>   mm/sl*b: Differentiate struct slab fields by sl*b implementations
>>>   mm/slub: Simplify struct slab slabs field definition
>>>   mm/slub: Define struct slab fields for CONFIG_SLUB_CPU_PARTIAL only
>>>     when enabled
>>>
>>>  arch/x86/mm/init_64.c          |    2 +-
>>>  drivers/iommu/amd/io_pgtable.c |   59 +-
>>>  drivers/iommu/dma-iommu.c      |   11 +-
>>>  drivers/iommu/intel/iommu.c    |   89 +--
>>>  include/linux/bootmem_info.h   |    2 +-
>>>  include/linux/iommu.h          |    3 +-
>>>  include/linux/kasan.h          |    9 +-
>>>  include/linux/memcontrol.h     |   48 --
>>>  include/linux/mm.h             |   12 +
>>>  include/linux/mm_types.h       |   38 +-
>>>  include/linux/page-flags.h     |   37 -
>>>  include/linux/slab.h           |    8 -
>>>  include/linux/slab_def.h       |   16 +-
>>>  include/linux/slub_def.h       |   29 +-
>>>  mm/bootmem_info.c              |    7 +-
>>>  mm/kasan/common.c              |   27 +-
>>>  mm/kasan/generic.c             |    8 +-
>>>  mm/kasan/kasan.h               |    1 +
>>>  mm/kasan/quarantine.c          |    2 +-
>>>  mm/kasan/report.c              |   13 +-
>>>  mm/kasan/report_tags.c         |   10 +-
>>>  mm/kfence/core.c               |   17 +-
>>>  mm/kfence/kfence_test.c        |    6 +-
>>>  mm/memcontrol.c                |   43 +-
>>>  mm/slab.c                      |  455 ++++++-------
>>>  mm/slab.h                      |  322 ++++++++-
>>>  mm/slab_common.c               |    8 +-
>>>  mm/slob.c                      |   46 +-
>>>  mm/slub.c                      | 1164 ++++++++++++++++----------------
>>>  mm/sparse.c                    |    2 +-
>>>  mm/usercopy.c                  |   13 +-
>>>  mm/zsmalloc.c                  |   18 +-
>>>  32 files changed, 1317 insertions(+), 1208 deletions(-)
>>>
>> 
> 
>