Re: [PATCH RFC 5/9] x86, mm: Use cache of page tables

["Edgecombe, Rick P" <rick.p.edgecombe@intel.com>]

On Wed, 2021-05-05 at 15:19 +0200, Peter Zijlstra wrote:
> On Wed, May 05, 2021 at 03:09:09PM +0300, Mike Rapoport wrote:
> > On Wed, May 05, 2021 at 10:51:55AM +0200, Peter Zijlstra wrote:
> > > On Tue, May 04, 2021 at 05:30:28PM -0700, Rick Edgecombe wrote:
> > > > @@ -54,6 +98,8 @@ void ___pte_free_tlb(struct mmu_gather *tlb,
> > > > struct page *pte)
> > > >  {
> > > >         pgtable_pte_page_dtor(pte);
> > > >         paravirt_release_pte(page_to_pfn(pte));
> > > > +       /* Set Page Table so swap knows how to free it */
> > > > +       __SetPageTable(pte);
> > > >         paravirt_tlb_remove_table(tlb, pte);
> > > >  }
> > > >  
> > > > @@ -70,12 +116,16 @@ void ___pmd_free_tlb(struct mmu_gather
> > > > *tlb, pmd_t *pmd)
> > > >         tlb->need_flush_all = 1;
> > > >  #endif
> > > >         pgtable_pmd_page_dtor(page);
> > > > +       /* Set Page Table so swap nows how to free it */
> > > > +       __SetPageTable(virt_to_page(pmd));
> > > >         paravirt_tlb_remove_table(tlb, page);
> > > >  }
> > > >  
> > > >  #if CONFIG_PGTABLE_LEVELS > 3
> > > >  void ___pud_free_tlb(struct mmu_gather *tlb, pud_t *pud)
> > > >  {
> > > > +       /* Set Page Table so swap nows how to free it */
> > > > +       __SetPageTable(virt_to_page(pud));
> > > >         paravirt_release_pud(__pa(pud) >> PAGE_SHIFT);
> > > >         paravirt_tlb_remove_table(tlb, virt_to_page(pud));
> > > >  }
> > > > @@ -83,6 +133,8 @@ void ___pud_free_tlb(struct mmu_gather *tlb,
> > > > pud_t *pud)
> > > >  #if CONFIG_PGTABLE_LEVELS > 4
> > > >  void ___p4d_free_tlb(struct mmu_gather *tlb, p4d_t *p4d)
> > > >  {
> > > > +       /* Set Page Table so swap nows how to free it */
> > > > +       __SetPageTable(virt_to_page(p4d));
> > > >         paravirt_release_p4d(__pa(p4d) >> PAGE_SHIFT);
> > > >         paravirt_tlb_remove_table(tlb, virt_to_page(p4d));
> > > >  }
> > > 
> > > This, to me, seems like a really weird place to __SetPageTable(),
> > > why
> > > can't we do that on allocation?
> > 
> > We call __ClearPageTable() at pgtable_pxy_page_dtor(), so at least
> > for pte
> > and pmd we need to somehow tell release_pages() what kind of page
> > it was.
> 
> Hurph, right, but then the added comment is misleading;
> s/Set/Reset/g.
> Still I'm thinking that if we do these allocators, moving the
> set/clear
> to the allocator would be the most natural place, perhaps we can
> remove
> them from the {c,d}tor.

Hmm, yes. I guess there could be just x86 specific versions of the
ctor/dtor that don't set the flag. Seems like it should work and be
less confusing. Thanks.