[PATCH 1/2] mm: Report success more often from filemap_map_folio_range()

[PATCH 1/2] mm: Report success more often from filemap_map_folio_range()

[Matthew Wilcox (Oracle)]

Even though we had successfully mapped the relevant page, we would
rarely return success from filemap_map_folio_range().  That leads to
falling back from the VMA lock path to the mmap_lock path, which is a
speed & scalability issue.  Found by inspection.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Fixes: 617c28ecab22 ("filemap: batch PTE mappings")
---
 mm/filemap.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/mm/filemap.c b/mm/filemap.c
index 582f5317ff71..580d0b2b1a7c 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -3506,7 +3506,7 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf,
 		if (count) {
 			set_pte_range(vmf, folio, page, count, addr);
 			folio_ref_add(folio, count);
-			if (in_range(vmf->address, addr, count))
+			if (in_range(vmf->address, addr, count * PAGE_SIZE))
 				ret = VM_FAULT_NOPAGE;
 		}
 
@@ -3520,7 +3520,7 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf,
 	if (count) {
 		set_pte_range(vmf, folio, page, count, addr);
 		folio_ref_add(folio, count);
-		if (in_range(vmf->address, addr, count))
+		if (in_range(vmf->address, addr, count * PAGE_SIZE))
 			ret = VM_FAULT_NOPAGE;
 	}
 
-- 
2.40.1

[PATCH 2/2] mm: Abstract moving to the next PFN

[Matthew Wilcox (Oracle)]

In order to fix the L1TF vulnerability, x86 can invert the PTE bits for
PROT_NONE VMAs, which means we cannot move from one PTE to the next by
adding 1 to the PFN field of the PTE.  Abstract advancing the PTE to
the next PFN through a pte_next_pfn() function/macro.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Fixes: bcc6cc832573 ("mm: add default definition of set_ptes()")
Reported-by: syzbot+55cc72f8cc3a549119df@syzkaller.appspotmail.com
---
 arch/x86/include/asm/pgtable.h | 8 ++++++++
 include/linux/pgtable.h        | 6 +++++-
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index d6ad98ca1288..e02b179ec659 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -955,6 +955,14 @@ static inline int pte_same(pte_t a, pte_t b)
 	return a.pte == b.pte;
 }
 
+static inline pte_t pte_next_pfn(pte_t pte)
+{
+	if (__pte_needs_invert(pte_val(pte)))
+		return __pte(pte_val(pte) - (1UL << PFN_PTE_SHIFT));
+	return __pte(pte_val(pte) + (1UL << PFN_PTE_SHIFT));
+}
+#define pte_next_pfn	pte_next_pfn
+
 static inline int pte_present(pte_t a)
 {
 	return pte_flags(a) & (_PAGE_PRESENT | _PAGE_PROTNONE);
diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
index 1fba072b3dac..5bdf78aa7852 100644
--- a/include/linux/pgtable.h
+++ b/include/linux/pgtable.h
@@ -205,6 +205,10 @@ static inline int pmd_young(pmd_t pmd)
 #define arch_flush_lazy_mmu_mode()	do {} while (0)
 #endif
 
+#ifndef pte_next_pfn
+#define pte_next_pfn(pte)	((pte) + (1UL << PFN_PTE_SHIFT))
+#endif
+
 #ifndef set_ptes
 /**
  * set_ptes - Map consecutive pages to a contiguous range of addresses.
@@ -231,7 +235,7 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr,
 		if (--nr == 0)
 			break;
 		ptep++;
-		pte = __pte(pte_val(pte) + (1UL << PFN_PTE_SHIFT));
+		pte = pte_next_pfn(pte);
 	}
 	arch_leave_lazy_mmu_mode();
 }
-- 
2.40.1

Re: [PATCH 2/2] mm: Abstract moving to the next PFN

[Matthew Wilcox]

On Wed, Sep 20, 2023 at 04:53:36AM +0100, Matthew Wilcox (Oracle) wrote:
> In order to fix the L1TF vulnerability, x86 can invert the PTE bits for
> PROT_NONE VMAs, which means we cannot move from one PTE to the next by
> adding 1 to the PFN field of the PTE.  Abstract advancing the PTE to
> the next PFN through a pte_next_pfn() function/macro.

Argh, wrong version.  New version coming up.

Re: [PATCH 2/2] mm: Abstract moving to the next PFN

[kernel test robot]

Hi Matthew,

kernel test robot noticed the following build errors:

[auto build test ERROR on akpm-mm/mm-everything]

url:    https://github.com/intel-lab-lkp/linux/commits/Matthew-Wilcox-Oracle/mm-Abstract-moving-to-the-next-PFN/20230920-115500
base:   https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link:    https://lore.kernel.org/r/20230920035336.854212-2-willy%40infradead.org
patch subject: [PATCH 2/2] mm: Abstract moving to the next PFN
config: um-i386_defconfig (https://download.01.org/0day-ci/archive/20230920/202309201731.Tr9SZYEz-lkp@intel.com/config)
compiler: gcc-7 (Ubuntu 7.5.0-6ubuntu2) 7.5.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20230920/202309201731.Tr9SZYEz-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202309201731.Tr9SZYEz-lkp@intel.com/

All errors (new ones prefixed by >>):

   In file included from include/linux/mm.h:29:0,
                    from include/linux/ring_buffer.h:5,
                    from include/linux/trace_events.h:6,
                    from include/trace/syscall.h:7,
                    from include/linux/syscalls.h:90,
                    from init/main.c:21:
   include/linux/pgtable.h: In function 'set_ptes':
>> include/linux/pgtable.h:209:34: error: invalid operands to binary + (have 'pte_t {aka struct <anonymous>}' and 'long unsigned int')
    #define pte_next_pfn(pte) ((pte) + (1UL << PFN_PTE_SHIFT))
                                     ^
   include/linux/pgtable.h:238:9: note: in expansion of macro 'pte_next_pfn'
      pte = pte_next_pfn(pte);
            ^~~~~~~~~~~~
--
   In file included from include/linux/mm.h:29:0,
                    from arch/x86/um/ptrace_32.c:6:
   include/linux/pgtable.h: In function 'set_ptes':
>> include/linux/pgtable.h:209:34: error: invalid operands to binary + (have 'pte_t {aka struct <anonymous>}' and 'long unsigned int')
    #define pte_next_pfn(pte) ((pte) + (1UL << PFN_PTE_SHIFT))
                                     ^
   include/linux/pgtable.h:238:9: note: in expansion of macro 'pte_next_pfn'
      pte = pte_next_pfn(pte);
            ^~~~~~~~~~~~
   arch/x86/um/ptrace_32.c: At top level:
   arch/x86/um/ptrace_32.c:15:6: warning: no previous declaration for 'arch_switch_to' [-Wmissing-declarations]
    void arch_switch_to(struct task_struct *to)
         ^~~~~~~~~~~~~~
   arch/x86/um/ptrace_32.c:28:5: warning: no previous declaration for 'is_syscall' [-Wmissing-declarations]
    int is_syscall(unsigned long addr)
        ^~~~~~~~~~
   arch/x86/um/ptrace_32.c:125:5: warning: no previous declaration for 'poke_user' [-Wmissing-declarations]
    int poke_user(struct task_struct *child, long addr, long data)
        ^~~~~~~~~
   arch/x86/um/ptrace_32.c:177:5: warning: no previous declaration for 'peek_user' [-Wmissing-declarations]
    int peek_user(struct task_struct *child, long addr, long data)
        ^~~~~~~~~
--
   In file included from include/linux/mm.h:29:0,
                    from include/linux/pid_namespace.h:7,
                    from include/linux/ptrace.h:10,
                    from arch/x86/um/signal.c:9:
   include/linux/pgtable.h: In function 'set_ptes':
>> include/linux/pgtable.h:209:34: error: invalid operands to binary + (have 'pte_t {aka struct <anonymous>}' and 'long unsigned int')
    #define pte_next_pfn(pte) ((pte) + (1UL << PFN_PTE_SHIFT))
                                     ^
   include/linux/pgtable.h:238:9: note: in expansion of macro 'pte_next_pfn'
      pte = pte_next_pfn(pte);
            ^~~~~~~~~~~~
   arch/x86/um/signal.c: At top level:
   arch/x86/um/signal.c:453:6: warning: no previous declaration for 'sys_sigreturn' [-Wmissing-declarations]
    long sys_sigreturn(void)
         ^~~~~~~~~~~~~
   arch/x86/um/signal.c:560:6: warning: no previous declaration for 'sys_rt_sigreturn' [-Wmissing-declarations]
    long sys_rt_sigreturn(void)
         ^~~~~~~~~~~~~~~~
--
   In file included from include/linux/mm.h:29:0,
                    from include/linux/ring_buffer.h:5,
                    from include/linux/trace_events.h:6,
                    from include/trace/syscall.h:7,
                    from include/linux/syscalls.h:90,
                    from arch/x86/um/tls_32.c:8:
   include/linux/pgtable.h: In function 'set_ptes':
>> include/linux/pgtable.h:209:34: error: invalid operands to binary + (have 'pte_t {aka struct <anonymous>}' and 'long unsigned int')
    #define pte_next_pfn(pte) ((pte) + (1UL << PFN_PTE_SHIFT))
                                     ^
   include/linux/pgtable.h:238:9: note: in expansion of macro 'pte_next_pfn'
      pte = pte_next_pfn(pte);
            ^~~~~~~~~~~~
   arch/x86/um/tls_32.c: At top level:
   arch/x86/um/tls_32.c:23:5: warning: no previous declaration for 'do_set_thread_area' [-Wmissing-declarations]
    int do_set_thread_area(struct user_desc *info)
        ^~~~~~~~~~~~~~~~~~
   arch/x86/um/tls_32.c:39:5: warning: no previous declaration for 'do_get_thread_area' [-Wmissing-declarations]
    int do_get_thread_area(struct user_desc *info)
        ^~~~~~~~~~~~~~~~~~
   arch/x86/um/tls_32.c:184:5: warning: no previous declaration for 'arch_switch_tls' [-Wmissing-declarations]
    int arch_switch_tls(struct task_struct *to)
        ^~~~~~~~~~~~~~~
--
   In file included from include/linux/mm.h:29:0,
                    from include/linux/coredump.h:6,
                    from arch/x86/um/elfcore.c:3:
   include/linux/pgtable.h: In function 'set_ptes':
>> include/linux/pgtable.h:209:34: error: invalid operands to binary + (have 'pte_t {aka struct <anonymous>}' and 'long unsigned int')
    #define pte_next_pfn(pte) ((pte) + (1UL << PFN_PTE_SHIFT))
                                     ^
   include/linux/pgtable.h:238:9: note: in expansion of macro 'pte_next_pfn'
      pte = pte_next_pfn(pte);
            ^~~~~~~~~~~~
   arch/x86/um/elfcore.c: At top level:
   arch/x86/um/elfcore.c:10:12: warning: no previous declaration for 'elf_core_extra_phdrs' [-Wmissing-declarations]
    Elf32_Half elf_core_extra_phdrs(struct coredump_params *cprm)
               ^~~~~~~~~~~~~~~~~~~~
   arch/x86/um/elfcore.c:15:5: warning: no previous declaration for 'elf_core_write_extra_phdrs' [-Wmissing-declarations]
    int elf_core_write_extra_phdrs(struct coredump_params *cprm, loff_t offset)
        ^~~~~~~~~~~~~~~~~~~~~~~~~~
   arch/x86/um/elfcore.c:42:5: warning: no previous declaration for 'elf_core_write_extra_data' [-Wmissing-declarations]
    int elf_core_write_extra_data(struct coredump_params *cprm)
        ^~~~~~~~~~~~~~~~~~~~~~~~~
   arch/x86/um/elfcore.c:63:8: warning: no previous declaration for 'elf_core_extra_data_size' [-Wmissing-declarations]
    size_t elf_core_extra_data_size(struct coredump_params *cprm)
           ^~~~~~~~~~~~~~~~~~~~~~~~
--
   In file included from include/linux/mm.h:29:0,
                    from include/linux/kallsyms.h:13,
                    from include/linux/ftrace.h:13,
                    from include/linux/kprobes.h:28,
                    from include/linux/kgdb.h:19,
                    from kernel/panic.c:15:
   include/linux/pgtable.h: In function 'set_ptes':
>> include/linux/pgtable.h:209:34: error: invalid operands to binary + (have 'pte_t {aka struct <anonymous>}' and 'long unsigned int')
    #define pte_next_pfn(pte) ((pte) + (1UL << PFN_PTE_SHIFT))
                                     ^
   include/linux/pgtable.h:238:9: note: in expansion of macro 'pte_next_pfn'
      pte = pte_next_pfn(pte);
            ^~~~~~~~~~~~
   kernel/panic.c: In function '__warn':
   kernel/panic.c:670:3: warning: function '__warn' might be a candidate for 'gnu_printf' format attribute [-Wsuggest-attribute=format]
      vprintk(args->fmt, args->args);
      ^~~~~~~
--
   In file included from include/linux/mm.h:29:0,
                    from include/linux/memblock.h:12,
                    from arch/um/kernel/mem.c:8:
   include/linux/pgtable.h: In function 'set_ptes':
>> include/linux/pgtable.h:209:34: error: invalid operands to binary + (have 'pte_t {aka struct <anonymous>}' and 'long unsigned int')
    #define pte_next_pfn(pte) ((pte) + (1UL << PFN_PTE_SHIFT))
                                     ^
   include/linux/pgtable.h:238:9: note: in expansion of macro 'pte_next_pfn'
      pte = pte_next_pfn(pte);
            ^~~~~~~~~~~~
   arch/um/kernel/mem.c: At top level:
   arch/um/kernel/mem.c:202:8: warning: no previous declaration for 'pgd_alloc' [-Wmissing-declarations]
    pgd_t *pgd_alloc(struct mm_struct *mm)
           ^~~~~~~~~
   arch/um/kernel/mem.c:215:7: warning: no previous declaration for 'uml_kmalloc' [-Wmissing-declarations]
    void *uml_kmalloc(int size, int flags)
          ^~~~~~~~~~~
--
   In file included from include/linux/mm.h:29:0,
                    from arch/um/kernel/process.c:12:
   include/linux/pgtable.h: In function 'set_ptes':
>> include/linux/pgtable.h:209:34: error: invalid operands to binary + (have 'pte_t {aka struct <anonymous>}' and 'long unsigned int')
    #define pte_next_pfn(pte) ((pte) + (1UL << PFN_PTE_SHIFT))
                                     ^
   include/linux/pgtable.h:238:9: note: in expansion of macro 'pte_next_pfn'
      pte = pte_next_pfn(pte);
            ^~~~~~~~~~~~
   arch/um/kernel/process.c: At top level:
   arch/um/kernel/process.c:51:5: warning: no previous declaration for 'pid_to_processor_id' [-Wmissing-declarations]
    int pid_to_processor_id(int pid)
        ^~~~~~~~~~~~~~~~~~~
   arch/um/kernel/process.c:87:7: warning: no previous declaration for '__switch_to' [-Wmissing-declarations]
    void *__switch_to(struct task_struct *from, struct task_struct *to)
          ^~~~~~~~~~~
   arch/um/kernel/process.c: In function 'new_thread_handler':
   arch/um/kernel/process.c:122:21: warning: variable 'n' set but not used [-Wunused-but-set-variable]
     int (*fn)(void *), n;
                        ^
   arch/um/kernel/process.c: At top level:
   arch/um/kernel/process.c:140:6: warning: no previous declaration for 'fork_handler' [-Wmissing-declarations]
    void fork_handler(void)
         ^~~~~~~~~~~~
   arch/um/kernel/process.c:217:6: warning: no previous declaration for 'arch_cpu_idle' [-Wmissing-declarations]
    void arch_cpu_idle(void)
         ^~~~~~~~~~~~~
   arch/um/kernel/process.c:253:5: warning: no previous declaration for 'copy_to_user_proc' [-Wmissing-declarations]
    int copy_to_user_proc(void __user *to, void *from, int size)
        ^~~~~~~~~~~~~~~~~
   arch/um/kernel/process.c:263:5: warning: no previous declaration for 'clear_user_proc' [-Wmissing-declarations]
    int clear_user_proc(void __user *buf, int size)
        ^~~~~~~~~~~~~~~
   arch/um/kernel/process.c:316:12: warning: no previous declaration for 'make_proc_sysemu' [-Wmissing-declarations]
    int __init make_proc_sysemu(void)
               ^~~~~~~~~~~~~~~~
   arch/um/kernel/process.c:356:15: warning: no previous declaration for 'arch_align_stack' [-Wmissing-declarations]
    unsigned long arch_align_stack(unsigned long sp)
                  ^~~~~~~~~~~~~~~~
--
   In file included from include/linux/mm.h:29:0,
                    from include/linux/oom.h:11,
                    from arch/um/kernel/reboot.c:11:
   include/linux/pgtable.h: In function 'set_ptes':
>> include/linux/pgtable.h:209:34: error: invalid operands to binary + (have 'pte_t {aka struct <anonymous>}' and 'long unsigned int')
    #define pte_next_pfn(pte) ((pte) + (1UL << PFN_PTE_SHIFT))
                                     ^
   include/linux/pgtable.h:238:9: note: in expansion of macro 'pte_next_pfn'
      pte = pte_next_pfn(pte);
            ^~~~~~~~~~~~
   arch/um/kernel/reboot.c: At top level:
   arch/um/kernel/reboot.c:45:6: warning: no previous declaration for 'machine_restart' [-Wmissing-declarations]
    void machine_restart(char * __unused)
         ^~~~~~~~~~~~~~~
   arch/um/kernel/reboot.c:51:6: warning: no previous declaration for 'machine_power_off' [-Wmissing-declarations]
    void machine_power_off(void)
         ^~~~~~~~~~~~~~~~~
   arch/um/kernel/reboot.c:57:6: warning: no previous declaration for 'machine_halt' [-Wmissing-declarations]
    void machine_halt(void)
         ^~~~~~~~~~~~
--
   In file included from include/linux/mm.h:29:0,
                    from arch/um/kernel/tlb.c:6:
   include/linux/pgtable.h: In function 'set_ptes':
>> include/linux/pgtable.h:209:34: error: invalid operands to binary + (have 'pte_t {aka struct <anonymous>}' and 'long unsigned int')
    #define pte_next_pfn(pte) ((pte) + (1UL << PFN_PTE_SHIFT))
                                     ^
   include/linux/pgtable.h:238:9: note: in expansion of macro 'pte_next_pfn'
      pte = pte_next_pfn(pte);
            ^~~~~~~~~~~~
   arch/um/kernel/tlb.c: At top level:
   arch/um/kernel/tlb.c:579:6: warning: no previous declaration for 'flush_tlb_mm_range' [-Wmissing-declarations]
    void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start,
         ^~~~~~~~~~~~~~~~~~
   arch/um/kernel/tlb.c:594:6: warning: no previous declaration for 'force_flush_all' [-Wmissing-declarations]
    void force_flush_all(void)
         ^~~~~~~~~~~~~~~
--
   In file included from include/linux/mm.h:29:0,
                    from arch/um/kernel/um_arch.c:9:
   include/linux/pgtable.h: In function 'set_ptes':
>> include/linux/pgtable.h:209:34: error: invalid operands to binary + (have 'pte_t {aka struct <anonymous>}' and 'long unsigned int')
    #define pte_next_pfn(pte) ((pte) + (1UL << PFN_PTE_SHIFT))
                                     ^
   include/linux/pgtable.h:238:9: note: in expansion of macro 'pte_next_pfn'
      pte = pte_next_pfn(pte);
            ^~~~~~~~~~~~
   arch/um/kernel/um_arch.c: At top level:
   arch/um/kernel/um_arch.c:408:19: warning: no previous declaration for 'read_initrd' [-Wmissing-declarations]
    int __init __weak read_initrd(void)
                      ^~~~~~~~~~~
   arch/um/kernel/um_arch.c:461:7: warning: no previous declaration for 'text_poke' [-Wmissing-declarations]
    void *text_poke(void *addr, const void *opcode, size_t len)
          ^~~~~~~~~
   arch/um/kernel/um_arch.c:473:6: warning: no previous declaration for 'text_poke_sync' [-Wmissing-declarations]
    void text_poke_sync(void)
         ^~~~~~~~~~~~~~
..


vim +209 include/linux/pgtable.h

   207	
   208	#ifndef pte_next_pfn
 > 209	#define pte_next_pfn(pte)	((pte) + (1UL << PFN_PTE_SHIFT))
   210	#endif
   211	

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

[PATCH 2/2] mm: Abstract moving to the next PFN

[Matthew Wilcox (Oracle)]

In order to fix the L1TF vulnerability, x86 can invert the PTE bits for
PROT_NONE VMAs, which means we cannot move from one PTE to the next by
adding 1 to the PFN field of the PTE.  Abstract advancing the PTE to
the next PFN through a pte_next_pfn() function/macro.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Fixes: bcc6cc832573 ("mm: add default definition of set_ptes()")
Reported-by: syzbot+55cc72f8cc3a549119df@syzkaller.appspotmail.com
---
 arch/x86/include/asm/pgtable.h |  8 ++++++++
 include/linux/pgtable.h        | 10 +++++++++-
 2 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index d6ad98ca1288..e02b179ec659 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -955,6 +955,14 @@ static inline int pte_same(pte_t a, pte_t b)
 	return a.pte == b.pte;
 }
 
+static inline pte_t pte_next_pfn(pte_t pte)
+{
+	if (__pte_needs_invert(pte_val(pte)))
+		return __pte(pte_val(pte) - (1UL << PFN_PTE_SHIFT));
+	return __pte(pte_val(pte) + (1UL << PFN_PTE_SHIFT));
+}
+#define pte_next_pfn	pte_next_pfn
+
 static inline int pte_present(pte_t a)
 {
 	return pte_flags(a) & (_PAGE_PRESENT | _PAGE_PROTNONE);
diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
index 1fba072b3dac..af7639c3b0a3 100644
--- a/include/linux/pgtable.h
+++ b/include/linux/pgtable.h
@@ -206,6 +206,14 @@ static inline int pmd_young(pmd_t pmd)
 #endif
 
 #ifndef set_ptes
+
+#ifndef pte_next_pfn
+static inline pte_t pte_next_pfn(pte_t pte)
+{
+	return __pte(pte_val(pte) + (1UL << PFN_PTE_SHIFT));
+}
+#endif
+
 /**
  * set_ptes - Map consecutive pages to a contiguous range of addresses.
  * @mm: Address space to map the pages into.
@@ -231,7 +239,7 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr,
 		if (--nr == 0)
 			break;
 		ptep++;
-		pte = __pte(pte_val(pte) + (1UL << PFN_PTE_SHIFT));
+		pte = pte_next_pfn(pte);
 	}
 	arch_leave_lazy_mmu_mode();
 }
-- 
2.40.1

Re: [PATCH 2/2] mm: Abstract moving to the next PFN

[Yin Fengwei]

On 9/20/23 12:09, Matthew Wilcox (Oracle) wrote:
> In order to fix the L1TF vulnerability, x86 can invert the PTE bits for
> PROT_NONE VMAs, which means we cannot move from one PTE to the next by
> adding 1 to the PFN field of the PTE.  Abstract advancing the PTE to
> the next PFN through a pte_next_pfn() function/macro.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> Fixes: bcc6cc832573 ("mm: add default definition of set_ptes()")
> Reported-by: syzbot+55cc72f8cc3a549119df@syzkaller.appspotmail.com
Reviewed-by: Yin Fengwei <fengwei.yin@intel.com>

Thanks a lot for taking care of this.


Regards
Yin, Fengwei

> ---
>  arch/x86/include/asm/pgtable.h |  8 ++++++++
>  include/linux/pgtable.h        | 10 +++++++++-
>  2 files changed, 17 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
> index d6ad98ca1288..e02b179ec659 100644
> --- a/arch/x86/include/asm/pgtable.h
> +++ b/arch/x86/include/asm/pgtable.h
> @@ -955,6 +955,14 @@ static inline int pte_same(pte_t a, pte_t b)
>  	return a.pte == b.pte;
>  }
>  
> +static inline pte_t pte_next_pfn(pte_t pte)
> +{
> +	if (__pte_needs_invert(pte_val(pte)))
> +		return __pte(pte_val(pte) - (1UL << PFN_PTE_SHIFT));
> +	return __pte(pte_val(pte) + (1UL << PFN_PTE_SHIFT));
> +}
> +#define pte_next_pfn	pte_next_pfn
> +
>  static inline int pte_present(pte_t a)
>  {
>  	return pte_flags(a) & (_PAGE_PRESENT | _PAGE_PROTNONE);
> diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
> index 1fba072b3dac..af7639c3b0a3 100644
> --- a/include/linux/pgtable.h
> +++ b/include/linux/pgtable.h
> @@ -206,6 +206,14 @@ static inline int pmd_young(pmd_t pmd)
>  #endif
>  
>  #ifndef set_ptes
> +
> +#ifndef pte_next_pfn
> +static inline pte_t pte_next_pfn(pte_t pte)
> +{
> +	return __pte(pte_val(pte) + (1UL << PFN_PTE_SHIFT));
> +}
> +#endif
> +
>  /**
>   * set_ptes - Map consecutive pages to a contiguous range of addresses.
>   * @mm: Address space to map the pages into.
> @@ -231,7 +239,7 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr,
>  		if (--nr == 0)
>  			break;
>  		ptep++;
> -		pte = __pte(pte_val(pte) + (1UL << PFN_PTE_SHIFT));
> +		pte = pte_next_pfn(pte);
>  	}
>  	arch_leave_lazy_mmu_mode();
>  }

Re: [PATCH 2/2] mm: Abstract moving to the next PFN

[Andrew Morton]

On Wed, 20 Sep 2023 05:09:58 +0100 "Matthew Wilcox (Oracle)" <willy@infradead.org> wrote:

> In order to fix the L1TF vulnerability, x86 can invert the PTE bits for
> PROT_NONE VMAs, which means we cannot move from one PTE to the next by
> adding 1 to the PFN field of the PTE.  Abstract advancing the PTE to
> the next PFN through a pte_next_pfn() function/macro.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> Fixes: bcc6cc832573 ("mm: add default definition of set_ptes()")
> Reported-by: syzbot+55cc72f8cc3a549119df@syzkaller.appspotmail.com

Is it just me, or is it a pain hunting down things via message IDs?

I tweaked the changelog thusly, pointing out that this fixes a BUG.


: In order to fix the L1TF vulnerability, x86 can invert the PTE bits for
: PROT_NONE VMAs, which means we cannot move from one PTE to the next by
: adding 1 to the PFN field of the PTE.  This results in the BUG reported at
: [1].
: 
: Abstract advancing the PTE to the next PFN through a pte_next_pfn()
: function/macro.
: 
: Link: https://lkml.kernel.org/r/20230920040958.866520-1-willy@infradead.org
: Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
: Fixes: bcc6cc832573 ("mm: add default definition of set_ptes()")
: Reported-by: syzbot+55cc72f8cc3a549119df@syzkaller.appspotmail.com
: Closes: https://lkml.kernel.org/r/000000000000d099fa0604f03351@google.com [1]

 

Re: [PATCH 1/2] mm: Report success more often from filemap_map_folio_range()

[Yin Fengwei]

On 9/20/23 11:53, Matthew Wilcox (Oracle) wrote:
> Even though we had successfully mapped the relevant page, we would
> rarely return success from filemap_map_folio_range().  That leads to
> falling back from the VMA lock path to the mmap_lock path, which is a
> speed & scalability issue.  Found by inspection.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> Fixes: 617c28ecab22 ("filemap: batch PTE mappings")
> ---
Reviewed-by: Yin Fengwei <fengwei.yin@intel.com>

Thanks a lot for taking care of this.

Regards
Yin, Fengwei



>  mm/filemap.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/filemap.c b/mm/filemap.c
> index 582f5317ff71..580d0b2b1a7c 100644
> --- a/mm/filemap.c
> +++ b/mm/filemap.c
> @@ -3506,7 +3506,7 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf,
>  		if (count) {
>  			set_pte_range(vmf, folio, page, count, addr);
>  			folio_ref_add(folio, count);
> -			if (in_range(vmf->address, addr, count))
> +			if (in_range(vmf->address, addr, count * PAGE_SIZE))
>  				ret = VM_FAULT_NOPAGE;
>  		}
>  
> @@ -3520,7 +3520,7 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf,
>  	if (count) {
>  		set_pte_range(vmf, folio, page, count, addr);
>  		folio_ref_add(folio, count);
> -		if (in_range(vmf->address, addr, count))
> +		if (in_range(vmf->address, addr, count * PAGE_SIZE))
>  			ret = VM_FAULT_NOPAGE;
>  	}
>