Linux-Next Archive on lore.kernel.org
 help / color / Atom feed
* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2014-01-07  6:00 Stephen Rothwell
  2014-01-07  6:34 ` Tang Chen
  0 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2014-01-07  6:00 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Lans Zhang, Yasuaki Ishimatsu,
	Lai Jiangshan, Tang Chen, Jiang Liu, Zhang Yanfei

[-- Attachment #1: Type: text/plain, Size: 1383 bytes --]

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in
arch/x86/mm/numa.c between commit f3d815cb854b ("x86/mm/numa: Fix 32-bit
kernel NUMA boot") from the tip tree and commit 1459be89954e ("x86: get
pg_data_t's memory from other node") from the akpm-current tree.

These appear to be two very similar solutions, I fixed it up (see below -
I (arbitrarily) chose to keep the actual allocation from the tip tree, but
the messages from the akpm-current tree) and can carry the fix as
necessary (no action is required).

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

diff --cc arch/x86/mm/numa.c
index c85da7bb6b60,f26b16f0d3e0..000000000000
--- a/arch/x86/mm/numa.c
+++ b/arch/x86/mm/numa.c
@@@ -211,11 -211,12 +211,12 @@@ static void __init setup_node_data(int 
  	 */
  	nd_pa = memblock_alloc_nid(nd_size, SMP_CACHE_BYTES, nid);
  	if (!nd_pa) {
+ 		pr_warn("Cannot find %zu bytes in node %d, so try other nodes",
+ 			nd_size, nid);
 -		nd_pa = memblock_alloc_nid(nd_size, SMP_CACHE_BYTES,
 -					   MAX_NUMNODES);
 +		nd_pa = __memblock_alloc_base(nd_size, SMP_CACHE_BYTES,
 +					      MEMBLOCK_ALLOC_ACCESSIBLE);
  		if (!nd_pa) {
- 			pr_err("Cannot find %zu bytes in node %d\n",
- 			       nd_size, nid);
+ 			pr_err("Cannot find %zu bytes in any node\n", nd_size);
  			return;
  		}
  	}

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2014-01-07  6:00 linux-next: manual merge of the akpm-current tree with the tip tree Stephen Rothwell
@ 2014-01-07  6:34 ` Tang Chen
  0 siblings, 0 replies; 87+ messages in thread
From: Tang Chen @ 2014-01-07  6:34 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra, linux-next, linux-kernel, Lans Zhang,
	Yasuaki Ishimatsu, Lai Jiangshan, Jiang Liu, Zhang Yanfei

On 01/07/2014 02:00 PM, Stephen Rothwell wrote:
> Hi Andrew,
>
> Today's linux-next merge of the akpm-current tree got a conflict in
> arch/x86/mm/numa.c between commit f3d815cb854b ("x86/mm/numa: Fix 32-bit
> kernel NUMA boot") from the tip tree and commit 1459be89954e ("x86: get
> pg_data_t's memory from other node") from the akpm-current tree.
>
> These appear to be two very similar solutions, I fixed it up (see below -
> I (arbitrarily) chose to keep the actual allocation from the tip tree, but
> the messages from the akpm-current tree) and can carry the fix as
> necessary (no action is required).
>

memblock_alloc_nid() and __memblock_alloc_base() will call
memblock_alloc_base_nid() in the end. So I think it is OK to me.

I will do some tests when they are merged.

Thanks.

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2019-10-31  5:43 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2019-10-31  5:43 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: Linux Next Mailing List, Linux Kernel Mailing List, Julien Grall

[-- Attachment #1: Type: text/plain, Size: 1640 bytes --]

Hi all,

Today's linux-next merge of the akpm-current tree got a conflict in:

  lib/ubsan.c

between commit:

  9a50dcaf0416 ("ubsan, x86: Annotate and allow __ubsan_handle_shift_out_of_bounds() in uaccess regions")

from the tip tree and commit:

  edbefc568464 ("lib/ubsan: don't serialize UBSAN report")

from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary. This
is now fixed as far as linux-next is concerned, but any non trivial
conflicts should be mentioned to your upstream maintainer when your tree
is submitted for merging.  You may also want to consider cooperating
with the maintainer of the conflicting tree to minimise any particularly
complex conflicts.

-- 
Cheers,
Stephen Rothwell

diff --cc lib/ubsan.c
index 0c4681118fcd,39d5952c4273..000000000000
--- a/lib/ubsan.c
+++ b/lib/ubsan.c
@@@ -374,12 -359,11 +359,12 @@@ void __ubsan_handle_shift_out_of_bounds
  	struct type_descriptor *lhs_type = data->lhs_type;
  	char rhs_str[VALUE_LENGTH];
  	char lhs_str[VALUE_LENGTH];
 +	unsigned long ua_flags = user_access_save();
  
  	if (suppress_report(&data->location))
 -		return;
 +		goto out;
  
- 	ubsan_prologue(&data->location, &flags);
+ 	ubsan_prologue(&data->location);
  
  	val_to_string(rhs_str, sizeof(rhs_str), rhs_type, rhs);
  	val_to_string(lhs_str, sizeof(lhs_str), lhs_type, lhs);
@@@ -402,9 -386,7 +387,9 @@@
  			lhs_str, rhs_str,
  			lhs_type->type_name);
  
- 	ubsan_epilogue(&flags);
+ 	ubsan_epilogue();
 +out:
 +	user_access_restore(ua_flags);
  }
  EXPORT_SYMBOL(__ubsan_handle_shift_out_of_bounds);
  

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2019-06-24 10:24 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2019-06-24 10:24 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: Linux Next Mailing List, Linux Kernel Mailing List, Waiman Long

[-- Attachment #1: Type: text/plain, Size: 799 bytes --]

Hi all,

Today's linux-next merge of the akpm-current tree got a conflict in:

  lib/debugobjects.c

between commit:

  d5f34153e526 ("debugobjects: Move printk out of db->lock critical sections")

from the tip tree and commit:

  8b6b497dfb11 ("lib/debugobjects.c: move printk out of db lock critical sections")

from the akpm-current tree.

I fixed it up (I reverted the akpm-current tree version) and can carry the
fix as necessary. This is now fixed as far as linux-next is concerned,
but any non trivial conflicts should be mentioned to your upstream
maintainer when your tree is submitted for merging.  You may also want
to consider cooperating with the maintainer of the conflicting tree to
minimise any particularly complex conflicts.

-- 
Cheers,
Stephen Rothwell

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2019-05-01 11:10 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2019-05-01 11:10 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: Linux Next Mailing List, Linux Kernel Mailing List,
	Rick Edgecombe, Roman Gushchin

[-- Attachment #1: Type: text/plain, Size: 4374 bytes --]

Hi all,

Today's linux-next merge of the akpm-current tree got a conflict in:

  mm/vmalloc.c

between commit:

  bade3b4bdcdb ("mm/vmalloc.c: refactor __vunmap() to avoid duplicated call to find_vm_area()")

from the tip tree and commit:

  868b104d7379 ("mm/vmalloc: Add flag for freeing of special permsissions")

from the akpm-current tree.

I fixed it up (I made an attempt ta a fix up - see below) and can carry
the fix as necessary. This is now fixed as far as linux-next is
concerned, but any non trivial conflicts should be mentioned to your
upstream maintainer when your tree is submitted for merging.  You may
also want to consider cooperating with the maintainer of the
conflicting tree to minimise any particularly complex conflicts.

-- 
Cheers,
Stephen Rothwell

diff --cc mm/vmalloc.c
index e5e9e1fcac01,4a91acce4b5f..000000000000
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@@ -1490,94 -2103,16 +2110,83 @@@ static struct vm_struct *__remove_vm_ar
   */
  struct vm_struct *remove_vm_area(const void *addr)
  {
+ 	struct vm_struct *vm = NULL;
  	struct vmap_area *va;
  
- 	might_sleep();
- 
  	va = find_vmap_area((unsigned long)addr);
- 	if (va && va->flags & VM_VM_AREA) {
- 		struct vm_struct *vm = va->vm;
- 
- 		spin_lock(&vmap_area_lock);
- 		va->vm = NULL;
- 		va->flags &= ~VM_VM_AREA;
- 		va->flags |= VM_LAZY_FREE;
- 		spin_unlock(&vmap_area_lock);
- 
- 		kasan_free_shadow(vm);
- 		free_unmap_vmap_area(va);
+ 	if (va && va->flags & VM_VM_AREA)
+ 		vm = __remove_vm_area(va);
  
- 		return vm;
- 	}
- 	return NULL;
+ 	return vm;
  }
  
 +static inline void set_area_direct_map(const struct vm_struct *area,
 +				       int (*set_direct_map)(struct page *page))
 +{
 +	int i;
 +
 +	for (i = 0; i < area->nr_pages; i++)
 +		if (page_address(area->pages[i]))
 +			set_direct_map(area->pages[i]);
 +}
 +
 +/* Handle removing and resetting vm mappings related to the vm_struct. */
- static void vm_remove_mappings(struct vm_struct *area, int deallocate_pages)
++static void vm_remove_mappings(struct vmap_area *va, int deallocate_pages)
 +{
++	struct vm_struct *area = va->vm;
 +	unsigned long addr = (unsigned long)area->addr;
 +	unsigned long start = ULONG_MAX, end = 0;
 +	int flush_reset = area->flags & VM_FLUSH_RESET_PERMS;
 +	int i;
 +
 +	/*
 +	 * The below block can be removed when all architectures that have
 +	 * direct map permissions also have set_direct_map_() implementations.
 +	 * This is concerned with resetting the direct map any an vm alias with
 +	 * execute permissions, without leaving a RW+X window.
 +	 */
 +	if (flush_reset && !IS_ENABLED(CONFIG_ARCH_HAS_SET_DIRECT_MAP)) {
 +		set_memory_nx(addr, area->nr_pages);
 +		set_memory_rw(addr, area->nr_pages);
 +	}
 +
- 	remove_vm_area(area->addr);
++	__remove_vm_area(va);
 +
 +	/* If this is not VM_FLUSH_RESET_PERMS memory, no need for the below. */
 +	if (!flush_reset)
 +		return;
 +
 +	/*
 +	 * If not deallocating pages, just do the flush of the VM area and
 +	 * return.
 +	 */
 +	if (!deallocate_pages) {
 +		vm_unmap_aliases();
 +		return;
 +	}
 +
 +	/*
 +	 * If execution gets here, flush the vm mapping and reset the direct
 +	 * map. Find the start and end range of the direct mappings to make sure
 +	 * the vm_unmap_aliases() flush includes the direct map.
 +	 */
 +	for (i = 0; i < area->nr_pages; i++) {
 +		if (page_address(area->pages[i])) {
 +			start = min(addr, start);
 +			end = max(addr, end);
 +		}
 +	}
 +
 +	/*
 +	 * Set direct map to something invalid so that it won't be cached if
 +	 * there are any accesses after the TLB flush, then flush the TLB and
 +	 * reset the direct map permissions to the default.
 +	 */
 +	set_area_direct_map(area, set_direct_map_invalid_noflush);
 +	_vm_unmap_aliases(start, end, 1);
 +	set_area_direct_map(area, set_direct_map_default_noflush);
 +}
 +
  static void __vunmap(const void *addr, int deallocate_pages)
  {
  	struct vm_struct *area;
@@@ -1599,8 -2136,7 +2210,8 @@@
  	debug_check_no_locks_freed(area->addr, get_vm_area_size(area));
  	debug_check_no_obj_freed(area->addr, get_vm_area_size(area));
  
- 	vm_remove_mappings(area, deallocate_pages);
 -	__remove_vm_area(va);
++	vm_remove_mappings(va, deallocate_pages);
 +
  	if (deallocate_pages) {
  		int i;
  

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2019-01-31  4:31 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2019-01-31  4:31 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: Linux Next Mailing List, Linux Kernel Mailing List, Aneesh Kumar K.V

[-- Attachment #1: Type: text/plain, Size: 1561 bytes --]

Hi all,

Today's linux-next merge of the akpm-current tree got a conflict in:

  include/linux/sched.h

between commit:

  15917dc02841 ("sched: Remove stale PF_MUTEX_TESTER bit")

from the tip tree and commit:

  ca299cb98649 ("mm/cma: add PF flag to force non cma alloc")

from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary. This
is now fixed as far as linux-next is concerned, but any non trivial
conflicts should be mentioned to your upstream maintainer when your tree
is submitted for merging.  You may also want to consider cooperating
with the maintainer of the conflicting tree to minimise any particularly
complex conflicts.

-- 
Cheers,
Stephen Rothwell

diff --cc include/linux/sched.h
index bb68abafac29,1ef3995b7564..000000000000
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@@ -1409,6 -1423,8 +1423,7 @@@ extern struct pid *cad_pid
  #define PF_UMH			0x02000000	/* I'm an Usermodehelper process */
  #define PF_NO_SETAFFINITY	0x04000000	/* Userland is not allowed to meddle with cpus_allowed */
  #define PF_MCE_EARLY		0x08000000      /* Early kill for mce process policy */
+ #define PF_MEMALLOC_NOCMA	0x10000000	/* All allocation request will have _GFP_MOVABLE cleared */
 -#define PF_MUTEX_TESTER		0x20000000	/* Thread belongs to the rt mutex tester */
  #define PF_FREEZER_SKIP		0x40000000	/* Freezer should not count it as freezable */
  #define PF_SUSPEND_TASK		0x80000000      /* This thread called freeze_processes() and should not be frozen */
  

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2018-08-20  4:32 Stephen Rothwell
@ 2018-08-20 19:52 ` Andrew Morton
  0 siblings, 0 replies; 87+ messages in thread
From: Andrew Morton @ 2018-08-20 19:52 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Thomas Gleixner, Ingo Molnar, H. Peter Anvin, Peter Zijlstra,
	Linux-Next Mailing List, Linux Kernel Mailing List,
	Adrian Hunter, Arnaldo Carvalho de Melo, James Morse,
	Omar Sandoval

On Mon, 20 Aug 2018 14:32:22 +1000 Stephen Rothwell <sfr@canb.auug.org.au> wrote:

> Today's linux-next merge of the akpm-current tree got conflicts in:
> 
>   fs/proc/kcore.c
>   include/linux/kcore.h
> 
> between commit:
> 
>   6855dc41b246 ("x86: Add entry trampolines to kcore")
> 
> from the tip tree and commits:
> 
>   4eb27c275abf ("fs/proc/kcore.c: use __pa_symbol() for KCORE_TEXT list entries")
>   ea551910d3f4 ("proc/kcore: clean up ELF header generation")
>   537412a2958f ("proc/kcore: don't grab lock for kclist_add()")
> 
> from the akpm-current tree.
> 
> I fixed it up (see below) and can carry the fix as necessary. This
> is now fixed as far as linux-next is concerned, but any non trivial
> conflicts should be mentioned to your upstream maintainer when your tree
> is submitted for merging.  You may also want to consider cooperating
> with the maintainer of the conflicting tree to minimise any particularly
> complex conflicts.

Yup.

What's happening here?  A two month old patch turns up in linux-next in the
middle of the merge window, in the "perf/urgent" branch.  That's a strange
branch for a June 6 patch!

Is it intended that this material be merged into 4.19-rc1?

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2018-08-20  4:32 Stephen Rothwell
  2018-08-20 19:52 ` Andrew Morton
  0 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2018-08-20  4:32 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: Linux-Next Mailing List, Linux Kernel Mailing List,
	Adrian Hunter, Arnaldo Carvalho de Melo, James Morse,
	Omar Sandoval

[-- Attachment #1: Type: text/plain, Size: 7173 bytes --]

Hi Andrew,

Today's linux-next merge of the akpm-current tree got conflicts in:

  fs/proc/kcore.c
  include/linux/kcore.h

between commit:

  6855dc41b246 ("x86: Add entry trampolines to kcore")

from the tip tree and commits:

  4eb27c275abf ("fs/proc/kcore.c: use __pa_symbol() for KCORE_TEXT list entries")
  ea551910d3f4 ("proc/kcore: clean up ELF header generation")
  537412a2958f ("proc/kcore: don't grab lock for kclist_add()")

from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary. This
is now fixed as far as linux-next is concerned, but any non trivial
conflicts should be mentioned to your upstream maintainer when your tree
is submitted for merging.  You may also want to consider cooperating
with the maintainer of the conflicting tree to minimise any particularly
complex conflicts.

-- 
Cheers,
Stephen Rothwell

diff --cc fs/proc/kcore.c
index 00282f134336,80464432dfe6..000000000000
--- a/fs/proc/kcore.c
+++ b/fs/proc/kcore.c
@@@ -448,53 -291,148 +291,151 @@@ static ssize_
  read_kcore(struct file *file, char __user *buffer, size_t buflen, loff_t *fpos)
  {
  	char *buf = file->private_data;
- 	ssize_t acc = 0;
- 	size_t size, tsz;
- 	size_t elf_buflen;
+ 	size_t phdrs_offset, notes_offset, data_offset;
+ 	size_t phdrs_len, notes_len;
+ 	struct kcore_list *m;
+ 	size_t tsz;
  	int nphdr;
  	unsigned long start;
+ 	size_t orig_buflen = buflen;
+ 	int ret = 0;
  
- 	read_lock(&kclist_lock);
- 	size = get_kcore_size(&nphdr, &elf_buflen);
+ 	down_read(&kclist_lock);
+ 
+ 	get_kcore_size(&nphdr, &phdrs_len, &notes_len, &data_offset);
+ 	phdrs_offset = sizeof(struct elfhdr);
+ 	notes_offset = phdrs_offset + phdrs_len;
+ 
+ 	/* ELF file header. */
+ 	if (buflen && *fpos < sizeof(struct elfhdr)) {
+ 		struct elfhdr ehdr = {
+ 			.e_ident = {
+ 				[EI_MAG0] = ELFMAG0,
+ 				[EI_MAG1] = ELFMAG1,
+ 				[EI_MAG2] = ELFMAG2,
+ 				[EI_MAG3] = ELFMAG3,
+ 				[EI_CLASS] = ELF_CLASS,
+ 				[EI_DATA] = ELF_DATA,
+ 				[EI_VERSION] = EV_CURRENT,
+ 				[EI_OSABI] = ELF_OSABI,
+ 			},
+ 			.e_type = ET_CORE,
+ 			.e_machine = ELF_ARCH,
+ 			.e_version = EV_CURRENT,
+ 			.e_phoff = sizeof(struct elfhdr),
+ 			.e_flags = ELF_CORE_EFLAGS,
+ 			.e_ehsize = sizeof(struct elfhdr),
+ 			.e_phentsize = sizeof(struct elf_phdr),
+ 			.e_phnum = nphdr,
+ 		};
+ 
+ 		tsz = min_t(size_t, buflen, sizeof(struct elfhdr) - *fpos);
+ 		if (copy_to_user(buffer, (char *)&ehdr + *fpos, tsz)) {
+ 			ret = -EFAULT;
+ 			goto out;
+ 		}
  
- 	if (buflen == 0 || *fpos >= size) {
- 		read_unlock(&kclist_lock);
- 		return 0;
+ 		buffer += tsz;
+ 		buflen -= tsz;
+ 		*fpos += tsz;
  	}
  
- 	/* trim buflen to not go beyond EOF */
- 	if (buflen > size - *fpos)
- 		buflen = size - *fpos;
- 
- 	/* construct an ELF core header if we'll need some of it */
- 	if (*fpos < elf_buflen) {
- 		char * elf_buf;
- 
- 		tsz = elf_buflen - *fpos;
- 		if (buflen < tsz)
- 			tsz = buflen;
- 		elf_buf = kzalloc(elf_buflen, GFP_ATOMIC);
- 		if (!elf_buf) {
- 			read_unlock(&kclist_lock);
- 			return -ENOMEM;
+ 	/* ELF program headers. */
+ 	if (buflen && *fpos < phdrs_offset + phdrs_len) {
+ 		struct elf_phdr *phdrs, *phdr;
+ 
+ 		phdrs = kzalloc(phdrs_len, GFP_KERNEL);
+ 		if (!phdrs) {
+ 			ret = -ENOMEM;
+ 			goto out;
  		}
- 		elf_kcore_store_hdr(elf_buf, nphdr, elf_buflen);
- 		read_unlock(&kclist_lock);
- 		if (copy_to_user(buffer, elf_buf + *fpos, tsz)) {
- 			kfree(elf_buf);
- 			return -EFAULT;
+ 
+ 		phdrs[0].p_type = PT_NOTE;
+ 		phdrs[0].p_offset = notes_offset;
+ 		phdrs[0].p_filesz = notes_len;
+ 
+ 		phdr = &phdrs[1];
+ 		list_for_each_entry(m, &kclist_head, list) {
+ 			phdr->p_type = PT_LOAD;
+ 			phdr->p_flags = PF_R | PF_W | PF_X;
+ 			phdr->p_offset = kc_vaddr_to_offset(m->addr) + data_offset;
 -			phdr->p_vaddr = (size_t)m->addr;
 -			if (m->type == KCORE_RAM)
++			if (m->type == KCORE_REMAP)
++				phdr->p_vaddr	= (size_t)m->vaddr;
++			else
++				phdr->p_vaddr	= (size_t)m->addr;
++			if (m->type == KCORE_RAM || m->type == KCORE_REMAP)
+ 				phdr->p_paddr = __pa(m->addr);
+ 			else if (m->type == KCORE_TEXT)
+ 				phdr->p_paddr = __pa_symbol(m->addr);
+ 			else
+ 				phdr->p_paddr = (elf_addr_t)-1;
+ 			phdr->p_filesz = phdr->p_memsz = m->size;
+ 			phdr->p_align = PAGE_SIZE;
+ 			phdr++;
  		}
- 		kfree(elf_buf);
+ 
+ 		tsz = min_t(size_t, buflen, phdrs_offset + phdrs_len - *fpos);
+ 		if (copy_to_user(buffer, (char *)phdrs + *fpos - phdrs_offset,
+ 				 tsz)) {
+ 			kfree(phdrs);
+ 			ret = -EFAULT;
+ 			goto out;
+ 		}
+ 		kfree(phdrs);
+ 
+ 		buffer += tsz;
  		buflen -= tsz;
  		*fpos += tsz;
- 		buffer += tsz;
- 		acc += tsz;
+ 	}
+ 
+ 	/* ELF note segment. */
+ 	if (buflen && *fpos < notes_offset + notes_len) {
+ 		struct elf_prstatus prstatus = {};
+ 		struct elf_prpsinfo prpsinfo = {
+ 			.pr_sname = 'R',
+ 			.pr_fname = "vmlinux",
+ 		};
+ 		char *notes;
+ 		size_t i = 0;
+ 
+ 		strlcpy(prpsinfo.pr_psargs, saved_command_line,
+ 			sizeof(prpsinfo.pr_psargs));
+ 
+ 		notes = kzalloc(notes_len, GFP_KERNEL);
+ 		if (!notes) {
+ 			ret = -ENOMEM;
+ 			goto out;
+ 		}
+ 
+ 		append_kcore_note(notes, &i, CORE_STR, NT_PRSTATUS, &prstatus,
+ 				  sizeof(prstatus));
+ 		append_kcore_note(notes, &i, CORE_STR, NT_PRPSINFO, &prpsinfo,
+ 				  sizeof(prpsinfo));
+ 		append_kcore_note(notes, &i, CORE_STR, NT_TASKSTRUCT, current,
+ 				  arch_task_struct_size);
+ 		/*
+ 		 * vmcoreinfo_size is mostly constant after init time, but it
+ 		 * can be changed by crash_save_vmcoreinfo(). Racing here with a
+ 		 * panic on another CPU before the machine goes down is insanely
+ 		 * unlikely, but it's better to not leave potential buffer
+ 		 * overflows lying around, regardless.
+ 		 */
+ 		append_kcore_note(notes, &i, VMCOREINFO_NOTE_NAME, 0,
+ 				  vmcoreinfo_data,
+ 				  min(vmcoreinfo_size, notes_len - i));
+ 
+ 		tsz = min_t(size_t, buflen, notes_offset + notes_len - *fpos);
+ 		if (copy_to_user(buffer, notes + *fpos - notes_offset, tsz)) {
+ 			kfree(notes);
+ 			ret = -EFAULT;
+ 			goto out;
+ 		}
+ 		kfree(notes);
  
- 		/* leave now if filled buffer already */
- 		if (buflen == 0)
- 			return acc;
- 	} else
- 		read_unlock(&kclist_lock);
+ 		buffer += tsz;
+ 		buflen -= tsz;
+ 		*fpos += tsz;
+ 	}
  
  	/*
  	 * Check to see if our file offset matches with any of
diff --cc include/linux/kcore.h
index bc088ef96358,c20f296438fb..000000000000
--- a/include/linux/kcore.h
+++ b/include/linux/kcore.h
@@@ -37,13 -35,7 +37,13 @@@ struct vmcoredd_node 
  };
  
  #ifdef CONFIG_PROC_KCORE
- extern void kclist_add(struct kcore_list *, void *, size_t, int type);
+ void __init kclist_add(struct kcore_list *, void *, size_t, int type);
 +static inline
 +void kclist_add_remap(struct kcore_list *m, void *addr, void *vaddr, size_t sz)
 +{
 +	m->vaddr = (unsigned long)vaddr;
 +	kclist_add(m, addr, sz, KCORE_REMAP);
 +}
  #else
  static inline
  void kclist_add(struct kcore_list *new, void *addr, size_t size, int type)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2018-03-23  5:59 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2018-03-23  5:59 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: Linux-Next Mailing List, Linux Kernel Mailing List, Gang He

[-- Attachment #1: Type: text/plain, Size: 816 bytes --]

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in:

  fs/ocfs2/filecheck.c

between commit:

  e24e960c7fe2 ("sched/wait, fs/ocfs2: Convert wait_on_atomic_t() usage to the new wait_var_event() API")

from the tip tree and commit:

  5a5b76d17dc4 ("ocfs2: add kobject for online file check")

from the akpm-current tree.

I fixed it up (the latter removed the code updated by the former) and
can carry the fix as necessary. This is now fixed as far as linux-next
is concerned, but any non trivial conflicts should be mentioned to your
upstream maintainer when your tree is submitted for merging.  You may
also want to consider cooperating with the maintainer of the conflicting
tree to minimise any particularly complex conflicts.

-- 
Cheers,
Stephen Rothwell

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2017-12-18  5:04 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2017-12-18  5:04 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: Linux-Next Mailing List, Linux Kernel Mailing List

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in:

  kernel/fork.c

between commit:

  5e28fd0b5fdb ("arch: Allow arch_dup_mmap() to fail")

from the tip tree and commit:

  120bd8608675 ("include/linux/sched/mm.h: uninline mmdrop_async(), etc")

from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary. This
is now fixed as far as linux-next is concerned, but any non trivial
conflicts should be mentioned to your upstream maintainer when your tree
is submitted for merging.  You may also want to consider cooperating
with the maintainer of the conflicting tree to minimise any particularly
complex conflicts.

-- 
Cheers,
Stephen Rothwell

diff --cc kernel/fork.c
index bed0eaf7233f,7fccd819866f..000000000000
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@@ -391,6 -391,241 +392,240 @@@ void free_task(struct task_struct *tsk
  }
  EXPORT_SYMBOL(free_task);
  
+ #ifdef CONFIG_MMU
+ static __latent_entropy int dup_mmap(struct mm_struct *mm,
+ 					struct mm_struct *oldmm)
+ {
+ 	struct vm_area_struct *mpnt, *tmp, *prev, **pprev;
+ 	struct rb_node **rb_link, *rb_parent;
+ 	int retval;
+ 	unsigned long charge;
+ 	LIST_HEAD(uf);
+ 
+ 	uprobe_start_dup_mmap();
+ 	if (down_write_killable(&oldmm->mmap_sem)) {
+ 		retval = -EINTR;
+ 		goto fail_uprobe_end;
+ 	}
+ 	flush_cache_dup_mm(oldmm);
+ 	uprobe_dup_mmap(oldmm, mm);
+ 	/*
+ 	 * Not linked in yet - no deadlock potential:
+ 	 */
+ 	down_write_nested(&mm->mmap_sem, SINGLE_DEPTH_NESTING);
+ 
+ 	/* No ordering required: file already has been exposed. */
+ 	RCU_INIT_POINTER(mm->exe_file, get_mm_exe_file(oldmm));
+ 
+ 	mm->total_vm = oldmm->total_vm;
+ 	mm->data_vm = oldmm->data_vm;
+ 	mm->exec_vm = oldmm->exec_vm;
+ 	mm->stack_vm = oldmm->stack_vm;
+ 
+ 	rb_link = &mm->mm_rb.rb_node;
+ 	rb_parent = NULL;
+ 	pprev = &mm->mmap;
+ 	retval = ksm_fork(mm, oldmm);
+ 	if (retval)
+ 		goto out;
+ 	retval = khugepaged_fork(mm, oldmm);
+ 	if (retval)
+ 		goto out;
+ 
+ 	prev = NULL;
+ 	for (mpnt = oldmm->mmap; mpnt; mpnt = mpnt->vm_next) {
+ 		struct file *file;
+ 
+ 		if (mpnt->vm_flags & VM_DONTCOPY) {
+ 			vm_stat_account(mm, mpnt->vm_flags, -vma_pages(mpnt));
+ 			continue;
+ 		}
+ 		charge = 0;
+ 		if (mpnt->vm_flags & VM_ACCOUNT) {
+ 			unsigned long len = vma_pages(mpnt);
+ 
+ 			if (security_vm_enough_memory_mm(oldmm, len)) /* sic */
+ 				goto fail_nomem;
+ 			charge = len;
+ 		}
+ 		tmp = kmem_cache_alloc(vm_area_cachep, GFP_KERNEL);
+ 		if (!tmp)
+ 			goto fail_nomem;
+ 		*tmp = *mpnt;
+ 		INIT_LIST_HEAD(&tmp->anon_vma_chain);
+ 		retval = vma_dup_policy(mpnt, tmp);
+ 		if (retval)
+ 			goto fail_nomem_policy;
+ 		tmp->vm_mm = mm;
+ 		retval = dup_userfaultfd(tmp, &uf);
+ 		if (retval)
+ 			goto fail_nomem_anon_vma_fork;
+ 		if (tmp->vm_flags & VM_WIPEONFORK) {
+ 			/* VM_WIPEONFORK gets a clean slate in the child. */
+ 			tmp->anon_vma = NULL;
+ 			if (anon_vma_prepare(tmp))
+ 				goto fail_nomem_anon_vma_fork;
+ 		} else if (anon_vma_fork(tmp, mpnt))
+ 			goto fail_nomem_anon_vma_fork;
+ 		tmp->vm_flags &= ~(VM_LOCKED | VM_LOCKONFAULT);
+ 		tmp->vm_next = tmp->vm_prev = NULL;
+ 		file = tmp->vm_file;
+ 		if (file) {
+ 			struct inode *inode = file_inode(file);
+ 			struct address_space *mapping = file->f_mapping;
+ 
+ 			get_file(file);
+ 			if (tmp->vm_flags & VM_DENYWRITE)
+ 				atomic_dec(&inode->i_writecount);
+ 			i_mmap_lock_write(mapping);
+ 			if (tmp->vm_flags & VM_SHARED)
+ 				atomic_inc(&mapping->i_mmap_writable);
+ 			flush_dcache_mmap_lock(mapping);
+ 			/* insert tmp into the share list, just after mpnt */
+ 			vma_interval_tree_insert_after(tmp, mpnt,
+ 					&mapping->i_mmap);
+ 			flush_dcache_mmap_unlock(mapping);
+ 			i_mmap_unlock_write(mapping);
+ 		}
+ 
+ 		/*
+ 		 * Clear hugetlb-related page reserves for children. This only
+ 		 * affects MAP_PRIVATE mappings. Faults generated by the child
+ 		 * are not guaranteed to succeed, even if read-only
+ 		 */
+ 		if (is_vm_hugetlb_page(tmp))
+ 			reset_vma_resv_huge_pages(tmp);
+ 
+ 		/*
+ 		 * Link in the new vma and copy the page table entries.
+ 		 */
+ 		*pprev = tmp;
+ 		pprev = &tmp->vm_next;
+ 		tmp->vm_prev = prev;
+ 		prev = tmp;
+ 
+ 		__vma_link_rb(mm, tmp, rb_link, rb_parent);
+ 		rb_link = &tmp->vm_rb.rb_right;
+ 		rb_parent = &tmp->vm_rb;
+ 
+ 		mm->map_count++;
+ 		if (!(tmp->vm_flags & VM_WIPEONFORK))
+ 			retval = copy_page_range(mm, oldmm, mpnt);
+ 
+ 		if (tmp->vm_ops && tmp->vm_ops->open)
+ 			tmp->vm_ops->open(tmp);
+ 
+ 		if (retval)
+ 			goto out;
+ 	}
+ 	/* a new mm has just been created */
 -	arch_dup_mmap(oldmm, mm);
 -	retval = 0;
++	retval = arch_dup_mmap(oldmm, mm);
+ out:
+ 	up_write(&mm->mmap_sem);
+ 	flush_tlb_mm(oldmm);
+ 	up_write(&oldmm->mmap_sem);
+ 	dup_userfaultfd_complete(&uf);
+ fail_uprobe_end:
+ 	uprobe_end_dup_mmap();
+ 	return retval;
+ fail_nomem_anon_vma_fork:
+ 	mpol_put(vma_policy(tmp));
+ fail_nomem_policy:
+ 	kmem_cache_free(vm_area_cachep, tmp);
+ fail_nomem:
+ 	retval = -ENOMEM;
+ 	vm_unacct_memory(charge);
+ 	goto out;
+ }
+ 
+ static inline int mm_alloc_pgd(struct mm_struct *mm)
+ {
+ 	mm->pgd = pgd_alloc(mm);
+ 	if (unlikely(!mm->pgd))
+ 		return -ENOMEM;
+ 	return 0;
+ }
+ 
+ static inline void mm_free_pgd(struct mm_struct *mm)
+ {
+ 	pgd_free(mm, mm->pgd);
+ }
+ #else
+ static int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)
+ {
+ 	down_write(&oldmm->mmap_sem);
+ 	RCU_INIT_POINTER(mm->exe_file, get_mm_exe_file(oldmm));
+ 	up_write(&oldmm->mmap_sem);
+ 	return 0;
+ }
+ #define mm_alloc_pgd(mm)	(0)
+ #define mm_free_pgd(mm)
+ #endif /* CONFIG_MMU */
+ 
+ static void check_mm(struct mm_struct *mm)
+ {
+ 	int i;
+ 
+ 	for (i = 0; i < NR_MM_COUNTERS; i++) {
+ 		long x = atomic_long_read(&mm->rss_stat.count[i]);
+ 
+ 		if (unlikely(x))
+ 			printk(KERN_ALERT "BUG: Bad rss-counter state "
+ 					  "mm:%p idx:%d val:%ld\n", mm, i, x);
+ 	}
+ 
+ 	if (mm_pgtables_bytes(mm))
+ 		pr_alert("BUG: non-zero pgtables_bytes on freeing mm: %ld\n",
+ 				mm_pgtables_bytes(mm));
+ 
+ #if defined(CONFIG_TRANSPARENT_HUGEPAGE) && !USE_SPLIT_PMD_PTLOCKS
+ 	VM_BUG_ON_MM(mm->pmd_huge_pte, mm);
+ #endif
+ }
+ 
+ #define allocate_mm()	(kmem_cache_alloc(mm_cachep, GFP_KERNEL))
+ #define free_mm(mm)	(kmem_cache_free(mm_cachep, (mm)))
+ 
+ /*
+  * Called when the last reference to the mm
+  * is dropped: either by a lazy thread or by
+  * mmput. Free the page directory and the mm.
+  */
+ static void __mmdrop(struct mm_struct *mm)
+ {
+ 	BUG_ON(mm == &init_mm);
+ 	mm_free_pgd(mm);
+ 	destroy_context(mm);
+ 	hmm_mm_destroy(mm);
+ 	mmu_notifier_mm_destroy(mm);
+ 	check_mm(mm);
+ 	put_user_ns(mm->user_ns);
+ 	free_mm(mm);
+ }
+ 
+ void mmdrop(struct mm_struct *mm)
+ {
+ 	if (unlikely(atomic_dec_and_test(&mm->mm_count)))
+ 		__mmdrop(mm);
+ }
+ EXPORT_SYMBOL_GPL(mmdrop);
+ 
+ static void mmdrop_async_fn(struct work_struct *work)
+ {
+ 	struct mm_struct *mm;
+ 
+ 	mm = container_of(work, struct mm_struct, async_put_work);
+ 	__mmdrop(mm);
+ }
+ 
+ static void mmdrop_async(struct mm_struct *mm)
+ {
+ 	if (unlikely(atomic_dec_and_test(&mm->mm_count))) {
+ 		INIT_WORK(&mm->async_put_work, mmdrop_async_fn);
+ 		schedule_work(&mm->async_put_work);
+ 	}
+ }
+ 
  static inline void free_signal_struct(struct signal_struct *sig)
  {
  	taskstats_tgid_free(sig);

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2017-11-10  4:33 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2017-11-10  4:33 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: Linux-Next Mailing List, Linux Kernel Mailing List, Sasha Levin,
	Frederic Weisbecker

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in:

  kernel/softirq.c

between commit:

  f71b74bca637 ("irq/softirqs: Use lockdep to assert IRQs are disabled/enabled")

from the tip tree and commit:

  275f9389fa4e ("kmemcheck: rip it out")

from the akpm-current tree.

I fixed it up (the latter removed code modified by the former) and can
carry the fix as necessary. This is now fixed as far as linux-next is
concerned, but any non trivial conflicts should be mentioned to your
upstream maintainer when your tree is submitted for merging.  You may
also want to consider cooperating with the maintainer of the conflicting
tree to minimise any particularly complex conflicts.

-- 
Cheers,
Stephen Rothwell

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2017-11-02  7:19 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2017-11-02  7:19 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: Linux-Next Mailing List, Linux Kernel Mailing List,
	Pavel Tatashin, Andrey Ryabinin, Kirill A. Shutemov

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in:

  arch/x86/mm/kasan_init_64.c

between commit:

  12a8cc7fcf54 ("x86/kasan: Use the same shadow offset for 4- and 5-level paging")

from the tip tree and commit:

  3af83426c380 ("x86/kasan: add and use kasan_map_populate()")

from the akpm-current tree.

I fixed it up (hopefully - see below) and can carry the fix as
necessary. This is now fixed as far as linux-next is concerned, but any
non trivial conflicts should be mentioned to your upstream maintainer
when your tree is submitted for merging.  You may also want to consider
cooperating with the maintainer of the conflicting tree to minimise any
particularly complex conflicts.

-- 
Cheers,
Stephen Rothwell

diff --cc arch/x86/mm/kasan_init_64.c
index fe5760db7b19,9778fec8a5dc..000000000000
--- a/arch/x86/mm/kasan_init_64.c
+++ b/arch/x86/mm/kasan_init_64.c
@@@ -15,8 -15,73 +15,75 @@@
  
  extern struct range pfn_mapped[E820_MAX_ENTRIES];
  
 +static p4d_t tmp_p4d_table[PTRS_PER_P4D] __initdata __aligned(PAGE_SIZE);
 +
+ /* Creates mappings for kasan during early boot. The mapped memory is zeroed */
+ static int __meminit kasan_map_populate(unsigned long start, unsigned long end,
+ 					int node)
+ {
+ 	unsigned long addr, pfn, next;
+ 	unsigned long long size;
+ 	pgd_t *pgd;
+ 	p4d_t *p4d;
+ 	pud_t *pud;
+ 	pmd_t *pmd;
+ 	pte_t *pte;
+ 	int ret;
+ 
+ 	ret = vmemmap_populate(start, end, node);
+ 	/*
+ 	 * We might have partially populated memory, so check for no entries,
+ 	 * and zero only those that actually exist.
+ 	 */
+ 	for (addr = start; addr < end; addr = next) {
+ 		pgd = pgd_offset_k(addr);
+ 		if (pgd_none(*pgd)) {
+ 			next = pgd_addr_end(addr, end);
+ 			continue;
+ 		}
+ 
+ 		p4d = p4d_offset(pgd, addr);
+ 		if (p4d_none(*p4d)) {
+ 			next = p4d_addr_end(addr, end);
+ 			continue;
+ 		}
+ 
+ 		pud = pud_offset(p4d, addr);
+ 		if (pud_none(*pud)) {
+ 			next = pud_addr_end(addr, end);
+ 			continue;
+ 		}
+ 		if (pud_large(*pud)) {
+ 			/* This is PUD size page */
+ 			next = pud_addr_end(addr, end);
+ 			size = PUD_SIZE;
+ 			pfn = pud_pfn(*pud);
+ 		} else {
+ 			pmd = pmd_offset(pud, addr);
+ 			if (pmd_none(*pmd)) {
+ 				next = pmd_addr_end(addr, end);
+ 				continue;
+ 			}
+ 			if (pmd_large(*pmd)) {
+ 				/* This is PMD size page */
+ 				next = pmd_addr_end(addr, end);
+ 				size = PMD_SIZE;
+ 				pfn = pmd_pfn(*pmd);
+ 			} else {
+ 				pte = pte_offset_kernel(pmd, addr);
+ 				next = addr + PAGE_SIZE;
+ 				if (pte_none(*pte))
+ 					continue;
+ 				/* This is base size page */
+ 				size = PAGE_SIZE;
+ 				pfn = pte_pfn(*pte);
+ 			}
+ 		}
+ 		memset(phys_to_virt(PFN_PHYS(pfn)), 0, size);
+ 	}
+ 	return ret;
+ }
+ 
  static int __init map_range(struct range *range)
  {
  	unsigned long start;

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-22  6:57 Stephen Rothwell
@ 2017-08-23  6:39 ` Vlastimil Babka
  0 siblings, 0 replies; 87+ messages in thread
From: Vlastimil Babka @ 2017-08-23  6:39 UTC (permalink / raw)
  To: Stephen Rothwell, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Peter Zijlstra
  Cc: Linux-Next Mailing List, Linux Kernel Mailing List, Waiman Long

On 08/22/2017 08:57 AM, Stephen Rothwell wrote:
> Hi Andrew,

Hi,

> Today's linux-next merge of the akpm-current tree got a conflict in:
> 
>   init/main.c
> 
> between commit:
> 
>   caba4cbbd27d ("debugobjects: Make kmemleak ignore debug objects")
> 
> from the tip tree and commit:
> 
>   50a7dc046b58 ("mm, page_ext: move page_ext_init() after page_alloc_init_late()")

This patch can be also dropped from mmotm. It was a RFC and review
suggested a different approach which I didn't get to try yet. (The other
patches in the series should be fine to stay in any case).

> from the akpm-current tree.
> 
> I fixed it up (see below) and can carry the fix as necessary. This
> is now fixed as far as linux-next is concerned, but any non trivial
> conflicts should be mentioned to your upstream maintainer when your tree
> is submitted for merging.  You may also want to consider cooperating
> with the maintainer of the conflicting tree to minimise any particularly
> complex conflicts.
> 

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2017-08-22  6:57 Stephen Rothwell
  2017-08-23  6:39 ` Vlastimil Babka
  0 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2017-08-22  6:57 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: Linux-Next Mailing List, Linux Kernel Mailing List,
	Vlastimil Babka, Waiman Long

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in:

  init/main.c

between commit:

  caba4cbbd27d ("debugobjects: Make kmemleak ignore debug objects")

from the tip tree and commit:

  50a7dc046b58 ("mm, page_ext: move page_ext_init() after page_alloc_init_late()")

from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary. This
is now fixed as far as linux-next is concerned, but any non trivial
conflicts should be mentioned to your upstream maintainer when your tree
is submitted for merging.  You may also want to consider cooperating
with the maintainer of the conflicting tree to minimise any particularly
complex conflicts.

-- 
Cheers,
Stephen Rothwell

diff --cc init/main.c
index aea41cf8f9a3,c401e5a38af3..000000000000
--- a/init/main.c
+++ b/init/main.c
@@@ -658,9 -651,8 +659,8 @@@ asmlinkage __visible void __init start_
  		initrd_start = 0;
  	}
  #endif
- 	page_ext_init();
 -	debug_objects_mem_init();
  	kmemleak_init();
 +	debug_objects_mem_init();
  	setup_per_cpu_pageset();
  	numa_policy_init();
  	if (late_time_init)

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-14 19:57                   ` Peter Zijlstra
@ 2017-08-16  4:14                     ` Minchan Kim
  0 siblings, 0 replies; 87+ messages in thread
From: Minchan Kim @ 2017-08-16  4:14 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Nadav Amit, Ingo Molnar, Stephen Rothwell, Andrew Morton,
	Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Linux-Next Mailing List, Linux Kernel Mailing List, Linus

On Mon, Aug 14, 2017 at 09:57:23PM +0200, Peter Zijlstra wrote:
> On Mon, Aug 14, 2017 at 05:38:39PM +0900, Minchan Kim wrote:
> > memory-barrier.txt always scares me. I have read it for a while
> > and IIUC, it seems semantic of spin_unlock(&same_pte) would be
> > enough without some memory-barrier inside mm_tlb_flush_nested.
> 
> Indeed, see the email I just send. Its both spin_lock() and
> spin_unlock() that we care about.
> 
> Aside from the semi permeable barrier of these primitives, RCpc ensures
> these orderings only work against the _same_ lock variable.
> 
> Let me try and explain the ordering for PPC (which is by far the worst
> we have in this regard):
> 
> 
> spin_lock(lock)
> {
> 	while (test_and_set(lock))
> 		cpu_relax();
> 	lwsync();
> }
> 
> 
> spin_unlock(lock)
> {
> 	lwsync();
> 	clear(lock);
> }
> 
> Now LWSYNC has fairly 'simple' semantics, but with fairly horrible
> ramifications. Consider LWSYNC to provide _local_ TSO ordering, this
> means that it allows 'stores reordered after loads'.
> 
> For the spin_lock() that implies that all load/store's inside the lock
> do indeed stay in, but the ACQUIRE is only on the LOAD of the
> test_and_set(). That is, the actual _set_ can leak in. After all it can
> re-order stores after load (inside the lock).
> 
> For unlock it again means all load/store's prior stay prior, and the
> RELEASE is on the store clearing the lock state (nothing surprising
> here).
> 
> Now the _local_ part, the main take-away is that these orderings are
> strictly CPU local. What makes the spinlock work across CPUs (as we'd
> very much expect it to) is the address dependency on the lock variable.
> 
> In order for the spin_lock() to succeed, it must observe the clear. Its
> this link that crosses between the CPUs and builds the ordering. But
> only the two CPUs agree on this order. A third CPU not involved in
> this transaction can disagree on the order of events.

The detail explanation in your previous reply makes me comfortable
from scary memory-barrier.txt but this reply makes me scared again. ;-)

Thanks for the kind clarification, Peter!

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-14 19:38                 ` Peter Zijlstra
@ 2017-08-15  7:51                   ` Nadav Amit
  0 siblings, 0 replies; 87+ messages in thread
From: Nadav Amit @ 2017-08-15  7:51 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Minchan Kim, Ingo Molnar, Stephen Rothwell, Andrew Morton,
	Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Linux-Next Mailing List, Linux Kernel Mailing List, Linus

Peter Zijlstra <peterz@infradead.org> wrote:

> On Mon, Aug 14, 2017 at 05:07:19AM +0000, Nadav Amit wrote:
>>>> So I'm not entirely clear about this yet.
>>>> 
>>>> How about:
>>>> 
>>>> 
>>>> 	CPU0				CPU1
>>>> 
>>>> 					tlb_gather_mmu()
>>>> 
>>>> 					lock PTLn
>>>> 					no mod
>>>> 					unlock PTLn
>>>> 
>>>> 	tlb_gather_mmu()
>>>> 
>>>> 					lock PTLm
>>>> 					mod
>>>> 					include in tlb range
>>>> 					unlock PTLm
>>>> 
>>>> 	lock PTLn
>>>> 	mod
>>>> 	unlock PTLn
>>>> 
>>>> 					tlb_finish_mmu()
>>>> 					  force = mm_tlb_flush_nested(tlb->mm);
>>>> 					  arch_tlb_finish_mmu(force);
>>>> 
>>>> 
>>>> 	... more ...
>>>> 
>>>> 	tlb_finish_mmu()
>>>> 
>>>> 
>>>> 
>>>> In this case you also want CPU1's mm_tlb_flush_nested() call to return
>>>> true, right?
>>> 
>>> No, because CPU 1 mofified pte and added it into tlb range
>>> so regardless of nested, it will flush TLB so there is no stale
>>> TLB problem.
> 
>> To clarify: the main problem that these patches address is when the first
>> CPU updates the PTE, and second CPU sees the updated value and thinks: “the
>> PTE is already what I wanted - no flush is needed”.
> 
> OK, that simplifies things.
> 
>> For some reason (I would assume intentional), all the examples here first
>> “do not modify” the PTE, and then modify it - which is not an “interesting”
>> case.
> 
> Depends on what you call 'interesting' :-) They are 'interesting' to
> make work from a memory ordering POV. And since I didn't get they were
> excluded from the set, I worried.
> 
> In fact, if they were to be included, I couldn't make it work at all. So
> I'm really glad to hear we can disregard them.
> 
>> However, based on what I understand on the memory barriers, I think
>> there is indeed a missing barrier before reading it in
>> mm_tlb_flush_nested(). IIUC using smp_mb__after_unlock_lock() in this case,
>> before reading, would solve the problem with least impact on systems with
>> strong memory ordering.
> 
> No, all is well. If, as you say, we're naturally constrained to the case
> where we only care about prior modification we can rely on the RCpc PTL
> locks.
> 
> Consider:
> 
> 
> 	CPU0				CPU1
> 
> 					tlb_gather_mmu()
> 
> 	tlb_gather_mmu()
> 	  inc	--------.
> 			| (inc is constrained by RELEASE)
> 	lock PTLn	|
> 	mod		^
> 	unlock PTLn ----------------->	lock PTLn
> 				v	no mod
> 				|	unlock PTLn
> 				|
> 				|	lock PTLm
> 				|	mod
> 				|	include in tlb range
> 				|	unlock PTLm
> 				|
> 	(read is constrained	|
> 	          by ACQUIRE)	|
> 				|	tlb_finish_mmu()
> 				`----	  force = mm_tlb_flush_nested(tlb->mm);
> 					  arch_tlb_finish_mmu(force);
> 
> 
> 	... more ...
> 
> 	tlb_finish_mmu()
> 
> 
> Then CPU1's acquire of PTLn orders against CPU0's release of that same
> PTLn which guarantees we observe both its (prior) modified PTE and the
> mm->tlb_flush_pending increment from tlb_gather_mmu().
> 
> So all we need for mm_tlb_flush_nested() to work is having acquired the
> right PTL at least once before calling it.
> 
> At the same time, the decrements need to be after the TLB invalidate is
> complete, this ensures that _IF_ we observe the decrement, we must've
> also observed the corresponding invalidate.
> 
> Something like the below is then sufficient.
> 
> ---
> Subject: mm: Clarify tlb_flush_pending barriers
> From: Peter Zijlstra <peterz@infradead.org>
> Date: Fri, 11 Aug 2017 16:04:50 +0200
> 
> Better document the ordering around tlb_flush_pending.
> 
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> ---
> include/linux/mm_types.h |   78 +++++++++++++++++++++++++++--------------------
> 1 file changed, 45 insertions(+), 33 deletions(-)
> 
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -526,30 +526,6 @@ extern void tlb_gather_mmu(struct mmu_ga
> extern void tlb_finish_mmu(struct mmu_gather *tlb,
> 				unsigned long start, unsigned long end);
> 
> -/*
> - * Memory barriers to keep this state in sync are graciously provided by
> - * the page table locks, outside of which no page table modifications happen.
> - * The barriers are used to ensure the order between tlb_flush_pending updates,
> - * which happen while the lock is not taken, and the PTE updates, which happen
> - * while the lock is taken, are serialized.
> - */
> -static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
> -{
> -	/*
> -	 * Must be called with PTL held; such that our PTL acquire will have
> -	 * observed the store from set_tlb_flush_pending().
> -	 */
> -	return atomic_read(&mm->tlb_flush_pending) > 0;
> -}
> -
> -/*
> - * Returns true if there are two above TLB batching threads in parallel.
> - */
> -static inline bool mm_tlb_flush_nested(struct mm_struct *mm)
> -{
> -	return atomic_read(&mm->tlb_flush_pending) > 1;
> -}
> -
> static inline void init_tlb_flush_pending(struct mm_struct *mm)
> {
> 	atomic_set(&mm->tlb_flush_pending, 0);
> @@ -558,7 +534,6 @@ static inline void init_tlb_flush_pendin
> static inline void inc_tlb_flush_pending(struct mm_struct *mm)
> {
> 	atomic_inc(&mm->tlb_flush_pending);
> -
> 	/*
> 	 * The only time this value is relevant is when there are indeed pages
> 	 * to flush. And we'll only flush pages after changing them, which
> @@ -580,24 +555,61 @@ static inline void inc_tlb_flush_pending
> 	 *	flush_tlb_range();
> 	 *	atomic_dec(&mm->tlb_flush_pending);
> 	 *
> -	 * So the =true store is constrained by the PTL unlock, and the =false
> -	 * store is constrained by the TLB invalidate.
> +	 * Where the increment if constrained by the PTL unlock, it thus
> +	 * ensures that the increment is visible if the PTE modification is
> +	 * visible. After all, if there is no PTE modification, nobody cares
> +	 * about TLB flushes either.
> +	 *
> +	 * This very much relies on users (mm_tlb_flush_pending() and
> +	 * mm_tlb_flush_nested()) only caring about _specific_ PTEs (and
> +	 * therefore specific PTLs), because with SPLIT_PTE_PTLOCKS and RCpc
> +	 * locks (PPC) the unlock of one doesn't order against the lock of
> +	 * another PTL.
> +	 *
> +	 * The decrement is ordered by the flush_tlb_range(), such that
> +	 * mm_tlb_flush_pending() will not return false unless all flushes have
> +	 * completed.
> 	 */
> }
> 
> -/* Clearing is done after a TLB flush, which also provides a barrier. */
> static inline void dec_tlb_flush_pending(struct mm_struct *mm)
> {
> 	/*
> -	 * Guarantee that the tlb_flush_pending does not not leak into the
> -	 * critical section, since we must order the PTE change and changes to
> -	 * the pending TLB flush indication. We could have relied on TLB flush
> -	 * as a memory barrier, but this behavior is not clearly documented.
> +	 * See inc_tlb_flush_pending().
> +	 *
> +	 * This cannot be smp_mb__before_atomic() because smp_mb() simply does
> +	 * not order against TLB invalidate completion, which is what we need.
> +	 *
> +	 * Therefore we must rely on tlb_flush_*() to guarantee order.
> 	 */
> -	smp_mb__before_atomic();
> 	atomic_dec(&mm->tlb_flush_pending);
> }
> 
> +static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
> +{
> +	/*
> +	 * Must be called after having acquired the PTL; orders against that
> +	 * PTLs release and therefore ensures that if we observe the modified
> +	 * PTE we must also observe the increment from inc_tlb_flush_pending().
> +	 *
> +	 * That is, it only guarantees to return true if there is a flush
> +	 * pending for _this_ PTL.
> +	 */
> +	return atomic_read(&mm->tlb_flush_pending);
> +}
> +
> +static inline bool mm_tlb_flush_nested(struct mm_struct *mm)
> +{
> +	/*
> +	 * Similar to mm_tlb_flush_pending(), we must have acquired the PTL
> +	 * for which there is a TLB flush pending in order to guarantee
> +	 * we've seen both that PTE modification and the increment.
> +	 *
> +	 * (no requirement on actually still holding the PTL, that is irrelevant)
> +	 */
> +	return atomic_read(&mm->tlb_flush_pending) > 1;
> +}
> +
> struct vm_fault;
> 
> struct vm_special_mapping {

Thanks for the detailed explanation. I will pay more attention next time.


^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-14  8:38                 ` Minchan Kim
@ 2017-08-14 19:57                   ` Peter Zijlstra
  2017-08-16  4:14                     ` Minchan Kim
  0 siblings, 1 reply; 87+ messages in thread
From: Peter Zijlstra @ 2017-08-14 19:57 UTC (permalink / raw)
  To: Minchan Kim
  Cc: Nadav Amit, Ingo Molnar, Stephen Rothwell, Andrew Morton,
	Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Linux-Next Mailing List, Linux Kernel Mailing List, Linus

On Mon, Aug 14, 2017 at 05:38:39PM +0900, Minchan Kim wrote:
> memory-barrier.txt always scares me. I have read it for a while
> and IIUC, it seems semantic of spin_unlock(&same_pte) would be
> enough without some memory-barrier inside mm_tlb_flush_nested.

Indeed, see the email I just send. Its both spin_lock() and
spin_unlock() that we care about.

Aside from the semi permeable barrier of these primitives, RCpc ensures
these orderings only work against the _same_ lock variable.

Let me try and explain the ordering for PPC (which is by far the worst
we have in this regard):


spin_lock(lock)
{
	while (test_and_set(lock))
		cpu_relax();
	lwsync();
}


spin_unlock(lock)
{
	lwsync();
	clear(lock);
}

Now LWSYNC has fairly 'simple' semantics, but with fairly horrible
ramifications. Consider LWSYNC to provide _local_ TSO ordering, this
means that it allows 'stores reordered after loads'.

For the spin_lock() that implies that all load/store's inside the lock
do indeed stay in, but the ACQUIRE is only on the LOAD of the
test_and_set(). That is, the actual _set_ can leak in. After all it can
re-order stores after load (inside the lock).

For unlock it again means all load/store's prior stay prior, and the
RELEASE is on the store clearing the lock state (nothing surprising
here).

Now the _local_ part, the main take-away is that these orderings are
strictly CPU local. What makes the spinlock work across CPUs (as we'd
very much expect it to) is the address dependency on the lock variable.

In order for the spin_lock() to succeed, it must observe the clear. Its
this link that crosses between the CPUs and builds the ordering. But
only the two CPUs agree on this order. A third CPU not involved in
this transaction can disagree on the order of events.

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-14  5:07               ` Nadav Amit
  2017-08-14  5:23                 ` Minchan Kim
  2017-08-14  8:38                 ` Minchan Kim
@ 2017-08-14 19:38                 ` Peter Zijlstra
  2017-08-15  7:51                   ` Nadav Amit
  2 siblings, 1 reply; 87+ messages in thread
From: Peter Zijlstra @ 2017-08-14 19:38 UTC (permalink / raw)
  To: Nadav Amit
  Cc: Minchan Kim, Ingo Molnar, Stephen Rothwell, Andrew Morton,
	Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Linux-Next Mailing List, Linux Kernel Mailing List, Linus

On Mon, Aug 14, 2017 at 05:07:19AM +0000, Nadav Amit wrote:
> >> So I'm not entirely clear about this yet.
> >> 
> >> How about:
> >> 
> >> 
> >> 	CPU0				CPU1
> >> 
> >> 					tlb_gather_mmu()
> >> 
> >> 					lock PTLn
> >> 					no mod
> >> 					unlock PTLn
> >> 
> >> 	tlb_gather_mmu()
> >> 
> >> 					lock PTLm
> >> 					mod
> >> 					include in tlb range
> >> 					unlock PTLm
> >> 
> >> 	lock PTLn
> >> 	mod
> >> 	unlock PTLn
> >> 
> >> 					tlb_finish_mmu()
> >> 					  force = mm_tlb_flush_nested(tlb->mm);
> >> 					  arch_tlb_finish_mmu(force);
> >> 
> >> 
> >> 	... more ...
> >> 
> >> 	tlb_finish_mmu()
> >> 
> >> 
> >> 
> >> In this case you also want CPU1's mm_tlb_flush_nested() call to return
> >> true, right?
> > 
> > No, because CPU 1 mofified pte and added it into tlb range
> > so regardless of nested, it will flush TLB so there is no stale
> > TLB problem.

> To clarify: the main problem that these patches address is when the first
> CPU updates the PTE, and second CPU sees the updated value and thinks: “the
> PTE is already what I wanted - no flush is needed”.

OK, that simplifies things.

> For some reason (I would assume intentional), all the examples here first
> “do not modify” the PTE, and then modify it - which is not an “interesting”
> case.

Depends on what you call 'interesting' :-) They are 'interesting' to
make work from a memory ordering POV. And since I didn't get they were
excluded from the set, I worried.

In fact, if they were to be included, I couldn't make it work at all. So
I'm really glad to hear we can disregard them.

> However, based on what I understand on the memory barriers, I think
> there is indeed a missing barrier before reading it in
> mm_tlb_flush_nested(). IIUC using smp_mb__after_unlock_lock() in this case,
> before reading, would solve the problem with least impact on systems with
> strong memory ordering.

No, all is well. If, as you say, we're naturally constrained to the case
where we only care about prior modification we can rely on the RCpc PTL
locks.

Consider:


	CPU0				CPU1

					tlb_gather_mmu()

	tlb_gather_mmu()
	  inc	--------.
			| (inc is constrained by RELEASE)
	lock PTLn	|
	mod		^
	unlock PTLn ----------------->	lock PTLn
				v	no mod
				|	unlock PTLn
				|
				|	lock PTLm
				|	mod
				|	include in tlb range
				|	unlock PTLm
				|
	(read is constrained	|
	          by ACQUIRE)	|
				|	tlb_finish_mmu()
				`----	  force = mm_tlb_flush_nested(tlb->mm);
					  arch_tlb_finish_mmu(force);


	... more ...

	tlb_finish_mmu()


Then CPU1's acquire of PTLn orders against CPU0's release of that same
PTLn which guarantees we observe both its (prior) modified PTE and the
mm->tlb_flush_pending increment from tlb_gather_mmu().

So all we need for mm_tlb_flush_nested() to work is having acquired the
right PTL at least once before calling it.

At the same time, the decrements need to be after the TLB invalidate is
complete, this ensures that _IF_ we observe the decrement, we must've
also observed the corresponding invalidate.

Something like the below is then sufficient.

---
Subject: mm: Clarify tlb_flush_pending barriers
From: Peter Zijlstra <peterz@infradead.org>
Date: Fri, 11 Aug 2017 16:04:50 +0200

Better document the ordering around tlb_flush_pending.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 include/linux/mm_types.h |   78 +++++++++++++++++++++++++++--------------------
 1 file changed, 45 insertions(+), 33 deletions(-)

--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -526,30 +526,6 @@ extern void tlb_gather_mmu(struct mmu_ga
 extern void tlb_finish_mmu(struct mmu_gather *tlb,
 				unsigned long start, unsigned long end);
 
-/*
- * Memory barriers to keep this state in sync are graciously provided by
- * the page table locks, outside of which no page table modifications happen.
- * The barriers are used to ensure the order between tlb_flush_pending updates,
- * which happen while the lock is not taken, and the PTE updates, which happen
- * while the lock is taken, are serialized.
- */
-static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
-{
-	/*
-	 * Must be called with PTL held; such that our PTL acquire will have
-	 * observed the store from set_tlb_flush_pending().
-	 */
-	return atomic_read(&mm->tlb_flush_pending) > 0;
-}
-
-/*
- * Returns true if there are two above TLB batching threads in parallel.
- */
-static inline bool mm_tlb_flush_nested(struct mm_struct *mm)
-{
-	return atomic_read(&mm->tlb_flush_pending) > 1;
-}
-
 static inline void init_tlb_flush_pending(struct mm_struct *mm)
 {
 	atomic_set(&mm->tlb_flush_pending, 0);
@@ -558,7 +534,6 @@ static inline void init_tlb_flush_pendin
 static inline void inc_tlb_flush_pending(struct mm_struct *mm)
 {
 	atomic_inc(&mm->tlb_flush_pending);
-
 	/*
 	 * The only time this value is relevant is when there are indeed pages
 	 * to flush. And we'll only flush pages after changing them, which
@@ -580,24 +555,61 @@ static inline void inc_tlb_flush_pending
 	 *	flush_tlb_range();
 	 *	atomic_dec(&mm->tlb_flush_pending);
 	 *
-	 * So the =true store is constrained by the PTL unlock, and the =false
-	 * store is constrained by the TLB invalidate.
+	 * Where the increment if constrained by the PTL unlock, it thus
+	 * ensures that the increment is visible if the PTE modification is
+	 * visible. After all, if there is no PTE modification, nobody cares
+	 * about TLB flushes either.
+	 *
+	 * This very much relies on users (mm_tlb_flush_pending() and
+	 * mm_tlb_flush_nested()) only caring about _specific_ PTEs (and
+	 * therefore specific PTLs), because with SPLIT_PTE_PTLOCKS and RCpc
+	 * locks (PPC) the unlock of one doesn't order against the lock of
+	 * another PTL.
+	 *
+	 * The decrement is ordered by the flush_tlb_range(), such that
+	 * mm_tlb_flush_pending() will not return false unless all flushes have
+	 * completed.
 	 */
 }
 
-/* Clearing is done after a TLB flush, which also provides a barrier. */
 static inline void dec_tlb_flush_pending(struct mm_struct *mm)
 {
 	/*
-	 * Guarantee that the tlb_flush_pending does not not leak into the
-	 * critical section, since we must order the PTE change and changes to
-	 * the pending TLB flush indication. We could have relied on TLB flush
-	 * as a memory barrier, but this behavior is not clearly documented.
+	 * See inc_tlb_flush_pending().
+	 *
+	 * This cannot be smp_mb__before_atomic() because smp_mb() simply does
+	 * not order against TLB invalidate completion, which is what we need.
+	 *
+	 * Therefore we must rely on tlb_flush_*() to guarantee order.
 	 */
-	smp_mb__before_atomic();
 	atomic_dec(&mm->tlb_flush_pending);
 }
 
+static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
+{
+	/*
+	 * Must be called after having acquired the PTL; orders against that
+	 * PTLs release and therefore ensures that if we observe the modified
+	 * PTE we must also observe the increment from inc_tlb_flush_pending().
+	 *
+	 * That is, it only guarantees to return true if there is a flush
+	 * pending for _this_ PTL.
+	 */
+	return atomic_read(&mm->tlb_flush_pending);
+}
+
+static inline bool mm_tlb_flush_nested(struct mm_struct *mm)
+{
+	/*
+	 * Similar to mm_tlb_flush_pending(), we must have acquired the PTL
+	 * for which there is a TLB flush pending in order to guarantee
+	 * we've seen both that PTE modification and the increment.
+	 *
+	 * (no requirement on actually still holding the PTL, that is irrelevant)
+	 */
+	return atomic_read(&mm->tlb_flush_pending) > 1;
+}
+
 struct vm_fault;
 
 struct vm_special_mapping {

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-14  3:09         ` Minchan Kim
@ 2017-08-14 18:54           ` Peter Zijlstra
  0 siblings, 0 replies; 87+ messages in thread
From: Peter Zijlstra @ 2017-08-14 18:54 UTC (permalink / raw)
  To: Minchan Kim
  Cc: Ingo Molnar, Stephen Rothwell, Andrew Morton, Thomas Gleixner,
	Ingo Molnar, H. Peter Anvin, Linux-Next Mailing List,
	Linux Kernel Mailing List, Nadav Amit, Linus

On Mon, Aug 14, 2017 at 12:09:14PM +0900, Minchan Kim wrote:
> @@ -446,9 +450,7 @@ void tlb_finish_mmu(struct mmu_gather *tlb,
>  	 *
>  	 */
>  	bool force = mm_tlb_flush_nested(tlb->mm);
> -
>  	arch_tlb_finish_mmu(tlb, start, end, force);
> -	dec_tlb_flush_pending(tlb->mm);
>  }

No, I think this breaks all the mm_tlb_flush_pending() users. They need
the decrement to not be visible until the TLB flush is complete.

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-14  5:07               ` Nadav Amit
  2017-08-14  5:23                 ` Minchan Kim
@ 2017-08-14  8:38                 ` Minchan Kim
  2017-08-14 19:57                   ` Peter Zijlstra
  2017-08-14 19:38                 ` Peter Zijlstra
  2 siblings, 1 reply; 87+ messages in thread
From: Minchan Kim @ 2017-08-14  8:38 UTC (permalink / raw)
  To: Nadav Amit
  Cc: Peter Zijlstra, Ingo Molnar, Stephen Rothwell, Andrew Morton,
	Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Linux-Next Mailing List, Linux Kernel Mailing List, Linus

Hi Nadav,

On Mon, Aug 14, 2017 at 05:07:19AM +0000, Nadav Amit wrote:
< snip >

> For some reason (I would assume intentional), all the examples here first
> “do not modify” the PTE, and then modify it - which is not an “interesting”
> case. However, based on what I understand on the memory barriers, I think
> there is indeed a missing barrier before reading it in
> mm_tlb_flush_nested(). IIUC using smp_mb__after_unlock_lock() in this case,

memory-barrier.txt always scares me. I have read it for a while
and IIUC, it seems semantic of spin_unlock(&same_pte) would be
enough without some memory-barrier inside mm_tlb_flush_nested.

I would be missing something totally.

Could you explain what kinds of sequence you have in mind to
have such problem?

Thanks.

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-14  5:07               ` Nadav Amit
@ 2017-08-14  5:23                 ` Minchan Kim
  2017-08-14  8:38                 ` Minchan Kim
  2017-08-14 19:38                 ` Peter Zijlstra
  2 siblings, 0 replies; 87+ messages in thread
From: Minchan Kim @ 2017-08-14  5:23 UTC (permalink / raw)
  To: Nadav Amit
  Cc: Peter Zijlstra, Ingo Molnar, Stephen Rothwell, Andrew Morton,
	Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Linux-Next Mailing List, Linux Kernel Mailing List, Linus

On Mon, Aug 14, 2017 at 05:07:19AM +0000, Nadav Amit wrote:
< snip >

> Minchan, as for the solution you proposed, it seems to open again a race,
> since the “pending” indication is removed before the actual TLB flush is
> performed.

Oops, you're right!

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-14  3:16             ` Minchan Kim
@ 2017-08-14  5:07               ` Nadav Amit
  2017-08-14  5:23                 ` Minchan Kim
                                   ` (2 more replies)
  0 siblings, 3 replies; 87+ messages in thread
From: Nadav Amit @ 2017-08-14  5:07 UTC (permalink / raw)
  To: Minchan Kim
  Cc: Peter Zijlstra, Ingo Molnar, Stephen Rothwell, Andrew Morton,
	Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Linux-Next Mailing List, Linux Kernel Mailing List, Linus

Minchan Kim <minchan@kernel.org> wrote:

> On Sun, Aug 13, 2017 at 02:50:19PM +0200, Peter Zijlstra wrote:
>> On Sun, Aug 13, 2017 at 06:06:32AM +0000, Nadav Amit wrote:
>>>> however mm_tlb_flush_nested() is a mystery, it appears to care about
>>>> anything inside the range. For now rely on it doing at least _a_ PTL
>>>> lock instead of taking  _the_ PTL lock.
>>> 
>>> It does not care about “anything” inside the range, but only on situations
>>> in which there is at least one (same) PT that was modified by one core and
>>> then read by the other. So, yes, it will always be _the_ same PTL, and not
>>> _a_ PTL - in the cases that flush is really needed.
>>> 
>>> The issue that might require additional barriers is that
>>> inc_tlb_flush_pending() and mm_tlb_flush_nested() are called when the PTL is
>>> not held. IIUC, since the release-acquire might not behave as a full memory
>>> barrier, this requires an explicit memory barrier.
>> 
>> So I'm not entirely clear about this yet.
>> 
>> How about:
>> 
>> 
>> 	CPU0				CPU1
>> 
>> 					tlb_gather_mmu()
>> 
>> 					lock PTLn
>> 					no mod
>> 					unlock PTLn
>> 
>> 	tlb_gather_mmu()
>> 
>> 					lock PTLm
>> 					mod
>> 					include in tlb range
>> 					unlock PTLm
>> 
>> 	lock PTLn
>> 	mod
>> 	unlock PTLn
>> 
>> 					tlb_finish_mmu()
>> 					  force = mm_tlb_flush_nested(tlb->mm);
>> 					  arch_tlb_finish_mmu(force);
>> 
>> 
>> 	... more ...
>> 
>> 	tlb_finish_mmu()
>> 
>> 
>> 
>> In this case you also want CPU1's mm_tlb_flush_nested() call to return
>> true, right?
> 
> No, because CPU 1 mofified pte and added it into tlb range
> so regardless of nested, it will flush TLB so there is no stale
> TLB problem.
> 
>> But even with an smp_mb__after_atomic() at CPU0's tlg_bather_mmu()
>> you're not guaranteed CPU1 sees the increment. The only way to do that
>> is to make the PTL locks RCsc and that is a much more expensive
>> proposition.
>> 
>> 
>> What about:
>> 
>> 
>> 	CPU0				CPU1
>> 
>> 					tlb_gather_mmu()
>> 
>> 					lock PTLn
>> 					no mod
>> 					unlock PTLn
>> 
>> 
>> 					lock PTLm
>> 					mod
>> 					include in tlb range
>> 					unlock PTLm
>> 
>> 	tlb_gather_mmu()
>> 
>> 	lock PTLn
>> 	mod
>> 	unlock PTLn
>> 
>> 					tlb_finish_mmu()
>> 					  force = mm_tlb_flush_nested(tlb->mm);
>> 					  arch_tlb_finish_mmu(force);
>> 
>> 
>> 	... more ...
>> 
>> 	tlb_finish_mmu()
>> 
>> Do we want CPU1 to see it here? If so, where does it end?
> 
> Ditto. Since CPU 1 has added range, it will flush TLB regardless
> of nested condition.
> 
>> CPU0				CPU1
>> 
>> 					tlb_gather_mmu()
>> 
>> 					lock PTLn
>> 					no mod
>> 					unlock PTLn
>> 
>> 
>> 					lock PTLm
>> 					mod
>> 					include in tlb range
>> 					unlock PTLm
>> 
>> 					tlb_finish_mmu()
>> 					  force = mm_tlb_flush_nested(tlb->mm);
>> 
>> 	tlb_gather_mmu()
>> 
>> 	lock PTLn
>> 	mod
>> 	unlock PTLn
>> 
>> 					  arch_tlb_finish_mmu(force);
>> 
>> 
>> 	... more ...
>> 
>> 	tlb_finish_mmu()
>> 
>> 
>> This?
>> 
>> 
>> Could you clarify under what exact condition mm_tlb_flush_nested() must
>> return true?
> 
> mm_tlb_flush_nested aims for the CPU side where there is no pte update
> but need TLB flush.
> As I wrote https://urldefense.proofpoint.com/v2/url?u=https-3A__marc.info_-3Fl-3Dlinux-2Dmm-26m-3D150267398226529-26w-3D2&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=x9zhXCtCLvTDtvE65-BGSA&m=v2Z7eDi7z1H9zdngcjZvlNeBudWzA9KvcXFNpU2A77s&s=amaSu_gurmBHHPcl3Pxfdl0Tk_uTnmf60tMQAsNDHVU&e= ,
> it has stable TLB problem if we don't flush TLB although there is no
> pte modification.

To clarify: the main problem that these patches address is when the first
CPU updates the PTE, and second CPU sees the updated value and thinks: “the
PTE is already what I wanted - no flush is needed”.

For some reason (I would assume intentional), all the examples here first
“do not modify” the PTE, and then modify it - which is not an “interesting”
case. However, based on what I understand on the memory barriers, I think
there is indeed a missing barrier before reading it in
mm_tlb_flush_nested(). IIUC using smp_mb__after_unlock_lock() in this case,
before reading, would solve the problem with least impact on systems with
strong memory ordering.

Minchan, as for the solution you proposed, it seems to open again a race,
since the “pending” indication is removed before the actual TLB flush is
performed.

Nadav

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-13 12:50           ` Peter Zijlstra
@ 2017-08-14  3:16             ` Minchan Kim
  2017-08-14  5:07               ` Nadav Amit
  0 siblings, 1 reply; 87+ messages in thread
From: Minchan Kim @ 2017-08-14  3:16 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Nadav Amit, Ingo Molnar, Stephen Rothwell, Andrew Morton,
	Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Linux-Next Mailing List, Linux Kernel Mailing List, Linus

On Sun, Aug 13, 2017 at 02:50:19PM +0200, Peter Zijlstra wrote:
> On Sun, Aug 13, 2017 at 06:06:32AM +0000, Nadav Amit wrote:
> > > however mm_tlb_flush_nested() is a mystery, it appears to care about
> > > anything inside the range. For now rely on it doing at least _a_ PTL
> > > lock instead of taking  _the_ PTL lock.
> > 
> > It does not care about “anything” inside the range, but only on situations
> > in which there is at least one (same) PT that was modified by one core and
> > then read by the other. So, yes, it will always be _the_ same PTL, and not
> > _a_ PTL - in the cases that flush is really needed.
> > 
> > The issue that might require additional barriers is that
> > inc_tlb_flush_pending() and mm_tlb_flush_nested() are called when the PTL is
> > not held. IIUC, since the release-acquire might not behave as a full memory
> > barrier, this requires an explicit memory barrier.
> 
> So I'm not entirely clear about this yet.
> 
> How about:
> 
> 
> 	CPU0				CPU1
> 
> 					tlb_gather_mmu()
> 
> 					lock PTLn
> 					no mod
> 					unlock PTLn
> 
> 	tlb_gather_mmu()
> 
> 					lock PTLm
> 					mod
> 					include in tlb range
> 					unlock PTLm
> 
> 	lock PTLn
> 	mod
> 	unlock PTLn
> 
> 					tlb_finish_mmu()
> 					  force = mm_tlb_flush_nested(tlb->mm);
> 					  arch_tlb_finish_mmu(force);
> 
> 
> 	... more ...
> 
> 	tlb_finish_mmu()
> 
> 
> 
> In this case you also want CPU1's mm_tlb_flush_nested() call to return
> true, right?

No, because CPU 1 mofified pte and added it into tlb range
so regardless of nested, it will flush TLB so there is no stale
TLB problem.

> 
> But even with an smp_mb__after_atomic() at CPU0's tlg_bather_mmu()
> you're not guaranteed CPU1 sees the increment. The only way to do that
> is to make the PTL locks RCsc and that is a much more expensive
> proposition.
> 
> 
> What about:
> 
> 
> 	CPU0				CPU1
> 
> 					tlb_gather_mmu()
> 
> 					lock PTLn
> 					no mod
> 					unlock PTLn
> 
> 
> 					lock PTLm
> 					mod
> 					include in tlb range
> 					unlock PTLm
> 
> 	tlb_gather_mmu()
> 
> 	lock PTLn
> 	mod
> 	unlock PTLn
> 
> 					tlb_finish_mmu()
> 					  force = mm_tlb_flush_nested(tlb->mm);
> 					  arch_tlb_finish_mmu(force);
> 
> 
> 	... more ...
> 
> 	tlb_finish_mmu()
> 
> Do we want CPU1 to see it here? If so, where does it end?

Ditto. Since CPU 1 has added range, it will flush TLB regardless
of nested condition.

> 
> 	CPU0				CPU1
> 
> 					tlb_gather_mmu()
> 
> 					lock PTLn
> 					no mod
> 					unlock PTLn
> 
> 
> 					lock PTLm
> 					mod
> 					include in tlb range
> 					unlock PTLm
> 
> 					tlb_finish_mmu()
> 					  force = mm_tlb_flush_nested(tlb->mm);
> 
> 	tlb_gather_mmu()
> 
> 	lock PTLn
> 	mod
> 	unlock PTLn
> 
> 					  arch_tlb_finish_mmu(force);
> 
> 
> 	... more ...
> 
> 	tlb_finish_mmu()
> 
> 
> This?
> 
> 
> Could you clarify under what exact condition mm_tlb_flush_nested() must
> return true?

mm_tlb_flush_nested aims for the CPU side where there is no pte update
but need TLB flush.
As I wrote https://marc.info/?l=linux-mm&m=150267398226529&w=2,
it has stable TLB problem if we don't flush TLB although there is no
pte modification.

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-11 14:04       ` Peter Zijlstra
  2017-08-13  6:06         ` Nadav Amit
@ 2017-08-14  3:09         ` Minchan Kim
  2017-08-14 18:54           ` Peter Zijlstra
  1 sibling, 1 reply; 87+ messages in thread
From: Minchan Kim @ 2017-08-14  3:09 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Ingo Molnar, Stephen Rothwell, Andrew Morton, Thomas Gleixner,
	Ingo Molnar, H. Peter Anvin, Linux-Next Mailing List,
	Linux Kernel Mailing List, Nadav Amit, Linus

Hi Peter,

On Fri, Aug 11, 2017 at 04:04:50PM +0200, Peter Zijlstra wrote:
> 
> Ok, so I have the below to still go on-top.
> 
> Ideally someone would clarify the situation around
> mm_tlb_flush_nested(), because ideally we'd remove the
> smp_mb__after_atomic() and go back to relying on PTL alone.
> 
> This also removes the pointless smp_mb__before_atomic()

I'm not an expert of barrier stuff but IIUC, mm_tlb_flush_nested's
side full memory barrier can go with removing smp_mb__after_atomic
in inc_tlb_flush_pending side?


diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 490af494c2da..5ad0e66df363 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -544,7 +544,12 @@ static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
  */
 static inline bool mm_tlb_flush_nested(struct mm_struct *mm)
 {
-	return atomic_read(&mm->tlb_flush_pending) > 1;
+	/*
+	 * atomic_dec_and_test's full memory barrier guarantees
+	 * to see uptodate tlb_flush_pending count in other CPU
+	 * without relying on page table lock.
+	 */
+	return !atomic_dec_and_test(&mm->tlb_flush_pending);
 }
 
 static inline void init_tlb_flush_pending(struct mm_struct *mm)
diff --git a/mm/memory.c b/mm/memory.c
index f571b0eb9816..e90b57bc65fb 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -407,6 +407,10 @@ void tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
 			unsigned long start, unsigned long end)
 {
 	arch_tlb_gather_mmu(tlb, mm, start, end);
+	/*
+	 * couterpart is mm_tlb_flush_nested in tlb_finish_mmu
+	 * which decreases pending count.
+	 */
 	inc_tlb_flush_pending(tlb->mm);
 }
 
@@ -446,9 +450,7 @@ void tlb_finish_mmu(struct mmu_gather *tlb,
 	 *
 	 */
 	bool force = mm_tlb_flush_nested(tlb->mm);
-
 	arch_tlb_finish_mmu(tlb, start, end, force);
-	dec_tlb_flush_pending(tlb->mm);
 }
 
 /*

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-13  6:06         ` Nadav Amit
@ 2017-08-13 12:50           ` Peter Zijlstra
  2017-08-14  3:16             ` Minchan Kim
  0 siblings, 1 reply; 87+ messages in thread
From: Peter Zijlstra @ 2017-08-13 12:50 UTC (permalink / raw)
  To: Nadav Amit
  Cc: Ingo Molnar, Stephen Rothwell, Andrew Morton, Thomas Gleixner,
	Ingo Molnar, H. Peter Anvin, Linux-Next Mailing List,
	Linux Kernel Mailing List, Linus, minchan

On Sun, Aug 13, 2017 at 06:06:32AM +0000, Nadav Amit wrote:
> > however mm_tlb_flush_nested() is a mystery, it appears to care about
> > anything inside the range. For now rely on it doing at least _a_ PTL
> > lock instead of taking  _the_ PTL lock.
> 
> It does not care about “anything” inside the range, but only on situations
> in which there is at least one (same) PT that was modified by one core and
> then read by the other. So, yes, it will always be _the_ same PTL, and not
> _a_ PTL - in the cases that flush is really needed.
> 
> The issue that might require additional barriers is that
> inc_tlb_flush_pending() and mm_tlb_flush_nested() are called when the PTL is
> not held. IIUC, since the release-acquire might not behave as a full memory
> barrier, this requires an explicit memory barrier.

So I'm not entirely clear about this yet.

How about:


	CPU0				CPU1

					tlb_gather_mmu()

					lock PTLn
					no mod
					unlock PTLn

	tlb_gather_mmu()

					lock PTLm
					mod
					include in tlb range
					unlock PTLm

	lock PTLn
	mod
	unlock PTLn

					tlb_finish_mmu()
					  force = mm_tlb_flush_nested(tlb->mm);
					  arch_tlb_finish_mmu(force);


	... more ...

	tlb_finish_mmu()



In this case you also want CPU1's mm_tlb_flush_nested() call to return
true, right?

But even with an smp_mb__after_atomic() at CPU0's tlg_bather_mmu()
you're not guaranteed CPU1 sees the increment. The only way to do that
is to make the PTL locks RCsc and that is a much more expensive
proposition.


What about:


	CPU0				CPU1

					tlb_gather_mmu()

					lock PTLn
					no mod
					unlock PTLn


					lock PTLm
					mod
					include in tlb range
					unlock PTLm

	tlb_gather_mmu()

	lock PTLn
	mod
	unlock PTLn

					tlb_finish_mmu()
					  force = mm_tlb_flush_nested(tlb->mm);
					  arch_tlb_finish_mmu(force);


	... more ...

	tlb_finish_mmu()

Do we want CPU1 to see it here? If so, where does it end?


	CPU0				CPU1

					tlb_gather_mmu()

					lock PTLn
					no mod
					unlock PTLn


					lock PTLm
					mod
					include in tlb range
					unlock PTLm

					tlb_finish_mmu()
					  force = mm_tlb_flush_nested(tlb->mm);

	tlb_gather_mmu()

	lock PTLn
	mod
	unlock PTLn

					  arch_tlb_finish_mmu(force);


	... more ...

	tlb_finish_mmu()


This?


Could you clarify under what exact condition mm_tlb_flush_nested() must
return true?

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-11 14:04       ` Peter Zijlstra
@ 2017-08-13  6:06         ` Nadav Amit
  2017-08-13 12:50           ` Peter Zijlstra
  2017-08-14  3:09         ` Minchan Kim
  1 sibling, 1 reply; 87+ messages in thread
From: Nadav Amit @ 2017-08-13  6:06 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Ingo Molnar, Stephen Rothwell, Andrew Morton, Thomas Gleixner,
	Ingo Molnar, H. Peter Anvin, Linux-Next Mailing List,
	Linux Kernel Mailing List, Linus, minchan

Peter Zijlstra <peterz@infradead.org> wrote:

> 
> Ok, so I have the below to still go on-top.
> 
> Ideally someone would clarify the situation around
> mm_tlb_flush_nested(), because ideally we'd remove the
> smp_mb__after_atomic() and go back to relying on PTL alone.
> 
> This also removes the pointless smp_mb__before_atomic()
> 
> ---
> Subject: mm: Fix barriers for the tlb_flush_pending thing
> From: Peter Zijlstra <peterz@infradead.org>
> Date: Fri Aug 11 12:43:33 CEST 2017
> 
> I'm not 100% sure we always care about the same PTL and when we have
> SPLIT_PTE_PTLOCKS and have RCpc locks (PPC) the UNLOCK of one does not
> in fact order against the LOCK of another lock. Therefore the
> documented scheme does not work if we care about multiple PTLs
> 
> mm_tlb_flush_pending() appears to only care about a single PTL:
> 
> - arch pte_accessible() (x86, arm64) only cares about that one PTE.
> - do_huge_pmd_numa_page() also only cares about a single (huge) page.
> - ksm write_protect_page() also only cares about a single page.
> 
> however mm_tlb_flush_nested() is a mystery, it appears to care about
> anything inside the range. For now rely on it doing at least _a_ PTL
> lock instead of taking  _the_ PTL lock.

It does not care about “anything” inside the range, but only on situations
in which there is at least one (same) PT that was modified by one core and
then read by the other. So, yes, it will always be _the_ same PTL, and not
_a_ PTL - in the cases that flush is really needed.

The issue that might require additional barriers is that
inc_tlb_flush_pending() and mm_tlb_flush_nested() are called when the PTL is
not held. IIUC, since the release-acquire might not behave as a full memory
barrier, this requires an explicit memory barrier.

> Therefore add an explicit smp_mb__after_atomic() to cure things.
> 
> Also remove the smp_mb__before_atomic() on the dec side, as its
> completely pointless. We must rely on flush_tlb_range() to DTRT.

Good. It seemed fishy to me, but I was focused on the TLB consistency and
less on the barriers (that’s my excuse).

Nadav


^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-11 11:56     ` Ingo Molnar
  2017-08-11 12:17       ` Peter Zijlstra
@ 2017-08-11 14:04       ` Peter Zijlstra
  2017-08-13  6:06         ` Nadav Amit
  2017-08-14  3:09         ` Minchan Kim
  1 sibling, 2 replies; 87+ messages in thread
From: Peter Zijlstra @ 2017-08-11 14:04 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Stephen Rothwell, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Linux-Next Mailing List,
	Linux Kernel Mailing List, Nadav Amit, Linus, minchan


Ok, so I have the below to still go on-top.

Ideally someone would clarify the situation around
mm_tlb_flush_nested(), because ideally we'd remove the
smp_mb__after_atomic() and go back to relying on PTL alone.

This also removes the pointless smp_mb__before_atomic()

---
Subject: mm: Fix barriers for the tlb_flush_pending thing
From: Peter Zijlstra <peterz@infradead.org>
Date: Fri Aug 11 12:43:33 CEST 2017

I'm not 100% sure we always care about the same PTL and when we have
SPLIT_PTE_PTLOCKS and have RCpc locks (PPC) the UNLOCK of one does not
in fact order against the LOCK of another lock. Therefore the
documented scheme does not work if we care about multiple PTLs

mm_tlb_flush_pending() appears to only care about a single PTL:

 - arch pte_accessible() (x86, arm64) only cares about that one PTE.
 - do_huge_pmd_numa_page() also only cares about a single (huge) page.
 - ksm write_protect_page() also only cares about a single page.

however mm_tlb_flush_nested() is a mystery, it appears to care about
anything inside the range. For now rely on it doing at least _a_ PTL
lock instead of taking  _the_ PTL lock.

Therefore add an explicit smp_mb__after_atomic() to cure things.

Also remove the smp_mb__before_atomic() on the dec side, as its
completely pointless. We must rely on flush_tlb_range() to DTRT.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 include/linux/mm_types.h |   38 ++++++++++++++++++++++----------------
 1 file changed, 22 insertions(+), 16 deletions(-)

--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -537,13 +537,13 @@ static inline bool mm_tlb_flush_pending(
 {
 	/*
 	 * Must be called with PTL held; such that our PTL acquire will have
-	 * observed the store from set_tlb_flush_pending().
+	 * observed the increment from inc_tlb_flush_pending().
 	 */
-	return atomic_read(&mm->tlb_flush_pending) > 0;
+	return atomic_read(&mm->tlb_flush_pending);
 }
 
 /*
- * Returns true if there are two above TLB batching threads in parallel.
+ * Returns true if there are two or more TLB batching threads in parallel.
  */
 static inline bool mm_tlb_flush_nested(struct mm_struct *mm)
 {
@@ -558,15 +558,12 @@ static inline void init_tlb_flush_pendin
 static inline void inc_tlb_flush_pending(struct mm_struct *mm)
 {
 	atomic_inc(&mm->tlb_flush_pending);
-
 	/*
-	 * The only time this value is relevant is when there are indeed pages
-	 * to flush. And we'll only flush pages after changing them, which
-	 * requires the PTL.
-	 *
 	 * So the ordering here is:
 	 *
 	 *	atomic_inc(&mm->tlb_flush_pending);
+	 *	smp_mb__after_atomic();
+	 *
 	 *	spin_lock(&ptl);
 	 *	...
 	 *	set_pte_at();
@@ -580,21 +577,30 @@ static inline void inc_tlb_flush_pending
 	 *	flush_tlb_range();
 	 *	atomic_dec(&mm->tlb_flush_pending);
 	 *
-	 * So the =true store is constrained by the PTL unlock, and the =false
-	 * store is constrained by the TLB invalidate.
+	 * Where we order the increment against the PTE modification with the
+	 * smp_mb__after_atomic(). It would appear that the spin_unlock(&ptl)
+	 * is sufficient to constrain the inc, because we only care about the
+	 * value if there is indeed a pending PTE modification. However with
+	 * SPLIT_PTE_PTLOCKS and RCpc locks (PPC) the UNLOCK of one lock does
+	 * not order against the LOCK of another lock.
+	 *
+	 * The decrement is ordered by the flush_tlb_range(), such that
+	 * mm_tlb_flush_pending() will not return false unless all flushes have
+	 * completed.
 	 */
+	smp_mb__after_atomic();
 }
 
-/* Clearing is done after a TLB flush, which also provides a barrier. */
 static inline void dec_tlb_flush_pending(struct mm_struct *mm)
 {
 	/*
-	 * Guarantee that the tlb_flush_pending does not not leak into the
-	 * critical section, since we must order the PTE change and changes to
-	 * the pending TLB flush indication. We could have relied on TLB flush
-	 * as a memory barrier, but this behavior is not clearly documented.
+	 * See inc_tlb_flush_pending().
+	 *
+	 * This cannot be smp_mb__before_atomic() because smp_mb() simply does
+	 * not order against TLB invalidate completion, which is what we need.
+	 *
+	 * Therefore we must rely on tlb_flush_*() to guarantee order.
 	 */
-	smp_mb__before_atomic();
 	atomic_dec(&mm->tlb_flush_pending);
 }
 

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-11 12:44         ` Ingo Molnar
@ 2017-08-11 13:49           ` Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2017-08-11 13:49 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Peter Zijlstra, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Linux-Next Mailing List,
	Linux Kernel Mailing List, Nadav Amit, Linus

Hi Ingo,

On Fri, 11 Aug 2017 14:44:25 +0200 Ingo Molnar <mingo@kernel.org> wrote:
>
> * Peter Zijlstra <peterz@infradead.org> wrote:
> 
> > On Fri, Aug 11, 2017 at 01:56:07PM +0200, Ingo Molnar wrote:  
> > > I've done a minimal conflict resolution merge locally. Peter, could you please 
> > > double check my resolution, in:
> > > 
> > >   040cca3ab2f6: Merge branch 'linus' into locking/core, to resolve conflicts  
> > 
> > That merge is a bit wonky, but not terminally broken afaict.
> > 
> > It now does two TLB flushes, the below cleans that up.  
> 
> Cool, thanks - I've applied it as a separate commit, to reduce the evilness of the 
> merge commit.
> 
> Will push it all out in time to make Stephen's Monday morning a bit less of a 
> Monday morning.

Thanks you very much.

-- 
Cheers,
Stephen Rothwell

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-11 12:17       ` Peter Zijlstra
@ 2017-08-11 12:44         ` Ingo Molnar
  2017-08-11 13:49           ` Stephen Rothwell
  0 siblings, 1 reply; 87+ messages in thread
From: Ingo Molnar @ 2017-08-11 12:44 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Stephen Rothwell, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Linux-Next Mailing List,
	Linux Kernel Mailing List, Nadav Amit, Linus


* Peter Zijlstra <peterz@infradead.org> wrote:

> On Fri, Aug 11, 2017 at 01:56:07PM +0200, Ingo Molnar wrote:
> > I've done a minimal conflict resolution merge locally. Peter, could you please 
> > double check my resolution, in:
> > 
> >   040cca3ab2f6: Merge branch 'linus' into locking/core, to resolve conflicts
> 
> That merge is a bit wonky, but not terminally broken afaict.
> 
> It now does two TLB flushes, the below cleans that up.

Cool, thanks - I've applied it as a separate commit, to reduce the evilness of the 
merge commit.

Will push it all out in time to make Stephen's Monday morning a bit less of a 
Monday morning.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-11 11:56     ` Ingo Molnar
@ 2017-08-11 12:17       ` Peter Zijlstra
  2017-08-11 12:44         ` Ingo Molnar
  2017-08-11 14:04       ` Peter Zijlstra
  1 sibling, 1 reply; 87+ messages in thread
From: Peter Zijlstra @ 2017-08-11 12:17 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Stephen Rothwell, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Linux-Next Mailing List,
	Linux Kernel Mailing List, Nadav Amit, Linus

On Fri, Aug 11, 2017 at 01:56:07PM +0200, Ingo Molnar wrote:
> I've done a minimal conflict resolution merge locally. Peter, could you please 
> double check my resolution, in:
> 
>   040cca3ab2f6: Merge branch 'linus' into locking/core, to resolve conflicts

That merge is a bit wonky, but not terminally broken afaict.

It now does two TLB flushes, the below cleans that up.

---
 mm/huge_memory.c | 22 +++++-----------------
 1 file changed, 5 insertions(+), 17 deletions(-)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index ce883459e246..08f6c1993832 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1410,7 +1410,6 @@ int do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd)
 	unsigned long haddr = vmf->address & HPAGE_PMD_MASK;
 	int page_nid = -1, this_nid = numa_node_id();
 	int target_nid, last_cpupid = -1;
-	bool need_flush = false;
 	bool page_locked;
 	bool migrated = false;
 	bool was_writable;
@@ -1497,22 +1496,18 @@ int do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd)
 	}
 
 	/*
-	 * The page_table_lock above provides a memory barrier
-	 * with change_protection_range.
-	 */
-	if (mm_tlb_flush_pending(vma->vm_mm))
-		flush_tlb_range(vma, haddr, haddr + HPAGE_PMD_SIZE);
-
-	/*
 	 * Since we took the NUMA fault, we must have observed the !accessible
 	 * bit. Make sure all other CPUs agree with that, to avoid them
 	 * modifying the page we're about to migrate.
 	 *
 	 * Must be done under PTL such that we'll observe the relevant
-	 * set_tlb_flush_pending().
+	 * inc_tlb_flush_pending().
+	 *
+	 * We are not sure a pending tlb flush here is for a huge page
+	 * mapping or not. Hence use the tlb range variant
 	 */
 	if (mm_tlb_flush_pending(vma->vm_mm))
-		need_flush = true;
+		flush_tlb_range(vma, haddr, haddr + HPAGE_PMD_SIZE);
 
 	/*
 	 * Migrate the THP to the requested node, returns with page unlocked
@@ -1520,13 +1515,6 @@ int do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd)
 	 */
 	spin_unlock(vmf->ptl);
 
-	/*
-	 * We are not sure a pending tlb flush here is for a huge page
-	 * mapping or not. Hence use the tlb range variant
-	 */
-	if (need_flush)
-		flush_tlb_range(vma, haddr, haddr + HPAGE_PMD_SIZE);
-
 	migrated = migrate_misplaced_transhuge_page(vma->vm_mm, vma,
 				vmf->pmd, pmd, vmf->address, page, target_nid);
 	if (migrated) {

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-11 11:45   ` Stephen Rothwell
@ 2017-08-11 11:56     ` Ingo Molnar
  2017-08-11 12:17       ` Peter Zijlstra
  2017-08-11 14:04       ` Peter Zijlstra
  0 siblings, 2 replies; 87+ messages in thread
From: Ingo Molnar @ 2017-08-11 11:56 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Peter Zijlstra, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Linux-Next Mailing List,
	Linux Kernel Mailing List, Nadav Amit, Linus


* Stephen Rothwell <sfr@canb.auug.org.au> wrote:

> Hi Peter,
> 
> On Fri, 11 Aug 2017 11:34:49 +0200 Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > On Fri, Aug 11, 2017 at 05:53:26PM +1000, Stephen Rothwell wrote:
> > > 
> > > Today's linux-next merge of the akpm-current tree got conflicts in:
> > > 
> > >   include/linux/mm_types.h
> > >   mm/huge_memory.c
> > > 
> > > between commit:
> > > 
> > >   8b1b436dd1cc ("mm, locking: Rework {set,clear,mm}_tlb_flush_pending()")
> > > 
> > > from the tip tree and commits:
> > > 
> > >   16af97dc5a89 ("mm: migrate: prevent racy access to tlb_flush_pending")
> > >   a9b802500ebb ("Revert "mm: numa: defer TLB flush for THP migration as long as possible"")
> > > 
> > > from the akpm-current tree.
> > > 
> > > The latter 2 are now in Linus' tree as well (but were not when I started
> > > the day).
> >
> > Here's two patches that apply on top of tip.
> 
> What I will really need (on Monday) is a merge resolution between
> Linus' tree and the tip tree ...

I've done a minimal conflict resolution merge locally. Peter, could you please 
double check my resolution, in:

  040cca3ab2f6: Merge branch 'linus' into locking/core, to resolve conflicts

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-11  9:34 ` Peter Zijlstra
  2017-08-11 10:48   ` Peter Zijlstra
@ 2017-08-11 11:45   ` Stephen Rothwell
  2017-08-11 11:56     ` Ingo Molnar
  1 sibling, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2017-08-11 11:45 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Linux-Next Mailing List, Linux Kernel Mailing List, Nadav Amit,
	Linus

Hi Peter,

On Fri, 11 Aug 2017 11:34:49 +0200 Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Fri, Aug 11, 2017 at 05:53:26PM +1000, Stephen Rothwell wrote:
> > 
> > Today's linux-next merge of the akpm-current tree got conflicts in:
> > 
> >   include/linux/mm_types.h
> >   mm/huge_memory.c
> > 
> > between commit:
> > 
> >   8b1b436dd1cc ("mm, locking: Rework {set,clear,mm}_tlb_flush_pending()")
> > 
> > from the tip tree and commits:
> > 
> >   16af97dc5a89 ("mm: migrate: prevent racy access to tlb_flush_pending")
> >   a9b802500ebb ("Revert "mm: numa: defer TLB flush for THP migration as long as possible"")
> > 
> > from the akpm-current tree.
> > 
> > The latter 2 are now in Linus' tree as well (but were not when I started
> > the day).
>
> Here's two patches that apply on top of tip.

What I will really need (on Monday) is a merge resolution between
Linus' tree and the tip tree ...

-- 
Cheers,
Stephen Rothwell

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-11  9:34 ` Peter Zijlstra
@ 2017-08-11 10:48   ` Peter Zijlstra
  2017-08-11 11:45   ` Stephen Rothwell
  1 sibling, 0 replies; 87+ messages in thread
From: Peter Zijlstra @ 2017-08-11 10:48 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Linux-Next Mailing List, Linux Kernel Mailing List, Nadav Amit,
	Linus

On Fri, Aug 11, 2017 at 11:34:49AM +0200, Peter Zijlstra wrote:
> On Fri, Aug 11, 2017 at 05:53:26PM +1000, Stephen Rothwell wrote:
> > Hi all,
> > 
> > Today's linux-next merge of the akpm-current tree got conflicts in:
> > 
> >   include/linux/mm_types.h
> >   mm/huge_memory.c
> > 
> > between commit:
> > 
> >   8b1b436dd1cc ("mm, locking: Rework {set,clear,mm}_tlb_flush_pending()")
> > 
> > from the tip tree and commits:
> > 
> >   16af97dc5a89 ("mm: migrate: prevent racy access to tlb_flush_pending")
> >   a9b802500ebb ("Revert "mm: numa: defer TLB flush for THP migration as long as possible"")
> > 
> > from the akpm-current tree.
> > 
> > The latter 2 are now in Linus' tree as well (but were not when I started
> > the day).
> > 
> > The only way forward I could see was to revert
> > 
> >   8b1b436dd1cc ("mm, locking: Rework {set,clear,mm}_tlb_flush_pending()")
> > 
> > and the three following commits
> > 
> >   ff7a5fb0f1d5 ("overlayfs, locking: Remove smp_mb__before_spinlock() usage")
> >   d89e588ca408 ("locking: Introduce smp_mb__after_spinlock()")
> >   a9668cd6ee28 ("locking: Remove smp_mb__before_spinlock()")
> > 
> > before merging the akpm-current tree again.
> 
> Here's two patches that apply on top of tip.
> 


And here's one to fix the PPC ordering issue I found while doing those
patches.


---
Subject: mm: Fix barrier for inc_tlb_flush_pending() for PPC
From: Peter Zijlstra <peterz@infradead.org>
Date: Fri Aug 11 12:43:33 CEST 2017

When we have SPLIT_PTE_PTLOCKS and have RCpc locks (PPC) the UNLOCK of
one does not in fact order against the LOCK of another lock. Therefore
the documented scheme does not work.

Add an explicit smp_mb__after_atomic() to cure things.

Also update the comment to reflect the new inc/dec thing.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 include/linux/mm_types.h |   34 ++++++++++++++++++++++++----------
 1 file changed, 24 insertions(+), 10 deletions(-)

--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -533,7 +533,7 @@ static inline bool mm_tlb_flush_pending(
 {
 	/*
 	 * Must be called with PTL held; such that our PTL acquire will have
-	 * observed the store from set_tlb_flush_pending().
+	 * observed the increment from inc_tlb_flush_pending().
 	 */
 	return atomic_read(&mm->tlb_flush_pending);
 }
@@ -547,13 +547,11 @@ static inline void inc_tlb_flush_pending
 {
 	atomic_inc(&mm->tlb_flush_pending);
 	/*
-	 * The only time this value is relevant is when there are indeed pages
-	 * to flush. And we'll only flush pages after changing them, which
-	 * requires the PTL.
-	 *
 	 * So the ordering here is:
 	 *
-	 *	mm->tlb_flush_pending = true;
+	 *	atomic_inc(&mm->tlb_flush_pending)
+	 *	smp_mb__after_atomic();
+	 *
 	 *	spin_lock(&ptl);
 	 *	...
 	 *	set_pte_at();
@@ -565,17 +563,33 @@ static inline void inc_tlb_flush_pending
 	 *				spin_unlock(&ptl);
 	 *
 	 *	flush_tlb_range();
-	 *	mm->tlb_flush_pending = false;
+	 *	atomic_dec(&mm->tlb_flush_pending);
 	 *
-	 * So the =true store is constrained by the PTL unlock, and the =false
-	 * store is constrained by the TLB invalidate.
+	 * Where we order the increment against the PTE modification with the
+	 * smp_mb__after_atomic(). It would appear that the spin_unlock(&ptl)
+	 * is sufficient to constrain the inc, because we only care about the
+	 * value if there is indeed a pending PTE modification. However with
+	 * SPLIT_PTE_PTLOCKS and RCpc locks (PPC) the UNLOCK of one lock does
+	 * not order against the LOCK of another lock.
+	 *
+	 * The decrement is ordered by the flush_tlb_range(), such that
+	 * mm_tlb_flush_pending() will not return false unless all flushes have
+	 * completed.
 	 */
+	smp_mb__after_atomic();
 }
 
 /* Clearing is done after a TLB flush, which also provides a barrier. */
 static inline void dec_tlb_flush_pending(struct mm_struct *mm)
 {
-	/* see set_tlb_flush_pending */
+	/*
+	 * See inc_tlb_flush_pending().
+	 *
+	 * This cannot be smp_mb__before_atomic() because smp_mb() simply does
+	 * not order against TLB invalidate completion, which is what we need.
+	 *
+	 * Therefore we must rely on tlb_flush_*() to guarantee order.
+	 */
 	atomic_dec(&mm->tlb_flush_pending);
 }
 #else

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-08-11  7:53 Stephen Rothwell
@ 2017-08-11  9:34 ` Peter Zijlstra
  2017-08-11 10:48   ` Peter Zijlstra
  2017-08-11 11:45   ` Stephen Rothwell
  0 siblings, 2 replies; 87+ messages in thread
From: Peter Zijlstra @ 2017-08-11  9:34 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Linux-Next Mailing List, Linux Kernel Mailing List, Nadav Amit,
	Linus

[-- Attachment #1: Type: text/plain, Size: 1108 bytes --]

On Fri, Aug 11, 2017 at 05:53:26PM +1000, Stephen Rothwell wrote:
> Hi all,
> 
> Today's linux-next merge of the akpm-current tree got conflicts in:
> 
>   include/linux/mm_types.h
>   mm/huge_memory.c
> 
> between commit:
> 
>   8b1b436dd1cc ("mm, locking: Rework {set,clear,mm}_tlb_flush_pending()")
> 
> from the tip tree and commits:
> 
>   16af97dc5a89 ("mm: migrate: prevent racy access to tlb_flush_pending")
>   a9b802500ebb ("Revert "mm: numa: defer TLB flush for THP migration as long as possible"")
> 
> from the akpm-current tree.
> 
> The latter 2 are now in Linus' tree as well (but were not when I started
> the day).
> 
> The only way forward I could see was to revert
> 
>   8b1b436dd1cc ("mm, locking: Rework {set,clear,mm}_tlb_flush_pending()")
> 
> and the three following commits
> 
>   ff7a5fb0f1d5 ("overlayfs, locking: Remove smp_mb__before_spinlock() usage")
>   d89e588ca408 ("locking: Introduce smp_mb__after_spinlock()")
>   a9668cd6ee28 ("locking: Remove smp_mb__before_spinlock()")
> 
> before merging the akpm-current tree again.

Here's two patches that apply on top of tip.


[-- Attachment #2: nadav_amit-mm-migrate__prevent_racy_access_to_tlb_flush_pending.patch --]
[-- Type: text/x-diff, Size: 4923 bytes --]

Subject: mm: migrate: prevent racy access to tlb_flush_pending
From: Nadav Amit <nadav.amit@gmail.com>
Date: Tue, 1 Aug 2017 17:08:12 -0700

Setting and clearing mm->tlb_flush_pending can be performed by multiple
threads, since mmap_sem may only be acquired for read in
task_numa_work(). If this happens, tlb_flush_pending might be cleared
while one of the threads still changes PTEs and batches TLB flushes.

This can lead to the same race between migration and
change_protection_range() that led to the introduction of
tlb_flush_pending. The result of this race was data corruption, which
means that this patch also addresses a theoretically possible data
corruption.

An actual data corruption was not observed, yet the race was
was confirmed by adding assertion to check tlb_flush_pending is not set
by two threads, adding artificial latency in change_protection_range()
and using sysctl to reduce kernel.numa_balancing_scan_delay_ms.

Fixes: 20841405940e ("mm: fix TLB flush race between migration, and
change_protection_range")


Cc: <akpm@linux-foundation.org>
Cc: CC:     <nadav.amit@gmail.com>
Cc: Andy Lutomirski <luto@kernel.org>
Signed-off-by: Nadav Amit <namit@vmware.com>
Acked-by: Mel Gorman <mgorman@suse.de>
Acked-by: Rik van Riel <riel@redhat.com>
Acked-by: Minchan Kim <minchan@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20170802000818.4760-2-namit@vmware.com
---
 include/linux/mm_types.h |   29 +++++++++++++++++++++--------
 kernel/fork.c            |    2 +-
 mm/debug.c               |    2 +-
 mm/mprotect.c            |    4 ++--
 4 files changed, 25 insertions(+), 12 deletions(-)

--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -493,7 +493,7 @@ struct mm_struct {
 	 * can move process memory needs to flush the TLB when moving a
 	 * PROT_NONE or PROT_NUMA mapped page.
 	 */
-	bool tlb_flush_pending;
+	atomic_t tlb_flush_pending;
 #endif
 #ifdef CONFIG_ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH
 	/* See flush_tlb_batched_pending() */
@@ -535,11 +535,17 @@ static inline bool mm_tlb_flush_pending(
 	 * Must be called with PTL held; such that our PTL acquire will have
 	 * observed the store from set_tlb_flush_pending().
 	 */
-	return mm->tlb_flush_pending;
+	return atomic_read(&mm->tlb_flush_pending);
 }
-static inline void set_tlb_flush_pending(struct mm_struct *mm)
+
+static inline void init_tlb_flush_pending(struct mm_struct *mm)
 {
-	mm->tlb_flush_pending = true;
+	atomic_set(&mm->tlb_flush_pending, 0);
+}
+
+static inline void inc_tlb_flush_pending(struct mm_struct *mm)
+{
+	atomic_inc(&mm->tlb_flush_pending);
 	/*
 	 * The only time this value is relevant is when there are indeed pages
 	 * to flush. And we'll only flush pages after changing them, which
@@ -565,21 +571,28 @@ static inline void set_tlb_flush_pending
 	 * store is constrained by the TLB invalidate.
 	 */
 }
+
 /* Clearing is done after a TLB flush, which also provides a barrier. */
-static inline void clear_tlb_flush_pending(struct mm_struct *mm)
+static inline void dec_tlb_flush_pending(struct mm_struct *mm)
 {
 	/* see set_tlb_flush_pending */
-	mm->tlb_flush_pending = false;
+	atomic_dec(&mm->tlb_flush_pending);
 }
 #else
 static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
 {
 	return false;
 }
-static inline void set_tlb_flush_pending(struct mm_struct *mm)
+
+static inline void init_tlb_flush_pending(struct mm_struct *mm)
 {
 }
-static inline void clear_tlb_flush_pending(struct mm_struct *mm)
+
+static inline void inc_tlb_flush_pending(struct mm_struct *mm)
+{
+}
+
+static inline void dec_tlb_flush_pending(struct mm_struct *mm)
 {
 }
 #endif
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -809,7 +809,7 @@ static struct mm_struct *mm_init(struct
 	mm_init_aio(mm);
 	mm_init_owner(mm, p);
 	mmu_notifier_mm_init(mm);
-	clear_tlb_flush_pending(mm);
+	init_tlb_flush_pending(mm);
 #if defined(CONFIG_TRANSPARENT_HUGEPAGE) && !USE_SPLIT_PMD_PTLOCKS
 	mm->pmd_huge_pte = NULL;
 #endif
--- a/mm/debug.c
+++ b/mm/debug.c
@@ -159,7 +159,7 @@ void dump_mm(const struct mm_struct *mm)
 		mm->numa_next_scan, mm->numa_scan_offset, mm->numa_scan_seq,
 #endif
 #if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
-		mm->tlb_flush_pending,
+		atomic_read(&mm->tlb_flush_pending),
 #endif
 		mm->def_flags, &mm->def_flags
 	);
--- a/mm/mprotect.c
+++ b/mm/mprotect.c
@@ -244,7 +244,7 @@ static unsigned long change_protection_r
 	BUG_ON(addr >= end);
 	pgd = pgd_offset(mm, addr);
 	flush_cache_range(vma, addr, end);
-	set_tlb_flush_pending(mm);
+	inc_tlb_flush_pending(mm);
 	do {
 		next = pgd_addr_end(addr, end);
 		if (pgd_none_or_clear_bad(pgd))
@@ -256,7 +256,7 @@ static unsigned long change_protection_r
 	/* Only flush the TLB if we actually modified any entries: */
 	if (pages)
 		flush_tlb_range(vma, start, end);
-	clear_tlb_flush_pending(mm);
+	dec_tlb_flush_pending(mm);
 
 	return pages;
 }

[-- Attachment #3: nadav_amit-revert__mm-numa__defer_tlb_flush_for_thp_migration_as_long_as_possible_.patch --]
[-- Type: text/x-diff, Size: 2576 bytes --]

Subject: Revert "mm: numa: defer TLB flush for THP migration as long as possible"
From:   Nadav Amit <namit@vmware.com>
Date: Tue, 1 Aug 2017 17:08:14 -0700

While deferring TLB flushes is a good practice, the reverted patch
caused pending TLB flushes to be checked while the page-table lock is
not taken. As a result, in architectures with weak memory model (PPC),
Linux may miss a memory-barrier, miss the fact TLB flushes are pending,
and cause (in theory) a memory corruption.

Since the alternative of using smp_mb__after_unlock_lock() was
considered a bit open-coded, and the performance impact is expected to
be small, the previous patch is reverted.

This reverts commit b0943d61b8fa420180f92f64ef67662b4f6cc493.

Cc: <akpm@linux-foundation.org>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: CC:     <nadav.amit@gmail.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Andy Lutomirski <luto@kernel.org>
Suggested-by: Mel Gorman <mgorman@suse.de>
Signed-off-by: Nadav Amit <namit@vmware.com>
Acked-by: Mel Gorman <mgorman@suse.de>
Acked-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20170802000818.4760-4-namit@vmware.com
---
 mm/huge_memory.c |   13 ++++---------
 1 file changed, 4 insertions(+), 9 deletions(-)

--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1410,7 +1410,6 @@ int do_huge_pmd_numa_page(struct vm_faul
 	unsigned long haddr = vmf->address & HPAGE_PMD_MASK;
 	int page_nid = -1, this_nid = numa_node_id();
 	int target_nid, last_cpupid = -1;
-	bool need_flush = false;
 	bool page_locked;
 	bool migrated = false;
 	bool was_writable;
@@ -1503,9 +1502,12 @@ int do_huge_pmd_numa_page(struct vm_faul
 	 *
 	 * Must be done under PTL such that we'll observe the relevant
 	 * set_tlb_flush_pending().
+	 *
+	 * We are not sure a pending tlb flush here is for a huge page
+	 * mapping or not. Hence use the tlb range variant
 	 */
 	if (mm_tlb_flush_pending(vma->vm_mm))
-		need_flush = true;
+		flush_tlb_range(vma, haddr, haddr + HPAGE_PMD_SIZE);
 
 	/*
 	 * Migrate the THP to the requested node, returns with page unlocked
@@ -1513,13 +1515,6 @@ int do_huge_pmd_numa_page(struct vm_faul
 	 */
 	spin_unlock(vmf->ptl);
 
-	/*
-	 * We are not sure a pending tlb flush here is for a huge page
-	 * mapping or not. Hence use the tlb range variant
-	 */
-	if (need_flush)
-		flush_tlb_range(vma, haddr, haddr + HPAGE_PMD_SIZE);
-
 	migrated = migrate_misplaced_transhuge_page(vma->vm_mm, vma,
 				vmf->pmd, pmd, vmf->address, page, target_nid);
 	if (migrated) {

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2017-08-11  7:53 Stephen Rothwell
  2017-08-11  9:34 ` Peter Zijlstra
  0 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2017-08-11  7:53 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: Linux-Next Mailing List, Linux Kernel Mailing List, Nadav Amit, Linus

Hi all,

Today's linux-next merge of the akpm-current tree got conflicts in:

  include/linux/mm_types.h
  mm/huge_memory.c

between commit:

  8b1b436dd1cc ("mm, locking: Rework {set,clear,mm}_tlb_flush_pending()")

from the tip tree and commits:

  16af97dc5a89 ("mm: migrate: prevent racy access to tlb_flush_pending")
  a9b802500ebb ("Revert "mm: numa: defer TLB flush for THP migration as long as possible"")

from the akpm-current tree.

The latter 2 are now in Linus' tree as well (but were not when I started
the day).

The only way forward I could see was to revert

  8b1b436dd1cc ("mm, locking: Rework {set,clear,mm}_tlb_flush_pending()")

and the three following commits

  ff7a5fb0f1d5 ("overlayfs, locking: Remove smp_mb__before_spinlock() usage")
  d89e588ca408 ("locking: Introduce smp_mb__after_spinlock()")
  a9668cd6ee28 ("locking: Remove smp_mb__before_spinlock()")

before merging the akpm-current tree again.

-- 
Cheers,
Stephen Rothwell

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-04-12 20:53 ` Vlastimil Babka
@ 2017-04-20  2:17   ` NeilBrown
  0 siblings, 0 replies; 87+ messages in thread
From: NeilBrown @ 2017-04-20  2:17 UTC (permalink / raw)
  To: Vlastimil Babka, Stephen Rothwell, Andrew Morton,
	Thomas Gleixner, Ingo Molnar, H. Peter Anvin, Peter Zijlstra
  Cc: Linux-Next Mailing List, Linux Kernel Mailing List, Michal Hocko

[-- Attachment #1: Type: text/plain, Size: 3430 bytes --]

On Wed, Apr 12 2017, Vlastimil Babka wrote:

> On 12.4.2017 8:46, Stephen Rothwell wrote:
>> Hi Andrew,
>> 
>> Today's linux-next merge of the akpm-current tree got conflicts in:
>> 
>>   drivers/block/nbd.c
>>   drivers/scsi/iscsi_tcp.c
>>   net/core/dev.c
>>   net/core/sock.c
>> 
>> between commit:
>> 
>>   717a94b5fc70 ("sched/core: Remove 'task' parameter and rename tsk_restore_flags() to current_restore_flags()")
>> 
>> from the tip tree and commit:
>> 
>>   61d5ad5b2e8a ("treewide: convert PF_MEMALLOC manipulations to new helpers")
>> 
>> from the akpm-current tree.
>
> Yeah, the first patch from Neil renames a function (as its subject says) and the
> second patch from me converts most of its users to new helpers specific to the
> PF_MEMALLOC flags.
>
>> I fixed it up (the latter is just a superset of the former, so I used
>
> It's not a complete superset though, more on that below.
>
>> that) and can carry the fix as necessary. This is now fixed as far as
>> linux-next is concerned, but any non trivial conflicts should be mentioned
>> to your upstream maintainer when your tree is submitted for merging.
>> You may also want to consider cooperating with the maintainer of the
>> conflicting tree to minimise any particularly complex conflicts.
>
> Hmm I could redo my patch on top of Neil's patch, but then Andrew would have to
> carry Neil's patch as well just to have a working mmotm? And then make sure to
> send my patch (but not Neil's) only after the tip tree is pulled? Would that
> work for the maintainers involved?
>
>> It looks like there may be more instances that the latter patch should
>> update.
>
> I see two remaining instances of current_restore_flags(). One in __do_softirq()
> is even for PF_MEMALLOC, but there the flag is cleared first and then set back,
> which is opposite of the common case that my helpers provide. The other in nfsd
> is for PF_LESS_THROTTLE which is not common enough to earn own helpers yet. IIRC
> Neil originally wanted to add a new one?

[Sorry - I thought I had sent this last week, but just noticed that I didn't]

In general, I'm not a fan of overly-specific helpers.

As a general rule, tsk_restore_flags() is probably better than
current_restore_flags() as it is more general.
However in this specific case, using any task other than 'current' would
almost certainly be incorrect code as locking is impossible.  So I
prefer the 'current' to be implicit, but the actual flag to be explicit.

If you are going to add helpers for setting/clearing PF flags, I would
much rather that you take

#define current_test_flags(f)   (current->flags & (f))
#define current_set_flags_nested(sp, f)         \
                (*(sp) = current->flags, current->flags |= (f))
#define current_clear_flags_nested(sp, f)       \
                (*(sp) = current->flags, current->flags &= ~(f))
#define current_restore_flags_nested(sp, f)     \
                (current->flags = ((current->flags & ~(f)) | (*(sp) & (f))))

out of fs/xfs/xfs_linux.h and use them globally.

Your
  noreclaim_flag = memalloc_reclaim_save()
becomes
  current_set_flags_nested&noreclaim_flag, PF_MEMALLOC)
which is more typing, but arguably easier to read.

If you then changed all uses of tsk_restore_flags() to use
current_restore_flags_nested(), my patch could be discarded as
irrelevant.

Thanks,
NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2017-04-12  6:46 Stephen Rothwell
@ 2017-04-12 20:53 ` Vlastimil Babka
  2017-04-20  2:17   ` NeilBrown
  0 siblings, 1 reply; 87+ messages in thread
From: Vlastimil Babka @ 2017-04-12 20:53 UTC (permalink / raw)
  To: Stephen Rothwell, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Peter Zijlstra
  Cc: Linux-Next Mailing List, Linux Kernel Mailing List, NeilBrown,
	Michal Hocko

On 12.4.2017 8:46, Stephen Rothwell wrote:
> Hi Andrew,
> 
> Today's linux-next merge of the akpm-current tree got conflicts in:
> 
>   drivers/block/nbd.c
>   drivers/scsi/iscsi_tcp.c
>   net/core/dev.c
>   net/core/sock.c
> 
> between commit:
> 
>   717a94b5fc70 ("sched/core: Remove 'task' parameter and rename tsk_restore_flags() to current_restore_flags()")
> 
> from the tip tree and commit:
> 
>   61d5ad5b2e8a ("treewide: convert PF_MEMALLOC manipulations to new helpers")
> 
> from the akpm-current tree.

Yeah, the first patch from Neil renames a function (as its subject says) and the
second patch from me converts most of its users to new helpers specific to the
PF_MEMALLOC flags.

> I fixed it up (the latter is just a superset of the former, so I used

It's not a complete superset though, more on that below.

> that) and can carry the fix as necessary. This is now fixed as far as
> linux-next is concerned, but any non trivial conflicts should be mentioned
> to your upstream maintainer when your tree is submitted for merging.
> You may also want to consider cooperating with the maintainer of the
> conflicting tree to minimise any particularly complex conflicts.

Hmm I could redo my patch on top of Neil's patch, but then Andrew would have to
carry Neil's patch as well just to have a working mmotm? And then make sure to
send my patch (but not Neil's) only after the tip tree is pulled? Would that
work for the maintainers involved?

> It looks like there may be more instances that the latter patch should
> update.

I see two remaining instances of current_restore_flags(). One in __do_softirq()
is even for PF_MEMALLOC, but there the flag is cleared first and then set back,
which is opposite of the common case that my helpers provide. The other in nfsd
is for PF_LESS_THROTTLE which is not common enough to earn own helpers yet. IIRC
Neil originally wanted to add a new one?

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2017-04-12  6:46 Stephen Rothwell
  2017-04-12 20:53 ` Vlastimil Babka
  0 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2017-04-12  6:46 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: Linux-Next Mailing List, Linux Kernel Mailing List,
	Vlastimil Babka, NeilBrown

Hi Andrew,

Today's linux-next merge of the akpm-current tree got conflicts in:

  drivers/block/nbd.c
  drivers/scsi/iscsi_tcp.c
  net/core/dev.c
  net/core/sock.c

between commit:

  717a94b5fc70 ("sched/core: Remove 'task' parameter and rename tsk_restore_flags() to current_restore_flags()")

from the tip tree and commit:

  61d5ad5b2e8a ("treewide: convert PF_MEMALLOC manipulations to new helpers")

from the akpm-current tree.

I fixed it up (the latter is just a superset of the former, so I used
that) and can carry the fix as necessary. This is now fixed as far as
linux-next is concerned, but any non trivial conflicts should be mentioned
to your upstream maintainer when your tree is submitted for merging.
You may also want to consider cooperating with the maintainer of the
conflicting tree to minimise any particularly complex conflicts.

It looks like there may be more instances that the latter patch should
update.
-- 
Cheers,
Stephen Rothwell

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2017-03-24  5:25 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2017-03-24  5:25 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Dmitry Vyukov

Hi all,

Today's linux-next merge of the akpm-current tree got conflicts in:

  arch/x86/include/asm/atomic.h
  arch/x86/include/asm/atomic64_64.h

between commits:

  a9ebf306f52c ("locking/atomic: Introduce atomic_try_cmpxchg()")
  e6790e4b5d5e ("locking/atomic/x86: Use atomic_try_cmpxchg()")

from the tip tree and commit:

  3f4ca3d25e1a ("asm-generic, x86: wrap atomic operations")

from the akpm-current tree.

I fixed it up (see below - though more work is probably needed) and can
carry the fix as necessary. This is now fixed as far as linux-next is
concerned, but any non trivial conflicts should be mentioned to your
upstream maintainer when your tree is submitted for merging.  You may
also want to consider cooperating with the maintainer of the conflicting
tree to minimise any particularly complex conflicts.

The below resolution is not quite right so I added this on top:

From: Stephen Rothwell <sfr@canb.auug.org.au>
Date: Fri, 24 Mar 2017 16:14:42 +1100
Subject: [PATCH] fix for bad merge fix

Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
---
 arch/x86/include/asm/atomic.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/atomic.h b/arch/x86/include/asm/atomic.h
index fc4412567a4a..f717b73182e7 100644
--- a/arch/x86/include/asm/atomic.h
+++ b/arch/x86/include/asm/atomic.h
@@ -217,7 +217,7 @@ static inline void arch_atomic_##op(int i, atomic_t *v)			\
 }
 
 #define ATOMIC_FETCH_OP(op, c_op)					\
-static inline int atomic_fetch_##op(int i, atomic_t *v)			\
+static inline int arch_atomic_fetch_##op(int i, atomic_t *v)		\
 {									\
 	int val = arch_atomic_read(v);					\
 	do {								\
-- 
2.11.0

-- 
Cheers,
Stephen Rothwell

diff --cc arch/x86/include/asm/atomic.h
index caa5798c92f4,95dd167eb3af..000000000000
--- a/arch/x86/include/asm/atomic.h
+++ b/arch/x86/include/asm/atomic.h
@@@ -181,20 -191,14 +191,20 @@@ static __always_inline int arch_atomic_
  	return xadd(&v->counter, -i);
  }
  
- static __always_inline int atomic_cmpxchg(atomic_t *v, int old, int new)
+ static __always_inline int arch_atomic_cmpxchg(atomic_t *v, int old, int new)
  {
- 	return cmpxchg(&v->counter, old, new);
+ 	return arch_cmpxchg(&v->counter, old, new);
  }
  
 +#define atomic_try_cmpxchg atomic_try_cmpxchg
 +static __always_inline bool atomic_try_cmpxchg(atomic_t *v, int *old, int new)
 +{
 +	return try_cmpxchg(&v->counter, old, new);
 +}
 +
- static inline int atomic_xchg(atomic_t *v, int new)
+ static inline int arch_atomic_xchg(atomic_t *v, int new)
  {
- 	return xchg(&v->counter, new);
+ 	return arch_xchg(&v->counter, new);
  }
  
  #define ATOMIC_OP(op)							\
@@@ -207,12 -211,16 +217,12 @@@ static inline void arch_atomic_##op(in
  }
  
  #define ATOMIC_FETCH_OP(op, c_op)					\
 -static inline int arch_atomic_fetch_##op(int i, atomic_t *v)		\
 +static inline int atomic_fetch_##op(int i, atomic_t *v)			\
  {									\
- 	int val = atomic_read(v);					\
 -	int old, val = arch_atomic_read(v);				\
 -	for (;;) {							\
 -		old = arch_atomic_cmpxchg(v, val, val c_op i);		\
 -		if (old == val)						\
 -			break;						\
 -		val = old;						\
 -	}								\
 -	return old;							\
++	int val = arch_atomic_read(v);					\
 +	do {								\
 +	} while (!atomic_try_cmpxchg(v, &val, val c_op i));		\
 +	return val;							\
  }
  
  #define ATOMIC_OPS(op, c_op)						\
@@@ -236,13 -244,18 +246,13 @@@ ATOMIC_OPS(xor, ^
   * Atomically adds @a to @v, so long as @v was not already @u.
   * Returns the old value of @v.
   */
- static __always_inline int __atomic_add_unless(atomic_t *v, int a, int u)
+ static __always_inline int __arch_atomic_add_unless(atomic_t *v, int a, int u)
  {
- 	int c = atomic_read(v);
 -	int c, old;
 -	c = arch_atomic_read(v);
 -	for (;;) {
 -		if (unlikely(c == (u)))
 -			break;
 -		old = arch_atomic_cmpxchg((v), c, c + (a));
 -		if (likely(old == c))
++	int c = arch_atomic_read(v);
 +	do {
 +		if (unlikely(c == u))
  			break;
 -		c = old;
 -	}
 +	} while (!atomic_try_cmpxchg(v, &c, c + a));
  	return c;
  }
  
diff --cc arch/x86/include/asm/atomic64_64.h
index 6189a433c9a9,de9555d35cb0..000000000000
--- a/arch/x86/include/asm/atomic64_64.h
+++ b/arch/x86/include/asm/atomic64_64.h
@@@ -168,23 -168,17 +168,23 @@@ static inline long arch_atomic64_fetch_
  	return xadd(&v->counter, -i);
  }
  
- #define atomic64_inc_return(v)  (atomic64_add_return(1, (v)))
- #define atomic64_dec_return(v)  (atomic64_sub_return(1, (v)))
+ #define arch_atomic64_inc_return(v)  (arch_atomic64_add_return(1, (v)))
+ #define arch_atomic64_dec_return(v)  (arch_atomic64_sub_return(1, (v)))
  
- static inline long atomic64_cmpxchg(atomic64_t *v, long old, long new)
+ static inline long arch_atomic64_cmpxchg(atomic64_t *v, long old, long new)
  {
- 	return cmpxchg(&v->counter, old, new);
+ 	return arch_cmpxchg(&v->counter, old, new);
  }
  
 +#define atomic64_try_cmpxchg atomic64_try_cmpxchg
 +static __always_inline bool atomic64_try_cmpxchg(atomic64_t *v, long *old, long new)
 +{
 +	return try_cmpxchg(&v->counter, old, new);
 +}
 +
- static inline long atomic64_xchg(atomic64_t *v, long new)
+ static inline long arch_atomic64_xchg(atomic64_t *v, long new)
  {
- 	return xchg(&v->counter, new);
+ 	return arch_xchg(&v->counter, new);
  }
  
  /**
@@@ -196,29 -190,35 +196,29 @@@
   * Atomically adds @a to @v, so long as it was not @u.
   * Returns the old value of @v.
   */
- static inline bool atomic64_add_unless(atomic64_t *v, long a, long u)
+ static inline bool arch_atomic64_add_unless(atomic64_t *v, long a, long u)
  {
- 	long c = atomic64_read(v);
 -	long c, old;
 -	c = arch_atomic64_read(v);
 -	for (;;) {
 -		if (unlikely(c == (u)))
 -			break;
 -		old = arch_atomic64_cmpxchg((v), c, c + (a));
 -		if (likely(old == c))
 -			break;
 -		c = old;
 -	}
 -	return c != (u);
++	long c = arch_atomic64_read(v);
 +	do {
 +		if (unlikely(c == u))
 +			return false;
 +	} while (!atomic64_try_cmpxchg(v, &c, c + a));
 +	return true;
  }
  
- #define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1, 0)
+ #define arch_atomic64_inc_not_zero(v) arch_atomic64_add_unless((v), 1, 0)
  
  /*
-  * atomic64_dec_if_positive - decrement by 1 if old value positive
+  * arch_atomic64_dec_if_positive - decrement by 1 if old value positive
   * @v: pointer of type atomic_t
   *
   * The function returns the old value of *v minus 1, even if
   * the atomic variable, v, was not decremented.
   */
- static inline long atomic64_dec_if_positive(atomic64_t *v)
+ static inline long arch_atomic64_dec_if_positive(atomic64_t *v)
  {
- 	long dec, c = atomic64_read(v);
 -	long c, old, dec;
 -	c = arch_atomic64_read(v);
 -	for (;;) {
++	long dec, c = arch_atomic64_read(v);
 +	do {
  		dec = c - 1;
  		if (unlikely(dec < 0))
  			break;
@@@ -236,12 -240,16 +236,12 @@@ static inline void arch_atomic64_##op(l
  }
  
  #define ATOMIC64_FETCH_OP(op, c_op)					\
- static inline long atomic64_fetch_##op(long i, atomic64_t *v)		\
+ static inline long arch_atomic64_fetch_##op(long i, atomic64_t *v)	\
  {									\
- 	long val = atomic64_read(v);					\
 -	long old, val = arch_atomic64_read(v);				\
 -	for (;;) {							\
 -		old = arch_atomic64_cmpxchg(v, val, val c_op i);	\
 -		if (old == val)						\
 -			break;						\
 -		val = old;						\
 -	}								\
 -	return old;							\
++	long val = arch_atomic64_read(v);				\
 +	do {								\
 +	} while (!atomic64_try_cmpxchg(v, &val, val c_op i));		\
 +	return val;							\
  }
  
  #define ATOMIC64_OPS(op, c_op)						\

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2017-02-17  4:40 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2017-02-17  4:40 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Frederic Weisbecker, Davidlohr Bueso

Hi all,

Today's linux-next merge of the akpm-current tree got conflicts in:

  arch/cris/include/asm/Kbuild
  arch/m32r/include/asm/Kbuild
  arch/parisc/include/asm/Kbuild
  arch/score/include/asm/Kbuild

between commit:

  b672592f0221 ("sched/cputime: Remove generic asm headers")

from the tip tree and commits:

  ccbd143eeee3 ("cris: use generic current.h")
  103c58f13b54 ("m32r: use generic current.h")
  35a25dde31aa ("score: remove asm/current.h")
  c6b552bc22c7 ("parisc: use generic current.h")

from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary. This
is now fixed as far as linux-next is concerned, but any non trivial
conflicts should be mentioned to your upstream maintainer when your tree
is submitted for merging.  You may also want to consider cooperating
with the maintainer of the conflicting tree to minimise any particularly
complex conflicts.

-- 
Cheers,
Stephen Rothwell

diff --cc arch/cris/include/asm/Kbuild
index 9f19e19bff9d,5e320f660c3c..000000000000
--- a/arch/cris/include/asm/Kbuild
+++ b/arch/cris/include/asm/Kbuild
@@@ -4,6 -4,8 +4,7 @@@ generic-y += barrier.
  generic-y += bitsperlong.h
  generic-y += clkdev.h
  generic-y += cmpxchg.h
 -generic-y += cputime.h
+ generic-y += current.h
  generic-y += device.h
  generic-y += div64.h
  generic-y += errno.h
diff --cc arch/m32r/include/asm/Kbuild
index 652100b64a71,30ee92ff0244..000000000000
--- a/arch/m32r/include/asm/Kbuild
+++ b/arch/m32r/include/asm/Kbuild
@@@ -1,5 -1,7 +1,6 @@@
  
  generic-y += clkdev.h
 -generic-y += cputime.h
+ generic-y += current.h
  generic-y += exec.h
  generic-y += irq_work.h
  generic-y += kvm_para.h
diff --cc arch/parisc/include/asm/Kbuild
index 4e179d770d69,7ac070267672..000000000000
--- a/arch/parisc/include/asm/Kbuild
+++ b/arch/parisc/include/asm/Kbuild
@@@ -2,6 -2,8 +2,7 @@@
  generic-y += auxvec.h
  generic-y += barrier.h
  generic-y += clkdev.h
 -generic-y += cputime.h
+ generic-y += current.h
  generic-y += device.h
  generic-y += div64.h
  generic-y += emergency-restart.h
diff --cc arch/score/include/asm/Kbuild
index 51970bb6c4fe,620970f837bc..000000000000
--- a/arch/score/include/asm/Kbuild
+++ b/arch/score/include/asm/Kbuild
@@@ -4,6 -4,8 +4,7 @@@ header-y +
  
  generic-y += barrier.h
  generic-y += clkdev.h
 -generic-y += cputime.h
+ generic-y += current.h
  generic-y += irq_work.h
  generic-y += mcs_spinlock.h
  generic-y += mm-arch-hooks.h

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2016-11-14  6:08 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2016-11-14  6:08 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Sebastian Andrzej Siewior, Vladimir Davydov

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in:

  mm/memcontrol.c

between commit:

  308167fcb330 ("mm/memcg: Convert to hotplug state machine")

from the tip tree and commit:

  2558c318449d ("mm: memcontrol: use special workqueue for creating per-memcg caches")

from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary. This
is now fixed as far as linux-next is concerned, but any non trivial
conflicts should be mentioned to your upstream maintainer when your tree
is submitted for merging.  You may also want to consider cooperating
with the maintainer of the conflicting tree to minimise any particularly
complex conflicts.

-- 
Cheers,
Stephen Rothwell

diff --cc mm/memcontrol.c
index 6c2043509fb5,91dfc7c5ce8f..000000000000
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@@ -5774,8 -5785,18 +5776,19 @@@ static int __init mem_cgroup_init(void
  {
  	int cpu, node;
  
+ #ifndef CONFIG_SLOB
+ 	/*
+ 	 * Kmem cache creation is mostly done with the slab_mutex held,
+ 	 * so use a special workqueue to avoid stalling all worker
+ 	 * threads in case lots of cgroups are created simultaneously.
+ 	 */
+ 	memcg_kmem_cache_create_wq =
+ 		alloc_ordered_workqueue("memcg_kmem_cache_create", 0);
+ 	BUG_ON(!memcg_kmem_cache_create_wq);
+ #endif
+ 
 -	hotcpu_notifier(memcg_cpu_hotplug_callback, 0);
 +	cpuhp_setup_state_nocalls(CPUHP_MM_MEMCQ_DEAD, "mm/memctrl:dead", NULL,
 +				  memcg_hotplug_cpu_dead);
  
  	for_each_possible_cpu(cpu)
  		INIT_WORK(&per_cpu_ptr(&memcg_stock, cpu)->work,

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2016-07-29  4:14 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2016-07-29  4:14 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Andy Lutomirski

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in:

  arch/x86/include/asm/thread_info.h

between commit:

  609c19a385c8 ("x86/ptrace: Stop setting TS_COMPAT in ptrace code")

from the tip tree and commit:

  58f9594bd42f ("signal: consolidate {TS,TLF}_RESTORE_SIGMASK code")

from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary. This
is now fixed as far as linux-next is concerned, but any non trivial
conflicts should be mentioned to your upstream maintainer when your tree
is submitted for merging.  You may also want to consider cooperating
with the maintainer of the conflicting tree to minimise any particularly
complex conflicts.

-- 
Cheers,
Stephen Rothwell

diff --cc arch/x86/include/asm/thread_info.h
index d4b0fd24a63e,b45ffdda3549..000000000000
--- a/arch/x86/include/asm/thread_info.h
+++ b/arch/x86/include/asm/thread_info.h
@@@ -263,35 -219,8 +263,11 @@@ static inline int arch_within_stack_fra
   * have to worry about atomic accesses.
   */
  #define TS_COMPAT		0x0002	/* 32bit syscall active (64BIT)*/
 +#ifdef CONFIG_COMPAT
 +#define TS_I386_REGS_POKED	0x0004	/* regs poked by 32-bit ptracer */
 +#endif
- #define TS_RESTORE_SIGMASK	0x0008	/* restore signal mask in do_signal() */
  
  #ifndef __ASSEMBLY__
- #define HAVE_SET_RESTORE_SIGMASK	1
- static inline void set_restore_sigmask(void)
- {
- 	struct thread_info *ti = current_thread_info();
- 	ti->status |= TS_RESTORE_SIGMASK;
- 	WARN_ON(!test_bit(TIF_SIGPENDING, (unsigned long *)&ti->flags));
- }
- static inline void clear_restore_sigmask(void)
- {
- 	current_thread_info()->status &= ~TS_RESTORE_SIGMASK;
- }
- static inline bool test_restore_sigmask(void)
- {
- 	return current_thread_info()->status & TS_RESTORE_SIGMASK;
- }
- static inline bool test_and_clear_restore_sigmask(void)
- {
- 	struct thread_info *ti = current_thread_info();
- 	if (!(ti->status & TS_RESTORE_SIGMASK))
- 		return false;
- 	ti->status &= ~TS_RESTORE_SIGMASK;
- 	return true;
- }
  
  static inline bool in_ia32_syscall(void)
  {

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2016-06-15  5:23 Stephen Rothwell
@ 2016-06-18 19:39 ` Manfred Spraul
  0 siblings, 0 replies; 87+ messages in thread
From: Manfred Spraul @ 2016-06-18 19:39 UTC (permalink / raw)
  To: Stephen Rothwell, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Peter Zijlstra
  Cc: linux-next, linux-kernel

Hi,

On 06/15/2016 07:23 AM, Stephen Rothwell wrote:
> Hi Andrew,
>
> Today's linux-next merge of the akpm-current tree got a conflict in:
>
>    ipc/sem.c
>
> between commit:
>
>    33ac279677dc ("locking/barriers: Introduce smp_acquire__after_ctrl_dep()")
>
> from the tip tree and commit:
>
>    a1c58ea067cb ("ipc/sem.c: Fix complex_count vs. simple op race")
>
> from the akpm-current tree.
Just in case, I have created a rediff of my patch against -tip.
And the patch with hysteresis would be ready as well.

I will send both patches.

More testers would be welcome, I can only test it on my laptop.

--
     Manfred

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2016-06-15  5:23 Stephen Rothwell
  2016-06-18 19:39 ` Manfred Spraul
  0 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2016-06-15  5:23 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Manfred Spraul

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in:

  ipc/sem.c

between commit:

  33ac279677dc ("locking/barriers: Introduce smp_acquire__after_ctrl_dep()")

from the tip tree and commit:

  a1c58ea067cb ("ipc/sem.c: Fix complex_count vs. simple op race")

from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary. This
is now fixed as far as linux-next is concerned, but any non trivial
conflicts should be mentioned to your upstream maintainer when your tree
is submitted for merging.  You may also want to consider cooperating
with the maintainer of the conflicting tree to minimise any particularly
complex conflicts.

-- 
Cheers,
Stephen Rothwell

diff --cc ipc/sem.c
index ae72b3cddc8d,11d9e605a619..000000000000
--- a/ipc/sem.c
+++ b/ipc/sem.c
@@@ -260,13 -267,20 +267,10 @@@ static void sem_rcu_free(struct rcu_hea
  }
  
  /*
-  * Wait until all currently ongoing simple ops have completed.
 - * spin_unlock_wait() and !spin_is_locked() are not memory barriers, they
 - * are only control barriers.
 - * The code must pair with spin_unlock(&sem->lock) or
 - * spin_unlock(&sem_perm.lock), thus just the control barrier is insufficient.
 - *
 - * smp_rmb() is sufficient, as writes cannot pass the control barrier.
 - */
 -#define ipc_smp_acquire__after_spin_is_unlocked()	smp_rmb()
 -
 -/*
+  * Enter the mode suitable for non-simple operations:
   * Caller must own sem_perm.lock.
-  * New simple ops cannot start, because simple ops first check
-  * that sem_perm.lock is free.
-  * that a) sem_perm.lock is free and b) complex_count is 0.
   */
- static void sem_wait_array(struct sem_array *sma)
+ static void complexmode_enter(struct sem_array *sma)
  {
  	int i;
  	struct sem *sem;

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2016-04-29  6:12 Stephen Rothwell
@ 2016-04-29  6:26 ` Ingo Molnar
  0 siblings, 0 replies; 87+ messages in thread
From: Ingo Molnar @ 2016-04-29  6:26 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra, linux-next, linux-kernel, Andy Shevchenko,
	Matt Fleming, Ard Biesheuvel


* Stephen Rothwell <sfr@canb.auug.org.au> wrote:

> Hi Andrew,
> 
> Today's linux-next merge of the akpm-current tree got a conflict in:
> 
>   include/linux/efi.h
> 
> between commit:
> 
>   2c23b73c2d02 ("Ard Biesheuvel <ard.biesheuvel@linaro.org>")
> 
> from the tip tree and commit:
> 
>   9f2c36a7b097 ("include/linux/efi.h: redefine type, constant, macro from generic code")
> 
> from the akpm-current tree.
> 
> I fixed it up (see below) and can carry the fix as necessary. This
> is now fixed as far as linux-next is concerned, but any non trivial
> conflicts should be mentioned to your upstream maintainer when your tree
> is submitted for merging.  You may also want to consider cooperating
> with the maintainer of the conflicting tree to minimise any particularly
> complex conflicts.

Btw., while looking at this, I noticed that akpm-current introduced this namespace 
collision:

include/acpi/acconfig.h:#define UUID_STRING_LENGTH          36  /* Total length of a UUID string */
include/linux/uuid.h:#define    UUID_STRING_LEN         36

I suspect the include/acpi/acconfig.h define should be renamed:

	UUID_STRING_LENGTH -> ACPI_UUID_STRING_LENGTH
	UUID_BUFFER_LENGTH -> ACPI_UUID_BUFFER_LENGTH

... before the collision causes any trouble.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2016-04-29  6:12 Stephen Rothwell
  2016-04-29  6:26 ` Ingo Molnar
  0 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2016-04-29  6:12 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Andy Shevchenko, Matt Fleming, Ard Biesheuvel

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in:

  include/linux/efi.h

between commit:

  2c23b73c2d02 ("Ard Biesheuvel <ard.biesheuvel@linaro.org>")

from the tip tree and commit:

  9f2c36a7b097 ("include/linux/efi.h: redefine type, constant, macro from generic code")

from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary. This
is now fixed as far as linux-next is concerned, but any non trivial
conflicts should be mentioned to your upstream maintainer when your tree
is submitted for merging.  You may also want to consider cooperating
with the maintainer of the conflicting tree to minimise any particularly
complex conflicts.

-- 
Cheers,
Stephen Rothwell

diff --cc include/linux/efi.h
index aa36fb8bea4b,5b1d5c5b4080..000000000000
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@@ -21,7 -21,7 +21,8 @@@
  #include <linux/pfn.h>
  #include <linux/pstore.h>
  #include <linux/reboot.h>
 +#include <linux/screen_info.h>
+ #include <linux/uuid.h>
  
  #include <asm/page.h>
  

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2016-03-02  5:40 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2016-03-02  5:40 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Dmitry Vyukov, Josh Poimboeuf

Hi Andrew,

Today's linux-next merge of the akpm-current tree got conflicts in:

  arch/x86/boot/Makefile
  arch/x86/boot/compressed/Makefile
  arch/x86/entry/vdso/Makefile
  arch/x86/kernel/Makefile
  arch/x86/realmode/rm/Makefile
  drivers/firmware/efi/libstub/Makefile

between commit:

  c0dd671686b2 ("objtool: Mark non-standard object files and directories")

from the tip tree and commit:

  9b1ad289b5e5 ("kernel: add kcov code coverage")

from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary (no action
is required).

-- 
Cheers,
Stephen Rothwell

diff --cc arch/x86/boot/Makefile
index 0bf6749522d9,5f93ca072b21..000000000000
--- a/arch/x86/boot/Makefile
+++ b/arch/x86/boot/Makefile
@@@ -9,8 -9,13 +9,14 @@@
  # Changed by many, many contributors over the years.
  #
  
 -KASAN_SANITIZE := n
 +KASAN_SANITIZE			:= n
 +OBJECT_FILES_NON_STANDARD	:= y
+ # Kernel does not boot with kcov instrumentation here.
+ # One of the problems observed was insertion of __sanitizer_cov_trace_pc()
+ # callback into middle of per-cpu data enabling code. Thus the callback observed
+ # inconsistent state and crashed. We are interested mostly in syscall coverage,
+ # so boot code is not interesting anyway.
+ KCOV_INSTRUMENT := n
  
  # If you want to preset the SVGA mode, uncomment the next line and
  # set SVGA_MODE to whatever number you want.
diff --cc arch/x86/boot/compressed/Makefile
index 5e1d26e09407,ad9e9fa5bb11..000000000000
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@@ -16,8 -16,9 +16,10 @@@
  #	(see scripts/Makefile.lib size_append)
  #	compressed vmlinux.bin.all + u32 size of vmlinux.bin.all
  
 -KASAN_SANITIZE := n
 +KASAN_SANITIZE			:= n
 +OBJECT_FILES_NON_STANDARD	:= y
+ # Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
+ KCOV_INSTRUMENT := n
  
  targets := vmlinux vmlinux.bin vmlinux.bin.gz vmlinux.bin.bz2 vmlinux.bin.lzma \
  	vmlinux.bin.xz vmlinux.bin.lzo vmlinux.bin.lz4
diff --cc arch/x86/entry/vdso/Makefile
index f9fb859c98b9,5a1993905ace..000000000000
--- a/arch/x86/entry/vdso/Makefile
+++ b/arch/x86/entry/vdso/Makefile
@@@ -3,9 -3,10 +3,11 @@@
  #
  
  KBUILD_CFLAGS += $(DISABLE_LTO)
 -KASAN_SANITIZE := n
 -UBSAN_SANITIZE := n
 +KASAN_SANITIZE			:= n
 +UBSAN_SANITIZE			:= n
 +OBJECT_FILES_NON_STANDARD	:= y
+ # Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
+ KCOV_INSTRUMENT := n
  
  VDSO64-$(CONFIG_X86_64)		:= y
  VDSOX32-$(CONFIG_X86_X32_ABI)	:= y
diff --cc arch/x86/kernel/Makefile
index d5fb0871aba3,4648960d1c4c..000000000000
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@@ -16,14 -16,14 +16,19 @@@ CFLAGS_REMOVE_ftrace.o = -p
  CFLAGS_REMOVE_early_printk.o = -pg
  endif
  
 -KASAN_SANITIZE_head$(BITS).o := n
 -KASAN_SANITIZE_dumpstack.o := n
 -KASAN_SANITIZE_dumpstack_$(BITS).o := n
 +KASAN_SANITIZE_head$(BITS).o				:= n
 +KASAN_SANITIZE_dumpstack.o				:= n
 +KASAN_SANITIZE_dumpstack_$(BITS).o			:= n
 +
 +OBJECT_FILES_NON_STANDARD_head_$(BITS).o		:= y
 +OBJECT_FILES_NON_STANDARD_relocate_kernel_$(BITS).o	:= y
 +OBJECT_FILES_NON_STANDARD_mcount_$(BITS).o		:= y
 +OBJECT_FILES_NON_STANDARD_test_nx.o			:= y
+ # If instrumentation of this dir is enabled, boot hangs during first second.
+ # Probably could be more selective here, but note that files related to irqs,
+ # boot, dumpstack/stacktrace, etc are either non-interesting or can lead to
+ # non-deterministic coverage.
+ KCOV_INSTRUMENT := n
  
  CFLAGS_irq.o := -I$(src)/../include/asm/trace
  
diff --cc arch/x86/realmode/rm/Makefile
index 053abe7b0ef7,35129dcdeb71..000000000000
--- a/arch/x86/realmode/rm/Makefile
+++ b/arch/x86/realmode/rm/Makefile
@@@ -6,8 -6,9 +6,10 @@@
  # for more details.
  #
  #
 -KASAN_SANITIZE := n
 +KASAN_SANITIZE			:= n
 +OBJECT_FILES_NON_STANDARD	:= y
+ # Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
+ KCOV_INSTRUMENT := n
  
  always := realmode.bin realmode.relocs
  
diff --cc drivers/firmware/efi/libstub/Makefile
index a15841eced4e,37cc9e395edb..000000000000
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@@ -23,7 -23,8 +23,9 @@@ KBUILD_CFLAGS			:= $(cflags-y) -DDISABL
  GCOV_PROFILE			:= n
  KASAN_SANITIZE			:= n
  UBSAN_SANITIZE			:= n
 +OBJECT_FILES_NON_STANDARD	:= y
+ # Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
+ KCOV_INSTRUMENT			:= n
  
  lib-y				:= efi-stub-helper.o
  

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2016-02-26  5:07 Stephen Rothwell
@ 2016-02-26 21:35 ` Andrew Morton
  0 siblings, 0 replies; 87+ messages in thread
From: Andrew Morton @ 2016-02-26 21:35 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Thomas Gleixner, Ingo Molnar, H. Peter Anvin, Peter Zijlstra,
	linux-next, linux-kernel, Dave Hansen, Piotr Kwapulinski

On Fri, 26 Feb 2016 16:07:12 +1100 Stephen Rothwell <sfr@canb.auug.org.au> wrote:

> Hi Andrew,
> 
> Today's linux-next merge of the akpm-current tree got a conflict in:
> 
>   mm/mprotect.c
> 
> between commit:
> 
>   62b5f7d013fc ("mm/core, x86/mm/pkeys: Add execute-only protection keys support")
> 
> from the tip tree and commit:
> 
>   aff3915ff831 ("mm/mprotect.c: don't imply PROT_EXEC on non-exec fs")
> 
> from the akpm-current tree.
> 
> I fixed it up (I think - see below) and can carry the fix as necessary
> (no action is required).
> 
> -- 
> Cheers,
> Stephen Rothwell
> 
> diff --cc mm/mprotect.c
> index fa37c4cd973a,6ff5dfa65b33..000000000000
> --- a/mm/mprotect.c
> +++ b/mm/mprotect.c
> @@@ -414,7 -409,11 +411,11 @@@ SYSCALL_DEFINE3(mprotect, unsigned long
>   
>   		/* Here we know that vma->vm_start <= nstart < vma->vm_end. */
>   
> + 		/* Does the application expect PROT_READ to imply PROT_EXEC */
> + 		if (rier && (vma->vm_flags & VM_MAYEXEC))
> + 			prot |= PROT_EXEC;
> + 
>  -		newflags = calc_vm_prot_bits(prot);
>  +		newflags = calc_vm_prot_bits(prot, pkey);
>   		newflags |= (vma->vm_flags & ~(VM_READ | VM_WRITE | VM_EXEC));
>   
>   		/* newflags >> 4 shift VM_MAY% in place of VM_% */

OK, thanks.

I moved this patch
(mm-mprotectc-dont-imply-prot_exec-on-non-exec-fs.patch) into the
"post-linux-next" section and reworked it to accommodate the -tip
changes.



From: Piotr Kwapulinski <kwapulinski.piotr@gmail.com>
Subject: mm/mprotect.c: don't imply PROT_EXEC on non-exec fs

The mprotect(PROT_READ) fails when called by the READ_IMPLIES_EXEC binary
on a memory mapped file located on non-exec fs.  The mprotect does not
check whether fs is _executable_ or not.  The PROT_EXEC flag is set
automatically even if a memory mapped file is located on non-exec fs.  Fix
it by checking whether a memory mapped file is located on a non-exec fs. 
If so the PROT_EXEC is not implied by the PROT_READ.  The implementation
uses the VM_MAYEXEC flag set properly in mmap.  Now it is consistent with
mmap.

I did the isolated tests (PT_GNU_STACK X/NX, multiple VMAs, X/NX fs).  I
also patched the official 3.19.0-47-generic Ubuntu 14.04 kernel and it
seems to work.

Signed-off-by: Piotr Kwapulinski <kwapulinski.piotr@gmail.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Konstantin Khlebnikov <koct9i@gmail.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/mprotect.c |   13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff -puN mm/mprotect.c~mm-mprotectc-dont-imply-prot_exec-on-non-exec-fs mm/mprotect.c
--- a/mm/mprotect.c~mm-mprotectc-dont-imply-prot_exec-on-non-exec-fs
+++ a/mm/mprotect.c
@@ -359,6 +359,9 @@ SYSCALL_DEFINE3(mprotect, unsigned long,
 	struct vm_area_struct *vma, *prev;
 	int error = -EINVAL;
 	const int grows = prot & (PROT_GROWSDOWN|PROT_GROWSUP);
+	const bool rier = (current->personality & READ_IMPLIES_EXEC) &&
+				(prot & PROT_READ);
+
 	prot &= ~(PROT_GROWSDOWN|PROT_GROWSUP);
 	if (grows == (PROT_GROWSDOWN|PROT_GROWSUP)) /* can't be both */
 		return -EINVAL;
@@ -375,11 +378,6 @@ SYSCALL_DEFINE3(mprotect, unsigned long,
 		return -EINVAL;
 
 	reqprot = prot;
-	/*
-	 * Does the application expect PROT_READ to imply PROT_EXEC:
-	 */
-	if ((prot & PROT_READ) && (current->personality & READ_IMPLIES_EXEC))
-		prot |= PROT_EXEC;
 
 	down_write(&current->mm->mmap_sem);
 
@@ -414,6 +412,10 @@ SYSCALL_DEFINE3(mprotect, unsigned long,
 
 		/* Here we know that vma->vm_start <= nstart < vma->vm_end. */
 
+		/* Does the application expect PROT_READ to imply PROT_EXEC */
+		if (rier && (vma->vm_flags & VM_MAYEXEC))
+			prot |= PROT_EXEC;
+
 		newflags = calc_vm_prot_bits(prot, pkey);
 		newflags |= (vma->vm_flags & ~(VM_READ | VM_WRITE | VM_EXEC));
 
@@ -445,6 +447,7 @@ SYSCALL_DEFINE3(mprotect, unsigned long,
 			error = -ENOMEM;
 			goto out;
 		}
+		prot = reqprot;
 	}
 out:
 	up_write(&current->mm->mmap_sem);
_

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2016-02-26  5:07 Stephen Rothwell
  2016-02-26 21:35 ` Andrew Morton
  0 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2016-02-26  5:07 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Dave Hansen, Piotr Kwapulinski

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in:

  mm/mprotect.c

between commit:

  62b5f7d013fc ("mm/core, x86/mm/pkeys: Add execute-only protection keys support")

from the tip tree and commit:

  aff3915ff831 ("mm/mprotect.c: don't imply PROT_EXEC on non-exec fs")

from the akpm-current tree.

I fixed it up (I think - see below) and can carry the fix as necessary
(no action is required).

-- 
Cheers,
Stephen Rothwell

diff --cc mm/mprotect.c
index fa37c4cd973a,6ff5dfa65b33..000000000000
--- a/mm/mprotect.c
+++ b/mm/mprotect.c
@@@ -414,7 -409,11 +411,11 @@@ SYSCALL_DEFINE3(mprotect, unsigned long
  
  		/* Here we know that vma->vm_start <= nstart < vma->vm_end. */
  
+ 		/* Does the application expect PROT_READ to imply PROT_EXEC */
+ 		if (rier && (vma->vm_flags & VM_MAYEXEC))
+ 			prot |= PROT_EXEC;
+ 
 -		newflags = calc_vm_prot_bits(prot);
 +		newflags = calc_vm_prot_bits(prot, pkey);
  		newflags |= (vma->vm_flags & ~(VM_READ | VM_WRITE | VM_EXEC));
  
  		/* newflags >> 4 shift VM_MAY% in place of VM_% */

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2016-02-19  4:09 Stephen Rothwell
@ 2016-02-19 15:26 ` Ard Biesheuvel
  0 siblings, 0 replies; 87+ messages in thread
From: Ard Biesheuvel @ 2016-02-19 15:26 UTC (permalink / raw)
  To: Andrew Morton, Catalin Marinas, H. Peter Anvin
  Cc: Stephen Rothwell, Thomas Gleixner, Ingo Molnar, Peter Zijlstra,
	linux-next, linux-kernel, Tony Luck

On 19 February 2016 at 05:09, Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> Hi Andrew,
>
> Today's linux-next merge of the akpm-current tree got a conflict in:
>
>   arch/x86/mm/extable.c
>
> between commit:
>
>   548acf19234d ("x86/mm: Expand the exception table logic to allow new handling options")
>
> from the tip tree and commit:
>
>   f1cd2c09ff09 ("x86/extable: use generic search and sort routines")
>
> from the akpm-current tree.
>
> I couldn't figure out how to fix this up, so I just dropped the
> akpm-current tree patch.
>

Hi Andrew,

Unfortunately, this is not the only problem currently with my extable
series. The arm64 patch now also conflicts with patches that are
queued in the arm64 tree.

So could you please drop all six of them for now? I will ask Catalin
to take the ones that are essential to the arm64 KASLR implementation
via the arm64 tree, and once that hits mainline, I will rebase and
resubmit the remaining patches.

extable-add-support-for-relative-extables-to-search-and-sort-routines.patch
alpha-extable-use-generic-search-and-sort-routines.patch
s390-extable-use-generic-search-and-sort-routines.patch
x86-extable-use-generic-search-and-sort-routines.patch
ia64-extable-use-generic-search-and-sort-routines.patch
arm64-switch-to-relative-exception-tables.patch

Thanks,
Ard.

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2016-02-19  4:09 Stephen Rothwell
  2016-02-19 15:26 ` Ard Biesheuvel
  0 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2016-02-19  4:09 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Tony Luck, Ard Biesheuvel

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in:

  arch/x86/mm/extable.c

between commit:

  548acf19234d ("x86/mm: Expand the exception table logic to allow new handling options")

from the tip tree and commit:

  f1cd2c09ff09 ("x86/extable: use generic search and sort routines")

from the akpm-current tree.

I couldn't figure out how to fix this up, so I just dropped the
akpm-current tree patch.

-- 
Cheers,
Stephen Rothwell

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2015-12-07  8:06 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2015-12-07  8:06 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Kirill A. Shutemov, Juergen Gross

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in:

  arch/x86/mm/pgtable.c

between commit:

  d6ccc3ec9525 ("x86/paravirt: Remove paravirt ops pmd_update[_defer] and pte_update_defer")

from the tip tree and commit:

  275461f0db1f ("x86, thp: remove infrastructure for handling splitting PMDs")

from the akpm-current tree.

I fixed it up (I removed the function (pmdp_splitting_flush) removed by
the latter) and can carry the fix as necessary (no action is required).



-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2015-10-02  4:21 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2015-10-02  4:21 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Kees Cook

Hi Andrew,

Today's linux-next merge of the akpm-current tree got conflicts in:

  Documentation/filesystems/proc.txt
  fs/proc/array.c
  fs/proc/base.c

between commit:

  b2f73922d119 ("fs/proc, core/debug: Don't expose absolute kernel addresses via wchan")

from the tip tree and commit:

  f01df89b6372 ("fs/proc: don't expose absolute kernel addresses via wchan")
  7adc347341f1 ("fs-proc-dont-expose-absolute-kernel-addresses-via-wchan-fix")

from the akpm-current tree.

I fixed it up (the tip tree version seemed newer, so I used that) and
can carry the fix as necessary (no action is required).

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2015-09-08 23:21           ` Andrew Morton
@ 2015-09-16  6:58             ` Geert Uytterhoeven
  0 siblings, 0 replies; 87+ messages in thread
From: Geert Uytterhoeven @ 2015-09-16  6:58 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Linus Torvalds, Stephen Rothwell, Andrea Arcangeli,
	Thomas Gleixner, Ingo Molnar, H. Peter Anvin, Peter Zijlstra,
	linux-next, Linux Kernel Mailing List, Andy Lutomirski,
	Eric B Munson, Dr. David Alan Gilbert

Hi Andrew,

On Wed, Sep 9, 2015 at 1:21 AM, Andrew Morton <akpm@linux-foundation.org> wrote:
> New syscalls are rather a pain, both from the patch-monkeying POV and
> also because nobody knows what the syscall numbers will be until
> everything lands in mainline.  Oh well, it doesn't happen often and
> it's easy stuff.

One more reason to let the assignment of syscall numbers be handled
(1) by the architecture maintainer, (2) after -rc1, even for x86.

If x86 is no more the canonical source, scripts/checksyscalls.sh needs an
update, though.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2015-09-08 23:03         ` Linus Torvalds
@ 2015-09-08 23:21           ` Andrew Morton
  2015-09-16  6:58             ` Geert Uytterhoeven
  0 siblings, 1 reply; 87+ messages in thread
From: Andrew Morton @ 2015-09-08 23:21 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Stephen Rothwell, Andrea Arcangeli, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Peter Zijlstra, linux-next,
	Linux Kernel Mailing List, Andy Lutomirski, Eric B Munson,
	Dr. David Alan Gilbert

On Tue, 8 Sep 2015 16:03:23 -0700 Linus Torvalds <torvalds@linux-foundation.org> wrote:

> On Tue, Sep 8, 2015 at 3:56 PM, Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> >
> > I have been applying that patch I sent to you to -next for some time.
> > I guess I expected Andrew to pick it up when he rebased his patch
> > series before submitting it to you.  These things sometimes slip
> > through the cracks.
> 
> I suspect Andrew saw that patch, and thought it was a merge fixup like
> you sometimes send out, and didn't realize that it actually applied
> directly to his series.

I've had it all the time, as a post-linux-next fixup - the idea being
that I send it to you after its linux-next preconditions have been
merged up.

However I failed to put that patch inside the stephen-take-these-bits
markers, so it never went from -mm into -next.

New syscalls are rather a pain, both from the patch-monkeying POV and
also because nobody knows what the syscall numbers will be until
everything lands in mainline.  Oh well, it doesn't happen often and
it's easy stuff.

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2015-09-08 22:56       ` Stephen Rothwell
@ 2015-09-08 23:03         ` Linus Torvalds
  2015-09-08 23:21           ` Andrew Morton
  0 siblings, 1 reply; 87+ messages in thread
From: Linus Torvalds @ 2015-09-08 23:03 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Andrea Arcangeli, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Peter Zijlstra, linux-next,
	Linux Kernel Mailing List, Andy Lutomirski, Eric B Munson,
	Dr. David Alan Gilbert

On Tue, Sep 8, 2015 at 3:56 PM, Stephen Rothwell <sfr@canb.auug.org.au> wrote:
>
> I have been applying that patch I sent to you to -next for some time.
> I guess I expected Andrew to pick it up when he rebased his patch
> series before submitting it to you.  These things sometimes slip
> through the cracks.

I suspect Andrew saw that patch, and thought it was a merge fixup like
you sometimes send out, and didn't realize that it actually applied
directly to his series.

              Linus

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2015-09-08 18:11     ` Linus Torvalds
@ 2015-09-08 22:56       ` Stephen Rothwell
  2015-09-08 23:03         ` Linus Torvalds
  0 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2015-09-08 22:56 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Andrea Arcangeli, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Peter Zijlstra, linux-next,
	Linux Kernel Mailing List, Andy Lutomirski, Eric B Munson,
	Dr. David Alan Gilbert

Hi Linus,

On Tue, 8 Sep 2015 11:11:25 -0700 Linus Torvalds <torvalds@linux-foundation.org> wrote:
>
> On Mon, Sep 7, 2015 at 4:35 PM, Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> >
> > The below patch was missed when the userfaultfd stuff and the x86 changes
> > were merged.  I have repeated the patch in the clear below.
> 
> When forwarding patches, please add your sign-off. I can see (and
> apply) the original in this thread, so I guess it doesn't matter in
> this particular case, but in general that's what you should be doing
> so that I don't then have to find the other email just to apply the
> patch from the original author.

Ooops, sorry about that.  I really should know better.

> Also, this wasn't actually a real merge error. This was just a bug in
> the whole process. Andrew's patches sent to me had been updated with
> the right number for all the other cases, why hadn't this been folded
> into the series too? Apparently Andrew's series has simply been buggy
> for the last month or more, and that was true even before it was sent
> to me and while it was cooking in -next.. Tssk.

I have been applying that patch I sent to you to -next for some time.
I guess I expected Andrew to pick it up when he rebased his patch
series before submitting it to you.  These things sometimes slip
through the cracks.

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2015-09-07 23:35   ` Stephen Rothwell
@ 2015-09-08 18:11     ` Linus Torvalds
  2015-09-08 22:56       ` Stephen Rothwell
  0 siblings, 1 reply; 87+ messages in thread
From: Linus Torvalds @ 2015-09-08 18:11 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Andrea Arcangeli, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Peter Zijlstra, linux-next,
	Linux Kernel Mailing List, Andy Lutomirski, Eric B Munson,
	Dr. David Alan Gilbert

On Mon, Sep 7, 2015 at 4:35 PM, Stephen Rothwell <sfr@canb.auug.org.au> wrote:
>
> The below patch was missed when the userfaultfd stuff and the x86 changes
> were merged.  I have repeated the patch in the clear below.

When forwarding patches, please add your sign-off. I can see (and
apply) the original in this thread, so I guess it doesn't matter in
this particular case, but in general that's what you should be doing
so that I don't then have to find the other email just to apply the
patch from the original author.

Also, this wasn't actually a real merge error. This was just a bug in
the whole process. Andrew's patches sent to me had been updated with
the right number for all the other cases, why hadn't this been folded
into the series too? Apparently Andrew's series has simply been buggy
for the last month or more, and that was true even before it was sent
to me and while it was cooking in -next.. Tssk.

            Linus

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2015-07-29 17:12 ` Andrea Arcangeli
  2015-07-29 17:47   ` Andy Lutomirski
  2015-07-29 23:06   ` Stephen Rothwell
@ 2015-09-07 23:35   ` Stephen Rothwell
  2015-09-08 18:11     ` Linus Torvalds
  2 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2015-09-07 23:35 UTC (permalink / raw)
  To: Linus
  Cc: Andrea Arcangeli, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Peter Zijlstra, linux-next, linux-kernel,
	Andy Lutomirski, Eric B Munson, Dr. David Alan Gilbert

Hi Linus,

On Wed, 29 Jul 2015 19:12:56 +0200 Andrea Arcangeli <aarcange@redhat.com> wrote:
>
> On Tue, Jul 28, 2015 at 04:00:15PM +1000, Stephen Rothwell wrote:
> >  -359	i386	userfaultfd		sys_userfaultfd
> > ++374	i386	userfaultfd		sys_userfaultfd
> 
> Do I understand correctly the syscall number of userfaultfd for x86
> 32bit has just changed from 359 to 374? Appreciated that you CCed me
> on such a relevant change to be sure I didn't miss it.
> 
> Then the below is needed as well.

The below patch was missed when the userfaultfd stuff and the x86 changes
were merged.  I have repeated the patch in the clear below.

From: Andrea Arcangeli <aarcange@redhat.com>
Date: Wed, 29 Jul 2015 18:53:17 +0200
Subject: [PATCH] userfaultfd: selftest: update userfaultfd x86 32bit syscall number

It changed as result of linux-next merge of other syscalls.

Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
---
 tools/testing/selftests/vm/userfaultfd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/vm/userfaultfd.c b/tools/testing/selftests/vm/userfaultfd.c
index 0c0b839..76071b1 100644
--- a/tools/testing/selftests/vm/userfaultfd.c
+++ b/tools/testing/selftests/vm/userfaultfd.c
@@ -69,7 +69,7 @@
 #ifdef __x86_64__
 #define __NR_userfaultfd 323
 #elif defined(__i386__)
-#define __NR_userfaultfd 359
+#define __NR_userfaultfd 374
 #elif defined(__powewrpc__)
 #define __NR_userfaultfd 364
 #else

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2015-07-29 18:46     ` Thomas Gleixner
@ 2015-07-30 15:38       ` Andrea Arcangeli
  0 siblings, 0 replies; 87+ messages in thread
From: Andrea Arcangeli @ 2015-07-30 15:38 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Andy Lutomirski, Stephen Rothwell, Andrew Morton, Ingo Molnar,
	H. Peter Anvin, Peter Zijlstra, linux-next, linux-kernel,
	Andy Lutomirski, Eric B Munson, Dr. David Alan Gilbert

On Wed, Jul 29, 2015 at 08:46:10PM +0200, Thomas Gleixner wrote:
> On Wed, 29 Jul 2015, Andy Lutomirski wrote:
> > -tip people: want to assign Andrea a pair of syscall numbers?
> 
> Sure, just send a patch ....

Awesome, I just sent the patch to register the syscall against -tip
with the usual placeholder that will return -ENOSYS.

Thanks,
Andrea

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2015-07-29 23:06   ` Stephen Rothwell
@ 2015-07-29 23:07     ` Thomas Gleixner
  0 siblings, 0 replies; 87+ messages in thread
From: Thomas Gleixner @ 2015-07-29 23:07 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Andrea Arcangeli, Andrew Morton, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra, linux-next, linux-kernel, Andy Lutomirski,
	Eric B Munson, Dr. David Alan Gilbert

On Thu, 30 Jul 2015, Stephen Rothwell wrote:
> Hi Andrea,
> 
> On Wed, 29 Jul 2015 19:12:56 +0200 Andrea Arcangeli <aarcange@redhat.com> wrote:
> >
> > On Tue, Jul 28, 2015 at 04:00:15PM +1000, Stephen Rothwell wrote:
> > >  -359	i386	userfaultfd		sys_userfaultfd
> > > ++374	i386	userfaultfd		sys_userfaultfd
> > 
> > Do I understand correctly the syscall number of userfaultfd for x86
> > 32bit has just changed from 359 to 374? Appreciated that you CCed me
> > on such a relevant change to be sure I didn't miss it.
> > 
> > Then the below is needed as well.
> 
> I have added the below patch to linux-next from today.
> 
> > One related question: is it ok to ship kernels in production right now
> > with the userfaultfd syscall number 374 for x86 32bit ABI (after the
> > above change) and 323 for x86-64 64bit ABI, with these syscalls number
> > registered in linux-next or it may keep changing like it has just
> > happened? I refer only to userfaultfd syscalls of x86 32bit and x86-64
> > 64bit, not all other syscalls in linux-next.
> 
> These numbers are certainly not in any way official, they are just the
> result of my merge conflict fixup.  So, yes, they could change again if
> someone adds another new syscall to any tree but Andrew's.
> 
> > Of course, I know full well that the standard answer is no, and in
> > fact the above is an expected and fine change. In other words what I'm
> > really asking is if I wonder if I could get an agreement here that
> > from now on, the syscall number of userfaultfd for x86 32bit and
> > x86-64 64bit won't change anymore in linux-next and it's already
> > reserved just like if it was already upstream.
> 
> Like Thomas said, send a patch to the x86 maintainers.  I suspect (if
> the rest of the implementation needs to stay in Andrew's tree) that it
> could be a simple as a patch to the syscall tables using ni_syscall and
> a comment.  Thomas?

Yes, that's all it takes to reserve a syscall number.

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2015-07-29 17:12 ` Andrea Arcangeli
  2015-07-29 17:47   ` Andy Lutomirski
@ 2015-07-29 23:06   ` Stephen Rothwell
  2015-07-29 23:07     ` Thomas Gleixner
  2015-09-07 23:35   ` Stephen Rothwell
  2 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2015-07-29 23:06 UTC (permalink / raw)
  To: Andrea Arcangeli
  Cc: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra, linux-next, linux-kernel, Andy Lutomirski,
	Eric B Munson, Dr. David Alan Gilbert

Hi Andrea,

On Wed, 29 Jul 2015 19:12:56 +0200 Andrea Arcangeli <aarcange@redhat.com> wrote:
>
> On Tue, Jul 28, 2015 at 04:00:15PM +1000, Stephen Rothwell wrote:
> >  -359	i386	userfaultfd		sys_userfaultfd
> > ++374	i386	userfaultfd		sys_userfaultfd
> 
> Do I understand correctly the syscall number of userfaultfd for x86
> 32bit has just changed from 359 to 374? Appreciated that you CCed me
> on such a relevant change to be sure I didn't miss it.
> 
> Then the below is needed as well.

I have added the below patch to linux-next from today.

> One related question: is it ok to ship kernels in production right now
> with the userfaultfd syscall number 374 for x86 32bit ABI (after the
> above change) and 323 for x86-64 64bit ABI, with these syscalls number
> registered in linux-next or it may keep changing like it has just
> happened? I refer only to userfaultfd syscalls of x86 32bit and x86-64
> 64bit, not all other syscalls in linux-next.

These numbers are certainly not in any way official, they are just the
result of my merge conflict fixup.  So, yes, they could change again if
someone adds another new syscall to any tree but Andrew's.

> Of course, I know full well that the standard answer is no, and in
> fact the above is an expected and fine change. In other words what I'm
> really asking is if I wonder if I could get an agreement here that
> from now on, the syscall number of userfaultfd for x86 32bit and
> x86-64 64bit won't change anymore in linux-next and it's already
> reserved just like if it was already upstream.

Like Thomas said, send a patch to the x86 maintainers.  I suspect (if
the rest of the implementation needs to stay in Andrew's tree) that it
could be a simple as a patch to the syscall tables using ni_syscall and
a comment.  Thomas?

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2015-07-29 17:47   ` Andy Lutomirski
@ 2015-07-29 18:46     ` Thomas Gleixner
  2015-07-30 15:38       ` Andrea Arcangeli
  0 siblings, 1 reply; 87+ messages in thread
From: Thomas Gleixner @ 2015-07-29 18:46 UTC (permalink / raw)
  To: Andy Lutomirski
  Cc: Andrea Arcangeli, Stephen Rothwell, Andrew Morton, Ingo Molnar,
	H. Peter Anvin, Peter Zijlstra, linux-next, linux-kernel,
	Andy Lutomirski, Eric B Munson, Dr. David Alan Gilbert

On Wed, 29 Jul 2015, Andy Lutomirski wrote:
> On Wed, Jul 29, 2015 at 10:12 AM, Andrea Arcangeli <aarcange@redhat.com> wrote:
> > Hello Stephen,
> >
> > On Tue, Jul 28, 2015 at 04:00:15PM +1000, Stephen Rothwell wrote:
> >>  -359 i386    userfaultfd             sys_userfaultfd
> >> ++374 i386    userfaultfd             sys_userfaultfd
> >
> > Do I understand correctly the syscall number of userfaultfd for x86
> > 32bit has just changed from 359 to 374? Appreciated that you CCed me
> > on such a relevant change to be sure I didn't miss it.
> >
> > Then the below is needed as well.
> >
> > One related question: is it ok to ship kernels in production right now
> > with the userfaultfd syscall number 374 for x86 32bit ABI (after the
> > above change) and 323 for x86-64 64bit ABI, with these syscalls number
> > registered in linux-next or it may keep changing like it has just
> > happened? I refer only to userfaultfd syscalls of x86 32bit and x86-64
> > 64bit, not all other syscalls in linux-next.
> >
> > Of course, I know full well that the standard answer is no, and in
> > fact the above is an expected and fine change. In other words what I'm
> > really asking is if I wonder if I could get an agreement here that
> > from now on, the syscall number of userfaultfd for x86 32bit and
> > x86-64 64bit won't change anymore in linux-next and it's already
> > reserved just like if it was already upstream.
> >
> > Again: I'd only seek such guarantee for the x86-64 64bit and x86 32bit
> > ABIs (not any other arch, and not any other syscall). If I could get
> > such a guarantee from you within the next week or two, that would
> > avoid me complications and some work, so I thought it was worth
> > asking. If it's not possible never mind.
> 
> My (limited) understanding is that this is up to the arch maintainers.
> I certainly didn't intend to preempt your syscall number, but my patch
> beat your patch to -tip :-p
> 
> -tip people: want to assign Andrea a pair of syscall numbers?

Sure, just send a patch ....

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2015-07-29 17:12 ` Andrea Arcangeli
@ 2015-07-29 17:47   ` Andy Lutomirski
  2015-07-29 18:46     ` Thomas Gleixner
  2015-07-29 23:06   ` Stephen Rothwell
  2015-09-07 23:35   ` Stephen Rothwell
  2 siblings, 1 reply; 87+ messages in thread
From: Andy Lutomirski @ 2015-07-29 17:47 UTC (permalink / raw)
  To: Andrea Arcangeli
  Cc: Stephen Rothwell, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Peter Zijlstra, linux-next, linux-kernel,
	Andy Lutomirski, Eric B Munson, Dr. David Alan Gilbert

On Wed, Jul 29, 2015 at 10:12 AM, Andrea Arcangeli <aarcange@redhat.com> wrote:
> Hello Stephen,
>
> On Tue, Jul 28, 2015 at 04:00:15PM +1000, Stephen Rothwell wrote:
>>  -359 i386    userfaultfd             sys_userfaultfd
>> ++374 i386    userfaultfd             sys_userfaultfd
>
> Do I understand correctly the syscall number of userfaultfd for x86
> 32bit has just changed from 359 to 374? Appreciated that you CCed me
> on such a relevant change to be sure I didn't miss it.
>
> Then the below is needed as well.
>
> One related question: is it ok to ship kernels in production right now
> with the userfaultfd syscall number 374 for x86 32bit ABI (after the
> above change) and 323 for x86-64 64bit ABI, with these syscalls number
> registered in linux-next or it may keep changing like it has just
> happened? I refer only to userfaultfd syscalls of x86 32bit and x86-64
> 64bit, not all other syscalls in linux-next.
>
> Of course, I know full well that the standard answer is no, and in
> fact the above is an expected and fine change. In other words what I'm
> really asking is if I wonder if I could get an agreement here that
> from now on, the syscall number of userfaultfd for x86 32bit and
> x86-64 64bit won't change anymore in linux-next and it's already
> reserved just like if it was already upstream.
>
> Again: I'd only seek such guarantee for the x86-64 64bit and x86 32bit
> ABIs (not any other arch, and not any other syscall). If I could get
> such a guarantee from you within the next week or two, that would
> avoid me complications and some work, so I thought it was worth
> asking. If it's not possible never mind.

My (limited) understanding is that this is up to the arch maintainers.
I certainly didn't intend to preempt your syscall number, but my patch
beat your patch to -tip :-p

-tip people: want to assign Andrea a pair of syscall numbers?

--Andy

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2015-07-28  6:00 Stephen Rothwell
@ 2015-07-29 17:12 ` Andrea Arcangeli
  2015-07-29 17:47   ` Andy Lutomirski
                     ` (2 more replies)
  0 siblings, 3 replies; 87+ messages in thread
From: Andrea Arcangeli @ 2015-07-29 17:12 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra, linux-next, linux-kernel, Andy Lutomirski,
	Eric B Munson, Dr. David Alan Gilbert

Hello Stephen,

On Tue, Jul 28, 2015 at 04:00:15PM +1000, Stephen Rothwell wrote:
>  -359	i386	userfaultfd		sys_userfaultfd
> ++374	i386	userfaultfd		sys_userfaultfd

Do I understand correctly the syscall number of userfaultfd for x86
32bit has just changed from 359 to 374? Appreciated that you CCed me
on such a relevant change to be sure I didn't miss it.

Then the below is needed as well.

One related question: is it ok to ship kernels in production right now
with the userfaultfd syscall number 374 for x86 32bit ABI (after the
above change) and 323 for x86-64 64bit ABI, with these syscalls number
registered in linux-next or it may keep changing like it has just
happened? I refer only to userfaultfd syscalls of x86 32bit and x86-64
64bit, not all other syscalls in linux-next.

Of course, I know full well that the standard answer is no, and in
fact the above is an expected and fine change. In other words what I'm
really asking is if I wonder if I could get an agreement here that
from now on, the syscall number of userfaultfd for x86 32bit and
x86-64 64bit won't change anymore in linux-next and it's already
reserved just like if it was already upstream.

Again: I'd only seek such guarantee for the x86-64 64bit and x86 32bit
ABIs (not any other arch, and not any other syscall). If I could get
such a guarantee from you within the next week or two, that would
avoid me complications and some work, so I thought it was worth
asking. If it's not possible never mind.

Thanks,
Andrea

===
>From 873093c32b4b1d0b6c3f18ec1e52b56c24f67457 Mon Sep 17 00:00:00 2001
From: Andrea Arcangeli <aarcange@redhat.com>
Date: Wed, 29 Jul 2015 18:53:17 +0200
Subject: [PATCH] userfaultfd: selftest: update userfaultfd x86 32bit syscall
 number

It changed as result of linux-next merge of other syscalls.

Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
---
 tools/testing/selftests/vm/userfaultfd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/vm/userfaultfd.c b/tools/testing/selftests/vm/userfaultfd.c
index 0c0b839..76071b1 100644
--- a/tools/testing/selftests/vm/userfaultfd.c
+++ b/tools/testing/selftests/vm/userfaultfd.c
@@ -69,7 +69,7 @@
 #ifdef __x86_64__
 #define __NR_userfaultfd 323
 #elif defined(__i386__)
-#define __NR_userfaultfd 359
+#define __NR_userfaultfd 374
 #elif defined(__powewrpc__)
 #define __NR_userfaultfd 364
 #else

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2015-07-28  6:00 Stephen Rothwell
  2015-07-29 17:12 ` Andrea Arcangeli
  0 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2015-07-28  6:00 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Andy Lutomirski, Eric B Munson,
	Andrea Arcangeli

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in:

  arch/x86/entry/syscalls/syscall_32.tbl

between commit:

  9dea5dc921b5 ("x86/entry/syscalls: Wire up 32-bit direct socket calls")

from the tip tree and commit:

  0a36ab281187 ("userfaultfd: activate syscall")
  f721d9f04de4 ("mm: mlock: add new mlock, munlock, and munlockall system calls")

from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary (no action
is required).

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

diff --cc arch/x86/entry/syscalls/syscall_32.tbl
index 25e3cf1cd8fd,d68b13925aa4..000000000000
--- a/arch/x86/entry/syscalls/syscall_32.tbl
+++ b/arch/x86/entry/syscalls/syscall_32.tbl
@@@ -365,18 -365,7 +365,22 @@@
  356	i386	memfd_create		sys_memfd_create
  357	i386	bpf			sys_bpf
  358	i386	execveat		sys_execveat			stub32_execveat
 -359	i386	userfaultfd		sys_userfaultfd
 -360	i386	mlock2			sys_mlock2
 -361	i386	munlock2		sys_munlock2
 -362	i386	munlockall2		sys_munlockall2
 +359	i386	socket			sys_socket
 +360	i386	socketpair		sys_socketpair
 +361	i386	bind			sys_bind
 +362	i386	connect			sys_connect
 +363	i386	listen			sys_listen
 +364	i386	accept4			sys_accept4
 +365	i386	getsockopt		sys_getsockopt			compat_sys_getsockopt
 +366	i386	setsockopt		sys_setsockopt			compat_sys_setsockopt
 +367	i386	getsockname		sys_getsockname
 +368	i386	getpeername		sys_getpeername
 +369	i386	sendto			sys_sendto
 +370	i386	sendmsg			sys_sendmsg			compat_sys_sendmsg
 +371	i386	recvfrom		sys_recvfrom			compat_sys_recvfrom
 +372	i386	recvmsg			sys_recvmsg			compat_sys_recvmsg
 +373	i386	shutdown		sys_shutdown
++374	i386	userfaultfd		sys_userfaultfd
++375	i386	mlock2			sys_mlock2
++376	i386	munlock2		sys_munlock2
++377	i386	munlockall2		sys_munlockall2

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2015-06-04 12:07 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2015-06-04 12:07 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Mel Gorman, Josh Triplett

[-- Attachment #1: Type: text/plain, Size: 6074 bytes --]

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in
arch/x86/Kconfig between commit 6471b825c41e ("x86/kconfig: Reorganize
arch feature Kconfig select's") from the tip tree and commits
be853e68c4b2 ("x86: mm: enable deferred struct page initialisation on
x86-64") and 84ebc29f19b2 ("x86: opt into HAVE_COPY_THREAD_TLS, for
both 32-bit and 64-bit") from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary (no action
is required).

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

diff --cc arch/x86/Kconfig
index fc1fd8a41540,0dd372f3111e..000000000000
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@@ -17,113 -23,61 +17,115 @@@ config X86_6
  ### Arch settings
  config X86
  	def_bool y
 -	select ACPI_SYSTEM_POWER_STATES_SUPPORT if ACPI
 -	select ARCH_MIGHT_HAVE_ACPI_PDC if ACPI
 +	select ACPI_LEGACY_TABLES_LOOKUP	if ACPI
 +	select ACPI_SYSTEM_POWER_STATES_SUPPORT	if ACPI
 +	select ANON_INODES
 +	select ARCH_CLOCKSOURCE_DATA
 +	select ARCH_DISCARD_MEMBLOCK
 +	select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE
  	select ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS
 +	select ARCH_HAS_ELF_RANDOMIZE
  	select ARCH_HAS_FAST_MULTIPLIER
  	select ARCH_HAS_GCOV_PROFILE_ALL
 +	select ARCH_HAS_SG_CHAIN
 +	select ARCH_HAVE_NMI_SAFE_CMPXCHG
 +	select ARCH_MIGHT_HAVE_ACPI_PDC		if ACPI
  	select ARCH_MIGHT_HAVE_PC_PARPORT
  	select ARCH_MIGHT_HAVE_PC_SERIO
 -	select HAVE_AOUT if X86_32
 -	select HAVE_UNSTABLE_SCHED_CLOCK
 -	select ARCH_SUPPORTS_NUMA_BALANCING if X86_64
 -	select ARCH_SUPPORTS_INT128 if X86_64
 -	select HAVE_IDE
 -	select HAVE_OPROFILE
 -	select HAVE_PCSPKR_PLATFORM
 -	select HAVE_PERF_EVENTS
 -	select HAVE_IOREMAP_PROT
 -	select HAVE_KPROBES
 -	select HAVE_MEMBLOCK
 -	select HAVE_MEMBLOCK_NODE_MAP
 -	select ARCH_DISCARD_MEMBLOCK
 -	select ARCH_WANT_OPTIONAL_GPIOLIB
 +	select ARCH_SUPPORTS_ATOMIC_RMW
++	select ARCH_SUPPORTS_DEFERRED_STRUCT_PAGE_INIT	if X86_64
 +	select ARCH_SUPPORTS_INT128		if X86_64
 +	select ARCH_SUPPORTS_NUMA_BALANCING	if X86_64
 +	select ARCH_USE_BUILTIN_BSWAP
 +	select ARCH_USE_CMPXCHG_LOCKREF		if X86_64
 +	select ARCH_USE_QUEUED_RWLOCKS
 +	select ARCH_USE_QUEUED_SPINLOCKS
  	select ARCH_WANT_FRAME_POINTERS
 -	select HAVE_DMA_ATTRS
 -	select HAVE_DMA_CONTIGUOUS
 -	select HAVE_KRETPROBES
 +	select ARCH_WANT_IPC_PARSE_VERSION	if X86_32
 +	select ARCH_WANT_OPTIONAL_GPIOLIB
 +	select BUILDTIME_EXTABLE_SORT
 +	select CLKEVT_I8253
 +	select CLKSRC_I8253			if X86_32
 +	select CLOCKSOURCE_VALIDATE_LAST_CYCLE
 +	select CLOCKSOURCE_WATCHDOG
 +	select CLONE_BACKWARDS			if X86_32
 +	select COMPAT_OLD_SIGACTION		if IA32_EMULATION
 +	select DCACHE_WORD_ACCESS
 +	select EDAC_ATOMIC_SCRUB
 +	select EDAC_SUPPORT
 +	select GENERIC_CLOCKEVENTS
 +	select GENERIC_CLOCKEVENTS_BROADCAST	if X86_64 || (X86_32 && X86_LOCAL_APIC)
 +	select GENERIC_CLOCKEVENTS_MIN_ADJUST
 +	select GENERIC_CMOS_UPDATE
 +	select GENERIC_CPU_AUTOPROBE
  	select GENERIC_EARLY_IOREMAP
 -	select HAVE_OPTPROBES
 -	select HAVE_KPROBES_ON_FTRACE
 -	select HAVE_FTRACE_MCOUNT_RECORD
 -	select HAVE_FENTRY if X86_64
 +	select GENERIC_FIND_FIRST_BIT
 +	select GENERIC_IOMAP
 +	select GENERIC_IRQ_PROBE
 +	select GENERIC_IRQ_SHOW
 +	select GENERIC_PENDING_IRQ		if SMP
 +	select GENERIC_SMP_IDLE_THREAD
 +	select GENERIC_STRNCPY_FROM_USER
 +	select GENERIC_STRNLEN_USER
 +	select GENERIC_TIME_VSYSCALL
 +	select HAVE_ACPI_APEI			if ACPI
 +	select HAVE_ACPI_APEI_NMI		if ACPI
 +	select HAVE_ALIGNED_STRUCT_PAGE		if SLUB
 +	select HAVE_AOUT			if X86_32
 +	select HAVE_ARCH_AUDITSYSCALL
 +	select HAVE_ARCH_HUGE_VMAP		if X86_64 || X86_PAE
 +	select HAVE_ARCH_JUMP_LABEL
 +	select HAVE_ARCH_KASAN			if X86_64 && SPARSEMEM_VMEMMAP
 +	select HAVE_ARCH_KGDB
 +	select HAVE_ARCH_KMEMCHECK
 +	select HAVE_ARCH_SECCOMP_FILTER
 +	select HAVE_ARCH_SOFT_DIRTY		if X86_64
 +	select HAVE_ARCH_TRACEHOOK
 +	select HAVE_ARCH_TRANSPARENT_HUGEPAGE
 +	select HAVE_BPF_JIT			if X86_64
 +	select HAVE_CC_STACKPROTECTOR
 +	select HAVE_CMPXCHG_DOUBLE
 +	select HAVE_CMPXCHG_LOCAL
 +	select HAVE_CONTEXT_TRACKING		if X86_64
++	select HAVE_COPY_THREAD_TLS
  	select HAVE_C_RECORDMCOUNT
 +	select HAVE_DEBUG_KMEMLEAK
 +	select HAVE_DEBUG_STACKOVERFLOW
 +	select HAVE_DMA_API_DEBUG
 +	select HAVE_DMA_ATTRS
 +	select HAVE_DMA_CONTIGUOUS
  	select HAVE_DYNAMIC_FTRACE
  	select HAVE_DYNAMIC_FTRACE_WITH_REGS
 -	select HAVE_FUNCTION_TRACER
 -	select HAVE_FUNCTION_GRAPH_TRACER
 -	select HAVE_FUNCTION_GRAPH_FP_TEST
 -	select HAVE_SYSCALL_TRACEPOINTS
 -	select SYSCTL_EXCEPTION_TRACE
 -	select HAVE_KVM
 -	select HAVE_ARCH_KGDB
 -	select HAVE_ARCH_TRACEHOOK
 -	select HAVE_GENERIC_DMA_COHERENT if X86_32
  	select HAVE_EFFICIENT_UNALIGNED_ACCESS
 -	select USER_STACKTRACE_SUPPORT
 -	select HAVE_REGS_AND_STACK_ACCESS_API
 -	select HAVE_DMA_API_DEBUG
 -	select HAVE_KERNEL_GZIP
 +	select HAVE_FENTRY			if X86_64
 +	select HAVE_FTRACE_MCOUNT_RECORD
 +	select HAVE_FUNCTION_GRAPH_FP_TEST
 +	select HAVE_FUNCTION_GRAPH_TRACER
 +	select HAVE_FUNCTION_TRACER
 +	select HAVE_GENERIC_DMA_COHERENT	if X86_32
 +	select HAVE_HW_BREAKPOINT
 +	select HAVE_IDE
 +	select HAVE_IOREMAP_PROT
 +	select HAVE_IRQ_EXIT_ON_IRQ_STACK	if X86_64
 +	select HAVE_IRQ_TIME_ACCOUNTING
  	select HAVE_KERNEL_BZIP2
 +	select HAVE_KERNEL_GZIP
 +	select HAVE_KERNEL_LZ4
  	select HAVE_KERNEL_LZMA
 -	select HAVE_KERNEL_XZ
  	select HAVE_KERNEL_LZO
 -	select HAVE_KERNEL_LZ4
 -	select HAVE_HW_BREAKPOINT
 +	select HAVE_KERNEL_XZ
 +	select HAVE_KPROBES
 +	select HAVE_KPROBES_ON_FTRACE
 +	select HAVE_KRETPROBES
 +	select HAVE_KVM
 +	select HAVE_LIVEPATCH			if X86_64
 +	select HAVE_MEMBLOCK
 +	select HAVE_MEMBLOCK_NODE_MAP
  	select HAVE_MIXED_BREAKPOINTS_REGS
 -	select PERF_EVENTS
 +	select HAVE_OPROFILE
 +	select HAVE_OPTPROBES
 +	select HAVE_PCSPKR_PLATFORM
 +	select HAVE_PERF_EVENTS
  	select HAVE_PERF_EVENTS_NMI
  	select HAVE_PERF_REGS
  	select HAVE_PERF_USER_STACK_DUMP

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2015-04-08  8:28 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2015-04-08  8:28 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Rasmus Villemoes, Xunlei Pang

[-- Attachment #1: Type: text/plain, Size: 1270 bytes --]

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in
drivers/rtc/rtc-mc13xxx.c between commit 0307b0d77a08
("drivers/rtc/mc13xxx: Update driver to address y2038/y2106 issues")
from the tip tree and commit 219451fa4da4 ("drivers/rtc/rtc-mc13xxx.c:
fix obfuscated and wrong format string") from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary (no action
is required).

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

diff --cc drivers/rtc/rtc-mc13xxx.c
index 32df1d812367,c8e78a560de7..000000000000
--- a/drivers/rtc/rtc-mc13xxx.c
+++ b/drivers/rtc/rtc-mc13xxx.c
@@@ -214,10 -215,12 +214,10 @@@ static int mc13xxx_rtc_set_alarm(struc
  	if (unlikely(ret))
  		goto out;
  
 -	ret = rtc_tm_to_time(&alarm->time, &s1970);
 -	if (unlikely(ret))
 -		goto out;
 +	s1970 = rtc_tm_to_time64(&alarm->time);
  
- 	dev_dbg(dev, "%s: o%2.s %lld\n", __func__, alarm->enabled ? "n" : "ff",
 -	dev_dbg(dev, "%s: %s %lu\n", __func__, alarm->enabled ? "on" : "off",
 -			s1970);
++	dev_dbg(dev, "%s: %s %lld\n", __func__, alarm->enabled ? "on" : "off",
 +			(long long)s1970);
  
  	ret = mc13xxx_rtc_irq_enable_unlocked(dev, alarm->enabled,
  			MC13XXX_IRQ_TODA);

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2015-04-08  8:25 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2015-04-08  8:25 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Matthew Garrett, Xunlei Pang

[-- Attachment #1: Type: text/plain, Size: 1790 bytes --]

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in
drivers/rtc/class.c between commit 0fa88cb4b82b ("time, drivers/rtc:
Don't bother with rtc_resume() for the nonstop clocksource") from the
tip tree and commit df9d6ec42558 ("rtc: restore alarm after resume")
from the akpm-current tree.

I fixed it up (I think - see below) and can carry the fix as necessary
(no action is required).

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

diff --cc drivers/rtc/class.c
index c29ba7e14304,d46549ce8fd9..000000000000
--- a/drivers/rtc/class.c
+++ b/drivers/rtc/class.c
@@@ -55,7 -55,9 +55,9 @@@ static int rtc_suspend(struct device *d
  	struct timespec64	delta, delta_delta;
  	int err;
  
+ 	rtc->valid_alarm = !rtc_read_alarm(rtc, &rtc->alarm);
+ 
 -	if (has_persistent_clock())
 +	if (timekeeping_rtc_skipsuspend())
  		return 0;
  
  	if (strcmp(dev_name(&rtc->dev), CONFIG_RTC_HCTOSYS_DEVICE) != 0)
@@@ -102,7 -104,28 +104,28 @@@ static int rtc_resume(struct device *de
  	struct timespec64	sleep_time;
  	int err;
  
+ 	/*
+ 	 * Ensure that the platform hasn't overwritten a pending alarm while
+ 	 * suspended
+ 	 */
+ 	if (rtc->valid_alarm) {
+ 		long now, scheduled;
+ 
+ 		rtc_read_time(rtc, &tm);
+ 		rtc_tm_to_time(&rtc->alarm.time, &scheduled);
+ 		rtc_tm_to_time(&tm, &now);
+ 
+ 		/* Clear the alarm registers if it went off during suspend */
+ 		if (scheduled <= now) {
+ 			rtc_time_to_tm(0, &rtc->alarm.time);
+ 			rtc->alarm.enabled = 0;
+ 		}
+ 
+ 		if (rtc->ops && rtc->ops->set_alarm)
+ 			rtc->ops->set_alarm(rtc->dev.parent, &rtc->alarm);
+ 	}
+ 
 -	if (has_persistent_clock())
 +	if (timekeeping_rtc_skipresume())
  		return 0;
  
  	rtc_hctosys_ret = -ENODEV;

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2014-03-17  9:36 ` Peter Zijlstra
@ 2014-03-19 23:27   ` Andrew Morton
  0 siblings, 0 replies; 87+ messages in thread
From: Andrew Morton @ 2014-03-19 23:27 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Stephen Rothwell, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	linux-next, linux-kernel, Josh Triplett

On Mon, 17 Mar 2014 10:36:02 +0100 Peter Zijlstra <peterz@infradead.org> wrote:

> On Mon, Mar 17, 2014 at 08:31:24PM +1100, Stephen Rothwell wrote:
> > Hi Andrew,
> > 
> > Today's linux-next merge of the akpm-current tree got a conflict in
> > kernel/locking/Makefile between commit fb0527bd5ea9 ("locking/mutexes:
> > Introduce cancelable MCS lock for adaptive spinning") from the  tree and
> > commit 4dc0fe493027 ("lglock: map to spinlock when !CONFIG_SMP") from the
> > akpm-current tree.
> > 
> > I fixed it up (see below) and can carry the fix as necessary (no action
> > is required).
> 
> I'm a bit sad of not having seen that lglock patch at all.

Here 'tis:

From: Josh Triplett <josh@joshtriplett.org>
Subject: lglock: map to spinlock when !CONFIG_SMP

When the system has only one CPU, lglock is effectively a spinlock; map it
directly to spinlock to eliminate the indirection and duplicate code.

In addition to removing overhead, this drops 1.6k of code with a defconfig
modified to have !CONFIG_SMP, and 1.1k with a minimal config.

Signed-off-by: Josh Triplett <josh@joshtriplett.org>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Michal Marek <mmarek@suse.cz>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: David Howells <dhowells@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Nick Piggin <npiggin@kernel.dk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/lglock.h  |   16 ++++++++++++++++
 kernel/locking/Makefile |    3 ++-
 2 files changed, 18 insertions(+), 1 deletion(-)

diff -puN include/linux/lglock.h~lglock-map-to-spinlock-when-config_smp include/linux/lglock.h
--- a/include/linux/lglock.h~lglock-map-to-spinlock-when-config_smp
+++ a/include/linux/lglock.h
@@ -25,6 +25,8 @@
 #include <linux/cpu.h>
 #include <linux/notifier.h>
 
+#ifdef CONFIG_SMP
+
 #ifdef CONFIG_DEBUG_LOCK_ALLOC
 #define LOCKDEP_INIT_MAP lockdep_init_map
 #else
@@ -57,4 +59,18 @@ void lg_local_unlock_cpu(struct lglock *
 void lg_global_lock(struct lglock *lg);
 void lg_global_unlock(struct lglock *lg);
 
+#else
+/* When !CONFIG_SMP, map lglock to spinlock */
+#define lglock spinlock
+#define DEFINE_LGLOCK(name) DEFINE_SPINLOCK(name)
+#define DEFINE_STATIC_LGLOCK(name) static DEFINE_SPINLOCK(name)
+#define lg_lock_init(lg, name) spin_lock_init(lg)
+#define lg_local_lock spin_lock
+#define lg_local_unlock spin_unlock
+#define lg_local_lock_cpu(lg, cpu) spin_lock(lg)
+#define lg_local_unlock_cpu(lg, cpu) spin_unlock(lg)
+#define lg_global_lock spin_lock
+#define lg_global_unlock spin_unlock
+#endif
+
 #endif
diff -puN kernel/locking/Makefile~lglock-map-to-spinlock-when-config_smp kernel/locking/Makefile
--- a/kernel/locking/Makefile~lglock-map-to-spinlock-when-config_smp
+++ a/kernel/locking/Makefile
@@ -1,5 +1,5 @@
 
-obj-y += mutex.o semaphore.o rwsem.o lglock.o mcs_spinlock.o
+obj-y += mutex.o semaphore.o rwsem.o mcs_spinlock.o
 
 ifdef CONFIG_FUNCTION_TRACER
 CFLAGS_REMOVE_lockdep.o = -pg
@@ -14,6 +14,7 @@ ifeq ($(CONFIG_PROC_FS),y)
 obj-$(CONFIG_LOCKDEP) += lockdep_proc.o
 endif
 obj-$(CONFIG_SMP) += spinlock.o
+obj-$(CONFIG_SMP) += lglock.o
 obj-$(CONFIG_PROVE_LOCKING) += spinlock.o
 obj-$(CONFIG_RT_MUTEXES) += rtmutex.o
 obj-$(CONFIG_DEBUG_RT_MUTEXES) += rtmutex-debug.o
_

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2014-03-17  9:31 Stephen Rothwell
@ 2014-03-17  9:36 ` Peter Zijlstra
  2014-03-19 23:27   ` Andrew Morton
  0 siblings, 1 reply; 87+ messages in thread
From: Peter Zijlstra @ 2014-03-17  9:36 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	linux-next, linux-kernel, Josh Triplett

On Mon, Mar 17, 2014 at 08:31:24PM +1100, Stephen Rothwell wrote:
> Hi Andrew,
> 
> Today's linux-next merge of the akpm-current tree got a conflict in
> kernel/locking/Makefile between commit fb0527bd5ea9 ("locking/mutexes:
> Introduce cancelable MCS lock for adaptive spinning") from the  tree and
> commit 4dc0fe493027 ("lglock: map to spinlock when !CONFIG_SMP") from the
> akpm-current tree.
> 
> I fixed it up (see below) and can carry the fix as necessary (no action
> is required).

I'm a bit sad of not having seen that lglock patch at all.

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2014-03-17  9:31 Stephen Rothwell
  2014-03-17  9:36 ` Peter Zijlstra
  0 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2014-03-17  9:31 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Josh Triplett

[-- Attachment #1: Type: text/plain, Size: 890 bytes --]

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in
kernel/locking/Makefile between commit fb0527bd5ea9 ("locking/mutexes:
Introduce cancelable MCS lock for adaptive spinning") from the  tree and
commit 4dc0fe493027 ("lglock: map to spinlock when !CONFIG_SMP") from the
akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary (no action
is required).

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

diff --cc kernel/locking/Makefile
index 306a76b51e0f,727fefd00c71..000000000000
--- a/kernel/locking/Makefile
+++ b/kernel/locking/Makefile
@@@ -1,5 -1,5 +1,5 @@@
  
- obj-y += mutex.o semaphore.o rwsem.o lglock.o mcs_spinlock.o
 -obj-y += mutex.o semaphore.o rwsem.o
++obj-y += mutex.o semaphore.o rwsem.o mcs_spinlock.o
  
  ifdef CONFIG_FUNCTION_TRACER
  CFLAGS_REMOVE_lockdep.o = -pg

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2014-01-14 13:17   ` Geert Uytterhoeven
  2014-01-14 13:33     ` Peter Zijlstra
@ 2014-01-14 16:19     ` H. Peter Anvin
  1 sibling, 0 replies; 87+ messages in thread
From: H. Peter Anvin @ 2014-01-14 16:19 UTC (permalink / raw)
  To: Geert Uytterhoeven, Peter Zijlstra
  Cc: Stephen Rothwell, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	Linux-Next, linux-kernel, Davidlohr Bueso

On 01/14/2014 05:17 AM, Geert Uytterhoeven wrote:
>>
>> This seems terribly broken, the *futex_value*() ops should not need
>> that; they are supposed to access userspace without any of that.
> 
> Why don't they need set_fs(USER_DS)?
> 
> Gr{oetje,eeting}s,
> 
>                         Geert

Because USER_DS is the normal operating state?  It would appear m68k is
the only(?) arch that calls initcalls with get_fs() == KERNEL_DS...

	-hpa

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2014-01-14 15:41       ` Peter Zijlstra
@ 2014-01-14 15:48         ` H. Peter Anvin
  0 siblings, 0 replies; 87+ messages in thread
From: H. Peter Anvin @ 2014-01-14 15:48 UTC (permalink / raw)
  To: Peter Zijlstra, Geert Uytterhoeven
  Cc: Stephen Rothwell, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	Linux-Next, linux-kernel, Davidlohr Bueso

On 01/14/2014 07:41 AM, Peter Zijlstra wrote:
>>>
>>> I am *guessing* that m68k is has get_fs() == KERNEL_DS at the point that
>>> futex_init() is called.  This would seem a bit of a peculiarity to m68k,
>>> and as such it would seem like it would be better for it to belong in
>>> the m68k-specific code, but since futex_init() is init code and only
>>> called once anyway it shouldn't cause any harm...
>>
>> Yes it does. So when getting the exception on 68030, we notice it's a kernel
>> space access error, not a user space access error, and crash.
> 
> Is there a good reason m68k works like this? That is, shouldn't we fix
> the arch code instead of littering the generic code with weirdness like
> this?
> 

Given that futex_init is called from initcall, this seems *really* weird
on the part of m68k.  I agree this should be fixed where the problem sits.

	-hpa

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2014-01-14 15:20     ` Geert Uytterhoeven
@ 2014-01-14 15:41       ` Peter Zijlstra
  2014-01-14 15:48         ` H. Peter Anvin
  0 siblings, 1 reply; 87+ messages in thread
From: Peter Zijlstra @ 2014-01-14 15:41 UTC (permalink / raw)
  To: Geert Uytterhoeven
  Cc: H. Peter Anvin, Stephen Rothwell, Andrew Morton, Thomas Gleixner,
	Ingo Molnar, Linux-Next, linux-kernel, Davidlohr Bueso

On Tue, Jan 14, 2014 at 04:20:36PM +0100, Geert Uytterhoeven wrote:
> On Tue, Jan 14, 2014 at 4:15 PM, H. Peter Anvin <hpa@zytor.com> wrote:
> > On 01/14/2014 04:51 AM, Peter Zijlstra wrote:
> >> On Tue, Jan 14, 2014 at 03:53:31PM +1100, Stephen Rothwell wrote:
> >>> Hi Andrew,
> >>>
> >>> Today's linux-next merge of the akpm-current tree got a conflict in
> >>> kernel/futex.c between commit a52b89ebb6d4 ("futexes: Increase hash table
> >>> size for better performance") from the tip tree and commit 61beee6c76e5
> >>> ("futex: switch to USER_DS for futex test") from the akpm-current tree.
> >>>
> >>> @@@ -2869,10 -2748,13 +2871,13 @@@
> >>>       * implementation, the non-functional ones will return
> >>>       * -ENOSYS.
> >>>       */
> >>> +    fs = get_fs();
> >>> +    set_fs(USER_DS);
> >>>      if (cmpxchg_futex_value_locked(&curval, NULL, 0, 0) == -EFAULT)
> >>>              futex_cmpxchg_enabled = 1;
> >>> +    set_fs(fs);
> >>>
> >>
> >> This seems terribly broken, the *futex_value*() ops should not need
> >> that; they are supposed to access userspace without any of that.
> >
> > I am *guessing* that m68k is has get_fs() == KERNEL_DS at the point that
> > futex_init() is called.  This would seem a bit of a peculiarity to m68k,
> > and as such it would seem like it would be better for it to belong in
> > the m68k-specific code, but since futex_init() is init code and only
> > called once anyway it shouldn't cause any harm...
> 
> Yes it does. So when getting the exception on 68030, we notice it's a kernel
> space access error, not a user space access error, and crash.

Is there a good reason m68k works like this? That is, shouldn't we fix
the arch code instead of littering the generic code with weirdness like
this?

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2014-01-14 15:15   ` H. Peter Anvin
@ 2014-01-14 15:20     ` Geert Uytterhoeven
  2014-01-14 15:41       ` Peter Zijlstra
  0 siblings, 1 reply; 87+ messages in thread
From: Geert Uytterhoeven @ 2014-01-14 15:20 UTC (permalink / raw)
  To: H. Peter Anvin
  Cc: Peter Zijlstra, Stephen Rothwell, Andrew Morton, Thomas Gleixner,
	Ingo Molnar, Linux-Next, linux-kernel, Davidlohr Bueso

On Tue, Jan 14, 2014 at 4:15 PM, H. Peter Anvin <hpa@zytor.com> wrote:
> On 01/14/2014 04:51 AM, Peter Zijlstra wrote:
>> On Tue, Jan 14, 2014 at 03:53:31PM +1100, Stephen Rothwell wrote:
>>> Hi Andrew,
>>>
>>> Today's linux-next merge of the akpm-current tree got a conflict in
>>> kernel/futex.c between commit a52b89ebb6d4 ("futexes: Increase hash table
>>> size for better performance") from the tip tree and commit 61beee6c76e5
>>> ("futex: switch to USER_DS for futex test") from the akpm-current tree.
>>>
>>> @@@ -2869,10 -2748,13 +2871,13 @@@
>>>       * implementation, the non-functional ones will return
>>>       * -ENOSYS.
>>>       */
>>> +    fs = get_fs();
>>> +    set_fs(USER_DS);
>>>      if (cmpxchg_futex_value_locked(&curval, NULL, 0, 0) == -EFAULT)
>>>              futex_cmpxchg_enabled = 1;
>>> +    set_fs(fs);
>>>
>>
>> This seems terribly broken, the *futex_value*() ops should not need
>> that; they are supposed to access userspace without any of that.
>
> I am *guessing* that m68k is has get_fs() == KERNEL_DS at the point that
> futex_init() is called.  This would seem a bit of a peculiarity to m68k,
> and as such it would seem like it would be better for it to belong in
> the m68k-specific code, but since futex_init() is init code and only
> called once anyway it shouldn't cause any harm...

Yes it does. So when getting the exception on 68030, we notice it's a kernel
space access error, not a user space access error, and crash.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2014-01-14 12:51 ` Peter Zijlstra
  2014-01-14 13:17   ` Geert Uytterhoeven
@ 2014-01-14 15:15   ` H. Peter Anvin
  2014-01-14 15:20     ` Geert Uytterhoeven
  1 sibling, 1 reply; 87+ messages in thread
From: H. Peter Anvin @ 2014-01-14 15:15 UTC (permalink / raw)
  To: Peter Zijlstra, Stephen Rothwell
  Cc: Andrew Morton, Thomas Gleixner, Ingo Molnar, linux-next,
	linux-kernel, Geert Uytterhoeven, Davidlohr Bueso

On 01/14/2014 04:51 AM, Peter Zijlstra wrote:
> On Tue, Jan 14, 2014 at 03:53:31PM +1100, Stephen Rothwell wrote:
>> Hi Andrew,
>>
>> Today's linux-next merge of the akpm-current tree got a conflict in
>> kernel/futex.c between commit a52b89ebb6d4 ("futexes: Increase hash table
>> size for better performance") from the tip tree and commit 61beee6c76e5
>> ("futex: switch to USER_DS for futex test") from the akpm-current tree.
>>
>> @@@ -2869,10 -2748,13 +2871,13 @@@
>>   	 * implementation, the non-functional ones will return
>>   	 * -ENOSYS.
>>   	 */
>> + 	fs = get_fs();
>> + 	set_fs(USER_DS);
>>   	if (cmpxchg_futex_value_locked(&curval, NULL, 0, 0) == -EFAULT)
>>   		futex_cmpxchg_enabled = 1;
>> + 	set_fs(fs);
>>   
> 
> This seems terribly broken, the *futex_value*() ops should not need
> that; they are supposed to access userspace without any of that.
> 

I am *guessing* that m68k is has get_fs() == KERNEL_DS at the point that
futex_init() is called.  This would seem a bit of a peculiarity to m68k,
and as such it would seem like it would be better for it to belong in
the m68k-specific code, but since futex_init() is init code and only
called once anyway it shouldn't cause any harm...

	-hpa

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2014-01-14 13:17   ` Geert Uytterhoeven
@ 2014-01-14 13:33     ` Peter Zijlstra
  2014-01-14 16:19     ` H. Peter Anvin
  1 sibling, 0 replies; 87+ messages in thread
From: Peter Zijlstra @ 2014-01-14 13:33 UTC (permalink / raw)
  To: Geert Uytterhoeven
  Cc: Stephen Rothwell, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Linux-Next, linux-kernel, Davidlohr Bueso

On Tue, Jan 14, 2014 at 02:17:55PM +0100, Geert Uytterhoeven wrote:
> On Tue, Jan 14, 2014 at 1:51 PM, Peter Zijlstra <peterz@infradead.org> wrote:
> > On Tue, Jan 14, 2014 at 03:53:31PM +1100, Stephen Rothwell wrote:
> >> Today's linux-next merge of the akpm-current tree got a conflict in
> >> kernel/futex.c between commit a52b89ebb6d4 ("futexes: Increase hash table
> >> size for better performance") from the tip tree and commit 61beee6c76e5
> >> ("futex: switch to USER_DS for futex test") from the akpm-current tree.
> >>
> >> @@@ -2869,10 -2748,13 +2871,13 @@@
> >>        * implementation, the non-functional ones will return
> >>        * -ENOSYS.
> >>        */
> >> +     fs = get_fs();
> >> +     set_fs(USER_DS);
> >>       if (cmpxchg_futex_value_locked(&curval, NULL, 0, 0) == -EFAULT)
> >>               futex_cmpxchg_enabled = 1;
> >> +     set_fs(fs);
> >>
> >
> > This seems terribly broken, the *futex_value*() ops should not need
> > that; they are supposed to access userspace without any of that.
> 
> Why don't they need set_fs(USER_DS)?

Why would they need it? These functions explicitly take a __user pointer
and are expected to do whatever is needed to perform the operation. None
of the other futex bits use USER_DS either.


Furthermore, from the Changelog of:
e4f2dfbb5e92be4e46c0625f4f8eb101110f756f

"   This patch adds that support, but only for uniprocessor machines,
    which is adequate for M68K.  For UP it's enough to disable preemption
    to ensure mutual exclusion (futexes don't need to care about other
    hardware agents), and the mandatory pagefault_disable() does just that.  "

It is wrong to rely on the fact that pagefault_disable() disables
preemption. This is fact not true on -rt.

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2014-01-14 12:51 ` Peter Zijlstra
@ 2014-01-14 13:17   ` Geert Uytterhoeven
  2014-01-14 13:33     ` Peter Zijlstra
  2014-01-14 16:19     ` H. Peter Anvin
  2014-01-14 15:15   ` H. Peter Anvin
  1 sibling, 2 replies; 87+ messages in thread
From: Geert Uytterhoeven @ 2014-01-14 13:17 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Stephen Rothwell, Andrew Morton, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, Linux-Next, linux-kernel, Davidlohr Bueso

On Tue, Jan 14, 2014 at 1:51 PM, Peter Zijlstra <peterz@infradead.org> wrote:
> On Tue, Jan 14, 2014 at 03:53:31PM +1100, Stephen Rothwell wrote:
>> Today's linux-next merge of the akpm-current tree got a conflict in
>> kernel/futex.c between commit a52b89ebb6d4 ("futexes: Increase hash table
>> size for better performance") from the tip tree and commit 61beee6c76e5
>> ("futex: switch to USER_DS for futex test") from the akpm-current tree.
>>
>> @@@ -2869,10 -2748,13 +2871,13 @@@
>>        * implementation, the non-functional ones will return
>>        * -ENOSYS.
>>        */
>> +     fs = get_fs();
>> +     set_fs(USER_DS);
>>       if (cmpxchg_futex_value_locked(&curval, NULL, 0, 0) == -EFAULT)
>>               futex_cmpxchg_enabled = 1;
>> +     set_fs(fs);
>>
>
> This seems terribly broken, the *futex_value*() ops should not need
> that; they are supposed to access userspace without any of that.

Why don't they need set_fs(USER_DS)?

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2014-01-14  4:53 Stephen Rothwell
  2014-01-14  5:04 ` Davidlohr Bueso
@ 2014-01-14 12:51 ` Peter Zijlstra
  2014-01-14 13:17   ` Geert Uytterhoeven
  2014-01-14 15:15   ` H. Peter Anvin
  1 sibling, 2 replies; 87+ messages in thread
From: Peter Zijlstra @ 2014-01-14 12:51 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	linux-next, linux-kernel, Geert Uytterhoeven, Davidlohr Bueso

On Tue, Jan 14, 2014 at 03:53:31PM +1100, Stephen Rothwell wrote:
> Hi Andrew,
> 
> Today's linux-next merge of the akpm-current tree got a conflict in
> kernel/futex.c between commit a52b89ebb6d4 ("futexes: Increase hash table
> size for better performance") from the tip tree and commit 61beee6c76e5
> ("futex: switch to USER_DS for futex test") from the akpm-current tree.
> 
> @@@ -2869,10 -2748,13 +2871,13 @@@
>   	 * implementation, the non-functional ones will return
>   	 * -ENOSYS.
>   	 */
> + 	fs = get_fs();
> + 	set_fs(USER_DS);
>   	if (cmpxchg_futex_value_locked(&curval, NULL, 0, 0) == -EFAULT)
>   		futex_cmpxchg_enabled = 1;
> + 	set_fs(fs);
>   

This seems terribly broken, the *futex_value*() ops should not need
that; they are supposed to access userspace without any of that.

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2014-01-14  4:53 Stephen Rothwell
@ 2014-01-14  5:04 ` Davidlohr Bueso
  2014-01-14 12:51 ` Peter Zijlstra
  1 sibling, 0 replies; 87+ messages in thread
From: Davidlohr Bueso @ 2014-01-14  5:04 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra, linux-next, linux-kernel, Geert Uytterhoeven

On Tue, 2014-01-14 at 15:53 +1100, Stephen Rothwell wrote:
> Hi Andrew,
> 
> Today's linux-next merge of the akpm-current tree got a conflict in
> kernel/futex.c between commit a52b89ebb6d4 ("futexes: Increase hash table
> size for better performance") from the tip tree and commit 61beee6c76e5
> ("futex: switch to USER_DS for futex test") from the akpm-current tree.
> 
> I fixed it up (see below) and can carry the fix as necessary (no action
> is required).

Thanks Stephen. At least for my bits the fix seems ok. In the future,
though, changes to this evil file should be routed only through one
tree.

Thanks,
Davidlohr

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2014-01-14  4:53 Stephen Rothwell
  2014-01-14  5:04 ` Davidlohr Bueso
  2014-01-14 12:51 ` Peter Zijlstra
  0 siblings, 2 replies; 87+ messages in thread
From: Stephen Rothwell @ 2014-01-14  4:53 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Geert Uytterhoeven, Davidlohr Bueso

[-- Attachment #1: Type: text/plain, Size: 1907 bytes --]

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in
kernel/futex.c between commit a52b89ebb6d4 ("futexes: Increase hash table
size for better performance") from the tip tree and commit 61beee6c76e5
("futex: switch to USER_DS for futex test") from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary (no action
is required).

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

diff --cc kernel/futex.c
index 3666aa0017ed,66d23727c6ab..000000000000
--- a/kernel/futex.c
+++ b/kernel/futex.c
@@@ -63,7 -63,7 +63,8 @@@
  #include <linux/sched/rt.h>
  #include <linux/hugetlb.h>
  #include <linux/freezer.h>
 +#include <linux/bootmem.h>
+ #include <linux/uaccess.h>
  
  #include <asm/futex.h>
  
@@@ -2845,19 -2734,9 +2846,20 @@@ SYSCALL_DEFINE6(futex, u32 __user *, ua
  
  static int __init futex_init(void)
  {
+ 	mm_segment_t fs;
  	u32 curval;
 -	int i;
 +	unsigned long i;
 +
 +#if CONFIG_BASE_SMALL
 +	futex_hashsize = 16;
 +#else
 +	futex_hashsize = roundup_pow_of_two(256 * num_possible_cpus());
 +#endif
 +
 +	futex_queues = alloc_large_system_hash("futex", sizeof(*futex_queues),
 +					       futex_hashsize, 0,
 +					       futex_hashsize < 256 ? HASH_SMALL : 0,
 +					       NULL, NULL, futex_hashsize, futex_hashsize);
  
  	/*
  	 * This will fail and we want it. Some arch implementations do
@@@ -2869,10 -2748,13 +2871,13 @@@
  	 * implementation, the non-functional ones will return
  	 * -ENOSYS.
  	 */
+ 	fs = get_fs();
+ 	set_fs(USER_DS);
  	if (cmpxchg_futex_value_locked(&curval, NULL, 0, 0) == -EFAULT)
  		futex_cmpxchg_enabled = 1;
+ 	set_fs(fs);
  
 -	for (i = 0; i < ARRAY_SIZE(futex_queues); i++) {
 +	for (i = 0; i < futex_hashsize; i++) {
  		plist_head_init(&futex_queues[i].chain);
  		spin_lock_init(&futex_queues[i].lock);
  	}

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2013-11-08 23:20   ` Stephen Rothwell
@ 2013-11-09  0:19     ` Josh Triplett
  0 siblings, 0 replies; 87+ messages in thread
From: Josh Triplett @ 2013-11-09  0:19 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra, linux-next, linux-kernel

On Sat, Nov 09, 2013 at 10:20:58AM +1100, Stephen Rothwell wrote:
> Hi Josh,
> 
> On Fri, 8 Nov 2013 10:58:12 -0800 Josh Triplett <josh@joshtriplett.org> wrote:
> >
> > Won't splitting the Makefile change into a separate commit break
> > bisection, in particular if you have the changes adding inlines but you
> > also compile in lglock.o?  Shouldn't this be squashed into the merge
> > itself, keeping the kernel/Makefile section of my original patch?
> 
> Actually it is not a problem because that fix patch was applied to the
> merge commit between the part of Andrew's tree that depends only on
> Linus' tree and the rest of linux-next.  So each side of the merge is ok
> and the merge commit itself fixes up the conflict.
> 
> I just split it this way for my work flow purposes.

Ah, I see.  That wasn't obvious to me from your previous mail explaining
your fix. :)

- Josh Triplett

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2013-11-08 18:58 ` Josh Triplett
@ 2013-11-08 23:20   ` Stephen Rothwell
  2013-11-09  0:19     ` Josh Triplett
  0 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2013-11-08 23:20 UTC (permalink / raw)
  To: Josh Triplett
  Cc: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra, linux-next, linux-kernel

[-- Attachment #1: Type: text/plain, Size: 775 bytes --]

Hi Josh,

On Fri, 8 Nov 2013 10:58:12 -0800 Josh Triplett <josh@joshtriplett.org> wrote:
>
> Won't splitting the Makefile change into a separate commit break
> bisection, in particular if you have the changes adding inlines but you
> also compile in lglock.o?  Shouldn't this be squashed into the merge
> itself, keeping the kernel/Makefile section of my original patch?

Actually it is not a problem because that fix patch was applied to the
merge commit between the part of Andrew's tree that depends only on
Linus' tree and the rest of linux-next.  So each side of the merge is ok
and the merge commit itself fixes up the conflict.

I just split it this way for my work flow purposes.
-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

* Re: linux-next: manual merge of the akpm-current tree with the tip tree
  2013-11-08  7:48 Stephen Rothwell
@ 2013-11-08 18:58 ` Josh Triplett
  2013-11-08 23:20   ` Stephen Rothwell
  0 siblings, 1 reply; 87+ messages in thread
From: Josh Triplett @ 2013-11-08 18:58 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra, linux-next, linux-kernel

On Fri, Nov 08, 2013 at 06:48:05PM +1100, Stephen Rothwell wrote:
> Hi Andrew,
> 
> Today's linux-next merge of the akpm-current tree got a conflict in
> kernel/Makefile between commits 60fc28746a7b ("locking: Move the spinlock
> code to kernel/locking/") and cd4d241d57c9 ("locking: Move the lglocks
> code to kernel/locking/") from the tip tree and commit f5639052d567
> ("lglock: map to spinlock when !CONFIG_SMP") from the akpm-current tree.
> 
> I fixed it up (dropping the kernel/Makefile section of the akpm-current
> commit and adding the below patch) and can carry the fix as necessary (no
> action is required).

Won't splitting the Makefile change into a separate commit break
bisection, in particular if you have the changes adding inlines but you
also compile in lglock.o?  Shouldn't this be squashed into the merge
itself, keeping the kernel/Makefile section of my original patch?

> From: Stephen Rothwell <sfr@canb.auug.org.au>
> Date: Fri, 8 Nov 2013 18:45:25 +1100
> Subject: [PATCH] lglock: fixup for code movement
> 
> Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
> ---
>  kernel/locking/Makefile | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/locking/Makefile b/kernel/locking/Makefile
> index baab8e5e7f66..bb3c65930a20 100644
> --- a/kernel/locking/Makefile
> +++ b/kernel/locking/Makefile
> @@ -1,5 +1,5 @@
>  
> -obj-y += mutex.o semaphore.o rwsem.o lglock.o
> +obj-y += mutex.o semaphore.o rwsem.o
>  
>  ifdef CONFIG_FUNCTION_TRACER
>  CFLAGS_REMOVE_lockdep.o = -pg
> @@ -8,6 +8,7 @@ CFLAGS_REMOVE_mutex-debug.o = -pg
>  CFLAGS_REMOVE_rtmutex-debug.o = -pg
>  endif
>  
> +obj-$(CONFIG_SMP) += lglock.o
>  obj-$(CONFIG_DEBUG_MUTEXES) += mutex-debug.o
>  obj-$(CONFIG_LOCKDEP) += lockdep.o
>  ifeq ($(CONFIG_PROC_FS),y)
> -- 
> 1.8.4.2
> 
> -- 
> Cheers,
> Stephen Rothwell                    sfr@canb.auug.org.au

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2013-11-08  7:48 Stephen Rothwell
  2013-11-08 18:58 ` Josh Triplett
  0 siblings, 1 reply; 87+ messages in thread
From: Stephen Rothwell @ 2013-11-08  7:48 UTC (permalink / raw)
  To: Andrew Morton, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	Peter Zijlstra
  Cc: linux-next, linux-kernel, Josh Triplett

[-- Attachment #1: Type: text/plain, Size: 1503 bytes --]

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in
kernel/Makefile between commits 60fc28746a7b ("locking: Move the spinlock
code to kernel/locking/") and cd4d241d57c9 ("locking: Move the lglocks
code to kernel/locking/") from the tip tree and commit f5639052d567
("lglock: map to spinlock when !CONFIG_SMP") from the akpm-current tree.

I fixed it up (dropping the kernel/Makefile section of the akpm-current
commit and adding the below patch) and can carry the fix as necessary (no
action is required).

From: Stephen Rothwell <sfr@canb.auug.org.au>
Date: Fri, 8 Nov 2013 18:45:25 +1100
Subject: [PATCH] lglock: fixup for code movement

Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
---
 kernel/locking/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/kernel/locking/Makefile b/kernel/locking/Makefile
index baab8e5e7f66..bb3c65930a20 100644
--- a/kernel/locking/Makefile
+++ b/kernel/locking/Makefile
@@ -1,5 +1,5 @@
 
-obj-y += mutex.o semaphore.o rwsem.o lglock.o
+obj-y += mutex.o semaphore.o rwsem.o
 
 ifdef CONFIG_FUNCTION_TRACER
 CFLAGS_REMOVE_lockdep.o = -pg
@@ -8,6 +8,7 @@ CFLAGS_REMOVE_mutex-debug.o = -pg
 CFLAGS_REMOVE_rtmutex-debug.o = -pg
 endif
 
+obj-$(CONFIG_SMP) += lglock.o
 obj-$(CONFIG_DEBUG_MUTEXES) += mutex-debug.o
 obj-$(CONFIG_LOCKDEP) += lockdep.o
 ifeq ($(CONFIG_PROC_FS),y)
-- 
1.8.4.2

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

* linux-next: manual merge of the akpm-current tree with the tip tree
@ 2013-10-30  6:40 Stephen Rothwell
  0 siblings, 0 replies; 87+ messages in thread
From: Stephen Rothwell @ 2013-10-30  6:40 UTC (permalink / raw)
  To: Andrew Morton
  Cc: linux-next, linux-kernel, Yinghai Lu, Thomas Gleixner,
	Ingo Molnar, H. Peter Anvin, Peter Zijlstra, Tang Chen,
	Zhang Yanfei

[-- Attachment #1: Type: text/plain, Size: 5339 bytes --]

Hi Andrew,

Today's linux-next merge of the akpm-current tree got a conflict in
arch/x86/mm/init.c between commit 6979287a7df6 ("x86/mm: Add 'step_size'
comments to init_mem_mapping()") from the tip tree and commits
6452c268c6d6 ("x86/mm: factor out of top-down direct mapping setup") and
f790250c098a ("x86/mem-hotplug: support initialize page tables in
bottom-up") from the akpm-current tree.

I fixed it up (see below) and can carry the fix as necessary (no action
is required).

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

diff --cc arch/x86/mm/init.c
index ce32017c5e38,b6892a71cbfc..000000000000
--- a/arch/x86/mm/init.c
+++ b/arch/x86/mm/init.c
@@@ -399,28 -399,23 +399,39 @@@ static unsigned long __init init_range_
  	return mapped_ram_size;
  }
  
 -/* (PUD_SHIFT-PMD_SHIFT)/2 */
 -#define STEP_SIZE_SHIFT 5
 +static unsigned long __init get_new_step_size(unsigned long step_size)
 +{
 +	/*
 +	 * Explain why we shift by 5 and why we don't have to worry about
 +	 * 'step_size << 5' overflowing:
 +	 *
 +	 * initial mapped size is PMD_SIZE (2M).
 +	 * We can not set step_size to be PUD_SIZE (1G) yet.
 +	 * In worse case, when we cross the 1G boundary, and
 +	 * PG_LEVEL_2M is not set, we will need 1+1+512 pages (2M + 8k)
 +	 * to map 1G range with PTE. Use 5 as shift for now.
 +	 *
 +	 * Don't need to worry about overflow, on 32bit, when step_size
 +	 * is 0, round_down() returns 0 for start, and that turns it
 +	 * into 0x100000000ULL.
 +	 */
 +	return step_size << 5;
 +}
  
- void __init init_mem_mapping(void)
+ /**
+  * memory_map_top_down - Map [map_start, map_end) top down
+  * @map_start: start address of the target memory range
+  * @map_end: end address of the target memory range
+  *
+  * This function will setup direct mapping for memory range
+  * [map_start, map_end) in top-down. That said, the page tables
+  * will be allocated at the end of the memory, and we map the
+  * memory in top-down.
+  */
+ static void __init memory_map_top_down(unsigned long map_start,
+ 				       unsigned long map_end)
  {
- 	unsigned long end, real_end, start, last_start;
+ 	unsigned long real_end, start, last_start;
  	unsigned long step_size;
  	unsigned long addr;
  	unsigned long mapped_ram_size = 0;
@@@ -470,8 -454,89 +470,89 @@@
  		mapped_ram_size += new_mapped_ram_size;
  	}
  
- 	if (real_end < end)
- 		init_range_memory_mapping(real_end, end);
+ 	if (real_end < map_end)
+ 		init_range_memory_mapping(real_end, map_end);
+ }
+ 
+ /**
+  * memory_map_bottom_up - Map [map_start, map_end) bottom up
+  * @map_start: start address of the target memory range
+  * @map_end: end address of the target memory range
+  *
+  * This function will setup direct mapping for memory range
+  * [map_start, map_end) in bottom-up. Since we have limited the
+  * bottom-up allocation above the kernel, the page tables will
+  * be allocated just above the kernel and we map the memory
+  * in [map_start, map_end) in bottom-up.
+  */
+ static void __init memory_map_bottom_up(unsigned long map_start,
+ 					unsigned long map_end)
+ {
+ 	unsigned long next, new_mapped_ram_size, start;
+ 	unsigned long mapped_ram_size = 0;
+ 	/* step_size need to be small so pgt_buf from BRK could cover it */
+ 	unsigned long step_size = PMD_SIZE;
+ 
+ 	start = map_start;
+ 	min_pfn_mapped = start >> PAGE_SHIFT;
+ 
+ 	/*
+ 	 * We start from the bottom (@map_start) and go to the top (@map_end).
+ 	 * The memblock_find_in_range() gets us a block of RAM from the
+ 	 * end of RAM in [min_pfn_mapped, max_pfn_mapped) used as new pages
+ 	 * for page table.
+ 	 */
+ 	while (start < map_end) {
+ 		if (map_end - start > step_size) {
+ 			next = round_up(start + 1, step_size);
+ 			if (next > map_end)
+ 				next = map_end;
+ 		} else
+ 			next = map_end;
+ 
+ 		new_mapped_ram_size = init_range_memory_mapping(start, next);
+ 		start = next;
+ 
+ 		if (new_mapped_ram_size > mapped_ram_size)
 -			step_size <<= STEP_SIZE_SHIFT;
++			step_size = get_new_step_size(step_size);
+ 		mapped_ram_size += new_mapped_ram_size;
+ 	}
+ }
+ 
+ void __init init_mem_mapping(void)
+ {
+ 	unsigned long end;
+ 
+ 	probe_page_size_mask();
+ 
+ #ifdef CONFIG_X86_64
+ 	end = max_pfn << PAGE_SHIFT;
+ #else
+ 	end = max_low_pfn << PAGE_SHIFT;
+ #endif
+ 
+ 	/* the ISA range is always mapped regardless of memory holes */
+ 	init_memory_mapping(0, ISA_END_ADDRESS);
+ 
+ 	/*
+ 	 * If the allocation is in bottom-up direction, we setup direct mapping
+ 	 * in bottom-up, otherwise we setup direct mapping in top-down.
+ 	 */
+ 	if (memblock_bottom_up()) {
+ 		unsigned long kernel_end = __pa_symbol(_end);
+ 
+ 		/*
+ 		 * we need two separate calls here. This is because we want to
+ 		 * allocate page tables above the kernel. So we first map
+ 		 * [kernel_end, end) to make memory above the kernel be mapped
+ 		 * as soon as possible. And then use page tables allocated above
+ 		 * the kernel to map [ISA_END_ADDRESS, kernel_end).
+ 		 */
+ 		memory_map_bottom_up(kernel_end, end);
+ 		memory_map_bottom_up(ISA_END_ADDRESS, kernel_end);
+ 	} else {
+ 		memory_map_top_down(ISA_END_ADDRESS, end);
+ 	}
  
  #ifdef CONFIG_X86_64
  	if (max_pfn > max_low_pfn) {

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 87+ messages in thread

end of thread, back to index

Thread overview: 87+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-01-07  6:00 linux-next: manual merge of the akpm-current tree with the tip tree Stephen Rothwell
2014-01-07  6:34 ` Tang Chen
  -- strict thread matches above, loose matches on Subject: below --
2019-10-31  5:43 Stephen Rothwell
2019-06-24 10:24 Stephen Rothwell
2019-05-01 11:10 Stephen Rothwell
2019-01-31  4:31 Stephen Rothwell
2018-08-20  4:32 Stephen Rothwell
2018-08-20 19:52 ` Andrew Morton
2018-03-23  5:59 Stephen Rothwell
2017-12-18  5:04 Stephen Rothwell
2017-11-10  4:33 Stephen Rothwell
2017-11-02  7:19 Stephen Rothwell
2017-08-22  6:57 Stephen Rothwell
2017-08-23  6:39 ` Vlastimil Babka
2017-08-11  7:53 Stephen Rothwell
2017-08-11  9:34 ` Peter Zijlstra
2017-08-11 10:48   ` Peter Zijlstra
2017-08-11 11:45   ` Stephen Rothwell
2017-08-11 11:56     ` Ingo Molnar
2017-08-11 12:17       ` Peter Zijlstra
2017-08-11 12:44         ` Ingo Molnar
2017-08-11 13:49           ` Stephen Rothwell
2017-08-11 14:04       ` Peter Zijlstra
2017-08-13  6:06         ` Nadav Amit
2017-08-13 12:50           ` Peter Zijlstra
2017-08-14  3:16             ` Minchan Kim
2017-08-14  5:07               ` Nadav Amit
2017-08-14  5:23                 ` Minchan Kim
2017-08-14  8:38                 ` Minchan Kim
2017-08-14 19:57                   ` Peter Zijlstra
2017-08-16  4:14                     ` Minchan Kim
2017-08-14 19:38                 ` Peter Zijlstra
2017-08-15  7:51                   ` Nadav Amit
2017-08-14  3:09         ` Minchan Kim
2017-08-14 18:54           ` Peter Zijlstra
2017-04-12  6:46 Stephen Rothwell
2017-04-12 20:53 ` Vlastimil Babka
2017-04-20  2:17   ` NeilBrown
2017-03-24  5:25 Stephen Rothwell
2017-02-17  4:40 Stephen Rothwell
2016-11-14  6:08 Stephen Rothwell
2016-07-29  4:14 Stephen Rothwell
2016-06-15  5:23 Stephen Rothwell
2016-06-18 19:39 ` Manfred Spraul
2016-04-29  6:12 Stephen Rothwell
2016-04-29  6:26 ` Ingo Molnar
2016-03-02  5:40 Stephen Rothwell
2016-02-26  5:07 Stephen Rothwell
2016-02-26 21:35 ` Andrew Morton
2016-02-19  4:09 Stephen Rothwell
2016-02-19 15:26 ` Ard Biesheuvel
2015-12-07  8:06 Stephen Rothwell
2015-10-02  4:21 Stephen Rothwell
2015-07-28  6:00 Stephen Rothwell
2015-07-29 17:12 ` Andrea Arcangeli
2015-07-29 17:47   ` Andy Lutomirski
2015-07-29 18:46     ` Thomas Gleixner
2015-07-30 15:38       ` Andrea Arcangeli
2015-07-29 23:06   ` Stephen Rothwell
2015-07-29 23:07     ` Thomas Gleixner
2015-09-07 23:35   ` Stephen Rothwell
2015-09-08 18:11     ` Linus Torvalds
2015-09-08 22:56       ` Stephen Rothwell
2015-09-08 23:03         ` Linus Torvalds
2015-09-08 23:21           ` Andrew Morton
2015-09-16  6:58             ` Geert Uytterhoeven
2015-06-04 12:07 Stephen Rothwell
2015-04-08  8:28 Stephen Rothwell
2015-04-08  8:25 Stephen Rothwell
2014-03-17  9:31 Stephen Rothwell
2014-03-17  9:36 ` Peter Zijlstra
2014-03-19 23:27   ` Andrew Morton
2014-01-14  4:53 Stephen Rothwell
2014-01-14  5:04 ` Davidlohr Bueso
2014-01-14 12:51 ` Peter Zijlstra
2014-01-14 13:17   ` Geert Uytterhoeven
2014-01-14 13:33     ` Peter Zijlstra
2014-01-14 16:19     ` H. Peter Anvin
2014-01-14 15:15   ` H. Peter Anvin
2014-01-14 15:20     ` Geert Uytterhoeven
2014-01-14 15:41       ` Peter Zijlstra
2014-01-14 15:48         ` H. Peter Anvin
2013-11-08  7:48 Stephen Rothwell
2013-11-08 18:58 ` Josh Triplett
2013-11-08 23:20   ` Stephen Rothwell
2013-11-09  0:19     ` Josh Triplett
2013-10-30  6:40 Stephen Rothwell

Linux-Next Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-next/0 linux-next/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-next linux-next/ https://lore.kernel.org/linux-next \
		linux-next@vger.kernel.org
	public-inbox-index linux-next

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-next


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git