linux-next: build failure after merge of the access_once tree

linux-next: build failure after merge of the access_once tree

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 935 bytes --]

Hi Christian,

After merging the access_once tree, today's linux-next build (x86_64
allmodconfig) failed like this:

In file included from include/linux/compiler.h:189:0,
                 from include/uapi/linux/stddef.h:1,
                 from include/linux/stddef.h:4,
                 from include/uapi/linux/posix_types.h:4,
                 from include/linux/fdtable.h:8,
                 from security/apparmor/domain.c:16:
include/linux/types.h:14:9: error: unknown type name '__kernel_fd_set'
 typedef __kernel_fd_set  fd_set;
         ^

And many more :-(

Presumably caused by commit b06f66134856 ("kernel: Provide READ_ONCE
and ASSIGN_ONCE") which added types.h into compiler.h, but I am not
sure.  It could be an interaction with something else in linux-next, of
course.

I have dropped the access_once tree for today, sorry.
-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: linux-next: build failure after merge of the access_once tree

[Christian Borntraeger]

Am 05.12.2014 um 11:32 schrieb Stephen Rothwell:
> Hi Christian,
> 
> After merging the access_once tree, today's linux-next build (x86_64
> allmodconfig) failed like this:
> 
> In file included from include/linux/compiler.h:189:0,
>                  from include/uapi/linux/stddef.h:1,
>                  from include/linux/stddef.h:4,
>                  from include/uapi/linux/posix_types.h:4,
>                  from include/linux/fdtable.h:8,
>                  from security/apparmor/domain.c:16:
> include/linux/types.h:14:9: error: unknown type name '__kernel_fd_set'
>  typedef __kernel_fd_set  fd_set;
>          ^
> 
> And many more :-(
> 
> Presumably caused by commit b06f66134856 ("kernel: Provide READ_ONCE
> and ASSIGN_ONCE") which added types.h into compiler.h, but I am not
> sure.  It could be an interaction with something else in linux-next, of
> course.
> 
> I have dropped the access_once tree for today, sorry.
> 

Interesting, I double checked with defconfig and cross compilation. But yes, using allmodconfig fails.
I have changed that to uapi/linux/types.h and __u<xx>. This seems to fix it as it only adds a small set of types.

Can you re-add?

Christian

Re: linux-next: build failure after merge of the access_once tree

[Christian Borntraeger]

[-- Attachment #1: Type: text/plain, Size: 1516 bytes --]

Am 07.12.2014 um 21:57 schrieb Christian Borntraeger:
> Am 05.12.2014 um 11:32 schrieb Stephen Rothwell:
>> Hi Christian,
>>
>> After merging the access_once tree, today's linux-next build (x86_64
>> allmodconfig) failed like this:
>>
>> In file included from include/linux/compiler.h:189:0,
>>                  from include/uapi/linux/stddef.h:1,
>>                  from include/linux/stddef.h:4,
>>                  from include/uapi/linux/posix_types.h:4,
>>                  from include/linux/fdtable.h:8,
>>                  from security/apparmor/domain.c:16:
>> include/linux/types.h:14:9: error: unknown type name '__kernel_fd_set'
>>  typedef __kernel_fd_set  fd_set;
>>          ^
>>
>> And many more :-(
>>
>> Presumably caused by commit b06f66134856 ("kernel: Provide READ_ONCE
>> and ASSIGN_ONCE") which added types.h into compiler.h, but I am not
>> sure.  It could be an interaction with something else in linux-next, of
>> course.
>>
>> I have dropped the access_once tree for today, sorry.
>>
> 
> Interesting, I double checked with defconfig and cross compilation. But yes, using allmodconfig fails.
> I have changed that to uapi/linux/types.h and __u<xx>. This seems to fix it as it only adds a small set of types.
> 
> Can you re-add?
> 
> Christian
> 

I also did an additional check and merged this tree with linux-next/master. This introduced a new problem.
The fix is simple, see patch. But I am not sure how to add that to next (via my tree), since this change is only in next.

Christian

[-- Attachment #2: 0001-x86-xen-p2m-Replace-ACCESS_ONCE-with-READ_ONCE.patch --]
[-- Type: text/x-patch, Size: 1113 bytes --]

>From ef62db716c9edb1343c00eae45ad8f6c0c1c9255 Mon Sep 17 00:00:00 2001
From: Christian Borntraeger <borntraeger@de.ibm.com>
Date: Sun, 7 Dec 2014 22:01:59 +0100
Subject: [PATCH] x86/xen/p2m: Replace ACCESS_ONCE with READ_ONCE

ACCESS_ONCE does not work reliably on non-scalar types. For
example gcc 4.6 and 4.7 might remove the volatile tag for such
accesses during the SRA (scalar replacement of aggregates) step
(https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58145)

Change the p2m code to replace ACCESS_ONCE with READ_ONCE.

Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/x86/xen/p2m.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/xen/p2m.c b/arch/x86/xen/p2m.c
index 8b5db51..08ddf91 100644
--- a/arch/x86/xen/p2m.c
+++ b/arch/x86/xen/p2m.c
@@ -554,7 +554,7 @@ static bool alloc_p2m(unsigned long pfn)
 		mid_mfn = NULL;
 	}
 
-	p2m_pfn = pte_pfn(ACCESS_ONCE(*ptep));
+	p2m_pfn = pte_pfn(READ_ONCE(*ptep));
 	if (p2m_pfn == PFN_DOWN(__pa(p2m_identity)) ||
 	    p2m_pfn == PFN_DOWN(__pa(p2m_missing))) {
 		/* p2m leaf page is missing */
-- 
1.8.5.5

Re: linux-next: build failure after merge of the access_once tree

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 628 bytes --]

Hi Christian,

On Sun, 07 Dec 2014 22:16:41 +0100 Christian Borntraeger <borntraeger@de.ibm.com> wrote:
>
> I also did an additional check and merged this tree with linux-next/master. This introduced a new problem.
> The fix is simple, see patch. But I am not sure how to add that to next (via my tree), since this change is only in next.

OK, thanks for that.  I will apply that patch as a merge fix to my tree
when I merge your tree.  You (or the owner of the conflicting tree)
will need to let Linus know when the merge happens in his tree.

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

linux-next: build failure after merge of the access_once tree

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 1098 bytes --]

Hi Christian,

After merging the acess_once tree, today's linux-next build (powerpc
allnoconfig) failed like this:

arch/powerpc/mm/hugetlbpage.c: In function 'find_linux_pte_or_hugepte':
arch/powerpc/mm/hugetlbpage.c:981:3: error: invalid initializer
   pud  = ACCESS_ONCE(*pudp);
   ^
arch/powerpc/mm/hugetlbpage.c:993:4: error: invalid initializer
    pmd  = ACCESS_ONCE(*pmdp);
    ^

These are preexisting ...

mm/gup.c: In function 'gup_pmd_range':
mm/gup.c:929:3: error: invalid initializer
   pmd_t pmd = ACCESS_ONCE(*pmdp);
   ^
mm/gup.c:929:3: error: (near initialization for 'pmd')

This is from commit f30c59e921f1 ("mm: Update generic gup
implementation to handle hugepage directory") from the powerpc-mpe (and
powerpc) tree and so will require a merge fix patch (presumable
s/ACCESS_ONCE/READ_ONCE/).

I am not sure how many architectures you are trying to cover, but
powerpc is one I care about :-)

I have dropped the access_once tree again today, sorry (its too late at
night).
-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: linux-next: build failure after merge of the access_once tree

[Christian Borntraeger]

Am 08.12.2014 um 12:26 schrieb Stephen Rothwell:
> Hi Christian,
> 
> After merging the acess_once tree, today's linux-next build (powerpc
> allnoconfig) failed like this:
> 
> arch/powerpc/mm/hugetlbpage.c: In function 'find_linux_pte_or_hugepte':
> arch/powerpc/mm/hugetlbpage.c:981:3: error: invalid initializer
>    pud  = ACCESS_ONCE(*pudp);
>    ^
> arch/powerpc/mm/hugetlbpage.c:993:4: error: invalid initializer
>     pmd  = ACCESS_ONCE(*pmdp);
>     ^
> 
> These are preexisting ...
> 
> mm/gup.c: In function 'gup_pmd_range':
> mm/gup.c:929:3: error: invalid initializer
>    pmd_t pmd = ACCESS_ONCE(*pmdp);
>    ^
> mm/gup.c:929:3: error: (near initialization for 'pmd')
> 
> This is from commit f30c59e921f1 ("mm: Update generic gup
> implementation to handle hugepage directory") from the powerpc-mpe (and
> powerpc) tree and so will require a merge fix patch (presumable
> s/ACCESS_ONCE/READ_ONCE/).
> 
> I am not sure how many architectures you are trying to cover, but
> powerpc is one I care about :-)
> 
> I have dropped the access_once tree again today, sorry (its too late at
> night).

Yes, we expect that tightening ACCESS_ONCE will trigger these bugs. I tried to cover everything in Linus master, but next is certainly a new game. It will be pretty hard for me to keep this running when hundreds of trees get merged during 3.19-rc. :-(

Linus, I see two options:

a: Either we go a full round in linux-next and expect all trees to follow the new rules, so we can merge this for 3.20
b: Only apply READ/ASSIGN_ONCE and the first fixups for 3.19 and put the ACCESS_ONCE change in next after rc1 so it will go in for 3.20 (or later) so that people can adopt. 

Any opinions?

Christian

Re: linux-next: build failure after merge of the access_once tree

[Christian Borntraeger]

Am 08.12.2014 um 12:26 schrieb Stephen Rothwell:
> Hi Christian,
> 
> After merging the acess_once tree, today's linux-next build (powerpc
> allnoconfig) failed like this:
> 
> arch/powerpc/mm/hugetlbpage.c: In function 'find_linux_pte_or_hugepte':
> arch/powerpc/mm/hugetlbpage.c:981:3: error: invalid initializer
>    pud  = ACCESS_ONCE(*pudp);
>    ^
> arch/powerpc/mm/hugetlbpage.c:993:4: error: invalid initializer
>     pmd  = ACCESS_ONCE(*pmdp);
>     ^
> 
> These are preexisting ...
> 
> mm/gup.c: In function 'gup_pmd_range':
> mm/gup.c:929:3: error: invalid initializer
>    pmd_t pmd = ACCESS_ONCE(*pmdp);
>    ^
> mm/gup.c:929:3: error: (near initialization for 'pmd')
> 
> This is from commit f30c59e921f1 ("mm: Update generic gup
> implementation to handle hugepage directory") from the powerpc-mpe (and
> powerpc) tree and so will require a merge fix patch (presumable
> s/ACCESS_ONCE/READ_ONCE/).
> 
> I am not sure how many architectures you are trying to cover, but
> powerpc is one I care about :-)
> 
> I have dropped the access_once tree again today, sorry (its too late at
> night).
> 

Next try. I removed the change of ACCESS_ONCE and just left the new interfaces and the simple reworks in the queue.
I will tackle everything else after rc1 when next should be small again. Keeping fingers crossed....

Christian

linux-next: build failure after merge of the access_once tree

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 1415 bytes --]

Hi Christian,

After merging the access_once tree, today's linux-next build (powerpc
ppc44x_defconfig) failed like this:

In file included from /scratch/sfr/next/include/uapi/linux/stddef.h:1:0,
                 from /scratch/sfr/next/include/linux/stddef.h:4,
                 from /scratch/sfr/next/include/uapi/linux/posix_types.h:4,
                 from /scratch/sfr/next/include/uapi/linux/types.h:13,
                 from /scratch/sfr/next/include/linux/types.h:5,
                 from /scratch/sfr/next/include/linux/smp.h:10,
                 from /scratch/sfr/next/include/linux/kernel_stat.h:4,
                 from /scratch/sfr/next/mm/memory.c:41:
In function '__read_once_size',
    inlined from 'handle_pte_fault' at /scratch/sfr/next/mm/memory.c:3192:10,
    inlined from '__handle_mm_fault' at /scratch/sfr/next/mm/memory.c:3324:2,
    inlined from 'handle_mm_fault' at /scratch/sfr/next/mm/memory.c:3353:6:
/scratch/sfr/next/include/linux/compiler.h:206:3: warning: call to 'data_access_exceeds_word_size' declared with attribute warning: data access exceeds word size and won't be atomic
   data_access_exceeds_word_size();
   ^
mm/built-in.o: In function `handle_mm_fault':
(.text+0x283a0): undefined reference to `data_access_exceeds_word_size'

I hate dropped the access_once tree again today.
-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: linux-next: build failure after merge of the access_once tree

[Christian Borntraeger]

Am 10.12.2014 um 09:20 schrieb Stephen Rothwell:
> Hi Christian,
> 
> After merging the access_once tree, today's linux-next build (powerpc
> ppc44x_defconfig) failed like this:
> 
> In file included from /scratch/sfr/next/include/uapi/linux/stddef.h:1:0,
>                  from /scratch/sfr/next/include/linux/stddef.h:4,
>                  from /scratch/sfr/next/include/uapi/linux/posix_types.h:4,
>                  from /scratch/sfr/next/include/uapi/linux/types.h:13,
>                  from /scratch/sfr/next/include/linux/types.h:5,
>                  from /scratch/sfr/next/include/linux/smp.h:10,
>                  from /scratch/sfr/next/include/linux/kernel_stat.h:4,
>                  from /scratch/sfr/next/mm/memory.c:41:
> In function '__read_once_size',
>     inlined from 'handle_pte_fault' at /scratch/sfr/next/mm/memory.c:3192:10,
>     inlined from '__handle_mm_fault' at /scratch/sfr/next/mm/memory.c:3324:2,
>     inlined from 'handle_mm_fault' at /scratch/sfr/next/mm/memory.c:3353:6:
> /scratch/sfr/next/include/linux/compiler.h:206:3: warning: call to 'data_access_exceeds_word_size' declared with attribute warning: data access exceeds word size and won't be atomic
>    data_access_exceeds_word_size();
>    ^
> mm/built-in.o: In function `handle_mm_fault':
> (.text+0x283a0): undefined reference to `data_access_exceeds_word_size'
> 
> I hate dropped the access_once tree again today.

hate dropped? typo or bad mood? ;-)

I fixed this two-ways 
1: a real bug: I did not provide data_access_exceeds_word_size
2: a change in semantics: change the memory.c rework to a barrier as if should be enough

Can you retry (I added ppc44x_defconfig to my cross compile test)?

Christian

Sorry for being such a pain, but the access_once rework will be prone to compile errors