Re: next/master boot bisection: next-20190215 on beaglebone-black

[Mathieu Desnoyers <mathieu.desnoyers@efficios.com>]

----- On Apr 16, 2019, at 3:25 PM, Mathieu Desnoyers mathieu.desnoyers@efficios.com wrote:

> ----- On Apr 16, 2019, at 3:17 PM, Mathieu Desnoyers
> mathieu.desnoyers@efficios.com wrote:
> 
>> ----- On Apr 16, 2019, at 2:54 PM, Dan Williams dan.j.williams@intel.com wrote:
>> 
>>> On Thu, Apr 11, 2019 at 1:54 PM Guenter Roeck <groeck@google.com> wrote:
>>> [..]
>>>> > > Boot tests report
>>>> > >
>>>> > > Qemu test results:
>>>> > >     total: 345 pass: 345 fail: 0
>>>> > >
>>>> > > This is on top of next-20190410 with CONFIG_SHUFFLE_PAGE_ALLOCATOR=y
>>>> > > and the known crashes fixed.
>>>> >
>>>> > In addition to CONFIG_SHUFFLE_PAGE_ALLOCATOR=y you also need the
>>>> > kernel command line option "page_alloc.shuffle=1"
>>>> >
>>>> > ...so I doubt you are running with shuffling enabled. Another way to
>>>> > double check is:
>>>> >
>>>> >    cat /sys/module/page_alloc/parameters/shuffle
>>>>
>>>> Yes, you are right. Because, with it enabled, I see:
>>>>
>>>> Kernel command line: rdinit=/sbin/init page_alloc.shuffle=1 panic=-1
>>>> console=ttyAMA0,115200 page_alloc.shuffle=1
>>>> ------------[ cut here ]------------
>>>> WARNING: CPU: 0 PID: 0 at ./include/linux/jump_label.h:303
>>>> page_alloc_shuffle+0x12c/0x1ac
>>>> static_key_enable(): static key 'page_alloc_shuffle_key+0x0/0x4' used
>>>> before call to jump_label_init()
>>> 
>>> This looks to be specific to ARM never having had to deal with
>>> DEFINE_STATIC_KEY_TRUE in the past.
>>> 
>>> I am able to avoid this warning by simply not enabling JUMP_LABEL
>>> support in my build.
>> 
>> How large is your kernel image in memory ? Is it larger than 32MB
>> by any chance ?
>> 
>> On arm, the arch_static_branch() uses a "nop" instruction, which seems
>> fine. However, I have a concern wrt arch_static_branch_jump():
>> 
>> arch/arm/include/asm/jump_label.h defines:
>> 
>> static __always_inline bool arch_static_branch_jump(struct static_key *key, bool
>> branch)
>> {
>>        asm_volatile_goto("1:\n\t"
>>                 WASM(b) " %l[l_yes]\n\t"
>>                 ".pushsection __jump_table,  \"aw\"\n\t"
>>                 ".word 1b, %l[l_yes], %c0\n\t"
>>                 ".popsection\n\t"
>>                 : :  "i" (&((char *)key)[branch]) :  : l_yes);
>> 
>>        return false;
>> l_yes:
>>        return true;
>> }
>> 
>> Which should work fine as long as the branch target is within +/-32MB range of
>> the branch instruction. However, based on
>> http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0489e/Cihfddaf.html
>> :
>> 
>> "Extending branch ranges
>> 
>> Machine-level B and BL instructions have restricted ranges from the address of
>> the current instruction. However, you can use these instructions even if label
>> is out of range. Often you do not know where the linker places label. When
>> necessary, the linker adds code to enable longer branches. The added code is
>> called a veneer."
>> 
>> So if by an odd chance this branch is turned into a longer branch by the linker,
>> then
>> the code pattern would be completely unexpected by arch/arm/kernel/jump_label.c.
>> 
>> Can you try with the following (untested) patch ?
> 

Updated logic of arch_static_branch_jump, and adding change that covers arch_static_branch()
as well (untested):

diff --git a/arch/arm/include/asm/jump_label.h b/arch/arm/include/asm/jump_label.h
index e12d7d096fc0..cec2f8a2b65e 100644
--- a/arch/arm/include/asm/jump_label.h
+++ b/arch/arm/include/asm/jump_label.h
@@ -9,12 +9,21 @@
 
 #define JUMP_LABEL_NOP_SIZE 4
 
+/*
+ * The linker adds veneer code if target of the branch is beyond +/-32MB
+ * range (+/-16MB for THUMB2), so ensure we never patch a branch
+ * instruction which target is outside of the inline asm.
+ */
 static __always_inline bool arch_static_branch(struct static_key *key, bool branch)
 {
        asm_volatile_goto("1:\n\t"
                 WASM(nop) "\n\t"
+                WASM(b) "2f\n\t"
+               "3:\n\t"
+                WASM(b) " %l[l_yes]\n\t"
+               "2:\n\t"
                 ".pushsection __jump_table,  \"aw\"\n\t"
-                ".word 1b, %l[l_yes], %c0\n\t"
+                ".word 1b, 3b, %c0\n\t"
                 ".popsection\n\t"
                 : :  "i" (&((char *)key)[branch]) :  : l_yes);
 
@@ -23,12 +32,21 @@ static __always_inline bool arch_static_branch(struct static_key *key, bool bran
        return true;
 }
 
+/*
+ * The linker adds veneer code if target of the branch is beyond +/-32MB
+ * range (+/-16MB for THUMB2), so ensure we never patch a branch
+ * instruction which target is outside of the inline asm.
+ */
 static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch)
 {
        asm_volatile_goto("1:\n\t"
+                WASM(b) "3f\n\t"
+                WASM(b) "2f\n\t"
+               "3:\n\t"
                 WASM(b) " %l[l_yes]\n\t"
+               "2:\n\t"
                 ".pushsection __jump_table,  \"aw\"\n\t"
-                ".word 1b, %l[l_yes], %c0\n\t"
+                ".word 1b, 3b, %c0\n\t"
                 ".popsection\n\t"
                 : :  "i" (&((char *)key)[branch]) :  : l_yes);


-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com