From: Thomas Garnier <thgarnie@google.com>
To: Andy Lutomirski <luto@kernel.org>
Cc: "Linus Torvalds" <torvalds@linux-foundation.org>,
"kernel test robot" <xiaolong.ye@intel.com>,
"Ingo Molnar" <mingo@kernel.org>,
"Alexander Potapenko" <glider@google.com>,
"Andrew Morton" <akpm@linux-foundation.org>,
"Andrey Ryabinin" <aryabinin@virtuozzo.com>,
"Ard Biesheuvel" <ard.biesheuvel@linaro.org>,
"Boris Ostrovsky" <boris.ostrovsky@oracle.com>,
"Borislav Petkov" <bp@suse.de>,
"Chris Wilson" <chris@chris-wilson.co.uk>,
"Christian Borntraeger" <borntraeger@de.ibm.com>,
"Dmitry Vyukov" <dvyukov@google.com>,
"Frederic Weisbecker" <fweisbec@gmail.com>,
"Jiri Kosina" <jikos@kernel.org>,
"Joerg Roedel" <joro@8bytes.org>,
"Jonathan Corbet" <corbet@lwn.net>,
"Josh Poimboeuf" <jpoimboe@redhat.com>,
"Juergen Gross" <jgross@suse.com>,
"Kees Cook" <keescook@chromium.org>,
"Len Brown" <len.brown@intel.com>,
"Lorenzo Stoakes" <lstoakes@gmail.com>,
"Luis R . Rodriguez" <mcgrof@kernel.org>,
"Matt Fleming" <matt@codeblueprint.co.uk>,
"Michal Hocko" <mhocko@suse.com>,
"Paolo Bonzini" <pbonzini@redhat.com>,
"Paul Gortmaker" <paul.gortmaker@windriver.com>,
"Pavel Machek" <pavel@ucw.cz>,
"Peter Zijlstra" <peterz@infradead.org>,
"Radim Krčmář" <rkrcmar@redhat.com>,
"Rafael J . Wysocki" <rjw@rjwysocki.net>,
"Rusty Russell" <rusty@rustcorp.com.au>,
"Stanislaw Gruszka" <sgruszka@redhat.com>,
"Thomas Gleixner" <tglx@linutronix.de>,
"Tim Chen" <tim.c.chen@linux.intel.com>,
"Vitaly Kuznetsov" <vkuznets@redhat.com>,
zijun_hu <zijun_hu@htc.com>, LKML <linux-kernel@vger.kernel.org>,
"Stephen Rothwell" <sfr@canb.auug.org.au>, LKP <lkp@01.org>
Subject: Re: [lkp-robot] [x86] 69218e4799: BUG:kernel_hang_in_boot_stage
Date: Tue, 21 Mar 2017 22:16:12 -0700 [thread overview]
Message-ID: <CAJcbSZEvR5GmxGt1K6JrA9KsDgv2TTbntwWFPZ+B01EhO9DSGQ@mail.gmail.com> (raw)
In-Reply-To: <CALCETrXOpdmEa+S8a6GimZ5D0-DeUih5tAZh97VY2SZq7GGKuA@mail.gmail.com>
On Tue, Mar 21, 2017 at 9:27 PM, Andy Lutomirski <luto@kernel.org> wrote:
> On Tue, Mar 21, 2017 at 5:41 PM, Thomas Garnier <thgarnie@google.com> wrote:
>> On Tue, Mar 21, 2017 at 4:51 PM, Andy Lutomirski <luto@kernel.org> wrote:
>>> On Tue, Mar 21, 2017 at 3:32 PM, Andy Lutomirski <luto@amacapital.net> wrote:
>>>> On Tue, Mar 21, 2017 at 2:11 PM, Linus Torvalds
>>>> <torvalds@linux-foundation.org> wrote:
>>>>> On Tue, Mar 21, 2017 at 1:25 PM, Thomas Garnier <thgarnie@google.com> wrote:
>>>>>> The issue seems to be related to exceptions happening in close pages
>>>>>> to the fixmap GDT remapping.
>>>>>>
>>>>>> The original page fault happen in do_test_wp_bit which set a fixmap
>>>>>> entry to test WP flag. If I grow the number of processors supported
>>>>>> increasing the distance between the remapped GDT page and the WP test
>>>>>> page, the error does not reproduce.
>>>>>>
>>>>>> I am still looking at the exact distance between repro and no-repro as
>>>>>> well as the exact root cause.
>>>>>
>>>>> Hmm. Have we set the GDT limit incorrectly, somehow? The GDT *can*
>>>>> cover 8k entries, which at 8 bytes each would be 64kB.
>>>>
>>>> The QEMU barf says the GDT limit is 0xff, for better or for worse.
>>>>
>>>>>
>>>>> So somebody trying to load an invalid segment (say, 0xffff) might end
>>>>> up causing an access to the GDT base + 64k - 8.
>>>>>
>>>>> It is also possible that the CPU might do a page table writability
>>>>> check *before* it does the limit check. That would sound odd, though.
>>>>> Might be a CPU errata.
>>>>>
>>>>
>>>
>>>> There's presumably something genuinely wrong with our GDT.
>>>
>>> This is suspicious. I added this code in test_wp_bit:
>>>
>>> if (memcmp(get_current_gdt_ro(), get_current_gdt_rw(), 4096) != 0) {
>>> pr_err("Oh crap\n");
>>> BUG_ON(1);
>>> }
>>>
>>> It printed "Oh crap" and blew up. Methinks something's wrong with the
>>> fixmap. Is it possible that we're crossing a PMD boundary and failing
>>> to translate the addresses right?
>>
>> I might be that. We crash when the PKMAP_BASE is just after the FIX_WP_TEST.
>>
>> I will continue testing couple scenarios and design a fix. Moving the
>> GDT FIXMAP at the beginning or align the base (or pad the end).
>>
>
> Talk about barking up the wrong tree...
>
> diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
> index f8e22dbad86c..c564f62c7a8d 100644
> --- a/arch/x86/kernel/cpu/common.c
> +++ b/arch/x86/kernel/cpu/common.c
> @@ -462,7 +464,8 @@ pgprot_t pg_fixmap_gdt_flags = PAGE_KERNEL;
> static inline void setup_fixmap_gdt(int cpu)
> {
> __set_fixmap(get_cpu_gdt_ro_index(cpu),
> - __pa(get_cpu_gdt_rw(cpu)), pg_fixmap_gdt_flags);
> + slow_virt_to_phys(get_cpu_gdt_rw(cpu)),
> + pg_fixmap_gdt_flags);
> }
>
> /* Load the original GDT from the per-cpu structure */
>
> This makes UP boot for me, but SMP (2 cpus) is still busted.
This change fixed boot for me:
diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
index b65155cc3760..4e30707d9f9a 100644
--- a/arch/x86/include/asm/fixmap.h
+++ b/arch/x86/include/asm/fixmap.h
@@ -104,7 +104,9 @@ enum fixed_addresses {
FIX_GDT_REMAP_BEGIN,
FIX_GDT_REMAP_END = FIX_GDT_REMAP_BEGIN + NR_CPUS - 1,
- __end_of_permanent_fixed_addresses,
+ __end_of_permanent_fixed_addresses =
+ (FIX_GDT_REMAP_END + PTRS_PER_PTE - 1) &
+ -PTRS_PER_PTE,
Just ensure PKMAP_BASE & FIX_WP_TEST are on a different PMD.
I don't think that the right fix but it might help understand the
exact root cause.
--
Thomas
next prev parent reply other threads:[~2017-03-22 5:16 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-21 4:57 [lkp-robot] [x86] 69218e4799: BUG:kernel_hang_in_boot_stage kernel test robot
2017-03-21 18:16 ` Thomas Garnier
2017-03-21 19:20 ` Linus Torvalds
2017-03-21 19:23 ` Thomas Garnier
2017-03-21 20:25 ` Thomas Garnier
2017-03-21 21:11 ` Linus Torvalds
2017-03-21 22:32 ` Andy Lutomirski
2017-03-21 23:51 ` Andy Lutomirski
2017-03-22 0:41 ` Thomas Garnier
2017-03-22 4:27 ` Andy Lutomirski
2017-03-22 5:16 ` Thomas Garnier [this message]
2017-03-22 7:36 ` Ingo Molnar
2017-03-22 16:33 ` Andy Lutomirski
2017-03-22 16:38 ` Thomas Garnier
2017-03-22 16:59 ` Andy Lutomirski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAJcbSZEvR5GmxGt1K6JrA9KsDgv2TTbntwWFPZ+B01EhO9DSGQ@mail.gmail.com \
--to=thgarnie@google.com \
--cc=akpm@linux-foundation.org \
--cc=ard.biesheuvel@linaro.org \
--cc=aryabinin@virtuozzo.com \
--cc=boris.ostrovsky@oracle.com \
--cc=borntraeger@de.ibm.com \
--cc=bp@suse.de \
--cc=chris@chris-wilson.co.uk \
--cc=corbet@lwn.net \
--cc=dvyukov@google.com \
--cc=fweisbec@gmail.com \
--cc=glider@google.com \
--cc=jgross@suse.com \
--cc=jikos@kernel.org \
--cc=joro@8bytes.org \
--cc=jpoimboe@redhat.com \
--cc=keescook@chromium.org \
--cc=len.brown@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lkp@01.org \
--cc=lstoakes@gmail.com \
--cc=luto@kernel.org \
--cc=matt@codeblueprint.co.uk \
--cc=mcgrof@kernel.org \
--cc=mhocko@suse.com \
--cc=mingo@kernel.org \
--cc=paul.gortmaker@windriver.com \
--cc=pavel@ucw.cz \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=rjw@rjwysocki.net \
--cc=rkrcmar@redhat.com \
--cc=rusty@rustcorp.com.au \
--cc=sfr@canb.auug.org.au \
--cc=sgruszka@redhat.com \
--cc=tglx@linutronix.de \
--cc=tim.c.chen@linux.intel.com \
--cc=torvalds@linux-foundation.org \
--cc=vkuznets@redhat.com \
--cc=xiaolong.ye@intel.com \
--cc=zijun_hu@htc.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).