From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S964866AbdCVQen (ORCPT ); Wed, 22 Mar 2017 12:34:43 -0400 Received: from mail-vk0-f44.google.com ([209.85.213.44]:32944 "EHLO mail-vk0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758917AbdCVQeR (ORCPT ); Wed, 22 Mar 2017 12:34:17 -0400 MIME-Version: 1.0 In-Reply-To: <20170322073608.GA5538@gmail.com> References: <20170322073608.GA5538@gmail.com> From: Andy Lutomirski Date: Wed, 22 Mar 2017 09:33:17 -0700 Message-ID: Subject: Re: [lkp-robot] [x86] 69218e4799: BUG:kernel_hang_in_boot_stage To: Ingo Molnar Cc: Thomas Garnier , Andy Lutomirski , Linus Torvalds , kernel test robot , Alexander Potapenko , Andrew Morton , Andrey Ryabinin , Ard Biesheuvel , Boris Ostrovsky , Borislav Petkov , Chris Wilson , Christian Borntraeger , Dmitry Vyukov , Frederic Weisbecker , Jiri Kosina , Joerg Roedel , Jonathan Corbet , Josh Poimboeuf , Juergen Gross , Kees Cook , Len Brown , Lorenzo Stoakes , "Luis R . Rodriguez" , Matt Fleming , Michal Hocko , Paolo Bonzini , Paul Gortmaker , Pavel Machek , Peter Zijlstra , =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , "Rafael J . Wysocki" , Rusty Russell , Stanislaw Gruszka , Thomas Gleixner , Tim Chen , Vitaly Kuznetsov , zijun_hu , LKML , Stephen Rothwell , LKP Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Mar 22, 2017 at 12:36 AM, Ingo Molnar wrote: > > * Thomas Garnier wrote: > >> > static inline void setup_fixmap_gdt(int cpu) >> > { >> > __set_fixmap(get_cpu_gdt_ro_index(cpu), >> > - __pa(get_cpu_gdt_rw(cpu)), pg_fixmap_gdt_flags); >> > + slow_virt_to_phys(get_cpu_gdt_rw(cpu)), >> > + pg_fixmap_gdt_flags); >> > } >> > >> > /* Load the original GDT from the per-cpu structure */ >> > >> > This makes UP boot for me, but SMP (2 cpus) is still busted. >> >> This change fixed boot for me: >> >> diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h >> index b65155cc3760..4e30707d9f9a 100644 >> --- a/arch/x86/include/asm/fixmap.h >> +++ b/arch/x86/include/asm/fixmap.h >> @@ -104,7 +104,9 @@ enum fixed_addresses { >> FIX_GDT_REMAP_BEGIN, >> FIX_GDT_REMAP_END = FIX_GDT_REMAP_BEGIN + NR_CPUS - 1, >> >> - __end_of_permanent_fixed_addresses, >> + __end_of_permanent_fixed_addresses = >> + (FIX_GDT_REMAP_END + PTRS_PER_PTE - 1) & >> + -PTRS_PER_PTE, >> >> Just ensure PKMAP_BASE & FIX_WP_TEST are on a different PMD. >> >> I don't think that the right fix but it might help understand the >> exact root cause. > > Could this be related to the permission bits in the PMD itself getting out of sync > with the PTEs? WP test marks a page writable/unwritable, and maybe we mess up the > restoration. If they are on separate PMDs then this is worked around because the > fixmap GDT is on a separate PMD. > I don't think so. I think it's a pair of bugs related to the way that percpu areas are virtually mapped. Bug 1: __pa is totally bogus on percpu pointers. Oddly, we have one older instance of exactly the same bug (on the same GDT address) in the kernel. I'll send a patch. Bug 2: Nothing syncs a freshly-set-up CPU's percpu area into initial_page_table. This makes access to the gdt fail in startup_32_smp. This looks like a longstanding bug, and I don't see what it has to do with Thomas' series. I'm still mulling over what to do about it. --Andy From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============4708482707602096270==" MIME-Version: 1.0 From: Andy Lutomirski To: lkp@lists.01.org Subject: Re: [lkp-robot] [x86] 69218e4799: BUG:kernel_hang_in_boot_stage Date: Wed, 22 Mar 2017 09:33:17 -0700 Message-ID: In-Reply-To: <20170322073608.GA5538@gmail.com> List-Id: --===============4708482707602096270== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Wed, Mar 22, 2017 at 12:36 AM, Ingo Molnar wrote: > > * Thomas Garnier wrote: > >> > static inline void setup_fixmap_gdt(int cpu) >> > { >> > __set_fixmap(get_cpu_gdt_ro_index(cpu), >> > - __pa(get_cpu_gdt_rw(cpu)), pg_fixmap_gdt_flags); >> > + slow_virt_to_phys(get_cpu_gdt_rw(cpu)), >> > + pg_fixmap_gdt_flags); >> > } >> > >> > /* Load the original GDT from the per-cpu structure */ >> > >> > This makes UP boot for me, but SMP (2 cpus) is still busted. >> >> This change fixed boot for me: >> >> diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap= .h >> index b65155cc3760..4e30707d9f9a 100644 >> --- a/arch/x86/include/asm/fixmap.h >> +++ b/arch/x86/include/asm/fixmap.h >> @@ -104,7 +104,9 @@ enum fixed_addresses { >> FIX_GDT_REMAP_BEGIN, >> FIX_GDT_REMAP_END =3D FIX_GDT_REMAP_BEGIN + NR_CPUS - 1, >> >> - __end_of_permanent_fixed_addresses, >> + __end_of_permanent_fixed_addresses =3D >> + (FIX_GDT_REMAP_END + PTRS_PER_PTE - 1) & >> + -PTRS_PER_PTE, >> >> Just ensure PKMAP_BASE & FIX_WP_TEST are on a different PMD. >> >> I don't think that the right fix but it might help understand the >> exact root cause. > > Could this be related to the permission bits in the PMD itself getting ou= t of sync > with the PTEs? WP test marks a page writable/unwritable, and maybe we mes= s up the > restoration. If they are on separate PMDs then this is worked around beca= use the > fixmap GDT is on a separate PMD. > I don't think so. I think it's a pair of bugs related to the way that percpu areas are virtually mapped. Bug 1: __pa is totally bogus on percpu pointers. Oddly, we have one older instance of exactly the same bug (on the same GDT address) in the kernel. I'll send a patch. Bug 2: Nothing syncs a freshly-set-up CPU's percpu area into initial_page_table. This makes access to the gdt fail in startup_32_smp. This looks like a longstanding bug, and I don't see what it has to do with Thomas' series. I'm still mulling over what to do about it. --Andy --===============4708482707602096270==--