From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 5F3C720D7649F for ; Wed, 12 Apr 2017 01:24:40 -0700 (PDT) Date: Wed, 12 Apr 2017 16:24:33 +0800 From: Dave Young Subject: Re: KASLR causes intermittent boot failures on some systems Message-ID: <20170412082433.GA3546@dhcp-128-65.nay.redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: Jeff Moyer Cc: bhe@redhat.com, linux-nvdimm@lists.01.org, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, mingo@kernel.org, thgarnie@google.com List-ID: On 04/07/17 at 10:41am, Jeff Moyer wrote: > Hi, > > commit 021182e52fe01 ("x86/mm: Enable KASLR for physical mapping memory > regions") causes some of my systems with persistent memory (whether real > or emulated) to fail to boot with a couple of different crash > signatures. The first signature is a NMI watchdog lockup of all but 1 > cpu, which causes much difficulty in extracting useful information from > the console. The second variant is an invalid paging request, listed > below. > > On some systems, I haven't hit this problem at all. Other systems > experience a failed boot maybe 20-30% of the time. To reproduce it, > configure some emulated pmem on your system. You can find directions > for that here: https://nvdimm.wiki.kernel.org/ > > Install ndctl (https://github.com/pmem/ndctl). > Configure the namespace: > # ndctl create-namespace -f -e namespace0.0 -m memory > > Then just reboot several times (5 should be enough), and hopefully > you'll hit the issue. > > I've attached both my .config and the dmesg output from a successful > boot at the end of this mail. > [snip] I did some tests about emulated pmem via memmap=, kdump kernel hangs or just reboots early during compressing kernel, no clue how to handle it. Since for kdump kernel kaslr is pointless a workaround is use "nokaslr" In Fedora or RHEL, just add "nokaslr" in KDUMP_COMMANDLINE_APPEND in /etc/sysconfig/kdump Can you try if this works? Thanks Dave _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753427AbdDLIYm (ORCPT ); Wed, 12 Apr 2017 04:24:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36544 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753209AbdDLIYk (ORCPT ); Wed, 12 Apr 2017 04:24:40 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B09F2C05974B Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dyoung@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com B09F2C05974B Date: Wed, 12 Apr 2017 16:24:33 +0800 From: Dave Young To: Jeff Moyer Cc: thgarnie@google.com, mingo@kernel.org, bhe@redhat.com, dan.j.williams@intel.com, linux-kernel@vger.kernel.org, linux-nvdimm@ml01.01.org, kexec@lists.infradead.org Subject: Re: KASLR causes intermittent boot failures on some systems Message-ID: <20170412082433.GA3546@dhcp-128-65.nay.redhat.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.1 (2016-10-04) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 12 Apr 2017 08:24:39 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 04/07/17 at 10:41am, Jeff Moyer wrote: > Hi, > > commit 021182e52fe01 ("x86/mm: Enable KASLR for physical mapping memory > regions") causes some of my systems with persistent memory (whether real > or emulated) to fail to boot with a couple of different crash > signatures. The first signature is a NMI watchdog lockup of all but 1 > cpu, which causes much difficulty in extracting useful information from > the console. The second variant is an invalid paging request, listed > below. > > On some systems, I haven't hit this problem at all. Other systems > experience a failed boot maybe 20-30% of the time. To reproduce it, > configure some emulated pmem on your system. You can find directions > for that here: https://nvdimm.wiki.kernel.org/ > > Install ndctl (https://github.com/pmem/ndctl). > Configure the namespace: > # ndctl create-namespace -f -e namespace0.0 -m memory > > Then just reboot several times (5 should be enough), and hopefully > you'll hit the issue. > > I've attached both my .config and the dmesg output from a successful > boot at the end of this mail. > [snip] I did some tests about emulated pmem via memmap=, kdump kernel hangs or just reboots early during compressing kernel, no clue how to handle it. Since for kdump kernel kaslr is pointless a workaround is use "nokaslr" In Fedora or RHEL, just add "nokaslr" in KDUMP_COMMANDLINE_APPEND in /etc/sysconfig/kdump Can you try if this works? Thanks Dave From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from mx1.redhat.com ([209.132.183.28]) by casper.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1cyDag-0000iL-CY for kexec@lists.infradead.org; Wed, 12 Apr 2017 08:25:28 +0000 Date: Wed, 12 Apr 2017 16:24:33 +0800 From: Dave Young Subject: Re: KASLR causes intermittent boot failures on some systems Message-ID: <20170412082433.GA3546@dhcp-128-65.nay.redhat.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "kexec" Errors-To: kexec-bounces+dwmw2=infradead.org@lists.infradead.org To: Jeff Moyer Cc: bhe@redhat.com, linux-nvdimm@lists.01.org, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, mingo@kernel.org, dan.j.williams@intel.com, thgarnie@google.com On 04/07/17 at 10:41am, Jeff Moyer wrote: > Hi, > > commit 021182e52fe01 ("x86/mm: Enable KASLR for physical mapping memory > regions") causes some of my systems with persistent memory (whether real > or emulated) to fail to boot with a couple of different crash > signatures. The first signature is a NMI watchdog lockup of all but 1 > cpu, which causes much difficulty in extracting useful information from > the console. The second variant is an invalid paging request, listed > below. > > On some systems, I haven't hit this problem at all. Other systems > experience a failed boot maybe 20-30% of the time. To reproduce it, > configure some emulated pmem on your system. You can find directions > for that here: https://nvdimm.wiki.kernel.org/ > > Install ndctl (https://github.com/pmem/ndctl). > Configure the namespace: > # ndctl create-namespace -f -e namespace0.0 -m memory > > Then just reboot several times (5 should be enough), and hopefully > you'll hit the issue. > > I've attached both my .config and the dmesg output from a successful > boot at the end of this mail. > [snip] I did some tests about emulated pmem via memmap=, kdump kernel hangs or just reboots early during compressing kernel, no clue how to handle it. Since for kdump kernel kaslr is pointless a workaround is use "nokaslr" In Fedora or RHEL, just add "nokaslr" in KDUMP_COMMANDLINE_APPEND in /etc/sysconfig/kdump Can you try if this works? Thanks Dave _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec