From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E853C433FE for ; Thu, 16 Sep 2021 09:37:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E8C13611CA for ; Thu, 16 Sep 2021 09:37:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235516AbhIPJjJ (ORCPT ); Thu, 16 Sep 2021 05:39:09 -0400 Received: from mail.kernel.org ([198.145.29.99]:50850 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235524AbhIPJjH (ORCPT ); Thu, 16 Sep 2021 05:39:07 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 334ED60F4A; Thu, 16 Sep 2021 09:37:43 +0000 (UTC) Date: Thu, 16 Sep 2021 10:37:40 +0100 From: Catalin Marinas To: Pavel Tatashin Cc: James Morris , Sasha Levin , "Eric W. Biederman" , kexec mailing list , LKML , Jonathan Corbet , Will Deacon , Linux ARM , Marc Zyngier , James Morse , Vladimir Murzin , Matthias Brugger , linux-mm , Mark Rutland , steve.capper@arm.com, rfontana@redhat.com, Thomas Gleixner , Selin Dag , Tyler Hicks , Pingfan Liu , Andrew Morton , madvenka@linux.microsoft.com Subject: Re: [PATCH v16 00/15] arm64: MMU enabled kexec relocation Message-ID: References: <20210802215408.804942-1-pasha.tatashin@soleen.com> <20210824180555.GD623@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Aug 26, 2021 at 11:03:21AM -0400, Pavel Tatashin wrote: > On Tue, Aug 24, 2021 at 2:06 PM Catalin Marinas wrote: > > > Enable MMU during kexec relocation in order to improve reboot performance. > > > > > > If kexec functionality is used for a fast system update, with a minimal > > > downtime, the relocation of kernel + initramfs takes a significant portion > > > of reboot. > > > > > > The reason for slow relocation is because it is done without MMU, and thus > > > not benefiting from D-Cache. > > > > The performance improvements are indeed significant on some platforms > > (going from 7s to ~40ms), so I think the merging the series is worth it. > > Some general questions so I better understand the impact: > > > > - Is the kdump path affected in any way? IIUC that doesn't need any > > relocation but we should also make sure we don't create the additional > > page table unnecessarily (should keep as much memory intact as > > possible). Maybe that's already handled. > > Because kdump does not need relocation, we do not reserve pages for > the page table in the kdump reboot case. In fact, with this series, > kdump reboot becomes more straightforward as we skip the relocation > function entirely, and jump directly into the crash kernel (or > purgatory if kexec tools loaded them). > > > - What happens if trans_pgd_create_copy() fails to allocate memory. Does > > it fall back to an MMU-off relocation? > > In case we are so low on memory that trans_pgd_create_copy() fails to > allocate the linear map that uses the large pages (the size of the > page table is tiny) the kexec fails during kexec load time (not during > reboot time), as out of memory. The MMU enabled kexec reboot is always > on, and we should not have several ways to do kexec reboot as it makes > the kexec reboot unpredictable in terms of performance, and also prone > to bugs by having a common MMU enabled path and less common path when > we are low on memory which is never tested. I think this makes sense, especially since it will fail during the kexec load time rather than reboot. I'm ok in principle with this series but I'd need to convince James Morse to have a another look since he followed it more closely than me. Could you please rebase it against 5.15-rc1? Thanks. -- Catalin From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1E25C433EF for ; Thu, 16 Sep 2021 09:39:57 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6D47B6112E for ; Thu, 16 Sep 2021 09:39:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 6D47B6112E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tFbocVAhK7WmKk6lLuu6eyTYdByBmnF09Vsj+K9tje8=; b=xCPI+FwGkWw664 yGXUqAEFkbFfHiUIVUAOSSnslxe/Xloz14xMDFky7Rtxn4SRa1X4pyAMXCFp+t8O7eM8dWx0LjsxD OKMrKUheD5yA/Ex1FRtTZd/AxlaV+bYeADQ2MFaa0KSI0EuODc9j+jMpxOiluBGfTEwHtRnKqRIw1 kwHuQU6VM7JWKNAUTBScVpjPo+XSh0k4B+8bjTYPDfj/J97iY5qwp/O+YHFJAFy8Ef99VSEQkjJTu eT3fWFioeKiftUpWh0fvfj/AI0RbU/sgd/JP5qezvdDlnJbUxphxlupccUkkvkbXvbbhcKNVAGSQs dYSERmZR5YHs1JCrGguQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mQnqE-00Ajg0-IA; Thu, 16 Sep 2021 09:38:02 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mQnpz-00AjaM-BI; Thu, 16 Sep 2021 09:37:48 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 334ED60F4A; Thu, 16 Sep 2021 09:37:43 +0000 (UTC) Date: Thu, 16 Sep 2021 10:37:40 +0100 From: Catalin Marinas To: Pavel Tatashin Cc: James Morris , Sasha Levin , "Eric W. Biederman" , kexec mailing list , LKML , Jonathan Corbet , Will Deacon , Linux ARM , Marc Zyngier , James Morse , Vladimir Murzin , Matthias Brugger , linux-mm , Mark Rutland , steve.capper@arm.com, rfontana@redhat.com, Thomas Gleixner , Selin Dag , Tyler Hicks , Pingfan Liu , Andrew Morton , madvenka@linux.microsoft.com Subject: Re: [PATCH v16 00/15] arm64: MMU enabled kexec relocation Message-ID: References: <20210802215408.804942-1-pasha.tatashin@soleen.com> <20210824180555.GD623@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_023747_456264_739C0532 X-CRM114-Status: GOOD ( 27.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Aug 26, 2021 at 11:03:21AM -0400, Pavel Tatashin wrote: > On Tue, Aug 24, 2021 at 2:06 PM Catalin Marinas wrote: > > > Enable MMU during kexec relocation in order to improve reboot performance. > > > > > > If kexec functionality is used for a fast system update, with a minimal > > > downtime, the relocation of kernel + initramfs takes a significant portion > > > of reboot. > > > > > > The reason for slow relocation is because it is done without MMU, and thus > > > not benefiting from D-Cache. > > > > The performance improvements are indeed significant on some platforms > > (going from 7s to ~40ms), so I think the merging the series is worth it. > > Some general questions so I better understand the impact: > > > > - Is the kdump path affected in any way? IIUC that doesn't need any > > relocation but we should also make sure we don't create the additional > > page table unnecessarily (should keep as much memory intact as > > possible). Maybe that's already handled. > > Because kdump does not need relocation, we do not reserve pages for > the page table in the kdump reboot case. In fact, with this series, > kdump reboot becomes more straightforward as we skip the relocation > function entirely, and jump directly into the crash kernel (or > purgatory if kexec tools loaded them). > > > - What happens if trans_pgd_create_copy() fails to allocate memory. Does > > it fall back to an MMU-off relocation? > > In case we are so low on memory that trans_pgd_create_copy() fails to > allocate the linear map that uses the large pages (the size of the > page table is tiny) the kexec fails during kexec load time (not during > reboot time), as out of memory. The MMU enabled kexec reboot is always > on, and we should not have several ways to do kexec reboot as it makes > the kexec reboot unpredictable in terms of performance, and also prone > to bugs by having a common MMU enabled path and less common path when > we are low on memory which is never tested. I think this makes sense, especially since it will fail during the kexec load time rather than reboot. I'm ok in principle with this series but I'd need to convince James Morse to have a another look since he followed it more closely than me. Could you please rebase it against 5.15-rc1? Thanks. -- Catalin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Date: Thu, 16 Sep 2021 10:37:40 +0100 From: Catalin Marinas Subject: Re: [PATCH v16 00/15] arm64: MMU enabled kexec relocation Message-ID: References: <20210802215408.804942-1-pasha.tatashin@soleen.com> <20210824180555.GD623@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "kexec" Errors-To: kexec-bounces+dwmw2=infradead.org@lists.infradead.org To: Pavel Tatashin Cc: James Morris , Sasha Levin , "Eric W. Biederman" , kexec mailing list , LKML , Jonathan Corbet , Will Deacon , Linux ARM , Marc Zyngier , James Morse , Vladimir Murzin , Matthias Brugger , linux-mm , Mark Rutland , steve.capper@arm.com, rfontana@redhat.com, Thomas Gleixner , Selin Dag , Tyler Hicks , Pingfan Liu , Andrew Morton , madvenka@linux.microsoft.com On Thu, Aug 26, 2021 at 11:03:21AM -0400, Pavel Tatashin wrote: > On Tue, Aug 24, 2021 at 2:06 PM Catalin Marinas wrote: > > > Enable MMU during kexec relocation in order to improve reboot performance. > > > > > > If kexec functionality is used for a fast system update, with a minimal > > > downtime, the relocation of kernel + initramfs takes a significant portion > > > of reboot. > > > > > > The reason for slow relocation is because it is done without MMU, and thus > > > not benefiting from D-Cache. > > > > The performance improvements are indeed significant on some platforms > > (going from 7s to ~40ms), so I think the merging the series is worth it. > > Some general questions so I better understand the impact: > > > > - Is the kdump path affected in any way? IIUC that doesn't need any > > relocation but we should also make sure we don't create the additional > > page table unnecessarily (should keep as much memory intact as > > possible). Maybe that's already handled. > > Because kdump does not need relocation, we do not reserve pages for > the page table in the kdump reboot case. In fact, with this series, > kdump reboot becomes more straightforward as we skip the relocation > function entirely, and jump directly into the crash kernel (or > purgatory if kexec tools loaded them). > > > - What happens if trans_pgd_create_copy() fails to allocate memory. Does > > it fall back to an MMU-off relocation? > > In case we are so low on memory that trans_pgd_create_copy() fails to > allocate the linear map that uses the large pages (the size of the > page table is tiny) the kexec fails during kexec load time (not during > reboot time), as out of memory. The MMU enabled kexec reboot is always > on, and we should not have several ways to do kexec reboot as it makes > the kexec reboot unpredictable in terms of performance, and also prone > to bugs by having a common MMU enabled path and less common path when > we are low on memory which is never tested. I think this makes sense, especially since it will fail during the kexec load time rather than reboot. I'm ok in principle with this series but I'd need to convince James Morse to have a another look since he followed it more closely than me. Could you please rebase it against 5.15-rc1? Thanks. -- Catalin _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec