From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E7B7C433FF for ; Wed, 31 Jul 2019 17:04:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 200DD21851 for ; Wed, 31 Jul 2019 17:04:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="aOi9HiQR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728763AbfGaREX (ORCPT ); Wed, 31 Jul 2019 13:04:23 -0400 Received: from mail-ed1-f66.google.com ([209.85.208.66]:45615 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726514AbfGaREW (ORCPT ); Wed, 31 Jul 2019 13:04:22 -0400 Received: by mail-ed1-f66.google.com with SMTP id x19so60438860eda.12 for ; Wed, 31 Jul 2019 10:04:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=w3v6TLclWYCTjAr1Tti2BDVXfhzMXBqkZT8bGQWpyYs=; b=aOi9HiQRPHO4FOr23H0QJg5L4BRnfgz+Y7QvXWhB/ngiIQaEGHpBRMnl1OxzoXIBD2 k43GqRNVGXw98UkWEJOgJJ2uCTXIijl6XMElm2r6cO8ZxA6p+VeKe0q36fAAh9KHjGXh YSgHNRaRuPZ5vFNks3rQ7KnUpdHhj099KPD3D99ZDMOC90fiBIPstXZKeDLSEGq6tymT 9ON4RFhkvi6jSB8/Oj5Xi0cdIYetxeUWiifFJKWczZ8hgQpqdsJUnl9/Uwx6rJkeFEJj 58AJWNsYbe4cmO3e3vyV70t0uKsb/DXhqgr2SLW8GdW3LoqfK74WF1+J9TrqIxeltae+ yGYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=w3v6TLclWYCTjAr1Tti2BDVXfhzMXBqkZT8bGQWpyYs=; b=d9hGXP8oDm+VfpMR7hqwRIUqf01zEbjIoT1Ujtxhk0heh6ddU0WFA/kzUVi2Rf08hg uGNq8nxXkPA34nBv5ZOHPU32SNumFYV+XVFkaNuQnvemoO04ODAq1GpJ0B1tLGyx+r8A WTFKRqxTxCI0J8DNh3wHHtjX6Ytsx8XK7JPqEM4ojhibEBtExMpIYuq3Q8sgZlfkZRac Sf9rpp7KqRCwjOxcBWv6dwVS20HFlWs9T9Ewv8hVLQ1aeIWkAJUrdwl4Dqh1Qu20jPqG U/4bKgXGaxouAmlechcIE85a0UdLX/AiUuCcqOzK2MPH9Gg93cPX35UybtBJTPV/Mi6F ZJXQ== X-Gm-Message-State: APjAAAWRFS0DSvXP7bNZB6Bb8d/IHJuE5sK8SIHeUDpNiv6vExbtonji j+9FQeZ3ywJrQzmL9nhnqpIE1RwptgFbyVl7ouQ= X-Google-Smtp-Source: APXvYqyW5t23CPvCpiYNBbUwB050BijLjhKoZGvCCP6vJyE4voxLk8m00zN6oc8wm1lTBdkVRCt4NjssIfYO0PsdD3Y= X-Received: by 2002:a17:906:9447:: with SMTP id z7mr30023478ejx.165.1564592659700; Wed, 31 Jul 2019 10:04:19 -0700 (PDT) MIME-Version: 1.0 References: <20190731153857.4045-1-pasha.tatashin@soleen.com> <20190731163258.GH39768@lakrids.cambridge.arm.com> <20190731165007.GJ39768@lakrids.cambridge.arm.com> In-Reply-To: <20190731165007.GJ39768@lakrids.cambridge.arm.com> From: Pavel Tatashin Date: Wed, 31 Jul 2019 13:04:08 -0400 Message-ID: Subject: Re: [RFC v2 0/8] arm64: MMU enabled kexec relocation To: Mark Rutland Cc: James Morris , Sasha Levin , "Eric W. Biederman" , kexec mailing list , LKML , Jonathan Corbet , Catalin Marinas , will@kernel.org, Linux Doc Mailing List , Linux ARM , Marc Zyngier , James Morse , Vladimir Murzin , Matthias Brugger , Bhupesh Sharma Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jul 31, 2019 at 12:50 PM Mark Rutland wrote: > > On Wed, Jul 31, 2019 at 12:40:51PM -0400, Pavel Tatashin wrote: > > On Wed, Jul 31, 2019 at 12:33 PM Mark Rutland wrote: > > > > > > Hi Pavel, > > > > > > Generally, the cover letter should state up-front what the goal is (or > > > what problem you're trying to solve). It would be really helpful to have > > > that so that we understand what you're trying to achieve, and why. > > [...] > > > > > Here is the current data from the real hardware: > > > > (because of bug, I forced EL1 mode by setting el2_switch always to zero in > > > > cpu_soft_restart()): > > > > > > > > For this experiment, the size of kernel plus initramfs is 25M. If initramfs > > > > was larger, than the improvements would be even greater, as time spent in > > > > relocation is proportional to the size of relocation. > > > > > > > > Previously: > > > > kernel shutdown 0.022131328s > > > > relocation 0.440510736s > > > > kernel startup 0.294706768s > > > > > > In total this takes ~0.76s... > > > > > > > > > > > Relocation was taking: 58.2% of reboot time > > > > > > > > Now: > > > > kernel shutdown 0.032066576s > > > > relocation 0.022158152s > > > > kernel startup 0.296055880s > > > > > > ... and this takes ~0.35s > > > > > > So do we really need this complexity for a few blinks of an eye? > > > > Yes, we have an extremely tight reboot budget, 0.35s is not an acceptable waste. > > Could you please elaborate on your use-case? > > Understanfin what you're trying to achieve would help us to understand > which solutions make sense. An extremely high availability device with an update story utilizing kexec functionality for a faster kernel update and also for being able to preserve some state in memory without wasting the time of copying it to and from a backing storage. We at Microsoft will be using a fleet of these devices. The total reboot budget is less than half a second, out of which 0.44s is currently spent in kexec relocation. Pasha > > Thanks, > Mark. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32277C32751 for ; Wed, 31 Jul 2019 17:04:30 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E530421851 for ; Wed, 31 Jul 2019 17:04:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="FYbq/pfu"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="aOi9HiQR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E530421851 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=XC3aXENJFFdjeJvtFkdGNQDdiJaTO2qCcxCgclAvguc=; b=FYbq/pfuyagx1o xuWtpAs5Opynljg4cftmCoAyi66jPuyl+gBU1cEOZ+Z8hU5e3YCkI3w5MmNiGtQYjnmX91YfYOi7M nitZDe72H5v2kP1umwoqfd3lnsZNw6yM9VGdUdD2C+PQqraRXp1HtNvzRl6CTP+iPU1VtfnxRvSZH VpXJZhXwGqUa+AdU3FqHB4DCQytkOviWkckrEq3QCkntF7ZLjP9GnOWNnbGcDd8fnxD2i3TIsZ6mE Frw8z7MjufIqV6HSof/5scYZJwc3iAePBA6cpbM73k7KTusQ5KNOccM95auHwGQxhY5GpeP2qxM76 lw1M/AgC179ySVLWTC5g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hss1Y-00076d-1a; Wed, 31 Jul 2019 17:04:24 +0000 Received: from mail-ed1-x541.google.com ([2a00:1450:4864:20::541]) by bombadil.infradead.org with esmtps (Exim 4.92 #3 (Red Hat Linux)) id 1hss1V-00075r-A5 for linux-arm-kernel@lists.infradead.org; Wed, 31 Jul 2019 17:04:22 +0000 Received: by mail-ed1-x541.google.com with SMTP id p15so66321036eds.8 for ; Wed, 31 Jul 2019 10:04:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=w3v6TLclWYCTjAr1Tti2BDVXfhzMXBqkZT8bGQWpyYs=; b=aOi9HiQRPHO4FOr23H0QJg5L4BRnfgz+Y7QvXWhB/ngiIQaEGHpBRMnl1OxzoXIBD2 k43GqRNVGXw98UkWEJOgJJ2uCTXIijl6XMElm2r6cO8ZxA6p+VeKe0q36fAAh9KHjGXh YSgHNRaRuPZ5vFNks3rQ7KnUpdHhj099KPD3D99ZDMOC90fiBIPstXZKeDLSEGq6tymT 9ON4RFhkvi6jSB8/Oj5Xi0cdIYetxeUWiifFJKWczZ8hgQpqdsJUnl9/Uwx6rJkeFEJj 58AJWNsYbe4cmO3e3vyV70t0uKsb/DXhqgr2SLW8GdW3LoqfK74WF1+J9TrqIxeltae+ yGYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=w3v6TLclWYCTjAr1Tti2BDVXfhzMXBqkZT8bGQWpyYs=; b=Fel44AFw8RxEjEXnppxMvr80XgT6Diwk7rjntFgY5hinXmPdq3cJx1sO66fvBIDtT9 SRO1gzyigIsxuA9uh+K/LtU76CkQUjzuidOUwnKf6hfC+WlF+JK3kRgU9XMSfWlf5CkZ u00eRYsT38w/ssWP8r4MtRUL2p7WF8C/3YSpm99SWkwedRLnfSLXtOT1SYXb8bccjmBR XP7/IdP2ToWo2fZX2TqHKi7dFGVjBWQP+JhuQ4X5qw4JqzvcQBd82chl0RXCtzkVOVXt Z5ojbUvgghzp3eWiGLyZbk/k852lYqg9F8tqGZG9cPDfQxXkBjcQTS2enqcTgJqIHVMP mD7g== X-Gm-Message-State: APjAAAVLGPdZqCvXNiaaPVN/7Q+K7XB1it+EmnAbM0pdvUkV/20shxPH I+eg73ClPO+9RUw5IkAPDI6WSism8nnlv98rxysHK4MB X-Google-Smtp-Source: APXvYqyW5t23CPvCpiYNBbUwB050BijLjhKoZGvCCP6vJyE4voxLk8m00zN6oc8wm1lTBdkVRCt4NjssIfYO0PsdD3Y= X-Received: by 2002:a17:906:9447:: with SMTP id z7mr30023478ejx.165.1564592659700; Wed, 31 Jul 2019 10:04:19 -0700 (PDT) MIME-Version: 1.0 References: <20190731153857.4045-1-pasha.tatashin@soleen.com> <20190731163258.GH39768@lakrids.cambridge.arm.com> <20190731165007.GJ39768@lakrids.cambridge.arm.com> In-Reply-To: <20190731165007.GJ39768@lakrids.cambridge.arm.com> From: Pavel Tatashin Date: Wed, 31 Jul 2019 13:04:08 -0400 Message-ID: Subject: Re: [RFC v2 0/8] arm64: MMU enabled kexec relocation To: Mark Rutland X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190731_100421_347249_C20A0D57 X-CRM114-Status: GOOD ( 18.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Sasha Levin , Vladimir Murzin , Jonathan Corbet , Marc Zyngier , Catalin Marinas , Bhupesh Sharma , Linux Doc Mailing List , kexec mailing list , LKML , James Morris , James Morse , "Eric W. Biederman" , Matthias Brugger , will@kernel.org, Linux ARM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Jul 31, 2019 at 12:50 PM Mark Rutland wrote: > > On Wed, Jul 31, 2019 at 12:40:51PM -0400, Pavel Tatashin wrote: > > On Wed, Jul 31, 2019 at 12:33 PM Mark Rutland wrote: > > > > > > Hi Pavel, > > > > > > Generally, the cover letter should state up-front what the goal is (or > > > what problem you're trying to solve). It would be really helpful to have > > > that so that we understand what you're trying to achieve, and why. > > [...] > > > > > Here is the current data from the real hardware: > > > > (because of bug, I forced EL1 mode by setting el2_switch always to zero in > > > > cpu_soft_restart()): > > > > > > > > For this experiment, the size of kernel plus initramfs is 25M. If initramfs > > > > was larger, than the improvements would be even greater, as time spent in > > > > relocation is proportional to the size of relocation. > > > > > > > > Previously: > > > > kernel shutdown 0.022131328s > > > > relocation 0.440510736s > > > > kernel startup 0.294706768s > > > > > > In total this takes ~0.76s... > > > > > > > > > > > Relocation was taking: 58.2% of reboot time > > > > > > > > Now: > > > > kernel shutdown 0.032066576s > > > > relocation 0.022158152s > > > > kernel startup 0.296055880s > > > > > > ... and this takes ~0.35s > > > > > > So do we really need this complexity for a few blinks of an eye? > > > > Yes, we have an extremely tight reboot budget, 0.35s is not an acceptable waste. > > Could you please elaborate on your use-case? > > Understanfin what you're trying to achieve would help us to understand > which solutions make sense. An extremely high availability device with an update story utilizing kexec functionality for a faster kernel update and also for being able to preserve some state in memory without wasting the time of copying it to and from a backing storage. We at Microsoft will be using a fleet of these devices. The total reboot budget is less than half a second, out of which 0.44s is currently spent in kexec relocation. Pasha > > Thanks, > Mark. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from mail-ed1-x543.google.com ([2a00:1450:4864:20::543]) by bombadil.infradead.org with esmtps (Exim 4.92 #3 (Red Hat Linux)) id 1hss1V-00075s-9C for kexec@lists.infradead.org; Wed, 31 Jul 2019 17:04:22 +0000 Received: by mail-ed1-x543.google.com with SMTP id s49so31545305edb.1 for ; Wed, 31 Jul 2019 10:04:20 -0700 (PDT) MIME-Version: 1.0 References: <20190731153857.4045-1-pasha.tatashin@soleen.com> <20190731163258.GH39768@lakrids.cambridge.arm.com> <20190731165007.GJ39768@lakrids.cambridge.arm.com> In-Reply-To: <20190731165007.GJ39768@lakrids.cambridge.arm.com> From: Pavel Tatashin Date: Wed, 31 Jul 2019 13:04:08 -0400 Message-ID: Subject: Re: [RFC v2 0/8] arm64: MMU enabled kexec relocation List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "kexec" Errors-To: kexec-bounces+dwmw2=infradead.org@lists.infradead.org To: Mark Rutland Cc: Sasha Levin , Vladimir Murzin , Jonathan Corbet , Marc Zyngier , Catalin Marinas , Bhupesh Sharma , Linux Doc Mailing List , kexec mailing list , LKML , James Morris , James Morse , "Eric W. Biederman" , Matthias Brugger , will@kernel.org, Linux ARM On Wed, Jul 31, 2019 at 12:50 PM Mark Rutland wrote: > > On Wed, Jul 31, 2019 at 12:40:51PM -0400, Pavel Tatashin wrote: > > On Wed, Jul 31, 2019 at 12:33 PM Mark Rutland wrote: > > > > > > Hi Pavel, > > > > > > Generally, the cover letter should state up-front what the goal is (or > > > what problem you're trying to solve). It would be really helpful to have > > > that so that we understand what you're trying to achieve, and why. > > [...] > > > > > Here is the current data from the real hardware: > > > > (because of bug, I forced EL1 mode by setting el2_switch always to zero in > > > > cpu_soft_restart()): > > > > > > > > For this experiment, the size of kernel plus initramfs is 25M. If initramfs > > > > was larger, than the improvements would be even greater, as time spent in > > > > relocation is proportional to the size of relocation. > > > > > > > > Previously: > > > > kernel shutdown 0.022131328s > > > > relocation 0.440510736s > > > > kernel startup 0.294706768s > > > > > > In total this takes ~0.76s... > > > > > > > > > > > Relocation was taking: 58.2% of reboot time > > > > > > > > Now: > > > > kernel shutdown 0.032066576s > > > > relocation 0.022158152s > > > > kernel startup 0.296055880s > > > > > > ... and this takes ~0.35s > > > > > > So do we really need this complexity for a few blinks of an eye? > > > > Yes, we have an extremely tight reboot budget, 0.35s is not an acceptable waste. > > Could you please elaborate on your use-case? > > Understanfin what you're trying to achieve would help us to understand > which solutions make sense. An extremely high availability device with an update story utilizing kexec functionality for a faster kernel update and also for being able to preserve some state in memory without wasting the time of copying it to and from a backing storage. We at Microsoft will be using a fleet of these devices. The total reboot budget is less than half a second, out of which 0.44s is currently spent in kexec relocation. Pasha > > Thanks, > Mark. _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec