From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965758AbeEIVwY (ORCPT ); Wed, 9 May 2018 17:52:24 -0400 Received: from foss.arm.com ([217.140.101.70]:50184 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S965697AbeEIVwW (ORCPT ); Wed, 9 May 2018 17:52:22 -0400 Subject: Re: [PATCH v2 2/6] arm64: alternative: Apply alternatives early in boot process To: Julien Thierry , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: mark.rutland@arm.com, marc.zyngier@arm.com, james.morse@arm.com, daniel.thompson@linaro.org, Catalin Marinas , Will Deacon References: <1516190084-18978-1-git-send-email-julien.thierry@arm.com> <1516190084-18978-3-git-send-email-julien.thierry@arm.com> From: Suzuki K Poulose Message-ID: <7a3afa42-821a-3d4a-3af8-00ba18653a4a@arm.com> Date: Wed, 9 May 2018 22:52:21 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/04/2018 11:06 AM, Julien Thierry wrote: > Hi, > > In order to prepare the v3 of this patchset, I'd like people's opinion > on what this patch does. More below. > > On 17/01/18 11:54, Julien Thierry wrote: >> From: Daniel Thompson >> >> Currently alternatives are applied very late in the boot process (and >> a long time after we enable scheduling). Some alternative sequences, >> such as those that alter the way CPU context is stored, must be applied >> much earlier in the boot sequence. >> +/* >> + * early-apply features are detected using only the boot CPU and >> checked on >> + * secondary CPUs startup, even then, >> + * These early-apply features should only include features where we must >> + * patch the kernel very early in the boot process. >> + * >> + * Note that the cpufeature logic *must* be made aware of early-apply >> + * features to ensure they are reported as enabled without waiting >> + * for other CPUs to boot. >> + */ >> +#define EARLY_APPLY_FEATURE_MASK BIT(ARM64_HAS_SYSREG_GIC_CPUIF) >> + > > Following the change in the cpufeature infrastructure, > ARM64_HAS_SYSREG_GIC_CPUIF will have the scope > ARM64_CPUCAP_SCOPE_BOOT_CPU in order to be checked early in the boot > process. Thats correct. > > Now, regarding the early application of alternative, I am wondering > whether we can apply all the alternatives associated with SCOPE_BOOT > features that *do not* have a cpu_enable callback. > I don't understand why would you skip the ones that have a "cpu_enable" callback. Could you explain this a bit ? Ideally you should be able to apply the alternatives for features with the SCOPE_BOOT, provided the cpu_enable() callback is written properly. > Otherwise we can keep the macro to list individually each feature that > is patchable at boot time as the current patch does (or put this info in > a flag within the arm64_cpu_capabilities structure) You may be able to build up the mask of *available* capabilities with SCOPE_BOOT at boot time by playing some trick in the setup_boot_cpu_capabilities(), rather than embedding it in the capabilities (and then parsing the entire table(s)) or manually keeping track of the capabilities by having a separate mask. Suzuki > > Any thoughts or preferences on this? > > Thanks, > >>   #define __ALT_PTR(a,f)        ((void *)&(a)->f + (a)->f) >>   #define ALT_ORIG_PTR(a)        __ALT_PTR(a, orig_offset) >>   #define ALT_REPL_PTR(a)        __ALT_PTR(a, alt_offset) >> @@ -105,7 +117,8 @@ static u32 get_alt_insn(struct alt_instr *alt, >> __le32 *insnptr, __le32 *altinsnp >>       return insn; >>   } >> >> -static void __apply_alternatives(void *alt_region, bool >> use_linear_alias) >> +static void __apply_alternatives(void *alt_region,  bool >> use_linear_alias, >> +                 unsigned long feature_mask) >>   { >>       struct alt_instr *alt; >>       struct alt_region *region = alt_region; >> @@ -115,6 +128,9 @@ static void __apply_alternatives(void *alt_region, >> bool use_linear_alias) >>           u32 insn; >>           int i, nr_inst; >> >> +        if ((BIT(alt->cpufeature) & feature_mask) == 0) >> +            continue; >> + >>           if (!cpus_have_cap(alt->cpufeature)) >>               continue; >> >> @@ -138,6 +154,21 @@ static void __apply_alternatives(void >> *alt_region, bool use_linear_alias) >>   } >> >>   /* >> + * This is called very early in the boot process (directly after we run >> + * a feature detect on the boot CPU). No need to worry about other CPUs >> + * here. >> + */ >> +void apply_alternatives_early(void) >> +{ >> +    struct alt_region region = { >> +        .begin    = (struct alt_instr *)__alt_instructions, >> +        .end    = (struct alt_instr *)__alt_instructions_end, >> +    }; >> + >> +    __apply_alternatives(®ion, true, EARLY_APPLY_FEATURE_MASK); >> +} >> + >> +/* >>    * We might be patching the stop_machine state machine, so implement a >>    * really simple polling protocol here. >>    */ >> @@ -156,7 +187,9 @@ static int __apply_alternatives_multi_stop(void >> *unused) >>           isb(); >>       } else { >>           BUG_ON(patched); >> -        __apply_alternatives(®ion, true); >> + >> +        __apply_alternatives(®ion, true, ~EARLY_APPLY_FEATURE_MASK); >> + >>           /* Barriers provided by the cache flushing */ >>           WRITE_ONCE(patched, 1); >>       } >> @@ -177,5 +210,5 @@ void apply_alternatives(void *start, size_t length) >>           .end    = start + length, >>       }; >> >> -    __apply_alternatives(®ion, false); >> +    __apply_alternatives(®ion, false, -1); >>   } >> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c >> index 551eb07..37361b5 100644 >> --- a/arch/arm64/kernel/smp.c >> +++ b/arch/arm64/kernel/smp.c >> @@ -453,6 +453,12 @@ void __init smp_prepare_boot_cpu(void) >>        * cpuinfo_store_boot_cpu() above. >>        */ >>       update_cpu_errata_workarounds(); >> +    /* >> +     * We now know enough about the boot CPU to apply the >> +     * alternatives that cannot wait until interrupt handling >> +     * and/or scheduling is enabled. >> +     */ >> +    apply_alternatives_early(); >>   } >> >>   static u64 __init of_get_cpu_mpidr(struct device_node *dn) >> -- >> 1.9.1 >> > From mboxrd@z Thu Jan 1 00:00:00 1970 From: suzuki.poulose@arm.com (Suzuki K Poulose) Date: Wed, 9 May 2018 22:52:21 +0100 Subject: [PATCH v2 2/6] arm64: alternative: Apply alternatives early in boot process In-Reply-To: References: <1516190084-18978-1-git-send-email-julien.thierry@arm.com> <1516190084-18978-3-git-send-email-julien.thierry@arm.com> Message-ID: <7a3afa42-821a-3d4a-3af8-00ba18653a4a@arm.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On 05/04/2018 11:06 AM, Julien Thierry wrote: > Hi, > > In order to prepare the v3 of this patchset, I'd like people's opinion > on what this patch does. More below. > > On 17/01/18 11:54, Julien Thierry wrote: >> From: Daniel Thompson >> >> Currently alternatives are applied very late in the boot process (and >> a long time after we enable scheduling). Some alternative sequences, >> such as those that alter the way CPU context is stored, must be applied >> much earlier in the boot sequence. >> +/* >> + * early-apply features are detected using only the boot CPU and >> checked on >> + * secondary CPUs startup, even then, >> + * These early-apply features should only include features where we must >> + * patch the kernel very early in the boot process. >> + * >> + * Note that the cpufeature logic *must* be made aware of early-apply >> + * features to ensure they are reported as enabled without waiting >> + * for other CPUs to boot. >> + */ >> +#define EARLY_APPLY_FEATURE_MASK BIT(ARM64_HAS_SYSREG_GIC_CPUIF) >> + > > Following the change in the cpufeature infrastructure, > ARM64_HAS_SYSREG_GIC_CPUIF will have the scope > ARM64_CPUCAP_SCOPE_BOOT_CPU in order to be checked early in the boot > process. Thats correct. > > Now, regarding the early application of alternative, I am wondering > whether we can apply all the alternatives associated with SCOPE_BOOT > features that *do not* have a cpu_enable callback. > I don't understand why would you skip the ones that have a "cpu_enable" callback. Could you explain this a bit ? Ideally you should be able to apply the alternatives for features with the SCOPE_BOOT, provided the cpu_enable() callback is written properly. > Otherwise we can keep the macro to list individually each feature that > is patchable at boot time as the current patch does (or put this info in > a flag within the arm64_cpu_capabilities structure) You may be able to build up the mask of *available* capabilities with SCOPE_BOOT at boot time by playing some trick in the setup_boot_cpu_capabilities(), rather than embedding it in the capabilities (and then parsing the entire table(s)) or manually keeping track of the capabilities by having a separate mask. Suzuki > > Any thoughts or preferences on this? > > Thanks, > >> ? #define __ALT_PTR(a,f)??????? ((void *)&(a)->f + (a)->f) >> ? #define ALT_ORIG_PTR(a)??????? __ALT_PTR(a, orig_offset) >> ? #define ALT_REPL_PTR(a)??????? __ALT_PTR(a, alt_offset) >> @@ -105,7 +117,8 @@ static u32 get_alt_insn(struct alt_instr *alt, >> __le32 *insnptr, __le32 *altinsnp >> ????? return insn; >> ? } >> >> -static void __apply_alternatives(void *alt_region, bool >> use_linear_alias) >> +static void __apply_alternatives(void *alt_region,? bool >> use_linear_alias, >> +???????????????? unsigned long feature_mask) >> ? { >> ????? struct alt_instr *alt; >> ????? struct alt_region *region = alt_region; >> @@ -115,6 +128,9 @@ static void __apply_alternatives(void *alt_region, >> bool use_linear_alias) >> ????????? u32 insn; >> ????????? int i, nr_inst; >> >> +??????? if ((BIT(alt->cpufeature) & feature_mask) == 0) >> +??????????? continue; >> + >> ????????? if (!cpus_have_cap(alt->cpufeature)) >> ????????????? continue; >> >> @@ -138,6 +154,21 @@ static void __apply_alternatives(void >> *alt_region, bool use_linear_alias) >> ? } >> >> ? /* >> + * This is called very early in the boot process (directly after we run >> + * a feature detect on the boot CPU). No need to worry about other CPUs >> + * here. >> + */ >> +void apply_alternatives_early(void) >> +{ >> +??? struct alt_region region = { >> +??????? .begin??? = (struct alt_instr *)__alt_instructions, >> +??????? .end??? = (struct alt_instr *)__alt_instructions_end, >> +??? }; >> + >> +??? __apply_alternatives(®ion, true, EARLY_APPLY_FEATURE_MASK); >> +} >> + >> +/* >> ?? * We might be patching the stop_machine state machine, so implement a >> ?? * really simple polling protocol here. >> ?? */ >> @@ -156,7 +187,9 @@ static int __apply_alternatives_multi_stop(void >> *unused) >> ????????? isb(); >> ????? } else { >> ????????? BUG_ON(patched); >> -??????? __apply_alternatives(®ion, true); >> + >> +??????? __apply_alternatives(®ion, true, ~EARLY_APPLY_FEATURE_MASK); >> + >> ????????? /* Barriers provided by the cache flushing */ >> ????????? WRITE_ONCE(patched, 1); >> ????? } >> @@ -177,5 +210,5 @@ void apply_alternatives(void *start, size_t length) >> ????????? .end??? = start + length, >> ????? }; >> >> -??? __apply_alternatives(®ion, false); >> +??? __apply_alternatives(®ion, false, -1); >> ? } >> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c >> index 551eb07..37361b5 100644 >> --- a/arch/arm64/kernel/smp.c >> +++ b/arch/arm64/kernel/smp.c >> @@ -453,6 +453,12 @@ void __init smp_prepare_boot_cpu(void) >> ?????? * cpuinfo_store_boot_cpu() above. >> ?????? */ >> ????? update_cpu_errata_workarounds(); >> +??? /* >> +???? * We now know enough about the boot CPU to apply the >> +???? * alternatives that cannot wait until interrupt handling >> +???? * and/or scheduling is enabled. >> +???? */ >> +??? apply_alternatives_early(); >> ? } >> >> ? static u64 __init of_get_cpu_mpidr(struct device_node *dn) >> -- >> 1.9.1 >> >