From: Marc Zyngier <marc.zyngier@arm.com> To: David Abdurachmanov <david.abdurachmanov@gmail.com> Cc: "Atish Patra" <atish.patra@wdc.com>, "Christoph Hellwig" <hch@infradead.org>, "Damien Le Moal" <Damien.LeMoal@wdc.com>, "Albert Ou" <aou@eecs.berkeley.edu>, "Jason Cooper" <jason@lakedaemon.net>, "Alan Kao" <alankao@andestech.com>, "Dmitriy Cherkasov" <dmitriy@oss-tech.org>, "Anup Patel" <anup@brainfault.org>, "Daniel Lezcano" <daniel.lezcano@linaro.org>, "Patrick Stählin" <me@packi.ch>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "Palmer Dabbelt" <palmer@sifive.com>, "Paul Walmsley" <paul.walmsley@sifive.com>, "Andreas Schwab" <schwab@suse.de>, "linux-riscv@lists.infradead.org" <linux-riscv@lists.infradead.org>, "Thomas Gleixner" <tglx@linutronix.de>, "Zong Li" <zongbox@gmail.com> Subject: Re: [v3 PATCH 8/8] RISC-V: Assign hwcap only according to boot cpu. Date: Sat, 09 Feb 2019 16:11:42 +0000 [thread overview] Message-ID: <86bm3lt001.wl-marc.zyngier@arm.com> (raw) In-Reply-To: <CAEn-LTqx8CQ7kY3kRKWFoAkSSu_FNoZkd9JnNYDNLAYX1kLHSA@mail.gmail.com> On Sat, 09 Feb 2019 04:26:07 +0000, David Abdurachmanov <david.abdurachmanov@gmail.com> wrote: > > On Sat, Feb 9, 2019 at 12:03 AM Atish Patra <atish.patra@wdc.com> wrote: > > > > On 2/8/19 1:11 AM, Christoph Hellwig wrote: > > >> + * We don't support running Linux on hertergenous ISA systems. > > >> + * But first "okay" processor might not be the boot cpu. > > >> + * Check the ISA of boot cpu. > > > > > > Please use up your available 80 characters per line in comments. > > > > > I will fix it. > > > > >> + /* > > >> + * All "okay" hart should have same isa. We don't know how to > > >> + * handle if they don't. Throw a warning for now. > > >> + */ > > >> + if (elf_hwcap && temp_hwcap != elf_hwcap) > > >> + pr_warn("isa mismatch: 0x%lx != 0x%lx\n", > > >> + elf_hwcap, temp_hwcap); > > >> + > > >> + if (hartid == boot_cpu_hartid) > > >> + boot_hwcap = temp_hwcap; > > >> + elf_hwcap = temp_hwcap; > > > > > > So we always set elf_hwcap to the capabilities of the previous cpu. > > > > > >> + temp_hwcap = 0; > > > > > > I think tmp_hwcap should be declared and initialized inside the outer loop > > > instead having to manually reset it like this. > > > > > >> + } > > >> > > >> + elf_hwcap = boot_hwcap; > > > > > > And then reset it here to the boot cpu. > > > > > > Shoudn't we only report the features supported by all cores? Otherwise > > > we'll still have problems if the boot cpu supports a feature, but not > > > others. > > > > > > > Hmm. The other side of the argument is boot cpu does have a feature that > > is not supported by other hart that didn't even boot. > > The user space may execute something based on boot cpu capability but > > that won't be enabled. > > > > At least, in this way we know that we are compatible completely with > > boot cpu capabilities. Thoughts ? > > There is one example on the market, e.g., Samsung Exynos 9810. > > Mongoose 3 (big cores) only support ARMv8.0, while Cortex-A55 > (little ones) support ARMv8.2 (and that brings atomics support). > I think, it's the only ARM SOC that supports different ISA extensions > between cores on the same package. > > Kernel scheduler doesn't know that big cores are missing atomics > support or that applications needs it and moves the thread > resulting in illegal instruction. Not quite. The scheduler doesn't have to know (thankfully). The problem is that the Samsung folks tampered with the detection logic in the kernel, and ended up advertising the LSE atomics to userspace (despite only being available on half the cores). If you run a mainline kernel on this things, it will just work, as the LSE atomics are not advertised to userspace at all. > > E.g., see Golang issue: https://github.com/golang/go/issues/28431 > > I also recall Jon Masters (Computer Architect at Red Hat) advocating > against having cores with mismatched capabilities on the server > market. Well, nobody recommends that, server or not. That being said, it is possible to handle it, and the arm64 kernel has been dealing with such thing from day 1. We can have CPUs with different PMUs, implemented page sizes, VA and PA spaces... What it takes is some work in the kernel to sanitize it, and be careful in what you expose to userspace. The thing to realise is that people will build stupid systems, no matter how loud you shout. You can either pretend they don't exist, or try to deal with them. Thanks, M. -- Jazz is not dead, it just smell funny.
WARNING: multiple messages have this Message-ID (diff)
From: Marc Zyngier <marc.zyngier@arm.com> To: David Abdurachmanov <david.abdurachmanov@gmail.com> Cc: "Damien Le Moal" <Damien.LeMoal@wdc.com>, "Albert Ou" <aou@eecs.berkeley.edu>, "Jason Cooper" <jason@lakedaemon.net>, "Alan Kao" <alankao@andestech.com>, "Dmitriy Cherkasov" <dmitriy@oss-tech.org>, "Anup Patel" <anup@brainfault.org>, "Daniel Lezcano" <daniel.lezcano@linaro.org>, "Patrick Stählin" <me@packi.ch>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "Christoph Hellwig" <hch@infradead.org>, "Atish Patra" <atish.patra@wdc.com>, "Palmer Dabbelt" <palmer@sifive.com>, "Paul Walmsley" <paul.walmsley@sifive.com>, "Andreas Schwab" <schwab@suse.de>, "linux-riscv@lists.infradead.org" <linux-riscv@lists.infradead.org>, "Thomas Gleixner" <tglx@linutronix.de>, "Zong Li" <zongbox@gmail.com> Subject: Re: [v3 PATCH 8/8] RISC-V: Assign hwcap only according to boot cpu. Date: Sat, 09 Feb 2019 16:11:42 +0000 [thread overview] Message-ID: <86bm3lt001.wl-marc.zyngier@arm.com> (raw) In-Reply-To: <CAEn-LTqx8CQ7kY3kRKWFoAkSSu_FNoZkd9JnNYDNLAYX1kLHSA@mail.gmail.com> On Sat, 09 Feb 2019 04:26:07 +0000, David Abdurachmanov <david.abdurachmanov@gmail.com> wrote: > > On Sat, Feb 9, 2019 at 12:03 AM Atish Patra <atish.patra@wdc.com> wrote: > > > > On 2/8/19 1:11 AM, Christoph Hellwig wrote: > > >> + * We don't support running Linux on hertergenous ISA systems. > > >> + * But first "okay" processor might not be the boot cpu. > > >> + * Check the ISA of boot cpu. > > > > > > Please use up your available 80 characters per line in comments. > > > > > I will fix it. > > > > >> + /* > > >> + * All "okay" hart should have same isa. We don't know how to > > >> + * handle if they don't. Throw a warning for now. > > >> + */ > > >> + if (elf_hwcap && temp_hwcap != elf_hwcap) > > >> + pr_warn("isa mismatch: 0x%lx != 0x%lx\n", > > >> + elf_hwcap, temp_hwcap); > > >> + > > >> + if (hartid == boot_cpu_hartid) > > >> + boot_hwcap = temp_hwcap; > > >> + elf_hwcap = temp_hwcap; > > > > > > So we always set elf_hwcap to the capabilities of the previous cpu. > > > > > >> + temp_hwcap = 0; > > > > > > I think tmp_hwcap should be declared and initialized inside the outer loop > > > instead having to manually reset it like this. > > > > > >> + } > > >> > > >> + elf_hwcap = boot_hwcap; > > > > > > And then reset it here to the boot cpu. > > > > > > Shoudn't we only report the features supported by all cores? Otherwise > > > we'll still have problems if the boot cpu supports a feature, but not > > > others. > > > > > > > Hmm. The other side of the argument is boot cpu does have a feature that > > is not supported by other hart that didn't even boot. > > The user space may execute something based on boot cpu capability but > > that won't be enabled. > > > > At least, in this way we know that we are compatible completely with > > boot cpu capabilities. Thoughts ? > > There is one example on the market, e.g., Samsung Exynos 9810. > > Mongoose 3 (big cores) only support ARMv8.0, while Cortex-A55 > (little ones) support ARMv8.2 (and that brings atomics support). > I think, it's the only ARM SOC that supports different ISA extensions > between cores on the same package. > > Kernel scheduler doesn't know that big cores are missing atomics > support or that applications needs it and moves the thread > resulting in illegal instruction. Not quite. The scheduler doesn't have to know (thankfully). The problem is that the Samsung folks tampered with the detection logic in the kernel, and ended up advertising the LSE atomics to userspace (despite only being available on half the cores). If you run a mainline kernel on this things, it will just work, as the LSE atomics are not advertised to userspace at all. > > E.g., see Golang issue: https://github.com/golang/go/issues/28431 > > I also recall Jon Masters (Computer Architect at Red Hat) advocating > against having cores with mismatched capabilities on the server > market. Well, nobody recommends that, server or not. That being said, it is possible to handle it, and the arm64 kernel has been dealing with such thing from day 1. We can have CPUs with different PMUs, implemented page sizes, VA and PA spaces... What it takes is some work in the kernel to sanitize it, and be careful in what you expose to userspace. The thing to realise is that people will build stupid systems, no matter how loud you shout. You can either pretend they don't exist, or try to deal with them. Thanks, M. -- Jazz is not dead, it just smell funny. _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv
next prev parent reply other threads:[~2019-02-09 16:11 UTC|newest] Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-02-08 1:51 [v3 PATCH 0/8] Various SMP related fixes Atish Patra 2019-02-08 1:51 ` Atish Patra 2019-02-08 1:51 ` [v3 PATCH 1/8] RISC-V: Do not wait indefinitely in __cpu_up Atish Patra 2019-02-08 1:51 ` Atish Patra 2019-02-08 9:01 ` Christoph Hellwig 2019-02-08 9:01 ` Christoph Hellwig 2019-02-08 1:51 ` [v3 PATCH 2/8] RISC-V: Move cpuid to hartid mapping to SMP Atish Patra 2019-02-08 1:51 ` Atish Patra 2019-02-08 9:03 ` Christoph Hellwig 2019-02-08 9:03 ` Christoph Hellwig 2019-02-08 22:56 ` Atish Patra 2019-02-08 22:56 ` Atish Patra 2019-02-08 1:51 ` [v3 PATCH 3/8] RISC-V: Remove NR_CPUs check during hartid search from DT Atish Patra 2019-02-08 1:51 ` Atish Patra 2019-02-08 1:51 ` [v3 PATCH 4/8] RISC-V: Allow hartid-to-cpuid function to fail Atish Patra 2019-02-08 1:51 ` Atish Patra 2019-02-08 1:51 ` [v3 PATCH 5/8] RISC-V: Compare cpuid with NR_CPUS before mapping Atish Patra 2019-02-08 1:51 ` Atish Patra 2019-02-08 1:51 ` [v3 PATCH 6/8] clocksource/drivers/riscv: Add required checks during clock source init Atish Patra 2019-02-08 1:51 ` Atish Patra 2019-02-08 9:04 ` Christoph Hellwig 2019-02-08 9:04 ` Christoph Hellwig 2019-02-08 22:56 ` Atish Patra 2019-02-08 22:56 ` Atish Patra 2019-02-08 1:51 ` [v3 PATCH 7/8] irqchip/irq-sifive-plic:: Check and continue in case of an invalid cpuid Atish Patra 2019-02-08 1:51 ` Atish Patra 2019-02-08 1:51 ` [v3 PATCH 8/8] RISC-V: Assign hwcap only according to boot cpu Atish Patra 2019-02-08 1:51 ` Atish Patra 2019-02-08 9:11 ` Christoph Hellwig 2019-02-08 9:11 ` Christoph Hellwig 2019-02-08 23:02 ` Atish Patra 2019-02-08 23:02 ` Atish Patra 2019-02-09 4:26 ` David Abdurachmanov 2019-02-09 4:26 ` David Abdurachmanov 2019-02-09 16:11 ` Marc Zyngier [this message] 2019-02-09 16:11 ` Marc Zyngier 2019-02-11 19:02 ` Palmer Dabbelt 2019-02-11 19:02 ` Palmer Dabbelt 2019-02-11 20:03 ` Atish Patra 2019-02-11 20:03 ` Atish Patra 2019-02-11 22:13 ` Marc Zyngier 2019-02-11 22:13 ` Marc Zyngier 2019-02-11 22:23 ` Palmer Dabbelt 2019-02-11 22:23 ` Palmer Dabbelt 2019-02-11 23:25 ` Atish Patra 2019-02-11 23:25 ` Atish Patra 2019-02-11 13:23 ` Andreas Schwab 2019-02-11 13:23 ` Andreas Schwab
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=86bm3lt001.wl-marc.zyngier@arm.com \ --to=marc.zyngier@arm.com \ --cc=Damien.LeMoal@wdc.com \ --cc=alankao@andestech.com \ --cc=anup@brainfault.org \ --cc=aou@eecs.berkeley.edu \ --cc=atish.patra@wdc.com \ --cc=daniel.lezcano@linaro.org \ --cc=david.abdurachmanov@gmail.com \ --cc=dmitriy@oss-tech.org \ --cc=hch@infradead.org \ --cc=jason@lakedaemon.net \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-riscv@lists.infradead.org \ --cc=me@packi.ch \ --cc=palmer@sifive.com \ --cc=paul.walmsley@sifive.com \ --cc=schwab@suse.de \ --cc=tglx@linutronix.de \ --cc=zongbox@gmail.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.