From: Atish Patra <Atish.Patra@wdc.com> To: "paul.walmsley@sifive.com" <paul.walmsley@sifive.com> Cc: "hch@lst.de" <hch@lst.de>, "linux-riscv@lists.infradead.org" <linux-riscv@lists.infradead.org>, Damien Le Moal <Damien.LeMoal@wdc.com>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Anup Patel <Anup.Patel@wdc.com>, "aou@eecs.berkeley.edu" <aou@eecs.berkeley.edu>, "palmer@sifive.com" <palmer@sifive.com> Subject: Re: [PATCH v4 0/3] Optimize tlbflush path Date: Sat, 28 Sep 2019 04:23:28 +0000 [thread overview] Message-ID: <3bd071e32ef564c91eed9e921423a95a092ec111.camel@wdc.com> (raw) In-Reply-To: <alpine.DEB.2.21.9999.1908301939300.16731@viisi.sifive.com> On Fri, 2019-08-30 at 19:50 -0700, Paul Walmsley wrote: > Hi Atish, > > On Thu, 22 Aug 2019, Atish Patra wrote: > > > This series adds few optimizations to reduce the trap cost in the > > tlb > > flush path. We should only make SBI calls to remote tlb flush only > > if > > absolutely required. > > The patches look great. My understanding is that these optimization > patches may actually be a partial workaround for the TLB flushing bug > that > we've been looking at for the last month or so, which can corrupt > memory > or crash the system. > > If that's the case, let's first root-cause the underlying > bug. Otherwise > we'll just be papering over the actual issue, which probably could > still > occur even with this series, correct? Since it contains no explicit > fixes? > > I have verified the glibc locale install issue both in Qemu and Unleashed. I don't see any issue with OpenSBI master + Linux v5.3 kernel. As per our investigation, it looks like a hardware errata with Unleashed board as the memory corruption issue only happens in case of tlb range flush. In RISC-V, sfence.vma can only be issued at page boundary. If the range is larger than that, OpenSBI has to issue multiple sfence.vma calls back to back leading to possible memory corruption. Currently, OpenSBI has a platform feature i.e. "tlb_range_flush_limit" that allows to configure tlb flush threshold per platform. Any tlb flush range request greater than this threshold, is converted to a full flush. Currently, it is set to the default value 4K for every platform[1]. Glibc locale install memory corruption only happens if this threshold is changed to a higher value i.e. 1G. This doesn't change anything in OpenSBI code path except the fact that it will issue many sfence.vma instructions back to back instead of one. If the hardware team at SiFive can look into this as well, it would be great. To conclude, we think this issue need to be investigated by hardware team and the kernel patch can be merged to get the performance benefit. [1] https://github.com/riscv/opensbi/blob/master/include/sbi/sbi_platform.h#L40 > - Paul > > _______________________________________________ > linux-riscv mailing list > linux-riscv@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-riscv -- Regards, Atish
WARNING: multiple messages have this Message-ID (diff)
From: Atish Patra <Atish.Patra@wdc.com> To: "paul.walmsley@sifive.com" <paul.walmsley@sifive.com> Cc: Damien Le Moal <Damien.LeMoal@wdc.com>, "aou@eecs.berkeley.edu" <aou@eecs.berkeley.edu>, Anup Patel <Anup.Patel@wdc.com>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "palmer@sifive.com" <palmer@sifive.com>, "linux-riscv@lists.infradead.org" <linux-riscv@lists.infradead.org>, "hch@lst.de" <hch@lst.de> Subject: Re: [PATCH v4 0/3] Optimize tlbflush path Date: Sat, 28 Sep 2019 04:23:28 +0000 [thread overview] Message-ID: <3bd071e32ef564c91eed9e921423a95a092ec111.camel@wdc.com> (raw) In-Reply-To: <alpine.DEB.2.21.9999.1908301939300.16731@viisi.sifive.com> On Fri, 2019-08-30 at 19:50 -0700, Paul Walmsley wrote: > Hi Atish, > > On Thu, 22 Aug 2019, Atish Patra wrote: > > > This series adds few optimizations to reduce the trap cost in the > > tlb > > flush path. We should only make SBI calls to remote tlb flush only > > if > > absolutely required. > > The patches look great. My understanding is that these optimization > patches may actually be a partial workaround for the TLB flushing bug > that > we've been looking at for the last month or so, which can corrupt > memory > or crash the system. > > If that's the case, let's first root-cause the underlying > bug. Otherwise > we'll just be papering over the actual issue, which probably could > still > occur even with this series, correct? Since it contains no explicit > fixes? > > I have verified the glibc locale install issue both in Qemu and Unleashed. I don't see any issue with OpenSBI master + Linux v5.3 kernel. As per our investigation, it looks like a hardware errata with Unleashed board as the memory corruption issue only happens in case of tlb range flush. In RISC-V, sfence.vma can only be issued at page boundary. If the range is larger than that, OpenSBI has to issue multiple sfence.vma calls back to back leading to possible memory corruption. Currently, OpenSBI has a platform feature i.e. "tlb_range_flush_limit" that allows to configure tlb flush threshold per platform. Any tlb flush range request greater than this threshold, is converted to a full flush. Currently, it is set to the default value 4K for every platform[1]. Glibc locale install memory corruption only happens if this threshold is changed to a higher value i.e. 1G. This doesn't change anything in OpenSBI code path except the fact that it will issue many sfence.vma instructions back to back instead of one. If the hardware team at SiFive can look into this as well, it would be great. To conclude, we think this issue need to be investigated by hardware team and the kernel patch can be merged to get the performance benefit. [1] https://github.com/riscv/opensbi/blob/master/include/sbi/sbi_platform.h#L40 > - Paul > > _______________________________________________ > linux-riscv mailing list > linux-riscv@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-riscv -- Regards, Atish _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv
next prev parent reply other threads:[~2019-09-28 4:23 UTC|newest] Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-08-22 7:51 [PATCH v4 0/3] Optimize tlbflush path Atish Patra 2019-08-22 7:51 ` Atish Patra 2019-08-22 7:51 ` [PATCH v4 1/3] RISC-V: Do not invoke SBI call if cpumask is empty Atish Patra 2019-08-22 7:51 ` Atish Patra 2019-08-22 8:10 ` Christoph Hellwig 2019-08-22 8:10 ` Christoph Hellwig 2019-08-22 7:51 ` [PATCH v4 2/3] RISC-V: Issue a local tlbflush if possible Atish Patra 2019-08-22 7:51 ` Atish Patra 2019-08-22 8:10 ` Christoph Hellwig 2019-08-22 8:10 ` Christoph Hellwig 2019-08-22 7:51 ` [PATCH v4 3/3] RISC-V: Issue a tlb page flush " Atish Patra 2019-08-22 7:51 ` Atish Patra 2019-08-22 8:11 ` Christoph Hellwig 2019-08-22 8:11 ` Christoph Hellwig 2019-08-22 18:31 ` Atish Patra 2019-08-22 18:31 ` Atish Patra 2019-08-31 2:50 ` [PATCH v4 0/3] Optimize tlbflush path Paul Walmsley 2019-08-31 2:50 ` Paul Walmsley 2019-09-28 4:23 ` Atish Patra [this message] 2019-09-28 4:23 ` Atish Patra 2019-10-25 3:09 ` Paul Walmsley 2019-10-25 3:09 ` Paul Walmsley 2019-10-31 15:13 ` Atish Patra 2019-10-31 15:13 ` Atish Patra
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=3bd071e32ef564c91eed9e921423a95a092ec111.camel@wdc.com \ --to=atish.patra@wdc.com \ --cc=Anup.Patel@wdc.com \ --cc=Damien.LeMoal@wdc.com \ --cc=aou@eecs.berkeley.edu \ --cc=hch@lst.de \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-riscv@lists.infradead.org \ --cc=palmer@sifive.com \ --cc=paul.walmsley@sifive.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.