From: Guo Ren <guoren@kernel.org>
To: Palmer Dabbelt <palmer@sifive.com>
Cc: ren_guo@c-sky.com, linux-riscv@lists.infradead.org,
linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org,
tech-privileged@lists.riscv.org,
Andrew Waterman <andrew@sifive.com>,
anup.patel@wdc.com, Arnd Bergmann <arnd@arndb.de>,
Christoph Hellwig <hch@lst.de>,
green.hu@gmail.com, m.szyprowski@samsung.com, rppt@linux.ibm.com,
robin.murphy@arm.com, swood@redhat.com, vincentc@andestech.com,
xiaoyan_xiang@c-sky.com
Subject: Re: [PATCH] riscv: Support non-coherency memory model
Date: Tue, 30 Apr 2019 11:29:16 +0800 [thread overview]
Message-ID: <20190430032916.GA649@guoren-Inspiron-7460> (raw)
In-Reply-To: <mhng-4889b94b-2734-4657-83c2-654d3677733e@palmer-si-x1e>
On Mon, Apr 29, 2019 at 01:11:43PM -0700, Palmer Dabbelt wrote:
> On Mon, 22 Apr 2019 08:44:30 PDT (-0700), guoren@kernel.org wrote:
> >From: Guo Ren <ren_guo@c-sky.com>
> >
> >The current riscv linux implementation requires SOC system to support
> >memory coherence between all I/O devices and CPUs. But some SOC systems
> >cannot maintain the coherence and they need support cache clean/invalid
> >operations to synchronize data.
> >
> >Current implementation is no problem with SiFive FU540, because FU540
> >keeps all IO devices and DMA master devices coherence with CPU. But to a
> >traditional SOC vendor, it may already have a stable non-coherency SOC
> >system, the need is simply to replace the CPU with RV CPU and rebuild
> >the whole system with IO-coherency is very expensive.
> >
> >So we should make riscv linux also support non-coherency memory model.
> >Here are the two points that riscv linux needs to be modified:
> >
> > - Add _PAGE_COHERENCY bit in current page table entry attributes. The bit
> > designates a coherence for this page mapping. Software set the bit to
> > tell the hardware that the region of the page's memory area must be
> > coherent with IOs devices in SOC system by PMA settings.
> > If IOs and CPU are already coherent in SOC system, CPU just ignore
> > this bit.
> >
> > PTE format:
> > | XLEN-1 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
> > PFN C RSW D A G U X W R V
> > ^
> > BIT(9): Coherence attribute bit
> > 0: hardware needn't keep the page coherenct and software will
> > maintain the coherence with cache clear/invalid operations.
> > 1: hardware must keep the page coherenct and software needn't
> > maintain the coherence.
> > BIT(8): Reserved for software and now it's _PAGE_SPECIAL in linux
> >
> > Add a new hardware bit in PTE also need to modify Privileged
> > Architecture Supervisor-Level ISA:
> > https://github.com/riscv/riscv-isa-manual/pull/374
>
> This is a RISC-V ISA modification, which isn't really appropriate to suggest on
> the kernel mailing lists. The right place to talk about this is at the RISC-V
> foundation, which owns the ISA -- we can't change the hardware with a patch to
> Linux :).
I just want a discussion and a wide discussion is good for all of us :)
>
> > - Add SBI_FENCE_DMA 9 in riscv-sbi.
> > sbi_fence_dma(start, size, dir) could synchronize CPU cache data with
> > DMA device in non-coherency memory model. The third param's definition
> > is the same with linux's in include/linux/dma-direction.h:
> >
> > enum dma_data_direction {
> > DMA_BIDIRECTIONAL = 0,
> > DMA_TO_DEVICE = 1,
> > DMA_FROM_DEVICE = 2,
> > DMA_NONE = 3,
> > };
> >
> > The first param:start must be physical address which could be handled
> > in M-state.
> >
> > Here is a pull request to the riscv-sbi-doc:
> > https://github.com/riscv/riscv-sbi-doc/pull/15
> >
> >We have tested the patch on our fpga SOC system which network controller
> >connected to a non-cache-coherency interconnect in and it couldn't work
> >without the patch.
> >
> >There is no side effect for FU540 whose CPU don't care _PAGE_COHERENCY
> >in PTE, but FU540's bbl also need to implement a simple sbi_fence_dma
> >by directly return. In fact, if you give a correct configuration for
> >dev_is_dma_conherent(), linux dma framework wouldn't call sbi_fence_dma
> >any more.
>
> Non-coherent fences also need to be discussed as part of a RISC-V ISA
^^^^^^^^^^^^ ^^^^^^
fences instructions? not page attributes?
> extension.
> I know people have expressed interest, but I don't know of a
> working group that's already been set up.
Is that mean current RISC-V ISA forces the SOC to be coherent memory model?
Best Regards
Guo Ren
prev parent reply other threads:[~2019-04-30 3:29 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-04-22 15:44 [PATCH] riscv: Support non-coherency memory model guoren
2019-04-22 16:18 ` Christoph Hellwig
2019-04-23 0:13 ` Guo Ren
2019-04-23 5:55 ` Christoph Hellwig
2019-04-23 15:46 ` Guo Ren
2019-04-23 15:57 ` Gary Guo
2019-04-24 2:08 ` Guo Ren
2019-04-24 3:21 ` Gary Guo
2019-04-24 5:57 ` Guo Ren
2019-04-24 12:45 ` Gary Guo
2019-04-24 14:23 ` Christoph Hellwig
2019-04-25 9:50 ` Arnd Bergmann
2019-04-26 16:05 ` Guo Ren
2019-04-26 18:42 ` Arnd Bergmann
2019-04-26 19:05 ` [tech-privileged] " Bill Huffman
2019-04-23 0:31 ` kbuild test robot
2019-04-29 20:11 ` Palmer Dabbelt
2019-04-30 3:29 ` Guo Ren [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190430032916.GA649@guoren-Inspiron-7460 \
--to=guoren@kernel.org \
--cc=andrew@sifive.com \
--cc=anup.patel@wdc.com \
--cc=arnd@arndb.de \
--cc=green.hu@gmail.com \
--cc=hch@lst.de \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-riscv@lists.infradead.org \
--cc=m.szyprowski@samsung.com \
--cc=palmer@sifive.com \
--cc=ren_guo@c-sky.com \
--cc=robin.murphy@arm.com \
--cc=rppt@linux.ibm.com \
--cc=swood@redhat.com \
--cc=tech-privileged@lists.riscv.org \
--cc=vincentc@andestech.com \
--cc=xiaoyan_xiang@c-sky.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).