* Re: [PATCH] arm64: mmu: no write cache for O_SYNC flag
[not found] <20200326163625.30714-1-li.wang@windriver.com>
@ 2020-03-26 16:55 ` Catalin Marinas
2020-03-27 14:29 ` Mark Rutland
1 sibling, 0 replies; 3+ messages in thread
From: Catalin Marinas @ 2020-03-26 16:55 UTC (permalink / raw)
To: Li Wang; +Cc: Will Deacon, linux-kernel, linux-arm-kernel
On Thu, Mar 26, 2020 at 09:36:25AM -0700, Li Wang wrote:
> reproduce steps:
> 1.
> disable CONFIG_STRICT_DEVMEM in linux kernel
> 2.
> Process A gets a Physical Address of global variable by
> "/proc/self/pagemap".
> 3.
> Process B writes a value to the same Physical Address by mmap():
> fd=open("/dev/mem",O_SYNC);
> Virtual Address=mmap(fd);
>
> problem symptom:
> after Process B write a value to the Physical Address,
> Process A of the value of global variable does not change.
> They both W/R the same Physical Address.
>
> technical reason:
> Process B writing the Physical Address is by the Virtual Address,
> and the Virtual Address comes from "/dev/mem" and mmap().
> In arm64 arch, the Virtual Address has write cache.
> So, maybe the value is not written into Physical Address.
>
> fix reason:
> giving write cache flag in arm64 is in phys_mem_access_prot():
> =====
> arch/arm64/mm/mmu.c
> phys_mem_access_prot()
> {
> if (!pfn_valid(pfn))
> return pgprot_noncached(vma_prot);
> else if (file->f_flags & O_SYNC)
> return pgprot_writecombine(vma_prot);
> return vma_prot;
> }
> ====
> the other arch and the share function drivers/char/mem.c of phys_mem_access_prot()
> does not add write cache flag.
> So, removing the flag to fix the issue
Other architectures may have transparent caches and don't require
different attributes.
> Signed-off-by: Li Wang <li.wang@windriver.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will@kernel.org>
> Cc: linux-arm-kernel@lists.infradead.org
> Cc: linux-kernel@vger.kernel.org
> ---
> arch/arm64/mm/mmu.c | 2 --
> 1 file changed, 2 deletions(-)
>
> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> index 128f70852bf3..d7083965ca17 100644
> --- a/arch/arm64/mm/mmu.c
> +++ b/arch/arm64/mm/mmu.c
> @@ -81,8 +81,6 @@ pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
> {
> if (!pfn_valid(pfn))
> return pgprot_noncached(vma_prot);
> - else if (file->f_flags & O_SYNC)
> - return pgprot_writecombine(vma_prot);
> return vma_prot;
> }
> EXPORT_SYMBOL(phys_mem_access_prot);
A better solution is for user space not to pass O_SYNC when opening
/dev/mem. We've had this ABI for a long time (arch/arm/ and several
other architectures do the same), why change it now?
--
Catalin
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] arm64: mmu: no write cache for O_SYNC flag
[not found] <20200326163625.30714-1-li.wang@windriver.com>
2020-03-26 16:55 ` [PATCH] arm64: mmu: no write cache for O_SYNC flag Catalin Marinas
@ 2020-03-27 14:29 ` Mark Rutland
[not found] ` <6fc201bf-ad0c-3dae-783e-c9c9e4ac034e@windriver.com>
1 sibling, 1 reply; 3+ messages in thread
From: Mark Rutland @ 2020-03-27 14:29 UTC (permalink / raw)
To: Li Wang; +Cc: Catalin Marinas, Will Deacon, linux-kernel, linux-arm-kernel
On Thu, Mar 26, 2020 at 09:36:25AM -0700, Li Wang wrote:
> reproduce steps:
> 1.
> disable CONFIG_STRICT_DEVMEM in linux kernel
> 2.
> Process A gets a Physical Address of global variable by
> "/proc/self/pagemap".
> 3.
> Process B writes a value to the same Physical Address by mmap():
> fd=open("/dev/mem",O_SYNC);
> Virtual Address=mmap(fd);
Is this just to demonstrate the behaviour, or is this meant to be
indicative of a real use-case? I'm struggling to see the latter.
> problem symptom:
> after Process B write a value to the Physical Address,
> Process A of the value of global variable does not change.
> They both W/R the same Physical Address.
If Process A is not using the same attributes as process B, there is no
guarantee of coherency. How did process A map this memory?
> technical reason:
> Process B writing the Physical Address is by the Virtual Address,
> and the Virtual Address comes from "/dev/mem" and mmap().
> In arm64 arch, the Virtual Address has write cache.
> So, maybe the value is not written into Physical Address.
I don't think that's true. I think what's happening here is:
* Process A has a Normal WBWA Cacheable mapping.
* Process B as a Normal Non-cacheable mapping.
* Process B's write does not snoop any caches, and goes straight to
memory.
* Process A reads a value from cache, which does not include process B's
write.
That's a natural result of using mismatched attributes, and is
consistent with the O_SYNC flag meaning that the write "is transferred
to the underlying hardware".
>
> fix reason:
> giving write cache flag in arm64 is in phys_mem_access_prot():
> =====
> arch/arm64/mm/mmu.c
> phys_mem_access_prot()
> {
> if (!pfn_valid(pfn))
> return pgprot_noncached(vma_prot);
> else if (file->f_flags & O_SYNC)
> return pgprot_writecombine(vma_prot);
> return vma_prot;
> }
> ====
> the other arch and the share function drivers/char/mem.c of phys_mem_access_prot()
> does not add write cache flag.
> So, removing the flag to fix the issue
This will change behaviour that other software may be relying upon, and
as above I do not believe this actually solves the problem you describe.
Thanks,
Mark.
>
> Signed-off-by: Li Wang <li.wang@windriver.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will@kernel.org>
> Cc: linux-arm-kernel@lists.infradead.org
> Cc: linux-kernel@vger.kernel.org
> ---
> arch/arm64/mm/mmu.c | 2 --
> 1 file changed, 2 deletions(-)
>
> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> index 128f70852bf3..d7083965ca17 100644
> --- a/arch/arm64/mm/mmu.c
> +++ b/arch/arm64/mm/mmu.c
> @@ -81,8 +81,6 @@ pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
> {
> if (!pfn_valid(pfn))
> return pgprot_noncached(vma_prot);
> - else if (file->f_flags & O_SYNC)
> - return pgprot_writecombine(vma_prot);
> return vma_prot;
> }
> EXPORT_SYMBOL(phys_mem_access_prot);
> --
> 2.24.1
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] arm64: mmu: no write cache for O_SYNC flag
[not found] ` <6fc201bf-ad0c-3dae-783e-c9c9e4ac034e@windriver.com>
@ 2020-03-27 17:02 ` Mark Rutland
0 siblings, 0 replies; 3+ messages in thread
From: Mark Rutland @ 2020-03-27 17:02 UTC (permalink / raw)
To: Wang, Li; +Cc: Catalin Marinas, Will Deacon, linux-kernel, linux-arm-kernel
On Sat, Mar 28, 2020 at 12:47:32AM +0800, Wang, Li wrote:
>
> 在 2020/3/27 22:29, Mark Rutland 写道:
> > On Thu, Mar 26, 2020 at 09:36:25AM -0700, Li Wang wrote:
> > > reproduce steps:
> > > 1.
> > > disable CONFIG_STRICT_DEVMEM in linux kernel
> > > 2.
> > > Process A gets a Physical Address of global variable by
> > > "/proc/self/pagemap".
> > > 3.
> > > Process B writes a value to the same Physical Address by mmap():
> > > fd=open("/dev/mem",O_SYNC);
> > > Virtual Address=mmap(fd);
> > Is this just to demonstrate the behaviour, or is this meant to be
> > indicative of a real use-case? I'm struggling to see the latter.
> >
> > > problem symptom:
> > > after Process B write a value to the Physical Address,
> > > Process A of the value of global variable does not change.
> > > They both W/R the same Physical Address.
> > If Process A is not using the same attributes as process B, there is no
> > guarantee of coherency. How did process A map this memory?
>
>
> about 2 Process:
>
> Process A:
>
> the memory is not declared by map function, it is just a global variable.
Then it is exactly as I described previously, and Process A has it
mapped with a Normal Write-Back Cacheable mappping.
Process B requests a mapping of that memory via /dev/mem. It passes the
O_SYNC flag, and to ensure that accesses go to "the underlying hardware"
the kernel makes this mapping Normal Non-Cacheable (which means it
should not look in a cache, or be allocated into one).
The two mappings are not coherent because process A uses the cache, but
process B does not. This is the expected behaviour, consistent with the
semantic of O_SYNC. If you need the two to be coherent, they must both
use the same attributes.
Process B can be coherent with process A if it does *not* pass O_SYNC,
which would give it a Normal Write-Back Cacheable mapping that was
coherent with process A.
> if you agree that O_SYNC flag means "is transferred to the underlying
> hardware",
>
> the arm64 does not do that:
>
> when use O_SYNC flag under arm64 arch, it adds write cache feature,
As above, this is not the case. O_SYNC causes the kernel to use a
non-cacheable mapping, where it would normally create a cacheable
mapping. i.e. O_SYNC *removes* cacheability.
It just happens that process A is using a cacheable mapping, which is
the case regardless of what process B does.
Thanks,
Mark.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-03-27 17:03 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20200326163625.30714-1-li.wang@windriver.com>
2020-03-26 16:55 ` [PATCH] arm64: mmu: no write cache for O_SYNC flag Catalin Marinas
2020-03-27 14:29 ` Mark Rutland
[not found] ` <6fc201bf-ad0c-3dae-783e-c9c9e4ac034e@windriver.com>
2020-03-27 17:02 ` Mark Rutland
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).