* Re: [PATCH] arm64: mmu: no write cache for O_SYNC flag [not found] <20200326163625.30714-1-li.wang@windriver.com> @ 2020-03-26 16:55 ` Catalin Marinas 2020-03-27 14:29 ` Mark Rutland 1 sibling, 0 replies; 3+ messages in thread From: Catalin Marinas @ 2020-03-26 16:55 UTC (permalink / raw) To: Li Wang; +Cc: Will Deacon, linux-kernel, linux-arm-kernel On Thu, Mar 26, 2020 at 09:36:25AM -0700, Li Wang wrote: > reproduce steps: > 1. > disable CONFIG_STRICT_DEVMEM in linux kernel > 2. > Process A gets a Physical Address of global variable by > "/proc/self/pagemap". > 3. > Process B writes a value to the same Physical Address by mmap(): > fd=open("/dev/mem",O_SYNC); > Virtual Address=mmap(fd); > > problem symptom: > after Process B write a value to the Physical Address, > Process A of the value of global variable does not change. > They both W/R the same Physical Address. > > technical reason: > Process B writing the Physical Address is by the Virtual Address, > and the Virtual Address comes from "/dev/mem" and mmap(). > In arm64 arch, the Virtual Address has write cache. > So, maybe the value is not written into Physical Address. > > fix reason: > giving write cache flag in arm64 is in phys_mem_access_prot(): > ===== > arch/arm64/mm/mmu.c > phys_mem_access_prot() > { > if (!pfn_valid(pfn)) > return pgprot_noncached(vma_prot); > else if (file->f_flags & O_SYNC) > return pgprot_writecombine(vma_prot); > return vma_prot; > } > ==== > the other arch and the share function drivers/char/mem.c of phys_mem_access_prot() > does not add write cache flag. > So, removing the flag to fix the issue Other architectures may have transparent caches and don't require different attributes. > Signed-off-by: Li Wang <li.wang@windriver.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Will Deacon <will@kernel.org> > Cc: linux-arm-kernel@lists.infradead.org > Cc: linux-kernel@vger.kernel.org > --- > arch/arm64/mm/mmu.c | 2 -- > 1 file changed, 2 deletions(-) > > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c > index 128f70852bf3..d7083965ca17 100644 > --- a/arch/arm64/mm/mmu.c > +++ b/arch/arm64/mm/mmu.c > @@ -81,8 +81,6 @@ pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, > { > if (!pfn_valid(pfn)) > return pgprot_noncached(vma_prot); > - else if (file->f_flags & O_SYNC) > - return pgprot_writecombine(vma_prot); > return vma_prot; > } > EXPORT_SYMBOL(phys_mem_access_prot); A better solution is for user space not to pass O_SYNC when opening /dev/mem. We've had this ABI for a long time (arch/arm/ and several other architectures do the same), why change it now? -- Catalin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] arm64: mmu: no write cache for O_SYNC flag [not found] <20200326163625.30714-1-li.wang@windriver.com> 2020-03-26 16:55 ` [PATCH] arm64: mmu: no write cache for O_SYNC flag Catalin Marinas @ 2020-03-27 14:29 ` Mark Rutland [not found] ` <6fc201bf-ad0c-3dae-783e-c9c9e4ac034e@windriver.com> 1 sibling, 1 reply; 3+ messages in thread From: Mark Rutland @ 2020-03-27 14:29 UTC (permalink / raw) To: Li Wang; +Cc: Catalin Marinas, Will Deacon, linux-kernel, linux-arm-kernel On Thu, Mar 26, 2020 at 09:36:25AM -0700, Li Wang wrote: > reproduce steps: > 1. > disable CONFIG_STRICT_DEVMEM in linux kernel > 2. > Process A gets a Physical Address of global variable by > "/proc/self/pagemap". > 3. > Process B writes a value to the same Physical Address by mmap(): > fd=open("/dev/mem",O_SYNC); > Virtual Address=mmap(fd); Is this just to demonstrate the behaviour, or is this meant to be indicative of a real use-case? I'm struggling to see the latter. > problem symptom: > after Process B write a value to the Physical Address, > Process A of the value of global variable does not change. > They both W/R the same Physical Address. If Process A is not using the same attributes as process B, there is no guarantee of coherency. How did process A map this memory? > technical reason: > Process B writing the Physical Address is by the Virtual Address, > and the Virtual Address comes from "/dev/mem" and mmap(). > In arm64 arch, the Virtual Address has write cache. > So, maybe the value is not written into Physical Address. I don't think that's true. I think what's happening here is: * Process A has a Normal WBWA Cacheable mapping. * Process B as a Normal Non-cacheable mapping. * Process B's write does not snoop any caches, and goes straight to memory. * Process A reads a value from cache, which does not include process B's write. That's a natural result of using mismatched attributes, and is consistent with the O_SYNC flag meaning that the write "is transferred to the underlying hardware". > > fix reason: > giving write cache flag in arm64 is in phys_mem_access_prot(): > ===== > arch/arm64/mm/mmu.c > phys_mem_access_prot() > { > if (!pfn_valid(pfn)) > return pgprot_noncached(vma_prot); > else if (file->f_flags & O_SYNC) > return pgprot_writecombine(vma_prot); > return vma_prot; > } > ==== > the other arch and the share function drivers/char/mem.c of phys_mem_access_prot() > does not add write cache flag. > So, removing the flag to fix the issue This will change behaviour that other software may be relying upon, and as above I do not believe this actually solves the problem you describe. Thanks, Mark. > > Signed-off-by: Li Wang <li.wang@windriver.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Will Deacon <will@kernel.org> > Cc: linux-arm-kernel@lists.infradead.org > Cc: linux-kernel@vger.kernel.org > --- > arch/arm64/mm/mmu.c | 2 -- > 1 file changed, 2 deletions(-) > > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c > index 128f70852bf3..d7083965ca17 100644 > --- a/arch/arm64/mm/mmu.c > +++ b/arch/arm64/mm/mmu.c > @@ -81,8 +81,6 @@ pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, > { > if (!pfn_valid(pfn)) > return pgprot_noncached(vma_prot); > - else if (file->f_flags & O_SYNC) > - return pgprot_writecombine(vma_prot); > return vma_prot; > } > EXPORT_SYMBOL(phys_mem_access_prot); > -- > 2.24.1 > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 3+ messages in thread
[parent not found: <6fc201bf-ad0c-3dae-783e-c9c9e4ac034e@windriver.com>]
* Re: [PATCH] arm64: mmu: no write cache for O_SYNC flag [not found] ` <6fc201bf-ad0c-3dae-783e-c9c9e4ac034e@windriver.com> @ 2020-03-27 17:02 ` Mark Rutland 0 siblings, 0 replies; 3+ messages in thread From: Mark Rutland @ 2020-03-27 17:02 UTC (permalink / raw) To: Wang, Li; +Cc: Catalin Marinas, Will Deacon, linux-kernel, linux-arm-kernel On Sat, Mar 28, 2020 at 12:47:32AM +0800, Wang, Li wrote: > > 在 2020/3/27 22:29, Mark Rutland 写道: > > On Thu, Mar 26, 2020 at 09:36:25AM -0700, Li Wang wrote: > > > reproduce steps: > > > 1. > > > disable CONFIG_STRICT_DEVMEM in linux kernel > > > 2. > > > Process A gets a Physical Address of global variable by > > > "/proc/self/pagemap". > > > 3. > > > Process B writes a value to the same Physical Address by mmap(): > > > fd=open("/dev/mem",O_SYNC); > > > Virtual Address=mmap(fd); > > Is this just to demonstrate the behaviour, or is this meant to be > > indicative of a real use-case? I'm struggling to see the latter. > > > > > problem symptom: > > > after Process B write a value to the Physical Address, > > > Process A of the value of global variable does not change. > > > They both W/R the same Physical Address. > > If Process A is not using the same attributes as process B, there is no > > guarantee of coherency. How did process A map this memory? > > > about 2 Process: > > Process A: > > the memory is not declared by map function, it is just a global variable. Then it is exactly as I described previously, and Process A has it mapped with a Normal Write-Back Cacheable mappping. Process B requests a mapping of that memory via /dev/mem. It passes the O_SYNC flag, and to ensure that accesses go to "the underlying hardware" the kernel makes this mapping Normal Non-Cacheable (which means it should not look in a cache, or be allocated into one). The two mappings are not coherent because process A uses the cache, but process B does not. This is the expected behaviour, consistent with the semantic of O_SYNC. If you need the two to be coherent, they must both use the same attributes. Process B can be coherent with process A if it does *not* pass O_SYNC, which would give it a Normal Write-Back Cacheable mapping that was coherent with process A. > if you agree that O_SYNC flag means "is transferred to the underlying > hardware", > > the arm64 does not do that: > > when use O_SYNC flag under arm64 arch, it adds write cache feature, As above, this is not the case. O_SYNC causes the kernel to use a non-cacheable mapping, where it would normally create a cacheable mapping. i.e. O_SYNC *removes* cacheability. It just happens that process A is using a cacheable mapping, which is the case regardless of what process B does. Thanks, Mark. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-03-27 17:03 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <20200326163625.30714-1-li.wang@windriver.com> 2020-03-26 16:55 ` [PATCH] arm64: mmu: no write cache for O_SYNC flag Catalin Marinas 2020-03-27 14:29 ` Mark Rutland [not found] ` <6fc201bf-ad0c-3dae-783e-c9c9e4ac034e@windriver.com> 2020-03-27 17:02 ` Mark Rutland
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).