change_page_attr() implementation for ARM?

change_page_attr() implementation for ARM?

[Vinod Rex]

On Cortex-A9, we have observed stale data being read from write-combine (C=0 B=1) memory regions mapped into userspace which have a duplicate cacheable mapping in the kernel address space (due to the kernel linear mapping).

The issue appears to be due to speculative prefetch on the cacheable kernel linear mapping which gets lines into the L2 cache. When reads are performed on the write-combine mapping for this address range, these reads get the stale data from L2 instead of memory.

As per the Cortex-A9 spec, behavior for double mappings with conflicting page attributes is undefined, so we need a way to make sure all duplicate mappings have the same memory type attributes. 

Similar issue on x86 is handled using the change_page_attr()/set_memory_*() functions defined in arch/x86/mm/pageattr.c. ?This function modifies the attributes of the page in kernel linear map to match the corresponding mapping in userspace to avoid having duplicate mappings with different page attributes. It accomplishes this by splitting section (large page) mappings into 4KB page mappings as needed so that the page attribute change is done only for the requested memory region.

Would a similar implementation for ARM be appropriate? We are experimenting with a port of change_page_attr() to ARM that seems to solve our problems.

Other approaches proposed to solve this on ARM, by remapping CMA reserved region using 4KB page mappings will not help us since IOMMU mappings don't use DMA contiguous allocation routines.

change_page_attr() implementation for ARM?

[Catalin Marinas]

Hi Vinod,

(I'm not sure how the quoted message ends up but your email client
didn't wrap lines properly)

On 21 September 2011 23:10, Vinod Rex <vrex@nvidia.com> wrote:
> On Cortex-A9, we have observed stale data being read from write-combine (C=0 B=1) memory regions mapped into userspace which have a duplicate cacheable mapping in the kernel address space (due to the kernel linear mapping).
>
> The issue appears to be due to speculative prefetch on the cacheable kernel linear mapping which gets lines into the L2 cache. When reads are performed on the write-combine mapping for this address range, these reads get the stale data from L2 instead of memory.

If your system has a PL310, there is bit 22 in the auxiliary control
register which makes reads via the non-cacheable mapping not to hit
the L2 cache. I had this patch queued in Russell's system for a long
time:

http://www.arm.linux.org.uk/developer/patches/viewpatch.php?id=6529/1

Alternatively, you can pass this bit via your platform code.

> As per the Cortex-A9 spec, behavior for double mappings with conflicting page attributes is undefined, so we need a way to make sure all duplicate mappings have the same memory type attributes.

Avoiding aliases would be better but we don't have a fully stable patch yet.

> Similar issue on x86 is handled using the change_page_attr()/set_memory_*() functions defined in arch/x86/mm/pageattr.c. ?This function modifies the attributes of the page in kernel linear map to match the corresponding mapping in userspace to avoid having duplicate mappings with different page attributes. It accomplishes this by splitting section (large page) mappings into 4KB page mappings as needed so that the page attribute change is done only for the requested memory region.
>
> Would a similar implementation for ARM be appropriate? We are experimenting with a port of change_page_attr() to ARM that seems to solve our problems.

The kernel linear mapping is done using sections, so we would have to
change the attributes for a full section. Russell's approach I think
is better but people reported some stability issues.

-- 
Catalin

change_page_attr() implementation for ARM?

[Krishna Reddy]

Catalin,

>> On Cortex-A9, we have observed stale data being read from write-combine (C=0 B=1) memory regions mapped into userspace which have a duplicate cacheable mapping in the kernel address space (due to the kernel linear mapping).
>>
>> The issue appears to be due to speculative prefetch on the cacheable kernel linear mapping which gets lines into the L2 cache. When reads are performed on the write-combine mapping for this address range, these reads get the stale data from L2 instead of memory.

>If your system has a PL310, there is bit 22 in the auxiliary control
>register which makes reads via the non-cacheable mapping not to hit
>the L2 cache. I had this patch queued in Russell's system for a long
>time:
>http://www.arm.linux.org.uk/developer/patches/viewpatch.php?id=6529/1
>Alternatively, you can pass this bit via your platform code.

Is this is a quick fix for ARM processors with PL310. Can this be expected in future versions of ARM processors with different L2 cache?

>The kernel linear mapping is done using sections, so we would have to
>change the attributes for a full section. Russell's approach I think
>is better but people reported some stability issues.

X86 way breaks the sections into pages and it only changes the attributes for the requested pages. the other pages of section would have same prot attributes as section. does Russell's approach refer to a different way? Is there any link on how Russell wants to do it?

--
nvpublic



-----Original Message-----
From: linux-arm-kernel-bounces@lists.infradead.org [mailto:linux-arm-kernel-bounces at lists.infradead.org] On Behalf Of Catalin Marinas
Sent: Friday, September 23, 2011 2:05 AM
To: Vinod Rex
Cc: linux-arm-kernel at lists.infradead.org
Subject: Re: change_page_attr() implementation for ARM?

Hi Vinod,

(I'm not sure how the quoted message ends up but your email client
didn't wrap lines properly)

On 21 September 2011 23:10, Vinod Rex <vrex@nvidia.com> wrote:
> On Cortex-A9, we have observed stale data being read from write-combine (C=0 B=1) memory regions mapped into userspace which have a duplicate cacheable mapping in the kernel address space (due to the kernel linear mapping).
>
> The issue appears to be due to speculative prefetch on the cacheable kernel linear mapping which gets lines into the L2 cache. When reads are performed on the write-combine mapping for this address range, these reads get the stale data from L2 instead of memory.

If your system has a PL310, there is bit 22 in the auxiliary control
register which makes reads via the non-cacheable mapping not to hit
the L2 cache. I had this patch queued in Russell's system for a long
time:

http://www.arm.linux.org.uk/developer/patches/viewpatch.php?id=6529/1

Alternatively, you can pass this bit via your platform code.

> As per the Cortex-A9 spec, behavior for double mappings with conflicting page attributes is undefined, so we need a way to make sure all duplicate mappings have the same memory type attributes.

Avoiding aliases would be better but we don't have a fully stable patch yet.

> Similar issue on x86 is handled using the change_page_attr()/set_memory_*() functions defined in arch/x86/mm/pageattr.c. ?This function modifies the attributes of the page in kernel linear map to match the corresponding mapping in userspace to avoid having duplicate mappings with different page attributes. It accomplishes this by splitting section (large page) mappings into 4KB page mappings as needed so that the page attribute change is done only for the requested memory region.
>
> Would a similar implementation for ARM be appropriate? We are experimenting with a port of change_page_attr() to ARM that seems to solve our problems.

The kernel linear mapping is done using sections, so we would have to
change the attributes for a full section. Russell's approach I think
is better but people reported some stability issues.

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

change_page_attr() implementation for ARM?

[Russell King - ARM Linux]

On Wed, Oct 12, 2011 at 07:27:58PM -0700, Krishna Reddy wrote:
> Catalin,
> >The kernel linear mapping is done using sections, so we would have to
> >change the attributes for a full section. Russell's approach I think
> >is better but people reported some stability issues.
> 
> X86 way breaks the sections into pages and it only changes the attributes
> for the requested pages. the other pages of section would have same prot
> attributes as section.

And doing that is the problem - you have multiple page tables to manipulate,
which in a SMP world could be in-use on different CPUs.  When you alter
such a page table, you will have a race between the CPU walking the
page tables and your update of that table.

Not only that but you will have to send a TLB flush to the other CPUs for
that address, and that requires an interruptible context.  There are
contexts which memory is allocated for DMA where that's not possible
(because IRQs are disabled by the caller.)

> does Russell's approach refer to a different way?
> Is there any link on how Russell wants to do it?

I had a patch which pre-allocates memory for the DMA allocators, but I
had to drop it because it caused regressions on platforms which had
very limited DMA memory available - it caused such platforms to panic
at boot.

So, this is proving to be an extremely difficult problem to solve -
and I've been wishing for quite a long time that ARM Ltd would get
their act together and realize that the combination of restrictions
(DMA incoherence plus not permitting aliases) is Very Bad News.

So, I'm hoping that there's an increasing chorus of voices trying to
persuade a move towards data-cache DMA coherence as standard.

change_page_attr() implementation for ARM?

[Catalin Marinas]

On Thu, Oct 13, 2011 at 03:27:58AM +0100, Krishna Reddy wrote:
> >> On Cortex-A9, we have observed stale data being read from
> >> write-combine (C=0 B=1) memory regions mapped into userspace which
> >> have a duplicate cacheable mapping in the kernel address space (due
> >> to the kernel linear mapping).
> >>
> >> The issue appears to be due to speculative prefetch on the
> >> cacheable kernel linear mapping which gets lines into the L2 cache.
> >> When reads are performed on the write-combine mapping for this
> >> address range, these reads get the stale data from L2 instead of
> >> memory.
> 
> >If your system has a PL310, there is bit 22 in the auxiliary control
> >register which makes reads via the non-cacheable mapping not to hit
> >the L2 cache. I had this patch queued in Russell's system for a long
> >time:
> >http://www.arm.linux.org.uk/developer/patches/viewpatch.php?id=6529/1
> >Alternatively, you can pass this bit via your platform code.
> 
> Is this is a quick fix for ARM processors with PL310. Can this be
> expected in future versions of ARM processors with different L2 cache?

That's specific to the PL310 cache controller, nothing to do with the
CPU. It's a configuration bit that may be useful in some scenarios.

-- 
Catalin

change_page_attr() implementation for ARM?

[Catalin Marinas]

On 13 October 2011 09:49, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Wed, Oct 12, 2011 at 07:27:58PM -0700, Krishna Reddy wrote:
>> >The kernel linear mapping is done using sections, so we would have to
>> >change the attributes for a full section. Russell's approach I think
>> >is better but people reported some stability issues.
...
>> does Russell's approach refer to a different way?
>> Is there any link on how Russell wants to do it?
>
> I had a patch which pre-allocates memory for the DMA allocators, but I
> had to drop it because it caused regressions on platforms which had
> very limited DMA memory available - it caused such platforms to panic
> at boot.
>
> So, this is proving to be an extremely difficult problem to solve -
> and I've been wishing for quite a long time that ARM Ltd would get
> their act together and realize that the combination of restrictions
> (DMA incoherence plus not permitting aliases) is Very Bad News.
>
> So, I'm hoping that there's an increasing chorus of voices trying to
> persuade a move towards data-cache DMA coherence as standard.

I mentioned the mismatched aliases clarification on several occasions,
though in private emails as the ARM ARM revC is not public yet. IIRC I
even sent a short document to a few people including you. Is that
clarification not enough (I can send it again if you want)?

Basically with Normal Non-cacheable DMA buffers (but not Strongly
Ordered) we are ok as long as the caches are cleaned on the other
Normal Cacheable mapping. Of course, you get different behaviour with
PL310 if you don't set bit 22 but that's a documented bit and not some
undocumented errata workaround (maybe this bit should have been 1 by
default).

-- 
Catalin

change_page_attr() implementation for ARM?

[Catalin Marinas]

On Tue, Nov 12, 2013 at 01:45:30PM +0000, Hiroshi Doyu wrote:
> Catalin Marinas <catalin.marinas@arm.com> wrote @ Thu, 17 Oct 2013 12:13:26 +0200:
> 
> > On Thu, Oct 17, 2013 at 06:05:23AM +0100, Hiroshi Doyu wrote:
> > > I found a bit old discussion[1] about CPA support on ARM32(Cortex-A9).
> > > 
> > > It seesm that CPA API on ARM wasn't possible because maintaining
> > > multiple pagetables under SMP is difficult. Instead currently we are
> > > using precallocated memory via DMA API.
> > > 
> > > Is this situation still same for Cortex-A15 and ARM64? If the
> > > mismatched aliasing problem doesn't happen, is CPA API feasible?
> > 
> > On ARMv7+LPAE and AArch64 we use separate TTBR1 for the kernel (linear)
> > mapping so in theory you could implement change_page_attr(). The
> > mismatched aliases restriction still exists but it is clarified on what
> > is and isn't allowed (so you can have Normal non-cacheable and Normal
> > cacheable aliases, as long as you take care of removing any dirty cache
> > lines that could potentially corrupt your data). The DMA API already
> > takes care of this.
> 
> In DMA API, creating a new map is done via ioremap_page_range() in
> kernel virtual address space. OTOH, the removing is done via
> unmap_kernel_range(). unmap_kernel_range() internally calls
> flush_cache_all(). So as long as kernel linear mapping side cache is
> kept clean, there's no risk that stale data(on linear side) is
> written back to RAM(corruption). That's why ARM doesn't need CPA API
> at all?

__dma_alloc_buffer() allocates the actual pages and flushes them, either
directly if in lowmem or does a kmap. After his, the buffer is
guaranteed not to have any dirty cache lines. The actual mapping
(non-cacheable) is done via ioremap_page_range() and the CPU is supposed
to access the buffer only via the new mapping to avoid hitting the cache
in the linear (and cacheable) mapping.

We don't really care about unmapping, there shouldn't be anything to
have dirtied the cache in the kernel linear mapping.

-- 
Catalin

change_page_attr() implementation for ARM?

[Hiroshi Doyu]

Hi Catalin,

Please clarify a few points inlined.

Catalin Marinas <catalin.marinas@arm.com> wrote @ Thu, 17 Oct 2013 12:13:26 +0200:

> On Thu, Oct 17, 2013 at 06:05:23AM +0100, Hiroshi Doyu wrote:
> > I found a bit old discussion[1] about CPA support on ARM32(Cortex-A9).
> > 
> > It seesm that CPA API on ARM wasn't possible because maintaining
> > multiple pagetables under SMP is difficult. Instead currently we are
> > using precallocated memory via DMA API.
> > 
> > Is this situation still same for Cortex-A15 and ARM64? If the
> > mismatched aliasing problem doesn't happen, is CPA API feasible?
> 
> On ARMv7+LPAE and AArch64 we use separate TTBR1 for the kernel (linear)
> mapping so in theory you could implement change_page_attr(). The
> mismatched aliases restriction still exists but it is clarified on what
> is and isn't allowed (so you can have Normal non-cacheable and Normal
> cacheable aliases, as long as you take care of removing any dirty cache
> lines that could potentially corrupt your data). The DMA API already
> takes care of this.

In DMA API, creating a new map is done via ioremap_page_range() in
kernel virtual address space. OTOH, the removing is done via
unmap_kernel_range(). unmap_kernel_range() internally calls
flush_cache_all(). So as long as kernel linear mapping side cache is
kept clean, there's no risk that stale data(on linear side) is
written back to RAM(corruption). That's why ARM doesn't need CPA API
at all?

> So, is the DMA API not enough for what you need?

In DMA API, get_vm_area_caller() is used to allocate kernel virtual
range, and the size VMALLOC range can be a limit if aggressively
used. This is minor issue, though.

Re: change_page_attr() implementation for ARM?

[Hiroshi Doyu]

Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org> wrote @ Thu, 17 Oct 2013 12:13:26 +0200:

> On Thu, Oct 17, 2013 at 06:05:23AM +0100, Hiroshi Doyu wrote:
> > I found a bit old discussion[1] about CPA support on ARM32(Cortex-A9).
> > 
> > It seesm that CPA API on ARM wasn't possible because maintaining
> > multiple pagetables under SMP is difficult. Instead currently we are
> > using precallocated memory via DMA API.
> > 
> > Is this situation still same for Cortex-A15 and ARM64? If the
> > mismatched aliasing problem doesn't happen, is CPA API feasible?
> 
> On ARMv7+LPAE and AArch64 we use separate TTBR1 for the kernel (linear)
> mapping so in theory you could implement change_page_attr(). The
> mismatched aliases restriction still exists but it is clarified on what
> is and isn't allowed (so you can have Normal non-cacheable and Normal
> cacheable aliases, as long as you take care of removing any dirty cache
> lines that could potentially corrupt your data). The DMA API already
> takes care of this.

Ok

> My (other) problem with change_page_attr() is that its not a standard
> in-kernel API. Powerpc and s390 implement change_page_attr() but with
> different arguments while x86 has change_page_attr_(set|clear).
> 
> So, is the DMA API not enough for what you need?

One of the possible usages may be to have buffers with some specific
page attributes, which could save some of cache maintenances, mainly
for better performance. For example, manipulating pagetables and some
special buffers shared by GPU.

change_page_attr() implementation for ARM?

[Hiroshi Doyu]

Catalin Marinas <catalin.marinas@arm.com> wrote @ Thu, 17 Oct 2013 12:13:26 +0200:

> On Thu, Oct 17, 2013 at 06:05:23AM +0100, Hiroshi Doyu wrote:
> > I found a bit old discussion[1] about CPA support on ARM32(Cortex-A9).
> > 
> > It seesm that CPA API on ARM wasn't possible because maintaining
> > multiple pagetables under SMP is difficult. Instead currently we are
> > using precallocated memory via DMA API.
> > 
> > Is this situation still same for Cortex-A15 and ARM64? If the
> > mismatched aliasing problem doesn't happen, is CPA API feasible?
> 
> On ARMv7+LPAE and AArch64 we use separate TTBR1 for the kernel (linear)
> mapping so in theory you could implement change_page_attr(). The
> mismatched aliases restriction still exists but it is clarified on what
> is and isn't allowed (so you can have Normal non-cacheable and Normal
> cacheable aliases, as long as you take care of removing any dirty cache
> lines that could potentially corrupt your data). The DMA API already
> takes care of this.

Ok

> My (other) problem with change_page_attr() is that its not a standard
> in-kernel API. Powerpc and s390 implement change_page_attr() but with
> different arguments while x86 has change_page_attr_(set|clear).
> 
> So, is the DMA API not enough for what you need?

One of the possible usages may be to have buffers with some specific
page attributes, which could save some of cache maintenances, mainly
for better performance. For example, manipulating pagetables and some
special buffers shared by GPU.

change_page_attr() implementation for ARM?

[Catalin Marinas]

On Thu, Oct 17, 2013 at 06:05:23AM +0100, Hiroshi Doyu wrote:
> I found a bit old discussion[1] about CPA support on ARM32(Cortex-A9).
> 
> It seesm that CPA API on ARM wasn't possible because maintaining
> multiple pagetables under SMP is difficult. Instead currently we are
> using precallocated memory via DMA API.
> 
> Is this situation still same for Cortex-A15 and ARM64? If the
> mismatched aliasing problem doesn't happen, is CPA API feasible?

On ARMv7+LPAE and AArch64 we use separate TTBR1 for the kernel (linear)
mapping so in theory you could implement change_page_attr(). The
mismatched aliases restriction still exists but it is clarified on what
is and isn't allowed (so you can have Normal non-cacheable and Normal
cacheable aliases, as long as you take care of removing any dirty cache
lines that could potentially corrupt your data). The DMA API already
takes care of this.

My (other) problem with change_page_attr() is that its not a standard
in-kernel API. Powerpc and s390 implement change_page_attr() but with
different arguments while x86 has change_page_attr_(set|clear).

So, is the DMA API not enough for what you need?

-- 
Catalin

change_page_attr() implementation for ARM?

[Hiroshi Doyu]

Hi,

I found a bit old discussion[1] about CPA support on ARM32(Cortex-A9).

It seesm that CPA API on ARM wasn't possible because maintaining
multiple pagetables under SMP is difficult. Instead currently we are
using precallocated memory via DMA API.

Is this situation still same for Cortex-A15 and ARM64? If the
mismatched aliasing problem doesn't happen, is CPA API feasible?

[1]
http://lists.infradead.org/pipermail/linux-arm-kernel/2011-October/069275.html
http://lists.infradead.org/pipermail/linux-arm-kernel/2011-October/069315.html
http://lists.infradead.org/pipermail/linux-arm-kernel/2011-October/069318.html