All of lore.kernel.org
 help / color / mirror / Atom feed
From: James Bottomley <James.Bottomley@HansenPartnership.com>
To: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>,
	Mark Salter <msalter@redhat.com>,
	Vineet Gupta <Vineet.Gupta1@synopsys.com>,
	linux-arch@vger.kernel.org, linux-c6x-dev@linux-c6x.org,
	linux-kernel@vger.kernel.org
Subject: Re: [Linux-c6x-dev] [PATCH 3/9] c6x: Provide dma_mmap_coherent() and dma_get_sgtable()
Date: Wed, 23 Jan 2013 10:29:46 +0000	[thread overview]
Message-ID: <1358936986.2584.36.camel@dabdike.int.hansenpartnership.com> (raw)
In-Reply-To: <50FFB19E.3020901@samsung.com>

On Wed, 2013-01-23 at 10:47 +0100, Marek Szyprowski wrote:
> On 1/22/2013 11:13 AM, James Bottomley wrote:
> > There might be a simple solution:  just replace void *cpu_addr with void
> > **cpu_addr in the API.  This is a bit nasty since compilers think that
> > void ** to void * conversion is quite legal, so it would be hard to pick
> > up misuse of this (uint8_t ** would be better).  That way VIPT could
> > remap the kernel pages to a coherent address.  This would probably have
> > to change in the dma_mmap_attr() and dma_ops structures.
> >
> > All consumers would have to expect cpu_addr might change, but that seems
> > doable.
> 
> I still don't get how this can help having a coherent buffer between DMA
> (devices) and CPU (kernel and user space mappings). The main purpose of
> the dma_mmap_coherent() call is to provide a common API for mapping a
> coherent buffer between DMA (device) and userspace. It is not about
> creating a coherent mapping for sharing memory between userspace and
> kernel space.

OK, so I assume you don't understand how VIPT architectures work?

On a VIPT architecture, the CPU cache is indexed by the virtual address
but tagged by the physical address.  This means that when an address
comes into the CPU, we can do simultaneous lookups in the cache and the
TLB (by the virtual address).  The cache doesn't have the full address
bits of the index, so it usually only looks up the lowest n bits.  The
value of n gives the congruency of the cache (sometimes called the
colour of the cache lines).  The cache usually produces a number of
possible lines depending on its associativity and the TLB lookup
produces the physical address.  We can now sweep through the cache lines
and if a physical address tag matches, return the answer from the cache
instead of having to consult main memory.  This gives a speed advantage
over PIPT (Physically Indexed Physically Tagged) caches because on PIPT
the cache lookup can only occur *after* the TLB lookup instead of
concurrently with.

As an aside, in practise every PIPT CPU actually has a VIPT cache,
purely because of the speed angle.  The trick to making it all work is
to shrink n so that n <= PAGE_SIZE_BITS and increase the associativity.
This means you can get VIPT speed without producing the aliasing
effects.

Coherence is achieved in VIPT CPUs when two mapping addresses for the
same physical page, say v1 and v2 are congruent i.e. (v1 & ((1<<n)-1))
== (v2 & ((1<<n)-1)).  For such mappings, the same cache line is used.
This is why we have an architecture override for
arch_get_unmapped_area(). we use this to ensure all shareable mappings
for all userspace process are at congruent addresses, so we don't get
any aliasing problems in shared VMAs.

Aliasing happens when v1 & ((1<<n)-1)) != (v2 & ((1<<n)-1)) where v1 and
v2 are virtual addresses of the same physical page.  Now that page has
*two* cache copies and you have to be very careful to resolve the
aliases.  This, unfortunately, is the default scenario for kernel
virtual and user virtual addresses because all kernel pages are offset
mapped which is why we have to be annoyingly careful about flushing
kernel pages before we hand control back to userspace.

As you can see, we can only share a mapping between the kernel and user
space without all the aliasing problems if the address is congruent.

dma_mmap_coherent() seeks to be coherent from the device to the kernel
to userspace.  On PARISC, we fix device coherency by loading a coherence
index into our IOMMU (it effectively tells the IOMMU the virtual alias
for the physical memory and allows the CPU cache to be managed as part
of the I/O transaction).  The only way to prevent aliasing between the
kernel and user space is to make the two addresses congruent.  We
already have a fixed user address (given to us by the vma), so our only
choice is to remap the page in the kernel to a congruent address and
bingo we have a buffer shared between device/kernel/user space with no
aliasing problems.

The reason for making the cpu_addr mutable is to allow the
implementation to remap the page and make full congruency happen.  We
can't allow the kernel and userspace addresses to be non-congruent
because kmap will fail horribly (it's required by the dma_buf_ops) and
we also run the risk of getting a machine check if the cache discovers
two dirty aliases of the same page.

James



  reply	other threads:[~2013-01-23 10:29 UTC|newest]

Thread overview: 52+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-01-13 10:44 [PATCH 1/9] avr32: Provide dma_mmap_coherent() and dma_get_sgtable() Geert Uytterhoeven
2013-01-13 10:44 ` [PATCH 2/9] blackfin: " Geert Uytterhoeven
2013-01-13 10:44   ` Geert Uytterhoeven
2013-01-15  8:12   ` [uclinux-dist-devel] " Jiang, Scott
2013-01-15  8:12     ` Jiang, Scott
2013-01-13 10:44 ` [PATCH 3/9] c6x: " Geert Uytterhoeven
2013-01-14 15:37   ` [Linux-c6x-dev] " Mark Salter
2013-01-15  4:16     ` Vineet Gupta
2013-01-15  4:16       ` Vineet Gupta
2013-01-15  9:13       ` Geert Uytterhoeven
2013-01-15 14:07         ` Marek Szyprowski
2013-01-15 16:56           ` James Bottomley
2013-01-21 20:00             ` Geert Uytterhoeven
2013-01-21 22:59               ` James Bottomley
2013-01-22 10:13                 ` James Bottomley
2013-01-22 10:16                   ` James Bottomley
2013-01-22 10:32                     ` James Bottomley
2013-01-22 13:42                       ` Mauro Carvalho Chehab
2013-01-22 13:23                     ` Mauro Carvalho Chehab
2013-01-22 13:23                       ` Mauro Carvalho Chehab
2013-01-22 10:33                   ` Marek Szyprowski
2013-01-22 10:47                     ` James Bottomley
2013-01-23  9:47                   ` Marek Szyprowski
2013-01-23 10:29                     ` James Bottomley [this message]
2013-01-23 17:44                       ` Marek Szyprowski
2013-01-24 11:14                         ` James Bottomley
2013-01-24 10:49               ` Marek Szyprowski
2013-01-22 10:15             ` Marek Szyprowski
2013-01-22 10:22               ` James Bottomley
2013-01-23  7:23                 ` Vineet Gupta
2013-01-23  7:23                   ` Vineet Gupta
2013-01-23  8:58                   ` James Bottomley
2013-01-13 10:44 ` [PATCH 4/9] cris: " Geert Uytterhoeven
2013-01-14  8:38   ` Jesper Nilsson
2013-01-13 10:44 ` [PATCH 5/9] frv: " Geert Uytterhoeven
2013-01-13 10:44 ` [PATCH 6/9] m68k: " Geert Uytterhoeven
2013-01-13 10:44   ` Geert Uytterhoeven
2013-01-13 10:44 ` [PATCH 7/9] mn10300: " Geert Uytterhoeven
2013-01-13 10:44 ` [PATCH 8/9] parisc: " Geert Uytterhoeven
2013-01-13 10:44   ` Geert Uytterhoeven
2013-01-13 11:36   ` James Bottomley
2013-01-13 11:36     ` James Bottomley
2013-01-13 13:12     ` Geert Uytterhoeven
2013-01-13 13:12       ` Geert Uytterhoeven
2013-01-13 13:49       ` James Bottomley
2013-01-13 13:49         ` James Bottomley
2013-01-13 14:52         ` Geert Uytterhoeven
2013-01-13 14:52           ` Geert Uytterhoeven
2013-01-13 16:37           ` James Bottomley
2013-01-13 16:37             ` James Bottomley
2013-01-13 10:44 ` [PATCH 9/9] xtensa: " Geert Uytterhoeven
2013-01-13 16:01 ` [PATCH 1/9] avr32: " Hans-Christian Egtvedt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1358936986.2584.36.camel@dabdike.int.hansenpartnership.com \
    --to=james.bottomley@hansenpartnership.com \
    --cc=Vineet.Gupta1@synopsys.com \
    --cc=geert@linux-m68k.org \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-c6x-dev@linux-c6x.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=m.szyprowski@samsung.com \
    --cc=msalter@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.