linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Missing cache flush.
@ 2001-06-04 23:16 David Woodhouse
  2001-06-04 23:23 ` Jeff Garzik
                   ` (4 more replies)
  0 siblings, 5 replies; 34+ messages in thread
From: David Woodhouse @ 2001-06-04 23:16 UTC (permalink / raw)
  To: bjornw; +Cc: linux-kernel, linux-mtd

The flash mapping driver arch/cris/drivers/axisflashmap.c uses a cached
mapping of the flash chips for bulk reads, but obviously an uncached mapping
for sending commands and reading status when we're actually writing to or
erasing parts of the chip.

However, it fails to flush the dcache for the range used when the flash is 
accessed through the uncached mapping. So after an erase or write, we may 
read old data from the cache for the changed area.

All the mapping driver needs to do is invalidate the dcache for the affected
area before the next copy_from() operation. No need to worry about writeback
in this case, because we never write to flash chips through the cached
mapping.

However, I can't see a cache operation which performs this function.
flush_dcache_page() is defined as a NOP on CRIS as, it seems, it is on most
architectures. On other architectures, there's dma_cache_wback_inv(), but
that also seems to be a NOP on i386, to pick a random example.

I'm aware that some architectures can't handle having both cached and
uncached mappings of the same physical range - so to prevent dismissal of
the question out of hand by people assuming all the world's a PeeCee -
consider the alternative situation where we have ROM or RAM chips in a paged
mapping such that only a 64K 'page' is visible by the CPU at a time
(remember XMS?). Using an uncached mapping is extremely suboptimal - all we
want to do is invalidate the cache when we change the page, or writeback 
and invalidate in the case of RAM.

I would have thought that's the function that dma_cache_wback_inv() is
supposed to perform - but it seems not to do so.

So how is this _supposed_ to be done?

I was pointed at Documentation/DMA-mapping.txt but that doesn't seem very
helpful - it's very PCI-specific, and a quick perusal of pci_dma_sync() on
i386 shows that it doesn't do what's required anyway.

--
dwmw2



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-04 23:16 Missing cache flush David Woodhouse
@ 2001-06-04 23:23 ` Jeff Garzik
  2001-06-04 23:29 ` David Woodhouse
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 34+ messages in thread
From: Jeff Garzik @ 2001-06-04 23:23 UTC (permalink / raw)
  To: David Woodhouse; +Cc: bjornw, linux-kernel, linux-mtd

David Woodhouse wrote:
> I was pointed at Documentation/DMA-mapping.txt but that doesn't seem very
> helpful - it's very PCI-specific, and a quick perusal of pci_dma_sync() on
> i386 shows that it doesn't do what's required anyway.

What should it do on i386?  mb()?

-- 
Jeff Garzik      | Echelon words of the day, from The Register:
Building 1024    | FRU Lebed HALO Spetznaz Al Amn al-Askari Glock 26 
MandrakeSoft     | Steak Knife Kill the President anarchy echelon
                 | nuclear assassinate Roswell Waco World Trade Center

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-04 23:16 Missing cache flush David Woodhouse
  2001-06-04 23:23 ` Jeff Garzik
@ 2001-06-04 23:29 ` David Woodhouse
  2001-06-05  2:03 ` David S. Miller
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 34+ messages in thread
From: David Woodhouse @ 2001-06-04 23:29 UTC (permalink / raw)
  To: Jeff Garzik; +Cc: bjornw, linux-kernel, linux-mtd


jgarzik@mandrakesoft.com said:
> > I was pointed at Documentation/DMA-mapping.txt but that doesn't seem
> > very helpful - it's very PCI-specific, and a quick perusal of
> > pci_dma_sync() on i386 shows that it doesn't do what's required anyway.

> What should it do on i386?  mb()? 

For it to have any use in the situation I described, it would need to 
writeback and invalidate the dcache for the affected range. It doesn't seem 
to do so, so it seems that it isn't what I require.

The situation is simple - I have a paged RAM setup and I need it cached. 
All I want to do is flush and invalidate the cache when I'm about to waggle 
whatever I/O ports I waggle to change pages. 

There are other situations in which I need the cache flushed, but the above 
is one of the simplest.

Even flush_page_to_ram() doesn't seem to do what its name implies, on most 
architectures.

--
dwmw2



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-04 23:16 Missing cache flush David Woodhouse
  2001-06-04 23:23 ` Jeff Garzik
  2001-06-04 23:29 ` David Woodhouse
@ 2001-06-05  2:03 ` David S. Miller
  2001-06-05  3:55   ` Chris Wedgwood
  2001-06-05  4:01   ` David S. Miller
  2001-06-05  2:04 ` David S. Miller
  2001-06-05  9:17 ` Bjorn Wesen
  4 siblings, 2 replies; 34+ messages in thread
From: David S. Miller @ 2001-06-05  2:03 UTC (permalink / raw)
  To: Jeff Garzik; +Cc: David Woodhouse, bjornw, linux-kernel, linux-mtd


Jeff Garzik writes:
 > David Woodhouse wrote:
 > > I was pointed at Documentation/DMA-mapping.txt but that doesn't seem very
 > > helpful - it's very PCI-specific, and a quick perusal of pci_dma_sync() on
 > > i386 shows that it doesn't do what's required anyway.
 > 
 > What should it do on i386?  mb()?

The x86 doesn't have dumb caches, therefore it really doesn't need to
flush anything.  Maybe a mb(), but that is it.

Later,
David S. Miller
davem@redhat.com

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-04 23:16 Missing cache flush David Woodhouse
                   ` (2 preceding siblings ...)
  2001-06-05  2:03 ` David S. Miller
@ 2001-06-05  2:04 ` David S. Miller
  2001-06-05  9:17 ` Bjorn Wesen
  4 siblings, 0 replies; 34+ messages in thread
From: David S. Miller @ 2001-06-05  2:04 UTC (permalink / raw)
  To: David Woodhouse; +Cc: Jeff Garzik, bjornw, linux-kernel, linux-mtd


David Woodhouse writes:
 > > What should it do on i386?  mb()? 
 > 
 > For it to have any use in the situation I described, it would need to 
 > writeback and invalidate the dcache for the affected range. It doesn't seem 
 > to do so, so it seems that it isn't what I require.

It only needs to do that on cpus where the cache is not consistent
with the rest of the system.  x86 caches are fully consistent with the
rest of the system, thus no flushing necessary.

Later,
David S. Miller
davem@redhat.com

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  2:03 ` David S. Miller
@ 2001-06-05  3:55   ` Chris Wedgwood
  2001-06-05  5:28     ` Linus Torvalds
                       ` (2 more replies)
  2001-06-05  4:01   ` David S. Miller
  1 sibling, 3 replies; 34+ messages in thread
From: Chris Wedgwood @ 2001-06-05  3:55 UTC (permalink / raw)
  To: David S. Miller
  Cc: Jeff Garzik, David Woodhouse, bjornw, linux-kernel, linux-mtd

On Mon, Jun 04, 2001 at 07:03:01PM -0700, David S. Miller wrote:

    The x86 doesn't have dumb caches, therefore it really doesn't
    need to flush anything.  Maybe a mb(), but that is it.

What if the memory is erased underneath the CPU being aware of this?
In such a way ig generates to bus traffic...



  --cw

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  2:03 ` David S. Miller
  2001-06-05  3:55   ` Chris Wedgwood
@ 2001-06-05  4:01   ` David S. Miller
  1 sibling, 0 replies; 34+ messages in thread
From: David S. Miller @ 2001-06-05  4:01 UTC (permalink / raw)
  To: Chris Wedgwood
  Cc: Jeff Garzik, David Woodhouse, bjornw, linux-kernel, linux-mtd


Chris Wedgwood writes:
 > On Mon, Jun 04, 2001 at 07:03:01PM -0700, David S. Miller wrote:
 > 
 >     The x86 doesn't have dumb caches, therefore it really doesn't
 >     need to flush anything.  Maybe a mb(), but that is it.
 > 
 > What if the memory is erased underneath the CPU being aware of this?
 > In such a way ig generates to bus traffic...

This doesn't happen on x86.  The processor snoops all transactions
done by other agents to/from main memory.  The processor caches are
always up to date.

Later,
David S. Miller
davem@redhat.com

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  3:55   ` Chris Wedgwood
@ 2001-06-05  5:28     ` Linus Torvalds
  2001-06-05  6:49       ` Alan Cox
  2001-06-05  8:29       ` Ingo Molnar
  2001-06-05  8:46     ` David Woodhouse
  2001-06-05  8:50     ` David S. Miller
  2 siblings, 2 replies; 34+ messages in thread
From: Linus Torvalds @ 2001-06-05  5:28 UTC (permalink / raw)
  To: linux-kernel

In article <20010605155550.C22741@metastasis.f00f.org>,
Chris Wedgwood  <cw@f00f.org> wrote:
>On Mon, Jun 04, 2001 at 07:03:01PM -0700, David S. Miller wrote:
>
>    The x86 doesn't have dumb caches, therefore it really doesn't
>    need to flush anything.  Maybe a mb(), but that is it.
>
>What if the memory is erased underneath the CPU being aware of this?
>In such a way ig generates to bus traffic...

Doing bank switching etc is outside the scope of the current DMA cache
flush macros - they are there only for "sane" cache coherency issues,
not to be used as generic "we have to flush the cache because we went
behind the back of the CPU and switched a bank of memory around". 

You will have to come up with some new primitive for this. 

The x86 has the "wbinval" instruction, although it should be noted that

 - it is buggy and will lock up some CPU's. Use with EXTREME CAUTION.
   Intel set a special field in the MP table for whether wbinval is
   usable or not, and you can probably find their errata on which CPU's
   it doesn't work on (I think it was some early PPro steppings).

   When wbinval doesn't work, there's another strategy to flush the
   cache, but I forget what it was. It was something ridiculous like
   reading in a few megabytes of memory from consecutive physical
   addresses to make sure that the cache has been replaced.

 - even when it works, it is necessarily very very very slow. Not to be
   used lightly. As you can imagine, the work-around is even slower.

On the whole, I would suggest avoiding this like the plague, and just
marking such memory to be non-cacheable, regardless of whether there is
a performance impact or not. If you mark it write-combining and
speculative, it's going to perform a bit better.

			Linus

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  5:28     ` Linus Torvalds
@ 2001-06-05  6:49       ` Alan Cox
  2001-06-05  8:29       ` Ingo Molnar
  1 sibling, 0 replies; 34+ messages in thread
From: Alan Cox @ 2001-06-05  6:49 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-kernel

>  - it is buggy and will lock up some CPU's. Use with EXTREME CAUTION.
>    Intel set a special field in the MP table for whether wbinval is
>    usable or not, and you can probably find their errata on which CPU's
>    it doesn't work on (I think it was some early PPro steppings).

Also some intel chipsets it didnt work with (notably earlier PCI ones) - indeed
that seems to be the cause of the 'bzImage wont boot on XYZ 486 tosh
laptop' bug.

> On the whole, I would suggest avoiding this like the plague, and just
> marking such memory to be non-cacheable, regardless of whether there is
> a performance impact or not. If you mark it write-combining and
> speculative, it's going to perform a bit better.

Or memcpy the blocks into the block or page cache when you need them. 


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  5:28     ` Linus Torvalds
  2001-06-05  6:49       ` Alan Cox
@ 2001-06-05  8:29       ` Ingo Molnar
  2001-06-05 15:10         ` Ralf Baechle
  1 sibling, 1 reply; 34+ messages in thread
From: Ingo Molnar @ 2001-06-05  8:29 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-kernel


On 4 Jun 2001, Linus Torvalds wrote:

>  - even when it works, it is necessarily very very very slow. Not to be
>    used lightly. As you can imagine, the work-around is even slower.

i've measured it once, IIRC it was around 10-15 millisecs on normal
pentiums, so while it's indeed the slowest x86 instruction on the planet,
it's still perhaps acceptable for hot-swapping ECC RAM.

	Ingo


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  3:55   ` Chris Wedgwood
  2001-06-05  5:28     ` Linus Torvalds
@ 2001-06-05  8:46     ` David Woodhouse
  2001-06-05  9:05       ` David Woodhouse
                         ` (3 more replies)
  2001-06-05  8:50     ` David S. Miller
  2 siblings, 4 replies; 34+ messages in thread
From: David Woodhouse @ 2001-06-05  8:46 UTC (permalink / raw)
  To: David S. Miller
  Cc: Chris Wedgwood, Jeff Garzik, bjornw, linux-kernel, linux-mtd


davem@redhat.com said:
> Chris Wedgwood writes:
> > What if the memory is erased underneath the CPU being aware of
> > this? In such a way ig generates to bus traffic...

> This doesn't happen on x86.  The processor snoops all transactions
> done by other agents to/from main memory.  The processor caches are
> always up to date. 

No. My original mail presented two situations in which this assumption is 
false.

1. Bank-switched RAM. You want to force a writeback before switching pages.
   I _guarantee_ you that the CPU isn't snooping my access to the I/O port
   which sets the latch that drives the upper few address bits of the RAM 
   chips.

2. Flash. A few writes of magic data to magic addresses and a whole erase
   block suddenly contains 0xFF. The CPU doesn't notice that either.

What I want is a function like simon_says_flush_page_to_ram(). In this 
case, I _do_ know better than the CPU. It is _not_ coherent with these 
devices.

I'm actually working on a MIPS box at the moment - the particular problems
with doing it on i386 don't interest me too much. To be honest, I could do
it by sticking asm instructions inside ifdefs in what is otherwise
arch-independent code. I'd rather not do it like that, though. 

Surely stuff like that should be exported by the arch-specific code or
include files somehow. Possibly with a #define or function I can use to tell
whether a selective flush is actually available on the current CPU. If it's
not possible to flush the dcache selectively, then the cost of doing a full
flush probably outweighs the benefit¹ of running the flash cached in the
first place. But it should still be possible to do it from arch-independent
code without manually inserting asm instructions to do it.

--
dwmw2

¹ The _assumed_ benefit, admittedly. I should get some benchmarks to back up
the comment about molasses in arch/cris/drivers/axisflashmap.c



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  3:55   ` Chris Wedgwood
  2001-06-05  5:28     ` Linus Torvalds
  2001-06-05  8:46     ` David Woodhouse
@ 2001-06-05  8:50     ` David S. Miller
  2001-06-05 17:01       ` Jamie Lokier
  2 siblings, 1 reply; 34+ messages in thread
From: David S. Miller @ 2001-06-05  8:50 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Chris Wedgwood, Jeff Garzik, bjornw, linux-kernel, linux-mtd


David Woodhouse writes:
 > What I want is a function like simon_says_flush_page_to_ram(). In
 > this case, I _do_ know better than the CPU. It is _not_ coherent
 > with these devices.

One way to do this, (even portably :-) is via displacement flushes.
Linus mentioned this.

Basically if you know the L2 cache size and the assosciativity you can
do this as long as you can get a "2 * L2 cache size * assosciativity"
piece of contiguous physical memory.  When you need this "simon says"
flush, you basically read this physical memory span and this will
guarentee that all dirty data has exited the L2 cache.

Later,
David S. Miller
davem@redhat.com

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  8:46     ` David Woodhouse
@ 2001-06-05  9:05       ` David Woodhouse
  2001-06-05 12:42         ` David Woodhouse
                           ` (2 more replies)
  2001-06-05  9:11       ` David S. Miller
                         ` (2 subsequent siblings)
  3 siblings, 3 replies; 34+ messages in thread
From: David Woodhouse @ 2001-06-05  9:05 UTC (permalink / raw)
  To: David S. Miller
  Cc: Chris Wedgwood, Jeff Garzik, bjornw, linux-kernel, linux-mtd


davem@redhat.com said:
>  One way to do this, (even portably :-) is via displacement flushes.
> Linus mentioned this.

> Basically if you know the L2 cache size and the assosciativity you can
> do this as long as you can get a "2 * L2 cache size * assosciativity"
> piece of contiguous physical memory.  When you need this "simon says"
> flush, you basically read this physical memory span and this will
> guarentee that all dirty data has exited the L2 cache. 

Fine. So it should be possible to do it on all architectures with 
physically-indexed caches - that's good. Architectures with 
virtually-indexed caches are going to have explicit cache management 
functionality anyway, presumably :)

Obviously the algorithm you describe should not be implemented in
arch-independent drivers. It should be in include/asm-*, for those
architectures which _can't_ do it with a single cache management instruction
(or loop of same).

What shall we call this function? The intuitive "flush_dcache_range" appears
to have already been taken.

--
dwmw2



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  8:46     ` David Woodhouse
  2001-06-05  9:05       ` David Woodhouse
@ 2001-06-05  9:11       ` David S. Miller
  2001-06-05  9:29       ` kira brown
  2001-06-05  9:43       ` Johan Adolfsson
  3 siblings, 0 replies; 34+ messages in thread
From: David S. Miller @ 2001-06-05  9:11 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Chris Wedgwood, Jeff Garzik, bjornw, linux-kernel, linux-mtd


David Woodhouse writes:
 > What shall we call this function? The intuitive "flush_dcache_range" appears
 > to have already been taken.

Call it flush_ecache_full() or something.

Many architectures need to implement this anyways if they have
parity error exception handling.  Most platforms sadly do not.
Sparc64 is one of the exceptions here...

Later,
David S. Miller
davem@redhat.com

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-04 23:16 Missing cache flush David Woodhouse
                   ` (3 preceding siblings ...)
  2001-06-05  2:04 ` David S. Miller
@ 2001-06-05  9:17 ` Bjorn Wesen
  2001-06-05 12:57   ` Chris Wedgwood
  2001-06-05 14:22   ` Vipin Malik
  4 siblings, 2 replies; 34+ messages in thread
From: Bjorn Wesen @ 2001-06-05  9:17 UTC (permalink / raw)
  To: David Woodhouse; +Cc: linux-kernel, linux-mtd

On Tue, 5 Jun 2001, David Woodhouse wrote:
> The flash mapping driver arch/cris/drivers/axisflashmap.c uses a cached
> mapping of the flash chips for bulk reads, but obviously an uncached mapping
> for sending commands and reading status when we're actually writing to or
> erasing parts of the chip.
> 
> However, it fails to flush the dcache for the range used when the flash is 
> accessed through the uncached mapping. So after an erase or write, we may 
> read old data from the cache for the changed area.

I'll start by saying that axisflashmap.c was not meant to be used by any
other archs, that's why it's in arch/cris. But if anyone find it useful,
that's great. Just be aware that it's not _designed_ for general use and
something like this might be just what that might mean.

CRIS is cache coherent just like the x86 cache and does not need any
explicit cache flushes for the write case. Even when doing cache bypass
writing, if a cacheline already exist with the referenced memory, the
cacheline is updated.

In the erase case though, yes there should be a flush. However during the
1-2 seconds it takes to erase a sector, you can with very high certainity
guarantee that the direct-mapped unified 8 kB cache on the CRIS is
flushed from any flash references at all.. I mean, it's one-way
associative, during 1-2 seconds it executes potentially 200 million
instructions. So we haven't really bothered to think about the problem..

For other CPU's it might be more dangerous, although I don't hold my
breath.. 1-2 seconds is a long time when talking about L1 caches.

> However, I can't see a cache operation which performs this function.
> flush_dcache_page() is defined as a NOP on CRIS as, it seems, it is on most
> architectures. On other architectures, there's dma_cache_wback_inv(), but
> that also seems to be a NOP on i386, to pick a random example.

I'd agree that to be really certain, a "flush_dcache()" function
should be implemented and used when an erase finishes. Like David Miller
wrote somewhere in the thread, one way is to use your knowledge of the
arch's cache and do suitable dummy accesses to flush it, if there is no
explicit command to do it. But that's just up to the arch coders..

-bw


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  8:46     ` David Woodhouse
  2001-06-05  9:05       ` David Woodhouse
  2001-06-05  9:11       ` David S. Miller
@ 2001-06-05  9:29       ` kira brown
  2001-06-05 17:16         ` Alan Cox
  2001-06-05  9:43       ` Johan Adolfsson
  3 siblings, 1 reply; 34+ messages in thread
From: kira brown @ 2001-06-05  9:29 UTC (permalink / raw)
  To: David Woodhouse
  Cc: David S. Miller, Chris Wedgwood, Jeff Garzik, bjornw,
	linux-kernel, linux-mtd



On Tue, 5 Jun 2001, David Woodhouse wrote:

>
> davem@redhat.com said:
> > Chris Wedgwood writes:
> > > What if the memory is erased underneath the CPU being aware of
> > > this? In such a way ig generates to bus traffic...
>
> > This doesn't happen on x86.  The processor snoops all transactions
> > done by other agents to/from main memory.  The processor caches are
> > always up to date.
>
> No. My original mail presented two situations in which this assumption is
> false.
>
> 1. Bank-switched RAM. You want to force a writeback before switching pages.
>    I _guarantee_ you that the CPU isn't snooping my access to the I/O port
>    which sets the latch that drives the upper few address bits of the RAM
>    chips.
>
> 2. Flash. A few writes of magic data to magic addresses and a whole erase
>    block suddenly contains 0xFF. The CPU doesn't notice that either.

3. Buggy implementations like the Cyrix 486es that don't properly maintain
   cache coherency.

kira.


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  8:46     ` David Woodhouse
                         ` (2 preceding siblings ...)
  2001-06-05  9:29       ` kira brown
@ 2001-06-05  9:43       ` Johan Adolfsson
  3 siblings, 0 replies; 34+ messages in thread
From: Johan Adolfsson @ 2001-06-05  9:43 UTC (permalink / raw)
  To: David Woodhouse, David S. Miller
  Cc: Chris Wedgwood, Jeff Garzik, bjorn.wesen, linux-kernel, linux-mtd

Possibly saying something extremly stupid here,
how about simply "fakewriting" 0xFF to the flash
after an erase to update any caches?

> 2. Flash. A few writes of magic data to magic addresses and a whole erase
>    block suddenly contains 0xFF. The CPU doesn't notice that either.

 do_erase_stuff();
 /* While verifying, update cache */
                for (address = adr; address < (adr + size); address++) {
                        if ((verify = map->read8(map, address)) != 0xFF) {
                                error = 1;
                                break;
                        }
                        /* "Fake" write 0xFF's to the erased sector so that
caches are updated
                         *  after we verified that uncached data is ok
                         */
                        *(unsigned char*)CACHED(address) = 0xFF;
                }

/Johan


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  9:05       ` David Woodhouse
@ 2001-06-05 12:42         ` David Woodhouse
  2001-06-05 12:52           ` David Woodhouse
  2001-06-05 12:48         ` David S. Miller
  2001-06-06 19:48         ` Pavel Machek
  2 siblings, 1 reply; 34+ messages in thread
From: David Woodhouse @ 2001-06-05 12:42 UTC (permalink / raw)
  To: David S. Miller
  Cc: Chris Wedgwood, Jeff Garzik, bjornw, linux-kernel, linux-mtd


davem@redhat.com said:
> David Woodhouse writes:
> > What shall we call this function? The intuitive "flush_dcache_range" 
> > appears to have already been taken.

> Call it flush_ecache_full() or something.

Strange name. Why? How about __flush_cache_range()?

--
dwmw2



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  9:05       ` David Woodhouse
  2001-06-05 12:42         ` David Woodhouse
@ 2001-06-05 12:48         ` David S. Miller
  2001-06-06  8:32           ` Albert D. Cahalan
  2001-06-06 19:48         ` Pavel Machek
  2 siblings, 1 reply; 34+ messages in thread
From: David S. Miller @ 2001-06-05 12:48 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Chris Wedgwood, Jeff Garzik, bjornw, linux-kernel, linux-mtd


David Woodhouse writes:
 > > Call it flush_ecache_full() or something.
 > 
 > Strange name. Why? How about __flush_cache_range()?

How about flush_cache_range_force() instead?

I want something in the name that tells the reader "this flushes the
caches, even though under every other ordinary circumstance you would
not need to".

Later,
David S. Miller
davem@redhat.com

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05 12:42         ` David Woodhouse
@ 2001-06-05 12:52           ` David Woodhouse
  0 siblings, 0 replies; 34+ messages in thread
From: David Woodhouse @ 2001-06-05 12:52 UTC (permalink / raw)
  To: David S. Miller
  Cc: Chris Wedgwood, Jeff Garzik, bjornw, linux-kernel, linux-mtd



davem@redhat.com said:
>  How about flush_cache_range_force() instead?
> I want something in the name that tells the reader "this flushes the
> caches, even though under every other ordinary circumstance you would
> not need to". 

OL, then. I would have thought it made more sense to have the
flush_dcache_range() unconditionally do what its name implies, and to have a
separate flush_dcache_range_for_dma() function which is optional. But that
decision was already made - I suppose we can't change the semantics now.

--
dwmw2



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  9:17 ` Bjorn Wesen
@ 2001-06-05 12:57   ` Chris Wedgwood
  2001-06-05 15:17     ` Ralf Baechle
  2001-06-05 17:41     ` Daniel Phillips
  2001-06-05 14:22   ` Vipin Malik
  1 sibling, 2 replies; 34+ messages in thread
From: Chris Wedgwood @ 2001-06-05 12:57 UTC (permalink / raw)
  To: Bjorn Wesen; +Cc: David Woodhouse, linux-kernel, linux-mtd

On Tue, Jun 05, 2001 at 11:17:48AM +0200, Bjorn Wesen wrote:

    In the erase case though, yes there should be a flush. However
    during the 1-2 seconds it takes to erase a sector, you can with
    very high certainity guarantee that the direct-mapped unified 8
    kB cache on the CRIS is flushed from any flash references at
    all.. I mean, it's one-way as\sociative, during 1-2 seconds it
    executes potentially 200 million instructions. So we haven't
    really bothered to think about the problem..
    
    For other CPU's it might be more dangerous, although I don't hold
    my breath.. 1-2 seconds is a long time when talking about L1
    caches.

I don't know about the CRIS (never heard of it, what is it?), but on
an Athlon when benchmarking stuff, I could still see L1 cache hits
from data that was 15 seconds old under certain work-loads (obviously
not gcc!). Does anyone know how old something may exisit in cache
before being written back to RAM?

Even though you potentially execute millions of instructions, they
are often the same ones over and over when the machine is near idle.



  --cw

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  9:17 ` Bjorn Wesen
  2001-06-05 12:57   ` Chris Wedgwood
@ 2001-06-05 14:22   ` Vipin Malik
  2001-06-05 14:40     ` Chris Wedgwood
  1 sibling, 1 reply; 34+ messages in thread
From: Vipin Malik @ 2001-06-05 14:22 UTC (permalink / raw)
  To: Bjorn Wesen; +Cc: David Woodhouse, linux-kernel, linux-mtd

Bjorn Wesen wrote:

>
> I'd agree that to be really certain, a "flush_dcache()" function
> should be implemented and used when an erase finishes. Like David Miller
> wrote somewhere in the thread, one way is to use your knowledge of the
> arch's cache and do suitable dummy accesses to flush it, if there is no
> explicit command to do it. But that's just up to the arch coders..
>

Here's a stupid question: Are there any processors out there that have a cache
but no explicit cache-flush command?

If not (i.e. no such "funny" processors), then what's wrong with the arch
dependent include through a define to execute the
arch specific asm command?

The only issue (besides knowing the cache size at run time) that I can think
about the "dummy" eviction scheme is that you now need to xfer potentially 3
times the cache
size data to and from memory:

#1. The dummy read
#2. The eviction of the entire cache data being evicted
#3. The refilling of the cache with good data again, as the dummy data cannot
really represent anything useful.

Is my thinking here completely non coherent with others? ;)

Vipin


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05 14:22   ` Vipin Malik
@ 2001-06-05 14:40     ` Chris Wedgwood
  0 siblings, 0 replies; 34+ messages in thread
From: Chris Wedgwood @ 2001-06-05 14:40 UTC (permalink / raw)
  To: Vipin Malik; +Cc: Bjorn Wesen, David Woodhouse, linux-kernel, linux-mtd

On Tue, Jun 05, 2001 at 09:22:13AM -0500, Vipin Malik wrote:

    Here's a stupid question: Are there any processors out there that
    have a cache but no explicit cache-flush command?

Yes, and as Linus pointed out, some with the ability are broken
anyhow.


  --cw

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  8:29       ` Ingo Molnar
@ 2001-06-05 15:10         ` Ralf Baechle
  0 siblings, 0 replies; 34+ messages in thread
From: Ralf Baechle @ 2001-06-05 15:10 UTC (permalink / raw)
  To: Ingo Molnar; +Cc: Linus Torvalds, linux-kernel

On Tue, Jun 05, 2001 at 10:29:28AM +0200, Ingo Molnar wrote:

> >  - even when it works, it is necessarily very very very slow. Not to be
> >    used lightly. As you can imagine, the work-around is even slower.
> 
> i've measured it once, IIRC it was around 10-15 millisecs on normal
> pentiums, so while it's indeed the slowest x86 instruction on the planet,
> it's still perhaps acceptable for hot-swapping ECC RAM.

I can already hear the realtime crowd bitch ...

  Ralf

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05 12:57   ` Chris Wedgwood
@ 2001-06-05 15:17     ` Ralf Baechle
  2001-06-05 17:41     ` Daniel Phillips
  1 sibling, 0 replies; 34+ messages in thread
From: Ralf Baechle @ 2001-06-05 15:17 UTC (permalink / raw)
  To: Chris Wedgwood; +Cc: Bjorn Wesen, David Woodhouse, linux-kernel, linux-mtd

On Wed, Jun 06, 2001 at 12:57:03AM +1200, Chris Wedgwood wrote:

> I don't know about the CRIS (never heard of it, what is it?), but on
> an Athlon when benchmarking stuff, I could still see L1 cache hits
> from data that was 15 seconds old under certain work-loads (obviously
> not gcc!). Does anyone know how old something may exisit in cache
> before being written back to RAM?

I know of no architecture that has a time limit that after which's expire
caches get written back to memory.  In other words cache lines may stay
dirty for an indefinate time if things are just right.

  Ralf

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  8:50     ` David S. Miller
@ 2001-06-05 17:01       ` Jamie Lokier
  2001-06-05 23:24         ` Chris Wedgwood
  0 siblings, 1 reply; 34+ messages in thread
From: Jamie Lokier @ 2001-06-05 17:01 UTC (permalink / raw)
  To: David S. Miller
  Cc: David Woodhouse, Chris Wedgwood, Jeff Garzik, bjornw,
	linux-kernel, linux-mtd

David S. Miller wrote:
> Basically if you know the L2 cache size and the assosciativity you can
> do this as long as you can get a "2 * L2 cache size * assosciativity"
> piece of contiguous physical memory.  When you need this "simon says"
> flush, you basically read this physical memory span and this will
> guarentee that all dirty data has exited the L2 cache.

Whether this works depends on the cache line replacement policy.  It
will always work with LRU, for example, and probably everything else
that exists.  But it is not guaranteed, is it?

-- Jamie

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  9:29       ` kira brown
@ 2001-06-05 17:16         ` Alan Cox
  2001-06-06 19:44           ` Earyly Cyrix CPUs was " Pavel Machek
  0 siblings, 1 reply; 34+ messages in thread
From: Alan Cox @ 2001-06-05 17:16 UTC (permalink / raw)
  To: kira brown
  Cc: David Woodhouse, David S. Miller, Chris Wedgwood, Jeff Garzik,
	bjornw, linux-kernel, linux-mtd

> 3. Buggy implementations like the Cyrix 486es that don't properly maintain
>    cache coherency.

The early Cyrix CPUs (and some 1.x rev 6x86 cpus I believe too) arent supported
and dont work. The problem is best corrected with a hammer and a visit to
ebay


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05 12:57   ` Chris Wedgwood
  2001-06-05 15:17     ` Ralf Baechle
@ 2001-06-05 17:41     ` Daniel Phillips
  1 sibling, 0 replies; 34+ messages in thread
From: Daniel Phillips @ 2001-06-05 17:41 UTC (permalink / raw)
  To: Chris Wedgwood; +Cc: linux-kernel

On Tuesday 05 June 2001 14:57, Chris Wedgwood wrote:
> I don't know about the CRIS (never heard of it, what is it?)

I wondered about that too.  From Documentation/cris:

What is CRIS ?
-------------- 
CRIS is an acronym for 'Code Reduced Instruction Set'. It is the CPU 
architecture in Axis Communication AB's range of embeddedn etwork 
CPU's, called ETRAX. The latest CPU is called

ETRAX 100LX, where LX stands for 'Linux' because the chip was designed 
to be a good host for the Linux operating system. 

Interesting, huh?

--
Daniel

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05 17:01       ` Jamie Lokier
@ 2001-06-05 23:24         ` Chris Wedgwood
  2001-06-06  8:43           ` James Sutherland
  0 siblings, 1 reply; 34+ messages in thread
From: Chris Wedgwood @ 2001-06-05 23:24 UTC (permalink / raw)
  To: Jamie Lokier
  Cc: David S. Miller, David Woodhouse, Jeff Garzik, bjornw,
	linux-kernel, linux-mtd

On Tue, Jun 05, 2001 at 07:01:28PM +0200, Jamie Lokier wrote:

    Whether this works depends on the cache line replacement policy. 
    It will always work with LRU, for example, and probably
    everything else that exists.  But it is not guaranteed, is it?

Getting way OT here... didn't the K6 or something have the ability to
lock cache lines?


  --cw

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05 12:48         ` David S. Miller
@ 2001-06-06  8:32           ` Albert D. Cahalan
  0 siblings, 0 replies; 34+ messages in thread
From: Albert D. Cahalan @ 2001-06-06  8:32 UTC (permalink / raw)
  To: David S. Miller
  Cc: David Woodhouse, Chris Wedgwood, Jeff Garzik, bjornw,
	linux-kernel, linux-mtd

David S. Miller writes:
> David Woodhouse writes:

>>> Call it flush_ecache_full() or something.
>>
>> Strange name. Why? How about __flush_cache_range()?
>
> How about flush_cache_range_force() instead?
>
> I want something in the name that tells the reader "this flushes
> the caches, even though under every other ordinary circumstance
> you would not need to".

"flush" means what to you?

write-back
write-back-and-invalidate
discard-and-invalidate

All 3 behaviors are useful to me, and a few more. I've been
using chunks of PowerPC assembly. Using PowerPC mnemonics...

dcba -- allocate a cache block with undefined content
dcbf -- write to RAM, then invalidate ("data cache block flush")
dcbi -- invalidate, discarding any data
dcbst -- initiate write if dirty
dcbt -- prefetch, hinting about future load instructions
dcbtst -- prefetch, hinting about future store instructions
dcbz -- allocate and zero a cache block (cacheable mem only!)

So dcbf_range() and dcbi_range() sound good to me. :-)

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05 23:24         ` Chris Wedgwood
@ 2001-06-06  8:43           ` James Sutherland
  0 siblings, 0 replies; 34+ messages in thread
From: James Sutherland @ 2001-06-06  8:43 UTC (permalink / raw)
  To: Chris Wedgwood
  Cc: Jamie Lokier, David S. Miller, David Woodhouse, Jeff Garzik,
	bjornw, linux-kernel, linux-mtd

> On Tue, Jun 05, 2001 at 07:01:28PM +0200, Jamie Lokier wrote:
>
>     Whether this works depends on the cache line replacement policy.
>     It will always work with LRU, for example, and probably
>     everything else that exists.  But it is not guaranteed, is it?

LFU, if used, wouldn't be flushed by this trick... Potentially, you could
have a slice of kernel interrupt handler sitting in a cache line, having
been accessed $BIG_NUMBER times... (Does anything actually use this
approach, though???)


James.


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Earyly Cyrix CPUs was Re: Missing cache flush.
  2001-06-05 17:16         ` Alan Cox
@ 2001-06-06 19:44           ` Pavel Machek
  2001-06-08 16:00             ` Alan Cox
  0 siblings, 1 reply; 34+ messages in thread
From: Pavel Machek @ 2001-06-06 19:44 UTC (permalink / raw)
  To: Alan Cox
  Cc: kira brown, David Woodhouse, David S. Miller, Chris Wedgwood,
	Jeff Garzik, bjornw, linux-kernel, linux-mtd

Hi1

> > 3. Buggy implementations like the Cyrix 486es that don't properly maintain
> >    cache coherency.
> 
> The early Cyrix CPUs (and some 1.x rev 6x86 cpus I believe too) arent supported
> and dont work. The problem is best corrected with a hammer and a visit to
> ebay

What is list of cpu's not supported? I want one ;-).

[It is even more broken than 386? Wow!]

[Is it really not work-aroundable? LIke declaring no DMA on system and using
PIO for floppy?]
								Pavel
-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Missing cache flush.
  2001-06-05  9:05       ` David Woodhouse
  2001-06-05 12:42         ` David Woodhouse
  2001-06-05 12:48         ` David S. Miller
@ 2001-06-06 19:48         ` Pavel Machek
  2 siblings, 0 replies; 34+ messages in thread
From: Pavel Machek @ 2001-06-06 19:48 UTC (permalink / raw)
  To: David Woodhouse
  Cc: David S. Miller, Chris Wedgwood, Jeff Garzik, bjornw,
	linux-kernel, linux-mtd

Hi!

> What shall we call this function? The intuitive "flush_dcache_range" appears
> to have already been taken.


Please do not use *_dcache_*. It would confuse me with dentry cache.

Flush data cche range would be ok.

[BTW what about just rewriting it with 0xffs after you zero the block?
Someone already suggested that, and it seems ok.]
-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Earyly Cyrix CPUs was Re: Missing cache flush.
  2001-06-06 19:44           ` Earyly Cyrix CPUs was " Pavel Machek
@ 2001-06-08 16:00             ` Alan Cox
  0 siblings, 0 replies; 34+ messages in thread
From: Alan Cox @ 2001-06-08 16:00 UTC (permalink / raw)
  To: Pavel Machek
  Cc: Alan Cox, kira brown, David Woodhouse, David S. Miller,
	Chris Wedgwood, Jeff Garzik, bjornw, linux-kernel, linux-mtd

> What is list of cpu's not supported? I want one ;-).
> [It is even more broken than 386? Wow!]

Some of the early Cyrix cpus get cache corruption. I dont know of any work
around. These are the CPU's that won't run some versions of gcc (optimised
egcs 1.1.2 segfaulted at random non optimised worked, both work on other
cpus)



^ permalink raw reply	[flat|nested] 34+ messages in thread

end of thread, other threads:[~2001-06-08 16:04 UTC | newest]

Thread overview: 34+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2001-06-04 23:16 Missing cache flush David Woodhouse
2001-06-04 23:23 ` Jeff Garzik
2001-06-04 23:29 ` David Woodhouse
2001-06-05  2:03 ` David S. Miller
2001-06-05  3:55   ` Chris Wedgwood
2001-06-05  5:28     ` Linus Torvalds
2001-06-05  6:49       ` Alan Cox
2001-06-05  8:29       ` Ingo Molnar
2001-06-05 15:10         ` Ralf Baechle
2001-06-05  8:46     ` David Woodhouse
2001-06-05  9:05       ` David Woodhouse
2001-06-05 12:42         ` David Woodhouse
2001-06-05 12:52           ` David Woodhouse
2001-06-05 12:48         ` David S. Miller
2001-06-06  8:32           ` Albert D. Cahalan
2001-06-06 19:48         ` Pavel Machek
2001-06-05  9:11       ` David S. Miller
2001-06-05  9:29       ` kira brown
2001-06-05 17:16         ` Alan Cox
2001-06-06 19:44           ` Earyly Cyrix CPUs was " Pavel Machek
2001-06-08 16:00             ` Alan Cox
2001-06-05  9:43       ` Johan Adolfsson
2001-06-05  8:50     ` David S. Miller
2001-06-05 17:01       ` Jamie Lokier
2001-06-05 23:24         ` Chris Wedgwood
2001-06-06  8:43           ` James Sutherland
2001-06-05  4:01   ` David S. Miller
2001-06-05  2:04 ` David S. Miller
2001-06-05  9:17 ` Bjorn Wesen
2001-06-05 12:57   ` Chris Wedgwood
2001-06-05 15:17     ` Ralf Baechle
2001-06-05 17:41     ` Daniel Phillips
2001-06-05 14:22   ` Vipin Malik
2001-06-05 14:40     ` Chris Wedgwood

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).