Re: [PATCH] O_STREAMING - flag for optimal streaming I/O

[Helge Hafting <helgehaf@aitel.hist.no>]

Mike Fedyk wrote:
> 
> On Thu, Oct 10, 2002 at 03:39:19PM +0200, Giuliano Pochini wrote:
[...]
> > When a process opens a file with O_STREAMING, it tells the kernel
> > it will use the data only once, but it tells nothing about other
> > tasks. If that process reads something which is already cached,
> > then it must not drop it because someone other used it recently
> > and IMHO pagecache only should be allowed to drop it.
> >
> 
> You are missing the point.  If the app thinks that might happen, it
> shouldn't use O_STREAMING.

The app _can't_ know, so nothing should use O_STREAMING?

I think the idea seems good - if a page requested for streaming
happens to be in cache already, don't mark it for early eviction.

This approach ensures that streaming apps don't affect the caches
of other apps at all.  There are way too many cases where
streaming is useful but we don't know if others might be using
the pages - perhaps in other ways.

Consider searching _all_ files on disk for some string.  Clearly
a cache-killer that would benefit from streaming, without
streaming this pushes everything else from cache except
for the last files searched.
But streaming that unconditionally marks pages for eviction
will also kill all cache in this case.  Both data and binaries.

Streaming that only evicts pages it _brought in_ can 
search the entire disk (or some reasonable but large subset)
leaving almost all other cache intact.

Now, if we could make updatedb use this kind of streaming
for its directory traversals... :-)

Helge Hafting