On Sun, 26 Jun 2005 17:35:48 CDT, David Masover said:

> > Right. So please explain what crypto/raw/foo and crypto/inflated/foo.gz give you.
> 
> In that example (shouldn't have used the name "crypto", but oh well), it
> should be crypto/raw/foo.gz and crypto/inflated/foo -- where foo.gz is
> the gzip'ed file and foo is the transparently compressed/decompressed
> file.  Basically, these are equivalent:
> 
> $ zcat crypto/raw/foo.gz
> $ cat crypto/inflated/foo

I'm *quite* aware of what your preconceived notions think it *should* be.

Maybe the two examples I asked for have *real-world* meanings that you should
be allowing for.  Like, for instance, on a mail server, where the A/V software
may need to unzip a file 5 or 6 times to find out if there's malicious content.

Or seeing if it's a ".zip bomb", where a small .zip will decompress to gigabytes.

Or I'm testing a new compression algorithm, to see if multiple compressions help
(yes, I know that it *shouldn't* help - but I've seen real-world cases where the
algorithm could only look at a 4K or 8K window at a time, and if you hit a *very*
long run of duplicate 4K segments, a second compression would compress all the
identical or near-identical "this is a 4K chunk" tokens...)


> > It's got a *LOT* to do with it if I created a *DIRECTORY*, to use *AS A DIRECTORY*,
> > the way Unix-style systems have done for 3 decades, and suddenly my system is
> > running like a pig because the kernel decided that it's a .zip file.
> 
> The kernel does not decide that.  You do.  And it doesn't automatically
> decide that every time you create a file.  You have to use some
> interface to trigger the plugins.

Oh, I'm waiting for the fun the first time somebody deploys a plugin that
has similar semantics to 'chmod g+s dirname/' ;)

> I guess I need a new name for this approach.  That's three possible ways
> of doing this?

I *said* you need to think this through in detail, didn't I? ;)
 
> I remember discussing that, actually.  It wouldn't automatically do this
> if you didn't want it to, but it would be nice if, say, it was something
> truly seekable like linux-2.6.12.zip, and linux-2.6.12 was a
> user-created symlink to linux-2.6.12.zip/.../contents, and we had a nice
> caching system...

I think you're highly deluded as to just how much or little performance gain
this will get you. Model what happens with a kernel 'make' on a 256M machine
with and without all that zipping and compressing, and assume that a constant
48M is available for caching of the linux-2.6.12/ tree.

> This is nice because then you get exactly the same performance during
> "make" as you would with "unzip && make", only better, because files you
> don't ever use (lots of arch, for instance) are not unpacked.

Go read http://www.tux.org/lkml/#s7-7 and ponder until enlightenment arrives.