archive mirror
 help / color / mirror / Atom feed
From: Linus Torvalds <>
To: "C. Scott Ananian" <>
Subject: Re: space compression (again)
Date: Fri, 15 Apr 2005 11:34:19 -0700 (PDT)	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On Fri, 15 Apr 2005, C. Scott Ananian wrote:
> Why are blobs per-file?  [After all, Linus insists that files are an 
> illusion.]  Why not just have 'chunks', and assemble *these* 
> into blobs (read, 'files')?  A good chunk size would fit evenly into some 
> number of disk blocks (no wasted space!).

I actually considered that. I ended up not doing it, because it's not 
obvious how to "block" things up (and even more so because while I like 
the notion, it flies in the face of the other issues I had: performance 
and simplicity).

The problem with chunking is:
 - it complicates a lot of the routines. Things like "is this file 
   unchanged" suddenly become "is this file still the same set of chunks",
   which is just a _lot_ more code and a lot more likely to have bugs.
 - you have to find a blocking factor. I thought of just going it fixed 
   chunks, and that just doesn't help at all. 
 - we already have wasted space due to the low-level filesystem (as 
   opposed to "git") usually being block-based, which means that space 
   utilization for small objects tends to suck. So you really want to 
   prefer objects that are several kB (compressed), and a small block just
   wastes tons of space.
 - there _is_ a natural blocking factor already. That's what a file 
   boundary really is within the project, and finding any other is really 
   quite hard.

So I'm personally 100% sure that it's not worth it. But I'm not opposed to
the _concept_: it makes total sense in the "filesystem" view, and is 100%
equivalent to having an inode with pointers to blocks. I just don't think 
the concept plays out well in reality.


  reply	other threads:[~2005-04-15 18:30 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2005-04-15 17:19 space compression (again) C. Scott Ananian
2005-04-15 18:34 ` Linus Torvalds [this message]
2005-04-15 18:45   ` C. Scott Ananian
2005-04-15 19:00     ` Derek Fawcus
2005-04-15 19:11     ` Linus Torvalds
2005-04-16 14:39       ` Martin Uecker
2005-04-16 15:11         ` C. Scott Ananian
2005-04-16 17:37           ` Martin Uecker
2005-04-19 12:39             ` Martin Uecker
2005-04-15 18:50 ` Derek Fawcus
2005-04-15 19:33 Ray Heasman
2005-04-16 12:29 ` David Lang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).