[RFC/PATCH 0/7] user-configurable git-archive output formats

[Jeff King <peff@github.com>]

On Tue, Jun 14, 2011 at 04:45:21PM -0400, Jeff King wrote:

> The gzip path is not configurable at all. Probably it should read the
> path and arguments from the config file. In fact, we could even allow
> arbitrary config like:
> 
>   [tarfilter "tgz"]
>     command = gzip -c
>     extension = tgz
>     extension = tar.gz

Here's a series implementing that. You can configure whatever you want,
and it includes builtin gzip configuration by default. You can override
to turn it off, or even switch it to run something like pigz instead.

My biggest reservation with the patches as-is is that they are very
tar-centric and not orthogonal. Specifically, they won't handle:

  1. Other streamable archive formats you would want to pipe through
     compressors. Do any of these actually exist? I guess we could offer
     "pax" as a format eventually, and it might be like tar with
     different defaults? I dunno.

     Fixing this would not be too hard. Instead of these being
     "tarfilters", they would be "archive filters", and they would chain
     to some format, defaulting to "tar".  Since there is no other
     format right now, we could even punt on writing most of the code
     until somebody adds one. But we would want to get the naming of the
     config options right, since those are user-facing. Maybe
     "archivefilter" (unfortunately the more readable archive.filter is
     a little awkward with the way we parse config files)?

  2. In theory you might want to plug in external helpers that are not
     just stream filters, but actually their own container formats (like
     zip). I think people who want 7zip would want this.

     But how does git-archive interact with the helper? By definition
     the data it wants is the set of files, not a single stream. So
     either:

       a. We give the helper a temporary exported checkout, and it
          generates the stream from that.

       b. We use tar as the lingua franca of streaming file containers,
          and let the helper deal with converting to its preferred
          output format.

      Option (a) seems horribly inefficient on disk I/O. And if we did
      want to do that, I think it's largely unrelated to this patch
      series.

      You can actually do option (b) with this series. In its worst
      case, you can do the same as (a): just untar into a temporary
      directory and compress from there. But a well-written helper could
      convert tar into the output format on the fly.

The patches are:

  [1/7]: archive: reorder option parsing and config reading
  [2/7]: archive: add user-configurable tar-filter infrastructure
  [3/7]: archive: support user tar-filters via --format
  [4/7]: archive: advertise user tar-filters in --list
  [5/7]: archive: refactor format-guessing from filename
  [6/7]: archive: match extensions from user-configured formats
  [7/7]: archive: provide builtin .tar.gz filter

-Peff