Re: [PATCH v5 5/5] convert: add "status=delayed" to filter process protocol

[Junio C Hamano <gitster@pobox.com>]

Lars Schneider <larsxschneider@gmail.com> writes:

> +Delay
> +^^^^^
> +
> +If the filter supports the "delay" capability, then Git can send the
> +flag "can-delay" after the filter command and pathname. This flag
> +denotes that the filter can delay filtering the current blob (e.g. to
> +compensate network latencies) by responding with no content but with
> +the status "delayed" and a flush packet.
> +------------------------
> +packet:          git> command=smudge
> +packet:          git> pathname=path/testfile.dat
> +packet:          git> can-delay=1
> +packet:          git> 0000
> +packet:          git> CONTENT
> +packet:          git> 0000
> +packet:          git< status=delayed
> +packet:          git< 0000
> +------------------------
> +
> +If the filter supports the "delay" capability then it must support the
> +"list_available_blobs" command. If Git sends this command, then the
> +filter is expected to return a list of pathnames of blobs that are
> +available. The list must be terminated with a flush packet followed
> +by a "success" status that is also terminated with a flush packet. If
> +no blobs for the delayed paths are available, yet, then the filter is
> +expected to block the response until at least one blob becomes
> +available. The filter can tell Git that it has no more delayed blobs
> +by sending an empty list.
> +------------------------
> +packet:          git> command=list_available_blobs
> +packet:          git> 0000
> +packet:          git< pathname=path/testfile.dat
> +packet:          git< pathname=path/otherfile.dat
> +packet:          git< 0000
> +packet:          git< status=success
> +packet:          git< 0000
> +------------------------
> +
> +After Git received the pathnames, it will request the corresponding
> +blobs again. These requests contain a pathname and an empty content
> +section. The filter is expected to respond with the smudged content
> +in the usual way as explained above.
> +------------------------
> +packet:          git> command=smudge
> +packet:          git> pathname=path/testfile.dat
> +packet:          git> 0000
> +packet:          git> 0000  # empty content!
> +packet:          git< status=success
> +packet:          git< 0000
> +packet:          git< SMUDGED_CONTENT
> +packet:          git< 0000
> +packet:          git< 0000  # empty list, keep "status=success" unchanged!
> +------------------------

Random things that come to mind (which I suspect has been dealt with
in code but not described in the above in detail):

 * Using pathname as a "delay key" would mean that we assume that a
   single helper process is spawned to serve only a single Git
   invocation (as opposed to be sitting as a daemon accepting
   requests from instances of Git invoked much later than the daemon
   started), which is OK, but it is somewhat unclear when a filter
   is allowed to discard the requests that used to be outstanding
   from the write-up.  Imagine Git asked for A and B, both of which
   was delayed, and then asked for a list of available ones and
   learned that A is now ready.  

   - Is it an error to ask for B at this point, and if so how is the
     error handled?  Or will it turn into a blocking operation?

   - As A is available, Git can ask for it.  After retrieving
     smudged content for A, can Git ask for it again?  Or is it an
     error to do so, without starting over with another can-delay=1
     request for the same path?

 * The pathname does not have to be encoded/quoted in any way as it
   is just a payload on the tail end of a packet line, so I am
   guessing it is just a sequence raw bytes.  It is somewhat unclear
   in the above write-up.  Also, is this considered to be a part of
   "textual" exchange over the packet-line protocol, where it is
   customary to remove the trailing LF from the packet?  "We do not
   support a file whose name ends with LF" may or may not be an
   acceptable stance (I am personally OK, but some people who use
   Git may not), but it needs to be documented if there is such an
   issue.

 * The continued request throws an empty content in the above
   illustration.  Is a filter allowed/encouraged to assume that an
   empty content in the request is a continuation?  I am guessing
   that the answer is NO (otherwise you cannot filter an empty
   file), and it somehow need to be documented, perhaps?