git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "brian m. carlson" <sandals@crustytoothpaste.net>
To: Bagas Sanjaya <bagasdotme@gmail.com>
Cc: Bryan Turner <bturner@atlassian.com>, git@vger.kernel.org
Subject: Re: Argument list too long when fetching many missing objects from partial clone
Date: Wed, 3 Mar 2021 00:42:27 +0000	[thread overview]
Message-ID: <YD7bczBsIR5rkqfc@camp.crustytoothpaste.net> (raw)
In-Reply-To: <f185e87e-d8d2-3ac4-9ca1-7b96ffb30562@gmail.com>

[-- Attachment #1: Type: text/plain, Size: 1552 bytes --]

[Note: we generally prefer inline responses.]

On 2021-03-02 at 06:55:27, Bagas Sanjaya wrote:
> After trying your suggestion, I now get:
> 
> error: RPC failed; HTTP 413 curl 22 The requested URL returned error: 413
> fatal: unable to write request to remote: Broken pipe

Hmmm, this means that your local Git server is unhappy with the size of
the request.  That means your request is probably very large, and it's
likely that it is kindly asking you not to DoS it.

> On 02/03/21 02.22, Bryan Turner wrote:
> > On Mon, Mar 1, 2021 at 5:20 AM Bagas Sanjaya <bagasdotme@gmail.com> wrote:
> > > Inside the partial clone, I tried to convert it to full clone. First, I gather
> > > list of missing objects:
> > > 
> > >      git rev-list --objects --all --missing=print | grep -oP '^\?\K\w+' > .git/missing.list
> > > 
> > > Then I fetched those:
> > > 
> > >      git fetch origin $(cat .git/missing.list)
> > 
> > Since you're on Git 2.30, instead of trying to place all the object
> > IDs on the `git fetch` command line, have you tried adding `--stdin`
> > and having it read from your `missing.list` directly? Something like
> > `git fetch --stdin origin <.git/missing.list` might do what you need.

You may want to try this:

  xargs git fetch origin < .git/missing.list

That will batch the requests to fit on the command line, which will
probably also have the pleasant side effect that it makes each
individual request smaller to avoid the 413.
-- 
brian m. carlson (he/him or they/them)
Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

      reply	other threads:[~2021-03-03  6:42 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-01 13:18 Bagas Sanjaya
2021-03-01 19:21 ` Jeff Hostetler
2021-03-01 19:22 ` Bryan Turner
2021-03-02  6:55   ` Bagas Sanjaya
2021-03-03  0:42     ` brian m. carlson [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YD7bczBsIR5rkqfc@camp.crustytoothpaste.net \
    --to=sandals@crustytoothpaste.net \
    --cc=bagasdotme@gmail.com \
    --cc=bturner@atlassian.com \
    --cc=git@vger.kernel.org \
    --subject='Re: Argument list too long when fetching many missing objects from partial clone' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
on how to clone and mirror all data and code used for this inbox