Re: [Buildroot] [PATCH 1/1] download/git: support Git LFS

From: Arnout Vandecappelle <arnout@mind.be>
To: Vincent Fazio <vfazio@xes-inc.com>, buildroot@buildroot.org
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>,
	John Keeping <john@metanate.com>,
	"Yann E . MORIN" <yann.morin.1998@free.fr>,
	Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Subject: Re: [Buildroot] [PATCH 1/1] download/git: support Git LFS
Date: Thu, 6 Jan 2022 11:35:21 +0100	[thread overview]
Message-ID: <d28ff21d-c2bf-f489-2d09-9f01936f1337@mind.be> (raw)
In-Reply-To: <20200121203337.30856-1-vfazio@xes-inc.com>

  Hi John, Vincent,

  I finally applied this to master, with a few minor changes.

On 21/01/2020 21:33, Vincent Fazio wrote:
> From: John Keeping <john@metanate.com>
> 
> Git Large File Storage replaces large files with text pointers in the
> Git repository while storing the contents on a remote server.  If a
> repository is using this extension, then git-lfs must be used to
> checkout the large files before the source archive is generated.
> 
> Signed-off-by: John Keeping <john@metanate.com>
> [vfazio:
>    - add git-lfs to DL_TOOLS_DEPENDENCIES
>    - fixup for 5a0d6813948fea2cdb88a2e35984520eec856dec
>      ("infra/pkg-download: make the DOWNLOAD macro fully parameterised")
> ]
> Signed-off-by: Vincent Fazio <vfazio@xes-inc.com>

[snip]

> +# If there are large files then fetch them.
> +if [ ${large_file} -eq 1 ]; then
> +    _git lfs install

  AFAICS an lfs install is not needed. IMHO it's pretty bad to do that because 
it updates ~/.gitconfig, which Buildroot shouldn't do. We could do lfs install 
--local, but even that is not needed. The lfs install just makes sure that lfs 
is done automatically whenever you do commit or pull or something like that, but 
in Buildroot we always do it explicitly so the smudge filter isn't needed.

> +    _git lfs fetch
> +    _git lfs checkout

  As already mentioned by Yann, this doesn't work when combined with submodules. 
The solution is pretty trivial:

     # If there are also submodules, recurse into them,
     if [ ${recurse} -eq 1 ]; then
         _git submodule foreach --recursive ${GIT} lfs fetch
         _git submodule foreach --recursive ${GIT} lfs checkout
     fi

  I haven't actually tested this though.

  Vincent (or anybody), it would be nice to have a runtime test for this 
feature. 
support/testing/tests/download/br2-external/git-refs/package/git-submodule-{enabled,disabled} 
can be a good source of inspiration. Only I'm not sure if it's possible to set 
up an LFS repository that is purely file-based.

  Regards,
  Arnout

> +fi
> +
>   # Generate the archive, sort with the C locale so that it is reproducible.
>   # We do not want the .git dir; we keep other .git files, in case they are the
>   # only files in their directory.
> 
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot