[PATCH 1/3] git.py: skip smudging if lfs=0 is set

[PATCH 1/3] git.py: skip smudging if lfs=0 is set

[Mauro Queirós]

Git-LFS objects were being fetched even when lfs=0 was not set.
This patch disables LFS smudging when lfs=0. That way, only the LFS pointers
are downloaded during checkout.

Signed-off-by: Mauro Queiros <maurofrqueiros@gmail.com>
---
 lib/bb/fetch2/git.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
index 5b3793a7..4c7d388e 100644
--- a/lib/bb/fetch2/git.py
+++ b/lib/bb/fetch2/git.py
@@ -475,6 +475,9 @@ class Git(FetchMethod):
 
         need_lfs = ud.parm.get("lfs", "1") == "1"
 
+        if not need_lfs:
+            ud.basecmd = "GIT_LFS_SKIP_SMUDGE=1 " + ud.basecmd
+
         source_found = False
         source_error = []
 
-- 
2.17.1

[PATCH 2/3] git.py: LFS bitbake note should not be printed if need_lfs is not set.

[Mauro Queirós]

The message "Repository %s has LFS content but it is not being fetched" was
being printed, even when Git-LFS was available and "lfs=1" was set. In those
situations, we want to fetch LFS content, so that message would not make sense.

Signed-off-by: Mauro Queiros <maurofrqueiros@gmail.com>
---
 lib/bb/fetch2/git.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
index 4c7d388e..eab76a10 100644
--- a/lib/bb/fetch2/git.py
+++ b/lib/bb/fetch2/git.py
@@ -509,7 +509,7 @@ class Git(FetchMethod):
         if self._contains_lfs(ud, d, destdir):
             if need_lfs and not self._find_git_lfs(d):
                 raise bb.fetch2.FetchError("Repository %s has LFS content, install git-lfs on host to download (or set lfs=0 to ignore it)" % (repourl))
-            else:
+            elif not need_lfs:
                 bb.note("Repository %s has LFS content but it is not being fetched" % (repourl))
 
         if not ud.nocheckout:
-- 
2.17.1

[PATCH 3/3] git.py: Use the correct branch to check if the repository has LFS objects.

[Mauro Queirós]

Function "contains_lfs" was only looking at the master branch when searching for LFS
content. LFS may be configured in specific branches only, so we need to use the
correct branch.

Signed-off-by: Mauro Queiros <maurofrqueiros@gmail.com>
---
 lib/bb/fetch2/git.py | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
index eab76a10..b7841c43 100644
--- a/lib/bb/fetch2/git.py
+++ b/lib/bb/fetch2/git.py
@@ -566,8 +566,15 @@ class Git(FetchMethod):
         """
         Check if the repository has 'lfs' (large file) content
         """
-        cmd = "%s grep lfs HEAD:.gitattributes | wc -l" % (
-                ud.basecmd)
+
+        if not ud.nobranch:
+            branchname = ud.branches[ud.names[0]]
+        else:
+            branchname = "master"
+
+        cmd = "%s show %s:.lfsconfig" % (
+                ud.basecmd, branchname)
+
         try:
             output = runfetchcmd(cmd, d, quiet=True, workdir=wd)
             if int(output) > 0:
-- 
2.17.1

Re: [bitbake-devel] [PATCH 1/3] git.py: skip smudging if lfs=0 is set

[Richard Purdie]

On Thu, 2020-05-28 at 15:18 +0100, Mauro Queirós wrote:
> Git-LFS objects were being fetched even when lfs=0 was not set.
> This patch disables LFS smudging when lfs=0. That way, only the LFS
> pointers
> are downloaded during checkout.
> 
> Signed-off-by: Mauro Queiros <maurofrqueiros@gmail.com>
> ---
>  lib/bb/fetch2/git.py | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
> index 5b3793a7..4c7d388e 100644
> --- a/lib/bb/fetch2/git.py
> +++ b/lib/bb/fetch2/git.py
> @@ -475,6 +475,9 @@ class Git(FetchMethod):
>  
>          need_lfs = ud.parm.get("lfs", "1") == "1"
>  
> +        if not need_lfs:
> +            ud.basecmd = "GIT_LFS_SKIP_SMUDGE=1 " + ud.basecmd
> +
>          source_found = False
>          source_error = []

Thanks for the patches, the sound good. Unfortunately they cause
bitbake-selftest to fail:

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/991/steps/8/logs/step1d

Cheers,

Richard

Re: [bitbake-devel] [PATCH 1/3] git.py: skip smudging if lfs=0 is set

[Mauro Queirós]

[-- Attachment #1: Type: text/plain, Size: 1345 bytes --]

That's because I was using .lfsconfig instead of .gitattributes in
"contains_lfs" function. self-tests only use the latter.
I've send v2 patches.

Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia sexta,
29/05/2020 à(s) 06:46:

> On Thu, 2020-05-28 at 15:18 +0100, Mauro Queirós wrote:
> > Git-LFS objects were being fetched even when lfs=0 was not set.
> > This patch disables LFS smudging when lfs=0. That way, only the LFS
> > pointers
> > are downloaded during checkout.
> >
> > Signed-off-by: Mauro Queiros <maurofrqueiros@gmail.com>
> > ---
> >  lib/bb/fetch2/git.py | 3 +++
> >  1 file changed, 3 insertions(+)
> >
> > diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
> > index 5b3793a7..4c7d388e 100644
> > --- a/lib/bb/fetch2/git.py
> > +++ b/lib/bb/fetch2/git.py
> > @@ -475,6 +475,9 @@ class Git(FetchMethod):
> >
> >          need_lfs = ud.parm.get("lfs", "1") == "1"
> >
> > +        if not need_lfs:
> > +            ud.basecmd = "GIT_LFS_SKIP_SMUDGE=1 " + ud.basecmd
> > +
> >          source_found = False
> >          source_error = []
>
> Thanks for the patches, the sound good. Unfortunately they cause
> bitbake-selftest to fail:
>
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/991/steps/8/logs/step1d
>
> Cheers,
>
> Richard
>
>

[-- Attachment #2: Type: text/html, Size: 2074 bytes --]