Re: [PATCH 4/5] fetch2/gitsm: Add clean function

[Mark Hatle <mark.hatle@windriver.com>]

On 3/14/19 9:50 PM, Robert Yang wrote:
> Hi,
> 
> On 3/14/19 8:34 PM, Lisicki, Raphael wrote:
>> I have not fully understand the logic but I am wondering what happens here is a submodule is located in two different repositories? Wouldn't a clean of one module delete submodules of the other one as well?
> 
> Here are more explanations about the logic, for example, repo_A has two
> submodules repo_B and repo_C, then after do_fetch,
> they will be"
> DL_DIR/repo_A
> DL_DIR/repo_B
> DL_DIR/repo_C
> 
> Without this patch, do_cleanall only removes DL_DIR/repo_A, but leaves
> repo_B and repo_C there, this patch makes it remove repo_B and C.

The issue Raphael is mentioning is when you have multiple projects that use the
same submodules.  I.e.:

recipeA:
 DL_DIR/repo_A
 DL_DIR/repo_B
 DL_DIR/repo_C

recipeB:
 DL_DIR/repo_D
 DL_DIR/repo_C

If you do a clean of A, you will end up removing repo_C which both items end up
using.

I don't know any way to resolve this, since the dependency is implicit, not
explicit -- so the system won't have any idea that the same thing is used in
multiple places.

> 
> For your question, if repo_B is submodule of both repo_A and repo_D, it
> is like foo and foo-native use the same git source, when you run
> "bitbake foo -cccleanall", both foo and foo-native's source will be
> removed since they use the same source, the similar to gitsm, so it is not
> a problem, the source will be re-downloaded automatically when needed.

I'm not sure if this situation will work, since the recipeB will have a stamp
that says the download is complete, even though part of it is now missing.

--Mark

> // Robert
> 
>>
>> Best regards
>> Raphael
>>
>>> -----Original Message-----
>>> From: bitbake-devel-bounces@lists.openembedded.org <bitbake-devel-
>>> bounces@lists.openembedded.org> On Behalf Of Robert Yang
>>> Sent: Thursday, March 14, 2019 10:28 AM
>>> To: bitbake-devel@lists.openembedded.org
>>> Subject: [bitbake-devel] [PATCH 4/5] fetch2/gitsm: Add clean function
>>>
>>> The git's clean can only remove parent git repo from DL_DIR, but doesn't
>>> remove
>>> submodules, this patch fixes the problem.
>>>
>>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>>> ---
>>>   bitbake/lib/bb/fetch2/gitsm.py | 15 +++++++++++++--
>>>   1 file changed, 13 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/bitbake/lib/bb/fetch2/gitsm.py b/bitbake/lib/bb/fetch2/gitsm.py
>>> index b21fed2..bb61093 100644
>>> --- a/bitbake/lib/bb/fetch2/gitsm.py
>>> +++ b/bitbake/lib/bb/fetch2/gitsm.py
>>> @@ -76,8 +76,8 @@ class GitSM(Git):
>>>           for name in ud.names:
>>>               try:
>>>                   gitmodules = runfetchcmd("%s show %s:.gitmodules" %
>>> (ud.basecmd, ud.revisions[name]), d, quiet=True, workdir=workdir)
>>> -            except:
>>> -                # No submodules to update
>>> +            except Exception as esc:
>>> +                logger.info("No submodules found: %s" % workdir)
>>>                   continue
>>>
>>>               for m, md in parse_gitmodules(gitmodules).items():
>>> @@ -164,6 +164,17 @@ class GitSM(Git):
>>>           Git.download(self, ud, d)
>>>           self.process_submodules(ud, ud.clonedir, download_submodule, d)
>>>
>>> +    def clean(self, ud, d):
>>> +        def clean_submodule(ud, url, module, modpath, d):
>>> +            try:
>>> +                newfetch = Fetch([url], d, cache=False)
>>> +                newfetch.clean()
>>> +            except Exception as e:
>>> +                logger.warn('gitsm: submodule clean failed: %s %s' %
>>> (type(e).__name__, str(e)))
>>> +
>>> +        self.process_submodules(ud, ud.clonedir, clean_submodule, d)
>>> +        Git.clean(self, ud, d)
>>> +
>>>       def unpack(self, ud, destdir, d):
>>>           def unpack_submodules(ud, url, module, modpath, d):
>>>               url += ";bareclone=1;nobranch=1"
>>> --
>>> 2.7.4
>>>
>>> --
>>> _______________________________________________
>>> bitbake-devel mailing list
>>> bitbake-devel@lists.openembedded.org
>>> http://lists.openembedded.org/mailman/listinfo/bitbake-devel