How to detect change on a repo manifest.git and rebuild the recipe with "repo://"

How to detect change on a repo manifest.git and rebuild the recipe with "repo://"

[GUAN Ben (ST-FIR/ENG1-Zhu)]

[-- Attachment #1: Type: text/plain, Size: 1438 bytes --]

The source code of my SW consist of multiple components from different repository, which is managed by a repo manifest repository. So I want to use "repo://" for the repo manifest repository instead of multiple "git://" to include all repositories in the SRC_URI.

I add the following to MyRecipe_git.bb:
```
SRC_URI = "repo://git@<MY_HOST>/manifest.git;protocol=ssh;branch=master"

S = "${WORKDIR}/repo/MyRecipe"
```

And in my local.conf, I add the following:
```
# Use this setting because I don't have internet connection to external Android repo.git, I mirror one in the internal host
REPO_URL = "git@<My_HOST>/repo.git"
# Use "repo" command in the host
HOSTTOOLS += "repo"
# Because the my Yocto meta-layers are managed by repo also, use a different directory for my recipe
REPODIR = "/home/benguan/projects/MyRecipeRepo"
```

Now I can build my recipe with Yocto, but there is an issue:  When there is any change required in the repositories of my application, I will change the code and update the tags in default.xml of manifest.git. I want Yocto could detect this change, fetch the change, them rebuild the recipe.

With "git://", it is easy to achieve via setting "${AUTOREV}" to "SRCREV". But the fetch2.repo doesn't support SRCREV. I have go through the existing recipe within meta-oe and oe-core, no recipe is using "repo://".

Does anybody have experience about it and help? Thank you in advance.


[-- Attachment #2: Type: text/html, Size: 3780 bytes --]

Re: How to detect change on a repo manifest.git and rebuild the recipe with "repo://"

[Leonardo Sandoval]

On Fri, 2017-08-25 at 02:06 +0000, GUAN Ben (ST-FIR/ENG1-Zhu) wrote:
> The source code of my SW consist of multiple components from different
> repository, which is managed by a repo manifest repository. So I want
> to use “repo://” for the repo manifest repository instead of multiple
> “git://” to include all repositories in the SRC_URI.
>  
> I add the following to MyRecipe_git.bb:
> ```
> SRC_URI =
> “repo://git@<MY_HOST>/manifest.git;protocol=ssh;branch=master”
>  
> S = “${WORKDIR}/repo/MyRecipe”
> ```
>  
> And in my local.conf, I add the following:
> ```
> # Use this setting because I don’t have internet connection to
> external Android repo.git, I mirror one in the internal host
> REPO_URL = “git@<My_HOST>/repo.git”
> # Use “repo” command in the host
> HOSTTOOLS += “repo”
> # Because the my Yocto meta-layers are managed by repo also, use a
> different directory for my recipe 
> REPODIR = “/home/benguan/projects/MyRecipeRepo”
> ```
>  
> Now I can build my recipe with Yocto, but there is an issue:  When
> there is any change required in the repositories of my application, I
> will change the code and update the tags in default.xml of
> manifest.git. I want Yocto could detect this change, fetch the change,
> them rebuild the recipe.  


the easiest would be to 'repo sync & bitbake'. bitbake just handles its
metadata and from it figures out if needs to re-execute a task, so
better if you split the repository update and the bitbake as two
separate processes/steps.

>  
> With “git://”, it is easy to achieve via setting “${AUTOREV}” to
> “SRCREV”. But the fetch2.repo doesn’t support SRCREV. I have go
> through the existing recipe within meta-oe and oe-core, no recipe is
> using “repo://”.
>  
> Does anybody have experience about it and help? Thank you in advance.
>  
> -- 
> _______________________________________________
> bitbake-devel mailing list
> bitbake-devel@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/bitbake-devel

Re: How to detect change on a repo manifest.git and rebuild the recipe with "repo://"

[Andre McCurdy]

On Thu, Aug 24, 2017 at 7:06 PM, GUAN Ben (ST-FIR/ENG1-Zhu)
<Ben.Guan@cn.bosch.com> wrote:
> The source code of my SW consist of multiple components from different
> repository, which is managed by a repo manifest repository. So I want to use
> “repo://” for the repo manifest repository instead of multiple “git://” to
> include all repositories in the SRC_URI.
>
> I add the following to MyRecipe_git.bb:
> ```
> SRC_URI = “repo://git@<MY_HOST>/manifest.git;protocol=ssh;branch=master”
>
> Now I can build my recipe with Yocto, but there is an issue:  When there is
> any change required in the repositories of my application, I will change the
> code and update the tags in default.xml of manifest.git. I want Yocto could
> detect this change, fetch the change, them rebuild the recipe.
>
> With “git://”, it is easy to achieve via setting “${AUTOREV}” to “SRCREV”.
> But the fetch2.repo doesn’t support SRCREV. I have go through the existing
> recipe within meta-oe and oe-core, no recipe is using “repo://”.
>
> Does anybody have experience about it and help? Thank you in advance.

Right, the repo fetcher doesn't support SRCREV. It's not even clear
how a repo manifest could map to a SRCREV, since the manifest could
define an arbitrary combination of fixed revisions and/or branches for
an arbitrary number of git repos.

You could perhaps append a call to "repo sync" to the do_patch etc
task in your recipe but it would certainly be a hack and you would run
into dependency problems (do_patch might not be re-run and without a
SRCREV bitbake wouldn't know if/when the code had changed and
therefore if/when to re-run dependent tasks).

As you have discovered by looking oe-core and meta-oe, the repo
fetcher isn't used much. The preferred solution for recipes which need
to fetch from multiple git repos is to include multiple git://
references in SRC_URI.