* [PATCH 0/1] [RFC]git fetch support for rebaseable repo (a.k.a crazy git fetcher) @ 2011-05-16 1:16 Yu Ke 2011-05-16 1:16 ` [PATCH 1/1] git fetcher: add support for rebaseable git repo Yu Ke 2011-05-17 14:20 ` [PATCH 0/1] [RFC]git fetch support for rebaseable repo (a.k.a crazy git fetcher) Richard Purdie 0 siblings, 2 replies; 7+ messages in thread From: Yu Ke @ 2011-05-16 1:16 UTC (permalink / raw) To: poky From: Yu Ke <ke.yu@intel.com> Pull URL: git://git.pokylinux.org/poky-contrib.git Branch: kyu3/crazygit Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=kyu3/crazygit Thanks, Yu Ke <ke.yu@intel.com> --- Yu Ke (1): git fetcher: add support for rebaseable git repo bitbake/lib/bb/fetch2/git.py | 28 ++++++++++++++++++++-------- 1 files changed, 20 insertions(+), 8 deletions(-) ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/1] git fetcher: add support for rebaseable git repo 2011-05-16 1:16 [PATCH 0/1] [RFC]git fetch support for rebaseable repo (a.k.a crazy git fetcher) Yu Ke @ 2011-05-16 1:16 ` Yu Ke 2011-05-20 16:30 ` Khem Raj 2011-05-17 14:20 ` [PATCH 0/1] [RFC]git fetch support for rebaseable repo (a.k.a crazy git fetcher) Richard Purdie 1 sibling, 1 reply; 7+ messages in thread From: Yu Ke @ 2011-05-16 1:16 UTC (permalink / raw) To: poky From: Yu Ke <ke.yu@intel.com> Some upstream git repo may rebase in the future, which means current revision may disappear from the upstream repo after the rebase. current git fetcher can not handle this case, because the git mirror tar ball is per repo, and may also change in the rebase and lost the current revision info. To fix this issue, this patch - add rebaseable tag in the SRC_URI - for rebaseable repo, make git mirror tar ball per revision, in this case, even upstream rebase, the git mirror still has the current revision info. - for rebaseable repo, generate mirror tar ball by default, since the repo may change in the future. Signed-off-by: Yu Ke <ke.yu@intel.com> --- bitbake/lib/bb/fetch2/git.py | 28 ++++++++++++++++++++-------- 1 files changed, 20 insertions(+), 8 deletions(-) diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py index 811acbf..82721c6 100644 --- a/bitbake/lib/bb/fetch2/git.py +++ b/bitbake/lib/bb/fetch2/git.py @@ -57,6 +57,11 @@ class Git(FetchMethod): if 'nocheckout' in ud.parm: ud.nocheckout = True + # rebaseable means the upstream git repo may rebase in the future, + # and current revision may disappear from upstream repo + # rebaseable is false by default. set rebaseable=1 in SRC_URI if rebaseable. + ud.rebaseable = ud.parm.get("rebaseable","0") == "1" + branches = ud.parm.get("branch", "master").split(',') if len(branches) != len(ud.names): raise bb.fetch2.ParameterError("The number of name and branch parameters is not balanced", ud.url) @@ -65,16 +70,9 @@ class Git(FetchMethod): branch = branches[ud.names.index(name)] ud.branches[name] = branch - gitsrcname = '%s%s' % (ud.host, ud.path.replace('/', '.')) - ud.mirrortarball = 'git2_%s.tar.gz' % (gitsrcname) - ud.fullmirror = os.path.join(data.getVar("DL_DIR", d, True), ud.mirrortarball) - ud.clonedir = os.path.join(data.expand('${GITDIR}', d), gitsrcname) - ud.basecmd = data.getVar("FETCHCMD_git", d, True) or "git" - ud.write_tarballs = (data.getVar("BB_GENERATE_MIRROR_TARBALLS", d, True) or "0") != "0" - - ud.localfile = ud.clonedir + ud.write_tarballs = ((data.getVar("BB_GENERATE_MIRROR_TARBALLS", d, True) or "0") != "0") or ud.rebaseable ud.setup_revisons(d) @@ -84,6 +82,20 @@ class Git(FetchMethod): ud.branches[name] = ud.revisions[name] ud.revisions[name] = self.latest_revision(ud.url, ud, d, name) + gitsrcname = '%s%s' % (ud.host, ud.path.replace('/', '.')) + # for rebaseable git repo, it is necessary to keep mirror tar ball + # per revision, so that even the revision disappears from the + # upstream repo in the future, the mirror will remain intact and still + # contains the revision + if ud.rebaseable: + for name in ud.names: + gitsrcname = gitsrcname + '_' + ud.revisions[name] + ud.mirrortarball = 'git2_%s.tar.gz' % (gitsrcname) + ud.fullmirror = os.path.join(data.getVar("DL_DIR", d, True), ud.mirrortarball) + ud.clonedir = os.path.join(data.expand('${GITDIR}', d), gitsrcname) + + ud.localfile = ud.clonedir + def localpath(self, url, ud, d): return ud.clonedir -- 1.7.0.4 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 1/1] git fetcher: add support for rebaseable git repo 2011-05-16 1:16 ` [PATCH 1/1] git fetcher: add support for rebaseable git repo Yu Ke @ 2011-05-20 16:30 ` Khem Raj 2011-05-21 0:07 ` Darren Hart 0 siblings, 1 reply; 7+ messages in thread From: Khem Raj @ 2011-05-20 16:30 UTC (permalink / raw) To: poky On 5/15/2011 6:16 PM, Yu Ke wrote: > From: Yu Ke<ke.yu@intel.com> > > Some upstream git repo may rebase in the future, which means current > revision may disappear from the upstream repo after the rebase. > > current git fetcher can not handle this case, because the git mirror > tar ball is per repo, and may also change in the rebase and lost the > current revision info. > > To fix this issue, this patch > - add rebaseable tag in the SRC_URI > - for rebaseable repo, make git mirror tar ball per revision, in this > case, even upstream rebase, the git mirror still has the current > revision info. > - for rebaseable repo, generate mirror tar ball by default, since the > repo may change in the future. Do we have the options to SRC_URI documented somewhere ? if yes then this should be added there too. > > Signed-off-by: Yu Ke<ke.yu@intel.com> > --- > bitbake/lib/bb/fetch2/git.py | 28 ++++++++++++++++++++-------- > 1 files changed, 20 insertions(+), 8 deletions(-) > > diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py > index 811acbf..82721c6 100644 > --- a/bitbake/lib/bb/fetch2/git.py > +++ b/bitbake/lib/bb/fetch2/git.py > @@ -57,6 +57,11 @@ class Git(FetchMethod): > if 'nocheckout' in ud.parm: > ud.nocheckout = True > > + # rebaseable means the upstream git repo may rebase in the future, > + # and current revision may disappear from upstream repo > + # rebaseable is false by default. set rebaseable=1 in SRC_URI if rebaseable. > + ud.rebaseable = ud.parm.get("rebaseable","0") == "1" > + > branches = ud.parm.get("branch", "master").split(',') > if len(branches) != len(ud.names): > raise bb.fetch2.ParameterError("The number of name and branch parameters is not balanced", ud.url) > @@ -65,16 +70,9 @@ class Git(FetchMethod): > branch = branches[ud.names.index(name)] > ud.branches[name] = branch > > - gitsrcname = '%s%s' % (ud.host, ud.path.replace('/', '.')) > - ud.mirrortarball = 'git2_%s.tar.gz' % (gitsrcname) > - ud.fullmirror = os.path.join(data.getVar("DL_DIR", d, True), ud.mirrortarball) > - ud.clonedir = os.path.join(data.expand('${GITDIR}', d), gitsrcname) > - > ud.basecmd = data.getVar("FETCHCMD_git", d, True) or "git" > > - ud.write_tarballs = (data.getVar("BB_GENERATE_MIRROR_TARBALLS", d, True) or "0") != "0" > - > - ud.localfile = ud.clonedir > + ud.write_tarballs = ((data.getVar("BB_GENERATE_MIRROR_TARBALLS", d, True) or "0") != "0") or ud.rebaseable > > ud.setup_revisons(d) > > @@ -84,6 +82,20 @@ class Git(FetchMethod): > ud.branches[name] = ud.revisions[name] > ud.revisions[name] = self.latest_revision(ud.url, ud, d, name) > > + gitsrcname = '%s%s' % (ud.host, ud.path.replace('/', '.')) > + # for rebaseable git repo, it is necessary to keep mirror tar ball > + # per revision, so that even the revision disappears from the > + # upstream repo in the future, the mirror will remain intact and still > + # contains the revision > + if ud.rebaseable: > + for name in ud.names: > + gitsrcname = gitsrcname + '_' + ud.revisions[name] > + ud.mirrortarball = 'git2_%s.tar.gz' % (gitsrcname) > + ud.fullmirror = os.path.join(data.getVar("DL_DIR", d, True), ud.mirrortarball) > + ud.clonedir = os.path.join(data.expand('${GITDIR}', d), gitsrcname) > + > + ud.localfile = ud.clonedir > + > def localpath(self, url, ud, d): > return ud.clonedir > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/1] git fetcher: add support for rebaseable git repo 2011-05-20 16:30 ` Khem Raj @ 2011-05-21 0:07 ` Darren Hart 2011-05-24 5:50 ` Yu Ke 0 siblings, 1 reply; 7+ messages in thread From: Darren Hart @ 2011-05-21 0:07 UTC (permalink / raw) To: Khem Raj; +Cc: poky On 05/20/2011 09:30 AM, Khem Raj wrote: > On 5/15/2011 6:16 PM, Yu Ke wrote: >> From: Yu Ke<ke.yu@intel.com> >> >> Some upstream git repo may rebase in the future, which means current >> revision may disappear from the upstream repo after the rebase. >> >> current git fetcher can not handle this case, because the git mirror >> tar ball is per repo, and may also change in the rebase and lost the >> current revision info. >> >> To fix this issue, this patch >> - add rebaseable tag in the SRC_URI >> - for rebaseable repo, make git mirror tar ball per revision, in this >> case, even upstream rebase, the git mirror still has the current >> revision info. >> - for rebaseable repo, generate mirror tar ball by default, since the >> repo may change in the future. > > Do we have the options to SRC_URI documented somewhere ? > if yes then this should be added there too. I was looking and didn't find them. At the very least these should be documented in the fetch2/git.py file for reference when reviewing/maintaining the source. -- Darren Hart Intel Open Source Technology Center Yocto Project - Linux Kernel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/1] git fetcher: add support for rebaseable git repo 2011-05-21 0:07 ` Darren Hart @ 2011-05-24 5:50 ` Yu Ke 2011-05-24 14:37 ` Darren Hart 0 siblings, 1 reply; 7+ messages in thread From: Yu Ke @ 2011-05-24 5:50 UTC (permalink / raw) To: Darren Hart; +Cc: poky on 2011-5-21 8:07, Darren Hart wrote: > > > On 05/20/2011 09:30 AM, Khem Raj wrote: >> On 5/15/2011 6:16 PM, Yu Ke wrote: >>> From: Yu Ke<ke.yu@intel.com> >>> >>> Some upstream git repo may rebase in the future, which means current >>> revision may disappear from the upstream repo after the rebase. >>> >>> current git fetcher can not handle this case, because the git mirror >>> tar ball is per repo, and may also change in the rebase and lost the >>> current revision info. >>> >>> To fix this issue, this patch >>> - add rebaseable tag in the SRC_URI >>> - for rebaseable repo, make git mirror tar ball per revision, in this >>> case, even upstream rebase, the git mirror still has the current >>> revision info. >>> - for rebaseable repo, generate mirror tar ball by default, since the >>> repo may change in the future. >> >> Do we have the options to SRC_URI documented somewhere ? >> if yes then this should be added there too. > > I was looking and didn't find them. At the very least these should be > documented in the fetch2/git.py file for reference when > reviewing/maintaining the source. > This is good reminder, Thanks. There are several places that hold this info: - bitbake manual: http://bitbake.berlios.de/manual/ch03s07.html - openembedded manual: http://docs.openembedded.org/usermanual/usermanual.html#id572539 - poky reference manual: http://www.yoctoproject.org/docs/poky-ref-manual/poky-ref-manual.html#ref-bitbake-fetchers It looks the most appropriate place would be bitbake manual, which is also refereed by poky reference manual. So does anyone know how can I update this mannual? Also, it sounds good idea to add the SRC_URI options doc in the source, I will cook a patch to add this info. Regards Ke ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/1] git fetcher: add support for rebaseable git repo 2011-05-24 5:50 ` Yu Ke @ 2011-05-24 14:37 ` Darren Hart 0 siblings, 0 replies; 7+ messages in thread From: Darren Hart @ 2011-05-24 14:37 UTC (permalink / raw) To: Yu Ke; +Cc: poky On 05/23/2011 10:50 PM, Yu Ke wrote: > on 2011-5-21 8:07, Darren Hart wrote: >> >> >> On 05/20/2011 09:30 AM, Khem Raj wrote: >>> On 5/15/2011 6:16 PM, Yu Ke wrote: >>>> From: Yu Ke<ke.yu@intel.com> >>>> >>>> Some upstream git repo may rebase in the future, which means current >>>> revision may disappear from the upstream repo after the rebase. >>>> >>>> current git fetcher can not handle this case, because the git mirror >>>> tar ball is per repo, and may also change in the rebase and lost the >>>> current revision info. >>>> >>>> To fix this issue, this patch >>>> - add rebaseable tag in the SRC_URI >>>> - for rebaseable repo, make git mirror tar ball per revision, in this >>>> case, even upstream rebase, the git mirror still has the current >>>> revision info. >>>> - for rebaseable repo, generate mirror tar ball by default, since the >>>> repo may change in the future. >>> >>> Do we have the options to SRC_URI documented somewhere ? >>> if yes then this should be added there too. >> >> I was looking and didn't find them. At the very least these should be >> documented in the fetch2/git.py file for reference when >> reviewing/maintaining the source. >> > > This is good reminder, Thanks. > > There are several places that hold this info: > > - bitbake manual: http://bitbake.berlios.de/manual/ch03s07.html > - openembedded manual: > http://docs.openembedded.org/usermanual/usermanual.html#id572539 > - poky reference manual: > http://www.yoctoproject.org/docs/poky-ref-manual/poky-ref-manual.html#ref-bitbake-fetchers > > It looks the most appropriate place would be bitbake manual, which is > also refereed by poky reference manual. So does anyone know how can I > update this mannual? I've provided a couple minor fixes to the manual. I just edited: poky.git/bitbake/doc/manual/usermanual.xml And sent the patch to the poky list. It might be preferable to develop against the actual bitbake repository, but nobody has complained yet :-) > > Also, it sounds good idea to add the SRC_URI options doc in the source, > I will cook a patch to add this info. Great! Thanks Ke, > > Regards > Ke -- Darren Hart Intel Open Source Technology Center Yocto Project - Linux Kernel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 0/1] [RFC]git fetch support for rebaseable repo (a.k.a crazy git fetcher) 2011-05-16 1:16 [PATCH 0/1] [RFC]git fetch support for rebaseable repo (a.k.a crazy git fetcher) Yu Ke 2011-05-16 1:16 ` [PATCH 1/1] git fetcher: add support for rebaseable git repo Yu Ke @ 2011-05-17 14:20 ` Richard Purdie 1 sibling, 0 replies; 7+ messages in thread From: Richard Purdie @ 2011-05-17 14:20 UTC (permalink / raw) To: Yu Ke; +Cc: poky On Mon, 2011-05-16 at 09:16 +0800, Yu Ke wrote: > From: Yu Ke <ke.yu@intel.com> > > > Pull URL: git://git.pokylinux.org/poky-contrib.git > Branch: kyu3/crazygit > Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=kyu3/crazygit > > Thanks, > Yu Ke <ke.yu@intel.com> > --- > > > Yu Ke (1): > git fetcher: add support for rebaseable git repo Merged to bitbake upstream and master, thanks. Richard ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2011-05-24 14:38 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2011-05-16 1:16 [PATCH 0/1] [RFC]git fetch support for rebaseable repo (a.k.a crazy git fetcher) Yu Ke 2011-05-16 1:16 ` [PATCH 1/1] git fetcher: add support for rebaseable git repo Yu Ke 2011-05-20 16:30 ` Khem Raj 2011-05-21 0:07 ` Darren Hart 2011-05-24 5:50 ` Yu Ke 2011-05-24 14:37 ` Darren Hart 2011-05-17 14:20 ` [PATCH 0/1] [RFC]git fetch support for rebaseable repo (a.k.a crazy git fetcher) Richard Purdie
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.