PREMIRROR

PREMIRROR

[Russell Peterson]

[-- Attachment #1: Type: text/plain, Size: 922 bytes --]

Hello,

I am looking to have bitbake pick up files for a particular recipe from a
local git repository using the PREMIRROR functionality.

Basically, the recipe (bb file) points to github but in my local build I
add PREMIRROR_prepend = "git://.*/.*
git:///local/path/BASENAME;protocol=file\n"

I will probably make the git regular expression more exact for my specific
github repo but this works for now.

This all works (as I deleted the github download from the local download
directory) because I can see in the do_fetch log and the correct (local)
repo was found and placed in the DL_DIR.

Problem is, do_unpack fails because it appears to be looking for the
original (github) SRC_URI.  Then it complains about "no up to date source
found: clone or directory not available or not up to date (shallow clone
not enabled)"

Any help on what I am missing would be appreciated.

Regards,

Russell

[-- Attachment #2: Type: text/html, Size: 1178 bytes --]

Re: PREMIRROR

[Yi Zhao]

[-- Attachment #1: Type: text/plain, Size: 1396 bytes --]


On 7/24/19 4:49 AM, Russell Peterson wrote:
> Hello,
>
> I am looking to have bitbake pick up files for a particular recipe 
> from a local git repository using the PREMIRROR functionality.
>
> Basically, the recipe (bb file) points to github but in my local build 
> I add PREMIRROR_prepend = "git://.*/.* 
> git:///local/path/BASENAME;protocol=file\n"
>
> I will probably make the git regular expression more exact for my 
> specific github repo but this works for now.
>
> This all works (as I deleted the github download from the local 
> download directory) because I can see in the do_fetch log and the 
> correct (local) repo was found and placed in the DL_DIR.
>
> Problem is, do_unpack fails because it appears to be looking for the 
> original (github) SRC_URI.  Then it complains about "no up to date 
> source found: clone or directory not available or not up to date 
> (shallow clone not enabled)"


Maybe you can use BB_GENERATE_MIRROR_TARBALLS to generate git repo 
tarball and put it to your download mirror.

See: 
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-bb-BB_GENERATE_MIRROR_TARBALLS

         and

https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-DL_DIR


Regards,

Yi


>
> Any help on what I am missing would be appreciated.
>
> Regards,
>
> Russell
>
>
>

[-- Attachment #2: Type: text/html, Size: 3122 bytes --]

Re: PREMIRROR

[Russell Peterson]

[-- Attachment #1: Type: text/plain, Size: 1665 bytes --]

Thanks, Yi.  I did try that and I see the tar file is created... but again,
the do_unpack function seems to ignore it and go directly to the github
repository.

--Russ


On Wed, Jul 24, 2019 at 4:28 AM Yi Zhao <yi.zhao@windriver.com> wrote:

>
> On 7/24/19 4:49 AM, Russell Peterson wrote:
>
> Hello,
>
> I am looking to have bitbake pick up files for a particular recipe from a
> local git repository using the PREMIRROR functionality.
>
> Basically, the recipe (bb file) points to github but in my local build I
> add PREMIRROR_prepend = "git://.*/.*
> git:///local/path/BASENAME;protocol=file\n"
>
> I will probably make the git regular expression more exact for my specific
> github repo but this works for now.
>
> This all works (as I deleted the github download from the local download
> directory) because I can see in the do_fetch log and the correct (local)
> repo was found and placed in the DL_DIR.
>
> Problem is, do_unpack fails because it appears to be looking for the
> original (github) SRC_URI.  Then it complains about "no up to date source
> found: clone or directory not available or not up to date (shallow clone
> not enabled)"
>
>
> Maybe you can use BB_GENERATE_MIRROR_TARBALLS to generate git repo tarball
> and put it to your download mirror.
>
> See:
> https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-bb-BB_GENERATE_MIRROR_TARBALLS
>
>         and
>
>
> https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-DL_DIR
>
>
> Regards,
>
> Yi
>
>
>
> Any help on what I am missing would be appreciated.
>
> Regards,
>
> Russell
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 3509 bytes --]

Re: PREMIRROR

[Rudolf J Streif]

[-- Attachment #1.1.1: Type: text/plain, Size: 1688 bytes --]

Russell,

You don't need PREMIRROR for this functionality. It's not exactly
intended for that use.

The simplest way to achieve what you are looking for is to use devtool.
If I understand you correctly you are downloading sources from a remote
repo on GitHub but want to have them locally to make modifications? If
so use:

devtool add myrecipe localsrc fetchuri

from your build environment (you have to source oe-init-build-env
first). devtool then fetches the source from fetchuri into a directory
localsrc as a git repo and automatically creates the recipe for it.

:rjs


On 7/23/19 1:49 PM, Russell Peterson wrote:
> Hello,
>
> I am looking to have bitbake pick up files for a particular recipe
> from a local git repository using the PREMIRROR functionality.
>
> Basically, the recipe (bb file) points to github but in my local build
> I add PREMIRROR_prepend = "git://.*/.* 
> git:///local/path/BASENAME;protocol=file\n"
>
> I will probably make the git regular expression more exact for my
> specific github repo but this works for now.
>
> This all works (as I deleted the github download from the local
> download directory) because I can see in the do_fetch log and the
> correct (local) repo was found and placed in the DL_DIR.
>
> Problem is, do_unpack fails because it appears to be looking for the
> original (github) SRC_URI.  Then it complains about "no up to date
> source found: clone or directory not available or not up to date
> (shallow clone not enabled)"
>
> Any help on what I am missing would be appreciated.
>
> Regards,
>
> Russell
>
>
>
-- 
-----
Rudolf J Streif
CEO/CTO ibeeto
+1.855.442.3396 x700


[-- Attachment #1.1.2: Type: text/html, Size: 2970 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 499 bytes --]

Re: PREMIRROR

[Russell Peterson]

[-- Attachment #1: Type: text/plain, Size: 2290 bytes --]

Hi, Rudolf.

Thanks for the reply.  Yes, I am aware of the eSDK functionality, however,
I have some unique requirements that I am trying to work around.
Regardless... what I am doing should work, no?  I simply want to use a
local git repo (the directory itself hence protocol=file) instead of what
the recipe specifies.  Looks like the fetch is working but the do_unpack
task is ignoring PREMIRRORS (or at least the localpath variable seems
wrong).

--Russ


On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif <rudolf.streif@ibeeto.com>
wrote:

> Russell,
>
> You don't need PREMIRROR for this functionality. It's not exactly intended
> for that use.
>
> The simplest way to achieve what you are looking for is to use devtool. If
> I understand you correctly you are downloading sources from a remote repo
> on GitHub but want to have them locally to make modifications? If so use:
>
> devtool add myrecipe localsrc fetchuri
>
> from your build environment (you have to source oe-init-build-env first).
> devtool then fetches the source from fetchuri into a directory localsrc as
> a git repo and automatically creates the recipe for it.
>
> :rjs
>
>
> On 7/23/19 1:49 PM, Russell Peterson wrote:
>
> Hello,
>
> I am looking to have bitbake pick up files for a particular recipe from a
> local git repository using the PREMIRROR functionality.
>
> Basically, the recipe (bb file) points to github but in my local build I
> add PREMIRROR_prepend = "git://.*/.*
> git:///local/path/BASENAME;protocol=file\n"
>
> I will probably make the git regular expression more exact for my specific
> github repo but this works for now.
>
> This all works (as I deleted the github download from the local download
> directory) because I can see in the do_fetch log and the correct (local)
> repo was found and placed in the DL_DIR.
>
> Problem is, do_unpack fails because it appears to be looking for the
> original (github) SRC_URI.  Then it complains about "no up to date source
> found: clone or directory not available or not up to date (shallow clone
> not enabled)"
>
> Any help on what I am missing would be appreciated.
>
> Regards,
>
> Russell
>
>
>
> --
> -----
> Rudolf J Streif
> CEO/CTO ibeeto
> +1.855.442.3396 x700
>
>

[-- Attachment #2: Type: text/html, Size: 3720 bytes --]

Re: PREMIRROR

[Rudolf J Streif]

[-- Attachment #1.1.1: Type: text/plain, Size: 3419 bytes --]

Hi Russell,

devtool and eSDK are different things. The purpose of PREMIRRORS is to
set a mirror for all recipes. It's a way for organizations to control
where their YP builds download sources from. It's not intended to be
used for a single recipe. There is no need for that. You simply set
SRC_URI in your recipe to your local git repo. That is what devtool does
after downloading the sources from a remote repo. If you already have
the remote repo cloned locally you can just point devtool to it.

You can do it manually by creating your recipe and setting SRC_URI like
this:

SRC_URI = "git:///local/path/${PN};protocol=file"

SRCREV = "${AUTOREV}"

S = "${WORKDIR}/git"

PREMIRRORS is only relevant for do_fetch not for do_unpack.

:rjs


On 7/24/19 11:28 AM, Russell Peterson wrote:
> Hi, Rudolf.
>
> Thanks for the reply.  Yes, I am aware of the eSDK functionality,
> however, I have some unique requirements that I am trying to work
> around.  Regardless... what I am doing should work, no?  I simply want
> to use a local git repo (the directory itself hence protocol=file)
> instead of what the recipe specifies.  Looks like the fetch is working
> but the do_unpack task is ignoring PREMIRRORS (or at least the
> localpath variable seems wrong).
>
> --Russ
>
>
> On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif
> <rudolf.streif@ibeeto.com <mailto:rudolf.streif@ibeeto.com>> wrote:
>
>     Russell,
>
>     You don't need PREMIRROR for this functionality. It's not exactly
>     intended for that use.
>
>     The simplest way to achieve what you are looking for is to use
>     devtool. If I understand you correctly you are downloading sources
>     from a remote repo on GitHub but want to have them locally to make
>     modifications? If so use:
>
>     devtool add myrecipe localsrc fetchuri
>
>     from your build environment (you have to source oe-init-build-env
>     first). devtool then fetches the source from fetchuri into a
>     directory localsrc as a git repo and automatically creates the
>     recipe for it.
>
>     :rjs
>
>
>     On 7/23/19 1:49 PM, Russell Peterson wrote:
>>     Hello,
>>
>>     I am looking to have bitbake pick up files for a particular
>>     recipe from a local git repository using the PREMIRROR functionality.
>>
>>     Basically, the recipe (bb file) points to github but in my local
>>     build I add PREMIRROR_prepend = "git://.*/.* 
>>     git:///local/path/BASENAME;protocol=file\n"
>>
>>     I will probably make the git regular expression more exact for my
>>     specific github repo but this works for now.
>>
>>     This all works (as I deleted the github download from the local
>>     download directory) because I can see in the do_fetch log and the
>>     correct (local) repo was found and placed in the DL_DIR.
>>
>>     Problem is, do_unpack fails because it appears to be looking for
>>     the original (github) SRC_URI.  Then it complains about "no up to
>>     date source found: clone or directory not available or not up to
>>     date (shallow clone not enabled)"
>>
>>     Any help on what I am missing would be appreciated.
>>
>>     Regards,
>>
>>     Russell
>>
>>
>>
>     -- 
>     -----
>     Rudolf J Streif
>     CEO/CTO ibeeto
>     +1.855.442.3396 x700
>
-- 
-----
Rudolf J Streif
CEO/CTO ibeeto
+1.855.442.3396 x700


[-- Attachment #1.1.2: Type: text/html, Size: 6112 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 499 bytes --]

Re: PREMIRROR

[Russell Peterson]

[-- Attachment #1: Type: text/plain, Size: 4497 bytes --]

Hi, Rudolf.

I apologize for not being clear.  The idea here is that my recipe points to
github while, for my local development environment, I set a premirror to
match a specific github repository and translate it to a local directory.
That works.  The fetch matches the PREMIRROR and places a copy of the LOCAL
git repo in my DL_DIR.  The problem is, the do_unpack task is looking for
the git repo in the DL_DIR (git2/etc...)... but it's looking for the git
repo from github, not my local repo that the fetch task just put in DL_DIR.

There are numerous reasons I'm doing this including the fact that I cannot
put a reference to my local repo in the bb file since I ship that recipe to
my customers and a local pathname is meaningless to them.  I prefer to
simply modify my local.conf with a PREMIRROR value that has a specific
regular expression that matches this particular github repo (and hence does
NOT effect all recipes).  This is why I wanted to use the PREMIRROR
function.  Unfortunately, it does not appear to work or at least not work
as I expected.

--Russ




On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif <rudolf.streif@ibeeto.com>
wrote:

> Hi Russell,
>
> devtool and eSDK are different things. The purpose of PREMIRRORS is to set
> a mirror for all recipes. It's a way for organizations to control where
> their YP builds download sources from. It's not intended to be used for a
> single recipe. There is no need for that. You simply set SRC_URI in your
> recipe to your local git repo. That is what devtool does after downloading
> the sources from a remote repo. If you already have the remote repo cloned
> locally you can just point devtool to it.
>
> You can do it manually by creating your recipe and setting SRC_URI like
> this:
>
> SRC_URI = "git:///local/path/${PN};protocol=file"
>
> SRCREV = "${AUTOREV}"
>
> S = "${WORKDIR}/git"
>
> PREMIRRORS is only relevant for do_fetch not for do_unpack.
>
> :rjs
>
>
> On 7/24/19 11:28 AM, Russell Peterson wrote:
>
> Hi, Rudolf.
>
> Thanks for the reply.  Yes, I am aware of the eSDK functionality, however,
> I have some unique requirements that I am trying to work around.
> Regardless... what I am doing should work, no?  I simply want to use a
> local git repo (the directory itself hence protocol=file) instead of what
> the recipe specifies.  Looks like the fetch is working but the do_unpack
> task is ignoring PREMIRRORS (or at least the localpath variable seems
> wrong).
>
> --Russ
>
>
> On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif <rudolf.streif@ibeeto.com>
> wrote:
>
>> Russell,
>>
>> You don't need PREMIRROR for this functionality. It's not exactly
>> intended for that use.
>>
>> The simplest way to achieve what you are looking for is to use devtool.
>> If I understand you correctly you are downloading sources from a remote
>> repo on GitHub but want to have them locally to make modifications? If so
>> use:
>>
>> devtool add myrecipe localsrc fetchuri
>>
>> from your build environment (you have to source oe-init-build-env first).
>> devtool then fetches the source from fetchuri into a directory localsrc as
>> a git repo and automatically creates the recipe for it.
>>
>> :rjs
>>
>>
>> On 7/23/19 1:49 PM, Russell Peterson wrote:
>>
>> Hello,
>>
>> I am looking to have bitbake pick up files for a particular recipe from a
>> local git repository using the PREMIRROR functionality.
>>
>> Basically, the recipe (bb file) points to github but in my local build I
>> add PREMIRROR_prepend = "git://.*/.*
>> git:///local/path/BASENAME;protocol=file\n"
>>
>> I will probably make the git regular expression more exact for my
>> specific github repo but this works for now.
>>
>> This all works (as I deleted the github download from the local download
>> directory) because I can see in the do_fetch log and the correct (local)
>> repo was found and placed in the DL_DIR.
>>
>> Problem is, do_unpack fails because it appears to be looking for the
>> original (github) SRC_URI.  Then it complains about "no up to date source
>> found: clone or directory not available or not up to date (shallow clone
>> not enabled)"
>>
>> Any help on what I am missing would be appreciated.
>>
>> Regards,
>>
>> Russell
>>
>>
>>
>> --
>> -----
>> Rudolf J Streif
>> CEO/CTO ibeeto
>> +1.855.442.3396 x700
>>
>> --
> -----
> Rudolf J Streif
> CEO/CTO ibeeto
> +1.855.442.3396 x700
>
>

[-- Attachment #2: Type: text/html, Size: 7567 bytes --]

Re: PREMIRROR

[Rudolf J Streif]

[-- Attachment #1.1.1: Type: text/plain, Size: 5367 bytes --]

Russell,

That is exactly what devtool and the externalsrc class do. PREMIRROR is
the wrong approach for that.

:rjs

On 7/24/19 12:53 PM, Russell Peterson wrote:
> Hi, Rudolf.
>
> I apologize for not being clear.  The idea here is that my recipe
> points to github while, for my local development environment, I set a
> premirror to match a specific github repository and translate it to a
> local directory.  That works.  The fetch matches the PREMIRROR and
> places a copy of the LOCAL git repo in my DL_DIR.  The problem is, the
> do_unpack task is looking for the git repo in the DL_DIR
> (git2/etc...)... but it's looking for the git repo from github, not my
> local repo that the fetch task just put in DL_DIR.
>
> There are numerous reasons I'm doing this including the fact that I
> cannot put a reference to my local repo in the bb file since I ship
> that recipe to my customers and a local pathname is meaningless to
> them.  I prefer to simply modify my local.conf with a PREMIRROR value
> that has a specific regular expression that matches this particular
> github repo (and hence does NOT effect all recipes).  This is why I
> wanted to use the PREMIRROR function.  Unfortunately, it does not
> appear to work or at least not work as I expected.
>
> --Russ
>
>
>
>
> On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif
> <rudolf.streif@ibeeto.com <mailto:rudolf.streif@ibeeto.com>> wrote:
>
>     Hi Russell,
>
>     devtool and eSDK are different things. The purpose of PREMIRRORS
>     is to set a mirror for all recipes. It's a way for organizations
>     to control where their YP builds download sources from. It's not
>     intended to be used for a single recipe. There is no need for
>     that. You simply set SRC_URI in your recipe to your local git
>     repo. That is what devtool does after downloading the sources from
>     a remote repo. If you already have the remote repo cloned locally
>     you can just point devtool to it.
>
>     You can do it manually by creating your recipe and setting SRC_URI
>     like this:
>
>     SRC_URI = "git:///local/path/${PN};protocol=file"
>
>     SRCREV = "${AUTOREV}"
>
>     S = "${WORKDIR}/git"
>
>     PREMIRRORS is only relevant for do_fetch not for do_unpack.
>
>     :rjs
>
>
>     On 7/24/19 11:28 AM, Russell Peterson wrote:
>>     Hi, Rudolf.
>>
>>     Thanks for the reply.  Yes, I am aware of the eSDK functionality,
>>     however, I have some unique requirements that I am trying to work
>>     around.  Regardless... what I am doing should work, no?  I simply
>>     want to use a local git repo (the directory itself hence
>>     protocol=file) instead of what the recipe specifies.  Looks like
>>     the fetch is working but the do_unpack task is ignoring
>>     PREMIRRORS (or at least the localpath variable seems wrong).
>>
>>     --Russ
>>
>>
>>     On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif
>>     <rudolf.streif@ibeeto.com <mailto:rudolf.streif@ibeeto.com>> wrote:
>>
>>         Russell,
>>
>>         You don't need PREMIRROR for this functionality. It's not
>>         exactly intended for that use.
>>
>>         The simplest way to achieve what you are looking for is to
>>         use devtool. If I understand you correctly you are
>>         downloading sources from a remote repo on GitHub but want to
>>         have them locally to make modifications? If so use:
>>
>>         devtool add myrecipe localsrc fetchuri
>>
>>         from your build environment (you have to source
>>         oe-init-build-env first). devtool then fetches the source
>>         from fetchuri into a directory localsrc as a git repo and
>>         automatically creates the recipe for it.
>>
>>         :rjs
>>
>>
>>         On 7/23/19 1:49 PM, Russell Peterson wrote:
>>>         Hello,
>>>
>>>         I am looking to have bitbake pick up files for a particular
>>>         recipe from a local git repository using the PREMIRROR
>>>         functionality.
>>>
>>>         Basically, the recipe (bb file) points to github but in my
>>>         local build I add PREMIRROR_prepend = "git://.*/.* 
>>>         git:///local/path/BASENAME;protocol=file\n"
>>>
>>>         I will probably make the git regular expression more exact
>>>         for my specific github repo but this works for now.
>>>
>>>         This all works (as I deleted the github download from the
>>>         local download directory) because I can see in the do_fetch
>>>         log and the correct (local) repo was found and placed in the
>>>         DL_DIR.
>>>
>>>         Problem is, do_unpack fails because it appears to be looking
>>>         for the original (github) SRC_URI.  Then it complains about
>>>         "no up to date source found: clone or directory not
>>>         available or not up to date (shallow clone not enabled)"
>>>
>>>         Any help on what I am missing would be appreciated.
>>>
>>>         Regards,
>>>
>>>         Russell
>>>
>>>
>>>
>>         -- 
>>         -----
>>         Rudolf J Streif
>>         CEO/CTO ibeeto
>>         +1.855.442.3396 x700
>>
>     -- 
>     -----
>     Rudolf J Streif
>     CEO/CTO ibeeto
>     +1.855.442.3396 x700
>
-- 
-----
Rudolf J Streif
CEO/CTO ibeeto
+1.855.442.3396 x700


[-- Attachment #1.1.2: Type: text/html, Size: 9997 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 499 bytes --]

Re: PREMIRROR

[Russell Peterson]

[-- Attachment #1: Type: text/plain, Size: 6648 bytes --]

I think I have a somewhat better understanding of what is going on.

First off, I was confused by the fact that the original error message I saw
from do_unpack referenced the file (URL) at
DL_DIR/git2/original_github_url.  What I didn't understand at the time was
that while that file existed,* it was actually a link* to
DL_DIR/git2/local_url.  So, do_unpack wasn't looking at the wrong bare
repository as I thought.  It was unhappy because it didn't see the git
commit in the local repo that it was looking for.  I use AUTOREV for
SRCREV.  Apparently, even with PREMIRROR set and matched, bitbake will
still fetch the git HEAD commit hash from the original URL in the recipe.
The local git repo doesn't contain that commit.  When I sync the local git
repo to the github repo things work as expected.

Here is my problem.  What I am ultimately trying to do here is have 2 git
repos.  1 public, 1 private.  For internal development we use the private
git repo (a local gerrit server).  Just before we ship we update the public
github repo.  The recipe always points to the github (public) repo so we
don't need to mess with that before a release.  This way we can do nightly
builds (I'm just talking nightly builds done by Jenkins... not developers
using eSDK and devtool) and test.  Customers using yocto will be given a
recipe that points to github and, importantly, those customers that do NOT
use yocto simply fetch the project from github and manually build it in
their own environment with their own tools.

I can't be the only one doing this.  Is there a best practices here
(private vs. public repo)?

--Russ


On Wed, Jul 24, 2019 at 3:55 PM Rudolf J Streif <rudolf.streif@ibeeto.com>
wrote:

> Russell,
>
> That is exactly what devtool and the externalsrc class do. PREMIRROR is
> the wrong approach for that.
>
> :rjs
> On 7/24/19 12:53 PM, Russell Peterson wrote:
>
> Hi, Rudolf.
>
> I apologize for not being clear.  The idea here is that my recipe points
> to github while, for my local development environment, I set a premirror to
> match a specific github repository and translate it to a local directory.
> That works.  The fetch matches the PREMIRROR and places a copy of the LOCAL
> git repo in my DL_DIR.  The problem is, the do_unpack task is looking for
> the git repo in the DL_DIR (git2/etc...)... but it's looking for the git
> repo from github, not my local repo that the fetch task just put in DL_DIR.
>
> There are numerous reasons I'm doing this including the fact that I cannot
> put a reference to my local repo in the bb file since I ship that recipe to
> my customers and a local pathname is meaningless to them.  I prefer to
> simply modify my local.conf with a PREMIRROR value that has a specific
> regular expression that matches this particular github repo (and hence does
> NOT effect all recipes).  This is why I wanted to use the PREMIRROR
> function.  Unfortunately, it does not appear to work or at least not work
> as I expected.
>
> --Russ
>
>
>
>
> On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif <rudolf.streif@ibeeto.com>
> wrote:
>
>> Hi Russell,
>>
>> devtool and eSDK are different things. The purpose of PREMIRRORS is to
>> set a mirror for all recipes. It's a way for organizations to control where
>> their YP builds download sources from. It's not intended to be used for a
>> single recipe. There is no need for that. You simply set SRC_URI in your
>> recipe to your local git repo. That is what devtool does after downloading
>> the sources from a remote repo. If you already have the remote repo cloned
>> locally you can just point devtool to it.
>>
>> You can do it manually by creating your recipe and setting SRC_URI like
>> this:
>>
>> SRC_URI = "git:///local/path/${PN};protocol=file"
>>
>> SRCREV = "${AUTOREV}"
>>
>> S = "${WORKDIR}/git"
>>
>> PREMIRRORS is only relevant for do_fetch not for do_unpack.
>>
>> :rjs
>>
>>
>> On 7/24/19 11:28 AM, Russell Peterson wrote:
>>
>> Hi, Rudolf.
>>
>> Thanks for the reply.  Yes, I am aware of the eSDK functionality,
>> however, I have some unique requirements that I am trying to work around.
>> Regardless... what I am doing should work, no?  I simply want to use a
>> local git repo (the directory itself hence protocol=file) instead of what
>> the recipe specifies.  Looks like the fetch is working but the do_unpack
>> task is ignoring PREMIRRORS (or at least the localpath variable seems
>> wrong).
>>
>> --Russ
>>
>>
>> On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif <
>> rudolf.streif@ibeeto.com> wrote:
>>
>>> Russell,
>>>
>>> You don't need PREMIRROR for this functionality. It's not exactly
>>> intended for that use.
>>>
>>> The simplest way to achieve what you are looking for is to use devtool.
>>> If I understand you correctly you are downloading sources from a remote
>>> repo on GitHub but want to have them locally to make modifications? If so
>>> use:
>>>
>>> devtool add myrecipe localsrc fetchuri
>>>
>>> from your build environment (you have to source oe-init-build-env
>>> first). devtool then fetches the source from fetchuri into a directory
>>> localsrc as a git repo and automatically creates the recipe for it.
>>>
>>> :rjs
>>>
>>>
>>> On 7/23/19 1:49 PM, Russell Peterson wrote:
>>>
>>> Hello,
>>>
>>> I am looking to have bitbake pick up files for a particular recipe from
>>> a local git repository using the PREMIRROR functionality.
>>>
>>> Basically, the recipe (bb file) points to github but in my local build I
>>> add PREMIRROR_prepend = "git://.*/.*
>>> git:///local/path/BASENAME;protocol=file\n"
>>>
>>> I will probably make the git regular expression more exact for my
>>> specific github repo but this works for now.
>>>
>>> This all works (as I deleted the github download from the local download
>>> directory) because I can see in the do_fetch log and the correct (local)
>>> repo was found and placed in the DL_DIR.
>>>
>>> Problem is, do_unpack fails because it appears to be looking for the
>>> original (github) SRC_URI.  Then it complains about "no up to date source
>>> found: clone or directory not available or not up to date (shallow clone
>>> not enabled)"
>>>
>>> Any help on what I am missing would be appreciated.
>>>
>>> Regards,
>>>
>>> Russell
>>>
>>>
>>>
>>> --
>>> -----
>>> Rudolf J Streif
>>> CEO/CTO ibeeto
>>> +1.855.442.3396 x700
>>>
>>> --
>> -----
>> Rudolf J Streif
>> CEO/CTO ibeeto
>> +1.855.442.3396 x700
>>
>> --
> -----
> Rudolf J Streif
> CEO/CTO ibeeto
> +1.855.442.3396 x700
>
>

[-- Attachment #2: Type: text/html, Size: 12007 bytes --]

Re: PREMIRROR

[Rudolf J Streif]

[-- Attachment #1.1.1: Type: text/plain, Size: 8845 bytes --]

Inlining below.

On 7/25/19 8:14 AM, Russell Peterson wrote:
> I think I have a somewhat better understanding of what is going on.
>
> First off, I was confused by the fact that the original error message
> I saw from do_unpack referenced the file (URL) at
> DL_DIR/git2/original_github_url.  What I didn't understand at the time
> was that while that file existed,/it was actually a link/ to
> DL_DIR/git2/local_url.  So, do_unpack wasn't looking at the wrong bare
> repository as I thought.  It was unhappy because it didn't see the git
> commit in the local repo that it was looking for.  I use AUTOREV for
> SRCREV.  Apparently, even with PREMIRROR set and matched, bitbake will
> still fetch the git HEAD commit hash from the original URL in the
> recipe.  The local git repo doesn't contain that commit.  When I sync
> the local git repo to the github repo things work as expected.
Correct. If the source repo or the correct commit cannot be found from
the PREMIRROR, bitbake will use SRC_URI from the recipe. You can block
that behavior by setting BB_NO_NETWORK = "1" in which case bitbake will
not be able to connect to any remote server. However, that does not help
if your PREMIRROR is a network server. In this case use BB_ALLOWED_HOSTS
to allow access to only the servers you want.
>
> Here is my problem.  What I am ultimately trying to do here is have 2
> git repos.  1 public, 1 private.  For internal development we use the
> private git repo (a local gerrit server).  Just before we ship we
> update the public github repo.  The recipe always points to the github
> (public) repo so we don't need to mess with that before a release. 
> This way we can do nightly builds (I'm just talking nightly builds
> done by Jenkins... not developers using eSDK and devtool) and test. 
> Customers using yocto will be given a recipe that points to github
> and, importantly, those customers that do NOT use yocto simply fetch
> the project from github and manually build it in their own environment
> with their own tools.
>
That is a common problem. I do this all the time. To do this add to your
conf/local.conf file:

INHERIT += "externalsrc"
EXTERNALSRC_pn-myrecipe = "/path/to/source/tree"

Alternatively, you could use a bbappend file for you recipe in a
development layer, myrecipe.bbappend:

inherit externalsrc
EXTERNALSRC = "/path/to/source/tree"

Now, that is exactly what devtool does for you but you can of course do
it manually.

:rjs



> I can't be the only one doing this.  Is there a best practices here
> (private vs. public repo)?
>
> --Russ
>
>
> On Wed, Jul 24, 2019 at 3:55 PM Rudolf J Streif
> <rudolf.streif@ibeeto.com <mailto:rudolf.streif@ibeeto.com>> wrote:
>
>     Russell,
>
>     That is exactly what devtool and the externalsrc class do.
>     PREMIRROR is the wrong approach for that.
>
>     :rjs
>
>     On 7/24/19 12:53 PM, Russell Peterson wrote:
>>     Hi, Rudolf.
>>
>>     I apologize for not being clear.  The idea here is that my recipe
>>     points to github while, for my local development environment, I
>>     set a premirror to match a specific github repository and
>>     translate it to a local directory.  That works.  The fetch
>>     matches the PREMIRROR and places a copy of the LOCAL git repo in
>>     my DL_DIR.  The problem is, the do_unpack task is looking for the
>>     git repo in the DL_DIR (git2/etc...)... but it's looking for the
>>     git repo from github, not my local repo that the fetch task just
>>     put in DL_DIR.
>>
>>     There are numerous reasons I'm doing this including the fact that
>>     I cannot put a reference to my local repo in the bb file since I
>>     ship that recipe to my customers and a local pathname is
>>     meaningless to them.  I prefer to simply modify my local.conf
>>     with a PREMIRROR value that has a specific regular expression
>>     that matches this particular github repo (and hence does NOT
>>     effect all recipes).  This is why I wanted to use the PREMIRROR
>>     function.  Unfortunately, it does not appear to work or at least
>>     not work as I expected.
>>
>>     --Russ
>>
>>
>>
>>
>>     On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif
>>     <rudolf.streif@ibeeto.com <mailto:rudolf.streif@ibeeto.com>> wrote:
>>
>>         Hi Russell,
>>
>>         devtool and eSDK are different things. The purpose of
>>         PREMIRRORS is to set a mirror for all recipes. It's a way for
>>         organizations to control where their YP builds download
>>         sources from. It's not intended to be used for a single
>>         recipe. There is no need for that. You simply set SRC_URI in
>>         your recipe to your local git repo. That is what devtool does
>>         after downloading the sources from a remote repo. If you
>>         already have the remote repo cloned locally you can just
>>         point devtool to it.
>>
>>         You can do it manually by creating your recipe and setting
>>         SRC_URI like this:
>>
>>         SRC_URI = "git:///local/path/${PN};protocol=file"
>>
>>         SRCREV = "${AUTOREV}"
>>
>>         S = "${WORKDIR}/git"
>>
>>         PREMIRRORS is only relevant for do_fetch not for do_unpack.
>>
>>         :rjs
>>
>>
>>         On 7/24/19 11:28 AM, Russell Peterson wrote:
>>>         Hi, Rudolf.
>>>
>>>         Thanks for the reply.  Yes, I am aware of the eSDK
>>>         functionality, however, I have some unique requirements that
>>>         I am trying to work around.  Regardless... what I am doing
>>>         should work, no?  I simply want to use a local git repo (the
>>>         directory itself hence protocol=file) instead of what the
>>>         recipe specifies.  Looks like the fetch is working but the
>>>         do_unpack task is ignoring PREMIRRORS (or at least the
>>>         localpath variable seems wrong).
>>>
>>>         --Russ
>>>
>>>
>>>         On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif
>>>         <rudolf.streif@ibeeto.com <mailto:rudolf.streif@ibeeto.com>>
>>>         wrote:
>>>
>>>             Russell,
>>>
>>>             You don't need PREMIRROR for this functionality. It's
>>>             not exactly intended for that use.
>>>
>>>             The simplest way to achieve what you are looking for is
>>>             to use devtool. If I understand you correctly you are
>>>             downloading sources from a remote repo on GitHub but
>>>             want to have them locally to make modifications? If so use:
>>>
>>>             devtool add myrecipe localsrc fetchuri
>>>
>>>             from your build environment (you have to source
>>>             oe-init-build-env first). devtool then fetches the
>>>             source from fetchuri into a directory localsrc as a git
>>>             repo and automatically creates the recipe for it.
>>>
>>>             :rjs
>>>
>>>
>>>             On 7/23/19 1:49 PM, Russell Peterson wrote:
>>>>             Hello,
>>>>
>>>>             I am looking to have bitbake pick up files for a
>>>>             particular recipe from a local git repository using the
>>>>             PREMIRROR functionality.
>>>>
>>>>             Basically, the recipe (bb file) points to github but in
>>>>             my local build I add PREMIRROR_prepend = "git://.*/.* 
>>>>             git:///local/path/BASENAME;protocol=file\n"
>>>>
>>>>             I will probably make the git regular expression more
>>>>             exact for my specific github repo but this works for now.
>>>>
>>>>             This all works (as I deleted the github download from
>>>>             the local download directory) because I can see in the
>>>>             do_fetch log and the correct (local) repo was found and
>>>>             placed in the DL_DIR.
>>>>
>>>>             Problem is, do_unpack fails because it appears to be
>>>>             looking for the original (github) SRC_URI.  Then it
>>>>             complains about "no up to date source found: clone or
>>>>             directory not available or not up to date (shallow
>>>>             clone not enabled)"
>>>>
>>>>             Any help on what I am missing would be appreciated.
>>>>
>>>>             Regards,
>>>>
>>>>             Russell
>>>>
>>>>
>>>>
>>>             -- 
>>>             -----
>>>             Rudolf J Streif
>>>             CEO/CTO ibeeto
>>>             +1.855.442.3396 x700
>>>
>>         -- 
>>         -----
>>         Rudolf J Streif
>>         CEO/CTO ibeeto
>>         +1.855.442.3396 x700
>>
>     -- 
>     -----
>     Rudolf J Streif
>     CEO/CTO ibeeto
>     +1.855.442.3396 x700
>
-- 
-----
Rudolf J Streif
CEO/CTO ibeeto
+1.855.442.3396 x700


[-- Attachment #1.1.2: Type: text/html, Size: 16582 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 499 bytes --]

Re: PREMIRROR

[Russell Peterson]

[-- Attachment #1: Type: text/plain, Size: 8123 bytes --]

Just tried the externalsrc feature.  Works perfectly.  Exactly what I was
looking for.  Thanks so much, Rudolf!

--Russ


On Thu, Jul 25, 2019 at 4:59 PM Rudolf J Streif <rudolf.streif@ibeeto.com>
wrote:

> Inlining below.
> On 7/25/19 8:14 AM, Russell Peterson wrote:
>
> I think I have a somewhat better understanding of what is going on.
>
> First off, I was confused by the fact that the original error message I
> saw from do_unpack referenced the file (URL) at
> DL_DIR/git2/original_github_url.  What I didn't understand at the time was
> that while that file existed,* it was actually a link* to
> DL_DIR/git2/local_url.  So, do_unpack wasn't looking at the wrong bare
> repository as I thought.  It was unhappy because it didn't see the git
> commit in the local repo that it was looking for.  I use AUTOREV for
> SRCREV.  Apparently, even with PREMIRROR set and matched, bitbake will
> still fetch the git HEAD commit hash from the original URL in the recipe.
> The local git repo doesn't contain that commit.  When I sync the local git
> repo to the github repo things work as expected.
>
> Correct. If the source repo or the correct commit cannot be found from the
> PREMIRROR, bitbake will use SRC_URI from the recipe. You can block that
> behavior by setting BB_NO_NETWORK = "1" in which case bitbake will not be
> able to connect to any remote server. However, that does not help if your
> PREMIRROR is a network server. In this case use BB_ALLOWED_HOSTS to allow
> access to only the servers you want.
>
>
> Here is my problem.  What I am ultimately trying to do here is have 2 git
> repos.  1 public, 1 private.  For internal development we use the private
> git repo (a local gerrit server).  Just before we ship we update the public
> github repo.  The recipe always points to the github (public) repo so we
> don't need to mess with that before a release.  This way we can do nightly
> builds (I'm just talking nightly builds done by Jenkins... not developers
> using eSDK and devtool) and test.  Customers using yocto will be given a
> recipe that points to github and, importantly, those customers that do NOT
> use yocto simply fetch the project from github and manually build it in
> their own environment with their own tools.
>
> That is a common problem. I do this all the time. To do this add to your
> conf/local.conf file:
>
> INHERIT += "externalsrc"
> EXTERNALSRC_pn-myrecipe = "/path/to/source/tree"
>
> Alternatively, you could use a bbappend file for you recipe in a
> development layer, myrecipe.bbappend:
>
> inherit externalsrc
> EXTERNALSRC = "/path/to/source/tree"
>
> Now, that is exactly what devtool does for you but you can of course do it
> manually.
>
> :rjs
>
>
>
> I can't be the only one doing this.  Is there a best practices here
> (private vs. public repo)?
>
> --Russ
>
>
> On Wed, Jul 24, 2019 at 3:55 PM Rudolf J Streif <rudolf.streif@ibeeto.com>
> wrote:
>
>> Russell,
>>
>> That is exactly what devtool and the externalsrc class do. PREMIRROR is
>> the wrong approach for that.
>>
>> :rjs
>> On 7/24/19 12:53 PM, Russell Peterson wrote:
>>
>> Hi, Rudolf.
>>
>> I apologize for not being clear.  The idea here is that my recipe points
>> to github while, for my local development environment, I set a premirror to
>> match a specific github repository and translate it to a local directory.
>> That works.  The fetch matches the PREMIRROR and places a copy of the LOCAL
>> git repo in my DL_DIR.  The problem is, the do_unpack task is looking for
>> the git repo in the DL_DIR (git2/etc...)... but it's looking for the git
>> repo from github, not my local repo that the fetch task just put in DL_DIR.
>>
>> There are numerous reasons I'm doing this including the fact that I
>> cannot put a reference to my local repo in the bb file since I ship that
>> recipe to my customers and a local pathname is meaningless to them.  I
>> prefer to simply modify my local.conf with a PREMIRROR value that has a
>> specific regular expression that matches this particular github repo (and
>> hence does NOT effect all recipes).  This is why I wanted to use the
>> PREMIRROR function.  Unfortunately, it does not appear to work or at least
>> not work as I expected.
>>
>> --Russ
>>
>>
>>
>>
>> On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif <rudolf.streif@ibeeto.com>
>> wrote:
>>
>>> Hi Russell,
>>>
>>> devtool and eSDK are different things. The purpose of PREMIRRORS is to
>>> set a mirror for all recipes. It's a way for organizations to control where
>>> their YP builds download sources from. It's not intended to be used for a
>>> single recipe. There is no need for that. You simply set SRC_URI in your
>>> recipe to your local git repo. That is what devtool does after downloading
>>> the sources from a remote repo. If you already have the remote repo cloned
>>> locally you can just point devtool to it.
>>>
>>> You can do it manually by creating your recipe and setting SRC_URI like
>>> this:
>>>
>>> SRC_URI = "git:///local/path/${PN};protocol=file"
>>>
>>> SRCREV = "${AUTOREV}"
>>>
>>> S = "${WORKDIR}/git"
>>>
>>> PREMIRRORS is only relevant for do_fetch not for do_unpack.
>>>
>>> :rjs
>>>
>>>
>>> On 7/24/19 11:28 AM, Russell Peterson wrote:
>>>
>>> Hi, Rudolf.
>>>
>>> Thanks for the reply.  Yes, I am aware of the eSDK functionality,
>>> however, I have some unique requirements that I am trying to work around.
>>> Regardless... what I am doing should work, no?  I simply want to use a
>>> local git repo (the directory itself hence protocol=file) instead of what
>>> the recipe specifies.  Looks like the fetch is working but the do_unpack
>>> task is ignoring PREMIRRORS (or at least the localpath variable seems
>>> wrong).
>>>
>>> --Russ
>>>
>>>
>>> On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif <
>>> rudolf.streif@ibeeto.com> wrote:
>>>
>>>> Russell,
>>>>
>>>> You don't need PREMIRROR for this functionality. It's not exactly
>>>> intended for that use.
>>>>
>>>> The simplest way to achieve what you are looking for is to use devtool.
>>>> If I understand you correctly you are downloading sources from a remote
>>>> repo on GitHub but want to have them locally to make modifications? If so
>>>> use:
>>>>
>>>> devtool add myrecipe localsrc fetchuri
>>>>
>>>> from your build environment (you have to source oe-init-build-env
>>>> first). devtool then fetches the source from fetchuri into a directory
>>>> localsrc as a git repo and automatically creates the recipe for it.
>>>>
>>>> :rjs
>>>>
>>>>
>>>> On 7/23/19 1:49 PM, Russell Peterson wrote:
>>>>
>>>> Hello,
>>>>
>>>> I am looking to have bitbake pick up files for a particular recipe from
>>>> a local git repository using the PREMIRROR functionality.
>>>>
>>>> Basically, the recipe (bb file) points to github but in my local build
>>>> I add PREMIRROR_prepend = "git://.*/.*
>>>> git:///local/path/BASENAME;protocol=file\n"
>>>>
>>>> I will probably make the git regular expression more exact for my
>>>> specific github repo but this works for now.
>>>>
>>>> This all works (as I deleted the github download from the local
>>>> download directory) because I can see in the do_fetch log and the correct
>>>> (local) repo was found and placed in the DL_DIR.
>>>>
>>>> Problem is, do_unpack fails because it appears to be looking for the
>>>> original (github) SRC_URI.  Then it complains about "no up to date source
>>>> found: clone or directory not available or not up to date (shallow clone
>>>> not enabled)"
>>>>
>>>> Any help on what I am missing would be appreciated.
>>>>
>>>> Regards,
>>>>
>>>> Russell
>>>>
>>>>
>>>>
>>>> --
>>>> -----
>>>> Rudolf J Streif
>>>> CEO/CTO ibeeto
>>>> +1.855.442.3396 x700
>>>>
>>>> --
>>> -----
>>> Rudolf J Streif
>>> CEO/CTO ibeeto
>>> +1.855.442.3396 x700
>>>
>>> --
>> -----
>> Rudolf J Streif
>> CEO/CTO ibeeto
>> +1.855.442.3396 x700
>>
>> --
> -----
> Rudolf J Streif
> CEO/CTO ibeeto
> +1.855.442.3396 x700
>
>

[-- Attachment #2: Type: text/html, Size: 16780 bytes --]

Re: PREMIRROR

[Rudolf J Streif]

[-- Attachment #1.1.1: Type: text/plain, Size: 10302 bytes --]

You are very welcome. Enjoy working with YP.

:rjs

On 7/25/19 4:48 PM, Russell Peterson wrote:
> Just tried the externalsrc feature.  Works perfectly.  Exactly what I
> was looking for.  Thanks so much, Rudolf!
>
> --Russ
>
>
> On Thu, Jul 25, 2019 at 4:59 PM Rudolf J Streif
> <rudolf.streif@ibeeto.com <mailto:rudolf.streif@ibeeto.com>> wrote:
>
>     Inlining below.
>
>     On 7/25/19 8:14 AM, Russell Peterson wrote:
>>     I think I have a somewhat better understanding of what is going on.
>>
>>     First off, I was confused by the fact that the original error
>>     message I saw from do_unpack referenced the file (URL) at
>>     DL_DIR/git2/original_github_url.  What I didn't understand at the
>>     time was that while that file existed,/it was actually a link/ to
>>     DL_DIR/git2/local_url.  So, do_unpack wasn't looking at the wrong
>>     bare repository as I thought.  It was unhappy because it didn't
>>     see the git commit in the local repo that it was looking for.  I
>>     use AUTOREV for SRCREV.  Apparently, even with PREMIRROR set and
>>     matched, bitbake will still fetch the git HEAD commit hash from
>>     the original URL in the recipe.  The local git repo doesn't
>>     contain that commit.  When I sync the local git repo to the
>>     github repo things work as expected.
>     Correct. If the source repo or the correct commit cannot be found
>     from the PREMIRROR, bitbake will use SRC_URI from the recipe. You
>     can block that behavior by setting BB_NO_NETWORK = "1" in which
>     case bitbake will not be able to connect to any remote server.
>     However, that does not help if your PREMIRROR is a network server.
>     In this case use BB_ALLOWED_HOSTS to allow access to only the
>     servers you want.
>>
>>     Here is my problem.  What I am ultimately trying to do here is
>>     have 2 git repos.  1 public, 1 private.  For internal development
>>     we use the private git repo (a local gerrit server).  Just before
>>     we ship we update the public github repo.  The recipe always
>>     points to the github (public) repo so we don't need to mess with
>>     that before a release.  This way we can do nightly builds (I'm
>>     just talking nightly builds done by Jenkins... not developers
>>     using eSDK and devtool) and test.  Customers using yocto will be
>>     given a recipe that points to github and, importantly, those
>>     customers that do NOT use yocto simply fetch the project from
>>     github and manually build it in their own environment with their
>>     own tools.
>>
>     That is a common problem. I do this all the time. To do this add
>     to your conf/local.conf file:
>
>     INHERIT += "externalsrc"
>     EXTERNALSRC_pn-myrecipe = "/path/to/source/tree"
>
>     Alternatively, you could use a bbappend file for you recipe in a
>     development layer, myrecipe.bbappend:
>
>     inherit externalsrc
>     EXTERNALSRC = "/path/to/source/tree"
>
>     Now, that is exactly what devtool does for you but you can of
>     course do it manually.
>
>     :rjs
>
>
>
>>     I can't be the only one doing this.  Is there a best practices
>>     here (private vs. public repo)?
>>
>>     --Russ
>>
>>
>>     On Wed, Jul 24, 2019 at 3:55 PM Rudolf J Streif
>>     <rudolf.streif@ibeeto.com <mailto:rudolf.streif@ibeeto.com>> wrote:
>>
>>         Russell,
>>
>>         That is exactly what devtool and the externalsrc class do.
>>         PREMIRROR is the wrong approach for that.
>>
>>         :rjs
>>
>>         On 7/24/19 12:53 PM, Russell Peterson wrote:
>>>         Hi, Rudolf.
>>>
>>>         I apologize for not being clear.  The idea here is that my
>>>         recipe points to github while, for my local development
>>>         environment, I set a premirror to match a specific github
>>>         repository and translate it to a local directory.  That
>>>         works.  The fetch matches the PREMIRROR and places a copy of
>>>         the LOCAL git repo in my DL_DIR.  The problem is, the
>>>         do_unpack task is looking for the git repo in the DL_DIR
>>>         (git2/etc...)... but it's looking for the git repo from
>>>         github, not my local repo that the fetch task just put in
>>>         DL_DIR.
>>>
>>>         There are numerous reasons I'm doing this including the fact
>>>         that I cannot put a reference to my local repo in the bb
>>>         file since I ship that recipe to my customers and a local
>>>         pathname is meaningless to them.  I prefer to simply modify
>>>         my local.conf with a PREMIRROR value that has a specific
>>>         regular expression that matches this particular github repo
>>>         (and hence does NOT effect all recipes).  This is why I
>>>         wanted to use the PREMIRROR function.  Unfortunately, it
>>>         does not appear to work or at least not work as I expected.
>>>
>>>         --Russ
>>>
>>>
>>>
>>>
>>>         On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif
>>>         <rudolf.streif@ibeeto.com <mailto:rudolf.streif@ibeeto.com>>
>>>         wrote:
>>>
>>>             Hi Russell,
>>>
>>>             devtool and eSDK are different things. The purpose of
>>>             PREMIRRORS is to set a mirror for all recipes. It's a
>>>             way for organizations to control where their YP builds
>>>             download sources from. It's not intended to be used for
>>>             a single recipe. There is no need for that. You simply
>>>             set SRC_URI in your recipe to your local git repo. That
>>>             is what devtool does after downloading the sources from
>>>             a remote repo. If you already have the remote repo
>>>             cloned locally you can just point devtool to it.
>>>
>>>             You can do it manually by creating your recipe and
>>>             setting SRC_URI like this:
>>>
>>>             SRC_URI = "git:///local/path/${PN};protocol=file"
>>>
>>>             SRCREV = "${AUTOREV}"
>>>
>>>             S = "${WORKDIR}/git"
>>>
>>>             PREMIRRORS is only relevant for do_fetch not for do_unpack.
>>>
>>>             :rjs
>>>
>>>
>>>             On 7/24/19 11:28 AM, Russell Peterson wrote:
>>>>             Hi, Rudolf.
>>>>
>>>>             Thanks for the reply.  Yes, I am aware of the eSDK
>>>>             functionality, however, I have some unique requirements
>>>>             that I am trying to work around.  Regardless... what I
>>>>             am doing should work, no?  I simply want to use a local
>>>>             git repo (the directory itself hence protocol=file)
>>>>             instead of what the recipe specifies.  Looks like the
>>>>             fetch is working but the do_unpack task is ignoring
>>>>             PREMIRRORS (or at least the localpath variable seems
>>>>             wrong).
>>>>
>>>>             --Russ
>>>>
>>>>
>>>>             On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif
>>>>             <rudolf.streif@ibeeto.com
>>>>             <mailto:rudolf.streif@ibeeto.com>> wrote:
>>>>
>>>>                 Russell,
>>>>
>>>>                 You don't need PREMIRROR for this functionality.
>>>>                 It's not exactly intended for that use.
>>>>
>>>>                 The simplest way to achieve what you are looking
>>>>                 for is to use devtool. If I understand you
>>>>                 correctly you are downloading sources from a remote
>>>>                 repo on GitHub but want to have them locally to
>>>>                 make modifications? If so use:
>>>>
>>>>                 devtool add myrecipe localsrc fetchuri
>>>>
>>>>                 from your build environment (you have to source
>>>>                 oe-init-build-env first). devtool then fetches the
>>>>                 source from fetchuri into a directory localsrc as a
>>>>                 git repo and automatically creates the recipe for it.
>>>>
>>>>                 :rjs
>>>>
>>>>
>>>>                 On 7/23/19 1:49 PM, Russell Peterson wrote:
>>>>>                 Hello,
>>>>>
>>>>>                 I am looking to have bitbake pick up files for a
>>>>>                 particular recipe from a local git repository
>>>>>                 using the PREMIRROR functionality.
>>>>>
>>>>>                 Basically, the recipe (bb file) points to github
>>>>>                 but in my local build I add PREMIRROR_prepend =
>>>>>                 "git://.*/.* 
>>>>>                 git:///local/path/BASENAME;protocol=file\n"
>>>>>
>>>>>                 I will probably make the git regular expression
>>>>>                 more exact for my specific github repo but this
>>>>>                 works for now.
>>>>>
>>>>>                 This all works (as I deleted the github download
>>>>>                 from the local download directory) because I can
>>>>>                 see in the do_fetch log and the correct (local)
>>>>>                 repo was found and placed in the DL_DIR.
>>>>>
>>>>>                 Problem is, do_unpack fails because it appears to
>>>>>                 be looking for the original (github) SRC_URI. 
>>>>>                 Then it complains about "no up to date source
>>>>>                 found: clone or directory not available or not up
>>>>>                 to date (shallow clone not enabled)"
>>>>>
>>>>>                 Any help on what I am missing would be appreciated.
>>>>>
>>>>>                 Regards,
>>>>>
>>>>>                 Russell
>>>>>
>>>>>
>>>>>
>>>>                 -- 
>>>>                 -----
>>>>                 Rudolf J Streif
>>>>                 CEO/CTO ibeeto
>>>>                 +1.855.442.3396 x700
>>>>
>>>             -- 
>>>             -----
>>>             Rudolf J Streif
>>>             CEO/CTO ibeeto
>>>             +1.855.442.3396 x700
>>>
>>         -- 
>>         -----
>>         Rudolf J Streif
>>         CEO/CTO ibeeto
>>         +1.855.442.3396 x700
>>
>     -- 
>     -----
>     Rudolf J Streif
>     CEO/CTO ibeeto
>     +1.855.442.3396 x700
>
-- 
-----
Rudolf J Streif
CEO/CTO ibeeto
+1.855.442.3396 x700


[-- Attachment #1.1.2: Type: text/html, Size: 21228 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 499 bytes --]