[PATCH] Gitsm does not use credentials when cloning submodules

[PATCH] Gitsm does not use credentials when cloning submodules

[Bernd Bauer]

If the git server needs authentication the credentials are
encoded in the URL of the repositories.
The credentials from the main repository are deposited in the
recipe so main repository can still be cloned.

But the URLs from the submodules are listed in the .gitmodules
file.
The submodule url is encoded with an username and a password
or a bitbucket URL token. See example.

The password is not stored in the ud.user but in the ud.pswd
variable. So in this case the url is made up by ud.user and
ud.pswd.

Example:
[submodule "sub1"]
        path = sub1
        url = https://user:password@test.org/url/to/sub1

Signed-off-by: Bernd Bauer <bernd.bauer@gmx.at>
---
 bitbake/lib/bb/fetch2/git.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py
index 5b3793a705..dc8f37b6c3 100644
--- a/bitbake/lib/bb/fetch2/git.py
+++ b/bitbake/lib/bb/fetch2/git.py
@@ -584,7 +584,9 @@ class Git(FetchMethod):
         """
         Return the repository URL
         """
-        if ud.user:
+        if ud.user and ud.pswd:
+            username = ud.user + ':' + ud.pswd + '@'
+        elif ud.user:
             username = ud.user + '@'
         else:
             username = ""
--
2.17.1

Re: [bitbake-devel] [PATCH] Gitsm does not use credentials when cloning submodules

[Richard Purdie]

On Thu, 2020-04-02 at 21:51 +0200, Bernd Bauer wrote:
> If the git server needs authentication the credentials are
> encoded in the URL of the repositories.
> The credentials from the main repository are deposited in the
> recipe so main repository can still be cloned.
> 
> But the URLs from the submodules are listed in the .gitmodules
> file.
> The submodule url is encoded with an username and a password
> or a bitbucket URL token. See example.
> 
> The password is not stored in the ud.user but in the ud.pswd
> variable. So in this case the url is made up by ud.user and
> ud.pswd.
> 
> Example:
> [submodule "sub1"]
>         path = sub1
>         url = https://user:password@test.org/url/to/sub1
> 
> Signed-off-by: Bernd Bauer <bernd.bauer@gmx.at>
> ---
>  bitbake/lib/bb/fetch2/git.py | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)

We have had requests for this before and we've said "no" since
encouraging people to put passwords in urls is bad and it leads to
people making accidental exposures. Its basically too risky.

For git submodules the line becomes blurred.

I'm open to the opinions of others but in this form the patch applies
to git and gitsm and that in itself makes it more risky.

I do understand why people want this, equally it really isn't a great
idea in general :/.

Cheers,

Richard

Re: [bitbake-devel] [PATCH] Gitsm does not use credentials when cloning submodules

[Bernd Bauer]

Am 02.04.20 um 23:08 schrieb Richard Purdie:
> On Thu, 2020-04-02 at 21:51 +0200, Bernd Bauer wrote:
>> If the git server needs authentication the credentials are
>> encoded in the URL of the repositories.
>> The credentials from the main repository are deposited in the
>> recipe so main repository can still be cloned.
>>
>> But the URLs from the submodules are listed in the .gitmodules
>> file.
>> The submodule url is encoded with an username and a password
>> or a bitbucket URL token. See example.
>>
>> The password is not stored in the ud.user but in the ud.pswd
>> variable. So in this case the url is made up by ud.user and
>> ud.pswd.
>>
>> Example:
>> [submodule "sub1"]
>>         path = sub1
>>         url = https://user:password@test.org/url/to/sub1
>>
>> Signed-off-by: Bernd Bauer <bernd.bauer@gmx.at>
>> ---
>>  bitbake/lib/bb/fetch2/git.py | 4 +++-
>>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> We have had requests for this before and we've said "no" since
> encouraging people to put passwords in urls is bad and it leads to
> people making accidental exposures. Its basically too risky.
>
> For git submodules the line becomes blurred.
>
> I'm open to the opinions of others but in this form the patch applies
> to git and gitsm and that in itself makes it more risky.
>
> I do understand why people want this, equally it really isn't a great
> idea in general :/.
>
> Cheers,
>
> Richard

Hi!

Of course it is not a good idea to write the credentials directly
in the recipe or in the .gitmodules.
Unfortunately I have no other choice than to encode the username
and password in the URL. This GIT server only allows HTTPS with
authentication. No SSH keys, no anonymous checkout.

But without my patch the result would be the same.
I would have the URL with username and password in the
recipe and also in the submodules.
The only difference would be that I would have to define a
task in the recipe to check out the submodules.
So my recipe is getting complexer.

do_configure_prepend() {
  cd ${WORKDIR}/git
  git submodules update --init --recursive
}

From my point of view, my patch would be more like
fixing a bug than an extension.

Just my two cents...

Greets Bernd

Re: [bitbake-devel] [PATCH] Gitsm does not use credentials when cloning submodules

[Khem Raj]

On 4/3/20 12:29 AM, Bernd Bauer wrote:
> Am 02.04.20 um 23:08 schrieb Richard Purdie:
>> On Thu, 2020-04-02 at 21:51 +0200, Bernd Bauer wrote:
>>> If the git server needs authentication the credentials are
>>> encoded in the URL of the repositories.
>>> The credentials from the main repository are deposited in the
>>> recipe so main repository can still be cloned.
>>>
>>> But the URLs from the submodules are listed in the .gitmodules
>>> file.
>>> The submodule url is encoded with an username and a password
>>> or a bitbucket URL token. See example.
>>>
>>> The password is not stored in the ud.user but in the ud.pswd
>>> variable. So in this case the url is made up by ud.user and
>>> ud.pswd.
>>>
>>> Example:
>>> [submodule "sub1"]
>>>          path = sub1
>>>          url = https://user:password@test.org/url/to/sub1
>>>
>>> Signed-off-by: Bernd Bauer <bernd.bauer@gmx.at>
>>> ---
>>>   bitbake/lib/bb/fetch2/git.py | 4 +++-
>>>   1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> We have had requests for this before and we've said "no" since
>> encouraging people to put passwords in urls is bad and it leads to
>> people making accidental exposures. Its basically too risky.
>>
>> For git submodules the line becomes blurred.
>>
>> I'm open to the opinions of others but in this form the patch applies
>> to git and gitsm and that in itself makes it more risky.
>>
>> I do understand why people want this, equally it really isn't a great
>> idea in general :/.
>>
>> Cheers,
>>
>> Richard
> 
> Hi!
> 
> Of course it is not a good idea to write the credentials directly
> in the recipe or in the .gitmodules.
> Unfortunately I have no other choice than to encode the username
> and password in the URL. This GIT server only allows HTTPS with
> authentication. No SSH keys, no anonymous checkout.

perhaps you should talk to your infosec about this.
this is unusual setup, using key based authentication is far better than 
username/password mechanism. but they might have their own reasons so 
basically work out with IT and infosec to allow right kind of access 
will get a long lasting solution

> 
> But without my patch the result would be the same.
> I would have the URL with username and password in the
> recipe and also in the submodules.
> The only difference would be that I would have to define a
> task in the recipe to check out the submodules.
> So my recipe is getting complexer.
> 
> do_configure_prepend() {
>    cd ${WORKDIR}/git
>    git submodules update --init --recursive
> }
> 
>  From my point of view, my patch would be more like
> fixing a bug than an extension.

> 
> Just my two cents...
> 
> Greets Bernd
> 
> 
> 
> 

Re: [bitbake-devel] [PATCH] Gitsm does not use credentials when cloning submodules

[Andre McCurdy]

On Fri, Apr 3, 2020 at 12:29 AM Bernd Bauer <bernd.bauer@gmx.at> wrote:
> Am 02.04.20 um 23:08 schrieb Richard Purdie:
> > On Thu, 2020-04-02 at 21:51 +0200, Bernd Bauer wrote:
> >> If the git server needs authentication the credentials are
> >> encoded in the URL of the repositories.
> >> The credentials from the main repository are deposited in the
> >> recipe so main repository can still be cloned.
> >>
> >> But the URLs from the submodules are listed in the .gitmodules
> >> file.
> >> The submodule url is encoded with an username and a password
> >> or a bitbucket URL token. See example.
> >>
> >> The password is not stored in the ud.user but in the ud.pswd
> >> variable. So in this case the url is made up by ud.user and
> >> ud.pswd.
> >>
> >> Example:
> >> [submodule "sub1"]
> >>         path = sub1
> >>         url = https://user:password@test.org/url/to/sub1
> >>
> >> Signed-off-by: Bernd Bauer <bernd.bauer@gmx.at>
> >> ---
> >>  bitbake/lib/bb/fetch2/git.py | 4 +++-
> >>  1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > We have had requests for this before and we've said "no" since
> > encouraging people to put passwords in urls is bad and it leads to
> > people making accidental exposures. Its basically too risky.
> >
> > For git submodules the line becomes blurred.
> >
> > I'm open to the opinions of others but in this form the patch applies
> > to git and gitsm and that in itself makes it more risky.
> >
> > I do understand why people want this, equally it really isn't a great
> > idea in general :/.
> >
> > Cheers,
> >
> > Richard
>
> Hi!
>
> Of course it is not a good idea to write the credentials directly
> in the recipe or in the .gitmodules.
> Unfortunately I have no other choice than to encode the username
> and password in the URL. This GIT server only allows HTTPS with
> authentication. No SSH keys, no anonymous checkout.

Google brings up a few possible alternatives:

  https://confluence.atlassian.com/bitbucketserver/permanently-authenticating-with-git-repositories-776639846.html

I also need to access a git server which supports only https with
authentication and .netrc has always worked fine for me.

> But without my patch the result would be the same.
> I would have the URL with username and password in the
> recipe and also in the submodules.
> The only difference would be that I would have to define a
> task in the recipe to check out the submodules.
> So my recipe is getting complexer.
>
> do_configure_prepend() {
>   cd ${WORKDIR}/git
>   git submodules update --init --recursive
> }
>
> From my point of view, my patch would be more like
> fixing a bug than an extension.
>
> Just my two cents...
>
> Greets Bernd
> 

Re: [bitbake-devel] [PATCH] Gitsm does not use credentials when cloning submodules

[Andre McCurdy]

On Fri, Apr 3, 2020 at 10:21 AM Khem Raj <raj.khem@gmail.com> wrote:
> On 4/3/20 12:29 AM, Bernd Bauer wrote:
> > Am 02.04.20 um 23:08 schrieb Richard Purdie:
> >> On Thu, 2020-04-02 at 21:51 +0200, Bernd Bauer wrote:
> >>> If the git server needs authentication the credentials are
> >>> encoded in the URL of the repositories.
> >>> The credentials from the main repository are deposited in the
> >>> recipe so main repository can still be cloned.
> >>>
> >>> But the URLs from the submodules are listed in the .gitmodules
> >>> file.
> >>> The submodule url is encoded with an username and a password
> >>> or a bitbucket URL token. See example.
> >>>
> >>> The password is not stored in the ud.user but in the ud.pswd
> >>> variable. So in this case the url is made up by ud.user and
> >>> ud.pswd.
> >>>
> >>> Example:
> >>> [submodule "sub1"]
> >>>          path = sub1
> >>>          url = https://user:password@test.org/url/to/sub1
> >>>
> >>> Signed-off-by: Bernd Bauer <bernd.bauer@gmx.at>
> >>> ---
> >>>   bitbake/lib/bb/fetch2/git.py | 4 +++-
> >>>   1 file changed, 3 insertions(+), 1 deletion(-)
> >>
> >> We have had requests for this before and we've said "no" since
> >> encouraging people to put passwords in urls is bad and it leads to
> >> people making accidental exposures. Its basically too risky.
> >>
> >> For git submodules the line becomes blurred.
> >>
> >> I'm open to the opinions of others but in this form the patch applies
> >> to git and gitsm and that in itself makes it more risky.
> >>
> >> I do understand why people want this, equally it really isn't a great
> >> idea in general :/.
> >>
> >> Cheers,
> >>
> >> Richard
> >
> > Hi!
> >
> > Of course it is not a good idea to write the credentials directly
> > in the recipe or in the .gitmodules.
> > Unfortunately I have no other choice than to encode the username
> > and password in the URL. This GIT server only allows HTTPS with
> > authentication. No SSH keys, no anonymous checkout.
>
> perhaps you should talk to your infosec about this.
> this is unusual setup, using key based authentication is far better than
> username/password mechanism.

Unfortunately there are some projects which haven't got that message.
Are you familiar with the RDK?

> > But without my patch the result would be the same.
> > I would have the URL with username and password in the
> > recipe and also in the submodules.
> > The only difference would be that I would have to define a
> > task in the recipe to check out the submodules.
> > So my recipe is getting complexer.
> >
> > do_configure_prepend() {
> >    cd ${WORKDIR}/git
> >    git submodules update --init --recursive
> > }
> >
> >  From my point of view, my patch would be more like
> > fixing a bug than an extension.
>
> >
> > Just my two cents...
> >
> > Greets Bernd
> >
> >
> >
> >
> 

Re: [bitbake-devel] [PATCH] Gitsm does not use credentials when cloning submodules

[Denys Dmytriyenko]

On Sun, Apr 05, 2020 at 02:51:02PM -0700, Andre McCurdy wrote:
> On Fri, Apr 3, 2020 at 10:21 AM Khem Raj <raj.khem@gmail.com> wrote:
> > On 4/3/20 12:29 AM, Bernd Bauer wrote:
> > > Am 02.04.20 um 23:08 schrieb Richard Purdie:
> > >> On Thu, 2020-04-02 at 21:51 +0200, Bernd Bauer wrote:
> > >>> If the git server needs authentication the credentials are
> > >>> encoded in the URL of the repositories.
> > >>> The credentials from the main repository are deposited in the
> > >>> recipe so main repository can still be cloned.
> > >>>
> > >>> But the URLs from the submodules are listed in the .gitmodules
> > >>> file.
> > >>> The submodule url is encoded with an username and a password
> > >>> or a bitbucket URL token. See example.
> > >>>
> > >>> The password is not stored in the ud.user but in the ud.pswd
> > >>> variable. So in this case the url is made up by ud.user and
> > >>> ud.pswd.
> > >>>
> > >>> Example:
> > >>> [submodule "sub1"]
> > >>>          path = sub1
> > >>>          url = https://user:password@test.org/url/to/sub1
> > >>>
> > >>> Signed-off-by: Bernd Bauer <bernd.bauer@gmx.at>
> > >>> ---
> > >>>   bitbake/lib/bb/fetch2/git.py | 4 +++-
> > >>>   1 file changed, 3 insertions(+), 1 deletion(-)
> > >>
> > >> We have had requests for this before and we've said "no" since
> > >> encouraging people to put passwords in urls is bad and it leads to
> > >> people making accidental exposures. Its basically too risky.
> > >>
> > >> For git submodules the line becomes blurred.
> > >>
> > >> I'm open to the opinions of others but in this form the patch applies
> > >> to git and gitsm and that in itself makes it more risky.
> > >>
> > >> I do understand why people want this, equally it really isn't a great
> > >> idea in general :/.
> > >>
> > >> Cheers,
> > >>
> > >> Richard
> > >
> > > Hi!
> > >
> > > Of course it is not a good idea to write the credentials directly
> > > in the recipe or in the .gitmodules.
> > > Unfortunately I have no other choice than to encode the username
> > > and password in the URL. This GIT server only allows HTTPS with
> > > authentication. No SSH keys, no anonymous checkout.
> >
> > perhaps you should talk to your infosec about this.
> > this is unusual setup, using key based authentication is far better than
> > username/password mechanism.
> 
> Unfortunately there are some projects which haven't got that message.
> Are you familiar with the RDK?

LOL


> > > But without my patch the result would be the same.
> > > I would have the URL with username and password in the
> > > recipe and also in the submodules.
> > > The only difference would be that I would have to define a
> > > task in the recipe to check out the submodules.
> > > So my recipe is getting complexer.
> > >
> > > do_configure_prepend() {
> > >    cd ${WORKDIR}/git
> > >    git submodules update --init --recursive
> > > }
> > >
> > >  From my point of view, my patch would be more like
> > > fixing a bug than an extension.
> >
> > >
> > > Just my two cents...
> > >
> > > Greets Bernd
> > >
> > >
> > >
> > >
> > 

>