gitsm fetcher does not work anymore

gitsm fetcher does not work anymore

[Florian Wickert]

Hi all,

relative URL schemes (e.g. submodules in 
https://github.com/xiph/Icecast-libshout) or SSH URLs (e.g. 
git@github.com:user/example.git) are not parsed correctly since the 
rework done in 5cd00e3e53819f3c10d1733480077ce9f3cc2ca8.
The SSH scheme is required for private repos on github because https 
doesn't work with SSH keys.

Regards,
Florian

-- 
Florian Wickert, M.Sc.
Javox Solutions GmbH
Gallierstr. 33, 52074 Aachen
Germany

Phone: +49 241 99034569
Web:www.javox-solutions.com

Re: gitsm fetcher does not work anymore

[Olof Johansson]

On 18-10-02 18:24 +0200, Florian Wickert wrote:
> The SSH scheme is required for private repos on github because https doesn't
> work with SSH keys.

SSH keys have never worked for https. You have to authenticate in
other ways, like authentication tokens. But saying that private
repos *require* ssh transport is not true.

See for instance https://gist.github.com/technoweenie/1072829 for
how to use ~/.netrc to do this in non-interactive environments
like OE (or when you just can't stand having to enter your
password over and over).

-- 
olofjn

Re: gitsm fetcher does not work anymore

[Mark Hatle]

On 10/2/18 11:24 AM, Florian Wickert wrote:
> Hi all,
> 
> relative URL schemes (e.g. submodules in 
> https://github.com/xiph/Icecast-libshout) or SSH URLs (e.g. 
> git@github.com:user/example.git) are not parsed correctly since the 

git@github.com:user/example.git that is not a valid URL.

ssh://git@github.com/user/example.git would be.

Is this an issue where it's specified inside of the git repository as a non-url
string?  If so, we'll have to learn how to recognize that and convert it
'somehow' to something that can be recognized as a URL.

(Does the git fetcher support ssh uris?)

--Mark

> rework done in 5cd00e3e53819f3c10d1733480077ce9f3cc2ca8.
> The SSH scheme is required for private repos on github because https 
> doesn't work with SSH keys.
> 
> Regards,
> Florian
> 

Re: gitsm fetcher does not work anymore

[Mark Hatle]

On 10/2/18 11:24 AM, Florian Wickert wrote:
> Hi all,
> 
> relative URL schemes (e.g. submodules in 
> https://github.com/xiph/Icecast-libshout) or SSH URLs (e.g. 

URIs in question are of the format:

[submodule "m4"]
	path = m4
	url = ../icecast-m4.git
[submodule "src/common"]
	path = src/common
	url = ../icecast-common.git

While I'm somewhat surprised this is valid, this should be something we can resolve.

(yes I intentionally replied to the two concerns with different emails, as I see
them as different problems.)

--Mark

> git@github.com:user/example.git) are not parsed correctly since the 
> rework done in 5cd00e3e53819f3c10d1733480077ce9f3cc2ca8.
> The SSH scheme is required for private repos on github because https 
> doesn't work with SSH keys.
> 
> Regards,
> Florian
> 

Re: gitsm fetcher does not work anymore

[Mark Hatle]

On 10/2/18 12:14 PM, Mark Hatle wrote:
> On 10/2/18 11:24 AM, Florian Wickert wrote:
>> Hi all,
>>
>> relative URL schemes (e.g. submodules in 
>> https://github.com/xiph/Icecast-libshout) or SSH URLs (e.g. 
>> git@github.com:user/example.git) are not parsed correctly since the 
> 
> git@github.com:user/example.git that is not a valid URL.
> 
> ssh://git@github.com/user/example.git would be.
> 
> Is this an issue where it's specified inside of the git repository as a non-url
> string?  If so, we'll have to learn how to recognize that and convert it
> 'somehow' to something that can be recognized as a URL.
> 
> (Does the git fetcher support ssh uris?)

FYI, the code that makes this work (for URIs) is:

http://git.openembedded.org/bitbake/tree/lib/bb/fetch2/gitsm.py

Lines 76+

            # Build new SRC_URI
            proto = uris[module].split(':', 1)[0]
            url = uris[module].replace('%s:' % proto, 'gitsm:', 1)
            url += ';protocol=%s' % proto

It splits at the :, since a proper URL should be <proto>://<path>  The it makes
the new URL into gitsm://<path>;protocol=<proto>

If if supporting the non-URI format of an SSH git is needed, then something will
need to be specifically added to this section to recognize and convert the URL
to a URL formatted configuration.

--Mark

> --Mark
> 
>> rework done in 5cd00e3e53819f3c10d1733480077ce9f3cc2ca8.
>> The SSH scheme is required for private repos on github because https 
>> doesn't work with SSH keys.
>>
>> Regards,
>> Florian
>>
> 

Re: gitsm fetcher does not work anymore

[Mark Hatle]

On 10/2/18 12:15 PM, Mark Hatle wrote:
> On 10/2/18 11:24 AM, Florian Wickert wrote:
>> Hi all,
>>
>> relative URL schemes (e.g. submodules in 
>> https://github.com/xiph/Icecast-libshout) or SSH URLs (e.g. 
> 
> URIs in question are of the format:
> 
> [submodule "m4"]
> 	path = m4
> 	url = ../icecast-m4.git
> [submodule "src/common"]
> 	path = src/common
> 	url = ../icecast-common.git
> 
> While I'm somewhat surprised this is valid, this should be something we can resolve.
> 
> (yes I intentionally replied to the two concerns with different emails, as I see
> them as different problems.)

Ok, I have a possible fix:

git://git.openembedded.org/bitbake-contrib mgh/gitsm

http://git.openembedded.org/bitbake-contrib/commit/?h=mgh/gitsm&id=f450ac0051f56683d43425d00e349c23ce6a16e4

Please try that and let me know if it resolves your issue.

--Mark

> --Mark
> 
>> git@github.com:user/example.git) are not parsed correctly since the 
>> rework done in 5cd00e3e53819f3c10d1733480077ce9f3cc2ca8.
>> The SSH scheme is required for private repos on github because https 
>> doesn't work with SSH keys.
>>
>> Regards,
>> Florian
>>
> 

Re: gitsm fetcher does not work anymore

[Olof Johansson]

On 18-10-02 12:14 -0500, Mark Hatle wrote:
> (Does the git fetcher support ssh uris?)

Well, not an ssh:// uri, but

  git://example.com/repo.git;protocol=ssh

-- 
olofjn