All of lore.kernel.org
 help / color / mirror / Atom feed
* Re: [bitbake-devel] [PATCH 1/2] fetch/git: Handle github dropping git:// support
       [not found] <16B3BB5E65837842.31260@lists.openembedded.org>
@ 2021-11-03 14:13 ` Richard Purdie
  2021-11-09 15:40   ` gabriel.valcazar
  0 siblings, 1 reply; 14+ messages in thread
From: Richard Purdie @ 2021-11-03 14:13 UTC (permalink / raw)
  To: bitbake-devel

On Tue, 2021-11-02 at 12:44 +0000, Richard Purdie via lists.openembedded.org
wrote:
> github is dropping support for git procotol iun git urls. Add code to remap
> this to https in a way that could be used in older bitbake versions.
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  lib/bb/fetch2/git.py | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
> index e974445fbe..f5be24d6ab 100644
> --- a/lib/bb/fetch2/git.py
> +++ b/lib/bb/fetch2/git.py
> @@ -142,6 +142,10 @@ class Git(FetchMethod):
>              ud.proto = 'file'
>          else:
>              ud.proto = "git"
> +        if ud.host == "github.com" and ud.proto == "git":
> +            # github stopped supporting git protocol
> +            # https://github.blog/2021-09-01-improving-git-protocol-security-github/#no-more-unauthenticated-git
> +            ud.proto = "https"
>  
>          if not ud.proto in ('git', 'file', 'ssh', 'http', 'https', 'rsync'):
>              raise bb.fetch2.ParameterError("Invalid protocol type", ud.url)

To make things a bit easier for everyone, I did backport this to a number of the
older bitbake branches back to 1.40.

Cheers,

Richard



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/2] fetch/git: Handle github dropping git:// support
  2021-11-03 14:13 ` [bitbake-devel] [PATCH 1/2] fetch/git: Handle github dropping git:// support Richard Purdie
@ 2021-11-09 15:40   ` gabriel.valcazar
  2021-11-09 15:44     ` [bitbake-devel] " Alexander Kanavin
  2021-11-09 15:47     ` Martin Jansa
  0 siblings, 2 replies; 14+ messages in thread
From: gabriel.valcazar @ 2021-11-09 15:40 UTC (permalink / raw)
  To: bitbake-devel

[-- Attachment #1: Type: text/plain, Size: 891 bytes --]

Hi Richard,

Are there any plans to backport this fix to older versions of bitbake/poky? My company has a line of legacy embedded products that are only supported in older versions of our Yocto distribution (1.3 in the most extreme case), and although we can manually change the protocol from git to https in the recipes in our layer, we can't do the same for other third party layers our distribution depends on (meta-qt5, meta-freescale, poky...). Since backporting the fix to older bitbake versions is trivial, do you intend to incorporate it in older bibake/poky versions in the near future?

Otherwise, are there any alternative approaches we can take to ensure that users that depend on our legacy products can continue to build our Yocto distribution? Upgrading our entire legacy line to newer Yocto versions is not a viable solution for us.

Thanks and best regards,
Gabriel

[-- Attachment #2: Type: text/html, Size: 919 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [bitbake-devel] [PATCH 1/2] fetch/git: Handle github dropping git:// support
  2021-11-09 15:40   ` gabriel.valcazar
@ 2021-11-09 15:44     ` Alexander Kanavin
  2021-11-09 15:47     ` Martin Jansa
  1 sibling, 0 replies; 14+ messages in thread
From: Alexander Kanavin @ 2021-11-09 15:44 UTC (permalink / raw)
  To: gabriel.valcazar; +Cc: bitbake-devel

[-- Attachment #1: Type: text/plain, Size: 1674 bytes --]

If the versions are very old (anything older than dunfell really), I think
the only viable path for you is to make private copies of respective yocto
repositories, and patch them as needed.

Alex

On Tue, 9 Nov 2021 at 16:40, gabriel.valcazar via lists.openembedded.org
<gabriel.valcazar=digi.com@lists.openembedded.org> wrote:

> Hi Richard,
>
> Are there any plans to backport this fix to older versions of
> bitbake/poky? My company has a line of legacy embedded products that are
> only supported in older versions of our Yocto distribution (1.3 in the most
> extreme case), and although we can manually change the protocol from git to
> https in the recipes in our layer, we can't do the same for other third
> party layers our distribution depends on (meta-qt5, meta-freescale,
> poky...). Since backporting the fix to older bitbake versions is trivial,
> do you intend to incorporate it in older bibake/poky versions in the near
> future?
>
> Otherwise, are there any alternative approaches we can take to ensure that
> users that depend on our legacy products can continue to build our Yocto
> distribution? Upgrading our entire legacy line to newer Yocto versions is
> not a viable solution for us.
>
> Thanks and best regards,
> Gabriel
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12982):
> https://lists.openembedded.org/g/bitbake-devel/message/12982
> Mute This Topic: https://lists.openembedded.org/mt/86764631/1686489
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
> alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 2511 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [bitbake-devel] [PATCH 1/2] fetch/git: Handle github dropping git:// support
  2021-11-09 15:40   ` gabriel.valcazar
  2021-11-09 15:44     ` [bitbake-devel] " Alexander Kanavin
@ 2021-11-09 15:47     ` Martin Jansa
  2021-11-10 15:15       ` gabriel.valcazar
  1 sibling, 1 reply; 14+ messages in thread
From: Martin Jansa @ 2021-11-09 15:47 UTC (permalink / raw)
  To: gabriel.valcazar; +Cc: bitbake-devel

[-- Attachment #1: Type: text/plain, Size: 1932 bytes --]

FWIW: It was already backported all the way to 1.40 version used in Yocto
2.6 Thud:
https://git.openembedded.org/bitbake/log/?h=1.40

All the way till 1.16 version used in Yocto 1.3 Danny, is still quite long.
Maybe you can backport the bitbake patch on your end? I would be surprised
if you don't have bunch of various other changes backported just to keep
these ancient builds going _and_ reliable PREMIRROR to never access
upstream repos from 2012.

On Tue, Nov 9, 2021 at 4:40 PM gabriel.valcazar via lists.openembedded.org
<gabriel.valcazar=digi.com@lists.openembedded.org> wrote:

> Hi Richard,
>
> Are there any plans to backport this fix to older versions of
> bitbake/poky? My company has a line of legacy embedded products that are
> only supported in older versions of our Yocto distribution (1.3 in the most
> extreme case), and although we can manually change the protocol from git to
> https in the recipes in our layer, we can't do the same for other third
> party layers our distribution depends on (meta-qt5, meta-freescale,
> poky...). Since backporting the fix to older bitbake versions is trivial,
> do you intend to incorporate it in older bibake/poky versions in the near
> future?
>
> Otherwise, are there any alternative approaches we can take to ensure that
> users that depend on our legacy products can continue to build our Yocto
> distribution? Upgrading our entire legacy line to newer Yocto versions is
> not a viable solution for us.
>
> Thanks and best regards,
> Gabriel
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12982):
> https://lists.openembedded.org/g/bitbake-devel/message/12982
> Mute This Topic: https://lists.openembedded.org/mt/86764631/3617156
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
> Martin.Jansa@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 2831 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/2] fetch/git: Handle github dropping git:// support
  2021-11-09 15:47     ` Martin Jansa
@ 2021-11-10 15:15       ` gabriel.valcazar
  2021-11-11 11:03         ` [bitbake-devel] " Richard Purdie
  0 siblings, 1 reply; 14+ messages in thread
From: gabriel.valcazar @ 2021-11-10 15:15 UTC (permalink / raw)
  To: bitbake-devel

[-- Attachment #1: Type: text/plain, Size: 508 bytes --]

Thanks all for the suggestions. Indeed, creating a fork of poky and applying the fix ourselves seems like a reasonable solution if there is no official fix for old versions.

I've just tested applying the patch to *bitbake 1.18* , and I'm able to build my Yocto 1.4 distribution with no problem. Is it not possible to officially backport the patch all the way back to bitbake 1.18? I could provide the patches, but they're literally just clean cherry-picks of the bitbake 1.40 patch.

Regards,
Gabriel

[-- Attachment #2: Type: text/html, Size: 542 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [bitbake-devel] [PATCH 1/2] fetch/git: Handle github dropping git:// support
  2021-11-10 15:15       ` gabriel.valcazar
@ 2021-11-11 11:03         ` Richard Purdie
  2021-11-11 12:18           ` gabriel.valcazar
  0 siblings, 1 reply; 14+ messages in thread
From: Richard Purdie @ 2021-11-11 11:03 UTC (permalink / raw)
  To: gabriel.valcazar, bitbake-devel

On Wed, 2021-11-10 at 07:15 -0800, gabriel.valcazar via lists.openembedded.org
wrote:
> Thanks all for the suggestions. Indeed, creating a fork of poky and applying
> the
> fix ourselves seems like a reasonable solution if there is no official fix for
> old versions.
> 
> I've just tested applying the patch to bitbake 1.18, and I'm able to build my
> Yocto 1.4 distribution with no problem. Is it not possible to officially
> backport the patch all the way back to bitbake 1.18? I could provide the
> patches, but they're literally just clean cherry-picks of the bitbake 1.40
> patch.

The trouble is there are a *lot* of fixes missing from 1.18. If we start
backporting this one, which other ones should we backport? If I don't say "no"
to this one, which other ones would I then need to take?

I appreciate this one is relatively safe and easy but it is also very easy to
backport something which applies cleanly but causes all kinds of issues.

We can't really start giving the impression we support things as old as 1.18,
much as I'd like to help. I can't imagine this is the only fix you need with
1.18?

Cheers,

Richard



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/2] fetch/git: Handle github dropping git:// support
  2021-11-11 11:03         ` [bitbake-devel] " Richard Purdie
@ 2021-11-11 12:18           ` gabriel.valcazar
  2021-11-11 12:27             ` [bitbake-devel] " Alexander Kanavin
  0 siblings, 1 reply; 14+ messages in thread
From: gabriel.valcazar @ 2021-11-11 12:18 UTC (permalink / raw)
  To: bitbake-devel

[-- Attachment #1: Type: text/plain, Size: 1920 bytes --]

Hi Richard,

Thanks for the explanation. I fully understand the reasoning behind your decision, but I really think this issue requires specific treatment due to its nature.

Ideally, customers using our legacy products (and thus, really old versions of our Yocto distribution) would migrate to newer products running newer software, and these situations would be avoided altogether. However, we're talking about products that are already in production, with several devices in the field that require periodic updates. Maintaining outdated software, while not recommendable from several standpoints, is oftentimes a more efficient approach than migrating to something newer, especially in these cases.

Older Yocto builds are still possible for us and our customers via docker containers or virtual machines, which provide "old" environments where the builds still work, despite all of the missing fixes in bitbake/poky. Basically, rather than patching the Yocto stack to fit our environment, we adapt our environment to fit the Yocto stack - again, because several legacy customers have made the decision to depend on old Yocto versions.

Having said this, I believe GitHub's deprecation of the git protocol is a special case because, regardless of the environment you're using or how many patches you've backported so far, builds will simply stop working altogether after the plug is pulled. No matter which environment we use, this is an external factor that's going to force changes in the Yocto stack, in areas that we have little to no control over. We're going to have to fix it either way, but it would be a much smoother transition if the fix were included by the community, and it's a unique enough case to warrant it being patched in older bitbake versions (in my opinion).

Forgive me for the insistence, but I truly believe this situation requires exceptional treatment.

Best regards,
Gabriel

[-- Attachment #2: Type: text/html, Size: 1981 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [bitbake-devel] [PATCH 1/2] fetch/git: Handle github dropping git:// support
  2021-11-11 12:18           ` gabriel.valcazar
@ 2021-11-11 12:27             ` Alexander Kanavin
  0 siblings, 0 replies; 14+ messages in thread
From: Alexander Kanavin @ 2021-11-11 12:27 UTC (permalink / raw)
  To: gabriel.valcazar; +Cc: bitbake-devel

[-- Attachment #1: Type: text/plain, Size: 2807 bytes --]

Are your customers paying anyone in the ‘community’ for this work or any
other work we do? Seriously, if you want anyone to pay attention to ancient
yocto versions, you have to support that financially.

Alex

On Thu 11. Nov 2021 at 13.18, gabriel.valcazar via lists.openembedded.org
<gabriel.valcazar=digi.com@lists.openembedded.org> wrote:

> Hi Richard,
>
> Thanks for the explanation. I fully understand the reasoning behind your
> decision, but I really think this issue requires specific treatment due to
> its nature.
>
> Ideally, customers using our legacy products (and thus, really old
> versions of our Yocto distribution) would migrate to newer products running
> newer software, and these situations would be avoided altogether. However,
> we're talking about products that are already in production, with several
> devices in the field that require periodic updates. Maintaining outdated
> software, while not recommendable from several standpoints, is oftentimes a
> more efficient approach than migrating to something newer, especially in
> these cases.
>
> Older Yocto builds are still possible for us and our customers via docker
> containers or virtual machines, which provide "old" environments where the
> builds still work, despite all of the missing fixes in bitbake/poky.
> Basically, rather than patching the Yocto stack to fit our environment, we
> adapt our environment to fit the Yocto stack - again, because several
> legacy customers have made the decision to depend on old Yocto versions.
>
> Having said this, I believe GitHub's deprecation of the *git *protocol is
> a special case because, regardless of the environment you're using or how
> many patches you've backported so far, builds will simply stop working
> altogether after the plug is pulled. No matter which environment we use,
> this is an external factor that's going to force changes in the Yocto
> stack, in areas that we have little to no control over. We're going to have
> to fix it either way, but it would be a much smoother transition if the fix
> were included by the community, and it's a unique enough case to warrant it
> being patched in older bitbake versions (in my opinion).
>
> Forgive me for the insistence, but I truly believe this situation requires
> exceptional treatment.
>
> Best regards,
> Gabriel
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#13015):
> https://lists.openembedded.org/g/bitbake-devel/message/13015
> Mute This Topic: https://lists.openembedded.org/mt/86764631/1686489
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
> alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 3674 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE: [bitbake-devel] [PATCH 1/2] fetch/git: Handle github dropping git:// support
  2021-11-02 12:44 Richard Purdie
  2021-11-02 13:19 ` [bitbake-devel] " Konrad Weihmann
@ 2021-11-02 18:00 ` Peter Kjellerstedt
  1 sibling, 0 replies; 14+ messages in thread
From: Peter Kjellerstedt @ 2021-11-02 18:00 UTC (permalink / raw)
  To: Richard Purdie, bitbake-devel

> -----Original Message-----
> From: bitbake-devel@lists.openembedded.org <bitbake-
> devel@lists.openembedded.org> On Behalf Of Richard Purdie
> Sent: den 2 november 2021 13:45
> To: bitbake-devel@lists.openembedded.org
> Subject: [bitbake-devel] [PATCH 1/2] fetch/git: Handle github dropping
> git:// support
> 
> github is dropping support for git procotol iun git urls. Add code to remap

Typo: procotol iun -> protocol in

> this to https in a way that could be used in older bitbake versions.

Though I would have written:

github is dropping support for the Git protocol in Git URLs. Add code to 
remap this to HTTPS in a way that can be used in older bitbake versions.

//Peter

> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  lib/bb/fetch2/git.py | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
> index e974445fbe..f5be24d6ab 100644
> --- a/lib/bb/fetch2/git.py
> +++ b/lib/bb/fetch2/git.py
> @@ -142,6 +142,10 @@ class Git(FetchMethod):
>              ud.proto = 'file'
>          else:
>              ud.proto = "git"
> +        if ud.host == "github.com" and ud.proto == "git":
> +            # github stopped supporting git protocol
> +            # https://github.blog/2021-09-01-improving-git-protocol-security-github/#no-more-unauthenticated-git
> +            ud.proto = "https"
> 
>          if not ud.proto in ('git', 'file', 'ssh', 'http', 'https', 'rsync'):
>              raise bb.fetch2.ParameterError("Invalid protocol type", ud.url)
> --
> 2.32.0



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [bitbake-devel] [PATCH 1/2] fetch/git: Handle github dropping git:// support
  2021-11-02 14:06     ` Frank Earl
@ 2021-11-02 14:57       ` Quentin Schulz
  0 siblings, 0 replies; 14+ messages in thread
From: Quentin Schulz @ 2021-11-02 14:57 UTC (permalink / raw)
  To: Frank Earl; +Cc: Richard Purdie, Konrad Weihmann, bitbake-devel

Hi Frank,

On Tue, Nov 02, 2021 at 09:06:19AM -0500, Frank Earl wrote:
> (Reply to all, since I simply hit reply...not nearly enough caffeine
> yet...)
> 
> This kind of breaks other things- what if you have a server that you need
> to do protocol=ssh and it's authenticated (We have some of this at Motorola

I think in that case you have ;protocol=ssh at the end of your URL?

In that case, the suggested code does handle that because it only
replaces the protocol from git to https. git protocol does not have
authentication, c.f. man git-clone:

"""
       The native transport (i.e. git:// URL) does no authentication and should be used with caution on unsecured
       networks.
"""

So if you needed authentication in the first place and it worked, I'm
pretty sure this change will not change anything for you.

Hopefully I understood your complain and am not completely off-topic :)

Cheers,
Quentin


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [bitbake-devel] [PATCH 1/2] fetch/git: Handle github dropping git:// support
  2021-11-02 13:27   ` Richard Purdie
  2021-11-02 14:06     ` Frank Earl
@ 2021-11-02 14:51     ` Jan-Simon Moeller
  1 sibling, 0 replies; 14+ messages in thread
From: Jan-Simon Moeller @ 2021-11-02 14:51 UTC (permalink / raw)
  To: bitbake-devel

Am Dienstag, 2. November 2021, 14:27:58 CET schrieb Richard Purdie:
>
> With 15 years of legacy branches and recipes out there, this change is
> intended to be backported to older bitbakes to at least give some support
> to people stuck on older versions. This is why this commit and the warning
> are separated.
>
> The warning is being added as we're not at 11.1.22 yet and people have
> previously said they don't like instant hard breaking error messages. We'll
> therefore give people some time to update, then change master to an error.

I think this is a good plan.

Best,
JS




^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [bitbake-devel] [PATCH 1/2] fetch/git: Handle github dropping git:// support
  2021-11-02 13:27   ` Richard Purdie
@ 2021-11-02 14:06     ` Frank Earl
  2021-11-02 14:57       ` Quentin Schulz
  2021-11-02 14:51     ` Jan-Simon Moeller
  1 sibling, 1 reply; 14+ messages in thread
From: Frank Earl @ 2021-11-02 14:06 UTC (permalink / raw)
  To: Richard Purdie; +Cc: Konrad Weihmann, bitbake-devel

[-- Attachment #1: Type: text/plain, Size: 3828 bytes --]

(Reply to all, since I simply hit reply...not nearly enough caffeine
yet...)

This kind of breaks other things- what if you have a server that you need
to do protocol=ssh and it's authenticated (We have some of this at Motorola
Solutions going on for security reasons...)?   The proposed change causes
access to that to go away.  You almost need to fix this by checking the
target host and if it's GitHub, do those changes since they're not allowed-
but that's a more invasive change for "fixing" the older branches (We're
working on catching up, PROMISE!  X-D)   Not sure what the right answer is
for this other than Yowl at GitHub and Microsoft for this one- the only
place you know of this and finding out about it is on their blog and when
you get punted out.



On Tue, Nov 2, 2021 at 8:28 AM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Tue, 2021-11-02 at 14:19 +0100, Konrad Weihmann wrote:
> >
> > On 02.11.21 13:44, Richard Purdie wrote:
> > > github is dropping support for git procotol iun git urls. Add code to
> remap
> > > this to https in a way that could be used in older bitbake versions.
> > >
> > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > > ---
> > >   lib/bb/fetch2/git.py | 4 ++++
> > >   1 file changed, 4 insertions(+)
> > >
> > > diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
> > > index e974445fbe..f5be24d6ab 100644
> > > --- a/lib/bb/fetch2/git.py
> > > +++ b/lib/bb/fetch2/git.py
> > > @@ -142,6 +142,10 @@ class Git(FetchMethod):
> > >               ud.proto = 'file'
> > >           else:
> > >               ud.proto = "git"
> > > +        if ud.host == "github.com" and ud.proto == "git":
> > > +            # github stopped supporting git protocol
> > > +            #
> https://github.blog/2021-09-01-improving-git-protocol-security-github/#no-more-unauthenticated-git
> >
> > While having a warning within the bitbake is something very much
> > appreciated, I don't think it's a good idea to remap to https here, as I
> > think rather the recipes that still use this should be fixed.
> > This will just lead to non-core layer (the ones being not so well
> > maintained) just ignoring the warning and leave the original offending
> > recipes as they are.
> >
> > I think this solution does scale very well.
> > For instance this assumes that github.com is used, but I can also think
> > of DNS aliases, where this rule here simply won't catch up... same for
> > the fact that other hosting platforms will likely go in the same
> > direction and we would end up with an ever growing condition here
> >
> > Bottom line: I like the warning, but would rather insist on the recipes
> > that still use git:// + protocol=git to be fixed - at the mentioned
> > 11.1.22 likely fetching them will fail with an explicit error message
> > anyway, so I actually don't see the need to magically fix up things here
>
> With 15 years of legacy branches and recipes out there, this change is
> intended
> to be backported to older bitbakes to at least give some support to people
> stuck
> on older versions. This is why this commit and the warning are separated.
>
> The warning is being added as we're not at 11.1.22 yet and people have
> previously said they don't like instant hard breaking error messages. We'll
> therefore give people some time to update, then change master to an error.
>
> Cheers,
>
> Richard
>
>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12859):
> https://lists.openembedded.org/g/bitbake-devel/message/12859
> Mute This Topic: https://lists.openembedded.org/mt/86764631/5804681
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
> linusti@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 5520 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [bitbake-devel] [PATCH 1/2] fetch/git: Handle github dropping git:// support
  2021-11-02 13:19 ` [bitbake-devel] " Konrad Weihmann
@ 2021-11-02 13:27   ` Richard Purdie
  2021-11-02 14:06     ` Frank Earl
  2021-11-02 14:51     ` Jan-Simon Moeller
  0 siblings, 2 replies; 14+ messages in thread
From: Richard Purdie @ 2021-11-02 13:27 UTC (permalink / raw)
  To: Konrad Weihmann, bitbake-devel

On Tue, 2021-11-02 at 14:19 +0100, Konrad Weihmann wrote:
> 
> On 02.11.21 13:44, Richard Purdie wrote:
> > github is dropping support for git procotol iun git urls. Add code to remap
> > this to https in a way that could be used in older bitbake versions.
> > 
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > ---
> >   lib/bb/fetch2/git.py | 4 ++++
> >   1 file changed, 4 insertions(+)
> > 
> > diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
> > index e974445fbe..f5be24d6ab 100644
> > --- a/lib/bb/fetch2/git.py
> > +++ b/lib/bb/fetch2/git.py
> > @@ -142,6 +142,10 @@ class Git(FetchMethod):
> >               ud.proto = 'file'
> >           else:
> >               ud.proto = "git"
> > +        if ud.host == "github.com" and ud.proto == "git":
> > +            # github stopped supporting git protocol
> > +            # https://github.blog/2021-09-01-improving-git-protocol-security-github/#no-more-unauthenticated-git
> 
> While having a warning within the bitbake is something very much 
> appreciated, I don't think it's a good idea to remap to https here, as I 
> think rather the recipes that still use this should be fixed.
> This will just lead to non-core layer (the ones being not so well 
> maintained) just ignoring the warning and leave the original offending 
> recipes as they are.
> 
> I think this solution does scale very well.
> For instance this assumes that github.com is used, but I can also think 
> of DNS aliases, where this rule here simply won't catch up... same for 
> the fact that other hosting platforms will likely go in the same 
> direction and we would end up with an ever growing condition here
> 
> Bottom line: I like the warning, but would rather insist on the recipes 
> that still use git:// + protocol=git to be fixed - at the mentioned 
> 11.1.22 likely fetching them will fail with an explicit error message 
> anyway, so I actually don't see the need to magically fix up things here

With 15 years of legacy branches and recipes out there, this change is intended
to be backported to older bitbakes to at least give some support to people stuck
on older versions. This is why this commit and the warning are separated.

The warning is being added as we're not at 11.1.22 yet and people have
previously said they don't like instant hard breaking error messages. We'll
therefore give people some time to update, then change master to an error.

Cheers,

Richard





^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [bitbake-devel] [PATCH 1/2] fetch/git: Handle github dropping git:// support
  2021-11-02 12:44 Richard Purdie
@ 2021-11-02 13:19 ` Konrad Weihmann
  2021-11-02 13:27   ` Richard Purdie
  2021-11-02 18:00 ` Peter Kjellerstedt
  1 sibling, 1 reply; 14+ messages in thread
From: Konrad Weihmann @ 2021-11-02 13:19 UTC (permalink / raw)
  To: Richard Purdie, bitbake-devel



On 02.11.21 13:44, Richard Purdie wrote:
> github is dropping support for git procotol iun git urls. Add code to remap
> this to https in a way that could be used in older bitbake versions.
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>   lib/bb/fetch2/git.py | 4 ++++
>   1 file changed, 4 insertions(+)
> 
> diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
> index e974445fbe..f5be24d6ab 100644
> --- a/lib/bb/fetch2/git.py
> +++ b/lib/bb/fetch2/git.py
> @@ -142,6 +142,10 @@ class Git(FetchMethod):
>               ud.proto = 'file'
>           else:
>               ud.proto = "git"
> +        if ud.host == "github.com" and ud.proto == "git":
> +            # github stopped supporting git protocol
> +            # https://github.blog/2021-09-01-improving-git-protocol-security-github/#no-more-unauthenticated-git

While having a warning within the bitbake is something very much 
appreciated, I don't think it's a good idea to remap to https here, as I 
think rather the recipes that still use this should be fixed.
This will just lead to non-core layer (the ones being not so well 
maintained) just ignoring the warning and leave the original offending 
recipes as they are.

I think this solution does scale very well.
For instance this assumes that github.com is used, but I can also think 
of DNS aliases, where this rule here simply won't catch up... same for 
the fact that other hosting platforms will likely go in the same 
direction and we would end up with an ever growing condition here

Bottom line: I like the warning, but would rather insist on the recipes 
that still use git:// + protocol=git to be fixed - at the mentioned 
11.1.22 likely fetching them will fail with an explicit error message 
anyway, so I actually don't see the need to magically fix up things here

> +            ud.proto = "https"
>   
>           if not ud.proto in ('git', 'file', 'ssh', 'http', 'https', 'rsync'):
>               raise bb.fetch2.ParameterError("Invalid protocol type", ud.url)
> 
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12856): https://lists.openembedded.org/g/bitbake-devel/message/12856
> Mute This Topic: https://lists.openembedded.org/mt/86764631/3647476
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [kweihmann@outlook.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2021-11-11 12:27 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <16B3BB5E65837842.31260@lists.openembedded.org>
2021-11-03 14:13 ` [bitbake-devel] [PATCH 1/2] fetch/git: Handle github dropping git:// support Richard Purdie
2021-11-09 15:40   ` gabriel.valcazar
2021-11-09 15:44     ` [bitbake-devel] " Alexander Kanavin
2021-11-09 15:47     ` Martin Jansa
2021-11-10 15:15       ` gabriel.valcazar
2021-11-11 11:03         ` [bitbake-devel] " Richard Purdie
2021-11-11 12:18           ` gabriel.valcazar
2021-11-11 12:27             ` [bitbake-devel] " Alexander Kanavin
2021-11-02 12:44 Richard Purdie
2021-11-02 13:19 ` [bitbake-devel] " Konrad Weihmann
2021-11-02 13:27   ` Richard Purdie
2021-11-02 14:06     ` Frank Earl
2021-11-02 14:57       ` Quentin Schulz
2021-11-02 14:51     ` Jan-Simon Moeller
2021-11-02 18:00 ` Peter Kjellerstedt

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.