[PATCH] migration-guides: prepare for release 3.5

[PATCH] migration-guides: prepare for release 3.5

[Michael Opdenacker]

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 documentation/migration-guides/index.rst      |  1 +
 .../migration-guides/migration-3.5.rst        | 37 +++++++++++++++++++
 2 files changed, 38 insertions(+)
 create mode 100644 documentation/migration-guides/migration-3.5.rst

diff --git a/documentation/migration-guides/index.rst b/documentation/migration-guides/index.rst
index 287b553195..6472cc2848 100644
--- a/documentation/migration-guides/index.rst
+++ b/documentation/migration-guides/index.rst
@@ -12,6 +12,7 @@ to move to one release of the Yocto Project from the previous one.
 .. toctree::
 
    migration-general
+   migration-3.5
    migration-3.4
    migration-3.3
    migration-3.2
diff --git a/documentation/migration-guides/migration-3.5.rst b/documentation/migration-guides/migration-3.5.rst
new file mode 100644
index 0000000000..c63c9dff05
--- /dev/null
+++ b/documentation/migration-guides/migration-3.5.rst
@@ -0,0 +1,37 @@
+Release 3.5 (kirkstone)
+=======================
+
+This section provides migration information for moving to the Yocto
+Project 3.5 Release (codename "kirkstone") from the prior release.
+
+Recipe changes
+--------------
+
+- Because of the uncertainty in future default branch names in git repositories,
+  it is now required to add a branch name to all URLs described
+  by ``git://`` and ``gitsm://`` :term:`SRC_URI` entries. For example::
+
+     SRC_URI = "git://git.denx.de/u-boot.git;branch=master"
+
+  A :oe_git:`convert-srcuri </openembedded-core/tree/scripts/contrib/convert-srcuri.py>`
+  script to convert your recipes is available in :term:`OpenEmbedded-Core (OE-Core)`
+  and in :term:`Poky`.
+
+- Because of `GitHub dropping support for the git:
+  protocol <https://github.blog/2021-09-01-improving-git-protocol-security-github/>`,
+  recipes now need to use ``;protocol=https`` at the end of GitHub
+  URLs. The same script as above can be used to convert the recipes.
+
+- The :term:`TOPDIR` variable and the current working directory are no longer modified
+  when parsing recipes. Any code depending on that behaviour will no longer worked as
+  expected (we don't know of any).
+
+- BitBake no longer supports ``append``, ``prepend`` and ``remove`` operators
+  combined with the ``+=``, ``=+``, ``.=``, ``=.`` and ``?=`` operators.
+  Such operations should be made using ``=`` alone. This means that
+  a leading space may need to be explicitly added.
+
+- :ref:`allarch <ref-classes-allarch>` packagegroups can no longer depend on packages
+  which use ``PKG:xxx`` renaming such as :ref:`debian.bbclass <ref-classes-debian>`.
+  Doing this will produce an error.
+
-- 
2.25.1

RE: [docs] [PATCH] migration-guides: prepare for release 3.5

[Peter Kjellerstedt]

> -----Original Message-----
> From: docs@lists.yoctoproject.org <docs@lists.yoctoproject.org> On Behalf Of Michael Opdenacker
> Sent: den 17 december 2021 15:04
> To: docs@lists.yoctoproject.org
> Cc: Michael Opdenacker <michael.opdenacker@bootlin.com>
> Subject: [docs] [PATCH] migration-guides: prepare for release 3.5
> 
> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
> ---
>  documentation/migration-guides/index.rst      |  1 +
>  .../migration-guides/migration-3.5.rst        | 37 +++++++++++++++++++
>  2 files changed, 38 insertions(+)
>  create mode 100644 documentation/migration-guides/migration-3.5.rst
> 
> diff --git a/documentation/migration-guides/index.rst b/documentation/migration-guides/index.rst
> index 287b553195..6472cc2848 100644
> --- a/documentation/migration-guides/index.rst
> +++ b/documentation/migration-guides/index.rst
> @@ -12,6 +12,7 @@ to move to one release of the Yocto Project from the previous one.
>  .. toctree::
> 
>     migration-general
> +   migration-3.5
>     migration-3.4
>     migration-3.3
>     migration-3.2
> diff --git a/documentation/migration-guides/migration-3.5.rst b/documentation/migration-guides/migration-3.5.rst
> new file mode 100644
> index 0000000000..c63c9dff05
> --- /dev/null
> +++ b/documentation/migration-guides/migration-3.5.rst
> @@ -0,0 +1,37 @@
> +Release 3.5 (kirkstone)
> +=======================
> +
> +This section provides migration information for moving to the Yocto
> +Project 3.5 Release (codename "kirkstone") from the prior release.
> +
> +Recipe changes
> +--------------
> +
> +- Because of the uncertainty in future default branch names in git repositories,
> +  it is now required to add a branch name to all URLs described
> +  by ``git://`` and ``gitsm://`` :term:`SRC_URI` entries. For example::
> +
> +     SRC_URI = "git://git.denx.de/u-boot.git;branch=master"
> +
> +  A :oe_git:`convert-srcuri </openembedded-core/tree/scripts/contrib/convert-srcuri.py>`
> +  script to convert your recipes is available in :term:`OpenEmbedded-Core (OE-Core)`
> +  and in :term:`Poky`.
> +
> +- Because of `GitHub dropping support for the git:
> +  protocol <https://github.blog/2021-09-01-improving-git-protocol-security-github/>`,
> +  recipes now need to use ``;protocol=https`` at the end of GitHub
> +  URLs. The same script as above can be used to convert the recipes.
> +
> +- The :term:`TOPDIR` variable and the current working directory are no longer modified
> +  when parsing recipes. Any code depending on that behaviour will no longer worked as

Change "worked" to "work".

> +  expected (we don't know of any).
> +
> +- BitBake no longer supports ``append``, ``prepend`` and ``remove`` operators
> +  combined with the ``+=``, ``=+``, ``.=``, ``=.`` and ``?=`` operators.
> +  Such operations should be made using ``=`` alone. This means that
> +  a leading space may need to be explicitly added.
> +
> +- :ref:`allarch <ref-classes-allarch>` packagegroups can no longer depend on packages
> +  which use ``PKG:xxx`` renaming such as :ref:`debian.bbclass <ref-classes-debian>`.
> +  Doing this will produce an error.
> +
> --
> 2.25.1

//Peter

Re: [docs] [PATCH] migration-guides: prepare for release 3.5

[Quentin Schulz]

Hi Michael,

On 12/17/21 15:04, Michael Opdenacker wrote:
> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
> ---
>   documentation/migration-guides/index.rst      |  1 +
>   .../migration-guides/migration-3.5.rst        | 37 +++++++++++++++++++
>   2 files changed, 38 insertions(+)
>   create mode 100644 documentation/migration-guides/migration-3.5.rst
> 
> diff --git a/documentation/migration-guides/index.rst b/documentation/migration-guides/index.rst
> index 287b553195..6472cc2848 100644
> --- a/documentation/migration-guides/index.rst
> +++ b/documentation/migration-guides/index.rst
> @@ -12,6 +12,7 @@ to move to one release of the Yocto Project from the previous one.
>   .. toctree::
>   
>      migration-general
> +   migration-3.5
>      migration-3.4
>      migration-3.3
>      migration-3.2
> diff --git a/documentation/migration-guides/migration-3.5.rst b/documentation/migration-guides/migration-3.5.rst
> new file mode 100644
> index 0000000000..c63c9dff05
> --- /dev/null
> +++ b/documentation/migration-guides/migration-3.5.rst
> @@ -0,0 +1,37 @@
> +Release 3.5 (kirkstone)
> +=======================
> +
> +This section provides migration information for moving to the Yocto
> +Project 3.5 Release (codename "kirkstone") from the prior release.
> +
> +Recipe changes
> +--------------
> +
> +- Because of the uncertainty in future default branch names in git repositories,
> +  it is now required to add a branch name to all URLs described
> +  by ``git://`` and ``gitsm://`` :term:`SRC_URI` entries. For example::
> +
> +     SRC_URI = "git://git.denx.de/u-boot.git;branch=master"
> +
> +  A :oe_git:`convert-srcuri </openembedded-core/tree/scripts/contrib/convert-srcuri.py>`
> +  script to convert your recipes is available in :term:`OpenEmbedded-Core (OE-Core)`
> +  and in :term:`Poky`.
> +
> +- Because of `GitHub dropping support for the git:
> +  protocol <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.blog_2021-2D09-2D01-2Dimproving-2Dgit-2Dprotocol-2Dsecurity-2Dgithub_&d=DwIDAg&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=BHtIscjgpnlAxrsnv58CSizo0w27BvdJhKyrnKN2iWSXMxSc6E9_hS4mbjYz1tJ7&s=dHvCcZgUMuYx5xpf5Vm0zUvn-LHw3OZ5u2E8xzrGUGM&e= >`,
> +  recipes now need to use ``;protocol=https`` at the end of GitHub
> +  URLs. The same script as above can be used to convert the recipes.
> +
> +- The :term:`TOPDIR` variable and the current working directory are no longer modified
> +  when parsing recipes. Any code depending on that behaviour will no longer worked as
> +  expected (we don't know of any).
> +

s/will no longer worked as expected (we don't know of any)/will no 
longer work/

> +- BitBake no longer supports ``append``, ``prepend`` and ``remove`` operators
> +  combined with the ``+=``, ``=+``, ``.=``, ``=.`` and ``?=`` operators.

I haven't checked, but what about ??= operator?

> +  Such operations should be made using ``=`` alone. This means that > +  a leading space may need to be explicitly added.
> +

May I suggest the following instead? For the whole paragraph (provided 
??= also isn't supported).

"""
The ``append``, ``prepend`` and ``remove`` operators can now only be 
combined with ``=`` and ``:=`` operators. To the exception of the 
``append`` plus ``+=`` and ``prepend`` plus ``=+`` combinations, all 
combinations could be factored up to the ``append``, ``prepend`` or 
``remove`` in the combination. This brought a lot of confusion on how 
the override style syntax operators work and should be used. Therefore, 
those combinations can simply be replaced by a single ``append``, 
``prepend`` or ``remove`` operator without any additional change.
For the ``append`` plus ``+=`` (and ``prepend`` plus ``=+``) 
combination, the content should be prefixed (respectively suffixed) by a 
space to maintain the same behavior.
You can learn more about override style syntax operators (``append``, 
``prepend`` and ``remove``) in the :ref:`BitBake documentation </don't 
know the path>`.
"""

The path to the doc: 
https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#appending-and-prepending-override-style-syntax 
and 
https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#removal-override-style-syntax

Looking for those links, I also noticed that we haven't updated the 
bitbake docs yet to remove the explanation on += combination in 
https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#override-style-operation-advantages.

> +- :ref:`allarch <ref-classes-allarch>` packagegroups can no longer depend on packages
> +  which use ``PKG:xxx`` renaming such as :ref:`debian.bbclass <ref-classes-debian>`.

s/``PKG:xxx``/:term:`PKG`/ ?

> +  Doing this will produce an error.
> +

I would remove this sentence or give the kind of message to be expected 
when such an error happens.

Cheers,
Quentin

Re: [docs] [PATCH] migration-guides: prepare for release 3.5

[Richard Purdie]

On Fri, 2021-12-17 at 15:04 +0100, Michael Opdenacker wrote:
> diff --git a/documentation/migration-guides/migration-3.5.rst b/documentation/migration-guides/migration-3.5.rst
> new file mode 100644
> index 0000000000..c63c9dff05
> --- /dev/null
> +++ b/documentation/migration-guides/migration-3.5.rst
> @@ -0,0 +1,37 @@
> +Release 3.5 (kirkstone)
> +=======================
> +
> +This section provides migration information for moving to the Yocto
> +Project 3.5 Release (codename "kirkstone") from the prior release.
> +
> +Recipe changes
> +--------------
> +
> +- Because of the uncertainty in future default branch names in git repositories,
> +  it is now required to add a branch name to all URLs described
> +  by ``git://`` and ``gitsm://`` :term:`SRC_URI` entries. For example::
> +
> +     SRC_URI = "git://git.denx.de/u-boot.git;branch=master"
> +
> +  A :oe_git:`convert-srcuri </openembedded-core/tree/scripts/contrib/convert-srcuri.py>`
> +  script to convert your recipes is available in :term:`OpenEmbedded-Core (OE-Core)`
> +  and in :term:`Poky`.
> +
> +- Because of `GitHub dropping support for the git:
> +  protocol <https://github.blog/2021-09-01-improving-git-protocol-security-github/>`,
> +  recipes now need to use ``;protocol=https`` at the end of GitHub
> +  URLs. The same script as above can be used to convert the recipes.
> +
> +- The :term:`TOPDIR` variable and the current working directory are no longer modified
> +  when parsing recipes. Any code depending on that behaviour will no longer worked as
> +  expected (we don't know of any).
> +
> +- BitBake no longer supports ``append``, ``prepend`` and ``remove`` operators
> +  combined with the ``+=``, ``=+``, ``.=``, ``=.`` and ``?=`` operators.
> +  Such operations should be made using ``=`` alone. This means that
> +  a leading space may need to be explicitly added.
> +
> +- :ref:`allarch <ref-classes-allarch>` packagegroups can no longer depend on packages
> +  which use ``PKG:xxx`` renaming such as :ref:`debian.bbclass <ref-classes-debian>`.
> +  Doing this will produce an error.
> +

Thanks for starting this. We need to now add something like:

"""
Network access from tasks is now disabled by default on kernels which support
this (most recent distros e.g. centos8 and debian11 onwards). This means that
tasks accessing the network need to be marked as such with the network flag,
e.g.:

do_mytask[network] = "1"

This is allowed by default from do_fetch but not from any of our other standard
tasks. Recipes shouldn't be accessing the network outside of do_fetch as it that
usually undermines fetcher source mirroring, image manifests and licence
manifests/auditing.
"""

Cheers,

Richard

Re: [docs] [PATCH] migration-guides: prepare for release 3.5

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 3512 bytes --]

It undermines software supply chain security as well, an important point to
make I think. It is not okay to pull random things from the internet
quietly.

Alex

On Mon, 17 Jan 2022 at 13:51, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Fri, 2021-12-17 at 15:04 +0100, Michael Opdenacker wrote:
> > diff --git a/documentation/migration-guides/migration-3.5.rst
> b/documentation/migration-guides/migration-3.5.rst
> > new file mode 100644
> > index 0000000000..c63c9dff05
> > --- /dev/null
> > +++ b/documentation/migration-guides/migration-3.5.rst
> > @@ -0,0 +1,37 @@
> > +Release 3.5 (kirkstone)
> > +=======================
> > +
> > +This section provides migration information for moving to the Yocto
> > +Project 3.5 Release (codename "kirkstone") from the prior release.
> > +
> > +Recipe changes
> > +--------------
> > +
> > +- Because of the uncertainty in future default branch names in git
> repositories,
> > +  it is now required to add a branch name to all URLs described
> > +  by ``git://`` and ``gitsm://`` :term:`SRC_URI` entries. For example::
> > +
> > +     SRC_URI = "git://git.denx.de/u-boot.git;branch=master"
> > +
> > +  A :oe_git:`convert-srcuri
> </openembedded-core/tree/scripts/contrib/convert-srcuri.py>`
> > +  script to convert your recipes is available in
> :term:`OpenEmbedded-Core (OE-Core)`
> > +  and in :term:`Poky`.
> > +
> > +- Because of `GitHub dropping support for the git:
> > +  protocol <
> https://github.blog/2021-09-01-improving-git-protocol-security-github/>`,
> > +  recipes now need to use ``;protocol=https`` at the end of GitHub
> > +  URLs. The same script as above can be used to convert the recipes.
> > +
> > +- The :term:`TOPDIR` variable and the current working directory are no
> longer modified
> > +  when parsing recipes. Any code depending on that behaviour will no
> longer worked as
> > +  expected (we don't know of any).
> > +
> > +- BitBake no longer supports ``append``, ``prepend`` and ``remove``
> operators
> > +  combined with the ``+=``, ``=+``, ``.=``, ``=.`` and ``?=`` operators.
> > +  Such operations should be made using ``=`` alone. This means that
> > +  a leading space may need to be explicitly added.
> > +
> > +- :ref:`allarch <ref-classes-allarch>` packagegroups can no longer
> depend on packages
> > +  which use ``PKG:xxx`` renaming such as :ref:`debian.bbclass
> <ref-classes-debian>`.
> > +  Doing this will produce an error.
> > +
>
> Thanks for starting this. We need to now add something like:
>
> """
> Network access from tasks is now disabled by default on kernels which
> support
> this (most recent distros e.g. centos8 and debian11 onwards). This means
> that
> tasks accessing the network need to be marked as such with the network
> flag,
> e.g.:
>
> do_mytask[network] = "1"
>
> This is allowed by default from do_fetch but not from any of our other
> standard
> tasks. Recipes shouldn't be accessing the network outside of do_fetch as
> it that
> usually undermines fetcher source mirroring, image manifests and licence
> manifests/auditing.
> """
>
> Cheers,
>
> Richard
>
>
>
>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#2414):
> https://lists.yoctoproject.org/g/docs/message/2414
> Mute This Topic: https://lists.yoctoproject.org/mt/87789405/1686489
> Group Owner: docs+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub [
> alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 4925 bytes --]

[PATCH] migration-3.5: mention task specific network access

[Michael Opdenacker]

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 documentation/migration-guides/migration-3.5.rst | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/documentation/migration-guides/migration-3.5.rst b/documentation/migration-guides/migration-3.5.rst
index bd807eb8c7..df15487b10 100644
--- a/documentation/migration-guides/migration-3.5.rst
+++ b/documentation/migration-guides/migration-3.5.rst
@@ -22,6 +22,18 @@ Recipe changes
   recipes now need to use ``;protocol=https`` at the end of GitHub
   URLs. The same script as above can be used to convert the recipes.
 
+- Network access from tasks is now disabled by default on kernels which support
+  this feature (on most recent distros such as CentOS 8 and Debian 11 onwards).
+  This means that tasks accessing the network need to be marked as such with the ``network``
+  flag. For example::
+
+     do_mytask[network] = "1"
+
+  This is allowed by default from ``do_fetch`` but not from any of our other standard
+  tasks. Recipes shouldn't be accessing the network outside of ``do_fetch`` as it
+  usually undermines fetcher source mirroring, image and licence manifests, software
+  auditing and supply chain security.
+
 - The :term:`TOPDIR` variable and the current working directory are no longer modified
   when parsing recipes. Any code depending on that behaviour will no longer work.
 
-- 
2.25.1

Re: [docs] [PATCH] migration-3.5: mention task specific network access

[Richard Purdie]

On Tue, 2022-01-18 at 14:07 +0100, Michael Opdenacker wrote:
> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
> ---
>  documentation/migration-guides/migration-3.5.rst | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/documentation/migration-guides/migration-3.5.rst b/documentation/migration-guides/migration-3.5.rst
> index bd807eb8c7..df15487b10 100644
> --- a/documentation/migration-guides/migration-3.5.rst
> +++ b/documentation/migration-guides/migration-3.5.rst
> @@ -22,6 +22,18 @@ Recipe changes
>    recipes now need to use ``;protocol=https`` at the end of GitHub
>    URLs. The same script as above can be used to convert the recipes.
>  
> +- Network access from tasks is now disabled by default on kernels which support
> +  this feature (on most recent distros such as CentOS 8 and Debian 11 onwards).
> +  This means that tasks accessing the network need to be marked as such with the ``network``
> +  flag. For example::
> +
> +     do_mytask[network] = "1"
> +
> +  This is allowed by default from ``do_fetch`` but not from any of our other standard
> +  tasks. Recipes shouldn't be accessing the network outside of ``do_fetch`` as it
> +  usually undermines fetcher source mirroring, image and licence manifests, software
> +  auditing and supply chain security.
> +
>  - The :term:`TOPDIR` variable and the current working directory are no longer modified
>    when parsing recipes. Any code depending on that behaviour will no longer work.

Looks good to me, thanks!

Cheers,

Richard