All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2] migration-guides: Add start of 3.4 guide with override migration notes
       [not found] <16968778436D0C00.13113@lists.yoctoproject.org>
@ 2021-07-30  9:45 ` Michael Opdenacker
  2021-07-30  9:47 ` Michael Opdenacker
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 6+ messages in thread
From: Michael Opdenacker @ 2021-07-30  9:45 UTC (permalink / raw)
  To: docs; +Cc: Richard Purdie, Quentin Schulz, Michael Opdenacker

From: Richard Purdie <richard.purdie@linuxfoundation.org>

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reviewed-by: Quentin Schulz <foss@0leil.net>
Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 documentation/migration-guides/index.rst      |  1 +
 .../migration-guides/migration-3.4.rst        | 72 +++++++++++++++++++
 2 files changed, 73 insertions(+)
 create mode 100644 documentation/migration-guides/migration-3.4.rst

diff --git a/documentation/migration-guides/index.rst b/documentation/migration-guides/index.rst
index 6304e6318c..287b553195 100644
--- a/documentation/migration-guides/index.rst
+++ b/documentation/migration-guides/index.rst
@@ -12,6 +12,7 @@ to move to one release of the Yocto Project from the previous one.
 .. toctree::
 
    migration-general
+   migration-3.4
    migration-3.3
    migration-3.2
    migration-3.1
diff --git a/documentation/migration-guides/migration-3.4.rst b/documentation/migration-guides/migration-3.4.rst
new file mode 100644
index 0000000000..45ce38cd6d
--- /dev/null
+++ b/documentation/migration-guides/migration-3.4.rst
@@ -0,0 +1,72 @@
+Release 3.4 (honister)
+======================
+
+This section provides migration information for moving to the Yocto
+Project 3.4 Release (codename "honister") from the prior release.
+
+Override syntax changes
+-----------------------
+
+This release requires changes to the metadata to indicate where overrides are
+being used in variable key names. This is done with the ":" character replacing
+the use of "_" previously. This means that an entry like::
+
+   SRC_URI_qemux86 = "file://somefile"
+
+becomes::
+
+   SRC_URI:qemux86 = " file://somefile"
+
+since qemux86 is an override. This applies to any use of override syntax so::
+
+   SRC_URI_append = " file://somefile"
+   SRC_URI_append_qemux86 = " file://somefile2"
+   SRC_URI_remove_qemux86-64 = " file://somefile3"
+   SRC_URI_prepend_qemuarm = "file://somefile4 "
+   FILES_${PN}-ptest = "${bindir}/xyz"
+   IMAGE_CMD_tar = "tar"
+   BASE_LIB_tune-coretexa76 = "lib"
+   SRCREV_pn-bash = "abc"
+   BB_TASK_NICE_LEVEL_task-testimage = '0'
+
+becomes::
+
+   SRC_URI:append = " file://somefile"
+   SRC_URI:append:qemux86 = " file://somefile2"
+   SRC_URI:remove:qemux86-64 = " file://somefile3"
+   SRC_URI:prepend:qemuarm = "file://somefile4 "
+   FILES:${PN}-ptest = "${bindir}/xyz"
+   IMAGE_CMD:tar = "tar"
+   BASE_LIB:tune-coretexa76 = "lib"
+   SRCREV:pn-bash = "abc"
+   BB_TASK_NICE_LEVEL:task-testimage = '0'
+
+This also applies to variable queries to the datastore, for example using getVar 
+and similar so d.getVar("RDEPENDS_${PN}") becomes d.getVar("RDEPENDS:${PN}").
+
+Whilst some of these are fairly obvious such as MACHINE and DISTRO overrides, some
+are less obvious, for example the packaging variables such as RDEPENDS, FILES and 
+so on taking package names (e.g. ${PN}, ${PN}-ptest) as overrides. These overrides 
+are not always applies in OVERRIDES but applied conditionally in specific contexts
+such as packaging. The task-<taskname> is another context specific override, the 
+context being specific tasks in that case. Tune overrides are another specialist
+case where some code does use them as overrides but some does not. We plan to try
+and make the tune code use overrides more consistently in the future.
+
+To help with migration of layers there is a script in OE-Core. Once configured
+with the overrides used by a layer, this can be run as::
+
+   <oe-core>/scripts/contrib/convert-overrides.py <layerdir>
+
+Please read the notes in the script as it isn't entirely automatic and it isn't
+expected to handle every case. In particular, it needs to be told which overrides
+the layer uses (usually machine and distro names/overrides) and the result should
+be carefully checked since it can be a little enthusiastic and will convert
+references to "_append", "_remove" and "_prepend" in function and variables names.
+
+For reference, this conversion is important as it allows bitbake to know what is
+an override and what is not. This should allow us to proceed with other syntax
+improvements and simplifications for usability. It also means bitbake no longer
+has to guess and maintain large lookup lists just in case "functionname" in
+"my_functionname" is an override and this should improve efficiency.
+
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH v2] migration-guides: Add start of 3.4 guide with override migration notes
       [not found] <16968778436D0C00.13113@lists.yoctoproject.org>
  2021-07-30  9:45 ` [PATCH v2] migration-guides: Add start of 3.4 guide with override migration notes Michael Opdenacker
@ 2021-07-30  9:47 ` Michael Opdenacker
  2021-07-30 10:33   ` [docs] " Quentin Schulz
       [not found] ` <1696886EF6A201F4.6707@lists.yoctoproject.org>
       [not found] ` <1696889367F9FDB7.13113@lists.yoctoproject.org>
  3 siblings, 1 reply; 6+ messages in thread
From: Michael Opdenacker @ 2021-07-30  9:47 UTC (permalink / raw)
  To: docs; +Cc: Richard Purdie, Quentin Schulz, Michael Opdenacker

From: Richard Purdie <richard.purdie@linuxfoundation.org>

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reviewed-by: Quentin Schulz <foss@0leil.net>
Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 documentation/migration-guides/index.rst      |  1 +
 .../migration-guides/migration-3.4.rst        | 74 +++++++++++++++++++
 2 files changed, 75 insertions(+)
 create mode 100644 documentation/migration-guides/migration-3.4.rst

diff --git a/documentation/migration-guides/index.rst b/documentation/migration-guides/index.rst
index 6304e6318c..287b553195 100644
--- a/documentation/migration-guides/index.rst
+++ b/documentation/migration-guides/index.rst
@@ -12,6 +12,7 @@ to move to one release of the Yocto Project from the previous one.
 .. toctree::
 
    migration-general
+   migration-3.4
    migration-3.3
    migration-3.2
    migration-3.1
diff --git a/documentation/migration-guides/migration-3.4.rst b/documentation/migration-guides/migration-3.4.rst
new file mode 100644
index 0000000000..4aa5271724
--- /dev/null
+++ b/documentation/migration-guides/migration-3.4.rst
@@ -0,0 +1,74 @@
+Release 3.4 (honister)
+======================
+
+This section provides migration information for moving to the Yocto
+Project 3.4 Release (codename "honister") from the prior release.
+
+Override syntax changes
+-----------------------
+
+This release requires changes to the metadata to indicate where overrides are
+being used in variable key names. This is done with the ``:`` character replacing
+the use of ``_`` previously. This means that an entry like::
+
+   SRC_URI_qemux86 = "file://somefile"
+
+becomes::
+
+   SRC_URI:qemux86 = "file://somefile"
+
+since ``qemux86`` is an override. This applies to any use of override syntax so::
+
+   SRC_URI_append = " file://somefile"
+   SRC_URI_append_qemux86 = " file://somefile2"
+   SRC_URI_remove_qemux86-64 = " file://somefile3"
+   SRC_URI_prepend_qemuarm = "file://somefile4 "
+   FILES_${PN}-ptest = "${bindir}/xyz"
+   IMAGE_CMD_tar = "tar"
+   BASE_LIB_tune-cortexa76 = "lib"
+   SRCREV_pn-bash = "abc"
+   BB_TASK_NICE_LEVEL_task-testimage = '0'
+
+becomes::
+
+   SRC_URI:append = " file://somefile"
+   SRC_URI:append:qemux86 = " file://somefile2"
+   SRC_URI:remove:qemux86-64 = " file://somefile3"
+   SRC_URI:prepend:qemuarm = "file://somefile4 "
+   FILES:${PN}-ptest = "${bindir}/xyz"
+   IMAGE_CMD:tar = "tar"
+   BASE_LIB:tune-cortexa76 = "lib"
+   SRCREV:pn-bash = "abc"
+   BB_TASK_NICE_LEVEL:task-testimage = '0'
+
+This also applies to variable queries to the datastore, for example using getVar
+and similar so ``d.getVar("RDEPENDS_${PN}")`` becomes ``d.getVar("RDEPENDS:${PN}")``.
+
+Whilst some of these are fairly obvious such as :term:`MACHINE` and :term:`DISTRO`
+overrides, some are less obvious, for example the packaging variables such as
+:term:`RDEPENDS`, :term:`FILES` and so on taking package names (e.g. ``${PN}``,
+``${PN}-ptest``) as overrides. These overrides are not always in
+:term:`OVERRIDES` but applied conditionally in specific contexts
+such as packaging. ``task-<taskname>`` is another context specific override, the
+context being specific tasks in that case. Tune overrides are another special
+case where some code does use them as overrides but some does not. We plan to try
+and make the tune code use overrides more consistently in the future.
+
+To help with migration of layers there is a script in OE-Core. Once configured
+with the overrides used by a layer, this can be run as::
+
+   <oe-core>/scripts/contrib/convert-overrides.py <layerdir>
+
+.. note::
+
+   Please read the notes in the script as it isn't entirely automatic and it isn't
+   expected to handle every case. In particular, it needs to be told which overrides
+   the layer uses (usually machine and distro names/overrides) and the result should
+   be carefully checked since it can be a little enthusiastic and will convert
+   references to ``_append``, ``_remove`` and ``_prepend`` in function and variables names.
+
+For reference, this conversion is important as it allows BitBake to know what is
+an override and what is not. This should allow us to proceed with other syntax
+improvements and simplifications for usability. It also means bitbake no longer
+has to guess and maintain large lookup lists just in case ``functionname`` in
+``my_functionname`` is an override and this should improve efficiency.
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [docs] [PATCH v2] migration-guides: Add start of 3.4 guide with override migration notes
       [not found] ` <1696886EF6A201F4.6707@lists.yoctoproject.org>
@ 2021-07-30  9:49   ` Michael Opdenacker
  0 siblings, 0 replies; 6+ messages in thread
From: Michael Opdenacker @ 2021-07-30  9:49 UTC (permalink / raw)
  To: docs; +Cc: Richard Purdie, Quentin Schulz

Forget about this one, this is still the V1 version.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [docs] [PATCH v2] migration-guides: Add start of 3.4 guide with override migration notes
       [not found] ` <1696889367F9FDB7.13113@lists.yoctoproject.org>
@ 2021-07-30 10:07   ` Michael Opdenacker
  0 siblings, 0 replies; 6+ messages in thread
From: Michael Opdenacker @ 2021-07-30 10:07 UTC (permalink / raw)
  To: docs; +Cc: Richard Purdie, Quentin Schulz


On 7/30/21 11:47 AM, Michael Opdenacker wrote:
> From: Richard Purdie <richard.purdie@linuxfoundation.org>
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> Reviewed-by: Quentin Schulz <foss@0leil.net>
> Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>


For further review, you can also check the generated HTML code on
https://docs.yoctoproject.org/next/migration-guides/migration-3.4.html
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [docs] [PATCH v2] migration-guides: Add start of 3.4 guide with override migration notes
  2021-07-30  9:47 ` Michael Opdenacker
@ 2021-07-30 10:33   ` Quentin Schulz
  2021-07-30 14:06     ` Michael Opdenacker
  0 siblings, 1 reply; 6+ messages in thread
From: Quentin Schulz @ 2021-07-30 10:33 UTC (permalink / raw)
  To: Michael Opdenacker; +Cc: docs, Richard Purdie, Quentin Schulz

Hi Michael,

On Fri, Jul 30, 2021 at 11:47:57AM +0200, Michael Opdenacker wrote:
> From: Richard Purdie <richard.purdie@linuxfoundation.org>
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> Reviewed-by: Quentin Schulz <foss@0leil.net>

Please don't add my Reviewed-by if I haven't given it :)

> Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
> ---
>  documentation/migration-guides/index.rst      |  1 +
>  .../migration-guides/migration-3.4.rst        | 74 +++++++++++++++++++
>  2 files changed, 75 insertions(+)
>  create mode 100644 documentation/migration-guides/migration-3.4.rst
> 
> diff --git a/documentation/migration-guides/index.rst b/documentation/migration-guides/index.rst
> index 6304e6318c..287b553195 100644
> --- a/documentation/migration-guides/index.rst
> +++ b/documentation/migration-guides/index.rst
> @@ -12,6 +12,7 @@ to move to one release of the Yocto Project from the previous one.
>  .. toctree::
>  
>     migration-general
> +   migration-3.4
>     migration-3.3
>     migration-3.2
>     migration-3.1
> diff --git a/documentation/migration-guides/migration-3.4.rst b/documentation/migration-guides/migration-3.4.rst
> new file mode 100644
> index 0000000000..4aa5271724
> --- /dev/null
> +++ b/documentation/migration-guides/migration-3.4.rst
> @@ -0,0 +1,74 @@
> +Release 3.4 (honister)
> +======================
> +
> +This section provides migration information for moving to the Yocto
> +Project 3.4 Release (codename "honister") from the prior release.
> +
> +Override syntax changes
> +-----------------------
> +
> +This release requires changes to the metadata to indicate where overrides are
> +being used in variable key names. This is done with the ``:`` character replacing
> +the use of ``_`` previously. This means that an entry like::
> +
> +   SRC_URI_qemux86 = "file://somefile"
> +
> +becomes::
> +
> +   SRC_URI:qemux86 = "file://somefile"
> +
> +since ``qemux86`` is an override. This applies to any use of override syntax so::
> +
> +   SRC_URI_append = " file://somefile"
> +   SRC_URI_append_qemux86 = " file://somefile2"
> +   SRC_URI_remove_qemux86-64 = " file://somefile3"
> +   SRC_URI_prepend_qemuarm = "file://somefile4 "
> +   FILES_${PN}-ptest = "${bindir}/xyz"
> +   IMAGE_CMD_tar = "tar"
> +   BASE_LIB_tune-cortexa76 = "lib"
> +   SRCREV_pn-bash = "abc"
> +   BB_TASK_NICE_LEVEL_task-testimage = '0'
> +
> +becomes::
> +
> +   SRC_URI:append = " file://somefile"
> +   SRC_URI:append:qemux86 = " file://somefile2"
> +   SRC_URI:remove:qemux86-64 = " file://somefile3"
> +   SRC_URI:prepend:qemuarm = "file://somefile4 "
> +   FILES:${PN}-ptest = "${bindir}/xyz"
> +   IMAGE_CMD:tar = "tar"
> +   BASE_LIB:tune-cortexa76 = "lib"
> +   SRCREV:pn-bash = "abc"
> +   BB_TASK_NICE_LEVEL:task-testimage = '0'
> +
> +This also applies to variable queries to the datastore, for example using getVar

getVar to be tick quoted?

Maybe we can link to:
https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html?functions-for-accessing-datastore-variables
for the various queries to the datastore?

Reviewed-by: Quentin Schulz <foss@0leil.net>

Thanks!

> +and similar so ``d.getVar("RDEPENDS_${PN}")`` becomes ``d.getVar("RDEPENDS:${PN}")``.
> +

Just something that popped up in my mind but we should be careful when
updating the docs to use : instead of _ since I seem to recall that we
have some ``RDEPENDS_*`` or similar that we could actually replace by
:term:`RDEPENDS` now :)

I've yet to read the udpated documentation (if there's something
available somewhere already?) and check a bit what's handled by the
script done by Richard :)

Cheers,
Quentin

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [docs] [PATCH v2] migration-guides: Add start of 3.4 guide with override migration notes
  2021-07-30 10:33   ` [docs] " Quentin Schulz
@ 2021-07-30 14:06     ` Michael Opdenacker
  0 siblings, 0 replies; 6+ messages in thread
From: Michael Opdenacker @ 2021-07-30 14:06 UTC (permalink / raw)
  To: Quentin Schulz; +Cc: docs, Richard Purdie, Quentin Schulz

Hi Quentin,

On 7/30/21 12:33 PM, Quentin Schulz wrote:
> Hi Michael,
>
> On Fri, Jul 30, 2021 at 11:47:57AM +0200, Michael Opdenacker wrote:
>> From: Richard Purdie <richard.purdie@linuxfoundation.org>
>>
>> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>> Reviewed-by: Quentin Schulz <foss@0leil.net>
> Please don't add my Reviewed-by if I haven't given it :)


Oops, my apologies!


> getVar to be tick quoted?
>
> Maybe we can link to:
> https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html?functions-for-accessing-datastore-variables
> for the various queries to the datastore?


Both done! I'm posting a V3 now.
Thanks,
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-07-30 14:06 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <16968778436D0C00.13113@lists.yoctoproject.org>
2021-07-30  9:45 ` [PATCH v2] migration-guides: Add start of 3.4 guide with override migration notes Michael Opdenacker
2021-07-30  9:47 ` Michael Opdenacker
2021-07-30 10:33   ` [docs] " Quentin Schulz
2021-07-30 14:06     ` Michael Opdenacker
     [not found] ` <1696886EF6A201F4.6707@lists.yoctoproject.org>
2021-07-30  9:49   ` Michael Opdenacker
     [not found] ` <1696889367F9FDB7.13113@lists.yoctoproject.org>
2021-07-30 10:07   ` Michael Opdenacker

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.