[PATCH v3 1/3] lib/oe/package: replace in place PN-locale-* packages in PACKAGES

[PATCH v3 1/3] lib/oe/package: replace in place PN-locale-* packages in PACKAGES

[Jonathan GUILLOT]

split_locales() removes PN-locale from PACKAGES and adds PN-locale-* to the end.
As the PN-locale package typically appears before PN base package, it may result
in paths not installed in PN-locale-* packages if already catched by PN. Now
insert PN-locale-* exactly where PN-locale was existing in list to avoid such
an issue.

Signed-off-by: Jonathan GUILLOT <jonathan@joggee.fr>
---
 meta/lib/oe/package.py | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
index 9a465eaa09..2f7a50e144 100644
--- a/meta/lib/oe/package.py
+++ b/meta/lib/oe/package.py
@@ -647,8 +647,11 @@ def split_locales(d):
     dvar = d.getVar('PKGD')
     pn = d.getVar('LOCALEBASEPN')
 
-    if pn + '-locale' in packages:
-        packages.remove(pn + '-locale')
+    try:
+        locale_index = packages.index(pn + '-locale')
+        packages.pop(locale_index)
+    except ValueError:
+        locale_index = len(packages)
 
     localedir = os.path.join(dvar + datadir, 'locale')
 
@@ -665,7 +668,8 @@ def split_locales(d):
     for l in sorted(locales):
         ln = legitimize_package_name(l)
         pkg = pn + '-locale-' + ln
-        packages.append(pkg)
+        packages.insert(locale_index, pkg)
+        locale_index += 1
         d.setVar('FILES:' + pkg, os.path.join(datadir, 'locale', l))
         d.setVar('RRECOMMENDS:' + pkg, '%svirtual-locale-%s' % (mlprefix, ln))
         d.setVar('RPROVIDES:' + pkg, '%s-locale %s%s-translation' % (pn, mlprefix, ln))
-- 
2.43.0

[PATCH v3 2/3] lib/oe/package: add LOCALE_EXTRA_PATHS to add extra locations for locales

[Jonathan GUILLOT]

Some packages may contain localized files not located in
${datadir}/locale. This new variables allows to add these directories in
dedicated locale packages.

Signed-off-by: Jonathan GUILLOT <jonathan@joggee.fr>
---
 meta/conf/documentation.conf |  1 +
 meta/lib/oe/package.py       | 23 ++++++++++++++++++-----
 2 files changed, 19 insertions(+), 5 deletions(-)

diff --git a/meta/conf/documentation.conf b/meta/conf/documentation.conf
index 486c62b6e8..336bddc9d8 100644
--- a/meta/conf/documentation.conf
+++ b/meta/conf/documentation.conf
@@ -271,6 +271,7 @@ LICENSE_PATH[doc] = "Path to additional licenses used during the build."
 LINUX_KERNEL_TYPE[doc] = "Defines the kernel type to be used in assembling the configuration."
 LINUX_VERSION[doc] = "The Linux version from kernel.org on which the Linux kernel image being built using the OpenEmbedded build system is based. You define this variable in the kernel recipe."
 LINUX_VERSION_EXTENSION[doc] = "A string extension compiled into the version string of the Linux kernel built with the OpenEmbedded build system. You define this variable in the kernel recipe."
+LOCALE_EXTRA_PATHS[doc] = "Whitespace separated list of paths extending base ${datadir}/locale that are scanned to construct locale packages."
 LOCALE_UTF8_IS_DEFAULT[doc] = "If set, locale names are renamed such that those lacking an explicit encoding (e.g. en_US) will always be UTF-8, and non-UTF-8 encodings are renamed to, e.g., en_US.ISO-8859-1. Otherwise, the encoding is specified by glibc's SUPPORTED file. Not supported for precompiled locales."
 LOG_DIR[doc] = "Specifies the directory to which the OpenEmbedded build system writes overall log files. The default directory is ${TMPDIR}/log"
 
diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
index 2f7a50e144..4718082bcd 100644
--- a/meta/lib/oe/package.py
+++ b/meta/lib/oe/package.py
@@ -653,24 +653,37 @@ def split_locales(d):
     except ValueError:
         locale_index = len(packages)
 
-    localedir = os.path.join(dvar + datadir, 'locale')
+    localedirs = [os.path.join(dvar + datadir, 'locale')]
 
-    if not cpath.isdir(localedir):
+    for localedir in (d.getVar('LOCALE_EXTRA_PATHS') or "").split():
+        localedirs.append(dvar + localedir)
+
+    locales = set()
+    locale_found = False
+    for localedir in localedirs:
+        if cpath.isdir(localedir):
+            locale_found = True
+            locales.update(os.listdir(localedir))
+
+    if not locale_found:
         bb.debug(1, "No locale files in this package")
         return
 
-    locales = os.listdir(localedir)
-
     summary = d.getVar('SUMMARY') or pn
     description = d.getVar('DESCRIPTION') or ""
     locale_section = d.getVar('LOCALE_SECTION')
     mlprefix = d.getVar('MLPREFIX') or ""
+    dvar_len = len(dvar)
     for l in sorted(locales):
         ln = legitimize_package_name(l)
         pkg = pn + '-locale-' + ln
         packages.insert(locale_index, pkg)
         locale_index += 1
-        d.setVar('FILES:' + pkg, os.path.join(datadir, 'locale', l))
+        files = []
+        for localedir in localedirs:
+            # Remove dvar prefix
+            files.append(os.path.join(localedir[dvar_len:], l))
+        d.setVar('FILES:' + pkg, " ".join(files))
         d.setVar('RRECOMMENDS:' + pkg, '%svirtual-locale-%s' % (mlprefix, ln))
         d.setVar('RPROVIDES:' + pkg, '%s-locale %s%s-translation' % (pn, mlprefix, ln))
         d.setVar('SUMMARY:' + pkg, '%s - %s translations' % (summary, l))
-- 
2.43.0

[PATCH v3 3/3] cups: use LOCALE_EXTRA_PATHS to split localized HTML templates

[Jonathan GUILLOT]

Localized HTML templates in /usr/share/cups/templates are now part of
locale packages.

Signed-off-by: Jonathan GUILLOT <jonathan@joggee.fr>
---
 meta/recipes-extended/cups/cups.inc | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-extended/cups/cups.inc b/meta/recipes-extended/cups/cups.inc
index 31f686cdfd..d0b77823a9 100644
--- a/meta/recipes-extended/cups/cups.inc
+++ b/meta/recipes-extended/cups/cups.inc
@@ -110,6 +110,8 @@ CONFFILES:${PN} += "${sysconfdir}/cups/cupsd.conf"
 
 MULTILIB_SCRIPTS = "${PN}-dev:${bindir}/cups-config"
 
+LOCALE_EXTRA_PATHS = "${datadir}/cups/templates/"
+
 SYSROOT_PREPROCESS_FUNCS += "cups_sysroot_preprocess"
 cups_sysroot_preprocess () {
 	sed -i ${SYSROOT_DESTDIR}${bindir_crossscripts}/cups-config -e 's:cups_datadir=.*:cups_datadir=${datadir}/cups:' -e 's:cups_serverbin=.*:cups_serverbin=${libexecdir}/cups:'
-- 
2.43.0

[PATCH v3 0/3] lib/oe/package: allow to scan extra directories for locale files

[Jonathan GUILLOT]

- Rename LOCALEEXTRADIRS to LOCALE_EXTRA_PATHS to improve readability
- Add documentation

----

Just playing with cups and observe the main packages includes localized HTML templates in /usr/share/cups/templates. Locale packages were only looking for files in ${datadir}/locale. This serie of patches allows to add extra directories to be scanned.
For a next patch, I would also like possible FILES:${PN}-locale-* variables defined in recipe not to be overriden by the split_locales() function.

Regards,
Jonathan GUILLOT

Jonathan GUILLOT (3):
  lib/oe/package: replace in place PN-locale-* packages in PACKAGES
  lib/oe/package: add LOCALE_EXTRA_PATHS to add extra locations for
    locales
  cups: use LOCALE_EXTRA_PATHS to split localized HTML templates

 meta/conf/documentation.conf        |  1 +
 meta/lib/oe/package.py              | 33 ++++++++++++++++++++++-------
 meta/recipes-extended/cups/cups.inc |  2 ++
 3 files changed, 28 insertions(+), 8 deletions(-)


base-commit: e1bffeab27b062884f6366cde24ce1c67e7ec03e
-- 
2.43.0

Re: [OE-core] [PATCH v3 2/3] lib/oe/package: add LOCALE_EXTRA_PATHS to add extra locations for locales

[Ross Burton]

I like the idea of this patch (hardcoding behaviour is bad), but I think it could be generalised some more.

> On 6 Dec 2023, at 16:03, Jonathan GUILLOT via lists.openembedded.org <jonathan=joggee.fr@lists.openembedded.org> wrote:
> -    localedir = os.path.join(dvar + datadir, 'locale')
> +    localedirs = [os.path.join(dvar + datadir, 'locale')]
> 
> -    if not cpath.isdir(localedir):
> +    for localedir in (d.getVar('LOCALE_EXTRA_PATHS') or "").split():
> +        localedirs.append(dvar + localedir)

Instead of hardcoding ${datadir}/locale and then working on LOCALE_EXTRA_PATHS, how about adding a new variable (LOCALE_PATHS?) which is set to ${datadir}/locale by default, and the population code just works on that.  The cups recipe can then LOCALE_PATHS += "${datadir}/cups/templates/“ to add in its own files.

Cheers
Ross

Re: [OE-core] [PATCH v3 2/3] lib/oe/package: add LOCALE_EXTRA_PATHS to add extra locations for locales

[Jonathan GUILLOT]

[-- Attachment #1: Type: text/plain, Size: 1135 bytes --]

Hi Ross,

Thanks for your suggestions. I gave it a try and the code is indeed
cleaner. I will send new patches soon.

Regards,
Jonathan GUILLOT

Le ven. 26 janv. 2024 à 18:13, Ross Burton <Ross.Burton@arm.com> a écrit :

> I like the idea of this patch (hardcoding behaviour is bad), but I think
> it could be generalised some more.
>
> > On 6 Dec 2023, at 16:03, Jonathan GUILLOT via lists.openembedded.org
> <jonathan=joggee.fr@lists.openembedded.org> wrote:
> > -    localedir = os.path.join(dvar + datadir, 'locale')
> > +    localedirs = [os.path.join(dvar + datadir, 'locale')]
> >
> > -    if not cpath.isdir(localedir):
> > +    for localedir in (d.getVar('LOCALE_EXTRA_PATHS') or "").split():
> > +        localedirs.append(dvar + localedir)
>
> Instead of hardcoding ${datadir}/locale and then working on
> LOCALE_EXTRA_PATHS, how about adding a new variable (LOCALE_PATHS?) which
> is set to ${datadir}/locale by default, and the population code just works
> on that.  The cups recipe can then LOCALE_PATHS +=
> "${datadir}/cups/templates/“ to add in its own files.
>
> Cheers
> Ross
>
>

[-- Attachment #2: Type: text/html, Size: 1720 bytes --]