* [PATCH] icu: Add knobs to generate a subset of ICU data
@ 2020-04-03 1:53 Khem Raj
2020-04-03 15:26 ` [OE-core] " Richard Purdie
0 siblings, 1 reply; 5+ messages in thread
From: Khem Raj @ 2020-04-03 1:53 UTC (permalink / raw)
To: openembedded-core; +Cc: Khem Raj, Wouter Meek
Recent versions of ICU (64+) provides a tool for configuring ICU
locale data file with finer granularity [1]
Default generated size for libicudata.so.66.1 is ~27M, which is quite
large for embedded systems and all of them may not even need all locale
data.
This patch calls the icudata buildtool during configure on the icudata
and utilizes a filter called 'filter.json` ( empty by default) to
create the data, default behavior should remain same but someone can
add a filter.json in own layer to configure this data, e.g.
{
"localeFilter": {
"filterType": "language",
"whitelist": [
"en",
"de",
"it"
]
}
}
would only generate the locale data for english/german/italian
This would reduce the size of libicudata.so.66.1 to 12M
[1] https://github.com/unicode-org/icu/blob/master/docs/userguide/icu_data/buildtool.md
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Suggested-by: Wouter Meek <w.meek@metrological.com>
---
meta/recipes-support/icu/icu/filter.json | 2 ++
meta/recipes-support/icu/icu_66.1.bb | 19 ++++++++++++++++---
2 files changed, 18 insertions(+), 3 deletions(-)
create mode 100644 meta/recipes-support/icu/icu/filter.json
diff --git a/meta/recipes-support/icu/icu/filter.json b/meta/recipes-support/icu/icu/filter.json
new file mode 100644
index 0000000000..2c63c08510
--- /dev/null
+++ b/meta/recipes-support/icu/icu/filter.json
@@ -0,0 +1,2 @@
+{
+}
diff --git a/meta/recipes-support/icu/icu_66.1.bb b/meta/recipes-support/icu/icu_66.1.bb
index 5018464c14..a8cfdf9eed 100644
--- a/meta/recipes-support/icu/icu_66.1.bb
+++ b/meta/recipes-support/icu/icu_66.1.bb
@@ -18,7 +18,10 @@ ARM_INSTRUCTION_SET_armv4 = "arm"
ARM_INSTRUCTION_SET_armv5 = "arm"
BASE_SRC_URI = "https://github.com/unicode-org/icu/releases/download/release-${ICU_FOLDER}/icu4c-${ICU_PV}-src.tgz"
-SRC_URI = "${BASE_SRC_URI} \
+DATA_SRC_URI = "https://github.com/unicode-org/icu/releases/download/release-${ICU_FOLDER}/icu4c-${ICU_PV}-data.zip"
+SRC_URI = "${BASE_SRC_URI};name=code \
+ ${DATA_SRC_URI};name=data \
+ file://filter.json \
file://icu-pkgdata-large-cmd.patch \
file://fix-install-manx.patch \
file://0001-Fix-big-endian-build.patch \
@@ -28,8 +31,18 @@ SRC_URI = "${BASE_SRC_URI} \
SRC_URI_append_class-target = "\
file://0001-Disable-LDFLAGSICUDT-for-Linux.patch \
"
-SRC_URI[md5sum] = "b33dc6766711517c98d318447e5110f8"
-SRC_URI[sha256sum] = "52a3f2209ab95559c1cf0a14f24338001f389615bf00e2585ef3dbc43ecf0a2e"
+SRC_URI[code.sha256sum] = "52a3f2209ab95559c1cf0a14f24338001f389615bf00e2585ef3dbc43ecf0a2e"
+SRC_URI[data.sha256sum] = "8be647f738891d2beb79d48f99077b3499948430eae6f1be112553b15ab0243e"
UPSTREAM_CHECK_REGEX = "icu4c-(?P<pver>\d+(_\d+)+)-src"
UPSTREAM_CHECK_URI = "https://github.com/unicode-org/icu/releases"
+
+CONFIGURE_SCRIPT_class-target = "${S}/runConfigureICU"
+CONFIGUREOPTS_prepend_class-target = "Linux "
+CACHED_CONFIGUREVARS_append_class-target = "ICU_DATA_FILTER_FILE=${WORKDIR}/filter.json"
+
+do_configure_prepend_class-target () {
+ rm -rf ${S}/data
+ cp -a ${WORKDIR}/data ${S}/
+}
+
--
2.26.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [OE-core] [PATCH] icu: Add knobs to generate a subset of ICU data
2020-04-03 1:53 [PATCH] icu: Add knobs to generate a subset of ICU data Khem Raj
@ 2020-04-03 15:26 ` Richard Purdie
2020-04-03 15:55 ` Khem Raj
0 siblings, 1 reply; 5+ messages in thread
From: Richard Purdie @ 2020-04-03 15:26 UTC (permalink / raw)
To: Khem Raj, openembedded-core; +Cc: Wouter Meek
On Thu, 2020-04-02 at 18:53 -0700, Khem Raj wrote:
> Recent versions of ICU (64+) provides a tool for configuring ICU
> locale data file with finer granularity [1]
>
> Default generated size for libicudata.so.66.1 is ~27M, which is quite
> large for embedded systems and all of them may not even need all
> locale
> data.
>
> This patch calls the icudata buildtool during configure on the
> icudata
> and utilizes a filter called 'filter.json` ( empty by default) to
> create the data, default behavior should remain same but someone can
> add a filter.json in own layer to configure this data, e.g.
Nice idea, fails on ppc:
https://autobuilder.yoctoproject.org/typhoon/#/builders/107/builds/411
Cheers,
Richard
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [OE-core] [PATCH] icu: Add knobs to generate a subset of ICU data
2020-04-03 15:26 ` [OE-core] " Richard Purdie
@ 2020-04-03 15:55 ` Khem Raj
2020-04-03 17:46 ` Richard Purdie
0 siblings, 1 reply; 5+ messages in thread
From: Khem Raj @ 2020-04-03 15:55 UTC (permalink / raw)
To: Richard Purdie
Cc: Patches and discussions about the oe-core layer, Wouter Meek
On Fri, Apr 3, 2020 at 8:26 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Thu, 2020-04-02 at 18:53 -0700, Khem Raj wrote:
> > Recent versions of ICU (64+) provides a tool for configuring ICU
> > locale data file with finer granularity [1]
> >
> > Default generated size for libicudata.so.66.1 is ~27M, which is quite
> > large for embedded systems and all of them may not even need all
> > locale
> > data.
> >
> > This patch calls the icudata buildtool during configure on the
> > icudata
> > and utilizes a filter called 'filter.json` ( empty by default) to
> > create the data, default behavior should remain same but someone can
> > add a filter.json in own layer to configure this data, e.g.
>
> Nice idea, fails on ppc:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/107/builds/411
>
yeah looking into it. Are there any other arch failures ?
> Cheers,
>
> Richard
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [OE-core] [PATCH] icu: Add knobs to generate a subset of ICU data
2020-04-03 15:55 ` Khem Raj
@ 2020-04-03 17:46 ` Richard Purdie
2020-04-03 21:34 ` Khem Raj
0 siblings, 1 reply; 5+ messages in thread
From: Richard Purdie @ 2020-04-03 17:46 UTC (permalink / raw)
To: Khem Raj; +Cc: Patches and discussions about the oe-core layer, Wouter Meek
On Fri, 2020-04-03 at 08:55 -0700, Khem Raj wrote:
> On Fri, Apr 3, 2020 at 8:26 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Thu, 2020-04-02 at 18:53 -0700, Khem Raj wrote:
> > > Recent versions of ICU (64+) provides a tool for configuring ICU
> > > locale data file with finer granularity [1]
> > >
> > > Default generated size for libicudata.so.66.1 is ~27M, which is
> > > quite
> > > large for embedded systems and all of them may not even need all
> > > locale
> > > data.
> > >
> > > This patch calls the icudata buildtool during configure on the
> > > icudata
> > > and utilizes a filter called 'filter.json` ( empty by default) to
> > > create the data, default behavior should remain same but someone
> > > can
> > > add a filter.json in own layer to configure this data, e.g.
> >
> > Nice idea, fails on ppc:
> >
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/107/builds/411
> >
>
> yeah looking into it. Are there any other arch failures ?
mips:
https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/1749
mips64:
https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/1753
edgerouter:
https://autobuilder.yoctoproject.org/typhoon/#/builders/111/builds/414
Cheers,
Richard
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [OE-core] [PATCH] icu: Add knobs to generate a subset of ICU data
2020-04-03 17:46 ` Richard Purdie
@ 2020-04-03 21:34 ` Khem Raj
0 siblings, 0 replies; 5+ messages in thread
From: Khem Raj @ 2020-04-03 21:34 UTC (permalink / raw)
To: Richard Purdie
Cc: Patches and discussions about the oe-core layer, Wouter Meek
On Fri, Apr 3, 2020 at 10:47 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Fri, 2020-04-03 at 08:55 -0700, Khem Raj wrote:
> > On Fri, Apr 3, 2020 at 8:26 AM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > > On Thu, 2020-04-02 at 18:53 -0700, Khem Raj wrote:
> > > > Recent versions of ICU (64+) provides a tool for configuring ICU
> > > > locale data file with finer granularity [1]
> > > >
> > > > Default generated size for libicudata.so.66.1 is ~27M, which is
> > > > quite
> > > > large for embedded systems and all of them may not even need all
> > > > locale
> > > > data.
> > > >
> > > > This patch calls the icudata buildtool during configure on the
> > > > icudata
> > > > and utilizes a filter called 'filter.json` ( empty by default) to
> > > > create the data, default behavior should remain same but someone
> > > > can
> > > > add a filter.json in own layer to configure this data, e.g.
> > >
> > > Nice idea, fails on ppc:
> > >
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/107/builds/411
> > >
> >
> > yeah looking into it. Are there any other arch failures ?
>
> mips:
> https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/1749
> mips64:
> https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/1753
> edgerouter:
> https://autobuilder.yoctoproject.org/typhoon/#/builders/111/builds/414
>
Thanks, I think I understand the problem, its when target endianness
is bigendian
and the host is little-endian, I have sent a v2 which should fix it.
Give v2 a shot.
> Cheers,
>
> Richard
>
>
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-04-03 21:34 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-04-03 1:53 [PATCH] icu: Add knobs to generate a subset of ICU data Khem Raj
2020-04-03 15:26 ` [OE-core] " Richard Purdie
2020-04-03 15:55 ` Khem Raj
2020-04-03 17:46 ` Richard Purdie
2020-04-03 21:34 ` Khem Raj
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.