libsoup:add a new routine & build as native package

libsoup:add a new routine & build as native package

[Jiang Lu]

Hi all,

This rr including 2 patch:
1.add a new routine for libsoup with name soup_uri_to_string_with_password()
and make libsoup compiled as a native package.
2.make glib-networking compiled as a native package.

Make this change for ostree utilies, which needs libsoup when building ostree image
on host.

Thanks
Jiang Lu

[oe-core 1/2] libsoup:add soup_uri_to_string_with_password

[Jiang Lu]

The existed soup_uri_to_string does not have password, add a
function to support it.

Besides, allow libsoup compiled as a native package and add
glib-networking into dependency list.

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Jiang Lu <lu.jiang@windriver.com>
---
 ...add-soup_uri_to_string_with_password.patch | 67 +++++++++++++++++++
 .../libsoup/libsoup-2.4_2.64.2.bb             |  5 +-
 2 files changed, 71 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-support/libsoup/libsoup-2.4/0001-add-soup_uri_to_string_with_password.patch

diff --git a/meta/recipes-support/libsoup/libsoup-2.4/0001-add-soup_uri_to_string_with_password.patch b/meta/recipes-support/libsoup/libsoup-2.4/0001-add-soup_uri_to_string_with_password.patch
new file mode 100644
index 0000000000..c679d6b9e5
--- /dev/null
+++ b/meta/recipes-support/libsoup/libsoup-2.4/0001-add-soup_uri_to_string_with_password.patch
@@ -0,0 +1,67 @@
+From 82c2f059bc7c491c8c4c728408eb913bfc4a6f0a Mon Sep 17 00:00:00 2001
+From: Hongxu Jia <hongxu.jia@windriver.com>
+Date: Wed, 6 Jun 2018 22:37:30 +0800
+Subject: [PATCH] add soup_uri_to_string_with_password
+
+The existed soup_uri_to_string does not have password, add a
+function to support it.
+
+Upstream-Status: Pending
+
+Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
+---
+ libsoup/soup-uri.c | 21 +++++++++++++++++++++
+ libsoup/soup-uri.h |  5 +++++
+ 2 files changed, 26 insertions(+)
+
+diff --git a/libsoup/soup-uri.c b/libsoup/soup-uri.c
+index 3eafd87..8421f1c 100644
+--- a/libsoup/soup-uri.c
++++ b/libsoup/soup-uri.c
+@@ -619,6 +619,27 @@ soup_uri_to_string (SoupURI *uri, gboolean just_path_and_query)
+ }
+ 
+ /**
++ * soup_uri_to_string_with_password:
++ * @uri: a #SoupURI
++ * @just_path_and_query: if %TRUE, output just the path and query portions
++ *
++ * Returns a string representing @uri.
++ *
++ * If @just_path_and_query is %TRUE, this concatenates the path and query
++ * together. That is, it constructs the string that would be needed in
++ * the Request-Line of an HTTP request for @uri.
++ *
++ * Note that the output will contain a password, if @uri does.
++ *
++ * Return value: a string representing @uri, which the caller must free.
++ **/
++char *
++soup_uri_to_string_with_password (SoupURI *uri, gboolean just_path_and_query)
++{
++	return soup_uri_to_string_internal (uri, just_path_and_query, TRUE, FALSE);
++}
++
++/**
+  * soup_uri_copy:
+  * @uri: a #SoupURI
+  *
+diff --git a/libsoup/soup-uri.h b/libsoup/soup-uri.h
+index b9360c6..af702d3 100644
+--- a/libsoup/soup-uri.h
++++ b/libsoup/soup-uri.h
+@@ -57,6 +57,11 @@ char   	   *soup_uri_to_string             (SoupURI    *uri,
+ 					    gboolean    just_path_and_query);
+ 
+ SOUP_AVAILABLE_IN_2_4
++char   	   *soup_uri_to_string_with_password             (SoupURI    *uri,
++					    gboolean    just_path_and_query);
++
++
++SOUP_AVAILABLE_IN_2_4
+ SoupURI	   *soup_uri_copy                  (SoupURI    *uri);
+ 
+ SOUP_AVAILABLE_IN_2_4
+-- 
+2.7.4
+
diff --git a/meta/recipes-support/libsoup/libsoup-2.4_2.64.2.bb b/meta/recipes-support/libsoup/libsoup-2.4_2.64.2.bb
index ae7c1a6863..c2284c0458 100644
--- a/meta/recipes-support/libsoup/libsoup-2.4_2.64.2.bb
+++ b/meta/recipes-support/libsoup/libsoup-2.4_2.64.2.bb
@@ -11,6 +11,7 @@ SHRT_VER = "${@d.getVar('PV').split('.')[0]}.${@d.getVar('PV').split('.')[1]}"
 
 SRC_URI = "${GNOME_MIRROR}/libsoup/${SHRT_VER}/libsoup-${PV}.tar.xz \
            file://0001-Do-not-enforce-no-introspection-when-cross-building.patch \
+           file://0001-add-soup_uri_to_string_with_password.patch \
            "
 SRC_URI[md5sum] = "cac755dc6c6acd6e0c70007f547548f5"
 SRC_URI[sha256sum] = "75ddc194a5b1d6f25033bb9d355f04bfe5c03e0e1c71ed0774104457b3a786c6"
@@ -48,4 +49,6 @@ EXTRA_OEMESON_append_class-target = " ${@bb.utils.contains('GTKDOC_ENABLED', 'Tr
 DEBIAN_NOAUTONAME_${PN} = "1"
 
 # glib-networking is needed for SSL, proxies, etc.
-RRECOMMENDS_${PN} = "glib-networking"
+DEPENDS += "glib-networking"
+
+BBCLASSEXTEND = "native"
-- 
2.17.1

[oe-core 2/2] glib-networking:enable glib-networking complie as native package

[Jiang Lu]

Enable glib-networking compile as a native package, for it is depended by
libsoup.

Signed-off-by: Jiang Lu <lu.jiang@windriver.com>
---
 .../glib-networking/glib-networking_2.58.0.bb | 31 +++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/meta/recipes-core/glib-networking/glib-networking_2.58.0.bb b/meta/recipes-core/glib-networking/glib-networking_2.58.0.bb
index f3190e1cae..35b4a3fa76 100644
--- a/meta/recipes-core/glib-networking/glib-networking_2.58.0.bb
+++ b/meta/recipes-core/glib-networking/glib-networking_2.58.0.bb
@@ -29,3 +29,34 @@ FILES_${PN} += "\
                 "
 FILES_${PN}-dev += "${libdir}/gio/modules/libgio*.la"
 FILES_${PN}-staticdev += "${libdir}/gio/modules/libgio*.a"
+
+# Make sure we compile with ca-certificates support enabled.
+PACKAGECONFIG_append = " ca-certificates"
+
+DEPENDS += "ca-certificates"
+RDEPENDS_${PN} += "ca-certificates"
+
+# We need native version for ostree-/flatpak-native.
+BBCLASSEXTEND = "native"
+# OE-core's relocatable.bbclass assumes that every package which
+# ends up creating a ${libdir}/pkgconfig directory in its sysroot
+# will always also install .pc-files there and tries to uncondi-
+# tionally update paths in those files using globbing that fails
+# if no such files are present. This presumption is not true for
+# glib-networking which happens to create a directory by dereferencing
+# a GIO pkgconfig variable which in turn is defined relative to
+# the pkgconfig directory (${pcfiledir}/../...), causing pkgconfig
+# to get created.
+#
+# Could be worked around in the upatream recipe but since that
+# does not provide/create native versions of the package and since
+# this problem is related to native packages, we work around it here.
+#
+do_install_append_class-native () {
+    for _pc in ${D}${libdir}/pkgconfig/*.pc; do
+        case $_pc in
+            *'*.pc') rm -fr ${D}${libdir}/pkgconfig;;
+            *.pc)    break;;
+        esac
+    done
+}
-- 
2.17.1

Re: [oe-core 2/2] glib-networking:enable glib-networking complie as native package

[Richard Purdie]

On Tue, 2019-03-19 at 15:54 +0800, Jiang Lu wrote:
> Enable glib-networking compile as a native package, for it is
> depended by
> libsoup.
> 
> Signed-off-by: Jiang Lu <lu.jiang@windriver.com>
> ---
>  .../glib-networking/glib-networking_2.58.0.bb | 31
> +++++++++++++++++++
>  1 file changed, 31 insertions(+)
> 
> diff --git a/meta/recipes-core/glib-networking/glib-
> networking_2.58.0.bb b/meta/recipes-core/glib-networking/glib-
> networking_2.58.0.bb
> index f3190e1cae..35b4a3fa76 100644
> --- a/meta/recipes-core/glib-networking/glib-networking_2.58.0.bb
> +++ b/meta/recipes-core/glib-networking/glib-networking_2.58.0.bb
> @@ -29,3 +29,34 @@ FILES_${PN} += "\
>                  "
>  FILES_${PN}-dev += "${libdir}/gio/modules/libgio*.la"
>  FILES_${PN}-staticdev += "${libdir}/gio/modules/libgio*.a"
> +
> +# Make sure we compile with ca-certificates support enabled.
> +PACKAGECONFIG_append = " ca-certificates"
> +
> +DEPENDS += "ca-certificates"
> +RDEPENDS_${PN} += "ca-certificates"
> +
> +# We need native version for ostree-/flatpak-native.
> +BBCLASSEXTEND = "native"
> +# OE-core's relocatable.bbclass assumes that every package which
> +# ends up creating a ${libdir}/pkgconfig directory in its sysroot
> +# will always also install .pc-files there and tries to uncondi-
> +# tionally update paths in those files using globbing that fails
> +# if no such files are present. This presumption is not true for
> +# glib-networking which happens to create a directory by
> dereferencing
> +# a GIO pkgconfig variable which in turn is defined relative to
> +# the pkgconfig directory (${pcfiledir}/../...), causing pkgconfig
> +# to get created.
> +#
> +# Could be worked around in the upatream recipe but since that
> +# does not provide/create native versions of the package and since
> +# this problem is related to native packages, we work around it
> here.
> +#
> +do_install_append_class-native () {
> +    for _pc in ${D}${libdir}/pkgconfig/*.pc; do
> +        case $_pc in
> +            *'*.pc') rm -fr ${D}${libdir}/pkgconfig;;
> +            *.pc)    break;;
> +        esac
> +    done
> +}

This reads as if there is a bbappend in some layer which is now being
dumped verbatim into the upstream recipe with no thought as to what
makes sense.

We don't do PACKAGECONFIG_append in recipes and we don't hack .pc files
like this. The dependencies suggest we should also fix the
PACKAGECONFIG DEPENDS/RDEPENDS too.

This needs more work.

Also, for the first patch, its unclear why we need/want the patch
present there and the commit shortlog doesn't really describe the
multiple things the patch is doing.

Cheers,

Richard

Re: [oe-core 2/2] glib-networking:enable glib-networking complie as native package

[Peter Kjellerstedt]

> -----Original Message-----
> From: openembedded-core-bounces@lists.openembedded.org <openembedded-
> core-bounces@lists.openembedded.org> On Behalf Of Jiang Lu
> Sent: den 19 mars 2019 08:55
> To: lu.jiang@windriver.com; openembedded-core@lists.openembedded.org
> Subject: [OE-core] [oe-core 2/2] glib-networking:enable glib-networking
> complie as native package
> 
> Enable glib-networking compile as a native package, for it is depended
> by
> libsoup.
> 
> Signed-off-by: Jiang Lu <lu.jiang@windriver.com>
> ---
>  .../glib-networking/glib-networking_2.58.0.bb | 31 +++++++++++++++++++
>  1 file changed, 31 insertions(+)
> 
> diff --git a/meta/recipes-core/glib-networking/glib-
> networking_2.58.0.bb b/meta/recipes-core/glib-networking/glib-
> networking_2.58.0.bb
> index f3190e1cae..35b4a3fa76 100644
> --- a/meta/recipes-core/glib-networking/glib-networking_2.58.0.bb
> +++ b/meta/recipes-core/glib-networking/glib-networking_2.58.0.bb
> @@ -29,3 +29,34 @@ FILES_${PN} += "\
>                  "
>  FILES_${PN}-dev += "${libdir}/gio/modules/libgio*.la"
>  FILES_${PN}-staticdev += "${libdir}/gio/modules/libgio*.a"
> +
> +# Make sure we compile with ca-certificates support enabled.
> +PACKAGECONFIG_append = " ca-certificates"
> +
> +DEPENDS += "ca-certificates"
> +RDEPENDS_${PN} += "ca-certificates"
> +
> +# We need native version for ostree-/flatpak-native.
> +BBCLASSEXTEND = "native"
> +# OE-core's relocatable.bbclass assumes that every package which
> +# ends up creating a ${libdir}/pkgconfig directory in its sysroot
> +# will always also install .pc-files there and tries to uncondi-
> +# tionally update paths in those files using globbing that fails
> +# if no such files are present. This presumption is not true for
> +# glib-networking which happens to create a directory by dereferencing
> +# a GIO pkgconfig variable which in turn is defined relative to
> +# the pkgconfig directory (${pcfiledir}/../...), causing pkgconfig
> +# to get created.

How about fixing relocatable_native_pcfiles() in relocatable.bbclass 
instead so that it ignores empty pkgconfig directories?

> +#
> +# Could be worked around in the upatream recipe but since that
> +# does not provide/create native versions of the package and since
> +# this problem is related to native packages, we work around it here.
> +#
> +do_install_append_class-native () {
> +    for _pc in ${D}${libdir}/pkgconfig/*.pc; do
> +        case $_pc in
> +            *'*.pc') rm -fr ${D}${libdir}/pkgconfig;;
> +            *.pc)    break;;
> +        esac
> +    done

Why complicate things? Just remove the directory if it exists 
and is empty:

	rmdir ${D}${libdir}/pkgconfig 2>/dev/null || :

> +}
> --
> 2.17.1

//Peter

Re: [oe-core 2/2] glib-networking:enable glib-networking complie as native package

[Burton, Ross]

On Tue, 19 Mar 2019 at 15:37, Peter Kjellerstedt
<peter.kjellerstedt@axis.com> wrote:
> > +do_install_append_class-native () {
> > +    for _pc in ${D}${libdir}/pkgconfig/*.pc; do
> > +        case $_pc in
> > +            *'*.pc') rm -fr ${D}${libdir}/pkgconfig;;
> > +            *.pc)    break;;
> > +        esac
> > +    done
>
> Why complicate things? Just remove the directory if it exists
> and is empty:
>
>         rmdir ${D}${libdir}/pkgconfig 2>/dev/null || :

Bikeshed!  Or

rmdir --ignore-fail-on-non-empty ${D}${libdir}/pkgconfig

Ross

Re: [oe-core 2/2] glib-networking:enable glib-networking complie as native package

[Jiang]

在 2019/3/19 下午11:36, Peter Kjellerstedt 写道:
>> -----Original Message-----
>> From: openembedded-core-bounces@lists.openembedded.org <openembedded-
>> core-bounces@lists.openembedded.org> On Behalf Of Jiang Lu
>> Sent: den 19 mars 2019 08:55
>> To: lu.jiang@windriver.com; openembedded-core@lists.openembedded.org
>> Subject: [OE-core] [oe-core 2/2] glib-networking:enable glib-networking
>> complie as native package
>>
>> Enable glib-networking compile as a native package, for it is depended
>> by
>> libsoup.
>>
>> Signed-off-by: Jiang Lu <lu.jiang@windriver.com>
>> ---
>>   .../glib-networking/glib-networking_2.58.0.bb | 31 +++++++++++++++++++
>>   1 file changed, 31 insertions(+)
>>
>> diff --git a/meta/recipes-core/glib-networking/glib-
>> networking_2.58.0.bb b/meta/recipes-core/glib-networking/glib-
>> networking_2.58.0.bb
>> index f3190e1cae..35b4a3fa76 100644
>> --- a/meta/recipes-core/glib-networking/glib-networking_2.58.0.bb
>> +++ b/meta/recipes-core/glib-networking/glib-networking_2.58.0.bb
>> @@ -29,3 +29,34 @@ FILES_${PN} += "\
>>                   "
>>   FILES_${PN}-dev += "${libdir}/gio/modules/libgio*.la"
>>   FILES_${PN}-staticdev += "${libdir}/gio/modules/libgio*.a"
>> +
>> +# Make sure we compile with ca-certificates support enabled.
>> +PACKAGECONFIG_append = " ca-certificates"
>> +
>> +DEPENDS += "ca-certificates"
>> +RDEPENDS_${PN} += "ca-certificates"
>> +
>> +# We need native version for ostree-/flatpak-native.
>> +BBCLASSEXTEND = "native"
>> +# OE-core's relocatable.bbclass assumes that every package which
>> +# ends up creating a ${libdir}/pkgconfig directory in its sysroot
>> +# will always also install .pc-files there and tries to uncondi-
>> +# tionally update paths in those files using globbing that fails
>> +# if no such files are present. This presumption is not true for
>> +# glib-networking which happens to create a directory by dereferencing
>> +# a GIO pkgconfig variable which in turn is defined relative to
>> +# the pkgconfig directory (${pcfiledir}/../...), causing pkgconfig
>> +# to get created.
> How about fixing relocatable_native_pcfiles() in relocatable.bbclass
> instead so that it ignores empty pkgconfig directories?


Thanks,

I prefer add a .pc file existence checking in relocatable_native_pcfiles()

-       if [ -d ${SYSROOT_DESTDIR}${libdir}/pkgconfig ]; then
+       filecnt=`ls -l ${SYSROOT_DESTDIR}${libdir}/pkgconfig/*.pc 
2>/dev/null | wc -l`
+       if [ $filecnt -gt 0 ]; then

Thanks

Jiang Lu

>
>> +#
>> +# Could be worked around in the upatream recipe but since that
>> +# does not provide/create native versions of the package and since
>> +# this problem is related to native packages, we work around it here.
>> +#
>> +do_install_append_class-native () {
>> +    for _pc in ${D}${libdir}/pkgconfig/*.pc; do
>> +        case $_pc in
>> +            *'*.pc') rm -fr ${D}${libdir}/pkgconfig;;
>> +            *.pc)    break;;
>> +        esac
>> +    done
> Why complicate things? Just remove the directory if it exists
> and is empty:
>
> 	rmdir ${D}${libdir}/pkgconfig 2>/dev/null || :
>
>> +}
>> --
>> 2.17.1
> //Peter
>
>