All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH v3 0/3] qt5webkit: fix build issue using system leveldb
@ 2019-01-02 20:56 Gaël PORTAY
  2019-01-02 20:56 ` [Buildroot] [PATCH v3 1/3] leveldb: install memenv static library and header Gaël PORTAY
                   ` (3 more replies)
  0 siblings, 4 replies; 6+ messages in thread
From: Gaël PORTAY @ 2019-01-02 20:56 UTC (permalink / raw)
  To: buildroot

Hi all,

This patchset fixes the build issue reported by autobuilder [0].

        /home/naourr/work/instance-2/output/build/qt5webkit-5.9.1/Source/WebCore//.obj/platform/leveldb/LevelDBDatabase.o: In function `WebCore::LevelDBDatabase::openInMemory(WebCore::LevelDBComparator const*)':
LevelDBDatabase.cpp.text._ZN7WebCore15LevelDBDatabase12openInMemoryEPKNS_17LevelDBComparatorE+0x34): undefined reference to `leveldb::NewMemEnv(leveldb::Env*)'
        collect2: error: ld returned 1 exit status
        make[3]: *** [Makefile.api:97: ../lib/libQt5WebKit.so.5.9.1] Error 1

The issue happens when the package leveldb is enabled.

QtWebKit builds its own copy of leveldb [1] (as a third-party) if the system
does not provided it (i.e. buildroot). It builds it differently and this is the
origin of that issue. Instead of using the Makefile provided by leveldb [2],
QtWebKit uses qmake to build that library [3].

The missing symbol issue happens because the symbol leveldb::NewMemEnv is
bundled in the static library libmemenv.a (aside libleveldb.so). This static
library consists of this single symbol which is like an extra that is built but
*NOT* shipped by default at installation in the staging directory.
Unfortunatly, that symbol is required later by WebCore [4].

The missing symbol issue happens because the symbol leveldb::NewMemEnv
is bundled in the static library libmemenv.a (aside libleveldb.so).
This static library consists of this single symbol which is like an
extra that is built but *NOT* shipped by default at installation in the
staging directory. Unfortunatly, that symbol is required later by
WebCore [4].

The copy built by QtWebKit is an all-in-one library including both
libleveldb and libmemenv; thus QtWebKit links against libleveldb only.
Also, the linker finds the buildroot's copy first (not the third-party):
that explains why it is complaining about a missing symbol. That copy
does not have the symbol leveldb::NewMemEnv.

Fortunatly, QtWebKit provides a facility to link against the system
leveldb package. The qmake flag WEBKIT_CONFIG+=use_system_leveldb tells
Qt5WebKit to link against libleveldb *AND* libmemenv [5].

To fix that issue, this patchset builds the leveldb package and tells QtWebKit
to use it instead of building its own copy.

The first two patches concern the leveldb package.

The first one adds installation of headers and the missing static library in
the staging directory.

The second one patches leveldb to generate position independant code for the
static library libmemenv.a; in order let it linkable by the Qt5WebKit module.
Without that patch, the build raises the error below:

        /home/gportay/src/buildroot/output/host/opt/ext-toolchain/bin/../lib/gcc/x86_64-amd-linux-gnu/6.2.0/../../../../x86_64-amd-linux-gnu/bin/ld: /home/gportay/src/buildroot/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/libmemenv.a(memenv.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
        /home/gportay/src/buildroot/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/libmemenv.a: error adding symbols: Bad value
        collect2: error: ld returned 1 exit status

It it not relevant to upstream that patch since the project has already moved
to cmake [6] for the (upcoming?) version 1.21. Backporting the upstreamed
patches was way too complicated.

The last patch selects the leveldb package in qt5webkit and sets the
appropriate qmake flag to tell it to use the package provided by
buildroot.

[0]: http://autobuild.buildroot.net/results/8afc8771741f62da3b2117d2124d1c6f2d941903/
[1]: https://github.com/qt/qtwebkit/tree/5.9/Source/ThirdParty/leveldb
[2]: https://github.com/qt/qtwebkit/blob/5.9/Source/ThirdParty/leveldb/Makefile#L167-L169
[3]: https://github.com/qt/qtwebkit/blob/5.9/Source/ThirdParty/leveldb/Target.pri#L80
[4]: https://github.com/qt/qtwebkit/blob/5.9/Source/WebCore/platform/leveldb/LevelDBDatabase.cpp#L185
[5]: https://github.com/qt/qtwebkit/blob/5.9/Source/WebCore/WebCore.pri#L254
[6]: https://github.com/google/leveldb/commit/739c25100e46576cdcdfff2d6f43f9f7008103c7

Changes since v2:
 - remove option BR2_PACKAGE_LEVELDB_MEMENV
 - use install -D option and full destination path

Changes since v1:
 - add option BR2_PACKAGE_LEVELDB_MEMENV to install static library and
   header

Regards,
Ga?l PORTAY (3):
  leveldb: install memenv static library and header
  leveldb: generate pic for static libraries
  qt5webkit: select leveldb package and memenv

 ...n-independant-code-for-static-librar.patch | 52 +++++++++++++++++++
 package/leveldb/leveldb.mk                    |  2 +
 package/qt5/qt5webkit/Config.in               |  3 ++
 package/qt5/qt5webkit/qt5webkit.mk            |  4 +-
 4 files changed, 59 insertions(+), 2 deletions(-)
 create mode 100644 package/leveldb/0001-Generate-position-independant-code-for-static-librar.patch

-- 
2.20.1

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v3 1/3] leveldb: install memenv static library and header
  2019-01-02 20:56 [Buildroot] [PATCH v3 0/3] qt5webkit: fix build issue using system leveldb Gaël PORTAY
@ 2019-01-02 20:56 ` Gaël PORTAY
  2019-01-02 20:56 ` [Buildroot] [PATCH v3 2/3] leveldb: generate pic for static libraries Gaël PORTAY
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 6+ messages in thread
From: Gaël PORTAY @ 2019-01-02 20:56 UTC (permalink / raw)
  To: buildroot

The project builds a tiny static library that consists of a single
symbol which creates an in-memory LevelDB database.

That library is not installed by default and may be used by other
projects.

This commit installs in the staging directory the libmemenv.a static
library and the memenv.h header file.

Signed-off-by: Ga?l PORTAY <gael.portay@collabora.com>
---
 package/leveldb/leveldb.mk | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/package/leveldb/leveldb.mk b/package/leveldb/leveldb.mk
index 54942a0f27..7999373770 100644
--- a/package/leveldb/leveldb.mk
+++ b/package/leveldb/leveldb.mk
@@ -25,6 +25,8 @@ define LEVELDB_INSTALL_STAGING_CMDS
 	$(TARGET_MAKE_ENV) $(MAKE) \
 		INSTALL_ROOT=$(STAGING_DIR) INSTALL_PREFIX=/usr \
 		$(LEVELDB_MAKE_ARGS) -C $(@D) install
+	$(INSTALL) -D -m 0644 $(@D)/out-static/libmemenv.a $(STAGING_DIR)/usr/lib/libmemenv.a
+	$(INSTALL) -D -m 0644 $(@D)/helpers/memenv/memenv.h $(STAGING_DIR)/usr/include/helpers/memenv/memenv.h
 endef
 
 define LEVELDB_INSTALL_TARGET_CMDS
-- 
2.20.1

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v3 2/3] leveldb: generate pic for static libraries
  2019-01-02 20:56 [Buildroot] [PATCH v3 0/3] qt5webkit: fix build issue using system leveldb Gaël PORTAY
  2019-01-02 20:56 ` [Buildroot] [PATCH v3 1/3] leveldb: install memenv static library and header Gaël PORTAY
@ 2019-01-02 20:56 ` Gaël PORTAY
  2019-01-02 20:56 ` [Buildroot] [PATCH v3 3/3] qt5webkit: select leveldb package and memenv Gaël PORTAY
  2019-03-07 22:27 ` [Buildroot] [PATCH v3 0/3] qt5webkit: fix build issue using system leveldb Arnout Vandecappelle
  3 siblings, 0 replies; 6+ messages in thread
From: Gaël PORTAY @ 2019-01-02 20:56 UTC (permalink / raw)
  To: buildroot

The project's static libraries are not compiled with the -fPIC compiler
flag that prevents any libraries to link against those libraries.

This commit adds a patch that sets the -fPIC compiler flag to the list of
CFLAGS/CXXFLAGS.

The project now generates position independant code for all of its
outputs (i.e. not limited anymore to its shared libraries).

Fixes:

	/home/gportay/src/buildroot/output/host/opt/ext-toolchain/bin/../lib/gcc/x86_64-amd-linux-gnu/6.2.0/../../../../x86_64-amd-linux-gnu/bin/ld: /home/gportay/src/buildroot/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/libmemenv.a(memenv.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
	/home/gportay/src/buildroot/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/libmemenv.a: error adding symbols: Bad value
	collect2: error: ld returned 1 exit status

Signed-off-by: Ga?l PORTAY <gael.portay@collabora.com>
---
 ...n-independant-code-for-static-librar.patch | 52 +++++++++++++++++++
 1 file changed, 52 insertions(+)
 create mode 100644 package/leveldb/0001-Generate-position-independant-code-for-static-librar.patch

diff --git a/package/leveldb/0001-Generate-position-independant-code-for-static-librar.patch b/package/leveldb/0001-Generate-position-independant-code-for-static-librar.patch
new file mode 100644
index 0000000000..dce06ec725
--- /dev/null
+++ b/package/leveldb/0001-Generate-position-independant-code-for-static-librar.patch
@@ -0,0 +1,52 @@
+From 6ed1b57ef6bcee0d497c181730710b2b0fafbfb3 Mon Sep 17 00:00:00 2001
+From: =?utf-8?q?Ga=C3=ABl=20PORTAY?= <gael.portay@savoirfairelinux.com>
+Date: Fri, 31 Aug 2018 12:23:46 -0400
+Subject: [PATCH] Generate position independant code for static library
+MIME-Version: 1.0
+Content-Type: text/plain; charset=utf-8
+Content-Transfer-Encoding: 8bit
+
+Currently, only shared libraries are using the PIC flag.
+
+Generalize this flag for static libraries in order to let them linkable
+by dynamic libraries.
+
+Fixes:
+
+	/home/gportay/src/buildroot/output/host/opt/ext-toolchain/bin/../lib/gcc/x86_64-amd-linux-gnu/6.2.0/../../../../x86_64-amd-linux-gnu/bin/ld: /home/gportay/src/buildroot/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/libmemenv.a(memenv.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
+	/home/gportay/src/buildroot/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/libmemenv.a: error adding symbols: Bad value
+	collect2: error: ld returned 1 exit status
+
+Upstream-Status: Inappropriate [upstream has migrated to cmake]
+Signed-off-by: Ga?l PORTAY <gael.portay@savoirfairelinux.com>
+---
+ build_detect_platform | 6 +++---
+ 1 file changed, 3 insertions(+), 3 deletions(-)
+
+diff --git a/build_detect_platform b/build_detect_platform
+index d2a20ce..4839444 100755
+--- a/build_detect_platform
++++ b/build_detect_platform
+@@ -55,8 +55,8 @@ fi
+ 
+ COMMON_FLAGS=
+ CROSS_COMPILE=
+-PLATFORM_CCFLAGS=
+-PLATFORM_CXXFLAGS=
++PLATFORM_CCFLAGS="-fPIC"
++PLATFORM_CXXFLAGS="-fPIC"
+ PLATFORM_LDFLAGS=
+ PLATFORM_LIBS=
+ PLATFORM_SHARED_EXT="so"
+@@ -197,7 +197,7 @@ else
+ EOF
+     if [ "$?" = 0 ]; then
+         COMMON_FLAGS="$COMMON_FLAGS -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT"
+-        PLATFORM_CXXFLAGS="-std=c++0x"
++        PLATFORM_CXXFLAGS="$PLATFORM_CXXFLAGS -std=c++0x"
+     else
+         COMMON_FLAGS="$COMMON_FLAGS -DLEVELDB_PLATFORM_POSIX"
+     fi
+-- 
+2.18.0
+
-- 
2.20.1

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v3 3/3] qt5webkit: select leveldb package and memenv
  2019-01-02 20:56 [Buildroot] [PATCH v3 0/3] qt5webkit: fix build issue using system leveldb Gaël PORTAY
  2019-01-02 20:56 ` [Buildroot] [PATCH v3 1/3] leveldb: install memenv static library and header Gaël PORTAY
  2019-01-02 20:56 ` [Buildroot] [PATCH v3 2/3] leveldb: generate pic for static libraries Gaël PORTAY
@ 2019-01-02 20:56 ` Gaël PORTAY
  2019-03-07 22:27 ` [Buildroot] [PATCH v3 0/3] qt5webkit: fix build issue using system leveldb Arnout Vandecappelle
  3 siblings, 0 replies; 6+ messages in thread
From: Gaël PORTAY @ 2019-01-02 20:56 UTC (permalink / raw)
  To: buildroot

This patch fixes the build issue reported by autobuilder [0].

        /home/naourr/work/instance-2/output/build/qt5webkit-5.9.1/Source/WebCore//.obj/platform/leveldb/LevelDBDatabase.o: In function
	`WebCore::LevelDBDatabase::openInMemory(WebCore::LevelDBComparator const*)':
	LevelDBDatabase.cpp.text._ZN7WebCore15LevelDBDatabase12openInMemoryEPKNS_17LevelDBComparatorE+0x34): undefined reference to `leveldb::NewMemEnv(leveldb::Env*)'
        collect2: error: ld returned 1 exit status
        make[3]: *** [Makefile.api:97: ../lib/libQt5WebKit.so.5.9.1]
	Error 1

The issue happens when both packages leveldb and qt5webkit are enabled.

QtWebKit builds its own copy of leveldb [1] (as a third-party) if the
system does not provided it (i.e. buildroot). It builds it differently
and this is the origin of that issue. Instead of using the Makefile
provided by leveldb [2], QtWebKit uses qmake to build that library [3].

The missing symbol issue happens because the symbol leveldb::NewMemEnv
is bundled in the static library libmemenv.a (aside libleveldb.so).
This static library consists of this single symbol which is like an
extra that is built but *NOT* shipped by default at installation in the
staging directory. Unfortunatly, that symbol is required later by
WebCore [4].

The copy built by QtWebKit is an all-in-one library including both
libleveldb and libmemenv; thus QtWebKit links against libleveldb only.
Also, the linker finds the buildroot's copy first (not the third-party):
that explains why it is complaining about a missing symbol. That copy
does not have the symbol leveldb::NewMemEnv.

Fortunatly, QtWebKit provides a facility to link against the system
leveldb package. The qmake flag WEBKIT_CONFIG+=use_system_leveldb tells
Qt5WebKit to link against libleveldb *AND* libmemenv [5].

To fix that issue, this commit selects the package leveldb that now
installs the libmemenv static library and its header. It ensures that
QtWebKit has every thing it needs to be built. It also sets the
appropriate qmake configure flags to tells QtWebKit to use the leveldb
copy build by the system (instead of the one that can be build internaly
as a third-party).

[0]: http://autobuild.buildroot.net/results/8afc8771741f62da3b2117d2124d1c6f2d941903/
[1]: https://github.com/qt/qtwebkit/tree/5.9/Source/ThirdParty/leveldb
[2]: https://github.com/qt/qtwebkit/blob/5.9/Source/ThirdParty/leveldb/Makefile#L167-L169
[3]: https://github.com/qt/qtwebkit/blob/5.9/Source/ThirdParty/leveldb/Target.pri#L80
[4]: https://github.com/qt/qtwebkit/blob/5.9/Source/WebCore/platform/leveldb/LevelDBDatabase.cpp#L185
[5]: https://github.com/qt/qtwebkit/blob/5.9/Source/WebCore/WebCore.pri#L254
[6]: https://github.com/google/leveldb/commit/739c25100e46576cdcdfff2d6f43f9f7008103c7

Signed-off-by: Ga?l PORTAY <gael.portay@collabora.com>
---
 package/qt5/qt5webkit/Config.in    | 3 +++
 package/qt5/qt5webkit/qt5webkit.mk | 4 ++--
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/package/qt5/qt5webkit/Config.in b/package/qt5/qt5webkit/Config.in
index 72c641b9f7..e8008f7bab 100644
--- a/package/qt5/qt5webkit/Config.in
+++ b/package/qt5/qt5webkit/Config.in
@@ -3,10 +3,13 @@ config BR2_PACKAGE_QT5WEBKIT
 	depends on !BR2_STATIC_LIBS
 	depends on BR2_PACKAGE_QT5_JSCORE_AVAILABLE
 	depends on BR2_HOST_GCC_AT_LEAST_4_8 # icu
+	depends on BR2_INSTALL_LIBSTDCPP # leveldb
 	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # icu
+	depends on BR2_TOOLCHAIN_HAS_THREADS # leveldb
 	depends on !BR2_BINFMT_FLAT # icu
 	# assumes a FPU is available on MIPS
 	depends on !BR2_MIPS_SOFT_FLOAT
+	select BR2_PACKAGE_LEVELDB
 	select BR2_PACKAGE_QT5BASE
 	select BR2_PACKAGE_QT5BASE_ICU
 	select BR2_PACKAGE_QT5BASE_GUI
diff --git a/package/qt5/qt5webkit/qt5webkit.mk b/package/qt5/qt5webkit/qt5webkit.mk
index e8d368fb01..a0a0998d12 100644
--- a/package/qt5/qt5webkit/qt5webkit.mk
+++ b/package/qt5/qt5webkit/qt5webkit.mk
@@ -16,7 +16,7 @@ endif
 QT5WEBKIT_SOURCE = qtwebkit-opensource-src-$(QT5WEBKIT_VERSION).tar.xz
 QT5WEBKIT_DEPENDENCIES = \
 	host-bison host-flex host-gperf host-python host-ruby \
-	qt5base sqlite
+	leveldb qt5base sqlite
 QT5WEBKIT_INSTALL_STAGING = YES
 
 QT5WEBKIT_LICENSE_FILES = Source/WebCore/LICENSE-LGPL-2 Source/WebCore/LICENSE-LGPL-2.1
@@ -45,7 +45,7 @@ endef
 QT5WEBKIT_PRE_CONFIGURE_HOOKS += QT5WEBKIT_PYTHON2_SYMLINK
 
 define QT5WEBKIT_CONFIGURE_CMDS
-	(cd $(@D); $(TARGET_MAKE_ENV) $(QT5WEBKIT_ENV) $(HOST_DIR)/bin/qmake)
+	(cd $(@D); $(TARGET_MAKE_ENV) $(QT5WEBKIT_ENV) $(HOST_DIR)/bin/qmake WEBKIT_CONFIG+=use_system_leveldb)
 endef
 
 define QT5WEBKIT_BUILD_CMDS
-- 
2.20.1

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v3 0/3] qt5webkit: fix build issue using system leveldb
  2019-01-02 20:56 [Buildroot] [PATCH v3 0/3] qt5webkit: fix build issue using system leveldb Gaël PORTAY
                   ` (2 preceding siblings ...)
  2019-01-02 20:56 ` [Buildroot] [PATCH v3 3/3] qt5webkit: select leveldb package and memenv Gaël PORTAY
@ 2019-03-07 22:27 ` Arnout Vandecappelle
  2019-03-25 12:20   ` Peter Korsgaard
  3 siblings, 1 reply; 6+ messages in thread
From: Arnout Vandecappelle @ 2019-03-07 22:27 UTC (permalink / raw)
  To: buildroot

 Hi Gael,

On 02/01/2019 21:56, Ga?l PORTAY wrote:
> This patchset fixes the build issue reported by autobuilder [0].
> 
>         /home/naourr/work/instance-2/output/build/qt5webkit-5.9.1/Source/WebCore//.obj/platform/leveldb/LevelDBDatabase.o: In function `WebCore::LevelDBDatabase::openInMemory(WebCore::LevelDBComparator const*)':
> LevelDBDatabase.cpp.text._ZN7WebCore15LevelDBDatabase12openInMemoryEPKNS_17LevelDBComparatorE+0x34): undefined reference to `leveldb::NewMemEnv(leveldb::Env*)'
>         collect2: error: ld returned 1 exit status
>         make[3]: *** [Makefile.api:97: ../lib/libQt5WebKit.so.5.9.1] Error 1
> 
> The issue happens when the package leveldb is enabled.

 Series applied to master, thanks. I renamed the patch because two other patches
to leveldb have been added in the meantime.

 Regards,
 Arnout

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v3 0/3] qt5webkit: fix build issue using system leveldb
  2019-03-07 22:27 ` [Buildroot] [PATCH v3 0/3] qt5webkit: fix build issue using system leveldb Arnout Vandecappelle
@ 2019-03-25 12:20   ` Peter Korsgaard
  0 siblings, 0 replies; 6+ messages in thread
From: Peter Korsgaard @ 2019-03-25 12:20 UTC (permalink / raw)
  To: buildroot

>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:

 >  Hi Gael,
 > On 02/01/2019 21:56, Ga?l PORTAY wrote:
 >> This patchset fixes the build issue reported by autobuilder [0].
 >> 
 >> /home/naourr/work/instance-2/output/build/qt5webkit-5.9.1/Source/WebCore//.obj/platform/leveldb/LevelDBDatabase.o:
 >> In function
 >> `WebCore::LevelDBDatabase::openInMemory(WebCore::LevelDBComparator
 >> const*)':
 >> LevelDBDatabase.cpp.text._ZN7WebCore15LevelDBDatabase12openInMemoryEPKNS_17LevelDBComparatorE+0x34):
 >> undefined reference to `leveldb::NewMemEnv(leveldb::Env*)'
 >> collect2: error: ld returned 1 exit status
 >> make[3]: *** [Makefile.api:97: ../lib/libQt5WebKit.so.5.9.1] Error 1
 >> 
 >> The issue happens when the package leveldb is enabled.

 >  Series applied to master, thanks. I renamed the patch because two other patches
 > to leveldb have been added in the meantime.

Series committed to 2018.02.x, 2018.11.x and 2019.02.x, thanks.

-- 
Bye, Peter Korsgaard

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2019-03-25 12:20 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-01-02 20:56 [Buildroot] [PATCH v3 0/3] qt5webkit: fix build issue using system leveldb Gaël PORTAY
2019-01-02 20:56 ` [Buildroot] [PATCH v3 1/3] leveldb: install memenv static library and header Gaël PORTAY
2019-01-02 20:56 ` [Buildroot] [PATCH v3 2/3] leveldb: generate pic for static libraries Gaël PORTAY
2019-01-02 20:56 ` [Buildroot] [PATCH v3 3/3] qt5webkit: select leveldb package and memenv Gaël PORTAY
2019-03-07 22:27 ` [Buildroot] [PATCH v3 0/3] qt5webkit: fix build issue using system leveldb Arnout Vandecappelle
2019-03-25 12:20   ` Peter Korsgaard

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.