* Re: [meta-virtualization] [PATCH] ceph: Upgrade to 16.2.0
2021-04-22 19:34 ` [meta-virtualization] [PATCH] ceph: Upgrade to 16.2.0 Saul Wold
@ 2021-04-23 16:51 ` Bruce Ashfield
0 siblings, 0 replies; 2+ messages in thread
From: Bruce Ashfield @ 2021-04-23 16:51 UTC (permalink / raw)
To: Saul Wold; +Cc: meta-virtualization
On Thu, Apr 22, 2021 at 3:34 PM Saul Wold <Saul.Wold@windriver.com> wrote:
>
> Bruce
>
> I am aware there is a 16.2.1 that was just published on the 19th, I
> wanted to get visibility to the base upgrade which I just completed
> while I work on the dot dot update.
>
Thanks Saul!
I'll do some test builds here, but will hold on the merge until you
have that next update ready. unless there's a reason to get this
version in first.
Cheers,
Bruce
> Ceph-17 is the next major release, but has not be released yet.
>
> Thanks
>
> Sau!
>
>
> On 4/22/21 12:32 PM, Saul Wold wrote:
> > This updates the Ceph version to 16.2.0 which required a couple
> > changes to base code via a rebased patch which also disabled the
> > sanity check due to it not working correctly, but the actual
> > compile worked. Also added a new patch to add soversion information
> > for the new libcephsqlite library so it would be packaged correctly
> >
> > Signed-off-by: Saul Wold <saul.wold@windriver.com>
> > ---
> > ...h-fix-build-errors-for-cross-compile.patch | 119 ++++++++++++++----
> > ...ite-Add-library-soversion-properties.patch | 31 +++++
> > recipes-extended/ceph/ceph_16.2.0.bb | 22 ++--
> > 3 files changed, 138 insertions(+), 34 deletions(-)
> > create mode 100644 recipes-extended/ceph/ceph/0001-libcephsqlite-Add-library-soversion-properties.patch
> >
> > diff --git a/recipes-extended/ceph/ceph/0001-ceph-fix-build-errors-for-cross-compile.patch b/recipes-extended/ceph/ceph/0001-ceph-fix-build-errors-for-cross-compile.patch
> > index 7764a2b..00c7e5b 100644
> > --- a/recipes-extended/ceph/ceph/0001-ceph-fix-build-errors-for-cross-compile.patch
> > +++ b/recipes-extended/ceph/ceph/0001-ceph-fix-build-errors-for-cross-compile.patch
> > @@ -1,10 +1,15 @@
> > -From 04b53b669dbd75c9a90bf7f8ae2b47ff96176239 Mon Sep 17 00:00:00 2001
> > +From 2a3ccc8079040a7e783b8a9091618878e7cf0719 Mon Sep 17 00:00:00 2001
> > From: Dengke Du <dengke.du@windriver.com>
> > Date: Mon, 11 Mar 2019 09:14:09 +0800
> > Subject: [PATCH] ceph: fix build errors for cross compile
> >
> > 1. set the cross compile sysroot to find the rocksdb library
> > 2. correct the install path for library in Distutils.cmake
> > +3. remove / disable the sanity_check(), for some reasong the
> > + sanity check gets the wrong settings and will not link
> > + correctly, but the actual build works correctly, this needs
> > + more investigation later, earlier versions of the patch also
> > + removed it but no comment on why. (sgw)
> >
> > Upstream-Status: Inappropriate [oe specific]
> >
> > @@ -14,16 +19,17 @@ Adjust context for v14.2.3
> >
> > Signed-off-by: He Zhe <zhe.he@windriver.com>
> > Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
> > +Signed-off-by: Saul Wold <saul.wold@windriver.com>
> >
> > ---
> > cmake/modules/Distutils.cmake | 25 +++++--------------------
> > cmake/modules/FindRocksDB.cmake | 4 ++--
> > src/compressor/zstd/CMakeLists.txt | 2 +-
> > - src/pybind/cephfs/setup.py | 1 -
> > - src/pybind/rados/setup.py | 1 -
> > - src/pybind/rbd/setup.py | 1 -
> > - src/pybind/rgw/setup.py | 1 -
> > - 7 files changed, 8 insertions(+), 27 deletions(-)
> > + src/pybind/cephfs/setup.py | 18 +++++-------------
> > + src/pybind/rados/setup.py | 11 ++---------
> > + src/pybind/rbd/setup.py | 18 +++++-------------
> > + src/pybind/rgw/setup.py | 18 +++++-------------
> > + 7 files changed, 25 insertions(+), 71 deletions(-)
> >
> > diff --git a/cmake/modules/Distutils.cmake b/cmake/modules/Distutils.cmake
> > index ff01aaf81..18656c5a5 100644
> > @@ -105,50 +111,111 @@ index b53b0d944..4955711a9 100644
> > -G${CMAKE_GENERATOR}
> > BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR}/libzstd
> > diff --git a/src/pybind/cephfs/setup.py b/src/pybind/cephfs/setup.py
> > -index c3d40df05..87cdeb473 100755
> > +index c3d40df05..24c26c3ed 100755
> > --- a/src/pybind/cephfs/setup.py
> > +++ b/src/pybind/cephfs/setup.py
> > -@@ -137,7 +137,6 @@ def check_sanity():
> > +@@ -137,19 +137,11 @@ def check_sanity():
> > finally:
> > shutil.rmtree(tmp_dir)
> >
> > -
> > - if 'BUILD_DOC' in os.environ or 'READTHEDOCS' in os.environ:
> > - ext_args = {}
> > - cython_constants = dict(BUILD_DOC=True)
> > +-if 'BUILD_DOC' in os.environ or 'READTHEDOCS' in os.environ:
> > +- ext_args = {}
> > +- cython_constants = dict(BUILD_DOC=True)
> > +- cythonize_args = dict(compile_time_env=cython_constants)
> > +-elif check_sanity():
> > +- ext_args = get_python_flags(['cephfs'])
> > +- cython_constants = dict(BUILD_DOC=False)
> > +- include_path = [os.path.join(os.path.dirname(__file__), "..", "rados")]
> > +- cythonize_args = dict(compile_time_env=cython_constants,
> > +- include_path=include_path)
> > +-else:
> > +- sys.exit(1)
> > ++ext_args = get_python_flags(['cephfs'])
> > ++cython_constants = dict(BUILD_DOC=False)
> > ++include_path = [os.path.join(os.path.dirname(__file__), "..", "rados")]
> > ++cythonize_args = dict(compile_time_env=cython_constants,
> > ++ include_path=include_path)
> > +
> > + cmdclass = {}
> > + try:
> > diff --git a/src/pybind/rados/setup.py b/src/pybind/rados/setup.py
> > -index 0d1047825..feafb605d 100755
> > +index 0d1047825..cff85a063 100755
> > --- a/src/pybind/rados/setup.py
> > +++ b/src/pybind/rados/setup.py
> > -@@ -131,7 +131,6 @@ def check_sanity():
> > +@@ -131,15 +131,8 @@ def check_sanity():
> > finally:
> > shutil.rmtree(tmp_dir)
> >
> > -
> > - if 'BUILD_DOC' in os.environ or 'READTHEDOCS' in os.environ:
> > - ext_args = {}
> > - cython_constants = dict(BUILD_DOC=True)
> > +-if 'BUILD_DOC' in os.environ or 'READTHEDOCS' in os.environ:
> > +- ext_args = {}
> > +- cython_constants = dict(BUILD_DOC=True)
> > +-elif check_sanity():
> > +- ext_args = get_python_flags(['rados'])
> > +- cython_constants = dict(BUILD_DOC=False)
> > +-else:
> > +- sys.exit(1)
> > ++ext_args = get_python_flags(['rados'])
> > ++cython_constants = dict(BUILD_DOC=False)
> > +
> > + cmdclass = {}
> > + try:
> > diff --git a/src/pybind/rbd/setup.py b/src/pybind/rbd/setup.py
> > -index 73ea4555f..b9c6a78d2 100755
> > +index 73ea4555f..e7f57ddb5 100755
> > --- a/src/pybind/rbd/setup.py
> > +++ b/src/pybind/rbd/setup.py
> > -@@ -135,7 +135,6 @@ def check_sanity():
> > +@@ -135,19 +135,11 @@ def check_sanity():
> > finally:
> > shutil.rmtree(tmp_dir)
> >
> > -
> > - if 'BUILD_DOC' in os.environ or 'READTHEDOCS' in os.environ:
> > - ext_args = {}
> > - cython_constants = dict(BUILD_DOC=True)
> > +-if 'BUILD_DOC' in os.environ or 'READTHEDOCS' in os.environ:
> > +- ext_args = {}
> > +- cython_constants = dict(BUILD_DOC=True)
> > +- cythonize_args = dict(compile_time_env=cython_constants)
> > +-elif check_sanity():
> > +- ext_args = get_python_flags(['rados', 'rbd'])
> > +- cython_constants = dict(BUILD_DOC=False)
> > +- include_path = [os.path.join(os.path.dirname(__file__), "..", "rados")]
> > +- cythonize_args = dict(compile_time_env=cython_constants,
> > +- include_path=include_path)
> > +-else:
> > +- sys.exit(1)
> > ++ext_args = get_python_flags(['rados', 'rbd'])
> > ++cython_constants = dict(BUILD_DOC=False)
> > ++include_path = [os.path.join(os.path.dirname(__file__), "..", "rados")]
> > ++cythonize_args = dict(compile_time_env=cython_constants,
> > ++ include_path=include_path)
> > +
> > + cmdclass = {}
> > + try:
> > diff --git a/src/pybind/rgw/setup.py b/src/pybind/rgw/setup.py
> > -index 663604e8f..695038c0e 100755
> > +index 663604e8f..489c9ba1f 100755
> > --- a/src/pybind/rgw/setup.py
> > +++ b/src/pybind/rgw/setup.py
> > -@@ -135,7 +135,6 @@ def check_sanity():
> > +@@ -135,19 +135,11 @@ def check_sanity():
> > finally:
> > shutil.rmtree(tmp_dir)
> >
> > -
> > - if 'BUILD_DOC' in os.environ or 'READTHEDOCS' in os.environ:
> > - ext_args = {}
> > - cython_constants = dict(BUILD_DOC=True)
> > +-if 'BUILD_DOC' in os.environ or 'READTHEDOCS' in os.environ:
> > +- ext_args = {}
> > +- cython_constants = dict(BUILD_DOC=True)
> > +- cythonize_args = dict(compile_time_env=cython_constants)
> > +-elif check_sanity():
> > +- ext_args = get_python_flags(['rados', 'rgw'])
> > +- cython_constants = dict(BUILD_DOC=False)
> > +- include_path = [os.path.join(os.path.dirname(__file__), "..", "rados")]
> > +- cythonize_args = dict(compile_time_env=cython_constants,
> > +- include_path=include_path)
> > +-else:
> > +- sys.exit(1)
> > ++ext_args = get_python_flags(['rados', 'rgw'])
> > ++cython_constants = dict(BUILD_DOC=False)
> > ++include_path = [os.path.join(os.path.dirname(__file__), "..", "rados")]
> > ++cythonize_args = dict(compile_time_env=cython_constants,
> > ++ include_path=include_path)
> > +
> > + cmdclass = {}
> > + try:
> > diff --git a/recipes-extended/ceph/ceph/0001-libcephsqlite-Add-library-soversion-properties.patch b/recipes-extended/ceph/ceph/0001-libcephsqlite-Add-library-soversion-properties.patch
> > new file mode 100644
> > index 0000000..2c9cd11
> > --- /dev/null
> > +++ b/recipes-extended/ceph/ceph/0001-libcephsqlite-Add-library-soversion-properties.patch
> > @@ -0,0 +1,31 @@
> > +From ce36ce7d1c0f954d1532a759f4d7b09d87287dac Mon Sep 17 00:00:00 2001
> > +From: Saul Wold <saul.wold@windriver.com>
> > +Date: Thu, 22 Apr 2021 08:05:57 -0700
> > +Subject: [PATCH] libcephsqlite: Add library soversion properties
> > +
> > +This additional properties are needed to ensure the library
> > +is installed with an soversion so that when the library is
> > +packaged it gets packaged corectly in the ceph package and
> > +ceph-dev package as the link.
> > +
> > +Upstream-Status: pending
> > +Signed-off-by: Saul Wold <saul.wold@windriver.com>
> > +---
> > + src/CMakeLists.txt | 4 ++++
> > + 1 file changed, 4 insertions(+)
> > +
> > +diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
> > +index 37db48f86..8dc7bdb56 100644
> > +--- a/src/CMakeLists.txt
> > ++++ b/src/CMakeLists.txt
> > +@@ -804,6 +804,10 @@ if(WITH_LIBCEPHSQLITE)
> > + set(cephsqlite_srcs libcephsqlite.cc SimpleRADOSStriper.cc)
> > + add_library(cephsqlite ${CEPH_SHARED} ${cephsqlite_srcs})
> > + target_link_libraries(cephsqlite PRIVATE cls_lock_client librados ceph-common SQLite3::SQLite3 ${EXTRALIBS})
> > ++ set_target_properties(cephsqlite PROPERTIES
> > ++ OUTPUT_NAME cephsqlite
> > ++ VERSION 1.0.0
> > ++ SOVERSION 1)
> > + install(TARGETS cephsqlite DESTINATION ${CMAKE_INSTALL_LIBDIR})
> > + endif(WITH_LIBCEPHSQLITE)
> > +
> > diff --git a/recipes-extended/ceph/ceph_16.2.0.bb b/recipes-extended/ceph/ceph_16.2.0.bb
> > index 806a03f..d55e602 100644
> > --- a/recipes-extended/ceph/ceph_16.2.0.bb
> > +++ b/recipes-extended/ceph/ceph_16.2.0.bb
> > @@ -58,7 +58,9 @@ LIC_FILES_CHKSUM = "file://COPYING-LGPL2.1;md5=fbc093901857fcd118f065f900982c24
> > file://COPYING-GPL2;md5=b234ee4d69f5fce4486a80fdaf4a4263 \
> > file://COPYING;md5=d140fb1fedb53047f0d0830883e7af9f \
> > "
> > -inherit cmake python3native python3-dir systemd
> > +#inherit cmake setuptools3 systemd
> > +inherit cmake distutils3-base systemd
> > +
> > # Disable python pybind support for ceph temporary, when corss compiling pybind,
> > # pybind mix cmake and python setup environment, would case a lot of errors.
> >
> > @@ -66,14 +68,9 @@ SRC_URI = "http://download.ceph.com/tarballs/ceph-${PV}.tar.gz \
> > file://0001-ceph-fix-build-errors-for-cross-compile.patch \
> > file://0001-fix-host-library-paths-were-used.patch \
> > file://ceph.conf \
> > + file://0001-libcephsqlite-Add-library-soversion-properties.patch \
> > "
> >
> > -#SRC_URI[md5sum] = "5a949b91199efe130bfe2e57f200eacd"
> > -#SRC_URI[sha256sum] = "ea7fa2be68b3570c5c3aa02774dca27de20b6f20a8498f65e5c4ef375a209ace"
> > -#SRC_URI[sha1sum] = "0503649d7c7d9268cb61b03bbe839106988f87d7"
> > -#SRC_URI[sha384sum] = "57208c126ea8e640c3979073fd64ce58606f17b4b7ed2e005b0792bdfd51b13391af325658d0f68f961f94175d93e891"
> > -#SRC_URI[sha512sum] = "1f55b0a13bf06df782831d0c9d5f8617c22442ab97fb186e6ccb08183e02bf1756caf8633fa6b1cf156c3fb7b1bdff90a60a1249adfad0b9450036e2329bf8ff"
> > -SRC_URI[sha256sum] = "dfeee542536607220dc2ca49f47c41127c8562dea234a886625e0f7ee1524904"
> > SRC_URI[sha1sum] = "f8cc85e4cc519d7e469a0c3c8327bd1675f0b287"
> > SRC_URI[sha384sum] = "29a75a733e5ae8039a5ad603558a9707e36b8c337419039ee97351cdbad57875fa57a365fcde29124cee72734608e245"
> > SRC_URI[sha512sum] = "aeb9a91c33221c64ea24603dc88cab346bf0abdf0d41ff85c2e1cf134130737ec1fab1246be0a2fa3af8a655ae1dabb69688855f229f54438e60cb8098175b8e"
> > @@ -149,6 +146,13 @@ do_install_append () {
> > mv ${D}${libexecdir}/ceph/ceph_common.sh ${D}${libdir}/ceph
> > # WITH_FUSE is set to OFF, remove ceph-fuse related units
> > rm ${D}${systemd_unitdir}/system/ceph-fuse.target ${D}${systemd_unitdir}/system/ceph-fuse@.service
> > +
> > + for i in ${D}${bindir}/* ${D}${sbindir}/*; do
> > + if [ -f "$i" ]; then
> > + sed -i -e s:${PYTHON}:${USRBINPATH}/env\ ${DISTUTILS_PYTHON}:g $i
> > + sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
> > + fi
> > + done
> > }
> >
> > do_install_append_class-target () {
> > @@ -173,7 +177,8 @@ FILES_${PN} += "\
> > ${libdir}/rados-classes/*.so.* \
> > ${libdir}/ceph/compressor/*.so \
> > ${libdir}/rados-classes/*.so \
> > - ${libdir}/ceph/*.so \
> > + ${libdir}/ceph/*.so \
> > + ${libdir}/libcephsqlite.so \
> > "
> >
> > FILES_${PN} += " \
> > @@ -190,6 +195,7 @@ RDEPENDS_${PN} += "\
> > python3-modules \
> > python3-prettytable \
> > ${PN}-python \
> > + gawk \
> > "
> > COMPATIBLE_HOST = "(x86_64).*"
> > PACKAGES += " \
> >
> >
> >
> >
> >
>
> --
> Sau!
>
>
>
--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
^ permalink raw reply [flat|nested] 2+ messages in thread