Re: [meta-oe][PATCHv3 2/2] protobuf-c: add patch for protobuf 3 compatibility

["pascal.bach@siemens.com" <pascal.bach@siemens.com>]

Hi Martin,

I haven’t observed this issue so far. But I haven’t rebuilt it 100 times.
I might find some time to have a look at the cmake or meson support and if it works submit a patch to switch the recipe to it.

Stay tuned.

Pascal

From: Martin Jansa <martin.jansa@gmail.com>
Sent: Mittwoch, 12. Juni 2019 20:23
To: Bach, Pascal (SI BP R&D ZG FW) <pascal.bach@siemens.com>
Cc: openembedded-devel <openembedded-devel@lists.openembedded.org>
Subject: Re: [oe] [meta-oe][PATCHv3 2/2] protobuf-c: add patch for protobuf 3 compatibility

It's not caused by this change, but if you're using protobuf-c aren't you seeing random build failures due to race-condition between generated test-full.pb.h and cxx-generate-packed-data.cc ?

I've rebuilt protobuf-c-native 100 times in 32 core system and got 4 failed builds, each with slightly different errors (only first 5 shown for each):

BUILD-40/work/x86_64-linux/protobuf-c-native/1.3.1+gitAUTOINC+269771b4b4-r0/temp/log.do_compile
../git/t/generated-code2/cxx-generate-packed-data.cc:28:17: error: ‘foo’ is not a namespace-name
../git/t/generated-code2/cxx-generate-packed-data.cc:28:20: error: expected namespace-name before ‘;’ token
../git/t/generated-code2/cxx-generate-packed-data.cc:31:36: error: ‘TestEnumSmall’ does not name a type
../git/t/generated-code2/cxx-generate-packed-data.cc:31:36: error: ‘TestEnumSmall’ does not name a type
../git/t/generated-code2/cxx-generate-packed-data.cc:31:36: error: ‘TestEnumSmall’ does not name a type

BUILD-76/work/x86_64-linux/protobuf-c-native/1.3.1+gitAUTOINC+269771b4b4-r0/temp/log.do_compile
./t/test-full.pb.h:4:0: error: unterminated #ifndef
/usr/include/inttypes.h:266:1: error: expected initializer before ‘extern’
../git/t/generated-code2/cxx-generate-packed-data.cc:1169:1: error: expected ‘}’ at end of input
BUILD-85/work/x86_64-linux/protobuf-c-native/1.3.1+gitAUTOINC+269771b4b4-r0/temp/log.do_compile
./t/test-full.pb.h:4:0: error: unterminated #ifndef
/usr/include/inttypes.h:266:1: error: expected ‘)’ before ‘extern’
../git/t/generated-code2/cxx-generate-packed-data.cc:42:1: error: a function-definition is not allowed here before ‘{’ token
../git/t/generated-code2/cxx-generate-packed-data.cc:64:1: error: a function-definition is not allowed here before ‘{’ token
../git/t/generated-code2/cxx-generate-packed-data.cc:70:1: error: a function-definition is not allowed here before ‘{’ token
BUILD-86/work/x86_64-linux/protobuf-c-native/1.3.1+gitAUTOINC+269771b4b4-r0/temp/log.do_compile
./t/test-full.pb.h:4:0: error: unterminated #ifndef
./t/test-full.pb.h:21888:15: error: ‘s’ in namespace ‘std’ does not name a type
../git/t/generated-code2/cxx-generate-packed-data.cc:1169:1: error: expected ‘}’ at end of input

It was supposed to be fixed with;
https://github.com/protobuf-c/protobuf-c/commit/b0bb56303366e2c072ee38eb5f1f11251b07239c<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fprotobuf-c%2Fprotobuf-c%2Fcommit%2Fb0bb56303366e2c072ee38eb5f1f11251b07239c&data=02%7C01%7Cpascal.bach%40siemens.com%7C6ea32c7da6344dc9a8b508d6ef631478%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C636959606157388338&sdata=aqCYxWpGVloQh7mki0O3p0Z0M%2BWXFuocYCi5rQsqxls%3D&reserved=0>
but the fix either doesn't work at all or is actually causing this (I haven't seen these failures with older protobuf-c-native).

Maybe we should switch from autotools to CMake when building protobuf-c.

There is also PR with meson support:
https://github.com/protobuf-c/protobuf-c/pull/340<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fprotobuf-c%2Fprotobuf-c%2Fpull%2F340&data=02%7C01%7Cpascal.bach%40siemens.com%7C6ea32c7da6344dc9a8b508d6ef631478%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C636959606157388338&sdata=8myitFPOaBfceG%2FdI2x9D0rHOtaswHoFkFyVixoQwKM%3D&reserved=0>

If you're not really interested in protobuf-c-native then you can ignore this, it's not caused by your change - I wasn't trying to hijack this thread and block your change.

Cheers,

On Tue, Jun 11, 2019 at 5:39 PM Pascal Bach <pascal.bach@siemens.com<mailto:pascal.bach@siemens.com>> wrote:
There is no release yet of protobuf-c that is compatible with protobuf 3.
Master is already patched so this is just a straight backport of this patch.

Signed-off-by: Pascal Bach <pascal.bach@siemens.com<mailto:pascal.bach@siemens.com>>
---
 .../protobuf-c/protobuf3-compatibility.patch       | 55 ++++++++++++++++++++++
 .../recipes-devtools/protobuf/protobuf-c_1.3.1.bb<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fprotobuf-c_1.3.1.bb&data=02%7C01%7Cpascal.bach%40siemens.com%7C6ea32c7da6344dc9a8b508d6ef631478%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C636959606157398333&sdata=1Nlxk8IRzE7hhJxxXcuqsuI8dqFGVRiF50tunWvE6r4%3D&reserved=0>  |  4 +-
 2 files changed, 58 insertions(+), 1 deletion(-)
 create mode 100644 meta-oe/recipes-devtools/protobuf/protobuf-c/protobuf3-compatibility.patch

diff --git a/meta-oe/recipes-devtools/protobuf/protobuf-c/protobuf3-compatibility.patch b/meta-oe/recipes-devtools/protobuf/protobuf-c/protobuf3-compatibility.patch
new file mode 100644
index 000000000..35914952d
--- /dev/null
+++ b/meta-oe/recipes-devtools/protobuf/protobuf-c/protobuf3-compatibility.patch
@@ -0,0 +1,55 @@
+From 7456d1621223d425b8a3fd74e435a79c046169fb Mon Sep 17 00:00:00 2001
+From: Robert Edmonds <edmonds@users.noreply.github.com<mailto:edmonds@users.noreply.github.com>>
+Date: Wed, 10 Apr 2019 20:47:48 -0400
+Subject: [PATCH] t/generated-code2/cxx-generate-packed-data.cc: Fix build
+ failure on newer protobuf
+
+ Upstream-Status: Backport [https://github.com/protobuf-c/protobuf-c/pull/369<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fprotobuf-c%2Fprotobuf-c%2Fpull%2F369&data=02%7C01%7Cpascal.bach%40siemens.com%7C6ea32c7da6344dc9a8b508d6ef631478%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C636959606157398333&sdata=3IiMfZQixmdOSe8meS8qBJKTfpmNzzk%2BnaU3z61q44A%3D&reserved=0>]
+
+google::protobuf::Message::Reflection has been removed in newer versions
+of protobuf. The replacement is google::protobuf::Reflection.
+
+protobuf in commit 779f61c6a3ce02a119e28e802f229e61b69b9046 ("Integrate
+recent changes from google3.", from August 2008) changed the following
+in message.h:
+
+    @@ -336,7 +337,8 @@ class LIBPROTOBUF_EXPORT Message {
+
+       // Introspection ---------------------------------------------------
+
+    -  class Reflection;  // Defined below.
+    +  // Typedef for backwards-compatibility.
+    +  typedef google::protobuf::Reflection Reflection;
+
+The "typedef for backwards-compatibility" apparently lasted ten years
+until protobuf commit 6bbe197e9c1b6fc38cbdc45e3bf83fa7ced792a3
+("Down-integrate from google3.", from August 2018) which finally removed
+the typedef:
+
+    @@ -327,8 +344,6 @@ class LIBPROTOBUF_EXPORT Message : public MessageLite {
+
+       // Introspection ---------------------------------------------------
+
+    -  // Typedef for backwards-compatibility.
+    -  typedef google::protobuf::Reflection Reflection;
+
+This commit updates the only use of this typedef (in the test suite) to
+directly refer to the replacement, google::protobuf::Reflection. This
+fixes the build failure in the test suite.
+---
+ t/generated-code2/cxx-generate-packed-data.cc | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/t/generated-code2/cxx-generate-packed-data.cc b/t/generated-code2/cxx-generate-packed-data.cc
+index 4fd3e25..0865d2e 100644
+--- a/t/generated-code2/cxx-generate-packed-data.cc
++++ b/t/generated-code2/cxx-generate-packed-data.cc
+@@ -998,7 +998,7 @@ static void dump_test_packed_repeated_enum (void)
+ static void dump_test_unknown_fields (void)
+ {
+   EmptyMess mess;
+-  const google::protobuf::Message::Reflection *reflection = mess.GetReflection();
++  const google::protobuf::Reflection *reflection = mess.GetReflection();
+   google::protobuf::UnknownFieldSet *fs = reflection->MutableUnknownFields(&mess);
+
+ #if GOOGLE_PROTOBUF_VERSION >= 2001000
diff --git a/meta-oe/recipes-devtools/protobuf/protobuf-c_1.3.1.bb<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fprotobuf-c_1.3.1.bb&data=02%7C01%7Cpascal.bach%40siemens.com%7C6ea32c7da6344dc9a8b508d6ef631478%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C636959606157398333&sdata=1Nlxk8IRzE7hhJxxXcuqsuI8dqFGVRiF50tunWvE6r4%3D&reserved=0> b/meta-oe/recipes-devtools/protobuf/protobuf-c_1.3.1.bb<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fprotobuf-c_1.3.1.bb&data=02%7C01%7Cpascal.bach%40siemens.com%7C6ea32c7da6344dc9a8b508d6ef631478%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C636959606157408326&sdata=laq3qyJKQ2UECciwgJQrulGbP2dez85fM8IOX3iX6kk%3D&reserved=0>
index 7ef030092..1d1782306 100644
--- a/meta-oe/recipes-devtools/protobuf/protobuf-c_1.3.1.bb<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fprotobuf-c_1.3.1.bb&data=02%7C01%7Cpascal.bach%40siemens.com%7C6ea32c7da6344dc9a8b508d6ef631478%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C636959606157408326&sdata=laq3qyJKQ2UECciwgJQrulGbP2dez85fM8IOX3iX6kk%3D&reserved=0>
+++ b/meta-oe/recipes-devtools/protobuf/protobuf-c_1.3.1.bb<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fprotobuf-c_1.3.1.bb&data=02%7C01%7Cpascal.bach%40siemens.com%7C6ea32c7da6344dc9a8b508d6ef631478%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C636959606157418320&sdata=ak1rCuM6bPK7MaGvbkcPBzS1PkZJRRUkqChchERhu8k%3D&reserved=0>
@@ -15,7 +15,9 @@ DEPENDS = "protobuf-native protobuf"
 PV .= "+git${SRCPV}"
 SRCREV = "269771b4b45d3aba04e59569f53600003db8d9ff"

-SRC_URI = "git://github.com/protobuf-c/protobuf-c.git<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2Fprotobuf-c%2Fprotobuf-c.git&data=02%7C01%7Cpascal.bach%40siemens.com%7C6ea32c7da6344dc9a8b508d6ef631478%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C636959606157418320&sdata=CyI4lVVbmKeIDw4PvtAlRbOKe8AAyRBOy79LsIChVZk%3D&reserved=0>"
+SRC_URI = "git://github.com/protobuf-c/protobuf-c.git<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2Fprotobuf-c%2Fprotobuf-c.git&data=02%7C01%7Cpascal.bach%40siemens.com%7C6ea32c7da6344dc9a8b508d6ef631478%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C636959606157428311&sdata=%2Bh8A2W6eQhRsZu4rDnaDZXAJ%2F%2BfTHs%2BSxLPYxHI5prY%3D&reserved=0> \
+           file://protobuf3-compatibility.patch \
+          "

 S = "${WORKDIR}/git"

--
2.11.0

--
_______________________________________________
Openembedded-devel mailing list
Openembedded-devel@lists.openembedded.org<mailto:Openembedded-devel@lists.openembedded.org>
http://lists.openembedded.org/mailman/listinfo/openembedded-devel<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.openembedded.org%2Fmailman%2Flistinfo%2Fopenembedded-devel&data=02%7C01%7Cpascal.bach%40siemens.com%7C6ea32c7da6344dc9a8b508d6ef631478%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C636959606157428311&sdata=aolVOgT563U4Mo17Y%2FFH88K8DLSgNlZ%2BNtECnwqeFLc%3D&reserved=0>