All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Marek Belisko" <marek.belisko@gmail.com>
To: Hongxu Jia <hongxu.jia@windriver.com>
Cc: Randy MacLeod <randy.macleod@windriver.com>,
	philip@balister.org,  akuster808 <akuster808@gmail.com>,
	yocto <yocto@yoctoproject.org>
Subject: Re: [meta-tensorflow][PATCH 8/25] tensorflow-estimator: 1.13 -> 2.4
Date: Mon, 4 Jan 2021 11:30:50 +0100	[thread overview]
Message-ID: <CAAfyv37zRgtJ_n2RZmWoHPOado1=LYw9cHVEVUZT1n497XB8yw@mail.gmail.com> (raw)
In-Reply-To: <20201216130822.28476-9-hongxu.jia@windriver.com>

Hi Hongxu,

On Wed, Dec 16, 2020 at 2:09 PM Hongxu Jia <hongxu.jia@windriver.com> wrote:
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
>  .../0001-customize-for-yocto.patch            | 28 +++++++++++++++++++
>  .../tensorflow/tensorflow-estimator_1.13.bb   | 12 ++++++--
>  2 files changed, 37 insertions(+), 3 deletions(-)
>  create mode 100644 recipes-framework/tensorflow/tensorflow-estimator/0001-customize-for-yocto.patch
>
> diff --git a/recipes-framework/tensorflow/tensorflow-estimator/0001-customize-for-yocto.patch b/recipes-framework/tensorflow/tensorflow-estimator/0001-customize-for-yocto.patch
> new file mode 100644
> index 0000000..e9b66d5
> --- /dev/null
> +++ b/recipes-framework/tensorflow/tensorflow-estimator/0001-customize-for-yocto.patch
> @@ -0,0 +1,28 @@
> +From a1bcf09a43fc44ad5e04c441ee45cc23d16cf7d2 Mon Sep 17 00:00:00 2001
> +From: Hongxu Jia <hongxu.jia@windriver.com>
> +Date: Wed, 9 Dec 2020 17:59:01 +0800
> +Subject: [PATCH] customize for yocto
> +
> +Upstream-Status: Inappropriate [oe specific]
> +
> +Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> +---
> + tensorflow_estimator/tools/pip_package/build_pip_package.sh | 2 +-
> + 1 file changed, 1 insertion(+), 1 deletion(-)
> +
> +diff --git a/tensorflow_estimator/tools/pip_package/build_pip_package.sh b/tensorflow_estimator/tools/pip_package/build_pip_package.sh
> +index d4953a6..e08cd8a 100755
> +--- a/tensorflow_estimator/tools/pip_package/build_pip_package.sh
> ++++ b/tensorflow_estimator/tools/pip_package/build_pip_package.sh
> +@@ -38,7 +38,7 @@ function prepare_src() {
> +
> +   # Verifies all expected files are in pip.
> +   # Creates init files in all directory in pip.
> +-  python tensorflow_estimator/tools/pip_package/create_pip_helper.py --pip-root "${TMPDIR}/tensorflow_estimator/" --bazel-root "./tensorflow_estimator"
> ++  nativepython3 tensorflow_estimator/tools/pip_package/create_pip_helper.py --pip-root "${TMPDIR}/tensorflow_estimator/" --bazel-root "./tensorflow_estimator"
> + }
> +
> + function build_wheel() {
> +--
> +2.18.2
> +
> diff --git a/recipes-framework/tensorflow/tensorflow-estimator_1.13.bb b/recipes-framework/tensorflow/tensorflow-estimator_1.13.bb
> index f3a5098..5b2fe5d 100644
> --- a/recipes-framework/tensorflow/tensorflow-estimator_1.13.bb
> +++ b/recipes-framework/tensorflow/tensorflow-estimator_1.13.bb
> @@ -3,9 +3,10 @@ learning programming."
>  LICENSE = "Apache-2.0"
>  LIC_FILES_CHKSUM = "file://LICENSE;md5=01e86893010a1b87e69a213faa753ebd"
>
> -SRC_URI = "git://github.com/tensorflow/estimator.git;branch=r1.13 \
> +SRC_URI = "git://github.com/tensorflow/estimator.git;branch=r2.4 \
> +           file://0001-customize-for-yocto.patch \
>            "
> -SRCREV = "340703eed78ba4854862f749ed94f91598826e79"
> +SRCREV = "c3e7f2b5bbcc35185ef71797955a28cadce28f60"
>  S = "${WORKDIR}/git"
>
>  inherit python3native bazel
> @@ -19,12 +20,18 @@ DEPENDS += " \
>      python3-astor-native \
>      python3-gast-native \
>      python3-termcolor-native \
> +    python3-wrapt-native \
> +    python3-opt-einsum-native \
> +    python3-astunparse-native \
> +    flatbuffers-native \
I'm getting issue when building tensorflow-estimator (I add small
changes to be able to build this patches on top of dunfell) and I'm
getting:
|   /bin/bash -c 'source
external/bazel_tools/tools/genrule/genrule-setup.sh;
bazel-out/k8-opt-exec-2B5CBBC6/bin/tensorflow_estimator/python/estimator/api/create_tensorflow_estimator.python.estimator_api_1_estimator_python_api_gen_compat_v1
 --apidir=bazel-out/k8-fastbuild/bin/tensorflow_estimator/python/estimator/api_v1/
--apiname=estimator --apiversion=1
--package=tensorflow_estimator.python.estimator
--output_package=tensorflow_estimator.python.estimator.api._v1
bazel-out/k8-fastbuild/bin/tensorflow_estimator/python/estimator/api/_v1/v1.py
bazel-out/k8-fastbuild/bin/tensorflow_estimator/python/estimator/api/_v1/estimator/__init__.py
bazel-out/k8-fastbuild/bin/tensorflow_estimator/python/estimator/api/_v1/estimator/experimental/__init__.py
bazel-out/k8-fastbuild/bin/tensorflow_estimator/python/estimator/api/_v1/estimator/export/__init__.py
bazel-out/k8-fastbuild/bin/tensorflow_estimator/python/estimator/api/_v1/estimator/inputs/__init__.py
bazel-out/k8-fastbuild/bin/tensorflow_estimator/python/estimator/api/_v1/estimator/tpu/__init__.py
bazel-out/k8-fastbuild/bin/tensorflow_estimator/python/estimator/api/_v1/estimator/tpu/experimental/__init__.py')
| Execution platform: @local_config_platform//:host
| /home/ubuntu/projects/test/build/tmp/work/aarch64-poky-linux/tensorflow-estimator/2.4-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/h5py/__init__.py:72:
UserWarning: h5py is running against HDF5 1.8.21 when it was built
against 1.8.19, this may cause problems
|   _warn(("h5py is running against HDF5 {0} when it was built against {1}, "
| Traceback (most recent call last):
|   File "/home/ubuntu/projects/test/build/tmp/work/aarch64-poky-linux/tensorflow-estimator/2.4-r0/bazel/output_base/execroot/org_tensorflow_estimator/bazel-out/k8-opt-exec-2B5CBBC6/bin/tensorflow_estimator/python/estimator/api/create_tensorflow_estimator.python.estimator_api_1_estimator_python_api_gen_compat_v1.runfiles/org_tensorflow_estimator/tensorflow_estimator/python/estimator/api/create_python_api_wrapper.py",
line 26, in <module>
|     from tensorflow_estimator.python.estimator import estimator_lib
# pylint: disable=unused-import
|   File "/home/ubuntu/projects/test/build/tmp/work/aarch64-poky-linux/tensorflow-estimator/2.4-r0/bazel/output_base/execroot/org_tensorflow_estimator/bazel-out/k8-opt-exec-2B5CBBC6/bin/tensorflow_estimator/python/estimator/api/create_tensorflow_estimator.python.estimator_api_1_estimator_python_api_gen_compat_v1.runfiles/org_tensorflow_estimator/tensorflow_estimator/python/estimator/estimator_lib.py",
line 22, in <module>
|     from tensorflow_estimator.python.estimator.canned.baseline
import BaselineClassifier
|   File "/home/ubuntu/projects/test/build/tmp/work/aarch64-poky-linux/tensorflow-estimator/2.4-r0/bazel/output_base/execroot/org_tensorflow_estimator/bazel-out/k8-opt-exec-2B5CBBC6/bin/tensorflow_estimator/python/estimator/api/create_tensorflow_estimator.python.estimator_api_1_estimator_python_api_gen_compat_v1.runfiles/org_tensorflow_estimator/tensorflow_estimator/python/estimator/canned/baseline.py",
line 53, in <module>
|     import tensorflow as tf
|   File "/home/ubuntu/projects/test/build/tmp/work/aarch64-poky-linux/tensorflow-estimator/2.4-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/tensorflow/__init__.py",
line 55, in <module>
|     from ._api.v2 import compat
|   File "/home/ubuntu/projects/test/build/tmp/work/aarch64-poky-linux/tensorflow-estimator/2.4-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/tensorflow/_api/v2/compat/__init__.py",
line 39, in <module>
|     from . import v1
|   File "/home/ubuntu/projects/test/build/tmp/work/aarch64-poky-linux/tensorflow-estimator/2.4-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/tensorflow/_api/v2/compat/v1/__init__.py",
line 34, in <module>
|     from . import compat
|   File "/home/ubuntu/projects/test/build/tmp/work/aarch64-poky-linux/tensorflow-estimator/2.4-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/tensorflow/_api/v2/compat/v1/compat/__init__.py",
line 39, in <module>
|     from . import v1
|   File "/home/ubuntu/projects/test/build/tmp/work/aarch64-poky-linux/tensorflow-estimator/2.4-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/tensorflow/_api/v2/compat/v1/compat/v1/__init__.py",
line 51, in <module>
|     from tensorflow._api.v2.compat.v1 import lite
|   File "/home/ubuntu/projects/test/build/tmp/work/aarch64-poky-linux/tensorflow-estimator/2.4-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/tensorflow/_api/v2/compat/v1/lite/__init__.py",
line 11, in <module>
|     from . import experimental
|   File "/home/ubuntu/projects/test/build/tmp/work/aarch64-poky-linux/tensorflow-estimator/2.4-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/tensorflow/_api/v2/compat/v1/lite/experimental/__init__.py",
line 10, in <module>
|     from . import nn
|   File "/home/ubuntu/projects/test/build/tmp/work/aarch64-poky-linux/tensorflow-estimator/2.4-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/tensorflow/_api/v2/compat/v1/lite/experimental/nn/__init__.py",
line 10, in <module>
|     from tensorflow.lite.python.lite import TFLiteLSTMCell
|   File "/home/ubuntu/projects/test/build/tmp/work/aarch64-poky-linux/tensorflow-estimator/2.4-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/tensorflow/lite/python/lite.py",
line 40, in <module>
|     from tensorflow.lite.python.convert import
build_toco_convert_protos  # pylint: disable=unused-import
|   File "/home/ubuntu/projects/test/build/tmp/work/aarch64-poky-linux/tensorflow-estimator/2.4-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/tensorflow/lite/python/convert.py",
line 33, in <module>
|     from tensorflow.lite.python import util
|   File "/home/ubuntu/projects/test/build/tmp/work/aarch64-poky-linux/tensorflow-estimator/2.4-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/tensorflow/lite/python/util.py",
line 30, in <module>
|     import flatbuffers
| ModuleNotFoundError: No module named 'flatbuffers'

FLatbuffers from openembedded are available version 1.12. Do you have
an idea what is can cause?
>      tensorflow-native \
>  "
>
>  do_compile () {
>      unset CC
>      export TMPDIR="${WORKDIR}"
> +    export PYTHON_BIN_PATH="${PYTHON}"
> +
>      ${BAZEL} build \
>          --subcommands --explain=${T}/explain.log \
>          --verbose_explanations --verbose_failures \
> @@ -32,7 +39,6 @@ do_compile () {
>          --python_path="${PYTHON}" \
>          //tensorflow_estimator/tools/pip_package:build_pip_package
>
> -    PYTHON_BIN_PATH="${PYTHON}" \
>      ${S}/bazel-bin/tensorflow_estimator/tools/pip_package/build_pip_package \
>          ${WORKDIR}/estimator_pip
>  }
> --
> 2.21.0
>

Thanks and BR,

marek
-- 
as simple and primitive as possible
-------------------------------------------------
Marek Belisko - OPEN-NANDRA
Freelance Developer

Ruska Nova Ves 219 | Presov, 08005 Slovak Republic
Tel: +421 915 052 184
skype: marekwhite
twitter: #opennandra
web: http://open-nandra.com

  reply	other threads:[~2021-01-04 10:31 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-16 13:07 Review request 0/25:[meta-tensorflow] upgrade tensorflow 1.13.1 -> 2.4.0.rc4 hongxu
2020-12-16 13:07 ` [meta-tensorflow][PATCH 1/25] openjdk-8-native: 212b04 -> 275b01 hongxu
2020-12-16 16:49   ` [yocto] " Kent Dorfman
2020-12-16 13:07 ` [meta-tensorflow][PATCH 2/25] bazel-native: 0.21.0 -> 3.1.0 hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 3/25] tensorflow: 1.13.0 -> 2.4.0 hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 4/25] add python3-gast 0.3.3 hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 5/25] python3-opt-einsum: add 3.3.0 hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 6/25] python3-google-pasta: add 0.2.0 hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 7/25] python3-astunparse: add 1.6.3 hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 8/25] tensorflow-estimator: 1.13 -> 2.4 hongxu
2021-01-04 10:30   ` Marek Belisko [this message]
2021-01-07 17:51     ` [yocto] " Randy MacLeod
2021-01-07 18:47       ` Marek Belisko
2021-01-07 20:47         ` Randy MacLeod
2021-01-25 18:57           ` Marek Belisko
2021-01-25 20:46             ` Randy MacLeod
2020-12-16 13:08 ` [meta-tensorflow][PATCH 9/25] bazel-native/bazel.bbclass: use default Bazel toolchain to build Yocto native tools hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 10/25] bazel-native/bazel.bbclass: replace deprecated --local_resources hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 11/25] tensorflow: update cross compile support hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 12/25] tensorflow: fix Multiple shlib providers for libtensorflow_framework.so hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 13/25] classes/bazel.bbclass: clean up bazel files before do_clean hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 14/25] python3-google-auth: 1.6.3 -> 1.24.0 hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 15/25] python3-google-auth-oauthlib: add version 0.4.2 hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 16/25] python3-pyasn1: add 0.4.8 hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 17/25] python3-pyasn1-modules: add version 0.2.8 hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 18/25] tensorboard: 1.12.2 -> 2.4 hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 19/25] tensorflow: clean up CROSSTOOL.tpl hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 20/25] bazel-native: 3.1.0 -> 3.7.1 hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 21/25] add classes/bazel-base.bbclass and imporve classes/bazel-base.bbclass hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 22/25] bazel-native/bazel.bbclass: support bazel build without limitation hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 23/25] BUILD.md: update quick start for tensorflow 2 hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 24/25] tensorflow: split sub packages hongxu
2020-12-16 13:08 ` [meta-tensorflow][PATCH 25/25] update LIMITATION.md hongxu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAAfyv37zRgtJ_n2RZmWoHPOado1=LYw9cHVEVUZT1n497XB8yw@mail.gmail.com' \
    --to=marek.belisko@gmail.com \
    --cc=akuster808@gmail.com \
    --cc=hongxu.jia@windriver.com \
    --cc=philip@balister.org \
    --cc=randy.macleod@windriver.com \
    --cc=yocto@yoctoproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.