* [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...`
@ 2021-12-06 11:44 Andy Shevchenko
2021-12-06 11:44 ` [PATCH v3 2/2] binman: Use less hard coded magic when inserting new PATH Andy Shevchenko
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Andy Shevchenko @ 2021-12-06 11:44 UTC (permalink / raw)
To: Andy Shevchenko, u-boot; +Cc: Simon Glass, Tom Rini
Importing libraries in Python caches the bytecode by default.
Since we run scripts in source tree it ignores the current directory
settings, which is $(srctree), and creates cache just in the middle
of the source tree. Move cache to the current directory.
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
---
v3: avoided crash (Simon), preserved tree hierarchy
tools/binman/main.py | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/tools/binman/main.py b/tools/binman/main.py
index 8c1e478d54ce..d19ded491af7 100755
--- a/tools/binman/main.py
+++ b/tools/binman/main.py
@@ -16,9 +16,20 @@ import sys
import traceback
import unittest
+# Get the absolute path to this file at run-time
+our_path = os.path.dirname(os.path.realpath(__file__))
+our1_path = os.path.dirname(our_path)
+our2_path = os.path.dirname(our1_path)
+
+#
+# Do not pollute source tree with cache files:
+# https://stackoverflow.com/a/60024195/2511795
+# https://bugs.python.org/issue33499
+#
+sys.pycache_prefix = os.path.relpath(our_path, os.environ.get('srctree', our2_path))
+
# Bring in the patman and dtoc libraries (but don't override the first path
# in PYTHONPATH)
-our_path = os.path.dirname(os.path.realpath(__file__))
sys.path.insert(2, os.path.join(our_path, '..'))
from patman import test_util
--
2.33.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v3 2/2] binman: Use less hard coded magic when inserting new PATH
2021-12-06 11:44 [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...` Andy Shevchenko
@ 2021-12-06 11:44 ` Andy Shevchenko
2021-12-15 0:33 ` Simon Glass
2021-12-15 0:33 ` [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...` Simon Glass
2 siblings, 0 replies; 7+ messages in thread
From: Andy Shevchenko @ 2021-12-06 11:44 UTC (permalink / raw)
To: Andy Shevchenko, u-boot; +Cc: Simon Glass, Tom Rini
Instead of joining hard coded '..' to the run-time path of the executable,
take just a dirname out of it. Besides that, use $(srctree) where it makes
sense.
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
---
v3: avoided crash (Simon), reused one level up folder variable
tools/binman/main.py | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/tools/binman/main.py b/tools/binman/main.py
index d19ded491af7..6f5ee4e21db7 100755
--- a/tools/binman/main.py
+++ b/tools/binman/main.py
@@ -21,24 +21,26 @@ our_path = os.path.dirname(os.path.realpath(__file__))
our1_path = os.path.dirname(our_path)
our2_path = os.path.dirname(our1_path)
+# Extract $(srctree) from Kbuild environment, or use relative paths below
+srctree = os.environ.get('srctree', our2_path)
+
#
# Do not pollute source tree with cache files:
# https://stackoverflow.com/a/60024195/2511795
# https://bugs.python.org/issue33499
#
-sys.pycache_prefix = os.path.relpath(our_path, os.environ.get('srctree', our2_path))
+sys.pycache_prefix = os.path.relpath(our_path, srctree)
# Bring in the patman and dtoc libraries (but don't override the first path
# in PYTHONPATH)
-sys.path.insert(2, os.path.join(our_path, '..'))
+sys.path.insert(2, our1_path)
from patman import test_util
# Bring in the libfdt module
sys.path.insert(2, 'scripts/dtc/pylibfdt')
-sys.path.insert(2, os.path.join(our_path, '../../scripts/dtc/pylibfdt'))
-sys.path.insert(2, os.path.join(our_path,
- '../../build-sandbox_spl/scripts/dtc/pylibfdt'))
+sys.path.insert(2, os.path.join(srctree, 'scripts/dtc/pylibfdt'))
+sys.path.insert(2, os.path.join(srctree, 'build-sandbox_spl/scripts/dtc/pylibfdt'))
# When running under python-coverage on Ubuntu 16.04, the dist-packages
# directories are dropped from the python path. Add them in so that we can find
--
2.33.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v3 2/2] binman: Use less hard coded magic when inserting new PATH
2021-12-06 11:44 [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...` Andy Shevchenko
2021-12-06 11:44 ` [PATCH v3 2/2] binman: Use less hard coded magic when inserting new PATH Andy Shevchenko
@ 2021-12-15 0:33 ` Simon Glass
2021-12-15 0:33 ` [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...` Simon Glass
2 siblings, 0 replies; 7+ messages in thread
From: Simon Glass @ 2021-12-15 0:33 UTC (permalink / raw)
To: Andy Shevchenko; +Cc: Simon Glass, Tom Rini, u-boot
Instead of joining hard coded '..' to the run-time path of the executable,
take just a dirname out of it. Besides that, use $(srctree) where it makes
sense.
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
---
v3: avoided crash (Simon), reused one level up folder variable
tools/binman/main.py | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
Applied to u-boot-dm/next, thanks!
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...`
2021-12-06 11:44 [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...` Andy Shevchenko
2021-12-06 11:44 ` [PATCH v3 2/2] binman: Use less hard coded magic when inserting new PATH Andy Shevchenko
2021-12-15 0:33 ` Simon Glass
@ 2021-12-15 0:33 ` Simon Glass
2021-12-23 5:07 ` Simon Glass
2 siblings, 1 reply; 7+ messages in thread
From: Simon Glass @ 2021-12-15 0:33 UTC (permalink / raw)
To: Andy Shevchenko; +Cc: Simon Glass, Tom Rini, u-boot
Importing libraries in Python caches the bytecode by default.
Since we run scripts in source tree it ignores the current directory
settings, which is $(srctree), and creates cache just in the middle
of the source tree. Move cache to the current directory.
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
---
v3: avoided crash (Simon), preserved tree hierarchy
tools/binman/main.py | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
Applied to u-boot-dm/next, thanks!
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...`
2021-12-15 0:33 ` [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...` Simon Glass
@ 2021-12-23 5:07 ` Simon Glass
2021-12-23 9:06 ` Andy Shevchenko
0 siblings, 1 reply; 7+ messages in thread
From: Simon Glass @ 2021-12-23 5:07 UTC (permalink / raw)
To: Andy Shevchenko; +Cc: Tom Rini, U-Boot Mailing List
Hi Andy,
On Tue, 14 Dec 2021 at 17:33, Simon Glass <sjg@chromium.org> wrote:
>
> Importing libraries in Python caches the bytecode by default.
> Since we run scripts in source tree it ignores the current directory
> settings, which is $(srctree), and creates cache just in the middle
> of the source tree. Move cache to the current directory.
>
> Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> ---
> v3: avoided crash (Simon), preserved tree hierarchy
> tools/binman/main.py | 13 ++++++++++++-
> 1 file changed, 12 insertions(+), 1 deletion(-)
>
> Applied to u-boot-dm/next, thanks!
I didn't notice this before, but this seems to create files like this:
./tools/binman/usr/lib/python3/dist-packages/elftools/common/construct_utils.cpython-39.pyc
We don't really want to 'recache' the common Python files. Do you
think we should revert this patch, or find another fix?
Regards,
Simon
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...`
2021-12-23 5:07 ` Simon Glass
@ 2021-12-23 9:06 ` Andy Shevchenko
2021-12-28 8:34 ` Simon Glass
0 siblings, 1 reply; 7+ messages in thread
From: Andy Shevchenko @ 2021-12-23 9:06 UTC (permalink / raw)
To: Simon Glass; +Cc: Andy Shevchenko, Tom Rini, U-Boot Mailing List
On Thu, Dec 23, 2021 at 7:08 AM Simon Glass <sjg@chromium.org> wrote:
> On Tue, 14 Dec 2021 at 17:33, Simon Glass <sjg@chromium.org> wrote:
> >
> > Importing libraries in Python caches the bytecode by default.
> > Since we run scripts in source tree it ignores the current directory
> > settings, which is $(srctree), and creates cache just in the middle
> > of the source tree. Move cache to the current directory.
> >
> > Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> > ---
> > v3: avoided crash (Simon), preserved tree hierarchy
> > tools/binman/main.py | 13 ++++++++++++-
> > 1 file changed, 12 insertions(+), 1 deletion(-)
> >
> > Applied to u-boot-dm/next, thanks!
>
> I didn't notice this before, but this seems to create files like this:
>
> ./tools/binman/usr/lib/python3/dist-packages/elftools/common/construct_utils.cpython-39.pyc
>
> We don't really want to 'recache' the common Python files. Do you
> think we should revert this patch, or find another fix?
I'm not sure I understand. efitools is not common, it's a separate
(non-standard) module and it's cached.
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...`
2021-12-23 9:06 ` Andy Shevchenko
@ 2021-12-28 8:34 ` Simon Glass
0 siblings, 0 replies; 7+ messages in thread
From: Simon Glass @ 2021-12-28 8:34 UTC (permalink / raw)
To: Andy Shevchenko; +Cc: Andy Shevchenko, Tom Rini, U-Boot Mailing List
Hi Andy,
On Thu, 23 Dec 2021 at 02:07, Andy Shevchenko <andy.shevchenko@gmail.com> wrote:
>
> On Thu, Dec 23, 2021 at 7:08 AM Simon Glass <sjg@chromium.org> wrote:
> > On Tue, 14 Dec 2021 at 17:33, Simon Glass <sjg@chromium.org> wrote:
> > >
> > > Importing libraries in Python caches the bytecode by default.
> > > Since we run scripts in source tree it ignores the current directory
> > > settings, which is $(srctree), and creates cache just in the middle
> > > of the source tree. Move cache to the current directory.
> > >
> > > Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> > > ---
> > > v3: avoided crash (Simon), preserved tree hierarchy
> > > tools/binman/main.py | 13 ++++++++++++-
> > > 1 file changed, 12 insertions(+), 1 deletion(-)
> > >
> > > Applied to u-boot-dm/next, thanks!
> >
> > I didn't notice this before, but this seems to create files like this:
> >
> > ./tools/binman/usr/lib/python3/dist-packages/elftools/common/construct_utils.cpython-39.pyc
> >
> > We don't really want to 'recache' the common Python files. Do you
> > think we should revert this patch, or find another fix?
>
> I'm not sure I understand. efitools is not common, it's a separate
> (non-standard) module and it's cached.
The problem is that we don't want to create this file in the source directory.
For me this file is in:
/usr/lib/python3/dist-packages/elftools/
but it could be in ~/.local I suppose.
In any case, the cache file should go there, shouldn't it?
Regards,
Simon
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-12-28 8:42 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-12-06 11:44 [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...` Andy Shevchenko
2021-12-06 11:44 ` [PATCH v3 2/2] binman: Use less hard coded magic when inserting new PATH Andy Shevchenko
2021-12-15 0:33 ` Simon Glass
2021-12-15 0:33 ` [PATCH v3 1/2] binman: Do not pollute source tree when build with `make O=...` Simon Glass
2021-12-23 5:07 ` Simon Glass
2021-12-23 9:06 ` Andy Shevchenko
2021-12-28 8:34 ` Simon Glass
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.