* [PATCH 1/2] cleanup-workdir: add a method for getting the build dir
@ 2015-05-18 20:08 Lucas Dutra Nunes
2015-05-18 20:08 ` [PATCH 2/2] cleanup-workdir: wait for bitbake instances to finish Lucas Dutra Nunes
0 siblings, 1 reply; 6+ messages in thread
From: Lucas Dutra Nunes @ 2015-05-18 20:08 UTC (permalink / raw)
To: openembedded-core
Signed-off-by: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
---
scripts/cleanup-workdir | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/scripts/cleanup-workdir b/scripts/cleanup-workdir
index a7f5a3a..bf37f90 100755
--- a/scripts/cleanup-workdir
+++ b/scripts/cleanup-workdir
@@ -64,6 +64,9 @@ def get_cur_arch_dirs(workdir, arch_dirs):
m = re.match(pattern, output)
arch_dirs.append(m.group(1))
+def get_build_dir():
+ return run_command('echo $BUILDDIR').strip()
+
def main():
global parser
parser = optparse.OptionParser(
@@ -77,7 +80,7 @@ will be deleted. Be CAUTIOUS.""")
options, args = parser.parse_args(sys.argv)
- builddir = run_command('echo $BUILDDIR').strip()
+ builddir = get_build_dir()
if len(builddir) == 0:
err_quit("Please source file \"oe-init-build-env\" first.\n")
--
2.1.4
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 2/2] cleanup-workdir: wait for bitbake instances to finish
2015-05-18 20:08 [PATCH 1/2] cleanup-workdir: add a method for getting the build dir Lucas Dutra Nunes
@ 2015-05-18 20:08 ` Lucas Dutra Nunes
2015-05-18 22:23 ` Christopher Larson
0 siblings, 1 reply; 6+ messages in thread
From: Lucas Dutra Nunes @ 2015-05-18 20:08 UTC (permalink / raw)
To: openembedded-core
bitbake uses a lock file on the build dir, "bitbake.lock", to prevent it
from running before an instance has exited. And sometimes the
cleanup-workdir script can call bibake too many times, too fast, before
the lock has been released.
By simply waiting that the lock has been released solves this problem.
Signed-off-by: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
---
scripts/cleanup-workdir | 31 +++++++++++++++++++++++++++----
1 file changed, 27 insertions(+), 4 deletions(-)
diff --git a/scripts/cleanup-workdir b/scripts/cleanup-workdir
index bf37f90..3e1df1f 100755
--- a/scripts/cleanup-workdir
+++ b/scripts/cleanup-workdir
@@ -21,6 +21,8 @@ import optparse
import re
import subprocess
import shutil
+import fcntl
+from contextlib import contextmanager
pkg_cur_dirs = {}
obsolete_dirs = []
@@ -51,7 +53,7 @@ def get_cur_arch_dirs(workdir, arch_dirs):
pattern = workdir + '/(.*?)/'
cmd = "bitbake -e | grep ^SDK_ARCH="
- output = run_command(cmd)
+ output = run_bitbake_command(cmd)
sdk_arch = output.split('"')[1]
# select thest 5 packages to get the dirs of current arch
@@ -59,7 +61,7 @@ def get_cur_arch_dirs(workdir, arch_dirs):
for pkg in pkgs:
cmd = "bitbake -e " + pkg + " | grep ^IMAGE_ROOTFS="
- output = run_command(cmd)
+ output = run_bitbake_command(cmd)
output = output.split('"')[1]
m = re.match(pattern, output)
arch_dirs.append(m.group(1))
@@ -67,6 +69,27 @@ def get_cur_arch_dirs(workdir, arch_dirs):
def get_build_dir():
return run_command('echo $BUILDDIR').strip()
+@contextmanager
+def wait_for_bitbake():
+ builddir = get_build_dir()
+ bitbake_lock_file = os.path.join(builddir, 'bitbake.lock')
+
+ with open(bitbake_lock_file, 'w+') as f:
+ fd = f.fileno()
+ try:
+ # Lock and unlock the lock file, to be sure that there are no other
+ # instances of bitbake running at the moment:
+ fcntl.flock(fd, fcntl.LOCK_EX)
+ fcntl.flock(fd, fcntl.LOCK_UN)
+ yield
+ finally:
+ # And unlock again, just to be safe:
+ fcntl.flock(fd, fcntl.LOCK_UN)
+
+def run_bitbake_command(cmd):
+ with wait_for_bitbake():
+ return run_command(cmd)
+
def main():
global parser
parser = optparse.OptionParser(
@@ -89,7 +112,7 @@ will be deleted. Be CAUTIOUS.""")
print 'Updating bitbake caches...'
cmd = "bitbake -s"
- output = run_command(cmd)
+ output = run_bitbake_command(cmd)
output = output.split('\n')
index = 0
@@ -111,7 +134,7 @@ will be deleted. Be CAUTIOUS.""")
pkg_cur_dirs[elems[0]] = version
cmd = "bitbake -e"
- output = run_command(cmd)
+ output = run_bitbake_command(cmd)
tmpdir = None
image_rootfs = None
--
2.1.4
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] cleanup-workdir: wait for bitbake instances to finish
2015-05-18 20:08 ` [PATCH 2/2] cleanup-workdir: wait for bitbake instances to finish Lucas Dutra Nunes
@ 2015-05-18 22:23 ` Christopher Larson
2015-05-18 22:26 ` Otavio Salvador
2015-05-19 14:21 ` Burton, Ross
0 siblings, 2 replies; 6+ messages in thread
From: Christopher Larson @ 2015-05-18 22:23 UTC (permalink / raw)
To: Lucas Dutra Nunes; +Cc: Patches and discussions about the oe-core layer
[-- Attachment #1: Type: text/plain, Size: 883 bytes --]
On Mon, May 18, 2015 at 1:08 PM, Lucas Dutra Nunes <ldnunes@ossystems.com.br
> wrote:
> bitbake uses a lock file on the build dir, "bitbake.lock", to prevent it
> from running before an instance has exited. And sometimes the
> cleanup-workdir script can call bibake too many times, too fast, before
> the lock has been released.
>
> By simply waiting that the lock has been released solves this problem.
>
> Signed-off-by: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
The fact that the bitbake UI can exit before the lock has been released is
a bug. Not being able to assume bitbake is done by the time the bitbake
command exits is not ideal, no one should have to block directly o the lock
like this.
--
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
[-- Attachment #2: Type: text/html, Size: 1342 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] cleanup-workdir: wait for bitbake instances to finish
2015-05-18 22:23 ` Christopher Larson
@ 2015-05-18 22:26 ` Otavio Salvador
2015-05-19 4:53 ` Anders Darander
2015-05-19 14:21 ` Burton, Ross
1 sibling, 1 reply; 6+ messages in thread
From: Otavio Salvador @ 2015-05-18 22:26 UTC (permalink / raw)
To: Christopher Larson; +Cc: Patches and discussions about the oe-core layer
On Mon, May 18, 2015 at 7:23 PM, Christopher Larson <clarson@kergoth.com> wrote:
>
>
> On Mon, May 18, 2015 at 1:08 PM, Lucas Dutra Nunes <ldnunes@ossystems.com.br> wrote:
>>
>> bitbake uses a lock file on the build dir, "bitbake.lock", to prevent it
>> from running before an instance has exited. And sometimes the
>> cleanup-workdir script can call bibake too many times, too fast, before
>> the lock has been released.
>>
>> By simply waiting that the lock has been released solves this problem.
>>
>> Signed-off-by: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
>
>
> The fact that the bitbake UI can exit before the lock has been released is a bug. Not being able to assume bitbake is done by the time the bitbake command exits is not ideal, no one should have to block directly o the lock like this.
Agreed so you would prefer this to be fixed there?
--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] cleanup-workdir: wait for bitbake instances to finish
2015-05-18 22:26 ` Otavio Salvador
@ 2015-05-19 4:53 ` Anders Darander
0 siblings, 0 replies; 6+ messages in thread
From: Anders Darander @ 2015-05-19 4:53 UTC (permalink / raw)
To: Otavio Salvador
Cc: Christopher Larson, Patches and discussions about the oe-core layer
* Otavio Salvador <otavio@ossystems.com.br> [150519 00:27]:
> On Mon, May 18, 2015 at 7:23 PM, Christopher Larson <clarson@kergoth.com> wrote:
> > On Mon, May 18, 2015 at 1:08 PM, Lucas Dutra Nunes <ldnunes@ossystems.com.br> wrote:
> >> bitbake uses a lock file on the build dir, "bitbake.lock", to prevent it
> >> from running before an instance has exited. And sometimes the
> >> cleanup-workdir script can call bibake too many times, too fast, before
> >> the lock has been released.
> >> By simply waiting that the lock has been released solves this problem.
> >> Signed-off-by: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
> > The fact that the bitbake UI can exit before the lock has been
> > released is a bug. Not being able to assume bitbake is done by the
> > time the bitbake command exits is not ideal, no one should have to
> > block directly o the lock like this.
> Agreed so you would prefer this to be fixed there?
Sure, that would benefit everyone, not just this use-case.
Cheers,
Anders
--
Anders Darander
ChargeStorm AB / eStorm AB
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] cleanup-workdir: wait for bitbake instances to finish
2015-05-18 22:23 ` Christopher Larson
2015-05-18 22:26 ` Otavio Salvador
@ 2015-05-19 14:21 ` Burton, Ross
1 sibling, 0 replies; 6+ messages in thread
From: Burton, Ross @ 2015-05-19 14:21 UTC (permalink / raw)
To: Christopher Larson; +Cc: Patches and discussions about the oe-core layer
[-- Attachment #1: Type: text/plain, Size: 449 bytes --]
On 18 May 2015 at 23:23, Christopher Larson <clarson@kergoth.com> wrote:
> The fact that the bitbake UI can exit before the lock has been released is
> a bug. Not being able to assume bitbake is done by the time the bitbake
> command exits is not ideal, no one should have to block directly o the lock
> like this.
>
Agreed and I personally seem to hit this often, when doing things like
bitbake foo -ccleansstate ; bitbake foo.
Ross
[-- Attachment #2: Type: text/html, Size: 876 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2015-05-19 14:21 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-05-18 20:08 [PATCH 1/2] cleanup-workdir: add a method for getting the build dir Lucas Dutra Nunes
2015-05-18 20:08 ` [PATCH 2/2] cleanup-workdir: wait for bitbake instances to finish Lucas Dutra Nunes
2015-05-18 22:23 ` Christopher Larson
2015-05-18 22:26 ` Otavio Salvador
2015-05-19 4:53 ` Anders Darander
2015-05-19 14:21 ` Burton, Ross
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.