* [Buildroot] [PATCH] autobuild-run: prepare_build(): ignore directories in dldir
@ 2018-04-04 7:07 Peter Korsgaard
2018-04-04 7:30 ` Thomas Petazzoni
0 siblings, 1 reply; 3+ messages in thread
From: Peter Korsgaard @ 2018-04-04 7:07 UTC (permalink / raw)
To: buildroot
With the change to use subdirs in dldir, prepare_build may end up trying to
remove a non-empty subdir, causing an exception:
Process Process-1:
Traceback (most recent call last):
File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home/peko/buildroot-test/scripts/autobuild-run", line 627, in run_instance
ret = prepare_build(**kwargs)
File "/home/peko/buildroot-test/scripts/autobuild-run", line 301, in prepare_build
os.remove(os.path.join(dldir, f))
OSError: [Errno 21] Is a directory: 'instance-0/dl/mtools'
We only want to delete individual download files, so drop directories from
the list of removal candidates.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
---
scripts/autobuild-run | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/scripts/autobuild-run b/scripts/autobuild-run
index 0f1c7d7..d2eb8a6 100755
--- a/scripts/autobuild-run
+++ b/scripts/autobuild-run
@@ -293,7 +293,9 @@ def prepare_build(**kwargs):
# regularly re-download files to check that their upstream
# location is still correct.
for i in range(0, 5):
- flist = os.listdir(dldir)
+ # ignore directories
+ flist = [f for f in os.listdir(dldir)
+ if os.path.isfile(os.path.join(dldir, f))]
if not flist:
break
f = flist[randint(0, len(flist) - 1)]
--
2.11.0
^ permalink raw reply related [flat|nested] 3+ messages in thread
* [Buildroot] [PATCH] autobuild-run: prepare_build(): ignore directories in dldir
2018-04-04 7:07 [Buildroot] [PATCH] autobuild-run: prepare_build(): ignore directories in dldir Peter Korsgaard
@ 2018-04-04 7:30 ` Thomas Petazzoni
2018-04-04 9:06 ` Peter Korsgaard
0 siblings, 1 reply; 3+ messages in thread
From: Thomas Petazzoni @ 2018-04-04 7:30 UTC (permalink / raw)
To: buildroot
Hello,
On Wed, 4 Apr 2018 09:07:28 +0200, Peter Korsgaard wrote:
> With the change to use subdirs in dldir, prepare_build may end up trying to
> remove a non-empty subdir, causing an exception:
>
> Process Process-1:
> Traceback (most recent call last):
> File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap
> self.run()
> File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
> self._target(*self._args, **self._kwargs)
> File "/home/peko/buildroot-test/scripts/autobuild-run", line 627, in run_instance
> ret = prepare_build(**kwargs)
> File "/home/peko/buildroot-test/scripts/autobuild-run", line 301, in prepare_build
> os.remove(os.path.join(dldir, f))
> OSError: [Errno 21] Is a directory: 'instance-0/dl/mtools'
>
> We only want to delete individual download files, so drop directories from
> the list of removal candidates.
>
> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
> ---
> scripts/autobuild-run | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/autobuild-run b/scripts/autobuild-run
> index 0f1c7d7..d2eb8a6 100755
> --- a/scripts/autobuild-run
> +++ b/scripts/autobuild-run
> @@ -293,7 +293,9 @@ def prepare_build(**kwargs):
> # regularly re-download files to check that their upstream
> # location is still correct.
> for i in range(0, 5):
> - flist = os.listdir(dldir)
> + # ignore directories
> + flist = [f for f in os.listdir(dldir)
> + if os.path.isfile(os.path.join(dldir, f))]
> if not flist:
> break
> f = flist[randint(0, len(flist) - 1)]
But this is not going to remove files in sub-directories I believe, and
therefore it no longer does the job it is supposed to do. I believe we
need to switch to os.walk() to walk down in the sub-directories, no ?
Best regards,
Thomas
--
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 3+ messages in thread
* [Buildroot] [PATCH] autobuild-run: prepare_build(): ignore directories in dldir
2018-04-04 7:30 ` Thomas Petazzoni
@ 2018-04-04 9:06 ` Peter Korsgaard
0 siblings, 0 replies; 3+ messages in thread
From: Peter Korsgaard @ 2018-04-04 9:06 UTC (permalink / raw)
To: buildroot
>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:
> Hello,
> On Wed, 4 Apr 2018 09:07:28 +0200, Peter Korsgaard wrote:
>> With the change to use subdirs in dldir, prepare_build may end up trying to
>> remove a non-empty subdir, causing an exception:
>>
>> Process Process-1:
>> Traceback (most recent call last):
>> File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap
>> self.run()
>> File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
>> self._target(*self._args, **self._kwargs)
>> File "/home/peko/buildroot-test/scripts/autobuild-run", line 627, in run_instance
>> ret = prepare_build(**kwargs)
>> File "/home/peko/buildroot-test/scripts/autobuild-run", line 301, in prepare_build
>> os.remove(os.path.join(dldir, f))
>> OSError: [Errno 21] Is a directory: 'instance-0/dl/mtools'
>>
>> We only want to delete individual download files, so drop directories from
>> the list of removal candidates.
>>
>> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
>> ---
>> scripts/autobuild-run | 4 +++-
>> 1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/scripts/autobuild-run b/scripts/autobuild-run
>> index 0f1c7d7..d2eb8a6 100755
>> --- a/scripts/autobuild-run
>> +++ b/scripts/autobuild-run
>> @@ -293,7 +293,9 @@ def prepare_build(**kwargs):
>> # regularly re-download files to check that their upstream
>> # location is still correct.
>> for i in range(0, 5):
>> - flist = os.listdir(dldir)
>> + # ignore directories
>> + flist = [f for f in os.listdir(dldir)
>> + if os.path.isfile(os.path.join(dldir, f))]
>> if not flist:
>> break
>> f = flist[randint(0, len(flist) - 1)]
> But this is not going to remove files in sub-directories I believe, and
> therefore it no longer does the job it is supposed to do. I believe we
> need to switch to os.walk() to walk down in the sub-directories, no ?
Ehh, you're right. For some reason I was convinced listdir would
recurse.
So something like:
flist = [map(lambda f: os.path.join(r, f), f) for (r, d, f) in os.walk(dldir)]
I'll do a test and resend.
--
Bye, Peter Korsgaard
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2018-04-04 9:06 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-04-04 7:07 [Buildroot] [PATCH] autobuild-run: prepare_build(): ignore directories in dldir Peter Korsgaard
2018-04-04 7:30 ` Thomas Petazzoni
2018-04-04 9:06 ` Peter Korsgaard
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.