ckmake in Python

ckmake in Python

[Ozan Çağlayan]

[-- Attachment #1: Type: text/plain, Size: 462 bytes --]

Hi,

The latest ckmake rewritten in Python doesn't work on Fedora 18
probably because of curses.
The list of kernels is shown but I can't go up/down or select
something. I have to hit CTRL^Z and kill
ckmake in order to get rid of it. The situation is the same in both
gnome-terminal and a real tty.

Attached is a screenshot.

-- 
Ozan Çağlayan
Research Assistant
Galatasaray University - Computer Engineering Dept.
http://www.ozancaglayan.com

[-- Attachment #2: ckmake-fedora.png --]
[-- Type: image/png, Size: 15624 bytes --]

Re: ckmake in Python

[Luis R. Rodriguez]

On Sat, Dec 22, 2012 at 3:21 AM, Ozan =C3=87a=C4=9Flayan <ozancag@gmail.com=
> wrote:
> Hi,
>
> The latest ckmake rewritten in Python doesn't work on Fedora 18
> probably because of curses.
> The list of kernels is shown but I can't go up/down or select
> something. I have to hit CTRL^Z and kill
> ckmake in order to get rid of it. The situation is the same in both
> gnome-terminal and a real tty.

It doesn't support letting you select the kernels via the boxes, but
that would certainly be a welcomed change. Right now when you run it,
it will actually kick off compilation across all kernels and you will
see a status update for each one as each kernel job thread finishes.
Hitting CTRL-C is handled on the program through a signal handler and
its supposed to clean things but for some reason its not doing that or
its taking a while.

  Luis

Re: ckmake in Python

[Ozan Çağlayan]

[-- Attachment #1: Type: text/plain, Size: 43 bytes --]

Ah, So what was the point of using curses?

[-- Attachment #2: Type: text/html, Size: 61 bytes --]

Re: ckmake in Python

[Luis R. Rodriguez]

On Fri, Dec 28, 2012 at 1:47 PM, Ozan =C3=87a=C4=9Flayan <ozancag@gmail.com=
> wrote:
> Ah, So what was the point of using curses?

Multithreading.

  Luis

Re: ckmake in Python

[Luis R. Rodriguez]

On Fri, Dec 28, 2012 at 2:49 PM, Luis R. Rodriguez <mcgrof@frijolero.org> w=
rote:
> On Fri, Dec 28, 2012 at 1:47 PM, Ozan =C3=87a=C4=9Flayan <ozancag@gmail.c=
om> wrote:
>> Ah, So what was the point of using curses?
>
> Multithreading.

Rather python --> multithreading. The fact that ncurses was used is
more of a wish to display live results from multithreaded processes
running in the background.

  Luis

Re: ckmake in Python

[Ozan Çağlayan]

On Sat, Dec 29, 2012 at 1:10 AM, Luis R. Rodriguez <mcgrof@frijolero.org> w=
rote:
> On Fri, Dec 28, 2012 at 2:49 PM, Luis R. Rodriguez <mcgrof@frijolero.org>=
 wrote:
>> On Fri, Dec 28, 2012 at 1:47 PM, Ozan =C3=87a=C4=9Flayan <ozancag@gmail.=
com> wrote:
>>> Ah, So what was the point of using curses?
>>
>> Multithreading.
>
> Rather python --> multithreading. The fact that ncurses was used is
> more of a wish to display live results from multithreaded processes
> running in the background.

Okay. This really doesn't work on Fedora :) When I start ckmake,
nothing happens. ckmake process runs, load average jumps to 15-20~,
ckmake hogs the cpu by %200 but building never finishes, even worse
last night there wasn't even any forked make process at all. I had to
stop it using CTRL+Z and then kill it after 20 minutes of waiting.

Also now I have a .tmp.ckmake folder in compat/ that I can't delete (I
cut 'rm -rf' after some 20 minutes, too. Maybe this should sit and
wait there for further compilations? I don't know)
[root@localhost .tmp.ckmake]# ll
total 40
drwxr-xr-x 6 root root 4096 Jan  7 18:56 2.6.24
drwxr-xr-x 6 root root 4096 Jan  7 18:56 2.6.27
drwxr-xr-x 3 root root 4096 Jan  8 12:17 2.6.28
drwxr-xr-x 6 root root 4096 Jan  7 18:56 2.6.30
drwxr-xr-x 6 root root 4096 Jan  7 18:56 2.6.31
drwxr-xr-x 6 root root 4096 Jan  7 18:56 2.6.33
drwxr-xr-x 6 root root 4096 Jan  7 18:56 2.6.35
drwxr-xr-x 6 root root 4096 Jan  7 18:56 2.6.36
drwxr-xr-x 6 root root 4096 Jan  7 18:56 3.1.10
drwxr-xr-x 6 root root 4096 Jan  7 18:56 3.2.33

Anyway I started to think about reviving ckmake.sh and keeping this
python-based one as experimental or development stuff. What do you
think about? Or how should I debug this to understand why things
stall?

Thanks :)

Re: ckmake in Python

[Luis R. Rodriguez]

On Tue, Jan 8, 2013 at 2:27 AM, Ozan =C3=87a=C4=9Flayan <ozancag@gmail.com>=
 wrote:
> On Sat, Dec 29, 2012 at 1:10 AM, Luis R. Rodriguez <mcgrof@frijolero.org>=
 wrote:
>> On Fri, Dec 28, 2012 at 2:49 PM, Luis R. Rodriguez <mcgrof@frijolero.org=
> wrote:
>>> On Fri, Dec 28, 2012 at 1:47 PM, Ozan =C3=87a=C4=9Flayan <ozancag@gmail=
.com> wrote:
>>>> Ah, So what was the point of using curses?
>>>
>>> Multithreading.
>>
>> Rather python --> multithreading. The fact that ncurses was used is
>> more of a wish to display live results from multithreaded processes
>> running in the background.
>
> Okay. This really doesn't work on Fedora :) When I start ckmake,
> nothing happens.

Nothing should happen unless a compilation completes for a kernel.

> ckmake process runs, load average jumps to 15-20~,
> ckmake hogs the cpu by %200 but building never finishes, even worse
> last night there wasn't even any forked make process at all. I had to
> stop it using CTRL+Z and then kill it after 20 minutes of waiting.

Hm odd, I just tested this on a Fedora 16 box and it worked without
even adding any new packages. I tested this by trying to run ckmake
only against compat.git, not compat-drivers. I'll try it now with
compat-drivers on the Fedora box to see if I see something similar.

> Also now I have a .tmp.ckmake folder in compat/ that I can't delete (I
> cut 'rm -rf' after some 20 minutes, too. Maybe this should sit and
> wait there for further compilations? I don't know)

Indeed, so it uses that directory to store compies of entire source
code of the current git *for each kernel* that it wants to test on. We
do this given that we cannot share the same directory for output
binary for different kernels when they are all running in parallel.

If we hit CTRL the signal handler should remove that directory. The
directory should also be nuked when it completes.

> [root@localhost .tmp.ckmake]# ll
> total 40
> drwxr-xr-x 6 root root 4096 Jan  7 18:56 2.6.24
> drwxr-xr-x 6 root root 4096 Jan  7 18:56 2.6.27
> drwxr-xr-x 3 root root 4096 Jan  8 12:17 2.6.28
> drwxr-xr-x 6 root root 4096 Jan  7 18:56 2.6.30
> drwxr-xr-x 6 root root 4096 Jan  7 18:56 2.6.31
> drwxr-xr-x 6 root root 4096 Jan  7 18:56 2.6.33
> drwxr-xr-x 6 root root 4096 Jan  7 18:56 2.6.35
> drwxr-xr-x 6 root root 4096 Jan  7 18:56 2.6.36
> drwxr-xr-x 6 root root 4096 Jan  7 18:56 3.1.10
> drwxr-xr-x 6 root root 4096 Jan  7 18:56 3.2.33
>
> Anyway I started to think about reviving ckmake.sh and keeping this
> python-based one as experimental or development stuff. What do you
> think about?

Sure.

> Or how should I debug this to understand why things
> stall?

Can you try ckmake just against compat.git ? Or perhaps get me access
to the box:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCeF/MMg+KqfzBfy+undzW4XCP8vJedmMO3/q5=
ypNz9P+825coqOfqVIJclO2/gPgYWwCP9oif7hVj9ASduaxQCQ1Cbqk7IU64MhdEKmsV77Z3f7+=
GC0jwr7L5iF45DvD+XBO5ZC0EFZJbd45Fh7iW2/IBde7QWu1DtzjTu3PCeavSJ3MtV129qicSPX=
SgWbEOZXngL8WrnQ5n7rbZ4lfJPag13SSoewhU582GTuroIe4f9g/4J6RpoKvR/Ck6wMwRt3evQ=
x/yet+BaqlydJoAB/3H8FR9Y0qoGHGG32493Flb8MBROjutcvRx0mdqgZL1QJWdwfRjKcSbWe9a=
Id699
mcgrof@frijol

  Luis

Re: ckmake in Python

[Luis R. Rodriguez]

On Mon, Jan 21, 2013 at 5:15 PM, Luis R. Rodriguez <mcgrof@frijolero.org> wrote:
> If we hit CTRL the signal handler should remove that directory. The
> directory should also be nuked when it completes.

Another thing you could do to debug is to tail -f the log file for each kernel:

tail -f .tmp.ckmake/2.6.24/ckmake.n.log

If that doesn't ever move you know something is going wrong. On a slow
box this might happen but the only reason why I suspect this might
happen on a slow box is the heuristics for the number of job processes
running at the same time to do compilation is too high. Right now use
use make -j $NUM_CPUS and this is computed by cpu_info_build_jobs() on
ckmake.

  Luis

Re: ckmake in Python

[Luis R. Rodriguez]

On Mon, Jan 21, 2013 at 6:45 PM, Luis R. Rodriguez <mcgrof@frijolero.org> wrote:
> On Mon, Jan 21, 2013 at 5:15 PM, Luis R. Rodriguez <mcgrof@frijolero.org> wrote:
>> If we hit CTRL the signal handler should remove that directory. The
>> directory should also be nuked when it completes.
>
> Another thing you could do to debug is to tail -f the log file for each kernel:
>
> tail -f .tmp.ckmake/2.6.24/ckmake.n.log
>
> If that doesn't ever move you know something is going wrong. On a slow
> box this might happen but the only reason why I suspect this might
> happen on a slow box is the heuristics for the number of job processes
> running at the same time to do compilation is too high. Right now use
> use make -j $NUM_CPUS and this is computed by cpu_info_build_jobs() on
> ckmake.

I just ran this on compat-drivers on a Fedora 16 box and it completed
just fine. It did take 55 minutes as its not the monster build server
we have for compat test builds though, but it did finish.

So what we do not have is a progress bar for each kernel -- and that'd
be nice but not even sure how to beging writing that. Maybe we should
have another thread that measures the number of lines on the kernel
build log file.. or something like that.. not sure, and display that
on the right column instead of just an empty box while things move on.

  Luis

Re: ckmake in Python

[Ozan Çağlayan]

[-- Attachment #1: Type: text/plain, Size: 103 bytes --]

Okay,

I think this was caused by a mess in the source tree. With a clean one, I
can't reproduce this.

[-- Attachment #2: Type: text/html, Size: 149 bytes --]