Problem with tty changes in linux-next

Problem with tty changes in linux-next

[Luotao Fu]

[-- Attachment #1: Type: text/plain, Size: 3744 bytes --]

Hi,

I just tried to get latest linux-next (HEAD=288092896e2097eebee7d4bf1df9a0c7b550e225)
run on my ARM system (a PXA270 base PCM990 board. The board maps its console to its
ttyS0. During the tests I experienced two problems with the new shiny bkl-free
tty stuff:

1. The tty mutex stuff depends on SMP, which I don't have on my target
system. So I still have to use the bkl for tty_lock and tty_unlock. This
leads to a WARN while booting:
------------[ cut here ]------------
WARNING: at include/linux/tty.h:590 tty_open+0x8c/0x60c()
Modules linked in:
Backtrace:
[<c00256c4>] (dump_backtrace+0x0/0x10c) from [<c0287418>] (dump_stack+0x18/0x1c)
 r6:c015841c r5:c03133f0 r4:0000024e
[<c0287400>] (dump_stack+0x0/0x1c) from [<c003529c>] (warn_slowpath_common+0x58/0x88)
[<c0035244>] (warn_slowpath_common+0x0/0x88) from [<c00352f0>] (warn_slowpath_null+0x24/0x2c)
 r8:c08f8754 r7:c39f2bc0 r6:00500001 r5:00000000 r4:00000000
[<c00352cc>] (warn_slowpath_null+0x0/0x2c) from [<c015841c>] (tty_open+0x8c/0x60c)
[<c0158390>] (tty_open+0x0/0x60c) from [<c009ca50>] (chrdev_open+0x110/0x1bc)
[<c009c940>] (chrdev_open+0x0/0x1bc) from [<c0097d94>] (__dentry_open+0xf0/0x2a0)
 r8:c3401270 r7:c009c940 r6:c3400038 r5:c39f2bc0 r4:00000000
[<c0097ca4>] (__dentry_open+0x0/0x2a0) from [<c009802c>] (nameidata_to_filp+0x58/0x60)
[<c0097fd4>] (nameidata_to_filp+0x0/0x60) from [<c00a461c>] (do_last+0x3d0/0x658)
 r5:00000000 r4:00000000
[<c00a424c>] (do_last+0x0/0x658) from [<c00a65e8>] (do_filp_open+0x18c/0x518)
[<c00a645c>] (do_filp_open+0x0/0x518) from [<c0097bb0>] (do_sys_open+0x70/0x108)
[<c0097b40>] (do_sys_open+0x0/0x108) from [<c0097c80>] (sys_open+0x24/0x28)
[<c0097c5c>] (sys_open+0x0/0x28) from [<c00084b8>] (kernel_init+0xe4/0x188)
[<c00083d4>] (kernel_init+0x0/0x188) from [<c0038c50>] (do_exit+0x0/0x688)
 r6:00000000 r5:00000000 r4:00000000
---[ end trace 7dd0bbd3f54aa46f ]---
tty_port_block_til_ready: flags 0x80000000
------------[ cut here ]------------

Indeed the bkl is held by kernel_init, so we will kind of _always_ run
into this. Seemed that the tty_lock stuff was reworked in
4a179218b78d346e0de37c6d428d5be01fadae9c by Arnd. I'd say that the
check of a holding lock in non-tty_mutex system should be changed here,
probably exclusive check for tty locks.

2. A more serious problem is that printing kernel message no more works
after running into userspace.
....
Freeing init memory: 100K
3sy||_|_|
phyCORE login:
....
The boot message between init and login sheel is printed only
partly. The cursor jumps back and forward. It seems that part the
special characters like new line etc. are cutted away so that the
printout is shown in such a funny manner. After a tty is spawned, every
thing works just well. I can log in onto the system and it seems to work
so far. I bisected the kernel and identified eventually
fb11bee14186af87ee6abb833cf1a2a6be59c65b as the
first bad commit. The actual problem should be, however,
36c621d82b956ff6ff72273f848af53e6c581aba, where tty_port_block_til_ready()
is introduced. Seems to me that there are locking problems. I
unfortunately don't have any insights of tty layer to tell where the
exact problem is.

BTW: We also experienced both the problems above on another board, which
is based on a I.MX27 SOC. So I can tell that this is no trouble with the
PXA itself.

Any hints/ideas?

Thanks

Cheers
Luotao Fu
-- 
Pengutronix e.K.                           | Dipl.-Ing. Luotao Fu        |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Problem with tty changes in linux-next

[Arnd Bergmann]

On Tuesday 06 July 2010, Luotao Fu wrote:

> I just tried to get latest linux-next (HEAD=288092896e2097eebee7d4bf1df9a0c7b550e225)
> run on my ARM system (a PXA270 base PCM990 board. The board maps its console to its
> ttyS0. During the tests I experienced two problems with the new shiny bkl-free
> tty stuff:

Sorry about the warning, I have been aware of this for some time but have
not yet pushed the fix into -next because of other dependencies.

The fix is the first patch in the 'config' branch of

git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl.git

It would probably be best if either Stephen pulls that branch into -next
or Frederic includes it in his tree that is already getting pulled in.

> Indeed the bkl is held by kernel_init, so we will kind of _always_ run
> into this. Seemed that the tty_lock stuff was reworked in
> 4a179218b78d346e0de37c6d428d5be01fadae9c by Arnd. I'd say that the
> check of a holding lock in non-tty_mutex system should be changed here,
> probably exclusive check for tty locks.

The check is only there for CONFIG_TTY_MUTEX=n, otherwise you get the
same information from lockdep. The warning is useful in general
because when it gets triggered this normally means that running with
CONFIG_TTY_MUTEX disabled would be broken.

The particular case of holding the lock from the init code is
a false positive though, because the init code does not get
converted to the tty mutex and in fact the patch referenced above
makes the init code BKL-free as well, which is the intended fix.
 
> 2. A more serious problem is that printing kernel message no more works
> after running into userspace.
> ....
> Freeing init memory: 100K
> 3sy||_|_|
> phyCORE login:
> ....
> The boot message between init and login sheel is printed only
> partly. The cursor jumps back and forward. It seems that part the
> special characters like new line etc. are cutted away so that the
> printout is shown in such a funny manner. After a tty is spawned, every
> thing works just well. I can log in onto the system and it seems to work
> so far. I bisected the kernel and identified eventually
> fb11bee14186af87ee6abb833cf1a2a6be59c65b as the
> first bad commit. The actual problem should be, however,
> 36c621d82b956ff6ff72273f848af53e6c581aba, where tty_port_block_til_ready()
> is introduced. Seems to me that there are locking problems. I
> unfortunately don't have any insights of tty layer to tell where the
> exact problem is.

I'm sorry you had to bisect this. I did the same bisect and already
submitted a patch for this, which probably got stuck in Greg's inbox
while he was preparing the stable releases. I can't find the patch
in the archives now, which could also mean that it never left my local
machine.

I have the patch on a different machine, but will resend it to Greg
when I get there to make sure it doesn't get lost.

	Arnd

Re: Problem with tty changes in linux-next

[Ilya Dryomov]

> > 2. A more serious problem is that printing kernel message no more works
> > after running into userspace.
> > ....
> > Freeing init memory: 100K
> > 3sy||_|_|
> > phyCORE login:
> > ....
> > The boot message between init and login sheel is printed only
> > partly. The cursor jumps back and forward. It seems that part the
> > special characters like new line etc. are cutted away so that the
> > printout is shown in such a funny manner. After a tty is spawned, every
> > thing works just well. I can log in onto the system and it seems to work
> > so far. I bisected the kernel and identified eventually
> > fb11bee14186af87ee6abb833cf1a2a6be59c65b as the
> > first bad commit. The actual problem should be, however,
> > 36c621d82b956ff6ff72273f848af53e6c581aba, where tty_port_block_til_ready()
> > is introduced. Seems to me that there are locking problems. I
> > unfortunately don't have any insights of tty layer to tell where the
> > exact problem is.
> 
> I'm sorry you had to bisect this. I did the same bisect and already
> submitted a patch for this, which probably got stuck in Greg's inbox
> while he was preparing the stable releases. I can't find the patch
> in the archives now, which could also mean that it never left my local
> machine.
> 
> I have the patch on a different machine, but will resend it to Greg
> when I get there to make sure it doesn't get lost.

I had the same issue. Here is the patch:

https://patchwork.kernel.org/patch/108700/

Thanks,

		Ilya

> 
> 	Arnd
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: Problem with tty changes in linux-next

[Greg KH]

On Tue, Jul 06, 2010 at 04:23:48PM +0200, Arnd Bergmann wrote:
> > 2. A more serious problem is that printing kernel message no more works
> > after running into userspace.
> > ....
> > Freeing init memory: 100K
> > 3sy||_|_|
> > phyCORE login:
> > ....
> > The boot message between init and login sheel is printed only
> > partly. The cursor jumps back and forward. It seems that part the
> > special characters like new line etc. are cutted away so that the
> > printout is shown in such a funny manner. After a tty is spawned, every
> > thing works just well. I can log in onto the system and it seems to work
> > so far. I bisected the kernel and identified eventually
> > fb11bee14186af87ee6abb833cf1a2a6be59c65b as the
> > first bad commit. The actual problem should be, however,
> > 36c621d82b956ff6ff72273f848af53e6c581aba, where tty_port_block_til_ready()
> > is introduced. Seems to me that there are locking problems. I
> > unfortunately don't have any insights of tty layer to tell where the
> > exact problem is.
> 
> I'm sorry you had to bisect this. I did the same bisect and already
> submitted a patch for this, which probably got stuck in Greg's inbox
> while he was preparing the stable releases. I can't find the patch
> in the archives now, which could also mean that it never left my local
> machine.
> 
> I have the patch on a different machine, but will resend it to Greg
> when I get there to make sure it doesn't get lost.

It's in my todo queue, sorry, I spent last week on the 5 stable kernel
releases, I should get to this today and push it into the next
linux-next release.

thanks,

greg k-h

Re: Problem with tty changes in linux-next

[Arnd Bergmann]

On Tuesday 06 July 2010 17:14:54 Greg KH wrote:
> I should get to this today and push it into the next
> linux-next release.

Ok, thanks!

On Tuesday 06 July 2010 16:40:52 Ilya Dryomov wrote:
> I had the same issue. Here is the patch:
> 
> https://patchwork.kernel.org/patch/108700/

Yes, that's the one. It would be good to get another confirmation
from Luotao Fu that this fixes all the problems with missing characters.
As I mentioned, the boot-time warning will get taken care of by
a different patch, but can be safely ignored.

	Arnd 

Re: Problem with tty changes in linux-next

[Luotao Fu]

[-- Attachment #1: Type: text/plain, Size: 1080 bytes --]

Hi,

On Tue, Jul 06, 2010 at 05:35:02PM +0200, Arnd Bergmann wrote:
> On Tuesday 06 July 2010 17:14:54 Greg KH wrote:
> > I should get to this today and push it into the next
> > linux-next release.
> 
> Ok, thanks!
> 
> On Tuesday 06 July 2010 16:40:52 Ilya Dryomov wrote:
> > I had the same issue. Here is the patch:
> > 
> > https://patchwork.kernel.org/patch/108700/
> 
> Yes, that's the one. It would be good to get another confirmation
> from Luotao Fu that this fixes all the problems with missing characters.
> As I mentioned, the boot-time warning will get taken care of by
> a different patch, but can be safely ignored.

I picked up the patch and did a quick test. The console message printing
works so far.

Thanks

Cheers
Luotao Fu
-- 
Pengutronix e.K.                           | Dipl.-Ing. Luotao Fu        |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]