Linux-ARM-Kernel Archive on lore.kernel.org
 help / color / Atom feed
From: peter@hurleysoftware.com (Peter Hurley)
To: linux-arm-kernel@lists.infradead.org
Subject: [PATCH 03/16] tty: serial: 8250_core: read only RX if there is something in the FIFO
Date: Tue, 10 Feb 2015 12:46:32 -0500
Message-ID: <54DA43F8.9090904@hurleysoftware.com> (raw)
In-Reply-To: <54D9F3C7.5000809@freebox.fr>

On 02/10/2015 07:04 AM, Nicolas Schichan wrote:
> On 02/10/2015 12:34 AM, Peter Hurley wrote:
>> Hi Nicolas,
>>
>> Thanks for the report.
>>
> [...]
>>> When a caracter is received on the UART while the kernel is printing
>>> the boot messages, as soon as the kernel configures the UART for
>>> receiving (after root filesystem mount), it gets stuck printing the
>>> following message repeatedly:
>>>
>>> serial8250: too much work for irq29
>>>
>>> Once stuck, the reception of another character allows the boot process
>>> to finish.
>>>
>>> From what I can gather, when we hit that, the UART_IIR_NO_INT is 0 (so the
>>> interrupt is raised), but the UART_LSR_DR bit is 0 as well so the UART_RX
>>> register is never read to clear the interrupt.
>>
>> The "too much work" message means serial8250_handle_irq() is returning 0,
>> ie., not handled. Which in turn means IIR indicates no interrupt is pending
>> (UART_IIR_NO_INT == 1).
>>
>> Can you log the register values for LSR and IIR at both patch locations
>> in serial8250_do_startup()?
>>
>> (I can get you a debug patch, if necessary. Let me know)
> 
> Hi Peter,
> 
> Thanks for your reply.
> 
> Here is what I have when the issue is triggered:
> 
> [   12.154877] lsr 0x60 / iir 0x01
> [   12.158071] lsr 0x60 / iir 0x01
> [   12.161438] serial8250: too much work for irq29
> [   12.165982] lsr 0x60 / iir 0x0c
> [   12.169354] serial8250: too much work for irq29
> [   12.173900] lsr 0x60 / iir 0x0c
> (previous two messages are repeated and printk_ratelimited())

Thanks for this information; I see I was wrong about the cause of message.

I think what happens during startup is that on this silicon clearing
the rx fifo (by serial8250_clear_fifos()) clears data ready but not
the rx timeout condition which causes a spurious rx interrupt when
interrupts are enabled.

So caught between two broken UARTs: one that underflows its rx fifo because
of unsolicited rx reads and the other that generates spurious interrupt
without unsolicited rx reads.


> When the issue is not triggered:
> 
> [   10.784871] lsr 0x60 / iir 0x01
> [   10.788066] lsr 0x60 / iir 0x01
> [   10.794734] VFS: Mounted root (nfs filesystem) readonly on device 0:13.
> [   10.801654] devtmpfs: mounted
> [   10.805169] Freeing unused kernel memory: 184K (807be000 - 807ec000)
> (userland takes over after that)
> 
> I have also displayed the IIR and LSR registers when the "too much fork for
> IRQ" condition is triggered.
> 
> In the serial8250_do_startup(), before the interrupt are unmasked at the end,
> the IIR looks sane and UART_IIR_NO_INT bit is set. When stuck
> serial8250_interrupt(), UART_IIR_NO_INT is cleared and the interrupt ID is set
> to 0xc which is not handled by the kernel at this time (the Kirkwood datasheet
>  indicates that it is some kind of timeout condition from what I can gather).

Yes, IIR == UART_IIR_RX_TIMEOUT is to used indicate that data is in the rx fifo
but has not reached the rx trigger level yet.

ATM, I'm not exactly sure if there is a safe way to clear the spurious interrupt
from the interrupt handler.

I'm fairly certain the only way to clear the rx timeout interrupt is to read
the rx fifo, but I think this would race with actual data arrival. IOW, there
might not be a way to determine if the data read is spurious or not.

Regards,
Peter Hurley

> Here is the corresponding debug patch, for reference (against a 3.19 kernel
> this time):
> 
> diff --git a/drivers/tty/serial/8250/8250_core.c b/drivers/tty/serial/8250/8250_
> index 11c6685..ed93741 100644
> --- a/drivers/tty/serial/8250/8250_core.c
> +++ b/drivers/tty/serial/8250/8250_core.c
> @@ -1661,6 +1661,12 @@ static int exar_handle_irq(struct uart_port *port)
>         return ret;
>  }
> 
> +static inline void show_lsr_iir(struct uart_port *port)
> +{
> +       printk("lsr 0x%02x / iir 0x%02x\n", serial_port_in(port, UART_LSR),
> +              serial_port_in(port, UART_IIR));
> +}
> +
>  /*
>   * This is the serial driver's interrupt routine.
>   *
> @@ -1705,6 +1711,8 @@ static irqreturn_t serial8250_interrupt(int irq, void *dev
>                         /* If we hit this, we're dead. */
>                         printk_ratelimited(KERN_ERR
>                                 "serial8250: too much work for irq%d\n", irq);
> +                       if (printk_ratelimit())
> +                               show_lsr_iir(port);
>                         break;
>                 }
>         } while (l != end);
> @@ -2107,6 +2115,7 @@ int serial8250_do_startup(struct uart_port *port)
>         /*
>          * Clear the interrupt registers.
>          */
> +       show_lsr_iir(port);
>         if (serial_port_in(port, UART_LSR) & UART_LSR_DR)
>                 serial_port_in(port, UART_RX);
>         serial_port_in(port, UART_IIR);
> @@ -2269,6 +2278,7 @@ dont_test_tx_en:
>          * saved flags to avoid getting false values from polling
>          * routines or the previous session.
>          */
> +       show_lsr_iir(port);
>         if (serial_port_in(port, UART_LSR) & UART_LSR_DR)
>                 serial_port_in(port, UART_RX);
>         serial_port_in(port, UART_IIR);
> 
> Regards,
> 

  reply index

Thread overview: 97+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-09-10 19:29 [PATCH 00/16 v9] omap 8250 based uart + DMA Sebastian Andrzej Siewior
2014-09-10 19:29 ` [PATCH 01/16] tty: serial: 8250_core: allow to set ->throttle / ->unthrottle callbacks Sebastian Andrzej Siewior
2014-09-10 19:29 ` [PATCH 02/16] tty: serial: 8250_core: add run time pm Sebastian Andrzej Siewior
2014-09-29  9:46   ` Frans Klaver
2014-09-29 13:39     ` Sebastian Andrzej Siewior
2014-09-10 19:29 ` [PATCH 03/16] tty: serial: 8250_core: read only RX if there is something in the FIFO Sebastian Andrzej Siewior
2015-02-09 13:34   ` Nicolas Schichan
2015-02-09 23:34     ` Peter Hurley
2015-02-10  9:32       ` Sebastian Andrzej Siewior
2015-02-10 12:04       ` Nicolas Schichan
2015-02-10 17:46         ` Peter Hurley [this message]
2015-02-11 20:01           ` Peter Hurley
2015-02-11 20:03             ` Tony Lindgren
2015-02-11 20:42               ` Peter Hurley
2015-02-12  8:45                 ` Sebastian Andrzej Siewior
2015-02-12  9:40                   ` Russell King - ARM Linux
2015-02-12 16:32                   ` Peter Hurley
2015-02-12 19:23                     ` Sebastian Andrzej Siewior
2015-02-12 19:55                       ` Peter Hurley
2015-02-12 20:34                         ` Sebastian Andrzej Siewior
2015-02-13 18:51                         ` Sebastian Andrzej Siewior
2015-02-13 23:15                           ` Russell King - ARM Linux
2015-02-15 17:32                             ` [PATCH] serial: 8250: Revert "tty: serial: 8250_core: read only RX if there is something in the FIFO" Sebastian Andrzej Siewior
2015-05-12 20:25                               ` Tony Lindgren
2014-09-10 19:29 ` [PATCH 04/16] tty: serial: 8250_core: use the ->line argument as a hint in serial8250_find_match_or_unused() Sebastian Andrzej Siewior
2014-09-10 19:30 ` [PATCH 05/16] tty: serial: 8250_core: remove UART_IER_RDI in serial8250_stop_rx() Sebastian Andrzej Siewior
2014-09-11 11:19   ` Heikki Krogerus
2014-09-10 19:30 ` [PATCH 06/16] tty: serial: Add 8250-core based omap driver Sebastian Andrzej Siewior
2014-09-11 11:57   ` Peter Hurley
2014-09-16 17:01     ` Sebastian Andrzej Siewior
2014-09-29  9:38   ` Frans Klaver
2014-09-29 13:27     ` Sebastian Andrzej Siewior
2014-09-29 13:34       ` Frans Klaver
2014-09-10 19:30 ` [PATCH 07/16] tty: serial: 8250_dma: handle error on TX submit Sebastian Andrzej Siewior
2014-09-10 19:30 ` [PATCH 08/16] tty: serial: 8250_dma: enqueue RX dma again on completion Sebastian Andrzej Siewior
2014-09-10 19:30 ` [PATCH 09/16] tty: serial: 8250_dma: Add a TX trigger workaround for AM33xx Sebastian Andrzej Siewior
2014-09-11 11:17   ` Heikki Krogerus
2014-09-11 11:42     ` Sebastian Andrzej Siewior
2014-09-11 12:32       ` Peter Hurley
2014-09-11 12:50         ` Sebastian Andrzej Siewior
2014-09-11 14:35           ` Peter Hurley
2014-09-15 17:01             ` Sebastian Andrzej Siewior
2014-09-16 16:55               ` Sebastian Andrzej Siewior
2014-09-17 12:20                 ` Peter Hurley
2014-09-17 16:25                   ` Sebastian Andrzej Siewior
2014-09-29 16:15                   ` Sebastian Andrzej Siewior
2014-09-11 15:11           ` Frans Klaver
2014-09-11 16:04             ` Sebastian Andrzej Siewior
2014-09-11 17:04               ` Frans Klaver
2014-09-12  7:23                 ` Sebastian Andrzej Siewior
     [not found]                   ` <20140912094010.GE28458@ci00147.xsens-tech.local>
2014-09-12  9:51                     ` Sebastian Andrzej Siewior
2014-09-12 10:28                       ` Frans Klaver
2014-09-15 16:42                         ` Sebastian Andrzej Siewior
2014-09-16  9:05                           ` Frans Klaver
2014-09-16 12:42                             ` Frans Klaver
2014-09-16 14:23                               ` Frans Klaver
2014-09-17 10:28                           ` Frans Klaver
2014-09-21 20:41                             ` Sebastian Andrzej Siewior
2014-09-22  9:28                               ` Frans Klaver
2014-09-24  7:56                                 ` Sebastian Andrzej Siewior
2014-09-25 15:14                                 ` Sebastian Andrzej Siewior
2014-09-25 15:18                                   ` Frans Klaver
2014-09-29  8:50                                   ` Frans Klaver
2014-09-29  9:54                                     ` Sebastian Andrzej Siewior
2014-09-29 10:30                                       ` Frans Klaver
2014-09-30  8:44                                         ` Frans Klaver
2014-10-02 10:27                                           ` Sebastian Andrzej Siewior
2014-10-13 14:55                                             ` Frans Klaver
2014-09-23 17:03                               ` Peter Hurley
2014-09-24  7:53                                 ` Sebastian Andrzej Siewior
2014-09-25 10:42                                   ` Sebastian Andrzej Siewior
2014-09-25 11:31                                     ` Peter Hurley
2014-09-25 13:11                                       ` Sebastian Andrzej Siewior
2014-09-17 16:34       ` Sebastian Andrzej Siewior
2014-09-19 10:22         ` Heikki Krogerus
2014-09-19 10:58           ` Sebastian Andrzej Siewior
2014-09-19 11:25             ` Peter Hurley
2014-09-22  7:46             ` Heikki Krogerus
2014-09-25  9:24               ` Sebastian Andrzej Siewior
2014-09-10 19:30 ` [PATCH 10/16] tty: serial: 8250_dma: optimize the xmit path due to UART_BUG_DMA_TX Sebastian Andrzej Siewior
2014-09-10 19:30 ` [PATCH 11/16] tty: serial: 8250_dma: keep own book keeping about RX transfers Sebastian Andrzej Siewior
2014-09-10 19:30 ` [PATCH 12/16] tty: serial: 8250_dma: handle the UART RDI event while DMA remains idle Sebastian Andrzej Siewior
2014-09-29  9:23   ` Frans Klaver
2014-09-10 19:30 ` [PATCH 13/16] tty: serial: 8250_dma: add pm runtime Sebastian Andrzej Siewior
2014-09-29  9:26   ` Frans Klaver
2014-09-29  9:56     ` Sebastian Andrzej Siewior
2014-09-10 19:30 ` [PATCH 14/16] arm: dts: am33xx: add DMA properties for UART Sebastian Andrzej Siewior
2014-09-10 19:30 ` [PATCH 15/16] arm: dts: dra7: " Sebastian Andrzej Siewior
2014-09-10 19:30 ` [PATCH 16/16] tty: serial: 8250: omap: add dma support Sebastian Andrzej Siewior
2014-09-12 22:43 ` [PATCH 00/16 v9] omap 8250 based uart + DMA Tony Lindgren
2014-09-15 11:50   ` Sebastian Andrzej Siewior
2014-09-16 12:57     ` Sebastian Andrzej Siewior
2014-09-16 16:48       ` Tony Lindgren
2014-09-16 21:30         ` Tony Lindgren
2014-09-17  8:38           ` Sebastian Andrzej Siewior
2014-09-17  9:05 ` Sebastian Andrzej Siewior
2014-09-26 16:02   ` Greg KH

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=54DA43F8.9090904@hurleysoftware.com \
    --to=peter@hurleysoftware.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-ARM-Kernel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-arm-kernel/0 linux-arm-kernel/git/0.git
	git clone --mirror https://lore.kernel.org/linux-arm-kernel/1 linux-arm-kernel/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-arm-kernel linux-arm-kernel/ https://lore.kernel.org/linux-arm-kernel \
		linux-arm-kernel@lists.infradead.org
	public-inbox-index linux-arm-kernel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.infradead.lists.linux-arm-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git