[PATCH v2] serial: stm32: optimize spin lock usage

[PATCH v2] serial: stm32: optimize spin lock usage

[dillon.minfei]

From: dillon min <dillon.minfei@gmail.com>

To avoid potential deadlock in spin_lock usage, use spin_lock_irqsave,
spin_trylock_irqsave(), spin_unlock_irqrestore() in process context.

remove unused local_irq_save/restore call.

Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
Cc: Gerald Baeza <gerald.baeza@foss.st.com>
Cc: Erwan Le Ray <erwan.leray@foss.st.com>
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: dillon min <dillon.minfei@gmail.com>
---
v2: remove unused code from stm32_usart_threaded_interrupt() according from
    Greg's review.

 drivers/tty/serial/stm32-usart.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/drivers/tty/serial/stm32-usart.c b/drivers/tty/serial/stm32-usart.c
index b3675cf25a69..b1ba5e36e36e 100644
--- a/drivers/tty/serial/stm32-usart.c
+++ b/drivers/tty/serial/stm32-usart.c
@@ -1354,13 +1354,12 @@ static void stm32_usart_console_write(struct console *co, const char *s,
 	u32 old_cr1, new_cr1;
 	int locked = 1;
 
-	local_irq_save(flags);
 	if (port->sysrq)
 		locked = 0;
 	else if (oops_in_progress)
-		locked = spin_trylock(&port->lock);
+		locked = spin_trylock_irqsave(&port->lock, flags);
 	else
-		spin_lock(&port->lock);
+		spin_lock_irqsave(&port->lock, flags);
 
 	/* Save and disable interrupts, enable the transmitter */
 	old_cr1 = readl_relaxed(port->membase + ofs->cr1);
@@ -1374,8 +1373,7 @@ static void stm32_usart_console_write(struct console *co, const char *s,
 	writel_relaxed(old_cr1, port->membase + ofs->cr1);
 
 	if (locked)
-		spin_unlock(&port->lock);
-	local_irq_restore(flags);
+		spin_unlock_irqrestore(&port->lock, flags);
 }
 
 static int stm32_usart_console_setup(struct console *co, char *options)
-- 
2.7.4

Re: [PATCH v2] serial: stm32: optimize spin lock usage

[Johan Hovold]

On Mon, Apr 12, 2021 at 05:31:38PM +0800, dillon.minfei@gmail.com wrote:
> From: dillon min <dillon.minfei@gmail.com>
> 
> To avoid potential deadlock in spin_lock usage, use spin_lock_irqsave,
> spin_trylock_irqsave(), spin_unlock_irqrestore() in process context.

This doesn't make much sense as console_write can be called in any
context. And where's the deadlock you claim to be fixing here?
 
> remove unused local_irq_save/restore call.
> 
> Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> Cc: Gerald Baeza <gerald.baeza@foss.st.com>
> Cc: Erwan Le Ray <erwan.leray@foss.st.com>
> Reported-by: kernel test robot <lkp@intel.com>
> Signed-off-by: dillon min <dillon.minfei@gmail.com>
> ---
> v2: remove unused code from stm32_usart_threaded_interrupt() according from
>     Greg's review.
> 
>  drivers/tty/serial/stm32-usart.c | 8 +++-----
>  1 file changed, 3 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/tty/serial/stm32-usart.c b/drivers/tty/serial/stm32-usart.c
> index b3675cf25a69..b1ba5e36e36e 100644
> --- a/drivers/tty/serial/stm32-usart.c
> +++ b/drivers/tty/serial/stm32-usart.c
> @@ -1354,13 +1354,12 @@ static void stm32_usart_console_write(struct console *co, const char *s,
>  	u32 old_cr1, new_cr1;
>  	int locked = 1;
>  
> -	local_irq_save(flags);
>  	if (port->sysrq)
>  		locked = 0;
>  	else if (oops_in_progress)
> -		locked = spin_trylock(&port->lock);
> +		locked = spin_trylock_irqsave(&port->lock, flags);
>  	else
> -		spin_lock(&port->lock);
> +		spin_lock_irqsave(&port->lock, flags);
>  
>  	/* Save and disable interrupts, enable the transmitter */
>  	old_cr1 = readl_relaxed(port->membase + ofs->cr1);
> @@ -1374,8 +1373,7 @@ static void stm32_usart_console_write(struct console *co, const char *s,
>  	writel_relaxed(old_cr1, port->membase + ofs->cr1);
>  
>  	if (locked)
> -		spin_unlock(&port->lock);
> -	local_irq_restore(flags);
> +		spin_unlock_irqrestore(&port->lock, flags);
>  }
>  
>  static int stm32_usart_console_setup(struct console *co, char *options)

Johan

Re: [PATCH v2] serial: stm32: optimize spin lock usage

[dillon min]

Hi Johan,

Yes, there is no deadlock. my fault.
I forget the local_irq_save() plus spin_lock() is spin_lock_irqsave().

Thanks for your review. please ignore this patch.

Best regards

Dillon

On Mon, Apr 12, 2021 at 9:08 PM Johan Hovold <johan@kernel.org> wrote:
>
> On Mon, Apr 12, 2021 at 05:31:38PM +0800, dillon.minfei@gmail.com wrote:
> > From: dillon min <dillon.minfei@gmail.com>
> >
> > To avoid potential deadlock in spin_lock usage, use spin_lock_irqsave,
> > spin_trylock_irqsave(), spin_unlock_irqrestore() in process context.
>
> This doesn't make much sense as console_write can be called in any
> context. And where's the deadlock you claim to be fixing here?
>
> > remove unused local_irq_save/restore call.
> >
> > Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> > Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> > Cc: Gerald Baeza <gerald.baeza@foss.st.com>
> > Cc: Erwan Le Ray <erwan.leray@foss.st.com>
> > Reported-by: kernel test robot <lkp@intel.com>
> > Signed-off-by: dillon min <dillon.minfei@gmail.com>
> > ---
> > v2: remove unused code from stm32_usart_threaded_interrupt() according from
> >     Greg's review.
> >
> >  drivers/tty/serial/stm32-usart.c | 8 +++-----
> >  1 file changed, 3 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/tty/serial/stm32-usart.c b/drivers/tty/serial/stm32-usart.c
> > index b3675cf25a69..b1ba5e36e36e 100644
> > --- a/drivers/tty/serial/stm32-usart.c
> > +++ b/drivers/tty/serial/stm32-usart.c
> > @@ -1354,13 +1354,12 @@ static void stm32_usart_console_write(struct console *co, const char *s,
> >       u32 old_cr1, new_cr1;
> >       int locked = 1;
> >
> > -     local_irq_save(flags);
> >       if (port->sysrq)
> >               locked = 0;
> >       else if (oops_in_progress)
> > -             locked = spin_trylock(&port->lock);
> > +             locked = spin_trylock_irqsave(&port->lock, flags);
> >       else
> > -             spin_lock(&port->lock);
> > +             spin_lock_irqsave(&port->lock, flags);
> >
> >       /* Save and disable interrupts, enable the transmitter */
> >       old_cr1 = readl_relaxed(port->membase + ofs->cr1);
> > @@ -1374,8 +1373,7 @@ static void stm32_usart_console_write(struct console *co, const char *s,
> >       writel_relaxed(old_cr1, port->membase + ofs->cr1);
> >
> >       if (locked)
> > -             spin_unlock(&port->lock);
> > -     local_irq_restore(flags);
> > +             spin_unlock_irqrestore(&port->lock, flags);
> >  }
> >
> >  static int stm32_usart_console_setup(struct console *co, char *options)
>
> Johan

Re: [PATCH v2] serial: stm32: optimize spin lock usage

[dillon min]

Hi Johan, Erwan

It seems still a bit of a problem in the current version, not deadlock
but access register at the same time.

For driver , we should consider it running under smp, let's think
about it for this case:

static void stm32_usart_console_write(struct console *co, const char *s,
                                      unsigned int cnt)
{
         .....
         local_irq_save(flags);
         if (port->sysrq)
                    locked = 0;
         .....
         access register cr1, tdr, isr
         .....

         local_irq_restore(flags);
}

if port->sysrq is 1, stm32_usart_console_write() just disable local
irq response by local_irq_save(), at the time of access register cr1,
tdr, isr. an TXE interrupt raised, for other cores(I know stm32
mpu/mcu do not have multi cores, just assume it has), it still has a
chance to handle interrupt.  Then there is no lock to protect the uart
register.

changes to below, should be more safe:

.....
if (port->sysrq || oops_in_progress)
      locked = spin_trylock_irqsave(&port->lock, flags);
else
      spin_lock_irqsave(&port->lock, flags);

....

if (locked)
     spin_unlock_irqrestore(&port->lock, flags);

For current stm32 soc, it shouldn't happen. just a reminder for future.

Thanks.

Dillon

On Mon, Apr 12, 2021 at 10:04 PM dillon min <dillon.minfei@gmail.com> wrote:
>
> Hi Johan,
>
> Yes, there is no deadlock. my fault.
> I forget the local_irq_save() plus spin_lock() is spin_lock_irqsave().
>
> Thanks for your review. please ignore this patch.
>
> Best regards
>
> Dillon
>
> On Mon, Apr 12, 2021 at 9:08 PM Johan Hovold <johan@kernel.org> wrote:
> >
> > On Mon, Apr 12, 2021 at 05:31:38PM +0800, dillon.minfei@gmail.com wrote:
> > > From: dillon min <dillon.minfei@gmail.com>
> > >
> > > To avoid potential deadlock in spin_lock usage, use spin_lock_irqsave,
> > > spin_trylock_irqsave(), spin_unlock_irqrestore() in process context.
> >
> > This doesn't make much sense as console_write can be called in any
> > context. And where's the deadlock you claim to be fixing here?
> >
> > > remove unused local_irq_save/restore call.
> > >
> > > Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> > > Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> > > Cc: Gerald Baeza <gerald.baeza@foss.st.com>
> > > Cc: Erwan Le Ray <erwan.leray@foss.st.com>
> > > Reported-by: kernel test robot <lkp@intel.com>
> > > Signed-off-by: dillon min <dillon.minfei@gmail.com>
> > > ---
> > > v2: remove unused code from stm32_usart_threaded_interrupt() according from
> > >     Greg's review.
> > >
> > >  drivers/tty/serial/stm32-usart.c | 8 +++-----
> > >  1 file changed, 3 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/drivers/tty/serial/stm32-usart.c b/drivers/tty/serial/stm32-usart.c
> > > index b3675cf25a69..b1ba5e36e36e 100644
> > > --- a/drivers/tty/serial/stm32-usart.c
> > > +++ b/drivers/tty/serial/stm32-usart.c
> > > @@ -1354,13 +1354,12 @@ static void stm32_usart_console_write(struct console *co, const char *s,
> > >       u32 old_cr1, new_cr1;
> > >       int locked = 1;
> > >
> > > -     local_irq_save(flags);
> > >       if (port->sysrq)
> > >               locked = 0;
> > >       else if (oops_in_progress)
> > > -             locked = spin_trylock(&port->lock);
> > > +             locked = spin_trylock_irqsave(&port->lock, flags);
> > >       else
> > > -             spin_lock(&port->lock);
> > > +             spin_lock_irqsave(&port->lock, flags);
> > >
> > >       /* Save and disable interrupts, enable the transmitter */
> > >       old_cr1 = readl_relaxed(port->membase + ofs->cr1);
> > > @@ -1374,8 +1373,7 @@ static void stm32_usart_console_write(struct console *co, const char *s,
> > >       writel_relaxed(old_cr1, port->membase + ofs->cr1);
> > >
> > >       if (locked)
> > > -             spin_unlock(&port->lock);
> > > -     local_irq_restore(flags);
> > > +             spin_unlock_irqrestore(&port->lock, flags);
> > >  }
> > >
> > >  static int stm32_usart_console_setup(struct console *co, char *options)
> >
> > Johan

Re: [PATCH v2] serial: stm32: optimize spin lock usage

[Erwan LE RAY]

Hi Dillon,

STM32MP151 is mono-core, but both STM32MP153 and STM32MP157 are 
dual-core (see 
https://www.st.com/content/st_com/en/products/microcontrollers-microprocessors/stm32-arm-cortex-mpus.html).
So your point is fully relevant, thanks.

ST already fixed the same issue in st-asc.c driver in the past (see 
ef49ffd8), because a systematic deadlock was detected with RT kernel.

You proposed a first implementation in your patch, and a second one in 
the discussion. It seems that your initial proposal (ie your V2 patch) 
is the most standard one (implemented in 6 drivers). The second 
implementation is implemented by only 1 company.

It looks that the solution is to avoid locking in the sysrq case and 
trylock in the oops_in_progress case (see detailed analysis in 
677fe555cbfb1).

So your initial patch looks to the right proposal, but it would be safer 
if Greg could confirm it.

BR, Erwan.


On 4/13/21 1:44 AM, dillon min wrote:
> Hi Johan, Erwan
> 
> It seems still a bit of a problem in the current version, not deadlock
> but access register at the same time.
> 
> For driver , we should consider it running under smp, let's think
> about it for this case:
> 
> static void stm32_usart_console_write(struct console *co, const char *s,
>                                        unsigned int cnt)
> {
>           .....
>           local_irq_save(flags);
>           if (port->sysrq)
>                      locked = 0;
>           .....
>           access register cr1, tdr, isr
>           .....
> 
>           local_irq_restore(flags);
> }
> 
> if port->sysrq is 1, stm32_usart_console_write() just disable local
> irq response by local_irq_save(), at the time of access register cr1,
> tdr, isr. an TXE interrupt raised, for other cores(I know stm32
> mpu/mcu do not have multi cores, just assume it has), it still has a
> chance to handle interrupt.  Then there is no lock to protect the uart
> register.
> 
> changes to below, should be more safe:
> 
> .....
> if (port->sysrq || oops_in_progress)
>        locked = spin_trylock_irqsave(&port->lock, flags);
> else
>        spin_lock_irqsave(&port->lock, flags);
> 
> ....
> 
> if (locked)
>       spin_unlock_irqrestore(&port->lock, flags);
> 
> For current stm32 soc, it shouldn't happen. just a reminder for future.
> 
> Thanks.
> 
> Dillon
> 
> On Mon, Apr 12, 2021 at 10:04 PM dillon min <dillon.minfei@gmail.com> wrote:
>>
>> Hi Johan,
>>
>> Yes, there is no deadlock. my fault.
>> I forget the local_irq_save() plus spin_lock() is spin_lock_irqsave().
>>
>> Thanks for your review. please ignore this patch.
>>
>> Best regards
>>
>> Dillon
>>
>> On Mon, Apr 12, 2021 at 9:08 PM Johan Hovold <johan@kernel.org> wrote:
>>>
>>> On Mon, Apr 12, 2021 at 05:31:38PM +0800, dillon.minfei@gmail.com wrote:
>>>> From: dillon min <dillon.minfei@gmail.com>
>>>>
>>>> To avoid potential deadlock in spin_lock usage, use spin_lock_irqsave,
>>>> spin_trylock_irqsave(), spin_unlock_irqrestore() in process context.
>>>
>>> This doesn't make much sense as console_write can be called in any
>>> context. And where's the deadlock you claim to be fixing here?
>>>
>>>> remove unused local_irq_save/restore call.
>>>>
>>>> Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
>>>> Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
>>>> Cc: Gerald Baeza <gerald.baeza@foss.st.com>
>>>> Cc: Erwan Le Ray <erwan.leray@foss.st.com>
>>>> Reported-by: kernel test robot <lkp@intel.com>
>>>> Signed-off-by: dillon min <dillon.minfei@gmail.com>
>>>> ---
>>>> v2: remove unused code from stm32_usart_threaded_interrupt() according from
>>>>      Greg's review.
>>>>
>>>>   drivers/tty/serial/stm32-usart.c | 8 +++-----
>>>>   1 file changed, 3 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/drivers/tty/serial/stm32-usart.c b/drivers/tty/serial/stm32-usart.c
>>>> index b3675cf25a69..b1ba5e36e36e 100644
>>>> --- a/drivers/tty/serial/stm32-usart.c
>>>> +++ b/drivers/tty/serial/stm32-usart.c
>>>> @@ -1354,13 +1354,12 @@ static void stm32_usart_console_write(struct console *co, const char *s,
>>>>        u32 old_cr1, new_cr1;
>>>>        int locked = 1;
>>>>
>>>> -     local_irq_save(flags);
>>>>        if (port->sysrq)
>>>>                locked = 0;
>>>>        else if (oops_in_progress)
>>>> -             locked = spin_trylock(&port->lock);
>>>> +             locked = spin_trylock_irqsave(&port->lock, flags);
>>>>        else
>>>> -             spin_lock(&port->lock);
>>>> +             spin_lock_irqsave(&port->lock, flags);
>>>>
>>>>        /* Save and disable interrupts, enable the transmitter */
>>>>        old_cr1 = readl_relaxed(port->membase + ofs->cr1);
>>>> @@ -1374,8 +1373,7 @@ static void stm32_usart_console_write(struct console *co, const char *s,
>>>>        writel_relaxed(old_cr1, port->membase + ofs->cr1);
>>>>
>>>>        if (locked)
>>>> -             spin_unlock(&port->lock);
>>>> -     local_irq_restore(flags);
>>>> +             spin_unlock_irqrestore(&port->lock, flags);
>>>>   }
>>>>
>>>>   static int stm32_usart_console_setup(struct console *co, char *options)
>>>
>>> Johan

Re: [PATCH v2] serial: stm32: optimize spin lock usage

[Hua Dillon]

Hi Erwan,

Erwan LE RAY <erwan.leray@foss.st.com> 于2021年4月16日周五 上午1:10写道：
>
> Hi Dillon,
>
> STM32MP151 is mono-core, but both STM32MP153 and STM32MP157 are
> dual-core (see
> https://www.st.com/content/st_com/en/products/microcontrollers-microprocessors/stm32-arm-cortex-mpus.html).
> So your point is fully relevant, thanks.

Thanks.
>
> ST already fixed the same issue in st-asc.c driver in the past (see
> ef49ffd8), because a systematic deadlock was detected with RT kernel.
>
> You proposed a first implementation in your patch, and a second one in
> the discussion. It seems that your initial proposal (ie your V2 patch)
> is the most standard one (implemented in 6 drivers). The second
> implementation is implemented by only 1 company.
>
> It looks that the solution is to avoid locking in the sysrq case and
> trylock in the oops_in_progress case (see detailed analysis in
> 677fe555cbfb1).

Thanks for the detail information. the V2 patch didn't cover this case:

    stm32_usart_threaded_interrupt()
      spin_lock(&port->lock);
      ...
      uart_handle_sysrq_char();
        sysrq_function();
          printk();
            stm32_usart_console_write();
              locked = spin_trylock_irqsave(&port->lock); //better
than no lock(locked = 0) if other uart interrupt coming at this point

Find a same solution on fsl_lpuart.c
commit abf1e0a98083fd0a1069ce68ad8c92bfb97a57db

Thanks.

Best regards
Dillon
>
> So your initial patch looks to the right proposal, but it would be safer
> if Greg could confirm it.
>
> BR, Erwan.
>
>
> On 4/13/21 1:44 AM, dillon min wrote:
> > Hi Johan, Erwan
> >
> > It seems still a bit of a problem in the current version, not deadlock
> > but access register at the same time.
> >
> > For driver , we should consider it running under smp, let's think
> > about it for this case:
> >
> > static void stm32_usart_console_write(struct console *co, const char *s,
> >                                        unsigned int cnt)
> > {
> >           .....
> >           local_irq_save(flags);
> >           if (port->sysrq)
> >                      locked = 0;
> >           .....
> >           access register cr1, tdr, isr
> >           .....
> >
> >           local_irq_restore(flags);
> > }
> >
> > if port->sysrq is 1, stm32_usart_console_write() just disable local
> > irq response by local_irq_save(), at the time of access register cr1,
> > tdr, isr. an TXE interrupt raised, for other cores(I know stm32
> > mpu/mcu do not have multi cores, just assume it has), it still has a
> > chance to handle interrupt.  Then there is no lock to protect the uart
> > register.
> >
> > changes to below, should be more safe:
> >
> > .....
> > if (port->sysrq || oops_in_progress)
> >        locked = spin_trylock_irqsave(&port->lock, flags);
> > else
> >        spin_lock_irqsave(&port->lock, flags);
> >
> > ....
> >
> > if (locked)
> >       spin_unlock_irqrestore(&port->lock, flags);
> >
> > For current stm32 soc, it shouldn't happen. just a reminder for future.
> >
> > Thanks.
> >
> > Dillon
> >
> > On Mon, Apr 12, 2021 at 10:04 PM dillon min <dillon.minfei@gmail.com> wrote:
> >>
> >> Hi Johan,
> >>
> >> Yes, there is no deadlock. my fault.
> >> I forget the local_irq_save() plus spin_lock() is spin_lock_irqsave().
> >>
> >> Thanks for your review. please ignore this patch.
> >>
> >> Best regards
> >>
> >> Dillon
> >>
> >> On Mon, Apr 12, 2021 at 9:08 PM Johan Hovold <johan@kernel.org> wrote:
> >>>
> >>> On Mon, Apr 12, 2021 at 05:31:38PM +0800, dillon.minfei@gmail.com wrote:
> >>>> From: dillon min <dillon.minfei@gmail.com>
> >>>>
> >>>> To avoid potential deadlock in spin_lock usage, use spin_lock_irqsave,
> >>>> spin_trylock_irqsave(), spin_unlock_irqrestore() in process context.
> >>>
> >>> This doesn't make much sense as console_write can be called in any
> >>> context. And where's the deadlock you claim to be fixing here?
> >>>
> >>>> remove unused local_irq_save/restore call.
> >>>>
> >>>> Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
> >>>> Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> >>>> Cc: Gerald Baeza <gerald.baeza@foss.st.com>
> >>>> Cc: Erwan Le Ray <erwan.leray@foss.st.com>
> >>>> Reported-by: kernel test robot <lkp@intel.com>
> >>>> Signed-off-by: dillon min <dillon.minfei@gmail.com>
> >>>> ---
> >>>> v2: remove unused code from stm32_usart_threaded_interrupt() according from
> >>>>      Greg's review.
> >>>>
> >>>>   drivers/tty/serial/stm32-usart.c | 8 +++-----
> >>>>   1 file changed, 3 insertions(+), 5 deletions(-)
> >>>>
> >>>> diff --git a/drivers/tty/serial/stm32-usart.c b/drivers/tty/serial/stm32-usart.c
> >>>> index b3675cf25a69..b1ba5e36e36e 100644
> >>>> --- a/drivers/tty/serial/stm32-usart.c
> >>>> +++ b/drivers/tty/serial/stm32-usart.c
> >>>> @@ -1354,13 +1354,12 @@ static void stm32_usart_console_write(struct console *co, const char *s,
> >>>>        u32 old_cr1, new_cr1;
> >>>>        int locked = 1;
> >>>>
> >>>> -     local_irq_save(flags);
> >>>>        if (port->sysrq)
> >>>>                locked = 0;
> >>>>        else if (oops_in_progress)
> >>>> -             locked = spin_trylock(&port->lock);
> >>>> +             locked = spin_trylock_irqsave(&port->lock, flags);
> >>>>        else
> >>>> -             spin_lock(&port->lock);
> >>>> +             spin_lock_irqsave(&port->lock, flags);
> >>>>
> >>>>        /* Save and disable interrupts, enable the transmitter */
> >>>>        old_cr1 = readl_relaxed(port->membase + ofs->cr1);
> >>>> @@ -1374,8 +1373,7 @@ static void stm32_usart_console_write(struct console *co, const char *s,
> >>>>        writel_relaxed(old_cr1, port->membase + ofs->cr1);
> >>>>
> >>>>        if (locked)
> >>>> -             spin_unlock(&port->lock);
> >>>> -     local_irq_restore(flags);
> >>>> +             spin_unlock_irqrestore(&port->lock, flags);
> >>>>   }
> >>>>
> >>>>   static int stm32_usart_console_setup(struct console *co, char *options)
> >>>
> >>> Johan

Re: [PATCH v2] serial: stm32: optimize spin lock usage

[Johan Hovold]

On Tue, Apr 13, 2021 at 07:44:39AM +0800, dillon min wrote:
> Hi Johan, Erwan
> 
> It seems still a bit of a problem in the current version, not deadlock
> but access register at the same time.
> 
> For driver , we should consider it running under smp, let's think
> about it for this case:
> 
> static void stm32_usart_console_write(struct console *co, const char *s,
>                                       unsigned int cnt)
> {
>          .....
>          local_irq_save(flags);
>          if (port->sysrq)
>                     locked = 0;
>          .....
>          access register cr1, tdr, isr
>          .....
> 
>          local_irq_restore(flags);
> }
> 
> if port->sysrq is 1, stm32_usart_console_write() just disable local
> irq response by local_irq_save(), at the time of access register cr1,
> tdr, isr. an TXE interrupt raised, for other cores(I know stm32
> mpu/mcu do not have multi cores, just assume it has), it still has a
> chance to handle interrupt.  Then there is no lock to protect the uart
> register.

Right, the sysrq handling is a bit of a hack.

> changes to below, should be more safe:
> 
> .....
> if (port->sysrq || oops_in_progress)
>       locked = spin_trylock_irqsave(&port->lock, flags);

Except that the lock debugging code would detect the attempt at
recursive locking here and complain loudly on UP.

If you really want to fix this, we have uart_unlock_and_check_sysrq()
which can be used to defer sysrq processing until the interrupt handler
has released the lock.

> else
>       spin_lock_irqsave(&port->lock, flags);
> 
> ....
> 
> if (locked)
>      spin_unlock_irqrestore(&port->lock, flags);

Johan

Re: [PATCH v2] serial: stm32: optimize spin lock usage

[Johan Hovold]

[ Please avoid top-posting. ]

On Thu, Apr 15, 2021 at 07:09:14PM +0200, Erwan LE RAY wrote:
> Hi Dillon,
> 
> STM32MP151 is mono-core, but both STM32MP153 and STM32MP157 are 
> dual-core (see 
> https://www.st.com/content/st_com/en/products/microcontrollers-microprocessors/stm32-arm-cortex-mpus.html).
> So your point is fully relevant, thanks.
> 
> ST already fixed the same issue in st-asc.c driver in the past (see 
> ef49ffd8), because a systematic deadlock was detected with RT kernel.

That's not the same issue. The above mentioned commit fixed an issue on
*RT* where local_irq_save() should be avoided.

> You proposed a first implementation in your patch, and a second one in 
> the discussion. It seems that your initial proposal (ie your V2 patch) 
> is the most standard one (implemented in 6 drivers). The second 
> implementation is implemented by only 1 company.
> 
> It looks that the solution is to avoid locking in the sysrq case and 
> trylock in the oops_in_progress case (see detailed analysis in 
> 677fe555cbfb1).
>
> So your initial patch looks to the right proposal, but it would be safer 
> if Greg could confirm it.

That would only fix the RT issue (and by making the sysrq one slightly
worse).

Using uart_unlock_and_check_sysrq() would address both issues.

Johan

Re: [PATCH v2] serial: stm32: optimize spin lock usage

[dillon min]

Hi Johan,

On Fri, Apr 16, 2021 at 4:35 PM Johan Hovold <johan@kernel.org> wrote:
>
> On Tue, Apr 13, 2021 at 07:44:39AM +0800, dillon min wrote:
> > Hi Johan, Erwan
> >
> > It seems still a bit of a problem in the current version, not deadlock
> > but access register at the same time.
> >
> > For driver , we should consider it running under smp, let's think
> > about it for this case:
> >
> > static void stm32_usart_console_write(struct console *co, const char *s,
> >                                       unsigned int cnt)
> > {
> >          .....
> >          local_irq_save(flags);
> >          if (port->sysrq)
> >                     locked = 0;
> >          .....
> >          access register cr1, tdr, isr
> >          .....
> >
> >          local_irq_restore(flags);
> > }
> >
> > if port->sysrq is 1, stm32_usart_console_write() just disable local
> > irq response by local_irq_save(), at the time of access register cr1,
> > tdr, isr. an TXE interrupt raised, for other cores(I know stm32
> > mpu/mcu do not have multi cores, just assume it has), it still has a
> > chance to handle interrupt.  Then there is no lock to protect the uart
> > register.
>
> Right, the sysrq handling is a bit of a hack.
>
> > changes to below, should be more safe:
> >
> > .....
> > if (port->sysrq || oops_in_progress)
> >       locked = spin_trylock_irqsave(&port->lock, flags);
>
> Except that the lock debugging code would detect the attempt at
> recursive locking here and complain loudly on UP.
>
> If you really want to fix this, we have uart_unlock_and_check_sysrq()
> which can be used to defer sysrq processing until the interrupt handler
> has released the lock.

Great, uart_unlock_and_check_sysrq() is fit to fix this. you mean make
the flow like below:

    stm32_usart_threaded_interrupt()
      spin_lock(&port->lock);
      uart_unlock_and_check_sysrq(port, flags);
      ...
      uart_prepare_sysrq_char();
          printk();
            stm32_usart_console_write();
              locked = spin_trylock_irqsave(&port->lock); //only
handle oops, normal case

If so, I will submit v3 as you suggested. thanks.

Best regards.
Dillon,
>
> > else
> >       spin_lock_irqsave(&port->lock, flags);
> >
> > ....
> >
> > if (locked)
> >      spin_unlock_irqrestore(&port->lock, flags);
>
> Johan