[PATCH] rwsem-spinlock: Fix EINTR branch in __down_write_common()

[PATCH] rwsem-spinlock: Fix EINTR branch in __down_write_common()

[Kirill Tkhai]

If a writer could been woken up, the above branch

	if (sem->count == 0)
		break;

would have moved us to taking the sem. So, it's
not the time to wake a writer now, and only readers
are allowed now. Thus, 0 must be passed to __rwsem_do_wake().

Next, __rwsem_do_wake() wakes readers unconditionally.
But we mustn't do that if the sem is owned by writer
in the moment. Otherwise, writer and reader own the sem
the same time, which leads to memory corruption in
callers.

rwsem-xadd.c does not need that, as:
1)the similar check is made lockless there,
2)in __rwsem_mark_wake::try_reader_grant we test,
that sem is not owned by writer.

Fixes: 17fcbd590d0c "locking/rwsem: Fix down_write_killable() for CONFIG_RWSEM_GENERIC_SPINLOCK=y"
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
CC: Niklas Cassel <niklas.cassel@axis.com>
CC: Peter Zijlstra (Intel) <peterz@infradead.org>
CC: Ingo Molnar <mingo@kernel.org>
---
 kernel/locking/rwsem-spinlock.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/locking/rwsem-spinlock.c b/kernel/locking/rwsem-spinlock.c
index c65f7989f850..20819df98125 100644
--- a/kernel/locking/rwsem-spinlock.c
+++ b/kernel/locking/rwsem-spinlock.c
@@ -231,8 +231,8 @@ int __sched __down_write_common(struct rw_semaphore *sem, int state)
 
 out_nolock:
 	list_del(&waiter.list);
-	if (!list_empty(&sem->wait_list))
-		__rwsem_do_wake(sem, 1);
+	if (!list_empty(&sem->wait_list) && sem->count >= 0)
+		__rwsem_do_wake(sem, 0);
 	raw_spin_unlock_irqrestore(&sem->wait_lock, flags);
 
 	return -EINTR;

Re: [PATCH] rwsem-spinlock: Fix EINTR branch in __down_write_common()

[Niklas Cassel]

Good catch!

This should go to -stable as well.


Perhaps

if (!list_empty(&sem->wait_list) && sem->count > 0)
	__rwsem_do_wake(sem, 0);

Rather than

if (!list_empty(&sem->wait_list) && sem->count >= 0)
	__rwsem_do_wake(sem, 0);

Since we have the spinlock, and since we just checked
if sem->count == 0, we still know that it can't be 0.

Either way:

Acked-by: Niklas Cassel <niklas.cassel@axis.com>

On 06/16/2017 03:44 PM, Kirill Tkhai wrote:
> If a writer could been woken up, the above branch
> 
> 	if (sem->count == 0)
> 		break;
> 
> would have moved us to taking the sem. So, it's
> not the time to wake a writer now, and only readers
> are allowed now. Thus, 0 must be passed to __rwsem_do_wake().
> 
> Next, __rwsem_do_wake() wakes readers unconditionally.
> But we mustn't do that if the sem is owned by writer
> in the moment. Otherwise, writer and reader own the sem
> the same time, which leads to memory corruption in
> callers.
> 
> rwsem-xadd.c does not need that, as:
> 1)the similar check is made lockless there,
> 2)in __rwsem_mark_wake::try_reader_grant we test,
> that sem is not owned by writer.
> 
> Fixes: 17fcbd590d0c "locking/rwsem: Fix down_write_killable() for CONFIG_RWSEM_GENERIC_SPINLOCK=y"
> Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
> CC: Niklas Cassel <niklas.cassel@axis.com>
> CC: Peter Zijlstra (Intel) <peterz@infradead.org>
> CC: Ingo Molnar <mingo@kernel.org>
> ---
>  kernel/locking/rwsem-spinlock.c |    4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/locking/rwsem-spinlock.c b/kernel/locking/rwsem-spinlock.c
> index c65f7989f850..20819df98125 100644
> --- a/kernel/locking/rwsem-spinlock.c
> +++ b/kernel/locking/rwsem-spinlock.c
> @@ -231,8 +231,8 @@ int __sched __down_write_common(struct rw_semaphore *sem, int state)
>  
>  out_nolock:
>  	list_del(&waiter.list);
> -	if (!list_empty(&sem->wait_list))
> -		__rwsem_do_wake(sem, 1);
> +	if (!list_empty(&sem->wait_list) && sem->count >= 0)
> +		__rwsem_do_wake(sem, 0);
>  	raw_spin_unlock_irqrestore(&sem->wait_lock, flags);
>  
>  	return -EINTR;
> 

Re: [PATCH] rwsem-spinlock: Fix EINTR branch in __down_write_common()

[Kirill Tkhai]

On Wed, Jun 28, 2017 at 15:20, Niklas Cassel wrote:
> Good catch!
> 
> This should go to -stable as well.
> 
> 
> Perhaps
> 
> if (!list_empty(&sem->wait_list) && sem->count > 0)
> 	__rwsem_do_wake(sem, 0);
> 
> Rather than
> 
> if (!list_empty(&sem->wait_list) && sem->count >= 0)
> 	__rwsem_do_wake(sem, 0);
> 
> Since we have the spinlock, and since we just checked
> if sem->count == 0, we still know that it can't be 0.
> 
> Either way:
> 
> Acked-by: Niklas Cassel <niklas.cassel@axis.com>
> 

[PATCH]rwsem-spinlock: Fix EINTR branch in __down_write_common()

If a writer could been woken up, the above branch

    if (sem->count == 0)
	    break;

would have moved us to taking the sem. So, it's
not the time to wake a writer now, and only readers
are allowed now. Thus, 0 must be passed to __rwsem_do_wake().

Next, __rwsem_do_wake() wakes readers unconditionally.
But we mustn't do that if the sem is owned by writer
in the moment. Otherwise, writer and reader own the sem
the same time, which leads to memory corruption in
callers.

rwsem-xadd.c does not need that, as:
1)the similar check is made lockless there,
2)in __rwsem_mark_wake::try_reader_grant we test,
that sem is not owned by writer.

Fixes: 17fcbd590d0c "locking/rwsem: Fix down_write_killable() for CONFIG_RWSEM_GENERIC_SPINLOCK=y"
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Niklas Cassel <niklas.cassel@axis.com>
CC: Peter Zijlstra (Intel) <peterz@infradead.org>
CC: Ingo Molnar <mingo@kernel.org>
---
 kernel/locking/rwsem-spinlock.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/locking/rwsem-spinlock.c b/kernel/locking/rwsem-spinlock.c
index c65f7989f850..3e6feccb8b56 100644
--- a/kernel/locking/rwsem-spinlock.c
+++ b/kernel/locking/rwsem-spinlock.c
@@ -231,8 +231,8 @@ int __sched __down_write_common(struct rw_semaphore *sem, int state)
 
 out_nolock:
 	list_del(&waiter.list);
-	if (!list_empty(&sem->wait_list))
-		__rwsem_do_wake(sem, 1);
+	if (!list_empty(&sem->wait_list) && sem->count > 0)
+		__rwsem_do_wake(sem, 0);
 	raw_spin_unlock_irqrestore(&sem->wait_lock, flags);
 
 	return -EINTR;

[tip:locking/urgent] locking/rwsem-spinlock: Fix EINTR branch in __down_write_common()

[tip-bot for Kirill Tkhai]

Commit-ID:  a0c4acd2c220376b4e9690e75782d0c0afdaab9f
Gitweb:     http://git.kernel.org/tip/a0c4acd2c220376b4e9690e75782d0c0afdaab9f
Author:     Kirill Tkhai <ktkhai@virtuozzo.com>
AuthorDate: Fri, 16 Jun 2017 16:44:34 +0300
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Wed, 5 Jul 2017 12:26:29 +0200

locking/rwsem-spinlock: Fix EINTR branch in __down_write_common()

If a writer could been woken up, the above branch

	if (sem->count == 0)
		break;

would have moved us to taking the sem. So, it's
not the time to wake a writer now, and only readers
are allowed now. Thus, 0 must be passed to __rwsem_do_wake().

Next, __rwsem_do_wake() wakes readers unconditionally.
But we mustn't do that if the sem is owned by writer
in the moment. Otherwise, writer and reader own the sem
the same time, which leads to memory corruption in
callers.

rwsem-xadd.c does not need that, as:

  1) the similar check is made lockless there,
  2) in __rwsem_mark_wake::try_reader_grant we test,

that sem is not owned by writer.

Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: <stable@vger.kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Niklas Cassel <niklas.cassel@axis.com>
Cc: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Fixes: 17fcbd590d0c "locking/rwsem: Fix down_write_killable() for CONFIG_RWSEM_GENERIC_SPINLOCK=y"
Link: http://lkml.kernel.org/r/149762063282.19811.9129615532201147826.stgit@localhost.localdomain
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 kernel/locking/rwsem-spinlock.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/locking/rwsem-spinlock.c b/kernel/locking/rwsem-spinlock.c
index c65f798..20819df 100644
--- a/kernel/locking/rwsem-spinlock.c
+++ b/kernel/locking/rwsem-spinlock.c
@@ -231,8 +231,8 @@ int __sched __down_write_common(struct rw_semaphore *sem, int state)
 
 out_nolock:
 	list_del(&waiter.list);
-	if (!list_empty(&sem->wait_list))
-		__rwsem_do_wake(sem, 1);
+	if (!list_empty(&sem->wait_list) && sem->count >= 0)
+		__rwsem_do_wake(sem, 0);
 	raw_spin_unlock_irqrestore(&sem->wait_lock, flags);
 
 	return -EINTR;

Re: [tip:locking/urgent] locking/rwsem-spinlock: Fix EINTR branch in __down_write_common()

[Niklas Cassel]

On 07/05/2017 04:27 PM, tip-bot for Kirill Tkhai wrote:
> Commit-ID:  a0c4acd2c220376b4e9690e75782d0c0afdaab9f
> Gitweb:     http://git.kernel.org/tip/a0c4acd2c220376b4e9690e75782d0c0afdaab9f
> Author:     Kirill Tkhai <ktkhai@virtuozzo.com>
> AuthorDate: Fri, 16 Jun 2017 16:44:34 +0300
> Committer:  Ingo Molnar <mingo@kernel.org>
> CommitDate: Wed, 5 Jul 2017 12:26:29 +0200
> 
> locking/rwsem-spinlock: Fix EINTR branch in __down_write_common()
> 
> If a writer could been woken up, the above branch
> 
> 	if (sem->count == 0)
> 		break;
> 
> would have moved us to taking the sem. So, it's
> not the time to wake a writer now, and only readers
> are allowed now. Thus, 0 must be passed to __rwsem_do_wake().
> 
> Next, __rwsem_do_wake() wakes readers unconditionally.
> But we mustn't do that if the sem is owned by writer
> in the moment. Otherwise, writer and reader own the sem
> the same time, which leads to memory corruption in
> callers.
> 
> rwsem-xadd.c does not need that, as:
> 
>   1) the similar check is made lockless there,
>   2) in __rwsem_mark_wake::try_reader_grant we test,
> 
> that sem is not owned by writer.
> 
> Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
> Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Cc: <stable@vger.kernel.org>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Niklas Cassel <niklas.cassel@axis.com>
> Cc: Peter Zijlstra (Intel) <peterz@infradead.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Fixes: 17fcbd590d0c "locking/rwsem: Fix down_write_killable() for CONFIG_RWSEM_GENERIC_SPINLOCK=y"
> Link: http://lkml.kernel.org/r/149762063282.19811.9129615532201147826.stgit@localhost.localdomain
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
> ---
>  kernel/locking/rwsem-spinlock.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/locking/rwsem-spinlock.c b/kernel/locking/rwsem-spinlock.c
> index c65f798..20819df 100644
> --- a/kernel/locking/rwsem-spinlock.c
> +++ b/kernel/locking/rwsem-spinlock.c
> @@ -231,8 +231,8 @@ int __sched __down_write_common(struct rw_semaphore *sem, int state)
>  
>  out_nolock:
>  	list_del(&waiter.list);
> -	if (!list_empty(&sem->wait_list))
> -		__rwsem_do_wake(sem, 1);
> +	if (!list_empty(&sem->wait_list) && sem->count >= 0)
> +		__rwsem_do_wake(sem, 0);
>  	raw_spin_unlock_irqrestore(&sem->wait_lock, flags);
>  
>  	return -EINTR;
> 

For the record, there is actually a v2 of this:

http://marc.info/?l=linux-kernel&m=149866422128912


Regards,
Niklas

Re: [tip:locking/urgent] locking/rwsem-spinlock: Fix EINTR branch in __down_write_common()

[Ingo Molnar]

* Niklas Cassel <niklas.cassel@axis.com> wrote:

> On 07/05/2017 04:27 PM, tip-bot for Kirill Tkhai wrote:
> > Commit-ID:  a0c4acd2c220376b4e9690e75782d0c0afdaab9f
> > Gitweb:     http://git.kernel.org/tip/a0c4acd2c220376b4e9690e75782d0c0afdaab9f
> > Author:     Kirill Tkhai <ktkhai@virtuozzo.com>
> > AuthorDate: Fri, 16 Jun 2017 16:44:34 +0300
> > Committer:  Ingo Molnar <mingo@kernel.org>
> > CommitDate: Wed, 5 Jul 2017 12:26:29 +0200
> > 
> > locking/rwsem-spinlock: Fix EINTR branch in __down_write_common()
> > 
> > If a writer could been woken up, the above branch
> > 
> > 	if (sem->count == 0)
> > 		break;
> > 
> > would have moved us to taking the sem. So, it's
> > not the time to wake a writer now, and only readers
> > are allowed now. Thus, 0 must be passed to __rwsem_do_wake().
> > 
> > Next, __rwsem_do_wake() wakes readers unconditionally.
> > But we mustn't do that if the sem is owned by writer
> > in the moment. Otherwise, writer and reader own the sem
> > the same time, which leads to memory corruption in
> > callers.
> > 
> > rwsem-xadd.c does not need that, as:
> > 
> >   1) the similar check is made lockless there,
> >   2) in __rwsem_mark_wake::try_reader_grant we test,
> > 
> > that sem is not owned by writer.
> > 
> > Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
> > Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > Cc: <stable@vger.kernel.org>
> > Cc: Linus Torvalds <torvalds@linux-foundation.org>
> > Cc: Niklas Cassel <niklas.cassel@axis.com>
> > Cc: Peter Zijlstra (Intel) <peterz@infradead.org>
> > Cc: Peter Zijlstra <peterz@infradead.org>
> > Cc: Thomas Gleixner <tglx@linutronix.de>
> > Fixes: 17fcbd590d0c "locking/rwsem: Fix down_write_killable() for CONFIG_RWSEM_GENERIC_SPINLOCK=y"
> > Link: http://lkml.kernel.org/r/149762063282.19811.9129615532201147826.stgit@localhost.localdomain
> > Signed-off-by: Ingo Molnar <mingo@kernel.org>
> > ---
> >  kernel/locking/rwsem-spinlock.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/kernel/locking/rwsem-spinlock.c b/kernel/locking/rwsem-spinlock.c
> > index c65f798..20819df 100644
> > --- a/kernel/locking/rwsem-spinlock.c
> > +++ b/kernel/locking/rwsem-spinlock.c
> > @@ -231,8 +231,8 @@ int __sched __down_write_common(struct rw_semaphore *sem, int state)
> >  
> >  out_nolock:
> >  	list_del(&waiter.list);
> > -	if (!list_empty(&sem->wait_list))
> > -		__rwsem_do_wake(sem, 1);
> > +	if (!list_empty(&sem->wait_list) && sem->count >= 0)
> > +		__rwsem_do_wake(sem, 0);
> >  	raw_spin_unlock_irqrestore(&sem->wait_lock, flags);
> >  
> >  	return -EINTR;
> > 
> 
> For the record, there is actually a v2 of this:
> 
> http://marc.info/?l=linux-kernel&m=149866422128912

Hm, so I missed that because it was within the discussion - please post v2 patches 
with a new subject line next time around.

But I also disagree with -v2 mildly: in practice a >= test has the same CPU 
overhead as a > test, and if we rely on the earlier "sem->count == 0" test then we 
should also comment on that.

It's more straightforward to just do the canonical sem->count >= 0 test that we do 
elsewhere in the rwsem-spinlock code.

PeterZ, what's your preference?

Thanks,

	Ingo

Re: [tip:locking/urgent] locking/rwsem-spinlock: Fix EINTR branch in __down_write_common()

[Peter Zijlstra]

On Thu, Jul 06, 2017 at 09:28:58AM +0200, Ingo Molnar wrote:
> It's more straightforward to just do the canonical sem->count >= 0 test that we do 
> elsewhere in the rwsem-spinlock code.
> 
> PeterZ, what's your preference?

Leave it as is.. it doesn't matter (the 0 case shouldn't happen) and as
you say >= 0 is what most other code does.