linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH] test_rhashtable: remove semaphore usage
@ 2018-12-10 21:17 Arnd Bergmann
  2018-12-11  1:29 ` David Miller
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Arnd Bergmann @ 2018-12-10 21:17 UTC (permalink / raw)
  To: Thomas Graf, Herbert Xu
  Cc: Arnd Bergmann, David S. Miller, NeilBrown, Tom Herbert,
	Kees Cook, Paul Blakey, netdev, linux-kernel

This is one of only two files that initialize a semaphore to a negative
value. We don't really need the two semaphores here at all, but can do
the same thing in more conventional and more effient way, by using a
single waitqueue and an atomic thread counter.

This gets us a little bit closer to eliminating classic semaphores from
the kernel. It also fixes a corner case where we fail to continue after
one of the threads fails to start up.

An alternative would be to use a split kthread_create()+wake_up_process()
and completely eliminate the separate synchronization.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
This is part of a longer, untested, series to remove semaphores
from the kernel, please review as such before applying.
---
 lib/test_rhashtable.c | 28 ++++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)

diff --git a/lib/test_rhashtable.c b/lib/test_rhashtable.c
index 18de5ff1255b..12bdea4f6c20 100644
--- a/lib/test_rhashtable.c
+++ b/lib/test_rhashtable.c
@@ -20,11 +20,11 @@
 #include <linux/module.h>
 #include <linux/rcupdate.h>
 #include <linux/rhashtable.h>
-#include <linux/semaphore.h>
 #include <linux/slab.h>
 #include <linux/sched.h>
 #include <linux/random.h>
 #include <linux/vmalloc.h>
+#include <linux/wait.h>
 
 #define MAX_ENTRIES	1000000
 #define TEST_INSERT_FAIL INT_MAX
@@ -112,7 +112,8 @@ static struct rhashtable_params test_rht_params_dup = {
 	.automatic_shrinking = false,
 };
 
-static struct semaphore prestart_sem, startup_sem;
+static atomic_t startup_count;
+static DECLARE_WAIT_QUEUE_HEAD(startup_wait);
 
 static int insert_retry(struct rhashtable *ht, struct test_obj *obj,
                         const struct rhashtable_params params)
@@ -635,8 +636,9 @@ static int threadfunc(void *data)
 	int i, step, err = 0, insert_retries = 0;
 	struct thread_data *tdata = data;
 
-	up(&prestart_sem);
-	if (down_interruptible(&startup_sem))
+	if (atomic_dec_and_test(&startup_count))
+		wake_up(&startup_wait);
+	if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == -1))
 		pr_err("  thread[%d]: down_interruptible failed\n", tdata->id);
 
 	for (i = 0; i < tdata->entries; i++) {
@@ -756,8 +758,7 @@ static int __init test_rht_init(void)
 
 	pr_info("Testing concurrent rhashtable access from %d threads\n",
 	        tcount);
-	sema_init(&prestart_sem, 1 - tcount);
-	sema_init(&startup_sem, 0);
+	atomic_set(&startup_count, tcount);
 	tdata = vzalloc(array_size(tcount, sizeof(struct thread_data)));
 	if (!tdata)
 		return -ENOMEM;
@@ -783,15 +784,18 @@ static int __init test_rht_init(void)
 		tdata[i].objs = objs + i * entries;
 		tdata[i].task = kthread_run(threadfunc, &tdata[i],
 		                            "rhashtable_thrad[%d]", i);
-		if (IS_ERR(tdata[i].task))
+		if (IS_ERR(tdata[i].task)) {
 			pr_err(" kthread_run failed for thread %d\n", i);
-		else
+			atomic_dec(&startup_count);
+		} else {
 			started_threads++;
+		}
 	}
-	if (down_interruptible(&prestart_sem))
-		pr_err("  down interruptible failed\n");
-	for (i = 0; i < tcount; i++)
-		up(&startup_sem);
+	if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == 0))
+		pr_err("  wait_event interruptible failed\n");
+	/* count is 0 now, set it to -1 and wake up all threads together */
+	atomic_dec(&startup_count);
+	wake_up_all(&startup_wait);
 	for (i = 0; i < tcount; i++) {
 		if (IS_ERR(tdata[i].task))
 			continue;
-- 
2.20.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH] test_rhashtable: remove semaphore usage
  2018-12-10 21:17 [PATCH] test_rhashtable: remove semaphore usage Arnd Bergmann
@ 2018-12-11  1:29 ` David Miller
  2018-12-11  5:45 ` Herbert Xu
  2018-12-14 21:25 ` David Miller
  2 siblings, 0 replies; 6+ messages in thread
From: David Miller @ 2018-12-11  1:29 UTC (permalink / raw)
  To: arnd; +Cc: tgraf, herbert, neilb, tom, keescook, paulb, netdev, linux-kernel

From: Arnd Bergmann <arnd@arndb.de>
Date: Mon, 10 Dec 2018 22:17:20 +0100

> This is one of only two files that initialize a semaphore to a negative
> value. We don't really need the two semaphores here at all, but can do
> the same thing in more conventional and more effient way, by using a
> single waitqueue and an atomic thread counter.
> 
> This gets us a little bit closer to eliminating classic semaphores from
> the kernel. It also fixes a corner case where we fail to continue after
> one of the threads fails to start up.
> 
> An alternative would be to use a split kthread_create()+wake_up_process()
> and completely eliminate the separate synchronization.
> 
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> ---
> This is part of a longer, untested, series to remove semaphores
> from the kernel, please review as such before applying.

This looks fine to me, although it seems kinda weird how the
synchronization works in that the N - 1 child threads will
be awoken two times, once when child N decrements the count
to zero and once when the parent decrements the count to -1
which lets us past the wait_event_interruptibel().

Nevertheless it should work just fine and I have no problems
with it.

Want me to apply this to net-next?

Thanks.


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] test_rhashtable: remove semaphore usage
  2018-12-10 21:17 [PATCH] test_rhashtable: remove semaphore usage Arnd Bergmann
  2018-12-11  1:29 ` David Miller
@ 2018-12-11  5:45 ` Herbert Xu
  2018-12-11  7:38   ` Phil Sutter
  2018-12-14 21:25 ` David Miller
  2 siblings, 1 reply; 6+ messages in thread
From: Herbert Xu @ 2018-12-11  5:45 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Thomas Graf, David S. Miller, NeilBrown, Tom Herbert, Kees Cook,
	Paul Blakey, netdev, linux-kernel, Phil Sutter

On Mon, Dec 10, 2018 at 10:17:20PM +0100, Arnd Bergmann wrote:
> This is one of only two files that initialize a semaphore to a negative
> value. We don't really need the two semaphores here at all, but can do
> the same thing in more conventional and more effient way, by using a
> single waitqueue and an atomic thread counter.
> 
> This gets us a little bit closer to eliminating classic semaphores from
> the kernel. It also fixes a corner case where we fail to continue after
> one of the threads fails to start up.
> 
> An alternative would be to use a split kthread_create()+wake_up_process()
> and completely eliminate the separate synchronization.
> 
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> ---
> This is part of a longer, untested, series to remove semaphores
> from the kernel, please review as such before applying.
> ---
>  lib/test_rhashtable.c | 28 ++++++++++++++++------------
>  1 file changed, 16 insertions(+), 12 deletions(-)

This was created by Phil Sutter so I am adding him to the cc list.

> diff --git a/lib/test_rhashtable.c b/lib/test_rhashtable.c
> index 18de5ff1255b..12bdea4f6c20 100644
> --- a/lib/test_rhashtable.c
> +++ b/lib/test_rhashtable.c
> @@ -20,11 +20,11 @@
>  #include <linux/module.h>
>  #include <linux/rcupdate.h>
>  #include <linux/rhashtable.h>
> -#include <linux/semaphore.h>
>  #include <linux/slab.h>
>  #include <linux/sched.h>
>  #include <linux/random.h>
>  #include <linux/vmalloc.h>
> +#include <linux/wait.h>
>  
>  #define MAX_ENTRIES	1000000
>  #define TEST_INSERT_FAIL INT_MAX
> @@ -112,7 +112,8 @@ static struct rhashtable_params test_rht_params_dup = {
>  	.automatic_shrinking = false,
>  };
>  
> -static struct semaphore prestart_sem, startup_sem;
> +static atomic_t startup_count;
> +static DECLARE_WAIT_QUEUE_HEAD(startup_wait);
>  
>  static int insert_retry(struct rhashtable *ht, struct test_obj *obj,
>                          const struct rhashtable_params params)
> @@ -635,8 +636,9 @@ static int threadfunc(void *data)
>  	int i, step, err = 0, insert_retries = 0;
>  	struct thread_data *tdata = data;
>  
> -	up(&prestart_sem);
> -	if (down_interruptible(&startup_sem))
> +	if (atomic_dec_and_test(&startup_count))
> +		wake_up(&startup_wait);
> +	if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == -1))
>  		pr_err("  thread[%d]: down_interruptible failed\n", tdata->id);
>  
>  	for (i = 0; i < tdata->entries; i++) {
> @@ -756,8 +758,7 @@ static int __init test_rht_init(void)
>  
>  	pr_info("Testing concurrent rhashtable access from %d threads\n",
>  	        tcount);
> -	sema_init(&prestart_sem, 1 - tcount);
> -	sema_init(&startup_sem, 0);
> +	atomic_set(&startup_count, tcount);
>  	tdata = vzalloc(array_size(tcount, sizeof(struct thread_data)));
>  	if (!tdata)
>  		return -ENOMEM;
> @@ -783,15 +784,18 @@ static int __init test_rht_init(void)
>  		tdata[i].objs = objs + i * entries;
>  		tdata[i].task = kthread_run(threadfunc, &tdata[i],
>  		                            "rhashtable_thrad[%d]", i);
> -		if (IS_ERR(tdata[i].task))
> +		if (IS_ERR(tdata[i].task)) {
>  			pr_err(" kthread_run failed for thread %d\n", i);
> -		else
> +			atomic_dec(&startup_count);
> +		} else {
>  			started_threads++;
> +		}
>  	}
> -	if (down_interruptible(&prestart_sem))
> -		pr_err("  down interruptible failed\n");
> -	for (i = 0; i < tcount; i++)
> -		up(&startup_sem);
> +	if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == 0))
> +		pr_err("  wait_event interruptible failed\n");
> +	/* count is 0 now, set it to -1 and wake up all threads together */
> +	atomic_dec(&startup_count);
> +	wake_up_all(&startup_wait);
>  	for (i = 0; i < tcount; i++) {
>  		if (IS_ERR(tdata[i].task))
>  			continue;
> -- 
> 2.20.0
> 

-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] test_rhashtable: remove semaphore usage
  2018-12-11  5:45 ` Herbert Xu
@ 2018-12-11  7:38   ` Phil Sutter
  0 siblings, 0 replies; 6+ messages in thread
From: Phil Sutter @ 2018-12-11  7:38 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Arnd Bergmann, Thomas Graf, David S. Miller, NeilBrown,
	Tom Herbert, Kees Cook, Paul Blakey, netdev, linux-kernel

Hi,

On Tue, Dec 11, 2018 at 01:45:52PM +0800, Herbert Xu wrote:
> On Mon, Dec 10, 2018 at 10:17:20PM +0100, Arnd Bergmann wrote:
> > This is one of only two files that initialize a semaphore to a negative
> > value. We don't really need the two semaphores here at all, but can do
> > the same thing in more conventional and more effient way, by using a
> > single waitqueue and an atomic thread counter.
> > 
> > This gets us a little bit closer to eliminating classic semaphores from
> > the kernel. It also fixes a corner case where we fail to continue after
> > one of the threads fails to start up.
> > 
> > An alternative would be to use a split kthread_create()+wake_up_process()
> > and completely eliminate the separate synchronization.
> > 
> > Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> > ---
> > This is part of a longer, untested, series to remove semaphores
> > from the kernel, please review as such before applying.
> > ---
> >  lib/test_rhashtable.c | 28 ++++++++++++++++------------
> >  1 file changed, 16 insertions(+), 12 deletions(-)
> 
> This was created by Phil Sutter so I am adding him to the cc list.

Thanks, I would have missed it otherwise. Just a minor nit:

> > diff --git a/lib/test_rhashtable.c b/lib/test_rhashtable.c
> > index 18de5ff1255b..12bdea4f6c20 100644
> > --- a/lib/test_rhashtable.c
> > +++ b/lib/test_rhashtable.c
[...]
> > @@ -635,8 +636,9 @@ static int threadfunc(void *data)
> >  	int i, step, err = 0, insert_retries = 0;
> >  	struct thread_data *tdata = data;
> >  
> > -	up(&prestart_sem);
> > -	if (down_interruptible(&startup_sem))
> > +	if (atomic_dec_and_test(&startup_count))
> > +		wake_up(&startup_wait);
> > +	if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == -1))
> >  		pr_err("  thread[%d]: down_interruptible failed\n", tdata->id);

The error message should probably be adjusted as well. Apart from that:

Acked-by: Phil Sutter <phil@nwl.cc>

Thanks, Phil

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] test_rhashtable: remove semaphore usage
  2018-12-10 21:17 [PATCH] test_rhashtable: remove semaphore usage Arnd Bergmann
  2018-12-11  1:29 ` David Miller
  2018-12-11  5:45 ` Herbert Xu
@ 2018-12-14 21:25 ` David Miller
  2018-12-16 19:50   ` Arnd Bergmann
  2 siblings, 1 reply; 6+ messages in thread
From: David Miller @ 2018-12-14 21:25 UTC (permalink / raw)
  To: arnd; +Cc: tgraf, herbert, neilb, tom, keescook, paulb, netdev, linux-kernel

From: Arnd Bergmann <arnd@arndb.de>
Date: Mon, 10 Dec 2018 22:17:20 +0100

> @@ -635,8 +636,9 @@ static int threadfunc(void *data)
>  	int i, step, err = 0, insert_retries = 0;
>  	struct thread_data *tdata = data;
>  
> -	up(&prestart_sem);
> -	if (down_interruptible(&startup_sem))
> +	if (atomic_dec_and_test(&startup_count))
> +		wake_up(&startup_wait);
> +	if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == -1))
>  		pr_err("  thread[%d]: down_interruptible failed\n", tdata->id);

Arnd, please adjust this pr_err() text to match the new code as pointed out by
Phil Sutter.

Thank you.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] test_rhashtable: remove semaphore usage
  2018-12-14 21:25 ` David Miller
@ 2018-12-16 19:50   ` Arnd Bergmann
  0 siblings, 0 replies; 6+ messages in thread
From: Arnd Bergmann @ 2018-12-16 19:50 UTC (permalink / raw)
  To: David Miller
  Cc: Thomas Graf, Herbert Xu, NeilBrown, Tom Herbert, Kees Cook,
	Paul Blakey, Networking, Linux Kernel Mailing List

On Fri, Dec 14, 2018 at 10:25 PM David Miller <davem@davemloft.net> wrote:
>
> From: Arnd Bergmann <arnd@arndb.de>
> Date: Mon, 10 Dec 2018 22:17:20 +0100
>
> > @@ -635,8 +636,9 @@ static int threadfunc(void *data)
> >       int i, step, err = 0, insert_retries = 0;
> >       struct thread_data *tdata = data;
> >
> > -     up(&prestart_sem);
> > -     if (down_interruptible(&startup_sem))
> > +     if (atomic_dec_and_test(&startup_count))
> > +             wake_up(&startup_wait);
> > +     if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == -1))
> >               pr_err("  thread[%d]: down_interruptible failed\n", tdata->id);
>
> Arnd, please adjust this pr_err() text to match the new code as pointed out by
> Phil Sutter.

Done, and resent as v2. I also noticed that the version I sent did not apply
cleanly on current kernels, so I fixed that, and also added a 'goto out' in
the kthread after printing the error message, so an interrupted thread would
exist straight away.

         Arnd

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2018-12-16 19:50 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-12-10 21:17 [PATCH] test_rhashtable: remove semaphore usage Arnd Bergmann
2018-12-11  1:29 ` David Miller
2018-12-11  5:45 ` Herbert Xu
2018-12-11  7:38   ` Phil Sutter
2018-12-14 21:25 ` David Miller
2018-12-16 19:50   ` Arnd Bergmann

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).