From: "Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com> To: paulus@samba.org, oleg@redhat.com, rusty@rustcorp.com.au, peterz@infradead.org, tglx@linutronix.de, akpm@linux-foundation.org Cc: mingo@kernel.org, paulmck@linux.vnet.ibm.com, tj@kernel.org, walken@google.com, ego@linux.vnet.ibm.com, linux@arm.linux.org.uk, linux-kernel@vger.kernel.org, srivatsa.bhat@linux.vnet.ibm.com, Robert Love <robert.w.love@intel.com>, "James E.J. Bottomley" <JBottomley@parallels.com>, fcoe-devel@open-fcoe.org, linux-scsi@vger.kernel.org, "Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com> Subject: [PATCH 33/51] scsi, fcoe: Fix CPU hotplug callback registration Date: Thu, 06 Feb 2014 03:40:34 +0530 [thread overview] Message-ID: <20140205221034.19080.28417.stgit@srivatsabhat.in.ibm.com> (raw) In-Reply-To: <20140205220251.19080.92336.stgit@srivatsabhat.in.ibm.com> Subsystems that want to register CPU hotplug callbacks, as well as perform initialization for the CPUs that are already online, often do it as shown below: get_online_cpus(); for_each_online_cpu(cpu) init_cpu(cpu); register_cpu_notifier(&foobar_cpu_notifier); put_online_cpus(); This is wrong, since it is prone to ABBA deadlocks involving the cpu_add_remove_lock and the cpu_hotplug.lock (when running concurrently with CPU hotplug operations). Instead, the correct and race-free way of performing the callback registration is: cpu_maps_update_begin(); for_each_online_cpu(cpu) init_cpu(cpu); /* Note the use of the double underscored version of the API */ __register_cpu_notifier(&foobar_cpu_notifier); cpu_maps_update_done(); Fix the fcoe code in scsi by using this latter form of callback registration. Cc: Robert Love <robert.w.love@intel.com> Cc: "James E.J. Bottomley" <JBottomley@parallels.com> Cc: fcoe-devel@open-fcoe.org Cc: linux-scsi@vger.kernel.org Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com> --- drivers/scsi/fcoe/fcoe.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/fcoe/fcoe.c b/drivers/scsi/fcoe/fcoe.c index f317000..1c299de 100644 --- a/drivers/scsi/fcoe/fcoe.c +++ b/drivers/scsi/fcoe/fcoe.c @@ -2633,14 +2633,18 @@ static int __init fcoe_init(void) skb_queue_head_init(&p->fcoe_rx_list); } + cpu_maps_update_begin(); + for_each_online_cpu(cpu) fcoe_percpu_thread_create(cpu); /* Initialize per CPU interrupt thread */ - rc = register_hotcpu_notifier(&fcoe_cpu_notifier); + rc = __register_hotcpu_notifier(&fcoe_cpu_notifier); if (rc) goto out_free; + cpu_maps_update_done(); + /* Setup link change notification */ fcoe_dev_setup(); @@ -2655,6 +2659,9 @@ out_free: for_each_online_cpu(cpu) { fcoe_percpu_thread_destroy(cpu); } + + cpu_maps_update_done(); + mutex_unlock(&fcoe_config_mutex); destroy_workqueue(fcoe_wq); return rc; @@ -2687,11 +2694,15 @@ static void __exit fcoe_exit(void) } rtnl_unlock(); - unregister_hotcpu_notifier(&fcoe_cpu_notifier); + cpu_maps_update_begin(); for_each_online_cpu(cpu) fcoe_percpu_thread_destroy(cpu); + __unregister_hotcpu_notifier(&fcoe_cpu_notifier); + + cpu_maps_update_done(); + mutex_unlock(&fcoe_config_mutex); /*
WARNING: multiple messages have this Message-ID (diff)
From: "Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com> To: paulus@samba.org, oleg@redhat.com, rusty@rustcorp.com.au, peterz@infradead.org, tglx@linutronix.de, akpm@linux-foundation.org Cc: mingo@kernel.org, paulmck@linux.vnet.ibm.com, tj@kernel.org, walken@google.com, ego@linux.vnet.ibm.com, linux@arm.linux.org.uk, linux-kernel@vger.kernel.org, srivatsa.bhat@linux.vnet.ibm.com, Robert Love <robert.w.love@intel.com>, "James E.J. Bottomley" <JBottomley@parallels.com>, fcoe-devel@open-fcoe.org, linux-scsi@vger.kernel.org"Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com> Subject: [PATCH 33/51] scsi, fcoe: Fix CPU hotplug callback registration Date: Thu, 06 Feb 2014 03:40:34 +0530 [thread overview] Message-ID: <20140205221034.19080.28417.stgit@srivatsabhat.in.ibm.com> (raw) In-Reply-To: <20140205220251.19080.92336.stgit@srivatsabhat.in.ibm.com> Subsystems that want to register CPU hotplug callbacks, as well as perform initialization for the CPUs that are already online, often do it as shown below: get_online_cpus(); for_each_online_cpu(cpu) init_cpu(cpu); register_cpu_notifier(&foobar_cpu_notifier); put_online_cpus(); This is wrong, since it is prone to ABBA deadlocks involving the cpu_add_remove_lock and the cpu_hotplug.lock (when running concurrently with CPU hotplug operations). Instead, the correct and race-free way of performing the callback registration is: cpu_maps_update_begin(); for_each_online_cpu(cpu) init_cpu(cpu); /* Note the use of the double underscored version of the API */ __register_cpu_notifier(&foobar_cpu_notifier); cpu_maps_update_done(); Fix the fcoe code in scsi by using this latter form of callback registration. Cc: Robert Love <robert.w.love@intel.com> Cc: "James E.J. Bottomley" <JBottomley@parallels.com> Cc: fcoe-devel@open-fcoe.org Cc: linux-scsi@vger.kernel.org Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com> --- drivers/scsi/fcoe/fcoe.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/fcoe/fcoe.c b/drivers/scsi/fcoe/fcoe.c index f317000..1c299de 100644 --- a/drivers/scsi/fcoe/fcoe.c +++ b/drivers/scsi/fcoe/fcoe.c @@ -2633,14 +2633,18 @@ static int __init fcoe_init(void) skb_queue_head_init(&p->fcoe_rx_list); } + cpu_maps_update_begin(); + for_each_online_cpu(cpu) fcoe_percpu_thread_create(cpu); /* Initialize per CPU interrupt thread */ - rc = register_hotcpu_notifier(&fcoe_cpu_notifier); + rc = __register_hotcpu_notifier(&fcoe_cpu_notifier); if (rc) goto out_free; + cpu_maps_update_done(); + /* Setup link change notification */ fcoe_dev_setup(); @@ -2655,6 +2659,9 @@ out_free: for_each_online_cpu(cpu) { fcoe_percpu_thread_destroy(cpu); } + + cpu_maps_update_done(); + mutex_unlock(&fcoe_config_mutex); destroy_workqueue(fcoe_wq); return rc; @@ -2687,11 +2694,15 @@ static void __exit fcoe_exit(void) } rtnl_unlock(); - unregister_hotcpu_notifier(&fcoe_cpu_notifier); + cpu_maps_update_begin(); for_each_online_cpu(cpu) fcoe_percpu_thread_destroy(cpu); + __unregister_hotcpu_notifier(&fcoe_cpu_notifier); + + cpu_maps_update_done(); + mutex_unlock(&fcoe_config_mutex); /*
next prev parent reply other threads:[~2014-02-05 22:16 UTC|newest] Thread overview: 159+ messages / expand[flat|nested] mbox.gz Atom feed top 2014-02-05 22:04 [PATCH 00/51] CPU hotplug: Fix issues with callback registration Srivatsa S. Bhat 2014-02-05 22:04 ` [PATCH 01/51] CPU hotplug: Provide lockless versions of callback registration functions Srivatsa S. Bhat 2014-02-06 18:41 ` Oleg Nesterov 2014-02-07 19:11 ` Gautham R Shenoy 2014-02-10 9:15 ` Srivatsa S. Bhat 2014-02-10 10:51 ` Gautham R Shenoy 2014-02-10 11:11 ` Srivatsa S. Bhat 2014-02-10 12:05 ` Gautham R Shenoy 2014-02-10 13:28 ` Srivatsa S. Bhat 2014-02-10 13:30 ` Srivatsa S. Bhat 2014-02-10 15:30 ` Oleg Nesterov 2014-02-10 17:27 ` Balbir Singh 2014-02-11 1:26 ` Toshi Kani 2014-02-11 9:27 ` Srivatsa S. Bhat 2014-02-11 16:33 ` Toshi Kani 2014-02-11 17:18 ` Gautham R Shenoy 2014-02-11 17:35 ` Toshi Kani 2014-02-11 19:20 ` Srivatsa S. Bhat 2014-02-11 20:51 ` Toshi Kani 2014-02-12 6:18 ` Srivatsa S. Bhat 2014-02-13 10:56 ` Srivatsa S. Bhat 2014-02-13 20:53 ` Toshi Kani 2014-02-11 17:15 ` Oleg Nesterov 2014-02-11 19:08 ` Srivatsa S. Bhat 2014-02-13 17:44 ` Oleg Nesterov 2014-02-13 17:54 ` Srivatsa S. Bhat 2014-02-13 11:06 ` Gautham R Shenoy 2014-02-05 22:04 ` [PATCH 02/51] Doc/cpu-hotplug: Specify race-free way to register CPU hotplug callbacks Srivatsa S. Bhat 2014-02-05 22:05 ` [PATCH 03/51] CPU hotplug, perf: Fix CPU hotplug callback registration Srivatsa S. Bhat 2014-02-05 22:05 ` [PATCH 04/51] ia64, salinfo: Fix " Srivatsa S. Bhat 2014-02-05 22:17 ` Srivatsa S. Bhat 2014-02-05 22:05 ` [PATCH 05/51] ia64, palinfo: Fix CPU " Srivatsa S. Bhat 2014-02-05 22:17 ` Srivatsa S. Bhat 2014-02-05 22:05 ` [PATCH 06/51] ia64, topology: " Srivatsa S. Bhat 2014-02-05 22:17 ` Srivatsa S. Bhat 2014-02-05 22:05 ` [PATCH 07/51] ia64, err-inject: " Srivatsa S. Bhat 2014-02-05 22:17 ` Srivatsa S. Bhat 2014-02-05 22:06 ` [PATCH 08/51] arm, hw-breakpoint: " Srivatsa S. Bhat 2014-02-05 22:06 ` Srivatsa S. Bhat 2014-02-06 10:57 ` Will Deacon 2014-02-06 10:57 ` Will Deacon 2014-02-06 11:25 ` Srivatsa S. Bhat 2014-02-06 11:25 ` Srivatsa S. Bhat 2014-02-06 11:39 ` Will Deacon 2014-02-06 11:39 ` Will Deacon 2014-02-06 11:38 ` Srivatsa S. Bhat 2014-02-06 11:38 ` Srivatsa S. Bhat 2014-02-05 22:06 ` [PATCH 09/51] arm, kvm: " Srivatsa S. Bhat 2014-02-05 22:06 ` Srivatsa S. Bhat 2014-02-05 22:06 ` [PATCH 10/51] s390, cacheinfo: " Srivatsa S. Bhat 2014-02-05 22:06 ` Srivatsa S. Bhat 2014-02-05 22:06 ` [PATCH 11/51] s390, smp: " Srivatsa S. Bhat 2014-02-05 22:06 ` Srivatsa S. Bhat 2014-02-05 22:06 ` [PATCH 12/51] sparc, sysfs: " Srivatsa S. Bhat 2014-02-05 22:18 ` Srivatsa S. Bhat 2014-02-05 22:06 ` [PATCH 13/51] powerpc, " Srivatsa S. Bhat 2014-02-05 22:06 ` Srivatsa S. Bhat 2014-02-14 6:47 ` Madhavan Srinivasan 2014-02-14 6:47 ` Madhavan Srinivasan 2014-02-05 22:07 ` [PATCH 14/51] x86, msr: " Srivatsa S. Bhat 2014-02-05 22:07 ` [PATCH 15/51] x86, cpuid: " Srivatsa S. Bhat 2014-02-05 22:07 ` [PATCH 16/51] x86, vsyscall: " Srivatsa S. Bhat 2014-02-10 18:50 ` Gautham R Shenoy 2014-02-11 6:58 ` Srivatsa S. Bhat 2014-02-05 22:07 ` [PATCH 17/51] x86, intel, uncore: " Srivatsa S. Bhat 2014-02-05 22:07 ` [PATCH 18/51] x86, mce: " Srivatsa S. Bhat 2014-02-05 22:08 ` [PATCH 19/51] x86, therm_throt.c: " Srivatsa S. Bhat 2014-02-10 15:53 ` Oleg Nesterov 2014-02-10 17:29 ` Srivatsa S. Bhat 2014-02-10 18:04 ` Srivatsa S. Bhat 2014-02-05 22:08 ` [PATCH 20/51] x86, amd, ibs: " Srivatsa S. Bhat 2014-02-05 22:08 ` [PATCH 21/51] x86, intel, cacheinfo: " Srivatsa S. Bhat 2014-02-05 22:08 ` [PATCH 22/51] x86, intel, rapl: " Srivatsa S. Bhat 2014-02-05 22:08 ` [PATCH 23/51] x86, amd, uncore: " Srivatsa S. Bhat 2014-02-05 22:09 ` [PATCH 24/51] x86, hpet: " Srivatsa S. Bhat 2014-02-10 18:58 ` Gautham R Shenoy 2014-02-11 6:59 ` Srivatsa S. Bhat 2014-02-05 22:09 ` [PATCH 25/51] x86, pci, amd-bus: " Srivatsa S. Bhat 2014-02-05 22:09 ` [PATCH 26/51] x86, oprofile, nmi: " Srivatsa S. Bhat 2014-02-10 19:07 ` Gautham R Shenoy 2014-02-10 19:27 ` Gautham R Shenoy 2014-02-11 7:01 ` Srivatsa S. Bhat 2014-02-05 22:09 ` [PATCH 27/51] x86, kvm: " Srivatsa S. Bhat 2014-02-05 22:09 ` [PATCH 28/51] arm64, hw_breakpoint.c: " Srivatsa S. Bhat 2014-02-05 22:09 ` Srivatsa S. Bhat 2014-02-06 11:41 ` Will Deacon 2014-02-06 11:41 ` Will Deacon 2014-02-05 22:09 ` [PATCH 29/51] arm64, debug-monitors: " Srivatsa S. Bhat 2014-02-05 22:09 ` Srivatsa S. Bhat 2014-02-06 11:41 ` Will Deacon 2014-02-06 11:41 ` Will Deacon 2014-02-05 22:10 ` [PATCH 30/51] powercap, intel-rapl: " Srivatsa S. Bhat 2014-02-05 22:10 ` [PATCH 31/51] scsi, bnx2i: " Srivatsa S. Bhat 2014-02-05 22:10 ` Srivatsa S. Bhat 2014-02-05 22:10 ` [PATCH 32/51] scsi, bnx2fc: " Srivatsa S. Bhat 2014-02-05 22:10 ` Srivatsa S. Bhat 2014-02-05 22:10 ` Srivatsa S. Bhat [this message] 2014-02-05 22:10 ` [PATCH 33/51] scsi, fcoe: " Srivatsa S. Bhat 2014-02-05 22:10 ` [PATCH 34/51] zsmalloc: " Srivatsa S. Bhat 2014-02-05 22:10 ` Srivatsa S. Bhat 2014-02-05 22:10 ` [PATCH 35/51] acpi-cpufreq: " Srivatsa S. Bhat 2014-02-05 22:10 ` Srivatsa S. Bhat 2014-02-06 12:43 ` Rafael J. Wysocki 2014-02-06 16:05 ` Srivatsa S. Bhat 2014-02-07 4:09 ` Viresh Kumar 2014-02-05 22:11 ` [PATCH 36/51] drivers/base/topology.c: " Srivatsa S. Bhat 2014-02-05 22:11 ` [PATCH 37/51] clocksource, dummy-timer: " Srivatsa S. Bhat 2014-02-05 22:11 ` [PATCH 38/51] intel-idle: " Srivatsa S. Bhat 2014-02-05 22:11 ` Srivatsa S. Bhat 2014-02-06 12:43 ` Rafael J. Wysocki 2014-02-06 16:04 ` Srivatsa S. Bhat 2014-02-05 22:11 ` [PATCH 39/51] oprofile, nmi-timer: " Srivatsa S. Bhat 2014-02-05 22:11 ` [PATCH 40/51] octeon, watchdog: " Srivatsa S. Bhat 2014-02-05 22:11 ` [PATCH 41/51] thermal, x86-pkg-temp: " Srivatsa S. Bhat 2014-02-05 22:11 ` Srivatsa S. Bhat 2014-02-05 22:12 ` [PATCH 42/51] hwmon, coretemp: " Srivatsa S. Bhat 2014-02-05 22:24 ` [lm-sensors] " Srivatsa S. Bhat 2014-02-06 0:44 ` Guenter Roeck 2014-02-06 0:44 ` [lm-sensors] " Guenter Roeck 2014-02-06 1:25 ` Guenter Roeck 2014-02-06 1:25 ` [lm-sensors] " Guenter Roeck 2014-02-06 10:03 ` Srivatsa S. Bhat 2014-02-06 10:15 ` [lm-sensors] " Srivatsa S. Bhat 2014-02-05 22:12 ` [PATCH 43/51] hwmon, via-cputemp: " Srivatsa S. Bhat 2014-02-05 22:24 ` [lm-sensors] " Srivatsa S. Bhat 2014-02-06 0:44 ` Guenter Roeck 2014-02-06 0:44 ` [lm-sensors] " Guenter Roeck 2014-02-06 1:26 ` Guenter Roeck 2014-02-06 1:26 ` [lm-sensors] " Guenter Roeck 2014-02-05 22:12 ` [PATCH 44/51] xen, balloon: " Srivatsa S. Bhat 2014-02-05 22:12 ` Srivatsa S. Bhat 2014-02-05 22:12 ` [PATCH 45/51] md, raid5: " Srivatsa S. Bhat 2014-02-05 22:12 ` Srivatsa S. Bhat 2014-02-06 1:11 ` NeilBrown 2014-02-06 10:05 ` Srivatsa S. Bhat 2014-02-06 18:43 ` Oleg Nesterov 2014-02-05 22:12 ` [PATCH 46/51] trace, ring-buffer: " Srivatsa S. Bhat 2014-02-05 23:41 ` Steven Rostedt 2014-02-05 22:13 ` [PATCH 47/51] profile: " Srivatsa S. Bhat 2014-02-05 22:13 ` [PATCH 48/51] mm, vmstat: " Srivatsa S. Bhat 2014-02-05 22:13 ` Srivatsa S. Bhat 2014-02-06 15:35 ` Christoph Lameter 2014-02-06 15:35 ` Christoph Lameter 2014-02-07 2:52 ` Yasuaki Ishimatsu 2014-02-07 2:52 ` Yasuaki Ishimatsu 2014-02-05 22:13 ` [PATCH 49/51] mm, zswap: " Srivatsa S. Bhat 2014-02-05 22:13 ` Srivatsa S. Bhat 2014-02-05 22:13 ` [PATCH 50/51] net/core/flow.c: " Srivatsa S. Bhat 2014-02-07 4:39 ` David Miller 2014-02-07 5:19 ` David Miller 2014-02-05 22:13 ` [PATCH 51/51] net/iucv/iucv.c: " Srivatsa S. Bhat 2014-02-05 22:13 ` Srivatsa S. Bhat 2014-02-07 4:39 ` David Miller 2014-02-07 5:19 ` David Miller 2014-02-06 9:38 ` [PATCH 00/51] CPU hotplug: Fix issues with " Gautham R Shenoy 2014-02-06 11:04 ` Srivatsa S. Bhat 2014-02-06 11:08 ` Srivatsa S. Bhat 2014-02-06 12:14 ` Gautham R Shenoy 2014-02-06 16:09 ` Srivatsa S. Bhat
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20140205221034.19080.28417.stgit@srivatsabhat.in.ibm.com \ --to=srivatsa.bhat@linux.vnet.ibm.com \ --cc=JBottomley@parallels.com \ --cc=akpm@linux-foundation.org \ --cc=ego@linux.vnet.ibm.com \ --cc=fcoe-devel@open-fcoe.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-scsi@vger.kernel.org \ --cc=linux@arm.linux.org.uk \ --cc=mingo@kernel.org \ --cc=oleg@redhat.com \ --cc=paulmck@linux.vnet.ibm.com \ --cc=paulus@samba.org \ --cc=peterz@infradead.org \ --cc=robert.w.love@intel.com \ --cc=rusty@rustcorp.com.au \ --cc=tglx@linutronix.de \ --cc=tj@kernel.org \ --cc=walken@google.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.