[PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Srivatsa S. Bhat]

Hi,

This patchset removes CPU hotplug's dependence on stop_machine() from the CPU
offline path and provides an alternative (set of APIs) to preempt_disable() to
prevent CPUs from going offline, which can be invoked from atomic context.
The motivation behind the removal of stop_machine() is to avoid its ill-effects
and thus improve the design of CPU hotplug. (More description regarding this
is available in the patches).

All the users of preempt_disable()/local_irq_disable() who used to use it to
prevent CPU offline, have been converted to the new primitives introduced in the
patchset. Also, the CPU_DYING notifiers have been audited to check whether
they can cope up with the removal of stop_machine() or whether they need to
use new locks for synchronization (all CPU_DYING notifiers looked OK, without
the need for any new locks).

Applies on v3.8-rc4. It currently has some locking issues with cpu idle (on
which even lockdep didn't provide any insight unfortunately). So for now, it
works with CONFIG_CPU_IDLE=n.

Overview of the patches:
-----------------------

Patches 1 to 6 introduce a generic, flexible Per-CPU Reader-Writer Locking
scheme.

Patch 7 uses this synchronization mechanism to build the
get/put_online_cpus_atomic() APIs which can be used from atomic context, to
prevent CPUs from going offline.

Patch 8 is a cleanup; it converts preprocessor macros to static inline
functions.

Patches 9 to 42 convert various call-sites to use the new APIs.

Patch 43 is the one which actually removes stop_machine() from the CPU
offline path.

Patch 44 decouples stop_machine() and CPU hotplug from Kconfig.

Patch 45 updates the documentation to reflect the new APIs.


Changes in v5:
--------------
  Exposed a new generic locking scheme: Flexible Per-CPU Reader-Writer locks,
  based on the synchronization schemes already discussed in the previous
  versions, and used it in CPU hotplug, to implement the new APIs.

  Audited the CPU_DYING notifiers in the kernel source tree and replaced
  usages of preempt_disable() with the new get/put_online_cpus_atomic() APIs
  where necessary.


Changes in v4:
--------------
  The synchronization scheme has been simplified quite a bit, which makes it
  look a lot less complex than before. Some highlights:

* Implicit ACKs:

  The earlier design required the readers to explicitly ACK the writer's
  signal. The new design uses implicit ACKs instead. The reader switching
  over to rwlock implicitly tells the writer to stop waiting for that reader.

* No atomic operations:

  Since we got rid of explicit ACKs, we no longer have the need for a reader
  and a writer to update the same counter. So we can get rid of atomic ops
  too.

Changes in v3:
--------------
* Dropped the _light() and _full() variants of the APIs. Provided a single
  interface: get/put_online_cpus_atomic().

* Completely redesigned the synchronization mechanism again, to make it
  fast and scalable at the reader-side in the fast-path (when no hotplug
  writers are active). This new scheme also ensures that there is no
  possibility of deadlocks due to circular locking dependency.
  In summary, this provides the scalability and speed of per-cpu rwlocks
  (without actually using them), while avoiding the downside (deadlock
  possibilities) which is inherent in any per-cpu locking scheme that is
  meant to compete with preempt_disable()/enable() in terms of flexibility.

  The problem with using per-cpu locking to replace preempt_disable()/enable
  was explained here:
  https://lkml.org/lkml/2012/12/6/290

  Basically we use per-cpu counters (for scalability) when no writers are
  active, and then switch to global rwlocks (for lock-safety) when a writer
  becomes active. It is a slightly complex scheme, but it is based on
  standard principles of distributed algorithms.

Changes in v2:
-------------
* Completely redesigned the synchronization scheme to avoid using any extra
  cpumasks.

* Provided APIs for 2 types of atomic hotplug readers: "light" (for
  light-weight) and "full". We wish to have more "light" readers than
  the "full" ones, to avoid indirectly inducing the "stop_machine effect"
  without even actually using stop_machine().

  And the patches show that it _is_ generally true: 5 patches deal with
  "light" readers, whereas only 1 patch deals with a "full" reader.

  Also, the "light" readers happen to be in very hot paths. So it makes a
  lot of sense to have such a distinction and a corresponding light-weight
  API.

Links to previous versions:
v4: https://lkml.org/lkml/2012/12/11/209
v3: https://lkml.org/lkml/2012/12/7/287
v2: https://lkml.org/lkml/2012/12/5/322
v1: https://lkml.org/lkml/2012/12/4/88

--

Paul E. McKenney (1):
      cpu: No more __stop_machine() in _cpu_down()

Srivatsa S. Bhat (44):
      percpu_rwlock: Introduce the global reader-writer lock backend
      percpu_rwlock: Introduce per-CPU variables for the reader and the writer
      percpu_rwlock: Provide a way to define and init percpu-rwlocks at compile time
      percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks
      percpu_rwlock: Make percpu-rwlocks IRQ-safe, optimally
      percpu_rwlock: Allow writers to be readers, and add lockdep annotations
      CPU hotplug: Provide APIs to prevent CPU offline from atomic context
      CPU hotplug: Convert preprocessor macros to static inline functions
      smp, cpu hotplug: Fix smp_call_function_*() to prevent CPU offline properly
      smp, cpu hotplug: Fix on_each_cpu_*() to prevent CPU offline properly
      sched/timer: Use get/put_online_cpus_atomic() to prevent CPU offline
      sched/migration: Use raw_spin_lock/unlock since interrupts are already disabled
      sched/rt: Use get/put_online_cpus_atomic() to prevent CPU offline
      rcu, CPU hotplug: Fix comment referring to stop_machine()
      tick: Use get/put_online_cpus_atomic() to prevent CPU offline
      time/clocksource: Use get/put_online_cpus_atomic() to prevent CPU offline
      softirq: Use get/put_online_cpus_atomic() to prevent CPU offline
      irq: Use get/put_online_cpus_atomic() to prevent CPU offline
      net: Use get/put_online_cpus_atomic() to prevent CPU offline
      block: Use get/put_online_cpus_atomic() to prevent CPU offline
      crypto: pcrypt - Protect access to cpu_online_mask with get/put_online_cpus()
      infiniband: ehca: Use get/put_online_cpus_atomic() to prevent CPU offline
      [SCSI] fcoe: Use get/put_online_cpus_atomic() to prevent CPU offline
      staging: octeon: Use get/put_online_cpus_atomic() to prevent CPU offline
      x86: Use get/put_online_cpus_atomic() to prevent CPU offline
      perf/x86: Use get/put_online_cpus_atomic() to prevent CPU offline
      KVM: Use get/put_online_cpus_atomic() to prevent CPU offline from atomic context
      kvm/vmx: Use get/put_online_cpus_atomic() to prevent CPU offline
      x86/xen: Use get/put_online_cpus_atomic() to prevent CPU offline
      alpha/smp: Use get/put_online_cpus_atomic() to prevent CPU offline
      blackfin/smp: Use get/put_online_cpus_atomic() to prevent CPU offline
      cris/smp: Use get/put_online_cpus_atomic() to prevent CPU offline
      hexagon/smp: Use get/put_online_cpus_atomic() to prevent CPU offline
      ia64: Use get/put_online_cpus_atomic() to prevent CPU offline
      m32r: Use get/put_online_cpus_atomic() to prevent CPU offline
      MIPS: Use get/put_online_cpus_atomic() to prevent CPU offline
      mn10300: Use get/put_online_cpus_atomic() to prevent CPU offline
      parisc: Use get/put_online_cpus_atomic() to prevent CPU offline
      powerpc: Use get/put_online_cpus_atomic() to prevent CPU offline
      sh: Use get/put_online_cpus_atomic() to prevent CPU offline
      sparc: Use get/put_online_cpus_atomic() to prevent CPU offline
      tile: Use get/put_online_cpus_atomic() to prevent CPU offline
      CPU hotplug, stop_machine: Decouple CPU hotplug from stop_machine() in Kconfig
      Documentation/cpu-hotplug: Remove references to stop_machine()


  Documentation/cpu-hotplug.txt                 |   17 +-
 arch/alpha/kernel/smp.c                       |   19 +-
 arch/arm/Kconfig                              |    1 
 arch/blackfin/Kconfig                         |    1 
 arch/blackfin/mach-common/smp.c               |    6 -
 arch/cris/arch-v32/kernel/smp.c               |    8 +
 arch/hexagon/kernel/smp.c                     |    5 +
 arch/ia64/Kconfig                             |    1 
 arch/ia64/kernel/irq_ia64.c                   |   13 +
 arch/ia64/kernel/perfmon.c                    |    6 +
 arch/ia64/kernel/smp.c                        |   23 ++
 arch/ia64/mm/tlb.c                            |    6 -
 arch/m32r/kernel/smp.c                        |   12 +
 arch/mips/Kconfig                             |    1 
 arch/mips/kernel/cevt-smtc.c                  |    8 +
 arch/mips/kernel/smp.c                        |   16 +-
 arch/mips/kernel/smtc.c                       |    3 
 arch/mips/mm/c-octeon.c                       |    4 
 arch/mn10300/Kconfig                          |    1 
 arch/mn10300/kernel/smp.c                     |    2 
 arch/mn10300/mm/cache-smp.c                   |    5 +
 arch/mn10300/mm/tlb-smp.c                     |   15 +-
 arch/parisc/Kconfig                           |    1 
 arch/parisc/kernel/smp.c                      |    4 
 arch/powerpc/Kconfig                          |    1 
 arch/powerpc/mm/mmu_context_nohash.c          |    2 
 arch/s390/Kconfig                             |    1 
 arch/sh/Kconfig                               |    1 
 arch/sh/kernel/smp.c                          |   12 +
 arch/sparc/Kconfig                            |    1 
 arch/sparc/kernel/leon_smp.c                  |    2 
 arch/sparc/kernel/smp_64.c                    |    9 +
 arch/sparc/kernel/sun4d_smp.c                 |    2 
 arch/sparc/kernel/sun4m_smp.c                 |    3 
 arch/tile/kernel/smp.c                        |    4 
 arch/x86/Kconfig                              |    1 
 arch/x86/include/asm/ipi.h                    |    5 +
 arch/x86/kernel/apic/apic_flat_64.c           |   10 +
 arch/x86/kernel/apic/apic_numachip.c          |    5 +
 arch/x86/kernel/apic/es7000_32.c              |    5 +
 arch/x86/kernel/apic/io_apic.c                |    7 +
 arch/x86/kernel/apic/ipi.c                    |   10 +
 arch/x86/kernel/apic/x2apic_cluster.c         |    4 
 arch/x86/kernel/apic/x2apic_uv_x.c            |    4 
 arch/x86/kernel/cpu/mcheck/therm_throt.c      |    4 
 arch/x86/kernel/cpu/perf_event_intel_uncore.c |    5 +
 arch/x86/kvm/vmx.c                            |    8 +
 arch/x86/mm/tlb.c                             |   14 +
 arch/x86/xen/mmu.c                            |   11 +
 arch/x86/xen/smp.c                            |    9 +
 block/blk-softirq.c                           |    4 
 crypto/pcrypt.c                               |    4 
 drivers/infiniband/hw/ehca/ehca_irq.c         |    8 +
 drivers/scsi/fcoe/fcoe.c                      |    7 +
 drivers/staging/octeon/ethernet-rx.c          |    3 
 include/linux/cpu.h                           |    8 +
 include/linux/percpu-rwlock.h                 |   86 +++++++++
 include/linux/stop_machine.h                  |    2 
 init/Kconfig                                  |    2 
 kernel/cpu.c                                  |   61 ++++++
 kernel/irq/manage.c                           |    7 +
 kernel/rcutree.c                              |    9 -
 kernel/sched/core.c                           |   36 +++-
 kernel/sched/fair.c                           |    5 -
 kernel/sched/rt.c                             |    3 
 kernel/smp.c                                  |   65 ++++---
 kernel/softirq.c                              |    3 
 kernel/time/clocksource.c                     |    5 +
 kernel/time/tick-broadcast.c                  |    2 
 kernel/timer.c                                |    2 
 lib/Kconfig                                   |    3 
 lib/Makefile                                  |    1 
 lib/percpu-rwlock.c                           |  242 +++++++++++++++++++++++++
 net/core/dev.c                                |    9 +
 virt/kvm/kvm_main.c                           |   10 +
 75 files changed, 776 insertions(+), 129 deletions(-)
 create mode 100644 include/linux/percpu-rwlock.h
 create mode 100644 lib/percpu-rwlock.c



Thanks,
Srivatsa S. Bhat
IBM Linux Technology Center

[PATCH v5 01/45] percpu_rwlock: Introduce the global reader-writer lock backend

[Srivatsa S. Bhat]

A straight-forward (and obvious) algorithm to implement Per-CPU Reader-Writer
locks can also lead to too many deadlock possibilities which can make it very
hard/impossible to use. This is explained in the example below, which helps
justify the need for a different algorithm to implement flexible Per-CPU
Reader-Writer locks.

We can use global rwlocks as shown below safely, without fear of deadlocks:

Readers:

         CPU 0                                CPU 1
         ------                               ------

1.    spin_lock(&random_lock);             read_lock(&my_rwlock);


2.    read_lock(&my_rwlock);               spin_lock(&random_lock);


Writer:

         CPU 2:
         ------

       write_lock(&my_rwlock);


We can observe that there is no possibility of deadlocks or circular locking
dependencies here. Its perfectly safe.

Now consider a blind/straight-forward conversion of global rwlocks to per-CPU
rwlocks like this:

The reader locks its own per-CPU rwlock for read, and proceeds.

Something like: read_lock(per-cpu rwlock of this cpu);

The writer acquires all per-CPU rwlocks for write and only then proceeds.

Something like:

  for_each_online_cpu(cpu)
	write_lock(per-cpu rwlock of 'cpu');


Now let's say that for performance reasons, the above scenario (which was
perfectly safe when using global rwlocks) was converted to use per-CPU rwlocks.


         CPU 0                                CPU 1
         ------                               ------

1.    spin_lock(&random_lock);             read_lock(my_rwlock of CPU 1);


2.    read_lock(my_rwlock of CPU 0);       spin_lock(&random_lock);


Writer:

         CPU 2:
         ------

      for_each_online_cpu(cpu)
        write_lock(my_rwlock of 'cpu');


Consider what happens if the writer begins his operation in between steps 1
and 2 at the reader side. It becomes evident that we end up in a (previously
non-existent) deadlock due to a circular locking dependency between the 3
entities, like this:


(holds              Waiting for
 random_lock) CPU 0 -------------> CPU 2  (holds my_rwlock of CPU 0
                                               for write)
               ^                   |
               |                   |
        Waiting|                   | Waiting
          for  |                   |  for
               |                   V
                ------ CPU 1 <------

                (holds my_rwlock of
                 CPU 1 for read)



So obviously this "straight-forward" way of implementing percpu rwlocks is
deadlock-prone. One simple measure for (or characteristic of) safe percpu
rwlock should be that if a user replaces global rwlocks with per-CPU rwlocks
(for performance reasons), he shouldn't suddenly end up in numerous deadlock
possibilities which never existed before. The replacement should continue to
remain safe, and perhaps improve the performance.

Observing the robustness of global rwlocks in providing a fair amount of
deadlock safety, we implement per-CPU rwlocks as nothing but global rwlocks,
as a first step.


Cc: David Howells <dhowells@redhat.com>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 include/linux/percpu-rwlock.h |   49 ++++++++++++++++++++++++++++++++
 lib/Kconfig                   |    3 ++
 lib/Makefile                  |    1 +
 lib/percpu-rwlock.c           |   63 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 116 insertions(+)
 create mode 100644 include/linux/percpu-rwlock.h
 create mode 100644 lib/percpu-rwlock.c

diff --git a/include/linux/percpu-rwlock.h b/include/linux/percpu-rwlock.h
new file mode 100644
index 0000000..45620d0
--- /dev/null
+++ b/include/linux/percpu-rwlock.h
@@ -0,0 +1,49 @@
+/*
+ * Flexible Per-CPU Reader-Writer Locks
+ * (with relaxed locking rules and reduced deadlock-possibilities)
+ *
+ * Copyright (C) IBM Corporation, 2012-2013
+ * Author: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
+ *
+ * With lots of invaluable suggestions from:
+ * 	   Oleg Nesterov <oleg@redhat.com>
+ * 	   Tejun Heo <tj@kernel.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#ifndef _LINUX_PERCPU_RWLOCK_H
+#define _LINUX_PERCPU_RWLOCK_H
+
+#include <linux/percpu.h>
+#include <linux/lockdep.h>
+#include <linux/spinlock.h>
+
+struct percpu_rwlock {
+	rwlock_t		global_rwlock;
+};
+
+extern void percpu_read_lock(struct percpu_rwlock *);
+extern void percpu_read_unlock(struct percpu_rwlock *);
+
+extern void percpu_write_lock(struct percpu_rwlock *);
+extern void percpu_write_unlock(struct percpu_rwlock *);
+
+extern int __percpu_init_rwlock(struct percpu_rwlock *,
+				const char *, struct lock_class_key *);
+
+#define percpu_init_rwlock(pcpu_rwlock)					\
+({	static struct lock_class_key rwlock_key;			\
+	__percpu_init_rwlock(pcpu_rwlock, #pcpu_rwlock, &rwlock_key);	\
+})
+
+#endif
diff --git a/lib/Kconfig b/lib/Kconfig
index 75cdb77..32fb0b9 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -45,6 +45,9 @@ config STMP_DEVICE
 config PERCPU_RWSEM
 	boolean
 
+config PERCPU_RWLOCK
+	boolean
+
 config CRC_CCITT
 	tristate "CRC-CCITT functions"
 	help
diff --git a/lib/Makefile b/lib/Makefile
index 02ed6c0..1854b5e 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -41,6 +41,7 @@ obj-$(CONFIG_DEBUG_SPINLOCK) += spinlock_debug.o
 lib-$(CONFIG_RWSEM_GENERIC_SPINLOCK) += rwsem-spinlock.o
 lib-$(CONFIG_RWSEM_XCHGADD_ALGORITHM) += rwsem.o
 lib-$(CONFIG_PERCPU_RWSEM) += percpu-rwsem.o
+lib-$(CONFIG_PERCPU_RWLOCK) += percpu-rwlock.o
 
 CFLAGS_hweight.o = $(subst $(quote),,$(CONFIG_ARCH_HWEIGHT_CFLAGS))
 obj-$(CONFIG_GENERIC_HWEIGHT) += hweight.o
diff --git a/lib/percpu-rwlock.c b/lib/percpu-rwlock.c
new file mode 100644
index 0000000..af0c714
--- /dev/null
+++ b/lib/percpu-rwlock.c
@@ -0,0 +1,63 @@
+/*
+ * Flexible Per-CPU Reader-Writer Locks
+ * (with relaxed locking rules and reduced deadlock-possibilities)
+ *
+ * Copyright (C) IBM Corporation, 2012-2013
+ * Author: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
+ *
+ * With lots of invaluable suggestions from:
+ * 	   Oleg Nesterov <oleg@redhat.com>
+ * 	   Tejun Heo <tj@kernel.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <linux/spinlock.h>
+#include <linux/percpu.h>
+#include <linux/lockdep.h>
+#include <linux/percpu-rwlock.h>
+#include <linux/errno.h>
+
+
+int __percpu_init_rwlock(struct percpu_rwlock *pcpu_rwlock,
+			 const char *name, struct lock_class_key *rwlock_key)
+{
+	/* ->global_rwlock represents the whole percpu_rwlock for lockdep */
+#ifdef CONFIG_DEBUG_SPINLOCK
+	__rwlock_init(&pcpu_rwlock->global_rwlock, name, rwlock_key);
+#else
+	pcpu_rwlock->global_rwlock =
+			__RW_LOCK_UNLOCKED(&pcpu_rwlock->global_rwlock);
+#endif
+	return 0;
+}
+
+void percpu_read_lock(struct percpu_rwlock *pcpu_rwlock)
+{
+	read_lock(&pcpu_rwlock->global_rwlock);
+}
+
+void percpu_read_unlock(struct percpu_rwlock *pcpu_rwlock)
+{
+	read_unlock(&pcpu_rwlock->global_rwlock);
+}
+
+void percpu_write_lock(struct percpu_rwlock *pcpu_rwlock)
+{
+	write_lock(&pcpu_rwlock->global_rwlock);
+}
+
+void percpu_write_unlock(struct percpu_rwlock *pcpu_rwlock)
+{
+	write_unlock(&pcpu_rwlock->global_rwlock);
+}
+

Re: [PATCH v5 01/45] percpu_rwlock: Introduce the global reader-writer lock backend

[Stephen Hemminger]

On Tue, 22 Jan 2013 13:03:22 +0530
"Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com> wrote:

> A straight-forward (and obvious) algorithm to implement Per-CPU Reader-Writer
> locks can also lead to too many deadlock possibilities which can make it very
> hard/impossible to use. This is explained in the example below, which helps
> justify the need for a different algorithm to implement flexible Per-CPU
> Reader-Writer locks.
> 
> We can use global rwlocks as shown below safely, without fear of deadlocks:
> 
> Readers:
> 
>          CPU 0                                CPU 1
>          ------                               ------
> 
> 1.    spin_lock(&random_lock);             read_lock(&my_rwlock);
> 
> 
> 2.    read_lock(&my_rwlock);               spin_lock(&random_lock);
> 
> 
> Writer:
> 
>          CPU 2:
>          ------
> 
>        write_lock(&my_rwlock);
> 
> 
> We can observe that there is no possibility of deadlocks or circular locking
> dependencies here. Its perfectly safe.
> 
> Now consider a blind/straight-forward conversion of global rwlocks to per-CPU
> rwlocks like this:
> 
> The reader locks its own per-CPU rwlock for read, and proceeds.
> 
> Something like: read_lock(per-cpu rwlock of this cpu);
> 
> The writer acquires all per-CPU rwlocks for write and only then proceeds.
> 
> Something like:
> 
>   for_each_online_cpu(cpu)
> 	write_lock(per-cpu rwlock of 'cpu');
> 
> 
> Now let's say that for performance reasons, the above scenario (which was
> perfectly safe when using global rwlocks) was converted to use per-CPU rwlocks.
> 
> 
>          CPU 0                                CPU 1
>          ------                               ------
> 
> 1.    spin_lock(&random_lock);             read_lock(my_rwlock of CPU 1);
> 
> 
> 2.    read_lock(my_rwlock of CPU 0);       spin_lock(&random_lock);
> 
> 
> Writer:
> 
>          CPU 2:
>          ------
> 
>       for_each_online_cpu(cpu)
>         write_lock(my_rwlock of 'cpu');
> 
> 
> Consider what happens if the writer begins his operation in between steps 1
> and 2 at the reader side. It becomes evident that we end up in a (previously
> non-existent) deadlock due to a circular locking dependency between the 3
> entities, like this:
> 
> 
> (holds              Waiting for
>  random_lock) CPU 0 -------------> CPU 2  (holds my_rwlock of CPU 0
>                                                for write)
>                ^                   |
>                |                   |
>         Waiting|                   | Waiting
>           for  |                   |  for
>                |                   V
>                 ------ CPU 1 <------
> 
>                 (holds my_rwlock of
>                  CPU 1 for read)
> 
> 
> 
> So obviously this "straight-forward" way of implementing percpu rwlocks is
> deadlock-prone. One simple measure for (or characteristic of) safe percpu
> rwlock should be that if a user replaces global rwlocks with per-CPU rwlocks
> (for performance reasons), he shouldn't suddenly end up in numerous deadlock
> possibilities which never existed before. The replacement should continue to
> remain safe, and perhaps improve the performance.
> 
> Observing the robustness of global rwlocks in providing a fair amount of
> deadlock safety, we implement per-CPU rwlocks as nothing but global rwlocks,
> as a first step.
> 
> 
> Cc: David Howells <dhowells@redhat.com>
> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>

We got rid of brlock years ago, do we have to reintroduce it like this?
The problem was that brlock caused starvation.

Re: [PATCH v5 01/45] percpu_rwlock: Introduce the global reader-writer lock backend

[Srivatsa S. Bhat]

On 01/23/2013 12:15 AM, Stephen Hemminger wrote:
> On Tue, 22 Jan 2013 13:03:22 +0530
> "Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com> wrote:
> 
>> A straight-forward (and obvious) algorithm to implement Per-CPU Reader-Writer
>> locks can also lead to too many deadlock possibilities which can make it very
>> hard/impossible to use. This is explained in the example below, which helps
>> justify the need for a different algorithm to implement flexible Per-CPU
>> Reader-Writer locks.
>>
>> We can use global rwlocks as shown below safely, without fear of deadlocks:
>>
>> Readers:
>>
>>          CPU 0                                CPU 1
>>          ------                               ------
>>
>> 1.    spin_lock(&random_lock);             read_lock(&my_rwlock);
>>
>>
>> 2.    read_lock(&my_rwlock);               spin_lock(&random_lock);
>>
>>
>> Writer:
>>
>>          CPU 2:
>>          ------
>>
>>        write_lock(&my_rwlock);
>>
>>
>> We can observe that there is no possibility of deadlocks or circular locking
>> dependencies here. Its perfectly safe.
>>
>> Now consider a blind/straight-forward conversion of global rwlocks to per-CPU
>> rwlocks like this:
>>
>> The reader locks its own per-CPU rwlock for read, and proceeds.
>>
>> Something like: read_lock(per-cpu rwlock of this cpu);
>>
>> The writer acquires all per-CPU rwlocks for write and only then proceeds.
>>
>> Something like:
>>
>>   for_each_online_cpu(cpu)
>> 	write_lock(per-cpu rwlock of 'cpu');
>>
>>
>> Now let's say that for performance reasons, the above scenario (which was
>> perfectly safe when using global rwlocks) was converted to use per-CPU rwlocks.
>>
>>
>>          CPU 0                                CPU 1
>>          ------                               ------
>>
>> 1.    spin_lock(&random_lock);             read_lock(my_rwlock of CPU 1);
>>
>>
>> 2.    read_lock(my_rwlock of CPU 0);       spin_lock(&random_lock);
>>
>>
>> Writer:
>>
>>          CPU 2:
>>          ------
>>
>>       for_each_online_cpu(cpu)
>>         write_lock(my_rwlock of 'cpu');
>>
>>
>> Consider what happens if the writer begins his operation in between steps 1
>> and 2 at the reader side. It becomes evident that we end up in a (previously
>> non-existent) deadlock due to a circular locking dependency between the 3
>> entities, like this:
>>
>>
>> (holds              Waiting for
>>  random_lock) CPU 0 -------------> CPU 2  (holds my_rwlock of CPU 0
>>                                                for write)
>>                ^                   |
>>                |                   |
>>         Waiting|                   | Waiting
>>           for  |                   |  for
>>                |                   V
>>                 ------ CPU 1 <------
>>
>>                 (holds my_rwlock of
>>                  CPU 1 for read)
>>
>>
>>
>> So obviously this "straight-forward" way of implementing percpu rwlocks is
>> deadlock-prone. One simple measure for (or characteristic of) safe percpu
>> rwlock should be that if a user replaces global rwlocks with per-CPU rwlocks
>> (for performance reasons), he shouldn't suddenly end up in numerous deadlock
>> possibilities which never existed before. The replacement should continue to
>> remain safe, and perhaps improve the performance.
>>
>> Observing the robustness of global rwlocks in providing a fair amount of
>> deadlock safety, we implement per-CPU rwlocks as nothing but global rwlocks,
>> as a first step.
>>
>>
>> Cc: David Howells <dhowells@redhat.com>
>> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
> 
> We got rid of brlock years ago, do we have to reintroduce it like this?
> The problem was that brlock caused starvation.
> 

Um? I still see it in include/linux/lglock.h and its users in fs/ directory.

BTW, I'm not advocating that everybody start converting their global reader-writer
locks to per-cpu rwlocks, because such a conversion probably won't make sense
in all scenarios.

The thing is, for CPU hotplug in particular, the "preempt_disable() at the reader;
stop_machine() at the writer" scheme had some very desirable properties at the
reader side (even though people might hate stop_machine() with all their
heart ;-)), namely : 

At the reader side:

o No need to hold locks to prevent CPU offline
o Extremely fast/optimized updates (the preempt count)
o No need for heavy memory barriers
o Extremely flexible nesting rules

So this made perfect sense at the reader for CPU hotplug, because it is expected
that CPU hotplug operations are very infrequent, and it is well-known that quite
a few atomic hotplug readers are in very hot paths. The problem was that the
stop_machine() at the writer was not only a little too heavy, but also inflicted
real-time latencies on the system because it needed cooperation from _all_ CPUs
synchronously, to take one CPU down.

So the idea is to get rid of stop_machine() without hurting the reader side.
And this scheme of per-cpu rwlocks comes close to ensuring that. (You can look
at the previous versions of this patchset [links given in cover letter] to see
what other schemes we hashed out before coming to this one).

The only reason I exposed this as a generic locking scheme was because Tejun
pointed out that, complex locking schemes implemented in individual subsystems
is not such a good idea. And also this comes at a time when per-cpu rwsemaphores
have just been introduced in the kernel and Oleg had ideas about converting the
cpu hotplug (sleepable) locking to use them.

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 01/45] percpu_rwlock: Introduce the global reader-writer lock backend

[Steven Rostedt]

On Tue, 2013-01-22 at 13:03 +0530, Srivatsa S. Bhat wrote:
> A straight-forward (and obvious) algorithm to implement Per-CPU Reader-Writer
> locks can also lead to too many deadlock possibilities which can make it very
> hard/impossible to use. This is explained in the example below, which helps
> justify the need for a different algorithm to implement flexible Per-CPU
> Reader-Writer locks.
> 
> We can use global rwlocks as shown below safely, without fear of deadlocks:
> 
> Readers:
> 
>          CPU 0                                CPU 1
>          ------                               ------
> 
> 1.    spin_lock(&random_lock);             read_lock(&my_rwlock);
> 
> 
> 2.    read_lock(&my_rwlock);               spin_lock(&random_lock);
> 
> 
> Writer:
> 
>          CPU 2:
>          ------
> 
>        write_lock(&my_rwlock);
> 

I thought global locks are now fair. That is, a reader will block if a
writer is waiting. Hence, the above should deadlock on the current
rwlock_t types.

We need to fix those locations (or better yet, remove all rwlocks ;-)

-- Steve

Re: [PATCH v5 01/45] percpu_rwlock: Introduce the global reader-writer lock backend

[Srivatsa S. Bhat]

On 01/23/2013 01:02 AM, Steven Rostedt wrote:
> On Tue, 2013-01-22 at 13:03 +0530, Srivatsa S. Bhat wrote:
>> A straight-forward (and obvious) algorithm to implement Per-CPU Reader-Writer
>> locks can also lead to too many deadlock possibilities which can make it very
>> hard/impossible to use. This is explained in the example below, which helps
>> justify the need for a different algorithm to implement flexible Per-CPU
>> Reader-Writer locks.
>>
>> We can use global rwlocks as shown below safely, without fear of deadlocks:
>>
>> Readers:
>>
>>          CPU 0                                CPU 1
>>          ------                               ------
>>
>> 1.    spin_lock(&random_lock);             read_lock(&my_rwlock);
>>
>>
>> 2.    read_lock(&my_rwlock);               spin_lock(&random_lock);
>>
>>
>> Writer:
>>
>>          CPU 2:
>>          ------
>>
>>        write_lock(&my_rwlock);
>>
> 
> I thought global locks are now fair. That is, a reader will block if a
> writer is waiting. Hence, the above should deadlock on the current
> rwlock_t types.
> 

Oh is it? Last I checked, lockdep didn't complain about this ABBA scenario!

> We need to fix those locations (or better yet, remove all rwlocks ;-)
> 

:-)

The challenge with stop_machine() removal is that the replacement on the
reader side must have the (locking) flexibility comparable to preempt_disable().
Otherwise, that solution most likely won't be viable because we'll hit way
too many locking problems and go crazy by the time we convert them over..(if
we can, that is!)

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 01/45] percpu_rwlock: Introduce the global reader-writer lock backend

[Steven Rostedt]

On Wed, 2013-01-23 at 01:28 +0530, Srivatsa S. Bhat wrote:

> > I thought global locks are now fair. That is, a reader will block if a
> > writer is waiting. Hence, the above should deadlock on the current
> > rwlock_t types.
> > 
> 
> Oh is it? Last I checked, lockdep didn't complain about this ABBA scenario!

It doesn't and Peter Zijlstra said we need to fix that ;-)  It only
recently became an issue with the new "fair" locking of rwlocks.

-- Steve

Re: [PATCH v5 01/45] percpu_rwlock: Introduce the global reader-writer lock backend

[Michel Lespinasse]

On Tue, Jan 22, 2013 at 11:32 AM, Steven Rostedt <rostedt@goodmis.org> wrote:
> On Tue, 2013-01-22 at 13:03 +0530, Srivatsa S. Bhat wrote:
>> A straight-forward (and obvious) algorithm to implement Per-CPU Reader-Writer
>> locks can also lead to too many deadlock possibilities which can make it very
>> hard/impossible to use. This is explained in the example below, which helps
>> justify the need for a different algorithm to implement flexible Per-CPU
>> Reader-Writer locks.
>>
>> We can use global rwlocks as shown below safely, without fear of deadlocks:
>>
>> Readers:
>>
>>          CPU 0                                CPU 1
>>          ------                               ------
>>
>> 1.    spin_lock(&random_lock);             read_lock(&my_rwlock);
>>
>>
>> 2.    read_lock(&my_rwlock);               spin_lock(&random_lock);
>>
>>
>> Writer:
>>
>>          CPU 2:
>>          ------
>>
>>        write_lock(&my_rwlock);
>>
>
> I thought global locks are now fair. That is, a reader will block if a
> writer is waiting. Hence, the above should deadlock on the current
> rwlock_t types.

I believe you are mistaken here. struct rw_semaphore is fair (and
blocking), but rwlock_t is unfair. The reason we can't easily make
rwlock_t fair is because tasklist_lock currently depends on the
rwlock_t unfairness - tasklist_lock readers typically don't disable
local interrupts, and tasklist_lock may be acquired again from within
an interrupt, which would deadlock if rwlock_t was fair and a writer
was queued by the time the interrupt is processed.

> We need to fix those locations (or better yet, remove all rwlocks ;-)

tasklist_lock is the main remaining user. I'm not sure about removing
rwlock_t, but I would like to at least make it fair somehow :)

-- 
Michel "Walken" Lespinasse
A program is never fully debugged until the last user dies.

[PATCH v5 02/45] percpu_rwlock: Introduce per-CPU variables for the reader and the writer

[Srivatsa S. Bhat]

Per-CPU rwlocks ought to give better performance than global rwlocks.
That is where the "per-CPU" component comes in. So introduce the necessary
per-CPU variables that would be necessary at the reader and the writer sides,
and add the support for dynamically initializing per-CPU rwlocks.
These per-CPU variables will be used subsequently to implement the core
algorithm behind per-CPU rwlocks.

Cc: David Howells <dhowells@redhat.com>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 include/linux/percpu-rwlock.h |    4 ++++
 lib/percpu-rwlock.c           |   21 +++++++++++++++++++++
 2 files changed, 25 insertions(+)

diff --git a/include/linux/percpu-rwlock.h b/include/linux/percpu-rwlock.h
index 45620d0..cd5eab5 100644
--- a/include/linux/percpu-rwlock.h
+++ b/include/linux/percpu-rwlock.h
@@ -29,6 +29,8 @@
 #include <linux/spinlock.h>
 
 struct percpu_rwlock {
+	unsigned long __percpu	*reader_refcnt;
+	bool __percpu		*writer_signal;
 	rwlock_t		global_rwlock;
 };
 
@@ -41,6 +43,8 @@ extern void percpu_write_unlock(struct percpu_rwlock *);
 extern int __percpu_init_rwlock(struct percpu_rwlock *,
 				const char *, struct lock_class_key *);
 
+extern void percpu_free_rwlock(struct percpu_rwlock *);
+
 #define percpu_init_rwlock(pcpu_rwlock)					\
 ({	static struct lock_class_key rwlock_key;			\
 	__percpu_init_rwlock(pcpu_rwlock, #pcpu_rwlock, &rwlock_key);	\
diff --git a/lib/percpu-rwlock.c b/lib/percpu-rwlock.c
index af0c714..80dad93 100644
--- a/lib/percpu-rwlock.c
+++ b/lib/percpu-rwlock.c
@@ -31,6 +31,17 @@
 int __percpu_init_rwlock(struct percpu_rwlock *pcpu_rwlock,
 			 const char *name, struct lock_class_key *rwlock_key)
 {
+	pcpu_rwlock->reader_refcnt = alloc_percpu(unsigned long);
+	if (unlikely(!pcpu_rwlock->reader_refcnt))
+		return -ENOMEM;
+
+	pcpu_rwlock->writer_signal = alloc_percpu(bool);
+	if (unlikely(!pcpu_rwlock->writer_signal)) {
+		free_percpu(pcpu_rwlock->reader_refcnt);
+		pcpu_rwlock->reader_refcnt = NULL;
+		return -ENOMEM;
+	}
+
 	/* ->global_rwlock represents the whole percpu_rwlock for lockdep */
 #ifdef CONFIG_DEBUG_SPINLOCK
 	__rwlock_init(&pcpu_rwlock->global_rwlock, name, rwlock_key);
@@ -41,6 +52,16 @@ int __percpu_init_rwlock(struct percpu_rwlock *pcpu_rwlock,
 	return 0;
 }
 
+void percpu_free_rwlock(struct percpu_rwlock *pcpu_rwlock)
+{
+	free_percpu(pcpu_rwlock->reader_refcnt);
+	free_percpu(pcpu_rwlock->writer_signal);
+
+	/* Catch use-after-free bugs */
+	pcpu_rwlock->reader_refcnt = NULL;
+	pcpu_rwlock->writer_signal = NULL;
+}
+
 void percpu_read_lock(struct percpu_rwlock *pcpu_rwlock)
 {
 	read_lock(&pcpu_rwlock->global_rwlock);

[PATCH v5 03/45] percpu_rwlock: Provide a way to define and init percpu-rwlocks at compile time

[Srivatsa S. Bhat]

Add the support for defining and initializing percpu-rwlocks at compile time
for those users who would like to use percpu-rwlocks really early in the boot
process (even before dynamic per-CPU allocations can begin).

Cc: David Howells <dhowells@redhat.com>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 include/linux/percpu-rwlock.h |   18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/include/linux/percpu-rwlock.h b/include/linux/percpu-rwlock.h
index cd5eab5..8dec8fe 100644
--- a/include/linux/percpu-rwlock.h
+++ b/include/linux/percpu-rwlock.h
@@ -45,6 +45,24 @@ extern int __percpu_init_rwlock(struct percpu_rwlock *,
 
 extern void percpu_free_rwlock(struct percpu_rwlock *);
 
+
+#define __PERCPU_RWLOCK_INIT(name)					\
+	{								\
+		.reader_refcnt = &name##_reader_refcnt,			\
+		.writer_signal = &name##_writer_signal,			\
+		.global_rwlock = __RW_LOCK_UNLOCKED(name.global_rwlock) \
+	}
+
+#define DEFINE_PERCPU_RWLOCK(name)					\
+	static DEFINE_PER_CPU(unsigned long, name##_reader_refcnt);	\
+	static DEFINE_PER_CPU(bool, name##_writer_signal);		\
+	struct percpu_rwlock (name) = __PERCPU_RWLOCK_INIT(name);
+
+#define DEFINE_STATIC_PERCPU_RWLOCK(name)				\
+	static DEFINE_PER_CPU(unsigned long, name##_reader_refcnt);	\
+	static DEFINE_PER_CPU(bool, name##_writer_signal);		\
+	static struct percpu_rwlock(name) = __PERCPU_RWLOCK_INIT(name);
+
 #define percpu_init_rwlock(pcpu_rwlock)					\
 ({	static struct lock_class_key rwlock_key;			\
 	__percpu_init_rwlock(pcpu_rwlock, #pcpu_rwlock, &rwlock_key);	\

[PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Srivatsa S. Bhat]

Using global rwlocks as the backend for per-CPU rwlocks helps us avoid many
lock-ordering related problems (unlike per-cpu locks). However, global
rwlocks lead to unnecessary cache-line bouncing even when there are no
writers present, which can slow down the system needlessly.

Per-cpu counters can help solve the cache-line bouncing problem. So we
actually use the best of both: per-cpu counters (no-waiting) at the reader
side in the fast-path, and global rwlocks in the slowpath.

[ Fastpath = no writer is active; Slowpath = a writer is active ]

IOW, the readers just increment/decrement their per-cpu refcounts (disabling
interrupts during the updates, if necessary) when no writer is active.
When a writer becomes active, he signals all readers to switch to global
rwlocks for the duration of his activity. The readers switch over when it
is safe for them (ie., when they are about to start a fresh, non-nested
read-side critical section) and start using (holding) the global rwlock for
read in their subsequent critical sections.

The writer waits for every existing reader to switch, and then acquires the
global rwlock for write and enters his critical section. Later, the writer
signals all readers that he is done, and that they can go back to using their
per-cpu refcounts again.

Note that the lock-safety (despite the per-cpu scheme) comes from the fact
that the readers can *choose* _when_ to switch to rwlocks upon the writer's
signal. And the readers don't wait on anybody based on the per-cpu counters.
The only true synchronization that involves waiting at the reader-side in this
scheme, is the one arising from the global rwlock, which is safe from circular
locking dependency issues.

Reader-writer locks and per-cpu counters are recursive, so they can be
used in a nested fashion in the reader-path, which makes per-CPU rwlocks also
recursive. Also, this design of switching the synchronization scheme ensures
that you can safely nest and use these locks in a very flexible manner.

I'm indebted to Michael Wang and Xiao Guangrong for their numerous thoughtful
suggestions and ideas, which inspired and influenced many of the decisions in
this as well as previous designs. Thanks a lot Michael and Xiao!

Cc: David Howells <dhowells@redhat.com>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 include/linux/percpu-rwlock.h |   10 +++
 lib/percpu-rwlock.c           |  128 ++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 136 insertions(+), 2 deletions(-)

diff --git a/include/linux/percpu-rwlock.h b/include/linux/percpu-rwlock.h
index 8dec8fe..6819bb8 100644
--- a/include/linux/percpu-rwlock.h
+++ b/include/linux/percpu-rwlock.h
@@ -68,4 +68,14 @@ extern void percpu_free_rwlock(struct percpu_rwlock *);
 	__percpu_init_rwlock(pcpu_rwlock, #pcpu_rwlock, &rwlock_key);	\
 })
 
+#define reader_uses_percpu_refcnt(pcpu_rwlock, cpu)			\
+		(ACCESS_ONCE(per_cpu(*((pcpu_rwlock)->reader_refcnt), cpu)))
+
+#define reader_nested_percpu(pcpu_rwlock)				\
+			(__this_cpu_read(*((pcpu_rwlock)->reader_refcnt)) > 1)
+
+#define writer_active(pcpu_rwlock)					\
+			(__this_cpu_read(*((pcpu_rwlock)->writer_signal)))
+
 #endif
+
diff --git a/lib/percpu-rwlock.c b/lib/percpu-rwlock.c
index 80dad93..992da5c 100644
--- a/lib/percpu-rwlock.c
+++ b/lib/percpu-rwlock.c
@@ -64,21 +64,145 @@ void percpu_free_rwlock(struct percpu_rwlock *pcpu_rwlock)
 
 void percpu_read_lock(struct percpu_rwlock *pcpu_rwlock)
 {
-	read_lock(&pcpu_rwlock->global_rwlock);
+	preempt_disable();
+
+	/* First and foremost, let the writer know that a reader is active */
+	this_cpu_inc(*pcpu_rwlock->reader_refcnt);
+
+	/*
+	 * If we are already using per-cpu refcounts, it is not safe to switch
+	 * the synchronization scheme. So continue using the refcounts.
+	 */
+	if (reader_nested_percpu(pcpu_rwlock)) {
+		goto out;
+	} else {
+		/*
+		 * The write to 'reader_refcnt' must be visible before we
+		 * read 'writer_signal'.
+		 */
+		smp_mb(); /* Paired with smp_rmb() in sync_reader() */
+
+		if (likely(!writer_active(pcpu_rwlock))) {
+			goto out;
+		} else {
+			/* Writer is active, so switch to global rwlock. */
+			read_lock(&pcpu_rwlock->global_rwlock);
+
+			/*
+			 * We might have raced with a writer going inactive
+			 * before we took the read-lock. So re-evaluate whether
+			 * we still need to hold the rwlock or if we can switch
+			 * back to per-cpu refcounts. (This also helps avoid
+			 * heterogeneous nesting of readers).
+			 */
+			if (writer_active(pcpu_rwlock))
+				this_cpu_dec(*pcpu_rwlock->reader_refcnt);
+			else
+				read_unlock(&pcpu_rwlock->global_rwlock);
+		}
+	}
+
+out:
+	/* Prevent reordering of any subsequent reads */
+	smp_rmb();
 }
 
 void percpu_read_unlock(struct percpu_rwlock *pcpu_rwlock)
 {
-	read_unlock(&pcpu_rwlock->global_rwlock);
+	/*
+	 * We never allow heterogeneous nesting of readers. So it is trivial
+	 * to find out the kind of reader we are, and undo the operation
+	 * done by our corresponding percpu_read_lock().
+	 */
+	if (__this_cpu_read(*pcpu_rwlock->reader_refcnt)) {
+		this_cpu_dec(*pcpu_rwlock->reader_refcnt);
+		smp_wmb(); /* Paired with smp_rmb() in sync_reader() */
+	} else {
+		read_unlock(&pcpu_rwlock->global_rwlock);
+	}
+
+	preempt_enable();
+}
+
+static inline void raise_writer_signal(struct percpu_rwlock *pcpu_rwlock,
+				       unsigned int cpu)
+{
+	per_cpu(*pcpu_rwlock->writer_signal, cpu) = true;
+}
+
+static inline void drop_writer_signal(struct percpu_rwlock *pcpu_rwlock,
+				      unsigned int cpu)
+{
+	per_cpu(*pcpu_rwlock->writer_signal, cpu) = false;
+}
+
+static void announce_writer_active(struct percpu_rwlock *pcpu_rwlock)
+{
+	unsigned int cpu;
+
+	for_each_online_cpu(cpu)
+		raise_writer_signal(pcpu_rwlock, cpu);
+
+	smp_mb(); /* Paired with smp_rmb() in percpu_read_[un]lock() */
+}
+
+static void announce_writer_inactive(struct percpu_rwlock *pcpu_rwlock)
+{
+	unsigned int cpu;
+
+	drop_writer_signal(pcpu_rwlock, smp_processor_id());
+
+	for_each_online_cpu(cpu)
+		drop_writer_signal(pcpu_rwlock, cpu);
+
+	smp_mb(); /* Paired with smp_rmb() in percpu_read_[un]lock() */
+}
+
+/*
+ * Wait for the reader to see the writer's signal and switch from percpu
+ * refcounts to global rwlock.
+ *
+ * If the reader is still using percpu refcounts, wait for him to switch.
+ * Else, we can safely go ahead, because either the reader has already
+ * switched over, or the next reader that comes along on that CPU will
+ * notice the writer's signal and will switch over to the rwlock.
+ */
+static inline void sync_reader(struct percpu_rwlock *pcpu_rwlock,
+			       unsigned int cpu)
+{
+	smp_rmb(); /* Paired with smp_[w]mb() in percpu_read_[un]lock() */
+
+	while (reader_uses_percpu_refcnt(pcpu_rwlock, cpu))
+		cpu_relax();
+}
+
+static void sync_all_readers(struct percpu_rwlock *pcpu_rwlock)
+{
+	unsigned int cpu;
+
+	for_each_online_cpu(cpu)
+		sync_reader(pcpu_rwlock, cpu);
 }
 
 void percpu_write_lock(struct percpu_rwlock *pcpu_rwlock)
 {
+	/*
+	 * Tell all readers that a writer is becoming active, so that they
+	 * start switching over to the global rwlock.
+	 */
+	announce_writer_active(pcpu_rwlock);
+	sync_all_readers(pcpu_rwlock);
 	write_lock(&pcpu_rwlock->global_rwlock);
 }
 
 void percpu_write_unlock(struct percpu_rwlock *pcpu_rwlock)
 {
+	/*
+	 * Inform all readers that we are done, so that they can switch back
+	 * to their per-cpu refcounts. (We don't need to wait for them to
+	 * see it).
+	 */
+	announce_writer_inactive(pcpu_rwlock);
 	write_unlock(&pcpu_rwlock->global_rwlock);
 }
 

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Tejun Heo]

Hello, Srivatsa.

First of all, I'm not sure whether we need to be this step-by-step
when introducing something new.  It's not like we're transforming an
existing implementation and it doesn't seem to help understanding the
series that much either.

On Tue, Jan 22, 2013 at 01:03:53PM +0530, Srivatsa S. Bhat wrote:
> Using global rwlocks as the backend for per-CPU rwlocks helps us avoid many
> lock-ordering related problems (unlike per-cpu locks). However, global

So, unfortunately, this already seems broken, right?  The problem here
seems to be that previously, say, read_lock() implied
preempt_disable() but as this series aims to move away from it, it
introduces the problem of locking order between such locks and the new
contruct.

The only two options are either punishing writers or identifying and
updating all such possible deadlocks.  percpu_rwsem does the former,
right?  I don't know how feasible the latter would be.  Srivatsa,
you've been looking at all the places which would require conversion,
how difficult would doing the latter be?

> +#define reader_uses_percpu_refcnt(pcpu_rwlock, cpu)			\
> +		(ACCESS_ONCE(per_cpu(*((pcpu_rwlock)->reader_refcnt), cpu)))
> +
> +#define reader_nested_percpu(pcpu_rwlock)				\
> +			(__this_cpu_read(*((pcpu_rwlock)->reader_refcnt)) > 1)
> +
> +#define writer_active(pcpu_rwlock)					\
> +			(__this_cpu_read(*((pcpu_rwlock)->writer_signal)))

Why are these in the public header file?  Are they gonna be used to
inline something?

> +static inline void raise_writer_signal(struct percpu_rwlock *pcpu_rwlock,
> +				       unsigned int cpu)
> +{
> +	per_cpu(*pcpu_rwlock->writer_signal, cpu) = true;
> +}
> +
> +static inline void drop_writer_signal(struct percpu_rwlock *pcpu_rwlock,
> +				      unsigned int cpu)
> +{
> +	per_cpu(*pcpu_rwlock->writer_signal, cpu) = false;
> +}
> +
> +static void announce_writer_active(struct percpu_rwlock *pcpu_rwlock)
> +{
> +	unsigned int cpu;
> +
> +	for_each_online_cpu(cpu)
> +		raise_writer_signal(pcpu_rwlock, cpu);
> +
> +	smp_mb(); /* Paired with smp_rmb() in percpu_read_[un]lock() */
> +}
> +
> +static void announce_writer_inactive(struct percpu_rwlock *pcpu_rwlock)
> +{
> +	unsigned int cpu;
> +
> +	drop_writer_signal(pcpu_rwlock, smp_processor_id());
> +
> +	for_each_online_cpu(cpu)
> +		drop_writer_signal(pcpu_rwlock, cpu);
> +
> +	smp_mb(); /* Paired with smp_rmb() in percpu_read_[un]lock() */
> +}

It could be just personal preference but I find the above one line
wrappers more obfuscating than anything else.  What's the point of
wrapping writer_signal = true/false into a separate function?  These
simple wrappers just add layers that people have to dig through to
figure out what's going on without adding anything of value.  I'd much
prefer collapsing these into the percpu_write_[un]lock().

Thanks.

-- 
tejun

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Srivatsa S. Bhat]

On 01/24/2013 12:25 AM, Tejun Heo wrote:
> Hello, Srivatsa.
> 
> First of all, I'm not sure whether we need to be this step-by-step
> when introducing something new.  It's not like we're transforming an
> existing implementation and it doesn't seem to help understanding the
> series that much either.
> 

Hmm.. I split it up into steps to help explain the reasoning behind
the code sufficiently, rather than spring all of the intricacies at
one go (which would make it very hard to write the changelog/comments
also). The split made it easier for me to document it well in the
changelog, because I could deal with reasonable chunks of code/complexity
at a time. IMHO that helps people reading it for the first time to
understand the logic easily.

> On Tue, Jan 22, 2013 at 01:03:53PM +0530, Srivatsa S. Bhat wrote:
>> Using global rwlocks as the backend for per-CPU rwlocks helps us avoid many
>> lock-ordering related problems (unlike per-cpu locks). However, global
> 
> So, unfortunately, this already seems broken, right?  The problem here
> seems to be that previously, say, read_lock() implied
> preempt_disable() but as this series aims to move away from it, it
> introduces the problem of locking order between such locks and the new
> contruct.
>

Not sure I got your point correctly. Are you referring to Steve's comment
that rwlocks are probably fair now (and hence not really safe when used
like this)? If yes, I haven't actually verified that yet, but yes, that
will make this hard to use, since we need to take care of locking rules.

But suppose rwlocks are unfair (as I had assumed them to be), then we
have absolutely no problems and no lock-ordering to worry about.
 
> The only two options are either punishing writers or identifying and
> updating all such possible deadlocks.  percpu_rwsem does the former,
> right?  I don't know how feasible the latter would be.

I don't think we can avoid looking into all the possible deadlocks,
as long as we use rwlocks inside get/put_online_cpus_atomic() (assuming
rwlocks are fair). Even with Oleg's idea of using synchronize_sched()
at the writer, we still need to take care of locking rules, because the
synchronize_sched() only helps avoid the memory barriers at the reader,
and doesn't help get rid of the rwlocks themselves.
So in short, I don't see how we can punish the writers and thereby somehow
avoid looking into possible deadlocks (if rwlocks are fair).

>  Srivatsa,
> you've been looking at all the places which would require conversion,
> how difficult would doing the latter be?
> 

The problem is that some APIs like smp_call_function() will need to use
get/put_online_cpus_atomic(). That is when the locking becomes tricky
in the subsystem which invokes these APIs with other (subsystem-specific,
internal) locks held. So we could potentially use a convention such as
"Make get/put_online_cpus_atomic() your outer-most calls, within which
you nest the other locks" to rule out all ABBA deadlock possibilities...
But we might still hit some hard-to-convert places.. 

BTW, Steve, fair rwlocks doesn't mean the following scenario will result
in a deadlock right?

CPU 0                          CPU 1

read_lock(&rwlock)

                              write_lock(&rwlock) //spins, because CPU 0
                              //has acquired the lock for read

read_lock(&rwlock)
   ^^^^^
What happens here? Does CPU 0 start spinning (and hence deadlock) or will
it continue realizing that it already holds the rwlock for read?

If the above ends in a deadlock, then its next to impossible to convert
all the places safely (because the above mentioned convention will simply
fall apart).

>> +#define reader_uses_percpu_refcnt(pcpu_rwlock, cpu)			\
>> +		(ACCESS_ONCE(per_cpu(*((pcpu_rwlock)->reader_refcnt), cpu)))
>> +
>> +#define reader_nested_percpu(pcpu_rwlock)				\
>> +			(__this_cpu_read(*((pcpu_rwlock)->reader_refcnt)) > 1)
>> +
>> +#define writer_active(pcpu_rwlock)					\
>> +			(__this_cpu_read(*((pcpu_rwlock)->writer_signal)))
> 
> Why are these in the public header file?  Are they gonna be used to
> inline something?
>

No, I can put it in the .c file itself. Will do.
 
>> +static inline void raise_writer_signal(struct percpu_rwlock *pcpu_rwlock,
>> +				       unsigned int cpu)
>> +{
>> +	per_cpu(*pcpu_rwlock->writer_signal, cpu) = true;
>> +}
>> +
>> +static inline void drop_writer_signal(struct percpu_rwlock *pcpu_rwlock,
>> +				      unsigned int cpu)
>> +{
>> +	per_cpu(*pcpu_rwlock->writer_signal, cpu) = false;
>> +}
>> +
>> +static void announce_writer_active(struct percpu_rwlock *pcpu_rwlock)
>> +{
>> +	unsigned int cpu;
>> +
>> +	for_each_online_cpu(cpu)
>> +		raise_writer_signal(pcpu_rwlock, cpu);
>> +
>> +	smp_mb(); /* Paired with smp_rmb() in percpu_read_[un]lock() */
>> +}
>> +
>> +static void announce_writer_inactive(struct percpu_rwlock *pcpu_rwlock)
>> +{
>> +	unsigned int cpu;
>> +
>> +	drop_writer_signal(pcpu_rwlock, smp_processor_id());
>> +
>> +	for_each_online_cpu(cpu)
>> +		drop_writer_signal(pcpu_rwlock, cpu);
>> +
>> +	smp_mb(); /* Paired with smp_rmb() in percpu_read_[un]lock() */
>> +}
> 
> It could be just personal preference but I find the above one line
> wrappers more obfuscating than anything else.  What's the point of
> wrapping writer_signal = true/false into a separate function?  These
> simple wrappers just add layers that people have to dig through to
> figure out what's going on without adding anything of value.  I'd much
> prefer collapsing these into the percpu_write_[un]lock().
>

Sure, I see your point. I'll change that.

Thanks a lot for your feedback Tejun!

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Tejun Heo]

Hello, Srivatsa.

On Thu, Jan 24, 2013 at 01:03:52AM +0530, Srivatsa S. Bhat wrote:
> Hmm.. I split it up into steps to help explain the reasoning behind
> the code sufficiently, rather than spring all of the intricacies at
> one go (which would make it very hard to write the changelog/comments
> also). The split made it easier for me to document it well in the
> changelog, because I could deal with reasonable chunks of code/complexity
> at a time. IMHO that helps people reading it for the first time to
> understand the logic easily.

I don't know.  It's a judgement call I guess.  I personally would much
prefer having ample documentation as comments in the source itself or
as a separate Documentation/ file as that's what most people are gonna
be looking at to figure out what's going on.  Maybe just compact it a
bit and add more in-line documentation instead?

> > The only two options are either punishing writers or identifying and
> > updating all such possible deadlocks.  percpu_rwsem does the former,
> > right?  I don't know how feasible the latter would be.
> 
> I don't think we can avoid looking into all the possible deadlocks,
> as long as we use rwlocks inside get/put_online_cpus_atomic() (assuming
> rwlocks are fair). Even with Oleg's idea of using synchronize_sched()
> at the writer, we still need to take care of locking rules, because the
> synchronize_sched() only helps avoid the memory barriers at the reader,
> and doesn't help get rid of the rwlocks themselves.

Well, percpu_rwlock don't have to use rwlock for the slow path.  It
can implement its own writer starving locking scheme.  It's not like
implementing slow path global rwlock logic is difficult.

> CPU 0                          CPU 1
> 
> read_lock(&rwlock)
> 
>                               write_lock(&rwlock) //spins, because CPU 0
>                               //has acquired the lock for read
> 
> read_lock(&rwlock)
>    ^^^^^
> What happens here? Does CPU 0 start spinning (and hence deadlock) or will
> it continue realizing that it already holds the rwlock for read?

I don't think rwlock allows nesting write lock inside read lock.
read_lock(); write_lock() will always deadlock.

Thanks.

-- 
tejun

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Srivatsa S. Bhat]

On 01/24/2013 01:27 AM, Tejun Heo wrote:
> Hello, Srivatsa.
> 
> On Thu, Jan 24, 2013 at 01:03:52AM +0530, Srivatsa S. Bhat wrote:
>> Hmm.. I split it up into steps to help explain the reasoning behind
>> the code sufficiently, rather than spring all of the intricacies at
>> one go (which would make it very hard to write the changelog/comments
>> also). The split made it easier for me to document it well in the
>> changelog, because I could deal with reasonable chunks of code/complexity
>> at a time. IMHO that helps people reading it for the first time to
>> understand the logic easily.
> 
> I don't know.  It's a judgement call I guess.  I personally would much
> prefer having ample documentation as comments in the source itself or
> as a separate Documentation/ file as that's what most people are gonna
> be looking at to figure out what's going on.  Maybe just compact it a
> bit and add more in-line documentation instead?
> 

OK, I'll think about this.

>>> The only two options are either punishing writers or identifying and
>>> updating all such possible deadlocks.  percpu_rwsem does the former,
>>> right?  I don't know how feasible the latter would be.
>>
>> I don't think we can avoid looking into all the possible deadlocks,
>> as long as we use rwlocks inside get/put_online_cpus_atomic() (assuming
>> rwlocks are fair). Even with Oleg's idea of using synchronize_sched()
>> at the writer, we still need to take care of locking rules, because the
>> synchronize_sched() only helps avoid the memory barriers at the reader,
>> and doesn't help get rid of the rwlocks themselves.
> 
> Well, percpu_rwlock don't have to use rwlock for the slow path.  It
> can implement its own writer starving locking scheme.  It's not like
> implementing slow path global rwlock logic is difficult.
>

Great idea! So probably I could use atomic ops or something similar in the
slow path to implement the scheme we need...

>> CPU 0                          CPU 1
>>
>> read_lock(&rwlock)
>>
>>                               write_lock(&rwlock) //spins, because CPU 0
>>                               //has acquired the lock for read
>>
>> read_lock(&rwlock)
>>    ^^^^^
>> What happens here? Does CPU 0 start spinning (and hence deadlock) or will
>> it continue realizing that it already holds the rwlock for read?
> 
> I don't think rwlock allows nesting write lock inside read lock.
> read_lock(); write_lock() will always deadlock.
> 

Sure, I understand that :-) My question was, what happens when *two* CPUs
are involved, as in, the read_lock() is invoked only on CPU 0 whereas the
write_lock() is invoked on CPU 1.

For example, the same scenario shown above, but with slightly different
timing, will NOT result in a deadlock:

Scenario 2:
  CPU 0                                CPU 1

read_lock(&rwlock)


read_lock(&rwlock) //doesn't spin

                                    write_lock(&rwlock) //spins, because CPU 0
                                    //has acquired the lock for read


So I was wondering whether the "fairness" logic of rwlocks would cause
the second read_lock() to spin (in the first scenario shown above) because
a writer is already waiting (and hence new readers should spin) and thus
cause a deadlock.

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Namhyung Kim]

On Thu, 24 Jan 2013 10:00:04 +0530, Srivatsa S. Bhat wrote:
> On 01/24/2013 01:27 AM, Tejun Heo wrote:
>> On Thu, Jan 24, 2013 at 01:03:52AM +0530, Srivatsa S. Bhat wrote:
>>> CPU 0                          CPU 1
>>>
>>> read_lock(&rwlock)
>>>
>>>                               write_lock(&rwlock) //spins, because CPU 0
>>>                               //has acquired the lock for read
>>>
>>> read_lock(&rwlock)
>>>    ^^^^^
>>> What happens here? Does CPU 0 start spinning (and hence deadlock) or will
>>> it continue realizing that it already holds the rwlock for read?
>> 
>> I don't think rwlock allows nesting write lock inside read lock.
>> read_lock(); write_lock() will always deadlock.
>> 
>
> Sure, I understand that :-) My question was, what happens when *two* CPUs
> are involved, as in, the read_lock() is invoked only on CPU 0 whereas the
> write_lock() is invoked on CPU 1.
>
> For example, the same scenario shown above, but with slightly different
> timing, will NOT result in a deadlock:
>
> Scenario 2:
>   CPU 0                                CPU 1
>
> read_lock(&rwlock)
>
>
> read_lock(&rwlock) //doesn't spin
>
>                                     write_lock(&rwlock) //spins, because CPU 0
>                                     //has acquired the lock for read
>
>
> So I was wondering whether the "fairness" logic of rwlocks would cause
> the second read_lock() to spin (in the first scenario shown above) because
> a writer is already waiting (and hence new readers should spin) and thus
> cause a deadlock.

In my understanding, current x86 rwlock does basically this (of course,
in an atomic fashion):


#define RW_LOCK_BIAS 0x10000

rwlock_init(rwlock)
{
	rwlock->lock = RW_LOCK_BIAS;
}

arch_read_lock(rwlock)
{
retry:
	if (--rwlock->lock >= 0)
		return;

        rwlock->lock++;
        while (rwlock->lock < 1)
        	continue;

        goto retry;
}

arch_write_lock(rwlock)
{
retry:
	if ((rwlock->lock -= RW_LOCK_BIAS) == 0)
        	return;

        rwlock->lock += RW_LOCK_BIAS;
	while (rwlock->lock != RW_LOCK_BIAS)
		continue;

        goto retry;
}


So I can't find where the 'fairness' logic comes from..

Thanks,
Namhyung

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Paul E. McKenney]

On Tue, Jan 29, 2013 at 08:12:37PM +0900, Namhyung Kim wrote:
> On Thu, 24 Jan 2013 10:00:04 +0530, Srivatsa S. Bhat wrote:
> > On 01/24/2013 01:27 AM, Tejun Heo wrote:
> >> On Thu, Jan 24, 2013 at 01:03:52AM +0530, Srivatsa S. Bhat wrote:
> >>> CPU 0                          CPU 1
> >>>
> >>> read_lock(&rwlock)
> >>>
> >>>                               write_lock(&rwlock) //spins, because CPU 0
> >>>                               //has acquired the lock for read
> >>>
> >>> read_lock(&rwlock)
> >>>    ^^^^^
> >>> What happens here? Does CPU 0 start spinning (and hence deadlock) or will
> >>> it continue realizing that it already holds the rwlock for read?
> >> 
> >> I don't think rwlock allows nesting write lock inside read lock.
> >> read_lock(); write_lock() will always deadlock.
> >> 
> >
> > Sure, I understand that :-) My question was, what happens when *two* CPUs
> > are involved, as in, the read_lock() is invoked only on CPU 0 whereas the
> > write_lock() is invoked on CPU 1.
> >
> > For example, the same scenario shown above, but with slightly different
> > timing, will NOT result in a deadlock:
> >
> > Scenario 2:
> >   CPU 0                                CPU 1
> >
> > read_lock(&rwlock)
> >
> >
> > read_lock(&rwlock) //doesn't spin
> >
> >                                     write_lock(&rwlock) //spins, because CPU 0
> >                                     //has acquired the lock for read
> >
> >
> > So I was wondering whether the "fairness" logic of rwlocks would cause
> > the second read_lock() to spin (in the first scenario shown above) because
> > a writer is already waiting (and hence new readers should spin) and thus
> > cause a deadlock.
> 
> In my understanding, current x86 rwlock does basically this (of course,
> in an atomic fashion):
> 
> 
> #define RW_LOCK_BIAS 0x10000
> 
> rwlock_init(rwlock)
> {
> 	rwlock->lock = RW_LOCK_BIAS;
> }
> 
> arch_read_lock(rwlock)
> {
> retry:
> 	if (--rwlock->lock >= 0)
> 		return;
> 
>         rwlock->lock++;
>         while (rwlock->lock < 1)
>         	continue;
> 
>         goto retry;
> }
> 
> arch_write_lock(rwlock)
> {
> retry:
> 	if ((rwlock->lock -= RW_LOCK_BIAS) == 0)
>         	return;
> 
>         rwlock->lock += RW_LOCK_BIAS;
> 	while (rwlock->lock != RW_LOCK_BIAS)
> 		continue;
> 
>         goto retry;
> }
> 
> 
> So I can't find where the 'fairness' logic comes from..

I looked through several of the rwlock implementations, and in all of
them the writer backs off if it sees readers -- or refrains from asserting
ownership of the lock to begin with.

So it should be OK to use rwlock as shown in the underlying patch.

							Thanx, Paul

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Srivatsa S. Bhat]

On 02/09/2013 04:17 AM, Paul E. McKenney wrote:
> On Tue, Jan 29, 2013 at 08:12:37PM +0900, Namhyung Kim wrote:
>> On Thu, 24 Jan 2013 10:00:04 +0530, Srivatsa S. Bhat wrote:
>>> On 01/24/2013 01:27 AM, Tejun Heo wrote:
>>>> On Thu, Jan 24, 2013 at 01:03:52AM +0530, Srivatsa S. Bhat wrote:
>>>>> CPU 0                          CPU 1
>>>>>
>>>>> read_lock(&rwlock)
>>>>>
>>>>>                               write_lock(&rwlock) //spins, because CPU 0
>>>>>                               //has acquired the lock for read
>>>>>
>>>>> read_lock(&rwlock)
>>>>>    ^^^^^
>>>>> What happens here? Does CPU 0 start spinning (and hence deadlock) or will
>>>>> it continue realizing that it already holds the rwlock for read?
>>>>
>>>> I don't think rwlock allows nesting write lock inside read lock.
>>>> read_lock(); write_lock() will always deadlock.
>>>>
>>>
>>> Sure, I understand that :-) My question was, what happens when *two* CPUs
>>> are involved, as in, the read_lock() is invoked only on CPU 0 whereas the
>>> write_lock() is invoked on CPU 1.
>>>
>>> For example, the same scenario shown above, but with slightly different
>>> timing, will NOT result in a deadlock:
>>>
>>> Scenario 2:
>>>   CPU 0                                CPU 1
>>>
>>> read_lock(&rwlock)
>>>
>>>
>>> read_lock(&rwlock) //doesn't spin
>>>
>>>                                     write_lock(&rwlock) //spins, because CPU 0
>>>                                     //has acquired the lock for read
>>>
>>>
>>> So I was wondering whether the "fairness" logic of rwlocks would cause
>>> the second read_lock() to spin (in the first scenario shown above) because
>>> a writer is already waiting (and hence new readers should spin) and thus
>>> cause a deadlock.
>>
>> In my understanding, current x86 rwlock does basically this (of course,
>> in an atomic fashion):
>>
>>
>> #define RW_LOCK_BIAS 0x10000
>>
>> rwlock_init(rwlock)
>> {
>> 	rwlock->lock = RW_LOCK_BIAS;
>> }
>>
>> arch_read_lock(rwlock)
>> {
>> retry:
>> 	if (--rwlock->lock >= 0)
>> 		return;
>>
>>         rwlock->lock++;
>>         while (rwlock->lock < 1)
>>         	continue;
>>
>>         goto retry;
>> }
>>
>> arch_write_lock(rwlock)
>> {
>> retry:
>> 	if ((rwlock->lock -= RW_LOCK_BIAS) == 0)
>>         	return;
>>
>>         rwlock->lock += RW_LOCK_BIAS;
>> 	while (rwlock->lock != RW_LOCK_BIAS)
>> 		continue;
>>
>>         goto retry;
>> }
>>
>>
>> So I can't find where the 'fairness' logic comes from..
> 
> I looked through several of the rwlock implementations, and in all of
> them the writer backs off if it sees readers -- or refrains from asserting
> ownership of the lock to begin with.
> 
> So it should be OK to use rwlock as shown in the underlying patch.
> 

Thanks a lot for confirming that Paul! So I guess we can use rwlocks
as it is, since its behaviour suits our needs perfectly. So I won't tinker
with atomic counters for a while, atleast not until someone starts making
rwlocks fair.. ;-)

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Paul E. McKenney]

On Tue, Jan 22, 2013 at 01:03:53PM +0530, Srivatsa S. Bhat wrote:
> Using global rwlocks as the backend for per-CPU rwlocks helps us avoid many
> lock-ordering related problems (unlike per-cpu locks). However, global
> rwlocks lead to unnecessary cache-line bouncing even when there are no
> writers present, which can slow down the system needlessly.
> 
> Per-cpu counters can help solve the cache-line bouncing problem. So we
> actually use the best of both: per-cpu counters (no-waiting) at the reader
> side in the fast-path, and global rwlocks in the slowpath.
> 
> [ Fastpath = no writer is active; Slowpath = a writer is active ]
> 
> IOW, the readers just increment/decrement their per-cpu refcounts (disabling
> interrupts during the updates, if necessary) when no writer is active.
> When a writer becomes active, he signals all readers to switch to global
> rwlocks for the duration of his activity. The readers switch over when it
> is safe for them (ie., when they are about to start a fresh, non-nested
> read-side critical section) and start using (holding) the global rwlock for
> read in their subsequent critical sections.
> 
> The writer waits for every existing reader to switch, and then acquires the
> global rwlock for write and enters his critical section. Later, the writer
> signals all readers that he is done, and that they can go back to using their
> per-cpu refcounts again.
> 
> Note that the lock-safety (despite the per-cpu scheme) comes from the fact
> that the readers can *choose* _when_ to switch to rwlocks upon the writer's
> signal. And the readers don't wait on anybody based on the per-cpu counters.
> The only true synchronization that involves waiting at the reader-side in this
> scheme, is the one arising from the global rwlock, which is safe from circular
> locking dependency issues.
> 
> Reader-writer locks and per-cpu counters are recursive, so they can be
> used in a nested fashion in the reader-path, which makes per-CPU rwlocks also
> recursive. Also, this design of switching the synchronization scheme ensures
> that you can safely nest and use these locks in a very flexible manner.
> 
> I'm indebted to Michael Wang and Xiao Guangrong for their numerous thoughtful
> suggestions and ideas, which inspired and influenced many of the decisions in
> this as well as previous designs. Thanks a lot Michael and Xiao!

Looks pretty close!  Some comments interspersed below.  Please either
fix the code or my confusion, as the case may be.  ;-)

							Thanx, Paul

> Cc: David Howells <dhowells@redhat.com>
> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
> ---
> 
>  include/linux/percpu-rwlock.h |   10 +++
>  lib/percpu-rwlock.c           |  128 ++++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 136 insertions(+), 2 deletions(-)
> 
> diff --git a/include/linux/percpu-rwlock.h b/include/linux/percpu-rwlock.h
> index 8dec8fe..6819bb8 100644
> --- a/include/linux/percpu-rwlock.h
> +++ b/include/linux/percpu-rwlock.h
> @@ -68,4 +68,14 @@ extern void percpu_free_rwlock(struct percpu_rwlock *);
>  	__percpu_init_rwlock(pcpu_rwlock, #pcpu_rwlock, &rwlock_key);	\
>  })
> 
> +#define reader_uses_percpu_refcnt(pcpu_rwlock, cpu)			\
> +		(ACCESS_ONCE(per_cpu(*((pcpu_rwlock)->reader_refcnt), cpu)))
> +
> +#define reader_nested_percpu(pcpu_rwlock)				\
> +			(__this_cpu_read(*((pcpu_rwlock)->reader_refcnt)) > 1)
> +
> +#define writer_active(pcpu_rwlock)					\
> +			(__this_cpu_read(*((pcpu_rwlock)->writer_signal)))
> +
>  #endif
> +
> diff --git a/lib/percpu-rwlock.c b/lib/percpu-rwlock.c
> index 80dad93..992da5c 100644
> --- a/lib/percpu-rwlock.c
> +++ b/lib/percpu-rwlock.c
> @@ -64,21 +64,145 @@ void percpu_free_rwlock(struct percpu_rwlock *pcpu_rwlock)
> 
>  void percpu_read_lock(struct percpu_rwlock *pcpu_rwlock)
>  {
> -	read_lock(&pcpu_rwlock->global_rwlock);
> +	preempt_disable();
> +
> +	/* First and foremost, let the writer know that a reader is active */
> +	this_cpu_inc(*pcpu_rwlock->reader_refcnt);
> +
> +	/*
> +	 * If we are already using per-cpu refcounts, it is not safe to switch
> +	 * the synchronization scheme. So continue using the refcounts.
> +	 */
> +	if (reader_nested_percpu(pcpu_rwlock)) {
> +		goto out;
> +	} else {
> +		/*
> +		 * The write to 'reader_refcnt' must be visible before we
> +		 * read 'writer_signal'.
> +		 */
> +		smp_mb(); /* Paired with smp_rmb() in sync_reader() */
> +
> +		if (likely(!writer_active(pcpu_rwlock))) {
> +			goto out;
> +		} else {
> +			/* Writer is active, so switch to global rwlock. */
> +			read_lock(&pcpu_rwlock->global_rwlock);
> +
> +			/*
> +			 * We might have raced with a writer going inactive
> +			 * before we took the read-lock. So re-evaluate whether
> +			 * we still need to hold the rwlock or if we can switch
> +			 * back to per-cpu refcounts. (This also helps avoid
> +			 * heterogeneous nesting of readers).
> +			 */
> +			if (writer_active(pcpu_rwlock))

The above writer_active() can be reordered with the following this_cpu_dec(),
strange though it might seem.  But this is OK because holding the rwlock
is conservative.  But might be worth a comment.

> +				this_cpu_dec(*pcpu_rwlock->reader_refcnt);
> +			else

In contrast, no reordering can happen here because read_unlock() is
required to keep the critical section underneath the lock.

> +				read_unlock(&pcpu_rwlock->global_rwlock);
> +		}
> +	}
> +
> +out:
> +	/* Prevent reordering of any subsequent reads */
> +	smp_rmb();

This should be smp_mb().  "Readers" really can do writes.  Hence the
name lglock -- "local/global" rather than "reader/writer".

>  }
> 
>  void percpu_read_unlock(struct percpu_rwlock *pcpu_rwlock)
>  {
> -	read_unlock(&pcpu_rwlock->global_rwlock);

We need an smp_mb() here to keep the critical section ordered before the
this_cpu_dec() below.  Otherwise, if a writer shows up just after we
exit the fastpath, that writer is not guaranteed to see the effects of
our critical section.  Equivalently, the prior read-side critical section
just might see some of the writer's updates, which could be a bit of
a surprise to the reader.

> +	/*
> +	 * We never allow heterogeneous nesting of readers. So it is trivial
> +	 * to find out the kind of reader we are, and undo the operation
> +	 * done by our corresponding percpu_read_lock().
> +	 */
> +	if (__this_cpu_read(*pcpu_rwlock->reader_refcnt)) {
> +		this_cpu_dec(*pcpu_rwlock->reader_refcnt);
> +		smp_wmb(); /* Paired with smp_rmb() in sync_reader() */

Given an smp_mb() above, I don't understand the need for this smp_wmb().
Isn't the idea that if the writer sees ->reader_refcnt decremented to
zero, it also needs to see the effects of the corresponding reader's
critical section?

Or am I missing something subtle here?  In any case, if this smp_wmb()
really is needed, there should be some subsequent write that the writer
might observe.  From what I can see, there is no subsequent write from
this reader that the writer cares about.

> +	} else {
> +		read_unlock(&pcpu_rwlock->global_rwlock);
> +	}
> +
> +	preempt_enable();
> +}
> +
> +static inline void raise_writer_signal(struct percpu_rwlock *pcpu_rwlock,
> +				       unsigned int cpu)
> +{
> +	per_cpu(*pcpu_rwlock->writer_signal, cpu) = true;
> +}
> +
> +static inline void drop_writer_signal(struct percpu_rwlock *pcpu_rwlock,
> +				      unsigned int cpu)
> +{
> +	per_cpu(*pcpu_rwlock->writer_signal, cpu) = false;
> +}
> +
> +static void announce_writer_active(struct percpu_rwlock *pcpu_rwlock)
> +{
> +	unsigned int cpu;
> +
> +	for_each_online_cpu(cpu)
> +		raise_writer_signal(pcpu_rwlock, cpu);
> +
> +	smp_mb(); /* Paired with smp_rmb() in percpu_read_[un]lock() */
> +}
> +
> +static void announce_writer_inactive(struct percpu_rwlock *pcpu_rwlock)
> +{
> +	unsigned int cpu;
> +
> +	drop_writer_signal(pcpu_rwlock, smp_processor_id());

Why do we drop ourselves twice?  More to the point, why is it important to
drop ourselves first?

> +
> +	for_each_online_cpu(cpu)
> +		drop_writer_signal(pcpu_rwlock, cpu);
> +
> +	smp_mb(); /* Paired with smp_rmb() in percpu_read_[un]lock() */
> +}
> +
> +/*
> + * Wait for the reader to see the writer's signal and switch from percpu
> + * refcounts to global rwlock.
> + *
> + * If the reader is still using percpu refcounts, wait for him to switch.
> + * Else, we can safely go ahead, because either the reader has already
> + * switched over, or the next reader that comes along on that CPU will
> + * notice the writer's signal and will switch over to the rwlock.
> + */
> +static inline void sync_reader(struct percpu_rwlock *pcpu_rwlock,
> +			       unsigned int cpu)
> +{
> +	smp_rmb(); /* Paired with smp_[w]mb() in percpu_read_[un]lock() */

As I understand it, the purpose of this memory barrier is to ensure
that the stores in drop_writer_signal() happen before the reads from
->reader_refcnt in reader_uses_percpu_refcnt(), thus preventing the
race between a new reader attempting to use the fastpath and this writer
acquiring the lock.  Unless I am confused, this must be smp_mb() rather
than smp_rmb().

Also, why not just have a single smp_mb() at the beginning of
sync_all_readers() instead of executing one barrier per CPU?

> +
> +	while (reader_uses_percpu_refcnt(pcpu_rwlock, cpu))
> +		cpu_relax();
> +}
> +
> +static void sync_all_readers(struct percpu_rwlock *pcpu_rwlock)
> +{
> +	unsigned int cpu;
> +
> +	for_each_online_cpu(cpu)
> +		sync_reader(pcpu_rwlock, cpu);
>  }
> 
>  void percpu_write_lock(struct percpu_rwlock *pcpu_rwlock)
>  {
> +	/*
> +	 * Tell all readers that a writer is becoming active, so that they
> +	 * start switching over to the global rwlock.
> +	 */
> +	announce_writer_active(pcpu_rwlock);
> +	sync_all_readers(pcpu_rwlock);
>  	write_lock(&pcpu_rwlock->global_rwlock);
>  }
> 
>  void percpu_write_unlock(struct percpu_rwlock *pcpu_rwlock)
>  {
> +	/*
> +	 * Inform all readers that we are done, so that they can switch back
> +	 * to their per-cpu refcounts. (We don't need to wait for them to
> +	 * see it).
> +	 */
> +	announce_writer_inactive(pcpu_rwlock);
>  	write_unlock(&pcpu_rwlock->global_rwlock);
>  }
> 
> 

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Oleg Nesterov]

On 02/08, Paul E. McKenney wrote:
>
> On Tue, Jan 22, 2013 at 01:03:53PM +0530, Srivatsa S. Bhat wrote:
> >
> >  void percpu_read_unlock(struct percpu_rwlock *pcpu_rwlock)
> >  {
> > -	read_unlock(&pcpu_rwlock->global_rwlock);
>
> We need an smp_mb() here to keep the critical section ordered before the
> this_cpu_dec() below.  Otherwise, if a writer shows up just after we
> exit the fastpath, that writer is not guaranteed to see the effects of
> our critical section.  Equivalently, the prior read-side critical section
> just might see some of the writer's updates, which could be a bit of
> a surprise to the reader.

Agreed, we should not assume that a "reader" doesn't write. And we should
ensure that this "read" section actually completes before this_cpu_dec().

> > +	/*
> > +	 * We never allow heterogeneous nesting of readers. So it is trivial
> > +	 * to find out the kind of reader we are, and undo the operation
> > +	 * done by our corresponding percpu_read_lock().
> > +	 */
> > +	if (__this_cpu_read(*pcpu_rwlock->reader_refcnt)) {
> > +		this_cpu_dec(*pcpu_rwlock->reader_refcnt);
> > +		smp_wmb(); /* Paired with smp_rmb() in sync_reader() */
>
> Given an smp_mb() above, I don't understand the need for this smp_wmb().
> Isn't the idea that if the writer sees ->reader_refcnt decremented to
> zero, it also needs to see the effects of the corresponding reader's
> critical section?

I am equally confused ;)

OTOH, we can probably aboid any barrier if reader_nested_percpu() == T.


> > +static void announce_writer_inactive(struct percpu_rwlock *pcpu_rwlock)
> > +{
> > +   unsigned int cpu;
> > +
> > +   drop_writer_signal(pcpu_rwlock, smp_processor_id());
>
> Why do we drop ourselves twice?  More to the point, why is it important to
> drop ourselves first?

And don't we need mb() _before_ we clear ->writer_signal ?

> > +static inline void sync_reader(struct percpu_rwlock *pcpu_rwlock,
> > +			       unsigned int cpu)
> > +{
> > +	smp_rmb(); /* Paired with smp_[w]mb() in percpu_read_[un]lock() */
>
> As I understand it, the purpose of this memory barrier is to ensure
> that the stores in drop_writer_signal() happen before the reads from
> ->reader_refcnt in reader_uses_percpu_refcnt(), thus preventing the
> race between a new reader attempting to use the fastpath and this writer
> acquiring the lock.  Unless I am confused, this must be smp_mb() rather
> than smp_rmb().

And note that before sync_reader() we call announce_writer_active() which
already adds mb() before sync_all_readers/sync_reader, so this rmb() looks
unneeded.

But, at the same time, could you confirm that we do not need another mb()
after sync_all_readers() in percpu_write_lock() ? I mean, without mb(),
can't this reader_uses_percpu_refcnt() LOAD leak into the critical section
protected by ->global_rwlock? Then this LOAD can be re-ordered with other
memory operations done by the writer.



Srivatsa, I think that the code would be more understandable if you kill
the helpers like sync_reader/raise_writer_signal. Perhaps even all "write"
helpers, I am not sure. At least, it seems to me that all barriers should
be moved to percpu_write_lock/unlock. But I won't insist of course, up to
you.

And cosmetic nit... How about

	struct xxx {
		unsigned long	reader_refcnt;
		bool		writer_signal;
	}

	struct percpu_rwlock {
		struct xxx __percpu	*xxx;
		rwlock_t		global_rwlock;
	};

?

This saves one alloc_percpu() and ensures that reader_refcnt/writer_signal
are always in the same cache-line.

Oleg.

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Srivatsa S. Bhat]

On 02/10/2013 11:36 PM, Oleg Nesterov wrote:
> On 02/08, Paul E. McKenney wrote:
>>
>> On Tue, Jan 22, 2013 at 01:03:53PM +0530, Srivatsa S. Bhat wrote:
>>>
>>>  void percpu_read_unlock(struct percpu_rwlock *pcpu_rwlock)
>>>  {
>>> -	read_unlock(&pcpu_rwlock->global_rwlock);
>>
>> We need an smp_mb() here to keep the critical section ordered before the
>> this_cpu_dec() below.  Otherwise, if a writer shows up just after we
>> exit the fastpath, that writer is not guaranteed to see the effects of
>> our critical section.  Equivalently, the prior read-side critical section
>> just might see some of the writer's updates, which could be a bit of
>> a surprise to the reader.
> 
> Agreed, we should not assume that a "reader" doesn't write. And we should
> ensure that this "read" section actually completes before this_cpu_dec().
>

Right, will fix.
 
>>> +	/*
>>> +	 * We never allow heterogeneous nesting of readers. So it is trivial
>>> +	 * to find out the kind of reader we are, and undo the operation
>>> +	 * done by our corresponding percpu_read_lock().
>>> +	 */
>>> +	if (__this_cpu_read(*pcpu_rwlock->reader_refcnt)) {
>>> +		this_cpu_dec(*pcpu_rwlock->reader_refcnt);
>>> +		smp_wmb(); /* Paired with smp_rmb() in sync_reader() */
>>
>> Given an smp_mb() above, I don't understand the need for this smp_wmb().
>> Isn't the idea that if the writer sees ->reader_refcnt decremented to
>> zero, it also needs to see the effects of the corresponding reader's
>> critical section?
> 
> I am equally confused ;)
> 
> OTOH, we can probably aboid any barrier if reader_nested_percpu() == T.
> 

Good point! Will add that optimization, thank you!

> 
>>> +static void announce_writer_inactive(struct percpu_rwlock *pcpu_rwlock)
>>> +{
>>> +   unsigned int cpu;
>>> +
>>> +   drop_writer_signal(pcpu_rwlock, smp_processor_id());
>>
>> Why do we drop ourselves twice?  More to the point, why is it important to
>> drop ourselves first?
> 
> And don't we need mb() _before_ we clear ->writer_signal ?
> 

Oh, right! Or, how about moving announce_writer_inactive() to _after_
write_unlock()?

>>> +static inline void sync_reader(struct percpu_rwlock *pcpu_rwlock,
>>> +			       unsigned int cpu)
>>> +{
>>> +	smp_rmb(); /* Paired with smp_[w]mb() in percpu_read_[un]lock() */
>>
>> As I understand it, the purpose of this memory barrier is to ensure
>> that the stores in drop_writer_signal() happen before the reads from
>> ->reader_refcnt in reader_uses_percpu_refcnt(), thus preventing the
>> race between a new reader attempting to use the fastpath and this writer
>> acquiring the lock.  Unless I am confused, this must be smp_mb() rather
>> than smp_rmb().
> 
> And note that before sync_reader() we call announce_writer_active() which
> already adds mb() before sync_all_readers/sync_reader, so this rmb() looks
> unneeded.
> 

My intention was to help the writer see the ->reader_refcnt drop to zero
ASAP; hence I used smp_wmb() at reader and smp_rmb() here at the writer.
Please correct me if my understanding of memory barriers is wrong here..

> But, at the same time, could you confirm that we do not need another mb()
> after sync_all_readers() in percpu_write_lock() ? I mean, without mb(),
> can't this reader_uses_percpu_refcnt() LOAD leak into the critical section
> protected by ->global_rwlock? Then this LOAD can be re-ordered with other
> memory operations done by the writer.
> 

Hmm.. it appears that we need a smp_mb() there.

> 
> 
> Srivatsa, I think that the code would be more understandable if you kill
> the helpers like sync_reader/raise_writer_signal. Perhaps even all "write"
> helpers, I am not sure. At least, it seems to me that all barriers should
> be moved to percpu_write_lock/unlock. But I won't insist of course, up to
> you.
> 

Sure, sure. Even Tejun pointed out that those helpers are getting in the way
of readability. I'll get rid of them in the next version.

> And cosmetic nit... How about
> 
> 	struct xxx {
> 		unsigned long	reader_refcnt;
> 		bool		writer_signal;
> 	}
> 
> 	struct percpu_rwlock {
> 		struct xxx __percpu	*xxx;
> 		rwlock_t		global_rwlock;
> 	};
> 
> ?
> 
> This saves one alloc_percpu() and ensures that reader_refcnt/writer_signal
> are always in the same cache-line.
>

Ok, that sounds better. Will make that change. Thanks a lot Oleg!

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Oleg Nesterov]

On 02/11, Srivatsa S. Bhat wrote:
>
> On 02/10/2013 11:36 PM, Oleg Nesterov wrote:
> >>> +static void announce_writer_inactive(struct percpu_rwlock *pcpu_rwlock)
> >>> +{
> >>> +   unsigned int cpu;
> >>> +
> >>> +   drop_writer_signal(pcpu_rwlock, smp_processor_id());
> >>
> >> Why do we drop ourselves twice?  More to the point, why is it important to
> >> drop ourselves first?
> >
> > And don't we need mb() _before_ we clear ->writer_signal ?
> >
>
> Oh, right! Or, how about moving announce_writer_inactive() to _after_
> write_unlock()?

Not sure this will help... but, either way it seems we have another
problem...

percpu_rwlock tries to be "generic". This means we should "ignore" its
usage in hotplug, and _write_lock should not race with _write_unlock.

IOW. Suppose that _write_unlock clears ->writer_signal. We need to ensure
that this can't race with another write which wants to set this flag.
Perhaps it should be counter as well, and it should be protected by
the same ->global_rwlock, but _write_lock() should drop it before
sync_all_readers() and then take it again?

> >>> +static inline void sync_reader(struct percpu_rwlock *pcpu_rwlock,
> >>> +			       unsigned int cpu)
> >>> +{
> >>> +	smp_rmb(); /* Paired with smp_[w]mb() in percpu_read_[un]lock() */
> >>
> >> As I understand it, the purpose of this memory barrier is to ensure
> >> that the stores in drop_writer_signal() happen before the reads from
> >> ->reader_refcnt in reader_uses_percpu_refcnt(), thus preventing the
> >> race between a new reader attempting to use the fastpath and this writer
> >> acquiring the lock.  Unless I am confused, this must be smp_mb() rather
> >> than smp_rmb().
> >
> > And note that before sync_reader() we call announce_writer_active() which
> > already adds mb() before sync_all_readers/sync_reader, so this rmb() looks
> > unneeded.
> >
>
> My intention was to help the writer see the ->reader_refcnt drop to zero
> ASAP; hence I used smp_wmb() at reader and smp_rmb() here at the writer.

Hmm, interesting... Not sure, but can't really comment. However I can
answer your next question:

> Please correct me if my understanding of memory barriers is wrong here..

Who? Me??? No we have paulmck for that ;)

Oleg.

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Srivatsa S. Bhat]

On 02/11/2013 01:20 AM, Oleg Nesterov wrote:
> On 02/11, Srivatsa S. Bhat wrote:
>>
>> On 02/10/2013 11:36 PM, Oleg Nesterov wrote:
>>>>> +static void announce_writer_inactive(struct percpu_rwlock *pcpu_rwlock)
>>>>> +{
>>>>> +   unsigned int cpu;
>>>>> +
>>>>> +   drop_writer_signal(pcpu_rwlock, smp_processor_id());
>>>>
>>>> Why do we drop ourselves twice?  More to the point, why is it important to
>>>> drop ourselves first?
>>>
>>> And don't we need mb() _before_ we clear ->writer_signal ?
>>>
>>
>> Oh, right! Or, how about moving announce_writer_inactive() to _after_
>> write_unlock()?
> 
> Not sure this will help... but, either way it seems we have another
> problem...
> 
> percpu_rwlock tries to be "generic". This means we should "ignore" its
> usage in hotplug, and _write_lock should not race with _write_unlock.
> 

Yes, good point!

> IOW. Suppose that _write_unlock clears ->writer_signal. We need to ensure
> that this can't race with another write which wants to set this flag.
> Perhaps it should be counter as well, and it should be protected by
> the same ->global_rwlock, but _write_lock() should drop it before
> sync_all_readers() and then take it again?

Hmm, or we could just add an extra mb() like you suggested, and keep it
simple...

> 
>>>>> +static inline void sync_reader(struct percpu_rwlock *pcpu_rwlock,
>>>>> +			       unsigned int cpu)
>>>>> +{
>>>>> +	smp_rmb(); /* Paired with smp_[w]mb() in percpu_read_[un]lock() */
>>>>
>>>> As I understand it, the purpose of this memory barrier is to ensure
>>>> that the stores in drop_writer_signal() happen before the reads from
>>>> ->reader_refcnt in reader_uses_percpu_refcnt(), thus preventing the
>>>> race between a new reader attempting to use the fastpath and this writer
>>>> acquiring the lock.  Unless I am confused, this must be smp_mb() rather
>>>> than smp_rmb().
>>>
>>> And note that before sync_reader() we call announce_writer_active() which
>>> already adds mb() before sync_all_readers/sync_reader, so this rmb() looks
>>> unneeded.
>>>
>>
>> My intention was to help the writer see the ->reader_refcnt drop to zero
>> ASAP; hence I used smp_wmb() at reader and smp_rmb() here at the writer.
> 
> Hmm, interesting... Not sure, but can't really comment. However I can
> answer your next question:
>

Paul told me in another mail that I was expecting too much out of memory
barriers, like increasing the speed of electrons and what not ;-)
[ It would have been cool though, if it had such magical powers :P ]
 
>> Please correct me if my understanding of memory barriers is wrong here..
> 
> Who? Me??? No we have paulmck for that ;)
>

Haha ;-)

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Paul E. McKenney]

On Mon, Feb 11, 2013 at 01:39:24AM +0530, Srivatsa S. Bhat wrote:
> On 02/11/2013 01:20 AM, Oleg Nesterov wrote:
> > On 02/11, Srivatsa S. Bhat wrote:
> >>
> >> On 02/10/2013 11:36 PM, Oleg Nesterov wrote:
> >>>>> +static void announce_writer_inactive(struct percpu_rwlock *pcpu_rwlock)
> >>>>> +{
> >>>>> +   unsigned int cpu;
> >>>>> +
> >>>>> +   drop_writer_signal(pcpu_rwlock, smp_processor_id());
> >>>>
> >>>> Why do we drop ourselves twice?  More to the point, why is it important to
> >>>> drop ourselves first?
> >>>
> >>> And don't we need mb() _before_ we clear ->writer_signal ?
> >>>
> >>
> >> Oh, right! Or, how about moving announce_writer_inactive() to _after_
> >> write_unlock()?
> > 
> > Not sure this will help... but, either way it seems we have another
> > problem...
> > 
> > percpu_rwlock tries to be "generic". This means we should "ignore" its
> > usage in hotplug, and _write_lock should not race with _write_unlock.
> > 
> 
> Yes, good point!
> 
> > IOW. Suppose that _write_unlock clears ->writer_signal. We need to ensure
> > that this can't race with another write which wants to set this flag.
> > Perhaps it should be counter as well, and it should be protected by
> > the same ->global_rwlock, but _write_lock() should drop it before
> > sync_all_readers() and then take it again?
> 
> Hmm, or we could just add an extra mb() like you suggested, and keep it
> simple...
> 
> > 
> >>>>> +static inline void sync_reader(struct percpu_rwlock *pcpu_rwlock,
> >>>>> +			       unsigned int cpu)
> >>>>> +{
> >>>>> +	smp_rmb(); /* Paired with smp_[w]mb() in percpu_read_[un]lock() */
> >>>>
> >>>> As I understand it, the purpose of this memory barrier is to ensure
> >>>> that the stores in drop_writer_signal() happen before the reads from
> >>>> ->reader_refcnt in reader_uses_percpu_refcnt(), thus preventing the
> >>>> race between a new reader attempting to use the fastpath and this writer
> >>>> acquiring the lock.  Unless I am confused, this must be smp_mb() rather
> >>>> than smp_rmb().
> >>>
> >>> And note that before sync_reader() we call announce_writer_active() which
> >>> already adds mb() before sync_all_readers/sync_reader, so this rmb() looks
> >>> unneeded.
> >>>
> >>
> >> My intention was to help the writer see the ->reader_refcnt drop to zero
> >> ASAP; hence I used smp_wmb() at reader and smp_rmb() here at the writer.
> > 
> > Hmm, interesting... Not sure, but can't really comment. However I can
> > answer your next question:
> 
> Paul told me in another mail that I was expecting too much out of memory
> barriers, like increasing the speed of electrons and what not ;-)
> [ It would have been cool though, if it had such magical powers :P ]

"But because you have used the special mb_tachyonic instruction, the
speed of light is 600,000 km/s for the next five clock cycles"...  ;-)

							Thanx, Paul

> >> Please correct me if my understanding of memory barriers is wrong here..
> > 
> > Who? Me??? No we have paulmck for that ;)
> >
> 
> Haha ;-)
> 
> Regards,
> Srivatsa S. Bhat
> 

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Paul E. McKenney]

On Sun, Feb 10, 2013 at 07:06:07PM +0100, Oleg Nesterov wrote:
> On 02/08, Paul E. McKenney wrote:

[ . . . ]

> > > +static inline void sync_reader(struct percpu_rwlock *pcpu_rwlock,
> > > +			       unsigned int cpu)
> > > +{
> > > +	smp_rmb(); /* Paired with smp_[w]mb() in percpu_read_[un]lock() */
> >
> > As I understand it, the purpose of this memory barrier is to ensure
> > that the stores in drop_writer_signal() happen before the reads from
> > ->reader_refcnt in reader_uses_percpu_refcnt(), thus preventing the
> > race between a new reader attempting to use the fastpath and this writer
> > acquiring the lock.  Unless I am confused, this must be smp_mb() rather
> > than smp_rmb().
> 
> And note that before sync_reader() we call announce_writer_active() which
> already adds mb() before sync_all_readers/sync_reader, so this rmb() looks
> unneeded.
> 
> But, at the same time, could you confirm that we do not need another mb()
> after sync_all_readers() in percpu_write_lock() ? I mean, without mb(),
> can't this reader_uses_percpu_refcnt() LOAD leak into the critical section
> protected by ->global_rwlock? Then this LOAD can be re-ordered with other
> memory operations done by the writer.

As soon as I get the rest of the way through Thomas's patchset.  ;-)

							Thanx, Paul

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Paul E. McKenney]

On Sun, Feb 10, 2013 at 11:54:17AM -0800, Paul E. McKenney wrote:
> On Sun, Feb 10, 2013 at 07:06:07PM +0100, Oleg Nesterov wrote:
> > On 02/08, Paul E. McKenney wrote:
> 
> [ . . . ]
> 
> > > > +static inline void sync_reader(struct percpu_rwlock *pcpu_rwlock,
> > > > +			       unsigned int cpu)
> > > > +{
> > > > +	smp_rmb(); /* Paired with smp_[w]mb() in percpu_read_[un]lock() */
> > >
> > > As I understand it, the purpose of this memory barrier is to ensure
> > > that the stores in drop_writer_signal() happen before the reads from
> > > ->reader_refcnt in reader_uses_percpu_refcnt(), thus preventing the
> > > race between a new reader attempting to use the fastpath and this writer
> > > acquiring the lock.  Unless I am confused, this must be smp_mb() rather
> > > than smp_rmb().
> > 
> > And note that before sync_reader() we call announce_writer_active() which
> > already adds mb() before sync_all_readers/sync_reader, so this rmb() looks
> > unneeded.
> > 
> > But, at the same time, could you confirm that we do not need another mb()
> > after sync_all_readers() in percpu_write_lock() ? I mean, without mb(),
> > can't this reader_uses_percpu_refcnt() LOAD leak into the critical section
> > protected by ->global_rwlock? Then this LOAD can be re-ordered with other
> > memory operations done by the writer.
> 
> As soon as I get the rest of the way through Thomas's patchset.  ;-)

There is a memory barrier associated with write_lock(), but it is
only required to keep the critical section inside the lock -- and is
permitted to allow stuff outside of the lock to be reordered into the
critical section.  So I believe we do indeed need an smp_mb() between
sync_all_readers() and write_lock() in percpu_write_lock().

Good eyes, Oleg!

							Thanx, Paul

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Srivatsa S. Bhat]

On 02/09/2013 04:40 AM, Paul E. McKenney wrote:
> On Tue, Jan 22, 2013 at 01:03:53PM +0530, Srivatsa S. Bhat wrote:
>> Using global rwlocks as the backend for per-CPU rwlocks helps us avoid many
>> lock-ordering related problems (unlike per-cpu locks). However, global
>> rwlocks lead to unnecessary cache-line bouncing even when there are no
>> writers present, which can slow down the system needlessly.
>>
[...]
> 
> Looks pretty close!  Some comments interspersed below.  Please either
> fix the code or my confusion, as the case may be.  ;-)
> 

Sure :-)

>> ---
>>
>>  include/linux/percpu-rwlock.h |   10 +++
>>  lib/percpu-rwlock.c           |  128 ++++++++++++++++++++++++++++++++++++++++-
>>  2 files changed, 136 insertions(+), 2 deletions(-)
>>
>> diff --git a/include/linux/percpu-rwlock.h b/include/linux/percpu-rwlock.h
>> index 8dec8fe..6819bb8 100644
>> --- a/include/linux/percpu-rwlock.h
>> +++ b/include/linux/percpu-rwlock.h
>> @@ -68,4 +68,14 @@ extern void percpu_free_rwlock(struct percpu_rwlock *);
>>  	__percpu_init_rwlock(pcpu_rwlock, #pcpu_rwlock, &rwlock_key);	\
>>  })
>>
>> +#define reader_uses_percpu_refcnt(pcpu_rwlock, cpu)			\
>> +		(ACCESS_ONCE(per_cpu(*((pcpu_rwlock)->reader_refcnt), cpu)))
>> +
>> +#define reader_nested_percpu(pcpu_rwlock)				\
>> +			(__this_cpu_read(*((pcpu_rwlock)->reader_refcnt)) > 1)
>> +
>> +#define writer_active(pcpu_rwlock)					\
>> +			(__this_cpu_read(*((pcpu_rwlock)->writer_signal)))
>> +
>>  #endif
>> +
>> diff --git a/lib/percpu-rwlock.c b/lib/percpu-rwlock.c
>> index 80dad93..992da5c 100644
>> --- a/lib/percpu-rwlock.c
>> +++ b/lib/percpu-rwlock.c
>> @@ -64,21 +64,145 @@ void percpu_free_rwlock(struct percpu_rwlock *pcpu_rwlock)
>>
>>  void percpu_read_lock(struct percpu_rwlock *pcpu_rwlock)
>>  {
>> -	read_lock(&pcpu_rwlock->global_rwlock);
>> +	preempt_disable();
>> +
>> +	/* First and foremost, let the writer know that a reader is active */
>> +	this_cpu_inc(*pcpu_rwlock->reader_refcnt);
>> +
>> +	/*
>> +	 * If we are already using per-cpu refcounts, it is not safe to switch
>> +	 * the synchronization scheme. So continue using the refcounts.
>> +	 */
>> +	if (reader_nested_percpu(pcpu_rwlock)) {
>> +		goto out;
>> +	} else {
>> +		/*
>> +		 * The write to 'reader_refcnt' must be visible before we
>> +		 * read 'writer_signal'.
>> +		 */
>> +		smp_mb(); /* Paired with smp_rmb() in sync_reader() */
>> +
>> +		if (likely(!writer_active(pcpu_rwlock))) {
>> +			goto out;
>> +		} else {
>> +			/* Writer is active, so switch to global rwlock. */
>> +			read_lock(&pcpu_rwlock->global_rwlock);
>> +
>> +			/*
>> +			 * We might have raced with a writer going inactive
>> +			 * before we took the read-lock. So re-evaluate whether
>> +			 * we still need to hold the rwlock or if we can switch
>> +			 * back to per-cpu refcounts. (This also helps avoid
>> +			 * heterogeneous nesting of readers).
>> +			 */
>> +			if (writer_active(pcpu_rwlock))
> 
> The above writer_active() can be reordered with the following this_cpu_dec(),
> strange though it might seem.  But this is OK because holding the rwlock
> is conservative.  But might be worth a comment.
> 

Ok..

>> +				this_cpu_dec(*pcpu_rwlock->reader_refcnt);
>> +			else
> 
> In contrast, no reordering can happen here because read_unlock() is
> required to keep the critical section underneath the lock.
>

Ok..
 
>> +				read_unlock(&pcpu_rwlock->global_rwlock);
>> +		}
>> +	}
>> +
>> +out:
>> +	/* Prevent reordering of any subsequent reads */
>> +	smp_rmb();
> 
> This should be smp_mb().  "Readers" really can do writes.  Hence the
> name lglock -- "local/global" rather than "reader/writer".
> 

Ok!

[ We were trying to avoid full memory barriers in get/put_online_cpus_atomic()
in the fastpath, as far as possible... Now it feels like all that discussion
was for nothing :-( ]

>>  }
>>
>>  void percpu_read_unlock(struct percpu_rwlock *pcpu_rwlock)
>>  {
>> -	read_unlock(&pcpu_rwlock->global_rwlock);
> 
> We need an smp_mb() here to keep the critical section ordered before the
> this_cpu_dec() below.  Otherwise, if a writer shows up just after we
> exit the fastpath, that writer is not guaranteed to see the effects of
> our critical section.  Equivalently, the prior read-side critical section
> just might see some of the writer's updates, which could be a bit of
> a surprise to the reader.
> 

Ok, will add it.

>> +	/*
>> +	 * We never allow heterogeneous nesting of readers. So it is trivial
>> +	 * to find out the kind of reader we are, and undo the operation
>> +	 * done by our corresponding percpu_read_lock().
>> +	 */
>> +	if (__this_cpu_read(*pcpu_rwlock->reader_refcnt)) {
>> +		this_cpu_dec(*pcpu_rwlock->reader_refcnt);
>> +		smp_wmb(); /* Paired with smp_rmb() in sync_reader() */
> 
> Given an smp_mb() above, I don't understand the need for this smp_wmb().
> Isn't the idea that if the writer sees ->reader_refcnt decremented to
> zero, it also needs to see the effects of the corresponding reader's
> critical section?
>

Not sure what you meant, but my idea here was that the writer should see
the reader_refcnt falling to zero as soon as possible, to avoid keeping the
writer waiting in a tight loop for longer than necessary.
I might have been a little over-zealous to use lighter memory barriers though,
(given our lengthy discussions in the previous versions to reduce the memory
barrier overheads), so the smp_wmb() used above might be wrong.

So, are you saying that the smp_mb() you indicated above would be enough
to make the writer observe the 1->0 transition of reader_refcnt immediately?
 
> Or am I missing something subtle here?  In any case, if this smp_wmb()
> really is needed, there should be some subsequent write that the writer
> might observe.  From what I can see, there is no subsequent write from
> this reader that the writer cares about.
> 

I thought the smp_wmb() here and the smp_rmb() at the writer would ensure
immediate reflection of the reader state at the writer side... Please correct
me if my understanding is incorrect.

>> +	} else {
>> +		read_unlock(&pcpu_rwlock->global_rwlock);
>> +	}
>> +
>> +	preempt_enable();
>> +}
>> +
>> +static inline void raise_writer_signal(struct percpu_rwlock *pcpu_rwlock,
>> +				       unsigned int cpu)
>> +{
>> +	per_cpu(*pcpu_rwlock->writer_signal, cpu) = true;
>> +}
>> +
>> +static inline void drop_writer_signal(struct percpu_rwlock *pcpu_rwlock,
>> +				      unsigned int cpu)
>> +{
>> +	per_cpu(*pcpu_rwlock->writer_signal, cpu) = false;
>> +}
>> +
>> +static void announce_writer_active(struct percpu_rwlock *pcpu_rwlock)
>> +{
>> +	unsigned int cpu;
>> +
>> +	for_each_online_cpu(cpu)
>> +		raise_writer_signal(pcpu_rwlock, cpu);
>> +
>> +	smp_mb(); /* Paired with smp_rmb() in percpu_read_[un]lock() */
>> +}
>> +
>> +static void announce_writer_inactive(struct percpu_rwlock *pcpu_rwlock)
>> +{
>> +	unsigned int cpu;
>> +
>> +	drop_writer_signal(pcpu_rwlock, smp_processor_id());
> 
> Why do we drop ourselves twice?  More to the point, why is it important to
> drop ourselves first?
> 

I don't see where we are dropping ourselves twice. Note that we are no longer
in the cpu_online_mask, so the 'for' loop below won't include us. So we need
to manually drop ourselves. It doesn't matter whether we drop ourselves first
or later.

>> +
>> +	for_each_online_cpu(cpu)
>> +		drop_writer_signal(pcpu_rwlock, cpu);
>> +
>> +	smp_mb(); /* Paired with smp_rmb() in percpu_read_[un]lock() */
>> +}
>> +
>> +/*
>> + * Wait for the reader to see the writer's signal and switch from percpu
>> + * refcounts to global rwlock.
>> + *
>> + * If the reader is still using percpu refcounts, wait for him to switch.
>> + * Else, we can safely go ahead, because either the reader has already
>> + * switched over, or the next reader that comes along on that CPU will
>> + * notice the writer's signal and will switch over to the rwlock.
>> + */
>> +static inline void sync_reader(struct percpu_rwlock *pcpu_rwlock,
>> +			       unsigned int cpu)
>> +{
>> +	smp_rmb(); /* Paired with smp_[w]mb() in percpu_read_[un]lock() */
> 
> As I understand it, the purpose of this memory barrier is to ensure
> that the stores in drop_writer_signal() happen before the reads from
> ->reader_refcnt in reader_uses_percpu_refcnt(),

No, that was not what I intended. announce_writer_inactive() already does
a full smp_mb() after calling drop_writer_signal().

I put the smp_rmb() here and the smp_wmb() at the reader side (after updates
to the ->reader_refcnt) to reflect the state change of ->reader_refcnt
immediately at the writer, so that the writer doesn't have to keep spinning
unnecessarily still referring to the old (non-zero) value of ->reader_refcnt.
Or perhaps I am confused about how to use memory barriers properly.. :-(

> thus preventing the
> race between a new reader attempting to use the fastpath and this writer
> acquiring the lock.  Unless I am confused, this must be smp_mb() rather
> than smp_rmb().
> 
> Also, why not just have a single smp_mb() at the beginning of
> sync_all_readers() instead of executing one barrier per CPU?

Well, since my intention was to help the writer see the update (->reader_refcnt
dropping to zero) ASAP, I kept the multiple smp_rmb()s.

> 
>> +
>> +	while (reader_uses_percpu_refcnt(pcpu_rwlock, cpu))
>> +		cpu_relax();
>> +}
>> +
>> +static void sync_all_readers(struct percpu_rwlock *pcpu_rwlock)
>> +{
>> +	unsigned int cpu;
>> +
>> +	for_each_online_cpu(cpu)
>> +		sync_reader(pcpu_rwlock, cpu);
>>  }
>>
>>  void percpu_write_lock(struct percpu_rwlock *pcpu_rwlock)
>>  {
>> +	/*
>> +	 * Tell all readers that a writer is becoming active, so that they
>> +	 * start switching over to the global rwlock.
>> +	 */
>> +	announce_writer_active(pcpu_rwlock);
>> +	sync_all_readers(pcpu_rwlock);
>>  	write_lock(&pcpu_rwlock->global_rwlock);
>>  }
>>
>>  void percpu_write_unlock(struct percpu_rwlock *pcpu_rwlock)
>>  {
>> +	/*
>> +	 * Inform all readers that we are done, so that they can switch back
>> +	 * to their per-cpu refcounts. (We don't need to wait for them to
>> +	 * see it).
>> +	 */
>> +	announce_writer_inactive(pcpu_rwlock);
>>  	write_unlock(&pcpu_rwlock->global_rwlock);
>>  }
>>
>>

Thanks a lot for your detailed review and comments! :-)

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Paul E. McKenney]

On Mon, Feb 11, 2013 at 12:40:56AM +0530, Srivatsa S. Bhat wrote:
> On 02/09/2013 04:40 AM, Paul E. McKenney wrote:
> > On Tue, Jan 22, 2013 at 01:03:53PM +0530, Srivatsa S. Bhat wrote:
> >> Using global rwlocks as the backend for per-CPU rwlocks helps us avoid many
> >> lock-ordering related problems (unlike per-cpu locks). However, global
> >> rwlocks lead to unnecessary cache-line bouncing even when there are no
> >> writers present, which can slow down the system needlessly.
> >>
> [...]
> > 
> > Looks pretty close!  Some comments interspersed below.  Please either
> > fix the code or my confusion, as the case may be.  ;-)
> > 
> 
> Sure :-)
> 
> >> ---
> >>
> >>  include/linux/percpu-rwlock.h |   10 +++
> >>  lib/percpu-rwlock.c           |  128 ++++++++++++++++++++++++++++++++++++++++-
> >>  2 files changed, 136 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/include/linux/percpu-rwlock.h b/include/linux/percpu-rwlock.h
> >> index 8dec8fe..6819bb8 100644
> >> --- a/include/linux/percpu-rwlock.h
> >> +++ b/include/linux/percpu-rwlock.h
> >> @@ -68,4 +68,14 @@ extern void percpu_free_rwlock(struct percpu_rwlock *);
> >>  	__percpu_init_rwlock(pcpu_rwlock, #pcpu_rwlock, &rwlock_key);	\
> >>  })
> >>
> >> +#define reader_uses_percpu_refcnt(pcpu_rwlock, cpu)			\
> >> +		(ACCESS_ONCE(per_cpu(*((pcpu_rwlock)->reader_refcnt), cpu)))
> >> +
> >> +#define reader_nested_percpu(pcpu_rwlock)				\
> >> +			(__this_cpu_read(*((pcpu_rwlock)->reader_refcnt)) > 1)
> >> +
> >> +#define writer_active(pcpu_rwlock)					\
> >> +			(__this_cpu_read(*((pcpu_rwlock)->writer_signal)))
> >> +
> >>  #endif
> >> +
> >> diff --git a/lib/percpu-rwlock.c b/lib/percpu-rwlock.c
> >> index 80dad93..992da5c 100644
> >> --- a/lib/percpu-rwlock.c
> >> +++ b/lib/percpu-rwlock.c
> >> @@ -64,21 +64,145 @@ void percpu_free_rwlock(struct percpu_rwlock *pcpu_rwlock)
> >>
> >>  void percpu_read_lock(struct percpu_rwlock *pcpu_rwlock)
> >>  {
> >> -	read_lock(&pcpu_rwlock->global_rwlock);
> >> +	preempt_disable();
> >> +
> >> +	/* First and foremost, let the writer know that a reader is active */
> >> +	this_cpu_inc(*pcpu_rwlock->reader_refcnt);
> >> +
> >> +	/*
> >> +	 * If we are already using per-cpu refcounts, it is not safe to switch
> >> +	 * the synchronization scheme. So continue using the refcounts.
> >> +	 */
> >> +	if (reader_nested_percpu(pcpu_rwlock)) {
> >> +		goto out;
> >> +	} else {
> >> +		/*
> >> +		 * The write to 'reader_refcnt' must be visible before we
> >> +		 * read 'writer_signal'.
> >> +		 */
> >> +		smp_mb(); /* Paired with smp_rmb() in sync_reader() */
> >> +
> >> +		if (likely(!writer_active(pcpu_rwlock))) {
> >> +			goto out;
> >> +		} else {
> >> +			/* Writer is active, so switch to global rwlock. */
> >> +			read_lock(&pcpu_rwlock->global_rwlock);
> >> +
> >> +			/*
> >> +			 * We might have raced with a writer going inactive
> >> +			 * before we took the read-lock. So re-evaluate whether
> >> +			 * we still need to hold the rwlock or if we can switch
> >> +			 * back to per-cpu refcounts. (This also helps avoid
> >> +			 * heterogeneous nesting of readers).
> >> +			 */
> >> +			if (writer_active(pcpu_rwlock))
> > 
> > The above writer_active() can be reordered with the following this_cpu_dec(),
> > strange though it might seem.  But this is OK because holding the rwlock
> > is conservative.  But might be worth a comment.
> > 
> 
> Ok..
> 
> >> +				this_cpu_dec(*pcpu_rwlock->reader_refcnt);
> >> +			else
> > 
> > In contrast, no reordering can happen here because read_unlock() is
> > required to keep the critical section underneath the lock.
> >
> 
> Ok..
> 
> >> +				read_unlock(&pcpu_rwlock->global_rwlock);
> >> +		}
> >> +	}
> >> +
> >> +out:
> >> +	/* Prevent reordering of any subsequent reads */
> >> +	smp_rmb();
> > 
> > This should be smp_mb().  "Readers" really can do writes.  Hence the
> > name lglock -- "local/global" rather than "reader/writer".
> > 
> 
> Ok!
> 
> [ We were trying to avoid full memory barriers in get/put_online_cpus_atomic()
> in the fastpath, as far as possible... Now it feels like all that discussion
> was for nothing :-( ]
> 
> >>  }
> >>
> >>  void percpu_read_unlock(struct percpu_rwlock *pcpu_rwlock)
> >>  {
> >> -	read_unlock(&pcpu_rwlock->global_rwlock);
> > 
> > We need an smp_mb() here to keep the critical section ordered before the
> > this_cpu_dec() below.  Otherwise, if a writer shows up just after we
> > exit the fastpath, that writer is not guaranteed to see the effects of
> > our critical section.  Equivalently, the prior read-side critical section
> > just might see some of the writer's updates, which could be a bit of
> > a surprise to the reader.
> > 
> 
> Ok, will add it.
> 
> >> +	/*
> >> +	 * We never allow heterogeneous nesting of readers. So it is trivial
> >> +	 * to find out the kind of reader we are, and undo the operation
> >> +	 * done by our corresponding percpu_read_lock().
> >> +	 */
> >> +	if (__this_cpu_read(*pcpu_rwlock->reader_refcnt)) {
> >> +		this_cpu_dec(*pcpu_rwlock->reader_refcnt);
> >> +		smp_wmb(); /* Paired with smp_rmb() in sync_reader() */
> > 
> > Given an smp_mb() above, I don't understand the need for this smp_wmb().
> > Isn't the idea that if the writer sees ->reader_refcnt decremented to
> > zero, it also needs to see the effects of the corresponding reader's
> > critical section?
> >
> 
> Not sure what you meant, but my idea here was that the writer should see
> the reader_refcnt falling to zero as soon as possible, to avoid keeping the
> writer waiting in a tight loop for longer than necessary.
> I might have been a little over-zealous to use lighter memory barriers though,
> (given our lengthy discussions in the previous versions to reduce the memory
> barrier overheads), so the smp_wmb() used above might be wrong.
> 
> So, are you saying that the smp_mb() you indicated above would be enough
> to make the writer observe the 1->0 transition of reader_refcnt immediately?
> 
> > Or am I missing something subtle here?  In any case, if this smp_wmb()
> > really is needed, there should be some subsequent write that the writer
> > might observe.  From what I can see, there is no subsequent write from
> > this reader that the writer cares about.
> 
> I thought the smp_wmb() here and the smp_rmb() at the writer would ensure
> immediate reflection of the reader state at the writer side... Please correct
> me if my understanding is incorrect.

Ah, but memory barriers are not so much about making data move faster
through the machine, but more about making sure that ordering constraints
are met.  After all, memory barriers cannot make electrons flow faster
through silicon.  You should therefore use memory barriers only to
constrain ordering, not to try to expedite electrons.

> >> +	} else {
> >> +		read_unlock(&pcpu_rwlock->global_rwlock);
> >> +	}
> >> +
> >> +	preempt_enable();
> >> +}
> >> +
> >> +static inline void raise_writer_signal(struct percpu_rwlock *pcpu_rwlock,
> >> +				       unsigned int cpu)
> >> +{
> >> +	per_cpu(*pcpu_rwlock->writer_signal, cpu) = true;
> >> +}
> >> +
> >> +static inline void drop_writer_signal(struct percpu_rwlock *pcpu_rwlock,
> >> +				      unsigned int cpu)
> >> +{
> >> +	per_cpu(*pcpu_rwlock->writer_signal, cpu) = false;
> >> +}
> >> +
> >> +static void announce_writer_active(struct percpu_rwlock *pcpu_rwlock)
> >> +{
> >> +	unsigned int cpu;
> >> +
> >> +	for_each_online_cpu(cpu)
> >> +		raise_writer_signal(pcpu_rwlock, cpu);
> >> +
> >> +	smp_mb(); /* Paired with smp_rmb() in percpu_read_[un]lock() */
> >> +}
> >> +
> >> +static void announce_writer_inactive(struct percpu_rwlock *pcpu_rwlock)
> >> +{
> >> +	unsigned int cpu;
> >> +
> >> +	drop_writer_signal(pcpu_rwlock, smp_processor_id());
> > 
> > Why do we drop ourselves twice?  More to the point, why is it important to
> > drop ourselves first?
> 
> I don't see where we are dropping ourselves twice. Note that we are no longer
> in the cpu_online_mask, so the 'for' loop below won't include us. So we need
> to manually drop ourselves. It doesn't matter whether we drop ourselves first
> or later.

Good point, apologies for my confusion!  Still worth a commment, though.

> >> +
> >> +	for_each_online_cpu(cpu)
> >> +		drop_writer_signal(pcpu_rwlock, cpu);
> >> +
> >> +	smp_mb(); /* Paired with smp_rmb() in percpu_read_[un]lock() */
> >> +}
> >> +
> >> +/*
> >> + * Wait for the reader to see the writer's signal and switch from percpu
> >> + * refcounts to global rwlock.
> >> + *
> >> + * If the reader is still using percpu refcounts, wait for him to switch.
> >> + * Else, we can safely go ahead, because either the reader has already
> >> + * switched over, or the next reader that comes along on that CPU will
> >> + * notice the writer's signal and will switch over to the rwlock.
> >> + */
> >> +static inline void sync_reader(struct percpu_rwlock *pcpu_rwlock,
> >> +			       unsigned int cpu)
> >> +{
> >> +	smp_rmb(); /* Paired with smp_[w]mb() in percpu_read_[un]lock() */
> > 
> > As I understand it, the purpose of this memory barrier is to ensure
> > that the stores in drop_writer_signal() happen before the reads from
> > ->reader_refcnt in reader_uses_percpu_refcnt(),
> 
> No, that was not what I intended. announce_writer_inactive() already does
> a full smp_mb() after calling drop_writer_signal().
> 
> I put the smp_rmb() here and the smp_wmb() at the reader side (after updates
> to the ->reader_refcnt) to reflect the state change of ->reader_refcnt
> immediately at the writer, so that the writer doesn't have to keep spinning
> unnecessarily still referring to the old (non-zero) value of ->reader_refcnt.
> Or perhaps I am confused about how to use memory barriers properly.. :-(

Sadly, no, memory barriers don't make electrons move faster.  So you
should only need the one -- the additional memory barriers are just
slowing things down.

> > thus preventing the
> > race between a new reader attempting to use the fastpath and this writer
> > acquiring the lock.  Unless I am confused, this must be smp_mb() rather
> > than smp_rmb().
> > 
> > Also, why not just have a single smp_mb() at the beginning of
> > sync_all_readers() instead of executing one barrier per CPU?
> 
> Well, since my intention was to help the writer see the update (->reader_refcnt
> dropping to zero) ASAP, I kept the multiple smp_rmb()s.

At least you were consistent.  ;-)

> >> +
> >> +	while (reader_uses_percpu_refcnt(pcpu_rwlock, cpu))
> >> +		cpu_relax();
> >> +}
> >> +
> >> +static void sync_all_readers(struct percpu_rwlock *pcpu_rwlock)
> >> +{
> >> +	unsigned int cpu;
> >> +
> >> +	for_each_online_cpu(cpu)
> >> +		sync_reader(pcpu_rwlock, cpu);
> >>  }
> >>
> >>  void percpu_write_lock(struct percpu_rwlock *pcpu_rwlock)
> >>  {
> >> +	/*
> >> +	 * Tell all readers that a writer is becoming active, so that they
> >> +	 * start switching over to the global rwlock.
> >> +	 */
> >> +	announce_writer_active(pcpu_rwlock);
> >> +	sync_all_readers(pcpu_rwlock);
> >>  	write_lock(&pcpu_rwlock->global_rwlock);
> >>  }
> >>
> >>  void percpu_write_unlock(struct percpu_rwlock *pcpu_rwlock)
> >>  {
> >> +	/*
> >> +	 * Inform all readers that we are done, so that they can switch back
> >> +	 * to their per-cpu refcounts. (We don't need to wait for them to
> >> +	 * see it).
> >> +	 */
> >> +	announce_writer_inactive(pcpu_rwlock);
> >>  	write_unlock(&pcpu_rwlock->global_rwlock);
> >>  }
> >>
> >>
> 
> Thanks a lot for your detailed review and comments! :-)

It will be good to get this in!

							Thanx, Paul

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Srivatsa S. Bhat]

On 02/11/2013 01:17 AM, Paul E. McKenney wrote:
> On Mon, Feb 11, 2013 at 12:40:56AM +0530, Srivatsa S. Bhat wrote:
>> On 02/09/2013 04:40 AM, Paul E. McKenney wrote:
>>> On Tue, Jan 22, 2013 at 01:03:53PM +0530, Srivatsa S. Bhat wrote:
>>>> Using global rwlocks as the backend for per-CPU rwlocks helps us avoid many
>>>> lock-ordering related problems (unlike per-cpu locks). However, global
>>>> rwlocks lead to unnecessary cache-line bouncing even when there are no
>>>> writers present, which can slow down the system needlessly.
>>>>
>> [...]
>>>> +	/*
>>>> +	 * We never allow heterogeneous nesting of readers. So it is trivial
>>>> +	 * to find out the kind of reader we are, and undo the operation
>>>> +	 * done by our corresponding percpu_read_lock().
>>>> +	 */
>>>> +	if (__this_cpu_read(*pcpu_rwlock->reader_refcnt)) {
>>>> +		this_cpu_dec(*pcpu_rwlock->reader_refcnt);
>>>> +		smp_wmb(); /* Paired with smp_rmb() in sync_reader() */
>>>
>>> Given an smp_mb() above, I don't understand the need for this smp_wmb().
>>> Isn't the idea that if the writer sees ->reader_refcnt decremented to
>>> zero, it also needs to see the effects of the corresponding reader's
>>> critical section?
>>>
>>
>> Not sure what you meant, but my idea here was that the writer should see
>> the reader_refcnt falling to zero as soon as possible, to avoid keeping the
>> writer waiting in a tight loop for longer than necessary.
>> I might have been a little over-zealous to use lighter memory barriers though,
>> (given our lengthy discussions in the previous versions to reduce the memory
>> barrier overheads), so the smp_wmb() used above might be wrong.
>>
>> So, are you saying that the smp_mb() you indicated above would be enough
>> to make the writer observe the 1->0 transition of reader_refcnt immediately?
>>
>>> Or am I missing something subtle here?  In any case, if this smp_wmb()
>>> really is needed, there should be some subsequent write that the writer
>>> might observe.  From what I can see, there is no subsequent write from
>>> this reader that the writer cares about.
>>
>> I thought the smp_wmb() here and the smp_rmb() at the writer would ensure
>> immediate reflection of the reader state at the writer side... Please correct
>> me if my understanding is incorrect.
> 
> Ah, but memory barriers are not so much about making data move faster
> through the machine, but more about making sure that ordering constraints
> are met.  After all, memory barriers cannot make electrons flow faster
> through silicon.  You should therefore use memory barriers only to
> constrain ordering, not to try to expedite electrons.
>

I guess I must have been confused after looking at that graph which showed
how much time it takes for other CPUs to notice the change in value of a
variable performed in a given CPU.. and must have gotten the (wrong) idea
that memory barriers also help speed that up! Very sorry about that!
 
>>>> +	} else {
>>>> +		read_unlock(&pcpu_rwlock->global_rwlock);
>>>> +	}
>>>> +
>>>> +	preempt_enable();
>>>> +}
>>>> +
>>>> +static inline void raise_writer_signal(struct percpu_rwlock *pcpu_rwlock,
>>>> +				       unsigned int cpu)
>>>> +{
>>>> +	per_cpu(*pcpu_rwlock->writer_signal, cpu) = true;
>>>> +}
>>>> +
>>>> +static inline void drop_writer_signal(struct percpu_rwlock *pcpu_rwlock,
>>>> +				      unsigned int cpu)
>>>> +{
>>>> +	per_cpu(*pcpu_rwlock->writer_signal, cpu) = false;
>>>> +}
>>>> +
>>>> +static void announce_writer_active(struct percpu_rwlock *pcpu_rwlock)
>>>> +{
>>>> +	unsigned int cpu;
>>>> +
>>>> +	for_each_online_cpu(cpu)
>>>> +		raise_writer_signal(pcpu_rwlock, cpu);
>>>> +
>>>> +	smp_mb(); /* Paired with smp_rmb() in percpu_read_[un]lock() */
>>>> +}
>>>> +
>>>> +static void announce_writer_inactive(struct percpu_rwlock *pcpu_rwlock)
>>>> +{
>>>> +	unsigned int cpu;
>>>> +
>>>> +	drop_writer_signal(pcpu_rwlock, smp_processor_id());
>>>
>>> Why do we drop ourselves twice?  More to the point, why is it important to
>>> drop ourselves first?
>>
>> I don't see where we are dropping ourselves twice. Note that we are no longer
>> in the cpu_online_mask, so the 'for' loop below won't include us. So we need
>> to manually drop ourselves. It doesn't matter whether we drop ourselves first
>> or later.
> 
> Good point, apologies for my confusion!  Still worth a commment, though.
> 

Sure, will add it.

>>>> +
>>>> +	for_each_online_cpu(cpu)
>>>> +		drop_writer_signal(pcpu_rwlock, cpu);
>>>> +
>>>> +	smp_mb(); /* Paired with smp_rmb() in percpu_read_[un]lock() */
>>>> +}
>>>> +
>>>> +/*
>>>> + * Wait for the reader to see the writer's signal and switch from percpu
>>>> + * refcounts to global rwlock.
>>>> + *
>>>> + * If the reader is still using percpu refcounts, wait for him to switch.
>>>> + * Else, we can safely go ahead, because either the reader has already
>>>> + * switched over, or the next reader that comes along on that CPU will
>>>> + * notice the writer's signal and will switch over to the rwlock.
>>>> + */
>>>> +static inline void sync_reader(struct percpu_rwlock *pcpu_rwlock,
>>>> +			       unsigned int cpu)
>>>> +{
>>>> +	smp_rmb(); /* Paired with smp_[w]mb() in percpu_read_[un]lock() */
>>>
>>> As I understand it, the purpose of this memory barrier is to ensure
>>> that the stores in drop_writer_signal() happen before the reads from
>>> ->reader_refcnt in reader_uses_percpu_refcnt(),
>>
>> No, that was not what I intended. announce_writer_inactive() already does
>> a full smp_mb() after calling drop_writer_signal().
>>
>> I put the smp_rmb() here and the smp_wmb() at the reader side (after updates
>> to the ->reader_refcnt) to reflect the state change of ->reader_refcnt
>> immediately at the writer, so that the writer doesn't have to keep spinning
>> unnecessarily still referring to the old (non-zero) value of ->reader_refcnt.
>> Or perhaps I am confused about how to use memory barriers properly.. :-(
> 
> Sadly, no, memory barriers don't make electrons move faster.  So you
> should only need the one -- the additional memory barriers are just
> slowing things down.
> 

Ok..

>>> thus preventing the
>>> race between a new reader attempting to use the fastpath and this writer
>>> acquiring the lock.  Unless I am confused, this must be smp_mb() rather
>>> than smp_rmb().
>>>
>>> Also, why not just have a single smp_mb() at the beginning of
>>> sync_all_readers() instead of executing one barrier per CPU?
>>
>> Well, since my intention was to help the writer see the update (->reader_refcnt
>> dropping to zero) ASAP, I kept the multiple smp_rmb()s.
> 
> At least you were consistent.  ;-)
>

Haha, that's an optimistic way of looking at it, but its no good if I was
consistently _wrong_! ;-)

>>>> +
>>>> +	while (reader_uses_percpu_refcnt(pcpu_rwlock, cpu))
>>>> +		cpu_relax();
>>>> +}
>>>> +
>>>> +static void sync_all_readers(struct percpu_rwlock *pcpu_rwlock)
>>>> +{
>>>> +	unsigned int cpu;
>>>> +
>>>> +	for_each_online_cpu(cpu)
>>>> +		sync_reader(pcpu_rwlock, cpu);
>>>>  }
>>>>
>>>>  void percpu_write_lock(struct percpu_rwlock *pcpu_rwlock)
>>>>  {
>>>> +	/*
>>>> +	 * Tell all readers that a writer is becoming active, so that they
>>>> +	 * start switching over to the global rwlock.
>>>> +	 */
>>>> +	announce_writer_active(pcpu_rwlock);
>>>> +	sync_all_readers(pcpu_rwlock);
>>>>  	write_lock(&pcpu_rwlock->global_rwlock);
>>>>  }
>>>>
>>>>  void percpu_write_unlock(struct percpu_rwlock *pcpu_rwlock)
>>>>  {
>>>> +	/*
>>>> +	 * Inform all readers that we are done, so that they can switch back
>>>> +	 * to their per-cpu refcounts. (We don't need to wait for them to
>>>> +	 * see it).
>>>> +	 */
>>>> +	announce_writer_inactive(pcpu_rwlock);
>>>>  	write_unlock(&pcpu_rwlock->global_rwlock);
>>>>  }
>>>>
>>>>
>>
>> Thanks a lot for your detailed review and comments! :-)
> 
> It will be good to get this in!
>

Thank you :-) I'll try to address the review comments and respin the
patchset soon.

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Oleg Nesterov]

On 02/11, Srivatsa S. Bhat wrote:
>
> On 02/09/2013 04:40 AM, Paul E. McKenney wrote:
> >> +static void announce_writer_inactive(struct percpu_rwlock *pcpu_rwlock)
> >> +{
> >> +	unsigned int cpu;
> >> +
> >> +	drop_writer_signal(pcpu_rwlock, smp_processor_id());
> >
> > Why do we drop ourselves twice?  More to the point, why is it important to
> > drop ourselves first?
> >
>
> I don't see where we are dropping ourselves twice. Note that we are no longer
> in the cpu_online_mask, so the 'for' loop below won't include us. So we need
> to manually drop ourselves. It doesn't matter whether we drop ourselves first
> or later.

Yes, but this just reflects its usage in cpu-hotplug. cpu goes away under
_write_lock.

Perhaps _write_lock/unlock shoud use for_each_possible_cpu() instead?

Hmm... I think this makes sense anyway. Otherwise, in theory,
percpu_write_lock(random_non_hotplug_lock) can race with cpu_up?

Oleg.

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Srivatsa S. Bhat]

On 02/11/2013 01:43 AM, Oleg Nesterov wrote:
> On 02/11, Srivatsa S. Bhat wrote:
>>
>> On 02/09/2013 04:40 AM, Paul E. McKenney wrote:
>>>> +static void announce_writer_inactive(struct percpu_rwlock *pcpu_rwlock)
>>>> +{
>>>> +	unsigned int cpu;
>>>> +
>>>> +	drop_writer_signal(pcpu_rwlock, smp_processor_id());
>>>
>>> Why do we drop ourselves twice?  More to the point, why is it important to
>>> drop ourselves first?
>>>
>>
>> I don't see where we are dropping ourselves twice. Note that we are no longer
>> in the cpu_online_mask, so the 'for' loop below won't include us. So we need
>> to manually drop ourselves. It doesn't matter whether we drop ourselves first
>> or later.
> 
> Yes, but this just reflects its usage in cpu-hotplug. cpu goes away under
> _write_lock.
> 

Ah, right. I guess the code still has remnants from the older version in which
this locking scheme wasn't generic and was tied to cpu-hotplug alone..

> Perhaps _write_lock/unlock shoud use for_each_possible_cpu() instead?
> 

Hmm, that wouldn't be too bad.

> Hmm... I think this makes sense anyway. Otherwise, in theory,
> percpu_write_lock(random_non_hotplug_lock) can race with cpu_up?
> 

Yeah, makes sense. Will change it to for_each_possible_cpu().
And I had previously fixed such races with lglocks with a complicated scheme (to
avoid the costly for_each_possible loop), which was finally rewritten to use
for_each_possible_cpu() for the sake of simplicity..
Regards,
Srivatsa S. Bhat

[PATCH v5 05/45] percpu_rwlock: Make percpu-rwlocks IRQ-safe, optimally

[Srivatsa S. Bhat]

If interrupt handlers can also be readers, then one of the ways to make
per-CPU rwlocks safe, is to disable interrupts at the reader side before
trying to acquire the per-CPU rwlock and keep it disabled throughout the
duration of the read-side critical section.

The goal is to avoid cases such as:

  1. writer is active and it holds the global rwlock for write

  2. a regular reader comes in and marks itself as present (by incrementing
     its per-CPU refcount) before checking whether writer is active.

  3. an interrupt hits the reader;
     [If it had not hit, the reader would have noticed that the writer is
      active and would have decremented its refcount and would have tried
      to acquire the global rwlock for read].
     Since the interrupt handler also happens to be a reader, it notices
     the non-zero refcount (which was due to the reader who got interrupted)
     and thinks that this is a nested read-side critical section and
     proceeds to take the fastpath, which is wrong. The interrupt handler
     should have noticed that the writer is active and taken the rwlock
     for read.

So, disabling interrupts can help avoid this problem (at the cost of keeping
the interrupts disabled for quite long).

But Oleg had a brilliant idea by which we can do much better than that:
we can manage with disabling interrupts _just_ during the updates (writes to
per-CPU refcounts) to safe-guard against races with interrupt handlers.
Beyond that, we can keep the interrupts enabled and still be safe w.r.t
interrupt handlers that can act as readers.

Basically the idea is that we differentiate between the *part* of the
per-CPU refcount that we use for reference counting vs the part that we use
merely to make the writer wait for us to switch over to the right
synchronization scheme.

The scheme involves splitting the per-CPU refcounts into 2 parts:
eg: the lower 16 bits are used to track the nesting depth of the reader
(a "nested-counter"), and the remaining (upper) bits are used to merely mark
the presence of the reader.

As long as the overall reader_refcnt is non-zero, the writer waits for the
reader (assuming that the reader is still actively using per-CPU refcounts for
synchronization).

The reader first sets one of the higher bits to mark its presence, and then
uses the lower 16 bits to manage the nesting depth. So, an interrupt handler
coming in as illustrated above will be able to distinguish between "this is
a nested read-side critical section" vs "we have merely marked our presence
to make the writer wait for us to switch" by looking at the same refcount.
Thus, it makes it unnecessary to keep interrupts disabled throughout the
read-side critical section, despite having the possibility of interrupt
handlers being readers themselves.


Implement this logic and rename the locking functions appropriately, to
reflect what they do.

Based-on-idea-by: Oleg Nesterov <oleg@redhat.com>
Cc: David Howells <dhowells@redhat.com>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 include/linux/percpu-rwlock.h |   15 ++++++++++-----
 lib/percpu-rwlock.c           |   41 +++++++++++++++++++++++++++--------------
 2 files changed, 37 insertions(+), 19 deletions(-)

diff --git a/include/linux/percpu-rwlock.h b/include/linux/percpu-rwlock.h
index 6819bb8..856ba6b 100644
--- a/include/linux/percpu-rwlock.h
+++ b/include/linux/percpu-rwlock.h
@@ -34,11 +34,13 @@ struct percpu_rwlock {
 	rwlock_t		global_rwlock;
 };
 
-extern void percpu_read_lock(struct percpu_rwlock *);
-extern void percpu_read_unlock(struct percpu_rwlock *);
+extern void percpu_read_lock_irqsafe(struct percpu_rwlock *);
+extern void percpu_read_unlock_irqsafe(struct percpu_rwlock *);
 
-extern void percpu_write_lock(struct percpu_rwlock *);
-extern void percpu_write_unlock(struct percpu_rwlock *);
+extern void percpu_write_lock_irqsave(struct percpu_rwlock *,
+				      unsigned long *flags);
+extern void percpu_write_unlock_irqrestore(struct percpu_rwlock *,
+					   unsigned long *flags);
 
 extern int __percpu_init_rwlock(struct percpu_rwlock *,
 				const char *, struct lock_class_key *);
@@ -68,11 +70,14 @@ extern void percpu_free_rwlock(struct percpu_rwlock *);
 	__percpu_init_rwlock(pcpu_rwlock, #pcpu_rwlock, &rwlock_key);	\
 })
 
+#define READER_PRESENT		(1UL << 16)
+#define READER_REFCNT_MASK	(READER_PRESENT - 1)
+
 #define reader_uses_percpu_refcnt(pcpu_rwlock, cpu)			\
 		(ACCESS_ONCE(per_cpu(*((pcpu_rwlock)->reader_refcnt), cpu)))
 
 #define reader_nested_percpu(pcpu_rwlock)				\
-			(__this_cpu_read(*((pcpu_rwlock)->reader_refcnt)) > 1)
+	(__this_cpu_read(*((pcpu_rwlock)->reader_refcnt)) & READER_REFCNT_MASK)
 
 #define writer_active(pcpu_rwlock)					\
 			(__this_cpu_read(*((pcpu_rwlock)->writer_signal)))
diff --git a/lib/percpu-rwlock.c b/lib/percpu-rwlock.c
index 992da5c..a8d177a 100644
--- a/lib/percpu-rwlock.c
+++ b/lib/percpu-rwlock.c
@@ -62,19 +62,19 @@ void percpu_free_rwlock(struct percpu_rwlock *pcpu_rwlock)
 	pcpu_rwlock->writer_signal = NULL;
 }
 
-void percpu_read_lock(struct percpu_rwlock *pcpu_rwlock)
+void percpu_read_lock_irqsafe(struct percpu_rwlock *pcpu_rwlock)
 {
 	preempt_disable();
 
 	/* First and foremost, let the writer know that a reader is active */
-	this_cpu_inc(*pcpu_rwlock->reader_refcnt);
+	this_cpu_add(*pcpu_rwlock->reader_refcnt, READER_PRESENT);
 
 	/*
 	 * If we are already using per-cpu refcounts, it is not safe to switch
 	 * the synchronization scheme. So continue using the refcounts.
 	 */
 	if (reader_nested_percpu(pcpu_rwlock)) {
-		goto out;
+		this_cpu_inc(*pcpu_rwlock->reader_refcnt);
 	} else {
 		/*
 		 * The write to 'reader_refcnt' must be visible before we
@@ -83,9 +83,19 @@ void percpu_read_lock(struct percpu_rwlock *pcpu_rwlock)
 		smp_mb(); /* Paired with smp_rmb() in sync_reader() */
 
 		if (likely(!writer_active(pcpu_rwlock))) {
-			goto out;
+			this_cpu_inc(*pcpu_rwlock->reader_refcnt);
 		} else {
 			/* Writer is active, so switch to global rwlock. */
+
+			/*
+			 * While we are spinning on ->global_rwlock, an
+			 * interrupt can hit us, and the interrupt handler
+			 * might call this function. The distinction between
+			 * READER_PRESENT and the refcnt helps ensure that the
+			 * interrupt handler also takes this branch and spins
+			 * on the ->global_rwlock, as long as the writer is
+			 * active.
+			 */
 			read_lock(&pcpu_rwlock->global_rwlock);
 
 			/*
@@ -95,26 +105,27 @@ void percpu_read_lock(struct percpu_rwlock *pcpu_rwlock)
 			 * back to per-cpu refcounts. (This also helps avoid
 			 * heterogeneous nesting of readers).
 			 */
-			if (writer_active(pcpu_rwlock))
-				this_cpu_dec(*pcpu_rwlock->reader_refcnt);
-			else
+			if (!writer_active(pcpu_rwlock)) {
+				this_cpu_inc(*pcpu_rwlock->reader_refcnt);
 				read_unlock(&pcpu_rwlock->global_rwlock);
+			}
 		}
 	}
 
-out:
+	this_cpu_sub(*pcpu_rwlock->reader_refcnt, READER_PRESENT);
+
 	/* Prevent reordering of any subsequent reads */
 	smp_rmb();
 }
 
-void percpu_read_unlock(struct percpu_rwlock *pcpu_rwlock)
+void percpu_read_unlock_irqsafe(struct percpu_rwlock *pcpu_rwlock)
 {
 	/*
 	 * We never allow heterogeneous nesting of readers. So it is trivial
 	 * to find out the kind of reader we are, and undo the operation
 	 * done by our corresponding percpu_read_lock().
 	 */
-	if (__this_cpu_read(*pcpu_rwlock->reader_refcnt)) {
+	if (reader_nested_percpu(pcpu_rwlock)) {
 		this_cpu_dec(*pcpu_rwlock->reader_refcnt);
 		smp_wmb(); /* Paired with smp_rmb() in sync_reader() */
 	} else {
@@ -184,7 +195,8 @@ static void sync_all_readers(struct percpu_rwlock *pcpu_rwlock)
 		sync_reader(pcpu_rwlock, cpu);
 }
 
-void percpu_write_lock(struct percpu_rwlock *pcpu_rwlock)
+void percpu_write_lock_irqsave(struct percpu_rwlock *pcpu_rwlock,
+			       unsigned long *flags)
 {
 	/*
 	 * Tell all readers that a writer is becoming active, so that they
@@ -192,10 +204,11 @@ void percpu_write_lock(struct percpu_rwlock *pcpu_rwlock)
 	 */
 	announce_writer_active(pcpu_rwlock);
 	sync_all_readers(pcpu_rwlock);
-	write_lock(&pcpu_rwlock->global_rwlock);
+	write_lock_irqsave(&pcpu_rwlock->global_rwlock, *flags);
 }
 
-void percpu_write_unlock(struct percpu_rwlock *pcpu_rwlock)
+void percpu_write_unlock_irqrestore(struct percpu_rwlock *pcpu_rwlock,
+			 unsigned long *flags)
 {
 	/*
 	 * Inform all readers that we are done, so that they can switch back
@@ -203,6 +216,6 @@ void percpu_write_unlock(struct percpu_rwlock *pcpu_rwlock)
 	 * see it).
 	 */
 	announce_writer_inactive(pcpu_rwlock);
-	write_unlock(&pcpu_rwlock->global_rwlock);
+	write_unlock_irqrestore(&pcpu_rwlock->global_rwlock, *flags);
 }
 

Re: [PATCH v5 05/45] percpu_rwlock: Make percpu-rwlocks IRQ-safe, optimally

[Paul E. McKenney]

On Tue, Jan 22, 2013 at 01:04:11PM +0530, Srivatsa S. Bhat wrote:
> If interrupt handlers can also be readers, then one of the ways to make
> per-CPU rwlocks safe, is to disable interrupts at the reader side before
> trying to acquire the per-CPU rwlock and keep it disabled throughout the
> duration of the read-side critical section.
> 
> The goal is to avoid cases such as:
> 
>   1. writer is active and it holds the global rwlock for write
> 
>   2. a regular reader comes in and marks itself as present (by incrementing
>      its per-CPU refcount) before checking whether writer is active.
> 
>   3. an interrupt hits the reader;
>      [If it had not hit, the reader would have noticed that the writer is
>       active and would have decremented its refcount and would have tried
>       to acquire the global rwlock for read].
>      Since the interrupt handler also happens to be a reader, it notices
>      the non-zero refcount (which was due to the reader who got interrupted)
>      and thinks that this is a nested read-side critical section and
>      proceeds to take the fastpath, which is wrong. The interrupt handler
>      should have noticed that the writer is active and taken the rwlock
>      for read.
> 
> So, disabling interrupts can help avoid this problem (at the cost of keeping
> the interrupts disabled for quite long).
> 
> But Oleg had a brilliant idea by which we can do much better than that:
> we can manage with disabling interrupts _just_ during the updates (writes to
> per-CPU refcounts) to safe-guard against races with interrupt handlers.
> Beyond that, we can keep the interrupts enabled and still be safe w.r.t
> interrupt handlers that can act as readers.
> 
> Basically the idea is that we differentiate between the *part* of the
> per-CPU refcount that we use for reference counting vs the part that we use
> merely to make the writer wait for us to switch over to the right
> synchronization scheme.
> 
> The scheme involves splitting the per-CPU refcounts into 2 parts:
> eg: the lower 16 bits are used to track the nesting depth of the reader
> (a "nested-counter"), and the remaining (upper) bits are used to merely mark
> the presence of the reader.
> 
> As long as the overall reader_refcnt is non-zero, the writer waits for the
> reader (assuming that the reader is still actively using per-CPU refcounts for
> synchronization).
> 
> The reader first sets one of the higher bits to mark its presence, and then
> uses the lower 16 bits to manage the nesting depth. So, an interrupt handler
> coming in as illustrated above will be able to distinguish between "this is
> a nested read-side critical section" vs "we have merely marked our presence
> to make the writer wait for us to switch" by looking at the same refcount.
> Thus, it makes it unnecessary to keep interrupts disabled throughout the
> read-side critical section, despite having the possibility of interrupt
> handlers being readers themselves.
> 
> 
> Implement this logic and rename the locking functions appropriately, to
> reflect what they do.

One nit below.  The issues called out in the previous patch still seem
to me to apply.

							Thanx, Paul

> Based-on-idea-by: Oleg Nesterov <oleg@redhat.com>
> Cc: David Howells <dhowells@redhat.com>
> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
> ---
> 
>  include/linux/percpu-rwlock.h |   15 ++++++++++-----
>  lib/percpu-rwlock.c           |   41 +++++++++++++++++++++++++++--------------
>  2 files changed, 37 insertions(+), 19 deletions(-)
> 
> diff --git a/include/linux/percpu-rwlock.h b/include/linux/percpu-rwlock.h
> index 6819bb8..856ba6b 100644
> --- a/include/linux/percpu-rwlock.h
> +++ b/include/linux/percpu-rwlock.h
> @@ -34,11 +34,13 @@ struct percpu_rwlock {
>  	rwlock_t		global_rwlock;
>  };
> 
> -extern void percpu_read_lock(struct percpu_rwlock *);
> -extern void percpu_read_unlock(struct percpu_rwlock *);
> +extern void percpu_read_lock_irqsafe(struct percpu_rwlock *);
> +extern void percpu_read_unlock_irqsafe(struct percpu_rwlock *);
> 
> -extern void percpu_write_lock(struct percpu_rwlock *);
> -extern void percpu_write_unlock(struct percpu_rwlock *);
> +extern void percpu_write_lock_irqsave(struct percpu_rwlock *,
> +				      unsigned long *flags);
> +extern void percpu_write_unlock_irqrestore(struct percpu_rwlock *,
> +					   unsigned long *flags);
> 
>  extern int __percpu_init_rwlock(struct percpu_rwlock *,
>  				const char *, struct lock_class_key *);
> @@ -68,11 +70,14 @@ extern void percpu_free_rwlock(struct percpu_rwlock *);
>  	__percpu_init_rwlock(pcpu_rwlock, #pcpu_rwlock, &rwlock_key);	\
>  })
> 
> +#define READER_PRESENT		(1UL << 16)
> +#define READER_REFCNT_MASK	(READER_PRESENT - 1)
> +
>  #define reader_uses_percpu_refcnt(pcpu_rwlock, cpu)			\
>  		(ACCESS_ONCE(per_cpu(*((pcpu_rwlock)->reader_refcnt), cpu)))
> 
>  #define reader_nested_percpu(pcpu_rwlock)				\
> -			(__this_cpu_read(*((pcpu_rwlock)->reader_refcnt)) > 1)
> +	(__this_cpu_read(*((pcpu_rwlock)->reader_refcnt)) & READER_REFCNT_MASK)
> 
>  #define writer_active(pcpu_rwlock)					\
>  			(__this_cpu_read(*((pcpu_rwlock)->writer_signal)))
> diff --git a/lib/percpu-rwlock.c b/lib/percpu-rwlock.c
> index 992da5c..a8d177a 100644
> --- a/lib/percpu-rwlock.c
> +++ b/lib/percpu-rwlock.c
> @@ -62,19 +62,19 @@ void percpu_free_rwlock(struct percpu_rwlock *pcpu_rwlock)
>  	pcpu_rwlock->writer_signal = NULL;
>  }
> 
> -void percpu_read_lock(struct percpu_rwlock *pcpu_rwlock)
> +void percpu_read_lock_irqsafe(struct percpu_rwlock *pcpu_rwlock)
>  {
>  	preempt_disable();
> 
>  	/* First and foremost, let the writer know that a reader is active */
> -	this_cpu_inc(*pcpu_rwlock->reader_refcnt);
> +	this_cpu_add(*pcpu_rwlock->reader_refcnt, READER_PRESENT);
> 
>  	/*
>  	 * If we are already using per-cpu refcounts, it is not safe to switch
>  	 * the synchronization scheme. So continue using the refcounts.
>  	 */
>  	if (reader_nested_percpu(pcpu_rwlock)) {
> -		goto out;
> +		this_cpu_inc(*pcpu_rwlock->reader_refcnt);

Hmmm...  If the reader is nested, it -doesn't- need the memory barrier at
the end of this function.  If there is lots of nesting, it might be
worth getting rid of it.

>  	} else {
>  		/*
>  		 * The write to 'reader_refcnt' must be visible before we
> @@ -83,9 +83,19 @@ void percpu_read_lock(struct percpu_rwlock *pcpu_rwlock)
>  		smp_mb(); /* Paired with smp_rmb() in sync_reader() */
> 
>  		if (likely(!writer_active(pcpu_rwlock))) {
> -			goto out;
> +			this_cpu_inc(*pcpu_rwlock->reader_refcnt);
>  		} else {
>  			/* Writer is active, so switch to global rwlock. */
> +
> +			/*
> +			 * While we are spinning on ->global_rwlock, an
> +			 * interrupt can hit us, and the interrupt handler
> +			 * might call this function. The distinction between
> +			 * READER_PRESENT and the refcnt helps ensure that the
> +			 * interrupt handler also takes this branch and spins
> +			 * on the ->global_rwlock, as long as the writer is
> +			 * active.
> +			 */
>  			read_lock(&pcpu_rwlock->global_rwlock);
> 
>  			/*
> @@ -95,26 +105,27 @@ void percpu_read_lock(struct percpu_rwlock *pcpu_rwlock)
>  			 * back to per-cpu refcounts. (This also helps avoid
>  			 * heterogeneous nesting of readers).
>  			 */
> -			if (writer_active(pcpu_rwlock))
> -				this_cpu_dec(*pcpu_rwlock->reader_refcnt);
> -			else
> +			if (!writer_active(pcpu_rwlock)) {
> +				this_cpu_inc(*pcpu_rwlock->reader_refcnt);
>  				read_unlock(&pcpu_rwlock->global_rwlock);
> +			}
>  		}
>  	}
> 
> -out:
> +	this_cpu_sub(*pcpu_rwlock->reader_refcnt, READER_PRESENT);
> +
>  	/* Prevent reordering of any subsequent reads */
>  	smp_rmb();
>  }
> 
> -void percpu_read_unlock(struct percpu_rwlock *pcpu_rwlock)
> +void percpu_read_unlock_irqsafe(struct percpu_rwlock *pcpu_rwlock)
>  {
>  	/*
>  	 * We never allow heterogeneous nesting of readers. So it is trivial
>  	 * to find out the kind of reader we are, and undo the operation
>  	 * done by our corresponding percpu_read_lock().
>  	 */
> -	if (__this_cpu_read(*pcpu_rwlock->reader_refcnt)) {
> +	if (reader_nested_percpu(pcpu_rwlock)) {
>  		this_cpu_dec(*pcpu_rwlock->reader_refcnt);
>  		smp_wmb(); /* Paired with smp_rmb() in sync_reader() */
>  	} else {
> @@ -184,7 +195,8 @@ static void sync_all_readers(struct percpu_rwlock *pcpu_rwlock)
>  		sync_reader(pcpu_rwlock, cpu);
>  }
> 
> -void percpu_write_lock(struct percpu_rwlock *pcpu_rwlock)
> +void percpu_write_lock_irqsave(struct percpu_rwlock *pcpu_rwlock,
> +			       unsigned long *flags)
>  {
>  	/*
>  	 * Tell all readers that a writer is becoming active, so that they
> @@ -192,10 +204,11 @@ void percpu_write_lock(struct percpu_rwlock *pcpu_rwlock)
>  	 */
>  	announce_writer_active(pcpu_rwlock);
>  	sync_all_readers(pcpu_rwlock);
> -	write_lock(&pcpu_rwlock->global_rwlock);
> +	write_lock_irqsave(&pcpu_rwlock->global_rwlock, *flags);
>  }
> 
> -void percpu_write_unlock(struct percpu_rwlock *pcpu_rwlock)
> +void percpu_write_unlock_irqrestore(struct percpu_rwlock *pcpu_rwlock,
> +			 unsigned long *flags)
>  {
>  	/*
>  	 * Inform all readers that we are done, so that they can switch back
> @@ -203,6 +216,6 @@ void percpu_write_unlock(struct percpu_rwlock *pcpu_rwlock)
>  	 * see it).
>  	 */
>  	announce_writer_inactive(pcpu_rwlock);
> -	write_unlock(&pcpu_rwlock->global_rwlock);
> +	write_unlock_irqrestore(&pcpu_rwlock->global_rwlock, *flags);
>  }
> 
> 

Re: [PATCH v5 05/45] percpu_rwlock: Make percpu-rwlocks IRQ-safe, optimally

[Srivatsa S. Bhat]

On 02/09/2013 05:14 AM, Paul E. McKenney wrote:
> On Tue, Jan 22, 2013 at 01:04:11PM +0530, Srivatsa S. Bhat wrote:
>> If interrupt handlers can also be readers, then one of the ways to make
>> per-CPU rwlocks safe, is to disable interrupts at the reader side before
>> trying to acquire the per-CPU rwlock and keep it disabled throughout the
>> duration of the read-side critical section.
[...]
>> -void percpu_read_lock(struct percpu_rwlock *pcpu_rwlock)
>> +void percpu_read_lock_irqsafe(struct percpu_rwlock *pcpu_rwlock)
>>  {
>>  	preempt_disable();
>>
>>  	/* First and foremost, let the writer know that a reader is active */
>> -	this_cpu_inc(*pcpu_rwlock->reader_refcnt);
>> +	this_cpu_add(*pcpu_rwlock->reader_refcnt, READER_PRESENT);
>>
>>  	/*
>>  	 * If we are already using per-cpu refcounts, it is not safe to switch
>>  	 * the synchronization scheme. So continue using the refcounts.
>>  	 */
>>  	if (reader_nested_percpu(pcpu_rwlock)) {
>> -		goto out;
>> +		this_cpu_inc(*pcpu_rwlock->reader_refcnt);
> 
> Hmmm...  If the reader is nested, it -doesn't- need the memory barrier at
> the end of this function.  If there is lots of nesting, it might be
> worth getting rid of it.
> 

Yes, good point! Will get rid of it.

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 05/45] percpu_rwlock: Make percpu-rwlocks IRQ-safe, optimally

[Oleg Nesterov]

only one cosmetic nit...

On 01/22, Srivatsa S. Bhat wrote:
>
> +#define READER_PRESENT		(1UL << 16)
> +#define READER_REFCNT_MASK	(READER_PRESENT - 1)
> +
>  #define reader_uses_percpu_refcnt(pcpu_rwlock, cpu)			\
>  		(ACCESS_ONCE(per_cpu(*((pcpu_rwlock)->reader_refcnt), cpu)))
>
>  #define reader_nested_percpu(pcpu_rwlock)				\
> -			(__this_cpu_read(*((pcpu_rwlock)->reader_refcnt)) > 1)
> +	(__this_cpu_read(*((pcpu_rwlock)->reader_refcnt)) & READER_REFCNT_MASK)
>
>  #define writer_active(pcpu_rwlock)					\
>  			(__this_cpu_read(*((pcpu_rwlock)->writer_signal)))

I think this all can go to lib/percpu-rwlock.c. Nobody needs to know
these implementation details.

Oleg.

Re: [PATCH v5 05/45] percpu_rwlock: Make percpu-rwlocks IRQ-safe, optimally

[Srivatsa S. Bhat]

On 02/11/2013 12:12 AM, Oleg Nesterov wrote:
> only one cosmetic nit...
> 
> On 01/22, Srivatsa S. Bhat wrote:
>>
>> +#define READER_PRESENT		(1UL << 16)
>> +#define READER_REFCNT_MASK	(READER_PRESENT - 1)
>> +
>>  #define reader_uses_percpu_refcnt(pcpu_rwlock, cpu)			\
>>  		(ACCESS_ONCE(per_cpu(*((pcpu_rwlock)->reader_refcnt), cpu)))
>>
>>  #define reader_nested_percpu(pcpu_rwlock)				\
>> -			(__this_cpu_read(*((pcpu_rwlock)->reader_refcnt)) > 1)
>> +	(__this_cpu_read(*((pcpu_rwlock)->reader_refcnt)) & READER_REFCNT_MASK)
>>
>>  #define writer_active(pcpu_rwlock)					\
>>  			(__this_cpu_read(*((pcpu_rwlock)->writer_signal)))
> 
> I think this all can go to lib/percpu-rwlock.c. Nobody needs to know
> these implementation details.
> 

Ok, will move them.
 
Regards,
Srivatsa S. Bhat

[PATCH v5 06/45] percpu_rwlock: Allow writers to be readers, and add lockdep annotations

[Srivatsa S. Bhat]

CPU hotplug (which will be the first user of per-CPU rwlocks) has a special
requirement with respect to locking: the writer, after acquiring the per-CPU
rwlock for write, must be allowed to take the same lock for read, without
deadlocking and without getting complaints from lockdep. In comparison, this
is similar to what get_online_cpus()/put_online_cpus() does today: it allows
a hotplug writer (who holds the cpu_hotplug.lock mutex) to invoke it without
locking issues, because it silently returns if the caller is the hotplug
writer itself.

This can be easily achieved with per-CPU rwlocks as well (even without a
"is this a writer?" check) by incrementing the per-CPU refcount of the writer
immediately after taking the global rwlock for write, and then decrementing
the per-CPU refcount before releasing the global rwlock.
This ensures that any reader that comes along on that CPU while the writer is
active (on that same CPU), notices the non-zero value of the nested counter
and assumes that it is a nested read-side critical section and proceeds by
just incrementing the refcount. Thus we prevent the reader from taking the
global rwlock for read, which prevents the writer from deadlocking itself.

Add that support and teach lockdep about this special locking scheme so
that it knows that this sort of usage is valid. Also add the required lockdep
annotations to enable it to detect common locking problems with per-CPU
rwlocks.

Cc: David Howells <dhowells@redhat.com>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 lib/percpu-rwlock.c |   21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/lib/percpu-rwlock.c b/lib/percpu-rwlock.c
index a8d177a..054a50a 100644
--- a/lib/percpu-rwlock.c
+++ b/lib/percpu-rwlock.c
@@ -84,6 +84,10 @@ void percpu_read_lock_irqsafe(struct percpu_rwlock *pcpu_rwlock)
 
 		if (likely(!writer_active(pcpu_rwlock))) {
 			this_cpu_inc(*pcpu_rwlock->reader_refcnt);
+
+			/* Pretend that we take global_rwlock for lockdep */
+			rwlock_acquire_read(&pcpu_rwlock->global_rwlock.dep_map,
+					    0, 0, _RET_IP_);
 		} else {
 			/* Writer is active, so switch to global rwlock. */
 
@@ -108,6 +112,12 @@ void percpu_read_lock_irqsafe(struct percpu_rwlock *pcpu_rwlock)
 			if (!writer_active(pcpu_rwlock)) {
 				this_cpu_inc(*pcpu_rwlock->reader_refcnt);
 				read_unlock(&pcpu_rwlock->global_rwlock);
+
+				/*
+				 * Pretend that we take global_rwlock for lockdep
+				 */
+				rwlock_acquire_read(&pcpu_rwlock->global_rwlock.dep_map,
+						    0, 0, _RET_IP_);
 			}
 		}
 	}
@@ -128,6 +138,14 @@ void percpu_read_unlock_irqsafe(struct percpu_rwlock *pcpu_rwlock)
 	if (reader_nested_percpu(pcpu_rwlock)) {
 		this_cpu_dec(*pcpu_rwlock->reader_refcnt);
 		smp_wmb(); /* Paired with smp_rmb() in sync_reader() */
+
+		/*
+		 * If this is the last decrement, then it is time to pretend
+		 * to lockdep that we are releasing the read lock.
+		 */
+		if (!reader_nested_percpu(pcpu_rwlock))
+			rwlock_release(&pcpu_rwlock->global_rwlock.dep_map,
+				       1, _RET_IP_);
 	} else {
 		read_unlock(&pcpu_rwlock->global_rwlock);
 	}
@@ -205,11 +223,14 @@ void percpu_write_lock_irqsave(struct percpu_rwlock *pcpu_rwlock,
 	announce_writer_active(pcpu_rwlock);
 	sync_all_readers(pcpu_rwlock);
 	write_lock_irqsave(&pcpu_rwlock->global_rwlock, *flags);
+	this_cpu_inc(*pcpu_rwlock->reader_refcnt);
 }
 
 void percpu_write_unlock_irqrestore(struct percpu_rwlock *pcpu_rwlock,
 			 unsigned long *flags)
 {
+	this_cpu_dec(*pcpu_rwlock->reader_refcnt);
+
 	/*
 	 * Inform all readers that we are done, so that they can switch back
 	 * to their per-cpu refcounts. (We don't need to wait for them to

Re: [PATCH v5 06/45] percpu_rwlock: Allow writers to be readers, and add lockdep annotations

[Paul E. McKenney]

On Tue, Jan 22, 2013 at 01:04:23PM +0530, Srivatsa S. Bhat wrote:
> CPU hotplug (which will be the first user of per-CPU rwlocks) has a special
> requirement with respect to locking: the writer, after acquiring the per-CPU
> rwlock for write, must be allowed to take the same lock for read, without
> deadlocking and without getting complaints from lockdep. In comparison, this
> is similar to what get_online_cpus()/put_online_cpus() does today: it allows
> a hotplug writer (who holds the cpu_hotplug.lock mutex) to invoke it without
> locking issues, because it silently returns if the caller is the hotplug
> writer itself.
> 
> This can be easily achieved with per-CPU rwlocks as well (even without a
> "is this a writer?" check) by incrementing the per-CPU refcount of the writer
> immediately after taking the global rwlock for write, and then decrementing
> the per-CPU refcount before releasing the global rwlock.
> This ensures that any reader that comes along on that CPU while the writer is
> active (on that same CPU), notices the non-zero value of the nested counter
> and assumes that it is a nested read-side critical section and proceeds by
> just incrementing the refcount. Thus we prevent the reader from taking the
> global rwlock for read, which prevents the writer from deadlocking itself.
> 
> Add that support and teach lockdep about this special locking scheme so
> that it knows that this sort of usage is valid. Also add the required lockdep
> annotations to enable it to detect common locking problems with per-CPU
> rwlocks.

Very nice!  The write-side interrupt disabling ensures that the task
stays on CPU, as required.

One request: Could we please have a comment explaining the reasons for
the writer incrementing and decrementing the reader reference count?

It looked really really strange to me until I came back and read the
commit log.  ;-)

							Thanx, Paul

> Cc: David Howells <dhowells@redhat.com>
> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
> ---
> 
>  lib/percpu-rwlock.c |   21 +++++++++++++++++++++
>  1 file changed, 21 insertions(+)
> 
> diff --git a/lib/percpu-rwlock.c b/lib/percpu-rwlock.c
> index a8d177a..054a50a 100644
> --- a/lib/percpu-rwlock.c
> +++ b/lib/percpu-rwlock.c
> @@ -84,6 +84,10 @@ void percpu_read_lock_irqsafe(struct percpu_rwlock *pcpu_rwlock)
> 
>  		if (likely(!writer_active(pcpu_rwlock))) {
>  			this_cpu_inc(*pcpu_rwlock->reader_refcnt);
> +
> +			/* Pretend that we take global_rwlock for lockdep */
> +			rwlock_acquire_read(&pcpu_rwlock->global_rwlock.dep_map,
> +					    0, 0, _RET_IP_);
>  		} else {
>  			/* Writer is active, so switch to global rwlock. */
> 
> @@ -108,6 +112,12 @@ void percpu_read_lock_irqsafe(struct percpu_rwlock *pcpu_rwlock)
>  			if (!writer_active(pcpu_rwlock)) {
>  				this_cpu_inc(*pcpu_rwlock->reader_refcnt);
>  				read_unlock(&pcpu_rwlock->global_rwlock);
> +
> +				/*
> +				 * Pretend that we take global_rwlock for lockdep
> +				 */
> +				rwlock_acquire_read(&pcpu_rwlock->global_rwlock.dep_map,
> +						    0, 0, _RET_IP_);
>  			}
>  		}
>  	}
> @@ -128,6 +138,14 @@ void percpu_read_unlock_irqsafe(struct percpu_rwlock *pcpu_rwlock)
>  	if (reader_nested_percpu(pcpu_rwlock)) {
>  		this_cpu_dec(*pcpu_rwlock->reader_refcnt);
>  		smp_wmb(); /* Paired with smp_rmb() in sync_reader() */
> +
> +		/*
> +		 * If this is the last decrement, then it is time to pretend
> +		 * to lockdep that we are releasing the read lock.
> +		 */
> +		if (!reader_nested_percpu(pcpu_rwlock))
> +			rwlock_release(&pcpu_rwlock->global_rwlock.dep_map,
> +				       1, _RET_IP_);
>  	} else {
>  		read_unlock(&pcpu_rwlock->global_rwlock);
>  	}
> @@ -205,11 +223,14 @@ void percpu_write_lock_irqsave(struct percpu_rwlock *pcpu_rwlock,
>  	announce_writer_active(pcpu_rwlock);
>  	sync_all_readers(pcpu_rwlock);
>  	write_lock_irqsave(&pcpu_rwlock->global_rwlock, *flags);
> +	this_cpu_inc(*pcpu_rwlock->reader_refcnt);
>  }
> 
>  void percpu_write_unlock_irqrestore(struct percpu_rwlock *pcpu_rwlock,
>  			 unsigned long *flags)
>  {
> +	this_cpu_dec(*pcpu_rwlock->reader_refcnt);
> +
>  	/*
>  	 * Inform all readers that we are done, so that they can switch back
>  	 * to their per-cpu refcounts. (We don't need to wait for them to
> 

Re: [PATCH v5 06/45] percpu_rwlock: Allow writers to be readers, and add lockdep annotations

[Srivatsa S. Bhat]

On 02/09/2013 05:17 AM, Paul E. McKenney wrote:
> On Tue, Jan 22, 2013 at 01:04:23PM +0530, Srivatsa S. Bhat wrote:
>> CPU hotplug (which will be the first user of per-CPU rwlocks) has a special
>> requirement with respect to locking: the writer, after acquiring the per-CPU
>> rwlock for write, must be allowed to take the same lock for read, without
>> deadlocking and without getting complaints from lockdep. In comparison, this
>> is similar to what get_online_cpus()/put_online_cpus() does today: it allows
>> a hotplug writer (who holds the cpu_hotplug.lock mutex) to invoke it without
>> locking issues, because it silently returns if the caller is the hotplug
>> writer itself.
>>
>> This can be easily achieved with per-CPU rwlocks as well (even without a
>> "is this a writer?" check) by incrementing the per-CPU refcount of the writer
>> immediately after taking the global rwlock for write, and then decrementing
>> the per-CPU refcount before releasing the global rwlock.
>> This ensures that any reader that comes along on that CPU while the writer is
>> active (on that same CPU), notices the non-zero value of the nested counter
>> and assumes that it is a nested read-side critical section and proceeds by
>> just incrementing the refcount. Thus we prevent the reader from taking the
>> global rwlock for read, which prevents the writer from deadlocking itself.
>>
>> Add that support and teach lockdep about this special locking scheme so
>> that it knows that this sort of usage is valid. Also add the required lockdep
>> annotations to enable it to detect common locking problems with per-CPU
>> rwlocks.
> 
> Very nice!  The write-side interrupt disabling ensures that the task
> stays on CPU, as required.
> 
> One request: Could we please have a comment explaining the reasons for
> the writer incrementing and decrementing the reader reference count?
> 
> It looked really really strange to me until I came back and read the
> commit log.  ;-)
> 

Sure :-)

Regards,
Srivatsa S. Bhat

[PATCH v5 07/45] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

[Srivatsa S. Bhat]

There are places where preempt_disable() or local_irq_disable() are used
to prevent any CPU from going offline during the critical section. Let us
call them as "atomic hotplug readers" ("atomic" because they run in atomic,
non-preemptible contexts).

Today, preempt_disable() or its equivalent works because the hotplug writer
uses stop_machine() to take CPUs offline. But once stop_machine() is gone
from the CPU hotplug offline path, the readers won't be able to prevent
CPUs from going offline using preempt_disable().

So the intent here is to provide synchronization APIs for such atomic hotplug
readers, to prevent (any) CPUs from going offline, without depending on
stop_machine() at the writer-side. The new APIs will look something like
this:  get_online_cpus_atomic() and put_online_cpus_atomic()

Some important design requirements and considerations:
-----------------------------------------------------

1. Scalable synchronization at the reader-side, especially in the fast-path

   Any synchronization at the atomic hotplug readers side must be highly
   scalable - avoid global single-holder locks/counters etc. Because, these
   paths currently use the extremely fast preempt_disable(); our replacement
   to preempt_disable() should not become ridiculously costly and also should
   not serialize the readers among themselves needlessly.

   At a minimum, the new APIs must be extremely fast at the reader side
   atleast in the fast-path, when no CPU offline writers are active.

2. preempt_disable() was recursive. The replacement should also be recursive.

3. No (new) lock-ordering restrictions

   preempt_disable() was super-flexible. It didn't impose any ordering
   restrictions or rules for nesting. Our replacement should also be equally
   flexible and usable.

4. No deadlock possibilities

   Regular per-cpu locking is not the way to go if we want to have relaxed
   rules for lock-ordering. Because, we can end up in circular-locking
   dependencies as explained in https://lkml.org/lkml/2012/12/6/290

   So, avoid the usual per-cpu locking schemes (per-cpu locks/per-cpu atomic
   counters with spin-on-contention etc) as much as possible, to avoid
   numerous deadlock possibilities from creeping in.


Implementation of the design:
----------------------------

We use per-CPU reader-writer locks for synchronization because:

  a. They are quite fast and scalable in the fast-path (when no writers are
     active), since they use fast per-cpu counters in those paths.

  b. They are recursive at the reader side.

  c. They provide a good amount of safety against deadlocks; they don't
     spring new deadlock possibilities on us from out of nowhere. As a
     result, they have relaxed locking rules and are quite flexible, and
     thus are best suited for replacing usages of preempt_disable() or
     local_irq_disable() at the reader side.

Together, these satisfy all the requirements mentioned above.

I'm indebted to Michael Wang and Xiao Guangrong for their numerous thoughtful
suggestions and ideas, which inspired and influenced many of the decisions in
this as well as previous designs. Thanks a lot Michael and Xiao!

Cc: Russell King <linux@arm.linux.org.uk>
Cc: Mike Frysinger <vapier@gentoo.org>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: David Howells <dhowells@redhat.com>
Cc: "James E.J. Bottomley" <jejb@parisc-linux.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: uclinux-dist-devel@blackfin.uclinux.org
Cc: linux-ia64@vger.kernel.org
Cc: linux-mips@linux-mips.org
Cc: linux-am33-list@redhat.com
Cc: linux-parisc@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-s390@vger.kernel.org
Cc: linux-sh@vger.kernel.org
Cc: sparclinux@vger.kernel.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/arm/Kconfig      |    1 +
 arch/blackfin/Kconfig |    1 +
 arch/ia64/Kconfig     |    1 +
 arch/mips/Kconfig     |    1 +
 arch/mn10300/Kconfig  |    1 +
 arch/parisc/Kconfig   |    1 +
 arch/powerpc/Kconfig  |    1 +
 arch/s390/Kconfig     |    1 +
 arch/sh/Kconfig       |    1 +
 arch/sparc/Kconfig    |    1 +
 arch/x86/Kconfig      |    1 +
 include/linux/cpu.h   |    4 +++
 kernel/cpu.c          |   57 ++++++++++++++++++++++++++++++++++++++++++++++---
 13 files changed, 69 insertions(+), 3 deletions(-)

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 67874b8..cb6b94b 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -1616,6 +1616,7 @@ config NR_CPUS
 config HOTPLUG_CPU
 	bool "Support for hot-pluggable CPUs"
 	depends on SMP && HOTPLUG
+	select PERCPU_RWLOCK
 	help
 	  Say Y here to experiment with turning CPUs off and on.  CPUs
 	  can be controlled through /sys/devices/system/cpu.
diff --git a/arch/blackfin/Kconfig b/arch/blackfin/Kconfig
index b6f3ad5..83d9882 100644
--- a/arch/blackfin/Kconfig
+++ b/arch/blackfin/Kconfig
@@ -261,6 +261,7 @@ config NR_CPUS
 config HOTPLUG_CPU
 	bool "Support for hot-pluggable CPUs"
 	depends on SMP && HOTPLUG
+	select PERCPU_RWLOCK
 	default y
 
 config BF_REV_MIN
diff --git a/arch/ia64/Kconfig b/arch/ia64/Kconfig
index 3279646..c246772 100644
--- a/arch/ia64/Kconfig
+++ b/arch/ia64/Kconfig
@@ -378,6 +378,7 @@ config HOTPLUG_CPU
 	bool "Support for hot-pluggable CPUs (EXPERIMENTAL)"
 	depends on SMP && EXPERIMENTAL
 	select HOTPLUG
+	select PERCPU_RWLOCK
 	default n
 	---help---
 	  Say Y here to experiment with turning CPUs off and on.  CPUs
diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 2ac626a..f97c479 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -956,6 +956,7 @@ config SYS_HAS_EARLY_PRINTK
 config HOTPLUG_CPU
 	bool "Support for hot-pluggable CPUs"
 	depends on SMP && HOTPLUG && SYS_SUPPORTS_HOTPLUG_CPU
+	select PERCPU_RWLOCK
 	help
 	  Say Y here to allow turning CPUs off and on. CPUs can be
 	  controlled through /sys/devices/system/cpu.
diff --git a/arch/mn10300/Kconfig b/arch/mn10300/Kconfig
index e70001c..a64e488 100644
--- a/arch/mn10300/Kconfig
+++ b/arch/mn10300/Kconfig
@@ -60,6 +60,7 @@ config ARCH_HAS_ILOG2_U32
 
 config HOTPLUG_CPU
 	def_bool n
+	select PERCPU_RWLOCK
 
 source "init/Kconfig"
 
diff --git a/arch/parisc/Kconfig b/arch/parisc/Kconfig
index b77feff..6f55cd4 100644
--- a/arch/parisc/Kconfig
+++ b/arch/parisc/Kconfig
@@ -226,6 +226,7 @@ config HOTPLUG_CPU
 	bool
 	default y if SMP
 	select HOTPLUG
+	select PERCPU_RWLOCK
 
 config ARCH_SELECT_MEMORY_MODEL
 	def_bool y
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 17903f1..56b1f15 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -336,6 +336,7 @@ config HOTPLUG_CPU
 	bool "Support for enabling/disabling CPUs"
 	depends on SMP && HOTPLUG && EXPERIMENTAL && (PPC_PSERIES || \
 	PPC_PMAC || PPC_POWERNV || (PPC_85xx && !PPC_E500MC))
+	select PERCPU_RWLOCK
 	---help---
 	  Say Y here to be able to disable and re-enable individual
 	  CPUs at runtime on SMP machines.
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index b5ea38c..a9aafb4 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -299,6 +299,7 @@ config HOTPLUG_CPU
 	prompt "Support for hot-pluggable CPUs"
 	depends on SMP
 	select HOTPLUG
+	select PERCPU_RWLOCK
 	help
 	  Say Y here to be able to turn CPUs off and on. CPUs
 	  can be controlled through /sys/devices/system/cpu/cpu#.
diff --git a/arch/sh/Kconfig b/arch/sh/Kconfig
index babc2b8..8c92eef 100644
--- a/arch/sh/Kconfig
+++ b/arch/sh/Kconfig
@@ -765,6 +765,7 @@ config NR_CPUS
 config HOTPLUG_CPU
 	bool "Support for hot-pluggable CPUs (EXPERIMENTAL)"
 	depends on SMP && HOTPLUG && EXPERIMENTAL
+	select PERCPU_RWLOCK
 	help
 	  Say Y here to experiment with turning CPUs off and on.  CPUs
 	  can be controlled through /sys/devices/system/cpu.
diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index 9f2edb5..e2bd573 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -253,6 +253,7 @@ config HOTPLUG_CPU
 	bool "Support for hot-pluggable CPUs"
 	depends on SPARC64 && SMP
 	select HOTPLUG
+	select PERCPU_RWLOCK
 	help
 	  Say Y here to experiment with turning CPUs off and on.  CPUs
 	  can be controlled through /sys/devices/system/cpu/cpu#.
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 79795af..a225d12 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1689,6 +1689,7 @@ config PHYSICAL_ALIGN
 config HOTPLUG_CPU
 	bool "Support for hot-pluggable CPUs"
 	depends on SMP && HOTPLUG
+	select PERCPU_RWLOCK
 	---help---
 	  Say Y here to allow turning CPUs off and on. CPUs can be
 	  controlled through /sys/devices/system/cpu.
diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index ce7a074..cf24da1 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -175,6 +175,8 @@ extern struct bus_type cpu_subsys;
 
 extern void get_online_cpus(void);
 extern void put_online_cpus(void);
+extern void get_online_cpus_atomic(void);
+extern void put_online_cpus_atomic(void);
 #define hotcpu_notifier(fn, pri)	cpu_notifier(fn, pri)
 #define register_hotcpu_notifier(nb)	register_cpu_notifier(nb)
 #define unregister_hotcpu_notifier(nb)	unregister_cpu_notifier(nb)
@@ -198,6 +200,8 @@ static inline void cpu_hotplug_driver_unlock(void)
 
 #define get_online_cpus()	do { } while (0)
 #define put_online_cpus()	do { } while (0)
+#define get_online_cpus_atomic()	do { } while (0)
+#define put_online_cpus_atomic()	do { } while (0)
 #define hotcpu_notifier(fn, pri)	do { (void)(fn); } while (0)
 /* These aren't inline functions due to a GCC bug. */
 #define register_hotcpu_notifier(nb)	({ (void)(nb); 0; })
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 3046a50..1c84138 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1,6 +1,18 @@
 /* CPU control.
  * (C) 2001, 2002, 2003, 2004 Rusty Russell
  *
+ * Rework of the CPU hotplug offline mechanism to remove its dependence on
+ * the heavy-weight stop_machine() primitive, by Srivatsa S. Bhat and
+ * Paul E. McKenney.
+ *
+ * Copyright (C) IBM Corporation, 2012-2013
+ * Authors: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
+ *          Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+ *
+ * With lots of invaluable suggestions from:
+ *	    Oleg Nesterov <oleg@redhat.com>
+ *	    Tejun Heo <tj@kernel.org>
+ *
  * This code is licenced under the GPL.
  */
 #include <linux/proc_fs.h>
@@ -19,6 +31,7 @@
 #include <linux/mutex.h>
 #include <linux/gfp.h>
 #include <linux/suspend.h>
+#include <linux/percpu-rwlock.h>
 
 #include "smpboot.h"
 
@@ -133,6 +146,38 @@ static void cpu_hotplug_done(void)
 	mutex_unlock(&cpu_hotplug.lock);
 }
 
+/*
+ * Per-CPU Reader-Writer lock to synchronize between atomic hotplug
+ * readers and the CPU offline hotplug writer.
+ */
+DEFINE_STATIC_PERCPU_RWLOCK(hotplug_pcpu_rwlock);
+
+/*
+ * Invoked by atomic hotplug reader (a task which wants to prevent
+ * CPU offline, but which can't afford to sleep), to prevent CPUs from
+ * going offline. So, you can call this function from atomic contexts
+ * (including interrupt handlers).
+ *
+ * Note: This does NOT prevent CPUs from coming online! It only prevents
+ * CPUs from going offline.
+ *
+ * You can call this function recursively.
+ *
+ * Returns with preemption disabled (but interrupts remain as they are;
+ * they are not disabled).
+ */
+void get_online_cpus_atomic(void)
+{
+	percpu_read_lock_irqsafe(&hotplug_pcpu_rwlock);
+}
+EXPORT_SYMBOL_GPL(get_online_cpus_atomic);
+
+void put_online_cpus_atomic(void)
+{
+	percpu_read_unlock_irqsafe(&hotplug_pcpu_rwlock);
+}
+EXPORT_SYMBOL_GPL(put_online_cpus_atomic);
+
 #else /* #if CONFIG_HOTPLUG_CPU */
 static void cpu_hotplug_begin(void) {}
 static void cpu_hotplug_done(void) {}
@@ -246,15 +291,21 @@ struct take_cpu_down_param {
 static int __ref take_cpu_down(void *_param)
 {
 	struct take_cpu_down_param *param = _param;
-	int err;
+	unsigned long flags;
+	int err = 0;
+
+	percpu_write_lock_irqsave(&hotplug_pcpu_rwlock, &flags);
 
 	/* Ensure this CPU doesn't handle any more interrupts. */
 	err = __cpu_disable();
 	if (err < 0)
-		return err;
+		goto out;
 
 	cpu_notify(CPU_DYING | param->mod, param->hcpu);
-	return 0;
+
+out:
+	percpu_write_unlock_irqrestore(&hotplug_pcpu_rwlock, &flags);
+	return err;
 }
 
 /* Requires cpu_add_remove_lock to be held */

Re: [PATCH v5 07/45] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

[Namhyung Kim]

Hi Srivatsa,

On Tue, 22 Jan 2013 13:04:54 +0530, Srivatsa S. Bhat wrote:
> @@ -246,15 +291,21 @@ struct take_cpu_down_param {
>  static int __ref take_cpu_down(void *_param)
>  {
>  	struct take_cpu_down_param *param = _param;
> -	int err;
> +	unsigned long flags;
> +	int err = 0;

It seems no need to set 'err' to 0.

Thanks,
Namhyung

> +
> +	percpu_write_lock_irqsave(&hotplug_pcpu_rwlock, &flags);
>  
>  	/* Ensure this CPU doesn't handle any more interrupts. */
>  	err = __cpu_disable();
>  	if (err < 0)
> -		return err;
> +		goto out;
>  
>  	cpu_notify(CPU_DYING | param->mod, param->hcpu);
> -	return 0;
> +
> +out:
> +	percpu_write_unlock_irqrestore(&hotplug_pcpu_rwlock, &flags);
> +	return err;
>  }
>  
>  /* Requires cpu_add_remove_lock to be held */
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v5 07/45] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

[Srivatsa S. Bhat]

Hi Namhyung,

On 01/29/2013 03:51 PM, Namhyung Kim wrote:
> Hi Srivatsa,
> 
> On Tue, 22 Jan 2013 13:04:54 +0530, Srivatsa S. Bhat wrote:
>> @@ -246,15 +291,21 @@ struct take_cpu_down_param {
>>  static int __ref take_cpu_down(void *_param)
>>  {
>>  	struct take_cpu_down_param *param = _param;
>> -	int err;
>> +	unsigned long flags;
>> +	int err = 0;
> 
> It seems no need to set 'err' to 0.
>

Sorry for the late reply. This mail got buried in my inbox and
I hadn't noticed it until now.. :-(

I'll remove the unnecessary initialization. Thank you!

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 07/45] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

[Paul E. McKenney]

On Tue, Jan 22, 2013 at 01:04:54PM +0530, Srivatsa S. Bhat wrote:
> There are places where preempt_disable() or local_irq_disable() are used
> to prevent any CPU from going offline during the critical section. Let us
> call them as "atomic hotplug readers" ("atomic" because they run in atomic,
> non-preemptible contexts).
> 
> Today, preempt_disable() or its equivalent works because the hotplug writer
> uses stop_machine() to take CPUs offline. But once stop_machine() is gone
> from the CPU hotplug offline path, the readers won't be able to prevent
> CPUs from going offline using preempt_disable().
> 
> So the intent here is to provide synchronization APIs for such atomic hotplug
> readers, to prevent (any) CPUs from going offline, without depending on
> stop_machine() at the writer-side. The new APIs will look something like
> this:  get_online_cpus_atomic() and put_online_cpus_atomic()
> 
> Some important design requirements and considerations:
> -----------------------------------------------------
> 
> 1. Scalable synchronization at the reader-side, especially in the fast-path
> 
>    Any synchronization at the atomic hotplug readers side must be highly
>    scalable - avoid global single-holder locks/counters etc. Because, these
>    paths currently use the extremely fast preempt_disable(); our replacement
>    to preempt_disable() should not become ridiculously costly and also should
>    not serialize the readers among themselves needlessly.
> 
>    At a minimum, the new APIs must be extremely fast at the reader side
>    atleast in the fast-path, when no CPU offline writers are active.
> 
> 2. preempt_disable() was recursive. The replacement should also be recursive.
> 
> 3. No (new) lock-ordering restrictions
> 
>    preempt_disable() was super-flexible. It didn't impose any ordering
>    restrictions or rules for nesting. Our replacement should also be equally
>    flexible and usable.
> 
> 4. No deadlock possibilities
> 
>    Regular per-cpu locking is not the way to go if we want to have relaxed
>    rules for lock-ordering. Because, we can end up in circular-locking
>    dependencies as explained in https://lkml.org/lkml/2012/12/6/290
> 
>    So, avoid the usual per-cpu locking schemes (per-cpu locks/per-cpu atomic
>    counters with spin-on-contention etc) as much as possible, to avoid
>    numerous deadlock possibilities from creeping in.
> 
> 
> Implementation of the design:
> ----------------------------
> 
> We use per-CPU reader-writer locks for synchronization because:
> 
>   a. They are quite fast and scalable in the fast-path (when no writers are
>      active), since they use fast per-cpu counters in those paths.
> 
>   b. They are recursive at the reader side.
> 
>   c. They provide a good amount of safety against deadlocks; they don't
>      spring new deadlock possibilities on us from out of nowhere. As a
>      result, they have relaxed locking rules and are quite flexible, and
>      thus are best suited for replacing usages of preempt_disable() or
>      local_irq_disable() at the reader side.
> 
> Together, these satisfy all the requirements mentioned above.
> 
> I'm indebted to Michael Wang and Xiao Guangrong for their numerous thoughtful
> suggestions and ideas, which inspired and influenced many of the decisions in
> this as well as previous designs. Thanks a lot Michael and Xiao!
> 
> Cc: Russell King <linux@arm.linux.org.uk>
> Cc: Mike Frysinger <vapier@gentoo.org>
> Cc: Tony Luck <tony.luck@intel.com>
> Cc: Ralf Baechle <ralf@linux-mips.org>
> Cc: David Howells <dhowells@redhat.com>
> Cc: "James E.J. Bottomley" <jejb@parisc-linux.org>
> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
> Cc: Paul Mundt <lethal@linux-sh.org>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: x86@kernel.org
> Cc: linux-arm-kernel@lists.infradead.org
> Cc: uclinux-dist-devel@blackfin.uclinux.org
> Cc: linux-ia64@vger.kernel.org
> Cc: linux-mips@linux-mips.org
> Cc: linux-am33-list@redhat.com
> Cc: linux-parisc@vger.kernel.org
> Cc: linuxppc-dev@lists.ozlabs.org
> Cc: linux-s390@vger.kernel.org
> Cc: linux-sh@vger.kernel.org
> Cc: sparclinux@vger.kernel.org
> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>

With the change suggested by Namhyung:

Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

> ---
> 
>  arch/arm/Kconfig      |    1 +
>  arch/blackfin/Kconfig |    1 +
>  arch/ia64/Kconfig     |    1 +
>  arch/mips/Kconfig     |    1 +
>  arch/mn10300/Kconfig  |    1 +
>  arch/parisc/Kconfig   |    1 +
>  arch/powerpc/Kconfig  |    1 +
>  arch/s390/Kconfig     |    1 +
>  arch/sh/Kconfig       |    1 +
>  arch/sparc/Kconfig    |    1 +
>  arch/x86/Kconfig      |    1 +
>  include/linux/cpu.h   |    4 +++
>  kernel/cpu.c          |   57 ++++++++++++++++++++++++++++++++++++++++++++++---
>  13 files changed, 69 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index 67874b8..cb6b94b 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -1616,6 +1616,7 @@ config NR_CPUS
>  config HOTPLUG_CPU
>  	bool "Support for hot-pluggable CPUs"
>  	depends on SMP && HOTPLUG
> +	select PERCPU_RWLOCK
>  	help
>  	  Say Y here to experiment with turning CPUs off and on.  CPUs
>  	  can be controlled through /sys/devices/system/cpu.
> diff --git a/arch/blackfin/Kconfig b/arch/blackfin/Kconfig
> index b6f3ad5..83d9882 100644
> --- a/arch/blackfin/Kconfig
> +++ b/arch/blackfin/Kconfig
> @@ -261,6 +261,7 @@ config NR_CPUS
>  config HOTPLUG_CPU
>  	bool "Support for hot-pluggable CPUs"
>  	depends on SMP && HOTPLUG
> +	select PERCPU_RWLOCK
>  	default y
> 
>  config BF_REV_MIN
> diff --git a/arch/ia64/Kconfig b/arch/ia64/Kconfig
> index 3279646..c246772 100644
> --- a/arch/ia64/Kconfig
> +++ b/arch/ia64/Kconfig
> @@ -378,6 +378,7 @@ config HOTPLUG_CPU
>  	bool "Support for hot-pluggable CPUs (EXPERIMENTAL)"
>  	depends on SMP && EXPERIMENTAL
>  	select HOTPLUG
> +	select PERCPU_RWLOCK
>  	default n
>  	---help---
>  	  Say Y here to experiment with turning CPUs off and on.  CPUs
> diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
> index 2ac626a..f97c479 100644
> --- a/arch/mips/Kconfig
> +++ b/arch/mips/Kconfig
> @@ -956,6 +956,7 @@ config SYS_HAS_EARLY_PRINTK
>  config HOTPLUG_CPU
>  	bool "Support for hot-pluggable CPUs"
>  	depends on SMP && HOTPLUG && SYS_SUPPORTS_HOTPLUG_CPU
> +	select PERCPU_RWLOCK
>  	help
>  	  Say Y here to allow turning CPUs off and on. CPUs can be
>  	  controlled through /sys/devices/system/cpu.
> diff --git a/arch/mn10300/Kconfig b/arch/mn10300/Kconfig
> index e70001c..a64e488 100644
> --- a/arch/mn10300/Kconfig
> +++ b/arch/mn10300/Kconfig
> @@ -60,6 +60,7 @@ config ARCH_HAS_ILOG2_U32
> 
>  config HOTPLUG_CPU
>  	def_bool n
> +	select PERCPU_RWLOCK
> 
>  source "init/Kconfig"
> 
> diff --git a/arch/parisc/Kconfig b/arch/parisc/Kconfig
> index b77feff..6f55cd4 100644
> --- a/arch/parisc/Kconfig
> +++ b/arch/parisc/Kconfig
> @@ -226,6 +226,7 @@ config HOTPLUG_CPU
>  	bool
>  	default y if SMP
>  	select HOTPLUG
> +	select PERCPU_RWLOCK
> 
>  config ARCH_SELECT_MEMORY_MODEL
>  	def_bool y
> diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
> index 17903f1..56b1f15 100644
> --- a/arch/powerpc/Kconfig
> +++ b/arch/powerpc/Kconfig
> @@ -336,6 +336,7 @@ config HOTPLUG_CPU
>  	bool "Support for enabling/disabling CPUs"
>  	depends on SMP && HOTPLUG && EXPERIMENTAL && (PPC_PSERIES || \
>  	PPC_PMAC || PPC_POWERNV || (PPC_85xx && !PPC_E500MC))
> +	select PERCPU_RWLOCK
>  	---help---
>  	  Say Y here to be able to disable and re-enable individual
>  	  CPUs at runtime on SMP machines.
> diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
> index b5ea38c..a9aafb4 100644
> --- a/arch/s390/Kconfig
> +++ b/arch/s390/Kconfig
> @@ -299,6 +299,7 @@ config HOTPLUG_CPU
>  	prompt "Support for hot-pluggable CPUs"
>  	depends on SMP
>  	select HOTPLUG
> +	select PERCPU_RWLOCK
>  	help
>  	  Say Y here to be able to turn CPUs off and on. CPUs
>  	  can be controlled through /sys/devices/system/cpu/cpu#.
> diff --git a/arch/sh/Kconfig b/arch/sh/Kconfig
> index babc2b8..8c92eef 100644
> --- a/arch/sh/Kconfig
> +++ b/arch/sh/Kconfig
> @@ -765,6 +765,7 @@ config NR_CPUS
>  config HOTPLUG_CPU
>  	bool "Support for hot-pluggable CPUs (EXPERIMENTAL)"
>  	depends on SMP && HOTPLUG && EXPERIMENTAL
> +	select PERCPU_RWLOCK
>  	help
>  	  Say Y here to experiment with turning CPUs off and on.  CPUs
>  	  can be controlled through /sys/devices/system/cpu.
> diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
> index 9f2edb5..e2bd573 100644
> --- a/arch/sparc/Kconfig
> +++ b/arch/sparc/Kconfig
> @@ -253,6 +253,7 @@ config HOTPLUG_CPU
>  	bool "Support for hot-pluggable CPUs"
>  	depends on SPARC64 && SMP
>  	select HOTPLUG
> +	select PERCPU_RWLOCK
>  	help
>  	  Say Y here to experiment with turning CPUs off and on.  CPUs
>  	  can be controlled through /sys/devices/system/cpu/cpu#.
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index 79795af..a225d12 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -1689,6 +1689,7 @@ config PHYSICAL_ALIGN
>  config HOTPLUG_CPU
>  	bool "Support for hot-pluggable CPUs"
>  	depends on SMP && HOTPLUG
> +	select PERCPU_RWLOCK
>  	---help---
>  	  Say Y here to allow turning CPUs off and on. CPUs can be
>  	  controlled through /sys/devices/system/cpu.
> diff --git a/include/linux/cpu.h b/include/linux/cpu.h
> index ce7a074..cf24da1 100644
> --- a/include/linux/cpu.h
> +++ b/include/linux/cpu.h
> @@ -175,6 +175,8 @@ extern struct bus_type cpu_subsys;
> 
>  extern void get_online_cpus(void);
>  extern void put_online_cpus(void);
> +extern void get_online_cpus_atomic(void);
> +extern void put_online_cpus_atomic(void);
>  #define hotcpu_notifier(fn, pri)	cpu_notifier(fn, pri)
>  #define register_hotcpu_notifier(nb)	register_cpu_notifier(nb)
>  #define unregister_hotcpu_notifier(nb)	unregister_cpu_notifier(nb)
> @@ -198,6 +200,8 @@ static inline void cpu_hotplug_driver_unlock(void)
> 
>  #define get_online_cpus()	do { } while (0)
>  #define put_online_cpus()	do { } while (0)
> +#define get_online_cpus_atomic()	do { } while (0)
> +#define put_online_cpus_atomic()	do { } while (0)
>  #define hotcpu_notifier(fn, pri)	do { (void)(fn); } while (0)
>  /* These aren't inline functions due to a GCC bug. */
>  #define register_hotcpu_notifier(nb)	({ (void)(nb); 0; })
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 3046a50..1c84138 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -1,6 +1,18 @@
>  /* CPU control.
>   * (C) 2001, 2002, 2003, 2004 Rusty Russell
>   *
> + * Rework of the CPU hotplug offline mechanism to remove its dependence on
> + * the heavy-weight stop_machine() primitive, by Srivatsa S. Bhat and
> + * Paul E. McKenney.
> + *
> + * Copyright (C) IBM Corporation, 2012-2013
> + * Authors: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
> + *          Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> + *
> + * With lots of invaluable suggestions from:
> + *	    Oleg Nesterov <oleg@redhat.com>
> + *	    Tejun Heo <tj@kernel.org>
> + *
>   * This code is licenced under the GPL.
>   */
>  #include <linux/proc_fs.h>
> @@ -19,6 +31,7 @@
>  #include <linux/mutex.h>
>  #include <linux/gfp.h>
>  #include <linux/suspend.h>
> +#include <linux/percpu-rwlock.h>
> 
>  #include "smpboot.h"
> 
> @@ -133,6 +146,38 @@ static void cpu_hotplug_done(void)
>  	mutex_unlock(&cpu_hotplug.lock);
>  }
> 
> +/*
> + * Per-CPU Reader-Writer lock to synchronize between atomic hotplug
> + * readers and the CPU offline hotplug writer.
> + */
> +DEFINE_STATIC_PERCPU_RWLOCK(hotplug_pcpu_rwlock);
> +
> +/*
> + * Invoked by atomic hotplug reader (a task which wants to prevent
> + * CPU offline, but which can't afford to sleep), to prevent CPUs from
> + * going offline. So, you can call this function from atomic contexts
> + * (including interrupt handlers).
> + *
> + * Note: This does NOT prevent CPUs from coming online! It only prevents
> + * CPUs from going offline.
> + *
> + * You can call this function recursively.
> + *
> + * Returns with preemption disabled (but interrupts remain as they are;
> + * they are not disabled).
> + */
> +void get_online_cpus_atomic(void)
> +{
> +	percpu_read_lock_irqsafe(&hotplug_pcpu_rwlock);
> +}
> +EXPORT_SYMBOL_GPL(get_online_cpus_atomic);
> +
> +void put_online_cpus_atomic(void)
> +{
> +	percpu_read_unlock_irqsafe(&hotplug_pcpu_rwlock);
> +}
> +EXPORT_SYMBOL_GPL(put_online_cpus_atomic);
> +
>  #else /* #if CONFIG_HOTPLUG_CPU */
>  static void cpu_hotplug_begin(void) {}
>  static void cpu_hotplug_done(void) {}
> @@ -246,15 +291,21 @@ struct take_cpu_down_param {
>  static int __ref take_cpu_down(void *_param)
>  {
>  	struct take_cpu_down_param *param = _param;
> -	int err;
> +	unsigned long flags;
> +	int err = 0;
> +
> +	percpu_write_lock_irqsave(&hotplug_pcpu_rwlock, &flags);
> 
>  	/* Ensure this CPU doesn't handle any more interrupts. */
>  	err = __cpu_disable();
>  	if (err < 0)
> -		return err;
> +		goto out;
> 
>  	cpu_notify(CPU_DYING | param->mod, param->hcpu);
> -	return 0;
> +
> +out:
> +	percpu_write_unlock_irqrestore(&hotplug_pcpu_rwlock, &flags);
> +	return err;
>  }
> 
>  /* Requires cpu_add_remove_lock to be held */
> 

[PATCH v5 08/45] CPU hotplug: Convert preprocessor macros to static inline functions

[Srivatsa S. Bhat]

On 12/05/2012 06:10 AM, Andrew Morton wrote:
"static inline C functions would be preferred if possible.  Feel free to
fix up the wrong crufty surrounding code as well ;-)"

Convert the macros in the CPU hotplug code to static inline C functions.

Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 include/linux/cpu.h |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index cf24da1..eb79f47 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -198,10 +198,10 @@ static inline void cpu_hotplug_driver_unlock(void)
 
 #else		/* CONFIG_HOTPLUG_CPU */
 
-#define get_online_cpus()	do { } while (0)
-#define put_online_cpus()	do { } while (0)
-#define get_online_cpus_atomic()	do { } while (0)
-#define put_online_cpus_atomic()	do { } while (0)
+static inline void get_online_cpus(void) {}
+static inline void put_online_cpus(void) {}
+static inline void get_online_cpus_atomic(void) {}
+static inline void put_online_cpus_atomic(void) {}
 #define hotcpu_notifier(fn, pri)	do { (void)(fn); } while (0)
 /* These aren't inline functions due to a GCC bug. */
 #define register_hotcpu_notifier(nb)	({ (void)(nb); 0; })

Re: [PATCH v5 08/45] CPU hotplug: Convert preprocessor macros to static inline functions

[Paul E. McKenney]

On Tue, Jan 22, 2013 at 01:05:02PM +0530, Srivatsa S. Bhat wrote:
> On 12/05/2012 06:10 AM, Andrew Morton wrote:
> "static inline C functions would be preferred if possible.  Feel free to
> fix up the wrong crufty surrounding code as well ;-)"
> 
> Convert the macros in the CPU hotplug code to static inline C functions.
> 
> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>

Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

> ---
> 
>  include/linux/cpu.h |    8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/include/linux/cpu.h b/include/linux/cpu.h
> index cf24da1..eb79f47 100644
> --- a/include/linux/cpu.h
> +++ b/include/linux/cpu.h
> @@ -198,10 +198,10 @@ static inline void cpu_hotplug_driver_unlock(void)
> 
>  #else		/* CONFIG_HOTPLUG_CPU */
> 
> -#define get_online_cpus()	do { } while (0)
> -#define put_online_cpus()	do { } while (0)
> -#define get_online_cpus_atomic()	do { } while (0)
> -#define put_online_cpus_atomic()	do { } while (0)
> +static inline void get_online_cpus(void) {}
> +static inline void put_online_cpus(void) {}
> +static inline void get_online_cpus_atomic(void) {}
> +static inline void put_online_cpus_atomic(void) {}
>  #define hotcpu_notifier(fn, pri)	do { (void)(fn); } while (0)
>  /* These aren't inline functions due to a GCC bug. */
>  #define register_hotcpu_notifier(nb)	({ (void)(nb); 0; })
> 

[PATCH v5 09/45] smp, cpu hotplug: Fix smp_call_function_*() to prevent CPU offline properly

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() to prevent CPUs from going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 kernel/smp.c |   40 ++++++++++++++++++++++++++--------------
 1 file changed, 26 insertions(+), 14 deletions(-)

diff --git a/kernel/smp.c b/kernel/smp.c
index 29dd40a..f421bcc 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -310,7 +310,8 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
 	 * prevent preemption and reschedule on another processor,
 	 * as well as CPU removal
 	 */
-	this_cpu = get_cpu();
+	get_online_cpus_atomic();
+	this_cpu = smp_processor_id();
 
 	/*
 	 * Can deadlock when called with interrupts disabled.
@@ -342,7 +343,7 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
 		}
 	}
 
-	put_cpu();
+	put_online_cpus_atomic();
 
 	return err;
 }
@@ -371,8 +372,10 @@ int smp_call_function_any(const struct cpumask *mask,
 	const struct cpumask *nodemask;
 	int ret;
 
+	get_online_cpus_atomic();
 	/* Try for same CPU (cheapest) */
-	cpu = get_cpu();
+	cpu = smp_processor_id();
+
 	if (cpumask_test_cpu(cpu, mask))
 		goto call;
 
@@ -388,7 +391,7 @@ int smp_call_function_any(const struct cpumask *mask,
 	cpu = cpumask_any_and(mask, cpu_online_mask);
 call:
 	ret = smp_call_function_single(cpu, func, info, wait);
-	put_cpu();
+	put_online_cpus_atomic();
 	return ret;
 }
 EXPORT_SYMBOL_GPL(smp_call_function_any);
@@ -409,25 +412,28 @@ void __smp_call_function_single(int cpu, struct call_single_data *data,
 	unsigned int this_cpu;
 	unsigned long flags;
 
-	this_cpu = get_cpu();
+	get_online_cpus_atomic();
+
+	this_cpu = smp_processor_id();
+
 	/*
 	 * Can deadlock when called with interrupts disabled.
 	 * We allow cpu's that are not yet online though, as no one else can
 	 * send smp call function interrupt to this cpu and as such deadlocks
 	 * can't happen.
 	 */
-	WARN_ON_ONCE(cpu_online(smp_processor_id()) && wait && irqs_disabled()
+	WARN_ON_ONCE(cpu_online(this_cpu) && wait && irqs_disabled()
 		     && !oops_in_progress);
 
 	if (cpu == this_cpu) {
 		local_irq_save(flags);
 		data->func(data->info);
 		local_irq_restore(flags);
-	} else {
+	} else if ((unsigned)cpu < nr_cpu_ids && cpu_online(cpu)) {
 		csd_lock(data);
 		generic_exec_single(cpu, data, wait);
 	}
-	put_cpu();
+	put_online_cpus_atomic();
 }
 
 /**
@@ -451,6 +457,8 @@ void smp_call_function_many(const struct cpumask *mask,
 	unsigned long flags;
 	int refs, cpu, next_cpu, this_cpu = smp_processor_id();
 
+	get_online_cpus_atomic();
+
 	/*
 	 * Can deadlock when called with interrupts disabled.
 	 * We allow cpu's that are not yet online though, as no one else can
@@ -467,17 +475,18 @@ void smp_call_function_many(const struct cpumask *mask,
 
 	/* No online cpus?  We're done. */
 	if (cpu >= nr_cpu_ids)
-		return;
+		goto out_unlock;
 
 	/* Do we have another CPU which isn't us? */
 	next_cpu = cpumask_next_and(cpu, mask, cpu_online_mask);
 	if (next_cpu == this_cpu)
-		next_cpu = cpumask_next_and(next_cpu, mask, cpu_online_mask);
+		next_cpu = cpumask_next_and(next_cpu, mask,
+						cpu_online_mask);
 
 	/* Fastpath: do that cpu by itself. */
 	if (next_cpu >= nr_cpu_ids) {
 		smp_call_function_single(cpu, func, info, wait);
-		return;
+		goto out_unlock;
 	}
 
 	data = &__get_cpu_var(cfd_data);
@@ -523,7 +532,7 @@ void smp_call_function_many(const struct cpumask *mask,
 	/* Some callers race with other cpus changing the passed mask */
 	if (unlikely(!refs)) {
 		csd_unlock(&data->csd);
-		return;
+		goto out_unlock;
 	}
 
 	raw_spin_lock_irqsave(&call_function.lock, flags);
@@ -554,6 +563,9 @@ void smp_call_function_many(const struct cpumask *mask,
 	/* Optionally wait for the CPUs to complete */
 	if (wait)
 		csd_lock_wait(&data->csd);
+
+out_unlock:
+	put_online_cpus_atomic();
 }
 EXPORT_SYMBOL(smp_call_function_many);
 
@@ -574,9 +586,9 @@ EXPORT_SYMBOL(smp_call_function_many);
  */
 int smp_call_function(smp_call_func_t func, void *info, int wait)
 {
-	preempt_disable();
+	get_online_cpus_atomic();
 	smp_call_function_many(cpu_online_mask, func, info, wait);
-	preempt_enable();
+	put_online_cpus_atomic();
 
 	return 0;
 }

Re: [PATCH v5 09/45] smp, cpu hotplug: Fix smp_call_function_*() to prevent CPU offline properly

[Paul E. McKenney]

On Tue, Jan 22, 2013 at 01:05:10PM +0530, Srivatsa S. Bhat wrote:
> Once stop_machine() is gone from the CPU offline path, we won't be able to
> depend on preempt_disable() to prevent CPUs from going offline from under us.
> 
> Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
> while invoking from atomic context.
> 
> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>

Would it make sense for get_online_cpus_atomic() to return the current
CPU number?  Looks good otherwise.

							Thanx, Paul

> ---
> 
>  kernel/smp.c |   40 ++++++++++++++++++++++++++--------------
>  1 file changed, 26 insertions(+), 14 deletions(-)
> 
> diff --git a/kernel/smp.c b/kernel/smp.c
> index 29dd40a..f421bcc 100644
> --- a/kernel/smp.c
> +++ b/kernel/smp.c
> @@ -310,7 +310,8 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
>  	 * prevent preemption and reschedule on another processor,
>  	 * as well as CPU removal
>  	 */
> -	this_cpu = get_cpu();
> +	get_online_cpus_atomic();
> +	this_cpu = smp_processor_id();
> 
>  	/*
>  	 * Can deadlock when called with interrupts disabled.
> @@ -342,7 +343,7 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
>  		}
>  	}
> 
> -	put_cpu();
> +	put_online_cpus_atomic();
> 
>  	return err;
>  }
> @@ -371,8 +372,10 @@ int smp_call_function_any(const struct cpumask *mask,
>  	const struct cpumask *nodemask;
>  	int ret;
> 
> +	get_online_cpus_atomic();
>  	/* Try for same CPU (cheapest) */
> -	cpu = get_cpu();
> +	cpu = smp_processor_id();
> +
>  	if (cpumask_test_cpu(cpu, mask))
>  		goto call;
> 
> @@ -388,7 +391,7 @@ int smp_call_function_any(const struct cpumask *mask,
>  	cpu = cpumask_any_and(mask, cpu_online_mask);
>  call:
>  	ret = smp_call_function_single(cpu, func, info, wait);
> -	put_cpu();
> +	put_online_cpus_atomic();
>  	return ret;
>  }
>  EXPORT_SYMBOL_GPL(smp_call_function_any);
> @@ -409,25 +412,28 @@ void __smp_call_function_single(int cpu, struct call_single_data *data,
>  	unsigned int this_cpu;
>  	unsigned long flags;
> 
> -	this_cpu = get_cpu();
> +	get_online_cpus_atomic();
> +
> +	this_cpu = smp_processor_id();
> +
>  	/*
>  	 * Can deadlock when called with interrupts disabled.
>  	 * We allow cpu's that are not yet online though, as no one else can
>  	 * send smp call function interrupt to this cpu and as such deadlocks
>  	 * can't happen.
>  	 */
> -	WARN_ON_ONCE(cpu_online(smp_processor_id()) && wait && irqs_disabled()
> +	WARN_ON_ONCE(cpu_online(this_cpu) && wait && irqs_disabled()
>  		     && !oops_in_progress);
> 
>  	if (cpu == this_cpu) {
>  		local_irq_save(flags);
>  		data->func(data->info);
>  		local_irq_restore(flags);
> -	} else {
> +	} else if ((unsigned)cpu < nr_cpu_ids && cpu_online(cpu)) {
>  		csd_lock(data);
>  		generic_exec_single(cpu, data, wait);
>  	}
> -	put_cpu();
> +	put_online_cpus_atomic();
>  }
> 
>  /**
> @@ -451,6 +457,8 @@ void smp_call_function_many(const struct cpumask *mask,
>  	unsigned long flags;
>  	int refs, cpu, next_cpu, this_cpu = smp_processor_id();
> 
> +	get_online_cpus_atomic();
> +
>  	/*
>  	 * Can deadlock when called with interrupts disabled.
>  	 * We allow cpu's that are not yet online though, as no one else can
> @@ -467,17 +475,18 @@ void smp_call_function_many(const struct cpumask *mask,
> 
>  	/* No online cpus?  We're done. */
>  	if (cpu >= nr_cpu_ids)
> -		return;
> +		goto out_unlock;
> 
>  	/* Do we have another CPU which isn't us? */
>  	next_cpu = cpumask_next_and(cpu, mask, cpu_online_mask);
>  	if (next_cpu == this_cpu)
> -		next_cpu = cpumask_next_and(next_cpu, mask, cpu_online_mask);
> +		next_cpu = cpumask_next_and(next_cpu, mask,
> +						cpu_online_mask);
> 
>  	/* Fastpath: do that cpu by itself. */
>  	if (next_cpu >= nr_cpu_ids) {
>  		smp_call_function_single(cpu, func, info, wait);
> -		return;
> +		goto out_unlock;
>  	}
> 
>  	data = &__get_cpu_var(cfd_data);
> @@ -523,7 +532,7 @@ void smp_call_function_many(const struct cpumask *mask,
>  	/* Some callers race with other cpus changing the passed mask */
>  	if (unlikely(!refs)) {
>  		csd_unlock(&data->csd);
> -		return;
> +		goto out_unlock;
>  	}
> 
>  	raw_spin_lock_irqsave(&call_function.lock, flags);
> @@ -554,6 +563,9 @@ void smp_call_function_many(const struct cpumask *mask,
>  	/* Optionally wait for the CPUs to complete */
>  	if (wait)
>  		csd_lock_wait(&data->csd);
> +
> +out_unlock:
> +	put_online_cpus_atomic();
>  }
>  EXPORT_SYMBOL(smp_call_function_many);
> 
> @@ -574,9 +586,9 @@ EXPORT_SYMBOL(smp_call_function_many);
>   */
>  int smp_call_function(smp_call_func_t func, void *info, int wait)
>  {
> -	preempt_disable();
> +	get_online_cpus_atomic();
>  	smp_call_function_many(cpu_online_mask, func, info, wait);
> -	preempt_enable();
> +	put_online_cpus_atomic();
> 
>  	return 0;
>  }
> 

Re: [PATCH v5 09/45] smp, cpu hotplug: Fix smp_call_function_*() to prevent CPU offline properly

[Srivatsa S. Bhat]

On 02/09/2013 05:37 AM, Paul E. McKenney wrote:
> On Tue, Jan 22, 2013 at 01:05:10PM +0530, Srivatsa S. Bhat wrote:
>> Once stop_machine() is gone from the CPU offline path, we won't be able to
>> depend on preempt_disable() to prevent CPUs from going offline from under us.
>>
>> Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
>> while invoking from atomic context.
>>
>> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
> 
> Would it make sense for get_online_cpus_atomic() to return the current
> CPU number?

Hmm, I'm not so sure. I tried to model it after get_online_cpus(), which doesn't
return anything (for other reasons, of course..)

Moreover, a function name like *_cpu_* returning the CPU number would be intuitive.
But a name such as *_cpus_* (plural) returning a CPU number might appear confusing..

And also I don't think it'll make a huge improvement in the callers.. (We might
be better off avoiding an smp_processor_id() if possible, since this function could
be called in very hot paths).. So I don't see a strong case for returning the
CPU number. But let me know if you think it'll still be worth it for some reason...

>  Looks good otherwise.
> 

Thank you very much for the detailed review, Paul!

Regards,
Srivatsa S. Bhat

> 
>> ---
>>
>>  kernel/smp.c |   40 ++++++++++++++++++++++++++--------------
>>  1 file changed, 26 insertions(+), 14 deletions(-)
>>
>> diff --git a/kernel/smp.c b/kernel/smp.c
>> index 29dd40a..f421bcc 100644
>> --- a/kernel/smp.c
>> +++ b/kernel/smp.c
>> @@ -310,7 +310,8 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
>>  	 * prevent preemption and reschedule on another processor,
>>  	 * as well as CPU removal
>>  	 */
>> -	this_cpu = get_cpu();
>> +	get_online_cpus_atomic();
>> +	this_cpu = smp_processor_id();
>>
>>  	/*
>>  	 * Can deadlock when called with interrupts disabled.
>> @@ -342,7 +343,7 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
>>  		}
>>  	}
>>
>> -	put_cpu();
>> +	put_online_cpus_atomic();
>>
>>  	return err;
>>  }
>> @@ -371,8 +372,10 @@ int smp_call_function_any(const struct cpumask *mask,
>>  	const struct cpumask *nodemask;
>>  	int ret;
>>
>> +	get_online_cpus_atomic();
>>  	/* Try for same CPU (cheapest) */
>> -	cpu = get_cpu();
>> +	cpu = smp_processor_id();
>> +
>>  	if (cpumask_test_cpu(cpu, mask))
>>  		goto call;
>>
>> @@ -388,7 +391,7 @@ int smp_call_function_any(const struct cpumask *mask,
>>  	cpu = cpumask_any_and(mask, cpu_online_mask);
>>  call:
>>  	ret = smp_call_function_single(cpu, func, info, wait);
>> -	put_cpu();
>> +	put_online_cpus_atomic();
>>  	return ret;
>>  }
>>  EXPORT_SYMBOL_GPL(smp_call_function_any);
>> @@ -409,25 +412,28 @@ void __smp_call_function_single(int cpu, struct call_single_data *data,
>>  	unsigned int this_cpu;
>>  	unsigned long flags;
>>
>> -	this_cpu = get_cpu();
>> +	get_online_cpus_atomic();
>> +
>> +	this_cpu = smp_processor_id();
>> +
>>  	/*
>>  	 * Can deadlock when called with interrupts disabled.
>>  	 * We allow cpu's that are not yet online though, as no one else can
>>  	 * send smp call function interrupt to this cpu and as such deadlocks
>>  	 * can't happen.
>>  	 */
>> -	WARN_ON_ONCE(cpu_online(smp_processor_id()) && wait && irqs_disabled()
>> +	WARN_ON_ONCE(cpu_online(this_cpu) && wait && irqs_disabled()
>>  		     && !oops_in_progress);
>>
>>  	if (cpu == this_cpu) {
>>  		local_irq_save(flags);
>>  		data->func(data->info);
>>  		local_irq_restore(flags);
>> -	} else {
>> +	} else if ((unsigned)cpu < nr_cpu_ids && cpu_online(cpu)) {
>>  		csd_lock(data);
>>  		generic_exec_single(cpu, data, wait);
>>  	}
>> -	put_cpu();
>> +	put_online_cpus_atomic();
>>  }
>>
>>  /**
>> @@ -451,6 +457,8 @@ void smp_call_function_many(const struct cpumask *mask,
>>  	unsigned long flags;
>>  	int refs, cpu, next_cpu, this_cpu = smp_processor_id();
>>
>> +	get_online_cpus_atomic();
>> +
>>  	/*
>>  	 * Can deadlock when called with interrupts disabled.
>>  	 * We allow cpu's that are not yet online though, as no one else can
>> @@ -467,17 +475,18 @@ void smp_call_function_many(const struct cpumask *mask,
>>
>>  	/* No online cpus?  We're done. */
>>  	if (cpu >= nr_cpu_ids)
>> -		return;
>> +		goto out_unlock;
>>
>>  	/* Do we have another CPU which isn't us? */
>>  	next_cpu = cpumask_next_and(cpu, mask, cpu_online_mask);
>>  	if (next_cpu == this_cpu)
>> -		next_cpu = cpumask_next_and(next_cpu, mask, cpu_online_mask);
>> +		next_cpu = cpumask_next_and(next_cpu, mask,
>> +						cpu_online_mask);
>>
>>  	/* Fastpath: do that cpu by itself. */
>>  	if (next_cpu >= nr_cpu_ids) {
>>  		smp_call_function_single(cpu, func, info, wait);
>> -		return;
>> +		goto out_unlock;
>>  	}
>>
>>  	data = &__get_cpu_var(cfd_data);
>> @@ -523,7 +532,7 @@ void smp_call_function_many(const struct cpumask *mask,
>>  	/* Some callers race with other cpus changing the passed mask */
>>  	if (unlikely(!refs)) {
>>  		csd_unlock(&data->csd);
>> -		return;
>> +		goto out_unlock;
>>  	}
>>
>>  	raw_spin_lock_irqsave(&call_function.lock, flags);
>> @@ -554,6 +563,9 @@ void smp_call_function_many(const struct cpumask *mask,
>>  	/* Optionally wait for the CPUs to complete */
>>  	if (wait)
>>  		csd_lock_wait(&data->csd);
>> +
>> +out_unlock:
>> +	put_online_cpus_atomic();
>>  }
>>  EXPORT_SYMBOL(smp_call_function_many);
>>
>> @@ -574,9 +586,9 @@ EXPORT_SYMBOL(smp_call_function_many);
>>   */
>>  int smp_call_function(smp_call_func_t func, void *info, int wait)
>>  {
>> -	preempt_disable();
>> +	get_online_cpus_atomic();
>>  	smp_call_function_many(cpu_online_mask, func, info, wait);
>> -	preempt_enable();
>> +	put_online_cpus_atomic();
>>
>>  	return 0;
>>  }
>>

Re: [PATCH v5 09/45] smp, cpu hotplug: Fix smp_call_function_*() to prevent CPU offline properly

[Paul E. McKenney]

On Mon, Feb 11, 2013 at 01:11:29AM +0530, Srivatsa S. Bhat wrote:
> On 02/09/2013 05:37 AM, Paul E. McKenney wrote:
> > On Tue, Jan 22, 2013 at 01:05:10PM +0530, Srivatsa S. Bhat wrote:
> >> Once stop_machine() is gone from the CPU offline path, we won't be able to
> >> depend on preempt_disable() to prevent CPUs from going offline from under us.
> >>
> >> Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
> >> while invoking from atomic context.
> >>
> >> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
> > 
> > Would it make sense for get_online_cpus_atomic() to return the current
> > CPU number?
> 
> Hmm, I'm not so sure. I tried to model it after get_online_cpus(), which doesn't
> return anything (for other reasons, of course..)
> 
> Moreover, a function name like *_cpu_* returning the CPU number would be intuitive.
> But a name such as *_cpus_* (plural) returning a CPU number might appear confusing..
> 
> And also I don't think it'll make a huge improvement in the callers.. (We might
> be better off avoiding an smp_processor_id() if possible, since this function could
> be called in very hot paths).. So I don't see a strong case for returning the
> CPU number. But let me know if you think it'll still be worth it for some reason...

I just noted a lot of two-line code sequences in your patch that would be
one line if the CPU number was returned.  But I don't feel strongly about
it, so if people are OK with the current version, no problem.

							Thanx, Paul

> >  Looks good otherwise.
> > 
> 
> Thank you very much for the detailed review, Paul!
> 
> Regards,
> Srivatsa S. Bhat
> 
> > 
> >> ---
> >>
> >>  kernel/smp.c |   40 ++++++++++++++++++++++++++--------------
> >>  1 file changed, 26 insertions(+), 14 deletions(-)
> >>
> >> diff --git a/kernel/smp.c b/kernel/smp.c
> >> index 29dd40a..f421bcc 100644
> >> --- a/kernel/smp.c
> >> +++ b/kernel/smp.c
> >> @@ -310,7 +310,8 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
> >>  	 * prevent preemption and reschedule on another processor,
> >>  	 * as well as CPU removal
> >>  	 */
> >> -	this_cpu = get_cpu();
> >> +	get_online_cpus_atomic();
> >> +	this_cpu = smp_processor_id();
> >>
> >>  	/*
> >>  	 * Can deadlock when called with interrupts disabled.
> >> @@ -342,7 +343,7 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
> >>  		}
> >>  	}
> >>
> >> -	put_cpu();
> >> +	put_online_cpus_atomic();
> >>
> >>  	return err;
> >>  }
> >> @@ -371,8 +372,10 @@ int smp_call_function_any(const struct cpumask *mask,
> >>  	const struct cpumask *nodemask;
> >>  	int ret;
> >>
> >> +	get_online_cpus_atomic();
> >>  	/* Try for same CPU (cheapest) */
> >> -	cpu = get_cpu();
> >> +	cpu = smp_processor_id();
> >> +
> >>  	if (cpumask_test_cpu(cpu, mask))
> >>  		goto call;
> >>
> >> @@ -388,7 +391,7 @@ int smp_call_function_any(const struct cpumask *mask,
> >>  	cpu = cpumask_any_and(mask, cpu_online_mask);
> >>  call:
> >>  	ret = smp_call_function_single(cpu, func, info, wait);
> >> -	put_cpu();
> >> +	put_online_cpus_atomic();
> >>  	return ret;
> >>  }
> >>  EXPORT_SYMBOL_GPL(smp_call_function_any);
> >> @@ -409,25 +412,28 @@ void __smp_call_function_single(int cpu, struct call_single_data *data,
> >>  	unsigned int this_cpu;
> >>  	unsigned long flags;
> >>
> >> -	this_cpu = get_cpu();
> >> +	get_online_cpus_atomic();
> >> +
> >> +	this_cpu = smp_processor_id();
> >> +
> >>  	/*
> >>  	 * Can deadlock when called with interrupts disabled.
> >>  	 * We allow cpu's that are not yet online though, as no one else can
> >>  	 * send smp call function interrupt to this cpu and as such deadlocks
> >>  	 * can't happen.
> >>  	 */
> >> -	WARN_ON_ONCE(cpu_online(smp_processor_id()) && wait && irqs_disabled()
> >> +	WARN_ON_ONCE(cpu_online(this_cpu) && wait && irqs_disabled()
> >>  		     && !oops_in_progress);
> >>
> >>  	if (cpu == this_cpu) {
> >>  		local_irq_save(flags);
> >>  		data->func(data->info);
> >>  		local_irq_restore(flags);
> >> -	} else {
> >> +	} else if ((unsigned)cpu < nr_cpu_ids && cpu_online(cpu)) {
> >>  		csd_lock(data);
> >>  		generic_exec_single(cpu, data, wait);
> >>  	}
> >> -	put_cpu();
> >> +	put_online_cpus_atomic();
> >>  }
> >>
> >>  /**
> >> @@ -451,6 +457,8 @@ void smp_call_function_many(const struct cpumask *mask,
> >>  	unsigned long flags;
> >>  	int refs, cpu, next_cpu, this_cpu = smp_processor_id();
> >>
> >> +	get_online_cpus_atomic();
> >> +
> >>  	/*
> >>  	 * Can deadlock when called with interrupts disabled.
> >>  	 * We allow cpu's that are not yet online though, as no one else can
> >> @@ -467,17 +475,18 @@ void smp_call_function_many(const struct cpumask *mask,
> >>
> >>  	/* No online cpus?  We're done. */
> >>  	if (cpu >= nr_cpu_ids)
> >> -		return;
> >> +		goto out_unlock;
> >>
> >>  	/* Do we have another CPU which isn't us? */
> >>  	next_cpu = cpumask_next_and(cpu, mask, cpu_online_mask);
> >>  	if (next_cpu == this_cpu)
> >> -		next_cpu = cpumask_next_and(next_cpu, mask, cpu_online_mask);
> >> +		next_cpu = cpumask_next_and(next_cpu, mask,
> >> +						cpu_online_mask);
> >>
> >>  	/* Fastpath: do that cpu by itself. */
> >>  	if (next_cpu >= nr_cpu_ids) {
> >>  		smp_call_function_single(cpu, func, info, wait);
> >> -		return;
> >> +		goto out_unlock;
> >>  	}
> >>
> >>  	data = &__get_cpu_var(cfd_data);
> >> @@ -523,7 +532,7 @@ void smp_call_function_many(const struct cpumask *mask,
> >>  	/* Some callers race with other cpus changing the passed mask */
> >>  	if (unlikely(!refs)) {
> >>  		csd_unlock(&data->csd);
> >> -		return;
> >> +		goto out_unlock;
> >>  	}
> >>
> >>  	raw_spin_lock_irqsave(&call_function.lock, flags);
> >> @@ -554,6 +563,9 @@ void smp_call_function_many(const struct cpumask *mask,
> >>  	/* Optionally wait for the CPUs to complete */
> >>  	if (wait)
> >>  		csd_lock_wait(&data->csd);
> >> +
> >> +out_unlock:
> >> +	put_online_cpus_atomic();
> >>  }
> >>  EXPORT_SYMBOL(smp_call_function_many);
> >>
> >> @@ -574,9 +586,9 @@ EXPORT_SYMBOL(smp_call_function_many);
> >>   */
> >>  int smp_call_function(smp_call_func_t func, void *info, int wait)
> >>  {
> >> -	preempt_disable();
> >> +	get_online_cpus_atomic();
> >>  	smp_call_function_many(cpu_online_mask, func, info, wait);
> >> -	preempt_enable();
> >> +	put_online_cpus_atomic();
> >>
> >>  	return 0;
> >>  }
> >>
> 

Re: [PATCH v5 09/45] smp, cpu hotplug: Fix smp_call_function_*() to prevent CPU offline properly

[Srivatsa S. Bhat]

On 02/11/2013 01:26 AM, Paul E. McKenney wrote:
> On Mon, Feb 11, 2013 at 01:11:29AM +0530, Srivatsa S. Bhat wrote:
>> On 02/09/2013 05:37 AM, Paul E. McKenney wrote:
>>> On Tue, Jan 22, 2013 at 01:05:10PM +0530, Srivatsa S. Bhat wrote:
>>>> Once stop_machine() is gone from the CPU offline path, we won't be able to
>>>> depend on preempt_disable() to prevent CPUs from going offline from under us.
>>>>
>>>> Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
>>>> while invoking from atomic context.
>>>>
>>>> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
>>>
>>> Would it make sense for get_online_cpus_atomic() to return the current
>>> CPU number?
>>
>> Hmm, I'm not so sure. I tried to model it after get_online_cpus(), which doesn't
>> return anything (for other reasons, of course..)
>>
>> Moreover, a function name like *_cpu_* returning the CPU number would be intuitive.
>> But a name such as *_cpus_* (plural) returning a CPU number might appear confusing..
>>
>> And also I don't think it'll make a huge improvement in the callers.. (We might
>> be better off avoiding an smp_processor_id() if possible, since this function could
>> be called in very hot paths).. So I don't see a strong case for returning the
>> CPU number. But let me know if you think it'll still be worth it for some reason...
> 
> I just noted a lot of two-line code sequences in your patch that would be
> one line if the CPU number was returned.

Ah, in that case, I'll reconsider your suggestion while working on the next version.
Thanks!

Regards,
Srivatsa S. Bhat

>  But I don't feel strongly about
> it, so if people are OK with the current version, no problem.
> 
> 							Thanx, Paul
> 

[PATCH v5 10/45] smp, cpu hotplug: Fix on_each_cpu_*() to prevent CPU offline properly

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() to prevent CPUs from going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 kernel/smp.c |   25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/kernel/smp.c b/kernel/smp.c
index f421bcc..d870bfe 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -688,12 +688,12 @@ int on_each_cpu(void (*func) (void *info), void *info, int wait)
 	unsigned long flags;
 	int ret = 0;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 	ret = smp_call_function(func, info, wait);
 	local_irq_save(flags);
 	func(info);
 	local_irq_restore(flags);
-	preempt_enable();
+	put_online_cpus_atomic();
 	return ret;
 }
 EXPORT_SYMBOL(on_each_cpu);
@@ -715,7 +715,11 @@ EXPORT_SYMBOL(on_each_cpu);
 void on_each_cpu_mask(const struct cpumask *mask, smp_call_func_t func,
 			void *info, bool wait)
 {
-	int cpu = get_cpu();
+	int cpu;
+
+	get_online_cpus_atomic();
+
+	cpu = smp_processor_id();
 
 	smp_call_function_many(mask, func, info, wait);
 	if (cpumask_test_cpu(cpu, mask)) {
@@ -723,7 +727,7 @@ void on_each_cpu_mask(const struct cpumask *mask, smp_call_func_t func,
 		func(info);
 		local_irq_enable();
 	}
-	put_cpu();
+	put_online_cpus_atomic();
 }
 EXPORT_SYMBOL(on_each_cpu_mask);
 
@@ -748,8 +752,9 @@ EXPORT_SYMBOL(on_each_cpu_mask);
  * The function might sleep if the GFP flags indicates a non
  * atomic allocation is allowed.
  *
- * Preemption is disabled to protect against CPUs going offline but not online.
- * CPUs going online during the call will not be seen or sent an IPI.
+ * We use get/put_online_cpus_atomic() to prevent CPUs from going
+ * offline in-between our operation. CPUs coming online during the
+ * call will not be seen or sent an IPI.
  *
  * You must not call this function with disabled interrupts or
  * from a hardware interrupt handler or from a bottom half handler.
@@ -764,26 +769,26 @@ void on_each_cpu_cond(bool (*cond_func)(int cpu, void *info),
 	might_sleep_if(gfp_flags & __GFP_WAIT);
 
 	if (likely(zalloc_cpumask_var(&cpus, (gfp_flags|__GFP_NOWARN)))) {
-		preempt_disable();
+		get_online_cpus_atomic();
 		for_each_online_cpu(cpu)
 			if (cond_func(cpu, info))
 				cpumask_set_cpu(cpu, cpus);
 		on_each_cpu_mask(cpus, func, info, wait);
-		preempt_enable();
+		put_online_cpus_atomic();
 		free_cpumask_var(cpus);
 	} else {
 		/*
 		 * No free cpumask, bother. No matter, we'll
 		 * just have to IPI them one by one.
 		 */
-		preempt_disable();
+		get_online_cpus_atomic();
 		for_each_online_cpu(cpu)
 			if (cond_func(cpu, info)) {
 				ret = smp_call_function_single(cpu, func,
 								info, wait);
 				WARN_ON_ONCE(!ret);
 			}
-		preempt_enable();
+		put_online_cpus_atomic();
 	}
 }
 EXPORT_SYMBOL(on_each_cpu_cond);

[PATCH v5 11/45] sched/timer: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from going
offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 kernel/sched/core.c |   24 +++++++++++++++++++++---
 kernel/sched/fair.c |    5 ++++-
 kernel/timer.c      |    2 ++
 3 files changed, 27 insertions(+), 4 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 257002c..c1596ac 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1117,11 +1117,11 @@ void kick_process(struct task_struct *p)
 {
 	int cpu;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 	cpu = task_cpu(p);
 	if ((cpu != smp_processor_id()) && task_curr(p))
 		smp_send_reschedule(cpu);
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 EXPORT_SYMBOL_GPL(kick_process);
 #endif /* CONFIG_SMP */
@@ -1129,6 +1129,10 @@ EXPORT_SYMBOL_GPL(kick_process);
 #ifdef CONFIG_SMP
 /*
  * ->cpus_allowed is protected by both rq->lock and p->pi_lock
+ *
+ *  Must be called under get/put_online_cpus_atomic() or
+ *  equivalent, to avoid CPUs from going offline from underneath
+ *  us.
  */
 static int select_fallback_rq(int cpu, struct task_struct *p)
 {
@@ -1192,6 +1196,9 @@ out:
 
 /*
  * The caller (fork, wakeup) owns p->pi_lock, ->cpus_allowed is stable.
+ *
+ * Must be called under get/put_online_cpus_atomic(), to prevent
+ * CPUs from going offline from underneath us.
  */
 static inline
 int select_task_rq(struct task_struct *p, int sd_flags, int wake_flags)
@@ -1432,6 +1439,7 @@ try_to_wake_up(struct task_struct *p, unsigned int state, int wake_flags)
 	int cpu, success = 0;
 
 	smp_wmb();
+	get_online_cpus_atomic();
 	raw_spin_lock_irqsave(&p->pi_lock, flags);
 	if (!(p->state & state))
 		goto out;
@@ -1472,6 +1480,7 @@ stat:
 	ttwu_stat(p, cpu, wake_flags);
 out:
 	raw_spin_unlock_irqrestore(&p->pi_lock, flags);
+	put_online_cpus_atomic();
 
 	return success;
 }
@@ -1692,6 +1701,7 @@ void wake_up_new_task(struct task_struct *p)
 	unsigned long flags;
 	struct rq *rq;
 
+	get_online_cpus_atomic();
 	raw_spin_lock_irqsave(&p->pi_lock, flags);
 #ifdef CONFIG_SMP
 	/*
@@ -1712,6 +1722,7 @@ void wake_up_new_task(struct task_struct *p)
 		p->sched_class->task_woken(rq, p);
 #endif
 	task_rq_unlock(rq, p, &flags);
+	put_online_cpus_atomic();
 }
 
 #ifdef CONFIG_PREEMPT_NOTIFIERS
@@ -2609,6 +2620,7 @@ void sched_exec(void)
 	unsigned long flags;
 	int dest_cpu;
 
+	get_online_cpus_atomic();
 	raw_spin_lock_irqsave(&p->pi_lock, flags);
 	dest_cpu = p->sched_class->select_task_rq(p, SD_BALANCE_EXEC, 0);
 	if (dest_cpu == smp_processor_id())
@@ -2618,11 +2630,13 @@ void sched_exec(void)
 		struct migration_arg arg = { p, dest_cpu };
 
 		raw_spin_unlock_irqrestore(&p->pi_lock, flags);
+		put_online_cpus_atomic();
 		stop_one_cpu(task_cpu(p), migration_cpu_stop, &arg);
 		return;
 	}
 unlock:
 	raw_spin_unlock_irqrestore(&p->pi_lock, flags);
+	put_online_cpus_atomic();
 }
 
 #endif
@@ -4372,6 +4386,7 @@ bool __sched yield_to(struct task_struct *p, bool preempt)
 	unsigned long flags;
 	bool yielded = 0;
 
+	get_online_cpus_atomic();
 	local_irq_save(flags);
 	rq = this_rq();
 
@@ -4399,13 +4414,14 @@ again:
 		 * Make p's CPU reschedule; pick_next_entity takes care of
 		 * fairness.
 		 */
-		if (preempt && rq != p_rq)
+		if (preempt && rq != p_rq && cpu_online(task_cpu(p)))
 			resched_task(p_rq->curr);
 	}
 
 out:
 	double_rq_unlock(rq, p_rq);
 	local_irq_restore(flags);
+	put_online_cpus_atomic();
 
 	if (yielded)
 		schedule();
@@ -4810,9 +4826,11 @@ static int migration_cpu_stop(void *data)
 	 * The original target cpu might have gone down and we might
 	 * be on another cpu but it doesn't matter.
 	 */
+	get_online_cpus_atomic();
 	local_irq_disable();
 	__migrate_task(arg->task, raw_smp_processor_id(), arg->dest_cpu);
 	local_irq_enable();
+	put_online_cpus_atomic();
 	return 0;
 }
 
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 5eea870..a846028 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -5695,8 +5695,11 @@ void trigger_load_balance(struct rq *rq, int cpu)
 	    likely(!on_null_domain(cpu)))
 		raise_softirq(SCHED_SOFTIRQ);
 #ifdef CONFIG_NO_HZ
-	if (nohz_kick_needed(rq, cpu) && likely(!on_null_domain(cpu)))
+	if (nohz_kick_needed(rq, cpu) && likely(!on_null_domain(cpu))) {
+		get_online_cpus_atomic();
 		nohz_balancer_kick(cpu);
+		put_online_cpus_atomic();
+	}
 #endif
 }
 
diff --git a/kernel/timer.c b/kernel/timer.c
index 367d008..b1820e3 100644
--- a/kernel/timer.c
+++ b/kernel/timer.c
@@ -924,6 +924,7 @@ void add_timer_on(struct timer_list *timer, int cpu)
 
 	timer_stats_timer_set_start_info(timer);
 	BUG_ON(timer_pending(timer) || !timer->function);
+	get_online_cpus_atomic();
 	spin_lock_irqsave(&base->lock, flags);
 	timer_set_base(timer, base);
 	debug_activate(timer, timer->expires);
@@ -938,6 +939,7 @@ void add_timer_on(struct timer_list *timer, int cpu)
 	 */
 	wake_up_idle_cpu(cpu);
 	spin_unlock_irqrestore(&base->lock, flags);
+	put_online_cpus_atomic();
 }
 EXPORT_SYMBOL_GPL(add_timer_on);
 

[PATCH v5 12/45] sched/migration: Use raw_spin_lock/unlock since interrupts are already disabled

[Srivatsa S. Bhat]

We need not use the raw_spin_lock_irqsave/restore primitives because
all CPU_DYING notifiers run with interrupts disabled. So just use
raw_spin_lock/unlock.

Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 kernel/sched/core.c |   12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index c1596ac..c2cec88 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4869,9 +4869,7 @@ static void calc_load_migrate(struct rq *rq)
  * Migrate all tasks from the rq, sleeping tasks will be migrated by
  * try_to_wake_up()->select_task_rq().
  *
- * Called with rq->lock held even though we'er in stop_machine() and
- * there's no concurrency possible, we hold the required locks anyway
- * because of lock validation efforts.
+ * Called with rq->lock held.
  */
 static void migrate_tasks(unsigned int dead_cpu)
 {
@@ -4883,8 +4881,8 @@ static void migrate_tasks(unsigned int dead_cpu)
 	 * Fudge the rq selection such that the below task selection loop
 	 * doesn't get stuck on the currently eligible stop task.
 	 *
-	 * We're currently inside stop_machine() and the rq is either stuck
-	 * in the stop_machine_cpu_stop() loop, or we're executing this code,
+	 * We're currently inside stop_one_cpu() and the rq is either stuck
+	 * in the cpu_stopper_thread(), or we're executing this code,
 	 * either way we should never end up calling schedule() until we're
 	 * done here.
 	 */
@@ -5153,14 +5151,14 @@ migration_call(struct notifier_block *nfb, unsigned long action, void *hcpu)
 	case CPU_DYING:
 		sched_ttwu_pending();
 		/* Update our root-domain */
-		raw_spin_lock_irqsave(&rq->lock, flags);
+		raw_spin_lock(&rq->lock); /* Interrupts already disabled */
 		if (rq->rd) {
 			BUG_ON(!cpumask_test_cpu(cpu, rq->rd->span));
 			set_rq_offline(rq);
 		}
 		migrate_tasks(cpu);
 		BUG_ON(rq->nr_running != 1); /* the migration thread */
-		raw_spin_unlock_irqrestore(&rq->lock, flags);
+		raw_spin_unlock(&rq->lock);
 		break;
 
 	case CPU_DEAD:

[PATCH v5 13/45] sched/rt: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 kernel/sched/rt.c |    3 +++
 1 file changed, 3 insertions(+)

diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index 418feb0..2a637be 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -6,6 +6,7 @@
 #include "sched.h"
 
 #include <linux/slab.h>
+#include <linux/cpu.h>
 
 static int do_sched_rt_period_timer(struct rt_bandwidth *rt_b, int overrun);
 
@@ -26,7 +27,9 @@ static enum hrtimer_restart sched_rt_period_timer(struct hrtimer *timer)
 		if (!overrun)
 			break;
 
+		get_online_cpus_atomic();
 		idle = do_sched_rt_period_timer(rt_b, overrun);
+		put_online_cpus_atomic();
 	}
 
 	return idle ? HRTIMER_NORESTART : HRTIMER_RESTART;

[PATCH v5 14/45] rcu, CPU hotplug: Fix comment referring to stop_machine()

[Srivatsa S. Bhat]

Don't refer to stop_machine() in the CPU hotplug path, since we are going
to get rid of it. Also, move the comment referring to callback adoption
to the CPU_DEAD case, because that's where it happens now.

Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 kernel/rcutree.c |    9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index e441b77..ac94474 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -2827,11 +2827,6 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
 		break;
 	case CPU_DYING:
 	case CPU_DYING_FROZEN:
-		/*
-		 * The whole machine is "stopped" except this CPU, so we can
-		 * touch any data without introducing corruption. We send the
-		 * dying CPU's callbacks to an arbitrarily chosen online CPU.
-		 */
 		for_each_rcu_flavor(rsp)
 			rcu_cleanup_dying_cpu(rsp);
 		rcu_cleanup_after_idle(cpu);
@@ -2840,6 +2835,10 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
 	case CPU_DEAD_FROZEN:
 	case CPU_UP_CANCELED:
 	case CPU_UP_CANCELED_FROZEN:
+		/*
+		 * We send the dead CPU's callbacks to an arbitrarily chosen
+		 * online CPU.
+		 */
 		for_each_rcu_flavor(rsp)
 			rcu_cleanup_dead_cpu(cpu, rsp);
 		break;

Re: [PATCH v5 14/45] rcu, CPU hotplug: Fix comment referring to stop_machine()

[Paul E. McKenney]

On Tue, Jan 22, 2013 at 01:06:34PM +0530, Srivatsa S. Bhat wrote:
> Don't refer to stop_machine() in the CPU hotplug path, since we are going
> to get rid of it. Also, move the comment referring to callback adoption
> to the CPU_DEAD case, because that's where it happens now.
> 
> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>

Ouch!  That comment is indeed obsolete and must die.

I queued this to -rcu with your Signed-off-by.  However, I omitted
the added comment, as it is imcomplete -- it is easy to look at
rcu_cleanup_dead_cpu() to see what it does.

							Thanx, Paul

> ---
> 
>  kernel/rcutree.c |    9 ++++-----
>  1 file changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/kernel/rcutree.c b/kernel/rcutree.c
> index e441b77..ac94474 100644
> --- a/kernel/rcutree.c
> +++ b/kernel/rcutree.c
> @@ -2827,11 +2827,6 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
>  		break;
>  	case CPU_DYING:
>  	case CPU_DYING_FROZEN:
> -		/*
> -		 * The whole machine is "stopped" except this CPU, so we can
> -		 * touch any data without introducing corruption. We send the
> -		 * dying CPU's callbacks to an arbitrarily chosen online CPU.
> -		 */
>  		for_each_rcu_flavor(rsp)
>  			rcu_cleanup_dying_cpu(rsp);
>  		rcu_cleanup_after_idle(cpu);
> @@ -2840,6 +2835,10 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
>  	case CPU_DEAD_FROZEN:
>  	case CPU_UP_CANCELED:
>  	case CPU_UP_CANCELED_FROZEN:
> +		/*
> +		 * We send the dead CPU's callbacks to an arbitrarily chosen
> +		 * online CPU.
> +		 */
>  		for_each_rcu_flavor(rsp)
>  			rcu_cleanup_dead_cpu(cpu, rsp);
>  		break;
> 

Re: [PATCH v5 14/45] rcu, CPU hotplug: Fix comment referring to stop_machine()

[Srivatsa S. Bhat]

On 02/09/2013 05:44 AM, Paul E. McKenney wrote:
> On Tue, Jan 22, 2013 at 01:06:34PM +0530, Srivatsa S. Bhat wrote:
>> Don't refer to stop_machine() in the CPU hotplug path, since we are going
>> to get rid of it. Also, move the comment referring to callback adoption
>> to the CPU_DEAD case, because that's where it happens now.
>>
>> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
> 
> Ouch!  That comment is indeed obsolete and must die.
> 
> I queued this to -rcu with your Signed-off-by.  However, I omitted
> the added comment, as it is imcomplete -- it is easy to look at
> rcu_cleanup_dead_cpu() to see what it does.
>

Cool! I'll drop it from my patch series then. Thank you!

Regards,
Srivatsa S. Bhat

[PATCH v5 15/45] tick: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 kernel/time/tick-broadcast.c |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/kernel/time/tick-broadcast.c b/kernel/time/tick-broadcast.c
index f113755..d123a2c 100644
--- a/kernel/time/tick-broadcast.c
+++ b/kernel/time/tick-broadcast.c
@@ -160,6 +160,7 @@ static void tick_do_broadcast(struct cpumask *mask)
  */
 static void tick_do_periodic_broadcast(void)
 {
+	get_online_cpus_atomic();
 	raw_spin_lock(&tick_broadcast_lock);
 
 	cpumask_and(to_cpumask(tmpmask),
@@ -167,6 +168,7 @@ static void tick_do_periodic_broadcast(void)
 	tick_do_broadcast(to_cpumask(tmpmask));
 
 	raw_spin_unlock(&tick_broadcast_lock);
+	put_online_cpus_atomic();
 }
 
 /*

[PATCH v5 16/45] time/clocksource: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from going
offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: John Stultz <john.stultz@linaro.org>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 kernel/time/clocksource.c |    5 +++++
 1 file changed, 5 insertions(+)

diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c
index c958338..1c8d735 100644
--- a/kernel/time/clocksource.c
+++ b/kernel/time/clocksource.c
@@ -30,6 +30,7 @@
 #include <linux/sched.h> /* for spin_unlock_irq() using preempt_count() m68k */
 #include <linux/tick.h>
 #include <linux/kthread.h>
+#include <linux/cpu.h>
 
 void timecounter_init(struct timecounter *tc,
 		      const struct cyclecounter *cc,
@@ -320,11 +321,13 @@ static void clocksource_watchdog(unsigned long data)
 	 * Cycle through CPUs to check if the CPUs stay synchronized
 	 * to each other.
 	 */
+	get_online_cpus_atomic();
 	next_cpu = cpumask_next(raw_smp_processor_id(), cpu_online_mask);
 	if (next_cpu >= nr_cpu_ids)
 		next_cpu = cpumask_first(cpu_online_mask);
 	watchdog_timer.expires += WATCHDOG_INTERVAL;
 	add_timer_on(&watchdog_timer, next_cpu);
+	put_online_cpus_atomic();
 out:
 	spin_unlock(&watchdog_lock);
 }
@@ -336,7 +339,9 @@ static inline void clocksource_start_watchdog(void)
 	init_timer(&watchdog_timer);
 	watchdog_timer.function = clocksource_watchdog;
 	watchdog_timer.expires = jiffies + WATCHDOG_INTERVAL;
+	get_online_cpus_atomic();
 	add_timer_on(&watchdog_timer, cpumask_first(cpu_online_mask));
+	put_online_cpus_atomic();
 	watchdog_running = 1;
 }
 

[PATCH v5 17/45] softirq: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Frederic Weisbecker <fweisbec@gmail.com>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 kernel/softirq.c |    3 +++
 1 file changed, 3 insertions(+)

diff --git a/kernel/softirq.c b/kernel/softirq.c
index ed567ba..98c3e27 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -631,6 +631,7 @@ static void remote_softirq_receive(void *data)
 
 static int __try_remote_softirq(struct call_single_data *cp, int cpu, int softirq)
 {
+	get_online_cpus_atomic();
 	if (cpu_online(cpu)) {
 		cp->func = remote_softirq_receive;
 		cp->info = cp;
@@ -638,8 +639,10 @@ static int __try_remote_softirq(struct call_single_data *cp, int cpu, int softir
 		cp->priv = softirq;
 
 		__smp_call_function_single(cpu, cp, 0);
+		put_online_cpus_atomic();
 		return 0;
 	}
+	put_online_cpus_atomic();
 	return 1;
 }
 #else /* CONFIG_USE_GENERIC_SMP_HELPERS */

[PATCH v5 18/45] irq: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 kernel/irq/manage.c |    7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index e49a288..b4240b9 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -16,6 +16,7 @@
 #include <linux/interrupt.h>
 #include <linux/slab.h>
 #include <linux/sched.h>
+#include <linux/cpu.h>
 #include <linux/task_work.h>
 
 #include "internals.h"
@@ -202,7 +203,9 @@ int irq_set_affinity(unsigned int irq, const struct cpumask *mask)
 		return -EINVAL;
 
 	raw_spin_lock_irqsave(&desc->lock, flags);
+	get_online_cpus_atomic();
 	ret =  __irq_set_affinity_locked(irq_desc_get_irq_data(desc), mask);
+	put_online_cpus_atomic();
 	raw_spin_unlock_irqrestore(&desc->lock, flags);
 	return ret;
 }
@@ -343,7 +346,9 @@ int irq_select_affinity_usr(unsigned int irq, struct cpumask *mask)
 	int ret;
 
 	raw_spin_lock_irqsave(&desc->lock, flags);
+	get_online_cpus_atomic();
 	ret = setup_affinity(irq, desc, mask);
+	put_online_cpus_atomic();
 	raw_spin_unlock_irqrestore(&desc->lock, flags);
 	return ret;
 }
@@ -1126,7 +1131,9 @@ __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new)
 		}
 
 		/* Set default affinity mask once everything is setup */
+		get_online_cpus_atomic();
 		setup_affinity(irq, desc, mask);
+		put_online_cpus_atomic();
 
 	} else if (new->flags & IRQF_TRIGGER_MASK) {
 		unsigned int nmsk = new->flags & IRQF_TRIGGER_MASK;

[PATCH v5 19/45] net: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: "David S. Miller" <davem@davemloft.net>
Cc: Eric Dumazet <edumazet@google.com>
Cc: netdev@vger.kernel.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 net/core/dev.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/net/core/dev.c b/net/core/dev.c
index f64e439..5421f96 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -3089,7 +3089,7 @@ int netif_rx(struct sk_buff *skb)
 		struct rps_dev_flow voidflow, *rflow = &voidflow;
 		int cpu;
 
-		preempt_disable();
+		get_online_cpus_atomic();
 		rcu_read_lock();
 
 		cpu = get_rps_cpu(skb->dev, skb, &rflow);
@@ -3099,7 +3099,7 @@ int netif_rx(struct sk_buff *skb)
 		ret = enqueue_to_backlog(skb, cpu, &rflow->last_qtail);
 
 		rcu_read_unlock();
-		preempt_enable();
+		put_online_cpus_atomic();
 	} else
 #endif
 	{
@@ -3498,6 +3498,7 @@ int netif_receive_skb(struct sk_buff *skb)
 		struct rps_dev_flow voidflow, *rflow = &voidflow;
 		int cpu, ret;
 
+		get_online_cpus_atomic();
 		rcu_read_lock();
 
 		cpu = get_rps_cpu(skb->dev, skb, &rflow);
@@ -3505,9 +3506,11 @@ int netif_receive_skb(struct sk_buff *skb)
 		if (cpu >= 0) {
 			ret = enqueue_to_backlog(skb, cpu, &rflow->last_qtail);
 			rcu_read_unlock();
+			put_online_cpus_atomic();
 			return ret;
 		}
 		rcu_read_unlock();
+		put_online_cpus_atomic();
 	}
 #endif
 	return __netif_receive_skb(skb);
@@ -3887,6 +3890,7 @@ static void net_rps_action_and_irq_enable(struct softnet_data *sd)
 		local_irq_enable();
 
 		/* Send pending IPI's to kick RPS processing on remote cpus. */
+		get_online_cpus_atomic();
 		while (remsd) {
 			struct softnet_data *next = remsd->rps_ipi_next;
 
@@ -3895,6 +3899,7 @@ static void net_rps_action_and_irq_enable(struct softnet_data *sd)
 							   &remsd->csd, 0);
 			remsd = next;
 		}
+		put_online_cpus_atomic();
 	} else
 #endif
 		local_irq_enable();

[PATCH v5 20/45] block: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 block/blk-softirq.c |    4 ++++
 1 file changed, 4 insertions(+)

diff --git a/block/blk-softirq.c b/block/blk-softirq.c
index 467c8de..448f9a9 100644
--- a/block/blk-softirq.c
+++ b/block/blk-softirq.c
@@ -58,6 +58,8 @@ static void trigger_softirq(void *data)
  */
 static int raise_blk_irq(int cpu, struct request *rq)
 {
+	get_online_cpus_atomic();
+
 	if (cpu_online(cpu)) {
 		struct call_single_data *data = &rq->csd;
 
@@ -66,9 +68,11 @@ static int raise_blk_irq(int cpu, struct request *rq)
 		data->flags = 0;
 
 		__smp_call_function_single(cpu, data, 0);
+		put_online_cpus_atomic();
 		return 0;
 	}
 
+	put_online_cpus_atomic();
 	return 1;
 }
 #else /* CONFIG_SMP && CONFIG_USE_GENERIC_SMP_HELPERS */

[PATCH v5 21/45] crypto: pcrypt - Protect access to cpu_online_mask with get/put_online_cpus()

[Srivatsa S. Bhat]

The pcrypt_aead_init_tfm() function access the cpu_online_mask without
disabling CPU hotplug. And it looks like it can afford to sleep, so use
the get/put_online_cpus() APIs to protect against CPU hotplug.

Cc: Steffen Klassert <steffen.klassert@secunet.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: linux-crypto@vger.kernel.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 crypto/pcrypt.c |    4 ++++
 1 file changed, 4 insertions(+)

diff --git a/crypto/pcrypt.c b/crypto/pcrypt.c
index b2c99dc..10f64e2 100644
--- a/crypto/pcrypt.c
+++ b/crypto/pcrypt.c
@@ -280,12 +280,16 @@ static int pcrypt_aead_init_tfm(struct crypto_tfm *tfm)
 
 	ictx->tfm_count++;
 
+	get_online_cpus();
+
 	cpu_index = ictx->tfm_count % cpumask_weight(cpu_online_mask);
 
 	ctx->cb_cpu = cpumask_first(cpu_online_mask);
 	for (cpu = 0; cpu < cpu_index; cpu++)
 		ctx->cb_cpu = cpumask_next(ctx->cb_cpu, cpu_online_mask);
 
+	put_online_cpus();
+
 	cipher = crypto_spawn_aead(crypto_instance_ctx(inst));
 
 	if (IS_ERR(cipher))

[PATCH v5 22/45] infiniband: ehca: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Roland Dreier <roland@kernel.org>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 drivers/infiniband/hw/ehca/ehca_irq.c |    8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/infiniband/hw/ehca/ehca_irq.c b/drivers/infiniband/hw/ehca/ehca_irq.c
index 8615d7c..d61936c 100644
--- a/drivers/infiniband/hw/ehca/ehca_irq.c
+++ b/drivers/infiniband/hw/ehca/ehca_irq.c
@@ -43,6 +43,7 @@
 
 #include <linux/slab.h>
 #include <linux/smpboot.h>
+#include <linux/cpu.h>
 
 #include "ehca_classes.h"
 #include "ehca_irq.h"
@@ -653,6 +654,9 @@ void ehca_tasklet_eq(unsigned long data)
 	ehca_process_eq((struct ehca_shca*)data, 1);
 }
 
+/*
+ * Must be called under get_online_cpus_atomic() and put_online_cpus_atomic().
+ */
 static int find_next_online_cpu(struct ehca_comp_pool *pool)
 {
 	int cpu;
@@ -703,6 +707,7 @@ static void queue_comp_task(struct ehca_cq *__cq)
 	int cq_jobs;
 	unsigned long flags;
 
+	get_online_cpus_atomic();
 	cpu_id = find_next_online_cpu(pool);
 	BUG_ON(!cpu_online(cpu_id));
 
@@ -720,6 +725,7 @@ static void queue_comp_task(struct ehca_cq *__cq)
 		BUG_ON(!cct || !thread);
 	}
 	__queue_comp_task(__cq, cct, thread);
+	put_online_cpus_atomic();
 }
 
 static void run_comp_task(struct ehca_cpu_comp_task *cct)
@@ -759,6 +765,7 @@ static void comp_task_park(unsigned int cpu)
 	list_splice_init(&cct->cq_list, &list);
 	spin_unlock_irq(&cct->task_lock);
 
+	get_online_cpus_atomic();
 	cpu = find_next_online_cpu(pool);
 	target = per_cpu_ptr(pool->cpu_comp_tasks, cpu);
 	thread = *per_cpu_ptr(pool->cpu_comp_threads, cpu);
@@ -768,6 +775,7 @@ static void comp_task_park(unsigned int cpu)
 		__queue_comp_task(cq, target, thread);
 	}
 	spin_unlock_irq(&target->task_lock);
+	put_online_cpus_atomic();
 }
 
 static void comp_task_stop(unsigned int cpu, bool online)

[PATCH v5 23/45] [SCSI] fcoe: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Robert Love <robert.w.love@intel.com>
Cc: "James E.J. Bottomley" <JBottomley@parallels.com>
Cc: devel@open-fcoe.org
Cc: linux-scsi@vger.kernel.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 drivers/scsi/fcoe/fcoe.c |    7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/fcoe/fcoe.c b/drivers/scsi/fcoe/fcoe.c
index 666b7ac..c971a17 100644
--- a/drivers/scsi/fcoe/fcoe.c
+++ b/drivers/scsi/fcoe/fcoe.c
@@ -1475,6 +1475,7 @@ static int fcoe_rcv(struct sk_buff *skb, struct net_device *netdev,
 	 * was originated, otherwise select cpu using rx exchange id
 	 * or fcoe_select_cpu().
 	 */
+	get_online_cpus_atomic();
 	if (ntoh24(fh->fh_f_ctl) & FC_FC_EX_CTX)
 		cpu = ntohs(fh->fh_ox_id) & fc_cpu_mask;
 	else {
@@ -1484,8 +1485,10 @@ static int fcoe_rcv(struct sk_buff *skb, struct net_device *netdev,
 			cpu = ntohs(fh->fh_rx_id) & fc_cpu_mask;
 	}
 
-	if (cpu >= nr_cpu_ids)
+	if (cpu >= nr_cpu_ids) {
+		put_online_cpus_atomic();
 		goto err;
+	}
 
 	fps = &per_cpu(fcoe_percpu, cpu);
 	spin_lock(&fps->fcoe_rx_list.lock);
@@ -1505,6 +1508,7 @@ static int fcoe_rcv(struct sk_buff *skb, struct net_device *netdev,
 		spin_lock(&fps->fcoe_rx_list.lock);
 		if (!fps->thread) {
 			spin_unlock(&fps->fcoe_rx_list.lock);
+			put_online_cpus_atomic();
 			goto err;
 		}
 	}
@@ -1526,6 +1530,7 @@ static int fcoe_rcv(struct sk_buff *skb, struct net_device *netdev,
 	if (fps->thread->state == TASK_INTERRUPTIBLE)
 		wake_up_process(fps->thread);
 	spin_unlock(&fps->fcoe_rx_list.lock);
+	put_online_cpus_atomic();
 
 	return 0;
 err:

[PATCH v5 24/45] staging: octeon: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: David Daney <david.daney@cavium.com>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 drivers/staging/octeon/ethernet-rx.c |    3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/staging/octeon/ethernet-rx.c b/drivers/staging/octeon/ethernet-rx.c
index 34afc16..8588b4d 100644
--- a/drivers/staging/octeon/ethernet-rx.c
+++ b/drivers/staging/octeon/ethernet-rx.c
@@ -36,6 +36,7 @@
 #include <linux/prefetch.h>
 #include <linux/ratelimit.h>
 #include <linux/smp.h>
+#include <linux/cpu.h>
 #include <linux/interrupt.h>
 #include <net/dst.h>
 #ifdef CONFIG_XFRM
@@ -97,6 +98,7 @@ static void cvm_oct_enable_one_cpu(void)
 		return;
 
 	/* ... if a CPU is available, Turn on NAPI polling for that CPU.  */
+	get_online_cpus_atomic();
 	for_each_online_cpu(cpu) {
 		if (!cpu_test_and_set(cpu, core_state.cpu_state)) {
 			v = smp_call_function_single(cpu, cvm_oct_enable_napi,
@@ -106,6 +108,7 @@ static void cvm_oct_enable_one_cpu(void)
 			break;
 		}
 	}
+	put_online_cpus_atomic();
 }
 
 static void cvm_oct_no_more_work(void)

[PATCH v5 25/45] x86: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Cc: Tony Luck <tony.luck@intel.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Daniel J Blueman <daniel@numascale-asia.com>
Cc: Steffen Persvold <sp@numascale.com>
Cc: Joerg Roedel <joerg.roedel@amd.com>
Cc: linux-edac@vger.kernel.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/x86/include/asm/ipi.h               |    5 +++++
 arch/x86/kernel/apic/apic_flat_64.c      |   10 ++++++++++
 arch/x86/kernel/apic/apic_numachip.c     |    5 +++++
 arch/x86/kernel/apic/es7000_32.c         |    5 +++++
 arch/x86/kernel/apic/io_apic.c           |    7 +++++--
 arch/x86/kernel/apic/ipi.c               |   10 ++++++++++
 arch/x86/kernel/apic/x2apic_cluster.c    |    4 ++++
 arch/x86/kernel/apic/x2apic_uv_x.c       |    4 ++++
 arch/x86/kernel/cpu/mcheck/therm_throt.c |    4 ++--
 arch/x86/mm/tlb.c                        |   14 +++++++-------
 10 files changed, 57 insertions(+), 11 deletions(-)

diff --git a/arch/x86/include/asm/ipi.h b/arch/x86/include/asm/ipi.h
index 615fa90..112249c 100644
--- a/arch/x86/include/asm/ipi.h
+++ b/arch/x86/include/asm/ipi.h
@@ -20,6 +20,7 @@
  * Subject to the GNU Public License, v.2
  */
 
+#include <linux/cpu.h>
 #include <asm/hw_irq.h>
 #include <asm/apic.h>
 #include <asm/smp.h>
@@ -131,18 +132,22 @@ extern int no_broadcast;
 
 static inline void __default_local_send_IPI_allbutself(int vector)
 {
+	get_online_cpus_atomic();
 	if (no_broadcast || vector == NMI_VECTOR)
 		apic->send_IPI_mask_allbutself(cpu_online_mask, vector);
 	else
 		__default_send_IPI_shortcut(APIC_DEST_ALLBUT, vector, apic->dest_logical);
+	put_online_cpus_atomic();
 }
 
 static inline void __default_local_send_IPI_all(int vector)
 {
+	get_online_cpus_atomic();
 	if (no_broadcast || vector == NMI_VECTOR)
 		apic->send_IPI_mask(cpu_online_mask, vector);
 	else
 		__default_send_IPI_shortcut(APIC_DEST_ALLINC, vector, apic->dest_logical);
+	put_online_cpus_atomic();
 }
 
 #ifdef CONFIG_X86_32
diff --git a/arch/x86/kernel/apic/apic_flat_64.c b/arch/x86/kernel/apic/apic_flat_64.c
index 00c77cf..8207ade 100644
--- a/arch/x86/kernel/apic/apic_flat_64.c
+++ b/arch/x86/kernel/apic/apic_flat_64.c
@@ -11,6 +11,7 @@
 #include <linux/errno.h>
 #include <linux/threads.h>
 #include <linux/cpumask.h>
+#include <linux/cpu.h>
 #include <linux/string.h>
 #include <linux/kernel.h>
 #include <linux/ctype.h>
@@ -92,6 +93,8 @@ static void flat_send_IPI_allbutself(int vector)
 #else
 	int hotplug = 0;
 #endif
+
+	get_online_cpus_atomic();
 	if (hotplug || vector == NMI_VECTOR) {
 		if (!cpumask_equal(cpu_online_mask, cpumask_of(cpu))) {
 			unsigned long mask = cpumask_bits(cpu_online_mask)[0];
@@ -105,16 +108,19 @@ static void flat_send_IPI_allbutself(int vector)
 		__default_send_IPI_shortcut(APIC_DEST_ALLBUT,
 					    vector, apic->dest_logical);
 	}
+	put_online_cpus_atomic();
 }
 
 static void flat_send_IPI_all(int vector)
 {
+	get_online_cpus_atomic();
 	if (vector == NMI_VECTOR) {
 		flat_send_IPI_mask(cpu_online_mask, vector);
 	} else {
 		__default_send_IPI_shortcut(APIC_DEST_ALLINC,
 					    vector, apic->dest_logical);
 	}
+	put_online_cpus_atomic();
 }
 
 static unsigned int flat_get_apic_id(unsigned long x)
@@ -255,12 +261,16 @@ static void physflat_send_IPI_mask_allbutself(const struct cpumask *cpumask,
 
 static void physflat_send_IPI_allbutself(int vector)
 {
+	get_online_cpus_atomic();
 	default_send_IPI_mask_allbutself_phys(cpu_online_mask, vector);
+	put_online_cpus_atomic();
 }
 
 static void physflat_send_IPI_all(int vector)
 {
+	get_online_cpus_atomic();
 	physflat_send_IPI_mask(cpu_online_mask, vector);
+	put_online_cpus_atomic();
 }
 
 static int physflat_probe(void)
diff --git a/arch/x86/kernel/apic/apic_numachip.c b/arch/x86/kernel/apic/apic_numachip.c
index 9c2aa89..7d19c1d 100644
--- a/arch/x86/kernel/apic/apic_numachip.c
+++ b/arch/x86/kernel/apic/apic_numachip.c
@@ -14,6 +14,7 @@
 #include <linux/errno.h>
 #include <linux/threads.h>
 #include <linux/cpumask.h>
+#include <linux/cpu.h>
 #include <linux/string.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
@@ -131,15 +132,19 @@ static void numachip_send_IPI_allbutself(int vector)
 	unsigned int this_cpu = smp_processor_id();
 	unsigned int cpu;
 
+	get_online_cpus_atomic();
 	for_each_online_cpu(cpu) {
 		if (cpu != this_cpu)
 			numachip_send_IPI_one(cpu, vector);
 	}
+	put_online_cpus_atomic();
 }
 
 static void numachip_send_IPI_all(int vector)
 {
+	get_online_cpus_atomic();
 	numachip_send_IPI_mask(cpu_online_mask, vector);
+	put_online_cpus_atomic();
 }
 
 static void numachip_send_IPI_self(int vector)
diff --git a/arch/x86/kernel/apic/es7000_32.c b/arch/x86/kernel/apic/es7000_32.c
index 0874799..ddf2995 100644
--- a/arch/x86/kernel/apic/es7000_32.c
+++ b/arch/x86/kernel/apic/es7000_32.c
@@ -45,6 +45,7 @@
 #include <linux/gfp.h>
 #include <linux/nmi.h>
 #include <linux/smp.h>
+#include <linux/cpu.h>
 #include <linux/io.h>
 
 #include <asm/apicdef.h>
@@ -412,12 +413,16 @@ static void es7000_send_IPI_mask(const struct cpumask *mask, int vector)
 
 static void es7000_send_IPI_allbutself(int vector)
 {
+	get_online_cpus_atomic();
 	default_send_IPI_mask_allbutself_phys(cpu_online_mask, vector);
+	put_online_cpus_atomic();
 }
 
 static void es7000_send_IPI_all(int vector)
 {
+	get_online_cpus_atomic();
 	es7000_send_IPI_mask(cpu_online_mask, vector);
+	put_online_cpus_atomic();
 }
 
 static int es7000_apic_id_registered(void)
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index b739d39..ca1c2a5 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -25,6 +25,7 @@
 #include <linux/init.h>
 #include <linux/delay.h>
 #include <linux/sched.h>
+#include <linux/cpu.h>
 #include <linux/pci.h>
 #include <linux/mc146818rtc.h>
 #include <linux/compiler.h>
@@ -1788,13 +1789,13 @@ __apicdebuginit(void) print_local_APICs(int maxcpu)
 	if (!maxcpu)
 		return;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 	for_each_online_cpu(cpu) {
 		if (cpu >= maxcpu)
 			break;
 		smp_call_function_single(cpu, print_local_APIC, NULL, 1);
 	}
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 __apicdebuginit(void) print_PIC(void)
@@ -2209,6 +2210,7 @@ void send_cleanup_vector(struct irq_cfg *cfg)
 {
 	cpumask_var_t cleanup_mask;
 
+	get_online_cpus_atomic();
 	if (unlikely(!alloc_cpumask_var(&cleanup_mask, GFP_ATOMIC))) {
 		unsigned int i;
 		for_each_cpu_and(i, cfg->old_domain, cpu_online_mask)
@@ -2219,6 +2221,7 @@ void send_cleanup_vector(struct irq_cfg *cfg)
 		free_cpumask_var(cleanup_mask);
 	}
 	cfg->move_in_progress = 0;
+	put_online_cpus_atomic();
 }
 
 asmlinkage void smp_irq_move_cleanup_interrupt(void)
diff --git a/arch/x86/kernel/apic/ipi.c b/arch/x86/kernel/apic/ipi.c
index cce91bf..c65aa77 100644
--- a/arch/x86/kernel/apic/ipi.c
+++ b/arch/x86/kernel/apic/ipi.c
@@ -29,12 +29,14 @@ void default_send_IPI_mask_sequence_phys(const struct cpumask *mask, int vector)
 	 * to an arbitrary mask, so I do a unicast to each CPU instead.
 	 * - mbligh
 	 */
+	get_online_cpus_atomic();
 	local_irq_save(flags);
 	for_each_cpu(query_cpu, mask) {
 		__default_send_IPI_dest_field(per_cpu(x86_cpu_to_apicid,
 				query_cpu), vector, APIC_DEST_PHYSICAL);
 	}
 	local_irq_restore(flags);
+	put_online_cpus_atomic();
 }
 
 void default_send_IPI_mask_allbutself_phys(const struct cpumask *mask,
@@ -46,6 +48,7 @@ void default_send_IPI_mask_allbutself_phys(const struct cpumask *mask,
 
 	/* See Hack comment above */
 
+	get_online_cpus_atomic();
 	local_irq_save(flags);
 	for_each_cpu(query_cpu, mask) {
 		if (query_cpu == this_cpu)
@@ -54,6 +57,7 @@ void default_send_IPI_mask_allbutself_phys(const struct cpumask *mask,
 				 query_cpu), vector, APIC_DEST_PHYSICAL);
 	}
 	local_irq_restore(flags);
+	put_online_cpus_atomic();
 }
 
 #ifdef CONFIG_X86_32
@@ -70,12 +74,14 @@ void default_send_IPI_mask_sequence_logical(const struct cpumask *mask,
 	 * should be modified to do 1 message per cluster ID - mbligh
 	 */
 
+	get_online_cpus_atomic();
 	local_irq_save(flags);
 	for_each_cpu(query_cpu, mask)
 		__default_send_IPI_dest_field(
 			early_per_cpu(x86_cpu_to_logical_apicid, query_cpu),
 			vector, apic->dest_logical);
 	local_irq_restore(flags);
+	put_online_cpus_atomic();
 }
 
 void default_send_IPI_mask_allbutself_logical(const struct cpumask *mask,
@@ -87,6 +93,7 @@ void default_send_IPI_mask_allbutself_logical(const struct cpumask *mask,
 
 	/* See Hack comment above */
 
+	get_online_cpus_atomic();
 	local_irq_save(flags);
 	for_each_cpu(query_cpu, mask) {
 		if (query_cpu == this_cpu)
@@ -96,6 +103,7 @@ void default_send_IPI_mask_allbutself_logical(const struct cpumask *mask,
 			vector, apic->dest_logical);
 		}
 	local_irq_restore(flags);
+	put_online_cpus_atomic();
 }
 
 /*
@@ -109,10 +117,12 @@ void default_send_IPI_mask_logical(const struct cpumask *cpumask, int vector)
 	if (WARN_ONCE(!mask, "empty IPI mask"))
 		return;
 
+	get_online_cpus_atomic();
 	local_irq_save(flags);
 	WARN_ON(mask & ~cpumask_bits(cpu_online_mask)[0]);
 	__default_send_IPI_dest_field(mask, vector, apic->dest_logical);
 	local_irq_restore(flags);
+	put_online_cpus_atomic();
 }
 
 void default_send_IPI_allbutself(int vector)
diff --git a/arch/x86/kernel/apic/x2apic_cluster.c b/arch/x86/kernel/apic/x2apic_cluster.c
index c88baa4..cb08e6b 100644
--- a/arch/x86/kernel/apic/x2apic_cluster.c
+++ b/arch/x86/kernel/apic/x2apic_cluster.c
@@ -88,12 +88,16 @@ x2apic_send_IPI_mask_allbutself(const struct cpumask *mask, int vector)
 
 static void x2apic_send_IPI_allbutself(int vector)
 {
+	get_online_cpus_atomic();
 	__x2apic_send_IPI_mask(cpu_online_mask, vector, APIC_DEST_ALLBUT);
+	put_online_cpus_atomic();
 }
 
 static void x2apic_send_IPI_all(int vector)
 {
+	get_online_cpus_atomic();
 	__x2apic_send_IPI_mask(cpu_online_mask, vector, APIC_DEST_ALLINC);
+	put_online_cpus_atomic();
 }
 
 static int
diff --git a/arch/x86/kernel/apic/x2apic_uv_x.c b/arch/x86/kernel/apic/x2apic_uv_x.c
index 8cfade9..cc469a3 100644
--- a/arch/x86/kernel/apic/x2apic_uv_x.c
+++ b/arch/x86/kernel/apic/x2apic_uv_x.c
@@ -244,15 +244,19 @@ static void uv_send_IPI_allbutself(int vector)
 	unsigned int this_cpu = smp_processor_id();
 	unsigned int cpu;
 
+	get_online_cpus_atomic();
 	for_each_online_cpu(cpu) {
 		if (cpu != this_cpu)
 			uv_send_IPI_one(cpu, vector);
 	}
+	put_online_cpus_atomic();
 }
 
 static void uv_send_IPI_all(int vector)
 {
+	get_online_cpus_atomic();
 	uv_send_IPI_mask(cpu_online_mask, vector);
+	put_online_cpus_atomic();
 }
 
 static int uv_apic_id_valid(int apicid)
diff --git a/arch/x86/kernel/cpu/mcheck/therm_throt.c b/arch/x86/kernel/cpu/mcheck/therm_throt.c
index 47a1870..d128ba4 100644
--- a/arch/x86/kernel/cpu/mcheck/therm_throt.c
+++ b/arch/x86/kernel/cpu/mcheck/therm_throt.c
@@ -82,13 +82,13 @@ static ssize_t therm_throt_device_show_##event##_##name(		\
 	unsigned int cpu = dev->id;					\
 	ssize_t ret;							\
 									\
-	preempt_disable();	/* CPU hotplug */			\
+	get_online_cpus_atomic();	/* CPU hotplug */		\
 	if (cpu_online(cpu)) {						\
 		ret = sprintf(buf, "%lu\n",				\
 			      per_cpu(thermal_state, cpu).event.name);	\
 	} else								\
 		ret = 0;						\
-	preempt_enable();						\
+	put_online_cpus_atomic();					\
 									\
 	return ret;							\
 }
diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 13a6b29..2c3ec76 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -147,12 +147,12 @@ void flush_tlb_current_task(void)
 {
 	struct mm_struct *mm = current->mm;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 
 	local_flush_tlb();
 	if (cpumask_any_but(mm_cpumask(mm), smp_processor_id()) < nr_cpu_ids)
 		flush_tlb_others(mm_cpumask(mm), mm, 0UL, TLB_FLUSH_ALL);
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 /*
@@ -187,7 +187,7 @@ void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start,
 	unsigned long addr;
 	unsigned act_entries, tlb_entries = 0;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 	if (current->active_mm != mm)
 		goto flush_all;
 
@@ -225,21 +225,21 @@ void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start,
 		if (cpumask_any_but(mm_cpumask(mm),
 				smp_processor_id()) < nr_cpu_ids)
 			flush_tlb_others(mm_cpumask(mm), mm, start, end);
-		preempt_enable();
+		put_online_cpus_atomic();
 		return;
 	}
 
 flush_all:
 	if (cpumask_any_but(mm_cpumask(mm), smp_processor_id()) < nr_cpu_ids)
 		flush_tlb_others(mm_cpumask(mm), mm, 0UL, TLB_FLUSH_ALL);
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 void flush_tlb_page(struct vm_area_struct *vma, unsigned long start)
 {
 	struct mm_struct *mm = vma->vm_mm;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 
 	if (current->active_mm == mm) {
 		if (current->mm)
@@ -251,7 +251,7 @@ void flush_tlb_page(struct vm_area_struct *vma, unsigned long start)
 	if (cpumask_any_but(mm_cpumask(mm), smp_processor_id()) < nr_cpu_ids)
 		flush_tlb_others(mm_cpumask(mm), mm, start, 0UL);
 
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 static void do_flush_tlb_all(void *info)

[PATCH v5 26/45] perf/x86: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

The CPU_DYING notifier modifies the per-cpu pointer pmu->box, and this can
race with functions such as uncore_pmu_to_box() and uncore_pci_remove() when
we remove stop_machine() from the CPU offline path. So protect them using
get/put_online_cpus_atomic().

Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/x86/kernel/cpu/perf_event_intel_uncore.c |    5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/x86/kernel/cpu/perf_event_intel_uncore.c b/arch/x86/kernel/cpu/perf_event_intel_uncore.c
index b43200d..6faae53 100644
--- a/arch/x86/kernel/cpu/perf_event_intel_uncore.c
+++ b/arch/x86/kernel/cpu/perf_event_intel_uncore.c
@@ -1,3 +1,4 @@
+#include <linux/cpu.h>
 #include "perf_event_intel_uncore.h"
 
 static struct intel_uncore_type *empty_uncore[] = { NULL, };
@@ -1965,6 +1966,7 @@ uncore_pmu_to_box(struct intel_uncore_pmu *pmu, int cpu)
 	if (box)
 		return box;
 
+	get_online_cpus_atomic();
 	raw_spin_lock(&uncore_box_lock);
 	list_for_each_entry(box, &pmu->box_list, list) {
 		if (box->phys_id == topology_physical_package_id(cpu)) {
@@ -1974,6 +1976,7 @@ uncore_pmu_to_box(struct intel_uncore_pmu *pmu, int cpu)
 		}
 	}
 	raw_spin_unlock(&uncore_box_lock);
+	put_online_cpus_atomic();
 
 	return *per_cpu_ptr(pmu->box, cpu);
 }
@@ -2556,6 +2559,7 @@ static void uncore_pci_remove(struct pci_dev *pdev)
 	if (WARN_ON_ONCE(phys_id != box->phys_id))
 		return;
 
+	get_online_cpus_atomic();
 	raw_spin_lock(&uncore_box_lock);
 	list_del(&box->list);
 	raw_spin_unlock(&uncore_box_lock);
@@ -2569,6 +2573,7 @@ static void uncore_pci_remove(struct pci_dev *pdev)
 
 	WARN_ON_ONCE(atomic_read(&box->refcnt) != 1);
 	kfree(box);
+	put_online_cpus_atomic();
 }
 
 static int uncore_pci_probe(struct pci_dev *pdev,

[PATCH v5 27/45] KVM: Use get/put_online_cpus_atomic() to prevent CPU offline from atomic context

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Marcelo Tosatti <mtosatti@redhat.com>
Cc: Gleb Natapov <gleb@redhat.com>
Cc: kvm@vger.kernel.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 virt/kvm/kvm_main.c |   10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 1cd693a..47f9c30 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -174,7 +174,8 @@ static bool make_all_cpus_request(struct kvm *kvm, unsigned int req)
 
 	zalloc_cpumask_var(&cpus, GFP_ATOMIC);
 
-	me = get_cpu();
+	get_online_cpus_atomic();
+	me = smp_processor_id();
 	kvm_for_each_vcpu(i, vcpu, kvm) {
 		kvm_make_request(req, vcpu);
 		cpu = vcpu->cpu;
@@ -192,7 +193,7 @@ static bool make_all_cpus_request(struct kvm *kvm, unsigned int req)
 		smp_call_function_many(cpus, ack_flush, NULL, 1);
 	else
 		called = false;
-	put_cpu();
+	put_online_cpus_atomic();
 	free_cpumask_var(cpus);
 	return called;
 }
@@ -1621,11 +1622,12 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
 		++vcpu->stat.halt_wakeup;
 	}
 
-	me = get_cpu();
+	get_online_cpus_atomic();
+	me = smp_processor_id();
 	if (cpu != me && (unsigned)cpu < nr_cpu_ids && cpu_online(cpu))
 		if (kvm_arch_vcpu_should_kick(vcpu))
 			smp_send_reschedule(cpu);
-	put_cpu();
+	put_online_cpus_atomic();
 }
 #endif /* !CONFIG_S390 */
 

[PATCH v5 28/45] kvm/vmx: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context (in vmx_vcpu_load() to prevent CPUs from
going offline while clearing vmcs).

Reported-by: Michael Wang <wangyun@linux.vnet.ibm.com>
Debugged-by: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com>
Cc: Marcelo Tosatti <mtosatti@redhat.com>
Cc: Gleb Natapov <gleb@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Cc: kvm@vger.kernel.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/x86/kvm/vmx.c |    8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 9120ae1..2886ff0 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -1557,10 +1557,14 @@ static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
 	u64 phys_addr = __pa(per_cpu(vmxarea, cpu));
 
-	if (!vmm_exclusive)
+	if (!vmm_exclusive) {
 		kvm_cpu_vmxon(phys_addr);
-	else if (vmx->loaded_vmcs->cpu != cpu)
+	} else if (vmx->loaded_vmcs->cpu != cpu) {
+		/* Prevent any CPU from going offline */
+		get_online_cpus_atomic();
 		loaded_vmcs_clear(vmx->loaded_vmcs);
+		put_online_cpus_atomic();
+	}
 
 	if (per_cpu(current_vmcs, cpu) != vmx->loaded_vmcs->vmcs) {
 		per_cpu(current_vmcs, cpu) = vmx->loaded_vmcs->vmcs;

[PATCH v5 29/45] x86/xen: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Cc: xen-devel@lists.xensource.com
Cc: virtualization@lists.linux-foundation.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/x86/xen/mmu.c |   11 +++++++++--
 arch/x86/xen/smp.c |    9 +++++++++
 2 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
index 01de35c..6a95a15 100644
--- a/arch/x86/xen/mmu.c
+++ b/arch/x86/xen/mmu.c
@@ -39,6 +39,7 @@
  * Jeremy Fitzhardinge <jeremy@xensource.com>, XenSource Inc, 2007
  */
 #include <linux/sched.h>
+#include <linux/cpu.h>
 #include <linux/highmem.h>
 #include <linux/debugfs.h>
 #include <linux/bug.h>
@@ -1163,9 +1164,13 @@ static void xen_drop_mm_ref(struct mm_struct *mm)
  */
 static void xen_exit_mmap(struct mm_struct *mm)
 {
-	get_cpu();		/* make sure we don't move around */
+	/*
+	 * Make sure we don't move around, and prevent CPUs from going
+	 * offline.
+	 */
+	get_online_cpus_atomic();
 	xen_drop_mm_ref(mm);
-	put_cpu();
+	put_online_cpus_atomic();
 
 	spin_lock(&mm->page_table_lock);
 
@@ -1371,6 +1376,7 @@ static void xen_flush_tlb_others(const struct cpumask *cpus,
 	args->op.arg2.vcpumask = to_cpumask(args->mask);
 
 	/* Remove us, and any offline CPUS. */
+	get_online_cpus_atomic();
 	cpumask_and(to_cpumask(args->mask), cpus, cpu_online_mask);
 	cpumask_clear_cpu(smp_processor_id(), to_cpumask(args->mask));
 
@@ -1383,6 +1389,7 @@ static void xen_flush_tlb_others(const struct cpumask *cpus,
 	MULTI_mmuext_op(mcs.mc, &args->op, 1, NULL, DOMID_SELF);
 
 	xen_mc_issue(PARAVIRT_LAZY_MMU);
+	put_online_cpus_atomic();
 }
 
 static unsigned long xen_read_cr3(void)
diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index 4f7d259..7d753ae 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -16,6 +16,7 @@
 #include <linux/err.h>
 #include <linux/slab.h>
 #include <linux/smp.h>
+#include <linux/cpu.h>
 #include <linux/irq_work.h>
 
 #include <asm/paravirt.h>
@@ -487,8 +488,10 @@ static void __xen_send_IPI_mask(const struct cpumask *mask,
 {
 	unsigned cpu;
 
+	get_online_cpus_atomic();
 	for_each_cpu_and(cpu, mask, cpu_online_mask)
 		xen_send_IPI_one(cpu, vector);
+	put_online_cpus_atomic();
 }
 
 static void xen_smp_send_call_function_ipi(const struct cpumask *mask)
@@ -551,8 +554,10 @@ void xen_send_IPI_all(int vector)
 {
 	int xen_vector = xen_map_vector(vector);
 
+	get_online_cpus_atomic();
 	if (xen_vector >= 0)
 		__xen_send_IPI_mask(cpu_online_mask, xen_vector);
+	put_online_cpus_atomic();
 }
 
 void xen_send_IPI_self(int vector)
@@ -572,20 +577,24 @@ void xen_send_IPI_mask_allbutself(const struct cpumask *mask,
 	if (!(num_online_cpus() > 1))
 		return;
 
+	get_online_cpus_atomic();
 	for_each_cpu_and(cpu, mask, cpu_online_mask) {
 		if (this_cpu == cpu)
 			continue;
 
 		xen_smp_send_call_function_single_ipi(cpu);
 	}
+	put_online_cpus_atomic();
 }
 
 void xen_send_IPI_allbutself(int vector)
 {
 	int xen_vector = xen_map_vector(vector);
 
+	get_online_cpus_atomic();
 	if (xen_vector >= 0)
 		xen_send_IPI_mask_allbutself(cpu_online_mask, xen_vector);
+	put_online_cpus_atomic();
 }
 
 static irqreturn_t xen_call_function_interrupt(int irq, void *dev_id)

Re: [PATCH v5 29/45] x86/xen: Use get/put_online_cpus_atomic() to prevent CPU offline

[Konrad Rzeszutek Wilk]

On Tue, Jan 22, 2013 at 01:10:51PM +0530, Srivatsa S. Bhat wrote:
> Once stop_machine() is gone from the CPU offline path, we won't be able to
> depend on preempt_disable() or local_irq_disable() to prevent CPUs from
> going offline from under us.
> 
> Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
> while invoking from atomic context.
> 
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

Weird. I see this in the patch but I don't see it in the header? Did you
explicitly suppress the CC part?


Anyhow, the patch looks sane enough, thought I need to to run it through
a test framework just to be on a sure side.

> Cc: Jeremy Fitzhardinge <jeremy@goop.org>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: x86@kernel.org
> Cc: xen-devel@lists.xensource.com
> Cc: virtualization@lists.linux-foundation.org
> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
> ---
> 
>  arch/x86/xen/mmu.c |   11 +++++++++--
>  arch/x86/xen/smp.c |    9 +++++++++
>  2 files changed, 18 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
> index 01de35c..6a95a15 100644
> --- a/arch/x86/xen/mmu.c
> +++ b/arch/x86/xen/mmu.c
> @@ -39,6 +39,7 @@
>   * Jeremy Fitzhardinge <jeremy@xensource.com>, XenSource Inc, 2007
>   */
>  #include <linux/sched.h>
> +#include <linux/cpu.h>
>  #include <linux/highmem.h>
>  #include <linux/debugfs.h>
>  #include <linux/bug.h>
> @@ -1163,9 +1164,13 @@ static void xen_drop_mm_ref(struct mm_struct *mm)
>   */
>  static void xen_exit_mmap(struct mm_struct *mm)
>  {
> -	get_cpu();		/* make sure we don't move around */
> +	/*
> +	 * Make sure we don't move around, and prevent CPUs from going
> +	 * offline.
> +	 */
> +	get_online_cpus_atomic();
>  	xen_drop_mm_ref(mm);
> -	put_cpu();
> +	put_online_cpus_atomic();
>  
>  	spin_lock(&mm->page_table_lock);
>  
> @@ -1371,6 +1376,7 @@ static void xen_flush_tlb_others(const struct cpumask *cpus,
>  	args->op.arg2.vcpumask = to_cpumask(args->mask);
>  
>  	/* Remove us, and any offline CPUS. */
> +	get_online_cpus_atomic();
>  	cpumask_and(to_cpumask(args->mask), cpus, cpu_online_mask);
>  	cpumask_clear_cpu(smp_processor_id(), to_cpumask(args->mask));
>  
> @@ -1383,6 +1389,7 @@ static void xen_flush_tlb_others(const struct cpumask *cpus,
>  	MULTI_mmuext_op(mcs.mc, &args->op, 1, NULL, DOMID_SELF);
>  
>  	xen_mc_issue(PARAVIRT_LAZY_MMU);
> +	put_online_cpus_atomic();
>  }
>  
>  static unsigned long xen_read_cr3(void)
> diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
> index 4f7d259..7d753ae 100644
> --- a/arch/x86/xen/smp.c
> +++ b/arch/x86/xen/smp.c
> @@ -16,6 +16,7 @@
>  #include <linux/err.h>
>  #include <linux/slab.h>
>  #include <linux/smp.h>
> +#include <linux/cpu.h>
>  #include <linux/irq_work.h>
>  
>  #include <asm/paravirt.h>
> @@ -487,8 +488,10 @@ static void __xen_send_IPI_mask(const struct cpumask *mask,
>  {
>  	unsigned cpu;
>  
> +	get_online_cpus_atomic();
>  	for_each_cpu_and(cpu, mask, cpu_online_mask)
>  		xen_send_IPI_one(cpu, vector);
> +	put_online_cpus_atomic();
>  }
>  
>  static void xen_smp_send_call_function_ipi(const struct cpumask *mask)
> @@ -551,8 +554,10 @@ void xen_send_IPI_all(int vector)
>  {
>  	int xen_vector = xen_map_vector(vector);
>  
> +	get_online_cpus_atomic();
>  	if (xen_vector >= 0)
>  		__xen_send_IPI_mask(cpu_online_mask, xen_vector);
> +	put_online_cpus_atomic();
>  }
>  
>  void xen_send_IPI_self(int vector)
> @@ -572,20 +577,24 @@ void xen_send_IPI_mask_allbutself(const struct cpumask *mask,
>  	if (!(num_online_cpus() > 1))
>  		return;
>  
> +	get_online_cpus_atomic();
>  	for_each_cpu_and(cpu, mask, cpu_online_mask) {
>  		if (this_cpu == cpu)
>  			continue;
>  
>  		xen_smp_send_call_function_single_ipi(cpu);
>  	}
> +	put_online_cpus_atomic();
>  }
>  
>  void xen_send_IPI_allbutself(int vector)
>  {
>  	int xen_vector = xen_map_vector(vector);
>  
> +	get_online_cpus_atomic();
>  	if (xen_vector >= 0)
>  		xen_send_IPI_mask_allbutself(cpu_online_mask, xen_vector);
> +	put_online_cpus_atomic();
>  }
>  
>  static irqreturn_t xen_call_function_interrupt(int irq, void *dev_id)
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [PATCH v5 29/45] x86/xen: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

On 02/19/2013 11:40 PM, Konrad Rzeszutek Wilk wrote:
> On Tue, Jan 22, 2013 at 01:10:51PM +0530, Srivatsa S. Bhat wrote:
>> Once stop_machine() is gone from the CPU offline path, we won't be able to
>> depend on preempt_disable() or local_irq_disable() to prevent CPUs from
>> going offline from under us.
>>
>> Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
>> while invoking from atomic context.
>>
>> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> 
> Weird. I see this in the patch but I don't see it in the header?

Meaning, you didn't get this email at all?

> Did you
> explicitly suppress the CC part?
> 

No.. I sent the entire patchset to a set of email ids and in addition to
that I CC'ed individual patches to the respective maintainers/lists (the
CC: list in the changelog). I used the --auto knob from stgit to do that.

> 
> Anyhow, the patch looks sane enough, thought I need to to run it through
> a test framework just to be on a sure side.
>

Sure, thank you. But you might want to test the v6 that I sent out
yesterday instead of v5. Oh, wait a min, you didn't get the v6 mail also?

Here it is, for your reference:
http://marc.info/?l=linux-kernel&m=136119260122255&w=2

Regards,
Srivatsa S. Bhat

>> Cc: Jeremy Fitzhardinge <jeremy@goop.org>
>> Cc: "H. Peter Anvin" <hpa@zytor.com>
>> Cc: x86@kernel.org
>> Cc: xen-devel@lists.xensource.com
>> Cc: virtualization@lists.linux-foundation.org
>> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
>> ---

[PATCH v5 30/45] alpha/smp: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Also, remove the non-ASCII character present in this file!

Cc: linux-alpha@vger.kernel.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/alpha/kernel/smp.c |   19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/arch/alpha/kernel/smp.c b/arch/alpha/kernel/smp.c
index 9603bc2..9213d5d 100644
--- a/arch/alpha/kernel/smp.c
+++ b/arch/alpha/kernel/smp.c
@@ -498,7 +498,6 @@ smp_cpus_done(unsigned int max_cpus)
 	       ((bogosum + 2500) / (5000/HZ)) % 100);
 }
 
-\f
 void
 smp_percpu_timer_interrupt(struct pt_regs *regs)
 {
@@ -682,7 +681,7 @@ ipi_flush_tlb_mm(void *x)
 void
 flush_tlb_mm(struct mm_struct *mm)
 {
-	preempt_disable();
+	get_online_cpus_atomic();
 
 	if (mm == current->active_mm) {
 		flush_tlb_current(mm);
@@ -694,7 +693,7 @@ flush_tlb_mm(struct mm_struct *mm)
 				if (mm->context[cpu])
 					mm->context[cpu] = 0;
 			}
-			preempt_enable();
+			put_online_cpus_atomic();
 			return;
 		}
 	}
@@ -703,7 +702,7 @@ flush_tlb_mm(struct mm_struct *mm)
 		printk(KERN_CRIT "flush_tlb_mm: timed out\n");
 	}
 
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 EXPORT_SYMBOL(flush_tlb_mm);
 
@@ -731,7 +730,7 @@ flush_tlb_page(struct vm_area_struct *vma, unsigned long addr)
 	struct flush_tlb_page_struct data;
 	struct mm_struct *mm = vma->vm_mm;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 
 	if (mm == current->active_mm) {
 		flush_tlb_current_page(mm, vma, addr);
@@ -743,7 +742,7 @@ flush_tlb_page(struct vm_area_struct *vma, unsigned long addr)
 				if (mm->context[cpu])
 					mm->context[cpu] = 0;
 			}
-			preempt_enable();
+			put_online_cpus_atomic();
 			return;
 		}
 	}
@@ -756,7 +755,7 @@ flush_tlb_page(struct vm_area_struct *vma, unsigned long addr)
 		printk(KERN_CRIT "flush_tlb_page: timed out\n");
 	}
 
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 EXPORT_SYMBOL(flush_tlb_page);
 
@@ -787,7 +786,7 @@ flush_icache_user_range(struct vm_area_struct *vma, struct page *page,
 	if ((vma->vm_flags & VM_EXEC) == 0)
 		return;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 
 	if (mm == current->active_mm) {
 		__load_new_mm_context(mm);
@@ -799,7 +798,7 @@ flush_icache_user_range(struct vm_area_struct *vma, struct page *page,
 				if (mm->context[cpu])
 					mm->context[cpu] = 0;
 			}
-			preempt_enable();
+			put_online_cpus_atomic();
 			return;
 		}
 	}
@@ -808,5 +807,5 @@ flush_icache_user_range(struct vm_area_struct *vma, struct page *page,
 		printk(KERN_CRIT "flush_icache_page: timed out\n");
 	}
 
-	preempt_enable();
+	put_online_cpus_atomic();
 }

[PATCH v5 31/45] blackfin/smp: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Mike Frysinger <vapier@gentoo.org>
Cc: Bob Liu <lliubbo@gmail.com>
Cc: Steven Miao <realmz6@gmail.com>
Cc: uclinux-dist-devel@blackfin.uclinux.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/blackfin/mach-common/smp.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/arch/blackfin/mach-common/smp.c b/arch/blackfin/mach-common/smp.c
index bb61ae4..6cc6d7a 100644
--- a/arch/blackfin/mach-common/smp.c
+++ b/arch/blackfin/mach-common/smp.c
@@ -194,6 +194,7 @@ void send_ipi(const struct cpumask *cpumask, enum ipi_message_type msg)
 	struct ipi_data *bfin_ipi_data;
 	unsigned long flags;
 
+	get_online_cpus_atomic();
 	local_irq_save(flags);
 	smp_mb();
 	for_each_cpu(cpu, cpumask) {
@@ -205,6 +206,7 @@ void send_ipi(const struct cpumask *cpumask, enum ipi_message_type msg)
 	}
 
 	local_irq_restore(flags);
+	put_online_cpus_atomic();
 }
 
 void arch_send_call_function_single_ipi(int cpu)
@@ -238,13 +240,13 @@ void smp_send_stop(void)
 {
 	cpumask_t callmap;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 	cpumask_copy(&callmap, cpu_online_mask);
 	cpumask_clear_cpu(smp_processor_id(), &callmap);
 	if (!cpumask_empty(&callmap))
 		send_ipi(&callmap, BFIN_IPI_CPU_STOP);
 
-	preempt_enable();
+	put_online_cpus_atomic();
 
 	return;
 }

Re: [PATCH v5 31/45] blackfin/smp: Use get/put_online_cpus_atomic() to prevent CPU offline

[Bob Liu]

On Tue, Jan 22, 2013 at 3:41 PM, Srivatsa S. Bhat
<srivatsa.bhat@linux.vnet.ibm.com> wrote:
> Once stop_machine() is gone from the CPU offline path, we won't be able to
> depend on preempt_disable() or local_irq_disable() to prevent CPUs from
> going offline from under us.
>
> Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
> while invoking from atomic context.
>
> Cc: Mike Frysinger <vapier@gentoo.org>
> Cc: Bob Liu <lliubbo@gmail.com>
> Cc: Steven Miao <realmz6@gmail.com>
> Cc: uclinux-dist-devel@blackfin.uclinux.org
> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>

Thanks, will be applied to my blackfin arch tree.

> ---
>
>  arch/blackfin/mach-common/smp.c |    6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/arch/blackfin/mach-common/smp.c b/arch/blackfin/mach-common/smp.c
> index bb61ae4..6cc6d7a 100644
> --- a/arch/blackfin/mach-common/smp.c
> +++ b/arch/blackfin/mach-common/smp.c
> @@ -194,6 +194,7 @@ void send_ipi(const struct cpumask *cpumask, enum ipi_message_type msg)
>         struct ipi_data *bfin_ipi_data;
>         unsigned long flags;
>
> +       get_online_cpus_atomic();
>         local_irq_save(flags);
>         smp_mb();
>         for_each_cpu(cpu, cpumask) {
> @@ -205,6 +206,7 @@ void send_ipi(const struct cpumask *cpumask, enum ipi_message_type msg)
>         }
>
>         local_irq_restore(flags);
> +       put_online_cpus_atomic();
>  }
>
>  void arch_send_call_function_single_ipi(int cpu)
> @@ -238,13 +240,13 @@ void smp_send_stop(void)
>  {
>         cpumask_t callmap;
>
> -       preempt_disable();
> +       get_online_cpus_atomic();
>         cpumask_copy(&callmap, cpu_online_mask);
>         cpumask_clear_cpu(smp_processor_id(), &callmap);
>         if (!cpumask_empty(&callmap))
>                 send_ipi(&callmap, BFIN_IPI_CPU_STOP);
>
> -       preempt_enable();
> +       put_online_cpus_atomic();
>
>         return;
>  }
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
Regards,
--Bob

Re: [PATCH v5 31/45] blackfin/smp: Use get/put_online_cpus_atomic() to prevent CPU offline

[Tejun Heo]

Hello, Bob.

On Mon, Jan 28, 2013 at 1:09 AM, Bob Liu <lliubbo@gmail.com> wrote:
> Thanks, will be applied to my blackfin arch tree.

I think we still have some work ahead of us to have this patchset
ready for inclusion and even then it probably would be best to route
these patches together, so probably not a very good idea to apply this
to blackfin right now.

Thanks.

-- 
tejun

Re: [PATCH v5 31/45] blackfin/smp: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

On 01/29/2013 06:06 AM, Tejun Heo wrote:
> Hello, Bob.
> 
> On Mon, Jan 28, 2013 at 1:09 AM, Bob Liu <lliubbo@gmail.com> wrote:
>> Thanks, will be applied to my blackfin arch tree.
> 
> I think we still have some work ahead of us to have this patchset
> ready for inclusion and even then it probably would be best to route
> these patches together, so probably not a very good idea to apply this
> to blackfin right now.
> 

Thanks Tejun for pointing that out! I'll address the review comments
soon and respin the patchset.
 
Regards,
Srivatsa S. Bhat

[PATCH v5 32/45] cris/smp: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Mikael Starvik <starvik@axis.com>
Cc: Jesper Nilsson <jesper.nilsson@axis.com>
Cc: linux-cris-kernel@axis.com
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/cris/arch-v32/kernel/smp.c |    8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/arch/cris/arch-v32/kernel/smp.c b/arch/cris/arch-v32/kernel/smp.c
index 04a16ed..644b358 100644
--- a/arch/cris/arch-v32/kernel/smp.c
+++ b/arch/cris/arch-v32/kernel/smp.c
@@ -15,6 +15,7 @@
 #include <linux/sched.h>
 #include <linux/kernel.h>
 #include <linux/cpumask.h>
+#include <linux/cpu.h>
 #include <linux/interrupt.h>
 #include <linux/module.h>
 
@@ -208,9 +209,12 @@ int __cpuinit __cpu_up(unsigned int cpu, struct task_struct *tidle)
 void smp_send_reschedule(int cpu)
 {
 	cpumask_t cpu_mask;
+
+	get_online_cpus_atomic();
 	cpumask_clear(&cpu_mask);
 	cpumask_set_cpu(cpu, &cpu_mask);
 	send_ipi(IPI_SCHEDULE, 0, cpu_mask);
+	put_online_cpus_atomic();
 }
 
 /* TLB flushing
@@ -224,6 +228,7 @@ void flush_tlb_common(struct mm_struct* mm, struct vm_area_struct* vma, unsigned
 	unsigned long flags;
 	cpumask_t cpu_mask;
 
+	get_online_cpus_atomic();
 	spin_lock_irqsave(&tlbstate_lock, flags);
 	cpu_mask = (mm == FLUSH_ALL ? cpu_all_mask : *mm_cpumask(mm));
 	cpumask_clear_cpu(smp_processor_id(), &cpu_mask);
@@ -232,6 +237,7 @@ void flush_tlb_common(struct mm_struct* mm, struct vm_area_struct* vma, unsigned
 	flush_addr = addr;
 	send_ipi(IPI_FLUSH_TLB, 1, cpu_mask);
 	spin_unlock_irqrestore(&tlbstate_lock, flags);
+	put_online_cpus_atomic();
 }
 
 void flush_tlb_all(void)
@@ -312,6 +318,7 @@ int smp_call_function(void (*func)(void *info), void *info, int wait)
 	struct call_data_struct data;
 	int ret;
 
+	get_online_cpus_atomic();
 	cpumask_setall(&cpu_mask);
 	cpumask_clear_cpu(smp_processor_id(), &cpu_mask);
 
@@ -325,6 +332,7 @@ int smp_call_function(void (*func)(void *info), void *info, int wait)
 	call_data = &data;
 	ret = send_ipi(IPI_CALL, wait, cpu_mask);
 	spin_unlock(&call_lock);
+	put_online_cpus_atomic();
 
 	return ret;
 }

[PATCH v5 33/45] hexagon/smp: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Richard Kuo <rkuo@codeaurora.org>
Cc: linux-hexagon@vger.kernel.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/hexagon/kernel/smp.c |    5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/hexagon/kernel/smp.c b/arch/hexagon/kernel/smp.c
index 8e095df..ec87de9 100644
--- a/arch/hexagon/kernel/smp.c
+++ b/arch/hexagon/kernel/smp.c
@@ -112,6 +112,7 @@ void send_ipi(const struct cpumask *cpumask, enum ipi_message_type msg)
 	unsigned long cpu;
 	unsigned long retval;
 
+	get_online_cpus_atomic();
 	local_irq_save(flags);
 
 	for_each_cpu(cpu, cpumask) {
@@ -128,6 +129,7 @@ void send_ipi(const struct cpumask *cpumask, enum ipi_message_type msg)
 	}
 
 	local_irq_restore(flags);
+	put_online_cpus_atomic();
 }
 
 static struct irqaction ipi_intdesc = {
@@ -241,9 +243,12 @@ void smp_send_reschedule(int cpu)
 void smp_send_stop(void)
 {
 	struct cpumask targets;
+
+	get_online_cpus_atomic();
 	cpumask_copy(&targets, cpu_online_mask);
 	cpumask_clear_cpu(smp_processor_id(), &targets);
 	send_ipi(&targets, IPI_CPU_STOP);
+	put_online_cpus_atomic();
 }
 
 void arch_send_call_function_single_ipi(int cpu)

[PATCH v5 34/45] ia64: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Tony Luck <tony.luck@intel.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: linux-ia64@vger.kernel.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/ia64/kernel/irq_ia64.c |   13 +++++++++++++
 arch/ia64/kernel/perfmon.c  |    6 ++++++
 arch/ia64/kernel/smp.c      |   23 ++++++++++++++++-------
 arch/ia64/mm/tlb.c          |    6 ++++--
 4 files changed, 39 insertions(+), 9 deletions(-)

diff --git a/arch/ia64/kernel/irq_ia64.c b/arch/ia64/kernel/irq_ia64.c
index 1034884..d0b4478 100644
--- a/arch/ia64/kernel/irq_ia64.c
+++ b/arch/ia64/kernel/irq_ia64.c
@@ -31,6 +31,7 @@
 #include <linux/ratelimit.h>
 #include <linux/acpi.h>
 #include <linux/sched.h>
+#include <linux/cpu.h>
 
 #include <asm/delay.h>
 #include <asm/intrinsics.h>
@@ -190,9 +191,11 @@ static void clear_irq_vector(int irq)
 {
 	unsigned long flags;
 
+	get_online_cpus_atomic();
 	spin_lock_irqsave(&vector_lock, flags);
 	__clear_irq_vector(irq);
 	spin_unlock_irqrestore(&vector_lock, flags);
+	put_online_cpus_atomic();
 }
 
 int
@@ -204,6 +207,7 @@ ia64_native_assign_irq_vector (int irq)
 
 	vector = -ENOSPC;
 
+	get_online_cpus_atomic();
 	spin_lock_irqsave(&vector_lock, flags);
 	for_each_online_cpu(cpu) {
 		domain = vector_allocation_domain(cpu);
@@ -218,6 +222,7 @@ ia64_native_assign_irq_vector (int irq)
 	BUG_ON(__bind_irq_vector(irq, vector, domain));
  out:
 	spin_unlock_irqrestore(&vector_lock, flags);
+	put_online_cpus_atomic();
 	return vector;
 }
 
@@ -302,9 +307,11 @@ int irq_prepare_move(int irq, int cpu)
 	unsigned long flags;
 	int ret;
 
+	get_online_cpus_atomic();
 	spin_lock_irqsave(&vector_lock, flags);
 	ret = __irq_prepare_move(irq, cpu);
 	spin_unlock_irqrestore(&vector_lock, flags);
+	put_online_cpus_atomic();
 	return ret;
 }
 
@@ -320,11 +327,13 @@ void irq_complete_move(unsigned irq)
 	if (unlikely(cpu_isset(smp_processor_id(), cfg->old_domain)))
 		return;
 
+	get_online_cpus_atomic();
 	cpumask_and(&cleanup_mask, &cfg->old_domain, cpu_online_mask);
 	cfg->move_cleanup_count = cpus_weight(cleanup_mask);
 	for_each_cpu_mask(i, cleanup_mask)
 		platform_send_ipi(i, IA64_IRQ_MOVE_VECTOR, IA64_IPI_DM_INT, 0);
 	cfg->move_in_progress = 0;
+	put_online_cpus_atomic();
 }
 
 static irqreturn_t smp_irq_move_cleanup_interrupt(int irq, void *dev_id)
@@ -409,6 +418,8 @@ int create_irq(void)
 	cpumask_t domain = CPU_MASK_NONE;
 
 	irq = vector = -ENOSPC;
+
+	get_online_cpus_atomic();
 	spin_lock_irqsave(&vector_lock, flags);
 	for_each_online_cpu(cpu) {
 		domain = vector_allocation_domain(cpu);
@@ -424,6 +435,8 @@ int create_irq(void)
 	BUG_ON(__bind_irq_vector(irq, vector, domain));
  out:
 	spin_unlock_irqrestore(&vector_lock, flags);
+	put_online_cpus_atomic();
+
 	if (irq >= 0)
 		dynamic_irq_init(irq);
 	return irq;
diff --git a/arch/ia64/kernel/perfmon.c b/arch/ia64/kernel/perfmon.c
index ea39eba..6c6a029 100644
--- a/arch/ia64/kernel/perfmon.c
+++ b/arch/ia64/kernel/perfmon.c
@@ -34,6 +34,7 @@
 #include <linux/poll.h>
 #include <linux/vfs.h>
 #include <linux/smp.h>
+#include <linux/cpu.h>
 #include <linux/pagemap.h>
 #include <linux/mount.h>
 #include <linux/bitops.h>
@@ -6485,6 +6486,7 @@ pfm_install_alt_pmu_interrupt(pfm_intr_handler_desc_t *hdl)
 	}
 
 	/* reserve our session */
+	get_online_cpus_atomic();
 	for_each_online_cpu(reserve_cpu) {
 		ret = pfm_reserve_session(NULL, 1, reserve_cpu);
 		if (ret) goto cleanup_reserve;
@@ -6500,6 +6502,7 @@ pfm_install_alt_pmu_interrupt(pfm_intr_handler_desc_t *hdl)
 	/* officially change to the alternate interrupt handler */
 	pfm_alt_intr_handler = hdl;
 
+	put_online_cpus_atomic();
 	spin_unlock(&pfm_alt_install_check);
 
 	return 0;
@@ -6512,6 +6515,7 @@ cleanup_reserve:
 		pfm_unreserve_session(NULL, 1, i);
 	}
 
+	put_online_cpus_atomic();
 	spin_unlock(&pfm_alt_install_check);
 
 	return ret;
@@ -6536,6 +6540,7 @@ pfm_remove_alt_pmu_interrupt(pfm_intr_handler_desc_t *hdl)
 
 	pfm_alt_intr_handler = NULL;
 
+	get_online_cpus_atomic();
 	ret = on_each_cpu(pfm_alt_restore_pmu_state, NULL, 1);
 	if (ret) {
 		DPRINT(("on_each_cpu() failed: %d\n", ret));
@@ -6545,6 +6550,7 @@ pfm_remove_alt_pmu_interrupt(pfm_intr_handler_desc_t *hdl)
 		pfm_unreserve_session(NULL, 1, i);
 	}
 
+	put_online_cpus_atomic();
 	spin_unlock(&pfm_alt_install_check);
 
 	return 0;
diff --git a/arch/ia64/kernel/smp.c b/arch/ia64/kernel/smp.c
index 9fcd4e6..d9a4636 100644
--- a/arch/ia64/kernel/smp.c
+++ b/arch/ia64/kernel/smp.c
@@ -24,6 +24,7 @@
 #include <linux/init.h>
 #include <linux/interrupt.h>
 #include <linux/smp.h>
+#include <linux/cpu.h>
 #include <linux/kernel_stat.h>
 #include <linux/mm.h>
 #include <linux/cache.h>
@@ -154,12 +155,15 @@ send_IPI_single (int dest_cpu, int op)
 static inline void
 send_IPI_allbutself (int op)
 {
-	unsigned int i;
+	unsigned int i, cpu;
 
+	get_online_cpus_atomic();
+	cpu = smp_processor_id();
 	for_each_online_cpu(i) {
-		if (i != smp_processor_id())
+		if (i != cpu)
 			send_IPI_single(i, op);
 	}
+	put_online_cpus_atomic();
 }
 
 /*
@@ -170,9 +174,11 @@ send_IPI_mask(const struct cpumask *mask, int op)
 {
 	unsigned int cpu;
 
+	get_online_cpus_atomic();
 	for_each_cpu(cpu, mask) {
 			send_IPI_single(cpu, op);
 	}
+	put_online_cpus_atomic();
 }
 
 /*
@@ -183,9 +189,11 @@ send_IPI_all (int op)
 {
 	int i;
 
+	get_online_cpus_atomic();
 	for_each_online_cpu(i) {
 		send_IPI_single(i, op);
 	}
+	put_online_cpus_atomic();
 }
 
 /*
@@ -259,7 +267,7 @@ smp_flush_tlb_cpumask(cpumask_t xcpumask)
 	cpumask_t cpumask = xcpumask;
 	int mycpu, cpu, flush_mycpu = 0;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 	mycpu = smp_processor_id();
 
 	for_each_cpu_mask(cpu, cpumask)
@@ -280,7 +288,7 @@ smp_flush_tlb_cpumask(cpumask_t xcpumask)
 		while(counts[cpu] == (local_tlb_flush_counts[cpu].count & 0xffff))
 			udelay(FLUSH_DELAY);
 
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 void
@@ -293,12 +301,13 @@ void
 smp_flush_tlb_mm (struct mm_struct *mm)
 {
 	cpumask_var_t cpus;
-	preempt_disable();
+
+	get_online_cpus_atomic();
 	/* this happens for the common case of a single-threaded fork():  */
 	if (likely(mm == current->active_mm && atomic_read(&mm->mm_users) == 1))
 	{
 		local_finish_flush_tlb_mm(mm);
-		preempt_enable();
+		put_online_cpus_atomic();
 		return;
 	}
 	if (!alloc_cpumask_var(&cpus, GFP_ATOMIC)) {
@@ -313,7 +322,7 @@ smp_flush_tlb_mm (struct mm_struct *mm)
 	local_irq_disable();
 	local_finish_flush_tlb_mm(mm);
 	local_irq_enable();
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 void arch_send_call_function_single_ipi(int cpu)
diff --git a/arch/ia64/mm/tlb.c b/arch/ia64/mm/tlb.c
index ed61297..8f03b58 100644
--- a/arch/ia64/mm/tlb.c
+++ b/arch/ia64/mm/tlb.c
@@ -20,6 +20,7 @@
 #include <linux/kernel.h>
 #include <linux/sched.h>
 #include <linux/smp.h>
+#include <linux/cpu.h>
 #include <linux/mm.h>
 #include <linux/bootmem.h>
 #include <linux/slab.h>
@@ -87,11 +88,12 @@ wrap_mmu_context (struct mm_struct *mm)
 	 * can't call flush_tlb_all() here because of race condition
 	 * with O(1) scheduler [EF]
 	 */
-	cpu = get_cpu(); /* prevent preemption/migration */
+	get_online_cpus_atomic(); /* prevent preemption/migration */
+	cpu = smp_processor_id();
 	for_each_online_cpu(i)
 		if (i != cpu)
 			per_cpu(ia64_need_tlb_flush, i) = 1;
-	put_cpu();
+	put_online_cpus_atomic();
 	local_flush_tlb_all();
 }
 

[PATCH v5 35/45] m32r: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Hirokazu Takata <takata@linux-m32r.org>
Cc: linux-m32r@ml.linux-m32r.org
Cc: linux-m32r-ja@ml.linux-m32r.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/m32r/kernel/smp.c |   12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/arch/m32r/kernel/smp.c b/arch/m32r/kernel/smp.c
index ce7aea3..0dad4d7 100644
--- a/arch/m32r/kernel/smp.c
+++ b/arch/m32r/kernel/smp.c
@@ -151,7 +151,7 @@ void smp_flush_cache_all(void)
 	cpumask_t cpumask;
 	unsigned long *mask;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 	cpumask_copy(&cpumask, cpu_online_mask);
 	cpumask_clear_cpu(smp_processor_id(), &cpumask);
 	spin_lock(&flushcache_lock);
@@ -162,7 +162,7 @@ void smp_flush_cache_all(void)
 	while (flushcache_cpumask)
 		mb();
 	spin_unlock(&flushcache_lock);
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 void smp_flush_cache_all_interrupt(void)
@@ -250,7 +250,7 @@ void smp_flush_tlb_mm(struct mm_struct *mm)
 	unsigned long *mmc;
 	unsigned long flags;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 	cpu_id = smp_processor_id();
 	mmc = &mm->context[cpu_id];
 	cpumask_copy(&cpu_mask, mm_cpumask(mm));
@@ -268,7 +268,7 @@ void smp_flush_tlb_mm(struct mm_struct *mm)
 	if (!cpumask_empty(&cpu_mask))
 		flush_tlb_others(cpu_mask, mm, NULL, FLUSH_ALL);
 
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 /*==========================================================================*
@@ -715,10 +715,12 @@ static void send_IPI_allbutself(int ipi_num, int try)
 {
 	cpumask_t cpumask;
 
+	get_online_cpus_atomic();
 	cpumask_copy(&cpumask, cpu_online_mask);
 	cpumask_clear_cpu(smp_processor_id(), &cpumask);
 
 	send_IPI_mask(&cpumask, ipi_num, try);
+	put_online_cpus_atomic();
 }
 
 /*==========================================================================*
@@ -750,6 +752,7 @@ static void send_IPI_mask(const struct cpumask *cpumask, int ipi_num, int try)
 	if (num_cpus <= 1)	/* NO MP */
 		return;
 
+	get_online_cpus_atomic();
 	cpumask_and(&tmp, cpumask, cpu_online_mask);
 	BUG_ON(!cpumask_equal(cpumask, &tmp));
 
@@ -760,6 +763,7 @@ static void send_IPI_mask(const struct cpumask *cpumask, int ipi_num, int try)
 	}
 
 	send_IPI_mask_phys(&physid_mask, ipi_num, try);
+	put_online_cpus_atomic();
 }
 
 /*==========================================================================*

[PATCH v5 36/45] MIPS: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs fom
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: David Daney <david.daney@cavium.com>
Cc: linux-mips@linux-mips.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/mips/kernel/cevt-smtc.c |    8 ++++++++
 arch/mips/kernel/smp.c       |   16 ++++++++--------
 arch/mips/kernel/smtc.c      |    3 +++
 arch/mips/mm/c-octeon.c      |    4 ++--
 4 files changed, 21 insertions(+), 10 deletions(-)

diff --git a/arch/mips/kernel/cevt-smtc.c b/arch/mips/kernel/cevt-smtc.c
index 2e72d30..6fb311b 100644
--- a/arch/mips/kernel/cevt-smtc.c
+++ b/arch/mips/kernel/cevt-smtc.c
@@ -11,6 +11,7 @@
 #include <linux/interrupt.h>
 #include <linux/percpu.h>
 #include <linux/smp.h>
+#include <linux/cpu.h>
 #include <linux/irq.h>
 
 #include <asm/smtc_ipi.h>
@@ -84,6 +85,8 @@ static int mips_next_event(unsigned long delta,
 	unsigned long nextcomp = 0L;
 	int vpe = current_cpu_data.vpe_id;
 	int cpu = smp_processor_id();
+
+	get_online_cpus_atomic();
 	local_irq_save(flags);
 	mtflags = dmt();
 
@@ -164,6 +167,7 @@ static int mips_next_event(unsigned long delta,
 	}
 	emt(mtflags);
 	local_irq_restore(flags);
+	put_online_cpus_atomic();
 	return 0;
 }
 
@@ -180,6 +184,7 @@ void smtc_distribute_timer(int vpe)
 
 repeat:
 	nextstamp = 0L;
+	get_online_cpus_atomic();
 	for_each_online_cpu(cpu) {
 	    /*
 	     * Find virtual CPUs within the current VPE who have
@@ -221,6 +226,9 @@ repeat:
 
 	    }
 	}
+
+	put_online_cpus_atomic()
+
 	/* Reprogram for interrupt at next soonest timestamp for VPE */
 	if (ISVALID(nextstamp)) {
 		write_c0_compare(nextstamp);
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index 66bf4e2..3828afa 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -248,12 +248,12 @@ static inline void smp_on_other_tlbs(void (*func) (void *info), void *info)
 
 static inline void smp_on_each_tlb(void (*func) (void *info), void *info)
 {
-	preempt_disable();
+	get_online_cpus_atomic();
 
 	smp_on_other_tlbs(func, info);
 	func(info);
 
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 /*
@@ -271,7 +271,7 @@ static inline void smp_on_each_tlb(void (*func) (void *info), void *info)
 
 void flush_tlb_mm(struct mm_struct *mm)
 {
-	preempt_disable();
+	get_online_cpus_atomic();
 
 	if ((atomic_read(&mm->mm_users) != 1) || (current->mm != mm)) {
 		smp_on_other_tlbs(flush_tlb_mm_ipi, mm);
@@ -285,7 +285,7 @@ void flush_tlb_mm(struct mm_struct *mm)
 	}
 	local_flush_tlb_mm(mm);
 
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 struct flush_tlb_data {
@@ -305,7 +305,7 @@ void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned l
 {
 	struct mm_struct *mm = vma->vm_mm;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 	if ((atomic_read(&mm->mm_users) != 1) || (current->mm != mm)) {
 		struct flush_tlb_data fd = {
 			.vma = vma,
@@ -323,7 +323,7 @@ void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned l
 		}
 	}
 	local_flush_tlb_range(vma, start, end);
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 static void flush_tlb_kernel_range_ipi(void *info)
@@ -352,7 +352,7 @@ static void flush_tlb_page_ipi(void *info)
 
 void flush_tlb_page(struct vm_area_struct *vma, unsigned long page)
 {
-	preempt_disable();
+	get_online_cpus_atomic();
 	if ((atomic_read(&vma->vm_mm->mm_users) != 1) || (current->mm != vma->vm_mm)) {
 		struct flush_tlb_data fd = {
 			.vma = vma,
@@ -369,7 +369,7 @@ void flush_tlb_page(struct vm_area_struct *vma, unsigned long page)
 		}
 	}
 	local_flush_tlb_page(vma, page);
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 static void flush_tlb_one_ipi(void *info)
diff --git a/arch/mips/kernel/smtc.c b/arch/mips/kernel/smtc.c
index 1d47843..caf081e 100644
--- a/arch/mips/kernel/smtc.c
+++ b/arch/mips/kernel/smtc.c
@@ -22,6 +22,7 @@
 #include <linux/sched.h>
 #include <linux/smp.h>
 #include <linux/cpumask.h>
+#include <linux/cpu.h>
 #include <linux/interrupt.h>
 #include <linux/kernel_stat.h>
 #include <linux/module.h>
@@ -1143,6 +1144,7 @@ static irqreturn_t ipi_interrupt(int irq, void *dev_idm)
 	 * for the current TC, so we ought not to have to do it explicitly here.
 	 */
 
+	get_online_cpus_atomic();
 	for_each_online_cpu(cpu) {
 		if (cpu_data[cpu].vpe_id != my_vpe)
 			continue;
@@ -1179,6 +1181,7 @@ static irqreturn_t ipi_interrupt(int irq, void *dev_idm)
 			}
 		}
 	}
+	put_online_cpus_atomic();
 
 	return IRQ_HANDLED;
 }
diff --git a/arch/mips/mm/c-octeon.c b/arch/mips/mm/c-octeon.c
index 6ec04da..cd2c1ce 100644
--- a/arch/mips/mm/c-octeon.c
+++ b/arch/mips/mm/c-octeon.c
@@ -73,7 +73,7 @@ static void octeon_flush_icache_all_cores(struct vm_area_struct *vma)
 	mb();
 	octeon_local_flush_icache();
 #ifdef CONFIG_SMP
-	preempt_disable();
+	get_online_cpus_atomic();
 	cpu = smp_processor_id();
 
 	/*
@@ -88,7 +88,7 @@ static void octeon_flush_icache_all_cores(struct vm_area_struct *vma)
 	for_each_cpu(cpu, &mask)
 		octeon_send_ipi_single(cpu, SMP_ICACHE_FLUSH);
 
-	preempt_enable();
+	put_online_cpus_atomic();
 #endif
 }
 

[PATCH v5 37/45] mn10300: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: David Howells <dhowells@redhat.com>
Cc: Koichi Yasutake <yasutake.koichi@jp.panasonic.com>
Cc: linux-am33-list@redhat.com
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/mn10300/kernel/smp.c   |    2 ++
 arch/mn10300/mm/cache-smp.c |    5 +++++
 arch/mn10300/mm/tlb-smp.c   |   15 +++++++++------
 3 files changed, 16 insertions(+), 6 deletions(-)

diff --git a/arch/mn10300/kernel/smp.c b/arch/mn10300/kernel/smp.c
index 5d7e152..9dfa172 100644
--- a/arch/mn10300/kernel/smp.c
+++ b/arch/mn10300/kernel/smp.c
@@ -349,9 +349,11 @@ void send_IPI_allbutself(int irq)
 {
 	cpumask_t cpumask;
 
+	get_online_cpus_atomic();
 	cpumask_copy(&cpumask, cpu_online_mask);
 	cpumask_clear_cpu(smp_processor_id(), &cpumask);
 	send_IPI_mask(&cpumask, irq);
+	put_online_cpus_atomic();
 }
 
 void arch_send_call_function_ipi_mask(const struct cpumask *mask)
diff --git a/arch/mn10300/mm/cache-smp.c b/arch/mn10300/mm/cache-smp.c
index 2d23b9e..47ca1c9 100644
--- a/arch/mn10300/mm/cache-smp.c
+++ b/arch/mn10300/mm/cache-smp.c
@@ -13,6 +13,7 @@
 #include <linux/mman.h>
 #include <linux/threads.h>
 #include <linux/interrupt.h>
+#include <linux/cpu.h>
 #include <asm/page.h>
 #include <asm/pgtable.h>
 #include <asm/processor.h>
@@ -94,6 +95,8 @@ void smp_cache_call(unsigned long opr_mask,
 	smp_cache_mask = opr_mask;
 	smp_cache_start = start;
 	smp_cache_end = end;
+
+	get_online_cpus_atomic();
 	cpumask_copy(&smp_cache_ipi_map, cpu_online_mask);
 	cpumask_clear_cpu(smp_processor_id(), &smp_cache_ipi_map);
 
@@ -102,4 +105,6 @@ void smp_cache_call(unsigned long opr_mask,
 	while (!cpumask_empty(&smp_cache_ipi_map))
 		/* nothing. lockup detection does not belong here */
 		mb();
+
+	put_online_cpus_atomic();
 }
diff --git a/arch/mn10300/mm/tlb-smp.c b/arch/mn10300/mm/tlb-smp.c
index 3e57faf..d47304d 100644
--- a/arch/mn10300/mm/tlb-smp.c
+++ b/arch/mn10300/mm/tlb-smp.c
@@ -23,6 +23,7 @@
 #include <linux/sched.h>
 #include <linux/profile.h>
 #include <linux/smp.h>
+#include <linux/cpu.h>
 #include <asm/tlbflush.h>
 #include <asm/bitops.h>
 #include <asm/processor.h>
@@ -105,6 +106,7 @@ static void flush_tlb_others(cpumask_t cpumask, struct mm_struct *mm,
 	BUG_ON(cpumask_empty(&cpumask));
 	BUG_ON(cpumask_test_cpu(smp_processor_id(), &cpumask));
 
+	get_online_cpus_atomic();
 	cpumask_and(&tmp, &cpumask, cpu_online_mask);
 	BUG_ON(!cpumask_equal(&cpumask, &tmp));
 
@@ -134,6 +136,7 @@ static void flush_tlb_others(cpumask_t cpumask, struct mm_struct *mm,
 	flush_mm = NULL;
 	flush_va = 0;
 	spin_unlock(&tlbstate_lock);
+	put_online_cpus_atomic();
 }
 
 /**
@@ -144,7 +147,7 @@ void flush_tlb_mm(struct mm_struct *mm)
 {
 	cpumask_t cpu_mask;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 	cpumask_copy(&cpu_mask, mm_cpumask(mm));
 	cpumask_clear_cpu(smp_processor_id(), &cpu_mask);
 
@@ -152,7 +155,7 @@ void flush_tlb_mm(struct mm_struct *mm)
 	if (!cpumask_empty(&cpu_mask))
 		flush_tlb_others(cpu_mask, mm, FLUSH_ALL);
 
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 /**
@@ -163,7 +166,7 @@ void flush_tlb_current_task(void)
 	struct mm_struct *mm = current->mm;
 	cpumask_t cpu_mask;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 	cpumask_copy(&cpu_mask, mm_cpumask(mm));
 	cpumask_clear_cpu(smp_processor_id(), &cpu_mask);
 
@@ -171,7 +174,7 @@ void flush_tlb_current_task(void)
 	if (!cpumask_empty(&cpu_mask))
 		flush_tlb_others(cpu_mask, mm, FLUSH_ALL);
 
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 /**
@@ -184,7 +187,7 @@ void flush_tlb_page(struct vm_area_struct *vma, unsigned long va)
 	struct mm_struct *mm = vma->vm_mm;
 	cpumask_t cpu_mask;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 	cpumask_copy(&cpu_mask, mm_cpumask(mm));
 	cpumask_clear_cpu(smp_processor_id(), &cpu_mask);
 
@@ -192,7 +195,7 @@ void flush_tlb_page(struct vm_area_struct *vma, unsigned long va)
 	if (!cpumask_empty(&cpu_mask))
 		flush_tlb_others(cpu_mask, mm, va);
 
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 /**

[PATCH v5 38/45] parisc: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: "James E.J. Bottomley" <jejb@parisc-linux.org>
Cc: linux-parisc@vger.kernel.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/parisc/kernel/smp.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/parisc/kernel/smp.c b/arch/parisc/kernel/smp.c
index 6266730..f7851b7 100644
--- a/arch/parisc/kernel/smp.c
+++ b/arch/parisc/kernel/smp.c
@@ -229,11 +229,13 @@ static inline void
 send_IPI_allbutself(enum ipi_message_type op)
 {
 	int i;
-	
+
+	get_online_cpus_atomic();
 	for_each_online_cpu(i) {
 		if (i != smp_processor_id())
 			send_IPI_single(i, op);
 	}
+	put_online_cpus_atomic();
 }
 
 

[PATCH v5 39/45] powerpc: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: linuxppc-dev@lists.ozlabs.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/powerpc/mm/mmu_context_nohash.c |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/powerpc/mm/mmu_context_nohash.c b/arch/powerpc/mm/mmu_context_nohash.c
index e779642..29f58cf 100644
--- a/arch/powerpc/mm/mmu_context_nohash.c
+++ b/arch/powerpc/mm/mmu_context_nohash.c
@@ -196,6 +196,7 @@ void switch_mmu_context(struct mm_struct *prev, struct mm_struct *next)
 
 	/* No lockless fast path .. yet */
 	raw_spin_lock(&context_lock);
+	get_online_cpus_atomic();
 
 	pr_hard("[%d] activating context for mm @%p, active=%d, id=%d",
 		cpu, next, next->context.active, next->context.id);
@@ -279,6 +280,7 @@ void switch_mmu_context(struct mm_struct *prev, struct mm_struct *next)
 	/* Flick the MMU and release lock */
 	pr_hardcont(" -> %d\n", id);
 	set_context(id, next->pgd);
+	put_online_cpus_atomic();
 	raw_spin_unlock(&context_lock);
 }
 

[PATCH v5 40/45] sh: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Paul Mundt <lethal@linux-sh.org>
Cc: linux-sh@vger.kernel.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/sh/kernel/smp.c |   12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/arch/sh/kernel/smp.c b/arch/sh/kernel/smp.c
index 2062aa8..232fabe 100644
--- a/arch/sh/kernel/smp.c
+++ b/arch/sh/kernel/smp.c
@@ -357,7 +357,7 @@ static void flush_tlb_mm_ipi(void *mm)
  */
 void flush_tlb_mm(struct mm_struct *mm)
 {
-	preempt_disable();
+	get_online_cpus_atomic();
 
 	if ((atomic_read(&mm->mm_users) != 1) || (current->mm != mm)) {
 		smp_call_function(flush_tlb_mm_ipi, (void *)mm, 1);
@@ -369,7 +369,7 @@ void flush_tlb_mm(struct mm_struct *mm)
 	}
 	local_flush_tlb_mm(mm);
 
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 struct flush_tlb_data {
@@ -390,7 +390,7 @@ void flush_tlb_range(struct vm_area_struct *vma,
 {
 	struct mm_struct *mm = vma->vm_mm;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 	if ((atomic_read(&mm->mm_users) != 1) || (current->mm != mm)) {
 		struct flush_tlb_data fd;
 
@@ -405,7 +405,7 @@ void flush_tlb_range(struct vm_area_struct *vma,
 				cpu_context(i, mm) = 0;
 	}
 	local_flush_tlb_range(vma, start, end);
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 static void flush_tlb_kernel_range_ipi(void *info)
@@ -433,7 +433,7 @@ static void flush_tlb_page_ipi(void *info)
 
 void flush_tlb_page(struct vm_area_struct *vma, unsigned long page)
 {
-	preempt_disable();
+	get_online_cpus_atomic();
 	if ((atomic_read(&vma->vm_mm->mm_users) != 1) ||
 	    (current->mm != vma->vm_mm)) {
 		struct flush_tlb_data fd;
@@ -448,7 +448,7 @@ void flush_tlb_page(struct vm_area_struct *vma, unsigned long page)
 				cpu_context(i, vma->vm_mm) = 0;
 	}
 	local_flush_tlb_page(vma, page);
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 static void flush_tlb_one_ipi(void *info)

[PATCH v5 41/45] sparc: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: "David S. Miller" <davem@davemloft.net>
Cc: Sam Ravnborg <sam@ravnborg.org>
Cc: sparclinux@vger.kernel.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/sparc/kernel/leon_smp.c  |    2 ++
 arch/sparc/kernel/smp_64.c    |    9 +++++----
 arch/sparc/kernel/sun4d_smp.c |    2 ++
 arch/sparc/kernel/sun4m_smp.c |    3 +++
 4 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/arch/sparc/kernel/leon_smp.c b/arch/sparc/kernel/leon_smp.c
index 0f3fb6d..441d3ac 100644
--- a/arch/sparc/kernel/leon_smp.c
+++ b/arch/sparc/kernel/leon_smp.c
@@ -420,6 +420,7 @@ static void leon_cross_call(smpfunc_t func, cpumask_t mask, unsigned long arg1,
 		unsigned long flags;
 
 		spin_lock_irqsave(&cross_call_lock, flags);
+		get_online_cpus_atomic();
 
 		{
 			/* If you make changes here, make sure gcc generates proper code... */
@@ -476,6 +477,7 @@ static void leon_cross_call(smpfunc_t func, cpumask_t mask, unsigned long arg1,
 			} while (++i <= high);
 		}
 
+		put_online_cpus_atomic();
 		spin_unlock_irqrestore(&cross_call_lock, flags);
 	}
 }
diff --git a/arch/sparc/kernel/smp_64.c b/arch/sparc/kernel/smp_64.c
index 537eb66..e1d7300 100644
--- a/arch/sparc/kernel/smp_64.c
+++ b/arch/sparc/kernel/smp_64.c
@@ -894,7 +894,8 @@ void smp_flush_dcache_page_impl(struct page *page, int cpu)
 	atomic_inc(&dcpage_flushes);
 #endif
 
-	this_cpu = get_cpu();
+	get_online_cpus_atomic();
+	this_cpu = smp_processor_id();
 
 	if (cpu == this_cpu) {
 		__local_flush_dcache_page(page);
@@ -920,7 +921,7 @@ void smp_flush_dcache_page_impl(struct page *page, int cpu)
 		}
 	}
 
-	put_cpu();
+	put_online_cpus_atomic();
 }
 
 void flush_dcache_page_all(struct mm_struct *mm, struct page *page)
@@ -931,7 +932,7 @@ void flush_dcache_page_all(struct mm_struct *mm, struct page *page)
 	if (tlb_type == hypervisor)
 		return;
 
-	preempt_disable();
+	get_online_cpus_atomic();
 
 #ifdef CONFIG_DEBUG_DCFLUSH
 	atomic_inc(&dcpage_flushes);
@@ -956,7 +957,7 @@ void flush_dcache_page_all(struct mm_struct *mm, struct page *page)
 	}
 	__local_flush_dcache_page(page);
 
-	preempt_enable();
+	put_online_cpus_atomic();
 }
 
 void __irq_entry smp_new_mmu_context_version_client(int irq, struct pt_regs *regs)
diff --git a/arch/sparc/kernel/sun4d_smp.c b/arch/sparc/kernel/sun4d_smp.c
index ddaea31..1fa7ff2 100644
--- a/arch/sparc/kernel/sun4d_smp.c
+++ b/arch/sparc/kernel/sun4d_smp.c
@@ -300,6 +300,7 @@ static void sun4d_cross_call(smpfunc_t func, cpumask_t mask, unsigned long arg1,
 		unsigned long flags;
 
 		spin_lock_irqsave(&cross_call_lock, flags);
+		get_online_cpus_atomic();
 
 		{
 			/*
@@ -356,6 +357,7 @@ static void sun4d_cross_call(smpfunc_t func, cpumask_t mask, unsigned long arg1,
 			} while (++i <= high);
 		}
 
+		put_online_cpus_atomic();
 		spin_unlock_irqrestore(&cross_call_lock, flags);
 	}
 }
diff --git a/arch/sparc/kernel/sun4m_smp.c b/arch/sparc/kernel/sun4m_smp.c
index 128af73..5599548 100644
--- a/arch/sparc/kernel/sun4m_smp.c
+++ b/arch/sparc/kernel/sun4m_smp.c
@@ -192,6 +192,7 @@ static void sun4m_cross_call(smpfunc_t func, cpumask_t mask, unsigned long arg1,
 		unsigned long flags;
 
 		spin_lock_irqsave(&cross_call_lock, flags);
+		get_online_cpus_atomic();
 
 		/* Init function glue. */
 		ccall_info.func = func;
@@ -238,6 +239,8 @@ static void sun4m_cross_call(smpfunc_t func, cpumask_t mask, unsigned long arg1,
 					barrier();
 			} while (++i < ncpus);
 		}
+
+		put_online_cpus_atomic();
 		spin_unlock_irqrestore(&cross_call_lock, flags);
 }
 

[PATCH v5 42/45] tile: Use get/put_online_cpus_atomic() to prevent CPU offline

[Srivatsa S. Bhat]

Once stop_machine() is gone from the CPU offline path, we won't be able to
depend on preempt_disable() or local_irq_disable() to prevent CPUs from
going offline from under us.

Use the get/put_online_cpus_atomic() APIs to prevent CPUs from going offline,
while invoking from atomic context.

Cc: Chris Metcalf <cmetcalf@tilera.com>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 arch/tile/kernel/smp.c |    4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/tile/kernel/smp.c b/arch/tile/kernel/smp.c
index cbc73a8..fb30624 100644
--- a/arch/tile/kernel/smp.c
+++ b/arch/tile/kernel/smp.c
@@ -15,6 +15,7 @@
  */
 
 #include <linux/smp.h>
+#include <linux/cpu.h>
 #include <linux/interrupt.h>
 #include <linux/io.h>
 #include <linux/irq.h>
@@ -82,9 +83,12 @@ void send_IPI_many(const struct cpumask *mask, int tag)
 void send_IPI_allbutself(int tag)
 {
 	struct cpumask mask;
+
+	get_online_cpus_atomic();
 	cpumask_copy(&mask, cpu_online_mask);
 	cpumask_clear_cpu(smp_processor_id(), &mask);
 	send_IPI_many(&mask, tag);
+	put_online_cpus_atomic();
 }
 
 /*

[PATCH v5 43/45] cpu: No more __stop_machine() in _cpu_down()

[Srivatsa S. Bhat]

From: Paul E. McKenney <paul.mckenney@linaro.org>

The _cpu_down() function invoked as part of the CPU-hotplug offlining
process currently invokes __stop_machine(), which is slow and inflicts
substantial real-time latencies on the entire system.  This patch
substitutes stop_one_cpu() for __stop_machine() in order to improve
both performance and real-time latency.

There were a number of uses of preempt_disable() or local_irq_disable()
that were intended to block CPU-hotplug offlining. These were fixed by
using get/put_online_cpus_atomic(), which is the new synchronization
primitive to prevent CPU offline, while invoking from atomic context.

Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
[ srivatsa.bhat@linux.vnet.ibm.com: Refer to the new sync primitives for
  readers (in the changelog); s/stop_cpus/stop_one_cpu and fix comment
  referring to stop_machine in the code]
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 kernel/cpu.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index 1c84138..7a51fb6 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -337,7 +337,7 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
 	}
 	smpboot_park_threads(cpu);
 
-	err = __stop_machine(take_cpu_down, &tcd_param, cpumask_of(cpu));
+	err = stop_one_cpu(cpu, take_cpu_down, &tcd_param);
 	if (err) {
 		/* CPU didn't die: tell everyone.  Can't complain. */
 		smpboot_unpark_threads(cpu);
@@ -349,7 +349,7 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
 	/*
 	 * The migration_call() CPU_DYING callback will have removed all
 	 * runnable tasks from the cpu, there's only the idle task left now
-	 * that the migration thread is done doing the stop_machine thing.
+	 * that the migration thread is done doing the stop_one_cpu() thing.
 	 *
 	 * Wait for the stop thread to go away.
 	 */

[PATCH v5 44/45] CPU hotplug, stop_machine: Decouple CPU hotplug from stop_machine() in Kconfig

[Srivatsa S. Bhat]

... and also cleanup a comment that refers to CPU hotplug being dependent on
stop_machine().

Cc: David Howells <dhowells@redhat.com>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 include/linux/stop_machine.h |    2 +-
 init/Kconfig                 |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/linux/stop_machine.h b/include/linux/stop_machine.h
index 3b5e910..ce2d3c4 100644
--- a/include/linux/stop_machine.h
+++ b/include/linux/stop_machine.h
@@ -120,7 +120,7 @@ int stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus);
  * @cpus: the cpus to run the @fn() on (NULL = any online cpu)
  *
  * Description: This is a special version of the above, which assumes cpus
- * won't come or go while it's being called.  Used by hotplug cpu.
+ * won't come or go while it's being called.
  */
 int __stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus);
 
diff --git a/init/Kconfig b/init/Kconfig
index be8b7f5..048a0c5 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1711,7 +1711,7 @@ config INIT_ALL_POSSIBLE
 config STOP_MACHINE
 	bool
 	default y
-	depends on (SMP && MODULE_UNLOAD) || HOTPLUG_CPU
+	depends on (SMP && MODULE_UNLOAD)
 	help
 	  Need stop_machine() primitive.
 

Re: [PATCH v5 44/45] CPU hotplug, stop_machine: Decouple CPU hotplug from stop_machine() in Kconfig

[Paul E. McKenney]

On Tue, Jan 22, 2013 at 01:15:22PM +0530, Srivatsa S. Bhat wrote:
> ... and also cleanup a comment that refers to CPU hotplug being dependent on
> stop_machine().
> 
> Cc: David Howells <dhowells@redhat.com>
> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>

Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

(Hey, I thought I owed myself an easy one!)

> ---
> 
>  include/linux/stop_machine.h |    2 +-
>  init/Kconfig                 |    2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/include/linux/stop_machine.h b/include/linux/stop_machine.h
> index 3b5e910..ce2d3c4 100644
> --- a/include/linux/stop_machine.h
> +++ b/include/linux/stop_machine.h
> @@ -120,7 +120,7 @@ int stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus);
>   * @cpus: the cpus to run the @fn() on (NULL = any online cpu)
>   *
>   * Description: This is a special version of the above, which assumes cpus
> - * won't come or go while it's being called.  Used by hotplug cpu.
> + * won't come or go while it's being called.
>   */
>  int __stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus);
> 
> diff --git a/init/Kconfig b/init/Kconfig
> index be8b7f5..048a0c5 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -1711,7 +1711,7 @@ config INIT_ALL_POSSIBLE
>  config STOP_MACHINE
>  	bool
>  	default y
> -	depends on (SMP && MODULE_UNLOAD) || HOTPLUG_CPU
> +	depends on (SMP && MODULE_UNLOAD)
>  	help
>  	  Need stop_machine() primitive.
> 
> 

Re: [PATCH v5 44/45] CPU hotplug, stop_machine: Decouple CPU hotplug from stop_machine() in Kconfig

[Srivatsa S. Bhat]

On 02/09/2013 05:45 AM, Paul E. McKenney wrote:
> On Tue, Jan 22, 2013 at 01:15:22PM +0530, Srivatsa S. Bhat wrote:
>> ... and also cleanup a comment that refers to CPU hotplug being dependent on
>> stop_machine().
>>
>> Cc: David Howells <dhowells@redhat.com>
>> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
> 
> Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> 
> (Hey, I thought I owed myself an easy one!)
> 

Haha ;-)

Regards,
Srivatsa S. Bhat

>> ---
>>
>>  include/linux/stop_machine.h |    2 +-
>>  init/Kconfig                 |    2 +-
>>  2 files changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/include/linux/stop_machine.h b/include/linux/stop_machine.h
>> index 3b5e910..ce2d3c4 100644
>> --- a/include/linux/stop_machine.h
>> +++ b/include/linux/stop_machine.h
>> @@ -120,7 +120,7 @@ int stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus);
>>   * @cpus: the cpus to run the @fn() on (NULL = any online cpu)
>>   *
>>   * Description: This is a special version of the above, which assumes cpus
>> - * won't come or go while it's being called.  Used by hotplug cpu.
>> + * won't come or go while it's being called.
>>   */
>>  int __stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus);
>>
>> diff --git a/init/Kconfig b/init/Kconfig
>> index be8b7f5..048a0c5 100644
>> --- a/init/Kconfig
>> +++ b/init/Kconfig
>> @@ -1711,7 +1711,7 @@ config INIT_ALL_POSSIBLE
>>  config STOP_MACHINE
>>  	bool
>>  	default y
>> -	depends on (SMP && MODULE_UNLOAD) || HOTPLUG_CPU
>> +	depends on (SMP && MODULE_UNLOAD)
>>  	help
>>  	  Need stop_machine() primitive.
>>
>>

[PATCH v5 45/45] Documentation/cpu-hotplug: Remove references to stop_machine()

[Srivatsa S. Bhat]

Since stop_machine() is no longer used in the CPU offline path, we cannot
disable CPU hotplug using preempt_disable()/local_irq_disable() etc. We
need to use the newly introduced get/put_online_cpus_atomic() APIs.
Reflect this in the documentation.

Cc: Rob Landley <rob@landley.net>
Cc: linux-doc@vger.kernel.org
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
---

 Documentation/cpu-hotplug.txt |   17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/Documentation/cpu-hotplug.txt b/Documentation/cpu-hotplug.txt
index 9f40135..7f907ec 100644
--- a/Documentation/cpu-hotplug.txt
+++ b/Documentation/cpu-hotplug.txt
@@ -113,13 +113,15 @@ Never use anything other than cpumask_t to represent bitmap of CPUs.
 	#include <linux/cpu.h>
 	get_online_cpus() and put_online_cpus():
 
-The above calls are used to inhibit cpu hotplug operations. While the
+The above calls are used to inhibit cpu hotplug operations, when invoked from
+non-atomic context (because the above functions can sleep). While the
 cpu_hotplug.refcount is non zero, the cpu_online_mask will not change.
-If you merely need to avoid cpus going away, you could also use
-preempt_disable() and preempt_enable() for those sections.
-Just remember the critical section cannot call any
-function that can sleep or schedule this process away. The preempt_disable()
-will work as long as stop_machine_run() is used to take a cpu down.
+
+However, if you are executing in atomic context (ie., you can't afford to
+sleep), and you merely need to avoid cpus going offline, you can use
+get_online_cpus_atomic() and put_online_cpus_atomic() for those sections.
+Just remember the critical section cannot call any function that can sleep or
+schedule this process away.
 
 CPU Hotplug - Frequently Asked Questions.
 
@@ -360,6 +362,9 @@ A: There are two ways.  If your code can be run in interrupt context, use
 		return err;
 	}
 
+   If my_func_on_cpu() itself cannot block, use get/put_online_cpus_atomic()
+   instead of get/put_online_cpus() to prevent CPUs from going offline.
+
 Q: How do we determine how many CPUs are available for hotplug.
 A: There is no clear spec defined way from ACPI that can give us that
    information today. Based on some input from Natalie of Unisys,

Re: [PATCH v5 45/45] Documentation/cpu-hotplug: Remove references to stop_machine()

[Paul E. McKenney]

On Tue, Jan 22, 2013 at 01:15:48PM +0530, Srivatsa S. Bhat wrote:
> Since stop_machine() is no longer used in the CPU offline path, we cannot
> disable CPU hotplug using preempt_disable()/local_irq_disable() etc. We
> need to use the newly introduced get/put_online_cpus_atomic() APIs.
> Reflect this in the documentation.
> 
> Cc: Rob Landley <rob@landley.net>
> Cc: linux-doc@vger.kernel.org
> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>

Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

> ---
> 
>  Documentation/cpu-hotplug.txt |   17 +++++++++++------
>  1 file changed, 11 insertions(+), 6 deletions(-)
> 
> diff --git a/Documentation/cpu-hotplug.txt b/Documentation/cpu-hotplug.txt
> index 9f40135..7f907ec 100644
> --- a/Documentation/cpu-hotplug.txt
> +++ b/Documentation/cpu-hotplug.txt
> @@ -113,13 +113,15 @@ Never use anything other than cpumask_t to represent bitmap of CPUs.
>  	#include <linux/cpu.h>
>  	get_online_cpus() and put_online_cpus():
> 
> -The above calls are used to inhibit cpu hotplug operations. While the
> +The above calls are used to inhibit cpu hotplug operations, when invoked from
> +non-atomic context (because the above functions can sleep). While the
>  cpu_hotplug.refcount is non zero, the cpu_online_mask will not change.
> -If you merely need to avoid cpus going away, you could also use
> -preempt_disable() and preempt_enable() for those sections.
> -Just remember the critical section cannot call any
> -function that can sleep or schedule this process away. The preempt_disable()
> -will work as long as stop_machine_run() is used to take a cpu down.
> +
> +However, if you are executing in atomic context (ie., you can't afford to
> +sleep), and you merely need to avoid cpus going offline, you can use
> +get_online_cpus_atomic() and put_online_cpus_atomic() for those sections.
> +Just remember the critical section cannot call any function that can sleep or
> +schedule this process away.
> 
>  CPU Hotplug - Frequently Asked Questions.
> 
> @@ -360,6 +362,9 @@ A: There are two ways.  If your code can be run in interrupt context, use
>  		return err;
>  	}
> 
> +   If my_func_on_cpu() itself cannot block, use get/put_online_cpus_atomic()
> +   instead of get/put_online_cpus() to prevent CPUs from going offline.
> +
>  Q: How do we determine how many CPUs are available for hotplug.
>  A: There is no clear spec defined way from ACPI that can give us that
>     information today. Based on some input from Natalie of Unisys,
> 

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Srivatsa S. Bhat]

[-- Attachment #1: Type: text/plain, Size: 3770 bytes --]

On 01/22/2013 01:03 PM, Srivatsa S. Bhat wrote:
> Hi,
> 
> This patchset removes CPU hotplug's dependence on stop_machine() from the CPU
> offline path and provides an alternative (set of APIs) to preempt_disable() to
> prevent CPUs from going offline, which can be invoked from atomic context.
> The motivation behind the removal of stop_machine() is to avoid its ill-effects
> and thus improve the design of CPU hotplug. (More description regarding this
> is available in the patches).
> 
> All the users of preempt_disable()/local_irq_disable() who used to use it to
> prevent CPU offline, have been converted to the new primitives introduced in the
> patchset. Also, the CPU_DYING notifiers have been audited to check whether
> they can cope up with the removal of stop_machine() or whether they need to
> use new locks for synchronization (all CPU_DYING notifiers looked OK, without
> the need for any new locks).
> 
> Applies on v3.8-rc4. It currently has some locking issues with cpu idle (on
> which even lockdep didn't provide any insight unfortunately). So for now, it
> works with CONFIG_CPU_IDLE=n.
> 

I ran this patchset on a POWER 7 machine with 32 cores (128 logical CPUs)
[POWER doesn't have the cpu idle issue]. And the results (latency or the time
taken for a single CPU offline) are shown below.

Experiment:
----------

Run a heavy workload (genload from LTP) that generates significant system time;
With '# online CPUs' online, measure the time it takes to complete the stop-m/c
phase in mainline and the equivalent phase in the patched kernel for 1 CPU
offline operation. (It is important to note here that the measurement shows the
average time it takes to perform a *single* CPU offline operation).

Expected results:
----------------

Since stop-machine doesn't scale with no. of online CPUs, we expect the
mainline kernel to take longer and longer for taking 1 CPU offline, with
increasing no. of online CPUs. The patched kernel is expected to take a
constant amount of time, irrespective of the number of online CPUs, because it
has a scalable design.


Experimental results:
---------------------

                 Avg. latency of 1 CPU offline (ms) [stop-cpu/stop-m/c latency]

# online CPUs    Mainline (with stop-m/c)       This patchset (no stop-m/c)

      8                 17.04                          7.73

     16                 18.05                          6.44

     32                 17.31                          7.39

     64                 32.40                          9.28

    128                 98.23                          7.35


Analysis and conclusion:
------------------------

The patched kernel performs pretty well and meets our expectations. It beats
mainline easily. As shown in the table above and the graph attached with this
mail, it has the following advantages:

1. Avg. latency is less than mainline (roughly half that of even the least
   in mainline).

2. The avg. latency is a constant, irrespective of number of online CPUs in
   the system, which proves that the design/synchronization scheme is scalable.

3. Throughout the duration shown above, mainline disables interrupts on all
   CPUs. But the patched kernel not only has a smaller duration of hotplug,
   but also keeps interrupts enabled on other CPUs, which makes CPU offline
   less disruptive on latency-sensitive workloads running on the system.


So, this gives us an idea of how this patchset actually performs. Of course
there are bugs and issues that still need fixing (even mainline crashes with
hotplug sometimes), but I did the above experiment to verify whether the
design is working as expected and whether it really shows significant
improvements over mainline. And thankfully, it does :-)

Regards,
Srivatsa S. Bhat


[-- Attachment #2: CPU hotplug latency.png --]
[-- Type: image/png, Size: 172574 bytes --]

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Rusty Russell]

"Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com> writes:
> On 01/22/2013 01:03 PM, Srivatsa S. Bhat wrote:
>                  Avg. latency of 1 CPU offline (ms) [stop-cpu/stop-m/c latency]
>
> # online CPUs    Mainline (with stop-m/c)       This patchset (no stop-m/c)
>
>       8                 17.04                          7.73
>
>      16                 18.05                          6.44
>
>      32                 17.31                          7.39
>
>      64                 32.40                          9.28
>
>     128                 98.23                          7.35

Nice!  I wonder how the ARM guys feel with their quad-cpu systems...

Thanks!
Rusty.

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Srivatsa S. Bhat]

On 02/07/2013 09:44 AM, Rusty Russell wrote:
> "Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com> writes:
>> On 01/22/2013 01:03 PM, Srivatsa S. Bhat wrote:
>>                  Avg. latency of 1 CPU offline (ms) [stop-cpu/stop-m/c latency]
>>
>> # online CPUs    Mainline (with stop-m/c)       This patchset (no stop-m/c)
>>
>>       8                 17.04                          7.73
>>
>>      16                 18.05                          6.44
>>
>>      32                 17.31                          7.39
>>
>>      64                 32.40                          9.28
>>
>>     128                 98.23                          7.35
> 
> Nice!

Thank you :-)

>  I wonder how the ARM guys feel with their quad-cpu systems...
> 

That would be definitely interesting to know :-)

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Russell King - ARM Linux]

On Thu, Feb 07, 2013 at 11:41:34AM +0530, Srivatsa S. Bhat wrote:
> On 02/07/2013 09:44 AM, Rusty Russell wrote:
> > "Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com> writes:
> >> On 01/22/2013 01:03 PM, Srivatsa S. Bhat wrote:
> >>                  Avg. latency of 1 CPU offline (ms) [stop-cpu/stop-m/c latency]
> >>
> >> # online CPUs    Mainline (with stop-m/c)       This patchset (no stop-m/c)
> >>
> >>       8                 17.04                          7.73
> >>
> >>      16                 18.05                          6.44
> >>
> >>      32                 17.31                          7.39
> >>
> >>      64                 32.40                          9.28
> >>
> >>     128                 98.23                          7.35
> > 
> > Nice!
> 
> Thank you :-)
> 
> >  I wonder how the ARM guys feel with their quad-cpu systems...
> > 
> 
> That would be definitely interesting to know :-)

That depends what exactly you'd like tested (and how) and whether you'd
like it to be a test-chip based quad core, or an OMAP dual-core SoC.

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Srivatsa S. Bhat]

On 02/08/2013 09:11 PM, Russell King - ARM Linux wrote:
> On Thu, Feb 07, 2013 at 11:41:34AM +0530, Srivatsa S. Bhat wrote:
>> On 02/07/2013 09:44 AM, Rusty Russell wrote:
>>> "Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com> writes:
>>>> On 01/22/2013 01:03 PM, Srivatsa S. Bhat wrote:
>>>>                  Avg. latency of 1 CPU offline (ms) [stop-cpu/stop-m/c latency]
>>>>
>>>> # online CPUs    Mainline (with stop-m/c)       This patchset (no stop-m/c)
>>>>
>>>>       8                 17.04                          7.73
>>>>
>>>>      16                 18.05                          6.44
>>>>
>>>>      32                 17.31                          7.39
>>>>
>>>>      64                 32.40                          9.28
>>>>
>>>>     128                 98.23                          7.35
>>>
>>> Nice!
>>
>> Thank you :-)
>>
>>>  I wonder how the ARM guys feel with their quad-cpu systems...
>>>
>>
>> That would be definitely interesting to know :-)
> 
> That depends what exactly you'd like tested (and how) and whether you'd
> like it to be a test-chip based quad core, or an OMAP dual-core SoC.
> 

The effect of stop_machine() doesn't really depend on the CPU architecture
used underneath or the platform. It depends only on the _number_ of
_logical_ CPUs used.

And stop_machine() has 2 noticeable drawbacks:
1. It makes the hotplug operation itself slow
2. and it causes disruptions to the workloads running on the other
CPUs by hijacking the entire machine for significant amounts of time.

In my experiments (mentioned above), I tried to measure how my patchset
improves (reduces) the duration of hotplug (CPU offline) itself. Which is
also slightly indicative of the impact it has on the rest of the system.

But what would be nice to test, is a setup where the workloads running on
the rest of the system are latency-sensitive, and measure the impact of
CPU offline on them, with this patchset applied. That would tell us how
far is this useful in making CPU hotplug less disruptive on the system.

Of course, it would be nice to also see whether we observe any reduction
in hotplug duration itself (point 1 above) on ARM platforms with lot
of CPUs. [This could potentially speed up suspend/resume, which is used
rather heavily on ARM platforms].

The benefits from this patchset over mainline (both in terms of points
1 and 2 above) is expected to increase, with increasing number of CPUs in
the system.

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Srivatsa S. Bhat]

On 02/08/2013 10:14 PM, Srivatsa S. Bhat wrote:
> On 02/08/2013 09:11 PM, Russell King - ARM Linux wrote:
>> On Thu, Feb 07, 2013 at 11:41:34AM +0530, Srivatsa S. Bhat wrote:
>>> On 02/07/2013 09:44 AM, Rusty Russell wrote:
>>>> "Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com> writes:
>>>>> On 01/22/2013 01:03 PM, Srivatsa S. Bhat wrote:
>>>>>                  Avg. latency of 1 CPU offline (ms) [stop-cpu/stop-m/c latency]
>>>>>
>>>>> # online CPUs    Mainline (with stop-m/c)       This patchset (no stop-m/c)
>>>>>
>>>>>       8                 17.04                          7.73
>>>>>
>>>>>      16                 18.05                          6.44
>>>>>
>>>>>      32                 17.31                          7.39
>>>>>
>>>>>      64                 32.40                          9.28
>>>>>
>>>>>     128                 98.23                          7.35
>>>>
>>>> Nice!
>>>
>>> Thank you :-)
>>>
>>>>  I wonder how the ARM guys feel with their quad-cpu systems...
>>>>
>>>
>>> That would be definitely interesting to know :-)
>>
>> That depends what exactly you'd like tested (and how) and whether you'd
>> like it to be a test-chip based quad core, or an OMAP dual-core SoC.
>>
> 
> The effect of stop_machine() doesn't really depend on the CPU architecture
> used underneath or the platform. It depends only on the _number_ of
> _logical_ CPUs used.
> 
> And stop_machine() has 2 noticeable drawbacks:
> 1. It makes the hotplug operation itself slow
> 2. and it causes disruptions to the workloads running on the other
> CPUs by hijacking the entire machine for significant amounts of time.
> 
> In my experiments (mentioned above), I tried to measure how my patchset
> improves (reduces) the duration of hotplug (CPU offline) itself. Which is
> also slightly indicative of the impact it has on the rest of the system.
> 
> But what would be nice to test, is a setup where the workloads running on
> the rest of the system are latency-sensitive, and measure the impact of
> CPU offline on them, with this patchset applied. That would tell us how
> far is this useful in making CPU hotplug less disruptive on the system.
> 
> Of course, it would be nice to also see whether we observe any reduction
> in hotplug duration itself (point 1 above) on ARM platforms with lot
> of CPUs. [This could potentially speed up suspend/resume, which is used
> rather heavily on ARM platforms].
> 
> The benefits from this patchset over mainline (both in terms of points
> 1 and 2 above) is expected to increase, with increasing number of CPUs in
> the system.
> 

Adding Vincent to CC, who had previously evaluated the performance and
latency implications of CPU hotplug on ARM platforms, IIRC.

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Vincent Guittot]

Hi Srivatsa,

I can try to run some of our stress tests on your patches. Have you
got a git tree that i can pull ?

Regards,
Vincent

On 8 February 2013 19:09, Srivatsa S. Bhat
<srivatsa.bhat@linux.vnet.ibm.com> wrote:
> On 02/08/2013 10:14 PM, Srivatsa S. Bhat wrote:
>> On 02/08/2013 09:11 PM, Russell King - ARM Linux wrote:
>>> On Thu, Feb 07, 2013 at 11:41:34AM +0530, Srivatsa S. Bhat wrote:
>>>> On 02/07/2013 09:44 AM, Rusty Russell wrote:
>>>>> "Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com> writes:
>>>>>> On 01/22/2013 01:03 PM, Srivatsa S. Bhat wrote:
>>>>>>                  Avg. latency of 1 CPU offline (ms) [stop-cpu/stop-m/c latency]
>>>>>>
>>>>>> # online CPUs    Mainline (with stop-m/c)       This patchset (no stop-m/c)
>>>>>>
>>>>>>       8                 17.04                          7.73
>>>>>>
>>>>>>      16                 18.05                          6.44
>>>>>>
>>>>>>      32                 17.31                          7.39
>>>>>>
>>>>>>      64                 32.40                          9.28
>>>>>>
>>>>>>     128                 98.23                          7.35
>>>>>
>>>>> Nice!
>>>>
>>>> Thank you :-)
>>>>
>>>>>  I wonder how the ARM guys feel with their quad-cpu systems...
>>>>>
>>>>
>>>> That would be definitely interesting to know :-)
>>>
>>> That depends what exactly you'd like tested (and how) and whether you'd
>>> like it to be a test-chip based quad core, or an OMAP dual-core SoC.
>>>
>>
>> The effect of stop_machine() doesn't really depend on the CPU architecture
>> used underneath or the platform. It depends only on the _number_ of
>> _logical_ CPUs used.
>>
>> And stop_machine() has 2 noticeable drawbacks:
>> 1. It makes the hotplug operation itself slow
>> 2. and it causes disruptions to the workloads running on the other
>> CPUs by hijacking the entire machine for significant amounts of time.
>>
>> In my experiments (mentioned above), I tried to measure how my patchset
>> improves (reduces) the duration of hotplug (CPU offline) itself. Which is
>> also slightly indicative of the impact it has on the rest of the system.
>>
>> But what would be nice to test, is a setup where the workloads running on
>> the rest of the system are latency-sensitive, and measure the impact of
>> CPU offline on them, with this patchset applied. That would tell us how
>> far is this useful in making CPU hotplug less disruptive on the system.
>>
>> Of course, it would be nice to also see whether we observe any reduction
>> in hotplug duration itself (point 1 above) on ARM platforms with lot
>> of CPUs. [This could potentially speed up suspend/resume, which is used
>> rather heavily on ARM platforms].
>>
>> The benefits from this patchset over mainline (both in terms of points
>> 1 and 2 above) is expected to increase, with increasing number of CPUs in
>> the system.
>>
>
> Adding Vincent to CC, who had previously evaluated the performance and
> latency implications of CPU hotplug on ARM platforms, IIRC.
>
> Regards,
> Srivatsa S. Bhat
>

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Srivatsa S. Bhat]

On 02/11/2013 05:28 PM, Vincent Guittot wrote:
> On 8 February 2013 19:09, Srivatsa S. Bhat
> <srivatsa.bhat@linux.vnet.ibm.com> wrote:
>> On 02/08/2013 10:14 PM, Srivatsa S. Bhat wrote:
>>> On 02/08/2013 09:11 PM, Russell King - ARM Linux wrote:
>>>> On Thu, Feb 07, 2013 at 11:41:34AM +0530, Srivatsa S. Bhat wrote:
>>>>> On 02/07/2013 09:44 AM, Rusty Russell wrote:
>>>>>> "Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com> writes:
>>>>>>> On 01/22/2013 01:03 PM, Srivatsa S. Bhat wrote:
>>>>>>>                  Avg. latency of 1 CPU offline (ms) [stop-cpu/stop-m/c latency]
>>>>>>>
>>>>>>> # online CPUs    Mainline (with stop-m/c)       This patchset (no stop-m/c)
>>>>>>>
>>>>>>>       8                 17.04                          7.73
>>>>>>>
>>>>>>>      16                 18.05                          6.44
>>>>>>>
>>>>>>>      32                 17.31                          7.39
>>>>>>>
>>>>>>>      64                 32.40                          9.28
>>>>>>>
>>>>>>>     128                 98.23                          7.35
>>>>>>
>>>>>> Nice!
>>>>>
>>>>> Thank you :-)
>>>>>
>>>>>>  I wonder how the ARM guys feel with their quad-cpu systems...
>>>>>>
>>>>>
>>>>> That would be definitely interesting to know :-)
>>>>
>>>> That depends what exactly you'd like tested (and how) and whether you'd
>>>> like it to be a test-chip based quad core, or an OMAP dual-core SoC.
>>>>
>>>
>>> The effect of stop_machine() doesn't really depend on the CPU architecture
>>> used underneath or the platform. It depends only on the _number_ of
>>> _logical_ CPUs used.
>>>
>>> And stop_machine() has 2 noticeable drawbacks:
>>> 1. It makes the hotplug operation itself slow
>>> 2. and it causes disruptions to the workloads running on the other
>>> CPUs by hijacking the entire machine for significant amounts of time.
>>>
>>> In my experiments (mentioned above), I tried to measure how my patchset
>>> improves (reduces) the duration of hotplug (CPU offline) itself. Which is
>>> also slightly indicative of the impact it has on the rest of the system.
>>>
>>> But what would be nice to test, is a setup where the workloads running on
>>> the rest of the system are latency-sensitive, and measure the impact of
>>> CPU offline on them, with this patchset applied. That would tell us how
>>> far is this useful in making CPU hotplug less disruptive on the system.
>>>
>>> Of course, it would be nice to also see whether we observe any reduction
>>> in hotplug duration itself (point 1 above) on ARM platforms with lot
>>> of CPUs. [This could potentially speed up suspend/resume, which is used
>>> rather heavily on ARM platforms].
>>>
>>> The benefits from this patchset over mainline (both in terms of points
>>> 1 and 2 above) is expected to increase, with increasing number of CPUs in
>>> the system.
>>>
>>
>> Adding Vincent to CC, who had previously evaluated the performance and
>> latency implications of CPU hotplug on ARM platforms, IIRC.
>>
>
> Hi Srivatsa,
> 
> I can try to run some of our stress tests on your patches.

Great!

> Have you
> got a git tree that i can pull ?
> 

Unfortunately, no, none at the moment..  :-(

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Paul E. McKenney]

On Mon, Feb 11, 2013 at 05:53:41PM +0530, Srivatsa S. Bhat wrote:
> On 02/11/2013 05:28 PM, Vincent Guittot wrote:
> > On 8 February 2013 19:09, Srivatsa S. Bhat
> > <srivatsa.bhat@linux.vnet.ibm.com> wrote:

[ . . . ]

> >> Adding Vincent to CC, who had previously evaluated the performance and
> >> latency implications of CPU hotplug on ARM platforms, IIRC.
> >>
> >
> > Hi Srivatsa,
> > 
> > I can try to run some of our stress tests on your patches.
> 
> Great!
> 
> > Have you
> > got a git tree that i can pull ?
> > 
> 
> Unfortunately, no, none at the moment..  :-(

You do need to create an externally visible git tree.  In the meantime,
I have added your series at rcu/bhat.2013.01.21a on -rcu:

git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git

This should appear soon on a kernel.org mirror near you.  ;-)

							Thanx, Paul

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Srivatsa S. Bhat]

On 02/12/2013 12:38 AM, Paul E. McKenney wrote:
> On Mon, Feb 11, 2013 at 05:53:41PM +0530, Srivatsa S. Bhat wrote:
>> On 02/11/2013 05:28 PM, Vincent Guittot wrote:
>>> On 8 February 2013 19:09, Srivatsa S. Bhat
>>> <srivatsa.bhat@linux.vnet.ibm.com> wrote:
> 
> [ . . . ]
> 
>>>> Adding Vincent to CC, who had previously evaluated the performance and
>>>> latency implications of CPU hotplug on ARM platforms, IIRC.
>>>>
>>>
>>> Hi Srivatsa,
>>>
>>> I can try to run some of our stress tests on your patches.
>>
>> Great!
>>
>>> Have you
>>> got a git tree that i can pull ?
>>>
>>
>> Unfortunately, no, none at the moment..  :-(
> 
> You do need to create an externally visible git tree.

Ok, I'll do that soon.

>  In the meantime,
> I have added your series at rcu/bhat.2013.01.21a on -rcu:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
> 
> This should appear soon on a kernel.org mirror near you.  ;-)
>

Thank you very much, Paul! :-)

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Vincent Guittot]

[-- Attachment #1: Type: text/plain, Size: 1764 bytes --]

Hi Srivatsa,

I have run some tests with you branch (thanks Paul for the git tree)
and you will find results below.

The tests condition are:
- 5 CPUs system in 2 clusters
- The test plugs/unplugs CPU2 and it increases the system load each 20
plug/unplug sequence with either more cyclictests threads
- The test is done with all CPUs online and with only CPU0 and CPU2

The main conclusion is that there is no differences with and without
your patches with my stress tests. I'm not sure that it was the
expected results but the cpu_down is already quite low : 4-5ms in
average

I have attached the raw data to the mail

Regards,
Vincent


On 12 February 2013 04:58, Srivatsa S. Bhat
<srivatsa.bhat@linux.vnet.ibm.com> wrote:
> On 02/12/2013 12:38 AM, Paul E. McKenney wrote:
>> On Mon, Feb 11, 2013 at 05:53:41PM +0530, Srivatsa S. Bhat wrote:
>>> On 02/11/2013 05:28 PM, Vincent Guittot wrote:
>>>> On 8 February 2013 19:09, Srivatsa S. Bhat
>>>> <srivatsa.bhat@linux.vnet.ibm.com> wrote:
>>
>> [ . . . ]
>>
>>>>> Adding Vincent to CC, who had previously evaluated the performance and
>>>>> latency implications of CPU hotplug on ARM platforms, IIRC.
>>>>>
>>>>
>>>> Hi Srivatsa,
>>>>
>>>> I can try to run some of our stress tests on your patches.
>>>
>>> Great!
>>>
>>>> Have you
>>>> got a git tree that i can pull ?
>>>>
>>>
>>> Unfortunately, no, none at the moment..  :-(
>>
>> You do need to create an externally visible git tree.
>
> Ok, I'll do that soon.
>
>>  In the meantime,
>> I have added your series at rcu/bhat.2013.01.21a on -rcu:
>>
>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
>>
>> This should appear soon on a kernel.org mirror near you.  ;-)
>>
>
> Thank you very much, Paul! :-)
>
> Regards,
> Srivatsa S. Bhat
>

[-- Attachment #2: test_load_5cpu_cyc_patched.txt --]
[-- Type: text/plain, Size: 70401 bytes --]

# tracer: function
#
# entries-in-buffer/entries-written: 880/880   #P:5
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
            bash-3457  [003] ....   111.373223: cpu_maps_update_begin <-cpu_down
            bash-3457  [003] ....   111.375965: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   112.389677: cpu_maps_update_begin <-cpu_up
            bash-3457  [004] ....   112.400081: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   114.424086: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   114.429025: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   115.439149: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   115.442572: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   117.461346: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   117.464135: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   118.475251: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   118.478228: cpu_maps_update_done <-cpu_up
            bash-3457  [004] ....   120.500161: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   120.504009: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   121.514516: cpu_maps_update_begin <-cpu_up
            bash-3457  [002] ....   121.517917: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   123.540914: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   123.543928: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   124.551646: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   124.554024: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   126.570624: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   126.573892: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   127.580496: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   127.583573: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   129.607839: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   129.611232: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   130.621265: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   130.623749: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   132.645309: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   132.656170: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   133.672326: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   133.678641: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   135.705470: cpu_maps_update_begin <-cpu_down
            bash-3457  [004] ....   135.709934: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   136.726657: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   136.732990: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   138.756512: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   138.759801: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   139.768289: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   139.771268: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   141.789552: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   141.792483: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   142.800886: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   142.803872: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   144.826592: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   144.829995: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   145.837351: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   145.840251: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   147.856430: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   147.861368: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   148.868134: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   148.871185: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   150.888007: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   150.890844: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   151.900297: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   151.902763: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   153.938258: cpu_maps_update_begin <-cpu_down
            bash-3457  [004] ....   153.943839: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   154.951721: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   154.954765: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   156.978737: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   156.981960: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   157.990111: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   157.992774: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   160.020775: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   160.024208: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   161.032041: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   161.034598: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   163.057241: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   163.060702: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   164.076951: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   164.080013: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   166.104979: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   166.108390: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   167.126996: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   167.129889: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   169.149223: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   169.152207: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   170.161147: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   170.164135: cpu_maps_update_done <-cpu_up
            bash-3457  [004] ....   172.367976: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   172.371556: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   173.378765: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   173.381104: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   175.397247: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   175.400774: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   176.408756: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   176.411817: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   178.431253: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   178.434427: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   179.444620: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   179.447555: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   181.474252: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   181.477569: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   182.493633: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   182.496844: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   184.523804: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   184.527240: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   185.535777: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   185.538953: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   187.559022: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   187.562369: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   188.578964: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   188.582021: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   190.602204: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   190.605491: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   191.624800: cpu_maps_update_begin <-cpu_up
            bash-3457  [002] ....   191.628278: cpu_maps_update_done <-cpu_up
            bash-3457  [004] ....   193.653565: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   193.657081: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   194.668657: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   194.674868: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   196.703848: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   196.707128: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   197.727124: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   197.730198: cpu_maps_update_done <-cpu_up
            bash-3457  [004] ....   199.754011: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   199.757428: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   200.768256: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   200.771467: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   202.798368: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   202.801846: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   203.818040: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   203.821292: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   205.843474: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   205.846776: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   206.871398: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   206.874954: cpu_maps_update_done <-cpu_up
            bash-3457  [004] ....   208.893176: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   208.895905: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   209.904317: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   209.906893: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   211.932694: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   211.935664: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   212.943456: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   212.946130: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   214.973682: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   214.977025: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   215.996893: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   216.000000: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   218.026550: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   218.030218: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   219.038032: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   219.040688: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   221.062053: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   221.065220: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   222.073403: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   222.075805: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   224.093216: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   224.097321: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   225.106637: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   225.109486: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   227.126549: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   227.131340: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   228.139517: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   228.142148: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   230.162121: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   230.165335: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   231.185261: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   231.188314: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   233.400634: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   233.404250: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   234.416540: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   234.419762: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   236.442320: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   236.445797: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   237.464758: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   237.468288: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   239.487335: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   239.491119: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   240.500745: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   240.503109: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   242.530746: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   242.534402: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   243.542575: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   243.545102: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   245.579018: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   245.582232: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   246.591081: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   246.593658: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   248.611092: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   248.614163: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   249.624877: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   249.628147: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   251.650368: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   251.654001: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   252.662600: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   252.665738: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   254.684046: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   254.687812: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   255.697046: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   255.700222: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   257.726606: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   257.730955: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   258.740695: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   258.743299: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   260.773966: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   260.777347: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   261.787813: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   261.790976: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   263.808754: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   263.812554: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   264.821548: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   264.824582: cpu_maps_update_done <-cpu_up
            bash-3457  [004] ....   266.847789: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   266.851764: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   267.859815: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   267.862282: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   269.884596: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   269.888070: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   270.897136: cpu_maps_update_begin <-cpu_up
            bash-3457  [004] ....   270.903204: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   272.927688: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   272.931462: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   273.949079: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   273.952213: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   275.972335: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   275.975825: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   276.984809: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   276.987805: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   279.004879: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   279.008673: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   280.021400: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   280.024586: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   282.043601: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   282.046825: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   283.055412: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   283.058367: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   285.079288: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   285.082954: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   286.090982: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   286.094095: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   288.113049: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   288.116578: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   289.124775: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   289.127358: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   291.154235: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   291.157241: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   292.166526: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   292.169665: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   294.324802: cpu_maps_update_begin <-cpu_down
            bash-3457  [003] ....   294.328760: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   295.338084: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   295.344339: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   297.373308: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   297.377199: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   298.386368: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   298.389550: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   300.414912: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] .N..   300.428243: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   301.439189: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   301.442590: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   303.462001: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   303.466231: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   304.475678: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   304.479023: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   306.504775: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   306.510636: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   307.519189: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   307.522752: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   309.543588: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   309.547735: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   310.557582: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   310.560833: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   312.584419: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   312.587836: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   313.597862: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   313.601449: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   315.623872: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   315.627584: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   316.654398: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   316.658271: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   318.682023: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   318.685777: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   319.695421: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   319.698621: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   321.725272: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   321.731461: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   322.742265: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   322.745631: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   324.769311: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   324.773186: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   325.783504: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   325.786758: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   327.809082: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   327.812707: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   328.824230: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   328.826796: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   330.848710: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   330.852573: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   331.863408: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   331.866032: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   333.895349: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   333.901583: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   334.910832: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   334.913986: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   336.938486: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   336.941783: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   337.951272: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   337.954438: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   339.978693: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   339.982881: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   340.992437: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   340.995804: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   343.020723: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   343.024499: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   344.034481: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   344.037264: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   346.060806: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   346.064924: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   347.088215: cpu_maps_update_begin <-cpu_up
            bash-3457  [004] ....   347.095341: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   349.116923: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   349.120213: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   350.131758: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   350.134402: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   352.161865: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   352.165963: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   353.178772: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   353.182014: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   355.360296: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   355.365098: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   356.378682: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   356.382488: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   358.404808: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   358.409544: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   359.420297: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   359.423564: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   361.449633: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   361.453336: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   362.471060: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   362.474842: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   364.503653: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   364.507413: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   365.520265: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   365.522737: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   367.541867: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   367.545684: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   368.555490: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   368.558807: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   370.577183: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   370.580575: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   371.608933: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   371.612525: cpu_maps_update_done <-cpu_up
            bash-3457  [004] ....   373.636792: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   373.641322: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   374.650234: cpu_maps_update_begin <-cpu_up
            bash-3457  [004] ....   374.653157: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   376.684421: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   376.688745: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   377.699472: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   377.702870: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   379.721029: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   379.726513: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   380.737772: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   380.741792: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   382.769514: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   382.773878: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   383.785502: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   383.788285: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   385.810116: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   385.813830: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   386.823367: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   386.826025: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   388.849599: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   388.854222: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   389.864381: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   389.867251: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   391.891659: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   391.895533: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   392.907039: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   392.909839: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   394.932024: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   394.935857: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   395.947073: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   395.949808: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   397.977268: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   397.980405: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   399.001994: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   399.005099: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   401.030925: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   401.034987: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   402.044792: cpu_maps_update_begin <-cpu_up
            bash-3457  [002] ....   402.048447: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   404.067969: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   404.071552: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   405.082879: cpu_maps_update_begin <-cpu_up
            bash-3457  [004] ....   405.085883: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   407.112269: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   407.117155: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   408.149997: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   408.153787: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   410.180293: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] .N..   410.184830: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   411.195536: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   411.199033: cpu_maps_update_done <-cpu_up
            bash-3457  [004] ....   413.226470: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   413.231277: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   414.241060: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   414.244570: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   416.428142: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   416.432667: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   417.444767: cpu_maps_update_begin <-cpu_up
            bash-3457  [002] ....   417.448443: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   419.478585: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   419.482401: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   420.495310: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   420.498744: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   422.525198: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   422.530249: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   423.542369: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   423.545741: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   425.575888: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   425.581176: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   426.592546: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   426.595934: cpu_maps_update_done <-cpu_up
            bash-3457  [004] ....   428.626165: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   428.630732: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   429.641634: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   429.644994: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   431.670868: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   431.675039: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   432.686296: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   432.689538: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   434.721343: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   434.725784: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   435.736625: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   435.740061: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   437.766640: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   437.770771: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   438.783137: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   438.786509: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   440.812617: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   440.816739: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   441.827964: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   441.831358: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   443.862950: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   443.866777: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   444.880010: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   444.882802: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   446.904282: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   446.908862: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   447.920757: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   447.924053: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   449.949861: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   449.954200: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   450.964422: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] .N..   450.967850: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   452.998868: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   453.003367: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   454.019026: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   454.022978: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   456.052024: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   456.057048: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   457.070427: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   457.075515: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   459.100722: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   459.104710: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   460.125126: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   460.129416: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   462.149327: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   462.154259: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   463.184367: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   463.188316: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   465.215288: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   465.220260: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   466.233334: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   466.236181: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   468.263330: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   468.267295: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   469.281083: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   469.284317: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   471.316334: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   471.320550: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   472.331534: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   472.334899: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   474.358854: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   474.363060: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   475.373514: cpu_maps_update_begin <-cpu_up
            bash-3457  [004] ....   475.378642: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   477.636401: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] .N..   477.641294: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   478.651704: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   478.655638: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   480.679056: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   480.683965: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   481.696735: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   481.701400: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   483.733245: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   483.738916: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   484.753549: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   484.757148: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   486.784651: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   486.789709: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   487.809715: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   487.813840: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   489.847352: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   489.852097: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   490.866450: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   490.869989: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   492.897700: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   492.903109: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   493.956171: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   493.963239: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   496.010015: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   496.014630: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   497.028101: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   497.031741: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   499.057883: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   499.062104: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   500.074095: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] .N..   500.079489: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   502.109017: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   502.113991: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   503.125996: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] .N..   503.130519: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   505.155440: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   505.160276: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   506.172326: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   506.177420: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   508.205764: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   508.210324: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   509.224078: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   509.230700: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   511.259148: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   511.263843: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   512.279871: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   512.284370: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   514.314066: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   514.319230: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   515.332794: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   515.336128: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   517.361857: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] .N..   517.366945: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   518.381963: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   518.386413: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   520.417495: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   520.422196: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   521.435955: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   521.439785: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   523.466862: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   523.471053: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   524.483366: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   524.487056: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   526.514171: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   526.518955: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   527.532983: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   527.536473: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   529.562836: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   529.567913: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   530.584358: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   530.587987: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   532.615743: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   532.620462: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   533.634569: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   533.638010: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   535.659612: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   535.663934: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   536.678510: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   536.683076: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   538.885631: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   538.901776: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   539.925451: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   539.930696: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   541.960201: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   541.964365: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   542.979530: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] .N..   542.984158: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   545.013384: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   545.018450: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   546.042979: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   546.048414: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   548.083769: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   548.089632: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   549.104690: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] .N..   549.109672: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   551.142609: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   551.151124: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   552.163330: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   552.167324: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   554.198293: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   554.203724: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   555.220488: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   555.224332: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   557.259925: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   557.263914: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   558.281901: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   558.285324: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   560.319072: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   560.323909: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   561.412001: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   561.416374: cpu_maps_update_done <-cpu_up
            bash-3457  [004] ....   563.455139: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] .N..   563.459935: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   564.476016: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   564.479998: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   566.509692: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   566.514283: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   567.528906: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] .N..   567.533787: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   569.567476: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   569.572422: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   570.587672: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   570.591471: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   572.619736: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   572.624266: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   573.644115: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   573.648175: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   575.684758: cpu_maps_update_begin <-cpu_down
            bash-3457  [003] .N..   575.689569: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   576.709204: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   576.714713: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   578.744939: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   578.753253: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   579.773928: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   579.778073: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   581.807273: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   581.813598: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   582.831399: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   582.835259: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   584.867938: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   584.873163: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   585.888883: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] .N..   585.893796: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   587.922934: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   587.929480: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   588.967778: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   588.971946: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   591.002838: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   591.006794: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   592.023673: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   592.027191: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   594.063606: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   594.072462: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   595.087581: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   595.092452: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   597.121644: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] .N..   597.127112: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   598.141132: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   598.145632: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   600.378536: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   600.385114: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   601.411874: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   601.416664: cpu_maps_update_done <-cpu_up
            bash-3457  [004] ....   603.447827: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   603.454229: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   604.467828: cpu_maps_update_begin <-cpu_up
            bash-3457  [002] ....   604.472473: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   606.501729: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   606.507478: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   607.528005: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] .N..   607.532622: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   609.561420: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] .N..   609.566307: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   610.586706: cpu_maps_update_begin <-cpu_up
            bash-3457  [004] ....   610.591551: cpu_maps_update_done <-cpu_up
            bash-3457  [004] ....   612.624309: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] .N..   612.631061: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   613.653440: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] .N..   613.658368: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   615.697086: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   615.702819: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   616.717924: cpu_maps_update_begin <-cpu_up
            bash-3457  [004] ....   616.723307: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   618.754454: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   618.767264: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   619.784476: cpu_maps_update_begin <-cpu_up
            bash-3457  [002] .N..   619.791325: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   621.815010: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   621.821959: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   622.837122: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   622.840626: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   624.867771: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   624.872747: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   625.888290: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] .N..   625.893324: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   627.925889: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] .N..   627.934670: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   628.957764: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] .N..   628.962630: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   630.989952: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   630.994253: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   632.007154: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   632.011876: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   634.033897: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   634.040446: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   635.056248: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   635.060307: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   637.094146: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   637.099997: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   638.115326: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] .N..   638.119588: cpu_maps_update_done <-cpu_up
            bash-3457  [004] ....   640.148130: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   640.152760: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   641.165439: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   641.172327: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   643.199571: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   643.203487: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   644.221912: cpu_maps_update_begin <-cpu_up
            bash-3457  [004] ....   644.226699: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   646.261411: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   646.265578: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   647.279187: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   647.284268: cpu_maps_update_done <-cpu_up
            bash-3457  [004] ....   649.313942: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] .N..   649.320365: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   650.336152: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   650.340084: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   652.367053: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   652.372236: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   653.391898: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   653.395822: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   655.430674: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   655.434591: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   656.454425: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   656.457934: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   658.484758: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] .N..   658.489242: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   659.504148: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   659.508341: cpu_maps_update_done <-cpu_up
            bash-3457  [004] ....   661.852330: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   661.861813: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   662.874062: cpu_maps_update_begin <-cpu_up
            bash-3457  [002] ....   662.882941: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   664.917366: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   664.923328: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   665.941143: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   665.945385: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   667.979362: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   667.984365: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   669.006295: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   669.010704: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   671.037624: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   671.041681: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   672.062433: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   672.066492: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   674.092637: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   674.097177: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   675.118122: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   675.121311: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   677.155262: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   677.159066: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   678.183031: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] .N..   678.186165: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   680.219529: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   680.223206: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   681.244471: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] .N..   681.248983: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   683.280102: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   683.284628: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   684.305344: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   684.310200: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   686.340627: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   686.344900: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   687.367323: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   687.370902: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   689.401505: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   689.407183: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   690.425354: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   690.429994: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   692.465114: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   692.472272: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   693.488432: cpu_maps_update_begin <-cpu_up
            bash-3457  [002] ....   693.492774: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   695.525160: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   695.528850: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   696.555884: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   696.559393: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   698.599518: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   698.605028: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   699.626630: cpu_maps_update_begin <-cpu_up
            bash-3457  [004] ....   699.631123: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   701.661491: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   701.665597: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   702.680910: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   702.684264: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   704.708028: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] .N..   704.711975: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   705.731832: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   705.735257: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   707.769044: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] .N..   707.772600: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   708.798959: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   708.802359: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   710.830744: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   710.834875: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   711.857738: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] .N..   711.861263: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   713.890926: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   713.896973: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   714.918760: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   714.923397: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   716.956437: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] .N..   716.963631: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   717.982784: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   717.987587: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   720.016393: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   720.020919: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   721.047742: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   721.051638: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   724.625061: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   724.636103: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   725.648988: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   725.656011: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   727.688788: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   727.693097: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   728.715334: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   728.719400: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   730.756029: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   730.763243: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   731.786591: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] .N..   731.791935: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   733.832712: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   733.839778: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   734.857343: cpu_maps_update_begin <-cpu_up
            bash-3457  [004] ....   734.862412: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   736.900007: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   736.906240: cpu_maps_update_done <-cpu_down
            bash-3457  [000] ....   737.932243: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   737.937409: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   739.981068: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   739.987606: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   741.008066: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   741.013659: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   743.061805: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   743.067230: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   744.100448: cpu_maps_update_begin <-cpu_up
            bash-3457  [002] ....   744.105299: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   746.141275: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   746.149712: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   747.167083: cpu_maps_update_begin <-cpu_up
            bash-3457  [002] ....   747.172403: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   749.212673: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   749.217448: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   750.234787: cpu_maps_update_begin <-cpu_up
            bash-3457  [003] ....   750.240044: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   752.266570: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   752.273151: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   753.297453: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] .N..   753.301765: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   755.335178: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   755.345617: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   756.360642: cpu_maps_update_begin <-cpu_up
            bash-3457  [002] ....   756.366350: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   758.401234: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   758.406489: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   759.426760: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   759.431933: cpu_maps_update_done <-cpu_up
            bash-3457  [002] ....   761.465079: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   761.472408: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   762.507344: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   762.511889: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   764.548024: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] .N..   764.553443: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   765.578859: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   765.583097: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   767.622021: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] .N..   767.625883: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   768.651567: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   768.655529: cpu_maps_update_done <-cpu_up
            bash-3457  [003] ....   770.690144: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] .N..   770.695791: cpu_maps_update_done <-cpu_down
            bash-3457  [001] ....   771.718498: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   771.722666: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   773.757711: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   773.764528: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   774.783655: cpu_maps_update_begin <-cpu_up
            bash-3457  [001] ....   774.788786: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   776.826457: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   776.834262: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   777.852445: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] ....   777.857247: cpu_maps_update_done <-cpu_up
            bash-3457  [000] ....   779.898926: cpu_maps_update_begin <-cpu_down
            bash-3457  [000] ....   779.904699: cpu_maps_update_done <-cpu_down
            bash-3457  [004] ....   780.924805: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] .N..   780.929819: cpu_maps_update_done <-cpu_up
            bash-3457  [001] ....   782.970791: cpu_maps_update_begin <-cpu_down
            bash-3457  [001] ....   782.975158: cpu_maps_update_done <-cpu_down
            bash-3457  [003] ....   784.004145: cpu_maps_update_begin <-cpu_up
            bash-3457  [000] .N..   784.008087: cpu_maps_update_done <-cpu_up

[-- Attachment #3: test_load_1cpu_cyc_patched.txt --]
[-- Type: text/plain, Size: 70401 bytes --]

# tracer: function
#
# entries-in-buffer/entries-written: 880/880   #P:2
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
            bash-6589  [002] ....  1145.644445: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1145.649403: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1146.666646: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1146.670469: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1148.690234: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1148.693662: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1149.710876: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1149.715002: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1151.741964: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1151.746386: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1152.763800: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1152.775161: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1154.808799: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1154.811922: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1155.829255: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1155.833549: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1157.870305: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1157.874211: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1158.891075: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1158.894907: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1160.930028: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1160.933255: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1161.974907: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1161.979648: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1164.008994: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1164.012002: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1165.029662: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1165.033448: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1167.065632: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1167.070026: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1168.087136: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1168.091129: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1170.124625: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1170.128608: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1171.144886: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1171.149575: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1173.191910: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1173.194741: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1174.211569: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1174.215397: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1176.250473: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1176.253885: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1177.271057: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1177.274772: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1179.308237: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1179.311519: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1180.328753: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1180.332431: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1182.361657: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1182.364810: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1183.382364: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1183.386096: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1185.421469: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1185.424717: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1186.441703: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1186.445482: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1188.480837: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1188.484031: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1189.501404: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1189.505143: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1191.529902: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1191.534153: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1192.551530: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1192.555421: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1194.591740: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1194.594929: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1195.611941: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1195.615604: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1197.640541: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1197.645029: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1198.661977: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1198.665792: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1200.702829: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1200.706472: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1201.723400: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1201.727123: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1203.764530: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1203.768583: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1204.784897: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1204.788927: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1207.044270: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1207.049552: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1208.074781: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1208.079808: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1210.104966: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1210.109360: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1211.135756: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1211.141334: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1213.170921: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1213.175057: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1214.200626: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1214.205109: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1216.257246: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1216.261794: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1217.287743: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1217.292488: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1219.340263: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1219.344921: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1220.370967: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1220.376064: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1222.411281: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1222.416800: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1223.443196: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1223.458005: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1225.487427: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1225.491775: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1226.517577: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1226.522364: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1228.549388: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1228.554444: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1229.580886: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1229.585476: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1231.612986: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1231.618894: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1232.644484: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1232.649720: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1234.679040: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1234.683590: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1235.710304: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1235.715829: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1237.766109: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1237.771242: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1238.796580: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1238.801449: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1240.828176: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1240.833874: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1241.860107: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1241.865260: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1243.906331: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1243.911650: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1244.937706: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1244.942453: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1246.973430: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1246.978922: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1248.004164: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1248.009154: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1250.059512: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1250.063670: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1251.090005: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1251.095183: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1253.154414: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1253.159664: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1254.185065: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1254.190034: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1256.209852: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1256.213473: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1257.240150: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1257.244745: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1259.279770: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1259.284173: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1260.309972: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1260.314769: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1262.356941: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1262.361586: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1263.388520: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1263.393784: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1265.422242: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1265.426378: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1266.452399: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1266.457615: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1268.851250: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1268.856151: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1269.906757: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1269.912758: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1271.955646: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1271.961668: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1273.014423: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1273.021127: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1275.071666: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1275.079251: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1276.128664: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1276.135427: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1278.177135: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1278.182363: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1279.233921: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1279.240321: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1281.265616: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1281.271251: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1282.330752: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1282.337228: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1284.386975: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1284.392423: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1285.446104: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1285.452381: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1287.509728: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1287.514637: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1288.564433: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1288.571339: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1290.625528: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1290.630554: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1291.684818: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1291.691056: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1293.721018: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1293.726540: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1294.779340: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1294.785805: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1296.836464: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1296.841713: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1297.890842: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1297.897252: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1299.948744: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1299.954441: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1301.005014: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1301.010836: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1303.057201: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1303.063425: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1304.114791: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1304.121188: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1306.161914: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1306.169404: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1307.218926: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1307.224483: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1309.273715: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1309.279481: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1310.329596: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1310.335834: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1312.389577: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1312.399631: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1313.449016: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1313.455260: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1315.507162: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1315.512247: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1316.564900: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1316.571053: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1318.614027: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1318.619398: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1319.670056: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1319.679452: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1321.717468: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1321.724021: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1322.775225: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1322.781399: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1324.855370: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1324.861291: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1325.913141: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1325.919891: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1327.959883: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1327.965170: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1329.018322: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1329.025258: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1331.388145: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1331.394715: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1333.208878: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1333.221124: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1335.261584: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1335.270260: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1337.037600: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1337.044342: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1339.105713: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1339.113741: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1340.872074: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1340.885057: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1342.941771: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1342.949882: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1344.698952: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1344.708619: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1346.760890: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1346.784006: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1348.724038: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1348.754953: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1350.798367: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1350.804104: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1352.553842: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1352.565009: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1354.609528: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1354.616210: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1356.399278: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1356.425247: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1358.473343: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1358.479289: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1360.222093: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1360.239489: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1362.276136: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1362.285634: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1364.032075: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1364.055499: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1366.116877: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1366.122553: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1367.902091: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1367.923952: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1369.961883: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1369.970938: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1371.744816: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1371.764873: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1373.810971: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1373.827777: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1375.581845: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1375.605144: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1377.660056: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1377.667483: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1379.403951: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1379.426855: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1381.470569: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1381.482016: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1383.291095: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] .N..  1383.314655: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1385.351872: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1385.360516: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1387.119651: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1387.142063: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1389.169304: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1389.175487: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1390.939426: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1391.039183: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1393.065933: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1393.072366: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1394.833625: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1394.863596: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1396.920054: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1396.927559: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1398.682037: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1398.705084: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1400.745248: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1400.755599: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1402.527886: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1402.545280: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1404.605461: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1404.612052: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1406.369692: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1406.384948: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1408.694530: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1408.769405: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1411.027387: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1411.164290: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1413.218345: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1413.365488: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1415.464557: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1415.602952: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1417.650476: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1417.664435: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1419.971249: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1420.109334: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1422.178531: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1422.216942: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1424.498700: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1424.634406: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1426.675524: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1426.782884: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1428.973963: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1429.110585: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1431.167416: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1431.318415: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1433.576729: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1433.716170: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1435.785223: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1435.820510: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1438.082245: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1438.219544: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1440.254312: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1440.262752: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1442.552548: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1442.690859: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1444.720582: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1444.734315: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1447.094345: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1447.231955: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1449.283515: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1449.292796: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1451.543295: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1451.681031: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1453.727573: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1453.767012: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1456.040258: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1456.179499: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1458.248445: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1458.412721: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1460.399030: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1460.536562: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1462.615509: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1462.626847: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1464.950384: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1465.086810: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1467.152771: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1467.176938: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1469.546352: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1469.686774: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1471.750695: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1471.767013: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1474.070842: cpu_maps_update_begin <-cpu_up
            bash-6589  [002] ....  1474.207148: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1476.265430: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1476.297008: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1478.585499: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1478.723621: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1480.788869: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1480.800913: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1483.058706: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1483.196683: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1485.268081: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1485.276685: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1487.601786: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1487.740084: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1489.778096: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1489.794911: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1492.111619: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1492.249772: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1494.290854: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1494.302963: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1496.614120: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1496.753507: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1499.304544: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1499.346674: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1501.770660: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1501.940053: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1504.033705: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1504.126840: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1506.467809: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1506.637070: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1508.749829: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1508.766850: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1511.364900: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1511.532687: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1513.595511: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1513.606772: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1515.978430: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1516.146533: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1518.207961: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1518.386303: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1520.800661: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1520.968928: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1523.026669: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1523.174019: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1525.601801: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1525.769646: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1527.835783: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1527.846669: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1530.266163: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1530.434259: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1532.515929: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1532.524825: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1535.024267: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1535.192808: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1537.267781: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1537.453752: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1539.751234: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1539.919809: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1541.972265: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1542.151899: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1544.583742: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1544.752993: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1546.815938: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1546.972780: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1549.343419: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1549.511272: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1551.563509: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1551.616893: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1554.008186: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1554.176388: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1556.230580: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1556.388934: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1558.798512: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1558.966821: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1561.018160: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1561.026538: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1563.499068: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1563.667759: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1565.772662: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1565.816672: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1568.250605: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1568.420250: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1570.475520: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1570.484715: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1572.966276: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1573.134392: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1575.214454: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1575.278837: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1577.731499: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1577.900082: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1579.990485: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1580.004360: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1582.476349: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1582.644510: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1584.743590: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1584.907805: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1587.288085: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1587.455477: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1589.541622: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1589.576623: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1591.926307: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1592.095228: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1594.680158: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1594.804785: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1597.276639: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1597.469587: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1599.613648: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1599.685506: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1602.140236: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1602.333419: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1604.490135: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1604.678455: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1607.321299: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1607.518146: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1609.726103: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1609.908855: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1612.393726: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1612.586506: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1614.834307: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1614.910394: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1617.471964: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] .N..  1617.664647: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1619.894166: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1620.071602: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1622.620902: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1622.814911: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1624.952917: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1625.052890: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1627.653803: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1627.847061: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1630.010649: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1630.022692: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1632.682493: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1632.875806: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1635.127866: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1635.242728: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1637.736548: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1637.930289: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1640.145976: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1640.172944: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1642.869139: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1643.062271: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1645.386300: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1645.586797: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1648.155358: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1648.349263: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1650.531562: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1650.703850: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1653.272136: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1653.465203: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1655.749870: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1655.866469: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1658.472643: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1658.666529: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1660.862693: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1661.007358: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1663.486477: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1663.680262: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1665.822174: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1666.032616: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1668.627241: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1668.820765: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1670.996819: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1671.070402: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1673.646244: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1673.839216: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1676.043182: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1676.182004: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1678.816476: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1679.009929: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1681.206000: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1681.356641: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1683.778875: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1683.971769: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1686.140023: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1686.267889: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1688.842123: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1689.033986: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1691.260683: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1691.380412: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1694.015049: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1694.207747: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1698.138150: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1698.163840: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1700.965599: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1701.184563: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1703.815498: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1703.825215: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1706.518592: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1706.738158: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1709.346911: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1709.362723: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1712.227847: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1712.447012: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1714.780609: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1714.786863: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1717.521981: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1717.740698: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1720.063885: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1720.071363: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1722.886826: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1723.105936: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1725.565230: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1725.572021: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1728.397950: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1728.616914: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1730.971418: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1730.980797: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1733.868476: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1734.087599: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1737.166395: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1737.173609: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1739.867625: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1740.086334: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1742.295161: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1742.303096: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1745.124906: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1745.344356: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1748.001180: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1748.007446: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1750.914729: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1751.134080: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1753.864367: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1753.881049: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1756.735974: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1756.955407: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1760.208643: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1760.217392: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1762.865778: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1763.083985: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1765.544251: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1765.551756: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1768.186777: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1768.407687: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1771.242439: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1771.252493: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1773.950710: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1774.169366: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1776.693146: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1776.707356: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1779.587787: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1779.806381: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1782.274325: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1782.283375: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1785.128705: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1785.348826: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1787.681581: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1787.692974: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1790.301957: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1790.520393: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1793.062653: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1793.070273: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1795.821067: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1796.041855: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1798.846725: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1798.858332: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1801.531270: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1801.753176: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1804.064681: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1804.072487: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1806.849978: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1807.069246: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1812.137316: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1812.143137: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1814.993067: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1815.237353: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1818.242462: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1818.247475: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1821.080793: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1821.325022: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1824.814163: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1824.821251: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1827.466691: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1827.711773: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1830.674096: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1830.683484: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1833.496527: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1833.741447: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1836.803251: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1836.810037: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1839.649195: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1839.893446: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1843.561792: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1843.569797: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1846.556333: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1846.801069: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1849.876534: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1849.883145: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1852.834664: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1853.078700: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1855.959973: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1855.965111: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1858.599975: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1858.846926: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1862.289439: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1862.296794: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1865.281591: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1865.525363: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1868.647534: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1868.653074: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1871.462620: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1871.713933: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1874.753523: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1874.759274: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1877.669818: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1877.916495: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1881.586943: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1881.593221: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1884.353131: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1884.597933: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1887.594396: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1887.600721: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1890.538484: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1890.783012: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1894.364132: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1894.370516: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1897.372257: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1897.617174: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1900.210439: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1900.215661: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1903.052980: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1903.297952: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1906.814087: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1906.820508: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1909.733338: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1909.978621: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1913.500800: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1913.506077: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1916.353147: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1916.598754: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1919.641378: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1919.646940: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1922.557203: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1922.802266: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1925.917029: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1925.922933: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1928.988823: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1929.233514: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1932.275451: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1932.280542: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1934.912652: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1935.157591: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1946.660946: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1946.666216: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1949.688684: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1949.958277: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1952.809324: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  1952.812792: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1955.651280: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1955.921662: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1959.415004: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1959.420045: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1962.370748: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1962.640013: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1965.677886: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1965.681529: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1968.611871: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1968.882358: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1971.848626: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1971.853094: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1974.773506: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1975.043870: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  1978.051065: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1978.056245: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1980.897154: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1981.167058: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1984.381781: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1984.385975: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1987.110914: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1987.380388: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1990.411054: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1990.415481: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1993.137767: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1993.407014: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  1996.581363: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  1996.585253: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  1999.363414: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  1999.633589: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2002.697858: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2002.702059: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2005.607273: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2005.876821: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  2009.761191: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  2009.766264: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2012.660264: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2012.931388: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2016.050591: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2016.054671: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2018.849471: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2019.118733: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2022.115053: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] .N..  2022.119844: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2024.989424: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2025.259602: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2028.291219: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2028.295382: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2031.249464: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2031.519413: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  2034.652328: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2034.657532: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2037.637855: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2037.907863: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2040.793256: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2040.797326: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2043.763941: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2044.034479: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2047.827601: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2047.831660: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2050.846612: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2051.116945: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2054.852328: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2054.856393: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2057.640065: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2057.910563: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2060.954740: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2060.959834: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2063.901198: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2064.171357: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  2067.097367: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2067.102288: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2070.132615: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2070.402267: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  2084.317257: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2084.321709: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2087.147204: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2087.443264: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2091.799611: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2091.803123: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2094.768587: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2095.065278: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2098.609331: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2098.612703: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2101.515478: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2101.811292: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2105.047115: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2105.051527: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2107.704373: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2108.001139: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  2111.958998: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2111.963242: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2114.885983: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2115.181654: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  2118.661945: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2118.666090: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2121.407856: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2121.703196: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  2124.888687: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2124.892792: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2127.497404: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2127.793095: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2131.208304: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2131.211778: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2134.123548: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2134.419654: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  2137.566763: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2137.571211: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2140.187585: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2140.483462: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  2143.951635: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2143.955373: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2146.834181: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2147.130467: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2150.350149: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2150.353415: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2153.335825: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2153.632025: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  2157.034745: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2157.039746: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2159.988365: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2160.284225: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2163.624430: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2163.628969: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2166.224809: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2166.521422: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2169.898621: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2169.902136: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2172.477491: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2172.772057: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  2176.165514: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2176.169178: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2179.205957: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2179.502428: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  2184.445413: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2184.449747: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2187.404479: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2187.701071: cpu_maps_update_done <-cpu_up
            bash-6589  [002] ....  2191.542431: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2191.546327: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2194.117699: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2194.414285: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2197.474220: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2197.477523: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2200.048612: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2200.344346: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2203.826792: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2203.830925: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2206.859112: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2207.155188: cpu_maps_update_done <-cpu_up
            bash-6589  [000] ....  2211.538675: cpu_maps_update_begin <-cpu_down
            bash-6589  [000] ....  2211.542310: cpu_maps_update_done <-cpu_down
            bash-6589  [000] ....  2214.000743: cpu_maps_update_begin <-cpu_up
            bash-6589  [000] ....  2214.296802: cpu_maps_update_done <-cpu_up

[-- Attachment #4: test_load_5cpus_cyc_main.txt --]
[-- Type: text/plain, Size: 70401 bytes --]

# tracer: function
#
# entries-in-buffer/entries-written: 880/880   #P:5
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
            bash-16504 [000] ....  8219.814186: cpu_maps_update_begin <-cpu_down
            bash-16504 [003] ....  8219.818517: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8220.826855: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8220.829808: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8222.848671: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8222.851886: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8223.860448: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8223.863307: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8225.879835: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8225.883007: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8226.892588: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8226.895512: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8228.920598: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8228.923753: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8229.932120: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8229.934417: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8231.950840: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8231.954104: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8232.962370: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8232.965345: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8234.981799: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8234.985122: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8235.992636: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8235.995500: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8238.011901: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8238.014947: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8239.021781: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8239.024778: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8241.046210: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8241.049522: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8242.067152: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8242.073497: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8244.090583: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8244.094019: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8245.101457: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8245.104354: cpu_maps_update_done <-cpu_up
            bash-16504 [004] ....  8247.120366: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8247.122981: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8248.130774: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8248.133974: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8250.155096: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8250.158456: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8251.181874: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] .N..  8251.188572: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8253.209721: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8253.212933: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8254.221614: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8254.224577: cpu_maps_update_done <-cpu_up
            bash-16504 [004] ....  8256.240881: cpu_maps_update_begin <-cpu_down
            bash-16504 [004] ....  8256.243501: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8257.252648: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8257.255224: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8259.271549: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8259.274781: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8260.295155: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8260.298231: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8262.322796: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8262.326004: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8263.345043: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8263.348381: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8265.366175: cpu_maps_update_begin <-cpu_down
            bash-16504 [003] ....  8265.369590: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8266.376679: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8266.382458: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8268.402120: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8268.405225: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8269.415034: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8269.424274: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8271.459606: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8271.465611: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8272.474242: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8272.477350: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8274.503606: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8274.506959: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8275.514669: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8275.517365: cpu_maps_update_done <-cpu_up
            bash-16504 [004] ....  8277.542986: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8277.546153: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8278.553885: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8278.556423: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8280.753559: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8280.766736: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8281.787106: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8281.790149: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8283.810278: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8283.813542: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8284.830765: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8284.839023: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8286.862263: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8286.865716: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8287.875363: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8287.878443: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8289.906797: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8289.909797: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8290.917518: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8290.919836: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8292.942378: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8292.945953: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8293.966691: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8293.969806: cpu_maps_update_done <-cpu_up
            bash-16504 [004] ....  8295.993760: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8295.997415: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8297.005273: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8297.007651: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8299.034646: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8299.038194: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8300.047965: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8300.050626: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8302.070802: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8302.073816: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8303.082258: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8303.084829: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8305.103126: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8305.106454: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8306.114311: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8306.117305: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8308.135477: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8308.143406: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8309.152478: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8309.155486: cpu_maps_update_done <-cpu_up
            bash-16504 [004] ....  8311.183161: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8311.186533: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8312.194194: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8312.196880: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8314.225372: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8314.232424: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8315.251310: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8315.254277: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8317.272928: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8317.276080: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8318.287454: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8318.290435: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8320.312159: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8320.315137: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8321.324193: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8321.327281: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8323.352380: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8323.355686: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8324.365790: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8324.368701: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8326.396663: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8326.400030: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8327.410244: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8327.413172: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8329.445664: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8329.452020: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8330.460356: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8330.463433: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8332.491725: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8332.494838: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8333.503548: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8333.506508: cpu_maps_update_done <-cpu_up
            bash-16504 [004] ....  8335.527074: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8335.530286: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8336.538502: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8336.541134: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8338.560040: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8338.563172: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8339.571322: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8339.573855: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8341.780544: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8341.784307: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8342.807730: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8342.814242: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8344.859508: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8344.863361: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8345.871410: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8345.874119: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8347.902478: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8347.905885: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8348.915007: cpu_maps_update_begin <-cpu_up
            bash-16504 [002] ....  8348.918228: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8350.938566: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8350.942189: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8351.950128: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8351.953602: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8353.982642: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8353.985686: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8354.995454: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8354.998599: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8357.022028: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8357.025119: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8358.034315: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8358.037933: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8360.056144: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8360.059330: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8361.067481: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8361.070511: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8363.095217: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8363.099373: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8364.108889: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8364.111343: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8366.136716: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8366.140276: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8367.148650: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8367.151677: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8369.168704: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8369.172887: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8370.181154: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8370.184635: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8372.203037: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8372.207110: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8373.217858: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8373.220391: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8375.237321: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8375.240810: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8376.260014: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8376.263111: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8378.289955: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8378.293690: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8379.313054: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8379.316201: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8381.345901: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8381.349214: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8382.358730: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8382.361807: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8384.386211: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8384.389689: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8385.397284: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8385.399696: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8387.424790: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8387.428920: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8388.437039: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8388.440112: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8390.465742: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8390.469488: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8391.478638: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8391.481048: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8393.502845: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8393.506424: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8394.531147: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8394.534289: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8396.567669: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8396.571545: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8397.580509: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8397.583589: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8399.612346: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8399.615863: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8400.623882: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8400.626480: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8402.872056: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8402.879944: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8403.892676: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8403.895901: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8405.920447: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8405.925123: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8406.965401: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8406.968926: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8408.994817: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8408.998919: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8410.008164: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8410.011033: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8412.041741: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8412.045462: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8413.055507: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8413.058662: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8415.076871: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8415.080424: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8416.089429: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8416.092018: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8418.115766: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8418.119763: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8419.130142: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8419.133206: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8421.159697: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8421.163504: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8422.172029: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8422.174820: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8424.193326: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8424.197146: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8425.222001: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8425.225364: cpu_maps_update_done <-cpu_up
            bash-16504 [004] ....  8427.251107: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8427.255056: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8428.264113: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8428.266658: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8430.285417: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8430.289600: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8431.298003: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8431.300826: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8433.333016: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8433.336456: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8434.345039: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8434.347936: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8436.380040: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8436.383900: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8437.402830: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8437.406104: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8439.432763: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8439.436734: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8440.445532: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8440.450564: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8442.470976: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8442.474789: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8443.484710: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8443.487411: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8445.510981: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8445.519401: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8446.529800: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8446.533463: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8448.555950: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8448.559848: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8449.571831: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8449.574976: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8451.598589: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8451.602285: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8452.612935: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8452.616158: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8454.642169: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8454.645763: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8455.654617: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8455.657840: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8457.680522: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8457.684178: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8458.696649: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8458.699182: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8460.717075: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8460.721022: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8461.734939: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8461.737614: cpu_maps_update_done <-cpu_up
            bash-16504 [004] ....  8463.984538: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8463.989212: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8464.998785: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8465.002059: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8467.029829: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8467.033843: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8468.044961: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8468.048096: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8470.073639: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8470.077463: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8471.088893: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8471.092084: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8473.117574: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8473.121545: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8474.130566: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8474.133582: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8476.164918: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8476.171768: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8477.180242: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8477.183011: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8479.211584: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8479.215955: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8480.226325: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8480.229603: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8482.256359: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8482.260163: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8483.270333: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8483.273336: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8485.292334: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8485.296079: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8486.309312: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8486.312645: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8488.331166: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8488.335072: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8489.347481: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8489.350922: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8491.374198: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8491.381184: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8492.413490: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8492.417383: cpu_maps_update_done <-cpu_up
            bash-16504 [004] ....  8494.442358: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8494.446445: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8495.457043: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8495.460252: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8497.479398: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8497.482875: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8498.495143: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8498.498625: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8500.520471: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8500.524796: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8501.536273: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8501.539500: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8503.566250: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8503.571125: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8504.596243: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8504.603341: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8506.631035: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8506.635458: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8507.645411: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8507.647992: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8509.666146: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8509.670125: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8510.679867: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8510.682953: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8512.705980: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8512.709670: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8513.720600: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8513.726042: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8515.752012: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8515.755653: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8516.768368: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8516.770994: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8518.795933: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8518.799941: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8519.809200: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8519.812768: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8521.838889: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8521.842986: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8522.853588: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8522.856807: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8525.084749: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8525.092361: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8526.102973: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8526.106224: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8528.133483: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8528.140267: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8529.167192: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8529.170886: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8531.190149: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8531.194012: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8532.204928: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8532.208417: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8534.238736: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8534.243367: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8535.253982: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8535.259370: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8537.288592: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8537.293644: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8538.304244: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8538.307679: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8540.327091: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8540.330989: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8541.340754: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8541.343417: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8543.366693: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8543.370479: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8544.382715: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8544.385889: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8546.410314: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8546.413747: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8547.434975: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8547.438691: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8549.466606: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] .N..  8549.470947: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8550.482357: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8550.485189: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8552.506329: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8552.510620: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8553.522664: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8553.527168: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8555.553116: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8555.557171: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8556.569818: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8556.572488: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8558.602544: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8558.607006: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8559.620495: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8559.623628: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8561.650127: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8561.655870: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8562.753940: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8562.757207: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8564.784832: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8564.794302: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8565.827946: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8565.832020: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8567.852011: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8567.856251: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8568.867493: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8568.870927: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8570.890080: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8570.893525: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8571.906555: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8571.910728: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8573.939010: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8573.942498: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8574.956395: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8574.959803: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8576.982841: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8576.989352: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8578.000285: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8578.003446: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8580.031040: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8580.034534: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8581.054590: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8581.058827: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8583.081918: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8583.085974: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8584.099688: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8584.102821: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8586.301723: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8586.306343: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8587.319782: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8587.323391: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8589.353831: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8589.359207: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8590.370587: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8590.373952: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8592.400054: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] .N..  8592.404420: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8593.418841: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8593.422049: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8595.452587: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] .N..  8595.464839: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8596.475230: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8596.478124: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8598.508473: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8598.513065: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8599.543077: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8599.547242: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8601.574606: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8601.579413: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8602.589899: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8602.593242: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8604.617715: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8604.621202: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8605.635617: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8605.638863: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8607.667639: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8607.672175: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8608.684886: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8608.688375: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8610.717624: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8610.722025: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8611.733942: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8611.737678: cpu_maps_update_done <-cpu_up
            bash-16504 [004] ....  8613.767057: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8613.771599: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8614.782894: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8614.786309: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8616.814471: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8616.821365: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8617.836196: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8617.840344: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8619.866677: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8619.871673: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8620.883959: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8620.887551: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8622.913438: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8622.920141: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8623.932562: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8623.936073: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8625.959606: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8625.964430: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8626.976898: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8626.980221: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8629.001106: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8629.005175: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8630.021735: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8630.024869: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8632.049357: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8632.053493: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8633.067239: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8633.072268: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8635.104543: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8635.111625: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8636.124221: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8636.129049: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8638.156784: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8638.161590: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8639.175112: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8639.178444: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8641.206752: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8641.211249: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8642.223713: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8642.229722: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8644.257525: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8644.262521: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8645.277909: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8645.282024: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8647.519094: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8647.525862: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8648.549887: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8648.554622: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8650.585139: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8650.589408: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8651.601378: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8651.604844: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8653.634744: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8653.640089: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8654.652085: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8654.655515: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8656.680215: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8656.684374: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8657.698956: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8657.702364: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8659.731306: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8659.735974: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8660.755222: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8660.758340: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8662.778533: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8662.783627: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8663.799330: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8663.802224: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8665.822598: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8665.826924: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8666.846573: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8666.850681: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8668.876245: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8668.881562: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8669.895931: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8669.901216: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8671.932646: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8671.937887: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8672.949761: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] .N..  8672.954209: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8674.981422: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8674.985519: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8675.998564: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8676.003239: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8678.031678: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8678.036323: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8679.056252: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] .N..  8679.061612: cpu_maps_update_done <-cpu_up
            bash-16504 [004] ....  8681.094974: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] .N..  8681.100616: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8682.112366: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8682.115353: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8684.144934: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8684.150706: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8685.165495: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] .N..  8685.169392: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8687.193447: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8687.200095: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8688.212434: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8688.219831: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8690.262642: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8690.270251: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8691.283886: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8691.288878: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8693.320397: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8693.325708: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8694.339029: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8694.342600: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8696.373307: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8696.380526: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8697.395267: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8697.399232: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8699.428172: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8699.432459: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8700.448238: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8700.452557: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8702.481685: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8702.486491: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8703.502864: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8703.508143: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8705.540670: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8705.544834: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8706.566886: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8706.572798: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8708.944182: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8708.952750: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8709.964840: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8709.969165: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8712.000737: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8712.005036: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8713.023124: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8713.027114: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8715.055431: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8715.065557: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8716.081081: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8716.086921: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8718.122336: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8718.128478: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8719.142221: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8719.146828: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8721.177323: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8721.182843: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8722.205540: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] .N..  8722.210429: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8724.242296: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8724.248021: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8725.267228: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] .N..  8725.271113: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8727.302140: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8727.306380: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8728.321186: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8728.325058: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8730.359402: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8730.365032: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8731.380655: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8731.385153: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8733.412420: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8733.417542: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8734.432964: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8734.436537: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8736.468589: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8736.473957: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8737.488131: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8737.492393: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8739.522058: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8739.526530: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8740.543018: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8740.547752: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8742.568586: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8742.572938: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8743.594367: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8743.597540: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8745.627420: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8745.631291: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8746.648015: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8746.651640: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8748.676449: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8748.680247: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8749.696374: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8749.700097: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8751.731408: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8751.734764: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8752.755348: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8752.759555: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8754.787201: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8754.791905: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8755.806805: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] .N..  8755.811184: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8757.839203: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8757.844324: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8758.862097: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8758.865957: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8760.896462: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] .N..  8760.901776: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8761.916556: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] .N..  8761.921522: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8763.954158: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8763.957473: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8764.972865: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8764.975896: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8767.005667: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8767.010813: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8768.025503: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8768.028827: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8770.282722: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8770.291718: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8771.312863: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8771.319594: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8773.354519: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8773.360936: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8774.380679: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8774.385282: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8776.415380: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8776.420446: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8777.437544: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8777.441613: cpu_maps_update_done <-cpu_up
            bash-16504 [004] ....  8779.470573: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8779.476135: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8780.495533: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8780.500026: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8782.522699: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8782.528371: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8783.570723: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8783.573752: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8785.612220: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8785.616084: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8786.638633: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8786.642136: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8788.676445: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8788.681255: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8789.702009: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8789.706503: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8791.743118: cpu_maps_update_begin <-cpu_down
            bash-16504 [003] ....  8791.749304: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8792.778459: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8792.783381: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8794.816978: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8794.820784: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8795.842470: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8795.845773: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8797.884741: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8797.893176: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8798.915423: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8798.920277: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8800.951285: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8800.955060: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8801.970949: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8801.975405: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8804.005731: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8804.010637: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8805.039856: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] .N..  8805.045124: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8807.078850: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8807.085245: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8808.105540: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8808.110416: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8810.133043: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8810.136706: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8811.153735: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] .N..  8811.158190: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8813.190896: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] .N..  8813.196855: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8814.232582: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8814.239936: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8816.275843: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] .N..  8816.281023: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8817.297611: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8817.302503: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8819.335697: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8819.342753: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8820.485242: cpu_maps_update_begin <-cpu_up
            bash-16504 [002] ....  8820.493986: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8822.525739: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8822.532216: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8823.546722: cpu_maps_update_begin <-cpu_up
            bash-16504 [002] ....  8823.551768: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8825.585630: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8825.591466: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8826.606944: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8826.612168: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8828.643761: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8828.649332: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8829.669306: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8829.674107: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8833.376450: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8833.383650: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8834.395299: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8834.406603: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8836.442495: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8836.446348: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8837.466165: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8837.470909: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8839.496107: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8839.501230: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8840.520703: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] .N..  8840.524747: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8842.553367: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8842.561011: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8843.582658: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8843.587859: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8845.620565: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8845.624427: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8846.648369: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8846.651916: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8848.691008: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8848.695353: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8849.729293: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8849.733579: cpu_maps_update_done <-cpu_up
            bash-16504 [002] ....  8851.768863: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] .N..  8851.775736: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8852.804126: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8852.808656: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8854.847918: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8854.853401: cpu_maps_update_done <-cpu_down
            bash-16504 [004] ....  8855.884566: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] .N..  8855.888156: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8857.929002: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8857.933993: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8858.969234: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8858.974494: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8860.997448: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8861.002098: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8862.026992: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8862.030398: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8864.077749: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8864.081877: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8865.110837: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] .N..  8865.115005: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8867.149152: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8867.154598: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8868.172254: cpu_maps_update_begin <-cpu_up
            bash-16504 [003] ....  8868.178597: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8870.213097: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8870.224161: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8871.259574: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8871.262893: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8873.301472: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] .N..  8873.307061: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8874.327611: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8874.331790: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8876.357021: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8876.360965: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8877.380573: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] .N..  8877.384785: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8879.417649: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8879.423257: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8880.446601: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8880.451140: cpu_maps_update_done <-cpu_up
            bash-16504 [004] ....  8882.483374: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8882.491304: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8883.511357: cpu_maps_update_begin <-cpu_up
            bash-16504 [001] ....  8883.515360: cpu_maps_update_done <-cpu_up
            bash-16504 [001] ....  8885.549541: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8885.554860: cpu_maps_update_done <-cpu_down
            bash-16504 [003] ....  8886.576686: cpu_maps_update_begin <-cpu_up
            bash-16504 [000] ....  8886.581784: cpu_maps_update_done <-cpu_up
            bash-16504 [000] ....  8888.605193: cpu_maps_update_begin <-cpu_down
            bash-16504 [000] ....  8888.610156: cpu_maps_update_done <-cpu_down
            bash-16504 [000] ....  8889.624462: cpu_maps_update_begin <-cpu_up
            bash-16504 [004] ....  8889.629413: cpu_maps_update_done <-cpu_up
            bash-16504 [003] ....  8891.664209: cpu_maps_update_begin <-cpu_down
            bash-16504 [001] ....  8891.670284: cpu_maps_update_done <-cpu_down
            bash-16504 [001] ....  8892.687249: cpu_maps_update_begin <-cpu_up
            bash-16504 [002] ....  8892.692844: cpu_maps_update_done <-cpu_up

[-- Attachment #5: test_load_1cpus_cyc_main.txt --]
[-- Type: text/plain, Size: 70401 bytes --]

# tracer: function
#
# entries-in-buffer/entries-written: 880/880   #P:2
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
            bash-13333 [000] ....  4802.852293: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4802.856127: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4803.872276: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4803.876251: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4805.907365: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4805.910639: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4806.927020: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4806.930651: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  4808.958826: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4808.963179: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4809.979750: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4809.983462: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4812.020320: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4812.023670: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4813.043633: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4813.048253: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4815.111323: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4815.114438: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4816.131331: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4816.135345: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4818.172235: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4818.175692: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4819.193694: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4819.198257: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4821.229896: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4821.232432: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4822.274689: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4822.278348: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4824.309929: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4824.313391: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4825.332239: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4825.336604: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  4827.400625: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4827.405685: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4828.422014: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4828.425784: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4830.451640: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4830.454831: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4831.473504: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4831.477449: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4833.511986: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4833.515251: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4834.531627: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4834.535266: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4836.558696: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4836.561797: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4837.578369: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4837.582203: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4839.615567: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4839.619586: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4840.635355: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4840.639221: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4842.670572: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4842.673915: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4843.690701: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4843.694396: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4845.719419: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4845.722555: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4846.738901: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4846.742683: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4848.780927: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4848.784301: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4849.800627: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4849.804334: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4851.827956: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4851.831422: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4852.866076: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4852.870307: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4854.906789: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4854.909982: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4855.927023: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4855.930762: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4857.965863: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4857.969778: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4858.985858: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4858.989660: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4861.029301: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4861.032388: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4862.048667: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4862.052901: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  4864.237358: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4864.243873: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4865.268402: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4865.273137: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4867.304208: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4867.309120: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4868.333674: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4868.338331: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4870.382913: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4870.386450: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4871.410670: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4871.415842: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4873.458196: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4873.462370: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4874.487431: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4874.491902: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4876.520097: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4876.524805: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4877.549463: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4877.553800: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  4879.587237: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4879.593292: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4880.617497: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4880.622419: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4882.651627: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4882.655860: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4883.716802: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4883.721425: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4885.765478: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4885.770453: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4886.794690: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4886.799712: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4888.838759: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4888.842337: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4889.866870: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4889.871592: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4891.901499: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4891.905855: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4892.930074: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4892.934750: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4894.965360: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4894.970034: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4895.994024: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4895.998762: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4898.020233: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4898.024208: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4899.048697: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4899.053332: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4901.084765: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4901.089543: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4902.113772: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4902.118398: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4904.181595: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4904.185616: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4905.210041: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4905.214394: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4907.246657: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4907.251290: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4908.275835: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4908.280615: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  4910.312212: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  4910.318607: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4911.343720: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4911.348301: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4913.391811: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4913.395114: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4914.419201: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4914.424211: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4916.466884: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4916.470689: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4917.494990: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4917.499796: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4919.525628: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4919.530549: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4920.554666: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4920.559355: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4922.580352: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4922.583627: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4923.607863: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4923.612719: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4925.832199: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4925.838867: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4926.884307: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  4926.890181: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4928.954143: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  4928.961156: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4930.007313: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  4930.012848: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4932.070584: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4932.075440: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4933.119267: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  4933.126391: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  4935.158461: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  4935.164628: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4936.208586: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4936.214797: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4938.259933: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4938.268713: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4939.313225: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  4939.319112: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  4941.346334: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  4941.353115: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4942.397350: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4942.403086: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4944.436853: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4944.442379: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4945.487808: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4945.494773: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4947.530022: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4947.534816: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4948.578222: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  4948.584144: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  4950.623957: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4950.631970: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4951.676491: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  4951.682814: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  4953.721318: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4953.729130: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4954.771752: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  4954.780170: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4956.836131: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4956.841043: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4957.886374: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  4957.892987: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4959.934976: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4959.940811: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4960.983219: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4960.989255: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4963.029887: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4963.034313: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4964.078523: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4964.085120: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4966.122896: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4966.129594: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4967.172341: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4967.179390: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4969.217156: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4969.223353: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4970.269748: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  4970.276162: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4972.314815: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4972.318786: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4973.362294: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4973.369126: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4975.403944: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4975.408798: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4976.451168: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4976.457960: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4978.498654: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4978.503815: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4979.615567: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4979.621322: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  4981.656504: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4981.663691: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4982.708870: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  4982.715005: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  4984.754792: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4984.761783: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4985.807333: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  4985.813204: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4988.070624: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4988.079068: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4989.406147: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  4989.414122: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4991.460329: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4991.469489: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4992.780860: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  4992.792215: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  4994.843749: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  4994.851509: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4996.166595: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  4996.180519: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  4998.231247: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  4998.239792: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  4999.579114: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  4999.589902: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5001.639865: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5001.649394: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5003.021541: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5003.030278: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5005.079220: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5005.089179: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5006.424943: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5006.434656: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5008.488584: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5008.499771: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5009.876689: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5009.885227: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5011.919025: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5011.925190: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5013.240999: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5013.251927: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5015.278512: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5015.285077: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5016.558948: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5016.566342: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5018.625470: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5018.632302: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5019.918770: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5019.930303: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5021.976547: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5021.982568: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5023.293768: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5023.309271: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5025.351885: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5025.359516: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5026.623763: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5026.632254: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5028.680232: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5028.685223: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5029.991375: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5030.000218: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5032.038114: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5032.045355: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5033.321208: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5033.329533: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5035.366791: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5035.372710: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5036.644005: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5036.652538: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5038.704703: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5038.711751: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5039.988946: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5039.997736: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5042.046553: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5042.052829: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5043.783481: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5043.792681: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5045.846897: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5045.854518: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5047.128976: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5047.140222: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5049.184111: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5049.191728: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5050.466567: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5050.483564: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5052.533618: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5052.541488: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5053.784358: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5053.794063: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5056.235827: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5056.243831: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5058.339897: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5058.476276: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5060.527923: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5060.541486: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5062.590879: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5062.728207: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5064.779983: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5064.791186: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5066.877754: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5067.014590: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5069.086499: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5069.095301: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5071.205936: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5071.343836: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5073.395248: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5073.416268: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5075.577572: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5075.713824: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5077.758942: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5077.766007: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5079.777524: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5079.915060: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5081.967875: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5081.981114: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5084.136430: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5084.274002: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5086.335224: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5086.342830: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5088.441481: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5088.578612: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5090.608469: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5090.621379: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5092.722982: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5092.861148: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5094.918463: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5094.925943: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5097.076302: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5097.214874: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5099.281898: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5099.290723: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5101.379694: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5101.517229: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5103.597629: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5103.606497: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5105.684890: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5105.822830: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5107.872893: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5107.896422: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5109.989951: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5110.129662: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5112.195637: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5112.202673: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5114.322576: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5114.460479: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5116.517970: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5116.526250: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5118.627714: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5118.764328: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5120.823758: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5120.861144: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5122.947892: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5123.085196: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5125.144354: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5125.153833: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5127.156332: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5127.295952: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5129.349806: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5129.363221: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5131.449665: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5131.587196: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5133.730155: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5133.736393: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5135.841735: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5135.977451: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5138.007311: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5138.016652: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5140.119671: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5140.255561: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5142.721628: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5142.751575: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5145.039155: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5145.209112: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5147.246869: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5147.407033: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5149.642684: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5149.810866: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5151.880686: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5152.073599: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5154.292764: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5154.459365: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5156.530387: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5156.549383: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5158.850012: cpu_maps_update_begin <-cpu_up
            bash-13333 [002] ....  5159.017025: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5161.165847: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5161.300435: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5163.570676: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5163.737770: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5165.787629: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5165.956418: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5168.164215: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5168.332694: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5170.412788: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5170.469360: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5172.759447: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5172.927540: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5175.107939: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5175.287484: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5177.509241: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5177.675844: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5179.720637: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5179.737033: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5182.063316: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5182.231000: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5184.342172: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5184.427638: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5186.715770: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5186.883436: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5188.933037: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5189.099137: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5191.335894: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5191.502558: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5193.553455: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5193.560353: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5195.797195: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5195.965723: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5198.043866: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5198.059594: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5200.363402: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5200.531108: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5202.658933: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5202.843926: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5204.971668: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5205.137207: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5207.194961: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5207.369359: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5209.485025: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5209.650147: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5211.697369: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5211.742882: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5213.966133: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5214.133374: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5216.189950: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5216.371381: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5218.560783: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5218.729035: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5220.789918: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5220.925016: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5223.163479: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5223.331616: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5225.383075: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5225.545476: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5227.794213: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5227.960774: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5230.013486: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5230.151372: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5232.376995: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5232.543489: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5235.219567: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5235.361789: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5237.565117: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5237.758065: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5239.859700: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5240.047230: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5242.315501: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5242.507740: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5244.648326: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5244.884708: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5247.209398: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5247.401701: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5249.524183: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5249.541615: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5252.042338: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5252.234898: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5254.390039: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5254.459424: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5256.874384: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5257.066954: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5259.182927: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5259.194709: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5261.698732: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5261.896316: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5264.024837: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5264.043530: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5266.440972: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5266.634304: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5268.756467: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5268.899494: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5271.263850: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5271.457259: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5273.588885: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5273.746753: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5276.063834: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5276.257902: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5278.380517: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5278.454833: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5280.775718: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5280.968560: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5283.125508: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5283.254726: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5285.691475: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5285.884352: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5288.007589: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5288.155185: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5290.481620: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5290.673592: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5292.816032: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5292.909396: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5295.438367: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5295.630046: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5297.695791: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5297.757204: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5300.090146: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5300.282939: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5302.397534: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5302.544638: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5304.858912: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5305.051622: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5307.236787: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5307.409433: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5309.750789: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5309.946426: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5312.058479: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5312.230966: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5314.439430: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5314.632220: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5316.767924: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5316.942149: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5319.335983: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5319.528024: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5321.654537: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5321.739413: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5324.221322: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5324.414262: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5326.495197: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5326.664632: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5329.101857: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5329.294229: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5332.484976: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5332.494034: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5335.061786: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5335.280058: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5338.024358: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5338.129275: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5340.592598: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5340.810925: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5343.425568: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5343.484162: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5346.129210: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5346.347825: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5348.505084: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5348.514433: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5351.112661: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5351.333738: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5353.751554: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5353.845016: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5356.461870: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5356.683722: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5359.166437: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5359.173018: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5361.652490: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5361.872369: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5364.293430: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5364.394247: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5366.818105: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5367.036284: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5369.187069: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5369.192965: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5371.787388: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5372.006212: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5374.268980: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5374.281398: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5376.803808: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5377.023250: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5379.200388: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5379.210190: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5381.817134: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5382.035306: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5384.198478: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5384.211132: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5386.600995: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5386.819953: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5389.595724: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5389.778885: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5392.158652: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5392.377342: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5394.526160: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5394.532349: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5396.902492: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5397.121053: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5399.237404: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5399.245093: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5401.885172: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5402.103618: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5404.271983: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5404.293157: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5406.760333: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5406.979506: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5409.124046: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5409.132918: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5411.655565: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5411.877108: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5414.324429: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5414.398287: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5417.013970: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5417.231738: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5419.636348: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5419.643975: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5422.354520: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5422.573590: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5425.034629: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5425.043926: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5427.603156: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5427.822371: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5430.254207: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5430.261458: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5433.005667: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5433.224329: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5439.851030: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5439.866866: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5442.504378: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5442.748075: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5445.257913: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5445.264060: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5448.168410: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5448.411886: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5451.254058: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5451.260768: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5454.254066: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5454.498742: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5457.483749: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5457.489956: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5460.310161: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5460.554653: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5463.558757: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5463.564418: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5466.432490: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5466.676304: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5469.613469: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5469.619811: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5472.561744: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5472.805274: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5476.358730: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5476.364916: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5479.200092: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5479.443987: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5482.773820: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5482.780008: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5485.565605: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5485.810146: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5488.691610: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5488.698668: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5491.405453: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5491.649765: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5494.621380: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5494.628787: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5497.274677: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5497.519034: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5500.363056: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5500.369387: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5502.946940: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5503.190819: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5506.121702: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5506.128407: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5509.029244: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5509.272999: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5512.823272: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5512.838481: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5515.514452: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5515.761572: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5518.255220: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5518.265748: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5521.217227: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5521.461000: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5524.473582: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5524.479907: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5527.244821: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5527.489378: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5530.064021: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5530.069944: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5532.994945: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5533.239282: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5536.061539: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5536.066496: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5538.857420: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5539.101670: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5542.034982: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5542.041625: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5544.781811: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5545.025997: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5548.043965: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5548.056103: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5551.010654: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5551.256933: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5554.379088: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5554.385708: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5557.020403: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5557.264266: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5569.318593: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5569.323070: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5571.885652: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5572.154760: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5575.146459: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5575.151146: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5578.041360: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5578.310521: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5581.474788: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5581.478576: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5584.321293: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5584.590707: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5587.537876: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5587.541535: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5590.111328: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5590.380356: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5593.598628: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5593.602995: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5596.189319: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5596.458186: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5599.668299: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5599.672112: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5602.515026: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5602.784431: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5606.484013: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5606.488601: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5609.259889: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5609.528991: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5612.622249: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5612.626278: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5615.547262: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5615.817558: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5618.912349: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5618.923122: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5621.553775: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5621.823605: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5624.893841: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5624.898225: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5627.543731: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5627.813926: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5630.931960: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5630.935616: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5633.643824: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5633.913575: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5636.987399: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5636.991285: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5639.857613: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5640.126848: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5643.098945: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5643.104229: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5645.815855: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5646.085071: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5649.877026: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5649.881058: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5652.469868: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5652.738461: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5655.843676: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5655.848697: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5658.754658: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5659.024448: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5662.161656: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5662.165594: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5664.816927: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5665.086468: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5668.038428: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5668.042530: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5670.644595: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5670.913941: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5674.131783: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5674.135783: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5677.069294: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5677.338503: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5680.396194: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5680.401335: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5683.257637: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5683.527090: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5686.533608: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5686.543833: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5689.456966: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5689.728866: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5702.565129: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5702.569706: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5705.255771: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5705.551485: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5708.668167: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5708.672768: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5711.668155: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5711.964075: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5715.123208: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5715.127489: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5717.709838: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5718.006697: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5720.934349: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] .N..  5720.937825: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5723.976792: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5724.274518: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5727.309167: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5727.313742: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5730.029823: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5730.326194: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5733.490313: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5733.494025: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5736.519038: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5736.814111: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5739.935479: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5739.939791: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5742.884578: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5743.179544: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5746.356687: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5746.360013: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5749.123035: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5749.418451: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5752.497903: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5752.502288: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5755.216125: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5755.512233: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5758.626066: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5758.633205: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5761.308401: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5761.606015: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5764.867046: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5764.870656: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5767.811277: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5768.108256: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5771.398007: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5771.401992: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5774.088946: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5774.384584: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5777.509014: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5777.513042: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5780.403772: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5780.698263: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5783.630441: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5783.634608: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5786.242653: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5786.538017: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5789.663725: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5789.668376: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5792.293635: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5792.588665: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5795.846598: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5795.850181: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5798.525916: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5798.821906: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5801.766878: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5801.770412: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5804.642810: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5804.939423: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5808.177612: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5808.181267: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5811.007482: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5811.303853: cpu_maps_update_done <-cpu_up
            bash-13333 [002] ....  5814.406780: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5814.410755: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5816.947341: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5817.246315: cpu_maps_update_done <-cpu_up
            bash-13333 [000] ....  5820.447436: cpu_maps_update_begin <-cpu_down
            bash-13333 [000] ....  5820.450795: cpu_maps_update_done <-cpu_down
            bash-13333 [000] ....  5822.997241: cpu_maps_update_begin <-cpu_up
            bash-13333 [000] ....  5823.293414: cpu_maps_update_done <-cpu_up

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Srivatsa S. Bhat]

Hi Vincent,

On 02/15/2013 06:58 PM, Vincent Guittot wrote:
> Hi Srivatsa,
> 
> I have run some tests with you branch (thanks Paul for the git tree)
> and you will find results below.
>

Thank you very much for testing this patchset!
 
> The tests condition are:
> - 5 CPUs system in 2 clusters
> - The test plugs/unplugs CPU2 and it increases the system load each 20
> plug/unplug sequence with either more cyclictests threads
> - The test is done with all CPUs online and with only CPU0 and CPU2
> 
> The main conclusion is that there is no differences with and without
> your patches with my stress tests. I'm not sure that it was the
> expected results but the cpu_down is already quite low : 4-5ms in
> average
> 

Atleast my patchset doesn't perform _worse_ than mainline, with respect
to cpu_down duration :-)

So, here is the analysis:
Stop-machine() doesn't really slow down CPU-down operation, if the rest
of the CPUs are mostly running in userspace all the time. Because, the
CPUs running userspace workloads cooperate very eagerly with the stop-machine
dance - they receive the resched IPI, and allow the per-cpu cpu-stopper
thread to monopolize the CPU, almost immediately.

The scenario where stop-machine() takes longer to take effect is when
most of the online CPUs are running in kernelspace, because, then the
probability that they call preempt_disable() frequently (and hence inhibit
stop-machine) is higher. That's why, in my tests, I ran genload from LTP
which generated a lot of system-time (system-time in 'top' indicates activity
in kernelspace). Hence my patchset showed significant improvement over
mainline in my tests.

However, your test is very useful too, if we measure a different parameter:
the latency impact on the workloads running on the system (cyclic test).
One other important aim of this patchset is to make hotplug as less intrusive
as possible, for other workloads running on the system. So if you measure
the cyclictest numbers, I would expect my patchset to show better numbers
than mainline, when you do cpu-hotplug in parallel (same test that you did).
Mainline would run stop-machine and hence interrupt the cyclic test tasks
too often. My patchset wouldn't do that, and hence cyclic test should
ideally show better numbers.

I'd really appreciate if you could try that out and let me know how it
goes.. :-) Thank you very much!

Regards,
Srivatsa S. Bhat

> 
> 
> On 12 February 2013 04:58, Srivatsa S. Bhat
> <srivatsa.bhat@linux.vnet.ibm.com> wrote:
>> On 02/12/2013 12:38 AM, Paul E. McKenney wrote:
>>> On Mon, Feb 11, 2013 at 05:53:41PM +0530, Srivatsa S. Bhat wrote:
>>>> On 02/11/2013 05:28 PM, Vincent Guittot wrote:
>>>>> On 8 February 2013 19:09, Srivatsa S. Bhat
>>>>> <srivatsa.bhat@linux.vnet.ibm.com> wrote:
>>>
>>> [ . . . ]
>>>
>>>>>> Adding Vincent to CC, who had previously evaluated the performance and
>>>>>> latency implications of CPU hotplug on ARM platforms, IIRC.
>>>>>>
>>>>>
>>>>> Hi Srivatsa,
>>>>>
>>>>> I can try to run some of our stress tests on your patches.
>>>>
>>>> Great!
>>>>
>>>>> Have you
>>>>> got a git tree that i can pull ?
>>>>>
>>>>
>>>> Unfortunately, no, none at the moment..  :-(
>>>
>>> You do need to create an externally visible git tree.
>>
>> Ok, I'll do that soon.
>>
>>>  In the meantime,
>>> I have added your series at rcu/bhat.2013.01.21a on -rcu:
>>>
>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
>>>
>>> This should appear soon on a kernel.org mirror near you.  ;-)
>>>
>>
>> Thank you very much, Paul! :-)
>>
>> Regards,
>> Srivatsa S. Bhat
>>

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Vincent Guittot]

On 15 February 2013 20:40, Srivatsa S. Bhat
<srivatsa.bhat@linux.vnet.ibm.com> wrote:
> Hi Vincent,
>
> On 02/15/2013 06:58 PM, Vincent Guittot wrote:
>> Hi Srivatsa,
>>
>> I have run some tests with you branch (thanks Paul for the git tree)
>> and you will find results below.
>>
>
> Thank you very much for testing this patchset!
>
>> The tests condition are:
>> - 5 CPUs system in 2 clusters
>> - The test plugs/unplugs CPU2 and it increases the system load each 20
>> plug/unplug sequence with either more cyclictests threads
>> - The test is done with all CPUs online and with only CPU0 and CPU2
>>
>> The main conclusion is that there is no differences with and without
>> your patches with my stress tests. I'm not sure that it was the
>> expected results but the cpu_down is already quite low : 4-5ms in
>> average
>>
>
> Atleast my patchset doesn't perform _worse_ than mainline, with respect
> to cpu_down duration :-)

yes exactly and it has pass  more than 400 consecutive plug/unplug on
an ARM platform

>
> So, here is the analysis:
> Stop-machine() doesn't really slow down CPU-down operation, if the rest
> of the CPUs are mostly running in userspace all the time. Because, the
> CPUs running userspace workloads cooperate very eagerly with the stop-machine
> dance - they receive the resched IPI, and allow the per-cpu cpu-stopper
> thread to monopolize the CPU, almost immediately.
>
> The scenario where stop-machine() takes longer to take effect is when
> most of the online CPUs are running in kernelspace, because, then the
> probability that they call preempt_disable() frequently (and hence inhibit
> stop-machine) is higher. That's why, in my tests, I ran genload from LTP
> which generated a lot of system-time (system-time in 'top' indicates activity
> in kernelspace). Hence my patchset showed significant improvement over
> mainline in my tests.
>

ok, I hadn't noticed this important point for the test

> However, your test is very useful too, if we measure a different parameter:
> the latency impact on the workloads running on the system (cyclic test).
> One other important aim of this patchset is to make hotplug as less intrusive
> as possible, for other workloads running on the system. So if you measure
> the cyclictest numbers, I would expect my patchset to show better numbers
> than mainline, when you do cpu-hotplug in parallel (same test that you did).
> Mainline would run stop-machine and hence interrupt the cyclic test tasks
> too often. My patchset wouldn't do that, and hence cyclic test should
> ideally show better numbers.

In fact, I haven't looked at the results as i was more interested by
the load that was generated

>
> I'd really appreciate if you could try that out and let me know how it
> goes.. :-) Thank you very much!

ok, I'm going to try to run a test series

Vincent
>
> Regards,
> Srivatsa S. Bhat
>
>>
>>
>> On 12 February 2013 04:58, Srivatsa S. Bhat
>> <srivatsa.bhat@linux.vnet.ibm.com> wrote:
>>> On 02/12/2013 12:38 AM, Paul E. McKenney wrote:
>>>> On Mon, Feb 11, 2013 at 05:53:41PM +0530, Srivatsa S. Bhat wrote:
>>>>> On 02/11/2013 05:28 PM, Vincent Guittot wrote:
>>>>>> On 8 February 2013 19:09, Srivatsa S. Bhat
>>>>>> <srivatsa.bhat@linux.vnet.ibm.com> wrote:
>>>>
>>>> [ . . . ]
>>>>
>>>>>>> Adding Vincent to CC, who had previously evaluated the performance and
>>>>>>> latency implications of CPU hotplug on ARM platforms, IIRC.
>>>>>>>
>>>>>>
>>>>>> Hi Srivatsa,
>>>>>>
>>>>>> I can try to run some of our stress tests on your patches.
>>>>>
>>>>> Great!
>>>>>
>>>>>> Have you
>>>>>> got a git tree that i can pull ?
>>>>>>
>>>>>
>>>>> Unfortunately, no, none at the moment..  :-(
>>>>
>>>> You do need to create an externally visible git tree.
>>>
>>> Ok, I'll do that soon.
>>>
>>>>  In the meantime,
>>>> I have added your series at rcu/bhat.2013.01.21a on -rcu:
>>>>
>>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
>>>>
>>>> This should appear soon on a kernel.org mirror near you.  ;-)
>>>>
>>>
>>> Thank you very much, Paul! :-)
>>>
>>> Regards,
>>> Srivatsa S. Bhat
>>>
>
>

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Srivatsa S. Bhat]

On 02/18/2013 03:54 PM, Vincent Guittot wrote:
> On 15 February 2013 20:40, Srivatsa S. Bhat
> <srivatsa.bhat@linux.vnet.ibm.com> wrote:
>> Hi Vincent,
>>
>> On 02/15/2013 06:58 PM, Vincent Guittot wrote:
>>> Hi Srivatsa,
>>>
>>> I have run some tests with you branch (thanks Paul for the git tree)
>>> and you will find results below.
>>>
>>
>> Thank you very much for testing this patchset!
>>
>>> The tests condition are:
>>> - 5 CPUs system in 2 clusters
>>> - The test plugs/unplugs CPU2 and it increases the system load each 20
>>> plug/unplug sequence with either more cyclictests threads
>>> - The test is done with all CPUs online and with only CPU0 and CPU2
>>>
>>> The main conclusion is that there is no differences with and without
>>> your patches with my stress tests. I'm not sure that it was the
>>> expected results but the cpu_down is already quite low : 4-5ms in
>>> average
>>>
>>
>> Atleast my patchset doesn't perform _worse_ than mainline, with respect
>> to cpu_down duration :-)
> 
> yes exactly and it has pass  more than 400 consecutive plug/unplug on
> an ARM platform
>

Great! However, did you turn on CPU_IDLE during your tests?

In my tests, I had turned off cpu idle in the .config, like I had mentioned in
the cover letter. I'm struggling to get it working with CPU_IDLE/INTEL_IDLE
turned on, because it gets into a lockup almost immediately. It appears that
the lock-holder of clockevents_lock never releases it, for some reason..
See below for the full log. Lockdep has not been useful in debugging this,
unfortunately :-(

>>
>> So, here is the analysis:
>> Stop-machine() doesn't really slow down CPU-down operation, if the rest
>> of the CPUs are mostly running in userspace all the time. Because, the
>> CPUs running userspace workloads cooperate very eagerly with the stop-machine
>> dance - they receive the resched IPI, and allow the per-cpu cpu-stopper
>> thread to monopolize the CPU, almost immediately.
>>
>> The scenario where stop-machine() takes longer to take effect is when
>> most of the online CPUs are running in kernelspace, because, then the
>> probability that they call preempt_disable() frequently (and hence inhibit
>> stop-machine) is higher. That's why, in my tests, I ran genload from LTP
>> which generated a lot of system-time (system-time in 'top' indicates activity
>> in kernelspace). Hence my patchset showed significant improvement over
>> mainline in my tests.
>>
> 
> ok, I hadn't noticed this important point for the test
> 
>> However, your test is very useful too, if we measure a different parameter:
>> the latency impact on the workloads running on the system (cyclic test).
>> One other important aim of this patchset is to make hotplug as less intrusive
>> as possible, for other workloads running on the system. So if you measure
>> the cyclictest numbers, I would expect my patchset to show better numbers
>> than mainline, when you do cpu-hotplug in parallel (same test that you did).
>> Mainline would run stop-machine and hence interrupt the cyclic test tasks
>> too often. My patchset wouldn't do that, and hence cyclic test should
>> ideally show better numbers.
> 
> In fact, I haven't looked at the results as i was more interested by
> the load that was generated
> 
>>
>> I'd really appreciate if you could try that out and let me know how it
>> goes.. :-) Thank you very much!
> 
> ok, I'm going to try to run a test series
> 

Great! Thank you :-)
Regards,
Srivatsa S. Bhat

--------------------------------------------------------------------------------

Lockup observed while running this patchset, with CPU_IDLE and INTEL_IDLE turned
on in the .config:

 smpboot: CPU 1 is now offline
Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 11
Pid: 0, comm: swapper/11 Not tainted 3.8.0-rc7+stpmch13-1 #8
Call Trace:
 <NMI>  [<ffffffff815a319e>] panic+0xc9/0x1ee
 [<ffffffff810fdd41>] watchdog_overflow_callback+0xb1/0xc0
 [<ffffffff8113ab5c>] __perf_event_overflow+0x9c/0x330
 [<ffffffff81028a88>] ? x86_perf_event_set_period+0xd8/0x160
 [<ffffffff8113b514>] perf_event_overflow+0x14/0x20
 [<ffffffff8102ee54>] intel_pmu_handle_irq+0x1c4/0x360
 [<ffffffff815a8ef1>] perf_event_nmi_handler+0x21/0x30
 [<ffffffff815a8366>] nmi_handle+0xb6/0x200
 [<ffffffff815a82b0>] ? oops_begin+0xd0/0xd0
 [<ffffffff815a85c8>] default_do_nmi+0x68/0x220
 [<ffffffff815a8840>] do_nmi+0xc0/0x110
 [<ffffffff815a7911>] end_repeat_nmi+0x1e/0x2e
 [<ffffffff812a3f98>] ? delay_tsc+0x38/0xb0
 [<ffffffff812a3f98>] ? delay_tsc+0x38/0xb0
 [<ffffffff812a3f98>] ? delay_tsc+0x38/0xb0
 <<EOE>>  [<ffffffff812a3f1f>] __delay+0xf/0x20
 [<ffffffff812aba1e>] do_raw_spin_lock+0x7e/0x150
 [<ffffffff815a64c1>] _raw_spin_lock_irqsave+0x61/0x70
 [<ffffffff810c0758>] ? clockevents_notify+0x28/0x150
 [<ffffffff815a6d37>] ? _raw_spin_unlock_irqrestore+0x77/0x80
 [<ffffffff810c0758>] clockevents_notify+0x28/0x150
 [<ffffffff8130459f>] intel_idle+0xaf/0xe0
 [<ffffffff81472ee0>] ? disable_cpuidle+0x20/0x20
 [<ffffffff81472ef9>] cpuidle_enter+0x19/0x20
 [<ffffffff814734c1>] cpuidle_wrap_enter+0x41/0xa0
 [<ffffffff81473530>] cpuidle_enter_tk+0x10/0x20
 [<ffffffff81472f17>] cpuidle_enter_state+0x17/0x50
 [<ffffffff81473899>] cpuidle_idle_call+0xd9/0x290
 [<ffffffff810203d5>] cpu_idle+0xe5/0x140
 [<ffffffff8159c603>] start_secondary+0xdd/0xdf
BUG: spinlock lockup suspected on CPU#2, migration/2/19
 lock: clockevents_lock+0x0/0x40, .magic: dead4ead, .owner: swapper/8/0, .owner_cpu: 8
Pid: 19, comm: migration/2 Not tainted 3.8.0-rc7+stpmch13-1 #8
Call Trace:
 [<ffffffff812ab878>] spin_dump+0x78/0xc0
 [<ffffffff812abac6>] do_raw_spin_lock+0x126/0x150
 [<ffffffff815a64c1>] _raw_spin_lock_irqsave+0x61/0x70
 [<ffffffff810c0758>] ? clockevents_notify+0x28/0x150
 [<ffffffff810c0758>] clockevents_notify+0x28/0x150
 [<ffffffff8159e08f>] hrtimer_cpu_notify+0xe3/0x107
 [<ffffffff815ab5ec>] notifier_call_chain+0x5c/0x120
 [<ffffffff8108d7de>] __raw_notifier_call_chain+0xe/0x10
 [<ffffffff8105e540>] __cpu_notify+0x20/0x40
 [<ffffffff81592003>] take_cpu_down+0x53/0x80
 [<ffffffff810ed3ba>] cpu_stopper_thread+0xfa/0x1e0
 [<ffffffff81591fb0>] ? enable_nonboot_cpus+0xf0/0xf0
 [<ffffffff815a5039>] ? __schedule+0x469/0x890
 [<ffffffff810ed2c0>] ? res_counter_init+0x60/0x60
 [<ffffffff810ed2c0>] ? res_counter_init+0x60/0x60
 [<ffffffff8108576e>] kthread+0xee/0x100
 [<ffffffff81085680>] ? __init_kthread_worker+0x70/0x70
 [<ffffffff815b092c>] ret_from_fork+0x7c/0xb0
 [<ffffffff81085680>] ? __init_kthread_worker+0x70/0x70
sending NMI to all CPUs:
NMI backtrace for cpu 3
CPU 3 
Pid: 0, comm: swapper/3 Not tainted 3.8.0-rc7+stpmch13-1 #8 IBM IBM System x -[7870C4Q]-/68Y8033     
RIP: 0010:[<ffffffff81304589>]  [<ffffffff81304589>] intel_idle+0x99/0xe0
RSP: 0018:ffff8808db57bdd8  EFLAGS: 00000046
RAX: 0000000000000020 RBX: 0000000000000008 RCX: 0000000000000001
RDX: 0000000000000000 RSI: ffff8808db57bfd8 RDI: ffffffff815a6d37
RBP: ffff8808db57be08 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000003
R13: 0000000000000020 R14: ffffffff81a9c000 R15: 0000000000000003
FS:  0000000000000000(0000) GS:ffff8808ffcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fffbbb902f8 CR3: 0000000001a0c000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process swapper/3 (pid: 0, threadinfo ffff8808db57a000, task ffff8808db578380)
Stack:
 0000000000000000 0000000300000001 ffff8808db57be18 ffff8808ffcda070
 000000bd1ad2c5de ffffffff81472ee0 ffff8808db57be18 ffffffff81472ef9
 ffff8808db57be78 ffffffff814734c1 ffff8808ffcc0000 0000000100000046
Call Trace:
 [<ffffffff81472ee0>] ? disable_cpuidle+0x20/0x20
 [<ffffffff81472ef9>] cpuidle_enter+0x19/0x20
 [<ffffffff814734c1>] cpuidle_wrap_enter+0x41/0xa0
 [<ffffffff81473530>] cpuidle_enter_tk+0x10/0x20
 [<ffffffff81472f17>] cpuidle_enter_state+0x17/0x50
 [<ffffffff81473899>] cpuidle_idle_call+0xd9/0x290
 [<ffffffff810203d5>] cpu_idle+0xe5/0x140
 [<ffffffff8159c603>] start_secondary+0xdd/0xdf
Code: ff 48 8d 86 38 e0 ff ff 83 e2 08 75 1e 31 d2 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d 69 7d 79 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 91 c1 db 
NMI backtrace for cpu 5
CPU 5 
Pid: 0, comm: swapper/5 Not tainted 3.8.0-rc7+stpmch13-1 #8 IBM IBM System x -[7870C4Q]-/68Y8033     
RIP: 0010:[<ffffffff81304589>]  [<ffffffff81304589>] intel_idle+0x99/0xe0
RSP: 0018:ffff8808db583dd8  EFLAGS: 00000046
RAX: 0000000000000020 RBX: 0000000000000008 RCX: 0000000000000001
RDX: 0000000000000000 RSI: ffff8808db583fd8 RDI: ffffffff815a6d37
RBP: ffff8808db583e08 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000003
R13: 0000000000000020 R14: ffffffff81a9c000 R15: 0000000000000003
FS:  0000000000000000(0000) GS:ffff88117fc40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffffffffff600400 CR3: 0000000001a0c000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process swapper/5 (pid: 0, threadinfo ffff8808db582000, task ffff8808db580400)
Stack:
 0000000000000000 0000000500000001 ffff8808db583e18 ffff88117fc5a070
 000000bd1af68ce5 ffffffff81472ee0 ffff8808db583e18 ffffffff81472ef9
 ffff8808db583e78 ffffffff814734c1 ffff88117fc40000 0000000100000046
Call Trace:
 [<ffffffff81472ee0>] ? disable_cpuidle+0x20/0x20
 [<ffffffff81472ef9>] cpuidle_enter+0x19/0x20
 [<ffffffff814734c1>] cpuidle_wrap_enter+0x41/0xa0
 [<ffffffff81473530>] cpuidle_enter_tk+0x10/0x20
 [<ffffffff81472f17>] cpuidle_enter_state+0x17/0x50
 [<ffffffff81473899>] cpuidle_idle_call+0xd9/0x290
 [<ffffffff810203d5>] cpu_idle+0xe5/0x140
 [<ffffffff8159c603>] start_secondary+0xdd/0xdf
Code: ff 48 8d 86 38 e0 ff ff 83 e2 08 75 1e 31 d2 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d 69 7d 79 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 91 c1 db 
NMI backtrace for cpu 6
CPU 6 
Pid: 0, comm: swapper/6 Not tainted 3.8.0-rc7+stpmch13-1 #8 IBM IBM System x -[7870C4Q]-/68Y8033     
RIP: 0010:[<ffffffff81304589>]  [<ffffffff81304589>] intel_idle+0x99/0xe0
RSP: 0018:ffff8808db589dd8  EFLAGS: 00000046
RAX: 0000000000000010 RBX: 0000000000000004 RCX: 0000000000000001
RDX: 0000000000000000 RSI: ffff8808db589fd8 RDI: ffffffff815a6d37
RBP: ffff8808db589e08 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000002
R13: 0000000000000010 R14: ffffffff81a9c000 R15: 0000000000000002
FS:  0000000000000000(0000) GS:ffff88117fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fde168df000 CR3: 0000000001a0c000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process swapper/6 (pid: 0, threadinfo ffff8808db588000, task ffff8808db586440)
Stack:
 0000000000000000 0000000600000001 ffff8808db589e18 ffff88117fc9a070
 000000bd1af4a55f ffffffff81472ee0 ffff8808db589e18 ffffffff81472ef9
 ffff8808db589e78 ffffffff814734c1 ffff88117fc80000 0000000b00000046
Call Trace:
 [<ffffffff81472ee0>] ? disable_cpuidle+0x20/0x20
 [<ffffffff81472ef9>] cpuidle_enter+0x19/0x20
 [<ffffffff814734c1>] cpuidle_wrap_enter+0x41/0xa0
 [<ffffffff81473530>] cpuidle_enter_tk+0x10/0x20
 [<ffffffff81472f17>] cpuidle_enter_state+0x17/0x50
 [<ffffffff81473899>] cpuidle_idle_call+0xd9/0x290
 [<ffffffff810203d5>] cpu_idle+0xe5/0x140
 [<ffffffff8159c603>] start_secondary+0xdd/0xdf
Code: ff 48 8d 86 38 e0 ff ff 83 e2 08 75 1e 31 d2 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d 69 7d 79 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 91 c1 db 
3h

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Srivatsa S. Bhat]

On 02/18/2013 04:04 PM, Srivatsa S. Bhat wrote:
> On 02/18/2013 03:54 PM, Vincent Guittot wrote:
>> On 15 February 2013 20:40, Srivatsa S. Bhat
>> <srivatsa.bhat@linux.vnet.ibm.com> wrote:
>>> Hi Vincent,
>>>
>>> On 02/15/2013 06:58 PM, Vincent Guittot wrote:
>>>> Hi Srivatsa,
>>>>
>>>> I have run some tests with you branch (thanks Paul for the git tree)
>>>> and you will find results below.
>>>>
>>>
>>> Thank you very much for testing this patchset!
>>>
>>>> The tests condition are:
>>>> - 5 CPUs system in 2 clusters
>>>> - The test plugs/unplugs CPU2 and it increases the system load each 20
>>>> plug/unplug sequence with either more cyclictests threads
>>>> - The test is done with all CPUs online and with only CPU0 and CPU2
>>>>
>>>> The main conclusion is that there is no differences with and without
>>>> your patches with my stress tests. I'm not sure that it was the
>>>> expected results but the cpu_down is already quite low : 4-5ms in
>>>> average
>>>>
>>>
>>> Atleast my patchset doesn't perform _worse_ than mainline, with respect
>>> to cpu_down duration :-)
>>
>> yes exactly and it has pass  more than 400 consecutive plug/unplug on
>> an ARM platform
>>
> 
> Great! However, did you turn on CPU_IDLE during your tests?
> 
> In my tests, I had turned off cpu idle in the .config, like I had mentioned in
> the cover letter. I'm struggling to get it working with CPU_IDLE/INTEL_IDLE
> turned on, because it gets into a lockup almost immediately. It appears that
> the lock-holder of clockevents_lock never releases it, for some reason..
> See below for the full log. Lockdep has not been useful in debugging this,
> unfortunately :-(
> 

Ah, nevermind, the following diff fixes it :-) I had applied this fix on v5
and tested but it still had races where I used to hit the lockups. Now after
I fixed all the memory barrier issues that Paul and Oleg pointed out in v5,
I applied this fix again and tested it just now - it works beautifully! :-)

I'll include this fix and post a v6 soon.

Regards,
Srivatsa S. Bhat

--------------------------------------------------------------------------->


diff --git a/kernel/time/clockevents.c b/kernel/time/clockevents.c
index 30b6de0..ca340fd 100644
--- a/kernel/time/clockevents.c
+++ b/kernel/time/clockevents.c
@@ -17,6 +17,7 @@
 #include <linux/module.h>
 #include <linux/notifier.h>
 #include <linux/smp.h>
+#include <linux/cpu.h>
 
 #include "tick-internal.h"
 
@@ -431,6 +432,7 @@ void clockevents_notify(unsigned long reason, void *arg)
 	unsigned long flags;
 	int cpu;
 
+	get_online_cpus_atomic();
 	raw_spin_lock_irqsave(&clockevents_lock, flags);
 	clockevents_do_notify(reason, arg);
 
@@ -459,6 +461,7 @@ void clockevents_notify(unsigned long reason, void *arg)
 		break;
 	}
 	raw_spin_unlock_irqrestore(&clockevents_lock, flags);
+	put_online_cpus_atomic();
 }
 EXPORT_SYMBOL_GPL(clockevents_notify);
 #endif

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Vincent Guittot]

On 18 February 2013 11:51, Srivatsa S. Bhat
<srivatsa.bhat@linux.vnet.ibm.com> wrote:
> On 02/18/2013 04:04 PM, Srivatsa S. Bhat wrote:
>> On 02/18/2013 03:54 PM, Vincent Guittot wrote:
>>> On 15 February 2013 20:40, Srivatsa S. Bhat
>>> <srivatsa.bhat@linux.vnet.ibm.com> wrote:
>>>> Hi Vincent,
>>>>
>>>> On 02/15/2013 06:58 PM, Vincent Guittot wrote:
>>>>> Hi Srivatsa,
>>>>>
>>>>> I have run some tests with you branch (thanks Paul for the git tree)
>>>>> and you will find results below.
>>>>>
>>>>
>>>> Thank you very much for testing this patchset!
>>>>
>>>>> The tests condition are:
>>>>> - 5 CPUs system in 2 clusters
>>>>> - The test plugs/unplugs CPU2 and it increases the system load each 20
>>>>> plug/unplug sequence with either more cyclictests threads
>>>>> - The test is done with all CPUs online and with only CPU0 and CPU2
>>>>>
>>>>> The main conclusion is that there is no differences with and without
>>>>> your patches with my stress tests. I'm not sure that it was the
>>>>> expected results but the cpu_down is already quite low : 4-5ms in
>>>>> average
>>>>>
>>>>
>>>> Atleast my patchset doesn't perform _worse_ than mainline, with respect
>>>> to cpu_down duration :-)
>>>
>>> yes exactly and it has pass  more than 400 consecutive plug/unplug on
>>> an ARM platform
>>>
>>
>> Great! However, did you turn on CPU_IDLE during your tests?
>>
>> In my tests, I had turned off cpu idle in the .config, like I had mentioned in
>> the cover letter. I'm struggling to get it working with CPU_IDLE/INTEL_IDLE
>> turned on, because it gets into a lockup almost immediately. It appears that
>> the lock-holder of clockevents_lock never releases it, for some reason..
>> See below for the full log. Lockdep has not been useful in debugging this,
>> unfortunately :-(
>>
>
> Ah, nevermind, the following diff fixes it :-) I had applied this fix on v5
> and tested but it still had races where I used to hit the lockups. Now after
> I fixed all the memory barrier issues that Paul and Oleg pointed out in v5,
> I applied this fix again and tested it just now - it works beautifully! :-)

My tests have been done without cpuidle because i have some issues
with function tracer and cpuidle

But the cpu hotplug and cpuidle work well when I run the tests without
enabling the function tracer

Vincent

>
> I'll include this fix and post a v6 soon.
>
> Regards,
> Srivatsa S. Bhat
>
> --------------------------------------------------------------------------->
>
>
> diff --git a/kernel/time/clockevents.c b/kernel/time/clockevents.c
> index 30b6de0..ca340fd 100644
> --- a/kernel/time/clockevents.c
> +++ b/kernel/time/clockevents.c
> @@ -17,6 +17,7 @@
>  #include <linux/module.h>
>  #include <linux/notifier.h>
>  #include <linux/smp.h>
> +#include <linux/cpu.h>
>
>  #include "tick-internal.h"
>
> @@ -431,6 +432,7 @@ void clockevents_notify(unsigned long reason, void *arg)
>         unsigned long flags;
>         int cpu;
>
> +       get_online_cpus_atomic();
>         raw_spin_lock_irqsave(&clockevents_lock, flags);
>         clockevents_do_notify(reason, arg);
>
> @@ -459,6 +461,7 @@ void clockevents_notify(unsigned long reason, void *arg)
>                 break;
>         }
>         raw_spin_unlock_irqrestore(&clockevents_lock, flags);
> +       put_online_cpus_atomic();
>  }
>  EXPORT_SYMBOL_GPL(clockevents_notify);
>  #endif
>

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Steven Rostedt]

On Mon, 2013-02-18 at 11:58 +0100, Vincent Guittot wrote:

> My tests have been done without cpuidle because i have some issues
> with function tracer and cpuidle
> 
> But the cpu hotplug and cpuidle work well when I run the tests without
> enabling the function tracer
> 

I know suspend and resume has issues with function tracing (because it
makes things like calling smp_processor_id() crash the system), but I'm
unaware of issues with hotplug itself. Could be some of the same issues.

Can you give me more details, I'll try to investigate it.

Thanks,

-- Steve

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Vincent Guittot]

On 18 February 2013 16:30, Steven Rostedt <rostedt@goodmis.org> wrote:
> On Mon, 2013-02-18 at 11:58 +0100, Vincent Guittot wrote:
>
>> My tests have been done without cpuidle because i have some issues
>> with function tracer and cpuidle
>>
>> But the cpu hotplug and cpuidle work well when I run the tests without
>> enabling the function tracer
>>
>
> I know suspend and resume has issues with function tracing (because it
> makes things like calling smp_processor_id() crash the system), but I'm
> unaware of issues with hotplug itself. Could be some of the same issues.
>
> Can you give me more details, I'll try to investigate it.

yes for sure.
The problem is more linked to cpuidle and function tracer.

cpu hotplug and function tracer work when cpuidle is disable.
cpu hotplug and cpuidle works if i don't enable function tracer.
my platform is dead as soon as I enable function tracer if cpuidle is
enabled. It looks like some notrace are missing in my platform driver
but we haven't completely fix the issue yet

Vincent

>
> Thanks,
>
> -- Steve
>
>

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Steven Rostedt]

On Mon, 2013-02-18 at 17:50 +0100, Vincent Guittot wrote:

> yes for sure.
> The problem is more linked to cpuidle and function tracer.
> 
> cpu hotplug and function tracer work when cpuidle is disable.
> cpu hotplug and cpuidle works if i don't enable function tracer.
> my platform is dead as soon as I enable function tracer if cpuidle is
> enabled. It looks like some notrace are missing in my platform driver
> but we haven't completely fix the issue yet
> 

You can bisect to find out exactly what function is the problem:

 cat /debug/tracing/available_filter_functions > t
 
f(t) {
 num=`wc -l t`
 sed -ne "1,${num}p" t > t1
 let num=num+1
 sed -ne "${num},$p" t > t2

 cat t1 > /debug/tracing/set_ftrace_filter
 # note this may take a long time to finish

 echo function > /debug/tracing/current_tracer

 <failed? bisect f(t1), if not bisect f(t2)>
}

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Steven Rostedt]

On Mon, 2013-02-18 at 17:50 +0100, Vincent Guittot wrote:

> yes for sure.
> The problem is more linked to cpuidle and function tracer.
> 
> cpu hotplug and function tracer work when cpuidle is disable.
> cpu hotplug and cpuidle works if i don't enable function tracer.
> my platform is dead as soon as I enable function tracer if cpuidle is
> enabled. It looks like some notrace are missing in my platform driver
> but we haven't completely fix the issue yet
> 

You can bisect to find out exactly what function is the problem:

 cat /debug/tracing/available_filter_functions > t
 
f(t) {
 num=`wc -l t`
 sed -ne "1,${num}p" t > t1
 let num=num+1
 sed -ne "${num},$p" t > t2

 cat t1 > /debug/tracing/set_ftrace_filter
 # note this may take a long time to finish

 echo function > /debug/tracing/current_tracer

 <failed? bisect f(t1), if not bisect f(t2)>
}

-- Steve

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Vincent Guittot]

On 18 February 2013 20:53, Steven Rostedt <rostedt@goodmis.org> wrote:
> On Mon, 2013-02-18 at 17:50 +0100, Vincent Guittot wrote:
>
>> yes for sure.
>> The problem is more linked to cpuidle and function tracer.
>>
>> cpu hotplug and function tracer work when cpuidle is disable.
>> cpu hotplug and cpuidle works if i don't enable function tracer.
>> my platform is dead as soon as I enable function tracer if cpuidle is
>> enabled. It looks like some notrace are missing in my platform driver
>> but we haven't completely fix the issue yet
>>
>
> You can bisect to find out exactly what function is the problem:
>
>  cat /debug/tracing/available_filter_functions > t
>
> f(t) {
>  num=`wc -l t`
>  sed -ne "1,${num}p" t > t1
>  let num=num+1
>  sed -ne "${num},$p" t > t2
>
>  cat t1 > /debug/tracing/set_ftrace_filter
>  # note this may take a long time to finish
>
>  echo function > /debug/tracing/current_tracer
>
>  <failed? bisect f(t1), if not bisect f(t2)>
> }
>

Thanks, i'm going to have a look

Vincent

> -- Steve
>
>
>

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Thomas Gleixner]

On Mon, 18 Feb 2013, Srivatsa S. Bhat wrote:
> Lockup observed while running this patchset, with CPU_IDLE and INTEL_IDLE turned
> on in the .config:
> 
>  smpboot: CPU 1 is now offline
> Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 11
> Pid: 0, comm: swapper/11 Not tainted 3.8.0-rc7+stpmch13-1 #8
> Call Trace:
>  [<ffffffff812aba1e>] do_raw_spin_lock+0x7e/0x150
>  [<ffffffff815a64c1>] _raw_spin_lock_irqsave+0x61/0x70
>  [<ffffffff810c0758>] ? clockevents_notify+0x28/0x150
>  [<ffffffff815a6d37>] ? _raw_spin_unlock_irqrestore+0x77/0x80
>  [<ffffffff810c0758>] clockevents_notify+0x28/0x150
>  [<ffffffff8130459f>] intel_idle+0xaf/0xe0
>  [<ffffffff81472ee0>] ? disable_cpuidle+0x20/0x20
>  [<ffffffff81472ef9>] cpuidle_enter+0x19/0x20
>  [<ffffffff814734c1>] cpuidle_wrap_enter+0x41/0xa0
>  [<ffffffff81473530>] cpuidle_enter_tk+0x10/0x20
>  [<ffffffff81472f17>] cpuidle_enter_state+0x17/0x50
>  [<ffffffff81473899>] cpuidle_idle_call+0xd9/0x290
>  [<ffffffff810203d5>] cpu_idle+0xe5/0x140
>  [<ffffffff8159c603>] start_secondary+0xdd/0xdf

> BUG: spinlock lockup suspected on CPU#2, migration/2/19
>  lock: clockevents_lock+0x0/0x40, .magic: dead4ead, .owner: swapper/8/0, .owner_cpu: 8

Unfortunately there is no back trace for cpu8. That's probably caused
by the watchdog -> panic setting.

So we have no idea why cpu2 and 11 get stuck on the clockevents_lock
and without that information it's impossible to decode.

Thanks,

	tglx

Re: [PATCH v5 00/45] CPU hotplug: stop_machine()-free CPU hotplug

[Srivatsa S. Bhat]

On 02/18/2013 04:24 PM, Thomas Gleixner wrote:
> On Mon, 18 Feb 2013, Srivatsa S. Bhat wrote:
>> Lockup observed while running this patchset, with CPU_IDLE and INTEL_IDLE turned
>> on in the .config:
>>
>>  smpboot: CPU 1 is now offline
>> Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 11
>> Pid: 0, comm: swapper/11 Not tainted 3.8.0-rc7+stpmch13-1 #8
>> Call Trace:
>>  [<ffffffff812aba1e>] do_raw_spin_lock+0x7e/0x150
>>  [<ffffffff815a64c1>] _raw_spin_lock_irqsave+0x61/0x70
>>  [<ffffffff810c0758>] ? clockevents_notify+0x28/0x150
>>  [<ffffffff815a6d37>] ? _raw_spin_unlock_irqrestore+0x77/0x80
>>  [<ffffffff810c0758>] clockevents_notify+0x28/0x150
>>  [<ffffffff8130459f>] intel_idle+0xaf/0xe0
>>  [<ffffffff81472ee0>] ? disable_cpuidle+0x20/0x20
>>  [<ffffffff81472ef9>] cpuidle_enter+0x19/0x20
>>  [<ffffffff814734c1>] cpuidle_wrap_enter+0x41/0xa0
>>  [<ffffffff81473530>] cpuidle_enter_tk+0x10/0x20
>>  [<ffffffff81472f17>] cpuidle_enter_state+0x17/0x50
>>  [<ffffffff81473899>] cpuidle_idle_call+0xd9/0x290
>>  [<ffffffff810203d5>] cpu_idle+0xe5/0x140
>>  [<ffffffff8159c603>] start_secondary+0xdd/0xdf
> 
>> BUG: spinlock lockup suspected on CPU#2, migration/2/19
>>  lock: clockevents_lock+0x0/0x40, .magic: dead4ead, .owner: swapper/8/0, .owner_cpu: 8
> 
> Unfortunately there is no back trace for cpu8.

Yes :-(

I had run this several times hoping to get a backtrace on the lock-holder,
expecting trigger_all_cpu_backtrace() to get it right at least once. But I
hadn't succeeded even once.

> That's probably caused
> by the watchdog -> panic setting.
> 

Oh, ok..

> So we have no idea why cpu2 and 11 get stuck on the clockevents_lock
> and without that information it's impossible to decode.
> 

But thankfully, the issue seems to have been resolved by the diff I posted
in my previous mail, along with the fixes related to memory barriers.

Regards,
Srivatsa S. Bhat

Re: [PATCH v5 01/45] percpu_rwlock: Introduce the global reader-writer lock backend

[David Howells]

Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com> wrote:

> We can use global rwlocks as shown below safely, without fear of deadlocks:
> 
> Readers:
> 
>          CPU 0                                CPU 1
>          ------                               ------
> 
> 1.    spin_lock(&random_lock);             read_lock(&my_rwlock);
> 
> 
> 2.    read_lock(&my_rwlock);               spin_lock(&random_lock);

The lock order on CPU 0 is unsafe if CPU2 can do:

	write_lock(&my_rwlock);
	spin_lock(&random_lock);

and on CPU 1 if CPU2 can do:

	spin_lock(&random_lock);
	write_lock(&my_rwlock);

I presume you were specifically excluding these situations?

David

Re: [PATCH v5 01/45] percpu_rwlock: Introduce the global reader-writer lock backend

[Srivatsa S. Bhat]

On 02/11/2013 06:11 PM, David Howells wrote:
> Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com> wrote:
> 
>> We can use global rwlocks as shown below safely, without fear of deadlocks:
>>
>> Readers:
>>
>>          CPU 0                                CPU 1
>>          ------                               ------
>>
>> 1.    spin_lock(&random_lock);             read_lock(&my_rwlock);
>>
>>
>> 2.    read_lock(&my_rwlock);               spin_lock(&random_lock);
> 
> The lock order on CPU 0 is unsafe if CPU2 can do:
> 
> 	write_lock(&my_rwlock);
> 	spin_lock(&random_lock);
> 
> and on CPU 1 if CPU2 can do:
> 
> 	spin_lock(&random_lock);
> 	write_lock(&my_rwlock);
> 

Right..

> I presume you were specifically excluding these situations?
>

Yes.. Those cases are simple to find out and fix (by changing the
lock ordering). My main problem was with CPU 0 and CPU 1 as shown above..
... and using a global rwlock helps ease that part out.

Regards,
Srivatsa S. Bhat