* [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask
@ 2008-10-20 17:03 Mike Travis
2008-10-20 17:03 ` [PATCH 01/35] x86: clean up speedctep-centrino and reduce cpumask_t usage Mike Travis
` (34 more replies)
0 siblings, 35 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Redesign cpumask API to explicitly declare struct cpumask pointers to
the cpumask_* operators and add functions to make it easier to support
struct cpumask pointers on the stack.
This patchset supplies the infrastructure going forward to implement
this new struct cpumask API. Patches have been written and tested
to verify the functionality of this API. These are still in review
between Rusty and myself.
Question: what's the status of the sparse-irqs? (I have two patches
to replace the static cpumask fields in the irq_desc and irq_cfg
arrays which depend on sparse irqs.)
Compiled and tested on x86_64.
Based on tip/master @ v2.6.27-5807-g03da394
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 01/35] x86: clean up speedctep-centrino and reduce cpumask_t usage
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-21 0:09 ` Stephen Rothwell
2008-10-20 17:03 ` [PATCH 02/35] cpumask: remove min from first_cpu/next_cpu Mike Travis
` (33 subsequent siblings)
34 siblings, 1 reply; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
1) The #ifdef CONFIG_HOTPLUG_CPU seems unnecessary these days.
2) The loop can simply skip over offline cpus, rather than creating a tmp mask.
3) set_mask is set to either a single cpu or all online cpus in a policy.
Since it's just used for set_cpus_allowed(), any offline cpus in a policy
don't matter, so we can just use cpumask_of_cpu() or the policy->cpus.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
arch/x86/kernel/cpu/cpufreq/speedstep-centrino.c | 51 ++++++++++-------------
1 file changed, 24 insertions(+), 27 deletions(-)
--- linux-2.6.28.orig/arch/x86/kernel/cpu/cpufreq/speedstep-centrino.c
+++ linux-2.6.28/arch/x86/kernel/cpu/cpufreq/speedstep-centrino.c
@@ -552,9 +552,7 @@ static int centrino_verify (struct cpufr
* Sets a new CPUFreq policy.
*/
struct allmasks {
- cpumask_t online_policy_cpus;
cpumask_t saved_mask;
- cpumask_t set_mask;
cpumask_t covered_cpus;
};
@@ -568,9 +566,7 @@ static int centrino_target (struct cpufr
int retval = 0;
unsigned int j, k, first_cpu, tmp;
CPUMASK_ALLOC(allmasks);
- CPUMASK_PTR(online_policy_cpus, allmasks);
CPUMASK_PTR(saved_mask, allmasks);
- CPUMASK_PTR(set_mask, allmasks);
CPUMASK_PTR(covered_cpus, allmasks);
if (unlikely(allmasks == NULL))
@@ -590,30 +586,28 @@ static int centrino_target (struct cpufr
goto out;
}
-#ifdef CONFIG_HOTPLUG_CPU
- /* cpufreq holds the hotplug lock, so we are safe from here on */
- cpus_and(*online_policy_cpus, cpu_online_map, policy->cpus);
-#else
- *online_policy_cpus = policy->cpus;
-#endif
-
*saved_mask = current->cpus_allowed;
first_cpu = 1;
cpus_clear(*covered_cpus);
- for_each_cpu_mask_nr(j, *online_policy_cpus) {
+ for_each_cpu_mask_nr(j, policy->cpus) {
+ const cpumask_t *mask;
+
+ /* cpufreq holds the hotplug lock, so we are safe here */
+ if (!cpu_online(j))
+ continue;
+
/*
* Support for SMP systems.
* Make sure we are running on CPU that wants to change freq
*/
- cpus_clear(*set_mask);
if (policy->shared_type == CPUFREQ_SHARED_TYPE_ANY)
- cpus_or(*set_mask, *set_mask, *online_policy_cpus);
+ mask = &policy->cpus;
else
- cpu_set(j, *set_mask);
+ mask = &cpumask_of_cpu(j);
- set_cpus_allowed_ptr(current, set_mask);
+ set_cpus_allowed_ptr(current, mask);
preempt_disable();
- if (unlikely(!cpu_isset(smp_processor_id(), *set_mask))) {
+ if (unlikely(!cpu_isset(smp_processor_id(), *mask))) {
dprintk("couldn't limit to CPUs in this domain\n");
retval = -EAGAIN;
if (first_cpu) {
@@ -641,7 +635,9 @@ static int centrino_target (struct cpufr
dprintk("target=%dkHz old=%d new=%d msr=%04x\n",
target_freq, freqs.old, freqs.new, msr);
- for_each_cpu_mask_nr(k, *online_policy_cpus) {
+ for_each_cpu_mask_nr(k, policy->cpus) {
+ if (!cpu_online(k))
+ continue;
freqs.cpu = k;
cpufreq_notify_transition(&freqs,
CPUFREQ_PRECHANGE);
@@ -664,7 +660,9 @@ static int centrino_target (struct cpufr
preempt_enable();
}
- for_each_cpu_mask_nr(k, *online_policy_cpus) {
+ for_each_cpu_mask_nr(k, policy->cpus) {
+ if (!cpu_online(k))
+ continue;
freqs.cpu = k;
cpufreq_notify_transition(&freqs, CPUFREQ_POSTCHANGE);
}
@@ -677,18 +675,17 @@ static int centrino_target (struct cpufr
* Best effort undo..
*/
- if (!cpus_empty(*covered_cpus))
- for_each_cpu_mask_nr(j, *covered_cpus) {
- set_cpus_allowed_ptr(current,
- &cpumask_of_cpu(j));
- wrmsr(MSR_IA32_PERF_CTL, oldmsr, h);
- }
+ for_each_cpu_mask_nr(j, *covered_cpus) {
+ set_cpus_allowed_ptr(current, &cpumask_of_cpu(j));
+ wrmsr(MSR_IA32_PERF_CTL, oldmsr, h);
+ }
tmp = freqs.new;
freqs.new = freqs.old;
freqs.old = tmp;
- for_each_cpu_mask_nr(j, *online_policy_cpus) {
- freqs.cpu = j;
+ for_each_cpu_mask_nr(j, policy->cpus) {
+ if (!cpu_online(j))
+ continue;
cpufreq_notify_transition(&freqs, CPUFREQ_PRECHANGE);
cpufreq_notify_transition(&freqs, CPUFREQ_POSTCHANGE);
}
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 02/35] cpumask: remove min from first_cpu/next_cpu
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
2008-10-20 17:03 ` [PATCH 01/35] x86: clean up speedctep-centrino and reduce cpumask_t usage Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 03/35] cpumask: add for_each_cpu_mask_and function Mike Travis
` (32 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Seems like this has been here forever, but I can't see why:
find_first_bit and find_next_bit both return >= NR_CPUS on failure.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
lib/cpumask.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
--- linux-2.6.28.orig/lib/cpumask.c
+++ linux-2.6.28/lib/cpumask.c
@@ -5,21 +5,20 @@
int __first_cpu(const cpumask_t *srcp)
{
- return min_t(int, NR_CPUS, find_first_bit(srcp->bits, NR_CPUS));
+ return find_first_bit(srcp->bits, NR_CPUS);
}
EXPORT_SYMBOL(__first_cpu);
int __next_cpu(int n, const cpumask_t *srcp)
{
- return min_t(int, NR_CPUS, find_next_bit(srcp->bits, NR_CPUS, n+1));
+ return find_next_bit(srcp->bits, NR_CPUS, n+1);
}
EXPORT_SYMBOL(__next_cpu);
#if NR_CPUS > 64
int __next_cpu_nr(int n, const cpumask_t *srcp)
{
- return min_t(int, nr_cpu_ids,
- find_next_bit(srcp->bits, nr_cpu_ids, n+1));
+ return find_next_bit(srcp->bits, nr_cpu_ids, n+1);
}
EXPORT_SYMBOL(__next_cpu_nr);
#endif
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 03/35] cpumask: add for_each_cpu_mask_and function
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
2008-10-20 17:03 ` [PATCH 01/35] x86: clean up speedctep-centrino and reduce cpumask_t usage Mike Travis
2008-10-20 17:03 ` [PATCH 02/35] cpumask: remove min from first_cpu/next_cpu Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 04/35] x86 smp: modify send_IPI_mask interface to accept cpumask_t pointers Mike Travis
` (31 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Add for_each_cpu_mask_and() function to eliminate need for a common use
of a temporary cpumask_t variable. When the following procedure is being
used:
funcproto(const cpumask_t *mask, ...)
{
cpumask_t temp;
cpus_and(temp, mask, cpu_online_map);
for_each_cpu_mask(cpu, temp)
...
It then becomes:
funcproto(cpumask_t *mask, ...)
{
for_each_cpu_mask_and(cpu, *mask, cpu_online_map)
...
... eliminating the need for the temp cpumask.
Applies to linux-2.6.tip/master.
Signed-off-by: Mike Travis <travis@sgi.com>
Acked-by: Rusty Russell <rusty@rustcorp.com.au>
---
include/linux/cpumask.h | 33 ++++++++++++++++++++++++---------
lib/cpumask.c | 9 +++++++++
2 files changed, 33 insertions(+), 9 deletions(-)
--- test-compile.orig/include/linux/cpumask.h
+++ test-compile/include/linux/cpumask.h
@@ -109,6 +109,7 @@
*
* for_each_cpu_mask(cpu, mask) for-loop cpu over mask using NR_CPUS
* for_each_cpu_mask_nr(cpu, mask) for-loop cpu over mask using nr_cpu_ids
+ * for_each_cpu_mask_and(cpu, mask, and) for-loop cpu over (mask & and).
*
* int num_online_cpus() Number of online CPUs
* int num_possible_cpus() Number of all possible CPUs
@@ -400,29 +401,41 @@ static inline void __cpus_fold(cpumask_t
#if NR_CPUS == 1
-#define nr_cpu_ids 1
-#define first_cpu(src) ({ (void)(src); 0; })
-#define next_cpu(n, src) ({ (void)(src); 1; })
-#define any_online_cpu(mask) 0
-#define for_each_cpu_mask(cpu, mask) \
+#define nr_cpu_ids 1
+#define first_cpu(src) ({ (void)(src); 0; })
+#define next_cpu(n, src) ({ (void)(src); 1; })
+#define cpumask_next_and(n, srcp, andp) ({ (void)(srcp), (void)(andp); 1; })
+#define any_online_cpu(mask) 0
+
+#define for_each_cpu_mask(cpu, mask) \
for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask)
+#define for_each_cpu_mask_and(cpu, mask, and) \
+ for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask, (void)and)
#else /* NR_CPUS > 1 */
extern int nr_cpu_ids;
int __first_cpu(const cpumask_t *srcp);
int __next_cpu(int n, const cpumask_t *srcp);
+int cpumask_next_and(int n, const cpumask_t *srcp, const cpumask_t *andp);
int __any_online_cpu(const cpumask_t *mask);
#define first_cpu(src) __first_cpu(&(src))
#define next_cpu(n, src) __next_cpu((n), &(src))
#define any_online_cpu(mask) __any_online_cpu(&(mask))
+
#define for_each_cpu_mask(cpu, mask) \
for ((cpu) = -1; \
(cpu) = next_cpu((cpu), (mask)), \
- (cpu) < NR_CPUS; )
+ (cpu) < NR_CPUS;)
+#define for_each_cpu_mask_and(cpu, mask, and) \
+ for ((cpu) = -1; \
+ (cpu) = cpumask_next_and((cpu), &(mask), &(and)), \
+ (cpu) < nr_cpu_ids;)
#endif
+#define cpumask_first_and(mask, and) cpumask_next_and(-1, (mask), (and))
+
#if NR_CPUS <= 64
#define next_cpu_nr(n, src) next_cpu(n, src)
@@ -432,12 +445,14 @@ int __any_online_cpu(const cpumask_t *ma
#else /* NR_CPUS > 64 */
int __next_cpu_nr(int n, const cpumask_t *srcp);
-#define next_cpu_nr(n, src) __next_cpu_nr((n), &(src))
-#define cpus_weight_nr(cpumask) __cpus_weight(&(cpumask), nr_cpu_ids)
+
+#define next_cpu_nr(n, src) __next_cpu_nr((n), &(src))
+#define cpus_weight_nr(cpumask) __cpus_weight(&(cpumask), nr_cpu_ids)
+
#define for_each_cpu_mask_nr(cpu, mask) \
for ((cpu) = -1; \
(cpu) = next_cpu_nr((cpu), (mask)), \
- (cpu) < nr_cpu_ids; )
+ (cpu) < nr_cpu_ids;)
#endif /* NR_CPUS > 64 */
--- test-compile.orig/lib/cpumask.c
+++ test-compile/lib/cpumask.c
@@ -15,6 +15,15 @@ int __next_cpu(int n, const cpumask_t *s
}
EXPORT_SYMBOL(__next_cpu);
+int cpumask_next_and(int n, const cpumask_t *srcp, const cpumask_t *andp)
+{
+ while ((n = next_cpu_nr(n, *srcp)) < nr_cpu_ids)
+ if (cpu_isset(n, *andp))
+ break;
+ return n;
+}
+EXPORT_SYMBOL(cpumask_next_and);
+
#if NR_CPUS > 64
int __next_cpu_nr(int n, const cpumask_t *srcp)
{
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 04/35] x86 smp: modify send_IPI_mask interface to accept cpumask_t pointers
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (2 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 03/35] cpumask: add for_each_cpu_mask_and function Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 05/35] sched: Reduce stack size requirements in kernel/sched.c Mike Travis
` (30 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
* Change genapic interfaces to accept cpumask_t pointers where possible.
* Modify external callers to use cpumask_t pointers in function calls.
* Create new send_IPI_mask_allbutself which is the same as the
send_IPI_mask functions but removes smp_processor_id() from list.
This removes another common need for a temporary cpumask_t variable.
* Functions that used a temp cpumask_t variable for:
cpumask_t allbutme = cpu_online_map;
cpu_clear(smp_processor_id(), allbutme);
if (!cpus_empty(allbutme))
...
become:
if (!cpus_equal(cpu_online_map, cpumask_of_cpu(cpu)))
...
* Other minor code optimizations (like using cpus_clear instead of
CPU_MASK_NONE, etc.)
Applies to linux-2.6.tip/master.
Signed-off-by: Mike Travis <travis@sgi.com>
Acked-by: Rusty Russell <rusty@rustcorp.com.au>
---
arch/x86/kernel/apic.c | 2
arch/x86/kernel/crash.c | 5 -
arch/x86/kernel/genapic_flat_64.c | 76 +++++++++++++------
arch/x86/kernel/genx2apic_cluster.c | 60 ++++++++++-----
arch/x86/kernel/genx2apic_phys.c | 54 +++++++++-----
arch/x86/kernel/genx2apic_uv_x.c | 43 ++++++-----
arch/x86/kernel/io_apic.c | 118 ++++++++++++++-----------------
arch/x86/kernel/ipi.c | 26 ++++--
arch/x86/kernel/smp.c | 8 +-
arch/x86/kernel/tlb_32.c | 2
arch/x86/kernel/tlb_64.c | 2
arch/x86/mach-generic/bigsmp.c | 5 -
arch/x86/mach-generic/es7000.c | 5 -
arch/x86/mach-generic/numaq.c | 5 -
arch/x86/mach-generic/summit.c | 5 -
arch/x86/xen/smp.c | 17 ++--
include/asm-x86/bigsmp/apic.h | 14 +--
include/asm-x86/bigsmp/ipi.h | 9 +-
include/asm-x86/es7000/apic.h | 18 ++--
include/asm-x86/es7000/ipi.h | 9 +-
include/asm-x86/genapic_32.h | 11 +-
include/asm-x86/genapic_64.h | 11 +-
include/asm-x86/ipi.h | 21 ++++-
include/asm-x86/mach-default/mach_apic.h | 17 ++--
include/asm-x86/mach-default/mach_ipi.h | 18 ++--
include/asm-x86/numaq/apic.h | 4 -
include/asm-x86/numaq/ipi.h | 9 +-
include/asm-x86/smp.h | 6 -
include/asm-x86/summit/apic.h | 12 +--
include/asm-x86/summit/ipi.h | 9 +-
30 files changed, 350 insertions(+), 251 deletions(-)
--- linux-2.6.28.orig/arch/x86/kernel/apic.c
+++ linux-2.6.28/arch/x86/kernel/apic.c
@@ -456,7 +456,7 @@ static void lapic_timer_setup(enum clock
static void lapic_timer_broadcast(cpumask_t mask)
{
#ifdef CONFIG_SMP
- send_IPI_mask(mask, LOCAL_TIMER_VECTOR);
+ send_IPI_mask(&mask, LOCAL_TIMER_VECTOR);
#endif
}
--- linux-2.6.28.orig/arch/x86/kernel/crash.c
+++ linux-2.6.28/arch/x86/kernel/crash.c
@@ -77,10 +77,7 @@ static int crash_nmi_callback(struct not
static void smp_send_nmi_allbutself(void)
{
- cpumask_t mask = cpu_online_map;
- cpu_clear(safe_smp_processor_id(), mask);
- if (!cpus_empty(mask))
- send_IPI_mask(mask, NMI_VECTOR);
+ send_IPI_allbutself(NMI_VECTOR);
}
static struct notifier_block crash_nmi_nb = {
--- linux-2.6.28.orig/arch/x86/kernel/genapic_flat_64.c
+++ linux-2.6.28/arch/x86/kernel/genapic_flat_64.c
@@ -30,12 +30,12 @@ static int __init flat_acpi_madt_oem_che
return 1;
}
-static cpumask_t flat_target_cpus(void)
+static const cpumask_t *flat_target_cpus(void)
{
- return cpu_online_map;
+ return &cpu_online_map;
}
-static cpumask_t flat_vector_allocation_domain(int cpu)
+static void flat_vector_allocation_domain(int cpu, cpumask_t *retmask)
{
/* Careful. Some cpus do not strictly honor the set of cpus
* specified in the interrupt destination when using lowest
@@ -45,8 +45,7 @@ static cpumask_t flat_vector_allocation_
* deliver interrupts to the wrong hyperthread when only one
* hyperthread was specified in the interrupt desitination.
*/
- cpumask_t domain = { { [0] = APIC_ALL_CPUS, } };
- return domain;
+ *retmask = (cpumask_t) { {[0] = APIC_ALL_CPUS, } };
}
/*
@@ -69,9 +68,8 @@ static void flat_init_apic_ldr(void)
apic_write(APIC_LDR, val);
}
-static void flat_send_IPI_mask(cpumask_t cpumask, int vector)
+static inline void _flat_send_IPI_mask(unsigned long mask, int vector)
{
- unsigned long mask = cpus_addr(cpumask)[0];
unsigned long flags;
local_irq_save(flags);
@@ -79,20 +77,40 @@ static void flat_send_IPI_mask(cpumask_t
local_irq_restore(flags);
}
+static void flat_send_IPI_mask(const cpumask_t *cpumask, int vector)
+{
+ unsigned long mask = cpus_addr(*cpumask)[0];
+
+ _flat_send_IPI_mask(mask, vector);
+}
+
+static void flat_send_IPI_mask_allbutself(const cpumask_t *cpumask, int vector)
+{
+ unsigned long mask = cpus_addr(*cpumask)[0];
+ int cpu = smp_processor_id();
+
+ if (cpu < BITS_PER_LONG)
+ clear_bit(cpu, &mask);
+ _flat_send_IPI_mask(mask, vector);
+}
+
static void flat_send_IPI_allbutself(int vector)
{
+ int cpu = smp_processor_id();
#ifdef CONFIG_HOTPLUG_CPU
int hotplug = 1;
#else
int hotplug = 0;
#endif
if (hotplug || vector == NMI_VECTOR) {
- cpumask_t allbutme = cpu_online_map;
+ if (!cpus_equal(cpu_online_map, cpumask_of_cpu(cpu))) {
+ unsigned long mask = cpus_addr(cpu_online_map)[0];
- cpu_clear(smp_processor_id(), allbutme);
+ if (cpu < BITS_PER_LONG)
+ clear_bit(cpu, &mask);
- if (!cpus_empty(allbutme))
- flat_send_IPI_mask(allbutme, vector);
+ _flat_send_IPI_mask(mask, vector);
+ }
} else if (num_online_cpus() > 1) {
__send_IPI_shortcut(APIC_DEST_ALLBUT, vector,APIC_DEST_LOGICAL);
}
@@ -101,7 +119,7 @@ static void flat_send_IPI_allbutself(int
static void flat_send_IPI_all(int vector)
{
if (vector == NMI_VECTOR)
- flat_send_IPI_mask(cpu_online_map, vector);
+ flat_send_IPI_mask(&cpu_online_map, vector);
else
__send_IPI_shortcut(APIC_DEST_ALLINC, vector, APIC_DEST_LOGICAL);
}
@@ -135,9 +153,9 @@ static int flat_apic_id_registered(void)
return physid_isset(read_xapic_id(), phys_cpu_present_map);
}
-static unsigned int flat_cpu_mask_to_apicid(cpumask_t cpumask)
+static unsigned int flat_cpu_mask_to_apicid(const cpumask_t *cpumask)
{
- return cpus_addr(cpumask)[0] & APIC_ALL_CPUS;
+ return cpus_addr(*cpumask)[0] & APIC_ALL_CPUS;
}
static unsigned int phys_pkg_id(int index_msb)
@@ -157,6 +175,7 @@ struct genapic apic_flat = {
.send_IPI_all = flat_send_IPI_all,
.send_IPI_allbutself = flat_send_IPI_allbutself,
.send_IPI_mask = flat_send_IPI_mask,
+ .send_IPI_mask_allbutself = flat_send_IPI_mask_allbutself,
.send_IPI_self = apic_send_IPI_self,
.cpu_mask_to_apicid = flat_cpu_mask_to_apicid,
.phys_pkg_id = phys_pkg_id,
@@ -188,35 +207,39 @@ static int __init physflat_acpi_madt_oem
return 0;
}
-static cpumask_t physflat_target_cpus(void)
+static const cpumask_t *physflat_target_cpus(void)
{
- return cpu_online_map;
+ return &cpu_online_map;
}
-static cpumask_t physflat_vector_allocation_domain(int cpu)
+static void physflat_vector_allocation_domain(int cpu, cpumask_t *retmask)
{
- return cpumask_of_cpu(cpu);
+ cpus_clear(*retmask);
+ cpu_set(cpu, *retmask);
}
-static void physflat_send_IPI_mask(cpumask_t cpumask, int vector)
+static void physflat_send_IPI_mask(const cpumask_t *cpumask, int vector)
{
send_IPI_mask_sequence(cpumask, vector);
}
-static void physflat_send_IPI_allbutself(int vector)
+static void physflat_send_IPI_mask_allbutself(const cpumask_t *cpumask,
+ int vector)
{
- cpumask_t allbutme = cpu_online_map;
+ send_IPI_mask_allbutself(cpumask, vector);
+}
- cpu_clear(smp_processor_id(), allbutme);
- physflat_send_IPI_mask(allbutme, vector);
+static void physflat_send_IPI_allbutself(int vector)
+{
+ send_IPI_mask_allbutself(&cpu_online_map, vector);
}
static void physflat_send_IPI_all(int vector)
{
- physflat_send_IPI_mask(cpu_online_map, vector);
+ physflat_send_IPI_mask(&cpu_online_map, vector);
}
-static unsigned int physflat_cpu_mask_to_apicid(cpumask_t cpumask)
+static unsigned int physflat_cpu_mask_to_apicid(const cpumask_t *cpumask)
{
int cpu;
@@ -224,7 +247,7 @@ static unsigned int physflat_cpu_mask_to
* We're using fixed IRQ delivery, can only return one phys APIC ID.
* May as well be the first.
*/
- cpu = first_cpu(cpumask);
+ cpu = first_cpu(*cpumask);
if ((unsigned)cpu < nr_cpu_ids)
return per_cpu(x86_cpu_to_apicid, cpu);
else
@@ -243,6 +266,7 @@ struct genapic apic_physflat = {
.send_IPI_all = physflat_send_IPI_all,
.send_IPI_allbutself = physflat_send_IPI_allbutself,
.send_IPI_mask = physflat_send_IPI_mask,
+ .send_IPI_mask_allbutself = physflat_send_IPI_mask_allbutself,
.send_IPI_self = apic_send_IPI_self,
.cpu_mask_to_apicid = physflat_cpu_mask_to_apicid,
.phys_pkg_id = phys_pkg_id,
--- linux-2.6.28.orig/arch/x86/kernel/genx2apic_cluster.c
+++ linux-2.6.28/arch/x86/kernel/genx2apic_cluster.c
@@ -22,19 +22,18 @@ static int __init x2apic_acpi_madt_oem_c
/* Start with all IRQs pointing to boot CPU. IRQ balancing will shift them. */
-static cpumask_t x2apic_target_cpus(void)
+static const cpumask_t *x2apic_target_cpus(void)
{
- return cpumask_of_cpu(0);
+ return &cpumask_of_cpu(0);
}
/*
* for now each logical cpu is in its own vector allocation domain.
*/
-static cpumask_t x2apic_vector_allocation_domain(int cpu)
+static void x2apic_vector_allocation_domain(int cpu, cpumask_t *retmask)
{
- cpumask_t domain = CPU_MASK_NONE;
- cpu_set(cpu, domain);
- return domain;
+ cpus_clear(*retmask);
+ cpu_set(cpu, *retmask);
}
static void __x2apic_send_IPI_dest(unsigned int apicid, int vector,
@@ -56,32 +55,52 @@ static void __x2apic_send_IPI_dest(unsig
* at once. We have 16 cpu's in a cluster. This will minimize IPI register
* writes.
*/
-static void x2apic_send_IPI_mask(cpumask_t mask, int vector)
+static void x2apic_send_IPI_mask(const cpumask_t *mask, int vector)
{
unsigned long flags;
unsigned long query_cpu;
local_irq_save(flags);
- for_each_cpu_mask(query_cpu, mask) {
- __x2apic_send_IPI_dest(per_cpu(x86_cpu_to_logical_apicid, query_cpu),
- vector, APIC_DEST_LOGICAL);
- }
+ for_each_cpu_mask_nr(query_cpu, *mask)
+ __x2apic_send_IPI_dest(
+ per_cpu(x86_cpu_to_logical_apicid, query_cpu),
+ vector, APIC_DEST_LOGICAL);
local_irq_restore(flags);
}
-static void x2apic_send_IPI_allbutself(int vector)
+static void x2apic_send_IPI_mask_allbutself(const cpumask_t *mask, int vector)
{
- cpumask_t mask = cpu_online_map;
+ unsigned long flags;
+ unsigned long query_cpu;
+ unsigned long this_cpu = smp_processor_id();
- cpu_clear(smp_processor_id(), mask);
+ local_irq_save(flags);
+ for_each_cpu_mask_nr(query_cpu, *mask)
+ if (query_cpu != this_cpu)
+ __x2apic_send_IPI_dest(
+ per_cpu(x86_cpu_to_logical_apicid, query_cpu),
+ vector, APIC_DEST_LOGICAL);
+ local_irq_restore(flags);
+}
- if (!cpus_empty(mask))
- x2apic_send_IPI_mask(mask, vector);
+static void x2apic_send_IPI_allbutself(int vector)
+{
+ unsigned long flags;
+ unsigned long query_cpu;
+ unsigned long this_cpu = smp_processor_id();
+
+ local_irq_save(flags);
+ for_each_online_cpu(query_cpu)
+ if (query_cpu != this_cpu)
+ __x2apic_send_IPI_dest(
+ per_cpu(x86_cpu_to_logical_apicid, query_cpu),
+ vector, APIC_DEST_LOGICAL);
+ local_irq_restore(flags);
}
static void x2apic_send_IPI_all(int vector)
{
- x2apic_send_IPI_mask(cpu_online_map, vector);
+ x2apic_send_IPI_mask(&cpu_online_map, vector);
}
static int x2apic_apic_id_registered(void)
@@ -89,7 +108,7 @@ static int x2apic_apic_id_registered(voi
return 1;
}
-static unsigned int x2apic_cpu_mask_to_apicid(cpumask_t cpumask)
+static unsigned int x2apic_cpu_mask_to_apicid(const cpumask_t *cpumask)
{
int cpu;
@@ -97,8 +116,8 @@ static unsigned int x2apic_cpu_mask_to_a
* We're using fixed IRQ delivery, can only return one phys APIC ID.
* May as well be the first.
*/
- cpu = first_cpu(cpumask);
- if ((unsigned)cpu < NR_CPUS)
+ cpu = first_cpu(*cpumask);
+ if ((unsigned)cpu < nr_cpu_ids)
return per_cpu(x86_cpu_to_logical_apicid, cpu);
else
return BAD_APICID;
@@ -150,6 +169,7 @@ struct genapic apic_x2apic_cluster = {
.send_IPI_all = x2apic_send_IPI_all,
.send_IPI_allbutself = x2apic_send_IPI_allbutself,
.send_IPI_mask = x2apic_send_IPI_mask,
+ .send_IPI_mask_allbutself = x2apic_send_IPI_mask_allbutself,
.send_IPI_self = x2apic_send_IPI_self,
.cpu_mask_to_apicid = x2apic_cpu_mask_to_apicid,
.phys_pkg_id = phys_pkg_id,
--- linux-2.6.28.orig/arch/x86/kernel/genx2apic_phys.c
+++ linux-2.6.28/arch/x86/kernel/genx2apic_phys.c
@@ -29,16 +29,15 @@ static int __init x2apic_acpi_madt_oem_c
/* Start with all IRQs pointing to boot CPU. IRQ balancing will shift them. */
-static cpumask_t x2apic_target_cpus(void)
+static const cpumask_t *x2apic_target_cpus(void)
{
- return cpumask_of_cpu(0);
+ return &cpumask_of_cpu(0);
}
-static cpumask_t x2apic_vector_allocation_domain(int cpu)
+static void x2apic_vector_allocation_domain(int cpu, cpumask_t *retmask)
{
- cpumask_t domain = CPU_MASK_NONE;
- cpu_set(cpu, domain);
- return domain;
+ cpus_clear(*retmask);
+ cpu_set(cpu, *retmask);
}
static void __x2apic_send_IPI_dest(unsigned int apicid, int vector,
@@ -54,32 +53,52 @@ static void __x2apic_send_IPI_dest(unsig
x2apic_icr_write(cfg, apicid);
}
-static void x2apic_send_IPI_mask(cpumask_t mask, int vector)
+static void x2apic_send_IPI_mask(const cpumask_t *mask, int vector)
{
unsigned long flags;
unsigned long query_cpu;
local_irq_save(flags);
- for_each_cpu_mask(query_cpu, mask) {
+ for_each_cpu_mask_nr(query_cpu, *mask) {
__x2apic_send_IPI_dest(per_cpu(x86_cpu_to_apicid, query_cpu),
vector, APIC_DEST_PHYSICAL);
}
local_irq_restore(flags);
}
-static void x2apic_send_IPI_allbutself(int vector)
+static void x2apic_send_IPI_mask_allbutself(const cpumask_t *mask, int vector)
{
- cpumask_t mask = cpu_online_map;
+ unsigned long flags;
+ unsigned long query_cpu;
+ unsigned long this_cpu = smp_processor_id();
+
+ local_irq_save(flags);
+ for_each_cpu_mask_nr(query_cpu, *mask) {
+ if (query_cpu != this_cpu)
+ __x2apic_send_IPI_dest(
+ per_cpu(x86_cpu_to_apicid, query_cpu),
+ vector, APIC_DEST_PHYSICAL);
+ }
+ local_irq_restore(flags);
+}
- cpu_clear(smp_processor_id(), mask);
+static void x2apic_send_IPI_allbutself(int vector)
+{
+ unsigned long flags;
+ unsigned long query_cpu;
+ unsigned long this_cpu = smp_processor_id();
- if (!cpus_empty(mask))
- x2apic_send_IPI_mask(mask, vector);
+ local_irq_save(flags);
+ for_each_online_cpu(query_cpu)
+ if (query_cpu != this_cpu)
+ __x2apic_send_IPI_dest(per_cpu(x86_cpu_to_apicid, query_cpu),
+ vector, APIC_DEST_PHYSICAL);
+ local_irq_restore(flags);
}
static void x2apic_send_IPI_all(int vector)
{
- x2apic_send_IPI_mask(cpu_online_map, vector);
+ x2apic_send_IPI_mask(&cpu_online_map, vector);
}
static int x2apic_apic_id_registered(void)
@@ -87,7 +106,7 @@ static int x2apic_apic_id_registered(voi
return 1;
}
-static unsigned int x2apic_cpu_mask_to_apicid(cpumask_t cpumask)
+static unsigned int x2apic_cpu_mask_to_apicid(const cpumask_t *cpumask)
{
int cpu;
@@ -95,8 +114,8 @@ static unsigned int x2apic_cpu_mask_to_a
* We're using fixed IRQ delivery, can only return one phys APIC ID.
* May as well be the first.
*/
- cpu = first_cpu(cpumask);
- if ((unsigned)cpu < NR_CPUS)
+ cpu = first_cpu(*cpumask);
+ if ((unsigned)cpu < nr_cpu_ids)
return per_cpu(x86_cpu_to_apicid, cpu);
else
return BAD_APICID;
@@ -145,6 +164,7 @@ struct genapic apic_x2apic_phys = {
.send_IPI_all = x2apic_send_IPI_all,
.send_IPI_allbutself = x2apic_send_IPI_allbutself,
.send_IPI_mask = x2apic_send_IPI_mask,
+ .send_IPI_mask_allbutself = x2apic_send_IPI_mask_allbutself,
.send_IPI_self = x2apic_send_IPI_self,
.cpu_mask_to_apicid = x2apic_cpu_mask_to_apicid,
.phys_pkg_id = phys_pkg_id,
--- linux-2.6.28.orig/arch/x86/kernel/genx2apic_uv_x.c
+++ linux-2.6.28/arch/x86/kernel/genx2apic_uv_x.c
@@ -76,16 +76,15 @@ EXPORT_SYMBOL(sn_rtc_cycles_per_second);
/* Start with all IRQs pointing to boot CPU. IRQ balancing will shift them. */
-static cpumask_t uv_target_cpus(void)
+static const cpumask_t *uv_target_cpus(void)
{
- return cpumask_of_cpu(0);
+ return &cpumask_of_cpu(0);
}
-static cpumask_t uv_vector_allocation_domain(int cpu)
+static void uv_vector_allocation_domain(int cpu, cpumask_t *retmask)
{
- cpumask_t domain = CPU_MASK_NONE;
- cpu_set(cpu, domain);
- return domain;
+ cpus_clear(*retmask);
+ cpu_set(cpu, *retmask);
}
int uv_wakeup_secondary(int phys_apicid, unsigned int start_rip)
@@ -124,28 +123,37 @@ static void uv_send_IPI_one(int cpu, int
uv_write_global_mmr64(pnode, UVH_IPI_INT, val);
}
-static void uv_send_IPI_mask(cpumask_t mask, int vector)
+static void uv_send_IPI_mask(const cpumask_t *mask, int vector)
{
unsigned int cpu;
- for_each_possible_cpu(cpu)
- if (cpu_isset(cpu, mask))
+ for_each_cpu_mask_nr(cpu, *mask)
+ uv_send_IPI_one(cpu, vector);
+}
+
+static void uv_send_IPI_mask_allbutself(const cpumask_t *mask, int vector)
+{
+ unsigned int cpu;
+ unsigned int this_cpu = smp_processor_id();
+
+ for_each_cpu_mask_nr(cpu, *mask)
+ if (cpu != this_cpu)
uv_send_IPI_one(cpu, vector);
}
static void uv_send_IPI_allbutself(int vector)
{
- cpumask_t mask = cpu_online_map;
-
- cpu_clear(smp_processor_id(), mask);
+ unsigned int cpu;
+ unsigned int this_cpu = smp_processor_id();
- if (!cpus_empty(mask))
- uv_send_IPI_mask(mask, vector);
+ for_each_online_cpu(cpu)
+ if (cpu != this_cpu)
+ uv_send_IPI_one(cpu, vector);
}
static void uv_send_IPI_all(int vector)
{
- uv_send_IPI_mask(cpu_online_map, vector);
+ uv_send_IPI_mask(&cpu_online_map, vector);
}
static int uv_apic_id_registered(void)
@@ -157,7 +165,7 @@ static void uv_init_apic_ldr(void)
{
}
-static unsigned int uv_cpu_mask_to_apicid(cpumask_t cpumask)
+static unsigned int uv_cpu_mask_to_apicid(const cpumask_t *cpumask)
{
int cpu;
@@ -165,7 +173,7 @@ static unsigned int uv_cpu_mask_to_apici
* We're using fixed IRQ delivery, can only return one phys APIC ID.
* May as well be the first.
*/
- cpu = first_cpu(cpumask);
+ cpu = first_cpu(*cpumask);
if ((unsigned)cpu < nr_cpu_ids)
return per_cpu(x86_cpu_to_apicid, cpu);
else
@@ -219,6 +227,7 @@ struct genapic apic_x2apic_uv_x = {
.send_IPI_all = uv_send_IPI_all,
.send_IPI_allbutself = uv_send_IPI_allbutself,
.send_IPI_mask = uv_send_IPI_mask,
+ .send_IPI_mask_allbutself = uv_send_IPI_mask_allbutself,
.send_IPI_self = uv_send_IPI_self,
.cpu_mask_to_apicid = uv_cpu_mask_to_apicid,
.phys_pkg_id = phys_pkg_id,
--- linux-2.6.28.orig/arch/x86/kernel/io_apic.c
+++ linux-2.6.28/arch/x86/kernel/io_apic.c
@@ -531,7 +531,7 @@ static void __target_IO_APIC_irq(unsigne
}
}
-static int assign_irq_vector(int irq, cpumask_t mask);
+static int assign_irq_vector(int irq, const cpumask_t *mask);
static void set_ioapic_affinity_irq(unsigned int irq, cpumask_t mask)
{
@@ -545,12 +545,12 @@ static void set_ioapic_affinity_irq(unsi
if (cpus_empty(tmp))
return;
- cfg = irq_cfg(irq);
- if (assign_irq_vector(irq, mask))
+ if (assign_irq_vector(irq, &mask))
return;
+ cfg = irq_cfg(irq);
cpus_and(tmp, cfg->domain, mask);
- dest = cpu_mask_to_apicid(tmp);
+ dest = cpu_mask_to_apicid(&tmp);
/*
* Only the high 8 bits are valid.
*/
@@ -1206,7 +1206,7 @@ void unlock_vector_lock(void)
spin_unlock(&vector_lock);
}
-static int __assign_irq_vector(int irq, cpumask_t mask)
+static int __assign_irq_vector(int irq, const cpumask_t *mask)
{
/*
* NOTE! The local APIC isn't very good at handling
@@ -1223,37 +1223,33 @@ static int __assign_irq_vector(int irq,
unsigned int old_vector;
int cpu;
struct irq_cfg *cfg;
+ cpumask_t tmp_mask;
cfg = irq_cfg(irq);
-
- /* Only try and allocate irqs on cpus that are present */
- cpus_and(mask, mask, cpu_online_map);
-
if ((cfg->move_in_progress) || cfg->move_cleanup_count)
return -EBUSY;
old_vector = cfg->vector;
if (old_vector) {
- cpumask_t tmp;
- cpus_and(tmp, cfg->domain, mask);
- if (!cpus_empty(tmp))
+ cpus_and(tmp_mask, *mask, cpu_online_map);
+ cpus_and(tmp_mask, cfg->domain, tmp_mask);
+ if (!cpus_empty(tmp_mask))
return 0;
}
- for_each_cpu_mask_nr(cpu, mask) {
- cpumask_t domain, new_mask;
+ /* Only try and allocate irqs on cpus that are present */
+ for_each_cpu_mask_and(cpu, *mask, cpu_online_map) {
int new_cpu;
int vector, offset;
- domain = vector_allocation_domain(cpu);
- cpus_and(new_mask, domain, cpu_online_map);
+ vector_allocation_domain(cpu, &tmp_mask);
vector = current_vector;
offset = current_offset;
next:
vector += 8;
if (vector >= first_system_vector) {
- /* If we run out of vectors on large boxen, must share them. */
+ /* If out of vectors on large boxen, must share them. */
offset = (offset + 1) % 8;
vector = FIRST_DEVICE_VECTOR + offset;
}
@@ -1266,7 +1262,7 @@ next:
if (vector == SYSCALL_VECTOR)
goto next;
#endif
- for_each_cpu_mask_nr(new_cpu, new_mask)
+ for_each_cpu_mask_and(new_cpu, tmp_mask, cpu_online_map)
if (per_cpu(vector_irq, new_cpu)[vector] != -1)
goto next;
/* Found one! */
@@ -1276,16 +1272,16 @@ next:
cfg->move_in_progress = 1;
cfg->old_domain = cfg->domain;
}
- for_each_cpu_mask_nr(new_cpu, new_mask)
+ for_each_cpu_mask_and(new_cpu, tmp_mask, cpu_online_map)
per_cpu(vector_irq, new_cpu)[vector] = irq;
cfg->vector = vector;
- cfg->domain = domain;
+ cfg->domain = tmp_mask;
return 0;
}
return -ENOSPC;
}
-static int assign_irq_vector(int irq, cpumask_t mask)
+static int assign_irq_vector(int irq, const cpumask_t *mask)
{
int err;
unsigned long flags;
@@ -1485,8 +1481,8 @@ static void setup_IO_APIC_irq(int apic,
cfg = irq_cfg(irq);
- mask = TARGET_CPUS;
- if (assign_irq_vector(irq, mask))
+ mask = *TARGET_CPUS;
+ if (assign_irq_vector(irq, &mask))
return;
cpus_and(mask, cfg->domain, mask);
@@ -1499,7 +1495,7 @@ static void setup_IO_APIC_irq(int apic,
if (setup_ioapic_entry(mp_ioapics[apic].mp_apicid, irq, &entry,
- cpu_mask_to_apicid(mask), trigger, polarity,
+ cpu_mask_to_apicid(&mask), trigger, polarity,
cfg->vector)) {
printk("Failed to setup ioapic entry for ioapic %d, pin %d\n",
mp_ioapics[apic].mp_apicid, pin);
@@ -2205,7 +2201,7 @@ static int ioapic_retrigger_irq(unsigned
unsigned long flags;
spin_lock_irqsave(&vector_lock, flags);
- send_IPI_mask(cpumask_of_cpu(first_cpu(cfg->domain)), cfg->vector);
+ send_IPI_mask(&cpumask_of_cpu(first_cpu(cfg->domain)), cfg->vector);
spin_unlock_irqrestore(&vector_lock, flags);
return 1;
@@ -2254,18 +2250,18 @@ static DECLARE_DELAYED_WORK(ir_migration
* as simple as edge triggered migration and we can do the irq migration
* with a simple atomic update to IO-APIC RTE.
*/
-static void migrate_ioapic_irq(int irq, cpumask_t mask)
+static void migrate_ioapic_irq(int irq, const cpumask_t *mask)
{
struct irq_cfg *cfg;
struct irq_desc *desc;
- cpumask_t tmp, cleanup_mask;
+ cpumask_t tmpmask;
struct irte irte;
int modify_ioapic_rte;
unsigned int dest;
unsigned long flags;
- cpus_and(tmp, mask, cpu_online_map);
- if (cpus_empty(tmp))
+ cpus_and(tmpmask, *mask, cpu_online_map);
+ if (cpus_empty(tmpmask))
return;
if (get_irte(irq, &irte))
@@ -2275,8 +2271,8 @@ static void migrate_ioapic_irq(int irq,
return;
cfg = irq_cfg(irq);
- cpus_and(tmp, cfg->domain, mask);
- dest = cpu_mask_to_apicid(tmp);
+ cpus_and(tmpmask, cfg->domain, *mask);
+ dest = cpu_mask_to_apicid(&tmpmask);
desc = irq_to_desc(irq);
modify_ioapic_rte = desc->status & IRQ_LEVEL;
@@ -2295,13 +2291,13 @@ static void migrate_ioapic_irq(int irq,
modify_irte(irq, &irte);
if (cfg->move_in_progress) {
- cpus_and(cleanup_mask, cfg->old_domain, cpu_online_map);
- cfg->move_cleanup_count = cpus_weight(cleanup_mask);
- send_IPI_mask(cleanup_mask, IRQ_MOVE_CLEANUP_VECTOR);
+ cpus_and(tmpmask, cfg->old_domain, cpu_online_map);
+ cfg->move_cleanup_count = cpus_weight(tmpmask);
+ send_IPI_mask(&tmpmask, IRQ_MOVE_CLEANUP_VECTOR);
cfg->move_in_progress = 0;
}
- desc->affinity = mask;
+ desc->affinity = *mask;
}
static int migrate_irq_remapped_level(int irq)
@@ -2323,7 +2319,7 @@ static int migrate_irq_remapped_level(in
}
/* everthing is clear. we have right of way */
- migrate_ioapic_irq(irq, desc->pending_mask);
+ migrate_ioapic_irq(irq, &desc->pending_mask);
ret = 0;
desc->status &= ~IRQ_MOVE_PENDING;
@@ -2371,7 +2367,7 @@ static void set_ir_ioapic_affinity_irq(u
return;
}
- migrate_ioapic_irq(irq, mask);
+ migrate_ioapic_irq(irq, &mask);
}
#endif
@@ -2427,7 +2423,7 @@ static void irq_complete_move(unsigned i
cpus_and(cleanup_mask, cfg->old_domain, cpu_online_map);
cfg->move_cleanup_count = cpus_weight(cleanup_mask);
- send_IPI_mask(cleanup_mask, IRQ_MOVE_CLEANUP_VECTOR);
+ send_IPI_mask(&cleanup_mask, IRQ_MOVE_CLEANUP_VECTOR);
cfg->move_in_progress = 0;
}
}
@@ -3132,14 +3128,14 @@ static int msi_compose_msg(struct pci_de
unsigned dest;
cpumask_t tmp;
- tmp = TARGET_CPUS;
- err = assign_irq_vector(irq, tmp);
+ tmp = *TARGET_CPUS;
+ err = assign_irq_vector(irq, &tmp);
if (err)
return err;
cfg = irq_cfg(irq);
cpus_and(tmp, cfg->domain, tmp);
- dest = cpu_mask_to_apicid(tmp);
+ dest = cpu_mask_to_apicid(&tmp);
#ifdef CONFIG_INTR_REMAP
if (irq_remapped(irq)) {
@@ -3205,12 +3201,12 @@ static void set_msi_irq_affinity(unsigne
if (cpus_empty(tmp))
return;
- if (assign_irq_vector(irq, mask))
+ if (assign_irq_vector(irq, &mask))
return;
cfg = irq_cfg(irq);
cpus_and(tmp, cfg->domain, mask);
- dest = cpu_mask_to_apicid(tmp);
+ dest = cpu_mask_to_apicid(&tmp);
read_msi_msg(irq, &msg);
@@ -3244,12 +3240,12 @@ static void ir_set_msi_irq_affinity(unsi
if (get_irte(irq, &irte))
return;
- if (assign_irq_vector(irq, mask))
+ if (assign_irq_vector(irq, &mask))
return;
cfg = irq_cfg(irq);
cpus_and(tmp, cfg->domain, mask);
- dest = cpu_mask_to_apicid(tmp);
+ dest = cpu_mask_to_apicid(&tmp);
irte.vector = cfg->vector;
irte.dest_id = IRTE_DEST(dest);
@@ -3267,7 +3263,7 @@ static void ir_set_msi_irq_affinity(unsi
if (cfg->move_in_progress) {
cpus_and(cleanup_mask, cfg->old_domain, cpu_online_map);
cfg->move_cleanup_count = cpus_weight(cleanup_mask);
- send_IPI_mask(cleanup_mask, IRQ_MOVE_CLEANUP_VECTOR);
+ send_IPI_mask(&cleanup_mask, IRQ_MOVE_CLEANUP_VECTOR);
cfg->move_in_progress = 0;
}
@@ -3486,12 +3482,12 @@ static void dmar_msi_set_affinity(unsign
if (cpus_empty(tmp))
return;
- if (assign_irq_vector(irq, mask))
+ if (assign_irq_vector(irq, &mask))
return;
cfg = irq_cfg(irq);
cpus_and(tmp, cfg->domain, mask);
- dest = cpu_mask_to_apicid(tmp);
+ dest = cpu_mask_to_apicid(&tmp);
dmar_msi_read(irq, &msg);
@@ -3547,12 +3543,12 @@ static void hpet_msi_set_affinity(unsign
if (cpus_empty(tmp))
return;
- if (assign_irq_vector(irq, mask))
+ if (assign_irq_vector(irq, &mask))
return;
cfg = irq_cfg(irq);
cpus_and(tmp, cfg->domain, mask);
- dest = cpu_mask_to_apicid(tmp);
+ dest = cpu_mask_to_apicid(&tmp);
hpet_msi_read(irq, &msg);
@@ -3628,12 +3624,12 @@ static void set_ht_irq_affinity(unsigned
if (cpus_empty(tmp))
return;
- if (assign_irq_vector(irq, mask))
+ if (assign_irq_vector(irq, &mask))
return;
cfg = irq_cfg(irq);
cpus_and(tmp, cfg->domain, mask);
- dest = cpu_mask_to_apicid(tmp);
+ dest = cpu_mask_to_apicid(&tmp);
target_ht_irq(irq, dest, cfg->vector);
desc = irq_to_desc(irq);
@@ -3658,15 +3654,15 @@ int arch_setup_ht_irq(unsigned int irq,
int err;
cpumask_t tmp;
- tmp = TARGET_CPUS;
- err = assign_irq_vector(irq, tmp);
+ tmp = *TARGET_CPUS;
+ err = assign_irq_vector(irq, &tmp);
if (!err) {
struct ht_irq_msg msg;
unsigned dest;
cfg = irq_cfg(irq);
cpus_and(tmp, cfg->domain, tmp);
- dest = cpu_mask_to_apicid(tmp);
+ dest = cpu_mask_to_apicid(&tmp);
msg.address_hi = HT_IRQ_HIGH_DEST_ID(dest);
@@ -3702,7 +3698,7 @@ int arch_setup_ht_irq(unsigned int irq,
int arch_enable_uv_irq(char *irq_name, unsigned int irq, int cpu, int mmr_blade,
unsigned long mmr_offset)
{
- const cpumask_t *eligible_cpu = get_cpu_mask(cpu);
+ const cpumask_t *eligible_cpu = &cpumask_of_cpu(cpu);
struct irq_cfg *cfg;
int mmr_pnode;
unsigned long mmr_value;
@@ -3710,8 +3706,8 @@ int arch_enable_uv_irq(char *irq_name, u
unsigned long flags;
int err;
- err = assign_irq_vector(irq, *eligible_cpu);
- if (err != 0)
+ err = assign_irq_vector(irq, eligible_cpu);
+ if (err)
return err;
spin_lock_irqsave(&vector_lock, flags);
@@ -3731,7 +3727,7 @@ int arch_enable_uv_irq(char *irq_name, u
entry->polarity = 0;
entry->trigger = 0;
entry->mask = 0;
- entry->dest = cpu_mask_to_apicid(*eligible_cpu);
+ entry->dest = cpu_mask_to_apicid(eligible_cpu);
mmr_pnode = uv_blade_to_pnode(mmr_blade);
uv_write_global_mmr64(mmr_pnode, mmr_offset, mmr_value);
@@ -3962,10 +3958,10 @@ void __init setup_ioapic_dest(void)
irq_polarity(irq_entry));
#ifdef CONFIG_INTR_REMAP
else if (intr_remapping_enabled)
- set_ir_ioapic_affinity_irq(irq, TARGET_CPUS);
+ set_ir_ioapic_affinity_irq(irq, *TARGET_CPUS);
#endif
else
- set_ioapic_affinity_irq(irq, TARGET_CPUS);
+ set_ioapic_affinity_irq(irq, *TARGET_CPUS);
}
}
--- linux-2.6.28.orig/arch/x86/kernel/ipi.c
+++ linux-2.6.28/arch/x86/kernel/ipi.c
@@ -116,9 +116,9 @@ static inline void __send_IPI_dest_field
/*
* This is only used on smaller machines.
*/
-void send_IPI_mask_bitmask(cpumask_t cpumask, int vector)
+void send_IPI_mask_bitmask(const cpumask_t *cpumask, int vector)
{
- unsigned long mask = cpus_addr(cpumask)[0];
+ unsigned long mask = cpus_addr(*cpumask)[0];
unsigned long flags;
local_irq_save(flags);
@@ -127,7 +127,7 @@ void send_IPI_mask_bitmask(cpumask_t cpu
local_irq_restore(flags);
}
-void send_IPI_mask_sequence(cpumask_t mask, int vector)
+void send_IPI_mask_sequence(const cpumask_t *mask, int vector)
{
unsigned long flags;
unsigned int query_cpu;
@@ -139,12 +139,24 @@ void send_IPI_mask_sequence(cpumask_t ma
*/
local_irq_save(flags);
- for_each_possible_cpu(query_cpu) {
- if (cpu_isset(query_cpu, mask)) {
+ for_each_cpu_mask_nr(query_cpu, *mask)
+ __send_IPI_dest_field(cpu_to_logical_apicid(query_cpu), vector);
+ local_irq_restore(flags);
+}
+
+void send_IPI_mask_allbutself(const cpumask_t *mask, int vector)
+{
+ unsigned long flags;
+ unsigned int query_cpu;
+ unsigned int this_cpu = smp_processor_id();
+
+ /* See Hack comment above */
+
+ local_irq_save(flags);
+ for_each_cpu_mask_nr(query_cpu, *mask)
+ if (query_cpu != this_cpu)
__send_IPI_dest_field(cpu_to_logical_apicid(query_cpu),
vector);
- }
- }
local_irq_restore(flags);
}
--- linux-2.6.28.orig/arch/x86/kernel/smp.c
+++ linux-2.6.28/arch/x86/kernel/smp.c
@@ -118,22 +118,22 @@ static void native_smp_send_reschedule(i
WARN_ON(1);
return;
}
- send_IPI_mask(cpumask_of_cpu(cpu), RESCHEDULE_VECTOR);
+ send_IPI_mask(&cpumask_of_cpu(cpu), RESCHEDULE_VECTOR);
}
void native_send_call_func_single_ipi(int cpu)
{
- send_IPI_mask(cpumask_of_cpu(cpu), CALL_FUNCTION_SINGLE_VECTOR);
+ send_IPI_mask(&cpumask_of_cpu(cpu), CALL_FUNCTION_SINGLE_VECTOR);
}
-void native_send_call_func_ipi(cpumask_t mask)
+void native_send_call_func_ipi(const cpumask_t *mask)
{
cpumask_t allbutself;
allbutself = cpu_online_map;
cpu_clear(smp_processor_id(), allbutself);
- if (cpus_equal(mask, allbutself) &&
+ if (cpus_equal(*mask, allbutself) &&
cpus_equal(cpu_online_map, cpu_callout_map))
send_IPI_allbutself(CALL_FUNCTION_VECTOR);
else
--- linux-2.6.28.orig/arch/x86/kernel/tlb_32.c
+++ linux-2.6.28/arch/x86/kernel/tlb_32.c
@@ -158,7 +158,7 @@ void native_flush_tlb_others(const cpuma
* We have to send the IPI only to
* CPUs affected.
*/
- send_IPI_mask(cpumask, INVALIDATE_TLB_VECTOR);
+ send_IPI_mask(&cpumask, INVALIDATE_TLB_VECTOR);
while (!cpus_empty(flush_cpumask))
/* nothing. lockup detection does not belong here */
--- linux-2.6.28.orig/arch/x86/kernel/tlb_64.c
+++ linux-2.6.28/arch/x86/kernel/tlb_64.c
@@ -186,7 +186,7 @@ void native_flush_tlb_others(const cpuma
* We have to send the IPI only to
* CPUs affected.
*/
- send_IPI_mask(cpumask, INVALIDATE_TLB_VECTOR_START + sender);
+ send_IPI_mask(&cpumask, INVALIDATE_TLB_VECTOR_START + sender);
while (!cpus_empty(f->flush_cpumask))
cpu_relax();
--- linux-2.6.28.orig/arch/x86/mach-generic/bigsmp.c
+++ linux-2.6.28/arch/x86/mach-generic/bigsmp.c
@@ -41,9 +41,10 @@ static const struct dmi_system_id bigsmp
{ }
};
-static cpumask_t vector_allocation_domain(int cpu)
+static void vector_allocation_domain(int cpu, cpumask_t *retmask)
{
- return cpumask_of_cpu(cpu);
+ cpus_clear(*retmask);
+ cpu_set(cpu, *retmask);
}
static int probe_bigsmp(void)
--- linux-2.6.28.orig/arch/x86/mach-generic/es7000.c
+++ linux-2.6.28/arch/x86/mach-generic/es7000.c
@@ -75,7 +75,7 @@ static int __init acpi_madt_oem_check(ch
}
#endif
-static cpumask_t vector_allocation_domain(int cpu)
+static void vector_allocation_domain(int cpu, cpumask_t *retmask)
{
/* Careful. Some cpus do not strictly honor the set of cpus
* specified in the interrupt destination when using lowest
@@ -85,8 +85,7 @@ static cpumask_t vector_allocation_domai
* deliver interrupts to the wrong hyperthread when only one
* hyperthread was specified in the interrupt desitination.
*/
- cpumask_t domain = { { [0] = APIC_ALL_CPUS, } };
- return domain;
+ *retmask = (cpumask_t){ { [0] = APIC_ALL_CPUS, } };
}
struct genapic __initdata_refok apic_es7000 = APIC_INIT("es7000", probe_es7000);
--- linux-2.6.28.orig/arch/x86/mach-generic/numaq.c
+++ linux-2.6.28/arch/x86/mach-generic/numaq.c
@@ -38,7 +38,7 @@ static int acpi_madt_oem_check(char *oem
return 0;
}
-static cpumask_t vector_allocation_domain(int cpu)
+static void vector_allocation_domain(int cpu, cpumask_t *retmask)
{
/* Careful. Some cpus do not strictly honor the set of cpus
* specified in the interrupt destination when using lowest
@@ -48,8 +48,7 @@ static cpumask_t vector_allocation_domai
* deliver interrupts to the wrong hyperthread when only one
* hyperthread was specified in the interrupt desitination.
*/
- cpumask_t domain = { { [0] = APIC_ALL_CPUS, } };
- return domain;
+ *retmask = (cpumask_t){ { [0] = APIC_ALL_CPUS, } };
}
struct genapic apic_numaq = APIC_INIT("NUMAQ", probe_numaq);
--- linux-2.6.28.orig/arch/x86/mach-generic/summit.c
+++ linux-2.6.28/arch/x86/mach-generic/summit.c
@@ -23,7 +23,7 @@ static int probe_summit(void)
return 0;
}
-static cpumask_t vector_allocation_domain(int cpu)
+static void vector_allocation_domain(int cpu, cpumask_t *retmask)
{
/* Careful. Some cpus do not strictly honor the set of cpus
* specified in the interrupt destination when using lowest
@@ -33,8 +33,7 @@ static cpumask_t vector_allocation_domai
* deliver interrupts to the wrong hyperthread when only one
* hyperthread was specified in the interrupt desitination.
*/
- cpumask_t domain = { { [0] = APIC_ALL_CPUS, } };
- return domain;
+ *retmask = (cpumask_t){ { [0] = APIC_ALL_CPUS, } };
}
struct genapic apic_summit = APIC_INIT("summit", probe_summit);
--- linux-2.6.28.orig/arch/x86/xen/smp.c
+++ linux-2.6.28/arch/x86/xen/smp.c
@@ -158,7 +158,7 @@ static void __init xen_fill_possible_map
{
int i, rc;
- for (i = 0; i < NR_CPUS; i++) {
+ for (i = 0; i < nr_cpu_ids; i++) {
rc = HYPERVISOR_vcpu_op(VCPUOP_is_up, i, NULL);
if (rc >= 0) {
num_processors++;
@@ -196,7 +196,7 @@ static void __init xen_smp_prepare_cpus(
/* Restrict the possible_map according to max_cpus. */
while ((num_possible_cpus() > 1) && (num_possible_cpus() > max_cpus)) {
- for (cpu = NR_CPUS - 1; !cpu_possible(cpu); cpu--)
+ for (cpu = nr_cpu_ids - 1; !cpu_possible(cpu); cpu--)
continue;
cpu_clear(cpu, cpu_possible_map);
}
@@ -408,24 +408,22 @@ static void xen_smp_send_reschedule(int
xen_send_IPI_one(cpu, XEN_RESCHEDULE_VECTOR);
}
-static void xen_send_IPI_mask(cpumask_t mask, enum ipi_vector vector)
+static void xen_send_IPI_mask(const cpumask_t *mask, enum ipi_vector vector)
{
unsigned cpu;
- cpus_and(mask, mask, cpu_online_map);
-
- for_each_cpu_mask_nr(cpu, mask)
+ for_each_cpu_mask_and(cpu, *mask, cpu_online_map)
xen_send_IPI_one(cpu, vector);
}
-static void xen_smp_send_call_function_ipi(cpumask_t mask)
+static void xen_smp_send_call_function_ipi(const cpumask_t *mask)
{
int cpu;
xen_send_IPI_mask(mask, XEN_CALL_FUNCTION_VECTOR);
/* Make sure other vcpus get a chance to run if they need to. */
- for_each_cpu_mask_nr(cpu, mask) {
+ for_each_cpu_mask_nr(cpu, *mask) {
if (xen_vcpu_stolen(cpu)) {
HYPERVISOR_sched_op(SCHEDOP_yield, 0);
break;
@@ -435,7 +433,8 @@ static void xen_smp_send_call_function_i
static void xen_smp_send_call_function_single_ipi(int cpu)
{
- xen_send_IPI_mask(cpumask_of_cpu(cpu), XEN_CALL_FUNCTION_SINGLE_VECTOR);
+ xen_send_IPI_mask(&cpumask_of_cpu(cpu),
+ XEN_CALL_FUNCTION_SINGLE_VECTOR);
}
static irqreturn_t xen_call_function_interrupt(int irq, void *dev_id)
--- linux-2.6.28.orig/include/asm-x86/bigsmp/apic.h
+++ linux-2.6.28/include/asm-x86/bigsmp/apic.h
@@ -9,12 +9,12 @@ static inline int apic_id_registered(voi
return (1);
}
-static inline cpumask_t target_cpus(void)
+static inline const cpumask_t *target_cpus(void)
{
#ifdef CONFIG_SMP
- return cpu_online_map;
+ return &cpu_online_map;
#else
- return cpumask_of_cpu(0);
+ return &cpumask_of_cpu(0);
#endif
}
@@ -81,7 +81,7 @@ static inline int apicid_to_node(int log
static inline int cpu_present_to_apicid(int mps_cpu)
{
- if (mps_cpu < NR_CPUS)
+ if (mps_cpu < nr_cpu_ids)
return (int) per_cpu(x86_bios_cpu_apicid, mps_cpu);
return BAD_APICID;
@@ -96,7 +96,7 @@ extern u8 cpu_2_logical_apicid[];
/* Mapping from cpu number to logical apicid */
static inline int cpu_to_logical_apicid(int cpu)
{
- if (cpu >= NR_CPUS)
+ if (cpu >= nr_cpu_ids)
return BAD_APICID;
return cpu_physical_id(cpu);
}
@@ -121,12 +121,12 @@ static inline int check_phys_apicid_pres
}
/* As we are using single CPU as destination, pick only one CPU here */
-static inline unsigned int cpu_mask_to_apicid(cpumask_t cpumask)
+static inline unsigned int cpu_mask_to_apicid(const cpumask_t *cpumask)
{
int cpu;
int apicid;
- cpu = first_cpu(cpumask);
+ cpu = first_cpu(*cpumask);
apicid = cpu_to_logical_apicid(cpu);
return apicid;
}
--- linux-2.6.28.orig/include/asm-x86/bigsmp/ipi.h
+++ linux-2.6.28/include/asm-x86/bigsmp/ipi.h
@@ -1,9 +1,10 @@
#ifndef __ASM_MACH_IPI_H
#define __ASM_MACH_IPI_H
-void send_IPI_mask_sequence(cpumask_t mask, int vector);
+void send_IPI_mask_sequence(const cpumask_t *mask, int vector);
+void send_IPI_mask_allbutself(const cpumask_t *mask, int vector);
-static inline void send_IPI_mask(cpumask_t mask, int vector)
+static inline void send_IPI_mask(const cpumask_t *mask, int vector)
{
send_IPI_mask_sequence(mask, vector);
}
@@ -14,12 +15,12 @@ static inline void send_IPI_allbutself(i
cpu_clear(smp_processor_id(), mask);
if (!cpus_empty(mask))
- send_IPI_mask(mask, vector);
+ send_IPI_mask(&mask, vector);
}
static inline void send_IPI_all(int vector)
{
- send_IPI_mask(cpu_online_map, vector);
+ send_IPI_mask(&cpu_online_map, vector);
}
#endif /* __ASM_MACH_IPI_H */
--- linux-2.6.28.orig/include/asm-x86/es7000/apic.h
+++ linux-2.6.28/include/asm-x86/es7000/apic.h
@@ -9,12 +9,12 @@ static inline int apic_id_registered(voi
return (1);
}
-static inline cpumask_t target_cpus(void)
+static inline const cpumask_t *target_cpus(void)
{
#if defined CONFIG_ES7000_CLUSTERED_APIC
- return CPU_MASK_ALL;
+ return &CPU_MASK_ALL;
#else
- return cpumask_of_cpu(smp_processor_id());
+ return &cpumask_of_cpu(smp_processor_id());
#endif
}
@@ -98,7 +98,7 @@ static inline int cpu_present_to_apicid(
{
if (!mps_cpu)
return boot_cpu_physical_apicid;
- else if (mps_cpu < NR_CPUS)
+ else if (mps_cpu < nr_cpu_ids)
return (int) per_cpu(x86_bios_cpu_apicid, mps_cpu);
else
return BAD_APICID;
@@ -118,7 +118,7 @@ extern u8 cpu_2_logical_apicid[];
static inline int cpu_to_logical_apicid(int cpu)
{
#ifdef CONFIG_SMP
- if (cpu >= NR_CPUS)
+ if (cpu >= nr_cpu_ids)
return BAD_APICID;
return (int)cpu_2_logical_apicid[cpu];
#else
@@ -144,14 +144,14 @@ static inline int check_phys_apicid_pres
return (1);
}
-static inline unsigned int cpu_mask_to_apicid(cpumask_t cpumask)
+static inline unsigned int cpu_mask_to_apicid(const cpumask_t *cpumask)
{
int num_bits_set;
int cpus_found = 0;
int cpu;
int apicid;
- num_bits_set = cpus_weight(cpumask);
+ num_bits_set = cpus_weight(*cpumask);
/* Return id to all */
if (num_bits_set == NR_CPUS)
#if defined CONFIG_ES7000_CLUSTERED_APIC
@@ -163,10 +163,10 @@ static inline unsigned int cpu_mask_to_a
* The cpus in the mask must all be on the apic cluster. If are not
* on the same apicid cluster return default value of TARGET_CPUS.
*/
- cpu = first_cpu(cpumask);
+ cpu = first_cpu(*cpumask);
apicid = cpu_to_logical_apicid(cpu);
while (cpus_found < num_bits_set) {
- if (cpu_isset(cpu, cpumask)) {
+ if (cpu_isset(cpu, *cpumask)) {
int new_apicid = cpu_to_logical_apicid(cpu);
if (apicid_cluster(apicid) !=
apicid_cluster(new_apicid)){
--- linux-2.6.28.orig/include/asm-x86/es7000/ipi.h
+++ linux-2.6.28/include/asm-x86/es7000/ipi.h
@@ -1,9 +1,10 @@
#ifndef __ASM_ES7000_IPI_H
#define __ASM_ES7000_IPI_H
-void send_IPI_mask_sequence(cpumask_t mask, int vector);
+void send_IPI_mask_sequence(const cpumask_t *mask, int vector);
+void send_IPI_mask_allbutself(const cpumask_t *mask, int vector);
-static inline void send_IPI_mask(cpumask_t mask, int vector)
+static inline void send_IPI_mask(const cpumask_t *mask, int vector)
{
send_IPI_mask_sequence(mask, vector);
}
@@ -13,12 +14,12 @@ static inline void send_IPI_allbutself(i
cpumask_t mask = cpu_online_map;
cpu_clear(smp_processor_id(), mask);
if (!cpus_empty(mask))
- send_IPI_mask(mask, vector);
+ send_IPI_mask(&mask, vector);
}
static inline void send_IPI_all(int vector)
{
- send_IPI_mask(cpu_online_map, vector);
+ send_IPI_mask(&cpu_online_map, vector);
}
#endif /* __ASM_ES7000_IPI_H */
--- linux-2.6.28.orig/include/asm-x86/genapic_32.h
+++ linux-2.6.28/include/asm-x86/genapic_32.h
@@ -23,7 +23,7 @@ struct genapic {
int (*probe)(void);
int (*apic_id_registered)(void);
- cpumask_t (*target_cpus)(void);
+ const cpumask_t *(*target_cpus)(void);
int int_delivery_mode;
int int_dest_mode;
int ESR_DISABLE;
@@ -56,12 +56,13 @@ struct genapic {
unsigned (*get_apic_id)(unsigned long x);
unsigned long apic_id_mask;
- unsigned int (*cpu_mask_to_apicid)(cpumask_t cpumask);
- cpumask_t (*vector_allocation_domain)(int cpu);
+ unsigned int (*cpu_mask_to_apicid)(const cpumask_t *cpumask);
+ void (*vector_allocation_domain)(int cpu, cpumask_t *retmask);
#ifdef CONFIG_SMP
/* ipi */
- void (*send_IPI_mask)(cpumask_t mask, int vector);
+ void (*send_IPI_mask)(const cpumask_t *mask, int vector);
+ void (*send_IPI_mask_allbutself)(const cpumask_t *mask, int vector);
void (*send_IPI_allbutself)(int vector);
void (*send_IPI_all)(int vector);
#endif
@@ -105,7 +106,7 @@ struct genapic {
APICFUNC(get_apic_id) \
.apic_id_mask = APIC_ID_MASK, \
APICFUNC(cpu_mask_to_apicid) \
- APICFUNC(vector_allocation_domain) \
+ APICFUNC(vector_allocation_domain) \
APICFUNC(acpi_madt_oem_check) \
IPIFUNC(send_IPI_mask) \
IPIFUNC(send_IPI_allbutself) \
--- linux-2.6.28.orig/include/asm-x86/genapic_64.h
+++ linux-2.6.28/include/asm-x86/genapic_64.h
@@ -1,6 +1,8 @@
#ifndef ASM_X86__GENAPIC_64_H
#define ASM_X86__GENAPIC_64_H
+#include <linux/cpumask.h>
+
/*
* Copyright 2004 James Cleverdon, IBM.
* Subject to the GNU Public License, v.2
@@ -18,16 +20,17 @@ struct genapic {
u32 int_delivery_mode;
u32 int_dest_mode;
int (*apic_id_registered)(void);
- cpumask_t (*target_cpus)(void);
- cpumask_t (*vector_allocation_domain)(int cpu);
+ const cpumask_t *(*target_cpus)(void);
+ void (*vector_allocation_domain)(int cpu, cpumask_t *retmask);
void (*init_apic_ldr)(void);
/* ipi */
- void (*send_IPI_mask)(cpumask_t mask, int vector);
+ void (*send_IPI_mask)(const cpumask_t *mask, int vector);
+ void (*send_IPI_mask_allbutself)(const cpumask_t *mask, int vector);
void (*send_IPI_allbutself)(int vector);
void (*send_IPI_all)(int vector);
void (*send_IPI_self)(int vector);
/* */
- unsigned int (*cpu_mask_to_apicid)(cpumask_t cpumask);
+ unsigned int (*cpu_mask_to_apicid)(const cpumask_t *cpumask);
unsigned int (*phys_pkg_id)(int index_msb);
unsigned int (*get_apic_id)(unsigned long x);
unsigned long (*set_apic_id)(unsigned int id);
--- linux-2.6.28.orig/include/asm-x86/ipi.h
+++ linux-2.6.28/include/asm-x86/ipi.h
@@ -117,7 +117,7 @@ static inline void __send_IPI_dest_field
native_apic_mem_write(APIC_ICR, cfg);
}
-static inline void send_IPI_mask_sequence(cpumask_t mask, int vector)
+static inline void send_IPI_mask_sequence(const cpumask_t *mask, int vector)
{
unsigned long flags;
unsigned long query_cpu;
@@ -128,11 +128,28 @@ static inline void send_IPI_mask_sequenc
* - mbligh
*/
local_irq_save(flags);
- for_each_cpu_mask_nr(query_cpu, mask) {
+ for_each_cpu_mask_nr(query_cpu, *mask) {
__send_IPI_dest_field(per_cpu(x86_cpu_to_apicid, query_cpu),
vector, APIC_DEST_PHYSICAL);
}
local_irq_restore(flags);
}
+static inline void send_IPI_mask_allbutself(const cpumask_t *mask, int vector)
+{
+ unsigned long flags;
+ unsigned int query_cpu;
+ unsigned int this_cpu = smp_processor_id();
+
+ /* See Hack comment above */
+
+ local_irq_save(flags);
+ for_each_cpu_mask_nr(query_cpu, *mask)
+ if (query_cpu != this_cpu)
+ __send_IPI_dest_field(
+ per_cpu(x86_cpu_to_apicid, query_cpu),
+ vector, APIC_DEST_PHYSICAL);
+ local_irq_restore(flags);
+}
+
#endif /* ASM_X86__IPI_H */
--- linux-2.6.28.orig/include/asm-x86/mach-default/mach_apic.h
+++ linux-2.6.28/include/asm-x86/mach-default/mach_apic.h
@@ -8,12 +8,12 @@
#define APIC_DFR_VALUE (APIC_DFR_FLAT)
-static inline cpumask_t target_cpus(void)
+static inline const cpumask_t *target_cpus(void)
{
#ifdef CONFIG_SMP
- return cpu_online_map;
+ return &cpu_online_map;
#else
- return cpumask_of_cpu(0);
+ return &cpumask_of_cpu(0);
#endif
}
@@ -59,9 +59,9 @@ static inline int apic_id_registered(voi
return physid_isset(read_apic_id(), phys_cpu_present_map);
}
-static inline unsigned int cpu_mask_to_apicid(cpumask_t cpumask)
+static inline unsigned int cpu_mask_to_apicid(const cpumask_t *cpumask)
{
- return cpus_addr(cpumask)[0];
+ return cpus_addr(*cpumask)[0];
}
static inline u32 phys_pkg_id(u32 cpuid_apic, int index_msb)
@@ -86,7 +86,7 @@ static inline int apicid_to_node(int log
#endif
}
-static inline cpumask_t vector_allocation_domain(int cpu)
+static inline void vector_allocation_domain(int cpu, cpumask_t *retmask)
{
/* Careful. Some cpus do not strictly honor the set of cpus
* specified in the interrupt destination when using lowest
@@ -96,8 +96,7 @@ static inline cpumask_t vector_allocatio
* deliver interrupts to the wrong hyperthread when only one
* hyperthread was specified in the interrupt desitination.
*/
- cpumask_t domain = { { [0] = APIC_ALL_CPUS, } };
- return domain;
+ *retmask = (cpumask_t){ { [0] = APIC_ALL_CPUS, } };
}
#endif
@@ -129,7 +128,7 @@ static inline int cpu_to_logical_apicid(
static inline int cpu_present_to_apicid(int mps_cpu)
{
- if (mps_cpu < NR_CPUS && cpu_present(mps_cpu))
+ if (mps_cpu < nr_cpu_ids && cpu_present(mps_cpu))
return (int)per_cpu(x86_bios_cpu_apicid, mps_cpu);
else
return BAD_APICID;
--- linux-2.6.28.orig/include/asm-x86/mach-default/mach_ipi.h
+++ linux-2.6.28/include/asm-x86/mach-default/mach_ipi.h
@@ -4,7 +4,8 @@
/* Avoid include hell */
#define NMI_VECTOR 0x02
-void send_IPI_mask_bitmask(cpumask_t mask, int vector);
+void send_IPI_mask_bitmask(const cpumask_t *mask, int vector);
+void send_IPI_mask_allbutself(const cpumask_t *mask, int vector);
void __send_IPI_shortcut(unsigned int shortcut, int vector);
extern int no_broadcast;
@@ -12,28 +13,27 @@ extern int no_broadcast;
#ifdef CONFIG_X86_64
#include <asm/genapic.h>
#define send_IPI_mask (genapic->send_IPI_mask)
+#define send_IPI_mask_allbutself (genapic->send_IPI_mask_allbutself)
#else
-static inline void send_IPI_mask(cpumask_t mask, int vector)
+static inline void send_IPI_mask(const cpumask_t *mask, int vector)
{
send_IPI_mask_bitmask(mask, vector);
}
+void send_IPI_mask_allbutself(const cpumask_t *mask, int vector);
#endif
static inline void __local_send_IPI_allbutself(int vector)
{
- if (no_broadcast || vector == NMI_VECTOR) {
- cpumask_t mask = cpu_online_map;
-
- cpu_clear(smp_processor_id(), mask);
- send_IPI_mask(mask, vector);
- } else
+ if (no_broadcast || vector == NMI_VECTOR)
+ send_IPI_mask_allbutself(&cpu_online_map, vector);
+ else
__send_IPI_shortcut(APIC_DEST_ALLBUT, vector);
}
static inline void __local_send_IPI_all(int vector)
{
if (no_broadcast || vector == NMI_VECTOR)
- send_IPI_mask(cpu_online_map, vector);
+ send_IPI_mask(&cpu_online_map, vector);
else
__send_IPI_shortcut(APIC_DEST_ALLINC, vector);
}
--- linux-2.6.28.orig/include/asm-x86/numaq/apic.h
+++ linux-2.6.28/include/asm-x86/numaq/apic.h
@@ -7,7 +7,7 @@
#define APIC_DFR_VALUE (APIC_DFR_CLUSTER)
-static inline cpumask_t target_cpus(void)
+static inline const cpumask_t *target_cpus(void)
{
return CPU_MASK_ALL;
}
@@ -122,7 +122,7 @@ static inline void enable_apic_mode(void
* We use physical apicids here, not logical, so just return the default
* physical broadcast to stop people from breaking us
*/
-static inline unsigned int cpu_mask_to_apicid(cpumask_t cpumask)
+static inline unsigned int cpu_mask_to_apicid(const cpumask_t *cpumask)
{
return (int) 0xF;
}
--- linux-2.6.28.orig/include/asm-x86/numaq/ipi.h
+++ linux-2.6.28/include/asm-x86/numaq/ipi.h
@@ -1,9 +1,10 @@
#ifndef __ASM_NUMAQ_IPI_H
#define __ASM_NUMAQ_IPI_H
-void send_IPI_mask_sequence(cpumask_t, int vector);
+void send_IPI_mask_sequence(const cpumask_t *mask, int vector);
+void send_IPI_mask_allbutself(const cpumask_t *mask, int vector);
-static inline void send_IPI_mask(cpumask_t mask, int vector)
+static inline void send_IPI_mask(const cpumask_t *mask, int vector)
{
send_IPI_mask_sequence(mask, vector);
}
@@ -14,12 +15,12 @@ static inline void send_IPI_allbutself(i
cpu_clear(smp_processor_id(), mask);
if (!cpus_empty(mask))
- send_IPI_mask(mask, vector);
+ send_IPI_mask(&mask, vector);
}
static inline void send_IPI_all(int vector)
{
- send_IPI_mask(cpu_online_map, vector);
+ send_IPI_mask(&cpu_online_map, vector);
}
#endif /* __ASM_NUMAQ_IPI_H */
--- linux-2.6.28.orig/include/asm-x86/smp.h
+++ linux-2.6.28/include/asm-x86/smp.h
@@ -60,7 +60,7 @@ struct smp_ops {
void (*cpu_die)(unsigned int cpu);
void (*play_dead)(void);
- void (*send_call_func_ipi)(cpumask_t mask);
+ void (*send_call_func_ipi)(const cpumask_t *mask);
void (*send_call_func_single_ipi)(int cpu);
};
@@ -125,7 +125,7 @@ static inline void arch_send_call_functi
static inline void arch_send_call_function_ipi(cpumask_t mask)
{
- smp_ops.send_call_func_ipi(mask);
+ smp_ops.send_call_func_ipi(&mask);
}
void cpu_disable_common(void);
@@ -138,7 +138,7 @@ void native_cpu_die(unsigned int cpu);
void native_play_dead(void);
void play_dead_common(void);
-void native_send_call_func_ipi(cpumask_t mask);
+void native_send_call_func_ipi(const cpumask_t *mask);
void native_send_call_func_single_ipi(int cpu);
extern void prefill_possible_map(void);
--- linux-2.6.28.orig/include/asm-x86/summit/apic.h
+++ linux-2.6.28/include/asm-x86/summit/apic.h
@@ -14,13 +14,13 @@
#define APIC_DFR_VALUE (APIC_DFR_CLUSTER)
-static inline cpumask_t target_cpus(void)
+static inline const cpumask_t *target_cpus(void)
{
/* CPU_MASK_ALL (0xff) has undefined behaviour with
* dest_LowestPrio mode logical clustered apic interrupt routing
* Just start on cpu 0. IRQ balancing will spread load
*/
- return cpumask_of_cpu(0);
+ return &cpumask_of_cpu(0);
}
#define INT_DELIVERY_MODE (dest_LowestPrio)
@@ -137,14 +137,14 @@ static inline void enable_apic_mode(void
{
}
-static inline unsigned int cpu_mask_to_apicid(cpumask_t cpumask)
+static inline unsigned int cpu_mask_to_apicid(const cpumask_t *cpumask)
{
int num_bits_set;
int cpus_found = 0;
int cpu;
int apicid;
- num_bits_set = cpus_weight(cpumask);
+ num_bits_set = cpus_weight(*cpumask);
/* Return id to all */
if (num_bits_set == NR_CPUS)
return (int) 0xFF;
@@ -152,10 +152,10 @@ static inline unsigned int cpu_mask_to_a
* The cpus in the mask must all be on the apic cluster. If are not
* on the same apicid cluster return default value of TARGET_CPUS.
*/
- cpu = first_cpu(cpumask);
+ cpu = first_cpu(*cpumask);
apicid = cpu_to_logical_apicid(cpu);
while (cpus_found < num_bits_set) {
- if (cpu_isset(cpu, cpumask)) {
+ if (cpu_isset(cpu, *cpumask)) {
int new_apicid = cpu_to_logical_apicid(cpu);
if (apicid_cluster(apicid) !=
apicid_cluster(new_apicid)){
--- linux-2.6.28.orig/include/asm-x86/summit/ipi.h
+++ linux-2.6.28/include/asm-x86/summit/ipi.h
@@ -1,9 +1,10 @@
#ifndef __ASM_SUMMIT_IPI_H
#define __ASM_SUMMIT_IPI_H
-void send_IPI_mask_sequence(cpumask_t mask, int vector);
+void send_IPI_mask_sequence(const cpumask_t *mask, int vector);
+void send_IPI_mask_allbutself(const cpumask_t *mask, int vector);
-static inline void send_IPI_mask(cpumask_t mask, int vector)
+static inline void send_IPI_mask(const cpumask_t *mask, int vector)
{
send_IPI_mask_sequence(mask, vector);
}
@@ -14,12 +15,12 @@ static inline void send_IPI_allbutself(i
cpu_clear(smp_processor_id(), mask);
if (!cpus_empty(mask))
- send_IPI_mask(mask, vector);
+ send_IPI_mask(&mask, vector);
}
static inline void send_IPI_all(int vector)
{
- send_IPI_mask(cpu_online_map, vector);
+ send_IPI_mask(&cpu_online_map, vector);
}
#endif /* __ASM_SUMMIT_IPI_H */
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 05/35] sched: Reduce stack size requirements in kernel/sched.c
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (3 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 04/35] x86 smp: modify send_IPI_mask interface to accept cpumask_t pointers Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 06/35] cpumask: introduce struct cpumask Mike Travis
` (29 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
* use node_to_cpumask_ptr in place of node_to_cpumask to reduce stack
requirements in sched.c
Applies to linux-2.6.tip/master.
Signed-off-by: Mike Travis <travis@sgi.com>
Acked-by: Rusty Russell <rusty@rustcorp.com.au>
---
kernel/sched.c | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
--- linux-2.6.28.orig/kernel/sched.c
+++ linux-2.6.28/kernel/sched.c
@@ -6117,8 +6117,9 @@ static void move_task_off_dead_cpu(int d
do {
/* On same node? */
- mask = node_to_cpumask(cpu_to_node(dead_cpu));
- cpus_and(mask, mask, p->cpus_allowed);
+ node_to_cpumask_ptr(pnodemask, cpu_to_node(dead_cpu));
+
+ cpus_and(mask, *pnodemask, p->cpus_allowed);
dest_cpu = any_online_cpu(mask);
/* On any allowed CPU? */
@@ -7126,9 +7127,9 @@ static int cpu_to_allnodes_group(int cpu
struct sched_group **sg, cpumask_t *nodemask)
{
int group;
+ node_to_cpumask_ptr(pnodemask, cpu_to_node(cpu));
- *nodemask = node_to_cpumask(cpu_to_node(cpu));
- cpus_and(*nodemask, *nodemask, *cpu_map);
+ cpus_and(*nodemask, *pnodemask, *cpu_map);
group = first_cpu(*nodemask);
if (sg)
@@ -7178,9 +7179,9 @@ static void free_sched_groups(const cpum
for (i = 0; i < nr_node_ids; i++) {
struct sched_group *oldsg, *sg = sched_group_nodes[i];
+ node_to_cpumask_ptr(pnodemask, i);
- *nodemask = node_to_cpumask(i);
- cpus_and(*nodemask, *nodemask, *cpu_map);
+ cpus_and(*nodemask, *pnodemask, *cpu_map);
if (cpus_empty(*nodemask))
continue;
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 06/35] cpumask: introduce struct cpumask.
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (4 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 05/35] sched: Reduce stack size requirements in kernel/sched.c Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 07/35] cpumask: change cpumask_scnprintf, cpumask_parse_user, cpulist_parse, and cpulist_scnprintf to take pointers Mike Travis
` (28 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
We want to move cpumasks off the stack: no local decls, no passing by
copy. We also don't want to allow assignment of them, so we can later
partially allocate them (ie. not all NR_CPUS bits).
Unfortunately, all the cpus_* functions are written perversely to take
cpumask_t not cpumask_t *; although they are in fact wrapper macros.
This sets a bad example. Also, we want to eventually make cpumasks an
undefined struct, so we can catch on-stack usage with a compile error.
So we create a 'struct cpumask', typedef cpumask_t to it during the
transition, and cleanup all the cpumask operators to be normal
functions (cpus_ -> cpumask_). Note that two functions already use
variants of the new names: they are fixed in the next patch.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 357 +++++++++++++++++++++++++-----------------------
lib/cpumask.c | 2
2 files changed, 192 insertions(+), 167 deletions(-)
--- linux-2.6.28.orig/include/linux/cpumask.h
+++ linux-2.6.28/include/linux/cpumask.h
@@ -5,17 +5,20 @@
* Cpumasks provide a bitmap suitable for representing the
* set of CPU's in a system, one bit position per CPU number.
*
+ * Old-style uses "cpumask_t", but new ops are "struct cpumask *";
+ * don't put "struct cpumask"s on the stack.
+ *
* See detailed comments in the file linux/bitmap.h describing the
* data type on which these cpumasks are based.
*
* For details of cpumask_scnprintf() and cpumask_parse_user(),
- * see bitmap_scnprintf() and bitmap_parse_user() in lib/bitmap.c.
- * For details of cpulist_scnprintf() and cpulist_parse(), see
- * bitmap_scnlistprintf() and bitmap_parselist(), also in bitmap.c.
- * For details of cpu_remap(), see bitmap_bitremap in lib/bitmap.c
- * For details of cpus_remap(), see bitmap_remap in lib/bitmap.c.
- * For details of cpus_onto(), see bitmap_onto in lib/bitmap.c.
- * For details of cpus_fold(), see bitmap_fold in lib/bitmap.c.
+ * see bitmap_scnprintf() and bitmap_parse_user() in lib/bitmap.c.
+ * For details of cpulist_scnprintf() and cpulist_parse(),
+ * see bitmap_scnlistprintf() and bitmap_parselist(), in lib/bitmap.c.
+ * For details of cpumask_cpuremap(), see bitmap_bitremap in lib/bitmap.c
+ * For details of cpumask_remap(), see bitmap_remap in lib/bitmap.c.
+ * For details of cpumask_onto(), see bitmap_onto in lib/bitmap.c.
+ * For details of cpumask_fold(), see bitmap_fold in lib/bitmap.c.
*
* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
* Note: The alternate operations with the suffix "_nr" are used
@@ -33,29 +36,29 @@
*
* The available cpumask operations are:
*
- * void cpu_set(cpu, mask) turn on bit 'cpu' in mask
- * void cpu_clear(cpu, mask) turn off bit 'cpu' in mask
- * void cpus_setall(mask) set all bits
- * void cpus_clear(mask) clear all bits
- * int cpu_isset(cpu, mask) true iff bit 'cpu' set in mask
- * int cpu_test_and_set(cpu, mask) test and set bit 'cpu' in mask
- *
- * void cpus_and(dst, src1, src2) dst = src1 & src2 [intersection]
- * void cpus_or(dst, src1, src2) dst = src1 | src2 [union]
- * void cpus_xor(dst, src1, src2) dst = src1 ^ src2
- * void cpus_andnot(dst, src1, src2) dst = src1 & ~src2
- * void cpus_complement(dst, src) dst = ~src
- *
- * int cpus_equal(mask1, mask2) Does mask1 == mask2?
- * int cpus_intersects(mask1, mask2) Do mask1 and mask2 intersect?
- * int cpus_subset(mask1, mask2) Is mask1 a subset of mask2?
- * int cpus_empty(mask) Is mask empty (no bits sets)?
- * int cpus_full(mask) Is mask full (all bits sets)?
- * int cpus_weight(mask) Hamming weigh - number of set bits
- * int cpus_weight_nr(mask) Same using nr_cpu_ids instead of NR_CPUS
+ * void cpumask_set_cpu(cpu, mask) turn on bit 'cpu' in mask
+ * void cpumask_clear_cpu(cpu, mask) turn off bit 'cpu' in mask
+ * int cpumask_test_and_set_cpu(cpu, mask) test and set bit 'cpu' in mask
+ * int cpumask_test_cpu(cpu, mask) true iff bit 'cpu' set in mask
+ * void cpumask_setall(mask) set all bits
+ * void cpumask_clear(mask) clear all bits
+ *
+ * void cpumask_and(dst, src1, src2) dst = src1 & src2 [intersection]
+ * void cpumask_or(dst, src1, src2) dst = src1 | src2 [union]
+ * void cpumask_xor(dst, src1, src2) dst = src1 ^ src2
+ * void cpumask_andnot(dst, src1, src2) dst = src1 & ~src2
+ * void cpumask_complement(dst, src) dst = ~src
+ *
+ * int cpumask_equal(mask1, mask2) Does mask1 == mask2?
+ * int cpumask_intersects(mask1, mask2) Do mask1 and mask2 intersect?
+ * int cpumask_subset(mask1, mask2) Is mask1 a subset of mask2?
+ * int cpumask_empty(mask) Is mask empty (no bits sets)?
+ * int cpumask_full(mask) Is mask full (all bits sets)?
+ * int cpumask_weight(mask) Hamming weigh - number of set bits
+ * int cpumask_weight_nr(mask) Same using nr_cpu_ids instead of NR_CPUS
*
- * void cpus_shift_right(dst, src, n) Shift right
- * void cpus_shift_left(dst, src, n) Shift left
+ * void cpumask_shift_right(dst, src, n) Shift right
+ * void cpumask_shift_left(dst, src, n) Shift left
*
* int first_cpu(mask) Number lowest set bit, or NR_CPUS
* int next_cpu(cpu, mask) Next cpu past 'cpu', or NR_CPUS
@@ -65,7 +68,7 @@
* (can be used as an lvalue)
* CPU_MASK_ALL Initializer - all bits set
* CPU_MASK_NONE Initializer - no bits set
- * unsigned long *cpus_addr(mask) Array of unsigned long's in mask
+ * unsigned long *cpumask_bits(mask) Array of unsigned long's in mask
*
* CPUMASK_ALLOC kmalloc's a structure that is a composite of many cpumask_t
* variables, and CPUMASK_PTR provides pointers to each field.
@@ -100,12 +103,12 @@
*
* int cpumask_scnprintf(buf, len, mask) Format cpumask for printing
* int cpumask_parse_user(ubuf, ulen, mask) Parse ascii string as cpumask
- * int cpulist_scnprintf(buf, len, mask) Format cpumask as list for printing
- * int cpulist_parse(buf, map) Parse ascii string as cpulist
+ * int cpumask_scnprintf(buf, len, mask) Format cpumask as list for printing
+ * int cpumask_parse(buf, map) Parse ascii string as cpumask
* int cpu_remap(oldbit, old, new) newbit = map(old, new)(oldbit)
- * void cpus_remap(dst, src, old, new) *dst = map(old, new)(src)
- * void cpus_onto(dst, orig, relmap) *dst = orig relative to relmap
- * void cpus_fold(dst, orig, sz) dst bits = orig bits mod sz
+ * void cpumask_remap(dst, src, old, new) *dst = map(old, new)(src)
+ * void cpumask_onto(dst, orig, relmap) *dst = orig relative to relmap
+ * void cpumask_fold(dst, orig, sz) dst bits = orig bits mod sz
*
* for_each_cpu_mask(cpu, mask) for-loop cpu over mask using NR_CPUS
* for_each_cpu_mask_nr(cpu, mask) for-loop cpu over mask using nr_cpu_ids
@@ -139,131 +142,217 @@
#include <linux/threads.h>
#include <linux/bitmap.h>
-typedef struct { DECLARE_BITMAP(bits, NR_CPUS); } cpumask_t;
+struct cpumask
+{
+ DECLARE_BITMAP(bits, NR_CPUS);
+};
+#define cpumask_bits(maskp) ((maskp)->bits)
+
+/* Deprecated. */
+typedef struct cpumask cpumask_t;
extern cpumask_t _unused_cpumask_arg_;
-#define cpu_set(cpu, dst) __cpu_set((cpu), &(dst))
-static inline void __cpu_set(int cpu, volatile cpumask_t *dstp)
+#define cpu_set(cpu, dst) cpumask_set_cpu((cpu), &(dst))
+#define cpu_clear(cpu, dst) cpumask_clear_cpu((cpu), &(dst))
+#define cpu_test_and_set(cpu, mask) cpumask_test_and_set_cpu((cpu), &(mask))
+/* No static inline type checking - see Subtlety (1) above. */
+#define cpu_isset(cpu, cpumask) test_bit((cpu), (cpumask).bits)
+#define cpus_setall(dst) cpumask_setall(&(dst))
+#define cpus_clear(dst) cpumask_clear(&(dst))
+#define cpus_and(dst, src1, src2) cpumask_and(&(dst), &(src1), &(src2))
+#define cpus_or(dst, src1, src2) cpumask_or(&(dst), &(src1), &(src2))
+#define cpus_xor(dst, src1, src2) cpumask_xor(&(dst), &(src1), &(src2))
+#define cpus_andnot(dst, src1, src2) \
+ cpumask_andnot(&(dst), &(src1), &(src2))
+#define cpus_complement(dst, src) cpumask_complement(&(dst), &(src))
+#define cpus_equal(src1, src2) cpumask_equal(&(src1), &(src2))
+#define cpus_intersects(src1, src2) cpumask_intersects(&(src1), &(src2))
+#define cpus_subset(src1, src2) cpumask_subset(&(src1), &(src2))
+#define cpus_empty(src) cpumask_empty(&(src))
+#define cpus_full(cpumask) cpumask_full(&(cpumask))
+#define cpus_weight(cpumask) cpumask_weight(&(cpumask))
+#define cpus_shift_right(dst, src, n) \
+ cpumask_shift_right(&(dst), &(src), (n))
+#define cpus_shift_left(dst, src, n) \
+ cpumask_shift_left(&(dst), &(src), (n))
+#define cpumask_scnprintf(buf, len, src) \
+ __cpumask_scnprintf((buf), (len), &(src))
+#define cpumask_parse_user(ubuf, ulen, dst) \
+ __cpumask_parse_user((ubuf), (ulen), &(dst))
+#define cpulist_scnprintf(buf, len, src) \
+ __cpulist_scnprintf((buf), (len), &(src))
+#define cpulist_parse(buf, dst) __cpulist_parse((buf), &(dst))
+#define cpu_remap(oldbit, old, new) \
+ cpumask_cpuremap((oldbit), &(old), &(new))
+#define cpus_remap(dst, src, old, new) \
+ cpumask_remap(&(dst), &(src), &(old), &(new))
+#define cpus_onto(dst, orig, relmap) \
+ cpumask_onto(&(dst), &(orig), &(relmap))
+#define cpus_fold(dst, orig, sz) \
+ cpumask_fold(&(dst), &(orig), sz)
+#define cpus_addr(src) ((src).bits)
+/* End deprecated region. */
+
+static inline void cpumask_set_cpu(int cpu, volatile struct cpumask *dstp)
{
set_bit(cpu, dstp->bits);
}
-#define cpu_clear(cpu, dst) __cpu_clear((cpu), &(dst))
-static inline void __cpu_clear(int cpu, volatile cpumask_t *dstp)
+static inline void cpumask_clear_cpu(int cpu, volatile struct cpumask *dstp)
{
clear_bit(cpu, dstp->bits);
}
-#define cpus_setall(dst) __cpus_setall(&(dst), NR_CPUS)
-static inline void __cpus_setall(cpumask_t *dstp, int nbits)
+/* No static inline type checking - see Subtlety (1) above. */
+#define cpumask_test_cpu(cpu, cpumask) test_bit((cpu), (cpumask)->bits)
+
+static inline int cpumask_test_and_set_cpu(int cpu, struct cpumask *addr)
{
- bitmap_fill(dstp->bits, nbits);
+ return test_and_set_bit(cpu, addr->bits);
}
-#define cpus_clear(dst) __cpus_clear(&(dst), NR_CPUS)
-static inline void __cpus_clear(cpumask_t *dstp, int nbits)
+static inline void cpumask_setall(struct cpumask *dstp)
{
- bitmap_zero(dstp->bits, nbits);
+ bitmap_fill(dstp->bits, NR_CPUS);
}
-/* No static inline type checking - see Subtlety (1) above. */
-#define cpu_isset(cpu, cpumask) test_bit((cpu), (cpumask).bits)
-
-#define cpu_test_and_set(cpu, cpumask) __cpu_test_and_set((cpu), &(cpumask))
-static inline int __cpu_test_and_set(int cpu, cpumask_t *addr)
+static inline void cpumask_clear(struct cpumask *dstp)
{
- return test_and_set_bit(cpu, addr->bits);
+ bitmap_zero(dstp->bits, NR_CPUS);
}
-#define cpus_and(dst, src1, src2) __cpus_and(&(dst), &(src1), &(src2), NR_CPUS)
-static inline void __cpus_and(cpumask_t *dstp, const cpumask_t *src1p,
- const cpumask_t *src2p, int nbits)
+static inline void cpumask_and(struct cpumask *dstp,
+ const struct cpumask *src1p,
+ const struct cpumask *src2p)
{
- bitmap_and(dstp->bits, src1p->bits, src2p->bits, nbits);
+ bitmap_and(dstp->bits, src1p->bits, src2p->bits, NR_CPUS);
}
-#define cpus_or(dst, src1, src2) __cpus_or(&(dst), &(src1), &(src2), NR_CPUS)
-static inline void __cpus_or(cpumask_t *dstp, const cpumask_t *src1p,
- const cpumask_t *src2p, int nbits)
+static inline void cpumask_or(struct cpumask *dstp, const struct cpumask *src1p,
+ const struct cpumask *src2p)
{
- bitmap_or(dstp->bits, src1p->bits, src2p->bits, nbits);
+ bitmap_or(dstp->bits, src1p->bits, src2p->bits, NR_CPUS);
}
-#define cpus_xor(dst, src1, src2) __cpus_xor(&(dst), &(src1), &(src2), NR_CPUS)
-static inline void __cpus_xor(cpumask_t *dstp, const cpumask_t *src1p,
- const cpumask_t *src2p, int nbits)
+static inline void cpumask_xor(struct cpumask *dstp,
+ const struct cpumask *src1p,
+ const struct cpumask *src2p)
{
- bitmap_xor(dstp->bits, src1p->bits, src2p->bits, nbits);
+ bitmap_xor(dstp->bits, src1p->bits, src2p->bits, NR_CPUS);
}
-#define cpus_andnot(dst, src1, src2) \
- __cpus_andnot(&(dst), &(src1), &(src2), NR_CPUS)
-static inline void __cpus_andnot(cpumask_t *dstp, const cpumask_t *src1p,
- const cpumask_t *src2p, int nbits)
+static inline void cpumask_andnot(struct cpumask *dstp,
+ const struct cpumask *src1p,
+ const struct cpumask *src2p)
{
- bitmap_andnot(dstp->bits, src1p->bits, src2p->bits, nbits);
+ bitmap_andnot(dstp->bits, src1p->bits, src2p->bits, NR_CPUS);
}
-#define cpus_complement(dst, src) __cpus_complement(&(dst), &(src), NR_CPUS)
-static inline void __cpus_complement(cpumask_t *dstp,
- const cpumask_t *srcp, int nbits)
+static inline void cpumask_complement(struct cpumask *dstp,
+ const struct cpumask *srcp)
{
- bitmap_complement(dstp->bits, srcp->bits, nbits);
+ bitmap_complement(dstp->bits, srcp->bits, NR_CPUS);
}
-#define cpus_equal(src1, src2) __cpus_equal(&(src1), &(src2), NR_CPUS)
-static inline int __cpus_equal(const cpumask_t *src1p,
- const cpumask_t *src2p, int nbits)
+static inline int cpumask_equal(const struct cpumask *src1p,
+ const struct cpumask *src2p)
{
- return bitmap_equal(src1p->bits, src2p->bits, nbits);
+ return bitmap_equal(src1p->bits, src2p->bits, NR_CPUS);
}
-#define cpus_intersects(src1, src2) __cpus_intersects(&(src1), &(src2), NR_CPUS)
-static inline int __cpus_intersects(const cpumask_t *src1p,
- const cpumask_t *src2p, int nbits)
+static inline int cpumask_intersects(const struct cpumask *src1p,
+ const struct cpumask *src2p)
{
- return bitmap_intersects(src1p->bits, src2p->bits, nbits);
+ return bitmap_intersects(src1p->bits, src2p->bits, NR_CPUS);
}
-#define cpus_subset(src1, src2) __cpus_subset(&(src1), &(src2), NR_CPUS)
-static inline int __cpus_subset(const cpumask_t *src1p,
- const cpumask_t *src2p, int nbits)
+static inline int cpumask_subset(const struct cpumask *src1p,
+ const struct cpumask *src2p)
{
- return bitmap_subset(src1p->bits, src2p->bits, nbits);
+ return bitmap_subset(src1p->bits, src2p->bits, NR_CPUS);
}
-#define cpus_empty(src) __cpus_empty(&(src), NR_CPUS)
-static inline int __cpus_empty(const cpumask_t *srcp, int nbits)
+static inline int cpumask_empty(const struct cpumask *srcp)
{
- return bitmap_empty(srcp->bits, nbits);
+ return bitmap_empty(srcp->bits, NR_CPUS);
}
-#define cpus_full(cpumask) __cpus_full(&(cpumask), NR_CPUS)
-static inline int __cpus_full(const cpumask_t *srcp, int nbits)
+static inline int cpumask_full(const struct cpumask *srcp)
{
- return bitmap_full(srcp->bits, nbits);
+ return bitmap_full(srcp->bits, NR_CPUS);
}
-#define cpus_weight(cpumask) __cpus_weight(&(cpumask), NR_CPUS)
static inline int __cpus_weight(const cpumask_t *srcp, int nbits)
{
return bitmap_weight(srcp->bits, nbits);
}
-#define cpus_shift_right(dst, src, n) \
- __cpus_shift_right(&(dst), &(src), (n), NR_CPUS)
-static inline void __cpus_shift_right(cpumask_t *dstp,
- const cpumask_t *srcp, int n, int nbits)
+static inline int cpumask_weight(const struct cpumask *srcp)
{
- bitmap_shift_right(dstp->bits, srcp->bits, n, nbits);
+ return bitmap_weight(srcp->bits, NR_CPUS);
}
-#define cpus_shift_left(dst, src, n) \
- __cpus_shift_left(&(dst), &(src), (n), NR_CPUS)
-static inline void __cpus_shift_left(cpumask_t *dstp,
- const cpumask_t *srcp, int n, int nbits)
+static inline void cpumask_shift_right(struct cpumask *dstp,
+ const struct cpumask *srcp, int n)
+{
+ bitmap_shift_right(dstp->bits, srcp->bits, n, NR_CPUS);
+}
+
+static inline void cpumask_shift_left(struct cpumask *dstp,
+ const struct cpumask *srcp, int n)
{
- bitmap_shift_left(dstp->bits, srcp->bits, n, nbits);
+ bitmap_shift_left(dstp->bits, srcp->bits, n, NR_CPUS);
+}
+
+static inline int __cpumask_scnprintf(char *buf, int len,
+ const struct cpumask *srcp)
+{
+ return bitmap_scnprintf(buf, len, srcp->bits, NR_CPUS);
+}
+
+static inline int __cpumask_parse_user(const char __user *buf, int len,
+ struct cpumask *dstp)
+{
+ return bitmap_parse_user(buf, len, dstp->bits, NR_CPUS);
+}
+
+static inline int __cpulist_scnprintf(char *buf, int len,
+ const struct cpumask *srcp)
+{
+ return bitmap_scnlistprintf(buf, len, srcp->bits, NR_CPUS);
+}
+
+static inline int __cpulist_parse(const char *buf, struct cpumask *dstp)
+{
+ return bitmap_parselist(buf, dstp->bits, NR_CPUS);
+}
+
+static inline int cpumask_cpuremap(int oldbit,
+ const struct cpumask *oldp,
+ const struct cpumask *newp)
+{
+ return bitmap_bitremap(oldbit, oldp->bits, newp->bits, NR_CPUS);
+}
+
+static inline void cpumask_remap(struct cpumask *dstp,
+ const struct cpumask *srcp,
+ const struct cpumask *oldp,
+ const struct cpumask *newp)
+{
+ bitmap_remap(dstp->bits, srcp->bits, oldp->bits, newp->bits, NR_CPUS);
+}
+
+static inline void cpumask_onto(struct cpumask *dstp,
+ const struct cpumask *origp,
+ const struct cpumask *relmapp)
+{
+ bitmap_onto(dstp->bits, origp->bits, relmapp->bits, NR_CPUS);
+}
+
+static inline void cpumask_fold(struct cpumask *dstp,
+ const struct cpumask *origp, int sz)
+{
+ bitmap_fold(dstp->bits, origp->bits, sz, NR_CPUS);
}
/*
@@ -326,8 +415,6 @@ extern cpumask_t cpu_mask_all;
[0] = 1UL \
} }
-#define cpus_addr(src) ((src).bits)
-
#if NR_CPUS > BITS_PER_LONG
#define CPUMASK_ALLOC(m) struct m *m = kmalloc(sizeof(*m), GFP_KERNEL)
#define CPUMASK_FREE(m) kfree(m)
@@ -337,68 +424,6 @@ extern cpumask_t cpu_mask_all;
#endif
#define CPUMASK_PTR(v, m) cpumask_t *v = &(m->v)
-#define cpumask_scnprintf(buf, len, src) \
- __cpumask_scnprintf((buf), (len), &(src), NR_CPUS)
-static inline int __cpumask_scnprintf(char *buf, int len,
- const cpumask_t *srcp, int nbits)
-{
- return bitmap_scnprintf(buf, len, srcp->bits, nbits);
-}
-
-#define cpumask_parse_user(ubuf, ulen, dst) \
- __cpumask_parse_user((ubuf), (ulen), &(dst), NR_CPUS)
-static inline int __cpumask_parse_user(const char __user *buf, int len,
- cpumask_t *dstp, int nbits)
-{
- return bitmap_parse_user(buf, len, dstp->bits, nbits);
-}
-
-#define cpulist_scnprintf(buf, len, src) \
- __cpulist_scnprintf((buf), (len), &(src), NR_CPUS)
-static inline int __cpulist_scnprintf(char *buf, int len,
- const cpumask_t *srcp, int nbits)
-{
- return bitmap_scnlistprintf(buf, len, srcp->bits, nbits);
-}
-
-#define cpulist_parse(buf, dst) __cpulist_parse((buf), &(dst), NR_CPUS)
-static inline int __cpulist_parse(const char *buf, cpumask_t *dstp, int nbits)
-{
- return bitmap_parselist(buf, dstp->bits, nbits);
-}
-
-#define cpu_remap(oldbit, old, new) \
- __cpu_remap((oldbit), &(old), &(new), NR_CPUS)
-static inline int __cpu_remap(int oldbit,
- const cpumask_t *oldp, const cpumask_t *newp, int nbits)
-{
- return bitmap_bitremap(oldbit, oldp->bits, newp->bits, nbits);
-}
-
-#define cpus_remap(dst, src, old, new) \
- __cpus_remap(&(dst), &(src), &(old), &(new), NR_CPUS)
-static inline void __cpus_remap(cpumask_t *dstp, const cpumask_t *srcp,
- const cpumask_t *oldp, const cpumask_t *newp, int nbits)
-{
- bitmap_remap(dstp->bits, srcp->bits, oldp->bits, newp->bits, nbits);
-}
-
-#define cpus_onto(dst, orig, relmap) \
- __cpus_onto(&(dst), &(orig), &(relmap), NR_CPUS)
-static inline void __cpus_onto(cpumask_t *dstp, const cpumask_t *origp,
- const cpumask_t *relmapp, int nbits)
-{
- bitmap_onto(dstp->bits, origp->bits, relmapp->bits, nbits);
-}
-
-#define cpus_fold(dst, orig, sz) \
- __cpus_fold(&(dst), &(orig), sz, NR_CPUS)
-static inline void __cpus_fold(cpumask_t *dstp, const cpumask_t *origp,
- int sz, int nbits)
-{
- bitmap_fold(dstp->bits, origp->bits, sz, nbits);
-}
-
#if NR_CPUS == 1
#define nr_cpu_ids 1
--- linux-2.6.28.orig/lib/cpumask.c
+++ linux-2.6.28/lib/cpumask.c
@@ -18,7 +18,7 @@ EXPORT_SYMBOL(__next_cpu);
int cpumask_next_and(int n, const cpumask_t *srcp, const cpumask_t *andp)
{
while ((n = next_cpu_nr(n, *srcp)) < nr_cpu_ids)
- if (cpu_isset(n, *andp))
+ if (cpumask_test_cpu(n, andp))
break;
return n;
}
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 07/35] cpumask: change cpumask_scnprintf, cpumask_parse_user, cpulist_parse, and cpulist_scnprintf to take pointers.
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (5 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 06/35] cpumask: introduce struct cpumask Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 08/35] cpumask: cpumask_size() Mike Travis
` (27 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
We want to move cpumasks off the stack: no local decls, no passing by
copy.
Most cpumask functions started with cpus_: these have been replaced by
cpumask_ ones which take struct cpumask pointers as expected, and macro
stubs used for the transition.
These four functions don't have good replacement names; fortunately
they're rarely used, so we just change them over.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
arch/ia64/kernel/topology.c | 2 +-
arch/mips/kernel/smp-cmp.c | 4 ++--
arch/powerpc/platforms/pseries/xics.c | 2 +-
arch/x86/kernel/cpu/intel_cacheinfo.c | 4 ++--
arch/x86/kernel/setup_percpu.c | 2 +-
drivers/base/cpu.c | 2 +-
drivers/base/node.c | 4 ++--
drivers/base/topology.c | 4 ++--
drivers/pci/pci-sysfs.c | 4 ++--
drivers/pci/probe.c | 4 ++--
include/linux/cpumask.h | 21 +++++++--------------
kernel/cpuset.c | 8 ++++----
kernel/irq/proc.c | 4 ++--
kernel/profile.c | 4 ++--
kernel/sched.c | 4 ++--
kernel/sched_stats.h | 2 +-
kernel/taskstats.c | 2 +-
kernel/trace/trace.c | 4 ++--
mm/slub.c | 2 +-
19 files changed, 38 insertions(+), 45 deletions(-)
--- test-compile.orig/arch/ia64/kernel/topology.c
+++ test-compile/arch/ia64/kernel/topology.c
@@ -217,7 +217,7 @@ static ssize_t show_shared_cpu_map(struc
cpumask_t shared_cpu_map;
cpus_and(shared_cpu_map, this_leaf->shared_cpu_map, cpu_online_map);
- len = cpumask_scnprintf(buf, NR_CPUS+1, shared_cpu_map);
+ len = cpumask_scnprintf(buf, NR_CPUS+1, &shared_cpu_map);
len += sprintf(buf+len, "\n");
return len;
}
--- test-compile.orig/arch/mips/kernel/smp-cmp.c
+++ test-compile/arch/mips/kernel/smp-cmp.c
@@ -51,10 +51,10 @@ static int __init allowcpus(char *str)
int len;
cpus_clear(cpu_allow_map);
- if (cpulist_parse(str, cpu_allow_map) == 0) {
+ if (cpulist_parse(str, &cpu_allow_map) == 0) {
cpu_set(0, cpu_allow_map);
cpus_and(cpu_possible_map, cpu_possible_map, cpu_allow_map);
- len = cpulist_scnprintf(buf, sizeof(buf)-1, cpu_possible_map);
+ len = cpulist_scnprintf(buf, sizeof(buf)-1, &cpu_possible_map);
buf[len] = '\0';
pr_debug("Allowable CPUs: %s\n", buf);
return 1;
--- test-compile.orig/arch/powerpc/platforms/pseries/xics.c
+++ test-compile/arch/powerpc/platforms/pseries/xics.c
@@ -358,7 +358,7 @@ static void xics_set_affinity(unsigned i
irq_server = get_irq_server(virq, 1);
if (irq_server == -1) {
char cpulist[128];
- cpumask_scnprintf(cpulist, sizeof(cpulist), cpumask);
+ cpumask_scnprintf(cpulist, sizeof(cpulist), &cpumask);
printk(KERN_WARNING
"%s: No online cpus in the mask %s for irq %d\n",
__func__, cpulist, virq);
--- test-compile.orig/arch/x86/kernel/cpu/intel_cacheinfo.c
+++ test-compile/arch/x86/kernel/cpu/intel_cacheinfo.c
@@ -626,8 +626,8 @@ static ssize_t show_shared_cpu_map_func(
cpumask_t *mask = &this_leaf->shared_cpu_map;
n = type?
- cpulist_scnprintf(buf, len-2, *mask):
- cpumask_scnprintf(buf, len-2, *mask);
+ cpulist_scnprintf(buf, len-2, mask) :
+ cpumask_scnprintf(buf, len-2, mask);
buf[n++] = '\n';
buf[n] = '\0';
}
--- test-compile.orig/arch/x86/kernel/setup_percpu.c
+++ test-compile/arch/x86/kernel/setup_percpu.c
@@ -282,7 +282,7 @@ static void __cpuinit numa_set_cpumask(i
else
cpu_clear(cpu, *mask);
- cpulist_scnprintf(buf, sizeof(buf), *mask);
+ cpulist_scnprintf(buf, sizeof(buf), mask);
printk(KERN_DEBUG "%s cpu %d node %d: mask now %s\n",
enable? "numa_add_cpu":"numa_remove_cpu", cpu, node, buf);
}
--- test-compile.orig/drivers/base/cpu.c
+++ test-compile/drivers/base/cpu.c
@@ -109,7 +109,7 @@ static SYSDEV_ATTR(crash_notes, 0400, sh
*/
static ssize_t print_cpus_map(char *buf, cpumask_t *map)
{
- int n = cpulist_scnprintf(buf, PAGE_SIZE-2, *map);
+ int n = cpulist_scnprintf(buf, PAGE_SIZE-2, map);
buf[n++] = '\n';
buf[n] = '\0';
--- test-compile.orig/drivers/base/node.c
+++ test-compile/drivers/base/node.c
@@ -29,8 +29,8 @@ static ssize_t node_read_cpumap(struct s
BUILD_BUG_ON((NR_CPUS/32 * 9) > (PAGE_SIZE-1));
len = type?
- cpulist_scnprintf(buf, PAGE_SIZE-2, *mask):
- cpumask_scnprintf(buf, PAGE_SIZE-2, *mask);
+ cpulist_scnprintf(buf, PAGE_SIZE-2, mask) :
+ cpumask_scnprintf(buf, PAGE_SIZE-2, mask);
buf[len++] = '\n';
buf[len] = '\0';
return len;
--- test-compile.orig/drivers/base/topology.c
+++ test-compile/drivers/base/topology.c
@@ -49,8 +49,8 @@ static ssize_t show_cpumap(int type, cpu
if (len > 1) {
n = type?
- cpulist_scnprintf(buf, len-2, *mask):
- cpumask_scnprintf(buf, len-2, *mask);
+ cpulist_scnprintf(buf, len-2, mask) :
+ cpumask_scnprintf(buf, len-2, mask);
buf[n++] = '\n';
buf[n] = '\0';
}
--- test-compile.orig/drivers/pci/pci-sysfs.c
+++ test-compile/drivers/pci/pci-sysfs.c
@@ -74,7 +74,7 @@ static ssize_t local_cpus_show(struct de
int len;
mask = pcibus_to_cpumask(to_pci_dev(dev)->bus);
- len = cpumask_scnprintf(buf, PAGE_SIZE-2, mask);
+ len = cpumask_scnprintf(buf, PAGE_SIZE-2, &mask);
buf[len++] = '\n';
buf[len] = '\0';
return len;
@@ -88,7 +88,7 @@ static ssize_t local_cpulist_show(struct
int len;
mask = pcibus_to_cpumask(to_pci_dev(dev)->bus);
- len = cpulist_scnprintf(buf, PAGE_SIZE-2, mask);
+ len = cpulist_scnprintf(buf, PAGE_SIZE-2, &mask);
buf[len++] = '\n';
buf[len] = '\0';
return len;
--- test-compile.orig/drivers/pci/probe.c
+++ test-compile/drivers/pci/probe.c
@@ -123,8 +123,8 @@ static ssize_t pci_bus_show_cpuaffinity(
cpumask = pcibus_to_cpumask(to_pci_bus(dev));
ret = type?
- cpulist_scnprintf(buf, PAGE_SIZE-2, cpumask):
- cpumask_scnprintf(buf, PAGE_SIZE-2, cpumask);
+ cpulist_scnprintf(buf, PAGE_SIZE-2, &cpumask) :
+ cpumask_scnprintf(buf, PAGE_SIZE-2, &cpumask);
buf[ret++] = '\n';
buf[ret] = '\0';
return ret;
--- test-compile.orig/include/linux/cpumask.h
+++ test-compile/include/linux/cpumask.h
@@ -175,13 +175,6 @@ extern cpumask_t _unused_cpumask_arg_;
cpumask_shift_right(&(dst), &(src), (n))
#define cpus_shift_left(dst, src, n) \
cpumask_shift_left(&(dst), &(src), (n))
-#define cpumask_scnprintf(buf, len, src) \
- __cpumask_scnprintf((buf), (len), &(src))
-#define cpumask_parse_user(ubuf, ulen, dst) \
- __cpumask_parse_user((ubuf), (ulen), &(dst))
-#define cpulist_scnprintf(buf, len, src) \
- __cpulist_scnprintf((buf), (len), &(src))
-#define cpulist_parse(buf, dst) __cpulist_parse((buf), &(dst))
#define cpu_remap(oldbit, old, new) \
cpumask_cpuremap((oldbit), &(old), &(new))
#define cpus_remap(dst, src, old, new) \
@@ -304,25 +297,25 @@ static inline void cpumask_shift_left(st
bitmap_shift_left(dstp->bits, srcp->bits, n, NR_CPUS);
}
-static inline int __cpumask_scnprintf(char *buf, int len,
- const struct cpumask *srcp)
+static inline int cpumask_scnprintf(char *buf, int len,
+ const struct cpumask *srcp)
{
return bitmap_scnprintf(buf, len, srcp->bits, NR_CPUS);
}
-static inline int __cpumask_parse_user(const char __user *buf, int len,
- struct cpumask *dstp)
+static inline int cpumask_parse_user(const char __user *buf, int len,
+ struct cpumask *dstp)
{
return bitmap_parse_user(buf, len, dstp->bits, NR_CPUS);
}
-static inline int __cpulist_scnprintf(char *buf, int len,
- const struct cpumask *srcp)
+static inline int cpulist_scnprintf(char *buf, int len,
+ const struct cpumask *srcp)
{
return bitmap_scnlistprintf(buf, len, srcp->bits, NR_CPUS);
}
-static inline int __cpulist_parse(const char *buf, struct cpumask *dstp)
+static inline int cpulist_parse(const char *buf, struct cpumask *dstp)
{
return bitmap_parselist(buf, dstp->bits, NR_CPUS);
}
--- test-compile.orig/kernel/cpuset.c
+++ test-compile/kernel/cpuset.c
@@ -891,7 +891,7 @@ static int update_cpumask(struct cpuset
if (!*buf) {
cpus_clear(trialcs.cpus_allowed);
} else {
- retval = cpulist_parse(buf, trialcs.cpus_allowed);
+ retval = cpulist_parse(buf, &trialcs.cpus_allowed);
if (retval < 0)
return retval;
@@ -1478,7 +1478,7 @@ static int cpuset_sprintf_cpulist(char *
mask = cs->cpus_allowed;
mutex_unlock(&callback_mutex);
- return cpulist_scnprintf(page, PAGE_SIZE, mask);
+ return cpulist_scnprintf(page, PAGE_SIZE, &mask);
}
static int cpuset_sprintf_memlist(char *page, struct cpuset *cs)
@@ -2438,11 +2438,11 @@ void cpuset_task_status_allowed(struct s
{
seq_printf(m, "Cpus_allowed:\t");
m->count += cpumask_scnprintf(m->buf + m->count, m->size - m->count,
- task->cpus_allowed);
+ &task->cpus_allowed);
seq_printf(m, "\n");
seq_printf(m, "Cpus_allowed_list:\t");
m->count += cpulist_scnprintf(m->buf + m->count, m->size - m->count,
- task->cpus_allowed);
+ &task->cpus_allowed);
seq_printf(m, "\n");
seq_printf(m, "Mems_allowed:\t");
m->count += nodemask_scnprintf(m->buf + m->count, m->size - m->count,
--- test-compile.orig/kernel/irq/proc.c
+++ test-compile/kernel/irq/proc.c
@@ -47,7 +47,7 @@ static ssize_t irq_affinity_proc_write(s
irq_balancing_disabled(irq))
return -EIO;
- err = cpumask_parse_user(buffer, count, new_value);
+ err = cpumask_parse_user(buffer, count, &new_value);
if (err)
return err;
@@ -95,7 +95,7 @@ static ssize_t default_affinity_write(st
cpumask_t new_value;
int err;
- err = cpumask_parse_user(buffer, count, new_value);
+ err = cpumask_parse_user(buffer, count, &new_value);
if (err)
return err;
--- test-compile.orig/kernel/profile.c
+++ test-compile/kernel/profile.c
@@ -442,7 +442,7 @@ void profile_tick(int type)
static int prof_cpu_mask_read_proc(char *page, char **start, off_t off,
int count, int *eof, void *data)
{
- int len = cpumask_scnprintf(page, count, *(cpumask_t *)data);
+ int len = cpumask_scnprintf(page, count, (cpumask_t *)data);
if (count - len < 2)
return -EINVAL;
len += sprintf(page + len, "\n");
@@ -456,7 +456,7 @@ static int prof_cpu_mask_write_proc(stru
unsigned long full_count = count, err;
cpumask_t new_value;
- err = cpumask_parse_user(buffer, count, new_value);
+ err = cpumask_parse_user(buffer, count, &new_value);
if (err)
return err;
--- test-compile.orig/kernel/sched.c
+++ test-compile/kernel/sched.c
@@ -6641,7 +6641,7 @@ static int sched_domain_debug_one(struct
struct sched_group *group = sd->groups;
char str[256];
- cpulist_scnprintf(str, sizeof(str), sd->span);
+ cpulist_scnprintf(str, sizeof(str), &sd->span);
cpus_clear(*groupmask);
printk(KERN_DEBUG "%*s domain %d: ", level, "", level);
@@ -6695,7 +6695,7 @@ static int sched_domain_debug_one(struct
cpus_or(*groupmask, *groupmask, group->cpumask);
- cpulist_scnprintf(str, sizeof(str), group->cpumask);
+ cpulist_scnprintf(str, sizeof(str), &group->cpumask);
printk(KERN_CONT " %s", str);
group = group->next;
--- test-compile.orig/kernel/sched_stats.h
+++ test-compile/kernel/sched_stats.h
@@ -42,7 +42,7 @@ static int show_schedstat(struct seq_fil
for_each_domain(cpu, sd) {
enum cpu_idle_type itype;
- cpumask_scnprintf(mask_str, mask_len, sd->span);
+ cpumask_scnprintf(mask_str, mask_len, &sd->span);
seq_printf(seq, "domain%d %s", dcount++, mask_str);
for (itype = CPU_IDLE; itype < CPU_MAX_IDLE_TYPES;
itype++) {
--- test-compile.orig/kernel/taskstats.c
+++ test-compile/kernel/taskstats.c
@@ -352,7 +352,7 @@ static int parse(struct nlattr *na, cpum
if (!data)
return -ENOMEM;
nla_strlcpy(data, na, len);
- ret = cpulist_parse(data, *mask);
+ ret = cpulist_parse(data, mask);
kfree(data);
return ret;
}
--- test-compile.orig/kernel/trace/trace.c
+++ test-compile/kernel/trace/trace.c
@@ -2117,7 +2117,7 @@ tracing_cpumask_read(struct file *filp,
mutex_lock(&tracing_cpumask_update_lock);
- len = cpumask_scnprintf(mask_str, count, tracing_cpumask);
+ len = cpumask_scnprintf(mask_str, count, &tracing_cpumask);
if (count - len < 2) {
count = -EINVAL;
goto out_err;
@@ -2138,7 +2138,7 @@ tracing_cpumask_write(struct file *filp,
int err, cpu;
mutex_lock(&tracing_cpumask_update_lock);
- err = cpumask_parse_user(ubuf, count, tracing_cpumask_new);
+ err = cpumask_parse_user(ubuf, count, &tracing_cpumask_new);
if (err)
goto err_unlock;
--- test-compile.orig/mm/slub.c
+++ test-compile/mm/slub.c
@@ -3637,7 +3637,7 @@ static int list_locations(struct kmem_ca
len < PAGE_SIZE - 60) {
len += sprintf(buf + len, " cpus=");
len += cpulist_scnprintf(buf + len, PAGE_SIZE - len - 50,
- l->cpus);
+ &l->cpus);
}
if (num_online_nodes() > 1 && !nodes_empty(l->nodes) &&
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 08/35] cpumask: cpumask_size()
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (6 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 07/35] cpumask: change cpumask_scnprintf, cpumask_parse_user, cpulist_parse, and cpulist_scnprintf to take pointers Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 09/35] cpumask: add cpumask_copy() Mike Travis
` (26 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Dynamic allocation of cpumasks requires the size.
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 6 ++++++
1 file changed, 6 insertions(+)
--- test-compile.orig/include/linux/cpumask.h
+++ test-compile/include/linux/cpumask.h
@@ -64,6 +64,7 @@
* int next_cpu(cpu, mask) Next cpu past 'cpu', or NR_CPUS
* int next_cpu_nr(cpu, mask) Next cpu past 'cpu', or nr_cpu_ids
*
+ * size_t cpumask_size() Length of cpumask in bytes.
* cpumask_t cpumask_of_cpu(cpu) Return cpumask with bit 'cpu' set
* (can be used as an lvalue)
* CPU_MASK_ALL Initializer - all bits set
@@ -148,6 +149,11 @@ struct cpumask
};
#define cpumask_bits(maskp) ((maskp)->bits)
+static inline ssize_t cpumask_size(void)
+{
+ return BITS_TO_LONGS(NR_CPUS) * sizeof(long);
+}
+
/* Deprecated. */
typedef struct cpumask cpumask_t;
extern cpumask_t _unused_cpumask_arg_;
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 09/35] cpumask: add cpumask_copy()
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (7 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 08/35] cpumask: cpumask_size() Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 10/35] cpumask: introduce cpumask_var_t for local cpumask vars Mike Travis
` (25 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Since cpumasks are to become pointers to undefined structs, we need to
replace assignments. Also, dynamically allocated ones will eventually
be nr_cpu_ids bits (<= NR_CPUS), so assignment is a definite no-no.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 8 ++++++++
1 file changed, 8 insertions(+)
--- linux-2.6.28.orig/include/linux/cpumask.h
+++ linux-2.6.28/include/linux/cpumask.h
@@ -64,6 +64,8 @@
* int next_cpu(cpu, mask) Next cpu past 'cpu', or NR_CPUS
* int next_cpu_nr(cpu, mask) Next cpu past 'cpu', or nr_cpu_ids
*
+ * void cpumask_copy(dmask, smask) dmask = smask
+ *
* size_t cpumask_size() Length of cpumask in bytes.
* cpumask_t cpumask_of_cpu(cpu) Return cpumask with bit 'cpu' set
* (can be used as an lvalue)
@@ -351,6 +353,12 @@ static inline void cpumask_fold(struct c
bitmap_fold(dstp->bits, origp->bits, sz, NR_CPUS);
}
+static inline void cpumask_copy(struct cpumask *dstp,
+ const struct cpumask *srcp)
+{
+ bitmap_copy(cpumask_bits(dstp), cpumask_bits(srcp), NR_CPUS);
+}
+
/*
* Special-case data structure for "single bit set only" constant CPU masks.
*
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 10/35] cpumask: introduce cpumask_var_t for local cpumask vars
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (8 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 09/35] cpumask: add cpumask_copy() Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 11/35] x86: enable MAXSMP Mike Travis
` (24 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
We want to move cpumasks off the stack: no local decls, no passing by
copy. Linus suggested we use the array-or-pointer trick for on-stack
vars; we introduce a new cpumask_var_t for this.
Rather than pick an arbitrary limit, I chose a new config option so
arch maintainers can decide where their threshold is.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 36 ++++++++++++++++++++++++++++++++++++
lib/cpumask.c | 31 +++++++++++++++++++++++++++++++
2 files changed, 67 insertions(+)
--- test-compile.orig/include/linux/cpumask.h
+++ test-compile/include/linux/cpumask.h
@@ -485,6 +485,42 @@ int __next_cpu_nr(int n, const cpumask_t
#endif /* NR_CPUS > 64 */
/*
+ * cpumask_var_t: struct cpumask for stack usage.
+ *
+ * Oh, the wicked games we play! In order to make kernel coding a
+ * little more difficult, we typedef cpumask_var_t to an array or a
+ * pointer: doing &mask on an array is a noop, so it still works.
+ *
+ * ie.
+ * cpumask_var_t tmpmask;
+ * if (!alloc_cpumask_var(&tmpmask, GFP_KERNEL))
+ * return -ENOMEM;
+ *
+ * ... use 'tmpmask' like a normal struct cpumask * ...
+ *
+ * free_cpumask_var(tmpmask);
+ */
+#ifdef CONFIG_CPUMASK_OFFSTACK
+typedef struct cpumask *cpumask_var_t;
+
+bool alloc_cpumask_var(cpumask_var_t *mask, gfp_t flags);
+void free_cpumask_var(cpumask_var_t mask);
+
+#else
+typedef struct cpumask cpumask_var_t[1];
+
+static inline bool alloc_cpumask_var(cpumask_var_t *mask, gfp_t flags)
+{
+ return true;
+}
+
+static inline void free_cpumask_var(cpumask_var_t mask)
+{
+}
+
+#endif /* CONFIG_CPUMASK_OFFSTACK */
+
+/*
* The following particular system cpumasks and operations manage
* possible, present, active and online cpus. Each of them is a fixed size
* bitmap of size NR_CPUS.
--- test-compile.orig/lib/cpumask.c
+++ test-compile/lib/cpumask.c
@@ -43,3 +43,34 @@ int __any_online_cpu(const cpumask_t *ma
return cpu;
}
EXPORT_SYMBOL(__any_online_cpu);
+
+/* These are not inline because of header tangles. */
+#ifdef CONFIG_CPUMASK_OFFSTACK
+bool alloc_cpumask_var(cpumask_var_t *mask, gfp_t flags)
+{
+ if (likely(slab_is_available()))
+ *mask = kmalloc(BITS_TO_LONGS(nr_cpu_ids)*sizeof(long), flags);
+ else {
+#ifdef CONFIG_DEBUG_PER_CPU_MAPS
+ printk(KERN_ERR
+ "=> alloc_cpumask_var: kmalloc not available!\n");
+ dump_stack();
+#endif
+ *mask = NULL;
+ }
+#ifdef CONFIG_DEBUG_PER_CPU_MAPS
+ if (!*mask) {
+ printk(KERN_ERR "=> alloc_cpumask_var: failed!\n");
+ dump_stack();
+ }
+#endif
+ return *mask != NULL;
+}
+EXPORT_SYMBOL(alloc_cpumask_var);
+
+void free_cpumask_var(cpumask_var_t mask)
+{
+ kfree(mask);
+}
+EXPORT_SYMBOL(free_cpumask_var);
+#endif
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 11/35] x86: enable MAXSMP
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (9 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 10/35] cpumask: introduce cpumask_var_t for local cpumask vars Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 12/35] cpumask: make CONFIG_NR_CPUS always valid Mike Travis
` (23 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Set MAXSMP to enable a configuration with the maximum amount of CPUS and
NODES. Also enables CONFIG_CPUMASK_OFFSTACK which moves cpumask's off
the stack (and in structs) when using cpumask_var_t.
From: Mike Travis <travis@sgi.com>
Acked-by: Rusty Russell <rusty@rustcorp.com.au>
---
arch/x86/Kconfig | 29 ++++++++++++++++++++++++++---
1 file changed, 26 insertions(+), 3 deletions(-)
--- test-compile.orig/arch/x86/Kconfig
+++ test-compile/arch/x86/Kconfig
@@ -571,12 +571,20 @@ config IOMMU_HELPER
config MAXSMP
bool "Configure Maximum number of SMP Processors and NUMA Nodes"
- depends on X86_64 && SMP && BROKEN
+ depends on X86_64 && SMP && DEBUG_KERNEL && EXPERIMENTAL
+ select CPUMASK_OFFSTACK
default n
help
Configure maximum number of CPUS and NUMA Nodes for this architecture.
If unsure, say N.
+if MAXSMP
+config NR_CPUS
+ int
+ default "4096"
+endif
+
+if !MAXSMP
config NR_CPUS
int "Maximum number of CPUs (2-512)" if !MAXSMP
range 2 512
@@ -591,6 +599,14 @@ config NR_CPUS
This is purely to save memory - each supported CPU adds
approximately eight kilobytes to the kernel image.
+endif
+
+config CPUMASK_OFFSTACK
+ bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPS
+ help
+ Use dynamic allocation for cpumask_var_t, instead of putting
+ them on the stack. This is a bit more expensive, but avoids
+ stack overflow.
config SCHED_SMT
bool "SMT (Hyperthreading) scheduler support"
@@ -1027,10 +1043,16 @@ config NUMA_EMU
into virtual nodes when booted with "numa=fake=N", where N is the
number of nodes. This is only useful for debugging.
+if MAXSMP
+config NODES_SHIFT
+ int
+ default "9"
+endif
+
+if !MAXSMP
config NODES_SHIFT
- int "Maximum NUMA Nodes (as a power of 2)" if !MAXSMP
+ int "Maximum NUMA Nodes (as a power of 2)"
range 1 9 if X86_64
- default "9" if MAXSMP
default "6" if X86_64
default "4" if X86_NUMAQ
default "3"
@@ -1038,6 +1060,7 @@ config NODES_SHIFT
help
Specify the maximum number of NUMA Nodes available on the target
system. Increases memory reserved to accomodate various tables.
+endif
config HAVE_ARCH_BOOTMEM_NODE
def_bool y
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 12/35] cpumask: make CONFIG_NR_CPUS always valid.
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (10 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 11/35] x86: enable MAXSMP Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 13/35] cpumask: use setup_nr_cpu_ids() instead of direct assignment Mike Travis
` (22 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Currently we have NR_CPUS, which is 1 on UP, and CONFIG_NR_CPUS on
SMP. If we make CONFIG_NR_CPUS always valid (and always 1 on !SMP),
we can skip the middleman.
This also allows us to find and check all the remaining NR_CPUS users.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
arch/alpha/Kconfig | 10 +++++-----
arch/arm/Kconfig | 6 +++---
arch/ia64/Kconfig | 8 ++++----
arch/m32r/Kconfig | 8 ++++----
arch/mips/Kconfig | 4 ++--
arch/parisc/Kconfig | 8 ++++----
arch/s390/Kconfig | 10 +++++-----
arch/sh/Kconfig | 10 +++++-----
arch/sparc/Kconfig | 8 ++++----
arch/sparc64/Kconfig | 8 ++++----
arch/um/Kconfig | 8 ++++----
arch/x86/Kconfig | 11 +++++------
include/linux/threads.h | 10 ++--------
13 files changed, 51 insertions(+), 58 deletions(-)
--- test-compile.orig/arch/alpha/Kconfig
+++ test-compile/arch/alpha/Kconfig
@@ -543,11 +543,11 @@ config HAVE_DEC_LOCK
default y
config NR_CPUS
- int "Maximum number of CPUs (2-32)"
- range 2 32
- depends on SMP
- default "32" if ALPHA_GENERIC || ALPHA_MARVEL
- default "4" if !ALPHA_GENERIC && !ALPHA_MARVEL
+ int "Maximum number of CPUs (2-32)" if SMP
+ range 2 32 if SMP
+ default "1" if !SMP
+ default "32" if SMP && (ALPHA_GENERIC || ALPHA_MARVEL)
+ default "4" if SMP && (!ALPHA_GENERIC && !ALPHA_MARVEL)
help
MARVEL support can handle a maximum of 32 CPUs, all the others
with working support have a maximum of 4 CPUs.
--- test-compile.orig/arch/arm/Kconfig
+++ test-compile/arch/arm/Kconfig
@@ -767,9 +767,9 @@ config PAGE_OFFSET
default 0xC0000000
config NR_CPUS
- int "Maximum number of CPUs (2-32)"
- range 2 32
- depends on SMP
+ int "Maximum number of CPUs (2-32)" if SMP
+ range 2 32 if SMP
+ default "1" if !SMP
default "4"
config HOTPLUG_CPU
--- test-compile.orig/arch/ia64/Kconfig
+++ test-compile/arch/ia64/Kconfig
@@ -313,10 +313,10 @@ config SMP
If you don't know what to do here, say N.
config NR_CPUS
- int "Maximum number of CPUs (2-4096)"
- range 2 4096
- depends on SMP
- default "4096"
+ int "Maximum number of CPUs (2-4096)" if SMP
+ range 2 4096 if SMP
+ default "1" if !SMP
+ default "4096" if SMP
help
You should set this to the number of CPUs in your system, but
keep in mind that a kernel compiled for, e.g., 2 CPUs will boot but
--- test-compile.orig/arch/m32r/Kconfig
+++ test-compile/arch/m32r/Kconfig
@@ -316,10 +316,10 @@ config CHIP_M32700_TS1
default n
config NR_CPUS
- int "Maximum number of CPUs (2-32)"
- range 2 32
- depends on SMP
- default "2"
+ int "Maximum number of CPUs (2-32)" if SMP
+ range 2 32 if SMP
+ default "1" if !SMP
+ default "2" if SMP
help
This allows you to specify the maximum number of CPUs which this
kernel will support. The maximum supported value is 32 and the
--- test-compile.orig/arch/mips/Kconfig
+++ test-compile/arch/mips/Kconfig
@@ -1719,9 +1719,9 @@ config NR_CPUS_DEFAULT_64
bool
config NR_CPUS
- int "Maximum number of CPUs (2-64)"
+ int "Maximum number of CPUs (2-64)" if SMP
range 1 64 if NR_CPUS_DEFAULT_1
- depends on SMP
+ default "1" if !SMP
default "1" if NR_CPUS_DEFAULT_1
default "2" if NR_CPUS_DEFAULT_2
default "4" if NR_CPUS_DEFAULT_4
--- test-compile.orig/arch/parisc/Kconfig
+++ test-compile/arch/parisc/Kconfig
@@ -251,10 +251,10 @@ config HPUX
depends on !64BIT
config NR_CPUS
- int "Maximum number of CPUs (2-32)"
- range 2 32
- depends on SMP
- default "32"
+ int "Maximum number of CPUs (2-32)" if SMP
+ range 2 32 if SMP
+ default "1" if !SMP
+ default "32" if SMP
endmenu
--- test-compile.orig/arch/s390/Kconfig
+++ test-compile/arch/s390/Kconfig
@@ -113,11 +113,11 @@ config SMP
Even if you don't know what to do here, say Y.
config NR_CPUS
- int "Maximum number of CPUs (2-64)"
- range 2 64
- depends on SMP
- default "32" if !64BIT
- default "64" if 64BIT
+ int "Maximum number of CPUs (2-64)" if SMP
+ range 2 64 if SMP
+ default "1" if !SMP
+ default "32" if SMP && !64BIT
+ default "64" if SMP && 64BIT
help
This allows you to specify the maximum number of CPUs which this
kernel will support. The maximum supported value is 64 and the
--- test-compile.orig/arch/sh/Kconfig
+++ test-compile/arch/sh/Kconfig
@@ -528,11 +528,11 @@ config SMP
If you don't know what to do here, say N.
config NR_CPUS
- int "Maximum number of CPUs (2-32)"
- range 2 32
- depends on SMP
- default "4" if CPU_SHX3
- default "2"
+ int "Maximum number of CPUs (2-32)" if SMP
+ range 2 32 if SMP
+ default "1" if !SMP
+ default "4" if SMP && CPU_SHX3
+ default "2" if SMP
help
This allows you to specify the maximum number of CPUs which this
kernel will support. The maximum supported value is 32 and the
--- test-compile.orig/arch/sparc/Kconfig
+++ test-compile/arch/sparc/Kconfig
@@ -62,10 +62,10 @@ config SMP
If you don't know what to do here, say N.
config NR_CPUS
- int "Maximum number of CPUs (2-32)"
- range 2 32
- depends on SMP
- default "32"
+ int "Maximum number of CPUs (2-32)" if SMP
+ range 2 32 if SMP
+ default "1" if SMP
+ default "32" if SMP
config SPARC
bool
--- test-compile.orig/arch/sparc64/Kconfig
+++ test-compile/arch/sparc64/Kconfig
@@ -168,10 +168,10 @@ config SMP
If you don't know what to do here, say N.
config NR_CPUS
- int "Maximum number of CPUs (2-1024)"
- range 2 1024
- depends on SMP
- default "64"
+ int "Maximum number of CPUs (2-1024)" if SMP
+ range 2 1024 if SMP
+ default "1" if !SMP
+ default "64" if SMP
source "drivers/cpufreq/Kconfig"
--- test-compile.orig/arch/um/Kconfig
+++ test-compile/arch/um/Kconfig
@@ -198,10 +198,10 @@ config SMP
If you don't know what to do, say N.
config NR_CPUS
- int "Maximum number of CPUs (2-32)"
- range 2 32
- depends on SMP
- default "32"
+ int "Maximum number of CPUs (2-32)" if SMP
+ range 2 32 if SMP
+ default "1" if !SMP
+ default "32" if SMP
config HIGHMEM
bool "Highmem support (EXPERIMENTAL)"
--- test-compile.orig/arch/x86/Kconfig
+++ test-compile/arch/x86/Kconfig
@@ -586,12 +586,11 @@ endif
if !MAXSMP
config NR_CPUS
- int "Maximum number of CPUs (2-512)" if !MAXSMP
- range 2 512
- depends on SMP
- default "4096" if MAXSMP
- default "32" if X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || X86_ES7000
- default "8"
+ int "Maximum number of CPUs (2-512)" if SMP
+ range 2 512 if SMP
+ default "1" if !SMP
+ default "32" if SMP && (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || X86_ES7000)
+ default "8" if SMP
help
This allows you to specify the maximum number of CPUs which this
kernel will support. The maximum supported value is 512 and the
--- test-compile.orig/include/linux/threads.h
+++ test-compile/include/linux/threads.h
@@ -8,16 +8,10 @@
*/
/*
- * Maximum supported processors that can run under SMP. This value is
- * set via configure setting. The maximum is equal to the size of the
- * bitmasks used on that platform, i.e. 32 or 64. Setting this smaller
- * saves quite a bit of memory.
+ * Maximum supported processors. Setting this smaller saves quite a
+ * bit of memory. Use nr_cpu_ids instead of this except for bitmaps.
*/
-#ifdef CONFIG_SMP
#define NR_CPUS CONFIG_NR_CPUS
-#else
-#define NR_CPUS 1
-#endif
#define MIN_THREADS_LEFT_FOR_ROOT 4
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 13/35] cpumask: use setup_nr_cpu_ids() instead of direct assignment.
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (11 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 12/35] cpumask: make CONFIG_NR_CPUS always valid Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 14/35] cpumask: make nr_cpu_ids valid in all configurations Mike Travis
` (21 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
nr_cpu_ids is going to become a constant under some configs, so don't
assign it. Currently only x86 seems to anyway.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
arch/x86/kernel/smpboot.c | 4 ++--
include/linux/cpumask.h | 4 ++++
init/main.c | 4 ++--
3 files changed, 8 insertions(+), 4 deletions(-)
--- linux-2.6.28.orig/arch/x86/kernel/smpboot.c
+++ linux-2.6.28/arch/x86/kernel/smpboot.c
@@ -1079,7 +1079,7 @@ static int __init smp_sanity_check(unsig
nr++;
}
- nr_cpu_ids = 8;
+ setup_nr_cpu_ids();
}
#endif
@@ -1296,7 +1296,7 @@ __init void prefill_possible_map(void)
for (i = 0; i < possible; i++)
cpu_set(i, cpu_possible_map);
- nr_cpu_ids = possible;
+ setup_nr_cpu_ids();
}
#ifdef CONFIG_HOTPLUG_CPU
--- linux-2.6.28.orig/include/linux/cpumask.h
+++ linux-2.6.28/include/linux/cpumask.h
@@ -144,6 +144,7 @@
#include <linux/kernel.h>
#include <linux/threads.h>
#include <linux/bitmap.h>
+#include <linux/init.h>
struct cpumask
{
@@ -636,4 +637,7 @@ extern cpumask_t cpu_active_map;
#define for_each_online_cpu(cpu) for_each_cpu_mask_nr((cpu), cpu_online_map)
#define for_each_present_cpu(cpu) for_each_cpu_mask_nr((cpu), cpu_present_map)
+/* Arch-specific code may call this to initialize nr_cpu_ids based on
+ * the cpu_possible_map. */
+void __init setup_nr_cpu_ids(void);
#endif /* __LINUX_CPUMASK_H */
--- linux-2.6.28.orig/init/main.c
+++ linux-2.6.28/init/main.c
@@ -362,7 +362,7 @@ static void __init smp_init(void)
#endif
static inline void setup_per_cpu_areas(void) { }
-static inline void setup_nr_cpu_ids(void) { }
+void __init setup_nr_cpu_ids(void) { }
static inline void smp_prepare_cpus(unsigned int maxcpus) { }
#else
@@ -377,7 +377,7 @@ int nr_cpu_ids __read_mostly = NR_CPUS;
EXPORT_SYMBOL(nr_cpu_ids);
/* An arch may set nr_cpu_ids earlier if needed, so this would be redundant */
-static void __init setup_nr_cpu_ids(void)
+void __init setup_nr_cpu_ids(void)
{
int cpu, highest_cpu = 0;
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 14/35] cpumask: make nr_cpu_ids valid in all configurations.
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (12 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 13/35] cpumask: use setup_nr_cpu_ids() instead of direct assignment Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 15/35] cpumask: prepare for iterators to only go to nr_cpu_ids Mike Travis
` (20 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
nr_cpu_ids is the (badly named) runtime limit on possible CPU numbers;
ie. the variable version of NR_CPUS.
This makes is valid in all configs, including UP.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 10 ++++++++--
init/main.c | 7 +++++++
2 files changed, 15 insertions(+), 2 deletions(-)
--- test-compile.orig/include/linux/cpumask.h
+++ test-compile/include/linux/cpumask.h
@@ -195,6 +195,14 @@ extern cpumask_t _unused_cpumask_arg_;
#define cpus_addr(src) ((src).bits)
/* End deprecated region. */
+#if NR_CPUS <= BITS_PER_LONG
+/* Constant is usually more efficient than a variable for small NR_CPUS */
+#define nr_cpu_ids NR_CPUS
+#else
+/* Starts at NR_CPUS until we know better. */
+extern int nr_cpu_ids;
+#endif /* NR_CPUS > BITS_PER_LONG */
+
static inline void cpumask_set_cpu(int cpu, volatile struct cpumask *dstp)
{
set_bit(cpu, dstp->bits);
@@ -434,7 +442,6 @@ extern cpumask_t cpu_mask_all;
#if NR_CPUS == 1
-#define nr_cpu_ids 1
#define first_cpu(src) ({ (void)(src); 0; })
#define next_cpu(n, src) ({ (void)(src); 1; })
#define cpumask_next_and(n, srcp, andp) ({ (void)(srcp), (void)(andp); 1; })
@@ -447,7 +454,6 @@ extern cpumask_t cpu_mask_all;
#else /* NR_CPUS > 1 */
-extern int nr_cpu_ids;
int __first_cpu(const cpumask_t *srcp);
int __next_cpu(int n, const cpumask_t *srcp);
int cpumask_next_and(int n, const cpumask_t *srcp, const cpumask_t *andp);
--- test-compile.orig/init/main.c
+++ test-compile/init/main.c
@@ -373,6 +373,8 @@ EXPORT_SYMBOL(cpu_mask_all);
#endif
/* Setup number of possible processor ids */
+/* nr_cpu_ids is a real variable for large NR_CPUS. */
+#ifndef nr_cpu_ids
int nr_cpu_ids __read_mostly = NR_CPUS;
EXPORT_SYMBOL(nr_cpu_ids);
@@ -386,6 +388,11 @@ void __init setup_nr_cpu_ids(void)
nr_cpu_ids = highest_cpu + 1;
}
+#else
+void __init setup_nr_cpu_ids(void)
+{
+}
+#endif /* ... nr_cpu_ids is a constant. */
#ifndef CONFIG_HAVE_SETUP_PER_CPU_AREA
unsigned long __per_cpu_offset[NR_CPUS] __read_mostly;
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 15/35] cpumask: prepare for iterators to only go to nr_cpu_ids.
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (13 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 14/35] cpumask: make nr_cpu_ids valid in all configurations Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 16/35] percpu: fix percpu accessors to potentially !cpu_possible() cpus Mike Travis
` (19 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
In fact, all cpumask ops will only be valid (in general) for bit
numbers < nr_cpu_ids. So use that instead of NR_CPUS in various
places.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
arch/alpha/kernel/irq.c | 3 ++-
arch/alpha/kernel/smp.c | 8 ++++----
arch/arm/kernel/irq.c | 2 +-
arch/cris/kernel/setup.c | 2 +-
arch/frv/kernel/setup.c | 2 +-
arch/h8300/kernel/setup.c | 2 +-
arch/ia64/kernel/acpi.c | 2 +-
arch/ia64/kernel/iosapic.c | 4 ++--
arch/ia64/kernel/irq.c | 2 +-
arch/ia64/kernel/mca.c | 6 +++---
arch/ia64/kernel/perfmon.c | 4 ++--
arch/ia64/kernel/salinfo.c | 6 +++---
arch/ia64/kernel/setup.c | 4 ++--
arch/ia64/sn/kernel/setup.c | 2 +-
arch/ia64/sn/kernel/sn2/sn2_smp.c | 6 +++---
arch/ia64/sn/kernel/sn2/sn_hwperf.c | 2 +-
arch/m32r/kernel/setup.c | 2 +-
arch/m68knommu/kernel/setup.c | 2 +-
arch/mips/kernel/irq-gic.c | 2 +-
arch/mips/kernel/proc.c | 2 +-
arch/mips/kernel/smp-cmp.c | 2 +-
arch/mips/kernel/smtc.c | 6 +++---
arch/mips/sibyte/bcm1480/irq.c | 2 +-
arch/mips/sibyte/bcm1480/smp.c | 2 +-
arch/mips/sibyte/sb1250/smp.c | 2 +-
arch/mn10300/kernel/irq.c | 4 ++--
arch/mn10300/kernel/setup.c | 2 +-
arch/parisc/kernel/irq.c | 4 ++--
arch/parisc/kernel/processor.c | 4 ++--
arch/powerpc/include/asm/cputhreads.h | 2 +-
arch/powerpc/kernel/irq.c | 2 +-
arch/powerpc/kernel/machine_kexec_64.c | 2 +-
arch/powerpc/kernel/process.c | 2 +-
arch/powerpc/kernel/setup-common.c | 10 +++++-----
arch/powerpc/mm/numa.c | 4 ++--
arch/powerpc/platforms/powermac/setup.c | 2 +-
arch/powerpc/platforms/powermac/smp.c | 4 ++--
arch/powerpc/platforms/pseries/hotplug-cpu.c | 2 +-
arch/powerpc/platforms/pseries/rtasd.c | 2 +-
arch/powerpc/platforms/pseries/xics.c | 2 +-
arch/powerpc/xmon/xmon.c | 4 ++--
arch/s390/kernel/smp.c | 10 +++++-----
arch/sh/kernel/setup.c | 2 +-
arch/sparc/kernel/smp.c | 11 +++++------
arch/sparc/kernel/sun4d_smp.c | 9 ++++-----
arch/sparc/kernel/sun4m_smp.c | 8 +++-----
arch/sparc/mm/srmmu.c | 2 +-
arch/sparc64/kernel/ds.c | 2 +-
arch/sparc64/kernel/irq.c | 4 ++--
arch/sparc64/mm/init.c | 2 +-
arch/um/kernel/um_arch.c | 2 +-
arch/x86/kernel/apic.c | 2 +-
arch/x86/kernel/irq_32.c | 2 +-
arch/x86/mach-voyager/voyager_smp.c | 2 +-
arch/x86/mm/numa_64.c | 4 ++--
arch/x86/mm/srat_64.c | 2 +-
drivers/infiniband/hw/ehca/ehca_irq.c | 2 +-
kernel/kexec.c | 2 +-
kernel/smp.c | 2 +-
net/core/neighbour.c | 4 ++--
net/ipv4/netfilter/nf_conntrack_l3proto_ipv4_compat.c | 4 ++--
net/ipv4/route.c | 4 ++--
net/netfilter/nf_conntrack_standalone.c | 4 ++--
security/selinux/selinuxfs.c | 2 +-
64 files changed, 108 insertions(+), 111 deletions(-)
--- test-compile.orig/arch/alpha/kernel/irq.c
+++ test-compile/arch/alpha/kernel/irq.c
@@ -50,8 +50,9 @@ int irq_select_affinity(unsigned int irq
if (!irq_desc[irq].chip->set_affinity || irq_user_affinity[irq])
return 1;
+ /* FIXME: This has an out-by-one error: inc then test! */
while (!cpu_possible(cpu) || !cpu_isset(cpu, irq_default_affinity))
- cpu = (cpu < (NR_CPUS-1) ? cpu + 1 : 0);
+ cpu = (cpu < (nr_cpu_ids-1) ? cpu + 1 : 0);
last_cpu = cpu;
irq_desc[irq].affinity = cpumask_of_cpu(cpu);
--- test-compile.orig/arch/alpha/kernel/smp.c
+++ test-compile/arch/alpha/kernel/smp.c
@@ -502,7 +502,7 @@ smp_cpus_done(unsigned int max_cpus)
int cpu;
unsigned long bogosum = 0;
- for(cpu = 0; cpu < NR_CPUS; cpu++)
+ for(cpu = 0; cpu < nr_cpu_ids; cpu++)
if (cpu_online(cpu))
bogosum += cpu_data[cpu].loops_per_jiffy;
@@ -703,7 +703,7 @@ flush_tlb_mm(struct mm_struct *mm)
flush_tlb_current(mm);
if (atomic_read(&mm->mm_users) <= 1) {
int cpu, this_cpu = smp_processor_id();
- for (cpu = 0; cpu < NR_CPUS; cpu++) {
+ for (cpu = 0; cpu < nr_cpu_ids; cpu++) {
if (!cpu_online(cpu) || cpu == this_cpu)
continue;
if (mm->context[cpu])
@@ -752,7 +752,7 @@ flush_tlb_page(struct vm_area_struct *vm
flush_tlb_current_page(mm, vma, addr);
if (atomic_read(&mm->mm_users) <= 1) {
int cpu, this_cpu = smp_processor_id();
- for (cpu = 0; cpu < NR_CPUS; cpu++) {
+ for (cpu = 0; cpu < nr_cpu_ids; cpu++) {
if (!cpu_online(cpu) || cpu == this_cpu)
continue;
if (mm->context[cpu])
@@ -808,7 +808,7 @@ flush_icache_user_range(struct vm_area_s
__load_new_mm_context(mm);
if (atomic_read(&mm->mm_users) <= 1) {
int cpu, this_cpu = smp_processor_id();
- for (cpu = 0; cpu < NR_CPUS; cpu++) {
+ for (cpu = 0; cpu < nr_cpu_ids; cpu++) {
if (!cpu_online(cpu) || cpu == this_cpu)
continue;
if (mm->context[cpu])
--- test-compile.orig/arch/arm/kernel/irq.c
+++ test-compile/arch/arm/kernel/irq.c
@@ -193,7 +193,7 @@ void migrate_irqs(void)
if (desc->cpu == cpu) {
unsigned int newcpu = any_online_cpu(desc->affinity);
- if (newcpu == NR_CPUS) {
+ if (newcpu >= nr_cpu_ids) {
if (printk_ratelimit())
printk(KERN_INFO "IRQ%u no longer affine to CPU%u\n",
i, cpu);
--- test-compile.orig/arch/cris/kernel/setup.c
+++ test-compile/arch/cris/kernel/setup.c
@@ -166,7 +166,7 @@ void __init setup_arch(char **cmdline_p)
static void *c_start(struct seq_file *m, loff_t *pos)
{
- return *pos < NR_CPUS ? (void *)(int)(*pos + 1): NULL;
+ return *pos < nr_cpu_ids ? (void *)(int)(*pos + 1) : NULL;
}
static void *c_next(struct seq_file *m, void *v, loff_t *pos)
--- test-compile.orig/arch/frv/kernel/setup.c
+++ test-compile/arch/frv/kernel/setup.c
@@ -1100,7 +1100,7 @@ static int show_cpuinfo(struct seq_file
static void *c_start(struct seq_file *m, loff_t *pos)
{
- return *pos < NR_CPUS ? (void *) 0x12345678 : NULL;
+ return *pos < nr_cpu_ids ? (void *) 0x12345678 : NULL;
}
static void *c_next(struct seq_file *m, void *v, loff_t *pos)
--- test-compile.orig/arch/h8300/kernel/setup.c
+++ test-compile/arch/h8300/kernel/setup.c
@@ -224,7 +224,7 @@ static int show_cpuinfo(struct seq_file
static void *c_start(struct seq_file *m, loff_t *pos)
{
- return *pos < NR_CPUS ? ((void *) 0x12345678) : NULL;
+ return *pos < nr_cpu_ids ? ((void *) 0x12345678) : NULL;
}
static void *c_next(struct seq_file *m, void *v, loff_t *pos)
--- test-compile.orig/arch/ia64/kernel/acpi.c
+++ test-compile/arch/ia64/kernel/acpi.c
@@ -885,7 +885,7 @@ int acpi_map_lsapic(acpi_handle handle,
cpus_complement(tmp_map, cpu_present_map);
cpu = first_cpu(tmp_map);
- if (cpu >= NR_CPUS)
+ if (cpu >= nr_cpu_ids)
return -EINVAL;
acpi_map_cpu2node(handle, cpu, physid);
--- test-compile.orig/arch/ia64/kernel/iosapic.c
+++ test-compile/arch/ia64/kernel/iosapic.c
@@ -720,7 +720,7 @@ get_target_cpu (unsigned int gsi, int ir
for (numa_cpu = first_cpu(cpu_mask) ; i < cpu_index ; i++)
numa_cpu = next_cpu(numa_cpu, cpu_mask);
- if (numa_cpu != NR_CPUS)
+ if (numa_cpu < nr_cpus_ids)
return cpu_physical_id(numa_cpu);
}
skip_numa_setup:
@@ -731,7 +731,7 @@ skip_numa_setup:
* case of NUMA.)
*/
do {
- if (++cpu >= NR_CPUS)
+ if (++cpu >= nr_cpu_ids)
cpu = 0;
} while (!cpu_online(cpu) || !cpu_isset(cpu, domain));
--- test-compile.orig/arch/ia64/kernel/irq.c
+++ test-compile/arch/ia64/kernel/irq.c
@@ -153,7 +153,7 @@ static void migrate_irqs(void)
continue;
cpus_and(mask, irq_desc[irq].affinity, cpu_online_map);
- if (any_online_cpu(mask) == NR_CPUS) {
+ if (any_online_cpu(mask) >= nr_cpu_ids) {
/*
* Save it for phase 2 processing
*/
--- test-compile.orig/arch/ia64/kernel/mca.c
+++ test-compile/arch/ia64/kernel/mca.c
@@ -1456,9 +1456,9 @@ ia64_mca_cmc_int_caller(int cmc_irq, voi
ia64_mca_cmc_int_handler(cmc_irq, arg);
- for (++cpuid ; cpuid < NR_CPUS && !cpu_online(cpuid) ; cpuid++);
+ cpuid = next_cpu(cpuid+1, cpu_online_map);
- if (cpuid < NR_CPUS) {
+ if (cpuid < nr_cpu_ids) {
platform_send_ipi(cpuid, IA64_CMCP_VECTOR, IA64_IPI_DM_INT, 0);
} else {
/* If no log record, switch out of polling mode */
@@ -1525,7 +1525,7 @@ ia64_mca_cpe_int_caller(int cpe_irq, voi
ia64_mca_cpe_int_handler(cpe_irq, arg);
- for (++cpuid ; cpuid < NR_CPUS && !cpu_online(cpuid) ; cpuid++);
+ cpuid = next_cpu(cpuid+1, cpu_online_map);
if (cpuid < NR_CPUS) {
platform_send_ipi(cpuid, IA64_CPEP_VECTOR, IA64_IPI_DM_INT, 0);
--- test-compile.orig/arch/ia64/kernel/perfmon.c
+++ test-compile/arch/ia64/kernel/perfmon.c
@@ -5598,7 +5598,7 @@ pfm_interrupt_handler(int irq, void *arg
* /proc/perfmon interface, for debug only
*/
-#define PFM_PROC_SHOW_HEADER ((void *)NR_CPUS+1)
+#define PFM_PROC_SHOW_HEADER ((void *)nr_cpu_ids+1)
static void *
pfm_proc_start(struct seq_file *m, loff_t *pos)
@@ -5607,7 +5607,7 @@ pfm_proc_start(struct seq_file *m, loff_
return PFM_PROC_SHOW_HEADER;
}
- while (*pos <= NR_CPUS) {
+ while (*pos <= nr_cpu_ids) {
if (cpu_online(*pos - 1)) {
return (void *)*pos;
}
--- test-compile.orig/arch/ia64/kernel/salinfo.c
+++ test-compile/arch/ia64/kernel/salinfo.c
@@ -317,7 +317,7 @@ retry:
}
n = data->cpu_check;
- for (i = 0; i < NR_CPUS; i++) {
+ for (i = 0; i < nr_cpu_ids; i++) {
if (cpu_isset(n, data->cpu_event)) {
if (!cpu_online(n)) {
cpu_clear(n, data->cpu_event);
@@ -326,7 +326,7 @@ retry:
cpu = n;
break;
}
- if (++n == NR_CPUS)
+ if (++n == nr_cpu_ids)
n = 0;
}
@@ -337,7 +337,7 @@ retry:
/* for next read, start checking at next CPU */
data->cpu_check = cpu;
- if (++data->cpu_check == NR_CPUS)
+ if (++data->cpu_check == nr_cpu_ids)
data->cpu_check = 0;
snprintf(cmd, sizeof(cmd), "read %d\n", cpu);
--- test-compile.orig/arch/ia64/kernel/setup.c
+++ test-compile/arch/ia64/kernel/setup.c
@@ -714,10 +714,10 @@ static void *
c_start (struct seq_file *m, loff_t *pos)
{
#ifdef CONFIG_SMP
- while (*pos < NR_CPUS && !cpu_isset(*pos, cpu_online_map))
+ while (*pos < nr_cpu_ids && !cpu_online(*pos))
++*pos;
#endif
- return *pos < NR_CPUS ? cpu_data(*pos) : NULL;
+ return *pos < nr_cpu_ids ? cpu_data(*pos) : NULL;
}
static void *
--- test-compile.orig/arch/ia64/sn/kernel/setup.c
+++ test-compile/arch/ia64/sn/kernel/setup.c
@@ -753,7 +753,7 @@ nasid_slice_to_cpuid(int nasid, int slic
{
long cpu;
- for (cpu = 0; cpu < NR_CPUS; cpu++)
+ for (cpu = 0; cpu < nr_cpu_ids; cpu++)
if (cpuid_to_nasid(cpu) == nasid &&
cpuid_to_slice(cpu) == slice)
return cpu;
--- test-compile.orig/arch/ia64/sn/kernel/sn2/sn2_smp.c
+++ test-compile/arch/ia64/sn/kernel/sn2/sn2_smp.c
@@ -461,7 +461,7 @@ bool sn_cpu_disable_allowed(int cpu)
static void *sn2_ptc_seq_start(struct seq_file *file, loff_t * offset)
{
- if (*offset < NR_CPUS)
+ if (*offset < nr_cpu_ids)
return offset;
return NULL;
}
@@ -469,7 +469,7 @@ static void *sn2_ptc_seq_start(struct se
static void *sn2_ptc_seq_next(struct seq_file *file, void *data, loff_t * offset)
{
(*offset)++;
- if (*offset < NR_CPUS)
+ if (*offset < nr_cpu_ids)
return offset;
return NULL;
}
@@ -491,7 +491,7 @@ static int sn2_ptc_seq_show(struct seq_f
seq_printf(file, "# ptctest %d, flushopt %d\n", sn2_ptctest, sn2_flush_opt);
}
- if (cpu < NR_CPUS && cpu_online(cpu)) {
+ if (cpu < nr_cpu_ids && cpu_online(cpu)) {
stat = &per_cpu(ptcstats, cpu);
seq_printf(file, "cpu %d %ld %ld %ld %ld %ld %ld %ld %ld %ld %ld %ld %ld\n", cpu, stat->ptc_l,
stat->change_rid, stat->shub_ptc_flushes, stat->nodes_flushed,
--- test-compile.orig/arch/ia64/sn/kernel/sn2/sn_hwperf.c
+++ test-compile/arch/ia64/sn/kernel/sn2/sn_hwperf.c
@@ -615,7 +615,7 @@ static int sn_hwperf_op_cpu(struct sn_hw
op_info->a->arg &= SN_HWPERF_ARG_OBJID_MASK;
if (cpu != SN_HWPERF_ARG_ANY_CPU) {
- if (cpu >= NR_CPUS || !cpu_online(cpu)) {
+ if (cpu >= nr_cpu_ids || !cpu_online(cpu)) {
r = -EINVAL;
goto out;
}
--- test-compile.orig/arch/m32r/kernel/setup.c
+++ test-compile/arch/m32r/kernel/setup.c
@@ -356,7 +356,7 @@ static int show_cpuinfo(struct seq_file
static void *c_start(struct seq_file *m, loff_t *pos)
{
- return *pos < NR_CPUS ? cpu_data + *pos : NULL;
+ return *pos < nr_cpu_ids ? cpu_data + *pos : NULL;
}
static void *c_next(struct seq_file *m, void *v, loff_t *pos)
--- test-compile.orig/arch/m68knommu/kernel/setup.c
+++ test-compile/arch/m68knommu/kernel/setup.c
@@ -248,7 +248,7 @@ static int show_cpuinfo(struct seq_file
static void *c_start(struct seq_file *m, loff_t *pos)
{
- return *pos < NR_CPUS ? ((void *) 0x12345678) : NULL;
+ return *pos < nr_cpu_ids ? ((void *) 0x12345678) : NULL;
}
static void *c_next(struct seq_file *m, void *v, loff_t *pos)
--- test-compile.orig/arch/mips/kernel/irq-gic.c
+++ test-compile/arch/mips/kernel/irq-gic.c
@@ -182,7 +182,7 @@ static void gic_set_affinity(unsigned in
_intrmap[irq].cpunum = first_cpu(tmp);
/* Update the pcpu_masks */
- for (i = 0; i < NR_CPUS; i++)
+ for (i = 0; i < nr_cpu_ids; i++)
clear_bit(irq, pcpu_masks[i].pcpu_mask);
set_bit(irq, pcpu_masks[first_cpu(tmp)].pcpu_mask);
--- test-compile.orig/arch/mips/kernel/proc.c
+++ test-compile/arch/mips/kernel/proc.c
@@ -86,7 +86,7 @@ static void *c_start(struct seq_file *m,
{
unsigned long i = *pos;
- return i < NR_CPUS ? (void *) (i + 1) : NULL;
+ return i < nr_cpu_ids ? (void *) (i + 1) : NULL;
}
static void *c_next(struct seq_file *m, void *v, loff_t *pos)
--- test-compile.orig/arch/mips/kernel/smp-cmp.c
+++ test-compile/arch/mips/kernel/smp-cmp.c
@@ -224,7 +224,7 @@ void __init cmp_smp_setup(void)
cpu_set(0, mt_fpu_cpumask);
#endif /* CONFIG_MIPS_MT_FPAFF */
- for (i = 1; i < NR_CPUS; i++) {
+ for (i = 1; i < nr_cpu_ids; i++) {
if (amon_cpu_avail(i)) {
cpu_set(i, phys_cpu_present_map);
__cpu_number_map[i] = ++ncpu;
--- test-compile.orig/arch/mips/kernel/smtc.c
+++ test-compile/arch/mips/kernel/smtc.c
@@ -303,7 +303,7 @@ int __init smtc_build_cpu_map(int start_
* everything up so that "logical" = "physical".
*/
ntcs = ((read_c0_mvpconf0() & MVPCONF0_PTC) >> MVPCONF0_PTC_SHIFT) + 1;
- for (i=start_cpu_slot; i<NR_CPUS && i<ntcs; i++) {
+ for (i = start_cpu_slot; i < nr_cpu_ids && i < ntcs; i++) {
cpu_set(i, phys_cpu_present_map);
__cpu_number_map[i] = i;
__cpu_logical_map[i] = i;
@@ -422,8 +422,8 @@ void smtc_prepare_cpus(int cpus)
if (vpelimit > 0 && nvpe > vpelimit)
nvpe = vpelimit;
ntc = ((val & MVPCONF0_PTC) >> MVPCONF0_PTC_SHIFT) + 1;
- if (ntc > NR_CPUS)
- ntc = NR_CPUS;
+ if (ntc > nr_cpu_ids)
+ ntc = nr_cpu_ids;
if (tclimit > 0 && ntc > tclimit)
ntc = tclimit;
slop = ntc % nvpe;
--- test-compile.orig/arch/mips/sibyte/bcm1480/irq.c
+++ test-compile/arch/mips/sibyte/bcm1480/irq.c
@@ -195,7 +195,7 @@ static void ack_bcm1480_irq(unsigned int
if (pending) {
#ifdef CONFIG_SMP
int i;
- for (i=0; i<NR_CPUS; i++) {
+ for (i=0; i < nr_cpu_ids; i++) {
/*
* Clear for all CPUs so an affinity switch
* doesn't find an old status
--- test-compile.orig/arch/mips/sibyte/bcm1480/smp.c
+++ test-compile/arch/mips/sibyte/bcm1480/smp.c
@@ -150,7 +150,7 @@ static void __init bcm1480_smp_setup(voi
__cpu_number_map[0] = 0;
__cpu_logical_map[0] = 0;
- for (i = 1, num = 0; i < NR_CPUS; i++) {
+ for (i = 1, num = 0; i < nr_cpu_ids; i++) {
if (cfe_cpu_stop(i) == 0) {
cpu_set(i, phys_cpu_present_map);
__cpu_number_map[i] = ++num;
--- test-compile.orig/arch/mips/sibyte/sb1250/smp.c
+++ test-compile/arch/mips/sibyte/sb1250/smp.c
@@ -138,7 +138,7 @@ static void __init sb1250_smp_setup(void
__cpu_number_map[0] = 0;
__cpu_logical_map[0] = 0;
- for (i = 1, num = 0; i < NR_CPUS; i++) {
+ for (i = 1, num = 0; i < nr_cpu_ids; i++) {
if (cfe_cpu_stop(i) == 0) {
cpu_set(i, phys_cpu_present_map);
__cpu_number_map[i] = ++num;
--- test-compile.orig/arch/mn10300/kernel/irq.c
+++ test-compile/arch/mn10300/kernel/irq.c
@@ -207,7 +207,7 @@ int show_interrupts(struct seq_file *p,
/* display column title bar naming CPUs */
case 0:
seq_printf(p, " ");
- for (j = 0; j < NR_CPUS; j++)
+ for (j = 0; j < nr_cpu_ids; j++)
if (cpu_online(j))
seq_printf(p, "CPU%d ", j);
seq_putc(p, '\n');
@@ -241,7 +241,7 @@ int show_interrupts(struct seq_file *p,
/* polish off with NMI and error counters */
case NR_IRQS:
seq_printf(p, "NMI: ");
- for (j = 0; j < NR_CPUS; j++)
+ for (j = 0; j < nr_cpu_ids; j++)
if (cpu_online(j))
seq_printf(p, "%10u ", nmi_count(j));
seq_putc(p, '\n');
--- test-compile.orig/arch/mn10300/kernel/setup.c
+++ test-compile/arch/mn10300/kernel/setup.c
@@ -276,7 +276,7 @@ static int show_cpuinfo(struct seq_file
static void *c_start(struct seq_file *m, loff_t *pos)
{
- return *pos < NR_CPUS ? cpu_data + *pos : NULL;
+ return *pos < nr_cpu_ids ? cpu_data + *pos : NULL;
}
static void *c_next(struct seq_file *m, void *v, loff_t *pos)
--- test-compile.orig/arch/parisc/kernel/irq.c
+++ test-compile/arch/parisc/kernel/irq.c
@@ -309,11 +309,11 @@ unsigned long txn_alloc_addr(unsigned in
next_cpu++; /* assign to "next" CPU we want this bugger on */
/* validate entry */
- while ((next_cpu < NR_CPUS) && (!cpu_data[next_cpu].txn_addr ||
+ while ((next_cpu < nr_cpu_ids) && (!cpu_data[next_cpu].txn_addr ||
!cpu_online(next_cpu)))
next_cpu++;
- if (next_cpu >= NR_CPUS)
+ if (next_cpu >= nr_cpu_ids)
next_cpu = 0; /* nothing else, assign monarch */
return txn_affinity_addr(virt_irq, next_cpu);
--- test-compile.orig/arch/parisc/kernel/processor.c
+++ test-compile/arch/parisc/kernel/processor.c
@@ -83,8 +83,8 @@ static int __cpuinit processor_probe(str
struct cpuinfo_parisc *p;
#ifdef CONFIG_SMP
- if (num_online_cpus() >= NR_CPUS) {
- printk(KERN_INFO "num_online_cpus() >= NR_CPUS\n");
+ if (num_online_cpus() >= nr_cpu_ids) {
+ printk(KERN_INFO "num_online_cpus() >= nr_cpu_ids\n");
return 1;
}
#else
--- test-compile.orig/arch/powerpc/include/asm/cputhreads.h
+++ test-compile/arch/powerpc/include/asm/cputhreads.h
@@ -34,7 +34,7 @@ static inline cpumask_t cpu_thread_mask_
int i;
res = CPU_MASK_NONE;
- for (i = 0; i < NR_CPUS; i += threads_per_core) {
+ for (i = 0; i < nr_cpu_ids; i += threads_per_core) {
cpus_shift_left(tmp, threads_core_mask, i);
if (cpus_intersects(threads, tmp))
cpu_set(i, res);
--- test-compile.orig/arch/powerpc/kernel/irq.c
+++ test-compile/arch/powerpc/kernel/irq.c
@@ -232,7 +232,7 @@ void fixup_irqs(cpumask_t map)
continue;
cpus_and(mask, irq_desc[irq].affinity, map);
- if (any_online_cpu(mask) == NR_CPUS) {
+ if (any_online_cpu(mask) >= nr_cpu_ids) {
printk("Breaking affinity for irq %i\n", irq);
mask = map;
}
--- test-compile.orig/arch/powerpc/kernel/machine_kexec_64.c
+++ test-compile/arch/powerpc/kernel/machine_kexec_64.c
@@ -176,7 +176,7 @@ static void kexec_prepare_cpus(void)
my_cpu = get_cpu();
/* check the others cpus are now down (via paca hw cpu id == -1) */
- for (i=0; i < NR_CPUS; i++) {
+ for (i = 0; i < nr_cpu_ids; i++) {
if (i == my_cpu)
continue;
--- test-compile.orig/arch/powerpc/kernel/process.c
+++ test-compile/arch/powerpc/kernel/process.c
@@ -941,7 +941,7 @@ static inline int valid_irq_stack(unsign
* Avoid crashing if the stack has overflowed and corrupted
* task_cpu(p), which is in the thread_info struct.
*/
- if (cpu < NR_CPUS && cpu_possible(cpu)) {
+ if (cpu < nr_cpu_ids && cpu_possible(cpu)) {
stack_page = (unsigned long) hardirq_ctx[cpu];
if (sp >= stack_page + sizeof(struct thread_struct)
&& sp <= stack_page + THREAD_SIZE - nbytes)
--- test-compile.orig/arch/powerpc/kernel/setup-common.c
+++ test-compile/arch/powerpc/kernel/setup-common.c
@@ -166,7 +166,7 @@ static int show_cpuinfo(struct seq_file
unsigned short maj;
unsigned short min;
- if (cpu_id == NR_CPUS) {
+ if (cpu_id == nr_cpu_ids) {
struct device_node *root;
const char *model = NULL;
#if defined(CONFIG_SMP) && defined(CONFIG_PPC32)
@@ -196,7 +196,7 @@ static int show_cpuinfo(struct seq_file
/* We only show online cpus: disable preempt (overzealous, I
* knew) to prevent cpu going down. */
preempt_disable();
- if (!cpu_online(cpu_id)) {
+ if (cpu_id >= nr_cpu_ids || !cpu_online(cpu_id)) {
preempt_enable();
return 0;
}
@@ -307,7 +307,7 @@ static void *c_start(struct seq_file *m,
{
unsigned long i = *pos;
- return i <= NR_CPUS ? (void *)(i + 1) : NULL;
+ return i <= nr_cpu_ids ? (void *)(i + 1) : NULL;
}
static void *c_next(struct seq_file *m, void *v, loff_t *pos)
@@ -402,7 +402,7 @@ void __init smp_setup_cpu_maps(void)
DBG("smp_setup_cpu_maps()\n");
- while ((dn = of_find_node_by_type(dn, "cpu")) && cpu < NR_CPUS) {
+ while ((dn = of_find_node_by_type(dn, "cpu")) && cpu < nr_cpu_ids) {
const int *intserv;
int j, len;
@@ -421,7 +421,7 @@ void __init smp_setup_cpu_maps(void)
intserv = &cpu; /* assume logical == phys */
}
- for (j = 0; j < nthreads && cpu < NR_CPUS; j++) {
+ for (j = 0; j < nthreads && cpu < nr_cpu_ids; j++) {
DBG(" thread %d -> cpu %d (hard id %d)\n",
j, cpu, intserv[j]);
cpu_set(cpu, cpu_present_map);
--- test-compile.orig/arch/powerpc/mm/numa.c
+++ test-compile/arch/powerpc/mm/numa.c
@@ -765,7 +765,7 @@ void __init dump_numa_cpu_topology(void)
* If we used a CPU iterator here we would miss printing
* the holes in the cpumap.
*/
- for (cpu = 0; cpu < NR_CPUS; cpu++) {
+ for (cpu = 0; cpu < nr_cpu_ids; cpu++) {
if (cpu_isset(cpu, numa_cpumask_lookup_table[node])) {
if (count == 0)
printk(" %u", cpu);
@@ -778,7 +778,7 @@ void __init dump_numa_cpu_topology(void)
}
if (count > 1)
- printk("-%u", NR_CPUS - 1);
+ printk("-%u", nr_cpu_ids - 1);
printk("\n");
}
}
--- test-compile.orig/arch/powerpc/platforms/powermac/setup.c
+++ test-compile/arch/powerpc/platforms/powermac/setup.c
@@ -365,7 +365,7 @@ static void __init pmac_setup_arch(void)
*/
int cpu;
- for (cpu = 1; cpu < 4 && cpu < NR_CPUS; ++cpu)
+ for (cpu = 1; cpu < 4 && cpu < nr_cpu_ids; ++cpu)
cpu_set(cpu, cpu_possible_map);
smp_ops = &psurge_smp_ops;
}
--- test-compile.orig/arch/powerpc/platforms/powermac/smp.c
+++ test-compile/arch/powerpc/platforms/powermac/smp.c
@@ -314,8 +314,8 @@ static int __init smp_psurge_probe(void)
* device tree for them, and smp_setup_cpu_maps hasn't
* set their bits in cpu_possible_map and cpu_present_map.
*/
- if (ncpus > NR_CPUS)
- ncpus = NR_CPUS;
+ if (ncpus > nr_cpu_ids)
+ ncpus = nr_cpu_ids;
for (i = 1; i < ncpus ; ++i) {
cpu_set(i, cpu_present_map);
set_hard_smp_processor_id(i, i);
--- test-compile.orig/arch/powerpc/platforms/pseries/hotplug-cpu.c
+++ test-compile/arch/powerpc/platforms/pseries/hotplug-cpu.c
@@ -221,7 +221,7 @@ static void pseries_remove_processor(str
set_hard_smp_processor_id(cpu, -1);
break;
}
- if (cpu == NR_CPUS)
+ if (cpu >= nr_cpu_ids)
printk(KERN_WARNING "Could not find cpu to remove "
"with physical id 0x%x\n", intserv[i]);
}
--- test-compile.orig/arch/powerpc/platforms/pseries/rtasd.c
+++ test-compile/arch/powerpc/platforms/pseries/rtasd.c
@@ -400,7 +400,7 @@ static void do_event_scan_all_cpus(long
get_online_cpus();
cpu = next_cpu(cpu, cpu_online_map);
- if (cpu == NR_CPUS)
+ if (cpu >= nr_cpu_ids)
break;
}
put_online_cpus();
--- test-compile.orig/arch/powerpc/platforms/pseries/xics.c
+++ test-compile/arch/powerpc/platforms/pseries/xics.c
@@ -164,7 +164,7 @@ static int get_irq_server(unsigned int v
server = first_cpu(tmp);
- if (server < NR_CPUS)
+ if (server < nr_cpu_ids)
return get_hard_smp_processor_id(server);
if (strict_check)
--- test-compile.orig/arch/powerpc/xmon/xmon.c
+++ test-compile/arch/powerpc/xmon/xmon.c
@@ -938,7 +938,7 @@ static int cpu_cmd(void)
/* print cpus waiting or in xmon */
printf("cpus stopped:");
count = 0;
- for (cpu = 0; cpu < NR_CPUS; ++cpu) {
+ for (cpu = 0; cpu < nr_cpu_ids; ++cpu) {
if (cpu_isset(cpu, cpus_in_xmon)) {
if (count == 0)
printf(" %x", cpu);
@@ -950,7 +950,7 @@ static int cpu_cmd(void)
}
}
if (count > 1)
- printf("-%x", NR_CPUS - 1);
+ printf("-%x", nr_cpu_ids - 1);
printf("\n");
return 0;
}
--- test-compile.orig/arch/s390/kernel/smp.c
+++ test-compile/arch/s390/kernel/smp.c
@@ -444,7 +444,7 @@ static int smp_rescan_cpus_sigp(cpumask_
int cpu_id, logical_cpu;
logical_cpu = first_cpu(avail);
- if (logical_cpu == NR_CPUS)
+ if (logical_cpu >= nr_cpu_ids)
return 0;
for (cpu_id = 0; cpu_id <= 65535; cpu_id++) {
if (cpu_known(cpu_id))
@@ -456,7 +456,7 @@ static int smp_rescan_cpus_sigp(cpumask_
cpu_set(logical_cpu, cpu_present_map);
smp_cpu_state[logical_cpu] = CPU_STATE_CONFIGURED;
logical_cpu = next_cpu(logical_cpu, avail);
- if (logical_cpu == NR_CPUS)
+ if (logical_cpu >= nr_cpu_ids)
break;
}
return 0;
@@ -469,7 +469,7 @@ static int smp_rescan_cpus_sclp(cpumask_
int rc;
logical_cpu = first_cpu(avail);
- if (logical_cpu == NR_CPUS)
+ if (logical_cpu >= nr_cpu_ids)
return 0;
info = kmalloc(sizeof(*info), GFP_KERNEL);
if (!info)
@@ -491,7 +491,7 @@ static int smp_rescan_cpus_sclp(cpumask_
else
smp_cpu_state[logical_cpu] = CPU_STATE_CONFIGURED;
logical_cpu = next_cpu(logical_cpu, avail);
- if (logical_cpu == NR_CPUS)
+ if (logical_cpu >= nr_cpu_ids)
break;
}
out:
@@ -733,7 +733,7 @@ static int __init setup_possible_cpus(ch
pcpus = simple_strtoul(s, NULL, 0);
cpu_possible_map = cpumask_of_cpu(0);
- for (cpu = 1; cpu < pcpus && cpu < NR_CPUS; cpu++)
+ for (cpu = 1; cpu < pcpus && cpu < nr_cpu_ids; cpu++)
cpu_set(cpu, cpu_possible_map);
return 0;
}
--- test-compile.orig/arch/sh/kernel/setup.c
+++ test-compile/arch/sh/kernel/setup.c
@@ -507,7 +507,7 @@ static int show_cpuinfo(struct seq_file
static void *c_start(struct seq_file *m, loff_t *pos)
{
- return *pos < NR_CPUS ? cpu_data + *pos : NULL;
+ return *pos < nr_cpu_ids ? cpu_data + *pos : NULL;
}
static void *c_next(struct seq_file *m, void *v, loff_t *pos)
{
--- test-compile.orig/arch/sparc/kernel/smp.c
+++ test-compile/arch/sparc/kernel/smp.c
@@ -72,13 +72,12 @@ void __init smp_cpus_done(unsigned int m
extern void smp4m_smp_done(void);
extern void smp4d_smp_done(void);
unsigned long bogosum = 0;
- int cpu, num;
+ int cpu, num = 0;
- for (cpu = 0, num = 0; cpu < NR_CPUS; cpu++)
- if (cpu_online(cpu)) {
- num++;
- bogosum += cpu_data(cpu).udelay_val;
- }
+ for_each_online_cpu(cpu) {
+ num++;
+ bogosum += cpu_data(cpu).udelay_val;
+ }
printk("Total of %d processors activated (%lu.%02lu BogoMIPS).\n",
num, bogosum/(500000/HZ),
--- test-compile.orig/arch/sparc/kernel/sun4d_smp.c
+++ test-compile/arch/sparc/kernel/sun4d_smp.c
@@ -228,11 +228,10 @@ void __init smp4d_smp_done(void)
/* setup cpu list for irq rotation */
first = 0;
prev = &first;
- for (i = 0; i < NR_CPUS; i++)
- if (cpu_online(i)) {
- *prev = i;
- prev = &cpu_data(i).next;
- }
+ for_each_online_cpu(i) {
+ *prev = i;
+ prev = &cpu_data(i).next;
+ }
*prev = first;
local_flush_cache_all();
--- test-compile.orig/arch/sparc/kernel/sun4m_smp.c
+++ test-compile/arch/sparc/kernel/sun4m_smp.c
@@ -185,11 +185,9 @@ void __init smp4m_smp_done(void)
/* setup cpu list for irq rotation */
first = 0;
prev = &first;
- for (i = 0; i < NR_CPUS; i++) {
- if (cpu_online(i)) {
- *prev = i;
- prev = &cpu_data(i).next;
- }
+ for_each_online_cpu(i) {
+ *prev = i;
+ prev = &cpu_data(i).next;
}
*prev = first;
local_flush_cache_all();
--- test-compile.orig/arch/sparc/mm/srmmu.c
+++ test-compile/arch/sparc/mm/srmmu.c
@@ -1427,7 +1427,7 @@ static void __init init_vac_layout(void)
min_line_size = vac_line_size;
//FIXME: cpus not contiguous!!
cpu++;
- if (cpu >= NR_CPUS || !cpu_online(cpu))
+ if (cpu >= nr_cpu_ids || !cpu_online(cpu))
break;
#else
break;
--- test-compile.orig/arch/sparc64/kernel/ds.c
+++ test-compile/arch/sparc64/kernel/ds.c
@@ -653,7 +653,7 @@ static void __cpuinit dr_cpu_data(struct
if (cpu_list[i] == CPU_SENTINEL)
continue;
- if (cpu_list[i] < NR_CPUS)
+ if (cpu_list[i] < nr_cpu_ids)
cpu_set(cpu_list[i], mask);
}
--- test-compile.orig/arch/sparc64/kernel/irq.c
+++ test-compile/arch/sparc64/kernel/irq.c
@@ -260,12 +260,12 @@ static int irq_choose_cpu(unsigned int v
spin_lock_irqsave(&irq_rover_lock, flags);
while (!cpu_online(irq_rover)) {
- if (++irq_rover >= NR_CPUS)
+ if (++irq_rover >= nr_cpu_ids)
irq_rover = 0;
}
cpuid = irq_rover;
do {
- if (++irq_rover >= NR_CPUS)
+ if (++irq_rover >= nr_cpu_ids)
irq_rover = 0;
} while (!cpu_online(irq_rover));
--- test-compile.orig/arch/sparc64/mm/init.c
+++ test-compile/arch/sparc64/mm/init.c
@@ -1080,7 +1080,7 @@ static void __init numa_parse_mdesc_grou
if (strcmp(name, "cpu"))
continue;
id = mdesc_get_property(md, target, "id", NULL);
- if (*id < NR_CPUS)
+ if (*id < nr_cpu_ids)
cpu_set(*id, *mask);
}
}
--- test-compile.orig/arch/um/kernel/um_arch.c
+++ test-compile/arch/um/kernel/um_arch.c
@@ -80,7 +80,7 @@ static int show_cpuinfo(struct seq_file
static void *c_start(struct seq_file *m, loff_t *pos)
{
- return *pos < NR_CPUS ? cpu_data + *pos : NULL;
+ return *pos < nr_cpu_ids ? cpu_data + *pos : NULL;
}
static void *c_next(struct seq_file *m, void *v, loff_t *pos)
--- test-compile.orig/arch/x86/kernel/apic.c
+++ test-compile/arch/x86/kernel/apic.c
@@ -2106,7 +2106,7 @@ __cpuinit int apic_is_clustered_box(void
bios_cpu_apicid = early_per_cpu_ptr(x86_bios_cpu_apicid);
bitmap_zero(clustermap, NUM_APIC_CLUSTERS);
- for (i = 0; i < NR_CPUS; i++) {
+ for (i = 0; i < nr_cpu_ids; i++) {
/* are we being called early in kernel startup? */
if (bios_cpu_apicid) {
id = bios_cpu_apicid[i];
--- test-compile.orig/arch/x86/kernel/irq_32.c
+++ test-compile/arch/x86/kernel/irq_32.c
@@ -246,7 +246,7 @@ void fixup_irqs(cpumask_t map)
continue;
cpus_and(mask, desc->affinity, map);
- if (any_online_cpu(mask) == NR_CPUS) {
+ if (any_online_cpu(mask) >= nr_cpu_ids) {
printk("Breaking affinity for irq %i\n", irq);
mask = map;
}
--- test-compile.orig/arch/x86/mach-voyager/voyager_smp.c
+++ test-compile/arch/x86/mach-voyager/voyager_smp.c
@@ -668,7 +668,7 @@ void __init smp_boot_cpus(void)
/* loop over all the extended VIC CPUs and boot them. The
* Quad CPUs must be bootstrapped by their extended VIC cpu */
- for (i = 0; i < NR_CPUS; i++) {
+ for (i = 0; i < nr_cpu_ids; i++) {
if (i == boot_cpu_id || !cpu_isset(i, phys_cpu_present_map))
continue;
do_boot_cpu(i);
--- test-compile.orig/arch/x86/mm/numa_64.c
+++ test-compile/arch/x86/mm/numa_64.c
@@ -278,7 +278,7 @@ void __init numa_init_array(void)
int rr, i;
rr = first_node(node_online_map);
- for (i = 0; i < NR_CPUS; i++) {
+ for (i = 0; i < nr_cpu_ids; i++) {
if (early_cpu_to_node(i) != NUMA_NO_NODE)
continue;
numa_set_node(i, rr);
@@ -549,7 +549,7 @@ void __init initmem_init(unsigned long s
memnodemap[0] = 0;
node_set_online(0);
node_set(0, node_possible_map);
- for (i = 0; i < NR_CPUS; i++)
+ for (i = 0; i < nr_cpu_ids; i++)
numa_set_node(i, 0);
e820_register_active_regions(0, start_pfn, last_pfn);
setup_node_bootmem(0, start_pfn << PAGE_SHIFT, last_pfn << PAGE_SHIFT);
--- test-compile.orig/arch/x86/mm/srat_64.c
+++ test-compile/arch/x86/mm/srat_64.c
@@ -382,7 +382,7 @@ int __init acpi_scan_nodes(unsigned long
if (!node_online(i))
setup_node_bootmem(i, nodes[i].start, nodes[i].end);
- for (i = 0; i < NR_CPUS; i++) {
+ for (i = 0; i < nr_cpu_ids; i++) {
int node = early_cpu_to_node(i);
if (node == NUMA_NO_NODE)
--- test-compile.orig/drivers/infiniband/hw/ehca/ehca_irq.c
+++ test-compile/drivers/infiniband/hw/ehca/ehca_irq.c
@@ -922,7 +922,7 @@ void ehca_destroy_comp_pool(void)
unregister_hotcpu_notifier(&comp_pool_callback_nb);
- for (i = 0; i < NR_CPUS; i++) {
+ for (i = 0; i < nr_cpu_ids; i++) {
if (cpu_online(i))
destroy_comp_task(pool, i);
}
--- test-compile.orig/kernel/kexec.c
+++ test-compile/kernel/kexec.c
@@ -1115,7 +1115,7 @@ void crash_save_cpu(struct pt_regs *regs
struct elf_prstatus prstatus;
u32 *buf;
- if ((cpu < 0) || (cpu >= NR_CPUS))
+ if ((cpu < 0) || (cpu >= nr_cpu_ids))
return;
/* Using ELF notes here is opportunistic.
--- test-compile.orig/kernel/smp.c
+++ test-compile/kernel/smp.c
@@ -222,7 +222,7 @@ int smp_call_function_single(int cpu, vo
local_irq_save(flags);
func(info);
local_irq_restore(flags);
- } else if ((unsigned)cpu < NR_CPUS && cpu_online(cpu)) {
+ } else if ((unsigned)cpu < nr_cpu_ids && cpu_online(cpu)) {
struct call_single_data *data = NULL;
if (!wait) {
--- test-compile.orig/net/core/neighbour.c
+++ test-compile/net/core/neighbour.c
@@ -2423,7 +2423,7 @@ static void *neigh_stat_seq_start(struct
if (*pos == 0)
return SEQ_START_TOKEN;
- for (cpu = *pos-1; cpu < NR_CPUS; ++cpu) {
+ for (cpu = *pos-1; cpu < nr_cpu_ids; ++cpu) {
if (!cpu_possible(cpu))
continue;
*pos = cpu+1;
@@ -2438,7 +2438,7 @@ static void *neigh_stat_seq_next(struct
struct neigh_table *tbl = pde->data;
int cpu;
- for (cpu = *pos; cpu < NR_CPUS; ++cpu) {
+ for (cpu = *pos; cpu < nr_cpu_ids; ++cpu) {
if (!cpu_possible(cpu))
continue;
*pos = cpu+1;
--- test-compile.orig/net/ipv4/netfilter/nf_conntrack_l3proto_ipv4_compat.c
+++ test-compile/net/ipv4/netfilter/nf_conntrack_l3proto_ipv4_compat.c
@@ -291,7 +291,7 @@ static void *ct_cpu_seq_start(struct seq
if (*pos == 0)
return SEQ_START_TOKEN;
- for (cpu = *pos-1; cpu < NR_CPUS; ++cpu) {
+ for (cpu = *pos-1; cpu < nr_cpu_ids; ++cpu) {
if (!cpu_possible(cpu))
continue;
*pos = cpu+1;
@@ -306,7 +306,7 @@ static void *ct_cpu_seq_next(struct seq_
struct net *net = seq_file_net(seq);
int cpu;
- for (cpu = *pos; cpu < NR_CPUS; ++cpu) {
+ for (cpu = *pos; cpu < nr_cpu_ids; ++cpu) {
if (!cpu_possible(cpu))
continue;
*pos = cpu+1;
--- test-compile.orig/net/ipv4/route.c
+++ test-compile/net/ipv4/route.c
@@ -427,7 +427,7 @@ static void *rt_cpu_seq_start(struct seq
if (*pos == 0)
return SEQ_START_TOKEN;
- for (cpu = *pos-1; cpu < NR_CPUS; ++cpu) {
+ for (cpu = *pos-1; cpu < nr_cpu_ids; ++cpu) {
if (!cpu_possible(cpu))
continue;
*pos = cpu+1;
@@ -440,7 +440,7 @@ static void *rt_cpu_seq_next(struct seq_
{
int cpu;
- for (cpu = *pos; cpu < NR_CPUS; ++cpu) {
+ for (cpu = *pos; cpu < nr_cpu_ids; ++cpu) {
if (!cpu_possible(cpu))
continue;
*pos = cpu+1;
--- test-compile.orig/net/netfilter/nf_conntrack_standalone.c
+++ test-compile/net/netfilter/nf_conntrack_standalone.c
@@ -200,7 +200,7 @@ static void *ct_cpu_seq_start(struct seq
if (*pos == 0)
return SEQ_START_TOKEN;
- for (cpu = *pos-1; cpu < NR_CPUS; ++cpu) {
+ for (cpu = *pos-1; cpu < nr_cpu_ids; ++cpu) {
if (!cpu_possible(cpu))
continue;
*pos = cpu + 1;
@@ -215,7 +215,7 @@ static void *ct_cpu_seq_next(struct seq_
struct net *net = seq_file_net(seq);
int cpu;
- for (cpu = *pos; cpu < NR_CPUS; ++cpu) {
+ for (cpu = *pos; cpu < nr_cpu_ids; ++cpu) {
if (!cpu_possible(cpu))
continue;
*pos = cpu + 1;
--- test-compile.orig/security/selinux/selinuxfs.c
+++ test-compile/security/selinux/selinuxfs.c
@@ -1206,7 +1206,7 @@ static struct avc_cache_stats *sel_avc_g
{
int cpu;
- for (cpu = *idx; cpu < NR_CPUS; ++cpu) {
+ for (cpu = *idx; cpu < nr_cpu_ids; ++cpu) {
if (!cpu_possible(cpu))
continue;
*idx = cpu + 1;
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 16/35] percpu: fix percpu accessors to potentially !cpu_possible() cpus
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (14 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 15/35] cpumask: prepare for iterators to only go to nr_cpu_ids Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 17/35] cpumask: make nr_cpu_ids the actual limit on bitmap size Mike Travis
` (18 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Percpu areas are only allocated for possible cpus. In general, you
shouldn't access random cpu's percpu areas: you're corrupting memory.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
arch/m32r/kernel/smpboot.c | 2 +-
arch/x86/mach-voyager/voyager_smp.c | 2 +-
drivers/pnp/pnpbios/bioscalls.c | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
--- test-compile.orig/arch/m32r/kernel/smpboot.c
+++ test-compile/arch/m32r/kernel/smpboot.c
@@ -598,7 +598,7 @@ int setup_profiling_timer(unsigned int m
* accounting. At that time they also adjust their APIC timers
* accordingly.
*/
- for (i = 0; i < NR_CPUS; ++i)
+ for_each_possible_cpu(i)
per_cpu(prof_multiplier, i) = multiplier;
return 0;
--- test-compile.orig/arch/x86/mach-voyager/voyager_smp.c
+++ test-compile/arch/x86/mach-voyager/voyager_smp.c
@@ -1223,7 +1223,7 @@ int setup_profiling_timer(unsigned int m
* new values until the next timer interrupt in which they do process
* accounting.
*/
- for (i = 0; i < NR_CPUS; ++i)
+ for_each_possible_cpu(i)
per_cpu(prof_multiplier, i) = multiplier;
return 0;
--- test-compile.orig/drivers/pnp/pnpbios/bioscalls.c
+++ test-compile/drivers/pnp/pnpbios/bioscalls.c
@@ -481,7 +481,7 @@ void pnpbios_calls_init(union pnp_bios_i
set_base(bad_bios_desc, __va((unsigned long)0x40 << 4));
_set_limit((char *)&bad_bios_desc, 4095 - (0x40 << 4));
- for (i = 0; i < NR_CPUS; i++) {
+ for_each_possible_cpu(i) {
struct desc_struct *gdt = get_cpu_gdt_table(i);
if (!gdt)
continue;
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 17/35] cpumask: make nr_cpu_ids the actual limit on bitmap size
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (15 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 16/35] percpu: fix percpu accessors to potentially !cpu_possible() cpus Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 18/35] cpumask: add nr_cpumask_bits Mike Travis
` (17 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
nr_cpu_ids is the (badly named) runtime limit on possible CPU numbers;
ie. the variable version of NR_CPUS.
If we use this in *all* the cpu ops it simplifies the API, and will
be possible to allocate cpumasks of the minimal length at runtime.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 110 +++++++++++++++++------------------------------
include/linux/seq_file.h | 2
include/linux/cpumask.h | 114 ++++++++++++++++-------------------------------
include/linux/seq_file.h | 2
lib/cpumask.c | 14 +----
3 files changed, 45 insertions(+), 85 deletions(-)
--- test-compile.orig/include/linux/cpumask.h
+++ test-compile/include/linux/cpumask.h
@@ -3,7 +3,8 @@
/*
* Cpumasks provide a bitmap suitable for representing the
- * set of CPU's in a system, one bit position per CPU number.
+ * set of CPU's in a system, one bit position per CPU number up to
+ * nr_cpu_ids (<= NR_CPUS).
*
* Old-style uses "cpumask_t", but new ops are "struct cpumask *";
* don't put "struct cpumask"s on the stack.
@@ -20,20 +21,6 @@
* For details of cpumask_onto(), see bitmap_onto in lib/bitmap.c.
* For details of cpumask_fold(), see bitmap_fold in lib/bitmap.c.
*
- * . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- * Note: The alternate operations with the suffix "_nr" are used
- * to limit the range of the loop to nr_cpu_ids instead of
- * NR_CPUS when NR_CPUS > 64 for performance reasons.
- * If NR_CPUS is <= 64 then most assembler bitmask
- * operators execute faster with a constant range, so
- * the operator will continue to use NR_CPUS.
- *
- * Another consideration is that nr_cpu_ids is initialized
- * to NR_CPUS and isn't lowered until the possible cpus are
- * discovered (including any disabled cpus). So early uses
- * will span the entire range of NR_CPUS.
- * . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- *
* The available cpumask operations are:
*
* void cpumask_set_cpu(cpu, mask) turn on bit 'cpu' in mask
@@ -55,14 +42,12 @@
* int cpumask_empty(mask) Is mask empty (no bits sets)?
* int cpumask_full(mask) Is mask full (all bits sets)?
* int cpumask_weight(mask) Hamming weigh - number of set bits
- * int cpumask_weight_nr(mask) Same using nr_cpu_ids instead of NR_CPUS
*
* void cpumask_shift_right(dst, src, n) Shift right
* void cpumask_shift_left(dst, src, n) Shift left
*
- * int first_cpu(mask) Number lowest set bit, or NR_CPUS
- * int next_cpu(cpu, mask) Next cpu past 'cpu', or NR_CPUS
- * int next_cpu_nr(cpu, mask) Next cpu past 'cpu', or nr_cpu_ids
+ * int first_cpu(mask) Number lowest set bit, or nr_cpu_ids
+ * int next_cpu(cpu, mask) Next cpu past 'cpu', or nr_cpu_ids
*
* void cpumask_copy(dmask, smask) dmask = smask
*
@@ -113,8 +98,7 @@
* void cpumask_onto(dst, orig, relmap) *dst = orig relative to relmap
* void cpumask_fold(dst, orig, sz) dst bits = orig bits mod sz
*
- * for_each_cpu_mask(cpu, mask) for-loop cpu over mask using NR_CPUS
- * for_each_cpu_mask_nr(cpu, mask) for-loop cpu over mask using nr_cpu_ids
+ * for_each_cpu_mask(cpu, mask) for-loop cpu over mask using nr_cpu_ids
* for_each_cpu_mask_and(cpu, mask, and) for-loop cpu over (mask & and).
*
* int num_online_cpus() Number of online CPUs
@@ -154,7 +138,7 @@ struct cpumask
static inline ssize_t cpumask_size(void)
{
- return BITS_TO_LONGS(NR_CPUS) * sizeof(long);
+ return BITS_TO_LONGS(nr_cpu_ids) * sizeof(long);
}
/* Deprecated. */
@@ -193,6 +177,9 @@ extern cpumask_t _unused_cpumask_arg_;
#define cpus_fold(dst, orig, sz) \
cpumask_fold(&(dst), &(orig), sz)
#define cpus_addr(src) ((src).bits)
+#define next_cpu_nr(n, src) next_cpu(n, src)
+#define cpus_weight_nr(cpumask) cpus_weight(cpumask)
+#define for_each_cpu_mask_nr(cpu, mask) for_each_cpu_mask(cpu, mask)
/* End deprecated region. */
#if NR_CPUS <= BITS_PER_LONG
@@ -223,73 +210,73 @@ static inline int cpumask_test_and_set_c
static inline void cpumask_setall(struct cpumask *dstp)
{
- bitmap_fill(dstp->bits, NR_CPUS);
+ bitmap_fill(dstp->bits, nr_cpu_ids);
}
static inline void cpumask_clear(struct cpumask *dstp)
{
- bitmap_zero(dstp->bits, NR_CPUS);
+ bitmap_zero(dstp->bits, nr_cpu_ids);
}
static inline void cpumask_and(struct cpumask *dstp,
const struct cpumask *src1p,
const struct cpumask *src2p)
{
- bitmap_and(dstp->bits, src1p->bits, src2p->bits, NR_CPUS);
+ bitmap_and(dstp->bits, src1p->bits, src2p->bits, nr_cpu_ids);
}
static inline void cpumask_or(struct cpumask *dstp, const struct cpumask *src1p,
const struct cpumask *src2p)
{
- bitmap_or(dstp->bits, src1p->bits, src2p->bits, NR_CPUS);
+ bitmap_or(dstp->bits, src1p->bits, src2p->bits, nr_cpu_ids);
}
static inline void cpumask_xor(struct cpumask *dstp,
const struct cpumask *src1p,
const struct cpumask *src2p)
{
- bitmap_xor(dstp->bits, src1p->bits, src2p->bits, NR_CPUS);
+ bitmap_xor(dstp->bits, src1p->bits, src2p->bits, nr_cpu_ids);
}
static inline void cpumask_andnot(struct cpumask *dstp,
const struct cpumask *src1p,
const struct cpumask *src2p)
{
- bitmap_andnot(dstp->bits, src1p->bits, src2p->bits, NR_CPUS);
+ bitmap_andnot(dstp->bits, src1p->bits, src2p->bits, nr_cpu_ids);
}
static inline void cpumask_complement(struct cpumask *dstp,
const struct cpumask *srcp)
{
- bitmap_complement(dstp->bits, srcp->bits, NR_CPUS);
+ bitmap_complement(dstp->bits, srcp->bits, nr_cpu_ids);
}
static inline int cpumask_equal(const struct cpumask *src1p,
const struct cpumask *src2p)
{
- return bitmap_equal(src1p->bits, src2p->bits, NR_CPUS);
+ return bitmap_equal(src1p->bits, src2p->bits, nr_cpu_ids);
}
static inline int cpumask_intersects(const struct cpumask *src1p,
const struct cpumask *src2p)
{
- return bitmap_intersects(src1p->bits, src2p->bits, NR_CPUS);
+ return bitmap_intersects(src1p->bits, src2p->bits, nr_cpu_ids);
}
static inline int cpumask_subset(const struct cpumask *src1p,
const struct cpumask *src2p)
{
- return bitmap_subset(src1p->bits, src2p->bits, NR_CPUS);
+ return bitmap_subset(src1p->bits, src2p->bits, nr_cpu_ids);
}
static inline int cpumask_empty(const struct cpumask *srcp)
{
- return bitmap_empty(srcp->bits, NR_CPUS);
+ return bitmap_empty(srcp->bits, nr_cpu_ids);
}
static inline int cpumask_full(const struct cpumask *srcp)
{
- return bitmap_full(srcp->bits, NR_CPUS);
+ return bitmap_full(srcp->bits, nr_cpu_ids);
}
static inline int __cpus_weight(const cpumask_t *srcp, int nbits)
@@ -299,49 +286,49 @@ static inline int __cpus_weight(const cp
static inline int cpumask_weight(const struct cpumask *srcp)
{
- return bitmap_weight(srcp->bits, NR_CPUS);
+ return bitmap_weight(srcp->bits, nr_cpu_ids);
}
static inline void cpumask_shift_right(struct cpumask *dstp,
const struct cpumask *srcp, int n)
{
- bitmap_shift_right(dstp->bits, srcp->bits, n, NR_CPUS);
+ bitmap_shift_right(dstp->bits, srcp->bits, n, nr_cpu_ids);
}
static inline void cpumask_shift_left(struct cpumask *dstp,
const struct cpumask *srcp, int n)
{
- bitmap_shift_left(dstp->bits, srcp->bits, n, NR_CPUS);
+ bitmap_shift_left(dstp->bits, srcp->bits, n, nr_cpu_ids);
}
static inline int cpumask_scnprintf(char *buf, int len,
const struct cpumask *srcp)
{
- return bitmap_scnprintf(buf, len, srcp->bits, NR_CPUS);
+ return bitmap_scnprintf(buf, len, srcp->bits, nr_cpu_ids);
}
static inline int cpumask_parse_user(const char __user *buf, int len,
struct cpumask *dstp)
{
- return bitmap_parse_user(buf, len, dstp->bits, NR_CPUS);
+ return bitmap_parse_user(buf, len, dstp->bits, nr_cpu_ids);
}
static inline int cpulist_scnprintf(char *buf, int len,
const struct cpumask *srcp)
{
- return bitmap_scnlistprintf(buf, len, srcp->bits, NR_CPUS);
+ return bitmap_scnlistprintf(buf, len, srcp->bits, nr_cpu_ids);
}
static inline int cpulist_parse(const char *buf, struct cpumask *dstp)
{
- return bitmap_parselist(buf, dstp->bits, NR_CPUS);
+ return bitmap_parselist(buf, dstp->bits, nr_cpu_ids);
}
static inline int cpumask_cpuremap(int oldbit,
const struct cpumask *oldp,
const struct cpumask *newp)
{
- return bitmap_bitremap(oldbit, oldp->bits, newp->bits, NR_CPUS);
+ return bitmap_bitremap(oldbit, oldp->bits, newp->bits, nr_cpu_ids);
}
static inline void cpumask_remap(struct cpumask *dstp,
@@ -349,26 +336,27 @@ static inline void cpumask_remap(struct
const struct cpumask *oldp,
const struct cpumask *newp)
{
- bitmap_remap(dstp->bits, srcp->bits, oldp->bits, newp->bits, NR_CPUS);
+ bitmap_remap(dstp->bits, srcp->bits, oldp->bits, newp->bits,
+ nr_cpu_ids);
}
static inline void cpumask_onto(struct cpumask *dstp,
const struct cpumask *origp,
const struct cpumask *relmapp)
{
- bitmap_onto(dstp->bits, origp->bits, relmapp->bits, NR_CPUS);
+ bitmap_onto(dstp->bits, origp->bits, relmapp->bits, nr_cpu_ids);
}
static inline void cpumask_fold(struct cpumask *dstp,
const struct cpumask *origp, int sz)
{
- bitmap_fold(dstp->bits, origp->bits, sz, NR_CPUS);
+ bitmap_fold(dstp->bits, origp->bits, sz, nr_cpu_ids);
}
static inline void cpumask_copy(struct cpumask *dstp,
const struct cpumask *srcp)
{
- bitmap_copy(cpumask_bits(dstp), cpumask_bits(srcp), NR_CPUS);
+ bitmap_copy(cpumask_bits(dstp), cpumask_bits(srcp), nr_cpu_ids);
}
/*
@@ -465,7 +453,7 @@ int __any_online_cpu(const cpumask_t *ma
#define for_each_cpu_mask(cpu, mask) \
for ((cpu) = -1; \
(cpu) = next_cpu((cpu), (mask)), \
- (cpu) < NR_CPUS;)
+ (cpu) < nr_cpu_ids;)
#define for_each_cpu_mask_and(cpu, mask, and) \
for ((cpu) = -1; \
(cpu) = cpumask_next_and((cpu), &(mask), &(and)), \
@@ -474,26 +462,6 @@ int __any_online_cpu(const cpumask_t *ma
#define cpumask_first_and(mask, and) cpumask_next_and(-1, (mask), (and))
-#if NR_CPUS <= 64
-
-#define next_cpu_nr(n, src) next_cpu(n, src)
-#define cpus_weight_nr(cpumask) cpus_weight(cpumask)
-#define for_each_cpu_mask_nr(cpu, mask) for_each_cpu_mask(cpu, mask)
-
-#else /* NR_CPUS > 64 */
-
-int __next_cpu_nr(int n, const cpumask_t *srcp);
-
-#define next_cpu_nr(n, src) __next_cpu_nr((n), &(src))
-#define cpus_weight_nr(cpumask) __cpus_weight(&(cpumask), nr_cpu_ids)
-
-#define for_each_cpu_mask_nr(cpu, mask) \
- for ((cpu) = -1; \
- (cpu) = next_cpu_nr((cpu), (mask)), \
- (cpu) < nr_cpu_ids;)
-
-#endif /* NR_CPUS > 64 */
-
/*
* cpumask_var_t: struct cpumask for stack usage.
*
@@ -593,9 +561,9 @@ extern cpumask_t cpu_present_map;
extern cpumask_t cpu_active_map;
#if NR_CPUS > 1
-#define num_online_cpus() cpus_weight_nr(cpu_online_map)
-#define num_possible_cpus() cpus_weight_nr(cpu_possible_map)
-#define num_present_cpus() cpus_weight_nr(cpu_present_map)
+#define num_online_cpus() cpus_weight(cpu_online_map)
+#define num_possible_cpus() cpus_weight(cpu_possible_map)
+#define num_present_cpus() cpus_weight(cpu_present_map)
#define cpu_online(cpu) cpu_isset((cpu), cpu_online_map)
#define cpu_possible(cpu) cpu_isset((cpu), cpu_possible_map)
#define cpu_present(cpu) cpu_isset((cpu), cpu_present_map)
@@ -612,9 +580,9 @@ extern cpumask_t cpu_active_map;
#define cpu_is_offline(cpu) unlikely(!cpu_online(cpu))
-#define for_each_possible_cpu(cpu) for_each_cpu_mask_nr((cpu), cpu_possible_map)
-#define for_each_online_cpu(cpu) for_each_cpu_mask_nr((cpu), cpu_online_map)
-#define for_each_present_cpu(cpu) for_each_cpu_mask_nr((cpu), cpu_present_map)
+#define for_each_possible_cpu(cpu) for_each_cpu_mask((cpu), cpu_possible_map)
+#define for_each_online_cpu(cpu) for_each_cpu_mask((cpu), cpu_online_map)
+#define for_each_present_cpu(cpu) for_each_cpu_mask((cpu), cpu_present_map)
/* Arch-specific code may call this to initialize nr_cpu_ids based on
* the cpu_possible_map. */
--- test-compile.orig/include/linux/seq_file.h
+++ test-compile/include/linux/seq_file.h
@@ -52,7 +52,7 @@ int seq_path_root(struct seq_file *m, st
int seq_bitmap(struct seq_file *m, unsigned long *bits, unsigned int nr_bits);
static inline int seq_cpumask(struct seq_file *m, cpumask_t *mask)
{
- return seq_bitmap(m, mask->bits, NR_CPUS);
+ return seq_bitmap(m, mask->bits, nr_cpu_ids);
}
static inline int seq_nodemask(struct seq_file *m, nodemask_t *mask)
--- test-compile.orig/lib/cpumask.c
+++ test-compile/lib/cpumask.c
@@ -5,33 +5,25 @@
int __first_cpu(const cpumask_t *srcp)
{
- return find_first_bit(srcp->bits, NR_CPUS);
+ return find_first_bit(srcp->bits, nr_cpu_ids);
}
EXPORT_SYMBOL(__first_cpu);
int __next_cpu(int n, const cpumask_t *srcp)
{
- return find_next_bit(srcp->bits, NR_CPUS, n+1);
+ return find_next_bit(srcp->bits, nr_cpu_ids, n+1);
}
EXPORT_SYMBOL(__next_cpu);
int cpumask_next_and(int n, const cpumask_t *srcp, const cpumask_t *andp)
{
- while ((n = next_cpu_nr(n, *srcp)) < nr_cpu_ids)
+ while ((n = next_cpu(n, *srcp)) < nr_cpu_ids)
if (cpumask_test_cpu(n, andp))
break;
return n;
}
EXPORT_SYMBOL(cpumask_next_and);
-#if NR_CPUS > 64
-int __next_cpu_nr(int n, const cpumask_t *srcp)
-{
- return find_next_bit(srcp->bits, nr_cpu_ids, n+1);
-}
-EXPORT_SYMBOL(__next_cpu_nr);
-#endif
-
int __any_online_cpu(const cpumask_t *mask)
{
int cpu;
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 18/35] cpumask: add nr_cpumask_bits
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (16 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 17/35] cpumask: make nr_cpu_ids the actual limit on bitmap size Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-21 12:26 ` Rusty Russell
2008-10-20 17:03 ` [PATCH 19/35] cpumask: use cpumask_bits() everywhere Mike Travis
` (16 subsequent siblings)
34 siblings, 1 reply; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
When nr_cpu_ids is set to CONFIG_NR_CPUS then references to nr_cpu_ids
will return the maximum index of the configured NR_CPUS (+1) instead
of the maximum index of the possible number of cpus (+1). This results
in extra unused memory being allocated by functions that are setting up
arrays of structs to keep track of per cpu items.
Since we do want to keep the ability to use constants for the cpu bit
operators on smaller systems (which are generally much faster assembler
ops), we introduce a separate "nr_cpumask_bits" to replace "nr_cpu_ids"
only for the inline assembly ops. This will be a constant when
CONFIG_CPUMASK_OFFSTACK is undefined and a variable when it is defined.
Thus "nr_cpu_ids" reverts back to being a variable representing the
maximum possible cpu (+1), except in the non-SMP case where it is a
constant value of 1. The relationship between the related variables
and constants is: (1 <= nr_cpu_ids <= nr_cpumask_bits <= NR_CPUS).
Signed-of-by: Mike Travis <travis@sgi.com>
---
arch/x86/kernel/setup_percpu.c | 7 +--
include/linux/cpumask.h | 94 +++++++++++++++++++++++------------------
init/main.c | 17 +++----
lib/cpumask.c | 8 +--
4 files changed, 72 insertions(+), 54 deletions(-)
--- test-compile.orig/arch/x86/kernel/setup_percpu.c
+++ test-compile/arch/x86/kernel/setup_percpu.c
@@ -155,6 +155,10 @@ void __init setup_per_cpu_areas(void)
printk(KERN_INFO "PERCPU: Allocating %zd bytes of per cpu data\n",
size);
+ printk(KERN_DEBUG
+ "NR_CPUS:%d nr_cpumask_bits:%d nr_cpu_ids:%d nr_node_ids:%d\n",
+ NR_CPUS, nr_cpumask_bits, nr_cpu_ids, nr_node_ids);
+
for_each_possible_cpu(cpu) {
#ifndef CONFIG_NEED_MULTIPLE_NODES
ptr = __alloc_bootmem(size, align,
@@ -183,9 +187,6 @@ void __init setup_per_cpu_areas(void)
memcpy(ptr, __per_cpu_start, __per_cpu_end - __per_cpu_start);
}
- printk(KERN_DEBUG "NR_CPUS: %d, nr_cpu_ids: %d, nr_node_ids %d\n",
- NR_CPUS, nr_cpu_ids, nr_node_ids);
-
/* Setup percpu data maps */
setup_per_cpu_maps();
--- test-compile.orig/include/linux/cpumask.h
+++ test-compile/include/linux/cpumask.h
@@ -4,7 +4,7 @@
/*
* Cpumasks provide a bitmap suitable for representing the
* set of CPU's in a system, one bit position per CPU number up to
- * nr_cpu_ids (<= NR_CPUS).
+ * nr_cpumask_bits (<= NR_CPUS).
*
* Old-style uses "cpumask_t", but new ops are "struct cpumask *";
* don't put "struct cpumask"s on the stack.
@@ -46,8 +46,8 @@
* void cpumask_shift_right(dst, src, n) Shift right
* void cpumask_shift_left(dst, src, n) Shift left
*
- * int first_cpu(mask) Number lowest set bit, or nr_cpu_ids
- * int next_cpu(cpu, mask) Next cpu past 'cpu', or nr_cpu_ids
+ * int first_cpu(mask) Number lowest set bit or nr_cpumask_bits
+ * int next_cpu(cpu, mask) Next cpu past 'cpu', or nr_cpumask_bits
*
* void cpumask_copy(dmask, smask) dmask = smask
*
@@ -98,7 +98,8 @@
* void cpumask_onto(dst, orig, relmap) *dst = orig relative to relmap
* void cpumask_fold(dst, orig, sz) dst bits = orig bits mod sz
*
- * for_each_cpu_mask(cpu, mask) for-loop cpu over mask using nr_cpu_ids
+ * for_each_cpu_mask(cpu, mask) for-loop cpu over mask
+ * using nr_cpumask_bits
* for_each_cpu_mask_and(cpu, mask, and) for-loop cpu over (mask & and).
*
* int num_online_cpus() Number of online CPUs
@@ -136,11 +137,34 @@ struct cpumask
};
#define cpumask_bits(maskp) ((maskp)->bits)
+/*
+ * nr_cpu_ids: max cpu index (+1)
+ * nr_cpumask_bits: number of bits in a struct cpumask
+ * 1 <= nr_cpu_ids <= nr_cpumask_bits <= NR_CPUS
+ *
+ * Use CONFIG_CPUMASK_OFFSTACK to force variable length struct cpumask
+ */
+#if NR_CPUS == 1
+#define nr_cpumask_bits NR_CPUS
+#define nr_cpu_ids NR_CPUS
+#else
+
+#ifdef CONFIG_CPUMASK_OFFSTACK
+extern int nr_cpumask_bits;
+extern int nr_cpu_ids;
+#else
+
+#define nr_cpumask_bits NR_CPUS
+extern int nr_cpu_ids;
+#endif
+#endif
+
static inline ssize_t cpumask_size(void)
{
- return BITS_TO_LONGS(nr_cpu_ids) * sizeof(long);
+ return BITS_TO_LONGS(nr_cpumask_bits) * sizeof(long);
}
+
/* Deprecated. */
typedef struct cpumask cpumask_t;
extern cpumask_t _unused_cpumask_arg_;
@@ -182,14 +206,6 @@ extern cpumask_t _unused_cpumask_arg_;
#define for_each_cpu_mask_nr(cpu, mask) for_each_cpu_mask(cpu, mask)
/* End deprecated region. */
-#if NR_CPUS <= BITS_PER_LONG
-/* Constant is usually more efficient than a variable for small NR_CPUS */
-#define nr_cpu_ids NR_CPUS
-#else
-/* Starts at NR_CPUS until we know better. */
-extern int nr_cpu_ids;
-#endif /* NR_CPUS > BITS_PER_LONG */
-
static inline void cpumask_set_cpu(int cpu, volatile struct cpumask *dstp)
{
set_bit(cpu, dstp->bits);
@@ -210,73 +226,73 @@ static inline int cpumask_test_and_set_c
static inline void cpumask_setall(struct cpumask *dstp)
{
- bitmap_fill(dstp->bits, nr_cpu_ids);
+ bitmap_fill(dstp->bits, nr_cpumask_bits);
}
static inline void cpumask_clear(struct cpumask *dstp)
{
- bitmap_zero(dstp->bits, nr_cpu_ids);
+ bitmap_zero(dstp->bits, nr_cpumask_bits);
}
static inline void cpumask_and(struct cpumask *dstp,
const struct cpumask *src1p,
const struct cpumask *src2p)
{
- bitmap_and(dstp->bits, src1p->bits, src2p->bits, nr_cpu_ids);
+ bitmap_and(dstp->bits, src1p->bits, src2p->bits, nr_cpumask_bits);
}
static inline void cpumask_or(struct cpumask *dstp, const struct cpumask *src1p,
const struct cpumask *src2p)
{
- bitmap_or(dstp->bits, src1p->bits, src2p->bits, nr_cpu_ids);
+ bitmap_or(dstp->bits, src1p->bits, src2p->bits, nr_cpumask_bits);
}
static inline void cpumask_xor(struct cpumask *dstp,
const struct cpumask *src1p,
const struct cpumask *src2p)
{
- bitmap_xor(dstp->bits, src1p->bits, src2p->bits, nr_cpu_ids);
+ bitmap_xor(dstp->bits, src1p->bits, src2p->bits, nr_cpumask_bits);
}
static inline void cpumask_andnot(struct cpumask *dstp,
const struct cpumask *src1p,
const struct cpumask *src2p)
{
- bitmap_andnot(dstp->bits, src1p->bits, src2p->bits, nr_cpu_ids);
+ bitmap_andnot(dstp->bits, src1p->bits, src2p->bits, nr_cpumask_bits);
}
static inline void cpumask_complement(struct cpumask *dstp,
const struct cpumask *srcp)
{
- bitmap_complement(dstp->bits, srcp->bits, nr_cpu_ids);
+ bitmap_complement(dstp->bits, srcp->bits, nr_cpumask_bits);
}
static inline int cpumask_equal(const struct cpumask *src1p,
const struct cpumask *src2p)
{
- return bitmap_equal(src1p->bits, src2p->bits, nr_cpu_ids);
+ return bitmap_equal(src1p->bits, src2p->bits, nr_cpumask_bits);
}
static inline int cpumask_intersects(const struct cpumask *src1p,
const struct cpumask *src2p)
{
- return bitmap_intersects(src1p->bits, src2p->bits, nr_cpu_ids);
+ return bitmap_intersects(src1p->bits, src2p->bits, nr_cpumask_bits);
}
static inline int cpumask_subset(const struct cpumask *src1p,
const struct cpumask *src2p)
{
- return bitmap_subset(src1p->bits, src2p->bits, nr_cpu_ids);
+ return bitmap_subset(src1p->bits, src2p->bits, nr_cpumask_bits);
}
static inline int cpumask_empty(const struct cpumask *srcp)
{
- return bitmap_empty(srcp->bits, nr_cpu_ids);
+ return bitmap_empty(srcp->bits, nr_cpumask_bits);
}
static inline int cpumask_full(const struct cpumask *srcp)
{
- return bitmap_full(srcp->bits, nr_cpu_ids);
+ return bitmap_full(srcp->bits, nr_cpumask_bits);
}
static inline int __cpus_weight(const cpumask_t *srcp, int nbits)
@@ -286,49 +302,49 @@ static inline int __cpus_weight(const cp
static inline int cpumask_weight(const struct cpumask *srcp)
{
- return bitmap_weight(srcp->bits, nr_cpu_ids);
+ return bitmap_weight(srcp->bits, nr_cpumask_bits);
}
static inline void cpumask_shift_right(struct cpumask *dstp,
const struct cpumask *srcp, int n)
{
- bitmap_shift_right(dstp->bits, srcp->bits, n, nr_cpu_ids);
+ bitmap_shift_right(dstp->bits, srcp->bits, n, nr_cpumask_bits);
}
static inline void cpumask_shift_left(struct cpumask *dstp,
const struct cpumask *srcp, int n)
{
- bitmap_shift_left(dstp->bits, srcp->bits, n, nr_cpu_ids);
+ bitmap_shift_left(dstp->bits, srcp->bits, n, nr_cpumask_bits);
}
static inline int cpumask_scnprintf(char *buf, int len,
const struct cpumask *srcp)
{
- return bitmap_scnprintf(buf, len, srcp->bits, nr_cpu_ids);
+ return bitmap_scnprintf(buf, len, srcp->bits, nr_cpumask_bits);
}
static inline int cpumask_parse_user(const char __user *buf, int len,
struct cpumask *dstp)
{
- return bitmap_parse_user(buf, len, dstp->bits, nr_cpu_ids);
+ return bitmap_parse_user(buf, len, dstp->bits, nr_cpumask_bits);
}
static inline int cpulist_scnprintf(char *buf, int len,
const struct cpumask *srcp)
{
- return bitmap_scnlistprintf(buf, len, srcp->bits, nr_cpu_ids);
+ return bitmap_scnlistprintf(buf, len, srcp->bits, nr_cpumask_bits);
}
static inline int cpulist_parse(const char *buf, struct cpumask *dstp)
{
- return bitmap_parselist(buf, dstp->bits, nr_cpu_ids);
+ return bitmap_parselist(buf, dstp->bits, nr_cpumask_bits);
}
static inline int cpumask_cpuremap(int oldbit,
const struct cpumask *oldp,
const struct cpumask *newp)
{
- return bitmap_bitremap(oldbit, oldp->bits, newp->bits, nr_cpu_ids);
+ return bitmap_bitremap(oldbit, oldp->bits, newp->bits, nr_cpumask_bits);
}
static inline void cpumask_remap(struct cpumask *dstp,
@@ -337,26 +353,26 @@ static inline void cpumask_remap(struct
const struct cpumask *newp)
{
bitmap_remap(dstp->bits, srcp->bits, oldp->bits, newp->bits,
- nr_cpu_ids);
+ nr_cpumask_bits);
}
static inline void cpumask_onto(struct cpumask *dstp,
const struct cpumask *origp,
const struct cpumask *relmapp)
{
- bitmap_onto(dstp->bits, origp->bits, relmapp->bits, nr_cpu_ids);
+ bitmap_onto(dstp->bits, origp->bits, relmapp->bits, nr_cpumask_bits);
}
static inline void cpumask_fold(struct cpumask *dstp,
const struct cpumask *origp, int sz)
{
- bitmap_fold(dstp->bits, origp->bits, sz, nr_cpu_ids);
+ bitmap_fold(dstp->bits, origp->bits, sz, nr_cpumask_bits);
}
static inline void cpumask_copy(struct cpumask *dstp,
const struct cpumask *srcp)
{
- bitmap_copy(cpumask_bits(dstp), cpumask_bits(srcp), nr_cpu_ids);
+ bitmap_copy(cpumask_bits(dstp), cpumask_bits(srcp), nr_cpumask_bits);
}
/*
@@ -454,11 +470,11 @@ int __any_online_cpu(const cpumask_t *ma
#define for_each_cpu_mask(cpu, mask) \
for ((cpu) = -1; \
(cpu) = next_cpu((cpu), (mask)), \
- (cpu) < nr_cpu_ids;)
+ (cpu) < nr_cpumask_bits;)
#define for_each_cpu_mask_and(cpu, mask, and) \
for ((cpu) = -1; \
(cpu) = cpumask_next_and((cpu), &(mask), &(and)), \
- (cpu) < nr_cpu_ids;)
+ (cpu) < nr_cpumask_bits;)
#endif
#define cpumask_first_and(mask, and) cpumask_next_and(-1, (mask), (and))
--- test-compile.orig/init/main.c
+++ test-compile/init/main.c
@@ -373,10 +373,13 @@ EXPORT_SYMBOL(cpu_mask_all);
#endif
/* Setup number of possible processor ids */
-/* nr_cpu_ids is a real variable for large NR_CPUS. */
-#ifndef nr_cpu_ids
+/* nr_cpumask_bits is a real variable for large NR_CPUS. */
+#ifndef nr_cpumask_bits
+int nr_cpumask_bits __read_mostly = NR_CPUS;
+EXPORT_SYMBOL(nr_cpumask_bits);
+#endif
+
int nr_cpu_ids __read_mostly = NR_CPUS;
-EXPORT_SYMBOL(nr_cpu_ids);
/* An arch may set nr_cpu_ids earlier if needed, so this would be redundant */
void __init setup_nr_cpu_ids(void)
@@ -387,12 +390,10 @@ void __init setup_nr_cpu_ids(void)
highest_cpu = cpu;
nr_cpu_ids = highest_cpu + 1;
+#ifndef nr_cpumask_bits
+ nr_cpumask_bits = highest_cpu + 1;
+#endif
}
-#else
-void __init setup_nr_cpu_ids(void)
-{
-}
-#endif /* ... nr_cpu_ids is a constant. */
#ifndef CONFIG_HAVE_SETUP_PER_CPU_AREA
unsigned long __per_cpu_offset[NR_CPUS] __read_mostly;
--- test-compile.orig/lib/cpumask.c
+++ test-compile/lib/cpumask.c
@@ -5,19 +5,19 @@
int __first_cpu(const cpumask_t *srcp)
{
- return find_first_bit(srcp->bits, nr_cpu_ids);
+ return find_first_bit(srcp->bits, nr_cpumask_bits);
}
EXPORT_SYMBOL(__first_cpu);
int __next_cpu(int n, const cpumask_t *srcp)
{
- return find_next_bit(srcp->bits, nr_cpu_ids, n+1);
+ return find_next_bit(srcp->bits, nr_cpumask_bits, n+1);
}
EXPORT_SYMBOL(__next_cpu);
int cpumask_next_and(int n, const cpumask_t *srcp, const cpumask_t *andp)
{
- while ((n = next_cpu(n, *srcp)) < nr_cpu_ids)
+ while ((n = next_cpu(n, *srcp)) < nr_cpumask_bits)
if (cpumask_test_cpu(n, andp))
break;
return n;
@@ -41,7 +41,7 @@ EXPORT_SYMBOL(__any_online_cpu);
bool alloc_cpumask_var(cpumask_var_t *mask, gfp_t flags)
{
if (likely(slab_is_available()))
- *mask = kmalloc(BITS_TO_LONGS(nr_cpu_ids)*sizeof(long), flags);
+ *mask = kmalloc(cpumask_size(), flags);
else {
#ifdef CONFIG_DEBUG_PER_CPU_MAPS
printk(KERN_ERR
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 19/35] cpumask: use cpumask_bits() everywhere.
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (17 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 18/35] cpumask: add nr_cpumask_bits Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 20/35] cpumask: cpumask_of(): cpumask_of_cpu() which returns a pointer Mike Travis
` (15 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Instead of accessing ->bits, we use cpumask_bits(). This will be very
useful when 'struct cpumask' has a hidden definition.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 70 ++++++++++++++++++++++++++++-------------------
include/linux/seq_file.h | 2 -
kernel/time/timer_list.c | 4 +-
lib/cpumask.c | 4 +-
4 files changed, 47 insertions(+), 33 deletions(-)
--- linux-2.6.28.orig/include/linux/cpumask.h
+++ linux-2.6.28/include/linux/cpumask.h
@@ -191,12 +191,12 @@ extern int nr_cpu_ids;
static inline void cpumask_set_cpu(int cpu, volatile struct cpumask *dstp)
{
- set_bit(cpu, dstp->bits);
+ set_bit(cpu, cpumask_bits(dstp));
}
static inline void cpumask_clear_cpu(int cpu, volatile struct cpumask *dstp)
{
- clear_bit(cpu, dstp->bits);
+ clear_bit(cpu, cpumask_bits(dstp));
}
/* No static inline type checking - see Subtlety (1) above. */
@@ -204,130 +204,142 @@ static inline void cpumask_clear_cpu(int
static inline int cpumask_test_and_set_cpu(int cpu, struct cpumask *addr)
{
- return test_and_set_bit(cpu, addr->bits);
+ return test_and_set_bit(cpu, cpumask_bits(addr));
}
static inline void cpumask_setall(struct cpumask *dstp)
{
- bitmap_fill(dstp->bits, nr_cpumask_bits);
+ bitmap_fill(cpumask_bits(dstp), nr_cpumask_bits);
}
static inline void cpumask_clear(struct cpumask *dstp)
{
- bitmap_zero(dstp->bits, nr_cpumask_bits);
+ bitmap_zero(cpumask_bits(dstp), nr_cpumask_bits);
}
static inline void cpumask_and(struct cpumask *dstp,
const struct cpumask *src1p,
const struct cpumask *src2p)
{
- bitmap_and(dstp->bits, src1p->bits, src2p->bits, nr_cpumask_bits);
+ bitmap_and(cpumask_bits(dstp), cpumask_bits(src1p),
+ cpumask_bits(src2p), nr_cpumask_bits);
}
static inline void cpumask_or(struct cpumask *dstp, const struct cpumask *src1p,
const struct cpumask *src2p)
{
- bitmap_or(dstp->bits, src1p->bits, src2p->bits, nr_cpumask_bits);
+ bitmap_or(cpumask_bits(dstp), cpumask_bits(src1p),
+ cpumask_bits(src2p), nr_cpumask_bits);
}
static inline void cpumask_xor(struct cpumask *dstp,
const struct cpumask *src1p,
const struct cpumask *src2p)
{
- bitmap_xor(dstp->bits, src1p->bits, src2p->bits, nr_cpumask_bits);
+ bitmap_xor(cpumask_bits(dstp), cpumask_bits(src1p),
+ cpumask_bits(src2p), nr_cpumask_bits);
}
static inline void cpumask_andnot(struct cpumask *dstp,
const struct cpumask *src1p,
const struct cpumask *src2p)
{
- bitmap_andnot(dstp->bits, src1p->bits, src2p->bits, nr_cpumask_bits);
+ bitmap_andnot(cpumask_bits(dstp), cpumask_bits(src1p),
+ cpumask_bits(src2p), nr_cpumask_bits);
}
static inline void cpumask_complement(struct cpumask *dstp,
const struct cpumask *srcp)
{
- bitmap_complement(dstp->bits, srcp->bits, nr_cpumask_bits);
+ bitmap_complement(cpumask_bits(dstp), cpumask_bits(srcp),
+ nr_cpumask_bits);
}
static inline int cpumask_equal(const struct cpumask *src1p,
const struct cpumask *src2p)
{
- return bitmap_equal(src1p->bits, src2p->bits, nr_cpumask_bits);
+ return bitmap_equal(cpumask_bits(src1p), cpumask_bits(src2p),
+ nr_cpumask_bits);
}
static inline int cpumask_intersects(const struct cpumask *src1p,
const struct cpumask *src2p)
{
- return bitmap_intersects(src1p->bits, src2p->bits, nr_cpumask_bits);
+ return bitmap_intersects(cpumask_bits(src1p), cpumask_bits(src2p),
+ nr_cpumask_bits);
}
static inline int cpumask_subset(const struct cpumask *src1p,
const struct cpumask *src2p)
{
- return bitmap_subset(src1p->bits, src2p->bits, nr_cpumask_bits);
+ return bitmap_subset(cpumask_bits(src1p), cpumask_bits(src2p),
+ nr_cpumask_bits);
}
static inline int cpumask_empty(const struct cpumask *srcp)
{
- return bitmap_empty(srcp->bits, nr_cpumask_bits);
+ return bitmap_empty(cpumask_bits(srcp), nr_cpumask_bits);
}
static inline int cpumask_full(const struct cpumask *srcp)
{
- return bitmap_full(srcp->bits, nr_cpumask_bits);
+ return bitmap_full(cpumask_bits(srcp), nr_cpumask_bits);
}
static inline int __cpus_weight(const cpumask_t *srcp, int nbits)
{
- return bitmap_weight(srcp->bits, nbits);
+ return bitmap_weight(cpumask_bits(srcp), nbits);
}
static inline int cpumask_weight(const struct cpumask *srcp)
{
- return bitmap_weight(srcp->bits, nr_cpumask_bits);
+ return bitmap_weight(cpumask_bits(srcp), nr_cpumask_bits);
}
static inline void cpumask_shift_right(struct cpumask *dstp,
const struct cpumask *srcp, int n)
{
- bitmap_shift_right(dstp->bits, srcp->bits, n, nr_cpumask_bits);
+ bitmap_shift_right(cpumask_bits(dstp), cpumask_bits(srcp), n,
+ nr_cpumask_bits);
}
static inline void cpumask_shift_left(struct cpumask *dstp,
const struct cpumask *srcp, int n)
{
- bitmap_shift_left(dstp->bits, srcp->bits, n, nr_cpumask_bits);
+ bitmap_shift_left(cpumask_bits(dstp), cpumask_bits(srcp), n,
+ nr_cpumask_bits);
}
static inline int cpumask_scnprintf(char *buf, int len,
const struct cpumask *srcp)
{
- return bitmap_scnprintf(buf, len, srcp->bits, nr_cpumask_bits);
+ return bitmap_scnprintf(buf, len, cpumask_bits(srcp), nr_cpumask_bits);
}
static inline int cpumask_parse_user(const char __user *buf, int len,
struct cpumask *dstp)
{
- return bitmap_parse_user(buf, len, dstp->bits, nr_cpumask_bits);
+ return bitmap_parse_user(buf, len, cpumask_bits(dstp), nr_cpumask_bits);
}
static inline int cpulist_scnprintf(char *buf, int len,
const struct cpumask *srcp)
{
- return bitmap_scnlistprintf(buf, len, srcp->bits, nr_cpumask_bits);
+ return bitmap_scnlistprintf(buf, len, cpumask_bits(srcp),
+ nr_cpumask_bits);
}
static inline int cpulist_parse(const char *buf, struct cpumask *dstp)
{
- return bitmap_parselist(buf, dstp->bits, nr_cpumask_bits);
+ return bitmap_parselist(buf, cpumask_bits(dstp), nr_cpumask_bits);
}
static inline int cpumask_cpuremap(int oldbit,
const struct cpumask *oldp,
const struct cpumask *newp)
{
- return bitmap_bitremap(oldbit, oldp->bits, newp->bits, nr_cpumask_bits);
+ return bitmap_bitremap(oldbit, cpumask_bits(oldp), cpumask_bits(newp),
+ nr_cpumask_bits);
}
static inline void cpumask_remap(struct cpumask *dstp,
@@ -335,21 +347,23 @@ static inline void cpumask_remap(struct
const struct cpumask *oldp,
const struct cpumask *newp)
{
- bitmap_remap(dstp->bits, srcp->bits, oldp->bits, newp->bits,
- nr_cpumask_bits);
+ bitmap_remap(cpumask_bits(dstp), cpumask_bits(srcp),
+ cpumask_bits(oldp), cpumask_bits(newp), nr_cpumask_bits);
}
static inline void cpumask_onto(struct cpumask *dstp,
const struct cpumask *origp,
const struct cpumask *relmapp)
{
- bitmap_onto(dstp->bits, origp->bits, relmapp->bits, nr_cpumask_bits);
+ bitmap_onto(cpumask_bits(dstp), cpumask_bits(origp),
+ cpumask_bits(relmapp), nr_cpumask_bits);
}
static inline void cpumask_fold(struct cpumask *dstp,
const struct cpumask *origp, int sz)
{
- bitmap_fold(dstp->bits, origp->bits, sz, nr_cpumask_bits);
+ bitmap_fold(cpumask_bits(dstp), cpumask_bits(origp), sz,
+ nr_cpumask_bits);
}
static inline void cpumask_copy(struct cpumask *dstp,
--- linux-2.6.28.orig/include/linux/seq_file.h
+++ linux-2.6.28/include/linux/seq_file.h
@@ -52,7 +52,7 @@ int seq_path_root(struct seq_file *m, st
int seq_bitmap(struct seq_file *m, unsigned long *bits, unsigned int nr_bits);
static inline int seq_cpumask(struct seq_file *m, cpumask_t *mask)
{
- return seq_bitmap(m, mask->bits, nr_cpu_ids);
+ return seq_bitmap(m, cpumask_bits(mask), nr_cpu_ids);
}
static inline int seq_nodemask(struct seq_file *m, nodemask_t *mask)
--- linux-2.6.28.orig/kernel/time/timer_list.c
+++ linux-2.6.28/kernel/time/timer_list.c
@@ -226,10 +226,10 @@ static void timer_list_show_tickdevices(
#ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
print_tickdevice(m, tick_get_broadcast_device());
SEQ_printf(m, "tick_broadcast_mask: %08lx\n",
- tick_get_broadcast_mask()->bits[0]);
+ cpumask_bits(tick_get_broadcast_mask())[0]);
#ifdef CONFIG_TICK_ONESHOT
SEQ_printf(m, "tick_broadcast_oneshot_mask: %08lx\n",
- tick_get_broadcast_oneshot_mask()->bits[0]);
+ cpumask_bits(tick_get_broadcast_oneshot_mask())[0]);
#endif
SEQ_printf(m, "\n");
#endif
--- linux-2.6.28.orig/lib/cpumask.c
+++ linux-2.6.28/lib/cpumask.c
@@ -5,13 +5,13 @@
int __first_cpu(const cpumask_t *srcp)
{
- return find_first_bit(srcp->bits, nr_cpumask_bits);
+ return find_first_bit(cpumask_bits(srcp), nr_cpumask_bits);
}
EXPORT_SYMBOL(__first_cpu);
int __next_cpu(int n, const cpumask_t *srcp)
{
- return find_next_bit(srcp->bits, nr_cpumask_bits, n+1);
+ return find_next_bit(cpumask_bits(srcp), nr_cpumask_bits, n+1);
}
EXPORT_SYMBOL(__next_cpu);
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 20/35] cpumask: cpumask_of(): cpumask_of_cpu() which returns a pointer.
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (18 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 19/35] cpumask: use cpumask_bits() everywhere Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 21/35] cpumask: for_each_cpu(): for_each_cpu_mask which takes " Mike Travis
` (14 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
This deprecates cpumask_of_cpu(), which returns a cpumask_t
(cpumask_of() returns a const pointer, instead).
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 16 ++++------------
1 file changed, 4 insertions(+), 12 deletions(-)
--- linux-2.6.28.orig/include/linux/cpumask.h
+++ linux-2.6.28/include/linux/cpumask.h
@@ -52,8 +52,7 @@
* void cpumask_copy(dmask, smask) dmask = smask
*
* size_t cpumask_size() Length of cpumask in bytes.
- * cpumask_t cpumask_of_cpu(cpu) Return cpumask with bit 'cpu' set
- * (can be used as an lvalue)
+ * const struct cpumask *cpumask_of(cpu) Return cpumask with bit 'cpu' set
* CPU_MASK_ALL Initializer - all bits set
* CPU_MASK_NONE Initializer - no bits set
* unsigned long *cpumask_bits(mask) Array of unsigned long's in mask
@@ -176,6 +175,7 @@ extern cpumask_t _unused_cpumask_arg_;
#define next_cpu_nr(n, src) next_cpu(n, src)
#define cpus_weight_nr(cpumask) cpus_weight(cpumask)
#define for_each_cpu_mask_nr(cpu, mask) for_each_cpu_mask(cpu, mask)
+#define cpumask_of_cpu(cpu) (*cpumask_of(cpu))
/* End deprecated region. */
#if NR_CPUS <= BITS_PER_LONG
@@ -382,21 +382,13 @@ static inline void cpumask_copy(struct c
extern const unsigned long
cpu_bit_bitmap[BITS_PER_LONG+1][BITS_TO_LONGS(NR_CPUS)];
-static inline const cpumask_t *get_cpu_mask(unsigned int cpu)
+static inline const struct cpumask *cpumask_of(unsigned int cpu)
{
const unsigned long *p = cpu_bit_bitmap[1 + cpu % BITS_PER_LONG];
p -= cpu / BITS_PER_LONG;
- return (const cpumask_t *)p;
+ return (const struct cpumask *)p;
}
-/*
- * In cases where we take the address of the cpumask immediately,
- * gcc optimizes it out (it's a constant) and there's no huge stack
- * variable created:
- */
-#define cpumask_of_cpu(cpu) (*get_cpu_mask(cpu))
-
-
#define CPU_MASK_LAST_WORD BITMAP_LAST_WORD_MASK(NR_CPUS)
#if NR_CPUS <= BITS_PER_LONG
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 21/35] cpumask: for_each_cpu(): for_each_cpu_mask which takes a pointer
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (19 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 20/35] cpumask: cpumask_of(): cpumask_of_cpu() which returns a pointer Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 22/35] cpumask: cpumask_first/cpumask_next Mike Travis
` (13 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
We want to wean people off handing around cpumask_t's, and have them
pass by pointer instead. This does for_each_cpu_mask().
We immediately convert core files who were doing
"for_each_cpu_mask(... *mask)" since this is clearer.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 27 +++++++++++++++------------
kernel/sched.c | 40 ++++++++++++++++++++--------------------
kernel/workqueue.c | 6 +++---
lib/cpumask.c | 2 +-
mm/allocpercpu.c | 4 ++--
mm/vmstat.c | 2 +-
6 files changed, 42 insertions(+), 39 deletions(-)
--- linux-2.6.28.orig/include/linux/cpumask.h
+++ linux-2.6.28/include/linux/cpumask.h
@@ -97,9 +97,9 @@
* void cpumask_onto(dst, orig, relmap) *dst = orig relative to relmap
* void cpumask_fold(dst, orig, sz) dst bits = orig bits mod sz
*
- * for_each_cpu_mask(cpu, mask) for-loop cpu over mask
+ * for_each_cpu(cpu, mask) for-loop cpu over mask
* using nr_cpumask_bits
- * for_each_cpu_mask_and(cpu, mask, and) for-loop cpu over (mask & and).
+ * for_each_cpu_and(cpu, mask, and) for-loop cpu over (mask & and).
*
* int num_online_cpus() Number of online CPUs
* int num_possible_cpus() Number of all possible CPUs
@@ -176,6 +176,9 @@ extern cpumask_t _unused_cpumask_arg_;
#define cpus_weight_nr(cpumask) cpus_weight(cpumask)
#define for_each_cpu_mask_nr(cpu, mask) for_each_cpu_mask(cpu, mask)
#define cpumask_of_cpu(cpu) (*cpumask_of(cpu))
+#define for_each_cpu_mask(cpu, mask) for_each_cpu(cpu, &(mask))
+#define for_each_cpu_mask_and(cpu, mask, and) \
+ for_each_cpu_and(cpu, &(mask), &(and))
/* End deprecated region. */
#if NR_CPUS <= BITS_PER_LONG
@@ -439,9 +442,9 @@ extern cpumask_t cpu_mask_all;
#define next_cpu(n, src) ({ (void)(src); 1; })
#define any_online_cpu(mask) 0
-#define for_each_cpu_mask(cpu, mask) \
+#define for_each_cpu(cpu, mask) \
for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask)
-#define for_each_cpu_mask_and(cpu, mask, and) \
+#define for_each_cpu_and(cpu, mask, and) \
for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask, (void)and)
#else /* NR_CPUS > 1 */
@@ -455,13 +458,13 @@ int __any_online_cpu(const cpumask_t *ma
#define next_cpu(n, src) __next_cpu((n), &(src))
#define any_online_cpu(mask) __any_online_cpu(&(mask))
-#define for_each_cpu_mask(cpu, mask) \
+#define for_each_cpu(cpu, mask) \
for ((cpu) = -1; \
- (cpu) = next_cpu((cpu), (mask)), \
+ (cpu) = __next_cpu((cpu), (mask)), \
(cpu) < nr_cpumask_bits;)
-#define for_each_cpu_mask_and(cpu, mask, and) \
- for ((cpu) = -1; \
- (cpu) = cpumask_next_and((cpu), &(mask), &(and)), \
+#define for_each_cpu_and(cpu, mask, and) \
+ for ((cpu) = -1; \
+ (cpu) = cpumask_next_and((cpu), (mask), (and)), \
(cpu) < nr_cpumask_bits;)
#endif
@@ -615,9 +618,9 @@ extern cpumask_t cpu_active_map;
#define cpu_is_offline(cpu) unlikely(!cpu_online(cpu))
-#define for_each_possible_cpu(cpu) for_each_cpu_mask((cpu), cpu_possible_map)
-#define for_each_online_cpu(cpu) for_each_cpu_mask((cpu), cpu_online_map)
-#define for_each_present_cpu(cpu) for_each_cpu_mask((cpu), cpu_present_map)
+#define for_each_possible_cpu(cpu) for_each_cpu((cpu), &cpu_possible_map)
+#define for_each_online_cpu(cpu) for_each_cpu((cpu), &cpu_online_map)
+#define for_each_present_cpu(cpu) for_each_cpu((cpu), &cpu_present_map)
/* Arch-specific code may call this to initialize nr_cpu_ids based on
* the cpu_possible_map. */
--- linux-2.6.28.orig/kernel/sched.c
+++ linux-2.6.28/kernel/sched.c
@@ -1512,7 +1512,7 @@ static int tg_shares_up(struct task_grou
struct sched_domain *sd = data;
int i;
- for_each_cpu_mask(i, sd->span) {
+ for_each_cpu(i, &sd->span) {
rq_weight += tg->cfs_rq[i]->load.weight;
shares += tg->cfs_rq[i]->shares;
}
@@ -1526,7 +1526,7 @@ static int tg_shares_up(struct task_grou
if (!rq_weight)
rq_weight = cpus_weight(sd->span) * NICE_0_LOAD;
- for_each_cpu_mask(i, sd->span) {
+ for_each_cpu(i, &sd->span) {
struct rq *rq = cpu_rq(i);
unsigned long flags;
@@ -2069,7 +2069,7 @@ find_idlest_group(struct sched_domain *s
/* Tally up the load of all CPUs in the group */
avg_load = 0;
- for_each_cpu_mask_nr(i, group->cpumask) {
+ for_each_cpu(i, &group->cpumask) {
/* Bias balancing toward cpus of our domain */
if (local_group)
load = source_load(i, load_idx);
@@ -2111,7 +2111,7 @@ find_idlest_cpu(struct sched_group *grou
/* Traverse only the allowed CPUs */
cpus_and(*tmp, group->cpumask, p->cpus_allowed);
- for_each_cpu_mask_nr(i, *tmp) {
+ for_each_cpu(i, tmp) {
load = weighted_cpuload(i);
if (load < min_load || (load == min_load && i == this_cpu)) {
@@ -3129,7 +3129,7 @@ find_busiest_group(struct sched_domain *
max_cpu_load = 0;
min_cpu_load = ~0UL;
- for_each_cpu_mask_nr(i, group->cpumask) {
+ for_each_cpu(i, &group->cpumask) {
struct rq *rq;
if (!cpu_isset(i, *cpus))
@@ -3408,7 +3408,7 @@ find_busiest_queue(struct sched_group *g
unsigned long max_load = 0;
int i;
- for_each_cpu_mask_nr(i, group->cpumask) {
+ for_each_cpu(i, &group->cpumask) {
unsigned long wl;
if (!cpu_isset(i, *cpus))
@@ -3950,7 +3950,7 @@ static void run_rebalance_domains(struct
int balance_cpu;
cpu_clear(this_cpu, cpus);
- for_each_cpu_mask_nr(balance_cpu, cpus) {
+ for_each_cpu(balance_cpu, &cpus) {
/*
* If this cpu gets work to do, stop the load balancing
* work being done for other cpus. Next load
@@ -6934,7 +6934,7 @@ init_sched_build_groups(const cpumask_t
cpus_clear(*covered);
- for_each_cpu_mask_nr(i, *span) {
+ for_each_cpu(i, span) {
struct sched_group *sg;
int group = group_fn(i, cpu_map, &sg, tmpmask);
int j;
@@ -6945,7 +6945,7 @@ init_sched_build_groups(const cpumask_t
cpus_clear(sg->cpumask);
sg->__cpu_power = 0;
- for_each_cpu_mask_nr(j, *span) {
+ for_each_cpu(j, span) {
if (group_fn(j, cpu_map, NULL, tmpmask) != group)
continue;
@@ -7145,7 +7145,7 @@ static void init_numa_sched_groups_power
if (!sg)
return;
do {
- for_each_cpu_mask_nr(j, sg->cpumask) {
+ for_each_cpu(j, &sg->cpumask) {
struct sched_domain *sd;
sd = &per_cpu(phys_domains, j);
@@ -7170,7 +7170,7 @@ static void free_sched_groups(const cpum
{
int cpu, i;
- for_each_cpu_mask_nr(cpu, *cpu_map) {
+ for_each_cpu(cpu, cpu_map) {
struct sched_group **sched_group_nodes
= sched_group_nodes_bycpu[cpu];
@@ -7417,7 +7417,7 @@ static int __build_sched_domains(const c
/*
* Set up domains for cpus specified by the cpu_map.
*/
- for_each_cpu_mask_nr(i, *cpu_map) {
+ for_each_cpu(i, cpu_map) {
struct sched_domain *sd = NULL, *p;
SCHED_CPUMASK_VAR(nodemask, allmasks);
@@ -7484,7 +7484,7 @@ static int __build_sched_domains(const c
#ifdef CONFIG_SCHED_SMT
/* Set up CPU (sibling) groups */
- for_each_cpu_mask_nr(i, *cpu_map) {
+ for_each_cpu(i, cpu_map) {
SCHED_CPUMASK_VAR(this_sibling_map, allmasks);
SCHED_CPUMASK_VAR(send_covered, allmasks);
@@ -7501,7 +7501,7 @@ static int __build_sched_domains(const c
#ifdef CONFIG_SCHED_MC
/* Set up multi-core groups */
- for_each_cpu_mask_nr(i, *cpu_map) {
+ for_each_cpu(i, cpu_map) {
SCHED_CPUMASK_VAR(this_core_map, allmasks);
SCHED_CPUMASK_VAR(send_covered, allmasks);
@@ -7568,7 +7568,7 @@ static int __build_sched_domains(const c
goto error;
}
sched_group_nodes[i] = sg;
- for_each_cpu_mask_nr(j, *nodemask) {
+ for_each_cpu(j, nodemask) {
struct sched_domain *sd;
sd = &per_cpu(node_domains, j);
@@ -7614,21 +7614,21 @@ static int __build_sched_domains(const c
/* Calculate CPU power for physical packages and nodes */
#ifdef CONFIG_SCHED_SMT
- for_each_cpu_mask_nr(i, *cpu_map) {
+ for_each_cpu(i, cpu_map) {
struct sched_domain *sd = &per_cpu(cpu_domains, i);
init_sched_groups_power(i, sd);
}
#endif
#ifdef CONFIG_SCHED_MC
- for_each_cpu_mask_nr(i, *cpu_map) {
+ for_each_cpu(i, cpu_map) {
struct sched_domain *sd = &per_cpu(core_domains, i);
init_sched_groups_power(i, sd);
}
#endif
- for_each_cpu_mask_nr(i, *cpu_map) {
+ for_each_cpu(i, cpu_map) {
struct sched_domain *sd = &per_cpu(phys_domains, i);
init_sched_groups_power(i, sd);
@@ -7648,7 +7648,7 @@ static int __build_sched_domains(const c
#endif
/* Attach the domains */
- for_each_cpu_mask_nr(i, *cpu_map) {
+ for_each_cpu(i, cpu_map) {
struct sched_domain *sd;
#ifdef CONFIG_SCHED_SMT
sd = &per_cpu(cpu_domains, i);
@@ -7731,7 +7731,7 @@ static void detach_destroy_domains(const
unregister_sched_domain_sysctl();
- for_each_cpu_mask_nr(i, *cpu_map)
+ for_each_cpu(i, cpu_map)
cpu_attach_domain(NULL, &def_root_domain, i);
synchronize_sched();
arch_destroy_sched_domains(cpu_map, &tmpmask);
--- linux-2.6.28.orig/kernel/workqueue.c
+++ linux-2.6.28/kernel/workqueue.c
@@ -415,7 +415,7 @@ void flush_workqueue(struct workqueue_st
might_sleep();
lock_map_acquire(&wq->lockdep_map);
lock_map_release(&wq->lockdep_map);
- for_each_cpu_mask_nr(cpu, *cpu_map)
+ for_each_cpu(cpu, cpu_map)
flush_cpu_workqueue(per_cpu_ptr(wq->cpu_wq, cpu));
}
EXPORT_SYMBOL_GPL(flush_workqueue);
@@ -546,7 +546,7 @@ static void wait_on_work(struct work_str
wq = cwq->wq;
cpu_map = wq_cpu_map(wq);
- for_each_cpu_mask_nr(cpu, *cpu_map)
+ for_each_cpu(cpu, cpu_map)
wait_on_cpu_work(per_cpu_ptr(wq->cpu_wq, cpu), work);
}
@@ -906,7 +906,7 @@ void destroy_workqueue(struct workqueue_
list_del(&wq->list);
spin_unlock(&workqueue_lock);
- for_each_cpu_mask_nr(cpu, *cpu_map)
+ for_each_cpu(cpu, cpu_map)
cleanup_workqueue_thread(per_cpu_ptr(wq->cpu_wq, cpu));
cpu_maps_update_done();
--- linux-2.6.28.orig/lib/cpumask.c
+++ linux-2.6.28/lib/cpumask.c
@@ -28,7 +28,7 @@ int __any_online_cpu(const cpumask_t *ma
{
int cpu;
- for_each_cpu_mask(cpu, *mask) {
+ for_each_cpu(cpu, mask) {
if (cpu_online(cpu))
break;
}
--- linux-2.6.28.orig/mm/allocpercpu.c
+++ linux-2.6.28/mm/allocpercpu.c
@@ -34,7 +34,7 @@ static void percpu_depopulate(void *__pd
static void __percpu_depopulate_mask(void *__pdata, cpumask_t *mask)
{
int cpu;
- for_each_cpu_mask_nr(cpu, *mask)
+ for_each_cpu(cpu, mask)
percpu_depopulate(__pdata, cpu);
}
@@ -86,7 +86,7 @@ static int __percpu_populate_mask(void *
int cpu;
cpus_clear(populated);
- for_each_cpu_mask_nr(cpu, *mask)
+ for_each_cpu(cpu, mask)
if (unlikely(!percpu_populate(__pdata, size, gfp, cpu))) {
__percpu_depopulate_mask(__pdata, &populated);
return -ENOMEM;
--- linux-2.6.28.orig/mm/vmstat.c
+++ linux-2.6.28/mm/vmstat.c
@@ -27,7 +27,7 @@ static void sum_vm_events(unsigned long
memset(ret, 0, NR_VM_EVENT_ITEMS * sizeof(unsigned long));
- for_each_cpu_mask_nr(cpu, *cpumask) {
+ for_each_cpu(cpu, cpumask) {
struct vm_event_state *this = &per_cpu(vm_event_states, cpu);
for (i = 0; i < NR_VM_EVENT_ITEMS; i++)
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 22/35] cpumask: cpumask_first/cpumask_next
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (20 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 21/35] cpumask: for_each_cpu(): for_each_cpu_mask which takes " Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 23/35] cpumask: deprecate any_online_cpu() in favour of cpumask_any/cpumask_any_and Mike Travis
` (12 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Pointer-taking variants of first_cpu/next_cpu.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 18 +++++++++---------
lib/cpumask.c | 10 +++++-----
2 files changed, 14 insertions(+), 14 deletions(-)
--- test-compile.orig/include/linux/cpumask.h
+++ test-compile/include/linux/cpumask.h
@@ -46,8 +46,8 @@
* void cpumask_shift_right(dst, src, n) Shift right
* void cpumask_shift_left(dst, src, n) Shift left
*
- * int first_cpu(mask) Number lowest set bit or nr_cpumask_bits
- * int next_cpu(cpu, mask) Next cpu past 'cpu', or nr_cpumask_bits
+ * int cpumask_first(mask) Number lowest set bit or nr_cpumask_bits
+ * int cpumask_next(cpu, mask) Next cpu past 'cpu', or nr_cpumask_bits
*
* void cpumask_copy(dmask, smask) dmask = smask
*
@@ -200,6 +200,8 @@ extern cpumask_t _unused_cpumask_arg_;
#define for_each_cpu_mask(cpu, mask) for_each_cpu(cpu, &(mask))
#define for_each_cpu_mask_and(cpu, mask, and) \
for_each_cpu_and(cpu, &(mask), &(and))
+#define first_cpu(src) cpumask_first(&(src))
+#define next_cpu(n, src) cpumask_next((n), &(src))
/* End deprecated region. */
static inline void cpumask_set_cpu(int cpu, volatile struct cpumask *dstp)
@@ -448,8 +450,8 @@ extern cpumask_t cpu_mask_all;
#if NR_CPUS == 1
-#define first_cpu(src) ({ (void)(src); 0; })
-#define next_cpu(n, src) ({ (void)(src); 1; })
+#define cpumask_first(src) ({ (void)(src); 0; })
+#define cpumask_next(n, src) ({ (void)(src); 1; })
#define cpumask_next_and(n, srcp, andp) ({ (void)(srcp), (void)(andp); 1; })
#define any_online_cpu(mask) 0
@@ -460,18 +462,16 @@ extern cpumask_t cpu_mask_all;
#else /* NR_CPUS > 1 */
-int __first_cpu(const cpumask_t *srcp);
-int __next_cpu(int n, const cpumask_t *srcp);
+int cpumask_first(const cpumask_t *srcp);
+int cpumask_next(int n, const cpumask_t *srcp);
int cpumask_next_and(int n, const cpumask_t *srcp, const cpumask_t *andp);
int __any_online_cpu(const cpumask_t *mask);
-#define first_cpu(src) __first_cpu(&(src))
-#define next_cpu(n, src) __next_cpu((n), &(src))
#define any_online_cpu(mask) __any_online_cpu(&(mask))
#define for_each_cpu(cpu, mask) \
for ((cpu) = -1; \
- (cpu) = __next_cpu((cpu), (mask)), \
+ (cpu) = cpumask_next((cpu), (mask)), \
(cpu) < nr_cpumask_bits;)
#define for_each_cpu_and(cpu, mask, and) \
for ((cpu) = -1; \
--- test-compile.orig/lib/cpumask.c
+++ test-compile/lib/cpumask.c
@@ -3,21 +3,21 @@
#include <linux/cpumask.h>
#include <linux/module.h>
-int __first_cpu(const cpumask_t *srcp)
+int cpumask_first(const cpumask_t *srcp)
{
return find_first_bit(cpumask_bits(srcp), nr_cpumask_bits);
}
-EXPORT_SYMBOL(__first_cpu);
+EXPORT_SYMBOL(cpumask_first);
-int __next_cpu(int n, const cpumask_t *srcp)
+int cpumask_next(int n, const cpumask_t *srcp)
{
return find_next_bit(cpumask_bits(srcp), nr_cpumask_bits, n+1);
}
-EXPORT_SYMBOL(__next_cpu);
+EXPORT_SYMBOL(cpumask_next);
int cpumask_next_and(int n, const cpumask_t *srcp, const cpumask_t *andp)
{
- while ((n = next_cpu(n, *srcp)) < nr_cpumask_bits)
+ while ((n = cpumask_next(n, srcp)) < nr_cpu_ids)
if (cpumask_test_cpu(n, andp))
break;
return n;
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 23/35] cpumask: deprecate any_online_cpu() in favour of cpumask_any/cpumask_any_and
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (21 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 22/35] cpumask: cpumask_first/cpumask_next Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 24/35] cpumask: cpumask_any_but() Mike Travis
` (11 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
any_online_cpu() is a good name, but it takes a cpumask_t, not a
pointer.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 11 ++++++-----
lib/cpumask.c | 12 ------------
2 files changed, 6 insertions(+), 17 deletions(-)
--- test-compile.orig/include/linux/cpumask.h
+++ test-compile/include/linux/cpumask.h
@@ -109,7 +109,8 @@
* int cpu_possible(cpu) Is some cpu possible?
* int cpu_present(cpu) Is some cpu present (can schedule)?
*
- * int any_online_cpu(mask) First online cpu in mask
+ * int cpumask_any(mask) Any cpu in mask
+ * int cpumask_any_and(mask1,mask2) Any cpu in both masks
*
* for_each_possible_cpu(cpu) for-loop cpu over cpu_possible_map
* for_each_online_cpu(cpu) for-loop cpu over cpu_online_map
@@ -202,6 +203,7 @@ extern cpumask_t _unused_cpumask_arg_;
for_each_cpu_and(cpu, &(mask), &(and))
#define first_cpu(src) cpumask_first(&(src))
#define next_cpu(n, src) cpumask_next((n), &(src))
+#define any_online_cpu(mask) cpumask_any_and(&(mask), &cpu_online_map)
/* End deprecated region. */
static inline void cpumask_set_cpu(int cpu, volatile struct cpumask *dstp)
@@ -387,6 +389,9 @@ static inline void cpumask_copy(struct c
bitmap_copy(cpumask_bits(dstp), cpumask_bits(srcp), nr_cpumask_bits);
}
+#define cpumask_any(srcp) cpumask_first(srcp)
+#define cpumask_any_and(mask1, mask2) cpumask_first_and((mask1), (mask2))
+
/*
* Special-case data structure for "single bit set only" constant CPU masks.
*
@@ -453,7 +458,6 @@ extern cpumask_t cpu_mask_all;
#define cpumask_first(src) ({ (void)(src); 0; })
#define cpumask_next(n, src) ({ (void)(src); 1; })
#define cpumask_next_and(n, srcp, andp) ({ (void)(srcp), (void)(andp); 1; })
-#define any_online_cpu(mask) 0
#define for_each_cpu(cpu, mask) \
for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask)
@@ -465,9 +469,6 @@ extern cpumask_t cpu_mask_all;
int cpumask_first(const cpumask_t *srcp);
int cpumask_next(int n, const cpumask_t *srcp);
int cpumask_next_and(int n, const cpumask_t *srcp, const cpumask_t *andp);
-int __any_online_cpu(const cpumask_t *mask);
-
-#define any_online_cpu(mask) __any_online_cpu(&(mask))
#define for_each_cpu(cpu, mask) \
for ((cpu) = -1; \
--- test-compile.orig/lib/cpumask.c
+++ test-compile/lib/cpumask.c
@@ -24,18 +24,6 @@ int cpumask_next_and(int n, const cpumas
}
EXPORT_SYMBOL(cpumask_next_and);
-int __any_online_cpu(const cpumask_t *mask)
-{
- int cpu;
-
- for_each_cpu(cpu, mask) {
- if (cpu_online(cpu))
- break;
- }
- return cpu;
-}
-EXPORT_SYMBOL(__any_online_cpu);
-
/* These are not inline because of header tangles. */
#ifdef CONFIG_CPUMASK_OFFSTACK
bool alloc_cpumask_var(cpumask_var_t *mask, gfp_t flags)
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 24/35] cpumask: cpumask_any_but()
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (22 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 23/35] cpumask: deprecate any_online_cpu() in favour of cpumask_any/cpumask_any_and Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 25/35] cpumask: Deprecate CPUMASK_ALLOC etc in favor of cpumask_var_t Mike Travis
` (10 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
There's a common case where we want any online cpu except a particular
one. This creates a helper to do that, otherwise we need a temp var
and cpumask_andnot().
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 3 +++
lib/cpumask.c | 10 ++++++++++
2 files changed, 13 insertions(+)
--- test-compile.orig/include/linux/cpumask.h
+++ test-compile/include/linux/cpumask.h
@@ -111,6 +111,7 @@
*
* int cpumask_any(mask) Any cpu in mask
* int cpumask_any_and(mask1,mask2) Any cpu in both masks
+ * int cpumask_any_but(mask,cpu) Any cpu in mask except cpu
*
* for_each_possible_cpu(cpu) for-loop cpu over cpu_possible_map
* for_each_online_cpu(cpu) for-loop cpu over cpu_online_map
@@ -458,6 +459,7 @@ extern cpumask_t cpu_mask_all;
#define cpumask_first(src) ({ (void)(src); 0; })
#define cpumask_next(n, src) ({ (void)(src); 1; })
#define cpumask_next_and(n, srcp, andp) ({ (void)(srcp), (void)(andp); 1; })
+#define cpumask_any_but(mask, cpu) ({ (void)(mask); (void)(cpu); 0; })
#define for_each_cpu(cpu, mask) \
for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask)
@@ -469,6 +471,7 @@ extern cpumask_t cpu_mask_all;
int cpumask_first(const cpumask_t *srcp);
int cpumask_next(int n, const cpumask_t *srcp);
int cpumask_next_and(int n, const cpumask_t *srcp, const cpumask_t *andp);
+int cpumask_any_but(const struct cpumask *mask, unsigned int cpu);
#define for_each_cpu(cpu, mask) \
for ((cpu) = -1; \
--- test-compile.orig/lib/cpumask.c
+++ test-compile/lib/cpumask.c
@@ -24,6 +24,16 @@ int cpumask_next_and(int n, const cpumas
}
EXPORT_SYMBOL(cpumask_next_and);
+int cpumask_any_but(const struct cpumask *mask, unsigned int cpu)
+{
+ unsigned int i;
+
+ for_each_cpu(i, mask)
+ if (i != cpu)
+ break;
+ return i;
+}
+
/* These are not inline because of header tangles. */
#ifdef CONFIG_CPUMASK_OFFSTACK
bool alloc_cpumask_var(cpumask_var_t *mask, gfp_t flags)
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 25/35] cpumask: Deprecate CPUMASK_ALLOC etc in favor of cpumask_var_t.
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (23 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 24/35] cpumask: cpumask_any_but() Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 26/35] cpumask: get rid of boutique sched.c allocations, use cpumask_var_t Mike Travis
` (9 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Remove CPUMASK_ALLOC() in favor of alloc_cpumask_var().
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 46 ++++++++--------------------------------------
1 file changed, 8 insertions(+), 38 deletions(-)
--- linux-2.6.28.orig/include/linux/cpumask.h
+++ linux-2.6.28/include/linux/cpumask.h
@@ -57,35 +57,6 @@
* CPU_MASK_NONE Initializer - no bits set
* unsigned long *cpumask_bits(mask) Array of unsigned long's in mask
*
- * CPUMASK_ALLOC kmalloc's a structure that is a composite of many cpumask_t
- * variables, and CPUMASK_PTR provides pointers to each field.
- *
- * The structure should be defined something like this:
- * struct my_cpumasks {
- * cpumask_t mask1;
- * cpumask_t mask2;
- * };
- *
- * Usage is then:
- * CPUMASK_ALLOC(my_cpumasks);
- * CPUMASK_PTR(mask1, my_cpumasks);
- * CPUMASK_PTR(mask2, my_cpumasks);
- *
- * --- DO NOT reference cpumask_t pointers until this check ---
- * if (my_cpumasks == NULL)
- * "kmalloc failed"...
- *
- * References are now pointers to the cpumask_t variables (*mask1, ...)
- *
- *if NR_CPUS > BITS_PER_LONG
- * CPUMASK_ALLOC(m) Declares and allocates struct m *m =
- * kmalloc(sizeof(*m), GFP_KERNEL)
- * CPUMASK_FREE(m) Macro for kfree(m)
- *else
- * CPUMASK_ALLOC(m) Declares struct m _m, *m = &_m
- * CPUMASK_FREE(m) Nop
- *endif
- * CPUMASK_PTR(v, m) Declares cpumask_t *v = &(m->v)
* ------------------------------------------------------------------------
*
* int cpumask_scnprintf(buf, len, mask) Format cpumask for printing
@@ -184,6 +155,14 @@ extern cpumask_t _unused_cpumask_arg_;
#define first_cpu(src) cpumask_first(&(src))
#define next_cpu(n, src) cpumask_next((n), &(src))
#define any_online_cpu(mask) cpumask_any_and(&(mask), &cpu_online_map)
+#if NR_CPUS > BITS_PER_LONG
+#define CPUMASK_ALLOC(m) struct m *m = kmalloc(sizeof(*m), GFP_KERNEL)
+#define CPUMASK_FREE(m) kfree(m)
+#else
+#define CPUMASK_ALLOC(m) struct m _m, *m = &_m
+#define CPUMASK_FREE(m)
+#endif
+#define CPUMASK_PTR(v, m) cpumask_t *v = &(m->v)
/* End deprecated region. */
#if NR_CPUS <= BITS_PER_LONG
@@ -435,15 +414,6 @@ extern cpumask_t cpu_mask_all;
[0] = 1UL \
} }
-#if NR_CPUS > BITS_PER_LONG
-#define CPUMASK_ALLOC(m) struct m *m = kmalloc(sizeof(*m), GFP_KERNEL)
-#define CPUMASK_FREE(m) kfree(m)
-#else
-#define CPUMASK_ALLOC(m) struct m _m, *m = &_m
-#define CPUMASK_FREE(m)
-#endif
-#define CPUMASK_PTR(v, m) cpumask_t *v = &(m->v)
-
#if NR_CPUS == 1
#define cpumask_first(src) ({ (void)(src); 0; })
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 26/35] cpumask: get rid of boutique sched.c allocations, use cpumask_var_t.
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (24 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 25/35] cpumask: Deprecate CPUMASK_ALLOC etc in favor of cpumask_var_t Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 27/35] cpumask: to_cpumask() Mike Travis
` (8 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Using lots of allocs rather than one big alloc is less efficient, but
who cares for this setup function?
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
kernel/sched.c | 133 ++++++++++++++++++++++++---------------------------------
1 file changed, 56 insertions(+), 77 deletions(-)
--- linux-2.6.28.orig/kernel/sched.c
+++ linux-2.6.28/kernel/sched.c
@@ -7291,40 +7291,6 @@ SD_INIT_FUNC(CPU)
SD_INIT_FUNC(MC)
#endif
-/*
- * To minimize stack usage kmalloc room for cpumasks and share the
- * space as the usage in build_sched_domains() dictates. Used only
- * if the amount of space is significant.
- */
-struct allmasks {
- cpumask_t tmpmask; /* make this one first */
- union {
- cpumask_t nodemask;
- cpumask_t this_sibling_map;
- cpumask_t this_core_map;
- };
- cpumask_t send_covered;
-
-#ifdef CONFIG_NUMA
- cpumask_t domainspan;
- cpumask_t covered;
- cpumask_t notcovered;
-#endif
-};
-
-#if NR_CPUS > 128
-#define SCHED_CPUMASK_ALLOC 1
-#define SCHED_CPUMASK_FREE(v) kfree(v)
-#define SCHED_CPUMASK_DECLARE(v) struct allmasks *v
-#else
-#define SCHED_CPUMASK_ALLOC 0
-#define SCHED_CPUMASK_FREE(v)
-#define SCHED_CPUMASK_DECLARE(v) struct allmasks _v, *v = &_v
-#endif
-
-#define SCHED_CPUMASK_VAR(v, a) cpumask_t *v = (cpumask_t *) \
- ((unsigned long)(a) + offsetof(struct allmasks, v))
-
static int default_relax_domain_level = -1;
static int __init setup_relax_domain_level(char *str)
@@ -7367,14 +7333,35 @@ static void set_domain_attribute(struct
static int __build_sched_domains(const cpumask_t *cpu_map,
struct sched_domain_attr *attr)
{
- int i;
+ int i, err = -ENOMEM;
struct root_domain *rd;
- SCHED_CPUMASK_DECLARE(allmasks);
- cpumask_t *tmpmask;
+ cpumask_var_t nodemask, this_sibling_map, this_core_map, send_covered,
+ tmpmask;
#ifdef CONFIG_NUMA
+ cpumask_var_t domainspan, covered, notcovered;
struct sched_group **sched_group_nodes = NULL;
int sd_allnodes = 0;
+ if (!alloc_cpumask_var(&domainspan, GFP_KERNEL))
+ goto out;
+ if (!alloc_cpumask_var(&covered, GFP_KERNEL))
+ goto free_domainspan;
+ if (!alloc_cpumask_var(¬covered, GFP_KERNEL))
+ goto free_covered;
+#endif
+
+ if (!alloc_cpumask_var(&nodemask, GFP_KERNEL))
+ goto free_notcovered;
+ if (!alloc_cpumask_var(&this_sibling_map, GFP_KERNEL))
+ goto free_nodemask;
+ if (!alloc_cpumask_var(&this_core_map, GFP_KERNEL))
+ goto free_this_sibling_map;
+ if (!alloc_cpumask_var(&send_covered, GFP_KERNEL))
+ goto free_this_core_map;
+ if (!alloc_cpumask_var(&tmpmask, GFP_KERNEL))
+ goto free_send_covered;
+
+#ifdef CONFIG_NUMA
/*
* Allocate the per-node list of sched groups
*/
@@ -7382,34 +7369,16 @@ static int __build_sched_domains(const c
GFP_KERNEL);
if (!sched_group_nodes) {
printk(KERN_WARNING "Can not alloc sched group node list\n");
- return -ENOMEM;
+ goto free_tmpmask;
}
#endif
rd = alloc_rootdomain();
if (!rd) {
printk(KERN_WARNING "Cannot alloc root domain\n");
-#ifdef CONFIG_NUMA
- kfree(sched_group_nodes);
-#endif
- return -ENOMEM;
+ goto free_sched_groups;
}
-#if SCHED_CPUMASK_ALLOC
- /* get space for all scratch cpumask variables */
- allmasks = kmalloc(sizeof(*allmasks), GFP_KERNEL);
- if (!allmasks) {
- printk(KERN_WARNING "Cannot alloc cpumask array\n");
- kfree(rd);
-#ifdef CONFIG_NUMA
- kfree(sched_group_nodes);
-#endif
- return -ENOMEM;
- }
-#endif
- tmpmask = (cpumask_t *)allmasks;
-
-
#ifdef CONFIG_NUMA
sched_group_nodes_bycpu[first_cpu(*cpu_map)] = sched_group_nodes;
#endif
@@ -7419,7 +7388,6 @@ static int __build_sched_domains(const c
*/
for_each_cpu(i, cpu_map) {
struct sched_domain *sd = NULL, *p;
- SCHED_CPUMASK_VAR(nodemask, allmasks);
*nodemask = node_to_cpumask(cpu_to_node(i));
cpus_and(*nodemask, *nodemask, *cpu_map);
@@ -7485,9 +7453,6 @@ static int __build_sched_domains(const c
#ifdef CONFIG_SCHED_SMT
/* Set up CPU (sibling) groups */
for_each_cpu(i, cpu_map) {
- SCHED_CPUMASK_VAR(this_sibling_map, allmasks);
- SCHED_CPUMASK_VAR(send_covered, allmasks);
-
*this_sibling_map = per_cpu(cpu_sibling_map, i);
cpus_and(*this_sibling_map, *this_sibling_map, *cpu_map);
if (i != first_cpu(*this_sibling_map))
@@ -7502,9 +7467,6 @@ static int __build_sched_domains(const c
#ifdef CONFIG_SCHED_MC
/* Set up multi-core groups */
for_each_cpu(i, cpu_map) {
- SCHED_CPUMASK_VAR(this_core_map, allmasks);
- SCHED_CPUMASK_VAR(send_covered, allmasks);
-
*this_core_map = cpu_coregroup_map(i);
cpus_and(*this_core_map, *this_core_map, *cpu_map);
if (i != first_cpu(*this_core_map))
@@ -7518,9 +7480,6 @@ static int __build_sched_domains(const c
/* Set up physical groups */
for (i = 0; i < nr_node_ids; i++) {
- SCHED_CPUMASK_VAR(nodemask, allmasks);
- SCHED_CPUMASK_VAR(send_covered, allmasks);
-
*nodemask = node_to_cpumask(i);
cpus_and(*nodemask, *nodemask, *cpu_map);
if (cpus_empty(*nodemask))
@@ -7534,8 +7493,6 @@ static int __build_sched_domains(const c
#ifdef CONFIG_NUMA
/* Set up node groups */
if (sd_allnodes) {
- SCHED_CPUMASK_VAR(send_covered, allmasks);
-
init_sched_build_groups(cpu_map, cpu_map,
&cpu_to_allnodes_group,
send_covered, tmpmask);
@@ -7544,9 +7501,6 @@ static int __build_sched_domains(const c
for (i = 0; i < nr_node_ids; i++) {
/* Set up node groups */
struct sched_group *sg, *prev;
- SCHED_CPUMASK_VAR(nodemask, allmasks);
- SCHED_CPUMASK_VAR(domainspan, allmasks);
- SCHED_CPUMASK_VAR(covered, allmasks);
int j;
*nodemask = node_to_cpumask(i);
@@ -7581,7 +7535,6 @@ static int __build_sched_domains(const c
prev = sg;
for (j = 0; j < nr_node_ids; j++) {
- SCHED_CPUMASK_VAR(notcovered, allmasks);
int n = (i + j) % nr_node_ids;
node_to_cpumask_ptr(pnodemask, n);
@@ -7660,14 +7613,40 @@ static int __build_sched_domains(const c
cpu_attach_domain(sd, rd, i);
}
- SCHED_CPUMASK_FREE((void *)allmasks);
- return 0;
+ err = 0;
+
+free_tmpmask:
+ free_cpumask_var(tmpmask);
+free_send_covered:
+ free_cpumask_var(send_covered);
+free_this_core_map:
+ free_cpumask_var(this_core_map);
+free_this_sibling_map:
+ free_cpumask_var(this_sibling_map);
+free_nodemask:
+ free_cpumask_var(nodemask);
+free_notcovered:
+#ifdef CONFIG_NUMA
+ free_cpumask_var(notcovered);
+free_covered:
+ free_cpumask_var(covered);
+free_domainspan:
+ free_cpumask_var(domainspan);
+out:
+#endif
+ return err;
+
+free_sched_groups:
+#ifdef CONFIG_NUMA
+ kfree(sched_group_nodes);
+#endif
+ goto free_tmpmask;
#ifdef CONFIG_NUMA
error:
free_sched_groups(cpu_map, tmpmask);
- SCHED_CPUMASK_FREE((void *)allmasks);
- return -ENOMEM;
+ kfree(rd);
+ goto free_tmpmask;
#endif
}
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 27/35] cpumask: to_cpumask()
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (25 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 26/35] cpumask: get rid of boutique sched.c allocations, use cpumask_var_t Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 28/35] cpumask: accessors to manipulate possible/present/online/active maps Mike Travis
` (7 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
There are times when we really want a static cpumask. Yet we don't
want to expose the struct cpumask definition, to avoid casual on-stack
usage and assignment.
So this macro allows you to do DECLARE_BITMAP(map, CONFIG_NR_CPUS); then use
to_cpumask() to turn it into a cpumask as needed.
Ugly? Yes, but as we move to fewer static cpumasks these calls vanish.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 14 ++++++++++++++
1 file changed, 14 insertions(+)
--- linux-2.6.28.orig/include/linux/cpumask.h
+++ linux-2.6.28/include/linux/cpumask.h
@@ -57,6 +57,9 @@
* CPU_MASK_NONE Initializer - no bits set
* unsigned long *cpumask_bits(mask) Array of unsigned long's in mask
*
+ * struct cpumask *to_cpumask(const unsigned long[])
+ * Convert a bitmap to a cpumask.
+ *
* ------------------------------------------------------------------------
*
* int cpumask_scnprintf(buf, len, mask) Format cpumask for printing
@@ -362,6 +365,17 @@ static inline void cpumask_copy(struct c
#define cpumask_any(srcp) cpumask_first(srcp)
#define cpumask_any_and(mask1, mask2) cpumask_first_and((mask1), (mask2))
+/* Used for static bitmaps of CONFIG_NR_CPUS bits. Must be a constant to use
+ * as an initializer. */
+#define to_cpumask(bitmap) \
+ ((struct cpumask *)(1 ? (bitmap) \
+ : (void *)sizeof(__check_is_bitmap(bitmap))))
+
+static inline int __check_is_bitmap(const unsigned long *bitmap)
+{
+ return 1;
+}
+
/*
* Special-case data structure for "single bit set only" constant CPU masks.
*
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 28/35] cpumask: accessors to manipulate possible/present/online/active maps
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (26 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 27/35] cpumask: to_cpumask() Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 29/35] cpumask: Use accessors code Mike Travis
` (6 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Since we are moving to const cpumask pointers for the maps, we need a
way to legitimately access them. Simple accessors work well.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 9 +++++++++
kernel/cpu.c | 42 +++++++++++++++++++++++++++++++++++++++++-
2 files changed, 50 insertions(+), 1 deletion(-)
--- linux-2.6.28.orig/include/linux/cpumask.h
+++ linux-2.6.28/include/linux/cpumask.h
@@ -604,6 +604,15 @@ extern cpumask_t cpu_active_map;
#define cpu_active(cpu) ((cpu) == 0)
#endif
+/* Wrappers to manipulate otherwise-constant masks. */
+void set_cpu_possible(unsigned int cpu, bool possible);
+void set_cpu_present(unsigned int cpu, bool present);
+void set_cpu_online(unsigned int cpu, bool online);
+void set_cpu_active(unsigned int cpu, bool active);
+void init_cpu_present(const struct cpumask *src);
+void init_cpu_possible(const struct cpumask *src);
+void init_cpu_online(const struct cpumask *src);
+
#define cpu_is_offline(cpu) unlikely(!cpu_online(cpu))
#define for_each_possible_cpu(cpu) for_each_cpu((cpu), &cpu_possible_map)
--- linux-2.6.28.orig/kernel/cpu.c
+++ linux-2.6.28/kernel/cpu.c
@@ -490,7 +490,6 @@ void notify_cpu_starting(unsigned int cp
#define MASK_DECLARE_8(x) MASK_DECLARE_4(x), MASK_DECLARE_4(x+4)
const unsigned long cpu_bit_bitmap[BITS_PER_LONG+1][BITS_TO_LONGS(NR_CPUS)] = {
-
MASK_DECLARE_8(0), MASK_DECLARE_8(8),
MASK_DECLARE_8(16), MASK_DECLARE_8(24),
#if BITS_PER_LONG > 32
@@ -499,3 +498,44 @@ const unsigned long cpu_bit_bitmap[BITS_
#endif
};
EXPORT_SYMBOL_GPL(cpu_bit_bitmap);
+
+void set_cpu_possible(unsigned int cpu, bool possible)
+{
+ if (possible)
+ cpumask_set_cpu(cpu, &cpu_possible_map);
+ else
+ cpumask_clear_cpu(cpu, &cpu_possible_map);
+}
+void set_cpu_present(unsigned int cpu, bool present)
+{
+ if (present)
+ cpumask_set_cpu(cpu, &cpu_present_map);
+ else
+ cpumask_clear_cpu(cpu, &cpu_present_map);
+}
+void set_cpu_online(unsigned int cpu, bool online)
+{
+ if (online)
+ cpumask_set_cpu(cpu, &cpu_online_map);
+ else
+ cpumask_clear_cpu(cpu, &cpu_online_map);
+}
+void set_cpu_active(unsigned int cpu, bool active)
+{
+ if (active)
+ cpumask_set_cpu(cpu, &cpu_active_map);
+ else
+ cpumask_clear_cpu(cpu, &cpu_active_map);
+}
+void init_cpu_present(const struct cpumask *src)
+{
+ cpumask_copy(&cpu_present_map, src);
+}
+void init_cpu_possible(const struct cpumask *src)
+{
+ cpumask_copy(&cpu_possible_map, src);
+}
+void init_cpu_online(const struct cpumask *src)
+{
+ cpumask_copy(&cpu_online_map, src);
+}
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 29/35] cpumask: Use accessors code.
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (27 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 28/35] cpumask: accessors to manipulate possible/present/online/active maps Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 30/35] cpumask: CONFIG_BITS_ALL, CONFIG_BITS_NONE and CONFIG_BITS_CPU0 Mike Travis
` (5 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Use the accessors rather than frobbing bits directly. Most of this is
in arch code I haven't even compiled, but is straightforward.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
arch/arm/mach-realview/platsmp.c | 2 +-
arch/cris/arch-v32/kernel/smp.c | 4 ++--
arch/ia64/kernel/acpi.c | 2 +-
arch/ia64/kernel/smpboot.c | 17 +++++++----------
arch/m32r/kernel/smpboot.c | 2 +-
arch/mips/kernel/smp-cmp.c | 6 ++++--
arch/mips/kernel/smp.c | 2 +-
arch/powerpc/kernel/setup-common.c | 4 ++--
arch/powerpc/platforms/powermac/setup.c | 2 +-
arch/s390/kernel/smp.c | 5 ++---
arch/sh/kernel/cpu/sh4a/smp-shx3.c | 5 ++---
arch/sh/kernel/smp.c | 6 +++---
arch/sparc64/kernel/mdesc.c | 2 +-
arch/sparc64/kernel/prom.c | 4 ++--
arch/um/kernel/smp.c | 2 +-
arch/x86/kernel/apic.c | 4 ++--
16 files changed, 33 insertions(+), 36 deletions(-)
--- linux-2.6.28.orig/arch/arm/mach-realview/platsmp.c
+++ linux-2.6.28/arch/arm/mach-realview/platsmp.c
@@ -193,7 +193,7 @@ void __init smp_init_cpus(void)
unsigned int i, ncores = get_core_count();
for (i = 0; i < ncores; i++)
- cpu_set(i, cpu_possible_map);
+ set_cpu_possible(i, true);
}
void __init smp_prepare_cpus(unsigned int max_cpus)
--- linux-2.6.28.orig/arch/cris/arch-v32/kernel/smp.c
+++ linux-2.6.28/arch/cris/arch-v32/kernel/smp.c
@@ -102,9 +102,9 @@ void __devinit smp_prepare_boot_cpu(void
SUPP_BANK_SEL(2);
SUPP_REG_WR(RW_MM_TLB_PGD, pgd);
- cpu_set(0, cpu_online_map);
+ set_cpu_online(0, true);
cpu_set(0, phys_cpu_present_map);
- cpu_set(0, cpu_possible_map);
+ set_cpu_possible(0, true);
}
void __init smp_cpus_done(unsigned int max_cpus)
--- linux-2.6.28.orig/arch/ia64/kernel/acpi.c
+++ linux-2.6.28/arch/ia64/kernel/acpi.c
@@ -845,7 +845,7 @@ __init void prefill_possible_map(void)
possible, max((possible - available_cpus), 0));
for (i = 0; i < possible; i++)
- cpu_set(i, cpu_possible_map);
+ set_cpu_possible(i, true);
}
int acpi_map_lsapic(acpi_handle handle, int *pcpu)
--- linux-2.6.28.orig/arch/ia64/kernel/smpboot.c
+++ linux-2.6.28/arch/ia64/kernel/smpboot.c
@@ -587,14 +587,14 @@ smp_build_cpu_map (void)
ia64_cpu_to_sapicid[0] = boot_cpu_id;
cpus_clear(cpu_present_map);
- cpu_set(0, cpu_present_map);
- cpu_set(0, cpu_possible_map);
+ set_cpu_present(0, true);
+ set_cpu_possible(0, true);
for (cpu = 1, i = 0; i < smp_boot_data.cpu_count; i++) {
sapicid = smp_boot_data.cpu_phys_id[i];
if (sapicid == boot_cpu_id)
continue;
- cpu_set(cpu, cpu_present_map);
- cpu_set(cpu, cpu_possible_map);
+ set_cpu_present(cpu, true);
+ set_cpu_possible(cpu, true);
ia64_cpu_to_sapicid[cpu] = sapicid;
cpu++;
}
@@ -632,12 +632,9 @@ smp_prepare_cpus (unsigned int max_cpus)
*/
if (!max_cpus) {
printk(KERN_INFO "SMP mode deactivated.\n");
- cpus_clear(cpu_online_map);
- cpus_clear(cpu_present_map);
- cpus_clear(cpu_possible_map);
- cpu_set(0, cpu_online_map);
- cpu_set(0, cpu_present_map);
- cpu_set(0, cpu_possible_map);
+ init_cpu_online(cpumask_of(0));
+ init_cpu_present(cpumask_of(0));
+ init_cpu_possible(cpumask_of(0));
return;
}
}
--- linux-2.6.28.orig/arch/m32r/kernel/smpboot.c
+++ linux-2.6.28/arch/m32r/kernel/smpboot.c
@@ -183,7 +183,7 @@ void __init smp_prepare_cpus(unsigned in
for (phys_id = 0 ; phys_id < nr_cpu ; phys_id++)
physid_set(phys_id, phys_cpu_present_map);
#ifndef CONFIG_HOTPLUG_CPU
- cpu_present_map = cpu_possible_map;
+ init_cpu_present(&cpu_possible_map);
#endif
show_mp_info(nr_cpu);
--- linux-2.6.28.orig/arch/mips/kernel/smp-cmp.c
+++ linux-2.6.28/arch/mips/kernel/smp-cmp.c
@@ -52,8 +52,10 @@ static int __init allowcpus(char *str)
cpus_clear(cpu_allow_map);
if (cpulist_parse(str, &cpu_allow_map) == 0) {
- cpu_set(0, cpu_allow_map);
- cpus_and(cpu_possible_map, cpu_possible_map, cpu_allow_map);
+ unsigned int i;
+ for (i = 1; i < nr_cpu_ids; i++)
+ if (!cpumask_test_cpu(i, cpu_allow_map))
+ set_cpu_possible(i, false);
len = cpulist_scnprintf(buf, sizeof(buf)-1, &cpu_possible_map);
buf[len] = '\0';
pr_debug("Allowable CPUs: %s\n", buf);
--- linux-2.6.28.orig/arch/mips/kernel/smp.c
+++ linux-2.6.28/arch/mips/kernel/smp.c
@@ -186,7 +186,7 @@ void __init smp_prepare_cpus(unsigned in
mp_ops->prepare_cpus(max_cpus);
set_cpu_sibling_map(0);
#ifndef CONFIG_HOTPLUG_CPU
- cpu_present_map = cpu_possible_map;
+ init_cpu_present(&cpu_possible_map);
#endif
}
--- linux-2.6.28.orig/arch/powerpc/kernel/setup-common.c
+++ linux-2.6.28/arch/powerpc/kernel/setup-common.c
@@ -413,7 +413,7 @@ void __init smp_setup_cpu_maps(void)
j, cpu, intserv[j]);
cpu_set(cpu, cpu_present_map);
set_hard_smp_processor_id(cpu, intserv[j]);
- cpu_set(cpu, cpu_possible_map);
+ set_cpu_possible(cpu, true);
cpu++;
}
}
@@ -459,7 +459,7 @@ void __init smp_setup_cpu_maps(void)
maxcpus);
for (cpu = 0; cpu < maxcpus; cpu++)
- cpu_set(cpu, cpu_possible_map);
+ set_cpu_possible(cpu, true);
out:
of_node_put(dn);
}
--- linux-2.6.28.orig/arch/powerpc/platforms/powermac/setup.c
+++ linux-2.6.28/arch/powerpc/platforms/powermac/setup.c
@@ -366,7 +366,7 @@ static void __init pmac_setup_arch(void)
int cpu;
for (cpu = 1; cpu < 4 && cpu < nr_cpu_ids; ++cpu)
- cpu_set(cpu, cpu_possible_map);
+ set_cpu_possible(cpu, true);
smp_ops = &psurge_smp_ops;
}
#endif
--- linux-2.6.28.orig/arch/s390/kernel/smp.c
+++ linux-2.6.28/arch/s390/kernel/smp.c
@@ -732,9 +732,8 @@ static int __init setup_possible_cpus(ch
int pcpus, cpu;
pcpus = simple_strtoul(s, NULL, 0);
- cpu_possible_map = cpumask_of_cpu(0);
- for (cpu = 1; cpu < pcpus && cpu < nr_cpu_ids; cpu++)
- cpu_set(cpu, cpu_possible_map);
+ for (cpu = 0; cpu < pcpus && cpu < nr_cpu_ids; cpu++)
+ set_cpu_possible(cpu, true);
return 0;
}
early_param("possible_cpus", setup_possible_cpus);
--- linux-2.6.28.orig/arch/sh/kernel/cpu/sh4a/smp-shx3.c
+++ linux-2.6.28/arch/sh/kernel/cpu/sh4a/smp-shx3.c
@@ -19,8 +19,7 @@ void __init plat_smp_setup(void)
unsigned int cpu = 0;
int i, num;
- cpus_clear(cpu_possible_map);
- cpu_set(cpu, cpu_possible_map);
+ init_cpu_possible(cpumask_of(cpu));
__cpu_number_map[0] = 0;
__cpu_logical_map[0] = 0;
@@ -30,7 +29,7 @@ void __init plat_smp_setup(void)
* for the total number of cores.
*/
for (i = 1, num = 0; i < NR_CPUS; i++) {
- cpu_set(i, cpu_possible_map);
+ set_cpu_possible(i, true);
__cpu_number_map[i] = ++num;
__cpu_logical_map[num] = i;
}
--- linux-2.6.28.orig/arch/sh/kernel/smp.c
+++ linux-2.6.28/arch/sh/kernel/smp.c
@@ -52,7 +52,7 @@ void __init smp_prepare_cpus(unsigned in
plat_prepare_cpus(max_cpus);
#ifndef CONFIG_HOTPLUG_CPU
- cpu_present_map = cpu_possible_map;
+ init_cpu_present(&cpu_possible_map);
#endif
}
@@ -63,8 +63,8 @@ void __devinit smp_prepare_boot_cpu(void
__cpu_number_map[0] = cpu;
__cpu_logical_map[0] = cpu;
- cpu_set(cpu, cpu_online_map);
- cpu_set(cpu, cpu_possible_map);
+ set_cpu_online(cpu, true);
+ set_cpu_possible(cpu, true);
}
asmlinkage void __cpuinit start_secondary(void)
--- linux-2.6.28.orig/arch/sparc64/kernel/mdesc.c
+++ linux-2.6.28/arch/sparc64/kernel/mdesc.c
@@ -566,7 +566,7 @@ static void __init report_platform_prope
max_cpu = NR_CPUS;
}
for (i = 0; i < max_cpu; i++)
- cpu_set(i, cpu_possible_map);
+ set_cpu_possible(i, true);
}
#endif
--- linux-2.6.28.orig/arch/sparc64/kernel/prom.c
+++ linux-2.6.28/arch/sparc64/kernel/prom.c
@@ -1601,8 +1601,8 @@ static void __init of_fill_in_cpu_data(v
}
#ifdef CONFIG_SMP
- cpu_set(cpuid, cpu_present_map);
- cpu_set(cpuid, cpu_possible_map);
+ set_cpu_present(cpuid, true);
+ set_cpu_possible(cpuid, true);
#endif
}
--- linux-2.6.28.orig/arch/um/kernel/smp.c
+++ linux-2.6.28/arch/um/kernel/smp.c
@@ -118,7 +118,7 @@ void smp_prepare_cpus(unsigned int maxcp
int i;
for (i = 0; i < ncpus; ++i)
- cpu_set(i, cpu_possible_map);
+ set_cpu_possible(i, true);
cpu_clear(me, cpu_online_map);
cpu_set(me, cpu_online_map);
--- linux-2.6.28.orig/arch/x86/kernel/apic.c
+++ linux-2.6.28/arch/x86/kernel/apic.c
@@ -1903,8 +1903,8 @@ void __cpuinit generic_processor_info(in
}
#endif
- cpu_set(cpu, cpu_possible_map);
- cpu_set(cpu, cpu_present_map);
+ set_cpu_possible(cpu, true);
+ set_cpu_present(cpu, true);
}
#ifdef CONFIG_X86_64
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 30/35] cpumask: CONFIG_BITS_ALL, CONFIG_BITS_NONE and CONFIG_BITS_CPU0
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (28 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 29/35] cpumask: Use accessors code Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 31/35] cpumask: switch over to cpu_online/possible/active/present_mask Mike Travis
` (4 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Since we're now preferring raw bitmaps for (eventually rare) static
cpumasks, we replace CPU_MASK_X with CPU_BITS_X.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 44 ++++++++++++++++++++++++--------------------
1 file changed, 24 insertions(+), 20 deletions(-)
--- linux-2.6.28.orig/include/linux/cpumask.h
+++ linux-2.6.28/include/linux/cpumask.h
@@ -53,8 +53,9 @@
*
* size_t cpumask_size() Length of cpumask in bytes.
* const struct cpumask *cpumask_of(cpu) Return cpumask with bit 'cpu' set
- * CPU_MASK_ALL Initializer - all bits set
- * CPU_MASK_NONE Initializer - no bits set
+ * CPU_BITS_ALL Initializer - all bits set
+ * CPU_BITS_NONE Initializer - no bits set
+ * CPU_BITS_CPU0 Initializer - first bit set
* unsigned long *cpumask_bits(mask) Array of unsigned long's in mask
*
* struct cpumask *to_cpumask(const unsigned long[])
@@ -116,6 +117,9 @@ struct cpumask
typedef struct cpumask cpumask_t;
extern cpumask_t _unused_cpumask_arg_;
+#define CPU_MASK_ALL ((cpumask_t){ CPU_BITS_ALL })
+#define CPU_MASK_NONE ((cpumask_t){ CPU_BITS_NONE })
+#define CPU_MASK_CPU0 ((cpumask_t){ CPU_BITS_CPU0 })
#define cpu_set(cpu, dst) cpumask_set_cpu((cpu), &(dst))
#define cpu_clear(cpu, dst) cpumask_clear_cpu((cpu), &(dst))
#define cpu_test_and_set(cpu, mask) cpumask_test_and_set_cpu((cpu), &(mask))
@@ -397,20 +401,20 @@ static inline const struct cpumask *cpum
#if NR_CPUS <= BITS_PER_LONG
-#define CPU_MASK_ALL \
-(cpumask_t) { { \
- [BITS_TO_LONGS(NR_CPUS)-1] = CPU_MASK_LAST_WORD \
-} }
+#define CPU_BITS_ALL \
+{ \
+ [BITS_TO_LONGS(CONFIG_NR_CPUS)-1] = CPU_MASK_LAST_WORD \
+}
#define CPU_MASK_ALL_PTR (&CPU_MASK_ALL)
#else
-#define CPU_MASK_ALL \
-(cpumask_t) { { \
- [0 ... BITS_TO_LONGS(NR_CPUS)-2] = ~0UL, \
- [BITS_TO_LONGS(NR_CPUS)-1] = CPU_MASK_LAST_WORD \
-} }
+#define CPU_BITS_ALL \
+{ \
+ [0 ... BITS_TO_LONGS(CONFIG_NR_CPUS)-2] = ~0UL, \
+ [BITS_TO_LONGS(CONFIG_NR_CPUS)-1] = CPU_MASK_LAST_WORD \
+}
/* cpu_mask_all is in init/main.c */
extern cpumask_t cpu_mask_all;
@@ -418,15 +422,15 @@ extern cpumask_t cpu_mask_all;
#endif
-#define CPU_MASK_NONE \
-(cpumask_t) { { \
- [0 ... BITS_TO_LONGS(NR_CPUS)-1] = 0UL \
-} }
-
-#define CPU_MASK_CPU0 \
-(cpumask_t) { { \
- [0] = 1UL \
-} }
+#define CPU_BITS_NONE \
+{ \
+ [0 ... BITS_TO_LONGS(CONFIG_NR_CPUS)-1] = 0UL \
+}
+
+#define CPU_BITS_CPU0 \
+{ \
+ [0] = 1UL \
+}
#if NR_CPUS == 1
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 31/35] cpumask: switch over to cpu_online/possible/active/present_mask
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (29 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 30/35] cpumask: CONFIG_BITS_ALL, CONFIG_BITS_NONE and CONFIG_BITS_CPU0 Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 32/35] cpumask: cpu_all_mask and cpu_none_mask Mike Travis
` (3 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
In order to hide the definition of struct cpumask, we need to expose
only pointers. Plus, it fits the new API far better to have pointers.
This deprecates the old _map versions, and defines them in terms of the
_mask versions. It also centralizes the definitions (finally!).
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
arch/alpha/kernel/smp.c | 5 --
arch/arm/kernel/smp.c | 10 ----
arch/cris/arch-v32/kernel/smp.c | 4 -
arch/ia64/kernel/smpboot.c | 6 --
arch/m32r/kernel/smpboot.c | 6 --
arch/mips/kernel/smp.c | 2
arch/parisc/kernel/smp.c | 15 ------
arch/powerpc/kernel/smp.c | 4 -
arch/s390/kernel/smp.c | 6 --
arch/sh/kernel/smp.c | 6 --
arch/sparc/kernel/sparc_ksyms.c | 2
arch/sparc64/kernel/smp.c | 4 -
arch/um/kernel/smp.c | 7 --
arch/x86/kernel/smpboot.c | 6 --
arch/x86/mach-voyager/voyager_smp.c | 7 --
include/linux/cpumask.h | 86 +++++++++++++++---------------------
kernel/cpu.c | 67 ++++++++++++----------------
17 files changed, 67 insertions(+), 176 deletions(-)
--- linux-2.6.28.orig/arch/alpha/kernel/smp.c
+++ linux-2.6.28/arch/alpha/kernel/smp.c
@@ -69,11 +69,6 @@ enum ipi_message_type {
/* Set to a secondary's cpuid when it comes online. */
static int smp_secondary_alive __devinitdata = 0;
-/* Which cpus ids came online. */
-cpumask_t cpu_online_map;
-
-EXPORT_SYMBOL(cpu_online_map);
-
int smp_num_probed; /* Internal processor count */
int smp_num_cpus = 1; /* Number that came online. */
EXPORT_SYMBOL(smp_num_cpus);
--- linux-2.6.28.orig/arch/arm/kernel/smp.c
+++ linux-2.6.28/arch/arm/kernel/smp.c
@@ -34,16 +34,6 @@
#include <asm/ptrace.h>
/*
- * bitmask of present and online CPUs.
- * The present bitmask indicates that the CPU is physically present.
- * The online bitmask indicates that the CPU is up and running.
- */
-cpumask_t cpu_possible_map;
-EXPORT_SYMBOL(cpu_possible_map);
-cpumask_t cpu_online_map;
-EXPORT_SYMBOL(cpu_online_map);
-
-/*
* as from 2.5, kernels no longer have an init_tasks structure
* so we need some other way of telling a new secondary core
* where to place its SVC stack
--- linux-2.6.28.orig/arch/cris/arch-v32/kernel/smp.c
+++ linux-2.6.28/arch/cris/arch-v32/kernel/smp.c
@@ -29,11 +29,7 @@
spinlock_t cris_atomic_locks[] = { [0 ... LOCK_COUNT - 1] = SPIN_LOCK_UNLOCKED};
/* CPU masks */
-cpumask_t cpu_online_map = CPU_MASK_NONE;
-EXPORT_SYMBOL(cpu_online_map);
cpumask_t phys_cpu_present_map = CPU_MASK_NONE;
-cpumask_t cpu_possible_map;
-EXPORT_SYMBOL(cpu_possible_map);
EXPORT_SYMBOL(phys_cpu_present_map);
/* Variables used during SMP boot */
--- linux-2.6.28.orig/arch/ia64/kernel/smpboot.c
+++ linux-2.6.28/arch/ia64/kernel/smpboot.c
@@ -131,12 +131,6 @@ struct task_struct *task_for_booting_cpu
*/
DEFINE_PER_CPU(int, cpu_state);
-/* Bitmasks of currently online, and possible CPUs */
-cpumask_t cpu_online_map;
-EXPORT_SYMBOL(cpu_online_map);
-cpumask_t cpu_possible_map = CPU_MASK_NONE;
-EXPORT_SYMBOL(cpu_possible_map);
-
cpumask_t cpu_core_map[NR_CPUS] __cacheline_aligned;
EXPORT_SYMBOL(cpu_core_map);
DEFINE_PER_CPU_SHARED_ALIGNED(cpumask_t, cpu_sibling_map);
--- linux-2.6.28.orig/arch/m32r/kernel/smpboot.c
+++ linux-2.6.28/arch/m32r/kernel/smpboot.c
@@ -72,17 +72,11 @@ static unsigned int bsp_phys_id = -1;
/* Bitmask of physically existing CPUs */
physid_mask_t phys_cpu_present_map;
-/* Bitmask of currently online CPUs */
-cpumask_t cpu_online_map;
-EXPORT_SYMBOL(cpu_online_map);
-
cpumask_t cpu_bootout_map;
cpumask_t cpu_bootin_map;
static cpumask_t cpu_callin_map;
cpumask_t cpu_callout_map;
EXPORT_SYMBOL(cpu_callout_map);
-cpumask_t cpu_possible_map = CPU_MASK_ALL;
-EXPORT_SYMBOL(cpu_possible_map);
/* Per CPU bogomips and other parameters */
struct cpuinfo_m32r cpu_data[NR_CPUS] __cacheline_aligned;
--- linux-2.6.28.orig/arch/mips/kernel/smp.c
+++ linux-2.6.28/arch/mips/kernel/smp.c
@@ -46,12 +46,10 @@
cpumask_t phys_cpu_present_map; /* Bitmask of available CPUs */
volatile cpumask_t cpu_callin_map; /* Bitmask of started secondaries */
-cpumask_t cpu_online_map; /* Bitmask of currently online CPUs */
int __cpu_number_map[NR_CPUS]; /* Map physical to logical */
int __cpu_logical_map[NR_CPUS]; /* Map logical to physical */
EXPORT_SYMBOL(phys_cpu_present_map);
-EXPORT_SYMBOL(cpu_online_map);
extern void cpu_idle(void);
--- linux-2.6.28.orig/arch/parisc/kernel/smp.c
+++ linux-2.6.28/arch/parisc/kernel/smp.c
@@ -67,21 +67,6 @@ static volatile int cpu_now_booting __re
static int parisc_max_cpus __read_mostly = 1;
-/* online cpus are ones that we've managed to bring up completely
- * possible cpus are all valid cpu
- * present cpus are all detected cpu
- *
- * On startup we bring up the "possible" cpus. Since we discover
- * CPUs later, we add them as hotplug, so the possible cpu mask is
- * empty in the beginning.
- */
-
-cpumask_t cpu_online_map __read_mostly = CPU_MASK_NONE; /* Bitmap of online CPUs */
-cpumask_t cpu_possible_map __read_mostly = CPU_MASK_ALL; /* Bitmap of Present CPUs */
-
-EXPORT_SYMBOL(cpu_online_map);
-EXPORT_SYMBOL(cpu_possible_map);
-
DEFINE_PER_CPU(spinlock_t, ipi_lock) = SPIN_LOCK_UNLOCKED;
enum ipi_message_type {
--- linux-2.6.28.orig/arch/powerpc/kernel/smp.c
+++ linux-2.6.28/arch/powerpc/kernel/smp.c
@@ -60,13 +60,9 @@
int smp_hw_index[NR_CPUS];
struct thread_info *secondary_ti;
-cpumask_t cpu_possible_map = CPU_MASK_NONE;
-cpumask_t cpu_online_map = CPU_MASK_NONE;
DEFINE_PER_CPU(cpumask_t, cpu_sibling_map) = CPU_MASK_NONE;
DEFINE_PER_CPU(cpumask_t, cpu_core_map) = CPU_MASK_NONE;
-EXPORT_SYMBOL(cpu_online_map);
-EXPORT_SYMBOL(cpu_possible_map);
EXPORT_PER_CPU_SYMBOL(cpu_sibling_map);
EXPORT_PER_CPU_SYMBOL(cpu_core_map);
--- linux-2.6.28.orig/arch/s390/kernel/smp.c
+++ linux-2.6.28/arch/s390/kernel/smp.c
@@ -52,12 +52,6 @@
struct _lowcore *lowcore_ptr[NR_CPUS];
EXPORT_SYMBOL(lowcore_ptr);
-cpumask_t cpu_online_map = CPU_MASK_NONE;
-EXPORT_SYMBOL(cpu_online_map);
-
-cpumask_t cpu_possible_map = CPU_MASK_ALL;
-EXPORT_SYMBOL(cpu_possible_map);
-
static struct task_struct *current_set[NR_CPUS];
static u8 smp_cpu_type;
--- linux-2.6.28.orig/arch/sh/kernel/smp.c
+++ linux-2.6.28/arch/sh/kernel/smp.c
@@ -30,12 +30,6 @@
int __cpu_number_map[NR_CPUS]; /* Map physical to logical */
int __cpu_logical_map[NR_CPUS]; /* Map logical to physical */
-cpumask_t cpu_possible_map;
-EXPORT_SYMBOL(cpu_possible_map);
-
-cpumask_t cpu_online_map;
-EXPORT_SYMBOL(cpu_online_map);
-
static inline void __init smp_store_cpu_info(unsigned int cpu)
{
struct sh_cpuinfo *c = cpu_data + cpu;
--- linux-2.6.28.orig/arch/sparc/kernel/sparc_ksyms.c
+++ linux-2.6.28/arch/sparc/kernel/sparc_ksyms.c
@@ -114,8 +114,6 @@ EXPORT_PER_CPU_SYMBOL(__cpu_data);
/* IRQ implementation. */
EXPORT_SYMBOL(synchronize_irq);
-/* CPU online map and active count. */
-EXPORT_SYMBOL(cpu_online_map);
EXPORT_SYMBOL(phys_cpu_present_map);
#endif
--- linux-2.6.28.orig/arch/sparc64/kernel/smp.c
+++ linux-2.6.28/arch/sparc64/kernel/smp.c
@@ -49,14 +49,10 @@
int sparc64_multi_core __read_mostly;
-cpumask_t cpu_possible_map __read_mostly = CPU_MASK_NONE;
-cpumask_t cpu_online_map __read_mostly = CPU_MASK_NONE;
DEFINE_PER_CPU(cpumask_t, cpu_sibling_map) = CPU_MASK_NONE;
cpumask_t cpu_core_map[NR_CPUS] __read_mostly =
{ [0 ... NR_CPUS-1] = CPU_MASK_NONE };
-EXPORT_SYMBOL(cpu_possible_map);
-EXPORT_SYMBOL(cpu_online_map);
EXPORT_PER_CPU_SYMBOL(cpu_sibling_map);
EXPORT_SYMBOL(cpu_core_map);
--- linux-2.6.28.orig/arch/um/kernel/smp.c
+++ linux-2.6.28/arch/um/kernel/smp.c
@@ -25,13 +25,6 @@ DEFINE_PER_CPU(struct mmu_gather, mmu_ga
#include "irq_user.h"
#include "os.h"
-/* CPU online map, set by smp_boot_cpus */
-cpumask_t cpu_online_map = CPU_MASK_NONE;
-cpumask_t cpu_possible_map = CPU_MASK_NONE;
-
-EXPORT_SYMBOL(cpu_online_map);
-EXPORT_SYMBOL(cpu_possible_map);
-
/* Per CPU bogomips and other parameters
* The only piece used here is the ipi pipe, which is set before SMP is
* started and never changed.
--- linux-2.6.28.orig/arch/x86/kernel/smpboot.c
+++ linux-2.6.28/arch/x86/kernel/smpboot.c
@@ -101,14 +101,8 @@ EXPORT_SYMBOL(smp_num_siblings);
/* Last level cache ID of each logical CPU */
DEFINE_PER_CPU(u16, cpu_llc_id) = BAD_APICID;
-/* bitmap of online cpus */
-cpumask_t cpu_online_map __read_mostly;
-EXPORT_SYMBOL(cpu_online_map);
-
cpumask_t cpu_callin_map;
cpumask_t cpu_callout_map;
-cpumask_t cpu_possible_map;
-EXPORT_SYMBOL(cpu_possible_map);
/* representing HT siblings of each logical CPU */
DEFINE_PER_CPU(cpumask_t, cpu_sibling_map);
--- linux-2.6.28.orig/arch/x86/mach-voyager/voyager_smp.c
+++ linux-2.6.28/arch/x86/mach-voyager/voyager_smp.c
@@ -62,11 +62,6 @@ static int voyager_extended_cpus = 1;
/* Used for the invalidate map that's also checked in the spinlock */
static volatile unsigned long smp_invalidate_needed;
-/* Bitmask of currently online CPUs - used by setup.c for
- /proc/cpuinfo, visible externally but still physical */
-cpumask_t cpu_online_map = CPU_MASK_NONE;
-EXPORT_SYMBOL(cpu_online_map);
-
/* Bitmask of CPUs present in the system - exported by i386_syms.c, used
* by scheduler but indexed physically */
cpumask_t phys_cpu_present_map = CPU_MASK_NONE;
@@ -216,8 +211,6 @@ static cpumask_t smp_commenced_mask = CP
/* This is for the new dynamic CPU boot code */
cpumask_t cpu_callin_map = CPU_MASK_NONE;
cpumask_t cpu_callout_map = CPU_MASK_NONE;
-cpumask_t cpu_possible_map = CPU_MASK_NONE;
-EXPORT_SYMBOL(cpu_possible_map);
/* The per processor IRQ masks (these are usually kept in sync) */
static __u16 vic_irq_mask[NR_CPUS] __cacheline_aligned;
--- linux-2.6.28.orig/include/linux/cpumask.h
+++ linux-2.6.28/include/linux/cpumask.h
@@ -88,9 +88,9 @@
* int cpumask_any_and(mask1,mask2) Any cpu in both masks
* int cpumask_any_but(mask,cpu) Any cpu in mask except cpu
*
- * for_each_possible_cpu(cpu) for-loop cpu over cpu_possible_map
- * for_each_online_cpu(cpu) for-loop cpu over cpu_online_map
- * for_each_present_cpu(cpu) for-loop cpu over cpu_present_map
+ * for_each_possible_cpu(cpu) for-loop cpu over cpu_possible_mask
+ * for_each_online_cpu(cpu) for-loop cpu over cpu_online_mask
+ * for_each_present_cpu(cpu) for-loop cpu over cpu_present_mask
*
* Subtlety:
* 1) The 'type-checked' form of cpu_isset() causes gcc (3.3.2, anyway)
@@ -161,7 +161,7 @@ extern cpumask_t _unused_cpumask_arg_;
for_each_cpu_and(cpu, &(mask), &(and))
#define first_cpu(src) cpumask_first(&(src))
#define next_cpu(n, src) cpumask_next((n), &(src))
-#define any_online_cpu(mask) cpumask_any_and(&(mask), &cpu_online_map)
+#define any_online_cpu(mask) cpumask_any_and(&(mask), cpu_online_mask)
#if NR_CPUS > BITS_PER_LONG
#define CPUMASK_ALLOC(m) struct m *m = kmalloc(sizeof(*m), GFP_KERNEL)
#define CPUMASK_FREE(m) kfree(m)
@@ -170,6 +170,11 @@ extern cpumask_t _unused_cpumask_arg_;
#define CPUMASK_FREE(m)
#endif
#define CPUMASK_PTR(v, m) cpumask_t *v = &(m->v)
+/* These strip const, as traditionally they weren't const. */
+#define cpu_possible_map (*(cpumask_t *)cpu_possible_mask)
+#define cpu_online_map (*(cpumask_t *)cpu_online_mask)
+#define cpu_present_map (*(cpumask_t *)cpu_present_mask)
+#define cpu_active_map (*(cpumask_t *)cpu_active_mask)
/* End deprecated region. */
#if NR_CPUS <= BITS_PER_LONG
@@ -530,65 +535,48 @@ static inline void free_cpumask_or_fallb
/*
* The following particular system cpumasks and operations manage
- * possible, present, active and online cpus. Each of them is a fixed size
- * bitmap of size NR_CPUS.
+ * possible, present, active and online cpus.
*
- * #ifdef CONFIG_HOTPLUG_CPU
- * cpu_possible_map - has bit 'cpu' set iff cpu is populatable
- * cpu_present_map - has bit 'cpu' set iff cpu is populated
- * cpu_online_map - has bit 'cpu' set iff cpu available to scheduler
- * cpu_active_map - has bit 'cpu' set iff cpu available to migration
- * #else
- * cpu_possible_map - has bit 'cpu' set iff cpu is populated
- * cpu_present_map - copy of cpu_possible_map
- * cpu_online_map - has bit 'cpu' set iff cpu available to scheduler
- * #endif
- *
- * In either case, NR_CPUS is fixed at compile time, as the static
- * size of these bitmaps. The cpu_possible_map is fixed at boot
- * time, as the set of CPU id's that it is possible might ever
- * be plugged in at anytime during the life of that system boot.
- * The cpu_present_map is dynamic(*), representing which CPUs
- * are currently plugged in. And cpu_online_map is the dynamic
- * subset of cpu_present_map, indicating those CPUs available
- * for scheduling.
+ * cpu_possible_mask- has bit 'cpu' set iff cpu is populatable
+ * cpu_present_mask - has bit 'cpu' set iff cpu is populated
+ * cpu_online_mask - has bit 'cpu' set iff cpu available to scheduler
+ * cpu_active_mask - has bit 'cpu' set iff cpu available to migration
+ *
+ * If !CONFIG_HOTPLUG_CPU, present == possible, and active == online.
+ *
+ * The cpu_possible_mask is fixed at boot time, as the set of CPU id's
+ * that it is possible might ever be plugged in at anytime during the
+ * life of that system boot. The cpu_present_mask is dynamic(*),
+ * representing which CPUs are currently plugged in. And
+ * cpu_online_mask is the dynamic subset of cpu_present_mask,
+ * indicating those CPUs available for scheduling.
*
- * If HOTPLUG is enabled, then cpu_possible_map is forced to have
+ * If HOTPLUG is enabled, then cpu_possible_mask is forced to have
* all NR_CPUS bits set, otherwise it is just the set of CPUs that
* ACPI reports present at boot.
*
- * If HOTPLUG is enabled, then cpu_present_map varies dynamically,
+ * If HOTPLUG is enabled, then cpu_present_mask varies dynamically,
* depending on what ACPI reports as currently plugged in, otherwise
- * cpu_present_map is just a copy of cpu_possible_map.
+ * cpu_present_mask is just a copy of cpu_possible_mask.
*
- * (*) Well, cpu_present_map is dynamic in the hotplug case. If not
- * hotplug, it's a copy of cpu_possible_map, hence fixed at boot.
+ * (*) Well, cpu_present_mask is dynamic in the hotplug case. If not
+ * hotplug, it's a copy of cpu_possible_mask, hence fixed at boot.
*
* Subtleties:
* 1) UP arch's (NR_CPUS == 1, CONFIG_SMP not defined) hardcode
* assumption that their single CPU is online. The UP
- * cpu_{online,possible,present}_maps are placebos. Changing them
+ * cpu_{online,possible,present}_masks are placebos. Changing them
* will have no useful affect on the following num_*_cpus()
* and cpu_*() macros in the UP case. This ugliness is a UP
* optimization - don't waste any instructions or memory references
* asking if you're online or how many CPUs there are if there is
* only one CPU.
- * 2) Most SMP arch's #define some of these maps to be some
- * other map specific to that arch. Therefore, the following
- * must be #define macros, not inlines. To see why, examine
- * the assembly code produced by the following. Note that
- * set1() writes phys_x_map, but set2() writes x_map:
- * int x_map, phys_x_map;
- * #define set1(a) x_map = a
- * inline void set2(int a) { x_map = a; }
- * #define x_map phys_x_map
- * main(){ set1(3); set2(5); }
*/
-extern cpumask_t cpu_possible_map;
-extern cpumask_t cpu_online_map;
-extern cpumask_t cpu_present_map;
-extern cpumask_t cpu_active_map;
+extern const struct cpumask *const cpu_possible_mask;
+extern const struct cpumask *const cpu_online_mask;
+extern const struct cpumask *const cpu_present_mask;
+extern const struct cpumask *const cpu_active_mask;
#if NR_CPUS > 1
#define num_online_cpus() cpus_weight(cpu_online_map)
@@ -619,11 +607,11 @@ void init_cpu_online(const struct cpumas
#define cpu_is_offline(cpu) unlikely(!cpu_online(cpu))
-#define for_each_possible_cpu(cpu) for_each_cpu((cpu), &cpu_possible_map)
-#define for_each_online_cpu(cpu) for_each_cpu((cpu), &cpu_online_map)
-#define for_each_present_cpu(cpu) for_each_cpu((cpu), &cpu_present_map)
+#define for_each_possible_cpu(cpu) for_each_cpu((cpu), cpu_possible_mask)
+#define for_each_online_cpu(cpu) for_each_cpu((cpu), cpu_online_mask)
+#define for_each_present_cpu(cpu) for_each_cpu((cpu), cpu_present_mask)
/* Arch-specific code may call this to initialize nr_cpu_ids based on
- * the cpu_possible_map. */
+ * the cpu_possible_mask. */
void __init setup_nr_cpu_ids(void);
#endif /* __LINUX_CPUMASK_H */
--- linux-2.6.28.orig/kernel/cpu.c
+++ linux-2.6.28/kernel/cpu.c
@@ -15,29 +15,8 @@
#include <linux/stop_machine.h>
#include <linux/mutex.h>
-/*
- * Represents all cpu's present in the system
- * In systems capable of hotplug, this map could dynamically grow
- * as new cpu's are detected in the system via any platform specific
- * method, such as ACPI for e.g.
- */
-cpumask_t cpu_present_map __read_mostly;
-EXPORT_SYMBOL(cpu_present_map);
-
-#ifndef CONFIG_SMP
-
-/*
- * Represents all cpu's that are currently online.
- */
-cpumask_t cpu_online_map __read_mostly = CPU_MASK_ALL;
-EXPORT_SYMBOL(cpu_online_map);
-
-cpumask_t cpu_possible_map __read_mostly = CPU_MASK_ALL;
-EXPORT_SYMBOL(cpu_possible_map);
-
-#else /* CONFIG_SMP */
-
-/* Serializes the updates to cpu_online_map, cpu_present_map */
+#ifdef CONFIG_SMP
+/* Serializes the updates to cpu_online_mask, cpu_present_mask */
static DEFINE_MUTEX(cpu_add_remove_lock);
static __cpuinitdata RAW_NOTIFIER_HEAD(cpu_chain);
@@ -64,8 +43,6 @@ void __init cpu_hotplug_init(void)
cpu_hotplug.refcount = 0;
}
-cpumask_t cpu_active_map;
-
#ifdef CONFIG_HOTPLUG_CPU
void get_online_cpus(void)
@@ -96,7 +73,7 @@ EXPORT_SYMBOL_GPL(put_online_cpus);
/*
* The following two API's must be used when attempting
- * to serialize the updates to cpu_online_map, cpu_present_map.
+ * to serialize the updates to cpu_online_mask, cpu_present_mask.
*/
void cpu_maps_update_begin(void)
{
@@ -499,43 +476,59 @@ const unsigned long cpu_bit_bitmap[BITS_
};
EXPORT_SYMBOL_GPL(cpu_bit_bitmap);
+static DECLARE_BITMAP(cpu_possible_bits, CONFIG_NR_CPUS) __read_mostly;
+const struct cpumask *const cpu_possible_mask = to_cpumask(cpu_possible_bits);
+EXPORT_SYMBOL(cpu_possible_mask);
+
+static DECLARE_BITMAP(cpu_online_bits, CONFIG_NR_CPUS) __read_mostly;
+const struct cpumask *const cpu_online_mask = to_cpumask(cpu_online_bits);
+EXPORT_SYMBOL(cpu_online_mask);
+
+static DECLARE_BITMAP(cpu_present_bits, CONFIG_NR_CPUS) __read_mostly;
+const struct cpumask *const cpu_present_mask = to_cpumask(cpu_present_bits);
+EXPORT_SYMBOL(cpu_present_mask);
+
+static DECLARE_BITMAP(cpu_active_bits, CONFIG_NR_CPUS) __read_mostly;
+const struct cpumask *const cpu_active_mask = to_cpumask(cpu_active_bits);
+EXPORT_SYMBOL(cpu_active_mask);
+
void set_cpu_possible(unsigned int cpu, bool possible)
{
if (possible)
- cpumask_set_cpu(cpu, &cpu_possible_map);
+ cpumask_set_cpu(cpu, to_cpumask(cpu_possible_bits));
else
- cpumask_clear_cpu(cpu, &cpu_possible_map);
+ cpumask_clear_cpu(cpu, to_cpumask(cpu_possible_bits));
}
void set_cpu_present(unsigned int cpu, bool present)
{
if (present)
- cpumask_set_cpu(cpu, &cpu_present_map);
+ cpumask_set_cpu(cpu, to_cpumask(cpu_present_bits));
else
- cpumask_clear_cpu(cpu, &cpu_present_map);
+ cpumask_clear_cpu(cpu, to_cpumask(cpu_present_bits));
}
void set_cpu_online(unsigned int cpu, bool online)
{
if (online)
- cpumask_set_cpu(cpu, &cpu_online_map);
+ cpumask_set_cpu(cpu, to_cpumask(cpu_online_bits));
else
- cpumask_clear_cpu(cpu, &cpu_online_map);
+ cpumask_clear_cpu(cpu, to_cpumask(cpu_online_bits));
}
void set_cpu_active(unsigned int cpu, bool active)
{
if (active)
- cpumask_set_cpu(cpu, &cpu_active_map);
+ cpumask_set_cpu(cpu, to_cpumask(cpu_active_bits));
else
- cpumask_clear_cpu(cpu, &cpu_active_map);
+ cpumask_clear_cpu(cpu, to_cpumask(cpu_active_bits));
}
void init_cpu_present(const struct cpumask *src)
{
- cpumask_copy(&cpu_present_map, src);
+ cpumask_copy(to_cpumask(cpu_present_bits), src);
}
void init_cpu_possible(const struct cpumask *src)
{
- cpumask_copy(&cpu_possible_map, src);
+ cpumask_copy(to_cpumask(cpu_possible_bits), src);
}
void init_cpu_online(const struct cpumask *src)
{
- cpumask_copy(&cpu_online_map, src);
+ cpumask_copy(to_cpumask(cpu_online_bits), src);
}
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 32/35] cpumask: cpu_all_mask and cpu_none_mask.
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (30 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 31/35] cpumask: switch over to cpu_online/possible/active/present_mask Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 33/35] cpumask: reorder header to minimize separate #ifdefs Mike Travis
` (2 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Instead of CPU_MASK_ALL_PTR and the SMP-only cpu_mask_all, this makes
cpu_all_mask and cpu_none_mask which are const cpumask pointers which
always exist.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 16 ++++++++++------
init/main.c | 7 +------
kernel/cpu.c | 2 ++
kernel/kmod.c | 2 +-
kernel/kthread.c | 4 ++--
kernel/sched.c | 2 +-
6 files changed, 17 insertions(+), 16 deletions(-)
--- linux-2.6.28.orig/include/linux/cpumask.h
+++ linux-2.6.28/include/linux/cpumask.h
@@ -117,6 +117,7 @@ struct cpumask
typedef struct cpumask cpumask_t;
extern cpumask_t _unused_cpumask_arg_;
+#define CPU_MASK_ALL_PTR (cpu_all_mask)
#define CPU_MASK_ALL ((cpumask_t){ CPU_BITS_ALL })
#define CPU_MASK_NONE ((cpumask_t){ CPU_BITS_NONE })
#define CPU_MASK_CPU0 ((cpumask_t){ CPU_BITS_CPU0 })
@@ -175,6 +176,7 @@ extern cpumask_t _unused_cpumask_arg_;
#define cpu_online_map (*(cpumask_t *)cpu_online_mask)
#define cpu_present_map (*(cpumask_t *)cpu_present_mask)
#define cpu_active_map (*(cpumask_t *)cpu_active_mask)
+#define cpu_mask_all (*(cpumask_t *)cpu_all_mask)
/* End deprecated region. */
#if NR_CPUS <= BITS_PER_LONG
@@ -411,8 +413,6 @@ static inline const struct cpumask *cpum
[BITS_TO_LONGS(CONFIG_NR_CPUS)-1] = CPU_MASK_LAST_WORD \
}
-#define CPU_MASK_ALL_PTR (&CPU_MASK_ALL)
-
#else
#define CPU_BITS_ALL \
@@ -421,10 +421,6 @@ static inline const struct cpumask *cpum
[BITS_TO_LONGS(CONFIG_NR_CPUS)-1] = CPU_MASK_LAST_WORD \
}
-/* cpu_mask_all is in init/main.c */
-extern cpumask_t cpu_mask_all;
-#define CPU_MASK_ALL_PTR (&cpu_mask_all)
-
#endif
#define CPU_BITS_NONE \
@@ -578,6 +574,14 @@ extern const struct cpumask *const cpu_o
extern const struct cpumask *const cpu_present_mask;
extern const struct cpumask *const cpu_active_mask;
+/* It's common to want to use cpu_all_mask in struct member initializers,
+ * so it has to refer to an address rather than a pointer. */
+extern const DECLARE_BITMAP(cpu_all_bits, CONFIG_NR_CPUS);
+#define cpu_all_mask to_cpumask(cpu_all_bits)
+
+/* First bits of cpu_bit_bitmap are in fact unset. */
+#define cpu_none_mask to_cpumask(cpu_bit_bitmap[0])
+
#if NR_CPUS > 1
#define num_online_cpus() cpus_weight(cpu_online_map)
#define num_possible_cpus() cpus_weight(cpu_possible_map)
--- linux-2.6.28.orig/init/main.c
+++ linux-2.6.28/init/main.c
@@ -367,11 +367,6 @@ static inline void smp_prepare_cpus(unsi
#else
-#if NR_CPUS > BITS_PER_LONG
-cpumask_t cpu_mask_all __read_mostly = CPU_MASK_ALL;
-EXPORT_SYMBOL(cpu_mask_all);
-#endif
-
/* Setup number of possible processor ids */
/* nr_cpumask_bits is a real variable for large NR_CPUS. */
#ifndef nr_cpumask_bits
@@ -869,7 +864,7 @@ static int __init kernel_init(void * unu
/*
* init can run on any cpu.
*/
- set_cpus_allowed_ptr(current, CPU_MASK_ALL_PTR);
+ set_cpus_allowed_ptr(current, cpu_all_mask);
/*
* Tell the world that we're going to be the grim
* reaper of innocent orphaned children.
--- linux-2.6.28.orig/kernel/cpu.c
+++ linux-2.6.28/kernel/cpu.c
@@ -489,6 +489,8 @@ EXPORT_SYMBOL(cpu_present_mask);
static DECLARE_BITMAP(cpu_active_bits, CONFIG_NR_CPUS) __read_mostly;
const struct cpumask *const cpu_active_mask = to_cpumask(cpu_active_bits);
EXPORT_SYMBOL(cpu_active_mask);
+const DECLARE_BITMAP(cpu_all_bits, CONFIG_NR_CPUS) = CPU_BITS_ALL;
+EXPORT_SYMBOL(cpu_all_bits);
void set_cpu_possible(unsigned int cpu, bool possible)
{
--- linux-2.6.28.orig/kernel/kmod.c
+++ linux-2.6.28/kernel/kmod.c
@@ -166,7 +166,7 @@ static int ____call_usermodehelper(void
}
/* We can run anywhere, unlike our parent keventd(). */
- set_cpus_allowed_ptr(current, CPU_MASK_ALL_PTR);
+ set_cpus_allowed_ptr(current, cpu_all_mask);
/*
* Our parent is keventd, which runs with elevated scheduling priority.
--- linux-2.6.28.orig/kernel/kthread.c
+++ linux-2.6.28/kernel/kthread.c
@@ -107,7 +107,7 @@ static void create_kthread(struct kthrea
*/
sched_setscheduler(create->result, SCHED_NORMAL, ¶m);
set_user_nice(create->result, KTHREAD_NICE_LEVEL);
- set_cpus_allowed_ptr(create->result, CPU_MASK_ALL_PTR);
+ set_cpus_allowed_ptr(create->result, cpu_all_mask);
}
complete(&create->done);
}
@@ -238,7 +238,7 @@ int kthreadd(void *unused)
set_task_comm(tsk, "kthreadd");
ignore_signals(tsk);
set_user_nice(tsk, KTHREAD_NICE_LEVEL);
- set_cpus_allowed_ptr(tsk, CPU_MASK_ALL_PTR);
+ set_cpus_allowed_ptr(tsk, cpu_all_mask);
current->flags |= PF_NOFREEZE | PF_FREEZER_NOSIG;
--- linux-2.6.28.orig/kernel/sched.c
+++ linux-2.6.28/kernel/sched.c
@@ -6166,7 +6166,7 @@ static void move_task_off_dead_cpu(int d
*/
static void migrate_nr_uninterruptible(struct rq *rq_src)
{
- struct rq *rq_dest = cpu_rq(any_online_cpu(*CPU_MASK_ALL_PTR));
+ struct rq *rq_dest = cpu_rq(cpumask_any(cpu_online_mask));
unsigned long flags;
local_irq_save(flags);
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 33/35] cpumask: reorder header to minimize separate #ifdefs
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (31 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 32/35] cpumask: cpu_all_mask and cpu_none_mask Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 34/35] cpumask: debug options for cpumasks Mike Travis
2008-10-20 17:03 ` [PATCH 35/35] cpumask: smp_call_function_many() Mike Travis
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
cpumask.h is pretty chaotic. Now we've replaced most of it, let's
group things together a bit better.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 72 ++++++++++++++++++++++--------------------------
1 file changed, 34 insertions(+), 38 deletions(-)
--- test-compile.orig/include/linux/cpumask.h
+++ test-compile/include/linux/cpumask.h
@@ -134,6 +134,8 @@ static inline ssize_t cpumask_size(void)
}
+#define cpumask_size() (BITS_TO_LONGS(nr_cpumask_bits) * sizeof(long))
+
/* Deprecated. */
typedef struct cpumask cpumask_t;
extern cpumask_t _unused_cpumask_arg_;
@@ -200,6 +202,22 @@ extern cpumask_t _unused_cpumask_arg_;
#define cpu_mask_all (*(cpumask_t *)cpu_all_mask)
/* End deprecated region. */
+/* static cpumask initializers */
+#define CPU_MASK_LAST_WORD BITMAP_LAST_WORD_MASK(CONFIG_NR_CPUS)
+#if CONFIG_NR_CPUS <= BITS_PER_LONG
+#define CPU_BITS_ALL \
+{ \
+ [BITS_TO_LONGS(CONFIG_NR_CPUS)-1] = CPU_MASK_LAST_WORD \
+}
+#else
+#define CPU_BITS_ALL \
+{ \
+ [0 ... BITS_TO_LONGS(CONFIG_NR_CPUS)-2] = ~0UL, \
+ [BITS_TO_LONGS(CONFIG_NR_CPUS)-1] = CPU_MASK_LAST_WORD \
+}
+#endif /* CONFIG_NR_CPUS > BITS_PER_LONG */
+
+/* cpumask_* operators */
static inline void cpumask_set_cpu(int cpu, volatile struct cpumask *dstp)
{
set_bit(cpu, cpumask_bits(dstp));
@@ -414,25 +432,6 @@ static inline const struct cpumask *cpum
return (const struct cpumask *)p;
}
-#define CPU_MASK_LAST_WORD BITMAP_LAST_WORD_MASK(NR_CPUS)
-
-#if NR_CPUS <= BITS_PER_LONG
-
-#define CPU_BITS_ALL \
-{ \
- [BITS_TO_LONGS(CONFIG_NR_CPUS)-1] = CPU_MASK_LAST_WORD \
-}
-
-#else
-
-#define CPU_BITS_ALL \
-{ \
- [0 ... BITS_TO_LONGS(CONFIG_NR_CPUS)-2] = ~0UL, \
- [BITS_TO_LONGS(CONFIG_NR_CPUS)-1] = CPU_MASK_LAST_WORD \
-}
-
-#endif
-
#define CPU_BITS_NONE \
{ \
[0 ... BITS_TO_LONGS(CONFIG_NR_CPUS)-1] = 0UL \
@@ -455,6 +454,13 @@ static inline const struct cpumask *cpum
#define for_each_cpu_and(cpu, mask, and) \
for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask, (void)and)
+#define num_online_cpus() 1
+#define num_possible_cpus() 1
+#define num_present_cpus() 1
+#define cpu_online(cpu) ((cpu) == 0)
+#define cpu_possible(cpu) ((cpu) == 0)
+#define cpu_present(cpu) ((cpu) == 0)
+#define cpu_active(cpu) ((cpu) == 0)
#else /* NR_CPUS > 1 */
int cpumask_first(const cpumask_t *srcp);
@@ -470,7 +476,15 @@ int cpumask_any_but(const struct cpumask
for ((cpu) = -1; \
(cpu) = cpumask_next_and((cpu), (mask), (and)), \
(cpu) < nr_cpumask_bits;)
-#endif
+
+#define num_online_cpus() cpus_weight(cpu_online_map)
+#define num_possible_cpus() cpus_weight(cpu_possible_map)
+#define num_present_cpus() cpus_weight(cpu_present_map)
+#define cpu_online(cpu) cpu_isset((cpu), cpu_online_map)
+#define cpu_possible(cpu) cpu_isset((cpu), cpu_possible_map)
+#define cpu_present(cpu) cpu_isset((cpu), cpu_present_map)
+#define cpu_active(cpu) cpu_isset((cpu), cpu_active_map)
+#endif /* NR_CPUS */
#define cpumask_first_and(mask, and) cpumask_next_and(-1, (mask), (and))
@@ -563,24 +577,6 @@ extern const DECLARE_BITMAP(cpu_all_bits
/* First bits of cpu_bit_bitmap are in fact unset. */
#define cpu_none_mask to_cpumask(cpu_bit_bitmap[0])
-#if NR_CPUS > 1
-#define num_online_cpus() cpus_weight(cpu_online_map)
-#define num_possible_cpus() cpus_weight(cpu_possible_map)
-#define num_present_cpus() cpus_weight(cpu_present_map)
-#define cpu_online(cpu) cpu_isset((cpu), cpu_online_map)
-#define cpu_possible(cpu) cpu_isset((cpu), cpu_possible_map)
-#define cpu_present(cpu) cpu_isset((cpu), cpu_present_map)
-#define cpu_active(cpu) cpu_isset((cpu), cpu_active_map)
-#else
-#define num_online_cpus() 1
-#define num_possible_cpus() 1
-#define num_present_cpus() 1
-#define cpu_online(cpu) ((cpu) == 0)
-#define cpu_possible(cpu) ((cpu) == 0)
-#define cpu_present(cpu) ((cpu) == 0)
-#define cpu_active(cpu) ((cpu) == 0)
-#endif
-
/* Wrappers to manipulate otherwise-constant masks. */
void set_cpu_possible(unsigned int cpu, bool possible);
void set_cpu_present(unsigned int cpu, bool present);
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 34/35] cpumask: debug options for cpumasks
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (32 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 33/35] cpumask: reorder header to minimize separate #ifdefs Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
2008-10-20 17:03 ` [PATCH 35/35] cpumask: smp_call_function_many() Mike Travis
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
It's useful to check that no one is accessing > nr_cpumask_bits for
cpumasks. This also allows you to turn on CONFIG_CPUMASKS_OFFSTACK
even for smaller CONFIG_NR_CPUS.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
include/linux/cpumask.h | 23 +++++++++++++++++------
lib/Kconfig.debug | 6 ++++++
lib/cpumask.c | 3 +++
3 files changed, 26 insertions(+), 6 deletions(-)
--- test-compile.orig/include/linux/cpumask.h
+++ test-compile/include/linux/cpumask.h
@@ -217,23 +217,34 @@ extern cpumask_t _unused_cpumask_arg_;
}
#endif /* CONFIG_NR_CPUS > BITS_PER_LONG */
+/* verify cpu argument to cpumask_* operators */
+static inline unsigned int cpumask_check(unsigned int cpu)
+{
+#ifdef CONFIG_DEBUG_PER_CPU_MAPS
+ /* This breaks at runtime. */
+ BUG_ON(cpu >= nr_cpumask_bits);
+#endif /* CONFIG_DEBUG_PER_CPU_MAPS */
+ return cpu;
+}
+
/* cpumask_* operators */
static inline void cpumask_set_cpu(int cpu, volatile struct cpumask *dstp)
{
- set_bit(cpu, cpumask_bits(dstp));
+ set_bit(cpumask_check(cpu), cpumask_bits(dstp));
}
static inline void cpumask_clear_cpu(int cpu, volatile struct cpumask *dstp)
{
- clear_bit(cpu, cpumask_bits(dstp));
+ clear_bit(cpumask_check(cpu), cpumask_bits(dstp));
}
/* No static inline type checking - see Subtlety (1) above. */
-#define cpumask_test_cpu(cpu, cpumask) test_bit((cpu), (cpumask)->bits)
+#define cpumask_test_cpu(cpu, cpumask) \
+ test_bit(cpumask_check(cpu), (cpumask)->bits)
static inline int cpumask_test_and_set_cpu(int cpu, struct cpumask *addr)
{
- return test_and_set_bit(cpu, cpumask_bits(addr));
+ return test_and_set_bit(cpumask_check(cpu), cpumask_bits(addr));
}
static inline void cpumask_setall(struct cpumask *dstp)
@@ -367,8 +378,8 @@ static inline int cpumask_cpuremap(int o
const struct cpumask *oldp,
const struct cpumask *newp)
{
- return bitmap_bitremap(oldbit, cpumask_bits(oldp), cpumask_bits(newp),
- nr_cpumask_bits);
+ return bitmap_bitremap(cpumask_check(oldbit), cpumask_bits(oldp),
+ cpumask_bits(newp), nr_cpumask_bits);
}
static inline void cpumask_remap(struct cpumask *dstp,
--- test-compile.orig/lib/Kconfig.debug
+++ test-compile/lib/Kconfig.debug
@@ -766,6 +766,12 @@ config SYSCTL_SYSCALL_CHECK
to properly maintain and use. This enables checks that help
you to keep things correct.
+config DEBUG_PER_CPU_MAPS
+ bool "Cpumask debug checks"
+ ---help---
+ Extra debugging for cpumasks.
+ eg. to make sure accesses to cpumasks are < nr_cpu_ids.
+
source kernel/trace/Kconfig
config PROVIDE_OHCI1394_DMA_INIT
--- test-compile.orig/lib/cpumask.c
+++ test-compile/lib/cpumask.c
@@ -11,6 +11,9 @@ EXPORT_SYMBOL(cpumask_first);
int cpumask_next(int n, const cpumask_t *srcp)
{
+ /* -1 is a legal arg here. */
+ if (n != -1)
+ cpumask_check(n);
return find_next_bit(cpumask_bits(srcp), nr_cpumask_bits, n+1);
}
EXPORT_SYMBOL(cpumask_next);
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 35/35] cpumask: smp_call_function_many()
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
` (33 preceding siblings ...)
2008-10-20 17:03 ` [PATCH 34/35] cpumask: debug options for cpumasks Mike Travis
@ 2008-10-20 17:03 ` Mike Travis
34 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-20 17:03 UTC (permalink / raw)
To: Ingo Molnar, Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, S390, peterz,
Jack Steiner, linux-kernel, Eric Dumazet, PowerPC, Andi Kleen,
Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Andrew Morton, David Miller
Transition from cpumask_t-taking smp_call_function_mask() to a new
smp_call_function_many() which takes a struct cpumask *.
(Naming is inspired by smp_call_function_single).
Note that the new one returns void: the old one couldn't fail either
unless there was a logic bug.
From: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
arch/s390/include/asm/smp.h | 3 -
arch/s390/kernel/smp.c | 30 +++++++++++--------
include/linux/smp.h | 13 +++++++-
kernel/smp.c | 68 +++++++++++++++++++++++---------------------
4 files changed, 64 insertions(+), 50 deletions(-)
--- test-compile.orig/arch/s390/include/asm/smp.h
+++ test-compile/arch/s390/include/asm/smp.h
@@ -90,9 +90,6 @@ extern int __cpu_up (unsigned int cpu);
extern struct mutex smp_cpu_state_mutex;
extern int smp_cpu_polarization[];
-
-extern int smp_call_function_mask(cpumask_t mask, void (*func)(void *),
- void *info, int wait);
#endif
#ifndef CONFIG_SMP
--- test-compile.orig/arch/s390/kernel/smp.c
+++ test-compile/arch/s390/kernel/smp.c
@@ -103,7 +103,7 @@ static void do_call_function(void)
}
static void __smp_call_function_map(void (*func) (void *info), void *info,
- int wait, cpumask_t map)
+ int wait, struct cpumask *map)
{
struct call_data_struct data;
int cpu, local = 0;
@@ -163,14 +163,16 @@ out:
* You must not call this function with disabled interrupts, from a
* hardware interrupt handler or from a bottom half.
*/
+
+/* protected by call_lock */
+static DEFINE_BITMAP(smp_call_map, CONFIG_NR_CPUS);
+
int smp_call_function(void (*func) (void *info), void *info, int wait)
{
- cpumask_t map;
-
spin_lock(&call_lock);
- map = cpu_online_map;
- cpu_clear(smp_processor_id(), map);
- __smp_call_function_map(func, info, wait, map);
+ cpumask_copy(to_cpumask(smp_call_map), cpu_online_mask);
+ cpumask_clear_cpu(smp_processor_id(), to_cpumask(smp_call_map));
+ __smp_call_function_map(func, info, wait, to_cpumask(smp_call_map));
spin_unlock(&call_lock);
return 0;
}
@@ -192,14 +194,15 @@ int smp_call_function_single(int cpu, vo
int wait)
{
spin_lock(&call_lock);
- __smp_call_function_map(func, info, wait, cpumask_of_cpu(cpu));
+ cpumask_copy(to_cpumask(smp_call_map), cpumask_of(cpu));
+ __smp_call_function_map(func, info, wait, cpumask_of(cpu));
spin_unlock(&call_lock);
return 0;
}
EXPORT_SYMBOL(smp_call_function_single);
/**
- * smp_call_function_mask(): Run a function on a set of other CPUs.
+ * smp_call_function_many(): Run a function on a set of other CPUs.
* @mask: The set of cpus to run on. Must not include the current cpu.
* @func: The function to run. This must be fast and non-blocking.
* @info: An arbitrary pointer to pass to the function.
@@ -213,16 +216,17 @@ EXPORT_SYMBOL(smp_call_function_single);
* You must not call this function with disabled interrupts or from a
* hardware interrupt handler or from a bottom half handler.
*/
-int smp_call_function_mask(cpumask_t mask, void (*func)(void *), void *info,
- int wait)
+int smp_call_function_many(const struct cpumask *mask,
+ void (*func)(void *), void *info, bool wait)
{
spin_lock(&call_lock);
- cpu_clear(smp_processor_id(), mask);
- __smp_call_function_map(func, info, wait, mask);
+ cpumask_copy(to_cpumask(smp_call_map), cpu_online_mask);
+ cpumask_clear_cpu(smp_processor_id(), to_cpumask(smp_call_map));
+ __smp_call_function_map(func, info, wait, to_cpumask(smp_call_map));
spin_unlock(&call_lock);
return 0;
}
-EXPORT_SYMBOL(smp_call_function_mask);
+EXPORT_SYMBOL(smp_call_function_many);
void smp_send_stop(void)
{
--- test-compile.orig/include/linux/smp.h
+++ test-compile/include/linux/smp.h
@@ -62,12 +62,21 @@ extern void smp_cpus_done(unsigned int m
* Call a function on all other processors
*/
int smp_call_function(void(*func)(void *info), void *info, int wait);
-int smp_call_function_mask(cpumask_t mask, void(*func)(void *info), void *info,
- int wait);
+void smp_call_function_many(const struct cpumask *mask,
+ void(*func)(void *info), void *info, bool wait);
int smp_call_function_single(int cpuid, void (*func) (void *info), void *info,
int wait);
void __smp_call_function_single(int cpuid, struct call_single_data *data);
+/* Use smp_call_function_many, which takes a pointer to the mask. */
+static inline int __deprecated
+smp_call_function_mask(cpumask_t mask, void(*func)(void *info), void *info,
+ int wait)
+{
+ smp_call_function_many(&mask, func, info, wait);
+ return 0;
+}
+
/*
* Generic and arch helpers
*/
--- test-compile.orig/kernel/smp.c
+++ test-compile/kernel/smp.c
@@ -24,7 +24,7 @@ struct call_function_data {
struct call_single_data csd;
spinlock_t lock;
unsigned int refs;
- cpumask_t cpumask;
+ struct cpumask *cpumask;
struct rcu_head rcu_head;
};
@@ -109,13 +109,13 @@ void generic_smp_call_function_interrupt
list_for_each_entry_rcu(data, &call_function_queue, csd.list) {
int refs;
- if (!cpu_isset(cpu, data->cpumask))
+ if (!cpumask_test_cpu(cpu, data->cpumask))
continue;
data->csd.func(data->csd.info);
spin_lock(&data->lock);
- cpu_clear(cpu, data->cpumask);
+ cpumask_clear_cpu(cpu, data->cpumask);
WARN_ON(data->refs == 0);
data->refs--;
refs = data->refs;
@@ -273,7 +273,7 @@ static void quiesce_dummy(void *unused)
/*
* Ensure stack based data used in call function mask is safe to free.
*
- * This is needed by smp_call_function_mask when using on-stack data, because
+ * This is needed by smp_call_function_many when using on-stack data, because
* a single call function queue is shared by all CPUs, and any CPU may pick up
* the data item on the queue at any time before it is deleted. So we need to
* ensure that all CPUs have transitioned through a quiescent state after
@@ -287,7 +287,7 @@ static void quiesce_dummy(void *unused)
* If a faster scheme can be made, we could go back to preferring stack based
* data -- the data allocation/free is non-zero cost.
*/
-static void smp_call_function_mask_quiesce_stack(cpumask_t mask)
+static void smp_call_function_mask_quiesce_stack(const struct cpumask *mask)
{
struct call_single_data data;
int cpu;
@@ -295,21 +295,19 @@ static void smp_call_function_mask_quies
data.func = quiesce_dummy;
data.info = NULL;
- for_each_cpu_mask(cpu, mask) {
+ for_each_cpu(cpu, mask) {
data.flags = CSD_FLAG_WAIT;
generic_exec_single(cpu, &data);
}
}
/**
- * smp_call_function_mask(): Run a function on a set of other CPUs.
- * @mask: The set of cpus to run on.
+ * smp_call_function_many(): Run a function on a set of other CPUs.
+ * @mask: The set of cpus to run on (only runs on online subset).
* @func: The function to run. This must be fast and non-blocking.
* @info: An arbitrary pointer to pass to the function.
* @wait: If true, wait (atomically) until function has completed on other CPUs.
*
- * Returns 0 on success, else a negative status code.
- *
* If @wait is true, then returns once @func has returned. Note that @wait
* will be implicitly turned on in case of allocation failures, since
* we fall back to on-stack allocation.
@@ -318,12 +316,13 @@ static void smp_call_function_mask_quies
* hardware interrupt handler or from a bottom half handler. Preemption
* must be disabled when calling this function.
*/
-int smp_call_function_mask(cpumask_t mask, void (*func)(void *), void *info,
- int wait)
+void smp_call_function_many(const struct cpumask *mask,
+ void (*func)(void *), void *info,
+ bool wait)
{
struct call_function_data d;
struct call_function_data *data = NULL;
- cpumask_t allbutself;
+ cpumask_var_t allbutself;
unsigned long flags;
int cpu, num_cpus;
int slowpath = 0;
@@ -331,21 +330,28 @@ int smp_call_function_mask(cpumask_t mas
/* Can deadlock when called with interrupts disabled */
WARN_ON(irqs_disabled());
- cpu = smp_processor_id();
- allbutself = cpu_online_map;
- cpu_clear(cpu, allbutself);
- cpus_and(mask, mask, allbutself);
- num_cpus = cpus_weight(mask);
+ if (!alloc_cpumask_var(&allbutself, GFP_ATOMIC)) {
+ /* Slow path. */
+ for_each_online_cpu(cpu) {
+ if (cpumask_test_cpu(cpu, mask))
+ smp_call_function_single(cpu, func, info, wait);
+ }
+ return;
+ }
+ cpumask_and(allbutself, cpu_online_mask, mask);
+ cpumask_clear_cpu(smp_processor_id(), allbutself);
+ num_cpus = cpumask_weight(allbutself);
/*
* If zero CPUs, return. If just a single CPU, turn this request
* into a targetted single call instead since it's faster.
*/
if (!num_cpus)
- return 0;
+ return;
else if (num_cpus == 1) {
- cpu = first_cpu(mask);
- return smp_call_function_single(cpu, func, info, wait);
+ cpu = cpumask_first(allbutself);
+ smp_call_function_single(cpu, func, info, wait);
+ goto out;
}
data = kmalloc(sizeof(*data), GFP_ATOMIC);
@@ -364,25 +370,25 @@ int smp_call_function_mask(cpumask_t mas
data->csd.func = func;
data->csd.info = info;
data->refs = num_cpus;
- data->cpumask = mask;
+ data->cpumask = allbutself;
spin_lock_irqsave(&call_function_lock, flags);
list_add_tail_rcu(&data->csd.list, &call_function_queue);
spin_unlock_irqrestore(&call_function_lock, flags);
/* Send a message to all CPUs in the map */
- arch_send_call_function_ipi(mask);
+ arch_send_call_function_ipi((cpumask_t)*allbutself);
/* optionally wait for the CPUs to complete */
if (wait) {
csd_flag_wait(&data->csd);
if (unlikely(slowpath))
- smp_call_function_mask_quiesce_stack(mask);
+ smp_call_function_mask_quiesce_stack(allbutself);
}
-
- return 0;
+out:
+ free_cpumask_var(allbutself);
}
-EXPORT_SYMBOL(smp_call_function_mask);
+EXPORT_SYMBOL(smp_call_function_many);
/**
* smp_call_function(): Run a function on all other CPUs.
@@ -390,7 +396,7 @@ EXPORT_SYMBOL(smp_call_function_mask);
* @info: An arbitrary pointer to pass to the function.
* @wait: If true, wait (atomically) until function has completed on other CPUs.
*
- * Returns 0 on success, else a negative status code.
+ * Returns 0.
*
* If @wait is true, then returns once @func has returned; otherwise
* it returns just before the target cpu calls @func. In case of allocation
@@ -401,12 +407,10 @@ EXPORT_SYMBOL(smp_call_function_mask);
*/
int smp_call_function(void (*func)(void *), void *info, int wait)
{
- int ret;
-
preempt_disable();
- ret = smp_call_function_mask(cpu_online_map, func, info, wait);
+ smp_call_function_many(cpu_online_mask, func, info, wait);
preempt_enable();
- return ret;
+ return 0;
}
EXPORT_SYMBOL(smp_call_function);
--
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [PATCH 01/35] x86: clean up speedctep-centrino and reduce cpumask_t usage
2008-10-20 17:03 ` [PATCH 01/35] x86: clean up speedctep-centrino and reduce cpumask_t usage Mike Travis
@ 2008-10-21 0:09 ` Stephen Rothwell
2008-10-21 12:28 ` Mike Travis
0 siblings, 1 reply; 40+ messages in thread
From: Stephen Rothwell @ 2008-10-21 0:09 UTC (permalink / raw)
To: Mike Travis
Cc: davej, Jeremy Fitzhardinge, Sorensen, IA64, Andrew Morton, Jes,
peterz, S390, David Miller, Rusty Russell, linux-kernel,
Eric Dumazet, PowerPC, Andi Kleen, Yinghai Lu, H. Peter Anvin,
SPARC, Ingo Molnar, Thomas Gleixner, Jack Steiner
[-- Attachment #1: Type: text/plain, Size: 859 bytes --]
Hi Mike,
Just a first small thing:
On Mon, 20 Oct 2008 10:03:20 -0700 Mike Travis <travis@sgi.com> wrote:
>
> 1) The #ifdef CONFIG_HOTPLUG_CPU seems unnecessary these days.
> 2) The loop can simply skip over offline cpus, rather than creating a tmp mask.
> 3) set_mask is set to either a single cpu or all online cpus in a policy.
> Since it's just used for set_cpus_allowed(), any offline cpus in a policy
> don't matter, so we can just use cpumask_of_cpu() or the policy->cpus.
>
> From: Rusty Russell <rusty@rustcorp.com.au>
> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
> Signed-off-by: Mike Travis <travis@sgi.com>
The From: line should be the first nonempty line in the mail to get the
attribution correct.
--
Cheers,
Stephen Rothwell sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/
[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [PATCH 18/35] cpumask: add nr_cpumask_bits
2008-10-20 17:03 ` [PATCH 18/35] cpumask: add nr_cpumask_bits Mike Travis
@ 2008-10-21 12:26 ` Rusty Russell
2008-10-21 13:53 ` Mike Travis
0 siblings, 1 reply; 40+ messages in thread
From: Rusty Russell @ 2008-10-21 12:26 UTC (permalink / raw)
To: Mike Travis
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, David Miller,
peterz, S390, Jack Steiner, linux-kernel, Eric Dumazet, PowerPC,
Andi Kleen, Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Ingo Molnar, Andrew Morton
On Tuesday 21 October 2008 04:03:37 Mike Travis wrote:
> When nr_cpu_ids is set to CONFIG_NR_CPUS then references to nr_cpu_ids
> will return the maximum index of the configured NR_CPUS (+1) instead
> of the maximum index of the possible number of cpus (+1). This results
> in extra unused memory being allocated by functions that are setting up
> arrays of structs to keep track of per cpu items.
1) I like the name in this context: it's a beacon of sanity after NR_CPUS and
nr_cpu_ids. But it's not so clearly a win when general code uses it:
if (cpumask_first(mymask) == nr_cpumask_bits) ...
vs:
if (cpumask_first(mymask) == nr_cpu_ids) ...
2) This breaks anyone who tests that the iterators etc. return == nr_cpu_ids.
One of the other patches tried to change them from NR_CPUS to nr_cpu_ids,
that should now be revisited & reaudited.
3) Noone should be naively allocating "* nr_cpu_ids" arrays, they should be
using per-cpu pointers. Not doing so wastes memory on non-contiguous
processor systems.
4) It should be a constant not be dependent on CONFIG_CPUMASK_OFFSTACK, but
rather as it was on NR_CPUS > BITS_PER_LONG. I think that's the sweet
spot, and should also make your 2MB "gain" vanish.
That's why I suggested a max_possible_cpu() function, and using that for those
who really want to do allocations, who should be audited anyway, see (3). I
don't want it as prominent as nr_cpu_ids, which is usually the Right Thing,
and always safe.
Cheers,
Rusty.
PS. I have part of a patch for this...
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [PATCH 01/35] x86: clean up speedctep-centrino and reduce cpumask_t usage
2008-10-21 0:09 ` Stephen Rothwell
@ 2008-10-21 12:28 ` Mike Travis
0 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-21 12:28 UTC (permalink / raw)
To: Stephen Rothwell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, Andrew Morton,
S390, peterz, David Miller, Rusty Russell, linux-kernel,
Eric Dumazet, PowerPC, Andi Kleen, Yinghai Lu, H. Peter Anvin,
SPARC, Ingo Molnar, Thomas Gleixner, Jack Steiner
Stephen Rothwell wrote:
> Hi Mike,
>
> Just a first small thing:
>
> On Mon, 20 Oct 2008 10:03:20 -0700 Mike Travis <travis@sgi.com> wrote:
>> 1) The #ifdef CONFIG_HOTPLUG_CPU seems unnecessary these days.
>> 2) The loop can simply skip over offline cpus, rather than creating a tmp mask.
>> 3) set_mask is set to either a single cpu or all online cpus in a policy.
>> Since it's just used for set_cpus_allowed(), any offline cpus in a policy
>> don't matter, so we can just use cpumask_of_cpu() or the policy->cpus.
>>
>> From: Rusty Russell <rusty@rustcorp.com.au>
>> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
>> Signed-off-by: Mike Travis <travis@sgi.com>
>
> The From: line should be the first nonempty line in the mail to get the
> attribution correct.
>
Ahh, ok, thanks!
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [PATCH 18/35] cpumask: add nr_cpumask_bits
2008-10-21 12:26 ` Rusty Russell
@ 2008-10-21 13:53 ` Mike Travis
0 siblings, 0 replies; 40+ messages in thread
From: Mike Travis @ 2008-10-21 13:53 UTC (permalink / raw)
To: Rusty Russell
Cc: davej, Jeremy Fitzhardinge, Jes Sorensen, IA64, David Miller,
peterz, S390, Jack Steiner, linux-kernel, Eric Dumazet, PowerPC,
Andi Kleen, Thomas Gleixner, Yinghai Lu, H. Peter Anvin, SPARC,
Ingo Molnar, Andrew Morton
Rusty Russell wrote:
> On Tuesday 21 October 2008 04:03:37 Mike Travis wrote:
>> When nr_cpu_ids is set to CONFIG_NR_CPUS then references to nr_cpu_ids
>> will return the maximum index of the configured NR_CPUS (+1) instead
>> of the maximum index of the possible number of cpus (+1). This results
>> in extra unused memory being allocated by functions that are setting up
>> arrays of structs to keep track of per cpu items.
>
> 1) I like the name in this context: it's a beacon of sanity after NR_CPUS and
> nr_cpu_ids. But it's not so clearly a win when general code uses it:
>
> if (cpumask_first(mymask) == nr_cpumask_bits) ...
>
> vs:
>
> if (cpumask_first(mymask) == nr_cpu_ids) ...
I think the correct use for iterators would be:
if (cpumask_first(mymask) >= nr_cpu_ids)
... since nr_cpu_ids is guaranteed to be <= nr_cpumask_bits. And nr_cpumask_bits
is not really meant to be used anywhere except in the bit operations supporting
the cpumask_* operators.
> 2) This breaks anyone who tests that the iterators etc. return == nr_cpu_ids.
I think that's broken anyways... ;-)
> One of the other patches tried to change them from NR_CPUS to nr_cpu_ids,
> that should now be revisited & reaudited.
The change from NR_CPUS to nr_cpu_ids is ok, but it should also be changed from:
(x == NR_CPUS)
to:
(x <= nr_cpu_ids)
> 3) Noone should be naively allocating "* nr_cpu_ids" arrays, they should be
> using per-cpu pointers. Not doing so wastes memory on non-contiguous
> processor systems.
The problem often arises where an array is allocated that will use the
cpu as an index into the array. They can be changed eventually to use a
percpu pointer, but in the interim keeping nr_cpu_ids intact maintains
compatibility without allocating unused memory.
> 4) It should be a constant not be dependent on CONFIG_CPUMASK_OFFSTACK, but
> rather as it was on NR_CPUS > BITS_PER_LONG. I think that's the sweet
> spot, and should also make your 2MB "gain" vanish.
I'll run the test again but most likely the result will be an extra 1Mb of
unused memory instead. ;-) One other note, that test compile used the default
config [and NR_CPUS=128] which turns off a lot of functions. A typical distro
config will have many more options turned on.
And the beauty of using a separate flag to enable variable length cpumasks,
is there may be cases where an arch or specific system config wants a multiple
word cpumask on the stack for performance reasons (like cache or node locality,
avoidance of the kmalloc's, etc.)
> That's why I suggested a max_possible_cpu() function, and using that for those
> who really want to do allocations, who should be audited anyway, see (3). I
> don't want it as prominent as nr_cpu_ids, which is usually the Right Thing,
> and always safe.
We could change all refs from nr_cpu_ids to max_possible_cpu but wouldn't
we still be leaving a window open where it could be incorrectly used?
So far all the cpumask conversions allow for "mixed-use" cpumask (i.e.,
cpumask_t and struct cpumask *) by maintaining backwards compatility,
until everything is eventually sorted out. Keeping nr_cpu_ids representing
the same value maintains this "compatibility bridge".
The most correct way would be to use the (not yet implemented) zero
based PERCPU allocator. Slightly less efficient would be to allocate
node local memory for each struct, in a loop using a per cpu pointer
and "for_each_possible_cpu()".
> Cheers,
> Rusty.
> PS. I have part of a patch for this...
But as I've said, it's not critical to the new functionality...
Thanks!
Mike
^ permalink raw reply [flat|nested] 40+ messages in thread
end of thread, other threads:[~2008-10-21 13:52 UTC | newest]
Thread overview: 40+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-10-20 17:03 [PATCH 00/35] cpumask: Replace cpumask_t with struct cpumask Mike Travis
2008-10-20 17:03 ` [PATCH 01/35] x86: clean up speedctep-centrino and reduce cpumask_t usage Mike Travis
2008-10-21 0:09 ` Stephen Rothwell
2008-10-21 12:28 ` Mike Travis
2008-10-20 17:03 ` [PATCH 02/35] cpumask: remove min from first_cpu/next_cpu Mike Travis
2008-10-20 17:03 ` [PATCH 03/35] cpumask: add for_each_cpu_mask_and function Mike Travis
2008-10-20 17:03 ` [PATCH 04/35] x86 smp: modify send_IPI_mask interface to accept cpumask_t pointers Mike Travis
2008-10-20 17:03 ` [PATCH 05/35] sched: Reduce stack size requirements in kernel/sched.c Mike Travis
2008-10-20 17:03 ` [PATCH 06/35] cpumask: introduce struct cpumask Mike Travis
2008-10-20 17:03 ` [PATCH 07/35] cpumask: change cpumask_scnprintf, cpumask_parse_user, cpulist_parse, and cpulist_scnprintf to take pointers Mike Travis
2008-10-20 17:03 ` [PATCH 08/35] cpumask: cpumask_size() Mike Travis
2008-10-20 17:03 ` [PATCH 09/35] cpumask: add cpumask_copy() Mike Travis
2008-10-20 17:03 ` [PATCH 10/35] cpumask: introduce cpumask_var_t for local cpumask vars Mike Travis
2008-10-20 17:03 ` [PATCH 11/35] x86: enable MAXSMP Mike Travis
2008-10-20 17:03 ` [PATCH 12/35] cpumask: make CONFIG_NR_CPUS always valid Mike Travis
2008-10-20 17:03 ` [PATCH 13/35] cpumask: use setup_nr_cpu_ids() instead of direct assignment Mike Travis
2008-10-20 17:03 ` [PATCH 14/35] cpumask: make nr_cpu_ids valid in all configurations Mike Travis
2008-10-20 17:03 ` [PATCH 15/35] cpumask: prepare for iterators to only go to nr_cpu_ids Mike Travis
2008-10-20 17:03 ` [PATCH 16/35] percpu: fix percpu accessors to potentially !cpu_possible() cpus Mike Travis
2008-10-20 17:03 ` [PATCH 17/35] cpumask: make nr_cpu_ids the actual limit on bitmap size Mike Travis
2008-10-20 17:03 ` [PATCH 18/35] cpumask: add nr_cpumask_bits Mike Travis
2008-10-21 12:26 ` Rusty Russell
2008-10-21 13:53 ` Mike Travis
2008-10-20 17:03 ` [PATCH 19/35] cpumask: use cpumask_bits() everywhere Mike Travis
2008-10-20 17:03 ` [PATCH 20/35] cpumask: cpumask_of(): cpumask_of_cpu() which returns a pointer Mike Travis
2008-10-20 17:03 ` [PATCH 21/35] cpumask: for_each_cpu(): for_each_cpu_mask which takes " Mike Travis
2008-10-20 17:03 ` [PATCH 22/35] cpumask: cpumask_first/cpumask_next Mike Travis
2008-10-20 17:03 ` [PATCH 23/35] cpumask: deprecate any_online_cpu() in favour of cpumask_any/cpumask_any_and Mike Travis
2008-10-20 17:03 ` [PATCH 24/35] cpumask: cpumask_any_but() Mike Travis
2008-10-20 17:03 ` [PATCH 25/35] cpumask: Deprecate CPUMASK_ALLOC etc in favor of cpumask_var_t Mike Travis
2008-10-20 17:03 ` [PATCH 26/35] cpumask: get rid of boutique sched.c allocations, use cpumask_var_t Mike Travis
2008-10-20 17:03 ` [PATCH 27/35] cpumask: to_cpumask() Mike Travis
2008-10-20 17:03 ` [PATCH 28/35] cpumask: accessors to manipulate possible/present/online/active maps Mike Travis
2008-10-20 17:03 ` [PATCH 29/35] cpumask: Use accessors code Mike Travis
2008-10-20 17:03 ` [PATCH 30/35] cpumask: CONFIG_BITS_ALL, CONFIG_BITS_NONE and CONFIG_BITS_CPU0 Mike Travis
2008-10-20 17:03 ` [PATCH 31/35] cpumask: switch over to cpu_online/possible/active/present_mask Mike Travis
2008-10-20 17:03 ` [PATCH 32/35] cpumask: cpu_all_mask and cpu_none_mask Mike Travis
2008-10-20 17:03 ` [PATCH 33/35] cpumask: reorder header to minimize separate #ifdefs Mike Travis
2008-10-20 17:03 ` [PATCH 34/35] cpumask: debug options for cpumasks Mike Travis
2008-10-20 17:03 ` [PATCH 35/35] cpumask: smp_call_function_many() Mike Travis
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).