* [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines
@ 2008-12-19 16:01 Mike Travis
2008-12-19 16:01 ` [PATCH 1/8] cpumask: convert kernel/compat.c Mike Travis
` (7 more replies)
0 siblings, 8 replies; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Rusty Russell, linux-kernel
Patches for review only. Will be pushed via one git conduit or another.
Remove 20 more cpumask_t references (down to *only* 518... ;-)
Pulled the following patches from:
http://ozlabs.org/~rusty/kernel/rr-2008-12-18-1.tar.bz2
cpumask:convert-kernel_compat.c.patch
cpumask:convert-kernel_workqueue.c.patch
cpumask:convert-kernel_time.patch
cpumask:convert-kernel_trace.patch
cpumask:convert-kernel.patch
cpumask:convert-mm.patch
cpumask:convert-drivers.patch
cpumask:convert-misc.patch
... and edited slightly/built/tested on x86_64
In addition, the following are OBSOLETE:
cpumask:get-rid-of-_nr-core.patch
(folded into the above patches to avoid inter-patch dependencies)
The affected files are:
(* - interdepency on 'irq_default_affinity')
(*) arch/alpha/kernel/irq.c
drivers/base/cpu.c
drivers/misc/sgi-xp/xpc_main.c
drivers/net/sfc/efx.c
drivers/oprofile/buffer_sync.c
drivers/oprofile/buffer_sync.h
drivers/oprofile/oprof.c
drivers/xen/manage.c
fs/seq_file.c
include/linux/interrupt.h
include/linux/rcuclassic.h
include/linux/seq_file.h
include/linux/stop_machine.h
include/linux/tick.h
kernel/compat.c
kernel/cpu.c
kernel/irq/manage.c
kernel/irq/proc.c
kernel/power/poweroff.c
kernel/profile.c
kernel/rcuclassic.c
kernel/rcupreempt.c
kernel/rcutorture.c
kernel/softlockup.c
kernel/stop_machine.c
kernel/taskstats.c
kernel/time/clocksource.c
kernel/time/tick-broadcast.c
kernel/time/tick-common.c
kernel/trace/ring_buffer.c
kernel/trace/trace.c
kernel/trace/trace_sysprof.c
kernel/workqueue.c
lib/smp_processor_id.c
mm/pdflush.c
mm/slab.c
mm/slub.c
mm/vmscan.c
mm/vmstat.c
net/iucv/iucv.c
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 1/8] cpumask: convert kernel/compat.c
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
2008-12-19 16:01 ` [PATCH 2/8] cpumask: convert kernel/workqueue.c Mike Travis
` (6 subsequent siblings)
7 siblings, 0 replies; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Rusty Russell, linux-kernel
[-- Attachment #1: cpumask:convert-kernel_compat.c.patch --]
[-- Type: text/plain, Size: 2705 bytes --]
Impact: Reduce stack usage, use new cpumask API.
Straightforward conversion; cpumasks' size is given by cpumask_size() (now
a variable rather than fixed) and on-stack cpu masks use cpumask_var_t.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
kernel/compat.c | 49 ++++++++++++++++++++++++++++++-------------------
1 file changed, 30 insertions(+), 19 deletions(-)
--- linux-2.6.28.orig/kernel/compat.c
+++ linux-2.6.28/kernel/compat.c
@@ -454,16 +454,16 @@ asmlinkage long compat_sys_waitid(int wh
}
static int compat_get_user_cpu_mask(compat_ulong_t __user *user_mask_ptr,
- unsigned len, cpumask_t *new_mask)
+ unsigned len, struct cpumask *new_mask)
{
unsigned long *k;
- if (len < sizeof(cpumask_t))
- memset(new_mask, 0, sizeof(cpumask_t));
- else if (len > sizeof(cpumask_t))
- len = sizeof(cpumask_t);
+ if (len < cpumask_size())
+ memset(new_mask, 0, cpumask_size());
+ else if (len > cpumask_size())
+ len = cpumask_size();
- k = cpus_addr(*new_mask);
+ k = cpumask_bits(new_mask);
return compat_get_bitmap(k, user_mask_ptr, len * 8);
}
@@ -471,40 +471,51 @@ asmlinkage long compat_sys_sched_setaffi
unsigned int len,
compat_ulong_t __user *user_mask_ptr)
{
- cpumask_t new_mask;
+ cpumask_var_t new_mask;
int retval;
- retval = compat_get_user_cpu_mask(user_mask_ptr, len, &new_mask);
+ if (!alloc_cpumask_var(&new_mask, GFP_KERNEL))
+ return -ENOMEM;
+
+ retval = compat_get_user_cpu_mask(user_mask_ptr, len, new_mask);
if (retval)
- return retval;
+ goto out;
- return sched_setaffinity(pid, &new_mask);
+ retval = sched_setaffinity(pid, new_mask);
+out:
+ free_cpumask_var(new_mask);
+ return retval;
}
asmlinkage long compat_sys_sched_getaffinity(compat_pid_t pid, unsigned int len,
compat_ulong_t __user *user_mask_ptr)
{
int ret;
- cpumask_t mask;
+ cpumask_var_t mask;
unsigned long *k;
- unsigned int min_length = sizeof(cpumask_t);
+ unsigned int min_length = cpumask_size();
- if (NR_CPUS <= BITS_PER_COMPAT_LONG)
+ if (nr_cpu_ids <= BITS_PER_COMPAT_LONG)
min_length = sizeof(compat_ulong_t);
if (len < min_length)
return -EINVAL;
- ret = sched_getaffinity(pid, &mask);
+ if (!alloc_cpumask_var(&mask, GFP_KERNEL))
+ return -ENOMEM;
+
+ ret = sched_getaffinity(pid, mask);
if (ret < 0)
- return ret;
+ goto out;
- k = cpus_addr(mask);
+ k = cpumask_bits(mask);
ret = compat_put_bitmap(user_mask_ptr, k, min_length * 8);
- if (ret)
- return ret;
+ if (ret == 0)
+ ret = min_length;
- return min_length;
+out:
+ free_cpumask_var(mask);
+ return ret;
}
int get_compat_itimerspec(struct itimerspec *dst,
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 2/8] cpumask: convert kernel/workqueue.c
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
2008-12-19 16:01 ` [PATCH 1/8] cpumask: convert kernel/compat.c Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
2008-12-19 16:01 ` [PATCH 3/8] cpumask: convert kernel time functions Mike Travis
` (5 subsequent siblings)
7 siblings, 0 replies; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Rusty Russell, linux-kernel, Heiko Carstens
[-- Attachment #1: cpumask:convert-kernel_workqueue.c.patch --]
[-- Type: text/plain, Size: 4695 bytes --]
Impact: Reduce memory usage, use new cpumask API.
In this case, we replace cpu_populated_map with a cpumask_var_t rather
than a bitmap because there's an obvious init function for workqueues
which happens after kmalloc is available.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
---
kernel/workqueue.c | 39 +++++++++++++++++++++------------------
1 file changed, 21 insertions(+), 18 deletions(-)
--- linux-2.6-for-ingo.orig/kernel/workqueue.c
+++ linux-2.6-for-ingo/kernel/workqueue.c
@@ -73,15 +73,15 @@ static DEFINE_SPINLOCK(workqueue_lock);
static LIST_HEAD(workqueues);
static int singlethread_cpu __read_mostly;
-static cpumask_t cpu_singlethread_map __read_mostly;
+static const struct cpumask *cpu_singlethread_map __read_mostly;
/*
- * _cpu_down() first removes CPU from cpu_online_map, then CPU_DEAD
+ * _cpu_down() first removes CPU from cpu_online_mask, then CPU_DEAD
* flushes cwq->worklist. This means that flush_workqueue/wait_on_work
* which comes in between can't use for_each_online_cpu(). We could
- * use cpu_possible_map, the cpumask below is more a documentation
+ * use cpu_possible_mask, the cpumask below is more a documentation
* than optimization.
*/
-static cpumask_t cpu_populated_map __read_mostly;
+static cpumask_var_t cpu_populated_map __read_mostly;
/* If it's single threaded, it isn't in the list of workqueues. */
static inline int is_single_threaded(struct workqueue_struct *wq)
@@ -89,10 +89,10 @@ static inline int is_single_threaded(str
return wq->singlethread;
}
-static const cpumask_t *wq_cpu_map(struct workqueue_struct *wq)
+static const struct cpumask *wq_cpu_map(struct workqueue_struct *wq)
{
return is_single_threaded(wq)
- ? &cpu_singlethread_map : &cpu_populated_map;
+ ? cpu_singlethread_map : cpu_populated_map;
}
static
@@ -410,13 +410,13 @@ static int flush_cpu_workqueue(struct cp
*/
void flush_workqueue(struct workqueue_struct *wq)
{
- const cpumask_t *cpu_map = wq_cpu_map(wq);
+ const struct cpumask *cpu_map = wq_cpu_map(wq);
int cpu;
might_sleep();
lock_map_acquire(&wq->lockdep_map);
lock_map_release(&wq->lockdep_map);
- for_each_cpu_mask_nr(cpu, *cpu_map)
+ for_each_cpu(cpu, cpu_map)
flush_cpu_workqueue(per_cpu_ptr(wq->cpu_wq, cpu));
}
EXPORT_SYMBOL_GPL(flush_workqueue);
@@ -532,7 +532,7 @@ static void wait_on_work(struct work_str
{
struct cpu_workqueue_struct *cwq;
struct workqueue_struct *wq;
- const cpumask_t *cpu_map;
+ const struct cpumask *cpu_map;
int cpu;
might_sleep();
@@ -547,7 +547,7 @@ static void wait_on_work(struct work_str
wq = cwq->wq;
cpu_map = wq_cpu_map(wq);
- for_each_cpu_mask_nr(cpu, *cpu_map)
+ for_each_cpu(cpu, cpu_map)
wait_on_cpu_work(per_cpu_ptr(wq->cpu_wq, cpu), work);
}
@@ -778,7 +778,7 @@ static int create_workqueue_thread(struc
* if (caller is __create_workqueue)
* nobody should see this wq
* else // caller is CPU_UP_PREPARE
- * cpu is not on cpu_online_map
+ * cpu is not on cpu_online_mask
* so we can abort safely.
*/
if (IS_ERR(p))
@@ -903,7 +903,7 @@ static void cleanup_workqueue_thread(str
*/
void destroy_workqueue(struct workqueue_struct *wq)
{
- const cpumask_t *cpu_map = wq_cpu_map(wq);
+ const struct cpumask *cpu_map = wq_cpu_map(wq);
int cpu;
cpu_maps_update_begin();
@@ -911,7 +911,7 @@ void destroy_workqueue(struct workqueue_
list_del(&wq->list);
spin_unlock(&workqueue_lock);
- for_each_cpu_mask_nr(cpu, *cpu_map)
+ for_each_cpu(cpu, cpu_map)
cleanup_workqueue_thread(per_cpu_ptr(wq->cpu_wq, cpu));
cpu_maps_update_done();
@@ -933,7 +933,7 @@ static int __devinit workqueue_cpu_callb
switch (action) {
case CPU_UP_PREPARE:
- cpu_set(cpu, cpu_populated_map);
+ cpumask_set_cpu(cpu, cpu_populated_map);
}
undo:
list_for_each_entry(wq, &workqueues, list) {
@@ -964,7 +964,7 @@ undo:
switch (action) {
case CPU_UP_CANCELED:
case CPU_POST_DEAD:
- cpu_clear(cpu, cpu_populated_map);
+ cpumask_clear_cpu(cpu, cpu_populated_map);
}
return ret;
@@ -1017,9 +1017,12 @@ EXPORT_SYMBOL_GPL(work_on_cpu);
void __init init_workqueues(void)
{
- cpu_populated_map = cpu_online_map;
- singlethread_cpu = first_cpu(cpu_possible_map);
- cpu_singlethread_map = cpumask_of_cpu(singlethread_cpu);
+ alloc_cpumask_var(&cpu_populated_map, GFP_KERNEL);
+ cpumask_copy(cpu_populated_map, cpu_online_mask);
+
+ singlethread_cpu = cpumask_first(cpu_possible_mask);
+ cpu_singlethread_map = cpumask_of(singlethread_cpu);
+
hotcpu_notifier(workqueue_cpu_callback, 0);
keventd_wq = create_workqueue("events");
BUG_ON(!keventd_wq);
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 3/8] cpumask: convert kernel time functions
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
2008-12-19 16:01 ` [PATCH 1/8] cpumask: convert kernel/compat.c Mike Travis
2008-12-19 16:01 ` [PATCH 2/8] cpumask: convert kernel/workqueue.c Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
2008-12-20 11:29 ` Rusty Russell
2008-12-19 16:01 ` [PATCH 4/8] cpumask: convert kernel trace functions Mike Travis
` (4 subsequent siblings)
7 siblings, 1 reply; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Rusty Russell, linux-kernel, Thomas Gleixner
[-- Attachment #1: cpumask:convert-kernel_time.patch --]
[-- Type: text/plain, Size: 14212 bytes --]
Impact: Reduce stack usage, use new cpumask API.
Convert kernel/time functions to use struct cpumask *.
Note the ugly bitmap declarations in tick-broadcast.c. These should
be cpumask_var_t, but there was no obvious initialization function to
put the alloc_cpumask_var() calls in. This was safe.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/tick.h | 4 -
kernel/time/clocksource.c | 9 +--
kernel/time/tick-broadcast.c | 115 ++++++++++++++++++++++---------------------
kernel/time/tick-common.c | 6 +-
4 files changed, 69 insertions(+), 65 deletions(-)
--- linux-2.6-for-ingo.orig/include/linux/tick.h
+++ linux-2.6-for-ingo/include/linux/tick.h
@@ -84,10 +84,10 @@ static inline void tick_cancel_sched_tim
# ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
extern struct tick_device *tick_get_broadcast_device(void);
-extern cpumask_t *tick_get_broadcast_mask(void);
+extern struct cpumask *tick_get_broadcast_mask(void);
# ifdef CONFIG_TICK_ONESHOT
-extern cpumask_t *tick_get_broadcast_oneshot_mask(void);
+extern struct cpumask *tick_get_broadcast_oneshot_mask(void);
# endif
# endif /* BROADCAST */
--- linux-2.6-for-ingo.orig/kernel/time/clocksource.c
+++ linux-2.6-for-ingo/kernel/time/clocksource.c
@@ -145,10 +145,11 @@ static void clocksource_watchdog(unsigne
* Cycle through CPUs to check if the CPUs stay
* synchronized to each other.
*/
- int next_cpu = next_cpu_nr(raw_smp_processor_id(), cpu_online_map);
+ int next_cpu = cpumask_next(raw_smp_processor_id(),
+ cpu_online_mask);
if (next_cpu >= nr_cpu_ids)
- next_cpu = first_cpu(cpu_online_map);
+ next_cpu = cpumask_first(cpu_online_mask);
watchdog_timer.expires += WATCHDOG_INTERVAL;
add_timer_on(&watchdog_timer, next_cpu);
}
@@ -173,7 +174,7 @@ static void clocksource_check_watchdog(s
watchdog_last = watchdog->read();
watchdog_timer.expires = jiffies + WATCHDOG_INTERVAL;
add_timer_on(&watchdog_timer,
- first_cpu(cpu_online_map));
+ cpumask_first(cpu_online_mask));
}
} else {
if (cs->flags & CLOCK_SOURCE_IS_CONTINUOUS)
@@ -195,7 +196,7 @@ static void clocksource_check_watchdog(s
watchdog_timer.expires =
jiffies + WATCHDOG_INTERVAL;
add_timer_on(&watchdog_timer,
- first_cpu(cpu_online_map));
+ cpumask_first(cpu_online_mask));
}
}
}
--- linux-2.6-for-ingo.orig/kernel/time/tick-broadcast.c
+++ linux-2.6-for-ingo/kernel/time/tick-broadcast.c
@@ -28,7 +28,9 @@
*/
struct tick_device tick_broadcast_device;
-static cpumask_t tick_broadcast_mask;
+/* FIXME: Use cpumask_var_t. */
+static DECLARE_BITMAP(tick_broadcast_mask, CONFIG_NR_CPUS);
+static DECLARE_BITMAP(tmpmask, CONFIG_NR_CPUS);
static DEFINE_SPINLOCK(tick_broadcast_lock);
static int tick_broadcast_force;
@@ -46,9 +48,9 @@ struct tick_device *tick_get_broadcast_d
return &tick_broadcast_device;
}
-cpumask_t *tick_get_broadcast_mask(void)
+struct cpumask *tick_get_broadcast_mask(void)
{
- return &tick_broadcast_mask;
+ return to_cpumask(tick_broadcast_mask);
}
/*
@@ -72,7 +74,7 @@ int tick_check_broadcast_device(struct c
clockevents_exchange_device(NULL, dev);
tick_broadcast_device.evtdev = dev;
- if (!cpus_empty(tick_broadcast_mask))
+ if (!cpumask_empty(tick_get_broadcast_mask()))
tick_broadcast_start_periodic(dev);
return 1;
}
@@ -104,7 +106,7 @@ int tick_device_uses_broadcast(struct cl
*/
if (!tick_device_is_functional(dev)) {
dev->event_handler = tick_handle_periodic;
- cpu_set(cpu, tick_broadcast_mask);
+ cpumask_set_cpu(cpu, tick_get_broadcast_mask());
tick_broadcast_start_periodic(tick_broadcast_device.evtdev);
ret = 1;
} else {
@@ -116,7 +118,7 @@ int tick_device_uses_broadcast(struct cl
if (!(dev->features & CLOCK_EVT_FEAT_C3STOP)) {
int cpu = smp_processor_id();
- cpu_clear(cpu, tick_broadcast_mask);
+ cpumask_clear_cpu(cpu, tick_get_broadcast_mask());
tick_broadcast_clear_oneshot(cpu);
}
}
@@ -125,9 +127,9 @@ int tick_device_uses_broadcast(struct cl
}
/*
- * Broadcast the event to the cpus, which are set in the mask
+ * Broadcast the event to the cpus, which are set in the mask (mangled).
*/
-static void tick_do_broadcast(cpumask_t mask)
+static void tick_do_broadcast(struct cpumask *mask)
{
int cpu = smp_processor_id();
struct tick_device *td;
@@ -135,22 +137,21 @@ static void tick_do_broadcast(cpumask_t
/*
* Check, if the current cpu is in the mask
*/
- if (cpu_isset(cpu, mask)) {
- cpu_clear(cpu, mask);
+ if (cpumask_test_cpu(cpu, mask)) {
+ cpumask_clear_cpu(cpu, mask);
td = &per_cpu(tick_cpu_device, cpu);
td->evtdev->event_handler(td->evtdev);
}
- if (!cpus_empty(mask)) {
+ if (!cpumask_empty(mask)) {
/*
* It might be necessary to actually check whether the devices
* have different broadcast functions. For now, just use the
* one of the first device. This works as long as we have this
* misfeature only on x86 (lapic)
*/
- cpu = first_cpu(mask);
- td = &per_cpu(tick_cpu_device, cpu);
- td->evtdev->broadcast(&mask);
+ td = &per_cpu(tick_cpu_device, cpumask_first(mask));
+ td->evtdev->broadcast(mask);
}
}
@@ -160,12 +161,11 @@ static void tick_do_broadcast(cpumask_t
*/
static void tick_do_periodic_broadcast(void)
{
- cpumask_t mask;
-
spin_lock(&tick_broadcast_lock);
- cpus_and(mask, cpu_online_map, tick_broadcast_mask);
- tick_do_broadcast(mask);
+ cpumask_and(to_cpumask(tmpmask),
+ cpu_online_mask, tick_get_broadcast_mask());
+ tick_do_broadcast(to_cpumask(tmpmask));
spin_unlock(&tick_broadcast_lock);
}
@@ -228,13 +228,13 @@ static void tick_do_broadcast_on_off(voi
if (!tick_device_is_functional(dev))
goto out;
- bc_stopped = cpus_empty(tick_broadcast_mask);
+ bc_stopped = cpumask_empty(tick_get_broadcast_mask());
switch (*reason) {
case CLOCK_EVT_NOTIFY_BROADCAST_ON:
case CLOCK_EVT_NOTIFY_BROADCAST_FORCE:
- if (!cpu_isset(cpu, tick_broadcast_mask)) {
- cpu_set(cpu, tick_broadcast_mask);
+ if (!cpumask_test_cpu(cpu, tick_get_broadcast_mask())) {
+ cpumask_set_cpu(cpu, tick_get_broadcast_mask());
if (tick_broadcast_device.mode ==
TICKDEV_MODE_PERIODIC)
clockevents_shutdown(dev);
@@ -244,8 +244,8 @@ static void tick_do_broadcast_on_off(voi
break;
case CLOCK_EVT_NOTIFY_BROADCAST_OFF:
if (!tick_broadcast_force &&
- cpu_isset(cpu, tick_broadcast_mask)) {
- cpu_clear(cpu, tick_broadcast_mask);
+ cpumask_test_cpu(cpu, tick_get_broadcast_mask())) {
+ cpumask_clear_cpu(cpu, tick_get_broadcast_mask());
if (tick_broadcast_device.mode ==
TICKDEV_MODE_PERIODIC)
tick_setup_periodic(dev, 0);
@@ -253,7 +253,7 @@ static void tick_do_broadcast_on_off(voi
break;
}
- if (cpus_empty(tick_broadcast_mask)) {
+ if (cpumask_empty(tick_get_broadcast_mask())) {
if (!bc_stopped)
clockevents_shutdown(bc);
} else if (bc_stopped) {
@@ -272,7 +272,7 @@ out:
*/
void tick_broadcast_on_off(unsigned long reason, int *oncpu)
{
- if (!cpu_isset(*oncpu, cpu_online_map))
+ if (!cpumask_test_cpu(*oncpu, cpu_online_mask))
printk(KERN_ERR "tick-broadcast: ignoring broadcast for "
"offline CPU #%d\n", *oncpu);
else
@@ -303,10 +303,10 @@ void tick_shutdown_broadcast(unsigned in
spin_lock_irqsave(&tick_broadcast_lock, flags);
bc = tick_broadcast_device.evtdev;
- cpu_clear(cpu, tick_broadcast_mask);
+ cpumask_clear_cpu(cpu, tick_get_broadcast_mask());
if (tick_broadcast_device.mode == TICKDEV_MODE_PERIODIC) {
- if (bc && cpus_empty(tick_broadcast_mask))
+ if (bc && cpumask_empty(tick_get_broadcast_mask()))
clockevents_shutdown(bc);
}
@@ -342,10 +342,10 @@ int tick_resume_broadcast(void)
switch (tick_broadcast_device.mode) {
case TICKDEV_MODE_PERIODIC:
- if(!cpus_empty(tick_broadcast_mask))
+ if (!cpumask_empty(tick_get_broadcast_mask()))
tick_broadcast_start_periodic(bc);
- broadcast = cpu_isset(smp_processor_id(),
- tick_broadcast_mask);
+ broadcast = cpumask_test_cpu(smp_processor_id(),
+ tick_get_broadcast_mask());
break;
case TICKDEV_MODE_ONESHOT:
broadcast = tick_resume_broadcast_oneshot(bc);
@@ -360,14 +360,15 @@ int tick_resume_broadcast(void)
#ifdef CONFIG_TICK_ONESHOT
-static cpumask_t tick_broadcast_oneshot_mask;
+/* FIXME: use cpumask_var_t. */
+static DECLARE_BITMAP(tick_broadcast_oneshot_mask, CONFIG_NR_CPUS);
/*
- * Debugging: see timer_list.c
+ * Exposed for debugging: see timer_list.c
*/
-cpumask_t *tick_get_broadcast_oneshot_mask(void)
+struct cpumask *tick_get_broadcast_oneshot_mask(void)
{
- return &tick_broadcast_oneshot_mask;
+ return to_cpumask(tick_broadcast_oneshot_mask);
}
static int tick_broadcast_set_event(ktime_t expires, int force)
@@ -389,7 +390,7 @@ int tick_resume_broadcast_oneshot(struct
*/
void tick_check_oneshot_broadcast(int cpu)
{
- if (cpu_isset(cpu, tick_broadcast_oneshot_mask)) {
+ if (cpumask_test_cpu(cpu, to_cpumask(tick_broadcast_oneshot_mask))) {
struct tick_device *td = &per_cpu(tick_cpu_device, cpu);
clockevents_set_mode(td->evtdev, CLOCK_EVT_MODE_ONESHOT);
@@ -402,7 +403,6 @@ void tick_check_oneshot_broadcast(int cp
static void tick_handle_oneshot_broadcast(struct clock_event_device *dev)
{
struct tick_device *td;
- cpumask_t mask;
ktime_t now, next_event;
int cpu;
@@ -410,13 +410,13 @@ static void tick_handle_oneshot_broadcas
again:
dev->next_event.tv64 = KTIME_MAX;
next_event.tv64 = KTIME_MAX;
- mask = CPU_MASK_NONE;
+ cpumask_clear(to_cpumask(tmpmask));
now = ktime_get();
/* Find all expired events */
- for_each_cpu_mask_nr(cpu, tick_broadcast_oneshot_mask) {
+ for_each_cpu(cpu, tick_get_broadcast_oneshot_mask()) {
td = &per_cpu(tick_cpu_device, cpu);
if (td->evtdev->next_event.tv64 <= now.tv64)
- cpu_set(cpu, mask);
+ cpumask_set_cpu(cpu, to_cpumask(tmpmask));
else if (td->evtdev->next_event.tv64 < next_event.tv64)
next_event.tv64 = td->evtdev->next_event.tv64;
}
@@ -424,7 +424,7 @@ again:
/*
* Wakeup the cpus which have an expired event.
*/
- tick_do_broadcast(mask);
+ tick_do_broadcast(to_cpumask(tmpmask));
/*
* Two reasons for reprogram:
@@ -476,15 +476,16 @@ void tick_broadcast_oneshot_control(unsi
goto out;
if (reason == CLOCK_EVT_NOTIFY_BROADCAST_ENTER) {
- if (!cpu_isset(cpu, tick_broadcast_oneshot_mask)) {
- cpu_set(cpu, tick_broadcast_oneshot_mask);
+ if (!cpumask_test_cpu(cpu, tick_get_broadcast_oneshot_mask())) {
+ cpumask_set_cpu(cpu, tick_get_broadcast_oneshot_mask());
clockevents_set_mode(dev, CLOCK_EVT_MODE_SHUTDOWN);
if (dev->next_event.tv64 < bc->next_event.tv64)
tick_broadcast_set_event(dev->next_event, 1);
}
} else {
- if (cpu_isset(cpu, tick_broadcast_oneshot_mask)) {
- cpu_clear(cpu, tick_broadcast_oneshot_mask);
+ if (cpumask_test_cpu(cpu, tick_get_broadcast_oneshot_mask())) {
+ cpumask_clear_cpu(cpu,
+ tick_get_broadcast_oneshot_mask());
clockevents_set_mode(dev, CLOCK_EVT_MODE_ONESHOT);
if (dev->next_event.tv64 != KTIME_MAX)
tick_program_event(dev->next_event, 1);
@@ -502,15 +503,16 @@ out:
*/
static void tick_broadcast_clear_oneshot(int cpu)
{
- cpu_clear(cpu, tick_broadcast_oneshot_mask);
+ cpumask_clear_cpu(cpu, tick_get_broadcast_oneshot_mask());
}
-static void tick_broadcast_init_next_event(cpumask_t *mask, ktime_t expires)
+static void tick_broadcast_init_next_event(struct cpumask *mask,
+ ktime_t expires)
{
struct tick_device *td;
int cpu;
- for_each_cpu_mask_nr(cpu, *mask) {
+ for_each_cpu(cpu, mask) {
td = &per_cpu(tick_cpu_device, cpu);
if (td->evtdev)
td->evtdev->next_event = expires;
@@ -526,7 +528,6 @@ void tick_broadcast_setup_oneshot(struct
if (bc->event_handler != tick_handle_oneshot_broadcast) {
int was_periodic = bc->mode == CLOCK_EVT_MODE_PERIODIC;
int cpu = smp_processor_id();
- cpumask_t mask;
bc->event_handler = tick_handle_oneshot_broadcast;
clockevents_set_mode(bc, CLOCK_EVT_MODE_ONESHOT);
@@ -540,13 +541,15 @@ void tick_broadcast_setup_oneshot(struct
* oneshot_mask bits for those and program the
* broadcast device to fire.
*/
- mask = tick_broadcast_mask;
- cpu_clear(cpu, mask);
- cpus_or(tick_broadcast_oneshot_mask,
- tick_broadcast_oneshot_mask, mask);
-
- if (was_periodic && !cpus_empty(mask)) {
- tick_broadcast_init_next_event(&mask, tick_next_period);
+ cpumask_copy(to_cpumask(tmpmask), tick_get_broadcast_mask());
+ cpumask_clear_cpu(cpu, to_cpumask(tmpmask));
+ cpumask_or(tick_get_broadcast_oneshot_mask(),
+ tick_get_broadcast_oneshot_mask(),
+ to_cpumask(tmpmask));
+
+ if (was_periodic && !cpumask_empty(to_cpumask(tmpmask))) {
+ tick_broadcast_init_next_event(to_cpumask(tmpmask),
+ tick_next_period);
tick_broadcast_set_event(tick_next_period, 1);
} else
bc->next_event.tv64 = KTIME_MAX;
@@ -585,7 +588,7 @@ void tick_shutdown_broadcast_oneshot(uns
* Clear the broadcast mask flag for the dead cpu, but do not
* stop the broadcast device!
*/
- cpu_clear(cpu, tick_broadcast_oneshot_mask);
+ cpumask_clear_cpu(cpu, tick_get_broadcast_oneshot_mask());
spin_unlock_irqrestore(&tick_broadcast_lock, flags);
}
--- linux-2.6-for-ingo.orig/kernel/time/tick-common.c
+++ linux-2.6-for-ingo/kernel/time/tick-common.c
@@ -254,7 +254,7 @@ static int tick_check_new_device(struct
curdev = NULL;
}
clockevents_exchange_device(curdev, newdev);
- tick_setup_device(td, newdev, cpu, &cpumask_of_cpu(cpu));
+ tick_setup_device(td, newdev, cpu, cpumask_of(cpu));
if (newdev->features & CLOCK_EVT_FEAT_ONESHOT)
tick_oneshot_notify();
@@ -299,9 +299,9 @@ static void tick_shutdown(unsigned int *
}
/* Transfer the do_timer job away from this cpu */
if (*cpup == tick_do_timer_cpu) {
- int cpu = first_cpu(cpu_online_map);
+ int cpu = cpumask_first(cpu_online_mask);
- tick_do_timer_cpu = (cpu != NR_CPUS) ? cpu :
+ tick_do_timer_cpu = (cpu < nr_cpu_ids) ? cpu :
TICK_DO_TIMER_NONE;
}
spin_unlock_irqrestore(&tick_device_lock, flags);
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 4/8] cpumask: convert kernel trace functions
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
` (2 preceding siblings ...)
2008-12-19 16:01 ` [PATCH 3/8] cpumask: convert kernel time functions Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
2008-12-19 16:16 ` Steven Rostedt
2008-12-19 16:01 ` [PATCH 5/8] cpumask: convert rest of files in kernel/ Mike Travis
` (3 subsequent siblings)
7 siblings, 1 reply; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Rusty Russell, linux-kernel, Steven Rostedt
[-- Attachment #1: cpumask:convert-kernel_trace.patch --]
[-- Type: text/plain, Size: 11402 bytes --]
Impact: Reduce memory usage, use new cpumask API.
Convert kernel trace functions to use struct cpumask.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
---
kernel/trace/ring_buffer.c | 42 +++++++++++++++++------------
kernel/trace/trace.c | 62 +++++++++++++++++++++++++------------------
kernel/trace/trace_sysprof.c | 13 ++-------
3 files changed, 65 insertions(+), 52 deletions(-)
--- linux-2.6-for-ingo.orig/kernel/trace/ring_buffer.c
+++ linux-2.6-for-ingo/kernel/trace/ring_buffer.c
@@ -189,7 +189,7 @@ void *ring_buffer_event_data(struct ring
}
#define for_each_buffer_cpu(buffer, cpu) \
- for_each_cpu_mask(cpu, buffer->cpumask)
+ for_each_cpu(cpu, buffer->cpumask)
#define TS_SHIFT 27
#define TS_MASK ((1ULL << TS_SHIFT) - 1)
@@ -262,7 +262,7 @@ struct ring_buffer {
unsigned pages;
unsigned flags;
int cpus;
- cpumask_t cpumask;
+ cpumask_var_t cpumask;
atomic_t record_disabled;
struct mutex mutex;
@@ -453,6 +453,9 @@ struct ring_buffer *ring_buffer_alloc(un
if (!buffer)
return NULL;
+ if (!alloc_cpumask_var(&buffer->cpumask, GFP_KERNEL))
+ goto fail_free_buffer;
+
buffer->pages = DIV_ROUND_UP(size, BUF_PAGE_SIZE);
buffer->flags = flags;
@@ -460,14 +463,14 @@ struct ring_buffer *ring_buffer_alloc(un
if (buffer->pages == 1)
buffer->pages++;
- buffer->cpumask = cpu_possible_map;
+ cpumask_copy(buffer->cpumask, cpu_possible_mask);
buffer->cpus = nr_cpu_ids;
bsize = sizeof(void *) * nr_cpu_ids;
buffer->buffers = kzalloc(ALIGN(bsize, cache_line_size()),
GFP_KERNEL);
if (!buffer->buffers)
- goto fail_free_buffer;
+ goto fail_free_cpumask;
for_each_buffer_cpu(buffer, cpu) {
buffer->buffers[cpu] =
@@ -487,6 +490,9 @@ struct ring_buffer *ring_buffer_alloc(un
}
kfree(buffer->buffers);
+ fail_free_cpumask:
+ free_cpumask_var(buffer->cpumask);
+
fail_free_buffer:
kfree(buffer);
return NULL;
@@ -504,6 +510,8 @@ ring_buffer_free(struct ring_buffer *buf
for_each_buffer_cpu(buffer, cpu)
rb_free_cpu_buffer(buffer->buffers[cpu]);
+ free_cpumask_var(buffer->cpumask);
+
kfree(buffer);
}
@@ -1260,7 +1268,7 @@ ring_buffer_lock_reserve(struct ring_buf
cpu = raw_smp_processor_id();
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
goto out;
cpu_buffer = buffer->buffers[cpu];
@@ -1371,7 +1379,7 @@ int ring_buffer_write(struct ring_buffer
cpu = raw_smp_processor_id();
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
goto out;
cpu_buffer = buffer->buffers[cpu];
@@ -1450,7 +1458,7 @@ void ring_buffer_record_disable_cpu(stru
{
struct ring_buffer_per_cpu *cpu_buffer;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return;
cpu_buffer = buffer->buffers[cpu];
@@ -1469,7 +1477,7 @@ void ring_buffer_record_enable_cpu(struc
{
struct ring_buffer_per_cpu *cpu_buffer;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return;
cpu_buffer = buffer->buffers[cpu];
@@ -1485,7 +1493,7 @@ unsigned long ring_buffer_entries_cpu(st
{
struct ring_buffer_per_cpu *cpu_buffer;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return 0;
cpu_buffer = buffer->buffers[cpu];
@@ -1501,7 +1509,7 @@ unsigned long ring_buffer_overrun_cpu(st
{
struct ring_buffer_per_cpu *cpu_buffer;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return 0;
cpu_buffer = buffer->buffers[cpu];
@@ -1814,7 +1822,7 @@ rb_buffer_peek(struct ring_buffer *buffe
struct buffer_page *reader;
int nr_loops = 0;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return NULL;
cpu_buffer = buffer->buffers[cpu];
@@ -1987,7 +1995,7 @@ ring_buffer_consume(struct ring_buffer *
struct ring_buffer_event *event;
unsigned long flags;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return NULL;
spin_lock_irqsave(&cpu_buffer->reader_lock, flags);
@@ -2023,7 +2031,7 @@ ring_buffer_read_start(struct ring_buffe
struct ring_buffer_iter *iter;
unsigned long flags;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return NULL;
iter = kmalloc(sizeof(*iter), GFP_KERNEL);
@@ -2129,7 +2137,7 @@ void ring_buffer_reset_cpu(struct ring_b
struct ring_buffer_per_cpu *cpu_buffer = buffer->buffers[cpu];
unsigned long flags;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return;
spin_lock_irqsave(&cpu_buffer->reader_lock, flags);
@@ -2182,7 +2190,7 @@ int ring_buffer_empty_cpu(struct ring_bu
{
struct ring_buffer_per_cpu *cpu_buffer;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return 1;
cpu_buffer = buffer->buffers[cpu];
@@ -2205,8 +2213,8 @@ int ring_buffer_swap_cpu(struct ring_buf
struct ring_buffer_per_cpu *cpu_buffer_a;
struct ring_buffer_per_cpu *cpu_buffer_b;
- if (!cpu_isset(cpu, buffer_a->cpumask) ||
- !cpu_isset(cpu, buffer_b->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer_a->cpumask) ||
+ !cpumask_test_cpu(cpu, buffer_b->cpumask))
return -EINVAL;
/* At least make sure the two buffers are somewhat the same */
--- linux-2.6-for-ingo.orig/kernel/trace/trace.c
+++ linux-2.6-for-ingo/kernel/trace/trace.c
@@ -90,10 +90,10 @@ static inline void ftrace_enable_cpu(voi
preempt_enable();
}
-static cpumask_t __read_mostly tracing_buffer_mask;
+static cpumask_var_t __read_mostly tracing_buffer_mask;
#define for_each_tracing_cpu(cpu) \
- for_each_cpu_mask(cpu, tracing_buffer_mask)
+ for_each_cpu(cpu, tracing_buffer_mask)
/*
* ftrace_dump_on_oops - variable to dump ftrace buffer on oops
@@ -2618,13 +2618,7 @@ static struct file_operations show_trace
/*
* Only trace on a CPU if the bitmask is set:
*/
-static cpumask_t tracing_cpumask = CPU_MASK_ALL;
-
-/*
- * When tracing/tracing_cpu_mask is modified then this holds
- * the new bitmask we are about to install:
- */
-static cpumask_t tracing_cpumask_new;
+static cpumask_var_t tracing_cpumask;
/*
* The tracer itself will not take this lock, but still we want
@@ -2646,7 +2640,7 @@ tracing_cpumask_read(struct file *filp,
mutex_lock(&tracing_cpumask_update_lock);
- len = cpumask_scnprintf(mask_str, count, &tracing_cpumask);
+ len = cpumask_scnprintf(mask_str, count, tracing_cpumask);
if (count - len < 2) {
count = -EINVAL;
goto out_err;
@@ -2665,9 +2659,13 @@ tracing_cpumask_write(struct file *filp,
size_t count, loff_t *ppos)
{
int err, cpu;
+ cpumask_var_t tracing_cpumask_new;
+
+ if (!alloc_cpumask_var(&tracing_cpumask_new, GFP_KERNEL))
+ return -ENOMEM;
mutex_lock(&tracing_cpumask_update_lock);
- err = cpumask_parse_user(ubuf, count, &tracing_cpumask_new);
+ err = cpumask_parse_user(ubuf, count, tracing_cpumask_new);
if (err)
goto err_unlock;
@@ -2678,26 +2676,28 @@ tracing_cpumask_write(struct file *filp,
* Increase/decrease the disabled counter if we are
* about to flip a bit in the cpumask:
*/
- if (cpu_isset(cpu, tracing_cpumask) &&
- !cpu_isset(cpu, tracing_cpumask_new)) {
+ if (cpumask_test_cpu(cpu, tracing_cpumask) &&
+ !cpumask_test_cpu(cpu, tracing_cpumask_new)) {
atomic_inc(&global_trace.data[cpu]->disabled);
}
- if (!cpu_isset(cpu, tracing_cpumask) &&
- cpu_isset(cpu, tracing_cpumask_new)) {
+ if (!cpumask_test_cpu(cpu, tracing_cpumask) &&
+ cpumask_test_cpu(cpu, tracing_cpumask_new)) {
atomic_dec(&global_trace.data[cpu]->disabled);
}
}
__raw_spin_unlock(&ftrace_max_lock);
local_irq_enable();
- tracing_cpumask = tracing_cpumask_new;
+ cpumask_copy(tracing_cpumask, tracing_cpumask_new);
mutex_unlock(&tracing_cpumask_update_lock);
+ free_cpumask_var(tracing_cpumask_new);
return count;
err_unlock:
mutex_unlock(&tracing_cpumask_update_lock);
+ free_cpumask_var(tracing_cpumask);
return err;
}
@@ -3724,7 +3724,6 @@ void ftrace_dump(void)
static DEFINE_SPINLOCK(ftrace_dump_lock);
/* use static because iter can be a bit big for the stack */
static struct trace_iterator iter;
- static cpumask_t mask;
static int dump_ran;
unsigned long flags;
int cnt = 0, cpu;
@@ -3758,8 +3757,6 @@ void ftrace_dump(void)
* and then release the locks again.
*/
- cpus_clear(mask);
-
while (!trace_empty(&iter)) {
if (!cnt)
@@ -3795,19 +3792,28 @@ __init static int tracer_alloc_buffers(v
{
struct trace_array_cpu *data;
int i;
+ int ret = -ENOMEM;
- /* TODO: make the number of buffers hot pluggable with CPUS */
- tracing_buffer_mask = cpu_possible_map;
+ if (!alloc_cpumask_var(&tracing_buffer_mask, GFP_KERNEL))
+ goto out;
+
+ if (!alloc_cpumask_var(&tracing_cpumask, GFP_KERNEL))
+ goto out_free_buffer_mask;
+ cpumask_copy(tracing_buffer_mask, cpu_possible_mask);
+ cpumask_copy(tracing_cpumask, cpu_all_mask);
+
+ /* TODO: make the number of buffers hot pluggable with CPUS */
global_trace.buffer = ring_buffer_alloc(trace_buf_size,
TRACE_BUFFER_FLAGS);
if (!global_trace.buffer) {
printk(KERN_ERR "tracer: failed to allocate ring buffer!\n");
WARN_ON(1);
- return 0;
+ goto out_free_cpumask;
}
global_trace.entries = ring_buffer_size(global_trace.buffer);
+
#ifdef CONFIG_TRACER_MAX_TRACE
max_tr.buffer = ring_buffer_alloc(trace_buf_size,
TRACE_BUFFER_FLAGS);
@@ -3815,7 +3821,7 @@ __init static int tracer_alloc_buffers(v
printk(KERN_ERR "tracer: failed to allocate max ring buffer!\n");
WARN_ON(1);
ring_buffer_free(global_trace.buffer);
- return 0;
+ goto out_free_cpumask;
}
max_tr.entries = ring_buffer_size(max_tr.buffer);
WARN_ON(max_tr.entries != global_trace.entries);
@@ -3842,11 +3848,17 @@ __init static int tracer_alloc_buffers(v
tracing_disabled = 0;
atomic_notifier_chain_register(&panic_notifier_list,
- &trace_panic_notifier);
+ &trace_panic_notifier);
register_die_notifier(&trace_die_notifier);
+ ret = 0;
- return 0;
+out_free_cpumask:
+ free_cpumask_var(tracing_cpumask);
+out_free_buffer_mask:
+ free_cpumask_var(tracing_buffer_mask);
+out:
+ return ret;
}
early_initcall(tracer_alloc_buffers);
fs_initcall(tracer_init_debugfs);
--- linux-2.6-for-ingo.orig/kernel/trace/trace_sysprof.c
+++ linux-2.6-for-ingo/kernel/trace/trace_sysprof.c
@@ -196,9 +196,9 @@ static enum hrtimer_restart stack_trace_
return HRTIMER_RESTART;
}
-static void start_stack_timer(int cpu)
+static void start_stack_timer(void *unused)
{
- struct hrtimer *hrtimer = &per_cpu(stack_trace_hrtimer, cpu);
+ struct hrtimer *hrtimer = &__get_cpu_var(stack_trace_hrtimer);
hrtimer_init(hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
hrtimer->function = stack_trace_timer_fn;
@@ -209,14 +209,7 @@ static void start_stack_timer(int cpu)
static void start_stack_timers(void)
{
- cpumask_t saved_mask = current->cpus_allowed;
- int cpu;
-
- for_each_online_cpu(cpu) {
- set_cpus_allowed_ptr(current, &cpumask_of_cpu(cpu));
- start_stack_timer(cpu);
- }
- set_cpus_allowed_ptr(current, &saved_mask);
+ on_each_cpu(start_stack_timer, NULL, 1);
}
static void stop_stack_timer(int cpu)
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 5/8] cpumask: convert rest of files in kernel/
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
` (3 preceding siblings ...)
2008-12-19 16:01 ` [PATCH 4/8] cpumask: convert kernel trace functions Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
2008-12-20 1:33 ` Lai Jiangshan
2008-12-20 13:02 ` Rusty Russell
2008-12-19 16:01 ` [PATCH 6/8] cpumask: convert kernel mm functions Mike Travis
` (2 subsequent siblings)
7 siblings, 2 replies; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar
Cc: Rusty Russell, linux-kernel, Dipankar Sarma, H. Peter Anvin,
Lai Jiangshan, Max Krasnyansky, Richard Henderson,
Thomas Gleixner, Yinghai Lu
[-- Attachment #1: cpumask:convert-kernel.patch --]
[-- Type: text/plain, Size: 27962 bytes --]
Impact: Reduce stack usage, use new cpumask API. ALPHA mod!
A couple of uses of cpumask_any_but() here to avoid temporary cpumasks
and carry the "const" modifier through the seq_cpumask() function all the
way to bitmap_scnprintf().
Note that prof_cpu_mask will be difficult to convert to a cpumask_var_t
since it needs to be ready from the first scheduler tick.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Cc: Dipankar Sarma <dipankar@in.ibm.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
Cc: Max Krasnyansky <maxk@qualcomm.com>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Yinghai Lu <yinghai@kernel.org>
---
arch/alpha/kernel/irq.c | 3 ++-
fs/seq_file.c | 3 ++-
include/linux/interrupt.h | 2 +-
include/linux/rcuclassic.h | 4 ++--
include/linux/seq_file.h | 5 +++--
include/linux/stop_machine.h | 6 +++---
kernel/cpu.c | 40 +++++++++++++++++++++-------------------
kernel/irq/manage.c | 11 +++++++++--
kernel/irq/proc.c | 32 +++++++++++++++++++++-----------
kernel/power/poweroff.c | 2 +-
kernel/profile.c | 31 ++++++++++++++++++-------------
kernel/rcuclassic.c | 32 +++++++++++++++++---------------
kernel/rcupreempt.c | 19 ++++++++++---------
kernel/rcutorture.c | 27 +++++++++++++++------------
kernel/softlockup.c | 6 ++----
kernel/stop_machine.c | 8 ++++----
kernel/taskstats.c | 39 ++++++++++++++++++++++++---------------
17 files changed, 155 insertions(+), 115 deletions(-)
--- linux-2.6-for-ingo.orig/arch/alpha/kernel/irq.c
+++ linux-2.6-for-ingo/arch/alpha/kernel/irq.c
@@ -50,7 +50,8 @@ int irq_select_affinity(unsigned int irq
if (!irq_desc[irq].chip->set_affinity || irq_user_affinity[irq])
return 1;
- while (!cpu_possible(cpu) || !cpu_isset(cpu, irq_default_affinity))
+ while (!cpu_possible(cpu) ||
+ !cpumask_test_cpu(cpu, irq_default_affinity))
cpu = (cpu < (NR_CPUS-1) ? cpu + 1 : 0);
last_cpu = cpu;
--- linux-2.6-for-ingo.orig/fs/seq_file.c
+++ linux-2.6-for-ingo/fs/seq_file.c
@@ -462,7 +462,8 @@ int seq_dentry(struct seq_file *m, struc
return -1;
}
-int seq_bitmap(struct seq_file *m, unsigned long *bits, unsigned int nr_bits)
+int seq_bitmap(struct seq_file *m, const unsigned long *bits,
+ unsigned int nr_bits)
{
if (m->count < m->size) {
int len = bitmap_scnprintf(m->buf + m->count,
--- linux-2.6-for-ingo.orig/include/linux/interrupt.h
+++ linux-2.6-for-ingo/include/linux/interrupt.h
@@ -109,7 +109,7 @@ extern void enable_irq(unsigned int irq)
#if defined(CONFIG_SMP) && defined(CONFIG_GENERIC_HARDIRQS)
-extern cpumask_t irq_default_affinity;
+extern cpumask_var_t irq_default_affinity;
extern int irq_set_affinity(unsigned int irq, const struct cpumask *cpumask);
extern int irq_can_set_affinity(unsigned int irq);
--- linux-2.6-for-ingo.orig/include/linux/rcuclassic.h
+++ linux-2.6-for-ingo/include/linux/rcuclassic.h
@@ -59,8 +59,8 @@ struct rcu_ctrlblk {
int signaled;
spinlock_t lock ____cacheline_internodealigned_in_smp;
- cpumask_t cpumask; /* CPUs that need to switch in order */
- /* for current batch to proceed. */
+ DECLARE_BITMAP(cpumask, NR_CPUS); /* CPUs that need to switch for */
+ /* current batch to proceed. */
} ____cacheline_internodealigned_in_smp;
/* Is batch a before batch b ? */
--- linux-2.6-for-ingo.orig/include/linux/seq_file.h
+++ linux-2.6-for-ingo/include/linux/seq_file.h
@@ -50,8 +50,9 @@ int seq_path(struct seq_file *, struct p
int seq_dentry(struct seq_file *, struct dentry *, char *);
int seq_path_root(struct seq_file *m, struct path *path, struct path *root,
char *esc);
-int seq_bitmap(struct seq_file *m, unsigned long *bits, unsigned int nr_bits);
-static inline int seq_cpumask(struct seq_file *m, cpumask_t *mask)
+int seq_bitmap(struct seq_file *m, const unsigned long *bits,
+ unsigned int nr_bits);
+static inline int seq_cpumask(struct seq_file *m, const struct cpumask *mask)
{
return seq_bitmap(m, mask->bits, NR_CPUS);
}
--- linux-2.6-for-ingo.orig/include/linux/stop_machine.h
+++ linux-2.6-for-ingo/include/linux/stop_machine.h
@@ -23,7 +23,7 @@
*
* This can be thought of as a very heavy write lock, equivalent to
* grabbing every spinlock in the kernel. */
-int stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus);
+int stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus);
/**
* __stop_machine: freeze the machine on all CPUs and run this function
@@ -34,11 +34,11 @@ int stop_machine(int (*fn)(void *), void
* Description: This is a special version of the above, which assumes cpus
* won't come or go while it's being called. Used by hotplug cpu.
*/
-int __stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus);
+int __stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus);
#else
static inline int stop_machine(int (*fn)(void *), void *data,
- const cpumask_t *cpus)
+ const struct cpumask *cpus)
{
int ret;
local_irq_disable();
--- linux-2.6-for-ingo.orig/kernel/cpu.c
+++ linux-2.6-for-ingo/kernel/cpu.c
@@ -218,7 +218,7 @@ static int __ref take_cpu_down(void *_pa
static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
{
int err, nr_calls = 0;
- cpumask_t old_allowed, tmp;
+ cpumask_var_t old_allowed;
void *hcpu = (void *)(long)cpu;
unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
struct take_cpu_down_param tcd_param = {
@@ -232,6 +232,9 @@ static int __ref _cpu_down(unsigned int
if (!cpu_online(cpu))
return -EINVAL;
+ if (!alloc_cpumask_var(&old_allowed, GFP_KERNEL))
+ return -ENOMEM;
+
cpu_hotplug_begin();
err = __raw_notifier_call_chain(&cpu_chain, CPU_DOWN_PREPARE | mod,
hcpu, -1, &nr_calls);
@@ -246,13 +249,11 @@ static int __ref _cpu_down(unsigned int
}
/* Ensure that we are not runnable on dying cpu */
- old_allowed = current->cpus_allowed;
- cpus_setall(tmp);
- cpu_clear(cpu, tmp);
- set_cpus_allowed_ptr(current, &tmp);
- tmp = cpumask_of_cpu(cpu);
+ cpumask_copy(old_allowed, ¤t->cpus_allowed);
+ set_cpus_allowed_ptr(current,
+ cpumask_of(cpumask_any_but(cpu_online_mask, cpu)));
- err = __stop_machine(take_cpu_down, &tcd_param, &tmp);
+ err = __stop_machine(take_cpu_down, &tcd_param, cpumask_of(cpu));
if (err) {
/* CPU didn't die: tell everyone. Can't complain. */
if (raw_notifier_call_chain(&cpu_chain, CPU_DOWN_FAILED | mod,
@@ -278,7 +279,7 @@ static int __ref _cpu_down(unsigned int
check_for_tasks(cpu);
out_allowed:
- set_cpus_allowed_ptr(current, &old_allowed);
+ set_cpus_allowed_ptr(current, old_allowed);
out_release:
cpu_hotplug_done();
if (!err) {
@@ -286,6 +287,7 @@ out_release:
hcpu) == NOTIFY_BAD)
BUG();
}
+ free_cpumask_var(old_allowed);
return err;
}
@@ -304,7 +306,7 @@ int __ref cpu_down(unsigned int cpu)
/*
* Make sure the all cpus did the reschedule and are not
- * using stale version of the cpu_active_map.
+ * using stale version of the cpu_active_mask.
* This is not strictly necessary becuase stop_machine()
* that we run down the line already provides the required
* synchronization. But it's really a side effect and we do not
@@ -368,7 +370,7 @@ out_notify:
int __cpuinit cpu_up(unsigned int cpu)
{
int err = 0;
- if (!cpu_isset(cpu, cpu_possible_map)) {
+ if (!cpu_possible(cpu)) {
printk(KERN_ERR "can't online cpu %d because it is not "
"configured as may-hotadd at boot time\n", cpu);
#if defined(CONFIG_IA64) || defined(CONFIG_X86_64)
@@ -393,25 +395,25 @@ out:
}
#ifdef CONFIG_PM_SLEEP_SMP
-static cpumask_t frozen_cpus;
+static DECLARE_BITMAP(frozen_cpus, CONFIG_NR_CPUS);
int disable_nonboot_cpus(void)
{
int cpu, first_cpu, error = 0;
cpu_maps_update_begin();
- first_cpu = first_cpu(cpu_online_map);
+ first_cpu = cpumask_first(cpu_online_mask);
/* We take down all of the non-boot CPUs in one shot to avoid races
* with the userspace trying to use the CPU hotplug at the same time
*/
- cpus_clear(frozen_cpus);
+ cpumask_clear(to_cpumask(frozen_cpus));
printk("Disabling non-boot CPUs ...\n");
for_each_online_cpu(cpu) {
if (cpu == first_cpu)
continue;
error = _cpu_down(cpu, 1);
if (!error) {
- cpu_set(cpu, frozen_cpus);
+ cpumask_set_cpu(cpu, to_cpumask(frozen_cpus));
printk("CPU%d is down\n", cpu);
} else {
printk(KERN_ERR "Error taking CPU%d down: %d\n",
@@ -437,11 +439,11 @@ void __ref enable_nonboot_cpus(void)
/* Allow everyone to use the CPU hotplug again */
cpu_maps_update_begin();
cpu_hotplug_disabled = 0;
- if (cpus_empty(frozen_cpus))
+ if (cpumask_empty(to_cpumask(frozen_cpus)))
goto out;
printk("Enabling non-boot CPUs ...\n");
- for_each_cpu_mask_nr(cpu, frozen_cpus) {
+ for_each_cpu(cpu, to_cpumask(frozen_cpus)) {
error = _cpu_up(cpu, 1);
if (!error) {
printk("CPU%d is up\n", cpu);
@@ -449,7 +451,7 @@ void __ref enable_nonboot_cpus(void)
}
printk(KERN_WARNING "Error taking CPU%d up: %d\n", cpu, error);
}
- cpus_clear(frozen_cpus);
+ cpumask_clear(to_cpumask(frozen_cpus));
out:
cpu_maps_update_done();
}
@@ -468,7 +470,7 @@ void __cpuinit notify_cpu_starting(unsig
unsigned long val = CPU_STARTING;
#ifdef CONFIG_PM_SLEEP_SMP
- if (cpu_isset(cpu, frozen_cpus))
+ if (cpumask_test_cpu(cpu, to_cpumask(frozen_cpus)))
val = CPU_STARTING_FROZEN;
#endif /* CONFIG_PM_SLEEP_SMP */
raw_notifier_call_chain(&cpu_chain, val, (void *)(long)cpu);
@@ -480,7 +482,7 @@ void __cpuinit notify_cpu_starting(unsig
* cpu_bit_bitmap[] is a special, "compressed" data structure that
* represents all NR_CPUS bits binary values of 1<<nr.
*
- * It is used by cpumask_of_cpu() to get a constant address to a CPU
+ * It is used by cpumask_of() to get a constant address to a CPU
* mask value that has a single bit set only.
*/
--- linux-2.6-for-ingo.orig/kernel/irq/manage.c
+++ linux-2.6-for-ingo/kernel/irq/manage.c
@@ -16,8 +16,15 @@
#include "internals.h"
#ifdef CONFIG_SMP
+cpumask_var_t irq_default_affinity;
-cpumask_t irq_default_affinity = CPU_MASK_ALL;
+static int init_irq_default_affinity(void)
+{
+ alloc_cpumask_var(&irq_default_affinity, GFP_KERNEL);
+ cpumask_setall(irq_default_affinity);
+ return 0;
+}
+core_initcall(init_irq_default_affinity);
/**
* synchronize_irq - wait for pending IRQ handlers (on other CPUs)
@@ -127,7 +134,7 @@ int do_irq_select_affinity(unsigned int
desc->status &= ~IRQ_AFFINITY_SET;
}
- cpumask_and(&desc->affinity, cpu_online_mask, &irq_default_affinity);
+ cpumask_and(&desc->affinity, cpu_online_mask, irq_default_affinity);
set_affinity:
desc->chip->set_affinity(irq, &desc->affinity);
--- linux-2.6-for-ingo.orig/kernel/irq/proc.c
+++ linux-2.6-for-ingo/kernel/irq/proc.c
@@ -20,7 +20,7 @@ static struct proc_dir_entry *root_irq_d
static int irq_affinity_proc_show(struct seq_file *m, void *v)
{
struct irq_desc *desc = irq_to_desc((long)m->private);
- cpumask_t *mask = &desc->affinity;
+ const struct cpumask *mask = &desc->affinity;
#ifdef CONFIG_GENERIC_PENDING_IRQ
if (desc->status & IRQ_MOVE_PENDING)
@@ -93,7 +93,7 @@ static const struct file_operations irq_
static int default_affinity_show(struct seq_file *m, void *v)
{
- seq_cpumask(m, &irq_default_affinity);
+ seq_cpumask(m, irq_default_affinity);
seq_putc(m, '\n');
return 0;
}
@@ -101,27 +101,37 @@ static int default_affinity_show(struct
static ssize_t default_affinity_write(struct file *file,
const char __user *buffer, size_t count, loff_t *ppos)
{
- cpumask_t new_value;
+ cpumask_var_t new_value;
int err;
- err = cpumask_parse_user(buffer, count, &new_value);
+ if (!alloc_cpumask_var(&new_value, GFP_KERNEL))
+ return -ENOMEM;
+
+ err = cpumask_parse_user(buffer, count, new_value);
if (err)
- return err;
+ goto out;
- if (!is_affinity_mask_valid(new_value))
- return -EINVAL;
+ if (!is_affinity_mask_valid(new_value)) {
+ err = -EINVAL;
+ goto out;
+ }
/*
* Do not allow disabling IRQs completely - it's a too easy
* way to make the system unusable accidentally :-) At least
* one online CPU still has to be targeted.
*/
- if (!cpus_intersects(new_value, cpu_online_map))
- return -EINVAL;
+ if (!cpumask_intersects(new_value, cpu_online_mask)) {
+ err = -EINVAL;
+ goto out;
+ }
- irq_default_affinity = new_value;
+ cpumask_copy(irq_default_affinity, new_value);
+ err = count;
- return count;
+out:
+ free_cpumask_var(new_value);
+ return err;
}
static int default_affinity_open(struct inode *inode, struct file *file)
--- linux-2.6-for-ingo.orig/kernel/power/poweroff.c
+++ linux-2.6-for-ingo/kernel/power/poweroff.c
@@ -27,7 +27,7 @@ static DECLARE_WORK(poweroff_work, do_po
static void handle_poweroff(int key, struct tty_struct *tty)
{
/* run sysrq poweroff on boot cpu */
- schedule_work_on(first_cpu(cpu_online_map), &poweroff_work);
+ schedule_work_on(cpumask_first(cpu_online_mask), &poweroff_work);
}
static struct sysrq_key_op sysrq_poweroff_op = {
--- linux-2.6-for-ingo.orig/kernel/profile.c
+++ linux-2.6-for-ingo/kernel/profile.c
@@ -45,7 +45,7 @@ static unsigned long prof_len, prof_shif
int prof_on __read_mostly;
EXPORT_SYMBOL_GPL(prof_on);
-static cpumask_t prof_cpu_mask = CPU_MASK_ALL;
+static DECLARE_BITMAP(prof_cpu_mask, CONFIG_NR_CPUS) = CPU_BITS_ALL;
#ifdef CONFIG_SMP
static DEFINE_PER_CPU(struct profile_hit *[2], cpu_profile_hits);
static DEFINE_PER_CPU(int, cpu_profile_flip);
@@ -386,13 +386,13 @@ out_free:
return NOTIFY_BAD;
case CPU_ONLINE:
case CPU_ONLINE_FROZEN:
- cpu_set(cpu, prof_cpu_mask);
+ cpumask_set_cpu(cpu, to_cpumask(prof_cpu_mask));
break;
case CPU_UP_CANCELED:
case CPU_UP_CANCELED_FROZEN:
case CPU_DEAD:
case CPU_DEAD_FROZEN:
- cpu_clear(cpu, prof_cpu_mask);
+ cpumask_clear_cpu(cpu, to_cpumask(prof_cpu_mask));
if (per_cpu(cpu_profile_hits, cpu)[0]) {
page = virt_to_page(per_cpu(cpu_profile_hits, cpu)[0]);
per_cpu(cpu_profile_hits, cpu)[0] = NULL;
@@ -430,7 +430,8 @@ void profile_tick(int type)
if (type == CPU_PROFILING && timer_hook)
timer_hook(regs);
- if (!user_mode(regs) && cpu_isset(smp_processor_id(), prof_cpu_mask))
+ if (!user_mode(regs) &&
+ cpumask_test_cpu(smp_processor_id(), to_cpumask(prof_cpu_mask)))
profile_hit(type, (void *)profile_pc(regs));
}
@@ -442,7 +443,7 @@ void profile_tick(int type)
static int prof_cpu_mask_read_proc(char *page, char **start, off_t off,
int count, int *eof, void *data)
{
- int len = cpumask_scnprintf(page, count, (cpumask_t *)data);
+ int len = cpumask_scnprintf(page, count, data);
if (count - len < 2)
return -EINVAL;
len += sprintf(page + len, "\n");
@@ -452,16 +453,20 @@ static int prof_cpu_mask_read_proc(char
static int prof_cpu_mask_write_proc(struct file *file,
const char __user *buffer, unsigned long count, void *data)
{
- cpumask_t *mask = (cpumask_t *)data;
+ struct cpumask *mask = data;
unsigned long full_count = count, err;
- cpumask_t new_value;
+ cpumask_var_t new_value;
- err = cpumask_parse_user(buffer, count, &new_value);
- if (err)
- return err;
+ if (!alloc_cpumask_var(&new_value, GFP_KERNEL))
+ return -ENOMEM;
- *mask = new_value;
- return full_count;
+ err = cpumask_parse_user(buffer, count, new_value);
+ if (!err) {
+ cpumask_copy(mask, new_value);
+ err = full_count;
+ }
+ free_cpumask_var(new_value);
+ return err;
}
void create_prof_cpu_mask(struct proc_dir_entry *root_irq_dir)
@@ -472,7 +477,7 @@ void create_prof_cpu_mask(struct proc_di
entry = create_proc_entry("prof_cpu_mask", 0600, root_irq_dir);
if (!entry)
return;
- entry->data = (void *)&prof_cpu_mask;
+ entry->data = prof_cpu_mask;
entry->read_proc = prof_cpu_mask_read_proc;
entry->write_proc = prof_cpu_mask_write_proc;
}
--- linux-2.6-for-ingo.orig/kernel/rcuclassic.c
+++ linux-2.6-for-ingo/kernel/rcuclassic.c
@@ -63,14 +63,14 @@ static struct rcu_ctrlblk rcu_ctrlblk =
.completed = -300,
.pending = -300,
.lock = __SPIN_LOCK_UNLOCKED(&rcu_ctrlblk.lock),
- .cpumask = CPU_MASK_NONE,
+ .cpumask = CPU_BITS_NONE,
};
static struct rcu_ctrlblk rcu_bh_ctrlblk = {
.cur = -300,
.completed = -300,
.pending = -300,
.lock = __SPIN_LOCK_UNLOCKED(&rcu_bh_ctrlblk.lock),
- .cpumask = CPU_MASK_NONE,
+ .cpumask = CPU_BITS_NONE,
};
DEFINE_PER_CPU(struct rcu_data, rcu_data) = { 0L };
@@ -85,7 +85,6 @@ static void force_quiescent_state(struct
struct rcu_ctrlblk *rcp)
{
int cpu;
- cpumask_t cpumask;
unsigned long flags;
set_need_resched();
@@ -96,10 +95,10 @@ static void force_quiescent_state(struct
* Don't send IPI to itself. With irqs disabled,
* rdp->cpu is the current cpu.
*
- * cpu_online_map is updated by the _cpu_down()
+ * cpu_online_mask is updated by the _cpu_down()
* using __stop_machine(). Since we're in irqs disabled
* section, __stop_machine() is not exectuting, hence
- * the cpu_online_map is stable.
+ * the cpu_online_mask is stable.
*
* However, a cpu might have been offlined _just_ before
* we disabled irqs while entering here.
@@ -107,13 +106,14 @@ static void force_quiescent_state(struct
* notification, leading to the offlined cpu's bit
* being set in the rcp->cpumask.
*
- * Hence cpumask = (rcp->cpumask & cpu_online_map) to prevent
+ * Hence cpumask = (rcp->cpumask & cpu_online_mask) to prevent
* sending smp_reschedule() to an offlined CPU.
*/
- cpus_and(cpumask, rcp->cpumask, cpu_online_map);
- cpu_clear(rdp->cpu, cpumask);
- for_each_cpu_mask_nr(cpu, cpumask)
- smp_send_reschedule(cpu);
+ for_each_cpu_and(cpu,
+ to_cpumask(rcp->cpumask), cpu_online_mask) {
+ if (cpu != rdp->cpu)
+ smp_send_reschedule(cpu);
+ }
}
spin_unlock_irqrestore(&rcp->lock, flags);
}
@@ -193,7 +193,7 @@ static void print_other_cpu_stall(struct
printk(KERN_ERR "INFO: RCU detected CPU stalls:");
for_each_possible_cpu(cpu) {
- if (cpu_isset(cpu, rcp->cpumask))
+ if (cpumask_test_cpu(cpu, to_cpumask(rcp->cpumask)))
printk(" %d", cpu);
}
printk(" (detected by %d, t=%ld jiffies)\n",
@@ -221,7 +221,8 @@ static void check_cpu_stall(struct rcu_c
long delta;
delta = jiffies - rcp->jiffies_stall;
- if (cpu_isset(smp_processor_id(), rcp->cpumask) && delta >= 0) {
+ if (cpumask_test_cpu(smp_processor_id(), to_cpumask(rcp->cpumask)) &&
+ delta >= 0) {
/* We haven't checked in, so go dump stack. */
print_cpu_stall(rcp);
@@ -393,7 +394,8 @@ static void rcu_start_batch(struct rcu_c
* unnecessarily.
*/
smp_mb();
- cpumask_andnot(&rcp->cpumask, cpu_online_mask, nohz_cpu_mask);
+ cpumask_andnot(to_cpumask(rcp->cpumask),
+ cpu_online_mask, nohz_cpu_mask);
rcp->signaled = 0;
}
@@ -406,8 +408,8 @@ static void rcu_start_batch(struct rcu_c
*/
static void cpu_quiet(int cpu, struct rcu_ctrlblk *rcp)
{
- cpu_clear(cpu, rcp->cpumask);
- if (cpus_empty(rcp->cpumask)) {
+ cpumask_clear_cpu(cpu, to_cpumask(rcp->cpumask));
+ if (cpumask_empty(to_cpumask(rcp->cpumask))) {
/* batch completed ! */
rcp->completed = rcp->cur;
rcu_start_batch(rcp);
--- linux-2.6-for-ingo.orig/kernel/rcupreempt.c
+++ linux-2.6-for-ingo/kernel/rcupreempt.c
@@ -164,7 +164,8 @@ static char *rcu_try_flip_state_names[]
{ "idle", "waitack", "waitzero", "waitmb" };
#endif /* #ifdef CONFIG_RCU_TRACE */
-static cpumask_t rcu_cpu_online_map __read_mostly = CPU_MASK_NONE;
+static DECLARE_BITMAP(rcu_cpu_online_map, CONFIG_NR_CPUS) __read_mostly
+ = CPU_BITS_NONE;
/*
* Enum and per-CPU flag to determine when each CPU has seen
@@ -748,7 +749,7 @@ rcu_try_flip_idle(void)
/* Now ask each CPU for acknowledgement of the flip. */
- for_each_cpu_mask_nr(cpu, rcu_cpu_online_map) {
+ for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map)) {
per_cpu(rcu_flip_flag, cpu) = rcu_flipped;
dyntick_save_progress_counter(cpu);
}
@@ -766,7 +767,7 @@ rcu_try_flip_waitack(void)
int cpu;
RCU_TRACE_ME(rcupreempt_trace_try_flip_a1);
- for_each_cpu_mask_nr(cpu, rcu_cpu_online_map)
+ for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map))
if (rcu_try_flip_waitack_needed(cpu) &&
per_cpu(rcu_flip_flag, cpu) != rcu_flip_seen) {
RCU_TRACE_ME(rcupreempt_trace_try_flip_ae1);
@@ -798,7 +799,7 @@ rcu_try_flip_waitzero(void)
/* Check to see if the sum of the "last" counters is zero. */
RCU_TRACE_ME(rcupreempt_trace_try_flip_z1);
- for_each_cpu_mask_nr(cpu, rcu_cpu_online_map)
+ for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map))
sum += RCU_DATA_CPU(cpu)->rcu_flipctr[lastidx];
if (sum != 0) {
RCU_TRACE_ME(rcupreempt_trace_try_flip_ze1);
@@ -813,7 +814,7 @@ rcu_try_flip_waitzero(void)
smp_mb(); /* ^^^^^^^^^^^^ */
/* Call for a memory barrier from each CPU. */
- for_each_cpu_mask_nr(cpu, rcu_cpu_online_map) {
+ for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map)) {
per_cpu(rcu_mb_flag, cpu) = rcu_mb_needed;
dyntick_save_progress_counter(cpu);
}
@@ -833,7 +834,7 @@ rcu_try_flip_waitmb(void)
int cpu;
RCU_TRACE_ME(rcupreempt_trace_try_flip_m1);
- for_each_cpu_mask_nr(cpu, rcu_cpu_online_map)
+ for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map))
if (rcu_try_flip_waitmb_needed(cpu) &&
per_cpu(rcu_mb_flag, cpu) != rcu_mb_done) {
RCU_TRACE_ME(rcupreempt_trace_try_flip_me1);
@@ -1022,7 +1023,7 @@ void rcu_offline_cpu(int cpu)
RCU_DATA_CPU(cpu)->rcu_flipctr[0] = 0;
RCU_DATA_CPU(cpu)->rcu_flipctr[1] = 0;
- cpu_clear(cpu, rcu_cpu_online_map);
+ cpumask_clear_cpu(cpu, to_cpumask(rcu_cpu_online_map));
spin_unlock_irqrestore(&rcu_ctrlblk.fliplock, flags);
@@ -1062,7 +1063,7 @@ void __cpuinit rcu_online_cpu(int cpu)
struct rcu_data *rdp;
spin_lock_irqsave(&rcu_ctrlblk.fliplock, flags);
- cpu_set(cpu, rcu_cpu_online_map);
+ cpumask_set_cpu(cpu, to_cpumask(rcu_cpu_online_map));
spin_unlock_irqrestore(&rcu_ctrlblk.fliplock, flags);
/*
@@ -1420,7 +1421,7 @@ void __init __rcu_init(void)
* We don't need protection against CPU-Hotplug here
* since
* a) If a CPU comes online while we are iterating over the
- * cpu_online_map below, we would only end up making a
+ * cpu_online_mask below, we would only end up making a
* duplicate call to rcu_online_cpu() which sets the corresponding
* CPU's mask in the rcu_cpu_online_map.
*
--- linux-2.6-for-ingo.orig/kernel/rcutorture.c
+++ linux-2.6-for-ingo/kernel/rcutorture.c
@@ -843,49 +843,52 @@ static int rcu_idle_cpu; /* Force all to
*/
static void rcu_torture_shuffle_tasks(void)
{
- cpumask_t tmp_mask;
+ cpumask_var_t tmp_mask;
int i;
- cpus_setall(tmp_mask);
+ if (!alloc_cpumask_var(&tmp_mask, GFP_KERNEL))
+ BUG();
+
+ cpumask_setall(tmp_mask);
get_online_cpus();
/* No point in shuffling if there is only one online CPU (ex: UP) */
- if (num_online_cpus() == 1) {
- put_online_cpus();
- return;
- }
+ if (num_online_cpus() == 1)
+ goto out;
if (rcu_idle_cpu != -1)
- cpu_clear(rcu_idle_cpu, tmp_mask);
+ cpumask_clear_cpu(rcu_idle_cpu, tmp_mask);
- set_cpus_allowed_ptr(current, &tmp_mask);
+ set_cpus_allowed_ptr(current, tmp_mask);
if (reader_tasks) {
for (i = 0; i < nrealreaders; i++)
if (reader_tasks[i])
set_cpus_allowed_ptr(reader_tasks[i],
- &tmp_mask);
+ tmp_mask);
}
if (fakewriter_tasks) {
for (i = 0; i < nfakewriters; i++)
if (fakewriter_tasks[i])
set_cpus_allowed_ptr(fakewriter_tasks[i],
- &tmp_mask);
+ tmp_mask);
}
if (writer_task)
- set_cpus_allowed_ptr(writer_task, &tmp_mask);
+ set_cpus_allowed_ptr(writer_task, tmp_mask);
if (stats_task)
- set_cpus_allowed_ptr(stats_task, &tmp_mask);
+ set_cpus_allowed_ptr(stats_task, tmp_mask);
if (rcu_idle_cpu == -1)
rcu_idle_cpu = num_online_cpus() - 1;
else
rcu_idle_cpu--;
+out:
put_online_cpus();
+ free_cpumask_var(tmp_mask);
}
/* Shuffle tasks across CPUs, with the intent of allowing each CPU in the
--- linux-2.6-for-ingo.orig/kernel/softlockup.c
+++ linux-2.6-for-ingo/kernel/softlockup.c
@@ -310,10 +310,8 @@ cpu_callback(struct notifier_block *nfb,
case CPU_DOWN_PREPARE:
case CPU_DOWN_PREPARE_FROZEN:
if (hotcpu == check_cpu) {
- cpumask_t temp_cpu_online_map = cpu_online_map;
-
- cpu_clear(hotcpu, temp_cpu_online_map);
- check_cpu = any_online_cpu(temp_cpu_online_map);
+ /* Pick any other online cpu. */
+ check_cpu = cpumask_any_but(cpu_online_mask, hotcpu);
}
break;
--- linux-2.6-for-ingo.orig/kernel/stop_machine.c
+++ linux-2.6-for-ingo/kernel/stop_machine.c
@@ -69,10 +69,10 @@ static void stop_cpu(struct work_struct
int err;
if (!active_cpus) {
- if (cpu == first_cpu(cpu_online_map))
+ if (cpu == cpumask_first(cpu_online_mask))
smdata = &active;
} else {
- if (cpu_isset(cpu, *active_cpus))
+ if (cpumask_test_cpu(cpu, active_cpus))
smdata = &active;
}
/* Simple state machine */
@@ -109,7 +109,7 @@ static int chill(void *unused)
return 0;
}
-int __stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus)
+int __stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus)
{
struct work_struct *sm_work;
int i, ret;
@@ -142,7 +142,7 @@ int __stop_machine(int (*fn)(void *), vo
return ret;
}
-int stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus)
+int stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus)
{
int ret;
--- linux-2.6-for-ingo.orig/kernel/taskstats.c
+++ linux-2.6-for-ingo/kernel/taskstats.c
@@ -290,18 +290,17 @@ ret:
return;
}
-static int add_del_listener(pid_t pid, cpumask_t *maskp, int isadd)
+static int add_del_listener(pid_t pid, const struct cpumask *mask, int isadd)
{
struct listener_list *listeners;
struct listener *s, *tmp;
unsigned int cpu;
- cpumask_t mask = *maskp;
- if (!cpus_subset(mask, cpu_possible_map))
+ if (!cpumask_subset(mask, cpu_possible_mask))
return -EINVAL;
if (isadd == REGISTER) {
- for_each_cpu_mask_nr(cpu, mask) {
+ for_each_cpu(cpu, mask) {
s = kmalloc_node(sizeof(struct listener), GFP_KERNEL,
cpu_to_node(cpu));
if (!s)
@@ -320,7 +319,7 @@ static int add_del_listener(pid_t pid, c
/* Deregister or cleanup */
cleanup:
- for_each_cpu_mask_nr(cpu, mask) {
+ for_each_cpu(cpu, mask) {
listeners = &per_cpu(listener_array, cpu);
down_write(&listeners->sem);
list_for_each_entry_safe(s, tmp, &listeners->list, list) {
@@ -335,7 +334,7 @@ cleanup:
return 0;
}
-static int parse(struct nlattr *na, cpumask_t *mask)
+static int parse(struct nlattr *na, struct cpumask *mask)
{
char *data;
int len;
@@ -428,23 +427,33 @@ err:
static int taskstats_user_cmd(struct sk_buff *skb, struct genl_info *info)
{
- int rc = 0;
+ int rc;
struct sk_buff *rep_skb;
struct taskstats *stats;
size_t size;
- cpumask_t mask;
+ cpumask_var_t mask;
+
+ if (!alloc_cpumask_var(&mask, GFP_KERNEL))
+ return -ENOMEM;
- rc = parse(info->attrs[TASKSTATS_CMD_ATTR_REGISTER_CPUMASK], &mask);
+ rc = parse(info->attrs[TASKSTATS_CMD_ATTR_REGISTER_CPUMASK], mask);
if (rc < 0)
- return rc;
- if (rc == 0)
- return add_del_listener(info->snd_pid, &mask, REGISTER);
+ goto free_return_rc;
+ if (rc == 0) {
+ rc = add_del_listener(info->snd_pid, mask, REGISTER);
+ goto free_return_rc;
+ }
- rc = parse(info->attrs[TASKSTATS_CMD_ATTR_DEREGISTER_CPUMASK], &mask);
+ rc = parse(info->attrs[TASKSTATS_CMD_ATTR_DEREGISTER_CPUMASK], mask);
if (rc < 0)
+ goto free_return_rc;
+ if (rc == 0) {
+ rc = add_del_listener(info->snd_pid, mask, DEREGISTER);
+free_return_rc:
+ free_cpumask_var(mask);
return rc;
- if (rc == 0)
- return add_del_listener(info->snd_pid, &mask, DEREGISTER);
+ }
+ free_cpumask_var(mask);
/*
* Size includes space for nested attributes
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 6/8] cpumask: convert kernel mm functions
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
` (4 preceding siblings ...)
2008-12-19 16:01 ` [PATCH 5/8] cpumask: convert rest of files in kernel/ Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
2008-12-19 18:08 ` Christoph Lameter
2008-12-19 16:01 ` [PATCH 7/8] cpumask: convert misc driver functions Mike Travis
2008-12-19 16:01 ` [PATCH 8/8] cpumask: convert other misc kernel functions Mike Travis
7 siblings, 1 reply; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar
Cc: Rusty Russell, linux-kernel, Christoph Lameter, Pekka Enberg,
Matt Mackall
[-- Attachment #1: cpumask:convert-mm.patch --]
[-- Type: text/plain, Size: 5341 bytes --]
Impact: Reduce stack usage, use new cpumask API.
Convert kernel mm functions to use struct cpumask.
We skip include/linux/percpu.h and mm/allocpercpu.c, which are in flux.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Cc: Christoph Lameter <cl@linux-foundation.org>
Cc: Pekka Enberg <penberg@cs.helsinki.fi>
Cc: Matt Mackall <mpm@selenic.com>
---
mm/pdflush.c | 16 +++++++++++++---
mm/slab.c | 2 +-
mm/slub.c | 20 +++++++++++---------
mm/vmscan.c | 2 +-
mm/vmstat.c | 4 ++--
5 files changed, 28 insertions(+), 16 deletions(-)
diff --git a/mm/pdflush.c b/mm/pdflush.c
--- a/mm/pdflush.c
+++ b/mm/pdflush.c
@@ -172,7 +172,16 @@ static int pdflush(void *dummy)
static int pdflush(void *dummy)
{
struct pdflush_work my_work;
- cpumask_t cpus_allowed;
+ cpumask_var_t cpus_allowed;
+
+ /*
+ * Since the caller doesn't even check kthread_run() worked, let's not
+ * freak out too much if this fails.
+ */
+ if (!alloc_cpumask_var(&cpus_allowed, GFP_KERNEL)) {
+ printk(KERN_WARNING "pdflush failed to allocate cpumask\n");
+ return 0;
+ }
/*
* pdflush can spend a lot of time doing encryption via dm-crypt. We
@@ -187,8 +196,9 @@ static int pdflush(void *dummy)
* This is needed as pdflush's are dynamically created and destroyed.
* The boottime pdflush's are easily placed w/o these 2 lines.
*/
- cpuset_cpus_allowed(current, &cpus_allowed);
- set_cpus_allowed_ptr(current, &cpus_allowed);
+ cpuset_cpus_allowed(current, cpus_allowed);
+ set_cpus_allowed_ptr(current, cpus_allowed);
+ free_cpumask_var(cpus_allowed);
return __pdflush(&my_work);
}
diff --git a/mm/slab.c b/mm/slab.c
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -2155,7 +2155,7 @@ kmem_cache_create (const char *name, siz
/*
* We use cache_chain_mutex to ensure a consistent view of
- * cpu_online_map as well. Please see cpuup_callback
+ * cpu_online_mask as well. Please see cpuup_callback
*/
get_online_cpus();
mutex_lock(&cache_chain_mutex);
diff --git a/mm/slub.c b/mm/slub.c
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1960,7 +1960,7 @@ static DEFINE_PER_CPU(struct kmem_cache_
kmem_cache_cpu)[NR_KMEM_CACHE_CPU];
static DEFINE_PER_CPU(struct kmem_cache_cpu *, kmem_cache_cpu_free);
-static cpumask_t kmem_cach_cpu_free_init_once = CPU_MASK_NONE;
+static DECLARE_BITMAP(kmem_cach_cpu_free_init_once, CONFIG_NR_CPUS);
static struct kmem_cache_cpu *alloc_kmem_cache_cpu(struct kmem_cache *s,
int cpu, gfp_t flags)
@@ -2035,13 +2035,13 @@ static void init_alloc_cpu_cpu(int cpu)
{
int i;
- if (cpu_isset(cpu, kmem_cach_cpu_free_init_once))
+ if (cpumask_test_cpu(cpu, to_cpumask(kmem_cach_cpu_free_init_once)))
return;
for (i = NR_KMEM_CACHE_CPU - 1; i >= 0; i--)
free_kmem_cache_cpu(&per_cpu(kmem_cache_cpu, cpu)[i], cpu);
- cpu_set(cpu, kmem_cach_cpu_free_init_once);
+ cpumask_set_cpu(cpu, to_cpumask(kmem_cach_cpu_free_init_once));
}
static void __init init_alloc_cpu(void)
@@ -3433,7 +3433,7 @@ struct location {
long max_time;
long min_pid;
long max_pid;
- cpumask_t cpus;
+ DECLARE_BITMAP(cpus, NR_CPUS);
nodemask_t nodes;
};
@@ -3508,7 +3508,8 @@ static int add_location(struct loc_track
if (track->pid > l->max_pid)
l->max_pid = track->pid;
- cpu_set(track->cpu, l->cpus);
+ cpumask_set_cpu(track->cpu,
+ to_cpumask(l->cpus));
}
node_set(page_to_nid(virt_to_page(track)), l->nodes);
return 1;
@@ -3538,8 +3539,8 @@ static int add_location(struct loc_track
l->max_time = age;
l->min_pid = track->pid;
l->max_pid = track->pid;
- cpus_clear(l->cpus);
- cpu_set(track->cpu, l->cpus);
+ cpumask_clear(to_cpumask(l->cpus));
+ cpumask_set_cpu(track->cpu, to_cpumask(l->cpus));
nodes_clear(l->nodes);
node_set(page_to_nid(virt_to_page(track)), l->nodes);
return 1;
@@ -3620,11 +3621,12 @@ static int list_locations(struct kmem_ca
len += sprintf(buf + len, " pid=%ld",
l->min_pid);
- if (num_online_cpus() > 1 && !cpus_empty(l->cpus) &&
+ if (num_online_cpus() > 1 &&
+ !cpumask_empty(to_cpumask(l->cpus)) &&
len < PAGE_SIZE - 60) {
len += sprintf(buf + len, " cpus=");
len += cpulist_scnprintf(buf + len, PAGE_SIZE - len - 50,
- &l->cpus);
+ to_cpumask(l->cpus));
}
if (num_online_nodes() > 1 && !nodes_empty(l->nodes) &&
diff --git a/mm/vmscan.c b/mm/vmscan.c
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1902,7 +1902,7 @@ static int kswapd(void *p)
};
node_to_cpumask_ptr(cpumask, pgdat->node_id);
- if (!cpus_empty(*cpumask))
+ if (!cpumask_empty(cpumask))
set_cpus_allowed_ptr(tsk, cpumask);
current->reclaim_state = &reclaim_state;
diff --git a/mm/vmstat.c b/mm/vmstat.c
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -20,7 +20,7 @@ DEFINE_PER_CPU(struct vm_event_state, vm
DEFINE_PER_CPU(struct vm_event_state, vm_event_states) = {{0}};
EXPORT_PER_CPU_SYMBOL(vm_event_states);
-static void sum_vm_events(unsigned long *ret, cpumask_t *cpumask)
+static void sum_vm_events(unsigned long *ret, const struct cpumask *cpumask)
{
int cpu;
int i;
@@ -43,7 +43,7 @@ void all_vm_events(unsigned long *ret)
void all_vm_events(unsigned long *ret)
{
get_online_cpus();
- sum_vm_events(ret, &cpu_online_map);
+ sum_vm_events(ret, cpu_online_mask);
put_online_cpus();
}
EXPORT_SYMBOL_GPL(all_vm_events);
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 7/8] cpumask: convert misc driver functions
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
` (5 preceding siblings ...)
2008-12-19 16:01 ` [PATCH 6/8] cpumask: convert kernel mm functions Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
2008-12-20 23:22 ` Ben Hutchings
2008-12-29 15:37 ` Dean Nelson
2008-12-19 16:01 ` [PATCH 8/8] cpumask: convert other misc kernel functions Mike Travis
7 siblings, 2 replies; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar
Cc: Rusty Russell, linux-kernel, Ben Hutchings, Dean Nelson,
Jeremy Fitzhardinge, Robert Richter
[-- Attachment #1: cpumask:convert-drivers.patch --]
[-- Type: text/plain, Size: 9473 bytes --]
Impact: Reduce stack usage, use new cpumask API.
Convert misc driver functions to use struct cpumask.
To Do:
- Convert iucv_buffer_cpumask to cpumask_var_t.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Cc: Ben Hutchings <bhutchings@solarflare.com>
Cc: Dean Nelson <dcn@sgi.com>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
Cc: Robert Richter <robert.richter@amd.com>
---
drivers/base/cpu.c | 2 +-
drivers/misc/sgi-xp/xpc_main.c | 2 +-
drivers/net/sfc/efx.c | 29 +++++++++++++++++------------
drivers/oprofile/buffer_sync.c | 22 ++++++++++++++++++----
drivers/oprofile/buffer_sync.h | 4 ++++
drivers/oprofile/oprof.c | 9 ++++++++-
drivers/xen/manage.c | 2 +-
net/iucv/iucv.c | 28 ++++++++++++++++++----------
8 files changed, 68 insertions(+), 30 deletions(-)
--- linux-2.6-for-ingo.orig/drivers/base/cpu.c
+++ linux-2.6-for-ingo/drivers/base/cpu.c
@@ -107,7 +107,7 @@ static SYSDEV_ATTR(crash_notes, 0400, sh
/*
* Print cpu online, possible, present, and system maps
*/
-static ssize_t print_cpus_map(char *buf, cpumask_t *map)
+static ssize_t print_cpus_map(char *buf, const struct cpumask *map)
{
int n = cpulist_scnprintf(buf, PAGE_SIZE-2, map);
--- linux-2.6-for-ingo.orig/drivers/misc/sgi-xp/xpc_main.c
+++ linux-2.6-for-ingo/drivers/misc/sgi-xp/xpc_main.c
@@ -318,7 +318,7 @@ xpc_hb_checker(void *ignore)
/* this thread was marked active by xpc_hb_init() */
- set_cpus_allowed_ptr(current, &cpumask_of_cpu(XPC_HB_CHECK_CPU));
+ set_cpus_allowed_ptr(current, cpumask_of(XPC_HB_CHECK_CPU));
/* set our heartbeating to other partitions into motion */
xpc_hb_check_timeout = jiffies + (xpc_hb_check_interval * HZ);
--- linux-2.6-for-ingo.orig/drivers/net/sfc/efx.c
+++ linux-2.6-for-ingo/drivers/net/sfc/efx.c
@@ -809,19 +809,18 @@ static void efx_fini_io(struct efx_nic *
/* Get number of RX queues wanted. Return number of online CPU
* packages in the expectation that an IRQ balancer will spread
* interrupts across them. */
-static int efx_wanted_rx_queues(void)
+static int efx_wanted_rx_queues(struct cpumask *scratch)
{
- cpumask_t core_mask;
int count;
int cpu;
- cpus_clear(core_mask);
+ cpumask_clear(scratch);
count = 0;
for_each_online_cpu(cpu) {
- if (!cpu_isset(cpu, core_mask)) {
+ if (!cpumask_test_cpu(cpu, scratch)) {
++count;
- cpus_or(core_mask, core_mask,
- topology_core_siblings(cpu));
+ cpumask_or(scratch, scratch,
+ topology_core_cpumask(cpu));
}
}
@@ -831,7 +830,7 @@ static int efx_wanted_rx_queues(void)
/* Probe the number and type of interrupts we are able to obtain, and
* the resulting numbers of channels and RX queues.
*/
-static void efx_probe_interrupts(struct efx_nic *efx)
+static void efx_probe_interrupts(struct efx_nic *efx, struct cpumask *scratch)
{
int max_channels =
min_t(int, efx->type->phys_addr_channels, EFX_MAX_CHANNELS);
@@ -845,7 +844,8 @@ static void efx_probe_interrupts(struct
* (or as specified by the rss_cpus module parameter).
* We will need one channel per interrupt.
*/
- wanted_ints = rss_cpus ? rss_cpus : efx_wanted_rx_queues();
+ wanted_ints = rss_cpus ? rss_cpus :
+ efx_wanted_rx_queues(scratch);
efx->n_rx_queues = min(wanted_ints, max_channels);
for (i = 0; i < efx->n_rx_queues; i++)
@@ -923,24 +923,29 @@ static void efx_set_channels(struct efx_
static int efx_probe_nic(struct efx_nic *efx)
{
int rc;
+ cpumask_var_t scratch;
EFX_LOG(efx, "creating NIC\n");
+ if (!alloc_cpumask_var(&scratch, GFP_KERNEL))
+ return -ENOMEM;
+
/* Carry out hardware-type specific initialisation */
rc = falcon_probe_nic(efx);
if (rc)
- return rc;
+ goto out;
/* Determine the number of channels and RX queues by trying to hook
* in MSI-X interrupts. */
- efx_probe_interrupts(efx);
+ efx_probe_interrupts(efx, scratch);
efx_set_channels(efx);
/* Initialise the interrupt moderation settings */
efx_init_irq_moderation(efx, tx_irq_mod_usec, rx_irq_mod_usec);
-
- return 0;
+out:
+ free_cpumask_var(scratch);
+ return rc;
}
static void efx_remove_nic(struct efx_nic *efx)
--- linux-2.6-for-ingo.orig/drivers/oprofile/buffer_sync.c
+++ linux-2.6-for-ingo/drivers/oprofile/buffer_sync.c
@@ -37,7 +37,7 @@
static LIST_HEAD(dying_tasks);
static LIST_HEAD(dead_tasks);
-static cpumask_t marked_cpus = CPU_MASK_NONE;
+static cpumask_var_t marked_cpus;
static DEFINE_SPINLOCK(task_mortuary);
static void process_task_mortuary(void);
@@ -524,10 +524,10 @@ static void mark_done(int cpu)
{
int i;
- cpu_set(cpu, marked_cpus);
+ cpumask_set_cpu(cpu, marked_cpus);
for_each_online_cpu(i) {
- if (!cpu_isset(i, marked_cpus))
+ if (!cpumask_test_cpu(i, marked_cpus))
return;
}
@@ -536,7 +536,7 @@ static void mark_done(int cpu)
*/
process_task_mortuary();
- cpus_clear(marked_cpus);
+ cpumask_clear(marked_cpus);
}
@@ -632,6 +632,20 @@ void sync_buffer(int cpu)
mutex_unlock(&buffer_mutex);
}
+int __init buffer_sync_init(void)
+{
+ if (!alloc_cpumask_var(&marked_cpus, GFP_KERNEL))
+ return -ENOMEM;
+
+ cpumask_copy(marked_cpus, cpu_none_mask);
+ return 0;
+}
+
+void __exit buffer_sync_cleanup(void)
+{
+ free_cpumask_var(marked_cpus);
+}
+
/* The function can be used to add a buffer worth of data directly to
* the kernel buffer. The buffer is assumed to be a circular buffer.
* Take the entries from index start and end at index end, wrapping
--- linux-2.6-for-ingo.orig/drivers/oprofile/buffer_sync.h
+++ linux-2.6-for-ingo/drivers/oprofile/buffer_sync.h
@@ -19,4 +19,8 @@ void sync_stop(void);
/* sync the given CPU's buffer */
void sync_buffer(int cpu);
+/* initialize/destroy the buffer system. */
+int buffer_sync_init(void);
+void buffer_sync_cleanup(void);
+
#endif /* OPROFILE_BUFFER_SYNC_H */
--- linux-2.6-for-ingo.orig/drivers/oprofile/oprof.c
+++ linux-2.6-for-ingo/drivers/oprofile/oprof.c
@@ -183,6 +183,10 @@ static int __init oprofile_init(void)
{
int err;
+ err = buffer_sync_init();
+ if (err)
+ return err;
+
err = oprofile_arch_init(&oprofile_ops);
if (err < 0 || timer) {
@@ -191,8 +195,10 @@ static int __init oprofile_init(void)
}
err = oprofilefs_register();
- if (err)
+ if (err) {
oprofile_arch_exit();
+ buffer_sync_cleanup();
+ }
return err;
}
@@ -202,6 +208,7 @@ static void __exit oprofile_exit(void)
{
oprofilefs_unregister();
oprofile_arch_exit();
+ buffer_sync_cleanup();
}
--- linux-2.6-for-ingo.orig/drivers/xen/manage.c
+++ linux-2.6-for-ingo/drivers/xen/manage.c
@@ -100,7 +100,7 @@ static void do_suspend(void)
/* XXX use normal device tree? */
xenbus_suspend();
- err = stop_machine(xen_suspend, &cancelled, &cpumask_of_cpu(0));
+ err = stop_machine(xen_suspend, &cancelled, cpumask_of(0));
if (err) {
printk(KERN_ERR "failed to start xen_suspend: %d\n", err);
goto out;
--- linux-2.6-for-ingo.orig/net/iucv/iucv.c
+++ linux-2.6-for-ingo/net/iucv/iucv.c
@@ -489,15 +489,14 @@ static void iucv_setmask_mp(void)
*
* Allow iucv interrupts on a single cpu.
*/
-static void iucv_setmask_up(void)
+static void iucv_setmask_up(struct cpumask *cpumask)
{
- cpumask_t cpumask;
int cpu;
/* Disable all cpu but the first in cpu_irq_cpumask. */
- cpumask = iucv_irq_cpumask;
- cpu_clear(first_cpu(iucv_irq_cpumask), cpumask);
- for_each_cpu_mask_nr(cpu, cpumask)
+ cpumask_copy(cpumask, iucv_irq_cpumask);
+ cpumask_clear_cpu(cpumask_first(iucv_irq_cpumask), cpumask);
+ for_each_cpu(cpu, cpumask)
smp_call_function_single(cpu, iucv_block_cpu, NULL, 1);
}
@@ -555,7 +554,7 @@ static void iucv_disable(void)
static int __cpuinit iucv_cpu_notify(struct notifier_block *self,
unsigned long action, void *hcpu)
{
- cpumask_t cpumask;
+ cpumask_var_t cpumask;
long cpu = (long) hcpu;
switch (action) {
@@ -590,15 +589,20 @@ static int __cpuinit iucv_cpu_notify(str
break;
case CPU_DOWN_PREPARE:
case CPU_DOWN_PREPARE_FROZEN:
- cpumask = iucv_buffer_cpumask;
- cpu_clear(cpu, cpumask);
- if (cpus_empty(cpumask))
+ if (!alloc_cpumask_var(&cpumask, GFP_KERNEL))
+ return NOTIFY_BAD;
+ cpumask_copy(cpumask, &iucv_buffer_cpumask);
+ cpumask_clear_cpu(cpu, cpumask);
+ if (cpumask_empty(cpumask)) {
/* Can't offline last IUCV enabled cpu. */
+ free_cpumask_var(cpumask);
return NOTIFY_BAD;
+ }
smp_call_function_single(cpu, iucv_retrieve_cpu, NULL, 1);
if (cpus_empty(iucv_irq_cpumask))
smp_call_function_single(first_cpu(iucv_buffer_cpumask),
iucv_allow_cpu, NULL, 1);
+ free_cpumask_var(cpumask);
break;
}
return NOTIFY_OK;
@@ -683,9 +687,12 @@ static void iucv_cleanup_queue(void)
int iucv_register(struct iucv_handler *handler, int smp)
{
int rc;
+ cpumask_var_t scratch;
if (!iucv_available)
return -ENOSYS;
+ if (!alloc_cpumask_var(&scratch, GFP_KERNEL))
+ return -ENOMEM;
mutex_lock(&iucv_register_mutex);
if (!smp)
iucv_nonsmp_handler++;
@@ -694,7 +701,7 @@ int iucv_register(struct iucv_handler *h
if (rc)
goto out_mutex;
} else if (!smp && iucv_nonsmp_handler == 1)
- iucv_setmask_up();
+ iucv_setmask_up(scratch);
INIT_LIST_HEAD(&handler->paths);
spin_lock_bh(&iucv_table_lock);
@@ -703,6 +710,7 @@ int iucv_register(struct iucv_handler *h
rc = 0;
out_mutex:
mutex_unlock(&iucv_register_mutex);
+ free_cpumask_var(scratch);
return rc;
}
EXPORT_SYMBOL(iucv_register);
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 8/8] cpumask: convert other misc kernel functions
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
` (6 preceding siblings ...)
2008-12-19 16:01 ` [PATCH 7/8] cpumask: convert misc driver functions Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
7 siblings, 0 replies; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Rusty Russell, linux-kernel, Steven Rostedt
[-- Attachment #1: cpumask:convert-misc.patch --]
[-- Type: text/plain, Size: 717 bytes --]
Impact: Use new cpumask API.
Convert other misc kernel functions to use struct cpumask.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Cc: Steven Rostedt <srostedt@redhat.com>
---
lib/smp_processor_id.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- linux-2.6-for-ingo.orig/lib/smp_processor_id.c
+++ linux-2.6-for-ingo/lib/smp_processor_id.c
@@ -22,7 +22,7 @@ notrace unsigned int debug_smp_processor
* Kernel threads bound to a single CPU can safely use
* smp_processor_id():
*/
- if (cpus_equal(current->cpus_allowed, cpumask_of_cpu(this_cpu)))
+ if (cpumask_equal(¤t->cpus_allowed, cpumask_of(this_cpu)))
goto out;
/*
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 4/8] cpumask: convert kernel trace functions
2008-12-19 16:01 ` [PATCH 4/8] cpumask: convert kernel trace functions Mike Travis
@ 2008-12-19 16:16 ` Steven Rostedt
2008-12-19 16:54 ` Mike Travis
0 siblings, 1 reply; 21+ messages in thread
From: Steven Rostedt @ 2008-12-19 16:16 UTC (permalink / raw)
To: Mike Travis; +Cc: Ingo Molnar, Rusty Russell, linux-kernel
On Fri, 19 Dec 2008, Mike Travis wrote:
> Impact: Reduce memory usage, use new cpumask API.
>
> Convert kernel trace functions to use struct cpumask.
>
[..]
> #define for_each_buffer_cpu(buffer, cpu) \
> - for_each_cpu_mask(cpu, buffer->cpumask)
> + for_each_cpu(cpu, buffer->cpumask)
>
> #define TS_SHIFT 27
> #define TS_MASK ((1ULL << TS_SHIFT) - 1)
> @@ -262,7 +262,7 @@ struct ring_buffer {
> unsigned pages;
> unsigned flags;
> int cpus;
> - cpumask_t cpumask;
> + cpumask_var_t cpumask;
> atomic_t record_disabled;
>
> struct mutex mutex;
> @@ -453,6 +453,9 @@ struct ring_buffer *ring_buffer_alloc(un
> if (!buffer)
> return NULL;
>
> + if (!alloc_cpumask_var(&buffer->cpumask, GFP_KERNEL))
> + goto fail_free_buffer;
> +
How does this save memory if we just allocate a cpumask var everytime
we allocate a ring buffer? Is cpumask_var_t a mask of possible CPUS and
not NR_CPUS?
Otherwise, I see this as just adding one extra pointer.
-- Steve
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 4/8] cpumask: convert kernel trace functions
2008-12-19 16:16 ` Steven Rostedt
@ 2008-12-19 16:54 ` Mike Travis
2008-12-19 17:01 ` Steven Rostedt
2008-12-20 1:41 ` Steven Rostedt
0 siblings, 2 replies; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:54 UTC (permalink / raw)
To: Steven Rostedt; +Cc: Ingo Molnar, Rusty Russell, linux-kernel
Steven Rostedt wrote:
> On Fri, 19 Dec 2008, Mike Travis wrote:
>
>> Impact: Reduce memory usage, use new cpumask API.
>>
>> Convert kernel trace functions to use struct cpumask.
>>
>
> [..]
>
>> #define for_each_buffer_cpu(buffer, cpu) \
>> - for_each_cpu_mask(cpu, buffer->cpumask)
>> + for_each_cpu(cpu, buffer->cpumask)
>>
>> #define TS_SHIFT 27
>> #define TS_MASK ((1ULL << TS_SHIFT) - 1)
>> @@ -262,7 +262,7 @@ struct ring_buffer {
>> unsigned pages;
>> unsigned flags;
>> int cpus;
>> - cpumask_t cpumask;
>> + cpumask_var_t cpumask;
>> atomic_t record_disabled;
>>
>> struct mutex mutex;
>> @@ -453,6 +453,9 @@ struct ring_buffer *ring_buffer_alloc(un
>> if (!buffer)
>> return NULL;
>>
>> + if (!alloc_cpumask_var(&buffer->cpumask, GFP_KERNEL))
>> + goto fail_free_buffer;
>> +
>
>
> How does this save memory if we just allocate a cpumask var everytime
> we allocate a ring buffer? Is cpumask_var_t a mask of possible CPUS and
> not NR_CPUS?
>
> Otherwise, I see this as just adding one extra pointer.
>
> -- Steve
Hi Steve,
Yes, eventually, the cpumask_var_t will be allocated based on cpumask_size()
which will become BITS_TO_LONG(nr_cpu_ids) instead of BITS_TO_LONGS(NR_CPUS)
as soon as the kernel becomes "cpumask" clean. (clean being that it ignores
all bits >= nr_cpu_ids.)
Note that on small NR_CPUS count systems, cpumask_var_t is a static array
and no allocation happens, so it only kicks in when CONFIG_CPUMASK_OFFSTACK=y.
Thanks,
Mike
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 4/8] cpumask: convert kernel trace functions
2008-12-19 16:54 ` Mike Travis
@ 2008-12-19 17:01 ` Steven Rostedt
2008-12-20 1:41 ` Steven Rostedt
1 sibling, 0 replies; 21+ messages in thread
From: Steven Rostedt @ 2008-12-19 17:01 UTC (permalink / raw)
To: Mike Travis; +Cc: Ingo Molnar, Rusty Russell, linux-kernel
On Fri, 19 Dec 2008, Mike Travis wrote:
> >
> >
> > How does this save memory if we just allocate a cpumask var everytime
> > we allocate a ring buffer? Is cpumask_var_t a mask of possible CPUS and
> > not NR_CPUS?
> >
> > Otherwise, I see this as just adding one extra pointer.
> >
> > -- Steve
>
> Hi Steve,
>
> Yes, eventually, the cpumask_var_t will be allocated based on cpumask_size()
> which will become BITS_TO_LONG(nr_cpu_ids) instead of BITS_TO_LONGS(NR_CPUS)
> as soon as the kernel becomes "cpumask" clean. (clean being that it ignores
> all bits >= nr_cpu_ids.)
>
> Note that on small NR_CPUS count systems, cpumask_var_t is a static array
> and no allocation happens, so it only kicks in when CONFIG_CPUMASK_OFFSTACK=y.
OK, that explains a lot. Thanks for the explanation.
-- Steve
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 6/8] cpumask: convert kernel mm functions
2008-12-19 16:01 ` [PATCH 6/8] cpumask: convert kernel mm functions Mike Travis
@ 2008-12-19 18:08 ` Christoph Lameter
0 siblings, 0 replies; 21+ messages in thread
From: Christoph Lameter @ 2008-12-19 18:08 UTC (permalink / raw)
To: Mike Travis
Cc: Ingo Molnar, Rusty Russell, linux-kernel, Pekka Enberg, Matt Mackall
Reviewed-by: Christoph Lameter <cl@linux-foundation.org>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 5/8] cpumask: convert rest of files in kernel/
2008-12-19 16:01 ` [PATCH 5/8] cpumask: convert rest of files in kernel/ Mike Travis
@ 2008-12-20 1:33 ` Lai Jiangshan
2008-12-20 13:02 ` Rusty Russell
1 sibling, 0 replies; 21+ messages in thread
From: Lai Jiangshan @ 2008-12-20 1:33 UTC (permalink / raw)
To: Mike Travis
Cc: Ingo Molnar, Rusty Russell, linux-kernel, Dipankar Sarma,
H. Peter Anvin, Max Krasnyansky, Richard Henderson,
Thomas Gleixner, Yinghai Lu, Paul E. McKenney
CC: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Mike Travis wrote:
> Impact: Reduce stack usage, use new cpumask API. ALPHA mod!
>
> A couple of uses of cpumask_any_but() here to avoid temporary cpumasks
> and carry the "const" modifier through the seq_cpumask() function all the
> way to bitmap_scnprintf().
>
> Note that prof_cpu_mask will be difficult to convert to a cpumask_var_t
> since it needs to be ready from the first scheduler tick.
>
> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
> Signed-off-by: Mike Travis <travis@sgi.com>
> Cc: Dipankar Sarma <dipankar@in.ibm.com>
> Cc: H. Peter Anvin <hpa@zytor.com>
> Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
> Cc: Max Krasnyansky <maxk@qualcomm.com>
> Cc: Richard Henderson <rth@twiddle.net>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Yinghai Lu <yinghai@kernel.org>
> ---
> arch/alpha/kernel/irq.c | 3 ++-
> fs/seq_file.c | 3 ++-
> include/linux/interrupt.h | 2 +-
> include/linux/rcuclassic.h | 4 ++--
> include/linux/seq_file.h | 5 +++--
> include/linux/stop_machine.h | 6 +++---
> kernel/cpu.c | 40 +++++++++++++++++++++-------------------
> kernel/irq/manage.c | 11 +++++++++--
> kernel/irq/proc.c | 32 +++++++++++++++++++++-----------
> kernel/power/poweroff.c | 2 +-
> kernel/profile.c | 31 ++++++++++++++++++-------------
> kernel/rcuclassic.c | 32 +++++++++++++++++---------------
> kernel/rcupreempt.c | 19 ++++++++++---------
> kernel/rcutorture.c | 27 +++++++++++++++------------
> kernel/softlockup.c | 6 ++----
> kernel/stop_machine.c | 8 ++++----
> kernel/taskstats.c | 39 ++++++++++++++++++++++++---------------
> 17 files changed, 155 insertions(+), 115 deletions(-)
>
> --- linux-2.6-for-ingo.orig/arch/alpha/kernel/irq.c
> +++ linux-2.6-for-ingo/arch/alpha/kernel/irq.c
> @@ -50,7 +50,8 @@ int irq_select_affinity(unsigned int irq
> if (!irq_desc[irq].chip->set_affinity || irq_user_affinity[irq])
> return 1;
>
> - while (!cpu_possible(cpu) || !cpu_isset(cpu, irq_default_affinity))
> + while (!cpu_possible(cpu) ||
> + !cpumask_test_cpu(cpu, irq_default_affinity))
> cpu = (cpu < (NR_CPUS-1) ? cpu + 1 : 0);
> last_cpu = cpu;
>
> --- linux-2.6-for-ingo.orig/fs/seq_file.c
> +++ linux-2.6-for-ingo/fs/seq_file.c
> @@ -462,7 +462,8 @@ int seq_dentry(struct seq_file *m, struc
> return -1;
> }
>
> -int seq_bitmap(struct seq_file *m, unsigned long *bits, unsigned int nr_bits)
> +int seq_bitmap(struct seq_file *m, const unsigned long *bits,
> + unsigned int nr_bits)
> {
> if (m->count < m->size) {
> int len = bitmap_scnprintf(m->buf + m->count,
> --- linux-2.6-for-ingo.orig/include/linux/interrupt.h
> +++ linux-2.6-for-ingo/include/linux/interrupt.h
> @@ -109,7 +109,7 @@ extern void enable_irq(unsigned int irq)
>
> #if defined(CONFIG_SMP) && defined(CONFIG_GENERIC_HARDIRQS)
>
> -extern cpumask_t irq_default_affinity;
> +extern cpumask_var_t irq_default_affinity;
>
> extern int irq_set_affinity(unsigned int irq, const struct cpumask *cpumask);
> extern int irq_can_set_affinity(unsigned int irq);
> --- linux-2.6-for-ingo.orig/include/linux/rcuclassic.h
> +++ linux-2.6-for-ingo/include/linux/rcuclassic.h
> @@ -59,8 +59,8 @@ struct rcu_ctrlblk {
> int signaled;
>
> spinlock_t lock ____cacheline_internodealigned_in_smp;
> - cpumask_t cpumask; /* CPUs that need to switch in order */
> - /* for current batch to proceed. */
> + DECLARE_BITMAP(cpumask, NR_CPUS); /* CPUs that need to switch for */
> + /* current batch to proceed. */
> } ____cacheline_internodealigned_in_smp;
>
> /* Is batch a before batch b ? */
> --- linux-2.6-for-ingo.orig/include/linux/seq_file.h
> +++ linux-2.6-for-ingo/include/linux/seq_file.h
> @@ -50,8 +50,9 @@ int seq_path(struct seq_file *, struct p
> int seq_dentry(struct seq_file *, struct dentry *, char *);
> int seq_path_root(struct seq_file *m, struct path *path, struct path *root,
> char *esc);
> -int seq_bitmap(struct seq_file *m, unsigned long *bits, unsigned int nr_bits);
> -static inline int seq_cpumask(struct seq_file *m, cpumask_t *mask)
> +int seq_bitmap(struct seq_file *m, const unsigned long *bits,
> + unsigned int nr_bits);
> +static inline int seq_cpumask(struct seq_file *m, const struct cpumask *mask)
> {
> return seq_bitmap(m, mask->bits, NR_CPUS);
> }
> --- linux-2.6-for-ingo.orig/include/linux/stop_machine.h
> +++ linux-2.6-for-ingo/include/linux/stop_machine.h
> @@ -23,7 +23,7 @@
> *
> * This can be thought of as a very heavy write lock, equivalent to
> * grabbing every spinlock in the kernel. */
> -int stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus);
> +int stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus);
>
> /**
> * __stop_machine: freeze the machine on all CPUs and run this function
> @@ -34,11 +34,11 @@ int stop_machine(int (*fn)(void *), void
> * Description: This is a special version of the above, which assumes cpus
> * won't come or go while it's being called. Used by hotplug cpu.
> */
> -int __stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus);
> +int __stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus);
> #else
>
> static inline int stop_machine(int (*fn)(void *), void *data,
> - const cpumask_t *cpus)
> + const struct cpumask *cpus)
> {
> int ret;
> local_irq_disable();
> --- linux-2.6-for-ingo.orig/kernel/cpu.c
> +++ linux-2.6-for-ingo/kernel/cpu.c
> @@ -218,7 +218,7 @@ static int __ref take_cpu_down(void *_pa
> static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
> {
> int err, nr_calls = 0;
> - cpumask_t old_allowed, tmp;
> + cpumask_var_t old_allowed;
> void *hcpu = (void *)(long)cpu;
> unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
> struct take_cpu_down_param tcd_param = {
> @@ -232,6 +232,9 @@ static int __ref _cpu_down(unsigned int
> if (!cpu_online(cpu))
> return -EINVAL;
>
> + if (!alloc_cpumask_var(&old_allowed, GFP_KERNEL))
> + return -ENOMEM;
> +
> cpu_hotplug_begin();
> err = __raw_notifier_call_chain(&cpu_chain, CPU_DOWN_PREPARE | mod,
> hcpu, -1, &nr_calls);
> @@ -246,13 +249,11 @@ static int __ref _cpu_down(unsigned int
> }
>
> /* Ensure that we are not runnable on dying cpu */
> - old_allowed = current->cpus_allowed;
> - cpus_setall(tmp);
> - cpu_clear(cpu, tmp);
> - set_cpus_allowed_ptr(current, &tmp);
> - tmp = cpumask_of_cpu(cpu);
> + cpumask_copy(old_allowed, ¤t->cpus_allowed);
> + set_cpus_allowed_ptr(current,
> + cpumask_of(cpumask_any_but(cpu_online_mask, cpu)));
>
> - err = __stop_machine(take_cpu_down, &tcd_param, &tmp);
> + err = __stop_machine(take_cpu_down, &tcd_param, cpumask_of(cpu));
> if (err) {
> /* CPU didn't die: tell everyone. Can't complain. */
> if (raw_notifier_call_chain(&cpu_chain, CPU_DOWN_FAILED | mod,
> @@ -278,7 +279,7 @@ static int __ref _cpu_down(unsigned int
> check_for_tasks(cpu);
>
> out_allowed:
> - set_cpus_allowed_ptr(current, &old_allowed);
> + set_cpus_allowed_ptr(current, old_allowed);
> out_release:
> cpu_hotplug_done();
> if (!err) {
> @@ -286,6 +287,7 @@ out_release:
> hcpu) == NOTIFY_BAD)
> BUG();
> }
> + free_cpumask_var(old_allowed);
> return err;
> }
>
> @@ -304,7 +306,7 @@ int __ref cpu_down(unsigned int cpu)
>
> /*
> * Make sure the all cpus did the reschedule and are not
> - * using stale version of the cpu_active_map.
> + * using stale version of the cpu_active_mask.
> * This is not strictly necessary becuase stop_machine()
> * that we run down the line already provides the required
> * synchronization. But it's really a side effect and we do not
> @@ -368,7 +370,7 @@ out_notify:
> int __cpuinit cpu_up(unsigned int cpu)
> {
> int err = 0;
> - if (!cpu_isset(cpu, cpu_possible_map)) {
> + if (!cpu_possible(cpu)) {
> printk(KERN_ERR "can't online cpu %d because it is not "
> "configured as may-hotadd at boot time\n", cpu);
> #if defined(CONFIG_IA64) || defined(CONFIG_X86_64)
> @@ -393,25 +395,25 @@ out:
> }
>
> #ifdef CONFIG_PM_SLEEP_SMP
> -static cpumask_t frozen_cpus;
> +static DECLARE_BITMAP(frozen_cpus, CONFIG_NR_CPUS);
>
> int disable_nonboot_cpus(void)
> {
> int cpu, first_cpu, error = 0;
>
> cpu_maps_update_begin();
> - first_cpu = first_cpu(cpu_online_map);
> + first_cpu = cpumask_first(cpu_online_mask);
> /* We take down all of the non-boot CPUs in one shot to avoid races
> * with the userspace trying to use the CPU hotplug at the same time
> */
> - cpus_clear(frozen_cpus);
> + cpumask_clear(to_cpumask(frozen_cpus));
> printk("Disabling non-boot CPUs ...\n");
> for_each_online_cpu(cpu) {
> if (cpu == first_cpu)
> continue;
> error = _cpu_down(cpu, 1);
> if (!error) {
> - cpu_set(cpu, frozen_cpus);
> + cpumask_set_cpu(cpu, to_cpumask(frozen_cpus));
> printk("CPU%d is down\n", cpu);
> } else {
> printk(KERN_ERR "Error taking CPU%d down: %d\n",
> @@ -437,11 +439,11 @@ void __ref enable_nonboot_cpus(void)
> /* Allow everyone to use the CPU hotplug again */
> cpu_maps_update_begin();
> cpu_hotplug_disabled = 0;
> - if (cpus_empty(frozen_cpus))
> + if (cpumask_empty(to_cpumask(frozen_cpus)))
> goto out;
>
> printk("Enabling non-boot CPUs ...\n");
> - for_each_cpu_mask_nr(cpu, frozen_cpus) {
> + for_each_cpu(cpu, to_cpumask(frozen_cpus)) {
> error = _cpu_up(cpu, 1);
> if (!error) {
> printk("CPU%d is up\n", cpu);
> @@ -449,7 +451,7 @@ void __ref enable_nonboot_cpus(void)
> }
> printk(KERN_WARNING "Error taking CPU%d up: %d\n", cpu, error);
> }
> - cpus_clear(frozen_cpus);
> + cpumask_clear(to_cpumask(frozen_cpus));
> out:
> cpu_maps_update_done();
> }
> @@ -468,7 +470,7 @@ void __cpuinit notify_cpu_starting(unsig
> unsigned long val = CPU_STARTING;
>
> #ifdef CONFIG_PM_SLEEP_SMP
> - if (cpu_isset(cpu, frozen_cpus))
> + if (cpumask_test_cpu(cpu, to_cpumask(frozen_cpus)))
> val = CPU_STARTING_FROZEN;
> #endif /* CONFIG_PM_SLEEP_SMP */
> raw_notifier_call_chain(&cpu_chain, val, (void *)(long)cpu);
> @@ -480,7 +482,7 @@ void __cpuinit notify_cpu_starting(unsig
> * cpu_bit_bitmap[] is a special, "compressed" data structure that
> * represents all NR_CPUS bits binary values of 1<<nr.
> *
> - * It is used by cpumask_of_cpu() to get a constant address to a CPU
> + * It is used by cpumask_of() to get a constant address to a CPU
> * mask value that has a single bit set only.
> */
>
> --- linux-2.6-for-ingo.orig/kernel/irq/manage.c
> +++ linux-2.6-for-ingo/kernel/irq/manage.c
> @@ -16,8 +16,15 @@
> #include "internals.h"
>
> #ifdef CONFIG_SMP
> +cpumask_var_t irq_default_affinity;
>
> -cpumask_t irq_default_affinity = CPU_MASK_ALL;
> +static int init_irq_default_affinity(void)
> +{
> + alloc_cpumask_var(&irq_default_affinity, GFP_KERNEL);
> + cpumask_setall(irq_default_affinity);
> + return 0;
> +}
> +core_initcall(init_irq_default_affinity);
>
> /**
> * synchronize_irq - wait for pending IRQ handlers (on other CPUs)
> @@ -127,7 +134,7 @@ int do_irq_select_affinity(unsigned int
> desc->status &= ~IRQ_AFFINITY_SET;
> }
>
> - cpumask_and(&desc->affinity, cpu_online_mask, &irq_default_affinity);
> + cpumask_and(&desc->affinity, cpu_online_mask, irq_default_affinity);
> set_affinity:
> desc->chip->set_affinity(irq, &desc->affinity);
>
> --- linux-2.6-for-ingo.orig/kernel/irq/proc.c
> +++ linux-2.6-for-ingo/kernel/irq/proc.c
> @@ -20,7 +20,7 @@ static struct proc_dir_entry *root_irq_d
> static int irq_affinity_proc_show(struct seq_file *m, void *v)
> {
> struct irq_desc *desc = irq_to_desc((long)m->private);
> - cpumask_t *mask = &desc->affinity;
> + const struct cpumask *mask = &desc->affinity;
>
> #ifdef CONFIG_GENERIC_PENDING_IRQ
> if (desc->status & IRQ_MOVE_PENDING)
> @@ -93,7 +93,7 @@ static const struct file_operations irq_
>
> static int default_affinity_show(struct seq_file *m, void *v)
> {
> - seq_cpumask(m, &irq_default_affinity);
> + seq_cpumask(m, irq_default_affinity);
> seq_putc(m, '\n');
> return 0;
> }
> @@ -101,27 +101,37 @@ static int default_affinity_show(struct
> static ssize_t default_affinity_write(struct file *file,
> const char __user *buffer, size_t count, loff_t *ppos)
> {
> - cpumask_t new_value;
> + cpumask_var_t new_value;
> int err;
>
> - err = cpumask_parse_user(buffer, count, &new_value);
> + if (!alloc_cpumask_var(&new_value, GFP_KERNEL))
> + return -ENOMEM;
> +
> + err = cpumask_parse_user(buffer, count, new_value);
> if (err)
> - return err;
> + goto out;
>
> - if (!is_affinity_mask_valid(new_value))
> - return -EINVAL;
> + if (!is_affinity_mask_valid(new_value)) {
> + err = -EINVAL;
> + goto out;
> + }
>
> /*
> * Do not allow disabling IRQs completely - it's a too easy
> * way to make the system unusable accidentally :-) At least
> * one online CPU still has to be targeted.
> */
> - if (!cpus_intersects(new_value, cpu_online_map))
> - return -EINVAL;
> + if (!cpumask_intersects(new_value, cpu_online_mask)) {
> + err = -EINVAL;
> + goto out;
> + }
>
> - irq_default_affinity = new_value;
> + cpumask_copy(irq_default_affinity, new_value);
> + err = count;
>
> - return count;
> +out:
> + free_cpumask_var(new_value);
> + return err;
> }
>
> static int default_affinity_open(struct inode *inode, struct file *file)
> --- linux-2.6-for-ingo.orig/kernel/power/poweroff.c
> +++ linux-2.6-for-ingo/kernel/power/poweroff.c
> @@ -27,7 +27,7 @@ static DECLARE_WORK(poweroff_work, do_po
> static void handle_poweroff(int key, struct tty_struct *tty)
> {
> /* run sysrq poweroff on boot cpu */
> - schedule_work_on(first_cpu(cpu_online_map), &poweroff_work);
> + schedule_work_on(cpumask_first(cpu_online_mask), &poweroff_work);
> }
>
> static struct sysrq_key_op sysrq_poweroff_op = {
> --- linux-2.6-for-ingo.orig/kernel/profile.c
> +++ linux-2.6-for-ingo/kernel/profile.c
> @@ -45,7 +45,7 @@ static unsigned long prof_len, prof_shif
> int prof_on __read_mostly;
> EXPORT_SYMBOL_GPL(prof_on);
>
> -static cpumask_t prof_cpu_mask = CPU_MASK_ALL;
> +static DECLARE_BITMAP(prof_cpu_mask, CONFIG_NR_CPUS) = CPU_BITS_ALL;
> #ifdef CONFIG_SMP
> static DEFINE_PER_CPU(struct profile_hit *[2], cpu_profile_hits);
> static DEFINE_PER_CPU(int, cpu_profile_flip);
> @@ -386,13 +386,13 @@ out_free:
> return NOTIFY_BAD;
> case CPU_ONLINE:
> case CPU_ONLINE_FROZEN:
> - cpu_set(cpu, prof_cpu_mask);
> + cpumask_set_cpu(cpu, to_cpumask(prof_cpu_mask));
> break;
> case CPU_UP_CANCELED:
> case CPU_UP_CANCELED_FROZEN:
> case CPU_DEAD:
> case CPU_DEAD_FROZEN:
> - cpu_clear(cpu, prof_cpu_mask);
> + cpumask_clear_cpu(cpu, to_cpumask(prof_cpu_mask));
> if (per_cpu(cpu_profile_hits, cpu)[0]) {
> page = virt_to_page(per_cpu(cpu_profile_hits, cpu)[0]);
> per_cpu(cpu_profile_hits, cpu)[0] = NULL;
> @@ -430,7 +430,8 @@ void profile_tick(int type)
>
> if (type == CPU_PROFILING && timer_hook)
> timer_hook(regs);
> - if (!user_mode(regs) && cpu_isset(smp_processor_id(), prof_cpu_mask))
> + if (!user_mode(regs) &&
> + cpumask_test_cpu(smp_processor_id(), to_cpumask(prof_cpu_mask)))
> profile_hit(type, (void *)profile_pc(regs));
> }
>
> @@ -442,7 +443,7 @@ void profile_tick(int type)
> static int prof_cpu_mask_read_proc(char *page, char **start, off_t off,
> int count, int *eof, void *data)
> {
> - int len = cpumask_scnprintf(page, count, (cpumask_t *)data);
> + int len = cpumask_scnprintf(page, count, data);
> if (count - len < 2)
> return -EINVAL;
> len += sprintf(page + len, "\n");
> @@ -452,16 +453,20 @@ static int prof_cpu_mask_read_proc(char
> static int prof_cpu_mask_write_proc(struct file *file,
> const char __user *buffer, unsigned long count, void *data)
> {
> - cpumask_t *mask = (cpumask_t *)data;
> + struct cpumask *mask = data;
> unsigned long full_count = count, err;
> - cpumask_t new_value;
> + cpumask_var_t new_value;
>
> - err = cpumask_parse_user(buffer, count, &new_value);
> - if (err)
> - return err;
> + if (!alloc_cpumask_var(&new_value, GFP_KERNEL))
> + return -ENOMEM;
>
> - *mask = new_value;
> - return full_count;
> + err = cpumask_parse_user(buffer, count, new_value);
> + if (!err) {
> + cpumask_copy(mask, new_value);
> + err = full_count;
> + }
> + free_cpumask_var(new_value);
> + return err;
> }
>
> void create_prof_cpu_mask(struct proc_dir_entry *root_irq_dir)
> @@ -472,7 +477,7 @@ void create_prof_cpu_mask(struct proc_di
> entry = create_proc_entry("prof_cpu_mask", 0600, root_irq_dir);
> if (!entry)
> return;
> - entry->data = (void *)&prof_cpu_mask;
> + entry->data = prof_cpu_mask;
> entry->read_proc = prof_cpu_mask_read_proc;
> entry->write_proc = prof_cpu_mask_write_proc;
> }
> --- linux-2.6-for-ingo.orig/kernel/rcuclassic.c
> +++ linux-2.6-for-ingo/kernel/rcuclassic.c
> @@ -63,14 +63,14 @@ static struct rcu_ctrlblk rcu_ctrlblk =
> .completed = -300,
> .pending = -300,
> .lock = __SPIN_LOCK_UNLOCKED(&rcu_ctrlblk.lock),
> - .cpumask = CPU_MASK_NONE,
> + .cpumask = CPU_BITS_NONE,
> };
> static struct rcu_ctrlblk rcu_bh_ctrlblk = {
> .cur = -300,
> .completed = -300,
> .pending = -300,
> .lock = __SPIN_LOCK_UNLOCKED(&rcu_bh_ctrlblk.lock),
> - .cpumask = CPU_MASK_NONE,
> + .cpumask = CPU_BITS_NONE,
> };
>
> DEFINE_PER_CPU(struct rcu_data, rcu_data) = { 0L };
> @@ -85,7 +85,6 @@ static void force_quiescent_state(struct
> struct rcu_ctrlblk *rcp)
> {
> int cpu;
> - cpumask_t cpumask;
> unsigned long flags;
>
> set_need_resched();
> @@ -96,10 +95,10 @@ static void force_quiescent_state(struct
> * Don't send IPI to itself. With irqs disabled,
> * rdp->cpu is the current cpu.
> *
> - * cpu_online_map is updated by the _cpu_down()
> + * cpu_online_mask is updated by the _cpu_down()
> * using __stop_machine(). Since we're in irqs disabled
> * section, __stop_machine() is not exectuting, hence
> - * the cpu_online_map is stable.
> + * the cpu_online_mask is stable.
> *
> * However, a cpu might have been offlined _just_ before
> * we disabled irqs while entering here.
> @@ -107,13 +106,14 @@ static void force_quiescent_state(struct
> * notification, leading to the offlined cpu's bit
> * being set in the rcp->cpumask.
> *
> - * Hence cpumask = (rcp->cpumask & cpu_online_map) to prevent
> + * Hence cpumask = (rcp->cpumask & cpu_online_mask) to prevent
> * sending smp_reschedule() to an offlined CPU.
> */
> - cpus_and(cpumask, rcp->cpumask, cpu_online_map);
> - cpu_clear(rdp->cpu, cpumask);
> - for_each_cpu_mask_nr(cpu, cpumask)
> - smp_send_reschedule(cpu);
> + for_each_cpu_and(cpu,
> + to_cpumask(rcp->cpumask), cpu_online_mask) {
> + if (cpu != rdp->cpu)
> + smp_send_reschedule(cpu);
> + }
> }
> spin_unlock_irqrestore(&rcp->lock, flags);
> }
> @@ -193,7 +193,7 @@ static void print_other_cpu_stall(struct
>
> printk(KERN_ERR "INFO: RCU detected CPU stalls:");
> for_each_possible_cpu(cpu) {
> - if (cpu_isset(cpu, rcp->cpumask))
> + if (cpumask_test_cpu(cpu, to_cpumask(rcp->cpumask)))
> printk(" %d", cpu);
> }
> printk(" (detected by %d, t=%ld jiffies)\n",
> @@ -221,7 +221,8 @@ static void check_cpu_stall(struct rcu_c
> long delta;
>
> delta = jiffies - rcp->jiffies_stall;
> - if (cpu_isset(smp_processor_id(), rcp->cpumask) && delta >= 0) {
> + if (cpumask_test_cpu(smp_processor_id(), to_cpumask(rcp->cpumask)) &&
> + delta >= 0) {
>
> /* We haven't checked in, so go dump stack. */
> print_cpu_stall(rcp);
> @@ -393,7 +394,8 @@ static void rcu_start_batch(struct rcu_c
> * unnecessarily.
> */
> smp_mb();
> - cpumask_andnot(&rcp->cpumask, cpu_online_mask, nohz_cpu_mask);
> + cpumask_andnot(to_cpumask(rcp->cpumask),
> + cpu_online_mask, nohz_cpu_mask);
>
> rcp->signaled = 0;
> }
> @@ -406,8 +408,8 @@ static void rcu_start_batch(struct rcu_c
> */
> static void cpu_quiet(int cpu, struct rcu_ctrlblk *rcp)
> {
> - cpu_clear(cpu, rcp->cpumask);
> - if (cpus_empty(rcp->cpumask)) {
> + cpumask_clear_cpu(cpu, to_cpumask(rcp->cpumask));
> + if (cpumask_empty(to_cpumask(rcp->cpumask))) {
> /* batch completed ! */
> rcp->completed = rcp->cur;
> rcu_start_batch(rcp);
> --- linux-2.6-for-ingo.orig/kernel/rcupreempt.c
> +++ linux-2.6-for-ingo/kernel/rcupreempt.c
> @@ -164,7 +164,8 @@ static char *rcu_try_flip_state_names[]
> { "idle", "waitack", "waitzero", "waitmb" };
> #endif /* #ifdef CONFIG_RCU_TRACE */
>
> -static cpumask_t rcu_cpu_online_map __read_mostly = CPU_MASK_NONE;
> +static DECLARE_BITMAP(rcu_cpu_online_map, CONFIG_NR_CPUS) __read_mostly
> + = CPU_BITS_NONE;
>
> /*
> * Enum and per-CPU flag to determine when each CPU has seen
> @@ -748,7 +749,7 @@ rcu_try_flip_idle(void)
>
> /* Now ask each CPU for acknowledgement of the flip. */
>
> - for_each_cpu_mask_nr(cpu, rcu_cpu_online_map) {
> + for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map)) {
> per_cpu(rcu_flip_flag, cpu) = rcu_flipped;
> dyntick_save_progress_counter(cpu);
> }
> @@ -766,7 +767,7 @@ rcu_try_flip_waitack(void)
> int cpu;
>
> RCU_TRACE_ME(rcupreempt_trace_try_flip_a1);
> - for_each_cpu_mask_nr(cpu, rcu_cpu_online_map)
> + for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map))
> if (rcu_try_flip_waitack_needed(cpu) &&
> per_cpu(rcu_flip_flag, cpu) != rcu_flip_seen) {
> RCU_TRACE_ME(rcupreempt_trace_try_flip_ae1);
> @@ -798,7 +799,7 @@ rcu_try_flip_waitzero(void)
> /* Check to see if the sum of the "last" counters is zero. */
>
> RCU_TRACE_ME(rcupreempt_trace_try_flip_z1);
> - for_each_cpu_mask_nr(cpu, rcu_cpu_online_map)
> + for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map))
> sum += RCU_DATA_CPU(cpu)->rcu_flipctr[lastidx];
> if (sum != 0) {
> RCU_TRACE_ME(rcupreempt_trace_try_flip_ze1);
> @@ -813,7 +814,7 @@ rcu_try_flip_waitzero(void)
> smp_mb(); /* ^^^^^^^^^^^^ */
>
> /* Call for a memory barrier from each CPU. */
> - for_each_cpu_mask_nr(cpu, rcu_cpu_online_map) {
> + for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map)) {
> per_cpu(rcu_mb_flag, cpu) = rcu_mb_needed;
> dyntick_save_progress_counter(cpu);
> }
> @@ -833,7 +834,7 @@ rcu_try_flip_waitmb(void)
> int cpu;
>
> RCU_TRACE_ME(rcupreempt_trace_try_flip_m1);
> - for_each_cpu_mask_nr(cpu, rcu_cpu_online_map)
> + for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map))
> if (rcu_try_flip_waitmb_needed(cpu) &&
> per_cpu(rcu_mb_flag, cpu) != rcu_mb_done) {
> RCU_TRACE_ME(rcupreempt_trace_try_flip_me1);
> @@ -1022,7 +1023,7 @@ void rcu_offline_cpu(int cpu)
> RCU_DATA_CPU(cpu)->rcu_flipctr[0] = 0;
> RCU_DATA_CPU(cpu)->rcu_flipctr[1] = 0;
>
> - cpu_clear(cpu, rcu_cpu_online_map);
> + cpumask_clear_cpu(cpu, to_cpumask(rcu_cpu_online_map));
>
> spin_unlock_irqrestore(&rcu_ctrlblk.fliplock, flags);
>
> @@ -1062,7 +1063,7 @@ void __cpuinit rcu_online_cpu(int cpu)
> struct rcu_data *rdp;
>
> spin_lock_irqsave(&rcu_ctrlblk.fliplock, flags);
> - cpu_set(cpu, rcu_cpu_online_map);
> + cpumask_set_cpu(cpu, to_cpumask(rcu_cpu_online_map));
> spin_unlock_irqrestore(&rcu_ctrlblk.fliplock, flags);
>
> /*
> @@ -1420,7 +1421,7 @@ void __init __rcu_init(void)
> * We don't need protection against CPU-Hotplug here
> * since
> * a) If a CPU comes online while we are iterating over the
> - * cpu_online_map below, we would only end up making a
> + * cpu_online_mask below, we would only end up making a
> * duplicate call to rcu_online_cpu() which sets the corresponding
> * CPU's mask in the rcu_cpu_online_map.
> *
> --- linux-2.6-for-ingo.orig/kernel/rcutorture.c
> +++ linux-2.6-for-ingo/kernel/rcutorture.c
> @@ -843,49 +843,52 @@ static int rcu_idle_cpu; /* Force all to
> */
> static void rcu_torture_shuffle_tasks(void)
> {
> - cpumask_t tmp_mask;
> + cpumask_var_t tmp_mask;
> int i;
>
> - cpus_setall(tmp_mask);
> + if (!alloc_cpumask_var(&tmp_mask, GFP_KERNEL))
> + BUG();
> +
> + cpumask_setall(tmp_mask);
> get_online_cpus();
>
> /* No point in shuffling if there is only one online CPU (ex: UP) */
> - if (num_online_cpus() == 1) {
> - put_online_cpus();
> - return;
> - }
> + if (num_online_cpus() == 1)
> + goto out;
>
> if (rcu_idle_cpu != -1)
> - cpu_clear(rcu_idle_cpu, tmp_mask);
> + cpumask_clear_cpu(rcu_idle_cpu, tmp_mask);
>
> - set_cpus_allowed_ptr(current, &tmp_mask);
> + set_cpus_allowed_ptr(current, tmp_mask);
>
> if (reader_tasks) {
> for (i = 0; i < nrealreaders; i++)
> if (reader_tasks[i])
> set_cpus_allowed_ptr(reader_tasks[i],
> - &tmp_mask);
> + tmp_mask);
> }
>
> if (fakewriter_tasks) {
> for (i = 0; i < nfakewriters; i++)
> if (fakewriter_tasks[i])
> set_cpus_allowed_ptr(fakewriter_tasks[i],
> - &tmp_mask);
> + tmp_mask);
> }
>
> if (writer_task)
> - set_cpus_allowed_ptr(writer_task, &tmp_mask);
> + set_cpus_allowed_ptr(writer_task, tmp_mask);
>
> if (stats_task)
> - set_cpus_allowed_ptr(stats_task, &tmp_mask);
> + set_cpus_allowed_ptr(stats_task, tmp_mask);
>
> if (rcu_idle_cpu == -1)
> rcu_idle_cpu = num_online_cpus() - 1;
> else
> rcu_idle_cpu--;
>
> +out:
> put_online_cpus();
> + free_cpumask_var(tmp_mask);
> }
>
> /* Shuffle tasks across CPUs, with the intent of allowing each CPU in the
> --- linux-2.6-for-ingo.orig/kernel/softlockup.c
> +++ linux-2.6-for-ingo/kernel/softlockup.c
> @@ -310,10 +310,8 @@ cpu_callback(struct notifier_block *nfb,
> case CPU_DOWN_PREPARE:
> case CPU_DOWN_PREPARE_FROZEN:
> if (hotcpu == check_cpu) {
> - cpumask_t temp_cpu_online_map = cpu_online_map;
> -
> - cpu_clear(hotcpu, temp_cpu_online_map);
> - check_cpu = any_online_cpu(temp_cpu_online_map);
> + /* Pick any other online cpu. */
> + check_cpu = cpumask_any_but(cpu_online_mask, hotcpu);
> }
> break;
>
> --- linux-2.6-for-ingo.orig/kernel/stop_machine.c
> +++ linux-2.6-for-ingo/kernel/stop_machine.c
> @@ -69,10 +69,10 @@ static void stop_cpu(struct work_struct
> int err;
>
> if (!active_cpus) {
> - if (cpu == first_cpu(cpu_online_map))
> + if (cpu == cpumask_first(cpu_online_mask))
> smdata = &active;
> } else {
> - if (cpu_isset(cpu, *active_cpus))
> + if (cpumask_test_cpu(cpu, active_cpus))
> smdata = &active;
> }
> /* Simple state machine */
> @@ -109,7 +109,7 @@ static int chill(void *unused)
> return 0;
> }
>
> -int __stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus)
> +int __stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus)
> {
> struct work_struct *sm_work;
> int i, ret;
> @@ -142,7 +142,7 @@ int __stop_machine(int (*fn)(void *), vo
> return ret;
> }
>
> -int stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus)
> +int stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus)
> {
> int ret;
>
> --- linux-2.6-for-ingo.orig/kernel/taskstats.c
> +++ linux-2.6-for-ingo/kernel/taskstats.c
> @@ -290,18 +290,17 @@ ret:
> return;
> }
>
> -static int add_del_listener(pid_t pid, cpumask_t *maskp, int isadd)
> +static int add_del_listener(pid_t pid, const struct cpumask *mask, int isadd)
> {
> struct listener_list *listeners;
> struct listener *s, *tmp;
> unsigned int cpu;
> - cpumask_t mask = *maskp;
>
> - if (!cpus_subset(mask, cpu_possible_map))
> + if (!cpumask_subset(mask, cpu_possible_mask))
> return -EINVAL;
>
> if (isadd == REGISTER) {
> - for_each_cpu_mask_nr(cpu, mask) {
> + for_each_cpu(cpu, mask) {
> s = kmalloc_node(sizeof(struct listener), GFP_KERNEL,
> cpu_to_node(cpu));
> if (!s)
> @@ -320,7 +319,7 @@ static int add_del_listener(pid_t pid, c
>
> /* Deregister or cleanup */
> cleanup:
> - for_each_cpu_mask_nr(cpu, mask) {
> + for_each_cpu(cpu, mask) {
> listeners = &per_cpu(listener_array, cpu);
> down_write(&listeners->sem);
> list_for_each_entry_safe(s, tmp, &listeners->list, list) {
> @@ -335,7 +334,7 @@ cleanup:
> return 0;
> }
>
> -static int parse(struct nlattr *na, cpumask_t *mask)
> +static int parse(struct nlattr *na, struct cpumask *mask)
> {
> char *data;
> int len;
> @@ -428,23 +427,33 @@ err:
>
> static int taskstats_user_cmd(struct sk_buff *skb, struct genl_info *info)
> {
> - int rc = 0;
> + int rc;
> struct sk_buff *rep_skb;
> struct taskstats *stats;
> size_t size;
> - cpumask_t mask;
> + cpumask_var_t mask;
> +
> + if (!alloc_cpumask_var(&mask, GFP_KERNEL))
> + return -ENOMEM;
>
> - rc = parse(info->attrs[TASKSTATS_CMD_ATTR_REGISTER_CPUMASK], &mask);
> + rc = parse(info->attrs[TASKSTATS_CMD_ATTR_REGISTER_CPUMASK], mask);
> if (rc < 0)
> - return rc;
> - if (rc == 0)
> - return add_del_listener(info->snd_pid, &mask, REGISTER);
> + goto free_return_rc;
> + if (rc == 0) {
> + rc = add_del_listener(info->snd_pid, mask, REGISTER);
> + goto free_return_rc;
> + }
>
> - rc = parse(info->attrs[TASKSTATS_CMD_ATTR_DEREGISTER_CPUMASK], &mask);
> + rc = parse(info->attrs[TASKSTATS_CMD_ATTR_DEREGISTER_CPUMASK], mask);
> if (rc < 0)
> + goto free_return_rc;
> + if (rc == 0) {
> + rc = add_del_listener(info->snd_pid, mask, DEREGISTER);
> +free_return_rc:
> + free_cpumask_var(mask);
> return rc;
> - if (rc == 0)
> - return add_del_listener(info->snd_pid, &mask, DEREGISTER);
> + }
> + free_cpumask_var(mask);
>
> /*
> * Size includes space for nested attributes
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 4/8] cpumask: convert kernel trace functions
2008-12-19 16:54 ` Mike Travis
2008-12-19 17:01 ` Steven Rostedt
@ 2008-12-20 1:41 ` Steven Rostedt
2008-12-20 11:22 ` Rusty Russell
1 sibling, 1 reply; 21+ messages in thread
From: Steven Rostedt @ 2008-12-20 1:41 UTC (permalink / raw)
To: Mike Travis; +Cc: Ingo Molnar, Rusty Russell, linux-kernel
On Fri, 19 Dec 2008, Mike Travis wrote:
> Steven Rostedt wrote:
> > On Fri, 19 Dec 2008, Mike Travis wrote:
> >
> >> Impact: Reduce memory usage, use new cpumask API.
> >>
> >> Convert kernel trace functions to use struct cpumask.
> >>
> > How does this save memory if we just allocate a cpumask var everytime
> > we allocate a ring buffer? Is cpumask_var_t a mask of possible CPUS and
> > not NR_CPUS?
> >
> > Otherwise, I see this as just adding one extra pointer.
> >
> > -- Steve
>
> Hi Steve,
>
> Yes, eventually, the cpumask_var_t will be allocated based on cpumask_size()
> which will become BITS_TO_LONG(nr_cpu_ids) instead of BITS_TO_LONGS(NR_CPUS)
> as soon as the kernel becomes "cpumask" clean. (clean being that it ignores
> all bits >= nr_cpu_ids.)
>
> Note that on small NR_CPUS count systems, cpumask_var_t is a static array
> and no allocation happens, so it only kicks in when CONFIG_CPUMASK_OFFSTACK=y.
For future memory savings...
Acked-by: Steven Rostedt <srostedt@redhat.com>
-- Steve
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 4/8] cpumask: convert kernel trace functions
2008-12-20 1:41 ` Steven Rostedt
@ 2008-12-20 11:22 ` Rusty Russell
0 siblings, 0 replies; 21+ messages in thread
From: Rusty Russell @ 2008-12-20 11:22 UTC (permalink / raw)
To: Steven Rostedt; +Cc: Mike Travis, Ingo Molnar, linux-kernel
On Saturday 20 December 2008 12:11:49 Steven Rostedt wrote:
> For future memory savings...
>
> Acked-by: Steven Rostedt <srostedt@redhat.com>
Thanks for this Mike, this seems a significant enhancement over the
one in my tree (AFAICT I didn't do ring_buffer.c, nor use on_each_cpu).
I've updated the description, and removed one gratuitous indent change.
It's now in the linux-next section of my tree:
cpumask: convert kernel trace functions
Impact: Reduce future memory usage, use new cpumask API.
(Eventually, cpumask_var_t will be allocated based on nr_cpu_ids, not NR_CPUS).
Convert kernel trace functions to use struct cpumask API:
1) Use cpumask_copy/cpumask_test_cpu/for_each_cpu.
2) Use cpumask_var_t and alloc_cpumask_var/free_cpumask_var everywhere.
3) Use on_each_cpu instead of playing with current->cpus_allowed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Acked-by: Steven Rostedt <rostedt@goodmis.org>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 3/8] cpumask: convert kernel time functions
2008-12-19 16:01 ` [PATCH 3/8] cpumask: convert kernel time functions Mike Travis
@ 2008-12-20 11:29 ` Rusty Russell
0 siblings, 0 replies; 21+ messages in thread
From: Rusty Russell @ 2008-12-20 11:29 UTC (permalink / raw)
To: Mike Travis; +Cc: Ingo Molnar, linux-kernel, Thomas Gleixner
On Saturday 20 December 2008 02:31:47 Mike Travis wrote:
> Impact: Reduce stack usage, use new cpumask API.
>
> Convert kernel/time functions to use struct cpumask *.
>
> Note the ugly bitmap declarations in tick-broadcast.c. These should
> be cpumask_var_t, but there was no obvious initialization function to
> put the alloc_cpumask_var() calls in. This was safe.
I still would like see those bitmaps turned into real cpumask_var_t. It
should be easy for someone who knows the code well, or has time to spend
making sure the initialization is early enough.
Cheers,
Rusty.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 5/8] cpumask: convert rest of files in kernel/
2008-12-19 16:01 ` [PATCH 5/8] cpumask: convert rest of files in kernel/ Mike Travis
2008-12-20 1:33 ` Lai Jiangshan
@ 2008-12-20 13:02 ` Rusty Russell
1 sibling, 0 replies; 21+ messages in thread
From: Rusty Russell @ 2008-12-20 13:02 UTC (permalink / raw)
To: Mike Travis
Cc: Ingo Molnar, linux-kernel, Dipankar Sarma, H. Peter Anvin,
Lai Jiangshan, Max Krasnyansky, Richard Henderson,
Thomas Gleixner, Yinghai Lu
On Saturday 20 December 2008 02:31:49 Mike Travis wrote:
> Impact: Reduce stack usage, use new cpumask API. ALPHA mod!
This need to be split, I think. I was just lazy in leaving the conversion
as one big patch.
> Note that prof_cpu_mask will be difficult to convert to a cpumask_var_t
> since it needs to be ready from the first scheduler tick.
Actually, I think this one can be fixed quite easily.
OK, I've done this. Compiles...
Rusty.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 7/8] cpumask: convert misc driver functions
2008-12-19 16:01 ` [PATCH 7/8] cpumask: convert misc driver functions Mike Travis
@ 2008-12-20 23:22 ` Ben Hutchings
2008-12-29 2:28 ` Rusty Russell
2008-12-29 15:37 ` Dean Nelson
1 sibling, 1 reply; 21+ messages in thread
From: Ben Hutchings @ 2008-12-20 23:22 UTC (permalink / raw)
To: Mike Travis
Cc: Ingo Molnar, Rusty Russell, linux-kernel, Dean Nelson,
Jeremy Fitzhardinge, Robert Richter
On Fri, 2008-12-19 at 08:01 -0800, Mike Travis wrote:
[...]
> --- linux-2.6-for-ingo.orig/drivers/net/sfc/efx.c
> +++ linux-2.6-for-ingo/drivers/net/sfc/efx.c
> @@ -809,19 +809,18 @@ static void efx_fini_io(struct efx_nic *
> /* Get number of RX queues wanted. Return number of online CPU
> * packages in the expectation that an IRQ balancer will spread
> * interrupts across them. */
> -static int efx_wanted_rx_queues(void)
> +static int efx_wanted_rx_queues(struct cpumask *scratch)
> {
> - cpumask_t core_mask;
> int count;
> int cpu;
>
> - cpus_clear(core_mask);
> + cpumask_clear(scratch);
> count = 0;
> for_each_online_cpu(cpu) {
> - if (!cpu_isset(cpu, core_mask)) {
> + if (!cpumask_test_cpu(cpu, scratch)) {
> ++count;
> - cpus_or(core_mask, core_mask,
> - topology_core_siblings(cpu));
> + cpumask_or(scratch, scratch,
> + topology_core_cpumask(cpu));
> }
> }
>
> @@ -831,7 +830,7 @@ static int efx_wanted_rx_queues(void)
> /* Probe the number and type of interrupts we are able to obtain, and
> * the resulting numbers of channels and RX queues.
> */
> -static void efx_probe_interrupts(struct efx_nic *efx)
> +static void efx_probe_interrupts(struct efx_nic *efx, struct cpumask *scratch)
> {
> int max_channels =
> min_t(int, efx->type->phys_addr_channels, EFX_MAX_CHANNELS);
> @@ -845,7 +844,8 @@ static void efx_probe_interrupts(struct
> * (or as specified by the rss_cpus module parameter).
> * We will need one channel per interrupt.
> */
> - wanted_ints = rss_cpus ? rss_cpus : efx_wanted_rx_queues();
> + wanted_ints = rss_cpus ? rss_cpus :
> + efx_wanted_rx_queues(scratch);
> efx->n_rx_queues = min(wanted_ints, max_channels);
>
> for (i = 0; i < efx->n_rx_queues; i++)
> @@ -923,24 +923,29 @@ static void efx_set_channels(struct efx_
> static int efx_probe_nic(struct efx_nic *efx)
> {
> int rc;
> + cpumask_var_t scratch;
>
> EFX_LOG(efx, "creating NIC\n");
>
> + if (!alloc_cpumask_var(&scratch, GFP_KERNEL))
> + return -ENOMEM;
> +
> /* Carry out hardware-type specific initialisation */
> rc = falcon_probe_nic(efx);
> if (rc)
> - return rc;
> + goto out;
>
> /* Determine the number of channels and RX queues by trying to hook
> * in MSI-X interrupts. */
> - efx_probe_interrupts(efx);
> + efx_probe_interrupts(efx, scratch);
>
> efx_set_channels(efx);
>
> /* Initialise the interrupt moderation settings */
> efx_init_irq_moderation(efx, tx_irq_mod_usec, rx_irq_mod_usec);
> -
> - return 0;
> +out:
> + free_cpumask_var(scratch);
> + return rc;
> }
>
> static void efx_remove_nic(struct efx_nic *efx)
[...]
I assume you're moving the allocation up to efx_probe_nic() because
efx_wanted_rx_queues() and efx_probe_interrupts() cannot return failure.
It really isn't worth exposing that detail up the call chain though. I
think it's acceptable for efx_wanted_rx_queues() to log an error message
and return 1 in the exceedingly unlikely case that the allocation fails.
Ben.
--
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 7/8] cpumask: convert misc driver functions
2008-12-20 23:22 ` Ben Hutchings
@ 2008-12-29 2:28 ` Rusty Russell
0 siblings, 0 replies; 21+ messages in thread
From: Rusty Russell @ 2008-12-29 2:28 UTC (permalink / raw)
To: Ben Hutchings
Cc: Mike Travis, Ingo Molnar, linux-kernel, Dean Nelson,
Jeremy Fitzhardinge, Robert Richter
On Sunday 21 December 2008 09:52:39 Ben Hutchings wrote:
> I assume you're moving the allocation up to efx_probe_nic() because
> efx_wanted_rx_queues() and efx_probe_interrupts() cannot return failure.
> It really isn't worth exposing that detail up the call chain though. I
> think it's acceptable for efx_wanted_rx_queues() to log an error message
> and return 1 in the exceedingly unlikely case that the allocation fails.
OK, fair call. I was trying trying hard not to break anything.
How's this?
cpumask: convert drivers/net/sfc
Remove a cpumask from the stack. Ben Hutchings indicated that printing
a warning and returning 1 was acceptable for the corner case where allocation
fails.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cc: Ben Hutchings <bhutchings@solarflare.com>
diff --git a/drivers/net/sfc/efx.c b/drivers/net/sfc/efx.c
--- a/drivers/net/sfc/efx.c
+++ b/drivers/net/sfc/efx.c
@@ -854,20 +854,27 @@ static void efx_fini_io(struct efx_nic *
* interrupts across them. */
static int efx_wanted_rx_queues(void)
{
- cpumask_t core_mask;
+ cpumask_var_t core_mask;
int count;
int cpu;
- cpus_clear(core_mask);
+ if (!alloc_cpumask_var(&core_mask, GFP_KERNEL)) {
+ printk(KERN_WARNING
+ "efx.c: allocation failure, irq balancing hobbled\n");
+ return 1;
+ }
+
+ cpumask_clear(core_mask);
count = 0;
for_each_online_cpu(cpu) {
- if (!cpu_isset(cpu, core_mask)) {
+ if (!cpumask_test_cpu(cpu, core_mask)) {
++count;
- cpumask_or(&core_mask, &core_mask,
+ cpumask_or(core_mask, core_mask,
topology_core_cpumask(cpu));
}
}
+ free_cpumask_var(core_mask);
return count;
}
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 7/8] cpumask: convert misc driver functions
2008-12-19 16:01 ` [PATCH 7/8] cpumask: convert misc driver functions Mike Travis
2008-12-20 23:22 ` Ben Hutchings
@ 2008-12-29 15:37 ` Dean Nelson
1 sibling, 0 replies; 21+ messages in thread
From: Dean Nelson @ 2008-12-29 15:37 UTC (permalink / raw)
To: Mike Travis
Cc: Ingo Molnar, Rusty Russell, linux-kernel, Ben Hutchings,
Jeremy Fitzhardinge, Robert Richter
On Fri, Dec 19, 2008 at 08:01:51AM -0800, Mike Travis wrote:
> Impact: Reduce stack usage, use new cpumask API.
>
> Convert misc driver functions to use struct cpumask.
>
> To Do:
> - Convert iucv_buffer_cpumask to cpumask_var_t.
>
> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
> Signed-off-by: Mike Travis <travis@sgi.com>
[...]
> --- linux-2.6-for-ingo.orig/drivers/misc/sgi-xp/xpc_main.c
> +++ linux-2.6-for-ingo/drivers/misc/sgi-xp/xpc_main.c
> @@ -318,7 +318,7 @@ xpc_hb_checker(void *ignore)
>
> /* this thread was marked active by xpc_hb_init() */
>
> - set_cpus_allowed_ptr(current, &cpumask_of_cpu(XPC_HB_CHECK_CPU));
> + set_cpus_allowed_ptr(current, cpumask_of(XPC_HB_CHECK_CPU));
>
> /* set our heartbeating to other partitions into motion */
> xpc_hb_check_timeout = jiffies + (xpc_hb_check_interval * HZ);
Acked-by: Dean Nelson <dcn@sgi.com>
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2008-12-29 15:37 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
2008-12-19 16:01 ` [PATCH 1/8] cpumask: convert kernel/compat.c Mike Travis
2008-12-19 16:01 ` [PATCH 2/8] cpumask: convert kernel/workqueue.c Mike Travis
2008-12-19 16:01 ` [PATCH 3/8] cpumask: convert kernel time functions Mike Travis
2008-12-20 11:29 ` Rusty Russell
2008-12-19 16:01 ` [PATCH 4/8] cpumask: convert kernel trace functions Mike Travis
2008-12-19 16:16 ` Steven Rostedt
2008-12-19 16:54 ` Mike Travis
2008-12-19 17:01 ` Steven Rostedt
2008-12-20 1:41 ` Steven Rostedt
2008-12-20 11:22 ` Rusty Russell
2008-12-19 16:01 ` [PATCH 5/8] cpumask: convert rest of files in kernel/ Mike Travis
2008-12-20 1:33 ` Lai Jiangshan
2008-12-20 13:02 ` Rusty Russell
2008-12-19 16:01 ` [PATCH 6/8] cpumask: convert kernel mm functions Mike Travis
2008-12-19 18:08 ` Christoph Lameter
2008-12-19 16:01 ` [PATCH 7/8] cpumask: convert misc driver functions Mike Travis
2008-12-20 23:22 ` Ben Hutchings
2008-12-29 2:28 ` Rusty Russell
2008-12-29 15:37 ` Dean Nelson
2008-12-19 16:01 ` [PATCH 8/8] cpumask: convert other misc kernel functions Mike Travis
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).