* [RFC PATCH 0/5] Add kernel-offset file and make rcu_read_[un]lock() included
@ 2011-03-28 2:55 Lai Jiangshan
2011-03-28 2:58 ` [RFC PATCH 1/5] Move task's RCU code to rcupdate.h Lai Jiangshan
` (7 more replies)
0 siblings, 8 replies; 67+ messages in thread
From: Lai Jiangshan @ 2011-03-28 2:55 UTC (permalink / raw)
To: Paul E. McKenney, Ingo Molnar, Peter Zijlstra, Dipankar Sarma,
Jan Beulich, David Howells, Alexander van Heukelum,
H. Peter Anvin, Andrew Morton, Roland McGrath, Oleg Nesterov,
Serge Hallyn, linux-kernel, Christoph Lameter, Sam Ravnborg
__rcu_read_[un]lock() are so simple functions and called so
frequent, it is not good that a function call overhead is
required when use them. It would be better if we declare
them inline and let the compiler do the choice.
We add kernel-offset.c for TASK_RCU_OFFSET, which enable we declare
rcu_read_[un]lock() as inline function.
Adding kernel-offset.c does not have any effect for the result
compiled kernel, it just adds a small and fast step in the kbuid stage.
Lai Jiangshan (5):
task_rcu_struct
kbuild_dedumplicated
add_kernel_offset
access task's task_rcu_struct without include sched.h
rcu inline
Kbuild | 99 ++++++++++++++++---------------
include/linux/init_task.h | 25 +--------
include/linux/rcupdate.h | 144 ++++++++++++++++++++++++++++++++++++++++++++-
include/linux/sched.h | 39 +------------
kernel/kernel-offsets.c | 17 +++++
kernel/rcutiny_plugin.h | 58 ++++--------------
kernel/rcutree_plugin.h | 69 ++++++----------------
7 files changed, 243 insertions(+), 208 deletions(-)
create mode 100644 kernel/kernel-offsets.c
--
1.7.4
^ permalink raw reply [flat|nested] 67+ messages in thread* [RFC PATCH 1/5] Move task's RCU code to rcupdate.h 2011-03-28 2:55 [RFC PATCH 0/5] Add kernel-offset file and make rcu_read_[un]lock() included Lai Jiangshan @ 2011-03-28 2:58 ` Lai Jiangshan 2011-03-31 11:31 ` Peter Zijlstra 2011-03-28 2:58 ` [RFC PATCH 2/5] kbuild: dedumplicated the generating code Lai Jiangshan ` (6 subsequent siblings) 7 siblings, 1 reply; 67+ messages in thread From: Lai Jiangshan @ 2011-03-28 2:58 UTC (permalink / raw) To: Paul E. McKenney, Ingo Molnar, Peter Zijlstra, Dipankar Sarma, Jan Beulich, David Howells, Alexander van Heukelum, H. Peter Anvin, Andrew Morton, Roland McGrath, Oleg Nesterov, Serge Hallyn, linux-kernel, Christoph Lameter, Sam Ravnborg Add struct task_rcu_struct to make code clearer and move them to rcupdate.h which help us for code maitainment. task_rcu_struct(), current_task_rcu_struct(), task_of_task_rcu() and rcu_copy_process() have to be implemented as MACRO, it is because we cann't access task's task_rcu_strut without linux/sched.h included. These MACROs can only be used with linux/sched.h included or used in some other MACROs which make this requirement propagate. Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> --- include/linux/init_task.h | 25 +---------------- include/linux/rcupdate.h | 68 ++++++++++++++++++++++++++++++++++++++++++++- include/linux/sched.h | 39 +------------------------- kernel/rcutiny_plugin.h | 26 +++++++++-------- kernel/rcutree_plugin.h | 37 ++++++++++++------------ 5 files changed, 102 insertions(+), 93 deletions(-) diff --git a/include/linux/init_task.h b/include/linux/init_task.h index caa151f..1749002 100644 --- a/include/linux/init_task.h +++ b/include/linux/init_task.h @@ -90,29 +90,6 @@ extern struct group_info init_groups; */ # define CAP_INIT_BSET CAP_FULL_SET -#ifdef CONFIG_RCU_BOOST -#define INIT_TASK_RCU_BOOST() \ - .rcu_boost_mutex = NULL, -#else -#define INIT_TASK_RCU_BOOST() -#endif -#ifdef CONFIG_TREE_PREEMPT_RCU -#define INIT_TASK_RCU_TREE_PREEMPT() \ - .rcu_blocked_node = NULL, -#else -#define INIT_TASK_RCU_TREE_PREEMPT(tsk) -#endif -#ifdef CONFIG_PREEMPT_RCU -#define INIT_TASK_RCU_PREEMPT(tsk) \ - .rcu_read_lock_nesting = 0, \ - .rcu_read_unlock_special = 0, \ - .rcu_node_entry = LIST_HEAD_INIT(tsk.rcu_node_entry), \ - INIT_TASK_RCU_TREE_PREEMPT() \ - INIT_TASK_RCU_BOOST() -#else -#define INIT_TASK_RCU_PREEMPT(tsk) -#endif - extern struct cred init_cred; #ifdef CONFIG_PERF_EVENTS @@ -191,7 +168,7 @@ extern struct cred init_cred; INIT_LOCKDEP \ INIT_FTRACE_GRAPH \ INIT_TRACE_RECURSION \ - INIT_TASK_RCU_PREEMPT(tsk) \ + .task_rcu_struct = INIT_TASK_RCU_STRUCT(tsk.task_rcu_struct), \ } diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 8037493..3500138 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -80,6 +80,72 @@ static inline void __rcu_read_unlock_bh(void) local_bh_enable(); } +/* Special flags for preemptible RCU */ +#define RCU_READ_UNLOCK_BLOCKED (1 << 0) /* blocked while in RCU read-side. */ +#define RCU_READ_UNLOCK_BOOSTED (1 << 1) /* boosted while in RCU read-side. */ +#define RCU_READ_UNLOCK_NEED_QS (1 << 2) /* RCU core needs CPU response. */ + +struct task_struct; + +struct task_rcu_struct { +#ifdef CONFIG_PREEMPT_RCU + int rcu_read_lock_nesting; + char rcu_read_unlock_special; + struct list_head rcu_node_entry; +#ifdef CONFIG_TREE_PREEMPT_RCU + struct rcu_node *rcu_blocked_node; +#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ +#ifdef CONFIG_RCU_BOOST + struct rt_mutex *rcu_boost_mutex; +#endif /* #ifdef CONFIG_RCU_BOOST */ +#endif /* #ifdef CONFIG_PREEMPT_RCU */ +}; + +#ifdef CONFIG_RCU_BOOST +#define INIT_TASK_RCU_BOOST() \ + .rcu_boost_mutex = NULL, +#else +#define INIT_TASK_RCU_BOOST() +#endif +#ifdef CONFIG_TREE_PREEMPT_RCU +#define INIT_TASK_RCU_TREE_PREEMPT() \ + .rcu_blocked_node = NULL, +#else +#define INIT_TASK_RCU_TREE_PREEMPT(tsk) +#endif +#ifdef CONFIG_PREEMPT_RCU +#define INIT_TASK_RCU_STRUCT(task_rcu) { \ + .rcu_read_lock_nesting = 0, \ + .rcu_read_unlock_special = 0, \ + .rcu_node_entry = LIST_HEAD_INIT(task_rcu.rcu_node_entry),\ + INIT_TASK_RCU_TREE_PREEMPT() \ + INIT_TASK_RCU_BOOST() \ + } +#else +#define INIT_TASK_RCU_STRUCT(tsk) +#endif + +static inline void init_task_rcu_struct(struct task_rcu_struct *task_rcu) +{ +#ifdef CONFIG_PREEMPT_RCU + task_rcu->rcu_read_lock_nesting = 0; + task_rcu->rcu_read_unlock_special = 0; + INIT_LIST_HEAD(&task_rcu->rcu_node_entry); +#ifdef CONFIG_TREE_PREEMPT_RCU + task_rcu->rcu_blocked_node = NULL; +#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ +#ifdef CONFIG_RCU_BOOST + task_rcu->rcu_boost_mutex = NULL; +#endif /* #ifdef CONFIG_RCU_BOOST */ +#endif +} + +#define task_rcu_struct(t) (&(t)->task_rcu_struct) +#define current_task_rcu_struct() task_rcu_struct(current) +#define task_of_task_rcu(task_rcu) container_of(task_rcu, struct task_struct,\ + task_rcu_struct) +#define rcu_copy_process(tsk) init_task_rcu_struct(task_rcu_struct(tsk)) + #ifdef CONFIG_PREEMPT_RCU extern void __rcu_read_lock(void); @@ -92,7 +158,7 @@ void synchronize_rcu(void); * nesting depth, but makes sense only if CONFIG_PREEMPT_RCU -- in other * types of kernel builds, the rcu_read_lock() nesting depth is unknowable. */ -#define rcu_preempt_depth() (current->rcu_read_lock_nesting) +#define rcu_preempt_depth() (current_task_rcu_struct()->rcu_read_lock_nesting) #else /* #ifdef CONFIG_PREEMPT_RCU */ diff --git a/include/linux/sched.h b/include/linux/sched.h index 777d8a5..8b359f3 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1232,17 +1232,7 @@ struct task_struct { unsigned int policy; cpumask_t cpus_allowed; -#ifdef CONFIG_PREEMPT_RCU - int rcu_read_lock_nesting; - char rcu_read_unlock_special; - struct list_head rcu_node_entry; -#endif /* #ifdef CONFIG_PREEMPT_RCU */ -#ifdef CONFIG_TREE_PREEMPT_RCU - struct rcu_node *rcu_blocked_node; -#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ -#ifdef CONFIG_RCU_BOOST - struct rt_mutex *rcu_boost_mutex; -#endif /* #ifdef CONFIG_RCU_BOOST */ + struct task_rcu_struct task_rcu_struct; #if defined(CONFIG_SCHEDSTATS) || defined(CONFIG_TASK_DELAY_ACCT) struct sched_info sched_info; @@ -1772,33 +1762,6 @@ extern void thread_group_times(struct task_struct *p, cputime_t *ut, cputime_t * #define tsk_used_math(p) ((p)->flags & PF_USED_MATH) #define used_math() tsk_used_math(current) -#ifdef CONFIG_PREEMPT_RCU - -#define RCU_READ_UNLOCK_BLOCKED (1 << 0) /* blocked while in RCU read-side. */ -#define RCU_READ_UNLOCK_BOOSTED (1 << 1) /* boosted while in RCU read-side. */ -#define RCU_READ_UNLOCK_NEED_QS (1 << 2) /* RCU core needs CPU response. */ - -static inline void rcu_copy_process(struct task_struct *p) -{ - p->rcu_read_lock_nesting = 0; - p->rcu_read_unlock_special = 0; -#ifdef CONFIG_TREE_PREEMPT_RCU - p->rcu_blocked_node = NULL; -#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ -#ifdef CONFIG_RCU_BOOST - p->rcu_boost_mutex = NULL; -#endif /* #ifdef CONFIG_RCU_BOOST */ - INIT_LIST_HEAD(&p->rcu_node_entry); -} - -#else - -static inline void rcu_copy_process(struct task_struct *p) -{ -} - -#endif - #ifdef CONFIG_SMP extern int set_cpus_allowed_ptr(struct task_struct *p, const struct cpumask *new_mask); diff --git a/kernel/rcutiny_plugin.h b/kernel/rcutiny_plugin.h index f259c67..425e892 100644 --- a/kernel/rcutiny_plugin.h +++ b/kernel/rcutiny_plugin.h @@ -150,7 +150,7 @@ static int rcu_cpu_blocking_cur_gp(void) */ static int rcu_preempt_running_reader(void) { - return current->rcu_read_lock_nesting; + return current_task_rcu_struct()->rcu_read_lock_nesting; } /* @@ -192,7 +192,7 @@ static int rcu_preempt_gp_in_progress(void) * Advance a ->blkd_tasks-list pointer to the next entry, instead * returning NULL if at the end of the list. */ -static struct list_head *rcu_next_node_entry(struct task_struct *t) +static struct list_head *rcu_next_node_entry(struct task_rcu_struct *t) { struct list_head *np; @@ -255,7 +255,7 @@ static int rcu_boost(void) { unsigned long flags; struct rt_mutex mtx; - struct task_struct *t; + struct task_rcu_struct *t; struct list_head *tb; if (rcu_preempt_ctrlblk.boost_tasks == NULL && @@ -298,8 +298,8 @@ static int rcu_boost(void) * simply acquiring this artificial rt_mutex will boost task * t's priority. (Thanks to tglx for suggesting this approach!) */ - t = container_of(tb, struct task_struct, rcu_node_entry); - rt_mutex_init_proxy_locked(&mtx, t); + t = container_of(tb, struct task_rcu_struct, rcu_node_entry); + rt_mutex_init_proxy_locked(&mtx, task_of_task_rcu(t)); t->rcu_boost_mutex = &mtx; t->rcu_read_unlock_special |= RCU_READ_UNLOCK_BOOSTED; raw_local_irq_restore(flags); @@ -402,9 +402,11 @@ static void rcu_preempt_boost_start_gp(void) */ static void rcu_preempt_cpu_qs(void) { + struct task_rcu_struct *t = current_task_rcu_struct(); + /* Record both CPU and task as having responded to current GP. */ rcu_preempt_ctrlblk.gpcpu = rcu_preempt_ctrlblk.gpnum; - current->rcu_read_unlock_special &= ~RCU_READ_UNLOCK_NEED_QS; + t->rcu_read_unlock_special &= ~RCU_READ_UNLOCK_NEED_QS; /* If there is no GP then there is nothing more to do. */ if (!rcu_preempt_gp_in_progress()) @@ -473,7 +475,7 @@ static void rcu_preempt_start_gp(void) */ void rcu_preempt_note_context_switch(void) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); unsigned long flags; local_irq_save(flags); /* must exclude scheduler_tick(). */ @@ -518,7 +520,7 @@ void rcu_preempt_note_context_switch(void) */ void __rcu_read_lock(void) { - current->rcu_read_lock_nesting++; + current_task_rcu_struct()->rcu_read_lock_nesting++; barrier(); /* needed if we ever invoke rcu_read_lock in rcutiny.c */ } EXPORT_SYMBOL_GPL(__rcu_read_lock); @@ -528,7 +530,7 @@ EXPORT_SYMBOL_GPL(__rcu_read_lock); * notify RCU core processing or task having blocked during the RCU * read-side critical section. */ -static void rcu_read_unlock_special(struct task_struct *t) +static void rcu_read_unlock_special(struct task_rcu_struct *t) { int empty; int empty_exp; @@ -617,7 +619,7 @@ static void rcu_read_unlock_special(struct task_struct *t) */ void __rcu_read_unlock(void) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); barrier(); /* needed if we ever invoke rcu_read_unlock in rcutiny.c */ --t->rcu_read_lock_nesting; @@ -640,7 +642,7 @@ EXPORT_SYMBOL_GPL(__rcu_read_unlock); */ static void rcu_preempt_check_callbacks(void) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); if (rcu_preempt_gp_in_progress() && (!rcu_preempt_running_reader() || @@ -841,7 +843,7 @@ int rcu_preempt_needs_cpu(void) */ void exit_rcu(void) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); if (t->rcu_read_lock_nesting == 0) return; diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h index 6d6079c..88a12d4 100644 --- a/kernel/rcutree_plugin.h +++ b/kernel/rcutree_plugin.h @@ -120,11 +120,12 @@ EXPORT_SYMBOL_GPL(rcu_force_quiescent_state); static void rcu_preempt_qs(int cpu) { struct rcu_data *rdp = &per_cpu(rcu_preempt_data, cpu); + struct task_rcu_struct *t = current_task_rcu_struct(); rdp->passed_quiesc_completed = rdp->gpnum - 1; barrier(); rdp->passed_quiesc = 1; - current->rcu_read_unlock_special &= ~RCU_READ_UNLOCK_NEED_QS; + t->rcu_read_unlock_special &= ~RCU_READ_UNLOCK_NEED_QS; } /* @@ -142,7 +143,7 @@ static void rcu_preempt_qs(int cpu) */ static void rcu_preempt_note_context_switch(int cpu) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); unsigned long flags; struct rcu_data *rdp; struct rcu_node *rnp; @@ -213,7 +214,7 @@ static void rcu_preempt_note_context_switch(int cpu) */ void __rcu_read_lock(void) { - current->rcu_read_lock_nesting++; + current_task_rcu_struct()->rcu_read_lock_nesting++; barrier(); /* needed if we ever invoke rcu_read_lock in rcutree.c */ } EXPORT_SYMBOL_GPL(__rcu_read_lock); @@ -268,7 +269,7 @@ static void rcu_report_unblock_qs_rnp(struct rcu_node *rnp, unsigned long flags) * Advance a ->blkd_tasks-list pointer to the next entry, instead * returning NULL if at the end of the list. */ -static struct list_head *rcu_next_node_entry(struct task_struct *t, +static struct list_head *rcu_next_node_entry(struct task_rcu_struct *t, struct rcu_node *rnp) { struct list_head *np; @@ -284,7 +285,7 @@ static struct list_head *rcu_next_node_entry(struct task_struct *t, * notify RCU core processing or task having blocked during the RCU * read-side critical section. */ -static void rcu_read_unlock_special(struct task_struct *t) +static void rcu_read_unlock_special(struct task_rcu_struct *t) { int empty; int empty_exp; @@ -384,7 +385,7 @@ static void rcu_read_unlock_special(struct task_struct *t) */ void __rcu_read_unlock(void) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); barrier(); /* needed if we ever invoke rcu_read_unlock in rcutree.c */ --t->rcu_read_lock_nesting; @@ -407,15 +408,15 @@ EXPORT_SYMBOL_GPL(__rcu_read_unlock); static void rcu_print_detail_task_stall_rnp(struct rcu_node *rnp) { unsigned long flags; - struct task_struct *t; + struct task_rcu_struct *t; if (!rcu_preempt_blocked_readers_cgp(rnp)) return; raw_spin_lock_irqsave(&rnp->lock, flags); t = list_entry(rnp->gp_tasks, - struct task_struct, rcu_node_entry); + struct task_rcu_struct, rcu_node_entry); list_for_each_entry_continue(t, &rnp->blkd_tasks, rcu_node_entry) - sched_show_task(t); + sched_show_task(task_of_task_rcu(t)); raw_spin_unlock_irqrestore(&rnp->lock, flags); } @@ -446,14 +447,14 @@ static void rcu_print_detail_task_stall(struct rcu_state *rsp) */ static void rcu_print_task_stall(struct rcu_node *rnp) { - struct task_struct *t; + struct task_rcu_struct *t; if (!rcu_preempt_blocked_readers_cgp(rnp)) return; t = list_entry(rnp->gp_tasks, - struct task_struct, rcu_node_entry); + struct task_rcu_struct, rcu_node_entry); list_for_each_entry_continue(t, &rnp->blkd_tasks, rcu_node_entry) - printk(" P%d", t->pid); + printk(" P%d", task_of_task_rcu(t)->pid); } /* @@ -508,7 +509,7 @@ static int rcu_preempt_offline_tasks(struct rcu_state *rsp, struct list_head *lp_root; int retval = 0; struct rcu_node *rnp_root = rcu_get_root(rsp); - struct task_struct *t; + struct task_rcu_struct *t; if (rnp == rnp_root) { WARN_ONCE(1, "Last CPU thought to be offlined?"); @@ -581,7 +582,7 @@ static void rcu_preempt_offline_cpu(int cpu) */ static void rcu_preempt_check_callbacks(int cpu) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); if (t->rcu_read_lock_nesting == 0) { rcu_preempt_qs(cpu); @@ -851,7 +852,7 @@ static void __init __rcu_init_preempt(void) */ void exit_rcu(void) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); if (t->rcu_read_lock_nesting == 0) return; @@ -1110,7 +1111,7 @@ static int rcu_boost(struct rcu_node *rnp) { unsigned long flags; struct rt_mutex mtx; - struct task_struct *t; + struct task_rcu_struct *t; struct list_head *tb; if (rnp->exp_tasks == NULL && rnp->boost_tasks == NULL) @@ -1158,8 +1159,8 @@ static int rcu_boost(struct rcu_node *rnp) * and task t's exiting its outermost RCU read-side critical * section. */ - t = container_of(tb, struct task_struct, rcu_node_entry); - rt_mutex_init_proxy_locked(&mtx, t); + t = container_of(tb, struct task_rcu_struct, rcu_node_entry); + rt_mutex_init_proxy_locked(&mtx, task_of_task_rcu(t)); t->rcu_boost_mutex = &mtx; t->rcu_read_unlock_special |= RCU_READ_UNLOCK_BOOSTED; raw_spin_unlock_irqrestore(&rnp->lock, flags); -- 1.7.4 ^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 1/5] Move task's RCU code to rcupdate.h 2011-03-28 2:58 ` [RFC PATCH 1/5] Move task's RCU code to rcupdate.h Lai Jiangshan @ 2011-03-31 11:31 ` Peter Zijlstra 2011-03-31 23:24 ` Paul E. McKenney 0 siblings, 1 reply; 67+ messages in thread From: Peter Zijlstra @ 2011-03-31 11:31 UTC (permalink / raw) To: Lai Jiangshan Cc: Paul E. McKenney, Ingo Molnar, Dipankar Sarma, Jan Beulich, David Howells, Alexander van Heukelum, H. Peter Anvin, Andrew Morton, Roland McGrath, Oleg Nesterov, Serge Hallyn, linux-kernel, Christoph Lameter, Sam Ravnborg On Mon, 2011-03-28 at 10:58 +0800, Lai Jiangshan wrote: > +struct task_rcu_struct { > +#ifdef CONFIG_PREEMPT_RCU > + int rcu_read_lock_nesting; > + char rcu_read_unlock_special; Is there a good reason that's a char? It'll leave a 3 byte hole in this location. > + struct list_head rcu_node_entry; > +#ifdef CONFIG_TREE_PREEMPT_RCU > + struct rcu_node *rcu_blocked_node; > +#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ > +#ifdef CONFIG_RCU_BOOST > + struct rt_mutex *rcu_boost_mutex; > +#endif /* #ifdef CONFIG_RCU_BOOST */ > +#endif /* #ifdef CONFIG_PREEMPT_RCU */ > +}; ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 1/5] Move task's RCU code to rcupdate.h 2011-03-31 11:31 ` Peter Zijlstra @ 2011-03-31 23:24 ` Paul E. McKenney 0 siblings, 0 replies; 67+ messages in thread From: Paul E. McKenney @ 2011-03-31 23:24 UTC (permalink / raw) To: Peter Zijlstra Cc: Lai Jiangshan, Ingo Molnar, Dipankar Sarma, Jan Beulich, David Howells, Alexander van Heukelum, H. Peter Anvin, Andrew Morton, Roland McGrath, Oleg Nesterov, Serge Hallyn, linux-kernel, Christoph Lameter, Sam Ravnborg On Thu, Mar 31, 2011 at 01:31:36PM +0200, Peter Zijlstra wrote: > On Mon, 2011-03-28 at 10:58 +0800, Lai Jiangshan wrote: > > +struct task_rcu_struct { > > +#ifdef CONFIG_PREEMPT_RCU > > + int rcu_read_lock_nesting; > > + char rcu_read_unlock_special; > > Is there a good reason that's a char? It'll leave a 3 byte hole in this > location. No good reason. Lai just copied the existing char from the task struct into his task_rcu_struct, so not his fault. ;-) Thanx, Paul > > + struct list_head rcu_node_entry; > > +#ifdef CONFIG_TREE_PREEMPT_RCU > > + struct rcu_node *rcu_blocked_node; > > +#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ > > +#ifdef CONFIG_RCU_BOOST > > + struct rt_mutex *rcu_boost_mutex; > > +#endif /* #ifdef CONFIG_RCU_BOOST */ > > +#endif /* #ifdef CONFIG_PREEMPT_RCU */ > > +}; ^ permalink raw reply [flat|nested] 67+ messages in thread
* [RFC PATCH 2/5] kbuild: dedumplicated the generating code 2011-03-28 2:55 [RFC PATCH 0/5] Add kernel-offset file and make rcu_read_[un]lock() included Lai Jiangshan 2011-03-28 2:58 ` [RFC PATCH 1/5] Move task's RCU code to rcupdate.h Lai Jiangshan @ 2011-03-28 2:58 ` Lai Jiangshan 2011-03-28 8:31 ` Jan Beulich 2011-03-28 2:59 ` [RFC PATCH 3/5] kbuild: add kernel-offset file Lai Jiangshan ` (5 subsequent siblings) 7 siblings, 1 reply; 67+ messages in thread From: Lai Jiangshan @ 2011-03-28 2:58 UTC (permalink / raw) To: Paul E. McKenney, Ingo Molnar, Peter Zijlstra, Dipankar Sarma, Jan Beulich, David Howells, Alexander van Heukelum, H. Peter Anvin, Andrew Morton, Roland McGrath, Oleg Nesterov, Serge Hallyn, linux-kernel, Christoph Lameter, Sam Ravnborg cmd_bounds and cmd_offsets are almost the same, dedumplicated them. The definition of sed-y is also moved up the top of the file. Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> --- Kbuild | 60 +++++++++++++++++++++--------------------------------------- 1 files changed, 21 insertions(+), 39 deletions(-) diff --git a/Kbuild b/Kbuild index 2114113..5c2ee83 100644 --- a/Kbuild +++ b/Kbuild @@ -5,19 +5,19 @@ # 2) Generate asm-offsets.h (may need bounds.h) # 3) Check for missing system calls -##### -# 1) Generate bounds.h - -bounds-file := include/generated/bounds.h - -always := $(bounds-file) -targets := $(bounds-file) kernel/bounds.s +# Default sed regexp - multiline due to syntax constraints +define sed-y + "/^->/{s:->#\(.*\):/* \1 */:; \ + s:^->\([^ ]*\) [\$$#]*\([-0-9]*\) \(.*\):#define \1 \2 /* \3 */:; \ + s:^->\([^ ]*\) [\$$#]*\([^ ]*\) \(.*\):#define \1 \2 /* \3 */:; \ + s:->::; p;}" +endef -quiet_cmd_bounds = GEN $@ -define cmd_bounds +quiet_cmd_kbuild_gen = GEN $@ +define cmd_kbuild_gen (set -e; \ - echo "#ifndef __LINUX_BOUNDS_H__"; \ - echo "#define __LINUX_BOUNDS_H__"; \ + echo "#ifndef __$$(echo $@ | tr -- '-./a-z' '___A-Z')__"; \ + echo "#define __$$(echo $@ | tr -- '-./a-z' '___A-Z')__"; \ echo "/*"; \ echo " * DO NOT MODIFY."; \ echo " *"; \ @@ -30,6 +30,14 @@ define cmd_bounds echo "#endif" ) > $@ endef +##### +# 1) Generate bounds.h + +bounds-file := include/generated/bounds.h + +always := $(bounds-file) +targets := $(bounds-file) kernel/bounds.s + # We use internal kbuild rules to avoid the "is up to date" message from make kernel/bounds.s: kernel/bounds.c FORCE $(Q)mkdir -p $(dir $@) @@ -37,7 +45,7 @@ kernel/bounds.s: kernel/bounds.c FORCE $(obj)/$(bounds-file): kernel/bounds.s Kbuild $(Q)mkdir -p $(dir $@) - $(call cmd,bounds) + $(call cmd,kbuild_gen) ##### # 2) Generate asm-offsets.h @@ -49,32 +57,6 @@ always += $(offsets-file) targets += $(offsets-file) targets += arch/$(SRCARCH)/kernel/asm-offsets.s - -# Default sed regexp - multiline due to syntax constraints -define sed-y - "/^->/{s:->#\(.*\):/* \1 */:; \ - s:^->\([^ ]*\) [\$$#]*\([-0-9]*\) \(.*\):#define \1 \2 /* \3 */:; \ - s:^->\([^ ]*\) [\$$#]*\([^ ]*\) \(.*\):#define \1 \2 /* \3 */:; \ - s:->::; p;}" -endef - -quiet_cmd_offsets = GEN $@ -define cmd_offsets - (set -e; \ - echo "#ifndef __ASM_OFFSETS_H__"; \ - echo "#define __ASM_OFFSETS_H__"; \ - echo "/*"; \ - echo " * DO NOT MODIFY."; \ - echo " *"; \ - echo " * This file was generated by Kbuild"; \ - echo " *"; \ - echo " */"; \ - echo ""; \ - sed -ne $(sed-y) $<; \ - echo ""; \ - echo "#endif" ) > $@ -endef - # We use internal kbuild rules to avoid the "is up to date" message from make arch/$(SRCARCH)/kernel/asm-offsets.s: arch/$(SRCARCH)/kernel/asm-offsets.c \ $(obj)/$(bounds-file) FORCE @@ -82,7 +64,7 @@ arch/$(SRCARCH)/kernel/asm-offsets.s: arch/$(SRCARCH)/kernel/asm-offsets.c \ $(call if_changed_dep,cc_s_c) $(obj)/$(offsets-file): arch/$(SRCARCH)/kernel/asm-offsets.s Kbuild - $(call cmd,offsets) + $(call cmd,kbuild_gen) ##### # 3) Check for missing system calls -- 1.7.4 ^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 2/5] kbuild: dedumplicated the generating code 2011-03-28 2:58 ` [RFC PATCH 2/5] kbuild: dedumplicated the generating code Lai Jiangshan @ 2011-03-28 8:31 ` Jan Beulich 0 siblings, 0 replies; 67+ messages in thread From: Jan Beulich @ 2011-03-28 8:31 UTC (permalink / raw) To: Lai Jiangshan Cc: Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Peter Zijlstra, Andrew Morton, H. Peter Anvin, Paul E. McKenney, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Christoph Lameter, Serge Hallyn, linux-kernel >>> On 28.03.11 at 04:58, Lai Jiangshan <laijs@cn.fujitsu.com> wrote: > > cmd_bounds and cmd_offsets are almost the same, dedumplicated them. > The definition of sed-y is also moved up the top of the file. > > Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> > --- > Kbuild | 60 +++++++++++++++++++++--------------------------------------- > 1 files changed, 21 insertions(+), 39 deletions(-) > > diff --git a/Kbuild b/Kbuild > index 2114113..5c2ee83 100644 > --- a/Kbuild > +++ b/Kbuild > @@ -5,19 +5,19 @@ > # 2) Generate asm-offsets.h (may need bounds.h) > # 3) Check for missing system calls > > -##### > -# 1) Generate bounds.h > - > -bounds-file := include/generated/bounds.h > - > -always := $(bounds-file) > -targets := $(bounds-file) kernel/bounds.s > +# Default sed regexp - multiline due to syntax constraints > +define sed-y > + "/^->/{s:->#\(.*\):/* \1 */:; \ > + s:^->\([^ ]*\) [\$$#]*\([-0-9]*\) \(.*\):#define \1 \2 /* \3 */:; \ > + s:^->\([^ ]*\) [\$$#]*\([^ ]*\) \(.*\):#define \1 \2 /* \3 */:; \ Having to touch (move) this code would imo be a very good opportunity to simplify it again: The replacement string is the same for those two lines, and the second match expression is a super set of the first one. That is, the first one can be dropped, and hence the whole thing changed back to its pre-3234282f33b29d349bcada40204fc7c8fda7fe72 form. Jan > + s:->::; p;}" > +endef > > -quiet_cmd_bounds = GEN $@ > -define cmd_bounds > +quiet_cmd_kbuild_gen = GEN $@ > +define cmd_kbuild_gen > (set -e; \ > - echo "#ifndef __LINUX_BOUNDS_H__"; \ > - echo "#define __LINUX_BOUNDS_H__"; \ > + echo "#ifndef __$$(echo $@ | tr -- '-./a-z' '___A-Z')__"; \ > + echo "#define __$$(echo $@ | tr -- '-./a-z' '___A-Z')__"; \ > echo "/*"; \ > echo " * DO NOT MODIFY."; \ > echo " *"; \ > @@ -30,6 +30,14 @@ define cmd_bounds > echo "#endif" ) > $@ > endef > > +##### > +# 1) Generate bounds.h > + > +bounds-file := include/generated/bounds.h > + > +always := $(bounds-file) > +targets := $(bounds-file) kernel/bounds.s > + > # We use internal kbuild rules to avoid the "is up to date" message from > make > kernel/bounds.s: kernel/bounds.c FORCE > $(Q)mkdir -p $(dir $@) > @@ -37,7 +45,7 @@ kernel/bounds.s: kernel/bounds.c FORCE > > $(obj)/$(bounds-file): kernel/bounds.s Kbuild > $(Q)mkdir -p $(dir $@) > - $(call cmd,bounds) > + $(call cmd,kbuild_gen) > > ##### > # 2) Generate asm-offsets.h > @@ -49,32 +57,6 @@ always += $(offsets-file) > targets += $(offsets-file) > targets += arch/$(SRCARCH)/kernel/asm-offsets.s > > - > -# Default sed regexp - multiline due to syntax constraints > -define sed-y > - "/^->/{s:->#\(.*\):/* \1 */:; \ > - s:^->\([^ ]*\) [\$$#]*\([-0-9]*\) \(.*\):#define \1 \2 /* \3 */:; \ > - s:^->\([^ ]*\) [\$$#]*\([^ ]*\) \(.*\):#define \1 \2 /* \3 */:; \ > - s:->::; p;}" > -endef > - > -quiet_cmd_offsets = GEN $@ > -define cmd_offsets > - (set -e; \ > - echo "#ifndef __ASM_OFFSETS_H__"; \ > - echo "#define __ASM_OFFSETS_H__"; \ > - echo "/*"; \ > - echo " * DO NOT MODIFY."; \ > - echo " *"; \ > - echo " * This file was generated by Kbuild"; \ > - echo " *"; \ > - echo " */"; \ > - echo ""; \ > - sed -ne $(sed-y) $<; \ > - echo ""; \ > - echo "#endif" ) > $@ > -endef > - > # We use internal kbuild rules to avoid the "is up to date" message from > make > arch/$(SRCARCH)/kernel/asm-offsets.s: arch/$(SRCARCH)/kernel/asm-offsets.c > \ > $(obj)/$(bounds-file) FORCE > @@ -82,7 +64,7 @@ arch/$(SRCARCH)/kernel/asm-offsets.s: > arch/$(SRCARCH)/kernel/asm-offsets.c \ > $(call if_changed_dep,cc_s_c) > > $(obj)/$(offsets-file): arch/$(SRCARCH)/kernel/asm-offsets.s Kbuild > - $(call cmd,offsets) > + $(call cmd,kbuild_gen) > > ##### > # 3) Check for missing system calls > -- > 1.7.4 ^ permalink raw reply [flat|nested] 67+ messages in thread
* [RFC PATCH 3/5] kbuild: add kernel-offset file 2011-03-28 2:55 [RFC PATCH 0/5] Add kernel-offset file and make rcu_read_[un]lock() included Lai Jiangshan 2011-03-28 2:58 ` [RFC PATCH 1/5] Move task's RCU code to rcupdate.h Lai Jiangshan 2011-03-28 2:58 ` [RFC PATCH 2/5] kbuild: dedumplicated the generating code Lai Jiangshan @ 2011-03-28 2:59 ` Lai Jiangshan 2011-03-28 3:00 ` [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET Lai Jiangshan ` (4 subsequent siblings) 7 siblings, 0 replies; 67+ messages in thread From: Lai Jiangshan @ 2011-03-28 2:59 UTC (permalink / raw) To: Paul E. McKenney, Ingo Molnar, Peter Zijlstra, Dipankar Sarma, Jan Beulich, David Howells, Alexander van Heukelum, H. Peter Anvin, Andrew Morton, Roland McGrath, Oleg Nesterov, Serge Hallyn, linux-kernel, Christoph Lameter, Sam Ravnborg Add kernel/kernel-offset.c, include/generated/kernel-offsets.h will be generated from this file and some difines need by kernel is generated in this file. This generating is assigned as the last step of the kbuild generating, it would require the other earier generated files. __KBUILD_GENARATING__ is defined when generating, it will be used to avoid compiling errror when generating. Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> --- Kbuild | 41 ++++++++++++++++++++++++++++++++--------- kernel/kernel-offsets.c | 14 ++++++++++++++ 2 files changed, 46 insertions(+), 9 deletions(-) diff --git a/Kbuild b/Kbuild index 5c2ee83..fdfde56 100644 --- a/Kbuild +++ b/Kbuild @@ -3,7 +3,8 @@ # This file takes care of the following: # 1) Generate bounds.h # 2) Generate asm-offsets.h (may need bounds.h) -# 3) Check for missing system calls +# 3) Generate kernel-offsets.h (may need bounds.h and asm-offsets.h) +# 4) Check for missing system calls # Default sed regexp - multiline due to syntax constraints define sed-y @@ -30,6 +31,10 @@ define cmd_kbuild_gen echo "#endif" ) > $@ endef +quiet_cmd_gen_cc_s_c = CC $(quiet_modtag) $@ +cmd_gen_cc_s_c = $(CC) -D__KBUILD_GENARATING__ \ + $(c_flags) -fverbose-asm -S -o $@ $< + ##### # 1) Generate bounds.h @@ -41,7 +46,7 @@ targets := $(bounds-file) kernel/bounds.s # We use internal kbuild rules to avoid the "is up to date" message from make kernel/bounds.s: kernel/bounds.c FORCE $(Q)mkdir -p $(dir $@) - $(call if_changed_dep,cc_s_c) + $(call if_changed_dep,gen_cc_s_c) $(obj)/$(bounds-file): kernel/bounds.s Kbuild $(Q)mkdir -p $(dir $@) @@ -51,23 +56,41 @@ $(obj)/$(bounds-file): kernel/bounds.s Kbuild # 2) Generate asm-offsets.h # -offsets-file := include/generated/asm-offsets.h +asm-offsets-file := include/generated/asm-offsets.h -always += $(offsets-file) -targets += $(offsets-file) +always += $(asm-offsets-file) +targets += $(asm-offsets-file) targets += arch/$(SRCARCH)/kernel/asm-offsets.s # We use internal kbuild rules to avoid the "is up to date" message from make arch/$(SRCARCH)/kernel/asm-offsets.s: arch/$(SRCARCH)/kernel/asm-offsets.c \ $(obj)/$(bounds-file) FORCE $(Q)mkdir -p $(dir $@) - $(call if_changed_dep,cc_s_c) + $(call if_changed_dep,gen_cc_s_c) + +$(obj)/$(asm-offsets-file): arch/$(SRCARCH)/kernel/asm-offsets.s Kbuild + $(call cmd,kbuild_gen) + +##### +# 3) Generate kernel-offsets.h +# + +kernel-offsets-file := include/generated/kernel-offsets.h + +always += $(kernel-offsets-file) +targets += $(kernel-offsets-file) kernel/kernel-offsets.s + +# We use internal kbuild rules to avoid the "is up to date" message from make +kernel/kernel-offsets.s: kernel/kernel-offsets.c $(obj)/$(asm-offsets-file) \ + $(obj)/$(bounds-file) FORCE + $(Q)mkdir -p $(dir $@) + $(call if_changed_dep,gen_cc_s_c) -$(obj)/$(offsets-file): arch/$(SRCARCH)/kernel/asm-offsets.s Kbuild +$(obj)/$(kernel-offsets-file): kernel/kernel-offsets.s Kbuild $(call cmd,kbuild_gen) ##### -# 3) Check for missing system calls +# 4) Check for missing system calls # quiet_cmd_syscalls = CALL $< @@ -78,4 +101,4 @@ missing-syscalls: scripts/checksyscalls.sh FORCE $(call cmd,syscalls) # Keep these two files during make clean -no-clean-files := $(bounds-file) $(offsets-file) +no-clean-files := $(bounds-file) $(asm-offsets-file) $(kernel-offsets-file) diff --git a/kernel/kernel-offsets.c b/kernel/kernel-offsets.c new file mode 100644 index 0000000..fb73ccf --- /dev/null +++ b/kernel/kernel-offsets.c @@ -0,0 +1,14 @@ +/* + * Generate definitions needed by assembly language modules. + * + * Copyright (C) 2010 Lai Jiangshan + */ + +#include <linux/kbuild.h> + +void foo(void); + +void foo(void) +{ +} + -- 1.7.4 ^ permalink raw reply related [flat|nested] 67+ messages in thread
* [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-28 2:55 [RFC PATCH 0/5] Add kernel-offset file and make rcu_read_[un]lock() included Lai Jiangshan ` (2 preceding siblings ...) 2011-03-28 2:59 ` [RFC PATCH 3/5] kbuild: add kernel-offset file Lai Jiangshan @ 2011-03-28 3:00 ` Lai Jiangshan 2011-03-28 8:35 ` Jan Beulich 2011-03-28 3:01 ` [RFC PATCH 5/5] RCU: declare preemptible __rcu_read_[un]lock() as inline function Lai Jiangshan ` (3 subsequent siblings) 7 siblings, 1 reply; 67+ messages in thread From: Lai Jiangshan @ 2011-03-28 3:00 UTC (permalink / raw) To: Paul E. McKenney, Ingo Molnar, Peter Zijlstra, Dipankar Sarma, Jan Beulich, David Howells, Alexander van Heukelum, H. Peter Anvin, Andrew Morton, Roland McGrath, Oleg Nesterov, Serge Hallyn, linux-kernel, Christoph Lameter, Sam Ravnborg We can not include linux/sched.h in linux/rcupdate.h. If we access task's task_rcu_strut without linux/sched.h included, We will failed with compiling error. So we add TASK_RCU_OFFSET, which help us access task's task_rcu_strut without linux/sched.h included. Now, task_rcu_struct(), current_task_rcu_struct(), task_of_task_rcu() and rcu_copy_process() can be used anywhere without linux/sched.h included. Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> --- include/linux/rcupdate.h | 49 +++++++++++++++++++++++++++++++++++++++++---- kernel/kernel-offsets.c | 3 ++ 2 files changed, 47 insertions(+), 5 deletions(-) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 3500138..a5ed3fe 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -140,11 +140,50 @@ static inline void init_task_rcu_struct(struct task_rcu_struct *task_rcu) #endif } -#define task_rcu_struct(t) (&(t)->task_rcu_struct) -#define current_task_rcu_struct() task_rcu_struct(current) -#define task_of_task_rcu(task_rcu) container_of(task_rcu, struct task_struct,\ - task_rcu_struct) -#define rcu_copy_process(tsk) init_task_rcu_struct(task_rcu_struct(tsk)) +#ifdef __KBUILD_GENARATING__ +/* + * Very eary kbuild stage, many important defines including TASK_RCU_OFFSET + * are not available, use dummies to avoid compiling errors. + */ +static inline +struct task_rcu_struct *task_rcu_struct(struct task_struct *t) +{ + (void)t; + return (struct task_rcu_struct *)0x3L; +} + +static inline +struct task_struct *task_of_task_rcu(struct task_rcu_struct *task_rcu) +{ + (void)task_rcu; + return (struct task_struct *)-0x3L; +} +#else +/* for generated TASK_RCU_OFFSET */ +#include <generated/kernel-offsets.h> + +static inline +struct task_rcu_struct *task_rcu_struct(struct task_struct *t) +{ + return (struct task_rcu_struct *)(((void *)t) + TASK_RCU_OFFSET); +} + +static inline +struct task_struct *task_of_task_rcu(struct task_rcu_struct *task_rcu) +{ + return (struct task_struct *)(((void *)task_rcu) - TASK_RCU_OFFSET); +} +#endif + +static inline struct task_rcu_struct *current_task_rcu_struct(void) +{ + return task_rcu_struct(current); +} + +static inline void rcu_copy_process(struct task_struct *tsk) +{ + init_task_rcu_struct(task_rcu_struct(tsk)); +} #ifdef CONFIG_PREEMPT_RCU diff --git a/kernel/kernel-offsets.c b/kernel/kernel-offsets.c index fb73ccf..a890511 100644 --- a/kernel/kernel-offsets.c +++ b/kernel/kernel-offsets.c @@ -4,11 +4,14 @@ * Copyright (C) 2010 Lai Jiangshan */ +#include <linux/rcupdate.h> +#include <linux/sched.h> #include <linux/kbuild.h> void foo(void); void foo(void) { + OFFSET(TASK_RCU_OFFSET, task_struct, task_rcu_struct); } -- 1.7.4 ^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-28 3:00 ` [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET Lai Jiangshan @ 2011-03-28 8:35 ` Jan Beulich 2011-03-29 9:41 ` Lai Jiangshan 2011-03-29 21:14 ` H. Peter Anvin 0 siblings, 2 replies; 67+ messages in thread From: Jan Beulich @ 2011-03-28 8:35 UTC (permalink / raw) To: Lai Jiangshan Cc: Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Peter Zijlstra, Andrew Morton, H. Peter Anvin, Paul E. McKenney, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel >>> On 28.03.11 at 05:00, Lai Jiangshan <laijs@cn.fujitsu.com> wrote: > > We can not include linux/sched.h in linux/rcupdate.h. > If we access task's task_rcu_strut without linux/sched.h included, > We will failed with compiling error. > > So we add TASK_RCU_OFFSET, which help us access > task's task_rcu_strut without linux/sched.h included. > Now, task_rcu_struct(), current_task_rcu_struct(), task_of_task_rcu() and > rcu_copy_process() can be used anywhere without linux/sched.h included. Aren't the offsets-generation methods meant for assembly consumption only? Header dependency problems normally can be solved by splitting headers into a type declaration one and a second one carrying inline function implementations. Is that indeed completely impossible here? Jan > Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> > --- > include/linux/rcupdate.h | 49 > +++++++++++++++++++++++++++++++++++++++++---- > kernel/kernel-offsets.c | 3 ++ > 2 files changed, 47 insertions(+), 5 deletions(-) > > diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h > index 3500138..a5ed3fe 100644 > --- a/include/linux/rcupdate.h > +++ b/include/linux/rcupdate.h > @@ -140,11 +140,50 @@ static inline void init_task_rcu_struct(struct > task_rcu_struct *task_rcu) > #endif > } > > -#define task_rcu_struct(t) (&(t)->task_rcu_struct) > -#define current_task_rcu_struct() task_rcu_struct(current) > -#define task_of_task_rcu(task_rcu) container_of(task_rcu, struct > task_struct,\ > - task_rcu_struct) > -#define rcu_copy_process(tsk) init_task_rcu_struct(task_rcu_struct(tsk)) > +#ifdef __KBUILD_GENARATING__ > +/* > + * Very eary kbuild stage, many important defines including TASK_RCU_OFFSET > + * are not available, use dummies to avoid compiling errors. > + */ > +static inline > +struct task_rcu_struct *task_rcu_struct(struct task_struct *t) > +{ > + (void)t; > + return (struct task_rcu_struct *)0x3L; > +} > + > +static inline > +struct task_struct *task_of_task_rcu(struct task_rcu_struct *task_rcu) > +{ > + (void)task_rcu; > + return (struct task_struct *)-0x3L; > +} > +#else > +/* for generated TASK_RCU_OFFSET */ > +#include <generated/kernel-offsets.h> > + > +static inline > +struct task_rcu_struct *task_rcu_struct(struct task_struct *t) > +{ > + return (struct task_rcu_struct *)(((void *)t) + TASK_RCU_OFFSET); > +} > + > +static inline > +struct task_struct *task_of_task_rcu(struct task_rcu_struct *task_rcu) > +{ > + return (struct task_struct *)(((void *)task_rcu) - TASK_RCU_OFFSET); > +} > +#endif > + > +static inline struct task_rcu_struct *current_task_rcu_struct(void) > +{ > + return task_rcu_struct(current); > +} > + > +static inline void rcu_copy_process(struct task_struct *tsk) > +{ > + init_task_rcu_struct(task_rcu_struct(tsk)); > +} > > #ifdef CONFIG_PREEMPT_RCU > > diff --git a/kernel/kernel-offsets.c b/kernel/kernel-offsets.c > index fb73ccf..a890511 100644 > --- a/kernel/kernel-offsets.c > +++ b/kernel/kernel-offsets.c > @@ -4,11 +4,14 @@ > * Copyright (C) 2010 Lai Jiangshan > */ > > +#include <linux/rcupdate.h> > +#include <linux/sched.h> > #include <linux/kbuild.h> > > void foo(void); > > void foo(void) > { > + OFFSET(TASK_RCU_OFFSET, task_struct, task_rcu_struct); > } > > -- > 1.7.4 ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-28 8:35 ` Jan Beulich @ 2011-03-29 9:41 ` Lai Jiangshan 2011-03-29 21:14 ` H. Peter Anvin 1 sibling, 0 replies; 67+ messages in thread From: Lai Jiangshan @ 2011-03-29 9:41 UTC (permalink / raw) To: Jan Beulich Cc: Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Peter Zijlstra, Andrew Morton, H. Peter Anvin, Paul E. McKenney, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, serge.hallyn, linux-kernel On 03/28/2011 04:35 PM, Jan Beulich wrote: >>>> On 28.03.11 at 05:00, Lai Jiangshan <laijs@cn.fujitsu.com> wrote: >> >> We can not include linux/sched.h in linux/rcupdate.h. >> If we access task's task_rcu_strut without linux/sched.h included, >> We will failed with compiling error. >> >> So we add TASK_RCU_OFFSET, which help us access >> task's task_rcu_strut without linux/sched.h included. >> Now, task_rcu_struct(), current_task_rcu_struct(), task_of_task_rcu() and >> rcu_copy_process() can be used anywhere without linux/sched.h included. > > Aren't the offsets-generation methods meant for assembly > consumption only? Header dependency problems normally can > be solved by splitting headers into a type declaration one and > a second one carrying inline function implementations. Is that > indeed completely impossible here? > That will need to split >100 header files, it is almost impossible. Thanks, Lai ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-28 8:35 ` Jan Beulich 2011-03-29 9:41 ` Lai Jiangshan @ 2011-03-29 21:14 ` H. Peter Anvin 2011-03-29 21:31 ` Paul E. McKenney 1 sibling, 1 reply; 67+ messages in thread From: H. Peter Anvin @ 2011-03-29 21:14 UTC (permalink / raw) To: Jan Beulich Cc: Lai Jiangshan, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Peter Zijlstra, Andrew Morton, Paul E. McKenney, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel On 03/28/2011 01:35 AM, Jan Beulich wrote: >>>> On 28.03.11 at 05:00, Lai Jiangshan <laijs@cn.fujitsu.com> wrote: >> >> We can not include linux/sched.h in linux/rcupdate.h. >> If we access task's task_rcu_strut without linux/sched.h included, >> We will failed with compiling error. >> >> So we add TASK_RCU_OFFSET, which help us access >> task's task_rcu_strut without linux/sched.h included. >> Now, task_rcu_struct(), current_task_rcu_struct(), task_of_task_rcu() and >> rcu_copy_process() can be used anywhere without linux/sched.h included. > > Aren't the offsets-generation methods meant for assembly > consumption only? Header dependency problems normally can > be solved by splitting headers into a type declaration one and > a second one carrying inline function implementations. Is that > indeed completely impossible here? > I have to say that if we have to use hardcoded offsets in C then we have bigger problems. -hpa ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-29 21:14 ` H. Peter Anvin @ 2011-03-29 21:31 ` Paul E. McKenney 2011-03-29 21:32 ` H. Peter Anvin 0 siblings, 1 reply; 67+ messages in thread From: Paul E. McKenney @ 2011-03-29 21:31 UTC (permalink / raw) To: H. Peter Anvin Cc: Jan Beulich, Lai Jiangshan, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Peter Zijlstra, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel On Tue, Mar 29, 2011 at 02:14:23PM -0700, H. Peter Anvin wrote: > On 03/28/2011 01:35 AM, Jan Beulich wrote: > >>>> On 28.03.11 at 05:00, Lai Jiangshan <laijs@cn.fujitsu.com> wrote: > >> > >> We can not include linux/sched.h in linux/rcupdate.h. > >> If we access task's task_rcu_strut without linux/sched.h included, > >> We will failed with compiling error. > >> > >> So we add TASK_RCU_OFFSET, which help us access > >> task's task_rcu_strut without linux/sched.h included. > >> Now, task_rcu_struct(), current_task_rcu_struct(), task_of_task_rcu() and > >> rcu_copy_process() can be used anywhere without linux/sched.h included. > > > > Aren't the offsets-generation methods meant for assembly > > consumption only? Header dependency problems normally can > > be solved by splitting headers into a type declaration one and > > a second one carrying inline function implementations. Is that > > indeed completely impossible here? > > I have to say that if we have to use hardcoded offsets in C then we have > bigger problems. In this case, the offsets are mechanically generated from the structure definitions. Or am I missing your point? Thanx, Paul ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-29 21:31 ` Paul E. McKenney @ 2011-03-29 21:32 ` H. Peter Anvin 2011-03-29 21:47 ` Paul E. McKenney 0 siblings, 1 reply; 67+ messages in thread From: H. Peter Anvin @ 2011-03-29 21:32 UTC (permalink / raw) To: paulmck Cc: Jan Beulich, Lai Jiangshan, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Peter Zijlstra, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel On 03/29/2011 02:31 PM, Paul E. McKenney wrote: >> >> I have to say that if we have to use hardcoded offsets in C then we have >> bigger problems. > > In this case, the offsets are mechanically generated from the structure > definitions. > > Or am I missing your point? > Yes. The point is if we have to pull out these kinds of hacks in *C* code, we are doing it wrong. Not just a little wrong, but completely and totally bonkers wrong. -hpa ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-29 21:32 ` H. Peter Anvin @ 2011-03-29 21:47 ` Paul E. McKenney 2011-03-29 22:01 ` H. Peter Anvin 0 siblings, 1 reply; 67+ messages in thread From: Paul E. McKenney @ 2011-03-29 21:47 UTC (permalink / raw) To: H. Peter Anvin Cc: Jan Beulich, Lai Jiangshan, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Peter Zijlstra, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel On Tue, Mar 29, 2011 at 02:32:30PM -0700, H. Peter Anvin wrote: > On 03/29/2011 02:31 PM, Paul E. McKenney wrote: > >> > >> I have to say that if we have to use hardcoded offsets in C then we have > >> bigger problems. > > > > In this case, the offsets are mechanically generated from the structure > > definitions. > > > > Or am I missing your point? > > Yes. The point is if we have to pull out these kinds of hacks in *C* > code, we are doing it wrong. Not just a little wrong, but completely > and totally bonkers wrong. OK, maybe we are doing it wrong. But in that case, how do you suggest restructuring include/linux/sched.h so that struct task_struct can be safely included everywhere rcu_read_lock() and friends are invoked? Or, on the other hand, what should we be doing so that we don't need to include task_struct everywhere? Thanx, Paul ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-29 21:47 ` Paul E. McKenney @ 2011-03-29 22:01 ` H. Peter Anvin 2011-03-30 0:47 ` Paul E. McKenney 0 siblings, 1 reply; 67+ messages in thread From: H. Peter Anvin @ 2011-03-29 22:01 UTC (permalink / raw) To: paulmck Cc: Jan Beulich, Lai Jiangshan, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Peter Zijlstra, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel On 03/29/2011 02:47 PM, Paul E. McKenney wrote: > On Tue, Mar 29, 2011 at 02:32:30PM -0700, H. Peter Anvin wrote: >> On 03/29/2011 02:31 PM, Paul E. McKenney wrote: >>>> >>>> I have to say that if we have to use hardcoded offsets in C then we have >>>> bigger problems. >>> >>> In this case, the offsets are mechanically generated from the structure >>> definitions. >>> >>> Or am I missing your point? >> >> Yes. The point is if we have to pull out these kinds of hacks in *C* >> code, we are doing it wrong. Not just a little wrong, but completely >> and totally bonkers wrong. > > OK, maybe we are doing it wrong. > > But in that case, how do you suggest restructuring include/linux/sched.h > so that struct task_struct can be safely included everywhere > rcu_read_lock() and friends are invoked? Or, on the other hand, > what should we be doing so that we don't need to include task_struct > everywhere? > Lai's text doesn't give any hint as to the specific nature of the conflict, which makes it hard to come up with a better alternative without having to rediscover the problem from first principles. However, a somewhat logical assumption is that the problem is that struct task_struct includes struct rcu_head, in which case the easiest thing to do is almost certainly to move the definition of struct rcu_head to its own header file, <linux/rcuhead.h>, and include that in <linux/sched.h>, which should make it possible to include <linux/sched.h> in <linux/rcupdate.h>. -hpa ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-29 22:01 ` H. Peter Anvin @ 2011-03-30 0:47 ` Paul E. McKenney 2011-03-30 5:25 ` Lai Jiangshan 0 siblings, 1 reply; 67+ messages in thread From: Paul E. McKenney @ 2011-03-30 0:47 UTC (permalink / raw) To: H. Peter Anvin Cc: Jan Beulich, Lai Jiangshan, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Peter Zijlstra, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel On Tue, Mar 29, 2011 at 03:01:19PM -0700, H. Peter Anvin wrote: > On 03/29/2011 02:47 PM, Paul E. McKenney wrote: > > On Tue, Mar 29, 2011 at 02:32:30PM -0700, H. Peter Anvin wrote: > >> On 03/29/2011 02:31 PM, Paul E. McKenney wrote: > >>>> > >>>> I have to say that if we have to use hardcoded offsets in C then we have > >>>> bigger problems. > >>> > >>> In this case, the offsets are mechanically generated from the structure > >>> definitions. > >>> > >>> Or am I missing your point? > >> > >> Yes. The point is if we have to pull out these kinds of hacks in *C* > >> code, we are doing it wrong. Not just a little wrong, but completely > >> and totally bonkers wrong. > > > > OK, maybe we are doing it wrong. > > > > But in that case, how do you suggest restructuring include/linux/sched.h > > so that struct task_struct can be safely included everywhere > > rcu_read_lock() and friends are invoked? Or, on the other hand, > > what should we be doing so that we don't need to include task_struct > > everywhere? > > Lai's text doesn't give any hint as to the specific nature of the > conflict, which makes it hard to come up with a better alternative > without having to rediscover the problem from first principles. > However, a somewhat logical assumption is that the problem is that > struct task_struct includes struct rcu_head, in which case the easiest > thing to do is almost certainly to move the definition of struct > rcu_head to its own header file, <linux/rcuhead.h>, and include that in > <linux/sched.h>, which should make it possible to include > <linux/sched.h> in <linux/rcupdate.h>. I believe that there are other circular dependencies -- there certainly were a few years back -- but I will defer to Lai. Thanx, Paul ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-30 0:47 ` Paul E. McKenney @ 2011-03-30 5:25 ` Lai Jiangshan 2011-03-30 7:22 ` Lai Jiangshan 2011-03-31 19:25 ` H. Peter Anvin 0 siblings, 2 replies; 67+ messages in thread From: Lai Jiangshan @ 2011-03-30 5:25 UTC (permalink / raw) To: H. Peter Anvin Cc: paulmck, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Peter Zijlstra, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On 03/30/2011 08:47 AM, Paul E. McKenney wrote: > On Tue, Mar 29, 2011 at 03:01:19PM -0700, H. Peter Anvin wrote: >> On 03/29/2011 02:47 PM, Paul E. McKenney wrote: >>> On Tue, Mar 29, 2011 at 02:32:30PM -0700, H. Peter Anvin wrote: >>>> On 03/29/2011 02:31 PM, Paul E. McKenney wrote: >>>>>> >>>>>> I have to say that if we have to use hardcoded offsets in C then we have >>>>>> bigger problems. >>>>> >>>>> In this case, the offsets are mechanically generated from the structure >>>>> definitions. >>>>> >>>>> Or am I missing your point? >>>> >>>> Yes. The point is if we have to pull out these kinds of hacks in *C* >>>> code, we are doing it wrong. Not just a little wrong, but completely >>>> and totally bonkers wrong. >>> >>> OK, maybe we are doing it wrong. >>> >>> But in that case, how do you suggest restructuring include/linux/sched.h >>> so that struct task_struct can be safely included everywhere >>> rcu_read_lock() and friends are invoked? Or, on the other hand, >>> what should we be doing so that we don't need to include task_struct >>> everywhere? >> >> Lai's text doesn't give any hint as to the specific nature of the >> conflict, which makes it hard to come up with a better alternative >> without having to rediscover the problem from first principles. >> However, a somewhat logical assumption is that the problem is that >> struct task_struct includes struct rcu_head, in which case the easiest >> thing to do is almost certainly to move the definition of struct >> rcu_head to its own header file, <linux/rcuhead.h>, and include that in >> <linux/sched.h>, which should make it possible to include >> <linux/sched.h> in <linux/rcupdate.h>. > > I believe that there are other circular dependencies -- there certainly > were a few years back -- but I will defer to Lai. > > Thanx, Paul > Yes, there are other circular dependencies, <linux/sched.h> includes many files which include or indirectly include <linux/rcupdate.h> for struct rcu_head or RCU apis. There are too many to be split all. Even we just create a <linux/task_struct_def.h>, it also needs to include many files which have included or indirectly included <linux/rcupdate.h> for struct rcu_head already. It is still not a easy work to split them, it still requires many subsystem maintainers to participate in. RCU is one of the most import subsystem, I think it is worth to do such "wrong" way. Thanks, Lai P.S. Circular dependencies is one of the original sins of C language. If it is a good change for lots of linux hackers participating in to split all un-split kernel headers, I'd like to do so too. ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-30 5:25 ` Lai Jiangshan @ 2011-03-30 7:22 ` Lai Jiangshan 2011-03-30 10:55 ` Michal Marek 2011-03-31 19:25 ` H. Peter Anvin 1 sibling, 1 reply; 67+ messages in thread From: Lai Jiangshan @ 2011-03-30 7:22 UTC (permalink / raw) To: H. Peter Anvin Cc: paulmck, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Peter Zijlstra, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On 03/30/2011 01:25 PM, Lai Jiangshan wrote: > On 03/30/2011 08:47 AM, Paul E. McKenney wrote: >> On Tue, Mar 29, 2011 at 03:01:19PM -0700, H. Peter Anvin wrote: >>> On 03/29/2011 02:47 PM, Paul E. McKenney wrote: >>>> On Tue, Mar 29, 2011 at 02:32:30PM -0700, H. Peter Anvin wrote: >>>>> On 03/29/2011 02:31 PM, Paul E. McKenney wrote: >>>>>>> >>>>>>> I have to say that if we have to use hardcoded offsets in C then we have >>>>>>> bigger problems. >>>>>> >>>>>> In this case, the offsets are mechanically generated from the structure >>>>>> definitions. >>>>>> >>>>>> Or am I missing your point? >>>>> >>>>> Yes. The point is if we have to pull out these kinds of hacks in *C* >>>>> code, we are doing it wrong. Not just a little wrong, but completely >>>>> and totally bonkers wrong. >>>> >>>> OK, maybe we are doing it wrong. >>>> >>>> But in that case, how do you suggest restructuring include/linux/sched.h >>>> so that struct task_struct can be safely included everywhere >>>> rcu_read_lock() and friends are invoked? Or, on the other hand, >>>> what should we be doing so that we don't need to include task_struct >>>> everywhere? >>> >>> Lai's text doesn't give any hint as to the specific nature of the >>> conflict, which makes it hard to come up with a better alternative >>> without having to rediscover the problem from first principles. >>> However, a somewhat logical assumption is that the problem is that >>> struct task_struct includes struct rcu_head, in which case the easiest >>> thing to do is almost certainly to move the definition of struct >>> rcu_head to its own header file, <linux/rcuhead.h>, and include that in >>> <linux/sched.h>, which should make it possible to include >>> <linux/sched.h> in <linux/rcupdate.h>. >> >> I believe that there are other circular dependencies -- there certainly >> were a few years back -- but I will defer to Lai. >> >> Thanx, Paul >> > > Yes, there are other circular dependencies, <linux/sched.h> includes > many files which include or indirectly include <linux/rcupdate.h> > for struct rcu_head or RCU apis. There are too many to be split all. > <linux/rcupdate.h> is so generic that it is included in many headers. <linux/sched.h> is so complex that it includes many headers. How many files which include(or indirectly include) <linux/rcupdate.h> are included(or indirectly included) by <linux/sched.h>? 714 files! So I can guess 100 files at least is needed to be touched if we resolve this problem by splitting. include/linux/sched.h include/linux/kernel.h include/linux/timex.h include/linux/jiffies.h include/linux/rbtree.h include/linux/thread_info.h include/linux/cpumask.h include/linux/nodemask.h include/linux/mm_types.h arch/alpha/include/asm/system.h arch/arm/include/asm/system.h arch/avr32/include/asm/system.h arch/blackfin/include/asm/system.h arch/cris/include/asm/system.h arch/frv/include/asm/system.h arch/h8300/include/asm/system.h arch/ia64/include/asm/system.h arch/m32r/include/asm/system.h arch/microblaze/include/asm/system.h arch/mips/include/asm/system.h arch/mn10300/include/asm/system.h arch/parisc/include/asm/system.h arch/powerpc/include/asm/system.h arch/s390/include/asm/system.h arch/score/include/asm/system.h arch/sh/include/asm/system.h arch/sparc/include/asm/system.h arch/tile/include/asm/system.h arch/unicore32/include/asm/system.h arch/x86/include/asm/system.h arch/xtensa/include/asm/system.h arch/mips/include/asm/page.h arch/sh/include/asm/page.h arch/powerpc/include/asm/ptrace.h arch/sh/include/asm/ptrace.h arch/sparc/include/asm/ptrace.h arch/alpha/include/asm/cputime.h arch/arm/include/asm/cputime.h arch/avr32/include/asm/cputime.h arch/blackfin/include/asm/cputime.h arch/cris/include/asm/cputime.h arch/frv/include/asm/cputime.h arch/h8300/include/asm/cputime.h arch/ia64/include/asm/cputime.h arch/m32r/include/asm/cputime.h arch/m68k/include/asm/cputime.h arch/microblaze/include/asm/cputime.h arch/mips/include/asm/cputime.h arch/mn10300/include/asm/cputime.h arch/parisc/include/asm/cputime.h arch/powerpc/include/asm/cputime.h arch/s390/include/asm/cputime.h arch/score/include/asm/cputime.h arch/sh/include/asm/cputime.h arch/sparc/include/asm/cputime.h arch/tile/include/asm/cputime.h arch/um/include/asm/cputime.h arch/x86/include/asm/cputime.h arch/xtensa/include/asm/cputime.h include/linux/smp.h include/linux/sem.h include/linux/signal.h include/linux/completion.h include/linux/pid.h include/linux/percpu.h include/linux/topology.h include/linux/proportions.h include/linux/seccomp.h include/linux/rculist.h include/linux/rtmutex.h include/linux/time.h include/linux/resource.h include/linux/timer.h include/linux/hrtimer.h include/linux/cred.h arch/cris/include/asm/processor.h arch/h8300/include/asm/processor.h arch/ia64/include/asm/processor.h arch/m32r/include/asm/processor.h arch/m68k/include/asm/processor.h arch/mips/include/asm/processor.h arch/mn10300/include/asm/processor.h arch/parisc/include/asm/processor.h arch/powerpc/include/asm/processor.h arch/score/include/asm/processor.h arch/sparc/include/asm/processor.h arch/x86/include/asm/processor.h include/linux/spinlock.h include/linux/aio.h arch/alpha/include/asm/current.h arch/arm/include/asm/current.h arch/avr32/include/asm/current.h arch/blackfin/include/asm/current.h arch/cris/include/asm/current.h arch/h8300/include/asm/current.h arch/m32r/include/asm/current.h arch/m68k/include/asm/current.h arch/mips/include/asm/current.h arch/mn10300/include/asm/current.h arch/parisc/include/asm/current.h arch/powerpc/include/asm/current.h arch/score/include/asm/current.h arch/sh/include/asm/current.h arch/sparc/include/asm/current.h arch/tile/include/asm/current.h arch/x86/include/asm/current.h arch/xtensa/include/asm/current.h include/linux/bitops.h include/linux/log2.h arch/s390/include/asm/bug.h arch/arm/include/asm/div64.h arch/ia64/include/asm/timex.h arch/mips/include/asm/timex.h arch/mn10300/include/asm/timex.h arch/parisc/include/asm/timex.h arch/sparc/include/asm/timex.h arch/x86/include/asm/timex.h include/linux/math64.h arch/cris/include/asm/thread_info.h arch/ia64/include/asm/thread_info.h arch/m32r/include/asm/thread_info.h arch/m68k/include/asm/thread_info.h arch/mips/include/asm/thread_info.h arch/parisc/include/asm/thread_info.h arch/powerpc/include/asm/thread_info.h arch/score/include/asm/thread_info.h arch/sh/include/asm/thread_info.h arch/sparc/include/asm/thread_info.h arch/x86/include/asm/thread_info.h include/linux/bitmap.h include/linux/list.h include/linux/rwsem.h arch/sh/include/asm/mmu.h arch/x86/include/asm/mmu.h include/linux/irqflags.h include/asm-generic/cmpxchg-local.h arch/blackfin/include/asm/irq.h arch/mips/include/asm/cmpxchg.h arch/mips/include/asm/dsp.h arch/mips/include/asm/watch.h arch/mn10300/include/asm/atomic.h arch/powerpc/include/asm/hw_irq.h arch/sh/include/asm/uncached.h arch/sparc/include/asm/system_64.h arch/sparc/include/asm/system_32.h arch/x86/include/asm/cpufeature.h arch/x86/include/asm/paravirt.h arch/mips/include/asm/io.h include/asm-generic/cputime.h arch/powerpc/include/asm/time.h include/linux/preempt.h arch/alpha/include/asm/smp.h arch/arm/include/asm/smp.h arch/blackfin/include/asm/smp.h arch/cris/include/asm/smp.h arch/ia64/include/asm/smp.h arch/m32r/include/asm/smp.h arch/mips/include/asm/smp.h arch/mn10300/include/asm/smp.h arch/parisc/include/asm/smp.h arch/powerpc/include/asm/smp.h arch/s390/include/asm/smp.h arch/sh/include/asm/smp.h arch/sparc/include/asm/smp.h arch/tile/include/asm/smp.h arch/x86/include/asm/smp.h include/linux/ipc.h arch/alpha/include/asm/atomic.h arch/arm/include/asm/atomic.h arch/avr32/include/asm/atomic.h arch/blackfin/include/asm/atomic.h arch/cris/include/asm/atomic.h arch/frv/include/asm/atomic.h arch/h8300/include/asm/atomic.h arch/ia64/include/asm/atomic.h arch/m32r/include/asm/atomic.h arch/microblaze/include/asm/atomic.h arch/mips/include/asm/atomic.h arch/parisc/include/asm/atomic.h arch/powerpc/include/asm/atomic.h arch/s390/include/asm/atomic.h arch/score/include/asm/atomic.h arch/sh/include/asm/atomic.h arch/sparc/include/asm/atomic.h arch/tile/include/asm/atomic.h arch/x86/include/asm/atomic.h arch/xtensa/include/asm/atomic.h include/linux/cache.h arch/blackfin/include/asm/signal.h arch/frv/include/asm/signal.h arch/m32r/include/asm/signal.h arch/microblaze/include/asm/signal.h arch/s390/include/asm/signal.h arch/score/include/asm/signal.h arch/sh/include/asm/signal.h arch/sparc/include/asm/signal.h arch/tile/include/asm/signal.h arch/x86/include/asm/signal.h arch/sparc/include/asm/siginfo.h include/linux/wait.h arch/powerpc/include/asm/percpu.h arch/x86/include/asm/percpu.h include/linux/mmzone.h arch/alpha/include/asm/topology.h arch/ia64/include/asm/topology.h arch/powerpc/include/asm/topology.h arch/s390/include/asm/topology.h arch/sparc/include/asm/topology.h arch/tile/include/asm/topology.h arch/x86/include/asm/topology.h include/linux/percpu_counter.h include/linux/mutex.h include/linux/plist.h include/linux/spinlock_types.h include/linux/seqlock.h include/linux/ktime.h include/linux/debugobjects.h include/linux/timerqueue.h include/linux/key.h arch/mips/include/asm/mipsregs.h arch/mn10300/include/asm/uaccess.h arch/sparc/include/asm/processor_64.h arch/sparc/include/asm/processor_32.h arch/x86/include/asm/msr.h arch/alpha/include/asm/spinlock.h arch/blackfin/include/asm/spinlock.h arch/ia64/include/asm/spinlock.h arch/m32r/include/asm/spinlock.h arch/mn10300/include/asm/spinlock.h arch/parisc/include/asm/spinlock.h arch/powerpc/include/asm/spinlock.h arch/s390/include/asm/spinlock.h arch/tile/include/asm/spinlock.h arch/x86/include/asm/spinlock.h arch/xtensa/include/asm/spinlock.h include/linux/workqueue.h include/asm-generic/current.h arch/powerpc/include/asm/paca.h arch/arm/include/asm/bitops.h arch/avr32/include/asm/bitops.h arch/blackfin/include/asm/bitops.h arch/cris/include/asm/bitops.h arch/h8300/include/asm/bitops.h arch/m32r/include/asm/bitops.h arch/microblaze/include/asm/bitops.h arch/mips/include/asm/bitops.h arch/parisc/include/asm/bitops.h arch/score/include/asm/bitops.h arch/sh/include/asm/bitops.h arch/tile/include/asm/bitops.h arch/unicore32/include/asm/bitops.h arch/x86/include/asm/bitops.h arch/xtensa/include/asm/bitops.h arch/mn10300/include/asm/hardirq.h arch/sparc/include/asm/timex_64.h arch/x86/include/asm/tsc.h arch/sparc/include/asm/thread_info_64.h arch/sparc/include/asm/thread_info_32.h include/linux/prefetch.h arch/alpha/include/asm/irqflags.h arch/m68k/include/asm/irqflags.h arch/mn10300/include/asm/irqflags.h arch/powerpc/include/asm/irqflags.h arch/x86/include/asm/irqflags.h include/asm-generic/atomic.h include/linux/bug.h arch/sparc/include/asm/visasm.h arch/powerpc/include/asm/iseries/hv_call.h arch/ia64/include/asm/io.h arch/mips/include/asm/smp-ops.h arch/s390/include/asm/sigp.h arch/sparc/include/asm/smp_64.h arch/sparc/include/asm/smp_32.h arch/x86/include/asm/apic.h arch/x86/include/asm/cpumask.h arch/sparc/include/asm/atomic_64.h arch/sparc/include/asm/atomic_32.h arch/x86/include/asm/alternative.h include/asm-generic/signal.h arch/sparc/include/asm/sigcontext.h include/linux/compat.h include/linux/memory_hotplug.h arch/alpha/include/asm/mmzone.h arch/m32r/include/asm/mmzone.h arch/mips/include/asm/mmzone.h arch/powerpc/include/asm/mmzone.h arch/sparc/include/asm/mmzone.h arch/ia64/include/asm/acpi.h arch/ia64/include/asm/numa.h arch/sparc/include/asm/topology_64.h include/linux/lockdep.h include/linux/mutex-debug.h include/linux/sysctl.h arch/tile/include/asm/spinlock_32.h arch/powerpc/include/asm/lppaca.h include/asm-generic/bitops/atomic.h include/asm-generic/bitops.h arch/tile/include/asm/bitops_32.h include/linux/irq.h arch/sparc/include/asm/timer.h include/linux/hardirq.h arch/ia64/include/asm/unaligned.h include/linux/delay.h include/linux/pm.h arch/x86/include/asm/fixmap.h arch/powerpc/include/asm/sigcontext.h include/linux/stat.h arch/mips/include/asm/compat.h arch/parisc/include/asm/compat.h arch/powerpc/include/asm/compat.h arch/s390/include/asm/compat.h arch/tile/include/asm/compat.h arch/x86/include/asm/compat.h include/linux/notifier.h include/linux/debug_locks.h include/linux/gfp.h include/linux/irqnr.h arch/ia64/include/asm/irq.h arch/m68k/include/asm/irq.h arch/mips/include/asm/irq.h arch/parisc/include/asm/irq.h arch/powerpc/include/asm/irq.h arch/s390/include/asm/irq.h arch/sh/include/asm/irq.h arch/sparc/include/asm/irq.h arch/tile/include/asm/irq.h arch/x86/include/asm/irq.h arch/alpha/include/asm/irq_regs.h arch/arm/include/asm/irq_regs.h arch/avr32/include/asm/irq_regs.h arch/blackfin/include/asm/irq_regs.h arch/cris/include/asm/irq_regs.h arch/h8300/include/asm/irq_regs.h arch/ia64/include/asm/irq_regs.h arch/m32r/include/asm/irq_regs.h arch/m68k/include/asm/irq_regs.h arch/microblaze/include/asm/irq_regs.h arch/mips/include/asm/irq_regs.h arch/parisc/include/asm/irq_regs.h arch/powerpc/include/asm/irq_regs.h arch/s390/include/asm/irq_regs.h arch/score/include/asm/irq_regs.h arch/sh/include/asm/irq_regs.h arch/sparc/include/asm/irq_regs.h arch/tile/include/asm/irq_regs.h arch/um/include/asm/irq_regs.h arch/x86/include/asm/irq_regs.h arch/xtensa/include/asm/irq_regs.h arch/ia64/include/asm/hw_irq.h arch/mips/include/asm/hw_irq.h arch/sh/include/asm/hw_irq.h arch/x86/include/asm/hw_irq.h arch/sparc/include/asm/timer_32.h arch/alpha/include/asm/hardirq.h arch/arm/include/asm/hardirq.h arch/avr32/include/asm/hardirq.h arch/blackfin/include/asm/hardirq.h arch/cris/include/asm/hardirq.h arch/frv/include/asm/hardirq.h arch/h8300/include/asm/hardirq.h arch/ia64/include/asm/hardirq.h arch/m32r/include/asm/hardirq.h arch/microblaze/include/asm/hardirq.h arch/mips/include/asm/hardirq.h arch/parisc/include/asm/hardirq.h arch/powerpc/include/asm/hardirq.h arch/score/include/asm/hardirq.h arch/sh/include/asm/hardirq.h arch/sparc/include/asm/hardirq.h arch/tile/include/asm/hardirq.h arch/um/include/asm/hardirq.h arch/x86/include/asm/hardirq.h arch/xtensa/include/asm/hardirq.h include/linux/unaligned/le_struct.h arch/ia64/include/asm/delay.h arch/parisc/include/asm/delay.h arch/powerpc/include/asm/delay.h arch/sparc/include/asm/delay.h arch/x86/include/asm/acpi.h arch/x86/include/asm/vsyscall.h arch/powerpc/include/asm/elf.h include/linux/srcu.h arch/mips/include/asm/mipsmtregs.h include/linux/radix-tree.h arch/sh/include/asm/machvec.h arch/sparc/include/asm/irq_64.h arch/sparc/include/asm/irq_32.h include/asm-generic/irq_regs.h include/linux/interrupt.h include/linux/profile.h arch/ia64/include/asm/paravirt.h arch/x86/include/asm/sections.h include/asm-generic/hardirq.h arch/sparc/include/asm/hardirq_64.h arch/sparc/include/asm/hardirq_32.h include/linux/unaligned/packed_struct.h arch/sparc/include/asm/delay_32.h arch/x86/include/asm/numa.h arch/x86/include/asm/trampoline.h include/trace/events/irq.h arch/x86/include/asm/uaccess.h arch/sparc/include/asm/cpudata.h arch/x86/include/asm/io.h include/linux/tracepoint.h include/trace/define_trace.h arch/sparc/include/asm/cpudata_64.h arch/sparc/include/asm/cpudata_32.h include/xen/xen.h include/linux/vmalloc.h include/linux/module.h include/trace/ftrace.h arch/ia64/include/asm/xen/hypervisor.h arch/x86/include/asm/xen/hypervisor.h include/linux/kmod.h include/linux/elf.h include/linux/kobject.h include/linux/moduleparam.h arch/mips/include/asm/module.h arch/powerpc/include/asm/module.h arch/score/include/asm/module.h include/trace/events/module.h include/linux/ftrace_event.h arch/mn10300/include/asm/elf.h arch/s390/include/asm/elf.h arch/score/include/asm/elf.h arch/sh/include/asm/elf.h arch/sparc/include/asm/elf.h arch/tile/include/asm/elf.h arch/x86/include/asm/elf.h include/linux/sysfs.h arch/mips/include/asm/uaccess.h arch/score/include/asm/uaccess.h include/linux/ring_buffer.h include/linux/trace_seq.h include/linux/perf_event.h include/linux/utsname.h arch/mn10300/include/asm/user.h arch/s390/include/asm/mmu_context.h include/linux/ptrace.h arch/sh/include/asm/user.h arch/sparc/include/asm/elf_64.h arch/sparc/include/asm/elf_32.h arch/x86/include/asm/desc.h include/linux/kmemcheck.h include/linux/mm.h include/linux/seq_file.h include/linux/fs.h include/linux/cgroup.h arch/powerpc/include/asm/perf_event.h arch/sparc/include/asm/perf_event.h arch/x86/include/asm/perf_event.h arch/alpha/include/asm/local64.h arch/arm/include/asm/local64.h arch/avr32/include/asm/local64.h arch/blackfin/include/asm/local64.h arch/cris/include/asm/local64.h arch/frv/include/asm/local64.h arch/h8300/include/asm/local64.h arch/ia64/include/asm/local64.h arch/m32r/include/asm/local64.h arch/m68k/include/asm/local64.h arch/microblaze/include/asm/local64.h arch/mips/include/asm/local64.h arch/mn10300/include/asm/local64.h arch/parisc/include/asm/local64.h arch/powerpc/include/asm/local64.h arch/s390/include/asm/local64.h arch/score/include/asm/local64.h arch/sh/include/asm/local64.h arch/sparc/include/asm/local64.h arch/x86/include/asm/local64.h arch/xtensa/include/asm/local64.h arch/powerpc/include/asm/hw_breakpoint.h arch/sh/include/asm/hw_breakpoint.h arch/x86/include/asm/hw_breakpoint.h include/linux/pid_namespace.h include/linux/ftrace.h include/linux/cpu.h include/linux/jump_label_ref.h arch/alpha/include/asm/local.h arch/arm/include/asm/local.h arch/avr32/include/asm/local.h arch/blackfin/include/asm/local.h arch/cris/include/asm/local.h arch/frv/include/asm/local.h arch/h8300/include/asm/local.h arch/ia64/include/asm/local.h arch/m32r/include/asm/local.h arch/m68k/include/asm/local.h arch/microblaze/include/asm/local.h arch/mips/include/asm/local.h arch/mn10300/include/asm/local.h arch/parisc/include/asm/local.h arch/powerpc/include/asm/local.h arch/s390/include/asm/local.h arch/score/include/asm/local.h arch/sh/include/asm/local.h arch/sparc/include/asm/local.h arch/tile/include/asm/local.h arch/x86/include/asm/local.h arch/xtensa/include/asm/local.h include/linux/nsproxy.h arch/s390/include/asm/pgalloc.h arch/s390/include/asm/uaccess.h arch/s390/include/asm/tlbflush.h arch/sparc/include/asm/uaccess.h arch/sparc/include/asm/mbus.h include/linux/bit_spinlock.h arch/alpha/include/asm/pgtable.h arch/arm/include/asm/pgtable.h arch/avr32/include/asm/pgtable.h arch/blackfin/include/asm/pgtable.h arch/cris/include/asm/pgtable.h arch/frv/include/asm/pgtable.h arch/h8300/include/asm/pgtable.h arch/ia64/include/asm/pgtable.h arch/m32r/include/asm/pgtable.h arch/microblaze/include/asm/pgtable.h arch/mips/include/asm/pgtable.h arch/mn10300/include/asm/pgtable.h arch/parisc/include/asm/pgtable.h arch/powerpc/include/asm/pgtable.h arch/s390/include/asm/pgtable.h arch/score/include/asm/pgtable.h arch/sh/include/asm/pgtable.h arch/sparc/include/asm/pgtable.h arch/tile/include/asm/pgtable.h arch/um/include/asm/pgtable.h arch/unicore32/include/asm/pgtable.h arch/x86/include/asm/pgtable.h arch/xtensa/include/asm/pgtable.h include/linux/page-flags.h include/linux/vmstat.h include/linux/dcache.h include/linux/semaphore.h include/linux/rculist_bl.h include/linux/quota.h include/linux/nfs_fs_i.h include/linux/prio_heap.h include/linux/idr.h arch/powerpc/include/asm/perf_event_server.h arch/powerpc/include/asm/perf_event_fsl_emb.h arch/x86/include/asm/stacktrace.h include/asm-generic/local64.h include/linux/kdebug.h include/linux/kallsyms.h arch/blackfin/include/asm/ftrace.h include/linux/sysdev.h include/linux/node.h include/asm-generic/local.h arch/sparc/include/asm/uaccess_64.h arch/sparc/include/asm/uaccess_32.h arch/sparc/include/asm/viking.h include/asm-generic/pgtable.h include/linux/slab.h arch/ia64/include/asm/cacheflush.h arch/ia64/include/asm/mmu_context.h arch/mips/include/asm/pgtable-32.h arch/mips/include/asm/pgtable-64.h arch/powerpc/include/asm/pgtable-ppc64.h arch/powerpc/include/asm/pgtable-ppc32.h arch/sh/include/asm/fixmap.h arch/sh/include/asm/pgtable_64.h arch/sparc/include/asm/pgtable_64.h arch/sparc/include/asm/pgtable_32.h arch/tile/include/asm/fixmap.h include/linux/list_bl.h include/linux/uaccess.h arch/mips/include/asm/kdebug.h arch/sparc/include/asm/kdebug.h arch/x86/include/asm/kdebug.h arch/sparc/include/asm/pgtsrmmu.h include/linux/slub_def.h include/linux/slab_def.h arch/mips/include/asm/fixmap.h arch/powerpc/include/asm/tlbflush.h arch/powerpc/include/asm/io.h include/linux/swap.h arch/sparc/include/asm/oplib.h arch/alpha/include/asm/uaccess.h arch/arm/include/asm/uaccess.h arch/avr32/include/asm/uaccess.h arch/blackfin/include/asm/uaccess.h arch/cris/include/asm/uaccess.h arch/frv/include/asm/uaccess.h arch/h8300/include/asm/uaccess.h arch/ia64/include/asm/uaccess.h arch/m32r/include/asm/uaccess.h arch/microblaze/include/asm/uaccess.h arch/parisc/include/asm/uaccess.h arch/powerpc/include/asm/uaccess.h arch/sh/include/asm/uaccess.h arch/tile/include/asm/uaccess.h arch/unicore32/include/asm/uaccess.h arch/xtensa/include/asm/uaccess.h arch/sparc/include/asm/kdebug_32.h include/trace/events/kmem.h include/linux/device.h include/linux/io.h arch/powerpc/include/asm/eeh.h include/linux/memcontrol.h arch/sparc/include/asm/oplib_64.h arch/sparc/include/asm/oplib_32.h include/asm-generic/uaccess.h arch/sparc/include/asm/openprom.h include/linux/klist.h arch/sparc/include/asm/device.h arch/alpha/include/asm/io.h arch/arm/include/asm/io.h arch/avr32/include/asm/io.h arch/blackfin/include/asm/io.h arch/cris/include/asm/io.h arch/frv/include/asm/io.h arch/microblaze/include/asm/io.h arch/mn10300/include/asm/io.h arch/parisc/include/asm/io.h arch/score/include/asm/io.h arch/sh/include/asm/io.h arch/sparc/include/asm/io.h arch/tile/include/asm/io.h arch/unicore32/include/asm/io.h arch/xtensa/include/asm/io.h arch/xtensa/include/asm/segment.h include/linux/of.h arch/alpha/include/asm/core_lca.h arch/alpha/include/asm/core_marvel.h arch/alpha/include/asm/core_t2.h include/asm-generic/io.h arch/mn10300/include/asm/cacheflush.h arch/sh/include/asm/io_trapped.h arch/sparc/include/asm/io_64.h arch/sparc/include/asm/io_32.h arch/microblaze/include/asm/prom.h arch/powerpc/include/asm/prom.h arch/sparc/include/asm/prom.h arch/x86/include/asm/prom.h arch/alpha/include/asm/cacheflush.h arch/arm/include/asm/cacheflush.h arch/avr32/include/asm/cacheflush.h arch/blackfin/include/asm/cacheflush.h arch/cris/include/asm/cacheflush.h arch/frv/include/asm/cacheflush.h arch/m32r/include/asm/cacheflush.h arch/microblaze/include/asm/cacheflush.h arch/mips/include/asm/cacheflush.h arch/parisc/include/asm/cacheflush.h arch/powerpc/include/asm/cacheflush.h arch/s390/include/asm/cacheflush.h arch/score/include/asm/cacheflush.h arch/sh/include/asm/cacheflush.h arch/sparc/include/asm/cacheflush.h arch/tile/include/asm/cacheflush.h arch/unicore32/include/asm/cacheflush.h arch/x86/include/asm/cacheflush.h arch/xtensa/include/asm/cacheflush.h include/linux/of_irq.h include/linux/platform_device.h include/linux/of_address.h include/linux/proc_fs.h include/linux/pci.h include/asm-generic/cacheflush.h arch/sparc/include/asm/cacheflush_64.h arch/sparc/include/asm/cacheflush_32.h include/linux/dmapool.h arch/alpha/include/asm/pci.h arch/arm/include/asm/pci.h arch/avr32/include/asm/pci.h arch/blackfin/include/asm/pci.h arch/cris/include/asm/pci.h arch/frv/include/asm/pci.h arch/ia64/include/asm/pci.h arch/m68k/include/asm/pci.h arch/microblaze/include/asm/pci.h arch/mips/include/asm/pci.h arch/mn10300/include/asm/pci.h arch/parisc/include/asm/pci.h arch/powerpc/include/asm/pci.h arch/sh/include/asm/pci.h arch/sparc/include/asm/pci.h arch/tile/include/asm/pci.h arch/unicore32/include/asm/pci.h arch/x86/include/asm/pci.h arch/xtensa/include/asm/pci.h arch/powerpc/include/asm/scatterlist.h include/linux/dma-mapping.h include/asm-generic/pci-dma-compat.h arch/microblaze/include/asm/pci-bridge.h arch/powerpc/include/asm/machdep.h arch/powerpc/include/asm/pci-bridge.h arch/sparc/include/asm/pci_64.h arch/sparc/include/asm/pci_32.h arch/powerpc/include/asm/dma.h include/linux/dma-attrs.h include/linux/scatterlist.h arch/alpha/include/asm/dma-mapping.h arch/arm/include/asm/dma-mapping.h arch/avr32/include/asm/dma-mapping.h arch/blackfin/include/asm/dma-mapping.h arch/cris/include/asm/dma-mapping.h arch/frv/include/asm/dma-mapping.h arch/ia64/include/asm/dma-mapping.h arch/microblaze/include/asm/dma-mapping.h arch/mips/include/asm/dma-mapping.h arch/mn10300/include/asm/dma-mapping.h arch/parisc/include/asm/dma-mapping.h arch/powerpc/include/asm/dma-mapping.h arch/sh/include/asm/dma-mapping.h arch/sparc/include/asm/dma-mapping.h arch/tile/include/asm/dma-mapping.h arch/unicore32/include/asm/dma-mapping.h arch/x86/include/asm/dma-mapping.h arch/xtensa/include/asm/dma-mapping.h include/asm-generic/dma-mapping-common.h arch/ia64/include/asm/swiotlb.h #!/bin/python import re include_re = re.compile(r'#\s*include\s*<([_\-\w]*)([/\._\-\w]*)>') leaf_file = ('linux', '/rcupdate.h') archs_dir = ( 'arch/alpha/', 'arch/arm/', 'arch/avr32/', 'arch/blackfin/', 'arch/cris/', 'arch/frv/', 'arch/h8300/', 'arch/ia64/', 'arch/m32r/', 'arch/m68k/', 'arch/m68knommu/', 'arch/microblaze/', 'arch/mips/', 'arch/mn10300/', 'arch/parisc/', 'arch/powerpc/', 'arch/s390/', 'arch/score/', 'arch/sh/', 'arch/sparc/', 'arch/tile/', 'arch/um/', 'arch/unicore32/', 'arch/x86/', 'arch/xtensa/', ) generic_dir = 'include/' searched = {('include/linux','/rcupdate.h'):1} searchqueue = [] def add(f): if searched.has_key(f): return searched[f] = 0 searchqueue.append(f) def addall(): i = 0 while i < len(searchqueue): f=searchqueue[i] i=i+1 try: ff = file(f[0]+f[1]) except: continue for ch in include_re.findall(ff.read()): if ch[0] == 'asm': if f[0][0:4] == 'arch': add((f[0], ch[1])) continue for dir in archs_dir: add((dir+'include/asm', ch[1])) else: add((generic_dir+ch[0], ch[1])) def markall(): for f in searchqueue: res = 0 try: ff = file(f[0]+f[1]) except: continue for ch in include_re.findall(ff.read()): if ch[0] == 'asm': if f[0][0:4] == 'arch': res = res or searched[(f[0], ch[1])] continue for dir in archs_dir: res = res or searched[(dir+'include/asm', ch[1])] else: res = res or searched[(generic_dir+ch[0], ch[1])] searched[f]=res add(('include/linux', '/sched.h')) addall() # I'm lazy to build a map and the reverse map..., thousands of headers is not # too much for this. add enough markall() until the result output is not changed markall() markall() markall() markall() markall() markall() markall() markall() markall() markall() markall() #markall() #print len(searchqueue) for f in searchqueue: if searched[f]: print f[0]+f[1] ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-30 7:22 ` Lai Jiangshan @ 2011-03-30 10:55 ` Michal Marek 2011-03-30 10:57 ` Peter Zijlstra 0 siblings, 1 reply; 67+ messages in thread From: Michal Marek @ 2011-03-30 10:55 UTC (permalink / raw) To: Lai Jiangshan Cc: H. Peter Anvin, paulmck, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Peter Zijlstra, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On Wed, Mar 30, 2011 at 03:22:26PM +0800, Lai Jiangshan wrote: > On 03/30/2011 01:25 PM, Lai Jiangshan wrote: > > Yes, there are other circular dependencies, <linux/sched.h> includes > > many files which include or indirectly include <linux/rcupdate.h> > > for struct rcu_head or RCU apis. There are too many to be split all. > > > > <linux/rcupdate.h> is so generic that it is included in many headers. > <linux/sched.h> is so complex that it includes many headers. > > How many files which include(or indirectly include) <linux/rcupdate.h> > are included(or indirectly included) by <linux/sched.h>? > > 714 files! So I can guess 100 files at least is needed to be touched if > we resolve this problem by splitting. It goes down quickly if you start reducing the dependencies. I wrote a quick&dirty patch that splits 9 headers and the preprocessed result of #include <linux/sched_types.h> is about 7800 lines. Most of it is caused by linux/signal.h which I skipped. What do you think? Michal Subject: [PATCH] headers: Allow for lightweight inclusion of task_struct definition Factor out a couple of type definitions to <header>_types.h to allow using task_struct without pulling tons of new dependencies via sched.h. Signed-off-by: Michal Marek <mmarek@suse.cz> --- include/linux/Kbuild | 1 + include/linux/mutex.h | 18 +-- include/linux/mutex_types.h | 22 ++ include/linux/nodemask.h | 2 +- include/linux/nodemask_types.h | 11 + include/linux/pid.h | 15 +- include/linux/pid_types.h | 18 ++ include/linux/proportions.h | 24 +-- include/linux/proportions_types.h | 29 ++ include/linux/rbtree.h | 8 +- include/linux/rbtree_types.h | 11 + include/linux/rcupdate.h | 11 +- include/linux/rcupdate_types.h | 14 + include/linux/sched.h | 481 +----------------------------------- include/linux/sched_types.h | 493 ++++++++++++++++++++++++++++++++++++ include/linux/seccomp.h | 5 +- include/linux/seccomp_types.h | 12 + include/linux/sem.h | 5 +- include/linux/sem_types.h | 10 + include/linux/task_io_accounting.h | 2 + 20 files changed, 633 insertions(+), 559 deletions(-) diff --git a/include/linux/Kbuild b/include/linux/Kbuild index b0ada6f..779afd63 100644 --- a/include/linux/Kbuild +++ b/include/linux/Kbuild @@ -325,6 +325,7 @@ header-y += sdla.h header-y += securebits.h header-y += selinux_netlink.h header-y += sem.h +header-y += sem_types.h header-y += serial.h header-y += serial_core.h header-y += serial_reg.h diff --git a/include/linux/mutex.h b/include/linux/mutex.h index 94b48bd..3cc7492 100644 --- a/include/linux/mutex.h +++ b/include/linux/mutex.h @@ -14,8 +14,8 @@ #include <linux/spinlock_types.h> #include <linux/linkage.h> #include <linux/lockdep.h> - #include <asm/atomic.h> +#include <linux/mutex_types.h> /* * Simple, straightforward mutexes with strict semantics: @@ -45,22 +45,6 @@ * - detects multi-task circular deadlocks and prints out all affected * locks and tasks (and only those tasks) */ -struct mutex { - /* 1: unlocked, 0: locked, negative: locked, possible waiters */ - atomic_t count; - spinlock_t wait_lock; - struct list_head wait_list; -#if defined(CONFIG_DEBUG_MUTEXES) || defined(CONFIG_SMP) - struct thread_info *owner; -#endif -#ifdef CONFIG_DEBUG_MUTEXES - const char *name; - void *magic; -#endif -#ifdef CONFIG_DEBUG_LOCK_ALLOC - struct lockdep_map dep_map; -#endif -}; /* * This is the control structure for tasks blocked on mutex, diff --git a/include/linux/mutex_types.h b/include/linux/mutex_types.h new file mode 100644 index 0000000..51113ab --- /dev/null +++ b/include/linux/mutex_types.h @@ -0,0 +1,22 @@ +#ifndef _LINUX_MUTEX_TYPES_H +#define _LINUX_MUTEX_TYPES_H + + +struct mutex { + /* 1: unlocked, 0: locked, negative: locked, possible waiters */ + atomic_t count; + spinlock_t wait_lock; + struct list_head wait_list; +#if defined(CONFIG_DEBUG_MUTEXES) || defined(CONFIG_SMP) + struct thread_info *owner; +#endif +#ifdef CONFIG_DEBUG_MUTEXES + const char *name; + void *magic; +#endif +#ifdef CONFIG_DEBUG_LOCK_ALLOC + struct lockdep_map dep_map; +#endif +}; + +#endif diff --git a/include/linux/nodemask.h b/include/linux/nodemask.h index dba35e4..ed02421 100644 --- a/include/linux/nodemask.h +++ b/include/linux/nodemask.h @@ -92,8 +92,8 @@ #include <linux/threads.h> #include <linux/bitmap.h> #include <linux/numa.h> +#include <linux/nodemask_types.h> -typedef struct { DECLARE_BITMAP(bits, MAX_NUMNODES); } nodemask_t; extern nodemask_t _unused_nodemask_arg_; #define node_set(node, dst) __node_set((node), &(dst)) diff --git a/include/linux/nodemask_types.h b/include/linux/nodemask_types.h new file mode 100644 index 0000000..97b6614 --- /dev/null +++ b/include/linux/nodemask_types.h @@ -0,0 +1,11 @@ +#ifndef _LINUX_NODEMASK_TYPES_H +#define _LINUX_NODEMASK_TYPES_H + +#include <linux/bitops.h> +#include <linux/kernel.h> +#include <linux/types.h> +#include <linux/numa.h> + +typedef struct { DECLARE_BITMAP(bits, MAX_NUMNODES); } nodemask_t; + +#endif diff --git a/include/linux/pid.h b/include/linux/pid.h index 49f1c2f..8b66f2c 100644 --- a/include/linux/pid.h +++ b/include/linux/pid.h @@ -2,14 +2,7 @@ #define _LINUX_PID_H #include <linux/rcupdate.h> - -enum pid_type -{ - PIDTYPE_PID, - PIDTYPE_PGID, - PIDTYPE_SID, - PIDTYPE_MAX -}; +#include <linux/pid_types.h> /* * What is struct pid? @@ -66,12 +59,6 @@ struct pid extern struct pid init_struct_pid; -struct pid_link -{ - struct hlist_node node; - struct pid *pid; -}; - static inline struct pid *get_pid(struct pid *pid) { if (pid) diff --git a/include/linux/pid_types.h b/include/linux/pid_types.h new file mode 100644 index 0000000..85ab12a --- /dev/null +++ b/include/linux/pid_types.h @@ -0,0 +1,18 @@ +#ifndef _LINUX_PID_TYPES_H +#define _LINUX_PID_TYPES_H + +struct pid; + +enum pid_type { + PIDTYPE_PID, + PIDTYPE_PGID, + PIDTYPE_SID, + PIDTYPE_MAX +}; + +struct pid_link { + struct hlist_node node; + struct pid *pid; +}; + +#endif diff --git a/include/linux/proportions.h b/include/linux/proportions.h index cf793bb..d332304 100644 --- a/include/linux/proportions.h +++ b/include/linux/proportions.h @@ -12,6 +12,7 @@ #include <linux/percpu_counter.h> #include <linux/spinlock.h> #include <linux/mutex.h> +#include <linux/proportions_types.h> struct prop_global { /* @@ -90,29 +91,6 @@ void __prop_inc_percpu_max(struct prop_descriptor *pd, struct prop_local_percpu *pl, long frac); -/* - * ----- SINGLE ------ - */ - -struct prop_local_single { - /* - * the local events counter - */ - unsigned long events; - - /* - * snapshot of the last seen global state - * and a lock protecting this state - */ - unsigned long period; - int shift; - spinlock_t lock; /* protect the snapshot state */ -}; - -#define INIT_PROP_LOCAL_SINGLE(name) \ -{ .lock = __SPIN_LOCK_UNLOCKED(name.lock), \ -} - int prop_local_init_single(struct prop_local_single *pl); void prop_local_destroy_single(struct prop_local_single *pl); void __prop_inc_single(struct prop_descriptor *pd, struct prop_local_single *pl); diff --git a/include/linux/proportions_types.h b/include/linux/proportions_types.h new file mode 100644 index 0000000..24451fb --- /dev/null +++ b/include/linux/proportions_types.h @@ -0,0 +1,29 @@ +#ifndef _LINUX_PROPORTIONS_TYPES_H +#define _LINUX_PROPORTIONS_TYPES_H + +/* + * ----- SINGLE ------ + */ + +#include <linux/spinlock_types.h> + +struct prop_local_single { + /* + * the local events counter + */ + unsigned long events; + + /* + * snapshot of the last seen global state + * and a lock protecting this state + */ + unsigned long period; + int shift; + spinlock_t lock; /* protect the snapshot state */ +}; + +#define INIT_PROP_LOCAL_SINGLE(name) \ +{ .lock = __SPIN_LOCK_UNLOCKED(name.lock), \ +} + +#endif diff --git a/include/linux/rbtree.h b/include/linux/rbtree.h index 7066acb..1ca842d 100644 --- a/include/linux/rbtree.h +++ b/include/linux/rbtree.h @@ -96,16 +96,10 @@ static inline struct page * rb_insert_page_cache(struct inode * inode, #include <linux/kernel.h> #include <linux/stddef.h> +#include <linux/rbtree_types.h> -struct rb_node -{ - unsigned long rb_parent_color; #define RB_RED 0 #define RB_BLACK 1 - struct rb_node *rb_right; - struct rb_node *rb_left; -} __attribute__((aligned(sizeof(long)))); - /* The alignment might seem pointless, but allegedly CRIS needs it */ struct rb_root { diff --git a/include/linux/rbtree_types.h b/include/linux/rbtree_types.h new file mode 100644 index 0000000..decfaef --- /dev/null +++ b/include/linux/rbtree_types.h @@ -0,0 +1,11 @@ +#ifndef _LINUX_RBTREE_TYPES_H +#define _LINUX_RBTREE_TYPES_H + +struct rb_node { + unsigned long rb_parent_color; + struct rb_node *rb_right; + struct rb_node *rb_left; +} __attribute__((aligned(sizeof(long)))); + /* The alignment might seem pointless, but allegedly CRIS needs it */ + +#endif diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index af56148..a901ede 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -42,6 +42,7 @@ #include <linux/completion.h> #include <linux/debugobjects.h> #include <linux/compiler.h> +#include <linux/rcupdate_types.h> #ifdef CONFIG_RCU_TORTURE_TEST extern int rcutorture_runnable; /* for sysctl */ @@ -52,16 +53,6 @@ extern int rcutorture_runnable; /* for sysctl */ #define ULONG_CMP_GE(a, b) (ULONG_MAX / 2 >= (a) - (b)) #define ULONG_CMP_LT(a, b) (ULONG_MAX / 2 < (a) - (b)) -/** - * struct rcu_head - callback structure for use with RCU - * @next: next update requests in a list - * @func: actual update function to call after the grace period. - */ -struct rcu_head { - struct rcu_head *next; - void (*func)(struct rcu_head *head); -}; - /* Exported common interfaces */ extern void call_rcu_sched(struct rcu_head *head, void (*func)(struct rcu_head *rcu)); diff --git a/include/linux/rcupdate_types.h b/include/linux/rcupdate_types.h new file mode 100644 index 0000000..5ee7b54 --- /dev/null +++ b/include/linux/rcupdate_types.h @@ -0,0 +1,14 @@ +#ifndef _LINUX_RCUPDATE_TYPES_H +#define _LINUX_RCUPDATE_TYPES_H + +/** + * struct rcu_head - callback structure for use with RCU + * @next: next update requests in a list + * @func: actual update function to call after the grace period. + */ +struct rcu_head { + struct rcu_head *next; + void (*func)(struct rcu_head *head); +}; + +#endif diff --git a/include/linux/sched.h b/include/linux/sched.h index 777d8a5..9aef559 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -87,12 +87,13 @@ struct sched_param { #include <linux/resource.h> #include <linux/timer.h> #include <linux/hrtimer.h> -#include <linux/task_io_accounting.h> #include <linux/latencytop.h> #include <linux/cred.h> #include <asm/processor.h> +#include <linux/sched_types.h> + struct exec_domain; struct futex_pi_state; struct robust_list_head; @@ -241,9 +242,6 @@ extern char ___assert_task_state[1 - 2*!!( #define set_current_state(state_value) \ set_mb(current->state, (state_value)) -/* Task command name length */ -#define TASK_COMM_LEN 16 - #include <linux/spinlock.h> /* @@ -460,34 +458,6 @@ struct cpu_itimer { u32 incr_error; }; -/** - * struct task_cputime - collected CPU time counts - * @utime: time spent in user mode, in &cputime_t units - * @stime: time spent in kernel mode, in &cputime_t units - * @sum_exec_runtime: total time spent on the CPU, in nanoseconds - * - * This structure groups together three kinds of CPU time that are - * tracked for threads and thread groups. Most things considering - * CPU time want to group these counts together and treat all three - * of them in parallel. - */ -struct task_cputime { - cputime_t utime; - cputime_t stime; - unsigned long long sum_exec_runtime; -}; -/* Alternate field names when used to cache expirations. */ -#define prof_exp stime -#define virt_exp utime -#define sched_exp sum_exec_runtime - -#define INIT_CPUTIME \ - (struct task_cputime) { \ - .utime = cputime_zero, \ - .stime = cputime_zero, \ - .sum_exec_runtime = 0, \ - } - /* * Disable preemption until the scheduler is running. * Reset by start_kernel()->sched_init()->init_idle(). @@ -721,22 +691,6 @@ extern struct user_struct root_user; struct backing_dev_info; struct reclaim_state; -#if defined(CONFIG_SCHEDSTATS) || defined(CONFIG_TASK_DELAY_ACCT) -struct sched_info { - /* cumulative counters */ - unsigned long pcount; /* # of times run on this cpu */ - unsigned long long run_delay; /* time spent waiting on a runqueue */ - - /* timestamps */ - unsigned long long last_arrival,/* when we last ran on a cpu */ - last_queued; /* when we were last queued to run */ -#ifdef CONFIG_SCHEDSTATS - /* BKL stats */ - unsigned int bkl_count; -#endif -}; -#endif /* defined(CONFIG_SCHEDSTATS) || defined(CONFIG_TASK_DELAY_ACCT) */ - #ifdef CONFIG_TASK_DELAY_ACCT struct task_delay_info { spinlock_t lock; @@ -1099,437 +1053,6 @@ struct sched_class { #endif }; -struct load_weight { - unsigned long weight, inv_weight; -}; - -#ifdef CONFIG_SCHEDSTATS -struct sched_statistics { - u64 wait_start; - u64 wait_max; - u64 wait_count; - u64 wait_sum; - u64 iowait_count; - u64 iowait_sum; - - u64 sleep_start; - u64 sleep_max; - s64 sum_sleep_runtime; - - u64 block_start; - u64 block_max; - u64 exec_max; - u64 slice_max; - - u64 nr_migrations_cold; - u64 nr_failed_migrations_affine; - u64 nr_failed_migrations_running; - u64 nr_failed_migrations_hot; - u64 nr_forced_migrations; - - u64 nr_wakeups; - u64 nr_wakeups_sync; - u64 nr_wakeups_migrate; - u64 nr_wakeups_local; - u64 nr_wakeups_remote; - u64 nr_wakeups_affine; - u64 nr_wakeups_affine_attempts; - u64 nr_wakeups_passive; - u64 nr_wakeups_idle; -}; -#endif - -struct sched_entity { - struct load_weight load; /* for load-balancing */ - struct rb_node run_node; - struct list_head group_node; - unsigned int on_rq; - - u64 exec_start; - u64 sum_exec_runtime; - u64 vruntime; - u64 prev_sum_exec_runtime; - - u64 nr_migrations; - -#ifdef CONFIG_SCHEDSTATS - struct sched_statistics statistics; -#endif - -#ifdef CONFIG_FAIR_GROUP_SCHED - struct sched_entity *parent; - /* rq on which this entity is (to be) queued: */ - struct cfs_rq *cfs_rq; - /* rq "owned" by this entity/group: */ - struct cfs_rq *my_q; -#endif -}; - -struct sched_rt_entity { - struct list_head run_list; - unsigned long timeout; - unsigned int time_slice; - int nr_cpus_allowed; - - struct sched_rt_entity *back; -#ifdef CONFIG_RT_GROUP_SCHED - struct sched_rt_entity *parent; - /* rq on which this entity is (to be) queued: */ - struct rt_rq *rt_rq; - /* rq "owned" by this entity/group: */ - struct rt_rq *my_q; -#endif -}; - -struct rcu_node; - -enum perf_event_task_context { - perf_invalid_context = -1, - perf_hw_context = 0, - perf_sw_context, - perf_nr_task_contexts, -}; - -struct task_struct { - volatile long state; /* -1 unrunnable, 0 runnable, >0 stopped */ - void *stack; - atomic_t usage; - unsigned int flags; /* per process flags, defined below */ - unsigned int ptrace; - - int lock_depth; /* BKL lock depth */ - -#ifdef CONFIG_SMP -#ifdef __ARCH_WANT_UNLOCKED_CTXSW - int oncpu; -#endif -#endif - - int prio, static_prio, normal_prio; - unsigned int rt_priority; - const struct sched_class *sched_class; - struct sched_entity se; - struct sched_rt_entity rt; - -#ifdef CONFIG_PREEMPT_NOTIFIERS - /* list of struct preempt_notifier: */ - struct hlist_head preempt_notifiers; -#endif - - /* - * fpu_counter contains the number of consecutive context switches - * that the FPU is used. If this is over a threshold, the lazy fpu - * saving becomes unlazy to save the trap. This is an unsigned char - * so that after 256 times the counter wraps and the behavior turns - * lazy again; this to deal with bursty apps that only use FPU for - * a short time - */ - unsigned char fpu_counter; -#ifdef CONFIG_BLK_DEV_IO_TRACE - unsigned int btrace_seq; -#endif - - unsigned int policy; - cpumask_t cpus_allowed; - -#ifdef CONFIG_PREEMPT_RCU - int rcu_read_lock_nesting; - char rcu_read_unlock_special; - struct list_head rcu_node_entry; -#endif /* #ifdef CONFIG_PREEMPT_RCU */ -#ifdef CONFIG_TREE_PREEMPT_RCU - struct rcu_node *rcu_blocked_node; -#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ -#ifdef CONFIG_RCU_BOOST - struct rt_mutex *rcu_boost_mutex; -#endif /* #ifdef CONFIG_RCU_BOOST */ - -#if defined(CONFIG_SCHEDSTATS) || defined(CONFIG_TASK_DELAY_ACCT) - struct sched_info sched_info; -#endif - - struct list_head tasks; -#ifdef CONFIG_SMP - struct plist_node pushable_tasks; -#endif - - struct mm_struct *mm, *active_mm; -#if defined(SPLIT_RSS_COUNTING) - struct task_rss_stat rss_stat; -#endif -/* task state */ - int exit_state; - int exit_code, exit_signal; - int pdeath_signal; /* The signal sent when the parent dies */ - /* ??? */ - unsigned int personality; - unsigned did_exec:1; - unsigned in_execve:1; /* Tell the LSMs that the process is doing an - * execve */ - unsigned in_iowait:1; - - - /* Revert to default priority/policy when forking */ - unsigned sched_reset_on_fork:1; - - pid_t pid; - pid_t tgid; - -#ifdef CONFIG_CC_STACKPROTECTOR - /* Canary value for the -fstack-protector gcc feature */ - unsigned long stack_canary; -#endif - - /* - * pointers to (original) parent process, youngest child, younger sibling, - * older sibling, respectively. (p->father can be replaced with - * p->real_parent->pid) - */ - struct task_struct *real_parent; /* real parent process */ - struct task_struct *parent; /* recipient of SIGCHLD, wait4() reports */ - /* - * children/sibling forms the list of my natural children - */ - struct list_head children; /* list of my children */ - struct list_head sibling; /* linkage in my parent's children list */ - struct task_struct *group_leader; /* threadgroup leader */ - - /* - * ptraced is the list of tasks this task is using ptrace on. - * This includes both natural children and PTRACE_ATTACH targets. - * p->ptrace_entry is p's link on the p->parent->ptraced list. - */ - struct list_head ptraced; - struct list_head ptrace_entry; - - /* PID/PID hash table linkage. */ - struct pid_link pids[PIDTYPE_MAX]; - struct list_head thread_group; - - struct completion *vfork_done; /* for vfork() */ - int __user *set_child_tid; /* CLONE_CHILD_SETTID */ - int __user *clear_child_tid; /* CLONE_CHILD_CLEARTID */ - - cputime_t utime, stime, utimescaled, stimescaled; - cputime_t gtime; -#ifndef CONFIG_VIRT_CPU_ACCOUNTING - cputime_t prev_utime, prev_stime; -#endif - unsigned long nvcsw, nivcsw; /* context switch counts */ - struct timespec start_time; /* monotonic time */ - struct timespec real_start_time; /* boot based time */ -/* mm fault and swap info: this can arguably be seen as either mm-specific or thread-specific */ - unsigned long min_flt, maj_flt; - - struct task_cputime cputime_expires; - struct list_head cpu_timers[3]; - -/* process credentials */ - const struct cred __rcu *real_cred; /* objective and real subjective task - * credentials (COW) */ - const struct cred __rcu *cred; /* effective (overridable) subjective task - * credentials (COW) */ - struct cred *replacement_session_keyring; /* for KEYCTL_SESSION_TO_PARENT */ - - char comm[TASK_COMM_LEN]; /* executable name excluding path - - access with [gs]et_task_comm (which lock - it with task_lock()) - - initialized normally by setup_new_exec */ -/* file system info */ - int link_count, total_link_count; -#ifdef CONFIG_SYSVIPC -/* ipc stuff */ - struct sysv_sem sysvsem; -#endif -#ifdef CONFIG_DETECT_HUNG_TASK -/* hung task detection */ - unsigned long last_switch_count; -#endif -/* CPU-specific state of this task */ - struct thread_struct thread; -/* filesystem information */ - struct fs_struct *fs; -/* open file information */ - struct files_struct *files; -/* namespaces */ - struct nsproxy *nsproxy; -/* signal handlers */ - struct signal_struct *signal; - struct sighand_struct *sighand; - - sigset_t blocked, real_blocked; - sigset_t saved_sigmask; /* restored if set_restore_sigmask() was used */ - struct sigpending pending; - - unsigned long sas_ss_sp; - size_t sas_ss_size; - int (*notifier)(void *priv); - void *notifier_data; - sigset_t *notifier_mask; - struct audit_context *audit_context; -#ifdef CONFIG_AUDITSYSCALL - uid_t loginuid; - unsigned int sessionid; -#endif - seccomp_t seccomp; - -/* Thread group tracking */ - u32 parent_exec_id; - u32 self_exec_id; -/* Protection of (de-)allocation: mm, files, fs, tty, keyrings, mems_allowed, - * mempolicy */ - spinlock_t alloc_lock; - -#ifdef CONFIG_GENERIC_HARDIRQS - /* IRQ handler threads */ - struct irqaction *irqaction; -#endif - - /* Protection of the PI data structures: */ - raw_spinlock_t pi_lock; - -#ifdef CONFIG_RT_MUTEXES - /* PI waiters blocked on a rt_mutex held by this task */ - struct plist_head pi_waiters; - /* Deadlock detection and priority inheritance handling */ - struct rt_mutex_waiter *pi_blocked_on; -#endif - -#ifdef CONFIG_DEBUG_MUTEXES - /* mutex deadlock detection */ - struct mutex_waiter *blocked_on; -#endif -#ifdef CONFIG_TRACE_IRQFLAGS - unsigned int irq_events; - unsigned long hardirq_enable_ip; - unsigned long hardirq_disable_ip; - unsigned int hardirq_enable_event; - unsigned int hardirq_disable_event; - int hardirqs_enabled; - int hardirq_context; - unsigned long softirq_disable_ip; - unsigned long softirq_enable_ip; - unsigned int softirq_disable_event; - unsigned int softirq_enable_event; - int softirqs_enabled; - int softirq_context; -#endif -#ifdef CONFIG_LOCKDEP -# define MAX_LOCK_DEPTH 48UL - u64 curr_chain_key; - int lockdep_depth; - unsigned int lockdep_recursion; - struct held_lock held_locks[MAX_LOCK_DEPTH]; - gfp_t lockdep_reclaim_gfp; -#endif - -/* journalling filesystem info */ - void *journal_info; - -/* stacked block device info */ - struct bio_list *bio_list; - -/* VM state */ - struct reclaim_state *reclaim_state; - - struct backing_dev_info *backing_dev_info; - - struct io_context *io_context; - - unsigned long ptrace_message; - siginfo_t *last_siginfo; /* For ptrace use. */ - struct task_io_accounting ioac; -#if defined(CONFIG_TASK_XACCT) - u64 acct_rss_mem1; /* accumulated rss usage */ - u64 acct_vm_mem1; /* accumulated virtual memory usage */ - cputime_t acct_timexpd; /* stime + utime since last update */ -#endif -#ifdef CONFIG_CPUSETS - nodemask_t mems_allowed; /* Protected by alloc_lock */ - int mems_allowed_change_disable; - int cpuset_mem_spread_rotor; - int cpuset_slab_spread_rotor; -#endif -#ifdef CONFIG_CGROUPS - /* Control Group info protected by css_set_lock */ - struct css_set __rcu *cgroups; - /* cg_list protected by css_set_lock and tsk->alloc_lock */ - struct list_head cg_list; -#endif -#ifdef CONFIG_FUTEX - struct robust_list_head __user *robust_list; -#ifdef CONFIG_COMPAT - struct compat_robust_list_head __user *compat_robust_list; -#endif - struct list_head pi_state_list; - struct futex_pi_state *pi_state_cache; -#endif -#ifdef CONFIG_PERF_EVENTS - struct perf_event_context *perf_event_ctxp[perf_nr_task_contexts]; - struct mutex perf_event_mutex; - struct list_head perf_event_list; -#endif -#ifdef CONFIG_NUMA - struct mempolicy *mempolicy; /* Protected by alloc_lock */ - short il_next; -#endif - atomic_t fs_excl; /* holding fs exclusive resources */ - struct rcu_head rcu; - - /* - * cache last used pipe for splice - */ - struct pipe_inode_info *splice_pipe; -#ifdef CONFIG_TASK_DELAY_ACCT - struct task_delay_info *delays; -#endif -#ifdef CONFIG_FAULT_INJECTION - int make_it_fail; -#endif - struct prop_local_single dirties; -#ifdef CONFIG_LATENCYTOP - int latency_record_count; - struct latency_record latency_record[LT_SAVECOUNT]; -#endif - /* - * time slack values; these are used to round up poll() and - * select() etc timeout values. These are in nanoseconds. - */ - unsigned long timer_slack_ns; - unsigned long default_timer_slack_ns; - - struct list_head *scm_work_list; -#ifdef CONFIG_FUNCTION_GRAPH_TRACER - /* Index of current stored address in ret_stack */ - int curr_ret_stack; - /* Stack of return addresses for return function tracing */ - struct ftrace_ret_stack *ret_stack; - /* time stamp for last schedule */ - unsigned long long ftrace_timestamp; - /* - * Number of functions that haven't been traced - * because of depth overrun. - */ - atomic_t trace_overrun; - /* Pause for the tracing */ - atomic_t tracing_graph_pause; -#endif -#ifdef CONFIG_TRACING - /* state flags for use by tracers */ - unsigned long trace; - /* bitmask of trace recursion */ - unsigned long trace_recursion; -#endif /* CONFIG_TRACING */ -#ifdef CONFIG_CGROUP_MEM_RES_CTLR /* memcg uses this to do batch job */ - struct memcg_batch_info { - int do_batch; /* incremented when batch uncharge started */ - struct mem_cgroup *memcg; /* target memcg of uncharge */ - unsigned long bytes; /* uncharged usage */ - unsigned long memsw_bytes; /* uncharged mem+swap usage */ - } memcg_batch; -#endif -}; - /* Future-safe accessor for struct task_struct's cpus_allowed. */ #define tsk_cpus_allowed(tsk) (&(tsk)->cpus_allowed) diff --git a/include/linux/sched_types.h b/include/linux/sched_types.h new file mode 100644 index 0000000..ab6df1e --- /dev/null +++ b/include/linux/sched_types.h @@ -0,0 +1,493 @@ +#ifndef _LINUX_SCHED_TYPES_H +#define _LINUX_SCHED_TYPES_H + +#include <linux/task_io_accounting.h> +#include <linux/proportions_types.h> +#include <linux/nodemask_types.h> +#include <linux/rcupdate_types.h> +#include <linux/seccomp_types.h> +#include <linux/rbtree_types.h> +#include <linux/mutex_types.h> +#include <linux/pid_types.h> +#include <linux/sem_types.h> +#include <linux/signal.h> +#include <asm/cputime.h> +#include <linux/plist.h> + +/* Task command name length */ +#define TASK_COMM_LEN 16 + +struct load_weight { + unsigned long weight, inv_weight; +}; + +enum perf_event_task_context { + perf_invalid_context = -1, + perf_hw_context = 0, + perf_sw_context, + perf_nr_task_contexts, +}; + +#if defined(CONFIG_SCHEDSTATS) || defined(CONFIG_TASK_DELAY_ACCT) +struct sched_info { + /* cumulative counters */ + unsigned long pcount; /* # of times run on this cpu */ + unsigned long long run_delay; /* time spent waiting on a runqueue */ + + /* timestamps */ + unsigned long long last_arrival,/* when we last ran on a cpu */ + last_queued; /* when we were last queued to run */ +#ifdef CONFIG_SCHEDSTATS + /* BKL stats */ + unsigned int bkl_count; +#endif +}; +#endif /* defined(CONFIG_SCHEDSTATS) || defined(CONFIG_TASK_DELAY_ACCT) */ + +#ifdef CONFIG_SCHEDSTATS +struct sched_statistics { + u64 wait_start; + u64 wait_max; + u64 wait_count; + u64 wait_sum; + u64 iowait_count; + u64 iowait_sum; + + u64 sleep_start; + u64 sleep_max; + s64 sum_sleep_runtime; + + u64 block_start; + u64 block_max; + u64 exec_max; + u64 slice_max; + + u64 nr_migrations_cold; + u64 nr_failed_migrations_affine; + u64 nr_failed_migrations_running; + u64 nr_failed_migrations_hot; + u64 nr_forced_migrations; + + u64 nr_wakeups; + u64 nr_wakeups_sync; + u64 nr_wakeups_migrate; + u64 nr_wakeups_local; + u64 nr_wakeups_remote; + u64 nr_wakeups_affine; + u64 nr_wakeups_affine_attempts; + u64 nr_wakeups_passive; + u64 nr_wakeups_idle; +}; +#endif + +struct sched_entity { + struct load_weight load; /* for load-balancing */ + struct rb_node run_node; + struct list_head group_node; + unsigned int on_rq; + + u64 exec_start; + u64 sum_exec_runtime; + u64 vruntime; + u64 prev_sum_exec_runtime; + + u64 nr_migrations; + +#ifdef CONFIG_SCHEDSTATS + struct sched_statistics statistics; +#endif + +#ifdef CONFIG_FAIR_GROUP_SCHED + struct sched_entity *parent; + /* rq on which this entity is (to be) queued: */ + struct cfs_rq *cfs_rq; + /* rq "owned" by this entity/group: */ + struct cfs_rq *my_q; +#endif +}; + +struct sched_rt_entity { + struct list_head run_list; + unsigned long timeout; + unsigned int time_slice; + int nr_cpus_allowed; + + struct sched_rt_entity *back; +#ifdef CONFIG_RT_GROUP_SCHED + struct sched_rt_entity *parent; + /* rq on which this entity is (to be) queued: */ + struct rt_rq *rt_rq; + /* rq "owned" by this entity/group: */ + struct rt_rq *my_q; +#endif +}; + +/** + * struct task_cputime - collected CPU time counts + * @utime: time spent in user mode, in &cputime_t units + * @stime: time spent in kernel mode, in &cputime_t units + * @sum_exec_runtime: total time spent on the CPU, in nanoseconds + * + * This structure groups together three kinds of CPU time that are + * tracked for threads and thread groups. Most things considering + * CPU time want to group these counts together and treat all three + * of them in parallel. + */ +struct task_cputime { + cputime_t utime; + cputime_t stime; + unsigned long long sum_exec_runtime; +}; +/* Alternate field names when used to cache expirations. */ +#define prof_exp stime +#define virt_exp utime +#define sched_exp sum_exec_runtime + +#define INIT_CPUTIME \ + (struct task_cputime) { \ + .utime = cputime_zero, \ + .stime = cputime_zero, \ + .sum_exec_runtime = 0, \ + } + +struct task_struct { + volatile long state; /* -1 unrunnable, 0 runnable, >0 stopped */ + void *stack; + atomic_t usage; + unsigned int flags; /* per process flags, defined below */ + unsigned int ptrace; + + int lock_depth; /* BKL lock depth */ + +#ifdef CONFIG_SMP +#ifdef __ARCH_WANT_UNLOCKED_CTXSW + int oncpu; +#endif +#endif + + int prio, static_prio, normal_prio; + unsigned int rt_priority; + const struct sched_class *sched_class; + struct sched_entity se; + struct sched_rt_entity rt; + +#ifdef CONFIG_PREEMPT_NOTIFIERS + /* list of struct preempt_notifier: */ + struct hlist_head preempt_notifiers; +#endif + + /* + * fpu_counter contains the number of consecutive context switches + * that the FPU is used. If this is over a threshold, the lazy fpu + * saving becomes unlazy to save the trap. This is an unsigned char + * so that after 256 times the counter wraps and the behavior turns + * lazy again; this to deal with bursty apps that only use FPU for + * a short time + */ + unsigned char fpu_counter; +#ifdef CONFIG_BLK_DEV_IO_TRACE + unsigned int btrace_seq; +#endif + + unsigned int policy; + cpumask_t cpus_allowed; + +#ifdef CONFIG_PREEMPT_RCU + int rcu_read_lock_nesting; + char rcu_read_unlock_special; + struct list_head rcu_node_entry; +#endif /* #ifdef CONFIG_PREEMPT_RCU */ +#ifdef CONFIG_TREE_PREEMPT_RCU + struct rcu_node *rcu_blocked_node; +#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ +#ifdef CONFIG_RCU_BOOST + struct rt_mutex *rcu_boost_mutex; +#endif /* #ifdef CONFIG_RCU_BOOST */ + +#if defined(CONFIG_SCHEDSTATS) || defined(CONFIG_TASK_DELAY_ACCT) + struct sched_info sched_info; +#endif + + struct list_head tasks; +#ifdef CONFIG_SMP + struct plist_node pushable_tasks; +#endif + + struct mm_struct *mm, *active_mm; +#if defined(SPLIT_RSS_COUNTING) + struct task_rss_stat rss_stat; +#endif +/* task state */ + int exit_state; + int exit_code, exit_signal; + int pdeath_signal; /* The signal sent when the parent dies */ + /* ??? */ + unsigned int personality; + unsigned did_exec:1; + unsigned in_execve:1; /* Tell the LSMs that the process is doing an + * execve */ + unsigned in_iowait:1; + + + /* Revert to default priority/policy when forking */ + unsigned sched_reset_on_fork:1; + + pid_t pid; + pid_t tgid; + +#ifdef CONFIG_CC_STACKPROTECTOR + /* Canary value for the -fstack-protector gcc feature */ + unsigned long stack_canary; +#endif + + /* + * pointers to (original) parent process, youngest child, younger sibling, + * older sibling, respectively. (p->father can be replaced with + * p->real_parent->pid) + */ + struct task_struct *real_parent; /* real parent process */ + struct task_struct *parent; /* recipient of SIGCHLD, wait4() reports */ + /* + * children/sibling forms the list of my natural children + */ + struct list_head children; /* list of my children */ + struct list_head sibling; /* linkage in my parent's children list */ + struct task_struct *group_leader; /* threadgroup leader */ + + /* + * ptraced is the list of tasks this task is using ptrace on. + * This includes both natural children and PTRACE_ATTACH targets. + * p->ptrace_entry is p's link on the p->parent->ptraced list. + */ + struct list_head ptraced; + struct list_head ptrace_entry; + + /* PID/PID hash table linkage. */ + struct pid_link pids[PIDTYPE_MAX]; + struct list_head thread_group; + + struct completion *vfork_done; /* for vfork() */ + int __user *set_child_tid; /* CLONE_CHILD_SETTID */ + int __user *clear_child_tid; /* CLONE_CHILD_CLEARTID */ + + cputime_t utime, stime, utimescaled, stimescaled; + cputime_t gtime; +#ifndef CONFIG_VIRT_CPU_ACCOUNTING + cputime_t prev_utime, prev_stime; +#endif + unsigned long nvcsw, nivcsw; /* context switch counts */ + struct timespec start_time; /* monotonic time */ + struct timespec real_start_time; /* boot based time */ +/* mm fault and swap info: this can arguably be seen as either mm-specific or thread-specific */ + unsigned long min_flt, maj_flt; + + struct task_cputime cputime_expires; + struct list_head cpu_timers[3]; + +/* process credentials */ + const struct cred __rcu *real_cred; /* objective and real subjective task + * credentials (COW) */ + const struct cred __rcu *cred; /* effective (overridable) subjective task + * credentials (COW) */ + struct cred *replacement_session_keyring; /* for KEYCTL_SESSION_TO_PARENT */ + + char comm[TASK_COMM_LEN]; /* executable name excluding path + - access with [gs]et_task_comm (which lock + it with task_lock()) + - initialized normally by setup_new_exec */ +/* file system info */ + int link_count, total_link_count; +#ifdef CONFIG_SYSVIPC +/* ipc stuff */ + struct sysv_sem sysvsem; +#endif +#ifdef CONFIG_DETECT_HUNG_TASK +/* hung task detection */ + unsigned long last_switch_count; +#endif +/* CPU-specific state of this task */ + struct thread_struct thread; +/* filesystem information */ + struct fs_struct *fs; +/* open file information */ + struct files_struct *files; +/* namespaces */ + struct nsproxy *nsproxy; +/* signal handlers */ + struct signal_struct *signal; + struct sighand_struct *sighand; + + sigset_t blocked, real_blocked; + sigset_t saved_sigmask; /* restored if set_restore_sigmask() was used */ + struct sigpending pending; + + unsigned long sas_ss_sp; + size_t sas_ss_size; + int (*notifier)(void *priv); + void *notifier_data; + sigset_t *notifier_mask; + struct audit_context *audit_context; +#ifdef CONFIG_AUDITSYSCALL + uid_t loginuid; + unsigned int sessionid; +#endif + seccomp_t seccomp; + +/* Thread group tracking */ + u32 parent_exec_id; + u32 self_exec_id; +/* Protection of (de-)allocation: mm, files, fs, tty, keyrings, mems_allowed, + * mempolicy */ + spinlock_t alloc_lock; + +#ifdef CONFIG_GENERIC_HARDIRQS + /* IRQ handler threads */ + struct irqaction *irqaction; +#endif + + /* Protection of the PI data structures: */ + raw_spinlock_t pi_lock; + +#ifdef CONFIG_RT_MUTEXES + /* PI waiters blocked on a rt_mutex held by this task */ + struct plist_head pi_waiters; + /* Deadlock detection and priority inheritance handling */ + struct rt_mutex_waiter *pi_blocked_on; +#endif + +#ifdef CONFIG_DEBUG_MUTEXES + /* mutex deadlock detection */ + struct mutex_waiter *blocked_on; +#endif +#ifdef CONFIG_TRACE_IRQFLAGS + unsigned int irq_events; + unsigned long hardirq_enable_ip; + unsigned long hardirq_disable_ip; + unsigned int hardirq_enable_event; + unsigned int hardirq_disable_event; + int hardirqs_enabled; + int hardirq_context; + unsigned long softirq_disable_ip; + unsigned long softirq_enable_ip; + unsigned int softirq_disable_event; + unsigned int softirq_enable_event; + int softirqs_enabled; + int softirq_context; +#endif +#ifdef CONFIG_LOCKDEP +# define MAX_LOCK_DEPTH 48UL + u64 curr_chain_key; + int lockdep_depth; + unsigned int lockdep_recursion; + struct held_lock held_locks[MAX_LOCK_DEPTH]; + gfp_t lockdep_reclaim_gfp; +#endif + +/* journalling filesystem info */ + void *journal_info; + +/* stacked block device info */ + struct bio_list *bio_list; + +/* VM state */ + struct reclaim_state *reclaim_state; + + struct backing_dev_info *backing_dev_info; + + struct io_context *io_context; + + unsigned long ptrace_message; + siginfo_t *last_siginfo; /* For ptrace use. */ + struct task_io_accounting ioac; +#if defined(CONFIG_TASK_XACCT) + u64 acct_rss_mem1; /* accumulated rss usage */ + u64 acct_vm_mem1; /* accumulated virtual memory usage */ + cputime_t acct_timexpd; /* stime + utime since last update */ +#endif +#ifdef CONFIG_CPUSETS + nodemask_t mems_allowed; /* Protected by alloc_lock */ + int mems_allowed_change_disable; + int cpuset_mem_spread_rotor; + int cpuset_slab_spread_rotor; +#endif +#ifdef CONFIG_CGROUPS + /* Control Group info protected by css_set_lock */ + struct css_set __rcu *cgroups; + /* cg_list protected by css_set_lock and tsk->alloc_lock */ + struct list_head cg_list; +#endif +#ifdef CONFIG_FUTEX + struct robust_list_head __user *robust_list; +#ifdef CONFIG_COMPAT + struct compat_robust_list_head __user *compat_robust_list; +#endif + struct list_head pi_state_list; + struct futex_pi_state *pi_state_cache; +#endif +#ifdef CONFIG_PERF_EVENTS + struct perf_event_context *perf_event_ctxp[perf_nr_task_contexts]; + struct mutex perf_event_mutex; + struct list_head perf_event_list; +#endif +#ifdef CONFIG_NUMA + struct mempolicy *mempolicy; /* Protected by alloc_lock */ + short il_next; +#endif + atomic_t fs_excl; /* holding fs exclusive resources */ + struct rcu_head rcu; + + /* + * cache last used pipe for splice + */ + struct pipe_inode_info *splice_pipe; +#ifdef CONFIG_TASK_DELAY_ACCT + struct task_delay_info *delays; +#endif +#ifdef CONFIG_FAULT_INJECTION + int make_it_fail; +#endif + struct prop_local_single dirties; +#ifdef CONFIG_LATENCYTOP + int latency_record_count; + struct latency_record latency_record[LT_SAVECOUNT]; +#endif + /* + * time slack values; these are used to round up poll() and + * select() etc timeout values. These are in nanoseconds. + */ + unsigned long timer_slack_ns; + unsigned long default_timer_slack_ns; + + struct list_head *scm_work_list; +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + /* Index of current stored address in ret_stack */ + int curr_ret_stack; + /* Stack of return addresses for return function tracing */ + struct ftrace_ret_stack *ret_stack; + /* time stamp for last schedule */ + unsigned long long ftrace_timestamp; + /* + * Number of functions that haven't been traced + * because of depth overrun. + */ + atomic_t trace_overrun; + /* Pause for the tracing */ + atomic_t tracing_graph_pause; +#endif +#ifdef CONFIG_TRACING + /* state flags for use by tracers */ + unsigned long trace; + /* bitmask of trace recursion */ + unsigned long trace_recursion; +#endif /* CONFIG_TRACING */ +#ifdef CONFIG_CGROUP_MEM_RES_CTLR /* memcg uses this to do batch job */ + struct memcg_batch_info { + int do_batch; /* incremented when batch uncharge started */ + struct mem_cgroup *memcg; /* target memcg of uncharge */ + unsigned long bytes; /* uncharged usage */ + unsigned long memsw_bytes; /* uncharged mem+swap usage */ + } memcg_batch; +#endif +}; + +#endif diff --git a/include/linux/seccomp.h b/include/linux/seccomp.h index 167c333..160860f 100644 --- a/include/linux/seccomp.h +++ b/include/linux/seccomp.h @@ -1,14 +1,13 @@ #ifndef _LINUX_SECCOMP_H #define _LINUX_SECCOMP_H +#include <linux/seccomp_types.h> #ifdef CONFIG_SECCOMP #include <linux/thread_info.h> #include <asm/seccomp.h> -typedef struct { int mode; } seccomp_t; - extern void __secure_computing(int); static inline void secure_computing(int this_syscall) { @@ -23,8 +22,6 @@ extern long prctl_set_seccomp(unsigned long); #include <linux/errno.h> -typedef struct { } seccomp_t; - #define secure_computing(x) do { } while (0) static inline long prctl_get_seccomp(void) diff --git a/include/linux/seccomp_types.h b/include/linux/seccomp_types.h new file mode 100644 index 0000000..6ccb36f --- /dev/null +++ b/include/linux/seccomp_types.h @@ -0,0 +1,12 @@ +#ifndef _LINUX_SECCOMP_TYPES_H +#define _LINUX_SECCOMP_TYPES_H + + +#ifdef CONFIG_SECCOMP +typedef struct { int mode; } seccomp_t; +#else +typedef struct { } seccomp_t; +#endif + +#endif + diff --git a/include/linux/sem.h b/include/linux/sem.h index f2961af..e109ed2 100644 --- a/include/linux/sem.h +++ b/include/linux/sem.h @@ -2,6 +2,7 @@ #define _LINUX_SEM_H #include <linux/ipc.h> +#include <linux/sem_types.h> /* semop flags */ #define SEM_UNDO 0x1000 /* undo the operation on exit */ @@ -138,10 +139,6 @@ struct sem_undo_list { struct list_head list_proc; }; -struct sysv_sem { - struct sem_undo_list *undo_list; -}; - #ifdef CONFIG_SYSVIPC extern int copy_semundo(unsigned long clone_flags, struct task_struct *tsk); diff --git a/include/linux/sem_types.h b/include/linux/sem_types.h new file mode 100644 index 0000000..4b938c3 --- /dev/null +++ b/include/linux/sem_types.h @@ -0,0 +1,10 @@ +#ifndef _LINUX_SEM_TYPES_H +#define _LINUX_SEM_TYPES_H + +struct sem_undo_list; + +struct sysv_sem { + struct sem_undo_list *undo_list; +}; + +#endif diff --git a/include/linux/task_io_accounting.h b/include/linux/task_io_accounting.h index bdf855c..777cdef 100644 --- a/include/linux/task_io_accounting.h +++ b/include/linux/task_io_accounting.h @@ -8,6 +8,8 @@ * Blame Andrew Morton for all this. */ +#include <linux/types.h> + struct task_io_accounting { #ifdef CONFIG_TASK_XACCT /* bytes read */ ^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-30 10:55 ` Michal Marek @ 2011-03-30 10:57 ` Peter Zijlstra 2011-03-30 11:46 ` Michal Marek 0 siblings, 1 reply; 67+ messages in thread From: Peter Zijlstra @ 2011-03-30 10:57 UTC (permalink / raw) To: Michal Marek Cc: Lai Jiangshan, H. Peter Anvin, paulmck, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On Wed, 2011-03-30 at 12:55 +0200, Michal Marek wrote: > Subject: [PATCH] headers: Allow for lightweight inclusion of task_struct definition > > Factor out a couple of type definitions to <header>_types.h to allow > using task_struct without pulling tons of new dependencies via sched.h. Urgh, not pretty.. so why not clean up sched.h properly? There's way too much cruft in there. ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-30 10:57 ` Peter Zijlstra @ 2011-03-30 11:46 ` Michal Marek 2011-03-31 1:02 ` Lai Jiangshan 0 siblings, 1 reply; 67+ messages in thread From: Michal Marek @ 2011-03-30 11:46 UTC (permalink / raw) To: Peter Zijlstra Cc: Lai Jiangshan, H. Peter Anvin, paulmck, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On 30.3.2011 12:57, Peter Zijlstra wrote: > On Wed, 2011-03-30 at 12:55 +0200, Michal Marek wrote: >> Subject: [PATCH] headers: Allow for lightweight inclusion of task_struct definition >> >> Factor out a couple of type definitions to <header>_types.h to allow >> using task_struct without pulling tons of new dependencies via sched.h. > > Urgh, not pretty.. so why not clean up sched.h properly? There's way too > much cruft in there. It was a proof-of-concept to show that it is doable to have proper definition of task_struct in rcupdate.h. Not an entry for any code beauty contest. Michal ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-30 11:46 ` Michal Marek @ 2011-03-31 1:02 ` Lai Jiangshan 2011-03-31 1:21 ` Paul E. McKenney 2011-03-31 8:04 ` Peter Zijlstra 0 siblings, 2 replies; 67+ messages in thread From: Lai Jiangshan @ 2011-03-31 1:02 UTC (permalink / raw) To: Michal Marek Cc: Peter Zijlstra, H. Peter Anvin, paulmck, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On 03/30/2011 07:46 PM, Michal Marek wrote: > On 30.3.2011 12:57, Peter Zijlstra wrote: >> On Wed, 2011-03-30 at 12:55 +0200, Michal Marek wrote: >>> Subject: [PATCH] headers: Allow for lightweight inclusion of task_struct definition >>> >>> Factor out a couple of type definitions to <header>_types.h to allow >>> using task_struct without pulling tons of new dependencies via sched.h. >> >> Urgh, not pretty.. so why not clean up sched.h properly? There's way too >> much cruft in there. > > It was a proof-of-concept to show that it is doable to have proper > definition of task_struct in rcupdate.h. Not an entry for any code > beauty contest. > > Michal > I like this cleanup, could you continue for this hard job? I will help you if required. Ingo & Peter - will you accept the patches when it is done. Acked-by: Lai Jiangshan <laijs@cn.fujitsu.com> ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-31 1:02 ` Lai Jiangshan @ 2011-03-31 1:21 ` Paul E. McKenney 2011-03-31 1:53 ` Lai Jiangshan 2011-03-31 8:04 ` Peter Zijlstra 1 sibling, 1 reply; 67+ messages in thread From: Paul E. McKenney @ 2011-03-31 1:21 UTC (permalink / raw) To: Lai Jiangshan Cc: Michal Marek, Peter Zijlstra, H. Peter Anvin, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On Thu, Mar 31, 2011 at 09:02:20AM +0800, Lai Jiangshan wrote: > On 03/30/2011 07:46 PM, Michal Marek wrote: > > On 30.3.2011 12:57, Peter Zijlstra wrote: > >> On Wed, 2011-03-30 at 12:55 +0200, Michal Marek wrote: > >>> Subject: [PATCH] headers: Allow for lightweight inclusion of task_struct definition > >>> > >>> Factor out a couple of type definitions to <header>_types.h to allow > >>> using task_struct without pulling tons of new dependencies via sched.h. > >> > >> Urgh, not pretty.. so why not clean up sched.h properly? There's way too > >> much cruft in there. > > > > It was a proof-of-concept to show that it is doable to have proper > > definition of task_struct in rcupdate.h. Not an entry for any code > > beauty contest. > > > > Michal > > > > I like this cleanup, could you continue for this hard job? I will help you if required. > > Ingo & Peter - will you accept the patches when it is done. > > Acked-by: Lai Jiangshan <laijs@cn.fujitsu.com> I certainly like the idea of being able to inline TREE_PREEMPT_RCU's rcu_read_lock() and rcu_read_unlock() using normal C code! Thanx, Paul ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-31 1:21 ` Paul E. McKenney @ 2011-03-31 1:53 ` Lai Jiangshan 2011-03-31 23:30 ` Paul E. McKenney 0 siblings, 1 reply; 67+ messages in thread From: Lai Jiangshan @ 2011-03-31 1:53 UTC (permalink / raw) To: paulmck Cc: Michal Marek, Peter Zijlstra, H. Peter Anvin, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On 03/31/2011 09:21 AM, Paul E. McKenney wrote: > On Thu, Mar 31, 2011 at 09:02:20AM +0800, Lai Jiangshan wrote: >> On 03/30/2011 07:46 PM, Michal Marek wrote: >>> On 30.3.2011 12:57, Peter Zijlstra wrote: >>>> On Wed, 2011-03-30 at 12:55 +0200, Michal Marek wrote: >>>>> Subject: [PATCH] headers: Allow for lightweight inclusion of task_struct definition >>>>> >>>>> Factor out a couple of type definitions to <header>_types.h to allow >>>>> using task_struct without pulling tons of new dependencies via sched.h. >>>> >>>> Urgh, not pretty.. so why not clean up sched.h properly? There's way too >>>> much cruft in there. >>> >>> It was a proof-of-concept to show that it is doable to have proper >>> definition of task_struct in rcupdate.h. Not an entry for any code >>> beauty contest. >>> >>> Michal >>> >> >> I like this cleanup, could you continue for this hard job? I will help you if required. >> >> Ingo & Peter - will you accept the patches when it is done. >> >> Acked-by: Lai Jiangshan <laijs@cn.fujitsu.com> > > I certainly like the idea of being able to inline TREE_PREEMPT_RCU's > rcu_read_lock() and rcu_read_unlock() using normal C code! > > Thanx, Paul > kernel-offset.c can be merged to some branch of your tree for testing inlined TREE_PREEMPT_RCU's rcu_read_lock() and rcu_read_unlock() before the kernel headers splitting is done. Can the [patch 1/5] be merged to mainline earlier? could you add it to origin/rcu/next? Thanks, Lai ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-31 1:53 ` Lai Jiangshan @ 2011-03-31 23:30 ` Paul E. McKenney 2011-04-01 3:28 ` Paul E. McKenney 0 siblings, 1 reply; 67+ messages in thread From: Paul E. McKenney @ 2011-03-31 23:30 UTC (permalink / raw) To: Lai Jiangshan Cc: Michal Marek, Peter Zijlstra, H. Peter Anvin, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On Thu, Mar 31, 2011 at 09:53:21AM +0800, Lai Jiangshan wrote: > On 03/31/2011 09:21 AM, Paul E. McKenney wrote: > > On Thu, Mar 31, 2011 at 09:02:20AM +0800, Lai Jiangshan wrote: > >> On 03/30/2011 07:46 PM, Michal Marek wrote: > >>> On 30.3.2011 12:57, Peter Zijlstra wrote: > >>>> On Wed, 2011-03-30 at 12:55 +0200, Michal Marek wrote: > >>>>> Subject: [PATCH] headers: Allow for lightweight inclusion of task_struct definition > >>>>> > >>>>> Factor out a couple of type definitions to <header>_types.h to allow > >>>>> using task_struct without pulling tons of new dependencies via sched.h. > >>>> > >>>> Urgh, not pretty.. so why not clean up sched.h properly? There's way too > >>>> much cruft in there. > >>> > >>> It was a proof-of-concept to show that it is doable to have proper > >>> definition of task_struct in rcupdate.h. Not an entry for any code > >>> beauty contest. > >>> > >>> Michal > >>> > >> > >> I like this cleanup, could you continue for this hard job? I will help you if required. > >> > >> Ingo & Peter - will you accept the patches when it is done. > >> > >> Acked-by: Lai Jiangshan <laijs@cn.fujitsu.com> > > > > I certainly like the idea of being able to inline TREE_PREEMPT_RCU's > > rcu_read_lock() and rcu_read_unlock() using normal C code! > > > > Thanx, Paul > > > > kernel-offset.c can be merged to some branch of your tree for > testing inlined TREE_PREEMPT_RCU's rcu_read_lock() and rcu_read_unlock() > before the kernel headers splitting is done. > > Can the [patch 1/5] be merged to mainline earlier? could you add it to origin/rcu/next? Let's first settle on the overall direction, on the off-chance that someone comes up with an approach that doesn't require task_rcu_struct, or that requires it to be located somewhere other than include/linux/rcupdate.h. Thanx, Paul ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-31 23:30 ` Paul E. McKenney @ 2011-04-01 3:28 ` Paul E. McKenney 0 siblings, 0 replies; 67+ messages in thread From: Paul E. McKenney @ 2011-04-01 3:28 UTC (permalink / raw) To: Lai Jiangshan Cc: Michal Marek, Peter Zijlstra, H. Peter Anvin, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On Thu, Mar 31, 2011 at 04:30:04PM -0700, Paul E. McKenney wrote: > On Thu, Mar 31, 2011 at 09:53:21AM +0800, Lai Jiangshan wrote: > > On 03/31/2011 09:21 AM, Paul E. McKenney wrote: > > > On Thu, Mar 31, 2011 at 09:02:20AM +0800, Lai Jiangshan wrote: > > >> On 03/30/2011 07:46 PM, Michal Marek wrote: > > >>> On 30.3.2011 12:57, Peter Zijlstra wrote: > > >>>> On Wed, 2011-03-30 at 12:55 +0200, Michal Marek wrote: > > >>>>> Subject: [PATCH] headers: Allow for lightweight inclusion of task_struct definition > > >>>>> > > >>>>> Factor out a couple of type definitions to <header>_types.h to allow > > >>>>> using task_struct without pulling tons of new dependencies via sched.h. > > >>>> > > >>>> Urgh, not pretty.. so why not clean up sched.h properly? There's way too > > >>>> much cruft in there. > > >>> > > >>> It was a proof-of-concept to show that it is doable to have proper > > >>> definition of task_struct in rcupdate.h. Not an entry for any code > > >>> beauty contest. > > >>> > > >>> Michal > > >>> > > >> > > >> I like this cleanup, could you continue for this hard job? I will help you if required. > > >> > > >> Ingo & Peter - will you accept the patches when it is done. > > >> > > >> Acked-by: Lai Jiangshan <laijs@cn.fujitsu.com> > > > > > > I certainly like the idea of being able to inline TREE_PREEMPT_RCU's > > > rcu_read_lock() and rcu_read_unlock() using normal C code! > > > > > > Thanx, Paul > > > > > > > kernel-offset.c can be merged to some branch of your tree for > > testing inlined TREE_PREEMPT_RCU's rcu_read_lock() and rcu_read_unlock() > > before the kernel headers splitting is done. > > > > Can the [patch 1/5] be merged to mainline earlier? could you add it to origin/rcu/next? > > Let's first settle on the overall direction, on the off-chance that someone > comes up with an approach that doesn't require task_rcu_struct, or that > requires it to be located somewhere other than include/linux/rcupdate.h. And I have created a branch rcu/dev that contains the commits that I don't yet trust enough to send to -next. As before, rcu/kfree_rcu contains the commits that apply kfree_rcu() elsewhere in the kernel. Thanx, Paul ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-31 1:02 ` Lai Jiangshan 2011-03-31 1:21 ` Paul E. McKenney @ 2011-03-31 8:04 ` Peter Zijlstra 2011-03-31 9:50 ` Lai Jiangshan 1 sibling, 1 reply; 67+ messages in thread From: Peter Zijlstra @ 2011-03-31 8:04 UTC (permalink / raw) To: Lai Jiangshan Cc: Michal Marek, H. Peter Anvin, paulmck, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On Thu, 2011-03-31 at 09:02 +0800, Lai Jiangshan wrote: > I like this cleanup, could you continue for this hard job? I will help > you if required. > > Ingo & Peter - will you accept the patches when it is done. > No, like I said, I think the proposed patch is utterly horrid. ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-31 8:04 ` Peter Zijlstra @ 2011-03-31 9:50 ` Lai Jiangshan 2011-03-31 11:18 ` Peter Zijlstra 2011-03-31 19:28 ` H. Peter Anvin 0 siblings, 2 replies; 67+ messages in thread From: Lai Jiangshan @ 2011-03-31 9:50 UTC (permalink / raw) To: Peter Zijlstra Cc: Michal Marek, H. Peter Anvin, paulmck, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On 03/31/2011 04:04 PM, Peter Zijlstra wrote: > On Thu, 2011-03-31 at 09:02 +0800, Lai Jiangshan wrote: >> I like this cleanup, could you continue for this hard job? I will help >> you if required. >> >> Ingo & Peter - will you accept the patches when it is done. >> > No, like I said, I think the proposed patch is utterly horrid. > But how about my kernel-offset.c patch? It is clean & simple, it just seems not so normal. If the proposed splitting patch is horrid, I think we will try to update it as you expect. If splitting sched.h is wrong, I will try to persuade more people accept the kernel-offset.c patch. ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-31 9:50 ` Lai Jiangshan @ 2011-03-31 11:18 ` Peter Zijlstra 2011-04-01 1:57 ` Lai Jiangshan 2011-03-31 19:28 ` H. Peter Anvin 1 sibling, 1 reply; 67+ messages in thread From: Peter Zijlstra @ 2011-03-31 11:18 UTC (permalink / raw) To: Lai Jiangshan Cc: Michal Marek, H. Peter Anvin, paulmck, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On Thu, 2011-03-31 at 17:50 +0800, Lai Jiangshan wrote: > On 03/31/2011 04:04 PM, Peter Zijlstra wrote: > > On Thu, 2011-03-31 at 09:02 +0800, Lai Jiangshan wrote: > >> I like this cleanup, could you continue for this hard job? I will help > >> you if required. > >> > >> Ingo & Peter - will you accept the patches when it is done. > >> > > No, like I said, I think the proposed patch is utterly horrid. > > > > But how about my kernel-offset.c patch? It is clean & simple, > it just seems not so normal. > > If the proposed splitting patch is horrid, I think we will try to > update it as you expect. > > If splitting sched.h is wrong, I will try to persuade more people > accept the kernel-offset.c patch. Well, I'm all for cleaning up sched.h, it includes way too much things not strongly related to kernel/sched*.c like a lot of the signal things and the misnamed signal_struct (should be called process_struct or somesuch). That also causes the inversion between sched.h and wait.h What I don't like is those _types.h headers, and definitely not the massive explosion of those as per the proposed patch. Nor do I quite get why all that is needed, sched_types which would define task_struct still needs the new task_rcu_struct bits, and as per the patch you need to split the rcu headers into two. Once you've done that, I don't see why sched.h still needs splitting too. ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-31 11:18 ` Peter Zijlstra @ 2011-04-01 1:57 ` Lai Jiangshan 2011-04-01 11:35 ` Peter Zijlstra 0 siblings, 1 reply; 67+ messages in thread From: Lai Jiangshan @ 2011-04-01 1:57 UTC (permalink / raw) To: Peter Zijlstra Cc: Michal Marek, H. Peter Anvin, paulmck, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On 03/31/2011 07:18 PM, Peter Zijlstra wrote: > On Thu, 2011-03-31 at 17:50 +0800, Lai Jiangshan wrote: >> On 03/31/2011 04:04 PM, Peter Zijlstra wrote: >>> On Thu, 2011-03-31 at 09:02 +0800, Lai Jiangshan wrote: >>>> I like this cleanup, could you continue for this hard job? I will help >>>> you if required. >>>> >>>> Ingo & Peter - will you accept the patches when it is done. >>>> >>> No, like I said, I think the proposed patch is utterly horrid. >>> >> >> But how about my kernel-offset.c patch? It is clean & simple, >> it just seems not so normal. >> >> If the proposed splitting patch is horrid, I think we will try to >> update it as you expect. >> >> If splitting sched.h is wrong, I will try to persuade more people >> accept the kernel-offset.c patch. > > Well, I'm all for cleaning up sched.h, it includes way too much things > not strongly related to kernel/sched*.c like a lot of the signal things > and the misnamed signal_struct (should be called process_struct or > somesuch). > > That also causes the inversion between sched.h and wait.h > > What I don't like is those _types.h headers, and definitely not the > massive explosion of those as per the proposed patch. > > Nor do I quite get why all that is needed, sched_types which would > define task_struct still needs the new task_rcu_struct bits, and as per > the patch you need to split the rcu headers into two. Once you've done > that, I don't see why sched.h still needs splitting too. > > "struct task_rcu_struct" is just a cleanup patch. We are trying to use inlined read_read_[un]lock(), it is required that sched.h needs include rcupdate.h which causes many recursive including. Splitting rcupdate.h only does not help. ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-01 1:57 ` Lai Jiangshan @ 2011-04-01 11:35 ` Peter Zijlstra 2011-04-05 21:54 ` Paul E. McKenney 0 siblings, 1 reply; 67+ messages in thread From: Peter Zijlstra @ 2011-04-01 11:35 UTC (permalink / raw) To: Lai Jiangshan Cc: Michal Marek, H. Peter Anvin, paulmck, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On Fri, 2011-04-01 at 09:57 +0800, Lai Jiangshan wrote: > > We are trying to use inlined read_read_[un]lock(), it is required that sched.h > needs include rcupdate.h which causes many recursive including. Splitting rcupdate.h > only does not help. You're not making any kind of sense there. linux-2.6# cat kernel/sched_tmp.c #include <linux/sched.h> linux-2.6# make kernel/sched_tmp.i linux-2.6# awk '/^#/ { t=$0 } /rcu_read_lock/ { print t " " $0 }' kernel/sched_tmp.i # 60 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline void __rcu_read_lock_bh(void) # 60 "/usr/src/linux-2.6/include/linux/rcupdate.h" extern void __rcu_read_lock(void); # 258 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline int rcu_read_lock_held(void) # 258 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline int rcu_read_lock_bh_held(void) # 258 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline int rcu_read_lock_sched_held(void) # 559 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline void rcu_read_lock(void) # 559 "/usr/src/linux-2.6/include/linux/rcupdate.h" __rcu_read_lock(); # 600 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline void rcu_read_lock_bh(void) # 600 "/usr/src/linux-2.6/include/linux/rcupdate.h" __rcu_read_lock_bh(); # 627 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline void rcu_read_lock_sched(void) # 627 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline __attribute__((no_instrument_function)) void rcu_read_lock_sched_notrace(void) # 70 "/usr/src/linux-2.6/include/linux/srcu.h" int __srcu_read_lock(struct srcu_struct *sp) ; # 103 "/usr/src/linux-2.6/include/linux/srcu.h" static inline int srcu_read_lock_held(struct srcu_struct *sp) # 149 "/usr/src/linux-2.6/include/linux/srcu.h" static inline int srcu_read_lock(struct srcu_struct *sp) # 149 "/usr/src/linux-2.6/include/linux/srcu.h" int retval = __srcu_read_lock(sp); # 1227 "/usr/src/linux-2.6/include/linux/sched.h" int rcu_read_lock_nesting; # 1786 "/usr/src/linux-2.6/include/linux/sched.h" p->rcu_read_lock_nesting = 0; There isn't a single rcu_read_lock() user in the include chain of sched.h, therefore its possible to split rcupdate.h such that one part includes the bits needed for sched.h and another part that includes sched.h and provides rcu_read_lock(). ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-01 11:35 ` Peter Zijlstra @ 2011-04-05 21:54 ` Paul E. McKenney 2011-04-05 23:07 ` Paul E. McKenney 0 siblings, 1 reply; 67+ messages in thread From: Paul E. McKenney @ 2011-04-05 21:54 UTC (permalink / raw) To: Peter Zijlstra Cc: Lai Jiangshan, Michal Marek, H. Peter Anvin, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On Fri, Apr 01, 2011 at 01:35:49PM +0200, Peter Zijlstra wrote: > On Fri, 2011-04-01 at 09:57 +0800, Lai Jiangshan wrote: > > > > We are trying to use inlined read_read_[un]lock(), it is required that sched.h > > needs include rcupdate.h which causes many recursive including. Splitting rcupdate.h > > only does not help. > > You're not making any kind of sense there. > > linux-2.6# cat kernel/sched_tmp.c > > #include <linux/sched.h> > > linux-2.6# make kernel/sched_tmp.i > linux-2.6# awk '/^#/ { t=$0 } /rcu_read_lock/ { print t " " $0 }' kernel/sched_tmp.i > # 60 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline void __rcu_read_lock_bh(void) > # 60 "/usr/src/linux-2.6/include/linux/rcupdate.h" extern void __rcu_read_lock(void); > # 258 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline int rcu_read_lock_held(void) > # 258 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline int rcu_read_lock_bh_held(void) > # 258 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline int rcu_read_lock_sched_held(void) > # 559 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline void rcu_read_lock(void) > # 559 "/usr/src/linux-2.6/include/linux/rcupdate.h" __rcu_read_lock(); > # 600 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline void rcu_read_lock_bh(void) > # 600 "/usr/src/linux-2.6/include/linux/rcupdate.h" __rcu_read_lock_bh(); > # 627 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline void rcu_read_lock_sched(void) > # 627 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline __attribute__((no_instrument_function)) void rcu_read_lock_sched_notrace(void) > # 70 "/usr/src/linux-2.6/include/linux/srcu.h" int __srcu_read_lock(struct srcu_struct *sp) ; > # 103 "/usr/src/linux-2.6/include/linux/srcu.h" static inline int srcu_read_lock_held(struct srcu_struct *sp) > # 149 "/usr/src/linux-2.6/include/linux/srcu.h" static inline int srcu_read_lock(struct srcu_struct *sp) > # 149 "/usr/src/linux-2.6/include/linux/srcu.h" int retval = __srcu_read_lock(sp); > # 1227 "/usr/src/linux-2.6/include/linux/sched.h" int rcu_read_lock_nesting; > # 1786 "/usr/src/linux-2.6/include/linux/sched.h" p->rcu_read_lock_nesting = 0; > > There isn't a single rcu_read_lock() user in the include chain of > sched.h, therefore its possible to split rcupdate.h such that one part > includes the bits needed for sched.h and another part that includes > sched.h and provides rcu_read_lock(). Yep, you can easily split struct rcu_head out into its own header file, and then include that into sched.h in place of rcupdate.h without trouble (at least in the few builds I tried). The trouble comes when you take the next step and include sched.h into include/linux/rcutree.h, which would be needed to allow TREE_PREEMPT_RCU's rcu_read_lock() and rcu_read_unlock() to be inlined. This pulls sched.h into files that it was not already included into, and we do get build failures (the first of which requires changes local to RCU, FWIW). Of course, it would be good to straighten this out. Thanx, Paul ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-05 21:54 ` Paul E. McKenney @ 2011-04-05 23:07 ` Paul E. McKenney 2011-04-06 8:10 ` Peter Zijlstra 2011-04-06 17:41 ` [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET Paul E. McKenney 0 siblings, 2 replies; 67+ messages in thread From: Paul E. McKenney @ 2011-04-05 23:07 UTC (permalink / raw) To: Peter Zijlstra Cc: Lai Jiangshan, Michal Marek, H. Peter Anvin, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On Tue, Apr 05, 2011 at 02:54:50PM -0700, Paul E. McKenney wrote: > On Fri, Apr 01, 2011 at 01:35:49PM +0200, Peter Zijlstra wrote: > > On Fri, 2011-04-01 at 09:57 +0800, Lai Jiangshan wrote: > > > > > > We are trying to use inlined read_read_[un]lock(), it is required that sched.h > > > needs include rcupdate.h which causes many recursive including. Splitting rcupdate.h > > > only does not help. > > > > You're not making any kind of sense there. > > > > linux-2.6# cat kernel/sched_tmp.c > > > > #include <linux/sched.h> > > > > linux-2.6# make kernel/sched_tmp.i > > linux-2.6# awk '/^#/ { t=$0 } /rcu_read_lock/ { print t " " $0 }' kernel/sched_tmp.i > > # 60 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline void __rcu_read_lock_bh(void) > > # 60 "/usr/src/linux-2.6/include/linux/rcupdate.h" extern void __rcu_read_lock(void); > > # 258 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline int rcu_read_lock_held(void) > > # 258 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline int rcu_read_lock_bh_held(void) > > # 258 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline int rcu_read_lock_sched_held(void) > > # 559 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline void rcu_read_lock(void) > > # 559 "/usr/src/linux-2.6/include/linux/rcupdate.h" __rcu_read_lock(); > > # 600 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline void rcu_read_lock_bh(void) > > # 600 "/usr/src/linux-2.6/include/linux/rcupdate.h" __rcu_read_lock_bh(); > > # 627 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline void rcu_read_lock_sched(void) > > # 627 "/usr/src/linux-2.6/include/linux/rcupdate.h" static inline __attribute__((no_instrument_function)) void rcu_read_lock_sched_notrace(void) > > # 70 "/usr/src/linux-2.6/include/linux/srcu.h" int __srcu_read_lock(struct srcu_struct *sp) ; > > # 103 "/usr/src/linux-2.6/include/linux/srcu.h" static inline int srcu_read_lock_held(struct srcu_struct *sp) > > # 149 "/usr/src/linux-2.6/include/linux/srcu.h" static inline int srcu_read_lock(struct srcu_struct *sp) > > # 149 "/usr/src/linux-2.6/include/linux/srcu.h" int retval = __srcu_read_lock(sp); > > # 1227 "/usr/src/linux-2.6/include/linux/sched.h" int rcu_read_lock_nesting; > > # 1786 "/usr/src/linux-2.6/include/linux/sched.h" p->rcu_read_lock_nesting = 0; > > > > There isn't a single rcu_read_lock() user in the include chain of > > sched.h, therefore its possible to split rcupdate.h such that one part > > includes the bits needed for sched.h and another part that includes > > sched.h and provides rcu_read_lock(). > > Yep, you can easily split struct rcu_head out into its own header file, > and then include that into sched.h in place of rcupdate.h without trouble > (at least in the few builds I tried). > > The trouble comes when you take the next step and include sched.h > into include/linux/rcutree.h, which would be needed to allow > TREE_PREEMPT_RCU's rcu_read_lock() and rcu_read_unlock() to be > inlined. This pulls sched.h into files that it was not already > included into, and we do get build failures (the first of which > requires changes local to RCU, FWIW). > > Of course, it would be good to straighten this out. And here is a start -- the build is -much- quieter with this patch. The next layer on this onion is the use of the ctl_table typedef in include/linux/key.h -- which is defined to be struct ctl_table. Anyone know where struct ctl_table is defined? Color me blind, but I am not seeing it... Thanx, Paul ------------------------------------------------------------------------ diff --git a/include/linux/rcu_head.h b/include/linux/rcu_head.h new file mode 100644 index 0000000..e0b8add --- /dev/null +++ b/include/linux/rcu_head.h @@ -0,0 +1,46 @@ +/* + * Read-Copy Update mechanism for mutual exclusion: base structures + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Copyright IBM Corporation, 2001 + * + * Author: Dipankar Sarma <dipankar@in.ibm.com> + * + * Based on the original work by Paul McKenney <paulmck@us.ibm.com> + * and inputs from Rusty Russell, Andrea Arcangeli and Andi Kleen. + * Papers: + * http://www.rdrop.com/users/paulmck/paper/rclockpdcsproof.pdf + * http://lse.sourceforge.net/locking/rclock_OLS.2001.05.01c.sc.pdf (OLS2001) + * + * For detailed explanation of Read-Copy Update mechanism see - + * http://lse.sourceforge.net/locking/rcupdate.html + * + */ + +#ifndef __LINUX_RCU_HEAD_H +#define __LINUX_RCU_HEAD_H + +/** + * struct rcu_head - callback structure for use with RCU + * @next: next update requests in a list + * @func: actual update function to call after the grace period. + */ +struct rcu_head { + struct rcu_head *next; + void (*func)(struct rcu_head *head); +}; + +#endif /* __LINUX_RCU_HEAD_H */ diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index ff422d2..ab94fcfb 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -52,15 +52,7 @@ extern int rcutorture_runnable; /* for sysctl */ #define ULONG_CMP_GE(a, b) (ULONG_MAX / 2 >= (a) - (b)) #define ULONG_CMP_LT(a, b) (ULONG_MAX / 2 < (a) - (b)) -/** - * struct rcu_head - callback structure for use with RCU - * @next: next update requests in a list - * @func: actual update function to call after the grace period. - */ -struct rcu_head { - struct rcu_head *next; - void (*func)(struct rcu_head *head); -}; +#include <linux/rcu_head.h> /* Exported common interfaces */ extern void call_rcu_sched(struct rcu_head *head, @@ -141,14 +133,6 @@ static inline void rcu_exit_nohz(void) #endif /* #else #ifdef CONFIG_NO_HZ */ -#if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) -#include <linux/rcutree.h> -#elif defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU) -#include <linux/rcutiny.h> -#else -#error "Unknown RCU implementation specified to kernel configuration" -#endif - /* * init_rcu_head_on_stack()/destroy_rcu_head_on_stack() are needed for dynamic * initialization and destruction of rcu_head on the stack. rcu_head structures @@ -797,4 +781,12 @@ static inline void debug_rcu_head_unqueue(struct rcu_head *head) } #endif /* #else !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ +#if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) +#include <linux/rcutree.h> +#elif defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU) +#include <linux/rcutiny.h> +#else +#error "Unknown RCU implementation specified to kernel configuration" +#endif + #endif /* __LINUX_RCUPDATE_H */ diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h index 3a93348..c317eec 100644 --- a/include/linux/rcutree.h +++ b/include/linux/rcutree.h @@ -30,6 +30,8 @@ #ifndef __LINUX_RCUTREE_H #define __LINUX_RCUTREE_H +#include <linux/sched.h> + extern void rcu_init(void); extern void rcu_note_context_switch(int cpu); extern int rcu_needs_cpu(int cpu); diff --git a/include/linux/sched.h b/include/linux/sched.h index 83bd2e2..e7c6a90 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -78,7 +78,7 @@ struct sched_param { #include <linux/topology.h> #include <linux/proportions.h> #include <linux/seccomp.h> -#include <linux/rcupdate.h> +#include <linux/rcu_head.h> #include <linux/rculist.h> #include <linux/rtmutex.h> @@ -2241,11 +2241,9 @@ int same_thread_group(struct task_struct *p1, struct task_struct *p2) return p1->tgid == p2->tgid; } -static inline struct task_struct *next_thread(const struct task_struct *p) -{ - return list_entry_rcu(p->thread_group.next, - struct task_struct, thread_group); -} +/* Avoid #include hell for inlining rcu_read_lock(). */ +#define next_thread(p) \ + list_entry_rcu((p)->thread_group.next, struct task_struct, thread_group) static inline int thread_group_empty(struct task_struct *p) { ^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-05 23:07 ` Paul E. McKenney @ 2011-04-06 8:10 ` Peter Zijlstra 2011-04-06 19:21 ` Paul E. McKenney 2011-04-06 17:41 ` [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET Paul E. McKenney 1 sibling, 1 reply; 67+ messages in thread From: Peter Zijlstra @ 2011-04-06 8:10 UTC (permalink / raw) To: paulmck Cc: Lai Jiangshan, Michal Marek, H. Peter Anvin, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On Tue, 2011-04-05 at 16:07 -0700, Paul E. McKenney wrote: > > The next layer on this onion is the use of the ctl_table typedef > in include/linux/key.h -- which is defined to be struct ctl_table. > Anyone know where struct ctl_table is defined? Color me blind, but I > am not seeing it... include/linux/sysctl.h:1015 /* A sysctl table is an array of struct ctl_table: */ struct ctl_table { const char *procname; /* Text ID for /proc/sys, or zero */ void *data; int maxlen; mode_t mode; struct ctl_table *child; struct ctl_table *parent; /* Automatically set */ proc_handler *proc_handler; /* Callback for text formatting */ void *extra1; void *extra2; }; ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-06 8:10 ` Peter Zijlstra @ 2011-04-06 19:21 ` Paul E. McKenney 2011-04-06 20:13 ` Paul E. McKenney 0 siblings, 1 reply; 67+ messages in thread From: Paul E. McKenney @ 2011-04-06 19:21 UTC (permalink / raw) To: Peter Zijlstra Cc: Lai Jiangshan, Michal Marek, H. Peter Anvin, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On Wed, Apr 06, 2011 at 10:10:28AM +0200, Peter Zijlstra wrote: > On Tue, 2011-04-05 at 16:07 -0700, Paul E. McKenney wrote: > > > > The next layer on this onion is the use of the ctl_table typedef > > in include/linux/key.h -- which is defined to be struct ctl_table. > > Anyone know where struct ctl_table is defined? Color me blind, but I > > am not seeing it... > > include/linux/sysctl.h:1015 Thank you! Thanx, Paul > /* A sysctl table is an array of struct ctl_table: */ > struct ctl_table > { > const char *procname; /* Text ID for /proc/sys, or zero */ > void *data; > int maxlen; > mode_t mode; > struct ctl_table *child; > struct ctl_table *parent; /* Automatically set */ > proc_handler *proc_handler; /* Callback for text formatting */ > void *extra1; > void *extra2; > }; > ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-06 19:21 ` Paul E. McKenney @ 2011-04-06 20:13 ` Paul E. McKenney 2011-04-06 21:06 ` Peter Zijlstra 0 siblings, 1 reply; 67+ messages in thread From: Paul E. McKenney @ 2011-04-06 20:13 UTC (permalink / raw) To: Peter Zijlstra Cc: Lai Jiangshan, Michal Marek, H. Peter Anvin, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On Wed, Apr 06, 2011 at 12:21:19PM -0700, Paul E. McKenney wrote: > On Wed, Apr 06, 2011 at 10:10:28AM +0200, Peter Zijlstra wrote: > > On Tue, 2011-04-05 at 16:07 -0700, Paul E. McKenney wrote: > > > > > > The next layer on this onion is the use of the ctl_table typedef > > > in include/linux/key.h -- which is defined to be struct ctl_table. > > > Anyone know where struct ctl_table is defined? Color me blind, but I > > > am not seeing it... > > > > include/linux/sysctl.h:1015 > > Thank you! And the following patch builds correctly for defconfig x86 builds, while allowing rcupdate.h to see the sched.h definitions as needed to inline rcu_read_lock() and rcu_read_unlock(). I will try allyesconfig builds and see what else breaks. Thanx, Paul include/linux/pid.h | 2 +- include/linux/rcupdate.h | 27 +++++++++------------------ include/linux/rcutree.h | 2 ++ include/linux/sched.h | 10 ++++------ include/linux/sem.h | 2 +- include/linux/sysctl.h | 5 +++-- include/linux/types.h | 10 ++++++++++ kernel/pid_namespace.c | 2 ++ 8 files changed, 32 insertions(+), 28 deletions(-) diff --git a/include/linux/pid.h b/include/linux/pid.h index efceda0..3c5719b 100644 --- a/include/linux/pid.h +++ b/include/linux/pid.h @@ -1,7 +1,7 @@ #ifndef _LINUX_PID_H #define _LINUX_PID_H -#include <linux/rcupdate.h> +#include <linux/types.h> enum pid_type { diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index ff422d2..2fe6402 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -33,6 +33,7 @@ #ifndef __LINUX_RCUPDATE_H #define __LINUX_RCUPDATE_H +#include <linux/types.h> #include <linux/cache.h> #include <linux/spinlock.h> #include <linux/threads.h> @@ -52,16 +53,6 @@ extern int rcutorture_runnable; /* for sysctl */ #define ULONG_CMP_GE(a, b) (ULONG_MAX / 2 >= (a) - (b)) #define ULONG_CMP_LT(a, b) (ULONG_MAX / 2 < (a) - (b)) -/** - * struct rcu_head - callback structure for use with RCU - * @next: next update requests in a list - * @func: actual update function to call after the grace period. - */ -struct rcu_head { - struct rcu_head *next; - void (*func)(struct rcu_head *head); -}; - /* Exported common interfaces */ extern void call_rcu_sched(struct rcu_head *head, void (*func)(struct rcu_head *rcu)); @@ -141,14 +132,6 @@ static inline void rcu_exit_nohz(void) #endif /* #else #ifdef CONFIG_NO_HZ */ -#if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) -#include <linux/rcutree.h> -#elif defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU) -#include <linux/rcutiny.h> -#else -#error "Unknown RCU implementation specified to kernel configuration" -#endif - /* * init_rcu_head_on_stack()/destroy_rcu_head_on_stack() are needed for dynamic * initialization and destruction of rcu_head on the stack. rcu_head structures @@ -797,4 +780,12 @@ static inline void debug_rcu_head_unqueue(struct rcu_head *head) } #endif /* #else !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ +#if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) +#include <linux/rcutree.h> +#elif defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU) +#include <linux/rcutiny.h> +#else +#error "Unknown RCU implementation specified to kernel configuration" +#endif + #endif /* __LINUX_RCUPDATE_H */ diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h index 3a93348..c317eec 100644 --- a/include/linux/rcutree.h +++ b/include/linux/rcutree.h @@ -30,6 +30,8 @@ #ifndef __LINUX_RCUTREE_H #define __LINUX_RCUTREE_H +#include <linux/sched.h> + extern void rcu_init(void); extern void rcu_note_context_switch(int cpu); extern int rcu_needs_cpu(int cpu); diff --git a/include/linux/sched.h b/include/linux/sched.h index 83bd2e2..30a4444 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -78,7 +78,7 @@ struct sched_param { #include <linux/topology.h> #include <linux/proportions.h> #include <linux/seccomp.h> -#include <linux/rcupdate.h> +#include <linux/types.h> #include <linux/rculist.h> #include <linux/rtmutex.h> @@ -2241,11 +2241,9 @@ int same_thread_group(struct task_struct *p1, struct task_struct *p2) return p1->tgid == p2->tgid; } -static inline struct task_struct *next_thread(const struct task_struct *p) -{ - return list_entry_rcu(p->thread_group.next, - struct task_struct, thread_group); -} +/* Avoid #include hell for inlining rcu_read_lock(). */ +#define next_thread(p) \ + list_entry_rcu((p)->thread_group.next, struct task_struct, thread_group) static inline int thread_group_empty(struct task_struct *p) { diff --git a/include/linux/sem.h b/include/linux/sem.h index f2961af..8489a1f 100644 --- a/include/linux/sem.h +++ b/include/linux/sem.h @@ -78,7 +78,7 @@ struct seminfo { #ifdef __KERNEL__ #include <asm/atomic.h> -#include <linux/rcupdate.h> +#include <linux/types.h> #include <linux/cache.h> struct task_struct; diff --git a/include/linux/sysctl.h b/include/linux/sysctl.h index 11684d9..be80a59 100644 --- a/include/linux/sysctl.h +++ b/include/linux/sysctl.h @@ -19,6 +19,8 @@ **************************************************************** */ +#include <linux/key.h> + #ifndef _LINUX_SYSCTL_H #define _LINUX_SYSCTL_H @@ -1012,8 +1014,7 @@ extern int proc_do_large_bitmap(struct ctl_table *, int, */ /* A sysctl table is an array of struct ctl_table: */ -struct ctl_table -{ +struct ctl_table { const char *procname; /* Text ID for /proc/sys, or zero */ void *data; int maxlen; diff --git a/include/linux/types.h b/include/linux/types.h index 176da8c..868ef8b 100644 --- a/include/linux/types.h +++ b/include/linux/types.h @@ -231,6 +231,16 @@ struct hlist_node { struct hlist_node *next, **pprev; }; +/** + * struct rcu_head - callback structure for use with RCU + * @next: next update requests in a list + * @func: actual update function to call after the grace period. + */ +struct rcu_head { + struct rcu_head *next; + void (*func)(struct rcu_head *head); +}; + struct ustat { __kernel_daddr_t f_tfree; __kernel_ino_t f_tinode; diff --git a/kernel/pid_namespace.c b/kernel/pid_namespace.c index e9c9adc..bfa75df 100644 --- a/kernel/pid_namespace.c +++ b/kernel/pid_namespace.c @@ -8,6 +8,8 @@ * */ +#include <linux/types.h> +#include <asm/atomic.h> #include <linux/pid.h> #include <linux/pid_namespace.h> #include <linux/syscalls.h> ^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-06 20:13 ` Paul E. McKenney @ 2011-04-06 21:06 ` Peter Zijlstra 2011-04-06 21:27 ` H. Peter Anvin 0 siblings, 1 reply; 67+ messages in thread From: Peter Zijlstra @ 2011-04-06 21:06 UTC (permalink / raw) To: paulmck Cc: Lai Jiangshan, Michal Marek, H. Peter Anvin, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On Wed, 2011-04-06 at 13:13 -0700, Paul E. McKenney wrote: > And the following patch builds correctly for defconfig x86 builds, > while allowing rcupdate.h to see the sched.h definitions as needed > to inline rcu_read_lock() and rcu_read_unlock(). > Looks like an entirely reasonable patch to me ;-) ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-06 21:06 ` Peter Zijlstra @ 2011-04-06 21:27 ` H. Peter Anvin 2011-04-07 0:30 ` Paul E. McKenney 0 siblings, 1 reply; 67+ messages in thread From: H. Peter Anvin @ 2011-04-06 21:27 UTC (permalink / raw) To: Peter Zijlstra Cc: paulmck, Lai Jiangshan, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On 04/06/2011 02:06 PM, Peter Zijlstra wrote: > On Wed, 2011-04-06 at 13:13 -0700, Paul E. McKenney wrote: >> And the following patch builds correctly for defconfig x86 builds, >> while allowing rcupdate.h to see the sched.h definitions as needed >> to inline rcu_read_lock() and rcu_read_unlock(). >> > Looks like an entirely reasonable patch to me ;-) > Quite... a lot better than the original proposal! -hpa ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-06 21:27 ` H. Peter Anvin @ 2011-04-07 0:30 ` Paul E. McKenney 2011-04-07 5:49 ` Lai Jiangshan ` (4 more replies) 0 siblings, 5 replies; 67+ messages in thread From: Paul E. McKenney @ 2011-04-07 0:30 UTC (permalink / raw) To: H. Peter Anvin Cc: Peter Zijlstra, Lai Jiangshan, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On Wed, Apr 06, 2011 at 02:27:39PM -0700, H. Peter Anvin wrote: > On 04/06/2011 02:06 PM, Peter Zijlstra wrote: > > On Wed, 2011-04-06 at 13:13 -0700, Paul E. McKenney wrote: > >> And the following patch builds correctly for defconfig x86 builds, > >> while allowing rcupdate.h to see the sched.h definitions as needed > >> to inline rcu_read_lock() and rcu_read_unlock(). > >> > > Looks like an entirely reasonable patch to me ;-) > > > > Quite... a lot better than the original proposal! Glad you both like it! When I do an allyesconfig build, I do get errors during the "CHECK" phase, when it is putting things into the usr/include in the build tree. I believe that this is because I am exposing different header files to the library-export scripts. The following patch silences some of them, but I am really out of my depth here. Sam, Jan, Michal, help? Thanx, Paul ------------------------------------------------------------------------ rcu: cleanups in CHECK portion of kernel build Silence some complaints about kernel-only functionality being exposed to userspace. Not-signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> diff --git a/include/linux/kernel.h b/include/linux/kernel.h index 00cec4d..a243c13 100644 --- a/include/linux/kernel.h +++ b/include/linux/kernel.h @@ -648,6 +648,8 @@ struct sysinfo { #define BUILD_BUG_ON_ZERO(e) (sizeof(struct { int:-!!(e); })) #define BUILD_BUG_ON_NULL(e) ((void *)sizeof(struct { int:-!!(e); })) +#ifdef __KERNEL__ + /** * BUILD_BUG_ON - break compile if a condition is true. * @condition: the condition which the compiler should know is false. @@ -673,6 +675,7 @@ extern int __build_bug_on_failed; if (condition) __build_bug_on_failed = 1; \ } while(0) #endif +#endif /* __KERNEL__ */ /* Trap pasters of __FUNCTION__ at compile-time */ #define __FUNCTION__ (__func__) diff --git a/include/linux/soundcard.h b/include/linux/soundcard.h index 1904afe..f99c32f 100644 --- a/include/linux/soundcard.h +++ b/include/linux/soundcard.h @@ -1064,7 +1064,9 @@ typedef struct mixer_vol_table { */ #define SEQ_DECLAREBUF() SEQ_USE_EXTBUF() +#ifdef __KERNEL__ void seqbuf_dump(void); /* This function must be provided by programs */ +#endif #define SEQ_PM_DEFINES int __foo_bar___ diff --git a/include/linux/sysctl.h b/include/linux/sysctl.h index be80a59..92fb6fa 100644 --- a/include/linux/sysctl.h +++ b/include/linux/sysctl.h @@ -19,7 +19,9 @@ **************************************************************** */ +#ifdef __KERNEL__ #include <linux/key.h> +#endif #ifndef _LINUX_SYSCTL_H #define _LINUX_SYSCTL_H ^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-07 0:30 ` Paul E. McKenney @ 2011-04-07 5:49 ` Lai Jiangshan 2011-04-07 15:47 ` Paul E. McKenney 2011-04-07 7:05 ` [PATCH 1/4] rcu: split rcupdate.h Lai Jiangshan ` (3 subsequent siblings) 4 siblings, 1 reply; 67+ messages in thread From: Lai Jiangshan @ 2011-04-07 5:49 UTC (permalink / raw) To: paulmck Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On 04/07/2011 08:30 AM, Paul E. McKenney wrote: > On Wed, Apr 06, 2011 at 02:27:39PM -0700, H. Peter Anvin wrote: >> On 04/06/2011 02:06 PM, Peter Zijlstra wrote: >>> On Wed, 2011-04-06 at 13:13 -0700, Paul E. McKenney wrote: >>>> And the following patch builds correctly for defconfig x86 builds, >>>> while allowing rcupdate.h to see the sched.h definitions as needed >>>> to inline rcu_read_lock() and rcu_read_unlock(). >>>> >>> Looks like an entirely reasonable patch to me ;-) >>> >> >> Quite... a lot better than the original proposal! > > Glad you both like it! > > When I do an allyesconfig build, I do get errors during the "CHECK" > phase, when it is putting things into the usr/include in the build tree. > I believe that this is because I am exposing different header files to > the library-export scripts. The following patch silences some of them, > but I am really out of my depth here. > > Sam, Jan, Michal, help? > > Thanx, Paul > > ------------------------------------------------------------------------ > Easy to split rcupdate.h, hard to resolve the dependence problem. You can apply the next additional patch when you test: diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 0b4496a..ccbebeb 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -788,4 +788,10 @@ static inline void debug_rcu_head_unqueue(struct rcu_head *head) #error "Unknown RCU implementation specified to kernel configuration" #endif +#include <linux/sched.h> +static inline long test_use_task_struct(void) +{ + return current->state; +} + #endif /* __LINUX_RCUPDATE_H */ ^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-07 5:49 ` Lai Jiangshan @ 2011-04-07 15:47 ` Paul E. McKenney 2011-04-07 16:26 ` Paul E. McKenney 0 siblings, 1 reply; 67+ messages in thread From: Paul E. McKenney @ 2011-04-07 15:47 UTC (permalink / raw) To: Lai Jiangshan Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On Thu, Apr 07, 2011 at 01:49:51PM +0800, Lai Jiangshan wrote: > On 04/07/2011 08:30 AM, Paul E. McKenney wrote: > > On Wed, Apr 06, 2011 at 02:27:39PM -0700, H. Peter Anvin wrote: > >> On 04/06/2011 02:06 PM, Peter Zijlstra wrote: > >>> On Wed, 2011-04-06 at 13:13 -0700, Paul E. McKenney wrote: > >>>> And the following patch builds correctly for defconfig x86 builds, > >>>> while allowing rcupdate.h to see the sched.h definitions as needed > >>>> to inline rcu_read_lock() and rcu_read_unlock(). > >>>> > >>> Looks like an entirely reasonable patch to me ;-) > >>> > >> > >> Quite... a lot better than the original proposal! > > > > Glad you both like it! > > > > When I do an allyesconfig build, I do get errors during the "CHECK" > > phase, when it is putting things into the usr/include in the build tree. > > I believe that this is because I am exposing different header files to > > the library-export scripts. The following patch silences some of them, > > but I am really out of my depth here. > > > > Sam, Jan, Michal, help? > > > > Thanx, Paul > > > > ------------------------------------------------------------------------ > > > > Easy to split rcupdate.h, hard to resolve the dependence problem. > > You can apply the next additional patch when you test: I am sure that you are quite correct. ;-) I am moving _rcu_read_lock() and _rcu_read_unlock() into include/linux/rcutree.h and include/linux/rcutiny.h, and I am sure that more pain will ensue. One thing I don't understand... How does is it helping to group the task_struct RCU-related fields into a structure? Is that generating better code on your platform due to smaller offsets or something? Also, does your patchset address the CHECK warnings? Thanx, Paul > diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h > index 0b4496a..ccbebeb 100644 > --- a/include/linux/rcupdate.h > +++ b/include/linux/rcupdate.h > @@ -788,4 +788,10 @@ static inline void debug_rcu_head_unqueue(struct rcu_head *head) > #error "Unknown RCU implementation specified to kernel configuration" > #endif > > +#include <linux/sched.h> > +static inline long test_use_task_struct(void) > +{ > + return current->state; > +} > + > #endif /* __LINUX_RCUPDATE_H */ > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-07 15:47 ` Paul E. McKenney @ 2011-04-07 16:26 ` Paul E. McKenney 2011-04-08 1:26 ` Lai Jiangshan 0 siblings, 1 reply; 67+ messages in thread From: Paul E. McKenney @ 2011-04-07 16:26 UTC (permalink / raw) To: Lai Jiangshan Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On Thu, Apr 07, 2011 at 08:47:37AM -0700, Paul E. McKenney wrote: > On Thu, Apr 07, 2011 at 01:49:51PM +0800, Lai Jiangshan wrote: > > On 04/07/2011 08:30 AM, Paul E. McKenney wrote: > > > On Wed, Apr 06, 2011 at 02:27:39PM -0700, H. Peter Anvin wrote: > > >> On 04/06/2011 02:06 PM, Peter Zijlstra wrote: > > >>> On Wed, 2011-04-06 at 13:13 -0700, Paul E. McKenney wrote: > > >>>> And the following patch builds correctly for defconfig x86 builds, > > >>>> while allowing rcupdate.h to see the sched.h definitions as needed > > >>>> to inline rcu_read_lock() and rcu_read_unlock(). > > >>>> > > >>> Looks like an entirely reasonable patch to me ;-) > > >>> > > >> > > >> Quite... a lot better than the original proposal! > > > > > > Glad you both like it! > > > > > > When I do an allyesconfig build, I do get errors during the "CHECK" > > > phase, when it is putting things into the usr/include in the build tree. > > > I believe that this is because I am exposing different header files to > > > the library-export scripts. The following patch silences some of them, > > > but I am really out of my depth here. > > > > > > Sam, Jan, Michal, help? > > > > > > Thanx, Paul > > > > > > ------------------------------------------------------------------------ > > > > > > > Easy to split rcupdate.h, hard to resolve the dependence problem. > > > > You can apply the next additional patch when you test: > > I am sure that you are quite correct. ;-) > > I am moving _rcu_read_lock() and _rcu_read_unlock() into > include/linux/rcutree.h and include/linux/rcutiny.h, and I am sure that > more pain will ensue. > > One thing I don't understand... How does is it helping to group the > task_struct RCU-related fields into a structure? Is that generating > better code on your platform due to smaller offsets or something? > > Also, does your patchset address the CHECK warnings? I take it back... I applied the following patch on top of my earlier one, and a defconfig x86 build completed without error. (Though I have not tested the results of the build.) One possible difference -- I did this work on top of a recent Linus git commit (b2a8b4b81966) rather than on top of my -rcu tree. Also, I have not yet tried an allyesconfig build, which will no doubt locate some more problems. Thanx, Paul ------------------------------------------------------------------------ rcu: inline preemptible rcu_read_lock() and rcu_read_unlock() Move the definitions of __rcu_read_lock() and __rcu_read_unlock() from kernel/rcutree_plugin.h to include/linux/rcutree.h and from kernel/rcutiny_plugin.h to include/linux/rcutiny.h, allowing these functions to be inlined. include/linux/rcutiny.h | 34 ++++++++++++++++++++++++++++++++++ include/linux/rcutree.h | 34 ++++++++++++++++++++++++++++++++++ kernel/rcutiny_plugin.h | 38 ++------------------------------------ kernel/rcutree_plugin.h | 38 ++------------------------------------ 4 files changed, 72 insertions(+), 72 deletions(-) rcu: inline preemptible rcu_read_lock() and rcu_read_unlock() Move the definitions of __rcu_read_lock() and __rcu_read_unlock() from kernel/rcutree_plugin.h to include/linux/rcutree.h and from kernel/rcutiny_plugin.h to include/linux/rcutiny.h, allowing these functions to be inlined. Not-signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h index 30ebd7c..227a3dd 100644 --- a/include/linux/rcutiny.h +++ b/include/linux/rcutiny.h @@ -47,6 +47,40 @@ static inline void rcu_barrier(void) void rcu_barrier(void); void synchronize_rcu_expedited(void); +void rcu_read_unlock_special(struct task_struct *t); + +/* + * Tiny-preemptible RCU implementation for rcu_read_lock(). + * Just increment ->rcu_read_lock_nesting, shared state will be updated + * if we block. + */ +static inline void __rcu_read_lock(void) +{ + current->rcu_read_lock_nesting++; + barrier(); +} + +/* + * Tiny-preemptible RCU implementation for rcu_read_unlock(). + * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost + * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then + * invoke rcu_read_unlock_special() to clean up after a context switch + * in an RCU read-side critical section and other special cases. + */ +static inline void __rcu_read_unlock(void) +{ + struct task_struct *t = current; + + barrier(); + --t->rcu_read_lock_nesting; + barrier(); /* decrement before load of ->rcu_read_unlock_special */ + if (t->rcu_read_lock_nesting == 0 && + unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) + rcu_read_unlock_special(t); +#ifdef CONFIG_PROVE_LOCKING + WARN_ON_ONCE(t->rcu_read_lock_nesting < 0); +#endif /* #ifdef CONFIG_PROVE_LOCKING */ +} #endif /* #else #ifdef CONFIG_TINY_RCU */ diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h index c317eec..00a2b88 100644 --- a/include/linux/rcutree.h +++ b/include/linux/rcutree.h @@ -40,6 +40,40 @@ extern void rcu_cpu_stall_reset(void); #ifdef CONFIG_TREE_PREEMPT_RCU extern void exit_rcu(void); +extern void rcu_read_unlock_special(struct task_struct *t); + +/* + * Tree-preemptable RCU implementation for rcu_read_lock(). + * Just increment ->rcu_read_lock_nesting, shared state will be updated + * if we block. + */ +static inline void __rcu_read_lock(void) +{ + current->rcu_read_lock_nesting++; + barrier(); +} + +/* + * Tree-preemptable RCU implementation for rcu_read_unlock(). + * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost + * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then + * invoke rcu_read_unlock_special() to clean up after a context switch + * in an RCU read-side critical section and other special cases. + */ +static inline void __rcu_read_unlock(void) +{ + struct task_struct *t = current; + + barrier(); + --t->rcu_read_lock_nesting; + barrier(); /* decrement before load of ->rcu_read_unlock_special */ + if (t->rcu_read_lock_nesting == 0 && + unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) + rcu_read_unlock_special(t); +#ifdef CONFIG_PROVE_LOCKING + WARN_ON_ONCE(ACCESS_ONCE(t->rcu_read_lock_nesting) < 0); +#endif /* #ifdef CONFIG_PROVE_LOCKING */ +} #else /* #ifdef CONFIG_TREE_PREEMPT_RCU */ diff --git a/kernel/rcutiny_plugin.h b/kernel/rcutiny_plugin.h index 3cb8e36..d0e1ac3 100644 --- a/kernel/rcutiny_plugin.h +++ b/kernel/rcutiny_plugin.h @@ -520,23 +520,11 @@ void rcu_preempt_note_context_switch(void) } /* - * Tiny-preemptible RCU implementation for rcu_read_lock(). - * Just increment ->rcu_read_lock_nesting, shared state will be updated - * if we block. - */ -void __rcu_read_lock(void) -{ - current->rcu_read_lock_nesting++; - barrier(); /* needed if we ever invoke rcu_read_lock in rcutiny.c */ -} -EXPORT_SYMBOL_GPL(__rcu_read_lock); - -/* * Handle special cases during rcu_read_unlock(), such as needing to * notify RCU core processing or task having blocked during the RCU * read-side critical section. */ -static void rcu_read_unlock_special(struct task_struct *t) +void rcu_read_unlock_special(struct task_struct *t) { int empty; int empty_exp; @@ -616,29 +604,7 @@ static void rcu_read_unlock_special(struct task_struct *t) #endif /* #ifdef CONFIG_RCU_BOOST */ local_irq_restore(flags); } - -/* - * Tiny-preemptible RCU implementation for rcu_read_unlock(). - * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost - * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then - * invoke rcu_read_unlock_special() to clean up after a context switch - * in an RCU read-side critical section and other special cases. - */ -void __rcu_read_unlock(void) -{ - struct task_struct *t = current; - - barrier(); /* needed if we ever invoke rcu_read_unlock in rcutiny.c */ - --t->rcu_read_lock_nesting; - barrier(); /* decrement before load of ->rcu_read_unlock_special */ - if (t->rcu_read_lock_nesting == 0 && - unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) - rcu_read_unlock_special(t); -#ifdef CONFIG_PROVE_LOCKING - WARN_ON_ONCE(t->rcu_read_lock_nesting < 0); -#endif /* #ifdef CONFIG_PROVE_LOCKING */ -} -EXPORT_SYMBOL_GPL(__rcu_read_unlock); +EXPORT_SYMBOL_GPL(rcu_read_unlock_special); /* * Check for a quiescent state from the current CPU. When a task blocks, diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h index a363871..4b27afd 100644 --- a/kernel/rcutree_plugin.h +++ b/kernel/rcutree_plugin.h @@ -196,18 +196,6 @@ static void rcu_preempt_note_context_switch(int cpu) } /* - * Tree-preemptable RCU implementation for rcu_read_lock(). - * Just increment ->rcu_read_lock_nesting, shared state will be updated - * if we block. - */ -void __rcu_read_lock(void) -{ - current->rcu_read_lock_nesting++; - barrier(); /* needed if we ever invoke rcu_read_lock in rcutree.c */ -} -EXPORT_SYMBOL_GPL(__rcu_read_lock); - -/* * Check for preempted RCU readers blocking the current grace period * for the specified rcu_node structure. If the caller needs a reliable * answer, it must hold the rcu_node's ->lock. @@ -261,7 +249,7 @@ static void rcu_report_unblock_qs_rnp(struct rcu_node *rnp, unsigned long flags) * notify RCU core processing or task having blocked during the RCU * read-side critical section. */ -static void rcu_read_unlock_special(struct task_struct *t) +void rcu_read_unlock_special(struct task_struct *t) { int empty; int empty_exp; @@ -332,29 +320,7 @@ static void rcu_read_unlock_special(struct task_struct *t) local_irq_restore(flags); } } - -/* - * Tree-preemptable RCU implementation for rcu_read_unlock(). - * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost - * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then - * invoke rcu_read_unlock_special() to clean up after a context switch - * in an RCU read-side critical section and other special cases. - */ -void __rcu_read_unlock(void) -{ - struct task_struct *t = current; - - barrier(); /* needed if we ever invoke rcu_read_unlock in rcutree.c */ - --t->rcu_read_lock_nesting; - barrier(); /* decrement before load of ->rcu_read_unlock_special */ - if (t->rcu_read_lock_nesting == 0 && - unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) - rcu_read_unlock_special(t); -#ifdef CONFIG_PROVE_LOCKING - WARN_ON_ONCE(ACCESS_ONCE(t->rcu_read_lock_nesting) < 0); -#endif /* #ifdef CONFIG_PROVE_LOCKING */ -} -EXPORT_SYMBOL_GPL(__rcu_read_unlock); +EXPORT_SYMBOL_GPL(rcu_read_unlock_special); #ifdef CONFIG_RCU_CPU_STALL_DETECTOR ^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-07 16:26 ` Paul E. McKenney @ 2011-04-08 1:26 ` Lai Jiangshan 2011-04-08 5:13 ` Paul E. McKenney 0 siblings, 1 reply; 67+ messages in thread From: Lai Jiangshan @ 2011-04-08 1:26 UTC (permalink / raw) To: paulmck Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On 04/08/2011 12:26 AM, Paul E. McKenney wrote: > On Thu, Apr 07, 2011 at 08:47:37AM -0700, Paul E. McKenney wrote: >> On Thu, Apr 07, 2011 at 01:49:51PM +0800, Lai Jiangshan wrote: >>> On 04/07/2011 08:30 AM, Paul E. McKenney wrote: >>>> On Wed, Apr 06, 2011 at 02:27:39PM -0700, H. Peter Anvin wrote: >>>>> On 04/06/2011 02:06 PM, Peter Zijlstra wrote: >>>>>> On Wed, 2011-04-06 at 13:13 -0700, Paul E. McKenney wrote: >>>>>>> And the following patch builds correctly for defconfig x86 builds, >>>>>>> while allowing rcupdate.h to see the sched.h definitions as needed >>>>>>> to inline rcu_read_lock() and rcu_read_unlock(). >>>>>>> >>>>>> Looks like an entirely reasonable patch to me ;-) >>>>>> >>>>> >>>>> Quite... a lot better than the original proposal! >>>> >>>> Glad you both like it! >>>> >>>> When I do an allyesconfig build, I do get errors during the "CHECK" >>>> phase, when it is putting things into the usr/include in the build tree. >>>> I believe that this is because I am exposing different header files to >>>> the library-export scripts. The following patch silences some of them, >>>> but I am really out of my depth here. >>>> >>>> Sam, Jan, Michal, help? >>>> >>>> Thanx, Paul >>>> >>>> ------------------------------------------------------------------------ >>>> >>> >>> Easy to split rcupdate.h, hard to resolve the dependence problem. >>> >>> You can apply the next additional patch when you test: >> >> I am sure that you are quite correct. ;-) >> >> I am moving _rcu_read_lock() and _rcu_read_unlock() into >> include/linux/rcutree.h and include/linux/rcutiny.h, and I am sure that >> more pain will ensue. >> >> One thing I don't understand... How does is it helping to group the >> task_struct RCU-related fields into a structure? Is that generating >> better code on your platform due to smaller offsets or something? You don't like task_rcu_struct patch? I think it can make code clearer, and it can also check the code even when CONFIG_PREEMPT_RCU=n. For rcu_read_[un]lock(), it generates the same code, no better, no worse. It is just a cleanup patch, it is helpless for making rcu_read_[un]lock() inline, if you don't like it, I will give up it. >> >> Also, does your patchset address the CHECK warnings? > > I take it back... I applied the following patch on top of my earlier > one, and a defconfig x86 build completed without error. (Though I have > not tested the results of the build.) > > One possible difference -- I did this work on top of a recent Linus > git commit (b2a8b4b81966) rather than on top of my -rcu tree. Also, > I have not yet tried an allyesconfig build, which will no doubt locate > some more problems. > > Thanx, Paul > when defconfig or allyesconfig, CONFIG_PREEMPT=n and CONFIG_TREE_PREEMPT_RCU=n when you make them "y": In file included from include/linux/rcupdate.h:764:0, from include/linux/tracepoint.h:19, from include/linux/module.h:18, from include/linux/crypto.h:21, from arch/x86/kernel/asm-offsets.c:8: include/linux/rcutree.h:50:20: error: static declaration of ‘__rcu_read_lock’ follows non-static declaration include/linux/rcupdate.h:76:13: note: previous declaration of ‘__rcu_read_lock’ was here include/linux/rcutree.h:63:20: error: static declaration of ‘__rcu_read_unlock’ follows non-static declaration include/linux/rcupdate.h:77:13: note: previous declaration of ‘__rcu_read_unlock’ was here make[1]: *** [arch/x86/kernel/asm-offsets.s] Error 1 make: *** [prepare0] Error 2 ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-08 1:26 ` Lai Jiangshan @ 2011-04-08 5:13 ` Paul E. McKenney 2011-04-11 3:08 ` Lai Jiangshan 0 siblings, 1 reply; 67+ messages in thread From: Paul E. McKenney @ 2011-04-08 5:13 UTC (permalink / raw) To: Lai Jiangshan Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On Fri, Apr 08, 2011 at 09:26:16AM +0800, Lai Jiangshan wrote: > On 04/08/2011 12:26 AM, Paul E. McKenney wrote: > > On Thu, Apr 07, 2011 at 08:47:37AM -0700, Paul E. McKenney wrote: > >> On Thu, Apr 07, 2011 at 01:49:51PM +0800, Lai Jiangshan wrote: > >>> On 04/07/2011 08:30 AM, Paul E. McKenney wrote: > >>>> On Wed, Apr 06, 2011 at 02:27:39PM -0700, H. Peter Anvin wrote: > >>>>> On 04/06/2011 02:06 PM, Peter Zijlstra wrote: > >>>>>> On Wed, 2011-04-06 at 13:13 -0700, Paul E. McKenney wrote: > >>>>>>> And the following patch builds correctly for defconfig x86 builds, > >>>>>>> while allowing rcupdate.h to see the sched.h definitions as needed > >>>>>>> to inline rcu_read_lock() and rcu_read_unlock(). > >>>>>>> > >>>>>> Looks like an entirely reasonable patch to me ;-) > >>>>>> > >>>>> > >>>>> Quite... a lot better than the original proposal! > >>>> > >>>> Glad you both like it! > >>>> > >>>> When I do an allyesconfig build, I do get errors during the "CHECK" > >>>> phase, when it is putting things into the usr/include in the build tree. > >>>> I believe that this is because I am exposing different header files to > >>>> the library-export scripts. The following patch silences some of them, > >>>> but I am really out of my depth here. > >>>> > >>>> Sam, Jan, Michal, help? > >>>> > >>>> Thanx, Paul > >>>> > >>>> ------------------------------------------------------------------------ > >>>> > >>> > >>> Easy to split rcupdate.h, hard to resolve the dependence problem. > >>> > >>> You can apply the next additional patch when you test: > >> > >> I am sure that you are quite correct. ;-) > >> > >> I am moving _rcu_read_lock() and _rcu_read_unlock() into > >> include/linux/rcutree.h and include/linux/rcutiny.h, and I am sure that > >> more pain will ensue. > >> > >> One thing I don't understand... How does is it helping to group the > >> task_struct RCU-related fields into a structure? Is that generating > >> better code on your platform due to smaller offsets or something? > > You don't like task_rcu_struct patch? I think it can make code clearer, > and it can also check the code even when CONFIG_PREEMPT_RCU=n. > > For rcu_read_[un]lock(), it generates the same code, no better, no worse. > > It is just a cleanup patch, it is helpless for making rcu_read_[un]lock() inline, > if you don't like it, I will give up it. I don't know that I feel strongly either way about it. It was necessary with the integer-offset approach, but optional now. > >> Also, does your patchset address the CHECK warnings? > > > > I take it back... I applied the following patch on top of my earlier > > one, and a defconfig x86 build completed without error. (Though I have > > not tested the results of the build.) > > > > One possible difference -- I did this work on top of a recent Linus > > git commit (b2a8b4b81966) rather than on top of my -rcu tree. Also, > > I have not yet tried an allyesconfig build, which will no doubt locate > > some more problems. > > > > Thanx, Paul > > > > when defconfig or allyesconfig, CONFIG_PREEMPT=n and CONFIG_TREE_PREEMPT_RCU=n > when you make them "y": > > In file included from include/linux/rcupdate.h:764:0, > from include/linux/tracepoint.h:19, > from include/linux/module.h:18, > from include/linux/crypto.h:21, > from arch/x86/kernel/asm-offsets.c:8: > include/linux/rcutree.h:50:20: error: static declaration of ‘__rcu_read_lock’ follows non-static declaration > include/linux/rcupdate.h:76:13: note: previous declaration of ‘__rcu_read_lock’ was here > include/linux/rcutree.h:63:20: error: static declaration of ‘__rcu_read_unlock’ follows non-static declaration > include/linux/rcupdate.h:77:13: note: previous declaration of ‘__rcu_read_unlock’ was here > make[1]: *** [arch/x86/kernel/asm-offsets.s] Error 1 > make: *** [prepare0] Error 2 Yep. I need to move the rcu_read_lock() APIs to follow the inclusion of rcutree.h and rcutiny.h. Also add include of sched.h to rcutiny.h. The code movement does bloat the patch a bit. But rcu_assign_pointer() must precede the inclusion of rcutree.h and rcutiny.h, so it is not possible to simply move the inclusions. See below. Thanx, Paul ------------------------------------------------------------------------ diff --git a/include/linux/kernel.h b/include/linux/kernel.h index 00cec4d..a243c13 100644 --- a/include/linux/kernel.h +++ b/include/linux/kernel.h @@ -648,6 +648,8 @@ struct sysinfo { #define BUILD_BUG_ON_ZERO(e) (sizeof(struct { int:-!!(e); })) #define BUILD_BUG_ON_NULL(e) ((void *)sizeof(struct { int:-!!(e); })) +#ifdef __KERNEL__ + /** * BUILD_BUG_ON - break compile if a condition is true. * @condition: the condition which the compiler should know is false. @@ -673,6 +675,7 @@ extern int __build_bug_on_failed; if (condition) __build_bug_on_failed = 1; \ } while(0) #endif +#endif /* __KERNEL__ */ /* Trap pasters of __FUNCTION__ at compile-time */ #define __FUNCTION__ (__func__) diff --git a/include/linux/pid.h b/include/linux/pid.h index efceda0..3c5719b 100644 --- a/include/linux/pid.h +++ b/include/linux/pid.h @@ -1,7 +1,7 @@ #ifndef _LINUX_PID_H #define _LINUX_PID_H -#include <linux/rcupdate.h> +#include <linux/types.h> enum pid_type { diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index ff422d2..55e941f 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -33,6 +33,7 @@ #ifndef __LINUX_RCUPDATE_H #define __LINUX_RCUPDATE_H +#include <linux/types.h> #include <linux/cache.h> #include <linux/spinlock.h> #include <linux/threads.h> @@ -52,16 +53,6 @@ extern int rcutorture_runnable; /* for sysctl */ #define ULONG_CMP_GE(a, b) (ULONG_MAX / 2 >= (a) - (b)) #define ULONG_CMP_LT(a, b) (ULONG_MAX / 2 < (a) - (b)) -/** - * struct rcu_head - callback structure for use with RCU - * @next: next update requests in a list - * @func: actual update function to call after the grace period. - */ -struct rcu_head { - struct rcu_head *next; - void (*func)(struct rcu_head *head); -}; - /* Exported common interfaces */ extern void call_rcu_sched(struct rcu_head *head, void (*func)(struct rcu_head *rcu)); @@ -82,8 +73,6 @@ static inline void __rcu_read_unlock_bh(void) #ifdef CONFIG_PREEMPT_RCU -extern void __rcu_read_lock(void); -extern void __rcu_read_unlock(void); void synchronize_rcu(void); /* @@ -141,14 +130,6 @@ static inline void rcu_exit_nohz(void) #endif /* #else #ifdef CONFIG_NO_HZ */ -#if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) -#include <linux/rcutree.h> -#elif defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU) -#include <linux/rcutiny.h> -#else -#error "Unknown RCU implementation specified to kernel configuration" -#endif - /* * init_rcu_head_on_stack()/destroy_rcu_head_on_stack() are needed for dynamic * initialization and destruction of rcu_head on the stack. rcu_head structures @@ -535,6 +516,134 @@ extern int rcu_my_thread_group_empty(void); #define rcu_dereference_sched(p) rcu_dereference_sched_check(p, 0) /** + * rcu_assign_pointer() - assign to RCU-protected pointer + * @p: pointer to assign to + * @v: value to assign (publish) + * + * Assigns the specified value to the specified RCU-protected + * pointer, ensuring that any concurrent RCU readers will see + * any prior initialization. Returns the value assigned. + * + * Inserts memory barriers on architectures that require them + * (pretty much all of them other than x86), and also prevents + * the compiler from reordering the code that initializes the + * structure after the pointer assignment. More importantly, this + * call documents which pointers will be dereferenced by RCU read-side + * code. + */ +#define rcu_assign_pointer(p, v) \ + __rcu_assign_pointer((p), (v), __rcu) + +/** + * RCU_INIT_POINTER() - initialize an RCU protected pointer + * + * Initialize an RCU-protected pointer in such a way to avoid RCU-lockdep + * splats. + */ +#define RCU_INIT_POINTER(p, v) \ + p = (typeof(*v) __force __rcu *)(v) + +/* Infrastructure to implement the synchronize_() primitives. */ + +struct rcu_synchronize { + struct rcu_head head; + struct completion completion; +}; + +extern void wakeme_after_rcu(struct rcu_head *head); + +#ifdef CONFIG_PREEMPT_RCU + +/** + * call_rcu() - Queue an RCU callback for invocation after a grace period. + * @head: structure to be used for queueing the RCU updates. + * @func: actual callback function to be invoked after the grace period + * + * The callback function will be invoked some time after a full grace + * period elapses, in other words after all pre-existing RCU read-side + * critical sections have completed. However, the callback function + * might well execute concurrently with RCU read-side critical sections + * that started after call_rcu() was invoked. RCU read-side critical + * sections are delimited by rcu_read_lock() and rcu_read_unlock(), + * and may be nested. + */ +extern void call_rcu(struct rcu_head *head, + void (*func)(struct rcu_head *head)); + +#else /* #ifdef CONFIG_PREEMPT_RCU */ + +/* In classic RCU, call_rcu() is just call_rcu_sched(). */ +#define call_rcu call_rcu_sched + +#endif /* #else #ifdef CONFIG_PREEMPT_RCU */ + +/** + * call_rcu_bh() - Queue an RCU for invocation after a quicker grace period. + * @head: structure to be used for queueing the RCU updates. + * @func: actual callback function to be invoked after the grace period + * + * The callback function will be invoked some time after a full grace + * period elapses, in other words after all currently executing RCU + * read-side critical sections have completed. call_rcu_bh() assumes + * that the read-side critical sections end on completion of a softirq + * handler. This means that read-side critical sections in process + * context must not be interrupted by softirqs. This interface is to be + * used when most of the read-side critical sections are in softirq context. + * RCU read-side critical sections are delimited by : + * - rcu_read_lock() and rcu_read_unlock(), if in interrupt context. + * OR + * - rcu_read_lock_bh() and rcu_read_unlock_bh(), if in process context. + * These may be nested. + */ +extern void call_rcu_bh(struct rcu_head *head, + void (*func)(struct rcu_head *head)); + +/* + * debug_rcu_head_queue()/debug_rcu_head_unqueue() are used internally + * by call_rcu() and rcu callback execution, and are therefore not part of the + * RCU API. Leaving in rcupdate.h because they are used by all RCU flavors. + */ + +#ifdef CONFIG_DEBUG_OBJECTS_RCU_HEAD +# define STATE_RCU_HEAD_READY 0 +# define STATE_RCU_HEAD_QUEUED 1 + +extern struct debug_obj_descr rcuhead_debug_descr; + +static inline void debug_rcu_head_queue(struct rcu_head *head) +{ + debug_object_activate(head, &rcuhead_debug_descr); + debug_object_active_state(head, &rcuhead_debug_descr, + STATE_RCU_HEAD_READY, + STATE_RCU_HEAD_QUEUED); +} + +static inline void debug_rcu_head_unqueue(struct rcu_head *head) +{ + debug_object_active_state(head, &rcuhead_debug_descr, + STATE_RCU_HEAD_QUEUED, + STATE_RCU_HEAD_READY); + debug_object_deactivate(head, &rcuhead_debug_descr); +} +#else /* !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ +static inline void debug_rcu_head_queue(struct rcu_head *head) +{ +} + +static inline void debug_rcu_head_unqueue(struct rcu_head *head) +{ +} +#endif /* #else !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ + +#if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) +#include <linux/rcutree.h> +#elif defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU) +#include <linux/rcutiny.h> +#else +#error "Unknown RCU implementation specified to kernel configuration" +#endif + +/** * rcu_read_lock() - mark the beginning of an RCU read-side critical section * * When synchronize_rcu() is invoked on one CPU while other CPUs @@ -677,124 +786,4 @@ static inline notrace void rcu_read_unlock_sched_notrace(void) preempt_enable_notrace(); } -/** - * rcu_assign_pointer() - assign to RCU-protected pointer - * @p: pointer to assign to - * @v: value to assign (publish) - * - * Assigns the specified value to the specified RCU-protected - * pointer, ensuring that any concurrent RCU readers will see - * any prior initialization. Returns the value assigned. - * - * Inserts memory barriers on architectures that require them - * (pretty much all of them other than x86), and also prevents - * the compiler from reordering the code that initializes the - * structure after the pointer assignment. More importantly, this - * call documents which pointers will be dereferenced by RCU read-side - * code. - */ -#define rcu_assign_pointer(p, v) \ - __rcu_assign_pointer((p), (v), __rcu) - -/** - * RCU_INIT_POINTER() - initialize an RCU protected pointer - * - * Initialize an RCU-protected pointer in such a way to avoid RCU-lockdep - * splats. - */ -#define RCU_INIT_POINTER(p, v) \ - p = (typeof(*v) __force __rcu *)(v) - -/* Infrastructure to implement the synchronize_() primitives. */ - -struct rcu_synchronize { - struct rcu_head head; - struct completion completion; -}; - -extern void wakeme_after_rcu(struct rcu_head *head); - -#ifdef CONFIG_PREEMPT_RCU - -/** - * call_rcu() - Queue an RCU callback for invocation after a grace period. - * @head: structure to be used for queueing the RCU updates. - * @func: actual callback function to be invoked after the grace period - * - * The callback function will be invoked some time after a full grace - * period elapses, in other words after all pre-existing RCU read-side - * critical sections have completed. However, the callback function - * might well execute concurrently with RCU read-side critical sections - * that started after call_rcu() was invoked. RCU read-side critical - * sections are delimited by rcu_read_lock() and rcu_read_unlock(), - * and may be nested. - */ -extern void call_rcu(struct rcu_head *head, - void (*func)(struct rcu_head *head)); - -#else /* #ifdef CONFIG_PREEMPT_RCU */ - -/* In classic RCU, call_rcu() is just call_rcu_sched(). */ -#define call_rcu call_rcu_sched - -#endif /* #else #ifdef CONFIG_PREEMPT_RCU */ - -/** - * call_rcu_bh() - Queue an RCU for invocation after a quicker grace period. - * @head: structure to be used for queueing the RCU updates. - * @func: actual callback function to be invoked after the grace period - * - * The callback function will be invoked some time after a full grace - * period elapses, in other words after all currently executing RCU - * read-side critical sections have completed. call_rcu_bh() assumes - * that the read-side critical sections end on completion of a softirq - * handler. This means that read-side critical sections in process - * context must not be interrupted by softirqs. This interface is to be - * used when most of the read-side critical sections are in softirq context. - * RCU read-side critical sections are delimited by : - * - rcu_read_lock() and rcu_read_unlock(), if in interrupt context. - * OR - * - rcu_read_lock_bh() and rcu_read_unlock_bh(), if in process context. - * These may be nested. - */ -extern void call_rcu_bh(struct rcu_head *head, - void (*func)(struct rcu_head *head)); - -/* - * debug_rcu_head_queue()/debug_rcu_head_unqueue() are used internally - * by call_rcu() and rcu callback execution, and are therefore not part of the - * RCU API. Leaving in rcupdate.h because they are used by all RCU flavors. - */ - -#ifdef CONFIG_DEBUG_OBJECTS_RCU_HEAD -# define STATE_RCU_HEAD_READY 0 -# define STATE_RCU_HEAD_QUEUED 1 - -extern struct debug_obj_descr rcuhead_debug_descr; - -static inline void debug_rcu_head_queue(struct rcu_head *head) -{ - debug_object_activate(head, &rcuhead_debug_descr); - debug_object_active_state(head, &rcuhead_debug_descr, - STATE_RCU_HEAD_READY, - STATE_RCU_HEAD_QUEUED); -} - -static inline void debug_rcu_head_unqueue(struct rcu_head *head) -{ - debug_object_active_state(head, &rcuhead_debug_descr, - STATE_RCU_HEAD_QUEUED, - STATE_RCU_HEAD_READY); - debug_object_deactivate(head, &rcuhead_debug_descr); -} -#else /* !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ -static inline void debug_rcu_head_queue(struct rcu_head *head) -{ -} - -static inline void debug_rcu_head_unqueue(struct rcu_head *head) -{ -} -#endif /* #else !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ - #endif /* __LINUX_RCUPDATE_H */ diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h index 30ebd7c..167fb19 100644 --- a/include/linux/rcutiny.h +++ b/include/linux/rcutiny.h @@ -26,6 +26,7 @@ #define __LINUX_TINY_H #include <linux/cache.h> +#include <linux/sched.h> static inline void rcu_init(void) { @@ -47,6 +48,40 @@ static inline void rcu_barrier(void) void rcu_barrier(void); void synchronize_rcu_expedited(void); +void rcu_read_unlock_special(struct task_struct *t); + +/* + * Tiny-preemptible RCU implementation for rcu_read_lock(). + * Just increment ->rcu_read_lock_nesting, shared state will be updated + * if we block. + */ +static inline void __rcu_read_lock(void) +{ + current->rcu_read_lock_nesting++; + barrier(); +} + +/* + * Tiny-preemptible RCU implementation for rcu_read_unlock(). + * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost + * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then + * invoke rcu_read_unlock_special() to clean up after a context switch + * in an RCU read-side critical section and other special cases. + */ +static inline void __rcu_read_unlock(void) +{ + struct task_struct *t = current; + + barrier(); + --t->rcu_read_lock_nesting; + barrier(); /* decrement before load of ->rcu_read_unlock_special */ + if (t->rcu_read_lock_nesting == 0 && + unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) + rcu_read_unlock_special(t); +#ifdef CONFIG_PROVE_LOCKING + WARN_ON_ONCE(t->rcu_read_lock_nesting < 0); +#endif /* #ifdef CONFIG_PROVE_LOCKING */ +} #endif /* #else #ifdef CONFIG_TINY_RCU */ diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h index 3a93348..00a2b88 100644 --- a/include/linux/rcutree.h +++ b/include/linux/rcutree.h @@ -30,6 +30,8 @@ #ifndef __LINUX_RCUTREE_H #define __LINUX_RCUTREE_H +#include <linux/sched.h> + extern void rcu_init(void); extern void rcu_note_context_switch(int cpu); extern int rcu_needs_cpu(int cpu); @@ -38,6 +40,40 @@ extern void rcu_cpu_stall_reset(void); #ifdef CONFIG_TREE_PREEMPT_RCU extern void exit_rcu(void); +extern void rcu_read_unlock_special(struct task_struct *t); + +/* + * Tree-preemptable RCU implementation for rcu_read_lock(). + * Just increment ->rcu_read_lock_nesting, shared state will be updated + * if we block. + */ +static inline void __rcu_read_lock(void) +{ + current->rcu_read_lock_nesting++; + barrier(); +} + +/* + * Tree-preemptable RCU implementation for rcu_read_unlock(). + * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost + * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then + * invoke rcu_read_unlock_special() to clean up after a context switch + * in an RCU read-side critical section and other special cases. + */ +static inline void __rcu_read_unlock(void) +{ + struct task_struct *t = current; + + barrier(); + --t->rcu_read_lock_nesting; + barrier(); /* decrement before load of ->rcu_read_unlock_special */ + if (t->rcu_read_lock_nesting == 0 && + unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) + rcu_read_unlock_special(t); +#ifdef CONFIG_PROVE_LOCKING + WARN_ON_ONCE(ACCESS_ONCE(t->rcu_read_lock_nesting) < 0); +#endif /* #ifdef CONFIG_PROVE_LOCKING */ +} #else /* #ifdef CONFIG_TREE_PREEMPT_RCU */ diff --git a/include/linux/sched.h b/include/linux/sched.h index 83bd2e2..30a4444 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -78,7 +78,7 @@ struct sched_param { #include <linux/topology.h> #include <linux/proportions.h> #include <linux/seccomp.h> -#include <linux/rcupdate.h> +#include <linux/types.h> #include <linux/rculist.h> #include <linux/rtmutex.h> @@ -2241,11 +2241,9 @@ int same_thread_group(struct task_struct *p1, struct task_struct *p2) return p1->tgid == p2->tgid; } -static inline struct task_struct *next_thread(const struct task_struct *p) -{ - return list_entry_rcu(p->thread_group.next, - struct task_struct, thread_group); -} +/* Avoid #include hell for inlining rcu_read_lock(). */ +#define next_thread(p) \ + list_entry_rcu((p)->thread_group.next, struct task_struct, thread_group) static inline int thread_group_empty(struct task_struct *p) { diff --git a/include/linux/sem.h b/include/linux/sem.h index f2961af..8489a1f 100644 --- a/include/linux/sem.h +++ b/include/linux/sem.h @@ -78,7 +78,7 @@ struct seminfo { #ifdef __KERNEL__ #include <asm/atomic.h> -#include <linux/rcupdate.h> +#include <linux/types.h> #include <linux/cache.h> struct task_struct; diff --git a/include/linux/soundcard.h b/include/linux/soundcard.h index 1904afe..f99c32f 100644 --- a/include/linux/soundcard.h +++ b/include/linux/soundcard.h @@ -1064,7 +1064,9 @@ typedef struct mixer_vol_table { */ #define SEQ_DECLAREBUF() SEQ_USE_EXTBUF() +#ifdef __KERNEL__ void seqbuf_dump(void); /* This function must be provided by programs */ +#endif #define SEQ_PM_DEFINES int __foo_bar___ diff --git a/include/linux/sysctl.h b/include/linux/sysctl.h index 11684d9..92fb6fa 100644 --- a/include/linux/sysctl.h +++ b/include/linux/sysctl.h @@ -19,6 +19,10 @@ **************************************************************** */ +#ifdef __KERNEL__ +#include <linux/key.h> +#endif + #ifndef _LINUX_SYSCTL_H #define _LINUX_SYSCTL_H @@ -1012,8 +1016,7 @@ extern int proc_do_large_bitmap(struct ctl_table *, int, */ /* A sysctl table is an array of struct ctl_table: */ -struct ctl_table -{ +struct ctl_table { const char *procname; /* Text ID for /proc/sys, or zero */ void *data; int maxlen; diff --git a/include/linux/types.h b/include/linux/types.h index 176da8c..868ef8b 100644 --- a/include/linux/types.h +++ b/include/linux/types.h @@ -231,6 +231,16 @@ struct hlist_node { struct hlist_node *next, **pprev; }; +/** + * struct rcu_head - callback structure for use with RCU + * @next: next update requests in a list + * @func: actual update function to call after the grace period. + */ +struct rcu_head { + struct rcu_head *next; + void (*func)(struct rcu_head *head); +}; + struct ustat { __kernel_daddr_t f_tfree; __kernel_ino_t f_tinode; diff --git a/kernel/pid_namespace.c b/kernel/pid_namespace.c index e9c9adc..bfa75df 100644 --- a/kernel/pid_namespace.c +++ b/kernel/pid_namespace.c @@ -8,6 +8,8 @@ * */ +#include <linux/types.h> +#include <asm/atomic.h> #include <linux/pid.h> #include <linux/pid_namespace.h> #include <linux/syscalls.h> diff --git a/kernel/rcutiny_plugin.h b/kernel/rcutiny_plugin.h index 3cb8e36..d0e1ac3 100644 --- a/kernel/rcutiny_plugin.h +++ b/kernel/rcutiny_plugin.h @@ -520,23 +520,11 @@ void rcu_preempt_note_context_switch(void) } /* - * Tiny-preemptible RCU implementation for rcu_read_lock(). - * Just increment ->rcu_read_lock_nesting, shared state will be updated - * if we block. - */ -void __rcu_read_lock(void) -{ - current->rcu_read_lock_nesting++; - barrier(); /* needed if we ever invoke rcu_read_lock in rcutiny.c */ -} -EXPORT_SYMBOL_GPL(__rcu_read_lock); - -/* * Handle special cases during rcu_read_unlock(), such as needing to * notify RCU core processing or task having blocked during the RCU * read-side critical section. */ -static void rcu_read_unlock_special(struct task_struct *t) +void rcu_read_unlock_special(struct task_struct *t) { int empty; int empty_exp; @@ -616,29 +604,7 @@ static void rcu_read_unlock_special(struct task_struct *t) #endif /* #ifdef CONFIG_RCU_BOOST */ local_irq_restore(flags); } - -/* - * Tiny-preemptible RCU implementation for rcu_read_unlock(). - * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost - * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then - * invoke rcu_read_unlock_special() to clean up after a context switch - * in an RCU read-side critical section and other special cases. - */ -void __rcu_read_unlock(void) -{ - struct task_struct *t = current; - - barrier(); /* needed if we ever invoke rcu_read_unlock in rcutiny.c */ - --t->rcu_read_lock_nesting; - barrier(); /* decrement before load of ->rcu_read_unlock_special */ - if (t->rcu_read_lock_nesting == 0 && - unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) - rcu_read_unlock_special(t); -#ifdef CONFIG_PROVE_LOCKING - WARN_ON_ONCE(t->rcu_read_lock_nesting < 0); -#endif /* #ifdef CONFIG_PROVE_LOCKING */ -} -EXPORT_SYMBOL_GPL(__rcu_read_unlock); +EXPORT_SYMBOL_GPL(rcu_read_unlock_special); /* * Check for a quiescent state from the current CPU. When a task blocks, diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h index a363871..4b27afd 100644 --- a/kernel/rcutree_plugin.h +++ b/kernel/rcutree_plugin.h @@ -196,18 +196,6 @@ static void rcu_preempt_note_context_switch(int cpu) } /* - * Tree-preemptable RCU implementation for rcu_read_lock(). - * Just increment ->rcu_read_lock_nesting, shared state will be updated - * if we block. - */ -void __rcu_read_lock(void) -{ - current->rcu_read_lock_nesting++; - barrier(); /* needed if we ever invoke rcu_read_lock in rcutree.c */ -} -EXPORT_SYMBOL_GPL(__rcu_read_lock); - -/* * Check for preempted RCU readers blocking the current grace period * for the specified rcu_node structure. If the caller needs a reliable * answer, it must hold the rcu_node's ->lock. @@ -261,7 +249,7 @@ static void rcu_report_unblock_qs_rnp(struct rcu_node *rnp, unsigned long flags) * notify RCU core processing or task having blocked during the RCU * read-side critical section. */ -static void rcu_read_unlock_special(struct task_struct *t) +void rcu_read_unlock_special(struct task_struct *t) { int empty; int empty_exp; @@ -332,29 +320,7 @@ static void rcu_read_unlock_special(struct task_struct *t) local_irq_restore(flags); } } - -/* - * Tree-preemptable RCU implementation for rcu_read_unlock(). - * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost - * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then - * invoke rcu_read_unlock_special() to clean up after a context switch - * in an RCU read-side critical section and other special cases. - */ -void __rcu_read_unlock(void) -{ - struct task_struct *t = current; - - barrier(); /* needed if we ever invoke rcu_read_unlock in rcutree.c */ - --t->rcu_read_lock_nesting; - barrier(); /* decrement before load of ->rcu_read_unlock_special */ - if (t->rcu_read_lock_nesting == 0 && - unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) - rcu_read_unlock_special(t); -#ifdef CONFIG_PROVE_LOCKING - WARN_ON_ONCE(ACCESS_ONCE(t->rcu_read_lock_nesting) < 0); -#endif /* #ifdef CONFIG_PROVE_LOCKING */ -} -EXPORT_SYMBOL_GPL(__rcu_read_unlock); +EXPORT_SYMBOL_GPL(rcu_read_unlock_special); #ifdef CONFIG_RCU_CPU_STALL_DETECTOR ^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-08 5:13 ` Paul E. McKenney @ 2011-04-11 3:08 ` Lai Jiangshan 2011-04-11 5:12 ` Paul E. McKenney 0 siblings, 1 reply; 67+ messages in thread From: Lai Jiangshan @ 2011-04-11 3:08 UTC (permalink / raw) To: paulmck Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On 04/08/2011 01:13 PM, Paul E. McKenney wrote: > On Fri, Apr 08, 2011 at 09:26:16AM +0800, Lai Jiangshan wrote: >> On 04/08/2011 12:26 AM, Paul E. McKenney wrote: >>> On Thu, Apr 07, 2011 at 08:47:37AM -0700, Paul E. McKenney wrote: >>>> On Thu, Apr 07, 2011 at 01:49:51PM +0800, Lai Jiangshan wrote: >>>>> On 04/07/2011 08:30 AM, Paul E. McKenney wrote: >>>>>> On Wed, Apr 06, 2011 at 02:27:39PM -0700, H. Peter Anvin wrote: >>>>>>> On 04/06/2011 02:06 PM, Peter Zijlstra wrote: >>>>>>>> On Wed, 2011-04-06 at 13:13 -0700, Paul E. McKenney wrote: >>>>>>>>> And the following patch builds correctly for defconfig x86 builds, >>>>>>>>> while allowing rcupdate.h to see the sched.h definitions as needed >>>>>>>>> to inline rcu_read_lock() and rcu_read_unlock(). >>>>>>>>> >>>>>>>> Looks like an entirely reasonable patch to me ;-) >>>>>>>> >>>>>>> >>>>>>> Quite... a lot better than the original proposal! >>>>>> >>>>>> Glad you both like it! >>>>>> >>>>>> When I do an allyesconfig build, I do get errors during the "CHECK" >>>>>> phase, when it is putting things into the usr/include in the build tree. >>>>>> I believe that this is because I am exposing different header files to >>>>>> the library-export scripts. The following patch silences some of them, >>>>>> but I am really out of my depth here. >>>>>> >>>>>> Sam, Jan, Michal, help? >>>>>> >>>>>> Thanx, Paul >>>>>> >>>>>> ------------------------------------------------------------------------ >>>>>> >>>>> >>>>> Easy to split rcupdate.h, hard to resolve the dependence problem. >>>>> >>>>> You can apply the next additional patch when you test: >>>> >>>> I am sure that you are quite correct. ;-) >>>> >>>> I am moving _rcu_read_lock() and _rcu_read_unlock() into >>>> include/linux/rcutree.h and include/linux/rcutiny.h, and I am sure that >>>> more pain will ensue. >>>> >>>> One thing I don't understand... How does is it helping to group the >>>> task_struct RCU-related fields into a structure? Is that generating >>>> better code on your platform due to smaller offsets or something? >> >> You don't like task_rcu_struct patch? I think it can make code clearer, >> and it can also check the code even when CONFIG_PREEMPT_RCU=n. >> >> For rcu_read_[un]lock(), it generates the same code, no better, no worse. >> >> It is just a cleanup patch, it is helpless for making rcu_read_[un]lock() inline, >> if you don't like it, I will give up it. > > I don't know that I feel strongly either way about it. It was necessary > with the integer-offset approach, but optional now. > >>>> Also, does your patchset address the CHECK warnings? >>> >>> I take it back... I applied the following patch on top of my earlier >>> one, and a defconfig x86 build completed without error. (Though I have >>> not tested the results of the build.) >>> >>> One possible difference -- I did this work on top of a recent Linus >>> git commit (b2a8b4b81966) rather than on top of my -rcu tree. Also, >>> I have not yet tried an allyesconfig build, which will no doubt locate >>> some more problems. >>> >>> Thanx, Paul >>> >> >> when defconfig or allyesconfig, CONFIG_PREEMPT=n and CONFIG_TREE_PREEMPT_RCU=n >> when you make them "y": >> >> In file included from include/linux/rcupdate.h:764:0, >> from include/linux/tracepoint.h:19, >> from include/linux/module.h:18, >> from include/linux/crypto.h:21, >> from arch/x86/kernel/asm-offsets.c:8: >> include/linux/rcutree.h:50:20: error: static declaration of ‘__rcu_read_lock’ follows non-static declaration >> include/linux/rcupdate.h:76:13: note: previous declaration of ‘__rcu_read_lock’ was here >> include/linux/rcutree.h:63:20: error: static declaration of ‘__rcu_read_unlock’ follows non-static declaration >> include/linux/rcupdate.h:77:13: note: previous declaration of ‘__rcu_read_unlock’ was here >> make[1]: *** [arch/x86/kernel/asm-offsets.s] Error 1 >> make: *** [prepare0] Error 2 > > Yep. I need to move the rcu_read_lock() APIs to follow the inclusion > of rcutree.h and rcutiny.h. Also add include of sched.h to rcutiny.h. > The code movement does bloat the patch a bit. But rcu_assign_pointer() > must precede the inclusion of rcutree.h and rcutiny.h, so it is not > possible to simply move the inclusions. See below. > > Thanx, Paul > sched.h still contains rcupdate.h after applied this patch. See my [PATCH 2/4] for more info. # make lib/is_single_threaded.o CHK include/linux/version.h CHK include/generated/utsrelease.h CALL scripts/checksyscalls.sh CC lib/is_single_threaded.o In file included from include/linux/rcupdate.h:639:0, from include/linux/rculist.h:10, from include/linux/sched.h:82, from lib/is_single_threaded.c:13: include/linux/rcutree.h: In function ‘__rcu_read_lock’: include/linux/rcutree.h:52:15: error: dereferencing pointer to incomplete type In file included from include/linux/rcupdate.h:639:0, from include/linux/rculist.h:10, from include/linux/sched.h:82, from lib/is_single_threaded.c:13: include/linux/rcutree.h: In function ‘__rcu_read_unlock’: include/linux/rcutree.h:68:5: error: dereferencing pointer to incomplete type include/linux/rcutree.h:70:7: error: dereferencing pointer to incomplete type include/linux/rcutree.h:71:46: error: dereferencing pointer to incomplete type include/linux/rcutree.h:71:78: error: dereferencing pointer to incomplete type include/linux/rcutree.h:71:6: warning: type defaults to ‘int’ in type name make[1]: *** [lib/is_single_threaded.o] Error 1 make: *** [lib/is_single_threaded.o] Error 2 ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-11 3:08 ` Lai Jiangshan @ 2011-04-11 5:12 ` Paul E. McKenney 2011-04-11 6:01 ` Lai Jiangshan ` (2 more replies) 0 siblings, 3 replies; 67+ messages in thread From: Paul E. McKenney @ 2011-04-11 5:12 UTC (permalink / raw) To: Lai Jiangshan Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On Mon, Apr 11, 2011 at 11:08:14AM +0800, Lai Jiangshan wrote: > On 04/08/2011 01:13 PM, Paul E. McKenney wrote: > > On Fri, Apr 08, 2011 at 09:26:16AM +0800, Lai Jiangshan wrote: > >> On 04/08/2011 12:26 AM, Paul E. McKenney wrote: > >>> On Thu, Apr 07, 2011 at 08:47:37AM -0700, Paul E. McKenney wrote: > >>>> On Thu, Apr 07, 2011 at 01:49:51PM +0800, Lai Jiangshan wrote: > >>>>> On 04/07/2011 08:30 AM, Paul E. McKenney wrote: > >>>>>> On Wed, Apr 06, 2011 at 02:27:39PM -0700, H. Peter Anvin wrote: > >>>>>>> On 04/06/2011 02:06 PM, Peter Zijlstra wrote: > >>>>>>>> On Wed, 2011-04-06 at 13:13 -0700, Paul E. McKenney wrote: > >>>>>>>>> And the following patch builds correctly for defconfig x86 builds, > >>>>>>>>> while allowing rcupdate.h to see the sched.h definitions as needed > >>>>>>>>> to inline rcu_read_lock() and rcu_read_unlock(). > >>>>>>>>> > >>>>>>>> Looks like an entirely reasonable patch to me ;-) > >>>>>>>> > >>>>>>> > >>>>>>> Quite... a lot better than the original proposal! > >>>>>> > >>>>>> Glad you both like it! > >>>>>> > >>>>>> When I do an allyesconfig build, I do get errors during the "CHECK" > >>>>>> phase, when it is putting things into the usr/include in the build tree. > >>>>>> I believe that this is because I am exposing different header files to > >>>>>> the library-export scripts. The following patch silences some of them, > >>>>>> but I am really out of my depth here. > >>>>>> > >>>>>> Sam, Jan, Michal, help? > >>>>>> > >>>>>> Thanx, Paul > >>>>>> > >>>>>> ------------------------------------------------------------------------ > >>>>>> > >>>>> > >>>>> Easy to split rcupdate.h, hard to resolve the dependence problem. > >>>>> > >>>>> You can apply the next additional patch when you test: > >>>> > >>>> I am sure that you are quite correct. ;-) > >>>> > >>>> I am moving _rcu_read_lock() and _rcu_read_unlock() into > >>>> include/linux/rcutree.h and include/linux/rcutiny.h, and I am sure that > >>>> more pain will ensue. > >>>> > >>>> One thing I don't understand... How does is it helping to group the > >>>> task_struct RCU-related fields into a structure? Is that generating > >>>> better code on your platform due to smaller offsets or something? > >> > >> You don't like task_rcu_struct patch? I think it can make code clearer, > >> and it can also check the code even when CONFIG_PREEMPT_RCU=n. > >> > >> For rcu_read_[un]lock(), it generates the same code, no better, no worse. > >> > >> It is just a cleanup patch, it is helpless for making rcu_read_[un]lock() inline, > >> if you don't like it, I will give up it. > > > > I don't know that I feel strongly either way about it. It was necessary > > with the integer-offset approach, but optional now. > > > >>>> Also, does your patchset address the CHECK warnings? > >>> > >>> I take it back... I applied the following patch on top of my earlier > >>> one, and a defconfig x86 build completed without error. (Though I have > >>> not tested the results of the build.) > >>> > >>> One possible difference -- I did this work on top of a recent Linus > >>> git commit (b2a8b4b81966) rather than on top of my -rcu tree. Also, > >>> I have not yet tried an allyesconfig build, which will no doubt locate > >>> some more problems. > >>> > >>> Thanx, Paul > >>> > >> > >> when defconfig or allyesconfig, CONFIG_PREEMPT=n and CONFIG_TREE_PREEMPT_RCU=n > >> when you make them "y": > >> > >> In file included from include/linux/rcupdate.h:764:0, > >> from include/linux/tracepoint.h:19, > >> from include/linux/module.h:18, > >> from include/linux/crypto.h:21, > >> from arch/x86/kernel/asm-offsets.c:8: > >> include/linux/rcutree.h:50:20: error: static declaration of ‘__rcu_read_lock’ follows non-static declaration > >> include/linux/rcupdate.h:76:13: note: previous declaration of ‘__rcu_read_lock’ was here > >> include/linux/rcutree.h:63:20: error: static declaration of ‘__rcu_read_unlock’ follows non-static declaration > >> include/linux/rcupdate.h:77:13: note: previous declaration of ‘__rcu_read_unlock’ was here > >> make[1]: *** [arch/x86/kernel/asm-offsets.s] Error 1 > >> make: *** [prepare0] Error 2 > > > > Yep. I need to move the rcu_read_lock() APIs to follow the inclusion > > of rcutree.h and rcutiny.h. Also add include of sched.h to rcutiny.h. > > The code movement does bloat the patch a bit. But rcu_assign_pointer() > > must precede the inclusion of rcutree.h and rcutiny.h, so it is not > > possible to simply move the inclusions. See below. > > > > Thanx, Paul > > > > sched.h still contains rcupdate.h after applied this patch. Then we are not in sync -- sched.h does not include rcupdate.h in my tree. It instead gets the struct rcu_head definition from include/linux/types.h. See below for a consolidated patch. Thanx, Paul > See my [PATCH 2/4] for more info. > > > # make lib/is_single_threaded.o > CHK include/linux/version.h > CHK include/generated/utsrelease.h > CALL scripts/checksyscalls.sh > CC lib/is_single_threaded.o > In file included from include/linux/rcupdate.h:639:0, > from include/linux/rculist.h:10, > from include/linux/sched.h:82, > from lib/is_single_threaded.c:13: > include/linux/rcutree.h: In function ‘__rcu_read_lock’: > include/linux/rcutree.h:52:15: error: dereferencing pointer to incomplete type > In file included from include/linux/rcupdate.h:639:0, > from include/linux/rculist.h:10, > from include/linux/sched.h:82, > from lib/is_single_threaded.c:13: > include/linux/rcutree.h: In function ‘__rcu_read_unlock’: > include/linux/rcutree.h:68:5: error: dereferencing pointer to incomplete type > include/linux/rcutree.h:70:7: error: dereferencing pointer to incomplete type > include/linux/rcutree.h:71:46: error: dereferencing pointer to incomplete type > include/linux/rcutree.h:71:78: error: dereferencing pointer to incomplete type > include/linux/rcutree.h:71:6: warning: type defaults to ‘int’ in type name > make[1]: *** [lib/is_single_threaded.o] Error 1 > make: *** [lib/is_single_threaded.o] Error 2 drivers/scsi/scsi_sysctl.c | 1 + fs/proc/proc_sysctl.c | 1 + include/linux/kernel.h | 3 +++ include/linux/pid.h | 2 +- include/linux/rcupdate.h | 29 +++++++++-------------------- include/linux/rcutiny.h | 35 +++++++++++++++++++++++++++++++++++ include/linux/rcutree.h | 37 +++++++++++++++++++++++++++++++++++++ include/linux/sched.h | 10 ++++------ include/linux/sem.h | 2 +- include/linux/soundcard.h | 2 ++ include/linux/sysctl.h | 5 +++-- include/linux/types.h | 10 ++++++++++ kernel/pid_namespace.c | 2 ++ kernel/rcutiny_plugin.h | 38 ++------------------------------------ kernel/rcutree_plugin.h | 38 ++------------------------------------ kernel/sysctl_binary.c | 1 + kernel/sysctl_check.c | 1 + net/core/sysctl_net_core.c | 1 + net/dccp/sysctl.c | 1 + net/ipv6/sysctl_net_ipv6.c | 1 + net/irda/irsysctl.c | 1 + net/phonet/sysctl.c | 1 + net/rds/ib_sysctl.c | 1 + net/rds/iw_sysctl.c | 1 + net/rds/sysctl.c | 1 + net/sctp/sysctl.c | 1 + net/sunrpc/sysctl.c | 1 + net/unix/sysctl_net_unix.c | 1 + net/xfrm/xfrm_sysctl.c | 1 + 29 files changed, 127 insertions(+), 102 deletions(-) diff --git a/drivers/scsi/scsi_sysctl.c b/drivers/scsi/scsi_sysctl.c index 2b6b93f..ecce1c6 100644 --- a/drivers/scsi/scsi_sysctl.c +++ b/drivers/scsi/scsi_sysctl.c @@ -6,6 +6,7 @@ #include <linux/errno.h> #include <linux/init.h> #include <linux/kernel.h> +#include <linux/key.h> #include <linux/sysctl.h> #include "scsi_logging.h" diff --git a/fs/proc/proc_sysctl.c b/fs/proc/proc_sysctl.c index f50133c..180bf3e 100644 --- a/fs/proc/proc_sysctl.c +++ b/fs/proc/proc_sysctl.c @@ -2,6 +2,7 @@ * /proc/sys support */ #include <linux/init.h> +#include <linux/key.h> #include <linux/sysctl.h> #include <linux/proc_fs.h> #include <linux/security.h> diff --git a/include/linux/kernel.h b/include/linux/kernel.h index 00cec4d..a243c13 100644 --- a/include/linux/kernel.h +++ b/include/linux/kernel.h @@ -648,6 +648,8 @@ struct sysinfo { #define BUILD_BUG_ON_ZERO(e) (sizeof(struct { int:-!!(e); })) #define BUILD_BUG_ON_NULL(e) ((void *)sizeof(struct { int:-!!(e); })) +#ifdef __KERNEL__ + /** * BUILD_BUG_ON - break compile if a condition is true. * @condition: the condition which the compiler should know is false. @@ -673,6 +675,7 @@ extern int __build_bug_on_failed; if (condition) __build_bug_on_failed = 1; \ } while(0) #endif +#endif /* __KERNEL__ */ /* Trap pasters of __FUNCTION__ at compile-time */ #define __FUNCTION__ (__func__) diff --git a/include/linux/pid.h b/include/linux/pid.h index efceda0..3c5719b 100644 --- a/include/linux/pid.h +++ b/include/linux/pid.h @@ -1,7 +1,7 @@ #ifndef _LINUX_PID_H #define _LINUX_PID_H -#include <linux/rcupdate.h> +#include <linux/types.h> enum pid_type { diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index ff422d2..d009ebf 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -33,6 +33,7 @@ #ifndef __LINUX_RCUPDATE_H #define __LINUX_RCUPDATE_H +#include <linux/types.h> #include <linux/cache.h> #include <linux/spinlock.h> #include <linux/threads.h> @@ -52,16 +53,6 @@ extern int rcutorture_runnable; /* for sysctl */ #define ULONG_CMP_GE(a, b) (ULONG_MAX / 2 >= (a) - (b)) #define ULONG_CMP_LT(a, b) (ULONG_MAX / 2 < (a) - (b)) -/** - * struct rcu_head - callback structure for use with RCU - * @next: next update requests in a list - * @func: actual update function to call after the grace period. - */ -struct rcu_head { - struct rcu_head *next; - void (*func)(struct rcu_head *head); -}; - /* Exported common interfaces */ extern void call_rcu_sched(struct rcu_head *head, void (*func)(struct rcu_head *rcu)); @@ -82,8 +73,6 @@ static inline void __rcu_read_unlock_bh(void) #ifdef CONFIG_PREEMPT_RCU -extern void __rcu_read_lock(void); -extern void __rcu_read_unlock(void); void synchronize_rcu(void); /* @@ -141,14 +130,6 @@ static inline void rcu_exit_nohz(void) #endif /* #else #ifdef CONFIG_NO_HZ */ -#if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) -#include <linux/rcutree.h> -#elif defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU) -#include <linux/rcutiny.h> -#else -#error "Unknown RCU implementation specified to kernel configuration" -#endif - /* * init_rcu_head_on_stack()/destroy_rcu_head_on_stack() are needed for dynamic * initialization and destruction of rcu_head on the stack. rcu_head structures @@ -797,4 +778,12 @@ static inline void debug_rcu_head_unqueue(struct rcu_head *head) } #endif /* #else !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ +#if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) +#include <linux/rcutree.h> +#elif defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU) +#include <linux/rcutiny.h> +#else +#error "Unknown RCU implementation specified to kernel configuration" +#endif + #endif /* __LINUX_RCUPDATE_H */ diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h index 30ebd7c..167fb19 100644 --- a/include/linux/rcutiny.h +++ b/include/linux/rcutiny.h @@ -26,6 +26,7 @@ #define __LINUX_TINY_H #include <linux/cache.h> +#include <linux/sched.h> static inline void rcu_init(void) { @@ -47,6 +48,40 @@ static inline void rcu_barrier(void) void rcu_barrier(void); void synchronize_rcu_expedited(void); +void rcu_read_unlock_special(struct task_struct *t); + +/* + * Tiny-preemptible RCU implementation for rcu_read_lock(). + * Just increment ->rcu_read_lock_nesting, shared state will be updated + * if we block. + */ +static inline void __rcu_read_lock(void) +{ + current->rcu_read_lock_nesting++; + barrier(); +} + +/* + * Tiny-preemptible RCU implementation for rcu_read_unlock(). + * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost + * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then + * invoke rcu_read_unlock_special() to clean up after a context switch + * in an RCU read-side critical section and other special cases. + */ +static inline void __rcu_read_unlock(void) +{ + struct task_struct *t = current; + + barrier(); + --t->rcu_read_lock_nesting; + barrier(); /* decrement before load of ->rcu_read_unlock_special */ + if (t->rcu_read_lock_nesting == 0 && + unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) + rcu_read_unlock_special(t); +#ifdef CONFIG_PROVE_LOCKING + WARN_ON_ONCE(t->rcu_read_lock_nesting < 0); +#endif /* #ifdef CONFIG_PROVE_LOCKING */ +} #endif /* #else #ifdef CONFIG_TINY_RCU */ diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h index 3a93348..012cc1c 100644 --- a/include/linux/rcutree.h +++ b/include/linux/rcutree.h @@ -30,6 +30,8 @@ #ifndef __LINUX_RCUTREE_H #define __LINUX_RCUTREE_H +#include <linux/sched.h> + extern void rcu_init(void); extern void rcu_note_context_switch(int cpu); extern int rcu_needs_cpu(int cpu); @@ -37,6 +39,41 @@ extern void rcu_cpu_stall_reset(void); #ifdef CONFIG_TREE_PREEMPT_RCU +extern void rcu_read_unlock_special(struct task_struct *t); + +/* + * Tree-preemptable RCU implementation for rcu_read_lock(). + * Just increment ->rcu_read_lock_nesting, shared state will be updated + * if we block. + */ +static inline void __rcu_read_lock(void) +{ + current->rcu_read_lock_nesting++; + barrier(); +} + +/* + * Tree-preemptable RCU implementation for rcu_read_unlock(). + * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost + * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then + * invoke rcu_read_unlock_special() to clean up after a context switch + * in an RCU read-side critical section and other special cases. + */ +static inline void __rcu_read_unlock(void) +{ + struct task_struct *t = current; + + barrier(); + --t->rcu_read_lock_nesting; + barrier(); /* decrement before load of ->rcu_read_unlock_special */ + if (t->rcu_read_lock_nesting == 0 && + unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) + rcu_read_unlock_special(t); +#ifdef CONFIG_PROVE_LOCKING + WARN_ON_ONCE(ACCESS_ONCE(t->rcu_read_lock_nesting) < 0); +#endif /* #ifdef CONFIG_PROVE_LOCKING */ +} + extern void exit_rcu(void); #else /* #ifdef CONFIG_TREE_PREEMPT_RCU */ diff --git a/include/linux/sched.h b/include/linux/sched.h index 83bd2e2..30a4444 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -78,7 +78,7 @@ struct sched_param { #include <linux/topology.h> #include <linux/proportions.h> #include <linux/seccomp.h> -#include <linux/rcupdate.h> +#include <linux/types.h> #include <linux/rculist.h> #include <linux/rtmutex.h> @@ -2241,11 +2241,9 @@ int same_thread_group(struct task_struct *p1, struct task_struct *p2) return p1->tgid == p2->tgid; } -static inline struct task_struct *next_thread(const struct task_struct *p) -{ - return list_entry_rcu(p->thread_group.next, - struct task_struct, thread_group); -} +/* Avoid #include hell for inlining rcu_read_lock(). */ +#define next_thread(p) \ + list_entry_rcu((p)->thread_group.next, struct task_struct, thread_group) static inline int thread_group_empty(struct task_struct *p) { diff --git a/include/linux/sem.h b/include/linux/sem.h index f2961af..8489a1f 100644 --- a/include/linux/sem.h +++ b/include/linux/sem.h @@ -78,7 +78,7 @@ struct seminfo { #ifdef __KERNEL__ #include <asm/atomic.h> -#include <linux/rcupdate.h> +#include <linux/types.h> #include <linux/cache.h> struct task_struct; diff --git a/include/linux/soundcard.h b/include/linux/soundcard.h index 1904afe..f99c32f 100644 --- a/include/linux/soundcard.h +++ b/include/linux/soundcard.h @@ -1064,7 +1064,9 @@ typedef struct mixer_vol_table { */ #define SEQ_DECLAREBUF() SEQ_USE_EXTBUF() +#ifdef __KERNEL__ void seqbuf_dump(void); /* This function must be provided by programs */ +#endif #define SEQ_PM_DEFINES int __foo_bar___ diff --git a/include/linux/sysctl.h b/include/linux/sysctl.h index 11684d9..be80a59 100644 --- a/include/linux/sysctl.h +++ b/include/linux/sysctl.h @@ -19,6 +19,8 @@ **************************************************************** */ +#include <linux/key.h> + #ifndef _LINUX_SYSCTL_H #define _LINUX_SYSCTL_H @@ -1012,8 +1014,7 @@ extern int proc_do_large_bitmap(struct ctl_table *, int, */ /* A sysctl table is an array of struct ctl_table: */ -struct ctl_table -{ +struct ctl_table { const char *procname; /* Text ID for /proc/sys, or zero */ void *data; int maxlen; diff --git a/include/linux/types.h b/include/linux/types.h index 176da8c..868ef8b 100644 --- a/include/linux/types.h +++ b/include/linux/types.h @@ -231,6 +231,16 @@ struct hlist_node { struct hlist_node *next, **pprev; }; +/** + * struct rcu_head - callback structure for use with RCU + * @next: next update requests in a list + * @func: actual update function to call after the grace period. + */ +struct rcu_head { + struct rcu_head *next; + void (*func)(struct rcu_head *head); +}; + struct ustat { __kernel_daddr_t f_tfree; __kernel_ino_t f_tinode; diff --git a/kernel/pid_namespace.c b/kernel/pid_namespace.c index e9c9adc..bfa75df 100644 --- a/kernel/pid_namespace.c +++ b/kernel/pid_namespace.c @@ -8,6 +8,8 @@ * */ +#include <linux/types.h> +#include <asm/atomic.h> #include <linux/pid.h> #include <linux/pid_namespace.h> #include <linux/syscalls.h> diff --git a/kernel/rcutiny_plugin.h b/kernel/rcutiny_plugin.h index 3cb8e36..d0e1ac3 100644 --- a/kernel/rcutiny_plugin.h +++ b/kernel/rcutiny_plugin.h @@ -520,23 +520,11 @@ void rcu_preempt_note_context_switch(void) } /* - * Tiny-preemptible RCU implementation for rcu_read_lock(). - * Just increment ->rcu_read_lock_nesting, shared state will be updated - * if we block. - */ -void __rcu_read_lock(void) -{ - current->rcu_read_lock_nesting++; - barrier(); /* needed if we ever invoke rcu_read_lock in rcutiny.c */ -} -EXPORT_SYMBOL_GPL(__rcu_read_lock); - -/* * Handle special cases during rcu_read_unlock(), such as needing to * notify RCU core processing or task having blocked during the RCU * read-side critical section. */ -static void rcu_read_unlock_special(struct task_struct *t) +void rcu_read_unlock_special(struct task_struct *t) { int empty; int empty_exp; @@ -616,29 +604,7 @@ static void rcu_read_unlock_special(struct task_struct *t) #endif /* #ifdef CONFIG_RCU_BOOST */ local_irq_restore(flags); } - -/* - * Tiny-preemptible RCU implementation for rcu_read_unlock(). - * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost - * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then - * invoke rcu_read_unlock_special() to clean up after a context switch - * in an RCU read-side critical section and other special cases. - */ -void __rcu_read_unlock(void) -{ - struct task_struct *t = current; - - barrier(); /* needed if we ever invoke rcu_read_unlock in rcutiny.c */ - --t->rcu_read_lock_nesting; - barrier(); /* decrement before load of ->rcu_read_unlock_special */ - if (t->rcu_read_lock_nesting == 0 && - unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) - rcu_read_unlock_special(t); -#ifdef CONFIG_PROVE_LOCKING - WARN_ON_ONCE(t->rcu_read_lock_nesting < 0); -#endif /* #ifdef CONFIG_PROVE_LOCKING */ -} -EXPORT_SYMBOL_GPL(__rcu_read_unlock); +EXPORT_SYMBOL_GPL(rcu_read_unlock_special); /* * Check for a quiescent state from the current CPU. When a task blocks, diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h index a363871..4b27afd 100644 --- a/kernel/rcutree_plugin.h +++ b/kernel/rcutree_plugin.h @@ -196,18 +196,6 @@ static void rcu_preempt_note_context_switch(int cpu) } /* - * Tree-preemptable RCU implementation for rcu_read_lock(). - * Just increment ->rcu_read_lock_nesting, shared state will be updated - * if we block. - */ -void __rcu_read_lock(void) -{ - current->rcu_read_lock_nesting++; - barrier(); /* needed if we ever invoke rcu_read_lock in rcutree.c */ -} -EXPORT_SYMBOL_GPL(__rcu_read_lock); - -/* * Check for preempted RCU readers blocking the current grace period * for the specified rcu_node structure. If the caller needs a reliable * answer, it must hold the rcu_node's ->lock. @@ -261,7 +249,7 @@ static void rcu_report_unblock_qs_rnp(struct rcu_node *rnp, unsigned long flags) * notify RCU core processing or task having blocked during the RCU * read-side critical section. */ -static void rcu_read_unlock_special(struct task_struct *t) +void rcu_read_unlock_special(struct task_struct *t) { int empty; int empty_exp; @@ -332,29 +320,7 @@ static void rcu_read_unlock_special(struct task_struct *t) local_irq_restore(flags); } } - -/* - * Tree-preemptable RCU implementation for rcu_read_unlock(). - * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost - * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then - * invoke rcu_read_unlock_special() to clean up after a context switch - * in an RCU read-side critical section and other special cases. - */ -void __rcu_read_unlock(void) -{ - struct task_struct *t = current; - - barrier(); /* needed if we ever invoke rcu_read_unlock in rcutree.c */ - --t->rcu_read_lock_nesting; - barrier(); /* decrement before load of ->rcu_read_unlock_special */ - if (t->rcu_read_lock_nesting == 0 && - unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) - rcu_read_unlock_special(t); -#ifdef CONFIG_PROVE_LOCKING - WARN_ON_ONCE(ACCESS_ONCE(t->rcu_read_lock_nesting) < 0); -#endif /* #ifdef CONFIG_PROVE_LOCKING */ -} -EXPORT_SYMBOL_GPL(__rcu_read_unlock); +EXPORT_SYMBOL_GPL(rcu_read_unlock_special); #ifdef CONFIG_RCU_CPU_STALL_DETECTOR diff --git a/kernel/sysctl_binary.c b/kernel/sysctl_binary.c index 3b8e028..58097b7 100644 --- a/kernel/sysctl_binary.c +++ b/kernel/sysctl_binary.c @@ -1,4 +1,5 @@ #include <linux/stat.h> +#include <linux/key.h> #include <linux/sysctl.h> #include "../fs/xfs/linux-2.6/xfs_sysctl.h" #include <linux/sunrpc/debug.h> diff --git a/kernel/sysctl_check.c b/kernel/sysctl_check.c index 4e4932a..a8a0df5 100644 --- a/kernel/sysctl_check.c +++ b/kernel/sysctl_check.c @@ -1,4 +1,5 @@ #include <linux/stat.h> +#include <linux/key.h> #include <linux/sysctl.h> #include "../fs/xfs/linux-2.6/xfs_sysctl.h" #include <linux/sunrpc/debug.h> diff --git a/net/core/sysctl_net_core.c b/net/core/sysctl_net_core.c index 385b609..528b975 100644 --- a/net/core/sysctl_net_core.c +++ b/net/core/sysctl_net_core.c @@ -6,6 +6,7 @@ */ #include <linux/mm.h> +#include <linux/key.h> #include <linux/sysctl.h> #include <linux/module.h> #include <linux/socket.h> diff --git a/net/dccp/sysctl.c b/net/dccp/sysctl.c index 4234882..bf609bc 100644 --- a/net/dccp/sysctl.c +++ b/net/dccp/sysctl.c @@ -10,6 +10,7 @@ */ #include <linux/mm.h> +#include <linux/key.h> #include <linux/sysctl.h> #include "dccp.h" #include "feat.h" diff --git a/net/ipv6/sysctl_net_ipv6.c b/net/ipv6/sysctl_net_ipv6.c index 6dcf5e7..97a3ce0 100644 --- a/net/ipv6/sysctl_net_ipv6.c +++ b/net/ipv6/sysctl_net_ipv6.c @@ -6,6 +6,7 @@ */ #include <linux/mm.h> +#include <linux/key.h> #include <linux/sysctl.h> #include <linux/in6.h> #include <linux/ipv6.h> diff --git a/net/irda/irsysctl.c b/net/irda/irsysctl.c index d0b70da..11aaf12 100644 --- a/net/irda/irsysctl.c +++ b/net/irda/irsysctl.c @@ -25,6 +25,7 @@ #include <linux/mm.h> #include <linux/ctype.h> +#include <linux/key.h> #include <linux/sysctl.h> #include <linux/init.h> diff --git a/net/phonet/sysctl.c b/net/phonet/sysctl.c index cea1c7d..2c2d479 100644 --- a/net/phonet/sysctl.c +++ b/net/phonet/sysctl.c @@ -23,6 +23,7 @@ */ #include <linux/seqlock.h> +#include <linux/key.h> #include <linux/sysctl.h> #include <linux/errno.h> #include <linux/init.h> diff --git a/net/rds/ib_sysctl.c b/net/rds/ib_sysctl.c index 1253b00..2f9a1a7 100644 --- a/net/rds/ib_sysctl.c +++ b/net/rds/ib_sysctl.c @@ -31,6 +31,7 @@ * */ #include <linux/kernel.h> +#include <linux/key.h> #include <linux/sysctl.h> #include <linux/proc_fs.h> diff --git a/net/rds/iw_sysctl.c b/net/rds/iw_sysctl.c index e2e4717..8c4051b 100644 --- a/net/rds/iw_sysctl.c +++ b/net/rds/iw_sysctl.c @@ -31,6 +31,7 @@ * */ #include <linux/kernel.h> +#include <linux/key.h> #include <linux/sysctl.h> #include <linux/proc_fs.h> diff --git a/net/rds/sysctl.c b/net/rds/sysctl.c index 25ad0c7..46d2d1d 100644 --- a/net/rds/sysctl.c +++ b/net/rds/sysctl.c @@ -31,6 +31,7 @@ * */ #include <linux/kernel.h> +#include <linux/key.h> #include <linux/sysctl.h> #include <linux/proc_fs.h> diff --git a/net/sctp/sysctl.c b/net/sctp/sysctl.c index 50cb57f..c2903ce 100644 --- a/net/sctp/sysctl.c +++ b/net/sctp/sysctl.c @@ -43,6 +43,7 @@ #include <net/sctp/structs.h> #include <net/sctp/sctp.h> +#include <linux/key.h> #include <linux/sysctl.h> static int zero = 0; diff --git a/net/sunrpc/sysctl.c b/net/sunrpc/sysctl.c index e65dcc6..1429601 100644 --- a/net/sunrpc/sysctl.c +++ b/net/sunrpc/sysctl.c @@ -11,6 +11,7 @@ #include <linux/linkage.h> #include <linux/ctype.h> #include <linux/fs.h> +#include <linux/key.h> #include <linux/sysctl.h> #include <linux/module.h> diff --git a/net/unix/sysctl_net_unix.c b/net/unix/sysctl_net_unix.c index 397cffe..6d2766e 100644 --- a/net/unix/sysctl_net_unix.c +++ b/net/unix/sysctl_net_unix.c @@ -11,6 +11,7 @@ #include <linux/mm.h> #include <linux/slab.h> +#include <linux/key.h> #include <linux/sysctl.h> #include <net/af_unix.h> diff --git a/net/xfrm/xfrm_sysctl.c b/net/xfrm/xfrm_sysctl.c index 05640bc..a50e4bf 100644 --- a/net/xfrm/xfrm_sysctl.c +++ b/net/xfrm/xfrm_sysctl.c @@ -1,3 +1,4 @@ +#include <linux/key.h> #include <linux/sysctl.h> #include <linux/slab.h> #include <net/net_namespace.h> ^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-11 5:12 ` Paul E. McKenney @ 2011-04-11 6:01 ` Lai Jiangshan 2011-04-11 8:31 ` Lai Jiangshan 2011-04-11 8:33 ` Lai Jiangshan 2 siblings, 0 replies; 67+ messages in thread From: Lai Jiangshan @ 2011-04-11 6:01 UTC (permalink / raw) To: paulmck Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On 04/11/2011 01:12 PM, Paul E. McKenney wrote: > On Mon, Apr 11, 2011 at 11:08:14AM +0800, Lai Jiangshan wrote: >> On 04/08/2011 01:13 PM, Paul E. McKenney wrote: >>> On Fri, Apr 08, 2011 at 09:26:16AM +0800, Lai Jiangshan wrote: >>>> On 04/08/2011 12:26 AM, Paul E. McKenney wrote: >>>>> On Thu, Apr 07, 2011 at 08:47:37AM -0700, Paul E. McKenney wrote: >>>>>> On Thu, Apr 07, 2011 at 01:49:51PM +0800, Lai Jiangshan wrote: >>>>>>> On 04/07/2011 08:30 AM, Paul E. McKenney wrote: >>>>>>>> On Wed, Apr 06, 2011 at 02:27:39PM -0700, H. Peter Anvin wrote: >>>>>>>>> On 04/06/2011 02:06 PM, Peter Zijlstra wrote: >>>>>>>>>> On Wed, 2011-04-06 at 13:13 -0700, Paul E. McKenney wrote: >>>>>>>>>>> And the following patch builds correctly for defconfig x86 builds, >>>>>>>>>>> while allowing rcupdate.h to see the sched.h definitions as needed >>>>>>>>>>> to inline rcu_read_lock() and rcu_read_unlock(). >>>>>>>>>>> >>>>>>>>>> Looks like an entirely reasonable patch to me ;-) >>>>>>>>>> >>>>>>>>> >>>>>>>>> Quite... a lot better than the original proposal! >>>>>>>> >>>>>>>> Glad you both like it! >>>>>>>> >>>>>>>> When I do an allyesconfig build, I do get errors during the "CHECK" >>>>>>>> phase, when it is putting things into the usr/include in the build tree. >>>>>>>> I believe that this is because I am exposing different header files to >>>>>>>> the library-export scripts. The following patch silences some of them, >>>>>>>> but I am really out of my depth here. >>>>>>>> >>>>>>>> Sam, Jan, Michal, help? >>>>>>>> >>>>>>>> Thanx, Paul >>>>>>>> >>>>>>>> ------------------------------------------------------------------------ >>>>>>>> >>>>>>> >>>>>>> Easy to split rcupdate.h, hard to resolve the dependence problem. >>>>>>> >>>>>>> You can apply the next additional patch when you test: >>>>>> >>>>>> I am sure that you are quite correct. ;-) >>>>>> >>>>>> I am moving _rcu_read_lock() and _rcu_read_unlock() into >>>>>> include/linux/rcutree.h and include/linux/rcutiny.h, and I am sure that >>>>>> more pain will ensue. >>>>>> >>>>>> One thing I don't understand... How does is it helping to group the >>>>>> task_struct RCU-related fields into a structure? Is that generating >>>>>> better code on your platform due to smaller offsets or something? >>>> >>>> You don't like task_rcu_struct patch? I think it can make code clearer, >>>> and it can also check the code even when CONFIG_PREEMPT_RCU=n. >>>> >>>> For rcu_read_[un]lock(), it generates the same code, no better, no worse. >>>> >>>> It is just a cleanup patch, it is helpless for making rcu_read_[un]lock() inline, >>>> if you don't like it, I will give up it. >>> >>> I don't know that I feel strongly either way about it. It was necessary >>> with the integer-offset approach, but optional now. >>> >>>>>> Also, does your patchset address the CHECK warnings? >>>>> >>>>> I take it back... I applied the following patch on top of my earlier >>>>> one, and a defconfig x86 build completed without error. (Though I have >>>>> not tested the results of the build.) >>>>> >>>>> One possible difference -- I did this work on top of a recent Linus >>>>> git commit (b2a8b4b81966) rather than on top of my -rcu tree. Also, >>>>> I have not yet tried an allyesconfig build, which will no doubt locate >>>>> some more problems. >>>>> >>>>> Thanx, Paul >>>>> >>>> >>>> when defconfig or allyesconfig, CONFIG_PREEMPT=n and CONFIG_TREE_PREEMPT_RCU=n >>>> when you make them "y": >>>> >>>> In file included from include/linux/rcupdate.h:764:0, >>>> from include/linux/tracepoint.h:19, >>>> from include/linux/module.h:18, >>>> from include/linux/crypto.h:21, >>>> from arch/x86/kernel/asm-offsets.c:8: >>>> include/linux/rcutree.h:50:20: error: static declaration of ‘__rcu_read_lock’ follows non-static declaration >>>> include/linux/rcupdate.h:76:13: note: previous declaration of ‘__rcu_read_lock’ was here >>>> include/linux/rcutree.h:63:20: error: static declaration of ‘__rcu_read_unlock’ follows non-static declaration >>>> include/linux/rcupdate.h:77:13: note: previous declaration of ‘__rcu_read_unlock’ was here >>>> make[1]: *** [arch/x86/kernel/asm-offsets.s] Error 1 >>>> make: *** [prepare0] Error 2 >>> >>> Yep. I need to move the rcu_read_lock() APIs to follow the inclusion >>> of rcutree.h and rcutiny.h. Also add include of sched.h to rcutiny.h. >>> The code movement does bloat the patch a bit. But rcu_assign_pointer() >>> must precede the inclusion of rcutree.h and rcutiny.h, so it is not >>> possible to simply move the inclusions. See below. >>> >>> Thanx, Paul >>> >> >> sched.h still contains rcupdate.h after applied this patch. > > Then we are not in sync -- sched.h does not include rcupdate.h in my tree. > It instead gets the struct rcu_head definition from include/linux/types.h. > See below for a consolidated patch. > > Thanx, Paul > >> See my [PATCH 2/4] for more info. >> >> >> # make lib/is_single_threaded.o >> CHK include/linux/version.h >> CHK include/generated/utsrelease.h >> CALL scripts/checksyscalls.sh >> CC lib/is_single_threaded.o >> In file included from include/linux/rcupdate.h:639:0, >> from include/linux/rculist.h:10, >> from include/linux/sched.h:82, >> from lib/is_single_threaded.c:13: >> include/linux/rcutree.h: In function ‘__rcu_read_lock’: >> include/linux/rcutree.h:52:15: error: dereferencing pointer to incomplete type >> In file included from include/linux/rcupdate.h:639:0, >> from include/linux/rculist.h:10, >> from include/linux/sched.h:82, >> from lib/is_single_threaded.c:13: >> include/linux/rcutree.h: In function ‘__rcu_read_unlock’: >> include/linux/rcutree.h:68:5: error: dereferencing pointer to incomplete type >> include/linux/rcutree.h:70:7: error: dereferencing pointer to incomplete type >> include/linux/rcutree.h:71:46: error: dereferencing pointer to incomplete type >> include/linux/rcutree.h:71:78: error: dereferencing pointer to incomplete type >> include/linux/rcutree.h:71:6: warning: type defaults to ‘int’ in type name >> make[1]: *** [lib/is_single_threaded.o] Error 1 >> make: *** [lib/is_single_threaded.o] Error 2 sched.h still contains rcupdate.h after applied this patch. so it still fails. Is your .config with CONFIG_TREE_PREEMPT_RCU=y ? > > drivers/scsi/scsi_sysctl.c | 1 + > fs/proc/proc_sysctl.c | 1 + > include/linux/kernel.h | 3 +++ > include/linux/pid.h | 2 +- > include/linux/rcupdate.h | 29 +++++++++-------------------- > include/linux/rcutiny.h | 35 +++++++++++++++++++++++++++++++++++ > include/linux/rcutree.h | 37 +++++++++++++++++++++++++++++++++++++ > include/linux/sched.h | 10 ++++------ > include/linux/sem.h | 2 +- > include/linux/soundcard.h | 2 ++ > include/linux/sysctl.h | 5 +++-- > include/linux/types.h | 10 ++++++++++ > kernel/pid_namespace.c | 2 ++ > kernel/rcutiny_plugin.h | 38 ++------------------------------------ > kernel/rcutree_plugin.h | 38 ++------------------------------------ > kernel/sysctl_binary.c | 1 + > kernel/sysctl_check.c | 1 + > net/core/sysctl_net_core.c | 1 + > net/dccp/sysctl.c | 1 + > net/ipv6/sysctl_net_ipv6.c | 1 + > net/irda/irsysctl.c | 1 + > net/phonet/sysctl.c | 1 + > net/rds/ib_sysctl.c | 1 + > net/rds/iw_sysctl.c | 1 + > net/rds/sysctl.c | 1 + > net/sctp/sysctl.c | 1 + > net/sunrpc/sysctl.c | 1 + > net/unix/sysctl_net_unix.c | 1 + > net/xfrm/xfrm_sysctl.c | 1 + > 29 files changed, 127 insertions(+), 102 deletions(-) > > diff --git a/drivers/scsi/scsi_sysctl.c b/drivers/scsi/scsi_sysctl.c > index 2b6b93f..ecce1c6 100644 > --- a/drivers/scsi/scsi_sysctl.c > +++ b/drivers/scsi/scsi_sysctl.c > @@ -6,6 +6,7 @@ > #include <linux/errno.h> > #include <linux/init.h> > #include <linux/kernel.h> > +#include <linux/key.h> > #include <linux/sysctl.h> > > #include "scsi_logging.h" > diff --git a/fs/proc/proc_sysctl.c b/fs/proc/proc_sysctl.c > index f50133c..180bf3e 100644 > --- a/fs/proc/proc_sysctl.c > +++ b/fs/proc/proc_sysctl.c > @@ -2,6 +2,7 @@ > * /proc/sys support > */ > #include <linux/init.h> > +#include <linux/key.h> > #include <linux/sysctl.h> > #include <linux/proc_fs.h> > #include <linux/security.h> > diff --git a/include/linux/kernel.h b/include/linux/kernel.h > index 00cec4d..a243c13 100644 > --- a/include/linux/kernel.h > +++ b/include/linux/kernel.h > @@ -648,6 +648,8 @@ struct sysinfo { > #define BUILD_BUG_ON_ZERO(e) (sizeof(struct { int:-!!(e); })) > #define BUILD_BUG_ON_NULL(e) ((void *)sizeof(struct { int:-!!(e); })) > > +#ifdef __KERNEL__ > + > /** > * BUILD_BUG_ON - break compile if a condition is true. > * @condition: the condition which the compiler should know is false. > @@ -673,6 +675,7 @@ extern int __build_bug_on_failed; > if (condition) __build_bug_on_failed = 1; \ > } while(0) > #endif > +#endif /* __KERNEL__ */ > > /* Trap pasters of __FUNCTION__ at compile-time */ > #define __FUNCTION__ (__func__) > diff --git a/include/linux/pid.h b/include/linux/pid.h > index efceda0..3c5719b 100644 > --- a/include/linux/pid.h > +++ b/include/linux/pid.h > @@ -1,7 +1,7 @@ > #ifndef _LINUX_PID_H > #define _LINUX_PID_H > > -#include <linux/rcupdate.h> > +#include <linux/types.h> > > enum pid_type > { > diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h > index ff422d2..d009ebf 100644 > --- a/include/linux/rcupdate.h > +++ b/include/linux/rcupdate.h > @@ -33,6 +33,7 @@ > #ifndef __LINUX_RCUPDATE_H > #define __LINUX_RCUPDATE_H > > +#include <linux/types.h> > #include <linux/cache.h> > #include <linux/spinlock.h> > #include <linux/threads.h> > @@ -52,16 +53,6 @@ extern int rcutorture_runnable; /* for sysctl */ > #define ULONG_CMP_GE(a, b) (ULONG_MAX / 2 >= (a) - (b)) > #define ULONG_CMP_LT(a, b) (ULONG_MAX / 2 < (a) - (b)) > > -/** > - * struct rcu_head - callback structure for use with RCU > - * @next: next update requests in a list > - * @func: actual update function to call after the grace period. > - */ > -struct rcu_head { > - struct rcu_head *next; > - void (*func)(struct rcu_head *head); > -}; > - > /* Exported common interfaces */ > extern void call_rcu_sched(struct rcu_head *head, > void (*func)(struct rcu_head *rcu)); > @@ -82,8 +73,6 @@ static inline void __rcu_read_unlock_bh(void) > > #ifdef CONFIG_PREEMPT_RCU > > -extern void __rcu_read_lock(void); > -extern void __rcu_read_unlock(void); > void synchronize_rcu(void); > > /* > @@ -141,14 +130,6 @@ static inline void rcu_exit_nohz(void) > > #endif /* #else #ifdef CONFIG_NO_HZ */ > > -#if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) > -#include <linux/rcutree.h> > -#elif defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU) > -#include <linux/rcutiny.h> > -#else > -#error "Unknown RCU implementation specified to kernel configuration" > -#endif > - > /* > * init_rcu_head_on_stack()/destroy_rcu_head_on_stack() are needed for dynamic > * initialization and destruction of rcu_head on the stack. rcu_head structures > @@ -797,4 +778,12 @@ static inline void debug_rcu_head_unqueue(struct rcu_head *head) > } > #endif /* #else !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ > > +#if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) > +#include <linux/rcutree.h> > +#elif defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU) > +#include <linux/rcutiny.h> > +#else > +#error "Unknown RCU implementation specified to kernel configuration" > +#endif > + > #endif /* __LINUX_RCUPDATE_H */ > diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h > index 30ebd7c..167fb19 100644 > --- a/include/linux/rcutiny.h > +++ b/include/linux/rcutiny.h > @@ -26,6 +26,7 @@ > #define __LINUX_TINY_H > > #include <linux/cache.h> > +#include <linux/sched.h> > > static inline void rcu_init(void) > { > @@ -47,6 +48,40 @@ static inline void rcu_barrier(void) > > void rcu_barrier(void); > void synchronize_rcu_expedited(void); > +void rcu_read_unlock_special(struct task_struct *t); > + > +/* > + * Tiny-preemptible RCU implementation for rcu_read_lock(). > + * Just increment ->rcu_read_lock_nesting, shared state will be updated > + * if we block. > + */ > +static inline void __rcu_read_lock(void) > +{ > + current->rcu_read_lock_nesting++; > + barrier(); > +} > + > +/* > + * Tiny-preemptible RCU implementation for rcu_read_unlock(). > + * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost > + * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then > + * invoke rcu_read_unlock_special() to clean up after a context switch > + * in an RCU read-side critical section and other special cases. > + */ > +static inline void __rcu_read_unlock(void) > +{ > + struct task_struct *t = current; > + > + barrier(); > + --t->rcu_read_lock_nesting; > + barrier(); /* decrement before load of ->rcu_read_unlock_special */ > + if (t->rcu_read_lock_nesting == 0 && > + unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) > + rcu_read_unlock_special(t); > +#ifdef CONFIG_PROVE_LOCKING > + WARN_ON_ONCE(t->rcu_read_lock_nesting < 0); > +#endif /* #ifdef CONFIG_PROVE_LOCKING */ > +} > > #endif /* #else #ifdef CONFIG_TINY_RCU */ > > diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h > index 3a93348..012cc1c 100644 > --- a/include/linux/rcutree.h > +++ b/include/linux/rcutree.h > @@ -30,6 +30,8 @@ > #ifndef __LINUX_RCUTREE_H > #define __LINUX_RCUTREE_H > > +#include <linux/sched.h> > + > extern void rcu_init(void); > extern void rcu_note_context_switch(int cpu); > extern int rcu_needs_cpu(int cpu); > @@ -37,6 +39,41 @@ extern void rcu_cpu_stall_reset(void); > > #ifdef CONFIG_TREE_PREEMPT_RCU > > +extern void rcu_read_unlock_special(struct task_struct *t); > + > +/* > + * Tree-preemptable RCU implementation for rcu_read_lock(). > + * Just increment ->rcu_read_lock_nesting, shared state will be updated > + * if we block. > + */ > +static inline void __rcu_read_lock(void) > +{ > + current->rcu_read_lock_nesting++; > + barrier(); > +} > + > +/* > + * Tree-preemptable RCU implementation for rcu_read_unlock(). > + * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost > + * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then > + * invoke rcu_read_unlock_special() to clean up after a context switch > + * in an RCU read-side critical section and other special cases. > + */ > +static inline void __rcu_read_unlock(void) > +{ > + struct task_struct *t = current; > + > + barrier(); > + --t->rcu_read_lock_nesting; > + barrier(); /* decrement before load of ->rcu_read_unlock_special */ > + if (t->rcu_read_lock_nesting == 0 && > + unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) > + rcu_read_unlock_special(t); > +#ifdef CONFIG_PROVE_LOCKING > + WARN_ON_ONCE(ACCESS_ONCE(t->rcu_read_lock_nesting) < 0); > +#endif /* #ifdef CONFIG_PROVE_LOCKING */ > +} > + > extern void exit_rcu(void); > > #else /* #ifdef CONFIG_TREE_PREEMPT_RCU */ > diff --git a/include/linux/sched.h b/include/linux/sched.h > index 83bd2e2..30a4444 100644 > --- a/include/linux/sched.h > +++ b/include/linux/sched.h > @@ -78,7 +78,7 @@ struct sched_param { > #include <linux/topology.h> > #include <linux/proportions.h> > #include <linux/seccomp.h> > -#include <linux/rcupdate.h> > +#include <linux/types.h> > #include <linux/rculist.h> > #include <linux/rtmutex.h> > > @@ -2241,11 +2241,9 @@ int same_thread_group(struct task_struct *p1, struct task_struct *p2) > return p1->tgid == p2->tgid; > } > > -static inline struct task_struct *next_thread(const struct task_struct *p) > -{ > - return list_entry_rcu(p->thread_group.next, > - struct task_struct, thread_group); > -} > +/* Avoid #include hell for inlining rcu_read_lock(). */ > +#define next_thread(p) \ > + list_entry_rcu((p)->thread_group.next, struct task_struct, thread_group) > > static inline int thread_group_empty(struct task_struct *p) > { > diff --git a/include/linux/sem.h b/include/linux/sem.h > index f2961af..8489a1f 100644 > --- a/include/linux/sem.h > +++ b/include/linux/sem.h > @@ -78,7 +78,7 @@ struct seminfo { > > #ifdef __KERNEL__ > #include <asm/atomic.h> > -#include <linux/rcupdate.h> > +#include <linux/types.h> > #include <linux/cache.h> > > struct task_struct; > diff --git a/include/linux/soundcard.h b/include/linux/soundcard.h > index 1904afe..f99c32f 100644 > --- a/include/linux/soundcard.h > +++ b/include/linux/soundcard.h > @@ -1064,7 +1064,9 @@ typedef struct mixer_vol_table { > */ > #define SEQ_DECLAREBUF() SEQ_USE_EXTBUF() > > +#ifdef __KERNEL__ > void seqbuf_dump(void); /* This function must be provided by programs */ > +#endif > > #define SEQ_PM_DEFINES int __foo_bar___ > > diff --git a/include/linux/sysctl.h b/include/linux/sysctl.h > index 11684d9..be80a59 100644 > --- a/include/linux/sysctl.h > +++ b/include/linux/sysctl.h > @@ -19,6 +19,8 @@ > **************************************************************** > */ > > +#include <linux/key.h> > + > #ifndef _LINUX_SYSCTL_H > #define _LINUX_SYSCTL_H > > @@ -1012,8 +1014,7 @@ extern int proc_do_large_bitmap(struct ctl_table *, int, > */ > > /* A sysctl table is an array of struct ctl_table: */ > -struct ctl_table > -{ > +struct ctl_table { > const char *procname; /* Text ID for /proc/sys, or zero */ > void *data; > int maxlen; > diff --git a/include/linux/types.h b/include/linux/types.h > index 176da8c..868ef8b 100644 > --- a/include/linux/types.h > +++ b/include/linux/types.h > @@ -231,6 +231,16 @@ struct hlist_node { > struct hlist_node *next, **pprev; > }; > > +/** > + * struct rcu_head - callback structure for use with RCU > + * @next: next update requests in a list > + * @func: actual update function to call after the grace period. > + */ > +struct rcu_head { > + struct rcu_head *next; > + void (*func)(struct rcu_head *head); > +}; > + > struct ustat { > __kernel_daddr_t f_tfree; > __kernel_ino_t f_tinode; > diff --git a/kernel/pid_namespace.c b/kernel/pid_namespace.c > index e9c9adc..bfa75df 100644 > --- a/kernel/pid_namespace.c > +++ b/kernel/pid_namespace.c > @@ -8,6 +8,8 @@ > * > */ > > +#include <linux/types.h> > +#include <asm/atomic.h> > #include <linux/pid.h> > #include <linux/pid_namespace.h> > #include <linux/syscalls.h> > diff --git a/kernel/rcutiny_plugin.h b/kernel/rcutiny_plugin.h > index 3cb8e36..d0e1ac3 100644 > --- a/kernel/rcutiny_plugin.h > +++ b/kernel/rcutiny_plugin.h > @@ -520,23 +520,11 @@ void rcu_preempt_note_context_switch(void) > } > > /* > - * Tiny-preemptible RCU implementation for rcu_read_lock(). > - * Just increment ->rcu_read_lock_nesting, shared state will be updated > - * if we block. > - */ > -void __rcu_read_lock(void) > -{ > - current->rcu_read_lock_nesting++; > - barrier(); /* needed if we ever invoke rcu_read_lock in rcutiny.c */ > -} > -EXPORT_SYMBOL_GPL(__rcu_read_lock); > - > -/* > * Handle special cases during rcu_read_unlock(), such as needing to > * notify RCU core processing or task having blocked during the RCU > * read-side critical section. > */ > -static void rcu_read_unlock_special(struct task_struct *t) > +void rcu_read_unlock_special(struct task_struct *t) > { > int empty; > int empty_exp; > @@ -616,29 +604,7 @@ static void rcu_read_unlock_special(struct task_struct *t) > #endif /* #ifdef CONFIG_RCU_BOOST */ > local_irq_restore(flags); > } > - > -/* > - * Tiny-preemptible RCU implementation for rcu_read_unlock(). > - * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost > - * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then > - * invoke rcu_read_unlock_special() to clean up after a context switch > - * in an RCU read-side critical section and other special cases. > - */ > -void __rcu_read_unlock(void) > -{ > - struct task_struct *t = current; > - > - barrier(); /* needed if we ever invoke rcu_read_unlock in rcutiny.c */ > - --t->rcu_read_lock_nesting; > - barrier(); /* decrement before load of ->rcu_read_unlock_special */ > - if (t->rcu_read_lock_nesting == 0 && > - unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) > - rcu_read_unlock_special(t); > -#ifdef CONFIG_PROVE_LOCKING > - WARN_ON_ONCE(t->rcu_read_lock_nesting < 0); > -#endif /* #ifdef CONFIG_PROVE_LOCKING */ > -} > -EXPORT_SYMBOL_GPL(__rcu_read_unlock); > +EXPORT_SYMBOL_GPL(rcu_read_unlock_special); > > /* > * Check for a quiescent state from the current CPU. When a task blocks, > diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h > index a363871..4b27afd 100644 > --- a/kernel/rcutree_plugin.h > +++ b/kernel/rcutree_plugin.h > @@ -196,18 +196,6 @@ static void rcu_preempt_note_context_switch(int cpu) > } > > /* > - * Tree-preemptable RCU implementation for rcu_read_lock(). > - * Just increment ->rcu_read_lock_nesting, shared state will be updated > - * if we block. > - */ > -void __rcu_read_lock(void) > -{ > - current->rcu_read_lock_nesting++; > - barrier(); /* needed if we ever invoke rcu_read_lock in rcutree.c */ > -} > -EXPORT_SYMBOL_GPL(__rcu_read_lock); > - > -/* > * Check for preempted RCU readers blocking the current grace period > * for the specified rcu_node structure. If the caller needs a reliable > * answer, it must hold the rcu_node's ->lock. > @@ -261,7 +249,7 @@ static void rcu_report_unblock_qs_rnp(struct rcu_node *rnp, unsigned long flags) > * notify RCU core processing or task having blocked during the RCU > * read-side critical section. > */ > -static void rcu_read_unlock_special(struct task_struct *t) > +void rcu_read_unlock_special(struct task_struct *t) > { > int empty; > int empty_exp; > @@ -332,29 +320,7 @@ static void rcu_read_unlock_special(struct task_struct *t) > local_irq_restore(flags); > } > } > - > -/* > - * Tree-preemptable RCU implementation for rcu_read_unlock(). > - * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost > - * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then > - * invoke rcu_read_unlock_special() to clean up after a context switch > - * in an RCU read-side critical section and other special cases. > - */ > -void __rcu_read_unlock(void) > -{ > - struct task_struct *t = current; > - > - barrier(); /* needed if we ever invoke rcu_read_unlock in rcutree.c */ > - --t->rcu_read_lock_nesting; > - barrier(); /* decrement before load of ->rcu_read_unlock_special */ > - if (t->rcu_read_lock_nesting == 0 && > - unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) > - rcu_read_unlock_special(t); > -#ifdef CONFIG_PROVE_LOCKING > - WARN_ON_ONCE(ACCESS_ONCE(t->rcu_read_lock_nesting) < 0); > -#endif /* #ifdef CONFIG_PROVE_LOCKING */ > -} > -EXPORT_SYMBOL_GPL(__rcu_read_unlock); > +EXPORT_SYMBOL_GPL(rcu_read_unlock_special); > > #ifdef CONFIG_RCU_CPU_STALL_DETECTOR > > diff --git a/kernel/sysctl_binary.c b/kernel/sysctl_binary.c > index 3b8e028..58097b7 100644 > --- a/kernel/sysctl_binary.c > +++ b/kernel/sysctl_binary.c > @@ -1,4 +1,5 @@ > #include <linux/stat.h> > +#include <linux/key.h> > #include <linux/sysctl.h> > #include "../fs/xfs/linux-2.6/xfs_sysctl.h" > #include <linux/sunrpc/debug.h> > diff --git a/kernel/sysctl_check.c b/kernel/sysctl_check.c > index 4e4932a..a8a0df5 100644 > --- a/kernel/sysctl_check.c > +++ b/kernel/sysctl_check.c > @@ -1,4 +1,5 @@ > #include <linux/stat.h> > +#include <linux/key.h> > #include <linux/sysctl.h> > #include "../fs/xfs/linux-2.6/xfs_sysctl.h" > #include <linux/sunrpc/debug.h> > diff --git a/net/core/sysctl_net_core.c b/net/core/sysctl_net_core.c > index 385b609..528b975 100644 > --- a/net/core/sysctl_net_core.c > +++ b/net/core/sysctl_net_core.c > @@ -6,6 +6,7 @@ > */ > > #include <linux/mm.h> > +#include <linux/key.h> > #include <linux/sysctl.h> > #include <linux/module.h> > #include <linux/socket.h> > diff --git a/net/dccp/sysctl.c b/net/dccp/sysctl.c > index 4234882..bf609bc 100644 > --- a/net/dccp/sysctl.c > +++ b/net/dccp/sysctl.c > @@ -10,6 +10,7 @@ > */ > > #include <linux/mm.h> > +#include <linux/key.h> > #include <linux/sysctl.h> > #include "dccp.h" > #include "feat.h" > diff --git a/net/ipv6/sysctl_net_ipv6.c b/net/ipv6/sysctl_net_ipv6.c > index 6dcf5e7..97a3ce0 100644 > --- a/net/ipv6/sysctl_net_ipv6.c > +++ b/net/ipv6/sysctl_net_ipv6.c > @@ -6,6 +6,7 @@ > */ > > #include <linux/mm.h> > +#include <linux/key.h> > #include <linux/sysctl.h> > #include <linux/in6.h> > #include <linux/ipv6.h> > diff --git a/net/irda/irsysctl.c b/net/irda/irsysctl.c > index d0b70da..11aaf12 100644 > --- a/net/irda/irsysctl.c > +++ b/net/irda/irsysctl.c > @@ -25,6 +25,7 @@ > > #include <linux/mm.h> > #include <linux/ctype.h> > +#include <linux/key.h> > #include <linux/sysctl.h> > #include <linux/init.h> > > diff --git a/net/phonet/sysctl.c b/net/phonet/sysctl.c > index cea1c7d..2c2d479 100644 > --- a/net/phonet/sysctl.c > +++ b/net/phonet/sysctl.c > @@ -23,6 +23,7 @@ > */ > > #include <linux/seqlock.h> > +#include <linux/key.h> > #include <linux/sysctl.h> > #include <linux/errno.h> > #include <linux/init.h> > diff --git a/net/rds/ib_sysctl.c b/net/rds/ib_sysctl.c > index 1253b00..2f9a1a7 100644 > --- a/net/rds/ib_sysctl.c > +++ b/net/rds/ib_sysctl.c > @@ -31,6 +31,7 @@ > * > */ > #include <linux/kernel.h> > +#include <linux/key.h> > #include <linux/sysctl.h> > #include <linux/proc_fs.h> > > diff --git a/net/rds/iw_sysctl.c b/net/rds/iw_sysctl.c > index e2e4717..8c4051b 100644 > --- a/net/rds/iw_sysctl.c > +++ b/net/rds/iw_sysctl.c > @@ -31,6 +31,7 @@ > * > */ > #include <linux/kernel.h> > +#include <linux/key.h> > #include <linux/sysctl.h> > #include <linux/proc_fs.h> > > diff --git a/net/rds/sysctl.c b/net/rds/sysctl.c > index 25ad0c7..46d2d1d 100644 > --- a/net/rds/sysctl.c > +++ b/net/rds/sysctl.c > @@ -31,6 +31,7 @@ > * > */ > #include <linux/kernel.h> > +#include <linux/key.h> > #include <linux/sysctl.h> > #include <linux/proc_fs.h> > > diff --git a/net/sctp/sysctl.c b/net/sctp/sysctl.c > index 50cb57f..c2903ce 100644 > --- a/net/sctp/sysctl.c > +++ b/net/sctp/sysctl.c > @@ -43,6 +43,7 @@ > > #include <net/sctp/structs.h> > #include <net/sctp/sctp.h> > +#include <linux/key.h> > #include <linux/sysctl.h> > > static int zero = 0; > diff --git a/net/sunrpc/sysctl.c b/net/sunrpc/sysctl.c > index e65dcc6..1429601 100644 > --- a/net/sunrpc/sysctl.c > +++ b/net/sunrpc/sysctl.c > @@ -11,6 +11,7 @@ > #include <linux/linkage.h> > #include <linux/ctype.h> > #include <linux/fs.h> > +#include <linux/key.h> > #include <linux/sysctl.h> > #include <linux/module.h> > > diff --git a/net/unix/sysctl_net_unix.c b/net/unix/sysctl_net_unix.c > index 397cffe..6d2766e 100644 > --- a/net/unix/sysctl_net_unix.c > +++ b/net/unix/sysctl_net_unix.c > @@ -11,6 +11,7 @@ > > #include <linux/mm.h> > #include <linux/slab.h> > +#include <linux/key.h> > #include <linux/sysctl.h> > > #include <net/af_unix.h> > diff --git a/net/xfrm/xfrm_sysctl.c b/net/xfrm/xfrm_sysctl.c > index 05640bc..a50e4bf 100644 > --- a/net/xfrm/xfrm_sysctl.c > +++ b/net/xfrm/xfrm_sysctl.c > @@ -1,3 +1,4 @@ > +#include <linux/key.h> > #include <linux/sysctl.h> > #include <linux/slab.h> > #include <net/net_namespace.h> > ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-11 5:12 ` Paul E. McKenney 2011-04-11 6:01 ` Lai Jiangshan @ 2011-04-11 8:31 ` Lai Jiangshan 2011-04-11 21:02 ` Paul E. McKenney 2011-04-11 8:33 ` Lai Jiangshan 2 siblings, 1 reply; 67+ messages in thread From: Lai Jiangshan @ 2011-04-11 8:31 UTC (permalink / raw) To: paulmck Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On 04/11/2011 01:12 PM, Paul E. McKenney wrote: > -static inline struct task_struct *next_thread(const struct task_struct *p) > -{ > - return list_entry_rcu(p->thread_group.next, > - struct task_struct, thread_group); > -} > +/* Avoid #include hell for inlining rcu_read_lock(). */ > +#define next_thread(p) \ > + list_entry_rcu((p)->thread_group.next, struct task_struct, thread_group) > It is strange for me. The user of the API "next_thread(p)" must has two headers included, sched.h and rculist.h I know this is a very popular pattern of user space code, is it OK for kernel? I think a file(even a header file) uses API(Marco), it should includes the the corresponding headers, it reduces surprises(example, the name of "next_thread()" has no "rcu", it is not expected that rcuxxxx.h is required). I admit the work will become very much simpler if this pattern is allowed. man fcntl: #include <unistd.h> #include <fcntl.h> int fcntl(int fd, int cmd, ... /* arg */ ); ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-11 8:31 ` Lai Jiangshan @ 2011-04-11 21:02 ` Paul E. McKenney 2011-04-11 21:24 ` Peter Zijlstra 2011-04-22 7:19 ` Lai Jiangshan 0 siblings, 2 replies; 67+ messages in thread From: Paul E. McKenney @ 2011-04-11 21:02 UTC (permalink / raw) To: Lai Jiangshan Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On Mon, Apr 11, 2011 at 04:31:10PM +0800, Lai Jiangshan wrote: > On 04/11/2011 01:12 PM, Paul E. McKenney wrote: > > > -static inline struct task_struct *next_thread(const struct task_struct *p) > > -{ > > - return list_entry_rcu(p->thread_group.next, > > - struct task_struct, thread_group); > > -} > > +/* Avoid #include hell for inlining rcu_read_lock(). */ > > +#define next_thread(p) \ > > + list_entry_rcu((p)->thread_group.next, struct task_struct, thread_group) > > > > > It is strange for me. > The user of the API "next_thread(p)" must has two headers included, sched.h and rculist.h > > I know this is a very popular pattern of user space code, is it OK for kernel? > I think a file(even a header file) uses API(Marco), it should includes the the corresponding > headers, it reduces surprises(example, the name of "next_thread()" has no "rcu", > it is not expected that rcuxxxx.h is required). > > I admit the work will become very much simpler if this pattern is allowed. The guy who maintains much of sched.h suggested it. ;-) Thanx, Paul > man fcntl: > #include <unistd.h> > #include <fcntl.h> > > int fcntl(int fd, int cmd, ... /* arg */ ); > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-11 21:02 ` Paul E. McKenney @ 2011-04-11 21:24 ` Peter Zijlstra 2011-04-22 7:19 ` Lai Jiangshan 1 sibling, 0 replies; 67+ messages in thread From: Peter Zijlstra @ 2011-04-11 21:24 UTC (permalink / raw) To: paulmck Cc: Lai Jiangshan, H. Peter Anvin, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On Mon, 2011-04-11 at 14:02 -0700, Paul E. McKenney wrote: > On Mon, Apr 11, 2011 at 04:31:10PM +0800, Lai Jiangshan wrote: > > On 04/11/2011 01:12 PM, Paul E. McKenney wrote: > > > > > -static inline struct task_struct *next_thread(const struct task_struct *p) > > > -{ > > > - return list_entry_rcu(p->thread_group.next, > > > - struct task_struct, thread_group); > > > -} > > > +/* Avoid #include hell for inlining rcu_read_lock(). */ > > > +#define next_thread(p) \ > > > + list_entry_rcu((p)->thread_group.next, struct task_struct, thread_group) > > > > > I admit the work will become very much simpler if this pattern is allowed. > > The guy who maintains much of sched.h suggested it. ;-) While its not the nicest thing its something that's on more than one occasion there's an existing site in sched.h and there's on in key.h and iirc there's another one that's relevant. A nice solution would be to move a lot of things out of sched.h into something like process.h, a header that deals with task groups and processes. But until that time, I see no problem with using this. ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-11 21:02 ` Paul E. McKenney 2011-04-11 21:24 ` Peter Zijlstra @ 2011-04-22 7:19 ` Lai Jiangshan 2011-04-22 8:10 ` Peter Zijlstra 2011-04-23 20:30 ` Paul E. McKenney 1 sibling, 2 replies; 67+ messages in thread From: Lai Jiangshan @ 2011-04-22 7:19 UTC (permalink / raw) To: paulmck Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On 04/12/2011 05:02 AM, Paul E. McKenney wrote: > On Mon, Apr 11, 2011 at 04:31:10PM +0800, Lai Jiangshan wrote: >> On 04/11/2011 01:12 PM, Paul E. McKenney wrote: >> >>> -static inline struct task_struct *next_thread(const struct task_struct *p) >>> -{ >>> - return list_entry_rcu(p->thread_group.next, >>> - struct task_struct, thread_group); >>> -} >>> +/* Avoid #include hell for inlining rcu_read_lock(). */ >>> +#define next_thread(p) \ >>> + list_entry_rcu((p)->thread_group.next, struct task_struct, thread_group) >>> >> >> >> It is strange for me. >> The user of the API "next_thread(p)" must has two headers included, sched.h and rculist.h >> >> I know this is a very popular pattern of user space code, is it OK for kernel? >> I think a file(even a header file) uses API(Marco), it should includes the the corresponding >> headers, it reduces surprises(example, the name of "next_thread()" has no "rcu", >> it is not expected that rcuxxxx.h is required). >> >> I admit the work will become very much simpler if this pattern is allowed. > > The guy who maintains much of sched.h suggested it. ;-) > > Thanx, Paul > >> man fcntl: >> #include <unistd.h> >> #include <fcntl.h> >> >> int fcntl(int fd, int cmd, ... /* arg */ ); >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> Please read the FAQ at http://www.tux.org/lkml/ > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > Hi, Paul What is the solution you prefer to? I insist on the solution which split rcupdate.h into 2 parts, the first part is rcupdate_defs.h which only contains: 1) struct rcu_head 2) MACRO rcu_dereference* 3) MACRO rcu_access_pointer rcu_access_index rcu_assign_pointer RCU_INIT_POINTER 4) rcu_*_lock_held() which is required by 2) All of these is required by sched.h, it is all about 450 lines of code. It does not just separate struct rcu_head out only, because rcu_dereference*() and APIs in rculist.h are required by in sched.h or headers included by sched.h. Any suggestion? Thanks, Lai ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-22 7:19 ` Lai Jiangshan @ 2011-04-22 8:10 ` Peter Zijlstra 2011-04-25 7:21 ` Lai Jiangshan 2011-04-23 20:30 ` Paul E. McKenney 1 sibling, 1 reply; 67+ messages in thread From: Peter Zijlstra @ 2011-04-22 8:10 UTC (permalink / raw) To: Lai Jiangshan Cc: paulmck, H. Peter Anvin, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On Fri, 2011-04-22 at 15:19 +0800, Lai Jiangshan wrote: > > What is the solution you prefer to? > > I insist on the solution which split rcupdate.h into 2 parts, > the first part is rcupdate_defs.h which only contains: > 1) struct rcu_head > 2) MACRO rcu_dereference* > 3) MACRO rcu_access_pointer rcu_access_index rcu_assign_pointer RCU_INIT_POINTER > 4) rcu_*_lock_held() which is required by 2) > > All of these is required by sched.h, it is all about 450 lines of code. > > It does not just separate struct rcu_head out only, because rcu_dereference*() > and APIs in rculist.h are required by in sched.h or headers included by sched.h. The only usage of rculist.h in the whole of sched.h is that _one_ list_entry_rcu, of which we already have a macro user, so why not make that one a macro too and safe ourselves a ton of grief? You're just making stuff harder than it needs to be. ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-22 8:10 ` Peter Zijlstra @ 2011-04-25 7:21 ` Lai Jiangshan 0 siblings, 0 replies; 67+ messages in thread From: Lai Jiangshan @ 2011-04-25 7:21 UTC (permalink / raw) To: Peter Zijlstra Cc: paulmck, H. Peter Anvin, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On 04/22/2011 04:10 PM, Peter Zijlstra wrote: > On Fri, 2011-04-22 at 15:19 +0800, Lai Jiangshan wrote: >> >> What is the solution you prefer to? >> >> I insist on the solution which split rcupdate.h into 2 parts, >> the first part is rcupdate_defs.h which only contains: >> 1) struct rcu_head >> 2) MACRO rcu_dereference* >> 3) MACRO rcu_access_pointer rcu_access_index rcu_assign_pointer RCU_INIT_POINTER >> 4) rcu_*_lock_held() which is required by 2) >> >> All of these is required by sched.h, it is all about 450 lines of code. >> >> It does not just separate struct rcu_head out only, because rcu_dereference*() >> and APIs in rculist.h are required by in sched.h or headers included by sched.h. > > The only usage of rculist.h in the whole of sched.h is that _one_ > list_entry_rcu, of which we already have a macro user, so why not make > that one a macro too and safe ourselves a ton of grief? > > You're just making stuff harder than it needs to be. > No, opposite. the more things(except rcu_read_[un]lock*()) are moved to rcupdate_defs.h the more easy. the less things are moved to rcupdate_defs.h the more hard. example, if rcu_dereference* is not moved to rcupdate_defs.h, I have to do 2 more things: 1) convert inline functions next_thread() to macro. 2) find all .c which uses next_thread()/next_task() and ensure they all have rcupdate.h included. 3?) handle the complaints from LKML and told them should convert their new APIs to macro and should also do the same work like 2) I'm just expressing my thinking, I will accept your requirement and make things progress. Thanks, Lai ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-22 7:19 ` Lai Jiangshan 2011-04-22 8:10 ` Peter Zijlstra @ 2011-04-23 20:30 ` Paul E. McKenney 1 sibling, 0 replies; 67+ messages in thread From: Paul E. McKenney @ 2011-04-23 20:30 UTC (permalink / raw) To: Lai Jiangshan Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On Fri, Apr 22, 2011 at 03:19:58PM +0800, Lai Jiangshan wrote: > On 04/12/2011 05:02 AM, Paul E. McKenney wrote: > > On Mon, Apr 11, 2011 at 04:31:10PM +0800, Lai Jiangshan wrote: > >> On 04/11/2011 01:12 PM, Paul E. McKenney wrote: > >> > >>> -static inline struct task_struct *next_thread(const struct task_struct *p) > >>> -{ > >>> - return list_entry_rcu(p->thread_group.next, > >>> - struct task_struct, thread_group); > >>> -} > >>> +/* Avoid #include hell for inlining rcu_read_lock(). */ > >>> +#define next_thread(p) \ > >>> + list_entry_rcu((p)->thread_group.next, struct task_struct, thread_group) > >>> > >> > >> > >> It is strange for me. > >> The user of the API "next_thread(p)" must has two headers included, sched.h and rculist.h > >> > >> I know this is a very popular pattern of user space code, is it OK for kernel? > >> I think a file(even a header file) uses API(Marco), it should includes the the corresponding > >> headers, it reduces surprises(example, the name of "next_thread()" has no "rcu", > >> it is not expected that rcuxxxx.h is required). > >> > >> I admit the work will become very much simpler if this pattern is allowed. > > > > The guy who maintains much of sched.h suggested it. ;-) > > > > Thanx, Paul > > > >> man fcntl: > >> #include <unistd.h> > >> #include <fcntl.h> > >> > >> int fcntl(int fd, int cmd, ... /* arg */ ); > >> -- > >> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > >> the body of a message to majordomo@vger.kernel.org > >> More majordomo info at http://vger.kernel.org/majordomo-info.html > >> Please read the FAQ at http://www.tux.org/lkml/ > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > Please read the FAQ at http://www.tux.org/lkml/ > > > > Hi, Paul > > What is the solution you prefer to? > > I insist on the solution which split rcupdate.h into 2 parts, > the first part is rcupdate_defs.h which only contains: > 1) struct rcu_head > 2) MACRO rcu_dereference* > 3) MACRO rcu_access_pointer rcu_access_index rcu_assign_pointer RCU_INIT_POINTER > 4) rcu_*_lock_held() which is required by 2) > > All of these is required by sched.h, it is all about 450 lines of code. > > It does not just separate struct rcu_head out only, because rcu_dereference*() > and APIs in rculist.h are required by in sched.h or headers included by sched.h. > > Any suggestion? My belief is that the Linux kernel's include files need to have a designed approach to reduce the include-file problems. For example, one approach would be to split out the data declarations from the access functions -- but it is not clear to me that this is the right approach. But whatever approach is chosen, that approach needs to have a clear design behind it, because otherwise we will end up continually refactoring. And whatever approach is chosen, putting simple/small data/type declarations into types.h is very likely to be part of the approach, given that types.h has worked well in a great many projects over a long period of time. I do hope that whatever overall approach we evolve to, that this approach will allow use of inline functions rather than macros. But it is easy to change these back and forth, so I am not too worried about them. In the meantime, for rcupdate.h, it appears that putting struct rcu_head into types.h and then converting a few inline functions to macros will suffice. Furthermore, this split of the rcupdate.h functionality will likely hold up for quite some time. You have been doing a much more thorough job of testing this change than I have, so it does make sense that you carry out the change. So, could you please try out this approach? Put rcu_head into types.h, re-order rcupdate.h as needed, and convert a few inline functions to macros? As I have said before, it will be really cool to allow rcu_read_lock() and rcu_read_unlock() to be inline functions for preemptible RCU!!! Thanx, Paul ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-11 5:12 ` Paul E. McKenney 2011-04-11 6:01 ` Lai Jiangshan 2011-04-11 8:31 ` Lai Jiangshan @ 2011-04-11 8:33 ` Lai Jiangshan 2 siblings, 0 replies; 67+ messages in thread From: Lai Jiangshan @ 2011-04-11 8:33 UTC (permalink / raw) To: paulmck Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On 04/11/2011 01:12 PM, Paul E. McKenney wrote: > On Mon, Apr 11, 2011 at 11:08:14AM +0800, Lai Jiangshan wrote: >> On 04/08/2011 01:13 PM, Paul E. McKenney wrote: >>> On Fri, Apr 08, 2011 at 09:26:16AM +0800, Lai Jiangshan wrote: >>>> On 04/08/2011 12:26 AM, Paul E. McKenney wrote: >>>>> On Thu, Apr 07, 2011 at 08:47:37AM -0700, Paul E. McKenney wrote: >>>>>> On Thu, Apr 07, 2011 at 01:49:51PM +0800, Lai Jiangshan wrote: >>>>>>> On 04/07/2011 08:30 AM, Paul E. McKenney wrote: >>>>>>>> On Wed, Apr 06, 2011 at 02:27:39PM -0700, H. Peter Anvin wrote: >>>>>>>>> On 04/06/2011 02:06 PM, Peter Zijlstra wrote: >>>>>>>>>> On Wed, 2011-04-06 at 13:13 -0700, Paul E. McKenney wrote: >>>>>>>>>>> And the following patch builds correctly for defconfig x86 builds, >>>>>>>>>>> while allowing rcupdate.h to see the sched.h definitions as needed >>>>>>>>>>> to inline rcu_read_lock() and rcu_read_unlock(). >>>>>>>>>>> >>>>>>>>>> Looks like an entirely reasonable patch to me ;-) >>>>>>>>>> >>>>>>>>> >>>>>>>>> Quite... a lot better than the original proposal! >>>>>>>> >>>>>>>> Glad you both like it! >>>>>>>> >>>>>>>> When I do an allyesconfig build, I do get errors during the "CHECK" >>>>>>>> phase, when it is putting things into the usr/include in the build tree. >>>>>>>> I believe that this is because I am exposing different header files to >>>>>>>> the library-export scripts. The following patch silences some of them, >>>>>>>> but I am really out of my depth here. >>>>>>>> >>>>>>>> Sam, Jan, Michal, help? >>>>>>>> >>>>>>>> Thanx, Paul >>>>>>>> >>>>>>>> ------------------------------------------------------------------------ >>>>>>>> >>>>>>> >>>>>>> Easy to split rcupdate.h, hard to resolve the dependence problem. >>>>>>> >>>>>>> You can apply the next additional patch when you test: >>>>>> >>>>>> I am sure that you are quite correct. ;-) >>>>>> >>>>>> I am moving _rcu_read_lock() and _rcu_read_unlock() into >>>>>> include/linux/rcutree.h and include/linux/rcutiny.h, and I am sure that >>>>>> more pain will ensue. >>>>>> >>>>>> One thing I don't understand... How does is it helping to group the >>>>>> task_struct RCU-related fields into a structure? Is that generating >>>>>> better code on your platform due to smaller offsets or something? >>>> >>>> You don't like task_rcu_struct patch? I think it can make code clearer, >>>> and it can also check the code even when CONFIG_PREEMPT_RCU=n. >>>> >>>> For rcu_read_[un]lock(), it generates the same code, no better, no worse. >>>> >>>> It is just a cleanup patch, it is helpless for making rcu_read_[un]lock() inline, >>>> if you don't like it, I will give up it. >>> >>> I don't know that I feel strongly either way about it. It was necessary >>> with the integer-offset approach, but optional now. >>> >>>>>> Also, does your patchset address the CHECK warnings? >>>>> >>>>> I take it back... I applied the following patch on top of my earlier >>>>> one, and a defconfig x86 build completed without error. (Though I have >>>>> not tested the results of the build.) >>>>> >>>>> One possible difference -- I did this work on top of a recent Linus >>>>> git commit (b2a8b4b81966) rather than on top of my -rcu tree. Also, >>>>> I have not yet tried an allyesconfig build, which will no doubt locate >>>>> some more problems. >>>>> >>>>> Thanx, Paul >>>>> >>>> >>>> when defconfig or allyesconfig, CONFIG_PREEMPT=n and CONFIG_TREE_PREEMPT_RCU=n >>>> when you make them "y": >>>> >>>> In file included from include/linux/rcupdate.h:764:0, >>>> from include/linux/tracepoint.h:19, >>>> from include/linux/module.h:18, >>>> from include/linux/crypto.h:21, >>>> from arch/x86/kernel/asm-offsets.c:8: >>>> include/linux/rcutree.h:50:20: error: static declaration of ‘__rcu_read_lock’ follows non-static declaration >>>> include/linux/rcupdate.h:76:13: note: previous declaration of ‘__rcu_read_lock’ was here >>>> include/linux/rcutree.h:63:20: error: static declaration of ‘__rcu_read_unlock’ follows non-static declaration >>>> include/linux/rcupdate.h:77:13: note: previous declaration of ‘__rcu_read_unlock’ was here >>>> make[1]: *** [arch/x86/kernel/asm-offsets.s] Error 1 >>>> make: *** [prepare0] Error 2 >>> >>> Yep. I need to move the rcu_read_lock() APIs to follow the inclusion >>> of rcutree.h and rcutiny.h. Also add include of sched.h to rcutiny.h. >>> The code movement does bloat the patch a bit. But rcu_assign_pointer() >>> must precede the inclusion of rcutree.h and rcutiny.h, so it is not >>> possible to simply move the inclusions. See below. >>> >>> Thanx, Paul >>> >> >> sched.h still contains rcupdate.h after applied this patch. > > Then we are not in sync -- sched.h does not include rcupdate.h in my tree. > It instead gets the struct rcu_head definition from include/linux/types.h. > See below for a consolidated patch. > > Thanx, Paul > >> See my [PATCH 2/4] for more info. >> >> >> # make lib/is_single_threaded.o >> CHK include/linux/version.h >> CHK include/generated/utsrelease.h >> CALL scripts/checksyscalls.sh >> CC lib/is_single_threaded.o >> In file included from include/linux/rcupdate.h:639:0, >> from include/linux/rculist.h:10, >> from include/linux/sched.h:82, >> from lib/is_single_threaded.c:13: >> include/linux/rcutree.h: In function ‘__rcu_read_lock’: >> include/linux/rcutree.h:52:15: error: dereferencing pointer to incomplete type >> In file included from include/linux/rcupdate.h:639:0, >> from include/linux/rculist.h:10, >> from include/linux/sched.h:82, >> from lib/is_single_threaded.c:13: >> include/linux/rcutree.h: In function ‘__rcu_read_unlock’: >> include/linux/rcutree.h:68:5: error: dereferencing pointer to incomplete type >> include/linux/rcutree.h:70:7: error: dereferencing pointer to incomplete type >> include/linux/rcutree.h:71:46: error: dereferencing pointer to incomplete type >> include/linux/rcutree.h:71:78: error: dereferencing pointer to incomplete type >> include/linux/rcutree.h:71:6: warning: type defaults to ‘int’ in type name >> make[1]: *** [lib/is_single_threaded.o] Error 1 >> make: *** [lib/is_single_threaded.o] Error 2 > > drivers/scsi/scsi_sysctl.c | 1 + > fs/proc/proc_sysctl.c | 1 + > include/linux/kernel.h | 3 +++ > include/linux/pid.h | 2 +- > include/linux/rcupdate.h | 29 +++++++++-------------------- > include/linux/rcutiny.h | 35 +++++++++++++++++++++++++++++++++++ > include/linux/rcutree.h | 37 +++++++++++++++++++++++++++++++++++++ > include/linux/sched.h | 10 ++++------ > include/linux/sem.h | 2 +- > include/linux/soundcard.h | 2 ++ > include/linux/sysctl.h | 5 +++-- > include/linux/types.h | 10 ++++++++++ > kernel/pid_namespace.c | 2 ++ > kernel/rcutiny_plugin.h | 38 ++------------------------------------ > kernel/rcutree_plugin.h | 38 ++------------------------------------ > kernel/sysctl_binary.c | 1 + > kernel/sysctl_check.c | 1 + > net/core/sysctl_net_core.c | 1 + > net/dccp/sysctl.c | 1 + > net/ipv6/sysctl_net_ipv6.c | 1 + > net/irda/irsysctl.c | 1 + > net/phonet/sysctl.c | 1 + > net/rds/ib_sysctl.c | 1 + > net/rds/iw_sysctl.c | 1 + > net/rds/sysctl.c | 1 + > net/sctp/sysctl.c | 1 + > net/sunrpc/sysctl.c | 1 + > net/unix/sysctl_net_unix.c | 1 + > net/xfrm/xfrm_sysctl.c | 1 + > 29 files changed, 127 insertions(+), 102 deletions(-) > sched.h still contains rcupdate.h after applied this patch. so it still fails. Is other part of patch(or patch 2/2) not sent? ^ permalink raw reply [flat|nested] 67+ messages in thread
* [PATCH 1/4] rcu: split rcupdate.h 2011-04-07 0:30 ` Paul E. McKenney 2011-04-07 5:49 ` Lai Jiangshan @ 2011-04-07 7:05 ` Lai Jiangshan 2011-04-07 7:07 ` [PATCH 2/4] rcu: make rcudpate.h can use struct task_struct Lai Jiangshan ` (2 subsequent siblings) 4 siblings, 0 replies; 67+ messages in thread From: Lai Jiangshan @ 2011-04-07 7:05 UTC (permalink / raw) To: paulmck Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt Split rcupdate.h and move basic defines to rcupdate_defs.h. It allows rcupdate.h using struct task_struct. Only rcu_read_lock*() (which require struct task_struct) are left in rcupdate.h. Some other code(example, the declaration of synchronize_rcu()) can also be left in rcupdate.h actually, but it is not required and it will make the works more complex. So I also move them. The result rcupdate_defs.h is a little big, but it just contains the RCU defines and declarations, it is OK. Maybe rcupdate_basic.h is a better name? Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> --- include/linux/rcupdate.h | 616 +-------------------------------------- include/linux/rcupdate_defs.h | 653 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 654 insertions(+), 615 deletions(-) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index dd4cd6d..c25d2a6 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -33,42 +33,7 @@ #ifndef __LINUX_RCUPDATE_H #define __LINUX_RCUPDATE_H -#include <linux/cache.h> -#include <linux/spinlock.h> -#include <linux/threads.h> -#include <linux/cpumask.h> -#include <linux/seqlock.h> -#include <linux/lockdep.h> -#include <linux/completion.h> -#include <linux/debugobjects.h> -#include <linux/compiler.h> - -#ifdef CONFIG_RCU_TORTURE_TEST -extern int rcutorture_runnable; /* for sysctl */ -#endif /* #ifdef CONFIG_RCU_TORTURE_TEST */ - -#define UINT_CMP_GE(a, b) (UINT_MAX / 2 >= (a) - (b)) -#define UINT_CMP_LT(a, b) (UINT_MAX / 2 < (a) - (b)) -#define ULONG_CMP_GE(a, b) (ULONG_MAX / 2 >= (a) - (b)) -#define ULONG_CMP_LT(a, b) (ULONG_MAX / 2 < (a) - (b)) - -/** - * struct rcu_head - callback structure for use with RCU - * @next: next update requests in a list - * @func: actual update function to call after the grace period. - */ -struct rcu_head { - struct rcu_head *next; - void (*func)(struct rcu_head *head); -}; - -/* Exported common interfaces */ -extern void call_rcu_sched(struct rcu_head *head, - void (*func)(struct rcu_head *rcu)); -extern void synchronize_sched(void); -extern void rcu_barrier_bh(void); -extern void rcu_barrier_sched(void); -extern int sched_expedited_torture_stats(char *page); +#include <linux/rcupdate_defs.h> static inline void __rcu_read_lock_bh(void) { @@ -84,7 +49,6 @@ static inline void __rcu_read_unlock_bh(void) extern void __rcu_read_lock(void); extern void __rcu_read_unlock(void); -void synchronize_rcu(void); /* * Defined as a macro as it is a very low level header included from @@ -106,11 +70,6 @@ static inline void __rcu_read_unlock(void) preempt_enable(); } -static inline void synchronize_rcu(void) -{ - synchronize_sched(); -} - static inline int rcu_preempt_depth(void) { return 0; @@ -118,402 +77,6 @@ static inline int rcu_preempt_depth(void) #endif /* #else #ifdef CONFIG_PREEMPT_RCU */ -/* Internal to kernel */ -extern void rcu_sched_qs(int cpu); -extern void rcu_bh_qs(int cpu); -extern void rcu_check_callbacks(int cpu, int user); -struct notifier_block; - -#ifdef CONFIG_NO_HZ - -extern void rcu_enter_nohz(void); -extern void rcu_exit_nohz(void); - -#else /* #ifdef CONFIG_NO_HZ */ - -static inline void rcu_enter_nohz(void) -{ -} - -static inline void rcu_exit_nohz(void) -{ -} - -#endif /* #else #ifdef CONFIG_NO_HZ */ - -#if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) -#include <linux/rcutree.h> -#elif defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU) -#include <linux/rcutiny.h> -#else -#error "Unknown RCU implementation specified to kernel configuration" -#endif - -/* - * init_rcu_head_on_stack()/destroy_rcu_head_on_stack() are needed for dynamic - * initialization and destruction of rcu_head on the stack. rcu_head structures - * allocated dynamically in the heap or defined statically don't need any - * initialization. - */ -#ifdef CONFIG_DEBUG_OBJECTS_RCU_HEAD -extern void init_rcu_head_on_stack(struct rcu_head *head); -extern void destroy_rcu_head_on_stack(struct rcu_head *head); -#else /* !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ -static inline void init_rcu_head_on_stack(struct rcu_head *head) -{ -} - -static inline void destroy_rcu_head_on_stack(struct rcu_head *head) -{ -} -#endif /* #else !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ - -#ifdef CONFIG_DEBUG_LOCK_ALLOC - -extern struct lockdep_map rcu_lock_map; -# define rcu_read_acquire() \ - lock_acquire(&rcu_lock_map, 0, 0, 2, 1, NULL, _THIS_IP_) -# define rcu_read_release() lock_release(&rcu_lock_map, 1, _THIS_IP_) - -extern struct lockdep_map rcu_bh_lock_map; -# define rcu_read_acquire_bh() \ - lock_acquire(&rcu_bh_lock_map, 0, 0, 2, 1, NULL, _THIS_IP_) -# define rcu_read_release_bh() lock_release(&rcu_bh_lock_map, 1, _THIS_IP_) - -extern struct lockdep_map rcu_sched_lock_map; -# define rcu_read_acquire_sched() \ - lock_acquire(&rcu_sched_lock_map, 0, 0, 2, 1, NULL, _THIS_IP_) -# define rcu_read_release_sched() \ - lock_release(&rcu_sched_lock_map, 1, _THIS_IP_) - -extern int debug_lockdep_rcu_enabled(void); - -/** - * rcu_read_lock_held() - might we be in RCU read-side critical section? - * - * If CONFIG_DEBUG_LOCK_ALLOC is selected, returns nonzero iff in an RCU - * read-side critical section. In absence of CONFIG_DEBUG_LOCK_ALLOC, - * this assumes we are in an RCU read-side critical section unless it can - * prove otherwise. This is useful for debug checks in functions that - * require that they be called within an RCU read-side critical section. - * - * Checks debug_lockdep_rcu_enabled() to prevent false positives during boot - * and while lockdep is disabled. - */ -static inline int rcu_read_lock_held(void) -{ - if (!debug_lockdep_rcu_enabled()) - return 1; - return lock_is_held(&rcu_lock_map); -} - -/* - * rcu_read_lock_bh_held() is defined out of line to avoid #include-file - * hell. - */ -extern int rcu_read_lock_bh_held(void); - -/** - * rcu_read_lock_sched_held() - might we be in RCU-sched read-side critical section? - * - * If CONFIG_DEBUG_LOCK_ALLOC is selected, returns nonzero iff in an - * RCU-sched read-side critical section. In absence of - * CONFIG_DEBUG_LOCK_ALLOC, this assumes we are in an RCU-sched read-side - * critical section unless it can prove otherwise. Note that disabling - * of preemption (including disabling irqs) counts as an RCU-sched - * read-side critical section. This is useful for debug checks in functions - * that required that they be called within an RCU-sched read-side - * critical section. - * - * Check debug_lockdep_rcu_enabled() to prevent false positives during boot - * and while lockdep is disabled. - */ -#ifdef CONFIG_PREEMPT -static inline int rcu_read_lock_sched_held(void) -{ - int lockdep_opinion = 0; - - if (!debug_lockdep_rcu_enabled()) - return 1; - if (debug_locks) - lockdep_opinion = lock_is_held(&rcu_sched_lock_map); - return lockdep_opinion || preempt_count() != 0 || irqs_disabled(); -} -#else /* #ifdef CONFIG_PREEMPT */ -static inline int rcu_read_lock_sched_held(void) -{ - return 1; -} -#endif /* #else #ifdef CONFIG_PREEMPT */ - -#else /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */ - -# define rcu_read_acquire() do { } while (0) -# define rcu_read_release() do { } while (0) -# define rcu_read_acquire_bh() do { } while (0) -# define rcu_read_release_bh() do { } while (0) -# define rcu_read_acquire_sched() do { } while (0) -# define rcu_read_release_sched() do { } while (0) - -static inline int rcu_read_lock_held(void) -{ - return 1; -} - -static inline int rcu_read_lock_bh_held(void) -{ - return 1; -} - -#ifdef CONFIG_PREEMPT -static inline int rcu_read_lock_sched_held(void) -{ - return preempt_count() != 0 || irqs_disabled(); -} -#else /* #ifdef CONFIG_PREEMPT */ -static inline int rcu_read_lock_sched_held(void) -{ - return 1; -} -#endif /* #else #ifdef CONFIG_PREEMPT */ - -#endif /* #else #ifdef CONFIG_DEBUG_LOCK_ALLOC */ - -#ifdef CONFIG_PROVE_RCU - -extern int rcu_my_thread_group_empty(void); - -/** - * rcu_lockdep_assert - emit lockdep splat if specified condition not met - * @c: condition to check - */ -#define rcu_lockdep_assert(c) \ - do { \ - static bool __warned; \ - if (debug_lockdep_rcu_enabled() && !__warned && !(c)) { \ - __warned = true; \ - lockdep_rcu_dereference(__FILE__, __LINE__); \ - } \ - } while (0) - -#else /* #ifdef CONFIG_PROVE_RCU */ - -#define rcu_lockdep_assert(c) do { } while (0) - -#endif /* #else #ifdef CONFIG_PROVE_RCU */ - -/* - * Helper functions for rcu_dereference_check(), rcu_dereference_protected() - * and rcu_assign_pointer(). Some of these could be folded into their - * callers, but they are left separate in order to ease introduction of - * multiple flavors of pointers to match the multiple flavors of RCU - * (e.g., __rcu_bh, * __rcu_sched, and __srcu), should this make sense in - * the future. - */ - -#ifdef __CHECKER__ -#define rcu_dereference_sparse(p, space) \ - ((void)(((typeof(*p) space *)p) == p)) -#else /* #ifdef __CHECKER__ */ -#define rcu_dereference_sparse(p, space) -#endif /* #else #ifdef __CHECKER__ */ - -#define __rcu_access_pointer(p, space) \ - ({ \ - typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ - rcu_dereference_sparse(p, space); \ - ((typeof(*p) __force __kernel *)(_________p1)); \ - }) -#define __rcu_dereference_check(p, c, space) \ - ({ \ - typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ - rcu_lockdep_assert(c); \ - rcu_dereference_sparse(p, space); \ - smp_read_barrier_depends(); \ - ((typeof(*p) __force __kernel *)(_________p1)); \ - }) -#define __rcu_dereference_protected(p, c, space) \ - ({ \ - rcu_lockdep_assert(c); \ - rcu_dereference_sparse(p, space); \ - ((typeof(*p) __force __kernel *)(p)); \ - }) - -#define __rcu_dereference_index_check(p, c) \ - ({ \ - typeof(p) _________p1 = ACCESS_ONCE(p); \ - rcu_lockdep_assert(c); \ - smp_read_barrier_depends(); \ - (_________p1); \ - }) -#define __rcu_assign_pointer(p, v, space) \ - ({ \ - if (!__builtin_constant_p(v) || \ - ((v) != NULL)) \ - smp_wmb(); \ - (p) = (typeof(*v) __force space *)(v); \ - }) - - -/** - * rcu_access_pointer() - fetch RCU pointer with no dereferencing - * @p: The pointer to read - * - * Return the value of the specified RCU-protected pointer, but omit the - * smp_read_barrier_depends() and keep the ACCESS_ONCE(). This is useful - * when the value of this pointer is accessed, but the pointer is not - * dereferenced, for example, when testing an RCU-protected pointer against - * NULL. Although rcu_access_pointer() may also be used in cases where - * update-side locks prevent the value of the pointer from changing, you - * should instead use rcu_dereference_protected() for this use case. - */ -#define rcu_access_pointer(p) __rcu_access_pointer((p), __rcu) - -/** - * rcu_dereference_check() - rcu_dereference with debug checking - * @p: The pointer to read, prior to dereferencing - * @c: The conditions under which the dereference will take place - * - * Do an rcu_dereference(), but check that the conditions under which the - * dereference will take place are correct. Typically the conditions - * indicate the various locking conditions that should be held at that - * point. The check should return true if the conditions are satisfied. - * An implicit check for being in an RCU read-side critical section - * (rcu_read_lock()) is included. - * - * For example: - * - * bar = rcu_dereference_check(foo->bar, lockdep_is_held(&foo->lock)); - * - * could be used to indicate to lockdep that foo->bar may only be dereferenced - * if either rcu_read_lock() is held, or that the lock required to replace - * the bar struct at foo->bar is held. - * - * Note that the list of conditions may also include indications of when a lock - * need not be held, for example during initialisation or destruction of the - * target struct: - * - * bar = rcu_dereference_check(foo->bar, lockdep_is_held(&foo->lock) || - * atomic_read(&foo->usage) == 0); - * - * Inserts memory barriers on architectures that require them - * (currently only the Alpha), prevents the compiler from refetching - * (and from merging fetches), and, more importantly, documents exactly - * which pointers are protected by RCU and checks that the pointer is - * annotated as __rcu. - */ -#define rcu_dereference_check(p, c) \ - __rcu_dereference_check((p), rcu_read_lock_held() || (c), __rcu) - -/** - * rcu_dereference_bh_check() - rcu_dereference_bh with debug checking - * @p: The pointer to read, prior to dereferencing - * @c: The conditions under which the dereference will take place - * - * This is the RCU-bh counterpart to rcu_dereference_check(). - */ -#define rcu_dereference_bh_check(p, c) \ - __rcu_dereference_check((p), rcu_read_lock_bh_held() || (c), __rcu) - -/** - * rcu_dereference_sched_check() - rcu_dereference_sched with debug checking - * @p: The pointer to read, prior to dereferencing - * @c: The conditions under which the dereference will take place - * - * This is the RCU-sched counterpart to rcu_dereference_check(). - */ -#define rcu_dereference_sched_check(p, c) \ - __rcu_dereference_check((p), rcu_read_lock_sched_held() || (c), \ - __rcu) - -#define rcu_dereference_raw(p) rcu_dereference_check(p, 1) /*@@@ needed? @@@*/ - -/** - * rcu_dereference_index_check() - rcu_dereference for indices with debug checking - * @p: The pointer to read, prior to dereferencing - * @c: The conditions under which the dereference will take place - * - * Similar to rcu_dereference_check(), but omits the sparse checking. - * This allows rcu_dereference_index_check() to be used on integers, - * which can then be used as array indices. Attempting to use - * rcu_dereference_check() on an integer will give compiler warnings - * because the sparse address-space mechanism relies on dereferencing - * the RCU-protected pointer. Dereferencing integers is not something - * that even gcc will put up with. - * - * Note that this function does not implicitly check for RCU read-side - * critical sections. If this function gains lots of uses, it might - * make sense to provide versions for each flavor of RCU, but it does - * not make sense as of early 2010. - */ -#define rcu_dereference_index_check(p, c) \ - __rcu_dereference_index_check((p), (c)) - -/** - * rcu_dereference_protected() - fetch RCU pointer when updates prevented - * @p: The pointer to read, prior to dereferencing - * @c: The conditions under which the dereference will take place - * - * Return the value of the specified RCU-protected pointer, but omit - * both the smp_read_barrier_depends() and the ACCESS_ONCE(). This - * is useful in cases where update-side locks prevent the value of the - * pointer from changing. Please note that this primitive does -not- - * prevent the compiler from repeating this reference or combining it - * with other references, so it should not be used without protection - * of appropriate locks. - * - * This function is only for update-side use. Using this function - * when protected only by rcu_read_lock() will result in infrequent - * but very ugly failures. - */ -#define rcu_dereference_protected(p, c) \ - __rcu_dereference_protected((p), (c), __rcu) - -/** - * rcu_dereference_bh_protected() - fetch RCU-bh pointer when updates prevented - * @p: The pointer to read, prior to dereferencing - * @c: The conditions under which the dereference will take place - * - * This is the RCU-bh counterpart to rcu_dereference_protected(). - */ -#define rcu_dereference_bh_protected(p, c) \ - __rcu_dereference_protected((p), (c), __rcu) - -/** - * rcu_dereference_sched_protected() - fetch RCU-sched pointer when updates prevented - * @p: The pointer to read, prior to dereferencing - * @c: The conditions under which the dereference will take place - * - * This is the RCU-sched counterpart to rcu_dereference_protected(). - */ -#define rcu_dereference_sched_protected(p, c) \ - __rcu_dereference_protected((p), (c), __rcu) - - -/** - * rcu_dereference() - fetch RCU-protected pointer for dereferencing - * @p: The pointer to read, prior to dereferencing - * - * This is a simple wrapper around rcu_dereference_check(). - */ -#define rcu_dereference(p) rcu_dereference_check(p, 0) - -/** - * rcu_dereference_bh() - fetch an RCU-bh-protected pointer for dereferencing - * @p: The pointer to read, prior to dereferencing - * - * Makes rcu_dereference_check() do the dirty work. - */ -#define rcu_dereference_bh(p) rcu_dereference_bh_check(p, 0) - -/** - * rcu_dereference_sched() - fetch RCU-sched-protected pointer for dereferencing - * @p: The pointer to read, prior to dereferencing - * - * Makes rcu_dereference_check() do the dirty work. - */ -#define rcu_dereference_sched(p) rcu_dereference_sched_check(p, 0) - /** * rcu_read_lock() - mark the beginning of an RCU read-side critical section * @@ -657,181 +220,4 @@ static inline notrace void rcu_read_unlock_sched_notrace(void) preempt_enable_notrace(); } -/** - * rcu_assign_pointer() - assign to RCU-protected pointer - * @p: pointer to assign to - * @v: value to assign (publish) - * - * Assigns the specified value to the specified RCU-protected - * pointer, ensuring that any concurrent RCU readers will see - * any prior initialization. Returns the value assigned. - * - * Inserts memory barriers on architectures that require them - * (pretty much all of them other than x86), and also prevents - * the compiler from reordering the code that initializes the - * structure after the pointer assignment. More importantly, this - * call documents which pointers will be dereferenced by RCU read-side - * code. - */ -#define rcu_assign_pointer(p, v) \ - __rcu_assign_pointer((p), (v), __rcu) - -/** - * RCU_INIT_POINTER() - initialize an RCU protected pointer - * - * Initialize an RCU-protected pointer in such a way to avoid RCU-lockdep - * splats. - */ -#define RCU_INIT_POINTER(p, v) \ - p = (typeof(*v) __force __rcu *)(v) - -/* Infrastructure to implement the synchronize_() primitives. */ - -struct rcu_synchronize { - struct rcu_head head; - struct completion completion; -}; - -extern void wakeme_after_rcu(struct rcu_head *head); - -#ifdef CONFIG_PREEMPT_RCU - -/** - * call_rcu() - Queue an RCU callback for invocation after a grace period. - * @head: structure to be used for queueing the RCU updates. - * @func: actual callback function to be invoked after the grace period - * - * The callback function will be invoked some time after a full grace - * period elapses, in other words after all pre-existing RCU read-side - * critical sections have completed. However, the callback function - * might well execute concurrently with RCU read-side critical sections - * that started after call_rcu() was invoked. RCU read-side critical - * sections are delimited by rcu_read_lock() and rcu_read_unlock(), - * and may be nested. - */ -extern void call_rcu(struct rcu_head *head, - void (*func)(struct rcu_head *head)); - -#else /* #ifdef CONFIG_PREEMPT_RCU */ - -/* In classic RCU, call_rcu() is just call_rcu_sched(). */ -#define call_rcu call_rcu_sched - -#endif /* #else #ifdef CONFIG_PREEMPT_RCU */ - -/** - * call_rcu_bh() - Queue an RCU for invocation after a quicker grace period. - * @head: structure to be used for queueing the RCU updates. - * @func: actual callback function to be invoked after the grace period - * - * The callback function will be invoked some time after a full grace - * period elapses, in other words after all currently executing RCU - * read-side critical sections have completed. call_rcu_bh() assumes - * that the read-side critical sections end on completion of a softirq - * handler. This means that read-side critical sections in process - * context must not be interrupted by softirqs. This interface is to be - * used when most of the read-side critical sections are in softirq context. - * RCU read-side critical sections are delimited by : - * - rcu_read_lock() and rcu_read_unlock(), if in interrupt context. - * OR - * - rcu_read_lock_bh() and rcu_read_unlock_bh(), if in process context. - * These may be nested. - */ -extern void call_rcu_bh(struct rcu_head *head, - void (*func)(struct rcu_head *head)); - -/* - * debug_rcu_head_queue()/debug_rcu_head_unqueue() are used internally - * by call_rcu() and rcu callback execution, and are therefore not part of the - * RCU API. Leaving in rcupdate.h because they are used by all RCU flavors. - */ - -#ifdef CONFIG_DEBUG_OBJECTS_RCU_HEAD -# define STATE_RCU_HEAD_READY 0 -# define STATE_RCU_HEAD_QUEUED 1 - -extern struct debug_obj_descr rcuhead_debug_descr; - -static inline void debug_rcu_head_queue(struct rcu_head *head) -{ - WARN_ON_ONCE((unsigned long)head & 0x3); - debug_object_activate(head, &rcuhead_debug_descr); - debug_object_active_state(head, &rcuhead_debug_descr, - STATE_RCU_HEAD_READY, - STATE_RCU_HEAD_QUEUED); -} - -static inline void debug_rcu_head_unqueue(struct rcu_head *head) -{ - debug_object_active_state(head, &rcuhead_debug_descr, - STATE_RCU_HEAD_QUEUED, - STATE_RCU_HEAD_READY); - debug_object_deactivate(head, &rcuhead_debug_descr); -} -#else /* !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ -static inline void debug_rcu_head_queue(struct rcu_head *head) -{ -} - -static inline void debug_rcu_head_unqueue(struct rcu_head *head) -{ -} -#endif /* #else !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ - -static __always_inline bool __is_kfree_rcu_offset(unsigned long offset) -{ - return offset < 4096; -} - -static __always_inline -void __kfree_rcu(struct rcu_head *head, unsigned long offset) -{ - typedef void (*rcu_callback)(struct rcu_head *); - - BUILD_BUG_ON(!__builtin_constant_p(offset)); - - /* See the kfree_rcu() header comment. */ - BUILD_BUG_ON(!__is_kfree_rcu_offset(offset)); - - call_rcu(head, (rcu_callback)offset); -} - -extern void kfree(const void *); - -static inline void __rcu_reclaim(struct rcu_head *head) -{ - unsigned long offset = (unsigned long)head->func; - - if (__is_kfree_rcu_offset(offset)) - kfree((void *)head - offset); - else - head->func(head); -} - -/** - * kfree_rcu() - kfree an object after a grace period. - * @ptr: pointer to kfree - * @rcu_head: the name of the struct rcu_head within the type of @ptr. - * - * Many rcu callbacks functions just call kfree() on the base structure. - * These functions are trivial, but their size adds up, and furthermore - * when they are used in a kernel module, that module must invoke the - * high-latency rcu_barrier() function at module-unload time. - * - * The kfree_rcu() function handles this issue. Rather than encoding a - * function address in the embedded rcu_head structure, kfree_rcu() instead - * encodes the offset of the rcu_head structure within the base structure. - * Because the functions are not allowed in the low-order 4096 bytes of - * kernel virtual memory, offsets up to 4095 bytes can be accommodated. - * If the offset is larger than 4095 bytes, a compile-time error will - * be generated in __kfree_rcu(). If this error is triggered, you can - * either fall back to use of call_rcu() or rearrange the structure to - * position the rcu_head structure into the first 4096 bytes. - * - * Note that the allowable offset might decrease in the future, for example, - * to allow something like kmem_cache_free_rcu(). - */ -#define kfree_rcu(ptr, rcu_head) \ - __kfree_rcu(&((ptr)->rcu_head), offsetof(typeof(*(ptr)), rcu_head)) - #endif /* __LINUX_RCUPDATE_H */ diff --git a/include/linux/rcupdate_defs.h b/include/linux/rcupdate_defs.h new file mode 100644 index 0000000..ae84745 --- /dev/null +++ b/include/linux/rcupdate_defs.h @@ -0,0 +1,653 @@ +/* + * Read-Copy Update mechanism for mutual exclusion + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Copyright IBM Corporation, 2001 + * + * Author: Dipankar Sarma <dipankar@in.ibm.com> + * + * Based on the original work by Paul McKenney <paulmck@us.ibm.com> + * and inputs from Rusty Russell, Andrea Arcangeli and Andi Kleen. + * Papers: + * http://www.rdrop.com/users/paulmck/paper/rclockpdcsproof.pdf + * http://lse.sourceforge.net/locking/rclock_OLS.2001.05.01c.sc.pdf (OLS2001) + * + * For detailed explanation of Read-Copy Update mechanism see - + * http://lse.sourceforge.net/locking/rcupdate.html + * + */ + +#ifndef __LINUX_RCUPDATE_DEFS_H +#define __LINUX_RCUPDATE_DEFS_H + +#include <linux/cache.h> +#include <linux/spinlock.h> +#include <linux/threads.h> +#include <linux/cpumask.h> +#include <linux/seqlock.h> +#include <linux/lockdep.h> +#include <linux/completion.h> +#include <linux/debugobjects.h> +#include <linux/compiler.h> + +#ifdef CONFIG_RCU_TORTURE_TEST +extern int rcutorture_runnable; /* for sysctl */ +#endif /* #ifdef CONFIG_RCU_TORTURE_TEST */ + +#define UINT_CMP_GE(a, b) (UINT_MAX / 2 >= (a) - (b)) +#define UINT_CMP_LT(a, b) (UINT_MAX / 2 < (a) - (b)) +#define ULONG_CMP_GE(a, b) (ULONG_MAX / 2 >= (a) - (b)) +#define ULONG_CMP_LT(a, b) (ULONG_MAX / 2 < (a) - (b)) + +/** + * struct rcu_head - callback structure for use with RCU + * @next: next update requests in a list + * @func: actual update function to call after the grace period. + */ +struct rcu_head { + struct rcu_head *next; + void (*func)(struct rcu_head *head); +}; + +/* Exported common interfaces */ +extern void call_rcu_sched(struct rcu_head *head, + void (*func)(struct rcu_head *rcu)); +extern void synchronize_sched(void); +extern void rcu_barrier_bh(void); +extern void rcu_barrier_sched(void); +extern int sched_expedited_torture_stats(char *page); + +/* Internal to kernel */ +extern void rcu_sched_qs(int cpu); +extern void rcu_bh_qs(int cpu); +extern void rcu_check_callbacks(int cpu, int user); +struct notifier_block; + +#ifdef CONFIG_NO_HZ + +extern void rcu_enter_nohz(void); +extern void rcu_exit_nohz(void); + +#else /* #ifdef CONFIG_NO_HZ */ + +static inline void rcu_enter_nohz(void) +{ +} + +static inline void rcu_exit_nohz(void) +{ +} + +#endif /* #else #ifdef CONFIG_NO_HZ */ + +#if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) +#include <linux/rcutree.h> +#elif defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU) +#include <linux/rcutiny.h> +#else +#error "Unknown RCU implementation specified to kernel configuration" +#endif + +/* + * init_rcu_head_on_stack()/destroy_rcu_head_on_stack() are needed for dynamic + * initialization and destruction of rcu_head on the stack. rcu_head structures + * allocated dynamically in the heap or defined statically don't need any + * initialization. + */ +#ifdef CONFIG_DEBUG_OBJECTS_RCU_HEAD +extern void init_rcu_head_on_stack(struct rcu_head *head); +extern void destroy_rcu_head_on_stack(struct rcu_head *head); +#else /* !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ +static inline void init_rcu_head_on_stack(struct rcu_head *head) +{ +} + +static inline void destroy_rcu_head_on_stack(struct rcu_head *head) +{ +} +#endif /* #else !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ + +#ifdef CONFIG_DEBUG_LOCK_ALLOC + +extern struct lockdep_map rcu_lock_map; +# define rcu_read_acquire() \ + lock_acquire(&rcu_lock_map, 0, 0, 2, 1, NULL, _THIS_IP_) +# define rcu_read_release() lock_release(&rcu_lock_map, 1, _THIS_IP_) + +extern struct lockdep_map rcu_bh_lock_map; +# define rcu_read_acquire_bh() \ + lock_acquire(&rcu_bh_lock_map, 0, 0, 2, 1, NULL, _THIS_IP_) +# define rcu_read_release_bh() lock_release(&rcu_bh_lock_map, 1, _THIS_IP_) + +extern struct lockdep_map rcu_sched_lock_map; +# define rcu_read_acquire_sched() \ + lock_acquire(&rcu_sched_lock_map, 0, 0, 2, 1, NULL, _THIS_IP_) +# define rcu_read_release_sched() \ + lock_release(&rcu_sched_lock_map, 1, _THIS_IP_) + +extern int debug_lockdep_rcu_enabled(void); + +/** + * rcu_read_lock_held() - might we be in RCU read-side critical section? + * + * If CONFIG_DEBUG_LOCK_ALLOC is selected, returns nonzero iff in an RCU + * read-side critical section. In absence of CONFIG_DEBUG_LOCK_ALLOC, + * this assumes we are in an RCU read-side critical section unless it can + * prove otherwise. This is useful for debug checks in functions that + * require that they be called within an RCU read-side critical section. + * + * Checks debug_lockdep_rcu_enabled() to prevent false positives during boot + * and while lockdep is disabled. + */ +static inline int rcu_read_lock_held(void) +{ + if (!debug_lockdep_rcu_enabled()) + return 1; + return lock_is_held(&rcu_lock_map); +} + +/* + * rcu_read_lock_bh_held() is defined out of line to avoid #include-file + * hell. + */ +extern int rcu_read_lock_bh_held(void); + +/** + * rcu_read_lock_sched_held() - might we be in RCU-sched read-side critical section? + * + * If CONFIG_DEBUG_LOCK_ALLOC is selected, returns nonzero iff in an + * RCU-sched read-side critical section. In absence of + * CONFIG_DEBUG_LOCK_ALLOC, this assumes we are in an RCU-sched read-side + * critical section unless it can prove otherwise. Note that disabling + * of preemption (including disabling irqs) counts as an RCU-sched + * read-side critical section. This is useful for debug checks in functions + * that required that they be called within an RCU-sched read-side + * critical section. + * + * Check debug_lockdep_rcu_enabled() to prevent false positives during boot + * and while lockdep is disabled. + */ +#ifdef CONFIG_PREEMPT +static inline int rcu_read_lock_sched_held(void) +{ + int lockdep_opinion = 0; + + if (!debug_lockdep_rcu_enabled()) + return 1; + if (debug_locks) + lockdep_opinion = lock_is_held(&rcu_sched_lock_map); + return lockdep_opinion || preempt_count() != 0 || irqs_disabled(); +} +#else /* #ifdef CONFIG_PREEMPT */ +static inline int rcu_read_lock_sched_held(void) +{ + return 1; +} +#endif /* #else #ifdef CONFIG_PREEMPT */ + +#else /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */ + +# define rcu_read_acquire() do { } while (0) +# define rcu_read_release() do { } while (0) +# define rcu_read_acquire_bh() do { } while (0) +# define rcu_read_release_bh() do { } while (0) +# define rcu_read_acquire_sched() do { } while (0) +# define rcu_read_release_sched() do { } while (0) + +static inline int rcu_read_lock_held(void) +{ + return 1; +} + +static inline int rcu_read_lock_bh_held(void) +{ + return 1; +} + +#ifdef CONFIG_PREEMPT +static inline int rcu_read_lock_sched_held(void) +{ + return preempt_count() != 0 || irqs_disabled(); +} +#else /* #ifdef CONFIG_PREEMPT */ +static inline int rcu_read_lock_sched_held(void) +{ + return 1; +} +#endif /* #else #ifdef CONFIG_PREEMPT */ + +#endif /* #else #ifdef CONFIG_DEBUG_LOCK_ALLOC */ + +#ifdef CONFIG_PROVE_RCU + +extern int rcu_my_thread_group_empty(void); + +/** + * rcu_lockdep_assert - emit lockdep splat if specified condition not met + * @c: condition to check + */ +#define rcu_lockdep_assert(c) \ + do { \ + static bool __warned; \ + if (debug_lockdep_rcu_enabled() && !__warned && !(c)) { \ + __warned = true; \ + lockdep_rcu_dereference(__FILE__, __LINE__); \ + } \ + } while (0) + +#else /* #ifdef CONFIG_PROVE_RCU */ + +#define rcu_lockdep_assert(c) do { } while (0) + +#endif /* #else #ifdef CONFIG_PROVE_RCU */ + +/* + * Helper functions for rcu_dereference_check(), rcu_dereference_protected() + * and rcu_assign_pointer(). Some of these could be folded into their + * callers, but they are left separate in order to ease introduction of + * multiple flavors of pointers to match the multiple flavors of RCU + * (e.g., __rcu_bh, * __rcu_sched, and __srcu), should this make sense in + * the future. + */ + +#ifdef __CHECKER__ +#define rcu_dereference_sparse(p, space) \ + ((void)(((typeof(*p) space *)p) == p)) +#else /* #ifdef __CHECKER__ */ +#define rcu_dereference_sparse(p, space) +#endif /* #else #ifdef __CHECKER__ */ + +#define __rcu_access_pointer(p, space) \ + ({ \ + typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ + rcu_dereference_sparse(p, space); \ + ((typeof(*p) __force __kernel *)(_________p1)); \ + }) +#define __rcu_dereference_check(p, c, space) \ + ({ \ + typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ + rcu_lockdep_assert(c); \ + rcu_dereference_sparse(p, space); \ + smp_read_barrier_depends(); \ + ((typeof(*p) __force __kernel *)(_________p1)); \ + }) +#define __rcu_dereference_protected(p, c, space) \ + ({ \ + rcu_lockdep_assert(c); \ + rcu_dereference_sparse(p, space); \ + ((typeof(*p) __force __kernel *)(p)); \ + }) + +#define __rcu_dereference_index_check(p, c) \ + ({ \ + typeof(p) _________p1 = ACCESS_ONCE(p); \ + rcu_lockdep_assert(c); \ + smp_read_barrier_depends(); \ + (_________p1); \ + }) +#define __rcu_assign_pointer(p, v, space) \ + ({ \ + if (!__builtin_constant_p(v) || \ + ((v) != NULL)) \ + smp_wmb(); \ + (p) = (typeof(*v) __force space *)(v); \ + }) + + +/** + * rcu_access_pointer() - fetch RCU pointer with no dereferencing + * @p: The pointer to read + * + * Return the value of the specified RCU-protected pointer, but omit the + * smp_read_barrier_depends() and keep the ACCESS_ONCE(). This is useful + * when the value of this pointer is accessed, but the pointer is not + * dereferenced, for example, when testing an RCU-protected pointer against + * NULL. Although rcu_access_pointer() may also be used in cases where + * update-side locks prevent the value of the pointer from changing, you + * should instead use rcu_dereference_protected() for this use case. + */ +#define rcu_access_pointer(p) __rcu_access_pointer((p), __rcu) + +/** + * rcu_dereference_check() - rcu_dereference with debug checking + * @p: The pointer to read, prior to dereferencing + * @c: The conditions under which the dereference will take place + * + * Do an rcu_dereference(), but check that the conditions under which the + * dereference will take place are correct. Typically the conditions + * indicate the various locking conditions that should be held at that + * point. The check should return true if the conditions are satisfied. + * An implicit check for being in an RCU read-side critical section + * (rcu_read_lock()) is included. + * + * For example: + * + * bar = rcu_dereference_check(foo->bar, lockdep_is_held(&foo->lock)); + * + * could be used to indicate to lockdep that foo->bar may only be dereferenced + * if either rcu_read_lock() is held, or that the lock required to replace + * the bar struct at foo->bar is held. + * + * Note that the list of conditions may also include indications of when a lock + * need not be held, for example during initialisation or destruction of the + * target struct: + * + * bar = rcu_dereference_check(foo->bar, lockdep_is_held(&foo->lock) || + * atomic_read(&foo->usage) == 0); + * + * Inserts memory barriers on architectures that require them + * (currently only the Alpha), prevents the compiler from refetching + * (and from merging fetches), and, more importantly, documents exactly + * which pointers are protected by RCU and checks that the pointer is + * annotated as __rcu. + */ +#define rcu_dereference_check(p, c) \ + __rcu_dereference_check((p), rcu_read_lock_held() || (c), __rcu) + +/** + * rcu_dereference_bh_check() - rcu_dereference_bh with debug checking + * @p: The pointer to read, prior to dereferencing + * @c: The conditions under which the dereference will take place + * + * This is the RCU-bh counterpart to rcu_dereference_check(). + */ +#define rcu_dereference_bh_check(p, c) \ + __rcu_dereference_check((p), rcu_read_lock_bh_held() || (c), __rcu) + +/** + * rcu_dereference_sched_check() - rcu_dereference_sched with debug checking + * @p: The pointer to read, prior to dereferencing + * @c: The conditions under which the dereference will take place + * + * This is the RCU-sched counterpart to rcu_dereference_check(). + */ +#define rcu_dereference_sched_check(p, c) \ + __rcu_dereference_check((p), rcu_read_lock_sched_held() || (c), \ + __rcu) + +#define rcu_dereference_raw(p) rcu_dereference_check(p, 1) /*@@@ needed? @@@*/ + +/** + * rcu_dereference_index_check() - rcu_dereference for indices with debug checking + * @p: The pointer to read, prior to dereferencing + * @c: The conditions under which the dereference will take place + * + * Similar to rcu_dereference_check(), but omits the sparse checking. + * This allows rcu_dereference_index_check() to be used on integers, + * which can then be used as array indices. Attempting to use + * rcu_dereference_check() on an integer will give compiler warnings + * because the sparse address-space mechanism relies on dereferencing + * the RCU-protected pointer. Dereferencing integers is not something + * that even gcc will put up with. + * + * Note that this function does not implicitly check for RCU read-side + * critical sections. If this function gains lots of uses, it might + * make sense to provide versions for each flavor of RCU, but it does + * not make sense as of early 2010. + */ +#define rcu_dereference_index_check(p, c) \ + __rcu_dereference_index_check((p), (c)) + +/** + * rcu_dereference_protected() - fetch RCU pointer when updates prevented + * @p: The pointer to read, prior to dereferencing + * @c: The conditions under which the dereference will take place + * + * Return the value of the specified RCU-protected pointer, but omit + * both the smp_read_barrier_depends() and the ACCESS_ONCE(). This + * is useful in cases where update-side locks prevent the value of the + * pointer from changing. Please note that this primitive does -not- + * prevent the compiler from repeating this reference or combining it + * with other references, so it should not be used without protection + * of appropriate locks. + * + * This function is only for update-side use. Using this function + * when protected only by rcu_read_lock() will result in infrequent + * but very ugly failures. + */ +#define rcu_dereference_protected(p, c) \ + __rcu_dereference_protected((p), (c), __rcu) + +/** + * rcu_dereference_bh_protected() - fetch RCU-bh pointer when updates prevented + * @p: The pointer to read, prior to dereferencing + * @c: The conditions under which the dereference will take place + * + * This is the RCU-bh counterpart to rcu_dereference_protected(). + */ +#define rcu_dereference_bh_protected(p, c) \ + __rcu_dereference_protected((p), (c), __rcu) + +/** + * rcu_dereference_sched_protected() - fetch RCU-sched pointer when updates prevented + * @p: The pointer to read, prior to dereferencing + * @c: The conditions under which the dereference will take place + * + * This is the RCU-sched counterpart to rcu_dereference_protected(). + */ +#define rcu_dereference_sched_protected(p, c) \ + __rcu_dereference_protected((p), (c), __rcu) + + +/** + * rcu_dereference() - fetch RCU-protected pointer for dereferencing + * @p: The pointer to read, prior to dereferencing + * + * This is a simple wrapper around rcu_dereference_check(). + */ +#define rcu_dereference(p) rcu_dereference_check(p, 0) + +/** + * rcu_dereference_bh() - fetch an RCU-bh-protected pointer for dereferencing + * @p: The pointer to read, prior to dereferencing + * + * Makes rcu_dereference_check() do the dirty work. + */ +#define rcu_dereference_bh(p) rcu_dereference_bh_check(p, 0) + +/** + * rcu_dereference_sched() - fetch RCU-sched-protected pointer for dereferencing + * @p: The pointer to read, prior to dereferencing + * + * Makes rcu_dereference_check() do the dirty work. + */ +#define rcu_dereference_sched(p) rcu_dereference_sched_check(p, 0) + +/** + * rcu_assign_pointer() - assign to RCU-protected pointer + * @p: pointer to assign to + * @v: value to assign (publish) + * + * Assigns the specified value to the specified RCU-protected + * pointer, ensuring that any concurrent RCU readers will see + * any prior initialization. Returns the value assigned. + * + * Inserts memory barriers on architectures that require them + * (pretty much all of them other than x86), and also prevents + * the compiler from reordering the code that initializes the + * structure after the pointer assignment. More importantly, this + * call documents which pointers will be dereferenced by RCU read-side + * code. + */ +#define rcu_assign_pointer(p, v) \ + __rcu_assign_pointer((p), (v), __rcu) + +/** + * RCU_INIT_POINTER() - initialize an RCU protected pointer + * + * Initialize an RCU-protected pointer in such a way to avoid RCU-lockdep + * splats. + */ +#define RCU_INIT_POINTER(p, v) \ + p = (typeof(*v) __force __rcu *)(v) + +/* Infrastructure to implement the synchronize_() primitives. */ + +struct rcu_synchronize { + struct rcu_head head; + struct completion completion; +}; + +extern void wakeme_after_rcu(struct rcu_head *head); + +#ifdef CONFIG_PREEMPT_RCU + +/** + * call_rcu() - Queue an RCU callback for invocation after a grace period. + * @head: structure to be used for queueing the RCU updates. + * @func: actual callback function to be invoked after the grace period + * + * The callback function will be invoked some time after a full grace + * period elapses, in other words after all pre-existing RCU read-side + * critical sections have completed. However, the callback function + * might well execute concurrently with RCU read-side critical sections + * that started after call_rcu() was invoked. RCU read-side critical + * sections are delimited by rcu_read_lock() and rcu_read_unlock(), + * and may be nested. + */ +extern void call_rcu(struct rcu_head *head, + void (*func)(struct rcu_head *head)); + +void synchronize_rcu(void); + +#else /* #ifdef CONFIG_PREEMPT_RCU */ + +/* In classic RCU, call_rcu() is just call_rcu_sched(). */ +#define call_rcu call_rcu_sched + +static inline void synchronize_rcu(void) +{ + synchronize_sched(); +} + +#endif /* #else #ifdef CONFIG_PREEMPT_RCU */ + +/** + * call_rcu_bh() - Queue an RCU for invocation after a quicker grace period. + * @head: structure to be used for queueing the RCU updates. + * @func: actual callback function to be invoked after the grace period + * + * The callback function will be invoked some time after a full grace + * period elapses, in other words after all currently executing RCU + * read-side critical sections have completed. call_rcu_bh() assumes + * that the read-side critical sections end on completion of a softirq + * handler. This means that read-side critical sections in process + * context must not be interrupted by softirqs. This interface is to be + * used when most of the read-side critical sections are in softirq context. + * RCU read-side critical sections are delimited by : + * - rcu_read_lock() and rcu_read_unlock(), if in interrupt context. + * OR + * - rcu_read_lock_bh() and rcu_read_unlock_bh(), if in process context. + * These may be nested. + */ +extern void call_rcu_bh(struct rcu_head *head, + void (*func)(struct rcu_head *head)); + +/* + * debug_rcu_head_queue()/debug_rcu_head_unqueue() are used internally + * by call_rcu() and rcu callback execution, and are therefore not part of the + * RCU API. Leaving in rcupdate.h because they are used by all RCU flavors. + */ + +#ifdef CONFIG_DEBUG_OBJECTS_RCU_HEAD +# define STATE_RCU_HEAD_READY 0 +# define STATE_RCU_HEAD_QUEUED 1 + +extern struct debug_obj_descr rcuhead_debug_descr; + +static inline void debug_rcu_head_queue(struct rcu_head *head) +{ + WARN_ON_ONCE((unsigned long)head & 0x3); + debug_object_activate(head, &rcuhead_debug_descr); + debug_object_active_state(head, &rcuhead_debug_descr, + STATE_RCU_HEAD_READY, + STATE_RCU_HEAD_QUEUED); +} + +static inline void debug_rcu_head_unqueue(struct rcu_head *head) +{ + debug_object_active_state(head, &rcuhead_debug_descr, + STATE_RCU_HEAD_QUEUED, + STATE_RCU_HEAD_READY); + debug_object_deactivate(head, &rcuhead_debug_descr); +} +#else /* !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ +static inline void debug_rcu_head_queue(struct rcu_head *head) +{ +} + +static inline void debug_rcu_head_unqueue(struct rcu_head *head) +{ +} +#endif /* #else !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ + +static __always_inline bool __is_kfree_rcu_offset(unsigned long offset) +{ + return offset < 4096; +} + +static __always_inline +void __kfree_rcu(struct rcu_head *head, unsigned long offset) +{ + typedef void (*rcu_callback)(struct rcu_head *); + + BUILD_BUG_ON(!__builtin_constant_p(offset)); + + /* See the kfree_rcu() header comment. */ + BUILD_BUG_ON(!__is_kfree_rcu_offset(offset)); + + call_rcu(head, (rcu_callback)offset); +} + +extern void kfree(const void *); + +static inline void __rcu_reclaim(struct rcu_head *head) +{ + unsigned long offset = (unsigned long)head->func; + + if (__is_kfree_rcu_offset(offset)) + kfree((void *)head - offset); + else + head->func(head); +} + +/** + * kfree_rcu() - kfree an object after a grace period. + * @ptr: pointer to kfree + * @rcu_head: the name of the struct rcu_head within the type of @ptr. + * + * Many rcu callbacks functions just call kfree() on the base structure. + * These functions are trivial, but their size adds up, and furthermore + * when they are used in a kernel module, that module must invoke the + * high-latency rcu_barrier() function at module-unload time. + * + * The kfree_rcu() function handles this issue. Rather than encoding a + * function address in the embedded rcu_head structure, kfree_rcu() instead + * encodes the offset of the rcu_head structure within the base structure. + * Because the functions are not allowed in the low-order 4096 bytes of + * kernel virtual memory, offsets up to 4095 bytes can be accommodated. + * If the offset is larger than 4095 bytes, a compile-time error will + * be generated in __kfree_rcu(). If this error is triggered, you can + * either fall back to use of call_rcu() or rearrange the structure to + * position the rcu_head structure into the first 4096 bytes. + * + * Note that the allowable offset might decrease in the future, for example, + * to allow something like kmem_cache_free_rcu(). + */ +#define kfree_rcu(ptr, rcu_head) \ + __kfree_rcu(&((ptr)->rcu_head), offsetof(typeof(*(ptr)), rcu_head)) + +#endif /* __LINUX_RCUPDATE_DEFS_H */ -- 1.7.4 ^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH 2/4] rcu: make rcudpate.h can use struct task_struct. 2011-04-07 0:30 ` Paul E. McKenney 2011-04-07 5:49 ` Lai Jiangshan 2011-04-07 7:05 ` [PATCH 1/4] rcu: split rcupdate.h Lai Jiangshan @ 2011-04-07 7:07 ` Lai Jiangshan 2011-04-07 7:07 ` [PATCH 3/4] rcu: introduce task_rcu_struct and move task's RCU code to rcupdate_defs.h Lai Jiangshan 2011-04-07 7:08 ` [PATCH 4/4] rcu: declare preemptible __rcu_read_[un]lock() as inline function Lai Jiangshan 4 siblings, 0 replies; 67+ messages in thread From: Lai Jiangshan @ 2011-04-07 7:07 UTC (permalink / raw) To: paulmck Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt Step1: Ensure sched.h that it does not include rcudpate.h. rcupdate.h is so generic that it is included in many headers. sched.h is so complex that it includes many headers. So we have to find out all header files which directly include rcupdate.h and are directly/indirectly included by sched.h. I found out these files by a script(No False Positive, No False Negetive): 2 include/linux/aio.h ---> include/linux/rcupdate.h 2 include/linux/key.h ---> include/linux/rcupdate.h 2 include/linux/pid.h ---> include/linux/rcupdate.h 2 include/linux/rculist.h ---> include/linux/rcupdate.h 2 include/linux/sched.h ---> include/linux/rcupdate.h 2 include/linux/sem.h ---> include/linux/rcupdate.h 2 include/linux/sysctl.h ---> include/linux/rcupdate.h So the step 1 changes these files just includes rcupdate_defs.h Step2: Ensure rcupdate.h included in the files which use rcu_read_lock*() When the step 1 is done, the kernel build will fail, because some files just include the above files and use rcu_read_lock*() without rcupdate.h included. # vi ~/is_include.py ########### command1 import re import sys include_re = re.compile(r'#\s*include\s*<([_\-\w]*/)([/\._\-\w]*)>') searched = {} def is_include(f, depth): if f == ('include/linux/', 'rcupdate.h'): sys.exit(0) if depth <= 0: return if searched.has_key(f): if searched[f] > depth: return searched[f] = depth try: ff = file(f[0]+f[1]) except: return () for ch in include_re.findall(ff.read()): if ch[0] != 'asm/': is_include(('include/' + ch[0], ch[1]), depth - 1) """usage: is_include.py file_name depth """ is_include(('', sys.argv[1]), int(sys.argv[2])) sys.exit(1) # vi ~/is_include.sh ############## command2 #!/bin/bash depth="$1" find -name "*.c" | xargs grep -l rcu_read_lock | while read file do python ~/is_include.py "$file" "$depth" || echo "$file" done # ~/is_include.sh 3 ################# command3(after step1 is done) ./drivers/base/power/wakeup.c ./drivers/misc/sgi-gru/grutlbpurge.c ./drivers/char/tpm/tpm.c ./drivers/net/wireless/iwlwifi/iwl-agn-rxon.c ./drivers/net/wireless/ath/ath9k/xmit.c ./drivers/net/wireless/ath/ath9k/htc_drv_main.c ./drivers/net/wireless/ath/ath9k/htc_drv_txrx.c ./drivers/net/wireless/ath/carl9170/mac.c ./drivers/net/wireless/ath/ar9170/mac.c ./drivers/staging/tidspbridge/pmgr/dspapi.c ./drivers/s390/cio/qdio_thinint.c ./lib/is_single_threaded.c ./fs/autofs4/waitq.c ./fs/ecryptfs/messaging.c ./fs/xfs/xfs_mount.c ./fs/xfs/linux-2.6/xfs_sync.c ./fs/xfs/xfs_iget.c ./fs/xfs/xfs_inode.c ./arch/ia64/sn/kernel/irq.c ./security/tomoyo/common.c ./security/apparmor/audit.c ./kernel/sched_fair.c ./kernel/tsacct.c ./kernel/sched_debug.c ./kernel/pid_namespace.c ./kernel/futex_compat.c ./kernel/sched_rt.c ./net/rds/ib_rdma.c ./net/rds/connection.c ./net/mac80211/debugfs_sta.c ./net/mac80211/mesh.c ./net/mac80211/mesh_hwmp.c ./net/mac80211/debugfs_key.c ./net/mac80211/mesh_plink.c ./net/batman-adv/send.c ./net/batman-adv/routing.c ./net/batman-adv/main.c ./net/batman-adv/originator.c (The simple scripts allow False Positive, disallow False Negetive) After manual check, only drivers/staging/tidspbridge/pmgr/dspapi.c kernel/futex_compat.c kernel/tsacct.c lib/is_single_threaded.c are required to add "#include <linux/rcupdate.h>" Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> --- drivers/staging/tidspbridge/pmgr/dspapi.c | 1 + include/linux/aio.h | 2 +- include/linux/key.h | 2 +- include/linux/pid.h | 2 +- include/linux/rculist.h | 2 +- include/linux/sched.h | 2 +- include/linux/sem.h | 2 +- include/linux/sysctl.h | 2 +- kernel/futex_compat.c | 1 + kernel/tsacct.c | 1 + lib/is_single_threaded.c | 1 + 11 files changed, 11 insertions(+), 7 deletions(-) diff --git a/drivers/staging/tidspbridge/pmgr/dspapi.c b/drivers/staging/tidspbridge/pmgr/dspapi.c index 86ca785..558654e 100644 --- a/drivers/staging/tidspbridge/pmgr/dspapi.c +++ b/drivers/staging/tidspbridge/pmgr/dspapi.c @@ -17,6 +17,7 @@ * WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE. */ #include <linux/types.h> +#include <linux/rcupdate.h> /* ----------------------------------- Host OS */ #include <dspbridge/host_os.h> diff --git a/include/linux/aio.h b/include/linux/aio.h index 7a8db41..1033572 100644 --- a/include/linux/aio.h +++ b/include/linux/aio.h @@ -5,7 +5,7 @@ #include <linux/workqueue.h> #include <linux/aio_abi.h> #include <linux/uio.h> -#include <linux/rcupdate.h> +#include <linux/rcupdate_defs.h> #include <asm/atomic.h> diff --git a/include/linux/key.h b/include/linux/key.h index 3db0adc..8f18036 100644 --- a/include/linux/key.h +++ b/include/linux/key.h @@ -18,7 +18,7 @@ #include <linux/types.h> #include <linux/list.h> #include <linux/rbtree.h> -#include <linux/rcupdate.h> +#include <linux/rcupdate_defs.h> #include <linux/sysctl.h> #include <linux/rwsem.h> #include <asm/atomic.h> diff --git a/include/linux/pid.h b/include/linux/pid.h index 49f1c2f..12d652c 100644 --- a/include/linux/pid.h +++ b/include/linux/pid.h @@ -1,7 +1,7 @@ #ifndef _LINUX_PID_H #define _LINUX_PID_H -#include <linux/rcupdate.h> +#include <linux/rcupdate_defs.h> enum pid_type { diff --git a/include/linux/rculist.h b/include/linux/rculist.h index 2dea94f..265e22d 100644 --- a/include/linux/rculist.h +++ b/include/linux/rculist.h @@ -7,7 +7,7 @@ * RCU-protected list version */ #include <linux/list.h> -#include <linux/rcupdate.h> +#include <linux/rcupdate_defs.h> /* * Why is there no list_empty_rcu()? Because list_empty() serves this diff --git a/include/linux/sched.h b/include/linux/sched.h index 777d8a5..69db9e7 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -78,7 +78,7 @@ struct sched_param { #include <linux/topology.h> #include <linux/proportions.h> #include <linux/seccomp.h> -#include <linux/rcupdate.h> +#include <linux/rcupdate_defs.h> #include <linux/rculist.h> #include <linux/rtmutex.h> diff --git a/include/linux/sem.h b/include/linux/sem.h index f2961af..98d96e8 100644 --- a/include/linux/sem.h +++ b/include/linux/sem.h @@ -78,7 +78,7 @@ struct seminfo { #ifdef __KERNEL__ #include <asm/atomic.h> -#include <linux/rcupdate.h> +#include <linux/rcupdate_defs.h> #include <linux/cache.h> struct task_struct; diff --git a/include/linux/sysctl.h b/include/linux/sysctl.h index 11684d9..553f127 100644 --- a/include/linux/sysctl.h +++ b/include/linux/sysctl.h @@ -930,7 +930,7 @@ enum #ifdef __KERNEL__ #include <linux/list.h> -#include <linux/rcupdate.h> +#include <linux/rcupdate_defs.h> /* For the /proc/sys support */ struct ctl_table; diff --git a/kernel/futex_compat.c b/kernel/futex_compat.c index a7934ac..4485705 100644 --- a/kernel/futex_compat.c +++ b/kernel/futex_compat.c @@ -10,6 +10,7 @@ #include <linux/compat.h> #include <linux/nsproxy.h> #include <linux/futex.h> +#include <linux/rcupdate.h> #include <asm/uaccess.h> diff --git a/kernel/tsacct.c b/kernel/tsacct.c index 24dc60d..2617467 100644 --- a/kernel/tsacct.c +++ b/kernel/tsacct.c @@ -21,6 +21,7 @@ #include <linux/tsacct_kern.h> #include <linux/acct.h> #include <linux/jiffies.h> +#include <linux/rcupdate.h> #include <linux/mm.h> /* diff --git a/lib/is_single_threaded.c b/lib/is_single_threaded.c index bd2bea9..efcb711 100644 --- a/lib/is_single_threaded.c +++ b/lib/is_single_threaded.c @@ -11,6 +11,7 @@ */ #include <linux/sched.h> +#include <linux/rcupdate.h> /* * Returns true if the task does not share ->mm with another thread/process. -- 1.7.4 ^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH 3/4] rcu: introduce task_rcu_struct and move task's RCU code to rcupdate_defs.h 2011-04-07 0:30 ` Paul E. McKenney ` (2 preceding siblings ...) 2011-04-07 7:07 ` [PATCH 2/4] rcu: make rcudpate.h can use struct task_struct Lai Jiangshan @ 2011-04-07 7:07 ` Lai Jiangshan 2011-04-07 13:40 ` Alexey Dobriyan 2011-04-07 7:08 ` [PATCH 4/4] rcu: declare preemptible __rcu_read_[un]lock() as inline function Lai Jiangshan 4 siblings, 1 reply; 67+ messages in thread From: Lai Jiangshan @ 2011-04-07 7:07 UTC (permalink / raw) To: paulmck Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt Add struct task_rcu_struct to make code clearer and move them to rcupdate_defs.h which help us for code maitainment. Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> --- include/linux/init_task.h | 25 +---------------- include/linux/rcupdate.h | 34 ++++++++++++++++++----- include/linux/rcupdate_defs.h | 60 +++++++++++++++++++++++++++++++++++++++++ include/linux/sched.h | 39 +-------------------------- kernel/rcutiny_plugin.h | 26 +++++++++-------- kernel/rcutree_plugin.h | 37 +++++++++++++------------ 6 files changed, 122 insertions(+), 99 deletions(-) diff --git a/include/linux/init_task.h b/include/linux/init_task.h index caa151f..1749002 100644 --- a/include/linux/init_task.h +++ b/include/linux/init_task.h @@ -90,29 +90,6 @@ extern struct group_info init_groups; */ # define CAP_INIT_BSET CAP_FULL_SET -#ifdef CONFIG_RCU_BOOST -#define INIT_TASK_RCU_BOOST() \ - .rcu_boost_mutex = NULL, -#else -#define INIT_TASK_RCU_BOOST() -#endif -#ifdef CONFIG_TREE_PREEMPT_RCU -#define INIT_TASK_RCU_TREE_PREEMPT() \ - .rcu_blocked_node = NULL, -#else -#define INIT_TASK_RCU_TREE_PREEMPT(tsk) -#endif -#ifdef CONFIG_PREEMPT_RCU -#define INIT_TASK_RCU_PREEMPT(tsk) \ - .rcu_read_lock_nesting = 0, \ - .rcu_read_unlock_special = 0, \ - .rcu_node_entry = LIST_HEAD_INIT(tsk.rcu_node_entry), \ - INIT_TASK_RCU_TREE_PREEMPT() \ - INIT_TASK_RCU_BOOST() -#else -#define INIT_TASK_RCU_PREEMPT(tsk) -#endif - extern struct cred init_cred; #ifdef CONFIG_PERF_EVENTS @@ -191,7 +168,7 @@ extern struct cred init_cred; INIT_LOCKDEP \ INIT_FTRACE_GRAPH \ INIT_TRACE_RECURSION \ - INIT_TASK_RCU_PREEMPT(tsk) \ + .task_rcu_struct = INIT_TASK_RCU_STRUCT(tsk.task_rcu_struct), \ } diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index c25d2a6..b24b288 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -34,6 +34,29 @@ #define __LINUX_RCUPDATE_H #include <linux/rcupdate_defs.h> +#include <linux/sched.h> + +static inline +struct task_rcu_struct *task_rcu_struct(struct task_struct *t) +{ + return &t->task_rcu_struct; +} + +static inline +struct task_struct *task_of_task_rcu(struct task_rcu_struct *task_rcu) +{ + return container_of(task_rcu, struct task_struct, task_rcu_struct); +} + +static inline struct task_rcu_struct *current_task_rcu_struct(void) +{ + return task_rcu_struct(current); +} + +static inline void rcu_copy_process(struct task_struct *tsk) +{ + init_task_rcu_struct(task_rcu_struct(tsk)); +} static inline void __rcu_read_lock_bh(void) { @@ -50,13 +73,10 @@ static inline void __rcu_read_unlock_bh(void) extern void __rcu_read_lock(void); extern void __rcu_read_unlock(void); -/* - * Defined as a macro as it is a very low level header included from - * areas that don't even know about current. This gives the rcu_read_lock() - * nesting depth, but makes sense only if CONFIG_PREEMPT_RCU -- in other - * types of kernel builds, the rcu_read_lock() nesting depth is unknowable. - */ -#define rcu_preempt_depth() (current->rcu_read_lock_nesting) +static inline int rcu_preempt_depth(void) +{ + return current_task_rcu_struct()->rcu_read_lock_nesting; +} #else /* #ifdef CONFIG_PREEMPT_RCU */ diff --git a/include/linux/rcupdate_defs.h b/include/linux/rcupdate_defs.h index ae84745..e4e1567 100644 --- a/include/linux/rcupdate_defs.h +++ b/include/linux/rcupdate_defs.h @@ -93,6 +93,66 @@ static inline void rcu_exit_nohz(void) #endif /* #else #ifdef CONFIG_NO_HZ */ +/* Special flags for preemptible RCU */ +#define RCU_READ_UNLOCK_BLOCKED (1 << 0) /* blocked while in RCU read-side. */ +#define RCU_READ_UNLOCK_BOOSTED (1 << 1) /* boosted while in RCU read-side. */ +#define RCU_READ_UNLOCK_NEED_QS (1 << 2) /* RCU core needs CPU response. */ + +struct task_struct; + +struct task_rcu_struct { +#ifdef CONFIG_PREEMPT_RCU + int rcu_read_lock_nesting; + char rcu_read_unlock_special; + struct list_head rcu_node_entry; +#ifdef CONFIG_TREE_PREEMPT_RCU + struct rcu_node *rcu_blocked_node; +#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ +#ifdef CONFIG_RCU_BOOST + struct rt_mutex *rcu_boost_mutex; +#endif /* #ifdef CONFIG_RCU_BOOST */ +#endif /* #ifdef CONFIG_PREEMPT_RCU */ +}; + +#ifdef CONFIG_RCU_BOOST +#define INIT_TASK_RCU_BOOST() \ + .rcu_boost_mutex = NULL, +#else +#define INIT_TASK_RCU_BOOST() +#endif +#ifdef CONFIG_TREE_PREEMPT_RCU +#define INIT_TASK_RCU_TREE_PREEMPT() \ + .rcu_blocked_node = NULL, +#else +#define INIT_TASK_RCU_TREE_PREEMPT(tsk) +#endif +#ifdef CONFIG_PREEMPT_RCU +#define INIT_TASK_RCU_STRUCT(task_rcu) { \ + .rcu_read_lock_nesting = 0, \ + .rcu_read_unlock_special = 0, \ + .rcu_node_entry = LIST_HEAD_INIT(task_rcu.rcu_node_entry),\ + INIT_TASK_RCU_TREE_PREEMPT() \ + INIT_TASK_RCU_BOOST() \ + } +#else +#define INIT_TASK_RCU_STRUCT(task_rcu) {} +#endif + +static inline void init_task_rcu_struct(struct task_rcu_struct *task_rcu) +{ +#ifdef CONFIG_PREEMPT_RCU + task_rcu->rcu_read_lock_nesting = 0; + task_rcu->rcu_read_unlock_special = 0; + INIT_LIST_HEAD(&task_rcu->rcu_node_entry); +#ifdef CONFIG_TREE_PREEMPT_RCU + task_rcu->rcu_blocked_node = NULL; +#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ +#ifdef CONFIG_RCU_BOOST + task_rcu->rcu_boost_mutex = NULL; +#endif /* #ifdef CONFIG_RCU_BOOST */ +#endif /* CONFIG_PREEMPT_RCU */ +} + #if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) #include <linux/rcutree.h> #elif defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU) diff --git a/include/linux/sched.h b/include/linux/sched.h index 69db9e7..5252e48 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1232,17 +1232,7 @@ struct task_struct { unsigned int policy; cpumask_t cpus_allowed; -#ifdef CONFIG_PREEMPT_RCU - int rcu_read_lock_nesting; - char rcu_read_unlock_special; - struct list_head rcu_node_entry; -#endif /* #ifdef CONFIG_PREEMPT_RCU */ -#ifdef CONFIG_TREE_PREEMPT_RCU - struct rcu_node *rcu_blocked_node; -#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ -#ifdef CONFIG_RCU_BOOST - struct rt_mutex *rcu_boost_mutex; -#endif /* #ifdef CONFIG_RCU_BOOST */ + struct task_rcu_struct task_rcu_struct; #if defined(CONFIG_SCHEDSTATS) || defined(CONFIG_TASK_DELAY_ACCT) struct sched_info sched_info; @@ -1772,33 +1762,6 @@ extern void thread_group_times(struct task_struct *p, cputime_t *ut, cputime_t * #define tsk_used_math(p) ((p)->flags & PF_USED_MATH) #define used_math() tsk_used_math(current) -#ifdef CONFIG_PREEMPT_RCU - -#define RCU_READ_UNLOCK_BLOCKED (1 << 0) /* blocked while in RCU read-side. */ -#define RCU_READ_UNLOCK_BOOSTED (1 << 1) /* boosted while in RCU read-side. */ -#define RCU_READ_UNLOCK_NEED_QS (1 << 2) /* RCU core needs CPU response. */ - -static inline void rcu_copy_process(struct task_struct *p) -{ - p->rcu_read_lock_nesting = 0; - p->rcu_read_unlock_special = 0; -#ifdef CONFIG_TREE_PREEMPT_RCU - p->rcu_blocked_node = NULL; -#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ -#ifdef CONFIG_RCU_BOOST - p->rcu_boost_mutex = NULL; -#endif /* #ifdef CONFIG_RCU_BOOST */ - INIT_LIST_HEAD(&p->rcu_node_entry); -} - -#else - -static inline void rcu_copy_process(struct task_struct *p) -{ -} - -#endif - #ifdef CONFIG_SMP extern int set_cpus_allowed_ptr(struct task_struct *p, const struct cpumask *new_mask); diff --git a/kernel/rcutiny_plugin.h b/kernel/rcutiny_plugin.h index f259c67..425e892 100644 --- a/kernel/rcutiny_plugin.h +++ b/kernel/rcutiny_plugin.h @@ -150,7 +150,7 @@ static int rcu_cpu_blocking_cur_gp(void) */ static int rcu_preempt_running_reader(void) { - return current->rcu_read_lock_nesting; + return current_task_rcu_struct()->rcu_read_lock_nesting; } /* @@ -192,7 +192,7 @@ static int rcu_preempt_gp_in_progress(void) * Advance a ->blkd_tasks-list pointer to the next entry, instead * returning NULL if at the end of the list. */ -static struct list_head *rcu_next_node_entry(struct task_struct *t) +static struct list_head *rcu_next_node_entry(struct task_rcu_struct *t) { struct list_head *np; @@ -255,7 +255,7 @@ static int rcu_boost(void) { unsigned long flags; struct rt_mutex mtx; - struct task_struct *t; + struct task_rcu_struct *t; struct list_head *tb; if (rcu_preempt_ctrlblk.boost_tasks == NULL && @@ -298,8 +298,8 @@ static int rcu_boost(void) * simply acquiring this artificial rt_mutex will boost task * t's priority. (Thanks to tglx for suggesting this approach!) */ - t = container_of(tb, struct task_struct, rcu_node_entry); - rt_mutex_init_proxy_locked(&mtx, t); + t = container_of(tb, struct task_rcu_struct, rcu_node_entry); + rt_mutex_init_proxy_locked(&mtx, task_of_task_rcu(t)); t->rcu_boost_mutex = &mtx; t->rcu_read_unlock_special |= RCU_READ_UNLOCK_BOOSTED; raw_local_irq_restore(flags); @@ -402,9 +402,11 @@ static void rcu_preempt_boost_start_gp(void) */ static void rcu_preempt_cpu_qs(void) { + struct task_rcu_struct *t = current_task_rcu_struct(); + /* Record both CPU and task as having responded to current GP. */ rcu_preempt_ctrlblk.gpcpu = rcu_preempt_ctrlblk.gpnum; - current->rcu_read_unlock_special &= ~RCU_READ_UNLOCK_NEED_QS; + t->rcu_read_unlock_special &= ~RCU_READ_UNLOCK_NEED_QS; /* If there is no GP then there is nothing more to do. */ if (!rcu_preempt_gp_in_progress()) @@ -473,7 +475,7 @@ static void rcu_preempt_start_gp(void) */ void rcu_preempt_note_context_switch(void) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); unsigned long flags; local_irq_save(flags); /* must exclude scheduler_tick(). */ @@ -518,7 +520,7 @@ void rcu_preempt_note_context_switch(void) */ void __rcu_read_lock(void) { - current->rcu_read_lock_nesting++; + current_task_rcu_struct()->rcu_read_lock_nesting++; barrier(); /* needed if we ever invoke rcu_read_lock in rcutiny.c */ } EXPORT_SYMBOL_GPL(__rcu_read_lock); @@ -528,7 +530,7 @@ EXPORT_SYMBOL_GPL(__rcu_read_lock); * notify RCU core processing or task having blocked during the RCU * read-side critical section. */ -static void rcu_read_unlock_special(struct task_struct *t) +static void rcu_read_unlock_special(struct task_rcu_struct *t) { int empty; int empty_exp; @@ -617,7 +619,7 @@ static void rcu_read_unlock_special(struct task_struct *t) */ void __rcu_read_unlock(void) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); barrier(); /* needed if we ever invoke rcu_read_unlock in rcutiny.c */ --t->rcu_read_lock_nesting; @@ -640,7 +642,7 @@ EXPORT_SYMBOL_GPL(__rcu_read_unlock); */ static void rcu_preempt_check_callbacks(void) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); if (rcu_preempt_gp_in_progress() && (!rcu_preempt_running_reader() || @@ -841,7 +843,7 @@ int rcu_preempt_needs_cpu(void) */ void exit_rcu(void) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); if (t->rcu_read_lock_nesting == 0) return; diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h index 69b8ab8..17e84f5 100644 --- a/kernel/rcutree_plugin.h +++ b/kernel/rcutree_plugin.h @@ -120,11 +120,12 @@ EXPORT_SYMBOL_GPL(rcu_force_quiescent_state); static void rcu_preempt_qs(int cpu) { struct rcu_data *rdp = &per_cpu(rcu_preempt_data, cpu); + struct task_rcu_struct *t = current_task_rcu_struct(); rdp->passed_quiesc_completed = rdp->gpnum - 1; barrier(); rdp->passed_quiesc = 1; - current->rcu_read_unlock_special &= ~RCU_READ_UNLOCK_NEED_QS; + t->rcu_read_unlock_special &= ~RCU_READ_UNLOCK_NEED_QS; } /* @@ -142,7 +143,7 @@ static void rcu_preempt_qs(int cpu) */ static void rcu_preempt_note_context_switch(int cpu) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); unsigned long flags; struct rcu_data *rdp; struct rcu_node *rnp; @@ -213,7 +214,7 @@ static void rcu_preempt_note_context_switch(int cpu) */ void __rcu_read_lock(void) { - current->rcu_read_lock_nesting++; + current_task_rcu_struct()->rcu_read_lock_nesting++; barrier(); /* needed if we ever invoke rcu_read_lock in rcutree.c */ } EXPORT_SYMBOL_GPL(__rcu_read_lock); @@ -268,7 +269,7 @@ static void rcu_report_unblock_qs_rnp(struct rcu_node *rnp, unsigned long flags) * Advance a ->blkd_tasks-list pointer to the next entry, instead * returning NULL if at the end of the list. */ -static struct list_head *rcu_next_node_entry(struct task_struct *t, +static struct list_head *rcu_next_node_entry(struct task_rcu_struct *t, struct rcu_node *rnp) { struct list_head *np; @@ -284,7 +285,7 @@ static struct list_head *rcu_next_node_entry(struct task_struct *t, * notify RCU core processing or task having blocked during the RCU * read-side critical section. */ -static void rcu_read_unlock_special(struct task_struct *t) +static void rcu_read_unlock_special(struct task_rcu_struct *t) { int empty; int empty_exp; @@ -384,7 +385,7 @@ static void rcu_read_unlock_special(struct task_struct *t) */ void __rcu_read_unlock(void) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); barrier(); /* needed if we ever invoke rcu_read_unlock in rcutree.c */ --t->rcu_read_lock_nesting; @@ -407,15 +408,15 @@ EXPORT_SYMBOL_GPL(__rcu_read_unlock); static void rcu_print_detail_task_stall_rnp(struct rcu_node *rnp) { unsigned long flags; - struct task_struct *t; + struct task_rcu_struct *t; if (!rcu_preempt_blocked_readers_cgp(rnp)) return; raw_spin_lock_irqsave(&rnp->lock, flags); t = list_entry(rnp->gp_tasks, - struct task_struct, rcu_node_entry); + struct task_rcu_struct, rcu_node_entry); list_for_each_entry_continue(t, &rnp->blkd_tasks, rcu_node_entry) - sched_show_task(t); + sched_show_task(task_of_task_rcu(t)); raw_spin_unlock_irqrestore(&rnp->lock, flags); } @@ -446,14 +447,14 @@ static void rcu_print_detail_task_stall(struct rcu_state *rsp) */ static void rcu_print_task_stall(struct rcu_node *rnp) { - struct task_struct *t; + struct task_rcu_struct *t; if (!rcu_preempt_blocked_readers_cgp(rnp)) return; t = list_entry(rnp->gp_tasks, - struct task_struct, rcu_node_entry); + struct task_rcu_struct, rcu_node_entry); list_for_each_entry_continue(t, &rnp->blkd_tasks, rcu_node_entry) - printk(" P%d", t->pid); + printk(" P%d", task_of_task_rcu(t)->pid); } /* @@ -508,7 +509,7 @@ static int rcu_preempt_offline_tasks(struct rcu_state *rsp, struct list_head *lp_root; int retval = 0; struct rcu_node *rnp_root = rcu_get_root(rsp); - struct task_struct *t; + struct task_rcu_struct *t; if (rnp == rnp_root) { WARN_ONCE(1, "Last CPU thought to be offlined?"); @@ -581,7 +582,7 @@ static void rcu_preempt_offline_cpu(int cpu) */ static void rcu_preempt_check_callbacks(int cpu) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); if (t->rcu_read_lock_nesting == 0) { rcu_preempt_qs(cpu); @@ -851,7 +852,7 @@ static void __init __rcu_init_preempt(void) */ void exit_rcu(void) { - struct task_struct *t = current; + struct task_rcu_struct *t = current_task_rcu_struct(); if (t->rcu_read_lock_nesting == 0) return; @@ -1110,7 +1111,7 @@ static int rcu_boost(struct rcu_node *rnp) { unsigned long flags; struct rt_mutex mtx; - struct task_struct *t; + struct task_rcu_struct *t; struct list_head *tb; if (rnp->exp_tasks == NULL && rnp->boost_tasks == NULL) @@ -1158,8 +1159,8 @@ static int rcu_boost(struct rcu_node *rnp) * and task t's exiting its outermost RCU read-side critical * section. */ - t = container_of(tb, struct task_struct, rcu_node_entry); - rt_mutex_init_proxy_locked(&mtx, t); + t = container_of(tb, struct task_rcu_struct, rcu_node_entry); + rt_mutex_init_proxy_locked(&mtx, task_of_task_rcu(t)); t->rcu_boost_mutex = &mtx; t->rcu_read_unlock_special |= RCU_READ_UNLOCK_BOOSTED; raw_spin_unlock_irqrestore(&rnp->lock, flags); -- 1.7.4 ^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [PATCH 3/4] rcu: introduce task_rcu_struct and move task's RCU code to rcupdate_defs.h 2011-04-07 7:07 ` [PATCH 3/4] rcu: introduce task_rcu_struct and move task's RCU code to rcupdate_defs.h Lai Jiangshan @ 2011-04-07 13:40 ` Alexey Dobriyan 0 siblings, 0 replies; 67+ messages in thread From: Alexey Dobriyan @ 2011-04-07 13:40 UTC (permalink / raw) To: Lai Jiangshan Cc: paulmck, H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt On Thu, Apr 7, 2011 at 10:07 AM, Lai Jiangshan <laijs@cn.fujitsu.com> wrote: > Add struct task_rcu_struct to make code clearer and move > them to rcupdate_defs.h which help us for code maitainment. > --- a/include/linux/sched.h > +++ b/include/linux/sched.h > @@ -1232,17 +1232,7 @@ struct task_struct { > unsigned int policy; > cpumask_t cpus_allowed; > > -#ifdef CONFIG_PREEMPT_RCU > - int rcu_read_lock_nesting; > - char rcu_read_unlock_special; > - struct list_head rcu_node_entry; > -#endif /* #ifdef CONFIG_PREEMPT_RCU */ > -#ifdef CONFIG_TREE_PREEMPT_RCU > - struct rcu_node *rcu_blocked_node; > -#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ > -#ifdef CONFIG_RCU_BOOST > - struct rt_mutex *rcu_boost_mutex; > -#endif /* #ifdef CONFIG_RCU_BOOST */ > + struct task_rcu_struct task_rcu_struct; Oh, wow, all these movements are for this. You could do struct task_rcu { ... } rcu; inside sched.h, drop pointless 1:1 current_() et al wrappers and avoid all this mess. ^ permalink raw reply [flat|nested] 67+ messages in thread
* [PATCH 4/4] rcu: declare preemptible __rcu_read_[un]lock() as inline function 2011-04-07 0:30 ` Paul E. McKenney ` (3 preceding siblings ...) 2011-04-07 7:07 ` [PATCH 3/4] rcu: introduce task_rcu_struct and move task's RCU code to rcupdate_defs.h Lai Jiangshan @ 2011-04-07 7:08 ` Lai Jiangshan 4 siblings, 0 replies; 67+ messages in thread From: Lai Jiangshan @ 2011-04-07 7:08 UTC (permalink / raw) To: paulmck Cc: H. Peter Anvin, Peter Zijlstra, Michal Marek, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, linux-kernel, Thomas Gleixner, Steven Rostedt __rcu_read_[un]lock() are so simple functions, make them inlined. Also rmove dumplicated code. Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> --- include/linux/rcupdate.h | 36 ++++++++++++++++++++++++++++++++++-- kernel/rcutiny_plugin.h | 38 ++------------------------------------ kernel/rcutree_plugin.h | 38 ++------------------------------------ 3 files changed, 38 insertions(+), 74 deletions(-) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index b24b288..1eb38b9 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -70,8 +70,40 @@ static inline void __rcu_read_unlock_bh(void) #ifdef CONFIG_PREEMPT_RCU -extern void __rcu_read_lock(void); -extern void __rcu_read_unlock(void); +void rcu_read_unlock_special(struct task_rcu_struct *t); + +/* + * Preemptible RCU implementation for rcu_read_lock(). + * Just increment ->rcu_read_lock_nesting, shared state will be updated + * if we block. + */ +static inline void __rcu_read_lock(void) +{ + current_task_rcu_struct()->rcu_read_lock_nesting++; + barrier(); +} + +/* + * Preemptible RCU implementation for rcu_read_unlock(). + * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost + * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then + * invoke rcu_read_unlock_special() to clean up after a context switch + * in an RCU read-side critical section and other special cases. + */ +static inline void __rcu_read_unlock(void) +{ + struct task_rcu_struct *t = current_task_rcu_struct(); + + barrier(); + --t->rcu_read_lock_nesting; + barrier(); /* decrement before load of ->rcu_read_unlock_special */ + if (t->rcu_read_lock_nesting == 0 && + unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) + rcu_read_unlock_special(t); +#ifdef CONFIG_PROVE_LOCKING + WARN_ON_ONCE(ACCESS_ONCE(t->rcu_read_lock_nesting) < 0); +#endif /* #ifdef CONFIG_PROVE_LOCKING */ +} static inline int rcu_preempt_depth(void) { diff --git a/kernel/rcutiny_plugin.h b/kernel/rcutiny_plugin.h index 425e892..49a7699 100644 --- a/kernel/rcutiny_plugin.h +++ b/kernel/rcutiny_plugin.h @@ -514,23 +514,11 @@ void rcu_preempt_note_context_switch(void) } /* - * Tiny-preemptible RCU implementation for rcu_read_lock(). - * Just increment ->rcu_read_lock_nesting, shared state will be updated - * if we block. - */ -void __rcu_read_lock(void) -{ - current_task_rcu_struct()->rcu_read_lock_nesting++; - barrier(); /* needed if we ever invoke rcu_read_lock in rcutiny.c */ -} -EXPORT_SYMBOL_GPL(__rcu_read_lock); - -/* * Handle special cases during rcu_read_unlock(), such as needing to * notify RCU core processing or task having blocked during the RCU * read-side critical section. */ -static void rcu_read_unlock_special(struct task_rcu_struct *t) +void rcu_read_unlock_special(struct task_rcu_struct *t) { int empty; int empty_exp; @@ -609,29 +597,7 @@ static void rcu_read_unlock_special(struct task_rcu_struct *t) #endif /* #ifdef CONFIG_RCU_BOOST */ local_irq_restore(flags); } - -/* - * Tiny-preemptible RCU implementation for rcu_read_unlock(). - * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost - * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then - * invoke rcu_read_unlock_special() to clean up after a context switch - * in an RCU read-side critical section and other special cases. - */ -void __rcu_read_unlock(void) -{ - struct task_rcu_struct *t = current_task_rcu_struct(); - - barrier(); /* needed if we ever invoke rcu_read_unlock in rcutiny.c */ - --t->rcu_read_lock_nesting; - barrier(); /* decrement before load of ->rcu_read_unlock_special */ - if (t->rcu_read_lock_nesting == 0 && - unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) - rcu_read_unlock_special(t); -#ifdef CONFIG_PROVE_LOCKING - WARN_ON_ONCE(t->rcu_read_lock_nesting < 0); -#endif /* #ifdef CONFIG_PROVE_LOCKING */ -} -EXPORT_SYMBOL_GPL(__rcu_read_unlock); +EXPORT_SYMBOL_GPL(rcu_read_unlock_special); /* * Check for a quiescent state from the current CPU. When a task blocks, diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h index 17e84f5..31e408f 100644 --- a/kernel/rcutree_plugin.h +++ b/kernel/rcutree_plugin.h @@ -208,18 +208,6 @@ static void rcu_preempt_note_context_switch(int cpu) } /* - * Tree-preemptible RCU implementation for rcu_read_lock(). - * Just increment ->rcu_read_lock_nesting, shared state will be updated - * if we block. - */ -void __rcu_read_lock(void) -{ - current_task_rcu_struct()->rcu_read_lock_nesting++; - barrier(); /* needed if we ever invoke rcu_read_lock in rcutree.c */ -} -EXPORT_SYMBOL_GPL(__rcu_read_lock); - -/* * Check for preempted RCU readers blocking the current grace period * for the specified rcu_node structure. If the caller needs a reliable * answer, it must hold the rcu_node's ->lock. @@ -285,7 +273,7 @@ static struct list_head *rcu_next_node_entry(struct task_rcu_struct *t, * notify RCU core processing or task having blocked during the RCU * read-side critical section. */ -static void rcu_read_unlock_special(struct task_rcu_struct *t) +void rcu_read_unlock_special(struct task_rcu_struct *t) { int empty; int empty_exp; @@ -375,29 +363,7 @@ static void rcu_read_unlock_special(struct task_rcu_struct *t) local_irq_restore(flags); } } - -/* - * Tree-preemptible RCU implementation for rcu_read_unlock(). - * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost - * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then - * invoke rcu_read_unlock_special() to clean up after a context switch - * in an RCU read-side critical section and other special cases. - */ -void __rcu_read_unlock(void) -{ - struct task_rcu_struct *t = current_task_rcu_struct(); - - barrier(); /* needed if we ever invoke rcu_read_unlock in rcutree.c */ - --t->rcu_read_lock_nesting; - barrier(); /* decrement before load of ->rcu_read_unlock_special */ - if (t->rcu_read_lock_nesting == 0 && - unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) - rcu_read_unlock_special(t); -#ifdef CONFIG_PROVE_LOCKING - WARN_ON_ONCE(ACCESS_ONCE(t->rcu_read_lock_nesting) < 0); -#endif /* #ifdef CONFIG_PROVE_LOCKING */ -} -EXPORT_SYMBOL_GPL(__rcu_read_unlock); +EXPORT_SYMBOL_GPL(rcu_read_unlock_special); #ifdef CONFIG_RCU_CPU_STALL_VERBOSE -- 1.7.4 ^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-04-05 23:07 ` Paul E. McKenney 2011-04-06 8:10 ` Peter Zijlstra @ 2011-04-06 17:41 ` Paul E. McKenney 1 sibling, 0 replies; 67+ messages in thread From: Paul E. McKenney @ 2011-04-06 17:41 UTC (permalink / raw) To: Peter Zijlstra Cc: Lai Jiangshan, Michal Marek, H. Peter Anvin, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On Tue, Apr 05, 2011 at 04:07:45PM -0700, Paul E. McKenney wrote: > On Tue, Apr 05, 2011 at 02:54:50PM -0700, Paul E. McKenney wrote: > > On Fri, Apr 01, 2011 at 01:35:49PM +0200, Peter Zijlstra wrote: [ . . . ] > > > There isn't a single rcu_read_lock() user in the include chain of > > > sched.h, therefore its possible to split rcupdate.h such that one part > > > includes the bits needed for sched.h and another part that includes > > > sched.h and provides rcu_read_lock(). > > > > Yep, you can easily split struct rcu_head out into its own header file, > > and then include that into sched.h in place of rcupdate.h without trouble > > (at least in the few builds I tried). > > > > The trouble comes when you take the next step and include sched.h > > into include/linux/rcutree.h, which would be needed to allow > > TREE_PREEMPT_RCU's rcu_read_lock() and rcu_read_unlock() to be > > inlined. This pulls sched.h into files that it was not already > > included into, and we do get build failures (the first of which > > requires changes local to RCU, FWIW). > > > > Of course, it would be good to straighten this out. > > And here is a start -- the build is -much- quieter with this patch. > > The next layer on this onion is the use of the ctl_table typedef > in include/linux/key.h -- which is defined to be struct ctl_table. > Anyone know where struct ctl_table is defined? Color me blind, but I > am not seeing it... Sigh... In include/linux/sysctl.h, but with the opening brace on the next line. Anyway, the problem is that include/linux/sysctl.h and include/linux/key.h indirectly include each other in a mutually recursive fashion. So easy to fix -- just include include/linux/key.h prior to the include of include/linux/sysctl.h in kernel/sysctl_binary.c. Thanx, Paul > ------------------------------------------------------------------------ > > diff --git a/include/linux/rcu_head.h b/include/linux/rcu_head.h > new file mode 100644 > index 0000000..e0b8add > --- /dev/null > +++ b/include/linux/rcu_head.h > @@ -0,0 +1,46 @@ > +/* > + * Read-Copy Update mechanism for mutual exclusion: base structures > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation; either version 2 of the License, or > + * (at your option) any later version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program; if not, write to the Free Software > + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. > + * > + * Copyright IBM Corporation, 2001 > + * > + * Author: Dipankar Sarma <dipankar@in.ibm.com> > + * > + * Based on the original work by Paul McKenney <paulmck@us.ibm.com> > + * and inputs from Rusty Russell, Andrea Arcangeli and Andi Kleen. > + * Papers: > + * http://www.rdrop.com/users/paulmck/paper/rclockpdcsproof.pdf > + * http://lse.sourceforge.net/locking/rclock_OLS.2001.05.01c.sc.pdf (OLS2001) > + * > + * For detailed explanation of Read-Copy Update mechanism see - > + * http://lse.sourceforge.net/locking/rcupdate.html > + * > + */ > + > +#ifndef __LINUX_RCU_HEAD_H > +#define __LINUX_RCU_HEAD_H > + > +/** > + * struct rcu_head - callback structure for use with RCU > + * @next: next update requests in a list > + * @func: actual update function to call after the grace period. > + */ > +struct rcu_head { > + struct rcu_head *next; > + void (*func)(struct rcu_head *head); > +}; > + > +#endif /* __LINUX_RCU_HEAD_H */ > diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h > index ff422d2..ab94fcfb 100644 > --- a/include/linux/rcupdate.h > +++ b/include/linux/rcupdate.h > @@ -52,15 +52,7 @@ extern int rcutorture_runnable; /* for sysctl */ > #define ULONG_CMP_GE(a, b) (ULONG_MAX / 2 >= (a) - (b)) > #define ULONG_CMP_LT(a, b) (ULONG_MAX / 2 < (a) - (b)) > > -/** > - * struct rcu_head - callback structure for use with RCU > - * @next: next update requests in a list > - * @func: actual update function to call after the grace period. > - */ > -struct rcu_head { > - struct rcu_head *next; > - void (*func)(struct rcu_head *head); > -}; > +#include <linux/rcu_head.h> > > /* Exported common interfaces */ > extern void call_rcu_sched(struct rcu_head *head, > @@ -141,14 +133,6 @@ static inline void rcu_exit_nohz(void) > > #endif /* #else #ifdef CONFIG_NO_HZ */ > > -#if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) > -#include <linux/rcutree.h> > -#elif defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU) > -#include <linux/rcutiny.h> > -#else > -#error "Unknown RCU implementation specified to kernel configuration" > -#endif > - > /* > * init_rcu_head_on_stack()/destroy_rcu_head_on_stack() are needed for dynamic > * initialization and destruction of rcu_head on the stack. rcu_head structures > @@ -797,4 +781,12 @@ static inline void debug_rcu_head_unqueue(struct rcu_head *head) > } > #endif /* #else !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ > > +#if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) > +#include <linux/rcutree.h> > +#elif defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU) > +#include <linux/rcutiny.h> > +#else > +#error "Unknown RCU implementation specified to kernel configuration" > +#endif > + > #endif /* __LINUX_RCUPDATE_H */ > diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h > index 3a93348..c317eec 100644 > --- a/include/linux/rcutree.h > +++ b/include/linux/rcutree.h > @@ -30,6 +30,8 @@ > #ifndef __LINUX_RCUTREE_H > #define __LINUX_RCUTREE_H > > +#include <linux/sched.h> > + > extern void rcu_init(void); > extern void rcu_note_context_switch(int cpu); > extern int rcu_needs_cpu(int cpu); > diff --git a/include/linux/sched.h b/include/linux/sched.h > index 83bd2e2..e7c6a90 100644 > --- a/include/linux/sched.h > +++ b/include/linux/sched.h > @@ -78,7 +78,7 @@ struct sched_param { > #include <linux/topology.h> > #include <linux/proportions.h> > #include <linux/seccomp.h> > -#include <linux/rcupdate.h> > +#include <linux/rcu_head.h> > #include <linux/rculist.h> > #include <linux/rtmutex.h> > > @@ -2241,11 +2241,9 @@ int same_thread_group(struct task_struct *p1, struct task_struct *p2) > return p1->tgid == p2->tgid; > } > > -static inline struct task_struct *next_thread(const struct task_struct *p) > -{ > - return list_entry_rcu(p->thread_group.next, > - struct task_struct, thread_group); > -} > +/* Avoid #include hell for inlining rcu_read_lock(). */ > +#define next_thread(p) \ > + list_entry_rcu((p)->thread_group.next, struct task_struct, thread_group) > > static inline int thread_group_empty(struct task_struct *p) > { ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-31 9:50 ` Lai Jiangshan 2011-03-31 11:18 ` Peter Zijlstra @ 2011-03-31 19:28 ` H. Peter Anvin 1 sibling, 0 replies; 67+ messages in thread From: H. Peter Anvin @ 2011-03-31 19:28 UTC (permalink / raw) To: Lai Jiangshan Cc: Peter Zijlstra, Michal Marek, paulmck, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On 03/31/2011 02:50 AM, Lai Jiangshan wrote: > > But how about my kernel-offset.c patch? It is clean & simple, > it just seems not so normal. > It is neither clear nor simple, sorry. It is quite the opposite. I think Michal is on the right track, but yes, much work remains to be done. -hpa ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET 2011-03-30 5:25 ` Lai Jiangshan 2011-03-30 7:22 ` Lai Jiangshan @ 2011-03-31 19:25 ` H. Peter Anvin 1 sibling, 0 replies; 67+ messages in thread From: H. Peter Anvin @ 2011-03-31 19:25 UTC (permalink / raw) To: Lai Jiangshan Cc: paulmck, Jan Beulich, Ingo Molnar, Alexander van Heukelum, Dipankar Sarma, Peter Zijlstra, Andrew Morton, Sam Ravnborg, David Howells, Oleg Nesterov, Roland McGrath, Serge Hallyn, linux-kernel, Thomas Gleixner, Steven Rostedt On 03/29/2011 10:25 PM, Lai Jiangshan wrote: > > RCU is one of the most import subsystem, > I think it is worth to do such "wrong" way. > No, that's the wrong way to look at it. RCU is important enough that it's worth creating disruption over doing it correctly. -hpa ^ permalink raw reply [flat|nested] 67+ messages in thread
* [RFC PATCH 5/5] RCU: declare preemptible __rcu_read_[un]lock() as inline function 2011-03-28 2:55 [RFC PATCH 0/5] Add kernel-offset file and make rcu_read_[un]lock() included Lai Jiangshan ` (3 preceding siblings ...) 2011-03-28 3:00 ` [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET Lai Jiangshan @ 2011-03-28 3:01 ` Lai Jiangshan 2011-03-28 22:17 ` [RFC PATCH 0/5] Add kernel-offset file and make rcu_read_[un]lock() included Paul E. McKenney ` (2 subsequent siblings) 7 siblings, 0 replies; 67+ messages in thread From: Lai Jiangshan @ 2011-03-28 3:01 UTC (permalink / raw) To: Paul E. McKenney, Ingo Molnar, Peter Zijlstra, Dipankar Sarma, Jan Beulich, David Howells, Alexander van Heukelum, H. Peter Anvin, Andrew Morton, Roland McGrath, Oleg Nesterov, Serge Hallyn, linux-kernel, Christoph Lameter, Sam Ravnborg __rcu_read_[un]lock() are so simple functions, make them inlined. Also rmove dumplicated code. Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> --- include/linux/rcupdate.h | 37 +++++++++++++++++++++++++++++++++++-- kernel/rcutiny_plugin.h | 38 ++------------------------------------ kernel/rcutree_plugin.h | 38 ++------------------------------------ 3 files changed, 39 insertions(+), 74 deletions(-) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index a5ed3fe..58e0350 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -187,8 +187,41 @@ static inline void rcu_copy_process(struct task_struct *tsk) #ifdef CONFIG_PREEMPT_RCU -extern void __rcu_read_lock(void); -extern void __rcu_read_unlock(void); +void rcu_read_unlock_special(struct task_rcu_struct *t); + +/* + * Preemptible RCU implementation for rcu_read_lock(). + * Just increment ->rcu_read_lock_nesting, shared state will be updated + * if we block. + */ +static inline void __rcu_read_lock(void) +{ + current_task_rcu_struct()->rcu_read_lock_nesting++; + barrier(); +} + +/* + * Preemptible RCU implementation for rcu_read_unlock(). + * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost + * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then + * invoke rcu_read_unlock_special() to clean up after a context switch + * in an RCU read-side critical section and other special cases. + */ +static inline void __rcu_read_unlock(void) +{ + struct task_rcu_struct *t = current_task_rcu_struct(); + + barrier(); + --t->rcu_read_lock_nesting; + barrier(); /* decrement before load of ->rcu_read_unlock_special */ + if (t->rcu_read_lock_nesting == 0 && + unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) + rcu_read_unlock_special(t); +#ifdef CONFIG_PROVE_LOCKING + WARN_ON_ONCE(ACCESS_ONCE(t->rcu_read_lock_nesting) < 0); +#endif /* #ifdef CONFIG_PROVE_LOCKING */ +} + void synchronize_rcu(void); /* diff --git a/kernel/rcutiny_plugin.h b/kernel/rcutiny_plugin.h index 425e892..49a7699 100644 --- a/kernel/rcutiny_plugin.h +++ b/kernel/rcutiny_plugin.h @@ -514,23 +514,11 @@ void rcu_preempt_note_context_switch(void) } /* - * Tiny-preemptible RCU implementation for rcu_read_lock(). - * Just increment ->rcu_read_lock_nesting, shared state will be updated - * if we block. - */ -void __rcu_read_lock(void) -{ - current_task_rcu_struct()->rcu_read_lock_nesting++; - barrier(); /* needed if we ever invoke rcu_read_lock in rcutiny.c */ -} -EXPORT_SYMBOL_GPL(__rcu_read_lock); - -/* * Handle special cases during rcu_read_unlock(), such as needing to * notify RCU core processing or task having blocked during the RCU * read-side critical section. */ -static void rcu_read_unlock_special(struct task_rcu_struct *t) +void rcu_read_unlock_special(struct task_rcu_struct *t) { int empty; int empty_exp; @@ -609,29 +597,7 @@ static void rcu_read_unlock_special(struct task_rcu_struct *t) #endif /* #ifdef CONFIG_RCU_BOOST */ local_irq_restore(flags); } - -/* - * Tiny-preemptible RCU implementation for rcu_read_unlock(). - * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost - * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then - * invoke rcu_read_unlock_special() to clean up after a context switch - * in an RCU read-side critical section and other special cases. - */ -void __rcu_read_unlock(void) -{ - struct task_rcu_struct *t = current_task_rcu_struct(); - - barrier(); /* needed if we ever invoke rcu_read_unlock in rcutiny.c */ - --t->rcu_read_lock_nesting; - barrier(); /* decrement before load of ->rcu_read_unlock_special */ - if (t->rcu_read_lock_nesting == 0 && - unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) - rcu_read_unlock_special(t); -#ifdef CONFIG_PROVE_LOCKING - WARN_ON_ONCE(t->rcu_read_lock_nesting < 0); -#endif /* #ifdef CONFIG_PROVE_LOCKING */ -} -EXPORT_SYMBOL_GPL(__rcu_read_unlock); +EXPORT_SYMBOL_GPL(rcu_read_unlock_special); /* * Check for a quiescent state from the current CPU. When a task blocks, diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h index 88a12d4..63c548a 100644 --- a/kernel/rcutree_plugin.h +++ b/kernel/rcutree_plugin.h @@ -208,18 +208,6 @@ static void rcu_preempt_note_context_switch(int cpu) } /* - * Tree-preemptible RCU implementation for rcu_read_lock(). - * Just increment ->rcu_read_lock_nesting, shared state will be updated - * if we block. - */ -void __rcu_read_lock(void) -{ - current_task_rcu_struct()->rcu_read_lock_nesting++; - barrier(); /* needed if we ever invoke rcu_read_lock in rcutree.c */ -} -EXPORT_SYMBOL_GPL(__rcu_read_lock); - -/* * Check for preempted RCU readers blocking the current grace period * for the specified rcu_node structure. If the caller needs a reliable * answer, it must hold the rcu_node's ->lock. @@ -285,7 +273,7 @@ static struct list_head *rcu_next_node_entry(struct task_rcu_struct *t, * notify RCU core processing or task having blocked during the RCU * read-side critical section. */ -static void rcu_read_unlock_special(struct task_rcu_struct *t) +void rcu_read_unlock_special(struct task_rcu_struct *t) { int empty; int empty_exp; @@ -375,29 +363,7 @@ static void rcu_read_unlock_special(struct task_rcu_struct *t) local_irq_restore(flags); } } - -/* - * Tree-preemptible RCU implementation for rcu_read_unlock(). - * Decrement ->rcu_read_lock_nesting. If the result is zero (outermost - * rcu_read_unlock()) and ->rcu_read_unlock_special is non-zero, then - * invoke rcu_read_unlock_special() to clean up after a context switch - * in an RCU read-side critical section and other special cases. - */ -void __rcu_read_unlock(void) -{ - struct task_rcu_struct *t = current_task_rcu_struct(); - - barrier(); /* needed if we ever invoke rcu_read_unlock in rcutree.c */ - --t->rcu_read_lock_nesting; - barrier(); /* decrement before load of ->rcu_read_unlock_special */ - if (t->rcu_read_lock_nesting == 0 && - unlikely(ACCESS_ONCE(t->rcu_read_unlock_special))) - rcu_read_unlock_special(t); -#ifdef CONFIG_PROVE_LOCKING - WARN_ON_ONCE(ACCESS_ONCE(t->rcu_read_lock_nesting) < 0); -#endif /* #ifdef CONFIG_PROVE_LOCKING */ -} -EXPORT_SYMBOL_GPL(__rcu_read_unlock); +EXPORT_SYMBOL_GPL(rcu_read_unlock_special); #ifdef CONFIG_RCU_CPU_STALL_VERBOSE -- 1.7.4 ^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 0/5] Add kernel-offset file and make rcu_read_[un]lock() included 2011-03-28 2:55 [RFC PATCH 0/5] Add kernel-offset file and make rcu_read_[un]lock() included Lai Jiangshan ` (4 preceding siblings ...) 2011-03-28 3:01 ` [RFC PATCH 5/5] RCU: declare preemptible __rcu_read_[un]lock() as inline function Lai Jiangshan @ 2011-03-28 22:17 ` Paul E. McKenney 2011-03-31 18:26 ` [RFC PATCH 2/5] kbuild: dedumplicated the generating code David Howells 2011-03-31 18:28 ` [RFC PATCH 3/5] kbuild: add kernel-offset file David Howells 7 siblings, 0 replies; 67+ messages in thread From: Paul E. McKenney @ 2011-03-28 22:17 UTC (permalink / raw) To: Lai Jiangshan Cc: Ingo Molnar, Peter Zijlstra, Dipankar Sarma, Jan Beulich, David Howells, Alexander van Heukelum, H. Peter Anvin, Andrew Morton, Roland McGrath, Oleg Nesterov, Serge Hallyn, linux-kernel, Christoph Lameter, Sam Ravnborg On Mon, Mar 28, 2011 at 10:55:41AM +0800, Lai Jiangshan wrote: > > > __rcu_read_[un]lock() are so simple functions and called so > frequent, it is not good that a function call overhead is > required when use them. It would be better if we declare > them inline and let the compiler do the choice. > > We add kernel-offset.c for TASK_RCU_OFFSET, which enable we declare > rcu_read_[un]lock() as inline function. > > Adding kernel-offset.c does not have any effect for the result > compiled kernel, it just adds a small and fast step in the kbuid stage. Good stuff, thank you! It will be really cool to have inlined rcu_read_lock() and rcu_read_unlock() for TREE_PREEMPT_RCU! I will queue this up in a few days, currently chasing down a problem located in the upcoming TREE_RCU softirq-to-kthread commit. Thanx, Paul > Lai Jiangshan (5): > task_rcu_struct > kbuild_dedumplicated > add_kernel_offset > access task's task_rcu_struct without include sched.h > rcu inline > > Kbuild | 99 ++++++++++++++++--------------- > include/linux/init_task.h | 25 +-------- > include/linux/rcupdate.h | 144 ++++++++++++++++++++++++++++++++++++++++++++- > include/linux/sched.h | 39 +------------ > kernel/kernel-offsets.c | 17 +++++ > kernel/rcutiny_plugin.h | 58 ++++-------------- > kernel/rcutree_plugin.h | 69 ++++++---------------- > 7 files changed, 243 insertions(+), 208 deletions(-) > create mode 100644 kernel/kernel-offsets.c > > -- > 1.7.4 ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 2/5] kbuild: dedumplicated the generating code 2011-03-28 2:55 [RFC PATCH 0/5] Add kernel-offset file and make rcu_read_[un]lock() included Lai Jiangshan ` (5 preceding siblings ...) 2011-03-28 22:17 ` [RFC PATCH 0/5] Add kernel-offset file and make rcu_read_[un]lock() included Paul E. McKenney @ 2011-03-31 18:26 ` David Howells 2011-03-31 18:28 ` [RFC PATCH 3/5] kbuild: add kernel-offset file David Howells 7 siblings, 0 replies; 67+ messages in thread From: David Howells @ 2011-03-31 18:26 UTC (permalink / raw) To: Lai Jiangshan Cc: dhowells, Paul E. McKenney, Ingo Molnar, Peter Zijlstra, Dipankar Sarma, Jan Beulich, Alexander van Heukelum, H. Peter Anvin, Andrew Morton, Roland McGrath, Oleg Nesterov, Serge Hallyn, linux-kernel, Christoph Lameter, Sam Ravnborg 'dedumplicated'? Did you mean 'deduplicated'? David ^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [RFC PATCH 3/5] kbuild: add kernel-offset file 2011-03-28 2:55 [RFC PATCH 0/5] Add kernel-offset file and make rcu_read_[un]lock() included Lai Jiangshan ` (6 preceding siblings ...) 2011-03-31 18:26 ` [RFC PATCH 2/5] kbuild: dedumplicated the generating code David Howells @ 2011-03-31 18:28 ` David Howells 7 siblings, 0 replies; 67+ messages in thread From: David Howells @ 2011-03-31 18:28 UTC (permalink / raw) To: Lai Jiangshan Cc: dhowells, Paul E. McKenney, Ingo Molnar, Peter Zijlstra, Dipankar Sarma, Jan Beulich, Alexander van Heukelum, H. Peter Anvin, Andrew Morton, Roland McGrath, Oleg Nesterov, Serge Hallyn, linux-kernel, Christoph Lameter, Sam Ravnborg Lai Jiangshan <laijs@cn.fujitsu.com> wrote: > __KBUILD_GENARATING__ Should that be __KBUILD_GENERATING__? David ^ permalink raw reply [flat|nested] 67+ messages in thread
end of thread, other threads:[~2011-04-25 7:33 UTC | newest] Thread overview: 67+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2011-03-28 2:55 [RFC PATCH 0/5] Add kernel-offset file and make rcu_read_[un]lock() included Lai Jiangshan 2011-03-28 2:58 ` [RFC PATCH 1/5] Move task's RCU code to rcupdate.h Lai Jiangshan 2011-03-31 11:31 ` Peter Zijlstra 2011-03-31 23:24 ` Paul E. McKenney 2011-03-28 2:58 ` [RFC PATCH 2/5] kbuild: dedumplicated the generating code Lai Jiangshan 2011-03-28 8:31 ` Jan Beulich 2011-03-28 2:59 ` [RFC PATCH 3/5] kbuild: add kernel-offset file Lai Jiangshan 2011-03-28 3:00 ` [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET Lai Jiangshan 2011-03-28 8:35 ` Jan Beulich 2011-03-29 9:41 ` Lai Jiangshan 2011-03-29 21:14 ` H. Peter Anvin 2011-03-29 21:31 ` Paul E. McKenney 2011-03-29 21:32 ` H. Peter Anvin 2011-03-29 21:47 ` Paul E. McKenney 2011-03-29 22:01 ` H. Peter Anvin 2011-03-30 0:47 ` Paul E. McKenney 2011-03-30 5:25 ` Lai Jiangshan 2011-03-30 7:22 ` Lai Jiangshan 2011-03-30 10:55 ` Michal Marek 2011-03-30 10:57 ` Peter Zijlstra 2011-03-30 11:46 ` Michal Marek 2011-03-31 1:02 ` Lai Jiangshan 2011-03-31 1:21 ` Paul E. McKenney 2011-03-31 1:53 ` Lai Jiangshan 2011-03-31 23:30 ` Paul E. McKenney 2011-04-01 3:28 ` Paul E. McKenney 2011-03-31 8:04 ` Peter Zijlstra 2011-03-31 9:50 ` Lai Jiangshan 2011-03-31 11:18 ` Peter Zijlstra 2011-04-01 1:57 ` Lai Jiangshan 2011-04-01 11:35 ` Peter Zijlstra 2011-04-05 21:54 ` Paul E. McKenney 2011-04-05 23:07 ` Paul E. McKenney 2011-04-06 8:10 ` Peter Zijlstra 2011-04-06 19:21 ` Paul E. McKenney 2011-04-06 20:13 ` Paul E. McKenney 2011-04-06 21:06 ` Peter Zijlstra 2011-04-06 21:27 ` H. Peter Anvin 2011-04-07 0:30 ` Paul E. McKenney 2011-04-07 5:49 ` Lai Jiangshan 2011-04-07 15:47 ` Paul E. McKenney 2011-04-07 16:26 ` Paul E. McKenney 2011-04-08 1:26 ` Lai Jiangshan 2011-04-08 5:13 ` Paul E. McKenney 2011-04-11 3:08 ` Lai Jiangshan 2011-04-11 5:12 ` Paul E. McKenney 2011-04-11 6:01 ` Lai Jiangshan 2011-04-11 8:31 ` Lai Jiangshan 2011-04-11 21:02 ` Paul E. McKenney 2011-04-11 21:24 ` Peter Zijlstra 2011-04-22 7:19 ` Lai Jiangshan 2011-04-22 8:10 ` Peter Zijlstra 2011-04-25 7:21 ` Lai Jiangshan 2011-04-23 20:30 ` Paul E. McKenney 2011-04-11 8:33 ` Lai Jiangshan 2011-04-07 7:05 ` [PATCH 1/4] rcu: split rcupdate.h Lai Jiangshan 2011-04-07 7:07 ` [PATCH 2/4] rcu: make rcudpate.h can use struct task_struct Lai Jiangshan 2011-04-07 7:07 ` [PATCH 3/4] rcu: introduce task_rcu_struct and move task's RCU code to rcupdate_defs.h Lai Jiangshan 2011-04-07 13:40 ` Alexey Dobriyan 2011-04-07 7:08 ` [PATCH 4/4] rcu: declare preemptible __rcu_read_[un]lock() as inline function Lai Jiangshan 2011-04-06 17:41 ` [RFC PATCH 4/5] RCU: Add TASK_RCU_OFFSET Paul E. McKenney 2011-03-31 19:28 ` H. Peter Anvin 2011-03-31 19:25 ` H. Peter Anvin 2011-03-28 3:01 ` [RFC PATCH 5/5] RCU: declare preemptible __rcu_read_[un]lock() as inline function Lai Jiangshan 2011-03-28 22:17 ` [RFC PATCH 0/5] Add kernel-offset file and make rcu_read_[un]lock() included Paul E. McKenney 2011-03-31 18:26 ` [RFC PATCH 2/5] kbuild: dedumplicated the generating code David Howells 2011-03-31 18:28 ` [RFC PATCH 3/5] kbuild: add kernel-offset file David Howells
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).