[PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Xiao Guangrong]

From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>

This patch is modified from Mathieu Desnoyers' patch. The original patch
can be found here: 
	http://marc.info/?l=linux-kernel&m=123791201816245&w=2
This tracepoint can trace the time stamp when softirq action is raised. 

Changelog for v1 -> v2: 
1: Use TRACE_EVENT instead of DEFINE_TRACE
2: Move the tracepoint from raise_softirq_irqoff() to
   __raise_softirq_irqoff()

Changelog for v2 -> v3: 
Move the definition of __raise_softifq_irqoff() to .c file when
CONFIG_TRACEPOINTS is enabled, to avoid recursive includes

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com>

---
 include/linux/interrupt.h  |    6 ++++++
 include/trace/events/irq.h |   18 ++++++++++++++++++
 kernel/softirq.c           |    8 ++++++++
 3 files changed, 32 insertions(+), 0 deletions(-)

diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
index dd574d5..3143341 100644
--- a/include/linux/interrupt.h
+++ b/include/linux/interrupt.h
@@ -361,7 +361,13 @@ asmlinkage void do_softirq(void);
 asmlinkage void __do_softirq(void);
 extern void open_softirq(int nr, void (*action)(struct softirq_action *));
 extern void softirq_init(void);
+
+#ifdef CONFIG_TRACEPOINTS
+extern void __raise_softirq_irqoff(unsigned int nr);
+#else
 #define __raise_softirq_irqoff(nr) do { or_softirq_pending(1UL << (nr)); } while (0)
+#endif
+
 extern void raise_softirq_irqoff(unsigned int nr);
 extern void raise_softirq(unsigned int nr);
 extern void wakeup_softirqd(void);
diff --git a/include/trace/events/irq.h b/include/trace/events/irq.h
index 32a9f7e..3c895bb 100644
--- a/include/trace/events/irq.h
+++ b/include/trace/events/irq.h
@@ -128,6 +128,24 @@ TRACE_EVENT(softirq_exit,
 	TP_printk("softirq=%d action=%s", __entry->vec, __get_str(name))
 );
 
+TRACE_EVENT(irq_softirq_raise,
+
+	TP_PROTO(unsigned int nr),
+
+	TP_ARGS(nr),
+
+	TP_STRUCT__entry(
+		__field(	unsigned int,	nr	)
+	),
+
+	TP_fast_assign(
+		__entry->nr	= nr;
+	),
+
+	TP_printk("softirq=%d action=%s is raised",
+		__entry->nr, softirq_to_name[__entry->nr])
+);
+
 #endif /*  _TRACE_IRQ_H */
 
 /* This part must be outside protection */
diff --git a/kernel/softirq.c b/kernel/softirq.c
index b41fb71..bd0546b 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -310,6 +310,14 @@ void irq_exit(void)
 	preempt_enable_no_resched();
 }
 
+#ifdef CONFIG_TRACEPOINTS
+inline void __raise_softirq_irqoff(unsigned int nr)
+{
+	trace_irq_softirq_raise(nr);
+	or_softirq_pending(1UL << (nr));
+}
+#endif
+
 /*
  * This function must run with irqs disabled!
  */
-- 
1.6.1.2

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Li Zefan]

> +TRACE_EVENT(irq_softirq_raise,

I think 'softirq_raise' is better.

> +
> +	TP_PROTO(unsigned int nr),
> +
> +	TP_ARGS(nr),
> +
> +	TP_STRUCT__entry(
> +		__field(	unsigned int,	nr	)
> +	),
> +
> +	TP_fast_assign(
> +		__entry->nr	= nr;
> +	),
> +
> +	TP_printk("softirq=%d action=%s is raised",
> +		__entry->nr, softirq_to_name[__entry->nr])

"softirq=%d action=%s" is sufficient.

Please see TRACE_EVENT(softirq_entry) and TRACE_EVENT(softirq_exit).


> +);
> +
>  #endif /*  _TRACE_IRQ_H */
>  
>  /* This part must be outside protection */
> diff --git a/kernel/softirq.c b/kernel/softirq.c
> index b41fb71..bd0546b 100644
> --- a/kernel/softirq.c
> +++ b/kernel/softirq.c
> @@ -310,6 +310,14 @@ void irq_exit(void)
>  	preempt_enable_no_resched();
>  }
>  
> +#ifdef CONFIG_TRACEPOINTS
> +inline void __raise_softirq_irqoff(unsigned int nr)
> +{
> +	trace_irq_softirq_raise(nr);
> +	or_softirq_pending(1UL << (nr));
> +}
> +#endif
> +
>  /*
>   * This function must run with irqs disabled!
>   */

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Mathieu Desnoyers]

* Xiao Guangrong (xiaoguangrong@cn.fujitsu.com) wrote:
> From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> 
> This patch is modified from Mathieu Desnoyers' patch. The original patch
> can be found here: 
> 	http://marc.info/?l=linux-kernel&m=123791201816245&w=2
> This tracepoint can trace the time stamp when softirq action is raised. 
> 
> Changelog for v1 -> v2: 
> 1: Use TRACE_EVENT instead of DEFINE_TRACE
> 2: Move the tracepoint from raise_softirq_irqoff() to
>    __raise_softirq_irqoff()
> 
> Changelog for v2 -> v3: 
> Move the definition of __raise_softifq_irqoff() to .c file when
> CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
> 
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com>
> 
> ---
>  include/linux/interrupt.h  |    6 ++++++
>  include/trace/events/irq.h |   18 ++++++++++++++++++
>  kernel/softirq.c           |    8 ++++++++
>  3 files changed, 32 insertions(+), 0 deletions(-)
> 
> diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
> index dd574d5..3143341 100644
> --- a/include/linux/interrupt.h
> +++ b/include/linux/interrupt.h
> @@ -361,7 +361,13 @@ asmlinkage void do_softirq(void);
>  asmlinkage void __do_softirq(void);
>  extern void open_softirq(int nr, void (*action)(struct softirq_action *));
>  extern void softirq_init(void);
> +
> +#ifdef CONFIG_TRACEPOINTS
> +extern void __raise_softirq_irqoff(unsigned int nr);
> +#else
>  #define __raise_softirq_irqoff(nr) do { or_softirq_pending(1UL << (nr)); } while (0)

Can you put the 
	trace_irq_softirq_raise(nr);

directly in the define rather than adding this weird CONFIG_TRACEPOINTS?
(and change the define for a static inline), something like :

static inline void __raise_softirq_irqoff(unsigned int nr)
{
	trace_irq_softirq_raise(nr);
	or_softirq_pending(1UL << (nr);
}

This would ensure we don't add a function call on the
__raise_softirq_irqoff() fast-path.

Beware of circular include dependencies though. The tracepoints are
meant not to have this kind of problems (I try to keep the dependencies
very minimalistic), but I wonder if Steven's TRACE_EVENT is now ok on
this aspect.

If TRACE_EVENT happens to pose problems with circular header
dependencies, then try moving to the DECLARE_TRACE/DEFINE_TRACE scheme
which has been more thoroughly tested as a first step.

Mathieu

> +#endif
> +
>  extern void raise_softirq_irqoff(unsigned int nr);
>  extern void raise_softirq(unsigned int nr);
>  extern void wakeup_softirqd(void);
> diff --git a/include/trace/events/irq.h b/include/trace/events/irq.h
> index 32a9f7e..3c895bb 100644
> --- a/include/trace/events/irq.h
> +++ b/include/trace/events/irq.h
> @@ -128,6 +128,24 @@ TRACE_EVENT(softirq_exit,
>  	TP_printk("softirq=%d action=%s", __entry->vec, __get_str(name))
>  );
>  
> +TRACE_EVENT(irq_softirq_raise,
> +
> +	TP_PROTO(unsigned int nr),
> +
> +	TP_ARGS(nr),
> +
> +	TP_STRUCT__entry(
> +		__field(	unsigned int,	nr	)
> +	),
> +
> +	TP_fast_assign(
> +		__entry->nr	= nr;
> +	),
> +
> +	TP_printk("softirq=%d action=%s is raised",
> +		__entry->nr, softirq_to_name[__entry->nr])
> +);
> +
>  #endif /*  _TRACE_IRQ_H */
>  
>  /* This part must be outside protection */
> diff --git a/kernel/softirq.c b/kernel/softirq.c
> index b41fb71..bd0546b 100644
> --- a/kernel/softirq.c
> +++ b/kernel/softirq.c
> @@ -310,6 +310,14 @@ void irq_exit(void)
>  	preempt_enable_no_resched();
>  }
>  
> +#ifdef CONFIG_TRACEPOINTS
> +inline void __raise_softirq_irqoff(unsigned int nr)
> +{
> +	trace_irq_softirq_raise(nr);
> +	or_softirq_pending(1UL << (nr));
> +}
> +#endif
> +
>  /*
>   * This function must run with irqs disabled!
>   */
> -- 
> 1.6.1.2
> 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Jason Baron]

On Tue, May 05, 2009 at 02:41:32PM +0800, Xiao Guangrong wrote:
> From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> 
> This patch is modified from Mathieu Desnoyers' patch. The original patch
> can be found here: 
> 	http://marc.info/?l=linux-kernel&m=123791201816245&w=2
> This tracepoint can trace the time stamp when softirq action is raised. 
> 
> Changelog for v1 -> v2: 
> 1: Use TRACE_EVENT instead of DEFINE_TRACE
> 2: Move the tracepoint from raise_softirq_irqoff() to
>    __raise_softirq_irqoff()
> 
> Changelog for v2 -> v3: 
> Move the definition of __raise_softifq_irqoff() to .c file when
> CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
> 
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com>
> 

hi,

Could you please add 'docbook' style comments above TRACE_EVENTi()? We
recently added a tracepoint docbook to the -tip and it already contains
an 'irq' chapter, so you just need to add the comments for the raise
softirq tracepoint. Please see: 

http://marc.info/?l=linux-kernel&m=124118471411710&w=2

thanks,

-Jason

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Xiao Guangrong]

Li Zefan wrote:
>> +TRACE_EVENT(irq_softirq_raise,
> 
> I think 'softirq_raise' is better.
> 
>> +
>> +	TP_PROTO(unsigned int nr),
>> +
>> +	TP_ARGS(nr),
>> +
>> +	TP_STRUCT__entry(
>> +		__field(	unsigned int,	nr	)
>> +	),
>> +
>> +	TP_fast_assign(
>> +		__entry->nr	= nr;
>> +	),
>> +
>> +	TP_printk("softirq=%d action=%s is raised",
>> +		__entry->nr, softirq_to_name[__entry->nr])
> 
> "softirq=%d action=%s" is sufficient.
> 
> Please see TRACE_EVENT(softirq_entry) and TRACE_EVENT(softirq_exit).
> 

Thanks, I will modify it as your suggestions. ;-)

> 
>> +);
>> +

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Xiao Guangrong]

Jason Baron wrote:
> On Tue, May 05, 2009 at 02:41:32PM +0800, Xiao Guangrong wrote:
>> From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
>>
>> This patch is modified from Mathieu Desnoyers' patch. The original patch
>> can be found here: 
>> 	http://marc.info/?l=linux-kernel&m=123791201816245&w=2
>> This tracepoint can trace the time stamp when softirq action is raised. 
>>
>> Changelog for v1 -> v2: 
>> 1: Use TRACE_EVENT instead of DEFINE_TRACE
>> 2: Move the tracepoint from raise_softirq_irqoff() to
>>    __raise_softirq_irqoff()
>>
>> Changelog for v2 -> v3: 
>> Move the definition of __raise_softifq_irqoff() to .c file when
>> CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
>>
>> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
>> Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com>
>>
> 
> hi,
> 
> Could you please add 'docbook' style comments above TRACE_EVENTi()? We
> recently added a tracepoint docbook to the -tip and it already contains
> an 'irq' chapter, so you just need to add the comments for the raise
> softirq tracepoint. Please see: 
> 
> http://marc.info/?l=linux-kernel&m=124118471411710&w=2
> 

Hi Jason: 

I will add the comments for this tracepoint. 
thanks! 

> thanks,
> 
> -Jason
> 
> 
> 
> 

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Xiao Guangrong]

Mathieu Desnoyers wrote:
> * Xiao Guangrong (xiaoguangrong@cn.fujitsu.com) wrote:
>> From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
>>

>> +#ifdef CONFIG_TRACEPOINTS
>> +extern void __raise_softirq_irqoff(unsigned int nr);
>> +#else
>>  #define __raise_softirq_irqoff(nr) do { or_softirq_pending(1UL << (nr)); } while (0)
> 
> Can you put the 
> 	trace_irq_softirq_raise(nr);
> 
> directly in the define rather than adding this weird CONFIG_TRACEPOINTS?
> (and change the define for a static inline), something like :
> 
> static inline void __raise_softirq_irqoff(unsigned int nr)
> {
> 	trace_irq_softirq_raise(nr);
> 	or_softirq_pending(1UL << (nr);
> }
> 
> This would ensure we don't add a function call on the
> __raise_softirq_irqoff() fast-path.
> 

We did this in v2, and we think it is better for same reason.
But ...

> Beware of circular include dependencies though. The tracepoints are
> meant not to have this kind of problems (I try to keep the dependencies
> very minimalistic), but I wonder if Steven's TRACE_EVENT is now ok on
> this aspect.
> 

We encount this type of problem in v2.
So we move to this version(v3).

> If TRACE_EVENT happens to pose problems with circular header
> dependencies, then try moving to the DECLARE_TRACE/DEFINE_TRACE scheme
> which has been more thoroughly tested as a first step.
> 

IMHO, TRACE_EVENT framework is better for its more generic as ingo said,
and it also provide ftrace support which means user can view tracepoint
information from /debug/tracing/events.

Although this TRACE_EVENT happens to expose problems with circular header
dependencies, we should not refuse using TRACE_EVENT, instead we should
try to fix it for the whole TRACE_EVENT facility later.

Thanks

> Mathieu
> 

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Mathieu Desnoyers]

* Xiao Guangrong (xiaoguangrong@cn.fujitsu.com) wrote:
> 
> 
> Mathieu Desnoyers wrote:
> > * Xiao Guangrong (xiaoguangrong@cn.fujitsu.com) wrote:
> >> From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> >>
> 
> >> +#ifdef CONFIG_TRACEPOINTS
> >> +extern void __raise_softirq_irqoff(unsigned int nr);
> >> +#else
> >>  #define __raise_softirq_irqoff(nr) do { or_softirq_pending(1UL << (nr)); } while (0)
> > 
> > Can you put the 
> > 	trace_irq_softirq_raise(nr);
> > 
> > directly in the define rather than adding this weird CONFIG_TRACEPOINTS?
> > (and change the define for a static inline), something like :
> > 
> > static inline void __raise_softirq_irqoff(unsigned int nr)
> > {
> > 	trace_irq_softirq_raise(nr);
> > 	or_softirq_pending(1UL << (nr);
> > }
> > 
> > This would ensure we don't add a function call on the
> > __raise_softirq_irqoff() fast-path.
> > 
> 
> We did this in v2, and we think it is better for same reason.
> But ...
> 
> > Beware of circular include dependencies though. The tracepoints are
> > meant not to have this kind of problems (I try to keep the dependencies
> > very minimalistic), but I wonder if Steven's TRACE_EVENT is now ok on
> > this aspect.
> > 
> 
> We encount this type of problem in v2.
> So we move to this version(v3).
> 
> > If TRACE_EVENT happens to pose problems with circular header
> > dependencies, then try moving to the DECLARE_TRACE/DEFINE_TRACE scheme
> > which has been more thoroughly tested as a first step.
> > 
> 
> IMHO, TRACE_EVENT framework is better for its more generic as ingo said,
> and it also provide ftrace support which means user can view tracepoint
> information from /debug/tracing/events.
> 
> Although this TRACE_EVENT happens to expose problems with circular header
> dependencies, we should not refuse using TRACE_EVENT, instead we should
> try to fix it for the whole TRACE_EVENT facility later.
> 

I partially agree with you :

Yes, we should try to fix TRACE_EVENT, but we should fix it _before_ we
start using it widely. Circular header dependencies is a real problem
with TRACE_EVENT right now.

Until we fix this, I will be tempted to stay with a known-good solution,
which is DECLARE/DEFINE_TRACE.

Mathieu

> Thanks
> 
> > Mathieu
> > 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Steven Rostedt]

On Mon, 11 May 2009, Mathieu Desnoyers wrote:
> 
> Yes, we should try to fix TRACE_EVENT, but we should fix it _before_ we
> start using it widely. Circular header dependencies is a real problem
> with TRACE_EVENT right now.
> 
> Until we fix this, I will be tempted to stay with a known-good solution,
> which is DECLARE/DEFINE_TRACE.

The majority of tracepoints happen is C files. Those few cases where they 
are used in headers is where the issues arise.

But...

I did not want to uglify all trace event headers with:

#ifdef CREATE_FOO_TRACE_POINTS
#undef CREATE_FOO_TRACE_POINTS
#include <trace/define_trace.h>
#endif

We would only need to do that for those trace points that need to be 
included in header files. Then the declaration C file would need to define 
both CREATE_FOO_TRACE_POINTS and CREATE_TRACE_POINTS

#define CREATE_FOO_TRACE_POINTS
#define CRATE_TRACE_POINTS
#include <trace/events/foo.h>


But this is pretty trivial to solve, and I do not consider it a show 
stopper or a major header dependency problem.

-- Steve

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Mathieu Desnoyers]

* Steven Rostedt (rostedt@goodmis.org) wrote:
> 
> On Mon, 11 May 2009, Mathieu Desnoyers wrote:
> > 
> > Yes, we should try to fix TRACE_EVENT, but we should fix it _before_ we
> > start using it widely. Circular header dependencies is a real problem
> > with TRACE_EVENT right now.
> > 
> > Until we fix this, I will be tempted to stay with a known-good solution,
> > which is DECLARE/DEFINE_TRACE.
> 
> The majority of tracepoints happen is C files. Those few cases where they 
> are used in headers is where the issues arise.
> 
> But...
> 
> I did not want to uglify all trace event headers with:
> 
> #ifdef CREATE_FOO_TRACE_POINTS
> #undef CREATE_FOO_TRACE_POINTS
> #include <trace/define_trace.h>
> #endif
> 
> We would only need to do that for those trace points that need to be 
> included in header files. Then the declaration C file would need to define 
> both CREATE_FOO_TRACE_POINTS and CREATE_TRACE_POINTS
> 
> #define CREATE_FOO_TRACE_POINTS
> #define CRATE_TRACE_POINTS
> #include <trace/events/foo.h>
> 
> 
> But this is pretty trivial to solve, and I do not consider it a show 
> stopper or a major header dependency problem.
> 
> -- Steve
> 

Hrm, is there any way to solve it elegantly ?

What we really need is to see the cases where TRACE_EVENT() is used as a
declaration vs the case where it expands
TP_STRUCT__entry/TP_fast_assign/TP_printk as having different
dependencies. The problem comes when we bring the include dependencies
of the TP_fast_assign part into the tracepoint header and it becomes
a dependency of the TRACE_EVENT() declaration-only part.

Can we do the following ?

All tracepoint headers could surround the include dependencies by :

#ifdef BUILD_EVENTS
#include <veryannoyingheaderdependency.h>
#endif

And then we follow this by the TRACE_EVENT() declarations.

BUILD_EVENTS would only be defined in kernel/trace/events.c.

I think it should work, but it looks a bit too simple, so I may have
missed something... ?

Mathieu


-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Steven Rostedt]

On Mon, 11 May 2009, Mathieu Desnoyers wrote:

> * Steven Rostedt (rostedt@goodmis.org) wrote:
> > 
> > On Mon, 11 May 2009, Mathieu Desnoyers wrote:
> > > 
> > > Yes, we should try to fix TRACE_EVENT, but we should fix it _before_ we
> > > start using it widely. Circular header dependencies is a real problem
> > > with TRACE_EVENT right now.
> > > 
> > > Until we fix this, I will be tempted to stay with a known-good solution,
> > > which is DECLARE/DEFINE_TRACE.
> > 
> > The majority of tracepoints happen is C files. Those few cases where they 
> > are used in headers is where the issues arise.
> > 
> > But...
> > 
> > I did not want to uglify all trace event headers with:
> > 
> > #ifdef CREATE_FOO_TRACE_POINTS
> > #undef CREATE_FOO_TRACE_POINTS
> > #include <trace/define_trace.h>
> > #endif
> > 
> > We would only need to do that for those trace points that need to be 
> > included in header files. Then the declaration C file would need to define 
> > both CREATE_FOO_TRACE_POINTS and CREATE_TRACE_POINTS
> > 
> > #define CREATE_FOO_TRACE_POINTS
> > #define CRATE_TRACE_POINTS
> > #include <trace/events/foo.h>
> > 
> > 
> > But this is pretty trivial to solve, and I do not consider it a show 
> > stopper or a major header dependency problem.
> > 
> > -- Steve
> > 
> 
> Hrm, is there any way to solve it elegantly ?
> 
> What we really need is to see the cases where TRACE_EVENT() is used as a
> declaration vs the case where it expands
> TP_STRUCT__entry/TP_fast_assign/TP_printk as having different
> dependencies. The problem comes when we bring the include dependencies
> of the TP_fast_assign part into the tracepoint header and it becomes
> a dependency of the TRACE_EVENT() declaration-only part.
> 
> Can we do the following ?
> 
> All tracepoint headers could surround the include dependencies by :
> 
> #ifdef BUILD_EVENTS
> #include <veryannoyingheaderdependency.h>
> #endif
> 
> And then we follow this by the TRACE_EVENT() declarations.
> 
> BUILD_EVENTS would only be defined in kernel/trace/events.c.

That file no longer exists. All the events are created with the 
define_trace.h or ftrace.h file.

We could do something like your

#ifdef BUILD_EVENTS /* or better name BUILDING_EVENTS ?? */
#include <allmyannoyingheaders.h>
[...]
#endif

And then have the define_trace.h file add define it. But this again adds 
more annoying CPP commands to all trace_event headers.

-- Steve

> 
> I think it should work, but it looks a bit too simple, so I may have
> missed something... ?
> 
> Mathieu
> 
> 
> -- 
> Mathieu Desnoyers
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> 

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Mathieu Desnoyers]

* Steven Rostedt (rostedt@goodmis.org) wrote:
> 
> 
> 
> On Mon, 11 May 2009, Mathieu Desnoyers wrote:
> 
> > * Steven Rostedt (rostedt@goodmis.org) wrote:
> > > 
> > > On Mon, 11 May 2009, Mathieu Desnoyers wrote:
> > > > 
> > > > Yes, we should try to fix TRACE_EVENT, but we should fix it _before_ we
> > > > start using it widely. Circular header dependencies is a real problem
> > > > with TRACE_EVENT right now.
> > > > 
> > > > Until we fix this, I will be tempted to stay with a known-good solution,
> > > > which is DECLARE/DEFINE_TRACE.
> > > 
> > > The majority of tracepoints happen is C files. Those few cases where they 
> > > are used in headers is where the issues arise.
> > > 
> > > But...
> > > 
> > > I did not want to uglify all trace event headers with:
> > > 
> > > #ifdef CREATE_FOO_TRACE_POINTS
> > > #undef CREATE_FOO_TRACE_POINTS
> > > #include <trace/define_trace.h>
> > > #endif
> > > 
> > > We would only need to do that for those trace points that need to be 
> > > included in header files. Then the declaration C file would need to define 
> > > both CREATE_FOO_TRACE_POINTS and CREATE_TRACE_POINTS
> > > 
> > > #define CREATE_FOO_TRACE_POINTS
> > > #define CRATE_TRACE_POINTS
> > > #include <trace/events/foo.h>
> > > 
> > > 
> > > But this is pretty trivial to solve, and I do not consider it a show 
> > > stopper or a major header dependency problem.
> > > 
> > > -- Steve
> > > 
> > 
> > Hrm, is there any way to solve it elegantly ?
> > 
> > What we really need is to see the cases where TRACE_EVENT() is used as a
> > declaration vs the case where it expands
> > TP_STRUCT__entry/TP_fast_assign/TP_printk as having different
> > dependencies. The problem comes when we bring the include dependencies
> > of the TP_fast_assign part into the tracepoint header and it becomes
> > a dependency of the TRACE_EVENT() declaration-only part.
> > 
> > Can we do the following ?
> > 
> > All tracepoint headers could surround the include dependencies by :
> > 
> > #ifdef BUILD_EVENTS
> > #include <veryannoyingheaderdependency.h>
> > #endif
> > 
> > And then we follow this by the TRACE_EVENT() declarations.
> > 
> > BUILD_EVENTS would only be defined in kernel/trace/events.c.
> 
> That file no longer exists. All the events are created with the 
> define_trace.h or ftrace.h file.
> 
> We could do something like your
> 
> #ifdef BUILD_EVENTS /* or better name BUILDING_EVENTS ?? */
> #include <allmyannoyingheaders.h>
> [...]
> #endif
> 
> And then have the define_trace.h file add define it. But this again adds 
> more annoying CPP commands to all trace_event headers.
> 

Yes, but this would solve most of the include dependency problems at
once. We would only have dependency on preempt.h left, which alone is
easier to deal with. And if headers don't need to include any annoying
headers, they don't need to use the ifdef BUILDING_EVENTS. But my point
is that when it's needed (and we already see two cases where it's
needed, with pvops instrumentation and softirq instrumentation, and I
guess we'll see much more), it's good to have this infrastructure in
place, so people don't end up trying to do hacks to the kernel code
changing inlines for function calls to try to deal with the circular
include issue.

I think it's

a) needed
b) it does not hurt anyone who does not need it.

So I would definitely recommend adding such define.

Mathieu

> -- Steve
> 
> > 
> > I think it should work, but it looks a bit too simple, so I may have
> > missed something... ?
> > 
> > Mathieu
> > 
> > 
> > -- 
> > Mathieu Desnoyers
> > OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> > 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Xiao Guangrong]

Mathieu Desnoyers wrote:
> * Steven Rostedt (rostedt@goodmis.org) wrote:
>>
>>
>> On Mon, 11 May 2009, Mathieu Desnoyers wrote:
>>
>>> * Steven Rostedt (rostedt@goodmis.org) wrote:
>>>> On Mon, 11 May 2009, Mathieu Desnoyers wrote:
>>>>> Yes, we should try to fix TRACE_EVENT, but we should fix it _before_ we
>>>>> start using it widely. Circular header dependencies is a real problem
>>>>> with TRACE_EVENT right now.

> Yes, but this would solve most of the include dependency problems at
> once. We would only have dependency on preempt.h left, which alone is
> easier to deal with. And if headers don't need to include any annoying
> headers, they don't need to use the ifdef BUILDING_EVENTS. But my point
> is that when it's needed (and we already see two cases where it's
> needed, with pvops instrumentation and softirq instrumentation, and I
> guess we'll see much more), it's good to have this infrastructure in
> place, so people don't end up trying to do hacks to the kernel code
> changing inlines for function calls to try to deal with the circular
> include issue.
> 
> I think it's
> 
> a) needed
> b) it does not hurt anyone who does not need it.
> 
> So I would definitely recommend adding such define.
> 

Sorry for my poor English.
Problem is not only in TP_printk, but also in TP_PROTO if we add a
tracer in header file.
So, it can't be solved only by "get rid of including ftrace parts"

Take v2 patch for example: 
(it is at: http://marc.info/?l=linux-kernel&m=124081169727739&w=2) 

In order to trace __raise_softirq_irqoff(), we should add 
a trace function in  __raise_softirq_irqoff() and include 
<include/trace/irq.h> in interrupte.h. 
If we put <include/trace/irq.h> on top of linux/irq.h,
we will see a warning of "struct softirq_action declared
inside parameter list" in compile. It is because struct irqaction's
definition is bypasswd before TP_PROTO.

To say it simple: 
sched.c:
include interrupte.h
	|
	|-->include irq.h
	|	|-->include interrupte.h (bypassed)
	|	|-->TP_PROTO(int irq, struct irqaction *action),
	|	    (but struct softirq_action is not declared before,
	|	     it raise a compile warning:struct softirq_action declared
	|	     inside parameter list.
	|-> ......
	
I can't see the solvent for this in your discuss or I understand wrong? 

Xiao

> Mathieu
> 
>> -- Steve
>>
>>> I think it should work, but it looks a bit too simple, so I may have
>>> missed something... ?
>>>
>>> Mathieu
>>>
>>>
>>> -- 
>>> Mathieu Desnoyers
>>> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
>>>
> 

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Mathieu Desnoyers]

* Xiao Guangrong (xiaoguangrong@cn.fujitsu.com) wrote:
> 
> 
> Mathieu Desnoyers wrote:
> > * Steven Rostedt (rostedt@goodmis.org) wrote:
> >>
> >>
> >> On Mon, 11 May 2009, Mathieu Desnoyers wrote:
> >>
> >>> * Steven Rostedt (rostedt@goodmis.org) wrote:
> >>>> On Mon, 11 May 2009, Mathieu Desnoyers wrote:
> >>>>> Yes, we should try to fix TRACE_EVENT, but we should fix it _before_ we
> >>>>> start using it widely. Circular header dependencies is a real problem
> >>>>> with TRACE_EVENT right now.
> 
> > Yes, but this would solve most of the include dependency problems at
> > once. We would only have dependency on preempt.h left, which alone is
> > easier to deal with. And if headers don't need to include any annoying
> > headers, they don't need to use the ifdef BUILDING_EVENTS. But my point
> > is that when it's needed (and we already see two cases where it's
> > needed, with pvops instrumentation and softirq instrumentation, and I
> > guess we'll see much more), it's good to have this infrastructure in
> > place, so people don't end up trying to do hacks to the kernel code
> > changing inlines for function calls to try to deal with the circular
> > include issue.
> > 
> > I think it's
> > 
> > a) needed
> > b) it does not hurt anyone who does not need it.
> > 
> > So I would definitely recommend adding such define.
> > 
> 
> Sorry for my poor English.
> Problem is not only in TP_printk, but also in TP_PROTO if we add a
> tracer in header file.
> So, it can't be solved only by "get rid of including ftrace parts"
> 
> Take v2 patch for example: 
> (it is at: http://marc.info/?l=linux-kernel&m=124081169727739&w=2) 
> 
> In order to trace __raise_softirq_irqoff(), we should add 
> a trace function in  __raise_softirq_irqoff() and include 
> <include/trace/irq.h> in interrupte.h. 
> If we put <include/trace/irq.h> on top of linux/irq.h,
> we will see a warning of "struct softirq_action declared
> inside parameter list" in compile. It is because struct irqaction's
> definition is bypasswd before TP_PROTO.
> 

Then add a forward declaration of

struct softirqaction;

At the top of trace/irq.h. I did it in quite a few places in the LTTng
tree. TP_PROTO just needs a forward declaration, not the full structure
declaration.

Only the ftrace-specific parts will need the #include <linux/irq.h>,
which we should put in a #ifdef :

sched.c:
#include linux/interrupt.h
        |
        |-->include <trace/irq.h>
            #ifdef BUILDING_EVENTS
            /*
             * Includes needed by TP_fast_assign, TP_printk, and
             * TP_STRUCT__entry :
             */
            # include <linux/interrupt.h>
            #else
	    /* Forward declarations for TP_PROTO */
            struct softirq_action;
            #endif

            TRACE_EVENT(......);
            |-->TP_PROTO(int irq, struct irqaction *action),
                (but struct softirq_action is not declared before,
                 it raise a compile warning:struct softirq_action declared
                 inside parameter list.
            |-> ......

Mathieu

> 	
> I can't see the solvent for this in your discuss or I understand wrong? 
> 
> Xiao
> 
> > Mathieu
> > 
> >> -- Steve
> >>
> >>> I think it should work, but it looks a bit too simple, so I may have
> >>> missed something... ?
> >>>
> >>> Mathieu
> >>>
> >>>
> >>> -- 
> >>> Mathieu Desnoyers
> >>> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> >>>
> > 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Lai Jiangshan]

Mathieu Desnoyers wrote:
> 
> I partially agree with you :
> 
> Yes, we should try to fix TRACE_EVENT, but we should fix it _before_ we
> start using it widely. Circular header dependencies is a real problem
> with TRACE_EVENT right now.
> 
> Until we fix this, I will be tempted to stay with a known-good solution,
> which is DECLARE/DEFINE_TRACE.
> 
> 

I partially agree with you:

Yes, Circular header dependencies is a real problem with TRACE_EVENT
right now. It is also a problem with DECLARE_TRACE. It's a stubborn
disease with C-Language (for complex headers). Can we fix C-Language?

o Macros in header (!CREATE_TRACE_POINTS)

When CREATE_TRACE_POINTS is not defined, TRACE_EVENT is definitely
the same as DECLARE_TRACE. Actually, TRACE_EVENT is:

#define TRACE_EVENT(name, proto, args, struct, assign, print)	\
	DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))

So TRACE_EVENT and DECLARE_TRACE are the same in header files.
And so TRACE_EVENT and DECLARE_TRACE have the same advantages and
disadvantages. More TRACE_EVENT equals to a known-good solution.

o Macros in c-file

tracepoint uses DEFINE_TRACE only.

ftrace uses CREATE_TRACE_POINTS + TRACE_EVENT:
	#define CREATE_TRACE_POINTS
	#include <trace/events/sched.h> (which uses TRACE_EVENT)

ftrace generates more code which uses the tracepoints.

> 
> Then add a forward declaration of
> 
> struct softirqaction;
> 
> At the top of trace/irq.h. I did it in quite a few places in the LTTng
> tree. TP_PROTO just needs a forward declaration, not the full structure
> declaration.
> 

Thank you for your valuable suggestions.

You are the father of tracepoint and LTTng, your experience in
LTTng is very useful for ftrace.

I'm glad for your suggestions.


Xiao Guangrong, could you add forward declarations of

struct irqaction;
struct softirq_action;

at the top of trace/irq.h as Mathieu's suggestions.
(and remove "#include <linux/interrupt.h>")

Lai

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Mathieu Desnoyers]

* Lai Jiangshan (laijs@cn.fujitsu.com) wrote:
> Mathieu Desnoyers wrote:
> > 
> > I partially agree with you :
> > 
> > Yes, we should try to fix TRACE_EVENT, but we should fix it _before_ we
> > start using it widely. Circular header dependencies is a real problem
> > with TRACE_EVENT right now.
> > 
> > Until we fix this, I will be tempted to stay with a known-good solution,
> > which is DECLARE/DEFINE_TRACE.
> > 
> > 
> 
> I partially agree with you:
> 
> Yes, Circular header dependencies is a real problem with TRACE_EVENT
> right now. It is also a problem with DECLARE_TRACE. It's a stubborn
> disease with C-Language (for complex headers). Can we fix C-Language?
> 
> o Macros in header (!CREATE_TRACE_POINTS)
> 
> When CREATE_TRACE_POINTS is not defined, TRACE_EVENT is definitely
> the same as DECLARE_TRACE. Actually, TRACE_EVENT is:
> 
> #define TRACE_EVENT(name, proto, args, struct, assign, print)	\
> 	DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))
> 
> So TRACE_EVENT and DECLARE_TRACE are the same in header files.
> And so TRACE_EVENT and DECLARE_TRACE have the same advantages and
> disadvantages. More TRACE_EVENT equals to a known-good solution.
> 
> o Macros in c-file
> 
> tracepoint uses DEFINE_TRACE only.
> 
> ftrace uses CREATE_TRACE_POINTS + TRACE_EVENT:
> 	#define CREATE_TRACE_POINTS
> 	#include <trace/events/sched.h> (which uses TRACE_EVENT)
> 
> ftrace generates more code which uses the tracepoints.
> 
> > 
> > Then add a forward declaration of
> > 
> > struct softirqaction;
> > 
> > At the top of trace/irq.h. I did it in quite a few places in the LTTng
> > tree. TP_PROTO just needs a forward declaration, not the full structure
> > declaration.
> > 
> 
> Thank you for your valuable suggestions.
> 
> You are the father of tracepoint and LTTng, your experience in
> LTTng is very useful for ftrace.
> 
> I'm glad for your suggestions.
> 
> 
> Xiao Guangrong, could you add forward declarations of
> 
> struct irqaction;
> struct softirq_action;
> 
> at the top of trace/irq.h as Mathieu's suggestions.
> (and remove "#include <linux/interrupt.h>")
> 

You will probably still need something like :

#ifdef CREATE_TRACE_POINTS
#include <linux/interrupt.h>
#else
struct irqaction;
struct softirq_action;
#endif

So that FTRACE has the header dependencies it needs to build.

Mathieu

> Lai
> 
> 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Lai Jiangshan]

Mathieu Desnoyers wrote:
> * Lai Jiangshan (laijs@cn.fujitsu.com) wrote:
>> Mathieu Desnoyers wrote:
>>> I partially agree with you :
>>>
>>> Yes, we should try to fix TRACE_EVENT, but we should fix it _before_ we
>>> start using it widely. Circular header dependencies is a real problem
>>> with TRACE_EVENT right now.
>>>
>>> Until we fix this, I will be tempted to stay with a known-good solution,
>>> which is DECLARE/DEFINE_TRACE.
>>>
>>>
>> I partially agree with you:
>>
>> Yes, Circular header dependencies is a real problem with TRACE_EVENT
>> right now. It is also a problem with DECLARE_TRACE. It's a stubborn
>> disease with C-Language (for complex headers). Can we fix C-Language?
>>
>> o Macros in header (!CREATE_TRACE_POINTS)
>>
>> When CREATE_TRACE_POINTS is not defined, TRACE_EVENT is definitely
>> the same as DECLARE_TRACE. Actually, TRACE_EVENT is:
>>
>> #define TRACE_EVENT(name, proto, args, struct, assign, print)	\
>> 	DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))
>>
>> So TRACE_EVENT and DECLARE_TRACE are the same in header files.
>> And so TRACE_EVENT and DECLARE_TRACE have the same advantages and
>> disadvantages. More TRACE_EVENT equals to a known-good solution.
>>
>> o Macros in c-file
>>
>> tracepoint uses DEFINE_TRACE only.
>>
>> ftrace uses CREATE_TRACE_POINTS + TRACE_EVENT:
>> 	#define CREATE_TRACE_POINTS
>> 	#include <trace/events/sched.h> (which uses TRACE_EVENT)
>>
>> ftrace generates more code which uses the tracepoints.
>>
>>> Then add a forward declaration of
>>>
>>> struct softirqaction;
>>>
>>> At the top of trace/irq.h. I did it in quite a few places in the LTTng
>>> tree. TP_PROTO just needs a forward declaration, not the full structure
>>> declaration.
>>>
>> Thank you for your valuable suggestions.
>>
>> You are the father of tracepoint and LTTng, your experience in
>> LTTng is very useful for ftrace.
>>
>> I'm glad for your suggestions.
>>
>>
>> Xiao Guangrong, could you add forward declarations of
>>
>> struct irqaction;
>> struct softirq_action;
>>
>> at the top of trace/irq.h as Mathieu's suggestions.
>> (and remove "#include <linux/interrupt.h>")
>>
> 
> You will probably still need something like :
> 
> #ifdef CREATE_TRACE_POINTS
> #include <linux/interrupt.h>
> #else
> struct irqaction;
> struct softirq_action;
> #endif
> 

It's not needed for trace/events/irq.h

Yes, it's a solution.

But I don't think we have to do this, CREATE_TRACE_POINTS is
needed only _once_ for every <trace/events/xxxx.h>

The .c file which defines CREATE_TRACE_POINTS can provide
(had provided likely) things like "#include <linux/interrupt.h>"

See kernel/softirq.c:
#include <linux/interrupt.h>
......
......
#define CREATE_TRACE_POINTS
#include <trace/events/irq.h>

I don't think it's a problem, CREATE_TRACE_POINTS is defined only
once for a <trace/events/xxxx.h>.

Lai.

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Xiao Guangrong]

Lai Jiangshan wrote:
> Mathieu Desnoyers wrote:
>> * Lai Jiangshan (laijs@cn.fujitsu.com) wrote:
>>> Mathieu Desnoyers wrote:

>> #endif
>>
> 
> It's not needed for trace/events/irq.h
> 
> Yes, it's a solution.
> 
> But I don't think we have to do this, CREATE_TRACE_POINTS is
> needed only _once_ for every <trace/events/xxxx.h>
> 
> The .c file which defines CREATE_TRACE_POINTS can provide
> (had provided likely) things like "#include <linux/interrupt.h>"
> 
> See kernel/softirq.c:
> #include <linux/interrupt.h>
> ......
> ......
> #define CREATE_TRACE_POINTS
> #include <trace/events/irq.h>
> 
> I don't think it's a problem, CREATE_TRACE_POINTS is defined only
> once for a <trace/events/xxxx.h>.
> 

Yes, I think Lai's argument is reasonable,
the .c file which use ftrace already include the relevant head file. 
I will make a patch as your suggestions. 

Thanks! ;-)

> Lai.
> 
> 
> 

[PATCH v4] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Xiao Guangrong]

From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>

This patch is modified from Mathieu Desnoyers' patch. The original patch
can be found here: 
	http://marc.info/?l=linux-kernel&m=123791201816245&w=2
This tracepoint can trace the time stamp when softirq action is raised. 

Changelog for v1 -> v2: 
1: Use TRACE_EVENT instead of DEFINE_TRACE
2: Move the tracepoint from raise_softirq_irqoff() to
   __raise_softirq_irqoff()

Changelog for v2 -> v3: 
Move the definition of __raise_softifq_irqoff() to .c file when
CONFIG_TRACEPOINTS is enabled, to avoid recursive includes

Changelog for v3 -> v4: 
1: Come back to v2, and use forward declarations to avoid
   recursive includes as Mathieu's suggestion
2: Modifiy the tracepoint name
3: Add comments for this tracepoint

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com>
Review-by: Lai Jiangshan <laijs@cn.fujitsu.com>
---
 include/linux/interrupt.h  |    9 ++++++++-
 include/trace/events/irq.h |   27 ++++++++++++++++++++++++++-
 2 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
index dd574d5..51e7909 100644
--- a/include/linux/interrupt.h
+++ b/include/linux/interrupt.h
@@ -18,6 +18,7 @@
 #include <asm/atomic.h>
 #include <asm/ptrace.h>
 #include <asm/system.h>
+#include <trace/events/irq.h>
 
 /*
  * These correspond to the IORESOURCE_IRQ_* defines in
@@ -361,7 +362,13 @@ asmlinkage void do_softirq(void);
 asmlinkage void __do_softirq(void);
 extern void open_softirq(int nr, void (*action)(struct softirq_action *));
 extern void softirq_init(void);
-#define __raise_softirq_irqoff(nr) do { or_softirq_pending(1UL << (nr)); } while (0)
+
+static inline void __raise_softirq_irqoff(unsigned int nr)
+{
+	trace_softirq_raise(nr);
+	or_softirq_pending(1UL << nr);
+}
+
 extern void raise_softirq_irqoff(unsigned int nr);
 extern void raise_softirq(unsigned int nr);
 extern void wakeup_softirqd(void);
diff --git a/include/trace/events/irq.h b/include/trace/events/irq.h
index 32a9f7e..08fa63b 100644
--- a/include/trace/events/irq.h
+++ b/include/trace/events/irq.h
@@ -2,11 +2,13 @@
 #define _TRACE_IRQ_H
 
 #include <linux/tracepoint.h>
-#include <linux/interrupt.h>
 
 #undef TRACE_SYSTEM
 #define TRACE_SYSTEM irq
 
+struct irqaction;
+struct softirq_action;
+
 /**
  * irq_handler_entry - called immediately before the irq action handler
  * @irq: irq number
@@ -128,6 +130,29 @@ TRACE_EVENT(softirq_exit,
 	TP_printk("softirq=%d action=%s", __entry->vec, __get_str(name))
 );
 
+/**
+ * softirq_raise - called immediately when a softirq is raised
+ * @nr: softirq vector number
+ */
+TRACE_EVENT(softirq_raise,
+
+	TP_PROTO(unsigned int nr),
+
+	TP_ARGS(nr),
+
+	TP_STRUCT__entry(
+		__field(	unsigned int,	nr			)
+		__string(	name,		softirq_to_name[nr]	)
+	),
+
+	TP_fast_assign(
+		__entry->nr	= nr;
+		__assign_str(name, softirq_to_name[nr]);
+	),
+
+	TP_printk("softirq=%u action=%s", __entry->nr, __get_str(name))
+);
+
 #endif /*  _TRACE_IRQ_H */
 
 /* This part must be outside protection */
-- 
1.6.1.2

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Mathieu Desnoyers]

* Lai Jiangshan (laijs@cn.fujitsu.com) wrote:
> Mathieu Desnoyers wrote:
> > * Lai Jiangshan (laijs@cn.fujitsu.com) wrote:
> >> Mathieu Desnoyers wrote:
> >>> I partially agree with you :
> >>>
> >>> Yes, we should try to fix TRACE_EVENT, but we should fix it _before_ we
> >>> start using it widely. Circular header dependencies is a real problem
> >>> with TRACE_EVENT right now.
> >>>
> >>> Until we fix this, I will be tempted to stay with a known-good solution,
> >>> which is DECLARE/DEFINE_TRACE.
> >>>
> >>>
> >> I partially agree with you:
> >>
> >> Yes, Circular header dependencies is a real problem with TRACE_EVENT
> >> right now. It is also a problem with DECLARE_TRACE. It's a stubborn
> >> disease with C-Language (for complex headers). Can we fix C-Language?
> >>
> >> o Macros in header (!CREATE_TRACE_POINTS)
> >>
> >> When CREATE_TRACE_POINTS is not defined, TRACE_EVENT is definitely
> >> the same as DECLARE_TRACE. Actually, TRACE_EVENT is:
> >>
> >> #define TRACE_EVENT(name, proto, args, struct, assign, print)	\
> >> 	DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))
> >>
> >> So TRACE_EVENT and DECLARE_TRACE are the same in header files.
> >> And so TRACE_EVENT and DECLARE_TRACE have the same advantages and
> >> disadvantages. More TRACE_EVENT equals to a known-good solution.
> >>
> >> o Macros in c-file
> >>
> >> tracepoint uses DEFINE_TRACE only.
> >>
> >> ftrace uses CREATE_TRACE_POINTS + TRACE_EVENT:
> >> 	#define CREATE_TRACE_POINTS
> >> 	#include <trace/events/sched.h> (which uses TRACE_EVENT)
> >>
> >> ftrace generates more code which uses the tracepoints.
> >>
> >>> Then add a forward declaration of
> >>>
> >>> struct softirqaction;
> >>>
> >>> At the top of trace/irq.h. I did it in quite a few places in the LTTng
> >>> tree. TP_PROTO just needs a forward declaration, not the full structure
> >>> declaration.
> >>>
> >> Thank you for your valuable suggestions.
> >>
> >> You are the father of tracepoint and LTTng, your experience in
> >> LTTng is very useful for ftrace.
> >>
> >> I'm glad for your suggestions.
> >>
> >>
> >> Xiao Guangrong, could you add forward declarations of
> >>
> >> struct irqaction;
> >> struct softirq_action;
> >>
> >> at the top of trace/irq.h as Mathieu's suggestions.
> >> (and remove "#include <linux/interrupt.h>")
> >>
> > 
> > You will probably still need something like :
> > 
> > #ifdef CREATE_TRACE_POINTS
> > #include <linux/interrupt.h>
> > #else
> > struct irqaction;
> > struct softirq_action;
> > #endif
> > 
> 
> It's not needed for trace/events/irq.h
> 
> Yes, it's a solution.
> 
> But I don't think we have to do this, CREATE_TRACE_POINTS is
> needed only _once_ for every <trace/events/xxxx.h>
> 
> The .c file which defines CREATE_TRACE_POINTS can provide
> (had provided likely) things like "#include <linux/interrupt.h>"
> 
> See kernel/softirq.c:
> #include <linux/interrupt.h>
> ......
> ......
> #define CREATE_TRACE_POINTS
> #include <trace/events/irq.h>
> 

Isn't it all included under kernel/trace/events.c ? e.g. :

#include <trace/trace_events.h>

#include "trace_output.h"

#include "trace_events_stage_1.h"
#include "trace_events_stage_2.h"
#include "trace_events_stage_3.h"

Therefore, adding a #include <linux/interrupt.h> would be just weird
here. We would have to do that for every tracepoint headers, and we
would just lose the ability to keep track of which tracepoint header has
which include dependency.

I thought the goal of TRACE_EVENT() was exactly the opposite : to
declare everything in one location....

Therefore, if you agree that it's good to have TRACE_EVENT() declaring
everything in one location, then I don't see why you would argue to move
the include dependencies in the very remote kernel/trace/events.c file ?

Mathieu

> I don't think it's a problem, CREATE_TRACE_POINTS is defined only
> once for a <trace/events/xxxx.h>.
> 
> Lai.
> 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [PATCH v4] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Mathieu Desnoyers]

* Xiao Guangrong (xiaoguangrong@cn.fujitsu.com) wrote:
> From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> 
> This patch is modified from Mathieu Desnoyers' patch. The original patch
> can be found here: 
> 	http://marc.info/?l=linux-kernel&m=123791201816245&w=2
> This tracepoint can trace the time stamp when softirq action is raised. 
> 
> Changelog for v1 -> v2: 
> 1: Use TRACE_EVENT instead of DEFINE_TRACE
> 2: Move the tracepoint from raise_softirq_irqoff() to
>    __raise_softirq_irqoff()
> 
> Changelog for v2 -> v3: 
> Move the definition of __raise_softifq_irqoff() to .c file when
> CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
> 
> Changelog for v3 -> v4: 
> 1: Come back to v2, and use forward declarations to avoid
>    recursive includes as Mathieu's suggestion
> 2: Modifiy the tracepoint name
> 3: Add comments for this tracepoint
> 

This is a step in the right direction, but please see my email to Lai
about the fact that this assumes correct and undocumented include
dependencies in kernel/trace/events.c. Not explicitely stating the
include dependencies is a build error waiting to happen.

Including interrupt.h under a ifdef would allow keeping track of
TRACE_EVENT specific build dependencies neatly on a per header basis.

Mathieu

> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com>
> Review-by: Lai Jiangshan <laijs@cn.fujitsu.com>
> ---
>  include/linux/interrupt.h  |    9 ++++++++-
>  include/trace/events/irq.h |   27 ++++++++++++++++++++++++++-
>  2 files changed, 34 insertions(+), 2 deletions(-)
> 
> diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
> index dd574d5..51e7909 100644
> --- a/include/linux/interrupt.h
> +++ b/include/linux/interrupt.h
> @@ -18,6 +18,7 @@
>  #include <asm/atomic.h>
>  #include <asm/ptrace.h>
>  #include <asm/system.h>
> +#include <trace/events/irq.h>
>  
>  /*
>   * These correspond to the IORESOURCE_IRQ_* defines in
> @@ -361,7 +362,13 @@ asmlinkage void do_softirq(void);
>  asmlinkage void __do_softirq(void);
>  extern void open_softirq(int nr, void (*action)(struct softirq_action *));
>  extern void softirq_init(void);
> -#define __raise_softirq_irqoff(nr) do { or_softirq_pending(1UL << (nr)); } while (0)
> +
> +static inline void __raise_softirq_irqoff(unsigned int nr)
> +{
> +	trace_softirq_raise(nr);
> +	or_softirq_pending(1UL << nr);
> +}
> +
>  extern void raise_softirq_irqoff(unsigned int nr);
>  extern void raise_softirq(unsigned int nr);
>  extern void wakeup_softirqd(void);
> diff --git a/include/trace/events/irq.h b/include/trace/events/irq.h
> index 32a9f7e..08fa63b 100644
> --- a/include/trace/events/irq.h
> +++ b/include/trace/events/irq.h
> @@ -2,11 +2,13 @@
>  #define _TRACE_IRQ_H
>  
>  #include <linux/tracepoint.h>
> -#include <linux/interrupt.h>
>  
>  #undef TRACE_SYSTEM
>  #define TRACE_SYSTEM irq
>  
> +struct irqaction;
> +struct softirq_action;
> +
>  /**
>   * irq_handler_entry - called immediately before the irq action handler
>   * @irq: irq number
> @@ -128,6 +130,29 @@ TRACE_EVENT(softirq_exit,
>  	TP_printk("softirq=%d action=%s", __entry->vec, __get_str(name))
>  );
>  
> +/**
> + * softirq_raise - called immediately when a softirq is raised
> + * @nr: softirq vector number
> + */
> +TRACE_EVENT(softirq_raise,
> +
> +	TP_PROTO(unsigned int nr),
> +
> +	TP_ARGS(nr),
> +
> +	TP_STRUCT__entry(
> +		__field(	unsigned int,	nr			)
> +		__string(	name,		softirq_to_name[nr]	)
> +	),
> +
> +	TP_fast_assign(
> +		__entry->nr	= nr;
> +		__assign_str(name, softirq_to_name[nr]);
> +	),
> +
> +	TP_printk("softirq=%u action=%s", __entry->nr, __get_str(name))
> +);
> +
>  #endif /*  _TRACE_IRQ_H */
>  
>  /* This part must be outside protection */
> -- 
> 1.6.1.2
> 
> 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [PATCH v3] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Steven Rostedt]

On Thu, 14 May 2009, Mathieu Desnoyers wrote:
> > > 
> > 
> > It's not needed for trace/events/irq.h
> > 
> > Yes, it's a solution.
> > 
> > But I don't think we have to do this, CREATE_TRACE_POINTS is
> > needed only _once_ for every <trace/events/xxxx.h>
> > 
> > The .c file which defines CREATE_TRACE_POINTS can provide
> > (had provided likely) things like "#include <linux/interrupt.h>"
> > 
> > See kernel/softirq.c:
> > #include <linux/interrupt.h>
> > ......
> > ......
> > #define CREATE_TRACE_POINTS
> > #include <trace/events/irq.h>
> > 
> 
> Isn't it all included under kernel/trace/events.c ? e.g. :

Mathieu,

I've told you this two or three times before. That file no longer exists. 
Maybe it is still there for .30, but these updates are for .31 (too late 
for .30). The code to make TRACE_EVENT work for modules removed the 
events.c file and got rid of those nasty stages.

-- Steve


> 
> #include <trace/trace_events.h>
> 
> #include "trace_output.h"
> 
> #include "trace_events_stage_1.h"
> #include "trace_events_stage_2.h"
> #include "trace_events_stage_3.h"
> 
> Therefore, adding a #include <linux/interrupt.h> would be just weird
> here. We would have to do that for every tracepoint headers, and we
> would just lose the ability to keep track of which tracepoint header has
> which include dependency.
> 
> I thought the goal of TRACE_EVENT() was exactly the opposite : to
> declare everything in one location....
> 
> Therefore, if you agree that it's good to have TRACE_EVENT() declaring
> everything in one location, then I don't see why you would argue to move
> the include dependencies in the very remote kernel/trace/events.c file ?
> 
> Mathieu
> 
> > I don't think it's a problem, CREATE_TRACE_POINTS is defined only
> > once for a <trace/events/xxxx.h>.
> > 
> > Lai.
> > 
> 
> -- 
> Mathieu Desnoyers
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> 

Re: [PATCH v4] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Steven Rostedt]

On Thu, 14 May 2009, Mathieu Desnoyers wrote:
> > From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> > 
> > This patch is modified from Mathieu Desnoyers' patch. The original patch
> > can be found here: 
> > 	http://marc.info/?l=linux-kernel&m=123791201816245&w=2
> > This tracepoint can trace the time stamp when softirq action is raised. 
> > 
> > Changelog for v1 -> v2: 
> > 1: Use TRACE_EVENT instead of DEFINE_TRACE
> > 2: Move the tracepoint from raise_softirq_irqoff() to
> >    __raise_softirq_irqoff()
> > 
> > Changelog for v2 -> v3: 
> > Move the definition of __raise_softifq_irqoff() to .c file when
> > CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
> > 
> > Changelog for v3 -> v4: 
> > 1: Come back to v2, and use forward declarations to avoid
> >    recursive includes as Mathieu's suggestion
> > 2: Modifiy the tracepoint name
> > 3: Add comments for this tracepoint
> > 
> 
> This is a step in the right direction, but please see my email to Lai
> about the fact that this assumes correct and undocumented include
> dependencies in kernel/trace/events.c. Not explicitely stating the
> include dependencies is a build error waiting to happen.
> 
> Including interrupt.h under a ifdef would allow keeping track of
> TRACE_EVENT specific build dependencies neatly on a per header basis.

This is all moot, the events.c file no longer exists and as not an issue.

-- Steve

Re: [PATCH v4] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Mathieu Desnoyers]

* Steven Rostedt (rostedt@goodmis.org) wrote:
> 
> On Thu, 14 May 2009, Mathieu Desnoyers wrote:
> > > From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> > > 
> > > This patch is modified from Mathieu Desnoyers' patch. The original patch
> > > can be found here: 
> > > 	http://marc.info/?l=linux-kernel&m=123791201816245&w=2
> > > This tracepoint can trace the time stamp when softirq action is raised. 
> > > 
> > > Changelog for v1 -> v2: 
> > > 1: Use TRACE_EVENT instead of DEFINE_TRACE
> > > 2: Move the tracepoint from raise_softirq_irqoff() to
> > >    __raise_softirq_irqoff()
> > > 
> > > Changelog for v2 -> v3: 
> > > Move the definition of __raise_softifq_irqoff() to .c file when
> > > CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
> > > 
> > > Changelog for v3 -> v4: 
> > > 1: Come back to v2, and use forward declarations to avoid
> > >    recursive includes as Mathieu's suggestion
> > > 2: Modifiy the tracepoint name
> > > 3: Add comments for this tracepoint
> > > 
> > 
> > This is a step in the right direction, but please see my email to Lai
> > about the fact that this assumes correct and undocumented include
> > dependencies in kernel/trace/events.c. Not explicitely stating the
> > include dependencies is a build error waiting to happen.
> > 
> > Including interrupt.h under a ifdef would allow keeping track of
> > TRACE_EVENT specific build dependencies neatly on a per header basis.
> 
> This is all moot, the events.c file no longer exists and as not an issue.
> 

OK, thanks for the update :)

Mathieu

> -- Steve
> 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [PATCH v4] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Xiao Guangrong]

Steven Rostedt wrote:
> On Thu, 14 May 2009, Mathieu Desnoyers wrote:
>>> From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
>>>
>>> This patch is modified from Mathieu Desnoyers' patch. The original patch
>>> can be found here: 
>>> 	http://marc.info/?l=linux-kernel&m=123791201816245&w=2
>>> This tracepoint can trace the time stamp when softirq action is raised. 
>>>
>>> Changelog for v1 -> v2: 
>>> 1: Use TRACE_EVENT instead of DEFINE_TRACE
>>> 2: Move the tracepoint from raise_softirq_irqoff() to
>>>    __raise_softirq_irqoff()
>>>
>>> Changelog for v2 -> v3: 
>>> Move the definition of __raise_softifq_irqoff() to .c file when
>>> CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
>>>
>>> Changelog for v3 -> v4: 
>>> 1: Come back to v2, and use forward declarations to avoid
>>>    recursive includes as Mathieu's suggestion
>>> 2: Modifiy the tracepoint name
>>> 3: Add comments for this tracepoint
>>>
>> This is a step in the right direction, but please see my email to Lai
>> about the fact that this assumes correct and undocumented include
>> dependencies in kernel/trace/events.c. Not explicitely stating the
>> include dependencies is a build error waiting to happen.
>>
>> Including interrupt.h under a ifdef would allow keeping track of
>> TRACE_EVENT specific build dependencies neatly on a per header basis.
> 
> This is all moot, the events.c file no longer exists and as not an issue.
> 

As Steve's says, use ftrace in ftrace.h not in events.c now. 
So, this mistake does not exist.
Dose this patch has other error? I expect for your views.

Thanks for your review, is great help to me. ;-) 

> -- Steve
> 
> 
> 

Re: [PATCH v4] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Zhaolei]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="UTF-8", Size: 2048 bytes --]

* From: "Xiao Guangrong" <xiaoguangrong@cn.fujitsu.com>
> Steven Rostedt wrote:
>> On Thu, 14 May 2009, Mathieu Desnoyers wrote:
>>>> From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
>>>>
>>>> This patch is modified from Mathieu Desnoyers' patch. The original patch
>>>> can be found here: 
>>>> http://marc.info/?l=linux-kernel&m=123791201816245&w=2
>>>> This tracepoint can trace the time stamp when softirq action is raised. 
>>>>
>>>> Changelog for v1 -> v2: 
>>>> 1: Use TRACE_EVENT instead of DEFINE_TRACE
>>>> 2: Move the tracepoint from raise_softirq_irqoff() to
>>>>    __raise_softirq_irqoff()
>>>>
>>>> Changelog for v2 -> v3: 
>>>> Move the definition of __raise_softifq_irqoff() to .c file when
>>>> CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
>>>>
>>>> Changelog for v3 -> v4: 
>>>> 1: Come back to v2, and use forward declarations to avoid
>>>>    recursive includes as Mathieu's suggestion
>>>> 2: Modifiy the tracepoint name
>>>> 3: Add comments for this tracepoint
>>>>
>>> This is a step in the right direction, but please see my email to Lai
>>> about the fact that this assumes correct and undocumented include
>>> dependencies in kernel/trace/events.c. Not explicitely stating the
>>> include dependencies is a build error waiting to happen.
>>>
>>> Including interrupt.h under a ifdef would allow keeping track of
>>> TRACE_EVENT specific build dependencies neatly on a per header basis.
>> 
>> This is all moot, the events.c file no longer exists and as not an issue.
>> 
> 
> As Steve's says, use ftrace in ftrace.h not in events.c now. 
> So, this mistake does not exist.
> Dose this patch has other error? I expect for your views.
> 
> Thanks for your review, is great help to me. ;-) 

Hello,

It seems Mathieu has no other comments on this patch now.
Ingo, what is your opinion on this patch?

Thanks
Zhaolei

> 
>> -- Steve
>> 
>> 
>> 
>ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: [PATCH v4] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Ingo Molnar]

* Zhaolei <zhaolei@cn.fujitsu.com> wrote:

> * From: "Xiao Guangrong" <xiaoguangrong@cn.fujitsu.com>
> > Steven Rostedt wrote:
> >> On Thu, 14 May 2009, Mathieu Desnoyers wrote:
> >>>> From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> >>>>
> >>>> This patch is modified from Mathieu Desnoyers' patch. The original patch
> >>>> can be found here: 
> >>>> http://marc.info/?l=linux-kernel&m=123791201816245&w=2
> >>>> This tracepoint can trace the time stamp when softirq action is raised. 
> >>>>
> >>>> Changelog for v1 -> v2: 
> >>>> 1: Use TRACE_EVENT instead of DEFINE_TRACE
> >>>> 2: Move the tracepoint from raise_softirq_irqoff() to
> >>>>    __raise_softirq_irqoff()
> >>>>
> >>>> Changelog for v2 -> v3: 
> >>>> Move the definition of __raise_softifq_irqoff() to .c file when
> >>>> CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
> >>>>
> >>>> Changelog for v3 -> v4: 
> >>>> 1: Come back to v2, and use forward declarations to avoid
> >>>>    recursive includes as Mathieu's suggestion
> >>>> 2: Modifiy the tracepoint name
> >>>> 3: Add comments for this tracepoint
> >>>>
> >>> This is a step in the right direction, but please see my email to Lai
> >>> about the fact that this assumes correct and undocumented include
> >>> dependencies in kernel/trace/events.c. Not explicitely stating the
> >>> include dependencies is a build error waiting to happen.
> >>>
> >>> Including interrupt.h under a ifdef would allow keeping track of
> >>> TRACE_EVENT specific build dependencies neatly on a per header basis.
> >> 
> >> This is all moot, the events.c file no longer exists and as not an issue.
> >> 
> > 
> > As Steve's says, use ftrace in ftrace.h not in events.c now. 
> > So, this mistake does not exist.
> > Dose this patch has other error? I expect for your views.
> > 
> > Thanks for your review, is great help to me. ;-) 
> 
> Hello,
> 
> It seems Mathieu has no other comments on this patch now.
> Ingo, what is your opinion on this patch?

There's a complication: this area of the softirq code needs fixes 
(unrelated to tracing).
 
This API:

inline void raise_softirq_irqoff(unsigned int nr)
{
        __raise_softirq_irqoff(nr);

        /*
         * If we're in an interrupt or softirq, we're done
         * (this also catches softirq-disabled code). We will
         * actually run the softirq once we return from
         * the irq or softirq.
         *
         * Otherwise we wake up ksoftirqd to make sure we
         * schedule the softirq soon.
         */
        if (!in_interrupt())
                wakeup_softirqd();
}

is broken with RT tasks (as recently reported to lkml), as when a 
real-time task wakes up ksoftirqd (which has lower priority) it wont 
execute and we starve softirq execution.

The proper solution would be to have a new API:

	raise_softirq_check()

and to remove the wakeup_softirqd() hack from raise_softirq_irqoff() 
- and put raise_softirq_check() to all places that use 
raise_softirq*() from process context.
 
raise_softirq_check() would execute softirq handlers from process 
context, if there's any pending ones. It has to be called outside of 
bh critical sections - i.e. often a bit after the raise_softirq() 
has been done.

__raise_softirq_irqoff() would be made private to kernel/softirq.c, 
and we'd only have two public APIs to trigger softirqs: 
raise_softirq() and raise_softirq_irqoff(). Both just set the 
pending flag and dont do any wakeup.

As a side-effect of these fixes, the tracepoints will be sorted out 
as well - there wont be any need to hack into 
__raise_softirq_irqoff().

	Ingo

Re: Re: [PATCH v4] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Zhaolei]

Ingo Molnar wrote:
> * Zhaolei <zhaolei@cn.fujitsu.com> wrote:
> 
>> * From: "Xiao Guangrong" <xiaoguangrong@cn.fujitsu.com>
>>> Steven Rostedt wrote:
>>>> On Thu, 14 May 2009, Mathieu Desnoyers wrote:
>>>>>> From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
>>>>>>
>>>>>> This patch is modified from Mathieu Desnoyers' patch. The original patch
>>>>>> can be found here: 
>>>>>> http://marc.info/?l=linux-kernel&m=123791201816245&w=2
>>>>>> This tracepoint can trace the time stamp when softirq action is raised. 
>>>>>>
>>>>>> Changelog for v1 -> v2: 
>>>>>> 1: Use TRACE_EVENT instead of DEFINE_TRACE
>>>>>> 2: Move the tracepoint from raise_softirq_irqoff() to
>>>>>>    __raise_softirq_irqoff()
>>>>>>
>>>>>> Changelog for v2 -> v3: 
>>>>>> Move the definition of __raise_softifq_irqoff() to .c file when
>>>>>> CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
>>>>>>
>>>>>> Changelog for v3 -> v4: 
>>>>>> 1: Come back to v2, and use forward declarations to avoid
>>>>>>    recursive includes as Mathieu's suggestion
>>>>>> 2: Modifiy the tracepoint name
>>>>>> 3: Add comments for this tracepoint
>>>>>>
>>>>> This is a step in the right direction, but please see my email to Lai
>>>>> about the fact that this assumes correct and undocumented include
>>>>> dependencies in kernel/trace/events.c. Not explicitely stating the
>>>>> include dependencies is a build error waiting to happen.
>>>>>
>>>>> Including interrupt.h under a ifdef would allow keeping track of
>>>>> TRACE_EVENT specific build dependencies neatly on a per header basis.
>>>> This is all moot, the events.c file no longer exists and as not an issue.
>>>>
>>> As Steve's says, use ftrace in ftrace.h not in events.c now. 
>>> So, this mistake does not exist.
>>> Dose this patch has other error? I expect for your views.
>>>
>>> Thanks for your review, is great help to me. ;-) 
>> Hello,
>>
>> It seems Mathieu has no other comments on this patch now.
>> Ingo, what is your opinion on this patch?
> 
> There's a complication: this area of the softirq code needs fixes 
> (unrelated to tracing).

Hello, Ingo

Thanks for your reply.

Since it is unrelated to tracing, why not add this tracepoint first
and fix the softirq code later? The tracepoint won't add any burden
to the fix work.

Thanks
Zhaolei

>  
> This API:
> 
> inline void raise_softirq_irqoff(unsigned int nr)
> {
>         __raise_softirq_irqoff(nr);
> 
>         /*
>          * If we're in an interrupt or softirq, we're done
>          * (this also catches softirq-disabled code). We will
>          * actually run the softirq once we return from
>          * the irq or softirq.
>          *
>          * Otherwise we wake up ksoftirqd to make sure we
>          * schedule the softirq soon.
>          */
>         if (!in_interrupt())
>                 wakeup_softirqd();
> }
> 
> is broken with RT tasks (as recently reported to lkml), as when a 
> real-time task wakes up ksoftirqd (which has lower priority) it wont 
> execute and we starve softirq execution.
> 
> The proper solution would be to have a new API:
> 
> 	raise_softirq_check()
> 
> and to remove the wakeup_softirqd() hack from raise_softirq_irqoff() 
> - and put raise_softirq_check() to all places that use 
> raise_softirq*() from process context.
>  
> raise_softirq_check() would execute softirq handlers from process 
> context, if there's any pending ones. It has to be called outside of 
> bh critical sections - i.e. often a bit after the raise_softirq() 
> has been done.
> 
> __raise_softirq_irqoff() would be made private to kernel/softirq.c, 
> and we'd only have two public APIs to trigger softirqs: 
> raise_softirq() and raise_softirq_irqoff(). Both just set the 
> pending flag and dont do any wakeup.
> 
> As a side-effect of these fixes, the tracepoints will be sorted out 
> as well - there wont be any need to hack into 
> __raise_softirq_irqoff().
> 
> 	Ingo
> 

Re: [PATCH v4] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Lai Jiangshan]

Ingo Molnar wrote:
> * Zhaolei <zhaolei@cn.fujitsu.com> wrote:
> 
>> * From: "Xiao Guangrong" <xiaoguangrong@cn.fujitsu.com>
>>> Steven Rostedt wrote:
>>>> On Thu, 14 May 2009, Mathieu Desnoyers wrote:
>>>>>> From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
>>>>>>
>>>>>> This patch is modified from Mathieu Desnoyers' patch. The original patch
>>>>>> can be found here: 
>>>>>> http://marc.info/?l=linux-kernel&m=123791201816245&w=2
>>>>>> This tracepoint can trace the time stamp when softirq action is raised. 
>>>>>>
>>>>>> Changelog for v1 -> v2: 
>>>>>> 1: Use TRACE_EVENT instead of DEFINE_TRACE
>>>>>> 2: Move the tracepoint from raise_softirq_irqoff() to
>>>>>>    __raise_softirq_irqoff()
>>>>>>
>>>>>> Changelog for v2 -> v3: 
>>>>>> Move the definition of __raise_softifq_irqoff() to .c file when
>>>>>> CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
>>>>>>
>>>>>> Changelog for v3 -> v4: 
>>>>>> 1: Come back to v2, and use forward declarations to avoid
>>>>>>    recursive includes as Mathieu's suggestion
>>>>>> 2: Modifiy the tracepoint name
>>>>>> 3: Add comments for this tracepoint
>>>>>>
>>>>> This is a step in the right direction, but please see my email to Lai
>>>>> about the fact that this assumes correct and undocumented include
>>>>> dependencies in kernel/trace/events.c. Not explicitely stating the
>>>>> include dependencies is a build error waiting to happen.
>>>>>
>>>>> Including interrupt.h under a ifdef would allow keeping track of
>>>>> TRACE_EVENT specific build dependencies neatly on a per header basis.
>>>> This is all moot, the events.c file no longer exists and as not an issue.
>>>>
>>> As Steve's says, use ftrace in ftrace.h not in events.c now. 
>>> So, this mistake does not exist.
>>> Dose this patch has other error? I expect for your views.
>>>
>>> Thanks for your review, is great help to me. ;-) 
>> Hello,
>>
>> It seems Mathieu has no other comments on this patch now.
>> Ingo, what is your opinion on this patch?
> 
> There's a complication: this area of the softirq code needs fixes 
> (unrelated to tracing).
>  
> This API:
> 
> inline void raise_softirq_irqoff(unsigned int nr)
> {
>         __raise_softirq_irqoff(nr);
> 
>         /*
>          * If we're in an interrupt or softirq, we're done
>          * (this also catches softirq-disabled code). We will
>          * actually run the softirq once we return from
>          * the irq or softirq.
>          *
>          * Otherwise we wake up ksoftirqd to make sure we
>          * schedule the softirq soon.
>          */
>         if (!in_interrupt())
>                 wakeup_softirqd();
> }
> 
> is broken with RT tasks (as recently reported to lkml), as when a 
> real-time task wakes up ksoftirqd (which has lower priority) it wont 
> execute and we starve softirq execution.
> 
> The proper solution would be to have a new API:
> 
> 	raise_softirq_check()
> 
> and to remove the wakeup_softirqd() hack from raise_softirq_irqoff() 
> - and put raise_softirq_check() to all places that use 
> raise_softirq*() from process context.

It's a nice solution. But I think it would be nicer when it is changed a little.

The new API raise_softirq_check() will become a very hard _burden_ to the users of raise_softirq*(). They have to find out a proper place to place the "raise_softirq_check();". It's not an easy things, functions are complicated and hard to be determined WHERE is the process context and WHEN(a function may be called from multi kinds of context).

Instead, I prefer that raise_softirq_check() is hidden from users. We call raise_softirq_check() from schedule(), it will handle the un-handle softirqs in time(if un-handle softirqs are too much, it'll wakeup the ksoftirqd).


Lai


>  
> raise_softirq_check() would execute softirq handlers from process 
> context, if there's any pending ones. It has to be called outside of 
> bh critical sections - i.e. often a bit after the raise_softirq() 
> has been done.
> 
> __raise_softirq_irqoff() would be made private to kernel/softirq.c, 
> and we'd only have two public APIs to trigger softirqs: 
> raise_softirq() and raise_softirq_irqoff(). Both just set the 
> pending flag and dont do any wakeup.
> 
> As a side-effect of these fixes, the tracepoints will be sorted out 
> as well - there wont be any need to hack into 
> __raise_softirq_irqoff().
> 
> 	Ingo
> 
> 
> 

[PATCH RFC] softirq: fix ksoftirq starved

[Lai Jiangshan]

Lai Jiangshan wrote:
> Ingo Molnar wrote:
>> * Zhaolei <zhaolei@cn.fujitsu.com> wrote:
>>
>>> * From: "Xiao Guangrong" <xiaoguangrong@cn.fujitsu.com>
>>>> Steven Rostedt wrote:
>>>>> On Thu, 14 May 2009, Mathieu Desnoyers wrote:
>>>>>>> From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
>>>>>>>
>>>>>>> This patch is modified from Mathieu Desnoyers' patch. The original patch
>>>>>>> can be found here: 
>>>>>>> http://marc.info/?l=linux-kernel&m=123791201816245&w=2
>>>>>>> This tracepoint can trace the time stamp when softirq action is raised. 
>>>>>>>
>>>>>>> Changelog for v1 -> v2: 
>>>>>>> 1: Use TRACE_EVENT instead of DEFINE_TRACE
>>>>>>> 2: Move the tracepoint from raise_softirq_irqoff() to
>>>>>>>    __raise_softirq_irqoff()
>>>>>>>
>>>>>>> Changelog for v2 -> v3: 
>>>>>>> Move the definition of __raise_softifq_irqoff() to .c file when
>>>>>>> CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
>>>>>>>
>>>>>>> Changelog for v3 -> v4: 
>>>>>>> 1: Come back to v2, and use forward declarations to avoid
>>>>>>>    recursive includes as Mathieu's suggestion
>>>>>>> 2: Modifiy the tracepoint name
>>>>>>> 3: Add comments for this tracepoint
>>>>>>>
>>>>>> This is a step in the right direction, but please see my email to Lai
>>>>>> about the fact that this assumes correct and undocumented include
>>>>>> dependencies in kernel/trace/events.c. Not explicitely stating the
>>>>>> include dependencies is a build error waiting to happen.
>>>>>>
>>>>>> Including interrupt.h under a ifdef would allow keeping track of
>>>>>> TRACE_EVENT specific build dependencies neatly on a per header basis.
>>>>> This is all moot, the events.c file no longer exists and as not an issue.
>>>>>
>>>> As Steve's says, use ftrace in ftrace.h not in events.c now. 
>>>> So, this mistake does not exist.
>>>> Dose this patch has other error? I expect for your views.
>>>>
>>>> Thanks for your review, is great help to me. ;-) 
>>> Hello,
>>>
>>> It seems Mathieu has no other comments on this patch now.
>>> Ingo, what is your opinion on this patch?
>> There's a complication: this area of the softirq code needs fixes 
>> (unrelated to tracing).
>>  
>> This API:
>>
>> inline void raise_softirq_irqoff(unsigned int nr)
>> {
>>         __raise_softirq_irqoff(nr);
>>
>>         /*
>>          * If we're in an interrupt or softirq, we're done
>>          * (this also catches softirq-disabled code). We will
>>          * actually run the softirq once we return from
>>          * the irq or softirq.
>>          *
>>          * Otherwise we wake up ksoftirqd to make sure we
>>          * schedule the softirq soon.
>>          */
>>         if (!in_interrupt())
>>                 wakeup_softirqd();
>> }
>>
>> is broken with RT tasks (as recently reported to lkml), as when a 
>> real-time task wakes up ksoftirqd (which has lower priority) it wont 
>> execute and we starve softirq execution.
>>
>> The proper solution would be to have a new API:
>>
>> 	raise_softirq_check()
>>
>> and to remove the wakeup_softirqd() hack from raise_softirq_irqoff() 
>> - and put raise_softirq_check() to all places that use 
>> raise_softirq*() from process context.
> 
> It's a nice solution. But I think it would be nicer when it is changed a little.
> 
> The new API raise_softirq_check() will become a very hard _burden_ to
> the users of raise_softirq*(). They have to find out a proper place to place
> the "raise_softirq_check();". It's not an easy things, functions are complicated
> and hard to be determined WHERE is the process context and WHEN(a function may be
> called from multi kinds of context).
> 
> Instead, I prefer that raise_softirq_check() is hidden from users. We call
> raise_softirq_check() from schedule(), it will handle the un-handle softirqs in time
> (if un-handle softirqs are too much, it'll wakeup the ksoftirqd).
> 
> 
> Lai
> 
> 
>>  
>> raise_softirq_check() would execute softirq handlers from process 
>> context, if there's any pending ones. It has to be called outside of 
>> bh critical sections - i.e. often a bit after the raise_softirq() 
>> has been done.
>>
>> __raise_softirq_irqoff() would be made private to kernel/softirq.c, 
>> and we'd only have two public APIs to trigger softirqs: 
>> raise_softirq() and raise_softirq_irqoff(). Both just set the 
>> pending flag and dont do any wakeup.
>>
>> As a side-effect of these fixes, the tracepoints will be sorted out 
>> as well - there wont be any need to hack into 
>> __raise_softirq_irqoff().
>>
>> 	Ingo
>>
>>
>>
> 

Subject: [PATCH RFC] softirq: fix ksoftirq starved

The API:
void raise_softirq_irqoff(unsigned int nr)
{
	__raise_softirq_irqoff(nr);

	/*
	 * If we're in an interrupt or softirq, we're done
	 * (this also catches softirq-disabled code). We will
	 * actually run the softirq once we return from
	 * the irq or softirq.
	 *
	 * Otherwise we wake up ksoftirqd to make sure we
	 * schedule the softirq soon.
	 */
	if (!in_interrupt())
		wakeup_softirqd();
}

is broken with RT tasks, as when a real-time task wakes up ksoftirqd
(which has lower priority) it wont execute and we starve softirq execution.

To make pending softirq is called in time, we check and run the pending
softirq in schedule().

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
---
diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
index dd574d5..f59e552 100644
--- a/include/linux/interrupt.h
+++ b/include/linux/interrupt.h
@@ -359,6 +359,7 @@ struct softirq_action
 
 asmlinkage void do_softirq(void);
 asmlinkage void __do_softirq(void);
+extern void schedule_softirq_check(void);
 extern void open_softirq(int nr, void (*action)(struct softirq_action *));
 extern void softirq_init(void);
 #define __raise_softirq_irqoff(nr) do { or_softirq_pending(1UL << (nr)); } while (0)
diff --git a/kernel/sched.c b/kernel/sched.c
index cfb0456..21b5f67 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -5307,6 +5307,7 @@ need_resched:
 	release_kernel_lock(prev);
 need_resched_nonpreemptible:
 
+	schedule_softirq_check();
 	schedule_debug(prev);
 
 	if (sched_feat(HRTICK))
diff --git a/kernel/softirq.c b/kernel/softirq.c
index b41fb71..59d142b 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -55,6 +55,7 @@ EXPORT_SYMBOL(irq_stat);
 static struct softirq_action softirq_vec[NR_SOFTIRQS] __cacheline_aligned_in_smp;
 
 static DEFINE_PER_CPU(struct task_struct *, ksoftirqd);
+static DEFINE_PER_CPU(int, schedule_softirq_check);
 
 char *softirq_to_name[NR_SOFTIRQS] = {
 	"HI", "TIMER", "NET_TX", "NET_RX", "BLOCK",
@@ -76,6 +77,14 @@ void wakeup_softirqd(void)
 		wake_up_process(tsk);
 }
 
+static void set_schedule_softirq_check(void)
+{
+	if (current != __get_cpu_var(ksoftirqd)) {
+		__get_cpu_var(schedule_softirq_check) = 1;
+		set_tsk_need_resched(current);
+	}
+}
+
 /*
  * This one is for softirq.c-internal use,
  * where hardirqs are disabled legitimately:
@@ -243,6 +252,7 @@ restart:
 
 	lockdep_softirq_exit();
 
+	__get_cpu_var(schedule_softirq_check) = 0;
 	account_system_vtime(current);
 	_local_bh_enable();
 }
@@ -269,6 +279,12 @@ asmlinkage void do_softirq(void)
 
 #endif
 
+void schedule_softirq_check(void)
+{
+	if (__get_cpu_var(schedule_softirq_check))
+		do_softirq();
+}
+
 /*
  * Enter an interrupt context.
  */
@@ -323,11 +339,16 @@ inline void raise_softirq_irqoff(unsigned int nr)
 	 * actually run the softirq once we return from
 	 * the irq or softirq.
 	 *
-	 * Otherwise we wake up ksoftirqd to make sure we
-	 * schedule the softirq soon.
+	 * Otherwise we check and run the pending softirq in schedule().
+	 *
+	 * NOTE: It's sometimes helpless to wakeup the ksoftirqd here.
+	 *       Because when current task is a real-time task(which
+	 *       has higher priority) or a real-time task is on standby
+	 *       in this cpu, ksoftirqd won't execute, we starve softirq
+	 *       execution.
 	 */
 	if (!in_interrupt())
-		wakeup_softirqd();
+		set_schedule_softirq_check();
 }
 
 void raise_softirq(unsigned int nr)

Re: [PATCH RFC] softirq: fix ksoftirq starved

[Ingo Molnar]

* Lai Jiangshan <laijs@cn.fujitsu.com> wrote:

> --- a/kernel/sched.c
> +++ b/kernel/sched.c
> @@ -5307,6 +5307,7 @@ need_resched:
>  	release_kernel_lock(prev);
>  need_resched_nonpreemptible:
>  
> +	schedule_softirq_check();
>  	schedule_debug(prev);

hm, this slows down the scheduler fast-path ...

	Ingo

Re: [PATCH RFC] softirq: fix ksoftirq starved

[Lai Jiangshan]

Ingo Molnar wrote:
> * Lai Jiangshan <laijs@cn.fujitsu.com> wrote:
> 
>> --- a/kernel/sched.c
>> +++ b/kernel/sched.c
>> @@ -5307,6 +5307,7 @@ need_resched:
>>  	release_kernel_lock(prev);
>>  need_resched_nonpreemptible:
>>  
>> +	schedule_softirq_check();
>>  	schedule_debug(prev);
> 
> hm, this slows down the scheduler fast-path ...
> 
> 	Ingo
> 
> 

It's true. But:

The overheads are:

Overhead-A: the function call statement "schedule_softirq_check();"
It can be gotten rid off by a macro or inline function.

Overhead-B: __get_cpu_var() and the test statement.

Overhead-C: do_softirq()
In my patch, we test a variable and then call do_softirq() when
the variable is true. do_softirq() can be called from process
context or from schedule() or by any other ways, but it must be
called and avoids starvation in this condition.
So we need pay this overhead. It is no worse than before.

Is it a critical thing when it slows down the scheduler fast-path
because of the "Overhead-B"?

Or I misunderstand something?

	Thanks, Lai.

Re: [PATCH RFC] softirq: fix ksoftirq starved

[Peter Zijlstra]

On Thu, 2009-06-18 at 11:19 +0800, Lai Jiangshan wrote:
> Ingo Molnar wrote:
> > * Lai Jiangshan <laijs@cn.fujitsu.com> wrote:
> > 
> >> --- a/kernel/sched.c
> >> +++ b/kernel/sched.c
> >> @@ -5307,6 +5307,7 @@ need_resched:
> >>  	release_kernel_lock(prev);
> >>  need_resched_nonpreemptible:
> >>  
> >> +	schedule_softirq_check();
> >>  	schedule_debug(prev);
> > 
> > hm, this slows down the scheduler fast-path ...
> > 
> > 	Ingo
> > 
> > 
> 
> It's true. But:
> 
> The overheads are:
> 
> Overhead-A: the function call statement "schedule_softirq_check();"
> It can be gotten rid off by a macro or inline function.
> 
> Overhead-B: __get_cpu_var() and the test statement.
> 
> Overhead-C: do_softirq()
> In my patch, we test a variable and then call do_softirq() when
> the variable is true. do_softirq() can be called from process
> context or from schedule() or by any other ways, but it must be
> called and avoids starvation in this condition.
> So we need pay this overhead. It is no worse than before.
> 
> Is it a critical thing when it slows down the scheduler fast-path
> because of the "Overhead-B"?
> 
> Or I misunderstand something?

I think its overhead-c, actually calling do_softirq from the schedule()
path that a really whacky idea.

Why not make whoemever needs the softirq to happen (eg. network ops)
poll this flag, so that regular RT processes don't get penalized by
random softirq muck?

It seems to me this approach basically gives softirqs higher prio than
RT processes, not something to be done lightly.

Re: [PATCH RFC] softirq: fix ksoftirq starved

[Ingo Molnar]

* Lai Jiangshan <laijs@cn.fujitsu.com> wrote:

> Ingo Molnar wrote:
> > * Lai Jiangshan <laijs@cn.fujitsu.com> wrote:
> > 
> >> --- a/kernel/sched.c
> >> +++ b/kernel/sched.c
> >> @@ -5307,6 +5307,7 @@ need_resched:
> >>  	release_kernel_lock(prev);
> >>  need_resched_nonpreemptible:
> >>  
> >> +	schedule_softirq_check();
> >>  	schedule_debug(prev);
> > 
> > hm, this slows down the scheduler fast-path ...
> > 
> > 	Ingo
> > 
> > 
> 
> It's true. But:
> 
> The overheads are:
> 
> Overhead-A: the function call statement "schedule_softirq_check();"
> It can be gotten rid off by a macro or inline function.
> 
> Overhead-B: __get_cpu_var() and the test statement.
> 
> Overhead-C: do_softirq()
> In my patch, we test a variable and then call do_softirq() when
> the variable is true. do_softirq() can be called from process
> context or from schedule() or by any other ways, but it must be
> called and avoids starvation in this condition.
> So we need pay this overhead. It is no worse than before.
> 
> Is it a critical thing when it slows down the scheduler fast-path
> because of the "Overhead-B"?
> 
> Or I misunderstand something?

The thing is that _any_ extra instruction in the scheduler fast-path 
should be avoided. I dont think it can be claimed that this problem 
can only be solved that way.

	Ingo

Re: [PATCH v4] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Lai Jiangshan]

Ingo Molnar wrote:
> * Zhaolei <zhaolei@cn.fujitsu.com> wrote:
> 
>> * From: "Xiao Guangrong" <xiaoguangrong@cn.fujitsu.com>
>>> Steven Rostedt wrote:
>>>> On Thu, 14 May 2009, Mathieu Desnoyers wrote:
>>>>>> From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
>>>>>>
>>>>>> This patch is modified from Mathieu Desnoyers' patch. The original patch
>>>>>> can be found here: 
>>>>>> http://marc.info/?l=linux-kernel&m=123791201816245&w=2
>>>>>> This tracepoint can trace the time stamp when softirq action is raised. 
>>>>>>
>>>>>> Changelog for v1 -> v2: 
>>>>>> 1: Use TRACE_EVENT instead of DEFINE_TRACE
>>>>>> 2: Move the tracepoint from raise_softirq_irqoff() to
>>>>>>    __raise_softirq_irqoff()
>>>>>>
>>>>>> Changelog for v2 -> v3: 
>>>>>> Move the definition of __raise_softifq_irqoff() to .c file when
>>>>>> CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
>>>>>>
>>>>>> Changelog for v3 -> v4: 
>>>>>> 1: Come back to v2, and use forward declarations to avoid
>>>>>>    recursive includes as Mathieu's suggestion
>>>>>> 2: Modifiy the tracepoint name
>>>>>> 3: Add comments for this tracepoint
>>>>>>
>>>>> This is a step in the right direction, but please see my email to Lai
>>>>> about the fact that this assumes correct and undocumented include
>>>>> dependencies in kernel/trace/events.c. Not explicitely stating the
>>>>> include dependencies is a build error waiting to happen.
>>>>>
>>>>> Including interrupt.h under a ifdef would allow keeping track of
>>>>> TRACE_EVENT specific build dependencies neatly on a per header basis.
>>>> This is all moot, the events.c file no longer exists and as not an issue.
>>>>
>>> As Steve's says, use ftrace in ftrace.h not in events.c now. 
>>> So, this mistake does not exist.
>>> Dose this patch has other error? I expect for your views.
>>>
>>> Thanks for your review, is great help to me. ;-) 
>> Hello,
>>
>> It seems Mathieu has no other comments on this patch now.
>> Ingo, what is your opinion on this patch?
> 
> There's a complication: this area of the softirq code needs fixes 
> (unrelated to tracing).
>  
> This API:
> 
> inline void raise_softirq_irqoff(unsigned int nr)
> {
>         __raise_softirq_irqoff(nr);
> 
>         /*
>          * If we're in an interrupt or softirq, we're done
>          * (this also catches softirq-disabled code). We will
>          * actually run the softirq once we return from
>          * the irq or softirq.
>          *
>          * Otherwise we wake up ksoftirqd to make sure we
>          * schedule the softirq soon.
>          */
>         if (!in_interrupt())
>                 wakeup_softirqd();
> }
> 
> is broken with RT tasks (as recently reported to lkml), as when a 
> real-time task wakes up ksoftirqd (which has lower priority) it wont 
> execute and we starve softirq execution.
> 
> The proper solution would be to have a new API:
> 
> 	raise_softirq_check()
> 
> and to remove the wakeup_softirqd() hack from raise_softirq_irqoff() 
> - and put raise_softirq_check() to all places that use 
> raise_softirq*() from process context.
>  
> raise_softirq_check() would execute softirq handlers from process 
> context, if there's any pending ones. It has to be called outside of 
> bh critical sections - i.e. often a bit after the raise_softirq() 
> has been done.
> 
> __raise_softirq_irqoff() would be made private to kernel/softirq.c, 
> and we'd only have two public APIs to trigger softirqs: 
> raise_softirq() and raise_softirq_irqoff(). Both just set the 
> pending flag and dont do any wakeup.
> 
> As a side-effect of these fixes, the tracepoints will be sorted out 
> as well - there wont be any need to hack into 
> __raise_softirq_irqoff().
> 
> 	Ingo
> 

Hi, Ingo

	Could you give me the .config file that this bug occurs?

Lai.

Re: [PATCH v4] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Ingo Molnar]

* Lai Jiangshan <laijs@cn.fujitsu.com> wrote:

> Ingo Molnar wrote:
> > * Zhaolei <zhaolei@cn.fujitsu.com> wrote:
> > 
> >> * From: "Xiao Guangrong" <xiaoguangrong@cn.fujitsu.com>
> >>> Steven Rostedt wrote:
> >>>> On Thu, 14 May 2009, Mathieu Desnoyers wrote:
> >>>>>> From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> >>>>>>
> >>>>>> This patch is modified from Mathieu Desnoyers' patch. The original patch
> >>>>>> can be found here: 
> >>>>>> http://marc.info/?l=linux-kernel&m=123791201816245&w=2
> >>>>>> This tracepoint can trace the time stamp when softirq action is raised. 
> >>>>>>
> >>>>>> Changelog for v1 -> v2: 
> >>>>>> 1: Use TRACE_EVENT instead of DEFINE_TRACE
> >>>>>> 2: Move the tracepoint from raise_softirq_irqoff() to
> >>>>>>    __raise_softirq_irqoff()
> >>>>>>
> >>>>>> Changelog for v2 -> v3: 
> >>>>>> Move the definition of __raise_softifq_irqoff() to .c file when
> >>>>>> CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
> >>>>>>
> >>>>>> Changelog for v3 -> v4: 
> >>>>>> 1: Come back to v2, and use forward declarations to avoid
> >>>>>>    recursive includes as Mathieu's suggestion
> >>>>>> 2: Modifiy the tracepoint name
> >>>>>> 3: Add comments for this tracepoint
> >>>>>>
> >>>>> This is a step in the right direction, but please see my email to Lai
> >>>>> about the fact that this assumes correct and undocumented include
> >>>>> dependencies in kernel/trace/events.c. Not explicitely stating the
> >>>>> include dependencies is a build error waiting to happen.
> >>>>>
> >>>>> Including interrupt.h under a ifdef would allow keeping track of
> >>>>> TRACE_EVENT specific build dependencies neatly on a per header basis.
> >>>> This is all moot, the events.c file no longer exists and as not an issue.
> >>>>
> >>> As Steve's says, use ftrace in ftrace.h not in events.c now. 
> >>> So, this mistake does not exist.
> >>> Dose this patch has other error? I expect for your views.
> >>>
> >>> Thanks for your review, is great help to me. ;-) 
> >> Hello,
> >>
> >> It seems Mathieu has no other comments on this patch now.
> >> Ingo, what is your opinion on this patch?
> > 
> > There's a complication: this area of the softirq code needs fixes 
> > (unrelated to tracing).
> >  
> > This API:
> > 
> > inline void raise_softirq_irqoff(unsigned int nr)
> > {
> >         __raise_softirq_irqoff(nr);
> > 
> >         /*
> >          * If we're in an interrupt or softirq, we're done
> >          * (this also catches softirq-disabled code). We will
> >          * actually run the softirq once we return from
> >          * the irq or softirq.
> >          *
> >          * Otherwise we wake up ksoftirqd to make sure we
> >          * schedule the softirq soon.
> >          */
> >         if (!in_interrupt())
> >                 wakeup_softirqd();
> > }
> > 
> > is broken with RT tasks (as recently reported to lkml), as when a 
> > real-time task wakes up ksoftirqd (which has lower priority) it wont 
> > execute and we starve softirq execution.
> > 
> > The proper solution would be to have a new API:
> > 
> > 	raise_softirq_check()
> > 
> > and to remove the wakeup_softirqd() hack from raise_softirq_irqoff() 
> > - and put raise_softirq_check() to all places that use 
> > raise_softirq*() from process context.
> >  
> > raise_softirq_check() would execute softirq handlers from process 
> > context, if there's any pending ones. It has to be called outside of 
> > bh critical sections - i.e. often a bit after the raise_softirq() 
> > has been done.
> > 
> > __raise_softirq_irqoff() would be made private to kernel/softirq.c, 
> > and we'd only have two public APIs to trigger softirqs: 
> > raise_softirq() and raise_softirq_irqoff(). Both just set the 
> > pending flag and dont do any wakeup.
> > 
> > As a side-effect of these fixes, the tracepoints will be sorted out 
> > as well - there wont be any need to hack into 
> > __raise_softirq_irqoff().
> > 
> > 	Ingo
> > 
> 
> Hi, Ingo
> 
> 	Could you give me the .config file that this bug occurs?

I havent reproduced this - it was reported in a recent thread on 
lkml. I dont have an URI for it - does anyone on the Cc: remember 
the details?

	Ingo

Re: [PATCH v4] ftrace: add a tracepoint for __raise_softirq_irqoff()

[Lai Jiangshan]

Ingo Molnar wrote:
> * Lai Jiangshan <laijs@cn.fujitsu.com> wrote:
> 
>> Ingo Molnar wrote:
>>> * Zhaolei <zhaolei@cn.fujitsu.com> wrote:
>>>
>>>> * From: "Xiao Guangrong" <xiaoguangrong@cn.fujitsu.com>
>>>>> Steven Rostedt wrote:
>>>>>> On Thu, 14 May 2009, Mathieu Desnoyers wrote:
>>>>>>>> From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
>>>>>>>>
>>>>>>>> This patch is modified from Mathieu Desnoyers' patch. The original patch
>>>>>>>> can be found here: 
>>>>>>>> http://marc.info/?l=linux-kernel&m=123791201816245&w=2
>>>>>>>> This tracepoint can trace the time stamp when softirq action is raised. 
>>>>>>>>
>>>>>>>> Changelog for v1 -> v2: 
>>>>>>>> 1: Use TRACE_EVENT instead of DEFINE_TRACE
>>>>>>>> 2: Move the tracepoint from raise_softirq_irqoff() to
>>>>>>>>    __raise_softirq_irqoff()
>>>>>>>>
>>>>>>>> Changelog for v2 -> v3: 
>>>>>>>> Move the definition of __raise_softifq_irqoff() to .c file when
>>>>>>>> CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
>>>>>>>>
>>>>>>>> Changelog for v3 -> v4: 
>>>>>>>> 1: Come back to v2, and use forward declarations to avoid
>>>>>>>>    recursive includes as Mathieu's suggestion
>>>>>>>> 2: Modifiy the tracepoint name
>>>>>>>> 3: Add comments for this tracepoint
>>>>>>>>
>>>>>>> This is a step in the right direction, but please see my email to Lai
>>>>>>> about the fact that this assumes correct and undocumented include
>>>>>>> dependencies in kernel/trace/events.c. Not explicitely stating the
>>>>>>> include dependencies is a build error waiting to happen.
>>>>>>>
>>>>>>> Including interrupt.h under a ifdef would allow keeping track of
>>>>>>> TRACE_EVENT specific build dependencies neatly on a per header basis.
>>>>>> This is all moot, the events.c file no longer exists and as not an issue.
>>>>>>
>>>>> As Steve's says, use ftrace in ftrace.h not in events.c now. 
>>>>> So, this mistake does not exist.
>>>>> Dose this patch has other error? I expect for your views.
>>>>>
>>>>> Thanks for your review, is great help to me. ;-) 
>>>> Hello,
>>>>
>>>> It seems Mathieu has no other comments on this patch now.
>>>> Ingo, what is your opinion on this patch?
>>> There's a complication: this area of the softirq code needs fixes 
>>> (unrelated to tracing).
>>>  
>>> This API:
>>>
>>> inline void raise_softirq_irqoff(unsigned int nr)
>>> {
>>>         __raise_softirq_irqoff(nr);
>>>
>>>         /*
>>>          * If we're in an interrupt or softirq, we're done
>>>          * (this also catches softirq-disabled code). We will
>>>          * actually run the softirq once we return from
>>>          * the irq or softirq.
>>>          *
>>>          * Otherwise we wake up ksoftirqd to make sure we
>>>          * schedule the softirq soon.
>>>          */
>>>         if (!in_interrupt())
>>>                 wakeup_softirqd();
>>> }
>>>
>>> is broken with RT tasks (as recently reported to lkml), as when a 
>>> real-time task wakes up ksoftirqd (which has lower priority) it wont 
>>> execute and we starve softirq execution.
>>>
>>> The proper solution would be to have a new API:
>>>
>>> 	raise_softirq_check()
>>>
>>> and to remove the wakeup_softirqd() hack from raise_softirq_irqoff() 
>>> - and put raise_softirq_check() to all places that use 
>>> raise_softirq*() from process context.
>>>  
>>> raise_softirq_check() would execute softirq handlers from process 
>>> context, if there's any pending ones. It has to be called outside of 
>>> bh critical sections - i.e. often a bit after the raise_softirq() 
>>> has been done.
>>>
>>> __raise_softirq_irqoff() would be made private to kernel/softirq.c, 
>>> and we'd only have two public APIs to trigger softirqs: 
>>> raise_softirq() and raise_softirq_irqoff(). Both just set the 
>>> pending flag and dont do any wakeup.
>>>
>>> As a side-effect of these fixes, the tracepoints will be sorted out 
>>> as well - there wont be any need to hack into 
>>> __raise_softirq_irqoff().
>>>
>>> 	Ingo
>>>
>> Hi, Ingo
>>
>> 	Could you give me the .config file that this bug occurs?
> 
> I havent reproduced this - it was reported in a recent thread on 
> lkml. I dont have an URI for it - does anyone on the Cc: remember 
> the details?
> 
> 	Ingo
> 
> 
> 

I haven't found the email you referred, I've searched it from
when you pointed out this softirq problem till now.
If anyone know it, please let me know.

If my analysis is correct, I will say this problem does NOT exist:
(non-PREEMPT_RT)

1) the (outmost) interrupt will call do_softirq() when it exits.
   softirq will not be starved when interrupts occur.

2) the clocks will provide tick-interrupt periodic.

3) even when CONFIG_NO_HZ=y, the cpu still provide periodic tick
   when current task is RT-task(which is non-idle task).

So any softirq will be called in time(one jiffy). Maybe my analysis is
incorrect or maybe the RT task disables the irq very long time or maybe
there is other bug in kernel which has influence and impacts softirq.

A) I will be still finding the email or the .config and probe
   this problem. The endeavors to ensure all softirq handled
   are not useful and it may not solve the problem. Because
   the tick-interrupt does ensure pending softirq handled.

B) If my thinking is correct, it's the best that we insert
   tracepoints at raise_softirq() functions, it's very helpful
   and one of its benefit is help us to probe problems
   which happen here.


	Lai