linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/7][RFC] function graph tracer port to PowerPC
@ 2009-02-12  1:10 Steven Rostedt
  2009-02-12  1:10 ` [PATCH 1/7][RFC] tracing/function-graph-tracer: make arch generic push pop functions Steven Rostedt
                   ` (10 more replies)
  0 siblings, 11 replies; 28+ messages in thread
From: Steven Rostedt @ 2009-02-12  1:10 UTC (permalink / raw)
  To: linux-kernel
  Cc: Ingo Molnar, Andrew Morton, Benjamin Herrenschmidt,
	Frederic Weisbecker, linuxppc-dev, Paul Mackerras


The following set of patches are RFC and not for inclusion
(unless everyone is fine with them as is).

This is the port to PowerPC of the function graph tracer that was written
by Frederic Weisbecker for the x86 architecture.  It is broken up
into a series of logical steps.

1) get generic code ready for other archs
2) get PowerPC 64-bit working with just static function tracing
3) get PowerPC 64-bit working with dynamic function tracing
4) get PowerPC 32-bit working with just static function tracing
5) get PowerPC 32-bit working with dynamic function tracing

(with some clean ups in between)


The function graph tracer not only traces the start of a function
(uses the function tracer part for that) but also uses the kprobes
trick to replace the return address with a hook to trace the exit
of the function. These hooks are generic in that other tracers
can also use them. But the function graph tracer itself is very
powerful. Simply doing the following:

# echo function_graph > /debug/tracing/current_tracer
# cat /debug/tracing/trace

# tracer: function_graph
#
# CPU OVERHEAD/DURATION            FUNCTION CALLS
# |   |        |                   |   |   |   |
 ------------------------------------------
 0)   less-2228    =>    cat-2229   
 ------------------------------------------

 0)               |    .__do_fault() {
 0)               |      .filemap_fault() {
 0)               |        .find_lock_page() {
 0)               |          .find_get_page() {
 0)   3.168 us    |            .__rcu_read_lock();
 0)   2.704 us    |            .__rcu_read_unlock();
 0) + 14.640 us   |          }
 0) + 20.112 us   |        }
 0) + 26.464 us   |      }
 0)   2.912 us    |      ._spin_lock();
 0)   2.656 us    |      .page_add_file_rmap();
 0)               |      .update_mmu_cache() {
 0)               |        .hash_preload() {
 0)   2.368 us    |          .get_slice_psize();
 0)   2.752 us    |          .hash_page_do_lazy_icache();
 0)   3.568 us    |          .native_hpte_insert();
 0) + 19.680 us   |        }
 0) + 24.960 us   |      }
 0)   2.336 us    |      ._spin_unlock();
 0)               |      .unlock_page() {
 0)   2.688 us    |        .page_waitqueue();
 0)   2.608 us    |        .__wake_up_bit();
 0) + 12.912 us   |      }
 0) ! 351.776 us  |    }
 0) ! 357.392 us  |  }
 0)   3.040 us    |  .up_read();
 0)               |  .compat_sys_ioctl() {
 0)   3.024 us    |    .fget_light();
 0)               |    .tty_compat_ioctl() {
 0)   2.704 us    |      .tty_paranoia_check();
 0)               |      .tty_ldisc_ref_wait() {
 0)               |        .tty_ldisc_try() {
 0)   2.880 us    |          ._spin_lock_irqsave();
 0)   2.928 us    |          ._spin_unlock_irqrestore();
 0) + 13.776 us   |        }
 0) + 19.424 us   |      }

[...]

As you can see, it gives a nice call trace of the functions being called
at run time, as well as a time stamp of how much time the function
took to execute.

Adding dynamic tracing to the mix, we can trace a single function:

# echo .do_fork > /debug/tracing/set_graph_function
# echo function_graph > /debug/tracing/current_tracer
# cat /debug/tracing/trace

# tracer: function_graph
#
# CPU OVERHEAD/DURATION            FUNCTION CALLS
# |   |        |                   |   |   |   |
 1)               |  .do_fork() {
 1)               |    .copy_process() {
 1)               |      .prepare_to_copy() {
 1)   2.944 us    |        .flush_fp_to_thread();
 1)   2.800 us    |        .flush_altivec_to_thread();
 1)   2.608 us    |        .flush_vsx_to_thread();
 1) + 19.184 us   |      }
 1)               |      .kmem_cache_alloc() {
 1)   2.464 us    |        .slab_should_failslab();
 1)   8.304 us    |      }
 1)               |      .alloc_thread_info() {
 1)               |        .kmem_cache_alloc() {
 1)   2.512 us    |          .slab_should_failslab();
 1)   8.224 us    |        }
 1) + 13.344 us   |      }
 1)   7.584 us    |      .arch_dup_task_struct();
 1)               |      .copy_creds() {
 1)   2.736 us    |        .__mutex_init();
 1)               |        .prepare_creds() {
 1)               |          .kmem_cache_alloc() {
 1)   2.640 us    |            .slab_should_failslab();
 1)   8.368 us    |          }

[...]


Note, the '.' in the '.do_fork' is a PowerPC64 thing. PowerPC32 and
other archs just need to do 'do_fork', without the dot.


The following patches are in:

  git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git

    branch: rfc/ppc/ftrace


Steven Rostedt (7):
      tracing/function-graph-tracer: make arch generic push pop functions
      powerpc64: port of the function graph tracer
      powerpc64, tracing: add function graph tracer with dynamic tracing
      powerpc64, ftrace: save toc only on modules for function graph
      powerpc32, ftrace: save and restore mcount regs with macro
      powerpc32, ftrace: port function graph tracer to ppc32, static only
      powerpc32, ftrace: dynamic function graph tracer

----
 arch/powerpc/Kconfig                 |    1 +
 arch/powerpc/include/asm/ftrace.h    |   39 ++++++++++-
 arch/powerpc/kernel/Makefile         |    9 +-
 arch/powerpc/kernel/entry_32.S       |  115 ++++++++++++++---------------
 arch/powerpc/kernel/entry_64.S       |   89 +++++++++++++++++++++-
 arch/powerpc/kernel/ftrace.c         |  135 ++++++++++++++++++++++++++++++++--
 arch/powerpc/kernel/process.c        |   16 ++++
 arch/powerpc/kernel/vmlinux.lds.S    |    1 +
 arch/x86/include/asm/ftrace.h        |   25 ------
 arch/x86/kernel/ftrace.c             |   75 +------------------
 include/linux/ftrace.h               |   24 ++++++
 kernel/trace/trace_functions_graph.c |   75 +++++++++++++++++++
 12 files changed, 428 insertions(+), 176 deletions(-)
-- 

^ permalink raw reply	[flat|nested] 28+ messages in thread

* [PATCH 1/7][RFC] tracing/function-graph-tracer: make arch generic push pop functions
  2009-02-12  1:10 [PATCH 0/7][RFC] function graph tracer port to PowerPC Steven Rostedt
@ 2009-02-12  1:10 ` Steven Rostedt
  2009-02-12  1:10 ` [PATCH 2/7][RFC] powerpc64: port of the function graph tracer Steven Rostedt
                   ` (9 subsequent siblings)
  10 siblings, 0 replies; 28+ messages in thread
From: Steven Rostedt @ 2009-02-12  1:10 UTC (permalink / raw)
  To: linux-kernel
  Cc: Ingo Molnar, Andrew Morton, Benjamin Herrenschmidt,
	Frederic Weisbecker, linuxppc-dev, Paul Mackerras,
	Steven Rostedt

[-- Attachment #1: 0001-tracing-function-graph-tracer-make-arch-generic-pus.patch --]
[-- Type: text/plain, Size: 7075 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

There is nothing really arch specific of the push and pop functions
used by the function graph tracer. This patch moves them to generic
code.

Signed-off-by: Steven Rostedt <srostedt@redhat.com>
---
 arch/x86/include/asm/ftrace.h        |   25 -----------
 arch/x86/kernel/ftrace.c             |   75 +---------------------------------
 include/linux/ftrace.h               |   24 +++++++++++
 kernel/trace/trace_functions_graph.c |   75 ++++++++++++++++++++++++++++++++++
 4 files changed, 100 insertions(+), 99 deletions(-)

diff --git a/arch/x86/include/asm/ftrace.h b/arch/x86/include/asm/ftrace.h
index b55b4a7..db24c22 100644
--- a/arch/x86/include/asm/ftrace.h
+++ b/arch/x86/include/asm/ftrace.h
@@ -55,29 +55,4 @@ struct dyn_arch_ftrace {
 #endif /* __ASSEMBLY__ */
 #endif /* CONFIG_FUNCTION_TRACER */
 
-#ifdef CONFIG_FUNCTION_GRAPH_TRACER
-
-#ifndef __ASSEMBLY__
-
-/*
- * Stack of return addresses for functions
- * of a thread.
- * Used in struct thread_info
- */
-struct ftrace_ret_stack {
-	unsigned long ret;
-	unsigned long func;
-	unsigned long long calltime;
-};
-
-/*
- * Primary handler of a function return.
- * It relays on ftrace_return_to_handler.
- * Defined in entry_32/64.S
- */
-extern void return_to_handler(void);
-
-#endif /* __ASSEMBLY__ */
-#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
-
 #endif /* _ASM_X86_FTRACE_H */
diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c
index 1b43086..258c8d5 100644
--- a/arch/x86/kernel/ftrace.c
+++ b/arch/x86/kernel/ftrace.c
@@ -389,79 +389,6 @@ void ftrace_nmi_exit(void)
 
 #endif /* !CONFIG_DYNAMIC_FTRACE */
 
-/* Add a function return address to the trace stack on thread info.*/
-static int push_return_trace(unsigned long ret, unsigned long long time,
-				unsigned long func, int *depth)
-{
-	int index;
-
-	if (!current->ret_stack)
-		return -EBUSY;
-
-	/* The return trace stack is full */
-	if (current->curr_ret_stack == FTRACE_RETFUNC_DEPTH - 1) {
-		atomic_inc(&current->trace_overrun);
-		return -EBUSY;
-	}
-
-	index = ++current->curr_ret_stack;
-	barrier();
-	current->ret_stack[index].ret = ret;
-	current->ret_stack[index].func = func;
-	current->ret_stack[index].calltime = time;
-	*depth = index;
-
-	return 0;
-}
-
-/* Retrieve a function return address to the trace stack on thread info.*/
-static void pop_return_trace(struct ftrace_graph_ret *trace, unsigned long *ret)
-{
-	int index;
-
-	index = current->curr_ret_stack;
-
-	if (unlikely(index < 0)) {
-		ftrace_graph_stop();
-		WARN_ON(1);
-		/* Might as well panic, otherwise we have no where to go */
-		*ret = (unsigned long)panic;
-		return;
-	}
-
-	*ret = current->ret_stack[index].ret;
-	trace->func = current->ret_stack[index].func;
-	trace->calltime = current->ret_stack[index].calltime;
-	trace->overrun = atomic_read(&current->trace_overrun);
-	trace->depth = index;
-	barrier();
-	current->curr_ret_stack--;
-
-}
-
-/*
- * Send the trace to the ring-buffer.
- * @return the original return address.
- */
-unsigned long ftrace_return_to_handler(void)
-{
-	struct ftrace_graph_ret trace;
-	unsigned long ret;
-
-	pop_return_trace(&trace, &ret);
-	trace.rettime = cpu_clock(raw_smp_processor_id());
-	ftrace_graph_return(&trace);
-
-	if (unlikely(!ret)) {
-		ftrace_graph_stop();
-		WARN_ON(1);
-		/* Might as well panic. What else to do? */
-		ret = (unsigned long)panic;
-	}
-
-	return ret;
-}
-
 /*
  * Hook the return address and push it in the stack of return addrs
  * in current thread info.
@@ -520,7 +447,7 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr)
 
 	calltime = cpu_clock(raw_smp_processor_id());
 
-	if (push_return_trace(old, calltime,
+	if (ftrace_push_return_trace(old, calltime,
 				self_addr, &trace.depth) == -EBUSY) {
 		*parent = old;
 		return;
diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h
index 677432b..a7f8134 100644
--- a/include/linux/ftrace.h
+++ b/include/linux/ftrace.h
@@ -380,6 +380,30 @@ struct ftrace_graph_ret {
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 
 /*
+ * Stack of return addresses for functions
+ * of a thread.
+ * Used in struct thread_info
+ */
+struct ftrace_ret_stack {
+	unsigned long ret;
+	unsigned long func;
+	unsigned long long calltime;
+};
+
+/*
+ * Primary handler of a function return.
+ * It relays on ftrace_return_to_handler.
+ * Defined in entry_32/64.S
+ */
+extern void return_to_handler(void);
+
+extern int
+ftrace_push_return_trace(unsigned long ret, unsigned long long time,
+			 unsigned long func, int *depth);
+extern void
+ftrace_pop_return_trace(struct ftrace_graph_ret *trace, unsigned long *ret);
+
+/*
  * Sometimes we don't want to trace a function with the function
  * graph tracer but we want them to keep traced by the usual function
  * tracer if the function graph tracer is not configured.
diff --git a/kernel/trace/trace_functions_graph.c b/kernel/trace/trace_functions_graph.c
index 930c08e..dce71a5 100644
--- a/kernel/trace/trace_functions_graph.c
+++ b/kernel/trace/trace_functions_graph.c
@@ -42,6 +42,81 @@ static struct tracer_flags tracer_flags = {
 /* pid on the last trace processed */
 static pid_t last_pid[NR_CPUS] = { [0 ... NR_CPUS-1] = -1 };
 
+/* Add a function return address to the trace stack on thread info.*/
+int
+ftrace_push_return_trace(unsigned long ret, unsigned long long time,
+			 unsigned long func, int *depth)
+{
+	int index;
+
+	if (!current->ret_stack)
+		return -EBUSY;
+
+	/* The return trace stack is full */
+	if (current->curr_ret_stack == FTRACE_RETFUNC_DEPTH - 1) {
+		atomic_inc(&current->trace_overrun);
+		return -EBUSY;
+	}
+
+	index = ++current->curr_ret_stack;
+	barrier();
+	current->ret_stack[index].ret = ret;
+	current->ret_stack[index].func = func;
+	current->ret_stack[index].calltime = time;
+	*depth = index;
+
+	return 0;
+}
+
+/* Retrieve a function return address to the trace stack on thread info.*/
+void
+ftrace_pop_return_trace(struct ftrace_graph_ret *trace, unsigned long *ret)
+{
+	int index;
+
+	index = current->curr_ret_stack;
+
+	if (unlikely(index < 0)) {
+		ftrace_graph_stop();
+		WARN_ON(1);
+		/* Might as well panic, otherwise we have no where to go */
+		*ret = (unsigned long)panic;
+		return;
+	}
+
+	*ret = current->ret_stack[index].ret;
+	trace->func = current->ret_stack[index].func;
+	trace->calltime = current->ret_stack[index].calltime;
+	trace->overrun = atomic_read(&current->trace_overrun);
+	trace->depth = index;
+	barrier();
+	current->curr_ret_stack--;
+
+}
+
+/*
+ * Send the trace to the ring-buffer.
+ * @return the original return address.
+ */
+unsigned long ftrace_return_to_handler(void)
+{
+	struct ftrace_graph_ret trace;
+	unsigned long ret;
+
+	ftrace_pop_return_trace(&trace, &ret);
+	trace.rettime = cpu_clock(raw_smp_processor_id());
+	ftrace_graph_return(&trace);
+
+	if (unlikely(!ret)) {
+		ftrace_graph_stop();
+		WARN_ON(1);
+		/* Might as well panic. What else to do? */
+		ret = (unsigned long)panic;
+	}
+
+	return ret;
+}
+
 static int graph_trace_init(struct trace_array *tr)
 {
 	int cpu, ret;
-- 
1.5.6.5

-- 

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH 2/7][RFC] powerpc64: port of the function graph tracer
  2009-02-12  1:10 [PATCH 0/7][RFC] function graph tracer port to PowerPC Steven Rostedt
  2009-02-12  1:10 ` [PATCH 1/7][RFC] tracing/function-graph-tracer: make arch generic push pop functions Steven Rostedt
@ 2009-02-12  1:10 ` Steven Rostedt
  2009-02-13  4:12   ` Benjamin Herrenschmidt
  2009-02-12  1:10 ` [PATCH 3/7][RFC] powerpc64, tracing: add function graph tracer with dynamic tracing Steven Rostedt
                   ` (8 subsequent siblings)
  10 siblings, 1 reply; 28+ messages in thread
From: Steven Rostedt @ 2009-02-12  1:10 UTC (permalink / raw)
  To: linux-kernel
  Cc: Ingo Molnar, Andrew Morton, Benjamin Herrenschmidt,
	Frederic Weisbecker, linuxppc-dev, Paul Mackerras,
	Steven Rostedt

[-- Attachment #1: 0002-powerpc64-port-of-the-function-graph-tracer.patch --]
[-- Type: text/plain, Size: 8695 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

This is a port of the function graph tracer that was written by
Frederic Weisbecker for the x86.

This only works for PPC64 at the moment and only for static tracing.
PPC32 and dynamic function graph tracing support will come later.

The trace produces a visual calling of functions:

 # tracer: function_graph
 #
 # CPU  DURATION                  FUNCTION CALLS
 # |     |   |                     |   |   |   |
  0)   2.224 us    |                        }
  0) ! 271.024 us  |                      }
  0) ! 320.080 us  |                    }
  0) ! 324.656 us  |                  }
  0) ! 329.136 us  |                }
  0)               |                .put_prev_task_fair() {
  0)               |                  .update_curr() {
  0)   2.240 us    |                    .update_min_vruntime();
  0)   6.512 us    |                  }
  0)   2.528 us    |                  .__enqueue_entity();
  0) + 15.536 us   |                }
  0)               |                .pick_next_task_fair() {
  0)   2.032 us    |                  .__pick_next_entity();
  0)   2.064 us    |                  .__clear_buddies();
  0)               |                  .set_next_entity() {
  0)   2.672 us    |                    .__dequeue_entity();
  0)   6.864 us    |                  }

Signed-off-by: Steven Rostedt <srostedt@redhat.com>
---
 arch/powerpc/Kconfig              |    1 +
 arch/powerpc/kernel/Makefile      |    9 ++--
 arch/powerpc/kernel/entry_64.S    |   58 +++++++++++++++++++++++++-
 arch/powerpc/kernel/ftrace.c      |   79 ++++++++++++++++++++++++++++++++++++-
 arch/powerpc/kernel/process.c     |   16 +++++++
 arch/powerpc/kernel/vmlinux.lds.S |    1 +
 6 files changed, 154 insertions(+), 10 deletions(-)

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 74cc312..ca4647e 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -111,6 +111,7 @@ config PPC
 	select HAVE_FTRACE_MCOUNT_RECORD
 	select HAVE_DYNAMIC_FTRACE
 	select HAVE_FUNCTION_TRACER
+	select HAVE_FUNCTION_GRAPH_TRACER if !DYNAMIC_FTRACE && PPC64
 	select ARCH_WANT_OPTIONAL_GPIOLIB
 	select HAVE_IDE
 	select HAVE_IOREMAP_PROT
diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile
index 8d1a419..a70aeea 100644
--- a/arch/powerpc/kernel/Makefile
+++ b/arch/powerpc/kernel/Makefile
@@ -18,12 +18,10 @@ CFLAGS_REMOVE_cputable.o = -pg -mno-sched-epilog
 CFLAGS_REMOVE_prom_init.o = -pg -mno-sched-epilog
 CFLAGS_REMOVE_btext.o = -pg -mno-sched-epilog
 CFLAGS_REMOVE_prom.o = -pg -mno-sched-epilog
-
-ifdef CONFIG_DYNAMIC_FTRACE
-# dynamic ftrace setup.
+# do not trace tracer code
 CFLAGS_REMOVE_ftrace.o = -pg -mno-sched-epilog
-endif
-
+# timers used by tracing
+CFLAGS_REMOVE_time.o = -pg -mno-sched-epilog
 endif
 
 obj-y				:= cputable.o ptrace.o syscalls.o \
@@ -94,6 +92,7 @@ obj-$(CONFIG_AUDIT)		+= audit.o
 obj64-$(CONFIG_AUDIT)		+= compat_audit.o
 
 obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
+obj-$(CONFIG_FUNCTION_GRAPH_TRACER)	+= ftrace.o
 
 obj-$(CONFIG_8XX_MINIMAL_FPEMU) += softemu8xx.o
 
diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index 383ed6e..a32699e 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -931,13 +931,65 @@ _GLOBAL(_mcount)
 	ld	r5,0(r5)
 	mtctr	r5
 	bctrl
-
 	nop
+
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+	b	ftrace_graph_caller
+#endif
 	ld	r0, 128(r1)
 	mtlr	r0
 	addi	r1, r1, 112
 _GLOBAL(ftrace_stub)
 	blr
 
-#endif
-#endif
+#endif /* CONFIG_DYNAMIC_FTRACE */
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+ftrace_graph_caller:
+	/* load r4 with local address */
+	ld	r4, 128(r1)
+	subi	r4, r4, MCOUNT_INSN_SIZE
+
+	/* get the parent address */
+	ld	r11, 112(r1)
+	addi	r3, r11, 16
+
+	bl	.prepare_ftrace_return
+	nop
+
+	ld	r0, 128(r1)
+	mtlr	r0
+	addi	r1, r1, 112
+	blr
+
+_GLOBAL(return_to_handler)
+	/* need to save return values */
+	std	r4,  -32(r1)
+	std	r3,  -24(r1)
+	/* save TOC */
+	std	r2,  -16(r1)
+	std	r31, -8(r1)
+	mr	r31, r1
+	stdu	r1, -112(r1)
+
+	/* update the TOC */
+	LOAD_REG_IMMEDIATE(r4,ftrace_return_to_handler)
+	ld	r2, 8(r4)
+
+	bl	.ftrace_return_to_handler
+	nop
+
+	/* return value has real return address */
+	mtlr	r3
+
+	ld	r1, 0(r1)
+	ld	r4,  -32(r1)
+	ld	r3,  -24(r1)
+	ld	r2,  -16(r1)
+	ld	r31, -8(r1)
+
+	/* Jump back to real return address */
+	blr
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
+#endif /* CONFIG_FUNCTION_TRACER */
diff --git a/arch/powerpc/kernel/ftrace.c b/arch/powerpc/kernel/ftrace.c
index 4112175..c9b1547 100644
--- a/arch/powerpc/kernel/ftrace.c
+++ b/arch/powerpc/kernel/ftrace.c
@@ -5,6 +5,9 @@
  *
  * Thanks goes out to P.A. Semi, Inc for supplying me with a PPC64 box.
  *
+ * Added function graph tracer code, taken from x86 that was written
+ * by Frederic Weisbecker, and ported to PPC by Steven Rostedt.
+ *
  */
 
 #include <linux/spinlock.h>
@@ -20,8 +23,6 @@
 #include <asm/code-patching.h>
 #include <asm/ftrace.h>
 
-static unsigned int ftrace_nop = PPC_NOP_INSTR;
-
 #ifdef CONFIG_PPC32
 # define GET_ADDR(addr) addr
 #else
@@ -29,6 +30,8 @@ static unsigned int ftrace_nop = PPC_NOP_INSTR;
 # define GET_ADDR(addr) (*(unsigned long *)addr)
 #endif
 
+#ifdef CONFIG_DYNAMIC_FTRACE
+static unsigned int ftrace_nop = PPC_NOP_INSTR;
 
 static unsigned int ftrace_calc_offset(long ip, long addr)
 {
@@ -525,3 +528,75 @@ int __init ftrace_dyn_arch_init(void *data)
 
 	return 0;
 }
+#endif /* CONFIG_DYNAMIC_FTRACE */
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+
+/*
+ * Hook the return address and push it in the stack of return addrs
+ * in current thread info.
+ */
+void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr)
+{
+	unsigned long old;
+	unsigned long long calltime;
+	int faulted;
+	struct ftrace_graph_ent trace;
+	unsigned long return_hooker = (unsigned long)
+				&return_to_handler;
+
+	if (unlikely(atomic_read(&current->tracing_graph_pause)))
+		return;
+
+	return_hooker = GET_ADDR(return_hooker);
+
+	/*
+	 * Protect against fault, even if it shouldn't
+	 * happen. This tool is too much intrusive to
+	 * ignore such a protection.
+	 */
+	asm volatile(
+		"1: " PPC_LL "%[old], 0(%[parent])\n"
+		"2: " PPC_STL "%[return_hooker], 0(%[parent])\n"
+		"   li %[faulted], 0\n"
+		"3:"
+
+		".section .fixup, \"ax\"\n"
+		"4: li %[faulted], 1\n"
+		"   b 3b\n"
+		".previous\n"
+
+		".section __ex_table,\"a\"\n"
+			PPC_LONG_ALIGN "\n"
+			PPC_LONG "1b,4b\n"
+			PPC_LONG "2b,4b\n"
+		".previous"
+
+		: [old] "=r" (old), [faulted] "=r" (faulted)
+		: [parent] "r" (parent), [return_hooker] "r" (return_hooker)
+		: "memory"
+	);
+
+	if (unlikely(faulted)) {
+		ftrace_graph_stop();
+		WARN_ON(1);
+		return;
+	}
+
+	calltime = cpu_clock(raw_smp_processor_id());
+
+	if (ftrace_push_return_trace(old, calltime,
+				self_addr, &trace.depth) == -EBUSY) {
+		*parent = old;
+		return;
+	}
+
+	trace.func = self_addr;
+
+	/* Only trace if the calling function expects to */
+	if (!ftrace_graph_entry(&trace)) {
+		current->curr_ret_stack--;
+		*parent = old;
+	}
+}
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index fb7049c..8ede428 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -33,6 +33,7 @@
 #include <linux/mqueue.h>
 #include <linux/hardirq.h>
 #include <linux/utsname.h>
+#include <linux/ftrace.h>
 #include <linux/kernel_stat.h>
 
 #include <asm/pgtable.h>
@@ -1008,6 +1009,14 @@ void show_stack(struct task_struct *tsk, unsigned long *stack)
 	unsigned long sp, ip, lr, newsp;
 	int count = 0;
 	int firstframe = 1;
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+	int curr_frame = current->curr_ret_stack;
+	extern void return_to_handler(void);
+	unsigned long addr = (unsigned long)return_to_handler;
+#ifdef CONFIG_PPC64
+	addr = *(unsigned long*)addr;
+#endif
+#endif
 
 	sp = (unsigned long) stack;
 	if (tsk == NULL)
@@ -1030,6 +1039,13 @@ void show_stack(struct task_struct *tsk, unsigned long *stack)
 		ip = stack[STACK_FRAME_LR_SAVE];
 		if (!firstframe || ip != lr) {
 			printk("["REG"] ["REG"] %pS", sp, ip, (void *)ip);
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+			if (ip == addr && curr_frame >= 0) {
+				printk(" (%pS)",
+				       (void *)current->ret_stack[curr_frame].ret);
+				curr_frame--;
+			}
+#endif
 			if (firstframe)
 				printk(" (unreliable)");
 			printk("\n");
diff --git a/arch/powerpc/kernel/vmlinux.lds.S b/arch/powerpc/kernel/vmlinux.lds.S
index 161b9b9..895af44 100644
--- a/arch/powerpc/kernel/vmlinux.lds.S
+++ b/arch/powerpc/kernel/vmlinux.lds.S
@@ -58,6 +58,7 @@ SECTIONS
 		SCHED_TEXT
 		LOCK_TEXT
 		KPROBES_TEXT
+		IRQENTRY_TEXT
 
 #ifdef CONFIG_PPC32
 		*(.got1)
-- 
1.5.6.5

-- 

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH 3/7][RFC] powerpc64, tracing: add function graph tracer with dynamic tracing
  2009-02-12  1:10 [PATCH 0/7][RFC] function graph tracer port to PowerPC Steven Rostedt
  2009-02-12  1:10 ` [PATCH 1/7][RFC] tracing/function-graph-tracer: make arch generic push pop functions Steven Rostedt
  2009-02-12  1:10 ` [PATCH 2/7][RFC] powerpc64: port of the function graph tracer Steven Rostedt
@ 2009-02-12  1:10 ` Steven Rostedt
  2009-02-13  4:15   ` Benjamin Herrenschmidt
  2009-02-12  1:10 ` [PATCH 4/7][RFC] powerpc64, ftrace: save toc only on modules for function graph Steven Rostedt
                   ` (7 subsequent siblings)
  10 siblings, 1 reply; 28+ messages in thread
From: Steven Rostedt @ 2009-02-12  1:10 UTC (permalink / raw)
  To: linux-kernel
  Cc: Ingo Molnar, Andrew Morton, Benjamin Herrenschmidt,
	Frederic Weisbecker, linuxppc-dev, Paul Mackerras,
	Steven Rostedt

[-- Attachment #1: 0003-powerpc64-tracing-add-function-graph-tracer-with-d.patch --]
[-- Type: text/plain, Size: 4609 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

This is the port of the function graph tracer to PowerPC with
dynamic tracing.

Signed-off-by: Steven Rostedt <srostedt@redhat.com>
---
 arch/powerpc/Kconfig           |    2 +-
 arch/powerpc/kernel/entry_64.S |    8 ++++++-
 arch/powerpc/kernel/ftrace.c   |   47 ++++++++++++++++++++++++++++++++++-----
 3 files changed, 49 insertions(+), 8 deletions(-)

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index ca4647e..6e1ee1b 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -111,7 +111,7 @@ config PPC
 	select HAVE_FTRACE_MCOUNT_RECORD
 	select HAVE_DYNAMIC_FTRACE
 	select HAVE_FUNCTION_TRACER
-	select HAVE_FUNCTION_GRAPH_TRACER if !DYNAMIC_FTRACE && PPC64
+	select HAVE_FUNCTION_GRAPH_TRACER if PPC64
 	select ARCH_WANT_OPTIONAL_GPIOLIB
 	select HAVE_IDE
 	select HAVE_IOREMAP_PROT
diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index a32699e..9f61fd6 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -908,6 +908,12 @@ _GLOBAL(ftrace_caller)
 ftrace_call:
 	bl	ftrace_stub
 	nop
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+.globl ftrace_graph_call
+ftrace_graph_call:
+	b	ftrace_graph_stub
+_GLOBAL(ftrace_graph_stub)
+#endif
 	ld	r0, 128(r1)
 	mtlr	r0
 	addi	r1, r1, 112
@@ -946,7 +952,7 @@ _GLOBAL(ftrace_stub)
 #endif /* CONFIG_DYNAMIC_FTRACE */
 
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
-ftrace_graph_caller:
+_GLOBAL(ftrace_graph_caller)
 	/* load r4 with local address */
 	ld	r4, 128(r1)
 	subi	r4, r4, MCOUNT_INSN_SIZE
diff --git a/arch/powerpc/kernel/ftrace.c b/arch/powerpc/kernel/ftrace.c
index c9b1547..7538b94 100644
--- a/arch/powerpc/kernel/ftrace.c
+++ b/arch/powerpc/kernel/ftrace.c
@@ -43,7 +43,8 @@ static unsigned char *ftrace_nop_replace(void)
 	return (char *)&ftrace_nop;
 }
 
-static unsigned char *ftrace_call_replace(unsigned long ip, unsigned long addr)
+static unsigned char *
+ftrace_call_replace(unsigned long ip, unsigned long addr, int link)
 {
 	static unsigned int op;
 
@@ -55,8 +56,9 @@ static unsigned char *ftrace_call_replace(unsigned long ip, unsigned long addr)
 	 */
 	addr = GET_ADDR(addr);
 
-	/* Set to "bl addr" */
-	op = 0x48000001 | (ftrace_calc_offset(ip, addr) & 0x03fffffc);
+	/* if (link) set op to 'bl' else 'b' */
+	op = 0x48000000 | (link ? 1 : 0);
+	op |= (ftrace_calc_offset(ip, addr) & 0x03fffffc);
 
 	/*
 	 * No locking needed, this must be called via kstop_machine
@@ -344,7 +346,7 @@ int ftrace_make_nop(struct module *mod,
 	 */
 	if (test_24bit_addr(ip, addr)) {
 		/* within range */
-		old = ftrace_call_replace(ip, addr);
+		old = ftrace_call_replace(ip, addr, 1);
 		new = ftrace_nop_replace();
 		return ftrace_modify_code(ip, old, new);
 	}
@@ -484,7 +486,7 @@ int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
 	if (test_24bit_addr(ip, addr)) {
 		/* within range */
 		old = ftrace_nop_replace();
-		new = ftrace_call_replace(ip, addr);
+		new = ftrace_call_replace(ip, addr, 1);
 		return ftrace_modify_code(ip, old, new);
 	}
 
@@ -513,7 +515,7 @@ int ftrace_update_ftrace_func(ftrace_func_t func)
 	int ret;
 
 	memcpy(old, &ftrace_call, MCOUNT_INSN_SIZE);
-	new = ftrace_call_replace(ip, (unsigned long)func);
+	new = ftrace_call_replace(ip, (unsigned long)func, 1);
 	ret = ftrace_modify_code(ip, old, new);
 
 	return ret;
@@ -532,6 +534,39 @@ int __init ftrace_dyn_arch_init(void *data)
 
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 
+#ifdef CONFIG_DYNAMIC_FTRACE
+extern void ftrace_graph_call(void);
+extern void ftrace_graph_stub(void);
+
+int ftrace_enable_ftrace_graph_caller(void)
+{
+	unsigned long ip = (unsigned long)(&ftrace_graph_call);
+	unsigned long addr = (unsigned long)(&ftrace_graph_caller);
+	unsigned long stub = (unsigned long)(&ftrace_graph_stub);
+	unsigned char old[MCOUNT_INSN_SIZE], *new;
+
+	new = ftrace_call_replace(ip, stub, 0);
+	memcpy(old, new, MCOUNT_INSN_SIZE);
+	new = ftrace_call_replace(ip, addr, 0);
+
+	return ftrace_modify_code(ip, old, new);
+}
+
+int ftrace_disable_ftrace_graph_caller(void)
+{
+	unsigned long ip = (unsigned long)(&ftrace_graph_call);
+	unsigned long addr = (unsigned long)(&ftrace_graph_caller);
+	unsigned long stub = (unsigned long)(&ftrace_graph_stub);
+	unsigned char old[MCOUNT_INSN_SIZE], *new;
+
+	new = ftrace_call_replace(ip, addr, 0);
+	memcpy(old, new, MCOUNT_INSN_SIZE);
+	new = ftrace_call_replace(ip, stub, 0);
+
+	return ftrace_modify_code(ip, old, new);
+}
+#endif /* CONFIG_DYNAMIC_FTRACE */
+
 /*
  * Hook the return address and push it in the stack of return addrs
  * in current thread info.
-- 
1.5.6.5

-- 

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH 4/7][RFC] powerpc64, ftrace: save toc only on modules for function graph
  2009-02-12  1:10 [PATCH 0/7][RFC] function graph tracer port to PowerPC Steven Rostedt
                   ` (2 preceding siblings ...)
  2009-02-12  1:10 ` [PATCH 3/7][RFC] powerpc64, tracing: add function graph tracer with dynamic tracing Steven Rostedt
@ 2009-02-12  1:10 ` Steven Rostedt
  2009-02-12  1:10 ` [PATCH 5/7][RFC] powerpc32, ftrace: save and restore mcount regs with macro Steven Rostedt
                   ` (6 subsequent siblings)
  10 siblings, 0 replies; 28+ messages in thread
From: Steven Rostedt @ 2009-02-12  1:10 UTC (permalink / raw)
  To: linux-kernel
  Cc: Ingo Molnar, Andrew Morton, Benjamin Herrenschmidt,
	Frederic Weisbecker, linuxppc-dev, Paul Mackerras,
	Steven Rostedt

[-- Attachment #1: 0004-powerpc64-ftrace-save-toc-only-on-modules-for-func.patch --]
[-- Type: text/plain, Size: 3061 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

The TOCS used by modules are different than the one used by
the core kernel code. The function graph tracer must save and
restore the TOC whenever it traces a module call. But this
is an added overhead to burden the majority of core kernel
code being traced.

Benjamin Herrenschmidt suggested in testing the entry of
the call to tell if it is a core kernel function or a module.
He recommended using the REGION_ID() macro to perform this test.

This patch implements Benjamin's idea, and uses a different
return_to_handler routine dependent on if the entry is a core
kernel function or not. The module version saves the TOC, where as
the core kernel version does not.

Signed-off-by: Steven Rostedt <srostedt@redhat.com>
---
 arch/powerpc/kernel/entry_64.S |   27 ++++++++++++++++++++++++++-
 arch/powerpc/kernel/ftrace.c   |   13 +++++++++++--
 2 files changed, 37 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index 9f61fd6..abfc323 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -971,6 +971,28 @@ _GLOBAL(ftrace_graph_caller)
 
 _GLOBAL(return_to_handler)
 	/* need to save return values */
+	std	r4,  -24(r1)
+	std	r3,  -16(r1)
+	std	r31, -8(r1)
+	mr	r31, r1
+	stdu	r1, -112(r1)
+
+	bl	.ftrace_return_to_handler
+	nop
+
+	/* return value has real return address */
+	mtlr	r3
+
+	ld	r1, 0(r1)
+	ld	r4,  -24(r1)
+	ld	r3,  -16(r1)
+	ld	r31, -8(r1)
+
+	/* Jump back to real return address */
+	blr
+
+_GLOBAL(mod_return_to_handler)
+	/* need to save return values */
 	std	r4,  -32(r1)
 	std	r3,  -24(r1)
 	/* save TOC */
@@ -979,7 +1001,10 @@ _GLOBAL(return_to_handler)
 	mr	r31, r1
 	stdu	r1, -112(r1)
 
-	/* update the TOC */
+	/*
+	 * We are in a module using the module's TOC.
+	 * Switch to our TOC to run inside the core kernel.
+	 */
 	LOAD_REG_IMMEDIATE(r4,ftrace_return_to_handler)
 	ld	r2, 8(r4)
 
diff --git a/arch/powerpc/kernel/ftrace.c b/arch/powerpc/kernel/ftrace.c
index 7538b94..5c6dfa9 100644
--- a/arch/powerpc/kernel/ftrace.c
+++ b/arch/powerpc/kernel/ftrace.c
@@ -567,6 +567,10 @@ int ftrace_disable_ftrace_graph_caller(void)
 }
 #endif /* CONFIG_DYNAMIC_FTRACE */
 
+#ifdef CONFIG_PPC64
+extern void mod_return_to_handler(void);
+#endif
+
 /*
  * Hook the return address and push it in the stack of return addrs
  * in current thread info.
@@ -577,12 +581,17 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr)
 	unsigned long long calltime;
 	int faulted;
 	struct ftrace_graph_ent trace;
-	unsigned long return_hooker = (unsigned long)
-				&return_to_handler;
+	unsigned long return_hooker = (unsigned long)&return_to_handler;
 
 	if (unlikely(atomic_read(&current->tracing_graph_pause)))
 		return;
 
+#if CONFIG_PPC64
+	/* non core kernel code needs to save and restore the TOC */
+	if (REGION_ID(self_addr) != KERNEL_REGION_ID)
+		return_hooker = (unsigned long)&mod_return_to_handler;
+#endif
+
 	return_hooker = GET_ADDR(return_hooker);
 
 	/*
-- 
1.5.6.5

-- 

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH 5/7][RFC] powerpc32, ftrace: save and restore mcount regs with macro
  2009-02-12  1:10 [PATCH 0/7][RFC] function graph tracer port to PowerPC Steven Rostedt
                   ` (3 preceding siblings ...)
  2009-02-12  1:10 ` [PATCH 4/7][RFC] powerpc64, ftrace: save toc only on modules for function graph Steven Rostedt
@ 2009-02-12  1:10 ` Steven Rostedt
  2009-02-12  1:10 ` [PATCH 6/7][RFC] powerpc32, ftrace: port function graph tracer to ppc32, static only Steven Rostedt
                   ` (5 subsequent siblings)
  10 siblings, 0 replies; 28+ messages in thread
From: Steven Rostedt @ 2009-02-12  1:10 UTC (permalink / raw)
  To: linux-kernel
  Cc: Ingo Molnar, Andrew Morton, Benjamin Herrenschmidt,
	Frederic Weisbecker, linuxppc-dev, Paul Mackerras

[-- Attachment #1: 0005-powerpc32-ftrace-save-and-restore-mcount-regs-with.patch --]
[-- Type: text/plain, Size: 3344 bytes --]

From: Steven Rostedt <rostedt@gollum.(none)>

Impact: clean up

Use a macro to save and restore the registers for PowerPC32,
since that code is duplicated.

This is similar to the work done by Cyrill Gorcunov for the
mcount code in x86_64.

Signed-off-by: Steven Rostedt <rostedt@gollum.(none)>
---
 arch/powerpc/include/asm/ftrace.h |   39 ++++++++++++++++++++-
 arch/powerpc/kernel/entry_32.S    |   68 +++++--------------------------------
 2 files changed, 47 insertions(+), 60 deletions(-)

diff --git a/arch/powerpc/include/asm/ftrace.h b/arch/powerpc/include/asm/ftrace.h
index e5f2ae8..dde1296 100644
--- a/arch/powerpc/include/asm/ftrace.h
+++ b/arch/powerpc/include/asm/ftrace.h
@@ -5,7 +5,44 @@
 #define MCOUNT_ADDR		((long)(_mcount))
 #define MCOUNT_INSN_SIZE	4 /* sizeof mcount call */
 
-#ifndef __ASSEMBLY__
+#ifdef __ASSEMBLY__
+
+/* Based off of objdump optput from glibc */
+
+#define MCOUNT_SAVE_FRAME			\
+	stwu	r1,-48(r1);			\
+	stw	r3, 12(r1);			\
+	stw	r4, 16(r1);			\
+	stw	r5, 20(r1);			\
+	stw	r6, 24(r1);			\
+	mflr	r3;				\
+	lwz	r4, 52(r1);			\
+	mfcr	r5;				\
+	stw	r7, 28(r1);			\
+	stw	r8, 32(r1);			\
+	stw	r9, 36(r1);			\
+	stw	r10,40(r1);			\
+	stw	r3, 44(r1);			\
+	stw	r5, 8(r1)
+
+#define MCOUNT_RESTORE_FRAME			\
+	lwz	r6, 8(r1);			\
+	lwz	r0, 44(r1);			\
+	lwz	r3, 12(r1);			\
+	mtctr	r0;				\
+	lwz	r4, 16(r1);			\
+	mtcr	r6;				\
+	lwz	r5, 20(r1);			\
+	lwz	r6, 24(r1);			\
+	lwz	r0, 52(r1);			\
+	lwz	r7, 28(r1);			\
+	lwz	r8, 32(r1);			\
+	mtlr	r0;				\
+	lwz	r9, 36(r1);			\
+	lwz	r10,40(r1);			\
+	addi	r1, r1, 48
+
+#else /* !__ASSEMBLY__ */
 extern void _mcount(void);
 
 #ifdef CONFIG_DYNAMIC_FTRACE
diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index 6f7eb7e..eb0c13e 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S
@@ -1176,59 +1176,22 @@ _GLOBAL(_mcount)
 	bctr
 
 _GLOBAL(ftrace_caller)
-	/* Based off of objdump optput from glibc */
-	stwu	r1,-48(r1)
-	stw	r3, 12(r1)
-	stw	r4, 16(r1)
-	stw	r5, 20(r1)
-	stw	r6, 24(r1)
-	mflr	r3
-	lwz	r4, 52(r1)
-	mfcr	r5
-	stw	r7, 28(r1)
-	stw	r8, 32(r1)
-	stw	r9, 36(r1)
-	stw	r10,40(r1)
-	stw	r3, 44(r1)
-	stw	r5, 8(r1)
+	MCOUNT_SAVE_FRAME
+	/* r3 ends up with link register */
 	subi	r3, r3, MCOUNT_INSN_SIZE
 .globl ftrace_call
 ftrace_call:
 	bl	ftrace_stub
 	nop
-	lwz	r6, 8(r1)
-	lwz	r0, 44(r1)
-	lwz	r3, 12(r1)
-	mtctr	r0
-	lwz	r4, 16(r1)
-	mtcr	r6
-	lwz	r5, 20(r1)
-	lwz	r6, 24(r1)
-	lwz	r0, 52(r1)
-	lwz	r7, 28(r1)
-	lwz	r8, 32(r1)
-	mtlr	r0
-	lwz	r9, 36(r1)
-	lwz	r10,40(r1)
-	addi	r1, r1, 48
+
+	MCOUNT_RESTORE_FRAME
+	/* old link register ends up in ctr reg */
 	bctr
 #else
 _GLOBAL(mcount)
 _GLOBAL(_mcount)
-	stwu	r1,-48(r1)
-	stw	r3, 12(r1)
-	stw	r4, 16(r1)
-	stw	r5, 20(r1)
-	stw	r6, 24(r1)
-	mflr	r3
-	lwz	r4, 52(r1)
-	mfcr	r5
-	stw	r7, 28(r1)
-	stw	r8, 32(r1)
-	stw	r9, 36(r1)
-	stw	r10,40(r1)
-	stw	r3, 44(r1)
-	stw	r5, 8(r1)
+
+	MCOUNT_SAVE_FRAME
 
 	subi	r3, r3, MCOUNT_INSN_SIZE
 	LOAD_REG_ADDR(r5, ftrace_trace_function)
@@ -1239,21 +1202,8 @@ _GLOBAL(_mcount)
 
 	nop
 
-	lwz	r6, 8(r1)
-	lwz	r0, 44(r1)
-	lwz	r3, 12(r1)
-	mtctr	r0
-	lwz	r4, 16(r1)
-	mtcr	r6
-	lwz	r5, 20(r1)
-	lwz	r6, 24(r1)
-	lwz	r0, 52(r1)
-	lwz	r7, 28(r1)
-	lwz	r8, 32(r1)
-	mtlr	r0
-	lwz	r9, 36(r1)
-	lwz	r10,40(r1)
-	addi	r1, r1, 48
+	MCOUNT_RESTORE_FRAME
+	
 	bctr
 #endif
 
-- 
1.5.6.5

-- 

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH 6/7][RFC] powerpc32, ftrace: port function graph tracer to ppc32, static only
  2009-02-12  1:10 [PATCH 0/7][RFC] function graph tracer port to PowerPC Steven Rostedt
                   ` (4 preceding siblings ...)
  2009-02-12  1:10 ` [PATCH 5/7][RFC] powerpc32, ftrace: save and restore mcount regs with macro Steven Rostedt
@ 2009-02-12  1:10 ` Steven Rostedt
  2009-02-12  1:10 ` [PATCH 7/7][RFC] powerpc32, ftrace: dynamic function graph tracer Steven Rostedt
                   ` (4 subsequent siblings)
  10 siblings, 0 replies; 28+ messages in thread
From: Steven Rostedt @ 2009-02-12  1:10 UTC (permalink / raw)
  To: linux-kernel
  Cc: Ingo Molnar, Andrew Morton, Benjamin Herrenschmidt,
	Frederic Weisbecker, linuxppc-dev, Paul Mackerras

[-- Attachment #1: 0006-powerpc32-ftrace-port-function-graph-tracer-to-ppc.patch --]
[-- Type: text/plain, Size: 2523 bytes --]

From: Steven Rostedt <rostedt@goodmis.org>

This patch ports the function graph tracer for PowerPC, but only
for static function tracing.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 arch/powerpc/Kconfig           |    2 +-
 arch/powerpc/kernel/entry_32.S |   43 +++++++++++++++++++++++++++++++++++++++-
 arch/powerpc/kernel/ftrace.c   |    2 +-
 3 files changed, 44 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 6e1ee1b..9e4bd20 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -111,7 +111,7 @@ config PPC
 	select HAVE_FTRACE_MCOUNT_RECORD
 	select HAVE_DYNAMIC_FTRACE
 	select HAVE_FUNCTION_TRACER
-	select HAVE_FUNCTION_GRAPH_TRACER if PPC64
+	select HAVE_FUNCTION_GRAPH_TRACER if PPC64 || !DYNAMIC_FTRACE
 	select ARCH_WANT_OPTIONAL_GPIOLIB
 	select HAVE_IDE
 	select HAVE_IOREMAP_PROT
diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index eb0c13e..9cf7083 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S
@@ -1199,9 +1199,11 @@ _GLOBAL(_mcount)
 
 	mtctr	r5
 	bctrl
-
 	nop
 
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+	b	ftrace_graph_caller
+#endif
 	MCOUNT_RESTORE_FRAME
 	
 	bctr
@@ -1210,4 +1212,43 @@ _GLOBAL(_mcount)
 _GLOBAL(ftrace_stub)
 	blr
 
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+_GLOBAL(ftrace_graph_caller)
+	/* load r4 with local address */
+	lwz	r4, 44(r1)
+	subi	r4, r4, MCOUNT_INSN_SIZE
+
+	/* get the parent address */
+	addi	r3, r1, 52
+
+	bl	prepare_ftrace_return
+	nop
+
+	MCOUNT_RESTORE_FRAME
+	/* old link register ends up in ctr reg */
+	bctr
+
+_GLOBAL(return_to_handler)
+	/* need to save return values */
+	stwu	r1, -32(r1)
+	stw	r3, 20(r1)
+	stw	r4, 16(r1)
+	stw	r31, 12(r1)
+	mr	r31, r1
+
+	bl	ftrace_return_to_handler
+	nop
+
+	/* return value has real return address */
+	mtlr	r3
+
+	lwz	r3, 20(r1)
+	lwz	r4, 16(r1)
+	lwz	r31,12(r1)
+	lwz	r1, 0(r1)
+
+	/* Jump back to real return address */
+	blr
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
+
 #endif /* CONFIG_MCOUNT */
diff --git a/arch/powerpc/kernel/ftrace.c b/arch/powerpc/kernel/ftrace.c
index 5c6dfa9..dddd99b 100644
--- a/arch/powerpc/kernel/ftrace.c
+++ b/arch/powerpc/kernel/ftrace.c
@@ -603,7 +603,7 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr)
 		"1: " PPC_LL "%[old], 0(%[parent])\n"
 		"2: " PPC_STL "%[return_hooker], 0(%[parent])\n"
 		"   li %[faulted], 0\n"
-		"3:"
+		"3:\n"
 
 		".section .fixup, \"ax\"\n"
 		"4: li %[faulted], 1\n"
-- 
1.5.6.5

-- 

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH 7/7][RFC] powerpc32, ftrace: dynamic function graph tracer
  2009-02-12  1:10 [PATCH 0/7][RFC] function graph tracer port to PowerPC Steven Rostedt
                   ` (5 preceding siblings ...)
  2009-02-12  1:10 ` [PATCH 6/7][RFC] powerpc32, ftrace: port function graph tracer to ppc32, static only Steven Rostedt
@ 2009-02-12  1:10 ` Steven Rostedt
  2009-02-12  1:55 ` [PATCH 0/7][RFC] function graph tracer port to PowerPC Frederic Weisbecker
                   ` (3 subsequent siblings)
  10 siblings, 0 replies; 28+ messages in thread
From: Steven Rostedt @ 2009-02-12  1:10 UTC (permalink / raw)
  To: linux-kernel
  Cc: Ingo Molnar, Andrew Morton, Benjamin Herrenschmidt,
	Frederic Weisbecker, linuxppc-dev, Paul Mackerras

[-- Attachment #1: 0007-powerpc32-ftrace-dynamic-function-graph-tracer.patch --]
[-- Type: text/plain, Size: 1396 bytes --]

From: Steven Rostedt <rostedt@gollum.(none)>

This patch gets function graph tracing working with dynamic function
tracer on PowerPC32.

Signed-off-by: Steven Rostedt <rostedt@gollum.(none)>
---
 arch/powerpc/Kconfig           |    2 +-
 arch/powerpc/kernel/entry_32.S |    8 ++++++--
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 9e4bd20..40b7981 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -111,7 +111,7 @@ config PPC
 	select HAVE_FTRACE_MCOUNT_RECORD
 	select HAVE_DYNAMIC_FTRACE
 	select HAVE_FUNCTION_TRACER
-	select HAVE_FUNCTION_GRAPH_TRACER if PPC64 || !DYNAMIC_FTRACE
+	select HAVE_FUNCTION_GRAPH_TRACER
 	select ARCH_WANT_OPTIONAL_GPIOLIB
 	select HAVE_IDE
 	select HAVE_IOREMAP_PROT
diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index 9cf7083..529bfbb 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S
@@ -1183,7 +1183,12 @@ _GLOBAL(ftrace_caller)
 ftrace_call:
 	bl	ftrace_stub
 	nop
-
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+.globl ftrace_graph_call
+ftrace_graph_call:
+	b	ftrace_graph_stub
+_GLOBAL(ftrace_graph_stub)
+#endif
 	MCOUNT_RESTORE_FRAME
 	/* old link register ends up in ctr reg */
 	bctr
@@ -1205,7 +1210,6 @@ _GLOBAL(_mcount)
 	b	ftrace_graph_caller
 #endif
 	MCOUNT_RESTORE_FRAME
-	
 	bctr
 #endif
 
-- 
1.5.6.5

-- 

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* Re: [PATCH 0/7][RFC] function graph tracer port to PowerPC
  2009-02-12  1:10 [PATCH 0/7][RFC] function graph tracer port to PowerPC Steven Rostedt
                   ` (6 preceding siblings ...)
  2009-02-12  1:10 ` [PATCH 7/7][RFC] powerpc32, ftrace: dynamic function graph tracer Steven Rostedt
@ 2009-02-12  1:55 ` Frederic Weisbecker
  2009-02-12  2:16   ` Steven Rostedt
                     ` (2 more replies)
  2009-02-12  2:23 ` Michael Ellerman
                   ` (2 subsequent siblings)
  10 siblings, 3 replies; 28+ messages in thread
From: Frederic Weisbecker @ 2009-02-12  1:55 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: linux-kernel, Ingo Molnar, Andrew Morton, Benjamin Herrenschmidt,
	linuxppc-dev, Paul Mackerras

On Wed, Feb 11, 2009 at 08:10:51PM -0500, Steven Rostedt wrote:
> 
> The following set of patches are RFC and not for inclusion
> (unless everyone is fine with them as is).
> 
> This is the port to PowerPC of the function graph tracer that was written
> by Frederic Weisbecker for the x86 architecture.  It is broken up
> into a series of logical steps.
> 
> 1) get generic code ready for other archs
> 2) get PowerPC 64-bit working with just static function tracing
> 3) get PowerPC 64-bit working with dynamic function tracing
> 4) get PowerPC 32-bit working with just static function tracing
> 5) get PowerPC 32-bit working with dynamic function tracing
> 
> (with some clean ups in between)
> 


Thanks a lot Steven!
I'm sad to not having a Power Pc to test it...

 
> The function graph tracer not only traces the start of a function
> (uses the function tracer part for that) but also uses the kprobes
> trick to replace the return address with a hook to trace the exit
> of the function. These hooks are generic in that other tracers
> can also use them. But the function graph tracer itself is very
> powerful. Simply doing the following:
> 
> # echo function_graph > /debug/tracing/current_tracer
> # cat /debug/tracing/trace
> 
> # tracer: function_graph
> #
> # CPU OVERHEAD/DURATION            FUNCTION CALLS
> # |   |        |                   |   |   |   |
>  ------------------------------------------
>  0)   less-2228    =>    cat-2229   
>  ------------------------------------------
> 
>  0)               |    .__do_fault() {
>  0)               |      .filemap_fault() {
>  0)               |        .find_lock_page() {
>  0)               |          .find_get_page() {
>  0)   3.168 us    |            .__rcu_read_lock();
>  0)   2.704 us    |            .__rcu_read_unlock();
>  0) + 14.640 us   |          }
>  0) + 20.112 us   |        }
>  0) + 26.464 us   |      }
>  0)   2.912 us    |      ._spin_lock();
>  0)   2.656 us    |      .page_add_file_rmap();
>  0)               |      .update_mmu_cache() {
>  0)               |        .hash_preload() {
>  0)   2.368 us    |          .get_slice_psize();
>  0)   2.752 us    |          .hash_page_do_lazy_icache();
>  0)   3.568 us    |          .native_hpte_insert();
>  0) + 19.680 us   |        }
>  0) + 24.960 us   |      }
>  0)   2.336 us    |      ._spin_unlock();
>  0)               |      .unlock_page() {
>  0)   2.688 us    |        .page_waitqueue();
>  0)   2.608 us    |        .__wake_up_bit();
>  0) + 12.912 us   |      }
>  0) ! 351.776 us  |    }
>  0) ! 357.392 us  |  }
>  0)   3.040 us    |  .up_read();
>  0)               |  .compat_sys_ioctl() {
>  0)   3.024 us    |    .fget_light();
>  0)               |    .tty_compat_ioctl() {
>  0)   2.704 us    |      .tty_paranoia_check();
>  0)               |      .tty_ldisc_ref_wait() {
>  0)               |        .tty_ldisc_try() {
>  0)   2.880 us    |          ._spin_lock_irqsave();
>  0)   2.928 us    |          ._spin_unlock_irqrestore();
>  0) + 13.776 us   |        }
>  0) + 19.424 us   |      }
> 
> [...]
> 
> As you can see, it gives a nice call trace of the functions being called
> at run time, as well as a time stamp of how much time the function
> took to execute.
> 
> Adding dynamic tracing to the mix, we can trace a single function:
> 
> # echo .do_fork > /debug/tracing/set_graph_function
> # echo function_graph > /debug/tracing/current_tracer
> # cat /debug/tracing/trace
> 
> # tracer: function_graph
> #
> # CPU OVERHEAD/DURATION            FUNCTION CALLS
> # |   |        |                   |   |   |   |
>  1)               |  .do_fork() {
>  1)               |    .copy_process() {
>  1)               |      .prepare_to_copy() {
>  1)   2.944 us    |        .flush_fp_to_thread();
>  1)   2.800 us    |        .flush_altivec_to_thread();
>  1)   2.608 us    |        .flush_vsx_to_thread();
>  1) + 19.184 us   |      }
>  1)               |      .kmem_cache_alloc() {
>  1)   2.464 us    |        .slab_should_failslab();
>  1)   8.304 us    |      }
>  1)               |      .alloc_thread_info() {
>  1)               |        .kmem_cache_alloc() {
>  1)   2.512 us    |          .slab_should_failslab();
>  1)   8.224 us    |        }
>  1) + 13.344 us   |      }
>  1)   7.584 us    |      .arch_dup_task_struct();
>  1)               |      .copy_creds() {
>  1)   2.736 us    |        .__mutex_init();
>  1)               |        .prepare_creds() {
>  1)               |          .kmem_cache_alloc() {
>  1)   2.640 us    |            .slab_should_failslab();
>  1)   8.368 us    |          }
> 
> [...]
> 
> 
> Note, the '.' in the '.do_fork' is a PowerPC64 thing. PowerPC32 and
> other archs just need to do 'do_fork', without the dot.
> 
> 
> The following patches are in:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
> 
>     branch: rfc/ppc/ftrace
> 
> 
> Steven Rostedt (7):
>       tracing/function-graph-tracer: make arch generic push pop functions
>       powerpc64: port of the function graph tracer
>       powerpc64, tracing: add function graph tracer with dynamic tracing
>       powerpc64, ftrace: save toc only on modules for function graph
>       powerpc32, ftrace: save and restore mcount regs with macro
>       powerpc32, ftrace: port function graph tracer to ppc32, static only
>       powerpc32, ftrace: dynamic function graph tracer
> 
> ----
>  arch/powerpc/Kconfig                 |    1 +
>  arch/powerpc/include/asm/ftrace.h    |   39 ++++++++++-
>  arch/powerpc/kernel/Makefile         |    9 +-
>  arch/powerpc/kernel/entry_32.S       |  115 ++++++++++++++---------------
>  arch/powerpc/kernel/entry_64.S       |   89 +++++++++++++++++++++-
>  arch/powerpc/kernel/ftrace.c         |  135 ++++++++++++++++++++++++++++++++--
>  arch/powerpc/kernel/process.c        |   16 ++++
>  arch/powerpc/kernel/vmlinux.lds.S    |    1 +
>  arch/x86/include/asm/ftrace.h        |   25 ------
>  arch/x86/kernel/ftrace.c             |   75 +------------------
>  include/linux/ftrace.h               |   24 ++++++
>  kernel/trace/trace_functions_graph.c |   75 +++++++++++++++++++
>  12 files changed, 428 insertions(+), 176 deletions(-)
> -- 


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 0/7][RFC] function graph tracer port to PowerPC
  2009-02-12  1:55 ` [PATCH 0/7][RFC] function graph tracer port to PowerPC Frederic Weisbecker
@ 2009-02-12  2:16   ` Steven Rostedt
  2009-02-12  4:08     ` Frederic Weisbecker
  2009-02-12 16:31   ` Steven Rostedt
  2009-02-13  4:18   ` Benjamin Herrenschmidt
  2 siblings, 1 reply; 28+ messages in thread
From: Steven Rostedt @ 2009-02-12  2:16 UTC (permalink / raw)
  To: Frederic Weisbecker
  Cc: linux-kernel, Ingo Molnar, Andrew Morton, Benjamin Herrenschmidt,
	linuxppc-dev, Paul Mackerras


On Thu, 12 Feb 2009, Frederic Weisbecker wrote:

> On Wed, Feb 11, 2009 at 08:10:51PM -0500, Steven Rostedt wrote:
> > 
> > The following set of patches are RFC and not for inclusion
> > (unless everyone is fine with them as is).
> > 
> > This is the port to PowerPC of the function graph tracer that was written
> > by Frederic Weisbecker for the x86 architecture.  It is broken up
> > into a series of logical steps.
> > 
> > 1) get generic code ready for other archs
> > 2) get PowerPC 64-bit working with just static function tracing
> > 3) get PowerPC 64-bit working with dynamic function tracing
> > 4) get PowerPC 32-bit working with just static function tracing
> > 5) get PowerPC 32-bit working with dynamic function tracing
> > 
> > (with some clean ups in between)
> > 
> 
> 
> Thanks a lot Steven!
> I'm sad to not having a Power Pc to test it...

If you had a PowerPC, I doubt I would have been the one to port it ;-)

-- Steve


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 0/7][RFC] function graph tracer port to PowerPC
  2009-02-12  1:10 [PATCH 0/7][RFC] function graph tracer port to PowerPC Steven Rostedt
                   ` (7 preceding siblings ...)
  2009-02-12  1:55 ` [PATCH 0/7][RFC] function graph tracer port to PowerPC Frederic Weisbecker
@ 2009-02-12  2:23 ` Michael Ellerman
  2009-02-12  2:37   ` Steven Rostedt
  2009-02-12 16:35 ` Steven Rostedt
  2009-02-12 23:32 ` Geoff Levand
  10 siblings, 1 reply; 28+ messages in thread
From: Michael Ellerman @ 2009-02-12  2:23 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: linux-kernel, linuxppc-dev, Paul Mackerras, Frederic Weisbecker,
	Ingo Molnar, Andrew Morton

[-- Attachment #1: Type: text/plain, Size: 1321 bytes --]

On Wed, 2009-02-11 at 20:10 -0500, Steven Rostedt wrote:
> The following set of patches are RFC and not for inclusion
> (unless everyone is fine with them as is).
> 
> This is the port to PowerPC of the function graph tracer that was written
> by Frederic Weisbecker for the x86 architecture.  It is broken up
> into a series of logical steps.
> 
> 1) get generic code ready for other archs
> 2) get PowerPC 64-bit working with just static function tracing
> 3) get PowerPC 64-bit working with dynamic function tracing
> 4) get PowerPC 32-bit working with just static function tracing
> 5) get PowerPC 32-bit working with dynamic function tracing
> 
> (with some clean ups in between)
> 
> 
> The function graph tracer not only traces the start of a function
> (uses the function tracer part for that) but also uses the kprobes
> trick to replace the return address with a hook to trace the exit
> of the function.

You use the "kprobes trick", but none of the kprobes code (AFAICS).
Couldn't there be some common code between the two?

cheers

-- 
Michael Ellerman
OzLabs, IBM Australia Development Lab

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 0/7][RFC] function graph tracer port to PowerPC
  2009-02-12  2:23 ` Michael Ellerman
@ 2009-02-12  2:37   ` Steven Rostedt
  0 siblings, 0 replies; 28+ messages in thread
From: Steven Rostedt @ 2009-02-12  2:37 UTC (permalink / raw)
  To: Michael Ellerman
  Cc: linux-kernel, linuxppc-dev, Paul Mackerras, Frederic Weisbecker,
	Ingo Molnar, Andrew Morton


On Thu, 12 Feb 2009, Michael Ellerman wrote:
> > 
> > 
> > The function graph tracer not only traces the start of a function
> > (uses the function tracer part for that) but also uses the kprobes
> > trick to replace the return address with a hook to trace the exit
> > of the function.
> 
> You use the "kprobes trick", but none of the kprobes code (AFAICS).
> Couldn't there be some common code between the two?

I'm not 100% sure how kprobes work, but I believe they use traps. And then 
they set the return address to take another trap, to fix it. I could be 
totally off here, on how kprobes does this.

But ftrace is about executing the code directly. The return code jumps to 
a another function in asm that will set up the call to do the tracing, and 
then fix the return pointer back. The only thing that is similar between
the two approaches that I can tell, is that we both modify the return 
address of the function. I took a quick peek at the kprobes code and I 
see no easy way to share it.

-- Steve


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 0/7][RFC] function graph tracer port to PowerPC
  2009-02-12  2:16   ` Steven Rostedt
@ 2009-02-12  4:08     ` Frederic Weisbecker
  0 siblings, 0 replies; 28+ messages in thread
From: Frederic Weisbecker @ 2009-02-12  4:08 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: linux-kernel, Ingo Molnar, Andrew Morton, Benjamin Herrenschmidt,
	linuxppc-dev, Paul Mackerras

On Wed, Feb 11, 2009 at 09:16:57PM -0500, Steven Rostedt wrote:
> 
> On Thu, 12 Feb 2009, Frederic Weisbecker wrote:
> 
> > On Wed, Feb 11, 2009 at 08:10:51PM -0500, Steven Rostedt wrote:
> > > 
> > > The following set of patches are RFC and not for inclusion
> > > (unless everyone is fine with them as is).
> > > 
> > > This is the port to PowerPC of the function graph tracer that was written
> > > by Frederic Weisbecker for the x86 architecture.  It is broken up
> > > into a series of logical steps.
> > > 
> > > 1) get generic code ready for other archs
> > > 2) get PowerPC 64-bit working with just static function tracing
> > > 3) get PowerPC 64-bit working with dynamic function tracing
> > > 4) get PowerPC 32-bit working with just static function tracing
> > > 5) get PowerPC 32-bit working with dynamic function tracing
> > > 
> > > (with some clean ups in between)
> > > 
> > 
> > 
> > Thanks a lot Steven!
> > I'm sad to not having a Power Pc to test it...
> 
> If you had a PowerPC, I doubt I would have been the one to port it ;-)


Especially since you already implemented ftrace on PowerPc :)
I will acquire an Arm board soon to adapt function graph on Arm...
 
> -- Steve
> 


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 0/7][RFC] function graph tracer port to PowerPC
  2009-02-12  1:55 ` [PATCH 0/7][RFC] function graph tracer port to PowerPC Frederic Weisbecker
  2009-02-12  2:16   ` Steven Rostedt
@ 2009-02-12 16:31   ` Steven Rostedt
  2009-02-12 16:47     ` Frederic Weisbecker
  2009-02-13  4:18   ` Benjamin Herrenschmidt
  2 siblings, 1 reply; 28+ messages in thread
From: Steven Rostedt @ 2009-02-12 16:31 UTC (permalink / raw)
  To: Frederic Weisbecker
  Cc: linux-kernel, Ingo Molnar, Andrew Morton, Benjamin Herrenschmidt,
	linuxppc-dev, Paul Mackerras


On Thu, 12 Feb 2009, Frederic Weisbecker wrote:

> On Wed, Feb 11, 2009 at 08:10:51PM -0500, Steven Rostedt wrote:
> > 
> > The following set of patches are RFC and not for inclusion
> > (unless everyone is fine with them as is).
> > 
> > This is the port to PowerPC of the function graph tracer that was written
> > by Frederic Weisbecker for the x86 architecture.  It is broken up
> > into a series of logical steps.
> > 
> > 1) get generic code ready for other archs
> > 2) get PowerPC 64-bit working with just static function tracing
> > 3) get PowerPC 64-bit working with dynamic function tracing
> > 4) get PowerPC 32-bit working with just static function tracing
> > 5) get PowerPC 32-bit working with dynamic function tracing
> > 
> > (with some clean ups in between)
> > 
> 
> 
> Thanks a lot Steven!
> I'm sad to not having a Power Pc to test it...

BTW, Can I count that as an Acked-by: for the first patch. Since the first 
patch does modify your code.

-- Steve


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 0/7][RFC] function graph tracer port to PowerPC
  2009-02-12  1:10 [PATCH 0/7][RFC] function graph tracer port to PowerPC Steven Rostedt
                   ` (8 preceding siblings ...)
  2009-02-12  2:23 ` Michael Ellerman
@ 2009-02-12 16:35 ` Steven Rostedt
  2009-02-12 23:32 ` Geoff Levand
  10 siblings, 0 replies; 28+ messages in thread
From: Steven Rostedt @ 2009-02-12 16:35 UTC (permalink / raw)
  To: linux-kernel
  Cc: Ingo Molnar, Andrew Morton, Benjamin Herrenschmidt,
	Frederic Weisbecker, linuxppc-dev, Paul Mackerras


On Wed, 11 Feb 2009, Steven Rostedt wrote:

> 
> The following set of patches are RFC and not for inclusion
> (unless everyone is fine with them as is).
> 
> This is the port to PowerPC of the function graph tracer that was written
> by Frederic Weisbecker for the x86 architecture.  It is broken up
> into a series of logical steps.
> 
> 1) get generic code ready for other archs
> 2) get PowerPC 64-bit working with just static function tracing
> 3) get PowerPC 64-bit working with dynamic function tracing
> 4) get PowerPC 32-bit working with just static function tracing
> 5) get PowerPC 32-bit working with dynamic function tracing
> 
> (with some clean ups in between)

Ben, if you get some time (no rush really), can you give an acked-by
on each of the PPC patches. The first patch is ftrace generic, so
you can ignore that one.

I'll keep it in the RFC state, until I have an ack from either you or
Paul.

I'm not sure if these changes should go via you or Ingo. I'm thinking 
that, since the first change modifies core ftrace code, I'll send it 
towards tip, since all the rest depends on that first change.

Thanks,

-- Steve

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 0/7][RFC] function graph tracer port to PowerPC
  2009-02-12 16:31   ` Steven Rostedt
@ 2009-02-12 16:47     ` Frederic Weisbecker
  2009-02-12 16:58       ` Steven Rostedt
  0 siblings, 1 reply; 28+ messages in thread
From: Frederic Weisbecker @ 2009-02-12 16:47 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: linux-kernel, Ingo Molnar, Andrew Morton, Benjamin Herrenschmidt,
	linuxppc-dev, Paul Mackerras

On Thu, Feb 12, 2009 at 11:31:44AM -0500, Steven Rostedt wrote:
> 
> On Thu, 12 Feb 2009, Frederic Weisbecker wrote:
> 
> > On Wed, Feb 11, 2009 at 08:10:51PM -0500, Steven Rostedt wrote:
> > > 
> > > The following set of patches are RFC and not for inclusion
> > > (unless everyone is fine with them as is).
> > > 
> > > This is the port to PowerPC of the function graph tracer that was written
> > > by Frederic Weisbecker for the x86 architecture.  It is broken up
> > > into a series of logical steps.
> > > 
> > > 1) get generic code ready for other archs
> > > 2) get PowerPC 64-bit working with just static function tracing
> > > 3) get PowerPC 64-bit working with dynamic function tracing
> > > 4) get PowerPC 32-bit working with just static function tracing
> > > 5) get PowerPC 32-bit working with dynamic function tracing
> > > 
> > > (with some clean ups in between)
> > > 
> > 
> > 
> > Thanks a lot Steven!
> > I'm sad to not having a Power Pc to test it...
> 
> BTW, Can I count that as an Acked-by: for the first patch. Since the first 
> patch does modify your code.
> 
> -- Steve
> 


Yes of course, I knew most of it was architecture independant but I delayed
this TODO for future ports, and you've done it.

Thanks.

Just a micro detail: the ftrace_push/pop_return_trace are parts of
the core of the entry/return probe, something that could be used
by other users than the function graph tracer itself.

Perhaps it would be better to put them in kernel/trace/ftrace.c
What do you think?

Anyway, Acked-by: Frederic Weisbecker <fweisbec@gmail.com>


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 0/7][RFC] function graph tracer port to PowerPC
  2009-02-12 16:47     ` Frederic Weisbecker
@ 2009-02-12 16:58       ` Steven Rostedt
  0 siblings, 0 replies; 28+ messages in thread
From: Steven Rostedt @ 2009-02-12 16:58 UTC (permalink / raw)
  To: Frederic Weisbecker
  Cc: linux-kernel, Ingo Molnar, Andrew Morton, Benjamin Herrenschmidt,
	linuxppc-dev, Paul Mackerras


On Thu, 12 Feb 2009, Frederic Weisbecker wrote:
> 
> 
> Yes of course, I knew most of it was architecture independant but I delayed
> this TODO for future ports, and you've done it.
> 
> Thanks.
> 
> Just a micro detail: the ftrace_push/pop_return_trace are parts of
> the core of the entry/return probe, something that could be used
> by other users than the function graph tracer itself.
> 
> Perhaps it would be better to put them in kernel/trace/ftrace.c
> What do you think?

I'll go with your above method. We'll move it when it comes to that ;-)

> 
> Anyway, Acked-by: Frederic Weisbecker <fweisbec@gmail.com>

Thanks!

I'll update my repo.

-- Steve


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 0/7][RFC] function graph tracer port to PowerPC
  2009-02-12  1:10 [PATCH 0/7][RFC] function graph tracer port to PowerPC Steven Rostedt
                   ` (9 preceding siblings ...)
  2009-02-12 16:35 ` Steven Rostedt
@ 2009-02-12 23:32 ` Geoff Levand
  2009-02-12 23:41   ` Steven Rostedt
  10 siblings, 1 reply; 28+ messages in thread
From: Geoff Levand @ 2009-02-12 23:32 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: linux-kernel, linuxppc-dev, Paul Mackerras, Frederic Weisbecker,
	Ingo Molnar, Andrew Morton

On 02/11/2009 05:10 PM, Steven Rostedt wrote:
> This is the port to PowerPC of the function graph tracer that was written
> by Frederic Weisbecker for the x86 architecture.  It is broken up
> into a series of logical steps.

I added these to my ps3-linux.git tree.  Very casual testing shows
they seem to work OK.


Tested-by: Geoff Levand <geoffrey.levand@am.sony.com>  Working OK on PS3.


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 0/7][RFC] function graph tracer port to PowerPC
  2009-02-12 23:32 ` Geoff Levand
@ 2009-02-12 23:41   ` Steven Rostedt
  2009-02-12 23:44     ` Josh Boyer
  2009-02-12 23:44     ` Geoff Levand
  0 siblings, 2 replies; 28+ messages in thread
From: Steven Rostedt @ 2009-02-12 23:41 UTC (permalink / raw)
  To: Geoff Levand
  Cc: linux-kernel, linuxppc-dev, Paul Mackerras, Frederic Weisbecker,
	Ingo Molnar, Andrew Morton


On Thu, 12 Feb 2009, Geoff Levand wrote:

> On 02/11/2009 05:10 PM, Steven Rostedt wrote:
> > This is the port to PowerPC of the function graph tracer that was written
> > by Frederic Weisbecker for the x86 architecture.  It is broken up
> > into a series of logical steps.
> 
> I added these to my ps3-linux.git tree.  Very casual testing shows
> they seem to work OK.
> 
> 
> Tested-by: Geoff Levand <geoffrey.levand@am.sony.com>  Working OK on PS3.

Thanks!

Is the ps3 64 or 32 bit? I'll put your tested by on the appropriate 
patches.

-- Steve


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 0/7][RFC] function graph tracer port to PowerPC
  2009-02-12 23:41   ` Steven Rostedt
@ 2009-02-12 23:44     ` Josh Boyer
  2009-02-12 23:44     ` Geoff Levand
  1 sibling, 0 replies; 28+ messages in thread
From: Josh Boyer @ 2009-02-12 23:44 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: Geoff Levand, Frederic Weisbecker, linux-kernel, linuxppc-dev,
	Paul Mackerras, Ingo Molnar, Andrew Morton

On Thu, Feb 12, 2009 at 06:41:26PM -0500, Steven Rostedt wrote:
>
>On Thu, 12 Feb 2009, Geoff Levand wrote:
>
>> On 02/11/2009 05:10 PM, Steven Rostedt wrote:
>> > This is the port to PowerPC of the function graph tracer that was written
>> > by Frederic Weisbecker for the x86 architecture.  It is broken up
>> > into a series of logical steps.
>> 
>> I added these to my ps3-linux.git tree.  Very casual testing shows
>> they seem to work OK.
>> 
>> 
>> Tested-by: Geoff Levand <geoffrey.levand@am.sony.com>  Working OK on PS3.
>
>Thanks!
>
>Is the ps3 64 or 32 bit? I'll put your tested by on the appropriate 
>patches.

64-bit.

josh

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 0/7][RFC] function graph tracer port to PowerPC
  2009-02-12 23:41   ` Steven Rostedt
  2009-02-12 23:44     ` Josh Boyer
@ 2009-02-12 23:44     ` Geoff Levand
  2009-02-12 23:51       ` Steven Rostedt
  1 sibling, 1 reply; 28+ messages in thread
From: Geoff Levand @ 2009-02-12 23:44 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: linux-kernel, linuxppc-dev, Paul Mackerras, Frederic Weisbecker,
	Ingo Molnar, Andrew Morton

On 02/12/2009 03:41 PM, Steven Rostedt wrote:
> On Thu, 12 Feb 2009, Geoff Levand wrote:
> 
>> On 02/11/2009 05:10 PM, Steven Rostedt wrote:
>> > This is the port to PowerPC of the function graph tracer that was written
>> > by Frederic Weisbecker for the x86 architecture.  It is broken up
>> > into a series of logical steps.
>> 
>> I added these to my ps3-linux.git tree.  Very casual testing shows
>> they seem to work OK.
>> 
>> 
>> Tested-by: Geoff Levand <geoffrey.levand@am.sony.com>  Working OK on PS3.
> 
> Thanks!
> 
> Is the ps3 64 or 32 bit? I'll put your tested by on the appropriate 
> patches.

It uses the Cell processor, which is 64 bit.

-Geoff


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 0/7][RFC] function graph tracer port to PowerPC
  2009-02-12 23:44     ` Geoff Levand
@ 2009-02-12 23:51       ` Steven Rostedt
  0 siblings, 0 replies; 28+ messages in thread
From: Steven Rostedt @ 2009-02-12 23:51 UTC (permalink / raw)
  To: Geoff Levand
  Cc: linux-kernel, linuxppc-dev, Paul Mackerras, Frederic Weisbecker,
	Ingo Molnar, Andrew Morton


On Thu, 12 Feb 2009, Geoff Levand wrote:

> On 02/12/2009 03:41 PM, Steven Rostedt wrote:
> > On Thu, 12 Feb 2009, Geoff Levand wrote:
> > 
> >> On 02/11/2009 05:10 PM, Steven Rostedt wrote:
> >> > This is the port to PowerPC of the function graph tracer that was written
> >> > by Frederic Weisbecker for the x86 architecture.  It is broken up
> >> > into a series of logical steps.
> >> 
> >> I added these to my ps3-linux.git tree.  Very casual testing shows
> >> they seem to work OK.
> >> 
> >> 
> >> Tested-by: Geoff Levand <geoffrey.levand@am.sony.com>  Working OK on PS3.
> > 
> > Thanks!
> > 
> > Is the ps3 64 or 32 bit? I'll put your tested by on the appropriate 
> > patches.
> 
> It uses the Cell processor, which is 64 bit.

Thanks guys!

-- Steve


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 2/7][RFC] powerpc64: port of the function graph tracer
  2009-02-12  1:10 ` [PATCH 2/7][RFC] powerpc64: port of the function graph tracer Steven Rostedt
@ 2009-02-13  4:12   ` Benjamin Herrenschmidt
  2009-02-13  4:18     ` Steven Rostedt
  0 siblings, 1 reply; 28+ messages in thread
From: Benjamin Herrenschmidt @ 2009-02-13  4:12 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: linux-kernel, Ingo Molnar, Andrew Morton, Frederic Weisbecker,
	linuxppc-dev, Paul Mackerras, Steven Rostedt

> On Wed, 2009-02-11 at 20:10 -0500, Steven Rostedt wrote:
> 
> +# timers used by tracing
> +CFLAGS_REMOVE_time.o = -pg -mno-sched-epilog
>  endif

That means no tracing of the timer interrupts etc... maybe we should
just move the specific function that we don't want traced out to a
separate file ? 

Appart from that, it looks ok, though I might have missed something :-)

Cheers,
Ben.


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 3/7][RFC] powerpc64, tracing: add function graph tracer with dynamic tracing
  2009-02-12  1:10 ` [PATCH 3/7][RFC] powerpc64, tracing: add function graph tracer with dynamic tracing Steven Rostedt
@ 2009-02-13  4:15   ` Benjamin Herrenschmidt
  2009-02-13  4:20     ` Steven Rostedt
  2009-02-13  5:22     ` Steven Rostedt
  0 siblings, 2 replies; 28+ messages in thread
From: Benjamin Herrenschmidt @ 2009-02-13  4:15 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: linux-kernel, Ingo Molnar, Andrew Morton, Frederic Weisbecker,
	linuxppc-dev, Paul Mackerras, Steven Rostedt


> @@ -55,8 +56,9 @@ static unsigned char *ftrace_call_replace(unsigned
> long ip, unsigned long addr)
>  	 */
>  	addr = GET_ADDR(addr);
>  
> -	/* Set to "bl addr" */
> -	op = 0x48000001 | (ftrace_calc_offset(ip, addr) & 0x03fffffc);
> +	/* if (link) set op to 'bl' else 'b' */
> +	op = 0x48000000 | (link ? 1 : 0);
> +	op |= (ftrace_calc_offset(ip, addr) & 0x03fffffc);

Any reason why you aren't using the code in
arch/powerpc/lib/code-patching.c here ?

> 	new = ftrace_call_replace(ip, stub, 0);
> +	memcpy(old, new, MCOUNT_INSN_SIZE);
> +	new = ftrace_call_replace(ip, addr, 0);
> +
> +	return ftrace_modify_code(ip, old, new);
> +}

Heh, memcpy of 4 bytes :-) I hope gcc is smart enough to turn that into
a simple load/store ..

Cheers,
Ben.


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 2/7][RFC] powerpc64: port of the function graph tracer
  2009-02-13  4:12   ` Benjamin Herrenschmidt
@ 2009-02-13  4:18     ` Steven Rostedt
  0 siblings, 0 replies; 28+ messages in thread
From: Steven Rostedt @ 2009-02-13  4:18 UTC (permalink / raw)
  To: Benjamin Herrenschmidt
  Cc: linux-kernel, Ingo Molnar, Andrew Morton, Frederic Weisbecker,
	linuxppc-dev, Paul Mackerras, Steven Rostedt


On Fri, 13 Feb 2009, Benjamin Herrenschmidt wrote:

> > On Wed, 2009-02-11 at 20:10 -0500, Steven Rostedt wrote:
> > 
> > +# timers used by tracing
> > +CFLAGS_REMOVE_time.o = -pg -mno-sched-epilog
> >  endif
> 
> That means no tracing of the timer interrupts etc... maybe we should
> just move the specific function that we don't want traced out to a
> separate file ? 

The function graph tracer calls cpu_clock, which calls sched_clock, 
to get the times. There's no protection against recursion here, since
we want to let interrupts still be recorded, and we do not need to disable
interrupts.

But if the cpu_clock calls something that is traced, it will recurse, and 
cause a lockup. What ever functions those are, we could annotate with
notrace. I just used the x86 blind method of 'dont trace this file'.


> 
> Appart from that, it looks ok, though I might have missed something :-)

Thanks,

-- Steve


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 0/7][RFC] function graph tracer port to PowerPC
  2009-02-12  1:55 ` [PATCH 0/7][RFC] function graph tracer port to PowerPC Frederic Weisbecker
  2009-02-12  2:16   ` Steven Rostedt
  2009-02-12 16:31   ` Steven Rostedt
@ 2009-02-13  4:18   ` Benjamin Herrenschmidt
  2 siblings, 0 replies; 28+ messages in thread
From: Benjamin Herrenschmidt @ 2009-02-13  4:18 UTC (permalink / raw)
  To: Frederic Weisbecker
  Cc: Steven Rostedt, linux-kernel, Ingo Molnar, Andrew Morton,
	linuxppc-dev, Paul Mackerras

For some reason I didn't get 7/7 ...

Cheers,
Ben.



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 3/7][RFC] powerpc64, tracing: add function graph tracer with dynamic tracing
  2009-02-13  4:15   ` Benjamin Herrenschmidt
@ 2009-02-13  4:20     ` Steven Rostedt
  2009-02-13  5:22     ` Steven Rostedt
  1 sibling, 0 replies; 28+ messages in thread
From: Steven Rostedt @ 2009-02-13  4:20 UTC (permalink / raw)
  To: Benjamin Herrenschmidt
  Cc: linux-kernel, Ingo Molnar, Andrew Morton, Frederic Weisbecker,
	linuxppc-dev, Paul Mackerras, Steven Rostedt


On Fri, 13 Feb 2009, Benjamin Herrenschmidt wrote:

> 
> > @@ -55,8 +56,9 @@ static unsigned char *ftrace_call_replace(unsigned
> > long ip, unsigned long addr)
> >  	 */
> >  	addr = GET_ADDR(addr);
> >  
> > -	/* Set to "bl addr" */
> > -	op = 0x48000001 | (ftrace_calc_offset(ip, addr) & 0x03fffffc);
> > +	/* if (link) set op to 'bl' else 'b' */
> > +	op = 0x48000000 | (link ? 1 : 0);
> > +	op |= (ftrace_calc_offset(ip, addr) & 0x03fffffc);
> 
> Any reason why you aren't using the code in
> arch/powerpc/lib/code-patching.c here ?
> 
> > 	new = ftrace_call_replace(ip, stub, 0);
> > +	memcpy(old, new, MCOUNT_INSN_SIZE);
> > +	new = ftrace_call_replace(ip, addr, 0);
> > +
> > +	return ftrace_modify_code(ip, old, new);
> > +}
> 
> Heh, memcpy of 4 bytes :-) I hope gcc is smart enough to turn that into
> a simple load/store ..

hehe, I hated writing that. I just did not want to touch the (already 
working code) of the dynamic ftrace. I guess I could still use longs and 
then typecast them to char pointers for the ftrace_modify_code.

Thanks,

-- Steve


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH 3/7][RFC] powerpc64, tracing: add function graph tracer with dynamic tracing
  2009-02-13  4:15   ` Benjamin Herrenschmidt
  2009-02-13  4:20     ` Steven Rostedt
@ 2009-02-13  5:22     ` Steven Rostedt
  1 sibling, 0 replies; 28+ messages in thread
From: Steven Rostedt @ 2009-02-13  5:22 UTC (permalink / raw)
  To: Benjamin Herrenschmidt
  Cc: linux-kernel, Ingo Molnar, Andrew Morton, Frederic Weisbecker,
	linuxppc-dev, Paul Mackerras, Steven Rostedt


On Fri, 13 Feb 2009, Benjamin Herrenschmidt wrote:

> 
> > @@ -55,8 +56,9 @@ static unsigned char *ftrace_call_replace(unsigned
> > long ip, unsigned long addr)
> >  	 */
> >  	addr = GET_ADDR(addr);
> >  
> > -	/* Set to "bl addr" */
> > -	op = 0x48000001 | (ftrace_calc_offset(ip, addr) & 0x03fffffc);
> > +	/* if (link) set op to 'bl' else 'b' */
> > +	op = 0x48000000 | (link ? 1 : 0);
> > +	op |= (ftrace_calc_offset(ip, addr) & 0x03fffffc);
> 
> Any reason why you aren't using the code in
> arch/powerpc/lib/code-patching.c here ?

Yes, because I did not know about it ;-)

I'll write up a patch to change this. But I'll post this series as is for 
now.

Thanks,

-- Steve


^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2009-02-13  5:23 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-02-12  1:10 [PATCH 0/7][RFC] function graph tracer port to PowerPC Steven Rostedt
2009-02-12  1:10 ` [PATCH 1/7][RFC] tracing/function-graph-tracer: make arch generic push pop functions Steven Rostedt
2009-02-12  1:10 ` [PATCH 2/7][RFC] powerpc64: port of the function graph tracer Steven Rostedt
2009-02-13  4:12   ` Benjamin Herrenschmidt
2009-02-13  4:18     ` Steven Rostedt
2009-02-12  1:10 ` [PATCH 3/7][RFC] powerpc64, tracing: add function graph tracer with dynamic tracing Steven Rostedt
2009-02-13  4:15   ` Benjamin Herrenschmidt
2009-02-13  4:20     ` Steven Rostedt
2009-02-13  5:22     ` Steven Rostedt
2009-02-12  1:10 ` [PATCH 4/7][RFC] powerpc64, ftrace: save toc only on modules for function graph Steven Rostedt
2009-02-12  1:10 ` [PATCH 5/7][RFC] powerpc32, ftrace: save and restore mcount regs with macro Steven Rostedt
2009-02-12  1:10 ` [PATCH 6/7][RFC] powerpc32, ftrace: port function graph tracer to ppc32, static only Steven Rostedt
2009-02-12  1:10 ` [PATCH 7/7][RFC] powerpc32, ftrace: dynamic function graph tracer Steven Rostedt
2009-02-12  1:55 ` [PATCH 0/7][RFC] function graph tracer port to PowerPC Frederic Weisbecker
2009-02-12  2:16   ` Steven Rostedt
2009-02-12  4:08     ` Frederic Weisbecker
2009-02-12 16:31   ` Steven Rostedt
2009-02-12 16:47     ` Frederic Weisbecker
2009-02-12 16:58       ` Steven Rostedt
2009-02-13  4:18   ` Benjamin Herrenschmidt
2009-02-12  2:23 ` Michael Ellerman
2009-02-12  2:37   ` Steven Rostedt
2009-02-12 16:35 ` Steven Rostedt
2009-02-12 23:32 ` Geoff Levand
2009-02-12 23:41   ` Steven Rostedt
2009-02-12 23:44     ` Josh Boyer
2009-02-12 23:44     ` Geoff Levand
2009-02-12 23:51       ` Steven Rostedt

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).