* kmemtrace.txt question: kernel parameter(s)?
@ 2009-12-17 22:13 Randy Dunlap
2009-12-18 0:57 ` Li Zefan
0 siblings, 1 reply; 8+ messages in thread
From: Randy Dunlap @ 2009-12-17 22:13 UTC (permalink / raw)
To: lkml; +Cc: eduard.munteanu, Pekka Enberg, Frederic Weisbecker
Hi,
Documentation/trace/kmemtrace.txt says:
Q: 'cat /sys/kernel/debug/kmemtrace/total_overruns' is non-zero, how do I fix
this? Should I worry?
A: If it's non-zero, this affects kmemtrace's accuracy, depending on how
large the number is. You can fix it by supplying a higher
'kmemtrace.subbufs=N' kernel parameter.
Where is this kernel parameter implemented and where is it documented?
or should this Answer just be fixed?
thanks,
---
~Randy
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: kmemtrace.txt question: kernel parameter(s)?
2009-12-17 22:13 kmemtrace.txt question: kernel parameter(s)? Randy Dunlap
@ 2009-12-18 0:57 ` Li Zefan
2009-12-24 0:00 ` Randy Dunlap
2010-03-27 4:27 ` Randy Dunlap
0 siblings, 2 replies; 8+ messages in thread
From: Li Zefan @ 2009-12-18 0:57 UTC (permalink / raw)
To: Randy Dunlap
Cc: lkml, eduard.munteanu, Pekka Enberg, Frederic Weisbecker, Ingo Molnar
Randy Dunlap wrote:
> Hi,
>
> Documentation/trace/kmemtrace.txt says:
>
> Q: 'cat /sys/kernel/debug/kmemtrace/total_overruns' is non-zero, how do I fix
> this? Should I worry?
> A: If it's non-zero, this affects kmemtrace's accuracy, depending on how
> large the number is. You can fix it by supplying a higher
> 'kmemtrace.subbufs=N' kernel parameter.
>
>
> Where is this kernel parameter implemented and where is it documented?
> or should this Answer just be fixed?
>
The whole documentation is out-dated. The original kmemtrace was based on
relay, and then it moved to ftrace, without updating the documentation.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: kmemtrace.txt question: kernel parameter(s)?
2009-12-18 0:57 ` Li Zefan
@ 2009-12-24 0:00 ` Randy Dunlap
2010-03-27 4:27 ` Randy Dunlap
1 sibling, 0 replies; 8+ messages in thread
From: Randy Dunlap @ 2009-12-24 0:00 UTC (permalink / raw)
To: Li Zefan
Cc: lkml, eduard.munteanu, Pekka Enberg, Frederic Weisbecker, Ingo Molnar
On Fri, 18 Dec 2009 08:57:49 +0800 Li Zefan wrote:
> Randy Dunlap wrote:
> > Hi,
> >
> > Documentation/trace/kmemtrace.txt says:
> >
> > Q: 'cat /sys/kernel/debug/kmemtrace/total_overruns' is non-zero, how do I fix
> > this? Should I worry?
> > A: If it's non-zero, this affects kmemtrace's accuracy, depending on how
> > large the number is. You can fix it by supplying a higher
> > 'kmemtrace.subbufs=N' kernel parameter.
> >
> >
> > Where is this kernel parameter implemented and where is it documented?
> > or should this Answer just be fixed?
> >
>
> The whole documentation is out-dated. The original kmemtrace was based on
> relay, and then it moved to ftrace, without updating the documentation.
Thank you. I guess that explains why kmemtraced (from kmemtrace-user.git)
fails with:
Could not open() file /sys/kernel/debug/kmemtrace/cpu0: No such file or directory
Eduard, can we get Documentation/trace/kmemtrace.txt updated, please?
---
~Randy
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: kmemtrace.txt question: kernel parameter(s)?
2009-12-18 0:57 ` Li Zefan
2009-12-24 0:00 ` Randy Dunlap
@ 2010-03-27 4:27 ` Randy Dunlap
2010-03-28 8:26 ` [PATCH] Remove Documentation/trace/kmemtrace.txt Eduard - Gabriel Munteanu
1 sibling, 1 reply; 8+ messages in thread
From: Randy Dunlap @ 2010-03-27 4:27 UTC (permalink / raw)
To: Li Zefan
Cc: lkml, eduard.munteanu, Pekka Enberg, Frederic Weisbecker, Ingo Molnar
On Fri, 18 Dec 2009 08:57:49 +0800 Li Zefan wrote:
> Randy Dunlap wrote:
> > Hi,
> >
> > Documentation/trace/kmemtrace.txt says:
> >
> > Q: 'cat /sys/kernel/debug/kmemtrace/total_overruns' is non-zero, how do I fix
> > this? Should I worry?
> > A: If it's non-zero, this affects kmemtrace's accuracy, depending on how
> > large the number is. You can fix it by supplying a higher
> > 'kmemtrace.subbufs=N' kernel parameter.
> >
> >
> > Where is this kernel parameter implemented and where is it documented?
> > or should this Answer just be fixed?
> >
>
> The whole documentation is out-dated. The original kmemtrace was based on
> relay, and then it moved to ftrace, without updating the documentation.
Sorry for the delay.
Shouldn't Documentation/trace/kmemtrace.txt just be completely deleted?
It's confusing and all relevant kmemtrace documentation is elsewhere.
---
~Randy
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH] Remove Documentation/trace/kmemtrace.txt
2010-03-27 4:27 ` Randy Dunlap
@ 2010-03-28 8:26 ` Eduard - Gabriel Munteanu
2010-03-28 8:33 ` Pekka Enberg
0 siblings, 1 reply; 8+ messages in thread
From: Eduard - Gabriel Munteanu @ 2010-03-28 8:26 UTC (permalink / raw)
To: mingo
Cc: randy.dunlap, lizf, fweisbec, penberg, linux-kernel,
Eduard - Gabriel Munteanu
The current implementation of kmemtrace has been based on ftrace for a
while. Documentation/trace/kmemtrace.txt concerns the old, relay-based
implementation, so it's outdated. This removes it to avoid confusion.
Signed-off-by: Eduard - Gabriel Munteanu <eduard.munteanu@linux360.ro>
---
Documentation/trace/kmemtrace.txt | 126 -------------------------------------
1 files changed, 0 insertions(+), 126 deletions(-)
delete mode 100644 Documentation/trace/kmemtrace.txt
diff --git a/Documentation/trace/kmemtrace.txt b/Documentation/trace/kmemtrace.txt
deleted file mode 100644
index 6308735..0000000
--- a/Documentation/trace/kmemtrace.txt
+++ /dev/null
@@ -1,126 +0,0 @@
- kmemtrace - Kernel Memory Tracer
-
- by Eduard - Gabriel Munteanu
- <eduard.munteanu@linux360.ro>
-
-I. Introduction
-===============
-
-kmemtrace helps kernel developers figure out two things:
-1) how different allocators (SLAB, SLUB etc.) perform
-2) how kernel code allocates memory and how much
-
-To do this, we trace every allocation and export information to the userspace
-through the relay interface. We export things such as the number of requested
-bytes, the number of bytes actually allocated (i.e. including internal
-fragmentation), whether this is a slab allocation or a plain kmalloc() and so
-on.
-
-The actual analysis is performed by a userspace tool (see section III for
-details on where to get it from). It logs the data exported by the kernel,
-processes it and (as of writing this) can provide the following information:
-- the total amount of memory allocated and fragmentation per call-site
-- the amount of memory allocated and fragmentation per allocation
-- total memory allocated and fragmentation in the collected dataset
-- number of cross-CPU allocation and frees (makes sense in NUMA environments)
-
-Moreover, it can potentially find inconsistent and erroneous behavior in
-kernel code, such as using slab free functions on kmalloc'ed memory or
-allocating less memory than requested (but not truly failed allocations).
-
-kmemtrace also makes provisions for tracing on some arch and analysing the
-data on another.
-
-II. Design and goals
-====================
-
-kmemtrace was designed to handle rather large amounts of data. Thus, it uses
-the relay interface to export whatever is logged to userspace, which then
-stores it. Analysis and reporting is done asynchronously, that is, after the
-data is collected and stored. By design, it allows one to log and analyse
-on different machines and different arches.
-
-As of writing this, the ABI is not considered stable, though it might not
-change much. However, no guarantees are made about compatibility yet. When
-deemed stable, the ABI should still allow easy extension while maintaining
-backward compatibility. This is described further in Documentation/ABI.
-
-Summary of design goals:
- - allow logging and analysis to be done across different machines
- - be fast and anticipate usage in high-load environments (*)
- - be reasonably extensible
- - make it possible for GNU/Linux distributions to have kmemtrace
- included in their repositories
-
-(*) - one of the reasons Pekka Enberg's original userspace data analysis
- tool's code was rewritten from Perl to C (although this is more than a
- simple conversion)
-
-
-III. Quick usage guide
-======================
-
-1) Get a kernel that supports kmemtrace and build it accordingly (i.e. enable
-CONFIG_KMEMTRACE).
-
-2) Get the userspace tool and build it:
-$ git clone git://repo.or.cz/kmemtrace-user.git # current repository
-$ cd kmemtrace-user/
-$ ./autogen.sh
-$ ./configure
-$ make
-
-3) Boot the kmemtrace-enabled kernel if you haven't, preferably in the
-'single' runlevel (so that relay buffers don't fill up easily), and run
-kmemtrace:
-# '$' does not mean user, but root here.
-$ mount -t debugfs none /sys/kernel/debug
-$ mount -t proc none /proc
-$ cd path/to/kmemtrace-user/
-$ ./kmemtraced
-Wait a bit, then stop it with CTRL+C.
-$ cat /sys/kernel/debug/kmemtrace/total_overruns # Check if we didn't
- # overrun, should
- # be zero.
-$ (Optionally) [Run kmemtrace_check separately on each cpu[0-9]*.out file to
- check its correctness]
-$ ./kmemtrace-report
-
-Now you should have a nice and short summary of how the allocator performs.
-
-IV. FAQ and known issues
-========================
-
-Q: 'cat /sys/kernel/debug/kmemtrace/total_overruns' is non-zero, how do I fix
-this? Should I worry?
-A: If it's non-zero, this affects kmemtrace's accuracy, depending on how
-large the number is. You can fix it by supplying a higher
-'kmemtrace.subbufs=N' kernel parameter.
----
-
-Q: kmemtrace_check reports errors, how do I fix this? Should I worry?
-A: This is a bug and should be reported. It can occur for a variety of
-reasons:
- - possible bugs in relay code
- - possible misuse of relay by kmemtrace
- - timestamps being collected unorderly
-Or you may fix it yourself and send us a patch.
----
-
-Q: kmemtrace_report shows many errors, how do I fix this? Should I worry?
-A: This is a known issue and I'm working on it. These might be true errors
-in kernel code, which may have inconsistent behavior (e.g. allocating memory
-with kmem_cache_alloc() and freeing it with kfree()). Pekka Enberg pointed
-out this behavior may work with SLAB, but may fail with other allocators.
-
-It may also be due to lack of tracing in some unusual allocator functions.
-
-We don't want bug reports regarding this issue yet.
----
-
-V. See also
-===========
-
-Documentation/kernel-parameters.txt
-Documentation/ABI/testing/debugfs-kmemtrace
-
--
1.6.4.4
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH] Remove Documentation/trace/kmemtrace.txt
2010-03-28 8:26 ` [PATCH] Remove Documentation/trace/kmemtrace.txt Eduard - Gabriel Munteanu
@ 2010-03-28 8:33 ` Pekka Enberg
0 siblings, 0 replies; 8+ messages in thread
From: Pekka Enberg @ 2010-03-28 8:33 UTC (permalink / raw)
To: Eduard - Gabriel Munteanu
Cc: mingo, randy.dunlap, lizf, fweisbec, linux-kernel
Eduard - Gabriel Munteanu wrote:
> The current implementation of kmemtrace has been based on ftrace for a
> while. Documentation/trace/kmemtrace.txt concerns the old, relay-based
> implementation, so it's outdated. This removes it to avoid confusion.
>
> Signed-off-by: Eduard - Gabriel Munteanu <eduard.munteanu@linux360.ro>
Acked-by: Pekka Enberg <penberg@cs.helsinki.fi>
> ---
> Documentation/trace/kmemtrace.txt | 126 -------------------------------------
> 1 files changed, 0 insertions(+), 126 deletions(-)
> delete mode 100644 Documentation/trace/kmemtrace.txt
>
> diff --git a/Documentation/trace/kmemtrace.txt b/Documentation/trace/kmemtrace.txt
> deleted file mode 100644
> index 6308735..0000000
> --- a/Documentation/trace/kmemtrace.txt
> +++ /dev/null
> @@ -1,126 +0,0 @@
> - kmemtrace - Kernel Memory Tracer
> -
> - by Eduard - Gabriel Munteanu
> - <eduard.munteanu@linux360.ro>
> -
> -I. Introduction
> -===============
> -
> -kmemtrace helps kernel developers figure out two things:
> -1) how different allocators (SLAB, SLUB etc.) perform
> -2) how kernel code allocates memory and how much
> -
> -To do this, we trace every allocation and export information to the userspace
> -through the relay interface. We export things such as the number of requested
> -bytes, the number of bytes actually allocated (i.e. including internal
> -fragmentation), whether this is a slab allocation or a plain kmalloc() and so
> -on.
> -
> -The actual analysis is performed by a userspace tool (see section III for
> -details on where to get it from). It logs the data exported by the kernel,
> -processes it and (as of writing this) can provide the following information:
> -- the total amount of memory allocated and fragmentation per call-site
> -- the amount of memory allocated and fragmentation per allocation
> -- total memory allocated and fragmentation in the collected dataset
> -- number of cross-CPU allocation and frees (makes sense in NUMA environments)
> -
> -Moreover, it can potentially find inconsistent and erroneous behavior in
> -kernel code, such as using slab free functions on kmalloc'ed memory or
> -allocating less memory than requested (but not truly failed allocations).
> -
> -kmemtrace also makes provisions for tracing on some arch and analysing the
> -data on another.
> -
> -II. Design and goals
> -====================
> -
> -kmemtrace was designed to handle rather large amounts of data. Thus, it uses
> -the relay interface to export whatever is logged to userspace, which then
> -stores it. Analysis and reporting is done asynchronously, that is, after the
> -data is collected and stored. By design, it allows one to log and analyse
> -on different machines and different arches.
> -
> -As of writing this, the ABI is not considered stable, though it might not
> -change much. However, no guarantees are made about compatibility yet. When
> -deemed stable, the ABI should still allow easy extension while maintaining
> -backward compatibility. This is described further in Documentation/ABI.
> -
> -Summary of design goals:
> - - allow logging and analysis to be done across different machines
> - - be fast and anticipate usage in high-load environments (*)
> - - be reasonably extensible
> - - make it possible for GNU/Linux distributions to have kmemtrace
> - included in their repositories
> -
> -(*) - one of the reasons Pekka Enberg's original userspace data analysis
> - tool's code was rewritten from Perl to C (although this is more than a
> - simple conversion)
> -
> -
> -III. Quick usage guide
> -======================
> -
> -1) Get a kernel that supports kmemtrace and build it accordingly (i.e. enable
> -CONFIG_KMEMTRACE).
> -
> -2) Get the userspace tool and build it:
> -$ git clone git://repo.or.cz/kmemtrace-user.git # current repository
> -$ cd kmemtrace-user/
> -$ ./autogen.sh
> -$ ./configure
> -$ make
> -
> -3) Boot the kmemtrace-enabled kernel if you haven't, preferably in the
> -'single' runlevel (so that relay buffers don't fill up easily), and run
> -kmemtrace:
> -# '$' does not mean user, but root here.
> -$ mount -t debugfs none /sys/kernel/debug
> -$ mount -t proc none /proc
> -$ cd path/to/kmemtrace-user/
> -$ ./kmemtraced
> -Wait a bit, then stop it with CTRL+C.
> -$ cat /sys/kernel/debug/kmemtrace/total_overruns # Check if we didn't
> - # overrun, should
> - # be zero.
> -$ (Optionally) [Run kmemtrace_check separately on each cpu[0-9]*.out file to
> - check its correctness]
> -$ ./kmemtrace-report
> -
> -Now you should have a nice and short summary of how the allocator performs.
> -
> -IV. FAQ and known issues
> -========================
> -
> -Q: 'cat /sys/kernel/debug/kmemtrace/total_overruns' is non-zero, how do I fix
> -this? Should I worry?
> -A: If it's non-zero, this affects kmemtrace's accuracy, depending on how
> -large the number is. You can fix it by supplying a higher
> -'kmemtrace.subbufs=N' kernel parameter.
> ----
> -
> -Q: kmemtrace_check reports errors, how do I fix this? Should I worry?
> -A: This is a bug and should be reported. It can occur for a variety of
> -reasons:
> - - possible bugs in relay code
> - - possible misuse of relay by kmemtrace
> - - timestamps being collected unorderly
> -Or you may fix it yourself and send us a patch.
> ----
> -
> -Q: kmemtrace_report shows many errors, how do I fix this? Should I worry?
> -A: This is a known issue and I'm working on it. These might be true errors
> -in kernel code, which may have inconsistent behavior (e.g. allocating memory
> -with kmem_cache_alloc() and freeing it with kfree()). Pekka Enberg pointed
> -out this behavior may work with SLAB, but may fail with other allocators.
> -
> -It may also be due to lack of tracing in some unusual allocator functions.
> -
> -We don't want bug reports regarding this issue yet.
> ----
> -
> -V. See also
> -===========
> -
> -Documentation/kernel-parameters.txt
> -Documentation/ABI/testing/debugfs-kmemtrace
> -
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] Remove Documentation/trace/kmemtrace.txt
2010-03-28 16:09 Randy Dunlap
@ 2010-03-28 18:56 ` Frederic Weisbecker
0 siblings, 0 replies; 8+ messages in thread
From: Frederic Weisbecker @ 2010-03-28 18:56 UTC (permalink / raw)
To: Randy Dunlap; +Cc: eduard.munteanu, mingo, linux-kernel, lizf, penberg
On Sun, Mar 28, 2010 at 09:09:17AM -0700, Randy Dunlap wrote:
> Acked-by: Randy Dunlap <randy.dunlap@oracle.com>
>
> Thanks.
Joining the party:
Acked-by: Frederic Weisbecker <fweisbec@gmail.com>
Btw, how far are we from removing the ftrace plugin, now
that we have the allocator trace events?
Thanks.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] Remove Documentation/trace/kmemtrace.txt
@ 2010-03-28 16:09 Randy Dunlap
2010-03-28 18:56 ` Frederic Weisbecker
0 siblings, 1 reply; 8+ messages in thread
From: Randy Dunlap @ 2010-03-28 16:09 UTC (permalink / raw)
To: eduard.munteanu; +Cc: fweisbec, mingo, linux-kernel, lizf, penberg
Acked-by: Randy Dunlap <randy.dunlap@oracle.com>
Thanks.
---
~Randy
----- Original Message -----
From: eduard.munteanu@linux360.ro
To: mingo@elte.hu
Cc: randy.dunlap@oracle.com, lizf@cn.fujitsu.com, fweisbec@gmail.com, penberg@cs.helsinki.fi, linux-kernel@vger.kernel.org, eduard.munteanu@linux360.ro
Sent: Sunday, March 28, 2010 1:27:31 AM GMT -08:00 US/Canada Pacific
Subject: [PATCH] Remove Documentation/trace/kmemtrace.txt
The current implementation of kmemtrace has been based on ftrace for a
while. Documentation/trace/kmemtrace.txt concerns the old, relay-based
implementation, so it's outdated. This removes it to avoid confusion.
Signed-off-by: Eduard - Gabriel Munteanu <eduard.munteanu@linux360.ro>
---
Documentation/trace/kmemtrace.txt | 126 -------------------------------------
1 files changed, 0 insertions(+), 126 deletions(-)
delete mode 100644 Documentation/trace/kmemtrace.txt
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2010-03-28 18:56 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-12-17 22:13 kmemtrace.txt question: kernel parameter(s)? Randy Dunlap
2009-12-18 0:57 ` Li Zefan
2009-12-24 0:00 ` Randy Dunlap
2010-03-27 4:27 ` Randy Dunlap
2010-03-28 8:26 ` [PATCH] Remove Documentation/trace/kmemtrace.txt Eduard - Gabriel Munteanu
2010-03-28 8:33 ` Pekka Enberg
2010-03-28 16:09 Randy Dunlap
2010-03-28 18:56 ` Frederic Weisbecker
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).