[PATCH v2 0/3] perf report: Recognize hugetlb mapping as anon mapping

[PATCH v2 0/3] perf report: Recognize hugetlb mapping as anon mapping

[Wang Nan]

The requirement of this function is first proposed at 2015.
Please refer to

http://lkml.iu.edu/hypermail/linux/kernel/1506.2/02372.html
http://lkml.iu.edu/hypermail/linux/kernel/1506.3/02290.html
http://lkml.iu.edu/hypermail/linux/kernel/1506.3/03512.html

For systems which use hugetlbfs, if a sample is captured inside
hugetlbfs, perf should not resolve symbols from the file shown in
/prof/<pid>/mmap, because 'files' in hugetlbfs are constructed
during execution and not ELF files. If user knows positions of
symbols, he/she should provide /tmp/perf-<pid>.map file.

This 3 patches makes perf recognize hugetlbfs mapping as anon mapping.
Before this 3 patches user has no chance to use his/her own .map file
to resolve symbols because perf tries to use hugetlbfs file.

v1 -> v2: Fix building when MAP_HUGETLB is missing.
          Note that simply define a MAP_HUGETLB not work because
	  its value depend on arch. This patch v2 disable this
	  'hugetlbfs as as anon mapping' feature if it is built
	  in very old system.

Wang Nan (3):
  perf tools: Recognize hugetlb mapping as anon mapping
  tools lib api fs: Add hugetlbfs filesystem detector
  perf record: Mark MAP_HUGETLB during synthesizing mmap events

 tools/lib/api/fs/fs.c   | 15 +++++++++++++++
 tools/lib/api/fs/fs.h   |  1 +
 tools/perf/util/event.c | 11 +++++++++++
 tools/perf/util/map.c   | 13 ++++++++++---
 4 files changed, 37 insertions(+), 3 deletions(-)

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Hou Pengyang <houpengyang@huawei.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Nilay Vaish <nilayvaish@gmail.com>

-- 
1.8.3.4

[PATCH v2 1/3] perf tools: Recognize hugetlb mapping as anon mapping

[Wang Nan]

Hugetlbfs mapping should be recognized as anon mapping so user has
a chance to create /tmp/perf-<pid>.map file for symbol resolving. This
patch utilizes MAP_HUGETLB to identify hugetlb mapping.

After this patch, if perf is started before the program starts using
huge pages (so perf gets MMAP2 events from kernel), perf is able to
recognize hugetlb mapping as anon mapping.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: Hou Pengyang <houpengyang@huawei.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Nilay Vaish <nilayvaish@gmail.com>
---
 tools/perf/util/map.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/tools/perf/util/map.c b/tools/perf/util/map.c
index 728129a..f52d460 100644
--- a/tools/perf/util/map.c
+++ b/tools/perf/util/map.c
@@ -6,6 +6,7 @@
 #include <string.h>
 #include <stdio.h>
 #include <unistd.h>
+#include <sys/mman.h>
 #include "map.h"
 #include "thread.h"
 #include "strlist.h"
@@ -24,9 +25,15 @@ const char *map_type__name[MAP__NR_TYPES] = {
 	[MAP__VARIABLE] = "Variables",
 };
 
-static inline int is_anon_memory(const char *filename)
+static inline int is_anon_memory(const char *filename, u32 flags)
 {
-	return !strcmp(filename, "//anon") ||
+	u32 anon_flags = 0;
+
+#ifdef MAP_HUGETLB
+	anon_flags |= MAP_HUGETLB;
+#endif
+	return flags & anon_flags ||
+	       !strcmp(filename, "//anon") ||
 	       !strncmp(filename, "/dev/zero", sizeof("/dev/zero") - 1) ||
 	       !strncmp(filename, "/anon_hugepage", sizeof("/anon_hugepage") - 1);
 }
@@ -155,7 +162,7 @@ struct map *map__new(struct machine *machine, u64 start, u64 len,
 		int anon, no_dso, vdso, android;
 
 		android = is_android_lib(filename);
-		anon = is_anon_memory(filename);
+		anon = is_anon_memory(filename, flags);
 		vdso = is_vdso_map(filename);
 		no_dso = is_no_dso_memory(filename);
 
-- 
1.8.3.4

[PATCH v2 2/3] tools lib api fs: Add hugetlbfs filesystem detector

[Wang Nan]

Detect hugetlbfs. hugetlbfs__mountpoint() will be used during recording
to help recorder identifying hugetlb mmaps: which should be recognized
as anon mapping.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Reviewed-by: Nilay Vaish <nilayvaish@gmail.com>
Cc: Hou Pengyang <houpengyang@huawei.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/lib/api/fs/fs.c | 15 +++++++++++++++
 tools/lib/api/fs/fs.h |  1 +
 2 files changed, 16 insertions(+)

diff --git a/tools/lib/api/fs/fs.c b/tools/lib/api/fs/fs.c
index ba7094b..f99f49e4 100644
--- a/tools/lib/api/fs/fs.c
+++ b/tools/lib/api/fs/fs.c
@@ -34,6 +34,10 @@
 #define TRACEFS_MAGIC          0x74726163
 #endif
 
+#ifndef HUGETLBFS_MAGIC
+#define HUGETLBFS_MAGIC        0x958458f6
+#endif
+
 static const char * const sysfs__fs_known_mountpoints[] = {
 	"/sys",
 	0,
@@ -67,6 +71,10 @@ static const char * const tracefs__known_mountpoints[] = {
 	0,
 };
 
+static const char * const hugetlbfs__known_mountpoints[] = {
+	0,
+};
+
 struct fs {
 	const char		*name;
 	const char * const	*mounts;
@@ -80,6 +88,7 @@ enum {
 	FS__PROCFS  = 1,
 	FS__DEBUGFS = 2,
 	FS__TRACEFS = 3,
+	FS__HUGETLBFS = 4,
 };
 
 #ifndef TRACEFS_MAGIC
@@ -107,6 +116,11 @@ static struct fs fs__entries[] = {
 		.mounts	= tracefs__known_mountpoints,
 		.magic	= TRACEFS_MAGIC,
 	},
+	[FS__HUGETLBFS] = {
+		.name	= "hugetlbfs",
+		.mounts = hugetlbfs__known_mountpoints,
+		.magic	= HUGETLBFS_MAGIC,
+	},
 };
 
 static bool fs__read_mounts(struct fs *fs)
@@ -265,6 +279,7 @@ FS(sysfs,   FS__SYSFS);
 FS(procfs,  FS__PROCFS);
 FS(debugfs, FS__DEBUGFS);
 FS(tracefs, FS__TRACEFS);
+FS(hugetlbfs, FS__HUGETLBFS);
 
 int filename__read_int(const char *filename, int *value)
 {
diff --git a/tools/lib/api/fs/fs.h b/tools/lib/api/fs/fs.h
index 16c9c2e..a63269f 100644
--- a/tools/lib/api/fs/fs.h
+++ b/tools/lib/api/fs/fs.h
@@ -21,6 +21,7 @@ FS(sysfs)
 FS(procfs)
 FS(debugfs)
 FS(tracefs)
+FS(hugetlbfs)
 
 #undef FS
 
-- 
1.8.3.4

[PATCH v2 3/3] perf record: Mark MAP_HUGETLB during synthesizing mmap events

[Wang Nan]

During synthesizing mmap events, add MAP_HUGETLB map flag if the
source of mapping is file in hugetlbfs.

After this patch, perf can identify hugetlb mapping even if perf
is started after the mapping of huge pages (like perf top).

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Reviewed-by: Nilay Vaish <nilayvaish@gmail.com>
Cc: Hou Pengyang <houpengyang@huawei.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/event.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/tools/perf/util/event.c b/tools/perf/util/event.c
index e20438b..23c3ebd 100644
--- a/tools/perf/util/event.c
+++ b/tools/perf/util/event.c
@@ -1,5 +1,6 @@
 #include <linux/types.h>
 #include <sys/mman.h>
+#include <api/fs/fs.h>
 #include "event.h"
 #include "debug.h"
 #include "hist.h"
@@ -248,6 +249,10 @@ int perf_event__synthesize_mmap_events(struct perf_tool *tool,
 	bool truncation = false;
 	unsigned long long timeout = proc_map_timeout * 1000000ULL;
 	int rc = 0;
+#ifdef MAP_HUGETLB
+	const char *hugetlbfs_mnt = hugetlbfs__mountpoint();
+	int hugetlbfs_mnt_len = hugetlbfs_mnt ? strlen(hugetlbfs_mnt) : 0;
+#endif
 
 	if (machine__is_default_guest(machine))
 		return 0;
@@ -342,6 +347,12 @@ out:
 
 		if (!strcmp(execname, ""))
 			strcpy(execname, anonstr);
+#ifdef MAP_HUGETLB
+		if (!strncmp(execname, hugetlbfs_mnt, hugetlbfs_mnt_len)) {
+			strcpy(execname, anonstr);
+			event->mmap2.flags |= MAP_HUGETLB;
+		}
+#endif
 
 		size = strlen(execname) + 1;
 		memcpy(event->mmap2.filename, execname, size);
-- 
1.8.3.4

[tip:perf/core] perf tools: Recognize hugetlb mapping as anon mapping

[tip-bot for Wang Nan]

Commit-ID:  0ac3348e502423cf2fe86beca83b8835a2e6d289
Gitweb:     http://git.kernel.org/tip/0ac3348e502423cf2fe86beca83b8835a2e6d289
Author:     Wang Nan <wangnan0@huawei.com>
AuthorDate: Tue, 6 Sep 2016 04:58:27 +0000
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Thu, 8 Sep 2016 12:28:06 -0300

perf tools: Recognize hugetlb mapping as anon mapping

Hugetlbfs mapping should be recognized as anon mapping so user has a
chance to create /tmp/perf-<pid>.map file for symbol resolving. This
patch utilizes MAP_HUGETLB to identify hugetlb mapping.

After this patch, if perf is started before a program starts using huge
pages (so perf gets MMAP2 events from kernel), perf is able to recognize
hugetlb mapping as anon mapping.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Nilay Vaish <nilayvaish@gmail.com>
Cc: Zefan Li <lizefan@huawei.com>
Link: http://lkml.kernel.org/r/1473137909-142064-2-git-send-email-wangnan0@huawei.com
Signed-off-by: Hou Pengyang <houpengyang@huawei.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/map.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/tools/perf/util/map.c b/tools/perf/util/map.c
index 0c54adb..d51a125 100644
--- a/tools/perf/util/map.c
+++ b/tools/perf/util/map.c
@@ -6,6 +6,7 @@
 #include <string.h>
 #include <stdio.h>
 #include <unistd.h>
+#include <sys/mman.h>
 #include "map.h"
 #include "thread.h"
 #include "strlist.h"
@@ -24,9 +25,15 @@ const char *map_type__name[MAP__NR_TYPES] = {
 	[MAP__VARIABLE] = "Variables",
 };
 
-static inline int is_anon_memory(const char *filename)
+static inline int is_anon_memory(const char *filename, u32 flags)
 {
-	return !strcmp(filename, "//anon") ||
+	u32 anon_flags = 0;
+
+#ifdef MAP_HUGETLB
+	anon_flags |= MAP_HUGETLB;
+#endif
+	return flags & anon_flags ||
+	       !strcmp(filename, "//anon") ||
 	       !strncmp(filename, "/dev/zero", sizeof("/dev/zero") - 1) ||
 	       !strncmp(filename, "/anon_hugepage", sizeof("/anon_hugepage") - 1);
 }
@@ -155,7 +162,7 @@ struct map *map__new(struct machine *machine, u64 start, u64 len,
 		int anon, no_dso, vdso, android;
 
 		android = is_android_lib(filename);
-		anon = is_anon_memory(filename);
+		anon = is_anon_memory(filename, flags);
 		vdso = is_vdso_map(filename);
 		no_dso = is_no_dso_memory(filename);
 

[tip:perf/core] tools lib api fs: Add hugetlbfs filesystem detector

[tip-bot for Wang Nan]

Commit-ID:  5e7be3e1f9d71ed03fac27df25e143815be662d2
Gitweb:     http://git.kernel.org/tip/5e7be3e1f9d71ed03fac27df25e143815be662d2
Author:     Wang Nan <wangnan0@huawei.com>
AuthorDate: Tue, 6 Sep 2016 04:58:28 +0000
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Thu, 8 Sep 2016 12:34:43 -0300

tools lib api fs: Add hugetlbfs filesystem detector

Detect hugetlbfs. hugetlbfs__mountpoint() will be used during recording
to help identifying hugetlb mmaps: which should be recognized as anon
mapping.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Reviewed-by: Nilay Vaish <nilayvaish@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Hou Pengyang <houpengyang@huawei.com>
Cc: Zefan Li <lizefan@huawei.com>
Link: http://lkml.kernel.org/r/1473137909-142064-3-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/lib/api/fs/fs.c | 15 +++++++++++++++
 tools/lib/api/fs/fs.h |  1 +
 2 files changed, 16 insertions(+)

diff --git a/tools/lib/api/fs/fs.c b/tools/lib/api/fs/fs.c
index ba7094b..f99f49e4 100644
--- a/tools/lib/api/fs/fs.c
+++ b/tools/lib/api/fs/fs.c
@@ -34,6 +34,10 @@
 #define TRACEFS_MAGIC          0x74726163
 #endif
 
+#ifndef HUGETLBFS_MAGIC
+#define HUGETLBFS_MAGIC        0x958458f6
+#endif
+
 static const char * const sysfs__fs_known_mountpoints[] = {
 	"/sys",
 	0,
@@ -67,6 +71,10 @@ static const char * const tracefs__known_mountpoints[] = {
 	0,
 };
 
+static const char * const hugetlbfs__known_mountpoints[] = {
+	0,
+};
+
 struct fs {
 	const char		*name;
 	const char * const	*mounts;
@@ -80,6 +88,7 @@ enum {
 	FS__PROCFS  = 1,
 	FS__DEBUGFS = 2,
 	FS__TRACEFS = 3,
+	FS__HUGETLBFS = 4,
 };
 
 #ifndef TRACEFS_MAGIC
@@ -107,6 +116,11 @@ static struct fs fs__entries[] = {
 		.mounts	= tracefs__known_mountpoints,
 		.magic	= TRACEFS_MAGIC,
 	},
+	[FS__HUGETLBFS] = {
+		.name	= "hugetlbfs",
+		.mounts = hugetlbfs__known_mountpoints,
+		.magic	= HUGETLBFS_MAGIC,
+	},
 };
 
 static bool fs__read_mounts(struct fs *fs)
@@ -265,6 +279,7 @@ FS(sysfs,   FS__SYSFS);
 FS(procfs,  FS__PROCFS);
 FS(debugfs, FS__DEBUGFS);
 FS(tracefs, FS__TRACEFS);
+FS(hugetlbfs, FS__HUGETLBFS);
 
 int filename__read_int(const char *filename, int *value)
 {
diff --git a/tools/lib/api/fs/fs.h b/tools/lib/api/fs/fs.h
index 16c9c2e..a63269f 100644
--- a/tools/lib/api/fs/fs.h
+++ b/tools/lib/api/fs/fs.h
@@ -21,6 +21,7 @@ FS(sysfs)
 FS(procfs)
 FS(debugfs)
 FS(tracefs)
+FS(hugetlbfs)
 
 #undef FS
 

[tip:perf/core] perf record: Mark MAP_HUGETLB when synthesizing mmap events

[tip-bot for Wang Nan]

Commit-ID:  d7e404af115bb4996afa4a0236020969ab007554
Gitweb:     http://git.kernel.org/tip/d7e404af115bb4996afa4a0236020969ab007554
Author:     Wang Nan <wangnan0@huawei.com>
AuthorDate: Tue, 6 Sep 2016 04:58:29 +0000
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Thu, 8 Sep 2016 12:36:01 -0300

perf record: Mark MAP_HUGETLB when synthesizing mmap events

When synthesizing mmap events, add MAP_HUGETLB map flag if the source of
mapping is file in hugetlbfs.

After this patch, perf can identify hugetlb mapping even if perf is
started after the mapping of huge pages (like with 'perf top').

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Reviewed-by: Nilay Vaish <nilayvaish@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Hou Pengyang <houpengyang@huawei.com>
Cc: Zefan Li <lizefan@huawei.com>
Link: http://lkml.kernel.org/r/1473137909-142064-4-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/event.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/tools/perf/util/event.c b/tools/perf/util/event.c
index 9ad7d32..6c30171 100644
--- a/tools/perf/util/event.c
+++ b/tools/perf/util/event.c
@@ -1,5 +1,6 @@
 #include <linux/types.h>
 #include <sys/mman.h>
+#include <api/fs/fs.h>
 #include "event.h"
 #include "debug.h"
 #include "hist.h"
@@ -248,6 +249,10 @@ int perf_event__synthesize_mmap_events(struct perf_tool *tool,
 	bool truncation = false;
 	unsigned long long timeout = proc_map_timeout * 1000000ULL;
 	int rc = 0;
+#ifdef MAP_HUGETLB
+	const char *hugetlbfs_mnt = hugetlbfs__mountpoint();
+	int hugetlbfs_mnt_len = hugetlbfs_mnt ? strlen(hugetlbfs_mnt) : 0;
+#endif
 
 	if (machine__is_default_guest(machine))
 		return 0;
@@ -342,6 +347,12 @@ out:
 
 		if (!strcmp(execname, ""))
 			strcpy(execname, anonstr);
+#ifdef MAP_HUGETLB
+		if (!strncmp(execname, hugetlbfs_mnt, hugetlbfs_mnt_len)) {
+			strcpy(execname, anonstr);
+			event->mmap2.flags |= MAP_HUGETLB;
+		}
+#endif
 
 		size = strlen(execname) + 1;
 		memcpy(event->mmap2.filename, execname, size);