[PATCH] perf machine: Search for modules in %s/lib/modules/%s

[PATCH] perf machine: Search for modules in %s/lib/modules/%s

[Richard Yao]

Modules installed outside of the kernel's build system should go into
"%s/lib/modules/%s/extra", but at present, perf will only look at them
when they are in "%s/lib/modules/%s/kernel". Lets encourage good
citizenship by relaxing this requirement to "%s/lib/modules/%s". This
way open source modules that are out-of-tree have no incentive to start
populating a directory reserved for in-kernel modules and I can stop
hex-editing my system's perf binary when profiling OSS out-of-tree
modules.

Feedback from Namhyung Kim correctly revealed that the hex-edits that I
had been doing meant that perf was also traversing the build and source
symlinks in %s/lib/modules/%s. That is undesireable, so we explicitly
exclude them from traversal with a minor tweak to the traversal routine.

Signed-off-by: Richard Yao <ryao@gentoo.org>
---
 tools/perf/util/machine.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
index a53cd0b..27c2a5e 100644
--- a/tools/perf/util/machine.c
+++ b/tools/perf/util/machine.c
@@ -717,7 +717,7 @@ static char *get_kernel_version(const char *root_dir)
 }
 
 static int map_groups__set_modules_path_dir(struct map_groups *mg,
-				const char *dir_name)
+				const char *dir_name, int depth)
 {
 	struct dirent *dent;
 	DIR *dir = opendir(dir_name);
@@ -742,7 +742,15 @@ static int map_groups__set_modules_path_dir(struct map_groups *mg,
 			    !strcmp(dent->d_name, ".."))
 				continue;
 
-			ret = map_groups__set_modules_path_dir(mg, path);
+			/* Do not follow top-level source and build symlinks */
+			if (depth == 0) {
+				if (!strcmp(dent->d_name, "source") ||
+				    !strcmp(dent->d_name, "build"))
+					continue;
+			}
+
+			ret = map_groups__set_modules_path_dir(mg, path,
+							       depth + 1);
 			if (ret < 0)
 				goto out;
 		} else {
@@ -786,11 +794,11 @@ static int machine__set_modules_path(struct machine *machine)
 	if (!version)
 		return -1;
 
-	snprintf(modules_path, sizeof(modules_path), "%s/lib/modules/%s/kernel",
+	snprintf(modules_path, sizeof(modules_path), "%s/lib/modules/%s",
 		 machine->root_dir, version);
 	free(version);
 
-	return map_groups__set_modules_path_dir(&machine->kmaps, modules_path);
+	return map_groups__set_modules_path_dir(&machine->kmaps, modules_path, 0);
 }
 
 static int machine__create_module(void *arg, const char *name, u64 start)
-- 
1.8.3.2

Re: [PATCH] perf machine: Search for modules in %s/lib/modules/%s

[Jiri Olsa]

On Sat, Apr 26, 2014 at 01:17:55PM -0400, Richard Yao wrote:
> Modules installed outside of the kernel's build system should go into
> "%s/lib/modules/%s/extra", but at present, perf will only look at them
> when they are in "%s/lib/modules/%s/kernel". Lets encourage good
> citizenship by relaxing this requirement to "%s/lib/modules/%s". This
> way open source modules that are out-of-tree have no incentive to start
> populating a directory reserved for in-kernel modules and I can stop
> hex-editing my system's perf binary when profiling OSS out-of-tree
> modules.
> 
> Feedback from Namhyung Kim correctly revealed that the hex-edits that I
> had been doing meant that perf was also traversing the build and source
> symlinks in %s/lib/modules/%s. That is undesireable, so we explicitly
> exclude them from traversal with a minor tweak to the traversal routine.

looks good to me, Namhyung?

thanks,
jirka

> 
> Signed-off-by: Richard Yao <ryao@gentoo.org>
> ---
>  tools/perf/util/machine.c | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
> 
> diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
> index a53cd0b..27c2a5e 100644
> --- a/tools/perf/util/machine.c
> +++ b/tools/perf/util/machine.c
> @@ -717,7 +717,7 @@ static char *get_kernel_version(const char *root_dir)
>  }
>  
>  static int map_groups__set_modules_path_dir(struct map_groups *mg,
> -				const char *dir_name)
> +				const char *dir_name, int depth)
>  {
>  	struct dirent *dent;
>  	DIR *dir = opendir(dir_name);
> @@ -742,7 +742,15 @@ static int map_groups__set_modules_path_dir(struct map_groups *mg,
>  			    !strcmp(dent->d_name, ".."))
>  				continue;
>  
> -			ret = map_groups__set_modules_path_dir(mg, path);
> +			/* Do not follow top-level source and build symlinks */
> +			if (depth == 0) {
> +				if (!strcmp(dent->d_name, "source") ||
> +				    !strcmp(dent->d_name, "build"))
> +					continue;
> +			}
> +
> +			ret = map_groups__set_modules_path_dir(mg, path,
> +							       depth + 1);
>  			if (ret < 0)
>  				goto out;
>  		} else {
> @@ -786,11 +794,11 @@ static int machine__set_modules_path(struct machine *machine)
>  	if (!version)
>  		return -1;
>  
> -	snprintf(modules_path, sizeof(modules_path), "%s/lib/modules/%s/kernel",
> +	snprintf(modules_path, sizeof(modules_path), "%s/lib/modules/%s",
>  		 machine->root_dir, version);
>  	free(version);
>  
> -	return map_groups__set_modules_path_dir(&machine->kmaps, modules_path);
> +	return map_groups__set_modules_path_dir(&machine->kmaps, modules_path, 0);
>  }
>  
>  static int machine__create_module(void *arg, const char *name, u64 start)
> -- 
> 1.8.3.2
> 

Re: [PATCH] perf machine: Search for modules in %s/lib/modules/%s

[Jiri Olsa]

On Sun, Apr 27, 2014 at 12:05:56PM +0200, Jiri Olsa wrote:
> On Sat, Apr 26, 2014 at 01:17:55PM -0400, Richard Yao wrote:
> > Modules installed outside of the kernel's build system should go into
> > "%s/lib/modules/%s/extra", but at present, perf will only look at them
> > when they are in "%s/lib/modules/%s/kernel". Lets encourage good
> > citizenship by relaxing this requirement to "%s/lib/modules/%s". This
> > way open source modules that are out-of-tree have no incentive to start
> > populating a directory reserved for in-kernel modules and I can stop
> > hex-editing my system's perf binary when profiling OSS out-of-tree
> > modules.
> > 
> > Feedback from Namhyung Kim correctly revealed that the hex-edits that I
> > had been doing meant that perf was also traversing the build and source
> > symlinks in %s/lib/modules/%s. That is undesireable, so we explicitly
> > exclude them from traversal with a minor tweak to the traversal routine.
> 
> looks good to me, Namhyung?

just realized Namhyung wasn't CC-ed on this one ;-)

jirka

> 
> thanks,
> jirka
> 
> > 
> > Signed-off-by: Richard Yao <ryao@gentoo.org>
> > ---
> >  tools/perf/util/machine.c | 16 ++++++++++++----
> >  1 file changed, 12 insertions(+), 4 deletions(-)
> > 
> > diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
> > index a53cd0b..27c2a5e 100644
> > --- a/tools/perf/util/machine.c
> > +++ b/tools/perf/util/machine.c
> > @@ -717,7 +717,7 @@ static char *get_kernel_version(const char *root_dir)
> >  }
> >  
> >  static int map_groups__set_modules_path_dir(struct map_groups *mg,
> > -				const char *dir_name)
> > +				const char *dir_name, int depth)
> >  {
> >  	struct dirent *dent;
> >  	DIR *dir = opendir(dir_name);
> > @@ -742,7 +742,15 @@ static int map_groups__set_modules_path_dir(struct map_groups *mg,
> >  			    !strcmp(dent->d_name, ".."))
> >  				continue;
> >  
> > -			ret = map_groups__set_modules_path_dir(mg, path);
> > +			/* Do not follow top-level source and build symlinks */
> > +			if (depth == 0) {
> > +				if (!strcmp(dent->d_name, "source") ||
> > +				    !strcmp(dent->d_name, "build"))
> > +					continue;
> > +			}
> > +
> > +			ret = map_groups__set_modules_path_dir(mg, path,
> > +							       depth + 1);
> >  			if (ret < 0)
> >  				goto out;
> >  		} else {
> > @@ -786,11 +794,11 @@ static int machine__set_modules_path(struct machine *machine)
> >  	if (!version)
> >  		return -1;
> >  
> > -	snprintf(modules_path, sizeof(modules_path), "%s/lib/modules/%s/kernel",
> > +	snprintf(modules_path, sizeof(modules_path), "%s/lib/modules/%s",
> >  		 machine->root_dir, version);
> >  	free(version);
> >  
> > -	return map_groups__set_modules_path_dir(&machine->kmaps, modules_path);
> > +	return map_groups__set_modules_path_dir(&machine->kmaps, modules_path, 0);
> >  }
> >  
> >  static int machine__create_module(void *arg, const char *name, u64 start)
> > -- 
> > 1.8.3.2
> > 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] perf machine: Search for modules in %s/lib/modules/%s

[Namhyung Kim]

Hi Jiri,

On Tue, 29 Apr 2014 10:06:17 +0200, Jiri Olsa wrote:
> On Sun, Apr 27, 2014 at 12:05:56PM +0200, Jiri Olsa wrote:
>> On Sat, Apr 26, 2014 at 01:17:55PM -0400, Richard Yao wrote:
>> > Modules installed outside of the kernel's build system should go into
>> > "%s/lib/modules/%s/extra", but at present, perf will only look at them
>> > when they are in "%s/lib/modules/%s/kernel". Lets encourage good
>> > citizenship by relaxing this requirement to "%s/lib/modules/%s". This
>> > way open source modules that are out-of-tree have no incentive to start
>> > populating a directory reserved for in-kernel modules and I can stop
>> > hex-editing my system's perf binary when profiling OSS out-of-tree
>> > modules.
>> > 
>> > Feedback from Namhyung Kim correctly revealed that the hex-edits that I
>> > had been doing meant that perf was also traversing the build and source
>> > symlinks in %s/lib/modules/%s. That is undesireable, so we explicitly
>> > exclude them from traversal with a minor tweak to the traversal routine.
>> 
>> looks good to me, Namhyung?
>
> just realized Namhyung wasn't CC-ed on this one ;-)

:)

Well, I still prefer the symlink check with lstat(), I can live with
this, so:

  Acked-by: Namhyung kim <namhyung@kernel.org>

Thanks,
Namhyung

>
>> 
>> thanks,
>> jirka
>> 
>> > 
>> > Signed-off-by: Richard Yao <ryao@gentoo.org>
>> > ---
>> >  tools/perf/util/machine.c | 16 ++++++++++++----
>> >  1 file changed, 12 insertions(+), 4 deletions(-)
>> > 
>> > diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
>> > index a53cd0b..27c2a5e 100644
>> > --- a/tools/perf/util/machine.c
>> > +++ b/tools/perf/util/machine.c
>> > @@ -717,7 +717,7 @@ static char *get_kernel_version(const char *root_dir)
>> >  }
>> >  
>> >  static int map_groups__set_modules_path_dir(struct map_groups *mg,
>> > -				const char *dir_name)
>> > +				const char *dir_name, int depth)
>> >  {
>> >  	struct dirent *dent;
>> >  	DIR *dir = opendir(dir_name);
>> > @@ -742,7 +742,15 @@ static int map_groups__set_modules_path_dir(struct map_groups *mg,
>> >  			    !strcmp(dent->d_name, ".."))
>> >  				continue;
>> >  
>> > -			ret = map_groups__set_modules_path_dir(mg, path);
>> > +			/* Do not follow top-level source and build symlinks */
>> > +			if (depth == 0) {
>> > +				if (!strcmp(dent->d_name, "source") ||
>> > +				    !strcmp(dent->d_name, "build"))
>> > +					continue;
>> > +			}
>> > +
>> > +			ret = map_groups__set_modules_path_dir(mg, path,
>> > +							       depth + 1);
>> >  			if (ret < 0)
>> >  				goto out;
>> >  		} else {
>> > @@ -786,11 +794,11 @@ static int machine__set_modules_path(struct machine *machine)
>> >  	if (!version)
>> >  		return -1;
>> >  
>> > -	snprintf(modules_path, sizeof(modules_path), "%s/lib/modules/%s/kernel",
>> > +	snprintf(modules_path, sizeof(modules_path), "%s/lib/modules/%s",
>> >  		 machine->root_dir, version);
>> >  	free(version);
>> >  
>> > -	return map_groups__set_modules_path_dir(&machine->kmaps, modules_path);
>> > +	return map_groups__set_modules_path_dir(&machine->kmaps, modules_path, 0);
>> >  }
>> >  
>> >  static int machine__create_module(void *arg, const char *name, u64 start)
>> > -- 
>> > 1.8.3.2
>> > 
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at  http://www.tux.org/lkml/

[tip:perf/core] perf machine: Search for modules in %s/lib/ modules/%s

[tip-bot for Richard Yao]

Commit-ID:  61d4290cc1f10588147b76b385875f06827d47ff
Gitweb:     http://git.kernel.org/tip/61d4290cc1f10588147b76b385875f06827d47ff
Author:     Richard Yao <ryao@gentoo.org>
AuthorDate: Sat, 26 Apr 2014 13:17:55 -0400
Committer:  Jiri Olsa <jolsa@kernel.org>
CommitDate: Wed, 30 Apr 2014 16:49:29 +0200

perf machine: Search for modules in %s/lib/modules/%s

Modules installed outside of the kernel's build system should go into
"%s/lib/modules/%s/extra", but at present, perf will only look at them
when they are in "%s/lib/modules/%s/kernel". Lets encourage good
citizenship by relaxing this requirement to "%s/lib/modules/%s". This
way open source modules that are out-of-tree have no incentive to start
populating a directory reserved for in-kernel modules and I can stop
hex-editing my system's perf binary when profiling OSS out-of-tree
modules.

Feedback from Namhyung Kim correctly revealed that the hex-edits that I
had been doing meant that perf was also traversing the build and source
symlinks in %s/lib/modules/%s. That is undesireable, so we explicitly
exclude them from traversal with a minor tweak to the traversal routine.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Acked-by: Namhyung kim <namhyung@kernel.org>
Link: http://lkml.kernel.org/r/1398532675-13684-1-git-send-email-ryao@gentoo.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/machine.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
index a53cd0b..27c2a5e 100644
--- a/tools/perf/util/machine.c
+++ b/tools/perf/util/machine.c
@@ -717,7 +717,7 @@ static char *get_kernel_version(const char *root_dir)
 }
 
 static int map_groups__set_modules_path_dir(struct map_groups *mg,
-				const char *dir_name)
+				const char *dir_name, int depth)
 {
 	struct dirent *dent;
 	DIR *dir = opendir(dir_name);
@@ -742,7 +742,15 @@ static int map_groups__set_modules_path_dir(struct map_groups *mg,
 			    !strcmp(dent->d_name, ".."))
 				continue;
 
-			ret = map_groups__set_modules_path_dir(mg, path);
+			/* Do not follow top-level source and build symlinks */
+			if (depth == 0) {
+				if (!strcmp(dent->d_name, "source") ||
+				    !strcmp(dent->d_name, "build"))
+					continue;
+			}
+
+			ret = map_groups__set_modules_path_dir(mg, path,
+							       depth + 1);
 			if (ret < 0)
 				goto out;
 		} else {
@@ -786,11 +794,11 @@ static int machine__set_modules_path(struct machine *machine)
 	if (!version)
 		return -1;
 
-	snprintf(modules_path, sizeof(modules_path), "%s/lib/modules/%s/kernel",
+	snprintf(modules_path, sizeof(modules_path), "%s/lib/modules/%s",
 		 machine->root_dir, version);
 	free(version);
 
-	return map_groups__set_modules_path_dir(&machine->kmaps, modules_path);
+	return map_groups__set_modules_path_dir(&machine->kmaps, modules_path, 0);
 }
 
 static int machine__create_module(void *arg, const char *name, u64 start)

Re: [PATCH] perf machine: Search for modules in %s/lib/modules/%s

[Richard Yao]

[-- Attachment #1: Type: text/plain, Size: 1601 bytes --]

Dear Jirka,

I have sent an updated patch, but instead of using your white list idea,
I went with a black list. That way things like the historical addon
directory are included and anyone who decides to use a custom directory
for their own in-development modules is free to do so.

Yours truly,
Richard Yao

On 04/15/2014 07:56 AM, Jiri Olsa wrote:
> On Tue, Apr 15, 2014 at 02:44:52PM +0900, Namhyung Kim wrote:
>> Hi Richard,
>>
>> On Thu, 10 Apr 2014 12:52:59 -0400, Richard Yao wrote:
>>> Modules installed outside of the kernel's build system should go into
>>> "%s/lib/modules/%s/extra", but at present, perf will only look at them
>>> when they are in "%s/lib/modules/%s/kernel". Lets encourage good
>>> citizenship by relaxing this requirement to "%s/lib/modules/%s". This
>>> way open source modules that are out-of-tree have no incentive to start
>>> populating a directory reserved for in-kernle modules and I can stop hex
>>> editing my system's perf binary when profiling OSS out-of-tree modules.
>>
>> But it'll make the perf traverses all the source and build directories
>> too, right?  I don't think it's a right thing to do.
>>
>> Maybe we can also change stat() in map_groups__set_modules_path_dir() to
>> lstat() so that it cannot go to unwanted directories in that case.  Or
>> else, just checking "kernel" and "extra" directories will work.
> 
> yay, forgot about source directory.. :-\  looks like lstat should
> help, but hardcoding kernel and extra sounds better to me.
> 
> Richard, please send updated patch
> 
> thanks,
> jirka
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]

Re: [PATCH] perf machine: Search for modules in %s/lib/modules/%s

[Jiri Olsa]

On Tue, Apr 15, 2014 at 02:44:52PM +0900, Namhyung Kim wrote:
> Hi Richard,
> 
> On Thu, 10 Apr 2014 12:52:59 -0400, Richard Yao wrote:
> > Modules installed outside of the kernel's build system should go into
> > "%s/lib/modules/%s/extra", but at present, perf will only look at them
> > when they are in "%s/lib/modules/%s/kernel". Lets encourage good
> > citizenship by relaxing this requirement to "%s/lib/modules/%s". This
> > way open source modules that are out-of-tree have no incentive to start
> > populating a directory reserved for in-kernle modules and I can stop hex
> > editing my system's perf binary when profiling OSS out-of-tree modules.
> 
> But it'll make the perf traverses all the source and build directories
> too, right?  I don't think it's a right thing to do.
> 
> Maybe we can also change stat() in map_groups__set_modules_path_dir() to
> lstat() so that it cannot go to unwanted directories in that case.  Or
> else, just checking "kernel" and "extra" directories will work.

yay, forgot about source directory.. :-\  looks like lstat should
help, but hardcoding kernel and extra sounds better to me.

Richard, please send updated patch

thanks,
jirka

Re: [PATCH] perf machine: Search for modules in %s/lib/modules/%s

[Namhyung Kim]

Hi Richard,

On Thu, 10 Apr 2014 12:52:59 -0400, Richard Yao wrote:
> Modules installed outside of the kernel's build system should go into
> "%s/lib/modules/%s/extra", but at present, perf will only look at them
> when they are in "%s/lib/modules/%s/kernel". Lets encourage good
> citizenship by relaxing this requirement to "%s/lib/modules/%s". This
> way open source modules that are out-of-tree have no incentive to start
> populating a directory reserved for in-kernle modules and I can stop hex
> editing my system's perf binary when profiling OSS out-of-tree modules.

But it'll make the perf traverses all the source and build directories
too, right?  I don't think it's a right thing to do.

Maybe we can also change stat() in map_groups__set_modules_path_dir() to
lstat() so that it cannot go to unwanted directories in that case.  Or
else, just checking "kernel" and "extra" directories will work.

Thanks,
Namhyung

Re: [PATCH] perf machine: Search for modules in %s/lib/modules/%s

[David Ahern]

On 4/10/14, 9:52 AM, Richard Yao wrote:
> Modules installed outside of the kernel's build system should go into
> "%s/lib/modules/%s/extra", but at present, perf will only look at them
> when they are in "%s/lib/modules/%s/kernel". Lets encourage good
> citizenship by relaxing this requirement to "%s/lib/modules/%s". This
> way open source modules that are out-of-tree have no incentive to start
> populating a directory reserved for in-kernle modules and I can stop hex
> editing my system's perf binary when profiling OSS out-of-tree modules.
>
> Signed-off-by: Richard Yao <ryao@gentoo.org>
> ---
>   tools/perf/util/machine.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
> index a53cd0b..116842e 100644
> --- a/tools/perf/util/machine.c
> +++ b/tools/perf/util/machine.c
> @@ -786,7 +786,7 @@ static int machine__set_modules_path(struct machine *machine)
>   	if (!version)
>   		return -1;
>
> -	snprintf(modules_path, sizeof(modules_path), "%s/lib/modules/%s/kernel",
> +	snprintf(modules_path, sizeof(modules_path), "%s/lib/modules/%s",
>   		 machine->root_dir, version);
>   	free(version);
>
>

Acked-by: David Ahern <dsahern@gmail.com>

[PATCH] perf machine: Search for modules in %s/lib/modules/%s

[Richard Yao]

Modules installed outside of the kernel's build system should go into
"%s/lib/modules/%s/extra", but at present, perf will only look at them
when they are in "%s/lib/modules/%s/kernel". Lets encourage good
citizenship by relaxing this requirement to "%s/lib/modules/%s". This
way open source modules that are out-of-tree have no incentive to start
populating a directory reserved for in-kernle modules and I can stop hex
editing my system's perf binary when profiling OSS out-of-tree modules.

Signed-off-by: Richard Yao <ryao@gentoo.org>
---
 tools/perf/util/machine.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
index a53cd0b..116842e 100644
--- a/tools/perf/util/machine.c
+++ b/tools/perf/util/machine.c
@@ -786,7 +786,7 @@ static int machine__set_modules_path(struct machine *machine)
 	if (!version)
 		return -1;
 
-	snprintf(modules_path, sizeof(modules_path), "%s/lib/modules/%s/kernel",
+	snprintf(modules_path, sizeof(modules_path), "%s/lib/modules/%s",
 		 machine->root_dir, version);
 	free(version);
 
-- 
1.8.3.2