[RFC PATCH 0/5] perf events patches for improved ARM64 support

[RFC PATCH 0/5] perf events patches for improved ARM64 support

[John Garry]

This patchset adds support for some perf events features,
targeted at ARM64, implemented in a generic fashion.

The two main features are as follows:
- support for arch/vendor/platform pmu events directory structure
- support for parsing architecture recommended pmu events

On the back of these, the Cavium ThunderX2 and HiSilicon hip08
JSONs are relocated/added.

TODO: 
	- Documentation needs to be updated for recommended
	events format in the JSON
	- tidy up patches a bit

John Garry (5):
  perf jevents: add support for pmu events vendor subdirectory
  perf jevents: add support for arch recommended events
  perf vendor events arm64: add armv8 recommended events JSON
  perf vendor events arm64: relocate thunderx2 JSON
  perf vendor events arm64: add HiSilicon hip08 JSON

 .../pmu-events/arch/arm64/armv8-recommended.json   | 452 +++++++++++++++++++++
 .../arch/arm64/cavium/thunderx2-imp-def.json       |  62 ---
 .../arch/arm64/cavium/thunderx2/core-imp-def.json  |  32 ++
 .../arch/arm64/hisilicon/hip08/core-imp-def.json   | 122 ++++++
 tools/perf/pmu-events/arch/arm64/mapfile.csv       |   3 +-
 tools/perf/pmu-events/jevents.c                    | 272 ++++++++++++-
 6 files changed, 859 insertions(+), 84 deletions(-)
 create mode 100644 tools/perf/pmu-events/arch/arm64/armv8-recommended.json
 delete mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2-imp-def.json
 create mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2/core-imp-def.json
 create mode 100644 tools/perf/pmu-events/arch/arm64/hisilicon/hip08/core-imp-def.json

-- 
1.9.1

[RFC PATCH 0/5] perf events patches for improved ARM64 support

[John Garry]

This patchset adds support for some perf events features,
targeted at ARM64, implemented in a generic fashion.

The two main features are as follows:
- support for arch/vendor/platform pmu events directory structure
- support for parsing architecture recommended pmu events

On the back of these, the Cavium ThunderX2 and HiSilicon hip08
JSONs are relocated/added.

TODO: 
	- Documentation needs to be updated for recommended
	events format in the JSON
	- tidy up patches a bit

John Garry (5):
  perf jevents: add support for pmu events vendor subdirectory
  perf jevents: add support for arch recommended events
  perf vendor events arm64: add armv8 recommended events JSON
  perf vendor events arm64: relocate thunderx2 JSON
  perf vendor events arm64: add HiSilicon hip08 JSON

 .../pmu-events/arch/arm64/armv8-recommended.json   | 452 +++++++++++++++++++++
 .../arch/arm64/cavium/thunderx2-imp-def.json       |  62 ---
 .../arch/arm64/cavium/thunderx2/core-imp-def.json  |  32 ++
 .../arch/arm64/hisilicon/hip08/core-imp-def.json   | 122 ++++++
 tools/perf/pmu-events/arch/arm64/mapfile.csv       |   3 +-
 tools/perf/pmu-events/jevents.c                    | 272 ++++++++++++-
 6 files changed, 859 insertions(+), 84 deletions(-)
 create mode 100644 tools/perf/pmu-events/arch/arm64/armv8-recommended.json
 delete mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2-imp-def.json
 create mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2/core-imp-def.json
 create mode 100644 tools/perf/pmu-events/arch/arm64/hisilicon/hip08/core-imp-def.json

-- 
1.9.1

[RFC PATCH 1/5] perf jevents: add support for pmu events vendor subdirectory

[John Garry]

For some architectures (like arm64), it is required to
support a vendor sub-directory and not put all the JSONs
for a given vendor in the same dir.

This is because all the events for the same vendor will be
in the same pmu events table, which may cause conflict.
This conflict would be in the instance that a vendor's custom
implemented events do have the same meaning on different
platforms. In addition, "perf list" command may list events
which are not even supported for a given platform.

This patch adds support for an arch/vendor/platform directory
hierarchy, while maintaining support for arch/platform
structure. In this, each platform would always have its own pmu
events table.

In generated file pmu_events.c, each platform table name is in
the format pme{_vendor}_platform, like this:

struct pmu_events_map pmu_events_map[] = {
{
	.cpuid = "0x00000000420f5160",
	.version = "v1",
	.type = "core",
	.table = pme_cavium_thunderx2
},
{
	.cpuid = 0,
	.version = 0,
	.type = 0,
	.table = 0,
},
};

or this:

struct pmu_events_map pmu_events_map[] = {
{
	.cpuid = "GenuineIntel-6-56",
	.version = "v5",
	.type = "core",
	.table = pme_broadwellde
},

[snip]

{
	.cpuid = 0,
	.version = 0,
	.type = 0,
	.table = 0,
},
};

Signed-off-by: John Garry <john.garry@huawei.com>
---
 tools/perf/pmu-events/jevents.c | 57 ++++++++++++++++++++++++++++++++++++++---
 1 file changed, 53 insertions(+), 4 deletions(-)

diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
index b578aa2..a0d489e 100644
--- a/tools/perf/pmu-events/jevents.c
+++ b/tools/perf/pmu-events/jevents.c
@@ -588,7 +588,7 @@ static char *file_name_to_table_name(char *fname)
 	 * Derive rest of table name from basename of the JSON file,
 	 * replacing hyphens and stripping out .json suffix.
 	 */
-	n = asprintf(&tblname, "pme_%s", basename(fname));
+	n = asprintf(&tblname, "pme_%s", fname);
 	if (n < 0) {
 		pr_info("%s: asprintf() error %s for file %s\n", prog,
 				strerror(errno), fname);
@@ -598,7 +598,7 @@ static char *file_name_to_table_name(char *fname)
 	for (i = 0; i < strlen(tblname); i++) {
 		c = tblname[i];
 
-		if (c == '-')
+		if (c == '-' || c == '/')
 			tblname[i] = '_';
 		else if (c == '.') {
 			tblname[i] = '\0';
@@ -755,15 +755,52 @@ static int get_maxfds(void)
 static FILE *eventsfp;
 static char *mapfile;
 
+static int isLeafDir(const char *fpath)
+{
+	DIR 		  *d;
+	struct dirent *dir;
+	int res = 1;
+	d = opendir(fpath);
+	if (!d)
+		return 0;
+
+	while ((dir = readdir(d)) != NULL) {
+		if (dir-> d_type == DT_DIR && dir->d_name[0] != '.') {
+			res = 0;
+			break;
+		}
+	}
+
+	closedir(d);
+
+	return res;
+}
+
 static int process_one_file(const char *fpath, const struct stat *sb,
 			    int typeflag, struct FTW *ftwbuf)
 {
-	char *tblname, *bname  = (char *) fpath + ftwbuf->base;
+	char *tblname, *bname;
 	int is_dir  = typeflag == FTW_D;
 	int is_file = typeflag == FTW_F;
 	int level   = ftwbuf->level;
 	int err = 0;
 
+	if (level == 2 && is_dir) {
+		/*
+		 * For level 2 directory, bname will include parent name,
+		 * like vendor/platform. So search back from platform dir
+		 * to find this.
+		 */
+		bname = (char *) fpath + ftwbuf->base - 2;
+		while (true) {
+			if (*bname == '/')
+				break;
+			bname--;
+		}
+		bname++;
+	} else
+		bname = (char *) fpath + ftwbuf->base;
+
 	pr_debug("%s %d %7jd %-20s %s\n",
 		 is_file ? "f" : is_dir ? "d" : "x",
 		 level, sb->st_size, bname, fpath);
@@ -773,7 +810,7 @@ static int process_one_file(const char *fpath, const struct stat *sb,
 		return 0;
 
 	/* model directory, reset topic */
-	if (level == 1 && is_dir) {
+	if (level == 1 && is_dir && isLeafDir(fpath)) {
 		if (close_table)
 			print_events_table_suffix(eventsfp);
 
@@ -791,6 +828,18 @@ static int process_one_file(const char *fpath, const struct stat *sb,
 
 		print_events_table_prefix(eventsfp, tblname);
 		return 0;
+	} else if (level == 2 && is_dir) {
+		if (close_table)
+			print_events_table_suffix(eventsfp);
+
+		tblname = file_name_to_table_name(bname);
+		if (!tblname) {
+			pr_info("%s: Error determining table name for %s, exiting\n", prog,
+				bname);
+			return -1;
+		}
+
+		print_events_table_prefix(eventsfp, tblname);
 	}
 
 	/*
-- 
1.9.1

[RFC PATCH 1/5] perf jevents: add support for pmu events vendor subdirectory

[John Garry]

For some architectures (like arm64), it is required to
support a vendor sub-directory and not put all the JSONs
for a given vendor in the same dir.

This is because all the events for the same vendor will be
in the same pmu events table, which may cause conflict.
This conflict would be in the instance that a vendor's custom
implemented events do have the same meaning on different
platforms. In addition, "perf list" command may list events
which are not even supported for a given platform.

This patch adds support for an arch/vendor/platform directory
hierarchy, while maintaining support for arch/platform
structure. In this, each platform would always have its own pmu
events table.

In generated file pmu_events.c, each platform table name is in
the format pme{_vendor}_platform, like this:

struct pmu_events_map pmu_events_map[] = {
{
	.cpuid = "0x00000000420f5160",
	.version = "v1",
	.type = "core",
	.table = pme_cavium_thunderx2
},
{
	.cpuid = 0,
	.version = 0,
	.type = 0,
	.table = 0,
},
};

or this:

struct pmu_events_map pmu_events_map[] = {
{
	.cpuid = "GenuineIntel-6-56",
	.version = "v5",
	.type = "core",
	.table = pme_broadwellde
},

[snip]

{
	.cpuid = 0,
	.version = 0,
	.type = 0,
	.table = 0,
},
};

Signed-off-by: John Garry <john.garry@huawei.com>
---
 tools/perf/pmu-events/jevents.c | 57 ++++++++++++++++++++++++++++++++++++++---
 1 file changed, 53 insertions(+), 4 deletions(-)

diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
index b578aa2..a0d489e 100644
--- a/tools/perf/pmu-events/jevents.c
+++ b/tools/perf/pmu-events/jevents.c
@@ -588,7 +588,7 @@ static char *file_name_to_table_name(char *fname)
 	 * Derive rest of table name from basename of the JSON file,
 	 * replacing hyphens and stripping out .json suffix.
 	 */
-	n = asprintf(&tblname, "pme_%s", basename(fname));
+	n = asprintf(&tblname, "pme_%s", fname);
 	if (n < 0) {
 		pr_info("%s: asprintf() error %s for file %s\n", prog,
 				strerror(errno), fname);
@@ -598,7 +598,7 @@ static char *file_name_to_table_name(char *fname)
 	for (i = 0; i < strlen(tblname); i++) {
 		c = tblname[i];
 
-		if (c == '-')
+		if (c == '-' || c == '/')
 			tblname[i] = '_';
 		else if (c == '.') {
 			tblname[i] = '\0';
@@ -755,15 +755,52 @@ static int get_maxfds(void)
 static FILE *eventsfp;
 static char *mapfile;
 
+static int isLeafDir(const char *fpath)
+{
+	DIR 		  *d;
+	struct dirent *dir;
+	int res = 1;
+	d = opendir(fpath);
+	if (!d)
+		return 0;
+
+	while ((dir = readdir(d)) != NULL) {
+		if (dir-> d_type == DT_DIR && dir->d_name[0] != '.') {
+			res = 0;
+			break;
+		}
+	}
+
+	closedir(d);
+
+	return res;
+}
+
 static int process_one_file(const char *fpath, const struct stat *sb,
 			    int typeflag, struct FTW *ftwbuf)
 {
-	char *tblname, *bname  = (char *) fpath + ftwbuf->base;
+	char *tblname, *bname;
 	int is_dir  = typeflag == FTW_D;
 	int is_file = typeflag == FTW_F;
 	int level   = ftwbuf->level;
 	int err = 0;
 
+	if (level == 2 && is_dir) {
+		/*
+		 * For level 2 directory, bname will include parent name,
+		 * like vendor/platform. So search back from platform dir
+		 * to find this.
+		 */
+		bname = (char *) fpath + ftwbuf->base - 2;
+		while (true) {
+			if (*bname == '/')
+				break;
+			bname--;
+		}
+		bname++;
+	} else
+		bname = (char *) fpath + ftwbuf->base;
+
 	pr_debug("%s %d %7jd %-20s %s\n",
 		 is_file ? "f" : is_dir ? "d" : "x",
 		 level, sb->st_size, bname, fpath);
@@ -773,7 +810,7 @@ static int process_one_file(const char *fpath, const struct stat *sb,
 		return 0;
 
 	/* model directory, reset topic */
-	if (level == 1 && is_dir) {
+	if (level == 1 && is_dir && isLeafDir(fpath)) {
 		if (close_table)
 			print_events_table_suffix(eventsfp);
 
@@ -791,6 +828,18 @@ static int process_one_file(const char *fpath, const struct stat *sb,
 
 		print_events_table_prefix(eventsfp, tblname);
 		return 0;
+	} else if (level == 2 && is_dir) {
+		if (close_table)
+			print_events_table_suffix(eventsfp);
+
+		tblname = file_name_to_table_name(bname);
+		if (!tblname) {
+			pr_info("%s: Error determining table name for %s, exiting\n", prog,
+				bname);
+			return -1;
+		}
+
+		print_events_table_prefix(eventsfp, tblname);
 	}
 
 	/*
-- 
1.9.1

Re: [RFC PATCH 1/5] perf jevents: add support for pmu events vendor subdirectory

[Jiri Olsa]

On Wed, Dec 06, 2017 at 12:13:15AM +0800, John Garry wrote:

SNIP

> diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
> index b578aa2..a0d489e 100644
> --- a/tools/perf/pmu-events/jevents.c
> +++ b/tools/perf/pmu-events/jevents.c
> @@ -588,7 +588,7 @@ static char *file_name_to_table_name(char *fname)
>  	 * Derive rest of table name from basename of the JSON file,
>  	 * replacing hyphens and stripping out .json suffix.
>  	 */
> -	n = asprintf(&tblname, "pme_%s", basename(fname));
> +	n = asprintf(&tblname, "pme_%s", fname);
>  	if (n < 0) {
>  		pr_info("%s: asprintf() error %s for file %s\n", prog,
>  				strerror(errno), fname);
> @@ -598,7 +598,7 @@ static char *file_name_to_table_name(char *fname)
>  	for (i = 0; i < strlen(tblname); i++) {
>  		c = tblname[i];
>  
> -		if (c == '-')
> +		if (c == '-' || c == '/')
>  			tblname[i] = '_';
>  		else if (c == '.') {
>  			tblname[i] = '\0';
> @@ -755,15 +755,52 @@ static int get_maxfds(void)
>  static FILE *eventsfp;
>  static char *mapfile;
>  
> +static int isLeafDir(const char *fpath)

we use _ to separate words in functions names

> +{
> +	DIR 		  *d;
> +	struct dirent *dir;
> +	int res = 1;
> +	d = opendir(fpath);
> +	if (!d)
> +		return 0;
> +
> +	while ((dir = readdir(d)) != NULL) {
> +		if (dir-> d_type == DT_DIR && dir->d_name[0] != '.') {
> +			res = 0;
> +			break;

just recently got into a issue on xfs when d_type is DT_UNKNOWN
for directory.. you need to handle it

thanks,
jirka

[RFC PATCH 1/5] perf jevents: add support for pmu events vendor subdirectory

[Jiri Olsa]

On Wed, Dec 06, 2017 at 12:13:15AM +0800, John Garry wrote:

SNIP

> diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
> index b578aa2..a0d489e 100644
> --- a/tools/perf/pmu-events/jevents.c
> +++ b/tools/perf/pmu-events/jevents.c
> @@ -588,7 +588,7 @@ static char *file_name_to_table_name(char *fname)
>  	 * Derive rest of table name from basename of the JSON file,
>  	 * replacing hyphens and stripping out .json suffix.
>  	 */
> -	n = asprintf(&tblname, "pme_%s", basename(fname));
> +	n = asprintf(&tblname, "pme_%s", fname);
>  	if (n < 0) {
>  		pr_info("%s: asprintf() error %s for file %s\n", prog,
>  				strerror(errno), fname);
> @@ -598,7 +598,7 @@ static char *file_name_to_table_name(char *fname)
>  	for (i = 0; i < strlen(tblname); i++) {
>  		c = tblname[i];
>  
> -		if (c == '-')
> +		if (c == '-' || c == '/')
>  			tblname[i] = '_';
>  		else if (c == '.') {
>  			tblname[i] = '\0';
> @@ -755,15 +755,52 @@ static int get_maxfds(void)
>  static FILE *eventsfp;
>  static char *mapfile;
>  
> +static int isLeafDir(const char *fpath)

we use _ to separate words in functions names

> +{
> +	DIR 		  *d;
> +	struct dirent *dir;
> +	int res = 1;
> +	d = opendir(fpath);
> +	if (!d)
> +		return 0;
> +
> +	while ((dir = readdir(d)) != NULL) {
> +		if (dir-> d_type == DT_DIR && dir->d_name[0] != '.') {
> +			res = 0;
> +			break;

just recently got into a issue on xfs when d_type is DT_UNKNOWN
for directory.. you need to handle it

thanks,
jirka

Re: [RFC PATCH 1/5] perf jevents: add support for pmu events vendor subdirectory

[John Garry]

On 06/12/2017 13:38, Jiri Olsa wrote:
> On Wed, Dec 06, 2017 at 12:13:15AM +0800, John Garry wrote:
>
> SNIP
>
>> diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
>> index b578aa2..a0d489e 100644
>> --- a/tools/perf/pmu-events/jevents.c
>> +++ b/tools/perf/pmu-events/jevents.c
>> @@ -588,7 +588,7 @@ static char *file_name_to_table_name(char *fname)
>>  	 * Derive rest of table name from basename of the JSON file,
>>  	 * replacing hyphens and stripping out .json suffix.
>>  	 */
>> -	n = asprintf(&tblname, "pme_%s", basename(fname));
>> +	n = asprintf(&tblname, "pme_%s", fname);
>>  	if (n < 0) {
>>  		pr_info("%s: asprintf() error %s for file %s\n", prog,
>>  				strerror(errno), fname);
>> @@ -598,7 +598,7 @@ static char *file_name_to_table_name(char *fname)
>>  	for (i = 0; i < strlen(tblname); i++) {
>>  		c = tblname[i];
>>
>> -		if (c == '-')
>> +		if (c == '-' || c == '/')
>>  			tblname[i] = '_';
>>  		else if (c == '.') {
>>  			tblname[i] = '\0';
>> @@ -755,15 +755,52 @@ static int get_maxfds(void)
>>  static FILE *eventsfp;
>>  static char *mapfile;
>>
>> +static int isLeafDir(const char *fpath)
>
> we use _ to separate words in functions names
>

Ok (I will also check functions added elsewhere)

>> +{
>> +	DIR 		  *d;
>> +	struct dirent *dir;
>> +	int res = 1;
>> +	d = opendir(fpath);
>> +	if (!d)
>> +		return 0;
>> +
>> +	while ((dir = readdir(d)) != NULL) {
>> +		if (dir-> d_type == DT_DIR && dir->d_name[0] != '.') {
>> +			res = 0;
>> +			break;
>
> just recently got into a issue on xfs when d_type is DT_UNKNOWN
> for directory.. you need to handle it
>

Sure

Cheers,
John

> thanks,
> jirka
>
> .
>

[RFC PATCH 1/5] perf jevents: add support for pmu events vendor subdirectory

[John Garry]

On 06/12/2017 13:38, Jiri Olsa wrote:
> On Wed, Dec 06, 2017 at 12:13:15AM +0800, John Garry wrote:
>
> SNIP
>
>> diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
>> index b578aa2..a0d489e 100644
>> --- a/tools/perf/pmu-events/jevents.c
>> +++ b/tools/perf/pmu-events/jevents.c
>> @@ -588,7 +588,7 @@ static char *file_name_to_table_name(char *fname)
>>  	 * Derive rest of table name from basename of the JSON file,
>>  	 * replacing hyphens and stripping out .json suffix.
>>  	 */
>> -	n = asprintf(&tblname, "pme_%s", basename(fname));
>> +	n = asprintf(&tblname, "pme_%s", fname);
>>  	if (n < 0) {
>>  		pr_info("%s: asprintf() error %s for file %s\n", prog,
>>  				strerror(errno), fname);
>> @@ -598,7 +598,7 @@ static char *file_name_to_table_name(char *fname)
>>  	for (i = 0; i < strlen(tblname); i++) {
>>  		c = tblname[i];
>>
>> -		if (c == '-')
>> +		if (c == '-' || c == '/')
>>  			tblname[i] = '_';
>>  		else if (c == '.') {
>>  			tblname[i] = '\0';
>> @@ -755,15 +755,52 @@ static int get_maxfds(void)
>>  static FILE *eventsfp;
>>  static char *mapfile;
>>
>> +static int isLeafDir(const char *fpath)
>
> we use _ to separate words in functions names
>

Ok (I will also check functions added elsewhere)

>> +{
>> +	DIR 		  *d;
>> +	struct dirent *dir;
>> +	int res = 1;
>> +	d = opendir(fpath);
>> +	if (!d)
>> +		return 0;
>> +
>> +	while ((dir = readdir(d)) != NULL) {
>> +		if (dir-> d_type == DT_DIR && dir->d_name[0] != '.') {
>> +			res = 0;
>> +			break;
>
> just recently got into a issue on xfs when d_type is DT_UNKNOWN
> for directory.. you need to handle it
>

Sure

Cheers,
John

> thanks,
> jirka
>
> .
>

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

For some architectures (like arm64), there are architecture-
defined recommended events. Vendors may not be obliged to
follow the recommendation and may implement their own pmu
event for a specific event code.

This patch adds support for parsing events from arch-defined
recommended JSONs, and then fixing up vendor events when
they have implemented these events as recommended.

In the vendor JSON, to specify that the event is supported
according to the recommendation, only the event code is
added to the JSON entry - no other event elements need be
added, like below:
[
    {
        "EventCode": "0x40",
    },

]

The pmu event parsing will check for "BriefDescription"
field presence only for this.

If "BriefDescription" is present, then it is implied
that the vendor has implemented their own custom event,
and there is no fixup. Other fields are ignored.

*TODO: update documentation

Signed-off-by: John Garry <john.garry@huawei.com>
---
 tools/perf/pmu-events/jevents.c | 215 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 198 insertions(+), 17 deletions(-)

diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
index a0d489e..a820ed4 100644
--- a/tools/perf/pmu-events/jevents.c
+++ b/tools/perf/pmu-events/jevents.c
@@ -42,6 +42,7 @@
 #include <dirent.h>
 #include <sys/time.h>			/* getrlimit */
 #include <sys/resource.h>		/* getrlimit */
+#include <sys/queue.h>
 #include <ftw.h>
 #include <sys/stat.h>
 #include "jsmn.h"
@@ -366,6 +367,94 @@ static int print_events_table_entry(void *data, char *name, char *event,
 	return 0;
 }
 
+struct event_struct {
+	char *name;
+	char *event;
+	char *desc;
+	char *long_desc;
+	char *pmu;
+	char *unit;
+	char *perpkg;
+	char *metric_expr;
+	char *metric_name;
+	char *metric_group;
+	LIST_ENTRY(event_struct) list;
+	char strings[];
+};
+
+LIST_HEAD(listhead, event_struct) recommended_events;
+
+static int save_recommended_events(void *data, char *name, char *event,
+				    char *desc, char *long_desc,
+				    char *pmu, char *unit, char *perpkg,
+				    char *metric_expr,
+				    char *metric_name, char *metric_group)
+{
+	static int count = 0;
+	char temp[1024];
+	struct event_struct *es;
+	struct stat *sb = data;
+	int len = 0;
+	char *strings;
+
+	/*
+	 * Lazily allocate size of the JSON file to hold the
+	 * strings, which would be more than large enough.
+	 */
+	len = sb->st_size;
+
+	es = malloc(sizeof(*es) + len);
+	if (!es)
+		return -ENOMEM;
+	memset(es, 0, sizeof(*es));
+	LIST_INSERT_HEAD(&recommended_events, es, list);
+
+	strings = &es->strings[0];
+
+	if (name) {
+		es->name = strings;
+		strings += snprintf(strings, len, "%s", name) + 1;
+	}
+	if (event) {
+		es->event = strings;
+		strings += snprintf(strings, len, "%s", event) + 1;
+	}
+	if (desc) {
+		es->desc = strings;
+		strings += snprintf(strings, len, "%s", desc) + 1;
+	}
+	if (long_desc) {
+		es->long_desc = strings;
+		strings += snprintf(strings, len, "%s", long_desc) + 1;
+	}
+	if (pmu) {
+		es->pmu = strings;
+		strings += snprintf(strings, len, "%s", pmu) + 1;
+	}
+	if (unit) {
+		es->unit = strings;
+		strings += snprintf(strings, len, "%s", unit) + 1;
+	}
+	if (perpkg) {
+		es->perpkg = strings;
+		strings += snprintf(strings, len, "%s", perpkg) + 1;
+	}
+	if (metric_expr) {
+		es->metric_expr = strings;
+		strings += snprintf(strings, len, "%s", metric_expr) + 1;
+	}
+	if (metric_name) {
+		es->metric_name = strings;
+		strings += snprintf(strings, len, "%s", metric_name) + 1;
+	}
+	if (metric_group) {
+		es->metric_group = strings;
+		strings += snprintf(strings, len, "%s", metric_group) + 1;
+	}
+
+	return 0;
+}
+
 static void print_events_table_suffix(FILE *outfp)
 {
 	fprintf(outfp, "{\n");
@@ -407,6 +496,61 @@ static char *real_event(const char *name, char *event)
 	return event;
 }
 
+static void fixup_field(char *from, char **to)
+{
+	/*
+	 * If we already had a valid pointer (string), then
+	 * don't allocate a new one, just reuse and overwrite.
+	 */
+	if (!*to)
+		*to = malloc(strlen(from));
+
+	strcpy(*to, from);
+}
+
+static int try_fixup(const char *fn, char *event, char **desc, char **name, char **long_desc, char **pmu, char **filter,
+				char **perpkg, char **unit, char **metric_expr, char **metric_name, char **metric_group)
+{
+	/* try to find matching event from recommended values */
+	struct event_struct *es;
+
+	LIST_FOREACH(es, &recommended_events, list) {
+		if (!strcmp(event, es->event)) {
+			/* now fixup */
+			if (es->desc)
+				fixup_field(es->desc, desc);
+			if (es->name)
+				fixup_field(es->name, name);
+			if (es->long_desc)
+				fixup_field(es->long_desc, long_desc);
+			if (es->pmu)
+				fixup_field(es->pmu, pmu);
+		//	if (event_struct->filter)
+		//		fixup_field(event_struct->filter, filter);
+			if (es->perpkg)
+				fixup_field(es->perpkg, perpkg);
+			if (es->unit)
+				fixup_field(es->unit, unit);
+			if (es->metric_expr)
+				fixup_field(es->metric_expr, metric_expr);
+			if (es->metric_name)
+				fixup_field(es->metric_name, metric_name);
+			if (es->metric_group)
+				fixup_field(es->metric_group, metric_group);
+
+			return 0;
+		}
+	}
+
+	pr_err("%s: could not find matching %s for %s\n", prog, event, fn);
+	return -1;
+}
+
+#define  FREE_MEMORIES \
+		free(event); free(desc); free(name); free(long_desc); \
+		free(extra_desc);  free(pmu); free(filter); free(perpkg); \
+		free(unit); free(metric_expr); free(metric_name);
+
 /* Call func with each event in the json file */
 int json_events(const char *fn,
 	  int (*func)(void *data, char *name, char *event, char *desc,
@@ -551,20 +695,22 @@ int json_events(const char *fn,
 		if (name)
 			fixname(name);
 
+		if (!desc) {
+			/*
+			 * If we have no valid desc, then fixup *all* values from recommended
+			 * by matching the event.
+			 */
+			err = try_fixup(fn, event, &desc, &name, &long_desc, &pmu, &filter, &perpkg, &unit, &metric_expr,
+					&metric_name, &metric_group);
+			if (err) {
+				FREE_MEMORIES
+				goto out_free;
+			}
+		}
+
 		err = func(data, name, real_event(name, event), desc, long_desc,
 			   pmu, unit, perpkg, metric_expr, metric_name, metric_group);
-		free(event);
-		free(desc);
-		free(name);
-		free(long_desc);
-		free(extra_desc);
-		free(pmu);
-		free(filter);
-		free(perpkg);
-		free(unit);
-		free(metric_expr);
-		free(metric_name);
-		free(metric_group);
+		FREE_MEMORIES
 		if (err)
 			break;
 		tok += j;
@@ -776,6 +922,32 @@ static int isLeafDir(const char *fpath)
 	return res;
 }
 
+static int isJsonFile(const char *name)
+{
+	const char *suffix;
+
+	if (strlen(name) < 5)
+		return 0;
+
+	suffix = name + strlen(name) - 5;
+
+	if (strncmp(suffix, ".json", 5) == 0)
+		return 1;
+	return 0;
+}
+
+static int preprocess_level0_files(const char *fpath, const struct stat *sb,
+				int typeflag, struct FTW *ftwbuf)
+{
+	int level	= ftwbuf->level;
+	int is_file = typeflag == FTW_F;
+
+	if (level == 1 && is_file && isJsonFile(fpath))
+		return json_events(fpath, save_recommended_events, (void *)sb);
+
+	return 0;
+}
+
 static int process_one_file(const char *fpath, const struct stat *sb,
 			    int typeflag, struct FTW *ftwbuf)
 {
@@ -806,8 +978,10 @@ static int process_one_file(const char *fpath, const struct stat *sb,
 		 level, sb->st_size, bname, fpath);
 
 	/* base dir */
-	if (level == 0)
-		return 0;
+	if (level == 0) {
+		LIST_INIT(&recommended_events);
+		return nftw(fpath, preprocess_level0_files, get_maxfds(), 0);
+	}
 
 	/* model directory, reset topic */
 	if (level == 1 && is_dir && isLeafDir(fpath)) {
@@ -869,9 +1043,7 @@ static int process_one_file(const char *fpath, const struct stat *sb,
 	 * ignore it. It could be a readme.txt for instance.
 	 */
 	if (is_file) {
-		char *suffix = bname + strlen(bname) - 5;
-
-		if (strncmp(suffix, ".json", 5)) {
+		if (!isJsonFile(bname)) {
 			pr_info("%s: Ignoring file without .json suffix %s\n", prog,
 				fpath);
 			return 0;
@@ -933,6 +1105,7 @@ int main(int argc, char *argv[])
 	const char *output_file;
 	const char *start_dirname;
 	struct stat stbuf;
+	struct event_struct *es1, *es2;
 
 	prog = basename(argv[0]);
 	if (argc < 4) {
@@ -988,6 +1161,14 @@ int main(int argc, char *argv[])
 		goto empty_map;
 	}
 
+	/* Free struct for recommended events */
+	es1 = LIST_FIRST(&recommended_events);
+	while (es1) {
+		es2 = LIST_NEXT(es1, list);
+		free(es1);
+		es1 = es2;
+	}
+
 	if (close_table)
 		print_events_table_suffix(eventsfp);
 
-- 
1.9.1

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

For some architectures (like arm64), there are architecture-
defined recommended events. Vendors may not be obliged to
follow the recommendation and may implement their own pmu
event for a specific event code.

This patch adds support for parsing events from arch-defined
recommended JSONs, and then fixing up vendor events when
they have implemented these events as recommended.

In the vendor JSON, to specify that the event is supported
according to the recommendation, only the event code is
added to the JSON entry - no other event elements need be
added, like below:
[
    {
        "EventCode": "0x40",
    },

]

The pmu event parsing will check for "BriefDescription"
field presence only for this.

If "BriefDescription" is present, then it is implied
that the vendor has implemented their own custom event,
and there is no fixup. Other fields are ignored.

*TODO: update documentation

Signed-off-by: John Garry <john.garry@huawei.com>
---
 tools/perf/pmu-events/jevents.c | 215 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 198 insertions(+), 17 deletions(-)

diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
index a0d489e..a820ed4 100644
--- a/tools/perf/pmu-events/jevents.c
+++ b/tools/perf/pmu-events/jevents.c
@@ -42,6 +42,7 @@
 #include <dirent.h>
 #include <sys/time.h>			/* getrlimit */
 #include <sys/resource.h>		/* getrlimit */
+#include <sys/queue.h>
 #include <ftw.h>
 #include <sys/stat.h>
 #include "jsmn.h"
@@ -366,6 +367,94 @@ static int print_events_table_entry(void *data, char *name, char *event,
 	return 0;
 }
 
+struct event_struct {
+	char *name;
+	char *event;
+	char *desc;
+	char *long_desc;
+	char *pmu;
+	char *unit;
+	char *perpkg;
+	char *metric_expr;
+	char *metric_name;
+	char *metric_group;
+	LIST_ENTRY(event_struct) list;
+	char strings[];
+};
+
+LIST_HEAD(listhead, event_struct) recommended_events;
+
+static int save_recommended_events(void *data, char *name, char *event,
+				    char *desc, char *long_desc,
+				    char *pmu, char *unit, char *perpkg,
+				    char *metric_expr,
+				    char *metric_name, char *metric_group)
+{
+	static int count = 0;
+	char temp[1024];
+	struct event_struct *es;
+	struct stat *sb = data;
+	int len = 0;
+	char *strings;
+
+	/*
+	 * Lazily allocate size of the JSON file to hold the
+	 * strings, which would be more than large enough.
+	 */
+	len = sb->st_size;
+
+	es = malloc(sizeof(*es) + len);
+	if (!es)
+		return -ENOMEM;
+	memset(es, 0, sizeof(*es));
+	LIST_INSERT_HEAD(&recommended_events, es, list);
+
+	strings = &es->strings[0];
+
+	if (name) {
+		es->name = strings;
+		strings += snprintf(strings, len, "%s", name) + 1;
+	}
+	if (event) {
+		es->event = strings;
+		strings += snprintf(strings, len, "%s", event) + 1;
+	}
+	if (desc) {
+		es->desc = strings;
+		strings += snprintf(strings, len, "%s", desc) + 1;
+	}
+	if (long_desc) {
+		es->long_desc = strings;
+		strings += snprintf(strings, len, "%s", long_desc) + 1;
+	}
+	if (pmu) {
+		es->pmu = strings;
+		strings += snprintf(strings, len, "%s", pmu) + 1;
+	}
+	if (unit) {
+		es->unit = strings;
+		strings += snprintf(strings, len, "%s", unit) + 1;
+	}
+	if (perpkg) {
+		es->perpkg = strings;
+		strings += snprintf(strings, len, "%s", perpkg) + 1;
+	}
+	if (metric_expr) {
+		es->metric_expr = strings;
+		strings += snprintf(strings, len, "%s", metric_expr) + 1;
+	}
+	if (metric_name) {
+		es->metric_name = strings;
+		strings += snprintf(strings, len, "%s", metric_name) + 1;
+	}
+	if (metric_group) {
+		es->metric_group = strings;
+		strings += snprintf(strings, len, "%s", metric_group) + 1;
+	}
+
+	return 0;
+}
+
 static void print_events_table_suffix(FILE *outfp)
 {
 	fprintf(outfp, "{\n");
@@ -407,6 +496,61 @@ static char *real_event(const char *name, char *event)
 	return event;
 }
 
+static void fixup_field(char *from, char **to)
+{
+	/*
+	 * If we already had a valid pointer (string), then
+	 * don't allocate a new one, just reuse and overwrite.
+	 */
+	if (!*to)
+		*to = malloc(strlen(from));
+
+	strcpy(*to, from);
+}
+
+static int try_fixup(const char *fn, char *event, char **desc, char **name, char **long_desc, char **pmu, char **filter,
+				char **perpkg, char **unit, char **metric_expr, char **metric_name, char **metric_group)
+{
+	/* try to find matching event from recommended values */
+	struct event_struct *es;
+
+	LIST_FOREACH(es, &recommended_events, list) {
+		if (!strcmp(event, es->event)) {
+			/* now fixup */
+			if (es->desc)
+				fixup_field(es->desc, desc);
+			if (es->name)
+				fixup_field(es->name, name);
+			if (es->long_desc)
+				fixup_field(es->long_desc, long_desc);
+			if (es->pmu)
+				fixup_field(es->pmu, pmu);
+		//	if (event_struct->filter)
+		//		fixup_field(event_struct->filter, filter);
+			if (es->perpkg)
+				fixup_field(es->perpkg, perpkg);
+			if (es->unit)
+				fixup_field(es->unit, unit);
+			if (es->metric_expr)
+				fixup_field(es->metric_expr, metric_expr);
+			if (es->metric_name)
+				fixup_field(es->metric_name, metric_name);
+			if (es->metric_group)
+				fixup_field(es->metric_group, metric_group);
+
+			return 0;
+		}
+	}
+
+	pr_err("%s: could not find matching %s for %s\n", prog, event, fn);
+	return -1;
+}
+
+#define  FREE_MEMORIES \
+		free(event); free(desc); free(name); free(long_desc); \
+		free(extra_desc);  free(pmu); free(filter); free(perpkg); \
+		free(unit); free(metric_expr); free(metric_name);
+
 /* Call func with each event in the json file */
 int json_events(const char *fn,
 	  int (*func)(void *data, char *name, char *event, char *desc,
@@ -551,20 +695,22 @@ int json_events(const char *fn,
 		if (name)
 			fixname(name);
 
+		if (!desc) {
+			/*
+			 * If we have no valid desc, then fixup *all* values from recommended
+			 * by matching the event.
+			 */
+			err = try_fixup(fn, event, &desc, &name, &long_desc, &pmu, &filter, &perpkg, &unit, &metric_expr,
+					&metric_name, &metric_group);
+			if (err) {
+				FREE_MEMORIES
+				goto out_free;
+			}
+		}
+
 		err = func(data, name, real_event(name, event), desc, long_desc,
 			   pmu, unit, perpkg, metric_expr, metric_name, metric_group);
-		free(event);
-		free(desc);
-		free(name);
-		free(long_desc);
-		free(extra_desc);
-		free(pmu);
-		free(filter);
-		free(perpkg);
-		free(unit);
-		free(metric_expr);
-		free(metric_name);
-		free(metric_group);
+		FREE_MEMORIES
 		if (err)
 			break;
 		tok += j;
@@ -776,6 +922,32 @@ static int isLeafDir(const char *fpath)
 	return res;
 }
 
+static int isJsonFile(const char *name)
+{
+	const char *suffix;
+
+	if (strlen(name) < 5)
+		return 0;
+
+	suffix = name + strlen(name) - 5;
+
+	if (strncmp(suffix, ".json", 5) == 0)
+		return 1;
+	return 0;
+}
+
+static int preprocess_level0_files(const char *fpath, const struct stat *sb,
+				int typeflag, struct FTW *ftwbuf)
+{
+	int level	= ftwbuf->level;
+	int is_file = typeflag == FTW_F;
+
+	if (level == 1 && is_file && isJsonFile(fpath))
+		return json_events(fpath, save_recommended_events, (void *)sb);
+
+	return 0;
+}
+
 static int process_one_file(const char *fpath, const struct stat *sb,
 			    int typeflag, struct FTW *ftwbuf)
 {
@@ -806,8 +978,10 @@ static int process_one_file(const char *fpath, const struct stat *sb,
 		 level, sb->st_size, bname, fpath);
 
 	/* base dir */
-	if (level == 0)
-		return 0;
+	if (level == 0) {
+		LIST_INIT(&recommended_events);
+		return nftw(fpath, preprocess_level0_files, get_maxfds(), 0);
+	}
 
 	/* model directory, reset topic */
 	if (level == 1 && is_dir && isLeafDir(fpath)) {
@@ -869,9 +1043,7 @@ static int process_one_file(const char *fpath, const struct stat *sb,
 	 * ignore it. It could be a readme.txt for instance.
 	 */
 	if (is_file) {
-		char *suffix = bname + strlen(bname) - 5;
-
-		if (strncmp(suffix, ".json", 5)) {
+		if (!isJsonFile(bname)) {
 			pr_info("%s: Ignoring file without .json suffix %s\n", prog,
 				fpath);
 			return 0;
@@ -933,6 +1105,7 @@ int main(int argc, char *argv[])
 	const char *output_file;
 	const char *start_dirname;
 	struct stat stbuf;
+	struct event_struct *es1, *es2;
 
 	prog = basename(argv[0]);
 	if (argc < 4) {
@@ -988,6 +1161,14 @@ int main(int argc, char *argv[])
 		goto empty_map;
 	}
 
+	/* Free struct for recommended events */
+	es1 = LIST_FIRST(&recommended_events);
+	while (es1) {
+		es2 = LIST_NEXT(es1, list);
+		free(es1);
+		es1 = es2;
+	}
+
 	if (close_table)
 		print_events_table_suffix(eventsfp);
 
-- 
1.9.1

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Andi Kleen]

On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
> For some architectures (like arm64), there are architecture-
> defined recommended events. Vendors may not be obliged to
> follow the recommendation and may implement their own pmu
> event for a specific event cod

I would just duplicate the architected events into the different 
vendor files.  Then you wouldn't need all this mess.

-Andi

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Andi Kleen]

On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
> For some architectures (like arm64), there are architecture-
> defined recommended events. Vendors may not be obliged to
> follow the recommendation and may implement their own pmu
> event for a specific event cod

I would just duplicate the architected events into the different 
vendor files.  Then you wouldn't need all this mess.

-Andi

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 05/12/2017 17:27, Andi Kleen wrote:
> On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
>> For some architectures (like arm64), there are architecture-
>> defined recommended events. Vendors may not be obliged to
>> follow the recommendation and may implement their own pmu
>> event for a specific event cod
>
> I would just duplicate the architected events into the different
> vendor files.  Then you wouldn't need all this mess.
>

This is what we were originally doing:
https://patchwork.kernel.org/patch/10010859/

But then we thought that we could avoid duplicating all these events for 
every platform from every vendor. Most, if not all, vendors will 
implement the events as recommended for any platform, so much 
unnecessary duplication.

cheers,
John


> -Andi
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
> .
>

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 05/12/2017 17:27, Andi Kleen wrote:
> On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
>> For some architectures (like arm64), there are architecture-
>> defined recommended events. Vendors may not be obliged to
>> follow the recommendation and may implement their own pmu
>> event for a specific event cod
>
> I would just duplicate the architected events into the different
> vendor files.  Then you wouldn't need all this mess.
>

This is what we were originally doing:
https://patchwork.kernel.org/patch/10010859/

But then we thought that we could avoid duplicating all these events for 
every platform from every vendor. Most, if not all, vendors will 
implement the events as recommended for any platform, so much 
unnecessary duplication.

cheers,
John


> -Andi
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
> .
>

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
> For some architectures (like arm64), there are architecture-
> defined recommended events. Vendors may not be obliged to
> follow the recommendation and may implement their own pmu
> event for a specific event code.
> 
> This patch adds support for parsing events from arch-defined
> recommended JSONs, and then fixing up vendor events when
> they have implemented these events as recommended.

in the previous patch you added the vendor support, so
you have arch|vendor|platform key for the event list
and perf have the most current/local event list

why would you need to fix it? if there's new event list,
the table gets updated, perf is rebuilt.. I'm clearly
missing something ;-)

> In the vendor JSON, to specify that the event is supported
> according to the recommendation, only the event code is
> added to the JSON entry - no other event elements need be
> added, like below:
> [
>     {
>         "EventCode": "0x40",
>     },
> 
> ]
> 
> The pmu event parsing will check for "BriefDescription"
> field presence only for this.
> 
> If "BriefDescription" is present, then it is implied
> that the vendor has implemented their own custom event,
> and there is no fixup. Other fields are ignored.

if we are going this way, please use some new token,
this list is supposed to be human readable

thanks,
jirka

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
> For some architectures (like arm64), there are architecture-
> defined recommended events. Vendors may not be obliged to
> follow the recommendation and may implement their own pmu
> event for a specific event code.
> 
> This patch adds support for parsing events from arch-defined
> recommended JSONs, and then fixing up vendor events when
> they have implemented these events as recommended.

in the previous patch you added the vendor support, so
you have arch|vendor|platform key for the event list
and perf have the most current/local event list

why would you need to fix it? if there's new event list,
the table gets updated, perf is rebuilt.. I'm clearly
missing something ;-)

> In the vendor JSON, to specify that the event is supported
> according to the recommendation, only the event code is
> added to the JSON entry - no other event elements need be
> added, like below:
> [
>     {
>         "EventCode": "0x40",
>     },
> 
> ]
> 
> The pmu event parsing will check for "BriefDescription"
> field presence only for this.
> 
> If "BriefDescription" is present, then it is implied
> that the vendor has implemented their own custom event,
> and there is no fixup. Other fields are ignored.

if we are going this way, please use some new token,
this list is supposed to be human readable

thanks,
jirka

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 06/12/2017 13:36, Jiri Olsa wrote:
> On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
>> For some architectures (like arm64), there are architecture-
>> defined recommended events. Vendors may not be obliged to
>> follow the recommendation and may implement their own pmu
>> event for a specific event code.
>>
>> This patch adds support for parsing events from arch-defined
>> recommended JSONs, and then fixing up vendor events when
>> they have implemented these events as recommended.
>
> in the previous patch you added the vendor support, so
> you have arch|vendor|platform key for the event list
> and perf have the most current/local event list
>
> why would you need to fix it? if there's new event list,
> the table gets updated, perf is rebuilt.. I'm clearly
> missing something ;-)

The 2 patches are quite separate. In the first patch, I just added 
support for the vendor subdirectory.

So this patch is not related to rebuilding when adding a new event list 
or dependency checking.

Here we are trying to allow the vendor to just specify that an event is 
supported as standard in their platform, without duplicating all the 
standard event fields in their JSON. When processing the vendor JSONs, 
the jevents tool can figure which events are standard and create the 
proper event entries in the pmu events table, referencing the 
architecture JSON.

>
>> In the vendor JSON, to specify that the event is supported
>> according to the recommendation, only the event code is
>> added to the JSON entry - no other event elements need be
>> added, like below:
>> [
>>     {
>>         "EventCode": "0x40",
>>     },
>>
>> ]
>>
>> The pmu event parsing will check for "BriefDescription"
>> field presence only for this.
>>
>> If "BriefDescription" is present, then it is implied
>> that the vendor has implemented their own custom event,
>> and there is no fixup. Other fields are ignored.
>
> if we are going this way, please use some new token,
> this list is supposed to be human readable

A new token could work also, but it would be just a flag to mark the 
event "standard".

Ideally we could reference another entry in another JSON, like a 
pointer, but I don't think that this is possible with JSONs; not unless 
we introduce some elaborate custom scheme to allow JSONs to be 
cross-referenced.

Cheers,
John

>
> thanks,
> jirka
>
> .
>

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 06/12/2017 13:36, Jiri Olsa wrote:
> On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
>> For some architectures (like arm64), there are architecture-
>> defined recommended events. Vendors may not be obliged to
>> follow the recommendation and may implement their own pmu
>> event for a specific event code.
>>
>> This patch adds support for parsing events from arch-defined
>> recommended JSONs, and then fixing up vendor events when
>> they have implemented these events as recommended.
>
> in the previous patch you added the vendor support, so
> you have arch|vendor|platform key for the event list
> and perf have the most current/local event list
>
> why would you need to fix it? if there's new event list,
> the table gets updated, perf is rebuilt.. I'm clearly
> missing something ;-)

The 2 patches are quite separate. In the first patch, I just added 
support for the vendor subdirectory.

So this patch is not related to rebuilding when adding a new event list 
or dependency checking.

Here we are trying to allow the vendor to just specify that an event is 
supported as standard in their platform, without duplicating all the 
standard event fields in their JSON. When processing the vendor JSONs, 
the jevents tool can figure which events are standard and create the 
proper event entries in the pmu events table, referencing the 
architecture JSON.

>
>> In the vendor JSON, to specify that the event is supported
>> according to the recommendation, only the event code is
>> added to the JSON entry - no other event elements need be
>> added, like below:
>> [
>>     {
>>         "EventCode": "0x40",
>>     },
>>
>> ]
>>
>> The pmu event parsing will check for "BriefDescription"
>> field presence only for this.
>>
>> If "BriefDescription" is present, then it is implied
>> that the vendor has implemented their own custom event,
>> and there is no fixup. Other fields are ignored.
>
> if we are going this way, please use some new token,
> this list is supposed to be human readable

A new token could work also, but it would be just a flag to mark the 
event "standard".

Ideally we could reference another entry in another JSON, like a 
pointer, but I don't think that this is possible with JSONs; not unless 
we introduce some elaborate custom scheme to allow JSONs to be 
cross-referenced.

Cheers,
John

>
> thanks,
> jirka
>
> .
>

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Wed, Dec 06, 2017 at 03:20:14PM +0000, John Garry wrote:
> On 06/12/2017 13:36, Jiri Olsa wrote:
> > On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
> > > For some architectures (like arm64), there are architecture-
> > > defined recommended events. Vendors may not be obliged to
> > > follow the recommendation and may implement their own pmu
> > > event for a specific event code.
> > > 
> > > This patch adds support for parsing events from arch-defined
> > > recommended JSONs, and then fixing up vendor events when
> > > they have implemented these events as recommended.
> > 
> > in the previous patch you added the vendor support, so
> > you have arch|vendor|platform key for the event list
> > and perf have the most current/local event list
> > 
> > why would you need to fix it? if there's new event list,
> > the table gets updated, perf is rebuilt.. I'm clearly
> > missing something ;-)
> 
> The 2 patches are quite separate. In the first patch, I just added support
> for the vendor subdirectory.
> 
> So this patch is not related to rebuilding when adding a new event list or
> dependency checking.
> 
> Here we are trying to allow the vendor to just specify that an event is
> supported as standard in their platform, without duplicating all the
> standard event fields in their JSON. When processing the vendor JSONs, the
> jevents tool can figure which events are standard and create the proper
> event entries in the pmu events table, referencing the architecture JSON.

I think we should keep this simple and mangle this with some pointer logic

now you have arch/vendor/platform directory structure.. why don't
you add events for every such directory? I understand there will
be duplications, but we already have them for other archs and it's
not big deal:

	[jolsa@krava perf]$ grep -r L2_RQSTS.DEMAND_DATA_RD_MISS pmu-events/arch/*
	pmu-events/arch/x86/broadwell/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
	pmu-events/arch/x86/haswell/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
	pmu-events/arch/x86/broadwellde/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
	pmu-events/arch/x86/haswellx/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
	pmu-events/arch/x86/skylake/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
	pmu-events/arch/x86/skylakex/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
	pmu-events/arch/x86/broadwellx/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",

thanks,
jirka

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Wed, Dec 06, 2017 at 03:20:14PM +0000, John Garry wrote:
> On 06/12/2017 13:36, Jiri Olsa wrote:
> > On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
> > > For some architectures (like arm64), there are architecture-
> > > defined recommended events. Vendors may not be obliged to
> > > follow the recommendation and may implement their own pmu
> > > event for a specific event code.
> > > 
> > > This patch adds support for parsing events from arch-defined
> > > recommended JSONs, and then fixing up vendor events when
> > > they have implemented these events as recommended.
> > 
> > in the previous patch you added the vendor support, so
> > you have arch|vendor|platform key for the event list
> > and perf have the most current/local event list
> > 
> > why would you need to fix it? if there's new event list,
> > the table gets updated, perf is rebuilt.. I'm clearly
> > missing something ;-)
> 
> The 2 patches are quite separate. In the first patch, I just added support
> for the vendor subdirectory.
> 
> So this patch is not related to rebuilding when adding a new event list or
> dependency checking.
> 
> Here we are trying to allow the vendor to just specify that an event is
> supported as standard in their platform, without duplicating all the
> standard event fields in their JSON. When processing the vendor JSONs, the
> jevents tool can figure which events are standard and create the proper
> event entries in the pmu events table, referencing the architecture JSON.

I think we should keep this simple and mangle this with some pointer logic

now you have arch/vendor/platform directory structure.. why don't
you add events for every such directory? I understand there will
be duplications, but we already have them for other archs and it's
not big deal:

	[jolsa at krava perf]$ grep -r L2_RQSTS.DEMAND_DATA_RD_MISS pmu-events/arch/*
	pmu-events/arch/x86/broadwell/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
	pmu-events/arch/x86/haswell/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
	pmu-events/arch/x86/broadwellde/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
	pmu-events/arch/x86/haswellx/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
	pmu-events/arch/x86/skylake/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
	pmu-events/arch/x86/skylakex/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
	pmu-events/arch/x86/broadwellx/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",

thanks,
jirka

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 08/12/2017 12:29, Jiri Olsa wrote:
> On Wed, Dec 06, 2017 at 03:20:14PM +0000, John Garry wrote:
>> On 06/12/2017 13:36, Jiri Olsa wrote:
>>> On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
>>>> For some architectures (like arm64), there are architecture-
>>>> defined recommended events. Vendors may not be obliged to
>>>> follow the recommendation and may implement their own pmu
>>>> event for a specific event code.
>>>>
>>>> This patch adds support for parsing events from arch-defined
>>>> recommended JSONs, and then fixing up vendor events when
>>>> they have implemented these events as recommended.
>>>
>>> in the previous patch you added the vendor support, so
>>> you have arch|vendor|platform key for the event list
>>> and perf have the most current/local event list
>>>
>>> why would you need to fix it? if there's new event list,
>>> the table gets updated, perf is rebuilt.. I'm clearly
>>> missing something ;-)
>>
>> The 2 patches are quite separate. In the first patch, I just added support
>> for the vendor subdirectory.
>>
>> So this patch is not related to rebuilding when adding a new event list or
>> dependency checking.
>>
>> Here we are trying to allow the vendor to just specify that an event is
>> supported as standard in their platform, without duplicating all the
>> standard event fields in their JSON. When processing the vendor JSONs, the
>> jevents tool can figure which events are standard and create the proper
>> event entries in the pmu events table, referencing the architecture JSON.
>

Hi jirka,

> I think we should keep this simple and mangle this with some pointer logic
>
> now you have arch/vendor/platform directory structure..

I'm glad that there seems to be no objection to this, as I feel that 
this was a problem.

why don't
> you add events for every such directory? I understand there will
> be duplications, but we already have them for other archs and it's
> not big deal:

The amount of duplication was the concern. As mentioned earlier, it 
would be anticipated that every vendor would implement these events as 
recommended, so a copy for every platform from every vendor. We're 
looking for a way to avoid this.

Actually having a scalable JSON standard format for pmu events, which 
allows us to define common events per architecture / vendor and 
reference them per platform JSON could be useful.

Here we're dealing with trade-off between duplication (simplicity) vs 
complexity (or over-engineering).

>
> 	[jolsa@krava perf]$ grep -r L2_RQSTS.DEMAND_DATA_RD_MISS pmu-events/arch/*
> 	pmu-events/arch/x86/broadwell/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
> 	pmu-events/arch/x86/haswell/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
> 	pmu-events/arch/x86/broadwellde/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
> 	pmu-events/arch/x86/haswellx/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
> 	pmu-events/arch/x86/skylake/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
> 	pmu-events/arch/x86/skylakex/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
> 	pmu-events/arch/x86/broadwellx/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
>
> thanks,
> jirka

Cheers,
John

>
> .
>

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 08/12/2017 12:29, Jiri Olsa wrote:
> On Wed, Dec 06, 2017 at 03:20:14PM +0000, John Garry wrote:
>> On 06/12/2017 13:36, Jiri Olsa wrote:
>>> On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
>>>> For some architectures (like arm64), there are architecture-
>>>> defined recommended events. Vendors may not be obliged to
>>>> follow the recommendation and may implement their own pmu
>>>> event for a specific event code.
>>>>
>>>> This patch adds support for parsing events from arch-defined
>>>> recommended JSONs, and then fixing up vendor events when
>>>> they have implemented these events as recommended.
>>>
>>> in the previous patch you added the vendor support, so
>>> you have arch|vendor|platform key for the event list
>>> and perf have the most current/local event list
>>>
>>> why would you need to fix it? if there's new event list,
>>> the table gets updated, perf is rebuilt.. I'm clearly
>>> missing something ;-)
>>
>> The 2 patches are quite separate. In the first patch, I just added support
>> for the vendor subdirectory.
>>
>> So this patch is not related to rebuilding when adding a new event list or
>> dependency checking.
>>
>> Here we are trying to allow the vendor to just specify that an event is
>> supported as standard in their platform, without duplicating all the
>> standard event fields in their JSON. When processing the vendor JSONs, the
>> jevents tool can figure which events are standard and create the proper
>> event entries in the pmu events table, referencing the architecture JSON.
>

Hi jirka,

> I think we should keep this simple and mangle this with some pointer logic
>
> now you have arch/vendor/platform directory structure..

I'm glad that there seems to be no objection to this, as I feel that 
this was a problem.

why don't
> you add events for every such directory? I understand there will
> be duplications, but we already have them for other archs and it's
> not big deal:

The amount of duplication was the concern. As mentioned earlier, it 
would be anticipated that every vendor would implement these events as 
recommended, so a copy for every platform from every vendor. We're 
looking for a way to avoid this.

Actually having a scalable JSON standard format for pmu events, which 
allows us to define common events per architecture / vendor and 
reference them per platform JSON could be useful.

Here we're dealing with trade-off between duplication (simplicity) vs 
complexity (or over-engineering).

>
> 	[jolsa at krava perf]$ grep -r L2_RQSTS.DEMAND_DATA_RD_MISS pmu-events/arch/*
> 	pmu-events/arch/x86/broadwell/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
> 	pmu-events/arch/x86/haswell/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
> 	pmu-events/arch/x86/broadwellde/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
> 	pmu-events/arch/x86/haswellx/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
> 	pmu-events/arch/x86/skylake/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
> 	pmu-events/arch/x86/skylakex/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
> 	pmu-events/arch/x86/broadwellx/cache.json:        "EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
>
> thanks,
> jirka

Cheers,
John

>
> .
>

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Fri, Dec 08, 2017 at 03:42:10PM +0000, John Garry wrote:
> On 08/12/2017 12:29, Jiri Olsa wrote:
> > On Wed, Dec 06, 2017 at 03:20:14PM +0000, John Garry wrote:
> > > On 06/12/2017 13:36, Jiri Olsa wrote:
> > > > On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
> > > > > For some architectures (like arm64), there are architecture-
> > > > > defined recommended events. Vendors may not be obliged to
> > > > > follow the recommendation and may implement their own pmu
> > > > > event for a specific event code.
> > > > > 
> > > > > This patch adds support for parsing events from arch-defined
> > > > > recommended JSONs, and then fixing up vendor events when
> > > > > they have implemented these events as recommended.
> > > > 
> > > > in the previous patch you added the vendor support, so
> > > > you have arch|vendor|platform key for the event list
> > > > and perf have the most current/local event list
> > > > 
> > > > why would you need to fix it? if there's new event list,
> > > > the table gets updated, perf is rebuilt.. I'm clearly
> > > > missing something ;-)
> > > 
> > > The 2 patches are quite separate. In the first patch, I just added support
> > > for the vendor subdirectory.
> > > 
> > > So this patch is not related to rebuilding when adding a new event list or
> > > dependency checking.
> > > 
> > > Here we are trying to allow the vendor to just specify that an event is
> > > supported as standard in their platform, without duplicating all the
> > > standard event fields in their JSON. When processing the vendor JSONs, the
> > > jevents tool can figure which events are standard and create the proper
> > > event entries in the pmu events table, referencing the architecture JSON.
> > 
> 
> Hi jirka,
> 
> > I think we should keep this simple and mangle this with some pointer logic

sry for confusion, of course it should have been '.. and NOT mangle..' ;-)

> > 
> > now you have arch/vendor/platform directory structure..
> 
> I'm glad that there seems to be no objection to this, as I feel that this
> was a problem.
> 
> why don't
> > you add events for every such directory? I understand there will
> > be duplications, but we already have them for other archs and it's
> > not big deal:
> 
> The amount of duplication was the concern. As mentioned earlier, it would be
> anticipated that every vendor would implement these events as recommended,
> so a copy for every platform from every vendor. We're looking for a way to
> avoid this.
> 
> Actually having a scalable JSON standard format for pmu events, which allows
> us to define common events per architecture / vendor and reference them per
> platform JSON could be useful.
> 
> Here we're dealing with trade-off between duplication (simplicity) vs
> complexity (or over-engineering).

understood, but as I said we already are ok with duplicates,
if it's reasonable size as is for x86 now..  how much amount
are we talking about for arm?

jirka

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Fri, Dec 08, 2017 at 03:42:10PM +0000, John Garry wrote:
> On 08/12/2017 12:29, Jiri Olsa wrote:
> > On Wed, Dec 06, 2017 at 03:20:14PM +0000, John Garry wrote:
> > > On 06/12/2017 13:36, Jiri Olsa wrote:
> > > > On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
> > > > > For some architectures (like arm64), there are architecture-
> > > > > defined recommended events. Vendors may not be obliged to
> > > > > follow the recommendation and may implement their own pmu
> > > > > event for a specific event code.
> > > > > 
> > > > > This patch adds support for parsing events from arch-defined
> > > > > recommended JSONs, and then fixing up vendor events when
> > > > > they have implemented these events as recommended.
> > > > 
> > > > in the previous patch you added the vendor support, so
> > > > you have arch|vendor|platform key for the event list
> > > > and perf have the most current/local event list
> > > > 
> > > > why would you need to fix it? if there's new event list,
> > > > the table gets updated, perf is rebuilt.. I'm clearly
> > > > missing something ;-)
> > > 
> > > The 2 patches are quite separate. In the first patch, I just added support
> > > for the vendor subdirectory.
> > > 
> > > So this patch is not related to rebuilding when adding a new event list or
> > > dependency checking.
> > > 
> > > Here we are trying to allow the vendor to just specify that an event is
> > > supported as standard in their platform, without duplicating all the
> > > standard event fields in their JSON. When processing the vendor JSONs, the
> > > jevents tool can figure which events are standard and create the proper
> > > event entries in the pmu events table, referencing the architecture JSON.
> > 
> 
> Hi jirka,
> 
> > I think we should keep this simple and mangle this with some pointer logic

sry for confusion, of course it should have been '.. and NOT mangle..' ;-)

> > 
> > now you have arch/vendor/platform directory structure..
> 
> I'm glad that there seems to be no objection to this, as I feel that this
> was a problem.
> 
> why don't
> > you add events for every such directory? I understand there will
> > be duplications, but we already have them for other archs and it's
> > not big deal:
> 
> The amount of duplication was the concern. As mentioned earlier, it would be
> anticipated that every vendor would implement these events as recommended,
> so a copy for every platform from every vendor. We're looking for a way to
> avoid this.
> 
> Actually having a scalable JSON standard format for pmu events, which allows
> us to define common events per architecture / vendor and reference them per
> platform JSON could be useful.
> 
> Here we're dealing with trade-off between duplication (simplicity) vs
> complexity (or over-engineering).

understood, but as I said we already are ok with duplicates,
if it's reasonable size as is for x86 now..  how much amount
are we talking about for arm?

jirka

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 09/12/2017 07:31, Jiri Olsa wrote:
> On Fri, Dec 08, 2017 at 03:42:10PM +0000, John Garry wrote:
>> On 08/12/2017 12:29, Jiri Olsa wrote:
>>> On Wed, Dec 06, 2017 at 03:20:14PM +0000, John Garry wrote:
>>>> On 06/12/2017 13:36, Jiri Olsa wrote:
>>>>> On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
>>>>>> For some architectures (like arm64), there are architecture-
>>>>>> defined recommended events. Vendors may not be obliged to
>>>>>> follow the recommendation and may implement their own pmu
>>>>>> event for a specific event code.
>>>>>>
>>>>>> This patch adds support for parsing events from arch-defined
>>>>>> recommended JSONs, and then fixing up vendor events when
>>>>>> they have implemented these events as recommended.
>>>>>
>>>>> in the previous patch you added the vendor support, so
>>>>> you have arch|vendor|platform key for the event list
>>>>> and perf have the most current/local event list
>>>>>
>>>>> why would you need to fix it? if there's new event list,
>>>>> the table gets updated, perf is rebuilt.. I'm clearly
>>>>> missing something ;-)
>>>>
>>>> The 2 patches are quite separate. In the first patch, I just added support
>>>> for the vendor subdirectory.
>>>>
>>>> So this patch is not related to rebuilding when adding a new event list or
>>>> dependency checking.
>>>>
>>>> Here we are trying to allow the vendor to just specify that an event is
>>>> supported as standard in their platform, without duplicating all the
>>>> standard event fields in their JSON. When processing the vendor JSONs, the
>>>> jevents tool can figure which events are standard and create the proper
>>>> event entries in the pmu events table, referencing the architecture JSON.
>>>
>>
>> Hi jirka,
>>
>>> I think we should keep this simple and mangle this with some pointer logic
>
> sry for confusion, of course it should have been '.. and NOT mangle..' ;-)
>
>>>
>>> now you have arch/vendor/platform directory structure..
>>
>> I'm glad that there seems to be no objection to this, as I feel that this
>> was a problem.
>>
>> why don't
>>> you add events for every such directory? I understand there will
>>> be duplications, but we already have them for other archs and it's
>>> not big deal:
>>
>> The amount of duplication was the concern. As mentioned earlier, it would be
>> anticipated that every vendor would implement these events as recommended,
>> so a copy for every platform from every vendor. We're looking for a way to
>> avoid this.
>>
>> Actually having a scalable JSON standard format for pmu events, which allows
>> us to define common events per architecture / vendor and reference them per
>> platform JSON could be useful.
>>
>> Here we're dealing with trade-off between duplication (simplicity) vs
>> complexity (or over-engineering).
>
> understood, but as I said we already are ok with duplicates,
> if it's reasonable size as is for x86 now..  how much amount
> are we talking about for arm?
>

Hi jirka,

These JSONs would only apply to vendors which have custom ARMv8 
implementations. If you check the ARMv8 ARM, there's 10 such companies 
recorded as ARMv8 implementators.

So this means that in the future we could have tens to hundreds of JSONs 
for arm64, all with these duplicated events.

At this point I'll ask Will Deacon to share his thoughts, as he 
originally requested this feature.

Thanks,
John

> jirka
>
> .
>

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 09/12/2017 07:31, Jiri Olsa wrote:
> On Fri, Dec 08, 2017 at 03:42:10PM +0000, John Garry wrote:
>> On 08/12/2017 12:29, Jiri Olsa wrote:
>>> On Wed, Dec 06, 2017 at 03:20:14PM +0000, John Garry wrote:
>>>> On 06/12/2017 13:36, Jiri Olsa wrote:
>>>>> On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
>>>>>> For some architectures (like arm64), there are architecture-
>>>>>> defined recommended events. Vendors may not be obliged to
>>>>>> follow the recommendation and may implement their own pmu
>>>>>> event for a specific event code.
>>>>>>
>>>>>> This patch adds support for parsing events from arch-defined
>>>>>> recommended JSONs, and then fixing up vendor events when
>>>>>> they have implemented these events as recommended.
>>>>>
>>>>> in the previous patch you added the vendor support, so
>>>>> you have arch|vendor|platform key for the event list
>>>>> and perf have the most current/local event list
>>>>>
>>>>> why would you need to fix it? if there's new event list,
>>>>> the table gets updated, perf is rebuilt.. I'm clearly
>>>>> missing something ;-)
>>>>
>>>> The 2 patches are quite separate. In the first patch, I just added support
>>>> for the vendor subdirectory.
>>>>
>>>> So this patch is not related to rebuilding when adding a new event list or
>>>> dependency checking.
>>>>
>>>> Here we are trying to allow the vendor to just specify that an event is
>>>> supported as standard in their platform, without duplicating all the
>>>> standard event fields in their JSON. When processing the vendor JSONs, the
>>>> jevents tool can figure which events are standard and create the proper
>>>> event entries in the pmu events table, referencing the architecture JSON.
>>>
>>
>> Hi jirka,
>>
>>> I think we should keep this simple and mangle this with some pointer logic
>
> sry for confusion, of course it should have been '.. and NOT mangle..' ;-)
>
>>>
>>> now you have arch/vendor/platform directory structure..
>>
>> I'm glad that there seems to be no objection to this, as I feel that this
>> was a problem.
>>
>> why don't
>>> you add events for every such directory? I understand there will
>>> be duplications, but we already have them for other archs and it's
>>> not big deal:
>>
>> The amount of duplication was the concern. As mentioned earlier, it would be
>> anticipated that every vendor would implement these events as recommended,
>> so a copy for every platform from every vendor. We're looking for a way to
>> avoid this.
>>
>> Actually having a scalable JSON standard format for pmu events, which allows
>> us to define common events per architecture / vendor and reference them per
>> platform JSON could be useful.
>>
>> Here we're dealing with trade-off between duplication (simplicity) vs
>> complexity (or over-engineering).
>
> understood, but as I said we already are ok with duplicates,
> if it's reasonable size as is for x86 now..  how much amount
> are we talking about for arm?
>

Hi jirka,

These JSONs would only apply to vendors which have custom ARMv8 
implementations. If you check the ARMv8 ARM, there's 10 such companies 
recorded as ARMv8 implementators.

So this means that in the future we could have tens to hundreds of JSONs 
for arm64, all with these duplicated events.

At this point I'll ask Will Deacon to share his thoughts, as he 
originally requested this feature.

Thanks,
John

> jirka
>
> .
>

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

>> Actually having a scalable JSON standard format for pmu events, which allows
>> us to define common events per architecture / vendor and reference them per
>> platform JSON could be useful.
>>
>> Here we're dealing with trade-off between duplication (simplicity) vs
>> complexity (or over-engineering).
>
> understood, but as I said we already are ok with duplicates,
> if it's reasonable size as is for x86 now..  how much amount
> are we talking about for arm?
>

Hi Jirka,

When you say reasonable size for x86, I ran a string duplication finder 
on the x86 JSONs and the results show a huge amount of duplication. 
Please check this:
https://gist.githubusercontent.com/johnpgarry/68bc87e823ae2ce0f7b475b4e55e5795/raw/f4cea138999d8b34151b9586d733592e01774d7a/x86%2520JSON%2520duplication

Extract:
"Found a 65 line (311 tokens) duplication in the following files:
Starting at line 100 of 
/linux/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
Starting at line 100 of 
/linux/tools/perf/pmu-events/arch/x86/ivytown/ivt-metrics.json
Starting at line 100 of 
/linux/tools/perf/pmu-events/arch/x86/broadwell/bdw-metrics.json
Starting at line 100 of 
/linux/tools/perf/pmu-events/arch/x86/skylakex/skx-metrics.json
Starting at line 76 of 
/linux/tools/perf/pmu-events/arch/x86/jaketown/jkt-metrics.json
Starting at line 100 of 
/linux/tools/perf/pmu-events/arch/x86/skylake/skl-metrics.json
Starting at line 76 of 
/linux/tools/perf/pmu-events/arch/x86/sandybridge/snb-metrics.json
Starting at line 100 of 
/linux/tools/perf/pmu-events/arch/x86/broadwellx/bdx-metrics.json"

Won't this all potentially have a big maintainence cost?

For example, I saw multiple JSON update patches which look identical:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/tools/perf/pmu-events/arch/x86?h=v4.15-rc3&id=7347bba5552f479d4292ffd008d18d41a965f021

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/tools/perf/pmu-events/arch/x86?h=v4.15-rc3&id=984d91f4c62f64026cbfef51f609971025934cec

I just don't know how this schema scales with more archs and more 
platforms supported. It's just early days now...

Regards,
John

> jirka
>
> .
>

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

>> Actually having a scalable JSON standard format for pmu events, which allows
>> us to define common events per architecture / vendor and reference them per
>> platform JSON could be useful.
>>
>> Here we're dealing with trade-off between duplication (simplicity) vs
>> complexity (or over-engineering).
>
> understood, but as I said we already are ok with duplicates,
> if it's reasonable size as is for x86 now..  how much amount
> are we talking about for arm?
>

Hi Jirka,

When you say reasonable size for x86, I ran a string duplication finder 
on the x86 JSONs and the results show a huge amount of duplication. 
Please check this:
https://gist.githubusercontent.com/johnpgarry/68bc87e823ae2ce0f7b475b4e55e5795/raw/f4cea138999d8b34151b9586d733592e01774d7a/x86%2520JSON%2520duplication

Extract:
"Found a 65 line (311 tokens) duplication in the following files:
Starting at line 100 of 
/linux/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
Starting at line 100 of 
/linux/tools/perf/pmu-events/arch/x86/ivytown/ivt-metrics.json
Starting at line 100 of 
/linux/tools/perf/pmu-events/arch/x86/broadwell/bdw-metrics.json
Starting at line 100 of 
/linux/tools/perf/pmu-events/arch/x86/skylakex/skx-metrics.json
Starting at line 76 of 
/linux/tools/perf/pmu-events/arch/x86/jaketown/jkt-metrics.json
Starting at line 100 of 
/linux/tools/perf/pmu-events/arch/x86/skylake/skl-metrics.json
Starting at line 76 of 
/linux/tools/perf/pmu-events/arch/x86/sandybridge/snb-metrics.json
Starting at line 100 of 
/linux/tools/perf/pmu-events/arch/x86/broadwellx/bdx-metrics.json"

Won't this all potentially have a big maintainence cost?

For example, I saw multiple JSON update patches which look identical:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/tools/perf/pmu-events/arch/x86?h=v4.15-rc3&id=7347bba5552f479d4292ffd008d18d41a965f021

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/tools/perf/pmu-events/arch/x86?h=v4.15-rc3&id=984d91f4c62f64026cbfef51f609971025934cec

I just don't know how this schema scales with more archs and more 
platforms supported. It's just early days now...

Regards,
John

> jirka
>
> .
>

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Andi Kleen]

> Won't this all potentially have a big maintainence cost?

No. It's all auto generated.

The only cost is slightly bigger binary size.

I would hope your event files are auto generated too.

> I just don't know how this schema scales with more archs and more platforms
> supported. It's just early days now...

Only perf will get slightly bigger, but memory is not exactly expensive.

In fact, the extra memory won't even be faulted in if it's not used,
so it's only disk space.

-Andi

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Andi Kleen]

> Won't this all potentially have a big maintainence cost?

No. It's all auto generated.

The only cost is slightly bigger binary size.

I would hope your event files are auto generated too.

> I just don't know how this schema scales with more archs and more platforms
> supported. It's just early days now...

Only perf will get slightly bigger, but memory is not exactly expensive.

In fact, the extra memory won't even be faulted in if it's not used,
so it's only disk space.

-Andi

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 16/12/2017 18:47, Andi Kleen wrote:

Hi Andi,

>> Won't this all potentially have a big maintainence cost?
>
> No. It's all auto generated.
>
> The only cost is slightly bigger binary size.
>
> I would hope your event files are auto generated too.
>

No, they're not - we are just manually transcribing the event data from 
the architecture reference manual (for arch-defined events) or internal 
text document (for custom events).

Can you describe how you autogenerate the JSONs? Do you have some 
internal proprietary HW file format describing events, with files 
supplied from HW designer, which you can just translate into a JSON? 
Would the files support deferencing events to improve scalability?

>> I just don't know how this schema scales with more archs and more platforms
>> supported. It's just early days now...
>
> Only perf will get slightly bigger, but memory is not exactly expensive.
>
> In fact, the extra memory won't even be faulted in if it's not used,
> so it's only disk space.

Much appreciated,
John

>
> -Andi
>
> .
>

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 16/12/2017 18:47, Andi Kleen wrote:

Hi Andi,

>> Won't this all potentially have a big maintainence cost?
>
> No. It's all auto generated.
>
> The only cost is slightly bigger binary size.
>
> I would hope your event files are auto generated too.
>

No, they're not - we are just manually transcribing the event data from 
the architecture reference manual (for arch-defined events) or internal 
text document (for custom events).

Can you describe how you autogenerate the JSONs? Do you have some 
internal proprietary HW file format describing events, with files 
supplied from HW designer, which you can just translate into a JSON? 
Would the files support deferencing events to improve scalability?

>> I just don't know how this schema scales with more archs and more platforms
>> supported. It's just early days now...
>
> Only perf will get slightly bigger, but memory is not exactly expensive.
>
> In fact, the extra memory won't even be faulted in if it's not used,
> so it's only disk space.

Much appreciated,
John

>
> -Andi
>
> .
>

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Andi Kleen]

[-- Attachment #1: Type: text/plain, Size: 847 bytes --]

> Can you describe how you autogenerate the JSONs? Do you have some internal
> proprietary HW file format describing events, with files supplied from HW
> designer, which you can just translate into a JSON? Would the files support
> deferencing events to improve scalability?

For Intel JSON is an official format, which is maintained for each CPU.
It is automatically generated from an internal database
https://download.01.org/perfmon/

I have some python scripts to convert these Intel JSONs into the perf
format (which has some additional headers, and is split into
different categories, and add metrics).  

They have some Intel specifics, so may not be useful for you. 

There's no support for dereference, each CPU gets its own unique file.

But you could do the a merge simply with the attached script which merges
two JSON files. 

-Andi

[-- Attachment #2: merge-json --]
[-- Type: text/plain, Size: 282 bytes --]

#!/usr/bin/python
# merge json event files
# merge-json file1.json file2... > merged.json
import sys
import json

all = []

for fn in sys.argv[1:]:
    jf = json.load(open(fn))
    for n in jf:
	all.append(n)

print json.dumps(all, sort_keys=True, indent=4, separators=(',', ': '))

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Andi Kleen]

> Can you describe how you autogenerate the JSONs? Do you have some internal
> proprietary HW file format describing events, with files supplied from HW
> designer, which you can just translate into a JSON? Would the files support
> deferencing events to improve scalability?

For Intel JSON is an official format, which is maintained for each CPU.
It is automatically generated from an internal database
https://download.01.org/perfmon/

I have some python scripts to convert these Intel JSONs into the perf
format (which has some additional headers, and is split into
different categories, and add metrics).  

They have some Intel specifics, so may not be useful for you. 

There's no support for dereference, each CPU gets its own unique file.

But you could do the a merge simply with the attached script which merges
two JSON files. 

-Andi
-------------- next part --------------
#!/usr/bin/python
# merge json event files
# merge-json file1.json file2... > merged.json
import sys
import json

all = []

for fn in sys.argv[1:]:
    jf = json.load(open(fn))
    for n in jf:
	all.append(n)

print json.dumps(all, sort_keys=True, indent=4, separators=(',', ': '))

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 02/01/2018 17:48, Andi Kleen wrote:
>> Can you describe how you autogenerate the JSONs? Do you have some internal
>> proprietary HW file format describing events, with files supplied from HW
>> designer, which you can just translate into a JSON? Would the files support
>> deferencing events to improve scalability?
>
> For Intel JSON is an official format, which is maintained for each CPU.
> It is automatically generated from an internal database
> https://download.01.org/perfmon/
>
> I have some python scripts to convert these Intel JSONs into the perf
> format (which has some additional headers, and is split into
> different categories, and add metrics).

OK, understood.

Unfortunately I could not see such a database being maintained for ARM 
implementors.

>
> They have some Intel specifics, so may not be useful for you.
>
> There's no support for dereference, each CPU gets its own unique file.

Right.

>
> But you could do the a merge simply with the attached script which merges
> two JSON files.

I assume that you're talking about simply merging the micro architecture 
and the platform specific event JSONS at build time.

If yes, this would not work for us:
- the microarchitecture JSON would contain definitions of all events, 
but there is no architectural method to check if they are implemented
- we need to deal with scenario of non-standard event implementations

But I could update the script to deal with this and add to the build 
(Jirka looked to be ok with the same in jevents, albeit a few caveats).

All the best,
John

>
> -Andi
>

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 02/01/2018 17:48, Andi Kleen wrote:
>> Can you describe how you autogenerate the JSONs? Do you have some internal
>> proprietary HW file format describing events, with files supplied from HW
>> designer, which you can just translate into a JSON? Would the files support
>> deferencing events to improve scalability?
>
> For Intel JSON is an official format, which is maintained for each CPU.
> It is automatically generated from an internal database
> https://download.01.org/perfmon/
>
> I have some python scripts to convert these Intel JSONs into the perf
> format (which has some additional headers, and is split into
> different categories, and add metrics).

OK, understood.

Unfortunately I could not see such a database being maintained for ARM 
implementors.

>
> They have some Intel specifics, so may not be useful for you.
>
> There's no support for dereference, each CPU gets its own unique file.

Right.

>
> But you could do the a merge simply with the attached script which merges
> two JSON files.

I assume that you're talking about simply merging the micro architecture 
and the platform specific event JSONS at build time.

If yes, this would not work for us:
- the microarchitecture JSON would contain definitions of all events, 
but there is no architectural method to check if they are implemented
- we need to deal with scenario of non-standard event implementations

But I could update the script to deal with this and add to the build 
(Jirka looked to be ok with the same in jevents, albeit a few caveats).

All the best,
John

>
> -Andi
>

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Fri, Dec 15, 2017 at 11:22:50AM +0000, John Garry wrote:
> > > Actually having a scalable JSON standard format for pmu events, which allows
> > > us to define common events per architecture / vendor and reference them per
> > > platform JSON could be useful.
> > > 
> > > Here we're dealing with trade-off between duplication (simplicity) vs
> > > complexity (or over-engineering).
> > 
> > understood, but as I said we already are ok with duplicates,
> > if it's reasonable size as is for x86 now..  how much amount
> > are we talking about for arm?
> > 
> 
> Hi Jirka,
> 
> When you say reasonable size for x86, I ran a string duplication finder on
> the x86 JSONs and the results show a huge amount of duplication. Please
> check this:
> https://gist.githubusercontent.com/johnpgarry/68bc87e823ae2ce0f7b475b4e55e5795/raw/f4cea138999d8b34151b9586d733592e01774d7a/x86%2520JSON%2520duplication
> 
> Extract:
> "Found a 65 line (311 tokens) duplication in the following files:
> Starting at line 100 of
> /linux/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
> Starting at line 100 of
> /linux/tools/perf/pmu-events/arch/x86/ivytown/ivt-metrics.json
> Starting at line 100 of
> /linux/tools/perf/pmu-events/arch/x86/broadwell/bdw-metrics.json
> Starting at line 100 of
> /linux/tools/perf/pmu-events/arch/x86/skylakex/skx-metrics.json
> Starting at line 76 of
> /linux/tools/perf/pmu-events/arch/x86/jaketown/jkt-metrics.json
> Starting at line 100 of
> /linux/tools/perf/pmu-events/arch/x86/skylake/skl-metrics.json
> Starting at line 76 of
> /linux/tools/perf/pmu-events/arch/x86/sandybridge/snb-metrics.json
> Starting at line 100 of
> /linux/tools/perf/pmu-events/arch/x86/broadwellx/bdx-metrics.json"
> 
> Won't this all potentially have a big maintainence cost?

as Andi said it's mostly just the disk space,
which is not big deal

I'm not doing JSON file updates, but I think having
simple single dir for platform/cpu could save us some
confusion in future

however I won't oppose if you want to add this logic,
but please:
  - use the list_head ;-)
  - leave the process_one_file function simple
    and separate the level0 processing
  - you are using 'EventCode' as an unique ID to find
    the base, but it's not unique for x86, you'll need
    to add some other ID scheme that fits to all archs

thanks,
jirka

> 
> For example, I saw multiple JSON update patches which look identical:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/tools/perf/pmu-events/arch/x86?h=v4.15-rc3&id=7347bba5552f479d4292ffd008d18d41a965f021
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/tools/perf/pmu-events/arch/x86?h=v4.15-rc3&id=984d91f4c62f64026cbfef51f609971025934cec
> 
> I just don't know how this schema scales with more archs and more platforms
> supported. It's just early days now...
> 
> Regards,
> John
> 
> > jirka
> > 
> > .
> > 
> 
> 

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Fri, Dec 15, 2017 at 11:22:50AM +0000, John Garry wrote:
> > > Actually having a scalable JSON standard format for pmu events, which allows
> > > us to define common events per architecture / vendor and reference them per
> > > platform JSON could be useful.
> > > 
> > > Here we're dealing with trade-off between duplication (simplicity) vs
> > > complexity (or over-engineering).
> > 
> > understood, but as I said we already are ok with duplicates,
> > if it's reasonable size as is for x86 now..  how much amount
> > are we talking about for arm?
> > 
> 
> Hi Jirka,
> 
> When you say reasonable size for x86, I ran a string duplication finder on
> the x86 JSONs and the results show a huge amount of duplication. Please
> check this:
> https://gist.githubusercontent.com/johnpgarry/68bc87e823ae2ce0f7b475b4e55e5795/raw/f4cea138999d8b34151b9586d733592e01774d7a/x86%2520JSON%2520duplication
> 
> Extract:
> "Found a 65 line (311 tokens) duplication in the following files:
> Starting at line 100 of
> /linux/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
> Starting at line 100 of
> /linux/tools/perf/pmu-events/arch/x86/ivytown/ivt-metrics.json
> Starting at line 100 of
> /linux/tools/perf/pmu-events/arch/x86/broadwell/bdw-metrics.json
> Starting at line 100 of
> /linux/tools/perf/pmu-events/arch/x86/skylakex/skx-metrics.json
> Starting at line 76 of
> /linux/tools/perf/pmu-events/arch/x86/jaketown/jkt-metrics.json
> Starting at line 100 of
> /linux/tools/perf/pmu-events/arch/x86/skylake/skl-metrics.json
> Starting at line 76 of
> /linux/tools/perf/pmu-events/arch/x86/sandybridge/snb-metrics.json
> Starting at line 100 of
> /linux/tools/perf/pmu-events/arch/x86/broadwellx/bdx-metrics.json"
> 
> Won't this all potentially have a big maintainence cost?

as Andi said it's mostly just the disk space,
which is not big deal

I'm not doing JSON file updates, but I think having
simple single dir for platform/cpu could save us some
confusion in future

however I won't oppose if you want to add this logic,
but please:
  - use the list_head ;-)
  - leave the process_one_file function simple
    and separate the level0 processing
  - you are using 'EventCode' as an unique ID to find
    the base, but it's not unique for x86, you'll need
    to add some other ID scheme that fits to all archs

thanks,
jirka

> 
> For example, I saw multiple JSON update patches which look identical:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/tools/perf/pmu-events/arch/x86?h=v4.15-rc3&id=7347bba5552f479d4292ffd008d18d41a965f021
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/tools/perf/pmu-events/arch/x86?h=v4.15-rc3&id=984d91f4c62f64026cbfef51f609971025934cec
> 
> I just don't know how this schema scales with more archs and more platforms
> supported. It's just early days now...
> 
> Regards,
> John
> 
> > jirka
> > 
> > .
> > 
> 
> 

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 21/12/2017 19:39, Jiri Olsa wrote:
>> Hi Jirka,
>> >
>> > When you say reasonable size for x86, I ran a string duplication finder on
>> > the x86 JSONs and the results show a huge amount of duplication. Please
>> > check this:
>> > https://gist.githubusercontent.com/johnpgarry/68bc87e823ae2ce0f7b475b4e55e5795/raw/f4cea138999d8b34151b9586d733592e01774d7a/x86%2520JSON%2520duplication
>> >
>> > Extract:
>> > "Found a 65 line (311 tokens) duplication in the following files:
>> > Starting at line 100 of
>> > /linux/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
>> > Starting at line 100 of
>> > /linux/tools/perf/pmu-events/arch/x86/ivytown/ivt-metrics.json
>> > Starting at line 100 of
>> > /linux/tools/perf/pmu-events/arch/x86/broadwell/bdw-metrics.json
>> > Starting at line 100 of
>> > /linux/tools/perf/pmu-events/arch/x86/skylakex/skx-metrics.json
>> > Starting at line 76 of
>> > /linux/tools/perf/pmu-events/arch/x86/jaketown/jkt-metrics.json
>> > Starting at line 100 of
>> > /linux/tools/perf/pmu-events/arch/x86/skylake/skl-metrics.json
>> > Starting at line 76 of
>> > /linux/tools/perf/pmu-events/arch/x86/sandybridge/snb-metrics.json
>> > Starting at line 100 of
>> > /linux/tools/perf/pmu-events/arch/x86/broadwellx/bdx-metrics.json"
>> >


Hi Jirka,

Sorry for the slow reply.

>> > Won't this all potentially have a big maintainence cost?
> as Andi said it's mostly just the disk space,
> which is not big deal
>
> I'm not doing JSON file updates, but I think having
> simple single dir for platform/cpu could save us some
> confusion in future

Understood. But for ARM, which has very standardised architecture 
events, it is good to reduce this event duplication between platforms.

>
> however I won't oppose if you want to add this logic,
> but please:
>   - use the list_head ;-)

Of course

>   - leave the process_one_file function simple
>     and separate the level0 processing

ok, this is how it should look already, albeit a couple of 
process_one_file() modifications. I'll re-check this.

>   - you are using 'EventCode' as an unique ID to find
>     the base, but it's not unique for x86, you'll need
>     to add some other ID scheme that fits to all archs

Right, so you mentioned earlier using a new keyword token to identify 
whether we use the standard event, so we can go his way - ok?

I would also like to mention at this point why I did the event 
pre-processing in jevents, and not a separate script:
- current build does not transverse the arch tree
	- tree transversal for JSON processing is done in jevents
- a script would mean derived objects, which means:
	- makefile changes for derived objects
	- jevents would have to deal with derived objects
- jevents already has support for JSON processing

The advantage of using a script is that we keep the JSON processing in 
jevents simple.

All the best,
John

>
> thanks,
> jirka
>

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 21/12/2017 19:39, Jiri Olsa wrote:
>> Hi Jirka,
>> >
>> > When you say reasonable size for x86, I ran a string duplication finder on
>> > the x86 JSONs and the results show a huge amount of duplication. Please
>> > check this:
>> > https://gist.githubusercontent.com/johnpgarry/68bc87e823ae2ce0f7b475b4e55e5795/raw/f4cea138999d8b34151b9586d733592e01774d7a/x86%2520JSON%2520duplication
>> >
>> > Extract:
>> > "Found a 65 line (311 tokens) duplication in the following files:
>> > Starting at line 100 of
>> > /linux/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
>> > Starting at line 100 of
>> > /linux/tools/perf/pmu-events/arch/x86/ivytown/ivt-metrics.json
>> > Starting at line 100 of
>> > /linux/tools/perf/pmu-events/arch/x86/broadwell/bdw-metrics.json
>> > Starting at line 100 of
>> > /linux/tools/perf/pmu-events/arch/x86/skylakex/skx-metrics.json
>> > Starting at line 76 of
>> > /linux/tools/perf/pmu-events/arch/x86/jaketown/jkt-metrics.json
>> > Starting at line 100 of
>> > /linux/tools/perf/pmu-events/arch/x86/skylake/skl-metrics.json
>> > Starting at line 76 of
>> > /linux/tools/perf/pmu-events/arch/x86/sandybridge/snb-metrics.json
>> > Starting at line 100 of
>> > /linux/tools/perf/pmu-events/arch/x86/broadwellx/bdx-metrics.json"
>> >


Hi Jirka,

Sorry for the slow reply.

>> > Won't this all potentially have a big maintainence cost?
> as Andi said it's mostly just the disk space,
> which is not big deal
>
> I'm not doing JSON file updates, but I think having
> simple single dir for platform/cpu could save us some
> confusion in future

Understood. But for ARM, which has very standardised architecture 
events, it is good to reduce this event duplication between platforms.

>
> however I won't oppose if you want to add this logic,
> but please:
>   - use the list_head ;-)

Of course

>   - leave the process_one_file function simple
>     and separate the level0 processing

ok, this is how it should look already, albeit a couple of 
process_one_file() modifications. I'll re-check this.

>   - you are using 'EventCode' as an unique ID to find
>     the base, but it's not unique for x86, you'll need
>     to add some other ID scheme that fits to all archs

Right, so you mentioned earlier using a new keyword token to identify 
whether we use the standard event, so we can go his way - ok?

I would also like to mention at this point why I did the event 
pre-processing in jevents, and not a separate script:
- current build does not transverse the arch tree
	- tree transversal for JSON processing is done in jevents
- a script would mean derived objects, which means:
	- makefile changes for derived objects
	- jevents would have to deal with derived objects
- jevents already has support for JSON processing

The advantage of using a script is that we keep the JSON processing in 
jevents simple.

All the best,
John

>
> thanks,
> jirka
>

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Thu, Jan 04, 2018 at 05:17:56PM +0000, John Garry wrote:

SNIP

> 
> 
> Hi Jirka,
> 
> Sorry for the slow reply.

np, just got back from holidays anyway ;-)

> 
> > > > Won't this all potentially have a big maintainence cost?
> > as Andi said it's mostly just the disk space,
> > which is not big deal
> > 
> > I'm not doing JSON file updates, but I think having
> > simple single dir for platform/cpu could save us some
> > confusion in future
> 
> Understood. But for ARM, which has very standardised architecture events, it
> is good to reduce this event duplication between platforms.
> 
> > 
> > however I won't oppose if you want to add this logic,
> > but please:
> >   - use the list_head ;-)
> 
> Of course
> 
> >   - leave the process_one_file function simple
> >     and separate the level0 processing
> 
> ok, this is how it should look already, albeit a couple of
> process_one_file() modifications. I'll re-check this.
> 
> >   - you are using 'EventCode' as an unique ID to find
> >     the base, but it's not unique for x86, you'll need
> >     to add some other ID scheme that fits to all archs
> 
> Right, so you mentioned earlier using a new keyword token to identify
> whether we use the standard event, so we can go his way - ok?

yes, something like that

> I would also like to mention at this point why I did the event
> pre-processing in jevents, and not a separate script:
> - current build does not transverse the arch tree
> 	- tree transversal for JSON processing is done in jevents
> - a script would mean derived objects, which means:
> 	- makefile changes for derived objects
> 	- jevents would have to deal with derived objects
> - jevents already has support for JSON processing
> 
> The advantage of using a script is that we keep the JSON processing in
> jevents simple.

I don't mind the extra functionality in jevents as long as the current
one keeps on working and the new one works for all archs ;-)

thanks,
jirka

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Thu, Jan 04, 2018 at 05:17:56PM +0000, John Garry wrote:

SNIP

> 
> 
> Hi Jirka,
> 
> Sorry for the slow reply.

np, just got back from holidays anyway ;-)

> 
> > > > Won't this all potentially have a big maintainence cost?
> > as Andi said it's mostly just the disk space,
> > which is not big deal
> > 
> > I'm not doing JSON file updates, but I think having
> > simple single dir for platform/cpu could save us some
> > confusion in future
> 
> Understood. But for ARM, which has very standardised architecture events, it
> is good to reduce this event duplication between platforms.
> 
> > 
> > however I won't oppose if you want to add this logic,
> > but please:
> >   - use the list_head ;-)
> 
> Of course
> 
> >   - leave the process_one_file function simple
> >     and separate the level0 processing
> 
> ok, this is how it should look already, albeit a couple of
> process_one_file() modifications. I'll re-check this.
> 
> >   - you are using 'EventCode' as an unique ID to find
> >     the base, but it's not unique for x86, you'll need
> >     to add some other ID scheme that fits to all archs
> 
> Right, so you mentioned earlier using a new keyword token to identify
> whether we use the standard event, so we can go his way - ok?

yes, something like that

> I would also like to mention at this point why I did the event
> pre-processing in jevents, and not a separate script:
> - current build does not transverse the arch tree
> 	- tree transversal for JSON processing is done in jevents
> - a script would mean derived objects, which means:
> 	- makefile changes for derived objects
> 	- jevents would have to deal with derived objects
> - jevents already has support for JSON processing
> 
> The advantage of using a script is that we keep the JSON processing in
> jevents simple.

I don't mind the extra functionality in jevents as long as the current
one keeps on working and the new one works for all archs ;-)

thanks,
jirka

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:

SNIP

> ---
>  tools/perf/pmu-events/jevents.c | 215 ++++++++++++++++++++++++++++++++++++----
>  1 file changed, 198 insertions(+), 17 deletions(-)
> 
> diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
> index a0d489e..a820ed4 100644
> --- a/tools/perf/pmu-events/jevents.c
> +++ b/tools/perf/pmu-events/jevents.c
> @@ -42,6 +42,7 @@
>  #include <dirent.h>
>  #include <sys/time.h>			/* getrlimit */
>  #include <sys/resource.h>		/* getrlimit */
> +#include <sys/queue.h>
>  #include <ftw.h>
>  #include <sys/stat.h>
>  #include "jsmn.h"
> @@ -366,6 +367,94 @@ static int print_events_table_entry(void *data, char *name, char *event,
>  	return 0;
>  }
>  
> +struct event_struct {
> +	char *name;
> +	char *event;
> +	char *desc;
> +	char *long_desc;
> +	char *pmu;
> +	char *unit;
> +	char *perpkg;
> +	char *metric_expr;
> +	char *metric_name;
> +	char *metric_group;
> +	LIST_ENTRY(event_struct) list;

is there any reason you don't use the 'struct list_head' ?
I dont think we want another thingie involved for lists

jirka

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:

SNIP

> ---
>  tools/perf/pmu-events/jevents.c | 215 ++++++++++++++++++++++++++++++++++++----
>  1 file changed, 198 insertions(+), 17 deletions(-)
> 
> diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
> index a0d489e..a820ed4 100644
> --- a/tools/perf/pmu-events/jevents.c
> +++ b/tools/perf/pmu-events/jevents.c
> @@ -42,6 +42,7 @@
>  #include <dirent.h>
>  #include <sys/time.h>			/* getrlimit */
>  #include <sys/resource.h>		/* getrlimit */
> +#include <sys/queue.h>
>  #include <ftw.h>
>  #include <sys/stat.h>
>  #include "jsmn.h"
> @@ -366,6 +367,94 @@ static int print_events_table_entry(void *data, char *name, char *event,
>  	return 0;
>  }
>  
> +struct event_struct {
> +	char *name;
> +	char *event;
> +	char *desc;
> +	char *long_desc;
> +	char *pmu;
> +	char *unit;
> +	char *perpkg;
> +	char *metric_expr;
> +	char *metric_name;
> +	char *metric_group;
> +	LIST_ENTRY(event_struct) list;

is there any reason you don't use the 'struct list_head' ?
I dont think we want another thingie involved for lists

jirka

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 06/12/2017 13:37, Jiri Olsa wrote:
> On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
>
> SNIP
>
>> ---
>>  tools/perf/pmu-events/jevents.c | 215 ++++++++++++++++++++++++++++++++++++----
>>  1 file changed, 198 insertions(+), 17 deletions(-)
>>
>> diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
>> index a0d489e..a820ed4 100644
>> --- a/tools/perf/pmu-events/jevents.c
>> +++ b/tools/perf/pmu-events/jevents.c
>> @@ -42,6 +42,7 @@
>>  #include <dirent.h>
>>  #include <sys/time.h>			/* getrlimit */
>>  #include <sys/resource.h>		/* getrlimit */
>> +#include <sys/queue.h>
>>  #include <ftw.h>
>>  #include <sys/stat.h>
>>  #include "jsmn.h"
>> @@ -366,6 +367,94 @@ static int print_events_table_entry(void *data, char *name, char *event,
>>  	return 0;
>>  }
>>
>> +struct event_struct {
>> +	char *name;
>> +	char *event;
>> +	char *desc;
>> +	char *long_desc;
>> +	char *pmu;
>> +	char *unit;
>> +	char *perpkg;
>> +	char *metric_expr;
>> +	char *metric_name;
>> +	char *metric_group;
>> +	LIST_ENTRY(event_struct) list;
>
> is there any reason you don't use the 'struct list_head' ?
> I dont think we want another thingie involved for lists

Hi jirka,

The linux kernel headers are not used for jevents tool. I would rather 
use them if possible...

Thanks,
John

>
> jirka
>
> .
>

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 06/12/2017 13:37, Jiri Olsa wrote:
> On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
>
> SNIP
>
>> ---
>>  tools/perf/pmu-events/jevents.c | 215 ++++++++++++++++++++++++++++++++++++----
>>  1 file changed, 198 insertions(+), 17 deletions(-)
>>
>> diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
>> index a0d489e..a820ed4 100644
>> --- a/tools/perf/pmu-events/jevents.c
>> +++ b/tools/perf/pmu-events/jevents.c
>> @@ -42,6 +42,7 @@
>>  #include <dirent.h>
>>  #include <sys/time.h>			/* getrlimit */
>>  #include <sys/resource.h>		/* getrlimit */
>> +#include <sys/queue.h>
>>  #include <ftw.h>
>>  #include <sys/stat.h>
>>  #include "jsmn.h"
>> @@ -366,6 +367,94 @@ static int print_events_table_entry(void *data, char *name, char *event,
>>  	return 0;
>>  }
>>
>> +struct event_struct {
>> +	char *name;
>> +	char *event;
>> +	char *desc;
>> +	char *long_desc;
>> +	char *pmu;
>> +	char *unit;
>> +	char *perpkg;
>> +	char *metric_expr;
>> +	char *metric_name;
>> +	char *metric_group;
>> +	LIST_ENTRY(event_struct) list;
>
> is there any reason you don't use the 'struct list_head' ?
> I dont think we want another thingie involved for lists

Hi jirka,

The linux kernel headers are not used for jevents tool. I would rather 
use them if possible...

Thanks,
John

>
> jirka
>
> .
>

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Wed, Dec 06, 2017 at 02:40:10PM +0000, John Garry wrote:
> On 06/12/2017 13:37, Jiri Olsa wrote:
> > On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
> > 
> > SNIP
> > 
> > > ---
> > >  tools/perf/pmu-events/jevents.c | 215 ++++++++++++++++++++++++++++++++++++----
> > >  1 file changed, 198 insertions(+), 17 deletions(-)
> > > 
> > > diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
> > > index a0d489e..a820ed4 100644
> > > --- a/tools/perf/pmu-events/jevents.c
> > > +++ b/tools/perf/pmu-events/jevents.c
> > > @@ -42,6 +42,7 @@
> > >  #include <dirent.h>
> > >  #include <sys/time.h>			/* getrlimit */
> > >  #include <sys/resource.h>		/* getrlimit */
> > > +#include <sys/queue.h>
> > >  #include <ftw.h>
> > >  #include <sys/stat.h>
> > >  #include "jsmn.h"
> > > @@ -366,6 +367,94 @@ static int print_events_table_entry(void *data, char *name, char *event,
> > >  	return 0;
> > >  }
> > > 
> > > +struct event_struct {
> > > +	char *name;
> > > +	char *event;
> > > +	char *desc;
> > > +	char *long_desc;
> > > +	char *pmu;
> > > +	char *unit;
> > > +	char *perpkg;
> > > +	char *metric_expr;
> > > +	char *metric_name;
> > > +	char *metric_group;
> > > +	LIST_ENTRY(event_struct) list;
> > 
> > is there any reason you don't use the 'struct list_head' ?
> > I dont think we want another thingie involved for lists
> 
> Hi jirka,
> 
> The linux kernel headers are not used for jevents tool. I would rather use
> them if possible...

should be as easy as adding  #include <linux/list.h> ;-)

it's heavily used within perf and I'm pretty sure we want
to keep around just one way of doing lists

thanks,
jirka

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Wed, Dec 06, 2017 at 02:40:10PM +0000, John Garry wrote:
> On 06/12/2017 13:37, Jiri Olsa wrote:
> > On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
> > 
> > SNIP
> > 
> > > ---
> > >  tools/perf/pmu-events/jevents.c | 215 ++++++++++++++++++++++++++++++++++++----
> > >  1 file changed, 198 insertions(+), 17 deletions(-)
> > > 
> > > diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
> > > index a0d489e..a820ed4 100644
> > > --- a/tools/perf/pmu-events/jevents.c
> > > +++ b/tools/perf/pmu-events/jevents.c
> > > @@ -42,6 +42,7 @@
> > >  #include <dirent.h>
> > >  #include <sys/time.h>			/* getrlimit */
> > >  #include <sys/resource.h>		/* getrlimit */
> > > +#include <sys/queue.h>
> > >  #include <ftw.h>
> > >  #include <sys/stat.h>
> > >  #include "jsmn.h"
> > > @@ -366,6 +367,94 @@ static int print_events_table_entry(void *data, char *name, char *event,
> > >  	return 0;
> > >  }
> > > 
> > > +struct event_struct {
> > > +	char *name;
> > > +	char *event;
> > > +	char *desc;
> > > +	char *long_desc;
> > > +	char *pmu;
> > > +	char *unit;
> > > +	char *perpkg;
> > > +	char *metric_expr;
> > > +	char *metric_name;
> > > +	char *metric_group;
> > > +	LIST_ENTRY(event_struct) list;
> > 
> > is there any reason you don't use the 'struct list_head' ?
> > I dont think we want another thingie involved for lists
> 
> Hi jirka,
> 
> The linux kernel headers are not used for jevents tool. I would rather use
> them if possible...

should be as easy as adding  #include <linux/list.h> ;-)

it's heavily used within perf and I'm pretty sure we want
to keep around just one way of doing lists

thanks,
jirka

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 08/12/2017 12:31, Jiri Olsa wrote:
> On Wed, Dec 06, 2017 at 02:40:10PM +0000, John Garry wrote:
>> On 06/12/2017 13:37, Jiri Olsa wrote:
>>> On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
>>>
>>> SNIP
>>>
>>>> ---
>>>>  tools/perf/pmu-events/jevents.c | 215 ++++++++++++++++++++++++++++++++++++----
>>>>  1 file changed, 198 insertions(+), 17 deletions(-)
>>>>
>>>> diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
>>>> index a0d489e..a820ed4 100644
>>>> --- a/tools/perf/pmu-events/jevents.c
>>>> +++ b/tools/perf/pmu-events/jevents.c
>>>> @@ -42,6 +42,7 @@
>>>>  #include <dirent.h>
>>>>  #include <sys/time.h>			/* getrlimit */
>>>>  #include <sys/resource.h>		/* getrlimit */
>>>> +#include <sys/queue.h>
>>>>  #include <ftw.h>
>>>>  #include <sys/stat.h>
>>>>  #include "jsmn.h"
>>>> @@ -366,6 +367,94 @@ static int print_events_table_entry(void *data, char *name, char *event,
>>>>  	return 0;
>>>>  }
>>>>
>>>> +struct event_struct {
>>>> +	char *name;
>>>> +	char *event;
>>>> +	char *desc;
>>>> +	char *long_desc;
>>>> +	char *pmu;
>>>> +	char *unit;
>>>> +	char *perpkg;
>>>> +	char *metric_expr;
>>>> +	char *metric_name;
>>>> +	char *metric_group;
>>>> +	LIST_ENTRY(event_struct) list;
>>>
>>> is there any reason you don't use the 'struct list_head' ?
>>> I dont think we want another thingie involved for lists
>>
>> Hi jirka,
>>

Hi jirka,

>> The linux kernel headers are not used for jevents tool. I would rather use
>> them if possible...
>
> should be as easy as adding  #include <linux/list.h> ;-)
>

Hi jirka,

I think the issue is that jevents is a "hostprogs", which does not use 
kernel headers.

FWIW, here is the symptom:
pmu-events/jevents.c:51:24: fatal error: linux/list.h: No such file or 
directory
  #include <linux/list.h>
                         ^
compilation terminated.
mv: cannot stat ‘pmu-events/.jevents.o.tmp’: No such file or directory

perf tool build is different.

Much appreciated,
John


> it's heavily used within perf and I'm pretty sure we want
> to keep around just one way of doing lists
>
> thanks,
> jirka
>
> .
>

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[John Garry]

On 08/12/2017 12:31, Jiri Olsa wrote:
> On Wed, Dec 06, 2017 at 02:40:10PM +0000, John Garry wrote:
>> On 06/12/2017 13:37, Jiri Olsa wrote:
>>> On Wed, Dec 06, 2017 at 12:13:16AM +0800, John Garry wrote:
>>>
>>> SNIP
>>>
>>>> ---
>>>>  tools/perf/pmu-events/jevents.c | 215 ++++++++++++++++++++++++++++++++++++----
>>>>  1 file changed, 198 insertions(+), 17 deletions(-)
>>>>
>>>> diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
>>>> index a0d489e..a820ed4 100644
>>>> --- a/tools/perf/pmu-events/jevents.c
>>>> +++ b/tools/perf/pmu-events/jevents.c
>>>> @@ -42,6 +42,7 @@
>>>>  #include <dirent.h>
>>>>  #include <sys/time.h>			/* getrlimit */
>>>>  #include <sys/resource.h>		/* getrlimit */
>>>> +#include <sys/queue.h>
>>>>  #include <ftw.h>
>>>>  #include <sys/stat.h>
>>>>  #include "jsmn.h"
>>>> @@ -366,6 +367,94 @@ static int print_events_table_entry(void *data, char *name, char *event,
>>>>  	return 0;
>>>>  }
>>>>
>>>> +struct event_struct {
>>>> +	char *name;
>>>> +	char *event;
>>>> +	char *desc;
>>>> +	char *long_desc;
>>>> +	char *pmu;
>>>> +	char *unit;
>>>> +	char *perpkg;
>>>> +	char *metric_expr;
>>>> +	char *metric_name;
>>>> +	char *metric_group;
>>>> +	LIST_ENTRY(event_struct) list;
>>>
>>> is there any reason you don't use the 'struct list_head' ?
>>> I dont think we want another thingie involved for lists
>>
>> Hi jirka,
>>

Hi jirka,

>> The linux kernel headers are not used for jevents tool. I would rather use
>> them if possible...
>
> should be as easy as adding  #include <linux/list.h> ;-)
>

Hi jirka,

I think the issue is that jevents is a "hostprogs", which does not use 
kernel headers.

FWIW, here is the symptom:
pmu-events/jevents.c:51:24: fatal error: linux/list.h: No such file or 
directory
  #include <linux/list.h>
                         ^
compilation terminated.
mv: cannot stat ?pmu-events/.jevents.o.tmp?: No such file or directory

perf tool build is different.

Much appreciated,
John


> it's heavily used within perf and I'm pretty sure we want
> to keep around just one way of doing lists
>
> thanks,
> jirka
>
> .
>

Re: [RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Fri, Dec 08, 2017 at 03:38:06PM +0000, John Garry wrote:

SNIP

> > > 
> > > Hi jirka,
> > > 
> 
> Hi jirka,
> 
> > > The linux kernel headers are not used for jevents tool. I would rather use
> > > them if possible...
> > 
> > should be as easy as adding  #include <linux/list.h> ;-)
> > 
> 
> Hi jirka,
> 
> I think the issue is that jevents is a "hostprogs", which does not use
> kernel headers.
> 
> FWIW, here is the symptom:
> pmu-events/jevents.c:51:24: fatal error: linux/list.h: No such file or
> directory
>  #include <linux/list.h>
>                         ^
> compilation terminated.
> mv: cannot stat ‘pmu-events/.jevents.o.tmp’: No such file or directory
> 
> perf tool build is different.

yep, need additional in Bukld file, attached

jirka


---
diff --git a/tools/perf/pmu-events/Build b/tools/perf/pmu-events/Build
index 999a4e878162..b7d2e0e9cbd0 100644
--- a/tools/perf/pmu-events/Build
+++ b/tools/perf/pmu-events/Build
@@ -1,5 +1,6 @@
 hostprogs := jevents
 
+CHOSTFLAGS      = -I$(srctree)/tools/include
 jevents-y	+= json.o jsmn.o jevents.o
 pmu-events-y	+= pmu-events.o
 JDIR		=  pmu-events/arch/$(SRCARCH)
diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
index b578aa26e375..5b9b1fee3dfe 100644
--- a/tools/perf/pmu-events/jevents.c
+++ b/tools/perf/pmu-events/jevents.c
@@ -47,6 +47,7 @@
 #include "jsmn.h"
 #include "json.h"
 #include "jevents.h"
+#include <linux/list.h>
 
 int verbose;
 char *prog;
@@ -884,6 +885,7 @@ int main(int argc, char *argv[])
 	const char *output_file;
 	const char *start_dirname;
 	struct stat stbuf;
+	struct list_head krava __maybe_unused;
 
 	prog = basename(argv[0]);
 	if (argc < 4) {

[RFC PATCH 2/5] perf jevents: add support for arch recommended events

[Jiri Olsa]

On Fri, Dec 08, 2017 at 03:38:06PM +0000, John Garry wrote:

SNIP

> > > 
> > > Hi jirka,
> > > 
> 
> Hi jirka,
> 
> > > The linux kernel headers are not used for jevents tool. I would rather use
> > > them if possible...
> > 
> > should be as easy as adding  #include <linux/list.h> ;-)
> > 
> 
> Hi jirka,
> 
> I think the issue is that jevents is a "hostprogs", which does not use
> kernel headers.
> 
> FWIW, here is the symptom:
> pmu-events/jevents.c:51:24: fatal error: linux/list.h: No such file or
> directory
>  #include <linux/list.h>
>                         ^
> compilation terminated.
> mv: cannot stat ?pmu-events/.jevents.o.tmp?: No such file or directory
> 
> perf tool build is different.

yep, need additional in Bukld file, attached

jirka


---
diff --git a/tools/perf/pmu-events/Build b/tools/perf/pmu-events/Build
index 999a4e878162..b7d2e0e9cbd0 100644
--- a/tools/perf/pmu-events/Build
+++ b/tools/perf/pmu-events/Build
@@ -1,5 +1,6 @@
 hostprogs := jevents
 
+CHOSTFLAGS      = -I$(srctree)/tools/include
 jevents-y	+= json.o jsmn.o jevents.o
 pmu-events-y	+= pmu-events.o
 JDIR		=  pmu-events/arch/$(SRCARCH)
diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
index b578aa26e375..5b9b1fee3dfe 100644
--- a/tools/perf/pmu-events/jevents.c
+++ b/tools/perf/pmu-events/jevents.c
@@ -47,6 +47,7 @@
 #include "jsmn.h"
 #include "json.h"
 #include "jevents.h"
+#include <linux/list.h>
 
 int verbose;
 char *prog;
@@ -884,6 +885,7 @@ int main(int argc, char *argv[])
 	const char *output_file;
 	const char *start_dirname;
 	struct stat stbuf;
+	struct list_head krava __maybe_unused;
 
 	prog = basename(argv[0]);
 	if (argc < 4) {

[RFC PATCH 3/5] perf vendor events arm64: add armv8 recommended events JSON

[John Garry]

Add JSON for arm64 IMPLEMENTATION DEFINED recommended events.

Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Shaokun Zhang <zhangshaokun@hisilicon.com>
---
 .../pmu-events/arch/arm64/armv8-recommended.json   | 452 +++++++++++++++++++++
 1 file changed, 452 insertions(+)
 create mode 100644 tools/perf/pmu-events/arch/arm64/armv8-recommended.json

diff --git a/tools/perf/pmu-events/arch/arm64/armv8-recommended.json b/tools/perf/pmu-events/arch/arm64/armv8-recommended.json
new file mode 100644
index 0000000..5584c31
--- /dev/null
+++ b/tools/perf/pmu-events/arch/arm64/armv8-recommended.json
@@ -0,0 +1,452 @@
+[
+    {
+        "PublicDescription": "Attributable Level 1 data cache access, read",
+        "EventCode": "0x40",
+        "EventName": "L1D_CACHE_RD",
+        "BriefDescription": "L1D cache access, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data cache access, write",
+        "EventCode": "0x41",
+        "EventName": "L1D_CACHE_WR",
+        "BriefDescription": "L1D cache access, write",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data cache refill, read",
+        "EventCode": "0x42",
+        "EventName": "L1D_CACHE_REFILL_RD",
+        "BriefDescription": "L1D cache refill, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data cache refill, write",
+        "EventCode": "0x43",
+        "EventName": "L1D_CACHE_REFILL_WR",
+        "BriefDescription": "L1D cache refill, write",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data cache refill, inner",
+        "EventCode": "0x44",
+        "EventName": "L1D_CACHE_REFILL_INNER",
+        "BriefDescription": "L1D cache refill, inner",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data cache refill, outer",
+        "EventCode": "0x45",
+        "EventName": "L1D_CACHE_REFILL_OUTER",
+        "BriefDescription": "L1D cache refill, outer",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data cache Write-Back, victim",
+        "EventCode": "0x46",
+        "EventName": "L1D_CACHE_WB_VICTIM",
+        "BriefDescription": "L1D cache Write-Back, victim",
+    },
+    {
+        "PublicDescription": "Level 1 data cache Write-Back, cleaning and coherency",
+        "EventCode": "0x47",
+        "EventName": "L1D_CACHE_WB_CLEAN",
+        "BriefDescription": "L1D cache Write-Back, cleaning and coherency",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data cache invalidate",
+        "EventCode": "0x48",
+        "EventName": "L1D_CACHE_INVAL",
+        "BriefDescription": "L1D cache invalidate",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data TLB refill, read",
+        "EventCode": "0x4C",
+        "EventName": "L1D_TLB_REFILL_RD",
+        "BriefDescription": "L1D tlb refill, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data TLB refill, write",
+        "EventCode": "0x4D",
+        "EventName": "L1D_TLB_REFILL_WR",
+        "BriefDescription": "L1D tlb refill, write",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data or unified TLB access, read",
+        "EventCode": "0x4E",
+        "EventName": "L1D_TLB_RD",
+        "BriefDescription": "L1D tlb access, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data or unified TLB access, write",
+        "EventCode": "0x4F",
+        "EventName": "L1D_TLB_WR",
+        "BriefDescription": "L1D tlb access, write",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data cache access, read",
+        "EventCode": "0x50",
+        "EventName": "L2D_CACHE_RD",
+        "BriefDescription": "L2D cache access, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data cache access, write",
+        "EventCode": "0x51",
+        "EventName": "L2D_CACHE_WR",
+        "BriefDescription": "L2D cache access, write",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data cache refill, read",
+        "EventCode": "0x52",
+        "EventName": "L2D_CACHE_REFILL_RD",
+        "BriefDescription": "L2D cache refill, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data cache refill, write",
+        "EventCode": "0x53",
+        "EventName": "L2D_CACHE_REFILL_WR",
+        "BriefDescription": "L2D cache refill, write",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data cache Write-Back, victim",
+        "EventCode": "0x56",
+        "EventName": "L2D_CACHE_WB_VICTIM",
+        "BriefDescription": "L2D cache Write-Back, victim",
+    },
+    {
+        "PublicDescription": "Level 2 data cache Write-Back, cleaning and coherency",
+        "EventCode": "0x57",
+        "EventName": "L2D_CACHE_WB_CLEAN",
+        "BriefDescription": "L2D cache Write-Back, cleaning and coherency",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data cache invalidate",
+        "EventCode": "0x58",
+        "EventName": "L2D_CACHE_INVAL",
+        "BriefDescription": "L2D cache invalidate",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data or unified TLB refill, read",
+        "EventCode": "0x5c",
+        "EventName": "L2D_TLB_REFILL_RD",
+        "BriefDescription": "L2D cache refill, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data or unified TLB refill, write",
+        "EventCode": "0x5d",
+        "EventName": "L2D_TLB_REFILL_WR",
+        "BriefDescription": "L2D cache refill, write",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data or unified TLB access, read",
+        "EventCode": "0x5e",
+        "EventName": "L2D_TLB_RD",
+        "BriefDescription": "L2D cache access, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data or unified TLB access, write",
+        "EventCode": "0x5f",
+        "EventName": "L2D_TLB_WR",
+        "BriefDescription": "L2D cache access, write",
+    },
+    {
+        "PublicDescription": "Bus access read",
+        "EventCode": "0x60",
+        "EventName": "BUS_ACCESS_RD",
+        "BriefDescription": "Bus access read",
+   },
+   {
+        "PublicDescription": "Bus access write",
+        "EventCode": "0x61",
+        "EventName": "BUS_ACCESS_WR",
+        "BriefDescription": "Bus access write",
+   }
+   {
+        "PublicDescription": "Bus access, Normal, Cacheable, Shareable",
+        "EventCode": "0x62",
+        "EventName": "BUS_ACCESS_SHARED",
+        "BriefDescription": "Bus access, Normal, Cacheable, Shareable",
+   }
+   {
+        "PublicDescription": "Bus access, not Normal, Cacheable, Shareable",
+        "EventCode": "0x63",
+        "EventName": "BUS_ACCESS_NOT_SHARED",
+        "BriefDescription": "Bus access, not Normal, Cacheable, Shareable",
+   }
+   {
+        "PublicDescription": "Bus access, Normal",
+        "EventCode": "0x64",
+        "EventName": "BUS_ACCESS_NORMAL",
+        "BriefDescription": "Bus access, Normal",
+   }
+   {
+        "PublicDescription": "Bus access, peripheral",
+        "EventCode": "0x65",
+        "EventName": "BUS_ACCESS_PERIPH",
+        "BriefDescription": "Bus access, peripheral",
+   }
+   {
+        "PublicDescription": "Data memory access, read",
+        "EventCode": "0x66",
+        "EventName": "MEM_ACCESS_RD",
+        "BriefDescription": "Data memory access, read",
+   }
+   {
+        "PublicDescription": "Data memory access, write",
+        "EventCode": "0x67",
+        "EventName": "MEM_ACCESS_WR",
+        "BriefDescription": "Data memory access, write",
+   }
+   {
+        "PublicDescription": "Unaligned access, read",
+        "EventCode": "0x68",
+        "EventName": "UNALIGNED_LD_SPEC",
+        "BriefDescription": "Unaligned access, read",
+   }
+   {
+        "PublicDescription": "Unaligned access, write",
+        "EventCode": "0x69",
+        "EventName": "UNALIGNED_ST_SPEC",
+        "BriefDescription": "Unaligned access, write",
+   }
+   {
+        "PublicDescription": "Unaligned access",
+        "EventCode": "0x6a",
+        "EventName": "UNALIGNED_LDST_SPEC",
+        "BriefDescription": "Unaligned access",
+   }
+   {
+        "PublicDescription": "Exclusive operation speculatively executed, LDREX or LDX",
+        "EventCode": "0x6c",
+        "EventName": "LDREX_SPEC",
+        "BriefDescription": "Exclusive operation speculatively executed, LDREX or LDX",
+   }
+   {
+        "PublicDescription": "Exclusive operation speculatively executed, STREX or STX pass",
+        "EventCode": "0x6d",
+        "EventName": "STREX_PASS_SPEC",
+        "BriefDescription": "Exclusive operation speculatively executed, STREX or STX pass",
+   }
+   {
+        "PublicDescription": "Exclusive operation speculatively executed, STREX or STX fail",
+        "EventCode": "0x6e",
+        "EventName": "STREX_FAIL_SPEC",
+        "BriefDescription": "Exclusive operation speculatively executed, STREX or STX fail",
+   }
+   {
+        "PublicDescription": "Exclusive operation speculatively executed, STREX or STX",
+        "EventCode": "0x6f",
+        "EventName": "STREX_SPEC",
+        "BriefDescription": "Exclusive operation speculatively executed, STREX or STX",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, load",
+        "EventCode": "0x70",
+        "EventName": "LD_SPEC",
+        "BriefDescription": "Operation speculatively executed, load",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, store",
+        "EventCode": "0x71",
+        "EventName": "ST_SPEC",
+        "BriefDescription": "Operation speculatively executed, store",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, load or store",
+        "EventCode": "0x72",
+        "EventName": "LDST_SPEC",
+        "BriefDescription": "Operation speculatively executed, load or store",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, integer data processing",
+        "EventCode": "0x73",
+        "EventName": "DP_SPEC",
+        "BriefDescription": "Operation speculatively executed, integer data processing",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, Advanced SIMD instruction",
+        "EventCode": "0x74",
+        "EventName": "ASE_SPEC",
+        "BriefDescription": "Operation speculatively executed, Advanced SIMD instruction",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, floating-point instruction",
+        "EventCode": "0x75",
+        "EventName": "VFP_SPEC",
+        "BriefDescription": "Operation speculatively executed, floating-point instruction",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, software change of the PC",
+        "EventCode": "0x76",
+        "EventName": "PC_WRITE_SPEC",
+        "BriefDescription": "Operation speculatively executed, software change of the PC",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, Cryptographic instruction",
+        "EventCode": "0x77",
+        "EventName": "CRYPTO_SPEC",
+        "BriefDescription": "Operation speculatively executed, Cryptographic instruction",
+   }
+   {
+        "PublicDescription": "Branch speculatively executed, immediate branch"
+        "EventCode": "0x78",
+        "EventName": "BR_IMMED_SPEC",
+        "BriefDescription": "Branch speculatively executed, immediate branch"
+   }
+   {
+        "PublicDescription": "Branch speculatively executed, procedure return"
+        "EventCode": "0x79",
+        "EventName": "BR_RETURN_SPEC",
+        "BriefDescription": "Branch speculatively executed, procedure return"
+   }
+   {
+        "PublicDescription": "Branch speculatively executed, indirect branch"
+        "EventCode": "0x7a",
+        "EventName": "BR_INDIRECT_SPEC",
+        "BriefDescription": "Branch speculatively executed, indirect branch"
+   }
+   {
+        "PublicDescription": "Barrier speculatively executed, ISB"
+        "EventCode": "0x7c",
+        "EventName": "ISB_SPEC",
+        "BriefDescription": "Barrier speculatively executed, ISB"
+   }
+   {
+        "PublicDescription": "Barrier speculatively executed, DSB"
+        "EventCode": "0x7d",
+        "EventName": "DSB_SPEC",
+        "BriefDescription": "Barrier speculatively executed, DSB"
+   }
+   {
+        "PublicDescription": "Barrier speculatively executed, DMB"
+        "EventCode": "0x7e",
+        "EventName": "DMB_SPEC",
+        "BriefDescription": "Barrier speculatively executed, DMB"
+   }
+   {
+        "PublicDescription": "Exception taken, Other synchronous"
+        "EventCode": "0x81",
+        "EventName": "EXC_UNDEF",
+        "BriefDescription": "Exception taken, Other synchronous"
+   }
+   {
+        "PublicDescription": "Exception taken, Supervisor Call"
+        "EventCode": "0x82",
+        "EventName": "EXC_SVC",
+        "BriefDescription": "Exception taken, Supervisor Call"
+   }
+   {
+        "PublicDescription": "Exception taken, Instruction Abort"
+        "EventCode": "0x83",
+        "EventName": "EXC_PABORT",
+        "BriefDescription": "Exception taken, Instruction Abort"
+   }
+   {
+        "PublicDescription": "Exception taken, Data Abort and SError"
+        "EventCode": "0x84",
+        "EventName": "EXC_DABORT",
+        "BriefDescription": "Exception taken, Data Abort and SError"
+   }
+   {
+        "PublicDescription": "Exception taken, IRQ"
+        "EventCode": "0x86",
+        "EventName": "EXC_IRQ",
+        "BriefDescription": "Exception taken, IRQ"
+   }
+   {
+        "PublicDescription": "Exception taken, FIQ"
+        "EventCode": "0x87",
+        "EventName": "EXC_FIQ",
+        "BriefDescription": "Exception taken, FIQ"
+   }
+   {
+        "PublicDescription": "Exception taken, Secure Monitor Call"
+        "EventCode": "0x88",
+        "EventName": "EXC_SMC",
+        "BriefDescription": "Exception taken, Secure Monitor Call"
+   }
+   {
+        "PublicDescription": "Exception taken, Hypervisor Call"
+        "EventCode": "0x8a",
+        "EventName": "EXC_HVC",
+        "BriefDescription": "Exception taken, Hypervisor Call"
+   }
+   {
+        "PublicDescription": "Exception taken, Instruction Abort not taken locally"
+        "EventCode": "0x8b",
+        "EventName": "EXC_TRAP_PABORT",
+        "BriefDescription": "Exception taken, Instruction Abort not taken locally"
+   }
+   {
+        "PublicDescription": "Exception taken, Data Abort or SError not taken locally"
+        "EventCode": "0x8c",
+        "EventName": "EXC_TRAP_DABORT",
+        "BriefDescription": "Exception taken, Data Abort or SError not taken locally"
+   }
+   {
+        "PublicDescription": "Exception taken, Other traps not taken locally"
+        "EventCode": "0x8d",
+        "EventName": "EXC_TRAP_OTHER",
+        "BriefDescription": "Exception taken, Other traps not taken locally"
+   }
+   {
+        "PublicDescription": "Exception taken, IRQ not taken locally"
+        "EventCode": "0x8e",
+        "EventName": "EXC_TRAP_IRQ",
+        "BriefDescription": "Exception taken, IRQ not taken locally"
+   }
+   {
+        "PublicDescription": "Exception taken, FIQ not taken locally"
+        "EventCode": "0x8f",
+        "EventName": "EXC_TRAP_FIQ",
+        "BriefDescription": "Exception taken, FIQ not taken locally"
+   }
+   {
+        "PublicDescription": "Release consistency operation speculatively executed, Load-Acquire"
+        "EventCode": "0x90",
+        "EventName": "RC_LD_SPEC",
+        "BriefDescription": "Release consistency operation speculatively executed, Load-Acquire"
+   }
+   {
+        "PublicDescription": "Release consistency operation speculatively executed, Store-Release"
+        "EventCode": "0x91",
+        "EventName": "RC_ST_SPEC",
+        "BriefDescription": "Release consistency operation speculatively executed, Store-Release"
+   }
+   {
+        "PublicDescription": "Attributable Level 3 data or unified cache access, read"
+        "EventCode": "0xa0",
+        "EventName": "L3D_CACHE_RD",
+        "BriefDescription": "Attributable Level 3 data or unified cache access, read"
+   }
+   {
+        "PublicDescription": "Attributable Level 3 data or unified cache access, write"
+        "EventCode": "0xa1",
+        "EventName": "L3D_CACHE_WR",
+        "BriefDescription": "Attributable Level 3 data or unified cache access, write"
+   }
+   {
+        "PublicDescription": "Attributable Level 3 data or unified cache refill, read"
+        "EventCode": "0xa2",
+        "EventName": "L3D_CACHE_REFILL_RD",
+        "BriefDescription": "Attributable Level 3 data or unified cache refill, read"
+   }
+   {
+        "PublicDescription": "Attributable Level 3 data or unified cache refill, write"
+        "EventCode": "0xa3",
+        "EventName": "L3D_CACHE_REFILL_WR",
+        "BriefDescription": "Attributable Level 3 data or unified cache refill, write"
+   }
+   {
+        "PublicDescription": "Attributable Level 3 data or unified cache Write-Back, victim"
+        "EventCode": "0xa6",
+        "EventName": "L3D_CACHE_WB_VICTIM",
+        "BriefDescription": "Attributable Level 3 data or unified cache Write-Back, victim"
+   }
+   {
+        "PublicDescription": "Attributable Level 3 data or unified cache Write-Back, cache clean"
+        "EventCode": "0xa7",
+        "EventName": "L3D_CACHE_WB_CLEAN",
+        "BriefDescription": "Attributable Level 3 data or unified cache Write-Back, cache clean"
+   }
+   {
+        "PublicDescription": "Attributable Level 3 data or unified cache access, invalidate"
+        "EventCode": "0xa8",
+        "EventName": "L3D_CACHE_INVAL",
+        "BriefDescription": "Attributable Level 3 data or unified cache access, invalidate"
+   }
+]
-- 
1.9.1

[RFC PATCH 3/5] perf vendor events arm64: add armv8 recommended events JSON

[John Garry]

Add JSON for arm64 IMPLEMENTATION DEFINED recommended events.

Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Shaokun Zhang <zhangshaokun@hisilicon.com>
---
 .../pmu-events/arch/arm64/armv8-recommended.json   | 452 +++++++++++++++++++++
 1 file changed, 452 insertions(+)
 create mode 100644 tools/perf/pmu-events/arch/arm64/armv8-recommended.json

diff --git a/tools/perf/pmu-events/arch/arm64/armv8-recommended.json b/tools/perf/pmu-events/arch/arm64/armv8-recommended.json
new file mode 100644
index 0000000..5584c31
--- /dev/null
+++ b/tools/perf/pmu-events/arch/arm64/armv8-recommended.json
@@ -0,0 +1,452 @@
+[
+    {
+        "PublicDescription": "Attributable Level 1 data cache access, read",
+        "EventCode": "0x40",
+        "EventName": "L1D_CACHE_RD",
+        "BriefDescription": "L1D cache access, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data cache access, write",
+        "EventCode": "0x41",
+        "EventName": "L1D_CACHE_WR",
+        "BriefDescription": "L1D cache access, write",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data cache refill, read",
+        "EventCode": "0x42",
+        "EventName": "L1D_CACHE_REFILL_RD",
+        "BriefDescription": "L1D cache refill, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data cache refill, write",
+        "EventCode": "0x43",
+        "EventName": "L1D_CACHE_REFILL_WR",
+        "BriefDescription": "L1D cache refill, write",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data cache refill, inner",
+        "EventCode": "0x44",
+        "EventName": "L1D_CACHE_REFILL_INNER",
+        "BriefDescription": "L1D cache refill, inner",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data cache refill, outer",
+        "EventCode": "0x45",
+        "EventName": "L1D_CACHE_REFILL_OUTER",
+        "BriefDescription": "L1D cache refill, outer",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data cache Write-Back, victim",
+        "EventCode": "0x46",
+        "EventName": "L1D_CACHE_WB_VICTIM",
+        "BriefDescription": "L1D cache Write-Back, victim",
+    },
+    {
+        "PublicDescription": "Level 1 data cache Write-Back, cleaning and coherency",
+        "EventCode": "0x47",
+        "EventName": "L1D_CACHE_WB_CLEAN",
+        "BriefDescription": "L1D cache Write-Back, cleaning and coherency",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data cache invalidate",
+        "EventCode": "0x48",
+        "EventName": "L1D_CACHE_INVAL",
+        "BriefDescription": "L1D cache invalidate",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data TLB refill, read",
+        "EventCode": "0x4C",
+        "EventName": "L1D_TLB_REFILL_RD",
+        "BriefDescription": "L1D tlb refill, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data TLB refill, write",
+        "EventCode": "0x4D",
+        "EventName": "L1D_TLB_REFILL_WR",
+        "BriefDescription": "L1D tlb refill, write",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data or unified TLB access, read",
+        "EventCode": "0x4E",
+        "EventName": "L1D_TLB_RD",
+        "BriefDescription": "L1D tlb access, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 1 data or unified TLB access, write",
+        "EventCode": "0x4F",
+        "EventName": "L1D_TLB_WR",
+        "BriefDescription": "L1D tlb access, write",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data cache access, read",
+        "EventCode": "0x50",
+        "EventName": "L2D_CACHE_RD",
+        "BriefDescription": "L2D cache access, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data cache access, write",
+        "EventCode": "0x51",
+        "EventName": "L2D_CACHE_WR",
+        "BriefDescription": "L2D cache access, write",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data cache refill, read",
+        "EventCode": "0x52",
+        "EventName": "L2D_CACHE_REFILL_RD",
+        "BriefDescription": "L2D cache refill, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data cache refill, write",
+        "EventCode": "0x53",
+        "EventName": "L2D_CACHE_REFILL_WR",
+        "BriefDescription": "L2D cache refill, write",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data cache Write-Back, victim",
+        "EventCode": "0x56",
+        "EventName": "L2D_CACHE_WB_VICTIM",
+        "BriefDescription": "L2D cache Write-Back, victim",
+    },
+    {
+        "PublicDescription": "Level 2 data cache Write-Back, cleaning and coherency",
+        "EventCode": "0x57",
+        "EventName": "L2D_CACHE_WB_CLEAN",
+        "BriefDescription": "L2D cache Write-Back, cleaning and coherency",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data cache invalidate",
+        "EventCode": "0x58",
+        "EventName": "L2D_CACHE_INVAL",
+        "BriefDescription": "L2D cache invalidate",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data or unified TLB refill, read",
+        "EventCode": "0x5c",
+        "EventName": "L2D_TLB_REFILL_RD",
+        "BriefDescription": "L2D cache refill, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data or unified TLB refill, write",
+        "EventCode": "0x5d",
+        "EventName": "L2D_TLB_REFILL_WR",
+        "BriefDescription": "L2D cache refill, write",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data or unified TLB access, read",
+        "EventCode": "0x5e",
+        "EventName": "L2D_TLB_RD",
+        "BriefDescription": "L2D cache access, read",
+    },
+    {
+        "PublicDescription": "Attributable Level 2 data or unified TLB access, write",
+        "EventCode": "0x5f",
+        "EventName": "L2D_TLB_WR",
+        "BriefDescription": "L2D cache access, write",
+    },
+    {
+        "PublicDescription": "Bus access read",
+        "EventCode": "0x60",
+        "EventName": "BUS_ACCESS_RD",
+        "BriefDescription": "Bus access read",
+   },
+   {
+        "PublicDescription": "Bus access write",
+        "EventCode": "0x61",
+        "EventName": "BUS_ACCESS_WR",
+        "BriefDescription": "Bus access write",
+   }
+   {
+        "PublicDescription": "Bus access, Normal, Cacheable, Shareable",
+        "EventCode": "0x62",
+        "EventName": "BUS_ACCESS_SHARED",
+        "BriefDescription": "Bus access, Normal, Cacheable, Shareable",
+   }
+   {
+        "PublicDescription": "Bus access, not Normal, Cacheable, Shareable",
+        "EventCode": "0x63",
+        "EventName": "BUS_ACCESS_NOT_SHARED",
+        "BriefDescription": "Bus access, not Normal, Cacheable, Shareable",
+   }
+   {
+        "PublicDescription": "Bus access, Normal",
+        "EventCode": "0x64",
+        "EventName": "BUS_ACCESS_NORMAL",
+        "BriefDescription": "Bus access, Normal",
+   }
+   {
+        "PublicDescription": "Bus access, peripheral",
+        "EventCode": "0x65",
+        "EventName": "BUS_ACCESS_PERIPH",
+        "BriefDescription": "Bus access, peripheral",
+   }
+   {
+        "PublicDescription": "Data memory access, read",
+        "EventCode": "0x66",
+        "EventName": "MEM_ACCESS_RD",
+        "BriefDescription": "Data memory access, read",
+   }
+   {
+        "PublicDescription": "Data memory access, write",
+        "EventCode": "0x67",
+        "EventName": "MEM_ACCESS_WR",
+        "BriefDescription": "Data memory access, write",
+   }
+   {
+        "PublicDescription": "Unaligned access, read",
+        "EventCode": "0x68",
+        "EventName": "UNALIGNED_LD_SPEC",
+        "BriefDescription": "Unaligned access, read",
+   }
+   {
+        "PublicDescription": "Unaligned access, write",
+        "EventCode": "0x69",
+        "EventName": "UNALIGNED_ST_SPEC",
+        "BriefDescription": "Unaligned access, write",
+   }
+   {
+        "PublicDescription": "Unaligned access",
+        "EventCode": "0x6a",
+        "EventName": "UNALIGNED_LDST_SPEC",
+        "BriefDescription": "Unaligned access",
+   }
+   {
+        "PublicDescription": "Exclusive operation speculatively executed, LDREX or LDX",
+        "EventCode": "0x6c",
+        "EventName": "LDREX_SPEC",
+        "BriefDescription": "Exclusive operation speculatively executed, LDREX or LDX",
+   }
+   {
+        "PublicDescription": "Exclusive operation speculatively executed, STREX or STX pass",
+        "EventCode": "0x6d",
+        "EventName": "STREX_PASS_SPEC",
+        "BriefDescription": "Exclusive operation speculatively executed, STREX or STX pass",
+   }
+   {
+        "PublicDescription": "Exclusive operation speculatively executed, STREX or STX fail",
+        "EventCode": "0x6e",
+        "EventName": "STREX_FAIL_SPEC",
+        "BriefDescription": "Exclusive operation speculatively executed, STREX or STX fail",
+   }
+   {
+        "PublicDescription": "Exclusive operation speculatively executed, STREX or STX",
+        "EventCode": "0x6f",
+        "EventName": "STREX_SPEC",
+        "BriefDescription": "Exclusive operation speculatively executed, STREX or STX",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, load",
+        "EventCode": "0x70",
+        "EventName": "LD_SPEC",
+        "BriefDescription": "Operation speculatively executed, load",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, store",
+        "EventCode": "0x71",
+        "EventName": "ST_SPEC",
+        "BriefDescription": "Operation speculatively executed, store",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, load or store",
+        "EventCode": "0x72",
+        "EventName": "LDST_SPEC",
+        "BriefDescription": "Operation speculatively executed, load or store",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, integer data processing",
+        "EventCode": "0x73",
+        "EventName": "DP_SPEC",
+        "BriefDescription": "Operation speculatively executed, integer data processing",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, Advanced SIMD instruction",
+        "EventCode": "0x74",
+        "EventName": "ASE_SPEC",
+        "BriefDescription": "Operation speculatively executed, Advanced SIMD instruction",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, floating-point instruction",
+        "EventCode": "0x75",
+        "EventName": "VFP_SPEC",
+        "BriefDescription": "Operation speculatively executed, floating-point instruction",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, software change of the PC",
+        "EventCode": "0x76",
+        "EventName": "PC_WRITE_SPEC",
+        "BriefDescription": "Operation speculatively executed, software change of the PC",
+   }
+   {
+        "PublicDescription": "Operation speculatively executed, Cryptographic instruction",
+        "EventCode": "0x77",
+        "EventName": "CRYPTO_SPEC",
+        "BriefDescription": "Operation speculatively executed, Cryptographic instruction",
+   }
+   {
+        "PublicDescription": "Branch speculatively executed, immediate branch"
+        "EventCode": "0x78",
+        "EventName": "BR_IMMED_SPEC",
+        "BriefDescription": "Branch speculatively executed, immediate branch"
+   }
+   {
+        "PublicDescription": "Branch speculatively executed, procedure return"
+        "EventCode": "0x79",
+        "EventName": "BR_RETURN_SPEC",
+        "BriefDescription": "Branch speculatively executed, procedure return"
+   }
+   {
+        "PublicDescription": "Branch speculatively executed, indirect branch"
+        "EventCode": "0x7a",
+        "EventName": "BR_INDIRECT_SPEC",
+        "BriefDescription": "Branch speculatively executed, indirect branch"
+   }
+   {
+        "PublicDescription": "Barrier speculatively executed, ISB"
+        "EventCode": "0x7c",
+        "EventName": "ISB_SPEC",
+        "BriefDescription": "Barrier speculatively executed, ISB"
+   }
+   {
+        "PublicDescription": "Barrier speculatively executed, DSB"
+        "EventCode": "0x7d",
+        "EventName": "DSB_SPEC",
+        "BriefDescription": "Barrier speculatively executed, DSB"
+   }
+   {
+        "PublicDescription": "Barrier speculatively executed, DMB"
+        "EventCode": "0x7e",
+        "EventName": "DMB_SPEC",
+        "BriefDescription": "Barrier speculatively executed, DMB"
+   }
+   {
+        "PublicDescription": "Exception taken, Other synchronous"
+        "EventCode": "0x81",
+        "EventName": "EXC_UNDEF",
+        "BriefDescription": "Exception taken, Other synchronous"
+   }
+   {
+        "PublicDescription": "Exception taken, Supervisor Call"
+        "EventCode": "0x82",
+        "EventName": "EXC_SVC",
+        "BriefDescription": "Exception taken, Supervisor Call"
+   }
+   {
+        "PublicDescription": "Exception taken, Instruction Abort"
+        "EventCode": "0x83",
+        "EventName": "EXC_PABORT",
+        "BriefDescription": "Exception taken, Instruction Abort"
+   }
+   {
+        "PublicDescription": "Exception taken, Data Abort and SError"
+        "EventCode": "0x84",
+        "EventName": "EXC_DABORT",
+        "BriefDescription": "Exception taken, Data Abort and SError"
+   }
+   {
+        "PublicDescription": "Exception taken, IRQ"
+        "EventCode": "0x86",
+        "EventName": "EXC_IRQ",
+        "BriefDescription": "Exception taken, IRQ"
+   }
+   {
+        "PublicDescription": "Exception taken, FIQ"
+        "EventCode": "0x87",
+        "EventName": "EXC_FIQ",
+        "BriefDescription": "Exception taken, FIQ"
+   }
+   {
+        "PublicDescription": "Exception taken, Secure Monitor Call"
+        "EventCode": "0x88",
+        "EventName": "EXC_SMC",
+        "BriefDescription": "Exception taken, Secure Monitor Call"
+   }
+   {
+        "PublicDescription": "Exception taken, Hypervisor Call"
+        "EventCode": "0x8a",
+        "EventName": "EXC_HVC",
+        "BriefDescription": "Exception taken, Hypervisor Call"
+   }
+   {
+        "PublicDescription": "Exception taken, Instruction Abort not taken locally"
+        "EventCode": "0x8b",
+        "EventName": "EXC_TRAP_PABORT",
+        "BriefDescription": "Exception taken, Instruction Abort not taken locally"
+   }
+   {
+        "PublicDescription": "Exception taken, Data Abort or SError not taken locally"
+        "EventCode": "0x8c",
+        "EventName": "EXC_TRAP_DABORT",
+        "BriefDescription": "Exception taken, Data Abort or SError not taken locally"
+   }
+   {
+        "PublicDescription": "Exception taken, Other traps not taken locally"
+        "EventCode": "0x8d",
+        "EventName": "EXC_TRAP_OTHER",
+        "BriefDescription": "Exception taken, Other traps not taken locally"
+   }
+   {
+        "PublicDescription": "Exception taken, IRQ not taken locally"
+        "EventCode": "0x8e",
+        "EventName": "EXC_TRAP_IRQ",
+        "BriefDescription": "Exception taken, IRQ not taken locally"
+   }
+   {
+        "PublicDescription": "Exception taken, FIQ not taken locally"
+        "EventCode": "0x8f",
+        "EventName": "EXC_TRAP_FIQ",
+        "BriefDescription": "Exception taken, FIQ not taken locally"
+   }
+   {
+        "PublicDescription": "Release consistency operation speculatively executed, Load-Acquire"
+        "EventCode": "0x90",
+        "EventName": "RC_LD_SPEC",
+        "BriefDescription": "Release consistency operation speculatively executed, Load-Acquire"
+   }
+   {
+        "PublicDescription": "Release consistency operation speculatively executed, Store-Release"
+        "EventCode": "0x91",
+        "EventName": "RC_ST_SPEC",
+        "BriefDescription": "Release consistency operation speculatively executed, Store-Release"
+   }
+   {
+        "PublicDescription": "Attributable Level 3 data or unified cache access, read"
+        "EventCode": "0xa0",
+        "EventName": "L3D_CACHE_RD",
+        "BriefDescription": "Attributable Level 3 data or unified cache access, read"
+   }
+   {
+        "PublicDescription": "Attributable Level 3 data or unified cache access, write"
+        "EventCode": "0xa1",
+        "EventName": "L3D_CACHE_WR",
+        "BriefDescription": "Attributable Level 3 data or unified cache access, write"
+   }
+   {
+        "PublicDescription": "Attributable Level 3 data or unified cache refill, read"
+        "EventCode": "0xa2",
+        "EventName": "L3D_CACHE_REFILL_RD",
+        "BriefDescription": "Attributable Level 3 data or unified cache refill, read"
+   }
+   {
+        "PublicDescription": "Attributable Level 3 data or unified cache refill, write"
+        "EventCode": "0xa3",
+        "EventName": "L3D_CACHE_REFILL_WR",
+        "BriefDescription": "Attributable Level 3 data or unified cache refill, write"
+   }
+   {
+        "PublicDescription": "Attributable Level 3 data or unified cache Write-Back, victim"
+        "EventCode": "0xa6",
+        "EventName": "L3D_CACHE_WB_VICTIM",
+        "BriefDescription": "Attributable Level 3 data or unified cache Write-Back, victim"
+   }
+   {
+        "PublicDescription": "Attributable Level 3 data or unified cache Write-Back, cache clean"
+        "EventCode": "0xa7",
+        "EventName": "L3D_CACHE_WB_CLEAN",
+        "BriefDescription": "Attributable Level 3 data or unified cache Write-Back, cache clean"
+   }
+   {
+        "PublicDescription": "Attributable Level 3 data or unified cache access, invalidate"
+        "EventCode": "0xa8",
+        "EventName": "L3D_CACHE_INVAL",
+        "BriefDescription": "Attributable Level 3 data or unified cache access, invalidate"
+   }
+]
-- 
1.9.1

[RFC PATCH 4/5] perf vendor events arm64: relocate thunderx2 JSON

[John Garry]

Since the pmu events architecture folder structure supports
arch/vendor/platform structure, relocate the ThunderX2 JSON.

Also since Cavium ThunderX2 has implemented its events
according to ARM recommendation, remove the fields apart from
"EventCode".

Signed-off-by: John Garry <john.garry@huawei.com>
---
 .../arch/arm64/cavium/thunderx2-imp-def.json       | 62 ----------------------
 .../arch/arm64/cavium/thunderx2/core-imp-def.json  | 32 +++++++++++
 tools/perf/pmu-events/arch/arm64/mapfile.csv       |  2 +-
 3 files changed, 33 insertions(+), 63 deletions(-)
 delete mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2-imp-def.json
 create mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2/core-imp-def.json

diff --git a/tools/perf/pmu-events/arch/arm64/cavium/thunderx2-imp-def.json b/tools/perf/pmu-events/arch/arm64/cavium/thunderx2-imp-def.json
deleted file mode 100644
index 2db45c4..0000000
--- a/tools/perf/pmu-events/arch/arm64/cavium/thunderx2-imp-def.json
+++ /dev/null
@@ -1,62 +0,0 @@
-[
-    {
-        "PublicDescription": "Attributable Level 1 data cache access, read",
-        "EventCode": "0x40",
-        "EventName": "l1d_cache_rd",
-        "BriefDescription": "L1D cache read",
-    },
-    {
-        "PublicDescription": "Attributable Level 1 data cache access, write ",
-        "EventCode": "0x41",
-        "EventName": "l1d_cache_wr",
-        "BriefDescription": "L1D cache write",
-    },
-    {
-        "PublicDescription": "Attributable Level 1 data cache refill, read",
-        "EventCode": "0x42",
-        "EventName": "l1d_cache_refill_rd",
-        "BriefDescription": "L1D cache refill read",
-    },
-    {
-        "PublicDescription": "Attributable Level 1 data cache refill, write",
-        "EventCode": "0x43",
-        "EventName": "l1d_cache_refill_wr",
-        "BriefDescription": "L1D refill write",
-    },
-    {
-        "PublicDescription": "Attributable Level 1 data TLB refill, read",
-        "EventCode": "0x4C",
-        "EventName": "l1d_tlb_refill_rd",
-        "BriefDescription": "L1D tlb refill read",
-    },
-    {
-        "PublicDescription": "Attributable Level 1 data TLB refill, write",
-        "EventCode": "0x4D",
-        "EventName": "l1d_tlb_refill_wr",
-        "BriefDescription": "L1D tlb refill write",
-    },
-    {
-        "PublicDescription": "Attributable Level 1 data or unified TLB access, read",
-        "EventCode": "0x4E",
-        "EventName": "l1d_tlb_rd",
-        "BriefDescription": "L1D tlb read",
-    },
-    {
-        "PublicDescription": "Attributable Level 1 data or unified TLB access, write",
-        "EventCode": "0x4F",
-        "EventName": "l1d_tlb_wr",
-        "BriefDescription": "L1D tlb write",
-    },
-    {
-        "PublicDescription": "Bus access read",
-        "EventCode": "0x60",
-        "EventName": "bus_access_rd",
-        "BriefDescription": "Bus access read",
-   },
-   {
-        "PublicDescription": "Bus access write",
-        "EventCode": "0x61",
-        "EventName": "bus_access_wr",
-        "BriefDescription": "Bus access write",
-   }
-]
diff --git a/tools/perf/pmu-events/arch/arm64/cavium/thunderx2/core-imp-def.json b/tools/perf/pmu-events/arch/arm64/cavium/thunderx2/core-imp-def.json
new file mode 100644
index 0000000..99313eb
--- /dev/null
+++ b/tools/perf/pmu-events/arch/arm64/cavium/thunderx2/core-imp-def.json
@@ -0,0 +1,32 @@
+[
+    {
+        "EventCode": "0x40",
+    },
+    {
+        "EventCode": "0x41",
+    },
+    {
+        "EventCode": "0x42",
+    },
+    {
+        "EventCode": "0x43",
+    },
+    {
+        "EventCode": "0x4C",
+    },
+    {
+        "EventCode": "0x4D",
+    },
+    {
+        "EventCode": "0x4E",
+    },
+    {
+        "EventCode": "0x4F",
+    },
+    {
+        "EventCode": "0x60",
+   },
+   {
+        "EventCode": "0x61",
+   }
+]
diff --git a/tools/perf/pmu-events/arch/arm64/mapfile.csv b/tools/perf/pmu-events/arch/arm64/mapfile.csv
index 219d675..32fa0d1 100644
--- a/tools/perf/pmu-events/arch/arm64/mapfile.csv
+++ b/tools/perf/pmu-events/arch/arm64/mapfile.csv
@@ -12,4 +12,4 @@
 #
 #
 #Family-model,Version,Filename,EventType
-0x00000000420f5160,v1,cavium,core
+0x00000000420f5160,v1,cavium/thunderx2,core
-- 
1.9.1

[RFC PATCH 4/5] perf vendor events arm64: relocate thunderx2 JSON

[John Garry]

Since the pmu events architecture folder structure supports
arch/vendor/platform structure, relocate the ThunderX2 JSON.

Also since Cavium ThunderX2 has implemented its events
according to ARM recommendation, remove the fields apart from
"EventCode".

Signed-off-by: John Garry <john.garry@huawei.com>
---
 .../arch/arm64/cavium/thunderx2-imp-def.json       | 62 ----------------------
 .../arch/arm64/cavium/thunderx2/core-imp-def.json  | 32 +++++++++++
 tools/perf/pmu-events/arch/arm64/mapfile.csv       |  2 +-
 3 files changed, 33 insertions(+), 63 deletions(-)
 delete mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2-imp-def.json
 create mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2/core-imp-def.json

diff --git a/tools/perf/pmu-events/arch/arm64/cavium/thunderx2-imp-def.json b/tools/perf/pmu-events/arch/arm64/cavium/thunderx2-imp-def.json
deleted file mode 100644
index 2db45c4..0000000
--- a/tools/perf/pmu-events/arch/arm64/cavium/thunderx2-imp-def.json
+++ /dev/null
@@ -1,62 +0,0 @@
-[
-    {
-        "PublicDescription": "Attributable Level 1 data cache access, read",
-        "EventCode": "0x40",
-        "EventName": "l1d_cache_rd",
-        "BriefDescription": "L1D cache read",
-    },
-    {
-        "PublicDescription": "Attributable Level 1 data cache access, write ",
-        "EventCode": "0x41",
-        "EventName": "l1d_cache_wr",
-        "BriefDescription": "L1D cache write",
-    },
-    {
-        "PublicDescription": "Attributable Level 1 data cache refill, read",
-        "EventCode": "0x42",
-        "EventName": "l1d_cache_refill_rd",
-        "BriefDescription": "L1D cache refill read",
-    },
-    {
-        "PublicDescription": "Attributable Level 1 data cache refill, write",
-        "EventCode": "0x43",
-        "EventName": "l1d_cache_refill_wr",
-        "BriefDescription": "L1D refill write",
-    },
-    {
-        "PublicDescription": "Attributable Level 1 data TLB refill, read",
-        "EventCode": "0x4C",
-        "EventName": "l1d_tlb_refill_rd",
-        "BriefDescription": "L1D tlb refill read",
-    },
-    {
-        "PublicDescription": "Attributable Level 1 data TLB refill, write",
-        "EventCode": "0x4D",
-        "EventName": "l1d_tlb_refill_wr",
-        "BriefDescription": "L1D tlb refill write",
-    },
-    {
-        "PublicDescription": "Attributable Level 1 data or unified TLB access, read",
-        "EventCode": "0x4E",
-        "EventName": "l1d_tlb_rd",
-        "BriefDescription": "L1D tlb read",
-    },
-    {
-        "PublicDescription": "Attributable Level 1 data or unified TLB access, write",
-        "EventCode": "0x4F",
-        "EventName": "l1d_tlb_wr",
-        "BriefDescription": "L1D tlb write",
-    },
-    {
-        "PublicDescription": "Bus access read",
-        "EventCode": "0x60",
-        "EventName": "bus_access_rd",
-        "BriefDescription": "Bus access read",
-   },
-   {
-        "PublicDescription": "Bus access write",
-        "EventCode": "0x61",
-        "EventName": "bus_access_wr",
-        "BriefDescription": "Bus access write",
-   }
-]
diff --git a/tools/perf/pmu-events/arch/arm64/cavium/thunderx2/core-imp-def.json b/tools/perf/pmu-events/arch/arm64/cavium/thunderx2/core-imp-def.json
new file mode 100644
index 0000000..99313eb
--- /dev/null
+++ b/tools/perf/pmu-events/arch/arm64/cavium/thunderx2/core-imp-def.json
@@ -0,0 +1,32 @@
+[
+    {
+        "EventCode": "0x40",
+    },
+    {
+        "EventCode": "0x41",
+    },
+    {
+        "EventCode": "0x42",
+    },
+    {
+        "EventCode": "0x43",
+    },
+    {
+        "EventCode": "0x4C",
+    },
+    {
+        "EventCode": "0x4D",
+    },
+    {
+        "EventCode": "0x4E",
+    },
+    {
+        "EventCode": "0x4F",
+    },
+    {
+        "EventCode": "0x60",
+   },
+   {
+        "EventCode": "0x61",
+   }
+]
diff --git a/tools/perf/pmu-events/arch/arm64/mapfile.csv b/tools/perf/pmu-events/arch/arm64/mapfile.csv
index 219d675..32fa0d1 100644
--- a/tools/perf/pmu-events/arch/arm64/mapfile.csv
+++ b/tools/perf/pmu-events/arch/arm64/mapfile.csv
@@ -12,4 +12,4 @@
 #
 #
 #Family-model,Version,Filename,EventType
-0x00000000420f5160,v1,cavium,core
+0x00000000420f5160,v1,cavium/thunderx2,core
-- 
1.9.1

[RFC PATCH 5/5] perf vendor events arm64: add HiSilicon hip08 JSON

[John Garry]

Add HiSilicon hip08 JSON. Since hip08 has its events
implementated according to ARM recommendation, only add
fields "EventCode" (where applicable - hip08 also has
implemented some other custom events).

Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Shaokun Zhang <zhangshaokun@hisilicon.com>
---
 .../arch/arm64/hisilicon/hip08/core-imp-def.json   | 122 +++++++++++++++++++++
 tools/perf/pmu-events/arch/arm64/mapfile.csv       |   1 +
 2 files changed, 123 insertions(+)
 create mode 100644 tools/perf/pmu-events/arch/arm64/hisilicon/hip08/core-imp-def.json

diff --git a/tools/perf/pmu-events/arch/arm64/hisilicon/hip08/core-imp-def.json b/tools/perf/pmu-events/arch/arm64/hisilicon/hip08/core-imp-def.json
new file mode 100644
index 0000000..94fde40
--- /dev/null
+++ b/tools/perf/pmu-events/arch/arm64/hisilicon/hip08/core-imp-def.json
@@ -0,0 +1,122 @@
+[
+    {
+        "EventCode": "0x40",
+    },
+    {
+        "EventCode": "0x41",
+    },
+    {
+        "EventCode": "0x42",
+    },
+    {
+        "EventCode": "0x43",
+    },
+    {
+        "EventCode": "0x46",
+    },
+    {
+        "EventCode": "0x47",
+    },
+    {
+        "EventCode": "0x48",
+    },
+    {
+        "EventCode": "0x4C",
+    },
+    {
+        "EventCode": "0x4D",
+    },
+    {
+        "EventCode": "0x4E",
+    },
+    {
+        "EventCode": "0x4F",
+    },
+    {
+        "EventCode": "0x50",
+    },
+    {
+        "EventCode": "0x51",
+    },
+    {
+        "EventCode": "0x52",
+    },
+    {
+        "EventCode": "0x53",
+    },
+    {
+        "EventCode": "0x56",
+    },
+    {
+        "EventCode": "0x57",
+    },
+    {
+        "EventCode": "0x58",
+    },
+    {
+        "PublicDescription": "Level 1 instruction cache prefetch access count",
+        "EventCode": "0x102e",
+        "EventName": "L1I_CACHE_PRF",
+        "BriefDescription": "L1I cache prefetch access count",
+    },
+    {
+        "PublicDescription": "Level 1 instruction cache miss due to prefetch access count",
+        "EventCode": "0x102f",
+        "EventName": "L1I_CACHE_PRF_REFILL",
+        "BriefDescription": "L1I cache miss due to prefetch access count",
+    },
+    {
+        "PublicDescription": "Instruction queue is empty",
+        "EventCode": "0x1043",
+        "EventName": "IQ_IS_EMPTY",
+        "BriefDescription": "Instruction queue is empty",
+    },
+    {
+        "PublicDescription": "Instruction fetch stall cycles",
+        "EventCode": "0x1044",
+        "EventName": "IF_IS_STALL",
+        "BriefDescription": "Instruction fetch stall cycles",
+    },
+    {
+        "PublicDescription": "Instructions can receive, but not send",
+        "EventCode": "0x2014",
+        "EventName": "FETCH_BUBBLE",
+        "BriefDescription": "Instructions can receive, but not send",
+    },
+    {
+        "PublicDescription": "Prefetch request from LSU",
+        "EventCode": "0x6013",
+        "EventName": "PRF_REQ",
+        "BriefDescription": "Prefetch request from LSU",
+    },
+    {
+        "PublicDescription": "Hit on prefetched data",
+        "EventCode": "0x6014",
+        "EventName": "HIT_ON_PRF",
+        "BriefDescription": "Hit on prefetched data",
+    },
+    {
+        "PublicDescription": "Cycles of that the number of issuing micro operations are less than 4",
+        "EventCode": "0x7001",
+        "EventName": "EXE_STALL_CYCLE",
+        "BriefDescription": "Cycles of that the number of issue ups are less than 4",
+    },
+    {
+        "PublicDescription": "No any micro operation is issued and meanwhile any load operation is not resolved",
+        "EventCode": "0x7004",
+        "EventName": "MEM_STALL_ANYLOAD",
+        "BriefDescription": "No any micro operation is issued and meanwhile any load operation is not resolved",
+    },
+    {
+        "PublicDescription": "No any micro operation is issued and meanwhile there is any load operation missing L1 cache and pending data refill",
+        "EventCode": "0x7006",
+        "EventName": "MEM_STALL_L1MISS",
+        "BriefDescription": "No any micro operation is issued and meanwhile there is any load operation missing L1 cache and pending data refill",
+    },
+    {
+        "PublicDescription": "No any micro operation is issued and meanwhile there is any load operation missing both L1 and L2 cache and pending data refill from L3 cache",
+        "EventCode": "0x7007",
+        "EventName": "MEM_STALL_L2MISS",
+        "BriefDescription": "No any micro operation is issued and meanwhile there is any load operation missing both L1 and L2 cache and pending data refill from L3 cache",
+    },
+]
diff --git a/tools/perf/pmu-events/arch/arm64/mapfile.csv b/tools/perf/pmu-events/arch/arm64/mapfile.csv
index 32fa0d1..9cc42da 100644
--- a/tools/perf/pmu-events/arch/arm64/mapfile.csv
+++ b/tools/perf/pmu-events/arch/arm64/mapfile.csv
@@ -13,3 +13,4 @@
 #
 #Family-model,Version,Filename,EventType
 0x00000000420f5160,v1,cavium/thunderx2,core
+0x00000000480fd010,v1,hisilicon/hip08,core
-- 
1.9.1

[RFC PATCH 5/5] perf vendor events arm64: add HiSilicon hip08 JSON

[John Garry]

Add HiSilicon hip08 JSON. Since hip08 has its events
implementated according to ARM recommendation, only add
fields "EventCode" (where applicable - hip08 also has
implemented some other custom events).

Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Shaokun Zhang <zhangshaokun@hisilicon.com>
---
 .../arch/arm64/hisilicon/hip08/core-imp-def.json   | 122 +++++++++++++++++++++
 tools/perf/pmu-events/arch/arm64/mapfile.csv       |   1 +
 2 files changed, 123 insertions(+)
 create mode 100644 tools/perf/pmu-events/arch/arm64/hisilicon/hip08/core-imp-def.json

diff --git a/tools/perf/pmu-events/arch/arm64/hisilicon/hip08/core-imp-def.json b/tools/perf/pmu-events/arch/arm64/hisilicon/hip08/core-imp-def.json
new file mode 100644
index 0000000..94fde40
--- /dev/null
+++ b/tools/perf/pmu-events/arch/arm64/hisilicon/hip08/core-imp-def.json
@@ -0,0 +1,122 @@
+[
+    {
+        "EventCode": "0x40",
+    },
+    {
+        "EventCode": "0x41",
+    },
+    {
+        "EventCode": "0x42",
+    },
+    {
+        "EventCode": "0x43",
+    },
+    {
+        "EventCode": "0x46",
+    },
+    {
+        "EventCode": "0x47",
+    },
+    {
+        "EventCode": "0x48",
+    },
+    {
+        "EventCode": "0x4C",
+    },
+    {
+        "EventCode": "0x4D",
+    },
+    {
+        "EventCode": "0x4E",
+    },
+    {
+        "EventCode": "0x4F",
+    },
+    {
+        "EventCode": "0x50",
+    },
+    {
+        "EventCode": "0x51",
+    },
+    {
+        "EventCode": "0x52",
+    },
+    {
+        "EventCode": "0x53",
+    },
+    {
+        "EventCode": "0x56",
+    },
+    {
+        "EventCode": "0x57",
+    },
+    {
+        "EventCode": "0x58",
+    },
+    {
+        "PublicDescription": "Level 1 instruction cache prefetch access count",
+        "EventCode": "0x102e",
+        "EventName": "L1I_CACHE_PRF",
+        "BriefDescription": "L1I cache prefetch access count",
+    },
+    {
+        "PublicDescription": "Level 1 instruction cache miss due to prefetch access count",
+        "EventCode": "0x102f",
+        "EventName": "L1I_CACHE_PRF_REFILL",
+        "BriefDescription": "L1I cache miss due to prefetch access count",
+    },
+    {
+        "PublicDescription": "Instruction queue is empty",
+        "EventCode": "0x1043",
+        "EventName": "IQ_IS_EMPTY",
+        "BriefDescription": "Instruction queue is empty",
+    },
+    {
+        "PublicDescription": "Instruction fetch stall cycles",
+        "EventCode": "0x1044",
+        "EventName": "IF_IS_STALL",
+        "BriefDescription": "Instruction fetch stall cycles",
+    },
+    {
+        "PublicDescription": "Instructions can receive, but not send",
+        "EventCode": "0x2014",
+        "EventName": "FETCH_BUBBLE",
+        "BriefDescription": "Instructions can receive, but not send",
+    },
+    {
+        "PublicDescription": "Prefetch request from LSU",
+        "EventCode": "0x6013",
+        "EventName": "PRF_REQ",
+        "BriefDescription": "Prefetch request from LSU",
+    },
+    {
+        "PublicDescription": "Hit on prefetched data",
+        "EventCode": "0x6014",
+        "EventName": "HIT_ON_PRF",
+        "BriefDescription": "Hit on prefetched data",
+    },
+    {
+        "PublicDescription": "Cycles of that the number of issuing micro operations are less than 4",
+        "EventCode": "0x7001",
+        "EventName": "EXE_STALL_CYCLE",
+        "BriefDescription": "Cycles of that the number of issue ups are less than 4",
+    },
+    {
+        "PublicDescription": "No any micro operation is issued and meanwhile any load operation is not resolved",
+        "EventCode": "0x7004",
+        "EventName": "MEM_STALL_ANYLOAD",
+        "BriefDescription": "No any micro operation is issued and meanwhile any load operation is not resolved",
+    },
+    {
+        "PublicDescription": "No any micro operation is issued and meanwhile there is any load operation missing L1 cache and pending data refill",
+        "EventCode": "0x7006",
+        "EventName": "MEM_STALL_L1MISS",
+        "BriefDescription": "No any micro operation is issued and meanwhile there is any load operation missing L1 cache and pending data refill",
+    },
+    {
+        "PublicDescription": "No any micro operation is issued and meanwhile there is any load operation missing both L1 and L2 cache and pending data refill from L3 cache",
+        "EventCode": "0x7007",
+        "EventName": "MEM_STALL_L2MISS",
+        "BriefDescription": "No any micro operation is issued and meanwhile there is any load operation missing both L1 and L2 cache and pending data refill from L3 cache",
+    },
+]
diff --git a/tools/perf/pmu-events/arch/arm64/mapfile.csv b/tools/perf/pmu-events/arch/arm64/mapfile.csv
index 32fa0d1..9cc42da 100644
--- a/tools/perf/pmu-events/arch/arm64/mapfile.csv
+++ b/tools/perf/pmu-events/arch/arm64/mapfile.csv
@@ -13,3 +13,4 @@
 #
 #Family-model,Version,Filename,EventType
 0x00000000420f5160,v1,cavium/thunderx2,core
+0x00000000480fd010,v1,hisilicon/hip08,core
-- 
1.9.1

Re: [RFC PATCH 0/5] perf events patches for improved ARM64 support

[William Cohen]

On 12/05/2017 11:13 AM, John Garry wrote:
> This patchset adds support for some perf events features,
> targeted at ARM64, implemented in a generic fashion.
> 
> The two main features are as follows:
> - support for arch/vendor/platform pmu events directory structure
> - support for parsing architecture recommended pmu events
> 
> On the back of these, the Cavium ThunderX2 and HiSilicon hip08
> JSONs are relocated/added.
> 
> TODO: 
> 	- Documentation needs to be updated for recommended
> 	events format in the JSON
> 	- tidy up patches a bit
> 
> John Garry (5):
>   perf jevents: add support for pmu events vendor subdirectory
>   perf jevents: add support for arch recommended events
>   perf vendor events arm64: add armv8 recommended events JSON
>   perf vendor events arm64: relocate thunderx2 JSON
>   perf vendor events arm64: add HiSilicon hip08 JSON
> 
>  .../pmu-events/arch/arm64/armv8-recommended.json   | 452 +++++++++++++++++++++
>  .../arch/arm64/cavium/thunderx2-imp-def.json       |  62 ---
>  .../arch/arm64/cavium/thunderx2/core-imp-def.json  |  32 ++
>  .../arch/arm64/hisilicon/hip08/core-imp-def.json   | 122 ++++++
>  tools/perf/pmu-events/arch/arm64/mapfile.csv       |   3 +-
>  tools/perf/pmu-events/jevents.c                    | 272 ++++++++++++-
>  6 files changed, 859 insertions(+), 84 deletions(-)
>  create mode 100644 tools/perf/pmu-events/arch/arm64/armv8-recommended.json
>  delete mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2-imp-def.json
>  create mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2/core-imp-def.json
>  create mode 100644 tools/perf/pmu-events/arch/arm64/hisilicon/hip08/core-imp-def.json
> 


Hi,

How is this code dealing with the possible different configurations within the aach64 architecture?  For example the Cortex-A53 implementations may exclude the L2 cache and not have L2 cache events implemented?  Is there some way that perf could make use of the PMCEID0 and PMCEID1 register information to accurately get the subset of events the processor supports?  Using this information could eliminate a lot of busy work adding implementation specific descriptions due to slight implementation differences.

-Will Cohen

[RFC PATCH 0/5] perf events patches for improved ARM64 support

[William Cohen]

On 12/05/2017 11:13 AM, John Garry wrote:
> This patchset adds support for some perf events features,
> targeted at ARM64, implemented in a generic fashion.
> 
> The two main features are as follows:
> - support for arch/vendor/platform pmu events directory structure
> - support for parsing architecture recommended pmu events
> 
> On the back of these, the Cavium ThunderX2 and HiSilicon hip08
> JSONs are relocated/added.
> 
> TODO: 
> 	- Documentation needs to be updated for recommended
> 	events format in the JSON
> 	- tidy up patches a bit
> 
> John Garry (5):
>   perf jevents: add support for pmu events vendor subdirectory
>   perf jevents: add support for arch recommended events
>   perf vendor events arm64: add armv8 recommended events JSON
>   perf vendor events arm64: relocate thunderx2 JSON
>   perf vendor events arm64: add HiSilicon hip08 JSON
> 
>  .../pmu-events/arch/arm64/armv8-recommended.json   | 452 +++++++++++++++++++++
>  .../arch/arm64/cavium/thunderx2-imp-def.json       |  62 ---
>  .../arch/arm64/cavium/thunderx2/core-imp-def.json  |  32 ++
>  .../arch/arm64/hisilicon/hip08/core-imp-def.json   | 122 ++++++
>  tools/perf/pmu-events/arch/arm64/mapfile.csv       |   3 +-
>  tools/perf/pmu-events/jevents.c                    | 272 ++++++++++++-
>  6 files changed, 859 insertions(+), 84 deletions(-)
>  create mode 100644 tools/perf/pmu-events/arch/arm64/armv8-recommended.json
>  delete mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2-imp-def.json
>  create mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2/core-imp-def.json
>  create mode 100644 tools/perf/pmu-events/arch/arm64/hisilicon/hip08/core-imp-def.json
> 


Hi,

How is this code dealing with the possible different configurations within the aach64 architecture?  For example the Cortex-A53 implementations may exclude the L2 cache and not have L2 cache events implemented?  Is there some way that perf could make use of the PMCEID0 and PMCEID1 register information to accurately get the subset of events the processor supports?  Using this information could eliminate a lot of busy work adding implementation specific descriptions due to slight implementation differences.

-Will Cohen

Re: [RFC PATCH 0/5] perf events patches for improved ARM64 support

[John Garry]

On 06/12/2017 16:42, William Cohen wrote:
> On 12/05/2017 11:13 AM, John Garry wrote:
>> This patchset adds support for some perf events features,
>> targeted at ARM64, implemented in a generic fashion.
>>
>> The two main features are as follows:
>> - support for arch/vendor/platform pmu events directory structure
>> - support for parsing architecture recommended pmu events
>>
>> On the back of these, the Cavium ThunderX2 and HiSilicon hip08
>> JSONs are relocated/added.
>>
>> TODO:
>> 	- Documentation needs to be updated for recommended
>> 	events format in the JSON
>> 	- tidy up patches a bit
>>
>> John Garry (5):
>>   perf jevents: add support for pmu events vendor subdirectory
>>   perf jevents: add support for arch recommended events
>>   perf vendor events arm64: add armv8 recommended events JSON
>>   perf vendor events arm64: relocate thunderx2 JSON
>>   perf vendor events arm64: add HiSilicon hip08 JSON
>>
>>  .../pmu-events/arch/arm64/armv8-recommended.json   | 452 +++++++++++++++++++++
>>  .../arch/arm64/cavium/thunderx2-imp-def.json       |  62 ---
>>  .../arch/arm64/cavium/thunderx2/core-imp-def.json  |  32 ++
>>  .../arch/arm64/hisilicon/hip08/core-imp-def.json   | 122 ++++++
>>  tools/perf/pmu-events/arch/arm64/mapfile.csv       |   3 +-
>>  tools/perf/pmu-events/jevents.c                    | 272 ++++++++++++-
>>  6 files changed, 859 insertions(+), 84 deletions(-)
>>  create mode 100644 tools/perf/pmu-events/arch/arm64/armv8-recommended.json
>>  delete mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2-imp-def.json
>>  create mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2/core-imp-def.json
>>  create mode 100644 tools/perf/pmu-events/arch/arm64/hisilicon/hip08/core-imp-def.json
>>
>
>
> Hi,
>
> How is this code dealing with the possible different configurations within the aach64 architecture?  For example the Cortex-A53 implementations may exclude the L2 cache and not have L2 cache events implemented?  Is there some way that perf could make use of the PMCEID0 and PMCEID1 register information to accurately get the subset of events the processor supports?  Using this information could eliminate a lot of busy work adding implementation specific descriptions due to slight implementation differences.
>

Hi Will,

Thanks for the input.

So today the PMCEID registers are used in the kernel to detect whether 
common architectural/microarchitectual events are implemented.

Here, in this patchset (specifically 2/5 onwards), we are trying to deal 
with the Recommended (IMPLEMENTATION DEFINED) 
architectural/microarchitectual events.

Cheers,
John

> -Will Cohen
>
> .
>

[RFC PATCH 0/5] perf events patches for improved ARM64 support

[John Garry]

On 06/12/2017 16:42, William Cohen wrote:
> On 12/05/2017 11:13 AM, John Garry wrote:
>> This patchset adds support for some perf events features,
>> targeted at ARM64, implemented in a generic fashion.
>>
>> The two main features are as follows:
>> - support for arch/vendor/platform pmu events directory structure
>> - support for parsing architecture recommended pmu events
>>
>> On the back of these, the Cavium ThunderX2 and HiSilicon hip08
>> JSONs are relocated/added.
>>
>> TODO:
>> 	- Documentation needs to be updated for recommended
>> 	events format in the JSON
>> 	- tidy up patches a bit
>>
>> John Garry (5):
>>   perf jevents: add support for pmu events vendor subdirectory
>>   perf jevents: add support for arch recommended events
>>   perf vendor events arm64: add armv8 recommended events JSON
>>   perf vendor events arm64: relocate thunderx2 JSON
>>   perf vendor events arm64: add HiSilicon hip08 JSON
>>
>>  .../pmu-events/arch/arm64/armv8-recommended.json   | 452 +++++++++++++++++++++
>>  .../arch/arm64/cavium/thunderx2-imp-def.json       |  62 ---
>>  .../arch/arm64/cavium/thunderx2/core-imp-def.json  |  32 ++
>>  .../arch/arm64/hisilicon/hip08/core-imp-def.json   | 122 ++++++
>>  tools/perf/pmu-events/arch/arm64/mapfile.csv       |   3 +-
>>  tools/perf/pmu-events/jevents.c                    | 272 ++++++++++++-
>>  6 files changed, 859 insertions(+), 84 deletions(-)
>>  create mode 100644 tools/perf/pmu-events/arch/arm64/armv8-recommended.json
>>  delete mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2-imp-def.json
>>  create mode 100644 tools/perf/pmu-events/arch/arm64/cavium/thunderx2/core-imp-def.json
>>  create mode 100644 tools/perf/pmu-events/arch/arm64/hisilicon/hip08/core-imp-def.json
>>
>
>
> Hi,
>
> How is this code dealing with the possible different configurations within the aach64 architecture?  For example the Cortex-A53 implementations may exclude the L2 cache and not have L2 cache events implemented?  Is there some way that perf could make use of the PMCEID0 and PMCEID1 register information to accurately get the subset of events the processor supports?  Using this information could eliminate a lot of busy work adding implementation specific descriptions due to slight implementation differences.
>

Hi Will,

Thanks for the input.

So today the PMCEID registers are used in the kernel to detect whether 
common architectural/microarchitectual events are implemented.

Here, in this patchset (specifically 2/5 onwards), we are trying to deal 
with the Recommended (IMPLEMENTATION DEFINED) 
architectural/microarchitectual events.

Cheers,
John

> -Will Cohen
>
> .
>