[PATCH 0/2] perf script python: Database export fixes

[PATCH 0/2] perf script python: Database export fixes

[Adrian Hunter]

Hi

Here are a couple of database export fixes.


Adrian Hunter (2):
      perf script python: Fix export-to-postgresql.py occasional failure
      perf script python: Fix export-to-sqlite.py sample columns

 tools/perf/scripts/python/export-to-postgresql.py | 9 +++++++++
 tools/perf/scripts/python/export-to-sqlite.py     | 6 +++++-
 2 files changed, 14 insertions(+), 1 deletion(-)


Regards
Adrian

[PATCH 1/2] perf script python: Fix export-to-postgresql.py occasional failure

[Adrian Hunter]

Occasional export failures were found to be caused by truncating 64-bit
pointers to 32-bits. Fix by explicitly setting types for all ctype
arguments and results.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: stable@vger.kernel.org
---
 tools/perf/scripts/python/export-to-postgresql.py | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/tools/perf/scripts/python/export-to-postgresql.py b/tools/perf/scripts/python/export-to-postgresql.py
index efcaf6cac2eb..e46f51b17513 100644
--- a/tools/perf/scripts/python/export-to-postgresql.py
+++ b/tools/perf/scripts/python/export-to-postgresql.py
@@ -204,14 +204,23 @@ from ctypes import *
 libpq = CDLL("libpq.so.5")
 PQconnectdb = libpq.PQconnectdb
 PQconnectdb.restype = c_void_p
+PQconnectdb.argtypes = [ c_char_p ]
 PQfinish = libpq.PQfinish
+PQfinish.argtypes = [ c_void_p ]
 PQstatus = libpq.PQstatus
+PQstatus.restype = c_int
+PQstatus.argtypes = [ c_void_p ]
 PQexec = libpq.PQexec
 PQexec.restype = c_void_p
+PQexec.argtypes = [ c_void_p, c_char_p ]
 PQresultStatus = libpq.PQresultStatus
+PQresultStatus.restype = c_int
+PQresultStatus.argtypes = [ c_void_p ]
 PQputCopyData = libpq.PQputCopyData
+PQputCopyData.restype = c_int
 PQputCopyData.argtypes = [ c_void_p, c_void_p, c_int ]
 PQputCopyEnd = libpq.PQputCopyEnd
+PQputCopyEnd.restype = c_int
 PQputCopyEnd.argtypes = [ c_void_p, c_void_p ]
 
 sys.path.append(os.environ['PERF_EXEC_PATH'] + \
-- 
2.17.1

[PATCH 2/2] perf script python: Fix export-to-sqlite.py sample columns

[Adrian Hunter]

With the "branches" export option, not all sample columns are exported.
However the unwanted columns are not at the end of the tuple, as assumed by
the code. Fix by taking the first 15 and last 3 values, instead of the
first 18.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: stable@vger.kernel.org
---
 tools/perf/scripts/python/export-to-sqlite.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/tools/perf/scripts/python/export-to-sqlite.py b/tools/perf/scripts/python/export-to-sqlite.py
index f827bf77e9d2..e4bb82c8aba9 100644
--- a/tools/perf/scripts/python/export-to-sqlite.py
+++ b/tools/perf/scripts/python/export-to-sqlite.py
@@ -440,7 +440,11 @@ def branch_type_table(*x):
 
 def sample_table(*x):
 	if branches:
-		bind_exec(sample_query, 18, x)
+		for xx in x[0:15]:
+			sample_query.addBindValue(str(xx))
+		for xx in x[19:22]:
+			sample_query.addBindValue(str(xx))
+		do_query_(sample_query)
 	else:
 		bind_exec(sample_query, 22, x)
 
-- 
2.17.1

Re: [PATCH 0/2] perf script python: Database export fixes

[Adrian Hunter]

On 11/09/18 14:45, Adrian Hunter wrote:
> Hi
> 
> Here are a couple of database export fixes.
> 
> 
> Adrian Hunter (2):
>       perf script python: Fix export-to-postgresql.py occasional failure
>       perf script python: Fix export-to-sqlite.py sample columns
> 
>  tools/perf/scripts/python/export-to-postgresql.py | 9 +++++++++
>  tools/perf/scripts/python/export-to-sqlite.py     | 6 +++++-
>  2 files changed, 14 insertions(+), 1 deletion(-)

Please consider these

Re: [PATCH 0/2] perf script python: Database export fixes

[Arnaldo Carvalho de Melo]

Em Tue, Sep 25, 2018 at 02:08:06PM +0300, Adrian Hunter escreveu:
> On 11/09/18 14:45, Adrian Hunter wrote:
> > Hi
> > 
> > Here are a couple of database export fixes.
> > 
> > 
> > Adrian Hunter (2):
> >       perf script python: Fix export-to-postgresql.py occasional failure
> >       perf script python: Fix export-to-sqlite.py sample columns
> > 
> >  tools/perf/scripts/python/export-to-postgresql.py | 9 +++++++++
> >  tools/perf/scripts/python/export-to-sqlite.py     | 6 +++++-
> >  2 files changed, 14 insertions(+), 1 deletion(-)
> 
> Please consider these

Applied to perf/urgent, please consider using the Fixes: tag, so that
the stable guys have a easier time (automatically even) figuring out how
far back the patch should be applied.

Thanks!

- Arnaldo

[tip:perf/urgent] perf script python: Fix export-to-postgresql.py occasional failure

[tip-bot for Adrian Hunter]

Commit-ID:  25e11700b54c7b6b5ebfc4361981dae12299557b
Gitweb:     https://git.kernel.org/tip/25e11700b54c7b6b5ebfc4361981dae12299557b
Author:     Adrian Hunter <adrian.hunter@intel.com>
AuthorDate: Tue, 11 Sep 2018 14:45:03 +0300
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Tue, 25 Sep 2018 11:33:06 -0300

perf script python: Fix export-to-postgresql.py occasional failure

Occasional export failures were found to be caused by truncating 64-bit
pointers to 32-bits. Fix by explicitly setting types for all ctype
arguments and results.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: stable@vger.kernel.org
Link: http://lkml.kernel.org/r/20180911114504.28516-2-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/scripts/python/export-to-postgresql.py | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/tools/perf/scripts/python/export-to-postgresql.py b/tools/perf/scripts/python/export-to-postgresql.py
index efcaf6cac2eb..e46f51b17513 100644
--- a/tools/perf/scripts/python/export-to-postgresql.py
+++ b/tools/perf/scripts/python/export-to-postgresql.py
@@ -204,14 +204,23 @@ from ctypes import *
 libpq = CDLL("libpq.so.5")
 PQconnectdb = libpq.PQconnectdb
 PQconnectdb.restype = c_void_p
+PQconnectdb.argtypes = [ c_char_p ]
 PQfinish = libpq.PQfinish
+PQfinish.argtypes = [ c_void_p ]
 PQstatus = libpq.PQstatus
+PQstatus.restype = c_int
+PQstatus.argtypes = [ c_void_p ]
 PQexec = libpq.PQexec
 PQexec.restype = c_void_p
+PQexec.argtypes = [ c_void_p, c_char_p ]
 PQresultStatus = libpq.PQresultStatus
+PQresultStatus.restype = c_int
+PQresultStatus.argtypes = [ c_void_p ]
 PQputCopyData = libpq.PQputCopyData
+PQputCopyData.restype = c_int
 PQputCopyData.argtypes = [ c_void_p, c_void_p, c_int ]
 PQputCopyEnd = libpq.PQputCopyEnd
+PQputCopyEnd.restype = c_int
 PQputCopyEnd.argtypes = [ c_void_p, c_void_p ]
 
 sys.path.append(os.environ['PERF_EXEC_PATH'] + \

[tip:perf/urgent] perf script python: Fix export-to-sqlite.py sample columns

[tip-bot for Adrian Hunter]

Commit-ID:  d005efe18db0b4a123dd92ea8e77e27aee8f99fd
Gitweb:     https://git.kernel.org/tip/d005efe18db0b4a123dd92ea8e77e27aee8f99fd
Author:     Adrian Hunter <adrian.hunter@intel.com>
AuthorDate: Tue, 11 Sep 2018 14:45:04 +0300
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Tue, 25 Sep 2018 11:37:05 -0300

perf script python: Fix export-to-sqlite.py sample columns

With the "branches" export option, not all sample columns are exported.
However the unwanted columns are not at the end of the tuple, as assumed
by the code. Fix by taking the first 15 and last 3 values, instead of
the first 18.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: stable@vger.kernel.org
Link: http://lkml.kernel.org/r/20180911114504.28516-3-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/scripts/python/export-to-sqlite.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/tools/perf/scripts/python/export-to-sqlite.py b/tools/perf/scripts/python/export-to-sqlite.py
index f827bf77e9d2..e4bb82c8aba9 100644
--- a/tools/perf/scripts/python/export-to-sqlite.py
+++ b/tools/perf/scripts/python/export-to-sqlite.py
@@ -440,7 +440,11 @@ def branch_type_table(*x):
 
 def sample_table(*x):
 	if branches:
-		bind_exec(sample_query, 18, x)
+		for xx in x[0:15]:
+			sample_query.addBindValue(str(xx))
+		for xx in x[19:22]:
+			sample_query.addBindValue(str(xx))
+		do_query_(sample_query)
 	else:
 		bind_exec(sample_query, 22, x)