[Fuego] LTP: fix the spreadsheet parser script

[Fuego] LTP: fix the spreadsheet parser script

[Daniel Sangorrin]

Hi Tim,

I noticed that the rtonly spec was failing because it just produces
an rt.log but no spreadsheet. The following patches fix that.

[PATCH 1/2] LTP:ltp_process: dont crash if no sheets are generated
[PATCH 2/2] LTP:ltp_process: parse the rt.log into a new sheet

I am planning to focus on improving LTP soon, please let me know if
you are going to work on it as well to avoid collisions. Also if you
have any ideas about what you want to have changed in LTP let me know.

Best regards,
Daniel

[Fuego] [PATCH 1/2] LTP:ltp_process: dont crash if no sheets are generated

[Daniel Sangorrin]

This fixes the following error that occurs when running the rtonly spec,
which does not generate any sheet.

Traceback (most recent call last):
  File "ltp_process.py", line 338, in <module>
    book.save('results.xlsx')
  File "/usr/lib/pymodules/python2.7/openpyxl/workbook.py", line 232, in save
    save_workbook(self, filename)
  File "/usr/lib/pymodules/python2.7/openpyxl/writer/excel.py", line 159, in save_workbook
    writer.save(filename)
  File "/usr/lib/pymodules/python2.7/openpyxl/writer/excel.py", line 142, in save
    self.write_data(archive)
  File "/usr/lib/pymodules/python2.7/openpyxl/writer/excel.py", line 73, in write_data
    archive.writestr(ARC_WORKBOOK, write_workbook(self.workbook))
  File "/usr/lib/pymodules/python2.7/openpyxl/writer/workbook.py", line 170, in write_workbook
    SubElement(book_views, 'workbookView', {'activeTab': '%d' % workbook.get_index(workbook.get_active_sheet()),
  File "/usr/lib/pymodules/python2.7/openpyxl/workbook.py", line 124, in get_active_sheet
    return self.worksheets[self._active_sheet_index]
IndexError: list index out of range

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/tests/Functional.LTP/ltp_process.py | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/engine/tests/Functional.LTP/ltp_process.py b/engine/tests/Functional.LTP/ltp_process.py
index 1658943..867ef7a 100644
--- a/engine/tests/Functional.LTP/ltp_process.py
+++ b/engine/tests/Functional.LTP/ltp_process.py
@@ -88,9 +88,6 @@ def read_output (test_category, test_case):
 # Open a workbook
 book = Workbook()
 
-# Delete the default sheet
-book.remove_sheet(book.get_sheet_by_name("Sheet"))
-
 # Create style setting before the for loop
 # This is needed to avoid excel errors (too many font settings)
 medium_border = Border()
@@ -334,6 +331,10 @@ if os.path.exists('pts.log'):
         # last working sheet
         pts_set_style(ws)
 
+# if we have added sheets, remove the default one ("Sheet")
+sheets = book.get_sheet_names()
+if len(sheets) > 1:
+    book.remove_sheet(book.get_sheet_by_name("Sheet"))
+
 # save the results
 book.save('results.xlsx')
-
-- 
2.7.4

[Fuego] [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new sheet

[Daniel Sangorrin]

[Note] I noticed that some tests didn't produce any machine
readable format result. This may need to be patched upstream.

TODO: add support for the normal parser as well.

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/tests/Functional.LTP/ltp_process.py | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/engine/tests/Functional.LTP/ltp_process.py b/engine/tests/Functional.LTP/ltp_process.py
index 867ef7a..eb059f4 100644
--- a/engine/tests/Functional.LTP/ltp_process.py
+++ b/engine/tests/Functional.LTP/ltp_process.py
@@ -331,6 +331,30 @@ if os.path.exists('pts.log'):
         # last working sheet
         pts_set_style(ws)
 
+if os.path.exists('rt.log'):
+    with open('rt.log') as f:
+        rt_testcase_regex = "^--- Running testcase (.*)  ---$"
+        rt_results_regex = "^\s*Result:\s*(.*)$"
+        ws = book.create_sheet(title="RT tests")
+        row = 0
+        ws.cell(row=row,column=0).value = "Test"
+        ws.cell(row=row,column=1).value = "Result"
+        ws.cell(row=row,column=2).value = "Log"
+        for line in f:
+            m = re.match(rt_testcase_regex, line.rstrip())
+            if m:
+                test_case = m.group(1)
+                row = row + 1
+                ws.cell(row=row,column=0).value = test_case
+            m = re.match(rt_results_regex, line.rstrip())
+            if m:
+                test_result = m.group(1)
+                ws.cell(row=row,column=1).value = test_result
+            if ws.cell(row=row,column=2).value:
+                ws.cell(row=row,column=2).value = ws.cell(row=row,column=2).value + line
+            else:
+                ws.cell(row=row,column=2).value = line
+
 # if we have added sheets, remove the default one ("Sheet")
 sheets = book.get_sheet_names()
 if len(sheets) > 1:
-- 
2.7.4

Re: [Fuego] [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new sheet

[Bird, Timothy]

> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Wednesday, December 13, 2017 12:42 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new
> sheet
> 
> [Note] I noticed that some tests didn't produce any machine
> readable format result. This may need to be patched upstream.


See my results below.

> 
> TODO: add support for the normal parser as well.
That would be much appreciated!

> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  engine/tests/Functional.LTP/ltp_process.py | 24
> ++++++++++++++++++++++++
>  1 file changed, 24 insertions(+)
> 
> diff --git a/engine/tests/Functional.LTP/ltp_process.py
> b/engine/tests/Functional.LTP/ltp_process.py
> index 867ef7a..eb059f4 100644
> --- a/engine/tests/Functional.LTP/ltp_process.py
> +++ b/engine/tests/Functional.LTP/ltp_process.py
> @@ -331,6 +331,30 @@ if os.path.exists('pts.log'):
>          # last working sheet
>          pts_set_style(ws)
> 
> +if os.path.exists('rt.log'):
> +    with open('rt.log') as f:
> +        rt_testcase_regex = "^--- Running testcase (.*)  ---$"
> +        rt_results_regex = "^\s*Result:\s*(.*)$"
> +        ws = book.create_sheet(title="RT tests")
> +        row = 0
> +        ws.cell(row=row,column=0).value = "Test"
> +        ws.cell(row=row,column=1).value = "Result"
> +        ws.cell(row=row,column=2).value = "Log"
> +        for line in f:
> +            m = re.match(rt_testcase_regex, line.rstrip())
> +            if m:
> +                test_case = m.group(1)
> +                row = row + 1
> +                ws.cell(row=row,column=0).value = test_case
> +            m = re.match(rt_results_regex, line.rstrip())
> +            if m:
> +                test_result = m.group(1)
> +                ws.cell(row=row,column=1).value = test_result
> +            if ws.cell(row=row,column=2).value:
> +                ws.cell(row=row,column=2).value =
> ws.cell(row=row,column=2).value + line
> +            else:
> +                ws.cell(row=row,column=2).value = line
> +
>  # if we have added sheets, remove the default one ("Sheet")
>  sheets = book.get_sheet_names()
>  if len(sheets) > 1:
> --
> 2.7.4

Not all results are reported.  Here are my results for one of my boards:
(in CSV format)
rt-migrate,PASS
sbrk_mutex,	
sched_jitter,	
sched_latency -d 1 -t 5 -c 100,PASS
testpi-0	,
testpi-1,PASS
testpi-2,PASS
testpi-4,PASS
testpi-5,PASS
testpi-6,PASS

However, this is a big  improvement over not having anything in the spreadsheet
for these tests.

Both patches applied - thanks!
 -- Tim

Re: [Fuego] [PATCH 1/2] LTP:ltp_process: dont crash if no sheets are generated

[Bird, Timothy]

Applied an pushed.

Thanks,
 -- Tim

> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Wednesday, December 13, 2017 12:42 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 1/2] LTP:ltp_process: dont crash if no sheets are
> generated
> 
> This fixes the following error that occurs when running the rtonly spec,
> which does not generate any sheet.
> 
> Traceback (most recent call last):
>   File "ltp_process.py", line 338, in <module>
>     book.save('results.xlsx')
>   File "/usr/lib/pymodules/python2.7/openpyxl/workbook.py", line 232, in
> save
>     save_workbook(self, filename)
>   File "/usr/lib/pymodules/python2.7/openpyxl/writer/excel.py", line 159, in
> save_workbook
>     writer.save(filename)
>   File "/usr/lib/pymodules/python2.7/openpyxl/writer/excel.py", line 142, in
> save
>     self.write_data(archive)
>   File "/usr/lib/pymodules/python2.7/openpyxl/writer/excel.py", line 73, in
> write_data
>     archive.writestr(ARC_WORKBOOK, write_workbook(self.workbook))
>   File "/usr/lib/pymodules/python2.7/openpyxl/writer/workbook.py", line
> 170, in write_workbook
>     SubElement(book_views, 'workbookView', {'activeTab': '%d' %
> workbook.get_index(workbook.get_active_sheet()),
>   File "/usr/lib/pymodules/python2.7/openpyxl/workbook.py", line 124, in
> get_active_sheet
>     return self.worksheets[self._active_sheet_index]
> IndexError: list index out of range
> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  engine/tests/Functional.LTP/ltp_process.py | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/engine/tests/Functional.LTP/ltp_process.py
> b/engine/tests/Functional.LTP/ltp_process.py
> index 1658943..867ef7a 100644
> --- a/engine/tests/Functional.LTP/ltp_process.py
> +++ b/engine/tests/Functional.LTP/ltp_process.py
> @@ -88,9 +88,6 @@ def read_output (test_category, test_case):
>  # Open a workbook
>  book = Workbook()
> 
> -# Delete the default sheet
> -book.remove_sheet(book.get_sheet_by_name("Sheet"))
> -
>  # Create style setting before the for loop
>  # This is needed to avoid excel errors (too many font settings)
>  medium_border = Border()
> @@ -334,6 +331,10 @@ if os.path.exists('pts.log'):
>          # last working sheet
>          pts_set_style(ws)
> 
> +# if we have added sheets, remove the default one ("Sheet")
> +sheets = book.get_sheet_names()
> +if len(sheets) > 1:
> +    book.remove_sheet(book.get_sheet_by_name("Sheet"))
> +
>  # save the results
>  book.save('results.xlsx')
> -
> --
> 2.7.4
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new sheet

[Bird, Timothy]

Applied and pushed.

Thanks,
 -- Tim

> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Wednesday, December 13, 2017 12:42 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new
> sheet
> 
> [Note] I noticed that some tests didn't produce any machine
> readable format result. This may need to be patched upstream.
> 
> TODO: add support for the normal parser as well.
> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  engine/tests/Functional.LTP/ltp_process.py | 24
> ++++++++++++++++++++++++
>  1 file changed, 24 insertions(+)
> 
> diff --git a/engine/tests/Functional.LTP/ltp_process.py
> b/engine/tests/Functional.LTP/ltp_process.py
> index 867ef7a..eb059f4 100644
> --- a/engine/tests/Functional.LTP/ltp_process.py
> +++ b/engine/tests/Functional.LTP/ltp_process.py
> @@ -331,6 +331,30 @@ if os.path.exists('pts.log'):
>          # last working sheet
>          pts_set_style(ws)
> 
> +if os.path.exists('rt.log'):
> +    with open('rt.log') as f:
> +        rt_testcase_regex = "^--- Running testcase (.*)  ---$"
> +        rt_results_regex = "^\s*Result:\s*(.*)$"
> +        ws = book.create_sheet(title="RT tests")
> +        row = 0
> +        ws.cell(row=row,column=0).value = "Test"
> +        ws.cell(row=row,column=1).value = "Result"
> +        ws.cell(row=row,column=2).value = "Log"
> +        for line in f:
> +            m = re.match(rt_testcase_regex, line.rstrip())
> +            if m:
> +                test_case = m.group(1)
> +                row = row + 1
> +                ws.cell(row=row,column=0).value = test_case
> +            m = re.match(rt_results_regex, line.rstrip())
> +            if m:
> +                test_result = m.group(1)
> +                ws.cell(row=row,column=1).value = test_result
> +            if ws.cell(row=row,column=2).value:
> +                ws.cell(row=row,column=2).value =
> ws.cell(row=row,column=2).value + line
> +            else:
> +                ws.cell(row=row,column=2).value = line
> +
>  # if we have added sheets, remove the default one ("Sheet")
>  sheets = book.get_sheet_names()
>  if len(sheets) > 1:
> --
> 2.7.4
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] LTP: fix the spreadsheet parser script

[Bird, Timothy]

> -----Original Message-----
> From: Daniel Sangorrin on Wednesday, December 13, 2017 12:42 AM
> I noticed that the rtonly spec was failing because it just produces
> an rt.log but no spreadsheet. The following patches fix that.

Thanks - this is good stuff.

> [PATCH 1/2] LTP:ltp_process: dont crash if no sheets are generated
> [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new sheet
> 
> I am planning to focus on improving LTP soon, please let me know if
> you are going to work on it as well to avoid collisions. Also if you
> have any ideas about what you want to have changed in LTP let me know.

I don't plan on working on LTP in the short term.  Feel free to improve things,
as it's currently kind of a mess.

I think it might be good to convert LTP back into separate tests:
Functional.LTP-posix, Functional.LTP-core, Functional.LTP-realtime.
They would share some common base functions, like netperf or
OpenSSL.  I'd have to revisit the build-directory sharing that I worked
on recently, but I think this is needed.  One issue I'm seeing is that
with all the different specs, it's almost impossible to settle in on a
criteria.json file that covers them.  This is true for other tests, but much
more so for super-tests like LTP, where the spec can have it run basically
an entire different test suite.

Let me know what you think.

Thanks for the patches.
 -- Tim