* [Fuego] LTP: fix the spreadsheet parser script
@ 2017-12-13 8:42 Daniel Sangorrin
2017-12-13 8:42 ` [Fuego] [PATCH 1/2] LTP:ltp_process: dont crash if no sheets are generated Daniel Sangorrin
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Daniel Sangorrin @ 2017-12-13 8:42 UTC (permalink / raw)
To: fuego
Hi Tim,
I noticed that the rtonly spec was failing because it just produces
an rt.log but no spreadsheet. The following patches fix that.
[PATCH 1/2] LTP:ltp_process: dont crash if no sheets are generated
[PATCH 2/2] LTP:ltp_process: parse the rt.log into a new sheet
I am planning to focus on improving LTP soon, please let me know if
you are going to work on it as well to avoid collisions. Also if you
have any ideas about what you want to have changed in LTP let me know.
Best regards,
Daniel
^ permalink raw reply [flat|nested] 7+ messages in thread* [Fuego] [PATCH 1/2] LTP:ltp_process: dont crash if no sheets are generated 2017-12-13 8:42 [Fuego] LTP: fix the spreadsheet parser script Daniel Sangorrin @ 2017-12-13 8:42 ` Daniel Sangorrin 2017-12-14 1:09 ` Bird, Timothy 2017-12-13 8:42 ` [Fuego] [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new sheet Daniel Sangorrin 2017-12-14 1:15 ` [Fuego] LTP: fix the spreadsheet parser script Bird, Timothy 2 siblings, 1 reply; 7+ messages in thread From: Daniel Sangorrin @ 2017-12-13 8:42 UTC (permalink / raw) To: fuego This fixes the following error that occurs when running the rtonly spec, which does not generate any sheet. Traceback (most recent call last): File "ltp_process.py", line 338, in <module> book.save('results.xlsx') File "/usr/lib/pymodules/python2.7/openpyxl/workbook.py", line 232, in save save_workbook(self, filename) File "/usr/lib/pymodules/python2.7/openpyxl/writer/excel.py", line 159, in save_workbook writer.save(filename) File "/usr/lib/pymodules/python2.7/openpyxl/writer/excel.py", line 142, in save self.write_data(archive) File "/usr/lib/pymodules/python2.7/openpyxl/writer/excel.py", line 73, in write_data archive.writestr(ARC_WORKBOOK, write_workbook(self.workbook)) File "/usr/lib/pymodules/python2.7/openpyxl/writer/workbook.py", line 170, in write_workbook SubElement(book_views, 'workbookView', {'activeTab': '%d' % workbook.get_index(workbook.get_active_sheet()), File "/usr/lib/pymodules/python2.7/openpyxl/workbook.py", line 124, in get_active_sheet return self.worksheets[self._active_sheet_index] IndexError: list index out of range Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> --- engine/tests/Functional.LTP/ltp_process.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/engine/tests/Functional.LTP/ltp_process.py b/engine/tests/Functional.LTP/ltp_process.py index 1658943..867ef7a 100644 --- a/engine/tests/Functional.LTP/ltp_process.py +++ b/engine/tests/Functional.LTP/ltp_process.py @@ -88,9 +88,6 @@ def read_output (test_category, test_case): # Open a workbook book = Workbook() -# Delete the default sheet -book.remove_sheet(book.get_sheet_by_name("Sheet")) - # Create style setting before the for loop # This is needed to avoid excel errors (too many font settings) medium_border = Border() @@ -334,6 +331,10 @@ if os.path.exists('pts.log'): # last working sheet pts_set_style(ws) +# if we have added sheets, remove the default one ("Sheet") +sheets = book.get_sheet_names() +if len(sheets) > 1: + book.remove_sheet(book.get_sheet_by_name("Sheet")) + # save the results book.save('results.xlsx') - -- 2.7.4 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [Fuego] [PATCH 1/2] LTP:ltp_process: dont crash if no sheets are generated 2017-12-13 8:42 ` [Fuego] [PATCH 1/2] LTP:ltp_process: dont crash if no sheets are generated Daniel Sangorrin @ 2017-12-14 1:09 ` Bird, Timothy 0 siblings, 0 replies; 7+ messages in thread From: Bird, Timothy @ 2017-12-14 1:09 UTC (permalink / raw) To: Daniel Sangorrin, fuego Applied an pushed. Thanks, -- Tim > -----Original Message----- > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego- > bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin > Sent: Wednesday, December 13, 2017 12:42 AM > To: fuego@lists.linuxfoundation.org > Subject: [Fuego] [PATCH 1/2] LTP:ltp_process: dont crash if no sheets are > generated > > This fixes the following error that occurs when running the rtonly spec, > which does not generate any sheet. > > Traceback (most recent call last): > File "ltp_process.py", line 338, in <module> > book.save('results.xlsx') > File "/usr/lib/pymodules/python2.7/openpyxl/workbook.py", line 232, in > save > save_workbook(self, filename) > File "/usr/lib/pymodules/python2.7/openpyxl/writer/excel.py", line 159, in > save_workbook > writer.save(filename) > File "/usr/lib/pymodules/python2.7/openpyxl/writer/excel.py", line 142, in > save > self.write_data(archive) > File "/usr/lib/pymodules/python2.7/openpyxl/writer/excel.py", line 73, in > write_data > archive.writestr(ARC_WORKBOOK, write_workbook(self.workbook)) > File "/usr/lib/pymodules/python2.7/openpyxl/writer/workbook.py", line > 170, in write_workbook > SubElement(book_views, 'workbookView', {'activeTab': '%d' % > workbook.get_index(workbook.get_active_sheet()), > File "/usr/lib/pymodules/python2.7/openpyxl/workbook.py", line 124, in > get_active_sheet > return self.worksheets[self._active_sheet_index] > IndexError: list index out of range > > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> > --- > engine/tests/Functional.LTP/ltp_process.py | 9 +++++---- > 1 file changed, 5 insertions(+), 4 deletions(-) > > diff --git a/engine/tests/Functional.LTP/ltp_process.py > b/engine/tests/Functional.LTP/ltp_process.py > index 1658943..867ef7a 100644 > --- a/engine/tests/Functional.LTP/ltp_process.py > +++ b/engine/tests/Functional.LTP/ltp_process.py > @@ -88,9 +88,6 @@ def read_output (test_category, test_case): > # Open a workbook > book = Workbook() > > -# Delete the default sheet > -book.remove_sheet(book.get_sheet_by_name("Sheet")) > - > # Create style setting before the for loop > # This is needed to avoid excel errors (too many font settings) > medium_border = Border() > @@ -334,6 +331,10 @@ if os.path.exists('pts.log'): > # last working sheet > pts_set_style(ws) > > +# if we have added sheets, remove the default one ("Sheet") > +sheets = book.get_sheet_names() > +if len(sheets) > 1: > + book.remove_sheet(book.get_sheet_by_name("Sheet")) > + > # save the results > book.save('results.xlsx') > - > -- > 2.7.4 > > > _______________________________________________ > Fuego mailing list > Fuego@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/fuego ^ permalink raw reply [flat|nested] 7+ messages in thread
* [Fuego] [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new sheet 2017-12-13 8:42 [Fuego] LTP: fix the spreadsheet parser script Daniel Sangorrin 2017-12-13 8:42 ` [Fuego] [PATCH 1/2] LTP:ltp_process: dont crash if no sheets are generated Daniel Sangorrin @ 2017-12-13 8:42 ` Daniel Sangorrin 2017-12-14 1:07 ` Bird, Timothy 2017-12-14 1:09 ` Bird, Timothy 2017-12-14 1:15 ` [Fuego] LTP: fix the spreadsheet parser script Bird, Timothy 2 siblings, 2 replies; 7+ messages in thread From: Daniel Sangorrin @ 2017-12-13 8:42 UTC (permalink / raw) To: fuego [Note] I noticed that some tests didn't produce any machine readable format result. This may need to be patched upstream. TODO: add support for the normal parser as well. Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> --- engine/tests/Functional.LTP/ltp_process.py | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/engine/tests/Functional.LTP/ltp_process.py b/engine/tests/Functional.LTP/ltp_process.py index 867ef7a..eb059f4 100644 --- a/engine/tests/Functional.LTP/ltp_process.py +++ b/engine/tests/Functional.LTP/ltp_process.py @@ -331,6 +331,30 @@ if os.path.exists('pts.log'): # last working sheet pts_set_style(ws) +if os.path.exists('rt.log'): + with open('rt.log') as f: + rt_testcase_regex = "^--- Running testcase (.*) ---$" + rt_results_regex = "^\s*Result:\s*(.*)$" + ws = book.create_sheet(title="RT tests") + row = 0 + ws.cell(row=row,column=0).value = "Test" + ws.cell(row=row,column=1).value = "Result" + ws.cell(row=row,column=2).value = "Log" + for line in f: + m = re.match(rt_testcase_regex, line.rstrip()) + if m: + test_case = m.group(1) + row = row + 1 + ws.cell(row=row,column=0).value = test_case + m = re.match(rt_results_regex, line.rstrip()) + if m: + test_result = m.group(1) + ws.cell(row=row,column=1).value = test_result + if ws.cell(row=row,column=2).value: + ws.cell(row=row,column=2).value = ws.cell(row=row,column=2).value + line + else: + ws.cell(row=row,column=2).value = line + # if we have added sheets, remove the default one ("Sheet") sheets = book.get_sheet_names() if len(sheets) > 1: -- 2.7.4 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [Fuego] [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new sheet 2017-12-13 8:42 ` [Fuego] [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new sheet Daniel Sangorrin @ 2017-12-14 1:07 ` Bird, Timothy 2017-12-14 1:09 ` Bird, Timothy 1 sibling, 0 replies; 7+ messages in thread From: Bird, Timothy @ 2017-12-14 1:07 UTC (permalink / raw) To: Daniel Sangorrin, fuego > -----Original Message----- > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego- > bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin > Sent: Wednesday, December 13, 2017 12:42 AM > To: fuego@lists.linuxfoundation.org > Subject: [Fuego] [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new > sheet > > [Note] I noticed that some tests didn't produce any machine > readable format result. This may need to be patched upstream. See my results below. > > TODO: add support for the normal parser as well. That would be much appreciated! > > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> > --- > engine/tests/Functional.LTP/ltp_process.py | 24 > ++++++++++++++++++++++++ > 1 file changed, 24 insertions(+) > > diff --git a/engine/tests/Functional.LTP/ltp_process.py > b/engine/tests/Functional.LTP/ltp_process.py > index 867ef7a..eb059f4 100644 > --- a/engine/tests/Functional.LTP/ltp_process.py > +++ b/engine/tests/Functional.LTP/ltp_process.py > @@ -331,6 +331,30 @@ if os.path.exists('pts.log'): > # last working sheet > pts_set_style(ws) > > +if os.path.exists('rt.log'): > + with open('rt.log') as f: > + rt_testcase_regex = "^--- Running testcase (.*) ---$" > + rt_results_regex = "^\s*Result:\s*(.*)$" > + ws = book.create_sheet(title="RT tests") > + row = 0 > + ws.cell(row=row,column=0).value = "Test" > + ws.cell(row=row,column=1).value = "Result" > + ws.cell(row=row,column=2).value = "Log" > + for line in f: > + m = re.match(rt_testcase_regex, line.rstrip()) > + if m: > + test_case = m.group(1) > + row = row + 1 > + ws.cell(row=row,column=0).value = test_case > + m = re.match(rt_results_regex, line.rstrip()) > + if m: > + test_result = m.group(1) > + ws.cell(row=row,column=1).value = test_result > + if ws.cell(row=row,column=2).value: > + ws.cell(row=row,column=2).value = > ws.cell(row=row,column=2).value + line > + else: > + ws.cell(row=row,column=2).value = line > + > # if we have added sheets, remove the default one ("Sheet") > sheets = book.get_sheet_names() > if len(sheets) > 1: > -- > 2.7.4 Not all results are reported. Here are my results for one of my boards: (in CSV format) rt-migrate,PASS sbrk_mutex, sched_jitter, sched_latency -d 1 -t 5 -c 100,PASS testpi-0 , testpi-1,PASS testpi-2,PASS testpi-4,PASS testpi-5,PASS testpi-6,PASS However, this is a big improvement over not having anything in the spreadsheet for these tests. Both patches applied - thanks! -- Tim ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Fuego] [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new sheet 2017-12-13 8:42 ` [Fuego] [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new sheet Daniel Sangorrin 2017-12-14 1:07 ` Bird, Timothy @ 2017-12-14 1:09 ` Bird, Timothy 1 sibling, 0 replies; 7+ messages in thread From: Bird, Timothy @ 2017-12-14 1:09 UTC (permalink / raw) To: Daniel Sangorrin, fuego Applied and pushed. Thanks, -- Tim > -----Original Message----- > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego- > bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin > Sent: Wednesday, December 13, 2017 12:42 AM > To: fuego@lists.linuxfoundation.org > Subject: [Fuego] [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new > sheet > > [Note] I noticed that some tests didn't produce any machine > readable format result. This may need to be patched upstream. > > TODO: add support for the normal parser as well. > > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> > --- > engine/tests/Functional.LTP/ltp_process.py | 24 > ++++++++++++++++++++++++ > 1 file changed, 24 insertions(+) > > diff --git a/engine/tests/Functional.LTP/ltp_process.py > b/engine/tests/Functional.LTP/ltp_process.py > index 867ef7a..eb059f4 100644 > --- a/engine/tests/Functional.LTP/ltp_process.py > +++ b/engine/tests/Functional.LTP/ltp_process.py > @@ -331,6 +331,30 @@ if os.path.exists('pts.log'): > # last working sheet > pts_set_style(ws) > > +if os.path.exists('rt.log'): > + with open('rt.log') as f: > + rt_testcase_regex = "^--- Running testcase (.*) ---$" > + rt_results_regex = "^\s*Result:\s*(.*)$" > + ws = book.create_sheet(title="RT tests") > + row = 0 > + ws.cell(row=row,column=0).value = "Test" > + ws.cell(row=row,column=1).value = "Result" > + ws.cell(row=row,column=2).value = "Log" > + for line in f: > + m = re.match(rt_testcase_regex, line.rstrip()) > + if m: > + test_case = m.group(1) > + row = row + 1 > + ws.cell(row=row,column=0).value = test_case > + m = re.match(rt_results_regex, line.rstrip()) > + if m: > + test_result = m.group(1) > + ws.cell(row=row,column=1).value = test_result > + if ws.cell(row=row,column=2).value: > + ws.cell(row=row,column=2).value = > ws.cell(row=row,column=2).value + line > + else: > + ws.cell(row=row,column=2).value = line > + > # if we have added sheets, remove the default one ("Sheet") > sheets = book.get_sheet_names() > if len(sheets) > 1: > -- > 2.7.4 > > > _______________________________________________ > Fuego mailing list > Fuego@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/fuego ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Fuego] LTP: fix the spreadsheet parser script 2017-12-13 8:42 [Fuego] LTP: fix the spreadsheet parser script Daniel Sangorrin 2017-12-13 8:42 ` [Fuego] [PATCH 1/2] LTP:ltp_process: dont crash if no sheets are generated Daniel Sangorrin 2017-12-13 8:42 ` [Fuego] [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new sheet Daniel Sangorrin @ 2017-12-14 1:15 ` Bird, Timothy 2 siblings, 0 replies; 7+ messages in thread From: Bird, Timothy @ 2017-12-14 1:15 UTC (permalink / raw) To: Daniel Sangorrin, fuego > -----Original Message----- > From: Daniel Sangorrin on Wednesday, December 13, 2017 12:42 AM > I noticed that the rtonly spec was failing because it just produces > an rt.log but no spreadsheet. The following patches fix that. Thanks - this is good stuff. > [PATCH 1/2] LTP:ltp_process: dont crash if no sheets are generated > [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new sheet > > I am planning to focus on improving LTP soon, please let me know if > you are going to work on it as well to avoid collisions. Also if you > have any ideas about what you want to have changed in LTP let me know. I don't plan on working on LTP in the short term. Feel free to improve things, as it's currently kind of a mess. I think it might be good to convert LTP back into separate tests: Functional.LTP-posix, Functional.LTP-core, Functional.LTP-realtime. They would share some common base functions, like netperf or OpenSSL. I'd have to revisit the build-directory sharing that I worked on recently, but I think this is needed. One issue I'm seeing is that with all the different specs, it's almost impossible to settle in on a criteria.json file that covers them. This is true for other tests, but much more so for super-tests like LTP, where the spec can have it run basically an entire different test suite. Let me know what you think. Thanks for the patches. -- Tim ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-12-14 1:15 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-12-13 8:42 [Fuego] LTP: fix the spreadsheet parser script Daniel Sangorrin 2017-12-13 8:42 ` [Fuego] [PATCH 1/2] LTP:ltp_process: dont crash if no sheets are generated Daniel Sangorrin 2017-12-14 1:09 ` Bird, Timothy 2017-12-13 8:42 ` [Fuego] [PATCH 2/2] LTP:ltp_process: parse the rt.log into a new sheet Daniel Sangorrin 2017-12-14 1:07 ` Bird, Timothy 2017-12-14 1:09 ` Bird, Timothy 2017-12-14 1:15 ` [Fuego] LTP: fix the spreadsheet parser script Bird, Timothy
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.