[Fuego] Split the output and add log links for each testcase

[Fuego] Split the output and add log links for each testcase

[Liu, Wenlong]

[-- Attachment #1: Type: text/plain, Size: 567 bytes --]

Hi Tim, Daniel and other friends

Functional.LTP has heavy outputs after testing, which makes it difficult for us to locate the error log easily.

LTP has the feature that can help us to split the output for different testcases.
So, I added some links to those separated log files in the test case summary page.
It helped me a lot to investigate different failures.

Now, I also added a generic function to split the output for different tests and try to apply it to different tests.
Please review the patch[01-09] in the maillist.

Best regards
Liu




[-- Attachment #2: Type: text/html, Size: 4595 bytes --]

Re: [Fuego] Split the output and add log links for each testcase

[Tim.Bird]

> -----Original Message-----
> From: Liu, Wenlong
> Sent: Tuesday, January 30, 2018 11:58 PM
> Hi Tim, Daniel and other friends
> 
> Functional.LTP has heavy outputs after testing, which makes it difficult for us
> to locate the error log easily.
> 
> LTP has the feature that can help us to split the output for different
> testcases.
> 
> So, I added some links to those separated log files in the test case summary
> page.
> 
> It helped me a lot to investigate different failures.

I like this a lot.  Fuego has not had very good facilities to isolate the
error information for each testcase, and this looks like it will do it.
 
> 
> Now, I also added a generic function to split the output for different tests
> and try to apply it to different tests.
> 
> Please review the patch[01-09] in the maillist.

I will do what I can tonight, but I probably won't be able to finish
before I stop working today.  I'll have to finish up later this week.
 -- Tim

Re: [Fuego] Split the output and add log links for each testcase

[Daniel Sangorrin]

> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-bounces@lists.linuxfoundation.org] On Behalf Of Tim.Bird@sony.com
> Sent: Thursday, February 01, 2018 10:30 AM
> To: liuwl.fnst@cn.fujitsu.com; fuego@lists.linuxfoundation.org
> Subject: Re: [Fuego] Split the output and add log links for each testcase
> 
> > -----Original Message-----
> > From: Liu, Wenlong
> > Sent: Tuesday, January 30, 2018 11:58 PM
> > Hi Tim, Daniel and other friends
> >
> > Functional.LTP has heavy outputs after testing, which makes it difficult for us
> > to locate the error log easily.
> >
> > LTP has the feature that can help us to split the output for different
> > testcases.
> >
> > So, I added some links to those separated log files in the test case summary
> > page.
> >
> > It helped me a lot to investigate different failures.
> 
> I like this a lot.  Fuego has not had very good facilities to isolate the
> error information for each testcase, and this looks like it will do it.

I think that separate log files are good. But if you patch LTP please keep in mind that
test case error logs are already separated by the parser. In LTP, the error log for 
non-successful test cases is in the corresponding row on the output excel sheet.
If you double-click on the cell, you will see the complete error log for that specific test case. 
# Next, I am planning to add another column with a guess on why the test case failed.

By the way, normally what I do to investigate a failure is:
  - set the target_postCleanup flag to False
  - ssh to the target
  - execute the test case individually with runltp
     # ./runltp -f syscalls -s madvise07
  - or execute the test directly with strace
    # strace bin/madvise07
    [Note] the arguments required for the test can be obtained from the test group inside runtests/ (e.g. syscalls file)

Thanks,
Daniel

> > Now, I also added a generic function to split the output for different tests
> > and try to apply it to different tests.
> >
> > Please review the patch[01-09] in the maillist.
> 
> I will do what I can tonight, but I probably won't be able to finish
> before I stop working today.  I'll have to finish up later this week.
>  -- Tim
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] Split the output and add log links for each testcase

[Liu, Wenlong]

> -----Original Message-----
> From: Daniel Sangorrin [mailto:daniel.sangorrin@toshiba.co.jp]
> Sent: Thursday, February 01, 2018 9:49 AM
> To: Tim.Bird@sony.com; Liu, Wenlong/刘 文龙 <liuwl.fnst@cn.fujitsu.com>;
> fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] Split the output and add log links for each testcase
> 
> 
> 
> > -----Original Message-----
> > From: fuego-bounces@lists.linuxfoundation.org
> > [mailto:fuego-bounces@lists.linuxfoundation.org] On Behalf Of
> > Tim.Bird@sony.com
> > Sent: Thursday, February 01, 2018 10:30 AM
> > To: liuwl.fnst@cn.fujitsu.com; fuego@lists.linuxfoundation.org
> > Subject: Re: [Fuego] Split the output and add log links for each
> > testcase
> >
> > > -----Original Message-----
> > > From: Liu, Wenlong
> > > Sent: Tuesday, January 30, 2018 11:58 PM Hi Tim, Daniel and other
> > > friends
> > >
> > > Functional.LTP has heavy outputs after testing, which makes it
> > > difficult for us to locate the error log easily.
> > >
> > > LTP has the feature that can help us to split the output for
> > > different testcases.
> > >
> > > So, I added some links to those separated log files in the test case
> > > summary page.
> > >
> > > It helped me a lot to investigate different failures.
> >
> > I like this a lot.  Fuego has not had very good facilities to isolate
> > the error information for each testcase, and this looks like it will do
> it.
> 
> I think that separate log files are good. But if you patch LTP please keep
> in mind that test case error logs are already separated by the parser. In
> LTP, the error log for non-successful test cases is in the corresponding
> row on the output excel sheet.
> If you double-click on the cell, you will see the complete error log for
> that specific test case.
> # Next, I am planning to add another column with a guess on why the test
> case failed.

Good.

> By the way, normally what I do to investigate a failure is:
>   - set the target_postCleanup flag to False
>   - ssh to the target
>   - execute the test case individually with runltp
>      # ./runltp -f syscalls -s madvise07
>   - or execute the test directly with strace
>     # strace bin/madvise07
>     [Note] the arguments required for the test can be obtained from the
> test group inside runtests/ (e.g. syscalls file)

Yes, right. Thanks, Daniel, I indeed did the same step with you for most LTP investigation.
But for some failures, sometimes, I just want to check if the error log same with before.
The excel sheet is very convenient to check and share the all test results, but I think the HTML page is better to compare the results of different builds and that's why I added this.

> Thanks,
> Daniel
> 
> > > Now, I also added a generic function to split the output for
> > > different tests and try to apply it to different tests.
> > >
> > > Please review the patch[01-09] in the maillist.
> >
> > I will do what I can tonight, but I probably won't be able to finish
> > before I stop working today.  I'll have to finish up later this week.
> >  -- Tim
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> 
> 
> 
> 

Re: [Fuego] Split the output and add log links for each testcase

[Tim.Bird]

Liu,

Just double-checking...  Are you planning on submitting this whole series again?
I like the feature, but lost track of the status.

Thanks,
 -- Tim


> -----Original Message-----
> From: Liu, Wenlong
>
> Functional.LTP has heavy outputs after testing, which makes it difficult for us
> to locate the error log easily.
> 
> 
> 
> LTP has the feature that can help us to split the output for different
> testcases.
> 
> So, I added some links to those separated log files in the test case summary
> page.
> 
> It helped me a lot to investigate different failures.
> 
> 
> 
> Now, I also added a generic function to split the output for different tests
> and try to apply it to different tests.
> 
> Please review the patch[01-09] in the maillist.