[Fuego] [Question] Will new tests be welcome by Fuego

[Fuego] [Question] Will new tests be welcome by Fuego

[Wang, Mingyu]

[-- Attachment #1: Type: text/plain, Size: 2229 bytes --]

Hello Fuego,
    I'm new to fuego.
    Currently our project is using fuego as the testing framework,
    and we'd like to share some general test cases.
    These test cases are for the fuctions of ebeded system,
    by sharing them, hopefully other prject can use it,
and it will also be easier for us to update and maintain them.
Will these new tests be welcome by Fuego?

Test target:
busybox

How to add:
a new test suite Functional.busybox is added in directory tests,
all the test case are scripts which can be called directly.

test case list:
busybox_ash
busybox_basename
busybox_bunzip2
busybox_busybox
busybox_bzcat
busybox_cat
busybox_chgrp
busybox_chmod
busybox_chown
busybox_chroot
busybox_chvt
busybox_clear
busybox_cmp
busybox_cp
busybox_cut
busybox_dd
busybox_date
busybox_deallocvt
busybox_df
busybox_dirname
busybox_dmesg
busybox_du
busybox_echo
busybox_egrep
busybox_env
busybox_expr
busybox_false
busybox_fgrep
busybox_find
busybox_free
busybox_grep
busybox_gunzip
busybox_gzip
busybox_head
busybox_hexdump
busybox_hostname
busybox_id
busybox_ifconfig
busybox_install
busybox_killall
busybox_kill
busybox_ln
busybox_logger
busybox_ls
busybox_mkdir
busybox_mknod
busybox_mktemp
busybox_more
busybox_mount
busybox_mv
busybox_openvt
busybox_passwd
busybox_pidof
busybox_ping
busybox_ps
busybox_pwd
busybox_readlink
busybox_reset
busybox_rmdir
busybox_rm
busybox_route
busybox_sed
busybox_busybox
busybox_sleep
busybox_sort
busybox_strings
busybox_swapoff
busybox_swapon
busybox_sync
busybox_syslogd
busybox_tail
busybox_tar
busybox_tee
busybox_test
busybox_time
busybox_touch
busybox_tr
busybox_true
busybox_umount
busybox_uname
busybox_uniq
busybox_uptime
busybox_usleep
busybox_wc
busybox_wget
busybox_whoami
busybox_xargs
busybox_zcat


  --
Best Regards
---------------------------------------------------
Wang Mingyu
Development Dept.I
Nanjing Fujitsu Nanda Software Tech. Co., Ltd.(FNST) No. 6 Wenzhu Road, Nanjing, 210012, China
TEL: +86+25-86630566-8568
COINS: 79988548
FAX: +86+25-83317685
MAIL: wangmy@cn.fujitsu.com<mailto:wangmy@cn.fujitsu.com>
http://www.fujitsu.com/cn/fnst/




[-- Attachment #2: Type: text/html, Size: 9702 bytes --]

Re: [Fuego] [Question] Will new tests be welcome by Fuego

[Tim.Bird]

> -----Original Message-----
> From: Wang, Mingyu
> 
> Hello Fuego,
> 
>     I'm new to fuego.
> 
>     Currently our project is using fuego as the testing framework,
> 
>     and we'd like to share some general test cases.
> 
>     These test cases are for the fuctions of ebeded system,
> 
>     by sharing them, hopefully other prject can use it,
> 
> and it will also be easier for us to update and maintain them.
> 
> Will these new tests be welcome by Fuego?

Yes.  Absolutely.  We very much appreciate new tests in Fuego.
 
> Test target:
> 
> busybox
> 
> 
> 
> How to add:
> 
> a new test suite Functional.busybox is added in directory tests,
> 
> all the test case are scripts which can be called directly.
> 
> 
> 
> test case list:
> 
> busybox_ash
> 
> busybox_basename
> 
> busybox_bunzip2
> 
> busybox_busybox
> 
> busybox_bzcat
> 
> busybox_cat
> 
> busybox_chgrp
> 
> busybox_chmod
> 
> busybox_chown
> 
> busybox_chroot
> 
> busybox_chvt
> 
> busybox_clear
> 
> busybox_cmp
> 
> busybox_cp
> 
> busybox_cut
> 
> busybox_dd
> 
> busybox_date
> 
> busybox_deallocvt
> 
> busybox_df
> 
> busybox_dirname
> 
> busybox_dmesg
> 
> busybox_du
> 
> busybox_echo
> 
> busybox_egrep
> 
> busybox_env
> 
> busybox_expr
> 
> busybox_false
> 
> busybox_fgrep
> 
> busybox_find
> 
> busybox_free
> 
> busybox_grep
> 
> busybox_gunzip
> 
> busybox_gzip
> 
> busybox_head
> 
> busybox_hexdump
> 
> busybox_hostname
> 
> busybox_id
> 
> busybox_ifconfig
> 
> busybox_install
> 
> busybox_killall
> 
> busybox_kill
> 
> busybox_ln
> 
> busybox_logger
> 
> busybox_ls
> 
> busybox_mkdir
> 
> busybox_mknod
> 
> busybox_mktemp
> 
> busybox_more
> 
> busybox_mount
> 
> busybox_mv
> 
> busybox_openvt
> 
> busybox_passwd
> 
> busybox_pidof
> 
> busybox_ping
> 
> busybox_ps
> 
> busybox_pwd
> 
> busybox_readlink
> 
> busybox_reset
> 
> busybox_rmdir
> 
> busybox_rm
> 
> busybox_route
> 
> busybox_sed
> 
> busybox_busybox
> 
> busybox_sleep
> 
> busybox_sort
> 
> busybox_strings
> 
> busybox_swapoff
> 
> busybox_swapon
> 
> busybox_sync
> 
> busybox_syslogd
> 
> busybox_tail
> 
> busybox_tar
> 
> busybox_tee
> 
> busybox_test
> 
> busybox_time
> 
> busybox_touch
> 
> busybox_tr
> 
> busybox_true
> 
> busybox_umount
> 
> busybox_uname
> 
> busybox_uniq
> 
> busybox_uptime
> 
> busybox_usleep
> 
> busybox_wc
> 
> busybox_wget
> 
> busybox_whoami
> 
> busybox_xargs
> 
> busybox_zcat
> 

Please do send these to the list.  I am anxious to see how the test works.
I believe many people will be interested in a nice test suite for busybox
in embedded.

Please format as a patch, and send the patch to the Fuego mailing list.
Please see http://fuegotest.org/wiki/License_And_Contribution_Policy for
information about the license, and patch submission policies.

We also have a coding style guide - see http://fuegotest.org/wiki/Coding_style
But don't let style issues block sending the patch.  I'd much rather get a
patch with non-conformant coding style, than none at all.

Thanks!  I look forward to seeing this test.
 -- Tim

Re: [Fuego] [Question] Will new tests be welcome by Fuego

[Wang, Mingyu]

Hi Tim,
 
I have submited several busybox related test cases.  I would like you to review them first. 
There are 89 busybox related test cases, and all test cases is placed in testsuits directory.
This time I only submitted 4 of them. I would like to modify the remaining cases according to your opinion and  submit the rest of them step by step.

 Thanks,
Wangmy

-----Original Message-----
From: Tim.Bird@sony.com [mailto:Tim.Bird@sony.com] 
Sent: Wednesday, May 16, 2018 2:03 AM
To: Wang, Mingyu/王 鸣瑜 <wangmy@cn.fujitsu.com>; fuego@lists.linuxfoundation.org
Subject: RE: [Question] Will new tests be welcome by Fuego

> -----Original Message-----
> From: Wang, Mingyu
> 
> Hello Fuego,
> 
>     I'm new to fuego.
> 
>     Currently our project is using fuego as the testing framework,
> 
>     and we'd like to share some general test cases.
> 
>     These test cases are for the fuctions of ebeded system,
> 
>     by sharing them, hopefully other prject can use it,
> 
> and it will also be easier for us to update and maintain them.
> 
> Will these new tests be welcome by Fuego?

Yes.  Absolutely.  We very much appreciate new tests in Fuego.
 
> Test target:
> 
> busybox
> 
> 
> 
> How to add:
> 
> a new test suite Functional.busybox is added in directory tests,
> 
> all the test case are scripts which can be called directly.
> 
> 
> 
> test case list:
> 
> busybox_ash
> 
> busybox_basename
> 
> busybox_bunzip2
> 
> busybox_busybox
> 
> busybox_bzcat
> 
> busybox_cat
> 
> busybox_chgrp
> 
> busybox_chmod
> 
> busybox_chown
> 
> busybox_chroot
> 
> busybox_chvt
> 
> busybox_clear
> 
> busybox_cmp
> 
> busybox_cp
> 
> busybox_cut
> 
> busybox_dd
> 
> busybox_date
> 
> busybox_deallocvt
> 
> busybox_df
> 
> busybox_dirname
> 
> busybox_dmesg
> 
> busybox_du
> 
> busybox_echo
> 
> busybox_egrep
> 
> busybox_env
> 
> busybox_expr
> 
> busybox_false
> 
> busybox_fgrep
> 
> busybox_find
> 
> busybox_free
> 
> busybox_grep
> 
> busybox_gunzip
> 
> busybox_gzip
> 
> busybox_head
> 
> busybox_hexdump
> 
> busybox_hostname
> 
> busybox_id
> 
> busybox_ifconfig
> 
> busybox_install
> 
> busybox_killall
> 
> busybox_kill
> 
> busybox_ln
> 
> busybox_logger
> 
> busybox_ls
> 
> busybox_mkdir
> 
> busybox_mknod
> 
> busybox_mktemp
> 
> busybox_more
> 
> busybox_mount
> 
> busybox_mv
> 
> busybox_openvt
> 
> busybox_passwd
> 
> busybox_pidof
> 
> busybox_ping
> 
> busybox_ps
> 
> busybox_pwd
> 
> busybox_readlink
> 
> busybox_reset
> 
> busybox_rmdir
> 
> busybox_rm
> 
> busybox_route
> 
> busybox_sed
> 
> busybox_busybox
> 
> busybox_sleep
> 
> busybox_sort
> 
> busybox_strings
> 
> busybox_swapoff
> 
> busybox_swapon
> 
> busybox_sync
> 
> busybox_syslogd
> 
> busybox_tail
> 
> busybox_tar
> 
> busybox_tee
> 
> busybox_test
> 
> busybox_time
> 
> busybox_touch
> 
> busybox_tr
> 
> busybox_true
> 
> busybox_umount
> 
> busybox_uname
> 
> busybox_uniq
> 
> busybox_uptime
> 
> busybox_usleep
> 
> busybox_wc
> 
> busybox_wget
> 
> busybox_whoami
> 
> busybox_xargs
> 
> busybox_zcat
> 

Please do send these to the list.  I am anxious to see how the test works.
I believe many people will be interested in a nice test suite for busybox in embedded.

Please format as a patch, and send the patch to the Fuego mailing list.
Please see http://fuegotest.org/wiki/License_And_Contribution_Policy for information about the license, and patch submission policies.

We also have a coding style guide - see http://fuegotest.org/wiki/Coding_style
But don't let style issues block sending the patch.  I'd much rather get a patch with non-conformant coding style, than none at all.

Thanks!  I look forward to seeing this test.
 -- Tim

Re: [Fuego] [Question] Will new tests be welcome by Fuego

[Tim.Bird]

> -----Original Message-----
> From: Wang, Mingyu [mailto:wangmy@cn.fujitsu.com]
> Sent: Friday, May 18, 2018 1:47 AM
> To: Bird, Timothy <Tim.Bird@sony.com>; fuego@lists.linuxfoundation.org
> Subject: RE: [Question] Will new tests be welcome by Fuego
> 
> Hi Tim,
> 
> I have submited several busybox related test cases.  I would like you to
> review them first.
> There are 89 busybox related test cases, and all test cases is placed in
> testsuits directory.
> This time I only submitted 4 of them. I would like to modify the remaining
> cases according to your opinion and  submit the rest of them step by step.

Here are the results for me running this on my beaglebone:
My comments are in ***comment*** marks.

...
##### doing fuego phase: run ########
WARNING: test log file parameter empty, so will use default
#!/bin/bash
for i in testsuits/*.sh; do
    sh $i
done
testsuits/busybox_ash.sh: 5: testsuits/busybox_ash.sh: expect: not found
*** my board doesn't have expect ***
testsuits/busybox_bunzip2.sh: 6: [: This is a test file: unexpected operator
*** maybe a shell syntax error? - I think a missing semi-colon. ***
bunzip2: test cases failed.
             total         used         free       shared      buffers
Mem:        508496        99328       409168            0          524
-/+ buffers:              98804       409692
Swap:            0            0            0
testsuits/busybox_free.sh: 5: [: 0: unexpected operator
*** I noticed you use `echo $?` in busybox_free.sh  You should just use "$?", and only use =, not == ***
free: test cases failed.
wget: bad port spec '192.168.7.1:'
ls: cannot access /home/test/: No such file or directory
wget: ls  /home/test/ failed.
*** this is the /home/test problem I mentioned in my other email ***
cat: /home/test/readme.txt: No such file or directory
wget: test cases failed.
rm: cannot remove `/home/test/readme.txt': No such file or directory
a
bin
boot
bootsave
dev
etc
home
lib
lost+found
media
mnt
opt
proc
root
run
sbin
selinux
srv
sys
tests
tmp
usr
var
work
*** this is from that stray 'ls' - what is this for? ***

##### doing fuego phase: post_test ########
Teardown board link
##### doing fuego phase: processing ########
...

----
I noticed that all your test scripts use #!/bin/bash.  You should use #!/bin/sh
as not every test machine will have bash.  Fuego tries not to rely on bash
being on a target board.  I know you are invoking these with 'sh'
in busybox_test.sh, but someone might try to run these scripts standalone.

 -- Tim

Re: [Fuego] [Question] Will new tests be welcome by Fuego

[Wang, Mingyu]

Hi Tim

Thank you for you comment. And I've been used 'sh' instead of 'bash'.

-- Wangmy

-----Original Message-----
From: Tim.Bird@sony.com [mailto:Tim.Bird@sony.com] 
Sent: Saturday, May 19, 2018 10:12 AM
To: Wang, Mingyu/王 鸣瑜 <wangmy@cn.fujitsu.com>; fuego@lists.linuxfoundation.org
Subject: RE: [Question] Will new tests be welcome by Fuego



> -----Original Message-----
> From: Wang, Mingyu [mailto:wangmy@cn.fujitsu.com]
> Sent: Friday, May 18, 2018 1:47 AM
> To: Bird, Timothy <Tim.Bird@sony.com>; fuego@lists.linuxfoundation.org
> Subject: RE: [Question] Will new tests be welcome by Fuego
> 
> Hi Tim,
> 
> I have submited several busybox related test cases.  I would like you 
> to review them first.
> There are 89 busybox related test cases, and all test cases is placed 
> in testsuits directory.
> This time I only submitted 4 of them. I would like to modify the 
> remaining cases according to your opinion and  submit the rest of them step by step.

Here are the results for me running this on my beaglebone:
My comments are in ***comment*** marks.

...
##### doing fuego phase: run ########
WARNING: test log file parameter empty, so will use default #!/bin/bash for i in testsuits/*.sh; do
    sh $i
done
testsuits/busybox_ash.sh: 5: testsuits/busybox_ash.sh: expect: not found
*** my board doesn't have expect ***
testsuits/busybox_bunzip2.sh: 6: [: This is a test file: unexpected operator
*** maybe a shell syntax error? - I think a missing semi-colon. ***
bunzip2: test cases failed.
             total         used         free       shared      buffers
Mem:        508496        99328       409168            0          524
-/+ buffers:              98804       409692
Swap:            0            0            0
testsuits/busybox_free.sh: 5: [: 0: unexpected operator
*** I noticed you use `echo $?` in busybox_free.sh  You should just use "$?", and only use =, not == ***
free: test cases failed.
wget: bad port spec '192.168.7.1:'
ls: cannot access /home/test/: No such file or directory
wget: ls  /home/test/ failed.
*** this is the /home/test problem I mentioned in my other email ***
cat: /home/test/readme.txt: No such file or directory
wget: test cases failed.
rm: cannot remove `/home/test/readme.txt': No such file or directory a bin boot bootsave dev etc home lib
lost+found
media
mnt
opt
proc
root
run
sbin
selinux
srv
sys
tests
tmp
usr
var
work
*** this is from that stray 'ls' - what is this for? ***

##### doing fuego phase: post_test ######## Teardown board link ##### doing fuego phase: processing ######## ...

----
I noticed that all your test scripts use #!/bin/bash.  You should use #!/bin/sh as not every test machine will have bash.  Fuego tries not to rely on bash being on a target board.  I know you are invoking these with 'sh'
in busybox_test.sh, but someone might try to run these scripts standalone.

 -- Tim