[PATCH v4 0/9] Various fixes/improvements for tests

* [PATCH v4 0/9] Various fixes/improvements for tests
@ 2021-10-13  0:39 Glenn Washburn
  2021-10-13  0:39 ` [PATCH v4 1/9] tests: Fix partmap_test for arm*-efi, disk numbering has changed Glenn Washburn
                   ` (8 more replies)
  0 siblings, 9 replies; 11+ messages in thread
From: Glenn Washburn @ 2021-10-13  0:39 UTC (permalink / raw)
  To: grub-devel, Daniel Kiper; +Cc: Glenn Washburn

Updates since v3:
* Dropped iso9660_test LANG patch
* Merged v3 in v2
* Rewrote commit message for #4 to hopefully be clearer
* Replace `...` with $(...)

These are mostly test improvement and fixes that stand on their own.

Glenn

Glenn Washburn (9):
  tests: Fix partmap_test for arm*-efi, disk numbering has changed
  tests: When checking squashfs fstime, use superblock last modified
    time
  tests: Add set -e to missing tests
  tests: Do not occlude subshell error codes when used as input to the
    test command
  tests: Do not occlude grub-shell return code
  tests: Make setup errors in grub-fs-tester hard errors
  tests: A failure of mktemp should cause the test script to exit with
    code 99
  tests: Exit with skipped exit code when test not performed
  tests: Use @BUILD_SHEBANG@ autoconf var instead of literal shell

 tests/ahci_test.in             | 19 ++++++++++++-------
 tests/cdboot_test.in           | 11 ++++++-----
 tests/core_compress_test.in    |  8 +++++---
 tests/ehci_test.in             | 19 ++++++++++++-------
 tests/f2fs_test.in             |  2 +-
 tests/fddboot_test.in          | 19 ++++++++++---------
 tests/gettext_strings_test.in  |  2 +-
 tests/grub_cmd_date.in         |  5 +++--
 tests/grub_cmd_set_date.in     |  6 +++---
 tests/grub_cmd_sleep.in        |  2 +-
 tests/grub_cmd_test.in         |  7 ++++---
 tests/grub_script_blockarg.in  |  3 ++-
 tests/grub_script_expansion.in |  3 ++-
 tests/gzcompress_test.in       |  3 ++-
 tests/hddboot_test.in          |  9 +++++----
 tests/lzocompress_test.in      |  3 ++-
 tests/netboot_test.in          | 15 ++++++++-------
 tests/ohci_test.in             | 19 ++++++++++++-------
 tests/partmap_test.in          | 18 +++++++++---------
 tests/pata_test.in             | 13 +++++++------
 tests/pseries_test.in          |  2 +-
 tests/syslinux_test.in         |  2 +-
 tests/test_sha512sum.in        |  7 ++++---
 tests/uhci_test.in             | 19 ++++++++++++-------
 tests/util/grub-fs-tester.in   | 17 ++++++++++++-----
 tests/xzcompress_test.in       |  3 ++-
 26 files changed, 139 insertions(+), 97 deletions(-)

Range-diff against v3:
 1:  3a2cc2738 =  1:  3a2cc2738 tests: Fix partmap_test for arm*-efi, disk numbering has changed
 2:  3da044f56 =  2:  3da044f56 tests: When checking squashfs fstime, use superblock last modified time
 3:  a061a35f5 =  3:  a061a35f5 tests: Add set -e to missing tests
 4:  ab85eb4c8 !  4:  5828e4c79 tests: Do not occlude subshell error codes
    @@ Metadata
     Author: Glenn Washburn <development@efficientek.com>

      ## Commit message ##
    -    tests: Do not occlude subshell error codes
    +    tests: Do not occlude subshell error codes when used as input to the test command

    -    When a subshell's output is used as input to a "simple command", its return
    -    code is not checked. These subshells contain an execution of the grub-shell
    -    script which does the work of the actual test. If grub-shell returns an
    -    error code, the test should fail. So refactor to not have the subshell which
    -    contains grub-shell be direct input into a simple command (usually the test
    -    command). Mostly this is accomplished by having the output first go to a
    -    shell variable, and then using the shell variable in the simple command.
    +    When using the output of a subshell as input, its error code is ignored in
    +    the context of "set -e". Many test scripts use grub-shell in a subshell with
    +    output used as an argument to the test command to test for expected output.
    +    Refactor these tests so that the subshell output goes to a shell variable,
    +    so that if the subshell errors the script will immediately exit with an
    +    error code.

      ## tests/ahci_test.in ##
     @@ tests/ahci_test.in: echo "hello" > "$outfile"
    @@ tests/cdboot_test.in: case "${grub_modinfo_target_cpu}-${grub_modinfo_platform}"
      esac

     -if [ "$(echo hello | "${grubshell}" --boot=cd)" != "Hello World" ]; then
    -+v=`echo hello | "${grubshell}" --boot=cd`
    ++v=$(echo hello | "${grubshell}" --boot=cd)
     +if [ "$v" != "Hello World" ]; then
         exit 1
      fi
    @@ tests/core_compress_test.in: case "${grub_modinfo_target_cpu}-${grub_modinfo_pla

     -if [ "$(echo hello | "${grubshell}" --grub-mkimage-extra=--compress=xz)" != "Hello World" ]; then
    -+v=`echo hello | "${grubshell}" --grub-mkimage-extra=--compress=xz`
    ++v=$(echo hello | "${grubshell}" --grub-mkimage-extra=--compress=xz)
     +if [ "$v" != "Hello World" ]; then
         exit 1
      fi

     -if [ "$(echo hello | "${grubshell}" --grub-mkimage-extra=--compress=none)" != "Hello World" ]; then
    -+v=`echo hello | "${grubshell}" --grub-mkimage-extra=--compress=none`
    ++v=$(echo hello | "${grubshell}" --grub-mkimage-extra=--compress=none)
     +if [ "$v" != "Hello World" ]; then
         exit 1
      fi
    @@ tests/fddboot_test.in: case "${grub_modinfo_target_cpu}-${grub_modinfo_platform}
      esac

     -if [ "$(echo hello | "${grubshell}" --boot=fd --mkrescue-arg="--compress=xz --fonts= --locales= --themes= -no-pad")" != "Hello World" ]; then
    -+v=`echo hello | "${grubshell}" --boot=fd --mkrescue-arg="--compress=xz --fonts= --locales= --themes= -no-pad"`
    ++v=$(echo hello | "${grubshell}" --boot=fd --mkrescue-arg="--compress=xz --fonts= --locales= --themes= -no-pad")
     +if [ "$v" != "Hello World" ]; then
         exit 1
      fi
    @@ tests/gzcompress_test.in: if ! which gzip >/dev/null 2>&1; then
      fi

     -if [ "$(echo hello | "${grubshell}" --mkrescue-arg=--compress=gz)" != "Hello World" ]; then
    -+v=`echo hello | "${grubshell}" --mkrescue-arg=--compress=gz`
    ++v=$(echo hello | "${grubshell}" --mkrescue-arg=--compress=gz)
     +if [ "$v" != "Hello World" ]; then
         exit 1
      fi
    @@ tests/hddboot_test.in: case "${grub_modinfo_target_cpu}-${grub_modinfo_platform}
      esac

     -if [ "$(echo hello | "${grubshell}" --boot=hd)" != "Hello World" ]; then
    -+v=`echo hello | "${grubshell}" --boot=hd`
    ++v=$(echo hello | "${grubshell}" --boot=hd)
     +if [ "$v" != "Hello World" ]; then
         exit 1
      fi
    @@ tests/lzocompress_test.in: if ! which lzop >/dev/null 2>&1; then
      fi

     -if [ "$(echo hello | "${grubshell}" --mkrescue-arg=--compress=lzo)" != "Hello World" ]; then
    -+v=`echo hello | "${grubshell}" --mkrescue-arg=--compress=lzo`
    ++v=$(echo hello | "${grubshell}" --mkrescue-arg=--compress=lzo)
     +if [ "$v" != "Hello World" ]; then
         exit 1
      fi
    @@ tests/netboot_test.in: case "${grub_modinfo_target_cpu}-${grub_modinfo_platform}
      esac

     -if [ "$(echo hello | "${grubshell}" --boot=net)" != "Hello World" ]; then
    -+v=`echo hello | "${grubshell}" --boot=net`
    ++v=$(echo hello | "${grubshell}" --boot=net)
     +if [ "$v" != "Hello World" ]; then
         exit 1
      fi
    @@ tests/pata_test.in: echo "hello" > "$outfile"
      tar cf "$imgfile" "$outfile"

     -if [ "$(echo "nativedisk; source '($indisk)/$outfile';" | "${grubshell}" --qemu-opts="-$disk $imgfile")" != "Hello World" ]; then
    -+v=`echo "nativedisk; source '($indisk)/$outfile';" | "${grubshell}" --qemu-opts="-$disk $imgfile"`
    ++v=$(echo "nativedisk; source '($indisk)/$outfile';" | "${grubshell}" --qemu-opts="-$disk $imgfile")
     +if [ "$v" != "Hello World" ]; then
         rm "$imgfile"
         rm "$outfile"
    @@ tests/xzcompress_test.in: if ! which xz >/dev/null 2>&1; then
      fi

     -if [ "$(echo hello | "${grubshell}" --mkrescue-arg=--compress=xz)" != "Hello World" ]; then
    -+v=`echo hello | "${grubshell}" --mkrescue-arg=--compress=xz`
    ++v=$(echo hello | "${grubshell}" --mkrescue-arg=--compress=xz)
     +if [ "$v" != "Hello World" ]; then
         exit 1
      fi
 5:  fac9d9cae =  5:  37097a173 tests: Do not occlude grub-shell return code
 6:  dae3591c4 =  6:  c703a1b2f tests: Make setup errors in grub-fs-tester hard errors
 7:  5f2fb19e9 =  7:  23a0d5358 tests: A failure of mktemp should cause the test script to exit with code 99
 8:  2a2e491bd !  8:  a4d79a908 tests: Exit with skipped exit code when test not performed
    @@ tests/cdboot_test.in: grubshell=@builddir@/grub-shell
     +	exit 77;;
      esac

    - v=`echo hello | "${grubshell}" --boot=cd`
    + v=$(echo hello | "${grubshell}" --boot=cd)

      ## tests/core_compress_test.in ##
     @@ tests/core_compress_test.in: grubshell=@builddir@/grub-shell
    @@ tests/fddboot_test.in: grubshell=@builddir@/grub-shell
     +	exit 77;;
      esac

    - v=`echo hello | "${grubshell}" --boot=fd --mkrescue-arg="--compress=xz --fonts= --locales= --themes= -no-pad"`
    + v=$(echo hello | "${grubshell}" --boot=fd --mkrescue-arg="--compress=xz --fonts= --locales= --themes= -no-pad")

      ## tests/grub_cmd_date.in ##
     @@ tests/grub_cmd_date.in: set -e
    @@ tests/hddboot_test.in: grubshell=@builddir@/grub-shell
     +	exit 77;;
      esac

    - v=`echo hello | "${grubshell}" --boot=hd`
    + v=$(echo hello | "${grubshell}" --boot=hd)

      ## tests/netboot_test.in ##
     @@ tests/netboot_test.in: grubshell=@builddir@/grub-shell
    @@ tests/netboot_test.in: grubshell=@builddir@/grub-shell
     +	exit 77;;
      esac

    - v=`echo hello | "${grubshell}" --boot=net`
    + v=$(echo hello | "${grubshell}" --boot=net)

      ## tests/ohci_test.in ##
     @@ tests/ohci_test.in: grubshell=@builddir@/grub-shell
 9:  606b9b818 =  9:  a7119dfc4 tests: Use @BUILD_SHEBANG@ autoconf var instead of literal shell
-- 
2.27.0

^ permalink raw reply	[flat|nested] 11+ messages in thread