[U-Boot] [RFC 0/3] test/py: add filesystem test scripts

[U-Boot] [RFC 0/3] test/py: add filesystem test scripts

[AKASHI Takahiro]

This patch series is an attempt to add tests scripts for filesystem
functionality to U-boot pytest suite.

Currently, we have a bit old-fashioned shell-script, fs-test.sh,
under test/fs for this purpose, which is obviously not an integrated
part of the suite.

So my aim here is:
* first, convert fs-test.sh to python script using pytest
* then add more test cases for exntensive test coverage,
  including added functions to be introduced by my patch set,
  "fs: fat: extend FAT write operations"[1]

The current code is a bit rough-edged yet and any comments or
feedbacks are very much welcomed.

The code itself was tested on v2018.09-rc with my patch set above.

[1] https://lists.denx.de/pipermail/u-boot/2018-July/335415.html

Thanks,
-Takahiro AKASHI

AKASHI Takahiro (3):
  test/py: convert fs-test.sh to pytest
  test/py: fs: add extended write operation test
  test/py: fs: add fstest/mkdir test

 test/py/tests/test_fs/conftest.py    | 289 +++++++++++++++++++++++++++
 test/py/tests/test_fs/fstest_defs.py |  13 ++
 test/py/tests/test_fs/test_basic.py  | 246 +++++++++++++++++++++++
 test/py/tests/test_fs/test_ext.py    | 197 ++++++++++++++++++
 test/py/tests/test_fs/test_mkdir.py  |  77 +++++++
 5 files changed, 822 insertions(+)
 create mode 100644 test/py/tests/test_fs/conftest.py
 create mode 100644 test/py/tests/test_fs/fstest_defs.py
 create mode 100644 test/py/tests/test_fs/test_basic.py
 create mode 100644 test/py/tests/test_fs/test_ext.py
 create mode 100644 test/py/tests/test_fs/test_mkdir.py

-- 
2.18.0

[U-Boot] [RFC 1/3] test/py: convert fs-test.sh to pytest

[AKASHI Takahiro]

In this commit, the same set of test cases as in test/fs/fs-test.sh
is provided using pytest framework.
Actually, fs-test.sh provides three variants:"sb" (sb command), "nonfs"
(fatxx and etc.) and "fs" (hostfs), and this patch currently supports
only "nonfs" variant; So it is not a replacement of fs-test.sh for now.

Simple usage:
  $ py.test test/py/tests/test_fs [<other options>]

You may also specify filesystem types to be tested:
  $ py.test test/py/tests/test_fs --fs-type fat32 [<other options>]

Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
---
 test/py/tests/test_fs/conftest.py    | 175 +++++++++++++++++++
 test/py/tests/test_fs/fstest_defs.py |  10 ++
 test/py/tests/test_fs/test_basic.py  | 246 +++++++++++++++++++++++++++
 3 files changed, 431 insertions(+)
 create mode 100644 test/py/tests/test_fs/conftest.py
 create mode 100644 test/py/tests/test_fs/fstest_defs.py
 create mode 100644 test/py/tests/test_fs/test_basic.py

diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
new file mode 100644
index 000000000000..fefeb4c9663f
--- /dev/null
+++ b/test/py/tests/test_fs/conftest.py
@@ -0,0 +1,175 @@
+# SPDX-License-Identifier:      GPL-2.0+
+# Copyright (c) 2018, Linaro Limited
+# Author: Takahiro Akashi <takahiro.akashi@linaro.org>
+
+import pytest
+import re
+from subprocess import call, check_call, check_output, CalledProcessError
+from fstest_defs import *
+
+supported_fs_basic = ['fat16', 'fat32', 'ext4']
+
+#
+# Filesystem test specific setup
+#
+def pytest_addoption(parser):
+    parser.addoption('--fs-type', action='append', default=None,
+        help='Targeting Filesystem Types')
+
+def pytest_configure(config):
+    global supported_fs_basic
+
+    def intersect(listA, listB):
+        return  [x for x in listA if x in listB]
+
+    supported_fs = config.getoption('fs_type')
+    if supported_fs:
+        print("*** FS TYPE modified: %s" % supported_fs)
+        supported_fs_basic =  intersect(supported_fs, supported_fs_basic)
+
+def pytest_generate_tests(metafunc):
+    if 'fs_obj_basic' in metafunc.fixturenames:
+        metafunc.parametrize('fs_obj_basic', supported_fs_basic,
+            indirect=True, scope='module')
+
+#
+# Helper functions
+#
+def fstype_to_ubname(fs_type):
+    if re.match('fat', fs_type):
+        return 'fat'
+    else:
+        return fs_type
+
+def check_ubconfig(config, fs_type):
+    if not config.buildconfig.get('config_cmd_%s' % fs_type, None):
+        pytest.skip('.config feature "CMD_%s" not enabled' % fs_type.upper())
+    if not config.buildconfig.get('config_%s_write' % fs_type, None):
+        pytest.skip('.config feature "%s_WRITE" not enabled'
+        % fs_type.upper())
+
+def mk_fs(config, fs_type, size, id):
+    fs_img = '%s.%s.img' % (id, fs_type)
+    fs_img = config.persistent_data_dir + '/' + fs_img
+
+    if fs_type == 'fat16':
+        mkfs_opt = '-F 16'
+    elif fs_type == 'fat32':
+        mkfs_opt = '-F 32'
+    else:
+        mkfs_opt = ''
+
+    if re.match('fat', fs_type):
+        fs_lnxtype = 'vfat'
+    else:
+        fs_lnxtype = fs_type
+
+    count = (size + 1023) / 1024
+
+    try:
+        check_call('rm -f %s' % fs_img, shell=True)
+        check_call('dd if=/dev/zero of=%s bs=1K count=%d'
+            % (fs_img, count), shell=True)
+        check_call('mkfs.%s %s %s'
+            % (fs_lnxtype, mkfs_opt, fs_img), shell=True)
+        return fs_img
+    except CalledProcessError:
+        call('rm -f %s' % fs_img, shell=True)
+        raise
+
+#
+# Fixture for basic fs test
+#     derived from test/fs/fs-test.sh
+#
+# NOTE: yield_fixture was deprecated since pytest-3.0
+ at pytest.yield_fixture()
+def fs_obj_basic(request, u_boot_config):
+    fs_type = request.param
+    fs_img = ''
+
+    fs_ubtype = fstype_to_ubname(fs_type)
+    check_ubconfig(u_boot_config, fs_ubtype)
+
+    mount_dir = u_boot_config.persistent_data_dir + '/mnt'
+    small_file = mount_dir + '/' + SMALL_FILE
+    big_file = mount_dir + '/' + BIG_FILE
+    try:
+
+        # 3GiB volume
+        fs_img = mk_fs(u_boot_config, fs_type, 0xc0000000, '3GB')
+
+        # Mount the image so we can populate it.
+        check_call('mkdir -p %s' % mount_dir, shell=True)
+        check_call('sudo mount -o loop,rw %s %s'
+            % (fs_img, mount_dir), shell=True)
+
+        # Create a subdirectory.
+        check_call('sudo mkdir %s/SUBDIR' % mount_dir, shell=True)
+
+        # Create big file in this image.
+        # Note that we work only on the start 1MB, couple MBs in the 2GB range
+        # and the last 1 MB of the huge 2.5GB file.
+        # So, just put random values only in those areas.
+        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=1'
+	    % big_file, shell=True)
+        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=2 seek=2047'
+            % big_file, shell=True)
+        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=1 seek=2499'
+            % big_file, shell=True)
+
+        # Create a small file in this image.
+        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=1'
+	    % small_file, shell=True)
+
+        # Delete the small file copies which possibly are written as part of a
+        # previous test.
+        # check_call('sudo rm -f "%s.w"' % MB1, shell=True)
+        # check_call('sudo rm -f "%s.w2"' % MB1, shell=True)
+
+        # Generate the md5sums of reads that we will test against small file
+        out = check_output(
+            'dd if=%s bs=1M skip=0 count=1 2> /dev/null | md5sum'
+	    % small_file, shell=True)
+        md5val = [ out.split()[0] ]
+
+        # Generate the md5sums of reads that we will test against big file
+        # One from beginning of file.
+        out = check_output(
+            'dd if=%s bs=1M skip=0 count=1 2> /dev/null | md5sum'
+	    % big_file, shell=True)
+        md5val.append(out.split()[0])
+
+        # One from end of file.
+        out = check_output(
+            'dd if=%s bs=1M skip=2499 count=1 2> /dev/null | md5sum'
+	    % big_file, shell=True)
+        md5val.append(out.split()[0])
+
+        # One from the last 1MB chunk of 2GB
+        out = check_output(
+            'dd if=%s bs=1M skip=2047 count=1 2> /dev/null | md5sum'
+	    % big_file, shell=True)
+        md5val.append(out.split()[0])
+
+        # One from the start 1MB chunk from 2GB
+        out = check_output(
+            'dd if=%s bs=1M skip=2048 count=1 2> /dev/null | md5sum'
+	    % big_file, shell=True)
+        md5val.append(out.split()[0])
+
+        # One 1MB chunk crossing the 2GB boundary
+        out = check_output(
+            'dd if=%s bs=512K skip=4095 count=2 2> /dev/null | md5sum'
+	    % big_file, shell=True)
+        md5val.append(out.split()[0])
+
+        check_call('sudo umount %s' % mount_dir, shell=True)
+    except CalledProcessError:
+        pytest.skip('Setup failed for filesystem: ' + fs_type)
+    else:
+        yield [fs_ubtype, fs_img, md5val]
+    finally:
+        call('sudo umount %s' % mount_dir, shell=True)
+	call('rmdir -rf %s' % mount_dir, shell=True)
+#        if fs_img:
+#            call('rm -f %s' % fs_img, shell=True)
diff --git a/test/py/tests/test_fs/fstest_defs.py b/test/py/tests/test_fs/fstest_defs.py
new file mode 100644
index 000000000000..f26dd06cacf2
--- /dev/null
+++ b/test/py/tests/test_fs/fstest_defs.py
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier:      GPL-2.0+
+
+# $SMALL_FILE is the name of the 1MB file in the file system image
+SMALL_FILE='1MB.file'
+
+# $BIG_FILE is the name of the 2.5GB file in the file system image
+BIG_FILE='2.5GB.file'
+
+ADDR=0x01000008
+LENGTH=0x00100000
diff --git a/test/py/tests/test_fs/test_basic.py b/test/py/tests/test_fs/test_basic.py
new file mode 100644
index 000000000000..5258d98d42a9
--- /dev/null
+++ b/test/py/tests/test_fs/test_basic.py
@@ -0,0 +1,246 @@
+# SPDX-License-Identifier:      GPL-2.0+
+# Copyright (c) 2018, Linaro Limited
+# Author: Takahiro Akashi <takahiro.akashi@linaro.org>
+#
+# U-Boot File System:Basic Test
+
+"""
+This test verifies basic read/write operation on file system.
+"""
+
+import pytest
+import re
+from fstest_defs import *
+
+ at pytest.mark.boardspec('sandbox')
+class TestFsBasic(object):
+    def test_fs1(self, u_boot_console, fs_obj_basic):
+        fs_type,fs_img,md5val = fs_obj_basic
+        with u_boot_console.log.section('Test Case 1 - ls'):
+            # Test Case 1 - ls
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sls host 0:0' % fs_type])
+            assert(re.search('2621440000 *%s' % BIG_FILE, ''.join(output)))
+            assert(re.search('1048576 *%s' % SMALL_FILE, ''.join(output)))
+
+            # In addition, test with a nonexistent directory to see if we crash.
+            output2 = u_boot_console.run_command(
+                '%sls host 0:0 invalid_d' % fs_type)
+            if fs_type == 'ext4':
+                assert('Can not find directory' in output2)
+            else:
+                assert('' == output2)
+
+    def test_fs2(self, u_boot_console, fs_obj_basic):
+        fs_type,fs_img,md5val = fs_obj_basic
+        with u_boot_console.log.section('Test Case 2 - size (small)'):
+            # 1MB is 0x0010 0000
+            # Test Case 2a - size of small file
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%ssize host 0:0 /%s' % (fs_type, SMALL_FILE),
+                'printenv filesize',
+                'setenv filesize'])
+            assert('filesize=100000' in ''.join(output))
+
+            # Test Case 2b - size of small file via a path using '..'
+            output = u_boot_console.run_command_list([
+                '%ssize host 0:0 /SUBDIR/../%s' % (fs_type, SMALL_FILE),
+                'printenv filesize',
+                'setenv filesize'])
+            assert('filesize=100000' in ''.join(output))
+
+    def test_fs3(self, u_boot_console, fs_obj_basic):
+        fs_type,fs_img,md5val = fs_obj_basic
+        with u_boot_console.log.section('Test Case 3 - size (big)'):
+            # 2.5GB (1024*1024*2500) is 0x9C40 0000
+            # Test Case 3 - size of big file
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%ssize host 0:0 /%s' % (fs_type, BIG_FILE),
+                'printenv filesize',
+                'setenv filesize'])
+            assert('filesize=9c400000' in ''.join(output))
+
+    def test_fs4(self, u_boot_console, fs_obj_basic):
+        fs_type,fs_img,md5val = fs_obj_basic
+        with u_boot_console.log.section('Test Case 4 - load (small)'):
+            # Test Case 4a - Read full 1MB of small file
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s' % (fs_type, ADDR, SMALL_FILE),
+                'printenv filesize'])
+            assert('filesize=100000' in ''.join(output))
+
+            # Test Case 4b - Read full 1MB of small file
+            output = u_boot_console.run_command_list([
+                'md5sum %x $filesize' % ADDR,
+                'setenv filesize'])
+            assert(md5val[0] in ''.join(output))
+
+    def test_fs5(self, u_boot_console, fs_obj_basic):
+        fs_type,fs_img,md5val = fs_obj_basic
+        with u_boot_console.log.section('Test Case 5 - load (big)'):
+            # Test Case 5a - First 1MB of big file
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s %x 0x0' % (fs_type, ADDR, BIG_FILE, LENGTH),
+                'printenv filesize'])
+            assert('filesize=100000' in ''.join(output))
+
+            # Test Case 5b - First 1MB of big file
+            output = u_boot_console.run_command_list([
+                'md5sum %x $filesize' % ADDR,
+                'setenv filesize'])
+            assert(md5val[1] in ''.join(output))
+
+    def test_fs6(self, u_boot_console, fs_obj_basic):
+        fs_type,fs_img,md5val = fs_obj_basic
+        with u_boot_console.log.section('Test Case 6 - load (big from last 1MB)'):
+            # fails for ext as no offset support
+            # Test Case 6a - Last 1MB of big file
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s %x 0x9c300000'
+                    % (fs_type, ADDR, BIG_FILE, LENGTH),
+                'printenv filesize'])
+            assert('filesize=100000' in ''.join(output))
+
+            # Test Case 6b - Last 1MB of big file
+            output = u_boot_console.run_command_list([
+                'md5sum %x $filesize' % ADDR,
+                'setenv filesize'])
+            assert(md5val[2] in ''.join(output))
+
+    def test_fs7(self, u_boot_console, fs_obj_basic):
+        fs_type,fs_img,md5val = fs_obj_basic
+        with u_boot_console.log.section('Test Case 7 - load (big from last 1MB in 2GB)'):
+            # fails for ext as no offset support
+            # Test Case 7a - One from the last 1MB chunk of 2GB
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s %x 0x7ff00000'
+                    % (fs_type, ADDR, BIG_FILE, LENGTH),
+                'printenv filesize'])
+            assert('filesize=100000' in ''.join(output))
+
+            # Test Case 7b - One from the last 1MB chunk of 2GB
+            output = u_boot_console.run_command_list([
+                'md5sum %x $filesize' % ADDR,
+                'setenv filesize'])
+            assert(md5val[3] in ''.join(output))
+
+    def test_fs8(self, u_boot_console, fs_obj_basic):
+        fs_type,fs_img,md5val = fs_obj_basic
+        with u_boot_console.log.section('Test Case 8 - load (big from first 1MB in 2GB)'):
+            # fails for ext as no offset support
+            # Test Case 8a - One from the start 1MB chunk from 2GB
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s %x 0x80000000'
+                    % (fs_type, ADDR, BIG_FILE, LENGTH),
+                'printenv filesize'])
+            assert('filesize=100000' in ''.join(output))
+
+            # Test Case 8b - One from the start 1MB chunk from 2GB
+            output = u_boot_console.run_command_list([
+                'md5sum %x $filesize' % ADDR,
+                'setenv filesize'])
+            assert(md5val[4] in ''.join(output))
+
+    def test_fs9(self, u_boot_console, fs_obj_basic):
+        fs_type,fs_img,md5val = fs_obj_basic
+        with u_boot_console.log.section('Test Case 9 - load (crossing 2GB boundary)'):
+            # fails for ext as no offset support
+            # Test Case 9a - One 1MB chunk crossing the 2GB boundary
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s %x 0x7ff80000'
+                    % (fs_type, ADDR, BIG_FILE, LENGTH),
+                'printenv filesize'])
+            assert('filesize=100000' in ''.join(output))
+
+            # Test Case 9b - One 1MB chunk crossing the 2GB boundary
+            output = u_boot_console.run_command_list([
+                'md5sum %x $filesize' % ADDR,
+                'setenv filesize'])
+            assert(md5val[5] in ''.join(output))
+
+    def test_fs10(self, u_boot_console, fs_obj_basic):
+        fs_type,fs_img,md5val = fs_obj_basic
+        with u_boot_console.log.section('Test Case 10 - load (over file end)'):
+            # Generic failure case
+            # Test Case 10 - 2MB chunk from the last 1MB of big file
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s 0x00200000 0x9c300000'
+                    % (fs_type, ADDR, BIG_FILE),
+                'printenv filesize',
+                'md5sum %x $filesize' % ADDR,
+                'setenv filesize'])
+        assert('filesize=100000' in ''.join(output))
+
+    def test_fs11(self, u_boot_console, fs_obj_basic):
+        fs_type,fs_img,md5val = fs_obj_basic
+        with u_boot_console.log.section('Test Case 11 - write (small)'):
+            # Read 1MB from small file
+            # Write it back to test the writes
+            # Test Case 11a - Check that the write succeeded
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s' % (fs_type, ADDR, SMALL_FILE),
+                '%swrite host 0:0 %x /%s.w $filesize'
+                    % (fs_type, ADDR, SMALL_FILE)])
+            assert('1048576 bytes written' in ''.join(output))
+
+            # Test Case 11b - Check md5 of written to is same
+            # as the one read from
+            output = u_boot_console.run_command_list([
+                '%sload host 0:0 %x /%s.w' % (fs_type, ADDR, SMALL_FILE),
+                'md5sum %x $filesize' % ADDR,
+                'setenv filesize'])
+            assert(md5val[0] in ''.join(output))
+
+    def test_fs12(self, u_boot_console, fs_obj_basic):
+        fs_type,fs_img,md5val = fs_obj_basic
+        with u_boot_console.log.section('Test Case 12 - write (".")'):
+            # Next test case checks writing a file whose dirent
+            # is the first in the block, which is always true for "."
+            # The write should fail, but the lookup should work
+            # Test Case 12 - Check directory traversal
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%swrite host 0:0 %x /. 0x10' % (fs_type, ADDR)])
+            assert('Unable to write' in ''.join(output))
+
+    def test_fs13(self, u_boot_console, fs_obj_basic):
+        fs_type,fs_img,md5val = fs_obj_basic
+        with u_boot_console.log.section('Test Case 13 - write (small w/ "./")'):
+            # Read 1MB from small file
+            # Write it via "same directory", i.e. "." dirent
+            # Test Case 13a - Check directory traversal
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s' % (fs_type, ADDR, SMALL_FILE),
+                '%swrite host 0:0 %x /./%s2 $filesize'
+                    % (fs_type, ADDR, SMALL_FILE)])
+            assert('1048576 bytes written' in ''.join(output))
+
+            # Test Case 13b - Check md5 of written to is same
+            # as the one read from
+            output = u_boot_console.run_command_list([
+                'mw.b %x 00 100' % ADDR,
+                '%sload host 0:0 %x /./%s2' % (fs_type, ADDR, SMALL_FILE),
+                'md5sum %x $filesize' % ADDR,
+                'setenv filesize'])
+            assert(md5val[0] in ''.join(output))
+
+            # Test Case 13c - Check md5 of written to is same
+            # as the one read from
+            output = u_boot_console.run_command_list([
+                'mw.b %x 00 100' % ADDR,
+                '%sload host 0:0 %x /%s2' % (fs_type, ADDR, SMALL_FILE),
+                'md5sum %x $filesize' % ADDR,
+                'setenv filesize'])
+            assert(md5val[0] in ''.join(output))
-- 
2.18.0

[U-Boot] [RFC 2/3] test/py: fs: add extended write operation test

[AKASHI Takahiro]

In this commit and the next one, test scripts for new filesystem
functionalities introduced by my patch set, "fs: fat: extend FAT write
operations," are provided.

In particular, this patch adds test cases for sub-directory write
and write with non-zero offset.

Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
---
 test/py/tests/test_fs/conftest.py    |  83 +++++++++++
 test/py/tests/test_fs/fstest_defs.py |   3 +
 test/py/tests/test_fs/test_ext.py    | 197 +++++++++++++++++++++++++++
 3 files changed, 283 insertions(+)
 create mode 100644 test/py/tests/test_fs/test_ext.py

diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
index fefeb4c9663f..087c9d30255c 100644
--- a/test/py/tests/test_fs/conftest.py
+++ b/test/py/tests/test_fs/conftest.py
@@ -8,6 +8,7 @@ from subprocess import call, check_call, check_output, CalledProcessError
 from fstest_defs import *
 
 supported_fs_basic = ['fat16', 'fat32', 'ext4']
+supported_fs_ext = ['fat16', 'fat32']
 
 #
 # Filesystem test specific setup
@@ -18,6 +19,7 @@ def pytest_addoption(parser):
 
 def pytest_configure(config):
     global supported_fs_basic
+    global supported_fs_ext
 
     def intersect(listA, listB):
         return  [x for x in listA if x in listB]
@@ -26,11 +28,15 @@ def pytest_configure(config):
     if supported_fs:
         print("*** FS TYPE modified: %s" % supported_fs)
         supported_fs_basic =  intersect(supported_fs, supported_fs_basic)
+        supported_fs_ext =  intersect(supported_fs, supported_fs_ext)
 
 def pytest_generate_tests(metafunc):
     if 'fs_obj_basic' in metafunc.fixturenames:
         metafunc.parametrize('fs_obj_basic', supported_fs_basic,
             indirect=True, scope='module')
+    if 'fs_obj_ext' in metafunc.fixturenames:
+        metafunc.parametrize('fs_obj_ext', supported_fs_ext,
+            indirect=True, scope='module')
 
 #
 # Helper functions
@@ -173,3 +179,80 @@ def fs_obj_basic(request, u_boot_config):
 	call('rmdir -rf %s' % mount_dir, shell=True)
 #        if fs_img:
 #            call('rm -f %s' % fs_img, shell=True)
+
+
+#
+# Fixture for extended fs test
+#
+# NOTE: yield_fixture was deprecated since pytest-3.0
+ at pytest.yield_fixture()
+def fs_obj_ext(request, u_boot_config):
+    fs_type = request.param
+    fs_img = ''
+
+    fs_ubtype = fstype_to_ubname(fs_type)
+    check_ubconfig(u_boot_config, fs_ubtype)
+
+    mount_dir = u_boot_config.persistent_data_dir + '/mnt'
+    min_file = mount_dir + '/' + MIN_FILE
+    tmp_file = mount_dir + '/tmpfile'
+
+    try:
+
+        # 256KiB volume
+        fs_img = mk_fs(u_boot_config, fs_type, 0x40000, '256KB')
+
+        # Mount the image so we can populate it.
+        check_call('mkdir -p %s' % mount_dir, shell=True)
+        check_call('sudo mount -o loop,rw,dmask=0000 %s %s'
+            % (fs_img, mount_dir), shell=True)
+
+        # Create a test directory
+        check_call('mkdir %s/dir1' % mount_dir, shell=True)
+
+        # Create a small file and calculate md5
+        check_call('dd if=/dev/urandom of=%s bs=1K count=20'
+            % min_file, shell=True)
+        out = check_output(
+            'dd if=%s bs=1K 2> /dev/null | md5sum'
+            % min_file, shell=True)
+        md5val = [ out.split()[0] ]
+
+        # Calculate md5sum of Test Case 4
+        check_call('dd if=%s of=%s bs=1K count=20'
+            % (min_file, tmp_file), shell=True)
+        check_call('dd if=%s of=%s bs=1K seek=5 count=20'
+            % (min_file, tmp_file), shell=True)
+        out = check_output('dd if=%s bs=1K 2> /dev/null | md5sum'
+            % tmp_file, shell=True)
+        md5val.append(out.split()[0])
+
+        # Calculate md5sum of Test Case 5
+        check_call('dd if=%s of=%s bs=1K count=20'
+            % (min_file, tmp_file), shell=True)
+        check_call('dd if=%s of=%s bs=1K seek=5 count=5'
+            % (min_file, tmp_file), shell=True)
+        out = check_output('dd if=%s bs=1K 2> /dev/null | md5sum'
+            % tmp_file, shell=True)
+        md5val.append(out.split()[0])
+
+        # Calculate md5sum of Test Case 7
+        check_call('dd if=%s of=%s bs=1K count=20'
+            % (min_file, tmp_file), shell=True)
+        check_call('dd if=%s of=%s bs=1K seek=20 count=20'
+            % (min_file, tmp_file), shell=True)
+        out = check_output('dd if=%s bs=1K 2> /dev/null | md5sum'
+            % tmp_file, shell=True)
+        md5val.append(out.split()[0])
+
+        check_call('rm %s' % tmp_file, shell=True)
+        check_call('sudo umount %s' % mount_dir, shell=True)
+    except CalledProcessError:
+        pytest.skip('Setup failed for filesystem: ' + fs_type)
+    else:
+        yield [fs_ubtype, fs_img, md5val]
+    finally:
+        call('sudo umount %s' % mount_dir, shell=True)
+        call('rmdir -rf %s' % mount_dir, shell=True)
+#        if fs_img:
+#            call('rm -f %s' % fs_img, shell=True)
diff --git a/test/py/tests/test_fs/fstest_defs.py b/test/py/tests/test_fs/fstest_defs.py
index f26dd06cacf2..5f107562d952 100644
--- a/test/py/tests/test_fs/fstest_defs.py
+++ b/test/py/tests/test_fs/fstest_defs.py
@@ -1,5 +1,8 @@
 # SPDX-License-Identifier:      GPL-2.0+
 
+# $MIN_FILE is the name of the 20KB file in the file system image
+MIN_FILE='testfile'
+
 # $SMALL_FILE is the name of the 1MB file in the file system image
 SMALL_FILE='1MB.file'
 
diff --git a/test/py/tests/test_fs/test_ext.py b/test/py/tests/test_fs/test_ext.py
new file mode 100644
index 000000000000..14695bd6f1e9
--- /dev/null
+++ b/test/py/tests/test_fs/test_ext.py
@@ -0,0 +1,197 @@
+# SPDX-License-Identifier:      GPL-2.0+
+# Copyright (c) 2018, Linaro Limited
+# Author: Takahiro Akashi <takahiro.akashi@linaro.org>
+#
+# U-Boot File System:Exntented Test
+
+"""
+This test verifies extented write operation on file system.
+"""
+
+import pytest
+import re
+from fstest_defs import *
+
+ at pytest.mark.boardspec('sandbox')
+class TestFsExt(object):
+    def test_fs_e1(self, u_boot_console, fs_obj_ext):
+        fs_type,fs_img,md5val = fs_obj_ext
+        with u_boot_console.log.section('Test Case 1 - write with abs path'):
+            # Test Case 1a - Check if command successfully returned
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s' % (fs_type, ADDR, MIN_FILE),
+                '%swrite host 0:0 %x /dir1/%s.w1 $filesize'
+                    % (fs_type, ADDR, MIN_FILE)])
+            assert('20480 bytes written' in ''.join(output))
+
+            # Test Case 1b - Check md5 of file content
+            output = u_boot_console.run_command_list([
+                'mw.b %x 00 100' % ADDR,
+                '%sload host 0:0 %x /dir1/%s.w1' % (fs_type, ADDR, MIN_FILE),
+                'md5sum %x $filesize' % ADDR,
+                'setenv filesize'])
+            assert(md5val[0] in ''.join(output))
+
+    def test_fs_e2(self, u_boot_console, fs_obj_ext):
+        fs_type,fs_img,md5val = fs_obj_ext
+        with u_boot_console.log.section('Test Case 2 - write with rel path'):
+            # Test Case 2a - Check if command successfully returned
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s' % (fs_type, ADDR, MIN_FILE),
+                '%swrite host 0:0 %x dir1/%s.w2 $filesize'
+                    % (fs_type, ADDR, MIN_FILE)])
+            assert('20480 bytes written' in ''.join(output))
+
+            # Test Case 2b - Check md5 of file content
+            output = u_boot_console.run_command_list([
+                'mw.b %x 00 100' % ADDR,
+                '%sload host 0:0 %x dir1/%s.w2' % (fs_type, ADDR, MIN_FILE),
+                'md5sum %x $filesize' % ADDR,
+                'setenv filesize'])
+            assert(md5val[0] in ''.join(output))
+
+    def test_fs_e3(self, u_boot_console, fs_obj_ext):
+        fs_type,fs_img,md5val = fs_obj_ext
+        with u_boot_console.log.section('Test Case 3 - write with invalid path'):
+            # Test Case 3 - Check if command expectedly failed
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s' % (fs_type, ADDR, MIN_FILE),
+                '%swrite host 0:0 %x /dir1/none/%s.w3 $filesize'
+                    % (fs_type, ADDR, MIN_FILE)])
+            assert('Unable to write "/dir1/none/' in ''.join(output))
+
+    def test_fs_e4(self, u_boot_console, fs_obj_ext):
+        fs_type,fs_img,md5val = fs_obj_ext
+        with u_boot_console.log.section('Test Case 4 - write with non-zero offset, enlarged'):
+            # Test Case 4a - Check if command successfully returned
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s' % (fs_type, ADDR, MIN_FILE),
+                '%swrite host 0:0 %x /dir1/%s.w4 $filesize'
+                    % (fs_type, ADDR, MIN_FILE)])
+            output = u_boot_console.run_command(
+                '%swrite host 0:0 %x /dir1/%s.w4 $filesize 0x1400'
+                    % (fs_type, ADDR, MIN_FILE))
+            assert('20480 bytes written' in output)
+
+            # Test Case 4b - Check size of written file
+            output = u_boot_console.run_command_list([
+                '%ssize host 0:0 /dir1/%s.w4' % (fs_type, MIN_FILE),
+                'printenv filesize',
+                'setenv filesize'])
+            assert('filesize=6400' in ''.join(output))
+
+            # Test Case 4c - Check md5 of file content
+            output = u_boot_console.run_command_list([
+                'mw.b %x 00 100' % ADDR,
+                '%sload host 0:0 %x /dir1/%s.w4' % (fs_type, ADDR, MIN_FILE),
+                'md5sum %x $filesize' % ADDR,
+                'setenv filesize'])
+            assert(md5val[1] in ''.join(output))
+
+    def test_fs_e5(self, u_boot_console, fs_obj_ext):
+        fs_type,fs_img,md5val = fs_obj_ext
+        with u_boot_console.log.section('Test Case 5 - write with non-zero offset, shrunk'):
+            # Test Case 5a - Check if command successfully returned
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s' % (fs_type, ADDR, MIN_FILE),
+                '%swrite host 0:0 %x /dir1/%s.w5 $filesize'
+                    % (fs_type, ADDR, MIN_FILE)])
+            output = u_boot_console.run_command(
+                '%swrite host 0:0 %x /dir1/%s.w5 0x1400 0x1400'
+                    % (fs_type, ADDR, MIN_FILE))
+            assert('5120 bytes written' in output)
+
+            # Test Case 5b - Check size of written file
+            output = u_boot_console.run_command_list([
+                '%ssize host 0:0 /dir1/%s.w5' % (fs_type, MIN_FILE),
+                'printenv filesize',
+                'setenv filesize'])
+            assert('filesize=2800' in ''.join(output))
+
+            # Test Case 5c - Check md5 of file content
+            output = u_boot_console.run_command_list([
+                'mw.b %x 00 100' % ADDR,
+                '%sload host 0:0 %x /dir1/%s.w5' % (fs_type, ADDR, MIN_FILE),
+                'md5sum %x $filesize' % ADDR,
+                'setenv filesize'])
+            assert(md5val[2] in ''.join(output))
+
+    def test_fs_e6(self, u_boot_console, fs_obj_ext):
+        fs_type,fs_img,md5val = fs_obj_ext
+        with u_boot_console.log.section('Test Case 6 - write at zero, truncating to zero'):
+            # Test Case 6a - Check if command successfully returned
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s' % (fs_type, ADDR, MIN_FILE),
+                '%swrite host 0:0 %x /dir1/%s.w6 $filesize'
+                    % (fs_type, ADDR, MIN_FILE)])
+            output = u_boot_console.run_command(
+                '%swrite host 0:0 %x /dir1/%s.w6 0 0'
+                    % (fs_type, ADDR, MIN_FILE))
+            assert('0 bytes written' in output)
+
+            # Test Case 6b - Check size of written file
+            output = u_boot_console.run_command_list([
+                '%ssize host 0:0 /dir1/%s.w6' % (fs_type, MIN_FILE),
+                'printenv filesize',
+                'setenv filesize'])
+            assert('filesize=0' in ''.join(output))
+
+    def test_fs_e7(self, u_boot_console, fs_obj_ext):
+        fs_type,fs_img,md5val = fs_obj_ext
+        with u_boot_console.log.section('Test Case 7 - write at the end (append)'):
+            # Test Case 7a - Check if command successfully returned
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s' % (fs_type, ADDR, MIN_FILE),
+                '%swrite host 0:0 %x /dir1/%s.w7 $filesize'
+                    % (fs_type, ADDR, MIN_FILE)])
+            output = u_boot_console.run_command(
+                '%swrite host 0:0 %x /dir1/%s.w7 $filesize $filesize'
+                    % (fs_type, ADDR, MIN_FILE))
+            assert('20480 bytes written' in output)
+
+            # Test Case 7b - Check size of written file
+            output = u_boot_console.run_command_list([
+                '%ssize host 0:0 /dir1/%s.w7' % (fs_type, MIN_FILE),
+                'printenv filesize',
+                'setenv filesize'])
+            assert('filesize=a000' in ''.join(output))
+
+            # Test Case 7c - Check md5 of file content
+            output = u_boot_console.run_command_list([
+                'mw.b %x 00 100' % ADDR,
+                '%sload host 0:0 %x /dir1/%s.w7' % (fs_type, ADDR, MIN_FILE),
+                'md5sum %x $filesize' % ADDR,
+                'setenv filesize'])
+            assert(md5val[3] in ''.join(output))
+
+    def test_fs_e8(self, u_boot_console, fs_obj_ext):
+        fs_type,fs_img,md5val = fs_obj_ext
+        with u_boot_console.log.section('Test Case 8 - write beyond the end'):
+            # Test Case 8a - Check if command expectedly failed
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s' % (fs_type, ADDR, MIN_FILE),
+                '%swrite host 0:0 %x /dir1/%s.w8 $filesize'
+                    % (fs_type, ADDR, MIN_FILE)])
+            output = u_boot_console.run_command(
+                '%swrite host 0:0 %x /dir1/%s.w8 0x1400 %x'
+                    % (fs_type, ADDR, MIN_FILE, 0x100000 + 0x1400))
+            assert('Unable to write "/dir1' in output)
+
+    def test_fs_e9(self, u_boot_console, fs_obj_ext):
+        fs_type,fs_img,md5val = fs_obj_ext
+        with u_boot_console.log.section('Test Case 8 - write to non-existing file with offset'):
+            # Test Case 9a - Check if command expectedly failed
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%sload host 0:0 %x /%s' % (fs_type, ADDR, MIN_FILE),
+                '%swrite host 0:0 %x /dir1/%s.w9 0x1400 0x1400'
+                    % (fs_type, ADDR, MIN_FILE)])
+            assert('Unable to write "/dir1' in ''.join(output))
-- 
2.18.0

[U-Boot] [RFC 3/3] test/py: fs: add fstest/mkdir test

[AKASHI Takahiro]

In this commit, test cases for mkdir interfaces are added as part of
"test_fs" test suite.

Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
---
 test/py/tests/test_fs/conftest.py   | 31 ++++++++++++
 test/py/tests/test_fs/test_mkdir.py | 77 +++++++++++++++++++++++++++++
 2 files changed, 108 insertions(+)
 create mode 100644 test/py/tests/test_fs/test_mkdir.py

diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
index 087c9d30255c..6cb308eca596 100644
--- a/test/py/tests/test_fs/conftest.py
+++ b/test/py/tests/test_fs/conftest.py
@@ -9,6 +9,7 @@ from fstest_defs import *
 
 supported_fs_basic = ['fat16', 'fat32', 'ext4']
 supported_fs_ext = ['fat16', 'fat32']
+supported_fs_mkdir = ['fat16', 'fat32']
 
 #
 # Filesystem test specific setup
@@ -20,6 +21,7 @@ def pytest_addoption(parser):
 def pytest_configure(config):
     global supported_fs_basic
     global supported_fs_ext
+    global supported_fs_mkdir
 
     def intersect(listA, listB):
         return  [x for x in listA if x in listB]
@@ -29,6 +31,7 @@ def pytest_configure(config):
         print("*** FS TYPE modified: %s" % supported_fs)
         supported_fs_basic =  intersect(supported_fs, supported_fs_basic)
         supported_fs_ext =  intersect(supported_fs, supported_fs_ext)
+        supported_fs_mkdir =  intersect(supported_fs, supported_fs_mkdir)
 
 def pytest_generate_tests(metafunc):
     if 'fs_obj_basic' in metafunc.fixturenames:
@@ -37,6 +40,9 @@ def pytest_generate_tests(metafunc):
     if 'fs_obj_ext' in metafunc.fixturenames:
         metafunc.parametrize('fs_obj_ext', supported_fs_ext,
             indirect=True, scope='module')
+    if 'fs_obj_mkdir' in metafunc.fixturenames:
+        metafunc.parametrize('fs_obj_mkdir', supported_fs_mkdir,
+            indirect=True, scope='module')
 
 #
 # Helper functions
@@ -256,3 +262,28 @@ def fs_obj_ext(request, u_boot_config):
         call('rmdir -rf %s' % mount_dir, shell=True)
 #        if fs_img:
 #            call('rm -f %s' % fs_img, shell=True)
+
+
+#
+# Fixture for mkdir test
+#
+# NOTE: yield_fixture was deprecated since pytest-3.0
+ at pytest.yield_fixture()
+def fs_obj_mkdir(request, u_boot_config):
+    fs_type = request.param
+    fs_img = ''
+
+    fs_ubtype = fstype_to_ubname(fs_type)
+    check_ubconfig(u_boot_config, fs_ubtype)
+
+    try:
+        # 256KiB volume
+        fs_img = mk_fs(u_boot_config, fs_type, 0x40000, '256KB')
+    except:
+        pytest.skip('Setup failed for filesystem: ' + fs_type)
+    else:
+        yield [fs_ubtype, fs_img]
+    finally:
+        print 'Dummy'
+#        if fs_img:
+#            call('rm -f %s' % fs_img, shell=True)
diff --git a/test/py/tests/test_fs/test_mkdir.py b/test/py/tests/test_fs/test_mkdir.py
new file mode 100644
index 000000000000..f42d461e84cb
--- /dev/null
+++ b/test/py/tests/test_fs/test_mkdir.py
@@ -0,0 +1,77 @@
+# SPDX-License-Identifier:      GPL-2.0+
+# Copyright (c) 2018, Linaro Limited
+# Author: Takahiro Akashi <takahiro.akashi@linaro.org>
+#
+# U-Boot File System:mkdir Test
+
+"""
+This test verifies mkdir operation on file system.
+"""
+
+import pytest
+
+ at pytest.mark.boardspec('sandbox')
+class TestMkdir(object):
+    def test_mkdir1(self, u_boot_console, fs_obj_mkdir):
+        fs_type,fs_img = fs_obj_mkdir
+        with u_boot_console.log.section('Test Case 1 - mkdir'):
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%smkdir host 0:0 dir1' % fs_type,
+                '%sls host 0:0 /' % fs_type])
+            assert('dir1/' in ''.join(output))
+
+            output = u_boot_console.run_command(
+                '%sls host 0:0 dir1' % fs_type)
+            assert('./'   in output)
+            assert('../'  in output)
+
+    def test_mkdir2(self, u_boot_console, fs_obj_mkdir):
+        fs_type,fs_img = fs_obj_mkdir
+        with u_boot_console.log.section('Test Case 2 - mkdir (sub-sub directory)'):
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%smkdir host 0:0 dir1/dir2' % fs_type,
+                '%sls host 0:0 dir1' % fs_type])
+            assert('dir2/' in ''.join(output))
+
+    def test_mkdir3(self, u_boot_console, fs_obj_mkdir):
+        fs_type,fs_img = fs_obj_mkdir
+        with u_boot_console.log.section('Test Case 3 - mkdir (non-existing path)'):
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%smkdir host 0:0 none/dir3' % fs_type])
+            assert('Unable to create a directory' in ''.join(output))
+
+    def test_mkdir4(self, u_boot_console, fs_obj_mkdir):
+        fs_type,fs_img = fs_obj_mkdir
+        with u_boot_console.log.section('Test Case 4 - mkdir (".")'):
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%smkdir host 0:0 .' % fs_type])
+            assert('Unable to create a directory' in ''.join(output))
+
+    def test_mkdir5(self, u_boot_console, fs_obj_mkdir):
+        fs_type,fs_img = fs_obj_mkdir
+        with u_boot_console.log.section('Test Case 5 - mkdir ("..")'):
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%smkdir host 0:0 ..' % fs_type])
+            assert('Unable to create a directory' in ''.join(output))
+
+    def test_mkdir6(self, u_boot_console, fs_obj_mkdir):
+        fs_type,fs_img = fs_obj_mkdir
+        with u_boot_console.log.section('Test Case 6 - mkdir (create many)'):
+            output = u_boot_console.run_command_list([
+                'host bind 0 %s' % fs_img,
+                '%smkdir host 0:0 dir6' % fs_type,
+                '%sls host 0:0 /' % fs_type])
+            assert('dir6/' in ''.join(output))
+
+            for i in range(0, 20):
+                output = u_boot_console.run_command(
+                    '%smkdir host 0:0 dir6/0123456789abcdef%02x'
+                    % (fs_type, i))
+            output = u_boot_console.run_command('%sls host 0:0 dir6' % fs_type)
+            assert('0123456789abcdef00/'  in output)
+            assert('0123456789abcdef13/'  in output)
-- 
2.18.0

[U-Boot] [RFC 1/3] test/py: convert fs-test.sh to pytest

[Heinrich Schuchardt]

On 08/23/2018 09:25 AM, AKASHI Takahiro wrote:
> In this commit, the same set of test cases as in test/fs/fs-test.sh
> is provided using pytest framework.
> Actually, fs-test.sh provides three variants:"sb" (sb command), "nonfs"
> (fatxx and etc.) and "fs" (hostfs), and this patch currently supports
> only "nonfs" variant; So it is not a replacement of fs-test.sh for now.
> 
> Simple usage:
>   $ py.test test/py/tests/test_fs [<other options>]
> 
> You may also specify filesystem types to be tested:
>   $ py.test test/py/tests/test_fs --fs-type fat32 [<other options>]
> 
> Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
> ---
>  test/py/tests/test_fs/conftest.py    | 175 +++++++++++++++++++
>  test/py/tests/test_fs/fstest_defs.py |  10 ++
>  test/py/tests/test_fs/test_basic.py  | 246 +++++++++++++++++++++++++++
>  3 files changed, 431 insertions(+)
>  create mode 100644 test/py/tests/test_fs/conftest.py
>  create mode 100644 test/py/tests/test_fs/fstest_defs.py
>  create mode 100644 test/py/tests/test_fs/test_basic.py
> 
> diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
> new file mode 100644
> index 000000000000..fefeb4c9663f
> --- /dev/null
> +++ b/test/py/tests/test_fs/conftest.py
> @@ -0,0 +1,175 @@
> +# SPDX-License-Identifier:      GPL-2.0+
> +# Copyright (c) 2018, Linaro Limited
> +# Author: Takahiro Akashi <takahiro.akashi@linaro.org>
> +
> +import pytest
> +import re
> +from subprocess import call, check_call, check_output, CalledProcessError
> +from fstest_defs import *
> +
> +supported_fs_basic = ['fat16', 'fat32', 'ext4']
> +
> +#
> +# Filesystem test specific setup
> +#
> +def pytest_addoption(parser):
> +    parser.addoption('--fs-type', action='append', default=None,
> +        help='Targeting Filesystem Types')
> +
> +def pytest_configure(config):
> +    global supported_fs_basic
> +
> +    def intersect(listA, listB):
> +        return  [x for x in listA if x in listB]
> +
> +    supported_fs = config.getoption('fs_type')
> +    if supported_fs:
> +        print("*** FS TYPE modified: %s" % supported_fs)
> +        supported_fs_basic =  intersect(supported_fs, supported_fs_basic)
> +
> +def pytest_generate_tests(metafunc):
> +    if 'fs_obj_basic' in metafunc.fixturenames:
> +        metafunc.parametrize('fs_obj_basic', supported_fs_basic,
> +            indirect=True, scope='module')
> +
> +#
> +# Helper functions
> +#
> +def fstype_to_ubname(fs_type):
> +    if re.match('fat', fs_type):
> +        return 'fat'
> +    else:
> +        return fs_type
> +
> +def check_ubconfig(config, fs_type):
> +    if not config.buildconfig.get('config_cmd_%s' % fs_type, None):
> +        pytest.skip('.config feature "CMD_%s" not enabled' % fs_type.upper())
> +    if not config.buildconfig.get('config_%s_write' % fs_type, None):
> +        pytest.skip('.config feature "%s_WRITE" not enabled'
> +        % fs_type.upper())
> +
> +def mk_fs(config, fs_type, size, id):
> +    fs_img = '%s.%s.img' % (id, fs_type)
> +    fs_img = config.persistent_data_dir + '/' + fs_img
> +
> +    if fs_type == 'fat16':
> +        mkfs_opt = '-F 16'
> +    elif fs_type == 'fat32':
> +        mkfs_opt = '-F 32'
> +    else:
> +        mkfs_opt = ''
> +
> +    if re.match('fat', fs_type):
> +        fs_lnxtype = 'vfat'
> +    else:
> +        fs_lnxtype = fs_type
> +
> +    count = (size + 1023) / 1024
> +
> +    try:
> +        check_call('rm -f %s' % fs_img, shell=True)
> +        check_call('dd if=/dev/zero of=%s bs=1K count=%d'
> +            % (fs_img, count), shell=True)
> +        check_call('mkfs.%s %s %s'
> +            % (fs_lnxtype, mkfs_opt, fs_img), shell=True)
> +        return fs_img
> +    except CalledProcessError:
> +        call('rm -f %s' % fs_img, shell=True)
> +        raise
> +
> +#
> +# Fixture for basic fs test
> +#     derived from test/fs/fs-test.sh
> +#
> +# NOTE: yield_fixture was deprecated since pytest-3.0
> + at pytest.yield_fixture()
> +def fs_obj_basic(request, u_boot_config):
> +    fs_type = request.param
> +    fs_img = ''
> +
> +    fs_ubtype = fstype_to_ubname(fs_type)
> +    check_ubconfig(u_boot_config, fs_ubtype)
> +
> +    mount_dir = u_boot_config.persistent_data_dir + '/mnt'
> +    small_file = mount_dir + '/' + SMALL_FILE
> +    big_file = mount_dir + '/' + BIG_FILE
> +    try:
> +
> +        # 3GiB volume
> +        fs_img = mk_fs(u_boot_config, fs_type, 0xc0000000, '3GB')
> +
> +        # Mount the image so we can populate it.
> +        check_call('mkdir -p %s' % mount_dir, shell=True)
> +        check_call('sudo mount -o loop,rw %s %s'
> +            % (fs_img, mount_dir), shell=True)

Should I grant sudo to anybody who can commit to U-Boot?

Just use exfat-fuse and fuse2fs.

> +
> +        # Create a subdirectory.
> +        check_call('sudo mkdir %s/SUBDIR' % mount_dir, shell=True)
> +
> +        # Create big file in this image.
> +        # Note that we work only on the start 1MB, couple MBs in the 2GB range
> +        # and the last 1 MB of the huge 2.5GB file.
> +        # So, just put random values only in those areas.
> +        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=1'
> +	    % big_file, shell=True)


No need to use sudo to create a file.

Best regards

Heinrich

> +        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=2 seek=2047'
> +            % big_file, shell=True)
> +        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=1 seek=2499'
> +            % big_file, shell=True)
> +
> +        # Create a small file in this image.
> +        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=1'
> +	    % small_file, shell=True)
> +
> +        # Delete the small file copies which possibly are written as part of a
> +        # previous test.
> +        # check_call('sudo rm -f "%s.w"' % MB1, shell=True)
> +        # check_call('sudo rm -f "%s.w2"' % MB1, shell=True)
> +
> +        # Generate the md5sums of reads that we will test against small file
> +        out = check_output(
> +            'dd if=%s bs=1M skip=0 count=1 2> /dev/null | md5sum'
> +	    % small_file, shell=True)
> +        md5val = [ out.split()[0] ]
> +
> +        # Generate the md5sums of reads that we will test against big file
> +        # One from beginning of file.
> +        out = check_output(
> +            'dd if=%s bs=1M skip=0 count=1 2> /dev/null | md5sum'
> +	    % big_file, shell=True)
> +        md5val.append(out.split()[0])
> +
> +        # One from end of file.
> +        out = check_output(
> +            'dd if=%s bs=1M skip=2499 count=1 2> /dev/null | md5sum'
> +	    % big_file, shell=True)
> +        md5val.append(out.split()[0])
> +
> +        # One from the last 1MB chunk of 2GB
> +        out = check_output(
> +            'dd if=%s bs=1M skip=2047 count=1 2> /dev/null | md5sum'
> +	    % big_file, shell=True)
> +        md5val.append(out.split()[0])
> +
> +        # One from the start 1MB chunk from 2GB
> +        out = check_output(
> +            'dd if=%s bs=1M skip=2048 count=1 2> /dev/null | md5sum'
> +	    % big_file, shell=True)
> +        md5val.append(out.split()[0])
> +
> +        # One 1MB chunk crossing the 2GB boundary
> +        out = check_output(
> +            'dd if=%s bs=512K skip=4095 count=2 2> /dev/null | md5sum'
> +	    % big_file, shell=True)
> +        md5val.append(out.split()[0])
> +
> +        check_call('sudo umount %s' % mount_dir, shell=True)
> +    except CalledProcessError:
> +        pytest.skip('Setup failed for filesystem: ' + fs_type)
> +    else:
> +        yield [fs_ubtype, fs_img, md5val]
> +    finally:
> +        call('sudo umount %s' % mount_dir, shell=True)
> +	call('rmdir -rf %s' % mount_dir, shell=True)
> +#        if fs_img:
> +#            call('rm -f %s' % fs_img, shell=True)
> diff --git a/test/py/tests/test_fs/fstest_defs.py b/test/py/tests/test_fs/fstest_defs.py
> new file mode 100644
> index 000000000000..f26dd06cacf2
> --- /dev/null
> +++ b/test/py/tests/test_fs/fstest_defs.py
> @@ -0,0 +1,10 @@
> +# SPDX-License-Identifier:      GPL-2.0+
> +
> +# $SMALL_FILE is the name of the 1MB file in the file system image
> +SMALL_FILE='1MB.file'
> +
> +# $BIG_FILE is the name of the 2.5GB file in the file system image
> +BIG_FILE='2.5GB.file'
> +
> +ADDR=0x01000008
> +LENGTH=0x00100000
> diff --git a/test/py/tests/test_fs/test_basic.py b/test/py/tests/test_fs/test_basic.py
> new file mode 100644
> index 000000000000..5258d98d42a9
> --- /dev/null
> +++ b/test/py/tests/test_fs/test_basic.py
> @@ -0,0 +1,246 @@
> +# SPDX-License-Identifier:      GPL-2.0+
> +# Copyright (c) 2018, Linaro Limited
> +# Author: Takahiro Akashi <takahiro.akashi@linaro.org>
> +#
> +# U-Boot File System:Basic Test
> +
> +"""
> +This test verifies basic read/write operation on file system.
> +"""
> +
> +import pytest
> +import re
> +from fstest_defs import *
> +
> + at pytest.mark.boardspec('sandbox')
> +class TestFsBasic(object):
> +    def test_fs1(self, u_boot_console, fs_obj_basic):
> +        fs_type,fs_img,md5val = fs_obj_basic
> +        with u_boot_console.log.section('Test Case 1 - ls'):
> +            # Test Case 1 - ls
> +            output = u_boot_console.run_command_list([
> +                'host bind 0 %s' % fs_img,
> +                '%sls host 0:0' % fs_type])
> +            assert(re.search('2621440000 *%s' % BIG_FILE, ''.join(output)))
> +            assert(re.search('1048576 *%s' % SMALL_FILE, ''.join(output)))
> +
> +            # In addition, test with a nonexistent directory to see if we crash.
> +            output2 = u_boot_console.run_command(
> +                '%sls host 0:0 invalid_d' % fs_type)
> +            if fs_type == 'ext4':
> +                assert('Can not find directory' in output2)
> +            else:
> +                assert('' == output2)
> +
> +    def test_fs2(self, u_boot_console, fs_obj_basic):
> +        fs_type,fs_img,md5val = fs_obj_basic
> +        with u_boot_console.log.section('Test Case 2 - size (small)'):
> +            # 1MB is 0x0010 0000
> +            # Test Case 2a - size of small file
> +            output = u_boot_console.run_command_list([
> +                'host bind 0 %s' % fs_img,
> +                '%ssize host 0:0 /%s' % (fs_type, SMALL_FILE),
> +                'printenv filesize',
> +                'setenv filesize'])
> +            assert('filesize=100000' in ''.join(output))
> +
> +            # Test Case 2b - size of small file via a path using '..'
> +            output = u_boot_console.run_command_list([
> +                '%ssize host 0:0 /SUBDIR/../%s' % (fs_type, SMALL_FILE),
> +                'printenv filesize',
> +                'setenv filesize'])
> +            assert('filesize=100000' in ''.join(output))
> +
> +    def test_fs3(self, u_boot_console, fs_obj_basic):
> +        fs_type,fs_img,md5val = fs_obj_basic
> +        with u_boot_console.log.section('Test Case 3 - size (big)'):
> +            # 2.5GB (1024*1024*2500) is 0x9C40 0000
> +            # Test Case 3 - size of big file
> +            output = u_boot_console.run_command_list([
> +                'host bind 0 %s' % fs_img,
> +                '%ssize host 0:0 /%s' % (fs_type, BIG_FILE),
> +                'printenv filesize',
> +                'setenv filesize'])
> +            assert('filesize=9c400000' in ''.join(output))
> +
> +    def test_fs4(self, u_boot_console, fs_obj_basic):
> +        fs_type,fs_img,md5val = fs_obj_basic
> +        with u_boot_console.log.section('Test Case 4 - load (small)'):
> +            # Test Case 4a - Read full 1MB of small file
> +            output = u_boot_console.run_command_list([
> +                'host bind 0 %s' % fs_img,
> +                '%sload host 0:0 %x /%s' % (fs_type, ADDR, SMALL_FILE),
> +                'printenv filesize'])
> +            assert('filesize=100000' in ''.join(output))
> +
> +            # Test Case 4b - Read full 1MB of small file
> +            output = u_boot_console.run_command_list([
> +                'md5sum %x $filesize' % ADDR,
> +                'setenv filesize'])
> +            assert(md5val[0] in ''.join(output))
> +
> +    def test_fs5(self, u_boot_console, fs_obj_basic):
> +        fs_type,fs_img,md5val = fs_obj_basic
> +        with u_boot_console.log.section('Test Case 5 - load (big)'):
> +            # Test Case 5a - First 1MB of big file
> +            output = u_boot_console.run_command_list([
> +                'host bind 0 %s' % fs_img,
> +                '%sload host 0:0 %x /%s %x 0x0' % (fs_type, ADDR, BIG_FILE, LENGTH),
> +                'printenv filesize'])
> +            assert('filesize=100000' in ''.join(output))
> +
> +            # Test Case 5b - First 1MB of big file
> +            output = u_boot_console.run_command_list([
> +                'md5sum %x $filesize' % ADDR,
> +                'setenv filesize'])
> +            assert(md5val[1] in ''.join(output))
> +
> +    def test_fs6(self, u_boot_console, fs_obj_basic):
> +        fs_type,fs_img,md5val = fs_obj_basic
> +        with u_boot_console.log.section('Test Case 6 - load (big from last 1MB)'):
> +            # fails for ext as no offset support
> +            # Test Case 6a - Last 1MB of big file
> +            output = u_boot_console.run_command_list([
> +                'host bind 0 %s' % fs_img,
> +                '%sload host 0:0 %x /%s %x 0x9c300000'
> +                    % (fs_type, ADDR, BIG_FILE, LENGTH),
> +                'printenv filesize'])
> +            assert('filesize=100000' in ''.join(output))
> +
> +            # Test Case 6b - Last 1MB of big file
> +            output = u_boot_console.run_command_list([
> +                'md5sum %x $filesize' % ADDR,
> +                'setenv filesize'])
> +            assert(md5val[2] in ''.join(output))
> +
> +    def test_fs7(self, u_boot_console, fs_obj_basic):
> +        fs_type,fs_img,md5val = fs_obj_basic
> +        with u_boot_console.log.section('Test Case 7 - load (big from last 1MB in 2GB)'):
> +            # fails for ext as no offset support
> +            # Test Case 7a - One from the last 1MB chunk of 2GB
> +            output = u_boot_console.run_command_list([
> +                'host bind 0 %s' % fs_img,
> +                '%sload host 0:0 %x /%s %x 0x7ff00000'
> +                    % (fs_type, ADDR, BIG_FILE, LENGTH),
> +                'printenv filesize'])
> +            assert('filesize=100000' in ''.join(output))
> +
> +            # Test Case 7b - One from the last 1MB chunk of 2GB
> +            output = u_boot_console.run_command_list([
> +                'md5sum %x $filesize' % ADDR,
> +                'setenv filesize'])
> +            assert(md5val[3] in ''.join(output))
> +
> +    def test_fs8(self, u_boot_console, fs_obj_basic):
> +        fs_type,fs_img,md5val = fs_obj_basic
> +        with u_boot_console.log.section('Test Case 8 - load (big from first 1MB in 2GB)'):
> +            # fails for ext as no offset support
> +            # Test Case 8a - One from the start 1MB chunk from 2GB
> +            output = u_boot_console.run_command_list([
> +                'host bind 0 %s' % fs_img,
> +                '%sload host 0:0 %x /%s %x 0x80000000'
> +                    % (fs_type, ADDR, BIG_FILE, LENGTH),
> +                'printenv filesize'])
> +            assert('filesize=100000' in ''.join(output))
> +
> +            # Test Case 8b - One from the start 1MB chunk from 2GB
> +            output = u_boot_console.run_command_list([
> +                'md5sum %x $filesize' % ADDR,
> +                'setenv filesize'])
> +            assert(md5val[4] in ''.join(output))
> +
> +    def test_fs9(self, u_boot_console, fs_obj_basic):
> +        fs_type,fs_img,md5val = fs_obj_basic
> +        with u_boot_console.log.section('Test Case 9 - load (crossing 2GB boundary)'):
> +            # fails for ext as no offset support
> +            # Test Case 9a - One 1MB chunk crossing the 2GB boundary
> +            output = u_boot_console.run_command_list([
> +                'host bind 0 %s' % fs_img,
> +                '%sload host 0:0 %x /%s %x 0x7ff80000'
> +                    % (fs_type, ADDR, BIG_FILE, LENGTH),
> +                'printenv filesize'])
> +            assert('filesize=100000' in ''.join(output))
> +
> +            # Test Case 9b - One 1MB chunk crossing the 2GB boundary
> +            output = u_boot_console.run_command_list([
> +                'md5sum %x $filesize' % ADDR,
> +                'setenv filesize'])
> +            assert(md5val[5] in ''.join(output))
> +
> +    def test_fs10(self, u_boot_console, fs_obj_basic):
> +        fs_type,fs_img,md5val = fs_obj_basic
> +        with u_boot_console.log.section('Test Case 10 - load (over file end)'):
> +            # Generic failure case
> +            # Test Case 10 - 2MB chunk from the last 1MB of big file
> +            output = u_boot_console.run_command_list([
> +                'host bind 0 %s' % fs_img,
> +                '%sload host 0:0 %x /%s 0x00200000 0x9c300000'
> +                    % (fs_type, ADDR, BIG_FILE),
> +                'printenv filesize',
> +                'md5sum %x $filesize' % ADDR,
> +                'setenv filesize'])
> +        assert('filesize=100000' in ''.join(output))
> +
> +    def test_fs11(self, u_boot_console, fs_obj_basic):
> +        fs_type,fs_img,md5val = fs_obj_basic
> +        with u_boot_console.log.section('Test Case 11 - write (small)'):
> +            # Read 1MB from small file
> +            # Write it back to test the writes
> +            # Test Case 11a - Check that the write succeeded
> +            output = u_boot_console.run_command_list([
> +                'host bind 0 %s' % fs_img,
> +                '%sload host 0:0 %x /%s' % (fs_type, ADDR, SMALL_FILE),
> +                '%swrite host 0:0 %x /%s.w $filesize'
> +                    % (fs_type, ADDR, SMALL_FILE)])
> +            assert('1048576 bytes written' in ''.join(output))
> +
> +            # Test Case 11b - Check md5 of written to is same
> +            # as the one read from
> +            output = u_boot_console.run_command_list([
> +                '%sload host 0:0 %x /%s.w' % (fs_type, ADDR, SMALL_FILE),
> +                'md5sum %x $filesize' % ADDR,
> +                'setenv filesize'])
> +            assert(md5val[0] in ''.join(output))
> +
> +    def test_fs12(self, u_boot_console, fs_obj_basic):
> +        fs_type,fs_img,md5val = fs_obj_basic
> +        with u_boot_console.log.section('Test Case 12 - write (".")'):
> +            # Next test case checks writing a file whose dirent
> +            # is the first in the block, which is always true for "."
> +            # The write should fail, but the lookup should work
> +            # Test Case 12 - Check directory traversal
> +            output = u_boot_console.run_command_list([
> +                'host bind 0 %s' % fs_img,
> +                '%swrite host 0:0 %x /. 0x10' % (fs_type, ADDR)])
> +            assert('Unable to write' in ''.join(output))
> +
> +    def test_fs13(self, u_boot_console, fs_obj_basic):
> +        fs_type,fs_img,md5val = fs_obj_basic
> +        with u_boot_console.log.section('Test Case 13 - write (small w/ "./")'):
> +            # Read 1MB from small file
> +            # Write it via "same directory", i.e. "." dirent
> +            # Test Case 13a - Check directory traversal
> +            output = u_boot_console.run_command_list([
> +                'host bind 0 %s' % fs_img,
> +                '%sload host 0:0 %x /%s' % (fs_type, ADDR, SMALL_FILE),
> +                '%swrite host 0:0 %x /./%s2 $filesize'
> +                    % (fs_type, ADDR, SMALL_FILE)])
> +            assert('1048576 bytes written' in ''.join(output))
> +
> +            # Test Case 13b - Check md5 of written to is same
> +            # as the one read from
> +            output = u_boot_console.run_command_list([
> +                'mw.b %x 00 100' % ADDR,
> +                '%sload host 0:0 %x /./%s2' % (fs_type, ADDR, SMALL_FILE),
> +                'md5sum %x $filesize' % ADDR,
> +                'setenv filesize'])
> +            assert(md5val[0] in ''.join(output))
> +
> +            # Test Case 13c - Check md5 of written to is same
> +            # as the one read from
> +            output = u_boot_console.run_command_list([
> +                'mw.b %x 00 100' % ADDR,
> +                '%sload host 0:0 %x /%s2' % (fs_type, ADDR, SMALL_FILE),
> +                'md5sum %x $filesize' % ADDR,
> +                'setenv filesize'])
> +            assert(md5val[0] in ''.join(output))
> 

[U-Boot] [RFC 1/3] test/py: convert fs-test.sh to pytest

[AKASHI Takahiro]

On Wed, Aug 29, 2018 at 11:36:51PM +0200, Heinrich Schuchardt wrote:
> On 08/23/2018 09:25 AM, AKASHI Takahiro wrote:
> > In this commit, the same set of test cases as in test/fs/fs-test.sh
> > is provided using pytest framework.
> > Actually, fs-test.sh provides three variants:"sb" (sb command), "nonfs"
> > (fatxx and etc.) and "fs" (hostfs), and this patch currently supports
> > only "nonfs" variant; So it is not a replacement of fs-test.sh for now.
> > 
> > Simple usage:
> >   $ py.test test/py/tests/test_fs [<other options>]
> > 
> > You may also specify filesystem types to be tested:
> >   $ py.test test/py/tests/test_fs --fs-type fat32 [<other options>]
> > 
> > Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
> > ---
> >  test/py/tests/test_fs/conftest.py    | 175 +++++++++++++++++++
> >  test/py/tests/test_fs/fstest_defs.py |  10 ++
> >  test/py/tests/test_fs/test_basic.py  | 246 +++++++++++++++++++++++++++
> >  3 files changed, 431 insertions(+)
> >  create mode 100644 test/py/tests/test_fs/conftest.py
> >  create mode 100644 test/py/tests/test_fs/fstest_defs.py
> >  create mode 100644 test/py/tests/test_fs/test_basic.py
> > 
> > diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
> > new file mode 100644
> > index 000000000000..fefeb4c9663f
> > --- /dev/null
> > +++ b/test/py/tests/test_fs/conftest.py
> > @@ -0,0 +1,175 @@
> > +# SPDX-License-Identifier:      GPL-2.0+
> > +# Copyright (c) 2018, Linaro Limited
> > +# Author: Takahiro Akashi <takahiro.akashi@linaro.org>
> > +
> > +import pytest
> > +import re
> > +from subprocess import call, check_call, check_output, CalledProcessError
> > +from fstest_defs import *
> > +
> > +supported_fs_basic = ['fat16', 'fat32', 'ext4']
> > +
> > +#
> > +# Filesystem test specific setup
> > +#
> > +def pytest_addoption(parser):
> > +    parser.addoption('--fs-type', action='append', default=None,
> > +        help='Targeting Filesystem Types')
> > +
> > +def pytest_configure(config):
> > +    global supported_fs_basic
> > +
> > +    def intersect(listA, listB):
> > +        return  [x for x in listA if x in listB]
> > +
> > +    supported_fs = config.getoption('fs_type')
> > +    if supported_fs:
> > +        print("*** FS TYPE modified: %s" % supported_fs)
> > +        supported_fs_basic =  intersect(supported_fs, supported_fs_basic)
> > +
> > +def pytest_generate_tests(metafunc):
> > +    if 'fs_obj_basic' in metafunc.fixturenames:
> > +        metafunc.parametrize('fs_obj_basic', supported_fs_basic,
> > +            indirect=True, scope='module')
> > +
> > +#
> > +# Helper functions
> > +#
> > +def fstype_to_ubname(fs_type):
> > +    if re.match('fat', fs_type):
> > +        return 'fat'
> > +    else:
> > +        return fs_type
> > +
> > +def check_ubconfig(config, fs_type):
> > +    if not config.buildconfig.get('config_cmd_%s' % fs_type, None):
> > +        pytest.skip('.config feature "CMD_%s" not enabled' % fs_type.upper())
> > +    if not config.buildconfig.get('config_%s_write' % fs_type, None):
> > +        pytest.skip('.config feature "%s_WRITE" not enabled'
> > +        % fs_type.upper())
> > +
> > +def mk_fs(config, fs_type, size, id):
> > +    fs_img = '%s.%s.img' % (id, fs_type)
> > +    fs_img = config.persistent_data_dir + '/' + fs_img
> > +
> > +    if fs_type == 'fat16':
> > +        mkfs_opt = '-F 16'
> > +    elif fs_type == 'fat32':
> > +        mkfs_opt = '-F 32'
> > +    else:
> > +        mkfs_opt = ''
> > +
> > +    if re.match('fat', fs_type):
> > +        fs_lnxtype = 'vfat'
> > +    else:
> > +        fs_lnxtype = fs_type
> > +
> > +    count = (size + 1023) / 1024
> > +
> > +    try:
> > +        check_call('rm -f %s' % fs_img, shell=True)
> > +        check_call('dd if=/dev/zero of=%s bs=1K count=%d'
> > +            % (fs_img, count), shell=True)
> > +        check_call('mkfs.%s %s %s'
> > +            % (fs_lnxtype, mkfs_opt, fs_img), shell=True)
> > +        return fs_img
> > +    except CalledProcessError:
> > +        call('rm -f %s' % fs_img, shell=True)
> > +        raise
> > +
> > +#
> > +# Fixture for basic fs test
> > +#     derived from test/fs/fs-test.sh
> > +#
> > +# NOTE: yield_fixture was deprecated since pytest-3.0
> > + at pytest.yield_fixture()
> > +def fs_obj_basic(request, u_boot_config):
> > +    fs_type = request.param
> > +    fs_img = ''
> > +
> > +    fs_ubtype = fstype_to_ubname(fs_type)
> > +    check_ubconfig(u_boot_config, fs_ubtype)
> > +
> > +    mount_dir = u_boot_config.persistent_data_dir + '/mnt'
> > +    small_file = mount_dir + '/' + SMALL_FILE
> > +    big_file = mount_dir + '/' + BIG_FILE
> > +    try:
> > +
> > +        # 3GiB volume
> > +        fs_img = mk_fs(u_boot_config, fs_type, 0xc0000000, '3GB')
> > +
> > +        # Mount the image so we can populate it.
> > +        check_call('mkdir -p %s' % mount_dir, shell=True)
> > +        check_call('sudo mount -o loop,rw %s %s'
> > +            % (fs_img, mount_dir), shell=True)
> 
> Should I grant sudo to anybody who can commit to U-Boot?

I don't get your point.
I think using "sudo" solely in testing should be allowed.

> Just use exfat-fuse and fuse2fs.

It will not be a good idea to use those tools which are not
provided in every distribution.
I'd like to leave "sudo mount" statement as a backstop.

> > +
> > +        # Create a subdirectory.
> > +        check_call('sudo mkdir %s/SUBDIR' % mount_dir, shell=True)
> > +
> > +        # Create big file in this image.
> > +        # Note that we work only on the start 1MB, couple MBs in the 2GB range
> > +        # and the last 1 MB of the huge 2.5GB file.
> > +        # So, just put random values only in those areas.
> > +        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=1'
> > +	    % big_file, shell=True)
> 
> 
> No need to use sudo to create a file.

Along with a change above, succeeding "sudo's" will be removed.

Thanks,
-Takahiro AKASHI


> Best regards
> 
> Heinrich
> 
> > +        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=2 seek=2047'
> > +            % big_file, shell=True)
> > +        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=1 seek=2499'
> > +            % big_file, shell=True)
> > +
> > +        # Create a small file in this image.
> > +        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=1'
> > +	    % small_file, shell=True)
> > +
> > +        # Delete the small file copies which possibly are written as part of a
> > +        # previous test.
> > +        # check_call('sudo rm -f "%s.w"' % MB1, shell=True)
> > +        # check_call('sudo rm -f "%s.w2"' % MB1, shell=True)
> > +
> > +        # Generate the md5sums of reads that we will test against small file
> > +        out = check_output(
> > +            'dd if=%s bs=1M skip=0 count=1 2> /dev/null | md5sum'
> > +	    % small_file, shell=True)
> > +        md5val = [ out.split()[0] ]
> > +
> > +        # Generate the md5sums of reads that we will test against big file
> > +        # One from beginning of file.
> > +        out = check_output(
> > +            'dd if=%s bs=1M skip=0 count=1 2> /dev/null | md5sum'
> > +	    % big_file, shell=True)
> > +        md5val.append(out.split()[0])
> > +
> > +        # One from end of file.
> > +        out = check_output(
> > +            'dd if=%s bs=1M skip=2499 count=1 2> /dev/null | md5sum'
> > +	    % big_file, shell=True)
> > +        md5val.append(out.split()[0])
> > +
> > +        # One from the last 1MB chunk of 2GB
> > +        out = check_output(
> > +            'dd if=%s bs=1M skip=2047 count=1 2> /dev/null | md5sum'
> > +	    % big_file, shell=True)
> > +        md5val.append(out.split()[0])
> > +
> > +        # One from the start 1MB chunk from 2GB
> > +        out = check_output(
> > +            'dd if=%s bs=1M skip=2048 count=1 2> /dev/null | md5sum'
> > +	    % big_file, shell=True)
> > +        md5val.append(out.split()[0])
> > +
> > +        # One 1MB chunk crossing the 2GB boundary
> > +        out = check_output(
> > +            'dd if=%s bs=512K skip=4095 count=2 2> /dev/null | md5sum'
> > +	    % big_file, shell=True)
> > +        md5val.append(out.split()[0])
> > +
> > +        check_call('sudo umount %s' % mount_dir, shell=True)
> > +    except CalledProcessError:
> > +        pytest.skip('Setup failed for filesystem: ' + fs_type)
> > +    else:
> > +        yield [fs_ubtype, fs_img, md5val]
> > +    finally:
> > +        call('sudo umount %s' % mount_dir, shell=True)
> > +	call('rmdir -rf %s' % mount_dir, shell=True)
> > +#        if fs_img:
> > +#            call('rm -f %s' % fs_img, shell=True)
> > diff --git a/test/py/tests/test_fs/fstest_defs.py b/test/py/tests/test_fs/fstest_defs.py
> > new file mode 100644
> > index 000000000000..f26dd06cacf2
> > --- /dev/null
> > +++ b/test/py/tests/test_fs/fstest_defs.py
> > @@ -0,0 +1,10 @@
> > +# SPDX-License-Identifier:      GPL-2.0+
> > +
> > +# $SMALL_FILE is the name of the 1MB file in the file system image
> > +SMALL_FILE='1MB.file'
> > +
> > +# $BIG_FILE is the name of the 2.5GB file in the file system image
> > +BIG_FILE='2.5GB.file'
> > +
> > +ADDR=0x01000008
> > +LENGTH=0x00100000
> > diff --git a/test/py/tests/test_fs/test_basic.py b/test/py/tests/test_fs/test_basic.py
> > new file mode 100644
> > index 000000000000..5258d98d42a9
> > --- /dev/null
> > +++ b/test/py/tests/test_fs/test_basic.py
> > @@ -0,0 +1,246 @@
> > +# SPDX-License-Identifier:      GPL-2.0+
> > +# Copyright (c) 2018, Linaro Limited
> > +# Author: Takahiro Akashi <takahiro.akashi@linaro.org>
> > +#
> > +# U-Boot File System:Basic Test
> > +
> > +"""
> > +This test verifies basic read/write operation on file system.
> > +"""
> > +
> > +import pytest
> > +import re
> > +from fstest_defs import *
> > +
> > + at pytest.mark.boardspec('sandbox')
> > +class TestFsBasic(object):
> > +    def test_fs1(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 1 - ls'):
> > +            # Test Case 1 - ls
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sls host 0:0' % fs_type])
> > +            assert(re.search('2621440000 *%s' % BIG_FILE, ''.join(output)))
> > +            assert(re.search('1048576 *%s' % SMALL_FILE, ''.join(output)))
> > +
> > +            # In addition, test with a nonexistent directory to see if we crash.
> > +            output2 = u_boot_console.run_command(
> > +                '%sls host 0:0 invalid_d' % fs_type)
> > +            if fs_type == 'ext4':
> > +                assert('Can not find directory' in output2)
> > +            else:
> > +                assert('' == output2)
> > +
> > +    def test_fs2(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 2 - size (small)'):
> > +            # 1MB is 0x0010 0000
> > +            # Test Case 2a - size of small file
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%ssize host 0:0 /%s' % (fs_type, SMALL_FILE),
> > +                'printenv filesize',
> > +                'setenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +            # Test Case 2b - size of small file via a path using '..'
> > +            output = u_boot_console.run_command_list([
> > +                '%ssize host 0:0 /SUBDIR/../%s' % (fs_type, SMALL_FILE),
> > +                'printenv filesize',
> > +                'setenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +    def test_fs3(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 3 - size (big)'):
> > +            # 2.5GB (1024*1024*2500) is 0x9C40 0000
> > +            # Test Case 3 - size of big file
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%ssize host 0:0 /%s' % (fs_type, BIG_FILE),
> > +                'printenv filesize',
> > +                'setenv filesize'])
> > +            assert('filesize=9c400000' in ''.join(output))
> > +
> > +    def test_fs4(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 4 - load (small)'):
> > +            # Test Case 4a - Read full 1MB of small file
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s' % (fs_type, ADDR, SMALL_FILE),
> > +                'printenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +            # Test Case 4b - Read full 1MB of small file
> > +            output = u_boot_console.run_command_list([
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[0] in ''.join(output))
> > +
> > +    def test_fs5(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 5 - load (big)'):
> > +            # Test Case 5a - First 1MB of big file
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s %x 0x0' % (fs_type, ADDR, BIG_FILE, LENGTH),
> > +                'printenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +            # Test Case 5b - First 1MB of big file
> > +            output = u_boot_console.run_command_list([
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[1] in ''.join(output))
> > +
> > +    def test_fs6(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 6 - load (big from last 1MB)'):
> > +            # fails for ext as no offset support
> > +            # Test Case 6a - Last 1MB of big file
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s %x 0x9c300000'
> > +                    % (fs_type, ADDR, BIG_FILE, LENGTH),
> > +                'printenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +            # Test Case 6b - Last 1MB of big file
> > +            output = u_boot_console.run_command_list([
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[2] in ''.join(output))
> > +
> > +    def test_fs7(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 7 - load (big from last 1MB in 2GB)'):
> > +            # fails for ext as no offset support
> > +            # Test Case 7a - One from the last 1MB chunk of 2GB
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s %x 0x7ff00000'
> > +                    % (fs_type, ADDR, BIG_FILE, LENGTH),
> > +                'printenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +            # Test Case 7b - One from the last 1MB chunk of 2GB
> > +            output = u_boot_console.run_command_list([
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[3] in ''.join(output))
> > +
> > +    def test_fs8(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 8 - load (big from first 1MB in 2GB)'):
> > +            # fails for ext as no offset support
> > +            # Test Case 8a - One from the start 1MB chunk from 2GB
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s %x 0x80000000'
> > +                    % (fs_type, ADDR, BIG_FILE, LENGTH),
> > +                'printenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +            # Test Case 8b - One from the start 1MB chunk from 2GB
> > +            output = u_boot_console.run_command_list([
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[4] in ''.join(output))
> > +
> > +    def test_fs9(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 9 - load (crossing 2GB boundary)'):
> > +            # fails for ext as no offset support
> > +            # Test Case 9a - One 1MB chunk crossing the 2GB boundary
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s %x 0x7ff80000'
> > +                    % (fs_type, ADDR, BIG_FILE, LENGTH),
> > +                'printenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +            # Test Case 9b - One 1MB chunk crossing the 2GB boundary
> > +            output = u_boot_console.run_command_list([
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[5] in ''.join(output))
> > +
> > +    def test_fs10(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 10 - load (over file end)'):
> > +            # Generic failure case
> > +            # Test Case 10 - 2MB chunk from the last 1MB of big file
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s 0x00200000 0x9c300000'
> > +                    % (fs_type, ADDR, BIG_FILE),
> > +                'printenv filesize',
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +        assert('filesize=100000' in ''.join(output))
> > +
> > +    def test_fs11(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 11 - write (small)'):
> > +            # Read 1MB from small file
> > +            # Write it back to test the writes
> > +            # Test Case 11a - Check that the write succeeded
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s' % (fs_type, ADDR, SMALL_FILE),
> > +                '%swrite host 0:0 %x /%s.w $filesize'
> > +                    % (fs_type, ADDR, SMALL_FILE)])
> > +            assert('1048576 bytes written' in ''.join(output))
> > +
> > +            # Test Case 11b - Check md5 of written to is same
> > +            # as the one read from
> > +            output = u_boot_console.run_command_list([
> > +                '%sload host 0:0 %x /%s.w' % (fs_type, ADDR, SMALL_FILE),
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[0] in ''.join(output))
> > +
> > +    def test_fs12(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 12 - write (".")'):
> > +            # Next test case checks writing a file whose dirent
> > +            # is the first in the block, which is always true for "."
> > +            # The write should fail, but the lookup should work
> > +            # Test Case 12 - Check directory traversal
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%swrite host 0:0 %x /. 0x10' % (fs_type, ADDR)])
> > +            assert('Unable to write' in ''.join(output))
> > +
> > +    def test_fs13(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 13 - write (small w/ "./")'):
> > +            # Read 1MB from small file
> > +            # Write it via "same directory", i.e. "." dirent
> > +            # Test Case 13a - Check directory traversal
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s' % (fs_type, ADDR, SMALL_FILE),
> > +                '%swrite host 0:0 %x /./%s2 $filesize'
> > +                    % (fs_type, ADDR, SMALL_FILE)])
> > +            assert('1048576 bytes written' in ''.join(output))
> > +
> > +            # Test Case 13b - Check md5 of written to is same
> > +            # as the one read from
> > +            output = u_boot_console.run_command_list([
> > +                'mw.b %x 00 100' % ADDR,
> > +                '%sload host 0:0 %x /./%s2' % (fs_type, ADDR, SMALL_FILE),
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[0] in ''.join(output))
> > +
> > +            # Test Case 13c - Check md5 of written to is same
> > +            # as the one read from
> > +            output = u_boot_console.run_command_list([
> > +                'mw.b %x 00 100' % ADDR,
> > +                '%sload host 0:0 %x /%s2' % (fs_type, ADDR, SMALL_FILE),
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[0] in ''.join(output))
> > 
> 

[U-Boot] [RFC 1/3] test/py: convert fs-test.sh to pytest

[Heinrich Schuchardt]

On 08/30/2018 08:52 AM, AKASHI Takahiro wrote:
> On Wed, Aug 29, 2018 at 11:36:51PM +0200, Heinrich Schuchardt wrote:
>> On 08/23/2018 09:25 AM, AKASHI Takahiro wrote:
>>> In this commit, the same set of test cases as in test/fs/fs-test.sh
>>> is provided using pytest framework.
>>> Actually, fs-test.sh provides three variants:"sb" (sb command), "nonfs"
>>> (fatxx and etc.) and "fs" (hostfs), and this patch currently supports
>>> only "nonfs" variant; So it is not a replacement of fs-test.sh for now.
>>>
>>> Simple usage:
>>>   $ py.test test/py/tests/test_fs [<other options>]
>>>
>>> You may also specify filesystem types to be tested:
>>>   $ py.test test/py/tests/test_fs --fs-type fat32 [<other options>]
>>>
>>> Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
>>> ---
>>>  test/py/tests/test_fs/conftest.py    | 175 +++++++++++++++++++
>>>  test/py/tests/test_fs/fstest_defs.py |  10 ++
>>>  test/py/tests/test_fs/test_basic.py  | 246 +++++++++++++++++++++++++++
>>>  3 files changed, 431 insertions(+)
>>>  create mode 100644 test/py/tests/test_fs/conftest.py
>>>  create mode 100644 test/py/tests/test_fs/fstest_defs.py
>>>  create mode 100644 test/py/tests/test_fs/test_basic.py
>>>
>>> diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
>>> new file mode 100644
>>> index 000000000000..fefeb4c9663f
>>> --- /dev/null
>>> +++ b/test/py/tests/test_fs/conftest.py
>>> @@ -0,0 +1,175 @@
>>> +# SPDX-License-Identifier:      GPL-2.0+
>>> +# Copyright (c) 2018, Linaro Limited
>>> +# Author: Takahiro Akashi <takahiro.akashi@linaro.org>
>>> +
>>> +import pytest
>>> +import re
>>> +from subprocess import call, check_call, check_output, CalledProcessError
>>> +from fstest_defs import *
>>> +
>>> +supported_fs_basic = ['fat16', 'fat32', 'ext4']
>>> +
>>> +#
>>> +# Filesystem test specific setup
>>> +#
>>> +def pytest_addoption(parser):
>>> +    parser.addoption('--fs-type', action='append', default=None,
>>> +        help='Targeting Filesystem Types')
>>> +
>>> +def pytest_configure(config):
>>> +    global supported_fs_basic
>>> +
>>> +    def intersect(listA, listB):
>>> +        return  [x for x in listA if x in listB]
>>> +
>>> +    supported_fs = config.getoption('fs_type')
>>> +    if supported_fs:
>>> +        print("*** FS TYPE modified: %s" % supported_fs)
>>> +        supported_fs_basic =  intersect(supported_fs, supported_fs_basic)
>>> +
>>> +def pytest_generate_tests(metafunc):
>>> +    if 'fs_obj_basic' in metafunc.fixturenames:
>>> +        metafunc.parametrize('fs_obj_basic', supported_fs_basic,
>>> +            indirect=True, scope='module')
>>> +
>>> +#
>>> +# Helper functions
>>> +#
>>> +def fstype_to_ubname(fs_type):
>>> +    if re.match('fat', fs_type):
>>> +        return 'fat'
>>> +    else:
>>> +        return fs_type
>>> +
>>> +def check_ubconfig(config, fs_type):
>>> +    if not config.buildconfig.get('config_cmd_%s' % fs_type, None):
>>> +        pytest.skip('.config feature "CMD_%s" not enabled' % fs_type.upper())
>>> +    if not config.buildconfig.get('config_%s_write' % fs_type, None):
>>> +        pytest.skip('.config feature "%s_WRITE" not enabled'
>>> +        % fs_type.upper())
>>> +
>>> +def mk_fs(config, fs_type, size, id):
>>> +    fs_img = '%s.%s.img' % (id, fs_type)
>>> +    fs_img = config.persistent_data_dir + '/' + fs_img
>>> +
>>> +    if fs_type == 'fat16':
>>> +        mkfs_opt = '-F 16'
>>> +    elif fs_type == 'fat32':
>>> +        mkfs_opt = '-F 32'
>>> +    else:
>>> +        mkfs_opt = ''
>>> +
>>> +    if re.match('fat', fs_type):
>>> +        fs_lnxtype = 'vfat'
>>> +    else:
>>> +        fs_lnxtype = fs_type
>>> +
>>> +    count = (size + 1023) / 1024
>>> +
>>> +    try:
>>> +        check_call('rm -f %s' % fs_img, shell=True)
>>> +        check_call('dd if=/dev/zero of=%s bs=1K count=%d'
>>> +            % (fs_img, count), shell=True)
>>> +        check_call('mkfs.%s %s %s'
>>> +            % (fs_lnxtype, mkfs_opt, fs_img), shell=True)
>>> +        return fs_img
>>> +    except CalledProcessError:
>>> +        call('rm -f %s' % fs_img, shell=True)
>>> +        raise
>>> +
>>> +#
>>> +# Fixture for basic fs test
>>> +#     derived from test/fs/fs-test.sh
>>> +#
>>> +# NOTE: yield_fixture was deprecated since pytest-3.0
>>> + at pytest.yield_fixture()
>>> +def fs_obj_basic(request, u_boot_config):
>>> +    fs_type = request.param
>>> +    fs_img = ''
>>> +
>>> +    fs_ubtype = fstype_to_ubname(fs_type)
>>> +    check_ubconfig(u_boot_config, fs_ubtype)
>>> +
>>> +    mount_dir = u_boot_config.persistent_data_dir + '/mnt'
>>> +    small_file = mount_dir + '/' + SMALL_FILE
>>> +    big_file = mount_dir + '/' + BIG_FILE
>>> +    try:
>>> +
>>> +        # 3GiB volume
>>> +        fs_img = mk_fs(u_boot_config, fs_type, 0xc0000000, '3GB')
>>> +
>>> +        # Mount the image so we can populate it.
>>> +        check_call('mkdir -p %s' % mount_dir, shell=True)
>>> +        check_call('sudo mount -o loop,rw %s %s'
>>> +            % (fs_img, mount_dir), shell=True)
>>
>> Should I grant sudo to anybody who can commit to U-Boot?
> 
> I don't get your point.
> I think using "sudo" solely in testing should be allowed.
> 
>> Just use exfat-fuse and fuse2fs.
> 
> It will not be a good idea to use those tools which are not
> provided in every distribution.
> I'd like to leave "sudo mount" statement as a backstop.

Fuse is a base functionality of Linux. On Travis we are using the Ubuntu
distribution which contains said fuse file systems. Which distribuition
does not have it?

Using sudo for me is a NO-NO. I will not run any test that uses sudo.

Best regards

Heinrich

[U-Boot] [RFC 1/3] test/py: convert fs-test.sh to pytest

[AKASHI Takahiro]

On Thu, Aug 30, 2018 at 12:01:32PM +0200, Heinrich Schuchardt wrote:
> On 08/30/2018 08:52 AM, AKASHI Takahiro wrote:
> > On Wed, Aug 29, 2018 at 11:36:51PM +0200, Heinrich Schuchardt wrote:
> >> On 08/23/2018 09:25 AM, AKASHI Takahiro wrote:
> >>> In this commit, the same set of test cases as in test/fs/fs-test.sh
> >>> is provided using pytest framework.
> >>> Actually, fs-test.sh provides three variants:"sb" (sb command), "nonfs"
> >>> (fatxx and etc.) and "fs" (hostfs), and this patch currently supports
> >>> only "nonfs" variant; So it is not a replacement of fs-test.sh for now.
> >>>
> >>> Simple usage:
> >>>   $ py.test test/py/tests/test_fs [<other options>]
> >>>
> >>> You may also specify filesystem types to be tested:
> >>>   $ py.test test/py/tests/test_fs --fs-type fat32 [<other options>]
> >>>
> >>> Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
> >>> ---
> >>>  test/py/tests/test_fs/conftest.py    | 175 +++++++++++++++++++
> >>>  test/py/tests/test_fs/fstest_defs.py |  10 ++
> >>>  test/py/tests/test_fs/test_basic.py  | 246 +++++++++++++++++++++++++++
> >>>  3 files changed, 431 insertions(+)
> >>>  create mode 100644 test/py/tests/test_fs/conftest.py
> >>>  create mode 100644 test/py/tests/test_fs/fstest_defs.py
> >>>  create mode 100644 test/py/tests/test_fs/test_basic.py
> >>>
> >>> diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
> >>> new file mode 100644
> >>> index 000000000000..fefeb4c9663f
> >>> --- /dev/null
> >>> +++ b/test/py/tests/test_fs/conftest.py
> >>> @@ -0,0 +1,175 @@
> >>> +# SPDX-License-Identifier:      GPL-2.0+
> >>> +# Copyright (c) 2018, Linaro Limited
> >>> +# Author: Takahiro Akashi <takahiro.akashi@linaro.org>
> >>> +
> >>> +import pytest
> >>> +import re
> >>> +from subprocess import call, check_call, check_output, CalledProcessError
> >>> +from fstest_defs import *
> >>> +
> >>> +supported_fs_basic = ['fat16', 'fat32', 'ext4']
> >>> +
> >>> +#
> >>> +# Filesystem test specific setup
> >>> +#
> >>> +def pytest_addoption(parser):
> >>> +    parser.addoption('--fs-type', action='append', default=None,
> >>> +        help='Targeting Filesystem Types')
> >>> +
> >>> +def pytest_configure(config):
> >>> +    global supported_fs_basic
> >>> +
> >>> +    def intersect(listA, listB):
> >>> +        return  [x for x in listA if x in listB]
> >>> +
> >>> +    supported_fs = config.getoption('fs_type')
> >>> +    if supported_fs:
> >>> +        print("*** FS TYPE modified: %s" % supported_fs)
> >>> +        supported_fs_basic =  intersect(supported_fs, supported_fs_basic)
> >>> +
> >>> +def pytest_generate_tests(metafunc):
> >>> +    if 'fs_obj_basic' in metafunc.fixturenames:
> >>> +        metafunc.parametrize('fs_obj_basic', supported_fs_basic,
> >>> +            indirect=True, scope='module')
> >>> +
> >>> +#
> >>> +# Helper functions
> >>> +#
> >>> +def fstype_to_ubname(fs_type):
> >>> +    if re.match('fat', fs_type):
> >>> +        return 'fat'
> >>> +    else:
> >>> +        return fs_type
> >>> +
> >>> +def check_ubconfig(config, fs_type):
> >>> +    if not config.buildconfig.get('config_cmd_%s' % fs_type, None):
> >>> +        pytest.skip('.config feature "CMD_%s" not enabled' % fs_type.upper())
> >>> +    if not config.buildconfig.get('config_%s_write' % fs_type, None):
> >>> +        pytest.skip('.config feature "%s_WRITE" not enabled'
> >>> +        % fs_type.upper())
> >>> +
> >>> +def mk_fs(config, fs_type, size, id):
> >>> +    fs_img = '%s.%s.img' % (id, fs_type)
> >>> +    fs_img = config.persistent_data_dir + '/' + fs_img
> >>> +
> >>> +    if fs_type == 'fat16':
> >>> +        mkfs_opt = '-F 16'
> >>> +    elif fs_type == 'fat32':
> >>> +        mkfs_opt = '-F 32'
> >>> +    else:
> >>> +        mkfs_opt = ''
> >>> +
> >>> +    if re.match('fat', fs_type):
> >>> +        fs_lnxtype = 'vfat'
> >>> +    else:
> >>> +        fs_lnxtype = fs_type
> >>> +
> >>> +    count = (size + 1023) / 1024
> >>> +
> >>> +    try:
> >>> +        check_call('rm -f %s' % fs_img, shell=True)
> >>> +        check_call('dd if=/dev/zero of=%s bs=1K count=%d'
> >>> +            % (fs_img, count), shell=True)
> >>> +        check_call('mkfs.%s %s %s'
> >>> +            % (fs_lnxtype, mkfs_opt, fs_img), shell=True)
> >>> +        return fs_img
> >>> +    except CalledProcessError:
> >>> +        call('rm -f %s' % fs_img, shell=True)
> >>> +        raise
> >>> +
> >>> +#
> >>> +# Fixture for basic fs test
> >>> +#     derived from test/fs/fs-test.sh
> >>> +#
> >>> +# NOTE: yield_fixture was deprecated since pytest-3.0
> >>> + at pytest.yield_fixture()
> >>> +def fs_obj_basic(request, u_boot_config):
> >>> +    fs_type = request.param
> >>> +    fs_img = ''
> >>> +
> >>> +    fs_ubtype = fstype_to_ubname(fs_type)
> >>> +    check_ubconfig(u_boot_config, fs_ubtype)
> >>> +
> >>> +    mount_dir = u_boot_config.persistent_data_dir + '/mnt'
> >>> +    small_file = mount_dir + '/' + SMALL_FILE
> >>> +    big_file = mount_dir + '/' + BIG_FILE
> >>> +    try:
> >>> +
> >>> +        # 3GiB volume
> >>> +        fs_img = mk_fs(u_boot_config, fs_type, 0xc0000000, '3GB')
> >>> +
> >>> +        # Mount the image so we can populate it.
> >>> +        check_call('mkdir -p %s' % mount_dir, shell=True)
> >>> +        check_call('sudo mount -o loop,rw %s %s'
> >>> +            % (fs_img, mount_dir), shell=True)
> >>
> >> Should I grant sudo to anybody who can commit to U-Boot?
> > 
> > I don't get your point.
> > I think using "sudo" solely in testing should be allowed.
> > 
> >> Just use exfat-fuse and fuse2fs.
> > 
> > It will not be a good idea to use those tools which are not
> > provided in every distribution.
> > I'd like to leave "sudo mount" statement as a backstop.
> 
> Fuse is a base functionality of Linux. On Travis we are using the Ubuntu
> distribution which contains said fuse file systems. Which distribuition
> does not have it?
> 
> Using sudo for me is a NO-NO. I will not run any test that uses sudo.

So this means that you have never tested file system using test-fs.sh.

Since my script is logically "general", it can, if we want, run against
other file systems as well. "sudo mount" is the only solution for those cases.

-Takahiro AKASHI


> Best regards
> 
> Heinrich

[U-Boot] [RFC 1/3] test/py: convert fs-test.sh to pytest

[Tuomas Tynkkynen]

Hi Heinrich, Takahiro

On 08/30/2018 01:26 PM, AKASHI Takahiro wrote:
> On Thu, Aug 30, 2018 at 12:01:32PM +0200, Heinrich Schuchardt wrote:
>> On 08/30/2018 08:52 AM, AKASHI Takahiro wrote:
>>> On Wed, Aug 29, 2018 at 11:36:51PM +0200, Heinrich Schuchardt wrote:
>>>> On 08/23/2018 09:25 AM, AKASHI Takahiro wrote:
>>>>> In this commit, the same set of test cases as in test/fs/fs-test.sh
>>>>> is provided using pytest framework.
>>>>> Actually, fs-test.sh provides three variants:"sb" (sb command), "nonfs"
>>>>> (fatxx and etc.) and "fs" (hostfs), and this patch currently supports
>>>>> only "nonfs" variant; So it is not a replacement of fs-test.sh for now.
>>>>>
>>>>> Simple usage:
>>>>>    $ py.test test/py/tests/test_fs [<other options>]
>>>>>
>>>>> You may also specify filesystem types to be tested:
>>>>>    $ py.test test/py/tests/test_fs --fs-type fat32 [<other options>]
>>>>>
>>>>> Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
>>>>> ---
>>>>>   test/py/tests/test_fs/conftest.py    | 175 +++++++++++++++++++
>>>>>   test/py/tests/test_fs/fstest_defs.py |  10 ++
>>>>>   test/py/tests/test_fs/test_basic.py  | 246 +++++++++++++++++++++++++++
>>>>>   3 files changed, 431 insertions(+)
>>>>>   create mode 100644 test/py/tests/test_fs/conftest.py
>>>>>   create mode 100644 test/py/tests/test_fs/fstest_defs.py
>>>>>   create mode 100644 test/py/tests/test_fs/test_basic.py
>>>>>
>>>>> diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
>>>>> new file mode 100644
>>>>> index 000000000000..fefeb4c9663f
>>>>> --- /dev/null
>>>>> +++ b/test/py/tests/test_fs/conftest.py
>>>>> @@ -0,0 +1,175 @@
>>>>> +# SPDX-License-Identifier:      GPL-2.0+
>>>>> +# Copyright (c) 2018, Linaro Limited
>>>>> +# Author: Takahiro Akashi <takahiro.akashi@linaro.org>
>>>>> +
>>>>> +import pytest
>>>>> +import re
>>>>> +from subprocess import call, check_call, check_output, CalledProcessError
>>>>> +from fstest_defs import *
>>>>> +
>>>>> +supported_fs_basic = ['fat16', 'fat32', 'ext4']
>>>>> +
>>>>> +#
>>>>> +# Filesystem test specific setup
>>>>> +#
>>>>> +def pytest_addoption(parser):
>>>>> +    parser.addoption('--fs-type', action='append', default=None,
>>>>> +        help='Targeting Filesystem Types')
>>>>> +
>>>>> +def pytest_configure(config):
>>>>> +    global supported_fs_basic
>>>>> +
>>>>> +    def intersect(listA, listB):
>>>>> +        return  [x for x in listA if x in listB]
>>>>> +
>>>>> +    supported_fs = config.getoption('fs_type')
>>>>> +    if supported_fs:
>>>>> +        print("*** FS TYPE modified: %s" % supported_fs)
>>>>> +        supported_fs_basic =  intersect(supported_fs, supported_fs_basic)
>>>>> +
>>>>> +def pytest_generate_tests(metafunc):
>>>>> +    if 'fs_obj_basic' in metafunc.fixturenames:
>>>>> +        metafunc.parametrize('fs_obj_basic', supported_fs_basic,
>>>>> +            indirect=True, scope='module')
>>>>> +
>>>>> +#
>>>>> +# Helper functions
>>>>> +#
>>>>> +def fstype_to_ubname(fs_type):
>>>>> +    if re.match('fat', fs_type):
>>>>> +        return 'fat'
>>>>> +    else:
>>>>> +        return fs_type
>>>>> +
>>>>> +def check_ubconfig(config, fs_type):
>>>>> +    if not config.buildconfig.get('config_cmd_%s' % fs_type, None):
>>>>> +        pytest.skip('.config feature "CMD_%s" not enabled' % fs_type.upper())
>>>>> +    if not config.buildconfig.get('config_%s_write' % fs_type, None):
>>>>> +        pytest.skip('.config feature "%s_WRITE" not enabled'
>>>>> +        % fs_type.upper())
>>>>> +
>>>>> +def mk_fs(config, fs_type, size, id):
>>>>> +    fs_img = '%s.%s.img' % (id, fs_type)
>>>>> +    fs_img = config.persistent_data_dir + '/' + fs_img
>>>>> +
>>>>> +    if fs_type == 'fat16':
>>>>> +        mkfs_opt = '-F 16'
>>>>> +    elif fs_type == 'fat32':
>>>>> +        mkfs_opt = '-F 32'
>>>>> +    else:
>>>>> +        mkfs_opt = ''
>>>>> +
>>>>> +    if re.match('fat', fs_type):
>>>>> +        fs_lnxtype = 'vfat'
>>>>> +    else:
>>>>> +        fs_lnxtype = fs_type
>>>>> +
>>>>> +    count = (size + 1023) / 1024
>>>>> +
>>>>> +    try:
>>>>> +        check_call('rm -f %s' % fs_img, shell=True)
>>>>> +        check_call('dd if=/dev/zero of=%s bs=1K count=%d'
>>>>> +            % (fs_img, count), shell=True)
>>>>> +        check_call('mkfs.%s %s %s'
>>>>> +            % (fs_lnxtype, mkfs_opt, fs_img), shell=True)
>>>>> +        return fs_img
>>>>> +    except CalledProcessError:
>>>>> +        call('rm -f %s' % fs_img, shell=True)
>>>>> +        raise
>>>>> +
>>>>> +#
>>>>> +# Fixture for basic fs test
>>>>> +#     derived from test/fs/fs-test.sh
>>>>> +#
>>>>> +# NOTE: yield_fixture was deprecated since pytest-3.0
>>>>> + at pytest.yield_fixture()
>>>>> +def fs_obj_basic(request, u_boot_config):
>>>>> +    fs_type = request.param
>>>>> +    fs_img = ''
>>>>> +
>>>>> +    fs_ubtype = fstype_to_ubname(fs_type)
>>>>> +    check_ubconfig(u_boot_config, fs_ubtype)
>>>>> +
>>>>> +    mount_dir = u_boot_config.persistent_data_dir + '/mnt'
>>>>> +    small_file = mount_dir + '/' + SMALL_FILE
>>>>> +    big_file = mount_dir + '/' + BIG_FILE
>>>>> +    try:
>>>>> +
>>>>> +        # 3GiB volume
>>>>> +        fs_img = mk_fs(u_boot_config, fs_type, 0xc0000000, '3GB')
>>>>> +
>>>>> +        # Mount the image so we can populate it.
>>>>> +        check_call('mkdir -p %s' % mount_dir, shell=True)
>>>>> +        check_call('sudo mount -o loop,rw %s %s'
>>>>> +            % (fs_img, mount_dir), shell=True)
>>>>
>>>> Should I grant sudo to anybody who can commit to U-Boot?
>>>
>>> I don't get your point.
>>> I think using "sudo" solely in testing should be allowed.
>>>
>>>> Just use exfat-fuse and fuse2fs.
>>>
>>> It will not be a good idea to use those tools which are not
>>> provided in every distribution.
>>> I'd like to leave "sudo mount" statement as a backstop.
>>
>> Fuse is a base functionality of Linux. On Travis we are using the Ubuntu
>> distribution which contains said fuse file systems. Which distribuition
>> does not have it?
>>
>> Using sudo for me is a NO-NO. I will not run any test that uses sudo.
> 
> So this means that you have never tested file system using test-fs.sh.
> 
> Since my script is logically "general", it can, if we want, run against
> other file systems as well. "sudo mount" is the only solution for those cases.
> 
There are two general non-root implementations that I know of:

1) http://libguestfs.org/ which IIRC launches a small Linux VM in QEMU
    to do the filesystem accesses. I am not sure if the performance would be
    acceptable without KVM (which I assume we don't have in Travis).

2) https://github.com/lkl/linux which is a port of Linux to run in userspace
    as a library. It comes with tools like cptofs and lklfuse to access any
    filesystem Linux has a driver for. Sadly lkl isn't packaged in many distros.

- Tuomas

[U-Boot] [RFC 1/3] test/py: convert fs-test.sh to pytest

[AKASHI Takahiro]

Hi Tuomas,

Thank you for interesting pointers.

On Thu, Aug 30, 2018 at 01:56:41PM +0300, Tuomas Tynkkynen wrote:
> Hi Heinrich, Takahiro
> 
> On 08/30/2018 01:26 PM, AKASHI Takahiro wrote:
> >On Thu, Aug 30, 2018 at 12:01:32PM +0200, Heinrich Schuchardt wrote:
> >>On 08/30/2018 08:52 AM, AKASHI Takahiro wrote:
> >>>On Wed, Aug 29, 2018 at 11:36:51PM +0200, Heinrich Schuchardt wrote:
> >>>>On 08/23/2018 09:25 AM, AKASHI Takahiro wrote:
> >>>>>In this commit, the same set of test cases as in test/fs/fs-test.sh
> >>>>>is provided using pytest framework.
> >>>>>Actually, fs-test.sh provides three variants:"sb" (sb command), "nonfs"
> >>>>>(fatxx and etc.) and "fs" (hostfs), and this patch currently supports
> >>>>>only "nonfs" variant; So it is not a replacement of fs-test.sh for now.
> >>>>>
> >>>>>Simple usage:
> >>>>>   $ py.test test/py/tests/test_fs [<other options>]
> >>>>>
> >>>>>You may also specify filesystem types to be tested:
> >>>>>   $ py.test test/py/tests/test_fs --fs-type fat32 [<other options>]
> >>>>>
> >>>>>Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
> >>>>>---
> >>>>>  test/py/tests/test_fs/conftest.py    | 175 +++++++++++++++++++
> >>>>>  test/py/tests/test_fs/fstest_defs.py |  10 ++
> >>>>>  test/py/tests/test_fs/test_basic.py  | 246 +++++++++++++++++++++++++++
> >>>>>  3 files changed, 431 insertions(+)
> >>>>>  create mode 100644 test/py/tests/test_fs/conftest.py
> >>>>>  create mode 100644 test/py/tests/test_fs/fstest_defs.py
> >>>>>  create mode 100644 test/py/tests/test_fs/test_basic.py
> >>>>>
> >>>>>diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
> >>>>>new file mode 100644
> >>>>>index 000000000000..fefeb4c9663f
> >>>>>--- /dev/null
> >>>>>+++ b/test/py/tests/test_fs/conftest.py
> >>>>>@@ -0,0 +1,175 @@
> >>>>>+# SPDX-License-Identifier:      GPL-2.0+
> >>>>>+# Copyright (c) 2018, Linaro Limited
> >>>>>+# Author: Takahiro Akashi <takahiro.akashi@linaro.org>
> >>>>>+
> >>>>>+import pytest
> >>>>>+import re
> >>>>>+from subprocess import call, check_call, check_output, CalledProcessError
> >>>>>+from fstest_defs import *
> >>>>>+
> >>>>>+supported_fs_basic = ['fat16', 'fat32', 'ext4']
> >>>>>+
> >>>>>+#
> >>>>>+# Filesystem test specific setup
> >>>>>+#
> >>>>>+def pytest_addoption(parser):
> >>>>>+    parser.addoption('--fs-type', action='append', default=None,
> >>>>>+        help='Targeting Filesystem Types')
> >>>>>+
> >>>>>+def pytest_configure(config):
> >>>>>+    global supported_fs_basic
> >>>>>+
> >>>>>+    def intersect(listA, listB):
> >>>>>+        return  [x for x in listA if x in listB]
> >>>>>+
> >>>>>+    supported_fs = config.getoption('fs_type')
> >>>>>+    if supported_fs:
> >>>>>+        print("*** FS TYPE modified: %s" % supported_fs)
> >>>>>+        supported_fs_basic =  intersect(supported_fs, supported_fs_basic)
> >>>>>+
> >>>>>+def pytest_generate_tests(metafunc):
> >>>>>+    if 'fs_obj_basic' in metafunc.fixturenames:
> >>>>>+        metafunc.parametrize('fs_obj_basic', supported_fs_basic,
> >>>>>+            indirect=True, scope='module')
> >>>>>+
> >>>>>+#
> >>>>>+# Helper functions
> >>>>>+#
> >>>>>+def fstype_to_ubname(fs_type):
> >>>>>+    if re.match('fat', fs_type):
> >>>>>+        return 'fat'
> >>>>>+    else:
> >>>>>+        return fs_type
> >>>>>+
> >>>>>+def check_ubconfig(config, fs_type):
> >>>>>+    if not config.buildconfig.get('config_cmd_%s' % fs_type, None):
> >>>>>+        pytest.skip('.config feature "CMD_%s" not enabled' % fs_type.upper())
> >>>>>+    if not config.buildconfig.get('config_%s_write' % fs_type, None):
> >>>>>+        pytest.skip('.config feature "%s_WRITE" not enabled'
> >>>>>+        % fs_type.upper())
> >>>>>+
> >>>>>+def mk_fs(config, fs_type, size, id):
> >>>>>+    fs_img = '%s.%s.img' % (id, fs_type)
> >>>>>+    fs_img = config.persistent_data_dir + '/' + fs_img
> >>>>>+
> >>>>>+    if fs_type == 'fat16':
> >>>>>+        mkfs_opt = '-F 16'
> >>>>>+    elif fs_type == 'fat32':
> >>>>>+        mkfs_opt = '-F 32'
> >>>>>+    else:
> >>>>>+        mkfs_opt = ''
> >>>>>+
> >>>>>+    if re.match('fat', fs_type):
> >>>>>+        fs_lnxtype = 'vfat'
> >>>>>+    else:
> >>>>>+        fs_lnxtype = fs_type
> >>>>>+
> >>>>>+    count = (size + 1023) / 1024
> >>>>>+
> >>>>>+    try:
> >>>>>+        check_call('rm -f %s' % fs_img, shell=True)
> >>>>>+        check_call('dd if=/dev/zero of=%s bs=1K count=%d'
> >>>>>+            % (fs_img, count), shell=True)
> >>>>>+        check_call('mkfs.%s %s %s'
> >>>>>+            % (fs_lnxtype, mkfs_opt, fs_img), shell=True)
> >>>>>+        return fs_img
> >>>>>+    except CalledProcessError:
> >>>>>+        call('rm -f %s' % fs_img, shell=True)
> >>>>>+        raise
> >>>>>+
> >>>>>+#
> >>>>>+# Fixture for basic fs test
> >>>>>+#     derived from test/fs/fs-test.sh
> >>>>>+#
> >>>>>+# NOTE: yield_fixture was deprecated since pytest-3.0
> >>>>>+ at pytest.yield_fixture()
> >>>>>+def fs_obj_basic(request, u_boot_config):
> >>>>>+    fs_type = request.param
> >>>>>+    fs_img = ''
> >>>>>+
> >>>>>+    fs_ubtype = fstype_to_ubname(fs_type)
> >>>>>+    check_ubconfig(u_boot_config, fs_ubtype)
> >>>>>+
> >>>>>+    mount_dir = u_boot_config.persistent_data_dir + '/mnt'
> >>>>>+    small_file = mount_dir + '/' + SMALL_FILE
> >>>>>+    big_file = mount_dir + '/' + BIG_FILE
> >>>>>+    try:
> >>>>>+
> >>>>>+        # 3GiB volume
> >>>>>+        fs_img = mk_fs(u_boot_config, fs_type, 0xc0000000, '3GB')
> >>>>>+
> >>>>>+        # Mount the image so we can populate it.
> >>>>>+        check_call('mkdir -p %s' % mount_dir, shell=True)
> >>>>>+        check_call('sudo mount -o loop,rw %s %s'
> >>>>>+            % (fs_img, mount_dir), shell=True)
> >>>>
> >>>>Should I grant sudo to anybody who can commit to U-Boot?
> >>>
> >>>I don't get your point.
> >>>I think using "sudo" solely in testing should be allowed.
> >>>
> >>>>Just use exfat-fuse and fuse2fs.
> >>>
> >>>It will not be a good idea to use those tools which are not
> >>>provided in every distribution.
> >>>I'd like to leave "sudo mount" statement as a backstop.
> >>
> >>Fuse is a base functionality of Linux. On Travis we are using the Ubuntu
> >>distribution which contains said fuse file systems. Which distribuition
> >>does not have it?
> >>
> >>Using sudo for me is a NO-NO. I will not run any test that uses sudo.
> >
> >So this means that you have never tested file system using test-fs.sh.
> >
> >Since my script is logically "general", it can, if we want, run against
> >other file systems as well. "sudo mount" is the only solution for those cases.
> >
> There are two general non-root implementations that I know of:
> 
> 1) http://libguestfs.org/ which IIRC launches a small Linux VM in QEMU
>    to do the filesystem accesses. I am not sure if the performance would be
>    acceptable without KVM (which I assume we don't have in Travis).

I didn't dig into this tool, but if it is all about VM on qemu,
the discussion here, whether we should be allowed to use sudo or not,
would be pointless as we can do whatever we want to do under VM.

> 2) https://github.com/lkl/linux which is a port of Linux to run in userspace
>    as a library. It comes with tools like cptofs and lklfuse to access any
>    filesystem Linux has a driver for.

It appears to be a kind of libos or unikernel, or rather,
resembles a sandbox of u-boot?

> Sadly lkl isn't packaged in many distros.

Too bad.

-Takahiro AKASHI

> - Tuomas

[U-Boot] [RFC 1/3] test/py: convert fs-test.sh to pytest

[AKASHI Takahiro]

On Wed, Aug 29, 2018 at 11:36:51PM +0200, Heinrich Schuchardt wrote:
> On 08/23/2018 09:25 AM, AKASHI Takahiro wrote:
> > In this commit, the same set of test cases as in test/fs/fs-test.sh
> > is provided using pytest framework.
> > Actually, fs-test.sh provides three variants:"sb" (sb command), "nonfs"
> > (fatxx and etc.) and "fs" (hostfs), and this patch currently supports
> > only "nonfs" variant; So it is not a replacement of fs-test.sh for now.
> > 
> > Simple usage:
> >   $ py.test test/py/tests/test_fs [<other options>]
> > 
> > You may also specify filesystem types to be tested:
> >   $ py.test test/py/tests/test_fs --fs-type fat32 [<other options>]
> > 
> > Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
> > ---
> >  test/py/tests/test_fs/conftest.py    | 175 +++++++++++++++++++
> >  test/py/tests/test_fs/fstest_defs.py |  10 ++
> >  test/py/tests/test_fs/test_basic.py  | 246 +++++++++++++++++++++++++++
> >  3 files changed, 431 insertions(+)
> >  create mode 100644 test/py/tests/test_fs/conftest.py
> >  create mode 100644 test/py/tests/test_fs/fstest_defs.py
> >  create mode 100644 test/py/tests/test_fs/test_basic.py
> > 
> > diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
> > new file mode 100644
> > index 000000000000..fefeb4c9663f
> > --- /dev/null
> > +++ b/test/py/tests/test_fs/conftest.py
> > @@ -0,0 +1,175 @@
> > +# SPDX-License-Identifier:      GPL-2.0+
> > +# Copyright (c) 2018, Linaro Limited
> > +# Author: Takahiro Akashi <takahiro.akashi@linaro.org>
> > +
> > +import pytest
> > +import re
> > +from subprocess import call, check_call, check_output, CalledProcessError
> > +from fstest_defs import *
> > +
> > +supported_fs_basic = ['fat16', 'fat32', 'ext4']
> > +
> > +#
> > +# Filesystem test specific setup
> > +#
> > +def pytest_addoption(parser):
> > +    parser.addoption('--fs-type', action='append', default=None,
> > +        help='Targeting Filesystem Types')
> > +
> > +def pytest_configure(config):
> > +    global supported_fs_basic
> > +
> > +    def intersect(listA, listB):
> > +        return  [x for x in listA if x in listB]
> > +
> > +    supported_fs = config.getoption('fs_type')
> > +    if supported_fs:
> > +        print("*** FS TYPE modified: %s" % supported_fs)
> > +        supported_fs_basic =  intersect(supported_fs, supported_fs_basic)
> > +
> > +def pytest_generate_tests(metafunc):
> > +    if 'fs_obj_basic' in metafunc.fixturenames:
> > +        metafunc.parametrize('fs_obj_basic', supported_fs_basic,
> > +            indirect=True, scope='module')
> > +
> > +#
> > +# Helper functions
> > +#
> > +def fstype_to_ubname(fs_type):
> > +    if re.match('fat', fs_type):
> > +        return 'fat'
> > +    else:
> > +        return fs_type
> > +
> > +def check_ubconfig(config, fs_type):
> > +    if not config.buildconfig.get('config_cmd_%s' % fs_type, None):
> > +        pytest.skip('.config feature "CMD_%s" not enabled' % fs_type.upper())
> > +    if not config.buildconfig.get('config_%s_write' % fs_type, None):
> > +        pytest.skip('.config feature "%s_WRITE" not enabled'
> > +        % fs_type.upper())
> > +
> > +def mk_fs(config, fs_type, size, id):
> > +    fs_img = '%s.%s.img' % (id, fs_type)
> > +    fs_img = config.persistent_data_dir + '/' + fs_img
> > +
> > +    if fs_type == 'fat16':
> > +        mkfs_opt = '-F 16'
> > +    elif fs_type == 'fat32':
> > +        mkfs_opt = '-F 32'
> > +    else:
> > +        mkfs_opt = ''
> > +
> > +    if re.match('fat', fs_type):
> > +        fs_lnxtype = 'vfat'
> > +    else:
> > +        fs_lnxtype = fs_type
> > +
> > +    count = (size + 1023) / 1024
> > +
> > +    try:
> > +        check_call('rm -f %s' % fs_img, shell=True)
> > +        check_call('dd if=/dev/zero of=%s bs=1K count=%d'
> > +            % (fs_img, count), shell=True)
> > +        check_call('mkfs.%s %s %s'
> > +            % (fs_lnxtype, mkfs_opt, fs_img), shell=True)
> > +        return fs_img
> > +    except CalledProcessError:
> > +        call('rm -f %s' % fs_img, shell=True)
> > +        raise
> > +
> > +#
> > +# Fixture for basic fs test
> > +#     derived from test/fs/fs-test.sh
> > +#
> > +# NOTE: yield_fixture was deprecated since pytest-3.0
> > + at pytest.yield_fixture()
> > +def fs_obj_basic(request, u_boot_config):
> > +    fs_type = request.param
> > +    fs_img = ''
> > +
> > +    fs_ubtype = fstype_to_ubname(fs_type)
> > +    check_ubconfig(u_boot_config, fs_ubtype)
> > +
> > +    mount_dir = u_boot_config.persistent_data_dir + '/mnt'
> > +    small_file = mount_dir + '/' + SMALL_FILE
> > +    big_file = mount_dir + '/' + BIG_FILE
> > +    try:
> > +
> > +        # 3GiB volume
> > +        fs_img = mk_fs(u_boot_config, fs_type, 0xc0000000, '3GB')
> > +
> > +        # Mount the image so we can populate it.
> > +        check_call('mkdir -p %s' % mount_dir, shell=True)
> > +        check_call('sudo mount -o loop,rw %s %s'
> > +            % (fs_img, mount_dir), shell=True)
> 
> Should I grant sudo to anybody who can commit to U-Boot?
> 
> Just use exfat-fuse and fuse2fs.

As far fas I tried, exfat-fuse will not be able to mount
a fat (vfat) file system, unlike fuse2fs handling ext2 as well as ext4.
So this cannot be a solution.
If you know how to mount fat fs with exfat-fuse as a non-root user,
please let me know.

-Takahiro AKASHI


> > +
> > +        # Create a subdirectory.
> > +        check_call('sudo mkdir %s/SUBDIR' % mount_dir, shell=True)
> > +
> > +        # Create big file in this image.
> > +        # Note that we work only on the start 1MB, couple MBs in the 2GB range
> > +        # and the last 1 MB of the huge 2.5GB file.
> > +        # So, just put random values only in those areas.
> > +        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=1'
> > +	    % big_file, shell=True)
> 
> 
> No need to use sudo to create a file.
> 
> Best regards
> 
> Heinrich
> 
> > +        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=2 seek=2047'
> > +            % big_file, shell=True)
> > +        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=1 seek=2499'
> > +            % big_file, shell=True)
> > +
> > +        # Create a small file in this image.
> > +        check_call('sudo dd if=/dev/urandom of=%s bs=1M count=1'
> > +	    % small_file, shell=True)
> > +
> > +        # Delete the small file copies which possibly are written as part of a
> > +        # previous test.
> > +        # check_call('sudo rm -f "%s.w"' % MB1, shell=True)
> > +        # check_call('sudo rm -f "%s.w2"' % MB1, shell=True)
> > +
> > +        # Generate the md5sums of reads that we will test against small file
> > +        out = check_output(
> > +            'dd if=%s bs=1M skip=0 count=1 2> /dev/null | md5sum'
> > +	    % small_file, shell=True)
> > +        md5val = [ out.split()[0] ]
> > +
> > +        # Generate the md5sums of reads that we will test against big file
> > +        # One from beginning of file.
> > +        out = check_output(
> > +            'dd if=%s bs=1M skip=0 count=1 2> /dev/null | md5sum'
> > +	    % big_file, shell=True)
> > +        md5val.append(out.split()[0])
> > +
> > +        # One from end of file.
> > +        out = check_output(
> > +            'dd if=%s bs=1M skip=2499 count=1 2> /dev/null | md5sum'
> > +	    % big_file, shell=True)
> > +        md5val.append(out.split()[0])
> > +
> > +        # One from the last 1MB chunk of 2GB
> > +        out = check_output(
> > +            'dd if=%s bs=1M skip=2047 count=1 2> /dev/null | md5sum'
> > +	    % big_file, shell=True)
> > +        md5val.append(out.split()[0])
> > +
> > +        # One from the start 1MB chunk from 2GB
> > +        out = check_output(
> > +            'dd if=%s bs=1M skip=2048 count=1 2> /dev/null | md5sum'
> > +	    % big_file, shell=True)
> > +        md5val.append(out.split()[0])
> > +
> > +        # One 1MB chunk crossing the 2GB boundary
> > +        out = check_output(
> > +            'dd if=%s bs=512K skip=4095 count=2 2> /dev/null | md5sum'
> > +	    % big_file, shell=True)
> > +        md5val.append(out.split()[0])
> > +
> > +        check_call('sudo umount %s' % mount_dir, shell=True)
> > +    except CalledProcessError:
> > +        pytest.skip('Setup failed for filesystem: ' + fs_type)
> > +    else:
> > +        yield [fs_ubtype, fs_img, md5val]
> > +    finally:
> > +        call('sudo umount %s' % mount_dir, shell=True)
> > +	call('rmdir -rf %s' % mount_dir, shell=True)
> > +#        if fs_img:
> > +#            call('rm -f %s' % fs_img, shell=True)
> > diff --git a/test/py/tests/test_fs/fstest_defs.py b/test/py/tests/test_fs/fstest_defs.py
> > new file mode 100644
> > index 000000000000..f26dd06cacf2
> > --- /dev/null
> > +++ b/test/py/tests/test_fs/fstest_defs.py
> > @@ -0,0 +1,10 @@
> > +# SPDX-License-Identifier:      GPL-2.0+
> > +
> > +# $SMALL_FILE is the name of the 1MB file in the file system image
> > +SMALL_FILE='1MB.file'
> > +
> > +# $BIG_FILE is the name of the 2.5GB file in the file system image
> > +BIG_FILE='2.5GB.file'
> > +
> > +ADDR=0x01000008
> > +LENGTH=0x00100000
> > diff --git a/test/py/tests/test_fs/test_basic.py b/test/py/tests/test_fs/test_basic.py
> > new file mode 100644
> > index 000000000000..5258d98d42a9
> > --- /dev/null
> > +++ b/test/py/tests/test_fs/test_basic.py
> > @@ -0,0 +1,246 @@
> > +# SPDX-License-Identifier:      GPL-2.0+
> > +# Copyright (c) 2018, Linaro Limited
> > +# Author: Takahiro Akashi <takahiro.akashi@linaro.org>
> > +#
> > +# U-Boot File System:Basic Test
> > +
> > +"""
> > +This test verifies basic read/write operation on file system.
> > +"""
> > +
> > +import pytest
> > +import re
> > +from fstest_defs import *
> > +
> > + at pytest.mark.boardspec('sandbox')
> > +class TestFsBasic(object):
> > +    def test_fs1(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 1 - ls'):
> > +            # Test Case 1 - ls
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sls host 0:0' % fs_type])
> > +            assert(re.search('2621440000 *%s' % BIG_FILE, ''.join(output)))
> > +            assert(re.search('1048576 *%s' % SMALL_FILE, ''.join(output)))
> > +
> > +            # In addition, test with a nonexistent directory to see if we crash.
> > +            output2 = u_boot_console.run_command(
> > +                '%sls host 0:0 invalid_d' % fs_type)
> > +            if fs_type == 'ext4':
> > +                assert('Can not find directory' in output2)
> > +            else:
> > +                assert('' == output2)
> > +
> > +    def test_fs2(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 2 - size (small)'):
> > +            # 1MB is 0x0010 0000
> > +            # Test Case 2a - size of small file
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%ssize host 0:0 /%s' % (fs_type, SMALL_FILE),
> > +                'printenv filesize',
> > +                'setenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +            # Test Case 2b - size of small file via a path using '..'
> > +            output = u_boot_console.run_command_list([
> > +                '%ssize host 0:0 /SUBDIR/../%s' % (fs_type, SMALL_FILE),
> > +                'printenv filesize',
> > +                'setenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +    def test_fs3(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 3 - size (big)'):
> > +            # 2.5GB (1024*1024*2500) is 0x9C40 0000
> > +            # Test Case 3 - size of big file
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%ssize host 0:0 /%s' % (fs_type, BIG_FILE),
> > +                'printenv filesize',
> > +                'setenv filesize'])
> > +            assert('filesize=9c400000' in ''.join(output))
> > +
> > +    def test_fs4(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 4 - load (small)'):
> > +            # Test Case 4a - Read full 1MB of small file
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s' % (fs_type, ADDR, SMALL_FILE),
> > +                'printenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +            # Test Case 4b - Read full 1MB of small file
> > +            output = u_boot_console.run_command_list([
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[0] in ''.join(output))
> > +
> > +    def test_fs5(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 5 - load (big)'):
> > +            # Test Case 5a - First 1MB of big file
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s %x 0x0' % (fs_type, ADDR, BIG_FILE, LENGTH),
> > +                'printenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +            # Test Case 5b - First 1MB of big file
> > +            output = u_boot_console.run_command_list([
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[1] in ''.join(output))
> > +
> > +    def test_fs6(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 6 - load (big from last 1MB)'):
> > +            # fails for ext as no offset support
> > +            # Test Case 6a - Last 1MB of big file
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s %x 0x9c300000'
> > +                    % (fs_type, ADDR, BIG_FILE, LENGTH),
> > +                'printenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +            # Test Case 6b - Last 1MB of big file
> > +            output = u_boot_console.run_command_list([
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[2] in ''.join(output))
> > +
> > +    def test_fs7(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 7 - load (big from last 1MB in 2GB)'):
> > +            # fails for ext as no offset support
> > +            # Test Case 7a - One from the last 1MB chunk of 2GB
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s %x 0x7ff00000'
> > +                    % (fs_type, ADDR, BIG_FILE, LENGTH),
> > +                'printenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +            # Test Case 7b - One from the last 1MB chunk of 2GB
> > +            output = u_boot_console.run_command_list([
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[3] in ''.join(output))
> > +
> > +    def test_fs8(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 8 - load (big from first 1MB in 2GB)'):
> > +            # fails for ext as no offset support
> > +            # Test Case 8a - One from the start 1MB chunk from 2GB
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s %x 0x80000000'
> > +                    % (fs_type, ADDR, BIG_FILE, LENGTH),
> > +                'printenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +            # Test Case 8b - One from the start 1MB chunk from 2GB
> > +            output = u_boot_console.run_command_list([
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[4] in ''.join(output))
> > +
> > +    def test_fs9(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 9 - load (crossing 2GB boundary)'):
> > +            # fails for ext as no offset support
> > +            # Test Case 9a - One 1MB chunk crossing the 2GB boundary
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s %x 0x7ff80000'
> > +                    % (fs_type, ADDR, BIG_FILE, LENGTH),
> > +                'printenv filesize'])
> > +            assert('filesize=100000' in ''.join(output))
> > +
> > +            # Test Case 9b - One 1MB chunk crossing the 2GB boundary
> > +            output = u_boot_console.run_command_list([
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[5] in ''.join(output))
> > +
> > +    def test_fs10(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 10 - load (over file end)'):
> > +            # Generic failure case
> > +            # Test Case 10 - 2MB chunk from the last 1MB of big file
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s 0x00200000 0x9c300000'
> > +                    % (fs_type, ADDR, BIG_FILE),
> > +                'printenv filesize',
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +        assert('filesize=100000' in ''.join(output))
> > +
> > +    def test_fs11(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 11 - write (small)'):
> > +            # Read 1MB from small file
> > +            # Write it back to test the writes
> > +            # Test Case 11a - Check that the write succeeded
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s' % (fs_type, ADDR, SMALL_FILE),
> > +                '%swrite host 0:0 %x /%s.w $filesize'
> > +                    % (fs_type, ADDR, SMALL_FILE)])
> > +            assert('1048576 bytes written' in ''.join(output))
> > +
> > +            # Test Case 11b - Check md5 of written to is same
> > +            # as the one read from
> > +            output = u_boot_console.run_command_list([
> > +                '%sload host 0:0 %x /%s.w' % (fs_type, ADDR, SMALL_FILE),
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[0] in ''.join(output))
> > +
> > +    def test_fs12(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 12 - write (".")'):
> > +            # Next test case checks writing a file whose dirent
> > +            # is the first in the block, which is always true for "."
> > +            # The write should fail, but the lookup should work
> > +            # Test Case 12 - Check directory traversal
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%swrite host 0:0 %x /. 0x10' % (fs_type, ADDR)])
> > +            assert('Unable to write' in ''.join(output))
> > +
> > +    def test_fs13(self, u_boot_console, fs_obj_basic):
> > +        fs_type,fs_img,md5val = fs_obj_basic
> > +        with u_boot_console.log.section('Test Case 13 - write (small w/ "./")'):
> > +            # Read 1MB from small file
> > +            # Write it via "same directory", i.e. "." dirent
> > +            # Test Case 13a - Check directory traversal
> > +            output = u_boot_console.run_command_list([
> > +                'host bind 0 %s' % fs_img,
> > +                '%sload host 0:0 %x /%s' % (fs_type, ADDR, SMALL_FILE),
> > +                '%swrite host 0:0 %x /./%s2 $filesize'
> > +                    % (fs_type, ADDR, SMALL_FILE)])
> > +            assert('1048576 bytes written' in ''.join(output))
> > +
> > +            # Test Case 13b - Check md5 of written to is same
> > +            # as the one read from
> > +            output = u_boot_console.run_command_list([
> > +                'mw.b %x 00 100' % ADDR,
> > +                '%sload host 0:0 %x /./%s2' % (fs_type, ADDR, SMALL_FILE),
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[0] in ''.join(output))
> > +
> > +            # Test Case 13c - Check md5 of written to is same
> > +            # as the one read from
> > +            output = u_boot_console.run_command_list([
> > +                'mw.b %x 00 100' % ADDR,
> > +                '%sload host 0:0 %x /%s2' % (fs_type, ADDR, SMALL_FILE),
> > +                'md5sum %x $filesize' % ADDR,
> > +                'setenv filesize'])
> > +            assert(md5val[0] in ''.join(output))
> > 
> 

[U-Boot] [RFC 1/3] test/py: convert fs-test.sh to pytest

[Alexander Graf]

> Am 31.08.2018 um 09:31 schrieb AKASHI Takahiro <takahiro.akashi@linaro.org>:
> 
>> On Wed, Aug 29, 2018 at 11:36:51PM +0200, Heinrich Schuchardt wrote:
>>> On 08/23/2018 09:25 AM, AKASHI Takahiro wrote:
>>> In this commit, the same set of test cases as in test/fs/fs-test.sh
>>> is provided using pytest framework.
>>> Actually, fs-test.sh provides three variants:"sb" (sb command), "nonfs"
>>> (fatxx and etc.) and "fs" (hostfs), and this patch currently supports
>>> only "nonfs" variant; So it is not a replacement of fs-test.sh for now.
>>> 
>>> Simple usage:
>>>  $ py.test test/py/tests/test_fs [<other options>]
>>> 
>>> You may also specify filesystem types to be tested:
>>>  $ py.test test/py/tests/test_fs --fs-type fat32 [<other options>]
>>> 
>>> Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
>>> ---
>>> test/py/tests/test_fs/conftest.py    | 175 +++++++++++++++++++
>>> test/py/tests/test_fs/fstest_defs.py |  10 ++
>>> test/py/tests/test_fs/test_basic.py  | 246 +++++++++++++++++++++++++++
>>> 3 files changed, 431 insertions(+)
>>> create mode 100644 test/py/tests/test_fs/conftest.py
>>> create mode 100644 test/py/tests/test_fs/fstest_defs.py
>>> create mode 100644 test/py/tests/test_fs/test_basic.py
>>> 
>>> diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
>>> new file mode 100644
>>> index 000000000000..fefeb4c9663f
>>> --- /dev/null
>>> +++ b/test/py/tests/test_fs/conftest.py
>>> @@ -0,0 +1,175 @@
>>> +# SPDX-License-Identifier:      GPL-2.0+
>>> +# Copyright (c) 2018, Linaro Limited
>>> +# Author: Takahiro Akashi <takahiro.akashi@linaro.org>
>>> +
>>> +import pytest
>>> +import re
>>> +from subprocess import call, check_call, check_output, CalledProcessError
>>> +from fstest_defs import *
>>> +
>>> +supported_fs_basic = ['fat16', 'fat32', 'ext4']
>>> +
>>> +#
>>> +# Filesystem test specific setup
>>> +#
>>> +def pytest_addoption(parser):
>>> +    parser.addoption('--fs-type', action='append', default=None,
>>> +        help='Targeting Filesystem Types')
>>> +
>>> +def pytest_configure(config):
>>> +    global supported_fs_basic
>>> +
>>> +    def intersect(listA, listB):
>>> +        return  [x for x in listA if x in listB]
>>> +
>>> +    supported_fs = config.getoption('fs_type')
>>> +    if supported_fs:
>>> +        print("*** FS TYPE modified: %s" % supported_fs)
>>> +        supported_fs_basic =  intersect(supported_fs, supported_fs_basic)
>>> +
>>> +def pytest_generate_tests(metafunc):
>>> +    if 'fs_obj_basic' in metafunc.fixturenames:
>>> +        metafunc.parametrize('fs_obj_basic', supported_fs_basic,
>>> +            indirect=True, scope='module')
>>> +
>>> +#
>>> +# Helper functions
>>> +#
>>> +def fstype_to_ubname(fs_type):
>>> +    if re.match('fat', fs_type):
>>> +        return 'fat'
>>> +    else:
>>> +        return fs_type
>>> +
>>> +def check_ubconfig(config, fs_type):
>>> +    if not config.buildconfig.get('config_cmd_%s' % fs_type, None):
>>> +        pytest.skip('.config feature "CMD_%s" not enabled' % fs_type.upper())
>>> +    if not config.buildconfig.get('config_%s_write' % fs_type, None):
>>> +        pytest.skip('.config feature "%s_WRITE" not enabled'
>>> +        % fs_type.upper())
>>> +
>>> +def mk_fs(config, fs_type, size, id):
>>> +    fs_img = '%s.%s.img' % (id, fs_type)
>>> +    fs_img = config.persistent_data_dir + '/' + fs_img
>>> +
>>> +    if fs_type == 'fat16':
>>> +        mkfs_opt = '-F 16'
>>> +    elif fs_type == 'fat32':
>>> +        mkfs_opt = '-F 32'
>>> +    else:
>>> +        mkfs_opt = ''
>>> +
>>> +    if re.match('fat', fs_type):
>>> +        fs_lnxtype = 'vfat'
>>> +    else:
>>> +        fs_lnxtype = fs_type
>>> +
>>> +    count = (size + 1023) / 1024
>>> +
>>> +    try:
>>> +        check_call('rm -f %s' % fs_img, shell=True)
>>> +        check_call('dd if=/dev/zero of=%s bs=1K count=%d'
>>> +            % (fs_img, count), shell=True)
>>> +        check_call('mkfs.%s %s %s'
>>> +            % (fs_lnxtype, mkfs_opt, fs_img), shell=True)
>>> +        return fs_img
>>> +    except CalledProcessError:
>>> +        call('rm -f %s' % fs_img, shell=True)
>>> +        raise
>>> +
>>> +#
>>> +# Fixture for basic fs test
>>> +#     derived from test/fs/fs-test.sh
>>> +#
>>> +# NOTE: yield_fixture was deprecated since pytest-3.0
>>> + at pytest.yield_fixture()
>>> +def fs_obj_basic(request, u_boot_config):
>>> +    fs_type = request.param
>>> +    fs_img = ''
>>> +
>>> +    fs_ubtype = fstype_to_ubname(fs_type)
>>> +    check_ubconfig(u_boot_config, fs_ubtype)
>>> +
>>> +    mount_dir = u_boot_config.persistent_data_dir + '/mnt'
>>> +    small_file = mount_dir + '/' + SMALL_FILE
>>> +    big_file = mount_dir + '/' + BIG_FILE
>>> +    try:
>>> +
>>> +        # 3GiB volume
>>> +        fs_img = mk_fs(u_boot_config, fs_type, 0xc0000000, '3GB')
>>> +
>>> +        # Mount the image so we can populate it.
>>> +        check_call('mkdir -p %s' % mount_dir, shell=True)
>>> +        check_call('sudo mount -o loop,rw %s %s'
>>> +            % (fs_img, mount_dir), shell=True)
>> 
>> Should I grant sudo to anybody who can commit to U-Boot?
>> 
>> Just use exfat-fuse and fuse2fs.
> 
> As far fas I tried, exfat-fuse will not be able to mount
> a fat (vfat) file system, unlike fuse2fs handling ext2 as well as ext4.
> So this cannot be a solution.
> If you know how to mount fat fs with exfat-fuse as a non-root user,
> please let me know.

Could we just make use of one of the many fat access libraries available in Python?

Alex

[U-Boot] [RFC 1/3] test/py: convert fs-test.sh to pytest

[Tuomas Tynkkynen]

Hi,

On 08/31/2018 10:22 AM, AKASHI Takahiro wrote:
> Hi Tuomas,
> 
> Thank you for interesting pointers.
> 
> On Thu, Aug 30, 2018 at 01:56:41PM +0300, Tuomas Tynkkynen wrote:
>> Hi Heinrich, Takahiro
>>
>> On 08/30/2018 01:26 PM, AKASHI Takahiro wrote:
>>> On Thu, Aug 30, 2018 at 12:01:32PM +0200, Heinrich Schuchardt wrote:
>>>> On 08/30/2018 08:52 AM, AKASHI Takahiro wrote:
>>>>> On Wed, Aug 29, 2018 at 11:36:51PM +0200, Heinrich Schuchardt wrote:
...
>>>> Using sudo for me is a NO-NO. I will not run any test that uses sudo.
>>>
>>> So this means that you have never tested file system using test-fs.sh.
>>>
>>> Since my script is logically "general", it can, if we want, run against
>>> other file systems as well. "sudo mount" is the only solution for those cases.
>>>
>> There are two general non-root implementations that I know of:
>>
>> 1) http://libguestfs.org/ which IIRC launches a small Linux VM in QEMU
>>     to do the filesystem accesses. I am not sure if the performance would be
>>     acceptable without KVM (which I assume we don't have in Travis).
> 
> I didn't dig into this tool, but if it is all about VM on qemu,
> the discussion here, whether we should be allowed to use sudo or not,
> would be pointless as we can do whatever we want to do under VM.
>

But that's exactly what was wanted - that the user running the tests for U-Boot
doesn't need to have sudo access on their build machine.

BTW I briefly benchmarked this on a AWS free-tier machine (no KVM available),
and copying a 2G file onto an ext4 partition took:

- 33s natively (sudo mount -o loop fs.img /mnt; sudo cp bigfile /mnt/; sudo umount /mnt)
- 2m 44s with guestfs (guestfish add fs.img : run : mount /dev/sda / : copy-in bigfile /)

so a 80% slowdown for things like Travis CI but still not unbearably slow.
I'm assuming here that it will run much faster with KVM and that developers usually
have KVM-capable machines so this shouldn't affect developers too badly.


>> 2) https://github.com/lkl/linux which is a port of Linux to run in userspace
>>     as a library. It comes with tools like cptofs and lklfuse to access any
>>     filesystem Linux has a driver for.
> 
> It appears to be a kind of libos or unikernel, or rather,
> resembles a sandbox of u-boot?

Correct.

> 
>> Sadly lkl isn't packaged in many distros.
> 
> Too bad.
> 

Yes, I'd say libguestfs is the most promising solution so far.