All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
@ 2021-05-13 11:40 Heinrich Schuchardt
  2021-05-17  6:33 ` Andy Shevchenko
  0 siblings, 1 reply; 18+ messages in thread
From: Heinrich Schuchardt @ 2021-05-13 11:40 UTC (permalink / raw)
  To: u-boot

Since commit 1ba21bb06b08 ("test: Don't unmount not (yet) mounted system")
the following tests are skipped:

test/py/tests/test_fs/test_basic.py
test/py/tests/test_fs/test_ext.py

SKIPPED [13] test/py/tests/test_fs/conftest.py:350: Setup failed for
filesystem: ext4. Command 'guestmount -a
build-sandbox/persistent-data/3GB.ext4.img -m /dev/sda
build-sandbox/persistent-data/mnt' returned non-zero exit status 1.

Let's revert the patch to get our tests back.

Fixes: 1ba21bb06b08 ("test: Don't unmount not (yet) mounted system")
Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
---
 test/py/tests/test_fs/conftest.py | 78 +++++++++----------------------
 test/run                          |  8 ++--
 2 files changed, 26 insertions(+), 60 deletions(-)

diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
index 50af9efcf7..ec70e8c4ef 100644
--- a/test/py/tests/test_fs/conftest.py
+++ b/test/py/tests/test_fs/conftest.py
@@ -270,20 +270,9 @@ def fs_obj_basic(request, u_boot_config):

         # 3GiB volume
         fs_img = mk_fs(u_boot_config, fs_type, 0xc0000000, '3GB')
-    except CalledProcessError as err:
-        pytest.skip('Creating failed for filesystem: ' + fs_type + '. {}'.format(err))
-        return
-
-    try:
-        check_call('mkdir -p %s' % mount_dir, shell=True)
-    except CalledProcessError as err:
-        pytest.skip('Preparing mount folder failed for filesystem: ' + fs_type + '. {}'.format(err))
-        return
-    finally:
-        call('rm -f %s' % fs_img, shell=True)

-    try:
         # Mount the image so we can populate it.
+        check_call('mkdir -p %s' % mount_dir, shell=True)
         mount_fs(fs_type, fs_img, mount_dir)

         # Create a subdirectory.
@@ -346,15 +335,18 @@ def fs_obj_basic(request, u_boot_config):
 	    % big_file, shell=True).decode()
         md5val.append(out.split()[0])

+        umount_fs(mount_dir)
     except CalledProcessError as err:
-        pytest.skip('Setup failed for filesystem: ' + fs_type + '. {}'.format(err))
+        pytest.skip('Setup failed for filesystem: ' + fs_type + \
+            '. {}'.format(err))
         return
     else:
         yield [fs_ubtype, fs_img, md5val]
     finally:
         umount_fs(mount_dir)
         call('rmdir %s' % mount_dir, shell=True)
-        call('rm -f %s' % fs_img, shell=True)
+        if fs_img:
+            call('rm -f %s' % fs_img, shell=True)

 #
 # Fixture for extended fs test
@@ -386,20 +378,9 @@ def fs_obj_ext(request, u_boot_config):

         # 128MiB volume
         fs_img = mk_fs(u_boot_config, fs_type, 0x8000000, '128MB')
-    except CalledProcessError as err:
-        pytest.skip('Creating failed for filesystem: ' + fs_type + '. {}'.format(err))
-        return
-
-    try:
-        check_call('mkdir -p %s' % mount_dir, shell=True)
-    except CalledProcessError as err:
-        pytest.skip('Preparing mount folder failed for filesystem: ' + fs_type + '. {}'.format(err))
-        return
-    finally:
-        call('rm -f %s' % fs_img, shell=True)

-    try:
         # Mount the image so we can populate it.
+        check_call('mkdir -p %s' % mount_dir, shell=True)
         mount_fs(fs_type, fs_img, mount_dir)

         # Create a test directory
@@ -441,6 +422,7 @@ def fs_obj_ext(request, u_boot_config):
         md5val.append(out.split()[0])

         check_call('rm %s' % tmp_file, shell=True)
+        umount_fs(mount_dir)
     except CalledProcessError:
         pytest.skip('Setup failed for filesystem: ' + fs_type)
         return
@@ -449,7 +431,8 @@ def fs_obj_ext(request, u_boot_config):
     finally:
         umount_fs(mount_dir)
         call('rmdir %s' % mount_dir, shell=True)
-        call('rm -f %s' % fs_img, shell=True)
+        if fs_img:
+            call('rm -f %s' % fs_img, shell=True)

 #
 # Fixture for mkdir test
@@ -477,10 +460,11 @@ def fs_obj_mkdir(request, u_boot_config):
         fs_img = mk_fs(u_boot_config, fs_type, 0x8000000, '128MB')
     except:
         pytest.skip('Setup failed for filesystem: ' + fs_type)
-        return
     else:
         yield [fs_ubtype, fs_img]
-    call('rm -f %s' % fs_img, shell=True)
+    finally:
+        if fs_img:
+            call('rm -f %s' % fs_img, shell=True)

 #
 # Fixture for unlink test
@@ -509,20 +493,9 @@ def fs_obj_unlink(request, u_boot_config):

         # 128MiB volume
         fs_img = mk_fs(u_boot_config, fs_type, 0x8000000, '128MB')
-    except CalledProcessError as err:
-        pytest.skip('Creating failed for filesystem: ' + fs_type + '. {}'.format(err))
-        return
-
-    try:
-        check_call('mkdir -p %s' % mount_dir, shell=True)
-    except CalledProcessError as err:
-        pytest.skip('Preparing mount folder failed for filesystem: ' + fs_type + '. {}'.format(err))
-        return
-    finally:
-        call('rm -f %s' % fs_img, shell=True)

-    try:
         # Mount the image so we can populate it.
+        check_call('mkdir -p %s' % mount_dir, shell=True)
         mount_fs(fs_type, fs_img, mount_dir)

         # Test Case 1 & 3
@@ -546,6 +519,7 @@ def fs_obj_unlink(request, u_boot_config):
         check_call('dd if=/dev/urandom of=%s/dir5/file1 bs=1K count=1'
                                     % mount_dir, shell=True)

+        umount_fs(mount_dir)
     except CalledProcessError:
         pytest.skip('Setup failed for filesystem: ' + fs_type)
         return
@@ -554,7 +528,8 @@ def fs_obj_unlink(request, u_boot_config):
     finally:
         umount_fs(mount_dir)
         call('rmdir %s' % mount_dir, shell=True)
-        call('rm -f %s' % fs_img, shell=True)
+        if fs_img:
+            call('rm -f %s' % fs_img, shell=True)

 #
 # Fixture for symlink fs test
@@ -584,22 +559,11 @@ def fs_obj_symlink(request, u_boot_config):

     try:

-        # 1GiB volume
+        # 3GiB volume
         fs_img = mk_fs(u_boot_config, fs_type, 0x40000000, '1GB')
-    except CalledProcessError as err:
-        pytest.skip('Creating failed for filesystem: ' + fs_type + '. {}'.format(err))
-        return

-    try:
-        check_call('mkdir -p %s' % mount_dir, shell=True)
-    except CalledProcessError as err:
-        pytest.skip('Preparing mount folder failed for filesystem: ' + fs_type + '. {}'.format(err))
-        return
-    finally:
-        call('rm -f %s' % fs_img, shell=True)
-
-    try:
         # Mount the image so we can populate it.
+        check_call('mkdir -p %s' % mount_dir, shell=True)
         mount_fs(fs_type, fs_img, mount_dir)

         # Create a subdirectory.
@@ -623,6 +587,7 @@ def fs_obj_symlink(request, u_boot_config):
             % medium_file, shell=True).decode()
         md5val.extend([out.split()[0]])

+        umount_fs(mount_dir)
     except CalledProcessError:
         pytest.skip('Setup failed for filesystem: ' + fs_type)
         return
@@ -631,4 +596,5 @@ def fs_obj_symlink(request, u_boot_config):
     finally:
         umount_fs(mount_dir)
         call('rmdir %s' % mount_dir, shell=True)
-        call('rm -f %s' % fs_img, shell=True)
+        if fs_img:
+            call('rm -f %s' % fs_img, shell=True)
diff --git a/test/run b/test/run
index 869406cd8d..04ec1a05e0 100755
--- a/test/run
+++ b/test/run
@@ -22,16 +22,16 @@ failures=0

 if [ -z "$tools_only" ]; then
 	# Run all tests that the standard sandbox build can support
-	run_test "sandbox" ./test/py/test.py --bd sandbox --build \
+	run_test "sandbox" ./test/py/test.py -ra --bd sandbox --build \
 		-m "${mark_expr}"
 fi

 # Run tests which require sandbox_spl
-run_test "sandbox_spl" ./test/py/test.py --bd sandbox_spl --build \
+run_test "sandbox_spl" ./test/py/test.py -ra --bd sandbox_spl --build \
 		-k 'test_ofplatdata or test_handoff or test_spl'

 # Run the sane tests with sandbox_noinst (i.e. without OF_PLATDATA_INST)
-run_test "sandbox_spl" ./test/py/test.py --bd sandbox_noinst --build \
+run_test "sandbox_spl" ./test/py/test.py -ra --bd sandbox_noinst --build \
 		-k 'test_ofplatdata or test_handoff or test_spl'

 if [ -z "$tools_only" ]; then
@@ -39,7 +39,7 @@ if [ -z "$tools_only" ]; then
 	# build which does not enable CONFIG_OF_LIVE for the live device tree, so we can
 	# check that functionality is the same. The standard sandbox build (above) uses
 	# CONFIG_OF_LIVE.
-	run_test "sandbox_flattree" ./test/py/test.py --bd sandbox_flattree \
+	run_test "sandbox_flattree" ./test/py/test.py -ra --bd sandbox_flattree \
 		--build -k test_ut
 fi

--
2.30.2

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-13 11:40 [PATCH 1/1] test: revert Don't unmount not (yet) mounted system Heinrich Schuchardt
@ 2021-05-17  6:33 ` Andy Shevchenko
  2021-05-17  8:48   ` Heinrich Schuchardt
  0 siblings, 1 reply; 18+ messages in thread
From: Andy Shevchenko @ 2021-05-17  6:33 UTC (permalink / raw)
  To: u-boot

On Thu, May 13, 2021 at 2:41 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> Since commit 1ba21bb06b08 ("test: Don't unmount not (yet) mounted system")
> the following tests are skipped:
>
> test/py/tests/test_fs/test_basic.py
> test/py/tests/test_fs/test_ext.py
>
> SKIPPED [13] test/py/tests/test_fs/conftest.py:350: Setup failed for
> filesystem: ext4. Command 'guestmount -a
> build-sandbox/persistent-data/3GB.ext4.img -m /dev/sda
> build-sandbox/persistent-data/mnt' returned non-zero exit status 1.
>
> Let's revert the patch to get our tests back.

Probably we may understand first what is the root cause of this issue?

In my case I can't allow this to happen, because it will annoy system
administrators as I mentioned earlier in the commit message.

So, NAK from me and let's investigate.
Can you provide a command line that I may run on my environment w/o root access?


-- 
With Best Regards,
Andy Shevchenko

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-17  6:33 ` Andy Shevchenko
@ 2021-05-17  8:48   ` Heinrich Schuchardt
  2021-05-17 11:16     ` Andy Shevchenko
  0 siblings, 1 reply; 18+ messages in thread
From: Heinrich Schuchardt @ 2021-05-17  8:48 UTC (permalink / raw)
  To: u-boot

On 17.05.21 08:33, Andy Shevchenko wrote:
> On Thu, May 13, 2021 at 2:41 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>
>> Since commit 1ba21bb06b08 ("test: Don't unmount not (yet) mounted system")
>> the following tests are skipped:
>>
>> test/py/tests/test_fs/test_basic.py
>> test/py/tests/test_fs/test_ext.py
>>
>> SKIPPED [13] test/py/tests/test_fs/conftest.py:350: Setup failed for
>> filesystem: ext4. Command 'guestmount -a
>> build-sandbox/persistent-data/3GB.ext4.img -m /dev/sda
>> build-sandbox/persistent-data/mnt' returned non-zero exit status 1.
>>
>> Let's revert the patch to get our tests back.
>
> Probably we may understand first what is the root cause of this issue?
>
> In my case I can't allow this to happen, because it will annoy system
> administrators as I mentioned earlier in the commit message.
>
> So, NAK from me and let's investigate.
> Can you provide a command line that I may run on my environment w/o root access?
>
>

Hello Andy,

The tests don't require root access if you have installed the
libguestfs-tools package and a Linux kernel.

How can I reproduce the problem with duplicate umount?

Best regards

Heinrich

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-17  8:48   ` Heinrich Schuchardt
@ 2021-05-17 11:16     ` Andy Shevchenko
  2021-05-17 11:35       ` Heinrich Schuchardt
  0 siblings, 1 reply; 18+ messages in thread
From: Andy Shevchenko @ 2021-05-17 11:16 UTC (permalink / raw)
  To: u-boot

On Mon, May 17, 2021 at 10:48:33AM +0200, Heinrich Schuchardt wrote:
> On 17.05.21 08:33, Andy Shevchenko wrote:
> > On Thu, May 13, 2021 at 2:41 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >>
> >> Since commit 1ba21bb06b08 ("test: Don't unmount not (yet) mounted system")
> >> the following tests are skipped:
> >>
> >> test/py/tests/test_fs/test_basic.py
> >> test/py/tests/test_fs/test_ext.py
> >>
> >> SKIPPED [13] test/py/tests/test_fs/conftest.py:350: Setup failed for
> >> filesystem: ext4. Command 'guestmount -a
> >> build-sandbox/persistent-data/3GB.ext4.img -m /dev/sda
> >> build-sandbox/persistent-data/mnt' returned non-zero exit status 1.
> >>
> >> Let's revert the patch to get our tests back.
> >
> > Probably we may understand first what is the root cause of this issue?
> >
> > In my case I can't allow this to happen, because it will annoy system
> > administrators as I mentioned earlier in the commit message.
> >
> > So, NAK from me and let's investigate.
> > Can you provide a command line that I may run on my environment w/o root access?
> 
> Hello Andy,
> 
> The tests don't require root access if you have installed the
> libguestfs-tools package and a Linux kernel.
> 
> How can I reproduce the problem with duplicate umount?

I was running this 2+ times in a row (*)

./test/py/test.py --bd sandbox --build

*) I can't run tests right now due to they are more or less constantly broken
one way or the other, now

============================================== test session starts ==============================================
platform linux -- Python 3.9.2, pytest-6.0.2, py-1.10.0, pluggy-0.13.0
rootdir: /home/andy/prj/u-boot/test/py, configfile: pytest.ini
collected 810 items / 1 error / 809 selected

___________________________________ ERROR collecting tests/test_fit_ecdsa.py ____________________________________
E   ModuleNotFoundError: No module named 'Cryptodome'


And suddenly this is a fatal error for unknown reason.

I will check later when I have time how to work around, or if meanwhile
somebody tells me what are the steps to fix it.

I'm using v2021.07-rc2

-- 
With Best Regards,
Andy Shevchenko

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-17 11:16     ` Andy Shevchenko
@ 2021-05-17 11:35       ` Heinrich Schuchardt
  2021-05-17 11:44         ` Andy Shevchenko
  0 siblings, 1 reply; 18+ messages in thread
From: Heinrich Schuchardt @ 2021-05-17 11:35 UTC (permalink / raw)
  To: u-boot

On 17.05.21 13:16, Andy Shevchenko wrote:
> On Mon, May 17, 2021 at 10:48:33AM +0200, Heinrich Schuchardt wrote:
>> On 17.05.21 08:33, Andy Shevchenko wrote:
>>> On Thu, May 13, 2021 at 2:41 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>>>
>>>> Since commit 1ba21bb06b08 ("test: Don't unmount not (yet) mounted system")
>>>> the following tests are skipped:
>>>>
>>>> test/py/tests/test_fs/test_basic.py
>>>> test/py/tests/test_fs/test_ext.py
>>>>
>>>> SKIPPED [13] test/py/tests/test_fs/conftest.py:350: Setup failed for
>>>> filesystem: ext4. Command 'guestmount -a
>>>> build-sandbox/persistent-data/3GB.ext4.img -m /dev/sda
>>>> build-sandbox/persistent-data/mnt' returned non-zero exit status 1.
>>>>
>>>> Let's revert the patch to get our tests back.
>>>
>>> Probably we may understand first what is the root cause of this issue?
>>>
>>> In my case I can't allow this to happen, because it will annoy system
>>> administrators as I mentioned earlier in the commit message.
>>>
>>> So, NAK from me and let's investigate.
>>> Can you provide a command line that I may run on my environment w/o root access?
>>
>> Hello Andy,
>>
>> The tests don't require root access if you have installed the
>> libguestfs-tools package and a Linux kernel.
>>
>> How can I reproduce the problem with duplicate umount?
>
> I was running this 2+ times in a row (*)
>
> ./test/py/test.py --bd sandbox --build
>
> *) I can't run tests right now due to they are more or less constantly broken
> one way or the other, now
>
> ============================================== test session starts ==============================================
> platform linux -- Python 3.9.2, pytest-6.0.2, py-1.10.0, pluggy-0.13.0
> rootdir: /home/andy/prj/u-boot/test/py, configfile: pytest.ini
> collected 810 items / 1 error / 809 selected
>
> ___________________________________ ERROR collecting tests/test_fit_ecdsa.py ____________________________________
> E   ModuleNotFoundError: No module named 'Cryptodome'

The missing package is available via

    apt-get install python3-pycryptodome # Debian/Ubuntu

or

    dnf install python3-pycryptodomex # Fedora

Best regards

Heinrich

>
>
> And suddenly this is a fatal error for unknown reason.
>
> I will check later when I have time how to work around, or if meanwhile
> somebody tells me what are the steps to fix it.
>
> I'm using v2021.07-rc2
>

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-17 11:35       ` Heinrich Schuchardt
@ 2021-05-17 11:44         ` Andy Shevchenko
  2021-05-17 13:21           ` Heinrich Schuchardt
  0 siblings, 1 reply; 18+ messages in thread
From: Andy Shevchenko @ 2021-05-17 11:44 UTC (permalink / raw)
  To: u-boot

On Mon, May 17, 2021 at 2:35 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> On 17.05.21 13:16, Andy Shevchenko wrote:
> > On Mon, May 17, 2021 at 10:48:33AM +0200, Heinrich Schuchardt wrote:
> >> On 17.05.21 08:33, Andy Shevchenko wrote:
> >>> On Thu, May 13, 2021 at 2:41 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >>>>
> >>>> Since commit 1ba21bb06b08 ("test: Don't unmount not (yet) mounted system")
> >>>> the following tests are skipped:
> >>>>
> >>>> test/py/tests/test_fs/test_basic.py
> >>>> test/py/tests/test_fs/test_ext.py
> >>>>
> >>>> SKIPPED [13] test/py/tests/test_fs/conftest.py:350: Setup failed for
> >>>> filesystem: ext4. Command 'guestmount -a
> >>>> build-sandbox/persistent-data/3GB.ext4.img -m /dev/sda
> >>>> build-sandbox/persistent-data/mnt' returned non-zero exit status 1.
> >>>>
> >>>> Let's revert the patch to get our tests back.
> >>>
> >>> Probably we may understand first what is the root cause of this issue?
> >>>
> >>> In my case I can't allow this to happen, because it will annoy system
> >>> administrators as I mentioned earlier in the commit message.
> >>>
> >>> So, NAK from me and let's investigate.
> >>> Can you provide a command line that I may run on my environment w/o root access?
> >>
> >> Hello Andy,
> >>
> >> The tests don't require root access if you have installed the
> >> libguestfs-tools package and a Linux kernel.
> >>
> >> How can I reproduce the problem with duplicate umount?
> >
> > I was running this 2+ times in a row (*)
> >
> > ./test/py/test.py --bd sandbox --build

(1)

> >
> > *) I can't run tests right now due to they are more or less constantly broken
> > one way or the other, now
> >
> > ============================================== test session starts ==============================================
> > platform linux -- Python 3.9.2, pytest-6.0.2, py-1.10.0, pluggy-0.13.0
> > rootdir: /home/andy/prj/u-boot/test/py, configfile: pytest.ini
> > collected 810 items / 1 error / 809 selected
> >
> > ___________________________________ ERROR collecting tests/test_fit_ecdsa.py ____________________________________
> > E   ModuleNotFoundError: No module named 'Cryptodome'
>
> The missing package is available via
>
>     apt-get install python3-pycryptodome # Debian/Ubuntu
>
> or
>
>     dnf install python3-pycryptodomex # Fedora

Thanks.

So, I have run above mentioned line (1) with current U-Boot (see
above), everything is fine, then I have reverted the commit (as your
patch does, correct), and oops

test/py/tests/test_efi_secboot/test_unsigned.py sss
                                   [ 88%]
test/py/tests/test_fs/test_basic.py [sudo] password for andy:
Sorry, try again.
[sudo] password for andy:
Sorry, try again.
[sudo] password for andy:
sssssssssssss[sudo] password for andy:

Now I'm waiting for a punishment from the admin, thanks to this test round.

I'm not going to repeat this again, please understand me correctly.

>
> Best regards
>
> Heinrich
>
> >
> >
> > And suddenly this is a fatal error for unknown reason.
> >
> > I will check later when I have time how to work around, or if meanwhile
> > somebody tells me what are the steps to fix it.
> >
> > I'm using v2021.07-rc2
> >
>


-- 
With Best Regards,
Andy Shevchenko

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-17 11:44         ` Andy Shevchenko
@ 2021-05-17 13:21           ` Heinrich Schuchardt
  2021-05-17 13:25             ` Andy Shevchenko
                               ` (3 more replies)
  0 siblings, 4 replies; 18+ messages in thread
From: Heinrich Schuchardt @ 2021-05-17 13:21 UTC (permalink / raw)
  To: u-boot

On 17.05.21 13:44, Andy Shevchenko wrote:
> On Mon, May 17, 2021 at 2:35 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>
>> On 17.05.21 13:16, Andy Shevchenko wrote:
>>> On Mon, May 17, 2021 at 10:48:33AM +0200, Heinrich Schuchardt wrote:
>>>> On 17.05.21 08:33, Andy Shevchenko wrote:
>>>>> On Thu, May 13, 2021 at 2:41 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>>>>>
>>>>>> Since commit 1ba21bb06b08 ("test: Don't unmount not (yet) mounted system")
>>>>>> the following tests are skipped:
>>>>>>
>>>>>> test/py/tests/test_fs/test_basic.py
>>>>>> test/py/tests/test_fs/test_ext.py
>>>>>>
>>>>>> SKIPPED [13] test/py/tests/test_fs/conftest.py:350: Setup failed for
>>>>>> filesystem: ext4. Command 'guestmount -a
>>>>>> build-sandbox/persistent-data/3GB.ext4.img -m /dev/sda
>>>>>> build-sandbox/persistent-data/mnt' returned non-zero exit status 1.
>>>>>>
>>>>>> Let's revert the patch to get our tests back.
>>>>>
>>>>> Probably we may understand first what is the root cause of this issue?
>>>>>
>>>>> In my case I can't allow this to happen, because it will annoy system
>>>>> administrators as I mentioned earlier in the commit message.
>>>>>
>>>>> So, NAK from me and let's investigate.
>>>>> Can you provide a command line that I may run on my environment w/o root access?
>>>>
>>>> Hello Andy,
>>>>
>>>> The tests don't require root access if you have installed the
>>>> libguestfs-tools package and a Linux kernel.
>>>>
>>>> How can I reproduce the problem with duplicate umount?
>>>
>>> I was running this 2+ times in a row (*)
>>>
>>> ./test/py/test.py --bd sandbox --build
>
> (1)
>
>>>
>>> *) I can't run tests right now due to they are more or less constantly broken
>>> one way or the other, now
>>>
>>> ============================================== test session starts ==============================================
>>> platform linux -- Python 3.9.2, pytest-6.0.2, py-1.10.0, pluggy-0.13.0
>>> rootdir: /home/andy/prj/u-boot/test/py, configfile: pytest.ini
>>> collected 810 items / 1 error / 809 selected
>>>
>>> ___________________________________ ERROR collecting tests/test_fit_ecdsa.py ____________________________________
>>> E   ModuleNotFoundError: No module named 'Cryptodome'
>>
>> The missing package is available via
>>
>>     apt-get install python3-pycryptodome # Debian/Ubuntu
>>
>> or
>>
>>     dnf install python3-pycryptodomex # Fedora
>
> Thanks.
>
> So, I have run above mentioned line (1) with current U-Boot (see
> above), everything is fine, then I have reverted the commit (as your
> patch does, correct), and oops
>
> test/py/tests/test_efi_secboot/test_unsigned.py sss
>                                    [ 88%]
> test/py/tests/test_fs/test_basic.py [sudo] password for andy:

If you are asked for a sudo password, you have not install libguestfs.

Please, install the missing package.

> Sorry, try again.
> [sudo] password for andy:
> Sorry, try again.
> [sudo] password for andy:
> sssssssssssss[sudo] password for andy:
>
> Now I'm waiting for a punishment from the admin, thanks to this test round.

make tests (on my local machine)

with origin/master:

test/py/tests/test_efi_secboot/test_unsigned.py ...
test/py/tests/test_fs/test_basic.py sssssssssssssssssssssssssssssssssssssss
test/py/tests/test_fs/test_ext.py ssssssssssssssssssssss
test/py/tests/test_fs/test_fs_cmd.py .
test/py/tests/test_fs/test_mkdir.py ............
test/py/tests/test_fs/test_symlink.py ssss
test/py/tests/test_fs/test_unlink.py ssssssssssssss

with your patch reverted

test/py/tests/test_efi_secboot/test_unsigned.py ...
test/py/tests/test_fs/test_basic.py F............F.........................
test/py/tests/test_fs/test_ext.py ......................
test/py/tests/test_fs/test_fs_cmd.py .
test/py/tests/test_fs/test_mkdir.py ............
test/py/tests/test_fs/test_symlink.py ....
test/py/tests/test_fs/test_unlink.py ..............

The failures are caused by dd being called with conv=fsync before
mounting with guestfs.

Obviously we have two scenarios to test separately:

1) using sudo for mounting
2) using guestfs for mounting

>
> I'm not going to repeat this again, please understand me correctly.

I assume that you possess a private laptop where your are the admin.
Where is the problem?

Best regards

Heinrich

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-17 13:21           ` Heinrich Schuchardt
@ 2021-05-17 13:25             ` Andy Shevchenko
  2021-05-17 13:29             ` Tom Rini
                               ` (2 subsequent siblings)
  3 siblings, 0 replies; 18+ messages in thread
From: Andy Shevchenko @ 2021-05-17 13:25 UTC (permalink / raw)
  To: u-boot

On Mon, May 17, 2021 at 4:21 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> On 17.05.21 13:44, Andy Shevchenko wrote:
> > On Mon, May 17, 2021 at 2:35 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >> On 17.05.21 13:16, Andy Shevchenko wrote:
> >>> On Mon, May 17, 2021 at 10:48:33AM +0200, Heinrich Schuchardt wrote:
> >>>> On 17.05.21 08:33, Andy Shevchenko wrote:
> >>>>> On Thu, May 13, 2021 at 2:41 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >>>>>>
> >>>>>> Since commit 1ba21bb06b08 ("test: Don't unmount not (yet) mounted system")
> >>>>>> the following tests are skipped:
> >>>>>>
> >>>>>> test/py/tests/test_fs/test_basic.py
> >>>>>> test/py/tests/test_fs/test_ext.py
> >>>>>>
> >>>>>> SKIPPED [13] test/py/tests/test_fs/conftest.py:350: Setup failed for
> >>>>>> filesystem: ext4. Command 'guestmount -a
> >>>>>> build-sandbox/persistent-data/3GB.ext4.img -m /dev/sda
> >>>>>> build-sandbox/persistent-data/mnt' returned non-zero exit status 1.
> >>>>>>
> >>>>>> Let's revert the patch to get our tests back.
> >>>>>
> >>>>> Probably we may understand first what is the root cause of this issue?
> >>>>>
> >>>>> In my case I can't allow this to happen, because it will annoy system
> >>>>> administrators as I mentioned earlier in the commit message.
> >>>>>
> >>>>> So, NAK from me and let's investigate.
> >>>>> Can you provide a command line that I may run on my environment w/o root access?
> >>>>
> >>>> Hello Andy,
> >>>>
> >>>> The tests don't require root access if you have installed the
> >>>> libguestfs-tools package and a Linux kernel.
> >>>>
> >>>> How can I reproduce the problem with duplicate umount?
> >>>
> >>> I was running this 2+ times in a row (*)
> >>>
> >>> ./test/py/test.py --bd sandbox --build
> >
> > (1)
> >
> >>>
> >>> *) I can't run tests right now due to they are more or less constantly broken
> >>> one way or the other, now
> >>>
> >>> ============================================== test session starts ==============================================
> >>> platform linux -- Python 3.9.2, pytest-6.0.2, py-1.10.0, pluggy-0.13.0
> >>> rootdir: /home/andy/prj/u-boot/test/py, configfile: pytest.ini
> >>> collected 810 items / 1 error / 809 selected
> >>>
> >>> ___________________________________ ERROR collecting tests/test_fit_ecdsa.py ____________________________________
> >>> E   ModuleNotFoundError: No module named 'Cryptodome'
> >>
> >> The missing package is available via
> >>
> >>     apt-get install python3-pycryptodome # Debian/Ubuntu
> >>
> >> or
> >>
> >>     dnf install python3-pycryptodomex # Fedora
> >
> > Thanks.
> >
> > So, I have run above mentioned line (1) with current U-Boot (see
> > above), everything is fine, then I have reverted the commit (as your
> > patch does, correct), and oops
> >
> > test/py/tests/test_efi_secboot/test_unsigned.py sss
> >                                    [ 88%]
> > test/py/tests/test_fs/test_basic.py [sudo] password for andy:
>
> If you are asked for a sudo password, you have not install libguestfs.
>
> Please, install the missing package.
>
> > Sorry, try again.
> > [sudo] password for andy:
> > Sorry, try again.
> > [sudo] password for andy:
> > sssssssssssss[sudo] password for andy:
> >
> > Now I'm waiting for a punishment from the admin, thanks to this test round.
>
> make tests (on my local machine)
>
> with origin/master:
>
> test/py/tests/test_efi_secboot/test_unsigned.py ...
> test/py/tests/test_fs/test_basic.py sssssssssssssssssssssssssssssssssssssss
> test/py/tests/test_fs/test_ext.py ssssssssssssssssssssss
> test/py/tests/test_fs/test_fs_cmd.py .
> test/py/tests/test_fs/test_mkdir.py ............
> test/py/tests/test_fs/test_symlink.py ssss
> test/py/tests/test_fs/test_unlink.py ssssssssssssss

Can you provide the reason why those tests were skipped?

> with your patch reverted
>
> test/py/tests/test_efi_secboot/test_unsigned.py ...
> test/py/tests/test_fs/test_basic.py F............F.........................
> test/py/tests/test_fs/test_ext.py ......................
> test/py/tests/test_fs/test_fs_cmd.py .
> test/py/tests/test_fs/test_mkdir.py ............
> test/py/tests/test_fs/test_symlink.py ....
> test/py/tests/test_fs/test_unlink.py ..............
>
> The failures are caused by dd being called with conv=fsync before
> mounting with guestfs.
>
> Obviously we have two scenarios to test separately:
>
> 1) using sudo for mounting
> 2) using guestfs for mounting



> > I'm not going to repeat this again, please understand me correctly.
>
> I assume that you possess a private laptop where your are the admin.

Nope, it's a collaborative server.

> Where is the problem?


-- 
With Best Regards,
Andy Shevchenko

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-17 13:21           ` Heinrich Schuchardt
  2021-05-17 13:25             ` Andy Shevchenko
@ 2021-05-17 13:29             ` Tom Rini
  2021-05-17 14:06               ` Andy Shevchenko
  2021-05-17 14:24             ` Heinrich Schuchardt
  2021-05-20 19:33             ` Alper Nebi Yasak
  3 siblings, 1 reply; 18+ messages in thread
From: Tom Rini @ 2021-05-17 13:29 UTC (permalink / raw)
  To: u-boot

On Mon, May 17, 2021 at 03:21:41PM +0200, Heinrich Schuchardt wrote:
> On 17.05.21 13:44, Andy Shevchenko wrote:
> > On Mon, May 17, 2021 at 2:35 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >>
> >> On 17.05.21 13:16, Andy Shevchenko wrote:
> >>> On Mon, May 17, 2021 at 10:48:33AM +0200, Heinrich Schuchardt wrote:
> >>>> On 17.05.21 08:33, Andy Shevchenko wrote:
> >>>>> On Thu, May 13, 2021 at 2:41 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >>>>>>
> >>>>>> Since commit 1ba21bb06b08 ("test: Don't unmount not (yet) mounted system")
> >>>>>> the following tests are skipped:
> >>>>>>
> >>>>>> test/py/tests/test_fs/test_basic.py
> >>>>>> test/py/tests/test_fs/test_ext.py
> >>>>>>
> >>>>>> SKIPPED [13] test/py/tests/test_fs/conftest.py:350: Setup failed for
> >>>>>> filesystem: ext4. Command 'guestmount -a
> >>>>>> build-sandbox/persistent-data/3GB.ext4.img -m /dev/sda
> >>>>>> build-sandbox/persistent-data/mnt' returned non-zero exit status 1.
> >>>>>>
> >>>>>> Let's revert the patch to get our tests back.
> >>>>>
> >>>>> Probably we may understand first what is the root cause of this issue?
> >>>>>
> >>>>> In my case I can't allow this to happen, because it will annoy system
> >>>>> administrators as I mentioned earlier in the commit message.
> >>>>>
> >>>>> So, NAK from me and let's investigate.
> >>>>> Can you provide a command line that I may run on my environment w/o root access?
> >>>>
> >>>> Hello Andy,
> >>>>
> >>>> The tests don't require root access if you have installed the
> >>>> libguestfs-tools package and a Linux kernel.
> >>>>
> >>>> How can I reproduce the problem with duplicate umount?
> >>>
> >>> I was running this 2+ times in a row (*)
> >>>
> >>> ./test/py/test.py --bd sandbox --build
> >
> > (1)
> >
> >>>
> >>> *) I can't run tests right now due to they are more or less constantly broken
> >>> one way or the other, now
> >>>
> >>> ============================================== test session starts ==============================================
> >>> platform linux -- Python 3.9.2, pytest-6.0.2, py-1.10.0, pluggy-0.13.0
> >>> rootdir: /home/andy/prj/u-boot/test/py, configfile: pytest.ini
> >>> collected 810 items / 1 error / 809 selected
> >>>
> >>> ___________________________________ ERROR collecting tests/test_fit_ecdsa.py ____________________________________
> >>> E   ModuleNotFoundError: No module named 'Cryptodome'
> >>
> >> The missing package is available via
> >>
> >>     apt-get install python3-pycryptodome # Debian/Ubuntu
> >>
> >> or
> >>
> >>     dnf install python3-pycryptodomex # Fedora
> >
> > Thanks.
> >
> > So, I have run above mentioned line (1) with current U-Boot (see
> > above), everything is fine, then I have reverted the commit (as your
> > patch does, correct), and oops
> >
> > test/py/tests/test_efi_secboot/test_unsigned.py sss
> >                                    [ 88%]
> > test/py/tests/test_fs/test_basic.py [sudo] password for andy:
> 
> If you are asked for a sudo password, you have not install libguestfs.
> 
> Please, install the missing package.
> 
> > Sorry, try again.
> > [sudo] password for andy:
> > Sorry, try again.
> > [sudo] password for andy:
> > sssssssssssss[sudo] password for andy:
> >
> > Now I'm waiting for a punishment from the admin, thanks to this test round.
> 
> make tests (on my local machine)
> 
> with origin/master:
> 
> test/py/tests/test_efi_secboot/test_unsigned.py ...
> test/py/tests/test_fs/test_basic.py sssssssssssssssssssssssssssssssssssssss
> test/py/tests/test_fs/test_ext.py ssssssssssssssssssssss
> test/py/tests/test_fs/test_fs_cmd.py .
> test/py/tests/test_fs/test_mkdir.py ............
> test/py/tests/test_fs/test_symlink.py ssss
> test/py/tests/test_fs/test_unlink.py ssssssssssssss
> 
> with your patch reverted
> 
> test/py/tests/test_efi_secboot/test_unsigned.py ...
> test/py/tests/test_fs/test_basic.py F............F.........................
> test/py/tests/test_fs/test_ext.py ......................
> test/py/tests/test_fs/test_fs_cmd.py .
> test/py/tests/test_fs/test_mkdir.py ............
> test/py/tests/test_fs/test_symlink.py ....
> test/py/tests/test_fs/test_unlink.py ..............
> 
> The failures are caused by dd being called with conv=fsync before
> mounting with guestfs.
> 
> Obviously we have two scenarios to test separately:
> 
> 1) using sudo for mounting
> 2) using guestfs for mounting
> 
> >
> > I'm not going to repeat this again, please understand me correctly.
> 
> I assume that you possess a private laptop where your are the admin.
> Where is the problem?

The problem here is that we have a number of different development /
testing scenarios that we need to support.  Tests need to run in CI.
They need to run in developer-controlled machines.  They need to run in
corporate-IT controlled machines.

The next problem is that as Andy has said, our python logic to handle
these cases is, to be polite, not working.  Check CI, we're skipping the
tests again, which I had missed.

Once we get these the fs tests working finally in all cases, we need to
update the relevant EFI tests too.

Finally, my python skills are not up to getting all of this correct, so
help greatly appreciated here.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20210517/e7c8956d/attachment.sig>

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-17 13:29             ` Tom Rini
@ 2021-05-17 14:06               ` Andy Shevchenko
  2021-05-17 17:08                 ` Heinrich Schuchardt
  2021-05-20 19:21                 ` Alper Nebi Yasak
  0 siblings, 2 replies; 18+ messages in thread
From: Andy Shevchenko @ 2021-05-17 14:06 UTC (permalink / raw)
  To: u-boot

On Mon, May 17, 2021 at 4:29 PM Tom Rini <trini@konsulko.com> wrote:
> On Mon, May 17, 2021 at 03:21:41PM +0200, Heinrich Schuchardt wrote:
> > On 17.05.21 13:44, Andy Shevchenko wrote:
> > > On Mon, May 17, 2021 at 2:35 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> > >>
> > >> On 17.05.21 13:16, Andy Shevchenko wrote:
> > >>> On Mon, May 17, 2021 at 10:48:33AM +0200, Heinrich Schuchardt wrote:
> > >>>> On 17.05.21 08:33, Andy Shevchenko wrote:
> > >>>>> On Thu, May 13, 2021 at 2:41 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> > >>>>>>
> > >>>>>> Since commit 1ba21bb06b08 ("test: Don't unmount not (yet) mounted system")
> > >>>>>> the following tests are skipped:
> > >>>>>>
> > >>>>>> test/py/tests/test_fs/test_basic.py
> > >>>>>> test/py/tests/test_fs/test_ext.py
> > >>>>>>
> > >>>>>> SKIPPED [13] test/py/tests/test_fs/conftest.py:350: Setup failed for
> > >>>>>> filesystem: ext4. Command 'guestmount -a
> > >>>>>> build-sandbox/persistent-data/3GB.ext4.img -m /dev/sda
> > >>>>>> build-sandbox/persistent-data/mnt' returned non-zero exit status 1.
> > >>>>>>
> > >>>>>> Let's revert the patch to get our tests back.
> > >>>>>
> > >>>>> Probably we may understand first what is the root cause of this issue?
> > >>>>>
> > >>>>> In my case I can't allow this to happen, because it will annoy system
> > >>>>> administrators as I mentioned earlier in the commit message.
> > >>>>>
> > >>>>> So, NAK from me and let's investigate.
> > >>>>> Can you provide a command line that I may run on my environment w/o root access?
> > >>>>
> > >>>> Hello Andy,
> > >>>>
> > >>>> The tests don't require root access if you have installed the
> > >>>> libguestfs-tools package and a Linux kernel.
> > >>>>
> > >>>> How can I reproduce the problem with duplicate umount?
> > >>>
> > >>> I was running this 2+ times in a row (*)
> > >>>
> > >>> ./test/py/test.py --bd sandbox --build
> > >
> > > (1)
> > >
> > >>>
> > >>> *) I can't run tests right now due to they are more or less constantly broken
> > >>> one way or the other, now
> > >>>
> > >>> ============================================== test session starts ==============================================
> > >>> platform linux -- Python 3.9.2, pytest-6.0.2, py-1.10.0, pluggy-0.13.0
> > >>> rootdir: /home/andy/prj/u-boot/test/py, configfile: pytest.ini
> > >>> collected 810 items / 1 error / 809 selected
> > >>>
> > >>> ___________________________________ ERROR collecting tests/test_fit_ecdsa.py ____________________________________
> > >>> E   ModuleNotFoundError: No module named 'Cryptodome'
> > >>
> > >> The missing package is available via
> > >>
> > >>     apt-get install python3-pycryptodome # Debian/Ubuntu
> > >>
> > >> or
> > >>
> > >>     dnf install python3-pycryptodomex # Fedora
> > >
> > > Thanks.
> > >
> > > So, I have run above mentioned line (1) with current U-Boot (see
> > > above), everything is fine, then I have reverted the commit (as your
> > > patch does, correct), and oops
> > >
> > > test/py/tests/test_efi_secboot/test_unsigned.py sss
> > >                                    [ 88%]
> > > test/py/tests/test_fs/test_basic.py [sudo] password for andy:
> >
> > If you are asked for a sudo password, you have not install libguestfs.
> >
> > Please, install the missing package.
> >
> > > Sorry, try again.
> > > [sudo] password for andy:
> > > Sorry, try again.
> > > [sudo] password for andy:
> > > sssssssssssss[sudo] password for andy:
> > >
> > > Now I'm waiting for a punishment from the admin, thanks to this test round.
> >
> > make tests (on my local machine)
> >
> > with origin/master:
> >
> > test/py/tests/test_efi_secboot/test_unsigned.py ...
> > test/py/tests/test_fs/test_basic.py sssssssssssssssssssssssssssssssssssssss
> > test/py/tests/test_fs/test_ext.py ssssssssssssssssssssss
> > test/py/tests/test_fs/test_fs_cmd.py .
> > test/py/tests/test_fs/test_mkdir.py ............
> > test/py/tests/test_fs/test_symlink.py ssss
> > test/py/tests/test_fs/test_unlink.py ssssssssssssss
> >
> > with your patch reverted
> >
> > test/py/tests/test_efi_secboot/test_unsigned.py ...
> > test/py/tests/test_fs/test_basic.py F............F.........................
> > test/py/tests/test_fs/test_ext.py ......................
> > test/py/tests/test_fs/test_fs_cmd.py .
> > test/py/tests/test_fs/test_mkdir.py ............
> > test/py/tests/test_fs/test_symlink.py ....
> > test/py/tests/test_fs/test_unlink.py ..............
> >
> > The failures are caused by dd being called with conv=fsync before
> > mounting with guestfs.
> >
> > Obviously we have two scenarios to test separately:
> >
> > 1) using sudo for mounting
> > 2) using guestfs for mounting
> >
> > >
> > > I'm not going to repeat this again, please understand me correctly.
> >
> > I assume that you possess a private laptop where your are the admin.
> > Where is the problem?
>
> The problem here is that we have a number of different development /
> testing scenarios that we need to support.  Tests need to run in CI.
> They need to run in developer-controlled machines.  They need to run in
> corporate-IT controlled machines.
>
> The next problem is that as Andy has said, our python logic to handle
> these cases is, to be polite, not working.  Check CI, we're skipping the
> tests again, which I had missed.
>
> Once we get these the fs tests working finally in all cases, we need to
> update the relevant EFI tests too.
>
> Finally, my python skills are not up to getting all of this correct, so
> help greatly appreciated here.

Thanks, Tom, for summarizing it.

I would like to be helpful here and when I have time, I'll look at it
closer if nobody beats me up to it. Currently I checked the reason why
we skip them in my scenario:
============================================ short test summary info
============================================
SKIPPED [13] test/py/tests/test_fs/conftest.py:274: Creating failed
for filesystem: fat16. Command 'mkfs.vfat -F
16 /home/andy/prj/u-boot/build-sandbox/persistent-data/3GB.fat16.img'
returned non-zero exit status 127.
SKIPPED [13] test/py/tests/test_fs/conftest.py:274: Creating failed
for filesystem: fat32. Command 'mkfs.vfat -F
32 /home/andy/prj/u-boot/build-sandbox/persistent-data/3GB.fat32.img'
returned non-zero exit status 127.
SKIPPED [13] test/py/tests/test_fs/conftest.py:274: Creating failed
for filesystem: ext4. Command 'mkfs.ext4  /ho
me/andy/prj/u-boot/build-sandbox/persistent-data/3GB.ext4.img'
returned non-zero exit status 127.
SKIPPED [11] test/py/tests/test_fs/conftest.py:390: Creating failed
for filesystem: fat16. Command 'mkfs.vfat -F
16 /home/andy/prj/u-boot/build-sandbox/persistent-data/128MB.fat16.img'
returned non-zero exit status 127.
SKIPPED [11] test/py/tests/test_fs/conftest.py:390: Creating failed
for filesystem: fat32. Command 'mkfs.vfat -F
32 /home/andy/prj/u-boot/build-sandbox/persistent-data/128MB.fat32.img'
returned non-zero exit status 127.
SKIPPED [6] test/py/tests/test_fs/conftest.py:479: Setup failed for
filesystem: fat16
SKIPPED [6] test/py/tests/test_fs/conftest.py:479: Setup failed for
filesystem: fat32
SKIPPED [4] test/py/tests/test_fs/conftest.py:590: Creating failed for
filesystem: ext4. Command 'mkfs.ext4  /hom
e/andy/prj/u-boot/build-sandbox/persistent-data/1GB.ext4.img' returned
non-zero exit status 127.
SKIPPED [7] test/py/tests/test_fs/conftest.py:513: Creating failed for
filesystem: fat16. Command 'mkfs.vfat -F 1
6 /home/andy/prj/u-boot/build-sandbox/persistent-data/128MB.fat16.img'
returned non-zero exit status 127.
SKIPPED [7] test/py/tests/test_fs/conftest.py:513: Creating failed for
filesystem: fat32. Command 'mkfs.vfat -F 3
2 /home/andy/prj/u-boot/build-sandbox/persistent-data/128MB.fat32.img'
returned non-zero exit status 127.
======================================== 3 passed, 91 skipped in
10.02s =========================================



-- 
With Best Regards,
Andy Shevchenko

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-17 13:21           ` Heinrich Schuchardt
  2021-05-17 13:25             ` Andy Shevchenko
  2021-05-17 13:29             ` Tom Rini
@ 2021-05-17 14:24             ` Heinrich Schuchardt
  2021-05-17 14:43               ` Tom Rini
  2021-05-20 19:33             ` Alper Nebi Yasak
  3 siblings, 1 reply; 18+ messages in thread
From: Heinrich Schuchardt @ 2021-05-17 14:24 UTC (permalink / raw)
  To: u-boot

On 17.05.21 15:21, Heinrich Schuchardt wrote:
> On 17.05.21 13:44, Andy Shevchenko wrote:
>> On Mon, May 17, 2021 at 2:35 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>>
>>> On 17.05.21 13:16, Andy Shevchenko wrote:
>>>> On Mon, May 17, 2021 at 10:48:33AM +0200, Heinrich Schuchardt wrote:
>>>>> On 17.05.21 08:33, Andy Shevchenko wrote:
>>>>>> On Thu, May 13, 2021 at 2:41 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>>>>>>
>>>>>>> Since commit 1ba21bb06b08 ("test: Don't unmount not (yet) mounted system")
>>>>>>> the following tests are skipped:
>>>>>>>
>>>>>>> test/py/tests/test_fs/test_basic.py
>>>>>>> test/py/tests/test_fs/test_ext.py
>>>>>>>
>>>>>>> SKIPPED [13] test/py/tests/test_fs/conftest.py:350: Setup failed for
>>>>>>> filesystem: ext4. Command 'guestmount -a
>>>>>>> build-sandbox/persistent-data/3GB.ext4.img -m /dev/sda
>>>>>>> build-sandbox/persistent-data/mnt' returned non-zero exit status 1.
>>>>>>>
>>>>>>> Let's revert the patch to get our tests back.
>>>>>>
>>>>>> Probably we may understand first what is the root cause of this issue?
>>>>>>
>>>>>> In my case I can't allow this to happen, because it will annoy system
>>>>>> administrators as I mentioned earlier in the commit message.
>>>>>>
>>>>>> So, NAK from me and let's investigate.
>>>>>> Can you provide a command line that I may run on my environment w/o root access?
>>>>>
>>>>> Hello Andy,
>>>>>
>>>>> The tests don't require root access if you have installed the
>>>>> libguestfs-tools package and a Linux kernel.
>>>>>
>>>>> How can I reproduce the problem with duplicate umount?
>>>>
>>>> I was running this 2+ times in a row (*)
>>>>
>>>> ./test/py/test.py --bd sandbox --build
>>
>> (1)
>>
>>>>
>>>> *) I can't run tests right now due to they are more or less constantly broken
>>>> one way or the other, now
>>>>
>>>> ============================================== test session starts ==============================================
>>>> platform linux -- Python 3.9.2, pytest-6.0.2, py-1.10.0, pluggy-0.13.0
>>>> rootdir: /home/andy/prj/u-boot/test/py, configfile: pytest.ini
>>>> collected 810 items / 1 error / 809 selected
>>>>
>>>> ___________________________________ ERROR collecting tests/test_fit_ecdsa.py ____________________________________
>>>> E   ModuleNotFoundError: No module named 'Cryptodome'
>>>
>>> The missing package is available via
>>>
>>>     apt-get install python3-pycryptodome # Debian/Ubuntu
>>>
>>> or
>>>
>>>     dnf install python3-pycryptodomex # Fedora
>>
>> Thanks.
>>
>> So, I have run above mentioned line (1) with current U-Boot (see
>> above), everything is fine, then I have reverted the commit (as your
>> patch does, correct), and oops
>>
>> test/py/tests/test_efi_secboot/test_unsigned.py sss
>>                                    [ 88%]
>> test/py/tests/test_fs/test_basic.py [sudo] password for andy:
>
> If you are asked for a sudo password, you have not install libguestfs.
>
> Please, install the missing package.
>
>> Sorry, try again.
>> [sudo] password for andy:
>> Sorry, try again.
>> [sudo] password for andy:
>> sssssssssssss[sudo] password for andy:
>>
>> Now I'm waiting for a punishment from the admin, thanks to this test round.
>
> make tests (on my local machine)
>
> with origin/master:
>
> test/py/tests/test_efi_secboot/test_unsigned.py ...
> test/py/tests/test_fs/test_basic.py sssssssssssssssssssssssssssssssssssssss
> test/py/tests/test_fs/test_ext.py ssssssssssssssssssssss
> test/py/tests/test_fs/test_fs_cmd.py .
> test/py/tests/test_fs/test_mkdir.py ............
> test/py/tests/test_fs/test_symlink.py ssss
> test/py/tests/test_fs/test_unlink.py ssssssssssssss
>
> with your patch reverted
>
> test/py/tests/test_efi_secboot/test_unsigned.py ...
> test/py/tests/test_fs/test_basic.py F............F.........................
> test/py/tests/test_fs/test_ext.py ......................
> test/py/tests/test_fs/test_fs_cmd.py .
> test/py/tests/test_fs/test_mkdir.py ............
> test/py/tests/test_fs/test_symlink.py ....
> test/py/tests/test_fs/test_unlink.py ..............
>
> The failures are caused by dd being called with conv=fsync before
> mounting with guestfs.
>
> Obviously we have two scenarios to test separately:
>
> 1) using sudo for mounting
> 2) using guestfs for mounting
>
>>
>> I'm not going to repeat this again, please understand me correctly.
>
> I assume that you possess a private laptop where your are the admin.
> Where is the problem?
>
> Best regards
>
> Heinrich
>

Here are the test results with guestfs-tools removed:

with origin/master

test/py/tests/test_efi_secboot/test_unsigned.py sss
test/py/tests/test_fs/test_basic.py [sudo] password for user:
sssssssssssssssssssssssssssssssssssssss
test/py/tests/test_fs/test_ext.py ssssssssssssssssssssss
test/py/tests/test_fs/test_fs_cmd.py .
test/py/tests/test_fs/test_mkdir.py ............
test/py/tests/test_fs/test_symlink.py ssss
test/py/tests/test_fs/test_unlink.py ssssssssssssss

with Andy's patch reverted

test/py/tests/test_efi_secboot/test_unsigned.py sss
test/py/tests/test_fs/test_basic.py [sudo] password for user:
.......................................
test/py/tests/test_fs/test_ext.py ......................
test/py/tests/test_fs/test_fs_cmd.py .
test/py/tests/test_fs/test_mkdir.py ............
test/py/tests/test_fs/test_symlink.py ....
test/py/tests/test_fs/test_unlink.py ..............

So the only effect of Andy's patch is to cause skipping tests. As a
secondary effect this may have led to less log entries.

Best regards

Heinrich

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-17 14:24             ` Heinrich Schuchardt
@ 2021-05-17 14:43               ` Tom Rini
  0 siblings, 0 replies; 18+ messages in thread
From: Tom Rini @ 2021-05-17 14:43 UTC (permalink / raw)
  To: u-boot

On Mon, May 17, 2021 at 04:24:46PM +0200, Heinrich Schuchardt wrote:
> On 17.05.21 15:21, Heinrich Schuchardt wrote:
> > On 17.05.21 13:44, Andy Shevchenko wrote:
> >> On Mon, May 17, 2021 at 2:35 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >>>
> >>> On 17.05.21 13:16, Andy Shevchenko wrote:
> >>>> On Mon, May 17, 2021 at 10:48:33AM +0200, Heinrich Schuchardt wrote:
> >>>>> On 17.05.21 08:33, Andy Shevchenko wrote:
> >>>>>> On Thu, May 13, 2021 at 2:41 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >>>>>>>
> >>>>>>> Since commit 1ba21bb06b08 ("test: Don't unmount not (yet) mounted system")
> >>>>>>> the following tests are skipped:
> >>>>>>>
> >>>>>>> test/py/tests/test_fs/test_basic.py
> >>>>>>> test/py/tests/test_fs/test_ext.py
> >>>>>>>
> >>>>>>> SKIPPED [13] test/py/tests/test_fs/conftest.py:350: Setup failed for
> >>>>>>> filesystem: ext4. Command 'guestmount -a
> >>>>>>> build-sandbox/persistent-data/3GB.ext4.img -m /dev/sda
> >>>>>>> build-sandbox/persistent-data/mnt' returned non-zero exit status 1.
> >>>>>>>
> >>>>>>> Let's revert the patch to get our tests back.
> >>>>>>
> >>>>>> Probably we may understand first what is the root cause of this issue?
> >>>>>>
> >>>>>> In my case I can't allow this to happen, because it will annoy system
> >>>>>> administrators as I mentioned earlier in the commit message.
> >>>>>>
> >>>>>> So, NAK from me and let's investigate.
> >>>>>> Can you provide a command line that I may run on my environment w/o root access?
> >>>>>
> >>>>> Hello Andy,
> >>>>>
> >>>>> The tests don't require root access if you have installed the
> >>>>> libguestfs-tools package and a Linux kernel.
> >>>>>
> >>>>> How can I reproduce the problem with duplicate umount?
> >>>>
> >>>> I was running this 2+ times in a row (*)
> >>>>
> >>>> ./test/py/test.py --bd sandbox --build
> >>
> >> (1)
> >>
> >>>>
> >>>> *) I can't run tests right now due to they are more or less constantly broken
> >>>> one way or the other, now
> >>>>
> >>>> ============================================== test session starts ==============================================
> >>>> platform linux -- Python 3.9.2, pytest-6.0.2, py-1.10.0, pluggy-0.13.0
> >>>> rootdir: /home/andy/prj/u-boot/test/py, configfile: pytest.ini
> >>>> collected 810 items / 1 error / 809 selected
> >>>>
> >>>> ___________________________________ ERROR collecting tests/test_fit_ecdsa.py ____________________________________
> >>>> E   ModuleNotFoundError: No module named 'Cryptodome'
> >>>
> >>> The missing package is available via
> >>>
> >>>     apt-get install python3-pycryptodome # Debian/Ubuntu
> >>>
> >>> or
> >>>
> >>>     dnf install python3-pycryptodomex # Fedora
> >>
> >> Thanks.
> >>
> >> So, I have run above mentioned line (1) with current U-Boot (see
> >> above), everything is fine, then I have reverted the commit (as your
> >> patch does, correct), and oops
> >>
> >> test/py/tests/test_efi_secboot/test_unsigned.py sss
> >>                                    [ 88%]
> >> test/py/tests/test_fs/test_basic.py [sudo] password for andy:
> >
> > If you are asked for a sudo password, you have not install libguestfs.
> >
> > Please, install the missing package.
> >
> >> Sorry, try again.
> >> [sudo] password for andy:
> >> Sorry, try again.
> >> [sudo] password for andy:
> >> sssssssssssss[sudo] password for andy:
> >>
> >> Now I'm waiting for a punishment from the admin, thanks to this test round.
> >
> > make tests (on my local machine)
> >
> > with origin/master:
> >
> > test/py/tests/test_efi_secboot/test_unsigned.py ...
> > test/py/tests/test_fs/test_basic.py sssssssssssssssssssssssssssssssssssssss
> > test/py/tests/test_fs/test_ext.py ssssssssssssssssssssss
> > test/py/tests/test_fs/test_fs_cmd.py .
> > test/py/tests/test_fs/test_mkdir.py ............
> > test/py/tests/test_fs/test_symlink.py ssss
> > test/py/tests/test_fs/test_unlink.py ssssssssssssss
> >
> > with your patch reverted
> >
> > test/py/tests/test_efi_secboot/test_unsigned.py ...
> > test/py/tests/test_fs/test_basic.py F............F.........................
> > test/py/tests/test_fs/test_ext.py ......................
> > test/py/tests/test_fs/test_fs_cmd.py .
> > test/py/tests/test_fs/test_mkdir.py ............
> > test/py/tests/test_fs/test_symlink.py ....
> > test/py/tests/test_fs/test_unlink.py ..............
> >
> > The failures are caused by dd being called with conv=fsync before
> > mounting with guestfs.
> >
> > Obviously we have two scenarios to test separately:
> >
> > 1) using sudo for mounting
> > 2) using guestfs for mounting
> >
> >>
> >> I'm not going to repeat this again, please understand me correctly.
> >
> > I assume that you possess a private laptop where your are the admin.
> > Where is the problem?
> >
> > Best regards
> >
> > Heinrich
> >
> 
> Here are the test results with guestfs-tools removed:
> 
> with origin/master
> 
> test/py/tests/test_efi_secboot/test_unsigned.py sss
> test/py/tests/test_fs/test_basic.py [sudo] password for user:
> sssssssssssssssssssssssssssssssssssssss

Which means it runs for nopassword groups in sudo, which can be a
reasonable thing to do in some cases / machine configurations.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20210517/9b2d94e9/attachment.sig>

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-17 14:06               ` Andy Shevchenko
@ 2021-05-17 17:08                 ` Heinrich Schuchardt
  2021-05-17 17:57                   ` Andy Shevchenko
  2021-05-20 19:21                 ` Alper Nebi Yasak
  1 sibling, 1 reply; 18+ messages in thread
From: Heinrich Schuchardt @ 2021-05-17 17:08 UTC (permalink / raw)
  To: u-boot

On 17.05.21 16:06, Andy Shevchenko wrote:
> On Mon, May 17, 2021 at 4:29 PM Tom Rini <trini@konsulko.com> wrote:
>> On Mon, May 17, 2021 at 03:21:41PM +0200, Heinrich Schuchardt wrote:
>>> On 17.05.21 13:44, Andy Shevchenko wrote:
>>>> On Mon, May 17, 2021 at 2:35 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>>>>
>>>>> On 17.05.21 13:16, Andy Shevchenko wrote:
>>>>>> On Mon, May 17, 2021 at 10:48:33AM +0200, Heinrich Schuchardt wrote:
>>>>>>> On 17.05.21 08:33, Andy Shevchenko wrote:
>>>>>>>> On Thu, May 13, 2021 at 2:41 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>>>>>>>>
>>>>>>>>> Since commit 1ba21bb06b08 ("test: Don't unmount not (yet) mounted system")
>>>>>>>>> the following tests are skipped:
>>>>>>>>>
>>>>>>>>> test/py/tests/test_fs/test_basic.py
>>>>>>>>> test/py/tests/test_fs/test_ext.py
>>>>>>>>>
>>>>>>>>> SKIPPED [13] test/py/tests/test_fs/conftest.py:350: Setup failed for
>>>>>>>>> filesystem: ext4. Command 'guestmount -a
>>>>>>>>> build-sandbox/persistent-data/3GB.ext4.img -m /dev/sda
>>>>>>>>> build-sandbox/persistent-data/mnt' returned non-zero exit status 1.
>>>>>>>>>
>>>>>>>>> Let's revert the patch to get our tests back.
>>>>>>>>
>>>>>>>> Probably we may understand first what is the root cause of this issue?
>>>>>>>>
>>>>>>>> In my case I can't allow this to happen, because it will annoy system
>>>>>>>> administrators as I mentioned earlier in the commit message.
>>>>>>>>
>>>>>>>> So, NAK from me and let's investigate.
>>>>>>>> Can you provide a command line that I may run on my environment w/o root access?
>>>>>>>
>>>>>>> Hello Andy,
>>>>>>>
>>>>>>> The tests don't require root access if you have installed the
>>>>>>> libguestfs-tools package and a Linux kernel.
>>>>>>>
>>>>>>> How can I reproduce the problem with duplicate umount?
>>>>>>
>>>>>> I was running this 2+ times in a row (*)
>>>>>>
>>>>>> ./test/py/test.py --bd sandbox --build
>>>>
>>>> (1)
>>>>
>>>>>>
>>>>>> *) I can't run tests right now due to they are more or less constantly broken
>>>>>> one way or the other, now
>>>>>>
>>>>>> ============================================== test session starts ==============================================
>>>>>> platform linux -- Python 3.9.2, pytest-6.0.2, py-1.10.0, pluggy-0.13.0
>>>>>> rootdir: /home/andy/prj/u-boot/test/py, configfile: pytest.ini
>>>>>> collected 810 items / 1 error / 809 selected
>>>>>>
>>>>>> ___________________________________ ERROR collecting tests/test_fit_ecdsa.py ____________________________________
>>>>>> E   ModuleNotFoundError: No module named 'Cryptodome'
>>>>>
>>>>> The missing package is available via
>>>>>
>>>>>     apt-get install python3-pycryptodome # Debian/Ubuntu
>>>>>
>>>>> or
>>>>>
>>>>>     dnf install python3-pycryptodomex # Fedora
>>>>
>>>> Thanks.
>>>>
>>>> So, I have run above mentioned line (1) with current U-Boot (see
>>>> above), everything is fine, then I have reverted the commit (as your
>>>> patch does, correct), and oops
>>>>
>>>> test/py/tests/test_efi_secboot/test_unsigned.py sss
>>>>                                    [ 88%]
>>>> test/py/tests/test_fs/test_basic.py [sudo] password for andy:
>>>
>>> If you are asked for a sudo password, you have not install libguestfs.
>>>
>>> Please, install the missing package.
>>>
>>>> Sorry, try again.
>>>> [sudo] password for andy:
>>>> Sorry, try again.
>>>> [sudo] password for andy:
>>>> sssssssssssss[sudo] password for andy:
>>>>
>>>> Now I'm waiting for a punishment from the admin, thanks to this test round.
>>>
>>> make tests (on my local machine)
>>>
>>> with origin/master:
>>>
>>> test/py/tests/test_efi_secboot/test_unsigned.py ...
>>> test/py/tests/test_fs/test_basic.py sssssssssssssssssssssssssssssssssssssss
>>> test/py/tests/test_fs/test_ext.py ssssssssssssssssssssss
>>> test/py/tests/test_fs/test_fs_cmd.py .
>>> test/py/tests/test_fs/test_mkdir.py ............
>>> test/py/tests/test_fs/test_symlink.py ssss
>>> test/py/tests/test_fs/test_unlink.py ssssssssssssss
>>>
>>> with your patch reverted
>>>
>>> test/py/tests/test_efi_secboot/test_unsigned.py ...
>>> test/py/tests/test_fs/test_basic.py F............F.........................
>>> test/py/tests/test_fs/test_ext.py ......................
>>> test/py/tests/test_fs/test_fs_cmd.py .
>>> test/py/tests/test_fs/test_mkdir.py ............
>>> test/py/tests/test_fs/test_symlink.py ....
>>> test/py/tests/test_fs/test_unlink.py ..............
>>>
>>> The failures are caused by dd being called with conv=fsync before
>>> mounting with guestfs.
>>>
>>> Obviously we have two scenarios to test separately:
>>>
>>> 1) using sudo for mounting
>>> 2) using guestfs for mounting
>>>
>>>>
>>>> I'm not going to repeat this again, please understand me correctly.
>>>
>>> I assume that you possess a private laptop where your are the admin.
>>> Where is the problem?
>>
>> The problem here is that we have a number of different development /
>> testing scenarios that we need to support.  Tests need to run in CI.
>> They need to run in developer-controlled machines.  They need to run in
>> corporate-IT controlled machines.
>>
>> The next problem is that as Andy has said, our python logic to handle
>> these cases is, to be polite, not working.  Check CI, we're skipping the
>> tests again, which I had missed.
>>
>> Once we get these the fs tests working finally in all cases, we need to
>> update the relevant EFI tests too.
>>
>> Finally, my python skills are not up to getting all of this correct, so
>> help greatly appreciated here.
>
> Thanks, Tom, for summarizing it.
>
> I would like to be helpful here and when I have time, I'll look at it
> closer if nobody beats me up to it. Currently I checked the reason why
> we skip them in my scenario:
> ============================================ short test summary info
> ============================================
> SKIPPED [13] test/py/tests/test_fs/conftest.py:274: Creating failed
> for filesystem: fat16. Command 'mkfs.vfat -F
> 16 /home/andy/prj/u-boot/build-sandbox/persistent-data/3GB.fat16.img'
> returned non-zero exit status 127.
> SKIPPED [13] test/py/tests/test_fs/conftest.py:274: Creating failed
> for filesystem: fat32. Command 'mkfs.vfat -F
> 32 /home/andy/prj/u-boot/build-sandbox/persistent-data/3GB.fat32.img'
> returned non-zero exit status 127.
> SKIPPED [13] test/py/tests/test_fs/conftest.py:274: Creating failed
> for filesystem: ext4. Command 'mkfs.ext4  /ho
> me/andy/prj/u-boot/build-sandbox/persistent-data/3GB.ext4.img'
> returned non-zero exit status 127.
> SKIPPED [11] test/py/tests/test_fs/conftest.py:390: Creating failed
> for filesystem: fat16. Command 'mkfs.vfat -F
> 16 /home/andy/prj/u-boot/build-sandbox/persistent-data/128MB.fat16.img'
> returned non-zero exit status 127.
> SKIPPED [11] test/py/tests/test_fs/conftest.py:390: Creating failed
> for filesystem: fat32. Command 'mkfs.vfat -F
> 32 /home/andy/prj/u-boot/build-sandbox/persistent-data/128MB.fat32.img'
> returned non-zero exit status 127.
> SKIPPED [6] test/py/tests/test_fs/conftest.py:479: Setup failed for
> filesystem: fat16
> SKIPPED [6] test/py/tests/test_fs/conftest.py:479: Setup failed for
> filesystem: fat32
> SKIPPED [4] test/py/tests/test_fs/conftest.py:590: Creating failed for
> filesystem: ext4. Command 'mkfs.ext4  /hom
> e/andy/prj/u-boot/build-sandbox/persistent-data/1GB.ext4.img' returned
> non-zero exit status 127.
> SKIPPED [7] test/py/tests/test_fs/conftest.py:513: Creating failed for
> filesystem: fat16. Command 'mkfs.vfat -F 1
> 6 /home/andy/prj/u-boot/build-sandbox/persistent-data/128MB.fat16.img'
> returned non-zero exit status 127.
> SKIPPED [7] test/py/tests/test_fs/conftest.py:513: Creating failed for
> filesystem: fat32. Command 'mkfs.vfat -F 3
> 2 /home/andy/prj/u-boot/build-sandbox/persistent-data/128MB.fat32.img'
> returned non-zero exit status 127.
> ======================================== 3 passed, 91 skipped in
> 10.02s =========================================
>
>
>

Let's look at the code without your patch:

We have multiple functions ending with:

    umount_fs(mount_dir)
except CalledProcessError as err:
    pytest.skip('Setup failed for filesystem: ' + fs_type + \
        '. {}'.format(err))
    return
else:
    yield [fs_ubtype, fs_img, md5val]
finally:
    umount_fs(mount_dir)
    call('rmdir %s' % mount_dir, shell=True)
    if fs_img:
        call('rm -f %s' % fs_img, shell=True)


If no exception occurs, unmount_fs() is called twice.

Both the unmount_fs() before except and the return statement should be
removed. Then umount_fs() will be called only in the finally branch.

I hope this change to the original code is enough to solve your problem.

I will look into creating a patch.

Best regards

Heinrich

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-17 17:08                 ` Heinrich Schuchardt
@ 2021-05-17 17:57                   ` Andy Shevchenko
  2021-05-17 18:01                     ` Andy Shevchenko
  0 siblings, 1 reply; 18+ messages in thread
From: Andy Shevchenko @ 2021-05-17 17:57 UTC (permalink / raw)
  To: u-boot

On Mon, May 17, 2021 at 8:08 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> On 17.05.21 16:06, Andy Shevchenko wrote:

...

> Let's look at the code without your patch:
>
> We have multiple functions ending with:
>
>     umount_fs(mount_dir)
> except CalledProcessError as err:
>     pytest.skip('Setup failed for filesystem: ' + fs_type + \
>         '. {}'.format(err))
>     return
> else:
>     yield [fs_ubtype, fs_img, md5val]
> finally:
>     umount_fs(mount_dir)
>     call('rmdir %s' % mount_dir, shell=True)
>     if fs_img:
>         call('rm -f %s' % fs_img, shell=True)
>
>
> If no exception occurs, unmount_fs() is called twice.
>
> Both the unmount_fs() before except and the return statement should be
> removed. Then umount_fs() will be called only in the finally branch.
>
> I hope this change to the original code is enough to solve your problem.

Probably, but as I mentioned in the commit message that Pythonic
exceptions are completely over engineered that no-one can get them. I
think you missed the fact that the exception can happen at any time
when FS is mounted and when it's not mounted yet. I suggest you think
more of this code... (Yet it admits that I may have made mistake
myself, see above)

> I will look into creating a patch.



-- 
With Best Regards,
Andy Shevchenko

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-17 17:57                   ` Andy Shevchenko
@ 2021-05-17 18:01                     ` Andy Shevchenko
  0 siblings, 0 replies; 18+ messages in thread
From: Andy Shevchenko @ 2021-05-17 18:01 UTC (permalink / raw)
  To: u-boot

On Mon, May 17, 2021 at 8:57 PM Andy Shevchenko
<andy.shevchenko@gmail.com> wrote:
>
> On Mon, May 17, 2021 at 8:08 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> > On 17.05.21 16:06, Andy Shevchenko wrote:
>
> ...
>
> > Let's look at the code without your patch:
> >
> > We have multiple functions ending with:
> >
> >     umount_fs(mount_dir)
> > except CalledProcessError as err:
> >     pytest.skip('Setup failed for filesystem: ' + fs_type + \
> >         '. {}'.format(err))
> >     return
> > else:
> >     yield [fs_ubtype, fs_img, md5val]
> > finally:
> >     umount_fs(mount_dir)
> >     call('rmdir %s' % mount_dir, shell=True)
> >     if fs_img:
> >         call('rm -f %s' % fs_img, shell=True)
> >
> >
> > If no exception occurs, unmount_fs() is called twice.
> >
> > Both the unmount_fs() before except and the return statement should be
> > removed.

No, AFAIU the return must be there, otherwise you will go after that
without properly prepared FS.

> Then umount_fs() will be called only in the finally branch.

Btw, that's what my patch does, but at better granularity.

> > I hope this change to the original code is enough to solve your problem.
>
> Probably, but as I mentioned in the commit message that Pythonic
> exceptions are completely over engineered that no-one can get them. I
> think you missed the fact that the exception can happen at any time
> when FS is mounted and when it's not mounted yet. I suggest you think
> more of this code... (Yet it admits that I may have made mistake
> myself, see above)
>
> > I will look into creating a patch.



-- 
With Best Regards,
Andy Shevchenko

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-17 14:06               ` Andy Shevchenko
  2021-05-17 17:08                 ` Heinrich Schuchardt
@ 2021-05-20 19:21                 ` Alper Nebi Yasak
  1 sibling, 0 replies; 18+ messages in thread
From: Alper Nebi Yasak @ 2021-05-20 19:21 UTC (permalink / raw)
  To: u-boot

On 17/05/2021 17:06, Andy Shevchenko wrote:
> I would like to be helpful here and when I have time, I'll look at it
> closer if nobody beats me up to it. Currently I checked the reason why
> we skip them in my scenario:
> ============================================ short test summary info
> ============================================
> SKIPPED [13] test/py/tests/test_fs/conftest.py:274: Creating failed
> for filesystem: fat16. Command 'mkfs.vfat -F
> 16 /home/andy/prj/u-boot/build-sandbox/persistent-data/3GB.fat16.img'
> returned non-zero exit status 127.
> SKIPPED [13] test/py/tests/test_fs/conftest.py:274: Creating failed
> for filesystem: fat32. Command 'mkfs.vfat -F
> 32 /home/andy/prj/u-boot/build-sandbox/persistent-data/3GB.fat32.img'
> returned non-zero exit status 127.
> SKIPPED [13] test/py/tests/test_fs/conftest.py:274: Creating failed
> for filesystem: ext4. Command 'mkfs.ext4  /ho
> me/andy/prj/u-boot/build-sandbox/persistent-data/3GB.ext4.img'
> returned non-zero exit status 127.
> SKIPPED [11] test/py/tests/test_fs/conftest.py:390: Creating failed
> for filesystem: fat16. Command 'mkfs.vfat -F
> 16 /home/andy/prj/u-boot/build-sandbox/persistent-data/128MB.fat16.img'
> returned non-zero exit status 127.
> SKIPPED [11] test/py/tests/test_fs/conftest.py:390: Creating failed
> for filesystem: fat32. Command 'mkfs.vfat -F
> 32 /home/andy/prj/u-boot/build-sandbox/persistent-data/128MB.fat32.img'
> returned non-zero exit status 127.
> SKIPPED [6] test/py/tests/test_fs/conftest.py:479: Setup failed for
> filesystem: fat16
> SKIPPED [6] test/py/tests/test_fs/conftest.py:479: Setup failed for
> filesystem: fat32
> SKIPPED [4] test/py/tests/test_fs/conftest.py:590: Creating failed for
> filesystem: ext4. Command 'mkfs.ext4  /hom
> e/andy/prj/u-boot/build-sandbox/persistent-data/1GB.ext4.img' returned
> non-zero exit status 127.
> SKIPPED [7] test/py/tests/test_fs/conftest.py:513: Creating failed for
> filesystem: fat16. Command 'mkfs.vfat -F 1
> 6 /home/andy/prj/u-boot/build-sandbox/persistent-data/128MB.fat16.img'
> returned non-zero exit status 127.
> SKIPPED [7] test/py/tests/test_fs/conftest.py:513: Creating failed for
> filesystem: fat32. Command 'mkfs.vfat -F 3
> 2 /home/andy/prj/u-boot/build-sandbox/persistent-data/128MB.fat32.img'
> returned non-zero exit status 127.
> ======================================== 3 passed, 91 skipped in
> 10.02s =========================================

I had similar errors as Debian has the mkfs.* executables in /usr/sbin
but that wasn't in my PATH. Running 'export PATH="/usr/sbin:$PATH"'
fixes these for me, maybe it's the same on your system?

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-17 13:21           ` Heinrich Schuchardt
                               ` (2 preceding siblings ...)
  2021-05-17 14:24             ` Heinrich Schuchardt
@ 2021-05-20 19:33             ` Alper Nebi Yasak
  2021-05-20 19:36               ` Heinrich Schuchardt
  3 siblings, 1 reply; 18+ messages in thread
From: Alper Nebi Yasak @ 2021-05-20 19:33 UTC (permalink / raw)
  To: u-boot

On 17/05/2021 16:21, Heinrich Schuchardt wrote:
> If you are asked for a sudo password, you have not install libguestfs.
> 
> Please, install the missing package.

This also might have ended up not in PATH like mkfs.* weren't in mine,
does e.g. "guestmount --version" work from your shell?

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/1] test: revert Don't unmount not (yet) mounted system
  2021-05-20 19:33             ` Alper Nebi Yasak
@ 2021-05-20 19:36               ` Heinrich Schuchardt
  0 siblings, 0 replies; 18+ messages in thread
From: Heinrich Schuchardt @ 2021-05-20 19:36 UTC (permalink / raw)
  To: u-boot

Am 20. Mai 2021 21:33:42 MESZ schrieb Alper Nebi Yasak <alpernebiyasak@gmail.com>:
>On 17/05/2021 16:21, Heinrich Schuchardt wrote:
>> If you are asked for a sudo password, you have not install
>libguestfs.
>> 
>> Please, install the missing package.
>
>This also might have ended up not in PATH like mkfs.* weren't in mine,
>does e.g. "guestmount --version" work from your shell?

On Debian I had to add /sbin and /usr/sbin to the path to run all tests.

Best regards

Heinrich

^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2021-05-20 19:36 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-13 11:40 [PATCH 1/1] test: revert Don't unmount not (yet) mounted system Heinrich Schuchardt
2021-05-17  6:33 ` Andy Shevchenko
2021-05-17  8:48   ` Heinrich Schuchardt
2021-05-17 11:16     ` Andy Shevchenko
2021-05-17 11:35       ` Heinrich Schuchardt
2021-05-17 11:44         ` Andy Shevchenko
2021-05-17 13:21           ` Heinrich Schuchardt
2021-05-17 13:25             ` Andy Shevchenko
2021-05-17 13:29             ` Tom Rini
2021-05-17 14:06               ` Andy Shevchenko
2021-05-17 17:08                 ` Heinrich Schuchardt
2021-05-17 17:57                   ` Andy Shevchenko
2021-05-17 18:01                     ` Andy Shevchenko
2021-05-20 19:21                 ` Alper Nebi Yasak
2021-05-17 14:24             ` Heinrich Schuchardt
2021-05-17 14:43               ` Tom Rini
2021-05-20 19:33             ` Alper Nebi Yasak
2021-05-20 19:36               ` Heinrich Schuchardt

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.