[PATCH] oe-selftest: test wic sparse_copy API

[PATCH] oe-selftest: test wic sparse_copy API

[Ed Bartosh]

Added new parameter 'api' to sparse_copy function to specify
underlying filemap API to use. By default sparse_copy will
try both available APIs.

Added test case for sparse_copy to wic test suite.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 meta/lib/oeqa/selftest/wic.py | 26 ++++++++++++++++++++++++++
 scripts/lib/wic/filemap.py    |  6 ++++--
 2 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/selftest/wic.py b/meta/lib/oeqa/selftest/wic.py
index df5e060..7e26dde 100644
--- a/meta/lib/oeqa/selftest/wic.py
+++ b/meta/lib/oeqa/selftest/wic.py
@@ -24,6 +24,7 @@
 """Test cases for wic."""
 
 import os
+import sys
 import unittest
 
 from glob import glob
@@ -758,3 +759,28 @@ part /etc --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path bin/ --r
         self.assertEqual(0, runCmd(cmd).status)
         self.remove_config(config)
         self.assertEqual(1, len(glob(self.resultdir + "sdimage-bootpart-*direct")))
+
+    def test_sparse_copy(self):
+        """Test sparse_copy with FIEMAP and SEEK_HOLE filemap APIs"""
+        libpath = os.path.join(get_bb_var('COREBASE'), 'scripts', 'lib', 'wic')
+        sys.path.insert(0, libpath)
+        from  filemap import FilemapFiemap, FilemapSeek, sparse_copy
+        with NamedTemporaryFile("w", suffix=".wic-sparse") as sparse:
+            src_name = sparse.name
+            src_size = 1024 * 10
+            sparse.truncate(src_size)
+            # write one byte to the file
+            with open(src_name, 'r+b') as sfile:
+                sfile.seek(1024 * 4)
+                sfile.write(b'\x00')
+            dest = sparse.name + '.out'
+            # copy src file to dest using different filemap APIs
+            for api in (FilemapFiemap, FilemapSeek, None):
+                if os.path.exists(dest):
+                    os.unlink(dest)
+                sparse_copy(sparse.name, dest, api=api)
+                dest_stat = os.stat(dest)
+                self.assertEqual(dest_stat.st_size, src_size)
+                # 8 blocks is 4K (physical sector size)
+                self.assertEqual(dest_stat.st_blocks, 8)
+            os.unlink(dest)
diff --git a/scripts/lib/wic/filemap.py b/scripts/lib/wic/filemap.py
index 080668e..1f1aacc 100644
--- a/scripts/lib/wic/filemap.py
+++ b/scripts/lib/wic/filemap.py
@@ -530,9 +530,11 @@ def filemap(image, log=None):
     except ErrorNotSupp:
         return FilemapSeek(image, log)
 
-def sparse_copy(src_fname, dst_fname, offset=0, skip=0):
+def sparse_copy(src_fname, dst_fname, offset=0, skip=0, api=None):
     """Efficiently copy sparse file to or into another file."""
-    fmap = filemap(src_fname)
+    if not api:
+        api = filemap
+    fmap = api(src_fname)
     try:
         dst_file = open(dst_fname, 'r+b')
     except IOError:
-- 
2.1.4

✗ patchtest: failure for oe-selftest: test wic sparse_copy API

[Patchwork]

== Series Details ==

Series: oe-selftest: test wic sparse_copy API
Revision: 1
URL   : https://patchwork.openembedded.org/series/6144/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             Series does not apply on top of target branch [test_series_merge_on_head] 
  Suggested fix    Rebase your series on top of targeted branch
  Targeted branch  master (currently at 3b7111b30d)



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

Re: [PATCH] oe-selftest: test wic sparse_copy API

[Richard Purdie]

On Tue, 2017-04-04 at 14:10 +0300, Ed Bartosh wrote:
> Added new parameter 'api' to sparse_copy function to specify
> underlying filemap API to use. By default sparse_copy will
> try both available APIs.
> 
> Added test case for sparse_copy to wic test suite.
> 
> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> ---
>  meta/lib/oeqa/selftest/wic.py | 26 ++++++++++++++++++++++++++
>  scripts/lib/wic/filemap.py    |  6 ++++--
>  2 files changed, 30 insertions(+), 2 deletions(-)

This failed on the autobuilder:

https://autobuilder.yocto.io/builders/nightly-oe-selftest/builds/244/steps/Running%20oe-selftest/logs/stdio

Cheers,

Richard

[PATCH v2] oe-selftest: test wic sparse_copy API

[Ed Bartosh]

Added new parameter 'api' to sparse_copy function to specify
underlying filemap API to use. By default sparse_copy will
try both available APIs.

Added test case for sparse_copy to wic test suite.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 meta/lib/oeqa/selftest/wic.py | 29 +++++++++++++++++++++++++++++
 scripts/lib/wic/filemap.py    |  6 ++++--
 2 files changed, 33 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/selftest/wic.py b/meta/lib/oeqa/selftest/wic.py
index df5e060..0a2f7ff 100644
--- a/meta/lib/oeqa/selftest/wic.py
+++ b/meta/lib/oeqa/selftest/wic.py
@@ -24,6 +24,7 @@
 """Test cases for wic."""
 
 import os
+import sys
 import unittest
 
 from glob import glob
@@ -758,3 +759,31 @@ part /etc --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path bin/ --r
         self.assertEqual(0, runCmd(cmd).status)
         self.remove_config(config)
         self.assertEqual(1, len(glob(self.resultdir + "sdimage-bootpart-*direct")))
+
+    def test_sparse_copy(self):
+        """Test sparse_copy with FIEMAP and SEEK_HOLE filemap APIs"""
+        libpath = os.path.join(get_bb_var('COREBASE'), 'scripts', 'lib', 'wic')
+        sys.path.insert(0, libpath)
+        from  filemap import FilemapFiemap, FilemapSeek, sparse_copy, ErrorNotSupp
+        with NamedTemporaryFile("w", suffix=".wic-sparse") as sparse:
+            src_name = sparse.name
+            src_size = 1024 * 10
+            sparse.truncate(src_size)
+            # write one byte to the file
+            with open(src_name, 'r+b') as sfile:
+                sfile.seek(1024 * 4)
+                sfile.write(b'\x00')
+            dest = sparse.name + '.out'
+            # copy src file to dest using different filemap APIs
+            for api in (FilemapFiemap, FilemapSeek, None):
+                if os.path.exists(dest):
+                    os.unlink(dest)
+                try:
+                    sparse_copy(sparse.name, dest, api=api)
+                except ErrorNotSupp:
+                    continue # skip unsupported API
+                dest_stat = os.stat(dest)
+                self.assertEqual(dest_stat.st_size, src_size)
+                # 8 blocks is 4K (physical sector size)
+                self.assertEqual(dest_stat.st_blocks, 8)
+            os.unlink(dest)
diff --git a/scripts/lib/wic/filemap.py b/scripts/lib/wic/filemap.py
index 080668e..1f1aacc 100644
--- a/scripts/lib/wic/filemap.py
+++ b/scripts/lib/wic/filemap.py
@@ -530,9 +530,11 @@ def filemap(image, log=None):
     except ErrorNotSupp:
         return FilemapSeek(image, log)
 
-def sparse_copy(src_fname, dst_fname, offset=0, skip=0):
+def sparse_copy(src_fname, dst_fname, offset=0, skip=0, api=None):
     """Efficiently copy sparse file to or into another file."""
-    fmap = filemap(src_fname)
+    if not api:
+        api = filemap
+    fmap = api(src_fname)
     try:
         dst_file = open(dst_fname, 'r+b')
     except IOError:
-- 
2.1.4