* Re: [PATCH] wic: fallback to dd, if sparse_copy does not work
2017-11-16 11:49 [PATCH] wic: fallback to dd, if sparse_copy does not work Dogukan Ergun
@ 2017-11-16 13:26 ` Martin Hundebøll
2017-11-16 13:28 ` Burton, Ross
2017-11-22 8:26 ` Ed Bartosh
2 siblings, 0 replies; 4+ messages in thread
From: Martin Hundebøll @ 2017-11-16 13:26 UTC (permalink / raw)
To: Dogukan Ergun, openembedded-core
On 2017-11-16 12:49, Dogukan Ergun wrote:
> Docker's aufs filesystem doesn't support file ioctl operations like FIGETBSZ
> or FIEMAP.
> Sparse_copy operation will fail if those ioctls are not supported.
> If sparse_copy fails while generating wic images, fallback to dd for copying
> filesystems on final image.
>
> Signed-off-by: Dogukan Ergun <dogukan.ergun@gmail.com>
> ---
> scripts/lib/wic/plugins/imager/direct.py | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py
> index da1c061..70f93ee 100644
> --- a/scripts/lib/wic/plugins/imager/direct.py
> +++ b/scripts/lib/wic/plugins/imager/direct.py
> @@ -577,8 +577,14 @@ class PartitionedImage():
> for part in self.partitions:
> source = part.source_file
> if source:
> - # install source_file contents into a partition
> - sparse_copy(source, self.path, seek=part.start * self.sector_size)
> + try:
> + # install source_file contents into a partition
> + sparse_copy(source, self.path, seek=part.start * self.sector_size)
> + except:
I guess a failing sparse_copy() would raise an IOError. Shouldn't you
limit the except to catching only those?
// Martin
> + # Sparse_copy failed, fallback to dd method
> + dd_cmd = "dd if=%s of=%s bs=%d seek=%d count=%d conv=notrunc" % \
> + (source, self.path, self.sector_size, part.start, part.size_sec)
> + exec_cmd(dd_cmd)
>
> logger.debug("Installed %s in partition %d, sectors %d-%d, "
> "size %d sectors", source, part.num, part.start,
>
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [PATCH] wic: fallback to dd, if sparse_copy does not work
2017-11-16 11:49 [PATCH] wic: fallback to dd, if sparse_copy does not work Dogukan Ergun
2017-11-16 13:26 ` Martin Hundebøll
@ 2017-11-16 13:28 ` Burton, Ross
2017-11-22 8:26 ` Ed Bartosh
2 siblings, 0 replies; 4+ messages in thread
From: Burton, Ross @ 2017-11-16 13:28 UTC (permalink / raw)
To: Dogukan Ergun; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 2120 bytes --]
Would it be neater to change sparse_copy itself so that it does the
fallback, not every caller?
Ross
On 16 November 2017 at 11:49, Dogukan Ergun <dogukan.ergun@gmail.com> wrote:
> Docker's aufs filesystem doesn't support file ioctl operations like
> FIGETBSZ
> or FIEMAP.
> Sparse_copy operation will fail if those ioctls are not supported.
> If sparse_copy fails while generating wic images, fallback to dd for
> copying
> filesystems on final image.
>
> Signed-off-by: Dogukan Ergun <dogukan.ergun@gmail.com>
> ---
> scripts/lib/wic/plugins/imager/direct.py | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/lib/wic/plugins/imager/direct.py
> b/scripts/lib/wic/plugins/imager/direct.py
> index da1c061..70f93ee 100644
> --- a/scripts/lib/wic/plugins/imager/direct.py
> +++ b/scripts/lib/wic/plugins/imager/direct.py
> @@ -577,8 +577,14 @@ class PartitionedImage():
> for part in self.partitions:
> source = part.source_file
> if source:
> - # install source_file contents into a partition
> - sparse_copy(source, self.path, seek=part.start *
> self.sector_size)
> + try:
> + # install source_file contents into a partition
> + sparse_copy(source, self.path, seek=part.start *
> self.sector_size)
> + except:
> + # Sparse_copy failed, fallback to dd method
> + dd_cmd = "dd if=%s of=%s bs=%d seek=%d count=%d
> conv=notrunc" % \
> + (source, self.path, self.sector_size, part.start,
> part.size_sec)
> + exec_cmd(dd_cmd)
>
> logger.debug("Installed %s in partition %d, sectors
> %d-%d, "
> "size %d sectors", source, part.num,
> part.start,
> --
> 2.7.4
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
[-- Attachment #2: Type: text/html, Size: 2993 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [PATCH] wic: fallback to dd, if sparse_copy does not work
2017-11-16 11:49 [PATCH] wic: fallback to dd, if sparse_copy does not work Dogukan Ergun
2017-11-16 13:26 ` Martin Hundebøll
2017-11-16 13:28 ` Burton, Ross
@ 2017-11-22 8:26 ` Ed Bartosh
2 siblings, 0 replies; 4+ messages in thread
From: Ed Bartosh @ 2017-11-22 8:26 UTC (permalink / raw)
To: Dogukan Ergun; +Cc: openembedded-core
On Thu, Nov 16, 2017 at 02:49:43PM +0300, Dogukan Ergun wrote:
> Docker's aufs filesystem doesn't support file ioctl operations like FIGETBSZ
> or FIEMAP.
> Sparse_copy operation will fail if those ioctls are not supported.
> If sparse_copy fails while generating wic images, fallback to dd for copying
> filesystems on final image.
>
You can make sparse_copy to use SEEK_HOLE / SEEK_DATA instead of FIEMAP.
This can be done by using 'api' parameter. If AUFS supports it this
there is no need to fall back to using dd.
> Signed-off-by: Dogukan Ergun <dogukan.ergun@gmail.com>
> ---
> scripts/lib/wic/plugins/imager/direct.py | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py
> index da1c061..70f93ee 100644
> --- a/scripts/lib/wic/plugins/imager/direct.py
> +++ b/scripts/lib/wic/plugins/imager/direct.py
> @@ -577,8 +577,14 @@ class PartitionedImage():
> for part in self.partitions:
> source = part.source_file
> if source:
> - # install source_file contents into a partition
> - sparse_copy(source, self.path, seek=part.start * self.sector_size)
> + try:
> + # install source_file contents into a partition
> + sparse_copy(source, self.path, seek=part.start * self.sector_size)
> + except:
sparse_copy raises ErrorNotSupp if some ioctl is not supported. Please,
catch this particular exception instead of any exception.
> + # Sparse_copy failed, fallback to dd method
> + dd_cmd = "dd if=%s of=%s bs=%d seek=%d count=%d conv=notrunc" % \
> + (source, self.path, self.sector_size, part.start, part.size_sec)
> + exec_cmd(dd_cmd)
>
> logger.debug("Installed %s in partition %d, sectors %d-%d, "
> "size %d sectors", source, part.num, part.start,
> --
> 2.7.4
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
--
Regards,
Ed
^ permalink raw reply [flat|nested] 4+ messages in thread