[meta-raspberrypi][PATCH] sdcard_image-rpi.bbclass: Allow optional compression

[meta-raspberrypi][PATCH] sdcard_image-rpi.bbclass: Allow optional compression

[Paul Barker]

Image files will typically contain lots of null blocks and should compress well.
This will help with distribution of images over slow network links.

Signed-off-by: Paul Barker <paul@paulbarker.me.uk>
---
 classes/sdcard_image-rpi.bbclass | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/classes/sdcard_image-rpi.bbclass b/classes/sdcard_image-rpi.bbclass
index a2fcd52..8d2a5f2 100644
--- a/classes/sdcard_image-rpi.bbclass
+++ b/classes/sdcard_image-rpi.bbclass
@@ -50,6 +50,11 @@ IMAGE_DEPENDS_rpi-sdimg = " \
 # SD card image name
 SDIMG = "${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.rpi-sdimg"
 
+# Compression method to apply to SDIMG after it has been created. May be "none",
+# "gzip", "bzip2" or "xz". Setting this to something other than "none" will
+# cause SDIMG to be renamed.
+SDIMG_COMPRESSION ?= "none"
+
 # Additional files and/or directories to be copied into the vfat partition from the IMAGE_ROOTFS.
 FATPAYLOAD ?= ""
 
@@ -107,6 +112,19 @@ IMAGE_CMD_rpi-sdimg () {
 	else
 		dd if=${SDIMG_ROOTFS} of=${SDIMG} conv=notrunc seek=1 bs=$(expr 1024 \* ${BOOT_SPACE_ALIGNED} + ${IMAGE_ROOTFS_ALIGNMENT} \* 1024) && sync && sync
 	fi
+
+	# Optionally apply compression
+	case "${SDIMG_COMPRESSION}" in
+	"gzip")
+		gzip -9 "${SDIMG}"
+		;;
+	"bzip2")
+		bzip2 -9 "${SDIMG}"
+		;;
+	"xz")
+		xz "${SDIMG}"
+		;;
+	esac
 }
 
 ROOTFS_POSTPROCESS_COMMAND += " rpi_generate_sysctl_config ; "
-- 
1.8.3.4

Re: [meta-raspberrypi][PATCH] sdcard_image-rpi.bbclass: Allow optional compression

[Andrei Gherzan]

[-- Attachment #1: Type: text/plain, Size: 1359 bytes --]

On Fri, Aug 16, 2013 at 1:48 PM, Paul Barker <paul@paulbarker.me.uk> wrote:

> Image files will typically contain lots of null blocks and should compress
> well.
> This will help with distribution of images over slow network links.
>
> Signed-off-by: Paul Barker <paul@paulbarker.me.uk>
> ---
>  classes/sdcard_image-rpi.bbclass | 18 ++++++++++++++++++
>  1 file changed, 18 insertions(+)
>
> diff --git a/classes/sdcard_image-rpi.bbclass
> b/classes/sdcard_image-rpi.bbclass
> index a2fcd52..8d2a5f2 100644
> --- a/classes/sdcard_image-rpi.bbclass
> +++ b/classes/sdcard_image-rpi.bbclass
> @@ -50,6 +50,11 @@ IMAGE_DEPENDS_rpi-sdimg = " \
>  # SD card image name
>  SDIMG = "${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.rpi-sdimg"
>
> +# Compression method to apply to SDIMG after it has been created. May be
> "none",
> +# "gzip", "bzip2" or "xz". Setting this to something other than "none"
> will
> +# cause SDIMG to be renamed.
> +SDIMG_COMPRESSION ?= "none"
> +


One thing I don't understand here. Where is the SDIMG renamed if for
example I set by mistake SDIMG_COMPRESSION to something like
"this-is-a-mistake"? If I check your code this will be ignored as the case
won't match any keywords.

Would you mind adding the variable explanation to README too?

*Andrei Gherzan*
m: +40.744.478.414 |  f: +40.31.816.28.12

[-- Attachment #2: Type: text/html, Size: 2345 bytes --]

Re: [meta-raspberrypi][PATCH] sdcard_image-rpi.bbclass: Allow optional compression

[Paul Barker]

On 20 August 2013 18:11, Andrei Gherzan <andrei@gherzan.ro> wrote:
> On Fri, Aug 16, 2013 at 1:48 PM, Paul Barker <paul@paulbarker.me.uk> wrote:
>>
>> +# Compression method to apply to SDIMG after it has been created. May be
>> "none",
>> +# "gzip", "bzip2" or "xz". Setting this to something other than "none"
>> will
>> +# cause SDIMG to be renamed.
>> +SDIMG_COMPRESSION ?= "none"
>> +
>
>
> One thing I don't understand here. Where is the SDIMG renamed if for example
> I set by mistake SDIMG_COMPRESSION to something like "this-is-a-mistake"? If
> I check your code this will be ignored as the case won't match any keywords.
>

Setting it to one of the other listed values will cause the SDIMG to
be re-named. I think the code is as I intended, the documentation
comment needs to be corrected though.

Actually I could just have the default be an empty string as anything
other than the understood compression types is ignored rather than
treated as an error.

> Would you mind adding the variable explanation to README too?

Will do.

I'll give this another spin this weekend.

-- 
Paul Barker

Email: paul@paulbarker.me.uk
http://www.paulbarker.me.uk

Re: [meta-raspberrypi][PATCH] sdcard_image-rpi.bbclass: Allow optional compression

[Andrei Gherzan]

[-- Attachment #1: Type: text/plain, Size: 2227 bytes --]

On Wed, Aug 21, 2013 at 11:12 PM, Paul Barker <paul@paulbarker.me.uk> wrote:

> On 20 August 2013 18:11, Andrei Gherzan <andrei@gherzan.ro> wrote:
> > On Fri, Aug 16, 2013 at 1:48 PM, Paul Barker <paul@paulbarker.me.uk>
> wrote:
> >>
> >> +# Compression method to apply to SDIMG after it has been created. May
> be
> >> "none",
> >> +# "gzip", "bzip2" or "xz". Setting this to something other than "none"
> >> will
> >> +# cause SDIMG to be renamed.
> >> +SDIMG_COMPRESSION ?= "none"
> >> +
> >
> >
> > One thing I don't understand here. Where is the SDIMG renamed if for
> example
> > I set by mistake SDIMG_COMPRESSION to something like
> "this-is-a-mistake"? If
> > I check your code this will be ignored as the case won't match any
> keywords.
> >
>
> Setting it to one of the other listed values will cause the SDIMG to
> be re-named. I think the code is as I intended, the documentation
> comment needs to be corrected though.
>
>
You are right. The comment confused me. So only if the COMPRESSION is "gzip",
"bzip2" or "xz" the SDIMG will be renamed. Understood now.

On the other hand, would it make sense to keep both uncompressed and
compressed images?


> Actually I could just have the default be an empty string as anything
> other than the understood compression types is ignored rather than
> treated as an error.
>
>
Indeed. Empty string would work without any issues.


>  > Would you mind adding the variable explanation to README too?
>
> Will do.
>
> I'll give this another spin this weekend.
>

Great. So all we need is a better comment, fix default value and a line or
two in readme.

Another idea. Would be a great feature to have a tool that would configure
the build interactively. Something like:

# configure-build
Want to have SD image compressed compressed? "y/n" y
What compression you want? "x/y/z" x
etc
etc

This script would just edit local.conf. This is because we defined a couple
of variables for this board - i would say pretty many (codecs keys, GPU,
mem etc etc) and it's pretty confusing for people who see this for the
first time. What do you think?

-- 
*Andrei Gherzan*
m: +40.744.478.414 |  f: +40.31.816.28.12

[-- Attachment #2: Type: text/html, Size: 4584 bytes --]

Re: [meta-raspberrypi][PATCH] sdcard_image-rpi.bbclass: Allow optional compression

[Paul Barker]

On 21 August 2013 23:41, Andrei Gherzan <andrei@gherzan.ro> wrote:
> On Wed, Aug 21, 2013 at 11:12 PM, Paul Barker <paul@paulbarker.me.uk> wrote:
>>
>> Setting it to one of the other listed values will cause the SDIMG to
>> be re-named. I think the code is as I intended, the documentation
>> comment needs to be corrected though.
>>
>
> You are right. The comment confused me. So only if the COMPRESSION is
> "gzip", "bzip2" or "xz" the SDIMG will be renamed. Understood now.
>
> On the other hand, would it make sense to keep both uncompressed and
> compressed images?
>

I was just going with the default behaviour of the tools, I can add
the '-k' argument to all of them to keep the uncompressed file.

>
> Another idea. Would be a great feature to have a tool that would configure
> the build interactively. Something like:
>
> # configure-build
> Want to have SD image compressed compressed? "y/n" y
> What compression you want? "x/y/z" x
> etc
> etc
>
> This script would just edit local.conf. This is because we defined a couple
> of variables for this board - i would say pretty many (codecs keys, GPU, mem
> etc etc) and it's pretty confusing for people who see this for the first
> time. What do you think?
>

local.conf could hold settings which relate to the distro or other
layers so I wouldn't want to write a script which directly edits it. I
think a well-documented sample file would be a good idea. Probably
can't call it local.conf.sample as it would only relate to raspberrypi
settings and the intention would be that you create your local.conf
from local.conf.sample and from this new file.

-- 
Paul Barker

Email: paul@paulbarker.me.uk
http://www.paulbarker.me.uk