[PATCH 0/9] Squashfs: Add support for LZMA compressed filesystems

[PATCH 0/9] Squashfs: Add support for LZMA compressed filesystems

[Phillip Lougher]

Hi,

The following patches add LZMA decompression support to Squashfs, using the
in-kernel LZMA decompression library.

The patches also add a decompression framework to Squashfs.
This allows LZMA decompression to be added cleanly, and it allows
additional decompressors to be easily added in the future.

To enable the in-kernel LZMA decompression code to be used by Squashfs,
there are two patches to the lzma code itself: one to make lzma available
to non-init code, and one to make lzma reentrant.  These are obviously
not restricted to Squashfs, but are needed by any non-init code that
may wish to use lzma compression.

These patches are available in my squashfs-devel git tree in a slightly
different format (these patches have been refactored for posting, they'll
be put into a git tree ASAP).

http://git.kernel.org/?p=linux/kernel/git/pkl/squashfs-devel.git;a=summary

I would like to thank the CE Linux Forum (CELF) for supporting this work.

Thanks

Phillip

Re: [PATCH 0/9] Squashfs: Add support for LZMA compressed filesystems

[Bruno Wolff III]

On Mon, Dec 07, 2009 at 08:37:48 +0000,
  Phillip Lougher <phillip@lougher.demon.co.uk> wrote:
> Hi,
> 
> The following patches add LZMA decompression support to Squashfs, using the
> in-kernel LZMA decompression library.

I am a member of the Spins SIG and the Games Spin maintainer for Fedora.
Thanks for working to get LZMA squashfs included in 2.6.33. If this
gets accepted, we will have livecd-creator using it in F13 in order
to include more packages in size limited images (such as the Games Spin)
or reduce the size other images.
Possibly it will be mentioned as a "feature" for the F13 release. I have
a proposed feature page set up and waiting for these patches to be accepted at:
https://fedoraproject.org/wiki/Features/LZMA_for_Live_Images

Re: [PATCH 0/9] Squashfs: Add support for LZMA compressed filesystems

[Ozan Çağlayan]

Bruno Wolff III wrote:
> On Mon, Dec 07, 2009 at 08:37:48 +0000,
>   Phillip Lougher <phillip@lougher.demon.co.uk> wrote:
>   
>> Hi,
>>
>> The following patches add LZMA decompression support to Squashfs, using the
>> in-kernel LZMA decompression library.
>>     
>
> I am a member of the Spins SIG and the Games Spin maintainer for Fedora.
> Thanks for working to get LZMA squashfs included in 2.6.33. If this
> gets accepted, we will have livecd-creator using it in F13 in order
> to include more packages in size limited images (such as the Games Spin)
> or reduce the size other images
>   

Similar, I was looking forward to squashfs-lzma to use it on our
distribution's installation/live medium. I once digged into the openwrt
patches and failed. Now I tested your patches (on top of 2.6.30.9) +
userspace from CVS patched with lzma 4.65, it's working flawlessly. With
lzma, our live cd image's size decreased from 820mb to 692mb.

Thanks a lot for your work
Ozan Caglayan

Re: [PATCH 0/9] Squashfs: Add support for LZMA compressed filesystems

[Bruno Wolff III]

On Mon, Dec 07, 2009 at 08:37:48 +0000,
  Phillip Lougher <phillip@lougher.demon.co.uk> wrote:
> Hi,
> 
> The following patches add LZMA decompression support to Squashfs, using the
> in-kernel LZMA decompression library.

Do these still have a chance to make 2.6.33?

Re: [PATCH 0/9] Squashfs: Add support for LZMA compressed filesystems

[Phillip Lougher]

Bruno Wolff III wrote:
> On Mon, Dec 07, 2009 at 08:37:48 +0000,
>   Phillip Lougher <phillip@lougher.demon.co.uk> wrote:
>> Hi,
>>
>> The following patches add LZMA decompression support to Squashfs, using the
>> in-kernel LZMA decompression library.
> 
> Do these still have a chance to make 2.6.33?
> 

It's very unlikely.  Andrew wanted it to spend a cycle in linux-next.  I'll then
try for 2.6.34 merge.  I'm happy with that.

Phillip

Re: [PATCH 0/9] Squashfs: Add support for LZMA compressed filesystems

[Bruno Wolff III]

On Mon, Dec 21, 2009 at 23:47:30 +0000,
  Phillip Lougher <phillip@lougher.demon.co.uk> wrote:
> Bruno Wolff III wrote:
> >On Mon, Dec 07, 2009 at 08:37:48 +0000,
> >  Phillip Lougher <phillip@lougher.demon.co.uk> wrote:
> >>Hi,
> >>
> >>The following patches add LZMA decompression support to Squashfs, using the
> >>in-kernel LZMA decompression library.
> >
> >Do these still have a chance to make 2.6.33?
> >
> 
> It's very unlikely.  Andrew wanted it to spend a cycle in linux-next.  I'll then
> try for 2.6.34 merge.  I'm happy with that.

I hadn't seen it show up there yet either. But maybe its a bit too early
for that. If it gets there I might have a chance of convincing one of
the Fedora kernel guys of back porting it, since F13 is likely to be released
with the 2,6,34 kernel in any case. (Along with some actual numbers for
our live iso images to show it's important to have. But I can get those
using the devel version of squashfs without the kernel changes.)
I'll keep an eye out for it showing up in linux-next. Thanks for the update.

Re: [PATCH 0/9] Squashfs: Add support for LZMA compressed filesystems

[Bruno Wolff III]

On Mon, Dec 21, 2009 at 23:47:30 +0000,
  Phillip Lougher <phillip@lougher.demon.co.uk> wrote:
> 
> It's very unlikely.  Andrew wanted it to spend a cycle in linux-next.  I'll then
> try for 2.6.34 merge.  I'm happy with that.
> 
> Phillip

linux-next still hasn't picked these patches up. It would give me more leverage
in getting the patches into Fedora 13 if they were to be in linux-next.

Re: [PATCH 0/9] Squashfs: Add support for LZMA compressed filesystems

[Bruno Wolff III]

On Thu, Dec 31, 2009 at 06:06:34 -0600,
  Bruno Wolff III <bruno@wolff.to> wrote:
> On Mon, Dec 21, 2009 at 23:47:30 +0000,
>   Phillip Lougher <phillip@lougher.demon.co.uk> wrote:
> > 
> > It's very unlikely.  Andrew wanted it to spend a cycle in linux-next.  I'll then
> > try for 2.6.34 merge.  I'm happy with that.
> > 
> > Phillip
> 
> linux-next still hasn't picked these patches up. It would give me more leverage
> in getting the patches into Fedora 13 if they were to be in linux-next.

I see that they made it into the January 1st linux-next tree. Thanks.

Re: [PATCH 0/9] Squashfs: Add support for LZMA compressed filesystems

[Arkadiusz Miśkiewicz]

Phillip Lougher wrote:

> Peter Korsgaard wrote:

>> Nice, do you also have an mksquashfs with lzma support?
>> 
> 
> Yes, in Squashfs CVS (http://sourceforge.net/projects/squashfs/develop).

Any reason for not using much nicer xz library instead of lzma sdk ? 

http://tukaani.org/xz/

> Phillip


-- 
Arkadiusz Miśkiewicz        PLD/Linux Team
arekm / maven.pl            http://ftp.pld-linux.org/

Re: [PATCH 0/9] Squashfs: Add support for LZMA compressed filesystems

[Phillip Lougher]

Peter Korsgaard wrote:
>>>>>> "root" == root  <root@lougher.demon.co.uk> writes:
> 
>  root> Hi,
>  root> The following patches add LZMA decompression support to Squashfs, using the
>  root> in-kernel LZMA decompression library.
> 
>  root> The patches also add a decompression framework to Squashfs.
>  root> This allows LZMA decompression to be added cleanly, and it allows
>  root> additional decompressors to be easily added in the future.
> 
>  root> To enable the in-kernel LZMA decompression code to be used by Squashfs,
>  root> there are two patches to the lzma code itself: one to make lzma available
>  root> to non-init code, and one to make lzma reentrant.  These are obviously
>  root> not restricted to Squashfs, but are needed by any non-init code that
>  root> may wish to use lzma compression.
> 
> Nice, do you also have an mksquashfs with lzma support?
> 

Yes, in Squashfs CVS (http://sourceforge.net/projects/squashfs/develop).
You'll need to edit the squashfs-tools Makefile to enable LZMA support.
The comments in the Makefile should, hopefully, explain how to
build LZMA support into Mksquashfs/Unsquashfs.

Once built-in, LZMA support can be specified using the -comp lzma option, i.e.

mksquashfs dir dir.img -comp lzma

Unsquashfs doesn't need any extra options, it automatically detects which
compression has been used.

You can tell which compression algorithms Mksquashfs/Unsquashfs support by
just typing the command on its own (i.e. mksquashfs, or unsquashfs).  The
(de-)compressors available are displayed at the end of the output.

Phillip

Re: [PATCH 0/9] Squashfs: Add support for LZMA compressed filesystems

[Peter Korsgaard]

>>>>> "root" == root  <root@lougher.demon.co.uk> writes:

 root> Hi,
 root> The following patches add LZMA decompression support to Squashfs, using the
 root> in-kernel LZMA decompression library.

 root> The patches also add a decompression framework to Squashfs.
 root> This allows LZMA decompression to be added cleanly, and it allows
 root> additional decompressors to be easily added in the future.

 root> To enable the in-kernel LZMA decompression code to be used by Squashfs,
 root> there are two patches to the lzma code itself: one to make lzma available
 root> to non-init code, and one to make lzma reentrant.  These are obviously
 root> not restricted to Squashfs, but are needed by any non-init code that
 root> may wish to use lzma compression.

Nice, do you also have an mksquashfs with lzma support?

-- 
Bye, Peter Korsgaard

[PATCH 0/9] Squashfs: Add support for LZMA compressed filesystems

[root]

Hi,

The following patches add LZMA decompression support to Squashfs, using the
in-kernel LZMA decompression library.

The patches also add a decompression framework to Squashfs.
This allows LZMA decompression to be added cleanly, and it allows
additional decompressors to be easily added in the future.

To enable the in-kernel LZMA decompression code to be used by Squashfs,
there are two patches to the lzma code itself: one to make lzma available
to non-init code, and one to make lzma reentrant.  These are obviously
not restricted to Squashfs, but are needed by any non-init code that
may wish to use lzma compression.

These patches are available in my squashfs-devel git tree in a slightly
different format (these patches have been refactored for posting, they'll
be put into a git tree ASAP).

http://git.kernel.org/?p=linux/kernel/git/pkl/squashfs-devel.git;a=summary

I would like to thank the CE Linux Forum (CELF) for supporting this work.

Thanks

Phillip