Re: [PATCH 0/3] Fix xz compression command and optimize compression time - McClintock Matthew-B29882

From: McClintock Matthew-B29882 <B29882@freescale.com>
To: Patches and discussions about the oe-core layer
	<openembedded-core@lists.openembedded.org>
Subject: Re: [PATCH 0/3] Fix xz compression command and optimize compression time
Date: Thu, 12 Jul 2012 19:20:28 +0000	[thread overview]
Message-ID: <CAEsOVNdmS9eFrdRbx4_9+RqEzr6=CAWmAwPo_4wNLYAEvyekVg@mail.gmail.com> (raw)
In-Reply-To: <8586287A-0F78-4778-8491-B8E1C70644B1@dominion.thruhere.net>

On Thu, Jul 12, 2012 at 1:58 PM, Koen Kooi <koen@dominion.thruhere.net> wrote:
> Any volunteers to test this on a system with >4 real cores?

How do you get the memory usage bit?

-M

>
> Op 12 jul. 2012, om 20:13 heeft Andrei Gherzan het volgende geschreven:
>
>> Koen suggested to add -T to xz commands. We have this option implemented in our current version .1alpha.
>>
>> -T threads, --threads=threads
>>              Specify the number of worker threads to use.  Setting threads to
>>              a special value 0 makes xz use as many threads as there are  CPU
>>              cores  on  the system.  The actual number of threads can be less
>>              than threads if the input file is not big enough  for  threading
>>              with  the  given  settings or if using more threads would exceed
>>              the memory usage limit.
>>
>> -------------------
>>
>> Memory: 7.8 GiB
>> Processor:: Intel® Core™ i5-2540M CPU @ 2.60GHz × 4
>>
>> File to compress: 1.9G
>>
>> xz -f -k -e -9 --check=crc32 (current configuration)
>> Memory: 673Mb
>> real    6m37.170s
>> user    6m35.389s
>> sys     0m0.884s
>> Compressed file size: 3.4Mb
>>
>> xz -f -k -e -9 -T 4 --check=crc32
>> Memory: Cannot allocate memory
>>
>> xz -f -k -e -8 -T 4 --check=crc32
>> Memory: 1.8G
>> real    3m24.462s
>> user    12m8.502s
>> sys     0m2.180s
>> Compressed file size: 3.4Mb
>>
>> xz -f -k -T 4 --check=crc32 (-e defaults to -6)
>> Memory: 471Mb
>> real    1m9.265s
>> user    4m8.972s
>> sys     0m0.944s
>> Compressed file size: 3.4Mb
>>
>> So my conclusion would be to use the default -e -6 with -T 0.
>>
>> The following changes since commit 90ad663909c0c8a405b22a510c9f957007d02669:
>>
>>  upstream_tracking: update boost (2012-07-09 17:21:38 +0100)
>>
>> are available in the git repository at:
>>
>>  git://git.yoctoproject.org/poky-contrib ag/xz
>>  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ag/xz
>>
>> Andrei Gherzan (3):
>>  image_types.bbclass: Fix COMPRESS_CMD for xz to redirect compressed
>>    data to file
>>  image_types.bbclass: Add XZ variable to set number of threads to be
>>    used while compressing
>>  image_types.bbclass: Default XZ_COMPRESSION_LEVEL to -e -6
>>
>> meta/classes/image_types.bbclass |    5 +++--
>> 1 file changed, 3 insertions(+), 2 deletions(-)
>>
>> --
>> 1.7.9.5
>>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core