All of lore.kernel.org
 help / color / mirror / Atom feed
From: Simon Glass <sjg@chromium.org>
To: U-Boot Mailing List <u-boot@lists.denx.de>
Cc: Simon Glass <sjg@chromium.org>
Subject: [PATCH v2 30/38] binman: Add a bintool implementation for lz4
Date: Sun,  9 Jan 2022 20:14:05 -0700	[thread overview]
Message-ID: <20220110031413.1970836-31-sjg@chromium.org> (raw)
In-Reply-To: <20220110031413.1970836-1-sjg@chromium.org>

Add a Bintool for this, which is used to compress and decompress data.
It supports the features needed by binman as well as installing via the
lz4 package.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v1)

 tools/binman/btool/lz4.py | 140 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 140 insertions(+)
 create mode 100644 tools/binman/btool/lz4.py

diff --git a/tools/binman/btool/lz4.py b/tools/binman/btool/lz4.py
new file mode 100644
index 00000000000..d165f52da92
--- /dev/null
+++ b/tools/binman/btool/lz4.py
@@ -0,0 +1,140 @@
+# SPDX-License-Identifier: GPL-2.0+
+# Copyright 2022 Google LLC
+#
+"""Bintool implementation for lz4
+
+lz4 allows compression and decompression of files.
+
+Documentation is available via::
+
+   man lz4
+
+Here is the help:
+
+*** LZ4 command line interface 64-bits v1.9.3, by Yann Collet ***
+Usage :
+      lz4 [arg] [input] [output]
+
+input   : a filename
+          with no FILE, or when FILE is - or stdin, read standard input
+Arguments :
+ -1     : Fast compression (default)
+ -9     : High compression
+ -d     : decompression (default for .lz4 extension)
+ -z     : force compression
+ -D FILE: use FILE as dictionary
+ -f     : overwrite output without prompting
+ -k     : preserve source files(s)  (default)
+--rm    : remove source file(s) after successful de/compression
+ -h/-H  : display help/long help and exit
+
+Advanced arguments :
+ -V     : display Version number and exit
+ -v     : verbose mode
+ -q     : suppress warnings; specify twice to suppress errors too
+ -c     : force write to standard output, even if it is the console
+ -t     : test compressed file integrity
+ -m     : multiple input files (implies automatic output filenames)
+ -r     : operate recursively on directories (sets also -m)
+ -l     : compress using Legacy format (Linux kernel compression)
+ -B#    : cut file into blocks of size # bytes [32+]
+                     or predefined block size [4-7] (default: 7)
+ -BI    : Block Independence (default)
+ -BD    : Block dependency (improves compression ratio)
+ -BX    : enable block checksum (default:disabled)
+--no-frame-crc : disable stream checksum (default:enabled)
+--content-size : compressed frame includes original size (default:not present)
+--list FILE : lists information about .lz4 files (useful for files compressed
+    with --content-size flag)
+--[no-]sparse  : sparse mode (default:enabled on file, disabled on stdout)
+--favor-decSpeed: compressed files decompress faster, but are less compressed
+--fast[=#]: switch to ultra fast compression level (default: 1)
+--best  : same as -12
+Benchmark arguments :
+ -b#    : benchmark file(s), using # compression level (default : 1)
+ -e#    : test all compression levels from -bX to # (default : 1)
+ -i#    : minimum evaluation time in seconds (default : 3s)
+"""
+
+import re
+import tempfile
+
+from binman import bintool
+from patman import tools
+
+# pylint: disable=C0103
+class Bintoollz4(bintool.Bintool):
+    """Compression/decompression using the LZ4 algorithm
+
+    This bintool supports running `lz4` to compress and decompress data, as
+    used by binman.
+
+    It is also possible to fetch the tool, which uses `apt` to install it.
+
+    Documentation is available via::
+
+        man lz4
+    """
+    def __init__(self, name):
+        super().__init__(name, 'lz4 compression')
+
+    def compress(self, indata):
+        """Compress data with lz4
+
+        Args:
+            indata (bytes): Data to compress
+
+        Returns:
+            bytes: Compressed data
+        """
+        with tempfile.NamedTemporaryFile(prefix='comp.tmp',
+                                         dir=tools.GetOutputDir()) as tmp:
+            tools.WriteFile(tmp.name, indata)
+            args = ['--no-frame-crc', '-B4', '-5', '-c', tmp.name]
+            return self.run_cmd(*args, binary=True)
+
+    def decompress(self, indata):
+        """Decompress data with lz4
+
+        Args:
+            indata (bytes): Data to decompress
+
+        Returns:
+            bytes: Decompressed data
+        """
+        with tempfile.NamedTemporaryFile(prefix='decomp.tmp',
+                                         dir=tools.GetOutputDir()) as inf:
+            tools.WriteFile(inf.name, indata)
+            args = ['-cd', inf.name]
+            return self.run_cmd(*args, binary=True)
+
+    def fetch(self, method):
+        """Fetch handler for lz4
+
+        This installs the lz4 package using the apt utility.
+
+        Args:
+            method (FETCH_...): Method to use
+
+        Returns:
+            True if the file was fetched and now installed, None if a method
+            other than FETCH_BIN was requested
+
+        Raises:
+            Valuerror: Fetching could not be completed
+        """
+        if method != bintool.FETCH_BIN:
+            return None
+        return self.apt_install('lz4')
+
+    def version(self):
+        """Version handler
+
+        Returns:
+            str: Version number of lz4
+        """
+        out = self.run_cmd('-V').strip()
+        if not out:
+            return super().version()
+        m_version = re.match(r'.* (v[0-9.]*),.*', out)
+        return m_version.group(1) if m_version else out
-- 
2.34.1.575.g55b058a8bb-goog


  parent reply	other threads:[~2022-01-10  3:20 UTC|newest]

Thread overview: 83+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-01-10  3:13 [PATCH v2 00/38] binman: Add support for bintools and missing tools Simon Glass
2022-01-10  3:13 ` [PATCH v2 01/38] Makefile: Fake external blobs by default with binman Simon Glass
2022-10-10 10:15   ` Rasmus Villemoes
2022-10-10 14:25     ` Tom Rini
2022-10-10 15:19     ` Simon Glass
2022-10-10 15:24       ` Tom Rini
2022-10-10 15:33         ` Simon Glass
2022-10-10 15:36           ` Tom Rini
2022-10-10 15:43             ` Simon Glass
2022-01-10  3:13 ` [PATCH v2 02/38] binman: Tweak elf tests for a toolchain change Simon Glass
2022-01-10  3:13 ` [PATCH v2 03/38] mkimage: Show the external-offset error Simon Glass
2022-01-10  3:13 ` [PATCH v2 04/38] binman: Expand the external FIT test a little Simon Glass
2022-01-10  3:13 ` [PATCH v2 05/38] patman: Allow running a tool and returning the full result Simon Glass
2022-01-10  3:13 ` [PATCH v2 06/38] buildman: Move the download function to tools Simon Glass
2022-01-10  3:13 ` [PATCH v2 07/38] patman: Tidy up the download function a little Simon Glass
2022-01-10  3:13 ` [PATCH v2 08/38] patman: Add a function to find a tool on the path Simon Glass
2022-01-10  3:13 ` [PATCH v2 09/38] binman: Write fake blobs to the output directory Simon Glass
2022-01-10  3:13 ` [PATCH v2 10/38] binman: Drop the image name from the fake-blob message Simon Glass
2022-01-10  3:13 ` [PATCH v2 11/38] binman: Allow faked blobs in blob-ext-list Simon Glass
2022-01-10  3:13 ` [PATCH v2 12/38] binman: Correct path for fip_util Simon Glass
2022-01-10  3:13 ` [PATCH v2 13/38] binman: Add installation instructions Simon Glass
2022-01-10  3:13 ` [PATCH v2 14/38] binman: Add support for bintools Simon Glass
2022-01-10  3:13 ` [PATCH v2 15/38] binman: Plumb in " Simon Glass
2022-01-10  3:13 ` [PATCH v2 16/38] binman: Add tests for bintool Simon Glass
2022-01-10  3:13 ` [PATCH v2 17/38] binman: Add a bintool implementation for cbfstool Simon Glass
2022-01-10  3:13 ` [PATCH v2 18/38] binman: Add a bintool implementation for fiptool Simon Glass
2022-01-10  3:13 ` [PATCH v2 19/38] binman: Add a bintool implementation for futility Simon Glass
2022-01-10  3:13 ` [PATCH v2 20/38] binman: Add a bintool implementation for ifwitool Simon Glass
2022-01-10  3:13 ` [PATCH v2 21/38] binman: Add a bintool implementation for mkimage Simon Glass
2022-01-10  3:13 ` [PATCH v2 22/38] binman: Enable bintool tests including cmdline processing Simon Glass
2022-01-10  3:13 ` [PATCH v2 23/38] binman: Convert to using the CBFS bintool Simon Glass
2022-01-10  3:13 ` [PATCH v2 24/38] binman: Convert to using the FIP bintool Simon Glass
2022-01-10  3:14 ` [PATCH v2 25/38] binman: Convert to using the futility bintool Simon Glass
2022-01-10  3:14 ` [PATCH v2 26/38] binman: Convert to using the ifwitool bintool Simon Glass
2022-01-10  3:14 ` [PATCH v2 27/38] binman: Convert to using the mkimage bintool Simon Glass
2022-01-10  3:14 ` [PATCH v2 28/38] binman: Move compression into binman Simon Glass
2022-01-10  3:14 ` [PATCH v2 29/38] binman: Tidy up pylint warnings in comp_util Simon Glass
2022-01-10  3:14 ` Simon Glass [this message]
2022-01-10  3:14 ` [PATCH v2 31/38] binman: Convert to using the lz4 bintool Simon Glass
2022-01-10  3:14 ` [PATCH v2 32/38] binman: Add a bintool implementation for lzma_alone Simon Glass
2022-01-10  3:14 ` [PATCH v2 33/38] binman: Convert to using the lzma_alone bintool Simon Glass
2022-01-10  3:14 ` [PATCH v2 34/38] binman: Plumb in support for missing bintools Simon Glass
2022-01-10  3:14 ` [PATCH v2 35/38] binman: Complete test coverage of comp_util Simon Glass
2022-01-10  3:14 ` [PATCH v2 36/38] binman: Add a command to generate bintool docs Simon Glass
2022-01-10  3:14 ` [PATCH v2 37/38] binman: Add documentation for bintools Simon Glass
2022-01-10  3:14 ` [PATCH v2 38/38] RFC: Move Odroid-C2 to use binman to produce the image Simon Glass
2022-01-21  2:02 ` [PATCH v2 00/38] binman: Add support for bintools and missing tools Simon Glass
2022-01-26 15:36 ` [PATCH v2 36/38] binman: Add a command to generate bintool docs Simon Glass
2022-01-26 15:36 ` [PATCH v2 37/38] binman: Add documentation for bintools Simon Glass
2022-01-26 15:36 ` [PATCH v2 35/38] binman: Complete test coverage of comp_util Simon Glass
2022-01-26 15:36 ` [PATCH v2 34/38] binman: Plumb in support for missing bintools Simon Glass
2022-01-26 15:36 ` [PATCH v2 32/38] binman: Add a bintool implementation for lzma_alone Simon Glass
2022-01-26 15:36 ` [PATCH v2 33/38] binman: Convert to using the lzma_alone bintool Simon Glass
2022-01-26 15:36 ` [PATCH v2 31/38] binman: Convert to using the lz4 bintool Simon Glass
2022-01-26 15:36 ` [PATCH v2 30/38] binman: Add a bintool implementation for lz4 Simon Glass
2022-01-26 15:36 ` [PATCH v2 29/38] binman: Tidy up pylint warnings in comp_util Simon Glass
2022-01-26 15:36 ` [PATCH v2 28/38] binman: Move compression into binman Simon Glass
2022-01-26 15:36 ` [PATCH v2 27/38] binman: Convert to using the mkimage bintool Simon Glass
2022-01-26 15:36 ` [PATCH v2 26/38] binman: Convert to using the ifwitool bintool Simon Glass
2022-01-26 15:36 ` [PATCH v2 25/38] binman: Convert to using the futility bintool Simon Glass
2022-01-26 15:36 ` [PATCH v2 24/38] binman: Convert to using the FIP bintool Simon Glass
2022-01-26 15:36 ` [PATCH v2 22/38] binman: Enable bintool tests including cmdline processing Simon Glass
2022-01-26 15:36 ` [PATCH v2 23/38] binman: Convert to using the CBFS bintool Simon Glass
2022-01-26 15:36 ` [PATCH v2 21/38] binman: Add a bintool implementation for mkimage Simon Glass
2022-01-26 15:36 ` [PATCH v2 20/38] binman: Add a bintool implementation for ifwitool Simon Glass
2022-01-26 15:36 ` [PATCH v2 19/38] binman: Add a bintool implementation for futility Simon Glass
2022-01-26 15:36 ` [PATCH v2 18/38] binman: Add a bintool implementation for fiptool Simon Glass
2022-01-26 15:37 ` [PATCH v2 17/38] binman: Add a bintool implementation for cbfstool Simon Glass
2022-01-26 15:37 ` [PATCH v2 16/38] binman: Add tests for bintool Simon Glass
2022-01-26 15:37 ` [PATCH v2 14/38] binman: Add support for bintools Simon Glass
2022-01-26 15:37 ` [PATCH v2 15/38] binman: Plumb in " Simon Glass
2022-01-26 15:37 ` [PATCH v2 12/38] binman: Correct path for fip_util Simon Glass
2022-01-26 15:37 ` [PATCH v2 13/38] binman: Add installation instructions Simon Glass
2022-01-26 15:37 ` [PATCH v2 11/38] binman: Allow faked blobs in blob-ext-list Simon Glass
2022-01-26 15:37 ` [PATCH v2 10/38] binman: Drop the image name from the fake-blob message Simon Glass
2022-01-26 15:37 ` [PATCH v2 08/38] patman: Add a function to find a tool on the path Simon Glass
2022-01-26 15:37 ` [PATCH v2 07/38] patman: Tidy up the download function a little Simon Glass
2022-01-26 15:37 ` [PATCH v2 06/38] buildman: Move the download function to tools Simon Glass
2022-01-26 15:37 ` [PATCH v2 05/38] patman: Allow running a tool and returning the full result Simon Glass
2022-01-26 15:37 ` [PATCH v2 04/38] binman: Expand the external FIT test a little Simon Glass
2022-01-26 15:37 ` [PATCH v2 03/38] mkimage: Show the external-offset error Simon Glass
2022-01-26 15:37 ` [PATCH v2 01/38] Makefile: Fake external blobs by default with binman Simon Glass
2022-01-26 15:37 ` [PATCH v2 02/38] binman: Tweak elf tests for a toolchain change Simon Glass

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220110031413.1970836-31-sjg@chromium.org \
    --to=sjg@chromium.org \
    --cc=u-boot@lists.denx.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.