From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 96240C433F5 for ; Wed, 24 Nov 2021 04:11:05 +0000 (UTC) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 6A43B836DC; Wed, 24 Nov 2021 05:10:51 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="k7LvovF0"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id D7CB1836E8; Wed, 24 Nov 2021 05:10:29 +0100 (CET) Received: from mail-io1-xd31.google.com (mail-io1-xd31.google.com [IPv6:2607:f8b0:4864:20::d31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 7B77683605 for ; Wed, 24 Nov 2021 05:10:23 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd31.google.com with SMTP id x10so1470171ioj.9 for ; Tue, 23 Nov 2021 20:10:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hZF8a8BVjVgIDe6SPlsuOSIMqIH+iyN8LKh/amFqGHI=; b=k7LvovF0tMVg9+Nw6MXLVwEQKDU7V14GzC1kmuHWuS0oBWQoqKFHNv2aFvkp4J4a79 TM+itOoWqnRJFHGL+TsqH4uJIA5MwthpQNyU1GWDlpa8A8648N5MEFRW6sSD2KShu4XP LqZMCpRquUFcFJ9hImEGp8OFdva4Lyz02mPGc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hZF8a8BVjVgIDe6SPlsuOSIMqIH+iyN8LKh/amFqGHI=; b=iFnzSyR6SGBlpKdYuUrce/Gij72ax2bRaSApbOM2PS0SDU3LB4U8j8vLCaXkfSNEPQ szE9S1ozTElUfmyKOwWx4O/Ygk0LtuGGgPw2jIvfPb+WzjyRmPHSwvS3+Czvfm5TWo9U jttlpl9C1Bly7FLWb2H/CRnz+P9QxSkZJxmXX1/rvY8356aXXbzuooYULdhveBdBL9t/ PCR/K1Y0Li4GuDQFY7guxjkZDC4AD9fKyHgVOj9w569x74KjPu1UZ3h7tcHchV8XtnWy eeFAFs+2TmBmfIYi8vYdj7DgVsZmMF63PxPtrQ2+XNz9Z+tXnxpNOPbK4hcxmb50IvhP y5uw== X-Gm-Message-State: AOAM530zHjp0YOjZY4l6XAT/W7IzxiIaNIq8yXip7atbmARkNnFi8JSB JRrXO05VqhjSRlpy6/ayoBo38QDPZXRukQ== X-Google-Smtp-Source: ABdhPJy2YWV2536lloDtZliN5wWJFrE5sIiJCnlC9l9J46/E0y3NQBVMn7TWha9nSSK9Y3bZBPs55Q== X-Received: by 2002:a05:6602:1648:: with SMTP id y8mr11005071iow.3.1637727021883; Tue, 23 Nov 2021 20:10:21 -0800 (PST) Received: from kiwi.bld.corp.google.com (c-67-190-101-114.hsd1.co.comcast.net. [67.190.101.114]) by smtp.gmail.com with ESMTPSA id x13sm681819ilp.43.2021.11.23.20.10.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 Nov 2021 20:10:21 -0800 (PST) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 5/7] binman: Support lists of external blobs Date: Tue, 23 Nov 2021 21:09:52 -0700 Message-Id: <20211124040954.699821-6-sjg@chromium.org> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog In-Reply-To: <20211124040954.699821-1-sjg@chromium.org> References: <20211124040954.699821-1-sjg@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.37 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.2 at phobos.denx.de X-Virus-Status: Clean Sometimes it is useful to have a list of related external blobs in a single entry. An example is the DDR binaries used by meson. There are 9 files in total. Add support for this, so we don't have to have a separate entry for each. Signed-off-by: Simon Glass --- scripts/pylint.base | 1 + tools/binman/entries.rst | 14 +++++ tools/binman/etype/blob.py | 16 ++++- tools/binman/etype/blob_ext_list.py | 58 +++++++++++++++++++ tools/binman/ftest.py | 20 +++++++ tools/binman/test/215_blob_ext_list.dts | 14 +++++ .../binman/test/216_blob_ext_list_missing.dts | 14 +++++ 7 files changed, 134 insertions(+), 3 deletions(-) create mode 100644 tools/binman/etype/blob_ext_list.py create mode 100644 tools/binman/test/215_blob_ext_list.dts create mode 100644 tools/binman/test/216_blob_ext_list_missing.dts diff --git a/scripts/pylint.base b/scripts/pylint.base index 3d891edf261..1b320e421e3 100644 --- a/scripts/pylint.base +++ b/scripts/pylint.base @@ -22,6 +22,7 @@ binman.state 3.30 blob -1.94 blob_dtb -10.00 blob_ext -20.00 +blob_ext_list -0.32 blob_named_by_arg -7.78 blob_phase -5.00 buildman.board 7.11 diff --git a/tools/binman/entries.rst b/tools/binman/entries.rst index cf3a6e69599..c47f7df0980 100644 --- a/tools/binman/entries.rst +++ b/tools/binman/entries.rst @@ -223,6 +223,20 @@ See 'blob' for Properties / Entry arguments. +Entry: blob-ext-list: List of externally built binary blobs +----------------------------------------------------------- + +This is like blob-ext except that a number of blobs can be provided, +typically with some sort of relationship, e.g. all are DDC parameters. + +If any of the external files needed by this llist is missing, binman can +optionally ignore it and produce a broken image with a warning. + +Args: + filenames: List of filenames to read and include + + + Entry: blob-named-by-arg: A blob entry which gets its filename property from its subclass ----------------------------------------------------------------------------------------- diff --git a/tools/binman/etype/blob.py b/tools/binman/etype/blob.py index fae86ca3ec0..8c1b809e8da 100644 --- a/tools/binman/etype/blob.py +++ b/tools/binman/etype/blob.py @@ -48,10 +48,10 @@ class Entry_blob(Entry): self.ReadBlobContents() return True - def ReadBlobContents(self): + def ReadFileContents(self, pathname): """Read blob contents into memory - This function compresses the data before storing if needed. + This function compresses the data before returning if needed. We assume the data is small enough to fit into memory. If this is used for large filesystem image that might not be true. @@ -59,13 +59,23 @@ class Entry_blob(Entry): new Entry method which can read in chunks. Then we could copy the data in chunks and avoid reading it all at once. For now this seems like an unnecessary complication. + + Args: + pathname (str): Pathname to read from + + Returns: + bytes: Data read """ state.TimingStart('read') - indata = tools.ReadFile(self._pathname) + indata = tools.ReadFile(pathname) state.TimingAccum('read') state.TimingStart('compress') data = self.CompressData(indata) state.TimingAccum('compress') + return data + + def ReadBlobContents(self): + data = self.ReadFileContents(self._pathname) self.SetContents(data) return True diff --git a/tools/binman/etype/blob_ext_list.py b/tools/binman/etype/blob_ext_list.py new file mode 100644 index 00000000000..136ae819946 --- /dev/null +++ b/tools/binman/etype/blob_ext_list.py @@ -0,0 +1,58 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2016 Google, Inc +# Written by Simon Glass +# +# Entry-type module for a list of external blobs, not built by U-Boot +# + +import os + +from binman.etype.blob import Entry_blob +from dtoc import fdt_util +from patman import tools +from patman import tout + +class Entry_blob_ext_list(Entry_blob): + """List of externally built binary blobs + + This is like blob-ext except that a number of blobs can be provided, + typically with some sort of relationship, e.g. all are DDC parameters. + + If any of the external files needed by this llist is missing, binman can + optionally ignore it and produce a broken image with a warning. + + Args: + filenames: List of filenames to read and include + """ + def __init__(self, section, etype, node): + Entry_blob.__init__(self, section, etype, node) + self.external = True + + def ReadNode(self): + super().ReadNode() + self._filenames = fdt_util.GetStringList(self._node, 'filenames') + self._pathnames = [] + + def ObtainContents(self): + missing = False + pathnames = [] + for fname in self._filenames: + pathname = tools.GetInputFilename( + fname, self.external and self.section.GetAllowMissing()) + # Allow the file to be missing + if not pathname: + missing = True + pathnames.append(pathname) + self._pathnames = pathnames + + if missing: + self.SetContents(b'') + self.missing = True + return True + + data = bytearray() + for pathname in pathnames: + data += self.ReadFileContents(pathname) + + self.SetContents(data) + return True diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 0ce829d1225..acbe107d6ae 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -4932,6 +4932,26 @@ fdt fdtmap Extract the devicetree blob from the fdtmap finally: shutil.rmtree(tmpdir) + def testExtblobList(self): + """Test an image with an external blob list""" + data = self._DoReadFile('215_blob_ext_list.dts') + self.assertEqual(REFCODE_DATA + FSP_M_DATA, data) + + def testExtblobListMissing(self): + """Test an image with a missing external blob""" + with self.assertRaises(ValueError) as e: + self._DoReadFile('216_blob_ext_list_missing.dts') + self.assertIn("Filename 'missing-file' not found in input path", + str(e.exception)) + + def testExtblobListMissingOk(self): + """Test an image with an missing external blob that is allowed""" + with test_util.capture_sys_output() as (stdout, stderr): + self._DoTestFile('216_blob_ext_list_missing.dts', + allow_missing=True) + err = stderr.getvalue() + self.assertRegex(err, "Image 'main-section'.*missing.*: blob-ext") + if __name__ == "__main__": unittest.main() diff --git a/tools/binman/test/215_blob_ext_list.dts b/tools/binman/test/215_blob_ext_list.dts new file mode 100644 index 00000000000..aad2f0300d3 --- /dev/null +++ b/tools/binman/test/215_blob_ext_list.dts @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + blob-ext-list { + filenames = "refcode.bin", "fsp_m.bin"; + }; + }; +}; diff --git a/tools/binman/test/216_blob_ext_list_missing.dts b/tools/binman/test/216_blob_ext_list_missing.dts new file mode 100644 index 00000000000..c02c335c760 --- /dev/null +++ b/tools/binman/test/216_blob_ext_list_missing.dts @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + blob-ext-list { + filenames = "refcode.bin", "missing-file"; + }; + }; +}; -- 2.34.0.rc2.393.gf8c9666880-goog