From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5E87BC433EF for ; Mon, 29 Nov 2021 09:48:39 +0000 (UTC) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 6C18A81EE5; Mon, 29 Nov 2021 10:48:36 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Kotyl23P"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 0B52480934; Mon, 29 Nov 2021 10:48:35 +0100 (CET) Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id B4FB382982 for ; Mon, 29 Nov 2021 10:48:29 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=heiko.thiery@gmail.com Received: by mail-wr1-x434.google.com with SMTP id a9so35373608wrr.8 for ; Mon, 29 Nov 2021 01:48:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=5ce8/IoRLlQlQRPdAxyOtlMBTPbacWpFEIdrf4NaOfg=; b=Kotyl23P7deYSS/0j45tNMSDnM7Ubr4DF+jR89n4ZoFBO0/+bJCpcgHrbQd7c/kUrR sAleoPe3PVuU52F+3RIoWPasZ/KVyNM+ZutloewM4orffISIuIS/FktJZxKMXpx6pkei 4rOxTgaxdI8fbOKJMvjN3AxB1ibq79Ts0gT9VXr5qi7FB1Dp85TgjW7oU2T62+Zhd82I UBETM1Lq/Kn2v34Z2PDi19ySgkgrn1MvPnatZmxDDA4SSZZ3csqMV8oc2MyLOyYIqJmt Ot1pnyvqQcDp//Vk6LoQbRUi4VPv/bDA2umEoP4FzDJwNxCQSvohWPJVvLcpb4BuraBj ShqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=5ce8/IoRLlQlQRPdAxyOtlMBTPbacWpFEIdrf4NaOfg=; b=cDdh/6o4IqyEYn2LKMxnR/LoN1xfLYUBFQ2HJL7G0kg0wAPKd9ZI34Pkr66H0ClSlf vEMGuaxZ0zZ80NjC6+BRP8wYCy1OFmJ6yytBLa46LyJTeG1G7Xw/P/oAyI0bxJV7PnrU YRn/CbXkrVnMMgQ71USnOhPY51iS0nA740CX1/Ka9v6B40ndaod09xBX4tGsiYTfvOZp lqyGnQjegoxQzCNB4AEDuC57nyEKzZn1DIgB+3w+W08cJMmlR8Cz3VpwgRPLjMlVca3W T323DYpiGjYUJ/YbBndxqULuKPdgCkH399Z5fbNxXGp/7/Qp9VfYj/Q5tp+xplg/AFIO hH2A== X-Gm-Message-State: AOAM530OWoWrirGZoyYW4V0hTPdy/I2a+Ov5bcbf3nGtwob8UIDAt0Y/ sF7VIFjQN8I744P9KUxzJzkpeifGaZc= X-Google-Smtp-Source: ABdhPJxxx2d59MWjr+AE4Y5LTAhrNrl2jorwxXxzx5+Lj1YQTMJeQlO1KbJbwjM8S8hHU59m+I0Qhg== X-Received: by 2002:adf:ec45:: with SMTP id w5mr31567272wrn.183.1638179309017; Mon, 29 Nov 2021 01:48:29 -0800 (PST) Received: from t450s.fritz.box (ip1f131103.dynamic.kabel-deutschland.de. [31.19.17.3]) by smtp.gmail.com with ESMTPSA id o25sm15112638wms.17.2021.11.29.01.48.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Nov 2021 01:48:28 -0800 (PST) From: Heiko Thiery To: u-boot@lists.denx.de Cc: Simon Glass , Stefano Babic , Fabio Estevam , Michael Walle , Tom Rini , Wolfgang Denk , Heiko Thiery Subject: [RFC Patch v2] binman: add support for creating dummy files for external blobs Date: Mon, 29 Nov 2021 10:48:13 +0100 Message-Id: <20211129094812.20835-1-heiko.thiery@gmail.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.37 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.2 at phobos.denx.de X-Virus-Status: Clean While converting to binman for an imx8mq board, it has been found that building in the u-boot CI fails. This is because an imx8mq requires an external binary (signed_hdmi_imx8m.bin). If this file cannot be found mkimage fails. To be able to build this board in the u-boot CI a binman option (--fake-ext-blobs) is introduced that can be switched on via the u-boot makefile option BINMAN_FAKE_EXT_BLOBS. With that the needed dummy files are created. Signed-off-by: Heiko Thiery --- v2: - pass allow_fake_blobs to ProcessImage() - set AllowAllowFakeBlob() to images/entries - create fake blob in Entry_blot.ObtainContents() when file is missing and creation is allowed still missing: - unittest - option to set BINMAN_FAKE_EXT_BLOBS in Makefile via environment variable. With that we could simply set this env variable in the CI (gitlab-ci.yml) with adding support to buildman. Makefile | 1 + tools/binman/cmdline.py | 2 ++ tools/binman/control.py | 9 +++++++-- tools/binman/entry.py | 11 +++++++++++ tools/binman/etype/blob.py | 7 +++++++ tools/binman/etype/blob_ext.py | 8 ++++++++ tools/binman/etype/mkimage.py | 9 +++++++++ tools/binman/etype/section.py | 9 +++++++++ 8 files changed, 54 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index f911f70344..1a833a1637 100644 --- a/Makefile +++ b/Makefile @@ -1307,6 +1307,7 @@ cmd_binman = $(srctree)/tools/binman/binman $(if $(BINMAN_DEBUG),-D) \ -a tpl-bss-pad=$(if $(CONFIG_TPL_SEPARATE_BSS),,1) \ -a spl-dtb=$(CONFIG_SPL_OF_REAL) \ -a tpl-dtb=$(CONFIG_SPL_OF_REAL) \ + $(if $(BINMAN_FAKE_EXT_BLOBS),--fake-ext-blobs) \ $(BINMAN_$(@F)) OBJCOPYFLAGS_u-boot.ldr.hex := -I binary -O ihex diff --git a/tools/binman/cmdline.py b/tools/binman/cmdline.py index d6156df408..2b29981cb4 100644 --- a/tools/binman/cmdline.py +++ b/tools/binman/cmdline.py @@ -52,6 +52,8 @@ controlled by a description in the board device tree.''' help='Configuration file (.dtb) to use') build_parser.add_argument('--fake-dtb', action='store_true', help='Use fake device tree contents (for testing only)') + build_parser.add_argument('--fake-ext-blobs', action='store_true', + help='Create fake ext blobs with dummy content (for testing only)') build_parser.add_argument('-i', '--image', type=str, action='append', help='Image filename to build (if not specified, build all)') build_parser.add_argument('-I', '--indir', action='append', diff --git a/tools/binman/control.py b/tools/binman/control.py index 0dbcbc28e9..fa034bddef 100644 --- a/tools/binman/control.py +++ b/tools/binman/control.py @@ -479,7 +479,8 @@ def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): def ProcessImage(image, update_fdt, write_map, get_contents=True, - allow_resize=True, allow_missing=False): + allow_resize=True, allow_missing=False, + allow_fake_blobs=False): """Perform all steps for this image, including checking and # writing it. This means that errors found with a later image will be reported after @@ -495,12 +496,14 @@ def ProcessImage(image, update_fdt, write_map, get_contents=True, allow_resize: True to allow entries to change size (this does a re-pack of the entries), False to raise an exception allow_missing: Allow blob_ext objects to be missing + allow_fake_blobs: Allow blob_ext objects to be faked with dummy files Returns: True if one or more external blobs are missing, False if all are present """ if get_contents: image.SetAllowMissing(allow_missing) + image.SetAllowFakeBlob(allow_fake_blobs) image.GetEntryContents() image.GetEntryOffsets() @@ -629,13 +632,15 @@ def Binman(args): images = PrepareImagesAndDtbs(dtb_fname, args.image, args.update_fdt, use_expanded) + if args.test_section_timeout: # Set the first image to timeout, used in testThreadTimeout() images[list(images.keys())[0]].test_section_timeout = True missing = False for image in images.values(): missing |= ProcessImage(image, args.update_fdt, args.map, - allow_missing=args.allow_missing) + allow_missing=args.allow_missing, + allow_fake_blobs=args.fake_ext_blobs) # Write the updated FDTs to our output files for dtb_item in state.GetAllFdts(): diff --git a/tools/binman/entry.py b/tools/binman/entry.py index 70222718ea..9aa4359be1 100644 --- a/tools/binman/entry.py +++ b/tools/binman/entry.py @@ -70,6 +70,8 @@ class Entry(object): missing: True if this entry is missing its contents allow_missing: Allow children of this entry to be missing (used by subclasses such as Entry_section) + allow_fake: Allow creating a dummy fake file if the blob file is not + available. This is mainly used for testing. external: True if this entry contains an external binary blob """ def __init__(self, section, etype, node, name_prefix=''): @@ -100,6 +102,7 @@ class Entry(object): self.missing = False self.external = False self.allow_missing = False + self.allow_fake = False @staticmethod def Lookup(node_path, etype, expanded): @@ -898,6 +901,14 @@ features to produce new behaviours. # This is meaningless for anything other than sections pass + def SetAllowFakeBlob(self, allow_fake): + """Set whether a section allows to create a fake blob + + Args: + allow_fake: True if allowed, False if not allowed + """ + pass + def CheckMissing(self, missing_list): """Check if any entries in this section have missing external blobs diff --git a/tools/binman/etype/blob.py b/tools/binman/etype/blob.py index fae86ca3ec..0c7469409f 100644 --- a/tools/binman/etype/blob.py +++ b/tools/binman/etype/blob.py @@ -5,6 +5,8 @@ # Entry-type module for blobs, which are binary objects read from files # +import pathlib + from binman.entry import Entry from binman import state from dtoc import fdt_util @@ -36,6 +38,11 @@ class Entry_blob(Entry): self._filename = fdt_util.GetString(self._node, 'filename', self.etype) def ObtainContents(self): + if self.allow_fake and not pathlib.Path(self._filename).is_file(): + tout.Warning("Missing blob '%s', fake it" % self._filename) + with open(self._filename, "wb") as out: + out.truncate(1024) + self._filename = self.GetDefaultFilename() self._pathname = tools.GetInputFilename(self._filename, self.external and self.section.GetAllowMissing()) diff --git a/tools/binman/etype/blob_ext.py b/tools/binman/etype/blob_ext.py index d6b0ca17c3..fba6271de2 100644 --- a/tools/binman/etype/blob_ext.py +++ b/tools/binman/etype/blob_ext.py @@ -26,3 +26,11 @@ class Entry_blob_ext(Entry_blob): def __init__(self, section, etype, node): Entry_blob.__init__(self, section, etype, node) self.external = True + + def SetAllowFakeBlob(self, allow_fake): + """Set whether the entry allows to create a fake blob + + Args: + allow_fake_blob: True if allowed, False if not allowed + """ + self.allow_fake = allow_fake diff --git a/tools/binman/etype/mkimage.py b/tools/binman/etype/mkimage.py index e49977522e..a797d9314e 100644 --- a/tools/binman/etype/mkimage.py +++ b/tools/binman/etype/mkimage.py @@ -61,3 +61,12 @@ class Entry_mkimage(Entry): entry = Entry.Create(self, node) entry.ReadNode() self._mkimage_entries[entry.name] = entry + + def SetAllowFakeBlob(self, allow_fake): + """Set whether the sub nodes allows to create a fake blob + + Args: + allow_fake: True if allowed, False if not allowed + """ + for entry in self._mkimage_entries.values(): + entry.SetAllowFakeBlob(allow_fake) diff --git a/tools/binman/etype/section.py b/tools/binman/etype/section.py index e2949fc916..52cbbe0f73 100644 --- a/tools/binman/etype/section.py +++ b/tools/binman/etype/section.py @@ -689,6 +689,15 @@ class Entry_section(Entry): for entry in self._entries.values(): entry.SetAllowMissing(allow_missing) + def SetAllowFakeBlob(self, allow_fake): + """Set whether a section allows to create a fake blob + + Args: + allow_fake_blob: True if allowed, False if not allowed + """ + for entry in self._entries.values(): + entry.SetAllowFakeBlob(allow_fake) + def CheckMissing(self, missing_list): """Check if any entries in this section have missing external blobs -- 2.20.1