Re: [SPAM] Re: Replace make-fit-atf.py with binman. Was: migrate u-boot-rockchip.bin to binman and generate an image for SPI

From: Simon Glass <sjg@chromium.org>
To: Xavier Drudis Ferran <xdrudis@tinet.cat>
Cc: "Quentin Schulz" <quentin.schulz@theobroma-systems.com>,
	"Quentin Schulz" <foss+uboot@0leil.net>,
	"Kever Yang" <kever.yang@rock-chips.com>,
	"Andy Yan" <andy.yan@rock-chips.com>,
	"Lin Huang" <hl@rock-chips.com>, 陈健洪 <chenjh@rock-chips.com>,
	"Manivannan Sadhasivam" <manivannan.sadhasivam@linaro.org>,
	"Nick Xie" <nick@khadas.com>, "Johan Jonker" <jbx6244@gmail.com>,
	deepakdas.linux@gmail.com, linux@alxd.me,
	"U-Boot Mailing List" <u-boot@lists.denx.de>
Subject: Re: [SPAM] Re: Replace make-fit-atf.py with binman. Was: migrate u-boot-rockchip.bin to binman and generate an image for SPI
Date: Tue, 2 Aug 2022 06:41:40 -0600	[thread overview]
Message-ID: <CAPnjgZ1er9eD2jBa_0PRUvt35+jAkqPzZx2RsHKHTyk4Ums3Bg@mail.gmail.com> (raw)
In-Reply-To: <YujdQcwc+I0/iOew@begut>

Hi Xavier,

On Tue, 2 Aug 2022 at 02:19, Xavier Drudis Ferran <xdrudis@tinet.cat> wrote:
>
> El Mon, Aug 01, 2022 at 01:13:27PM -0600, Simon Glass deia:
> > >
> > > Am I completely lost or does what I want to do make some kind of sense?
> >
> > Well I still feel that we should handle this properly in binman.
> >
>
> I respect your feelings but would you care explaining the goal of binman?
> I first thought it was about handling binaries (things like copy binary content,
> aligning it, padding, extracting and selecting parts from one binary to form
> the final image...). A little like a linker but for other kinds of binary files,
> with maybe headers but no symbol resolution and all. Maybe more like GNU ar but
> for images instead of executables.
>
> But then you seem to say it should handle dependencies between build files and
> be called only once ? So should it eventually become a sort of make itself ?
>
> > I don't even think we need to allow images to be incorporated in other
> > images. We can probably just allow a section to have a filename, a bit
> > like this patch [1].
> >
>
> There's a image r-sd and an image r-spi. Both need to have inside the
> same binary itb, packed differently. If r-sd has a section with itb
> and writes that section to a file, r-spi still needs to incorporate
> that section as a binary, and to do that it needs to know the other
> image has already written its section to a file or built into
> somewhere in memory. So there're dependencies, and synchronisation if
> you want to built images in parallel. make can handle that if you call
> binman multiple times. But if you don't want that, then you need to
> either include dependency management into binman (turning it into a
> little make working from dts subtrees instead of makefiles) or forbid
> that binaries used by binman could be images produced by binman, and
> then you need other binary manipulation tools (like make_fit_atf.py,
> which is basically what you have now, but that's apparently not
> desired either).
>
> If it was just me who feels lost, then it'd be just that I don't get it,
> but Quentin is saying something similar.

It seems we need a lot more guidance here. I can help write more into
the packaging docs, perhaps:

https://u-boot.readthedocs.io/en/latest/develop/package/binman.html#motivation

Can you please suggest a few questions to answer?

>
> In any case, don't forget what Jerome said about tee.bin needing different
> parsing than split-elf does in binman. The sections of tee.bin can also
> end up in u-boot.itb. Adding that to binman would maybe make more sense
> that what I was trying to add, in the sense that binman be a swiss army knive
> of parsing and building binaries. But then you'd still have the problem
> that if images built by binman cannot be incorporated in images built by binman,
> then something else has to build u-boot.itb. And that something else is
> currently make_fit_atf.py and works fine, so why the trouble ?

Because we have lots of scripts with no tests and it will proliferate
even more. Binman is data-driven, has tests and allows us to build
common functionality used by all SoCs.

>
> We're kind of running in circles, and for me it would be helpful to
> understand the goal and scope of binman to understand what would be
> desirable, because the status quo seems preferable to some kind of
> feature creep in binman that I can't seem to reconcile with the
> philosophy of one tool doing one task and well.
>
> Maybe most people here already understand all this, but sometimes
> explaining the basics to a foreign bystander who knows little of
> the subject (hello, that's me) can help people sort out thoughts?
>
> Thanks, and sorry for the wall of text.

Have you looked at osfc binman talk from a few years ago?

Binman is not to replace make. The dependency we are talking about is
between different images generated by Binman. Part of the problem is
that you know what you want, but it is a bit foggy to me.

To move forward, can I suggest you send a patch with a binman
description file containing what you want. Obviously it won't work,
due to the dependencies, but I can then figure out how to enable that
in binman. Can you try that?

Regards,
Simon