[PATCH] binman: Avoid requiring a home directory on startup

[PATCH] binman: Avoid requiring a home directory on startup

[Simon Glass]

This is needed to download tools, but we may not need to do this. At
present binman fails to start if HOME is not set.

Use the current directory as a default to avoid this.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 tools/binman/bintool.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/binman/bintool.py b/tools/binman/bintool.py
index 8fda13ff012..f460243e796 100644
--- a/tools/binman/bintool.py
+++ b/tools/binman/bintool.py
@@ -43,7 +43,7 @@ FETCH_NAMES = {
 # Status of tool fetching
 FETCHED, FAIL, PRESENT, STATUS_COUNT = range(4)
 
-DOWNLOAD_DESTDIR = os.path.join(os.getenv('HOME'), 'bin')
+DOWNLOAD_DESTDIR = os.path.expanduser('~/bin')
 
 class Bintool:
     """Tool which operates on binaries to help produce entry contents
-- 
2.39.1.581.gbfd45094c4-goog

Re: [PATCH] binman: Avoid requiring a home directory on startup

[Mark Kettenis]

> From: Simon Glass <sjg@chromium.org>
> Date: Fri, 10 Feb 2023 13:59:46 -0700
> 
> This is needed to download tools, but we may not need to do this. At
> present binman fails to start if HOME is not set.
> 
> Use the current directory as a default to avoid this.
> 
> Signed-off-by: Simon Glass <sjg@chromium.org>

Not new, but $HOME/bin seems like an *extremely* poor choice for a
directory to download random binaries from the internet into as this
directory is often part of a user's $PATH.


> ---
> 
>  tools/binman/bintool.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/binman/bintool.py b/tools/binman/bintool.py
> index 8fda13ff012..f460243e796 100644
> --- a/tools/binman/bintool.py
> +++ b/tools/binman/bintool.py
> @@ -43,7 +43,7 @@ FETCH_NAMES = {
>  # Status of tool fetching
>  FETCHED, FAIL, PRESENT, STATUS_COUNT = range(4)
>  
> -DOWNLOAD_DESTDIR = os.path.join(os.getenv('HOME'), 'bin')
> +DOWNLOAD_DESTDIR = os.path.expanduser('~/bin')
>  
>  class Bintool:
>      """Tool which operates on binaries to help produce entry contents
> -- 
> 2.39.1.581.gbfd45094c4-goog
> 
> 

Re: [PATCH] binman: Avoid requiring a home directory on startup

[Simon Glass]

Hi Mark,

On Fri, 10 Feb 2023 at 14:23, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
>
> > From: Simon Glass <sjg@chromium.org>
> > Date: Fri, 10 Feb 2023 13:59:46 -0700
> >
> > This is needed to download tools, but we may not need to do this. At
> > present binman fails to start if HOME is not set.
> >
> > Use the current directory as a default to avoid this.
> >
> > Signed-off-by: Simon Glass <sjg@chromium.org>
>
> Not new, but $HOME/bin seems like an *extremely* poor choice for a
> directory to download random binaries from the internet into as this
> directory is often part of a user's $PATH.

But isn't that the point? It is designed to bring in tools for use by the user.

Regards,
Simon

Re: [PATCH] binman: Avoid requiring a home directory on startup

[Mike Frysinger]

considering, iiuc, the user has to execute the command to opt-in to
installing the programs, putting it into a dir that's in $PATH is
reasonable

i would quibble that ~/bin is archaic and everyone should be using
~/.local/bin nowadays.  but that can be a followup.
https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
-mike

Re: [PATCH] binman: Avoid requiring a home directory on startup

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 1055 bytes --]

On Tue, Feb 14, 2023 at 12:48:56PM -0700, Simon Glass wrote:
> Hi Mark,
> 
> On Fri, 10 Feb 2023 at 14:23, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> >
> > > From: Simon Glass <sjg@chromium.org>
> > > Date: Fri, 10 Feb 2023 13:59:46 -0700
> > >
> > > This is needed to download tools, but we may not need to do this. At
> > > present binman fails to start if HOME is not set.
> > >
> > > Use the current directory as a default to avoid this.
> > >
> > > Signed-off-by: Simon Glass <sjg@chromium.org>
> >
> > Not new, but $HOME/bin seems like an *extremely* poor choice for a
> > directory to download random binaries from the internet into as this
> > directory is often part of a user's $PATH.
> 
> But isn't that the point? It is designed to bring in tools for use by the user.

Downloading things from the internet and putting them in to the default
PATH always and forever is also kinda not great? buildman --fetch-arch
places things elsewhere and some sub directory of ~/.buildman-toolchains
is less risky.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH] binman: Avoid requiring a home directory on startup

[Mike Frysinger]

On Tue, Feb 14, 2023 at 3:08 PM Tom Rini <trini@konsulko.com> wrote:
> Downloading things from the internet and putting them in to the default
> PATH always and forever is also kinda not great?

you just described a standard distribution.  this is like literally
how all of them work.  not to mention every other language-specific
distro tool out there (e.g. Python pip, Perl cpan, Go, etc...).

maybe you'd like more guarantees on top (e.g. signature verification)
which is reasonable.

but to be clear, this script is already merged & in the tree, so your
feedback doesn't block this patch.
-mike

Re: [PATCH] binman: Avoid requiring a home directory on startup

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 1279 bytes --]

On Tue, Feb 14, 2023 at 03:12:46PM -0500, Mike Frysinger wrote:
> On Tue, Feb 14, 2023 at 3:08 PM Tom Rini <trini@konsulko.com> wrote:
> > Downloading things from the internet and putting them in to the default
> > PATH always and forever is also kinda not great?
> 
> you just described a standard distribution.  this is like literally
> how all of them work.  not to mention every other language-specific
> distro tool out there (e.g. Python pip, Perl cpan, Go, etc...).
> 
> maybe you'd like more guarantees on top (e.g. signature verification)
> which is reasonable.
> 
> but to be clear, this script is already merged & in the tree, so your
> feedback doesn't block this patch.

Yes, exactly. This is a fix on top of what we do today, so it should go
in. But modern distributions only install signed packages, and
language-specific tools tend to be a hive of bad examples. Looking over
binman right now, I see that we're either using apt (and oh, there's
"aot" typo in one spot) or downloading from a known Google drive, for
only a few less common tools.

So yes, I would like to see some ideas on how to improve things in the
future so we aren't putting the binaries somewhere that's not a default
(or frequently common) PATH location.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH] binman: Avoid requiring a home directory on startup

[Simon Glass]

Hi Tom,

On Tue, 14 Feb 2023 at 13:27, Tom Rini <trini@konsulko.com> wrote:
>
> On Tue, Feb 14, 2023 at 03:12:46PM -0500, Mike Frysinger wrote:
> > On Tue, Feb 14, 2023 at 3:08 PM Tom Rini <trini@konsulko.com> wrote:
> > > Downloading things from the internet and putting them in to the default
> > > PATH always and forever is also kinda not great?
> >
> > you just described a standard distribution.  this is like literally
> > how all of them work.  not to mention every other language-specific
> > distro tool out there (e.g. Python pip, Perl cpan, Go, etc...).
> >
> > maybe you'd like more guarantees on top (e.g. signature verification)
> > which is reasonable.
> >
> > but to be clear, this script is already merged & in the tree, so your
> > feedback doesn't block this patch.
>
> Yes, exactly. This is a fix on top of what we do today, so it should go
> in. But modern distributions only install signed packages, and
> language-specific tools tend to be a hive of bad examples. Looking over
> binman right now, I see that we're either using apt (and oh, there's
> "aot" typo in one spot) or downloading from a known Google drive, for
> only a few less common tools.
>
> So yes, I would like to see some ideas on how to improve things in the
> future so we aren't putting the binaries somewhere that's not a default
> (or frequently common) PATH location.

Are you thinking they should go in ~/.binman-tools or something like
that? Then we would need to tell people to add it to their path. But
we could make binman look there automatically.

Regards,
Simon

Re: [PATCH] binman: Avoid requiring a home directory on startup

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 1933 bytes --]

On Thu, Feb 16, 2023 at 05:12:33PM -0700, Simon Glass wrote:
> Hi Tom,
> 
> On Tue, 14 Feb 2023 at 13:27, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Tue, Feb 14, 2023 at 03:12:46PM -0500, Mike Frysinger wrote:
> > > On Tue, Feb 14, 2023 at 3:08 PM Tom Rini <trini@konsulko.com> wrote:
> > > > Downloading things from the internet and putting them in to the default
> > > > PATH always and forever is also kinda not great?
> > >
> > > you just described a standard distribution.  this is like literally
> > > how all of them work.  not to mention every other language-specific
> > > distro tool out there (e.g. Python pip, Perl cpan, Go, etc...).
> > >
> > > maybe you'd like more guarantees on top (e.g. signature verification)
> > > which is reasonable.
> > >
> > > but to be clear, this script is already merged & in the tree, so your
> > > feedback doesn't block this patch.
> >
> > Yes, exactly. This is a fix on top of what we do today, so it should go
> > in. But modern distributions only install signed packages, and
> > language-specific tools tend to be a hive of bad examples. Looking over
> > binman right now, I see that we're either using apt (and oh, there's
> > "aot" typo in one spot) or downloading from a known Google drive, for
> > only a few less common tools.
> >
> > So yes, I would like to see some ideas on how to improve things in the
> > future so we aren't putting the binaries somewhere that's not a default
> > (or frequently common) PATH location.
> 
> Are you thinking they should go in ~/.binman-tools or something like
> that? Then we would need to tell people to add it to their path. But
> we could make binman look there automatically.

We should document that it's where we're putting stuff, not so much
"tell" them, unless you mean as a note when downloading.  But yes,
~/.binman-tools sounds reasonable.  Maybe a flag to point elsewhere?

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH] binman: Avoid requiring a home directory on startup

[Simon Glass]

Hi Tom,

On Thu, 16 Feb 2023 at 17:19, Tom Rini <trini@konsulko.com> wrote:
>
> On Thu, Feb 16, 2023 at 05:12:33PM -0700, Simon Glass wrote:
> > Hi Tom,
> >
> > On Tue, 14 Feb 2023 at 13:27, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Tue, Feb 14, 2023 at 03:12:46PM -0500, Mike Frysinger wrote:
> > > > On Tue, Feb 14, 2023 at 3:08 PM Tom Rini <trini@konsulko.com> wrote:
> > > > > Downloading things from the internet and putting them in to the default
> > > > > PATH always and forever is also kinda not great?
> > > >
> > > > you just described a standard distribution.  this is like literally
> > > > how all of them work.  not to mention every other language-specific
> > > > distro tool out there (e.g. Python pip, Perl cpan, Go, etc...).
> > > >
> > > > maybe you'd like more guarantees on top (e.g. signature verification)
> > > > which is reasonable.
> > > >
> > > > but to be clear, this script is already merged & in the tree, so your
> > > > feedback doesn't block this patch.
> > >
> > > Yes, exactly. This is a fix on top of what we do today, so it should go
> > > in. But modern distributions only install signed packages, and
> > > language-specific tools tend to be a hive of bad examples. Looking over
> > > binman right now, I see that we're either using apt (and oh, there's
> > > "aot" typo in one spot) or downloading from a known Google drive, for
> > > only a few less common tools.
> > >
> > > So yes, I would like to see some ideas on how to improve things in the
> > > future so we aren't putting the binaries somewhere that's not a default
> > > (or frequently common) PATH location.
> >
> > Are you thinking they should go in ~/.binman-tools or something like
> > that? Then we would need to tell people to add it to their path. But
> > we could make binman look there automatically.
>
> We should document that it's where we're putting stuff, not so much
> "tell" them, unless you mean as a note when downloading.  But yes,
> ~/.binman-tools sounds reasonable.  Maybe a flag to point elsewhere?

OK I will take a look.

Regards,
Simon

Re: [PATCH] binman: Avoid requiring a home directory on startup

[Quentin Schulz]

Hi all,

On 2/17/23 03:55, Simon Glass wrote:
> Hi Tom,
> 
> On Thu, 16 Feb 2023 at 17:19, Tom Rini <trini@konsulko.com> wrote:
>>
>> On Thu, Feb 16, 2023 at 05:12:33PM -0700, Simon Glass wrote:
>>> Hi Tom,
>>>
>>> On Tue, 14 Feb 2023 at 13:27, Tom Rini <trini@konsulko.com> wrote:
>>>>
>>>> On Tue, Feb 14, 2023 at 03:12:46PM -0500, Mike Frysinger wrote:
>>>>> On Tue, Feb 14, 2023 at 3:08 PM Tom Rini <trini@konsulko.com> wrote:
>>>>>> Downloading things from the internet and putting them in to the default
>>>>>> PATH always and forever is also kinda not great?
>>>>>
>>>>> you just described a standard distribution.  this is like literally
>>>>> how all of them work.  not to mention every other language-specific
>>>>> distro tool out there (e.g. Python pip, Perl cpan, Go, etc...).
>>>>>
>>>>> maybe you'd like more guarantees on top (e.g. signature verification)
>>>>> which is reasonable.
>>>>>
>>>>> but to be clear, this script is already merged & in the tree, so your
>>>>> feedback doesn't block this patch.
>>>>
>>>> Yes, exactly. This is a fix on top of what we do today, so it should go
>>>> in. But modern distributions only install signed packages, and
>>>> language-specific tools tend to be a hive of bad examples. Looking over
>>>> binman right now, I see that we're either using apt (and oh, there's
>>>> "aot" typo in one spot) or downloading from a known Google drive, for
>>>> only a few less common tools.
>>>>
>>>> So yes, I would like to see some ideas on how to improve things in the
>>>> future so we aren't putting the binaries somewhere that's not a default
>>>> (or frequently common) PATH location.
>>>
>>> Are you thinking they should go in ~/.binman-tools or something like
>>> that? Then we would need to tell people to add it to their path. But
>>> we could make binman look there automatically.
>>
>> We should document that it's where we're putting stuff, not so much
>> "tell" them, unless you mean as a note when downloading.  But yes,
>> ~/.binman-tools sounds reasonable.  Maybe a flag to point elsewhere?
> 
> OK I will take a look.
> 

I think this should be directly put into the output/build directory used 
by U-Boot, because what happens when you have two U-Boot git repos with 
different version requirements for those host tools? Then you need to 
make sure you're not building both at the same time, that you update 
them properly before each build, etc.

We also have an issue with buildman creating (image) temporary files in 
the current git repo which makes people unhappy because they then get a 
dirty tree because we tend to forget to update the .gitignore. What 
about building out-of-tree by default like Buildroot does? e.g. with 
make O=$PWD/output/ ?

Cheers,
Quentin

Re: [PATCH] binman: Avoid requiring a home directory on startup

[Simon Glass]

Hi Quentin,

On Fri, 17 Feb 2023 at 05:21, Quentin Schulz
<quentin.schulz@theobroma-systems.com> wrote:
>
> Hi all,
>
> On 2/17/23 03:55, Simon Glass wrote:
> > Hi Tom,
> >
> > On Thu, 16 Feb 2023 at 17:19, Tom Rini <trini@konsulko.com> wrote:
> >>
> >> On Thu, Feb 16, 2023 at 05:12:33PM -0700, Simon Glass wrote:
> >>> Hi Tom,
> >>>
> >>> On Tue, 14 Feb 2023 at 13:27, Tom Rini <trini@konsulko.com> wrote:
> >>>>
> >>>> On Tue, Feb 14, 2023 at 03:12:46PM -0500, Mike Frysinger wrote:
> >>>>> On Tue, Feb 14, 2023 at 3:08 PM Tom Rini <trini@konsulko.com> wrote:
> >>>>>> Downloading things from the internet and putting them in to the default
> >>>>>> PATH always and forever is also kinda not great?
> >>>>>
> >>>>> you just described a standard distribution.  this is like literally
> >>>>> how all of them work.  not to mention every other language-specific
> >>>>> distro tool out there (e.g. Python pip, Perl cpan, Go, etc...).
> >>>>>
> >>>>> maybe you'd like more guarantees on top (e.g. signature verification)
> >>>>> which is reasonable.
> >>>>>
> >>>>> but to be clear, this script is already merged & in the tree, so your
> >>>>> feedback doesn't block this patch.
> >>>>
> >>>> Yes, exactly. This is a fix on top of what we do today, so it should go
> >>>> in. But modern distributions only install signed packages, and
> >>>> language-specific tools tend to be a hive of bad examples. Looking over
> >>>> binman right now, I see that we're either using apt (and oh, there's
> >>>> "aot" typo in one spot) or downloading from a known Google drive, for
> >>>> only a few less common tools.
> >>>>
> >>>> So yes, I would like to see some ideas on how to improve things in the
> >>>> future so we aren't putting the binaries somewhere that's not a default
> >>>> (or frequently common) PATH location.
> >>>
> >>> Are you thinking they should go in ~/.binman-tools or something like
> >>> that? Then we would need to tell people to add it to their path. But
> >>> we could make binman look there automatically.
> >>
> >> We should document that it's where we're putting stuff, not so much
> >> "tell" them, unless you mean as a note when downloading.  But yes,
> >> ~/.binman-tools sounds reasonable.  Maybe a flag to point elsewhere?
> >
> > OK I will take a look.
> >
>
> I think this should be directly put into the output/build directory used
> by U-Boot, because what happens when you have two U-Boot git repos with
> different version requirements for those host tools? Then you need to
> make sure you're not building both at the same time, that you update
> them properly before each build, etc.

My advice: *Don't do that*

So far as binman is concerned, a tool is a tool. Tools should be
backwards compatible so updating to the new one should fix all the
problems.

The problem with using the output dir is we then have to download them
for each build, or cache them somewhere. To my mind, the 'binman tool'
feature is a convenience to reduce the pain involved in obtaining
tools needed to build. It is a not a panacea for strange situations.

>
> We also have an issue with buildman creating (image) temporary files in
> the current git repo which makes people unhappy because they then get a
> dirty tree because we tend to forget to update the .gitignore. What
> about building out-of-tree by default like Buildroot does? e.g. with
> make O=$PWD/output/ ?

I don't tend to use in-tree builds, but, yes, this is a problem.

I believe the best solution is, I think, to make binman put its
intermediate files in a subdir. At present there is no distinction
between output files and files produced along the way for debugging
purposes. This has been on my mind for a while and has been discussed
at least once before, too.

We do want the output files to be in the build directory. It would be
annoying to put u-boot.itb in a subdir.

Binman should all track all the files it produced (both final output
and intermediate) so that info is available.

For now, the solution is to use 'make O=xxx' as you say. With that,
binman should not write anything to the current dir. If it does, that
is a bug.

Regards,
Simon

Re: [PATCH] binman: Avoid requiring a home directory on startup

[Quentin Schulz]

Hi Simon,

On 2/18/23 00:49, Simon Glass wrote:
> Hi Quentin,
> 
> On Fri, 17 Feb 2023 at 05:21, Quentin Schulz
> <quentin.schulz@theobroma-systems.com> wrote:
>>
>> Hi all,
>>
>> On 2/17/23 03:55, Simon Glass wrote:
>>> Hi Tom,
>>>
>>> On Thu, 16 Feb 2023 at 17:19, Tom Rini <trini@konsulko.com> wrote:
>>>>
>>>> On Thu, Feb 16, 2023 at 05:12:33PM -0700, Simon Glass wrote:
>>>>> Hi Tom,
>>>>>
>>>>> On Tue, 14 Feb 2023 at 13:27, Tom Rini <trini@konsulko.com> wrote:
>>>>>>
>>>>>> On Tue, Feb 14, 2023 at 03:12:46PM -0500, Mike Frysinger wrote:
>>>>>>> On Tue, Feb 14, 2023 at 3:08 PM Tom Rini <trini@konsulko.com> wrote:
>>>>>>>> Downloading things from the internet and putting them in to the default
>>>>>>>> PATH always and forever is also kinda not great?
>>>>>>>
>>>>>>> you just described a standard distribution.  this is like literally
>>>>>>> how all of them work.  not to mention every other language-specific
>>>>>>> distro tool out there (e.g. Python pip, Perl cpan, Go, etc...).
>>>>>>>
>>>>>>> maybe you'd like more guarantees on top (e.g. signature verification)
>>>>>>> which is reasonable.
>>>>>>>
>>>>>>> but to be clear, this script is already merged & in the tree, so your
>>>>>>> feedback doesn't block this patch.
>>>>>>
>>>>>> Yes, exactly. This is a fix on top of what we do today, so it should go
>>>>>> in. But modern distributions only install signed packages, and
>>>>>> language-specific tools tend to be a hive of bad examples. Looking over
>>>>>> binman right now, I see that we're either using apt (and oh, there's
>>>>>> "aot" typo in one spot) or downloading from a known Google drive, for
>>>>>> only a few less common tools.
>>>>>>
>>>>>> So yes, I would like to see some ideas on how to improve things in the
>>>>>> future so we aren't putting the binaries somewhere that's not a default
>>>>>> (or frequently common) PATH location.
>>>>>
>>>>> Are you thinking they should go in ~/.binman-tools or something like
>>>>> that? Then we would need to tell people to add it to their path. But
>>>>> we could make binman look there automatically.
>>>>
>>>> We should document that it's where we're putting stuff, not so much
>>>> "tell" them, unless you mean as a note when downloading.  But yes,
>>>> ~/.binman-tools sounds reasonable.  Maybe a flag to point elsewhere?
>>>
>>> OK I will take a look.
>>>
>>
>> I think this should be directly put into the output/build directory used
>> by U-Boot, because what happens when you have two U-Boot git repos with
>> different version requirements for those host tools? Then you need to
>> make sure you're not building both at the same time, that you update
>> them properly before each build, etc.
> 
> My advice: *Don't do that*
> 
> So far as binman is concerned, a tool is a tool. Tools should be
> backwards compatible so updating to the new one should fix all the
> problems.
> 

That's a very bold claim :)

> The problem with using the output dir is we then have to download them
> for each build, or cache them somewhere. To my mind, the 'binman tool'
> feature is a convenience to reduce the pain involved in obtaining
> tools needed to build. It is a not a panacea for strange situations.
> 

Have the default in the build directory and allow the user to define an 
out-of-tree directory if they want to cache them somewhere? Similar to 
Yocto with SSTATE_DIR/DL_DIR, Buildroot with BR2_DL_DIR for example.

Cheers,
Quentin

Re: [PATCH] binman: Avoid requiring a home directory on startup

[Simon Glass]

Hi Quentin,

On Mon, 20 Feb 2023 at 04:15, Quentin Schulz
<quentin.schulz@theobroma-systems.com> wrote:
>
> Hi Simon,
>
> On 2/18/23 00:49, Simon Glass wrote:
> > Hi Quentin,
> >
> > On Fri, 17 Feb 2023 at 05:21, Quentin Schulz
> > <quentin.schulz@theobroma-systems.com> wrote:
> >>
> >> Hi all,
> >>
> >> On 2/17/23 03:55, Simon Glass wrote:
> >>> Hi Tom,
> >>>
> >>> On Thu, 16 Feb 2023 at 17:19, Tom Rini <trini@konsulko.com> wrote:
> >>>>
> >>>> On Thu, Feb 16, 2023 at 05:12:33PM -0700, Simon Glass wrote:
> >>>>> Hi Tom,
> >>>>>
> >>>>> On Tue, 14 Feb 2023 at 13:27, Tom Rini <trini@konsulko.com> wrote:
> >>>>>>
> >>>>>> On Tue, Feb 14, 2023 at 03:12:46PM -0500, Mike Frysinger wrote:
> >>>>>>> On Tue, Feb 14, 2023 at 3:08 PM Tom Rini <trini@konsulko.com> wrote:
> >>>>>>>> Downloading things from the internet and putting them in to the default
> >>>>>>>> PATH always and forever is also kinda not great?
> >>>>>>>
> >>>>>>> you just described a standard distribution.  this is like literally
> >>>>>>> how all of them work.  not to mention every other language-specific
> >>>>>>> distro tool out there (e.g. Python pip, Perl cpan, Go, etc...).
> >>>>>>>
> >>>>>>> maybe you'd like more guarantees on top (e.g. signature verification)
> >>>>>>> which is reasonable.
> >>>>>>>
> >>>>>>> but to be clear, this script is already merged & in the tree, so your
> >>>>>>> feedback doesn't block this patch.
> >>>>>>
> >>>>>> Yes, exactly. This is a fix on top of what we do today, so it should go
> >>>>>> in. But modern distributions only install signed packages, and
> >>>>>> language-specific tools tend to be a hive of bad examples. Looking over
> >>>>>> binman right now, I see that we're either using apt (and oh, there's
> >>>>>> "aot" typo in one spot) or downloading from a known Google drive, for
> >>>>>> only a few less common tools.
> >>>>>>
> >>>>>> So yes, I would like to see some ideas on how to improve things in the
> >>>>>> future so we aren't putting the binaries somewhere that's not a default
> >>>>>> (or frequently common) PATH location.
> >>>>>
> >>>>> Are you thinking they should go in ~/.binman-tools or something like
> >>>>> that? Then we would need to tell people to add it to their path. But
> >>>>> we could make binman look there automatically.
> >>>>
> >>>> We should document that it's where we're putting stuff, not so much
> >>>> "tell" them, unless you mean as a note when downloading.  But yes,
> >>>> ~/.binman-tools sounds reasonable.  Maybe a flag to point elsewhere?
> >>>
> >>> OK I will take a look.
> >>>
> >>
> >> I think this should be directly put into the output/build directory used
> >> by U-Boot, because what happens when you have two U-Boot git repos with
> >> different version requirements for those host tools? Then you need to
> >> make sure you're not building both at the same time, that you update
> >> them properly before each build, etc.
> >
> > My advice: *Don't do that*
> >
> > So far as binman is concerned, a tool is a tool. Tools should be
> > backwards compatible so updating to the new one should fix all the
> > problems.
> >
>
> That's a very bold claim :)
>
> > The problem with using the output dir is we then have to download them
> > for each build, or cache them somewhere. To my mind, the 'binman tool'
> > feature is a convenience to reduce the pain involved in obtaining
> > tools needed to build. It is a not a panacea for strange situations.
> >
>
> Have the default in the build directory and allow the user to define an
> out-of-tree directory if they want to cache them somewhere? Similar to
> Yocto with SSTATE_DIR/DL_DIR, Buildroot with BR2_DL_DIR for example.

OK, but why do you want to use the build directory at all? It seems
like a hassle to set up. With the series I sent it is automatic and
all that is needed is to add ~/.binman-tools to your path.

Regards,
Simon

Re: [PATCH] binman: Avoid requiring a home directory on startup

[Simon Glass]

Hi Quentin,

On Mon, 20 Feb 2023 at 04:15, Quentin Schulz
<quentin.schulz@theobroma-systems.com> wrote:
>
> Hi Simon,
>
> On 2/18/23 00:49, Simon Glass wrote:
> > Hi Quentin,
> >
> > On Fri, 17 Feb 2023 at 05:21, Quentin Schulz
> > <quentin.schulz@theobroma-systems.com> wrote:
> >>
> >> Hi all,
> >>
> >> On 2/17/23 03:55, Simon Glass wrote:
> >>> Hi Tom,
> >>>
> >>> On Thu, 16 Feb 2023 at 17:19, Tom Rini <trini@konsulko.com> wrote:
> >>>>
> >>>> On Thu, Feb 16, 2023 at 05:12:33PM -0700, Simon Glass wrote:
> >>>>> Hi Tom,
> >>>>>
> >>>>> On Tue, 14 Feb 2023 at 13:27, Tom Rini <trini@konsulko.com> wrote:
> >>>>>>
> >>>>>> On Tue, Feb 14, 2023 at 03:12:46PM -0500, Mike Frysinger wrote:
> >>>>>>> On Tue, Feb 14, 2023 at 3:08 PM Tom Rini <trini@konsulko.com> wrote:
> >>>>>>>> Downloading things from the internet and putting them in to the default
> >>>>>>>> PATH always and forever is also kinda not great?
> >>>>>>>
> >>>>>>> you just described a standard distribution.  this is like literally
> >>>>>>> how all of them work.  not to mention every other language-specific
> >>>>>>> distro tool out there (e.g. Python pip, Perl cpan, Go, etc...).
> >>>>>>>
> >>>>>>> maybe you'd like more guarantees on top (e.g. signature verification)
> >>>>>>> which is reasonable.
> >>>>>>>
> >>>>>>> but to be clear, this script is already merged & in the tree, so your
> >>>>>>> feedback doesn't block this patch.
> >>>>>>
> >>>>>> Yes, exactly. This is a fix on top of what we do today, so it should go
> >>>>>> in. But modern distributions only install signed packages, and
> >>>>>> language-specific tools tend to be a hive of bad examples. Looking over
> >>>>>> binman right now, I see that we're either using apt (and oh, there's
> >>>>>> "aot" typo in one spot) or downloading from a known Google drive, for
> >>>>>> only a few less common tools.
> >>>>>>
> >>>>>> So yes, I would like to see some ideas on how to improve things in the
> >>>>>> future so we aren't putting the binaries somewhere that's not a default
> >>>>>> (or frequently common) PATH location.
> >>>>>
> >>>>> Are you thinking they should go in ~/.binman-tools or something like
> >>>>> that? Then we would need to tell people to add it to their path. But
> >>>>> we could make binman look there automatically.
> >>>>
> >>>> We should document that it's where we're putting stuff, not so much
> >>>> "tell" them, unless you mean as a note when downloading.  But yes,
> >>>> ~/.binman-tools sounds reasonable.  Maybe a flag to point elsewhere?
> >>>
> >>> OK I will take a look.
> >>>
> >>
> >> I think this should be directly put into the output/build directory used
> >> by U-Boot, because what happens when you have two U-Boot git repos with
> >> different version requirements for those host tools? Then you need to
> >> make sure you're not building both at the same time, that you update
> >> them properly before each build, etc.
> >
> > My advice: *Don't do that*
> >
> > So far as binman is concerned, a tool is a tool. Tools should be
> > backwards compatible so updating to the new one should fix all the
> > problems.
> >
>
> That's a very bold claim :)
>
> > The problem with using the output dir is we then have to download them
> > for each build, or cache them somewhere. To my mind, the 'binman tool'
> > feature is a convenience to reduce the pain involved in obtaining
> > tools needed to build. It is a not a panacea for strange situations.
> >
>
> Have the default in the build directory and allow the user to define an
> out-of-tree directory if they want to cache them somewhere? Similar to
> Yocto with SSTATE_DIR/DL_DIR, Buildroot with BR2_DL_DIR for example.

OK, but why do you want to use the build directory at all? It seems
like a hassle to set up. With the series I sent it is automatic and
all that is needed is to add ~/.binman-tools to your path.

Regards,
Simon

Applied to u-boot-dm/next, thanks!