Re: [bitbake-devel] [PATCH 0/3] fetch2: ssh: fixups and minimal checkstatus implementation

Re: [bitbake-devel] [PATCH 0/3] fetch2: ssh: fixups and minimal checkstatus implementation

[Richard Purdie]

On Wed, 2022-03-02 at 20:50 +0100, Daniel Wagenknecht wrote:
> Hello!
> 
> I recently tried using a sstate mirror accessible to me only via ssh. The
> first two patches fix problems I encountered with the ssh fetcher
> implementation in general, the last patch provides a very basic implementation
> for the checkstatus method needed for sstate mirrors.
> 
> Consider the last patch as a request for comments and feel free to merge the
> first two independently.
> 
> There's been some discussion on sstate mirrors via ssh already. Especially this
> point
> > Performance of that function is key and creating a new ssh connection
> > to check for each sstate artefact may make the build prohibitively
> > slow. There are some connection caching mechanisms present which may
> > help with that.
> Richard makes in https://lists.yoctoproject.org/g/yocto/message/47769 is not
> yet adressed with my implementation. With connection caching configured in my
> workstations ssh config (SSH Options: ControlMaster, ControlPath,
> ControlPersist) the performance was good. For caching the connection inside
> bitbake I'm not sure how to approach this. Technically passing the necessary
> options to each ssh invocation should be trivial, but I'm not sure if it aligns
> with users expectations. I'd probably be suprised if I moved my ssh key (or
> removed the hardware token) and ssh access via bitbake continued to work. How
> is connection caching for other fetchers configured? Do we have a directory
> with sufficiently restrictive permissions for storing the
> cached-connection-access-files (SSH Option: ControlPath)? What are reasonable
> timeouts to use?

I guess the patches are ok in that they don't really break anything and whilst
I'm not convinced about using sstate over ssh but if people want to do that...

For the caching, see the checkstatus code in sstate.bbclass. It is establishing
multiple threads and maintains a connection in each thread which is being reused
for the http request.

I'm not sure how this would be best handled for ssh. As you say, there are some
interesting challenges...

Cheers,

Richard