All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Rusty Howell" <rustyhowell@gmail.com>
To: "yocto@lists.yoctoproject.org" <yocto@lists.yoctoproject.org>
Subject: Sharing sstate cache across build nodes
Date: Fri, 10 Sep 2021 11:16:48 -0600	[thread overview]
Message-ID: <CAE+BM3=TP+2KtovT+csfZKRZTnvG68aoQbX=JPS2H6=umoN0zg@mail.gmail.com> (raw)

[-- Attachment #1: Type: text/plain, Size: 1532 bytes --]

Hello,
We are having a problem with the PR server resetting the PR number it is
returning for a given package/arch/checksum.  Our setup is as follows:  We
have multiple linux servers being used as build nodes. Each node can build
any one of four MACHINE types that we have defined in our distro. These
builds actually happen inside a docker container on the build nodes.  We
have a global PR server running on a separate server.

Each node has it's own SSTATE_DIR, DL_DIR, and all bitbake builds on a node
will use the same SSTATE_DIR, DL_DIR. Those directories are shared with the
docker containers.

Our distro has many recipes that have a git SRC_URI with a branch name as
the rev. So they need to use AUTOINC.
What we are noticing is that once in a while, the revs being served out by
the PR server will be reset back to 0, and thus break upgrade-ability with
the debian packages.

I have been unable to find much information about how to properly configure
multiple build nodes so that they all have consistent PR values from the PR
server.

I know there are several directories that might be necessary to achieve my
goal.
PERSISTENT_DIR, SSTATE_DIR, BUILDHISTORY_DIR, DL_DIR

Can someone help explain which dirs should/must be shared via NFS/smb
across all build nodes? Which directories are node-specific and only need
to be cached locally (but not NFS) and used for all local build jobs? Does
changing the MACHINE type on the build affect how/if these directories can
be shared?
Thanks a lot

[-- Attachment #2: Type: text/html, Size: 1711 bytes --]

             reply	other threads:[~2021-09-10 17:17 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-10 17:16 Rusty Howell [this message]
2021-09-11 10:24 ` [yocto] Sharing sstate cache across build nodes Richard Purdie
2021-09-16  4:26   ` Rusty Howell
2021-09-16  4:34   ` Rusty Howell
2021-09-16  4:50     ` [yocto] " Khem Raj
2021-09-20  9:37     ` Ross Burton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAE+BM3=TP+2KtovT+csfZKRZTnvG68aoQbX=JPS2H6=umoN0zg@mail.gmail.com' \
    --to=rustyhowell@gmail.com \
    --cc=yocto@lists.yoctoproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.