[PATCH] ref-manual: Document BB_GIT_SHALLOW and friends

[PATCH] ref-manual: Document BB_GIT_SHALLOW and friends

[Paul Barker]

Documentation is based on the commit message of bitbake rev 5ed7d85fda
and mailing list discussion.

[YOCTO #14493]

Signed-off-by: Paul Barker <paul@pbarker.dev>
---
 documentation/ref-manual/variables.rst | 55 ++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index f6d248a19..edab054f1 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -524,6 +524,61 @@ system and gives an overview of their function and contents.
       clean up your :term:`DL_DIR` directory by deleting any Git or other
       source control work directories.
 
+   :term:`BB_GENERATE_SHALLOW_TARBALLS`
+      Setting this variable to "1" when :term:`BB_GIT_SHALLOW` is also set to
+      "1" causes bitbake to generate shallow mirror tarballs when fetching git
+      repositories. The number of commits included in the shallow mirror
+      tarballs is controlled by :term:`BB_GIT_SHALLOW_DEPTH`.
+
+      If both :term:`BB_GIT_SHALLOW` and :term:`BB_GENERATE_MIRROR_TARBALLS` are
+      enabled, bitbake will generate shallow mirror tarballs by default for git
+      repositories. This separate variable exists so that shallow tarball
+      generation can be enabled without needing to also enable normal mirror
+      generation if it is not desired.
+
+      For example usage, see :term:`BB_GIT_SHALLOW`.
+
+   :term:`BB_GIT_SHALLOW`
+      Setting this variable to "1" enables the support for fetching, using and
+      generating mirror tarballs of shallow git repositories. The external
+      git-make-shallow script is used for shallow mirror tarball creation.
+
+      When BB_GIT_SHALLOW is enabled, bitbake will attempt to fetch a shallow
+      mirror tarball. If the shallow mirror tarball cannot be fetched, it will
+      try to fetch the full mirror tarball and use that.
+
+      When a mirror tarball is not available, a full git clone will be performed
+      regardless of whether this variable is set or not. Support for shallow
+      clones is not currently implemented as git does not directly support
+      shallow cloning a particular git commit hash (it only supports cloning
+      from a tag or branch reference).
+
+      See also :term:`BB_GIT_SHALLOW_DEPTH` and
+      :term:`BB_GENERATE_SHALLOW_TARBALLS`.
+
+      Example usage::
+
+         BB_GIT_SHALLOW ?= "1"
+
+         # Keep only the top commit
+         BB_GIT_SHALLOW_DEPTH ?= "1"
+
+         # This defaults to enabled if both BB_GIT_SHALLOW and
+         # BB_GENERATE_MIRROR_TARBALLS are enabled
+         BB_GENERATE_SHALLOW_TARBALLS ?= "1"
+
+   :term:`BB_GIT_SHALLOW_DEPTH`
+      When used with :term:`BB_GENERATE_SHALLOW_TARBALLS`, this variable sets
+      the number of commits to include in generated shallow mirror tarballs.
+      With a depth of 1, only the commit referenced in :term:`SRCREV` is
+      included in the shallow mirror tarball. Increasing the depth includes
+      additional parent commits, working back through the commit history.
+
+      If this variable is unset, bitbake will default to a depth of 1 when
+      generating shallow mirror tarballs.
+
+      For example usage, see :term:`BB_GIT_SHALLOW`.
+
    :term:`BB_NUMBER_THREADS`
       The maximum number of tasks BitBake should run in parallel at any one
       time. The OpenEmbedded build system automatically configures this
-- 
2.31.1

Re: [docs] [PATCH] ref-manual: Document BB_GIT_SHALLOW and friends

[Quentin Schulz]

Hi Paul,

On Fri, Aug 13, 2021 at 04:11:48PM +0100, Paul Barker wrote:
> Documentation is based on the commit message of bitbake rev 5ed7d85fda
> and mailing list discussion.
> 
> [YOCTO #14493]
> 
> Signed-off-by: Paul Barker <paul@pbarker.dev>
> ---
>  documentation/ref-manual/variables.rst | 55 ++++++++++++++++++++++++++
>  1 file changed, 55 insertions(+)
> 

Since those variables are used in Bitbake source code, I'd say this
addition would make more sense in Bitbake documentation.

> diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
> index f6d248a19..edab054f1 100644
> --- a/documentation/ref-manual/variables.rst
> +++ b/documentation/ref-manual/variables.rst
> @@ -524,6 +524,61 @@ system and gives an overview of their function and contents.
>        clean up your :term:`DL_DIR` directory by deleting any Git or other
>        source control work directories.
>  
> +   :term:`BB_GENERATE_SHALLOW_TARBALLS`
> +      Setting this variable to "1" when :term:`BB_GIT_SHALLOW` is also set to
> +      "1" causes bitbake to generate shallow mirror tarballs when fetching git
> +      repositories. The number of commits included in the shallow mirror
> +      tarballs is controlled by :term:`BB_GIT_SHALLOW_DEPTH`.
> +
> +      If both :term:`BB_GIT_SHALLOW` and :term:`BB_GENERATE_MIRROR_TARBALLS` are
> +      enabled, bitbake will generate shallow mirror tarballs by default for git
> +      repositories. This separate variable exists so that shallow tarball
> +      generation can be enabled without needing to also enable normal mirror
> +      generation if it is not desired.
> +
> +      For example usage, see :term:`BB_GIT_SHALLOW`.
> +
> +   :term:`BB_GIT_SHALLOW`
> +      Setting this variable to "1" enables the support for fetching, using and
> +      generating mirror tarballs of shallow git repositories. The external
> +      git-make-shallow script is used for shallow mirror tarball creation.
> +
> +      When BB_GIT_SHALLOW is enabled, bitbake will attempt to fetch a shallow

s/BB_GIT_SHALLOW/:term:`BB_GIT_SHALLOW`/

> +      mirror tarball. If the shallow mirror tarball cannot be fetched, it will
> +      try to fetch the full mirror tarball and use that.
> +
> +      When a mirror tarball is not available, a full git clone will be performed
> +      regardless of whether this variable is set or not. Support for shallow
> +      clones is not currently implemented as git does not directly support
> +      shallow cloning a particular git commit hash (it only supports cloning
> +      from a tag or branch reference).
> +
> +      See also :term:`BB_GIT_SHALLOW_DEPTH` and
> +      :term:`BB_GENERATE_SHALLOW_TARBALLS`.
> +
> +      Example usage::
> +
> +         BB_GIT_SHALLOW ?= "1"
> +
> +         # Keep only the top commit
> +         BB_GIT_SHALLOW_DEPTH ?= "1"
> +
> +         # This defaults to enabled if both BB_GIT_SHALLOW and
> +         # BB_GENERATE_MIRROR_TARBALLS are enabled
> +         BB_GENERATE_SHALLOW_TARBALLS ?= "1"
> +
> +   :term:`BB_GIT_SHALLOW_DEPTH`
> +      When used with :term:`BB_GENERATE_SHALLOW_TARBALLS`, this variable sets
> +      the number of commits to include in generated shallow mirror tarballs.
> +      With a depth of 1, only the commit referenced in :term:`SRCREV` is
> +      included in the shallow mirror tarball. Increasing the depth includes
> +      additional parent commits, working back through the commit history.
> +
> +      If this variable is unset, bitbake will default to a depth of 1 when
> +      generating shallow mirror tarballs.
> +
> +      For example usage, see :term:`BB_GIT_SHALLOW`.
> +

I'd have liked a link to a small article/tutorial/man page explaining
what shallow cloning is and what are its advantages and downsides since I have
never used this before. But in any case, I guess people looking for this
variable will know what it's for or could Google this.

Except for the location of this doc (for me should be in bitbake docs):
Reviewed-by: Quentin Schulz <foss@0leil.net>

Thanks!
Quentin