All of lore.kernel.org
 help / color / mirror / Atom feed
From: Yann E. MORIN <yann.morin.1998@free.fr>
To: buildroot@busybox.net
Subject: [Buildroot] [PATCH 08/10 v3] support/download: change format of archives generated from svn
Date: Tue, 29 Dec 2020 12:01:16 +0100	[thread overview]
Message-ID: <ab12b3a7a8aa528805008e540cec4cb486954921.1609239666.git.yann.morin.1998@free.fr> (raw)
In-Reply-To: <cover.1609239666.git.yann.morin.1998@free.fr>

Like we recently did for git, switch the archives generated from
subversion to be reproducible whatever the tar version.

We have no in-tree users of the svn backend which also has hashes,
so no hash to update.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Heiko Thiery <heiko.thiery@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Vincent Fazio <vfazio@xes-inc.com>
---
 package/pkg-download.mk |  1 +
 support/download/svn    | 22 +++++++++-------------
 2 files changed, 10 insertions(+), 13 deletions(-)

diff --git a/package/pkg-download.mk b/package/pkg-download.mk
index 272f56a826..21b8493e3f 100644
--- a/package/pkg-download.mk
+++ b/package/pkg-download.mk
@@ -20,6 +20,7 @@ export LOCALFILES := $(call qstrip,$(BR2_LOCALFILES))
 # Version of the format of the archives we generate in the corresponding
 # download backend:
 BR_FMT_VERSION_git = _br1
+BR_FMT_VERSION_svn = _br1
 
 DL_WRAPPER = support/download/dl-wrapper
 
diff --git a/support/download/svn b/support/download/svn
index b7a6ac7443..839dccaf62 100755
--- a/support/download/svn
+++ b/support/download/svn
@@ -1,5 +1,10 @@
 #!/usr/bin/env bash
 
+# NOTE: if the output of this backend has to change (e.g. we change what gets
+# included in the archive, or we change the format of the archive (e.g. tar
+# options, compression ratio or method)), we MUST update the format version
+# in the variable BR_FTM_VERSION_svn, in package/pkg-download.mk.
+
 # We want to catch any unexpected failure, and exit immediately
 set -e
 
@@ -15,6 +20,8 @@ set -e
 # Environment:
 #   SVN      : the svn command to call
 
+. "${0%/*}/helpers"
+
 verbose=
 while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do
     case "${OPT}" in
@@ -45,18 +52,7 @@ _svn export ${verbose} "${@}" "'${uri}@${rev}'" "'${basename}'"
 # last line (svn outputs everything on stdout)
 date="$( _svn info --show-item last-changed-date "'${uri}@${rev}'" |tail -n 1 )"
 
-# Generate the archive, sort with the C locale so that it is reproducible.
+# Generate the archive.
 # We did a 'svn export' above, so it's not a working copy (there is no .svn
 # directory or file to ignore).
-find "${basename}" -not -type d >"${output}.list"
-LC_ALL=C sort <"${output}.list" >"${output}.list.sorted"
-
-# Create GNU-format tarballs, since that's the format of the tarballs on
-# sources.buildroot.org and used in the *.hash files
-tar cf - --transform="s#^\./#${basename}/#" \
-         --numeric-owner --owner=0 --group=0 --mtime="${date}" --format=gnu \
-         -T "${output}.list.sorted" >"${output}.tar"
-gzip -6 -n <"${output}.tar" >"${output}"
-
-rm -f "${output}.list"
-rm -f "${output}.list.sorted"
+mk_tar_gz "${basename}" "${basename}" "${date}" "${output}"
-- 
2.25.1

  parent reply	other threads:[~2020-12-29 11:01 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-29 11:01 [Buildroot] [PATCH 00/10 v3] support/download: reproducible archives whatever tar version (branch yem/dl-git-tar-pax-2) Yann E. MORIN
2020-12-29 11:01 ` [Buildroot] [PATCH 01/10 v3] core/pkg-infra: prepare for alternate default source archives Yann E. MORIN
2021-01-05 21:54   ` Arnout Vandecappelle
2021-01-07 19:52   ` Vincent Fazio
2020-12-29 11:01 ` [Buildroot] [PATCH 02/10 v3] core/pkg-infra: allow per site-method sub-version strings Yann E. MORIN
2021-01-05 21:58   ` Arnout Vandecappelle
2021-01-07 19:52   ` Vincent Fazio
2020-12-29 11:01 ` [Buildroot] [PATCH 03/10 v3] support/download: add helper to generate a reproducible archive Yann E. MORIN
2020-12-29 14:26   ` Vincent Fazio
2020-12-29 14:37     ` Yann E. MORIN
2021-01-05 22:05   ` Arnout Vandecappelle
2021-01-07 19:50   ` Vincent Fazio
2020-12-29 11:01 ` [Buildroot] [PATCH 04/10 v3] WIP: support/download: change format of archives generated from git Yann E. MORIN
2021-01-05 22:13   ` Arnout Vandecappelle
2021-01-09 13:45     ` Yann E. MORIN
2021-01-07 19:50   ` Vincent Fazio
2020-12-29 11:01 ` [Buildroot] [PATCH 05/10 v3] WIP: boot+packages: update hash to new git-tarballs format Yann E. MORIN
2021-01-05 22:30   ` Arnout Vandecappelle
2021-01-09 13:46     ` Yann E. MORIN
2021-01-07 19:47   ` Vincent Fazio
2020-12-29 11:01 ` [Buildroot] [PATCH 06/10 v3] WIP: support/testing: update git-hash checks with new archive format Yann E. MORIN
2021-01-05 22:32   ` Arnout Vandecappelle
2021-01-09 11:16     ` Yann E. MORIN
2021-01-07 19:46   ` Vincent Fazio
2020-12-29 11:01 ` [Buildroot] [PATCH 07/10 v3] support/download: cleanup svn backend Yann E. MORIN
2021-01-05 22:33   ` Arnout Vandecappelle
2021-01-07 19:42   ` Vincent Fazio
2020-12-29 11:01 ` Yann E. MORIN [this message]
2021-01-05 22:38   ` [Buildroot] [PATCH 08/10 v3] support/download: change format of archives generated from svn Arnout Vandecappelle
2021-01-07 19:44   ` Vincent Fazio
2020-12-29 11:01 ` [Buildroot] [PATCH 09/10 v3] support/dependencies: drop check for maximal tar version Yann E. MORIN
2021-01-05 22:41   ` Arnout Vandecappelle
2021-01-07 19:40   ` Vincent Fazio
2020-12-29 11:01 ` [Buildroot] [PATCH 10/10 v3] package/tar: drop specific version for host variant Yann E. MORIN
2021-01-05 22:46   ` Arnout Vandecappelle
2021-01-07 19:40   ` Vincent Fazio
2020-12-29 13:09 ` [Buildroot] [PATCH 00/10 v3] support/download: reproducible archives whatever tar version (branch yem/dl-git-tar-pax-2) Thomas Petazzoni
2020-12-29 13:46   ` Yann E. MORIN
2020-12-29 14:15     ` Thomas Petazzoni
2020-12-29 14:25       ` Yann E. MORIN

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ab12b3a7a8aa528805008e540cec4cb486954921.1609239666.git.yann.morin.1998@free.fr \
    --to=yann.morin.1998@free.fr \
    --cc=buildroot@busybox.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.