All of lore.kernel.org
 help / color / mirror / Atom feed
From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Cc: Han-Wen Nienhuys <hanwenn@gmail.com>,
	Josh Steadmon <steadmon@google.com>,
	Luca Milanesio <luca.milanesio@gmail.com>,
	Eric Sunshine <sunshine@sunshineco.com>,
	Toon Claes <toon@iotcl.com>, Justin Tobler <jltobler@gmail.com>
Subject: [PATCH v3 08/13] ci: merge scripts which install dependencies
Date: Thu, 11 Apr 2024 11:10:36 +0200	[thread overview]
Message-ID: <7748f87f8cd23826adbbfbaa7616d8b5be346c42.1712825204.git.ps@pks.im> (raw)
In-Reply-To: <cover.1712825204.git.ps@pks.im>

[-- Attachment #1: Type: text/plain, Size: 6337 bytes --]

We have two different scripts which install dependencies, one for
dockerized jobs and one for non-dockerized ones. Naturally, these
scripts have quite some duplication. Furthermore, either of these
scripts is missing some test dependencies that the respective other
script has, thus reducing test coverage.

Merge those two scripts such that there is a single source of truth for
test dependencies, only.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 .github/workflows/main.yml        |  2 +-
 .gitlab-ci.yml                    |  4 +--
 ci/install-dependencies.sh        | 32 ++++++++++++++++++---
 ci/install-docker-dependencies.sh | 46 -------------------------------
 4 files changed, 31 insertions(+), 53 deletions(-)
 delete mode 100755 ci/install-docker-dependencies.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 71cd4e5486..5838986895 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -359,7 +359,7 @@ jobs:
       if: matrix.vector.jobname != 'linux32'
     - uses: actions/checkout@v1 # cannot be upgraded because Node.js Actions aren't supported in this container
       if: matrix.vector.jobname == 'linux32'
-    - run: ci/install-docker-dependencies.sh
+    - run: ci/install-dependencies.sh
     - run: ci/run-build-and-tests.sh
     - name: print test failures
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 3a0ef4d4d4..ba65f50aac 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -12,7 +12,7 @@ test:linux:
   variables:
     CUSTOM_PATH: "/custom"
   before_script:
-    - ./ci/install-docker-dependencies.sh
+    - ./ci/install-dependencies.sh
   script:
     - useradd builder --create-home
     - chown -R builder "${CI_PROJECT_DIR}"
@@ -100,7 +100,7 @@ static-analysis:
   variables:
     jobname: StaticAnalysis
   before_script:
-    - ./ci/install-docker-dependencies.sh
+    - ./ci/install-dependencies.sh
   script:
     - ./ci/run-static-analysis.sh
     - ./ci/check-directional-formatting.bash
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index bafe37f2d1..e673797115 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -5,6 +5,8 @@
 
 . ${0%/*}/lib.sh
 
+begin_group "Install dependencies"
+
 P4WHENCE=https://cdist2.perforce.com/perforce/r21.2
 LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
 
@@ -20,14 +22,27 @@ then
 fi
 
 case "$distro" in
+alpine-*)
+	apk add --update shadow sudo build-base curl-dev openssl-dev expat-dev gettext \
+		pcre2-dev python3 musl-libintl perl-utils ncurses \
+		apache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \
+		bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null
+	;;
+fedora-*)
+	dnf -yq update >/dev/null &&
+	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
+	;;
 ubuntu-*)
+	# Required so that apt doesn't wait for user input on certain packages.
+	export DEBIAN_FRONTEND=noninteractive
+
 	sudo apt-get -q update
 	sudo apt-get -q -y install \
-		language-pack-is libsvn-perl apache2 \
-		make libssl-dev libcurl4-openssl-dev libexpat-dev \
+		language-pack-is libsvn-perl apache2 cvs cvsps git gnupg subversion \
+		make libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo \
 		tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \
-		libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl \
-		$CC_PACKAGE $PYTHON_PACKAGE
+		libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \
+		${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE
 
 	mkdir --parents "$CUSTOM_PATH"
 	wget --quiet --directory-prefix="$CUSTOM_PATH" \
@@ -39,6 +54,13 @@ ubuntu-*)
 		-C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs"
 	rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
 	;;
+ubuntu32-*)
+	sudo linux32 --32bit i386 sh -c '
+		apt update >/dev/null &&
+		apt install -y build-essential libcurl4-openssl-dev \
+			libssl-dev libexpat-dev gettext python >/dev/null
+	'
+	;;
 macos-*)
 	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
 	# Uncomment this if you want to run perf tests:
@@ -98,3 +120,5 @@ then
 else
 	echo >&2 "WARNING: git-lfs wasn't installed, see above for clues why"
 fi
+
+end_group "Install dependencies"
diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh
deleted file mode 100755
index eb2c9e1eca..0000000000
--- a/ci/install-docker-dependencies.sh
+++ /dev/null
@@ -1,46 +0,0 @@
-#!/bin/sh
-#
-# Install dependencies required to build and test Git inside container
-#
-
-. ${0%/*}/lib.sh
-
-begin_group "Install dependencies"
-
-case "$jobname" in
-linux32)
-	linux32 --32bit i386 sh -c '
-		apt update >/dev/null &&
-		apt install -y build-essential libcurl4-openssl-dev \
-			libssl-dev libexpat-dev gettext python >/dev/null
-	'
-	;;
-linux-musl)
-	apk add --update shadow sudo build-base curl-dev openssl-dev expat-dev gettext \
-		pcre2-dev python3 musl-libintl perl-utils ncurses \
-		apache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \
-		bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null
-	;;
-linux-*|StaticAnalysis)
-	# Required so that apt doesn't wait for user input on certain packages.
-	export DEBIAN_FRONTEND=noninteractive
-
-	apt update -q &&
-	apt install -q -y sudo git make language-pack-is libsvn-perl apache2 libssl-dev \
-		libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev \
-		perl-modules liberror-perl libauthen-sasl-perl libemail-valid-perl \
-		libdbd-sqlite3-perl libio-socket-ssl-perl libnet-smtp-ssl-perl ${CC_PACKAGE:-${CC:-gcc}} \
-		apache2 cvs cvsps gnupg libcgi-pm-perl subversion
-
-	if test "$jobname" = StaticAnalysis
-	then
-		apt install -q -y coccinelle
-	fi
-	;;
-pedantic)
-	dnf -yq update >/dev/null &&
-	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
-	;;
-esac
-
-end_group "Install dependencies"
-- 
2.44.GIT


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

  parent reply	other threads:[~2024-04-11  9:10 UTC|newest]

Thread overview: 106+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-04-04 13:25 [PATCH 00/12] t: exercise Git/JGit reftable compatibility Patrick Steinhardt
2024-04-04 13:25 ` [PATCH 01/12] ci: rename "runs_on_pool" to "distro" Patrick Steinhardt
2024-04-04 13:25 ` [PATCH 02/12] ci: expose distro name in dockerized GitHub jobs Patrick Steinhardt
2024-04-04 13:25 ` [PATCH 03/12] ci: allow skipping sudo on dockerized jobs Patrick Steinhardt
2024-04-04 13:25 ` [PATCH 04/12] ci: drop duplicate package installation for "linux-gcc-default" Patrick Steinhardt
2024-04-04 13:25 ` [PATCH 05/12] ci: convert "install-dependencies.sh" to use "/bin/sh" Patrick Steinhardt
2024-04-04 13:25 ` [PATCH 06/12] ci: merge custom PATH directories Patrick Steinhardt
2024-04-04 13:25 ` [PATCH 07/12] ci: merge scripts which install dependencies Patrick Steinhardt
2024-04-04 13:25 ` [PATCH 08/12] ci: make Perforce binaries executable for all users Patrick Steinhardt
2024-04-05 20:01   ` Josh Steadmon
2024-04-08  5:48     ` Patrick Steinhardt
2024-04-04 13:25 ` [PATCH 09/12] ci: install JGit dependency Patrick Steinhardt
2024-04-04 13:25 ` [PATCH 10/12] t06xx: always execute backend-specific tests Patrick Steinhardt
2024-04-04 13:25 ` [PATCH 11/12] t0610: fix non-portable variable assignment Patrick Steinhardt
2024-04-05  9:14   ` Eric Sunshine
2024-04-05  9:40     ` Patrick Steinhardt
2024-04-05 16:02   ` Junio C Hamano
2024-04-05 16:12     ` [PATCH] CodingGuidelines: quote assigned value with "local" and "export" Junio C Hamano
2024-04-05 16:21       ` Junio C Hamano
2024-04-05 17:48       ` Jeff King
2024-04-05 19:36         ` Junio C Hamano
2024-04-05 23:15           ` Junio C Hamano
2024-04-07  1:23             ` Jeff King
2024-04-05 16:44     ` [PATCH 11/12] t0610: fix non-portable variable assignment Patrick Steinhardt
2024-04-04 13:25 ` [PATCH 12/12] t0612: add tests to exercise Git/JGit reftable compatibility Patrick Steinhardt
2024-04-04 21:40   ` Han-Wen Nienhuys
2024-04-05  9:41     ` Patrick Steinhardt
2024-04-06 13:03       ` Han-Wen Nienhuys
2024-04-06 15:53         ` Patrick Steinhardt
2024-04-08  5:24     ` Patrick Steinhardt
2024-04-04 13:37 ` [PATCH 00/12] t: " Patrick Steinhardt
2024-04-08  6:45 ` [PATCH v2 " Patrick Steinhardt
2024-04-08  6:46   ` [PATCH v2 02/12] ci: expose distro name in dockerized GitHub jobs Patrick Steinhardt
2024-04-08  6:46   ` [PATCH v2 03/12] ci: allow skipping sudo on dockerized jobs Patrick Steinhardt
2024-04-10 16:53     ` Toon claes
2024-04-10 17:21       ` Junio C Hamano
2024-04-11  8:55         ` Patrick Steinhardt
2024-04-08  6:46   ` [PATCH v2 04/12] ci: drop duplicate package installation for "linux-gcc-default" Patrick Steinhardt
2024-04-08  6:46   ` [PATCH v2 05/12] ci: convert "install-dependencies.sh" to use "/bin/sh" Patrick Steinhardt
2024-04-10 20:46     ` Justin Tobler
2024-04-11  8:55       ` Patrick Steinhardt
2024-04-08  6:46   ` [PATCH v2 06/12] ci: merge custom PATH directories Patrick Steinhardt
2024-04-08  6:46   ` [PATCH v2 07/12] ci: merge scripts which install dependencies Patrick Steinhardt
2024-04-08  6:46   ` [PATCH v2 08/12] ci: make Perforce binaries executable for all users Patrick Steinhardt
2024-04-08  6:46   ` [PATCH v2 09/12] ci: install JGit dependency Patrick Steinhardt
2024-04-08  6:46   ` [PATCH v2 10/12] t06xx: always execute backend-specific tests Patrick Steinhardt
2024-04-08  6:47   ` [PATCH v2 11/12] t0610: fix non-portable variable assignment Patrick Steinhardt
2024-04-08  6:57     ` Eric Sunshine
2024-04-08  8:56       ` Patrick Steinhardt
2024-04-08  6:47   ` [PATCH v2 12/12] t0612: add tests to exercise Git/JGit reftable compatibility Patrick Steinhardt
2024-04-08  7:10     ` Eric Sunshine
2024-04-08 16:07       ` Junio C Hamano
2024-04-08 16:19         ` Eric Sunshine
2024-04-08 16:29           ` Eric Sunshine
2024-04-08 16:22         ` Patrick Steinhardt
2024-04-08 17:26           ` Jeff King
2024-04-08 21:52             ` Junio C Hamano
2024-04-08 21:51           ` Junio C Hamano
2024-04-10 20:43     ` Justin Tobler
2024-04-11  8:55       ` Patrick Steinhardt
2024-04-08  6:47   ` [PATCH v2 01/12] ci: rename "runs_on_pool" to "distro" Patrick Steinhardt
2024-04-09  2:17   ` [PATCH v2 00/12] t: exercise Git/JGit reftable compatibility Junio C Hamano
2024-04-09  3:43     ` Patrick Steinhardt
2024-04-09  5:34       ` Junio C Hamano
2024-04-09  6:07         ` Patrick Steinhardt
2024-04-09  9:57     ` [PATCH 0/2] t0610: fix umask tests Patrick Steinhardt
2024-04-09  9:57       ` [PATCH 1/2] t0610: make `--shared=` tests reusable Patrick Steinhardt
2024-04-09  9:57       ` [PATCH 2/2] t0610: execute git-pack-refs(1) with specified umask Patrick Steinhardt
2024-04-10 21:49       ` [PATCH 0/2] t0610: fix umask tests Justin Tobler
2024-04-11  9:09 ` [PATCH v3 00/13] t: exercise Git/JGit reftable compatibility Patrick Steinhardt
2024-04-11  9:09   ` [PATCH v3 01/13] ci: rename "runs_on_pool" to "distro" Patrick Steinhardt
2024-04-11  9:10   ` [PATCH v3 02/13] ci: expose distro name in dockerized GitHub jobs Patrick Steinhardt
2024-04-11  9:10   ` [PATCH v3 03/13] ci: skip sudo when we are already root Patrick Steinhardt
2024-04-11  9:10   ` [PATCH v3 04/13] ci: drop duplicate package installation for "linux-gcc-default" Patrick Steinhardt
2024-04-11  9:10   ` [PATCH v3 05/13] ci: convert "install-dependencies.sh" to use "/bin/sh" Patrick Steinhardt
2024-04-11  9:10   ` [PATCH v3 06/13] ci: merge custom PATH directories Patrick Steinhardt
2024-04-11  9:10   ` [PATCH v3 07/13] ci: fix setup of custom path for GitLab CI Patrick Steinhardt
2024-04-11 10:06     ` Eric Sunshine
2024-04-11 10:26       ` Patrick Steinhardt
2024-04-11  9:10   ` Patrick Steinhardt [this message]
2024-04-11  9:10   ` [PATCH v3 09/13] ci: make Perforce binaries executable for all users Patrick Steinhardt
2024-04-11  9:10   ` [PATCH v3 10/13] ci: install JGit dependency Patrick Steinhardt
2024-04-11  9:11   ` [PATCH v3 11/13] t06xx: always execute backend-specific tests Patrick Steinhardt
2024-04-11  9:11   ` [PATCH v3 12/13] t0610: fix non-portable variable assignment Patrick Steinhardt
2024-04-11  9:11   ` [PATCH v3 13/13] t0612: add tests to exercise Git/JGit reftable compatibility Patrick Steinhardt
2024-04-12  4:43 ` [PATCH v4 00/13] t: " Patrick Steinhardt
2024-04-12  4:43   ` [PATCH v4 01/13] ci: rename "runs_on_pool" to "distro" Patrick Steinhardt
2024-04-12  4:44   ` [PATCH v4 02/13] ci: expose distro name in dockerized GitHub jobs Patrick Steinhardt
2024-04-12  4:44   ` [PATCH v4 03/13] ci: skip sudo when we are already root Patrick Steinhardt
2024-04-12  4:44   ` [PATCH v4 04/13] ci: drop duplicate package installation for "linux-gcc-default" Patrick Steinhardt
2024-04-12  4:44   ` [PATCH v4 05/13] ci: convert "install-dependencies.sh" to use "/bin/sh" Patrick Steinhardt
2024-04-12  4:44   ` [PATCH v4 06/13] ci: merge custom PATH directories Patrick Steinhardt
2024-04-12  4:44   ` [PATCH v4 07/13] ci: fix setup of custom path for GitLab CI Patrick Steinhardt
2024-04-12  4:44   ` [PATCH v4 08/13] ci: merge scripts which install dependencies Patrick Steinhardt
2024-04-12  4:44   ` [PATCH v4 09/13] ci: make Perforce binaries executable for all users Patrick Steinhardt
2024-04-12  4:44   ` [PATCH v4 10/13] ci: install JGit dependency Patrick Steinhardt
2024-04-12  4:44   ` [PATCH v4 11/13] t06xx: always execute backend-specific tests Patrick Steinhardt
2024-04-12  4:44   ` [PATCH v4 12/13] t0610: fix non-portable variable assignment Patrick Steinhardt
2024-04-12  4:44   ` [PATCH v4 13/13] t0612: add tests to exercise Git/JGit reftable compatibility Patrick Steinhardt
2024-05-03 18:42   ` [PATCH v4 00/13] t: " Justin Tobler
2024-05-03 18:48     ` Patrick Steinhardt
2024-05-03 18:57       ` Patrick Steinhardt
2024-05-03 19:35         ` Justin Tobler
2024-05-03 19:49           ` Patrick Steinhardt
2024-05-04 17:32             ` Justin Tobler
2024-05-06  5:53               ` Patrick Steinhardt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=7748f87f8cd23826adbbfbaa7616d8b5be346c42.1712825204.git.ps@pks.im \
    --to=ps@pks.im \
    --cc=git@vger.kernel.org \
    --cc=hanwenn@gmail.com \
    --cc=jltobler@gmail.com \
    --cc=luca.milanesio@gmail.com \
    --cc=steadmon@google.com \
    --cc=sunshine@sunshineco.com \
    --cc=toon@iotcl.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.