* [Qemu-devel] [PATCH v4] scripts: use git archive in archive-source
@ 2019-02-05 11:23 Gerd Hoffmann
2019-02-05 11:26 ` no-reply
2019-02-05 11:50 ` no-reply
0 siblings, 2 replies; 3+ messages in thread
From: Gerd Hoffmann @ 2019-02-05 11:23 UTC (permalink / raw)
To: qemu-devel; +Cc: famz, Gerd Hoffmann
Use git archive to create tarballs of qemu and submodules instead of
cloning the repository and the submodules. This is a order of magnitude
faster because it doesn't fetch the submodules from the internet each
time the script runs.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
scripts/archive-source.sh | 67 ++++++++++++++++++++++-------------------------
1 file changed, 31 insertions(+), 36 deletions(-)
diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh
index 6eed2a29bd..a4ea98ded4 100755
--- a/scripts/archive-source.sh
+++ b/scripts/archive-source.sh
@@ -19,16 +19,25 @@ if test $# -lt 1; then
fi
tar_file=$(realpath "$1")
-list_file="${tar_file}.list"
-vroot_dir="${tar_file}.vroot"
+sub_tdir=$(mktemp -d "${tar_file%.tar}.sub.XXXXXXXX")
+sub_file="${sub_tdir}/submodule.tar"
# We want a predictable list of submodules for builds, that is
# independent of what the developer currently has initialized
# in their checkout, because the build environment is completely
# different to the host OS.
submodules="dtc ui/keycodemapdb tests/fp/berkeley-softfloat-3 tests/fp/berkeley-testfloat-3"
+sub_deinit=""
-trap "status=$?; rm -rf \"$list_file\" \"$vroot_dir\"; exit \$status" 0 1 2 3 15
+function cleanup() {
+ local status=$?
+ rm -rf "$sub_tdir"
+ if test "$sub_deinit" != ""; then
+ git submodule deinit $sub_deinit
+ fi
+ exit $status
+}
+trap "cleanup" 0 1 2 3 15
if git diff-index --quiet HEAD -- &>/dev/null
then
@@ -36,38 +45,24 @@ then
else
HEAD=$(git stash create)
fi
-git clone --shared . "$vroot_dir"
-test $? -ne 0 && error "failed to clone into '$vroot_dir'"
-
-cd "$vroot_dir"
-test $? -ne 0 && error "failed to change into '$vroot_dir'"
-
-git checkout $HEAD
-test $? -ne 0 && error "failed to checkout $HEAD revision"
-
+git archive --format tar $HEAD > "$tar_file"
+test $? -ne 0 && error "failed to archive qemu"
for sm in $submodules; do
- git submodule update --init $sm
- test $? -ne 0 && error "failed to init submodule $sm"
+ status="$(git submodule status "$sm")"
+ smhash="${status#[ +-]}"
+ smhash="${smhash%% *}"
+ case "$status" in
+ -*)
+ sub_deinit="$sub_deinit $sm"
+ git submodule update --init "$sm"
+ test $? -ne 0 && error "failed to update submodule $sm"
+ ;;
+ +*)
+ echo "WARNING: submodule $sm is out of sync"
+ ;;
+ esac
+ (cd $sm; git archive --format tar --prefix "$sm/" $smhash) > "$sub_file"
+ test $? -ne 0 && error "failed to archive submodule $sm ($smhash)"
+ tar --concatenate --file "$tar_file" "$sub_file"
+ test $? -ne 0 && error "failed append submodule $sm to $tar_file"
done
-
-if test -n "$submodules"; then
- {
- git ls-files || error "git ls-files failed"
- for sm in $submodules; do
- (cd $sm; git ls-files) | sed "s:^:$sm/:"
- if test "${PIPESTATUS[*]}" != "0 0"; then
- error "git ls-files in submodule $sm failed"
- fi
- done
- } | grep -x -v $(for sm in $submodules; do echo "-e $sm"; done) > "$list_file"
-else
- git ls-files > "$list_file"
-fi
-
-if test $? -ne 0; then
- error "failed to generate list file"
-fi
-
-tar -cf "$tar_file" -T "$list_file" || error "failed to create tar file"
-
-exit 0
--
2.9.3
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [Qemu-devel] [PATCH v4] scripts: use git archive in archive-source
2019-02-05 11:23 [Qemu-devel] [PATCH v4] scripts: use git archive in archive-source Gerd Hoffmann
@ 2019-02-05 11:26 ` no-reply
2019-02-05 11:50 ` no-reply
1 sibling, 0 replies; 3+ messages in thread
From: no-reply @ 2019-02-05 11:26 UTC (permalink / raw)
To: kraxel; +Cc: fam, qemu-devel, famz
Patchew URL: https://patchew.org/QEMU/20190205112315.23454-1-kraxel@redhat.com/
Hi,
This series seems to have some coding style problems. See output below for
more information:
Subject: [Qemu-devel] [PATCH v4] scripts: use git archive in archive-source
Type: series
Message-id: 20190205112315.23454-1-kraxel@redhat.com
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
* [new tag] patchew/20190205112315.23454-1-kraxel@redhat.com -> patchew/20190205112315.23454-1-kraxel@redhat.com
Switched to a new branch 'test'
e92ad529b4 scripts: use git archive in archive-source
=== OUTPUT BEGIN ===
ERROR: code indent should never use tabs
#40: FILE: scripts/archive-source.sh:36:
+^Igit submodule deinit $sub_deinit$
ERROR: code indent should never use tabs
#70: FILE: scripts/archive-source.sh:55:
+^I-*)$
ERROR: code indent should never use tabs
#71: FILE: scripts/archive-source.sh:56:
+^I sub_deinit="$sub_deinit $sm"$
ERROR: code indent should never use tabs
#72: FILE: scripts/archive-source.sh:57:
+^I git submodule update --init "$sm"$
ERROR: code indent should never use tabs
#73: FILE: scripts/archive-source.sh:58:
+^I test $? -ne 0 && error "failed to update submodule $sm"$
ERROR: code indent should never use tabs
#74: FILE: scripts/archive-source.sh:59:
+^I ;;$
ERROR: code indent should never use tabs
#75: FILE: scripts/archive-source.sh:60:
+^I+*)$
ERROR: code indent should never use tabs
#76: FILE: scripts/archive-source.sh:61:
+^I echo "WARNING: submodule $sm is out of sync"$
ERROR: code indent should never use tabs
#77: FILE: scripts/archive-source.sh:62:
+^I ;;$
total: 9 errors, 0 warnings, 63 lines checked
Commit e92ad529b444 (scripts: use git archive in archive-source) has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/20190205112315.23454-1-kraxel@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [Qemu-devel] [PATCH v4] scripts: use git archive in archive-source
2019-02-05 11:23 [Qemu-devel] [PATCH v4] scripts: use git archive in archive-source Gerd Hoffmann
2019-02-05 11:26 ` no-reply
@ 2019-02-05 11:50 ` no-reply
1 sibling, 0 replies; 3+ messages in thread
From: no-reply @ 2019-02-05 11:50 UTC (permalink / raw)
To: kraxel; +Cc: fam, qemu-devel, famz
Patchew URL: https://patchew.org/QEMU/20190205112315.23454-1-kraxel@redhat.com/
Hi,
This series failed the docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-mingw@fedora SHOW_ENV=1 J=14
=== TEST SCRIPT END ===
The full log is available at
http://patchew.org/logs/20190205112315.23454-1-kraxel@redhat.com/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-02-05 11:51 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-02-05 11:23 [Qemu-devel] [PATCH v4] scripts: use git archive in archive-source Gerd Hoffmann
2019-02-05 11:26 ` no-reply
2019-02-05 11:50 ` no-reply
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.