* [Buildroot] [PATCH 1/3] support/download/git: log checked out sha1
@ 2016-11-01 19:33 Ricardo Martincoski
2016-11-01 19:33 ` [Buildroot] [PATCH 2/3] test/support/download/git: new test Ricardo Martincoski
` (2 more replies)
0 siblings, 3 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2016-11-01 19:33 UTC (permalink / raw)
To: buildroot
In preparation to have automated tests for this script, display the
checked out sha1 in the build log.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
---
Another solution would be to use an environment variable set by the test
to print that info, e.g. if [ "1" == "${GIT_DOWNLOAD_AUTOTEST}" ]; then
---
support/download/git | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/support/download/git b/support/download/git
index 7921411..7c44c65 100755
--- a/support/download/git
+++ b/support/download/git
@@ -85,6 +85,12 @@ if [ ${recurse} -eq 1 ]; then
_git submodule update --init --recursive
fi
+# Log the sha1. It can be used for automated tests and debug.
+if [ -z ${verbose} ]; then
+ sha1=$(_git rev-parse HEAD)
+ printf "Checked out '%s'.\n" "${sha1}"
+fi
+
# We do not need the .git dir; we keep other .git files, in case they
# are the only files in their directory.
rm -rf .git
--
2.9.3
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 2/3] test/support/download/git: new test
2016-11-01 19:33 [Buildroot] [PATCH 1/3] support/download/git: log checked out sha1 Ricardo Martincoski
@ 2016-11-01 19:33 ` Ricardo Martincoski
2016-11-01 21:27 ` Ricardo Martincoski
` (3 more replies)
2016-11-01 19:33 ` [Buildroot] [PATCH 3/3] support/download/git: do not use git clone Ricardo Martincoski
2016-11-05 21:50 ` [Buildroot] [PATCH 1/3] support/download/git: log checked out sha1 Arnout Vandecappelle
2 siblings, 4 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2016-11-01 19:33 UTC (permalink / raw)
To: buildroot
This test assumes:
- it is run from the buildroot base dir;
- there is a ssh server installed;
- there is a key pair to allow the current user to ssh to localhost
without being prompted for a password;
- there is no ./.config ./stdout ./package/package;
For each testcase, it creates a new git repo, and then download from
this repo using a fake package named 'package'.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
---
This test needs some instrumentation in the source code (previous patch)
to work properly.
---
support/download/git | 1 +
test/support/download/git | 202 ++++++++++++++++++++++++++++++++++
test/support/download/git_keywords.py | 84 ++++++++++++++
test/support/download/git_util.py | 196 +++++++++++++++++++++++++++++++++
4 files changed, 483 insertions(+)
create mode 100755 test/support/download/git
create mode 100755 test/support/download/git_keywords.py
create mode 100644 test/support/download/git_util.py
diff --git a/support/download/git b/support/download/git
index 7c44c65..f7eef15 100755
--- a/support/download/git
+++ b/support/download/git
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# Please test this script using test/support/download/git
# We want to catch any unexpected failure, and exit immediately
set -e
diff --git a/test/support/download/git b/test/support/download/git
new file mode 100755
index 0000000..3aa4d43
--- /dev/null
+++ b/test/support/download/git
@@ -0,0 +1,202 @@
+#!/usr/bin/env python
+# this script conforms to pyflakes && pep8 --ignore=E501
+
+# BEFORE calling this script, make sure:
+# - there is a ssh server installed
+# - there is a key pair generated that allow the current user to run 'ssh localhost' without being prompted for entering a password
+
+# TODO test recursive submodules
+# TODO test another transport protocols: git, https, ... - need manual setup of git server
+# TODO test extra download options, especially --reference
+# TODO test which cases should have optimized download, and which cases use the full fetch
+
+
+import git_keywords
+
+
+class TestSuiteSupportDownloadGitRefs(git_keywords.Keywords):
+ @classmethod
+ def setUpClass(cls):
+ print("Setup")
+ cls.given_the_the_patch_instrumenting_the_source_code_is_present()
+ cls.given_the_localhost_can_be_accessed_by_current_user_without_a_password_prompt()
+ cls.given_the_test_will_not_overwrite_user_files()
+ cls.create_a_fake_package()
+ cls.create_a_config_file()
+
+ @classmethod
+ def tearDownClass(cls):
+ print("Teardown")
+ cls.remove_temporary_files()
+
+ def setUp(self):
+ self.create_test_repo()
+ self.create_commit_in_test_repo() # initial commit
+ self.remove_dowloaded_tarball()
+
+ def tearDown(self):
+ self.remove_test_repo()
+
+ def test101_tag_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_tag_named("feature101")
+ self.when_i_download_the_version("feature101")
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists("feature101")
+
+ def test102_tag_can_be_preferred_over_branch_by_using_full_name(self):
+ tag_sha1 = self.given_there_is_a_commit_pointed_only_by_the_tag_named("feature102")
+ branch_sha1 = self.given_there_is_a_commit_pointed_only_by_the_branch_named("feature102")
+ self.given_the_commits_are_not_the_same(branch_sha1, tag_sha1)
+ self.when_i_download_the_version("refs/tags/feature102")
+ self.then_i_can_see_the_downloaded_sha1_is(tag_sha1)
+ self.then_i_can_see_the_tarball_exists("refs_tags_feature102")
+
+ def test201_sha1_not_branch_head_but_in_a_branch_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_branch_named("feature201")
+ sha1_head = self.given_there_is_another_commit_in_the_upstream()
+ self.given_the_commits_are_not_the_same(sha1, sha1_head)
+ self.when_i_download_the_version(sha1)
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists(sha1)
+
+ def test201b_sha1_not_branch_head_but_in_a_branch_does_not_use_optimized_download(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_branch_named("feature201")
+ sha1_head = self.given_there_is_another_commit_in_the_upstream()
+ self.given_the_commits_are_not_the_same(sha1, sha1_head)
+ self.when_i_download_the_version(sha1)
+ self.then_i_can_see_the_tarball_exists(sha1)
+ self.then_i_can_see_a_full_clone_was_needed()
+
+ def test202_sha1_of_branch_head_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_branch_named("feature202")
+ self.when_i_download_the_version(sha1)
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists(sha1)
+
+ def test203_sha1_of_branch_head_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_branch_named("feature203")
+ partial_sha1 = sha1[:20]
+ self.when_i_download_the_version(partial_sha1)
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists(partial_sha1)
+
+ def test204_sha1_of_commit_pointed_by_a_tag_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_tag_named("feature204")
+ self.when_i_download_the_version(sha1)
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists(sha1)
+
+ def test204b_sha1_of_commit_pointed_by_a_tag_uses_optimized_download(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_tag_named("feature204b")
+ self.when_i_download_the_version(sha1)
+ self.then_i_can_see_the_tarball_exists(sha1)
+ self.then_i_can_see_a_full_clone_was_not_needed()
+
+ def test205_sha1_head_of_branch_with_slash_in_the_name_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_branch_named("feat/ure205")
+ self.when_i_download_the_version(sha1)
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists(sha1)
+
+ def test206_sha1_not_branch_head_but_in_a_branch_with_slash_in_the_name_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_branch_named("feat/ure206")
+ sha1_head = self.given_there_is_another_commit_in_the_upstream()
+ self.given_the_commits_are_not_the_same(sha1, sha1_head)
+ self.when_i_download_the_version(sha1)
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists(sha1)
+
+ def test207_partial_sha1_of_tagged_commit_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_tag_named("feature207")
+ partial_sha1 = sha1[:20]
+ self.when_i_download_the_version(partial_sha1)
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists(partial_sha1)
+
+ def test208_sha1_of_special_ref_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_special_ref_named("changes/02/2/8")
+ self.when_i_download_the_version(sha1)
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists(sha1)
+
+ def test209_sha1_of_merged_special_ref_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_special_ref_named("changes/02/2/9")
+ self.given_the_same_commit_is_pointed_by_the_branch_named("feature209")
+ self.when_i_download_the_version(sha1)
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists(sha1)
+
+ def test210_partial_sha1_of_special_ref_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_special_ref_named("changes/02/2/10")
+ partial_sha1 = sha1[:20]
+ self.when_i_download_the_version(partial_sha1)
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists(partial_sha1)
+
+ def test211_partial_sha1_of_merged_special_ref_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_special_ref_named("changes/02/2/11")
+ self.given_the_same_commit_is_pointed_by_the_branch_named("feature211")
+ sha1_head = self.given_there_is_another_commit_in_the_upstream()
+ self.given_the_commits_are_not_the_same(sha1, sha1_head)
+ partial_sha1 = sha1[:20]
+ self.when_i_download_the_version(partial_sha1)
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists(partial_sha1)
+
+ def test301_special_ref_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_special_ref_named("changes/03/3/1")
+ self.when_i_download_the_version("refs/changes/03/3/1")
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists("refs_changes_03_3_1")
+
+ def test302_merged_special_ref_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_special_ref_named("changes/03/3/2")
+ self.given_the_same_commit_is_pointed_by_the_branch_named("feature302")
+ self.when_i_download_the_version("refs/changes/03/3/2")
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists("refs_changes_03_3_2")
+
+ def test401_branch_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_branch_named("feature401")
+ self.when_i_download_the_version("feature401")
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists("feature401")
+
+ def test402_branch_is_preferred_over_tag(self): # XXX behavior enforced by git clone / fetch / checkout
+ tag_sha1 = self.given_there_is_a_commit_pointed_only_by_the_tag_named("feature402")
+ branch_sha1 = self.given_there_is_a_commit_pointed_only_by_the_branch_named("feature402")
+ self.given_the_commits_are_not_the_same(branch_sha1, tag_sha1)
+ self.when_i_download_the_version("feature402")
+ self.then_i_can_see_the_downloaded_sha1_is(branch_sha1)
+ self.then_i_can_see_the_tarball_exists("feature402")
+
+ def test403_branch_with_slash_in_the_name_can_be_used(self):
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_branch_named("feat/ure403")
+ self.when_i_download_the_version("feat/ure403")
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists("feat_ure403")
+
+ def test501_tag_can_be_used_when_there_are_submodules_without_support_to_git_modules(self):
+ self.given_there_is_a_submodule_named("submodule501")
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_tag_named("feature501")
+ self.when_i_download_the_version("feature501")
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists("feature501")
+ self.then_i_can_see_the_tarball_does_not_contain_the_submodule("feature501", "submodule501")
+
+ def test502_tag_can_be_used_when_there_are_submodules_with_support_to_git_modules(self):
+ self.given_there_is_a_submodule_named("submodule502")
+ sha1 = self.given_there_is_a_commit_pointed_only_by_the_tag_named("feature502")
+ self.when_i_download_with_support_to_git_modules_the_version("feature502")
+ self.then_i_can_see_the_downloaded_sha1_is(sha1)
+ self.then_i_can_see_the_tarball_exists("feature502")
+ self.then_i_can_see_the_tarball_contains_the_submodule("feature502", "submodule502")
+
+ def test901_head_can_be_used(self):
+ self.when_i_download_the_version("HEAD")
+ self.then_i_can_see_the_tarball_exists("HEAD")
+
+
+if __name__ == '__main__':
+ suite = git_keywords.unittest.TestLoader().loadTestsFromTestCase(TestSuiteSupportDownloadGitRefs)
+ git_keywords.unittest.TextTestRunner(verbosity=2).run(suite)
diff --git a/test/support/download/git_keywords.py b/test/support/download/git_keywords.py
new file mode 100755
index 0000000..34ee5eb
--- /dev/null
+++ b/test/support/download/git_keywords.py
@@ -0,0 +1,84 @@
+#!/usr/bin/env python
+# this script conforms to pyflakes && pep8 --ignore=E501
+
+import git_util
+import unittest
+
+
+class Keywords(unittest.TestCase, git_util.Utilities):
+ @classmethod
+ def given_the_the_patch_instrumenting_the_source_code_is_present(cls):
+ cls.check_the_patch_instrumenting_the_source_code_is_present()
+
+ @classmethod
+ def given_the_localhost_can_be_accessed_by_current_user_without_a_password_prompt(cls):
+ cls.check_user_can_run_command_in_localhost_through_ssh_without_password_prompt()
+
+ @classmethod
+ def given_the_test_will_not_overwrite_user_files(cls):
+ cls.check_temporary_files_do_not_exist_or_exit()
+
+ def given_there_is_a_commit_pointed_only_by_the_branch_named(self, branch):
+ self.checkout_a_new_branch_in_test_repo(branch)
+ sha1 = self.create_commit_in_test_repo()
+ return sha1
+
+ def given_the_same_commit_is_pointed_by_the_branch_named(self, branch):
+ self.checkout_a_new_branch_in_test_repo(branch)
+
+ def given_there_is_another_commit_in_the_upstream(self):
+ sha1 = self.create_commit_in_test_repo()
+ return sha1
+
+ def given_there_is_a_commit_pointed_only_by_the_tag_named(self, tag):
+ sha1 = self.create_commit_in_test_repo()
+ self.create_a_new_tag_in_test_repog(tag)
+ self.create_commit_in_test_repo()
+ return sha1
+
+ def given_there_is_a_commit_pointed_only_by_the_special_ref_named(self, special_ref):
+ self.detach_head_in_test_repo()
+ sha1 = self.create_commit_in_test_repo()
+ self.create_a_special_ref_in_test_repo(special_ref)
+ self.checkout_branch_in_test_repo("master")
+ return sha1
+
+ def given_the_commits_are_not_the_same(self, sha1_1st, sha1_2nd):
+ self.assertNotEqual(sha1_1st, sha1_2nd)
+
+ def given_the_commits_are_the_same(self, sha1_1st, sha1_2nd):
+ self.assertEqual(sha1_1st, sha1_2nd)
+
+ def when_i_download_the_version(self, version):
+ result = self.download_source_using_buildroot(version)
+ self.assertEqual(0, result, "download of version %s failed" % version)
+
+ def when_i_download_with_support_to_git_modules_the_version(self, version):
+ result = self.download_source_using_buildroot(version, 'PACKAGE_GIT_SUBMODULES=YES')
+ self.assertEqual(0, result, "download with support to git modules of version %s failed" % version)
+
+ def then_i_can_see_the_tarball_exists(self, version):
+ result = self.check_tarball_exists(version)
+ self.assertTrue(result, "Tarball for version %s does not exist" % version)
+
+ def then_i_can_see_the_tarball_contains_the_submodule(self, version, submodule):
+ result = self.check_tarball_contains_the_submodule(version, submodule)
+ self.assertTrue(result, "Tarball does not contain the submodule data")
+
+ def then_i_can_see_the_tarball_does_not_contain_the_submodule(self, version, submodule):
+ result = self.check_tarball_contains_the_submodule(version, submodule)
+ self.assertFalse(result, "Tarball does contain the submodule data, but it shouldn't")
+
+ def then_i_can_see_the_downloaded_sha1_is(self, sha1):
+ self.check_sha1_appeared_in_build_log(sha1)
+
+ def then_i_can_see_a_full_clone_was_not_needed(self):
+ result = self.check_in_the_build_log_if_a_full_clone_or_fetch_was_needed()
+ self.assertFalse(result, "A full clone or fetch was needed")
+
+ def then_i_can_see_a_full_clone_was_needed(self):
+ result = self.check_in_the_build_log_if_a_full_clone_or_fetch_was_needed()
+ self.assertTrue(result, "A full clone or fetch was not needed")
+
+ def given_there_is_a_submodule_named(self, submodule):
+ self.create_a_submodule(submodule)
diff --git a/test/support/download/git_util.py b/test/support/download/git_util.py
new file mode 100644
index 0000000..5e6e3e5
--- /dev/null
+++ b/test/support/download/git_util.py
@@ -0,0 +1,196 @@
+#!/usr/bin/env python
+# this script conforms to pyflakes && pep8 --ignore=E501
+
+import os
+import subprocess
+
+
+class Utilities():
+ commit_data = 10000
+ # TODO allow customizing the url
+ remote_url_of_local_repo = subprocess.Popen('echo ssh://localhost/$(readlink -f $(pwd))/package/package', shell=True, stdout=subprocess.PIPE).communicate()[0].strip()
+ # debug intentionally left on. It pollutes the test log but it shows relevant info when a test fails
+ # TODO allow setting this at command line
+ verbose = True
+
+ @classmethod
+ def run_commands_or_exit(cls, commands):
+ for command in commands:
+ result = os.system(command)
+ if result != 0:
+ raise OSError("command '%s' failed with error %d" % (command, result / 256))
+
+ @classmethod
+ def check_the_patch_instrumenting_the_source_code_is_present(cls):
+ cls.run_commands_or_exit(
+ ['grep -q "Checked out" support/download/git'])
+
+ @classmethod
+ def check_user_can_run_command_in_localhost_through_ssh_without_password_prompt(cls):
+ cls.run_commands_or_exit(
+ ['ssh localhost true'])
+
+ @classmethod
+ def create_a_fake_package(cls):
+ cls.run_commands_or_exit(
+ ['mkdir package/package',
+ 'echo "PACKAGE_VERSION = invalid" >> package/package/package.mk',
+ 'echo "PACKAGE_SITE = %s/package.git" >> package/package/package.mk' % cls.remote_url_of_local_repo,
+ 'echo "PACKAGE_SITE_METHOD = git" >> package/package/package.mk',
+ 'echo \'$(eval $(generic-package))\' >> package/package/package.mk'])
+
+ @classmethod
+ def create_a_config_file(cls):
+ cls.run_commands_or_exit(
+ ['make defconfig >/dev/null 2>/dev/null',
+ 'echo \'BR2_DL_DIR="$(TOPDIR)/package/package/dl"\' >> .config',
+ 'echo \'BR2_BACKUP_SITE=""\' >> .config',
+ 'make oldconfig >/dev/null 2>/dev/null'])
+
+ @classmethod
+ def check_temporary_files_do_not_exist_or_exit(cls):
+ if not os.path.isfile('support/download/git'):
+ raise OSError('These tests must be run from Buildroot base dir')
+ if os.path.isfile('.config'):
+ raise OSError('File .config should not exist')
+ if os.path.isfile('stdout'): # FIXME ugly abstraction error, but it does the job
+ raise OSError('File stdout should not exist')
+ if os.path.isdir('package/package/'):
+ raise OSError('The fake package package/package/ should not exist')
+
+ @classmethod
+ def remove_temporary_files(cls):
+ cls.run_commands_or_exit(
+ ['rm -rf package/package/',
+ 'rm -f .config',
+ 'rm -f stdout'])
+
+ def remove_dowloaded_tarball(self):
+ self.__class__.run_commands_or_exit(
+ ['rm -rf package/package/dl'])
+
+ def run_commands_or_fail(self, commands):
+ for command in commands:
+ result = os.system(command)
+ # FIXME ugly abstraction error, but it does the job
+ self.assertEqual(0, result, "command %s failed with error %d" % (command, result / 256))
+
+ def create_test_repo(self):
+ self.run_commands_or_fail(
+ ['git init -q package/package/package.git'])
+
+ def remove_test_repo(self):
+ self.run_commands_or_fail(
+ ['rm -rf package/package/package.git'])
+
+ def create_commit_in_test_repo(self):
+ base_dir = os.path.realpath('.')
+ os.chdir('package/package/package.git')
+ self.run_commands_or_fail(
+ ['echo %i > %i' % (self.commit_data, self.commit_data),
+ 'git add %i' % self.commit_data,
+ 'git commit -q -m %i' % self.commit_data])
+ self.commit_data += 1
+ sha1 = self.get_sha1_from_checkout()
+ os.chdir(base_dir)
+ return sha1
+
+ def checkout_a_new_branch_in_test_repo(self, branch):
+ base_dir = os.path.realpath('.')
+ os.chdir('package/package/package.git')
+ self.run_commands_or_fail(
+ ['git branch -q -f "%s"' % branch,
+ 'git checkout -q -f "%s"' % branch])
+ os.chdir(base_dir)
+
+ def create_a_special_ref_in_test_repo(self, special_ref):
+ base_dir = os.path.realpath('.')
+ os.chdir('package/package/package.git')
+ directory = '/'.join(special_ref.split('/')[:-1])
+ self.run_commands_or_fail(
+ ['git branch -q -f temporary',
+ 'mkdir -p .git/refs/%s' % directory,
+ 'mv .git/refs/heads/temporary .git/refs/%s' % special_ref])
+ os.chdir(base_dir)
+
+ def create_a_new_tag_in_test_repog(self, tag):
+ base_dir = os.path.realpath('.')
+ os.chdir('package/package/package.git')
+ self.run_commands_or_fail(
+ ['git tag -a "%s" -m "%s"' % (tag, self.commit_data)])
+ self.commit_data += 1
+ os.chdir(base_dir)
+
+ def download_source_using_buildroot(self, version, extra_options=''):
+ redirect_stderr = '2>/dev/null'
+ if self.verbose:
+ os.system('echo; git -C package/package/package.git log --all --graph --abbrev-commit --pretty=oneline --decorate; git ls-remote package/package/package.git')
+ redirect_stderr = ''
+ # Using 'make PACKAGE_VERSION=' does not work for references with slash, so write to the .mk
+ self.run_commands_or_fail(
+ ['sed -e \'s#^PACKAGE_VERSION.*$#PACKAGE_VERSION = %s#g\' -i package/package/package.mk' % version])
+ result = os.system(
+ 'make %s package-dirclean package-source >stdout %s' % (extra_options, redirect_stderr))
+ if self.verbose:
+ os.system('echo -; cat stdout; echo -')
+ return result
+
+ def check_tarball_exists(self, expected_download):
+ if self.verbose:
+ os.system('echo -n "dl: " ; ls package/package/dl/')
+ return os.path.isfile('package/package/dl/package-' + expected_download + '.tar.gz')
+
+ def get_sha1_from_checkout(self):
+ pipe = subprocess.Popen("git rev-parse HEAD", shell=True, stdout=subprocess.PIPE)
+ sha1 = pipe.communicate()[0].strip()
+ if len(sha1) != 40:
+ raise OSError("Failed to create commit, got sha1 '%s'" % sha1)
+ return sha1
+
+ def check_sha1_appeared_in_build_log(self, sha1): # it needs some instrumentation in the code
+ self.run_commands_or_fail(
+ ['grep -q "Checked out \'%s\'" stdout' % sha1])
+
+ def check_in_the_build_log_if_a_full_clone_or_fetch_was_needed(self): # it is very coupled to the source code
+ return 0 == os.system('grep -q "Doing full" stdout')
+
+ def detach_head_in_test_repo(self):
+ base_dir = os.path.realpath('.')
+ os.chdir('package/package/package.git')
+ self.run_commands_or_fail(
+ ['git checkout -q -f HEAD~0'])
+ os.chdir(base_dir)
+
+ def checkout_branch_in_test_repo(self, branch):
+ base_dir = os.path.realpath('.')
+ os.chdir('package/package/package.git')
+ self.run_commands_or_fail(
+ ['git checkout -q -f "%s"' % branch])
+ os.chdir(base_dir)
+
+ def create_a_submodule(self, submodule):
+ base_dir = os.path.realpath('.')
+ submodule_dir = 'package/package/%s.git' % submodule
+ self.run_commands_or_fail(
+ ['git init -q %s' % submodule_dir])
+ os.chdir(submodule_dir)
+ self.run_commands_or_fail(
+ ['echo %s > %s' % (submodule, submodule),
+ 'git add %s' % submodule,
+ 'git commit -q -m %s' % submodule])
+ os.chdir(base_dir)
+ os.chdir('package/package/package.git')
+ self.run_commands_or_fail(
+ ['git submodule add %s/%s.git ./%s 2>/dev/null' % (self.__class__.remote_url_of_local_repo, submodule, submodule),
+ 'echo %i > %i' % (self.commit_data, self.commit_data),
+ 'git add %i' % self.commit_data,
+ 'git commit -q -m %i' % self.commit_data])
+ self.commit_data += 1
+ sha1 = self.get_sha1_from_checkout()
+ os.chdir(base_dir)
+ return sha1
+
+ def check_tarball_contains_the_submodule(self, expected_download, submodule):
+ if self.verbose:
+ os.system('echo "tar:"; tar -tf "package/package/dl/package-%s.tar.gz"' % expected_download)
+ return 0 == os.system('tar -tf "package/package/dl/package-%s.tar.gz" | grep -q "%s/%s"' % (expected_download, submodule, submodule))
--
2.9.3
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 3/3] support/download/git: do not use git clone
2016-11-01 19:33 [Buildroot] [PATCH 1/3] support/download/git: log checked out sha1 Ricardo Martincoski
2016-11-01 19:33 ` [Buildroot] [PATCH 2/3] test/support/download/git: new test Ricardo Martincoski
@ 2016-11-01 19:33 ` Ricardo Martincoski
2016-11-06 1:19 ` Arnout Vandecappelle
2016-11-05 21:50 ` [Buildroot] [PATCH 1/3] support/download/git: log checked out sha1 Arnout Vandecappelle
2 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2016-11-01 19:33 UTC (permalink / raw)
To: buildroot
The logic of the script was completely rewritten based in some ideas
discussed during the review of [1].
Always using git init + git fetch has these advantages:
- git fetch works with all kinds of refs, while git clone supports only
branches and tags without the ref/heads/ and ref/tags/ prefixes;
- a shallow fetch can be done for the head of those refs.
The remote is first asked for its references using git ls-remote saving
the output to a file.
This file is later on compared to the desired change set, determining if
it is a branch, tag, special ref or sha1.
A concern that arrives from this method is that the remote can change
between the git ls-remote and the git fetch, but in this case, once the
script creates a shallow fetch, the checkout does what it is expected:
- for a named reference (branch, tag or special ref), the fetch and
checkout are successful using the "new" sha1 from the remote;
- for a sha1 of a branch, the fetch fails (because the sha1 it is not
anymore at branch head), falling back to a successful full fetch;
- for a sha1 of a commit pointed by a tag (that should not be changed
but it can be changed) the same behavior of a sha1 of a branch takes
place;
- for a sha1 only accessible by a special refs, the checkout fails,
falling back to an unsuccessful checkout after full fetch. This can be
caused only be a error from a developer.
An analytical solution is used instead of a single awk line. It makes
each line of code simple: grep to check the entry is in the ls-remote
output and awk to actually get the reference to use.
[1] http://patchwork.ozlabs.org/patch/681841/
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
---
support/download/git | 91 ++++++++++++++++++++++++++++++++++------------------
1 file changed, 60 insertions(+), 31 deletions(-)
diff --git a/support/download/git b/support/download/git
index f7eef15..1411d94 100755
--- a/support/download/git
+++ b/support/download/git
@@ -39,43 +39,72 @@ _git() {
eval ${GIT} "${@}"
}
-# Try a shallow clone, since it is faster than a full clone - but that only
-# works if the version is a ref (tag or branch). Before trying to do a shallow
-# clone we check if ${cset} is in the list provided by git ls-remote. If not
-# we fall back on a full clone.
-#
-# Messages for the type of clone used are provided to ease debugging in case of
-# problems
-git_done=0
-if [ -n "$(_git ls-remote "'${repo}'" "'${cset}'" 2>&1)" ]; then
- printf "Doing shallow clone\n"
- if _git clone ${verbose} "${@}" --depth 1 -b "'${cset}'" "'${repo}'" "'${basename}'"; then
- git_done=1
- else
- printf "Shallow clone failed, falling back to doing a full clone\n"
+_git init ${verbose} "'${basename}'"
+
+pushd "${basename}" >/dev/null
+
+# Save the temporary file inside the .git directory that will be deleted after the checkout is done.
+tmpf=".git/ls-remote"
+_git ls-remote "'${repo}'" > "${tmpf}"
+
+do_a_shallow_fetch=0
+if grep "\<\(\|\(\|refs/\)\(heads\|tags\)/\)${cset}$" "${tmpf}" >/dev/null 2>&1; then
+ printf "The desired version is a named reference\n"
+ # Support branches and tags in the simplified form.
+ # Support branches and tags and special refs in the complete form refs/heads/branch.
+ # When the name is ambiguous, the branch will be selected (by git fetch or git clone).
+ ref="${cset}"
+ do_a_shallow_fetch=1
+elif grep "^${cset}" "${tmpf}" >/dev/null 2>&1; then
+ printf "The desired version is a sha1\n"
+ if [ ${#cset} -lt 40 -a "1" != "$(awk "/^${cset}/{print \$1|\"sort -u | wc -l\"}" "${tmpf}")" ]; then
+ printf "Ambiguous partial sha1\n"
+ awk "/^${cset}/{print \$1|\"sort -u | wc -l\"}" "${tmpf}"
+ exit 1
+ fi
+ # Accept full or unambiguous partial sha1. A sha1 can be referenced by many names.
+ # Prefer sha1 of commit pointed by a tag or sha1 of the tag itself,
+ # then sha1 pointed by any ref/*, and only then sha1 pointed by *HEAD.
+ ref="$(awk -F'[\t^]' "/^${cset}.*\trefs\/tags\//{ print \$2; exit }" "${tmpf}")"
+ if [ -z "${ref}" ]; then
+ ref="$(awk -F'[\t^]' "/^${cset}.*\trefs\//{ print \$2; exit }" "${tmpf}")"
+ fi
+ if [ -z "${ref}" ]; then
+ # sha1 is referenced by HEAD
+ ref="$(awk -F'[\t^]' "/^${cset}/{ print \$2; exit }" "${tmpf}")"
+ fi
+ if [ -n "${ref}" ]; then
+ printf "Fetching '%s' to get '%s'\n" "${ref}" "${cset}"
+ do_a_shallow_fetch=1
fi
-fi
-if [ ${git_done} -eq 0 ]; then
- printf "Doing full clone\n"
- _git clone ${verbose} "${@}" "'${repo}'" "'${basename}'"
fi
-pushd "${basename}" >/dev/null
+git_fetch_done=0
+if [ ${do_a_shallow_fetch} -eq 1 ]; then
+ printf "Doing shallow fetch\n"
+ if _git fetch ${verbose} "${@}" --depth 1 "'${repo}'" "'${ref}:refs/buildroot/${cset}'" 2>&1; then
+ git_fetch_done=1
+ else
+ # It catches the case the remote changed after ls-remote.
+ printf "Shallow fetch failed, falling back to doing a full fetch\n"
+ fi
+fi
-# Try to get the special refs exposed by some forges (pull-requests for
-# github, changes for gerrit...). There is no easy way to know whether
-# the cset the user passed us is such a special ref or a tag or a sha1
-# or whatever else. We'll eventually fail at checking out that cset,
-# below, if there is an issue anyway. Since most of the cset we're gonna
-# have to clone are not such special refs, consign the output to oblivion
-# so as not to alarm unsuspecting users, but still trace it as a warning.
-if ! _git fetch origin "'${cset}:${cset}'" >/dev/null 2>&1; then
- printf "Could not fetch special ref '%s'; assuming it is not special.\n" "${cset}"
+git_checkout_done=0
+if [ ${git_fetch_done} -eq 1 ]; then
+ if _git checkout -q "'refs/buildroot/${cset}'" 2>&1; then
+ git_checkout_done=1
+ else
+ printf "Checkout failed, falling back to doing a full fetch\n"
+ fi
fi
-# Checkout the required changeset, so that we can update the required
-# submodules.
-_git checkout -q "'${cset}'"
+if [ ${git_checkout_done} -eq 0 ]; then
+ printf "Doing full fetch\n"
+ _git remote add origin "'${repo}'"
+ _git fetch ${verbose} "${@}" origin
+ _git checkout -q "'${cset}'"
+fi
# Get date of commit to generate a reproducible archive.
# %cD is RFC2822, so it's fully qualified, with TZ and all.
--
2.9.3
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 2/3] test/support/download/git: new test
2016-11-01 19:33 ` [Buildroot] [PATCH 2/3] test/support/download/git: new test Ricardo Martincoski
@ 2016-11-01 21:27 ` Ricardo Martincoski
2016-11-06 0:19 ` Arnout Vandecappelle
` (2 subsequent siblings)
3 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2016-11-01 21:27 UTC (permalink / raw)
To: buildroot
All,
On Tue, Nov 01, 2016 at 05:33 PM, Ricardo Martincoski <ricardo.martincoski@datacom.ind.br> wrote:
[snip]
WARNING to anyone willing to test this script:
- please test it :)
- but, since it is a experimental script, I recommend to run it in a complete
SEPARATE new clone of buildroot in order to avoid any problems if the script
behaves poorly in some scenario (I don't want anyone risking to loose any work
in progress!). I didn't see any problem in the 2 days I am using it, but
it's always better to be safe!
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 1/3] support/download/git: log checked out sha1
2016-11-01 19:33 [Buildroot] [PATCH 1/3] support/download/git: log checked out sha1 Ricardo Martincoski
2016-11-01 19:33 ` [Buildroot] [PATCH 2/3] test/support/download/git: new test Ricardo Martincoski
2016-11-01 19:33 ` [Buildroot] [PATCH 3/3] support/download/git: do not use git clone Ricardo Martincoski
@ 2016-11-05 21:50 ` Arnout Vandecappelle
2016-11-06 23:17 ` Ricardo Martincoski
2 siblings, 1 reply; 108+ messages in thread
From: Arnout Vandecappelle @ 2016-11-05 21:50 UTC (permalink / raw)
To: buildroot
On 01-11-16 20:33, Ricardo Martincoski wrote:
> In preparation to have automated tests for this script, display the
> checked out sha1 in the build log.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> ---
> Another solution would be to use an environment variable set by the test
> to print that info, e.g. if [ "1" == "${GIT_DOWNLOAD_AUTOTEST}" ]; then
> ---
> support/download/git | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/support/download/git b/support/download/git
> index 7921411..7c44c65 100755
> --- a/support/download/git
> +++ b/support/download/git
> @@ -85,6 +85,12 @@ if [ ${recurse} -eq 1 ]; then
> _git submodule update --init --recursive
> fi
>
> +# Log the sha1. It can be used for automated tests and debug.
> +if [ -z ${verbose} ]; then
It's quite weird to check that verbose is empty, but that's how the variable is
defined... Perhaps s/verbose/quiet/ would be a good idea...
Regards,
Arnout
> + sha1=$(_git rev-parse HEAD)
> + printf "Checked out '%s'.\n" "${sha1}"
> +fi
> +
> # We do not need the .git dir; we keep other .git files, in case they
> # are the only files in their directory.
> rm -rf .git
>
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 2/3] test/support/download/git: new test
2016-11-01 19:33 ` [Buildroot] [PATCH 2/3] test/support/download/git: new test Ricardo Martincoski
2016-11-01 21:27 ` Ricardo Martincoski
@ 2016-11-06 0:19 ` Arnout Vandecappelle
2016-11-06 0:51 ` Arnout Vandecappelle
2016-12-02 2:34 ` Ricardo Martincoski
2016-11-06 1:24 ` Arnout Vandecappelle
2017-08-26 22:20 ` [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build() Ricardo Martincoski
3 siblings, 2 replies; 108+ messages in thread
From: Arnout Vandecappelle @ 2016-11-06 0:19 UTC (permalink / raw)
To: buildroot
On 01-11-16 20:33, Ricardo Martincoski wrote:
> This test assumes:
> - it is run from the buildroot base dir;
> - there is a ssh server installed;
> - there is a key pair to allow the current user to ssh to localhost
> without being prompted for a password;
> - there is no ./.config ./stdout ./package/package;
>
> For each testcase, it creates a new git repo, and then download from
> this repo using a fake package named 'package'.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> ---
> This test needs some instrumentation in the source code (previous patch)
> to work properly.
> ---
> support/download/git | 1 +
> test/support/download/git | 202 ++++++++++++++++++++++++++++++++++
> test/support/download/git_keywords.py | 84 ++++++++++++++
> test/support/download/git_util.py | 196 +++++++++++++++++++++++++++++++++
> 4 files changed, 483 insertions(+)
> create mode 100755 test/support/download/git
> create mode 100755 test/support/download/git_keywords.py
> create mode 100644 test/support/download/git_util.py
>
> diff --git a/support/download/git b/support/download/git
> index 7c44c65..f7eef15 100755
> --- a/support/download/git
> +++ b/support/download/git
> @@ -1,4 +1,5 @@
> #!/usr/bin/env bash
> +# Please test this script using test/support/download/git
>
> # We want to catch any unexpected failure, and exit immediately
> set -e
> diff --git a/test/support/download/git b/test/support/download/git
It would be nice if you could put this in a place that is convenient for
integration with the overall test infrastructure that Thomas has been working
on. However, I can't find the git repo with that test infrastructure anywhere...
Thomas?
I would call it testgit.py instead, so it can be run with 'python -m unittest
discover -s test'.
> new file mode 100755
> index 0000000..3aa4d43
> --- /dev/null
> +++ b/test/support/download/git
> @@ -0,0 +1,202 @@
> +#!/usr/bin/env python
> +# this script conforms to pyflakes && pep8 --ignore=E501
> +
> +# BEFORE calling this script, make sure:
> +# - there is a ssh server installed
> +# - there is a key pair generated that allow the current user to run 'ssh localhost' without being prompted for entering a password
That's not very nice. It would be better to use a git daemon (and the git://
instead of ssh:// protocol). You can start it with:
git daemon --reuseaddr --verbose --listen=localhost --port={port}
It will fail to start if the port is already in use, so you can try a different
port if you find 'Address already in use' in the output. And it will output
'Ready to rumble' once it started, so you can use that to delay the test until
the daemon is operational.
> +
> +# TODO test recursive submodules
> +# TODO test another transport protocols: git, https, ... - need manual setup of git server
That's not very useful, we're pretty much agnostic of the protocol used.
> +# TODO test extra download options, especially --reference
Er, that option isn't used anywhere at the moment...
> +# TODO test which cases should have optimized download, and which cases use the full fetch
> +
> +
> +import git_keywords
> +
> +
> +class TestSuiteSupportDownloadGitRefs(git_keywords.Keywords):
I'm going to skip the tests themselves for the moment, because I first want to
comment on the test infrastructure (which is unfortunately further below). I
have a *lot* of remarks on the pythonesqueness of the code. That's not so great,
because:
- it's a lot of bikeshedding and doesn't affect the functionality (much);
- it's just my opinion and I'm not even much of a Python expert;
- changing it may be a lot of work and it's unfortunate to have a patch held up
because of it (especially adding a test, because that is *always useful*);
- these things can anyway be rewritten later, after the patch has been accepted.
So feel free to disregard some or all of my bikeshedding remarks.
There are, however, also a large number of functional remarks. I hope you'll be
able to distinguish them :-)
[snip]
> +if __name__ == '__main__':
> + suite = git_keywords.unittest.TestLoader().loadTestsFromTestCase(TestSuiteSupportDownloadGitRefs)
It's not very nice to rely on git_keywords' import of unittest. Just import
unittest here.
Any reason why the usual unittest.main() doesn't work?
> + git_keywords.unittest.TextTestRunner(verbosity=2).run(suite)
> diff --git a/test/support/download/git_keywords.py b/test/support/download/git_keywords.py
> new file mode 100755
> index 0000000..34ee5eb
> --- /dev/null
> +++ b/test/support/download/git_keywords.py
> @@ -0,0 +1,84 @@
> +#!/usr/bin/env python
> +# this script conforms to pyflakes && pep8 --ignore=E501
> +
> +import git_util
> +import unittest
> +
> +
> +class Keywords(unittest.TestCase, git_util.Utilities):
This class looks like nothing more than a GWT wrapper around git_util. I don't
think that that's very useful. Buildroot developers are generally developers,
not test engineers who are not so used to programming, and I think that for most
BR developers the GWT standard is more confusing than useful.
Second, instead of all these given's etc., it is much simpler to set up a
single test repo, once, that has all the features you need. Setting it up could
either be through explicit git commands, but I think it's even simpler if you
just create a bare repository and commit that into Buildroot. It's not very nice
for review, but I think it simplifies things enough to make it worth it. And
anyway, the contents of the objects are not so important for review. If you do
this, remove the hooks and info directories to make it smaller, and make sure
the objects directory is binary. You know, I'll send a patch to show you how.
> + @classmethod
> + def given_the_the_patch_instrumenting_the_source_code_is_present(cls):
> + cls.check_the_patch_instrumenting_the_source_code_is_present()
> +
> + @classmethod
> + def given_the_localhost_can_be_accessed_by_current_user_without_a_password_prompt(cls):
> + cls.check_user_can_run_command_in_localhost_through_ssh_without_password_prompt()
> +
> + @classmethod
> + def given_the_test_will_not_overwrite_user_files(cls):
> + cls.check_temporary_files_do_not_exist_or_exit()
> +
> + def given_there_is_a_commit_pointed_only_by_the_branch_named(self, branch):
> + self.checkout_a_new_branch_in_test_repo(branch)
> + sha1 = self.create_commit_in_test_repo()
> + return sha1
> +
> + def given_the_same_commit_is_pointed_by_the_branch_named(self, branch):
> + self.checkout_a_new_branch_in_test_repo(branch)
> +
> + def given_there_is_another_commit_in_the_upstream(self):
> + sha1 = self.create_commit_in_test_repo()
> + return sha1
> +
> + def given_there_is_a_commit_pointed_only_by_the_tag_named(self, tag):
> + sha1 = self.create_commit_in_test_repo()
> + self.create_a_new_tag_in_test_repog(tag)
> + self.create_commit_in_test_repo()
> + return sha1
> +
> + def given_there_is_a_commit_pointed_only_by_the_special_ref_named(self, special_ref):
> + self.detach_head_in_test_repo()
> + sha1 = self.create_commit_in_test_repo()
> + self.create_a_special_ref_in_test_repo(special_ref)
> + self.checkout_branch_in_test_repo("master")
> + return sha1
> +
> + def given_the_commits_are_not_the_same(self, sha1_1st, sha1_2nd):
> + self.assertNotEqual(sha1_1st, sha1_2nd)
This can't be right... An assertion shouldn't occur in a given clause, only in
a then or maybe an unless clause.
> +
> + def given_the_commits_are_the_same(self, sha1_1st, sha1_2nd):
> + self.assertEqual(sha1_1st, sha1_2nd)
> +
> + def when_i_download_the_version(self, version):
> + result = self.download_source_using_buildroot(version)
> + self.assertEqual(0, result, "download of version %s failed" % version)
This is an example why GWT doesn't really work. For GWT, the assert should be
in the then clause. But to be able to do that, you have to store the result in
some stateful object, and that unnecessarily complicates thing. Just use the
asserts directly in the test.
Also, for this specific case, it's IMHO cleaner to raise an exception when the
download fails, which unittest will interpret as a failure as well. Exceptions
are something which are acceptable in a which clause I think. And when a
download failure is expected, use assertRaises.
> +
> + def when_i_download_with_support_to_git_modules_the_version(self, version):
> + result = self.download_source_using_buildroot(version, 'PACKAGE_GIT_SUBMODULES=YES')
> + self.assertEqual(0, result, "download with support to git modules of version %s failed" % version)
> +
> + def then_i_can_see_the_tarball_exists(self, version):
> + result = self.check_tarball_exists(version)
> + self.assertTrue(result, "Tarball for version %s does not exist" % version)
> +
> + def then_i_can_see_the_tarball_contains_the_submodule(self, version, submodule):
> + result = self.check_tarball_contains_the_submodule(version, submodule)
> + self.assertTrue(result, "Tarball does not contain the submodule data")
> +
> + def then_i_can_see_the_tarball_does_not_contain_the_submodule(self, version, submodule):
> + result = self.check_tarball_contains_the_submodule(version, submodule)
> + self.assertFalse(result, "Tarball does contain the submodule data, but it shouldn't")
> +
> + def then_i_can_see_the_downloaded_sha1_is(self, sha1):
> + self.check_sha1_appeared_in_build_log(sha1)
> +
> + def then_i_can_see_a_full_clone_was_not_needed(self):
> + result = self.check_in_the_build_log_if_a_full_clone_or_fetch_was_needed()
> + self.assertFalse(result, "A full clone or fetch was needed")
> +
> + def then_i_can_see_a_full_clone_was_needed(self):
> + result = self.check_in_the_build_log_if_a_full_clone_or_fetch_was_needed()
> + self.assertTrue(result, "A full clone or fetch was not needed")
> +
> + def given_there_is_a_submodule_named(self, submodule):
> + self.create_a_submodule(submodule)
> diff --git a/test/support/download/git_util.py b/test/support/download/git_util.py
> new file mode 100644
> index 0000000..5e6e3e5
> --- /dev/null
> +++ b/test/support/download/git_util.py
> @@ -0,0 +1,196 @@
> +#!/usr/bin/env python
> +# this script conforms to pyflakes && pep8 --ignore=E501
> +
> +import os
> +import subprocess
> +
> +
> +class Utilities():
> + commit_data = 10000
As far as I can see, instances of this class are completely stateless, there is
only the commit_data class attribute that gets updated (for reasons I don't
understand BTW). Python is not Java, so it's perfectly OK to have module
functions. In other words, remove this class and just define everything as
functions of the git_util module.
Anyway, with my suggestion of including the test repo directly rather than
building it up, most of this module becomes redundant.
> + # TODO allow customizing the url
> + remote_url_of_local_repo = subprocess.Popen('echo ssh://localhost/$(readlink -f $(pwd))/package/package', shell=True, stdout=subprocess.PIPE).communicate()[0].strip()
Er, are you trying to do
remote_url_of_local_repo = 'ssh://localhost' + os.path.abspath('package/package')
?
BTW, wrap at around 80 columns when that makes sense.
> + # debug intentionally left on. It pollutes the test log but it shows relevant info when a test fails
> + # TODO allow setting this at command line
> + verbose = True
Perhaps use the logging module.
> +
> + @classmethod
> + def run_commands_or_exit(cls, commands):
> + for command in commands:
> + result = os.system(command)
> + if result != 0:
> + raise OSError("command '%s' failed with error %d" % (command, result / 256))
Use subprocess.check_call instead. The loop is kind of useless, constructing
the list or having multiple check_call's is more or less equivalent.
> +
> + @classmethod
> + def check_the_patch_instrumenting_the_source_code_is_present(cls):
> + cls.run_commands_or_exit(
> + ['grep -q "Checked out" support/download/git'])
Just assume that that patch is present. We're not going to make tests that run
in different version of Buildroot. If something changes in Buildroot, the tests
should be updated along with it.
> +
> + @classmethod
> + def check_user_can_run_command_in_localhost_through_ssh_without_password_prompt(cls):
> + cls.run_commands_or_exit(
> + ['ssh localhost true'])
> +
> + @classmethod
> + def create_a_fake_package(cls):
> + cls.run_commands_or_exit(
> + ['mkdir package/package',
Use os.makedirs instead.
> + 'echo "PACKAGE_VERSION = invalid" >> package/package/package.mk',
> + 'echo "PACKAGE_SITE = %s/package.git" >> package/package/package.mk' % cls.remote_url_of_local_repo,
> + 'echo "PACKAGE_SITE_METHOD = git" >> package/package/package.mk',
> + 'echo \'$(eval $(generic-package))\' >> package/package/package.mk'])
with open('package/package/package.mk', 'wt') as package_mk:
package_mk.write('''\
PACKAGE_VERSION = ...
...''' % remote_url_of_local_repo)
Although actually, .format is preferred over % nowadays.
That said, I think what we're testing here is just the git download wrapper,
not the entire download infrastructure. So I wouldn't create a package, but
instead call support/download/dl-wrapper with the appropriate arguments. This is
a unit test after all, not an integration test.
If you do keep the package, then it shouldn't be created in the Buildroot tree,
but it should be used as a BR2_EXTERNAL package. Also, instead of creating the
files here, just add them to the buildroot tree and use them from the test, like
I proposed for the testgit repository.
> +
> + @classmethod
> + def create_a_config_file(cls):
> + cls.run_commands_or_exit(
> + ['make defconfig >/dev/null 2>/dev/null',
> + 'echo \'BR2_DL_DIR="$(TOPDIR)/package/package/dl"\' >> .config',
BR2_DL_DIR is overridden by the environment variable. I have BR2_DL_DIR in my
environment, so all the tests fail for me :-) So just make sure it is exported.
Or override it on the make commandline whenever you call make, that works as well.
> + 'echo \'BR2_BACKUP_SITE=""\' >> .config',
> + 'make oldconfig >/dev/null 2>/dev/null'])
You shouldn't use the output directory, instead use a O= to some temporary
directory.
> +
> + @classmethod
> + def check_temporary_files_do_not_exist_or_exit(cls):
> + if not os.path.isfile('support/download/git'):
> + raise OSError('These tests must be run from Buildroot base dir')
> + if os.path.isfile('.config'):
> + raise OSError('File .config should not exist')
> + if os.path.isfile('stdout'): # FIXME ugly abstraction error, but it does the job
> + raise OSError('File stdout should not exist')
> + if os.path.isdir('package/package/'):
> + raise OSError('The fake package package/package/ should not exist')
> +
> + @classmethod
> + def remove_temporary_files(cls):
> + cls.run_commands_or_exit(
> + ['rm -rf package/package/',
> + 'rm -f .config',
> + 'rm -f stdout'])
> +
> + def remove_dowloaded_tarball(self):
> + self.__class__.run_commands_or_exit(
> + ['rm -rf package/package/dl'])
Use shutil.rmtree
> +
> + def run_commands_or_fail(self, commands):
> + for command in commands:
> + result = os.system(command)
> + # FIXME ugly abstraction error, but it does the job
> + self.assertEqual(0, result, "command %s failed with error %d" % (command, result / 256))
The only difference with the other run_commands definition is that this one
raises AssertionError instead of OSError. Not much of a difference... If the
difference is important, use subprocess.check() instead.
> +
> + def create_test_repo(self):
> + self.run_commands_or_fail(
> + ['git init -q package/package/package.git'])
> +
> + def remove_test_repo(self):
> + self.run_commands_or_fail(
> + ['rm -rf package/package/package.git'])
> +
> + def create_commit_in_test_repo(self):
> + base_dir = os.path.realpath('.')
> + os.chdir('package/package/package.git')
> + self.run_commands_or_fail(
> + ['echo %i > %i' % (self.commit_data, self.commit_data),
> + 'git add %i' % self.commit_data,
> + 'git commit -q -m %i' % self.commit_data])
> + self.commit_data += 1
> + sha1 = self.get_sha1_from_checkout()
> + os.chdir(base_dir)
> + return sha1
> +
> + def checkout_a_new_branch_in_test_repo(self, branch):
> + base_dir = os.path.realpath('.')
> + os.chdir('package/package/package.git')
> + self.run_commands_or_fail(
> + ['git branch -q -f "%s"' % branch,
> + 'git checkout -q -f "%s"' % branch])
> + os.chdir(base_dir)
> +
> + def create_a_special_ref_in_test_repo(self, special_ref):
> + base_dir = os.path.realpath('.')
> + os.chdir('package/package/package.git')
> + directory = '/'.join(special_ref.split('/')[:-1])
> + self.run_commands_or_fail(
> + ['git branch -q -f temporary',
> + 'mkdir -p .git/refs/%s' % directory,
> + 'mv .git/refs/heads/temporary .git/refs/%s' % special_ref])
You should probably read the git update-ref man page.
> + os.chdir(base_dir)
> +
> + def create_a_new_tag_in_test_repog(self, tag):
> + base_dir = os.path.realpath('.')
> + os.chdir('package/package/package.git')
> + self.run_commands_or_fail(
> + ['git tag -a "%s" -m "%s"' % (tag, self.commit_data)])
> + self.commit_data += 1
> + os.chdir(base_dir)
> +
> + def download_source_using_buildroot(self, version, extra_options=''):
> + redirect_stderr = '2>/dev/null'
> + if self.verbose:
> + os.system('echo; git -C package/package/package.git log --all --graph --abbrev-commit --pretty=oneline --decorate; git ls-remote package/package/package.git')
> + redirect_stderr = ''
> + # Using 'make PACKAGE_VERSION=' does not work for references with slash, so write to the .mk
> + self.run_commands_or_fail(
> + ['sed -e \'s#^PACKAGE_VERSION.*$#PACKAGE_VERSION = %s#g\' -i package/package/package.mk' % version])
> + result = os.system(
> + 'make %s package-dirclean package-source >stdout %s' % (extra_options, redirect_stderr))
> + if self.verbose:
> + os.system('echo -; cat stdout; echo -')
> + return result
> +
> + def check_tarball_exists(self, expected_download):
> + if self.verbose:
> + os.system('echo -n "dl: " ; ls package/package/dl/')
> + return os.path.isfile('package/package/dl/package-' + expected_download + '.tar.gz')
It would be good to also check the contents of the tarball. That way the
submodule stuff is immediately covered as well.
> +
> + def get_sha1_from_checkout(self):
> + pipe = subprocess.Popen("git rev-parse HEAD", shell=True, stdout=subprocess.PIPE)
> + sha1 = pipe.communicate()[0].strip()
> + if len(sha1) != 40:
> + raise OSError("Failed to create commit, got sha1 '%s'" % sha1)
> + return sha1
> +
> + def check_sha1_appeared_in_build_log(self, sha1): # it needs some instrumentation in the code
> + self.run_commands_or_fail(
> + ['grep -q "Checked out \'%s\'" stdout' % sha1])
Instead of using this stdout file, just use subprocess.check_output, save the
output somewhere, and use it in the asserts. And obviously instead of grep use
re.match or str.find.
Regards,
Arnout
> +
> + def check_in_the_build_log_if_a_full_clone_or_fetch_was_needed(self): # it is very coupled to the source code
> + return 0 == os.system('grep -q "Doing full" stdout')
> +
> + def detach_head_in_test_repo(self):
> + base_dir = os.path.realpath('.')
> + os.chdir('package/package/package.git')
> + self.run_commands_or_fail(
> + ['git checkout -q -f HEAD~0'])
> + os.chdir(base_dir)
> +
> + def checkout_branch_in_test_repo(self, branch):
> + base_dir = os.path.realpath('.')
> + os.chdir('package/package/package.git')
> + self.run_commands_or_fail(
> + ['git checkout -q -f "%s"' % branch])
> + os.chdir(base_dir)
> +
> + def create_a_submodule(self, submodule):
> + base_dir = os.path.realpath('.')
> + submodule_dir = 'package/package/%s.git' % submodule
> + self.run_commands_or_fail(
> + ['git init -q %s' % submodule_dir])
> + os.chdir(submodule_dir)
> + self.run_commands_or_fail(
> + ['echo %s > %s' % (submodule, submodule),
> + 'git add %s' % submodule,
> + 'git commit -q -m %s' % submodule])
> + os.chdir(base_dir)
> + os.chdir('package/package/package.git')
> + self.run_commands_or_fail(
> + ['git submodule add %s/%s.git ./%s 2>/dev/null' % (self.__class__.remote_url_of_local_repo, submodule, submodule),
> + 'echo %i > %i' % (self.commit_data, self.commit_data),
> + 'git add %i' % self.commit_data,
> + 'git commit -q -m %i' % self.commit_data])
> + self.commit_data += 1
> + sha1 = self.get_sha1_from_checkout()
> + os.chdir(base_dir)
> + return sha1
> +
> + def check_tarball_contains_the_submodule(self, expected_download, submodule):
> + if self.verbose:
> + os.system('echo "tar:"; tar -tf "package/package/dl/package-%s.tar.gz"' % expected_download)
> + return 0 == os.system('tar -tf "package/package/dl/package-%s.tar.gz" | grep -q "%s/%s"' % (expected_download, submodule, submodule))
>
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 2/3] test/support/download/git: new test
2016-11-06 0:19 ` Arnout Vandecappelle
@ 2016-11-06 0:51 ` Arnout Vandecappelle
2016-12-02 2:34 ` Ricardo Martincoski
1 sibling, 0 replies; 108+ messages in thread
From: Arnout Vandecappelle @ 2016-11-06 0:51 UTC (permalink / raw)
To: buildroot
On 06-11-16 01:19, Arnout Vandecappelle wrote:
> Second, instead of all these given's etc., it is much simpler to set up a
> single test repo, once, that has all the features you need. Setting it up could
> either be through explicit git commands, but I think it's even simpler if you
> just create a bare repository and commit that into Buildroot. It's not very nice
> for review, but I think it simplifies things enough to make it worth it. And
> anyway, the contents of the objects are not so important for review. If you do
> this, remove the hooks and info directories to make it smaller, and make sure
> the objects directory is binary. You know, I'll send a patch to show you how.
So I've taken another look at the tests you created, and it looks like it is
indeed a lot simpler and more effective to create the repo on the fly rather
than having a repo checked in into Buildroot. So forget this remark.
Regards,
Arnout
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 3/3] support/download/git: do not use git clone
2016-11-01 19:33 ` [Buildroot] [PATCH 3/3] support/download/git: do not use git clone Ricardo Martincoski
@ 2016-11-06 1:19 ` Arnout Vandecappelle
2016-11-10 0:07 ` Ricardo Martincoski
2016-11-18 7:33 ` Ricardo Martincoski
0 siblings, 2 replies; 108+ messages in thread
From: Arnout Vandecappelle @ 2016-11-06 1:19 UTC (permalink / raw)
To: buildroot
On 01-11-16 20:33, Ricardo Martincoski wrote:
> The logic of the script was completely rewritten based in some ideas
> discussed during the review of [1].
>
> Always using git init + git fetch has these advantages:
> - git fetch works with all kinds of refs, while git clone supports only
> branches and tags without the ref/heads/ and ref/tags/ prefixes;
> - a shallow fetch can be done for the head of those refs.
>
> The remote is first asked for its references using git ls-remote saving
> the output to a file.
> This file is later on compared to the desired change set, determining if
> it is a branch, tag, special ref or sha1.
> A concern that arrives from this method is that the remote can change
> between the git ls-remote and the git fetch, but in this case, once the
> script creates a shallow fetch, the checkout does what it is expected:
> - for a named reference (branch, tag or special ref), the fetch and
> checkout are successful using the "new" sha1 from the remote;
> - for a sha1 of a branch, the fetch fails (because the sha1 it is not
> anymore at branch head), falling back to a successful full fetch;
> - for a sha1 of a commit pointed by a tag (that should not be changed
> but it can be changed) the same behavior of a sha1 of a branch takes
> place;
> - for a sha1 only accessible by a special refs, the checkout fails,
> falling back to an unsuccessful checkout after full fetch. This can be
> caused only be a error from a developer.
>
> An analytical solution is used instead of a single awk line. It makes
> each line of code simple: grep to check the entry is in the ls-remote
> output and awk to actually get the reference to use.
Excellent commit message!
>
> [1] http://patchwork.ozlabs.org/patch/681841/
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
I have a bunch of remarks below, but nothing that should stop the acceptance of
this patch. So:
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This is for next, obviously.
> ---
> support/download/git | 91 ++++++++++++++++++++++++++++++++++------------------
> 1 file changed, 60 insertions(+), 31 deletions(-)
Note that, although lines are added here, it really is a simplification and
explicitation of the logic.
>
> diff --git a/support/download/git b/support/download/git
> index f7eef15..1411d94 100755
> --- a/support/download/git
> +++ b/support/download/git
> @@ -39,43 +39,72 @@ _git() {
> eval ${GIT} "${@}"
> }
>
> -# Try a shallow clone, since it is faster than a full clone - but that only
> -# works if the version is a ref (tag or branch). Before trying to do a shallow
> -# clone we check if ${cset} is in the list provided by git ls-remote. If not
> -# we fall back on a full clone.
> -#
> -# Messages for the type of clone used are provided to ease debugging in case of
> -# problems
> -git_done=0
> -if [ -n "$(_git ls-remote "'${repo}'" "'${cset}'" 2>&1)" ]; then
> - printf "Doing shallow clone\n"
> - if _git clone ${verbose} "${@}" --depth 1 -b "'${cset}'" "'${repo}'" "'${basename}'"; then
> - git_done=1
> - else
> - printf "Shallow clone failed, falling back to doing a full clone\n"
> +_git init ${verbose} "'${basename}'"
> +
> +pushd "${basename}" >/dev/null
> +
> +# Save the temporary file inside the .git directory that will be deleted after the checkout is done.
Don't forget to wrap at 80 columns.
> +tmpf=".git/ls-remote"
> +_git ls-remote "'${repo}'" > "${tmpf}"
> +
> +do_a_shallow_fetch=0
> +if grep "\<\(\|\(\|refs/\)\(heads\|tags\)/\)${cset}$" "${tmpf}" >/dev/null 2>&1; then
This is where grep -E becomes useful :-)
> + printf "The desired version is a named reference\n"
> + # Support branches and tags in the simplified form.
> + # Support branches and tags and special refs in the complete form refs/heads/branch.
> + # When the name is ambiguous, the branch will be selected (by git fetch or git clone).
> + ref="${cset}"
> + do_a_shallow_fetch=1
> +elif grep "^${cset}" "${tmpf}" >/dev/null 2>&1; then
> + printf "The desired version is a sha1\n"
> + if [ ${#cset} -lt 40 -a "1" != "$(awk "/^${cset}/{print \$1|\"sort -u | wc -l\"}" "${tmpf}")" ]; then
I think the check for -lt 40 is redundant.
I think all the logic here could be made a bit simpler by saving to a temporary
file again:
elif grep "^${cset}" "${tmpf}" > .git/matching_refs 2>/dev/null; then
printf ...
if "$(cut -f 1 .git/matching_refs | sort -u | wc -l) != 2; then
...
> + printf "Ambiguous partial sha1\n"
> + awk "/^${cset}/{print \$1|\"sort -u | wc -l\"}" "${tmpf}"
Here we should print the full lines, so without sort -u and wc. Sort is still
useful though.
> + exit 1
> + fi
> + # Accept full or unambiguous partial sha1. A sha1 can be referenced by many names.
> + # Prefer sha1 of commit pointed by a tag or sha1 of the tag itself,
> + # then sha1 pointed by any ref/*, and only then sha1 pointed by *HEAD.
I don't understand why this bit is needed. Any of the refs will do, right? It's
true that there is a risk that the ref gets updated between the ls-remote and
the fetch, and that for tags the chance that it gets updated is smaller than for
e.g. HEAD. But still, this is very much a corner case, and if it _does_ happen,
we still fall back to full clone. So the only important thing is to avoid HEAD.
Since ls-remotes already seems to sort the refs, which typically puts tags near
the end, just take the last one and we're done:
ref="$(cut -f 2 .git/matching_refs | tail -1)"
> + ref="$(awk -F'[\t^]' "/^${cset}.*\trefs\/tags\//{ print \$2; exit }" "${tmpf}")"
> + if [ -z "${ref}" ]; then
> + ref="$(awk -F'[\t^]' "/^${cset}.*\trefs\//{ print \$2; exit }" "${tmpf}")"
> + fi
> + if [ -z "${ref}" ]; then
> + # sha1 is referenced by HEAD
> + ref="$(awk -F'[\t^]' "/^${cset}/{ print \$2; exit }" "${tmpf}")"
> + fi
> + if [ -n "${ref}" ]; then
> + printf "Fetching '%s' to get '%s'\n" "${ref}" "${cset}"
> + do_a_shallow_fetch=1
> fi
> -fi
> -if [ ${git_done} -eq 0 ]; then
> - printf "Doing full clone\n"
> - _git clone ${verbose} "${@}" "'${repo}'" "'${basename}'"
> fi
>
> -pushd "${basename}" >/dev/null
> +git_fetch_done=0
> +if [ ${do_a_shallow_fetch} -eq 1 ]; then
Actually, if [ "$ref" ] would be sufficient.
> + printf "Doing shallow fetch\n"
> + if _git fetch ${verbose} "${@}" --depth 1 "'${repo}'" "'${ref}:refs/buildroot/${cset}'" 2>&1; then
> + git_fetch_done=1
> + else
> + # It catches the case the remote changed after ls-remote.
> + printf "Shallow fetch failed, falling back to doing a full fetch\n"
> + fi
> +fi
>
> -# Try to get the special refs exposed by some forges (pull-requests for
> -# github, changes for gerrit...). There is no easy way to know whether
> -# the cset the user passed us is such a special ref or a tag or a sha1
> -# or whatever else. We'll eventually fail at checking out that cset,
> -# below, if there is an issue anyway. Since most of the cset we're gonna
> -# have to clone are not such special refs, consign the output to oblivion
> -# so as not to alarm unsuspecting users, but still trace it as a warning.
Yann will be happy that this gets removed :-)
> -if ! _git fetch origin "'${cset}:${cset}'" >/dev/null 2>&1; then
> - printf "Could not fetch special ref '%s'; assuming it is not special.\n" "${cset}"
> +git_checkout_done=0
> +if [ ${git_fetch_done} -eq 1 ]; then
> + if _git checkout -q "'refs/buildroot/${cset}'" 2>&1; then
I'm not sure if I like this split. git_fetch_done gets set in only one place,
so we could just do the checkout directly there. That would also make it more
explicit that this is a checkout specifically for shallow fetches, while the
checkout below is specifically for full fetches.
> + git_checkout_done=1
> + else
> + printf "Checkout failed, falling back to doing a full fetch\n"
> + fi
> fi
>
> -# Checkout the required changeset, so that we can update the required
> -# submodules.
> -_git checkout -q "'${cset}'"
> +if [ ${git_checkout_done} -eq 0 ]; then
> + printf "Doing full fetch\n"
> + _git remote add origin "'${repo}'"
> + _git fetch ${verbose} "${@}" origin
Why do we need to create a remote here?
> + _git checkout -q "'${cset}'"
> +fi
At a later stage, we could update the fetch with --submodules=yes/no depending
on -r option.
Another interesting feature would be to first try a shallow fetch of all tags
and branches with a depth of, say, 100. That is still way less than a full clone
for large repositories, and has a good chance to still capture a sha1 that is
not at a branch or tag head. So it would provide a nice middle ground for the
cases for which we fail to do a shallow clone now. And if it does fail, nothing
is wasted because the full clone can start from the part that was already
downloaded.
Regards,
Arnout
>
> # Get date of commit to generate a reproducible archive.
> # %cD is RFC2822, so it's fully qualified, with TZ and all.
>
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 2/3] test/support/download/git: new test
2016-11-01 19:33 ` [Buildroot] [PATCH 2/3] test/support/download/git: new test Ricardo Martincoski
2016-11-01 21:27 ` Ricardo Martincoski
2016-11-06 0:19 ` Arnout Vandecappelle
@ 2016-11-06 1:24 ` Arnout Vandecappelle
2016-11-06 12:13 ` Henrique Marks
2017-08-26 22:20 ` [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build() Ricardo Martincoski
3 siblings, 1 reply; 108+ messages in thread
From: Arnout Vandecappelle @ 2016-11-06 1:24 UTC (permalink / raw)
To: buildroot
On 01-11-16 20:33, Ricardo Martincoski wrote:
> This test assumes:
> - it is run from the buildroot base dir;
> - there is a ssh server installed;
> - there is a key pair to allow the current user to ssh to localhost
> without being prompted for a password;
> - there is no ./.config ./stdout ./package/package;
>
> For each testcase, it creates a new git repo, and then download from
> this repo using a fake package named 'package'.
I've given comments on this patch which I think are important, so I've marked
it as Changes Requested in patchwork. However, patch 3/3 can be committed
without this one.
Regards,
Arnout
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> ---
> This test needs some instrumentation in the source code (previous patch)
> to work properly.
[snip]
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 2/3] test/support/download/git: new test
2016-11-06 1:24 ` Arnout Vandecappelle
@ 2016-11-06 12:13 ` Henrique Marks
2016-11-06 15:34 ` Arnout Vandecappelle
0 siblings, 1 reply; 108+ messages in thread
From: Henrique Marks @ 2016-11-06 12:13 UTC (permalink / raw)
To: buildroot
Hello All
----- Mensagem original -----
> De: "Arnout Vandecappelle" <arnout@mind.be>
> Para: "ricardo martincoski" <ricardo.martincoski@datacom.ind.br>, buildroot at buildroot.org
> Cc: "Yann E . MORIN" <yann.morin.1998@free.fr>
> Enviadas: S?bado, 5 de novembro de 2016 23:24:14
> Assunto: Re: [Buildroot] [PATCH 2/3] test/support/download/git: new test
> On 01-11-16 20:33, Ricardo Martincoski wrote:
>> This test assumes:
>> - it is run from the buildroot base dir;
>> - there is a ssh server installed;
>> - there is a key pair to allow the current user to ssh to localhost
>> without being prompted for a password;
>> - there is no ./.config ./stdout ./package/package;
>>
>> For each testcase, it creates a new git repo, and then download from
>> this repo using a fake package named 'package'.
>
> I've given comments on this patch which I think are important, so I've marked
> it as Changes Requested in patchwork. However, patch 3/3 can be committed
> without this one.
>
> Regards,
> Arnout
>
>>
Regarding the test comments, i think that part of the proposal is to show the tests written in Given-When-Then format without using nothing more than python and unittest class. Using GWT, it becomes easier to separate the pre-conditions, the operations to be tested, and the results. I think GWT exposes this nicely.
But, i must say that we (at Datacom) accepts both formats to tests, because there are cases where a tabular format is easier, and using GWT is just cumbersome. For instance, testing (ssh, sftp, https) connections is tabular, there is no preparation, and the operation and result comes together.
Another advantage of GWT here is to serve as an example. As buildroot is discussing test infra-structure, the GWT git tests are useful as an example on how to do it with GWT, python and unittest.
Thanks all.
--
Dr. Henrique Marks
henrique.marks at datacom.ind.br
R. Am?rica, 1000 - Eldorado do Sul - RS
CEP: 92990-000 - Brasil
Fone: +55 51 3933 3000 - Ramal 3466
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 2/3] test/support/download/git: new test
2016-11-06 12:13 ` Henrique Marks
@ 2016-11-06 15:34 ` Arnout Vandecappelle
0 siblings, 0 replies; 108+ messages in thread
From: Arnout Vandecappelle @ 2016-11-06 15:34 UTC (permalink / raw)
To: buildroot
On 06-11-16 13:13, Henrique Marks wrote:
> Regarding the test comments, i think that part of the proposal is to show the
> tests written in Given-When-Then format without using nothing more than
> python and unittest class. Using GWT, it becomes easier to separate the
> pre-conditions, the operations to be tested, and the results. I think GWT
> exposes this nicely.
>
> But, i must say that we (at Datacom) accepts both formats to tests, because
> there are cases where a tabular format is easier, and using GWT is just
> cumbersome. For instance, testing (ssh, sftp, https) connections is tabular,
> there is no preparation, and the operation and result comes together.
>
> Another advantage of GWT here is to serve as an example. As buildroot is
> discussing test infra-structure, the GWT git tests are useful as an example
> on how to do it with GWT, python and unittest.
Yes, it's good to have such an example so other Buildroot developers can get a
feel of it. But I suspect that most Buildroot developers will not find it
attractive. Maybe I shouldn't speak for others however :-)
In my experience, the GWT format works well for integration tests because they
are completely black-box, and because the Given and When often hide complicated
setups and stubs. For unit tests, however, I don't see them working that well
because they tend to be more gray box than black box, and they often involve
sequential tests (do this, expect that, then do this, expect that). Doing a
when-then-when-then that relies on the sequencing is kind of breaking the concept.
BTW, why do you distinguish GWT from tabular format? I've always used them
combined, like:
Given
Given
----------------------------
+ Foo + Bar + Expect +
----------------------------
+ When + When + Then +
+ When + When + Then +
+ When + When + Then +
+ When + When + Then +
----------------------------
Regards,
Arnout
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 1/3] support/download/git: log checked out sha1
2016-11-05 21:50 ` [Buildroot] [PATCH 1/3] support/download/git: log checked out sha1 Arnout Vandecappelle
@ 2016-11-06 23:17 ` Ricardo Martincoski
0 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2016-11-06 23:17 UTC (permalink / raw)
To: buildroot
Arnout,
Thank you for reviewing this series.
On Sat, Nov 05, 2016 at 07:50 PM, Arnout Vandecappelle wrote:
[snip]
>> +if [ -z ${verbose} ]; then
>
> It's quite weird to check that verbose is empty, but that's how the variable is
> defined... Perhaps s/verbose/quiet/ would be a good idea...
OK, makes sense. I can send a followup patch.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 3/3] support/download/git: do not use git clone
2016-11-06 1:19 ` Arnout Vandecappelle
@ 2016-11-10 0:07 ` Ricardo Martincoski
2016-11-18 7:33 ` Ricardo Martincoski
1 sibling, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2016-11-10 0:07 UTC (permalink / raw)
To: buildroot
Arnout,
Sorry the delay.
On Sat, Nov 05, 2016 at 11:19 PM, Arnout Vandecappelle wrote:
[snip]
Your suggestions will simplify the script.
I will answer all questions in each review and resend the series in the next
days. So I marked 1/3 and 3/3 as Changes Requested.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 3/3] support/download/git: do not use git clone
2016-11-06 1:19 ` Arnout Vandecappelle
2016-11-10 0:07 ` Ricardo Martincoski
@ 2016-11-18 7:33 ` Ricardo Martincoski
1 sibling, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2016-11-18 7:33 UTC (permalink / raw)
To: buildroot
Arnout,
On Sat, Nov 05, 2016 at 11:19 PM, Arnout Vandecappelle wrote:
[snip]
> Excellent commit message!
Thanks.
[snip]
> I have a bunch of remarks below, but nothing that should stop the acceptance of
> this patch. So:
>
> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
I won't carry these for v2 because there will be a bunch of improvements based
on your review ;)
>
> This is for next, obviously.
>
[snip]
>> +# Save the temporary file inside the .git directory that will be deleted after the checkout is done.
>
> Don't forget to wrap at 80 columns.
Sorry. This line is way too long.
But just to be clear:
I think you mean to wrap to 80 as a rule of thumb (e.g. comments are easy to do)
and not as a hard requirement.
Right?
I ask this because I think breaking some lines with commands would make it
harder to read, e.g.
if _git fetch ${verbose} "${@}" --depth 1 "'${repo}'" "'${ref}:refs/buildroot/${cset}'" 2>&1; then
[snip]
>> +if grep "\<\(\|\(\|refs/\)\(heads\|tags\)/\)${cset}$" "${tmpf}" >/dev/null 2>&1; then
>
> This is where grep -E becomes useful :-)
Thanks. Much nicer:
if grep -E "\<(|(|refs/)(heads|tags)/)${cset}$" "${tmpf}" >/dev/null 2>&1; then
[snip]
>> +elif grep "^${cset}" "${tmpf}" >/dev/null 2>&1; then
>> + printf "The desired version is a sha1\n"
>> + if [ ${#cset} -lt 40 -a "1" != "$(awk "/^${cset}/{print \$1|\"sort -u | wc -l\"}" "${tmpf}")" ]; then
>
> I think the check for -lt 40 is redundant.
You are right.
But this line will be removed in v2 because it is not needed and not wanted.
It is not needed because unambiguous partial sha1 (of any commit) can be
checked out after the full fetch (but without optimized download, of course).
It is not wanted because, as I realized after your comment, the output of
ls-remote is a subset of all sha1s and can't be used to test a sha1 is not
ambiguous. We could end up checking out a wrong sha1 when we really need to let
git checkout fail.
BTW, an example of partial sha1 is in the current master, see commit 956fcc2.
>
> I think all the logic here could be made a bit simpler by saving to a temporary
> file again:
>
> elif grep "^${cset}" "${tmpf}" > .git/matching_refs 2>/dev/null; then
> printf ...
> if "$(cut -f 1 .git/matching_refs | sort -u | wc -l) != 2; then
> ...
Yes. Much simpler.
Actually this 'if' is removed in v2 (see above) but I will use temporary file
and cut instead of awk in v2 (see below).
>
>
>> + printf "Ambiguous partial sha1\n"
>> + awk "/^${cset}/{print \$1|\"sort -u | wc -l\"}" "${tmpf}"
>
> Here we should print the full lines, so without sort -u and wc. Sort is still
> useful though.
Indeed. But will be removed in v2.
If the partial sha1 is ambiguous, let git checkout fail. Git checkout fails even
when the sha1 of the commit is ambiguous to a sha1 of a tree.
BTW, the best real example I have so far between 2 refs is b3766 from
https://github.com/vim/vim.git .
>
>> + exit 1
>> + fi
>> + # Accept full or unambiguous partial sha1. A sha1 can be referenced by many names.
>> + # Prefer sha1 of commit pointed by a tag or sha1 of the tag itself,
>> + # then sha1 pointed by any ref/*, and only then sha1 pointed by *HEAD.
>
> I don't understand why this bit is needed. Any of the refs will do, right? It's
> true that there is a risk that the ref gets updated between the ls-remote and
> the fetch, and that for tags the chance that it gets updated is smaller than for
> e.g. HEAD. But still, this is very much a corner case, and if it _does_ happen,
> we still fall back to full clone. So the only important thing is to avoid HEAD.
> Since ls-remotes already seems to sort the refs, which typically puts tags near
> the end, just take the last one and we're done:
>
> ref="$(cut -f 2 .git/matching_refs | tail -1)"
Yes. It is simpler.
I just noticed I missed to explain the awk -F'[\t^]' in the comments (or in the
commit log) :/ .
Its is needed because annotated tags have their own sha1 and the commit pointed
by a tag has ^{} at the end of the ref name in the ls-remote output.
9395452b4aab7bc2475ef8935b4a4fb99d778d70 refs/tags/v4.8-rc6^{}
But after 'cut' I can use bash's 'Pattern substitution'
ref=${ref/%"^{}"}
and, of course, add a nice comment for it.
[snip]
>> +if [ ${do_a_shallow_fetch} -eq 1 ]; then
>
> Actually, if [ "$ref" ] would be sufficient.
OK.
[snip]
>> -if ! _git fetch origin "'${cset}:${cset}'" >/dev/null 2>&1; then
>> - printf "Could not fetch special ref '%s'; assuming it is not special.\n" "${cset}"
>> +git_checkout_done=0
>> +if [ ${git_fetch_done} -eq 1 ]; then
>> + if _git checkout -q "'refs/buildroot/${cset}'" 2>&1; then
>
> I'm not sure if I like this split. git_fetch_done gets set in only one place,
> so we could just do the checkout directly there. That would also make it more
> explicit that this is a checkout specifically for shallow fetches, while the
> checkout below is specifically for full fetches.
OK.
[snip]
>> + _git remote add origin "'${repo}'"
>> + _git fetch ${verbose} "${@}" origin
>
> Why do we need to create a remote here?
Actually, we don't.
I missed your comment in http://patchwork.ozlabs.org/patch/681841/
"just use $repo everywhere"
I will do this in v2.
I think I can do this too:
"Do it in two patches: first convert the current situation to fetches, then add
the shallow download of tag SHAs."
>
>> + _git checkout -q "'${cset}'"
>> +fi
>
> At a later stage, we could update the fetch with --submodules=yes/no depending
> on -r option.
In theory, OK. But because of a bug in git maybe it's better to keep the
current 'git submodule update --init --recursive'.
git fetch --help # git 2.10.2
BUGS
Using --recurse-submodules can only fetch new commits in already
checked out submodules right now. When e.g. upstream added a new
submodule in the just fetched commits of the superproject the submodule
itself can not be fetched, making it impossible to check out that
submodule later without having to do a fetch again. This is expected to
be fixed in a future Git version.
>
> Another interesting feature would be to first try a shallow fetch of all tags
> and branches with a depth of, say, 100. That is still way less than a full clone
> for large repositories, and has a good chance to still capture a sha1 that is
> not at a branch or tag head. So it would provide a nice middle ground for the
> cases for which we fail to do a shallow clone now. And if it does fail, nothing
> is wasted because the full clone can start from the part that was already
> downloaded.
Yes. It should be useful especially for hardware-specific kernel repos when
using sha1 as custom repo version.
The buildroot policy is to use preferably a tag, and if a tag cannot be used,
use the sha1 in a branch, right?
So I think we will have the best gain by fetching only from branches in this
case.
A quick test shows good results: make warp7_defconfig ; make linux-source
without this feature: 1.69 GiB
5148191
with this feature (100): 945.59 MiB + 73.89 KiB
1997387 + 138 = 1997525
with this feature (10+100): 402.04 MiB + 470.10 MiB
347225 + 32 + 1650245 = 1997502
(I didn't dig to understand why the sum is not exactly the same)
Notice there is some extra processing in the server side ('Compressing objects')
that takes some more seconds but I guess the bandwidth saved worth this.
I can add this patch (depth 100) at the end of the series as a RFC.
This way people can comment/test/argue a different depth to use.
1) without this feature
Doing full clone
Cloning into 'linux-efaf36531fe7b1fc15a48033e5972825c91f9fc6'...
remote: Counting objects: 5148191, done.
remote: Compressing objects: 100% (395/395), done.
remote: Total 5148191 (delta 377), reused 183 (delta 183), pack-reused 5147613
Receiving objects: 100% (5148191/5148191), 1.69 GiB | 1.73 MiB/s, done.
2) with this feature, only for branches, depth 100
Doing shallow fetch of all branches
remote: Counting objects: 1997387, done.
remote: Compressing objects: 100% (453860/453860), done.
remote: Total 1997387 (delta 1605759), reused 1902796 (delta 1529294), pack-reused 0
Receiving objects: 100% (1997387/1997387), 945.59 MiB | 1.74 MiB/s, done.
...
remote: Counting objects: 138, done.
remote: Compressing objects: 100% (138/138), done.
remote: Total 138 (delta 0), reused 138 (delta 0), pack-reused 0
Receiving objects: 100% (138/138), 73.89 KiB | 0 bytes/s, done.
3) with this feature, only for branches, 1st step depth 10, 2nd step depth 100
Doing shallow fetch of all branches (10)
remote: Counting objects: 347225, done.
remote: Compressing objects: 100% (177196/177196), done.
remote: Total 347225 (delta 239120), reused 251422 (delta 167173), pack-reused 0
Receiving objects: 100% (347225/347225), 402.04 MiB | 1.56 MiB/s, done.
...
remote: Counting objects: 32, done.
remote: Compressing objects: 100% (32/32), done.
remote: Total 32 (delta 0), reused 32 (delta 0), pack-reused 0
Unpacking objects: 100% (32/32), done.
...
fatal: reference is not a tree: efaf36531fe7b1fc15a48033e5972825c91f9fc6
Doing shallow fetch of all branches (100)
remote: Counting objects: 1650245, done.
remote: Compressing objects: 100% (459302/459302), done.
remote: Total 1650245 (delta 1364443), reused 1452442 (delta 1179566), pack-reused 0
Receiving objects: 100% (1650245/1650245), 470.10 MiB | 1.68 MiB/s, done.
Resolving deltas: 100% (1364443/1364443), completed with 38986 local objects.
Checked out 'efaf36531fe7b1fc15a48033e5972825c91f9fc6'.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 2/3] test/support/download/git: new test
2016-11-06 0:19 ` Arnout Vandecappelle
2016-11-06 0:51 ` Arnout Vandecappelle
@ 2016-12-02 2:34 ` Ricardo Martincoski
2016-12-05 17:26 ` Arnout Vandecappelle
1 sibling, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2016-12-02 2:34 UTC (permalink / raw)
To: buildroot
Arnout,
Right now I will resend this series without the test and later on I will send
two follow-up series adding tests already based on the test infra.
The first series will introduce changes and/or new module(s) to the test infra
to support the test of the git download script, and also 1 or 2 testcases;
The second series will introduce the rest of the testcases.
The advantages of this approach are:
- the tests I wrote works enough for me to reduce the load of manual tests if
other improvements are needed in the download script;
- avoid duplicate rework (as the test infra already solves some issues my test
has, especially some ugly code);
- avoid duplicate new work (as the test infra already brings features that I
need/want to implement, such as the out-of-tree build);
- avoid non-trivial rebase (it is not that complex too, but it takes time) as
the infra is currently functional but is right now under stylistic rework;
- the review of each patch/series should become easier.
I think it makes more sense to keep the test and the patch "log checked out
sha1" in the same series, so I won't resend that just yet.
On Sat, Nov 05, 2016 at 10:19 PM, Arnout Vandecappelle wrote:
[snip]
>> diff --git a/test/support/download/git b/test/support/download/git
>
> It would be nice if you could put this in a place that is convenient for
> integration with the overall test infrastructure that Thomas has been working
> on. However, I can't find the git repo with that test infrastructure anywhere...
> Thomas?
A working version is here:
http://git.free-electrons.com/users/thomas-petazzoni/buildroot/?h=runtime-tests
but it is being refactored before merge.
>
> I would call it testgit.py instead, so it can be run with 'python -m unittest
> discover -s test'.
Yes.
Probably will become support/testing/tests/download/test_git.py
[snip]
>> +# BEFORE calling this script, make sure:
>> +# - there is a ssh server installed
>> +# - there is a key pair generated that allow the current user to run 'ssh localhost' without being prompted for entering a password
>
> That's not very nice. It would be better to use a git daemon (and the git://
> instead of ssh:// protocol). You can start it with:
>
> git daemon --reuseaddr --verbose --listen=localhost --port={port}
OK. Probably it can be done about the same way the qemu is started by the test
infra.
>
> It will fail to start if the port is already in use, so you can try a different
> port if you find 'Address already in use' in the output. And it will output
> 'Ready to rumble' once it started, so you can use that to delay the test until
> the daemon is operational.
Something similar (delay the test) is done for qemu in the infra.
[snip]
>> +# TODO test another transport protocols: git, https, ... - need manual setup of git server
>
> That's not very useful, we're pretty much agnostic of the protocol used.
OK.
The only corner case I found so far regarding the transport protocol is when the
remote server does not support smart http/https. In this case any use of --depth
is refused (git clone, git fetch) and fallback to a full clone. This case can
easily be tested manually:
make defconfig ; make am33x-cm3-source
>
>> +# TODO test extra download options, especially --reference
>
> Er, that option isn't used anywhere at the moment...
Indeed. Sorry the wrong comment.
[snip]
>> +if __name__ == '__main__':
>> + suite = git_keywords.unittest.TestLoader().loadTestsFromTestCase(TestSuiteSupportDownloadGitRefs)
>
> It's not very nice to rely on git_keywords' import of unittest. Just import
> unittest here.
OK.
But in the test infra I don't need to write this code :)
>
> Any reason why the usual unittest.main() doesn't work?
I used this because verbosity=2 can only be passed for main() when using at
least python 2.7 and I wanted to keep compatibility to 2.6.
But in the test infra I don't need to write this code :)
[snip]
>> +class Keywords(unittest.TestCase, git_util.Utilities):
>
> This class looks like nothing more than a GWT wrapper around git_util. I don't
> think that that's very useful. Buildroot developers are generally developers,
> not test engineers who are not so used to programming, and I think that for most
> BR developers the GWT standard is more confusing than useful.
I will use the same testcase style as other tests in test infra.
>
[snip]
>> + def given_the_commits_are_not_the_same(self, sha1_1st, sha1_2nd):
>> + self.assertNotEqual(sha1_1st, sha1_2nd)
>
> This can't be right... An assertion shouldn't occur in a given clause, only in
> a then or maybe an unless clause.
Yes. Another error in my abstraction.
[snip]
>> + def when_i_download_the_version(self, version):
>> + result = self.download_source_using_buildroot(version)
>> + self.assertEqual(0, result, "download of version %s failed" % version)
>
[snip]
>
> Also, for this specific case, it's IMHO cleaner to raise an exception when the
> download fails, which unittest will interpret as a failure as well. Exceptions
> are something which are acceptable in a which clause I think. And when a
> download failure is expected, use assertRaises.
I will raise an exception.
[snip]
>> +class Utilities():
>> + commit_data = 10000
>
> As far as I can see, instances of this class are completely stateless, there is
> only the commit_data class attribute that gets updated (for reasons I don't
> understand BTW). Python is not Java, so it's perfectly OK to have module
> functions. In other words, remove this class and just define everything as
> functions of the git_util module.
When using the test infra I will use a module.
>
[snip]
>> + remote_url_of_local_repo = subprocess.Popen('echo ssh://localhost/$(readlink -f $(pwd))/package/package', shell=True, stdout=subprocess.PIPE).communicate()[0].strip()
>
> Er, are you trying to do
> remote_url_of_local_repo = 'ssh://localhost' + os.path.abspath('package/package')
> ?
Yes. Much better.
>
> BTW, wrap at around 80 columns when that makes sense.
OK.
[snip]
>> + verbose = True
>
> Perhaps use the logging module.
Won't be needed in the test infra, that generates a log for each stage (build,
run).
[snip]
>> + result = os.system(command)
>> + if result != 0:
>> + raise OSError("command '%s' failed with error %d" % (command, result / 256))
>
> Use subprocess.check_call instead. The loop is kind of useless, constructing
> the list or having multiple check_call's is more or less equivalent.
I will use the same as in the test infra, that uses subprocess.
[snip]
>> + def check_the_patch_instrumenting_the_source_code_is_present(cls):
>> + cls.run_commands_or_exit(
>> + ['grep -q "Checked out" support/download/git'])
>
> Just assume that that patch is present. We're not going to make tests that run
> in different version of Buildroot. If something changes in Buildroot, the tests
> should be updated along with it.
OK. Fair enough.
[snip]
>> + cls.run_commands_or_exit(
>> + ['mkdir package/package',
>
> Use os.makedirs instead.
OK.
I first started to write these tests in a bash script.
Some ugly things like this survived the port to python.
>
>> + 'echo "PACKAGE_VERSION = invalid" >> package/package/package.mk',
>> + 'echo "PACKAGE_SITE = %s/package.git" >> package/package/package.mk' % cls.remote_url_of_local_repo,
>> + 'echo "PACKAGE_SITE_METHOD = git" >> package/package/package.mk',
>> + 'echo \'$(eval $(generic-package))\' >> package/package/package.mk'])
>
> with open('package/package/package.mk', 'wt') as package_mk:
> package_mk.write('''\
> PACKAGE_VERSION = ...
> ...''' % remote_url_of_local_repo)
This will move to a committed br2-external in the test infra.
>
> Although actually, .format is preferred over % nowadays.
I will use the same style as the test infra after stylistic refactor.
>
> That said, I think what we're testing here is just the git download wrapper,
> not the entire download infrastructure. So I wouldn't create a package, but
> instead call support/download/dl-wrapper with the appropriate arguments. This is
> a unit test after all, not an integration test.
Of course it's possible to create a unit test.
But in this case I think the total effort (including maintenance) will be
smaller if we keep calling make ...-source.
The test infra is ready to do calls to the main Buildroot makefile.
There are some cases, e.g. package version with slash, that we would need to
mimic other scripts in the infra to test the wrapper by itself.
>
> If you do keep the package, then it shouldn't be created in the Buildroot tree,
> but it should be used as a BR2_EXTERNAL package. Also, instead of creating the
> files here, just add them to the buildroot tree and use them from the test, like
> I proposed for the testgit repository.
I will use a package in a br2-external.
[snip]
>> + 'echo \'BR2_DL_DIR="$(TOPDIR)/package/package/dl"\' >> .config',
>
> BR2_DL_DIR is overridden by the environment variable. I have BR2_DL_DIR in my
> environment, so all the tests fail for me :-) So just make sure it is exported.
> Or override it on the make commandline whenever you call make, that works as well.
OK.
>
>
>> + 'echo \'BR2_BACKUP_SITE=""\' >> .config',
>> + 'make oldconfig >/dev/null 2>/dev/null'])
>
> You shouldn't use the output directory, instead use a O= to some temporary
> directory.
That I get for free from the test infra.
[snip]
>> + ['rm -rf package/package/dl'])
>
> Use shutil.rmtree
I will point the download directory inside the builddir, so the infra will take
care of this for me. BTW the test infra uses shutil.rmtree.
>
>> +
>> + def run_commands_or_fail(self, commands):
>> + for command in commands:
>> + result = os.system(command)
>> + # FIXME ugly abstraction error, but it does the job
>> + self.assertEqual(0, result, "command %s failed with error %d" % (command, result / 256))
>
> The only difference with the other run_commands definition is that this one
> raises AssertionError instead of OSError. Not much of a difference... If the
> difference is important, use subprocess.check() instead.
OK.
[snip]
>> + def create_a_special_ref_in_test_repo(self, special_ref):
>> + base_dir = os.path.realpath('.')
>> + os.chdir('package/package/package.git')
>> + directory = '/'.join(special_ref.split('/')[:-1])
>> + self.run_commands_or_fail(
>> + ['git branch -q -f temporary',
>> + 'mkdir -p .git/refs/%s' % directory,
>> + 'mv .git/refs/heads/temporary .git/refs/%s' % special_ref])
>
> You should probably read the git update-ref man page.
Thanks, much cleaner solution.
[snip]
>> + def check_tarball_exists(self, expected_download):
>> + if self.verbose:
>> + os.system('echo -n "dl: " ; ls package/package/dl/')
>> + return os.path.isfile('package/package/dl/package-' + expected_download + '.tar.gz')
>
> It would be good to also check the contents of the tarball. That way the
> submodule stuff is immediately covered as well.
Indeed. This is the only one I am not sure how I will do in the test infra, but
should not be too hard. I will keep this suggestion in mind when rewriting the
tests.
[snip]
>> + def check_sha1_appeared_in_build_log(self, sha1): # it needs some instrumentation in the code
>> + self.run_commands_or_fail(
>> + ['grep -q "Checked out \'%s\'" stdout' % sha1])
>
> Instead of using this stdout file, just use subprocess.check_output, save the
> output somewhere, and use it in the asserts. And obviously instead of grep use
> re.match or str.find.
Will do.
That grep was another ugly code that survived the port from shell script.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 2/3] test/support/download/git: new test
2016-12-02 2:34 ` Ricardo Martincoski
@ 2016-12-05 17:26 ` Arnout Vandecappelle
2016-12-06 1:35 ` Ricardo Martincoski
0 siblings, 1 reply; 108+ messages in thread
From: Arnout Vandecappelle @ 2016-12-05 17:26 UTC (permalink / raw)
To: buildroot
On 02-12-16 03:34, Ricardo Martincoski wrote:
[snip]
>>> + def check_tarball_exists(self, expected_download):
>>> + if self.verbose:
>>> + os.system('echo -n "dl: " ; ls package/package/dl/')
>>> + return os.path.isfile('package/package/dl/package-' + expected_download + '.tar.gz')
>> It would be good to also check the contents of the tarball. That way the
>> submodule stuff is immediately covered as well.
> Indeed. This is the only one I am not sure how I will do in the test infra, but
> should not be too hard. I will keep this suggestion in mind when rewriting the
> tests.
Here's a hint:
import tarfile
with tarfile.open(expected_download) as tf:
files = tf.getnames()
...
Regards,
Arnout
[snip]
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH 2/3] test/support/download/git: new test
2016-12-05 17:26 ` Arnout Vandecappelle
@ 2016-12-06 1:35 ` Ricardo Martincoski
0 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2016-12-06 1:35 UTC (permalink / raw)
To: buildroot
Arnout,
On Mon, Dec 05, 2016 at 03:26 PM, Arnout Vandecappelle wrote:
> On 02-12-16 03:34, Ricardo Martincoski wrote:
> [snip]
>>>> + def check_tarball_exists(self, expected_download):
>>>> + if self.verbose:
>>>> + os.system('echo -n "dl: " ; ls package/package/dl/')
>>>> + return os.path.isfile('package/package/dl/package-' + expected_download + '.tar.gz')
>>> It would be good to also check the contents of the tarball. That way the
>>> submodule stuff is immediately covered as well.
>> Indeed. This is the only one I am not sure how I will do in the test infra, but
>> should not be too hard. I will keep this suggestion in mind when rewriting the
>> tests.
>
> Here's a hint:
>
> import tarfile
> with tarfile.open(expected_download) as tf:
> files = tf.getnames()
> ...
Thank you very much.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build()
2016-11-01 19:33 ` [Buildroot] [PATCH 2/3] test/support/download/git: new test Ricardo Martincoski
` (2 preceding siblings ...)
2016-11-06 1:24 ` Arnout Vandecappelle
@ 2017-08-26 22:20 ` Ricardo Martincoski
2017-08-26 22:20 ` [Buildroot] [next v2 2/7] testing/infra/builder: build with target and environment Ricardo Martincoski
` (7 more replies)
3 siblings, 8 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-08-26 22:20 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Some test cases don't use a full build as setup, so split the build()
method into configure() and build().
It allows a test case to perform configuration at the setup stage and
the build inside the test itself.
Call this new method just before build in the BRTest base class, to keep
the current behavior for existing test cases.
This change will be needed when adding a common class to test the git
download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
---
Changes v1 -> v2:
- new patch to adapt the test infra to test git download
---
support/testing/infra/basetest.py | 1 +
support/testing/infra/builder.py | 3 ++-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
index 431605b23f..d61f4d337b 100644
--- a/support/testing/infra/basetest.py
+++ b/support/testing/infra/basetest.py
@@ -57,6 +57,7 @@ class BRTest(unittest.TestCase):
if not self.b.is_finished():
self.show_msg("Building")
+ self.b.configure()
self.b.build()
self.show_msg("Building done")
diff --git a/support/testing/infra/builder.py b/support/testing/infra/builder.py
index 905b127c91..e7c8e0102c 100644
--- a/support/testing/infra/builder.py
+++ b/support/testing/infra/builder.py
@@ -10,7 +10,7 @@ class Builder(object):
self.builddir = builddir
self.logfile = infra.open_log_file(builddir, "build", logtofile)
- def build(self):
+ def configure(self):
if not os.path.isdir(self.builddir):
os.makedirs(self.builddir)
@@ -29,6 +29,7 @@ class Builder(object):
if ret != 0:
raise SystemError("Cannot olddefconfig")
+ def build(self):
cmd = ["make", "-C", self.builddir]
ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile)
if ret != 0:
--
2.13.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 2/7] testing/infra/builder: build with target and environment
2017-08-26 22:20 ` [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build() Ricardo Martincoski
@ 2017-08-26 22:20 ` Ricardo Martincoski
2017-10-06 20:42 ` Arnout Vandecappelle
2017-08-26 22:20 ` [Buildroot] [next v2 3/7] testing/infra/builder: allow to override logfile Ricardo Martincoski
` (6 subsequent siblings)
7 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2017-08-26 22:20 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Make the builder able to call 'VAR1=1 make VAR2=2 target'.
Allow to send extra parameters to be added to the end of make command
line. It can be used for these purposes:
- to configure a br2-external, by passing 'BR2_EXTERNAL="dir"';
- to specify a make target, such as 'foo-source'.
Allow to add variables to the environment in which make runs. It can be
used for these purposes:
- to override values from environment, such as 'BR2_DL_DIR="dl"';
- to set variables not set in the .mk files (for testing purpose), as
FOO_VERSION='1/1', triggering the preprocessing of values.
This change will be needed when adding a common class to test the git
download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
---
Changes v1 -> v2:
- new patch to adapt the test infra to test git download
---
support/testing/infra/builder.py | 37 +++++++++++++++++++++++++++++++------
1 file changed, 31 insertions(+), 6 deletions(-)
diff --git a/support/testing/infra/builder.py b/support/testing/infra/builder.py
index e7c8e0102c..de7c141a76 100644
--- a/support/testing/infra/builder.py
+++ b/support/testing/infra/builder.py
@@ -10,7 +10,12 @@ class Builder(object):
self.builddir = builddir
self.logfile = infra.open_log_file(builddir, "build", logtofile)
- def configure(self):
+ # Configure the build
+ #
+ # makecmdline: a list of arguments to be passed to the make command.
+ # e.g. makecmdline=["BR2_EXTERNAL=/path"]
+ #
+ def configure(self, makecmdline=None):
if not os.path.isdir(self.builddir):
os.makedirs(self.builddir)
@@ -22,16 +27,36 @@ class Builder(object):
"> end defconfig\n")
self.logfile.flush()
- cmd = ["make",
- "O={}".format(self.builddir),
- "olddefconfig"]
+ cmd = ["make", "O={}".format(self.builddir)]
+ if makecmdline:
+ cmd += makecmdline
+ cmd += ["olddefconfig"]
+
ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile)
if ret != 0:
raise SystemError("Cannot olddefconfig")
- def build(self):
+ # Perform the build
+ #
+ # makecmdline: a list of arguments to be passed to the make command. It can
+ # include a make target.
+ # e.g. makecmdline=["foo-source"]
+ #
+ # env: a dict of variables to be appended (or replaced) in the environment
+ # that calls make.
+ # e.g. env={"BR2_DL_DIR": "/path"}
+ #
+ def build(self, makecmdline=None, env=None):
+ buildenv = os.environ.copy()
+ if env:
+ buildenv.update(env)
+
cmd = ["make", "-C", self.builddir]
- ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile)
+ if makecmdline:
+ cmd += makecmdline
+
+ ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
+ env=buildenv)
if ret != 0:
raise SystemError("Build failed")
--
2.13.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 3/7] testing/infra/builder: allow to override logfile
2017-08-26 22:20 ` [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build() Ricardo Martincoski
2017-08-26 22:20 ` [Buildroot] [next v2 2/7] testing/infra/builder: build with target and environment Ricardo Martincoski
@ 2017-08-26 22:20 ` Ricardo Martincoski
2017-10-06 20:50 ` Arnout Vandecappelle
2017-08-26 22:20 ` [Buildroot] [next v2 4/7] testing/tests/download: add infra for git tests Ricardo Martincoski
` (5 subsequent siblings)
7 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2017-08-26 22:20 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Some test cases can use only the configure step as setup, while the
build is part of the test itself.
Allow the caller to build() to override the logfile used. This way all
the log for the actual test can be directed to *-run.log.
This change will be needed to make nice logs for git downloader test
cases.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
---
Changes v1 -> v2:
- new patch to adapt the test infra to test git download
---
support/testing/infra/builder.py | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/support/testing/infra/builder.py b/support/testing/infra/builder.py
index de7c141a76..d634c211ac 100644
--- a/support/testing/infra/builder.py
+++ b/support/testing/infra/builder.py
@@ -46,7 +46,13 @@ class Builder(object):
# that calls make.
# e.g. env={"BR2_DL_DIR": "/path"}
#
- def build(self, makecmdline=None, env=None):
+ # ovrdlogfile: when the build is part of the actual test case instead of
+ # the setup, this parameter can be used to override the logfile used.
+ # e.g. ovrdlogfile=handler_for_the_run_log
+ #
+ def build(self, makecmdline=None, env=None, ovrdlogfile=None):
+ logfile = ovrdlogfile or self.logfile
+
buildenv = os.environ.copy()
if env:
buildenv.update(env)
@@ -55,7 +61,7 @@ class Builder(object):
if makecmdline:
cmd += makecmdline
- ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
+ ret = subprocess.call(cmd, stdout=logfile, stderr=logfile,
env=buildenv)
if ret != 0:
raise SystemError("Build failed")
--
2.13.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 4/7] testing/tests/download: add infra for git tests
2017-08-26 22:20 ` [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build() Ricardo Martincoski
2017-08-26 22:20 ` [Buildroot] [next v2 2/7] testing/infra/builder: build with target and environment Ricardo Martincoski
2017-08-26 22:20 ` [Buildroot] [next v2 3/7] testing/infra/builder: allow to override logfile Ricardo Martincoski
@ 2017-08-26 22:20 ` Ricardo Martincoski
2017-10-06 21:30 ` Arnout Vandecappelle
2017-08-26 22:20 ` [Buildroot] [next v2 5/7] testing/tests/download: add git hash tests Ricardo Martincoski
` (4 subsequent siblings)
7 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2017-08-26 22:20 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Add GitRemote class, that can start a git server in the host machine to
emulate a remote git server under the control of the test.
Add a new base class, called GitTestBase, that inherits from BRTest and
must be subclassed by all git test cases.
Its setUp() method takes care of configuring the build with a
br2-external, avoiding to hit http://sources.buildroot.net by using an
empty BR2_BACKUP_SITE. It also avoids downloading not pre-installed
dependencies (i.e. lzip) every time by calling 'make dependencies' using
the common dl directory, and it instantiates the GitRemote object.
Its tearDown() method cleans things up similarly to the one from BRTest.
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
---
Changes v1 -> v2:
- use git daemon + git:// instead of ssh (Arnout);
- remove __main__ handling since the test infra already does that;
- use the logging support from test infra;
- this patch is part of series 1/3 of a new version of
http://patchwork.ozlabs.org/patch/690097/
---
support/testing/tests/download/__init__.py | 0
support/testing/tests/download/gitbase.py | 41 +++++++++++++++++++++++
support/testing/tests/download/gitremote.py | 50 +++++++++++++++++++++++++++++
3 files changed, 91 insertions(+)
create mode 100644 support/testing/tests/download/__init__.py
create mode 100644 support/testing/tests/download/gitbase.py
create mode 100644 support/testing/tests/download/gitremote.py
diff --git a/support/testing/tests/download/__init__.py b/support/testing/tests/download/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/support/testing/tests/download/gitbase.py b/support/testing/tests/download/gitbase.py
new file mode 100644
index 0000000000..b259914c77
--- /dev/null
+++ b/support/testing/tests/download/gitbase.py
@@ -0,0 +1,41 @@
+import infra.basetest
+from infra.builder import Builder
+from gitremote import GitRemote
+
+
+class GitTestBase(infra.basetest.BRTest):
+ config = \
+ """
+ BR2_BACKUP_SITE=""
+ """
+ br2_external = None
+ serveddir = None
+ gitremote = None
+ logfile = None
+
+ def setUp(self):
+ self.show_msg("Starting")
+ self.b = Builder(self.config, self.builddir, self.logtofile)
+
+ if not self.keepbuilds:
+ self.b.delete()
+
+ if not self.b.is_finished():
+ self.show_msg("Configuring")
+ self.b.configure(["BR2_EXTERNAL={}".format(self.br2_external)])
+ # do not download not pre-installed dependencies (i.e. lzip) every
+ # time a test runs
+ self.b.build(["dependencies"])
+
+ self.gitremote = GitRemote(self.builddir, self.serveddir,
+ self.logtofile)
+ # send output from the test to the logfile created by GitRemote
+ self.logfile = self.gitremote.logfile
+ self.gitremote.start()
+
+ def tearDown(self):
+ self.show_msg("Cleaning up")
+ if self.gitremote:
+ self.gitremote.stop()
+ if self.b and not self.keepbuilds:
+ self.b.delete()
diff --git a/support/testing/tests/download/gitremote.py b/support/testing/tests/download/gitremote.py
new file mode 100644
index 0000000000..84d1036889
--- /dev/null
+++ b/support/testing/tests/download/gitremote.py
@@ -0,0 +1,50 @@
+# subprocess does not kill the child daemon when a test case fails by raising
+# an exception. So use pexpect instead.
+import pexpect
+
+import infra
+
+GIT_REMOTE_PORT_INITIAL = 9418
+GIT_REMOTE_PORT_LAST = GIT_REMOTE_PORT_INITIAL + 99
+
+
+class GitRemote(object):
+
+ def __init__(self, builddir, serveddir, logtofile):
+ self.daemon = None
+ self.port = None
+ self.serveddir = serveddir
+ self.logfile = infra.open_log_file(builddir, "run", logtofile)
+
+ # Start a local git server
+ #
+ # In order to support test cases in parallel, select the port the server
+ # will listen to in runtime. Since there is no reliable way to allocate the
+ # port prior to starting the server (another process in the host machine
+ # can use the port between it is selected from a list and it is really
+ # allocated to the server) try to start the server in a port and in the
+ # case it is already in use, try the next one in the allowed range.
+ #
+ def start(self):
+ daemon_cmd = ["git", "daemon", "--reuseaddr", "--verbose",
+ "--listen=localhost", "--export-all",
+ "--base-path={}".format(self.serveddir)]
+ for port in range(GIT_REMOTE_PORT_INITIAL, GIT_REMOTE_PORT_LAST):
+ port_arg = ["--port={}".format(port)]
+ cmd = daemon_cmd + port_arg
+ self.logfile.write("> starting git remote with '{}'\n"
+ .format(" ".join(cmd)))
+ self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile)
+ ret = self.daemon.expect(["Ready to rumble",
+ "Address already in use"])
+ if ret == 0:
+ self.port = port
+ break
+ self.daemon.read()
+ if not self.port:
+ raise SystemError("Could not find a free port to run git remote")
+
+ def stop(self):
+ if self.daemon is None:
+ return
+ self.daemon.terminate(force=True)
--
2.13.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 5/7] testing/tests/download: add git hash tests
2017-08-26 22:20 ` [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build() Ricardo Martincoski
` (2 preceding siblings ...)
2017-08-26 22:20 ` [Buildroot] [next v2 4/7] testing/tests/download: add infra for git tests Ricardo Martincoski
@ 2017-08-26 22:20 ` Ricardo Martincoski
2017-08-26 22:38 ` Ricardo Martincoski
` (2 more replies)
2017-08-26 22:20 ` [Buildroot] [next v2 6/7] testing/tests/download: add test for sha1 as git ref Ricardo Martincoski
` (3 subsequent siblings)
7 siblings, 3 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-08-26 22:20 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Add three test cases for hash of a git package:
- correct hash;
- wrong hash;
- no hash file.
Besides the Python script, add some fixtures used during the tests:
- a br2-external (git-hash) with one package for each test case;
- a static git bare repo (repo.git) to be served using GitRemote class
for all these test cases.
Neither the br2-external nor the check hash functionalities are the
subject of these tests per se, so for simplicity limit the check to the
error codes and don't look for the messages in the log.
Thanks to Arnout for the hint about how to add a bare repo to test.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Changes v1 -> v2:
- new patch
---
.gitlab-ci.yml | 3 ++
.../tests/download/br2-external/git-hash/Config.in | 0
.../download/br2-external/git-hash/external.desc | 1 +
.../download/br2-external/git-hash/external.mk | 1 +
.../br2-external/git-hash/package/bad/bad.hash | 1 +
.../br2-external/git-hash/package/bad/bad.mk | 12 ++++++++
.../br2-external/git-hash/package/good/good.hash | 1 +
.../br2-external/git-hash/package/good/good.mk | 12 ++++++++
.../br2-external/git-hash/package/nohash/nohash.mk | 12 ++++++++
.../download/git-remote/repo.git/.gitattributes | 1 +
.../tests/download/git-remote/repo.git/HEAD | 1 +
.../tests/download/git-remote/repo.git/config | 4 +++
.../12/33c8f882d130978246a30da75892321f13e5bb | Bin 0 -> 49 bytes
.../74/46f8dab6cdb166aaf130aa88633dc627a4e8ad | Bin 0 -> 153 bytes
.../8e/27be7d6154a1f68ea9160ef0e18691d20560dc | Bin 0 -> 20 bytes
.../download/git-remote/repo.git/refs/heads/master | 1 +
support/testing/tests/download/test_git_hash.py | 34 +++++++++++++++++++++
17 files changed, 84 insertions(+)
create mode 100644 support/testing/tests/download/br2-external/git-hash/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-hash/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-hash/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.hash
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk
create mode 100644 support/testing/tests/download/git-remote/repo.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/repo.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/repo.git/config
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/12/33c8f882d130978246a30da75892321f13e5bb
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/74/46f8dab6cdb166aaf130aa88633dc627a4e8ad
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/8e/27be7d6154a1f68ea9160ef0e18691d20560dc
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/master
create mode 100644 support/testing/tests/download/test_git_hash.py
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index fb50692976..98022b6d7f 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -219,6 +219,9 @@ tests.core.test_rootfs_overlay.TestRootfsOverlay: *runtime_test
tests.core.test_timezone.TestGlibcAllTimezone: *runtime_test
tests.core.test_timezone.TestGlibcNonDefaultLimitedTimezone: *runtime_test
tests.core.test_timezone.TestNoTimezone: *runtime_test
+tests.download.test_git_hash.TestGitBadHash: *runtime_test
+tests.download.test_git_hash.TestGitGoodHash: *runtime_test
+tests.download.test_git_hash.TestGitNoHash: *runtime_test
tests.fs.test_ext.TestExt2: *runtime_test
tests.fs.test_ext.TestExt2r1: *runtime_test
tests.fs.test_ext.TestExt3: *runtime_test
diff --git a/support/testing/tests/download/br2-external/git-hash/Config.in b/support/testing/tests/download/br2-external/git-hash/Config.in
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/support/testing/tests/download/br2-external/git-hash/external.desc b/support/testing/tests/download/br2-external/git-hash/external.desc
new file mode 100644
index 0000000000..41316c8b25
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/external.desc
@@ -0,0 +1 @@
+name: GIT_HASH
diff --git a/support/testing/tests/download/br2-external/git-hash/external.mk b/support/testing/tests/download/br2-external/git-hash/external.mk
new file mode 100644
index 0000000000..b69746e06e
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/external.mk
@@ -0,0 +1 @@
+include $(sort $(wildcard $(BR2_EXTERNAL_GIT_HASH_PATH)/package/*/*.mk))
diff --git a/support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
new file mode 100644
index 0000000000..90bfb63085
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
@@ -0,0 +1 @@
+sha256 0000000000000000000000000000000000000000000000000000000000000000 bad-7446f8dab6cdb166aaf130aa88633dc627a4e8ad.tar.gz
diff --git a/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
new file mode 100644
index 0000000000..2ad06fa47f
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
@@ -0,0 +1,12 @@
+################################################################################
+#
+# bad
+#
+################################################################################
+
+BAD_VERSION = 7446f8dab6cdb166aaf130aa88633dc627a4e8ad
+# Get the git server port number from the test infra
+GITREMOTE_PORT_NUMBER ?= 9418
+BAD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-hash/package/good/good.hash b/support/testing/tests/download/br2-external/git-hash/package/good/good.hash
new file mode 100644
index 0000000000..ccb2efbffe
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/good/good.hash
@@ -0,0 +1 @@
+sha256 9302294b11ac6742c97eec6b7fd677fc3ac79b3f9348fbb1a46ba3ccb5287f9f good-7446f8dab6cdb166aaf130aa88633dc627a4e8ad.tar.gz
diff --git a/support/testing/tests/download/br2-external/git-hash/package/good/good.mk b/support/testing/tests/download/br2-external/git-hash/package/good/good.mk
new file mode 100644
index 0000000000..0e82e0e6c6
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/good/good.mk
@@ -0,0 +1,12 @@
+################################################################################
+#
+# good
+#
+################################################################################
+
+GOOD_VERSION = 7446f8dab6cdb166aaf130aa88633dc627a4e8ad
+# Get the git server port number from the test infra
+GITREMOTE_PORT_NUMBER ?= 9418
+GOOD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk b/support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk
new file mode 100644
index 0000000000..4e3943211c
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk
@@ -0,0 +1,12 @@
+################################################################################
+#
+# nohash
+#
+################################################################################
+
+NOHASH_VERSION = 7446f8dab6cdb166aaf130aa88633dc627a4e8ad
+# Get the git server port number from the test infra
+GITREMOTE_PORT_NUMBER ?= 9418
+NOHASH_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/repo.git/.gitattributes b/support/testing/tests/download/git-remote/repo.git/.gitattributes
new file mode 100644
index 0000000000..eb50c64a21
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/.gitattributes
@@ -0,0 +1 @@
+objects/*/* binary
diff --git a/support/testing/tests/download/git-remote/repo.git/HEAD b/support/testing/tests/download/git-remote/repo.git/HEAD
new file mode 100644
index 0000000000..cb089cd89a
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/HEAD
@@ -0,0 +1 @@
+ref: refs/heads/master
diff --git a/support/testing/tests/download/git-remote/repo.git/config b/support/testing/tests/download/git-remote/repo.git/config
new file mode 100644
index 0000000000..07d359d07c
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/config
@@ -0,0 +1,4 @@
+[core]
+ repositoryformatversion = 0
+ filemode = true
+ bare = true
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/12/33c8f882d130978246a30da75892321f13e5bb b/support/testing/tests/download/git-remote/repo.git/objects/12/33c8f882d130978246a30da75892321f13e5bb
new file mode 100644
index 0000000000000000000000000000000000000000..9f711b51ca36f85619d9cff0226e163f84a9d232
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5^+oXg@YxcZK1AR5GTg_{e_Jgs$#(}c19v6=
Fb^xfh5%K^4
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/74/46f8dab6cdb166aaf130aa88633dc627a4e8ad b/support/testing/tests/download/git-remote/repo.git/objects/74/46f8dab6cdb166aaf130aa88633dc627a4e8ad
new file mode 100644
index 0000000000000000000000000000000000000000..f25a70acdb1fe93d61fa64cee624b3eaee653adf
GIT binary patch
literal 153
zcmV;K0A~Mq0j<ux4Z<)GKw)Z6aSJ4}eKD~SLM%W76I}cQC*oK#reJ#_QP3mWuXuXC
z<jf89=r*m!2udS*o2=DAp>(6w!<JA68jW=twMq)3nI=Kcb}JRm?5TzVC#sDXUmkac
zef?mczj(Z6;<@)FA3zPx8Dpg}&`Fe1th!FL_@|tp5Fqg!g}Cr_iJ>bgb<=D2%dL0=
HR6}0pH2X&9
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/8e/27be7d6154a1f68ea9160ef0e18691d20560dc b/support/testing/tests/download/git-remote/repo.git/objects/8e/27be7d6154a1f68ea9160ef0e18691d20560dc
new file mode 100644
index 0000000000000000000000000000000000000000..a8e78a9d1b29e6afe9554cd3ac26c33ab697b782
GIT binary patch
literal 20
bcmb<m^geacKgb|e!*i|1Lna2vt;|ONPIm`P
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/heads/master b/support/testing/tests/download/git-remote/repo.git/refs/heads/master
new file mode 100644
index 0000000000..0180a4d15f
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/heads/master
@@ -0,0 +1 @@
+7446f8dab6cdb166aaf130aa88633dc627a4e8ad
diff --git a/support/testing/tests/download/test_git_hash.py b/support/testing/tests/download/test_git_hash.py
new file mode 100644
index 0000000000..1d483844fa
--- /dev/null
+++ b/support/testing/tests/download/test_git_hash.py
@@ -0,0 +1,34 @@
+import os
+
+import infra
+from gitbase import GitTestBase
+
+
+class TestGitHash(GitTestBase):
+ br2_external = infra.filepath("tests/download/br2-external/git-hash")
+ serveddir = infra.filepath("tests/download/git-remote")
+
+ def check_hash(self, package):
+ # store downloaded tarball inside the output dir so the test infra
+ # cleans it up@the end
+ env = {"BR2_DL_DIR": os.path.join(self.builddir, "dl"),
+ "GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
+ self.b.build(["{}-dirclean".format(package),
+ "{}-source".format(package)],
+ env, self.logfile)
+
+
+class TestGitGoodHash(TestGitHash):
+ def test_run(self):
+ self.check_hash("good")
+
+
+class TestGitBadHash(TestGitHash):
+ def test_run(self):
+ with self.assertRaises(SystemError):
+ self.check_hash("bad")
+
+
+class TestGitNoHash(TestGitHash):
+ def test_run(self):
+ self.check_hash("nohash")
--
2.13.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 6/7] testing/tests/download: add test for sha1 as git ref
2017-08-26 22:20 ` [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build() Ricardo Martincoski
` (3 preceding siblings ...)
2017-08-26 22:20 ` [Buildroot] [next v2 5/7] testing/tests/download: add git hash tests Ricardo Martincoski
@ 2017-08-26 22:20 ` Ricardo Martincoski
2017-10-06 21:57 ` Arnout Vandecappelle
2017-08-26 22:20 ` [Buildroot] [next v2 7/7] testing/tests/download: add test for git submodule Ricardo Martincoski
` (2 subsequent siblings)
7 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2017-08-26 22:20 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Add a test to download the package with git method and its version set
to a sha1 reachable by a branch name, but not pointed by it.
This is the most common use case for git refs in the tree.
Besides the test case Python script, add:
- a br2-external (git-refs) with a fake package (foo) as a fixture for
this and upcoming tests;
- support in GitTestBase for repos created on the fly;
- a new class GitRepo to create and manipulate repos. It is a class and
not a simple module because it will become very useful when testing
submodules.
After the download, compare the tarball to a checkout of the original
remote repo.
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
---
Changes v1 -> v2:
- rename the main test file to testgit (Arnout). Actually I broke it
down to a common gitbase and the two test_git_refs and test_git_hash
(this last one is new, in the previous patch);
- remove some weird/wrong TODO I had in v1 code (Arnout);
- use the same structure for test cases as used in the test infra;
- raise an exception when the download fails (Arnout). I did not add
code for this since I let the builder class to raise it;
- I reimplemented git_util as gitrepo. Arnout suggested to use it as
module and I first implemented that way locally, but then I created
test cases for submodules and I realized now I have a reason to use
a class (see next patch);
- move package (now called foo) to a BR2_EXTERNAL (Arnout);
- override BR2_DL_DIR when calling make (Arnout);
- the test infra uses O= so I don't need to;
- instead of removing the files in the dl/ folder, use a different dir
as dl/ dir since now each repo has a different name;
- open the tarball to check its contents (Arnout). I create a fresh
clone of the repo before the test to compare against;
- this patch is not checking for the actual sha1 anymore, it can be
done later. But the check for the contents should cover most cases;
- my old argument to not test only the support/download/dl-wrapper is
not valid anymore since I mimic the logic from the scripts to know
the name of the tarball. But I still think calling make ...-source
has better maintenance. And also the previous patch (test for hash
of packages with git method) can use the same base class;
- this patch is part of series 1/3 of a new version of
http://patchwork.ozlabs.org/patch/690097/
---
.gitlab-ci.yml | 1 +
.../tests/download/br2-external/git-refs/Config.in | 0
.../download/br2-external/git-refs/external.desc | 1 +
.../download/br2-external/git-refs/external.mk | 1 +
.../br2-external/git-refs/package/foo/foo.mk | 13 +++
support/testing/tests/download/gitbase.py | 13 +++
support/testing/tests/download/gitrepo.py | 82 ++++++++++++++++++
support/testing/tests/download/test_git_refs.py | 96 ++++++++++++++++++++++
8 files changed, 207 insertions(+)
create mode 100644 support/testing/tests/download/br2-external/git-refs/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-refs/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-refs/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/foo/foo.mk
create mode 100644 support/testing/tests/download/gitrepo.py
create mode 100644 support/testing/tests/download/test_git_refs.py
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 98022b6d7f..6d64a2c0b6 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -222,6 +222,7 @@ tests.core.test_timezone.TestNoTimezone: *runtime_test
tests.download.test_git_hash.TestGitBadHash: *runtime_test
tests.download.test_git_hash.TestGitGoodHash: *runtime_test
tests.download.test_git_hash.TestGitNoHash: *runtime_test
+tests.download.test_git_refs.TestGitSha1InsideBranch: *runtime_test
tests.fs.test_ext.TestExt2: *runtime_test
tests.fs.test_ext.TestExt2r1: *runtime_test
tests.fs.test_ext.TestExt3: *runtime_test
diff --git a/support/testing/tests/download/br2-external/git-refs/Config.in b/support/testing/tests/download/br2-external/git-refs/Config.in
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/support/testing/tests/download/br2-external/git-refs/external.desc b/support/testing/tests/download/br2-external/git-refs/external.desc
new file mode 100644
index 0000000000..69f40d46c6
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/external.desc
@@ -0,0 +1 @@
+name: GIT_REFS
diff --git a/support/testing/tests/download/br2-external/git-refs/external.mk b/support/testing/tests/download/br2-external/git-refs/external.mk
new file mode 100644
index 0000000000..de1aad635f
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/external.mk
@@ -0,0 +1 @@
+include $(sort $(wildcard $(BR2_EXTERNAL_GIT_REFS_PATH)/package/*/*.mk))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/foo/foo.mk b/support/testing/tests/download/br2-external/git-refs/package/foo/foo.mk
new file mode 100644
index 0000000000..01c926e177
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/foo/foo.mk
@@ -0,0 +1,13 @@
+################################################################################
+#
+# foo
+#
+################################################################################
+
+# Get all the data from the test infra
+FOO_VERSION ?= 0
+GITREMOTE_PORT_NUMBER ?= 9418
+GITREMOTE_REPO ?= repo.git
+FOO_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/$(GITREMOTE_REPO)
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/gitbase.py b/support/testing/tests/download/gitbase.py
index b259914c77..b70e6400bc 100644
--- a/support/testing/tests/download/gitbase.py
+++ b/support/testing/tests/download/gitbase.py
@@ -1,6 +1,9 @@
+import os
+
import infra.basetest
from infra.builder import Builder
from gitremote import GitRemote
+from gitrepo import GitRepo
class GitTestBase(infra.basetest.BRTest):
@@ -9,6 +12,7 @@ class GitTestBase(infra.basetest.BRTest):
BR2_BACKUP_SITE=""
"""
br2_external = None
+ repo = None
serveddir = None
gitremote = None
logfile = None
@@ -27,6 +31,15 @@ class GitTestBase(infra.basetest.BRTest):
# time a test runs
self.b.build(["dependencies"])
+ # for the case we are dinamically creating repos
+ if self.serveddir is None:
+ # place the repo in the images/ dir so it appears as an artifact
+ # when the test runs in gitlab infra
+ self.serveddir = os.path.join(self.builddir, "images")
+ if not os.path.exists(self.serveddir): # for run-tests -k
+ os.mkdir(self.serveddir)
+ self.repo = GitRepo(self.builddir, self.serveddir, self.logtofile)
+
self.gitremote = GitRemote(self.builddir, self.serveddir,
self.logtofile)
# send output from the test to the logfile created by GitRemote
diff --git a/support/testing/tests/download/gitrepo.py b/support/testing/tests/download/gitrepo.py
new file mode 100644
index 0000000000..47bdba1b7c
--- /dev/null
+++ b/support/testing/tests/download/gitrepo.py
@@ -0,0 +1,82 @@
+import os
+import subprocess
+import tempfile
+
+import infra
+
+
+class GitRepo(object):
+
+ def __init__(self, builddir, serveddir, logtofile):
+ self.logfile = infra.open_log_file(builddir, "run", logtofile)
+ self.remotedir = tempfile.mkdtemp(dir=serveddir)
+
+ # Run a git command in the emulated remote repo
+ def _git(self, git_cmd):
+ cmd = ["git"] + git_cmd
+ self.logfile.write("> Running command '{}' @ '{}'\n"
+ .format(" ".join(cmd), self.remotedir))
+ self.logfile.flush()
+ r = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
+ cwd=self.remotedir)
+ if r != 0:
+ raise SystemError("'{}' returned {}".format(" ".join(cmd), r))
+
+ def _sha1(self, ref="HEAD"):
+ cmd = ["git", "rev-parse", ref]
+ out = subprocess.check_output(cmd, stderr=self.logfile,
+ cwd=self.remotedir)
+ sha1 = out.strip()
+ self.logfile.write("> {}\n".format(sha1))
+ return sha1
+
+ # Initialize a repo with an initial commit. Git by default gives us the
+ # master branch pointing to this commit. Leave it there by detaching the
+ # HEAD before creating any more commits. It will become handy before we
+ # start the actual download test, since we can checkout this branch to move
+ # HEAD away from the ref to be tested, thus avoiding false OK in the case
+ # the HEAD can be checked out but the ref doesn't.
+ def git_init(self):
+ self._git(["init"])
+ # Ensure git know us before commit (needed to run in the docker image)
+ self._git(["config", "user.email", "'you at example.com'"])
+ self._git(["config", "user.name", "'Your Name'"])
+ self.git_commit()
+ self.git_checkout("HEAD~0")
+ return os.path.basename(self.remotedir)
+
+ # Create a commit with random data (actually a randomly named empty file).
+ # This way the tarball can be checked by comparing the contents of a clone
+ # of the repo against the contents of the tarball.
+ # Use the name of the new file as the commit message so when the git tree
+ # is dumped to the logfile, it displays useful info.
+ def git_commit(self):
+ _, f = tempfile.mkstemp(dir=self.remotedir)
+ self._git(["add", os.path.basename(f)])
+ self._git(["commit", "-m", os.path.basename(f)])
+ return self._sha1()
+
+ def git_branch(self, branch):
+ self._git(["branch", branch])
+ return self._sha1()
+
+ def git_checkout(self, ref):
+ self._git(["checkout", "-f", ref])
+ return self._sha1()
+
+ # Create a fresh clone of the repo to be compared to the tarball.
+ # For simplicity, create this copy inside the repo itself, so all go away
+ # when the test ends (when run-tests is called without -k).
+ def create_a_copy_to_compare(self):
+ self._git(["clone", self.remotedir, "copy"])
+ return os.path.join(self.remotedir, "copy")
+
+ # Move HEAD of emulated remote away from desired commit to avoid false OK
+ # when the reference under test cannot be fetched but HEAD can.
+ def move_head_away_from_ref_under_test(self):
+ self.git_checkout("master")
+
+ # For debug purposes, dump to the log a nice ascii art of all commits in
+ # the repo, including the short sha1, commit title and the refs.
+ def save_git_tree_to_log(self):
+ self._git(["log", "--all", "--oneline", "--graph", "--decorate"])
diff --git a/support/testing/tests/download/test_git_refs.py b/support/testing/tests/download/test_git_refs.py
new file mode 100644
index 0000000000..2f0709d703
--- /dev/null
+++ b/support/testing/tests/download/test_git_refs.py
@@ -0,0 +1,96 @@
+import os
+import tarfile
+
+import infra
+from gitbase import GitTestBase
+
+
+def files_from_tarball(tarball):
+ filelist = None
+ with tarfile.open(tarball) as tf:
+ filelist = tf.getnames()
+ path_inside_tarball = os.path.basename(tarball).split('.')[0]
+ files = [os.path.relpath(f, start=path_inside_tarball) for f in filelist]
+ return sorted(files)
+
+
+def files_from_checkout(checkout):
+ filelist = []
+ for dirpath, dirnames, filenames in os.walk(checkout):
+ if '.git' in dirnames:
+ dirnames.remove('.git')
+ filelist += [os.path.join(dirpath, f) for f in filenames]
+ files = [os.path.relpath(f, start=checkout) for f in filelist]
+ return sorted(files)
+
+
+def compare_tarball_with_checkout(tarball, checkout, logfile):
+ t_files = files_from_tarball(tarball)
+ c_files = files_from_checkout(checkout)
+ only_in_t = [f for f in t_files if f not in c_files]
+ only_in_c = [f for f in c_files if f not in t_files]
+ common = [f for f in t_files if f in c_files]
+ logfile.write("> Comparing resulting tarball with a checkout of the repo\n"
+ "> only in tarball: " + " ".join(only_in_t) + "\n"
+ "> missing from tarball: " + " ".join(only_in_c) + "\n"
+ "> common files: " + " ".join(common) + "\n")
+ return len(only_in_t + only_in_c)
+
+
+class TestGitRefs(GitTestBase):
+ br2_external = infra.filepath("tests/download/br2-external/git-refs")
+
+ # Common preparations of the repo before the actual test
+ def _prepare_the_repo(self, ref):
+ # create a checkout from current commit to compare the generated
+ # tarball against it later
+ self.repo.git_checkout(ref)
+ self.checkoutdir = self.repo.create_a_copy_to_compare()
+ self.repo.move_head_away_from_ref_under_test()
+ # save debug info about the git repos
+ self.repo.save_git_tree_to_log()
+
+ # Download the sources from the emulated remote
+ def _download_ref(self, ref, remotedir):
+ self.dldir = os.path.join(self.builddir, "dl",
+ os.path.basename(remotedir))
+ env = {"BR2_DL_DIR": self.dldir,
+ "FOO_VERSION": ref,
+ "GITREMOTE_REPO": remotedir,
+ "GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
+ # download the sources
+ self.b.build(["foo-dirclean", "foo-source"], env, self.logfile)
+
+ # Check the tarball files against a clone of the emulated remote repo
+ def _check_tarball(self, ref):
+ tarball = os.path.join(self.dldir,
+ "foo-{}.tar.gz".format(ref.replace('/', '_')))
+ ret = compare_tarball_with_checkout(tarball, self.checkoutdir,
+ self.logfile)
+ self.assertEqual(ret, 0, "downloaded source does not match")
+
+ # Download the sources and check
+ #
+ # ref: the git reference to be tested (tag, branch, sha1, special ref).
+ # e.g. "mybranch"
+ #
+ # remotedir: local path to the repo exported by git server.
+ # usually self.repo.git_init()
+ #
+ def check_download(self, ref, remotedir):
+ self._prepare_the_repo(ref)
+ self._download_ref(ref, remotedir)
+ self._check_tarball(ref)
+
+
+class TestGitSha1InsideBranch(TestGitRefs):
+ # Repo layout under test:
+ # * sha1_3 (mybranch)
+ # * sha1_2<<<
+ # * sha1_1 (HEAD -> master)
+ def test_run(self):
+ remotedir = self.repo.git_init()
+ ref = self.repo.git_commit()
+ self.repo.git_commit()
+ self.repo.git_branch("mybranch")
+ self.check_download(ref, remotedir)
--
2.13.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 7/7] testing/tests/download: add test for git submodule
2017-08-26 22:20 ` [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build() Ricardo Martincoski
` (4 preceding siblings ...)
2017-08-26 22:20 ` [Buildroot] [next v2 6/7] testing/tests/download: add test for sha1 as git ref Ricardo Martincoski
@ 2017-08-26 22:20 ` Ricardo Martincoski
2017-10-06 20:31 ` [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build() Arnout Vandecappelle
2017-10-29 14:05 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
7 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-08-26 22:20 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Add a test that downloads a git package with recursive submodules.
Add the required support in the GitRepo class.
Use the br2-external added by the first test.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
---
Changes v1 -> v2:
- rewrite using new git test infra;
- this patch is part of series 1/3 of a new version of
http://patchwork.ozlabs.org/patch/690097/
---
.gitlab-ci.yml | 1 +
support/testing/tests/download/gitrepo.py | 35 +++++++++++++++++++--
support/testing/tests/download/test_git_refs.py | 42 +++++++++++++++++++++----
3 files changed, 70 insertions(+), 8 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 6d64a2c0b6..c782f66d5d 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -222,6 +222,7 @@ tests.core.test_timezone.TestNoTimezone: *runtime_test
tests.download.test_git_hash.TestGitBadHash: *runtime_test
tests.download.test_git_hash.TestGitGoodHash: *runtime_test
tests.download.test_git_hash.TestGitNoHash: *runtime_test
+tests.download.test_git_refs.TestGitRecursiveSubmodule: *runtime_test
tests.download.test_git_refs.TestGitSha1InsideBranch: *runtime_test
tests.fs.test_ext.TestExt2: *runtime_test
tests.fs.test_ext.TestExt2r1: *runtime_test
diff --git a/support/testing/tests/download/gitrepo.py b/support/testing/tests/download/gitrepo.py
index 47bdba1b7c..5940f94522 100644
--- a/support/testing/tests/download/gitrepo.py
+++ b/support/testing/tests/download/gitrepo.py
@@ -8,8 +8,12 @@ import infra
class GitRepo(object):
def __init__(self, builddir, serveddir, logtofile):
+ self.builddir = builddir
+ self.serveddir = serveddir
+ self.logtofile = logtofile
self.logfile = infra.open_log_file(builddir, "run", logtofile)
self.remotedir = tempfile.mkdtemp(dir=serveddir)
+ self.submodule = None
# Run a git command in the emulated remote repo
def _git(self, git_cmd):
@@ -64,19 +68,46 @@ class GitRepo(object):
self._git(["checkout", "-f", ref])
return self._sha1()
+ def git_submodule(self, recurse=1):
+ if recurse < 1:
+ return self.git_commit()
+
+ self.submodule = GitRepo(self.builddir, self.serveddir, self.logtofile)
+ d = self.submodule.git_init()
+ ref = self.submodule.git_submodule(recurse - 1)
+
+ # make submodule commit reachable by a branch but not@its head
+ self.submodule.git_commit()
+ self.submodule.git_branch("anybranch")
+ self.submodule.git_checkout(ref)
+
+ self._git(["submodule", "add",
+ os.path.join("..", os.path.basename(d))])
+ self._git(["commit", "-m", os.path.basename(d)])
+ return self._sha1()
+
# Create a fresh clone of the repo to be compared to the tarball.
# For simplicity, create this copy inside the repo itself, so all go away
# when the test ends (when run-tests is called without -k).
- def create_a_copy_to_compare(self):
- self._git(["clone", self.remotedir, "copy"])
+ def create_a_copy_to_compare(self, submodules):
+ if submodules:
+ self._git(["clone", "--recursive", self.remotedir, "copy"])
+ else:
+ self._git(["clone", self.remotedir, "copy"])
return os.path.join(self.remotedir, "copy")
# Move HEAD of emulated remote away from desired commit to avoid false OK
# when the reference under test cannot be fetched but HEAD can.
def move_head_away_from_ref_under_test(self):
self.git_checkout("master")
+ if self.submodule:
+ self.submodule.move_head_away_from_ref_under_test()
# For debug purposes, dump to the log a nice ascii art of all commits in
# the repo, including the short sha1, commit title and the refs.
def save_git_tree_to_log(self):
self._git(["log", "--all", "--oneline", "--graph", "--decorate"])
+ # don't use 'git submodule foreach', it only works as expected if the
+ # desired sha1 (that contains the submodule) is checked out
+ if self.submodule:
+ self.submodule.save_git_tree_to_log()
diff --git a/support/testing/tests/download/test_git_refs.py b/support/testing/tests/download/test_git_refs.py
index 2f0709d703..a58d806924 100644
--- a/support/testing/tests/download/test_git_refs.py
+++ b/support/testing/tests/download/test_git_refs.py
@@ -41,23 +41,25 @@ class TestGitRefs(GitTestBase):
br2_external = infra.filepath("tests/download/br2-external/git-refs")
# Common preparations of the repo before the actual test
- def _prepare_the_repo(self, ref):
+ def _prepare_the_repo(self, ref, submodules):
# create a checkout from current commit to compare the generated
# tarball against it later
self.repo.git_checkout(ref)
- self.checkoutdir = self.repo.create_a_copy_to_compare()
+ self.checkoutdir = self.repo.create_a_copy_to_compare(submodules)
self.repo.move_head_away_from_ref_under_test()
# save debug info about the git repos
self.repo.save_git_tree_to_log()
# Download the sources from the emulated remote
- def _download_ref(self, ref, remotedir):
+ def _download_ref(self, ref, remotedir, submodules):
self.dldir = os.path.join(self.builddir, "dl",
os.path.basename(remotedir))
env = {"BR2_DL_DIR": self.dldir,
"FOO_VERSION": ref,
"GITREMOTE_REPO": remotedir,
"GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
+ if submodules:
+ env["FOO_GIT_SUBMODULES"] = "YES"
# download the sources
self.b.build(["foo-dirclean", "foo-source"], env, self.logfile)
@@ -77,9 +79,12 @@ class TestGitRefs(GitTestBase):
# remotedir: local path to the repo exported by git server.
# usually self.repo.git_init()
#
- def check_download(self, ref, remotedir):
- self._prepare_the_repo(ref)
- self._download_ref(ref, remotedir)
+ # submodules: enable git submodules support on the recipe?
+ # e.g. False
+ #
+ def check_download(self, ref, remotedir, submodules=False):
+ self._prepare_the_repo(ref, submodules)
+ self._download_ref(ref, remotedir, submodules)
self._check_tarball(ref)
@@ -94,3 +99,28 @@ class TestGitSha1InsideBranch(TestGitRefs):
self.repo.git_commit()
self.repo.git_branch("mybranch")
self.check_download(ref, remotedir)
+
+
+class TestGitRecursiveSubmodule(TestGitRefs):
+ # Repo layout under test:
+ # * sha1_3 (mybranch)
+ # * sha1_2<<< submodule_a sha1_2a
+ # * sha1_1 (HEAD -> master)
+ # submodule_a:
+ # * sha1_3a (anybranch)
+ # * sha1_2a<<< submodule_b sha1_2b
+ # * sha1_1a (HEAD -> master)
+ # submodule_b:
+ # * sha1_3b (anybranch)
+ # * sha1_2b<<< submodule_c sha1_2c
+ # * sha1_1b (HEAD -> master)
+ # submodule_c:
+ # * sha1_3c (anybranch)
+ # * sha1_2c<<<
+ # * sha1_1c (HEAD -> master)
+ def test_run(self):
+ remotedir = self.repo.git_init()
+ ref = self.repo.git_submodule(3)
+ self.repo.git_commit()
+ self.repo.git_branch("mybranch")
+ self.check_download(ref, remotedir, submodules=True)
--
2.13.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 5/7] testing/tests/download: add git hash tests
2017-08-26 22:20 ` [Buildroot] [next v2 5/7] testing/tests/download: add git hash tests Ricardo Martincoski
@ 2017-08-26 22:38 ` Ricardo Martincoski
2017-08-27 4:00 ` Baruch Siach
2017-10-06 21:36 ` Arnout Vandecappelle
2017-10-06 21:44 ` Arnout Vandecappelle
2 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2017-08-26 22:38 UTC (permalink / raw)
To: buildroot
Hello,
On Sat, Aug 26, 2017 at 07:20 PM, Ricardo Martincoski wrote:
> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>
> Add three test cases for hash of a git package:
> - correct hash;
> - wrong hash;
> - no hash file.
>
> Besides the Python script, add some fixtures used during the tests:
> - a br2-external (git-hash) with one package for each test case;
> - a static git bare repo (repo.git) to be served using GitRemote class
> for all these test cases.
>
> Neither the br2-external nor the check hash functionalities are the
> subject of these tests per se, so for simplicity limit the check to the
> error codes and don't look for the messages in the log.
>
> Thanks to Arnout for the hint about how to add a bare repo to test.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> ---
> Changes v1 -> v2:
> - new patch
> ---
> .gitlab-ci.yml | 3 ++
> .../tests/download/br2-external/git-hash/Config.in | 0
> .../download/br2-external/git-hash/external.desc | 1 +
> .../download/br2-external/git-hash/external.mk | 1 +
> .../br2-external/git-hash/package/bad/bad.hash | 1 +
> .../br2-external/git-hash/package/bad/bad.mk | 12 ++++++++
> .../br2-external/git-hash/package/good/good.hash | 1 +
> .../br2-external/git-hash/package/good/good.mk | 12 ++++++++
> .../br2-external/git-hash/package/nohash/nohash.mk | 12 ++++++++
> .../download/git-remote/repo.git/.gitattributes | 1 +
> .../tests/download/git-remote/repo.git/HEAD | 1 +
> .../tests/download/git-remote/repo.git/config | 4 +++
> .../12/33c8f882d130978246a30da75892321f13e5bb | Bin 0 -> 49 bytes
> .../74/46f8dab6cdb166aaf130aa88633dc627a4e8ad | Bin 0 -> 153 bytes
> .../8e/27be7d6154a1f68ea9160ef0e18691d20560dc | Bin 0 -> 20 bytes
> .../download/git-remote/repo.git/refs/heads/master | 1 +
> support/testing/tests/download/test_git_hash.py | 34 +++++++++++++++++++++
> 17 files changed, 84 insertions(+)
This patch didn't get to mailing list.
I guess it is because of binary files added.
Did I miss something to properly send this?
Anyway it can be seen here:
https://gitlab.com/RicardoMartincoski/buildroot/commit/32611754f77a17dbbfae26e10da55300ebea0b83
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 5/7] testing/tests/download: add git hash tests
2017-08-26 22:38 ` Ricardo Martincoski
@ 2017-08-27 4:00 ` Baruch Siach
0 siblings, 0 replies; 108+ messages in thread
From: Baruch Siach @ 2017-08-27 4:00 UTC (permalink / raw)
To: buildroot
Hi Ricardo,
On Sat, Aug 26, 2017 at 07:38:31PM -0300, Ricardo Martincoski wrote:
> On Sat, Aug 26, 2017 at 07:20 PM, Ricardo Martincoski wrote:
> > From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> >
> > Add three test cases for hash of a git package:
> > - correct hash;
> > - wrong hash;
> > - no hash file.
> >
> > Besides the Python script, add some fixtures used during the tests:
> > - a br2-external (git-hash) with one package for each test case;
> > - a static git bare repo (repo.git) to be served using GitRemote class
> > for all these test cases.
> >
> > Neither the br2-external nor the check hash functionalities are the
> > subject of these tests per se, so for simplicity limit the check to the
> > error codes and don't look for the messages in the log.
> >
> > Thanks to Arnout for the hint about how to add a bare repo to test.
> >
> > Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> > Cc: Arnout Vandecappelle <arnout@mind.be>
> > ---
> > Changes v1 -> v2:
> > - new patch
> > ---
> > .gitlab-ci.yml | 3 ++
> > .../tests/download/br2-external/git-hash/Config.in | 0
> > .../download/br2-external/git-hash/external.desc | 1 +
> > .../download/br2-external/git-hash/external.mk | 1 +
> > .../br2-external/git-hash/package/bad/bad.hash | 1 +
> > .../br2-external/git-hash/package/bad/bad.mk | 12 ++++++++
> > .../br2-external/git-hash/package/good/good.hash | 1 +
> > .../br2-external/git-hash/package/good/good.mk | 12 ++++++++
> > .../br2-external/git-hash/package/nohash/nohash.mk | 12 ++++++++
> > .../download/git-remote/repo.git/.gitattributes | 1 +
> > .../tests/download/git-remote/repo.git/HEAD | 1 +
> > .../tests/download/git-remote/repo.git/config | 4 +++
> > .../12/33c8f882d130978246a30da75892321f13e5bb | Bin 0 -> 49 bytes
> > .../74/46f8dab6cdb166aaf130aa88633dc627a4e8ad | Bin 0 -> 153 bytes
> > .../8e/27be7d6154a1f68ea9160ef0e18691d20560dc | Bin 0 -> 20 bytes
> > .../download/git-remote/repo.git/refs/heads/master | 1 +
> > support/testing/tests/download/test_git_hash.py | 34 +++++++++++++++++++++
> > 17 files changed, 84 insertions(+)
>
> This patch didn't get to mailing list.
> I guess it is because of binary files added.
>
> Did I miss something to properly send this?
>
> Anyway it can be seen here:
> https://gitlab.com/RicardoMartincoski/buildroot/commit/32611754f77a17dbbfae26e10da55300ebea0b83
I got this patch from the list. It is also archived at
http://lists.busybox.net/pipermail/buildroot/2017-August/200933.html.
baruch
--
http://baruch.siach.name/blog/ ~. .~ Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
- baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il -
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build()
2017-08-26 22:20 ` [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build() Ricardo Martincoski
` (5 preceding siblings ...)
2017-08-26 22:20 ` [Buildroot] [next v2 7/7] testing/tests/download: add test for git submodule Ricardo Martincoski
@ 2017-10-06 20:31 ` Arnout Vandecappelle
2017-10-23 2:31 ` Ricardo Martincoski
2017-10-29 14:05 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
7 siblings, 1 reply; 108+ messages in thread
From: Arnout Vandecappelle @ 2017-10-06 20:31 UTC (permalink / raw)
To: buildroot
I believe you might respin this series soonish, so time to review it :-)
BTW, when you do respin, could you push it to a gitlab clone, trigger the
build, and report the result in your cover letter?
On 27-08-17 00:20, Ricardo Martincoski wrote:
> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>
> Some test cases don't use a full build as setup, so split the build()
> method into configure() and build().
> It allows a test case to perform configuration at the setup stage and
> the build inside the test itself.
>
> Call this new method just before build in the BRTest base class, to keep
> the current behavior for existing test cases.
>
> This change will be needed when adding a common class to test the git
> download infra.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
It would be nice though to have an example of its use earlier in the series.
Regards,
Arnout
> ---
> Changes v1 -> v2:
> - new patch to adapt the test infra to test git download
> ---
> support/testing/infra/basetest.py | 1 +
> support/testing/infra/builder.py | 3 ++-
> 2 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
> index 431605b23f..d61f4d337b 100644
> --- a/support/testing/infra/basetest.py
> +++ b/support/testing/infra/basetest.py
> @@ -57,6 +57,7 @@ class BRTest(unittest.TestCase):
>
> if not self.b.is_finished():
> self.show_msg("Building")
> + self.b.configure()
> self.b.build()
> self.show_msg("Building done")
>
> diff --git a/support/testing/infra/builder.py b/support/testing/infra/builder.py
> index 905b127c91..e7c8e0102c 100644
> --- a/support/testing/infra/builder.py
> +++ b/support/testing/infra/builder.py
> @@ -10,7 +10,7 @@ class Builder(object):
> self.builddir = builddir
> self.logfile = infra.open_log_file(builddir, "build", logtofile)
>
> - def build(self):
> + def configure(self):
> if not os.path.isdir(self.builddir):
> os.makedirs(self.builddir)
>
> @@ -29,6 +29,7 @@ class Builder(object):
> if ret != 0:
> raise SystemError("Cannot olddefconfig")
>
> + def build(self):
> cmd = ["make", "-C", self.builddir]
> ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile)
> if ret != 0:
>
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 2/7] testing/infra/builder: build with target and environment
2017-08-26 22:20 ` [Buildroot] [next v2 2/7] testing/infra/builder: build with target and environment Ricardo Martincoski
@ 2017-10-06 20:42 ` Arnout Vandecappelle
2017-10-06 21:02 ` Arnout Vandecappelle
2017-10-23 2:34 ` Ricardo Martincoski
0 siblings, 2 replies; 108+ messages in thread
From: Arnout Vandecappelle @ 2017-10-06 20:42 UTC (permalink / raw)
To: buildroot
On 27-08-17 00:20, Ricardo Martincoski wrote:
> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>
> Make the builder able to call 'VAR1=1 make VAR2=2 target'.
>
> Allow to send extra parameters to be added to the end of make command
> line. It can be used for these purposes:
> - to configure a br2-external, by passing 'BR2_EXTERNAL="dir"';
> - to specify a make target, such as 'foo-source'.
>
> Allow to add variables to the environment in which make runs. It can be
> used for these purposes:
> - to override values from environment, such as 'BR2_DL_DIR="dl"';
> - to set variables not set in the .mk files (for testing purpose), as
> FOO_VERSION='1/1', triggering the preprocessing of values.
>
> This change will be needed when adding a common class to test the git
> download infra.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> ---
> Changes v1 -> v2:
> - new patch to adapt the test infra to test git download
> ---
> support/testing/infra/builder.py | 37 +++++++++++++++++++++++++++++++------
> 1 file changed, 31 insertions(+), 6 deletions(-)
>
> diff --git a/support/testing/infra/builder.py b/support/testing/infra/builder.py
> index e7c8e0102c..de7c141a76 100644
> --- a/support/testing/infra/builder.py
> +++ b/support/testing/infra/builder.py
> @@ -10,7 +10,12 @@ class Builder(object):
> self.builddir = builddir
> self.logfile = infra.open_log_file(builddir, "build", logtofile)
>
> - def configure(self):
> + # Configure the build
> + #
> + # makecmdline: a list of arguments to be passed to the make command.
> + # e.g. makecmdline=["BR2_EXTERNAL=/path"]
> + #
Better to do this as a docstring rather than a comment. And use the docstring
formatting as well.
> + def configure(self, makecmdline=None):
Maybe makecmdline -> make_extra_opts ?
For symmetry, I'd add both make_extra_opts and make_extra_env here.
> if not os.path.isdir(self.builddir):
> os.makedirs(self.builddir)
>
> @@ -22,16 +27,36 @@ class Builder(object):
> "> end defconfig\n")
> self.logfile.flush()
>
> - cmd = ["make",
> - "O={}".format(self.builddir),
> - "olddefconfig"]
> + cmd = ["make", "O={}".format(self.builddir)]
> + if makecmdline:
> + cmd += makecmdline
This allows only a single options to be passed. I would require make_extra_opts
to be an iterable and do cmd.extend(make_extra_opts) here.
Also, if you default it to () instead of None, you don't need the condition.
> + cmd += ["olddefconfig"]
> +
> ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile)
> if ret != 0:
> raise SystemError("Cannot olddefconfig")
>
> - def build(self):
> + # Perform the build
> + #
> + # makecmdline: a list of arguments to be passed to the make command. It can
> + # include a make target.
> + # e.g. makecmdline=["foo-source"]
> + #
> + # env: a dict of variables to be appended (or replaced) in the environment
> + # that calls make.
> + # e.g. env={"BR2_DL_DIR": "/path"}
> + #
> + def build(self, makecmdline=None, env=None):
env -> make_extra_env
> + buildenv = os.environ.copy()
> + if env:
> + buildenv.update(env)
> +
Here you could default make_extra_env to {} and do:
env = os.environ.copy()
env.update(make_extra_env)
However, do we really want to keep the environment? Perhaps it's better to
start with an empty environment anyway...
Regards,
Arnout
> cmd = ["make", "-C", self.builddir]
> - ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile)
> + if makecmdline:
> + cmd += makecmdline
> +
> + ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
> + env=buildenv)
> if ret != 0:
> raise SystemError("Build failed")
>
>
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 3/7] testing/infra/builder: allow to override logfile
2017-08-26 22:20 ` [Buildroot] [next v2 3/7] testing/infra/builder: allow to override logfile Ricardo Martincoski
@ 2017-10-06 20:50 ` Arnout Vandecappelle
2017-10-23 2:32 ` Ricardo Martincoski
0 siblings, 1 reply; 108+ messages in thread
From: Arnout Vandecappelle @ 2017-10-06 20:50 UTC (permalink / raw)
To: buildroot
On 27-08-17 00:20, Ricardo Martincoski wrote:
> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>
> Some test cases can use only the configure step as setup, while the
> build is part of the test itself.
> Allow the caller to build() to override the logfile used. This way all
> the log for the actual test can be directed to *-run.log.
This feels weird to me. If you need this kind of argument, then there is
something wrong with the logfile member of Builder.
Your explanation here seems to suggest that the only reason to have this is to
split the -build and the -run logs so the -run logs contains something useful
for the git tests. I don't think that that's a good enough reason. What's wrong
with collecting all the output in the -build log?
But looking at later patches, it looks like you're actually trying to solve a
problem here that a file is opened from two places and they run over each other.
If that is the case, I think a better solution needs to be found to handle the
log files. Could be as simple as the log file being maintained by the test class
itself, and passed as an argument to the Builder functions every time.
Regards,
Arnout
>
> This change will be needed to make nice logs for git downloader test
> cases.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> ---
> Changes v1 -> v2:
> - new patch to adapt the test infra to test git download
> ---
> support/testing/infra/builder.py | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/support/testing/infra/builder.py b/support/testing/infra/builder.py
> index de7c141a76..d634c211ac 100644
> --- a/support/testing/infra/builder.py
> +++ b/support/testing/infra/builder.py
> @@ -46,7 +46,13 @@ class Builder(object):
> # that calls make.
> # e.g. env={"BR2_DL_DIR": "/path"}
> #
> - def build(self, makecmdline=None, env=None):
> + # ovrdlogfile: when the build is part of the actual test case instead of
> + # the setup, this parameter can be used to override the logfile used.
> + # e.g. ovrdlogfile=handler_for_the_run_log
> + #
> + def build(self, makecmdline=None, env=None, ovrdlogfile=None):
> + logfile = ovrdlogfile or self.logfile
> +
> buildenv = os.environ.copy()
> if env:
> buildenv.update(env)
> @@ -55,7 +61,7 @@ class Builder(object):
> if makecmdline:
> cmd += makecmdline
>
> - ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
> + ret = subprocess.call(cmd, stdout=logfile, stderr=logfile,
> env=buildenv)
> if ret != 0:
> raise SystemError("Build failed")
>
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 2/7] testing/infra/builder: build with target and environment
2017-10-06 20:42 ` Arnout Vandecappelle
@ 2017-10-06 21:02 ` Arnout Vandecappelle
2017-10-23 2:34 ` Ricardo Martincoski
1 sibling, 0 replies; 108+ messages in thread
From: Arnout Vandecappelle @ 2017-10-06 21:02 UTC (permalink / raw)
To: buildroot
On 06-10-17 22:42, Arnout Vandecappelle wrote:
>
> On 27-08-17 00:20, Ricardo Martincoski wrote:
>> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
[snip]
>> + cmd = ["make", "O={}".format(self.builddir)]
>> + if makecmdline:
>> + cmd += makecmdline
> This allows only a single options to be passed. I would require make_extra_opts
> to be an iterable and do cmd.extend(make_extra_opts) here.
Sorry, += is the same as extend, of course...
Regards,
Arnout
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 4/7] testing/tests/download: add infra for git tests
2017-08-26 22:20 ` [Buildroot] [next v2 4/7] testing/tests/download: add infra for git tests Ricardo Martincoski
@ 2017-10-06 21:30 ` Arnout Vandecappelle
2017-10-23 2:35 ` Ricardo Martincoski
0 siblings, 1 reply; 108+ messages in thread
From: Arnout Vandecappelle @ 2017-10-06 21:30 UTC (permalink / raw)
To: buildroot
I would personally have split the patches differently: include a very minimal
GitTestBase in the patch that adds the hash tests, then extend it with features
needed for the later tests.
On 27-08-17 00:20, Ricardo Martincoski wrote:
> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>
> Add GitRemote class, that can start a git server in the host machine to
> emulate a remote git server under the control of the test.
>
> Add a new base class, called GitTestBase, that inherits from BRTest and
> must be subclassed by all git test cases.
> Its setUp() method takes care of configuring the build with a
> br2-external, avoiding to hit http://sources.buildroot.net by using an
> empty BR2_BACKUP_SITE. It also avoids downloading not pre-installed
> dependencies (i.e. lzip) every time by calling 'make dependencies' using
> the common dl directory, and it instantiates the GitRemote object.
> Its tearDown() method cleans things up similarly to the one from BRTest.
>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> ---
> Changes v1 -> v2:
> - use git daemon + git:// instead of ssh (Arnout);
> - remove __main__ handling since the test infra already does that;
> - use the logging support from test infra;
> - this patch is part of series 1/3 of a new version of
> http://patchwork.ozlabs.org/patch/690097/
> ---
> support/testing/tests/download/__init__.py | 0
These empty __init__.py files were removed at some point.
> support/testing/tests/download/gitbase.py | 41 +++++++++++++++++++++++
> support/testing/tests/download/gitremote.py | 50 +++++++++++++++++++++++++++++
> 3 files changed, 91 insertions(+)
> create mode 100644 support/testing/tests/download/__init__.py
> create mode 100644 support/testing/tests/download/gitbase.py
> create mode 100644 support/testing/tests/download/gitremote.py
>
> diff --git a/support/testing/tests/download/__init__.py b/support/testing/tests/download/__init__.py
> new file mode 100644
> index 0000000000..e69de29bb2
> diff --git a/support/testing/tests/download/gitbase.py b/support/testing/tests/download/gitbase.py
> new file mode 100644
> index 0000000000..b259914c77
> --- /dev/null
> +++ b/support/testing/tests/download/gitbase.py
> @@ -0,0 +1,41 @@
> +import infra.basetest
> +from infra.builder import Builder
> +from gitremote import GitRemote
> +
> +
> +class GitTestBase(infra.basetest.BRTest):
> + config = \
> + """
> + BR2_BACKUP_SITE=""
> + """
> + br2_external = None
> + serveddir = None
> + gitremote = None
> + logfile = None
> +
> + def setUp(self):
> + self.show_msg("Starting")
> + self.b = Builder(self.config, self.builddir, self.logtofile)
> +
> + if not self.keepbuilds:
> + self.b.delete()
> +
> + if not self.b.is_finished():
> + self.show_msg("Configuring")> + self.b.configure(["BR2_EXTERNAL={}".format(self.br2_external)])
I'm not at all happy with this, too much copying of the logic of BRTest. How
about this:
- configure extra options are a member (like self.config is) and gets picked up
by the base class.
- Base class stops after configure.
- The rest of BRTest goes into a new class RuntimeTestBase, which does the build
step in setUp and starts the emulator.
- RuntimeTestBase and GitTestBase call super().setUp(self) to extend the setUp
method. (Or however you do super() in Python2, it's different IIRC.)
Note BTW that specifically BR2_EXTERNAL is typically passed in the environment,
not as an option. But it doesn't matter much.
Perhaps we could even put the support for BR2_EXTERNAL directly in BRTest. It
would then of course have to depend on self.br2_external is not None.
> + # do not download not pre-installed dependencies (i.e. lzip) every
> + # time a test runs
> + self.b.build(["dependencies"])
Since we have only one test per class, this is anyway run every time, so
there's not much point I think.
> +
> + self.gitremote = GitRemote(self.builddir, self.serveddir,
> + self.logtofile)
> + # send output from the test to the logfile created by GitRemote
> + self.logfile = self.gitremote.logfile
> + self.gitremote.start()
Emulator starts directly from its __init__, perhaps the same should be done here?
> +
> + def tearDown(self):
> + self.show_msg("Cleaning up")
> + if self.gitremote:
> + self.gitremote.stop()
> + if self.b and not self.keepbuilds:
> + self.b.delete()
Same here with super().tearDown(self) at the end.
> diff --git a/support/testing/tests/download/gitremote.py b/support/testing/tests/download/gitremote.py
> new file mode 100644
> index 0000000000..84d1036889
> --- /dev/null
> +++ b/support/testing/tests/download/gitremote.py
> @@ -0,0 +1,50 @@
> +# subprocess does not kill the child daemon when a test case fails by raising
> +# an exception. So use pexpect instead.
> +import pexpect
> +
> +import infra
> +
> +GIT_REMOTE_PORT_INITIAL = 9418
> +GIT_REMOTE_PORT_LAST = GIT_REMOTE_PORT_INITIAL + 99
> +
> +
> +class GitRemote(object):
> +
> + def __init__(self, builddir, serveddir, logtofile):
> + self.daemon = None
> + self.port = None
> + self.serveddir = serveddir
> + self.logfile = infra.open_log_file(builddir, "run", logtofile)
> +
> + # Start a local git server
> + #
> + # In order to support test cases in parallel, select the port the server
> + # will listen to in runtime. Since there is no reliable way to allocate the
> + # port prior to starting the server (another process in the host machine
> + # can use the port between it is selected from a list and it is really
> + # allocated to the server) try to start the server in a port and in the
> + # case it is already in use, try the next one in the allowed range.
> + #
> + def start(self):
> + daemon_cmd = ["git", "daemon", "--reuseaddr", "--verbose",
> + "--listen=localhost", "--export-all",
> + "--base-path={}".format(self.serveddir)]
> + for port in range(GIT_REMOTE_PORT_INITIAL, GIT_REMOTE_PORT_LAST):
> + port_arg = ["--port={}".format(port)]
> + cmd = daemon_cmd + port_arg
> + self.logfile.write("> starting git remote with '{}'\n"
> + .format(" ".join(cmd)))
> + self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile)
> + ret = self.daemon.expect(["Ready to rumble",
> + "Address already in use"])
> + if ret == 0:
> + self.port = port
> + break
> + self.daemon.read()
Maybe add a comment that explains why this is needed?
Regards,
Arnout
> + if not self.port:
> + raise SystemError("Could not find a free port to run git remote")
> +
> + def stop(self):
> + if self.daemon is None:
> + return
> + self.daemon.terminate(force=True)
>
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 5/7] testing/tests/download: add git hash tests
2017-08-26 22:20 ` [Buildroot] [next v2 5/7] testing/tests/download: add git hash tests Ricardo Martincoski
2017-08-26 22:38 ` Ricardo Martincoski
@ 2017-10-06 21:36 ` Arnout Vandecappelle
2017-10-06 21:44 ` Arnout Vandecappelle
2 siblings, 0 replies; 108+ messages in thread
From: Arnout Vandecappelle @ 2017-10-06 21:36 UTC (permalink / raw)
To: buildroot
On 27-08-17 00:20, Ricardo Martincoski wrote:
> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>
> Add three test cases for hash of a git package:
> - correct hash;
> - wrong hash;
> - no hash file.
>
> Besides the Python script, add some fixtures used during the tests:
> - a br2-external (git-hash) with one package for each test case;
> - a static git bare repo (repo.git) to be served using GitRemote class
> for all these test cases.
>
> Neither the br2-external nor the check hash functionalities are the
> subject of these tests per se, so for simplicity limit the check to the
> error codes and don't look for the messages in the log.
>
> Thanks to Arnout for the hint about how to add a bare repo to test.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Regards,
Arnout
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 5/7] testing/tests/download: add git hash tests
2017-08-26 22:20 ` [Buildroot] [next v2 5/7] testing/tests/download: add git hash tests Ricardo Martincoski
2017-08-26 22:38 ` Ricardo Martincoski
2017-10-06 21:36 ` Arnout Vandecappelle
@ 2017-10-06 21:44 ` Arnout Vandecappelle
2017-10-23 2:36 ` Ricardo Martincoski
2 siblings, 1 reply; 108+ messages in thread
From: Arnout Vandecappelle @ 2017-10-06 21:44 UTC (permalink / raw)
To: buildroot
Darn, I was too quick with my Rev-by tag...
On 27-08-17 00:20, Ricardo Martincoski wrote:
[snip]
> diff --git a/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
> new file mode 100644
> index 0000000000..2ad06fa47f
> --- /dev/null
> +++ b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
> @@ -0,0 +1,12 @@
> +################################################################################
> +#
> +# bad
> +#
> +################################################################################
> +
> +BAD_VERSION = 7446f8dab6cdb166aaf130aa88633dc627a4e8ad
> +# Get the git server port number from the test infra
> +GITREMOTE_PORT_NUMBER ?= 9418
This is not OK, since the same variable is defined in multiple places. You can
just put it in the external.mk instead.
Regards,
Arnout
> +BAD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
> +
> +$(eval $(generic-package))
[snip]
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 6/7] testing/tests/download: add test for sha1 as git ref
2017-08-26 22:20 ` [Buildroot] [next v2 6/7] testing/tests/download: add test for sha1 as git ref Ricardo Martincoski
@ 2017-10-06 21:57 ` Arnout Vandecappelle
2017-10-23 2:38 ` Ricardo Martincoski
0 siblings, 1 reply; 108+ messages in thread
From: Arnout Vandecappelle @ 2017-10-06 21:57 UTC (permalink / raw)
To: buildroot
On 27-08-17 00:20, Ricardo Martincoski wrote:
> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>
> Add a test to download the package with git method and its version set
> to a sha1 reachable by a branch name, but not pointed by it.
> This is the most common use case for git refs in the tree.
>
> Besides the test case Python script, add:
> - a br2-external (git-refs) with a fake package (foo) as a fixture for
> this and upcoming tests;
I'd use a more descriptive name than foo, e.g. git-refs.mk
> - support in GitTestBase for repos created on the fly;
> - a new class GitRepo to create and manipulate repos. It is a class and
> not a simple module because it will become very useful when testing
> submodules.
So why do you suddenly need to create repos on the fly now? I still believe it
is much simpler to include the repo directly. I also think it's easier if the
same repo is used for all git tests - which should be possible, I think.
Regards,
Arnout
> After the download, compare the tarball to a checkout of the original
> remote repo.
>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build()
2017-10-06 20:31 ` [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build() Arnout Vandecappelle
@ 2017-10-23 2:31 ` Ricardo Martincoski
0 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-23 2:31 UTC (permalink / raw)
To: buildroot
Hello,
On Fri, Oct 06, 2017 at 05:31 PM, Arnout Vandecappelle wrote:
> from: Arnout Vandecappelle <arnout@mind.be>
> date: Fri, Oct 06 10:31 PM +02:00 2017
> to: Ricardo Martincoski <ricardo.martincoski@gmail.com>, buildroot at buildroot.org
> cc: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> subject: Re: [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build()
>
> I believe you might respin this series soonish, so time to review it :-)
Thank you very much for reviewing the series.
Sorry for the long delay.
>
> BTW, when you do respin, could you push it to a gitlab clone, trigger the
> build, and report the result in your cover letter?
Sure. I will do.
>
> On 27-08-17 00:20, Ricardo Martincoski wrote:
[snip]
> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The only reason I won't keep it is because the rebase is not trivial (automatic)
after I add the new patch "call make with empty env" to the beginning of the
series.
> It would be nice though to have an example of its use earlier in the series.
It will have a use (but not a complete example) when I add the new patch "split
runtime test from BRTest"
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 3/7] testing/infra/builder: allow to override logfile
2017-10-06 20:50 ` Arnout Vandecappelle
@ 2017-10-23 2:32 ` Ricardo Martincoski
0 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-23 2:32 UTC (permalink / raw)
To: buildroot
Hello,
On Fri, Oct 06, 2017 at 05:50 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
> On 27-08-17 00:20, Ricardo Martincoski wrote:
>> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>>
>> Some test cases can use only the configure step as setup, while the
>> build is part of the test itself.
>> Allow the caller to build() to override the logfile used. This way all
>> the log for the actual test can be directed to *-run.log.
>
> This feels weird to me. If you need this kind of argument, then there is
Rethinking now, after your comments, this patch is hackish.
> something wrong with the logfile member of Builder.
>
> Your explanation here seems to suggest that the only reason to have this is to
> split the -build and the -run logs so the -run logs contains something useful
> for the git tests. I don't think that that's a good enough reason. What's wrong
> with collecting all the output in the -build log?
Nothing wrong. I will remove this patch from the series.
>
> But looking at later patches, it looks like you're actually trying to solve a
> problem here that a file is opened from two places and they run over each other.
> If that is the case, I think a better solution needs to be found to handle the
> log files. Could be as simple as the log file being maintained by the test class
> itself, and passed as an argument to the Builder functions every time.
That's not the case. I was only trying to get all the "setup" output to
-build.log and all the "run" output to -run.log. Silly and not needed.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 2/7] testing/infra/builder: build with target and environment
2017-10-06 20:42 ` Arnout Vandecappelle
2017-10-06 21:02 ` Arnout Vandecappelle
@ 2017-10-23 2:34 ` Ricardo Martincoski
1 sibling, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-23 2:34 UTC (permalink / raw)
To: buildroot
Hello,
On Fri, Oct 06, 2017 at 05:42 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
> On 27-08-17 00:20, Ricardo Martincoski wrote:
[snip]
>> - def configure(self):
>> + # Configure the build
>> + #
>> + # makecmdline: a list of arguments to be passed to the make command.
>> + # e.g. makecmdline=["BR2_EXTERNAL=/path"]
>> + #
>
> Better to do this as a docstring rather than a comment. And use the docstring
> formatting as well.
OK.
>
>> + def configure(self, makecmdline=None):
>
> Maybe makecmdline -> make_extra_opts ?
Yes. It's better.
>
> For symmetry, I'd add both make_extra_opts and make_extra_env here.
OK.
>
>> if not os.path.isdir(self.builddir):
>> os.makedirs(self.builddir)
>>
>> @@ -22,16 +27,36 @@ class Builder(object):
>> "> end defconfig\n")
>> self.logfile.flush()
>>
>> - cmd = ["make",
>> - "O={}".format(self.builddir),
>> - "olddefconfig"]
>> + cmd = ["make", "O={}".format(self.builddir)]
>> + if makecmdline:
>> + cmd += makecmdline
>
> This allows only a single options to be passed. I would require make_extra_opts
> to be an iterable and do cmd.extend(make_extra_opts) here.
>
> Also, if you default it to () instead of None, you don't need the condition.
I will default it to [].
>
>> + cmd += ["olddefconfig"]
>> +
>> ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile)
>> if ret != 0:
>> raise SystemError("Cannot olddefconfig")
>>
>> - def build(self):
>> + # Perform the build
>> + #
>> + # makecmdline: a list of arguments to be passed to the make command. It can
>> + # include a make target.
>> + # e.g. makecmdline=["foo-source"]
>> + #
>> + # env: a dict of variables to be appended (or replaced) in the environment
>> + # that calls make.
>> + # e.g. env={"BR2_DL_DIR": "/path"}
>> + #
>> + def build(self, makecmdline=None, env=None):
>
> env -> make_extra_env
OK.
It also allows me to rename buildenv -> env as you did below.
>
>> + buildenv = os.environ.copy()
>> + if env:
>> + buildenv.update(env)
>> +
>
> Here you could default make_extra_env to {} and do:
>
> env = os.environ.copy()
> env.update(make_extra_env)
Indeed. As long we never change this parameter inside the function we are safe
and the parameter will always point to the same empty dict, NOT falling in
following scenario:
>>> def f1(a, b={}):
... b[a]=1
... print b
...
>>> f1(1)
{1: 1}
>>> f1(2)
{1: 1, 2: 1}
>
> However, do we really want to keep the environment? Perhaps it's better to
> start with an empty environment anyway...
A completely empty environment won't work because
subprocess.call(..., env={})
is equivalent to calling 'env -i make' in the buildroot directory, it lacks the
path to host libraries, for instance.
But we can import only PATH from the environment. I will add a separate patch
in the beginning of the series to deal with this while keeping the current -d
precedence over BR2_DL_DIR.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 4/7] testing/tests/download: add infra for git tests
2017-10-06 21:30 ` Arnout Vandecappelle
@ 2017-10-23 2:35 ` Ricardo Martincoski
2017-10-23 8:18 ` Arnout Vandecappelle
0 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-23 2:35 UTC (permalink / raw)
To: buildroot
Hello,
On Fri, Oct 06, 2017 at 06:30 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
> I would personally have split the patches differently: include a very minimal
> GitTestBase in the patch that adds the hash tests, then extend it with features
> needed for the later tests.
Good idea. I will do that.
>
> On 27-08-17 00:20, Ricardo Martincoski wrote:
[snip]
>> ---
>> support/testing/tests/download/__init__.py | 0
>
> These empty __init__.py files were removed at some point.
Sorry. I did not understand what you meant.
>
>> support/testing/tests/download/gitbase.py | 41 +++++++++++++++++++++++
>> support/testing/tests/download/gitremote.py | 50 +++++++++++++++++++++++++++++
>> 3 files changed, 91 insertions(+)
>> create mode 100644 support/testing/tests/download/__init__.py
>> create mode 100644 support/testing/tests/download/gitbase.py
>> create mode 100644 support/testing/tests/download/gitremote.py
>>
[snip]
>> + def setUp(self):
>> + self.show_msg("Starting")
>> + self.b = Builder(self.config, self.builddir, self.logtofile)
>> +
>> + if not self.keepbuilds:
>> + self.b.delete()
>> +
>> + if not self.b.is_finished():
>> + self.show_msg("Configuring")> + self.b.configure(["BR2_EXTERNAL={}".format(self.br2_external)])
>
> I'm not at all happy with this, too much copying of the logic of BRTest. How
> about this:
>
> - configure extra options are a member (like self.config is) and gets picked up
> by the base class.
I will put the support for BR2_EXTERNAL directly in BRTest instead.
> - Base class stops after configure.
>
> - The rest of BRTest goes into a new class RuntimeTestBase, which does the build
> step in setUp and starts the emulator.
I will create a patch for these 2.
> - RuntimeTestBase and GitTestBase call super().setUp(self) to extend the setUp
> method. (Or however you do super() in Python2, it's different IIRC.)
OK. Much better this way avoiding code duplication.
>
> Note BTW that specifically BR2_EXTERNAL is typically passed in the environment,
> not as an option. But it doesn't matter much.
I didn't know that! In my everyday use I always passed a make variable.
The example in the manual does the same so I will keep it.
>
>
> Perhaps we could even put the support for BR2_EXTERNAL directly in BRTest. It
> would then of course have to depend on self.br2_external is not None.
I will create a patch for this.
>
>> + # do not download not pre-installed dependencies (i.e. lzip) every
>> + # time a test runs
>> + self.b.build(["dependencies"])
>
> Since we have only one test per class, this is anyway run every time, so
> there's not much point I think.
This code matters when running the tests in a local machine.
I will add this info to the comment.
>
>> +
>> + self.gitremote = GitRemote(self.builddir, self.serveddir,
>> + self.logtofile)
>> + # send output from the test to the logfile created by GitRemote
>> + self.logfile = self.gitremote.logfile
>> + self.gitremote.start()
>
> Emulator starts directly from its __init__, perhaps the same should be done here?
Er... I don't see such code on Emulator.
But anyway... I will move to __init__. Few lines of code can removed this way.
>
>> +
>> + def tearDown(self):
>> + self.show_msg("Cleaning up")
>> + if self.gitremote:
>> + self.gitremote.stop()
>> + if self.b and not self.keepbuilds:
>> + self.b.delete()
>
> Same here with super().tearDown(self) at the end.
OK.
[snip]
>> + self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile)
>> + ret = self.daemon.expect(["Ready to rumble",
>> + "Address already in use"])
>> + if ret == 0:
>> + self.port = port
>> + break
>> + self.daemon.read()
>
> Maybe add a comment that explains why this is needed?
I will remove it, it will not be needed.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 5/7] testing/tests/download: add git hash tests
2017-10-06 21:44 ` Arnout Vandecappelle
@ 2017-10-23 2:36 ` Ricardo Martincoski
0 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-23 2:36 UTC (permalink / raw)
To: buildroot
Hello,
On Fri, Oct 06, 2017 at 06:44 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
> Darn, I was too quick with my Rev-by tag...
OK, I will remove it.
> On 27-08-17 00:20, Ricardo Martincoski wrote:
> [snip]
>> diff --git a/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
>> new file mode 100644
>> index 0000000000..2ad06fa47f
>> --- /dev/null
>> +++ b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
>> @@ -0,0 +1,12 @@
>> +################################################################################
>> +#
>> +# bad
>> +#
>> +################################################################################
>> +
>> +BAD_VERSION = 7446f8dab6cdb166aaf130aa88633dc627a4e8ad
>> +# Get the git server port number from the test infra
>> +GITREMOTE_PORT_NUMBER ?= 9418
>
> This is not OK, since the same variable is defined in multiple places. You can
> just put it in the external.mk instead.
OK.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 6/7] testing/tests/download: add test for sha1 as git ref
2017-10-06 21:57 ` Arnout Vandecappelle
@ 2017-10-23 2:38 ` Ricardo Martincoski
0 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-23 2:38 UTC (permalink / raw)
To: buildroot
Hello,
On Fri, Oct 06, 2017 at 06:57 PM, Arnout Vandecappelle wrote:
> On 27-08-17 00:20, Ricardo Martincoski wrote:
>> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>>
>> Add a test to download the package with git method and its version set
>> to a sha1 reachable by a branch name, but not pointed by it.
>> This is the most common use case for git refs in the tree.
>>
>> Besides the test case Python script, add:
>> - a br2-external (git-refs) with a fake package (foo) as a fixture for
>> this and upcoming tests;
>
> I'd use a more descriptive name than foo, e.g. git-refs.mk
OK. In order to use a static repo I will create one package for each ref type to
check so I can have hashes for each tarball so I don't need to manually open the
tarball to know it is OK. But I will keep the tip for "more descriptive name" in
mind while creating the packages.
>
>> - support in GitTestBase for repos created on the fly;
>> - a new class GitRepo to create and manipulate repos. It is a class and
>> not a simple module because it will become very useful when testing
>> submodules.
>
> So why do you suddenly need to create repos on the fly now? I still believe it
> is much simpler to include the repo directly. I also think it's easier if the
> same repo is used for all git tests - which should be possible, I think.
Fair enough. I will switch to a single static repo in the next spin.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 4/7] testing/tests/download: add infra for git tests
2017-10-23 2:35 ` Ricardo Martincoski
@ 2017-10-23 8:18 ` Arnout Vandecappelle
2017-10-29 4:00 ` Ricardo Martincoski
0 siblings, 1 reply; 108+ messages in thread
From: Arnout Vandecappelle @ 2017-10-23 8:18 UTC (permalink / raw)
To: buildroot
On 23-10-17 04:35, Ricardo Martincoski wrote:
> Hello,
>
> On Fri, Oct 06, 2017 at 06:30 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
>> On 27-08-17 00:20, Ricardo Martincoski wrote:
> [snip]
>>> ---
>>> support/testing/tests/download/__init__.py | 0
>>
>> These empty __init__.py files were removed at some point.
>
> Sorry. I did not understand what you meant.
My mistake, I was confusing with another project where we used to have empty
__init__.py files everywhere but we removed them again because they turned out
not to be needed. But in that project we use dynamic loading with
importlib.import_module, that's why they could be removed.
[snip]
>>> +
>>> + self.gitremote = GitRemote(self.builddir, self.serveddir,
>>> + self.logtofile)
>>> + # send output from the test to the logfile created by GitRemote
>>> + self.logfile = self.gitremote.logfile
>>> + self.gitremote.start()
>>
>> Emulator starts directly from its __init__, perhaps the same should be done here?
>
> Er... I don't see such code on Emulator.
Er, I don't know what I was thinking... I must have been tired when writing
this mail :-)
> But anyway... I will move to __init__. Few lines of code can removed this way.
Maybe that is not such a good idea after all... I think __init__ is still
called when you do run-tests -l, but we don't want the git server to be started
in that case.
Regards,
Arnout
[snip]
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [next v2 4/7] testing/tests/download: add infra for git tests
2017-10-23 8:18 ` Arnout Vandecappelle
@ 2017-10-29 4:00 ` Ricardo Martincoski
0 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-29 4:00 UTC (permalink / raw)
To: buildroot
Hello,
On Mon, Oct 23, 2017 at 06:18 AM, Arnout Vandecappelle wrote:
> On 23-10-17 04:35, Ricardo Martincoski wrote:
>> On Fri, Oct 06, 2017 at 06:30 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
>>> On 27-08-17 00:20, Ricardo Martincoski wrote:
[snip]
>>>> + self.gitremote = GitRemote(self.builddir, self.serveddir,
>>>> + self.logtofile)
>>>> + # send output from the test to the logfile created by GitRemote
>>>> + self.logfile = self.gitremote.logfile
>>>> + self.gitremote.start()
>>>
>>> Emulator starts directly from its __init__, perhaps the same should be done here?
>>
>> Er... I don't see such code on Emulator.
>
> Er, I don't know what I was thinking... I must have been tired when writing
> this mail :-)
>
>> But anyway... I will move to __init__. Few lines of code can removed this way.
>
> Maybe that is not such a good idea after all... I think __init__ is still
> called when you do run-tests -l, but we don't want the git server to be started
> in that case.
>
Indeed __init__ is called when running run-tests -l for all classes that
inherits from unittest.TestCase, but since in this case GitRemote is
instantiated from GitTestBase.setUp it is safe to start the server in
GitRemote.__init__ so I will do that.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3)
2017-08-26 22:20 ` [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build() Ricardo Martincoski
` (6 preceding siblings ...)
2017-10-06 20:31 ` [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build() Arnout Vandecappelle
@ 2017-10-29 14:05 ` Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 1/9] testing/infra/builder: call make with empty env Ricardo Martincoski
` (10 more replies)
7 siblings, 11 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-29 14:05 UTC (permalink / raw)
To: buildroot
Hello,
This is a new iteration of the series that first adds automated tests for the
git download infra and then rewrites the script to use git fetch.
The first 5 patches prepare the test infra to have the git test cases.
Patch 6 adds a test case that ensures hash checking is enabled for git packages.
The last 3 patches add a test case that checks each type of git ref can be used
as version of a git package.
Changes v2 -> v3:
- remove patch not needed "allow to override logfile";
- add support for br2-external to the base test class;
- call make from test infra with empty env;
- split the runtime test from the base test class;
- use static repo to test git download;
- use a single test case for git hash and a single test case for git refs
because both do not take long to run and each do test a feature that needs
all checks to pass to be considered OK;
Changes v1 -> v2:
- use test infra for the tests;
- split the patch of the download script;
- remove patch "log checked out sha1" as it is not needed when using the test
infra (maybe it can be resent later as an improvement);
The detailed changelog is part of each patch.
Here I include an overview of the series history:
v1 had these patches:
support/download/git: log checked out sha1
test/support/download/git: new test
support/download/git: do not use git clone
For the next iterations I split the series in 3:
- s1/3: tests 1/2 (test infra and 2 test cases)
- s2/3: tests 2/2 (rest of the test cases)
- s3/3: changes to the git download infra
s3/3 v2 - the test infra was not merged yet at the time of sending, so I did not
include the tests:
support/download/git: do not use git clone
support/download/git: optimized download of sha1
DEVELOPERS: add entry for support/download/git
[RFC] support/download/git: shallow fetch of all branches
s1/3 v2 - rewrite using the test infra:
testing/infra/builder: split configure() from build()
testing/infra/builder: build with target and environment
testing/infra/builder: allow to override logfile
testing/tests/download: add infra for git tests
testing/tests/download: add git hash tests
testing/tests/download: add test for sha1 as git ref
testing/tests/download: add test for git submodule
CURRENT s1/3 v3 - rewrite using static repo:
testing/infra/builder: call make with empty env
testing/infra/builder: split configure() from build()
testing/infra/builder: build with target and environment
testing/infra: split runtime test from BRTest
testing/infra/basetest: support br2-external
testing/tests/download: add git hash test
testing/tests/download: test case for git refs
testing/tests/download: test git branch
testing/tests/download: test git submodules
You can see a run in this gitlab url:
https://gitlab.com/RicardoMartincoski/buildroot/pipelines/13324257
upcoming s2/3 v3 - rewrite using the test infra and a static repo:
testing/tests/download: test git tag
testing/tests/download: test git tag/branch precedence
testing/tests/download: test git special ref
testing/tests/download: test git branch with slash
upcoming s3/3 v3 - I know this series or 'git cache' will need to rebase in the
top of the other:
DEVELOPERS: add entry for support/download/git
support/download/git: do not use git clone
testing/tests/download: test sha1 of special ref
support/download/git: optimized download of sha1
support/download/git: shallow fetch of all branches
Regards,
Ricardo
Ricardo Martincoski (9):
testing/infra/builder: call make with empty env
testing/infra/builder: split configure() from build()
testing/infra/builder: build with target and environment
testing/infra: split runtime test from BRTest
testing/infra/basetest: support br2-external
testing/tests/download: add git hash test
testing/tests/download: test case for git refs
testing/tests/download: test git branch
testing/tests/download: test git submodules
.gitlab-ci.yml | 2 +
support/testing/infra/basetest.py | 16 +++---
support/testing/infra/builder.py | 45 +++++++++++++++--
support/testing/infra/runtimetest.py | 23 +++++++++
support/testing/run-tests | 1 -
support/testing/tests/core/test_post_scripts.py | 3 +-
support/testing/tests/core/test_rootfs_overlay.py | 3 +-
support/testing/tests/core/test_timezone.py | 7 +--
support/testing/tests/download/__init__.py | 0
.../tests/download/br2-external/git-hash/Config.in | 0
.../download/br2-external/git-hash/external.desc | 1 +
.../download/br2-external/git-hash/external.mk | 4 ++
.../br2-external/git-hash/package/bad/bad.hash | 1 +
.../br2-external/git-hash/package/bad/bad.mk | 10 ++++
.../br2-external/git-hash/package/good/good.hash | 1 +
.../br2-external/git-hash/package/good/good.mk | 10 ++++
.../br2-external/git-hash/package/nohash/nohash.mk | 10 ++++
.../tests/download/br2-external/git-refs/Config.in | 0
.../download/br2-external/git-refs/external.desc | 1 +
.../download/br2-external/git-refs/external.mk | 4 ++
.../git-refs/package/git-branch/git-branch.hash | 2 +
.../git-refs/package/git-branch/git-branch.mk | 11 +++++
.../git-partial-sha1-branch-head.hash | 2 +
.../git-partial-sha1-branch-head.mk | 11 +++++
.../git-partial-sha1-reachable-by-branch.hash | 2 +
.../git-partial-sha1-reachable-by-branch.mk | 11 +++++
.../git-sha1-branch-head/git-sha1-branch-head.hash | 2 +
.../git-sha1-branch-head/git-sha1-branch-head.mk | 11 +++++
.../git-sha1-reachable-by-branch.hash | 2 +
.../git-sha1-reachable-by-branch.mk | 11 +++++
.../git-submodule-disabled.hash | 2 +
.../git-submodule-disabled.mk | 11 +++++
.../git-submodule-enabled.hash | 3 ++
.../git-submodule-enabled/git-submodule-enabled.mk | 28 +++++++++++
.../git-wrong-content/git-wrong-content.hash | 2 +
.../package/git-wrong-content/git-wrong-content.mk | 11 +++++
.../package/git-wrong-sha1/git-wrong-sha1.mk | 11 +++++
.../git-remote/refs-sub1.git/.gitattributes | 1 +
.../tests/download/git-remote/refs-sub1.git/HEAD | 1 +
.../tests/download/git-remote/refs-sub1.git/config | 4 ++
.../07/cabc655213bdf7087d8dd50fda95124e935570 | Bin 0 -> 22 bytes
.../1d/f823cb8a6d1866148ae50a8009762a9c4c777f | Bin 0 -> 180 bytes
.../7d/52f458bdb0f9f5a4beb46fa82824421b8b988e | Bin 0 -> 65 bytes
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a | Bin 0 -> 122 bytes
.../bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23 | Bin 0 -> 22 bytes
.../cb/545facf77bbc5f24f95b6d503c338d10b7b717 | Bin 0 -> 151 bytes
.../dd/130f6f4629514adaf2e03407f3ed9344eb6cd6 | Bin 0 -> 122 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../f8/001e5780100962a5e30a25cbc4c9609cfc7bf3 | Bin 0 -> 184 bytes
.../git-remote/refs-sub1.git/refs/heads/submodule | 1 +
.../git-remote/refs-sub2.git/.gitattributes | 1 +
.../tests/download/git-remote/refs-sub2.git/HEAD | 1 +
.../tests/download/git-remote/refs-sub2.git/config | 4 ++
.../0a/846af45c3e455789435f49f80d70e86b65b9d7 | Bin 0 -> 22 bytes
.../0b/32ebd8fc52cec991f18c94be980e85a8341585 | Bin 0 -> 22 bytes
.../32/d61bae693af7879da63b89a60d3ae67f851e56 | Bin 0 -> 151 bytes
.../3e/9b0a5198c64cea9c00f820433411e3b4d50c1c | Bin 0 -> 48 bytes
.../85/8f666af75b7c0dfba6b8be7eac5f196e7a1221 | Bin 0 -> 49 bytes
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb | Bin 0 -> 184 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../e8/3f6f805bd016b90acafc8702c52d778eb57310 | Bin 0 -> 182 bytes
.../git-remote/refs-sub2.git/refs/heads/submodule | 1 +
.../download/git-remote/repo.git/.gitattributes | 1 +
.../tests/download/git-remote/repo.git/HEAD | 1 +
.../tests/download/git-remote/repo.git/config | 4 ++
.../0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343 | Bin 0 -> 22 bytes
.../11/93ff46343f4f6a0522e2b28b871e905178c1f0 | Bin 0 -> 23 bytes
.../25/59d83bfe937fc0412d96ed664663c9e8a99055 | Bin 0 -> 22 bytes
.../2f/a37f6885d7eb746df75eccaddbacf3ac82799d | Bin 0 -> 180 bytes
.../31/7406308d9259e2231bd0d6ddad3de3832bce08 | Bin 0 -> 182 bytes
.../34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5 | Bin 0 -> 23 bytes
.../68/c197d0879d485f4f6cee85544722b79e68e59f | Bin 0 -> 184 bytes
.../8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7 | Bin 0 -> 65 bytes
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../a2/38b1dfcd825d47d834af3c5223417c8411d90d | Bin 0 -> 152 bytes
.../a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f | Bin 0 -> 183 bytes
.../b2/4b387624edc78d0292a127c43cad9ba97c6232 | Bin 0 -> 49 bytes
.../d4/e2941d18a63535400476324ddeb7f40164be41 | Bin 0 -> 121 bytes
.../e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e | Bin 0 -> 49 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../fe/74231105841041d5f441e70399d37f0e600aa5 | Bin 0 -> 121 bytes
.../download/git-remote/repo.git/refs/heads/master | 1 +
.../git-remote/repo.git/refs/heads/mybranch | 1 +
.../git-remote/repo.git/refs/heads/submodule | 1 +
support/testing/tests/download/gitbase.py | 27 +++++++++++
support/testing/tests/download/gitremote.py | 52 ++++++++++++++++++++
support/testing/tests/download/test_git.py | 54 +++++++++++++++++++++
support/testing/tests/fs/test_ext.py | 9 ++--
support/testing/tests/fs/test_iso9660.py | 9 ++--
support/testing/tests/fs/test_jffs2.py | 3 +-
support/testing/tests/fs/test_squashfs.py | 3 +-
support/testing/tests/fs/test_ubi.py | 3 +-
support/testing/tests/fs/test_yaffs2.py | 3 +-
support/testing/tests/init/base.py | 3 +-
support/testing/tests/package/test_dropbear.py | 3 +-
support/testing/tests/package/test_python.py | 3 +-
support/testing/tests/toolchain/test_external.py | 3 +-
99 files changed, 436 insertions(+), 37 deletions(-)
create mode 100644 support/testing/infra/runtimetest.py
create mode 100644 support/testing/tests/download/__init__.py
create mode 100644 support/testing/tests/download/br2-external/git-hash/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-hash/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-hash/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.hash
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-refs/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-refs/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-sha1/git-wrong-sha1.mk
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/config
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/07/cabc655213bdf7087d8dd50fda95124e935570
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/1d/f823cb8a6d1866148ae50a8009762a9c4c777f
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/7d/52f458bdb0f9f5a4beb46fa82824421b8b988e
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/cb/545facf77bbc5f24f95b6d503c338d10b7b717
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/dd/130f6f4629514adaf2e03407f3ed9344eb6cd6
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/f8/001e5780100962a5e30a25cbc4c9609cfc7bf3
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/refs/heads/submodule
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/config
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/0a/846af45c3e455789435f49f80d70e86b65b9d7
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/0b/32ebd8fc52cec991f18c94be980e85a8341585
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/32/d61bae693af7879da63b89a60d3ae67f851e56
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/3e/9b0a5198c64cea9c00f820433411e3b4d50c1c
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/85/8f666af75b7c0dfba6b8be7eac5f196e7a1221
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/e8/3f6f805bd016b90acafc8702c52d778eb57310
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/refs/heads/submodule
create mode 100644 support/testing/tests/download/git-remote/repo.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/repo.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/repo.git/config
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/11/93ff46343f4f6a0522e2b28b871e905178c1f0
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/25/59d83bfe937fc0412d96ed664663c9e8a99055
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/2f/a37f6885d7eb746df75eccaddbacf3ac82799d
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/31/7406308d9259e2231bd0d6ddad3de3832bce08
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/68/c197d0879d485f4f6cee85544722b79e68e59f
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/a2/38b1dfcd825d47d834af3c5223417c8411d90d
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/b2/4b387624edc78d0292a127c43cad9ba97c6232
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/d4/e2941d18a63535400476324ddeb7f40164be41
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/fe/74231105841041d5f441e70399d37f0e600aa5
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/master
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/mybranch
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/submodule
create mode 100644 support/testing/tests/download/gitbase.py
create mode 100644 support/testing/tests/download/gitremote.py
create mode 100644 support/testing/tests/download/test_git.py
--
2.14.2
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v3 1/9] testing/infra/builder: call make with empty env
2017-10-29 14:05 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
@ 2017-10-29 14:06 ` Ricardo Martincoski
2018-04-01 17:58 ` Thomas Petazzoni
2017-10-29 14:06 ` [Buildroot] [PATCH v3 2/9] testing/infra/builder: split configure() from build() Ricardo Martincoski
` (9 subsequent siblings)
10 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-29 14:06 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Use an empty environment when calling make, but import PATH so the
buildroot tree under test can find libraries from the host machine.
Since environment variables are now ignored, move the handling of
BR2_DL_DIR to the defconfig to keep the current precedence of -d:
BR2_DL_DIR | -d DIR | test downloads | BR downloads
------------+----------+------------------+--------------
unset | unset | [error] | [error]
unset | set | in $(DIR) | in $(DIR)
set | unset | in $(BR2_DL_DIR) | in $(BR2_DL_DIR)
set | set | in $(DIR) | in $(DIR)
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
---
Changes v2 -> v3:
- new patch
- search for "start with an empty environment" in
http://patchwork.ozlabs.org/patch/806159/
---
support/testing/infra/basetest.py | 1 +
support/testing/infra/builder.py | 7 +++++--
support/testing/run-tests | 1 -
3 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
index 754922692c..b85e8627fd 100644
--- a/support/testing/infra/basetest.py
+++ b/support/testing/infra/basetest.py
@@ -42,6 +42,7 @@ class BRTest(unittest.TestCase):
self.testname = self.__class__.__name__
self.builddir = self.outputdir and os.path.join(self.outputdir, self.testname)
self.emulator = None
+ self.config += '\nBR2_DL_DIR="{}"\n'.format(self.downloaddir)
self.config += "\nBR2_JLEVEL={}\n".format(self.jlevel)
def show_msg(self, msg):
diff --git a/support/testing/infra/builder.py b/support/testing/infra/builder.py
index 7512339ae1..36f4801a2d 100644
--- a/support/testing/infra/builder.py
+++ b/support/testing/infra/builder.py
@@ -24,15 +24,18 @@ class Builder(object):
"> end defconfig\n")
self.logfile.flush()
+ env = {"PATH": os.environ["PATH"]}
cmd = ["make",
"O={}".format(self.builddir),
"olddefconfig"]
- ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile)
+ ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
+ env=env)
if ret != 0:
raise SystemError("Cannot olddefconfig")
cmd = ["make", "-C", self.builddir]
- ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile)
+ ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
+ env=env)
if ret != 0:
raise SystemError("Build failed")
diff --git a/support/testing/run-tests b/support/testing/run-tests
index f8cee09ed1..270e78cff7 100755
--- a/support/testing/run-tests
+++ b/support/testing/run-tests
@@ -58,7 +58,6 @@ def main():
return 1
BRTest.downloaddir = os.path.abspath(args.download)
- os.putenv("BR2_DL_DIR", BRTest.downloaddir)
if args.output is None:
print "Missing output directory, please use -o/--output"
--
2.14.2
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v3 2/9] testing/infra/builder: split configure() from build()
2017-10-29 14:05 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 1/9] testing/infra/builder: call make with empty env Ricardo Martincoski
@ 2017-10-29 14:06 ` Ricardo Martincoski
2018-04-01 17:59 ` Thomas Petazzoni
2017-10-29 14:06 ` [Buildroot] [PATCH v3 3/9] testing/infra/builder: build with target and environment Ricardo Martincoski
` (8 subsequent siblings)
10 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-29 14:06 UTC (permalink / raw)
To: buildroot
Some test cases don't use a full build as setup, so split the build()
method into configure() and build().
It allows a test case to perform configuration at the setup stage and
the build inside the test itself.
Call this new method just before build in the BRTest base class, to keep
the current behavior for existing test cases.
This change will be needed when adding a common class to test the git
download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
Changes v2 -> v3:
- renumber 1 -> 2;
- rebased after new patch 1 "call make with empty env";
- removed reviewed tag because the rebase over new patch 1 is not
trivial (automatic);
Changes v1 -> v2:
- new patch to adapt the test infra to test git download
---
support/testing/infra/basetest.py | 1 +
support/testing/infra/builder.py | 4 +++-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
index b85e8627fd..f3f13ad97f 100644
--- a/support/testing/infra/basetest.py
+++ b/support/testing/infra/basetest.py
@@ -58,6 +58,7 @@ class BRTest(unittest.TestCase):
if not self.b.is_finished():
self.show_msg("Building")
+ self.b.configure()
self.b.build()
self.show_msg("Building done")
diff --git a/support/testing/infra/builder.py b/support/testing/infra/builder.py
index 36f4801a2d..faf1eb1494 100644
--- a/support/testing/infra/builder.py
+++ b/support/testing/infra/builder.py
@@ -12,7 +12,7 @@ class Builder(object):
self.builddir = builddir
self.logfile = infra.open_log_file(builddir, "build", logtofile)
- def build(self):
+ def configure(self):
if not os.path.isdir(self.builddir):
os.makedirs(self.builddir)
@@ -33,6 +33,8 @@ class Builder(object):
if ret != 0:
raise SystemError("Cannot olddefconfig")
+ def build(self):
+ env = {"PATH": os.environ["PATH"]}
cmd = ["make", "-C", self.builddir]
ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
env=env)
--
2.14.2
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v3 3/9] testing/infra/builder: build with target and environment
2017-10-29 14:05 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 1/9] testing/infra/builder: call make with empty env Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 2/9] testing/infra/builder: split configure() from build() Ricardo Martincoski
@ 2017-10-29 14:06 ` Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 4/9] testing/infra: split runtime test from BRTest Ricardo Martincoski
` (7 subsequent siblings)
10 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-29 14:06 UTC (permalink / raw)
To: buildroot
Make the builder able to call 'VAR1=1 make VAR2=2 target'.
Allow to send extra parameters to be added to the end of make command
line. It can be used for these purposes:
- to configure a br2-external, by passing 'BR2_EXTERNAL="dir"';
- to specify a make target, such as 'foo-source'.
Allow to add variables to the environment in which make runs. It can be
used to override values from environment, such as 'BR2_DL_DIR="dl"'.
This change will be needed when adding a common class to test the git
download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Changes v2 -> v3:
- rebase after adding patch 1 to the series, which runs make using an
empty env;
- use docstring (Arnout Vandecappelle);
- use more descriptive parameter names (make_extra_opts,
make_extra_env) (Arnout Vandecappelle);
- default make_extra_env to {} and use update() (Arnout
Vandecappelle);
- for consistence I did the equivalent to make_extra_opts;
- for symmetry, use make_extra_env also in the configure() (Arnout
Vandecappelle);
- remove old example from commit message since using a static repo to
test git refs do not need to override _VERSION.
Changes v1 -> v2:
- new patch to adapt the test infra to test git download;
---
support/testing/infra/builder.py | 38 ++++++++++++++++++++++++++++++++++----
1 file changed, 34 insertions(+), 4 deletions(-)
diff --git a/support/testing/infra/builder.py b/support/testing/infra/builder.py
index faf1eb1494..d478c0d212 100644
--- a/support/testing/infra/builder.py
+++ b/support/testing/infra/builder.py
@@ -12,7 +12,18 @@ class Builder(object):
self.builddir = builddir
self.logfile = infra.open_log_file(builddir, "build", logtofile)
- def configure(self):
+ def configure(self, make_extra_opts=[], make_extra_env={}):
+ """
+ Configure the build.
+
+ make_extra_opts: a list of arguments to be passed to the make
+ command.
+ e.g. make_extra_opts=["BR2_EXTERNAL=/path"]
+
+ make_extra_env: a dict of variables to be appended (or replaced)
+ in the environment that calls make.
+ e.g. make_extra_env={"BR2_DL_DIR": "/path"}
+ """
if not os.path.isdir(self.builddir):
os.makedirs(self.builddir)
@@ -25,17 +36,36 @@ class Builder(object):
self.logfile.flush()
env = {"PATH": os.environ["PATH"]}
+ env.update(make_extra_env)
+
cmd = ["make",
- "O={}".format(self.builddir),
- "olddefconfig"]
+ "O={}".format(self.builddir)]
+ cmd += make_extra_opts
+ cmd += ["olddefconfig"]
+
ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
env=env)
if ret != 0:
raise SystemError("Cannot olddefconfig")
- def build(self):
+ def build(self, make_extra_opts=[], make_extra_env={}):
+ """
+ Perform the build.
+
+ make_extra_opts: a list of arguments to be passed to the make
+ command. It can include a make target.
+ e.g. make_extra_opts=["foo-source"]
+
+ make_extra_env: a dict of variables to be appended (or replaced)
+ in the environment that calls make.
+ e.g. make_extra_env={"BR2_DL_DIR": "/path"}
+ """
env = {"PATH": os.environ["PATH"]}
+ env.update(make_extra_env)
+
cmd = ["make", "-C", self.builddir]
+ cmd += make_extra_opts
+
ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
env=env)
if ret != 0:
--
2.14.2
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v3 4/9] testing/infra: split runtime test from BRTest
2017-10-29 14:05 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
` (2 preceding siblings ...)
2017-10-29 14:06 ` [Buildroot] [PATCH v3 3/9] testing/infra/builder: build with target and environment Ricardo Martincoski
@ 2017-10-29 14:06 ` Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 5/9] testing/infra/basetest: support br2-external Ricardo Martincoski
` (6 subsequent siblings)
10 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-29 14:06 UTC (permalink / raw)
To: buildroot
Move the setup of emulator to a new class, RuntimeTestBase, that behaves
exactly like BRTest currently does.
It will avoid duplicating code when adding a common class to test the
git download infra.
Change all current test cases to use the new class.
Do this by first using automatic replace:
$ find support/testing/ -name '*.py' | \
xargs grep -l BRTest | \
xargs sed -i \
-e 's,import infra.basetest,\0\nimport infra.runtimetest,g' \
-e 's,infra.basetest.BRTest,infra.runtimetest.RuntimeTestBase,g'
and then manually add code to import runtimetest in test_external.py to
avoid this error:
AttributeError: 'module' object has no attribute 'LoadTestsFailure'
This explicit import was not need before because run-tests imports
BRTest and this is the only test file that do not use the defconfig
fragments from basetest.py in its code.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Changes v2 -> v3:
- new patch
- search for "RuntimeTestBase" in
http://patchwork.ozlabs.org/patch/806161/
---
support/testing/infra/basetest.py | 10 ----------
support/testing/infra/runtimetest.py | 23 +++++++++++++++++++++++
support/testing/tests/core/test_post_scripts.py | 3 ++-
support/testing/tests/core/test_rootfs_overlay.py | 3 ++-
support/testing/tests/core/test_timezone.py | 7 ++++---
support/testing/tests/fs/test_ext.py | 9 +++++----
support/testing/tests/fs/test_iso9660.py | 9 +++++----
support/testing/tests/fs/test_jffs2.py | 3 ++-
support/testing/tests/fs/test_squashfs.py | 3 ++-
support/testing/tests/fs/test_ubi.py | 3 ++-
support/testing/tests/fs/test_yaffs2.py | 3 ++-
support/testing/tests/init/base.py | 3 ++-
support/testing/tests/package/test_dropbear.py | 3 ++-
support/testing/tests/package/test_python.py | 3 ++-
support/testing/tests/toolchain/test_external.py | 3 ++-
15 files changed, 57 insertions(+), 31 deletions(-)
create mode 100644 support/testing/infra/runtimetest.py
diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
index f3f13ad97f..4773312585 100644
--- a/support/testing/infra/basetest.py
+++ b/support/testing/infra/basetest.py
@@ -3,7 +3,6 @@ import os
import datetime
from infra.builder import Builder
-from infra.emulator import Emulator
BASIC_TOOLCHAIN_CONFIG = \
"""
@@ -41,7 +40,6 @@ class BRTest(unittest.TestCase):
super(BRTest, self).__init__(names)
self.testname = self.__class__.__name__
self.builddir = self.outputdir and os.path.join(self.outputdir, self.testname)
- self.emulator = None
self.config += '\nBR2_DL_DIR="{}"\n'.format(self.downloaddir)
self.config += "\nBR2_JLEVEL={}\n".format(self.jlevel)
@@ -57,17 +55,9 @@ class BRTest(unittest.TestCase):
self.b.delete()
if not self.b.is_finished():
- self.show_msg("Building")
self.b.configure()
- self.b.build()
- self.show_msg("Building done")
-
- self.emulator = Emulator(self.builddir, self.downloaddir,
- self.logtofile, self.timeout_multiplier)
def tearDown(self):
self.show_msg("Cleaning up")
- if self.emulator:
- self.emulator.stop()
if self.b and not self.keepbuilds:
self.b.delete()
diff --git a/support/testing/infra/runtimetest.py b/support/testing/infra/runtimetest.py
new file mode 100644
index 0000000000..68bb03d091
--- /dev/null
+++ b/support/testing/infra/runtimetest.py
@@ -0,0 +1,23 @@
+from infra.basetest import BRTest
+from infra.emulator import Emulator
+
+
+class RuntimeTestBase(BRTest):
+ def __init__(self, names):
+ super(RuntimeTestBase, self).__init__(names)
+ self.emulator = None
+
+ def setUp(self):
+ super(RuntimeTestBase, self).setUp()
+ if not self.b.is_finished():
+ self.show_msg("Building")
+ self.b.build()
+ self.show_msg("Building done")
+
+ self.emulator = Emulator(self.builddir, self.downloaddir,
+ self.logtofile, self.timeout_multiplier)
+
+ def tearDown(self):
+ if self.emulator:
+ self.emulator.stop()
+ super(RuntimeTestBase, self).tearDown()
diff --git a/support/testing/tests/core/test_post_scripts.py b/support/testing/tests/core/test_post_scripts.py
index 1db568b0d6..9b821abb57 100644
--- a/support/testing/tests/core/test_post_scripts.py
+++ b/support/testing/tests/core/test_post_scripts.py
@@ -2,9 +2,10 @@ import os
import csv
import infra.basetest
+import infra.runtimetest
-class TestPostScripts(infra.basetest.BRTest):
+class TestPostScripts(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_INIT_NONE=y
diff --git a/support/testing/tests/core/test_rootfs_overlay.py b/support/testing/tests/core/test_rootfs_overlay.py
index fedd40d8ac..66b5531221 100644
--- a/support/testing/tests/core/test_rootfs_overlay.py
+++ b/support/testing/tests/core/test_rootfs_overlay.py
@@ -2,13 +2,14 @@ import os
import subprocess
import infra.basetest
+import infra.runtimetest
def compare_file(file1, file2):
return subprocess.call(["cmp", file1, file2])
-class TestRootfsOverlay(infra.basetest.BRTest):
+class TestRootfsOverlay(infra.runtimetest.RuntimeTestBase):
rootfs_overlay_path = infra.filepath("tests/core/rootfs-overlay")
diff --git a/support/testing/tests/core/test_timezone.py b/support/testing/tests/core/test_timezone.py
index 050624e0aa..dca38bebe8 100644
--- a/support/testing/tests/core/test_timezone.py
+++ b/support/testing/tests/core/test_timezone.py
@@ -1,6 +1,7 @@
import os
import infra.basetest
+import infra.runtimetest
def boot_armv5_cpio(emulator, builddir):
@@ -10,7 +11,7 @@ def boot_armv5_cpio(emulator, builddir):
emulator.login()
-class TestNoTimezone(infra.basetest.BRTest):
+class TestNoTimezone(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
# BR2_TARGET_TZ_INFO is not set
@@ -26,7 +27,7 @@ class TestNoTimezone(infra.basetest.BRTest):
self.assertEqual(tz[0].strip(), "UTC")
-class TestGlibcAllTimezone(infra.basetest.BRTest):
+class TestGlibcAllTimezone(infra.runtimetest.RuntimeTestBase):
config = \
"""
BR2_arm=y
@@ -48,7 +49,7 @@ class TestGlibcAllTimezone(infra.basetest.BRTest):
self.assertEqual(tz[0].strip(), "CET")
-class TestGlibcNonDefaultLimitedTimezone(infra.basetest.BRTest):
+class TestGlibcNonDefaultLimitedTimezone(infra.runtimetest.RuntimeTestBase):
config = \
"""
BR2_arm=y
diff --git a/support/testing/tests/fs/test_ext.py b/support/testing/tests/fs/test_ext.py
index f5f9e9fdf1..a8f68bc54c 100644
--- a/support/testing/tests/fs/test_ext.py
+++ b/support/testing/tests/fs/test_ext.py
@@ -2,6 +2,7 @@ import os
import subprocess
import infra.basetest
+import infra.runtimetest
VOLNAME_PROP = "Filesystem volume name"
REVISION_PROP = "Filesystem revision #"
@@ -41,7 +42,7 @@ def boot_img_and_check_fs_type(emulator, builddir, fs_type):
return exit_code
-class TestExt2(infra.basetest.BRTest):
+class TestExt2(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_EXT2=y
@@ -60,7 +61,7 @@ class TestExt2(infra.basetest.BRTest):
self.assertEqual(exit_code, 0)
-class TestExt2r1(infra.basetest.BRTest):
+class TestExt2r1(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_EXT2=y
@@ -80,7 +81,7 @@ class TestExt2r1(infra.basetest.BRTest):
self.assertEqual(exit_code, 0)
-class TestExt3(infra.basetest.BRTest):
+class TestExt3(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_EXT2=y
@@ -99,7 +100,7 @@ class TestExt3(infra.basetest.BRTest):
self.assertEqual(exit_code, 0)
-class TestExt4(infra.basetest.BRTest):
+class TestExt4(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_EXT2=y
diff --git a/support/testing/tests/fs/test_iso9660.py b/support/testing/tests/fs/test_iso9660.py
index 5d945a347a..36227b438d 100644
--- a/support/testing/tests/fs/test_iso9660.py
+++ b/support/testing/tests/fs/test_iso9660.py
@@ -1,6 +1,7 @@
import os
import infra.basetest
+import infra.runtimetest
BASIC_CONFIG = \
"""
@@ -47,7 +48,7 @@ def test_touch_file(emulator):
# Grub 2
-class TestIso9660Grub2External(infra.basetest.BRTest):
+class TestIso9660Grub2External(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
@@ -67,7 +68,7 @@ class TestIso9660Grub2External(infra.basetest.BRTest):
self.assertEqual(exit_code, 1)
-class TestIso9660Grub2Internal(infra.basetest.BRTest):
+class TestIso9660Grub2Internal(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
@@ -90,7 +91,7 @@ class TestIso9660Grub2Internal(infra.basetest.BRTest):
# Syslinux
-class TestIso9660SyslinuxExternal(infra.basetest.BRTest):
+class TestIso9660SyslinuxExternal(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
@@ -109,7 +110,7 @@ class TestIso9660SyslinuxExternal(infra.basetest.BRTest):
self.assertEqual(exit_code, 1)
-class TestIso9660SyslinuxInternal(infra.basetest.BRTest):
+class TestIso9660SyslinuxInternal(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
diff --git a/support/testing/tests/fs/test_jffs2.py b/support/testing/tests/fs/test_jffs2.py
index 2ff5099180..8f73cf1f06 100644
--- a/support/testing/tests/fs/test_jffs2.py
+++ b/support/testing/tests/fs/test_jffs2.py
@@ -2,6 +2,7 @@ import os
import subprocess
import infra.basetest
+import infra.runtimetest
def jffs2dump_find_file(files_list, fname):
@@ -12,7 +13,7 @@ def jffs2dump_find_file(files_list, fname):
return False
-class TestJffs2(infra.basetest.BRTest):
+class TestJffs2(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_JFFS2=y
diff --git a/support/testing/tests/fs/test_squashfs.py b/support/testing/tests/fs/test_squashfs.py
index 066c054342..f80149f9d9 100644
--- a/support/testing/tests/fs/test_squashfs.py
+++ b/support/testing/tests/fs/test_squashfs.py
@@ -2,9 +2,10 @@ import os
import subprocess
import infra.basetest
+import infra.runtimetest
-class TestSquashfs(infra.basetest.BRTest):
+class TestSquashfs(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_SQUASHFS=y
diff --git a/support/testing/tests/fs/test_ubi.py b/support/testing/tests/fs/test_ubi.py
index 015d82f769..c724f4f740 100644
--- a/support/testing/tests/fs/test_ubi.py
+++ b/support/testing/tests/fs/test_ubi.py
@@ -2,9 +2,10 @@ import subprocess
import os
import infra.basetest
+import infra.runtimetest
-class TestUbi(infra.basetest.BRTest):
+class TestUbi(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_UBIFS=y
diff --git a/support/testing/tests/fs/test_yaffs2.py b/support/testing/tests/fs/test_yaffs2.py
index b60e90e660..c7c8c1f724 100644
--- a/support/testing/tests/fs/test_yaffs2.py
+++ b/support/testing/tests/fs/test_yaffs2.py
@@ -1,9 +1,10 @@
import os
import infra.basetest
+import infra.runtimetest
-class TestYaffs2(infra.basetest.BRTest):
+class TestYaffs2(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
infra.basetest.MINIMAL_CONFIG + \
"""
diff --git a/support/testing/tests/init/base.py b/support/testing/tests/init/base.py
index 75cfbe9c59..1b736af657 100644
--- a/support/testing/tests/init/base.py
+++ b/support/testing/tests/init/base.py
@@ -1,9 +1,10 @@
import os
import subprocess
import infra.basetest
+import infra.runtimetest
-class InitSystemBase(infra.basetest.BRTest):
+class InitSystemBase(infra.runtimetest.RuntimeTestBase):
def start_emulator(self, fs_type, kernel=None, dtb=None, init=None):
img = os.path.join(self.builddir, "images", "rootfs.{}".format(fs_type))
diff --git a/support/testing/tests/package/test_dropbear.py b/support/testing/tests/package/test_dropbear.py
index 8f7f1fee82..8f7f30e3af 100644
--- a/support/testing/tests/package/test_dropbear.py
+++ b/support/testing/tests/package/test_dropbear.py
@@ -1,9 +1,10 @@
import os
import infra.basetest
+import infra.runtimetest
-class TestDropbear(infra.basetest.BRTest):
+class TestDropbear(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_GENERIC_ROOT_PASSWD="testpwd"
diff --git a/support/testing/tests/package/test_python.py b/support/testing/tests/package/test_python.py
index 26cf49947b..787364c719 100644
--- a/support/testing/tests/package/test_python.py
+++ b/support/testing/tests/package/test_python.py
@@ -1,9 +1,10 @@
import os
import infra.basetest
+import infra.runtimetest
-class TestPythonBase(infra.basetest.BRTest):
+class TestPythonBase(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_CPIO=y
diff --git a/support/testing/tests/toolchain/test_external.py b/support/testing/tests/toolchain/test_external.py
index 881d2b00db..b72e19f740 100644
--- a/support/testing/tests/toolchain/test_external.py
+++ b/support/testing/tests/toolchain/test_external.py
@@ -1,5 +1,6 @@
import os
import infra
+import infra.runtimetest
BASIC_CONFIG = \
"""
@@ -17,7 +18,7 @@ def has_broken_links(path):
return False
-class TestExternalToolchain(infra.basetest.BRTest):
+class TestExternalToolchain(infra.runtimetest.RuntimeTestBase):
def common_check(self):
# Check for broken symlinks
for d in ["lib", "usr/lib"]:
--
2.14.2
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v3 5/9] testing/infra/basetest: support br2-external
2017-10-29 14:05 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
` (3 preceding siblings ...)
2017-10-29 14:06 ` [Buildroot] [PATCH v3 4/9] testing/infra: split runtime test from BRTest Ricardo Martincoski
@ 2017-10-29 14:06 ` Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 6/9] testing/tests/download: add git hash test Ricardo Martincoski
` (5 subsequent siblings)
10 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-29 14:06 UTC (permalink / raw)
To: buildroot
This change will be needed when adding a common class to test the git
download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Changes v2 -> v3:
- new patch
- search for "support for BR2_EXTERNAL directly in BRTest" in
http://patchwork.ozlabs.org/patch/806161/
---
support/testing/infra/basetest.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
index 4773312585..641f1961f2 100644
--- a/support/testing/infra/basetest.py
+++ b/support/testing/infra/basetest.py
@@ -29,6 +29,7 @@ MINIMAL_CONFIG = \
class BRTest(unittest.TestCase):
config = None
+ br2_external = None
downloaddir = None
outputdir = None
logtofile = True
@@ -55,7 +56,10 @@ class BRTest(unittest.TestCase):
self.b.delete()
if not self.b.is_finished():
- self.b.configure()
+ if self.br2_external is None:
+ self.b.configure()
+ else:
+ self.b.configure(["BR2_EXTERNAL={}".format(self.br2_external)])
def tearDown(self):
self.show_msg("Cleaning up")
--
2.14.2
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v3 6/9] testing/tests/download: add git hash test
2017-10-29 14:05 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
` (4 preceding siblings ...)
2017-10-29 14:06 ` [Buildroot] [PATCH v3 5/9] testing/infra/basetest: support br2-external Ricardo Martincoski
@ 2017-10-29 14:06 ` Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 7/9] testing/tests/download: test case for git refs Ricardo Martincoski
` (4 subsequent siblings)
10 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-29 14:06 UTC (permalink / raw)
To: buildroot
Add one test case to ensure the hash is checked for git packages:
- correct hash;
- wrong hash;
- no hash file.
Add required infra:
- a GitRemote class, that can start a git server in the host machine to
emulate a remote git server under the control of the test;
- a new base class, called GitTestBase, that inherits from BRTest and
must be subclassed by all git test cases.
Its setUp() method takes care of configuring the build with a
br2-external, avoiding to hit http://sources.buildroot.net by using
an empty BR2_BACKUP_SITE. It also avoids downloading not
pre-installed dependencies (i.e. lzip) every time by calling 'make
dependencies' using the common dl directory, and it instantiates the
GitRemote object.
Besides the Python scripts, add some fixtures used during the tests:
- a br2-external (git-hash) with one package for each part of the test
case;
- a static git bare repo (repo.git) to be served using GitRemote class.
Neither the br2-external nor the check hash functionalities are the
subject of these tests per se, so for simplicity limit the check to the
error codes and don't look for the messages in the log.
Thanks to Arnout for the hint about how to add a bare repo to test.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repo with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* a238b1d (HEAD -> master) initial
Changes v2 -> v3:
- renumber 5 -> 6;
- rebase after removing patch 3 "allow to override logfile" from the
series and adding new patches 4 "split runtime test from BRTest" and
5 "support br2-external";
- merge part of old patch 4 (very minimal GitTestBase) to old patch 5
(Arnout Vandecappelle);
- merge and tweak part of commit log from old patch 4;
- move GITREMOTE_PORT_NUMBER to external.mk (Arnout Vandecappelle);
- use dosctrings;
- start git remote from __init__ (Arnout Vandecappelle), but I kept a
separate method and call start();
- remove dead code (Arnout Vandecappelle);
- use the same git repo for all tests (Arnout Vandecappelle);
- rename the file to test_git and merge the 3 test cases in one, since
the feature "check hash for git packages" is only OK if all work
well and also because the test does not take too much time;
- update commit log accordingly.
Changes v1 -> v2 (patch 4):
- new patch
Changes v1 -> v2 (patch 5):
- use git daemon + git:// instead of ssh (Arnout);
- remove __main__ handling since the test infra already does that;
- use the logging support from test infra;
- this patch is part of series 1/3 of a new version of
http://patchwork.ozlabs.org/patch/690097/
---
.gitlab-ci.yml | 1 +
support/testing/tests/download/__init__.py | 0
.../tests/download/br2-external/git-hash/Config.in | 0
.../download/br2-external/git-hash/external.desc | 1 +
.../download/br2-external/git-hash/external.mk | 4 ++
.../br2-external/git-hash/package/bad/bad.hash | 1 +
.../br2-external/git-hash/package/bad/bad.mk | 10 ++++
.../br2-external/git-hash/package/good/good.hash | 1 +
.../br2-external/git-hash/package/good/good.mk | 10 ++++
.../br2-external/git-hash/package/nohash/nohash.mk | 10 ++++
.../download/git-remote/repo.git/.gitattributes | 1 +
.../tests/download/git-remote/repo.git/HEAD | 1 +
.../tests/download/git-remote/repo.git/config | 4 ++
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../a2/38b1dfcd825d47d834af3c5223417c8411d90d | Bin 0 -> 152 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../download/git-remote/repo.git/refs/heads/master | 1 +
support/testing/tests/download/gitbase.py | 27 +++++++++++
support/testing/tests/download/gitremote.py | 52 +++++++++++++++++++++
support/testing/tests/download/test_git.py | 23 +++++++++
20 files changed, 147 insertions(+)
create mode 100644 support/testing/tests/download/__init__.py
create mode 100644 support/testing/tests/download/br2-external/git-hash/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-hash/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-hash/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.hash
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk
create mode 100644 support/testing/tests/download/git-remote/repo.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/repo.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/repo.git/config
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/a2/38b1dfcd825d47d834af3c5223417c8411d90d
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/master
create mode 100644 support/testing/tests/download/gitbase.py
create mode 100644 support/testing/tests/download/gitremote.py
create mode 100644 support/testing/tests/download/test_git.py
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index ebaac7a3d9..fbbc4c0243 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -227,6 +227,7 @@ tests.core.test_rootfs_overlay.TestRootfsOverlay: *runtime_test
tests.core.test_timezone.TestGlibcAllTimezone: *runtime_test
tests.core.test_timezone.TestGlibcNonDefaultLimitedTimezone: *runtime_test
tests.core.test_timezone.TestNoTimezone: *runtime_test
+tests.download.test_git.TestGitHash: *runtime_test
tests.fs.test_ext.TestExt2: *runtime_test
tests.fs.test_ext.TestExt2r1: *runtime_test
tests.fs.test_ext.TestExt3: *runtime_test
diff --git a/support/testing/tests/download/__init__.py b/support/testing/tests/download/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/support/testing/tests/download/br2-external/git-hash/Config.in b/support/testing/tests/download/br2-external/git-hash/Config.in
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/support/testing/tests/download/br2-external/git-hash/external.desc b/support/testing/tests/download/br2-external/git-hash/external.desc
new file mode 100644
index 0000000000..41316c8b25
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/external.desc
@@ -0,0 +1 @@
+name: GIT_HASH
diff --git a/support/testing/tests/download/br2-external/git-hash/external.mk b/support/testing/tests/download/br2-external/git-hash/external.mk
new file mode 100644
index 0000000000..4646dfe2b0
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/external.mk
@@ -0,0 +1,4 @@
+include $(sort $(wildcard $(BR2_EXTERNAL_GIT_HASH_PATH)/package/*/*.mk))
+
+# Get the git server port number from the test infra
+GITREMOTE_PORT_NUMBER ?= 9418
diff --git a/support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
new file mode 100644
index 0000000000..b9e1baec84
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
@@ -0,0 +1 @@
+sha256 0000000000000000000000000000000000000000000000000000000000000000 bad-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
diff --git a/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
new file mode 100644
index 0000000000..5497bd6bfe
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
@@ -0,0 +1,10 @@
+################################################################################
+#
+# bad
+#
+################################################################################
+
+BAD_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+BAD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-hash/package/good/good.hash b/support/testing/tests/download/br2-external/git-hash/package/good/good.hash
new file mode 100644
index 0000000000..9e92ab8ab9
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/good/good.hash
@@ -0,0 +1 @@
+sha256 d00ae598e9e770d607621a86766030b42eaa58156cb8d482b043969da7963c23 good-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
diff --git a/support/testing/tests/download/br2-external/git-hash/package/good/good.mk b/support/testing/tests/download/br2-external/git-hash/package/good/good.mk
new file mode 100644
index 0000000000..0f0eefd944
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/good/good.mk
@@ -0,0 +1,10 @@
+################################################################################
+#
+# good
+#
+################################################################################
+
+GOOD_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+GOOD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk b/support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk
new file mode 100644
index 0000000000..1da19d88c6
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk
@@ -0,0 +1,10 @@
+################################################################################
+#
+# nohash
+#
+################################################################################
+
+NOHASH_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+NOHASH_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/repo.git/.gitattributes b/support/testing/tests/download/git-remote/repo.git/.gitattributes
new file mode 100644
index 0000000000..eb50c64a21
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/.gitattributes
@@ -0,0 +1 @@
+objects/*/* binary
diff --git a/support/testing/tests/download/git-remote/repo.git/HEAD b/support/testing/tests/download/git-remote/repo.git/HEAD
new file mode 100644
index 0000000000..cb089cd89a
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/HEAD
@@ -0,0 +1 @@
+ref: refs/heads/master
diff --git a/support/testing/tests/download/git-remote/repo.git/config b/support/testing/tests/download/git-remote/repo.git/config
new file mode 100644
index 0000000000..07d359d07c
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/config
@@ -0,0 +1,4 @@
+[core]
+ repositoryformatversion = 0
+ filemode = true
+ bare = true
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab b/support/testing/tests/download/git-remote/repo.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
new file mode 100644
index 0000000000000000000000000000000000000000..9db72668cf9374b0b85a25a19f30084fd460072d
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5wZcw)UHz=nDZYB`lCt5lYr9_mN?Omvz#_so
F4FHVv5b^*3
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/a2/38b1dfcd825d47d834af3c5223417c8411d90d b/support/testing/tests/download/git-remote/repo.git/objects/a2/38b1dfcd825d47d834af3c5223417c8411d90d
new file mode 100644
index 0000000000000000000000000000000000000000..31b6bcf34de247d14531c70b36e28d895ba20af3
GIT binary patch
literal 152
zcmV;J0B8Sr0j<tW4#FT1Kw;OMVlL1IsFY%i at d9r21jApQv_OKm@b)%w;oi9Y7BA10
zl&FE!)2`JJz?dk*5QMWrMPrqWC`}wkKTO<v<fHILXtWHU?OrNe$zk;cE?667R~`$&
zv3{^mUp&tVY3*G}ClHEjqx7`b(D8wDw&^y}!k==CGBv`Lu^Y`f_^!m*+1KHhS@s5O
G+F(P1IZe3$
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5 b/support/testing/tests/download/git-remote/repo.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
new file mode 100644
index 0000000000000000000000000000000000000000..df2037a2d2d30afb20e31cd558c92c2edabe7cf6
GIT binary patch
literal 23
fcmb<m^geacKWIb3i8H5lPWqgA$i$$b%kl;Qg;xrl
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/heads/master b/support/testing/tests/download/git-remote/repo.git/refs/heads/master
new file mode 100644
index 0000000000..b6bccc1c17
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/heads/master
@@ -0,0 +1 @@
+a238b1dfcd825d47d834af3c5223417c8411d90d
diff --git a/support/testing/tests/download/gitbase.py b/support/testing/tests/download/gitbase.py
new file mode 100644
index 0000000000..d0a7b95e5e
--- /dev/null
+++ b/support/testing/tests/download/gitbase.py
@@ -0,0 +1,27 @@
+import infra.basetest
+from gitremote import GitRemote
+
+
+class GitTestBase(infra.basetest.BRTest):
+ config = \
+ """
+ BR2_BACKUP_SITE=""
+ """
+ serveddir = infra.filepath("tests/download/git-remote")
+
+ def setUp(self):
+ super(GitTestBase, self).setUp()
+ if not self.b.is_finished():
+ # when running the tests in a local machine, do not download not
+ # pre-installed dependencies (i.e. lzip) every time a test runs
+ self.b.build(["dependencies"])
+
+ self.gitremote = GitRemote(self.builddir, self.serveddir,
+ self.logtofile)
+
+ def tearDown(self):
+ self.show_msg("Cleaning up")
+ if self.gitremote:
+ self.gitremote.stop()
+ if self.b and not self.keepbuilds:
+ self.b.delete()
diff --git a/support/testing/tests/download/gitremote.py b/support/testing/tests/download/gitremote.py
new file mode 100644
index 0000000000..b3a1357193
--- /dev/null
+++ b/support/testing/tests/download/gitremote.py
@@ -0,0 +1,52 @@
+# subprocess does not kill the child daemon when a test case fails by raising
+# an exception. So use pexpect instead.
+import pexpect
+
+import infra
+
+GIT_REMOTE_PORT_INITIAL = 9418
+GIT_REMOTE_PORT_LAST = GIT_REMOTE_PORT_INITIAL + 99
+
+
+class GitRemote(object):
+ def __init__(self, builddir, serveddir, logtofile):
+ self.daemon = None
+ self.port = None
+ self.serveddir = serveddir
+ self.logfile = infra.open_log_file(builddir, "run", logtofile)
+
+ self.start()
+
+ def start(self):
+ """
+ Start a local git server.
+
+ In order to support test cases in parallel, select the port the
+ server will listen to in runtime. Since there is no reliable way
+ to allocate the port prior to starting the server (another
+ process in the host machine can use the port between it is
+ selected from a list and it is really allocated to the server)
+ try to start the server in a port and in the case it is already
+ in use, try the next one in the allowed range.
+ """
+ daemon_cmd = ["git", "daemon", "--reuseaddr", "--verbose",
+ "--listen=localhost", "--export-all",
+ "--base-path={}".format(self.serveddir)]
+ for port in range(GIT_REMOTE_PORT_INITIAL, GIT_REMOTE_PORT_LAST):
+ port_arg = ["--port={}".format(port)]
+ cmd = daemon_cmd + port_arg
+ self.logfile.write("> starting git remote with '{}'\n"
+ .format(" ".join(cmd)))
+ self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile)
+ ret = self.daemon.expect(["Ready to rumble",
+ "Address already in use"])
+ if ret == 0:
+ self.port = port
+ break
+ if not self.port:
+ raise SystemError("Could not find a free port to run git remote")
+
+ def stop(self):
+ if self.daemon is None:
+ return
+ self.daemon.terminate(force=True)
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
new file mode 100644
index 0000000000..994a8afc52
--- /dev/null
+++ b/support/testing/tests/download/test_git.py
@@ -0,0 +1,23 @@
+import os
+
+import infra
+from gitbase import GitTestBase
+
+
+class TestGitHash(GitTestBase):
+ br2_external = infra.filepath("tests/download/br2-external/git-hash")
+
+ def check_hash(self, package):
+ # store downloaded tarball inside the output dir so the test infra
+ # cleans it up at the end
+ env = {"BR2_DL_DIR": os.path.join(self.builddir, "dl"),
+ "GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
+ self.b.build(["{}-dirclean".format(package),
+ "{}-source".format(package)],
+ env)
+
+ def test_run(self):
+ with self.assertRaises(SystemError):
+ self.check_hash("bad")
+ self.check_hash("good")
+ self.check_hash("nohash")
--
2.14.2
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v3 7/9] testing/tests/download: test case for git refs
2017-10-29 14:05 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
` (5 preceding siblings ...)
2017-10-29 14:06 ` [Buildroot] [PATCH v3 6/9] testing/tests/download: add git hash test Ricardo Martincoski
@ 2017-10-29 14:06 ` Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 8/9] testing/tests/download: test git branch Ricardo Martincoski
` (3 subsequent siblings)
10 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-29 14:06 UTC (permalink / raw)
To: buildroot
All upcoming tests for git refs will rely on the return code of make to
determine whether a git ref can be downloaded or not and also to
determine whether the downloaded content is correct (all of this taking
advantage of the check-hash mechanism already in place for git
packages).
So to avoid false results i.e. in the case the check-hash mechanism
become broken in the master branch, add some sanity checks before the
actual test of download git refs.
Add the minimum test case for git refs containing only sanity checks.
Reuse the commit in the static repo.
Add a br2-external with two packages to check that:
- trying to download an invalid sha1 generates an error;
- downloading a valid sha1 that contains unexpected content generates
an error.
In order to ease the maintenance and review, each upcoming patch adding
checks to this test case will add at same time the commits to the static
repo, the equivalent packages to the br2-external and code to the test
case.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repo with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* a238b1d (HEAD -> master) initial
Changes v2 -> v3:
- rewrite from v2, using an static repo;
- separate on this patch the bare minimum static repo (actually reuse
from previous commit), the next patches on the series will gradually
add new commits to the static repo together to the equivalent
packages and call to check_download. This approach allows better
maintenance, even during the review of the series, because a patch
can be more easily reworked/discarded/reordered;
Changes v1 -> v2:
- rename the main test file to testgit (Arnout). Actually I broke it
down to a common gitbase and the two test_git_refs and test_git_hash
(this last one is new, in the previous patch);
- remove some weird/wrong TODO I had in v1 code (Arnout);
- use the same structure for test cases as used in the test infra;
- raise an exception when the download fails (Arnout). I did not add
code for this since I let the builder class to raise it;
- I reimplemented git_util as gitrepo. Arnout suggested to use it as
module and I first implemented that way locally, but then I created
test cases for submodules and I realized now I have a reason to use
a class (see next patch);
- move package (now called foo) to a BR2_EXTERNAL (Arnout);
- override BR2_DL_DIR when calling make (Arnout);
- the test infra uses O= so I don't need to;
- instead of removing the files in the dl/ folder, use a different dir
as dl/ dir since now each repo has a different name;
- open the tarball to check its contents (Arnout). I create a fresh
clone of the repo before the test to compare against;
- this patch is not checking for the actual sha1 anymore, it can be
done later. But the check for the contents should cover most cases;
- my old argument to not test only the support/download/dl-wrapper is
not valid anymore since I mimic the logic from the scripts to know
the name of the tarball. But I still think calling make ...-source
has better maintenance. And also the previous patch (test for hash
of packages with git method) can use the same base class;
- this patch is part of series 1/3 of a new version of
http://patchwork.ozlabs.org/patch/690097/
---
.gitlab-ci.yml | 1 +
.../tests/download/br2-external/git-refs/Config.in | 0
.../download/br2-external/git-refs/external.desc | 1 +
.../tests/download/br2-external/git-refs/external.mk | 4 ++++
.../package/git-wrong-content/git-wrong-content.hash | 2 ++
.../package/git-wrong-content/git-wrong-content.mk | 11 +++++++++++
.../git-refs/package/git-wrong-sha1/git-wrong-sha1.mk | 11 +++++++++++
support/testing/tests/download/test_git.py | 19 +++++++++++++++++++
8 files changed, 49 insertions(+)
create mode 100644 support/testing/tests/download/br2-external/git-refs/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-refs/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-refs/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-sha1/git-wrong-sha1.mk
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index fbbc4c0243..60d01bdfc8 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -228,6 +228,7 @@ tests.core.test_timezone.TestGlibcAllTimezone: *runtime_test
tests.core.test_timezone.TestGlibcNonDefaultLimitedTimezone: *runtime_test
tests.core.test_timezone.TestNoTimezone: *runtime_test
tests.download.test_git.TestGitHash: *runtime_test
+tests.download.test_git.TestGitRefs: *runtime_test
tests.fs.test_ext.TestExt2: *runtime_test
tests.fs.test_ext.TestExt2r1: *runtime_test
tests.fs.test_ext.TestExt3: *runtime_test
diff --git a/support/testing/tests/download/br2-external/git-refs/Config.in b/support/testing/tests/download/br2-external/git-refs/Config.in
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/support/testing/tests/download/br2-external/git-refs/external.desc b/support/testing/tests/download/br2-external/git-refs/external.desc
new file mode 100644
index 0000000000..69f40d46c6
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/external.desc
@@ -0,0 +1 @@
+name: GIT_REFS
diff --git a/support/testing/tests/download/br2-external/git-refs/external.mk b/support/testing/tests/download/br2-external/git-refs/external.mk
new file mode 100644
index 0000000000..ab38c27eb7
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/external.mk
@@ -0,0 +1,4 @@
+include $(sort $(wildcard $(BR2_EXTERNAL_GIT_REFS_PATH)/package/*/*.mk))
+
+# Get the git server port number from the test infra
+GITREMOTE_PORT_NUMBER ?= 9418
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash
new file mode 100644
index 0000000000..47b2b8b7d7
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash
@@ -0,0 +1,2 @@
+ sha256 04715901977503d1df650e0959f4d31d8e7b105e2ac99a2182e0652b8f13baa1 git-wrong-content-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
+ sha256 0000000000000000000000000000000000000000000000000000000000000000 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.mk b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.mk
new file mode 100644
index 0000000000..786224dad9
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-wrong-content
+#
+################################################################################
+
+GIT_WRONG_CONTENT_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+GIT_WRONG_CONTENT_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_WRONG_CONTENT_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-wrong-sha1/git-wrong-sha1.mk b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-sha1/git-wrong-sha1.mk
new file mode 100644
index 0000000000..f9d0d2226c
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-sha1/git-wrong-sha1.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-wrong-sha1
+#
+################################################################################
+
+GIT_WRONG_SHA1_VERSION = 0000000000000000000000000000000000000000
+GIT_WRONG_SHA1_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_WRONG_SHA1_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index 994a8afc52..29b5cd4091 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -21,3 +21,22 @@ class TestGitHash(GitTestBase):
self.check_hash("bad")
self.check_hash("good")
self.check_hash("nohash")
+
+
+class TestGitRefs(GitTestBase):
+ br2_external = infra.filepath("tests/download/br2-external/git-refs")
+
+ def check_download(self, package):
+ # store downloaded tarball inside the output dir so the test infra
+ # cleans it up at the end
+ env = {"BR2_DL_DIR": os.path.join(self.builddir, "dl"),
+ "GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
+ self.b.build(["{}-dirclean".format(package),
+ "{}-legal-info".format(package)],
+ env)
+
+ def test_run(self):
+ with self.assertRaises(SystemError):
+ self.check_download("git-wrong-content")
+ with self.assertRaises(SystemError):
+ self.check_download("git-wrong-sha1")
--
2.14.2
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v3 8/9] testing/tests/download: test git branch
2017-10-29 14:05 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
` (6 preceding siblings ...)
2017-10-29 14:06 ` [Buildroot] [PATCH v3 7/9] testing/tests/download: test case for git refs Ricardo Martincoski
@ 2017-10-29 14:06 ` Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 9/9] testing/tests/download: test git submodules Ricardo Martincoski
` (2 subsequent siblings)
10 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-29 14:06 UTC (permalink / raw)
To: buildroot
Add a branch to the static repo and check on the git refs test case the
download of a git package:
- with a sha1 reachable by a branch name, but not pointed by it, as
version. This is the most common use case for git refs in the tree;
- with a partial sha1 of a commit reachable by a branch as version;
- with the name of a branch as version;
- with a sha1 of the commit head of a branch as version;
- with a partial sha1 of the commit head of a branch as version;
Enforce the download always occurs by removing the BR2_DL_DIR used for
the tarballs generated by the git download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repo with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* 68c197d (mybranch) branch2
* 3174063 branch1
* a238b1d (HEAD -> master) initial
Changes v2 -> v3:
- complete rewrite using a static repo;
---
.../git-refs/package/git-branch/git-branch.hash | 2 ++
.../git-refs/package/git-branch/git-branch.mk | 11 +++++++++++
.../git-partial-sha1-branch-head.hash | 2 ++
.../git-partial-sha1-branch-head.mk | 11 +++++++++++
.../git-partial-sha1-reachable-by-branch.hash | 2 ++
.../git-partial-sha1-reachable-by-branch.mk | 11 +++++++++++
.../git-sha1-branch-head/git-sha1-branch-head.hash | 2 ++
.../package/git-sha1-branch-head/git-sha1-branch-head.mk | 11 +++++++++++
.../git-sha1-reachable-by-branch.hash | 2 ++
.../git-sha1-reachable-by-branch.mk | 11 +++++++++++
.../objects/11/93ff46343f4f6a0522e2b28b871e905178c1f0 | Bin 0 -> 23 bytes
.../objects/31/7406308d9259e2231bd0d6ddad3de3832bce08 | Bin 0 -> 182 bytes
.../objects/34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5 | Bin 0 -> 23 bytes
.../objects/68/c197d0879d485f4f6cee85544722b79e68e59f | Bin 0 -> 184 bytes
.../objects/b2/4b387624edc78d0292a127c43cad9ba97c6232 | Bin 0 -> 49 bytes
.../objects/e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e | Bin 0 -> 49 bytes
.../download/git-remote/repo.git/refs/heads/mybranch | 1 +
support/testing/tests/download/test_git.py | 12 +++++++++++-
18 files changed, 77 insertions(+), 1 deletion(-)
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.mk
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/11/93ff46343f4f6a0522e2b28b871e905178c1f0
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/31/7406308d9259e2231bd0d6ddad3de3832bce08
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/68/c197d0879d485f4f6cee85544722b79e68e59f
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/b2/4b387624edc78d0292a127c43cad9ba97c6232
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/mybranch
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.hash b/support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.hash
new file mode 100644
index 0000000000..0061618b0d
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.hash
@@ -0,0 +1,2 @@
+sha256 9697b68a51149188922b5df596f0f1f363c2abf9ee8e6feff0ec44b6bca3e69d git-branch-mybranch.tar.gz
+sha256 2c1126513651b0d346a4e6d1bb75ac1c9999217e18026302d27bea47b06c7fb2 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.mk b/support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.mk
new file mode 100644
index 0000000000..624d07aad1
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-branch
+#
+################################################################################
+
+GIT_BRANCH_VERSION = mybranch
+GIT_BRANCH_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_BRANCH_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash
new file mode 100644
index 0000000000..648bcceca0
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash
@@ -0,0 +1,2 @@
+sha256 70b76187369e47db69dac02c5696e63b35199cd20490fa473d289dd377774613 git-partial-sha1-branch-head-68c197d0879d485f4f6c.tar.gz
+sha256 2c1126513651b0d346a4e6d1bb75ac1c9999217e18026302d27bea47b06c7fb2 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.mk b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.mk
new file mode 100644
index 0000000000..6d4177c5bf
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-partial-sha1-branch-head
+#
+################################################################################
+
+GIT_PARTIAL_SHA1_BRANCH_HEAD_VERSION = 68c197d0879d485f4f6c
+GIT_PARTIAL_SHA1_BRANCH_HEAD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_PARTIAL_SHA1_BRANCH_HEAD_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash
new file mode 100644
index 0000000000..d064362e2c
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash
@@ -0,0 +1,2 @@
+sha256 9db079b9e215799d59975db7b2b26671eff1932ee6cf1075296f2ace3e2cb746 git-partial-sha1-reachable-by-branch-317406308d9259e2231b.tar.gz
+sha256 fabbc65c442bacb5e69b7adfea6d14fbbfc1327134322efd12771dc84387d507 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.mk b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.mk
new file mode 100644
index 0000000000..2f65b5c9a9
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-partial-sha1-reachable-by-branch
+#
+################################################################################
+
+GIT_PARTIAL_SHA1_REACHABLE_BY_BRANCH_VERSION = 317406308d9259e2231b
+GIT_PARTIAL_SHA1_REACHABLE_BY_BRANCH_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_PARTIAL_SHA1_REACHABLE_BY_BRANCH_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash
new file mode 100644
index 0000000000..3e8f76d31a
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash
@@ -0,0 +1,2 @@
+sha256 a21a2507b6d94ad484e49e3a9ae698f672a57469aab8e1779da77df7c9d4d337 git-sha1-branch-head-68c197d0879d485f4f6cee85544722b79e68e59f.tar.gz
+sha256 2c1126513651b0d346a4e6d1bb75ac1c9999217e18026302d27bea47b06c7fb2 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.mk b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.mk
new file mode 100644
index 0000000000..d3ecaa8593
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-sha1-branch-head
+#
+################################################################################
+
+GIT_SHA1_BRANCH_HEAD_VERSION = 68c197d0879d485f4f6cee85544722b79e68e59f
+GIT_SHA1_BRANCH_HEAD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SHA1_BRANCH_HEAD_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash
new file mode 100644
index 0000000000..f8d7b5dc48
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash
@@ -0,0 +1,2 @@
+sha256 8909e76d898e651af0bc23fae4103b87888bfe77448d71aaf1fbec3da97a3ce2 git-sha1-reachable-by-branch-317406308d9259e2231bd0d6ddad3de3832bce08.tar.gz
+sha256 fabbc65c442bacb5e69b7adfea6d14fbbfc1327134322efd12771dc84387d507 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.mk b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.mk
new file mode 100644
index 0000000000..badf9e13ca
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-sha1-reachable-by-branch
+#
+################################################################################
+
+GIT_SHA1_REACHABLE_BY_BRANCH_VERSION = 317406308d9259e2231bd0d6ddad3de3832bce08
+GIT_SHA1_REACHABLE_BY_BRANCH_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SHA1_REACHABLE_BY_BRANCH_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/11/93ff46343f4f6a0522e2b28b871e905178c1f0 b/support/testing/tests/download/git-remote/repo.git/objects/11/93ff46343f4f6a0522e2b28b871e905178c1f0
new file mode 100644
index 0000000000000000000000000000000000000000..3541cd14f0bf5f3a172f868c7ec730dcc255ebd2
GIT binary patch
literal 23
fcmb<m^geacKWIaOhnCkF?{fxEm>AT(SQY{RanuOb
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/31/7406308d9259e2231bd0d6ddad3de3832bce08 b/support/testing/tests/download/git-remote/repo.git/objects/31/7406308d9259e2231bd0d6ddad3de3832bce08
new file mode 100644
index 0000000000000000000000000000000000000000..c2c986572a7dec5f263e461155a09124d0f05844
GIT binary patch
literal 182
zcmV;n07?IN0j<tEP6II%Kv8P1!W)>#-;9k=lnv036+FLBVg-+p;}q<kNGa$M?J2G<
z_SWju-tAxJVAkwV*zA+69E${E??l+TU<sm!f}90XX7vSwx2eJTR4fTna*@JH$p;XR
z(K#RNiYZu2*^q+!y!4?zD#D<y-(k$UVZU9gej88RwO>4*8`RGo_IA)Vnl;DdW7doD
k#;AurrYHYXPL%;0UaV55dfu3pF0z#G*T2lF9~m)jQ^MU{p#T5?
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5 b/support/testing/tests/download/git-remote/repo.git/objects/34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5
new file mode 100644
index 0000000000000000000000000000000000000000..29f2d4fd004afa03b43e3148207e258e7aa9fc41
GIT binary patch
literal 23
fcmb<m^geacKWIaOhnCkF?{fx^m>AT3SQY^Qam@(V
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/68/c197d0879d485f4f6cee85544722b79e68e59f b/support/testing/tests/download/git-remote/repo.git/objects/68/c197d0879d485f4f6cee85544722b79e68e59f
new file mode 100644
index 0000000000000000000000000000000000000000..3e421bb97a40ab93592d94107f3a1258a99b76fa
GIT binary patch
literal 184
zcmV;p07w6L0j<tGP6IIzhGA--!WY=cJG1K_gb){?AtxBmOcE=2jf^R{Ju9V1AJIP2
z_vz(UYZ(^7KaH+xGb|Zn at UW<Gh~gjt%@BA&CP|r6U=Il9i at I+6a1M*}=%OT8lLFwJ
z#fcXoK~andnbk!@KjYH6{U}Ubnti7}%Es;cTI@@|;i2B~?OdsR9=O$4JFhOJHKIp5
mIdsnKx<?QFr#q87ZF~V^j^(_mOzm`<^6&E_vE~mXVr;h@Dp>;n
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/b2/4b387624edc78d0292a127c43cad9ba97c6232 b/support/testing/tests/download/git-remote/repo.git/objects/b2/4b387624edc78d0292a127c43cad9ba97c6232
new file mode 100644
index 0000000000000000000000000000000000000000..20491e3f176096db13c0b0c7857406e1c4119619
GIT binary patch
literal 49
zcmV-10M7q-0V^p=O;s>9VlXr?Ff%bxNXyJgWiYvTtI+!UJ(=J**;<>Gt?O at iAAbq}
H3k46<$LbTr
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e b/support/testing/tests/download/git-remote/repo.git/objects/e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e
new file mode 100644
index 0000000000000000000000000000000000000000..57be6485778c7f743a82f71ef718103cecfd4f35
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5RbbAa`)7~bv+oXld2!jhvYnibqU`nz3}1Tr
F^a08O6TJWc
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/heads/mybranch b/support/testing/tests/download/git-remote/repo.git/refs/heads/mybranch
new file mode 100644
index 0000000000..45cec54673
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/heads/mybranch
@@ -0,0 +1 @@
+68c197d0879d485f4f6cee85544722b79e68e59f
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index 29b5cd4091..aa95519e74 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -1,4 +1,5 @@
import os
+import shutil
import infra
from gitbase import GitTestBase
@@ -29,7 +30,11 @@ class TestGitRefs(GitTestBase):
def check_download(self, package):
# store downloaded tarball inside the output dir so the test infra
# cleans it up@the end
- env = {"BR2_DL_DIR": os.path.join(self.builddir, "dl"),
+ dl_dir = os.path.join(self.builddir, "dl")
+ # enforce we test the download even when using run-tests -k
+ if os.path.exists(dl_dir):
+ shutil.rmtree(dl_dir)
+ env = {"BR2_DL_DIR": dl_dir,
"GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
self.b.build(["{}-dirclean".format(package),
"{}-legal-info".format(package)],
@@ -40,3 +45,8 @@ class TestGitRefs(GitTestBase):
self.check_download("git-wrong-content")
with self.assertRaises(SystemError):
self.check_download("git-wrong-sha1")
+ self.check_download("git-branch")
+ self.check_download("git-partial-sha1-branch-head")
+ self.check_download("git-partial-sha1-reachable-by-branch")
+ self.check_download("git-sha1-branch-head")
+ self.check_download("git-sha1-reachable-by-branch")
--
2.14.2
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v3 9/9] testing/tests/download: test git submodules
2017-10-29 14:05 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
` (7 preceding siblings ...)
2017-10-29 14:06 ` [Buildroot] [PATCH v3 8/9] testing/tests/download: test git branch Ricardo Martincoski
@ 2017-10-29 14:06 ` Ricardo Martincoski
2018-04-25 20:58 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
10 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2017-10-29 14:06 UTC (permalink / raw)
To: buildroot
Add two submodules as static repos, add a branch to the main static repo
and check on the git refs test case the download of a git package:
- repo with submodule but without support in the package;
- repo with recursive submodules with support in the package.
For the test case with submodule support enabled, check the hash of
files in the checkout instead of the tarball since some versions of git
client do not generate reproducible tarballs when there are a second
level submodule.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repos with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* 2fa37f6 (submodule) sub0-2
* a9dbc1e sub0-1
| * 68c197d (mybranch) branch2
| * 3174063 branch1
|/
* a238b1d (HEAD -> master) initial
$ git -C support/testing/tests/download/git-remote/refs-sub1.git \
log --all --decorate --graph --oneline --decorate
* 1df823c (submodule) sub1-2
* f8001e5 sub1-1
* cb545fa (HEAD) initial
$ git -C support/testing/tests/download/git-remote/refs-sub2.git \
log --all --decorate --graph --oneline --decorate
* e83f6f8 (submodule) sub2-2
* cd4d62f sub2-1
* 32d61ba (HEAD) initial
In this layout:
- sub0-1 adds refs-sub1.git as submodule to refs.git pointing to sub1-1
commit;
- sub1-1 adds refs-sub2.git as submodule to refs-sub1.git pointing to
sub2-1 commit.
Changes v2 -> v3:
- complete rewrite using a static repo;
Changes v1 -> v2:
- rewrite using new git test infra;
- this patch is part of series 1/3 of a new version of
http://patchwork.ozlabs.org/patch/690097/
---
.../git-submodule-disabled.hash | 2 ++
.../git-submodule-disabled.mk | 11 ++++++++
.../git-submodule-enabled.hash | 3 +++
.../git-submodule-enabled/git-submodule-enabled.mk | 28 +++++++++++++++++++++
.../git-remote/refs-sub1.git/.gitattributes | 1 +
.../tests/download/git-remote/refs-sub1.git/HEAD | 1 +
.../tests/download/git-remote/refs-sub1.git/config | 4 +++
.../07/cabc655213bdf7087d8dd50fda95124e935570 | Bin 0 -> 22 bytes
.../1d/f823cb8a6d1866148ae50a8009762a9c4c777f | Bin 0 -> 180 bytes
.../7d/52f458bdb0f9f5a4beb46fa82824421b8b988e | Bin 0 -> 65 bytes
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a | Bin 0 -> 122 bytes
.../bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23 | Bin 0 -> 22 bytes
.../cb/545facf77bbc5f24f95b6d503c338d10b7b717 | Bin 0 -> 151 bytes
.../dd/130f6f4629514adaf2e03407f3ed9344eb6cd6 | Bin 0 -> 122 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../f8/001e5780100962a5e30a25cbc4c9609cfc7bf3 | Bin 0 -> 184 bytes
.../git-remote/refs-sub1.git/refs/heads/submodule | 1 +
.../git-remote/refs-sub2.git/.gitattributes | 1 +
.../tests/download/git-remote/refs-sub2.git/HEAD | 1 +
.../tests/download/git-remote/refs-sub2.git/config | 4 +++
.../0a/846af45c3e455789435f49f80d70e86b65b9d7 | Bin 0 -> 22 bytes
.../0b/32ebd8fc52cec991f18c94be980e85a8341585 | Bin 0 -> 22 bytes
.../32/d61bae693af7879da63b89a60d3ae67f851e56 | Bin 0 -> 151 bytes
.../3e/9b0a5198c64cea9c00f820433411e3b4d50c1c | Bin 0 -> 48 bytes
.../85/8f666af75b7c0dfba6b8be7eac5f196e7a1221 | Bin 0 -> 49 bytes
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb | Bin 0 -> 184 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../e8/3f6f805bd016b90acafc8702c52d778eb57310 | Bin 0 -> 182 bytes
.../git-remote/refs-sub2.git/refs/heads/submodule | 1 +
.../0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343 | Bin 0 -> 22 bytes
.../25/59d83bfe937fc0412d96ed664663c9e8a99055 | Bin 0 -> 22 bytes
.../2f/a37f6885d7eb746df75eccaddbacf3ac82799d | Bin 0 -> 180 bytes
.../8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7 | Bin 0 -> 65 bytes
.../a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f | Bin 0 -> 183 bytes
.../d4/e2941d18a63535400476324ddeb7f40164be41 | Bin 0 -> 121 bytes
.../fe/74231105841041d5f441e70399d37f0e600aa5 | Bin 0 -> 121 bytes
.../git-remote/repo.git/refs/heads/submodule | 1 +
support/testing/tests/download/test_git.py | 2 ++
40 files changed, 61 insertions(+)
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.mk
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/config
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/07/cabc655213bdf7087d8dd50fda95124e935570
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/1d/f823cb8a6d1866148ae50a8009762a9c4c777f
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/7d/52f458bdb0f9f5a4beb46fa82824421b8b988e
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/cb/545facf77bbc5f24f95b6d503c338d10b7b717
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/dd/130f6f4629514adaf2e03407f3ed9344eb6cd6
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/f8/001e5780100962a5e30a25cbc4c9609cfc7bf3
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/refs/heads/submodule
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/config
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/0a/846af45c3e455789435f49f80d70e86b65b9d7
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/0b/32ebd8fc52cec991f18c94be980e85a8341585
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/32/d61bae693af7879da63b89a60d3ae67f851e56
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/3e/9b0a5198c64cea9c00f820433411e3b4d50c1c
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/85/8f666af75b7c0dfba6b8be7eac5f196e7a1221
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/e8/3f6f805bd016b90acafc8702c52d778eb57310
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/refs/heads/submodule
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/25/59d83bfe937fc0412d96ed664663c9e8a99055
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/2f/a37f6885d7eb746df75eccaddbacf3ac82799d
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/d4/e2941d18a63535400476324ddeb7f40164be41
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/fe/74231105841041d5f441e70399d37f0e600aa5
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/submodule
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash
new file mode 100644
index 0000000000..3bd0a44693
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash
@@ -0,0 +1,2 @@
+sha256 f9d46ff8a1a344c6c31fa4211220f3085c446abd31626232540703158276f22c git-submodule-disabled-a9dbc1e23c45e8e1b88c0448763f54d714eb6f8f.tar.gz
+sha256 ba8b6ddc4726bfb6a05045ebfd8c43263c968ad1bc601bd46a25bc055008eddc file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.mk b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.mk
new file mode 100644
index 0000000000..7a35b3b51a
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-submodule-disabled
+#
+################################################################################
+
+GIT_SUBMODULE_DISABLED_VERSION = a9dbc1e23c45e8e1b88c0448763f54d714eb6f8f
+GIT_SUBMODULE_DISABLED_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SUBMODULE_DISABLED_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash
new file mode 100644
index 0000000000..a4ba6947c8
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash
@@ -0,0 +1,3 @@
+sha256 ba8b6ddc4726bfb6a05045ebfd8c43263c968ad1bc601bd46a25bc055008eddc file
+sha256 5219dcb50c7753bcdd72bc68a7b48af33ac2e42c5c61de78a9dd81589c4d50b6 refs-sub1/file
+sha256 b93c1fa9014d25a17fee36771d26ae023f043da656315ffe8947c30ad1ba141f refs-sub1/refs-sub2/file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.mk b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.mk
new file mode 100644
index 0000000000..019dd076e3
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.mk
@@ -0,0 +1,28 @@
+################################################################################
+#
+# git-submodule-enabled
+#
+################################################################################
+
+GIT_SUBMODULE_ENABLED_VERSION = a9dbc1e23c45e8e1b88c0448763f54d714eb6f8f
+GIT_SUBMODULE_ENABLED_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SUBMODULE_ENABLED_GIT_SUBMODULES = YES
+GIT_SUBMODULE_ENABLED_LICENSE_FILES = \
+ file \
+ refs-sub1/file \
+ refs-sub1/refs-sub2/file
+
+# Some versions of git client fill the .git file for the second level submodule
+# using the absolute path to the main .git directory, which in the case of the
+# buildroot download helper is always different since it uses a temporary
+# directory. This version of git have the issue:
+# - 2.7.4 included in Ubuntu 16.04;
+# The same does not occur using newer versions of git:
+# - 2.11.0 included in Debian 9;
+# - 2.14.2 latest at the time of writing;
+# In order to allow running this test case locally in many systems, do not check
+# for the hash of the tarball, but instead check the hash of each meaningful
+# file included in the tarball.
+BR_NO_CHECK_HASH_FOR += $(GIT_SUBMODULE_ENABLED_SOURCE)
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/.gitattributes b/support/testing/tests/download/git-remote/refs-sub1.git/.gitattributes
new file mode 100644
index 0000000000..eb50c64a21
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub1.git/.gitattributes
@@ -0,0 +1 @@
+objects/*/* binary
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/HEAD b/support/testing/tests/download/git-remote/refs-sub1.git/HEAD
new file mode 100644
index 0000000000..4e92c1f200
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub1.git/HEAD
@@ -0,0 +1 @@
+cb545facf77bbc5f24f95b6d503c338d10b7b717
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/config b/support/testing/tests/download/git-remote/refs-sub1.git/config
new file mode 100644
index 0000000000..07d359d07c
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub1.git/config
@@ -0,0 +1,4 @@
+[core]
+ repositoryformatversion = 0
+ filemode = true
+ bare = true
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/07/cabc655213bdf7087d8dd50fda95124e935570 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/07/cabc655213bdf7087d8dd50fda95124e935570
new file mode 100644
index 0000000000000000000000000000000000000000..cbf5085af8cdca19c43017c92369ddc8002f1a3d
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXfis>UJ21Qop+W=aH2UY+8
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/1d/f823cb8a6d1866148ae50a8009762a9c4c777f b/support/testing/tests/download/git-remote/refs-sub1.git/objects/1d/f823cb8a6d1866148ae50a8009762a9c4c777f
new file mode 100644
index 0000000000000000000000000000000000000000..04055b899ecbdc3bceb2d7936757f360e90544a5
GIT binary patch
literal 180
zcmV;l089UP0j-Wf4#FT106p^xK2WpJg47t}2fXMDb{9~h6hrag?``72yYV{7OomI&
zsmWfORjVOLR6Sr+R$C(U-VSb1q?wp3@(?`O-VrJk53FHoGCBYi%*KHN7_CMVmVh>k
zG4N<%<j5O}y})O?mMTw)vHBv9SX-L7%(o<W^#j-W3lEo!>AvQY_fi>fV64+lcAx>o
iLYL_l{*)8*lT at dLIH$`Ld{<)Z=(YXjRlEVZ`fTHP_gG8-
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/7d/52f458bdb0f9f5a4beb46fa82824421b8b988e b/support/testing/tests/download/git-remote/refs-sub1.git/objects/7d/52f458bdb0f9f5a4beb46fa82824421b8b988e
new file mode 100644
index 0000000000000000000000000000000000000000..2b78a01965068cc6f7c89402eea4e7f9a03ba807
GIT binary patch
literal 65
zcmV-H0KWft0ZYosPf{>5V~8#;P0GzrDa}b$P%27IE7k?_jg(@!I13U>G8AkT;9^{y
XrA0YF0X;o^xR_pgW(gMne83k<GdCCZ
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab b/support/testing/tests/download/git-remote/refs-sub1.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
new file mode 100644
index 0000000000000000000000000000000000000000..9db72668cf9374b0b85a25a19f30084fd460072d
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5wZcw)UHz=nDZYB`lCt5lYr9_mN?Omvz#_so
F4FHVv5b^*3
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a b/support/testing/tests/download/git-remote/refs-sub1.git/objects/a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a
new file mode 100644
index 0000000000000000000000000000000000000000..f49d53326e0b22b95fd16670f0a25e3cd84718ae
GIT binary patch
literal 122
zcmV-=0EPc}0V^p=O;s>7uwXDWFfcPQQP4}zEXmDJDa}bOW~dGN60vu~&#z1NZOLDu
zq2eUnJ);k*EG;u9m4W@#p41 at Wz27-%d$01}nkwWsIkdpg%m4@!ic-^xb&E at rj2O=P
cCjD3J+P-ChfArrU3~L^=ueW&(03qxzkxyhgmjD0&
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23
new file mode 100644
index 0000000000000000000000000000000000000000..7d2a9d55c1a92dfa2122308cc05a4a5b96d70d8f
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXfis=(321RD(TL4;&2UP$7
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/cb/545facf77bbc5f24f95b6d503c338d10b7b717 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/cb/545facf77bbc5f24f95b6d503c338d10b7b717
new file mode 100644
index 0000000000000000000000000000000000000000..dc7de29459f4156b999f14fdeb6cb58b58c21a19
GIT binary patch
literal 151
zcmV;I0BHYs0j<ux4Z<)GKw)Z6aSJ3mJF*iagjj$ECOH3v6LBmVQ?NaeDCiOGS3Esm
za^?nVoOZ3o2-YSQhbX<1I$5VxB4v{?VVE!kLL-BfBRN6Ob}JPw?5TzVXR3`?Umkac
zL;YZ%zj&H6@zVQ}k03QpsEsrT9V$ez={C{gpK^j%ZcK|1SDx1xx{^}oUi)8W#T!=;
FU>B}CKEMC~
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/dd/130f6f4629514adaf2e03407f3ed9344eb6cd6 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/dd/130f6f4629514adaf2e03407f3ed9344eb6cd6
new file mode 100644
index 0000000000000000000000000000000000000000..37037d3920820ae2dab7e40d0c1df612af6f7a94
GIT binary patch
literal 122
zcmV-=0EPc}0V^p=O;s>7uwXDWFfcPQQP4}zEXmDJDa}bOW~dGN60vu~&#z1NZOLDu
zq2eUnJ);k*EG;u9m0`~b>19v%c7N5Gn$h**^0Vm=4Sy*cni&9rLQ!g3v2Jl`k`cpM
c-=zPFUE8-T at Q?ocgJI2s_VqTe0l at My>J95W4gdfE
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
new file mode 100644
index 0000000000000000000000000000000000000000..df2037a2d2d30afb20e31cd558c92c2edabe7cf6
GIT binary patch
literal 23
fcmb<m^geacKWIb3i8H5lPWqgA$i$$b%kl;Qg;xrl
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/f8/001e5780100962a5e30a25cbc4c9609cfc7bf3 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/f8/001e5780100962a5e30a25cbc4c9609cfc7bf3
new file mode 100644
index 0000000000000000000000000000000000000000..d824d3c17c1cc1c13a18eb1d2c0a7e59f66ba08b
GIT binary patch
literal 184
zcmV;p07w6L0j<tGP6IIzKw)a1!WY<R?D5YoLWm2{Bq#8AJi!95k>eEHo=7RuBigTc
zdbzh&r$+0a<|suo8w3gBg5A7viIdm@=pYIf-q5*1A#+gQX_S5HOx}4(Sz;v0UaXVN
zQ(zC65mz=qk)nwzUFSz1`dD)s-1U1Jv+lWHKC6BicRaKQzMWgDUx(b=t2Q21@L>hi
m3s?Yk)7$hc|CCdgE1AV8b*ksR at X|#V`g#7#tNI184{X$IS6W&C
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/refs/heads/submodule b/support/testing/tests/download/git-remote/refs-sub1.git/refs/heads/submodule
new file mode 100644
index 0000000000..bf97dc4185
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub1.git/refs/heads/submodule
@@ -0,0 +1 @@
+1df823cb8a6d1866148ae50a8009762a9c4c777f
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/.gitattributes b/support/testing/tests/download/git-remote/refs-sub2.git/.gitattributes
new file mode 100644
index 0000000000..eb50c64a21
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub2.git/.gitattributes
@@ -0,0 +1 @@
+objects/*/* binary
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/HEAD b/support/testing/tests/download/git-remote/refs-sub2.git/HEAD
new file mode 100644
index 0000000000..bbf19a4bb4
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub2.git/HEAD
@@ -0,0 +1 @@
+32d61bae693af7879da63b89a60d3ae67f851e56
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/config b/support/testing/tests/download/git-remote/refs-sub2.git/config
new file mode 100644
index 0000000000..07d359d07c
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub2.git/config
@@ -0,0 +1,4 @@
+[core]
+ repositoryformatversion = 0
+ filemode = true
+ bare = true
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/0a/846af45c3e455789435f49f80d70e86b65b9d7 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/0a/846af45c3e455789435f49f80d70e86b65b9d7
new file mode 100644
index 0000000000000000000000000000000000000000..aaa29dbd8a5a74a24f51a8522b695a7474349f3c
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXflIasB21Rz}+W=aJ2Uq|A
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/0b/32ebd8fc52cec991f18c94be980e85a8341585 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/0b/32ebd8fc52cec991f18c94be980e85a8341585
new file mode 100644
index 0000000000000000000000000000000000000000..50caf8b6282d2287878d1b3481c14431c2a557af
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXflIbHR21QQhI{;dM2U!3B
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/32/d61bae693af7879da63b89a60d3ae67f851e56 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/32/d61bae693af7879da63b89a60d3ae67f851e56
new file mode 100644
index 0000000000000000000000000000000000000000..a8887b1e611d8f76d7cfe6ab0bb693fdebc2ef19
GIT binary patch
literal 151
zcmV;I0BHYs0j<ux4Z<)GKw)Z6aSJ3mJF*iagjj$ECOH3v6LBmVQ?NaeDCiOGS3Esm
za^?nVoOZ3o2-YSQhbX<1I$5VxB4v{?VVE!kLL-BfBRN6Ob}JPw?5TzVXR3`?Umkac
zL;YZ%zj&H6 at zVQ}k03QpsEr&gbf^%;rrSh|f6577xiKw5TzOt&=t@eRd+mRj6>nh7
FU at NX0KJNek
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/3e/9b0a5198c64cea9c00f820433411e3b4d50c1c b/support/testing/tests/download/git-remote/refs-sub2.git/objects/3e/9b0a5198c64cea9c00f820433411e3b4d50c1c
new file mode 100644
index 0000000000000000000000000000000000000000..b1fb722c53d827847c6d13029e601a5434b84ac4
GIT binary patch
literal 48
zcmV-00MGw;0V^p=O;s>9VlXr?Ff%bxNXyJgW#DSb`VwR38s6y~@A-qb;6--o&g%g8
GZVq_HQWiM?
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/85/8f666af75b7c0dfba6b8be7eac5f196e7a1221 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/85/8f666af75b7c0dfba6b8be7eac5f196e7a1221
new file mode 100644
index 0000000000000000000000000000000000000000..10f24a087ff08320b8fc5922db728472bd276e9a
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5^?_Mc?PLE7J0}a<fAhI%-n_oFwKGGTLGrBt
FHvqLX67T>3
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab b/support/testing/tests/download/git-remote/refs-sub2.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
new file mode 100644
index 0000000000000000000000000000000000000000..9db72668cf9374b0b85a25a19f30084fd460072d
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5wZcw)UHz=nDZYB`lCt5lYr9_mN?Omvz#_so
F4FHVv5b^*3
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb b/support/testing/tests/download/git-remote/refs-sub2.git/objects/cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb
new file mode 100644
index 0000000000000000000000000000000000000000..35d253f0d15b31ea0bc79ed1a287819deed9987d
GIT binary patch
literal 184
zcmV;p07w6L0j<tGZUZ3@hGEy7f(xuQEI$j8W%&YWk`oNWU?YdcVsVk%XQLG9qqL9o
zeR`?YT87lKJ&evQHHQdjRq*5tF$Mz2eg>npQHpgm$pR at 75A3{t50g_7*#(Vx at k>bP
zbo2ooB)#*Jy;W?Tz~{KNF5e1aPffmJA7!Ps-HSZ+A3WAyyj&}mk0Z5umdb($>r7bW
m1T%oxb&nqWPj_N_#(7dgyqD|BX=*u7@$=;`t>O#gcx)*r3s*J(
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
new file mode 100644
index 0000000000000000000000000000000000000000..df2037a2d2d30afb20e31cd558c92c2edabe7cf6
GIT binary patch
literal 23
fcmb<m^geacKWIb3i8H5lPWqgA$i$$b%kl;Qg;xrl
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/e8/3f6f805bd016b90acafc8702c52d778eb57310 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/e8/3f6f805bd016b90acafc8702c52d778eb57310
new file mode 100644
index 0000000000000000000000000000000000000000..05c04d2887e9354ec43ad69cef20ed4e7166013d
GIT binary patch
literal 182
zcmV;n07?IN0j<tKPD3FOfMM61f(yK6pl<+;F<!upo?r%MT2gr!N*CVVCN8>fy8RY^
z{@h!u)9yV!%^_kFCJiBwM&y{GC{y6X5`{9VdkzxGdGE{*4cVs6Wi7$0dY6bJuLP?i
zYtaNiSs+EyeHJ!!na4itXU#N9w;wcS-Eu#ls(l^*xNA3jKQyY}yWHEGbqEPCphLTW
k2QXK?PEUDI&YUmo7jMj0J#12zuDbB=%RR5=4+?N at DHiEdA^-pY
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/refs/heads/submodule b/support/testing/tests/download/git-remote/refs-sub2.git/refs/heads/submodule
new file mode 100644
index 0000000000..39dc53f019
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub2.git/refs/heads/submodule
@@ -0,0 +1 @@
+e83f6f805bd016b90acafc8702c52d778eb57310
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343 b/support/testing/tests/download/git-remote/repo.git/objects/0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343
new file mode 100644
index 0000000000000000000000000000000000000000..ef7fbd4154aea755d8ee2684e420700467e9245c
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXfg6R_`28I93HvwH&2ulC}
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/25/59d83bfe937fc0412d96ed664663c9e8a99055 b/support/testing/tests/download/git-remote/repo.git/objects/25/59d83bfe937fc0412d96ed664663c9e8a99055
new file mode 100644
index 0000000000000000000000000000000000000000..cbf60fc7298ec12b3d7cd388fecac606ca31536e
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXfg6ShB21Q2ZTL4;A2TcF~
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/2f/a37f6885d7eb746df75eccaddbacf3ac82799d b/support/testing/tests/download/git-remote/repo.git/objects/2f/a37f6885d7eb746df75eccaddbacf3ac82799d
new file mode 100644
index 0000000000000000000000000000000000000000..2dd14580e666d35483727598f006148f3eeb03be
GIT binary patch
literal 180
zcmV;l089UP0j-Wp4#FT509|tmF3{!yj~0zFUcil>fX_pTQVgXFZ*LP9?v2|?W-`9y
zoEleBZ(1ed0ZDFA1Yzu;w9*KG<DevlKyHi(Vn9bI*wa>;8n at fvydbGOYBD68F&+ at j
z>O_rU6i9A}CNjIUTdDk<yse?|ldUaHzRdT;_w at t&{DsGBw&~IPk`G*HFrddZ%R7(&
iY^CdTqd(=?yg2A2W0!QDD0C&p&b@ZOys|fCz-zg3O;?Bj
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7 b/support/testing/tests/download/git-remote/repo.git/objects/8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7
new file mode 100644
index 0000000000000000000000000000000000000000..70cf1795f3210f81951d0995823bb958867ba710
GIT binary patch
literal 65
zcmb<m^geacKghs1xkFFS_taT`&9k1N0a{+Z+Sgb=@lH+JX~8YzZ6uN-`XDK0&Qe!P
V2S!`l+8s0Es?s>c8ET5e-2otN7bXAz
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f b/support/testing/tests/download/git-remote/repo.git/objects/a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f
new file mode 100644
index 0000000000000000000000000000000000000000..2cbdce1f95493b284df1f8ca3ca953a5fb813072
GIT binary patch
literal 183
zcmV;o07(CM0j<t4ZUZ3@K+)8kf(xuQ14Ce~D9Q!g#3z_#z(F=kgmK~9XC)Wzt+cQB
z`mxnohPKANjh>Ru$&;~&-~u5KipxZpJlJ^_>ocTx0A{Ctxu<n#Ha09MIZ7~2$V;%q
z*+yrKCG>Htpv)kOZ{w$R{aqq=Y5Kr at lr^^Nx#*Anz+HXf*Qs*3>|(2*8XW|19*ox$
l7yz~DHa*fm<<#}Iz=TR2%V|wAwVbErzJKLa{Q<P{YY&#BUEBZw
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/d4/e2941d18a63535400476324ddeb7f40164be41 b/support/testing/tests/download/git-remote/repo.git/objects/d4/e2941d18a63535400476324ddeb7f40164be41
new file mode 100644
index 0000000000000000000000000000000000000000..4d82169300d0d8a2e31ced481501bff6647ef6b7
GIT binary patch
literal 121
zcmV-<0EYi~0V^p=O;s>7uwXDWFfcPQQP4}zEXmDJDa}bOX6Wv!>xg2JU9K(na!F0}
zWQL7JE1yG^rDf)%GN?x0u>Lo>{(z(Iw6|$)$tPc|oDgbgW&i{VMX71Uy2YhQh73O#
b<iZ;SIFpt>=2AU<<YdB}Kh>WBpF}NR{H!<P
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/fe/74231105841041d5f441e70399d37f0e600aa5 b/support/testing/tests/download/git-remote/repo.git/objects/fe/74231105841041d5f441e70399d37f0e600aa5
new file mode 100644
index 0000000000000000000000000000000000000000..6d0ae484c3333d6b18cf726d5bb468669f9d16c1
GIT binary patch
literal 121
zcmV-<0EYi~0V^p=O;s>7uwXDWFfcPQQP4}zEXmDJDa}bOX6Wv!>xg2JU9K(na!F0}
zWQL7JE1yG^rDf)%GH~BGw=?nZrcDChywb1Md&gYMyzFdfW&i{VMX71Uy2YhQh73O#
b<iZ;SIFpt>=2AU<<YdB}Kh>WBwy!O?8)rGE
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/heads/submodule b/support/testing/tests/download/git-remote/repo.git/refs/heads/submodule
new file mode 100644
index 0000000000..e9f4f32c30
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/heads/submodule
@@ -0,0 +1 @@
+2fa37f6885d7eb746df75eccaddbacf3ac82799d
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index aa95519e74..2b0bd18a2c 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -50,3 +50,5 @@ class TestGitRefs(GitTestBase):
self.check_download("git-partial-sha1-reachable-by-branch")
self.check_download("git-sha1-branch-head")
self.check_download("git-sha1-reachable-by-branch")
+ self.check_download("git-submodule-disabled")
+ self.check_download("git-submodule-enabled")
--
2.14.2
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v3 1/9] testing/infra/builder: call make with empty env
2017-10-29 14:06 ` [Buildroot] [PATCH v3 1/9] testing/infra/builder: call make with empty env Ricardo Martincoski
@ 2018-04-01 17:58 ` Thomas Petazzoni
0 siblings, 0 replies; 108+ messages in thread
From: Thomas Petazzoni @ 2018-04-01 17:58 UTC (permalink / raw)
To: buildroot
Hello,
On Sun, 29 Oct 2017 12:06:00 -0200, Ricardo Martincoski wrote:
> From: Ricardo Martincoski <ricardo.martincoski@gmail.com>
>
> Use an empty environment when calling make, but import PATH so the
> buildroot tree under test can find libraries from the host machine.
>
> Since environment variables are now ignored, move the handling of
> BR2_DL_DIR to the defconfig to keep the current precedence of -d:
>
> BR2_DL_DIR | -d DIR | test downloads | BR downloads
> ------------+----------+------------------+--------------
> unset | unset | [error] | [error]
> unset | set | in $(DIR) | in $(DIR)
> set | unset | in $(BR2_DL_DIR) | in $(BR2_DL_DIR)
> set | set | in $(DIR) | in $(DIR)
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Yann E. MORIN <yann.morin.1998@free.fr>
> ---
> Changes v2 -> v3:
> - new patch
> - search for "start with an empty environment" in
> http://patchwork.ozlabs.org/patch/806159/
> ---
> support/testing/infra/basetest.py | 1 +
> support/testing/infra/builder.py | 7 +++++--
> support/testing/run-tests | 1 -
> 3 files changed, 6 insertions(+), 3 deletions(-)
Applied to master, thanks.
Thomas
--
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v3 2/9] testing/infra/builder: split configure() from build()
2017-10-29 14:06 ` [Buildroot] [PATCH v3 2/9] testing/infra/builder: split configure() from build() Ricardo Martincoski
@ 2018-04-01 17:59 ` Thomas Petazzoni
2018-04-01 21:32 ` Ricardo Martincoski
0 siblings, 1 reply; 108+ messages in thread
From: Thomas Petazzoni @ 2018-04-01 17:59 UTC (permalink / raw)
To: buildroot
Hello,
On Sun, 29 Oct 2017 12:06:01 -0200, Ricardo Martincoski wrote:
> Some test cases don't use a full build as setup, so split the build()
> method into configure() and build().
> It allows a test case to perform configuration at the setup stage and
> the build inside the test itself.
>
> Call this new method just before build in the BRTest base class, to keep
> the current behavior for existing test cases.
>
> This change will be needed when adding a common class to test the git
> download infra.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
I've applied, but I have a comment, see below.
> + def build(self):
> + env = {"PATH": os.environ["PATH"]}
So now, we have the same local "env" variable in configure() and
build(). Would it make sense to do:
self.env = ...
in the constructor, and then use it in configure() and build() ?
Thanks!
Thomas
--
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v3 2/9] testing/infra/builder: split configure() from build()
2018-04-01 17:59 ` Thomas Petazzoni
@ 2018-04-01 21:32 ` Ricardo Martincoski
2018-04-01 21:37 ` Thomas Petazzoni
0 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-01 21:32 UTC (permalink / raw)
To: buildroot
Hello,
On Sun, Apr 01, 2018 at 02:59 PM, Thomas Petazzoni wrote:
> On Sun, 29 Oct 2017 12:06:01 -0200, Ricardo Martincoski wrote:
> I've applied, but I have a comment, see below.
>
>
>> + def build(self):
>> + env = {"PATH": os.environ["PATH"]}
>
> So now, we have the same local "env" variable in configure() and
> build(). Would it make sense to do:
>
> self.env = ...
>
> in the constructor, and then use it in configure() and build() ?
It is possible to do that.
The next patch on the same series uses .update() so the local "env" is changed.
env = {"PATH": os.environ["PATH"]}
env.update(make_extra_env)
Note: I did not tested the code below.
In order to always have a clean environment, we could do in the constructor:
self.env = {"PATH": os.environ["PATH"]}
And change the methods to:
env = self.env.copy()
env.update(make_extra_env)
Or we assume we will need only PATH from env in the future, and do in the
constructor:
self.env_path = os.environ["PATH"]
And change the methods to:
env = self.make_extra_env.copy()
env.update({"PATH": self.env_path})
The access to os.environ should be inexpensive (I did not measured it), so I am
not sure it needs to be done. I am not against it either.
At first glance, it seems just more code to me.
Maybe I am missing something.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v3 2/9] testing/infra/builder: split configure() from build()
2018-04-01 21:32 ` Ricardo Martincoski
@ 2018-04-01 21:37 ` Thomas Petazzoni
0 siblings, 0 replies; 108+ messages in thread
From: Thomas Petazzoni @ 2018-04-01 21:37 UTC (permalink / raw)
To: buildroot
Hello,
On Sun, 01 Apr 2018 18:32:01 -0300, Ricardo Martincoski wrote:
> It is possible to do that.
>
> The next patch on the same series uses .update() so the local "env" is changed.
> env = {"PATH": os.environ["PATH"]}
> env.update(make_extra_env)
>
> Note: I did not tested the code below.
>
> In order to always have a clean environment, we could do in the constructor:
> self.env = {"PATH": os.environ["PATH"]}
> And change the methods to:
> env = self.env.copy()
> env.update(make_extra_env)
>
> Or we assume we will need only PATH from env in the future, and do in the
> constructor:
> self.env_path = os.environ["PATH"]
> And change the methods to:
> env = self.make_extra_env.copy()
> env.update({"PATH": self.env_path})
>
> The access to os.environ should be inexpensive (I did not measured it), so I am
> not sure it needs to be done. I am not against it either.
> At first glance, it seems just more code to me.
> Maybe I am missing something.
Hum, indeed. Let's see what it gives with the next patches, and we can
always adjust that kind of details later.
Thanks for your feedback!
Thomas
--
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3)
2017-10-29 14:05 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
` (8 preceding siblings ...)
2017-10-29 14:06 ` [Buildroot] [PATCH v3 9/9] testing/tests/download: test git submodules Ricardo Martincoski
@ 2018-04-25 20:58 ` Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
10 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-25 20:58 UTC (permalink / raw)
To: buildroot
This week I will send a new iteration of this series after refreshing it in
current master (with the new download infra) and adding a few patches, so I am
marking it as Changes Requested.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v4] tests for git download infra v4
2017-10-29 14:05 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
` (9 preceding siblings ...)
2018-04-25 20:58 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
@ 2018-04-29 14:33 ` Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/infra/builder: build with target and environment Ricardo Martincoski
` (14 more replies)
10 siblings, 15 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-29 14:33 UTC (permalink / raw)
To: buildroot
Hello,
This series adds automated tests for the git download infra.
Changes v3 -> v4:
- the first 2 patches from v3 got applied;
- refresh the series in current master (with new download infra);
- add few patches I was holding back to send later;
- allow the use of multiple br2-external trees, simplifying patch 3;
- many code improvements to patch 4;
The detailed changelog is part of each patch.
This series does not yet test the cache function, all test cases run with an
empty git cache, but it is a step towards it.
The first 3 patches prepare the test infra to have the git test cases.
Patch 4 adds a test case that ensures hash checking is enabled for git packages.
Remaining patches, except the last one, add a test case that checks each type of
git ref (tag, sha1, ...) that can be used as version for a package with download
method git.
I created a single test case because:
- for all ref types it only takes 3 minutes to run;
- the feature under test is OK if all kinds of supported refs work.
But I split it in many patches because a static git repo is used, which is not
trivial to review. So each patch adds the needed git objects and the associated
.mk and .hash files. This way any rework/drop/rebase/revert is easier.
In order to make review easier I also added to the comment area of each patch
(after ---) the layout of the static repo with all patches until that one
applied (generated by git log --graph).
Last patch is a RFC to detect regressions after the bug found by Luca becomes
fixed by Yann's series. See:
http://lists.busybox.net/pipermail/buildroot/2018-April/219627.html
http://lists.busybox.net/pipermail/buildroot/2018-April/219665.html
Here a complete run on Gitlab CI, with the series applied on master (7a801da8e):
https://gitlab.com/RicardoMartincoski/buildroot/pipelines/21247818
Regards,
Ricardo
Ricardo Martincoski (12):
testing/infra/builder: build with target and environment
testing/infra: split runtime test from BRTest
testing/infra/basetest: support br2-external
testing/tests/download: add git hash test
testing/tests/download: test case for git refs
testing/tests/download: test git branch
testing/tests/download: test git submodules
testing/tests/download: test git tag
testing/tests/download: test git tag/branch precedence
testing/tests/download: test git special ref
testing/tests/download: test git branch with slash
support/testing: test extra download with site method git
.gitlab-ci.yml | 3 +
support/testing/infra/basetest.py | 13 +--
support/testing/infra/builder.py | 38 +++++++-
support/testing/infra/runtimetest.py | 23 +++++
support/testing/infra/server.py | 39 ++++++++
support/testing/tests/boot/test_atf.py | 7 +-
support/testing/tests/core/test_post_scripts.py | 3 +-
support/testing/tests/core/test_rootfs_overlay.py | 3 +-
support/testing/tests/core/test_timezone.py | 7 +-
support/testing/tests/download/__init__.py | 0
.../br2-external/git-extra-download/Config.in | 0
.../br2-external/git-extra-download/external.desc | 1 +
.../br2-external/git-extra-download/external.mk | 6 ++
.../package/extra-fails/extra-fails.hash | 3 +
.../package/extra-fails/extra-fails.mk | 12 +++
.../package/main-fails/main-fails.hash | 3 +
.../package/main-fails/main-fails.mk | 12 +++
.../git-extra-download/package/ok/ok.hash | 3 +
.../git-extra-download/package/ok/ok.mk | 12 +++
.../tests/download/br2-external/git-hash/Config.in | 0
.../download/br2-external/git-hash/external.desc | 1 +
.../download/br2-external/git-hash/external.mk | 4 +
.../br2-external/git-hash/package/bad/bad.hash | 1 +
.../br2-external/git-hash/package/bad/bad.mk | 10 ++
.../br2-external/git-hash/package/good/good.hash | 1 +
.../br2-external/git-hash/package/good/good.mk | 10 ++
.../br2-external/git-hash/package/nohash/nohash.mk | 10 ++
.../tests/download/br2-external/git-refs/Config.in | 0
.../download/br2-external/git-refs/external.desc | 1 +
.../download/br2-external/git-refs/external.mk | 4 +
.../git-branch-takes-precedence-over-tag.hash | 3 +
.../git-branch-takes-precedence-over-tag.mk | 11 +++
.../git-branch-with-slash.hash | 2 +
.../git-branch-with-slash/git-branch-with-slash.mk | 11 +++
.../git-refs/package/git-branch/git-branch.hash | 2 +
.../git-refs/package/git-branch/git-branch.mk | 11 +++
.../git-partial-sha1-branch-head.hash | 2 +
.../git-partial-sha1-branch-head.mk | 11 +++
.../git-partial-sha1-reachable-by-branch.hash | 2 +
.../git-partial-sha1-reachable-by-branch.mk | 11 +++
.../git-partial-sha1-reachable-by-tag.hash | 2 +
.../git-partial-sha1-reachable-by-tag.mk | 11 +++
.../git-partial-sha1-tag-itself.hash | 2 +
.../git-partial-sha1-tag-itself.mk | 11 +++
.../git-partial-sha1-tag-points-to.hash | 2 +
.../git-partial-sha1-tag-points-to.mk | 11 +++
.../git-sha1-branch-head/git-sha1-branch-head.hash | 2 +
.../git-sha1-branch-head/git-sha1-branch-head.mk | 11 +++
.../git-sha1-reachable-by-branch.hash | 2 +
.../git-sha1-reachable-by-branch.mk | 11 +++
.../git-sha1-reachable-by-tag.hash | 2 +
.../git-sha1-reachable-by-tag.mk | 11 +++
.../git-sha1-tag-itself/git-sha1-tag-itself.hash | 2 +
.../git-sha1-tag-itself/git-sha1-tag-itself.mk | 11 +++
.../git-sha1-tag-points-to.hash | 2 +
.../git-sha1-tag-points-to.mk | 11 +++
.../package/git-special-ref/git-special-ref.hash | 2 +
.../package/git-special-ref/git-special-ref.mk | 11 +++
.../git-submodule-disabled.hash | 2 +
.../git-submodule-disabled.mk | 11 +++
.../git-submodule-enabled.hash | 4 +
.../git-submodule-enabled/git-submodule-enabled.mk | 28 ++++++
.../git-refs/package/git-tag/git-tag.hash | 2 +
.../git-refs/package/git-tag/git-tag.mk | 11 +++
.../git-wrong-content/git-wrong-content.hash | 2 +
.../package/git-wrong-content/git-wrong-content.mk | 11 +++
.../package/git-wrong-sha1/git-wrong-sha1.mk | 11 +++
.../git-remote/refs-sub1.git/.gitattributes | 1 +
.../tests/download/git-remote/refs-sub1.git/HEAD | 1 +
.../tests/download/git-remote/refs-sub1.git/config | 4 +
.../07/cabc655213bdf7087d8dd50fda95124e935570 | Bin 0 -> 22 bytes
.../1d/f823cb8a6d1866148ae50a8009762a9c4c777f | Bin 0 -> 180 bytes
.../7d/52f458bdb0f9f5a4beb46fa82824421b8b988e | Bin 0 -> 65 bytes
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a | Bin 0 -> 122 bytes
.../bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23 | Bin 0 -> 22 bytes
.../cb/545facf77bbc5f24f95b6d503c338d10b7b717 | Bin 0 -> 151 bytes
.../dd/130f6f4629514adaf2e03407f3ed9344eb6cd6 | Bin 0 -> 122 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../f8/001e5780100962a5e30a25cbc4c9609cfc7bf3 | Bin 0 -> 184 bytes
.../git-remote/refs-sub1.git/refs/heads/submodule | 1 +
.../git-remote/refs-sub2.git/.gitattributes | 1 +
.../tests/download/git-remote/refs-sub2.git/HEAD | 1 +
.../tests/download/git-remote/refs-sub2.git/config | 4 +
.../0a/846af45c3e455789435f49f80d70e86b65b9d7 | Bin 0 -> 22 bytes
.../0b/32ebd8fc52cec991f18c94be980e85a8341585 | Bin 0 -> 22 bytes
.../32/d61bae693af7879da63b89a60d3ae67f851e56 | Bin 0 -> 151 bytes
.../3e/9b0a5198c64cea9c00f820433411e3b4d50c1c | Bin 0 -> 48 bytes
.../85/8f666af75b7c0dfba6b8be7eac5f196e7a1221 | Bin 0 -> 49 bytes
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb | Bin 0 -> 184 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../e8/3f6f805bd016b90acafc8702c52d778eb57310 | Bin 0 -> 182 bytes
.../git-remote/refs-sub2.git/refs/heads/submodule | 1 +
.../download/git-remote/repo.git/.gitattributes | 1 +
.../tests/download/git-remote/repo.git/HEAD | 1 +
.../tests/download/git-remote/repo.git/config | 4 +
.../0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343 | Bin 0 -> 22 bytes
.../11/93ff46343f4f6a0522e2b28b871e905178c1f0 | Bin 0 -> 23 bytes
.../17/c409592968d17600ff9912e96fdd461bb72e74 | Bin 0 -> 48 bytes
.../25/59d83bfe937fc0412d96ed664663c9e8a99055 | Bin 0 -> 22 bytes
.../2b/0e0d98a49c97da6a618ab36337e2058eb733a2 | Bin 0 -> 137 bytes
.../2f/a37f6885d7eb746df75eccaddbacf3ac82799d | Bin 0 -> 180 bytes
.../31/7406308d9259e2231bd0d6ddad3de3832bce08 | Bin 0 -> 182 bytes
.../34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5 | Bin 0 -> 23 bytes
.../42/c7c82cc86a07a73cb4499345eff2eccfe668bc | Bin 0 -> 19 bytes
.../46/bae5b639e5a18e2cc4dc508f080d566baeff59 | Bin 0 -> 182 bytes
.../47/3b7c2f803ad64eac57695ff57a5a6a8b34cbdd | Bin 0 -> 49 bytes
.../51/6c9c5f64ec66534d4d069c2e408d9ae4dce023 | Bin 0 -> 182 bytes
.../53/122a7b0454b33c6d9e159c10657173db77899e | Bin 0 -> 27 bytes
.../68/c197d0879d485f4f6cee85544722b79e68e59f | Bin 0 -> 184 bytes
.../80/858c1ab821392fd59a897dbaedd7d07e1ac403 | Bin 0 -> 22 bytes
.../83/e0f402891d9de91e7007f8534884ec4b9c7c2c | Bin 0 -> 183 bytes
.../8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7 | Bin 0 -> 65 bytes
.../8e/a6edb6df557ee935de556113bab2b5f9ecbabf | Bin 0 -> 49 bytes
.../92/ef85be57d627f280d8ce3724452ac21c9a6452 | Bin 0 -> 20 bytes
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../a2/38b1dfcd825d47d834af3c5223417c8411d90d | Bin 0 -> 152 bytes
.../a8/f73649910ce50c48626de0329a0174263ebb5b | Bin 0 -> 183 bytes
.../a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f | Bin 0 -> 183 bytes
.../b2/4b387624edc78d0292a127c43cad9ba97c6232 | Bin 0 -> 49 bytes
.../b7/2ff6078f62522a87f5cae5e9f34dedf5ec3885 | Bin 0 -> 186 bytes
.../b8/5af389ad1ce39dfb897f8403b12f8d830053f2 | Bin 0 -> 49 bytes
.../ce/5d9d43cc09b37eb0ebe677b18f81933b950b60 | Bin 0 -> 187 bytes
.../cf/0f4f85d7a1237e377a2d25b996518a877ea001 | Bin 0 -> 49 bytes
.../d4/e2941d18a63535400476324ddeb7f40164be41 | Bin 0 -> 121 bytes
.../e0/de76f5d12508fa80efca376f0e2a822fedcfdd | Bin 0 -> 144 bytes
.../e2/2695cbf976fed1f543ad7486a531c0af473482 | Bin 0 -> 49 bytes
.../e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e | Bin 0 -> 49 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../ed/aa7c3edd6ff3866f476824babd820ae42f1040 | Bin 0 -> 31 bytes
.../f6/476b879f65e956d7dedd5b08736369e9a24acc | Bin 0 -> 20 bytes
.../fe/74231105841041d5f441e70399d37f0e600aa5 | Bin 0 -> 121 bytes
.../git-remote/repo.git/refs/changes/01/1/2 | 1 +
.../download/git-remote/repo.git/refs/heads/master | 1 +
.../git-remote/repo.git/refs/heads/mybranch | 1 +
.../git-remote/repo.git/refs/heads/mybranchandtag | 1 +
.../repo.git/refs/heads/myfeature/mybranch | 1 +
.../git-remote/repo.git/refs/heads/submodule | 1 +
.../git-remote/repo.git/refs/tags/mybranchandtag | 1 +
.../download/git-remote/repo.git/refs/tags/mytag | 1 +
support/testing/tests/download/gitremote.py | 15 +++
support/testing/tests/download/http-server/extra | 1 +
support/testing/tests/download/httpserver.py | 14 +++
support/testing/tests/download/test_git.py | 105 +++++++++++++++++++++
support/testing/tests/fs/test_ext.py | 9 +-
support/testing/tests/fs/test_iso9660.py | 13 +--
support/testing/tests/fs/test_jffs2.py | 3 +-
support/testing/tests/fs/test_squashfs.py | 3 +-
support/testing/tests/fs/test_ubi.py | 3 +-
support/testing/tests/fs/test_yaffs2.py | 3 +-
support/testing/tests/init/base.py | 3 +-
support/testing/tests/package/test_dropbear.py | 3 +-
support/testing/tests/package/test_python.py | 3 +-
support/testing/tests/package/test_rust.py | 3 +-
support/testing/tests/package/test_syslog_ng.py | 3 +-
support/testing/tests/toolchain/test_external.py | 3 +-
157 files changed, 667 insertions(+), 43 deletions(-)
create mode 100644 support/testing/infra/runtimetest.py
create mode 100644 support/testing/infra/server.py
create mode 100644 support/testing/tests/download/__init__.py
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.hash
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.mk
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.hash
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.mk
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.hash
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-hash/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-hash/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.hash
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-refs/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-refs/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch-takes-precedence-over-tag/git-branch-takes-precedence-over-tag.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch-takes-precedence-over-tag/git-branch-takes-precedence-over-tag.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch-with-slash/git-branch-with-slash.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch-with-slash/git-branch-with-slash.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-sha1/git-wrong-sha1.mk
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/config
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/07/cabc655213bdf7087d8dd50fda95124e935570
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/1d/f823cb8a6d1866148ae50a8009762a9c4c777f
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/7d/52f458bdb0f9f5a4beb46fa82824421b8b988e
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/cb/545facf77bbc5f24f95b6d503c338d10b7b717
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/dd/130f6f4629514adaf2e03407f3ed9344eb6cd6
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/f8/001e5780100962a5e30a25cbc4c9609cfc7bf3
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/refs/heads/submodule
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/config
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/0a/846af45c3e455789435f49f80d70e86b65b9d7
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/0b/32ebd8fc52cec991f18c94be980e85a8341585
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/32/d61bae693af7879da63b89a60d3ae67f851e56
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/3e/9b0a5198c64cea9c00f820433411e3b4d50c1c
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/85/8f666af75b7c0dfba6b8be7eac5f196e7a1221
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/e8/3f6f805bd016b90acafc8702c52d778eb57310
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/refs/heads/submodule
create mode 100644 support/testing/tests/download/git-remote/repo.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/repo.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/repo.git/config
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/11/93ff46343f4f6a0522e2b28b871e905178c1f0
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/17/c409592968d17600ff9912e96fdd461bb72e74
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/25/59d83bfe937fc0412d96ed664663c9e8a99055
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/2b/0e0d98a49c97da6a618ab36337e2058eb733a2
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/2f/a37f6885d7eb746df75eccaddbacf3ac82799d
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/31/7406308d9259e2231bd0d6ddad3de3832bce08
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/42/c7c82cc86a07a73cb4499345eff2eccfe668bc
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/46/bae5b639e5a18e2cc4dc508f080d566baeff59
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/47/3b7c2f803ad64eac57695ff57a5a6a8b34cbdd
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/51/6c9c5f64ec66534d4d069c2e408d9ae4dce023
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/53/122a7b0454b33c6d9e159c10657173db77899e
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/68/c197d0879d485f4f6cee85544722b79e68e59f
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/80/858c1ab821392fd59a897dbaedd7d07e1ac403
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/83/e0f402891d9de91e7007f8534884ec4b9c7c2c
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/8e/a6edb6df557ee935de556113bab2b5f9ecbabf
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/92/ef85be57d627f280d8ce3724452ac21c9a6452
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/a2/38b1dfcd825d47d834af3c5223417c8411d90d
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/a8/f73649910ce50c48626de0329a0174263ebb5b
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/b2/4b387624edc78d0292a127c43cad9ba97c6232
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/b7/2ff6078f62522a87f5cae5e9f34dedf5ec3885
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/b8/5af389ad1ce39dfb897f8403b12f8d830053f2
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/ce/5d9d43cc09b37eb0ebe677b18f81933b950b60
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/cf/0f4f85d7a1237e377a2d25b996518a877ea001
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/d4/e2941d18a63535400476324ddeb7f40164be41
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e0/de76f5d12508fa80efca376f0e2a822fedcfdd
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e2/2695cbf976fed1f543ad7486a531c0af473482
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/ed/aa7c3edd6ff3866f476824babd820ae42f1040
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/f6/476b879f65e956d7dedd5b08736369e9a24acc
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/fe/74231105841041d5f441e70399d37f0e600aa5
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/changes/01/1/2
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/master
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/mybranch
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/mybranchandtag
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/myfeature/mybranch
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/submodule
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/tags/mybranchandtag
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/tags/mytag
create mode 100644 support/testing/tests/download/gitremote.py
create mode 100644 support/testing/tests/download/http-server/extra
create mode 100644 support/testing/tests/download/httpserver.py
create mode 100644 support/testing/tests/download/test_git.py
--
2.14.1
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v4] testing/infra/builder: build with target and environment
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
@ 2018-04-29 14:33 ` Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/infra: split runtime test from BRTest Ricardo Martincoski
` (13 subsequent siblings)
14 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-29 14:33 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Make the builder able to call 'VAR1=1 make VAR2=2 target'.
Allow to send extra parameters to be added to the end of make command
line. It can be used for these purposes:
- to configure a br2-external, by passing 'BR2_EXTERNAL="dir"';
- to specify a make target, such as 'foo-source'.
Allow to add variables to the environment in which make runs. It can be
used to override values from environment, such as 'BR2_DL_DIR="dl"'.
This change will be needed when adding a common class to test the git
download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Changes v3 -> v4:
- no changes
Changes v2 -> v3:
- rebase after adding patch 1 to the series, which runs make using an
empty env;
- use docstring (Arnout Vandecappelle);
- use more descriptive parameter names (make_extra_opts,
make_extra_env) (Arnout Vandecappelle);
- default make_extra_env to {} and use update() (Arnout
Vandecappelle);
- for consistence I did the equivalent to make_extra_opts;
- for symmetry, use make_extra_env also in the configure() (Arnout
Vandecappelle);
- remove old example from commit message since using a static repo to
test git refs do not need to override _VERSION.
Changes v1 -> v2:
- new patch to adapt the test infra to test git download;
---
support/testing/infra/builder.py | 38 ++++++++++++++++++++++++++++++++++----
1 file changed, 34 insertions(+), 4 deletions(-)
diff --git a/support/testing/infra/builder.py b/support/testing/infra/builder.py
index faf1eb1494..d478c0d212 100644
--- a/support/testing/infra/builder.py
+++ b/support/testing/infra/builder.py
@@ -12,7 +12,18 @@ class Builder(object):
self.builddir = builddir
self.logfile = infra.open_log_file(builddir, "build", logtofile)
- def configure(self):
+ def configure(self, make_extra_opts=[], make_extra_env={}):
+ """
+ Configure the build.
+
+ make_extra_opts: a list of arguments to be passed to the make
+ command.
+ e.g. make_extra_opts=["BR2_EXTERNAL=/path"]
+
+ make_extra_env: a dict of variables to be appended (or replaced)
+ in the environment that calls make.
+ e.g. make_extra_env={"BR2_DL_DIR": "/path"}
+ """
if not os.path.isdir(self.builddir):
os.makedirs(self.builddir)
@@ -25,17 +36,36 @@ class Builder(object):
self.logfile.flush()
env = {"PATH": os.environ["PATH"]}
+ env.update(make_extra_env)
+
cmd = ["make",
- "O={}".format(self.builddir),
- "olddefconfig"]
+ "O={}".format(self.builddir)]
+ cmd += make_extra_opts
+ cmd += ["olddefconfig"]
+
ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
env=env)
if ret != 0:
raise SystemError("Cannot olddefconfig")
- def build(self):
+ def build(self, make_extra_opts=[], make_extra_env={}):
+ """
+ Perform the build.
+
+ make_extra_opts: a list of arguments to be passed to the make
+ command. It can include a make target.
+ e.g. make_extra_opts=["foo-source"]
+
+ make_extra_env: a dict of variables to be appended (or replaced)
+ in the environment that calls make.
+ e.g. make_extra_env={"BR2_DL_DIR": "/path"}
+ """
env = {"PATH": os.environ["PATH"]}
+ env.update(make_extra_env)
+
cmd = ["make", "-C", self.builddir]
+ cmd += make_extra_opts
+
ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
env=env)
if ret != 0:
--
2.14.1
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v4] testing/infra: split runtime test from BRTest
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/infra/builder: build with target and environment Ricardo Martincoski
@ 2018-04-29 14:33 ` Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/infra/basetest: support br2-external Ricardo Martincoski
` (12 subsequent siblings)
14 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-29 14:33 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Move the setup of emulator to a new class, RuntimeTestBase, that bahaves
exactly like BRTest currently does.
It will avoid duplicating code when adding a common class to test the
git download infra.
Change all current test cases to use the new class.
Do this by first using automatic replace:
$ find support/testing/ -name '*.py' | \
xargs grep -l BRTest | \
xargs sed -i \
-e 's,import infra.basetest,\0\nimport infra.runtimetest,g' \
-e 's,infra.basetest.BRTest,infra.runtimetest.RuntimeTestBase,g'
and then manually add code to import runtimetest in test_external.py to
avoid this error:
AttributeError: 'module' object has no attribute 'LoadTestsFailure'
This explicit import was not need before because run-tests imports
BRTest and this is the only test file that do not use the defconfig
fragments from basetest.py in its code.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
WARNING: this patch changes all current test cases, so if a new test case was
applied after this patch was sent, 'git am' will apply this patch cleanly, yet
any testcase created in the meantime will be broken.
But I can refresh it if need. It will probably only need the automatic replace
from the commit log to be re-run.
Changes v3 -> v4:
- re-run the automatic replace command
Changes v2 -> v3:
- new patch
- search for "RuntimeTestBase" in
http://patchwork.ozlabs.org/patch/806161/
---
support/testing/infra/basetest.py | 10 ----------
support/testing/infra/runtimetest.py | 23 +++++++++++++++++++++++
support/testing/tests/boot/test_atf.py | 7 ++++---
support/testing/tests/core/test_post_scripts.py | 3 ++-
support/testing/tests/core/test_rootfs_overlay.py | 3 ++-
support/testing/tests/core/test_timezone.py | 7 ++++---
support/testing/tests/fs/test_ext.py | 9 +++++----
support/testing/tests/fs/test_iso9660.py | 13 +++++++------
support/testing/tests/fs/test_jffs2.py | 3 ++-
support/testing/tests/fs/test_squashfs.py | 3 ++-
support/testing/tests/fs/test_ubi.py | 3 ++-
support/testing/tests/fs/test_yaffs2.py | 3 ++-
support/testing/tests/init/base.py | 3 ++-
support/testing/tests/package/test_dropbear.py | 3 ++-
support/testing/tests/package/test_python.py | 3 ++-
support/testing/tests/package/test_rust.py | 3 ++-
support/testing/tests/package/test_syslog_ng.py | 3 ++-
support/testing/tests/toolchain/test_external.py | 3 ++-
18 files changed, 67 insertions(+), 38 deletions(-)
create mode 100644 support/testing/infra/runtimetest.py
diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
index f3f13ad97f..4773312585 100644
--- a/support/testing/infra/basetest.py
+++ b/support/testing/infra/basetest.py
@@ -3,7 +3,6 @@ import os
import datetime
from infra.builder import Builder
-from infra.emulator import Emulator
BASIC_TOOLCHAIN_CONFIG = \
"""
@@ -41,7 +40,6 @@ class BRTest(unittest.TestCase):
super(BRTest, self).__init__(names)
self.testname = self.__class__.__name__
self.builddir = self.outputdir and os.path.join(self.outputdir, self.testname)
- self.emulator = None
self.config += '\nBR2_DL_DIR="{}"\n'.format(self.downloaddir)
self.config += "\nBR2_JLEVEL={}\n".format(self.jlevel)
@@ -57,17 +55,9 @@ class BRTest(unittest.TestCase):
self.b.delete()
if not self.b.is_finished():
- self.show_msg("Building")
self.b.configure()
- self.b.build()
- self.show_msg("Building done")
-
- self.emulator = Emulator(self.builddir, self.downloaddir,
- self.logtofile, self.timeout_multiplier)
def tearDown(self):
self.show_msg("Cleaning up")
- if self.emulator:
- self.emulator.stop()
if self.b and not self.keepbuilds:
self.b.delete()
diff --git a/support/testing/infra/runtimetest.py b/support/testing/infra/runtimetest.py
new file mode 100644
index 0000000000..68bb03d091
--- /dev/null
+++ b/support/testing/infra/runtimetest.py
@@ -0,0 +1,23 @@
+from infra.basetest import BRTest
+from infra.emulator import Emulator
+
+
+class RuntimeTestBase(BRTest):
+ def __init__(self, names):
+ super(RuntimeTestBase, self).__init__(names)
+ self.emulator = None
+
+ def setUp(self):
+ super(RuntimeTestBase, self).setUp()
+ if not self.b.is_finished():
+ self.show_msg("Building")
+ self.b.build()
+ self.show_msg("Building done")
+
+ self.emulator = Emulator(self.builddir, self.downloaddir,
+ self.logtofile, self.timeout_multiplier)
+
+ def tearDown(self):
+ if self.emulator:
+ self.emulator.stop()
+ super(RuntimeTestBase, self).tearDown()
diff --git a/support/testing/tests/boot/test_atf.py b/support/testing/tests/boot/test_atf.py
index 75cea01fc8..0ea486fb5b 100644
--- a/support/testing/tests/boot/test_atf.py
+++ b/support/testing/tests/boot/test_atf.py
@@ -1,7 +1,8 @@
import infra.basetest
+import infra.runtimetest
-class TestATFVexpress(infra.basetest.BRTest):
+class TestATFVexpress(infra.runtimetest.RuntimeTestBase):
config = \
"""
BR2_aarch64=y
@@ -25,7 +26,7 @@ class TestATFVexpress(infra.basetest.BRTest):
pass
-class TestATFAllwinner(infra.basetest.BRTest):
+class TestATFAllwinner(infra.runtimetest.RuntimeTestBase):
config = \
"""
BR2_aarch64=y
@@ -54,7 +55,7 @@ class TestATFAllwinner(infra.basetest.BRTest):
pass
-class TestATFMarvell(infra.basetest.BRTest):
+class TestATFMarvell(infra.runtimetest.RuntimeTestBase):
config = \
"""
BR2_aarch64=y
diff --git a/support/testing/tests/core/test_post_scripts.py b/support/testing/tests/core/test_post_scripts.py
index a0e5b6b454..c4dfb91550 100644
--- a/support/testing/tests/core/test_post_scripts.py
+++ b/support/testing/tests/core/test_post_scripts.py
@@ -2,9 +2,10 @@ import os
import csv
import infra.basetest
+import infra.runtimetest
-class TestPostScripts(infra.basetest.BRTest):
+class TestPostScripts(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_INIT_NONE=y
diff --git a/support/testing/tests/core/test_rootfs_overlay.py b/support/testing/tests/core/test_rootfs_overlay.py
index fedd40d8ac..66b5531221 100644
--- a/support/testing/tests/core/test_rootfs_overlay.py
+++ b/support/testing/tests/core/test_rootfs_overlay.py
@@ -2,13 +2,14 @@ import os
import subprocess
import infra.basetest
+import infra.runtimetest
def compare_file(file1, file2):
return subprocess.call(["cmp", file1, file2])
-class TestRootfsOverlay(infra.basetest.BRTest):
+class TestRootfsOverlay(infra.runtimetest.RuntimeTestBase):
rootfs_overlay_path = infra.filepath("tests/core/rootfs-overlay")
diff --git a/support/testing/tests/core/test_timezone.py b/support/testing/tests/core/test_timezone.py
index 050624e0aa..dca38bebe8 100644
--- a/support/testing/tests/core/test_timezone.py
+++ b/support/testing/tests/core/test_timezone.py
@@ -1,6 +1,7 @@
import os
import infra.basetest
+import infra.runtimetest
def boot_armv5_cpio(emulator, builddir):
@@ -10,7 +11,7 @@ def boot_armv5_cpio(emulator, builddir):
emulator.login()
-class TestNoTimezone(infra.basetest.BRTest):
+class TestNoTimezone(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
# BR2_TARGET_TZ_INFO is not set
@@ -26,7 +27,7 @@ class TestNoTimezone(infra.basetest.BRTest):
self.assertEqual(tz[0].strip(), "UTC")
-class TestGlibcAllTimezone(infra.basetest.BRTest):
+class TestGlibcAllTimezone(infra.runtimetest.RuntimeTestBase):
config = \
"""
BR2_arm=y
@@ -48,7 +49,7 @@ class TestGlibcAllTimezone(infra.basetest.BRTest):
self.assertEqual(tz[0].strip(), "CET")
-class TestGlibcNonDefaultLimitedTimezone(infra.basetest.BRTest):
+class TestGlibcNonDefaultLimitedTimezone(infra.runtimetest.RuntimeTestBase):
config = \
"""
BR2_arm=y
diff --git a/support/testing/tests/fs/test_ext.py b/support/testing/tests/fs/test_ext.py
index f5f9e9fdf1..a8f68bc54c 100644
--- a/support/testing/tests/fs/test_ext.py
+++ b/support/testing/tests/fs/test_ext.py
@@ -2,6 +2,7 @@ import os
import subprocess
import infra.basetest
+import infra.runtimetest
VOLNAME_PROP = "Filesystem volume name"
REVISION_PROP = "Filesystem revision #"
@@ -41,7 +42,7 @@ def boot_img_and_check_fs_type(emulator, builddir, fs_type):
return exit_code
-class TestExt2(infra.basetest.BRTest):
+class TestExt2(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_EXT2=y
@@ -60,7 +61,7 @@ class TestExt2(infra.basetest.BRTest):
self.assertEqual(exit_code, 0)
-class TestExt2r1(infra.basetest.BRTest):
+class TestExt2r1(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_EXT2=y
@@ -80,7 +81,7 @@ class TestExt2r1(infra.basetest.BRTest):
self.assertEqual(exit_code, 0)
-class TestExt3(infra.basetest.BRTest):
+class TestExt3(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_EXT2=y
@@ -99,7 +100,7 @@ class TestExt3(infra.basetest.BRTest):
self.assertEqual(exit_code, 0)
-class TestExt4(infra.basetest.BRTest):
+class TestExt4(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_EXT2=y
diff --git a/support/testing/tests/fs/test_iso9660.py b/support/testing/tests/fs/test_iso9660.py
index 68f4840852..77255ac93b 100644
--- a/support/testing/tests/fs/test_iso9660.py
+++ b/support/testing/tests/fs/test_iso9660.py
@@ -1,6 +1,7 @@
import os
import infra.basetest
+import infra.runtimetest
BASIC_CONFIG = \
"""
@@ -47,7 +48,7 @@ def test_touch_file(emulator):
# Grub 2
-class TestIso9660Grub2External(infra.basetest.BRTest):
+class TestIso9660Grub2External(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
@@ -67,7 +68,7 @@ class TestIso9660Grub2External(infra.basetest.BRTest):
self.assertEqual(exit_code, 1)
-class TestIso9660Grub2ExternalCompress(infra.basetest.BRTest):
+class TestIso9660Grub2ExternalCompress(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
@@ -88,7 +89,7 @@ class TestIso9660Grub2ExternalCompress(infra.basetest.BRTest):
self.assertEqual(exit_code, 1)
-class TestIso9660Grub2Internal(infra.basetest.BRTest):
+class TestIso9660Grub2Internal(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
@@ -111,7 +112,7 @@ class TestIso9660Grub2Internal(infra.basetest.BRTest):
# Syslinux
-class TestIso9660SyslinuxExternal(infra.basetest.BRTest):
+class TestIso9660SyslinuxExternal(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
@@ -130,7 +131,7 @@ class TestIso9660SyslinuxExternal(infra.basetest.BRTest):
self.assertEqual(exit_code, 1)
-class TestIso9660SyslinuxExternalCompress(infra.basetest.BRTest):
+class TestIso9660SyslinuxExternalCompress(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
@@ -150,7 +151,7 @@ class TestIso9660SyslinuxExternalCompress(infra.basetest.BRTest):
self.assertEqual(exit_code, 1)
-class TestIso9660SyslinuxInternal(infra.basetest.BRTest):
+class TestIso9660SyslinuxInternal(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
diff --git a/support/testing/tests/fs/test_jffs2.py b/support/testing/tests/fs/test_jffs2.py
index 2ff5099180..8f73cf1f06 100644
--- a/support/testing/tests/fs/test_jffs2.py
+++ b/support/testing/tests/fs/test_jffs2.py
@@ -2,6 +2,7 @@ import os
import subprocess
import infra.basetest
+import infra.runtimetest
def jffs2dump_find_file(files_list, fname):
@@ -12,7 +13,7 @@ def jffs2dump_find_file(files_list, fname):
return False
-class TestJffs2(infra.basetest.BRTest):
+class TestJffs2(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_JFFS2=y
diff --git a/support/testing/tests/fs/test_squashfs.py b/support/testing/tests/fs/test_squashfs.py
index 066c054342..f80149f9d9 100644
--- a/support/testing/tests/fs/test_squashfs.py
+++ b/support/testing/tests/fs/test_squashfs.py
@@ -2,9 +2,10 @@ import os
import subprocess
import infra.basetest
+import infra.runtimetest
-class TestSquashfs(infra.basetest.BRTest):
+class TestSquashfs(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_SQUASHFS=y
diff --git a/support/testing/tests/fs/test_ubi.py b/support/testing/tests/fs/test_ubi.py
index 015d82f769..c724f4f740 100644
--- a/support/testing/tests/fs/test_ubi.py
+++ b/support/testing/tests/fs/test_ubi.py
@@ -2,9 +2,10 @@ import subprocess
import os
import infra.basetest
+import infra.runtimetest
-class TestUbi(infra.basetest.BRTest):
+class TestUbi(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_UBIFS=y
diff --git a/support/testing/tests/fs/test_yaffs2.py b/support/testing/tests/fs/test_yaffs2.py
index b60e90e660..c7c8c1f724 100644
--- a/support/testing/tests/fs/test_yaffs2.py
+++ b/support/testing/tests/fs/test_yaffs2.py
@@ -1,9 +1,10 @@
import os
import infra.basetest
+import infra.runtimetest
-class TestYaffs2(infra.basetest.BRTest):
+class TestYaffs2(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
infra.basetest.MINIMAL_CONFIG + \
"""
diff --git a/support/testing/tests/init/base.py b/support/testing/tests/init/base.py
index 75cfbe9c59..1b736af657 100644
--- a/support/testing/tests/init/base.py
+++ b/support/testing/tests/init/base.py
@@ -1,9 +1,10 @@
import os
import subprocess
import infra.basetest
+import infra.runtimetest
-class InitSystemBase(infra.basetest.BRTest):
+class InitSystemBase(infra.runtimetest.RuntimeTestBase):
def start_emulator(self, fs_type, kernel=None, dtb=None, init=None):
img = os.path.join(self.builddir, "images", "rootfs.{}".format(fs_type))
diff --git a/support/testing/tests/package/test_dropbear.py b/support/testing/tests/package/test_dropbear.py
index 8f7f1fee82..8f7f30e3af 100644
--- a/support/testing/tests/package/test_dropbear.py
+++ b/support/testing/tests/package/test_dropbear.py
@@ -1,9 +1,10 @@
import os
import infra.basetest
+import infra.runtimetest
-class TestDropbear(infra.basetest.BRTest):
+class TestDropbear(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_GENERIC_ROOT_PASSWD="testpwd"
diff --git a/support/testing/tests/package/test_python.py b/support/testing/tests/package/test_python.py
index 26cf49947b..787364c719 100644
--- a/support/testing/tests/package/test_python.py
+++ b/support/testing/tests/package/test_python.py
@@ -1,9 +1,10 @@
import os
import infra.basetest
+import infra.runtimetest
-class TestPythonBase(infra.basetest.BRTest):
+class TestPythonBase(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_CPIO=y
diff --git a/support/testing/tests/package/test_rust.py b/support/testing/tests/package/test_rust.py
index 2dc814f99d..df445f1186 100644
--- a/support/testing/tests/package/test_rust.py
+++ b/support/testing/tests/package/test_rust.py
@@ -4,9 +4,10 @@ import subprocess
import shutil
import infra.basetest
+import infra.runtimetest
-class TestRustBase(infra.basetest.BRTest):
+class TestRustBase(infra.runtimetest.RuntimeTestBase):
target = 'armv7-unknown-linux-gnueabihf'
crate = 'hello-world'
diff --git a/support/testing/tests/package/test_syslog_ng.py b/support/testing/tests/package/test_syslog_ng.py
index 0155ef14e4..df6ce8b8d0 100644
--- a/support/testing/tests/package/test_syslog_ng.py
+++ b/support/testing/tests/package/test_syslog_ng.py
@@ -1,9 +1,10 @@
import os
import infra.basetest
+import infra.runtimetest
-class TestSyslogNg(infra.basetest.BRTest):
+class TestSyslogNg(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_PACKAGE_BUSYBOX_SHOW_OTHERS=y
diff --git a/support/testing/tests/toolchain/test_external.py b/support/testing/tests/toolchain/test_external.py
index 881d2b00db..b72e19f740 100644
--- a/support/testing/tests/toolchain/test_external.py
+++ b/support/testing/tests/toolchain/test_external.py
@@ -1,5 +1,6 @@
import os
import infra
+import infra.runtimetest
BASIC_CONFIG = \
"""
@@ -17,7 +18,7 @@ def has_broken_links(path):
return False
-class TestExternalToolchain(infra.basetest.BRTest):
+class TestExternalToolchain(infra.runtimetest.RuntimeTestBase):
def common_check(self):
# Check for broken symlinks
for d in ["lib", "usr/lib"]:
--
2.14.1
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v4] testing/infra/basetest: support br2-external
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/infra/builder: build with target and environment Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/infra: split runtime test from BRTest Ricardo Martincoski
@ 2018-04-29 14:33 ` Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: add git hash test Ricardo Martincoski
` (11 subsequent siblings)
14 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-29 14:33 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
This change will be needed when adding a common class to test the git
download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Changes v3 -> v4:
- allow the use of multiple br2-external trees, simplifying the code
Changes v2 -> v3:
- new patch
- search for "support for BR2_EXTERNAL directly in BRTest" in
http://patchwork.ozlabs.org/patch/806161/
---
support/testing/infra/basetest.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
index 4773312585..67076701e7 100644
--- a/support/testing/infra/basetest.py
+++ b/support/testing/infra/basetest.py
@@ -29,6 +29,7 @@ MINIMAL_CONFIG = \
class BRTest(unittest.TestCase):
config = None
+ br2_external = list()
downloaddir = None
outputdir = None
logtofile = True
@@ -55,7 +56,7 @@ class BRTest(unittest.TestCase):
self.b.delete()
if not self.b.is_finished():
- self.b.configure()
+ self.b.configure(["BR2_EXTERNAL={}".format(":".join(self.br2_external))])
def tearDown(self):
self.show_msg("Cleaning up")
--
2.14.1
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v4] testing/tests/download: add git hash test
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
` (2 preceding siblings ...)
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/infra/basetest: support br2-external Ricardo Martincoski
@ 2018-04-29 14:33 ` Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test case for git refs Ricardo Martincoski
` (10 subsequent siblings)
14 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-29 14:33 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Add one test case to ensure the hash is checked for git packages:
- correct hash;
- wrong hash;
- no hash file.
Add required infra:
- a GitRemote class, that can start a git server in the host machine to
emulate a remote git server under the control of the test;
- a new base class, called GitTestBase, that inherits from BRTest and
must be subclassed by all git test cases.
Its setUp() method takes care of configuring the build with a
br2-external, avoiding to hit http://sources.buildroot.net by using
an empty BR2_BACKUP_SITE. It also avoids downloading not
pre-installed dependencies (i.e. lzip) every time by calling 'make
dependencies' using the common dl directory, and it instantiates the
GitRemote object.
Besides the Python scripts, add some fixtures used during the tests:
- a br2-external (git-hash) with one package for each part of the test
case;
- a static git bare repo (repo.git) to be served using GitRemote class.
Neither the br2-external nor the check hash functionalities are the
subject of these tests per se, so for simplicity limit the check to the
error codes and don't look for the messages in the log.
Thanks to Arnout for the hint about how to add a bare repo to test.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repo with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* a238b1d (HEAD -> master) initial
Changes v3 -> v4:
- remove call to 'make dependencies';
- merge generic git base file to the main test case as the first was
only used by the second;
- use variable naming similar to builder;
- move useful methods to the base class;
- rename logfile to -gitremote (it will become handy when adding the
http server on a later patch);
- use more columns (our limit is now 132);
- merge start to __init__ as it was called only there;
- remove temp variable used only once (port_arg);
- use a more future-proof string format for the port arg (it will
become handy when adding the http server on a later patch);
- use simpler code to raise exception if there is no free port;
- fix the range for ports to use;
- br2_external is now a list;
Changes v2 -> v3:
- renumber 5 -> 6;
- rebase after removing patch 3 "allow to override logfile" from the
series and adding new patches 4 "split runtime test from BRTest" and
5 "support br2-external";
- merge part of old patch 4 (very minimal GitTestBase) to old patch 5
(Arnout Vandecappelle);
- merge and tweak part of commit log from old patch 4;
- move GITREMOTE_PORT_NUMBER to external.mk (Arnout Vandecappelle);
- use dosctrings;
- start git remote from __init__ (Arnout Vandecappelle), but I kept a
separate method and call start();
- remove dead code (Arnout Vandecappelle);
- use the same git repo for all tests (Arnout Vandecappelle);
- rename the file to test_git and merge the 3 test cases in one, since
the feature "check hash for git packages" is only OK if all work
well and also because the test does not take too much time;
- update commit log accordingly.
Changes v1 -> v2 (patch 4):
- new patch
Changes v1 -> v2 (patch 5):
- use git daemon + git:// instead of ssh (Arnout);
- remove __main__ handling since the test infra already does that;
- use the logging support from test infra;
- this patch is part of series 1/3 of a new version of
http://patchwork.ozlabs.org/patch/690097/
---
.gitlab-ci.yml | 1 +
support/testing/tests/download/__init__.py | 0
.../tests/download/br2-external/git-hash/Config.in | 0
.../download/br2-external/git-hash/external.desc | 1 +
.../download/br2-external/git-hash/external.mk | 4 ++
.../br2-external/git-hash/package/bad/bad.hash | 1 +
.../br2-external/git-hash/package/bad/bad.mk | 10 +++++
.../br2-external/git-hash/package/good/good.hash | 1 +
.../br2-external/git-hash/package/good/good.mk | 10 +++++
.../br2-external/git-hash/package/nohash/nohash.mk | 10 +++++
.../download/git-remote/repo.git/.gitattributes | 1 +
.../tests/download/git-remote/repo.git/HEAD | 1 +
.../tests/download/git-remote/repo.git/config | 4 ++
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../a2/38b1dfcd825d47d834af3c5223417c8411d90d | Bin 0 -> 152 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../download/git-remote/repo.git/refs/heads/master | 1 +
support/testing/tests/download/gitremote.py | 44 +++++++++++++++++++++
support/testing/tests/download/test_git.py | 43 ++++++++++++++++++++
19 files changed, 132 insertions(+)
create mode 100644 support/testing/tests/download/__init__.py
create mode 100644 support/testing/tests/download/br2-external/git-hash/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-hash/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-hash/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.hash
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk
create mode 100644 support/testing/tests/download/git-remote/repo.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/repo.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/repo.git/config
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/a2/38b1dfcd825d47d834af3c5223417c8411d90d
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/master
create mode 100644 support/testing/tests/download/gitremote.py
create mode 100644 support/testing/tests/download/test_git.py
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 568fb7fc19..42559e0365 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -266,6 +266,7 @@ tests.core.test_rootfs_overlay.TestRootfsOverlay: *runtime_test
tests.core.test_timezone.TestGlibcAllTimezone: *runtime_test
tests.core.test_timezone.TestGlibcNonDefaultLimitedTimezone: *runtime_test
tests.core.test_timezone.TestNoTimezone: *runtime_test
+tests.download.test_git.TestGitHash: *runtime_test
tests.fs.test_ext.TestExt2: *runtime_test
tests.fs.test_ext.TestExt2r1: *runtime_test
tests.fs.test_ext.TestExt3: *runtime_test
diff --git a/support/testing/tests/download/__init__.py b/support/testing/tests/download/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/support/testing/tests/download/br2-external/git-hash/Config.in b/support/testing/tests/download/br2-external/git-hash/Config.in
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/support/testing/tests/download/br2-external/git-hash/external.desc b/support/testing/tests/download/br2-external/git-hash/external.desc
new file mode 100644
index 0000000000..41316c8b25
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/external.desc
@@ -0,0 +1 @@
+name: GIT_HASH
diff --git a/support/testing/tests/download/br2-external/git-hash/external.mk b/support/testing/tests/download/br2-external/git-hash/external.mk
new file mode 100644
index 0000000000..4646dfe2b0
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/external.mk
@@ -0,0 +1,4 @@
+include $(sort $(wildcard $(BR2_EXTERNAL_GIT_HASH_PATH)/package/*/*.mk))
+
+# Get the git server port number from the test infra
+GITREMOTE_PORT_NUMBER ?= 9418
diff --git a/support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
new file mode 100644
index 0000000000..b9e1baec84
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
@@ -0,0 +1 @@
+sha256 0000000000000000000000000000000000000000000000000000000000000000 bad-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
diff --git a/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
new file mode 100644
index 0000000000..5497bd6bfe
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
@@ -0,0 +1,10 @@
+################################################################################
+#
+# bad
+#
+################################################################################
+
+BAD_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+BAD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-hash/package/good/good.hash b/support/testing/tests/download/br2-external/git-hash/package/good/good.hash
new file mode 100644
index 0000000000..9e92ab8ab9
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/good/good.hash
@@ -0,0 +1 @@
+sha256 d00ae598e9e770d607621a86766030b42eaa58156cb8d482b043969da7963c23 good-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
diff --git a/support/testing/tests/download/br2-external/git-hash/package/good/good.mk b/support/testing/tests/download/br2-external/git-hash/package/good/good.mk
new file mode 100644
index 0000000000..0f0eefd944
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/good/good.mk
@@ -0,0 +1,10 @@
+################################################################################
+#
+# good
+#
+################################################################################
+
+GOOD_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+GOOD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk b/support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk
new file mode 100644
index 0000000000..1da19d88c6
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk
@@ -0,0 +1,10 @@
+################################################################################
+#
+# nohash
+#
+################################################################################
+
+NOHASH_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+NOHASH_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/repo.git/.gitattributes b/support/testing/tests/download/git-remote/repo.git/.gitattributes
new file mode 100644
index 0000000000..eb50c64a21
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/.gitattributes
@@ -0,0 +1 @@
+objects/*/* binary
diff --git a/support/testing/tests/download/git-remote/repo.git/HEAD b/support/testing/tests/download/git-remote/repo.git/HEAD
new file mode 100644
index 0000000000..cb089cd89a
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/HEAD
@@ -0,0 +1 @@
+ref: refs/heads/master
diff --git a/support/testing/tests/download/git-remote/repo.git/config b/support/testing/tests/download/git-remote/repo.git/config
new file mode 100644
index 0000000000..07d359d07c
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/config
@@ -0,0 +1,4 @@
+[core]
+ repositoryformatversion = 0
+ filemode = true
+ bare = true
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab b/support/testing/tests/download/git-remote/repo.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
new file mode 100644
index 0000000000000000000000000000000000000000..9db72668cf9374b0b85a25a19f30084fd460072d
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5wZcw)UHz=nDZYB`lCt5lYr9_mN?Omvz#_so
F4FHVv5b^*3
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/a2/38b1dfcd825d47d834af3c5223417c8411d90d b/support/testing/tests/download/git-remote/repo.git/objects/a2/38b1dfcd825d47d834af3c5223417c8411d90d
new file mode 100644
index 0000000000000000000000000000000000000000..31b6bcf34de247d14531c70b36e28d895ba20af3
GIT binary patch
literal 152
zcmV;J0B8Sr0j<tW4#FT1Kw;OMVlL1IsFY%i at d9r21jApQv_OKm@b)%w;oi9Y7BA10
zl&FE!)2`JJz?dk*5QMWrMPrqWC`}wkKTO<v<fHILXtWHU?OrNe$zk;cE?667R~`$&
zv3{^mUp&tVY3*G}ClHEjqx7`b(D8wDw&^y}!k==CGBv`Lu^Y`f_^!m*+1KHhS@s5O
G+F(P1IZe3$
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5 b/support/testing/tests/download/git-remote/repo.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
new file mode 100644
index 0000000000000000000000000000000000000000..df2037a2d2d30afb20e31cd558c92c2edabe7cf6
GIT binary patch
literal 23
fcmb<m^geacKWIb3i8H5lPWqgA$i$$b%kl;Qg;xrl
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/heads/master b/support/testing/tests/download/git-remote/repo.git/refs/heads/master
new file mode 100644
index 0000000000..b6bccc1c17
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/heads/master
@@ -0,0 +1 @@
+a238b1dfcd825d47d834af3c5223417c8411d90d
diff --git a/support/testing/tests/download/gitremote.py b/support/testing/tests/download/gitremote.py
new file mode 100644
index 0000000000..60bc49fbf8
--- /dev/null
+++ b/support/testing/tests/download/gitremote.py
@@ -0,0 +1,44 @@
+# subprocess does not kill the child daemon when a test case fails by raising
+# an exception. So use pexpect instead.
+import pexpect
+
+import infra
+
+GIT_REMOTE_PORT_INITIAL = 9418
+GIT_REMOTE_PORT_LAST = GIT_REMOTE_PORT_INITIAL + 99
+
+
+class GitRemote(object):
+ def __init__(self, builddir, serveddir, logtofile):
+ """
+ Start a local git server.
+
+ In order to support test cases in parallel, select the port the
+ server will listen to in runtime. Since there is no reliable way
+ to allocate the port prior to starting the server (another
+ process in the host machine can use the port between it is
+ selected from a list and it is really allocated to the server)
+ try to start the server in a port and in the case it is already
+ in use, try the next one in the allowed range.
+ """
+ self.daemon = None
+ self.port = None
+ self.logfile = infra.open_log_file(builddir, "gitremote", logtofile)
+
+ daemon_cmd = ["git", "daemon", "--reuseaddr", "--verbose", "--listen=localhost", "--export-all",
+ "--base-path={}".format(serveddir)]
+ for port in range(GIT_REMOTE_PORT_INITIAL, GIT_REMOTE_PORT_LAST + 1):
+ cmd = daemon_cmd + ["--port={port}".format(port=port)]
+ self.logfile.write("> starting git remote with '{}'\n".format(" ".join(cmd)))
+ self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile)
+ ret = self.daemon.expect(["Ready to rumble",
+ "Address already in use"])
+ if ret == 0:
+ self.port = port
+ return
+ raise SystemError("Could not find a free port to run git remote")
+
+ def stop(self):
+ if self.daemon is None:
+ return
+ self.daemon.terminate(force=True)
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
new file mode 100644
index 0000000000..14fc8e4da3
--- /dev/null
+++ b/support/testing/tests/download/test_git.py
@@ -0,0 +1,43 @@
+import os
+
+import infra
+from gitremote import GitRemote
+
+
+class GitTestBase(infra.basetest.BRTest):
+ config = \
+ """
+ BR2_BACKUP_SITE=""
+ """
+ gitremotedir = infra.filepath("tests/download/git-remote")
+ gitremote = None
+
+ def setUp(self):
+ super(GitTestBase, self).setUp()
+ self.gitremote = GitRemote(self.builddir, self.gitremotedir, self.logtofile)
+
+ def tearDown(self):
+ self.show_msg("Cleaning up")
+ if self.gitremote:
+ self.gitremote.stop()
+ if self.b and not self.keepbuilds:
+ self.b.delete()
+
+ def check_hash(self, package):
+ # store downloaded tarball inside the output dir so the test infra
+ # cleans it up at the end
+ env = {"BR2_DL_DIR": os.path.join(self.builddir, "dl"),
+ "GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
+ self.b.build(["{}-dirclean".format(package),
+ "{}-source".format(package)],
+ env)
+
+
+class TestGitHash(GitTestBase):
+ br2_external = [infra.filepath("tests/download/br2-external/git-hash")]
+
+ def test_run(self):
+ with self.assertRaises(SystemError):
+ self.check_hash("bad")
+ self.check_hash("good")
+ self.check_hash("nohash")
--
2.14.1
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v4] testing/tests/download: test case for git refs
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
` (3 preceding siblings ...)
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: add git hash test Ricardo Martincoski
@ 2018-04-29 14:33 ` Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git branch Ricardo Martincoski
` (9 subsequent siblings)
14 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-29 14:33 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
All upcoming tests for git refs will rely on the return code of make to
determine whether a git ref can be downloaded or not and also to
determine whether the downloaded content is correct (all of this taking
advantage of the check-hash mechanism already in place for git
packages).
So to avoid false results i.e. in the case the check-hash mechanism
become broken in the master branch, add some sanity checks before the
actual test of download git refs.
Add the minimum test case for git refs containing only sanity checks.
Reuse the commit in the static repo.
Add a br2-external with two packages to check that:
- trying to download an invalid sha1 generates an error;
- downloading a valid sha1 that contains unexpected content generates
an error.
In order to ease the maintenance and review, each upcoming patch adding
checks to this test case will add at same time the commits to the static
repo, the equivalent packages to the br2-external and code to the test
case.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repo with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* a238b1d (HEAD -> master) initial
Changes v3 -> v4:
- move useful methods to the git base class;
- br2_external is now a list;
Changes v2 -> v3:
- rewrite from v2, using an static repo;
- separate on this patch the bare minimum static repo (actually reuse
from previous commit), the next patches on the series will gradually
add new commits to the static repo together to the equivalent
packages and call to check_download. This approach allows better
maintenance, even during the review of the series, because a patch
can be more easily reworked/discarded/reordered;
Changes v1 -> v2:
- rename the main test file to testgit (Arnout). Actually I broke it
down to a common gitbase and the two test_git_refs and test_git_hash
(this last one is new, in the previous patch);
- remove some weird/wrong TODO I had in v1 code (Arnout);
- use the same structure for test cases as used in the test infra;
- raise an exception when the download fails (Arnout). I did not add
code for this since I let the builder class to raise it;
- I reimplemented git_util as gitrepo. Arnout suggested to use it as
module and I first implemented that way locally, but then I created
test cases for submodules and I realized now I have a reason to use
a class (see next patch);
- move package (now called foo) to a BR2_EXTERNAL (Arnout);
- override BR2_DL_DIR when calling make (Arnout);
- the test infra uses O= so I don't need to;
- instead of removing the files in the dl/ folder, use a different dir
as dl/ dir since now each repo has a different name;
- open the tarball to check its contents (Arnout). I create a fresh
clone of the repo before the test to compare against;
- this patch is not checking for the actual sha1 anymore, it can be
done later. But the check for the contents should cover most cases;
- my old argument to not test only the support/download/dl-wrapper is
not valid anymore since I mimic the logic from the scripts to know
the name of the tarball. But I still think calling make ...-source
has better maintenance. And also the previous patch (test for hash
of packages with git method) can use the same base class;
- this patch is part of series 1/3 of a new version of
http://patchwork.ozlabs.org/patch/690097/
---
.gitlab-ci.yml | 1 +
.../tests/download/br2-external/git-refs/Config.in | 0
.../download/br2-external/git-refs/external.desc | 1 +
.../tests/download/br2-external/git-refs/external.mk | 4 ++++
.../package/git-wrong-content/git-wrong-content.hash | 2 ++
.../package/git-wrong-content/git-wrong-content.mk | 11 +++++++++++
.../git-refs/package/git-wrong-sha1/git-wrong-sha1.mk | 11 +++++++++++
support/testing/tests/download/test_git.py | 19 +++++++++++++++++++
8 files changed, 49 insertions(+)
create mode 100644 support/testing/tests/download/br2-external/git-refs/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-refs/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-refs/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-sha1/git-wrong-sha1.mk
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 42559e0365..26e2af3799 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -267,6 +267,7 @@ tests.core.test_timezone.TestGlibcAllTimezone: *runtime_test
tests.core.test_timezone.TestGlibcNonDefaultLimitedTimezone: *runtime_test
tests.core.test_timezone.TestNoTimezone: *runtime_test
tests.download.test_git.TestGitHash: *runtime_test
+tests.download.test_git.TestGitRefs: *runtime_test
tests.fs.test_ext.TestExt2: *runtime_test
tests.fs.test_ext.TestExt2r1: *runtime_test
tests.fs.test_ext.TestExt3: *runtime_test
diff --git a/support/testing/tests/download/br2-external/git-refs/Config.in b/support/testing/tests/download/br2-external/git-refs/Config.in
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/support/testing/tests/download/br2-external/git-refs/external.desc b/support/testing/tests/download/br2-external/git-refs/external.desc
new file mode 100644
index 0000000000..69f40d46c6
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/external.desc
@@ -0,0 +1 @@
+name: GIT_REFS
diff --git a/support/testing/tests/download/br2-external/git-refs/external.mk b/support/testing/tests/download/br2-external/git-refs/external.mk
new file mode 100644
index 0000000000..ab38c27eb7
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/external.mk
@@ -0,0 +1,4 @@
+include $(sort $(wildcard $(BR2_EXTERNAL_GIT_REFS_PATH)/package/*/*.mk))
+
+# Get the git server port number from the test infra
+GITREMOTE_PORT_NUMBER ?= 9418
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash
new file mode 100644
index 0000000000..47b2b8b7d7
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash
@@ -0,0 +1,2 @@
+ sha256 04715901977503d1df650e0959f4d31d8e7b105e2ac99a2182e0652b8f13baa1 git-wrong-content-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
+ sha256 0000000000000000000000000000000000000000000000000000000000000000 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.mk b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.mk
new file mode 100644
index 0000000000..786224dad9
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-wrong-content
+#
+################################################################################
+
+GIT_WRONG_CONTENT_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+GIT_WRONG_CONTENT_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_WRONG_CONTENT_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-wrong-sha1/git-wrong-sha1.mk b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-sha1/git-wrong-sha1.mk
new file mode 100644
index 0000000000..f9d0d2226c
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-sha1/git-wrong-sha1.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-wrong-sha1
+#
+################################################################################
+
+GIT_WRONG_SHA1_VERSION = 0000000000000000000000000000000000000000
+GIT_WRONG_SHA1_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_WRONG_SHA1_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index 14fc8e4da3..fdfc858233 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -32,6 +32,15 @@ class GitTestBase(infra.basetest.BRTest):
"{}-source".format(package)],
env)
+ def check_download(self, package):
+ # store downloaded tarball inside the output dir so the test infra
+ # cleans it up at the end
+ env = {"BR2_DL_DIR": os.path.join(self.builddir, "dl"),
+ "GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
+ self.b.build(["{}-dirclean".format(package),
+ "{}-legal-info".format(package)],
+ env)
+
class TestGitHash(GitTestBase):
br2_external = [infra.filepath("tests/download/br2-external/git-hash")]
@@ -41,3 +50,13 @@ class TestGitHash(GitTestBase):
self.check_hash("bad")
self.check_hash("good")
self.check_hash("nohash")
+
+
+class TestGitRefs(GitTestBase):
+ br2_external = [infra.filepath("tests/download/br2-external/git-refs")]
+
+ def test_run(self):
+ with self.assertRaises(SystemError):
+ self.check_download("git-wrong-content")
+ with self.assertRaises(SystemError):
+ self.check_download("git-wrong-sha1")
--
2.14.1
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v4] testing/tests/download: test git branch
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
` (4 preceding siblings ...)
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test case for git refs Ricardo Martincoski
@ 2018-04-29 14:33 ` Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git submodules Ricardo Martincoski
` (8 subsequent siblings)
14 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-29 14:33 UTC (permalink / raw)
To: buildroot
Add a branch to the static repo and check on the git refs test case the
download of a git package:
- with a sha1 reachable by a branch name, but not pointed by it, as
version. This is the most common use case for git refs in the tree;
- with a partial sha1 of a commit reachable by a branch as version;
- with the name of a branch as version;
- with a sha1 of the commit head of a branch as version;
- with a partial sha1 of the commit head of a branch as version;
Enforce the download always occurs by removing the BR2_DL_DIR used for
the tarballs generated by the git download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repo with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* 68c197d (mybranch) branch2
* 3174063 branch1
* a238b1d (HEAD -> master) initial
Changes v3 -> v4:
- no functional changes;
- fix old comment on the code;
Changes v2 -> v3:
- complete rewrite using a static repo;
---
.../git-refs/package/git-branch/git-branch.hash | 2 ++
.../git-refs/package/git-branch/git-branch.mk | 11 +++++++++++
.../git-partial-sha1-branch-head.hash | 2 ++
.../git-partial-sha1-branch-head.mk | 11 +++++++++++
.../git-partial-sha1-reachable-by-branch.hash | 2 ++
.../git-partial-sha1-reachable-by-branch.mk | 11 +++++++++++
.../git-sha1-branch-head/git-sha1-branch-head.hash | 2 ++
.../package/git-sha1-branch-head/git-sha1-branch-head.mk | 11 +++++++++++
.../git-sha1-reachable-by-branch.hash | 2 ++
.../git-sha1-reachable-by-branch.mk | 11 +++++++++++
.../objects/11/93ff46343f4f6a0522e2b28b871e905178c1f0 | Bin 0 -> 23 bytes
.../objects/31/7406308d9259e2231bd0d6ddad3de3832bce08 | Bin 0 -> 182 bytes
.../objects/34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5 | Bin 0 -> 23 bytes
.../objects/68/c197d0879d485f4f6cee85544722b79e68e59f | Bin 0 -> 184 bytes
.../objects/b2/4b387624edc78d0292a127c43cad9ba97c6232 | Bin 0 -> 49 bytes
.../objects/e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e | Bin 0 -> 49 bytes
.../download/git-remote/repo.git/refs/heads/mybranch | 1 +
support/testing/tests/download/test_git.py | 12 +++++++++++-
18 files changed, 77 insertions(+), 1 deletion(-)
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.mk
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/11/93ff46343f4f6a0522e2b28b871e905178c1f0
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/31/7406308d9259e2231bd0d6ddad3de3832bce08
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/68/c197d0879d485f4f6cee85544722b79e68e59f
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/b2/4b387624edc78d0292a127c43cad9ba97c6232
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/mybranch
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.hash b/support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.hash
new file mode 100644
index 0000000000..0061618b0d
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.hash
@@ -0,0 +1,2 @@
+sha256 9697b68a51149188922b5df596f0f1f363c2abf9ee8e6feff0ec44b6bca3e69d git-branch-mybranch.tar.gz
+sha256 2c1126513651b0d346a4e6d1bb75ac1c9999217e18026302d27bea47b06c7fb2 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.mk b/support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.mk
new file mode 100644
index 0000000000..624d07aad1
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-branch/git-branch.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-branch
+#
+################################################################################
+
+GIT_BRANCH_VERSION = mybranch
+GIT_BRANCH_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_BRANCH_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash
new file mode 100644
index 0000000000..648bcceca0
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash
@@ -0,0 +1,2 @@
+sha256 70b76187369e47db69dac02c5696e63b35199cd20490fa473d289dd377774613 git-partial-sha1-branch-head-68c197d0879d485f4f6c.tar.gz
+sha256 2c1126513651b0d346a4e6d1bb75ac1c9999217e18026302d27bea47b06c7fb2 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.mk b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.mk
new file mode 100644
index 0000000000..6d4177c5bf
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-partial-sha1-branch-head
+#
+################################################################################
+
+GIT_PARTIAL_SHA1_BRANCH_HEAD_VERSION = 68c197d0879d485f4f6c
+GIT_PARTIAL_SHA1_BRANCH_HEAD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_PARTIAL_SHA1_BRANCH_HEAD_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash
new file mode 100644
index 0000000000..d064362e2c
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash
@@ -0,0 +1,2 @@
+sha256 9db079b9e215799d59975db7b2b26671eff1932ee6cf1075296f2ace3e2cb746 git-partial-sha1-reachable-by-branch-317406308d9259e2231b.tar.gz
+sha256 fabbc65c442bacb5e69b7adfea6d14fbbfc1327134322efd12771dc84387d507 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.mk b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.mk
new file mode 100644
index 0000000000..2f65b5c9a9
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-partial-sha1-reachable-by-branch
+#
+################################################################################
+
+GIT_PARTIAL_SHA1_REACHABLE_BY_BRANCH_VERSION = 317406308d9259e2231b
+GIT_PARTIAL_SHA1_REACHABLE_BY_BRANCH_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_PARTIAL_SHA1_REACHABLE_BY_BRANCH_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash
new file mode 100644
index 0000000000..3e8f76d31a
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash
@@ -0,0 +1,2 @@
+sha256 a21a2507b6d94ad484e49e3a9ae698f672a57469aab8e1779da77df7c9d4d337 git-sha1-branch-head-68c197d0879d485f4f6cee85544722b79e68e59f.tar.gz
+sha256 2c1126513651b0d346a4e6d1bb75ac1c9999217e18026302d27bea47b06c7fb2 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.mk b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.mk
new file mode 100644
index 0000000000..d3ecaa8593
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-sha1-branch-head
+#
+################################################################################
+
+GIT_SHA1_BRANCH_HEAD_VERSION = 68c197d0879d485f4f6cee85544722b79e68e59f
+GIT_SHA1_BRANCH_HEAD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SHA1_BRANCH_HEAD_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash
new file mode 100644
index 0000000000..f8d7b5dc48
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash
@@ -0,0 +1,2 @@
+sha256 8909e76d898e651af0bc23fae4103b87888bfe77448d71aaf1fbec3da97a3ce2 git-sha1-reachable-by-branch-317406308d9259e2231bd0d6ddad3de3832bce08.tar.gz
+sha256 fabbc65c442bacb5e69b7adfea6d14fbbfc1327134322efd12771dc84387d507 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.mk b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.mk
new file mode 100644
index 0000000000..badf9e13ca
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-sha1-reachable-by-branch
+#
+################################################################################
+
+GIT_SHA1_REACHABLE_BY_BRANCH_VERSION = 317406308d9259e2231bd0d6ddad3de3832bce08
+GIT_SHA1_REACHABLE_BY_BRANCH_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SHA1_REACHABLE_BY_BRANCH_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/11/93ff46343f4f6a0522e2b28b871e905178c1f0 b/support/testing/tests/download/git-remote/repo.git/objects/11/93ff46343f4f6a0522e2b28b871e905178c1f0
new file mode 100644
index 0000000000000000000000000000000000000000..3541cd14f0bf5f3a172f868c7ec730dcc255ebd2
GIT binary patch
literal 23
fcmb<m^geacKWIaOhnCkF?{fxEm>AT(SQY{RanuOb
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/31/7406308d9259e2231bd0d6ddad3de3832bce08 b/support/testing/tests/download/git-remote/repo.git/objects/31/7406308d9259e2231bd0d6ddad3de3832bce08
new file mode 100644
index 0000000000000000000000000000000000000000..c2c986572a7dec5f263e461155a09124d0f05844
GIT binary patch
literal 182
zcmV;n07?IN0j<tEP6II%Kv8P1!W)>#-;9k=lnv036+FLBVg-+p;}q<kNGa$M?J2G<
z_SWju-tAxJVAkwV*zA+69E${E??l+TU<sm!f}90XX7vSwx2eJTR4fTna*@JH$p;XR
z(K#RNiYZu2*^q+!y!4?zD#D<y-(k$UVZU9gej88RwO>4*8`RGo_IA)Vnl;DdW7doD
k#;AurrYHYXPL%;0UaV55dfu3pF0z#G*T2lF9~m)jQ^MU{p#T5?
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5 b/support/testing/tests/download/git-remote/repo.git/objects/34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5
new file mode 100644
index 0000000000000000000000000000000000000000..29f2d4fd004afa03b43e3148207e258e7aa9fc41
GIT binary patch
literal 23
fcmb<m^geacKWIaOhnCkF?{fx^m>AT3SQY^Qam@(V
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/68/c197d0879d485f4f6cee85544722b79e68e59f b/support/testing/tests/download/git-remote/repo.git/objects/68/c197d0879d485f4f6cee85544722b79e68e59f
new file mode 100644
index 0000000000000000000000000000000000000000..3e421bb97a40ab93592d94107f3a1258a99b76fa
GIT binary patch
literal 184
zcmV;p07w6L0j<tGP6IIzhGA--!WY=cJG1K_gb){?AtxBmOcE=2jf^R{Ju9V1AJIP2
z_vz(UYZ(^7KaH+xGb|Zn at UW<Gh~gjt%@BA&CP|r6U=Il9i at I+6a1M*}=%OT8lLFwJ
z#fcXoK~andnbk!@KjYH6{U}Ubnti7}%Es;cTI@@|;i2B~?OdsR9=O$4JFhOJHKIp5
mIdsnKx<?QFr#q87ZF~V^j^(_mOzm`<^6&E_vE~mXVr;h@Dp>;n
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/b2/4b387624edc78d0292a127c43cad9ba97c6232 b/support/testing/tests/download/git-remote/repo.git/objects/b2/4b387624edc78d0292a127c43cad9ba97c6232
new file mode 100644
index 0000000000000000000000000000000000000000..20491e3f176096db13c0b0c7857406e1c4119619
GIT binary patch
literal 49
zcmV-10M7q-0V^p=O;s>9VlXr?Ff%bxNXyJgWiYvTtI+!UJ(=J**;<>Gt?O at iAAbq}
H3k46<$LbTr
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e b/support/testing/tests/download/git-remote/repo.git/objects/e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e
new file mode 100644
index 0000000000000000000000000000000000000000..57be6485778c7f743a82f71ef718103cecfd4f35
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5RbbAa`)7~bv+oXld2!jhvYnibqU`nz3}1Tr
F^a08O6TJWc
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/heads/mybranch b/support/testing/tests/download/git-remote/repo.git/refs/heads/mybranch
new file mode 100644
index 0000000000..45cec54673
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/heads/mybranch
@@ -0,0 +1 @@
+68c197d0879d485f4f6cee85544722b79e68e59f
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index fdfc858233..d603e6bb6d 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -1,4 +1,5 @@
import os
+import shutil
import infra
from gitremote import GitRemote
@@ -35,7 +36,11 @@ class GitTestBase(infra.basetest.BRTest):
def check_download(self, package):
# store downloaded tarball inside the output dir so the test infra
# cleans it up@the end
- env = {"BR2_DL_DIR": os.path.join(self.builddir, "dl"),
+ dl_dir = os.path.join(self.builddir, "dl")
+ # enforce we test the download
+ if os.path.exists(dl_dir):
+ shutil.rmtree(dl_dir)
+ env = {"BR2_DL_DIR": dl_dir,
"GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
self.b.build(["{}-dirclean".format(package),
"{}-legal-info".format(package)],
@@ -60,3 +65,8 @@ class TestGitRefs(GitTestBase):
self.check_download("git-wrong-content")
with self.assertRaises(SystemError):
self.check_download("git-wrong-sha1")
+ self.check_download("git-branch")
+ self.check_download("git-partial-sha1-branch-head")
+ self.check_download("git-partial-sha1-reachable-by-branch")
+ self.check_download("git-sha1-branch-head")
+ self.check_download("git-sha1-reachable-by-branch")
--
2.14.1
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v4] testing/tests/download: test git submodules
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
` (5 preceding siblings ...)
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git branch Ricardo Martincoski
@ 2018-04-29 14:33 ` Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git tag Ricardo Martincoski
` (7 subsequent siblings)
14 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-29 14:33 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Add two submodules as static repos, add a branch to the main static repo
and check on the git refs test case the download of a git package:
- repo with submodule but without support in the package;
- repo with recursive submodules with support in the package.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repos with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* 2fa37f6 (submodule) sub0-2
* a9dbc1e sub0-1
| * 68c197d (mybranch) branch2
| * 3174063 branch1
|/
* a238b1d (HEAD -> master) initial
$ git -C support/testing/tests/download/git-remote/refs-sub1.git \
log --all --decorate --graph --oneline --decorate
* 1df823c (submodule) sub1-2
* f8001e5 sub1-1
* cb545fa (HEAD) initial
$ git -C support/testing/tests/download/git-remote/refs-sub2.git \
log --all --decorate --graph --oneline --decorate
* e83f6f8 (submodule) sub2-2
* cd4d62f sub2-1
* 32d61ba (HEAD) initial
In this layout:
- sub0-1 adds refs-sub1.git as submodule to refs.git pointing to sub1-1
commit;
- sub1-1 adds refs-sub2.git as submodule to refs-sub1.git pointing to
sub2-1 commit.
Changes v3 -> v4:
- add hash for submodule enabled tarball;
Changes v2 -> v3:
- complete rewrite using a static repo;
Changes v1 -> v2:
- rewrite using new git test infra;
- this patch is part of series 1/3 of a new version of
http://patchwork.ozlabs.org/patch/690097/
---
.../git-submodule-disabled.hash | 2 ++
.../git-submodule-disabled.mk | 11 ++++++++
.../git-submodule-enabled.hash | 4 +++
.../git-submodule-enabled/git-submodule-enabled.mk | 28 +++++++++++++++++++++
.../git-remote/refs-sub1.git/.gitattributes | 1 +
.../tests/download/git-remote/refs-sub1.git/HEAD | 1 +
.../tests/download/git-remote/refs-sub1.git/config | 4 +++
.../07/cabc655213bdf7087d8dd50fda95124e935570 | Bin 0 -> 22 bytes
.../1d/f823cb8a6d1866148ae50a8009762a9c4c777f | Bin 0 -> 180 bytes
.../7d/52f458bdb0f9f5a4beb46fa82824421b8b988e | Bin 0 -> 65 bytes
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a | Bin 0 -> 122 bytes
.../bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23 | Bin 0 -> 22 bytes
.../cb/545facf77bbc5f24f95b6d503c338d10b7b717 | Bin 0 -> 151 bytes
.../dd/130f6f4629514adaf2e03407f3ed9344eb6cd6 | Bin 0 -> 122 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../f8/001e5780100962a5e30a25cbc4c9609cfc7bf3 | Bin 0 -> 184 bytes
.../git-remote/refs-sub1.git/refs/heads/submodule | 1 +
.../git-remote/refs-sub2.git/.gitattributes | 1 +
.../tests/download/git-remote/refs-sub2.git/HEAD | 1 +
.../tests/download/git-remote/refs-sub2.git/config | 4 +++
.../0a/846af45c3e455789435f49f80d70e86b65b9d7 | Bin 0 -> 22 bytes
.../0b/32ebd8fc52cec991f18c94be980e85a8341585 | Bin 0 -> 22 bytes
.../32/d61bae693af7879da63b89a60d3ae67f851e56 | Bin 0 -> 151 bytes
.../3e/9b0a5198c64cea9c00f820433411e3b4d50c1c | Bin 0 -> 48 bytes
.../85/8f666af75b7c0dfba6b8be7eac5f196e7a1221 | Bin 0 -> 49 bytes
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb | Bin 0 -> 184 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../e8/3f6f805bd016b90acafc8702c52d778eb57310 | Bin 0 -> 182 bytes
.../git-remote/refs-sub2.git/refs/heads/submodule | 1 +
.../0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343 | Bin 0 -> 22 bytes
.../25/59d83bfe937fc0412d96ed664663c9e8a99055 | Bin 0 -> 22 bytes
.../2f/a37f6885d7eb746df75eccaddbacf3ac82799d | Bin 0 -> 180 bytes
.../8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7 | Bin 0 -> 65 bytes
.../a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f | Bin 0 -> 183 bytes
.../d4/e2941d18a63535400476324ddeb7f40164be41 | Bin 0 -> 121 bytes
.../fe/74231105841041d5f441e70399d37f0e600aa5 | Bin 0 -> 121 bytes
.../git-remote/repo.git/refs/heads/submodule | 1 +
support/testing/tests/download/test_git.py | 2 ++
40 files changed, 62 insertions(+)
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.mk
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/config
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/07/cabc655213bdf7087d8dd50fda95124e935570
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/1d/f823cb8a6d1866148ae50a8009762a9c4c777f
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/7d/52f458bdb0f9f5a4beb46fa82824421b8b988e
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/cb/545facf77bbc5f24f95b6d503c338d10b7b717
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/dd/130f6f4629514adaf2e03407f3ed9344eb6cd6
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/f8/001e5780100962a5e30a25cbc4c9609cfc7bf3
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/refs/heads/submodule
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/config
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/0a/846af45c3e455789435f49f80d70e86b65b9d7
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/0b/32ebd8fc52cec991f18c94be980e85a8341585
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/32/d61bae693af7879da63b89a60d3ae67f851e56
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/3e/9b0a5198c64cea9c00f820433411e3b4d50c1c
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/85/8f666af75b7c0dfba6b8be7eac5f196e7a1221
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/e8/3f6f805bd016b90acafc8702c52d778eb57310
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/refs/heads/submodule
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/25/59d83bfe937fc0412d96ed664663c9e8a99055
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/2f/a37f6885d7eb746df75eccaddbacf3ac82799d
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/d4/e2941d18a63535400476324ddeb7f40164be41
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/fe/74231105841041d5f441e70399d37f0e600aa5
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/submodule
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash
new file mode 100644
index 0000000000..3bd0a44693
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash
@@ -0,0 +1,2 @@
+sha256 f9d46ff8a1a344c6c31fa4211220f3085c446abd31626232540703158276f22c git-submodule-disabled-a9dbc1e23c45e8e1b88c0448763f54d714eb6f8f.tar.gz
+sha256 ba8b6ddc4726bfb6a05045ebfd8c43263c968ad1bc601bd46a25bc055008eddc file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.mk b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.mk
new file mode 100644
index 0000000000..7a35b3b51a
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-submodule-disabled
+#
+################################################################################
+
+GIT_SUBMODULE_DISABLED_VERSION = a9dbc1e23c45e8e1b88c0448763f54d714eb6f8f
+GIT_SUBMODULE_DISABLED_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SUBMODULE_DISABLED_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash
new file mode 100644
index 0000000000..089eed2365
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash
@@ -0,0 +1,4 @@
+sha256 139a34c3c844c844dee74b6746418cfa75fbcc4205794ac8c0b3cd7d55a76792 git-submodule-enabled-a9dbc1e23c45e8e1b88c0448763f54d714eb6f8f.tar.gz
+sha256 ba8b6ddc4726bfb6a05045ebfd8c43263c968ad1bc601bd46a25bc055008eddc file
+sha256 5219dcb50c7753bcdd72bc68a7b48af33ac2e42c5c61de78a9dd81589c4d50b6 refs-sub1/file
+sha256 b93c1fa9014d25a17fee36771d26ae023f043da656315ffe8947c30ad1ba141f refs-sub1/refs-sub2/file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.mk b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.mk
new file mode 100644
index 0000000000..019dd076e3
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.mk
@@ -0,0 +1,28 @@
+################################################################################
+#
+# git-submodule-enabled
+#
+################################################################################
+
+GIT_SUBMODULE_ENABLED_VERSION = a9dbc1e23c45e8e1b88c0448763f54d714eb6f8f
+GIT_SUBMODULE_ENABLED_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SUBMODULE_ENABLED_GIT_SUBMODULES = YES
+GIT_SUBMODULE_ENABLED_LICENSE_FILES = \
+ file \
+ refs-sub1/file \
+ refs-sub1/refs-sub2/file
+
+# Some versions of git client fill the .git file for the second level submodule
+# using the absolute path to the main .git directory, which in the case of the
+# buildroot download helper is always different since it uses a temporary
+# directory. This version of git have the issue:
+# - 2.7.4 included in Ubuntu 16.04;
+# The same does not occur using newer versions of git:
+# - 2.11.0 included in Debian 9;
+# - 2.14.2 latest at the time of writing;
+# In order to allow running this test case locally in many systems, do not check
+# for the hash of the tarball, but instead check the hash of each meaningful
+# file included in the tarball.
+BR_NO_CHECK_HASH_FOR += $(GIT_SUBMODULE_ENABLED_SOURCE)
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/.gitattributes b/support/testing/tests/download/git-remote/refs-sub1.git/.gitattributes
new file mode 100644
index 0000000000..eb50c64a21
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub1.git/.gitattributes
@@ -0,0 +1 @@
+objects/*/* binary
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/HEAD b/support/testing/tests/download/git-remote/refs-sub1.git/HEAD
new file mode 100644
index 0000000000..4e92c1f200
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub1.git/HEAD
@@ -0,0 +1 @@
+cb545facf77bbc5f24f95b6d503c338d10b7b717
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/config b/support/testing/tests/download/git-remote/refs-sub1.git/config
new file mode 100644
index 0000000000..07d359d07c
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub1.git/config
@@ -0,0 +1,4 @@
+[core]
+ repositoryformatversion = 0
+ filemode = true
+ bare = true
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/07/cabc655213bdf7087d8dd50fda95124e935570 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/07/cabc655213bdf7087d8dd50fda95124e935570
new file mode 100644
index 0000000000000000000000000000000000000000..cbf5085af8cdca19c43017c92369ddc8002f1a3d
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXfis>UJ21Qop+W=aH2UY+8
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/1d/f823cb8a6d1866148ae50a8009762a9c4c777f b/support/testing/tests/download/git-remote/refs-sub1.git/objects/1d/f823cb8a6d1866148ae50a8009762a9c4c777f
new file mode 100644
index 0000000000000000000000000000000000000000..04055b899ecbdc3bceb2d7936757f360e90544a5
GIT binary patch
literal 180
zcmV;l089UP0j-Wf4#FT106p^xK2WpJg47t}2fXMDb{9~h6hrag?``72yYV{7OomI&
zsmWfORjVOLR6Sr+R$C(U-VSb1q?wp3@(?`O-VrJk53FHoGCBYi%*KHN7_CMVmVh>k
zG4N<%<j5O}y})O?mMTw)vHBv9SX-L7%(o<W^#j-W3lEo!>AvQY_fi>fV64+lcAx>o
iLYL_l{*)8*lT at dLIH$`Ld{<)Z=(YXjRlEVZ`fTHP_gG8-
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/7d/52f458bdb0f9f5a4beb46fa82824421b8b988e b/support/testing/tests/download/git-remote/refs-sub1.git/objects/7d/52f458bdb0f9f5a4beb46fa82824421b8b988e
new file mode 100644
index 0000000000000000000000000000000000000000..2b78a01965068cc6f7c89402eea4e7f9a03ba807
GIT binary patch
literal 65
zcmV-H0KWft0ZYosPf{>5V~8#;P0GzrDa}b$P%27IE7k?_jg(@!I13U>G8AkT;9^{y
XrA0YF0X;o^xR_pgW(gMne83k<GdCCZ
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab b/support/testing/tests/download/git-remote/refs-sub1.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
new file mode 100644
index 0000000000000000000000000000000000000000..9db72668cf9374b0b85a25a19f30084fd460072d
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5wZcw)UHz=nDZYB`lCt5lYr9_mN?Omvz#_so
F4FHVv5b^*3
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a b/support/testing/tests/download/git-remote/refs-sub1.git/objects/a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a
new file mode 100644
index 0000000000000000000000000000000000000000..f49d53326e0b22b95fd16670f0a25e3cd84718ae
GIT binary patch
literal 122
zcmV-=0EPc}0V^p=O;s>7uwXDWFfcPQQP4}zEXmDJDa}bOW~dGN60vu~&#z1NZOLDu
zq2eUnJ);k*EG;u9m4W@#p41 at Wz27-%d$01}nkwWsIkdpg%m4@!ic-^xb&E at rj2O=P
cCjD3J+P-ChfArrU3~L^=ueW&(03qxzkxyhgmjD0&
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23
new file mode 100644
index 0000000000000000000000000000000000000000..7d2a9d55c1a92dfa2122308cc05a4a5b96d70d8f
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXfis=(321RD(TL4;&2UP$7
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/cb/545facf77bbc5f24f95b6d503c338d10b7b717 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/cb/545facf77bbc5f24f95b6d503c338d10b7b717
new file mode 100644
index 0000000000000000000000000000000000000000..dc7de29459f4156b999f14fdeb6cb58b58c21a19
GIT binary patch
literal 151
zcmV;I0BHYs0j<ux4Z<)GKw)Z6aSJ3mJF*iagjj$ECOH3v6LBmVQ?NaeDCiOGS3Esm
za^?nVoOZ3o2-YSQhbX<1I$5VxB4v{?VVE!kLL-BfBRN6Ob}JPw?5TzVXR3`?Umkac
zL;YZ%zj&H6@zVQ}k03QpsEsrT9V$ez={C{gpK^j%ZcK|1SDx1xx{^}oUi)8W#T!=;
FU>B}CKEMC~
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/dd/130f6f4629514adaf2e03407f3ed9344eb6cd6 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/dd/130f6f4629514adaf2e03407f3ed9344eb6cd6
new file mode 100644
index 0000000000000000000000000000000000000000..37037d3920820ae2dab7e40d0c1df612af6f7a94
GIT binary patch
literal 122
zcmV-=0EPc}0V^p=O;s>7uwXDWFfcPQQP4}zEXmDJDa}bOW~dGN60vu~&#z1NZOLDu
zq2eUnJ);k*EG;u9m0`~b>19v%c7N5Gn$h**^0Vm=4Sy*cni&9rLQ!g3v2Jl`k`cpM
c-=zPFUE8-T at Q?ocgJI2s_VqTe0l at My>J95W4gdfE
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
new file mode 100644
index 0000000000000000000000000000000000000000..df2037a2d2d30afb20e31cd558c92c2edabe7cf6
GIT binary patch
literal 23
fcmb<m^geacKWIb3i8H5lPWqgA$i$$b%kl;Qg;xrl
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/f8/001e5780100962a5e30a25cbc4c9609cfc7bf3 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/f8/001e5780100962a5e30a25cbc4c9609cfc7bf3
new file mode 100644
index 0000000000000000000000000000000000000000..d824d3c17c1cc1c13a18eb1d2c0a7e59f66ba08b
GIT binary patch
literal 184
zcmV;p07w6L0j<tGP6IIzKw)a1!WY<R?D5YoLWm2{Bq#8AJi!95k>eEHo=7RuBigTc
zdbzh&r$+0a<|suo8w3gBg5A7viIdm@=pYIf-q5*1A#+gQX_S5HOx}4(Sz;v0UaXVN
zQ(zC65mz=qk)nwzUFSz1`dD)s-1U1Jv+lWHKC6BicRaKQzMWgDUx(b=t2Q21@L>hi
m3s?Yk)7$hc|CCdgE1AV8b*ksR at X|#V`g#7#tNI184{X$IS6W&C
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/refs/heads/submodule b/support/testing/tests/download/git-remote/refs-sub1.git/refs/heads/submodule
new file mode 100644
index 0000000000..bf97dc4185
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub1.git/refs/heads/submodule
@@ -0,0 +1 @@
+1df823cb8a6d1866148ae50a8009762a9c4c777f
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/.gitattributes b/support/testing/tests/download/git-remote/refs-sub2.git/.gitattributes
new file mode 100644
index 0000000000..eb50c64a21
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub2.git/.gitattributes
@@ -0,0 +1 @@
+objects/*/* binary
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/HEAD b/support/testing/tests/download/git-remote/refs-sub2.git/HEAD
new file mode 100644
index 0000000000..bbf19a4bb4
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub2.git/HEAD
@@ -0,0 +1 @@
+32d61bae693af7879da63b89a60d3ae67f851e56
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/config b/support/testing/tests/download/git-remote/refs-sub2.git/config
new file mode 100644
index 0000000000..07d359d07c
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub2.git/config
@@ -0,0 +1,4 @@
+[core]
+ repositoryformatversion = 0
+ filemode = true
+ bare = true
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/0a/846af45c3e455789435f49f80d70e86b65b9d7 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/0a/846af45c3e455789435f49f80d70e86b65b9d7
new file mode 100644
index 0000000000000000000000000000000000000000..aaa29dbd8a5a74a24f51a8522b695a7474349f3c
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXflIasB21Rz}+W=aJ2Uq|A
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/0b/32ebd8fc52cec991f18c94be980e85a8341585 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/0b/32ebd8fc52cec991f18c94be980e85a8341585
new file mode 100644
index 0000000000000000000000000000000000000000..50caf8b6282d2287878d1b3481c14431c2a557af
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXflIbHR21QQhI{;dM2U!3B
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/32/d61bae693af7879da63b89a60d3ae67f851e56 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/32/d61bae693af7879da63b89a60d3ae67f851e56
new file mode 100644
index 0000000000000000000000000000000000000000..a8887b1e611d8f76d7cfe6ab0bb693fdebc2ef19
GIT binary patch
literal 151
zcmV;I0BHYs0j<ux4Z<)GKw)Z6aSJ3mJF*iagjj$ECOH3v6LBmVQ?NaeDCiOGS3Esm
za^?nVoOZ3o2-YSQhbX<1I$5VxB4v{?VVE!kLL-BfBRN6Ob}JPw?5TzVXR3`?Umkac
zL;YZ%zj&H6 at zVQ}k03QpsEr&gbf^%;rrSh|f6577xiKw5TzOt&=t@eRd+mRj6>nh7
FU at NX0KJNek
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/3e/9b0a5198c64cea9c00f820433411e3b4d50c1c b/support/testing/tests/download/git-remote/refs-sub2.git/objects/3e/9b0a5198c64cea9c00f820433411e3b4d50c1c
new file mode 100644
index 0000000000000000000000000000000000000000..b1fb722c53d827847c6d13029e601a5434b84ac4
GIT binary patch
literal 48
zcmV-00MGw;0V^p=O;s>9VlXr?Ff%bxNXyJgW#DSb`VwR38s6y~@A-qb;6--o&g%g8
GZVq_HQWiM?
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/85/8f666af75b7c0dfba6b8be7eac5f196e7a1221 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/85/8f666af75b7c0dfba6b8be7eac5f196e7a1221
new file mode 100644
index 0000000000000000000000000000000000000000..10f24a087ff08320b8fc5922db728472bd276e9a
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5^?_Mc?PLE7J0}a<fAhI%-n_oFwKGGTLGrBt
FHvqLX67T>3
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab b/support/testing/tests/download/git-remote/refs-sub2.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
new file mode 100644
index 0000000000000000000000000000000000000000..9db72668cf9374b0b85a25a19f30084fd460072d
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5wZcw)UHz=nDZYB`lCt5lYr9_mN?Omvz#_so
F4FHVv5b^*3
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb b/support/testing/tests/download/git-remote/refs-sub2.git/objects/cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb
new file mode 100644
index 0000000000000000000000000000000000000000..35d253f0d15b31ea0bc79ed1a287819deed9987d
GIT binary patch
literal 184
zcmV;p07w6L0j<tGZUZ3@hGEy7f(xuQEI$j8W%&YWk`oNWU?YdcVsVk%XQLG9qqL9o
zeR`?YT87lKJ&evQHHQdjRq*5tF$Mz2eg>npQHpgm$pR at 75A3{t50g_7*#(Vx at k>bP
zbo2ooB)#*Jy;W?Tz~{KNF5e1aPffmJA7!Ps-HSZ+A3WAyyj&}mk0Z5umdb($>r7bW
m1T%oxb&nqWPj_N_#(7dgyqD|BX=*u7@$=;`t>O#gcx)*r3s*J(
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
new file mode 100644
index 0000000000000000000000000000000000000000..df2037a2d2d30afb20e31cd558c92c2edabe7cf6
GIT binary patch
literal 23
fcmb<m^geacKWIb3i8H5lPWqgA$i$$b%kl;Qg;xrl
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/e8/3f6f805bd016b90acafc8702c52d778eb57310 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/e8/3f6f805bd016b90acafc8702c52d778eb57310
new file mode 100644
index 0000000000000000000000000000000000000000..05c04d2887e9354ec43ad69cef20ed4e7166013d
GIT binary patch
literal 182
zcmV;n07?IN0j<tKPD3FOfMM61f(yK6pl<+;F<!upo?r%MT2gr!N*CVVCN8>fy8RY^
z{@h!u)9yV!%^_kFCJiBwM&y{GC{y6X5`{9VdkzxGdGE{*4cVs6Wi7$0dY6bJuLP?i
zYtaNiSs+EyeHJ!!na4itXU#N9w;wcS-Eu#ls(l^*xNA3jKQyY}yWHEGbqEPCphLTW
k2QXK?PEUDI&YUmo7jMj0J#12zuDbB=%RR5=4+?N at DHiEdA^-pY
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/refs/heads/submodule b/support/testing/tests/download/git-remote/refs-sub2.git/refs/heads/submodule
new file mode 100644
index 0000000000..39dc53f019
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub2.git/refs/heads/submodule
@@ -0,0 +1 @@
+e83f6f805bd016b90acafc8702c52d778eb57310
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343 b/support/testing/tests/download/git-remote/repo.git/objects/0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343
new file mode 100644
index 0000000000000000000000000000000000000000..ef7fbd4154aea755d8ee2684e420700467e9245c
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXfg6R_`28I93HvwH&2ulC}
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/25/59d83bfe937fc0412d96ed664663c9e8a99055 b/support/testing/tests/download/git-remote/repo.git/objects/25/59d83bfe937fc0412d96ed664663c9e8a99055
new file mode 100644
index 0000000000000000000000000000000000000000..cbf60fc7298ec12b3d7cd388fecac606ca31536e
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXfg6ShB21Q2ZTL4;A2TcF~
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/2f/a37f6885d7eb746df75eccaddbacf3ac82799d b/support/testing/tests/download/git-remote/repo.git/objects/2f/a37f6885d7eb746df75eccaddbacf3ac82799d
new file mode 100644
index 0000000000000000000000000000000000000000..2dd14580e666d35483727598f006148f3eeb03be
GIT binary patch
literal 180
zcmV;l089UP0j-Wp4#FT509|tmF3{!yj~0zFUcil>fX_pTQVgXFZ*LP9?v2|?W-`9y
zoEleBZ(1ed0ZDFA1Yzu;w9*KG<DevlKyHi(Vn9bI*wa>;8n at fvydbGOYBD68F&+ at j
z>O_rU6i9A}CNjIUTdDk<yse?|ldUaHzRdT;_w at t&{DsGBw&~IPk`G*HFrddZ%R7(&
iY^CdTqd(=?yg2A2W0!QDD0C&p&b@ZOys|fCz-zg3O;?Bj
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7 b/support/testing/tests/download/git-remote/repo.git/objects/8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7
new file mode 100644
index 0000000000000000000000000000000000000000..70cf1795f3210f81951d0995823bb958867ba710
GIT binary patch
literal 65
zcmb<m^geacKghs1xkFFS_taT`&9k1N0a{+Z+Sgb=@lH+JX~8YzZ6uN-`XDK0&Qe!P
V2S!`l+8s0Es?s>c8ET5e-2otN7bXAz
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f b/support/testing/tests/download/git-remote/repo.git/objects/a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f
new file mode 100644
index 0000000000000000000000000000000000000000..2cbdce1f95493b284df1f8ca3ca953a5fb813072
GIT binary patch
literal 183
zcmV;o07(CM0j<t4ZUZ3@K+)8kf(xuQ14Ce~D9Q!g#3z_#z(F=kgmK~9XC)Wzt+cQB
z`mxnohPKANjh>Ru$&;~&-~u5KipxZpJlJ^_>ocTx0A{Ctxu<n#Ha09MIZ7~2$V;%q
z*+yrKCG>Htpv)kOZ{w$R{aqq=Y5Kr at lr^^Nx#*Anz+HXf*Qs*3>|(2*8XW|19*ox$
l7yz~DHa*fm<<#}Iz=TR2%V|wAwVbErzJKLa{Q<P{YY&#BUEBZw
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/d4/e2941d18a63535400476324ddeb7f40164be41 b/support/testing/tests/download/git-remote/repo.git/objects/d4/e2941d18a63535400476324ddeb7f40164be41
new file mode 100644
index 0000000000000000000000000000000000000000..4d82169300d0d8a2e31ced481501bff6647ef6b7
GIT binary patch
literal 121
zcmV-<0EYi~0V^p=O;s>7uwXDWFfcPQQP4}zEXmDJDa}bOX6Wv!>xg2JU9K(na!F0}
zWQL7JE1yG^rDf)%GN?x0u>Lo>{(z(Iw6|$)$tPc|oDgbgW&i{VMX71Uy2YhQh73O#
b<iZ;SIFpt>=2AU<<YdB}Kh>WBpF}NR{H!<P
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/fe/74231105841041d5f441e70399d37f0e600aa5 b/support/testing/tests/download/git-remote/repo.git/objects/fe/74231105841041d5f441e70399d37f0e600aa5
new file mode 100644
index 0000000000000000000000000000000000000000..6d0ae484c3333d6b18cf726d5bb468669f9d16c1
GIT binary patch
literal 121
zcmV-<0EYi~0V^p=O;s>7uwXDWFfcPQQP4}zEXmDJDa}bOX6Wv!>xg2JU9K(na!F0}
zWQL7JE1yG^rDf)%GH~BGw=?nZrcDChywb1Md&gYMyzFdfW&i{VMX71Uy2YhQh73O#
b<iZ;SIFpt>=2AU<<YdB}Kh>WBwy!O?8)rGE
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/heads/submodule b/support/testing/tests/download/git-remote/repo.git/refs/heads/submodule
new file mode 100644
index 0000000000..e9f4f32c30
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/heads/submodule
@@ -0,0 +1 @@
+2fa37f6885d7eb746df75eccaddbacf3ac82799d
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index d603e6bb6d..07c5762710 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -70,3 +70,5 @@ class TestGitRefs(GitTestBase):
self.check_download("git-partial-sha1-reachable-by-branch")
self.check_download("git-sha1-branch-head")
self.check_download("git-sha1-reachable-by-branch")
+ self.check_download("git-submodule-disabled")
+ self.check_download("git-submodule-enabled")
--
2.14.1
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v4] testing/tests/download: test git tag
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
` (6 preceding siblings ...)
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git submodules Ricardo Martincoski
@ 2018-04-29 14:33 ` Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git tag/branch precedence Ricardo Martincoski
` (6 subsequent siblings)
14 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-29 14:33 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Add a tag to the static repo and check on the git refs test case the
download of a git package:
- with the name of a tag as version;
- with the sha1 of a tag itself as version;
- with the partial sha1 of a tag itself as version;
- with the sha1 of a commit pointed by a tag as version;
- with the partial sha1 of a commit pointed by a tag as version;
- with the sha1 of a commit reachable only by a tag as version;
- with the partial sha1 of a commit reachable only by a tag as version.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repo with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* 2fa37f6 (submodule) sub0-2
* a9dbc1e sub0-1
| * 68c197d (mybranch) branch2
| * 3174063 branch1
|/
| * 516c9c5 (tag: mytag) tag2
| * 46bae5b tag1
|/
* a238b1d (HEAD -> master) initial
Changes v1 -> v4:
- not included on v2 or v3;
- rewrite from v1 using new git test infra with static repo;
---
.../git-partial-sha1-reachable-by-tag.hash | 2 ++
.../git-partial-sha1-reachable-by-tag.mk | 11 +++++++++++
.../git-partial-sha1-tag-itself.hash | 2 ++
.../git-partial-sha1-tag-itself.mk | 11 +++++++++++
.../git-partial-sha1-tag-points-to.hash | 2 ++
.../git-partial-sha1-tag-points-to.mk | 11 +++++++++++
.../git-sha1-reachable-by-tag.hash | 2 ++
.../git-sha1-reachable-by-tag.mk | 11 +++++++++++
.../package/git-sha1-tag-itself/git-sha1-tag-itself.hash | 2 ++
.../package/git-sha1-tag-itself/git-sha1-tag-itself.mk | 11 +++++++++++
.../git-sha1-tag-points-to/git-sha1-tag-points-to.hash | 2 ++
.../git-sha1-tag-points-to/git-sha1-tag-points-to.mk | 11 +++++++++++
.../br2-external/git-refs/package/git-tag/git-tag.hash | 2 ++
.../br2-external/git-refs/package/git-tag/git-tag.mk | 11 +++++++++++
.../objects/2b/0e0d98a49c97da6a618ab36337e2058eb733a2 | Bin 0 -> 137 bytes
.../objects/46/bae5b639e5a18e2cc4dc508f080d566baeff59 | Bin 0 -> 182 bytes
.../objects/51/6c9c5f64ec66534d4d069c2e408d9ae4dce023 | Bin 0 -> 182 bytes
.../objects/92/ef85be57d627f280d8ce3724452ac21c9a6452 | Bin 0 -> 20 bytes
.../objects/cf/0f4f85d7a1237e377a2d25b996518a877ea001 | Bin 0 -> 49 bytes
.../objects/e2/2695cbf976fed1f543ad7486a531c0af473482 | Bin 0 -> 49 bytes
.../objects/f6/476b879f65e956d7dedd5b08736369e9a24acc | Bin 0 -> 20 bytes
.../tests/download/git-remote/repo.git/refs/tags/mytag | 1 +
support/testing/tests/download/test_git.py | 7 +++++++
23 files changed, 99 insertions(+)
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.mk
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/2b/0e0d98a49c97da6a618ab36337e2058eb733a2
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/46/bae5b639e5a18e2cc4dc508f080d566baeff59
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/51/6c9c5f64ec66534d4d069c2e408d9ae4dce023
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/92/ef85be57d627f280d8ce3724452ac21c9a6452
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/cf/0f4f85d7a1237e377a2d25b996518a877ea001
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e2/2695cbf976fed1f543ad7486a531c0af473482
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/f6/476b879f65e956d7dedd5b08736369e9a24acc
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/tags/mytag
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.hash b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.hash
new file mode 100644
index 0000000000..7696086690
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.hash
@@ -0,0 +1,2 @@
+sha256 f2ef9772f13a9ef9a2c7cde0795e179defb12320d1747fc8d2408748ef5844c2 git-partial-sha1-reachable-by-tag-46bae5b639e5a18e2cc4.tar.gz
+sha256 2de87d77a2f226813f2d9bda906e970e4195605cdba6680443c0c04d89c532b6 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.mk b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.mk
new file mode 100644
index 0000000000..05aa659dd2
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-partial-sha1-reachable-by-tag
+#
+################################################################################
+
+GIT_PARTIAL_SHA1_REACHABLE_BY_TAG_VERSION = 46bae5b639e5a18e2cc4
+GIT_PARTIAL_SHA1_REACHABLE_BY_TAG_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_PARTIAL_SHA1_REACHABLE_BY_TAG_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.hash b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.hash
new file mode 100644
index 0000000000..e627caf91e
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.hash
@@ -0,0 +1,2 @@
+sha256 721143b41b8e56cfd9025833f1602e900a490627db2504e5b4907baa23e0019e git-partial-sha1-tag-itself-2b0e0d98a49c97da6a61.tar.gz
+sha256 6de8772a0a58fa62e2b8c58d4dae55c9db7534ad3b3918ecc849a9008d58f081 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.mk b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.mk
new file mode 100644
index 0000000000..515492397e
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-partial-sha1-tag-itself
+#
+################################################################################
+
+GIT_PARTIAL_SHA1_TAG_ITSELF_VERSION = 2b0e0d98a49c97da6a61
+GIT_PARTIAL_SHA1_TAG_ITSELF_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_PARTIAL_SHA1_TAG_ITSELF_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.hash b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.hash
new file mode 100644
index 0000000000..f957a0e23c
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.hash
@@ -0,0 +1,2 @@
+sha256 0fbf7fe935f962ceaafcf8e0ffd25dd2a83752c3f0fd055a942a53f8c9235fa7 git-partial-sha1-tag-points-to-516c9c5f64ec66534d4d.tar.gz
+sha256 6de8772a0a58fa62e2b8c58d4dae55c9db7534ad3b3918ecc849a9008d58f081 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.mk b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.mk
new file mode 100644
index 0000000000..c810e81175
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-partial-sha1-tag-points-to
+#
+################################################################################
+
+GIT_PARTIAL_SHA1_TAG_POINTS_TO_VERSION = 516c9c5f64ec66534d4d
+GIT_PARTIAL_SHA1_TAG_POINTS_TO_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_PARTIAL_SHA1_TAG_POINTS_TO_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.hash b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.hash
new file mode 100644
index 0000000000..0eb0ca0917
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.hash
@@ -0,0 +1,2 @@
+sha256 9b20256a3058221a8e91031f11700d9945ea84e8f328fa8e42e1cb9f7a30e3b2 git-sha1-reachable-by-tag-46bae5b639e5a18e2cc4dc508f080d566baeff59.tar.gz
+sha256 2de87d77a2f226813f2d9bda906e970e4195605cdba6680443c0c04d89c532b6 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.mk b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.mk
new file mode 100644
index 0000000000..b5fde7b586
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-sha1-reachable-by-tag
+#
+################################################################################
+
+GIT_SHA1_REACHABLE_BY_TAG_VERSION = 46bae5b639e5a18e2cc4dc508f080d566baeff59
+GIT_SHA1_REACHABLE_BY_TAG_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SHA1_REACHABLE_BY_TAG_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.hash b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.hash
new file mode 100644
index 0000000000..48c1348538
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.hash
@@ -0,0 +1,2 @@
+sha256 7d301c1a1054d6aee49193ca9e938f4da561ff73fb01719662865aa38bdc4361 git-sha1-tag-itself-2b0e0d98a49c97da6a618ab36337e2058eb733a2.tar.gz
+sha256 6de8772a0a58fa62e2b8c58d4dae55c9db7534ad3b3918ecc849a9008d58f081 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.mk b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.mk
new file mode 100644
index 0000000000..8914496653
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-sha1-tag-itself
+#
+################################################################################
+
+GIT_SHA1_TAG_ITSELF_VERSION = 2b0e0d98a49c97da6a618ab36337e2058eb733a2
+GIT_SHA1_TAG_ITSELF_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SHA1_TAG_ITSELF_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.hash b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.hash
new file mode 100644
index 0000000000..3dcafc2094
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.hash
@@ -0,0 +1,2 @@
+sha256 c1f9f5734529a31140a71c031534460811f001b4db37d26833f386358150ab47 git-sha1-tag-points-to-516c9c5f64ec66534d4d069c2e408d9ae4dce023.tar.gz
+sha256 6de8772a0a58fa62e2b8c58d4dae55c9db7534ad3b3918ecc849a9008d58f081 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.mk b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.mk
new file mode 100644
index 0000000000..adaae7329c
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-sha1-tag-points-to
+#
+################################################################################
+
+GIT_SHA1_TAG_POINTS_TO_VERSION = 516c9c5f64ec66534d4d069c2e408d9ae4dce023
+GIT_SHA1_TAG_POINTS_TO_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SHA1_TAG_POINTS_TO_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.hash b/support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.hash
new file mode 100644
index 0000000000..1cd0b15f27
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.hash
@@ -0,0 +1,2 @@
+sha256 85dcb5bcf9bed496688d0eb01c7a3ce05c5b46b984cc1e9e76a6dbefd976e6b3 git-tag-mytag.tar.gz
+sha256 6de8772a0a58fa62e2b8c58d4dae55c9db7534ad3b3918ecc849a9008d58f081 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.mk b/support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.mk
new file mode 100644
index 0000000000..6960ceb2cb
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-tag
+#
+################################################################################
+
+GIT_TAG_VERSION = mytag
+GIT_TAG_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_TAG_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/2b/0e0d98a49c97da6a618ab36337e2058eb733a2 b/support/testing/tests/download/git-remote/repo.git/objects/2b/0e0d98a49c97da6a618ab36337e2058eb733a2
new file mode 100644
index 0000000000000000000000000000000000000000..39c913094a3f2d8d9d84cda13a89306bc577f982
GIT binary patch
literal 137
zcmV;40CxX)0Y#2W4gw(%1zmHBzJQY+enMi57jWYZuxSY at gF!ZPdoi<c^HQlr<@JzY
zHfmRI52g-gw&cm&t!A>;C>?b~>q)Xk7kywIiBT%i*Ea*zQgRo4G|D>mM~NF;Gldo_
rTte%4q564dIJQkr<uA{vg#4OD<}^;=!5b&h!5$T&5L at yK5lk}^QNuc<
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/46/bae5b639e5a18e2cc4dc508f080d566baeff59 b/support/testing/tests/download/git-remote/repo.git/objects/46/bae5b639e5a18e2cc4dc508f080d566baeff59
new file mode 100644
index 0000000000000000000000000000000000000000..3bf8865ea67386bd0fd88ab7222392d5b66b3607
GIT binary patch
literal 182
zcmV;n07?IN0j<t4ZUZ3 at KvCD6f(xw0FyLXMDDnl;#3z_#V2u)%2$RCMx1|*BEw``u
z`mxnohDM9K(Noefo9uG%LJSruArV`#ewpWkhrod-u`!^2xu<Ptwip(W93@yUjwGPt
zjL}<*4k!i(AhQw0f8*S`{wk5XH2ukalr6T)chT?tANTcv$F*|#*~eBtG<ZX^4{-2$
kGH8st>wS8p7v<F86_h%abxSg}oTugb_ROie0nJ8hV&q3%KL7v#
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/51/6c9c5f64ec66534d4d069c2e408d9ae4dce023 b/support/testing/tests/download/git-remote/repo.git/objects/51/6c9c5f64ec66534d4d069c2e408d9ae4dce023
new file mode 100644
index 0000000000000000000000000000000000000000..0bda23ab4d6487524fc254c54b673e53bfe69435
GIT binary patch
literal 182
zcmV;n07?IN0j<tUP6IIzMN!6Hg*VWVf8`HCQ8vJYtf0DFZmi%oa-0FXClLeYi0OIK
z)1}^8ozfcnGzYU}V|)TyOA5Yl)&-o!99;AWR+GZwf_2e|PaM2Y>HLZetG9^(b!0>2
zGJ%RkMU}xn=29RDyv_GM<Xa^ixyx4^v+mTdmnvVz4<6btp0<Yd<3PO~r3MueL>;VL
klvPUH^&UO>pYFuO%?NR>+s=9Er7Y{$^IuBEH_8ueWztGniU0rr
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/92/ef85be57d627f280d8ce3724452ac21c9a6452 b/support/testing/tests/download/git-remote/repo.git/objects/92/ef85be57d627f280d8ce3724452ac21c9a6452
new file mode 100644
index 0000000000000000000000000000000000000000..1f2474ba3209802c0f297f390eb517b4419621f9
GIT binary patch
literal 20
bcmb<m^geacKgb|e!_&w72@`|F8|EAUOXLRI
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/cf/0f4f85d7a1237e377a2d25b996518a877ea001 b/support/testing/tests/download/git-remote/repo.git/objects/cf/0f4f85d7a1237e377a2d25b996518a877ea001
new file mode 100644
index 0000000000000000000000000000000000000000..fd224552518d57b0b8b0a1d618eef507f798555f
GIT binary patch
literal 49
zcmV-10M7q-0V^p=O;s>9VlXr?Ff%bxNXyJgWtjB7bzk^3^-m2q&Y7#YY8{f9l at bI1
H7n%=ExH=M#
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/e2/2695cbf976fed1f543ad7486a531c0af473482 b/support/testing/tests/download/git-remote/repo.git/objects/e2/2695cbf976fed1f543ad7486a531c0af473482
new file mode 100644
index 0000000000000000000000000000000000000000..15d80c78fac8655d4dfe4b8b543076c8bd26408e
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5^~bY$ckS%Irr!8kUB1||YqD{?-X|+&2Jxc;
FIRMOk6Sn{W
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/f6/476b879f65e956d7dedd5b08736369e9a24acc b/support/testing/tests/download/git-remote/repo.git/objects/f6/476b879f65e956d7dedd5b08736369e9a24acc
new file mode 100644
index 0000000000000000000000000000000000000000..ee01041e0abcefa3c783ef3fc91e31c33edfb7cf
GIT binary patch
literal 20
bcmb<m^geacKgb|e!_&w75fg*NJLX&fOWg+C
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/tags/mytag b/support/testing/tests/download/git-remote/repo.git/refs/tags/mytag
new file mode 100644
index 0000000000..8c09448ad2
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/tags/mytag
@@ -0,0 +1 @@
+2b0e0d98a49c97da6a618ab36337e2058eb733a2
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index 07c5762710..a5fff4dd90 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -68,7 +68,14 @@ class TestGitRefs(GitTestBase):
self.check_download("git-branch")
self.check_download("git-partial-sha1-branch-head")
self.check_download("git-partial-sha1-reachable-by-branch")
+ self.check_download("git-partial-sha1-reachable-by-tag")
+ self.check_download("git-partial-sha1-tag-itself")
+ self.check_download("git-partial-sha1-tag-points-to")
self.check_download("git-sha1-branch-head")
self.check_download("git-sha1-reachable-by-branch")
+ self.check_download("git-sha1-reachable-by-tag")
+ self.check_download("git-sha1-tag-itself")
+ self.check_download("git-sha1-tag-points-to")
self.check_download("git-submodule-disabled")
self.check_download("git-submodule-enabled")
+ self.check_download("git-tag")
--
2.14.1
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v4] testing/tests/download: test git tag/branch precedence
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
` (7 preceding siblings ...)
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git tag Ricardo Martincoski
@ 2018-04-29 14:33 ` Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git special ref Ricardo Martincoski
` (5 subsequent siblings)
14 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-29 14:33 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Add a tag and a branch with the same name to the static repo and check
on the git refs test case:
- given there is a tag and a branch with the same name, download the
branch (this is the default behavior from git command line and thus
our git download infra must do the same);
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repo with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* 2fa37f6 (submodule) sub0-2
* a9dbc1e sub0-1
| * a8f7364 (tag: mybranchandtag) tag
|/
| * 83e0f40 (mybranchandtag) branch
|/
| * 68c197d (mybranch) branch2
| * 3174063 branch1
|/
| * 516c9c5 (tag: mytag) tag2
| * 46bae5b tag1
|/
* a238b1d (HEAD -> master) initial
Changes v1 -> v4:
- not included on v2 or v3;
- rewrite from v1 using new git test infra with static repo;
---
.../git-branch-takes-precedence-over-tag.hash | 3 +++
.../git-branch-takes-precedence-over-tag.mk | 11 +++++++++++
.../objects/42/c7c82cc86a07a73cb4499345eff2eccfe668bc | Bin 0 -> 19 bytes
.../objects/47/3b7c2f803ad64eac57695ff57a5a6a8b34cbdd | Bin 0 -> 49 bytes
.../objects/80/858c1ab821392fd59a897dbaedd7d07e1ac403 | Bin 0 -> 22 bytes
.../objects/83/e0f402891d9de91e7007f8534884ec4b9c7c2c | Bin 0 -> 183 bytes
.../objects/8e/a6edb6df557ee935de556113bab2b5f9ecbabf | Bin 0 -> 49 bytes
.../objects/a8/f73649910ce50c48626de0329a0174263ebb5b | Bin 0 -> 183 bytes
.../objects/e0/de76f5d12508fa80efca376f0e2a822fedcfdd | Bin 0 -> 144 bytes
.../git-remote/repo.git/refs/heads/mybranchandtag | 1 +
.../download/git-remote/repo.git/refs/tags/mybranchandtag | 1 +
support/testing/tests/download/test_git.py | 1 +
12 files changed, 17 insertions(+)
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch-takes-precedence-over-tag/git-branch-takes-precedence-over-tag.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch-takes-precedence-over-tag/git-branch-takes-precedence-over-tag.mk
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/42/c7c82cc86a07a73cb4499345eff2eccfe668bc
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/47/3b7c2f803ad64eac57695ff57a5a6a8b34cbdd
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/80/858c1ab821392fd59a897dbaedd7d07e1ac403
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/83/e0f402891d9de91e7007f8534884ec4b9c7c2c
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/8e/a6edb6df557ee935de556113bab2b5f9ecbabf
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/a8/f73649910ce50c48626de0329a0174263ebb5b
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e0/de76f5d12508fa80efca376f0e2a822fedcfdd
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/mybranchandtag
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/tags/mybranchandtag
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-branch-takes-precedence-over-tag/git-branch-takes-precedence-over-tag.hash b/support/testing/tests/download/br2-external/git-refs/package/git-branch-takes-precedence-over-tag/git-branch-takes-precedence-over-tag.hash
new file mode 100644
index 0000000000..5eb1451efe
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-branch-takes-precedence-over-tag/git-branch-takes-precedence-over-tag.hash
@@ -0,0 +1,3 @@
+sha256 fff39726219f51cc4d3f7c090e733fed2514fcf1a759068df0d7503f9827e781 git-branch-takes-precedence-over-tag-mybranchandtag.tar.gz
+# check file contains "branch" as in commit 83e0f402891d9de91e7007f8534884ec4b9c7c2c
+sha256 2bf74fd761df1710dae82a5459f89af15375f36e700c31156c991e22b9fe7edb file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-branch-takes-precedence-over-tag/git-branch-takes-precedence-over-tag.mk b/support/testing/tests/download/br2-external/git-refs/package/git-branch-takes-precedence-over-tag/git-branch-takes-precedence-over-tag.mk
new file mode 100644
index 0000000000..ce034644aa
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-branch-takes-precedence-over-tag/git-branch-takes-precedence-over-tag.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-branch-takes-precedence-over-tag
+#
+################################################################################
+
+GIT_BRANCH_TAKES_PRECEDENCE_OVER_TAG_VERSION = mybranchandtag
+GIT_BRANCH_TAKES_PRECEDENCE_OVER_TAG_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_BRANCH_TAKES_PRECEDENCE_OVER_TAG_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/42/c7c82cc86a07a73cb4499345eff2eccfe668bc b/support/testing/tests/download/git-remote/repo.git/objects/42/c7c82cc86a07a73cb4499345eff2eccfe668bc
new file mode 100644
index 0000000000000000000000000000000000000000..314ffcd789086900b009fe7d4e9195d7524e8eb8
GIT binary patch
literal 19
acmb<m^geacKgb|a!_((E6N6|lvlReC!3EU-
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/47/3b7c2f803ad64eac57695ff57a5a6a8b34cbdd b/support/testing/tests/download/git-remote/repo.git/objects/47/3b7c2f803ad64eac57695ff57a5a6a8b34cbdd
new file mode 100644
index 0000000000000000000000000000000000000000..b8690704ec849ea1054dc7ce7633fadcc26cce78
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5H6!Eg#9Iq9R@Rq>rN3Rb{kr}YPuWB^2G+&=
F#sIxn67~Q9
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/80/858c1ab821392fd59a897dbaedd7d07e1ac403 b/support/testing/tests/download/git-remote/repo.git/objects/80/858c1ab821392fd59a897dbaedd7d07e1ac403
new file mode 100644
index 0000000000000000000000000000000000000000..c9412113d9b2278589980786be5f726480ad6625
GIT binary patch
literal 22
dcmb<m^geacKgb~6L(A)o_qhj53`&(O`2b?K2q6Ff
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/83/e0f402891d9de91e7007f8534884ec4b9c7c2c b/support/testing/tests/download/git-remote/repo.git/objects/83/e0f402891d9de91e7007f8534884ec4b9c7c2c
new file mode 100644
index 0000000000000000000000000000000000000000..08ea238ac69e06859e7bfb230587f15047af0024
GIT binary patch
literal 183
zcmV;o07(CM0j<t4ZUZ3 at K+)8kf(xt#470FN6y*YG;u8!rU?YbmBBaReW66bkEA1=3
zKKIt@)Zp-Gj-0g*kOI3B4Unup6JuC6ECmB0trQb@P9o}yM&73;2QgVGEYYF(AQ3zj
zU~~?=4II6-vKdkIGtYhKkIFQp>vtNn?%cmGRlkfI9 at -t>PL1m4fqQ$^7ERbN;%fC`
lz!-Ja>-5b3lv8QY&S#~L^|WVMdMQi#eSYLs{Q<+8ZB-QXS4{u_
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/8e/a6edb6df557ee935de556113bab2b5f9ecbabf b/support/testing/tests/download/git-remote/repo.git/objects/8e/a6edb6df557ee935de556113bab2b5f9ecbabf
new file mode 100644
index 0000000000000000000000000000000000000000..78cbc40c0e3df2b143e9832ecf422f3c4098602b
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5wZN**>A8#8 at 1ohd^CrjMudgcp7kNyMf!A1I
FCjg-q5`_Q&
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/a8/f73649910ce50c48626de0329a0174263ebb5b b/support/testing/tests/download/git-remote/repo.git/objects/a8/f73649910ce50c48626de0329a0174263ebb5b
new file mode 100644
index 0000000000000000000000000000000000000000..511cb7a629c3bb89f5e11311dcade00d18c48d85
GIT binary patch
literal 183
zcmV;o07(CM0j<tKZo?oDh2gF_1s8AxjK{`R73Biiv?myt!9@usB4m-<$5I#FSKU2*
zi>E)<QgYYe+*@xV+6zs>Vd8{1i!6XFh?BJtD7b)Wk+`p<Zq#J!njH9GIYsskd7Rn9
zNC_f32cw;%AFbuZu%bu*u1$aDNR6vLQ|oz)^?u3vqy6Ba{NiaXl&^<a%TZf2^MZZ?
ls|N$ds9pEyF8_3=I;~25<#iJtYDz<RzW*guy#P{>Y}~=KTUh`A
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/e0/de76f5d12508fa80efca376f0e2a822fedcfdd b/support/testing/tests/download/git-remote/repo.git/objects/e0/de76f5d12508fa80efca376f0e2a822fedcfdd
new file mode 100644
index 0000000000000000000000000000000000000000..10ae03c5c822cbd526709f853b2ad44f766234d0
GIT binary patch
literal 144
zcmV;B0B`?z0bR^N4#FT1KvCD6VlH4aK!K7N;|1J!gJFslEyW<4-rgD)CNBS5{6-T<
zCwC>>BR8Po>a^{>m&nl|_k&e-jHs0-l&)7+#}G^qZGA at Il5=XJZI8T$N(<j;8Mpk<
y6f0a3QymMOskXFmc|8&w>KD8G!_$;0&E57pf;8y78LYFgM~x`N-}?>I^*O{+OGOy~
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/heads/mybranchandtag b/support/testing/tests/download/git-remote/repo.git/refs/heads/mybranchandtag
new file mode 100644
index 0000000000..030351c2f7
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/heads/mybranchandtag
@@ -0,0 +1 @@
+83e0f402891d9de91e7007f8534884ec4b9c7c2c
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/tags/mybranchandtag b/support/testing/tests/download/git-remote/repo.git/refs/tags/mybranchandtag
new file mode 100644
index 0000000000..1f3290059a
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/tags/mybranchandtag
@@ -0,0 +1 @@
+e0de76f5d12508fa80efca376f0e2a822fedcfdd
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index a5fff4dd90..878cae87d6 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -66,6 +66,7 @@ class TestGitRefs(GitTestBase):
with self.assertRaises(SystemError):
self.check_download("git-wrong-sha1")
self.check_download("git-branch")
+ self.check_download("git-branch-takes-precedence-over-tag")
self.check_download("git-partial-sha1-branch-head")
self.check_download("git-partial-sha1-reachable-by-branch")
self.check_download("git-partial-sha1-reachable-by-tag")
--
2.14.1
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v4] testing/tests/download: test git special ref
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
` (8 preceding siblings ...)
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git tag/branch precedence Ricardo Martincoski
@ 2018-04-29 14:33 ` Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git branch with slash Ricardo Martincoski
` (4 subsequent siblings)
14 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-29 14:33 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Add a special ref to the static repo and check on the git refs test case
the download of a git package:
- with the name of a special ref as version;
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repo with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* 2fa37f6 (submodule) sub0-2
* a9dbc1e sub0-1
| * b72ff60 (refs/changes/01/1/2) specialref
|/
| * a8f7364 (tag: mybranchandtag) tag
|/
| * 83e0f40 (mybranchandtag) branch
|/
| * 68c197d (mybranch) branch2
| * 3174063 branch1
|/
| * 516c9c5 (tag: mytag) tag2
| * 46bae5b tag1
|/
* a238b1d (HEAD -> master) initial
Changes v1 -> v4:
- not included on v2 or v3;
- rewrite from v1 using new git test infra with static repo;
---
.../git-refs/package/git-special-ref/git-special-ref.hash | 2 ++
.../git-refs/package/git-special-ref/git-special-ref.mk | 11 +++++++++++
.../objects/17/c409592968d17600ff9912e96fdd461bb72e74 | Bin 0 -> 48 bytes
.../objects/53/122a7b0454b33c6d9e159c10657173db77899e | Bin 0 -> 27 bytes
.../objects/b7/2ff6078f62522a87f5cae5e9f34dedf5ec3885 | Bin 0 -> 186 bytes
.../download/git-remote/repo.git/refs/changes/01/1/2 | 1 +
support/testing/tests/download/test_git.py | 1 +
7 files changed, 15 insertions(+)
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.mk
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/17/c409592968d17600ff9912e96fdd461bb72e74
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/53/122a7b0454b33c6d9e159c10657173db77899e
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/b7/2ff6078f62522a87f5cae5e9f34dedf5ec3885
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/changes/01/1/2
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.hash b/support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.hash
new file mode 100644
index 0000000000..13e21aaeca
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.hash
@@ -0,0 +1,2 @@
+sha256 c639b15260d098d94b8802cae812453c9f878ffd5f80cca6d87c58c01eed5c5a git-special-ref-refs_changes_01_1_2.tar.gz
+sha256 b252b594be4679764e2e06cfea77a3e50638dd445af9b472b0f298c43b134518 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.mk b/support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.mk
new file mode 100644
index 0000000000..e7561bafca
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-special-ref
+#
+################################################################################
+
+GIT_SPECIAL_REF_VERSION = refs/changes/01/1/2
+GIT_SPECIAL_REF_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SPECIAL_REF_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/17/c409592968d17600ff9912e96fdd461bb72e74 b/support/testing/tests/download/git-remote/repo.git/objects/17/c409592968d17600ff9912e96fdd461bb72e74
new file mode 100644
index 0000000000000000000000000000000000000000..8c19de6455168ac65c81d70219e01eb2431c069e
GIT binary patch
literal 48
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5l|$^(sx;1<Ig_pZOZptL5<kBEbgq)&X$db2
E0EbQzfdBvi
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/53/122a7b0454b33c6d9e159c10657173db77899e b/support/testing/tests/download/git-remote/repo.git/objects/53/122a7b0454b33c6d9e159c10657173db77899e
new file mode 100644
index 0000000000000000000000000000000000000000..d72d49eed56298f84aaf85967dbf051da94f3871
GIT binary patch
literal 27
icmb<m^geacKghr&MMKZS_ngleO)uZaObmAQY+e9~zX?PD
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/b7/2ff6078f62522a87f5cae5e9f34dedf5ec3885 b/support/testing/tests/download/git-remote/repo.git/objects/b7/2ff6078f62522a87f5cae5e9f34dedf5ec3885
new file mode 100644
index 0000000000000000000000000000000000000000..6727b2e8083f2a31c48e849caa4cd772a6778e07
GIT binary patch
literal 186
zcmV;r07d_J0j<tKP6QzkKvCD6f(!Jd3TU8=F<!upo`9x`k<8Fc7+iRJJ8|LOxP6PC
zpL=U{YHR3W4iSwZJD`;e`N$YNKq*9MgnVIk9=9zR2~OS8kbUZ8Y}}Ab=4e)SfunV)
z*t{BJ-4TlpkqKC(&-u}ZeyceR?)o*2S@+!Um#Uw}51!gDzMNaC*HiB8S>p<UJc0{)
o0SllGJ*KDpQ%>D)l54sSDN4Q9^DeyfQkL!O at h`vX8x-PgzB#pAi~s-t
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/changes/01/1/2 b/support/testing/tests/download/git-remote/repo.git/refs/changes/01/1/2
new file mode 100644
index 0000000000..7066394dd0
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/changes/01/1/2
@@ -0,0 +1 @@
+b72ff6078f62522a87f5cae5e9f34dedf5ec3885
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index 878cae87d6..36ed04937e 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -77,6 +77,7 @@ class TestGitRefs(GitTestBase):
self.check_download("git-sha1-reachable-by-tag")
self.check_download("git-sha1-tag-itself")
self.check_download("git-sha1-tag-points-to")
+ self.check_download("git-special-ref")
self.check_download("git-submodule-disabled")
self.check_download("git-submodule-enabled")
self.check_download("git-tag")
--
2.14.1
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v4] testing/tests/download: test git branch with slash
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
` (9 preceding siblings ...)
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git special ref Ricardo Martincoski
@ 2018-04-29 14:33 ` Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [RFC PATCH v4] support/testing: test extra download with site method git Ricardo Martincoski
` (3 subsequent siblings)
14 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-29 14:33 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Add a branch with slash in its name (often called feature branch) to the
static repo and check on the git refs test case the download of a git
package:
- with the name of a branch with slash as version.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repo with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* ce5d9d4 (myfeature/mybranch) feature/branch
| * 2fa37f6 (submodule) sub0-2
| * a9dbc1e sub0-1
|/
| * b72ff60 (refs/changes/01/1/2) specialref
|/
| * a8f7364 (tag: mybranchandtag) tag
|/
| * 83e0f40 (mybranchandtag) branch
|/
| * 68c197d (mybranch) branch2
| * 3174063 branch1
|/
| * 516c9c5 (tag: mytag) tag2
| * 46bae5b tag1
|/
* a238b1d (HEAD -> master) initial
Changes v1 -> v4:
- not included on v2 or v3;
- rewrite from v1 using new git test infra with static repo;
---
.../git-branch-with-slash/git-branch-with-slash.hash | 2 ++
.../git-branch-with-slash/git-branch-with-slash.mk | 11 +++++++++++
.../objects/b8/5af389ad1ce39dfb897f8403b12f8d830053f2 | Bin 0 -> 49 bytes
.../objects/ce/5d9d43cc09b37eb0ebe677b18f81933b950b60 | Bin 0 -> 187 bytes
.../objects/ed/aa7c3edd6ff3866f476824babd820ae42f1040 | Bin 0 -> 31 bytes
.../git-remote/repo.git/refs/heads/myfeature/mybranch | 1 +
support/testing/tests/download/test_git.py | 1 +
7 files changed, 15 insertions(+)
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch-with-slash/git-branch-with-slash.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-branch-with-slash/git-branch-with-slash.mk
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/b8/5af389ad1ce39dfb897f8403b12f8d830053f2
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/ce/5d9d43cc09b37eb0ebe677b18f81933b950b60
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/ed/aa7c3edd6ff3866f476824babd820ae42f1040
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/myfeature/mybranch
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-branch-with-slash/git-branch-with-slash.hash b/support/testing/tests/download/br2-external/git-refs/package/git-branch-with-slash/git-branch-with-slash.hash
new file mode 100644
index 0000000000..c1357f994f
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-branch-with-slash/git-branch-with-slash.hash
@@ -0,0 +1,2 @@
+sha256 3a4012781e10f4fb29863683c1ca913e94c3472ab3ce9289a95173bb41827d3c git-branch-with-slash-myfeature_mybranch.tar.gz
+sha256 7352fd35e0917ad431ec18051e641a4ca1b6ac8b8d523384040346d70a7f0efc file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-branch-with-slash/git-branch-with-slash.mk b/support/testing/tests/download/br2-external/git-refs/package/git-branch-with-slash/git-branch-with-slash.mk
new file mode 100644
index 0000000000..d39ab246c6
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-branch-with-slash/git-branch-with-slash.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-branch-with-slash
+#
+################################################################################
+
+GIT_BRANCH_WITH_SLASH_VERSION = myfeature/mybranch
+GIT_BRANCH_WITH_SLASH_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_BRANCH_WITH_SLASH_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/b8/5af389ad1ce39dfb897f8403b12f8d830053f2 b/support/testing/tests/download/git-remote/repo.git/objects/b8/5af389ad1ce39dfb897f8403b12f8d830053f2
new file mode 100644
index 0000000000000000000000000000000000000000..c41f3bdbffe7d7d13bff65a0a1a60d4587e90d67
GIT binary patch
literal 49
zcmV-10M7q-0V^p=O;s>9VlXr?Ff%bxNXyJgWq7-)#_n$Z=eB(J43%Abo4B6n3pfA(
H6kiVG);$!C
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/ce/5d9d43cc09b37eb0ebe677b18f81933b950b60 b/support/testing/tests/download/git-remote/repo.git/objects/ce/5d9d43cc09b37eb0ebe677b18f81933b950b60
new file mode 100644
index 0000000000000000000000000000000000000000..76ec78178dbf70b5d6b94a7083763e14cd4cdb08
GIT binary patch
literal 187
zcmV;s07U<I0j-WfP6Qzk06p^xJ}_%TgVSs_#t(Q|U(nF7OO(MNd+_%=i3jh-Yo$`D
z)LW}lBjUpxIqOKC3sPV)DPxclDXfJYU^KRn5CV8CR(<lw+th54qLGq>tQWWLg4+Yh
zTZ_)DiJUPKKosBRwGVx+iHCIk%460o^{@A;PvZw3+An;$G_D_q)Z4Q*9tfTStVS<j
p0n|?S>6!m2r%L8|8~HH~-qKa6w|d#KEWMN^ejoqxtnRJqZ_li-TuT4|
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/ed/aa7c3edd6ff3866f476824babd820ae42f1040 b/support/testing/tests/download/git-remote/repo.git/objects/ed/aa7c3edd6ff3866f476824babd820ae42f1040
new file mode 100644
index 0000000000000000000000000000000000000000..1f0a114eb866aff4797ae3f69388b581f2346312
GIT binary patch
literal 31
ncmb<m^geacKghr&)x+CYM^o49s=t=k8Sirsm>8nxvF`)`vi1vx
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/heads/myfeature/mybranch b/support/testing/tests/download/git-remote/repo.git/refs/heads/myfeature/mybranch
new file mode 100644
index 0000000000..2c6892cc06
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/heads/myfeature/mybranch
@@ -0,0 +1 @@
+ce5d9d43cc09b37eb0ebe677b18f81933b950b60
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index 36ed04937e..9fca2bb693 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -67,6 +67,7 @@ class TestGitRefs(GitTestBase):
self.check_download("git-wrong-sha1")
self.check_download("git-branch")
self.check_download("git-branch-takes-precedence-over-tag")
+ self.check_download("git-branch-with-slash")
self.check_download("git-partial-sha1-branch-head")
self.check_download("git-partial-sha1-reachable-by-branch")
self.check_download("git-partial-sha1-reachable-by-tag")
--
2.14.1
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [RFC PATCH v4] support/testing: test extra download with site method git
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
` (10 preceding siblings ...)
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git branch with slash Ricardo Martincoski
@ 2018-04-29 14:33 ` Ricardo Martincoski
2018-04-30 1:38 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
` (2 subsequent siblings)
14 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-29 14:33 UTC (permalink / raw)
To: buildroot
Add one test case to ensure the use of extra download works for git
packages:
- an extra download is correctly downloaded using wget after the main
download finished using the git backend;
- when the main download using the git backend fails, the build fails;
- when the extra download using wget fails, the build fails.
Reuse the commit in the static repo.
Add required infra and fixture:
- a HttpServer class, that starts a http server in the host machine to
emulate a remote http server under the control of the test;
- a file to be served by the http server;
- a br2-external (git-extra-download) with one package for each part of
the test case.
Since the HttpServer is similar to GitRemote, extract the commonalities
to a new base class called Server.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
---
Changes v3 -> v4:
- new patch
In the master branch (7a801da8e):
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/65546799
master + these 2 patches:
http://patchwork.ozlabs.org/patch/904722/
http://patchwork.ozlabs.org/patch/904721/
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/65550316
I use docker only with the Buildroot image for the test infra.
Sorry, I give up for now on trying to run apache on the docker image.
As a midground solution between:
- adding more files to http://autobuild.buildroot.net/artefacts/ that
won't scale well if we need different files for different test cases;
- installing a full web server to the docker image auto-loading as root
when the container is started as user and allowing the test infra to
copy files to the served path as user;
and inspired by this article:
https://www.pcsuggest.com/best-lightweight-web-server-linux/
I created this RFC patch using SimpleHTTPServer being started by the
test infra as a user.
I know it is weird to use pexpect to fork a server written in Python,
but it has these advantages:
- theoretically replacing SimpleHTTPServer with a similar solution (say
the busybox httpd) in the future would require only small changes to
the test infra code;
- SimpleHTTPServer is already on the docker image, busybox isn't, so no
need to change the Dockerfile, generate the image, publish it and
update the docker image tag on .gitlab-ci.yml;
- the log file generated by the server is handled in the same way as
other log files generated by the test infra;
---
.gitlab-ci.yml | 1 +
support/testing/infra/server.py | 39 ++++++++++++++++++++
| 0
| 1 +
| 6 +++
| 3 ++
| 12 ++++++
.../package/main-fails/main-fails.hash | 3 ++
.../package/main-fails/main-fails.mk | 12 ++++++
| 3 ++
| 12 ++++++
support/testing/tests/download/gitremote.py | 43 ++++------------------
| 1 +
support/testing/tests/download/httpserver.py | 14 +++++++
support/testing/tests/download/test_git.py | 21 +++++++++++
15 files changed, 135 insertions(+), 36 deletions(-)
create mode 100644 support/testing/infra/server.py
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.hash
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.mk
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.hash
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.mk
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.hash
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.mk
create mode 100644 support/testing/tests/download/http-server/extra
create mode 100644 support/testing/tests/download/httpserver.py
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 26e2af3799..5df21b8787 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -266,6 +266,7 @@ tests.core.test_rootfs_overlay.TestRootfsOverlay: *runtime_test
tests.core.test_timezone.TestGlibcAllTimezone: *runtime_test
tests.core.test_timezone.TestGlibcNonDefaultLimitedTimezone: *runtime_test
tests.core.test_timezone.TestNoTimezone: *runtime_test
+tests.download.test_git.TestGitExtraDownload: *runtime_test
tests.download.test_git.TestGitHash: *runtime_test
tests.download.test_git.TestGitRefs: *runtime_test
tests.fs.test_ext.TestExt2: *runtime_test
diff --git a/support/testing/infra/server.py b/support/testing/infra/server.py
new file mode 100644
index 0000000000..0fbffdf846
--- /dev/null
+++ b/support/testing/infra/server.py
@@ -0,0 +1,39 @@
+# subprocess does not kill the child daemon when a test case fails by raising
+# an exception. So use pexpect instead.
+import pexpect
+
+import infra
+
+
+class Server(object):
+ def __init__(self, builddir, serveddir, logtofile, daemon_cmd, port_arg, port_initial, port_last, good_msg, bad_msg):
+ """
+ Start a local server.
+
+ In order to support test cases in parallel, select the port the
+ server will listen to in runtime. Since there is no reliable way
+ to allocate the port prior to starting the server (another
+ process in the host machine can use the port between it is
+ selected from a list and it is really allocated to the server)
+ try to start the server in a port and in the case it is already
+ in use, try the next one in the allowed range.
+ """
+ self.name = self.__class__.__name__.lower()
+ self.daemon = None
+ self.port = None
+ self.logfile = infra.open_log_file(builddir, self.name, logtofile)
+
+ for port in range(port_initial, port_last + 1):
+ cmd = daemon_cmd + [port_arg.format(port=port)]
+ self.logfile.write("> starting {} with 'cd {} && {}'\n".format(self.name, serveddir, " ".join(cmd)))
+ self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile, cwd=serveddir)
+ ret = self.daemon.expect(good_msg + bad_msg)
+ if ret < len(good_msg):
+ self.port = port
+ return
+ raise SystemError("Could not find a free port to run {}".format(self.name))
+
+ def stop(self):
+ if self.daemon is None:
+ return
+ self.daemon.terminate(force=True)
diff --git a/support/testing/tests/download/br2-external/git-extra-download/Config.in b/support/testing/tests/download/br2-external/git-extra-download/Config.in
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/support/testing/tests/download/br2-external/git-extra-download/external.desc b/support/testing/tests/download/br2-external/git-extra-download/external.desc
new file mode 100644
index 0000000000..6ebd5a534d
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/external.desc
@@ -0,0 +1 @@
+name: GIT_EXTRA_DOWNLOAD
diff --git a/support/testing/tests/download/br2-external/git-extra-download/external.mk b/support/testing/tests/download/br2-external/git-extra-download/external.mk
new file mode 100644
index 0000000000..c6080f571b
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/external.mk
@@ -0,0 +1,6 @@
+include $(sort $(wildcard $(BR2_EXTERNAL_GIT_EXTRA_DOWNLOAD_PATH)/package/*/*.mk))
+
+# Get the git server port number from the test infra
+GITREMOTE_PORT_NUMBER ?= 9418
+# Get the http server port number from the test infra
+HTTP_SERVER_PORT_NUMBER ?= 8000
diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.hash b/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.hash
new file mode 100644
index 0000000000..89b1b1f682
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.hash
@@ -0,0 +1,3 @@
+sha256 1e6bc73fabdcce8857361e36e3c812c4ee42d8ffa30d56492bc56f8fcad7eb90 extra-fails-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
+sha256 b9e68e1bea3e5b19ca6b2f98b73a54b73daafaa250484902e09982e07a12e733 notfound
+sha256 da68f54607d5f5644954096ce1597c006c5bb9f2497e07441bf064b81003ef8a file
diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.mk b/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.mk
new file mode 100644
index 0000000000..dcf3ebe7a6
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.mk
@@ -0,0 +1,12 @@
+################################################################################
+#
+# extra-fails
+#
+################################################################################
+
+EXTRA_FAILS_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+EXTRA_FAILS_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+EXTRA_FAILS_EXTRA_DOWNLOADS = http://localhost:$(HTTP_SERVER_PORT_NUMBER)/notfound
+EXTRA_FAILS_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.hash b/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.hash
new file mode 100644
index 0000000000..ad81243751
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.hash
@@ -0,0 +1,3 @@
+sha256 cd6851ef519a83345e4547f376b33d6bbd622d4ccbb234af9997c43854c602de main-fails-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
+sha256 b9e68e1bea3e5b19ca6b2f98b73a54b73daafaa250484902e09982e07a12e733 extra
+sha256 da68f54607d5f5644954096ce1597c006c5bb9f2497e07441bf064b81003ef8a file
diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.mk b/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.mk
new file mode 100644
index 0000000000..022bb37cbb
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.mk
@@ -0,0 +1,12 @@
+################################################################################
+#
+# main-fails
+#
+################################################################################
+
+MAIN_FAILS_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+MAIN_FAILS_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/notfound.git
+MAIN_FAILS_EXTRA_DOWNLOADS = http://localhost:$(HTTP_SERVER_PORT_NUMBER)/extra
+MAIN_FAILS_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.hash b/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.hash
new file mode 100644
index 0000000000..366940754b
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.hash
@@ -0,0 +1,3 @@
+sha256 737b4fd21506dbaa34cedc93c53c1ebb1f950db2c7644572bb006ae9297961da ok-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
+sha256 b9e68e1bea3e5b19ca6b2f98b73a54b73daafaa250484902e09982e07a12e733 extra
+sha256 da68f54607d5f5644954096ce1597c006c5bb9f2497e07441bf064b81003ef8a file
diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.mk b/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.mk
new file mode 100644
index 0000000000..ce102cb0de
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.mk
@@ -0,0 +1,12 @@
+################################################################################
+#
+# ok
+#
+################################################################################
+
+OK_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+OK_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+OK_EXTRA_DOWNLOADS = http://localhost:$(HTTP_SERVER_PORT_NUMBER)/extra
+OK_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/gitremote.py b/support/testing/tests/download/gitremote.py
index 60bc49fbf8..9766b19ce0 100644
--- a/support/testing/tests/download/gitremote.py
+++ b/support/testing/tests/download/gitremote.py
@@ -1,44 +1,15 @@
-# subprocess does not kill the child daemon when a test case fails by raising
-# an exception. So use pexpect instead.
-import pexpect
-
-import infra
+from infra.server import Server
GIT_REMOTE_PORT_INITIAL = 9418
GIT_REMOTE_PORT_LAST = GIT_REMOTE_PORT_INITIAL + 99
-class GitRemote(object):
+class GitRemote(Server):
def __init__(self, builddir, serveddir, logtofile):
- """
- Start a local git server.
-
- In order to support test cases in parallel, select the port the
- server will listen to in runtime. Since there is no reliable way
- to allocate the port prior to starting the server (another
- process in the host machine can use the port between it is
- selected from a list and it is really allocated to the server)
- try to start the server in a port and in the case it is already
- in use, try the next one in the allowed range.
- """
- self.daemon = None
- self.port = None
- self.logfile = infra.open_log_file(builddir, "gitremote", logtofile)
-
daemon_cmd = ["git", "daemon", "--reuseaddr", "--verbose", "--listen=localhost", "--export-all",
"--base-path={}".format(serveddir)]
- for port in range(GIT_REMOTE_PORT_INITIAL, GIT_REMOTE_PORT_LAST + 1):
- cmd = daemon_cmd + ["--port={port}".format(port=port)]
- self.logfile.write("> starting git remote with '{}'\n".format(" ".join(cmd)))
- self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile)
- ret = self.daemon.expect(["Ready to rumble",
- "Address already in use"])
- if ret == 0:
- self.port = port
- return
- raise SystemError("Could not find a free port to run git remote")
-
- def stop(self):
- if self.daemon is None:
- return
- self.daemon.terminate(force=True)
+ super(GitRemote, self).__init__(
+ builddir, serveddir, logtofile,
+ daemon_cmd, "--port={port}", GIT_REMOTE_PORT_INITIAL, GIT_REMOTE_PORT_LAST,
+ ["Ready to rumble"],
+ ["Address already in use"])
--git a/support/testing/tests/download/http-server/extra b/support/testing/tests/download/http-server/extra
new file mode 100644
index 0000000000..8e27be7d61
--- /dev/null
+++ b/support/testing/tests/download/http-server/extra
@@ -0,0 +1 @@
+text
diff --git a/support/testing/tests/download/httpserver.py b/support/testing/tests/download/httpserver.py
new file mode 100644
index 0000000000..9f4f947304
--- /dev/null
+++ b/support/testing/tests/download/httpserver.py
@@ -0,0 +1,14 @@
+from infra.server import Server
+
+HTTP_SERVER_PORT_INITIAL = 8000
+HTTP_SERVER_PORT_LAST = HTTP_SERVER_PORT_INITIAL + 99
+
+
+class HttpServer(Server):
+ def __init__(self, builddir, serveddir, logtofile):
+ daemon_cmd = ["python", "-m", "SimpleHTTPServer"]
+ super(HttpServer, self).__init__(
+ builddir, serveddir, logtofile,
+ daemon_cmd, "{port}", HTTP_SERVER_PORT_INITIAL, HTTP_SERVER_PORT_LAST,
+ ["Serving HTTP"],
+ ["Address already in use"])
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index 9fca2bb693..a06ce8bc50 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -3,6 +3,7 @@ import shutil
import infra
from gitremote import GitRemote
+from httpserver import HttpServer
class GitTestBase(infra.basetest.BRTest):
@@ -12,13 +13,19 @@ class GitTestBase(infra.basetest.BRTest):
"""
gitremotedir = infra.filepath("tests/download/git-remote")
gitremote = None
+ httpserverdir = None
+ httpserver = None
def setUp(self):
super(GitTestBase, self).setUp()
self.gitremote = GitRemote(self.builddir, self.gitremotedir, self.logtofile)
+ if self.httpserverdir:
+ self.httpserver = HttpServer(self.builddir, self.httpserverdir, self.logtofile)
def tearDown(self):
self.show_msg("Cleaning up")
+ if self.httpserver:
+ self.httpserver.stop()
if self.gitremote:
self.gitremote.stop()
if self.b and not self.keepbuilds:
@@ -42,11 +49,25 @@ class GitTestBase(infra.basetest.BRTest):
shutil.rmtree(dl_dir)
env = {"BR2_DL_DIR": dl_dir,
"GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
+ if self.httpserver:
+ env["HTTP_SERVER_PORT_NUMBER"] = str(self.httpserver.port)
self.b.build(["{}-dirclean".format(package),
"{}-legal-info".format(package)],
env)
+class TestGitExtraDownload(GitTestBase):
+ br2_external = [infra.filepath("tests/download/br2-external/git-extra-download")]
+ httpserverdir = infra.filepath("tests/download/http-server")
+
+ def test_run(self):
+ with self.assertRaises(SystemError):
+ self.check_download("extra-fails")
+ with self.assertRaises(SystemError):
+ self.check_download("main-fails")
+ self.check_download("ok")
+
+
class TestGitHash(GitTestBase):
br2_external = [infra.filepath("tests/download/br2-external/git-hash")]
--
2.14.1
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v4] tests for git download infra v4
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
` (11 preceding siblings ...)
2018-04-29 14:33 ` [Buildroot] [RFC PATCH v4] support/testing: test extra download with site method git Ricardo Martincoski
@ 2018-04-30 1:38 ` Ricardo Martincoski
2018-05-11 3:09 ` Ricardo Martincoski
2018-05-12 2:58 ` [Buildroot] [PATCH v5 00/10] tests for git download infra v5 Ricardo Martincoski
14 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-04-30 1:38 UTC (permalink / raw)
To: buildroot
Hello,
Each patch showed up as a separate series on patchwork.
I used -N with format-patch by mistake and just noticed now.
Sorry for the mess.
I can resend if needed, just let me know.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v4] tests for git download infra v4
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
` (12 preceding siblings ...)
2018-04-30 1:38 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
@ 2018-05-11 3:09 ` Ricardo Martincoski
2018-05-12 2:58 ` [Buildroot] [PATCH v5 00/10] tests for git download infra v5 Ricardo Martincoski
14 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-05-11 3:09 UTC (permalink / raw)
To: buildroot
Hello,
I am marking this series as Changes Requested because all tests for named
branches must be removed. For details see:
http://patchwork.ozlabs.org/patch/911451/
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 00/10] tests for git download infra v5
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
` (13 preceding siblings ...)
2018-05-11 3:09 ` Ricardo Martincoski
@ 2018-05-12 2:58 ` Ricardo Martincoski
2018-05-12 2:58 ` [Buildroot] [PATCH v5 01/10] testing/infra/builder: build with target and environment Ricardo Martincoski
` (9 more replies)
14 siblings, 10 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-05-12 2:58 UTC (permalink / raw)
To: buildroot
Hello,
This series adds automated tests for the git download infra.
Changes v4 -> v5:
- do not test a named branch, Buildroot does not support it (1 patch changed
and 2 removed);
- double-check I correctly used git format-patch -n before sending;
Changes v3 -> v4:
- the first 2 patches from v3 got applied;
- refresh the series in current master (with new download infra);
- add few patches I was holding back to send later;
- allow the use of multiple br2-external trees, simplifying patch 3;
- many code improvements to patch 4;
The detailed changelog is part of each patch.
This series does not yet test the cache function, all test cases run with an
empty git cache, but it is a step towards it.
The first 3 patches prepare the test infra to have the git test cases.
Patch 4 adds a test case that ensures hash checking is enabled for git packages.
Remaining patches, except the last one, add a test case that checks each type of
git ref (tag, sha1, ...) that can be used as version for a package with download
method git.
I created a single test case because:
- for all ref types it only takes 3 minutes to run;
- the feature under test is OK if all kinds of supported refs work.
But I split it in many patches because a static git repo is used, which is not
trivial to review. So each patch adds the needed git objects and the associated
.mk and .hash files. This way any rework/drop/rebase/revert is easier.
In order to make review easier I also added to the comment area of each patch
(after ---) the layout of the static repo with all patches until that one
applied (generated by git log --graph).
Last patch tests that when an extra download is used with a package with site
method git, the main download is done using git but the extra download is done
using wget.
Here a complete run on Gitlab CI, with the series applied on master (8f2d402b):
https://gitlab.com/RicardoMartincoski/buildroot/pipelines/21903594
Regards,
Ricardo
Ricardo Martincoski (10):
testing/infra/builder: build with target and environment
testing/infra: split runtime test from BRTest
testing/infra/basetest: support br2-external
testing/tests/download: add git hash test
testing/tests/download: test case for git refs
testing/tests/download: test git branch
testing/tests/download: test git submodules
testing/tests/download: test git tag
testing/tests/download: test git special ref
support/testing: test extra download with site method git
.gitlab-ci.yml | 3 +
support/testing/infra/basetest.py | 13 +--
support/testing/infra/builder.py | 38 ++++++-
support/testing/infra/runtimetest.py | 23 ++++
support/testing/infra/server.py | 39 +++++++
support/testing/tests/boot/test_atf.py | 7 +-
.../testing/tests/core/test_post_scripts.py | 3 +-
.../testing/tests/core/test_rootfs_overlay.py | 3 +-
support/testing/tests/core/test_timezone.py | 7 +-
support/testing/tests/download/__init__.py | 0
.../br2-external/git-extra-download/Config.in | 0
.../git-extra-download/external.desc | 1 +
.../git-extra-download/external.mk | 6 ++
.../package/extra-fails/extra-fails.hash | 3 +
.../package/extra-fails/extra-fails.mk | 12 +++
.../package/main-fails/main-fails.hash | 3 +
.../package/main-fails/main-fails.mk | 12 +++
.../git-extra-download/package/ok/ok.hash | 3 +
.../git-extra-download/package/ok/ok.mk | 12 +++
.../download/br2-external/git-hash/Config.in | 0
.../br2-external/git-hash/external.desc | 1 +
.../br2-external/git-hash/external.mk | 4 +
.../git-hash/package/bad/bad.hash | 1 +
.../br2-external/git-hash/package/bad/bad.mk | 10 ++
.../git-hash/package/good/good.hash | 1 +
.../git-hash/package/good/good.mk | 10 ++
.../git-hash/package/nohash/nohash.mk | 10 ++
.../download/br2-external/git-refs/Config.in | 0
.../br2-external/git-refs/external.desc | 1 +
.../br2-external/git-refs/external.mk | 4 +
.../git-partial-sha1-branch-head.hash | 2 +
.../git-partial-sha1-branch-head.mk | 11 ++
.../git-partial-sha1-reachable-by-branch.hash | 2 +
.../git-partial-sha1-reachable-by-branch.mk | 11 ++
.../git-partial-sha1-reachable-by-tag.hash | 2 +
.../git-partial-sha1-reachable-by-tag.mk | 11 ++
.../git-partial-sha1-tag-itself.hash | 2 +
.../git-partial-sha1-tag-itself.mk | 11 ++
.../git-partial-sha1-tag-points-to.hash | 2 +
.../git-partial-sha1-tag-points-to.mk | 11 ++
.../git-sha1-branch-head.hash | 2 +
.../git-sha1-branch-head.mk | 11 ++
.../git-sha1-reachable-by-branch.hash | 2 +
.../git-sha1-reachable-by-branch.mk | 11 ++
.../git-sha1-reachable-by-tag.hash | 2 +
.../git-sha1-reachable-by-tag.mk | 11 ++
.../git-sha1-tag-itself.hash | 2 +
.../git-sha1-tag-itself.mk | 11 ++
.../git-sha1-tag-points-to.hash | 2 +
.../git-sha1-tag-points-to.mk | 11 ++
.../git-special-ref/git-special-ref.hash | 2 +
.../git-special-ref/git-special-ref.mk | 11 ++
.../git-submodule-disabled.hash | 2 +
.../git-submodule-disabled.mk | 11 ++
.../git-submodule-enabled.hash | 4 +
.../git-submodule-enabled.mk | 28 +++++
.../git-refs/package/git-tag/git-tag.hash | 2 +
.../git-refs/package/git-tag/git-tag.mk | 11 ++
.../git-wrong-content/git-wrong-content.hash | 2 +
.../git-wrong-content/git-wrong-content.mk | 11 ++
.../package/git-wrong-sha1/git-wrong-sha1.mk | 11 ++
.../git-remote/refs-sub1.git/.gitattributes | 1 +
.../download/git-remote/refs-sub1.git/HEAD | 1 +
.../download/git-remote/refs-sub1.git/config | 4 +
.../07/cabc655213bdf7087d8dd50fda95124e935570 | Bin 0 -> 22 bytes
.../1d/f823cb8a6d1866148ae50a8009762a9c4c777f | Bin 0 -> 180 bytes
.../7d/52f458bdb0f9f5a4beb46fa82824421b8b988e | Bin 0 -> 65 bytes
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a | Bin 0 -> 122 bytes
.../bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23 | Bin 0 -> 22 bytes
.../cb/545facf77bbc5f24f95b6d503c338d10b7b717 | Bin 0 -> 151 bytes
.../dd/130f6f4629514adaf2e03407f3ed9344eb6cd6 | Bin 0 -> 122 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../f8/001e5780100962a5e30a25cbc4c9609cfc7bf3 | Bin 0 -> 184 bytes
.../refs-sub1.git/refs/heads/submodule | 1 +
.../git-remote/refs-sub2.git/.gitattributes | 1 +
.../download/git-remote/refs-sub2.git/HEAD | 1 +
.../download/git-remote/refs-sub2.git/config | 4 +
.../0a/846af45c3e455789435f49f80d70e86b65b9d7 | Bin 0 -> 22 bytes
.../0b/32ebd8fc52cec991f18c94be980e85a8341585 | Bin 0 -> 22 bytes
.../32/d61bae693af7879da63b89a60d3ae67f851e56 | Bin 0 -> 151 bytes
.../3e/9b0a5198c64cea9c00f820433411e3b4d50c1c | Bin 0 -> 48 bytes
.../85/8f666af75b7c0dfba6b8be7eac5f196e7a1221 | Bin 0 -> 49 bytes
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb | Bin 0 -> 184 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../e8/3f6f805bd016b90acafc8702c52d778eb57310 | Bin 0 -> 182 bytes
.../refs-sub2.git/refs/heads/submodule | 1 +
.../git-remote/repo.git/.gitattributes | 1 +
.../tests/download/git-remote/repo.git/HEAD | 1 +
.../tests/download/git-remote/repo.git/config | 4 +
.../0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343 | Bin 0 -> 22 bytes
.../11/93ff46343f4f6a0522e2b28b871e905178c1f0 | Bin 0 -> 23 bytes
.../17/c409592968d17600ff9912e96fdd461bb72e74 | Bin 0 -> 48 bytes
.../25/59d83bfe937fc0412d96ed664663c9e8a99055 | Bin 0 -> 22 bytes
.../2b/0e0d98a49c97da6a618ab36337e2058eb733a2 | Bin 0 -> 137 bytes
.../2f/a37f6885d7eb746df75eccaddbacf3ac82799d | Bin 0 -> 180 bytes
.../31/7406308d9259e2231bd0d6ddad3de3832bce08 | Bin 0 -> 182 bytes
.../34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5 | Bin 0 -> 23 bytes
.../46/bae5b639e5a18e2cc4dc508f080d566baeff59 | Bin 0 -> 182 bytes
.../51/6c9c5f64ec66534d4d069c2e408d9ae4dce023 | Bin 0 -> 182 bytes
.../53/122a7b0454b33c6d9e159c10657173db77899e | Bin 0 -> 27 bytes
.../68/c197d0879d485f4f6cee85544722b79e68e59f | Bin 0 -> 184 bytes
.../8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7 | Bin 0 -> 65 bytes
.../92/ef85be57d627f280d8ce3724452ac21c9a6452 | Bin 0 -> 20 bytes
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../a2/38b1dfcd825d47d834af3c5223417c8411d90d | Bin 0 -> 152 bytes
.../a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f | Bin 0 -> 183 bytes
.../b2/4b387624edc78d0292a127c43cad9ba97c6232 | Bin 0 -> 49 bytes
.../b7/2ff6078f62522a87f5cae5e9f34dedf5ec3885 | Bin 0 -> 186 bytes
.../cf/0f4f85d7a1237e377a2d25b996518a877ea001 | Bin 0 -> 49 bytes
.../d4/e2941d18a63535400476324ddeb7f40164be41 | Bin 0 -> 121 bytes
.../e2/2695cbf976fed1f543ad7486a531c0af473482 | Bin 0 -> 49 bytes
.../e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e | Bin 0 -> 49 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../f6/476b879f65e956d7dedd5b08736369e9a24acc | Bin 0 -> 20 bytes
.../fe/74231105841041d5f441e70399d37f0e600aa5 | Bin 0 -> 121 bytes
.../git-remote/repo.git/refs/changes/01/1/2 | 1 +
.../git-remote/repo.git/refs/heads/master | 1 +
.../git-remote/repo.git/refs/heads/mybranch | 1 +
.../git-remote/repo.git/refs/heads/submodule | 1 +
.../git-remote/repo.git/refs/tags/mytag | 1 +
support/testing/tests/download/gitremote.py | 15 +++
.../testing/tests/download/http-server/extra | 1 +
support/testing/tests/download/httpserver.py | 14 +++
support/testing/tests/download/test_git.py | 102 ++++++++++++++++++
support/testing/tests/fs/test_ext.py | 9 +-
support/testing/tests/fs/test_iso9660.py | 13 +--
support/testing/tests/fs/test_jffs2.py | 3 +-
support/testing/tests/fs/test_squashfs.py | 3 +-
support/testing/tests/fs/test_ubi.py | 3 +-
support/testing/tests/fs/test_yaffs2.py | 3 +-
support/testing/tests/init/base.py | 3 +-
.../testing/tests/package/test_dropbear.py | 3 +-
support/testing/tests/package/test_python.py | 3 +-
support/testing/tests/package/test_rust.py | 3 +-
.../testing/tests/package/test_syslog_ng.py | 3 +-
.../testing/tests/toolchain/test_external.py | 3 +-
138 files changed, 621 insertions(+), 43 deletions(-)
create mode 100644 support/testing/infra/runtimetest.py
create mode 100644 support/testing/infra/server.py
create mode 100644 support/testing/tests/download/__init__.py
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.hash
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.mk
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.hash
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.mk
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.hash
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-hash/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-hash/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.hash
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-refs/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-refs/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-sha1/git-wrong-sha1.mk
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/config
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/07/cabc655213bdf7087d8dd50fda95124e935570
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/1d/f823cb8a6d1866148ae50a8009762a9c4c777f
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/7d/52f458bdb0f9f5a4beb46fa82824421b8b988e
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/cb/545facf77bbc5f24f95b6d503c338d10b7b717
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/dd/130f6f4629514adaf2e03407f3ed9344eb6cd6
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/f8/001e5780100962a5e30a25cbc4c9609cfc7bf3
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/refs/heads/submodule
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/config
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/0a/846af45c3e455789435f49f80d70e86b65b9d7
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/0b/32ebd8fc52cec991f18c94be980e85a8341585
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/32/d61bae693af7879da63b89a60d3ae67f851e56
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/3e/9b0a5198c64cea9c00f820433411e3b4d50c1c
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/85/8f666af75b7c0dfba6b8be7eac5f196e7a1221
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/e8/3f6f805bd016b90acafc8702c52d778eb57310
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/refs/heads/submodule
create mode 100644 support/testing/tests/download/git-remote/repo.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/repo.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/repo.git/config
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/11/93ff46343f4f6a0522e2b28b871e905178c1f0
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/17/c409592968d17600ff9912e96fdd461bb72e74
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/25/59d83bfe937fc0412d96ed664663c9e8a99055
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/2b/0e0d98a49c97da6a618ab36337e2058eb733a2
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/2f/a37f6885d7eb746df75eccaddbacf3ac82799d
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/31/7406308d9259e2231bd0d6ddad3de3832bce08
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/46/bae5b639e5a18e2cc4dc508f080d566baeff59
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/51/6c9c5f64ec66534d4d069c2e408d9ae4dce023
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/53/122a7b0454b33c6d9e159c10657173db77899e
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/68/c197d0879d485f4f6cee85544722b79e68e59f
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/92/ef85be57d627f280d8ce3724452ac21c9a6452
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/a2/38b1dfcd825d47d834af3c5223417c8411d90d
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/b2/4b387624edc78d0292a127c43cad9ba97c6232
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/b7/2ff6078f62522a87f5cae5e9f34dedf5ec3885
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/cf/0f4f85d7a1237e377a2d25b996518a877ea001
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/d4/e2941d18a63535400476324ddeb7f40164be41
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e2/2695cbf976fed1f543ad7486a531c0af473482
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/f6/476b879f65e956d7dedd5b08736369e9a24acc
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/fe/74231105841041d5f441e70399d37f0e600aa5
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/changes/01/1/2
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/master
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/mybranch
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/submodule
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/tags/mytag
create mode 100644 support/testing/tests/download/gitremote.py
create mode 100644 support/testing/tests/download/http-server/extra
create mode 100644 support/testing/tests/download/httpserver.py
create mode 100644 support/testing/tests/download/test_git.py
--
2.17.0
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 01/10] testing/infra/builder: build with target and environment
2018-05-12 2:58 ` [Buildroot] [PATCH v5 00/10] tests for git download infra v5 Ricardo Martincoski
@ 2018-05-12 2:58 ` Ricardo Martincoski
2018-05-12 2:58 ` [Buildroot] [PATCH v5 02/10] testing/infra: split runtime test from BRTest Ricardo Martincoski
` (8 subsequent siblings)
9 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-05-12 2:58 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Make the builder able to call 'VAR1=1 make VAR2=2 target'.
Allow to send extra parameters to be added to the end of make command
line. It can be used for these purposes:
- to configure a br2-external, by passing 'BR2_EXTERNAL="dir"';
- to specify a make target, such as 'foo-source'.
Allow to add variables to the environment in which make runs. It can be
used to override values from environment, such as 'BR2_DL_DIR="dl"'.
This change will be needed when adding a common class to test the git
download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Changes v4 -> v5:
- no changes
Changes v3 -> v4:
- no changes
Changes v2 -> v3:
- rebase after adding patch 1 to the series, which runs make using an
empty env;
- use docstring (Arnout Vandecappelle);
- use more descriptive parameter names (make_extra_opts,
make_extra_env) (Arnout Vandecappelle);
- default make_extra_env to {} and use update() (Arnout
Vandecappelle);
- for consistence I did the equivalent to make_extra_opts;
- for symmetry, use make_extra_env also in the configure() (Arnout
Vandecappelle);
- remove old example from commit message since using a static repo to
test git refs do not need to override _VERSION.
Changes v1 -> v2:
- new patch to adapt the test infra to test git download;
---
support/testing/infra/builder.py | 38 ++++++++++++++++++++++++++++----
1 file changed, 34 insertions(+), 4 deletions(-)
diff --git a/support/testing/infra/builder.py b/support/testing/infra/builder.py
index faf1eb1494..d478c0d212 100644
--- a/support/testing/infra/builder.py
+++ b/support/testing/infra/builder.py
@@ -12,7 +12,18 @@ class Builder(object):
self.builddir = builddir
self.logfile = infra.open_log_file(builddir, "build", logtofile)
- def configure(self):
+ def configure(self, make_extra_opts=[], make_extra_env={}):
+ """
+ Configure the build.
+
+ make_extra_opts: a list of arguments to be passed to the make
+ command.
+ e.g. make_extra_opts=["BR2_EXTERNAL=/path"]
+
+ make_extra_env: a dict of variables to be appended (or replaced)
+ in the environment that calls make.
+ e.g. make_extra_env={"BR2_DL_DIR": "/path"}
+ """
if not os.path.isdir(self.builddir):
os.makedirs(self.builddir)
@@ -25,17 +36,36 @@ class Builder(object):
self.logfile.flush()
env = {"PATH": os.environ["PATH"]}
+ env.update(make_extra_env)
+
cmd = ["make",
- "O={}".format(self.builddir),
- "olddefconfig"]
+ "O={}".format(self.builddir)]
+ cmd += make_extra_opts
+ cmd += ["olddefconfig"]
+
ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
env=env)
if ret != 0:
raise SystemError("Cannot olddefconfig")
- def build(self):
+ def build(self, make_extra_opts=[], make_extra_env={}):
+ """
+ Perform the build.
+
+ make_extra_opts: a list of arguments to be passed to the make
+ command. It can include a make target.
+ e.g. make_extra_opts=["foo-source"]
+
+ make_extra_env: a dict of variables to be appended (or replaced)
+ in the environment that calls make.
+ e.g. make_extra_env={"BR2_DL_DIR": "/path"}
+ """
env = {"PATH": os.environ["PATH"]}
+ env.update(make_extra_env)
+
cmd = ["make", "-C", self.builddir]
+ cmd += make_extra_opts
+
ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
env=env)
if ret != 0:
--
2.17.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 02/10] testing/infra: split runtime test from BRTest
2018-05-12 2:58 ` [Buildroot] [PATCH v5 00/10] tests for git download infra v5 Ricardo Martincoski
2018-05-12 2:58 ` [Buildroot] [PATCH v5 01/10] testing/infra/builder: build with target and environment Ricardo Martincoski
@ 2018-05-12 2:58 ` Ricardo Martincoski
2019-02-04 15:55 ` Arnout Vandecappelle
2018-05-12 2:58 ` [Buildroot] [PATCH v5 03/10] testing/infra/basetest: support br2-external Ricardo Martincoski
` (7 subsequent siblings)
9 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2018-05-12 2:58 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Move the setup of emulator to a new class, RuntimeTestBase, that bahaves
exactly like BRTest currently does.
It will avoid duplicating code when adding a common class to test the
git download infra.
Change all current test cases to use the new class.
Do this by first using automatic replace:
$ find support/testing/ -name '*.py' | \
xargs grep -l BRTest | \
xargs sed -i \
-e 's,import infra.basetest,\0\nimport infra.runtimetest,g' \
-e 's,infra.basetest.BRTest,infra.runtimetest.RuntimeTestBase,g'
and then manually add code to import runtimetest in test_external.py to
avoid this error:
AttributeError: 'module' object has no attribute 'LoadTestsFailure'
This explicit import was not need before because run-tests imports
BRTest and this is the only test file that do not use the defconfig
fragments from basetest.py in its code.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
WARNING: this patch changes all current test cases, so if a new test case was
applied after this patch was sent, 'git am' will apply this patch cleanly, yet
any testcase created in the meantime will be broken.
But I can refresh it if need. It will probably only need the automatic replace
from the commit log to be re-run.
Changes v4 -> v5:
- no changes
Changes v3 -> v4:
- re-run the automatic replace command
Changes v2 -> v3:
- new patch
- search for "RuntimeTestBase" in
http://patchwork.ozlabs.org/patch/806161/
---
support/testing/infra/basetest.py | 10 --------
support/testing/infra/runtimetest.py | 23 +++++++++++++++++++
support/testing/tests/boot/test_atf.py | 7 +++---
.../testing/tests/core/test_post_scripts.py | 3 ++-
.../testing/tests/core/test_rootfs_overlay.py | 3 ++-
support/testing/tests/core/test_timezone.py | 7 +++---
support/testing/tests/fs/test_ext.py | 9 ++++----
support/testing/tests/fs/test_iso9660.py | 13 ++++++-----
support/testing/tests/fs/test_jffs2.py | 3 ++-
support/testing/tests/fs/test_squashfs.py | 3 ++-
support/testing/tests/fs/test_ubi.py | 3 ++-
support/testing/tests/fs/test_yaffs2.py | 3 ++-
support/testing/tests/init/base.py | 3 ++-
.../testing/tests/package/test_dropbear.py | 3 ++-
support/testing/tests/package/test_python.py | 3 ++-
support/testing/tests/package/test_rust.py | 3 ++-
.../testing/tests/package/test_syslog_ng.py | 3 ++-
.../testing/tests/toolchain/test_external.py | 3 ++-
18 files changed, 67 insertions(+), 38 deletions(-)
create mode 100644 support/testing/infra/runtimetest.py
diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
index f3f13ad97f..4773312585 100644
--- a/support/testing/infra/basetest.py
+++ b/support/testing/infra/basetest.py
@@ -3,7 +3,6 @@ import os
import datetime
from infra.builder import Builder
-from infra.emulator import Emulator
BASIC_TOOLCHAIN_CONFIG = \
"""
@@ -41,7 +40,6 @@ class BRTest(unittest.TestCase):
super(BRTest, self).__init__(names)
self.testname = self.__class__.__name__
self.builddir = self.outputdir and os.path.join(self.outputdir, self.testname)
- self.emulator = None
self.config += '\nBR2_DL_DIR="{}"\n'.format(self.downloaddir)
self.config += "\nBR2_JLEVEL={}\n".format(self.jlevel)
@@ -57,17 +55,9 @@ class BRTest(unittest.TestCase):
self.b.delete()
if not self.b.is_finished():
- self.show_msg("Building")
self.b.configure()
- self.b.build()
- self.show_msg("Building done")
-
- self.emulator = Emulator(self.builddir, self.downloaddir,
- self.logtofile, self.timeout_multiplier)
def tearDown(self):
self.show_msg("Cleaning up")
- if self.emulator:
- self.emulator.stop()
if self.b and not self.keepbuilds:
self.b.delete()
diff --git a/support/testing/infra/runtimetest.py b/support/testing/infra/runtimetest.py
new file mode 100644
index 0000000000..68bb03d091
--- /dev/null
+++ b/support/testing/infra/runtimetest.py
@@ -0,0 +1,23 @@
+from infra.basetest import BRTest
+from infra.emulator import Emulator
+
+
+class RuntimeTestBase(BRTest):
+ def __init__(self, names):
+ super(RuntimeTestBase, self).__init__(names)
+ self.emulator = None
+
+ def setUp(self):
+ super(RuntimeTestBase, self).setUp()
+ if not self.b.is_finished():
+ self.show_msg("Building")
+ self.b.build()
+ self.show_msg("Building done")
+
+ self.emulator = Emulator(self.builddir, self.downloaddir,
+ self.logtofile, self.timeout_multiplier)
+
+ def tearDown(self):
+ if self.emulator:
+ self.emulator.stop()
+ super(RuntimeTestBase, self).tearDown()
diff --git a/support/testing/tests/boot/test_atf.py b/support/testing/tests/boot/test_atf.py
index 75cea01fc8..0ea486fb5b 100644
--- a/support/testing/tests/boot/test_atf.py
+++ b/support/testing/tests/boot/test_atf.py
@@ -1,7 +1,8 @@
import infra.basetest
+import infra.runtimetest
-class TestATFVexpress(infra.basetest.BRTest):
+class TestATFVexpress(infra.runtimetest.RuntimeTestBase):
config = \
"""
BR2_aarch64=y
@@ -25,7 +26,7 @@ class TestATFVexpress(infra.basetest.BRTest):
pass
-class TestATFAllwinner(infra.basetest.BRTest):
+class TestATFAllwinner(infra.runtimetest.RuntimeTestBase):
config = \
"""
BR2_aarch64=y
@@ -54,7 +55,7 @@ class TestATFAllwinner(infra.basetest.BRTest):
pass
-class TestATFMarvell(infra.basetest.BRTest):
+class TestATFMarvell(infra.runtimetest.RuntimeTestBase):
config = \
"""
BR2_aarch64=y
diff --git a/support/testing/tests/core/test_post_scripts.py b/support/testing/tests/core/test_post_scripts.py
index a0e5b6b454..c4dfb91550 100644
--- a/support/testing/tests/core/test_post_scripts.py
+++ b/support/testing/tests/core/test_post_scripts.py
@@ -2,9 +2,10 @@ import os
import csv
import infra.basetest
+import infra.runtimetest
-class TestPostScripts(infra.basetest.BRTest):
+class TestPostScripts(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_INIT_NONE=y
diff --git a/support/testing/tests/core/test_rootfs_overlay.py b/support/testing/tests/core/test_rootfs_overlay.py
index fedd40d8ac..66b5531221 100644
--- a/support/testing/tests/core/test_rootfs_overlay.py
+++ b/support/testing/tests/core/test_rootfs_overlay.py
@@ -2,13 +2,14 @@ import os
import subprocess
import infra.basetest
+import infra.runtimetest
def compare_file(file1, file2):
return subprocess.call(["cmp", file1, file2])
-class TestRootfsOverlay(infra.basetest.BRTest):
+class TestRootfsOverlay(infra.runtimetest.RuntimeTestBase):
rootfs_overlay_path = infra.filepath("tests/core/rootfs-overlay")
diff --git a/support/testing/tests/core/test_timezone.py b/support/testing/tests/core/test_timezone.py
index 050624e0aa..dca38bebe8 100644
--- a/support/testing/tests/core/test_timezone.py
+++ b/support/testing/tests/core/test_timezone.py
@@ -1,6 +1,7 @@
import os
import infra.basetest
+import infra.runtimetest
def boot_armv5_cpio(emulator, builddir):
@@ -10,7 +11,7 @@ def boot_armv5_cpio(emulator, builddir):
emulator.login()
-class TestNoTimezone(infra.basetest.BRTest):
+class TestNoTimezone(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
# BR2_TARGET_TZ_INFO is not set
@@ -26,7 +27,7 @@ class TestNoTimezone(infra.basetest.BRTest):
self.assertEqual(tz[0].strip(), "UTC")
-class TestGlibcAllTimezone(infra.basetest.BRTest):
+class TestGlibcAllTimezone(infra.runtimetest.RuntimeTestBase):
config = \
"""
BR2_arm=y
@@ -48,7 +49,7 @@ class TestGlibcAllTimezone(infra.basetest.BRTest):
self.assertEqual(tz[0].strip(), "CET")
-class TestGlibcNonDefaultLimitedTimezone(infra.basetest.BRTest):
+class TestGlibcNonDefaultLimitedTimezone(infra.runtimetest.RuntimeTestBase):
config = \
"""
BR2_arm=y
diff --git a/support/testing/tests/fs/test_ext.py b/support/testing/tests/fs/test_ext.py
index f5f9e9fdf1..a8f68bc54c 100644
--- a/support/testing/tests/fs/test_ext.py
+++ b/support/testing/tests/fs/test_ext.py
@@ -2,6 +2,7 @@ import os
import subprocess
import infra.basetest
+import infra.runtimetest
VOLNAME_PROP = "Filesystem volume name"
REVISION_PROP = "Filesystem revision #"
@@ -41,7 +42,7 @@ def boot_img_and_check_fs_type(emulator, builddir, fs_type):
return exit_code
-class TestExt2(infra.basetest.BRTest):
+class TestExt2(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_EXT2=y
@@ -60,7 +61,7 @@ class TestExt2(infra.basetest.BRTest):
self.assertEqual(exit_code, 0)
-class TestExt2r1(infra.basetest.BRTest):
+class TestExt2r1(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_EXT2=y
@@ -80,7 +81,7 @@ class TestExt2r1(infra.basetest.BRTest):
self.assertEqual(exit_code, 0)
-class TestExt3(infra.basetest.BRTest):
+class TestExt3(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_EXT2=y
@@ -99,7 +100,7 @@ class TestExt3(infra.basetest.BRTest):
self.assertEqual(exit_code, 0)
-class TestExt4(infra.basetest.BRTest):
+class TestExt4(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_EXT2=y
diff --git a/support/testing/tests/fs/test_iso9660.py b/support/testing/tests/fs/test_iso9660.py
index 68f4840852..77255ac93b 100644
--- a/support/testing/tests/fs/test_iso9660.py
+++ b/support/testing/tests/fs/test_iso9660.py
@@ -1,6 +1,7 @@
import os
import infra.basetest
+import infra.runtimetest
BASIC_CONFIG = \
"""
@@ -47,7 +48,7 @@ def test_touch_file(emulator):
# Grub 2
-class TestIso9660Grub2External(infra.basetest.BRTest):
+class TestIso9660Grub2External(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
@@ -67,7 +68,7 @@ class TestIso9660Grub2External(infra.basetest.BRTest):
self.assertEqual(exit_code, 1)
-class TestIso9660Grub2ExternalCompress(infra.basetest.BRTest):
+class TestIso9660Grub2ExternalCompress(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
@@ -88,7 +89,7 @@ class TestIso9660Grub2ExternalCompress(infra.basetest.BRTest):
self.assertEqual(exit_code, 1)
-class TestIso9660Grub2Internal(infra.basetest.BRTest):
+class TestIso9660Grub2Internal(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
@@ -111,7 +112,7 @@ class TestIso9660Grub2Internal(infra.basetest.BRTest):
# Syslinux
-class TestIso9660SyslinuxExternal(infra.basetest.BRTest):
+class TestIso9660SyslinuxExternal(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
@@ -130,7 +131,7 @@ class TestIso9660SyslinuxExternal(infra.basetest.BRTest):
self.assertEqual(exit_code, 1)
-class TestIso9660SyslinuxExternalCompress(infra.basetest.BRTest):
+class TestIso9660SyslinuxExternalCompress(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
@@ -150,7 +151,7 @@ class TestIso9660SyslinuxExternalCompress(infra.basetest.BRTest):
self.assertEqual(exit_code, 1)
-class TestIso9660SyslinuxInternal(infra.basetest.BRTest):
+class TestIso9660SyslinuxInternal(infra.runtimetest.RuntimeTestBase):
config = BASIC_CONFIG + \
"""
BR2_TARGET_ROOTFS_ISO9660=y
diff --git a/support/testing/tests/fs/test_jffs2.py b/support/testing/tests/fs/test_jffs2.py
index 2ff5099180..8f73cf1f06 100644
--- a/support/testing/tests/fs/test_jffs2.py
+++ b/support/testing/tests/fs/test_jffs2.py
@@ -2,6 +2,7 @@ import os
import subprocess
import infra.basetest
+import infra.runtimetest
def jffs2dump_find_file(files_list, fname):
@@ -12,7 +13,7 @@ def jffs2dump_find_file(files_list, fname):
return False
-class TestJffs2(infra.basetest.BRTest):
+class TestJffs2(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_JFFS2=y
diff --git a/support/testing/tests/fs/test_squashfs.py b/support/testing/tests/fs/test_squashfs.py
index 066c054342..f80149f9d9 100644
--- a/support/testing/tests/fs/test_squashfs.py
+++ b/support/testing/tests/fs/test_squashfs.py
@@ -2,9 +2,10 @@ import os
import subprocess
import infra.basetest
+import infra.runtimetest
-class TestSquashfs(infra.basetest.BRTest):
+class TestSquashfs(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_SQUASHFS=y
diff --git a/support/testing/tests/fs/test_ubi.py b/support/testing/tests/fs/test_ubi.py
index 015d82f769..c724f4f740 100644
--- a/support/testing/tests/fs/test_ubi.py
+++ b/support/testing/tests/fs/test_ubi.py
@@ -2,9 +2,10 @@ import subprocess
import os
import infra.basetest
+import infra.runtimetest
-class TestUbi(infra.basetest.BRTest):
+class TestUbi(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_UBIFS=y
diff --git a/support/testing/tests/fs/test_yaffs2.py b/support/testing/tests/fs/test_yaffs2.py
index b60e90e660..c7c8c1f724 100644
--- a/support/testing/tests/fs/test_yaffs2.py
+++ b/support/testing/tests/fs/test_yaffs2.py
@@ -1,9 +1,10 @@
import os
import infra.basetest
+import infra.runtimetest
-class TestYaffs2(infra.basetest.BRTest):
+class TestYaffs2(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
infra.basetest.MINIMAL_CONFIG + \
"""
diff --git a/support/testing/tests/init/base.py b/support/testing/tests/init/base.py
index 75cfbe9c59..1b736af657 100644
--- a/support/testing/tests/init/base.py
+++ b/support/testing/tests/init/base.py
@@ -1,9 +1,10 @@
import os
import subprocess
import infra.basetest
+import infra.runtimetest
-class InitSystemBase(infra.basetest.BRTest):
+class InitSystemBase(infra.runtimetest.RuntimeTestBase):
def start_emulator(self, fs_type, kernel=None, dtb=None, init=None):
img = os.path.join(self.builddir, "images", "rootfs.{}".format(fs_type))
diff --git a/support/testing/tests/package/test_dropbear.py b/support/testing/tests/package/test_dropbear.py
index 8f7f1fee82..8f7f30e3af 100644
--- a/support/testing/tests/package/test_dropbear.py
+++ b/support/testing/tests/package/test_dropbear.py
@@ -1,9 +1,10 @@
import os
import infra.basetest
+import infra.runtimetest
-class TestDropbear(infra.basetest.BRTest):
+class TestDropbear(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_GENERIC_ROOT_PASSWD="testpwd"
diff --git a/support/testing/tests/package/test_python.py b/support/testing/tests/package/test_python.py
index 26cf49947b..787364c719 100644
--- a/support/testing/tests/package/test_python.py
+++ b/support/testing/tests/package/test_python.py
@@ -1,9 +1,10 @@
import os
import infra.basetest
+import infra.runtimetest
-class TestPythonBase(infra.basetest.BRTest):
+class TestPythonBase(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_CPIO=y
diff --git a/support/testing/tests/package/test_rust.py b/support/testing/tests/package/test_rust.py
index 2dc814f99d..df445f1186 100644
--- a/support/testing/tests/package/test_rust.py
+++ b/support/testing/tests/package/test_rust.py
@@ -4,9 +4,10 @@ import subprocess
import shutil
import infra.basetest
+import infra.runtimetest
-class TestRustBase(infra.basetest.BRTest):
+class TestRustBase(infra.runtimetest.RuntimeTestBase):
target = 'armv7-unknown-linux-gnueabihf'
crate = 'hello-world'
diff --git a/support/testing/tests/package/test_syslog_ng.py b/support/testing/tests/package/test_syslog_ng.py
index 0155ef14e4..df6ce8b8d0 100644
--- a/support/testing/tests/package/test_syslog_ng.py
+++ b/support/testing/tests/package/test_syslog_ng.py
@@ -1,9 +1,10 @@
import os
import infra.basetest
+import infra.runtimetest
-class TestSyslogNg(infra.basetest.BRTest):
+class TestSyslogNg(infra.runtimetest.RuntimeTestBase):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_PACKAGE_BUSYBOX_SHOW_OTHERS=y
diff --git a/support/testing/tests/toolchain/test_external.py b/support/testing/tests/toolchain/test_external.py
index 881d2b00db..b72e19f740 100644
--- a/support/testing/tests/toolchain/test_external.py
+++ b/support/testing/tests/toolchain/test_external.py
@@ -1,5 +1,6 @@
import os
import infra
+import infra.runtimetest
BASIC_CONFIG = \
"""
@@ -17,7 +18,7 @@ def has_broken_links(path):
return False
-class TestExternalToolchain(infra.basetest.BRTest):
+class TestExternalToolchain(infra.runtimetest.RuntimeTestBase):
def common_check(self):
# Check for broken symlinks
for d in ["lib", "usr/lib"]:
--
2.17.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 03/10] testing/infra/basetest: support br2-external
2018-05-12 2:58 ` [Buildroot] [PATCH v5 00/10] tests for git download infra v5 Ricardo Martincoski
2018-05-12 2:58 ` [Buildroot] [PATCH v5 01/10] testing/infra/builder: build with target and environment Ricardo Martincoski
2018-05-12 2:58 ` [Buildroot] [PATCH v5 02/10] testing/infra: split runtime test from BRTest Ricardo Martincoski
@ 2018-05-12 2:58 ` Ricardo Martincoski
2018-05-12 2:58 ` [Buildroot] [PATCH v5 04/10] testing/tests/download: add git hash test Ricardo Martincoski
` (6 subsequent siblings)
9 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2018-05-12 2:58 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
This change will be needed when adding a common class to test the git
download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Changes v4 -> v5:
- no changes
Changes v3 -> v4:
- allow the use of multiple br2-external trees, simplifying the code
Changes v2 -> v3:
- new patch
- search for "support for BR2_EXTERNAL directly in BRTest" in
http://patchwork.ozlabs.org/patch/806161/
---
support/testing/infra/basetest.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
index 4773312585..67076701e7 100644
--- a/support/testing/infra/basetest.py
+++ b/support/testing/infra/basetest.py
@@ -29,6 +29,7 @@ MINIMAL_CONFIG = \
class BRTest(unittest.TestCase):
config = None
+ br2_external = list()
downloaddir = None
outputdir = None
logtofile = True
@@ -55,7 +56,7 @@ class BRTest(unittest.TestCase):
self.b.delete()
if not self.b.is_finished():
- self.b.configure()
+ self.b.configure(["BR2_EXTERNAL={}".format(":".join(self.br2_external))])
def tearDown(self):
self.show_msg("Cleaning up")
--
2.17.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 04/10] testing/tests/download: add git hash test
2018-05-12 2:58 ` [Buildroot] [PATCH v5 00/10] tests for git download infra v5 Ricardo Martincoski
` (2 preceding siblings ...)
2018-05-12 2:58 ` [Buildroot] [PATCH v5 03/10] testing/infra/basetest: support br2-external Ricardo Martincoski
@ 2018-05-12 2:58 ` Ricardo Martincoski
2019-02-04 15:52 ` Arnout Vandecappelle
2018-05-12 2:58 ` [Buildroot] [PATCH v5 05/10] testing/tests/download: test case for git refs Ricardo Martincoski
` (5 subsequent siblings)
9 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2018-05-12 2:58 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Add one test case to ensure the hash is checked for git packages:
- correct hash;
- wrong hash;
- no hash file.
Add required infra:
- a GitRemote class, that can start a git server in the host machine to
emulate a remote git server under the control of the test;
- a new base class, called GitTestBase, that inherits from BRTest and
must be subclassed by all git test cases.
Its setUp() method takes care of configuring the build with a
br2-external, avoiding to hit http://sources.buildroot.net by using
an empty BR2_BACKUP_SITE. It also avoids downloading not
pre-installed dependencies (i.e. lzip) every time by calling 'make
dependencies' using the common dl directory, and it instantiates the
GitRemote object.
Besides the Python scripts, add some fixtures used during the tests:
- a br2-external (git-hash) with one package for each part of the test
case;
- a static git bare repo (repo.git) to be served using GitRemote class.
Neither the br2-external nor the check hash functionalities are the
subject of these tests per se, so for simplicity limit the check to the
error codes and don't look for the messages in the log.
Thanks to Arnout for the hint about how to add a bare repo to test.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repo with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* a238b1d (HEAD -> master) initial
Changes v4 -> v5:
- no changes
Changes v3 -> v4:
- remove call to 'make dependencies';
- merge generic git base file to the main test case as the first was
only used by the second;
- use variable naming similar to builder;
- move useful methods to the base class;
- rename logfile to -gitremote (it will become handy when adding the
http server on a later patch);
- use more columns (our limit is now 132);
- merge start to __init__ as it was called only there;
- remove temp variable used only once (port_arg);
- use a more future-proof string format for the port arg (it will
become handy when adding the http server on a later patch);
- use simpler code to raise exception if there is no free port;
- fix the range for ports to use;
- br2_external is now a list;
Changes v2 -> v3:
- renumber 5 -> 6;
- rebase after removing patch 3 "allow to override logfile" from the
series and adding new patches 4 "split runtime test from BRTest" and
5 "support br2-external";
- merge part of old patch 4 (very minimal GitTestBase) to old patch 5
(Arnout Vandecappelle);
- merge and tweak part of commit log from old patch 4;
- move GITREMOTE_PORT_NUMBER to external.mk (Arnout Vandecappelle);
- use dosctrings;
- start git remote from __init__ (Arnout Vandecappelle), but I kept a
separate method and call start();
- remove dead code (Arnout Vandecappelle);
- use the same git repo for all tests (Arnout Vandecappelle);
- rename the file to test_git and merge the 3 test cases in one, since
the feature "check hash for git packages" is only OK if all work
well and also because the test does not take too much time;
- update commit log accordingly.
Changes v1 -> v2 (patch 4):
- new patch
Changes v1 -> v2 (patch 5):
- use git daemon + git:// instead of ssh (Arnout);
- remove __main__ handling since the test infra already does that;
- use the logging support from test infra;
- this patch is part of series 1/3 of a new version of
http://patchwork.ozlabs.org/patch/690097/
---
.gitlab-ci.yml | 1 +
support/testing/tests/download/__init__.py | 0
.../download/br2-external/git-hash/Config.in | 0
.../br2-external/git-hash/external.desc | 1 +
.../br2-external/git-hash/external.mk | 4 ++
.../git-hash/package/bad/bad.hash | 1 +
.../br2-external/git-hash/package/bad/bad.mk | 10 ++++
.../git-hash/package/good/good.hash | 1 +
.../git-hash/package/good/good.mk | 10 ++++
.../git-hash/package/nohash/nohash.mk | 10 ++++
.../git-remote/repo.git/.gitattributes | 1 +
.../tests/download/git-remote/repo.git/HEAD | 1 +
.../tests/download/git-remote/repo.git/config | 4 ++
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../a2/38b1dfcd825d47d834af3c5223417c8411d90d | Bin 0 -> 152 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../git-remote/repo.git/refs/heads/master | 1 +
support/testing/tests/download/gitremote.py | 44 ++++++++++++++++++
support/testing/tests/download/test_git.py | 43 +++++++++++++++++
19 files changed, 132 insertions(+)
create mode 100644 support/testing/tests/download/__init__.py
create mode 100644 support/testing/tests/download/br2-external/git-hash/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-hash/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-hash/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.hash
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.mk
create mode 100644 support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk
create mode 100644 support/testing/tests/download/git-remote/repo.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/repo.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/repo.git/config
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/a2/38b1dfcd825d47d834af3c5223417c8411d90d
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/master
create mode 100644 support/testing/tests/download/gitremote.py
create mode 100644 support/testing/tests/download/test_git.py
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index e80491cdde..27b0255343 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -263,6 +263,7 @@ tests.core.test_rootfs_overlay.TestRootfsOverlay: *runtime_test
tests.core.test_timezone.TestGlibcAllTimezone: *runtime_test
tests.core.test_timezone.TestGlibcNonDefaultLimitedTimezone: *runtime_test
tests.core.test_timezone.TestNoTimezone: *runtime_test
+tests.download.test_git.TestGitHash: *runtime_test
tests.fs.test_ext.TestExt2: *runtime_test
tests.fs.test_ext.TestExt2r1: *runtime_test
tests.fs.test_ext.TestExt3: *runtime_test
diff --git a/support/testing/tests/download/__init__.py b/support/testing/tests/download/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/support/testing/tests/download/br2-external/git-hash/Config.in b/support/testing/tests/download/br2-external/git-hash/Config.in
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/support/testing/tests/download/br2-external/git-hash/external.desc b/support/testing/tests/download/br2-external/git-hash/external.desc
new file mode 100644
index 0000000000..41316c8b25
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/external.desc
@@ -0,0 +1 @@
+name: GIT_HASH
diff --git a/support/testing/tests/download/br2-external/git-hash/external.mk b/support/testing/tests/download/br2-external/git-hash/external.mk
new file mode 100644
index 0000000000..4646dfe2b0
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/external.mk
@@ -0,0 +1,4 @@
+include $(sort $(wildcard $(BR2_EXTERNAL_GIT_HASH_PATH)/package/*/*.mk))
+
+# Get the git server port number from the test infra
+GITREMOTE_PORT_NUMBER ?= 9418
diff --git a/support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
new file mode 100644
index 0000000000..b9e1baec84
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
@@ -0,0 +1 @@
+sha256 0000000000000000000000000000000000000000000000000000000000000000 bad-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
diff --git a/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
new file mode 100644
index 0000000000..5497bd6bfe
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk
@@ -0,0 +1,10 @@
+################################################################################
+#
+# bad
+#
+################################################################################
+
+BAD_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+BAD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-hash/package/good/good.hash b/support/testing/tests/download/br2-external/git-hash/package/good/good.hash
new file mode 100644
index 0000000000..9e92ab8ab9
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/good/good.hash
@@ -0,0 +1 @@
+sha256 d00ae598e9e770d607621a86766030b42eaa58156cb8d482b043969da7963c23 good-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
diff --git a/support/testing/tests/download/br2-external/git-hash/package/good/good.mk b/support/testing/tests/download/br2-external/git-hash/package/good/good.mk
new file mode 100644
index 0000000000..0f0eefd944
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/good/good.mk
@@ -0,0 +1,10 @@
+################################################################################
+#
+# good
+#
+################################################################################
+
+GOOD_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+GOOD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk b/support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk
new file mode 100644
index 0000000000..1da19d88c6
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk
@@ -0,0 +1,10 @@
+################################################################################
+#
+# nohash
+#
+################################################################################
+
+NOHASH_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+NOHASH_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/repo.git/.gitattributes b/support/testing/tests/download/git-remote/repo.git/.gitattributes
new file mode 100644
index 0000000000..eb50c64a21
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/.gitattributes
@@ -0,0 +1 @@
+objects/*/* binary
diff --git a/support/testing/tests/download/git-remote/repo.git/HEAD b/support/testing/tests/download/git-remote/repo.git/HEAD
new file mode 100644
index 0000000000..cb089cd89a
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/HEAD
@@ -0,0 +1 @@
+ref: refs/heads/master
diff --git a/support/testing/tests/download/git-remote/repo.git/config b/support/testing/tests/download/git-remote/repo.git/config
new file mode 100644
index 0000000000..07d359d07c
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/config
@@ -0,0 +1,4 @@
+[core]
+ repositoryformatversion = 0
+ filemode = true
+ bare = true
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab b/support/testing/tests/download/git-remote/repo.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
new file mode 100644
index 0000000000000000000000000000000000000000..9db72668cf9374b0b85a25a19f30084fd460072d
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5wZcw)UHz=nDZYB`lCt5lYr9_mN?Omvz#_so
F4FHVv5b^*3
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/a2/38b1dfcd825d47d834af3c5223417c8411d90d b/support/testing/tests/download/git-remote/repo.git/objects/a2/38b1dfcd825d47d834af3c5223417c8411d90d
new file mode 100644
index 0000000000000000000000000000000000000000..31b6bcf34de247d14531c70b36e28d895ba20af3
GIT binary patch
literal 152
zcmV;J0B8Sr0j<tW4#FT1Kw;OMVlL1IsFY%i at d9r21jApQv_OKm@b)%w;oi9Y7BA10
zl&FE!)2`JJz?dk*5QMWrMPrqWC`}wkKTO<v<fHILXtWHU?OrNe$zk;cE?667R~`$&
zv3{^mUp&tVY3*G}ClHEjqx7`b(D8wDw&^y}!k==CGBv`Lu^Y`f_^!m*+1KHhS@s5O
G+F(P1IZe3$
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5 b/support/testing/tests/download/git-remote/repo.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
new file mode 100644
index 0000000000000000000000000000000000000000..df2037a2d2d30afb20e31cd558c92c2edabe7cf6
GIT binary patch
literal 23
fcmb<m^geacKWIb3i8H5lPWqgA$i$$b%kl;Qg;xrl
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/heads/master b/support/testing/tests/download/git-remote/repo.git/refs/heads/master
new file mode 100644
index 0000000000..b6bccc1c17
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/heads/master
@@ -0,0 +1 @@
+a238b1dfcd825d47d834af3c5223417c8411d90d
diff --git a/support/testing/tests/download/gitremote.py b/support/testing/tests/download/gitremote.py
new file mode 100644
index 0000000000..60bc49fbf8
--- /dev/null
+++ b/support/testing/tests/download/gitremote.py
@@ -0,0 +1,44 @@
+# subprocess does not kill the child daemon when a test case fails by raising
+# an exception. So use pexpect instead.
+import pexpect
+
+import infra
+
+GIT_REMOTE_PORT_INITIAL = 9418
+GIT_REMOTE_PORT_LAST = GIT_REMOTE_PORT_INITIAL + 99
+
+
+class GitRemote(object):
+ def __init__(self, builddir, serveddir, logtofile):
+ """
+ Start a local git server.
+
+ In order to support test cases in parallel, select the port the
+ server will listen to in runtime. Since there is no reliable way
+ to allocate the port prior to starting the server (another
+ process in the host machine can use the port between it is
+ selected from a list and it is really allocated to the server)
+ try to start the server in a port and in the case it is already
+ in use, try the next one in the allowed range.
+ """
+ self.daemon = None
+ self.port = None
+ self.logfile = infra.open_log_file(builddir, "gitremote", logtofile)
+
+ daemon_cmd = ["git", "daemon", "--reuseaddr", "--verbose", "--listen=localhost", "--export-all",
+ "--base-path={}".format(serveddir)]
+ for port in range(GIT_REMOTE_PORT_INITIAL, GIT_REMOTE_PORT_LAST + 1):
+ cmd = daemon_cmd + ["--port={port}".format(port=port)]
+ self.logfile.write("> starting git remote with '{}'\n".format(" ".join(cmd)))
+ self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile)
+ ret = self.daemon.expect(["Ready to rumble",
+ "Address already in use"])
+ if ret == 0:
+ self.port = port
+ return
+ raise SystemError("Could not find a free port to run git remote")
+
+ def stop(self):
+ if self.daemon is None:
+ return
+ self.daemon.terminate(force=True)
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
new file mode 100644
index 0000000000..14fc8e4da3
--- /dev/null
+++ b/support/testing/tests/download/test_git.py
@@ -0,0 +1,43 @@
+import os
+
+import infra
+from gitremote import GitRemote
+
+
+class GitTestBase(infra.basetest.BRTest):
+ config = \
+ """
+ BR2_BACKUP_SITE=""
+ """
+ gitremotedir = infra.filepath("tests/download/git-remote")
+ gitremote = None
+
+ def setUp(self):
+ super(GitTestBase, self).setUp()
+ self.gitremote = GitRemote(self.builddir, self.gitremotedir, self.logtofile)
+
+ def tearDown(self):
+ self.show_msg("Cleaning up")
+ if self.gitremote:
+ self.gitremote.stop()
+ if self.b and not self.keepbuilds:
+ self.b.delete()
+
+ def check_hash(self, package):
+ # store downloaded tarball inside the output dir so the test infra
+ # cleans it up at the end
+ env = {"BR2_DL_DIR": os.path.join(self.builddir, "dl"),
+ "GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
+ self.b.build(["{}-dirclean".format(package),
+ "{}-source".format(package)],
+ env)
+
+
+class TestGitHash(GitTestBase):
+ br2_external = [infra.filepath("tests/download/br2-external/git-hash")]
+
+ def test_run(self):
+ with self.assertRaises(SystemError):
+ self.check_hash("bad")
+ self.check_hash("good")
+ self.check_hash("nohash")
--
2.17.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 05/10] testing/tests/download: test case for git refs
2018-05-12 2:58 ` [Buildroot] [PATCH v5 00/10] tests for git download infra v5 Ricardo Martincoski
` (3 preceding siblings ...)
2018-05-12 2:58 ` [Buildroot] [PATCH v5 04/10] testing/tests/download: add git hash test Ricardo Martincoski
@ 2018-05-12 2:58 ` Ricardo Martincoski
2019-02-04 19:48 ` Arnout Vandecappelle
2018-05-12 2:58 ` [Buildroot] [PATCH v5 06/10] testing/tests/download: test git branch Ricardo Martincoski
` (4 subsequent siblings)
9 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2018-05-12 2:58 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
All upcoming tests for git refs will rely on the return code of make to
determine whether a git ref can be downloaded or not and also to
determine whether the downloaded content is correct (all of this taking
advantage of the check-hash mechanism already in place for git
packages).
So to avoid false results i.e. in the case the check-hash mechanism
become broken in the master branch, add some sanity checks before the
actual test of download git refs.
Add the minimum test case for git refs containing only sanity checks.
Reuse the commit in the static repo.
Add a br2-external with two packages to check that:
- trying to download an invalid sha1 generates an error;
- downloading a valid sha1 that contains unexpected content generates
an error.
In order to ease the maintenance and review, each upcoming patch adding
checks to this test case will add at same time the commits to the static
repo, the equivalent packages to the br2-external and code to the test
case.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repo with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* a238b1d (HEAD -> master) initial
Changes v4 -> v5:
- no changes
Changes v3 -> v4:
- move useful methods to the git base class;
- br2_external is now a list;
Changes v2 -> v3:
- rewrite from v2, using an static repo;
- separate on this patch the bare minimum static repo (actually reuse
from previous commit), the next patches on the series will gradually
add new commits to the static repo together to the equivalent
packages and call to check_download. This approach allows better
maintenance, even during the review of the series, because a patch
can be more easily reworked/discarded/reordered;
Changes v1 -> v2:
- rename the main test file to testgit (Arnout). Actually I broke it
down to a common gitbase and the two test_git_refs and test_git_hash
(this last one is new, in the previous patch);
- remove some weird/wrong TODO I had in v1 code (Arnout);
- use the same structure for test cases as used in the test infra;
- raise an exception when the download fails (Arnout). I did not add
code for this since I let the builder class to raise it;
- I reimplemented git_util as gitrepo. Arnout suggested to use it as
module and I first implemented that way locally, but then I created
test cases for submodules and I realized now I have a reason to use
a class (see next patch);
- move package (now called foo) to a BR2_EXTERNAL (Arnout);
- override BR2_DL_DIR when calling make (Arnout);
- the test infra uses O= so I don't need to;
- instead of removing the files in the dl/ folder, use a different dir
as dl/ dir since now each repo has a different name;
- open the tarball to check its contents (Arnout). I create a fresh
clone of the repo before the test to compare against;
- this patch is not checking for the actual sha1 anymore, it can be
done later. But the check for the contents should cover most cases;
- my old argument to not test only the support/download/dl-wrapper is
not valid anymore since I mimic the logic from the scripts to know
the name of the tarball. But I still think calling make ...-source
has better maintenance. And also the previous patch (test for hash
of packages with git method) can use the same base class;
- this patch is part of series 1/3 of a new version of
http://patchwork.ozlabs.org/patch/690097/
---
.gitlab-ci.yml | 1 +
.../download/br2-external/git-refs/Config.in | 0
.../br2-external/git-refs/external.desc | 1 +
.../br2-external/git-refs/external.mk | 4 ++++
.../git-wrong-content/git-wrong-content.hash | 2 ++
.../git-wrong-content/git-wrong-content.mk | 11 +++++++++++
.../package/git-wrong-sha1/git-wrong-sha1.mk | 11 +++++++++++
support/testing/tests/download/test_git.py | 19 +++++++++++++++++++
8 files changed, 49 insertions(+)
create mode 100644 support/testing/tests/download/br2-external/git-refs/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-refs/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-refs/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-wrong-sha1/git-wrong-sha1.mk
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 27b0255343..61ccfdfd2d 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -264,6 +264,7 @@ tests.core.test_timezone.TestGlibcAllTimezone: *runtime_test
tests.core.test_timezone.TestGlibcNonDefaultLimitedTimezone: *runtime_test
tests.core.test_timezone.TestNoTimezone: *runtime_test
tests.download.test_git.TestGitHash: *runtime_test
+tests.download.test_git.TestGitRefs: *runtime_test
tests.fs.test_ext.TestExt2: *runtime_test
tests.fs.test_ext.TestExt2r1: *runtime_test
tests.fs.test_ext.TestExt3: *runtime_test
diff --git a/support/testing/tests/download/br2-external/git-refs/Config.in b/support/testing/tests/download/br2-external/git-refs/Config.in
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/support/testing/tests/download/br2-external/git-refs/external.desc b/support/testing/tests/download/br2-external/git-refs/external.desc
new file mode 100644
index 0000000000..69f40d46c6
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/external.desc
@@ -0,0 +1 @@
+name: GIT_REFS
diff --git a/support/testing/tests/download/br2-external/git-refs/external.mk b/support/testing/tests/download/br2-external/git-refs/external.mk
new file mode 100644
index 0000000000..ab38c27eb7
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/external.mk
@@ -0,0 +1,4 @@
+include $(sort $(wildcard $(BR2_EXTERNAL_GIT_REFS_PATH)/package/*/*.mk))
+
+# Get the git server port number from the test infra
+GITREMOTE_PORT_NUMBER ?= 9418
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash
new file mode 100644
index 0000000000..47b2b8b7d7
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash
@@ -0,0 +1,2 @@
+ sha256 04715901977503d1df650e0959f4d31d8e7b105e2ac99a2182e0652b8f13baa1 git-wrong-content-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
+ sha256 0000000000000000000000000000000000000000000000000000000000000000 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.mk b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.mk
new file mode 100644
index 0000000000..786224dad9
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-wrong-content
+#
+################################################################################
+
+GIT_WRONG_CONTENT_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+GIT_WRONG_CONTENT_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_WRONG_CONTENT_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-wrong-sha1/git-wrong-sha1.mk b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-sha1/git-wrong-sha1.mk
new file mode 100644
index 0000000000..f9d0d2226c
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-wrong-sha1/git-wrong-sha1.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-wrong-sha1
+#
+################################################################################
+
+GIT_WRONG_SHA1_VERSION = 0000000000000000000000000000000000000000
+GIT_WRONG_SHA1_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_WRONG_SHA1_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index 14fc8e4da3..fdfc858233 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -32,6 +32,15 @@ class GitTestBase(infra.basetest.BRTest):
"{}-source".format(package)],
env)
+ def check_download(self, package):
+ # store downloaded tarball inside the output dir so the test infra
+ # cleans it up at the end
+ env = {"BR2_DL_DIR": os.path.join(self.builddir, "dl"),
+ "GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
+ self.b.build(["{}-dirclean".format(package),
+ "{}-legal-info".format(package)],
+ env)
+
class TestGitHash(GitTestBase):
br2_external = [infra.filepath("tests/download/br2-external/git-hash")]
@@ -41,3 +50,13 @@ class TestGitHash(GitTestBase):
self.check_hash("bad")
self.check_hash("good")
self.check_hash("nohash")
+
+
+class TestGitRefs(GitTestBase):
+ br2_external = [infra.filepath("tests/download/br2-external/git-refs")]
+
+ def test_run(self):
+ with self.assertRaises(SystemError):
+ self.check_download("git-wrong-content")
+ with self.assertRaises(SystemError):
+ self.check_download("git-wrong-sha1")
--
2.17.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 06/10] testing/tests/download: test git branch
2018-05-12 2:58 ` [Buildroot] [PATCH v5 00/10] tests for git download infra v5 Ricardo Martincoski
` (4 preceding siblings ...)
2018-05-12 2:58 ` [Buildroot] [PATCH v5 05/10] testing/tests/download: test case for git refs Ricardo Martincoski
@ 2018-05-12 2:58 ` Ricardo Martincoski
2019-02-05 9:42 ` Arnout Vandecappelle
2018-05-12 2:58 ` [Buildroot] [PATCH v5 07/10] testing/tests/download: test git submodules Ricardo Martincoski
` (3 subsequent siblings)
9 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2018-05-12 2:58 UTC (permalink / raw)
To: buildroot
Add a branch to the static repo and check on the git refs test case the
download of a git package:
- with a sha1 reachable by a branch name, but not pointed by it, as
version. This is the most common use case for git refs in the tree;
- with a partial sha1 of a commit reachable by a branch as version;
- with a sha1 of the commit head of a branch as version;
- with a partial sha1 of the commit head of a branch as version;
Enforce the download always occurs by removing the BR2_DL_DIR used for
the tarballs generated by the git download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repo with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* 68c197d (mybranch) branch2
* 3174063 branch1
* a238b1d (HEAD -> master) initial
Changes v4 -> v5:
- do not test a named branch, Buildroot does not support it;
Changes v3 -> v4:
- no functional changes;
- fix old comment on the code;
Changes v2 -> v3:
- complete rewrite using a static repo;
---
.../git-partial-sha1-branch-head.hash | 2 ++
.../git-partial-sha1-branch-head.mk | 11 +++++++++++
.../git-partial-sha1-reachable-by-branch.hash | 2 ++
.../git-partial-sha1-reachable-by-branch.mk | 11 +++++++++++
.../git-sha1-branch-head.hash | 2 ++
.../git-sha1-branch-head/git-sha1-branch-head.mk | 11 +++++++++++
.../git-sha1-reachable-by-branch.hash | 2 ++
.../git-sha1-reachable-by-branch.mk | 11 +++++++++++
.../11/93ff46343f4f6a0522e2b28b871e905178c1f0 | Bin 0 -> 23 bytes
.../31/7406308d9259e2231bd0d6ddad3de3832bce08 | Bin 0 -> 182 bytes
.../34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5 | Bin 0 -> 23 bytes
.../68/c197d0879d485f4f6cee85544722b79e68e59f | Bin 0 -> 184 bytes
.../b2/4b387624edc78d0292a127c43cad9ba97c6232 | Bin 0 -> 49 bytes
.../e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e | Bin 0 -> 49 bytes
.../git-remote/repo.git/refs/heads/mybranch | 1 +
support/testing/tests/download/test_git.py | 11 ++++++++++-
16 files changed, 63 insertions(+), 1 deletion(-)
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.mk
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/11/93ff46343f4f6a0522e2b28b871e905178c1f0
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/31/7406308d9259e2231bd0d6ddad3de3832bce08
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/68/c197d0879d485f4f6cee85544722b79e68e59f
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/b2/4b387624edc78d0292a127c43cad9ba97c6232
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/mybranch
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash
new file mode 100644
index 0000000000..648bcceca0
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash
@@ -0,0 +1,2 @@
+sha256 70b76187369e47db69dac02c5696e63b35199cd20490fa473d289dd377774613 git-partial-sha1-branch-head-68c197d0879d485f4f6c.tar.gz
+sha256 2c1126513651b0d346a4e6d1bb75ac1c9999217e18026302d27bea47b06c7fb2 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.mk b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.mk
new file mode 100644
index 0000000000..6d4177c5bf
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-partial-sha1-branch-head
+#
+################################################################################
+
+GIT_PARTIAL_SHA1_BRANCH_HEAD_VERSION = 68c197d0879d485f4f6c
+GIT_PARTIAL_SHA1_BRANCH_HEAD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_PARTIAL_SHA1_BRANCH_HEAD_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash
new file mode 100644
index 0000000000..d064362e2c
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash
@@ -0,0 +1,2 @@
+sha256 9db079b9e215799d59975db7b2b26671eff1932ee6cf1075296f2ace3e2cb746 git-partial-sha1-reachable-by-branch-317406308d9259e2231b.tar.gz
+sha256 fabbc65c442bacb5e69b7adfea6d14fbbfc1327134322efd12771dc84387d507 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.mk b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.mk
new file mode 100644
index 0000000000..2f65b5c9a9
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-partial-sha1-reachable-by-branch
+#
+################################################################################
+
+GIT_PARTIAL_SHA1_REACHABLE_BY_BRANCH_VERSION = 317406308d9259e2231b
+GIT_PARTIAL_SHA1_REACHABLE_BY_BRANCH_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_PARTIAL_SHA1_REACHABLE_BY_BRANCH_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash
new file mode 100644
index 0000000000..3e8f76d31a
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash
@@ -0,0 +1,2 @@
+sha256 a21a2507b6d94ad484e49e3a9ae698f672a57469aab8e1779da77df7c9d4d337 git-sha1-branch-head-68c197d0879d485f4f6cee85544722b79e68e59f.tar.gz
+sha256 2c1126513651b0d346a4e6d1bb75ac1c9999217e18026302d27bea47b06c7fb2 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.mk b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.mk
new file mode 100644
index 0000000000..d3ecaa8593
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-sha1-branch-head
+#
+################################################################################
+
+GIT_SHA1_BRANCH_HEAD_VERSION = 68c197d0879d485f4f6cee85544722b79e68e59f
+GIT_SHA1_BRANCH_HEAD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SHA1_BRANCH_HEAD_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash
new file mode 100644
index 0000000000..f8d7b5dc48
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash
@@ -0,0 +1,2 @@
+sha256 8909e76d898e651af0bc23fae4103b87888bfe77448d71aaf1fbec3da97a3ce2 git-sha1-reachable-by-branch-317406308d9259e2231bd0d6ddad3de3832bce08.tar.gz
+sha256 fabbc65c442bacb5e69b7adfea6d14fbbfc1327134322efd12771dc84387d507 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.mk b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.mk
new file mode 100644
index 0000000000..badf9e13ca
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-sha1-reachable-by-branch
+#
+################################################################################
+
+GIT_SHA1_REACHABLE_BY_BRANCH_VERSION = 317406308d9259e2231bd0d6ddad3de3832bce08
+GIT_SHA1_REACHABLE_BY_BRANCH_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SHA1_REACHABLE_BY_BRANCH_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/11/93ff46343f4f6a0522e2b28b871e905178c1f0 b/support/testing/tests/download/git-remote/repo.git/objects/11/93ff46343f4f6a0522e2b28b871e905178c1f0
new file mode 100644
index 0000000000000000000000000000000000000000..3541cd14f0bf5f3a172f868c7ec730dcc255ebd2
GIT binary patch
literal 23
fcmb<m^geacKWIaOhnCkF?{fxEm>AT(SQY{RanuOb
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/31/7406308d9259e2231bd0d6ddad3de3832bce08 b/support/testing/tests/download/git-remote/repo.git/objects/31/7406308d9259e2231bd0d6ddad3de3832bce08
new file mode 100644
index 0000000000000000000000000000000000000000..c2c986572a7dec5f263e461155a09124d0f05844
GIT binary patch
literal 182
zcmV;n07?IN0j<tEP6II%Kv8P1!W)>#-;9k=lnv036+FLBVg-+p;}q<kNGa$M?J2G<
z_SWju-tAxJVAkwV*zA+69E${E??l+TU<sm!f}90XX7vSwx2eJTR4fTna*@JH$p;XR
z(K#RNiYZu2*^q+!y!4?zD#D<y-(k$UVZU9gej88RwO>4*8`RGo_IA)Vnl;DdW7doD
k#;AurrYHYXPL%;0UaV55dfu3pF0z#G*T2lF9~m)jQ^MU{p#T5?
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5 b/support/testing/tests/download/git-remote/repo.git/objects/34/d1da713bf7de1c535e1d7d3ca985afd84bc7e5
new file mode 100644
index 0000000000000000000000000000000000000000..29f2d4fd004afa03b43e3148207e258e7aa9fc41
GIT binary patch
literal 23
fcmb<m^geacKWIaOhnCkF?{fx^m>AT3SQY^Qam@(V
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/68/c197d0879d485f4f6cee85544722b79e68e59f b/support/testing/tests/download/git-remote/repo.git/objects/68/c197d0879d485f4f6cee85544722b79e68e59f
new file mode 100644
index 0000000000000000000000000000000000000000..3e421bb97a40ab93592d94107f3a1258a99b76fa
GIT binary patch
literal 184
zcmV;p07w6L0j<tGP6IIzhGA--!WY=cJG1K_gb){?AtxBmOcE=2jf^R{Ju9V1AJIP2
z_vz(UYZ(^7KaH+xGb|Zn at UW<Gh~gjt%@BA&CP|r6U=Il9i at I+6a1M*}=%OT8lLFwJ
z#fcXoK~andnbk!@KjYH6{U}Ubnti7}%Es;cTI@@|;i2B~?OdsR9=O$4JFhOJHKIp5
mIdsnKx<?QFr#q87ZF~V^j^(_mOzm`<^6&E_vE~mXVr;h@Dp>;n
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/b2/4b387624edc78d0292a127c43cad9ba97c6232 b/support/testing/tests/download/git-remote/repo.git/objects/b2/4b387624edc78d0292a127c43cad9ba97c6232
new file mode 100644
index 0000000000000000000000000000000000000000..20491e3f176096db13c0b0c7857406e1c4119619
GIT binary patch
literal 49
zcmV-10M7q-0V^p=O;s>9VlXr?Ff%bxNXyJgWiYvTtI+!UJ(=J**;<>Gt?O at iAAbq}
H3k46<$LbTr
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e b/support/testing/tests/download/git-remote/repo.git/objects/e2/cfe068f7e5bf4de32ffe1241da53abce9fa89e
new file mode 100644
index 0000000000000000000000000000000000000000..57be6485778c7f743a82f71ef718103cecfd4f35
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5RbbAa`)7~bv+oXld2!jhvYnibqU`nz3}1Tr
F^a08O6TJWc
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/heads/mybranch b/support/testing/tests/download/git-remote/repo.git/refs/heads/mybranch
new file mode 100644
index 0000000000..45cec54673
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/heads/mybranch
@@ -0,0 +1 @@
+68c197d0879d485f4f6cee85544722b79e68e59f
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index fdfc858233..279d0ed8eb 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -1,4 +1,5 @@
import os
+import shutil
import infra
from gitremote import GitRemote
@@ -35,7 +36,11 @@ class GitTestBase(infra.basetest.BRTest):
def check_download(self, package):
# store downloaded tarball inside the output dir so the test infra
# cleans it up@the end
- env = {"BR2_DL_DIR": os.path.join(self.builddir, "dl"),
+ dl_dir = os.path.join(self.builddir, "dl")
+ # enforce we test the download
+ if os.path.exists(dl_dir):
+ shutil.rmtree(dl_dir)
+ env = {"BR2_DL_DIR": dl_dir,
"GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
self.b.build(["{}-dirclean".format(package),
"{}-legal-info".format(package)],
@@ -60,3 +65,7 @@ class TestGitRefs(GitTestBase):
self.check_download("git-wrong-content")
with self.assertRaises(SystemError):
self.check_download("git-wrong-sha1")
+ self.check_download("git-partial-sha1-branch-head")
+ self.check_download("git-partial-sha1-reachable-by-branch")
+ self.check_download("git-sha1-branch-head")
+ self.check_download("git-sha1-reachable-by-branch")
--
2.17.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 07/10] testing/tests/download: test git submodules
2018-05-12 2:58 ` [Buildroot] [PATCH v5 00/10] tests for git download infra v5 Ricardo Martincoski
` (5 preceding siblings ...)
2018-05-12 2:58 ` [Buildroot] [PATCH v5 06/10] testing/tests/download: test git branch Ricardo Martincoski
@ 2018-05-12 2:58 ` Ricardo Martincoski
2019-02-05 10:03 ` Arnout Vandecappelle
2018-05-12 2:58 ` [Buildroot] [PATCH v5 08/10] testing/tests/download: test git tag Ricardo Martincoski
` (2 subsequent siblings)
9 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2018-05-12 2:58 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Add two submodules as static repos, add a branch to the main static repo
and check on the git refs test case the download of a git package:
- repo with submodule but without support in the package;
- repo with recursive submodules with support in the package.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repos with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* 2fa37f6 (submodule) sub0-2
* a9dbc1e sub0-1
| * 68c197d (mybranch) branch2
| * 3174063 branch1
|/
* a238b1d (HEAD -> master) initial
$ git -C support/testing/tests/download/git-remote/refs-sub1.git \
log --all --decorate --graph --oneline --decorate
* 1df823c (submodule) sub1-2
* f8001e5 sub1-1
* cb545fa (HEAD) initial
$ git -C support/testing/tests/download/git-remote/refs-sub2.git \
log --all --decorate --graph --oneline --decorate
* e83f6f8 (submodule) sub2-2
* cd4d62f sub2-1
* 32d61ba (HEAD) initial
In this layout:
- sub0-1 adds refs-sub1.git as submodule to refs.git pointing to sub1-1
commit;
- sub1-1 adds refs-sub2.git as submodule to refs-sub1.git pointing to
sub2-1 commit.
Changes v4 -> v5:
- no changes
Changes v3 -> v4:
- add hash for submodule enabled tarball;
Changes v2 -> v3:
- complete rewrite using a static repo;
Changes v1 -> v2:
- rewrite using new git test infra;
- this patch is part of series 1/3 of a new version of
http://patchwork.ozlabs.org/patch/690097/
---
.../git-submodule-disabled.hash | 2 ++
.../git-submodule-disabled.mk | 11 +++++++
.../git-submodule-enabled.hash | 4 +++
.../git-submodule-enabled.mk | 28 ++++++++++++++++++
.../git-remote/refs-sub1.git/.gitattributes | 1 +
.../download/git-remote/refs-sub1.git/HEAD | 1 +
.../download/git-remote/refs-sub1.git/config | 4 +++
.../07/cabc655213bdf7087d8dd50fda95124e935570 | Bin 0 -> 22 bytes
.../1d/f823cb8a6d1866148ae50a8009762a9c4c777f | Bin 0 -> 180 bytes
.../7d/52f458bdb0f9f5a4beb46fa82824421b8b988e | Bin 0 -> 65 bytes
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a | Bin 0 -> 122 bytes
.../bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23 | Bin 0 -> 22 bytes
.../cb/545facf77bbc5f24f95b6d503c338d10b7b717 | Bin 0 -> 151 bytes
.../dd/130f6f4629514adaf2e03407f3ed9344eb6cd6 | Bin 0 -> 122 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../f8/001e5780100962a5e30a25cbc4c9609cfc7bf3 | Bin 0 -> 184 bytes
.../refs-sub1.git/refs/heads/submodule | 1 +
.../git-remote/refs-sub2.git/.gitattributes | 1 +
.../download/git-remote/refs-sub2.git/HEAD | 1 +
.../download/git-remote/refs-sub2.git/config | 4 +++
.../0a/846af45c3e455789435f49f80d70e86b65b9d7 | Bin 0 -> 22 bytes
.../0b/32ebd8fc52cec991f18c94be980e85a8341585 | Bin 0 -> 22 bytes
.../32/d61bae693af7879da63b89a60d3ae67f851e56 | Bin 0 -> 151 bytes
.../3e/9b0a5198c64cea9c00f820433411e3b4d50c1c | Bin 0 -> 48 bytes
.../85/8f666af75b7c0dfba6b8be7eac5f196e7a1221 | Bin 0 -> 49 bytes
.../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes
.../cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb | Bin 0 -> 184 bytes
.../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes
.../e8/3f6f805bd016b90acafc8702c52d778eb57310 | Bin 0 -> 182 bytes
.../refs-sub2.git/refs/heads/submodule | 1 +
.../0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343 | Bin 0 -> 22 bytes
.../25/59d83bfe937fc0412d96ed664663c9e8a99055 | Bin 0 -> 22 bytes
.../2f/a37f6885d7eb746df75eccaddbacf3ac82799d | Bin 0 -> 180 bytes
.../8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7 | Bin 0 -> 65 bytes
.../a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f | Bin 0 -> 183 bytes
.../d4/e2941d18a63535400476324ddeb7f40164be41 | Bin 0 -> 121 bytes
.../fe/74231105841041d5f441e70399d37f0e600aa5 | Bin 0 -> 121 bytes
.../git-remote/repo.git/refs/heads/submodule | 1 +
support/testing/tests/download/test_git.py | 2 ++
40 files changed, 62 insertions(+)
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.mk
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/config
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/07/cabc655213bdf7087d8dd50fda95124e935570
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/1d/f823cb8a6d1866148ae50a8009762a9c4c777f
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/7d/52f458bdb0f9f5a4beb46fa82824421b8b988e
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/cb/545facf77bbc5f24f95b6d503c338d10b7b717
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/dd/130f6f4629514adaf2e03407f3ed9344eb6cd6
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/objects/f8/001e5780100962a5e30a25cbc4c9609cfc7bf3
create mode 100644 support/testing/tests/download/git-remote/refs-sub1.git/refs/heads/submodule
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/.gitattributes
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/HEAD
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/config
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/0a/846af45c3e455789435f49f80d70e86b65b9d7
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/0b/32ebd8fc52cec991f18c94be980e85a8341585
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/32/d61bae693af7879da63b89a60d3ae67f851e56
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/3e/9b0a5198c64cea9c00f820433411e3b4d50c1c
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/85/8f666af75b7c0dfba6b8be7eac5f196e7a1221
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/objects/e8/3f6f805bd016b90acafc8702c52d778eb57310
create mode 100644 support/testing/tests/download/git-remote/refs-sub2.git/refs/heads/submodule
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/25/59d83bfe937fc0412d96ed664663c9e8a99055
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/2f/a37f6885d7eb746df75eccaddbacf3ac82799d
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/d4/e2941d18a63535400476324ddeb7f40164be41
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/fe/74231105841041d5f441e70399d37f0e600aa5
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/submodule
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash
new file mode 100644
index 0000000000..3bd0a44693
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash
@@ -0,0 +1,2 @@
+sha256 f9d46ff8a1a344c6c31fa4211220f3085c446abd31626232540703158276f22c git-submodule-disabled-a9dbc1e23c45e8e1b88c0448763f54d714eb6f8f.tar.gz
+sha256 ba8b6ddc4726bfb6a05045ebfd8c43263c968ad1bc601bd46a25bc055008eddc file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.mk b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.mk
new file mode 100644
index 0000000000..7a35b3b51a
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-submodule-disabled
+#
+################################################################################
+
+GIT_SUBMODULE_DISABLED_VERSION = a9dbc1e23c45e8e1b88c0448763f54d714eb6f8f
+GIT_SUBMODULE_DISABLED_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SUBMODULE_DISABLED_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash
new file mode 100644
index 0000000000..089eed2365
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash
@@ -0,0 +1,4 @@
+sha256 139a34c3c844c844dee74b6746418cfa75fbcc4205794ac8c0b3cd7d55a76792 git-submodule-enabled-a9dbc1e23c45e8e1b88c0448763f54d714eb6f8f.tar.gz
+sha256 ba8b6ddc4726bfb6a05045ebfd8c43263c968ad1bc601bd46a25bc055008eddc file
+sha256 5219dcb50c7753bcdd72bc68a7b48af33ac2e42c5c61de78a9dd81589c4d50b6 refs-sub1/file
+sha256 b93c1fa9014d25a17fee36771d26ae023f043da656315ffe8947c30ad1ba141f refs-sub1/refs-sub2/file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.mk b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.mk
new file mode 100644
index 0000000000..019dd076e3
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.mk
@@ -0,0 +1,28 @@
+################################################################################
+#
+# git-submodule-enabled
+#
+################################################################################
+
+GIT_SUBMODULE_ENABLED_VERSION = a9dbc1e23c45e8e1b88c0448763f54d714eb6f8f
+GIT_SUBMODULE_ENABLED_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SUBMODULE_ENABLED_GIT_SUBMODULES = YES
+GIT_SUBMODULE_ENABLED_LICENSE_FILES = \
+ file \
+ refs-sub1/file \
+ refs-sub1/refs-sub2/file
+
+# Some versions of git client fill the .git file for the second level submodule
+# using the absolute path to the main .git directory, which in the case of the
+# buildroot download helper is always different since it uses a temporary
+# directory. This version of git have the issue:
+# - 2.7.4 included in Ubuntu 16.04;
+# The same does not occur using newer versions of git:
+# - 2.11.0 included in Debian 9;
+# - 2.14.2 latest at the time of writing;
+# In order to allow running this test case locally in many systems, do not check
+# for the hash of the tarball, but instead check the hash of each meaningful
+# file included in the tarball.
+BR_NO_CHECK_HASH_FOR += $(GIT_SUBMODULE_ENABLED_SOURCE)
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/.gitattributes b/support/testing/tests/download/git-remote/refs-sub1.git/.gitattributes
new file mode 100644
index 0000000000..eb50c64a21
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub1.git/.gitattributes
@@ -0,0 +1 @@
+objects/*/* binary
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/HEAD b/support/testing/tests/download/git-remote/refs-sub1.git/HEAD
new file mode 100644
index 0000000000..4e92c1f200
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub1.git/HEAD
@@ -0,0 +1 @@
+cb545facf77bbc5f24f95b6d503c338d10b7b717
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/config b/support/testing/tests/download/git-remote/refs-sub1.git/config
new file mode 100644
index 0000000000..07d359d07c
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub1.git/config
@@ -0,0 +1,4 @@
+[core]
+ repositoryformatversion = 0
+ filemode = true
+ bare = true
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/07/cabc655213bdf7087d8dd50fda95124e935570 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/07/cabc655213bdf7087d8dd50fda95124e935570
new file mode 100644
index 0000000000000000000000000000000000000000..cbf5085af8cdca19c43017c92369ddc8002f1a3d
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXfis>UJ21Qop+W=aH2UY+8
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/1d/f823cb8a6d1866148ae50a8009762a9c4c777f b/support/testing/tests/download/git-remote/refs-sub1.git/objects/1d/f823cb8a6d1866148ae50a8009762a9c4c777f
new file mode 100644
index 0000000000000000000000000000000000000000..04055b899ecbdc3bceb2d7936757f360e90544a5
GIT binary patch
literal 180
zcmV;l089UP0j-Wf4#FT106p^xK2WpJg47t}2fXMDb{9~h6hrag?``72yYV{7OomI&
zsmWfORjVOLR6Sr+R$C(U-VSb1q?wp3@(?`O-VrJk53FHoGCBYi%*KHN7_CMVmVh>k
zG4N<%<j5O}y})O?mMTw)vHBv9SX-L7%(o<W^#j-W3lEo!>AvQY_fi>fV64+lcAx>o
iLYL_l{*)8*lT at dLIH$`Ld{<)Z=(YXjRlEVZ`fTHP_gG8-
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/7d/52f458bdb0f9f5a4beb46fa82824421b8b988e b/support/testing/tests/download/git-remote/refs-sub1.git/objects/7d/52f458bdb0f9f5a4beb46fa82824421b8b988e
new file mode 100644
index 0000000000000000000000000000000000000000..2b78a01965068cc6f7c89402eea4e7f9a03ba807
GIT binary patch
literal 65
zcmV-H0KWft0ZYosPf{>5V~8#;P0GzrDa}b$P%27IE7k?_jg(@!I13U>G8AkT;9^{y
XrA0YF0X;o^xR_pgW(gMne83k<GdCCZ
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab b/support/testing/tests/download/git-remote/refs-sub1.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
new file mode 100644
index 0000000000000000000000000000000000000000..9db72668cf9374b0b85a25a19f30084fd460072d
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5wZcw)UHz=nDZYB`lCt5lYr9_mN?Omvz#_so
F4FHVv5b^*3
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a b/support/testing/tests/download/git-remote/refs-sub1.git/objects/a1/30af1626bbadd64841b2cbcb6ac4ed0638ba1a
new file mode 100644
index 0000000000000000000000000000000000000000..f49d53326e0b22b95fd16670f0a25e3cd84718ae
GIT binary patch
literal 122
zcmV-=0EPc}0V^p=O;s>7uwXDWFfcPQQP4}zEXmDJDa}bOW~dGN60vu~&#z1NZOLDu
zq2eUnJ);k*EG;u9m4W@#p41 at Wz27-%d$01}nkwWsIkdpg%m4@!ic-^xb&E at rj2O=P
cCjD3J+P-ChfArrU3~L^=ueW&(03qxzkxyhgmjD0&
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/bc/c81ba6e5bd8bf52c95688ae8d3e697e131fa23
new file mode 100644
index 0000000000000000000000000000000000000000..7d2a9d55c1a92dfa2122308cc05a4a5b96d70d8f
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXfis=(321RD(TL4;&2UP$7
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/cb/545facf77bbc5f24f95b6d503c338d10b7b717 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/cb/545facf77bbc5f24f95b6d503c338d10b7b717
new file mode 100644
index 0000000000000000000000000000000000000000..dc7de29459f4156b999f14fdeb6cb58b58c21a19
GIT binary patch
literal 151
zcmV;I0BHYs0j<ux4Z<)GKw)Z6aSJ3mJF*iagjj$ECOH3v6LBmVQ?NaeDCiOGS3Esm
za^?nVoOZ3o2-YSQhbX<1I$5VxB4v{?VVE!kLL-BfBRN6Ob}JPw?5TzVXR3`?Umkac
zL;YZ%zj&H6@zVQ}k03QpsEsrT9V$ez={C{gpK^j%ZcK|1SDx1xx{^}oUi)8W#T!=;
FU>B}CKEMC~
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/dd/130f6f4629514adaf2e03407f3ed9344eb6cd6 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/dd/130f6f4629514adaf2e03407f3ed9344eb6cd6
new file mode 100644
index 0000000000000000000000000000000000000000..37037d3920820ae2dab7e40d0c1df612af6f7a94
GIT binary patch
literal 122
zcmV-=0EPc}0V^p=O;s>7uwXDWFfcPQQP4}zEXmDJDa}bOW~dGN60vu~&#z1NZOLDu
zq2eUnJ);k*EG;u9m0`~b>19v%c7N5Gn$h**^0Vm=4Sy*cni&9rLQ!g3v2Jl`k`cpM
c-=zPFUE8-T at Q?ocgJI2s_VqTe0l at My>J95W4gdfE
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
new file mode 100644
index 0000000000000000000000000000000000000000..df2037a2d2d30afb20e31cd558c92c2edabe7cf6
GIT binary patch
literal 23
fcmb<m^geacKWIb3i8H5lPWqgA$i$$b%kl;Qg;xrl
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/objects/f8/001e5780100962a5e30a25cbc4c9609cfc7bf3 b/support/testing/tests/download/git-remote/refs-sub1.git/objects/f8/001e5780100962a5e30a25cbc4c9609cfc7bf3
new file mode 100644
index 0000000000000000000000000000000000000000..d824d3c17c1cc1c13a18eb1d2c0a7e59f66ba08b
GIT binary patch
literal 184
zcmV;p07w6L0j<tGP6IIzKw)a1!WY<R?D5YoLWm2{Bq#8AJi!95k>eEHo=7RuBigTc
zdbzh&r$+0a<|suo8w3gBg5A7viIdm@=pYIf-q5*1A#+gQX_S5HOx}4(Sz;v0UaXVN
zQ(zC65mz=qk)nwzUFSz1`dD)s-1U1Jv+lWHKC6BicRaKQzMWgDUx(b=t2Q21@L>hi
m3s?Yk)7$hc|CCdgE1AV8b*ksR at X|#V`g#7#tNI184{X$IS6W&C
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub1.git/refs/heads/submodule b/support/testing/tests/download/git-remote/refs-sub1.git/refs/heads/submodule
new file mode 100644
index 0000000000..bf97dc4185
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub1.git/refs/heads/submodule
@@ -0,0 +1 @@
+1df823cb8a6d1866148ae50a8009762a9c4c777f
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/.gitattributes b/support/testing/tests/download/git-remote/refs-sub2.git/.gitattributes
new file mode 100644
index 0000000000..eb50c64a21
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub2.git/.gitattributes
@@ -0,0 +1 @@
+objects/*/* binary
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/HEAD b/support/testing/tests/download/git-remote/refs-sub2.git/HEAD
new file mode 100644
index 0000000000..bbf19a4bb4
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub2.git/HEAD
@@ -0,0 +1 @@
+32d61bae693af7879da63b89a60d3ae67f851e56
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/config b/support/testing/tests/download/git-remote/refs-sub2.git/config
new file mode 100644
index 0000000000..07d359d07c
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub2.git/config
@@ -0,0 +1,4 @@
+[core]
+ repositoryformatversion = 0
+ filemode = true
+ bare = true
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/0a/846af45c3e455789435f49f80d70e86b65b9d7 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/0a/846af45c3e455789435f49f80d70e86b65b9d7
new file mode 100644
index 0000000000000000000000000000000000000000..aaa29dbd8a5a74a24f51a8522b695a7474349f3c
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXflIasB21Rz}+W=aJ2Uq|A
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/0b/32ebd8fc52cec991f18c94be980e85a8341585 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/0b/32ebd8fc52cec991f18c94be980e85a8341585
new file mode 100644
index 0000000000000000000000000000000000000000..50caf8b6282d2287878d1b3481c14431c2a557af
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXflIbHR21QQhI{;dM2U!3B
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/32/d61bae693af7879da63b89a60d3ae67f851e56 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/32/d61bae693af7879da63b89a60d3ae67f851e56
new file mode 100644
index 0000000000000000000000000000000000000000..a8887b1e611d8f76d7cfe6ab0bb693fdebc2ef19
GIT binary patch
literal 151
zcmV;I0BHYs0j<ux4Z<)GKw)Z6aSJ3mJF*iagjj$ECOH3v6LBmVQ?NaeDCiOGS3Esm
za^?nVoOZ3o2-YSQhbX<1I$5VxB4v{?VVE!kLL-BfBRN6Ob}JPw?5TzVXR3`?Umkac
zL;YZ%zj&H6 at zVQ}k03QpsEr&gbf^%;rrSh|f6577xiKw5TzOt&=t@eRd+mRj6>nh7
FU at NX0KJNek
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/3e/9b0a5198c64cea9c00f820433411e3b4d50c1c b/support/testing/tests/download/git-remote/refs-sub2.git/objects/3e/9b0a5198c64cea9c00f820433411e3b4d50c1c
new file mode 100644
index 0000000000000000000000000000000000000000..b1fb722c53d827847c6d13029e601a5434b84ac4
GIT binary patch
literal 48
zcmV-00MGw;0V^p=O;s>9VlXr?Ff%bxNXyJgW#DSb`VwR38s6y~@A-qb;6--o&g%g8
GZVq_HQWiM?
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/85/8f666af75b7c0dfba6b8be7eac5f196e7a1221 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/85/8f666af75b7c0dfba6b8be7eac5f196e7a1221
new file mode 100644
index 0000000000000000000000000000000000000000..10f24a087ff08320b8fc5922db728472bd276e9a
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5^?_Mc?PLE7J0}a<fAhI%-n_oFwKGGTLGrBt
FHvqLX67T>3
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab b/support/testing/tests/download/git-remote/refs-sub2.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab
new file mode 100644
index 0000000000000000000000000000000000000000..9db72668cf9374b0b85a25a19f30084fd460072d
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5wZcw)UHz=nDZYB`lCt5lYr9_mN?Omvz#_so
F4FHVv5b^*3
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb b/support/testing/tests/download/git-remote/refs-sub2.git/objects/cd/4d62ff218ab7b4a04f5bfdf800ace087af3ceb
new file mode 100644
index 0000000000000000000000000000000000000000..35d253f0d15b31ea0bc79ed1a287819deed9987d
GIT binary patch
literal 184
zcmV;p07w6L0j<tGZUZ3@hGEy7f(xuQEI$j8W%&YWk`oNWU?YdcVsVk%XQLG9qqL9o
zeR`?YT87lKJ&evQHHQdjRq*5tF$Mz2eg>npQHpgm$pR at 75A3{t50g_7*#(Vx at k>bP
zbo2ooB)#*Jy;W?Tz~{KNF5e1aPffmJA7!Ps-HSZ+A3WAyyj&}mk0Z5umdb($>r7bW
m1T%oxb&nqWPj_N_#(7dgyqD|BX=*u7@$=;`t>O#gcx)*r3s*J(
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5
new file mode 100644
index 0000000000000000000000000000000000000000..df2037a2d2d30afb20e31cd558c92c2edabe7cf6
GIT binary patch
literal 23
fcmb<m^geacKWIb3i8H5lPWqgA$i$$b%kl;Qg;xrl
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/objects/e8/3f6f805bd016b90acafc8702c52d778eb57310 b/support/testing/tests/download/git-remote/refs-sub2.git/objects/e8/3f6f805bd016b90acafc8702c52d778eb57310
new file mode 100644
index 0000000000000000000000000000000000000000..05c04d2887e9354ec43ad69cef20ed4e7166013d
GIT binary patch
literal 182
zcmV;n07?IN0j<tKPD3FOfMM61f(yK6pl<+;F<!upo?r%MT2gr!N*CVVCN8>fy8RY^
z{@h!u)9yV!%^_kFCJiBwM&y{GC{y6X5`{9VdkzxGdGE{*4cVs6Wi7$0dY6bJuLP?i
zYtaNiSs+EyeHJ!!na4itXU#N9w;wcS-Eu#ls(l^*xNA3jKQyY}yWHEGbqEPCphLTW
k2QXK?PEUDI&YUmo7jMj0J#12zuDbB=%RR5=4+?N at DHiEdA^-pY
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/refs-sub2.git/refs/heads/submodule b/support/testing/tests/download/git-remote/refs-sub2.git/refs/heads/submodule
new file mode 100644
index 0000000000..39dc53f019
--- /dev/null
+++ b/support/testing/tests/download/git-remote/refs-sub2.git/refs/heads/submodule
@@ -0,0 +1 @@
+e83f6f805bd016b90acafc8702c52d778eb57310
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343 b/support/testing/tests/download/git-remote/repo.git/objects/0b/d8ceb961c3b2b210f64a67d57f4b5cd669d343
new file mode 100644
index 0000000000000000000000000000000000000000..ef7fbd4154aea755d8ee2684e420700467e9245c
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXfg6R_`28I93HvwH&2ulC}
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/25/59d83bfe937fc0412d96ed664663c9e8a99055 b/support/testing/tests/download/git-remote/repo.git/objects/25/59d83bfe937fc0412d96ed664663c9e8a99055
new file mode 100644
index 0000000000000000000000000000000000000000..cbf60fc7298ec12b3d7cd388fecac606ca31536e
GIT binary patch
literal 22
dcmb<m^geacKgb|mL(kXfg6ShB21Q2ZTL4;A2TcF~
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/2f/a37f6885d7eb746df75eccaddbacf3ac82799d b/support/testing/tests/download/git-remote/repo.git/objects/2f/a37f6885d7eb746df75eccaddbacf3ac82799d
new file mode 100644
index 0000000000000000000000000000000000000000..2dd14580e666d35483727598f006148f3eeb03be
GIT binary patch
literal 180
zcmV;l089UP0j-Wp4#FT509|tmF3{!yj~0zFUcil>fX_pTQVgXFZ*LP9?v2|?W-`9y
zoEleBZ(1ed0ZDFA1Yzu;w9*KG<DevlKyHi(Vn9bI*wa>;8n at fvydbGOYBD68F&+ at j
z>O_rU6i9A}CNjIUTdDk<yse?|ldUaHzRdT;_w at t&{DsGBw&~IPk`G*HFrddZ%R7(&
iY^CdTqd(=?yg2A2W0!QDD0C&p&b@ZOys|fCz-zg3O;?Bj
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7 b/support/testing/tests/download/git-remote/repo.git/objects/8b/8a7e885a041da72b1ee9a47c5b9300b172a9e7
new file mode 100644
index 0000000000000000000000000000000000000000..70cf1795f3210f81951d0995823bb958867ba710
GIT binary patch
literal 65
zcmb<m^geacKghs1xkFFS_taT`&9k1N0a{+Z+Sgb=@lH+JX~8YzZ6uN-`XDK0&Qe!P
V2S!`l+8s0Es?s>c8ET5e-2otN7bXAz
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f b/support/testing/tests/download/git-remote/repo.git/objects/a9/dbc1e23c45e8e1b88c0448763f54d714eb6f8f
new file mode 100644
index 0000000000000000000000000000000000000000..2cbdce1f95493b284df1f8ca3ca953a5fb813072
GIT binary patch
literal 183
zcmV;o07(CM0j<t4ZUZ3@K+)8kf(xuQ14Ce~D9Q!g#3z_#z(F=kgmK~9XC)Wzt+cQB
z`mxnohPKANjh>Ru$&;~&-~u5KipxZpJlJ^_>ocTx0A{Ctxu<n#Ha09MIZ7~2$V;%q
z*+yrKCG>Htpv)kOZ{w$R{aqq=Y5Kr at lr^^Nx#*Anz+HXf*Qs*3>|(2*8XW|19*ox$
l7yz~DHa*fm<<#}Iz=TR2%V|wAwVbErzJKLa{Q<P{YY&#BUEBZw
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/d4/e2941d18a63535400476324ddeb7f40164be41 b/support/testing/tests/download/git-remote/repo.git/objects/d4/e2941d18a63535400476324ddeb7f40164be41
new file mode 100644
index 0000000000000000000000000000000000000000..4d82169300d0d8a2e31ced481501bff6647ef6b7
GIT binary patch
literal 121
zcmV-<0EYi~0V^p=O;s>7uwXDWFfcPQQP4}zEXmDJDa}bOX6Wv!>xg2JU9K(na!F0}
zWQL7JE1yG^rDf)%GN?x0u>Lo>{(z(Iw6|$)$tPc|oDgbgW&i{VMX71Uy2YhQh73O#
b<iZ;SIFpt>=2AU<<YdB}Kh>WBpF}NR{H!<P
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/fe/74231105841041d5f441e70399d37f0e600aa5 b/support/testing/tests/download/git-remote/repo.git/objects/fe/74231105841041d5f441e70399d37f0e600aa5
new file mode 100644
index 0000000000000000000000000000000000000000..6d0ae484c3333d6b18cf726d5bb468669f9d16c1
GIT binary patch
literal 121
zcmV-<0EYi~0V^p=O;s>7uwXDWFfcPQQP4}zEXmDJDa}bOX6Wv!>xg2JU9K(na!F0}
zWQL7JE1yG^rDf)%GH~BGw=?nZrcDChywb1Md&gYMyzFdfW&i{VMX71Uy2YhQh73O#
b<iZ;SIFpt>=2AU<<YdB}Kh>WBwy!O?8)rGE
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/heads/submodule b/support/testing/tests/download/git-remote/repo.git/refs/heads/submodule
new file mode 100644
index 0000000000..e9f4f32c30
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/heads/submodule
@@ -0,0 +1 @@
+2fa37f6885d7eb746df75eccaddbacf3ac82799d
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index 279d0ed8eb..803a50d054 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -69,3 +69,5 @@ class TestGitRefs(GitTestBase):
self.check_download("git-partial-sha1-reachable-by-branch")
self.check_download("git-sha1-branch-head")
self.check_download("git-sha1-reachable-by-branch")
+ self.check_download("git-submodule-disabled")
+ self.check_download("git-submodule-enabled")
--
2.17.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 08/10] testing/tests/download: test git tag
2018-05-12 2:58 ` [Buildroot] [PATCH v5 00/10] tests for git download infra v5 Ricardo Martincoski
` (6 preceding siblings ...)
2018-05-12 2:58 ` [Buildroot] [PATCH v5 07/10] testing/tests/download: test git submodules Ricardo Martincoski
@ 2018-05-12 2:58 ` Ricardo Martincoski
2019-02-06 10:03 ` Arnout Vandecappelle
2018-05-12 2:58 ` [Buildroot] [PATCH v5 09/10] testing/tests/download: test git special ref Ricardo Martincoski
2018-05-12 2:58 ` [Buildroot] [PATCH v5 10/10] support/testing: test extra download with site method git Ricardo Martincoski
9 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2018-05-12 2:58 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Add a tag to the static repo and check on the git refs test case the
download of a git package:
- with the name of a tag as version;
- with the sha1 of a tag itself as version;
- with the partial sha1 of a tag itself as version;
- with the sha1 of a commit pointed by a tag as version;
- with the partial sha1 of a commit pointed by a tag as version;
- with the sha1 of a commit reachable only by a tag as version;
- with the partial sha1 of a commit reachable only by a tag as version.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repo with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* 2fa37f6 (submodule) sub0-2
* a9dbc1e sub0-1
| * 68c197d (mybranch) branch2
| * 3174063 branch1
|/
| * 516c9c5 (tag: mytag) tag2
| * 46bae5b tag1
|/
* a238b1d (HEAD -> master) initial
Changes v4 -> v5:
- no changes
Changes v1 -> v4:
- not included on v2 or v3;
- rewrite from v1 using new git test infra with static repo;
---
.../git-partial-sha1-reachable-by-tag.hash | 2 ++
.../git-partial-sha1-reachable-by-tag.mk | 11 +++++++++++
.../git-partial-sha1-tag-itself.hash | 2 ++
.../git-partial-sha1-tag-itself.mk | 11 +++++++++++
.../git-partial-sha1-tag-points-to.hash | 2 ++
.../git-partial-sha1-tag-points-to.mk | 11 +++++++++++
.../git-sha1-reachable-by-tag.hash | 2 ++
.../git-sha1-reachable-by-tag.mk | 11 +++++++++++
.../git-sha1-tag-itself/git-sha1-tag-itself.hash | 2 ++
.../git-sha1-tag-itself/git-sha1-tag-itself.mk | 11 +++++++++++
.../git-sha1-tag-points-to.hash | 2 ++
.../git-sha1-tag-points-to.mk | 11 +++++++++++
.../git-refs/package/git-tag/git-tag.hash | 2 ++
.../git-refs/package/git-tag/git-tag.mk | 11 +++++++++++
.../2b/0e0d98a49c97da6a618ab36337e2058eb733a2 | Bin 0 -> 137 bytes
.../46/bae5b639e5a18e2cc4dc508f080d566baeff59 | Bin 0 -> 182 bytes
.../51/6c9c5f64ec66534d4d069c2e408d9ae4dce023 | Bin 0 -> 182 bytes
.../92/ef85be57d627f280d8ce3724452ac21c9a6452 | Bin 0 -> 20 bytes
.../cf/0f4f85d7a1237e377a2d25b996518a877ea001 | Bin 0 -> 49 bytes
.../e2/2695cbf976fed1f543ad7486a531c0af473482 | Bin 0 -> 49 bytes
.../f6/476b879f65e956d7dedd5b08736369e9a24acc | Bin 0 -> 20 bytes
.../download/git-remote/repo.git/refs/tags/mytag | 1 +
support/testing/tests/download/test_git.py | 7 +++++++
23 files changed, 99 insertions(+)
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.mk
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.mk
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/2b/0e0d98a49c97da6a618ab36337e2058eb733a2
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/46/bae5b639e5a18e2cc4dc508f080d566baeff59
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/51/6c9c5f64ec66534d4d069c2e408d9ae4dce023
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/92/ef85be57d627f280d8ce3724452ac21c9a6452
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/cf/0f4f85d7a1237e377a2d25b996518a877ea001
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e2/2695cbf976fed1f543ad7486a531c0af473482
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/f6/476b879f65e956d7dedd5b08736369e9a24acc
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/tags/mytag
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.hash b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.hash
new file mode 100644
index 0000000000..7696086690
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.hash
@@ -0,0 +1,2 @@
+sha256 f2ef9772f13a9ef9a2c7cde0795e179defb12320d1747fc8d2408748ef5844c2 git-partial-sha1-reachable-by-tag-46bae5b639e5a18e2cc4.tar.gz
+sha256 2de87d77a2f226813f2d9bda906e970e4195605cdba6680443c0c04d89c532b6 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.mk b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.mk
new file mode 100644
index 0000000000..05aa659dd2
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-partial-sha1-reachable-by-tag
+#
+################################################################################
+
+GIT_PARTIAL_SHA1_REACHABLE_BY_TAG_VERSION = 46bae5b639e5a18e2cc4
+GIT_PARTIAL_SHA1_REACHABLE_BY_TAG_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_PARTIAL_SHA1_REACHABLE_BY_TAG_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.hash b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.hash
new file mode 100644
index 0000000000..e627caf91e
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.hash
@@ -0,0 +1,2 @@
+sha256 721143b41b8e56cfd9025833f1602e900a490627db2504e5b4907baa23e0019e git-partial-sha1-tag-itself-2b0e0d98a49c97da6a61.tar.gz
+sha256 6de8772a0a58fa62e2b8c58d4dae55c9db7534ad3b3918ecc849a9008d58f081 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.mk b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.mk
new file mode 100644
index 0000000000..515492397e
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-partial-sha1-tag-itself
+#
+################################################################################
+
+GIT_PARTIAL_SHA1_TAG_ITSELF_VERSION = 2b0e0d98a49c97da6a61
+GIT_PARTIAL_SHA1_TAG_ITSELF_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_PARTIAL_SHA1_TAG_ITSELF_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.hash b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.hash
new file mode 100644
index 0000000000..f957a0e23c
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.hash
@@ -0,0 +1,2 @@
+sha256 0fbf7fe935f962ceaafcf8e0ffd25dd2a83752c3f0fd055a942a53f8c9235fa7 git-partial-sha1-tag-points-to-516c9c5f64ec66534d4d.tar.gz
+sha256 6de8772a0a58fa62e2b8c58d4dae55c9db7534ad3b3918ecc849a9008d58f081 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.mk b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.mk
new file mode 100644
index 0000000000..c810e81175
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-partial-sha1-tag-points-to
+#
+################################################################################
+
+GIT_PARTIAL_SHA1_TAG_POINTS_TO_VERSION = 516c9c5f64ec66534d4d
+GIT_PARTIAL_SHA1_TAG_POINTS_TO_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_PARTIAL_SHA1_TAG_POINTS_TO_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.hash b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.hash
new file mode 100644
index 0000000000..0eb0ca0917
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.hash
@@ -0,0 +1,2 @@
+sha256 9b20256a3058221a8e91031f11700d9945ea84e8f328fa8e42e1cb9f7a30e3b2 git-sha1-reachable-by-tag-46bae5b639e5a18e2cc4dc508f080d566baeff59.tar.gz
+sha256 2de87d77a2f226813f2d9bda906e970e4195605cdba6680443c0c04d89c532b6 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.mk b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.mk
new file mode 100644
index 0000000000..b5fde7b586
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-sha1-reachable-by-tag
+#
+################################################################################
+
+GIT_SHA1_REACHABLE_BY_TAG_VERSION = 46bae5b639e5a18e2cc4dc508f080d566baeff59
+GIT_SHA1_REACHABLE_BY_TAG_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SHA1_REACHABLE_BY_TAG_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.hash b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.hash
new file mode 100644
index 0000000000..48c1348538
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.hash
@@ -0,0 +1,2 @@
+sha256 7d301c1a1054d6aee49193ca9e938f4da561ff73fb01719662865aa38bdc4361 git-sha1-tag-itself-2b0e0d98a49c97da6a618ab36337e2058eb733a2.tar.gz
+sha256 6de8772a0a58fa62e2b8c58d4dae55c9db7534ad3b3918ecc849a9008d58f081 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.mk b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.mk
new file mode 100644
index 0000000000..8914496653
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-sha1-tag-itself
+#
+################################################################################
+
+GIT_SHA1_TAG_ITSELF_VERSION = 2b0e0d98a49c97da6a618ab36337e2058eb733a2
+GIT_SHA1_TAG_ITSELF_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SHA1_TAG_ITSELF_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.hash b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.hash
new file mode 100644
index 0000000000..3dcafc2094
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.hash
@@ -0,0 +1,2 @@
+sha256 c1f9f5734529a31140a71c031534460811f001b4db37d26833f386358150ab47 git-sha1-tag-points-to-516c9c5f64ec66534d4d069c2e408d9ae4dce023.tar.gz
+sha256 6de8772a0a58fa62e2b8c58d4dae55c9db7534ad3b3918ecc849a9008d58f081 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.mk b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.mk
new file mode 100644
index 0000000000..adaae7329c
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-sha1-tag-points-to
+#
+################################################################################
+
+GIT_SHA1_TAG_POINTS_TO_VERSION = 516c9c5f64ec66534d4d069c2e408d9ae4dce023
+GIT_SHA1_TAG_POINTS_TO_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SHA1_TAG_POINTS_TO_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.hash b/support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.hash
new file mode 100644
index 0000000000..1cd0b15f27
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.hash
@@ -0,0 +1,2 @@
+sha256 85dcb5bcf9bed496688d0eb01c7a3ce05c5b46b984cc1e9e76a6dbefd976e6b3 git-tag-mytag.tar.gz
+sha256 6de8772a0a58fa62e2b8c58d4dae55c9db7534ad3b3918ecc849a9008d58f081 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.mk b/support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.mk
new file mode 100644
index 0000000000..6960ceb2cb
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-tag
+#
+################################################################################
+
+GIT_TAG_VERSION = mytag
+GIT_TAG_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_TAG_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/2b/0e0d98a49c97da6a618ab36337e2058eb733a2 b/support/testing/tests/download/git-remote/repo.git/objects/2b/0e0d98a49c97da6a618ab36337e2058eb733a2
new file mode 100644
index 0000000000000000000000000000000000000000..39c913094a3f2d8d9d84cda13a89306bc577f982
GIT binary patch
literal 137
zcmV;40CxX)0Y#2W4gw(%1zmHBzJQY+enMi57jWYZuxSY at gF!ZPdoi<c^HQlr<@JzY
zHfmRI52g-gw&cm&t!A>;C>?b~>q)Xk7kywIiBT%i*Ea*zQgRo4G|D>mM~NF;Gldo_
rTte%4q564dIJQkr<uA{vg#4OD<}^;=!5b&h!5$T&5L at yK5lk}^QNuc<
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/46/bae5b639e5a18e2cc4dc508f080d566baeff59 b/support/testing/tests/download/git-remote/repo.git/objects/46/bae5b639e5a18e2cc4dc508f080d566baeff59
new file mode 100644
index 0000000000000000000000000000000000000000..3bf8865ea67386bd0fd88ab7222392d5b66b3607
GIT binary patch
literal 182
zcmV;n07?IN0j<t4ZUZ3 at KvCD6f(xw0FyLXMDDnl;#3z_#V2u)%2$RCMx1|*BEw``u
z`mxnohDM9K(Noefo9uG%LJSruArV`#ewpWkhrod-u`!^2xu<Ptwip(W93@yUjwGPt
zjL}<*4k!i(AhQw0f8*S`{wk5XH2ukalr6T)chT?tANTcv$F*|#*~eBtG<ZX^4{-2$
kGH8st>wS8p7v<F86_h%abxSg}oTugb_ROie0nJ8hV&q3%KL7v#
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/51/6c9c5f64ec66534d4d069c2e408d9ae4dce023 b/support/testing/tests/download/git-remote/repo.git/objects/51/6c9c5f64ec66534d4d069c2e408d9ae4dce023
new file mode 100644
index 0000000000000000000000000000000000000000..0bda23ab4d6487524fc254c54b673e53bfe69435
GIT binary patch
literal 182
zcmV;n07?IN0j<tUP6IIzMN!6Hg*VWVf8`HCQ8vJYtf0DFZmi%oa-0FXClLeYi0OIK
z)1}^8ozfcnGzYU}V|)TyOA5Yl)&-o!99;AWR+GZwf_2e|PaM2Y>HLZetG9^(b!0>2
zGJ%RkMU}xn=29RDyv_GM<Xa^ixyx4^v+mTdmnvVz4<6btp0<Yd<3PO~r3MueL>;VL
klvPUH^&UO>pYFuO%?NR>+s=9Er7Y{$^IuBEH_8ueWztGniU0rr
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/92/ef85be57d627f280d8ce3724452ac21c9a6452 b/support/testing/tests/download/git-remote/repo.git/objects/92/ef85be57d627f280d8ce3724452ac21c9a6452
new file mode 100644
index 0000000000000000000000000000000000000000..1f2474ba3209802c0f297f390eb517b4419621f9
GIT binary patch
literal 20
bcmb<m^geacKgb|e!_&w72@`|F8|EAUOXLRI
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/cf/0f4f85d7a1237e377a2d25b996518a877ea001 b/support/testing/tests/download/git-remote/repo.git/objects/cf/0f4f85d7a1237e377a2d25b996518a877ea001
new file mode 100644
index 0000000000000000000000000000000000000000..fd224552518d57b0b8b0a1d618eef507f798555f
GIT binary patch
literal 49
zcmV-10M7q-0V^p=O;s>9VlXr?Ff%bxNXyJgWtjB7bzk^3^-m2q&Y7#YY8{f9l at bI1
H7n%=ExH=M#
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/e2/2695cbf976fed1f543ad7486a531c0af473482 b/support/testing/tests/download/git-remote/repo.git/objects/e2/2695cbf976fed1f543ad7486a531c0af473482
new file mode 100644
index 0000000000000000000000000000000000000000..15d80c78fac8655d4dfe4b8b543076c8bd26408e
GIT binary patch
literal 49
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5^~bY$ckS%Irr!8kUB1||YqD{?-X|+&2Jxc;
FIRMOk6Sn{W
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/f6/476b879f65e956d7dedd5b08736369e9a24acc b/support/testing/tests/download/git-remote/repo.git/objects/f6/476b879f65e956d7dedd5b08736369e9a24acc
new file mode 100644
index 0000000000000000000000000000000000000000..ee01041e0abcefa3c783ef3fc91e31c33edfb7cf
GIT binary patch
literal 20
bcmb<m^geacKgb|e!_&w75fg*NJLX&fOWg+C
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/tags/mytag b/support/testing/tests/download/git-remote/repo.git/refs/tags/mytag
new file mode 100644
index 0000000000..8c09448ad2
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/tags/mytag
@@ -0,0 +1 @@
+2b0e0d98a49c97da6a618ab36337e2058eb733a2
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index 803a50d054..ce3655121c 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -67,7 +67,14 @@ class TestGitRefs(GitTestBase):
self.check_download("git-wrong-sha1")
self.check_download("git-partial-sha1-branch-head")
self.check_download("git-partial-sha1-reachable-by-branch")
+ self.check_download("git-partial-sha1-reachable-by-tag")
+ self.check_download("git-partial-sha1-tag-itself")
+ self.check_download("git-partial-sha1-tag-points-to")
self.check_download("git-sha1-branch-head")
self.check_download("git-sha1-reachable-by-branch")
+ self.check_download("git-sha1-reachable-by-tag")
+ self.check_download("git-sha1-tag-itself")
+ self.check_download("git-sha1-tag-points-to")
self.check_download("git-submodule-disabled")
self.check_download("git-submodule-enabled")
+ self.check_download("git-tag")
--
2.17.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 09/10] testing/tests/download: test git special ref
2018-05-12 2:58 ` [Buildroot] [PATCH v5 00/10] tests for git download infra v5 Ricardo Martincoski
` (7 preceding siblings ...)
2018-05-12 2:58 ` [Buildroot] [PATCH v5 08/10] testing/tests/download: test git tag Ricardo Martincoski
@ 2018-05-12 2:58 ` Ricardo Martincoski
2019-02-06 10:05 ` Arnout Vandecappelle
2018-05-12 2:58 ` [Buildroot] [PATCH v5 10/10] support/testing: test extra download with site method git Ricardo Martincoski
9 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2018-05-12 2:58 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Add a special ref to the static repo and check on the git refs test case
the download of a git package:
- with the name of a special ref as version;
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
Layout of static repo with the series applied until this patch:
$ git -C support/testing/tests/download/git-remote/repo.git \
log --all --decorate --graph --oneline --decorate
* 2fa37f6 (submodule) sub0-2
* a9dbc1e sub0-1
| * b72ff60 (refs/changes/01/1/2) specialref
|/
| * 68c197d (mybranch) branch2
| * 3174063 branch1
|/
| * 516c9c5 (tag: mytag) tag2
| * 46bae5b tag1
|/
* a238b1d (HEAD -> master) initial
Changes v4 -> v5:
- no changes
Changes v1 -> v4:
- not included on v2 or v3;
- rewrite from v1 using new git test infra with static repo;
---
.../package/git-special-ref/git-special-ref.hash | 2 ++
.../package/git-special-ref/git-special-ref.mk | 11 +++++++++++
.../17/c409592968d17600ff9912e96fdd461bb72e74 | Bin 0 -> 48 bytes
.../53/122a7b0454b33c6d9e159c10657173db77899e | Bin 0 -> 27 bytes
.../b7/2ff6078f62522a87f5cae5e9f34dedf5ec3885 | Bin 0 -> 186 bytes
.../git-remote/repo.git/refs/changes/01/1/2 | 1 +
support/testing/tests/download/test_git.py | 1 +
7 files changed, 15 insertions(+)
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.hash
create mode 100644 support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.mk
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/17/c409592968d17600ff9912e96fdd461bb72e74
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/53/122a7b0454b33c6d9e159c10657173db77899e
create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/b7/2ff6078f62522a87f5cae5e9f34dedf5ec3885
create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/changes/01/1/2
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.hash b/support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.hash
new file mode 100644
index 0000000000..13e21aaeca
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.hash
@@ -0,0 +1,2 @@
+sha256 c639b15260d098d94b8802cae812453c9f878ffd5f80cca6d87c58c01eed5c5a git-special-ref-refs_changes_01_1_2.tar.gz
+sha256 b252b594be4679764e2e06cfea77a3e50638dd445af9b472b0f298c43b134518 file
diff --git a/support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.mk b/support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.mk
new file mode 100644
index 0000000000..e7561bafca
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-refs/package/git-special-ref/git-special-ref.mk
@@ -0,0 +1,11 @@
+################################################################################
+#
+# git-special-ref
+#
+################################################################################
+
+GIT_SPECIAL_REF_VERSION = refs/changes/01/1/2
+GIT_SPECIAL_REF_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+GIT_SPECIAL_REF_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/17/c409592968d17600ff9912e96fdd461bb72e74 b/support/testing/tests/download/git-remote/repo.git/objects/17/c409592968d17600ff9912e96fdd461bb72e74
new file mode 100644
index 0000000000000000000000000000000000000000..8c19de6455168ac65c81d70219e01eb2431c069e
GIT binary patch
literal 48
zcmb<m)YkO!4K*-JGB7bPFg6VIIDN)5l|$^(sx;1<Ig_pZOZptL5<kBEbgq)&X$db2
E0EbQzfdBvi
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/53/122a7b0454b33c6d9e159c10657173db77899e b/support/testing/tests/download/git-remote/repo.git/objects/53/122a7b0454b33c6d9e159c10657173db77899e
new file mode 100644
index 0000000000000000000000000000000000000000..d72d49eed56298f84aaf85967dbf051da94f3871
GIT binary patch
literal 27
icmb<m^geacKghr&MMKZS_ngleO)uZaObmAQY+e9~zX?PD
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/objects/b7/2ff6078f62522a87f5cae5e9f34dedf5ec3885 b/support/testing/tests/download/git-remote/repo.git/objects/b7/2ff6078f62522a87f5cae5e9f34dedf5ec3885
new file mode 100644
index 0000000000000000000000000000000000000000..6727b2e8083f2a31c48e849caa4cd772a6778e07
GIT binary patch
literal 186
zcmV;r07d_J0j<tKP6QzkKvCD6f(!Jd3TU8=F<!upo`9x`k<8Fc7+iRJJ8|LOxP6PC
zpL=U{YHR3W4iSwZJD`;e`N$YNKq*9MgnVIk9=9zR2~OS8kbUZ8Y}}Ab=4e)SfunV)
z*t{BJ-4TlpkqKC(&-u}ZeyceR?)o*2S@+!Um#Uw}51!gDzMNaC*HiB8S>p<UJc0{)
o0SllGJ*KDpQ%>D)l54sSDN4Q9^DeyfQkL!O at h`vX8x-PgzB#pAi~s-t
literal 0
HcmV?d00001
diff --git a/support/testing/tests/download/git-remote/repo.git/refs/changes/01/1/2 b/support/testing/tests/download/git-remote/repo.git/refs/changes/01/1/2
new file mode 100644
index 0000000000..7066394dd0
--- /dev/null
+++ b/support/testing/tests/download/git-remote/repo.git/refs/changes/01/1/2
@@ -0,0 +1 @@
+b72ff6078f62522a87f5cae5e9f34dedf5ec3885
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index ce3655121c..162c03623b 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -75,6 +75,7 @@ class TestGitRefs(GitTestBase):
self.check_download("git-sha1-reachable-by-tag")
self.check_download("git-sha1-tag-itself")
self.check_download("git-sha1-tag-points-to")
+ self.check_download("git-special-ref")
self.check_download("git-submodule-disabled")
self.check_download("git-submodule-enabled")
self.check_download("git-tag")
--
2.17.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 10/10] support/testing: test extra download with site method git
2018-05-12 2:58 ` [Buildroot] [PATCH v5 00/10] tests for git download infra v5 Ricardo Martincoski
` (8 preceding siblings ...)
2018-05-12 2:58 ` [Buildroot] [PATCH v5 09/10] testing/tests/download: test git special ref Ricardo Martincoski
@ 2018-05-12 2:58 ` Ricardo Martincoski
2019-02-06 10:34 ` Arnout Vandecappelle
9 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2018-05-12 2:58 UTC (permalink / raw)
To: buildroot
Add one test case to ensure the use of extra download works for git
packages:
- an extra download is correctly downloaded using wget after the main
download finished using the git backend;
- when the main download using the git backend fails, the build fails;
- when the extra download using wget fails, the build fails.
Reuse the commit in the static repo.
Add required infra and fixture:
- a HttpServer class, that starts a http server in the host machine to
emulate a remote http server under the control of the test;
- a file to be served by the http server;
- a br2-external (git-extra-download) with one package for each part of
the test case.
Since the HttpServer is similar to GitRemote, extract the commonalities
to a new base class called Server.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
---
Changes v4 -> v5:
- no changes
Changes v3 -> v4:
- new patch
I use docker only with the Buildroot image for the test infra.
Sorry, I give up for now on trying to run apache on the docker image.
As a midground solution between:
- adding more files to http://autobuild.buildroot.net/artefacts/ that
won't scale well if we need different files for different test cases;
- installing a full web server to the docker image auto-loading as root
when the container is started as user and allowing the test infra to
copy files to the served path as user;
and inspired by this article:
https://www.pcsuggest.com/best-lightweight-web-server-linux/
I created this patch using SimpleHTTPServer being started by the test
infra as a user.
I know it is weird to use pexpect to fork a server written in Python,
but it has these advantages:
- theoretically replacing SimpleHTTPServer with a similar solution (say
the busybox httpd) in the future would require only small changes to
the test infra code;
- SimpleHTTPServer is already on the docker image, busybox isn't, so no
need to change the Dockerfile, generate the image, publish it and
update the docker image tag on .gitlab-ci.yml;
- the log file generated by the server is handled in the same way as
other log files generated by the test infra;
---
.gitlab-ci.yml | 1 +
support/testing/infra/server.py | 39 +++++++++++++++++
| 0
| 1 +
| 6 +++
| 3 ++
| 12 ++++++
.../package/main-fails/main-fails.hash | 3 ++
.../package/main-fails/main-fails.mk | 12 ++++++
| 3 ++
| 12 ++++++
support/testing/tests/download/gitremote.py | 43 +++----------------
| 1 +
support/testing/tests/download/httpserver.py | 14 ++++++
support/testing/tests/download/test_git.py | 21 +++++++++
15 files changed, 135 insertions(+), 36 deletions(-)
create mode 100644 support/testing/infra/server.py
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/Config.in
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/external.desc
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/external.mk
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.hash
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.mk
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.hash
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.mk
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.hash
create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.mk
create mode 100644 support/testing/tests/download/http-server/extra
create mode 100644 support/testing/tests/download/httpserver.py
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 61ccfdfd2d..53f7c72ac3 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -263,6 +263,7 @@ tests.core.test_rootfs_overlay.TestRootfsOverlay: *runtime_test
tests.core.test_timezone.TestGlibcAllTimezone: *runtime_test
tests.core.test_timezone.TestGlibcNonDefaultLimitedTimezone: *runtime_test
tests.core.test_timezone.TestNoTimezone: *runtime_test
+tests.download.test_git.TestGitExtraDownload: *runtime_test
tests.download.test_git.TestGitHash: *runtime_test
tests.download.test_git.TestGitRefs: *runtime_test
tests.fs.test_ext.TestExt2: *runtime_test
diff --git a/support/testing/infra/server.py b/support/testing/infra/server.py
new file mode 100644
index 0000000000..0fbffdf846
--- /dev/null
+++ b/support/testing/infra/server.py
@@ -0,0 +1,39 @@
+# subprocess does not kill the child daemon when a test case fails by raising
+# an exception. So use pexpect instead.
+import pexpect
+
+import infra
+
+
+class Server(object):
+ def __init__(self, builddir, serveddir, logtofile, daemon_cmd, port_arg, port_initial, port_last, good_msg, bad_msg):
+ """
+ Start a local server.
+
+ In order to support test cases in parallel, select the port the
+ server will listen to in runtime. Since there is no reliable way
+ to allocate the port prior to starting the server (another
+ process in the host machine can use the port between it is
+ selected from a list and it is really allocated to the server)
+ try to start the server in a port and in the case it is already
+ in use, try the next one in the allowed range.
+ """
+ self.name = self.__class__.__name__.lower()
+ self.daemon = None
+ self.port = None
+ self.logfile = infra.open_log_file(builddir, self.name, logtofile)
+
+ for port in range(port_initial, port_last + 1):
+ cmd = daemon_cmd + [port_arg.format(port=port)]
+ self.logfile.write("> starting {} with 'cd {} && {}'\n".format(self.name, serveddir, " ".join(cmd)))
+ self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile, cwd=serveddir)
+ ret = self.daemon.expect(good_msg + bad_msg)
+ if ret < len(good_msg):
+ self.port = port
+ return
+ raise SystemError("Could not find a free port to run {}".format(self.name))
+
+ def stop(self):
+ if self.daemon is None:
+ return
+ self.daemon.terminate(force=True)
diff --git a/support/testing/tests/download/br2-external/git-extra-download/Config.in b/support/testing/tests/download/br2-external/git-extra-download/Config.in
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/support/testing/tests/download/br2-external/git-extra-download/external.desc b/support/testing/tests/download/br2-external/git-extra-download/external.desc
new file mode 100644
index 0000000000..6ebd5a534d
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/external.desc
@@ -0,0 +1 @@
+name: GIT_EXTRA_DOWNLOAD
diff --git a/support/testing/tests/download/br2-external/git-extra-download/external.mk b/support/testing/tests/download/br2-external/git-extra-download/external.mk
new file mode 100644
index 0000000000..c6080f571b
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/external.mk
@@ -0,0 +1,6 @@
+include $(sort $(wildcard $(BR2_EXTERNAL_GIT_EXTRA_DOWNLOAD_PATH)/package/*/*.mk))
+
+# Get the git server port number from the test infra
+GITREMOTE_PORT_NUMBER ?= 9418
+# Get the http server port number from the test infra
+HTTP_SERVER_PORT_NUMBER ?= 8000
diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.hash b/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.hash
new file mode 100644
index 0000000000..89b1b1f682
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.hash
@@ -0,0 +1,3 @@
+sha256 1e6bc73fabdcce8857361e36e3c812c4ee42d8ffa30d56492bc56f8fcad7eb90 extra-fails-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
+sha256 b9e68e1bea3e5b19ca6b2f98b73a54b73daafaa250484902e09982e07a12e733 notfound
+sha256 da68f54607d5f5644954096ce1597c006c5bb9f2497e07441bf064b81003ef8a file
diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.mk b/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.mk
new file mode 100644
index 0000000000..dcf3ebe7a6
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.mk
@@ -0,0 +1,12 @@
+################################################################################
+#
+# extra-fails
+#
+################################################################################
+
+EXTRA_FAILS_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+EXTRA_FAILS_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+EXTRA_FAILS_EXTRA_DOWNLOADS = http://localhost:$(HTTP_SERVER_PORT_NUMBER)/notfound
+EXTRA_FAILS_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.hash b/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.hash
new file mode 100644
index 0000000000..ad81243751
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.hash
@@ -0,0 +1,3 @@
+sha256 cd6851ef519a83345e4547f376b33d6bbd622d4ccbb234af9997c43854c602de main-fails-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
+sha256 b9e68e1bea3e5b19ca6b2f98b73a54b73daafaa250484902e09982e07a12e733 extra
+sha256 da68f54607d5f5644954096ce1597c006c5bb9f2497e07441bf064b81003ef8a file
diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.mk b/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.mk
new file mode 100644
index 0000000000..022bb37cbb
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.mk
@@ -0,0 +1,12 @@
+################################################################################
+#
+# main-fails
+#
+################################################################################
+
+MAIN_FAILS_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+MAIN_FAILS_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/notfound.git
+MAIN_FAILS_EXTRA_DOWNLOADS = http://localhost:$(HTTP_SERVER_PORT_NUMBER)/extra
+MAIN_FAILS_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.hash b/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.hash
new file mode 100644
index 0000000000..366940754b
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.hash
@@ -0,0 +1,3 @@
+sha256 737b4fd21506dbaa34cedc93c53c1ebb1f950db2c7644572bb006ae9297961da ok-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
+sha256 b9e68e1bea3e5b19ca6b2f98b73a54b73daafaa250484902e09982e07a12e733 extra
+sha256 da68f54607d5f5644954096ce1597c006c5bb9f2497e07441bf064b81003ef8a file
diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.mk b/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.mk
new file mode 100644
index 0000000000..ce102cb0de
--- /dev/null
+++ b/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.mk
@@ -0,0 +1,12 @@
+################################################################################
+#
+# ok
+#
+################################################################################
+
+OK_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
+OK_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
+OK_EXTRA_DOWNLOADS = http://localhost:$(HTTP_SERVER_PORT_NUMBER)/extra
+OK_LICENSE_FILES = file
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/download/gitremote.py b/support/testing/tests/download/gitremote.py
index 60bc49fbf8..9766b19ce0 100644
--- a/support/testing/tests/download/gitremote.py
+++ b/support/testing/tests/download/gitremote.py
@@ -1,44 +1,15 @@
-# subprocess does not kill the child daemon when a test case fails by raising
-# an exception. So use pexpect instead.
-import pexpect
-
-import infra
+from infra.server import Server
GIT_REMOTE_PORT_INITIAL = 9418
GIT_REMOTE_PORT_LAST = GIT_REMOTE_PORT_INITIAL + 99
-class GitRemote(object):
+class GitRemote(Server):
def __init__(self, builddir, serveddir, logtofile):
- """
- Start a local git server.
-
- In order to support test cases in parallel, select the port the
- server will listen to in runtime. Since there is no reliable way
- to allocate the port prior to starting the server (another
- process in the host machine can use the port between it is
- selected from a list and it is really allocated to the server)
- try to start the server in a port and in the case it is already
- in use, try the next one in the allowed range.
- """
- self.daemon = None
- self.port = None
- self.logfile = infra.open_log_file(builddir, "gitremote", logtofile)
-
daemon_cmd = ["git", "daemon", "--reuseaddr", "--verbose", "--listen=localhost", "--export-all",
"--base-path={}".format(serveddir)]
- for port in range(GIT_REMOTE_PORT_INITIAL, GIT_REMOTE_PORT_LAST + 1):
- cmd = daemon_cmd + ["--port={port}".format(port=port)]
- self.logfile.write("> starting git remote with '{}'\n".format(" ".join(cmd)))
- self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile)
- ret = self.daemon.expect(["Ready to rumble",
- "Address already in use"])
- if ret == 0:
- self.port = port
- return
- raise SystemError("Could not find a free port to run git remote")
-
- def stop(self):
- if self.daemon is None:
- return
- self.daemon.terminate(force=True)
+ super(GitRemote, self).__init__(
+ builddir, serveddir, logtofile,
+ daemon_cmd, "--port={port}", GIT_REMOTE_PORT_INITIAL, GIT_REMOTE_PORT_LAST,
+ ["Ready to rumble"],
+ ["Address already in use"])
diff --git a/support/testing/tests/download/http-server/extra b/support/testing/tests/download/http-server/extra
new file mode 100644
index 0000000000..8e27be7d61
--- /dev/null
+++ b/support/testing/tests/download/http-server/extra
@@ -0,0 +1 @@
+text
diff --git a/support/testing/tests/download/httpserver.py b/support/testing/tests/download/httpserver.py
new file mode 100644
index 0000000000..9f4f947304
--- /dev/null
+++ b/support/testing/tests/download/httpserver.py
@@ -0,0 +1,14 @@
+from infra.server import Server
+
+HTTP_SERVER_PORT_INITIAL = 8000
+HTTP_SERVER_PORT_LAST = HTTP_SERVER_PORT_INITIAL + 99
+
+
+class HttpServer(Server):
+ def __init__(self, builddir, serveddir, logtofile):
+ daemon_cmd = ["python", "-m", "SimpleHTTPServer"]
+ super(HttpServer, self).__init__(
+ builddir, serveddir, logtofile,
+ daemon_cmd, "{port}", HTTP_SERVER_PORT_INITIAL, HTTP_SERVER_PORT_LAST,
+ ["Serving HTTP"],
+ ["Address already in use"])
diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
index 162c03623b..e0abc3fa65 100644
--- a/support/testing/tests/download/test_git.py
+++ b/support/testing/tests/download/test_git.py
@@ -3,6 +3,7 @@ import shutil
import infra
from gitremote import GitRemote
+from httpserver import HttpServer
class GitTestBase(infra.basetest.BRTest):
@@ -12,13 +13,19 @@ class GitTestBase(infra.basetest.BRTest):
"""
gitremotedir = infra.filepath("tests/download/git-remote")
gitremote = None
+ httpserverdir = None
+ httpserver = None
def setUp(self):
super(GitTestBase, self).setUp()
self.gitremote = GitRemote(self.builddir, self.gitremotedir, self.logtofile)
+ if self.httpserverdir:
+ self.httpserver = HttpServer(self.builddir, self.httpserverdir, self.logtofile)
def tearDown(self):
self.show_msg("Cleaning up")
+ if self.httpserver:
+ self.httpserver.stop()
if self.gitremote:
self.gitremote.stop()
if self.b and not self.keepbuilds:
@@ -42,11 +49,25 @@ class GitTestBase(infra.basetest.BRTest):
shutil.rmtree(dl_dir)
env = {"BR2_DL_DIR": dl_dir,
"GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
+ if self.httpserver:
+ env["HTTP_SERVER_PORT_NUMBER"] = str(self.httpserver.port)
self.b.build(["{}-dirclean".format(package),
"{}-legal-info".format(package)],
env)
+class TestGitExtraDownload(GitTestBase):
+ br2_external = [infra.filepath("tests/download/br2-external/git-extra-download")]
+ httpserverdir = infra.filepath("tests/download/http-server")
+
+ def test_run(self):
+ with self.assertRaises(SystemError):
+ self.check_download("extra-fails")
+ with self.assertRaises(SystemError):
+ self.check_download("main-fails")
+ self.check_download("ok")
+
+
class TestGitHash(GitTestBase):
br2_external = [infra.filepath("tests/download/br2-external/git-hash")]
--
2.17.0
^ permalink raw reply related [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 04/10] testing/tests/download: add git hash test
2018-05-12 2:58 ` [Buildroot] [PATCH v5 04/10] testing/tests/download: add git hash test Ricardo Martincoski
@ 2019-02-04 15:52 ` Arnout Vandecappelle
2019-02-05 0:52 ` Ricardo Martincoski
0 siblings, 1 reply; 108+ messages in thread
From: Arnout Vandecappelle @ 2019-02-04 15:52 UTC (permalink / raw)
To: buildroot
Hi Ricardo,
I'll apply this today, but I have some comment...
On 12/05/2018 04:58, Ricardo Martincoski wrote:
[snip]
> support/testing/tests/download/gitremote.py | 44 ++++++++++++++++++
> support/testing/tests/download/test_git.py | 43 +++++++++++++++++
flake8 says:
support/testing/tests/download/gitremote.py:5:1: I100 Import statements are in
the wrong order. 'import infra' should be before 'import pexpect' and in a
different group.
support/testing/tests/download/test_git.py:4:1: I100 Import statements are in
the wrong order. 'from gitremote import GitRemote' should be before 'import
infra' and in a different group.
support/testing/tests/download/test_git.py:4:1: I201 Missing newline between
import groups. 'from gitremote import GitRemote' is identified as Third Party
and 'import infra' is identified as Third Party.
I fixed this while applying.
[snip]
> diff --git a/support/testing/tests/download/gitremote.py b/support/testing/tests/download/gitremote.py
> new file mode 100644
> index 0000000000..60bc49fbf8
> --- /dev/null
> +++ b/support/testing/tests/download/gitremote.py
> @@ -0,0 +1,44 @@
> +# subprocess does not kill the child daemon when a test case fails by raising
> +# an exception. So use pexpect instead.
> +import pexpect
> +
> +import infra
> +
> +GIT_REMOTE_PORT_INITIAL = 9418
> +GIT_REMOTE_PORT_LAST = GIT_REMOTE_PORT_INITIAL + 99
> +
> +
> +class GitRemote(object):
> + def __init__(self, builddir, serveddir, logtofile):
> + """
> + Start a local git server.
> +
> + In order to support test cases in parallel, select the port the
> + server will listen to in runtime. Since there is no reliable way
> + to allocate the port prior to starting the server (another
> + process in the host machine can use the port between it is
> + selected from a list and it is really allocated to the server)
> + try to start the server in a port and in the case it is already
> + in use, try the next one in the allowed range.
> + """
> + self.daemon = None
> + self.port = None
> + self.logfile = infra.open_log_file(builddir, "gitremote", logtofile)
> +
> + daemon_cmd = ["git", "daemon", "--reuseaddr", "--verbose", "--listen=localhost", "--export-all",
Line is too long. Fixed as well.
> + "--base-path={}".format(serveddir)]
> + for port in range(GIT_REMOTE_PORT_INITIAL, GIT_REMOTE_PORT_LAST + 1):
> + cmd = daemon_cmd + ["--port={port}".format(port=port)]
> + self.logfile.write("> starting git remote with '{}'\n".format(" ".join(cmd)))
> + self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile)
> + ret = self.daemon.expect(["Ready to rumble",
> + "Address already in use"])
Shouldn't we add a timeout here, just to be safe?
This can be fixed in a follow-up patch.
> + if ret == 0:
> + self.port = port
> + return
> + raise SystemError("Could not find a free port to run git remote")
> +
> + def stop(self):
> + if self.daemon is None:
> + return
> + self.daemon.terminate(force=True)
> diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
> new file mode 100644
> index 0000000000..14fc8e4da3
> --- /dev/null
> +++ b/support/testing/tests/download/test_git.py
> @@ -0,0 +1,43 @@
> +import os
> +
> +import infra
> +from gitremote import GitRemote
> +
> +
> +class GitTestBase(infra.basetest.BRTest):
> + config = \
> + """
> + BR2_BACKUP_SITE=""
> + """
> + gitremotedir = infra.filepath("tests/download/git-remote")
> + gitremote = None
> +
> + def setUp(self):
> + super(GitTestBase, self).setUp()
> + self.gitremote = GitRemote(self.builddir, self.gitremotedir, self.logtofile)
> +
> + def tearDown(self):
> + self.show_msg("Cleaning up")
> + if self.gitremote:
> + self.gitremote.stop()
> + if self.b and not self.keepbuilds:
This is actually a bit useless, since self.b is not initialzed in
BRTest.__init__ you'll actually get either an exception or self.b evaluates to
True. But the same pattern is used in other places, so OK. Can be fixed in a
follow-up patch.
Regards,
Arnout
> + self.b.delete()
> +
> + def check_hash(self, package):
> + # store downloaded tarball inside the output dir so the test infra
> + # cleans it up at the end
> + env = {"BR2_DL_DIR": os.path.join(self.builddir, "dl"),
> + "GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
> + self.b.build(["{}-dirclean".format(package),
> + "{}-source".format(package)],
> + env)
> +
> +
> +class TestGitHash(GitTestBase):
> + br2_external = [infra.filepath("tests/download/br2-external/git-hash")]
> +
> + def test_run(self):
> + with self.assertRaises(SystemError):
> + self.check_hash("bad")
> + self.check_hash("good")
> + self.check_hash("nohash")
>
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 02/10] testing/infra: split runtime test from BRTest
2018-05-12 2:58 ` [Buildroot] [PATCH v5 02/10] testing/infra: split runtime test from BRTest Ricardo Martincoski
@ 2019-02-04 15:55 ` Arnout Vandecappelle
2019-02-04 18:19 ` Matthew Weber
2019-02-05 1:00 ` Ricardo Martincoski
0 siblings, 2 replies; 108+ messages in thread
From: Arnout Vandecappelle @ 2019-02-04 15:55 UTC (permalink / raw)
To: buildroot
On 12/05/2018 04:58, Ricardo Martincoski wrote:
> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>
> Move the setup of emulator to a new class, RuntimeTestBase, that bahaves
> exactly like BRTest currently does.
> It will avoid duplicating code when adding a common class to test the
> git download infra.
>
> Change all current test cases to use the new class.
> Do this by first using automatic replace:
> $ find support/testing/ -name '*.py' | \
> xargs grep -l BRTest | \
> xargs sed -i \
> -e 's,import infra.basetest,\0\nimport infra.runtimetest,g' \
> -e 's,infra.basetest.BRTest,infra.runtimetest.RuntimeTestBase,g'
> and then manually add code to import runtimetest in test_external.py to
> avoid this error:
> AttributeError: 'module' object has no attribute 'LoadTestsFailure'
> This explicit import was not need before because run-tests imports
> BRTest and this is the only test file that do not use the defconfig
> fragments from basetest.py in its code.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> ---
> WARNING: this patch changes all current test cases, so if a new test case was
> applied after this patch was sent, 'git am' will apply this patch cleanly, yet
> any testcase created in the meantime will be broken.
> But I can refresh it if need. It will probably only need the automatic replace
> from the commit log to be re-run.
This seems to be an optimisation rather than a requirement for the other
patches, so I haven't applied this - it needs to be refreshed first.
If you do so before Wednesday, we might still apply it.
Regards,
Arnout
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 02/10] testing/infra: split runtime test from BRTest
2019-02-04 15:55 ` Arnout Vandecappelle
@ 2019-02-04 18:19 ` Matthew Weber
2019-02-04 19:42 ` Matthew Weber
2019-02-05 1:00 ` Ricardo Martincoski
1 sibling, 1 reply; 108+ messages in thread
From: Matthew Weber @ 2019-02-04 18:19 UTC (permalink / raw)
To: buildroot
Ricardo,
On Mon, Feb 4, 2019 at 9:55 AM Arnout Vandecappelle <arnout@mind.be> wrote:
>
>
>
> On 12/05/2018 04:58, Ricardo Martincoski wrote:
> > From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> >
> > Move the setup of emulator to a new class, RuntimeTestBase, that bahaves
> > exactly like BRTest currently does.
> > It will avoid duplicating code when adding a common class to test the
> > git download infra.
> >
> > Change all current test cases to use the new class.
> > Do this by first using automatic replace:
> > $ find support/testing/ -name '*.py' | \
> > xargs grep -l BRTest | \
> > xargs sed -i \
> > -e 's,import infra.basetest,\0\nimport infra.runtimetest,g' \
> > -e 's,infra.basetest.BRTest,infra.runtimetest.RuntimeTestBase,g'
> > and then manually add code to import runtimetest in test_external.py to
> > avoid this error:
> > AttributeError: 'module' object has no attribute 'LoadTestsFailure'
> > This explicit import was not need before because run-tests imports
> > BRTest and this is the only test file that do not use the defconfig
> > fragments from basetest.py in its code.
> >
> > Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> > Cc: Arnout Vandecappelle <arnout@mind.be>
> > ---
> > WARNING: this patch changes all current test cases, so if a new test case was
> > applied after this patch was sent, 'git am' will apply this patch cleanly, yet
> > any testcase created in the meantime will be broken.
> > But I can refresh it if need. It will probably only need the automatic replace
> > from the commit log to be re-run.
>
> This seems to be an optimisation rather than a requirement for the other
> patches, so I haven't applied this - it needs to be refreshed first.
>
> If you do so before Wednesday, we might still apply it.
>
Let me know if you won't have a chance and I'll get it updated for tomorrow.
Matt
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 02/10] testing/infra: split runtime test from BRTest
2019-02-04 18:19 ` Matthew Weber
@ 2019-02-04 19:42 ` Matthew Weber
2019-02-05 1:19 ` Ricardo Martincoski
0 siblings, 1 reply; 108+ messages in thread
From: Matthew Weber @ 2019-02-04 19:42 UTC (permalink / raw)
To: buildroot
Ricardo,
On Mon, Feb 4, 2019 at 12:19 PM Matthew Weber
<matthew.weber@rockwellcollins.com> wrote:
>
> Ricardo,
>
> On Mon, Feb 4, 2019 at 9:55 AM Arnout Vandecappelle <arnout@mind.be> wrote:
> >
> >
> >
> > On 12/05/2018 04:58, Ricardo Martincoski wrote:
> > > From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> > >
> > > Move the setup of emulator to a new class, RuntimeTestBase, that bahaves
> > > exactly like BRTest currently does.
> > > It will avoid duplicating code when adding a common class to test the
> > > git download infra.
> > >
> > > Change all current test cases to use the new class.
> > > Do this by first using automatic replace:
> > > $ find support/testing/ -name '*.py' | \
> > > xargs grep -l BRTest | \
> > > xargs sed -i \
> > > -e 's,import infra.basetest,\0\nimport infra.runtimetest,g' \
> > > -e 's,infra.basetest.BRTest,infra.runtimetest.RuntimeTestBase,g'
> > > and then manually add code to import runtimetest in test_external.py to
> > > avoid this error:
> > > AttributeError: 'module' object has no attribute 'LoadTestsFailure'
> > > This explicit import was not need before because run-tests imports
> > > BRTest and this is the only test file that do not use the defconfig
> > > fragments from basetest.py in its code.
> > >
> > > Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> > > Cc: Arnout Vandecappelle <arnout@mind.be>
> > > ---
> > > WARNING: this patch changes all current test cases, so if a new test case was
> > > applied after this patch was sent, 'git am' will apply this patch cleanly, yet
> > > any testcase created in the meantime will be broken.
> > > But I can refresh it if need. It will probably only need the automatic replace
> > > from the commit log to be re-run.
> >
> > This seems to be an optimisation rather than a requirement for the other
> > patches, so I haven't applied this - it needs to be refreshed first.
> >
> > If you do so before Wednesday, we might still apply it.
> >
>
> Let me know if you won't have a chance and I'll get it updated for tomorrow.
>
Ricardo, I just realized the scope of retesting this. I went ahead
and fixed up the patch for v6 (this included rebase changes for
br2-external and new test cases needing the new class). I also kicked
off a test run to check it out and I'll send the patch later if things
pass.
Matt
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 05/10] testing/tests/download: test case for git refs
2018-05-12 2:58 ` [Buildroot] [PATCH v5 05/10] testing/tests/download: test case for git refs Ricardo Martincoski
@ 2019-02-04 19:48 ` Arnout Vandecappelle
2019-02-05 0:53 ` Ricardo Martincoski
0 siblings, 1 reply; 108+ messages in thread
From: Arnout Vandecappelle @ 2019-02-04 19:48 UTC (permalink / raw)
To: buildroot
On 12/05/2018 04:58, Ricardo Martincoski wrote:
[snip]
> create mode 100644 support/testing/tests/download/br2-external/git-refs/Config.in
> create mode 100644 support/testing/tests/download/br2-external/git-refs/external.desc
> create mode 100644 support/testing/tests/download/br2-external/git-refs/external.mk
I don't think it makes a whole lot of sense to make a separate external for
this; it would seem easier to put everything in a single external (and move the
br2_external setting to GitTestBase). But that can be fixed in a follow-up
patch, so applied to master, thanks.
[snip]
> + def check_download(self, package):
> + # store downloaded tarball inside the output dir so the test infra
> + # cleans it up at the end
> + env = {"BR2_DL_DIR": os.path.join(self.builddir, "dl"),
> + "GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
> + self.b.build(["{}-dirclean".format(package),
> + "{}-legal-info".format(package)],
Here there's also a refactoring opportunity, since this function only differs
with check_hash in this one line.
Regards,
Arnout
> + env)
> +
>
> class TestGitHash(GitTestBase):
> br2_external = [infra.filepath("tests/download/br2-external/git-hash")]
> @@ -41,3 +50,13 @@ class TestGitHash(GitTestBase):
> self.check_hash("bad")
> self.check_hash("good")
> self.check_hash("nohash")
> +
> +
> +class TestGitRefs(GitTestBase):
> + br2_external = [infra.filepath("tests/download/br2-external/git-refs")]
> +
> + def test_run(self):
> + with self.assertRaises(SystemError):
> + self.check_download("git-wrong-content")
> + with self.assertRaises(SystemError):
> + self.check_download("git-wrong-sha1")
>
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 04/10] testing/tests/download: add git hash test
2019-02-04 15:52 ` Arnout Vandecappelle
@ 2019-02-05 0:52 ` Ricardo Martincoski
2019-02-05 8:09 ` Arnout Vandecappelle
0 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2019-02-05 0:52 UTC (permalink / raw)
To: buildroot
Hello,
On Mon, Feb 04, 2019 at 01:52 PM, Arnout Vandecappelle wrote:
> On 12/05/2018 04:58, Ricardo Martincoski wrote:
> [snip]
>> support/testing/tests/download/gitremote.py | 44 ++++++++++++++++++
>> support/testing/tests/download/test_git.py | 43 +++++++++++++++++
>
> flake8 says:
>
> support/testing/tests/download/gitremote.py:5:1: I100 Import statements are in
> the wrong order. 'import infra' should be before 'import pexpect' and in a
> different group.
> support/testing/tests/download/test_git.py:4:1: I100 Import statements are in
> the wrong order. 'from gitremote import GitRemote' should be before 'import
> infra' and in a different group.
> support/testing/tests/download/test_git.py:4:1: I201 Missing newline between
> import groups. 'from gitremote import GitRemote' is identified as Third Party
> and 'import infra' is identified as Third Party.
>
> I fixed this while applying.
So you have the plugin flake8-import-order.
I don't have it locally and we don't have it yet in the docker image.
But thank you for fixing.
[snip]
>> + daemon_cmd = ["git", "daemon", "--reuseaddr", "--verbose", "--listen=localhost", "--export-all",
>
> Line is too long. Fixed as well.
But it is shorter than 132 (as we set in .flake8), no?
Any chance you are using some version of flake8/pycodestyle that does not
support configs in .flake8 or max-line-length=132 ?
>
>> + "--base-path={}".format(serveddir)]
>> + for port in range(GIT_REMOTE_PORT_INITIAL, GIT_REMOTE_PORT_LAST + 1):
>> + cmd = daemon_cmd + ["--port={port}".format(port=port)]
>> + self.logfile.write("> starting git remote with '{}'\n".format(" ".join(cmd)))
>> + self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile)
>> + ret = self.daemon.expect(["Ready to rumble",
>> + "Address already in use"])
>
> Shouldn't we add a timeout here, just to be safe?
I think the expect() method uses the default from the class if not defined.
And the default timeout for spawn is 30 seconds AFAICR.
I did not double-checked the docs yet.
[snip]
>> +class GitTestBase(infra.basetest.BRTest):
>> + config = \
>> + """
>> + BR2_BACKUP_SITE=""
>> + """
>> + gitremotedir = infra.filepath("tests/download/git-remote")
>> + gitremote = None
>> +
>> + def setUp(self):
>> + super(GitTestBase, self).setUp()
>> + self.gitremote = GitRemote(self.builddir, self.gitremotedir, self.logtofile)
>> +
>> + def tearDown(self):
>> + self.show_msg("Cleaning up")
>> + if self.gitremote:
>> + self.gitremote.stop()
>> + if self.b and not self.keepbuilds:
>
> This is actually a bit useless, since self.b is not initialzed in
> BRTest.__init__ you'll actually get either an exception or self.b evaluates to
> True. But the same pattern is used in other places, so OK. Can be fixed in a
> follow-up patch.
Probably some leftover from a rebase. I will look into it.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 05/10] testing/tests/download: test case for git refs
2019-02-04 19:48 ` Arnout Vandecappelle
@ 2019-02-05 0:53 ` Ricardo Martincoski
0 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2019-02-05 0:53 UTC (permalink / raw)
To: buildroot
Hello,
On Mon, Feb 04, 2019 at 05:48 PM, Arnout Vandecappelle wrote:
> On 12/05/2018 04:58, Ricardo Martincoski wrote:
> [snip]
>> create mode 100644 support/testing/tests/download/br2-external/git-refs/Config.in
>> create mode 100644 support/testing/tests/download/br2-external/git-refs/external.desc
>> create mode 100644 support/testing/tests/download/br2-external/git-refs/external.mk
>
> I don't think it makes a whole lot of sense to make a separate external for
> this; it would seem easier to put everything in a single external (and move the
> br2_external setting to GitTestBase). But that can be fixed in a follow-up
> patch, so applied to master, thanks.
I don't remember if there was any reason to keep them separate.
Indeed it seems they can be merged. I will add to my todo list.
>
> [snip]
>> + def check_download(self, package):
>> + # store downloaded tarball inside the output dir so the test infra
>> + # cleans it up at the end
>> + env = {"BR2_DL_DIR": os.path.join(self.builddir, "dl"),
>> + "GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
>> + self.b.build(["{}-dirclean".format(package),
>> + "{}-legal-info".format(package)],
>
> Here there's also a refactoring opportunity, since this function only differs
> with check_hash in this one line.
I will take a look.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 02/10] testing/infra: split runtime test from BRTest
2019-02-04 15:55 ` Arnout Vandecappelle
2019-02-04 18:19 ` Matthew Weber
@ 2019-02-05 1:00 ` Ricardo Martincoski
2019-02-05 1:12 ` Matthew Weber
1 sibling, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2019-02-05 1:00 UTC (permalink / raw)
To: buildroot
Hello,
+ Matt
On Mon, Feb 04, 2019 at 01:55 PM, Arnout Vandecappelle wrote:
> On 12/05/2018 04:58, Ricardo Martincoski wrote:
>> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>>
>> Move the setup of emulator to a new class, RuntimeTestBase, that bahaves
>> exactly like BRTest currently does.
>> It will avoid duplicating code when adding a common class to test the
>> git download infra.
>>
>> Change all current test cases to use the new class.
>> Do this by first using automatic replace:
>> $ find support/testing/ -name '*.py' | \
>> xargs grep -l BRTest | \
>> xargs sed -i \
>> -e 's,import infra.basetest,\0\nimport infra.runtimetest,g' \
>> -e 's,infra.basetest.BRTest,infra.runtimetest.RuntimeTestBase,g'
>> and then manually add code to import runtimetest in test_external.py to
>> avoid this error:
>> AttributeError: 'module' object has no attribute 'LoadTestsFailure'
>> This explicit import was not need before because run-tests imports
>> BRTest and this is the only test file that do not use the defconfig
>> fragments from basetest.py in its code.
>>
>> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>> Cc: Arnout Vandecappelle <arnout@mind.be>
>> ---
>> WARNING: this patch changes all current test cases, so if a new test case was
>> applied after this patch was sent, 'git am' will apply this patch cleanly, yet
>> any testcase created in the meantime will be broken.
>> But I can refresh it if need. It will probably only need the automatic replace
>> from the commit log to be re-run.
>
> This seems to be an optimisation rather than a requirement for the other
> patches, so I haven't applied this - it needs to be refreshed first.
It's an optimisation, but I failed to describe why it should be done first...
The download test cases do not need to build anything before they run, so by
splitting the two classes and making test_git to inherit from the new BRTest
instead of RuntimeTestBase makes the download test cases to run really fast.
The two test cases already on master, after applying and refreshing this patch,
run in 35 seconds total in my machine.
Less than 2 minutes each in the Gitlab CI auto-scale runners:
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/156194310
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/156194313
Unfortunately there is still some problem in my WIP refresh of the patch, so I
can't send it now.
>
> If you do so before Wednesday, we might still apply it.
Unfortunately it's unlikely I can do in time.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 02/10] testing/infra: split runtime test from BRTest
2019-02-05 1:00 ` Ricardo Martincoski
@ 2019-02-05 1:12 ` Matthew Weber
2019-02-05 1:47 ` Ricardo Martincoski
2019-02-05 9:28 ` Matthew Weber
0 siblings, 2 replies; 108+ messages in thread
From: Matthew Weber @ 2019-02-05 1:12 UTC (permalink / raw)
To: buildroot
Ricardo,
On Mon, Feb 4, 2019 at 7:00 PM Ricardo Martincoski
<ricardo.martincoski@gmail.com> wrote:
>
> Hello,
>
> + Matt
>
> On Mon, Feb 04, 2019 at 01:55 PM, Arnout Vandecappelle wrote:
>
> > On 12/05/2018 04:58, Ricardo Martincoski wrote:
> >> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> >>
> >> Move the setup of emulator to a new class, RuntimeTestBase, that bahaves
> >> exactly like BRTest currently does.
> >> It will avoid duplicating code when adding a common class to test the
> >> git download infra.
> >>
> >> Change all current test cases to use the new class.
> >> Do this by first using automatic replace:
> >> $ find support/testing/ -name '*.py' | \
> >> xargs grep -l BRTest | \
> >> xargs sed -i \
> >> -e 's,import infra.basetest,\0\nimport infra.runtimetest,g' \
> >> -e 's,infra.basetest.BRTest,infra.runtimetest.RuntimeTestBase,g'
> >> and then manually add code to import runtimetest in test_external.py to
> >> avoid this error:
> >> AttributeError: 'module' object has no attribute 'LoadTestsFailure'
> >> This explicit import was not need before because run-tests imports
> >> BRTest and this is the only test file that do not use the defconfig
> >> fragments from basetest.py in its code.
> >>
> >> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> >> Cc: Arnout Vandecappelle <arnout@mind.be>
> >> ---
> >> WARNING: this patch changes all current test cases, so if a new test case was
> >> applied after this patch was sent, 'git am' will apply this patch cleanly, yet
> >> any testcase created in the meantime will be broken.
> >> But I can refresh it if need. It will probably only need the automatic replace
> >> from the commit log to be re-run.
> >
> > This seems to be an optimisation rather than a requirement for the other
> > patches, so I haven't applied this - it needs to be refreshed first.
>
> It's an optimisation, but I failed to describe why it should be done first...
>
> The download test cases do not need to build anything before they run, so by
> splitting the two classes and making test_git to inherit from the new BRTest
> instead of RuntimeTestBase makes the download test cases to run really fast.
> The two test cases already on master, after applying and refreshing this patch,
> run in 35 seconds total in my machine.
> Less than 2 minutes each in the Gitlab CI auto-scale runners:
> https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/156194310
> https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/156194313
If I understand correctly those are the only two at this point that
should use the BRTest class instead of the RuntimeTestBase, right?
> Unfortunately there is still some problem in my WIP refresh of the patch, so I
> can't send it now.
>
> >
> > If you do so before Wednesday, we might still apply it.
>
> Unfortunately it's unlikely I can do in time.
>
I might be in the same situation after looking at the build failures.
I'll spend a bit more on it tonight but there are a lot of tests to
either spot check or run all. I'm not sure yet if it is my build
machine's OS and python version or qemu at this point.
Matt
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 02/10] testing/infra: split runtime test from BRTest
2019-02-04 19:42 ` Matthew Weber
@ 2019-02-05 1:19 ` Ricardo Martincoski
2019-02-05 8:29 ` Arnout Vandecappelle
0 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2019-02-05 1:19 UTC (permalink / raw)
To: buildroot
Hello,
On Mon, Feb 04, 2019 at 05:42 PM, Matthew Weber wrote:
> On Mon, Feb 4, 2019 at 12:19 PM Matthew Weber wrote:
>>
>> On Mon, Feb 4, 2019 at 9:55 AM Arnout Vandecappelle <arnout@mind.be> wrote:
[snip]
>> > This seems to be an optimisation rather than a requirement for the other
>> > patches, so I haven't applied this - it needs to be refreshed first.
>> >
>> > If you do so before Wednesday, we might still apply it.
>> >
>>
>> Let me know if you won't have a chance and I'll get it updated for tomorrow.
>>
>
> Ricardo, I just realized the scope of retesting this. I went ahead
> and fixed up the patch for v6 (this included rebase changes for
> br2-external and new test cases needing the new class). I also kicked
> off a test run to check it out and I'll send the patch later if things
> pass.
If you find the time, please do.
It's unlikely I have enough time before Wednesday.
I started to refresh the patch, but there are some test cases failing, I don't
know why. Probably I made some mistake.
https://gitlab.com/RicardoMartincoski/buildroot/pipelines/46127542
I did not checked if any of these test cases already fail without this patch.
I just kicked off a test run for this:
https://gitlab.com/RicardoMartincoski/buildroot/pipelines/46142012
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 02/10] testing/infra: split runtime test from BRTest
2019-02-05 1:12 ` Matthew Weber
@ 2019-02-05 1:47 ` Ricardo Martincoski
2019-02-05 9:28 ` Matthew Weber
1 sibling, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2019-02-05 1:47 UTC (permalink / raw)
To: buildroot
Hello,
On Mon, Feb 04, 2019 at 11:12 PM, Matthew Weber wrote:
> On Mon, Feb 4, 2019 at 7:00 PM Ricardo Martincoski wrote:
[snip]
>> The download test cases do not need to build anything before they run, so by
>> splitting the two classes and making test_git to inherit from the new BRTest
>> instead of RuntimeTestBase makes the download test cases to run really fast.
>> The two test cases already on master, after applying and refreshing this patch,
>> run in 35 seconds total in my machine.
>> Less than 2 minutes each in the Gitlab CI auto-scale runners:
>> https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/156194310
>> https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/156194313
>
> If I understand correctly those are the only two at this point that
> should use the BRTest class instead of the RuntimeTestBase, right?
Right.
>
>> Unfortunately there is still some problem in my WIP refresh of the patch, so I
>> can't send it now.
>>
>> >
>> > If you do so before Wednesday, we might still apply it.
>>
>> Unfortunately it's unlikely I can do in time.
>>
>
> I might be in the same situation after looking at the build failures.
No problem. We can do it later. Thank you for looking into this.
> I'll spend a bit more on it tonight but there are a lot of tests to
> either spot check or run all. I'm not sure yet if it is my build
> machine's OS and python version or qemu at this point.
I found myself in this situation a few times in the past :-)
You can also download the docker image and run the test cases inside it.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 04/10] testing/tests/download: add git hash test
2019-02-05 0:52 ` Ricardo Martincoski
@ 2019-02-05 8:09 ` Arnout Vandecappelle
2019-02-05 8:55 ` Peter Korsgaard
0 siblings, 1 reply; 108+ messages in thread
From: Arnout Vandecappelle @ 2019-02-05 8:09 UTC (permalink / raw)
To: buildroot
On 05/02/2019 01:52, Ricardo Martincoski wrote:
> Hello,
>
> On Mon, Feb 04, 2019 at 01:52 PM, Arnout Vandecappelle wrote:
>
>> On 12/05/2018 04:58, Ricardo Martincoski wrote:
>> [snip]
>>> support/testing/tests/download/gitremote.py | 44 ++++++++++++++++++
>>> support/testing/tests/download/test_git.py | 43 +++++++++++++++++
>>
>> flake8 says:
>>
>> support/testing/tests/download/gitremote.py:5:1: I100 Import statements are in
>> the wrong order. 'import infra' should be before 'import pexpect' and in a
>> different group.
>> support/testing/tests/download/test_git.py:4:1: I100 Import statements are in
>> the wrong order. 'from gitremote import GitRemote' should be before 'import
>> infra' and in a different group.
>> support/testing/tests/download/test_git.py:4:1: I201 Missing newline between
>> import groups. 'from gitremote import GitRemote' is identified as Third Party
>> and 'import infra' is identified as Third Party.
>>
>> I fixed this while applying.
>
> So you have the plugin flake8-import-order.
> I don't have it locally and we don't have it yet in the docker image.
We could add it, but in fact a lot of things "break" with it.
> But thank you for fixing.
>
> [snip]
>>> + daemon_cmd = ["git", "daemon", "--reuseaddr", "--verbose", "--listen=localhost", "--export-all",
>>
>> Line is too long. Fixed as well.
>
> But it is shorter than 132 (as we set in .flake8), no?
>
> Any chance you are using some version of flake8/pycodestyle that does not
> support configs in .flake8 or max-line-length=132 ?
No, I just manually check in my editor.
The idea is that our target line length is 80, but in many cases lines are
difficult to split (it looks worse). Since we have the flake8 run in our CI, we
shouldn't error out on too long lines. And adding an exception comment for each
long line does not make the code more readable. So as a compromise, we set the
line length to 132 in .flake8. But the intention is to have line length of 80.
Perhaps we should remove it from flake8, and instead add the option to
.gitlab-ci.yml.in. Hm, yes, that would make a lot of sense...
>
>>
>>> + "--base-path={}".format(serveddir)]
>>> + for port in range(GIT_REMOTE_PORT_INITIAL, GIT_REMOTE_PORT_LAST + 1):
>>> + cmd = daemon_cmd + ["--port={port}".format(port=port)]
>>> + self.logfile.write("> starting git remote with '{}'\n".format(" ".join(cmd)))
>>> + self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile)
>>> + ret = self.daemon.expect(["Ready to rumble",
>>> + "Address already in use"])
>>
>> Shouldn't we add a timeout here, just to be safe?
>
> I think the expect() method uses the default from the class if not defined.
> And the default timeout for spawn is 30 seconds AFAICR.
> I did not double-checked the docs yet.
I had only diagonally read the documentation and I saw that timeout in expect()
defaults to -1, so I assumed it was infinity... But in fact it indeed means
"inherit from the spawn class", and there it defaults to 30.
Regards,
Arnout
>
> [snip]
>>> +class GitTestBase(infra.basetest.BRTest):
>>> + config = \
>>> + """
>>> + BR2_BACKUP_SITE=""
>>> + """
>>> + gitremotedir = infra.filepath("tests/download/git-remote")
>>> + gitremote = None
>>> +
>>> + def setUp(self):
>>> + super(GitTestBase, self).setUp()
>>> + self.gitremote = GitRemote(self.builddir, self.gitremotedir, self.logtofile)
>>> +
>>> + def tearDown(self):
>>> + self.show_msg("Cleaning up")
>>> + if self.gitremote:
>>> + self.gitremote.stop()
>>> + if self.b and not self.keepbuilds:
>>
>> This is actually a bit useless, since self.b is not initialzed in
>> BRTest.__init__ you'll actually get either an exception or self.b evaluates to
>> True. But the same pattern is used in other places, so OK. Can be fixed in a
>> follow-up patch.
>
> Probably some leftover from a rebase. I will look into it.
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 02/10] testing/infra: split runtime test from BRTest
2019-02-05 1:19 ` Ricardo Martincoski
@ 2019-02-05 8:29 ` Arnout Vandecappelle
0 siblings, 0 replies; 108+ messages in thread
From: Arnout Vandecappelle @ 2019-02-05 8:29 UTC (permalink / raw)
To: buildroot
On 05/02/2019 02:19, Ricardo Martincoski wrote:
> Hello,
>
> On Mon, Feb 04, 2019 at 05:42 PM, Matthew Weber wrote:
>
>> On Mon, Feb 4, 2019 at 12:19 PM Matthew Weber wrote:
>>>
>>> On Mon, Feb 4, 2019 at 9:55 AM Arnout Vandecappelle <arnout@mind.be> wrote:
> [snip]
>>>> This seems to be an optimisation rather than a requirement for the other
>>>> patches, so I haven't applied this - it needs to be refreshed first.
>>>>
>>>> If you do so before Wednesday, we might still apply it.
>>>>
>>>
>>> Let me know if you won't have a chance and I'll get it updated for tomorrow.
>>>
>>
>> Ricardo, I just realized the scope of retesting this. I went ahead
>> and fixed up the patch for v6 (this included rebase changes for
>> br2-external and new test cases needing the new class). I also kicked
>> off a test run to check it out and I'll send the patch later if things
>> pass.
>
> If you find the time, please do.
> It's unlikely I have enough time before Wednesday.
>
> I started to refresh the patch, but there are some test cases failing, I don't
> know why. Probably I made some mistake.
> https://gitlab.com/RicardoMartincoski/buildroot/pipelines/46127542
I think these failures are already happening in master:
https://gitlab.com/buildroot.org/buildroot/pipelines/45710311
Someone really should look at the CI results :-)
Regards,
Arnout
>
> I did not checked if any of these test cases already fail without this patch.
> I just kicked off a test run for this:
> https://gitlab.com/RicardoMartincoski/buildroot/pipelines/46142012
>
>
> Regards,
> Ricardo
>
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 04/10] testing/tests/download: add git hash test
2019-02-05 8:09 ` Arnout Vandecappelle
@ 2019-02-05 8:55 ` Peter Korsgaard
0 siblings, 0 replies; 108+ messages in thread
From: Peter Korsgaard @ 2019-02-05 8:55 UTC (permalink / raw)
To: buildroot
>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:
Hi,
>>> Shouldn't we add a timeout here, just to be safe?
>>
>> I think the expect() method uses the default from the class if not defined.
>> And the default timeout for spawn is 30 seconds AFAICR.
>> I did not double-checked the docs yet.
> I had only diagonally read the documentation and I saw that timeout in expect()
> defaults to -1, so I assumed it was infinity... But in fact it indeed means
> "inherit from the spawn class", and there it defaults to 30.
I got confused by that as well when I wrote the docker/compose
tests.
Notice that we set timeout to 5 in emulator.py:
timeout=5 * self.timeout_multiplier
So I believe -1 means 5 seconds, not 30.
--
Bye, Peter Korsgaard
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 02/10] testing/infra: split runtime test from BRTest
2019-02-05 1:12 ` Matthew Weber
2019-02-05 1:47 ` Ricardo Martincoski
@ 2019-02-05 9:28 ` Matthew Weber
1 sibling, 0 replies; 108+ messages in thread
From: Matthew Weber @ 2019-02-05 9:28 UTC (permalink / raw)
To: buildroot
Ricardo / Arnout,
On Mon, Feb 4, 2019 at 7:12 PM Matthew Weber
<matthew.weber@rockwellcollins.com> wrote:
>
> Ricardo,
> On Mon, Feb 4, 2019 at 7:00 PM Ricardo Martincoski
> <ricardo.martincoski@gmail.com> wrote:
> >
> > Hello,
> >
> > + Matt
> >
> > On Mon, Feb 04, 2019 at 01:55 PM, Arnout Vandecappelle wrote:
> >
> > > On 12/05/2018 04:58, Ricardo Martincoski wrote:
> > >> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> > >>
> > >> Move the setup of emulator to a new class, RuntimeTestBase, that bahaves
> > >> exactly like BRTest currently does.
> > >> It will avoid duplicating code when adding a common class to test the
> > >> git download infra.
> > >>
> > >> Change all current test cases to use the new class.
> > >> Do this by first using automatic replace:
> > >> $ find support/testing/ -name '*.py' | \
> > >> xargs grep -l BRTest | \
> > >> xargs sed -i \
> > >> -e 's,import infra.basetest,\0\nimport infra.runtimetest,g' \
> > >> -e 's,infra.basetest.BRTest,infra.runtimetest.RuntimeTestBase,g'
> > >> and then manually add code to import runtimetest in test_external.py to
> > >> avoid this error:
> > >> AttributeError: 'module' object has no attribute 'LoadTestsFailure'
> > >> This explicit import was not need before because run-tests imports
> > >> BRTest and this is the only test file that do not use the defconfig
> > >> fragments from basetest.py in its code.
> > >>
> > >> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> > >> Cc: Arnout Vandecappelle <arnout@mind.be>
> > >> ---
> > >> WARNING: this patch changes all current test cases, so if a new test case was
> > >> applied after this patch was sent, 'git am' will apply this patch cleanly, yet
> > >> any testcase created in the meantime will be broken.
> > >> But I can refresh it if need. It will probably only need the automatic replace
> > >> from the commit log to be re-run.
> > >
> > > This seems to be an optimisation rather than a requirement for the other
> > > patches, so I haven't applied this - it needs to be refreshed first.
> >
> > It's an optimisation, but I failed to describe why it should be done first...
> >
> > The download test cases do not need to build anything before they run, so by
> > splitting the two classes and making test_git to inherit from the new BRTest
> > instead of RuntimeTestBase makes the download test cases to run really fast.
> > The two test cases already on master, after applying and refreshing this patch,
> > run in 35 seconds total in my machine.
> > Less than 2 minutes each in the Gitlab CI auto-scale runners:
> > https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/156194310
> > https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/156194313
>
> If I understand correctly those are the only two at this point that
> should use the BRTest class instead of the RuntimeTestBase, right?
>
> > Unfortunately there is still some problem in my WIP refresh of the patch, so I
> > can't send it now.
> >
> > >
> > > If you do so before Wednesday, we might still apply it.
> >
> > Unfortunately it's unlikely I can do in time.
> >
>
> I might be in the same situation after looking at the build failures.
> I'll spend a bit more on it tonight but there are a lot of tests to
> either spot check or run all. I'm not sure yet if it is my build
> machine's OS and python version or qemu at this point.
>
Here's the v6 and I'll start looking through the differences in the CI
reports (looked similar enough I figured I'd get the patch out first).
http://patchwork.ozlabs.org/patch/1036559/
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 06/10] testing/tests/download: test git branch
2018-05-12 2:58 ` [Buildroot] [PATCH v5 06/10] testing/tests/download: test git branch Ricardo Martincoski
@ 2019-02-05 9:42 ` Arnout Vandecappelle
0 siblings, 0 replies; 108+ messages in thread
From: Arnout Vandecappelle @ 2019-02-05 9:42 UTC (permalink / raw)
To: buildroot
On 12/05/2018 04:58, Ricardo Martincoski wrote:
> Add a branch to the static repo and check on the git refs test case the
> download of a git package:
> - with a sha1 reachable by a branch name, but not pointed by it, as
> version. This is the most common use case for git refs in the tree;
> - with a partial sha1 of a commit reachable by a branch as version;
> - with a sha1 of the commit head of a branch as version;
> - with a partial sha1 of the commit head of a branch as version;
>
> Enforce the download always occurs by removing the BR2_DL_DIR used for
> the tarballs generated by the git download infra.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> Cc: Arnout Vandecappelle <arnout@mind.be>
Applied to master, thanks.
Regards,
Arnout
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 07/10] testing/tests/download: test git submodules
2018-05-12 2:58 ` [Buildroot] [PATCH v5 07/10] testing/tests/download: test git submodules Ricardo Martincoski
@ 2019-02-05 10:03 ` Arnout Vandecappelle
2019-02-06 9:08 ` Arnout Vandecappelle
2019-02-06 9:14 ` Yann E. MORIN
0 siblings, 2 replies; 108+ messages in thread
From: Arnout Vandecappelle @ 2019-02-05 10:03 UTC (permalink / raw)
To: buildroot
On 12/05/2018 04:58, Ricardo Martincoski wrote:
> +################################################################################
> +#
> +# git-submodule-enabled
> +#
> +################################################################################
> +
> +GIT_SUBMODULE_ENABLED_VERSION = a9dbc1e23c45e8e1b88c0448763f54d714eb6f8f
> +GIT_SUBMODULE_ENABLED_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
> +GIT_SUBMODULE_ENABLED_GIT_SUBMODULES = YES
> +GIT_SUBMODULE_ENABLED_LICENSE_FILES = \
> + file \
> + refs-sub1/file \
> + refs-sub1/refs-sub2/file
> +
> +# Some versions of git client fill the .git file for the second level submodule
> +# using the absolute path to the main .git directory, which in the case of the
> +# buildroot download helper is always different since it uses a temporary
> +# directory. This version of git have the issue:
> +# - 2.7.4 included in Ubuntu 16.04;
> +# The same does not occur using newer versions of git:
> +# - 2.11.0 included in Debian 9;
> +# - 2.14.2 latest at the time of writing;
> +# In order to allow running this test case locally in many systems, do not check
> +# for the hash of the tarball, but instead check the hash of each meaningful
> +# file included in the tarball.
This is a bug! Funny that we never hit it in the autobuilders, don't we have an
Ubuntu 16.04 autobuilder and don't we have packages with submodules and a hash?
Anyway, I think maybe we should fix the bug first... Ideally there should be a
test case for it as well, but that's difficult since it depends on the installed
git version.
Yann, how about if we change the find command in the git helper to
find . -name .git -prune -o -print
?
Regards,
Arnout
> +BR_NO_CHECK_HASH_FOR += $(GIT_SUBMODULE_ENABLED_SOURCE)
> +
> +$(eval $(generic-package))
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 07/10] testing/tests/download: test git submodules
2019-02-05 10:03 ` Arnout Vandecappelle
@ 2019-02-06 9:08 ` Arnout Vandecappelle
2019-02-06 9:14 ` Yann E. MORIN
1 sibling, 0 replies; 108+ messages in thread
From: Arnout Vandecappelle @ 2019-02-06 9:08 UTC (permalink / raw)
To: buildroot
On 05/02/2019 11:03, Arnout Vandecappelle wrote:
>
>
> On 12/05/2018 04:58, Ricardo Martincoski wrote:
>> +################################################################################
>> +#
>> +# git-submodule-enabled
>> +#
>> +################################################################################
>> +
>> +GIT_SUBMODULE_ENABLED_VERSION = a9dbc1e23c45e8e1b88c0448763f54d714eb6f8f
>> +GIT_SUBMODULE_ENABLED_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
>> +GIT_SUBMODULE_ENABLED_GIT_SUBMODULES = YES
>> +GIT_SUBMODULE_ENABLED_LICENSE_FILES = \
>> + file \
>> + refs-sub1/file \
>> + refs-sub1/refs-sub2/file
>> +
>> +# Some versions of git client fill the .git file for the second level submodule
>> +# using the absolute path to the main .git directory, which in the case of the
>> +# buildroot download helper is always different since it uses a temporary
>> +# directory. This version of git have the issue:
>> +# - 2.7.4 included in Ubuntu 16.04;
>> +# The same does not occur using newer versions of git:
>> +# - 2.11.0 included in Debian 9;
>> +# - 2.14.2 latest at the time of writing;
>> +# In order to allow running this test case locally in many systems, do not check
>> +# for the hash of the tarball, but instead check the hash of each meaningful
>> +# file included in the tarball.
>
> This is a bug!
So, I've removed this workaround from the test and pushed. It will most likely
fail in gitlab because our container is based on Ubuntu 16.04. But thats exactly
what we want, because it is a bug.
Regards,
Arnout
> Funny that we never hit it in the autobuilders, don't we have an
> Ubuntu 16.04 autobuilder and don't we have packages with submodules and a hash?
>
> Anyway, I think maybe we should fix the bug first... Ideally there should be a
> test case for it as well, but that's difficult since it depends on the installed
> git version.
>
> Yann, how about if we change the find command in the git helper to
>
> find . -name .git -prune -o -print
>
> ?
>
> Regards,
> Arnout
>
>> +BR_NO_CHECK_HASH_FOR += $(GIT_SUBMODULE_ENABLED_SOURCE)
>> +
>> +$(eval $(generic-package))
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 07/10] testing/tests/download: test git submodules
2019-02-05 10:03 ` Arnout Vandecappelle
2019-02-06 9:08 ` Arnout Vandecappelle
@ 2019-02-06 9:14 ` Yann E. MORIN
1 sibling, 0 replies; 108+ messages in thread
From: Yann E. MORIN @ 2019-02-06 9:14 UTC (permalink / raw)
To: buildroot
Arnout, All,
On 2019-02-05 11:03 +0100, Arnout Vandecappelle spake thusly:
> On 12/05/2018 04:58, Ricardo Martincoski wrote:
> > +################################################################################
> > +#
> > +# git-submodule-enabled
> > +#
> > +################################################################################
> > +
> > +GIT_SUBMODULE_ENABLED_VERSION = a9dbc1e23c45e8e1b88c0448763f54d714eb6f8f
> > +GIT_SUBMODULE_ENABLED_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
> > +GIT_SUBMODULE_ENABLED_GIT_SUBMODULES = YES
> > +GIT_SUBMODULE_ENABLED_LICENSE_FILES = \
> > + file \
> > + refs-sub1/file \
> > + refs-sub1/refs-sub2/file
> > +
> > +# Some versions of git client fill the .git file for the second level submodule
> > +# using the absolute path to the main .git directory, which in the case of the
> > +# buildroot download helper is always different since it uses a temporary
> > +# directory. This version of git have the issue:
> > +# - 2.7.4 included in Ubuntu 16.04;
> > +# The same does not occur using newer versions of git:
> > +# - 2.11.0 included in Debian 9;
> > +# - 2.14.2 latest at the time of writing;
> > +# In order to allow running this test case locally in many systems, do not check
> > +# for the hash of the tarball, but instead check the hash of each meaningful
> > +# file included in the tarball.
>
> This is a bug! Funny that we never hit it in the autobuilders, don't we have an
> Ubuntu 16.04 autobuilder and don't we have packages with submodules and a hash?
>
> Anyway, I think maybe we should fix the bug first... Ideally there should be a
> test case for it as well, but that's difficult since it depends on the installed
> git version.
>
> Yann, how about if we change the find command in the git helper to
> find . -name .git -prune -o -print
> ?
Yeah, I too consider it a bug, and this should be fixed. I'll tackle
this as part of my git-backend rework in the upcoming days.
Sorry I did not reply to the mail earlier, with all the live discussion
we had yesterday it just slipped my mind...
Regards,
Yann E. MORIN.
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 08/10] testing/tests/download: test git tag
2018-05-12 2:58 ` [Buildroot] [PATCH v5 08/10] testing/tests/download: test git tag Ricardo Martincoski
@ 2019-02-06 10:03 ` Arnout Vandecappelle
0 siblings, 0 replies; 108+ messages in thread
From: Arnout Vandecappelle @ 2019-02-06 10:03 UTC (permalink / raw)
To: buildroot
On 12/05/2018 04:58, Ricardo Martincoski wrote:
> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>
> Add a tag to the static repo and check on the git refs test case the
> download of a git package:
> - with the name of a tag as version;
> - with the sha1 of a tag itself as version;
> - with the partial sha1 of a tag itself as version;
> - with the sha1 of a commit pointed by a tag as version;
> - with the partial sha1 of a commit pointed by a tag as version;
> - with the sha1 of a commit reachable only by a tag as version;
> - with the partial sha1 of a commit reachable only by a tag as version.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> Cc: Arnout Vandecappelle <arnout@mind.be>
Applied to master, thanks.
Regards,
Arnout
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 09/10] testing/tests/download: test git special ref
2018-05-12 2:58 ` [Buildroot] [PATCH v5 09/10] testing/tests/download: test git special ref Ricardo Martincoski
@ 2019-02-06 10:05 ` Arnout Vandecappelle
2019-02-18 2:46 ` Ricardo Martincoski
0 siblings, 1 reply; 108+ messages in thread
From: Arnout Vandecappelle @ 2019-02-06 10:05 UTC (permalink / raw)
To: buildroot
On 12/05/2018 04:58, Ricardo Martincoski wrote:
> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>
> Add a special ref to the static repo and check on the git refs test case
> the download of a git package:
> - with the name of a special ref as version;
We might remove support for getting special refs by name, but keep on
supporting getting special refs by sha1. So I've changed the test into getting
it by sha1 instead.
Ideally there should also be a check for getting the parent of the special ref
by sha1, but I think that actually does *not* work now.
Regards,
Arnout
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 10/10] support/testing: test extra download with site method git
2018-05-12 2:58 ` [Buildroot] [PATCH v5 10/10] support/testing: test extra download with site method git Ricardo Martincoski
@ 2019-02-06 10:34 ` Arnout Vandecappelle
0 siblings, 0 replies; 108+ messages in thread
From: Arnout Vandecappelle @ 2019-02-06 10:34 UTC (permalink / raw)
To: buildroot
On 12/05/2018 04:58, Ricardo Martincoski wrote:
> Add one test case to ensure the use of extra download works for git
> packages:
> - an extra download is correctly downloaded using wget after the main
> download finished using the git backend;
> - when the main download using the git backend fails, the build fails;
> - when the extra download using wget fails, the build fails.
>
> Reuse the commit in the static repo.
> Add required infra and fixture:
> - a HttpServer class, that starts a http server in the host machine to
> emulate a remote http server under the control of the test;
> - a file to be served by the http server;
> - a br2-external (git-extra-download) with one package for each part of
> the test case.
> Since the HttpServer is similar to GitRemote, extract the commonalities
> to a new base class called Server.
This one is a little too complicated to just apply, so I'm going to leave it
here for the time being...
It should for sure be split into two patches, one adding the common Server
class to the infra, and the next one adding the http test.
Also I'd probably prefer first just adding an http download test case and only
later adding the extra downloads test.
However, I also don't really like the approach of launching our own http
server. I think it's much easier to rely on an existing one, perhaps using
sources.buildroot.org. You could say that it makes it impossible to run the
tests offline, but that's already the case for a most of them, because they
download sources and kernel binaries etc.
So I've marked this one as Changes Requested in patchwork.
Regards,
Arnout
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
> Cc: Luca Ceresoli <luca@lucaceresoli.net>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> Cc: Yann E. MORIN <yann.morin.1998@free.fr>
> ---
> Changes v4 -> v5:
> - no changes
>
> Changes v3 -> v4:
> - new patch
>
> I use docker only with the Buildroot image for the test infra.
> Sorry, I give up for now on trying to run apache on the docker image.
>
> As a midground solution between:
> - adding more files to http://autobuild.buildroot.net/artefacts/ that
> won't scale well if we need different files for different test cases;
> - installing a full web server to the docker image auto-loading as root
> when the container is started as user and allowing the test infra to
> copy files to the served path as user;
> and inspired by this article:
> https://www.pcsuggest.com/best-lightweight-web-server-linux/
> I created this patch using SimpleHTTPServer being started by the test
> infra as a user.
> I know it is weird to use pexpect to fork a server written in Python,
> but it has these advantages:
> - theoretically replacing SimpleHTTPServer with a similar solution (say
> the busybox httpd) in the future would require only small changes to
> the test infra code;
> - SimpleHTTPServer is already on the docker image, busybox isn't, so no
> need to change the Dockerfile, generate the image, publish it and
> update the docker image tag on .gitlab-ci.yml;
> - the log file generated by the server is handled in the same way as
> other log files generated by the test infra;
> ---
> .gitlab-ci.yml | 1 +
> support/testing/infra/server.py | 39 +++++++++++++++++
> .../br2-external/git-extra-download/Config.in | 0
> .../git-extra-download/external.desc | 1 +
> .../git-extra-download/external.mk | 6 +++
> .../package/extra-fails/extra-fails.hash | 3 ++
> .../package/extra-fails/extra-fails.mk | 12 ++++++
> .../package/main-fails/main-fails.hash | 3 ++
> .../package/main-fails/main-fails.mk | 12 ++++++
> .../git-extra-download/package/ok/ok.hash | 3 ++
> .../git-extra-download/package/ok/ok.mk | 12 ++++++
> support/testing/tests/download/gitremote.py | 43 +++----------------
> .../testing/tests/download/http-server/extra | 1 +
> support/testing/tests/download/httpserver.py | 14 ++++++
> support/testing/tests/download/test_git.py | 21 +++++++++
> 15 files changed, 135 insertions(+), 36 deletions(-)
> create mode 100644 support/testing/infra/server.py
> create mode 100644 support/testing/tests/download/br2-external/git-extra-download/Config.in
> create mode 100644 support/testing/tests/download/br2-external/git-extra-download/external.desc
> create mode 100644 support/testing/tests/download/br2-external/git-extra-download/external.mk
> create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.hash
> create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.mk
> create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.hash
> create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.mk
> create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.hash
> create mode 100644 support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.mk
> create mode 100644 support/testing/tests/download/http-server/extra
> create mode 100644 support/testing/tests/download/httpserver.py
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 61ccfdfd2d..53f7c72ac3 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -263,6 +263,7 @@ tests.core.test_rootfs_overlay.TestRootfsOverlay: *runtime_test
> tests.core.test_timezone.TestGlibcAllTimezone: *runtime_test
> tests.core.test_timezone.TestGlibcNonDefaultLimitedTimezone: *runtime_test
> tests.core.test_timezone.TestNoTimezone: *runtime_test
> +tests.download.test_git.TestGitExtraDownload: *runtime_test
> tests.download.test_git.TestGitHash: *runtime_test
> tests.download.test_git.TestGitRefs: *runtime_test
> tests.fs.test_ext.TestExt2: *runtime_test
> diff --git a/support/testing/infra/server.py b/support/testing/infra/server.py
> new file mode 100644
> index 0000000000..0fbffdf846
> --- /dev/null
> +++ b/support/testing/infra/server.py
> @@ -0,0 +1,39 @@
> +# subprocess does not kill the child daemon when a test case fails by raising
> +# an exception. So use pexpect instead.
> +import pexpect
> +
> +import infra
> +
> +
> +class Server(object):
> + def __init__(self, builddir, serveddir, logtofile, daemon_cmd, port_arg, port_initial, port_last, good_msg, bad_msg):
> + """
> + Start a local server.
> +
> + In order to support test cases in parallel, select the port the
> + server will listen to in runtime. Since there is no reliable way
> + to allocate the port prior to starting the server (another
> + process in the host machine can use the port between it is
> + selected from a list and it is really allocated to the server)
> + try to start the server in a port and in the case it is already
> + in use, try the next one in the allowed range.
> + """
> + self.name = self.__class__.__name__.lower()
> + self.daemon = None
> + self.port = None
> + self.logfile = infra.open_log_file(builddir, self.name, logtofile)
> +
> + for port in range(port_initial, port_last + 1):
> + cmd = daemon_cmd + [port_arg.format(port=port)]
> + self.logfile.write("> starting {} with 'cd {} && {}'\n".format(self.name, serveddir, " ".join(cmd)))
> + self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile, cwd=serveddir)
> + ret = self.daemon.expect(good_msg + bad_msg)
> + if ret < len(good_msg):
> + self.port = port
> + return
> + raise SystemError("Could not find a free port to run {}".format(self.name))
> +
> + def stop(self):
> + if self.daemon is None:
> + return
> + self.daemon.terminate(force=True)
> diff --git a/support/testing/tests/download/br2-external/git-extra-download/Config.in b/support/testing/tests/download/br2-external/git-extra-download/Config.in
> new file mode 100644
> index 0000000000..e69de29bb2
> diff --git a/support/testing/tests/download/br2-external/git-extra-download/external.desc b/support/testing/tests/download/br2-external/git-extra-download/external.desc
> new file mode 100644
> index 0000000000..6ebd5a534d
> --- /dev/null
> +++ b/support/testing/tests/download/br2-external/git-extra-download/external.desc
> @@ -0,0 +1 @@
> +name: GIT_EXTRA_DOWNLOAD
> diff --git a/support/testing/tests/download/br2-external/git-extra-download/external.mk b/support/testing/tests/download/br2-external/git-extra-download/external.mk
> new file mode 100644
> index 0000000000..c6080f571b
> --- /dev/null
> +++ b/support/testing/tests/download/br2-external/git-extra-download/external.mk
> @@ -0,0 +1,6 @@
> +include $(sort $(wildcard $(BR2_EXTERNAL_GIT_EXTRA_DOWNLOAD_PATH)/package/*/*.mk))
> +
> +# Get the git server port number from the test infra
> +GITREMOTE_PORT_NUMBER ?= 9418
> +# Get the http server port number from the test infra
> +HTTP_SERVER_PORT_NUMBER ?= 8000
> diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.hash b/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.hash
> new file mode 100644
> index 0000000000..89b1b1f682
> --- /dev/null
> +++ b/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.hash
> @@ -0,0 +1,3 @@
> +sha256 1e6bc73fabdcce8857361e36e3c812c4ee42d8ffa30d56492bc56f8fcad7eb90 extra-fails-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
> +sha256 b9e68e1bea3e5b19ca6b2f98b73a54b73daafaa250484902e09982e07a12e733 notfound
> +sha256 da68f54607d5f5644954096ce1597c006c5bb9f2497e07441bf064b81003ef8a file
> diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.mk b/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.mk
> new file mode 100644
> index 0000000000..dcf3ebe7a6
> --- /dev/null
> +++ b/support/testing/tests/download/br2-external/git-extra-download/package/extra-fails/extra-fails.mk
> @@ -0,0 +1,12 @@
> +################################################################################
> +#
> +# extra-fails
> +#
> +################################################################################
> +
> +EXTRA_FAILS_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
> +EXTRA_FAILS_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
> +EXTRA_FAILS_EXTRA_DOWNLOADS = http://localhost:$(HTTP_SERVER_PORT_NUMBER)/notfound
> +EXTRA_FAILS_LICENSE_FILES = file
> +
> +$(eval $(generic-package))
> diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.hash b/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.hash
> new file mode 100644
> index 0000000000..ad81243751
> --- /dev/null
> +++ b/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.hash
> @@ -0,0 +1,3 @@
> +sha256 cd6851ef519a83345e4547f376b33d6bbd622d4ccbb234af9997c43854c602de main-fails-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
> +sha256 b9e68e1bea3e5b19ca6b2f98b73a54b73daafaa250484902e09982e07a12e733 extra
> +sha256 da68f54607d5f5644954096ce1597c006c5bb9f2497e07441bf064b81003ef8a file
> diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.mk b/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.mk
> new file mode 100644
> index 0000000000..022bb37cbb
> --- /dev/null
> +++ b/support/testing/tests/download/br2-external/git-extra-download/package/main-fails/main-fails.mk
> @@ -0,0 +1,12 @@
> +################################################################################
> +#
> +# main-fails
> +#
> +################################################################################
> +
> +MAIN_FAILS_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
> +MAIN_FAILS_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/notfound.git
> +MAIN_FAILS_EXTRA_DOWNLOADS = http://localhost:$(HTTP_SERVER_PORT_NUMBER)/extra
> +MAIN_FAILS_LICENSE_FILES = file
> +
> +$(eval $(generic-package))
> diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.hash b/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.hash
> new file mode 100644
> index 0000000000..366940754b
> --- /dev/null
> +++ b/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.hash
> @@ -0,0 +1,3 @@
> +sha256 737b4fd21506dbaa34cedc93c53c1ebb1f950db2c7644572bb006ae9297961da ok-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
> +sha256 b9e68e1bea3e5b19ca6b2f98b73a54b73daafaa250484902e09982e07a12e733 extra
> +sha256 da68f54607d5f5644954096ce1597c006c5bb9f2497e07441bf064b81003ef8a file
> diff --git a/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.mk b/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.mk
> new file mode 100644
> index 0000000000..ce102cb0de
> --- /dev/null
> +++ b/support/testing/tests/download/br2-external/git-extra-download/package/ok/ok.mk
> @@ -0,0 +1,12 @@
> +################################################################################
> +#
> +# ok
> +#
> +################################################################################
> +
> +OK_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d
> +OK_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git
> +OK_EXTRA_DOWNLOADS = http://localhost:$(HTTP_SERVER_PORT_NUMBER)/extra
> +OK_LICENSE_FILES = file
> +
> +$(eval $(generic-package))
> diff --git a/support/testing/tests/download/gitremote.py b/support/testing/tests/download/gitremote.py
> index 60bc49fbf8..9766b19ce0 100644
> --- a/support/testing/tests/download/gitremote.py
> +++ b/support/testing/tests/download/gitremote.py
> @@ -1,44 +1,15 @@
> -# subprocess does not kill the child daemon when a test case fails by raising
> -# an exception. So use pexpect instead.
> -import pexpect
> -
> -import infra
> +from infra.server import Server
>
> GIT_REMOTE_PORT_INITIAL = 9418
> GIT_REMOTE_PORT_LAST = GIT_REMOTE_PORT_INITIAL + 99
>
>
> -class GitRemote(object):
> +class GitRemote(Server):
> def __init__(self, builddir, serveddir, logtofile):
> - """
> - Start a local git server.
> -
> - In order to support test cases in parallel, select the port the
> - server will listen to in runtime. Since there is no reliable way
> - to allocate the port prior to starting the server (another
> - process in the host machine can use the port between it is
> - selected from a list and it is really allocated to the server)
> - try to start the server in a port and in the case it is already
> - in use, try the next one in the allowed range.
> - """
> - self.daemon = None
> - self.port = None
> - self.logfile = infra.open_log_file(builddir, "gitremote", logtofile)
> -
> daemon_cmd = ["git", "daemon", "--reuseaddr", "--verbose", "--listen=localhost", "--export-all",
> "--base-path={}".format(serveddir)]
> - for port in range(GIT_REMOTE_PORT_INITIAL, GIT_REMOTE_PORT_LAST + 1):
> - cmd = daemon_cmd + ["--port={port}".format(port=port)]
> - self.logfile.write("> starting git remote with '{}'\n".format(" ".join(cmd)))
> - self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile)
> - ret = self.daemon.expect(["Ready to rumble",
> - "Address already in use"])
> - if ret == 0:
> - self.port = port
> - return
> - raise SystemError("Could not find a free port to run git remote")
> -
> - def stop(self):
> - if self.daemon is None:
> - return
> - self.daemon.terminate(force=True)
> + super(GitRemote, self).__init__(
> + builddir, serveddir, logtofile,
> + daemon_cmd, "--port={port}", GIT_REMOTE_PORT_INITIAL, GIT_REMOTE_PORT_LAST,
> + ["Ready to rumble"],
> + ["Address already in use"])
> diff --git a/support/testing/tests/download/http-server/extra b/support/testing/tests/download/http-server/extra
> new file mode 100644
> index 0000000000..8e27be7d61
> --- /dev/null
> +++ b/support/testing/tests/download/http-server/extra
> @@ -0,0 +1 @@
> +text
> diff --git a/support/testing/tests/download/httpserver.py b/support/testing/tests/download/httpserver.py
> new file mode 100644
> index 0000000000..9f4f947304
> --- /dev/null
> +++ b/support/testing/tests/download/httpserver.py
> @@ -0,0 +1,14 @@
> +from infra.server import Server
> +
> +HTTP_SERVER_PORT_INITIAL = 8000
> +HTTP_SERVER_PORT_LAST = HTTP_SERVER_PORT_INITIAL + 99
> +
> +
> +class HttpServer(Server):
> + def __init__(self, builddir, serveddir, logtofile):
> + daemon_cmd = ["python", "-m", "SimpleHTTPServer"]
> + super(HttpServer, self).__init__(
> + builddir, serveddir, logtofile,
> + daemon_cmd, "{port}", HTTP_SERVER_PORT_INITIAL, HTTP_SERVER_PORT_LAST,
> + ["Serving HTTP"],
> + ["Address already in use"])
> diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py
> index 162c03623b..e0abc3fa65 100644
> --- a/support/testing/tests/download/test_git.py
> +++ b/support/testing/tests/download/test_git.py
> @@ -3,6 +3,7 @@ import shutil
>
> import infra
> from gitremote import GitRemote
> +from httpserver import HttpServer
>
>
> class GitTestBase(infra.basetest.BRTest):
> @@ -12,13 +13,19 @@ class GitTestBase(infra.basetest.BRTest):
> """
> gitremotedir = infra.filepath("tests/download/git-remote")
> gitremote = None
> + httpserverdir = None
> + httpserver = None
>
> def setUp(self):
> super(GitTestBase, self).setUp()
> self.gitremote = GitRemote(self.builddir, self.gitremotedir, self.logtofile)
> + if self.httpserverdir:
> + self.httpserver = HttpServer(self.builddir, self.httpserverdir, self.logtofile)
>
> def tearDown(self):
> self.show_msg("Cleaning up")
> + if self.httpserver:
> + self.httpserver.stop()
> if self.gitremote:
> self.gitremote.stop()
> if self.b and not self.keepbuilds:
> @@ -42,11 +49,25 @@ class GitTestBase(infra.basetest.BRTest):
> shutil.rmtree(dl_dir)
> env = {"BR2_DL_DIR": dl_dir,
> "GITREMOTE_PORT_NUMBER": str(self.gitremote.port)}
> + if self.httpserver:
> + env["HTTP_SERVER_PORT_NUMBER"] = str(self.httpserver.port)
> self.b.build(["{}-dirclean".format(package),
> "{}-legal-info".format(package)],
> env)
>
>
> +class TestGitExtraDownload(GitTestBase):
> + br2_external = [infra.filepath("tests/download/br2-external/git-extra-download")]
> + httpserverdir = infra.filepath("tests/download/http-server")
> +
> + def test_run(self):
> + with self.assertRaises(SystemError):
> + self.check_download("extra-fails")
> + with self.assertRaises(SystemError):
> + self.check_download("main-fails")
> + self.check_download("ok")
> +
> +
> class TestGitHash(GitTestBase):
> br2_external = [infra.filepath("tests/download/br2-external/git-hash")]
>
>
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 09/10] testing/tests/download: test git special ref
2019-02-06 10:05 ` Arnout Vandecappelle
@ 2019-02-18 2:46 ` Ricardo Martincoski
2019-02-19 9:01 ` Arnout Vandecappelle
0 siblings, 1 reply; 108+ messages in thread
From: Ricardo Martincoski @ 2019-02-18 2:46 UTC (permalink / raw)
To: buildroot
Hello,
On Wed, Feb 06, 2019 at 08:05 AM, Arnout Vandecappelle wrote:
> On 12/05/2018 04:58, Ricardo Martincoski wrote:
>> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>>
>> Add a special ref to the static repo and check on the git refs test case
>> the download of a git package:
>> - with the name of a special ref as version;
>
> We might remove support for getting special refs by name, but keep on
> supporting getting special refs by sha1. So I've changed the test into getting
> it by sha1 instead.
OK... but it depends on a recent git version to work.
The same does not occur when getting special refs by name.
When using recent versions of git the download of the sha1 of a special ref
works because of this line in support/download/git:
_git fetch origin "'${cset}:${cset}'"
I tested using:
git version 2.17.1
The git version installed in our docker image does not support it.
I am not sure what is the best fix for the TestGitRefs failure in GitLab CI.
I would change the test to use special ref by name, and *when* we change the
behavior, we change the test in the same commit to expect it to fail.
>
> Ideally there should also be a check for getting the parent of the special ref
> by sha1, but I think that actually does *not* work now.
I expect it to not work. But I didn't tested recently.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 09/10] testing/tests/download: test git special ref
2019-02-18 2:46 ` Ricardo Martincoski
@ 2019-02-19 9:01 ` Arnout Vandecappelle
2019-02-26 3:01 ` Ricardo Martincoski
0 siblings, 1 reply; 108+ messages in thread
From: Arnout Vandecappelle @ 2019-02-19 9:01 UTC (permalink / raw)
To: buildroot
On 18/02/2019 03:46, Ricardo Martincoski wrote:
> Hello,
>
> On Wed, Feb 06, 2019 at 08:05 AM, Arnout Vandecappelle wrote:
>
>> On 12/05/2018 04:58, Ricardo Martincoski wrote:
>>> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>>>
>>> Add a special ref to the static repo and check on the git refs test case
>>> the download of a git package:
>>> - with the name of a special ref as version;
>>
>> We might remove support for getting special refs by name, but keep on
>> supporting getting special refs by sha1. So I've changed the test into getting
>> it by sha1 instead.
>
> OK... but it depends on a recent git version to work.
Ah, darn, I forgot that...
> The same does not occur when getting special refs by name.
>
> When using recent versions of git the download of the sha1 of a special ref
> works because of this line in support/download/git:
> _git fetch origin "'${cset}:${cset}'"
So, IIRC, Yann had an idea for how to rewrite this in a way that we would (1)
no longer support branch names and (2) we would support hashes that refer to
special refs or ancestors of special refs. But I don't remember what it was.
Maybe it was
_git fetch origin '+refs/*:refs/remotes/origin/*' '+refs/tags/*:refs/tags/*'
That way (I think) a branch name would no longer be usable (because when doing
the checkout the origin/ bit would be missing), but a tag name still would be
usable.
> I tested using:
> git version 2.17.1
>
> The git version installed in our docker image does not support it.
>
> I am not sure what is the best fix for the TestGitRefs failure in GitLab CI.
> I would change the test to use special ref by name, and *when* we change the
> behavior, we change the test in the same commit to expect it to fail.
Or maybe just remove the test for now.
Regards,
Arnout
>
>>
>> Ideally there should also be a check for getting the parent of the special ref
>> by sha1, but I think that actually does *not* work now.
>
> I expect it to not work. But I didn't tested recently.
>
> Regards,
> Ricardo
>
^ permalink raw reply [flat|nested] 108+ messages in thread
* [Buildroot] [PATCH v5 09/10] testing/tests/download: test git special ref
2019-02-19 9:01 ` Arnout Vandecappelle
@ 2019-02-26 3:01 ` Ricardo Martincoski
0 siblings, 0 replies; 108+ messages in thread
From: Ricardo Martincoski @ 2019-02-26 3:01 UTC (permalink / raw)
To: buildroot
Hello,
On Tue, Feb 19, 2019 at 06:01 AM, Arnout Vandecappelle wrote:
> On 18/02/2019 03:46, Ricardo Martincoski wrote:
>>
>> On Wed, Feb 06, 2019 at 08:05 AM, Arnout Vandecappelle wrote:
>>
>>> On 12/05/2018 04:58, Ricardo Martincoski wrote:
>>>> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>>>>
>>>> Add a special ref to the static repo and check on the git refs test case
>>>> the download of a git package:
>>>> - with the name of a special ref as version;
>>>
>>> We might remove support for getting special refs by name, but keep on
>>> supporting getting special refs by sha1. So I've changed the test into getting
>>> it by sha1 instead.
>>
>> OK... but it depends on a recent git version to work.
>
> Ah, darn, I forgot that...
>
>> The same does not occur when getting special refs by name.
>>
>> When using recent versions of git the download of the sha1 of a special ref
>> works because of this line in support/download/git:
>> _git fetch origin "'${cset}:${cset}'"
>
> So, IIRC, Yann had an idea for how to rewrite this in a way that we would (1)
> no longer support branch names and (2) we would support hashes that refer to
> special refs or ancestors of special refs. But I don't remember what it was.
> Maybe it was
>
> _git fetch origin '+refs/*:refs/remotes/origin/*' '+refs/tags/*:refs/tags/*'
>
> That way (I think) a branch name would no longer be usable (because when doing
> the checkout the origin/ bit would be missing), but a tag name still would be
> usable.
I didn't tested it but I like the idea.
>
>> I tested using:
>> git version 2.17.1
>>
>> The git version installed in our docker image does not support it.
>>
>> I am not sure what is the best fix for the TestGitRefs failure in GitLab CI.
>> I would change the test to use special ref by name, and *when* we change the
>> behavior, we change the test in the same commit to expect it to fail.
>
> Or maybe just remove the test for now.
OK. When new code is added to support hashes that refer to special refs the test
can be added again.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 108+ messages in thread
end of thread, other threads:[~2019-02-26 3:01 UTC | newest]
Thread overview: 108+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-11-01 19:33 [Buildroot] [PATCH 1/3] support/download/git: log checked out sha1 Ricardo Martincoski
2016-11-01 19:33 ` [Buildroot] [PATCH 2/3] test/support/download/git: new test Ricardo Martincoski
2016-11-01 21:27 ` Ricardo Martincoski
2016-11-06 0:19 ` Arnout Vandecappelle
2016-11-06 0:51 ` Arnout Vandecappelle
2016-12-02 2:34 ` Ricardo Martincoski
2016-12-05 17:26 ` Arnout Vandecappelle
2016-12-06 1:35 ` Ricardo Martincoski
2016-11-06 1:24 ` Arnout Vandecappelle
2016-11-06 12:13 ` Henrique Marks
2016-11-06 15:34 ` Arnout Vandecappelle
2017-08-26 22:20 ` [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build() Ricardo Martincoski
2017-08-26 22:20 ` [Buildroot] [next v2 2/7] testing/infra/builder: build with target and environment Ricardo Martincoski
2017-10-06 20:42 ` Arnout Vandecappelle
2017-10-06 21:02 ` Arnout Vandecappelle
2017-10-23 2:34 ` Ricardo Martincoski
2017-08-26 22:20 ` [Buildroot] [next v2 3/7] testing/infra/builder: allow to override logfile Ricardo Martincoski
2017-10-06 20:50 ` Arnout Vandecappelle
2017-10-23 2:32 ` Ricardo Martincoski
2017-08-26 22:20 ` [Buildroot] [next v2 4/7] testing/tests/download: add infra for git tests Ricardo Martincoski
2017-10-06 21:30 ` Arnout Vandecappelle
2017-10-23 2:35 ` Ricardo Martincoski
2017-10-23 8:18 ` Arnout Vandecappelle
2017-10-29 4:00 ` Ricardo Martincoski
2017-08-26 22:20 ` [Buildroot] [next v2 5/7] testing/tests/download: add git hash tests Ricardo Martincoski
2017-08-26 22:38 ` Ricardo Martincoski
2017-08-27 4:00 ` Baruch Siach
2017-10-06 21:36 ` Arnout Vandecappelle
2017-10-06 21:44 ` Arnout Vandecappelle
2017-10-23 2:36 ` Ricardo Martincoski
2017-08-26 22:20 ` [Buildroot] [next v2 6/7] testing/tests/download: add test for sha1 as git ref Ricardo Martincoski
2017-10-06 21:57 ` Arnout Vandecappelle
2017-10-23 2:38 ` Ricardo Martincoski
2017-08-26 22:20 ` [Buildroot] [next v2 7/7] testing/tests/download: add test for git submodule Ricardo Martincoski
2017-10-06 20:31 ` [Buildroot] [next v2 1/7] testing/infra/builder: split configure() from build() Arnout Vandecappelle
2017-10-23 2:31 ` Ricardo Martincoski
2017-10-29 14:05 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 1/9] testing/infra/builder: call make with empty env Ricardo Martincoski
2018-04-01 17:58 ` Thomas Petazzoni
2017-10-29 14:06 ` [Buildroot] [PATCH v3 2/9] testing/infra/builder: split configure() from build() Ricardo Martincoski
2018-04-01 17:59 ` Thomas Petazzoni
2018-04-01 21:32 ` Ricardo Martincoski
2018-04-01 21:37 ` Thomas Petazzoni
2017-10-29 14:06 ` [Buildroot] [PATCH v3 3/9] testing/infra/builder: build with target and environment Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 4/9] testing/infra: split runtime test from BRTest Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 5/9] testing/infra/basetest: support br2-external Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 6/9] testing/tests/download: add git hash test Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 7/9] testing/tests/download: test case for git refs Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 8/9] testing/tests/download: test git branch Ricardo Martincoski
2017-10-29 14:06 ` [Buildroot] [PATCH v3 9/9] testing/tests/download: test git submodules Ricardo Martincoski
2018-04-25 20:58 ` [Buildroot] [PATCH v3 0/9] tests for git download infra (series 1/3) Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/infra/builder: build with target and environment Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/infra: split runtime test from BRTest Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/infra/basetest: support br2-external Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: add git hash test Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test case for git refs Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git branch Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git submodules Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git tag Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git tag/branch precedence Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git special ref Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [PATCH v4] testing/tests/download: test git branch with slash Ricardo Martincoski
2018-04-29 14:33 ` [Buildroot] [RFC PATCH v4] support/testing: test extra download with site method git Ricardo Martincoski
2018-04-30 1:38 ` [Buildroot] [PATCH v4] tests for git download infra v4 Ricardo Martincoski
2018-05-11 3:09 ` Ricardo Martincoski
2018-05-12 2:58 ` [Buildroot] [PATCH v5 00/10] tests for git download infra v5 Ricardo Martincoski
2018-05-12 2:58 ` [Buildroot] [PATCH v5 01/10] testing/infra/builder: build with target and environment Ricardo Martincoski
2018-05-12 2:58 ` [Buildroot] [PATCH v5 02/10] testing/infra: split runtime test from BRTest Ricardo Martincoski
2019-02-04 15:55 ` Arnout Vandecappelle
2019-02-04 18:19 ` Matthew Weber
2019-02-04 19:42 ` Matthew Weber
2019-02-05 1:19 ` Ricardo Martincoski
2019-02-05 8:29 ` Arnout Vandecappelle
2019-02-05 1:00 ` Ricardo Martincoski
2019-02-05 1:12 ` Matthew Weber
2019-02-05 1:47 ` Ricardo Martincoski
2019-02-05 9:28 ` Matthew Weber
2018-05-12 2:58 ` [Buildroot] [PATCH v5 03/10] testing/infra/basetest: support br2-external Ricardo Martincoski
2018-05-12 2:58 ` [Buildroot] [PATCH v5 04/10] testing/tests/download: add git hash test Ricardo Martincoski
2019-02-04 15:52 ` Arnout Vandecappelle
2019-02-05 0:52 ` Ricardo Martincoski
2019-02-05 8:09 ` Arnout Vandecappelle
2019-02-05 8:55 ` Peter Korsgaard
2018-05-12 2:58 ` [Buildroot] [PATCH v5 05/10] testing/tests/download: test case for git refs Ricardo Martincoski
2019-02-04 19:48 ` Arnout Vandecappelle
2019-02-05 0:53 ` Ricardo Martincoski
2018-05-12 2:58 ` [Buildroot] [PATCH v5 06/10] testing/tests/download: test git branch Ricardo Martincoski
2019-02-05 9:42 ` Arnout Vandecappelle
2018-05-12 2:58 ` [Buildroot] [PATCH v5 07/10] testing/tests/download: test git submodules Ricardo Martincoski
2019-02-05 10:03 ` Arnout Vandecappelle
2019-02-06 9:08 ` Arnout Vandecappelle
2019-02-06 9:14 ` Yann E. MORIN
2018-05-12 2:58 ` [Buildroot] [PATCH v5 08/10] testing/tests/download: test git tag Ricardo Martincoski
2019-02-06 10:03 ` Arnout Vandecappelle
2018-05-12 2:58 ` [Buildroot] [PATCH v5 09/10] testing/tests/download: test git special ref Ricardo Martincoski
2019-02-06 10:05 ` Arnout Vandecappelle
2019-02-18 2:46 ` Ricardo Martincoski
2019-02-19 9:01 ` Arnout Vandecappelle
2019-02-26 3:01 ` Ricardo Martincoski
2018-05-12 2:58 ` [Buildroot] [PATCH v5 10/10] support/testing: test extra download with site method git Ricardo Martincoski
2019-02-06 10:34 ` Arnout Vandecappelle
2016-11-01 19:33 ` [Buildroot] [PATCH 3/3] support/download/git: do not use git clone Ricardo Martincoski
2016-11-06 1:19 ` Arnout Vandecappelle
2016-11-10 0:07 ` Ricardo Martincoski
2016-11-18 7:33 ` Ricardo Martincoski
2016-11-05 21:50 ` [Buildroot] [PATCH 1/3] support/download/git: log checked out sha1 Arnout Vandecappelle
2016-11-06 23:17 ` Ricardo Martincoski
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.