All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Alex Bennée" <alex.bennee@linaro.org>
To: peter.maydell@linaro.org
Cc: qemu-devel@nongnu.org, "Philippe Mathieu-Daudé" <f4bug@amsat.org>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	"Fam Zheng" <famz@redhat.com>
Subject: [Qemu-devel] [PULL 04/21] docker: rebuild image if 'extra files' checksum does not match
Date: Wed, 21 Jun 2017 15:47:57 +0100	[thread overview]
Message-ID: <20170621144814.15324-5-alex.bennee@linaro.org> (raw)
In-Reply-To: <20170621144814.15324-1-alex.bennee@linaro.org>

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 10fa907905..e707e5bcca 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -38,6 +38,9 @@ def _text_checksum(text):
     """Calculate a digest string unique to the text content"""
     return hashlib.sha1(text).hexdigest()
 
+def _file_checksum(filename):
+    return _text_checksum(open(filename, 'rb').read())
+
 def _guess_docker_command():
     """ Guess a working docker command or raise exception if not found"""
     commands = [["docker"], ["sudo", "-n", "docker"]]
@@ -154,7 +157,7 @@ class Docker(object):
         return labels.get("com.qemu.dockerfile-checksum", "")
 
     def build_image(self, tag, docker_dir, dockerfile,
-                    quiet=True, user=False, argv=None):
+                    quiet=True, user=False, argv=None, extra_files_cksum=[]):
         if argv == None:
             argv = []
 
@@ -170,7 +173,8 @@ class Docker(object):
 
         tmp_df.write("\n")
         tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" %
-                     _text_checksum(dockerfile))
+                     _text_checksum("\n".join([dockerfile] +
+                                    extra_files_cksum)))
         tmp_df.flush()
 
         self._do(["build", "-t", tag, "-f", tmp_df.name] + argv + \
@@ -276,16 +280,22 @@ class BuildCommand(SubCommand):
 
             # Copy any extra files into the Docker context. These can be
             # included by the use of the ADD directive in the Dockerfile.
+            cksum = []
             if args.include_executable:
+                # FIXME: there is no checksum of this executable and the linked
+                # libraries, once the image built any change of this executable
+                # or any library won't trigger another build.
                 _copy_binary_with_libs(args.include_executable, docker_dir)
             for filename in args.extra_files or []:
                 _copy_with_mkdir(filename, docker_dir)
+                cksum += [_file_checksum(filename)]
 
             argv += ["--build-arg=" + k.lower() + "=" + v
                         for k, v in os.environ.iteritems()
                         if k.lower() in FILTERED_ENV_NAMES]
             dkr.build_image(tag, docker_dir, dockerfile,
-                            quiet=args.quiet, user=args.user, argv=argv)
+                            quiet=args.quiet, user=args.user, argv=argv,
+                            extra_files_cksum=cksum)
 
             rmtree(docker_dir)
 
-- 
2.13.0

  parent reply	other threads:[~2017-06-21 14:47 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-06-21 14:47 [Qemu-devel] [PULL 00/21] Docker and shippable updates Alex Bennée
2017-06-21 14:47 ` [Qemu-devel] [PULL 01/21] docker: update qemu:debian base following stretch release Alex Bennée
2017-06-21 14:47 ` [Qemu-devel] [PULL 02/21] docker: let _copy_with_mkdir() sub_path argument be optional Alex Bennée
2017-06-21 14:47 ` [Qemu-devel] [PULL 03/21] docker: add --include-files argument to 'build' command Alex Bennée
2017-06-21 14:47 ` Alex Bennée [this message]
2017-06-21 14:47 ` [Qemu-devel] [PULL 05/21] docker: install ca-certificates package in base image Alex Bennée
2017-06-21 14:47 ` [Qemu-devel] [PULL 06/21] docker: use better regex to generate deb-src entries Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 07/21] docker: use eatmydata, install common build packages in base image Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 08/21] docker: use eatmydata in debian armhf image Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 09/21] docker: use eatmydata in debian arm64 image Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 10/21] docker: add extra libs to armhf target to extend codebase coverage Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 11/21] docker: add extra libs to arm64 " Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 12/21] docker: add extra libs to s390x " Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 13/21] docker: add mipsel build target Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 14/21] shippable: use C locale to simplify console output Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 15/21] shippable: build using all available cpus Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 16/21] shippable: do not initialize submodules automatically Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 17/21] shippable: be verbose while building docker images Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 18/21] shippable: add armeb-linux-user target Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 19/21] shippable: add mipsel target Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 20/21] MAINTAINERS: add Shippable automation platform URL Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 21/21] MAINTAINERS: self-appoint me as reviewer in build/test automation Alex Bennée
2017-06-22 14:09 ` [Qemu-devel] [PULL 00/21] Docker and shippable updates Peter Maydell
2017-06-22 14:58   ` Philippe Mathieu-Daudé
2017-06-22 15:13     ` Alex Bennée
2017-06-22 15:42       ` Philippe Mathieu-Daudé
2017-07-10 12:04         ` Philippe Mathieu-Daudé
2017-07-10 14:17           ` Alex Bennée
2017-07-10 14:22             ` Peter Maydell
2017-07-12 12:17               ` Philippe Mathieu-Daudé

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170621144814.15324-5-alex.bennee@linaro.org \
    --to=alex.bennee@linaro.org \
    --cc=f4bug@amsat.org \
    --cc=famz@redhat.com \
    --cc=peter.maydell@linaro.org \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.