Re: [Qemu-devel] [PATCH v3 00/13] tests: Introducing docker tests

From: "Alex Bennée" <alex.bennee@linaro.org>
To: Fam Zheng <famz@redhat.com>
Cc: kwolf@redhat.com, peter.maydell@linaro.org, sw@weilnetz.de,
	qemu-devel@nongnu.org, stefanha@redhat.com,
	Paolo Bonzini <pbonzini@redhat.com>,
	jsnow@redhat.com, david@gibson.dropbear.id.au
Subject: Re: [Qemu-devel] [PATCH v3 00/13] tests: Introducing docker tests
Date: Fri, 11 Mar 2016 16:16:58 +0000	[thread overview]
Message-ID: <87fuvxt2f9.fsf@linaro.org> (raw)
In-Reply-To: <1457086720-30391-1-git-send-email-famz@redhat.com>

Fam Zheng <famz@redhat.com> writes:

> v3 changes:

I think we are almost there. There a just a few tweaks to be made to
help text and prompts. Can you ensure that all examples in commit
messages and help text actually do run as expected?

Is it proposed this goes through Daniel's treee?

>   - Merge all docker_* tools into docker.py as subcommands, and add simple help
>     texts; clean up docker.py a bit.
>   - For ease of management of the series, squashed Alex's COPY_SRC patch into
>     patch 2.
>   - Pick up Alex's tweaks on Makefile changes.
>   - Change the way we copy source. Now we send source tarballs to the
>     container, and untar it there. Beside qemu.tgz, also copy dtc.tgz and
>     pixman.tgz if submodules are initialized in the tree.  The first is
>     required by mingw test.
>   - Update test runner to adapt to above change.
>   - Tweak "make docker" help text. Dropped "docker-@IMAGE" because it is
>     equivalent to "make docker-test IMAGES=XXX". Too many targets will be hard
>     to memorize, and we can always add more shortcuts if desired.
>   - Drop "PAUSE=1" env and add "DEBUG=1", which will also enable networking,
>     and drop to shell when test fails.
>   - Add "NOCACHE=1" env var to add "--no-cache" to "docker build" command,
>     which is useful in certain cases to workaround image build failure. For
>     example in ubuntu when "apt-get update" is cached, "apt-get install xxx"
>     can get 404.
>   - Add libfdt-devel to images.
>   - Add epel, and ccache in centos6.
>   - Add "TARGET_LIST=" env var.
>   - Add "sparse" from multiverse for ubuntu, which is used by travis tool.
>
> This series adds a new "docker" make target family to run tests in created
> docker containers.
>
> To begin with, this can be a place to store standard env/command combinations to
> build and test QEMU.
>
> Secondly, CI usually provides "docker" capability, where we specify
> standard/repeatable test environments, and run tests in them.  However, what
> tests to cover is better maintained in-tree, in order to keep in sync with the
> code development.
>
> Lastly, this makes it very simple for developers to replicate such tests
> themselves.
>
>
> Fam Zheng (13):
>   tests: Add utilities for docker testing
>   Makefile: Rules for docker testing
>   docker: Add images
>   docker: Add test runner
>   docker: Add common.rc
>   docker: Add quick test
>   docker: Add full test
>   docker: Add clang test
>   docker: Add mingw test
>   docker: Add travis tool
>   docs: Add text for tests/docker in build-system.txt
>   .gitignore: Ignore temporary dockerfile
>   MAINTAINERS: Add tests/docker
>
>  .gitignore                              |   1 +
>  MAINTAINERS                             |   7 ++
>  Makefile                                |   4 +-
>  docs/build-system.txt                   |   5 +
>  tests/docker/Makefile.include           | 121 +++++++++++++++++++++
>  tests/docker/common.rc                  |  31 ++++++
>  tests/docker/docker.py                  | 180 ++++++++++++++++++++++++++++++++
>  tests/docker/dockerfiles/centos6.docker |   6 ++
>  tests/docker/dockerfiles/fedora.docker  |   7 ++
>  tests/docker/dockerfiles/ubuntu.docker  |  11 ++
>  tests/docker/run                        |  58 ++++++++++
>  tests/docker/test-clang                 |  25 +++++
>  tests/docker/test-full                  |  17 +++
>  tests/docker/test-mingw                 |  34 ++++++
>  tests/docker/test-quick                 |  19 ++++
>  tests/docker/travis                     |  21 ++++
>  tests/docker/travis.py                  |  48 +++++++++
>  17 files changed, 594 insertions(+), 1 deletion(-)
>  create mode 100644 tests/docker/Makefile.include
>  create mode 100755 tests/docker/common.rc
>  create mode 100755 tests/docker/docker.py
>  create mode 100644 tests/docker/dockerfiles/centos6.docker
>  create mode 100644 tests/docker/dockerfiles/fedora.docker
>  create mode 100644 tests/docker/dockerfiles/ubuntu.docker
>  create mode 100755 tests/docker/run
>  create mode 100755 tests/docker/test-clang
>  create mode 100755 tests/docker/test-full
>  create mode 100755 tests/docker/test-mingw
>  create mode 100755 tests/docker/test-quick
>  create mode 100755 tests/docker/travis
>  create mode 100755 tests/docker/travis.py

--
Alex Bennée