From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59242) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e6W8K-00046D-SF for qemu-devel@nongnu.org; Mon, 23 Oct 2017 02:22:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e6W8G-00065f-T0 for qemu-devel@nongnu.org; Mon, 23 Oct 2017 02:22:44 -0400 Received: from mx1.redhat.com ([209.132.183.28]:34854) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e6W8G-00065S-K8 for qemu-devel@nongnu.org; Mon, 23 Oct 2017 02:22:40 -0400 Date: Mon, 23 Oct 2017 14:22:37 +0800 From: Fam Zheng Message-ID: <20171023062237.GG29815@lemon> References: <20171018073841.30062-1-famz@redhat.com> <150831409774.377.7665751901178175493@b58463cdfd5f> <20171018082836.GD9443@lemon> <20171018083549.GB9719@redhat.com> <20171020125107.GM21172@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20171020125107.GM21172@redhat.com> Subject: Re: [Qemu-devel] [PATCH] docker: Fix PATH for ccache List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Daniel P. Berrange" Cc: alex.bennee@linaro.org, qemu-devel@nongnu.org, f4bug@amsat.org On Fri, 10/20 13:51, Daniel P. Berrange wrote: > On Wed, Oct 18, 2017 at 09:35:49AM +0100, Daniel P. Berrange wrote: > > On Wed, Oct 18, 2017 at 04:28:36PM +0800, Fam Zheng wrote: > > > On Wed, 10/18 01:08, no-reply@patchew.org wrote: > > > > Your branch is up-to-date with 'origin/test'. > > > > Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' > > > > Cloning into '/var/tmp/patchew-tester-tmp-jhpnlaz0/src/docker-src.2017-10-18-04.03.32.32287/qemu.tar.vroot/dtc'... > > > > fatal: unable to connect to git.qemu-project.org: > > > > git.qemu-project.org[0: 172.99.69.163]: errno=Connection timed out > > > > > > > > fatal: clone of 'git://git.qemu-project.org/dtc.git' into submodule path '/var/tmp/patchew-tester-tmp-jhpnlaz0/src/docker-src.2017-10-18-04.03.32.32287/qemu.tar.vroot/dtc' failed > > > > Failed to clone 'dtc'. Retry scheduled > > > > Cloning into '/var/tmp/patchew-tester-tmp-jhpnlaz0/src/docker-src.2017-10-18-04.03.32.32287/qemu.tar.vroot/dtc'... > > > > fatal: unable to connect to git.qemu-project.org: > > > > git.qemu-project.org[0: 172.99.69.163]: errno=Connection timed out > > > > > > > > fatal: clone of 'git://git.qemu-project.org/dtc.git' into submodule path '/var/tmp/patchew-tester-tmp-jhpnlaz0/src/docker-src.2017-10-18-04.03.32.32287/qemu.tar.vroot/dtc' failed > > > > Failed to clone 'dtc' a second time, aborting > > > > failed to init submodule dtc > > > > > > So now patchew is less robust on network hiccups: a number of the same "git > > > submodule update --init" calls have succeeded above, so this is incidental. Any > > > idea how to improve that? Maybe if there is a checkout from $QEMU_SRC, clone/cp > > > from there instead of fetch from submodule remote? > > > > If patchew has unreliable connectivity to git.qemu.org then the best thing > > is probably have a background job that mirrors the git.qemu.org repos on > > the patchew server. Then have the test job modify .gitmodules to point to > > your local mirror. That way, network access is used during the background > > job only, and actual builds just get a local clone from the cache. > > I thought of a simpler way to address this. > > Currently patchew runs 4 docker jobs, and each one triggers creation of > the $(DOCKER_SRC_COPY) rule. This runs archive-source.sh 4 times, because > $(DOCKER_SRC_COPY) contains the current timestamp and so is different > each time we run it. > > If you override DOCKER_SRC_COPY with the current git change hash, then > you would end up using the same archive for all 4 jobs, and so only do > the checkouts once > > make docker-foo DOCKER_SRC_COPY=docker-src.`git rev-parse HEAD` I'm fine with that, just not sure about if this handles uncommitted changes correctly? It certainly isn't a concern on patchew, but may be a problem for developers. Your previous idea about mirroring submodules to local makes much sense, I can try it though. Fam