All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH OSSTEST 0/5] Tweaks to allow running non-master production instances
@ 2015-04-01 15:09 Ian Campbell
  2015-04-01 15:10 ` [PATCH OSSTEST 1/5] cri-args-hostslists: allow instance specific settings Ian Campbell
                   ` (4 more replies)
  0 siblings, 5 replies; 14+ messages in thread
From: Ian Campbell @ 2015-04-01 15:09 UTC (permalink / raw)
  To: Ian Jackson; +Cc: xen-devel

Now that the Cambridge osstest instance is no longer the XenProject
master instance it needs some tweaks to allow us to keep using it for
local testing.

Ian.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH OSSTEST 1/5] cri-args-hostslists: allow instance specific settings
  2015-04-01 15:09 [PATCH OSSTEST 0/5] Tweaks to allow running non-master production instances Ian Campbell
@ 2015-04-01 15:10 ` Ian Campbell
  2015-04-02 10:52   ` Ian Jackson
  2015-04-01 15:10 ` [PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances Ian Campbell
                   ` (3 subsequent siblings)
  4 siblings, 1 reply; 14+ messages in thread
From: Ian Campbell @ 2015-04-01 15:10 UTC (permalink / raw)
  To: ian.jackson; +Cc: Ian Campbell, xen-devel

In particular this new $HOME/.xen-osstest/cri-args-hostslists.settings
can contain things like "export
OSSTEST_CONFIG=production-config-cambridge" to tailor things for a
particular instance of osstest running in production mode.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 cri-args-hostlists | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/cri-args-hostlists b/cri-args-hostlists
index 04cd6be..bf847f8 100644
--- a/cri-args-hostlists
+++ b/cri-args-hostlists
@@ -16,7 +16,9 @@
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-
+if [ -e $HOME/.xen-osstest/cri-args-hostslists.settings ]; then
+     source $HOME/.xen-osstest/cri-args-hostslists.settings
+fi
 export OSSTEST_CONFIG=${OSSTEST_CONFIG:-production-config}
 
 check_stop_core () {
-- 
2.1.4

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances
  2015-04-01 15:09 [PATCH OSSTEST 0/5] Tweaks to allow running non-master production instances Ian Campbell
  2015-04-01 15:10 ` [PATCH OSSTEST 1/5] cri-args-hostslists: allow instance specific settings Ian Campbell
@ 2015-04-01 15:10 ` Ian Campbell
  2015-04-02 11:13   ` Ian Jackson
  2015-04-01 15:10 ` [PATCH OSSTEST 3/5] cambridge: Add a crontab Ian Campbell
                   ` (2 subsequent siblings)
  4 siblings, 1 reply; 14+ messages in thread
From: Ian Campbell @ 2015-04-01 15:10 UTC (permalink / raw)
  To: ian.jackson; +Cc: Ian Campbell, xen-devel

We want to arrange that the master XenProject instance continues to
test its own pretest branch while any downstream instances will pickup
changes from the master instance's production (i.e. tested) branch,
which is published at git://xenbits.xen.org/osstest.git#master.

Add a new configuration option OsstestUpstream which by default is
"git://xenbits.xen.org/osstest.git master" and which is cleared to
nothing on the master instance via production-config.

If the option is not set then the existing behaviour is unchanged.

If the option is set then osstest branch flights proceed by merging,
the referenced branch into the local pretest branch and testing the
result.

This must be done:
 - in a clone not in the main testing.git in order to avoid inserting
   merge conflict markers into the active set of scripts.
 - in a non-bare repo because git merge requires it.

$repos/osstest is a bare repo which we want to keep that way because
using repo_tree_rev_fetch_git to fetch the remote branch is
convenient.

So we use $repos/osstest-merge as a temporary merge repo and reclone
from the active local repo each time.

All of this happens in ap-fetch version which also fetches the result
into FETCH_HEAD into the active repo (the one running the test), such
that the existing hard reset in cr-daily-branch can find the commit.

If the merge fails then manual intervention (i.e. a manual merge and
push to the _local_ pretest) will be required.

In ap-push we stop pushing to xenbits#master except for the master
instance if an upstream is defined. At some point it might be useful
to add a configuration option for where to push to but I don't have
that requirement right now.

ap-fetch-version-old requires no changes.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 Osstest.pm        |  2 ++
 ap-common         |  3 +++
 ap-fetch-version  | 19 ++++++++++++++++++-
 ap-push           |  8 +++++++-
 production-config |  2 ++
 5 files changed, 32 insertions(+), 2 deletions(-)

diff --git a/Osstest.pm b/Osstest.pm
index 7989129..7f72bc6 100644
--- a/Osstest.pm
+++ b/Osstest.pm
@@ -102,6 +102,8 @@ sub readglobalconfig () {
     return if $readglobalconfig_done;
     $readglobalconfig_done=1;
 
+    $c{OsstestUpstream} = 'git://xenbits.xen.org/osstest.git master';
+
     $c{HostProp_DhcpWatchMethod} = 'leases dhcp3 infra:5556';
     $c{AuthorizedKeysFiles} = '';
     $c{AuthorizedKeysAppend} = '';
diff --git a/ap-common b/ap-common
index 64749e3..fadeb2b 100644
--- a/ap-common
+++ b/ap-common
@@ -21,6 +21,8 @@
 
 : ${XENBITS:=osstest@xenbits.xen.org}
 
+: ${TREEBRANCH_OSSTEST_UPSTREAM=`getconfig OsstestUpstream`}
+
 : ${TREE_XEN:=git://xenbits.xen.org/xen.git}
 : ${PUSH_TREE_XEN:=$XENBITS:/home/xen/git/xen.git}
 
@@ -76,6 +78,7 @@ fi
 : ${TREE_LINUX2639:=${KERNEL_SCM}/konrad/xen.git}
 : ${TAG_LINUX2639:=stable/2.6.39.x}
 
+: ${LOCALREV_OSSTEST:=daily-cron.$branch}
 : ${LOCALREV_XEN:=daily-cron.$branch}
 : ${LOCALREV_LINUX:=daily-cron.$branch}
 : ${LOCALREV_LIBVIRT:=daily-cron.$branch}
diff --git a/ap-fetch-version b/ap-fetch-version
index 33aaf00..efd3139 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -90,7 +90,24 @@ ovmf)
 		$TREE_OVMF_UPSTREAM master $LOCALREV_OVMF
 	;;
 osstest)
-	if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
+	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
+	    OSSTEST_REVISION_MERGE=`repo_tree_rev_fetch_git osstest \
+		$TREEBRANCH_OSSTEST_UPSTREAM $LOCALREV_OVMF`
+	    echo >&2 "$TREEBRANCH_OSSTEST_UPSTREAM = $OSSTEST_REVISION_MERGE"
+
+	    rm -rf $repos/osstest-merge >&2
+	    git clone -b pretest $HOME/testing.git $repos/osstest-merge >&2
+
+	    git -C $repos/osstest-merge \
+		fetch $repos/osstest $LOCALREV_OVMF:ap-merge >&2
+	    git -C $repos/osstest-merge merge --no-edit \
+		-m "Automerge of $TREEBRANCH_OSSTEST_UPSTREAM into pretest" \
+		ap-merge >&2
+
+	    # cr-daily-branch will reset the local tree to this version.
+	    git fetch $repos/osstest-merge pretest
+	    git	rev-parse FETCH_HEAD^0
+	elif [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
 	    git fetch $HOME/testing.git pretest:ap-fetch >&2
             git rev-parse ap-fetch^0
 	else
diff --git a/ap-push b/ap-push
index c141464..5967b42 100755
--- a/ap-push
+++ b/ap-push
@@ -99,7 +99,13 @@ ovmf)
 	;;
 osstest)
 	git push $HOME/testing.git $revision:production
-	git push $XENBITS:/home/xen/git/osstest.git $revision:master
+	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
+	    # could push to instance specific location, but
+	    # certainly not to master instance's xenbits repo!
+	    :
+	else
+	    git push $XENBITS:/home/xen/git/osstest.git $revision:master
+	fi
 	;;
 *)
 	echo >&2 "branch $branch ?"
diff --git a/production-config b/production-config
index 2ffefdd..9983025 100644
--- a/production-config
+++ b/production-config
@@ -15,6 +15,8 @@
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+# This is the master instance, therefore it has no upstream
+OsstestUpstream ''
 
 JobDB Executive
 HostDB Executive
-- 
2.1.4

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH OSSTEST 3/5] cambridge: Add a crontab
  2015-04-01 15:09 [PATCH OSSTEST 0/5] Tweaks to allow running non-master production instances Ian Campbell
  2015-04-01 15:10 ` [PATCH OSSTEST 1/5] cri-args-hostslists: allow instance specific settings Ian Campbell
  2015-04-01 15:10 ` [PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances Ian Campbell
@ 2015-04-01 15:10 ` Ian Campbell
  2015-04-02 11:13   ` Ian Jackson
  2015-04-01 15:10 ` [PATCH OSSTEST 4/5] cambridge: Do not try to push harness to XenProject instance output Ian Campbell
  2015-04-01 15:10 ` [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark Ian Campbell
  4 siblings, 1 reply; 14+ messages in thread
From: Ian Campbell @ 2015-04-01 15:10 UTC (permalink / raw)
  To: ian.jackson; +Cc: Ian Campbell, xen-devel

Right now all it does is keep osstest itself up to date.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 crontab-cambridge | 5 +++++
 1 file changed, 5 insertions(+)
 create mode 100644 crontab-cambridge

diff --git a/crontab-cambridge b/crontab-cambridge
new file mode 100644
index 0000000..ef0d234
--- /dev/null
+++ b/crontab-cambridge
@@ -0,0 +1,5 @@
+PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
+MAILTO=ian.jackson@citrix.com,ian.campbell@eu.citrix.com
+# m		h	dom mon dow	command
+49		1	* * *		cd testing.git && BRANCHES=osstest		./cr-for-branches branches -w "./cr-daily-branch --real"
+3		4	* * *		savelog -c28 testing.git/tmp/cr-for-branches.log >/dev/null
-- 
2.1.4

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH OSSTEST 4/5] cambridge: Do not try to push harness to XenProject instance output
  2015-04-01 15:09 [PATCH OSSTEST 0/5] Tweaks to allow running non-master production instances Ian Campbell
                   ` (2 preceding siblings ...)
  2015-04-01 15:10 ` [PATCH OSSTEST 3/5] cambridge: Add a crontab Ian Campbell
@ 2015-04-01 15:10 ` Ian Campbell
  2015-04-02 11:13   ` Ian Jackson
  2015-04-01 15:10 ` [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark Ian Campbell
  4 siblings, 1 reply; 14+ messages in thread
From: Ian Campbell @ 2015-04-01 15:10 UTC (permalink / raw)
  To: ian.jackson; +Cc: Ian Campbell, xen-devel

By arranging for cr-publish-flight-logs to ignore --push-harness if
either of HarnessPublishGitRepoDir or HarnessPublishGitUserHost are
not specified

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
Perhaps we should have our own tree for such things, but for now just
nobble it.
---
 cr-publish-flight-logs      | 4 ++++
 production-config-cambridge | 3 ---
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/cr-publish-flight-logs b/cr-publish-flight-logs
index 1feb52d..5311015 100755
--- a/cr-publish-flight-logs
+++ b/cr-publish-flight-logs
@@ -43,6 +43,10 @@ if ($push_harness) {
     my $githost= $c{HarnessPublishGitUserHost};
     my $gitdir= $c{HarnessPublishGitRepoDir};
 
+    if (!$githost || !$gitdir) {
+        logm("Nowhere to publish harness to");
+        return 0;
+    }
     system_checked("git push $githost:$gitdir HEAD:refs/heads/flight-$flight");
     system_checked("ssh $githost 'cd $gitdir && git update-server-info'");
 }
diff --git a/production-config-cambridge b/production-config-cambridge
index ad4e44c..043a73c 100644
--- a/production-config-cambridge
+++ b/production-config-cambridge
@@ -50,9 +50,6 @@ GlobalLockDir /export/home/osstest/testing.git
 LogsPublish= "$c{Publish}/logs"
 ResultsPublish= "$c{Publish}/results"
 
-HarnessPublishGitUserHost xen@xenbits.xensource.com
-HarnessPublishGitRepoDir git/osstest.git
-
 ReportTrailer= <<END
 Logs, config files, etc. are available at
     $c{ReportHtmlPubBaseUrl}
-- 
2.1.4

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark
  2015-04-01 15:09 [PATCH OSSTEST 0/5] Tweaks to allow running non-master production instances Ian Campbell
                   ` (3 preceding siblings ...)
  2015-04-01 15:10 ` [PATCH OSSTEST 4/5] cambridge: Do not try to push harness to XenProject instance output Ian Campbell
@ 2015-04-01 15:10 ` Ian Campbell
  2015-04-02 11:16   ` Ian Jackson
  4 siblings, 1 reply; 14+ messages in thread
From: Ian Campbell @ 2015-04-01 15:10 UTC (permalink / raw)
  To: ian.jackson; +Cc: Ian Campbell, xen-devel

http://osstest.cam.xci-test.com/~osstest/testlogs already exists and
points to the live logs directory, so switch PubBaseUrl to that in the
Cambridge config such that email reports etc contain it. This won't be
externally accessible but I think that won't matter now that the
master production instance is elsewhere.

Arrange that cr-publish-flight-logs doesn't publish the corresponding
thing if either LogsPublish or ResultsPublish is not set, and unset
them in the Cambridge config.

Likewise arrange that cr-ensure-disk-space doesn't do anything if the
configuration variable passed as an option is not set, and unset
Publish (the base for {Logs,Results}Publish) in the Cambridge config.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 cr-ensure-disk-space        | 2 ++
 cr-publish-flight-logs      | 4 ++--
 production-config-cambridge | 6 +-----
 3 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/cr-ensure-disk-space b/cr-ensure-disk-space
index 0ee84c7..84b1890 100755
--- a/cr-ensure-disk-space
+++ b/cr-ensure-disk-space
@@ -40,6 +40,8 @@ die unless @ARGV==1;
 
 our ($cfgbase) = @ARGV;
 
+exit 0 unless $cfgbase;
+
 csreadconfig();
 
 open LOCK, "> $c{GlobalLockDir}/publish-lock" or die $!;
diff --git a/cr-publish-flight-logs b/cr-publish-flight-logs
index 5311015..3894351 100755
--- a/cr-publish-flight-logs
+++ b/cr-publish-flight-logs
@@ -61,5 +61,5 @@ sub copydir ($$) {
     $!=0; $?=0; system @cmd; die "rsync $? $!" if $? or $!;
 }
 
-copydir("$c{Logs}/$flight/", "$c{LogsPublish}/$flight");
-copydir("$c{Results}/", "$c{ResultsPublish}");
+copydir("$c{Logs}/$flight/", "$c{LogsPublish}/$flight") if $c{LogsPublish};
+copydir("$c{Results}/", "$c{ResultsPublish}") if $c{ResultsPublish};
diff --git a/production-config-cambridge b/production-config-cambridge
index 043a73c..b3a7da5 100644
--- a/production-config-cambridge
+++ b/production-config-cambridge
@@ -40,16 +40,12 @@ TestHostKeypairPath /export/home/osstest/.ssh/id_rsa_osstest
 
 GitCacheProxy git://drall.uk.xensource.com:9419/
 
-PubBaseUrl http://www.chiark.greenend.org.uk/~xensrcts
+PubBaseUrl http://osstest.cam.xci-test.com/~osstest/testlogs
 ReportHtmlPubBaseUrl="$c{PubBaseUrl}/logs"
 ResultsHtmlPubBaseUrl="$c{PubBaseUrl}/results"
 
-Publish xensrcts@login.chiark.greenend.org.uk:/home/ian/work/xc_osstest
 GlobalLockDir /export/home/osstest/testing.git
 
-LogsPublish= "$c{Publish}/logs"
-ResultsPublish= "$c{Publish}/results"
-
 ReportTrailer= <<END
 Logs, config files, etc. are available at
     $c{ReportHtmlPubBaseUrl}
-- 
2.1.4

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: [PATCH OSSTEST 1/5] cri-args-hostslists: allow instance specific settings
  2015-04-01 15:10 ` [PATCH OSSTEST 1/5] cri-args-hostslists: allow instance specific settings Ian Campbell
@ 2015-04-02 10:52   ` Ian Jackson
  0 siblings, 0 replies; 14+ messages in thread
From: Ian Jackson @ 2015-04-02 10:52 UTC (permalink / raw)
  To: Ian Campbell; +Cc: xen-devel

Ian Campbell writes ("[PATCH OSSTEST 1/5] cri-args-hostslists: allow instance specific settings"):
> In particular this new $HOME/.xen-osstest/cri-args-hostslists.settings
> can contain things like "export
> OSSTEST_CONFIG=production-config-cambridge" to tailor things for a
> particular instance of osstest running in production mode.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances
  2015-04-01 15:10 ` [PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances Ian Campbell
@ 2015-04-02 11:13   ` Ian Jackson
  2015-04-02 11:21     ` Ian Campbell
  0 siblings, 1 reply; 14+ messages in thread
From: Ian Jackson @ 2015-04-02 11:13 UTC (permalink / raw)
  To: Ian Campbell; +Cc: xen-devel

Ian Campbell writes ("[PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances"):
> We want to arrange that the master XenProject instance continues to
> test its own pretest branch while any downstream instances will pickup
> changes from the master instance's production (i.e. tested) branch,
> which is published at git://xenbits.xen.org/osstest.git#master.
...	;;
>  osstest)
> -	if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
> +	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then

Shouldn't the OSSTEST_USE_HEAD option override this ?  Ie I think this
is the wrong way round.

> +	    OSSTEST_REVISION_MERGE=`repo_tree_rev_fetch_git osstest \
> +		$TREEBRANCH_OSSTEST_UPSTREAM $LOCALREV_OVMF`

OSSTEST_REVISION_MERGE doesn't seem to be an environment variable.  I
think it should either be honoured if pre-set, or just be an ordinary
(lowercase) variable name (in which case it doesn't need the OSSTEST
prefix).

> +	    echo >&2 "$TREEBRANCH_OSSTEST_UPSTREAM = $OSSTEST_REVISION_MERGE"
> +
> +	    rm -rf $repos/osstest-merge >&2
> +	    git clone -b pretest $HOME/testing.git $repos/osstest-merge >&2
> +
> +	    git -C $repos/osstest-merge \
> +		fetch $repos/osstest $LOCALREV_OVMF:ap-merge >&2

LOCALREV_OVMF (here and earlier).  It's not clear to me that you
actually need to indirect it; you could just specify a branch name.

The rest looks fine.

Having thought about how to override things, particularly:

> +	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
> +	    # could push to instance specific location, but
> +	    # certainly not to master instance's xenbits repo!
> +	    :
> +	else

I think ap-fetch-* and ap-push should have a general purpose hook
which occurs just after ap-common is loaded, something like this:

  diff --git a/ap-fetch-version b/ap-fetch-version
  index 33aaf00..a3028f6 100755
  --- a/ap-fetch-version
  +++ b/ap-fetch-version
  @@ -31,6 +31,8 @@ if info_linux_tree "$branch"; then
          exit 0
   fi

  +. ${OSSTEST_HOOK_AP_FETCH-/dev/null}
  +
   case "$branch" in
   xen-3.*)
          ./sg-hg-heads sh -ec '

We can postpone actually writing that until someone needs it.

Ian.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH OSSTEST 3/5] cambridge: Add a crontab
  2015-04-01 15:10 ` [PATCH OSSTEST 3/5] cambridge: Add a crontab Ian Campbell
@ 2015-04-02 11:13   ` Ian Jackson
  0 siblings, 0 replies; 14+ messages in thread
From: Ian Jackson @ 2015-04-02 11:13 UTC (permalink / raw)
  To: Ian Campbell; +Cc: xen-devel

Ian Campbell writes ("[PATCH OSSTEST 3/5] cambridge: Add a crontab"):
> Right now all it does is keep osstest itself up to date.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH OSSTEST 4/5] cambridge: Do not try to push harness to XenProject instance output
  2015-04-01 15:10 ` [PATCH OSSTEST 4/5] cambridge: Do not try to push harness to XenProject instance output Ian Campbell
@ 2015-04-02 11:13   ` Ian Jackson
  0 siblings, 0 replies; 14+ messages in thread
From: Ian Jackson @ 2015-04-02 11:13 UTC (permalink / raw)
  To: Ian Campbell; +Cc: ian.jackson, xen-devel

Ian Campbell writes ("[PATCH OSSTEST 4/5] cambridge: Do not try to push harness to XenProject instance output"):
> By arranging for cr-publish-flight-logs to ignore --push-harness if
> either of HarnessPublishGitRepoDir or HarnessPublishGitUserHost are
> not specified

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark
  2015-04-01 15:10 ` [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark Ian Campbell
@ 2015-04-02 11:16   ` Ian Jackson
  2015-04-02 11:25     ` Ian Campbell
  0 siblings, 1 reply; 14+ messages in thread
From: Ian Jackson @ 2015-04-02 11:16 UTC (permalink / raw)
  To: Ian Campbell; +Cc: xen-devel

Ian Campbell writes ("[PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark"):
> http://osstest.cam.xci-test.com/~osstest/testlogs already exists and
> points to the live logs directory, so switch PubBaseUrl to that in the
> Cambridge config such that email reports etc contain it. This won't be
> externally accessible but I think that won't matter now that the
> master production instance is elsewhere.
> 
> Arrange that cr-publish-flight-logs doesn't publish the corresponding
> thing if either LogsPublish or ResultsPublish is not set, and unset
> them in the Cambridge config.
> 
> Likewise arrange that cr-ensure-disk-space doesn't do anything if the
> configuration variable passed as an option is not set, and unset
> Publish (the base for {Logs,Results}Publish) in the Cambridge config.
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> ---
>  cr-ensure-disk-space        | 2 ++
>  cr-publish-flight-logs      | 4 ++--
>  production-config-cambridge | 6 +-----
>  3 files changed, 5 insertions(+), 7 deletions(-)
> 
> diff --git a/cr-ensure-disk-space b/cr-ensure-disk-space
> index 0ee84c7..84b1890 100755
> --- a/cr-ensure-disk-space
> +++ b/cr-ensure-disk-space
> @@ -40,6 +40,8 @@ die unless @ARGV==1;
>  
>  our ($cfgbase) = @ARGV;
>  
> +exit 0 unless $cfgbase;
>  
>  csreadconfig();

Don't you mean "unless $c{$cfgbase}" ?  And then it has to come after
csreadconfig.  (Did you test this ?)

The rest LGTM.

Thanks,
Ian.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances
  2015-04-02 11:13   ` Ian Jackson
@ 2015-04-02 11:21     ` Ian Campbell
  0 siblings, 0 replies; 14+ messages in thread
From: Ian Campbell @ 2015-04-02 11:21 UTC (permalink / raw)
  To: Ian Jackson; +Cc: xen-devel

On Thu, 2015-04-02 at 12:13 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances"):
> > We want to arrange that the master XenProject instance continues to
> > test its own pretest branch while any downstream instances will pickup
> > changes from the master instance's production (i.e. tested) branch,
> > which is published at git://xenbits.xen.org/osstest.git#master.
> ...	;;
> >  osstest)
> > -	if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
> > +	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
> 
> Shouldn't the OSSTEST_USE_HEAD option override this ?  Ie I think this
> is the wrong way round.

Yes, I think you are right.

I think I just ended up with this because I was testing via the
standalone wrapper which sets USE_HEAD, but I'll test another way.

> > +	    OSSTEST_REVISION_MERGE=`repo_tree_rev_fetch_git osstest \
> > +		$TREEBRANCH_OSSTEST_UPSTREAM $LOCALREV_OVMF`
> 
> OSSTEST_REVISION_MERGE doesn't seem to be an environment variable.  I
> think it should either be honoured if pre-set, or just be an ordinary
> (lowercase) variable name (in which case it doesn't need the OSSTEST
> prefix).

OK. Any preference as to which?

As it stands the only reason for the variable is to sink the output from
repo_tree_rev_fetch and log it.

> 
> > +	    echo >&2 "$TREEBRANCH_OSSTEST_UPSTREAM = $OSSTEST_REVISION_MERGE"
> > +
> > +	    rm -rf $repos/osstest-merge >&2
> > +	    git clone -b pretest $HOME/testing.git $repos/osstest-merge >&2
> > +
> > +	    git -C $repos/osstest-merge \
> > +		fetch $repos/osstest $LOCALREV_OVMF:ap-merge >&2
> 
> LOCALREV_OVMF (here and earlier).  It's not clear to me that you
> actually need to indirect it; you could just specify a branch name.

OK.

> The rest looks fine.
> 
> Having thought about how to override things, particularly:
> 
> > +	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
> > +	    # could push to instance specific location, but
> > +	    # certainly not to master instance's xenbits repo!
> > +	    :
> > +	else
> 
> I think ap-fetch-* and ap-push should have a general purpose hook
> which occurs just after ap-common is loaded, something like this:
> 
>   diff --git a/ap-fetch-version b/ap-fetch-version
>   index 33aaf00..a3028f6 100755
>   --- a/ap-fetch-version
>   +++ b/ap-fetch-version
>   @@ -31,6 +31,8 @@ if info_linux_tree "$branch"; then
>           exit 0
>    fi
> 
>   +. ${OSSTEST_HOOK_AP_FETCH-/dev/null}
>   +
>    case "$branch" in
>    xen-3.*)
>           ./sg-hg-heads sh -ec '
> 
> We can postpone actually writing that until someone needs it.

Gladly ;-)

Ian.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark
  2015-04-02 11:16   ` Ian Jackson
@ 2015-04-02 11:25     ` Ian Campbell
  2015-04-02 15:34       ` Ian Campbell
  0 siblings, 1 reply; 14+ messages in thread
From: Ian Campbell @ 2015-04-02 11:25 UTC (permalink / raw)
  To: Ian Jackson; +Cc: xen-devel

On Thu, 2015-04-02 at 12:16 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark"):
> > http://osstest.cam.xci-test.com/~osstest/testlogs already exists and
> > points to the live logs directory, so switch PubBaseUrl to that in the
> > Cambridge config such that email reports etc contain it. This won't be
> > externally accessible but I think that won't matter now that the
> > master production instance is elsewhere.
> > 
> > Arrange that cr-publish-flight-logs doesn't publish the corresponding
> > thing if either LogsPublish or ResultsPublish is not set, and unset
> > them in the Cambridge config.
> > 
> > Likewise arrange that cr-ensure-disk-space doesn't do anything if the
> > configuration variable passed as an option is not set, and unset
> > Publish (the base for {Logs,Results}Publish) in the Cambridge config.
> > 
> > Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> > ---
> >  cr-ensure-disk-space        | 2 ++
> >  cr-publish-flight-logs      | 4 ++--
> >  production-config-cambridge | 6 +-----
> >  3 files changed, 5 insertions(+), 7 deletions(-)
> > 
> > diff --git a/cr-ensure-disk-space b/cr-ensure-disk-space
> > index 0ee84c7..84b1890 100755
> > --- a/cr-ensure-disk-space
> > +++ b/cr-ensure-disk-space
> > @@ -40,6 +40,8 @@ die unless @ARGV==1;
> >  
> >  our ($cfgbase) = @ARGV;
> >  
> > +exit 0 unless $cfgbase;
> >  
> >  csreadconfig();
> 
> Don't you mean "unless $c{$cfgbase}" ?

I think I do, yes.

> And then it has to come after csreadconfig.  (Did you test this ?)

I was running in standalone mode, so I don't think this code will have
actually been run, I was relying on my ability to reason about the code,
which apparently failed :-/.

Shall I run an adhoc flight with this change in it?

Ian.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark
  2015-04-02 11:25     ` Ian Campbell
@ 2015-04-02 15:34       ` Ian Campbell
  0 siblings, 0 replies; 14+ messages in thread
From: Ian Campbell @ 2015-04-02 15:34 UTC (permalink / raw)
  To: Ian Jackson; +Cc: xen-devel

On Thu, 2015-04-02 at 12:25 +0100, Ian Campbell wrote:
> Shall I run an adhoc flight with this change in it?

FYI, I think I've now figured out how to run something approximating a
"proper" osstest flight from something other than ~osstest/testing.git
via the cronjob, so one is in progress now...

Ian.

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2015-04-02 15:34 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-04-01 15:09 [PATCH OSSTEST 0/5] Tweaks to allow running non-master production instances Ian Campbell
2015-04-01 15:10 ` [PATCH OSSTEST 1/5] cri-args-hostslists: allow instance specific settings Ian Campbell
2015-04-02 10:52   ` Ian Jackson
2015-04-01 15:10 ` [PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances Ian Campbell
2015-04-02 11:13   ` Ian Jackson
2015-04-02 11:21     ` Ian Campbell
2015-04-01 15:10 ` [PATCH OSSTEST 3/5] cambridge: Add a crontab Ian Campbell
2015-04-02 11:13   ` Ian Jackson
2015-04-01 15:10 ` [PATCH OSSTEST 4/5] cambridge: Do not try to push harness to XenProject instance output Ian Campbell
2015-04-02 11:13   ` Ian Jackson
2015-04-01 15:10 ` [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark Ian Campbell
2015-04-02 11:16   ` Ian Jackson
2015-04-02 11:25     ` Ian Campbell
2015-04-02 15:34       ` Ian Campbell

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.