From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Pang, LongtaoX" Subject: Re: [OSSTEST Nested PATCH v11 6/7] Compose the main recipe of nested test job Date: Wed, 17 Jun 2015 08:54:32 +0000 Message-ID: <86C3224E41A7434B904EC364302132D80E4C30D5@SHSMSX101.ccr.corp.intel.com> References: <1432631304-27347-1-git-send-email-longtaox.pang@intel.com> <1432631304-27347-7-git-send-email-longtaox.pang@intel.com> <21880.23264.813171.606123@mariner.uk.xensource.com> <86C3224E41A7434B904EC364302132D80E4BFEC4@SHSMSX101.ccr.corp.intel.com> <21881.42757.84361.654532@mariner.uk.xensource.com> <86C3224E41A7434B904EC364302132D80E4C0A6B@SHSMSX101.ccr.corp.intel.com> <21882.64076.306554.733060@mariner.uk.xensource.com> <1434286285.25577.6.camel@localhost> <1434359303.13744.16.camel@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1434359303.13744.16.camel@citrix.com> Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Ian Campbell , "Hu, Robert" Cc: "wei.liu2@citrix.com" , Ian Jackson , "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org > -----Original Message----- > From: Ian Campbell [mailto:ian.campbell@citrix.com] > Sent: Monday, June 15, 2015 5:08 PM > To: Hu, Robert > Cc: Ian Jackson; Pang, LongtaoX; xen-devel@lists.xen.org; wei.liu2@citrix.com > Subject: Re: [OSSTEST Nested PATCH v11 6/7] Compose the main recipe of nested > test job > > On Sun, 2015-06-14 at 20:51 +0800, Robert Hu wrote: > > On Fri, 2015-06-12 at 16:27 +0100, Ian Jackson wrote: > > > Pang, LongtaoX writes ("RE: [OSSTEST Nested PATCH v11 6/7] Compose the > main recipe of nested test job"): > > > > > From: Ian Jackson [mailto:Ian.Jackson@eu.citrix.com] > > > ... > > > > > leak-check compares the set of objects present at the `leak-check > > > > > check' step with the set of objects present at the `basis' step, and > > > > > the check fails if there are any new objects. For this purpose, > > > > > objects includes domains, corefiles, etc. > > > > > > > > > OK, so the recipe in sg-run-job should be like below, please correct me if > something wrong. > > > > proc need-hosts/test-nested {} {return host} > > > > proc run-job/test-nested {} { > > > > > > This is roughly right, but thinking about it, you want ts-logs-capture > > > to run even if the previous steps fail. > > > > > > I think it might be better to reuse (subvert?) the existing machinery > > > in sg-run-job, by adding the l1 to need_xen_hosts. > > > > > > Maybe something like > > > > > > proc add-xen-host-retrospectively {ident} { > > > global need_xen_hosts > > > ts-leak-check $ident + basis > > > lappend need_xen_hosts $ident > > > } > > > > > > ? > > > > > > And then call > > > > > > add-xen-host-retrospectively l1 > > > > > > at the appropriate point. > > Thanks Ian J.. Since I'm not familiar with tcl and your sg-run-job > > framework, does here 'appropriate point' refers to before > > per-host-ts . =(*) {ts-leak-check basis} > > in proc run-job {job}? but then l1 doesn't exist yet I'm afraid. > > If after that point, the l1 has missd check basis step. > > Note that Ian included a ts-leak-check in his > add-xen-host-retrospectively function, so you needn't worry about this, > you should do it jsut after the L1 has booted into Xen, I think. > > What you get automatically here is the final leak check (i.e. the > comparison against the basis). Thanks Ian C. So, I modify the nested job recipe in sg-run-job as below. proc add-xen-host-retrospectively {ident} { global need_xen_hosts run-ts . = ts-leak-check basis $ident lappend need_xen_hosts $ident } proc need-hosts/test-nested {} {return host} proc run-job/test-nested {} { run-ts . = ts-debian-hvm-install + host nestedl1 run-ts . = ts-nested-setup + host nestedl1 run-ts . = ts-xen-install nestedl1 run-ts . = ts-host-reboot nestedl1 add-xen-host-retrospectively nestedl1 run-ts . = ts-debian-hvm-install nestedl1 nestedl2 run-ts . = ts-guest-stop nestedl1 nestedl2 run-ts . = ts-guest-destroy + host nestedl1 } After executing command ' ./standalone run-job --simulate -h dummy test-amd64-amd64-qemuu-nested | grep testid', get below information: root@OSSTEST:~/v11_pretest_3/osstest_v11# ./standalone run-job --simulate -h dummy test-amd64-amd64-qemuu-nested | grep testid 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 1 testid build-check(1) ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 2 testid hosts-allocate ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 3 testid host-install(3) ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 4 testid host-ping-check-native ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 5 testid xen-install ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 6 testid xen-boot ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 7 testid host-ping-check-xen ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 8 testid leak-check/basis(8) ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 9 testid debian-hvm-install ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 10 testid nested-setup ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 11 testid xen-install/nestedl1 ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 12 testid host-reboot/nestedl1 ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 13 testid leak-check/basis/nestedl1 ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 14 testid debian-hvm-install/nestedl1/nestedl2 ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 15 testid guest-stop/nestedl1/nestedl2 ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 16 testid guest-destroy ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 17 testid leak-check/check ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 18 testid leak-check/check/nestedl1 ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 19 testid capture-logs(19) ========== 2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 20 testid capture-logs/nestedl1(20) ========== root@OSSTEST:~/v11_pretest_3/osstest_v11# But, for testid of '18', I think it will failed to execute 'leak-check/check/nestedl1', since 'nestedl1' has been destroyed via the action of 'run-ts . = ts-guest-destroy + host nestedl1'. Please correct me if I am wrong. Another question, after execute testid of '13'(leak-check/basis/nestedl1), the testing will fail and exit with below message. Something wrong with my test environment? 2015-06-17 08:10:25 Z executing ssh ... root@192.168.199.25 xl list 2015-06-17 08:10:25 Z executing ssh ... root@192.168.199.25 ps -wwef 2015-06-17 08:10:25 Z executing ssh ... root@192.168.199.25 ps -wwef 2015-06-17 08:10:25 Z executing ssh ... root@192.168.199.25 xenstore-ls -fp 2015-06-17 08:10:26 Z executing ssh ... root@192.168.199.25 find /tmp /var/run /var/tmp /var/lib/xen /var/core ! -type d -print0 -ls -printf '\0' find: `/var/core': No such file or directory 2015-06-17 08:10:26 Z command nonzero waitstatus 256: timeout 60 ssh -o StrictHostKeyChecking=no -o BatchMode=yes -o ConnectTimeout=100 -o ServerAliveInterval=100 -o PasswordAuthentication=no -o ChallengeResponseAuthentication=no -o UserKnownHostsFile=tmp/t.known_hosts_standalone.test-amd64-amd64-qemuu-nested root@192.168.199.25 find /tmp /var/run /var/tmp /var/lib/xen /var/core ! -type d -print0 -ls -printf '\0' BROKEN: listing/checking leakable objects: status 256 at Osstest/TestSupport.pm line 400. ; marked standalone.test-amd64-amd64-qemuu-nested broken at Osstest/TestSupport.pm line 218. + rc=2 + date -u +%Y-%m-%d %H:%M:%S Z exit status 2 2015-06-17 08:10:26 Z exit status 2 + exit 2 2015-06-17 08:10:26 Z standalone.test-amd64-amd64-qemuu-nested 13 status status fail > > > > If you do this then the main run-job proc will automatically do the > > > leak-check and the logs-capture for you. > > > > > > > > > Thinking about this leads me to ask another question. Suppose that a > > > bug causes the l1 to lock up completely. ts-logs-capture will attempt > > > to hard reboot a locked-up host. If it can't fetch any logs, it calls > > > target_reboot_hard($ho); > > > > > > What will that do if $ho refers to the l1 ? It relies on the power > > > method. Does your nested l1 "host" have a power method ? > > I'm afraid l1 won't like normal hosts has power cycle operations. Maybe > > we need to simulate it? > > Perhaps arrange for an appropriate PowerMethod for "hosts which are > actually guests"? > I think maybe we need to refactor 'power_cycle' function in TestSupport.pm. I have not try it, something like below? sub power_cycle ($) { my ($ho) = @_; + if (guest_var($ho,"enable_nestedhvm",'') =~ m/true/) { + guest_destroy($ho); + guest_create($gho); + guest_await_dhcp_tcp($gho,300); + guest_check_up($gho); + } else { $mjobdb->host_check_allocated($ho); die "refusing to set power state for host $ho->{Name}". " possibly shared with other jobs\n" if $ho->{SharedMaybeOthers}; power_state($ho, 0); power_cycle_sleep($ho); power_state($ho, 1); + } } > Ian.