OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP

OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP

[Hu, Robert]

Hi Ians,

Current ts-guest-destory will invoke guest_await_dhcp_tcp(); 
but in nested case, after l1 turns into Xen environment, it then has
fixed IP address; which in turn has failed at dhcp lease check.

So, how about if I in ts-guest-destroy bypass guest_await_dhcp_tcp()
if we have $r{guest->Guest_ip}?

Best Regards,
Robert Ho

Re: OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP

[Hu, Robert]

> -----Original Message-----
> From: xen-devel-bounces@lists.xen.org
> [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of Hu, Robert
> Sent: Wednesday, August 5, 2015 2:22 PM
> To: ian.jackson@eu.citrix.com; Ian Campbell; wei.liu2@citrix.com
> Cc: xen-devel@lists.xen.org
> Subject: [Xen-devel] OSSTEST -- nested test case development, RFC:
> ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP
> 
> Hi Ians,
> 
> Current ts-guest-destory will invoke guest_await_dhcp_tcp();
> but in nested case, after l1 turns into Xen environment, it then has
> fixed IP address; which in turn has failed at dhcp lease check.
> 
> So, how about if I in ts-guest-destroy bypass guest_await_dhcp_tcp()
> if we have $r{guest->Guest_ip}?
Detailed diff

@@ -1354,6 +1355,8 @@ sub selectguest ($$) {
         $gho->{Options}{$opt}++;
     }
     logm("guest: using $gn on $gho->{Host}{Name}");
+    $gho->{Ip} = $r{"$gho->{Guest}_ip"};
+    logm("guest: $gn has fixed IP $gho->{Ip}");
     guest_find_lv($gho);
     guest_find_ether($gho);
     guest_find_tcpcheckport($gho);
@@ -1758,7 +1761,7 @@ sub guest_await_dhcp_tcp ($$) {
              " $gho->{TcpCheckPort}".
               " link/ip/tcp",
               sub {
-        my $err= guest_check_ip($gho);
+        my $err= guest_check_ip($gho) if !$gho->{Ip};
         return $err if defined $err;
 
         return
> 
> Best Regards,
> Robert Ho
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP

[Ian Campbell]

On Wed, 2015-08-05 at 06:22 +0000, Hu, Robert wrote:
> Hi Ians,

I don't 100% recall how this is supposed to fit together.

IIRC:

1# L0 is installed as usual

#2 An L1 guest is installed. That L1 guest gets an IP address from DHCP in
the normal way.

3# Then ts-nested setup customises the L1 guest into an L1 host, storing
the DHCP assigned address in $r{"${l1ident}_ip"}. (I'm not sure if it is
actually called l1ident, but whatever it is).

4# Then operations which selecthost(l1ident) see that $r{"${l1ident}_ip"}
and use it as the $ho->{Ip} instead of looking for it in the host db.

5# At some point an L2 guest is installed on the L1 host and it also gets
an IP from DHCP in the usual way.

Is that all correct?

> Current ts-guest-destory will invoke guest_await_dhcp_tcp(); 
> but in nested case, after l1 turns into Xen environment, it then has
> fixed IP address; which in turn has failed at dhcp lease check.

So here are we talking about ts-guest-destroy of an L2 guest on the L1
host, or of the L1 guest on the L0 host?

I think you are talking about the L1 guest on the L0 host.

In that context ts-guest-destroy will be considering the L1 as a guest, so
I would expect that guest_await_dhcp_tcp should work, because the L1
guest's IP was assigned via DHCP in #2 above.

So I suppose the question is how/why is guest_await_dhcp_tcp failing when
operating on the L1 guest? It should be "just a guest" in this context I
think.

> 
> So, how about if I in ts-guest-destroy bypass guest_await_dhcp_tcp()
> if we have $r{guest->Guest_ip}?
> 
> Best Regards,
> Robert Ho
> 

Re: OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP

[Hu, Robert]

> -----Original Message-----
> From: Ian Campbell [mailto:ian.campbell@citrix.com]
> Sent: Wednesday, August 5, 2015 8:27 PM
> To: Hu, Robert; ian.jackson@eu.citrix.com; wei.liu2@citrix.com
> Cc: xen-devel@lists.xen.org
> Subject: Re: OSSTEST -- nested test case development, RFC: ts-guest-destroy
> doesn't call guest_await_dhcp_tcp() if guest has fixed IP
> 
> On Wed, 2015-08-05 at 06:22 +0000, Hu, Robert wrote:
> > Hi Ians,
> 
> I don't 100% recall how this is supposed to fit together.
> 
> IIRC:
> 
> 1# L0 is installed as usual
> 
> #2 An L1 guest is installed. That L1 guest gets an IP address from DHCP in
> the normal way.
yes
> 
> 3# Then ts-nested setup customises the L1 guest into an L1 host, storing
> the DHCP assigned address in $r{"${l1ident}_ip"}. (I'm not sure if it is
> actually called l1ident, but whatever it is).
After l1 installed Xen and turned into Xen environment (12 testid xen-install/l1 below),
its IP get fixed (this is current normal Xen installation behavior, same in l0's Xen install).
See /etc/network/interface in l1:
root@l1:~# cat /etc/network/interfaces 
...
auto xenbr0
iface xenbr0 inet static
    address 192.168.199.175
    netmask 255.255.255.0
    gateway 192.168.199.1
    bridge_ports eth0
    bridge_fd 0
    bridge_stp off

Then, in guest-destroy, we cannot get its IP by searching lease file.
> 
> 4# Then operations which selecthost(l1ident) see that $r{"${l1ident}_ip"}
> and use it as the $ho->{Ip} instead of looking for it in the host db.
Yes
> 
> 5# At some point an L2 guest is installed on the L1 host and it also gets
> an IP from DHCP in the usual way.
Yes, L2 will get IP from dhcp.
> 
> Is that all correct?
Here is current test steps

2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 1 testid build-check(1) ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 2 testid hosts-allocate ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 3 testid host-install(3) ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 4 testid host-ping-check-native ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 5 testid xen-install ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 6 testid xen-boot ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 7 testid host-ping-check-xen ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 8 testid leak-check/basis(8) ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 9 testid debian-hvm-install ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 10 testid nested-setup ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 11 testid host-ping-check-native/l1 ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 12 testid xen-install/l1 ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 13 testid xen-boot/l1 ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 14 testid host-ping-check-xen/l1 ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 15 testid leak-check/basis/l1(15) ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 16 testid debian-hvm-install/l1/l2 ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 17 testid guest-stop/l1/l2 ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 18 testid guest-destroy ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 19 testid leak-check/check/l1 ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 20 testid capture-logs/l1(20) ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 21 testid final-poweroff/l1 ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 22 testid leak-check/check ==========
2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 23 testid capture-logs(23) ==========
> 
> > Current ts-guest-destory will invoke guest_await_dhcp_tcp();
> > but in nested case, after l1 turns into Xen environment, it then has
> > fixed IP address; which in turn has failed at dhcp lease check.
> 
> So here are we talking about ts-guest-destroy of an L2 guest on the L1
> host, or of the L1 guest on the L0 host?
l1 on l0.
> 
> I think you are talking about the L1 guest on the L0 host.
> 
> In that context ts-guest-destroy will be considering the L1 as a guest, so
> I would expect that guest_await_dhcp_tcp should work, because the L1
> guest's IP was assigned via DHCP in #2 above.
explained above, it is not after we xen-install l1.
> 
> So I suppose the question is how/why is guest_await_dhcp_tcp failing when
> operating on the L1 guest? It should be "just a guest" in this context I
> think.
It is a special guest now.
> 
> >
> > So, how about if I in ts-guest-destroy bypass guest_await_dhcp_tcp()
> > if we have $r{guest->Guest_ip}?
> >
> > Best Regards,
> > Robert Ho
> >

Re: OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP

[Ian Campbell]

On Thu, 2015-08-06 at 01:57 +0000, Hu, Robert wrote:
> > 
> > -----Original Message-----
> > From: Ian Campbell [mailto:ian.campbell@citrix.com]
> > Sent: Wednesday, August 5, 2015 8:27 PM
> > To: Hu, Robert; ian.jackson@eu.citrix.com; wei.liu2@citrix.com
> > Cc: xen-devel@lists.xen.org
> > Subject: Re: OSSTEST -- nested test case development, RFC: ts-guest
> > -destroy
> > doesn't call guest_await_dhcp_tcp() if guest has fixed IP
> > 
> > On Wed, 2015-08-05 at 06:22 +0000, Hu, Robert wrote:
> > > Hi Ians,
> > 
> > I don't 100% recall how this is supposed to fit together.
> > 
> > IIRC:
> > 
> > 1# L0 is installed as usual
> > 
> > #2 An L1 guest is installed. That L1 guest gets an IP address from DHCP 
> > in
> > the normal way.
> yes
> > 
> > 3# Then ts-nested setup customises the L1 guest into an L1 host, 
> > storing
> > the DHCP assigned address in $r{"${l1ident}_ip"}. (I'm not sure if it 
> > is
> > actually called l1ident, but whatever it is).
> After l1 installed Xen and turned into Xen environment (12 testid xen-install/l1 below),
> its IP get fixed (this is current normal Xen installation behavior, same in l0's Xen install).

Reconfiguring things on a system to be static rather than dynamic has no
effect on how the network administrator (i.e. the person who operated the
DHCP server) views the IP address. As far as they are concerned the IP
address remains part of the dynamic allocation pool.

However by reconfiguring things to be static the L1 host will no longer be
generating DHCP RENEW requests when the lease times out, so the DHCP server
is at liberty to release the lease when it times out or, worse, reuse the
IP address for something else.

This doesn't apply for L0 in our deployments because we statically assign
IP addresses to MAC addresses associated with physical network interfaces.

So I think we do actually need to start supporting a dynamic mode for at
least L1 hosts (and that may well easily extend to L0 hosts too). Although
it is not 100% accurate I think we can assume that DHCP renewal will always
work, i.e. once a host is given a particular IP address so long as it keeps
renewing the lease it will keep the same address.

So I think we can still discover the DHCP address assigned to the L1 guest,
and propagate it into $r{"${l1ident}_ip"} when we convert it to an L1 host,
but we then also need to modify the Xen installation runs to use dhcp mode
for such cases and not switch to static as we do for an L0 host.

I'm not quite sure how this should be recorded in the runvars. I think we
may want to wait for Ian to return from vacation next week.

The alternative would be that selecthost needs to query the DHCP leases
file for these kinds of hosts, that would have the benefit of handling
potential lease expiry over a reboot.

Ian?


> See /etc/network/interface in l1:
> root@l1:~# cat /etc/network/interfaces 
> ...
> auto xenbr0
> iface xenbr0 inet static
>     address 192.168.199.175
>     netmask 255.255.255.0
>     gateway 192.168.199.1
>     bridge_ports eth0
>     bridge_fd 0
>     bridge_stp off
> 
> Then, in guest-destroy, we cannot get its IP by searching lease file.
> > 
> > 4# Then operations which selecthost(l1ident) see that 
> > $r{"${l1ident}_ip"}
> > and use it as the $ho->{Ip} instead of looking for it in the host db.
> Yes
> > 
> > 5# At some point an L2 guest is installed on the L1 host and it also 
> > gets
> > an IP from DHCP in the usual way.
> Yes, L2 will get IP from dhcp.
> > 
> > Is that all correct?
> Here is current test steps
> 
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 1 testid build-check(1) ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 2 testid hosts-allocate ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 3 testid host-install(3) ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 4 testid host-ping-check-native ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 5 testid xen-install ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 6 testid xen-boot ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 7 testid host-ping-check-xen ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 8 testid leak-check/basis(8) ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 9 testid debian-hvm-install ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 10 testid nested-setup ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 11 testid host-ping-check-native/l1 ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 12 testid xen-install/l1 ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 13 testid xen-boot/l1 ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 14 testid host-ping-check-xen/l1 ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 15 testid leak-check/basis/l1(15) ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 16 testid debian-hvm-install/l1/l2 ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 17 testid guest-stop/l1/l2 ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 18 testid guest-destroy ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 19 testid leak-check/check/l1 ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 20 testid capture-logs/l1(20) ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 21 testid final-poweroff/l1 ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 22 testid leak-check/check ==========
> 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested ========== 23 testid capture-logs(23) ==========
> > 
> > > Current ts-guest-destory will invoke guest_await_dhcp_tcp();
> > > but in nested case, after l1 turns into Xen environment, it then has
> > > fixed IP address; which in turn has failed at dhcp lease check.
> > 
> > So here are we talking about ts-guest-destroy of an L2 guest on the L1
> > host, or of the L1 guest on the L0 host?
> l1 on l0.
> > 
> > I think you are talking about the L1 guest on the L0 host.
> > 
> > In that context ts-guest-destroy will be considering the L1 as a guest, 
> > so
> > I would expect that guest_await_dhcp_tcp should work, because the L1
> > guest's IP was assigned via DHCP in #2 above.
> explained above, it is not after we xen-install l1.
> > 
> > So I suppose the question is how/why is guest_await_dhcp_tcp failing 
> > when
> > operating on the L1 guest? It should be "just a guest" in this context 
> > I
> > think.
> It is a special guest now.
> > 
> > > 
> > > So, how about if I in ts-guest-destroy bypass guest_await_dhcp_tcp()
> > > if we have $r{guest->Guest_ip}?
> > > 
> > > Best Regards,
> > > Robert Ho
> > > 

Re: OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP

[Hu, Robert]

> -----Original Message-----
> From: Ian Campbell [mailto:ian.campbell@citrix.com]
> Sent: Thursday, August 6, 2015 5:01 PM
> To: Hu, Robert; ian.jackson@eu.citrix.com; wei.liu2@citrix.com
> Cc: xen-devel@lists.xen.org
> Subject: Re: OSSTEST -- nested test case development, RFC: ts-guest-destroy
> doesn't call guest_await_dhcp_tcp() if guest has fixed IP
> 
> On Thu, 2015-08-06 at 01:57 +0000, Hu, Robert wrote:
> > >
> > > -----Original Message-----
> > > From: Ian Campbell [mailto:ian.campbell@citrix.com]
> > > Sent: Wednesday, August 5, 2015 8:27 PM
> > > To: Hu, Robert; ian.jackson@eu.citrix.com; wei.liu2@citrix.com
> > > Cc: xen-devel@lists.xen.org
> > > Subject: Re: OSSTEST -- nested test case development, RFC: ts-guest
> > > -destroy
> > > doesn't call guest_await_dhcp_tcp() if guest has fixed IP
> > >
> > > On Wed, 2015-08-05 at 06:22 +0000, Hu, Robert wrote:
> > > > Hi Ians,
> > >
> > > I don't 100% recall how this is supposed to fit together.
> > >
> > > IIRC:
> > >
> > > 1# L0 is installed as usual
> > >
> > > #2 An L1 guest is installed. That L1 guest gets an IP address from DHCP
> > > in
> > > the normal way.
> > yes
> > >
> > > 3# Then ts-nested setup customises the L1 guest into an L1 host,
> > > storing
> > > the DHCP assigned address in $r{"${l1ident}_ip"}. (I'm not sure if it
> > > is
> > > actually called l1ident, but whatever it is).
> > After l1 installed Xen and turned into Xen environment (12 testid
> xen-install/l1 below),
> > its IP get fixed (this is current normal Xen installation behavior, same in l0's
> Xen install).
> 
> Reconfiguring things on a system to be static rather than dynamic has no
> effect on how the network administrator (i.e. the person who operated the
> DHCP server) views the IP address. As far as they are concerned the IP
> address remains part of the dynamic allocation pool.
Yes.
> 
> However by reconfiguring things to be static the L1 host will no longer be
> generating DHCP RENEW requests when the lease times out, so the DHCP
> server
> is at liberty to release the lease when it times out or, worse, reuse the
> IP address for something else.
Yeah, this is probably the reason of no findings.
> 
> This doesn't apply for L0 in our deployments because we statically assign
> IP addresses to MAC addresses associated with physical network interfaces.
> 
> So I think we do actually need to start supporting a dynamic mode for at
> least L1 hosts (and that may well easily extend to L0 hosts too). Although
> it is not 100% accurate I think we can assume that DHCP renewal will always
> work, i.e. once a host is given a particular IP address so long as it keeps
> renewing the lease it will keep the same address.
Well, I'm afraid we cannot assume dhcp server behaves in this way: always assign
its last time IP. You know l1 will reboot after xen install. After reboot, it shall
'ask for' not renew its IP. Though, in most practices, it will get its last time IP
but cannot guarantee that.
> 
> So I think we can still discover the DHCP address assigned to the L1 guest,
> and propagate it into $r{"${l1ident}_ip"} when we convert it to an L1 host,
> but we then also need to modify the Xen installation runs to use dhcp mode
> for such cases and not switch to static as we do for an L0 host.
This will make current code more complex; while seems no other chooses.
I would suggest unify this: dynamic all cases, no matter L1 or L0 xen install; since
you can fix L0's IP in server side, it will always get same IP.
> 
> I'm not quite sure how this should be recorded in the runvars. I think we
> may want to wait for Ian to return from vacation next week.
OK
> 
> The alternative would be that selecthost needs to query the DHCP leases
> file for these kinds of hosts, that would have the benefit of handling
> potential lease expiry over a reboot.
> 
> Ian?
> 
> 
> > See /etc/network/interface in l1:
> > root@l1:~# cat /etc/network/interfaces
> > ...
> > auto xenbr0
> > iface xenbr0 inet static
> >     address 192.168.199.175
> >     netmask 255.255.255.0
> >     gateway 192.168.199.1
> >     bridge_ports eth0
> >     bridge_fd 0
> >     bridge_stp off
> >
> > Then, in guest-destroy, we cannot get its IP by searching lease file.
> > >
> > > 4# Then operations which selecthost(l1ident) see that
> > > $r{"${l1ident}_ip"}
> > > and use it as the $ho->{Ip} instead of looking for it in the host db.
> > Yes
> > >
> > > 5# At some point an L2 guest is installed on the L1 host and it also
> > > gets
> > > an IP from DHCP in the usual way.
> > Yes, L2 will get IP from dhcp.
> > >
> > > Is that all correct?
> > Here is current test steps
> >
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 1 testid build-check(1) ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 2 testid hosts-allocate ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 3 testid host-install(3) ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 4 testid host-ping-check-native ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 5 testid xen-install ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 6 testid xen-boot ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 7 testid host-ping-check-xen ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 8 testid leak-check/basis(8) ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 9 testid debian-hvm-install ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 10 testid nested-setup ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 11 testid host-ping-check-native/l1 ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 12 testid xen-install/l1 ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 13 testid xen-boot/l1 ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 14 testid host-ping-check-xen/l1 ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 15 testid leak-check/basis/l1(15) ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 16 testid debian-hvm-install/l1/l2 ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 17 testid guest-stop/l1/l2 ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 18 testid guest-destroy ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 19 testid leak-check/check/l1 ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 20 testid capture-logs/l1(20) ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 21 testid final-poweroff/l1 ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 22 testid leak-check/check ==========
> > 2015-08-06 01:44:58 Z standalone.test-amd64-amd64-qemuu-nested
> ========== 23 testid capture-logs(23) ==========
> > >
> > > > Current ts-guest-destory will invoke guest_await_dhcp_tcp();
> > > > but in nested case, after l1 turns into Xen environment, it then has
> > > > fixed IP address; which in turn has failed at dhcp lease check.
> > >
> > > So here are we talking about ts-guest-destroy of an L2 guest on the L1
> > > host, or of the L1 guest on the L0 host?
> > l1 on l0.
> > >
> > > I think you are talking about the L1 guest on the L0 host.
> > >
> > > In that context ts-guest-destroy will be considering the L1 as a guest,
> > > so
> > > I would expect that guest_await_dhcp_tcp should work, because the L1
> > > guest's IP was assigned via DHCP in #2 above.
> > explained above, it is not after we xen-install l1.
> > >
> > > So I suppose the question is how/why is guest_await_dhcp_tcp failing
> > > when
> > > operating on the L1 guest? It should be "just a guest" in this context
> > > I
> > > think.
> > It is a special guest now.
> > >
> > > >
> > > > So, how about if I in ts-guest-destroy bypass guest_await_dhcp_tcp()
> > > > if we have $r{guest->Guest_ip}?
> > > >
> > > > Best Regards,
> > > > Robert Ho
> > > >

Re: OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP

[Ian Jackson]

Ian Campbell writes ("Re: OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP"):
> However by reconfiguring things to be static the L1 host will no longer be
> generating DHCP RENEW requests when the lease times out, so the DHCP server
> is at liberty to release the lease when it times out or, worse, reuse the
> IP address for something else.

Indeed.  This is wrong.

> So I think we do actually need to start supporting a dynamic mode for at
> least L1 hosts (and that may well easily extend to L0 hosts too). Although
> it is not 100% accurate I think we can assume that DHCP renewal will always
> work, i.e. once a host is given a particular IP address so long as it keeps
> renewing the lease it will keep the same address.

It isn't clear to me that we need to make this assumption, in the
general case.  We probably need to assume that the DHCP-assigned IP
addresses don't change unexpectedly during the execution of a
particular ts-* script (where `unexpectedly' means `other than as an
obvious consequence of actions such as rebooting).

> So I think we can still discover the DHCP address assigned to the L1 guest,
> and propagate it into $r{"${l1ident}_ip"} when we convert it to an L1 host,
> but we then also need to modify the Xen installation runs to use dhcp mode
> for such cases and not switch to static as we do for an L0 host.

This would be the right approach, but ...

> I'm not quite sure how this should be recorded in the runvars. I think we
> may want to wait for Ian to return from vacation next week.

... having looked at it like this, I think recording the L1 IP
addresss in the runvars is wrong.  It should be looked up each time
(by something called by selecthost).

> The alternative would be that selecthost needs to query the DHCP leases
> file for these kinds of hosts, that would have the benefit of handling
> potential lease expiry over a reboot.

Exactly.

Ian.

Re: OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP

[Hu, Robert]

> -----Original Message-----
> From: Ian Jackson [mailto:Ian.Jackson@eu.citrix.com]
> Sent: Tuesday, August 11, 2015 8:55 PM
> To: Ian Campbell
> Cc: Hu, Robert; wei.liu2@citrix.com; xen-devel@lists.xen.org
> Subject: Re: OSSTEST -- nested test case development, RFC: ts-guest-destroy
> doesn't call guest_await_dhcp_tcp() if guest has fixed IP
> 
> Ian Campbell writes ("Re: OSSTEST -- nested test case development, RFC:
> ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP"):
> > However by reconfiguring things to be static the L1 host will no longer be
> > generating DHCP RENEW requests when the lease times out, so the DHCP
> server
> > is at liberty to release the lease when it times out or, worse, reuse the
> > IP address for something else.
> 
> Indeed.  This is wrong.
> 
> > So I think we do actually need to start supporting a dynamic mode for at
> > least L1 hosts (and that may well easily extend to L0 hosts too). Although
> > it is not 100% accurate I think we can assume that DHCP renewal will
> always
> > work, i.e. once a host is given a particular IP address so long as it keeps
> > renewing the lease it will keep the same address.
> 
> It isn't clear to me that we need to make this assumption, in the
> general case.  We probably need to assume that the DHCP-assigned IP
> addresses don't change unexpectedly during the execution of a
> particular ts-* script (where `unexpectedly' means `other than as an
> obvious consequence of actions such as rebooting).
> 
> > So I think we can still discover the DHCP address assigned to the L1 guest,
> > and propagate it into $r{"${l1ident}_ip"} when we convert it to an L1 host,
> > but we then also need to modify the Xen installation runs to use dhcp
> mode
> > for such cases and not switch to static as we do for an L0 host.
> 
> This would be the right approach, but ...
> 
> > I'm not quite sure how this should be recorded in the runvars. I think we
> > may want to wait for Ian to return from vacation next week.
> 
> ... having looked at it like this, I think recording the L1 IP
> addresss in the runvars is wrong.  It should be looked up each time
> (by something called by selecthost).
> 
> > The alternative would be that selecthost needs to query the DHCP leases
> > file for these kinds of hosts, that would have the benefit of handling
> > potential lease expiry over a reboot.
> 
> Exactly.
So shall I conclude as following?:
1. Xen install shall create xenbr0 bridge with dhcp mode; Apply to both L0 and L1.
2. in each selecthost invoke, query DHCP lease for host's IP. (This assumes no more
fixed IP used in any cases).

And who's going to make the patch for these? me or you will help provide?
> 
> Ian.

Re: OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP

[Ian Jackson]

Hu, Robert writes ("RE: OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP"):

> So shall I conclude as following?:

> 1. Xen install shall create xenbr0 bridge with dhcp mode; Apply to
> both L0 and L1.

I think ts-xen-install should respect the existing configuration for
the primary network device, rather than insisting it be static, or
insisting it be dhcp.

> 2. in each selecthost invoke, query DHCP lease for host's IP. (This
> assumes no more fixed IP used in any cases).

We should continue to use fixed IP addresses where we have them.

Ian.

Re: OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP

[Hu, Robert]

> -----Original Message-----
> From: Ian Jackson [mailto:Ian.Jackson@eu.citrix.com]
> Sent: Wednesday, August 12, 2015 11:27 PM
> To: Hu, Robert
> Cc: Ian Campbell; wei.liu2@citrix.com; xen-devel@lists.xen.org
> Subject: RE: OSSTEST -- nested test case development, RFC: ts-guest-destroy
> doesn't call guest_await_dhcp_tcp() if guest has fixed IP
> 
> Hu, Robert writes ("RE: OSSTEST -- nested test case development, RFC:
> ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP"):
> 
> > So shall I conclude as following?:
> 
> > 1. Xen install shall create xenbr0 bridge with dhcp mode; Apply to
> > both L0 and L1.
> 
> I think ts-xen-install should respect the existing configuration for
> the primary network device, rather than insisting it be static, or
> insisting it be dhcp.
Yes, in fact it is the only network interface (eth0 before Xen intall; xenbr0 after
Xen install, eth0 becomes its slave right now).
So you won't object turn xenbr0 config to 'dhcp' as following, right?
diff --git a/ts-xen-install b/ts-xen-install
index 0f53382..74104cb 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -301,10 +301,7 @@ END
             if (m/^\s* iface \s+ (?: $physif | xenbr0 ) \s+ inet \s /x) {
                 $suppress= 1;
                 print EO <<END;
-iface $iface inet static
-    address $ho->{Ip}
-    netmask $netmask
-    gateway $gateway
+iface $iface inet dhcp
 $bridgex
 END
             }
> 
> > 2. in each selecthost invoke, query DHCP lease for host's IP. (This
> > assumes no more fixed IP used in any cases).
> 
> We should continue to use fixed IP addresses where we have them.
Then I'm not confident to change selecthost(). Would you help give me
the patch addressing this?
> 
> Ian.

Re: OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP

[Hu, Robert]

> -----Original Message-----
> From: Hu, Robert
> Sent: Thursday, August 13, 2015 11:46 AM
> To: Ian Jackson
> Cc: Ian Campbell; wei.liu2@citrix.com; xen-devel@lists.xen.org
> Subject: RE: OSSTEST -- nested test case development, RFC: ts-guest-destroy
> doesn't call guest_await_dhcp_tcp() if guest has fixed IP
> 
> > -----Original Message-----
> > From: Ian Jackson [mailto:Ian.Jackson@eu.citrix.com]
> > Sent: Wednesday, August 12, 2015 11:27 PM
> > To: Hu, Robert
> > Cc: Ian Campbell; wei.liu2@citrix.com; xen-devel@lists.xen.org
> > Subject: RE: OSSTEST -- nested test case development, RFC:
> ts-guest-destroy
> > doesn't call guest_await_dhcp_tcp() if guest has fixed IP
> >
> > Hu, Robert writes ("RE: OSSTEST -- nested test case development, RFC:
> > ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed
> IP"):
> >
> > > So shall I conclude as following?:
> >
> > > 1. Xen install shall create xenbr0 bridge with dhcp mode; Apply to
> > > both L0 and L1.
> >
> > I think ts-xen-install should respect the existing configuration for
> > the primary network device, rather than insisting it be static, or
> > insisting it be dhcp.
> Yes, in fact it is the only network interface (eth0 before Xen intall; xenbr0
> after
> Xen install, eth0 becomes its slave right now).
> So you won't object turn xenbr0 config to 'dhcp' as following, right?
> diff --git a/ts-xen-install b/ts-xen-install
> index 0f53382..74104cb 100755
> --- a/ts-xen-install
> +++ b/ts-xen-install
> @@ -301,10 +301,7 @@ END
>              if (m/^\s* iface \s+ (?: $physif | xenbr0 ) \s+ inet \s /x) {
>                  $suppress= 1;
>                  print EO <<END;
> -iface $iface inet static
> -    address $ho->{Ip}
> -    netmask $netmask
> -    gateway $gateway
> +iface $iface inet dhcp
>  $bridgex
>  END
>              }
> >
> > > 2. in each selecthost invoke, query DHCP lease for host's IP. (This
> > > assumes no more fixed IP used in any cases).
> >
> > We should continue to use fixed IP addresses where we have them.
> Then I'm not confident to change selecthost(). Would you help give me
> the patch addressing this?
I changed like this. It works (I tested with both nested case and 
test-amd64-amd64-xl-qemuu-debianhvm-amd64). How would you like it?

index e9be7f8..b367cfb 100755
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -875,7 +875,7 @@ sub selecthost ($) {
     $ho->{Fqdn} = get_host_property($ho,'fqdn',$defaultfqdn);
 
 
-    $ho->{Ether}= get_host_property($ho,'ether');
+    $ho->{Ether}= get_host_property($ho,'ether') || $r{"$ho->{Ident}_ether"};
     $ho->{DiskDevice}= get_host_property($ho,'disk-device');
     $ho->{Power}= get_host_property($ho,'power-method');
 
@@ -886,17 +886,23 @@ sub selecthost ($) {
     serial_host_setup($ho);
 
     $ho->{IpStatic} = get_host_property($ho,'ip-addr');
-    if (defined $r{"${ident}_ip"}) {
-        $ho->{Ip} = $r{"${ident}_ip"};
-    } else {
-        if (!defined $ho->{IpStatic}) {
-           my $ip_packed= gethostbyname($ho->{Fqdn});
-           die "$ho->{Fqdn} ?" unless $ip_packed;
+    if (!defined $ho->{IpStatic}) {
+       my $ip_packed= gethostbyname($ho->{Fqdn});
+       if (!$ip_packed) {
+           logm("Going to look for $ho->{Name}'s ether: $ho->{Ether}");
+           $ho->{DhcpWatch}->check_ip($ho);
+           $ho->{IpStatic}=$ho->{Ip};
+       }
+#      die "$ho->{Fqdn} ?" unless $ip_packed;
+       else {
            $ho->{IpStatic}= inet_ntoa($ip_packed);
            die "$ho->{Fqdn} ?" unless defined $ho->{IpStatic};
-        }
-        $ho->{Ip}= $ho->{IpStatic};
+           $ho->{Ip}= $ho->{IpStatic};
+       }
     }
> >
> > Ian.

Re: OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP

[Ian Jackson]

Hu, Robert writes ("RE: OSSTEST -- nested test case development, RFC: ts-guest-destroy doesn't call guest_await_dhcp_tcp() if guest has fixed IP"):
> So you won't object turn xenbr0 config to 'dhcp' as following, right?

No, I'm afraid that's not right.  The logic in this function needs to
be changed to always preserve what it finds, not replace it.

> > We should continue to use fixed IP addresses where we have them.
>
> Then I'm not confident to change selecthost(). Would you help give me
> the patch addressing this?

I will try to come up with something.

Ian.

Re: [OSSTest Nested v12 18/21] After Xen install, configure its xenbr0 interface as dhcp

[Ian Jackson]

Ian Jackson:
> Robert Ho:
> > Ian Campbell:
> > > We should continue to use fixed IP addresses where we have them.
> >
> > Then I'm not confident to change selecthost(). Would you help give me
> > the patch addressing this?
> 
> I will try to come up with something.

This is the kind of thing I would suggest.  NB that this is COMPLETELY
UNTESTED.  If you have trouble testing and/or debugging this please
let me know and I will do it.

I have split it into 4 patches but I'm not necessarily convinced that
it needs to be so fine-grained.

Robert, Ian: Let me know what you think.

Thanks,
Ian.

[OSSTEST PATCH 1/4] ts-xen-install: networking: Do nothing if there's already a bridge

[Ian Jackson]

The main effect is that old Xend tests will no longer replace the
bridge network interface.  If it was using dhcp then it will keep
using dhcp, rather than being switched to have the dhcp-obtained
address hardwired in a static configuration.

A side effect is that after this, `nodhcp' will do nothing if it has
already been run, rather than replacing the effects of a previous run.
This case is only relevant for ad-hoc by-hand invocations, where
changes to the script may not be reflected when it is rerun.

Most invocations remain unaffected.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 ts-xen-install |    7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/ts-xen-install b/ts-xen-install
index b511e2b..433a6d3 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -255,7 +255,6 @@ sub nodhcp () {
 	if (!defined $physif) {
 	    # preread /etc/network/interfaces to figure out the interface
 	    my %candidates;
-	    my @bridges;
 	    while (<EI>) {
 		if (
 		    m{^ \s* (  auto \s+ (\S+)               ) \s* $}x ||
@@ -266,20 +265,18 @@ sub nodhcp () {
 		}
 		if (
 			(
-			 (m{^ \s* iface \s+ (xenbr\w*) \s+ inet \s+ }x
-			      ? ((push @bridges, $1), 1) : 0)
+			 m{^ \s* iface \s+ (xenbr\w*) \s+ inet \s+ }x
 			 ...
 			 !m{^ [ \t] }x
 			)
 		    &&
 			m{^ \s+ bridge_ports \s+ (\S+) \s }x
 		    ) {
-		    push @{ $candidates{$1} }, "$bridges[$#bridges] $_";
+		    targt_editfile_cancel("already have a bridge");
 		}
 	    }
 	    EI->error and die $!;
 	    delete $candidates{'lo'};
-	    delete $candidates{$_} foreach @bridges;
 	    die Dumper(\%candidates, \@bridges).
 		" -- cannot determine default interface"
 		unless (scalar keys %candidates) == 1;
-- 
1.7.10.4

[OSSTEST PATCH 2/4] ts-xen-install: networking: Bail if we expected a bridge but there wasn't one

[Ian Jackson]

If we have been told that there should already be a bridge, but there
isn't one, we would previously rewrite the networking config just in
case (mainly to make it static).  Instead, quit the /e/n/i editing in
this case.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 ts-xen-install |   12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/ts-xen-install b/ts-xen-install
index 433a6d3..b148163 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -285,17 +285,15 @@ sub nodhcp () {
 	}
 	my ($iface,$bridgex);
 
-	if ($initscripts_nobridge) {
-	    $iface= 'xenbr0';
-	    $bridgex= <<END;
+	if (!$initscripts_nobridge) {
+	    target_editfile_cancel("no bridge but old xend setup");
+	}
+	$iface= 'xenbr0';
+	$bridgex= <<END;
     bridge_ports $physif
     bridge_fd 0
     bridge_stp off
 END
-	} else {
-	    $iface= $physif;
-	    $bridgex= '';
-        }
 
 	my $routes= target_cmd_output_root($ho, "route -n");
 
-- 
1.7.10.4

[OSSTEST PATCH 3/4] ts-xen-install: networking: Never rewrite the interface config

[Ian Jackson]

Instead, if we need to, simply change the physical interface in the
`iface' line to refer to xenbr0, and add the physical interface as a
`bridge_ports' along with the other bridge settings.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 ts-xen-install |   33 +++++----------------------------
 1 file changed, 5 insertions(+), 28 deletions(-)

diff --git a/ts-xen-install b/ts-xen-install
index b148163..ef26cba 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -295,36 +295,13 @@ sub nodhcp () {
     bridge_stp off
 END
 
-	my $routes= target_cmd_output_root($ho, "route -n");
+	logm("iface=$iface physif=$iface");
 
-	$routes =~ m/^ [0-9.]+ \s+ 0\.0\.0\.0 \s+ ([0-9.]+) \s+ \S*U\S* \s /mxi
-	    or die "no own local network in route ?  $routes ";
-	my $netmask= $1;
-
-	$routes =~
-	    m/^ 0\.0\.0\.0 \s+ ([0-9.]+) \s+ 0\.0\.0\.0 \s+ \S*UG\S* \s /mxi
-	    or die "no default gateway ?  $routes ";
-	my $gateway= $1;
-
-	logm("iface $iface mask=$netmask gw=$gateway");
-
-        my $suppress= 0;
         while (<EI>) {
-            $suppress= 0 unless m/^\s+/;
-            s/^ \s* allow-hotplug \s+ (?: $physif | xenbr0 ) \s* $
-                /auto $iface\n/x;
-            if (m/^\s* iface \s+ (?: $physif | xenbr0 ) \s+ inet \s /x) {
-                $suppress= 1;
-                print EO <<END;
-iface $iface inet static
-    address $ho->{Ip}
-    netmask $netmask
-    gateway $gateway
-$bridgex
-END
-            }
-            print EO or die $!
-                unless $suppress;
+            if (s/^\s* iface \s+ \Q$physif\E \s+ /iface $iface /x) {
+		$_ .= $bridgex;
+	    }
+            print EO or die $!;
         }
     });
 }    
-- 
1.7.10.4

Re: [OSSTEST PATCH 3/4] ts-xen-install: networking: Never rewrite the interface config

[Ian Jackson]

Ian Jackson writes ("[OSSTEST PATCH 3/4] ts-xen-install: networking: Never rewrite the interface config"):
> Instead, if we need to, simply change the physical interface in the
> `iface' line to refer to xenbr0, and add the physical interface as a
> `bridge_ports' along with the other bridge settings.

Having looked at this again I decided there was a better (much
simpler!) way, which I will incorporate in the nestedhvm v14 series.

Ian.

[OSSTEST PATCH 4/4] ts-xen-install: networking: Rename `nodhcp' to `ensurebridge'

[Ian Jackson]

The code that used to be in nodhcp replaced any existing dhcp
configuration with a static one.  What it does now is ensure that
there is a bridge, without messing with how the network is configured.

So rename the function.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 ts-xen-install |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ts-xen-install b/ts-xen-install
index ef26cba..69884a3 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -247,7 +247,7 @@ sub hosts () {
     });
 }
 
-sub nodhcp () {
+sub ensurebridge () {
     target_editfile_root($ho, "/etc/network/interfaces",
                          "etc-network-interfaces", sub {
         my $physif= get_host_property($ho,'interface force',undef);
@@ -339,6 +339,6 @@ if ($checkmode) {
     adjustconfig();
     setupboot();
     setupinitd();
-    nodhcp();
+    ensurebridge();
     hosts();
 }
-- 
1.7.10.4