default xen "make world" again dependent on non-xenbits repositories and non-http protocol

default xen "make world" again dependent on non-xenbits repositories and non-http protocol

[Dan Magenheimer]

Argh!  I know there's been a lot of effort over the years
on driving the Xen build mechanism to work from a single
server (xenbits).  I notice that (probably for awhile, but
I just noticed it) a default "make world" on a fresh
clone again depends on a different server, namely
git.kernel.org.  If it is down or otherwise inaccessible
(e.g. due to git proxying) the Xen "make world" fails.

I guess one can get around it by avoiding the pv_ops
kernel with:

KERNELS=linux-2.6-xen0 make world

but it would be nice if "make world" defaulted to fetch
all the bits via http and preferably from one server.
It's no fun to launch a "make world" on a fresh clone
on a fresh machine, expecting it to take hours and then
come back to that machine hours later only to find that
the build died hours ago.

</gripe off>
Dan

Re: default xen "make world" again dependent on non-xenbits repositories and non-http protocol

[Ian Jackson]

Dan Magenheimer writes ("[Xen-devel] default xen "make world" again dependent on non-xenbits repositories and non-http protocol"):
> Argh!  I know there's been a lot of effort over the years
> on driving the Xen build mechanism to work from a single
> server (xenbits).  I notice that (probably for awhile, but
> I just noticed it) a default "make world" on a fresh
> clone again depends on a different server, namely
> git.kernel.org.

As far as I'm concerned we've made a conscious decision to make an
exception for git.kernel.org, which is after all a very high-profile
site that a lot of people depend on which ought not to be down very
often or for very long.

The kernel is big and has complex development processes and running
our own infrastructure for hosting it doesn't seem sensible.

And, of course, building it is optional (although enabled by default).

Ian.

RE: default xen "make world" again dependent on non-xenbits repositories and non-http protocol

[Dan Magenheimer]

> As far as I'm concerned we've made a conscious decision to make an
> exception for git.kernel.org, which is after all a very high-profile
> site that a lot of people depend on which ought not to be down very
> often or for very long.

Understood, but the problem (in this case) is not the server
but the protocol... I don't claim to understand this well, but
I'm told that git isn't handled by standard proxies.

I believe there's an http hg mirror for the kernel but
I don't know if it works for anything but the main branch.

Lacking any other solution, is there any easy way for "make world"
to determine immediately if git.kernel.org is NOT accessible
via the git protocol (or at least via something like "make check")?

OR as an absolute minimum the issue should be in the main README.

Yes, I know I should be able to work around this, but I am
in part speaking up for all the newbies that would like to work
on Xen but find "getting started" is too difficult.

Thanks,
Dan

> -----Original Message-----
> From: Ian Jackson [mailto:Ian.Jackson@eu.citrix.com]
> Sent: Wednesday, October 07, 2009 9:24 AM
> To: Dan Magenheimer
> Cc: Xen-Devel (E-mail)
> Subject: Re: [Xen-devel] default xen "make world" again dependent on
> non-xenbits repositories and non-http protocol
> 
> 
> Dan Magenheimer writes ("[Xen-devel] default xen "make world" 
> again dependent on non-xenbits repositories and non-http protocol"):
> > Argh!  I know there's been a lot of effort over the years
> > on driving the Xen build mechanism to work from a single
> > server (xenbits).  I notice that (probably for awhile, but
> > I just noticed it) a default "make world" on a fresh
> > clone again depends on a different server, namely
> > git.kernel.org.
> 
> As far as I'm concerned we've made a conscious decision to make an
> exception for git.kernel.org, which is after all a very high-profile
> site that a lot of people depend on which ought not to be down very
> often or for very long.
> 
> The kernel is big and has complex development processes and running
> our own infrastructure for hosting it doesn't seem sensible.
> 
> And, of course, building it is optional (although enabled by default).
> 
> Ian.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

RE: default xen "make world" again dependent on non-xenbits repositories and non-http protocol

[Stefano Stabellini]

On Wed, 7 Oct 2009, Dan Magenheimer wrote:
> > As far as I'm concerned we've made a conscious decision to make an
> > exception for git.kernel.org, which is after all a very high-profile
> > site that a lot of people depend on which ought not to be down very
> > often or for very long.
> 
> Understood, but the problem (in this case) is not the server
> but the protocol... I don't claim to understand this well, but
> I'm told that git isn't handled by standard proxies.
> 
> I believe there's an http hg mirror for the kernel but
> I don't know if it works for anything but the main branch.
> 
> Lacking any other solution, is there any easy way for "make world"
> to determine immediately if git.kernel.org is NOT accessible
> via the git protocol (or at least via something like "make check")?
> 
> OR as an absolute minimum the issue should be in the main README.
> 
> Yes, I know I should be able to work around this, but I am
> in part speaking up for all the newbies that would like to work
> on Xen but find "getting started" is too difficult.
> 

We could make everyone happy using git over http:

diff -r 6f63970032a3 buildconfigs/mk.linux-2.6-pvops
--- a/buildconfigs/mk.linux-2.6-pvops	Fri Oct 02 09:10:27 2009 +0100
+++ b/buildconfigs/mk.linux-2.6-pvops	Wed Oct 07 17:11:07 2009 +0100
@@ -5,7 +5,7 @@
 
 XEN_LINUX_CONFIG_UPDATE := buildconfigs/enable-xen-config
 
-XEN_LINUX_GIT_URL ?= git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
+XEN_LINUX_GIT_URL ?= http://www.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
 XEN_LINUX_GIT_REMOTEBRANCH ?= xen/master
 
 EXTRAVERSION ?=
diff -r 6f63970032a3 buildconfigs/mk.linux-2.6-tip-latest
--- a/buildconfigs/mk.linux-2.6-tip-latest	Fri Oct 02 09:10:27 2009 +0100
+++ b/buildconfigs/mk.linux-2.6-tip-latest	Wed Oct 07 17:11:07 2009 +0100
@@ -5,7 +5,7 @@
 
 XEN_LINUX_CONFIG_UPDATE := buildconfigs/enable-xen-config
 
-XEN_LINUX_GIT_URL ?= git://git.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git
+XEN_LINUX_GIT_URL ?= http://www.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git
 XEN_LINUX_GIT_REMOTEBRANCH ?= auto-latest
 
 EXTRAVERSION ?=

---

but the problem is that it seems to be broken for Jeremy's tree (the
others work fine).

RE: default xen "make world" again dependent on non-xenbits repositories and non-http protocol

[Dan Magenheimer]

> We could make everyone happy using git over http:

Excellent.  +1!

> +XEN_LINUX_GIT_URL ?= 
> http://www.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
>  :
> but the problem is that it seems to be broken for Jeremy's tree (the
> others work fine).

I'm told that Jeremy needs to set something in his git config
file on kernel.org for this to work.

> -----Original Message-----
> From: Stefano Stabellini [mailto:stefano.stabellini@eu.citrix.com]
> Sent: Wednesday, October 07, 2009 10:12 AM
> To: Dan Magenheimer
> Cc: Ian Jackson; Xen-Devel (E-mail)
> Subject: RE: [Xen-devel] default xen "make world" again dependent on
> non-xenbits repositories and non-http protocol
> 
> 
> On Wed, 7 Oct 2009, Dan Magenheimer wrote:
> > > As far as I'm concerned we've made a conscious decision to make an
> > > exception for git.kernel.org, which is after all a very 
> high-profile
> > > site that a lot of people depend on which ought not to be 
> down very
> > > often or for very long.
> > 
> > Understood, but the problem (in this case) is not the server
> > but the protocol... I don't claim to understand this well, but
> > I'm told that git isn't handled by standard proxies.
> > 
> > I believe there's an http hg mirror for the kernel but
> > I don't know if it works for anything but the main branch.
> > 
> > Lacking any other solution, is there any easy way for "make world"
> > to determine immediately if git.kernel.org is NOT accessible
> > via the git protocol (or at least via something like "make check")?
> > 
> > OR as an absolute minimum the issue should be in the main README.
> > 
> > Yes, I know I should be able to work around this, but I am
> > in part speaking up for all the newbies that would like to work
> > on Xen but find "getting started" is too difficult.
> > 
> 
> We could make everyone happy using git over http:
> 
> diff -r 6f63970032a3 buildconfigs/mk.linux-2.6-pvops
> --- a/buildconfigs/mk.linux-2.6-pvops	Fri Oct 02 09:10:27 2009 +0100
> +++ b/buildconfigs/mk.linux-2.6-pvops	Wed Oct 07 17:11:07 2009 +0100
> @@ -5,7 +5,7 @@
>  
>  XEN_LINUX_CONFIG_UPDATE := buildconfigs/enable-xen-config
>  
> -XEN_LINUX_GIT_URL ?= 
> git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
> +XEN_LINUX_GIT_URL ?= 
> http://www.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
>  XEN_LINUX_GIT_REMOTEBRANCH ?= xen/master
>  
>  EXTRAVERSION ?=
> diff -r 6f63970032a3 buildconfigs/mk.linux-2.6-tip-latest
> --- a/buildconfigs/mk.linux-2.6-tip-latest	Fri Oct 02 
> 09:10:27 2009 +0100
> +++ b/buildconfigs/mk.linux-2.6-tip-latest	Wed Oct 07 
> 17:11:07 2009 +0100
> @@ -5,7 +5,7 @@
>  
>  XEN_LINUX_CONFIG_UPDATE := buildconfigs/enable-xen-config
>  
> -XEN_LINUX_GIT_URL ?= 
> git://git.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git
> +XEN_LINUX_GIT_URL ?= 
> http://www.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git
>  XEN_LINUX_GIT_REMOTEBRANCH ?= auto-latest
>  
>  EXTRAVERSION ?=
> 
> ---
> 
> but the problem is that it seems to be broken for Jeremy's tree (the
> others work fine).
> 
>

Re: default xen "make world" again dependent on non-xenbits repositories and non-http protocol

[Konrad Rzeszutek Wilk]

On Wed, Oct 07, 2009 at 08:42:22AM -0700, Dan Magenheimer wrote:
> > As far as I'm concerned we've made a conscious decision to make an
> > exception for git.kernel.org, which is after all a very high-profile
> > site that a lot of people depend on which ought not to be down very
> > often or for very long.
> 
> Understood, but the problem (in this case) is not the server
> but the protocol... I don't claim to understand this well, but
> I'm told that git isn't handled by standard proxies.

You can use work-arounds. 'corkscrew' I believe does the job?

http://www.mtu.net/~engstrom/ssh-proxy.php

http://groups.google.com/group/android-platform/browse_thread/thread/1e082b3c4a9d85ca?fwc=1&pli=1

> 
> I believe there's an http hg mirror for the kernel but
> I don't know if it works for anything but the main branch.
> 
> Lacking any other solution, is there any easy way for "make world"
> to determine immediately if git.kernel.org is NOT accessible
> via the git protocol (or at least via something like "make check")?
> 
> OR as an absolute minimum the issue should be in the main README.
> 
> Yes, I know I should be able to work around this, but I am
> in part speaking up for all the newbies that would like to work
> on Xen but find "getting started" is too difficult.

Re: default xen "make world" again dependent on non-xenbits repositories and non-http protocol

[Keir Fraser]

On 07/10/2009 17:26, "Dan Magenheimer" <dan.magenheimer@oracle.com> wrote:

>> +XEN_LINUX_GIT_URL ?=
>> http://www.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
>>  :
>> but the problem is that it seems to be broken for Jeremy's tree (the
>> others work fine).
> 
> I'm told that Jeremy needs to set something in his git config
> file on kernel.org for this to work.

If Jeremy fixes that I will apply Stefano's patch.

 -- Keir

Re: default xen "make world" again dependent on non-xenbits repositories and non-http protocol

[Jeremy Fitzhardinge]

On 10/07/09 09:30, Keir Fraser wrote:
> If Jeremy fixes that I will apply Stefano's patch.
>   

OK, http://www.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
appears to work now.

    J