Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[Lars Kurth]

Hi,

as part of a number of tasks to move Xen Project websites to https, we investigated whether we can move our tarballs to a new Xen Project owned domain to download tarballs. Currently tarballs are stored on http://bits.xensource.com, which is a http site only. We do not have sufficient control of bits.xensource.com (which is an Akamai site) to convert the site to https, and are thus potentially exposed to MiM attacks. 

To fix this, the current plan of record is to
- Copy existing tarballs to an existing or new VM
- To expose that VM via the new public URL ftp.xenproject.org (this is non-browsable, thus ftp - we also already have https://downloads.xenproject.org/ to host legacy content)
- To only publish new tarballs on https://ftp.xenproject.org
- To update http://xenproject.org/downloads/xen-archives.html to use the new VM

In most cases, the ftp.xenproject.org site would *not* be exposed directly to users, but via the download manager on xenproject.org. The exception are blog posts and xen-devel@/etc. mails such as https://blog.xenproject.org/2016/05/11/announcing-xen-project-4-7-rc-and-test-day-schedule/

We would either keep existing tarballs on bits.xensource.com OR - if we have sufficient control - implement a 301 redirect to the new site. This would ensure that 3rd party links to tarballs are not broken. 

Does anyone have any objection regarding the name of the site and/or proposal. I am assuming this is non-controversial: if I don't get any objections by end of day Friday 12th, Aug assume we can go ahead with the change.

Best Regards
Lars
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[George Dunlap]

On 08/08/16 12:43, Lars Kurth wrote:
> Hi,
> 
> as part of a number of tasks to move Xen Project websites to https, we investigated whether we can move our tarballs to a new Xen Project owned domain to download tarballs. Currently tarballs are stored on http://bits.xensource.com, which is a http site only. We do not have sufficient control of bits.xensource.com (which is an Akamai site) to convert the site to https, and are thus potentially exposed to MiM attacks. 
> 
> To fix this, the current plan of record is to
> - Copy existing tarballs to an existing or new VM
> - To expose that VM via the new public URL ftp.xenproject.org (this is non-browsable, thus ftp - we also already have https://downloads.xenproject.org/ to host legacy content)
> - To only publish new tarballs on https://ftp.xenproject.org

The pedant in me thinks it's strange to have a server titled "ftp" that
is speaking https rather than ftp.  But we're apparently already using
"downloads.xenproject.org" for something else, and I don't have a better
name, so I suppose it will have to do. :-)

 -George


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[Ian Jackson]

Lars Kurth writes ("Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)"):
> To fix this, the current plan of record is to
> - Copy existing tarballs to an existing or new VM
> - To expose that VM via the new public URL ftp.xenproject.org (this is non-browsable, thus ftp - we also already have https://downloads.xenproject.org/ to host legacy content)
> - To only publish new tarballs on https://ftp.xenproject.org
> - To update http://xenproject.org/downloads/xen-archives.html to use the new VM

We should consider whether the release tarballs (and ancient Xen 3.x
docs archive) could be put on https://downloads.xenproject.org/.

Looking at the webserver directory listing there suggests that we
could fit the `new' content alongside the old.

> We would either keep existing tarballs on bits.xensource.com OR - if we have sufficient control - implement a 301 redirect to the new site. This would ensure that 3rd party links to tarballs are not broken. 

We might be able to manage that.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[Lars Kurth]

> On 8 Aug 2016, at 14:51, Ian Jackson <ian.jackson@eu.citrix.com> wrote:
> 
> Lars Kurth writes ("Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)"):
>> To fix this, the current plan of record is to
>> - Copy existing tarballs to an existing or new VM
>> - To expose that VM via the new public URL ftp.xenproject.org (this is non-browsable, thus ftp - we also already have https://downloads.xenproject.org/ to host legacy content)
>> - To only publish new tarballs on https://ftp.xenproject.org
>> - To update http://xenproject.org/downloads/xen-archives.html to use the new VM
> 
> We should consider whether the release tarballs (and ancient Xen 3.x
> docs archive) could be put on https://downloads.xenproject.org/.
> 
> Looking at the webserver directory listing there suggests that we
> could fit the `new' content alongside the old.

I don't mind. I can ask Credativ whether that is doable from a load perspective. But the answer is probably yes.

I am assuming this would not create any extra complexities for the workflow of creating tarballs? Please confirm.

>> We would either keep existing tarballs on bits.xensource.com OR - if we have sufficient control - implement a 301 redirect to the new site. This would ensure that 3rd party links to tarballs are not broken. 
> 
> We might be able to manage that.

What I thought: a simple change to .htaccess should be all that is needed

Lars
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[Ian Jackson]

Lars Kurth writes ("Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)"):
> I don't mind. I can ask Credativ whether that is doable from a load perspective. But the answer is probably yes.

If it isn't doable then moving the release tarballs load elsewhere is
no help, since it's all coming out of the same rackspace account.

Currently downloads.xenproject.org is hosted on the mail-and-dns VM.

> I am assuming this would not create any extra complexities for the workflow of creating tarballs? Please confirm.

No.

> > We might be able to manage that.
> 
> What I thought: a simple change to .htaccess should be all that is needed

I doubt that.  Akamai are not using Apache.  But we can investigate.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[Lars Kurth]

> On 8 Aug 2016, at 15:04, Ian Jackson <ian.jackson@eu.citrix.com> wrote:
> 
> Lars Kurth writes ("Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)"):
>> I don't mind. I can ask Credativ whether that is doable from a load perspective. But the answer is probably yes.
> 
> If it isn't doable then moving the release tarballs load elsewhere is
> no help, since it's all coming out of the same rackspace account.
> 
> Currently downloads.xenproject.org is hosted on the mail-and-dns VM.
> 
>> I am assuming this would not create any extra complexities for the workflow of creating tarballs? Please confirm.
> 
> No.

Using downloads.xenproject.org seems to be the best way then. 


>>> We might be able to manage that.
>> 
>> What I thought: a simple change to .htaccess should be all that is needed
> 
> I doubt that.  Akamai are not using Apache.  But we can investigate.

Sure

Lars

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[Ian Jackson]

Lars Kurth writes ("Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)"):
> Using downloads.xenproject.org seems to be the best way then. 

I have:

 * Used cvs-repomove to move the primary cvs repository for the
   Xen releases to mail.xenproject.org aka downloads.xenproject.org.
   The repo is in
      /home/downloads-cvs/cvs-repos
   It can be checked out with
      cvs -d mail.xenproject.org:/home/downloads-cvs/cvs-repos co xen.org
   (if you have the appropriate permission, of course)

 * Used cvs-repomove to update my own personal working tree.  If there
   are other working trees, `cvs-repomove' (with no arguments, in the
   appropriate directory) will adjust them.  (cvs-repomove is in the
   Debian package chiark-scripts.)

 * Checked out a copy into a new directory
     /data/downloads.xenproject.org/xen.org

 * Made a symlink `release' in the root of the
   downloads.xen[project].org webtree pointing into the cvs checkout's
   `release'.

See https://downloads.xenproject.org/ for the result.

Lars, what do you think of this ?

Things I have not done:

 * Carefully considered whether the name `release' there is right.

 * Adjusted any web pages referring to the tarballs.

 * Anything about tidying up the other things found in
   https://downloads.xenproject.org/ (most of which are historical, I
   think).

 * Provided any anonymous access to the cvs repo containing the
   downloads webtree.  I think this is unnecessary.

 * Updated the release checklist.

 * Thought properly about what to do about the akamai account.
   I guess we should keep updating the files on akamai for the
   foreseeable future - at least, as long as we can.  Many downstreams
   seem to have url-guessing arrangements which use the
   `bits.xensource.com' URLs.  I'm also not sure about the status of
   the other files in the akamai account.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[Lars Kurth]

Ian,

thank you!

> On 4 Oct 2016, at 16:20, Ian Jackson <ian.jackson@eu.citrix.com> wrote:
> 
> Lars Kurth writes ("Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)"):
>> Using downloads.xenproject.org seems to be the best way then. 
> 
> I have:
> 
> * Used cvs-repomove to move the primary cvs repository for the
>   Xen releases to mail.xenproject.org aka downloads.xenproject.org.
>   The repo is in
>      /home/downloads-cvs/cvs-repos
>   It can be checked out with
>      cvs -d mail.xenproject.org:/home/downloads-cvs/cvs-repos co xen.org
>   (if you have the appropriate permission, of course)
> 
> * Used cvs-repomove to update my own personal working tree.  If there
>   are other working trees, `cvs-repomove' (with no arguments, in the
>   appropriate directory) will adjust them.  (cvs-repomove is in the
>   Debian package chiark-scripts.)
> 
> * Checked out a copy into a new directory
>     /data/downloads.xenproject.org/xen.org
> 
> * Made a symlink `release' in the root of the
>   downloads.xen[project].org webtree pointing into the cvs checkout's
>   `release'.
> 
> See https://downloads.xenproject.org/ for the result.

As far as I can see, these are at https://downloads.xenproject.org/release/

> Lars, what do you think of this ?

That works for me.

> 
> Things I have not done:
> 
> * Carefully considered whether the name `release' there is right.

As a not, it may be better to use release/xen/... instead of release/...
That would allow us to use the same directory for drivers and other releases we may make in future

> * Adjusted any web pages referring to the tarballs.

Not done this either. I can try with a couple, once we agreed the above and see whether it works. But I don't expect any issues.

> * Anything about tidying up the other things found in
>   https://downloads.xenproject.org/ (most of which are historical, I
>   think).

I think we can probably archive some stuff in https://downloads.xenproject.org/ (aka move it into an archive directory and at some point delete it).
I just noticed that I can't ssh into that machine, which is mostly why I have not cleaned this up.

> 
> * Provided any anonymous access to the cvs repo containing the
>   downloads webtree.  I think this is unnecessary.
> 
> * Updated the release checklist.
> 
> * Thought properly about what to do about the akamai account.
>   I guess we should keep updating the files on akamai for the
>   foreseeable future - at least, as long as we can.  Many downstreams
>   seem to have url-guessing arrangements which use the
>   `bits.xensource.com' URLs.  I'm also not sure about the status of
>   the other files in the akamai account.

I don't know enough about akamai to know what redirect capabilities exist

Lars
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[Ian Jackson]

Lars Kurth writes ("Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)"):
> > On 4 Oct 2016, at 16:20, Ian Jackson <ian.jackson@eu.citrix.com> wrote:
> > See https://downloads.xenproject.org/ for the result.
> 
> As far as I can see, these are at https://downloads.xenproject.org/release/
> 
> > Lars, what do you think of this ?
> 
> That works for me.
...
> As a not, it may be better to use release/xen/... instead of
> release/...  That would allow us to use the same directory for
> drivers and other releases we may make in future

Sure, I don't have an opinion.  I have changed this, so it's now
under:
  https://downloads.xenproject.org/release/xen/

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[Ian Jackson]

Ian Jackson writes ("Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)"):
> Sure, I don't have an opinion.  I have changed this, so it's now
> under:
>   https://downloads.xenproject.org/release/xen/

No-one has objected, so we are now committing to this.  The new URLs
will be primary for the forthcoming RC (Wei will send an announcement
when it's ready).

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[Lars Kurth]

> On 12 Oct 2016, at 11:38, Ian Jackson <ian.jackson@eu.citrix.com> wrote:
> 
> Ian Jackson writes ("Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)"):
>> Sure, I don't have an opinion.  I have changed this, so it's now
>> under:
>>  https://downloads.xenproject.org/release/xen/
> 
> No-one has objected, so we are now committing to this.  The new URLs
> will be primary for the forthcoming RC (Wei will send an announcement
> when it's ready).
> 
> Ian.

Wei, I will update the Test Day wiki and blog post
Lars


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[Steven Haigh]

[-- Attachment #1.1.1: Type: text/plain, Size: 970 bytes --]

On 12/10/16 21:38, Ian Jackson wrote:
> Ian Jackson writes ("Re: Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)"):
>> Sure, I don't have an opinion.  I have changed this, so it's now
>> under:
>>   https://downloads.xenproject.org/release/xen/
> 
> No-one has objected, so we are now committing to this.  The new URLs
> will be primary for the forthcoming RC (Wei will send an announcement
> when it's ready).

I missed this previously in the rest of the list happenings.

I'm actually glad this is happening. Having predictable naming / pathing
of the xen tarballs is fantastic.

I lothe going via the web site to download the file.html which ends up
being the filename on the system. Would be much nicer to have a direct
download link automatically generated that works.

As such, +1 from me :)

-- 
Steven Haigh

Email: netwiz@crc.id.au
Web: https://www.crc.id.au
Phone: (03) 9001 6090 - 0412 935 897


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel