Re: qemu.org server bandwidth report (May 2021)

[Stefan Hajnoczi <stefanha@gmail.com>]

On Mon, May 10, 2021 at 4:53 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>
>
> Stefan Hajnoczi <stefanha@gmail.com> writes:
>
> > On Mon, May 10, 2021 at 11:31 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
> >>
> >> On Mon, May 10, 2021 at 10:49:19AM +0100, Stefan Hajnoczi wrote:
> >>
> >> > qemu.org bandwidth usage has been as follows:
> >> > - Jan: 12.56 TB
> >> > - Feb: 10.55 TB
> >> > - Mar: 10.28 TB
> >> > - Apr: 7.62 TB
> >> >
> >> > In May qemu.org has averaged 232.25 GB/day so far putting it on track
> >> > for 7 TB total this month.
> <snip>
> >>
> >> For the https:// URIs should we setup a HTTP redirect ?
> >>
> >> When git clones via https it fetches some specific paths which
> >> I believe we have rules for in httpd conf:
> >>
> >>   ScriptAliasMatch "^/git/(.*\.git/(HEAD|info/refs))$" \
> >>     /usr/libexec/git-core/git-http-backend/$1
> >>   ScriptAliasMatch "^/git/(.*\.git/git-(upload|receive)-pack)$" \
> >>     /usr/libexec/git-core/git-http-backend/$1
> >>
> >> If we set those URI path matches to send a HTTP 307 redirect
> >> to gitlab, that would essentially kill off our git traffic on
> >> qemu.org, while still allowing the qemu.org gitweb UI to
> >> work normally. The downside is that people won't notice to
> >> update their clone URIs. Still feels like an easy win and
> >> we can easily remove the redirect if we use code 307.
> >
> > I remember there were concerns about warning messages that
> > git-clone(1) prints when an HTTP redirect is encountered? If everyone
> > is okay I can turn the git-http-backend(1) aliases into HTTP 307
> > redirects to GitLab.
> >
> >> Third, qemu 4.2.0....
> >>
> >> I wonder why this is the most popular. Something must be linking
> >> to this, as you would otherwise have to go out of your way to
> >> search it out.
> >>
> >> Do we have any stats on the referrer URLs ?
> >>
> >> I wonder if there's some key page(s) that need updating ?
> >>
> >> If we're unlucky there might be some CI system that hardcoded
> >> use of qemu 4.2.0 that's frequently pulling it.
> >
> > The majority of qemu-4.2.0.tar.xz downloads have the wget user agent
> > and no referrer. The IP addresses don't have a clear pattern (there
> > are many).
>
> I've just checked my Gentoo box and I can see it pulls directly from:
>
>   SRC_URI="https://download.qemu.org/${P}.tar.xz"
>
> and the *9999* builds (HEAD, which I doubt many people use) points to:
>
>   EGIT_REPO_URI="https://git.qemu.org/git/qemu.git"
>
> but the lowest version is 5.2.0 and 6.0.0 is already in the repo so
> these particular users probably are a minority.
>
> However Google does point to a number of instructions online that have
> wget and "qemu-4.2.0.tar.xz" in them.

Thank you for checking this! Are you in touch with the maintainers or
able to tweak the ebuilds or documentation?

Thanks,
Stefan