[Qemu-devel] buildbot failure in qemu on xen_x86_64_debian_6_0

[Qemu-devel] buildbot failure in qemu on xen_x86_64_debian_6_0

[qemu]

The Buildbot has detected a new failure on builder xen_x86_64_debian_6_0 while building qemu.
Full details are available at:
 http://buildbot.b1-systems.de/qemu/builders/xen_x86_64_debian_6_0/builds/38

Buildbot URL: http://buildbot.b1-systems.de/qemu/

Buildslave for this Build: yuzuki

Build Reason: The Nightly scheduler named 'nightly_xen' triggered this build
Build Source Stamp: [branch xen-next] HEAD
Blamelist: 

BUILD FAILED: failed git

sincerely,
 -The Buildbot

[Qemu-devel] buildbot failure in qemu on xen_x86_64_debian_6_0

[qemu]

The Buildbot has detected a new failure on builder xen_x86_64_debian_6_0 while building qemu.
Full details are available at:
 http://buildbot.b1-systems.de/qemu/builders/xen_x86_64_debian_6_0/builds/361

Buildbot URL: http://buildbot.b1-systems.de/qemu/

Buildslave for this Build: yuzuki

Build Reason: The Nightly scheduler named 'nightly_xen_next' triggered this build
Build Source Stamp: [branch xen-next] HEAD
Blamelist: 

BUILD FAILED: failed git

sincerely,
 -The Buildbot

[Qemu-devel] buildbot failure in qemu on xen_x86_64_debian_6_0

[qemu]

The Buildbot has detected a new failure on builder xen_x86_64_debian_6_0 while building qemu.
Full details are available at:
 http://buildbot.b1-systems.de/qemu/builders/xen_x86_64_debian_6_0/builds/356

Buildbot URL: http://buildbot.b1-systems.de/qemu/

Buildslave for this Build: yuzuki

Build Reason: The Nightly scheduler named 'nightly_xen_next' triggered this build
Build Source Stamp: [branch xen-next] HEAD
Blamelist: 

BUILD FAILED: failed git

sincerely,
 -The Buildbot

Re: [Qemu-devel] buildbot failure in qemu on xen_x86_64_debian_6_0

[Gerd Hoffmann]

  Hi,

> i finally took a closer look on this failed git reports. It seems like all the 
> "git failed" mails are not due to a too short reply period. It is due to the 
> buildslave version used on "yuzuki" is not recent enough to perform retries at 
> all.

Another git issue:  Fedora 16 fails because git 1.7.7 errors out on 'git
branch -M master'.  Guess that needs to be fixed in the GitPoller @
buildmaster?

I've also some python bits to send out the log tail with the mails, see
below.

cheers,
  Gerd

===================== [ cut here ] ========================

def kraxelMessageFormatter(mode, name, build, results, master_status):
    result = Results[results]
    defmsg = mail.defaultMessage(mode, name, build, results, master_status);
    text = list();
    text.append(defmsg['body']);

    # get log for last step
    logs = build.getLogs()
    for log in reversed(logs):
        if log.getName() == 'stdio':
            break
    content = log.getText().splitlines() # Note: can be VERY LARGE
    url = "%s/steps/%s/logs/%s" % (master_status.getURLForThing(build),
                                   log.getStep().getName(),
                                   log.getName())

    # append log info to standard message
    text.append("========== log tail ==========")
    for line in content[-32:]:
        text.append(unicode(line,'utf8'))
    text.append("")
    text.append("========== full log ==========")
    text.append(url);

    return { 'body' : "\n".join(text), 'type' : 'plain' }

[ ... ]

mn = mail.MailNotifier([ other args ]
                       messageFormatter=kraxelMessageFormatter);

Re: [Qemu-devel] buildbot failure in qemu on xen_x86_64_debian_6_0

[Stefan Hajnoczi]

On Tue, Nov 1, 2011 at 8:07 AM, Daniel Gollub <gollub@b1-systems.de> wrote:
> Hi Stefan,
>
> On Tuesday, November 01, 2011 08:36:46 AM Stefan Weil wrote:
>> As far as I know from previous mails, the buildbots already do a retry
>> when git fails. The retry period used today is obviously too short.
>
> i finally took a closer look on this failed git reports. It seems like all the
> "git failed" mails are not due to a too short reply period. It is due to the
> buildslave version used on "yuzuki" is not recent enough to perform retries at
> all.
>
> Other buildslaves which have buildbot version 0.8.2 and greater perform
> retries correctly and seem to require only one retry to finally succeed.
> "yuzuki" is running buildbot version 0.7.12 and doesn't perform any retry and
> cause nearly all "git failed" mails since we introduced git-fetch retries.
>
> Stefan, could you update your buildslave "yuzuki"? In meanwhile I try to setup
> mine buildslave also with a new version of buildbot.

Thanks for the pointer.  I have upgraded yuzuki to buildbot 0.8.5.

Stefan

Re: [Qemu-devel] buildbot failure in qemu on xen_x86_64_debian_6_0

[Daniel Gollub]

[-- Attachment #1: Type: Text/Plain, Size: 1200 bytes --]

Hi Stefan,

On Tuesday, November 01, 2011 08:36:46 AM Stefan Weil wrote:
> As far as I know from previous mails, the buildbots already do a retry
> when git fails. The retry period used today is obviously too short.

i finally took a closer look on this failed git reports. It seems like all the 
"git failed" mails are not due to a too short reply period. It is due to the 
buildslave version used on "yuzuki" is not recent enough to perform retries at 
all.

Other buildslaves which have buildbot version 0.8.2 and greater perform 
retries correctly and seem to require only one retry to finally succeed. 
"yuzuki" is running buildbot version 0.7.12 and doesn't perform any retry and 
cause nearly all "git failed" mails since we introduced git-fetch retries.

Stefan, could you update your buildslave "yuzuki"? In meanwhile I try to setup 
mine buildslave also with a new version of buildbot.


Best Regards,
Daniel

-- 
Daniel Gollub
Linux Consultant & Developer
Tel.: +49-160 47 73 970 
Mail: gollub@b1-systems.de

B1 Systems GmbH
Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de
GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [Qemu-devel] buildbot failure in qemu on xen_x86_64_debian_6_0

[Stefan Weil]

Am 01.11.2011 07:19, schrieb qemu@buildbot.b1-systems.de:
> The Buildbot has detected a new failure on builder 
> xen_x86_64_debian_6_0 while building qemu.
> Full details are available at:
> http://buildbot.b1-systems.de/qemu/builders/xen_x86_64_debian_6_0/builds/78
>
> Buildbot URL: http://buildbot.b1-systems.de/qemu/
>
> Buildslave for this Build: yuzuki
>
> Build Reason: The Nightly scheduler named 'nightly_xen' triggered this 
> build
> Build Source Stamp: [branch xen-next] HEAD
> Blamelist:
>
> BUILD FAILED: failed git
>
> sincerely,
> -The Buildbot

As far as I know from previous mails, the buildbots already do a retry
when git fails. The retry period used today is obviously too short.

We know that git fails rather often and that those failures take some
time. Is there any reason why a buildbot cannot retry until it
succeeds with a much larger time limit than today?

A git retry limit of several hours or even a day would not harm.
Mails from the buildbot are only useful if some user action is
required - when git fails, you can only wait.

Developers who want a fast feedback for their latest commits
can either poll the buildbot, or the buildbot must be configured
to send individual mails with more information.

Regards,
Stefan Weil

[Qemu-devel] buildbot failure in qemu on xen_x86_64_debian_6_0

[qemu]

The Buildbot has detected a new failure on builder xen_x86_64_debian_6_0 while building qemu.
Full details are available at:
 http://buildbot.b1-systems.de/qemu/builders/xen_x86_64_debian_6_0/builds/78

Buildbot URL: http://buildbot.b1-systems.de/qemu/

Buildslave for this Build: yuzuki

Build Reason: The Nightly scheduler named 'nightly_xen' triggered this build
Build Source Stamp: [branch xen-next] HEAD
Blamelist: 

BUILD FAILED: failed git

sincerely,
 -The Buildbot

[Qemu-devel] buildbot failure in qemu on xen_x86_64_debian_6_0

[qemu]

The Buildbot has detected a new failure on builder xen_x86_64_debian_6_0 while building qemu.
Full details are available at:
 http://buildbot.b1-systems.de/qemu/builders/xen_x86_64_debian_6_0/builds/55

Buildbot URL: http://buildbot.b1-systems.de/qemu/

Buildslave for this Build: yuzuki

Build Reason: The Nightly scheduler named 'nightly_xen' triggered this build
Build Source Stamp: [branch xen-next] HEAD
Blamelist: 

BUILD FAILED: failed git

sincerely,
 -The Buildbot

[Qemu-devel] buildbot failure in qemu on xen_x86_64_debian_6_0

[qemu]

The Buildbot has detected a new failure on builder xen_x86_64_debian_6_0 while building qemu.
Full details are available at:
 http://buildbot.b1-systems.de/qemu/builders/xen_x86_64_debian_6_0/builds/50

Buildbot URL: http://buildbot.b1-systems.de/qemu/

Buildslave for this Build: yuzuki

Build Reason: The Nightly scheduler named 'nightly_xen' triggered this build
Build Source Stamp: [branch xen-next] HEAD
Blamelist: 

BUILD FAILED: failed git

sincerely,
 -The Buildbot

[Qemu-devel] buildbot failure in qemu on xen_x86_64_debian_6_0

[qemu]

The Buildbot has detected a new failure on builder xen_x86_64_debian_6_0 while building qemu.
Full details are available at:
 http://buildbot.b1-systems.de/qemu/builders/xen_x86_64_debian_6_0/builds/21

Buildbot URL: http://buildbot.b1-systems.de/qemu/

Buildslave for this Build: yuzuki

Build Reason: The Nightly scheduler named 'nightly_xen' triggered this build
Build Source Stamp: [branch xen-next] HEAD
Blamelist: 

BUILD FAILED: failed git

sincerely,
 -The Buildbot