All of lore.kernel.org
 help / color / mirror / Atom feed
* [Bug 97260] R9 290 low performance in Linux 4.7
@ 2016-08-09 13:45 bugzilla-daemon
  2016-08-09 13:48 ` bugzilla-daemon
                   ` (24 more replies)
  0 siblings, 25 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-09 13:45 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 1405 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

            Bug ID: 97260
           Summary: R9 290 low performance in Linux 4.7
           Product: DRI
           Version: unspecified
          Hardware: x86-64 (AMD64)
                OS: Linux (All)
            Status: NEW
          Keywords: regression
          Severity: major
          Priority: high
         Component: DRM/Radeon
          Assignee: dri-devel@lists.freedesktop.org
          Reporter: magolobel@yahoo.com.br

Starting with Linux Kernel 4.7, the Radeon R9 290 started showing very low
performance, about 20/30% of what is available up to kernel 4.6. As can be seen
in this article, the performance regression started even before the launch of
kernel 4.7 RC1, using Alex Deucher's drm-next-4.7 branch:

http://www.phoronix.com/scan.php?page=article&item=radeon-drm-next-47&num=1

Some time later it was speculated that the problem may be with DPM:

http://www.phoronix.com/scan.php?page=news_item&px=Linux-4.7-R9-290-Regression

To replicate the bug, install Ubuntu 16.04 64bits and install kernel 4.7 from
here (preferably above rc5):

http://kernel.ubuntu.com/~kernel-ppa/mainline/ 

The performance regression will show right away. Rebooting the system and
choosing a kernel version up to 4.6 will fix the bug.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 3027 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
@ 2016-08-09 13:48 ` bugzilla-daemon
  2016-08-10 18:45 ` bugzilla-daemon
                   ` (23 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-09 13:48 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 211 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #1 from Alex Deucher <alexdeucher@gmail.com> ---
Can you bisect?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 966 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
  2016-08-09 13:48 ` bugzilla-daemon
@ 2016-08-10 18:45 ` bugzilla-daemon
  2016-08-10 19:17 ` bugzilla-daemon
                   ` (22 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-10 18:45 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 343 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #2 from Jan Ziak <0xe2.0x9a.0x9b@gmail.com> ---
Just a note:

I am not experiencing such a performance regression on R9 390 with kernel 4.7.0
- 4.8.0-rc1, amdgpu.ko kernel module, Gentoo Linux.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1097 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
  2016-08-09 13:48 ` bugzilla-daemon
  2016-08-10 18:45 ` bugzilla-daemon
@ 2016-08-10 19:17 ` bugzilla-daemon
  2016-08-10 19:40 ` bugzilla-daemon
                   ` (21 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-10 19:17 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 541 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #3 from Clésio Luiz <magolobel@yahoo.com.br> ---
(In reply to Alex Deucher from comment #1)
> Can you bisect?

Unfortunately, no. But the best I can say is that it started very early in
kernel 4.7 schedule, since the Phoronix benchmark on the first link was made
before 4.7 RC1 was out, using Alex Deucher's drm-next-4.7 branch, in
05/14/2016.

The problem is still present in 4.8 RC1.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1367 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (2 preceding siblings ...)
  2016-08-10 19:17 ` bugzilla-daemon
@ 2016-08-10 19:40 ` bugzilla-daemon
  2016-08-10 19:43 ` bugzilla-daemon
                   ` (20 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-10 19:40 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 574 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #4 from Alex Deucher <alexdeucher@gmail.com> ---
Unfortunately, none of the power management code for these asics changed during
that time period, so nothing jumps out as a likely culprit.  Did you also
change your mesa stack or firmware in that time period?  With everything else
the same, does the issue go away when using an older kernel?  If so, any chance
you could narrow it down which kernels have the issue vs. not?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1329 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (3 preceding siblings ...)
  2016-08-10 19:40 ` bugzilla-daemon
@ 2016-08-10 19:43 ` bugzilla-daemon
  2016-08-10 19:51 ` bugzilla-daemon
                   ` (19 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-10 19:43 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 235 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #5 from Alex Deucher <alexdeucher@gmail.com> ---
Can you verify that 4.6 works properly?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 990 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (4 preceding siblings ...)
  2016-08-10 19:43 ` bugzilla-daemon
@ 2016-08-10 19:51 ` bugzilla-daemon
  2016-08-10 20:25 ` bugzilla-daemon
                   ` (18 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-10 19:51 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 329 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #6 from Alex Deucher <alexdeucher@gmail.com> ---
Does reverting 5e031d9fe8b0741f11d49667dfc3ebf5454121fd (drm/radeon/pm: update
current crtc info after setting the powerstate) help?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1084 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (5 preceding siblings ...)
  2016-08-10 19:51 ` bugzilla-daemon
@ 2016-08-10 20:25 ` bugzilla-daemon
  2016-08-10 20:52 ` bugzilla-daemon
                   ` (17 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-10 20:25 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 1605 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

Kai <kai@dev.carbon-project.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kai@dev.carbon-project.org

--- Comment #7 from Kai <kai@dev.carbon-project.org> ---
I can confirm this issue with 4.7.0 (vs. 4.6.4 in my case). In XCOM 2 I'm
losing a third in FPS (down to <= 19 FPS from 26-29 FPS with 4.6.4, "measured"
with Gallium's HUD) or about a fifth in The Talos Principle 64 bit (down to 43
Avg FPS from 53 Avg FPS, measured by the benchmark of TTP when run for 60
seconds). The only change between these numbers is the different kernel (and
that's with all the VM faults I'm seeing for XCOM 2 with 4.6.4 on occasion and
haven't been able to reproduce with 4.7.0 yet).

The stack I'm using (Debian testing as a base) is:
GPU: Hawaii PRO [Radeon R9 290] (ChipID = 0x67b1)
Mesa: Git:master/3fb4a9b3b3
libdrm: 2.4.70-1
LLVM: SVN:trunk/r277307 (4.0 devel)
X.Org: 2:1.18.4-1
Linux: 4.6.4 / 4.7.0
Firmware: Git:master/c170c8d957 (placed in /lib/firmware/updates; fallback
would be firmware-amd-graphics/20160110-1)
libclc: Git:master/785bfd3719
DDX: 1:7.7.0-1

I'm not going to be able to do a bisect before the weekend and even then I
don't have high hopes, since the last time I tried bisecting the kernel all the
merge commits screwed me over and I was unable to find a single offending
commit.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 2897 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (6 preceding siblings ...)
  2016-08-10 20:25 ` bugzilla-daemon
@ 2016-08-10 20:52 ` bugzilla-daemon
  2016-08-10 21:00 ` bugzilla-daemon
                   ` (16 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-10 20:52 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 902 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #8 from Kai <kai@dev.carbon-project.org> ---
(In reply to Alex Deucher from comment #6)
> Does reverting 5e031d9fe8b0741f11d49667dfc3ebf5454121fd (drm/radeon/pm:
> update current crtc info after setting the powerstate) help?

No, it does not for me (I fetched the patch from
<https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/?id=5e031d9fe8b0741f11d49667dfc3ebf5454121fd>
and applied it with patch -p1 -R < /path/to.patch). The Talos Principle is
still down to 43 FPS, haven't tested XCOM 2 or anything else.

Btw, not sure if it can help tracking down the cause: the GPU doesn't seem to
go into its highest performance mode or at least the fan is not reaching its
highest RPM when I'm on 4.7.0 (with or without the revert).

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1864 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (7 preceding siblings ...)
  2016-08-10 20:52 ` bugzilla-daemon
@ 2016-08-10 21:00 ` bugzilla-daemon
  2016-08-11 17:07 ` bugzilla-daemon
                   ` (15 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-10 21:00 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 1191 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #9 from Alex Deucher <alexdeucher@gmail.com> ---
(In reply to Kai from comment #8)
> (In reply to Alex Deucher from comment #6)
> > Does reverting 5e031d9fe8b0741f11d49667dfc3ebf5454121fd (drm/radeon/pm:
> > update current crtc info after setting the powerstate) help?
> 
> No, it does not for me (I fetched the patch from
> <https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/
> ?id=5e031d9fe8b0741f11d49667dfc3ebf5454121fd> and applied it with patch -p1
> -R < /path/to.patch). The Talos Principle is still down to 43 FPS, haven't
> tested XCOM 2 or anything else.
> 
> Btw, not sure if it can help tracking down the cause: the GPU doesn't seem
> to go into its highest performance mode or at least the fan is not reaching
> its highest RPM when I'm on 4.7.0 (with or without the revert).

You can check by running apps and monitoring the content of
/sys/kernel/debug/dri/64/radeon_pm_info
Do you see it scaling up under load on the problematic kernels?  Does forcing
the clocks to high via sysfs work ok?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 2191 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (8 preceding siblings ...)
  2016-08-10 21:00 ` bugzilla-daemon
@ 2016-08-11 17:07 ` bugzilla-daemon
  2016-08-11 19:35 ` bugzilla-daemon
                   ` (14 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-11 17:07 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 2267 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #10 from Kai <kai@dev.carbon-project.org> ---
(In reply to Alex Deucher from comment #9)
> (In reply to Kai from comment #8)
> > (In reply to Alex Deucher from comment #6)
> > > Does reverting 5e031d9fe8b0741f11d49667dfc3ebf5454121fd (drm/radeon/pm:
> > > update current crtc info after setting the powerstate) help?
> > 
> > No, it does not for me (I fetched the patch from
> > <https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/
> > ?id=5e031d9fe8b0741f11d49667dfc3ebf5454121fd> and applied it with patch -p1
> > -R < /path/to.patch). The Talos Principle is still down to 43 FPS, haven't
> > tested XCOM 2 or anything else.
> > 
> > Btw, not sure if it can help tracking down the cause: the GPU doesn't seem
> > to go into its highest performance mode or at least the fan is not reaching
> > its highest RPM when I'm on 4.7.0 (with or without the revert).
> 
> You can check by running apps and monitoring the content of
> /sys/kernel/debug/dri/64/radeon_pm_info
> Do you see it scaling up under load on the problematic kernels?  Does
> forcing the clocks to high via sysfs work ok?

While the clocks *do* scale up on 4.7.0, it doesn't go as high as 4.6.4. On
4.6.4 I'm reaching the maximum clocks for my GPU (power level avg    sclk:
98000 mclk: 125000). With 4.7.0 the highest I'm seeing is "power level avg   
sclk: 90843 mclk: 125000" and the overall level is way lower (lots of sclk
values starting with a 3) compared to 4.6.4 where most values are > 60000 for
sclk.

Note: I used watch to dump /sys/kernel/debug/dri/64/radeon_pm_info every five
seconds to a file. Between runs the actual numbers logged vary, but the general
trend stays the same.

Writing "high" to /sys/class/drm/card0/device/power_dpm_force_performance_level
locks the clock indeed to the highest values (sclk: 98000 mclk: 125000) on
4.7.0 and recovers almost all of the lost performance. There's still a small
hit to performance.

Note: I've only tested "The Talos Principle" so far, if you need me to test
XCOM 2 or other titles as well, let me know.

Do you still need the bisect?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 3393 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (9 preceding siblings ...)
  2016-08-11 17:07 ` bugzilla-daemon
@ 2016-08-11 19:35 ` bugzilla-daemon
  2016-08-12 19:58 ` [Bug 97260] [bisected] " bugzilla-daemon
                   ` (13 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-11 19:35 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 507 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #11 from Alex Deucher <alexdeucher@gmail.com> ---
(In reply to Kai from comment #10)
> 
> Do you still need the bisect?

That would be great.  As I mentioned before, there weren't really any dpm
related changes for these parts in 4.7 (there weren't many changes to radeon
period), so I don't really have any ideas on what would have caused it right
now.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1337 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] [bisected] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (10 preceding siblings ...)
  2016-08-11 19:35 ` bugzilla-daemon
@ 2016-08-12 19:58 ` bugzilla-daemon
  2016-08-12 19:58 ` bugzilla-daemon
                   ` (12 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-12 19:58 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 1909 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

Kai <kai@dev.carbon-project.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |bisected
                 CC|                            |michel@daenzer.net
            Summary|R9 290 low performance in   |[bisected] R9 290 low
                   |Linux 4.7                   |performance in Linux 4.7

--- Comment #12 from Kai <kai@dev.carbon-project.org> ---
Ok, after 14 steps of bisection I identified the first bad commit as:

c63dd758589b1f7e8398841d1f443f06ebfbcefc is the first bad commit
commit c63dd758589b1f7e8398841d1f443f06ebfbcefc
Author: Michel Dänzer <michel.daenzer@amd.com>
Date:   Fri Apr 1 18:51:34 2016 +0900

    drm/radeon: Support DRM_MODE_PAGE_FLIP_ASYNC

    When this flag is set, we program the hardware to execute the flip
    during horizontal blank (i.e. for the next scanline) instead of during
    vertical blank (i.e. for the next frame).

    Currently this is only supported on ASICs which have a page flip
    completion interrupt (>= R600), and only if the use_pflipirq parameter
    has value 2 (the default).

    Reviewed-by: Christian König <christian.koenig@amd.com>
    Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
    Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

:040000 040000 2f3d8295e7fa2809a3546a23c64da33311e624b9
6cd9fd9b53df0942efab559295e4c11fc6cc0463 M      drivers

An additional build of 4.7.0 with c63dd758589b1f7e8398841d1f443f06ebfbcefc
reverted maintains the performance level of 4.6.x.

Adding Michel to the CC list of this bug, since he authored the offending
commit. Let me know, if you need anything else.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 3685 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] [bisected] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (11 preceding siblings ...)
  2016-08-12 19:58 ` [Bug 97260] [bisected] " bugzilla-daemon
@ 2016-08-12 19:58 ` bugzilla-daemon
  2016-08-13 18:49 ` bugzilla-daemon
                   ` (11 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-12 19:58 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 389 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #13 from Kai <kai@dev.carbon-project.org> ---
Created attachment 125755
  --> https://bugs.freedesktop.org/attachment.cgi?id=125755&action=edit
git bisect log output for the bisection run leading to
c63dd758589b1f7e8398841d1f443f06ebfbcefc

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1445 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] [bisected] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (12 preceding siblings ...)
  2016-08-12 19:58 ` bugzilla-daemon
@ 2016-08-13 18:49 ` bugzilla-daemon
  2016-08-15 17:57 ` bugzilla-daemon
                   ` (10 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-13 18:49 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 341 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #14 from Alexander Tsoy <alexander@tsoy.me> ---
(In reply to Kai from comment #12)

This is interesting. The same bad commit was found here:
https://bugzilla.kernel.org/show_bug.cgi?id=119631

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1222 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] [bisected] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (13 preceding siblings ...)
  2016-08-13 18:49 ` bugzilla-daemon
@ 2016-08-15 17:57 ` bugzilla-daemon
  2016-08-15 18:12 ` bugzilla-daemon
                   ` (9 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-15 17:57 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 452 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #15 from nadro-linux@wp.pl ---
I also noticed a big performance regression since kernel v4.7rc1. With kernel
4.6 all works fine. I have Radeon R9 380 2GB (Sapphire ITX Compact). I'll try
to build a custom kernel without "support for DRM_MODE_PAGE_FLIP_ASYNC" commit
and check if performance will be ok.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1214 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] [bisected] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (14 preceding siblings ...)
  2016-08-15 17:57 ` bugzilla-daemon
@ 2016-08-15 18:12 ` bugzilla-daemon
  2016-08-15 20:02 ` bugzilla-daemon
                   ` (8 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-15 18:12 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 676 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #16 from Alex Deucher <alexdeucher@gmail.com> ---
Does just reverting this chunk fix the issue?

@@ -1630,6 +1631,9 @@ int radeon_modeset_init(struct radeon_device *rdev)

        rdev->ddev->mode_config.funcs = &radeon_mode_funcs;

+       if (radeon_use_pflipirq == 2 && rdev->family >= CHIP_R600)
+               rdev->ddev->mode_config.async_page_flip = true;
+
        if (ASIC_IS_DCE5(rdev)) {
                rdev->ddev->mode_config.max_width = 16384;
                rdev->ddev->mode_config.max_height = 16384;

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1512 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] [bisected] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (15 preceding siblings ...)
  2016-08-15 18:12 ` bugzilla-daemon
@ 2016-08-15 20:02 ` bugzilla-daemon
  2016-08-16  3:52 ` bugzilla-daemon
                   ` (7 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-15 20:02 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 398 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #17 from nadro-linux@wp.pl ---
Thanks for your help, when I removed followind line:
"adev->ddev->mode_config.async_page_flip = true;"
from all three files "dce_v8_0.c", "dce_v10_0.c", "dce_v11_0.c" (I use AMDGPU
driver) a performance is fine again!

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1196 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] [bisected] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (16 preceding siblings ...)
  2016-08-15 20:02 ` bugzilla-daemon
@ 2016-08-16  3:52 ` bugzilla-daemon
  2016-08-16  6:15 ` bugzilla-daemon
                   ` (6 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-16  3:52 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 1141 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #18 from Michel Dänzer <michel@daenzer.net> ---
Created attachment 125808
  --> https://bugs.freedesktop.org/attachment.cgi?id=125808&action=edit
radeon: Add some page flip debugging output

Well, that's a surprising result of the bisection.

I can imagine two possible causes, or possibly some combination thereof:

* The processing of asynchronous flips or the corresponding completion
interrupts
  is delayed for some reason
* Using flips instead of blits for buffer swaps lowers the load on the GPU 3D
  engine, so the SMU doesn't switch to higher clocks

The attached debugging patch should give us more information about the former.
With it applied, run the following while an affected application is running in
fullscreen:

sudo sh -c 'echo 2 >/sys/module/drm/parameters/debug'; sleep 1; sudo sh -c
'echo 0 >/sys/module/drm/parameters/debug'

Then attach the resulting dmesg output.

BTW, does the problem still happen with Alex's current drm-next-4.9-wip branch?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 2187 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] [bisected] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (17 preceding siblings ...)
  2016-08-16  3:52 ` bugzilla-daemon
@ 2016-08-16  6:15 ` bugzilla-daemon
  2016-08-16 15:49 ` bugzilla-daemon
                   ` (5 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-16  6:15 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 414 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #19 from Michel Dänzer <michel@daenzer.net> ---
BTW, there are some potential workarounds:

* Disable DRI3 for affected games with the environment variable
  LIBGL_DRI3_DISABLE=1

* Enable sync-to-vblank in affected applications, or force it with
vblank_mode=3

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1191 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] [bisected] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (18 preceding siblings ...)
  2016-08-16  6:15 ` bugzilla-daemon
@ 2016-08-16 15:49 ` bugzilla-daemon
  2016-08-17  8:05 ` bugzilla-daemon
                   ` (4 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-16 15:49 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 3078 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #20 from Kai <kai@dev.carbon-project.org> ---
Created attachment 125820
  --> https://bugs.freedesktop.org/attachment.cgi?id=125820&action=edit
dmesg output with additional debug info from attachment 125808

(In reply to Alex Deucher from comment #16)
> Does just reverting this chunk fix the issue?
> 
> @@ -1630,6 +1631,9 @@ int radeon_modeset_init(struct radeon_device *rdev)
>  
>         rdev->ddev->mode_config.funcs = &radeon_mode_funcs;
>  
> +       if (radeon_use_pflipirq == 2 && rdev->family >= CHIP_R600)
> +               rdev->ddev->mode_config.async_page_flip = true;
> +
>         if (ASIC_IS_DCE5(rdev)) {
>                 rdev->ddev->mode_config.max_width = 16384;
>                 rdev->ddev->mode_config.max_height = 16384;

Yes, just removing the default enable here has the same effect as reverting the
entire patch.

(In reply to Michel Dänzer from comment #18)
> Created attachment 125808
> radeon: Add some page flip debugging output
> 
> Well, that's a surprising result of the bisection.
> 
> I can imagine two possible causes, or possibly some combination thereof:
> 
> * The processing of asynchronous flips or the corresponding completion
> interrupts
>   is delayed for some reason
> * Using flips instead of blits for buffer swaps lowers the load on the GPU 3D
>   engine, so the SMU doesn't switch to higher clocks
> 
> The attached debugging patch should give us more information about the
> former. With it applied, run the following while an affected application is
> running in fullscreen:
> 
> sudo sh -c 'echo 2 >/sys/module/drm/parameters/debug'; sleep 1; sudo sh -c
> 'echo 0 >/sys/module/drm/parameters/debug'
> 
> Then attach the resulting dmesg output.

Here you go. That was generated by running XCOM 2.

> BTW, does the problem still happen with Alex's current drm-next-4.9-wip
> branch?

Haven't tested that yet. Maybe somebody else can do that. ;-)

(In reply to Michel Dänzer from comment #19)
> BTW, there are some potential workarounds:
> 
> * Disable DRI3 for affected games with the environment variable
>   LIBGL_DRI3_DISABLE=1
> 
> * Enable sync-to-vblank in affected applications, or force it with
> vblank_mode=3

Well, this is going to be odd: I had VSync enabled in XCOM, since without that
option I got poorer performance in the past than with it. Now, after your note
here I actually *disabled* the VSync option in the game. 4.6.4 (or 4.7.0
without the offending commit/the enable removed) shows no longer a performance
difference and I'm getting ~30 FPS in XCOM 2. BUT with your ASYNC patch
(vanilla 4.7.0) and VSync turned of in the game gives me ALSO ~30 FPS! I'd
still say this is a regression as there is no difference without your patch,
but maybe this information can help you in narrowing down the cause?

I hope I haven't missed any open question. Let me know if you need anything
else.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 5162 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] [bisected] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (19 preceding siblings ...)
  2016-08-16 15:49 ` bugzilla-daemon
@ 2016-08-17  8:05 ` bugzilla-daemon
  2016-08-17 16:31 ` bugzilla-daemon
                   ` (3 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-17  8:05 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 378 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #21 from Michel Dänzer <michel@daenzer.net> ---
Created attachment 125838
  --> https://bugs.freedesktop.org/attachment.cgi?id=125838&action=edit
loader/dri3: Always use 3 back buffers when flipping

Does this Mesa patch help?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1436 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] [bisected] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (20 preceding siblings ...)
  2016-08-17  8:05 ` bugzilla-daemon
@ 2016-08-17 16:31 ` bugzilla-daemon
  2016-08-18  1:02 ` bugzilla-daemon
                   ` (2 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-17 16:31 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 993 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #22 from Kai <kai@dev.carbon-project.org> ---
(In reply to Michel Dänzer from comment #21)
> Created attachment 125838
> loader/dri3: Always use 3 back buffers when flipping
> 
> Does this Mesa patch help?

Yes! With attachment 125838 applied on top of Mesa master 607ab6d3bf I'm seeing
the same performance on 4.7.1 as on 4.6.4.
You can have my
 Tested-by: Kai Wasserbäch <kai@dev.carbon-project.org>

The full stack used (Debian testing as a base) was:
GPU: Hawaii PRO [Radeon R9 290] (ChipID = 0x67b1)
Mesa: Git:master/3fb4a9b3b3 / Git:master/607ab6d3bf + attachment 125838
libdrm: 2.4.70-1
LLVM: SVN:trunk/r278555 (4.0 devel)
X.Org: 2:1.18.4-1
Linux: 4.7.1
Firmware: Git:master/c170c8d957 (placed in /lib/firmware/updates; fallback
would be firmware-amd-graphics/20160110-1)
libclc: Git:master/785bfd3719
DDX: 1:7.7.0-1

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 2971 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] [bisected] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (21 preceding siblings ...)
  2016-08-17 16:31 ` bugzilla-daemon
@ 2016-08-18  1:02 ` bugzilla-daemon
  2016-08-18  1:03 ` bugzilla-daemon
  2016-08-19  6:22 ` bugzilla-daemon
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-18  1:02 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 342 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #23 from Michel Dänzer <michel@daenzer.net> ---
Note that since the crash happens when Xorg tries to use the GPU, testing with
DRI3 and Xorg not using the GPU directly cannot trigger the crash.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1119 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] [bisected] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (22 preceding siblings ...)
  2016-08-18  1:02 ` bugzilla-daemon
@ 2016-08-18  1:03 ` bugzilla-daemon
  2016-08-19  6:22 ` bugzilla-daemon
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-18  1:03 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 216 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

--- Comment #24 from Michel Dänzer <michel@daenzer.net> ---
Sorry, wrong bug. :(

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 993 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug 97260] [bisected] R9 290 low performance in Linux 4.7
  2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
                   ` (23 preceding siblings ...)
  2016-08-18  1:03 ` bugzilla-daemon
@ 2016-08-19  6:22 ` bugzilla-daemon
  24 siblings, 0 replies; 26+ messages in thread
From: bugzilla-daemon @ 2016-08-19  6:22 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 747 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=97260

Michel Dänzer <michel@daenzer.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|unspecified                 |git
          Component|DRM/Radeon                  |Mesa core
         QA Contact|                            |mesa-dev@lists.freedesktop.
                   |                            |org
            Product|DRI                         |Mesa
           Assignee|dri-devel@lists.freedesktop |mesa-dev@lists.freedesktop.
                   |.org                        |org

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1852 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2016-08-19  6:22 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-08-09 13:45 [Bug 97260] R9 290 low performance in Linux 4.7 bugzilla-daemon
2016-08-09 13:48 ` bugzilla-daemon
2016-08-10 18:45 ` bugzilla-daemon
2016-08-10 19:17 ` bugzilla-daemon
2016-08-10 19:40 ` bugzilla-daemon
2016-08-10 19:43 ` bugzilla-daemon
2016-08-10 19:51 ` bugzilla-daemon
2016-08-10 20:25 ` bugzilla-daemon
2016-08-10 20:52 ` bugzilla-daemon
2016-08-10 21:00 ` bugzilla-daemon
2016-08-11 17:07 ` bugzilla-daemon
2016-08-11 19:35 ` bugzilla-daemon
2016-08-12 19:58 ` [Bug 97260] [bisected] " bugzilla-daemon
2016-08-12 19:58 ` bugzilla-daemon
2016-08-13 18:49 ` bugzilla-daemon
2016-08-15 17:57 ` bugzilla-daemon
2016-08-15 18:12 ` bugzilla-daemon
2016-08-15 20:02 ` bugzilla-daemon
2016-08-16  3:52 ` bugzilla-daemon
2016-08-16  6:15 ` bugzilla-daemon
2016-08-16 15:49 ` bugzilla-daemon
2016-08-17  8:05 ` bugzilla-daemon
2016-08-17 16:31 ` bugzilla-daemon
2016-08-18  1:02 ` bugzilla-daemon
2016-08-18  1:03 ` bugzilla-daemon
2016-08-19  6:22 ` bugzilla-daemon

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.