All of lore.kernel.org
 help / color / mirror / Atom feed
* [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads.
@ 2017-02-12 15:34 bugzilla-daemon
  2017-02-12 15:38 ` bugzilla-daemon
                   ` (12 more replies)
  0 siblings, 13 replies; 14+ messages in thread
From: bugzilla-daemon @ 2017-02-12 15:34 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 4070 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=99780

            Bug ID: 99780
           Summary: Flickering artifacts in radeonsi driver with divergent
                    texture reads.
           Product: Mesa
           Version: unspecified
          Hardware: x86-64 (AMD64)
                OS: Linux (All)
            Status: NEW
          Severity: normal
          Priority: medium
         Component: Drivers/Gallium/radeonsi
          Assignee: dri-devel@lists.freedesktop.org
          Reporter: dark_sylinc@yahoo.com.ar
        QA Contact: dri-devel@lists.freedesktop.org

Created attachment 129530
  --> https://bugs.freedesktop.org/attachment.cgi?id=129530&action=edit
Sample repro

I'm an Ogre3D developer.

We noticed after updating to LLVM 4.0 that there were visual glitches:
http://imgur.com/43tIuI6
http://imgur.com/iBhxoBj
(this glitches tend to flicker over time *even if everything is stationary*)
This bug is *NOT* present if using LLVM 3.9.1
This bug happens with radeon open source drivers. Using llvmpipe works as
intended.

I tried to reproduce this bug in a clean environment but failed. So instead I
stripped down as much as possible to a minimal repro sample out of our code;
which I am attaching.

We noticed that this flicker happens alongside the difference between the PSSM
splits, but not always:
http://imgur.com/130B4mE
We added a tint to differentiate the splits. But as you can see there are a few
artifacts that happens within one tint; though the vast majority of the
artifacts happens alongside the line.

A PSSM split in shader code means this:
uniform sampler2DShadow texShadowMap[2];
float fShadow = 1.0;
vec3 tint = vec3( 1, 1, 1 );
if( inPs.depth <= pass.pssmSplitPoints0 )
{
    fShadow = getShadow( texShadowMap[0], inPs.posL0, vec4( 0.000488281,
0.000488281, 1, 1 ) );
    tint = vec3( 0.0, 0, 1 );
}
else if( inPs.depth <= pass.pssmSplitPoints1 )
{
    fShadow = getShadow( texShadowMap[1], inPs.posL1, vec4( 0.000976562,
0.000976562, 1, 1 ) );
    tint = vec3(0.0, 1.0, 0.0 );
}

Everything from inPs comes from the vertex shader.

After stripping down as much as possible, our getShadow declaration is this:
float getShadow( sampler2DShadow shadowMap, vec4 psPosLN, vec4 invShadowMapSize
)
{
        float fDepth = psPosLN.z;
        vec2 uv = psPosLN.xy / psPosLN.w;

        float retVal = 0;

        vec2 fW;
        vec4 c;

        retVal += texture( shadowMap, vec3( uv, fDepth ) ).r;

        return retVal;
}


I've seen a similar artifact before in the Windows AMD drivers back when I was
experimenting with bindless textures and I purposedly managed to mix textures
from divergent branches into the same wavefront. Not sure if that's what's
happening here though.

Just run launchapp.sh; select OpenGL renderer, and click accept. It will launch
the sample.

NOTE: If when running the sample you encounter this problem:
Created GL 4.3 context
libGL error: failed to create drawable
libGL error: failed to create drawable
X Error of failed request:  0
  Major opcode of failed request:  156 (GLX)
  Minor opcode of failed request:  26 (X_GLXMakeContextCurrent)
  Serial number of failed request:  28
  Current serial number in output stream:  28

The just run it again. I do not know why this happens, it happens rarely (like
1 in 10 runs) and I haven't yet been able to determine why. It sounds like a
race condition inside Mesa (because the bug won't manifest while debugging
where loading is significantly slowed down loading all symbols), but I don't
have enough evidence to back this up, and it's not the reason of this bug
ticket. 

If you have more issues running the sample let me know, I can provide
assistance. This sample needs SDL2 to be installed in your system.

I'm using AMD Radeon HD 7770 1GB
Ubuntu 16.10
Mesa git 5bc222ebafddd14f2329f5096287b51d798a6431
LLVM 4.0 SVN 293947
Kernel 4.8.0-37-generic

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 5682 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads.
  2017-02-12 15:34 [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads bugzilla-daemon
@ 2017-02-12 15:38 ` bugzilla-daemon
  2017-02-12 15:39 ` bugzilla-daemon
                   ` (11 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla-daemon @ 2017-02-12 15:38 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 383 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=99780

--- Comment #1 from Matias N. Goldberg <dark_sylinc@yahoo.com.ar> ---
Created attachment 129531
  --> https://bugs.freedesktop.org/attachment.cgi?id=129531&action=edit
Pixel Shader used to receive these shadows (already stripped down & with tint)

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1475 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads.
  2017-02-12 15:34 [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads bugzilla-daemon
  2017-02-12 15:38 ` bugzilla-daemon
@ 2017-02-12 15:39 ` bugzilla-daemon
  2017-02-12 16:19 ` bugzilla-daemon
                   ` (10 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla-daemon @ 2017-02-12 15:39 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 384 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=99780

--- Comment #2 from Matias N. Goldberg <dark_sylinc@yahoo.com.ar> ---
Created attachment 129532
  --> https://bugs.freedesktop.org/attachment.cgi?id=129532&action=edit
Vertex Shader used to receive these shadows (already stripped down & with tint)

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1478 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads.
  2017-02-12 15:34 [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads bugzilla-daemon
  2017-02-12 15:38 ` bugzilla-daemon
  2017-02-12 15:39 ` bugzilla-daemon
@ 2017-02-12 16:19 ` bugzilla-daemon
  2017-02-12 16:20 ` bugzilla-daemon
                   ` (9 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla-daemon @ 2017-02-12 16:19 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 648 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=99780

--- Comment #3 from Matias N. Goldberg <dark_sylinc@yahoo.com.ar> ---
Created attachment 129533
  --> https://bugs.freedesktop.org/attachment.cgi?id=129533&action=edit
IR Generated by LLVM 3.9.1

Took me a while to find the IR out of all the noise.

You should consider adding a Mesa extension to give human-readable names to
shaders. It would help everyone in debugging A LOT.

I don't mind having
glSetNameMESA( shaderId, "My Filename" );
and internally Mesa keeps a std::map<int, string> or similar.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1640 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads.
  2017-02-12 15:34 [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads bugzilla-daemon
                   ` (2 preceding siblings ...)
  2017-02-12 16:19 ` bugzilla-daemon
@ 2017-02-12 16:20 ` bugzilla-daemon
  2017-02-12 16:30 ` bugzilla-daemon
                   ` (8 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla-daemon @ 2017-02-12 16:20 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 329 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=99780

--- Comment #4 from Matias N. Goldberg <dark_sylinc@yahoo.com.ar> ---
Created attachment 129534
  --> https://bugs.freedesktop.org/attachment.cgi?id=129534&action=edit
IR Generated by LLVM 4.0

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1301 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads.
  2017-02-12 15:34 [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads bugzilla-daemon
                   ` (3 preceding siblings ...)
  2017-02-12 16:20 ` bugzilla-daemon
@ 2017-02-12 16:30 ` bugzilla-daemon
  2017-02-12 16:31 ` bugzilla-daemon
                   ` (7 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla-daemon @ 2017-02-12 16:30 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 374 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=99780

--- Comment #5 from Matias N. Goldberg <dark_sylinc@yahoo.com.ar> ---
Created attachment 129535
  --> https://bugs.freedesktop.org/attachment.cgi?id=129535&action=edit
ASM 4.0

Those readfirstlane instructions are incredibly suspicious

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1312 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads.
  2017-02-12 15:34 [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads bugzilla-daemon
                   ` (4 preceding siblings ...)
  2017-02-12 16:30 ` bugzilla-daemon
@ 2017-02-12 16:31 ` bugzilla-daemon
  2017-02-12 17:34 ` bugzilla-daemon
                   ` (6 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla-daemon @ 2017-02-12 16:31 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 319 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=99780

--- Comment #6 from Matias N. Goldberg <dark_sylinc@yahoo.com.ar> ---
Created attachment 129536
  --> https://bugs.freedesktop.org/attachment.cgi?id=129536&action=edit
ASM from 3.9.1

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1271 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads.
  2017-02-12 15:34 [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads bugzilla-daemon
                   ` (5 preceding siblings ...)
  2017-02-12 16:31 ` bugzilla-daemon
@ 2017-02-12 17:34 ` bugzilla-daemon
  2017-02-12 17:54 ` bugzilla-daemon
                   ` (5 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla-daemon @ 2017-02-12 17:34 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 420 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=99780

--- Comment #7 from Matias N. Goldberg <dark_sylinc@yahoo.com.ar> ---
Analyzing the ASM it's clear what's happening: the 3.9.1 version branches
everything.
The 4.0 version branches the calculations, then at the end uses readfirstlane
(this is WRONG) and finally performs one sample.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1252 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads.
  2017-02-12 15:34 [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads bugzilla-daemon
                   ` (6 preceding siblings ...)
  2017-02-12 17:34 ` bugzilla-daemon
@ 2017-02-12 17:54 ` bugzilla-daemon
  2017-02-12 23:42 ` bugzilla-daemon
                   ` (4 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla-daemon @ 2017-02-12 17:54 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 333 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=99780

--- Comment #8 from Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl> ---
I think this is the issue described in 

https://reviews.llvm.org/D26348

Not idea when the fix is going to go through though.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1209 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads.
  2017-02-12 15:34 [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads bugzilla-daemon
                   ` (7 preceding siblings ...)
  2017-02-12 17:54 ` bugzilla-daemon
@ 2017-02-12 23:42 ` bugzilla-daemon
  2017-02-15 15:21 ` bugzilla-daemon
                   ` (3 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla-daemon @ 2017-02-12 23:42 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 525 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=99780

--- Comment #9 from Matias N. Goldberg <dark_sylinc@yahoo.com.ar> ---
Thanks for the referral.

I commented there, and they replied back with a potential fix:

> I believe it is very easy to fix, no need to change LLVM: marking the texture operation as having unknown side-effect should be enough.

Any comments? (perhaps you already know this but the performance hit could be
huge?)

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1388 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads.
  2017-02-12 15:34 [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads bugzilla-daemon
                   ` (8 preceding siblings ...)
  2017-02-12 23:42 ` bugzilla-daemon
@ 2017-02-15 15:21 ` bugzilla-daemon
  2017-02-16 16:24 ` bugzilla-daemon
                   ` (2 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla-daemon @ 2017-02-15 15:21 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 428 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=99780

--- Comment #10 from Matias N. Goldberg <dark_sylinc@yahoo.com.ar> ---
I've built LLVM 5.0 from latest SVN.
I've applied the patch from https://reviews.llvm.org/D26348

The issue still persists. I don't know if anything else was needed (e.g.
changes to radeonsi to make use of the patch?).

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1307 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads.
  2017-02-12 15:34 [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads bugzilla-daemon
                   ` (9 preceding siblings ...)
  2017-02-15 15:21 ` bugzilla-daemon
@ 2017-02-16 16:24 ` bugzilla-daemon
  2017-02-16 18:47 ` bugzilla-daemon
  2017-03-15 13:46 ` bugzilla-daemon
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla-daemon @ 2017-02-16 16:24 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 418 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=99780

--- Comment #11 from Nicolai Hähnle <nhaehnle@gmail.com> ---
Created attachment 129675
  --> https://bugs.freedesktop.org/attachment.cgi?id=129675&action=edit
Mesa patch

Yes, you need this patch for Mesa as well.

I'm going to pick up the LLVM patch again as well these days.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1439 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads.
  2017-02-12 15:34 [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads bugzilla-daemon
                   ` (10 preceding siblings ...)
  2017-02-16 16:24 ` bugzilla-daemon
@ 2017-02-16 18:47 ` bugzilla-daemon
  2017-03-15 13:46 ` bugzilla-daemon
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla-daemon @ 2017-02-16 18:47 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 315 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=99780

--- Comment #12 from Matias N. Goldberg <dark_sylinc@yahoo.com.ar> ---
I can confirm applying the patch you uploaded (plus LLVM's patch of course)
fixes the problem.

Thank you.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1148 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads.
  2017-02-12 15:34 [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads bugzilla-daemon
                   ` (11 preceding siblings ...)
  2017-02-16 18:47 ` bugzilla-daemon
@ 2017-03-15 13:46 ` bugzilla-daemon
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla-daemon @ 2017-03-15 13:46 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 653 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=99780

Samuel Pitoiset <samuel.pitoiset@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #13 from Samuel Pitoiset <samuel.pitoiset@gmail.com> ---
Just pushed a workaround for this issue in Mesa.

https://cgit.freedesktop.org/mesa/mesa/commit/?id=7751ed39e40e08e5aa0633d018c9f25ad17f9bb0

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 2333 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2017-03-15 13:46 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-02-12 15:34 [Bug 99780] Flickering artifacts in radeonsi driver with divergent texture reads bugzilla-daemon
2017-02-12 15:38 ` bugzilla-daemon
2017-02-12 15:39 ` bugzilla-daemon
2017-02-12 16:19 ` bugzilla-daemon
2017-02-12 16:20 ` bugzilla-daemon
2017-02-12 16:30 ` bugzilla-daemon
2017-02-12 16:31 ` bugzilla-daemon
2017-02-12 17:34 ` bugzilla-daemon
2017-02-12 17:54 ` bugzilla-daemon
2017-02-12 23:42 ` bugzilla-daemon
2017-02-15 15:21 ` bugzilla-daemon
2017-02-16 16:24 ` bugzilla-daemon
2017-02-16 18:47 ` bugzilla-daemon
2017-03-15 13:46 ` bugzilla-daemon

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.