All of lore.kernel.org
 help / color / mirror / Atom feed
* Pixel-perfect frame checks in IGT Chamelium tests and CRC
@ 2017-06-15 13:57 Paul Kocialkowski
  2017-06-15 17:37 ` Lyude Paul
  2017-06-16  4:08 ` chihchung
  0 siblings, 2 replies; 8+ messages in thread
From: Paul Kocialkowski @ 2017-06-15 13:57 UTC (permalink / raw)
  To: Lyude Paul, chamelium-external; +Cc: intel-gfx

Hi,

So far, there are two ways of testing for pixel-perfect frames using the
Chamelium that are in IGT. The first one grabs a full frame from the Chamelium
and compares it pixel-to-pixel with the cairo reference, which works well for
DP/HDMI.

For VGA, this is probably not the case (because the link is analogue). In that
case, I will look into implementing some fuzzy testing, probably inspired by
what piglit (probably) does to compare output frames with references.

For pixel-perfect testing, grabbing a full frame and testing it with memcmp
comes with a significant time penalty (about 2 seconds for 1080p). The Chamelium
also provides a CRC mechanism that is faster and does not require retrieving the
frame, that IGT currently also supports. It compares the CRC calculated by the
Chamelium (implemented in the HDL) with a hardcoded reference value.

This approach currently fails for me (the values I get don't match the hardcoded
reference). There are reasons why it is not really reasonable: fonts rendering
may change between machines (e.g. use of anti-aliasing) and cairo version
changes could introduce slight rendering changes too (not to mention changes in
the test pattern itself). So instead of comparing the CRC with a hardcoded
reference value, I think it would make a lot more sense to actually calculate
the CRC based on the cairo image that is the actual reference (and that we
should assume may change between runs/machines).

I am currently looking into the CRC calculation mechanism used by the Chamelium
and trying to reproduce it in C code. Is this a known algorithm for which a
reference/optimized implementation exists, or something custom that the folks
over at Google came up with?

Any thoughts, comments or suggestions?

Cheers!

-- 
Paul Kocialkowski <paul.kocialkowski@linux.intel.com>
Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Pixel-perfect frame checks in IGT Chamelium tests and CRC
  2017-06-15 13:57 Pixel-perfect frame checks in IGT Chamelium tests and CRC Paul Kocialkowski
@ 2017-06-15 17:37 ` Lyude Paul
  2017-06-15 18:38   ` Jani Nikula
  2017-06-16  8:32   ` Paul Kocialkowski
  2017-06-16  4:08 ` chihchung
  1 sibling, 2 replies; 8+ messages in thread
From: Lyude Paul @ 2017-06-15 17:37 UTC (permalink / raw)
  To: Paul Kocialkowski, chamelium-external; +Cc: intel-gfx

JFYI: Hardcoded CRCs are fine I'm pretty sure, but I might be wrong. As
well, put the chamelium in a metal box. The way the IO board is hooked
up is not really the way something running DP should be hooked up, so
it's very susceptible to electromagnetic interference. This will
usually cause CRC mismatches with DP.

This being said however, I think we should have some better functions
for the chamelium for doing frame comparisons, mainly something that
does fuzzy frame comparisons for stuff like VGA.

On Thu, 2017-06-15 at 16:57 +0300, Paul Kocialkowski wrote:
> Hi,
> 
> So far, there are two ways of testing for pixel-perfect frames using
> the
> Chamelium that are in IGT. The first one grabs a full frame from the
> Chamelium
> and compares it pixel-to-pixel with the cairo reference, which works
> well for
> DP/HDMI.
> 
> For VGA, this is probably not the case (because the link is
> analogue). In that
> case, I will look into implementing some fuzzy testing, probably
> inspired by
> what piglit (probably) does to compare output frames with references.
> 
> For pixel-perfect testing, grabbing a full frame and testing it with
> memcmp
> comes with a significant time penalty (about 2 seconds for 1080p).
> The Chamelium
> also provides a CRC mechanism that is faster and does not require
> retrieving the
> frame, that IGT currently also supports. It compares the CRC
> calculated by the
> Chamelium (implemented in the HDL) with a hardcoded reference value.
> 
> This approach currently fails for me (the values I get don't match
> the hardcoded
> reference). There are reasons why it is not really reasonable: fonts
> rendering
> may change between machines (e.g. use of anti-aliasing) and cairo
> version
> changes could introduce slight rendering changes too (not to mention
> changes in
> the test pattern itself). So instead of comparing the CRC with a
> hardcoded
> reference value, I think it would make a lot more sense to actually
> calculate
> the CRC based on the cairo image that is the actual reference (and
> that we
> should assume may change between runs/machines).
> 
> I am currently looking into the CRC calculation mechanism used by the
> Chamelium
> and trying to reproduce it in C code. Is this a known algorithm for
> which a
> reference/optimized implementation exists, or something custom that
> the folks
> over at Google came up with?
> 
> Any thoughts, comments or suggestions?
> 
> Cheers!
> 
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Pixel-perfect frame checks in IGT Chamelium tests and CRC
  2017-06-15 17:37 ` Lyude Paul
@ 2017-06-15 18:38   ` Jani Nikula
  2017-06-16  8:32   ` Paul Kocialkowski
  1 sibling, 0 replies; 8+ messages in thread
From: Jani Nikula @ 2017-06-15 18:38 UTC (permalink / raw)
  To: Lyude Paul, Paul Kocialkowski, chamelium-external; +Cc: intel-gfx

On Thu, 15 Jun 2017, Lyude Paul <lyude@redhat.com> wrote:
> This being said however, I think we should have some better functions
> for the chamelium for doing frame comparisons, mainly something that
> does fuzzy frame comparisons for stuff like VGA.

Something that takes into account dithering might be of use too.

BR,
Jani.

-- 
Jani Nikula, Intel Open Source Technology Center
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Pixel-perfect frame checks in IGT Chamelium tests and CRC
  2017-06-15 13:57 Pixel-perfect frame checks in IGT Chamelium tests and CRC Paul Kocialkowski
  2017-06-15 17:37 ` Lyude Paul
@ 2017-06-16  4:08 ` chihchung
  2017-06-16  9:28   ` Paul Kocialkowski
  1 sibling, 1 reply; 8+ messages in thread
From: chihchung @ 2017-06-16  4:08 UTC (permalink / raw)
  To: chamelium-external; +Cc: intel-gfx, lyude


[-- Attachment #1.1: Type: text/plain, Size: 2525 bytes --]



Paul Kocialkowski於 2017年6月15日星期四 UTC+8下午9時57分09秒寫道:
>
> Hi, 
>
> So far, there are two ways of testing for pixel-perfect frames using the 
> Chamelium that are in IGT. The first one grabs a full frame from the 
> Chamelium 
> and compares it pixel-to-pixel with the cairo reference, which works well 
> for 
> DP/HDMI. 
>
> For VGA, this is probably not the case (because the link is analogue). In 
> that 
> case, I will look into implementing some fuzzy testing, probably inspired 
> by 
> what piglit (probably) does to compare output frames with references. 
>
> For pixel-perfect testing, grabbing a full frame and testing it with 
> memcmp 
> comes with a significant time penalty (about 2 seconds for 1080p). The 
> Chamelium 
> also provides a CRC mechanism that is faster and does not require 
> retrieving the 
> frame, that IGT currently also supports. It compares the CRC calculated by 
> the 
> Chamelium (implemented in the HDL) with a hardcoded reference value. 
>
> This approach currently fails for me (the values I get don't match the 
> hardcoded 
> reference). There are reasons why it is not really reasonable: fonts 
> rendering 
> may change between machines (e.g. use of anti-aliasing) and cairo version 
> changes could introduce slight rendering changes too (not to mention 
> changes in 
> the test pattern itself). So instead of comparing the CRC with a hardcoded 
> reference value, I think it would make a lot more sense to actually 
> calculate 
> the CRC based on the cairo image that is the actual reference (and that we 
> should assume may change between runs/machines). 
>
> I am currently looking into the CRC calculation mechanism used by the 
> Chamelium 
> and trying to reproduce it in C code. Is this a known algorithm for which 
> a 
> reference/optimized implementation exists, or something custom that the 
> folks 
> over at Google came up with? 
>
> Any thoughts, comments or suggestions? 
>

I feel bad about the stupid hash algorithm I came up with, but here is the 
document:
https://docs.google.com/document/d/1_HjEMA8fBoHkUbpUZq-OXtITfiomCb1HBKN07T-POlY/edit#heading=h.jqek3kkh9qjm
You can also ask it to hash just part of the frame instead of the whole 
frame (i.e. cropping before hashing).

Thanks,
Chih-Chung Chang

 

>
> Cheers! 
>
> -- 
> Paul Kocialkowski <paul.kocialkowski@linux.intel.com> 
> Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo 
>

[-- Attachment #1.2: Type: text/html, Size: 3400 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Pixel-perfect frame checks in IGT Chamelium tests and CRC
  2017-06-15 17:37 ` Lyude Paul
  2017-06-15 18:38   ` Jani Nikula
@ 2017-06-16  8:32   ` Paul Kocialkowski
  1 sibling, 0 replies; 8+ messages in thread
From: Paul Kocialkowski @ 2017-06-16  8:32 UTC (permalink / raw)
  To: Lyude Paul, chamelium-external; +Cc: intel-gfx

On Thu, 2017-06-15 at 13:37 -0400, Lyude Paul wrote:
> JFYI: Hardcoded CRCs are fine I'm pretty sure, but I might be wrong. As
> well, put the chamelium in a metal box. The way the IO board is hooked
> up is not really the way something running DP should be hooked up, so
> it's very susceptible to electromagnetic interference. This will
> usually cause CRC mismatches with DP.

I don't think that is the case here, as pixel-to-pixel comparison succeeds
reliably. I also get the same hashes (that don't match the hardcoded ones)
everytime I run the test. I think the reference picture I have is just different
from the one you hardcoded the CRCs for, as I pointed out (font rendering cannot
be trusted, among other things that can change).

This is why I think we should calculate the CRC instead of using hardcoded ones.

> This being said however, I think we should have some better functions
> for the chamelium for doing frame comparisons, mainly something that
> does fuzzy frame comparisons for stuff like VGA.

Agreed. I will most likely look into that next week.

> On Thu, 2017-06-15 at 16:57 +0300, Paul Kocialkowski wrote:
> > Hi,
> > 
> > So far, there are two ways of testing for pixel-perfect frames using
> > the
> > Chamelium that are in IGT. The first one grabs a full frame from the
> > Chamelium
> > and compares it pixel-to-pixel with the cairo reference, which works
> > well for
> > DP/HDMI.
> > 
> > For VGA, this is probably not the case (because the link is
> > analogue). In that
> > case, I will look into implementing some fuzzy testing, probably
> > inspired by
> > what piglit (probably) does to compare output frames with references.
> > 
> > For pixel-perfect testing, grabbing a full frame and testing it with
> > memcmp
> > comes with a significant time penalty (about 2 seconds for 1080p).
> > The Chamelium
> > also provides a CRC mechanism that is faster and does not require
> > retrieving the
> > frame, that IGT currently also supports. It compares the CRC
> > calculated by the
> > Chamelium (implemented in the HDL) with a hardcoded reference value.
> > 
> > This approach currently fails for me (the values I get don't match
> > the hardcoded
> > reference). There are reasons why it is not really reasonable: fonts
> > rendering
> > may change between machines (e.g. use of anti-aliasing) and cairo
> > version
> > changes could introduce slight rendering changes too (not to mention
> > changes in
> > the test pattern itself). So instead of comparing the CRC with a
> > hardcoded
> > reference value, I think it would make a lot more sense to actually
> > calculate
> > the CRC based on the cairo image that is the actual reference (and
> > that we
> > should assume may change between runs/machines).
> > 
> > I am currently looking into the CRC calculation mechanism used by the
> > Chamelium
> > and trying to reproduce it in C code. Is this a known algorithm for
> > which a
> > reference/optimized implementation exists, or something custom that
> > the folks
> > over at Google came up with?
> > 
> > Any thoughts, comments or suggestions?
> > 
> > Cheers!
> > 
-- 
Paul Kocialkowski <paul.kocialkowski@linux.intel.com>
Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Pixel-perfect frame checks in IGT Chamelium tests and CRC
  2017-06-16  4:08 ` chihchung
@ 2017-06-16  9:28   ` Paul Kocialkowski
  2017-06-16 10:09     ` Chih-Chung Chang
  0 siblings, 1 reply; 8+ messages in thread
From: Paul Kocialkowski @ 2017-06-16  9:28 UTC (permalink / raw)
  To: chihchung, chamelium-external; +Cc: intel-gfx, lyude

Hi,

On Thu, 2017-06-15 at 21:08 -0700, chihchung@google.com wrote:
> Paul Kocialkowski於 2017年6月15日星期四 UTC+8下午9時57分09秒寫道:
> > Hi, 
> > 
> > So far, there are two ways of testing for pixel-perfect frames using the 
> > Chamelium that are in IGT. The first one grabs a full frame from the
> > Chamelium 
> > and compares it pixel-to-pixel with the cairo reference, which works well
> > for 
> > DP/HDMI. 
> > 
> > For VGA, this is probably not the case (because the link is analogue). In
> > that 
> > case, I will look into implementing some fuzzy testing, probably inspired
> > by 
> > what piglit (probably) does to compare output frames with references. 
> > 
> > For pixel-perfect testing, grabbing a full frame and testing it with memcmp 
> > comes with a significant time penalty (about 2 seconds for 1080p). The
> > Chamelium 
> > also provides a CRC mechanism that is faster and does not require retrieving
> > the 
> > frame, that IGT currently also supports. It compares the CRC calculated by
> > the 
> > Chamelium (implemented in the HDL) with a hardcoded reference value. 
> > 
> > This approach currently fails for me (the values I get don't match the
> > hardcoded 
> > reference). There are reasons why it is not really reasonable: fonts
> > rendering 
> > may change between machines (e.g. use of anti-aliasing) and cairo version 
> > changes could introduce slight rendering changes too (not to mention changes
> > in 
> > the test pattern itself). So instead of comparing the CRC with a hardcoded 
> > reference value, I think it would make a lot more sense to actually
> > calculate 
> > the CRC based on the cairo image that is the actual reference (and that we 
> > should assume may change between runs/machines). 
> > 
> > I am currently looking into the CRC calculation mechanism used by the
> > Chamelium 
> > and trying to reproduce it in C code. Is this a known algorithm for which a 
> > reference/optimized implementation exists, or something custom that the
> > folks 
> > over at Google came up with? 
> > 
> > Any thoughts, comments or suggestions? 
> 
> I feel bad about the stupid hash algorithm I came up with, but here is the
> document:
> https://docs.google.com/document/d/1_HjEMA8fBoHkUbpUZq-OXtITfiomCb1HBKN07T-POl
> Y/edit#heading=h.jqek3kkh9qjm
> You can also ask it to hash just part of the frame instead of the whole frame
> (i.e. cropping before hashing).

Thanks for the link to the relevant documentation! I have been following it
closely and came up with the following implementation, which does not produce
the same result. Would you have any idea of what I'm doing wrong?

That function is called with m = 4 and k = { 0, 1, 2, 3 }. I am using a single-
color-frame and the 4 hash16 I get from the Chamelium are identical. They are
however different from the 4 (also identical) that I get from :

int hash16(unsigned char *buffer, int width, int height, int k, int m)
{
	unsigned char r, g, b;
	long int sum = 0;
	int count = 0;
	int index;
	int hash;
	int value;
	int i;

	for (i=0; i < width * height; i++) {
		if ((i % m) != k)
			continue;

		index = i * 3;

		r = buffer[index + 0];
		g = buffer[index + 1];
		b = buffer[index + 2];

		value = r | (g << 8) | (b << 16);
		sum += ++count * value;
	}

	hash = ((sum >> 48) ^ (sum >> 32) ^ (sum >> 16) ^ (sum >> 0)) & 0xffff;

	return hash;
}

I am certain that the r, g, b values are correct.

Thanks!

-- 
Paul Kocialkowski <paul.kocialkowski@linux.intel.com>
Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Pixel-perfect frame checks in IGT Chamelium tests and CRC
  2017-06-16  9:28   ` Paul Kocialkowski
@ 2017-06-16 10:09     ` Chih-Chung Chang
  2017-06-16 10:21       ` Paul Kocialkowski
  0 siblings, 1 reply; 8+ messages in thread
From: Chih-Chung Chang @ 2017-06-16 10:09 UTC (permalink / raw)
  To: Paul Kocialkowski; +Cc: chamelium-external, lyude, intel-gfx


[-- Attachment #1.1: Type: text/plain, Size: 4961 bytes --]

On Fri, Jun 16, 2017 at 5:28 PM, Paul Kocialkowski <
paul.kocialkowski@linux.intel.com> wrote:

> Hi,
>
> On Thu, 2017-06-15 at 21:08 -0700, chihchung@google.com wrote:
> > Paul Kocialkowski於 2017年6月15日星期四 UTC+8下午9時57分09秒寫道:
> > > Hi,
> > >
> > > So far, there are two ways of testing for pixel-perfect frames using
> the
> > > Chamelium that are in IGT. The first one grabs a full frame from the
> > > Chamelium
> > > and compares it pixel-to-pixel with the cairo reference, which works
> well
> > > for
> > > DP/HDMI.
> > >
> > > For VGA, this is probably not the case (because the link is analogue).
> In
> > > that
> > > case, I will look into implementing some fuzzy testing, probably
> inspired
> > > by
> > > what piglit (probably) does to compare output frames with references.
> > >
> > > For pixel-perfect testing, grabbing a full frame and testing it with
> memcmp
> > > comes with a significant time penalty (about 2 seconds for 1080p). The
> > > Chamelium
> > > also provides a CRC mechanism that is faster and does not require
> retrieving
> > > the
> > > frame, that IGT currently also supports. It compares the CRC
> calculated by
> > > the
> > > Chamelium (implemented in the HDL) with a hardcoded reference value.
> > >
> > > This approach currently fails for me (the values I get don't match the
> > > hardcoded
> > > reference). There are reasons why it is not really reasonable: fonts
> > > rendering
> > > may change between machines (e.g. use of anti-aliasing) and cairo
> version
> > > changes could introduce slight rendering changes too (not to mention
> changes
> > > in
> > > the test pattern itself). So instead of comparing the CRC with a
> hardcoded
> > > reference value, I think it would make a lot more sense to actually
> > > calculate
> > > the CRC based on the cairo image that is the actual reference (and
> that we
> > > should assume may change between runs/machines).
> > >
> > > I am currently looking into the CRC calculation mechanism used by the
> > > Chamelium
> > > and trying to reproduce it in C code. Is this a known algorithm for
> which a
> > > reference/optimized implementation exists, or something custom that the
> > > folks
> > > over at Google came up with?
> > >
> > > Any thoughts, comments or suggestions?
> >
> > I feel bad about the stupid hash algorithm I came up with, but here is
> the
> > document:
> > https://docs.google.com/document/d/1_HjEMA8fBoHkUbpUZq-
> OXtITfiomCb1HBKN07T-POl
> > Y/edit#heading=h.jqek3kkh9qjm
> > You can also ask it to hash just part of the frame instead of the whole
> frame
> > (i.e. cropping before hashing).
>
> Thanks for the link to the relevant documentation! I have been following it
> closely and came up with the following implementation, which does not
> produce
> the same result. Would you have any idea of what I'm doing wrong?
>
> That function is called with m = 4 and k = { 0, 1, 2, 3 }. I am using a
> single-
> color-frame and the 4 hash16 I get from the Chamelium are identical. They
> are
> however different from the 4 (also identical) that I get from :
>
> int hash16(unsigned char *buffer, int width, int height, int k, int m)
> {
>         unsigned char r, g, b;
>         long int sum = 0;
>         int count = 0;
>         int index;
>         int hash;
>         int value;
>         int i;
>
>         for (i=0; i < width * height; i++) {
>                 if ((i % m) != k)
>                         continue;
>
>                 index = i * 3;
>
>                 r = buffer[index + 0];
>                 g = buffer[index + 1];
>                 b = buffer[index + 2];
>
>                 value = r | (g << 8) | (b << 16);
>                 sum += ++count * value;
>

Maybe this multiplication overflows?


>         }
>
>         hash = ((sum >> 48) ^ (sum >> 32) ^ (sum >> 16) ^ (sum >> 0)) &
> 0xffff;
>
>         return hash;
> }
>
> I am certain that the r, g, b values are correct.
>
> Thanks!
>
> --
> Paul Kocialkowski <paul.kocialkowski@linux.intel.com>
> Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo
>
> --
> N.B. This list is shared with folks outside Google so please make sure no
> confidential information is being discussed.
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "chamelium-external" group.
> To unsubscribe from this topic, visit https://groups.google.com/a/
> google.com/d/topic/chamelium-external/ITCjBMJHfGM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> chamelium-external+unsubscribe@google.com.
> To post to this group, send email to chamelium-external@google.com.
> To view this discussion on the web visit https://groups.google.com/a/
> google.com/d/msgid/chamelium-external/1497605300.1451.9.
> camel%40linux.intel.com.
>

[-- Attachment #1.2: Type: text/html, Size: 6802 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Pixel-perfect frame checks in IGT Chamelium tests and CRC
  2017-06-16 10:09     ` Chih-Chung Chang
@ 2017-06-16 10:21       ` Paul Kocialkowski
  0 siblings, 0 replies; 8+ messages in thread
From: Paul Kocialkowski @ 2017-06-16 10:21 UTC (permalink / raw)
  To: Chih-Chung Chang; +Cc: chamelium-external, lyude, intel-gfx

Hey,

On Fri, 2017-06-16 at 18:09 +0800, Chih-Chung Chang wrote:
> On Fri, Jun 16, 2017 at 5:28 PM, Paul Kocialkowski
> <paul.kocialkowski@linux.intel.com> wrote:
> > Hi,
> > 
> > On Thu, 2017-06-15 at 21:08 -0700, chihchung@google.com wrote:
> > > Paul Kocialkowski於 2017年6月15日星期四 UTC+8下午9時57分09秒寫道:
> > > > Hi,
> > > >
> > > > So far, there are two ways of testing for pixel-perfect frames using the
> > > > Chamelium that are in IGT. The first one grabs a full frame from the
> > > > Chamelium
> > > > and compares it pixel-to-pixel with the cairo reference, which works
> > well
> > > > for
> > > > DP/HDMI.
> > > >
> > > > For VGA, this is probably not the case (because the link is analogue).
> > In
> > > > that
> > > > case, I will look into implementing some fuzzy testing, probably
> > inspired
> > > > by
> > > > what piglit (probably) does to compare output frames with references.
> > > >
> > > > For pixel-perfect testing, grabbing a full frame and testing it with
> > memcmp
> > > > comes with a significant time penalty (about 2 seconds for 1080p). The
> > > > Chamelium
> > > > also provides a CRC mechanism that is faster and does not require
> > retrieving
> > > > the
> > > > frame, that IGT currently also supports. It compares the CRC calculated
> > by
> > > > the
> > > > Chamelium (implemented in the HDL) with a hardcoded reference value.
> > > >
> > > > This approach currently fails for me (the values I get don't match the
> > > > hardcoded
> > > > reference). There are reasons why it is not really reasonable: fonts
> > > > rendering
> > > > may change between machines (e.g. use of anti-aliasing) and cairo
> > version
> > > > changes could introduce slight rendering changes too (not to mention
> > changes
> > > > in
> > > > the test pattern itself). So instead of comparing the CRC with a
> > hardcoded
> > > > reference value, I think it would make a lot more sense to actually
> > > > calculate
> > > > the CRC based on the cairo image that is the actual reference (and that
> > we
> > > > should assume may change between runs/machines).
> > > >
> > > > I am currently looking into the CRC calculation mechanism used by the
> > > > Chamelium
> > > > and trying to reproduce it in C code. Is this a known algorithm for
> > which a
> > > > reference/optimized implementation exists, or something custom that the
> > > > folks
> > > > over at Google came up with?
> > > >
> > > > Any thoughts, comments or suggestions?
> > >
> > > I feel bad about the stupid hash algorithm I came up with, but here is the
> > > document:
> > > https://docs.google.com/document/d/1_HjEMA8fBoHkUbpUZq-OXtITfiomCb1HBKN07T
> > -POl
> > > Y/edit#heading=h.jqek3kkh9qjm
> > > You can also ask it to hash just part of the frame instead of the whole
> > frame
> > > (i.e. cropping before hashing).
> > 
> > Thanks for the link to the relevant documentation! I have been following it
> > closely and came up with the following implementation, which does not
> > produce
> > the same result. Would you have any idea of what I'm doing wrong?
> > 
> > That function is called with m = 4 and k = { 0, 1, 2, 3 }. I am using a
> > single-
> > color-frame and the 4 hash16 I get from the Chamelium are identical. They
> > are
> > however different from the 4 (also identical) that I get from :
> > 
> > int hash16(unsigned char *buffer, int width, int height, int k, int m)
> > {
> >         unsigned char r, g, b;
> >         long int sum = 0;
> >         int count = 0;
> >         int index;
> >         int hash;
> >         int value;
> >         int i;
> > 
> >         for (i=0; i < width * height; i++) {
> >                 if ((i % m) != k)
> >                         continue;
> > 
> >                 index = i * 3;
> > 
> >                 r = buffer[index + 0];
> >                 g = buffer[index + 1];
> >                 b = buffer[index + 2];
> > 
> >                 value = r | (g << 8) | (b << 16);
> >                 sum += ++count * value;
> 
> Maybe this multiplication overflows?

Good pick, that was it! Now the CRC is the same as the one obtained from the
Chamelium.

Thanks a lot!

> >         }
> > 
> >         hash = ((sum >> 48) ^ (sum >> 32) ^ (sum >> 16) ^ (sum >> 0)) &
> > 0xffff;
> > 
> >         return hash;
> > }
> > 
> > I am certain that the r, g, b values are correct.
> > 
> > Thanks!
> > 
> > --
> > Paul Kocialkowski <paul.kocialkowski@linux.intel.com>
> > Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo
> > 
> > --
> > N.B. This list is shared with folks outside Google so please make sure no
> > confidential information is being discussed.
> > ---
> > You received this message because you are subscribed to a topic in the
> > Google Groups "chamelium-external" group.
> > To unsubscribe from this topic, visit https://groups.google.com/a/google.com
> > /d/topic/chamelium-external/ITCjBMJHfGM/unsubscribe.
> > To unsubscribe from this group and all its topics, send an email to chameliu
> > m-external+unsubscribe@google.com.
> > To post to this group, send email to chamelium-external@google.com.
> > To view this discussion on the web visit https://groups.google.com/a/google.
> > com/d/msgid/chamelium-external/1497605300.1451.9.camel%40linux.intel.com.
> 
> 
-- 
Paul Kocialkowski <paul.kocialkowski@linux.intel.com>
Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2017-06-16 10:21 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-06-15 13:57 Pixel-perfect frame checks in IGT Chamelium tests and CRC Paul Kocialkowski
2017-06-15 17:37 ` Lyude Paul
2017-06-15 18:38   ` Jani Nikula
2017-06-16  8:32   ` Paul Kocialkowski
2017-06-16  4:08 ` chihchung
2017-06-16  9:28   ` Paul Kocialkowski
2017-06-16 10:09     ` Chih-Chung Chang
2017-06-16 10:21       ` Paul Kocialkowski

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.