Dhrystone -- userland version

Dhrystone -- userland version

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 419 bytes --]

Hi!

Distributions do not usually carry dhrystone, and I don't think anyone
really maintains it. It is useful tool, and it seems we'll maintain
it.

I'd like to add enough glue so that it would be runnable from
userspace, too? Userland version is what is actually useful to me, and
it should not be hard.

Best regards,
									Pavel
-- 
People of Russia, stop Putin before his war on Ukraine escalates.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Dhrystone -- userland version

[Theodore Ts'o]

On Sat, Jan 07, 2023 at 11:30:15PM +0100, Pavel Machek wrote:
> 
> Distributions do not usually carry dhrystone, and I don't think anyone
> really maintains it. It is useful tool, and it seems we'll maintain
> it.
> 
> I'd like to add enough glue so that it would be runnable from
> userspace, too? Userland version is what is actually useful to me, and
> it should not be hard.

I don't see whatever message you were replying to, and it doesn't seem
to be archived in lore[1], so I'm not sure about the context.  But you
are talking about the Dhrystone benchmark[2], right?

[1] https://lore.kernel.org/all/Y7nyd4hPeXsdiibH@duo.ucw.cz/T/#u
[2] https://wiki.cdot.senecacollege.ca/wiki/Dhrystone_howto

If so, I'm confused what you mean by "add enough glue so that it would
be runnable from userspace" --- Dhrystone is a userspace benchmark,
dating from the 1980's, although what it benchmarks is often more about
the compiler than the CPU's performace.

   "When first released, the Dhrystone benchmark fulfilled a useful
   function – at least it gave an alternative indicator to vendors’
   literal MIPS ratings. However, more than twenty years later, there
   are undoubtedly better benchmarks available for measuring processor
   performance."

						- Ted

Re: Dhrystone -- userland version

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1422 bytes --]

Hi!

> > Distributions do not usually carry dhrystone, and I don't think anyone
> > really maintains it. It is useful tool, and it seems we'll maintain
> > it.
> > 
> > I'd like to add enough glue so that it would be runnable from
> > userspace, too? Userland version is what is actually useful to me, and
> > it should not be hard.
> 
> I don't see whatever message you were replying to, and it doesn't seem
> to be archived in lore[1], so I'm not sure about the context.  But you
> are talking about the Dhrystone benchmark[2], right?
> 
> [1] https://lore.kernel.org/all/Y7nyd4hPeXsdiibH@duo.ucw.cz/T/#u
> [2] https://wiki.cdot.senecacollege.ca/wiki/Dhrystone_howto
> 
> If so, I'm confused what you mean by "add enough glue so that it would
> be runnable from userspace" --- Dhrystone is a userspace benchmark,
> dating from the 1980's, although what it benchmarks is often more about
> the compiler than the CPU's performace.

Yes, I'm talking about Dhrystone benchmark. We are carrying
kernel-only version in lib/dhry_*.c, it is in -next now.

commit cfbd4cc940275240e97f8b922c8f18a44fe15c07
Author: Geert Uytterhoeven <geert+renesas@glider.be>
Date:   Thu Dec 8 15:31:28 2022 +0100

    lib: add Dhrystone benchmark test

I'd like userspace-too version, at the same place :-).

Best regards,
								Pavel

-- 
People of Russia, stop Putin before his war on Ukraine escalates.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Dhrystone -- userland version

[Geert Uytterhoeven]

Hi Pavel,

On Sun, Jan 8, 2023 at 8:40 AM Pavel Machek <pavel@ucw.cz> wrote:
> > > Distributions do not usually carry dhrystone, and I don't think anyone
> > > really maintains it. It is useful tool, and it seems we'll maintain
> > > it.
> > >
> > > I'd like to add enough glue so that it would be runnable from
> > > userspace, too? Userland version is what is actually useful to me, and
> > > it should not be hard.
> >
> > I don't see whatever message you were replying to, and it doesn't seem
> > to be archived in lore[1], so I'm not sure about the context.  But you
> > are talking about the Dhrystone benchmark[2], right?
> >
> > [1] https://lore.kernel.org/all/Y7nyd4hPeXsdiibH@duo.ucw.cz/T/#u
> > [2] https://wiki.cdot.senecacollege.ca/wiki/Dhrystone_howto
> >
> > If so, I'm confused what you mean by "add enough glue so that it would
> > be runnable from userspace" --- Dhrystone is a userspace benchmark,
> > dating from the 1980's, although what it benchmarks is often more about
> > the compiler than the CPU's performace.
>
> Yes, I'm talking about Dhrystone benchmark. We are carrying
> kernel-only version in lib/dhry_*.c, it is in -next now.
>
> commit cfbd4cc940275240e97f8b922c8f18a44fe15c07
> Author: Geert Uytterhoeven <geert+renesas@glider.be>
> Date:   Thu Dec 8 15:31:28 2022 +0100
>
>     lib: add Dhrystone benchmark test
>
> I'd like userspace-too version, at the same place :-).

So you want to add some glue code to tools/testing/, for building a
userspace version?

However, as this is not Linux-specific, how hard can it be to convince
your distro to include https://github.com/qris/dhrystone-deb.git?
Usually, when I have a full userspace available, I just clone the above,
and debuild it myself.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Dhrystone -- userland version

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 2045 bytes --]

Hi!

> On Sun, Jan 8, 2023 at 8:40 AM Pavel Machek <pavel@ucw.cz> wrote:
> > > > Distributions do not usually carry dhrystone, and I don't think anyone
> > > > really maintains it. It is useful tool, and it seems we'll maintain
> > > > it.
> > > >
> > > > I'd like to add enough glue so that it would be runnable from
> > > > userspace, too? Userland version is what is actually useful to me, and
> > > > it should not be hard.
> > >
> > > I don't see whatever message you were replying to, and it doesn't seem
> > > to be archived in lore[1], so I'm not sure about the context.  But you
> > > are talking about the Dhrystone benchmark[2], right?
> > >
> > > [1] https://lore.kernel.org/all/Y7nyd4hPeXsdiibH@duo.ucw.cz/T/#u
> > > [2] https://wiki.cdot.senecacollege.ca/wiki/Dhrystone_howto
> > >
> > > If so, I'm confused what you mean by "add enough glue so that it would
> > > be runnable from userspace" --- Dhrystone is a userspace benchmark,
> > > dating from the 1980's, although what it benchmarks is often more about
> > > the compiler than the CPU's performace.
> >
> > Yes, I'm talking about Dhrystone benchmark. We are carrying
> > kernel-only version in lib/dhry_*.c, it is in -next now.
> >
> > commit cfbd4cc940275240e97f8b922c8f18a44fe15c07
> > Author: Geert Uytterhoeven <geert+renesas@glider.be>
> > Date:   Thu Dec 8 15:31:28 2022 +0100
> >
> >     lib: add Dhrystone benchmark test
> >
> > I'd like userspace-too version, at the same place :-).
> 
> So you want to add some glue code to tools/testing/, for building a
> userspace version?

Yes, that would be the plan.

> However, as this is not Linux-specific, how hard can it be to convince
> your distro to include https://github.com/qris/dhrystone-deb.git?
> Usually, when I have a full userspace available, I just clone the above,
> and debuild it myself.

Dunno. I'd not solve it if package was in Debian, but it is not.

Best regards,
								Pavel
-- 
People of Russia, stop Putin before his war on Ukraine escalates.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Dhrystone -- userland version

[Theodore Ts'o]

On Sun, Jan 08, 2023 at 07:47:07PM +0100, Pavel Machek wrote:
> > However, as this is not Linux-specific, how hard can it be to convince
> > your distro to include https://github.com/qris/dhrystone-deb.git?
> > Usually, when I have a full userspace available, I just clone the above,
> > and debuild it myself.
> 
> Dunno. I'd not solve it if package was in Debian, but it is not.

I would suspect the better long-term solution would be to get the
package into Debian, since that will be easier for people to use.  I
suspect the reason why most distros don't include it is because it
really is a **terrible** benchmark for most use cases.

(I'm not even convinced that using it to try to auto-tune the schedule
by including a random dhrystone number in a device tree is reallty all
that useful, but hey, I'm not a scheduler expert.)

So I took a quick look at dhrystone-deb.git, as well as the original
version of the Dhrystone source code at [1], and I do see huge red
flag show-stopper that would prevent it from getting it into Debian.

There are no copyright license notices anywhere in the source code,
and from what I can tell, it was originally published in ACM's SIGPLAN
Notices --- which even today is behind a paywall, so I presume it's
under an ACM copyright.

Which makes me wonder --- did someone get copyright clearance from ACM
and/or the original authors before this was published in a linux git
tree?  And if not, we should get this clarified before someone sends a
pull request to Linus with a potential copyright licensing problem.

     	     	      	     	       		 - Ted

[1] https://netlib.org/benchmark/dhry-c

P.S.  This is problem is not unique to Dhrystone.  This is also one of
the reasons why xfstests will probably never be packaged in Debian ---
one of the file system exerciser code has been contaminated by code
copyrighted by Avadis Tevanian while he was at NeXT, which means that
the copyright is now with Apple.  There is a much older version of
fsx.c that is fully in the public domain if I remember correctly from
the last time I looked at it.  But someone would have to do a
clean-room reimplementation of the enhancements made by Avie from the
public domain version, and then layer on all of the changes made by
SGI, and Linux developers since then.  The last time I asked about
whether SGI had managed to get copyright clearance for fsx as I recall
there was a lot of mumbling and everyone has decided to let sleeping
dogs lie.

However, there are people who use the Linux kernel who are much
touchier about copyright concerns, so if we can't build a Linux kernel
without including lib/dhrystone.c, and we don't have copyright
permission from ACM, I suspect some corporate lawyers would
be.... uneasy.

Re: Dhrystone -- userland version

[Geert Uytterhoeven]

Hi Ted,

On Mon, Jan 9, 2023 at 1:45 AM Theodore Ts'o <tytso@mit.edu> wrote:
> On Sun, Jan 08, 2023 at 07:47:07PM +0100, Pavel Machek wrote:
> > > However, as this is not Linux-specific, how hard can it be to convince
> > > your distro to include https://github.com/qris/dhrystone-deb.git?
> > > Usually, when I have a full userspace available, I just clone the above,
> > > and debuild it myself.
> >
> > Dunno. I'd not solve it if package was in Debian, but it is not.
>
> I would suspect the better long-term solution would be to get the
> package into Debian, since that will be easier for people to use.  I
> suspect the reason why most distros don't include it is because it
> really is a **terrible** benchmark for most use cases.

My use case is verifying the CPU core clock rate when working on
the clock driver and/or cpufreq.  I can easily measure e.g. SPI or
UART clock rates
externally, but not CPU core clock rates.

> (I'm not even convinced that using it to try to auto-tune the schedule
> by including a random dhrystone number in a device tree is reallty all
> that useful, but hey, I'm not a scheduler expert.)

I leave that to the scheduler people, too ;-)

> So I took a quick look at dhrystone-deb.git, as well as the original
> version of the Dhrystone source code at [1], and I do see huge red
> flag show-stopper that would prevent it from getting it into Debian.
>
> There are no copyright license notices anywhere in the source code,
> and from what I can tell, it was originally published in ACM's SIGPLAN
> Notices --- which even today is behind a paywall, so I presume it's
> under an ACM copyright.

Note that the version published by ACM was the original ADA version...

> Which makes me wonder --- did someone get copyright clearance from ACM
> and/or the original authors before this was published in a linux git
> tree?  And if not, we should get this clarified before someone sends a
> pull request to Linus with a potential copyright licensing problem.

I used "GPL-2.0-only OR BSD-2-Clause"[1], as the latter is what FreeBSD
uses, now augmented with calls into Linux-specific code.

Contacting the original author is no longer possible, unfortunately[3].
I don't know if Rick Richardson is still around.

> However, there are people who use the Linux kernel who are much
> touchier about copyright concerns, so if we can't build a Linux kernel
> without including lib/dhrystone.c, and we don't have copyright
> permission from ACM, I suspect some corporate lawyers would
> be.... uneasy.

Chromium hosts the Dhrystone benchmark as part of the LLVM testsuite,
under the LLVM license[4].  The Fuego Test System says MIT[5].

[1] https://lore.kernel.org/all/CAMuHMdXVkdmnKDobfZHkk+Lxradq238wAcaLOyW-cmS7J3MS4g@mail.gmail.com
[2] https://www.freshports.org/benchmarks/dhrystone/
[3] https://www.spec.org/spec/memories/reinhold/
[4] https://chromium.googlesource.com/native_client/pnacl-llvm-testsuite/+/c5be109f343430594ecec866dfa6d9833b921e1e/SingleSource/Benchmarks/Dhrystone/LICENSE.TXT
[5] http://fuegotest.org/server/Benchmark.Dhrystone

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Dhrystone -- userland version

[Willy Tarreau]

Hi Geert,

On Mon, Jan 09, 2023 at 10:28:09AM +0100, Geert Uytterhoeven wrote:
> Hi Ted,
> 
> On Mon, Jan 9, 2023 at 1:45 AM Theodore Ts'o <tytso@mit.edu> wrote:
> > On Sun, Jan 08, 2023 at 07:47:07PM +0100, Pavel Machek wrote:
> > > > However, as this is not Linux-specific, how hard can it be to convince
> > > > your distro to include https://github.com/qris/dhrystone-deb.git?
> > > > Usually, when I have a full userspace available, I just clone the above,
> > > > and debuild it myself.
> > >
> > > Dunno. I'd not solve it if package was in Debian, but it is not.
> >
> > I would suspect the better long-term solution would be to get the
> > package into Debian, since that will be easier for people to use.  I
> > suspect the reason why most distros don't include it is because it
> > really is a **terrible** benchmark for most use cases.
> 
> My use case is verifying the CPU core clock rate when working on
> the clock driver and/or cpufreq.  I can easily measure e.g. SPI or
> UART clock rates
> externally, but not CPU core clock rates.

Then maybe you'd rather use this:

   https://github.com/wtarreau/mhz

E.g.:

  $ ./mhz 10
  count=1666675 us50=20089 us250=100748 diff=80659 cpu_MHz=4132.645 tsc50=42427696 tsc250=212780620 diff=102 rdtsc_MHz=2112.014
  count=1666675 us50=20271 us250=101282 diff=81011 cpu_MHz=4114.688 tsc50=42813060 tsc250=213905908 diff=102 rdtsc_MHz=2111.970
  count=1666675 us50=20405 us250=100344 diff=79939 cpu_MHz=4169.867 tsc50=43094116 tsc250=211927426 diff=101 rdtsc_MHz=2112.027
  count=1666675 us50=20239 us250=100249 diff=80010 cpu_MHz=4166.167 tsc50=42744302 tsc250=211727114 diff=101 rdtsc_MHz=2112.021
  count=1666675 us50=20182 us250=100482 diff=80300 cpu_MHz=4151.121 tsc50=42624768 tsc250=212217884 diff=101 rdtsc_MHz=2111.994
  count=1666675 us50=20153 us250=100151 diff=79998 cpu_MHz=4166.792 tsc50=42562818 tsc250=211519326 diff=101 rdtsc_MHz=2112.009
  count=1666675 us50=20281 us250=99803 diff=79522 cpu_MHz=4191.733 tsc50=42832298 tsc250=210784252 diff=100 rdtsc_MHz=2112.019
  count=1666675 us50=20104 us250=99841 diff=79737 cpu_MHz=4180.431 tsc50=42461284 tsc250=210862414 diff=101 rdtsc_MHz=2111.957
  count=1666675 us50=20088 us250=100287 diff=80199 cpu_MHz=4156.349 tsc50=42423974 tsc250=211807066 diff=101 rdtsc_MHz=2112.035
  count=1666675 us50=20178 us250=99913 diff=79735 cpu_MHz=4180.536 tsc50=42614718 tsc250=211016124 diff=101 rdtsc_MHz=2112.014

It's also what's used by sbc-bench to verify that CPU vendors are not
cheating in the embedded world. It's very simple, and not arch-dependent
nor compiler-dependent (at least as long as you don't build at -O0 :-)).

Just my two cents,
Willy

Re: Dhrystone -- userland version

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1237 bytes --]

Hi!

> > > However, as this is not Linux-specific, how hard can it be to convince
> > > your distro to include https://github.com/qris/dhrystone-deb.git?
> > > Usually, when I have a full userspace available, I just clone the above,
> > > and debuild it myself.
> > 
> > Dunno. I'd not solve it if package was in Debian, but it is not.
> 
> I would suspect the better long-term solution would be to get the
> package into Debian, since that will be easier for people to use.  I
> suspect the reason why most distros don't include it is because it
> really is a **terrible** benchmark for most use cases.

Yeah, so... every benchmark is terrible. Is this one worse than most?

I tried benchmarking using:
time factor $[65863223*65863159]
     - but that's division test, not a CPU test, and there are two
     versions of factor floating around
time cat /dev/urandom | head -c 10000000 |  bzip2 -9 - | wc -c
     - but that seems to be memory test, not a CPU test.

Dhrystone seems to be making some kind of comeback in RISC-V land. Is
there something better? Something better that is still reasonably small?

Best regards,
								Pavel
-- 
People of Russia, stop Putin before his war on Ukraine escalates.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Dhrystone -- userland version

[Geert Uytterhoeven]

Hi Willy,

On Mon, Jan 9, 2023 at 3:31 PM Willy Tarreau <w@1wt.eu> wrote:
> On Mon, Jan 09, 2023 at 10:28:09AM +0100, Geert Uytterhoeven wrote:
> > On Mon, Jan 9, 2023 at 1:45 AM Theodore Ts'o <tytso@mit.edu> wrote:
> > > On Sun, Jan 08, 2023 at 07:47:07PM +0100, Pavel Machek wrote:
> > > > > However, as this is not Linux-specific, how hard can it be to convince
> > > > > your distro to include https://github.com/qris/dhrystone-deb.git?
> > > > > Usually, when I have a full userspace available, I just clone the above,
> > > > > and debuild it myself.
> > > >
> > > > Dunno. I'd not solve it if package was in Debian, but it is not.
> > >
> > > I would suspect the better long-term solution would be to get the
> > > package into Debian, since that will be easier for people to use.  I
> > > suspect the reason why most distros don't include it is because it
> > > really is a **terrible** benchmark for most use cases.
> >
> > My use case is verifying the CPU core clock rate when working on
> > the clock driver and/or cpufreq.  I can easily measure e.g. SPI or
> > UART clock rates
> > externally, but not CPU core clock rates.
>
> Then maybe you'd rather use this:
>
>    https://github.com/wtarreau/mhz

Thanks!

> E.g.:
>
>   $ ./mhz 10
>   count=1666675 us50=20089 us250=100748 diff=80659 cpu_MHz=4132.645 tsc50=42427696 tsc250=212780620 diff=102 rdtsc_MHz=2112.014
>   count=1666675 us50=20271 us250=101282 diff=81011 cpu_MHz=4114.688 tsc50=42813060 tsc250=213905908 diff=102 rdtsc_MHz=2111.970
>   count=1666675 us50=20405 us250=100344 diff=79939 cpu_MHz=4169.867 tsc50=43094116 tsc250=211927426 diff=101 rdtsc_MHz=2112.027
>   count=1666675 us50=20239 us250=100249 diff=80010 cpu_MHz=4166.167 tsc50=42744302 tsc250=211727114 diff=101 rdtsc_MHz=2112.021
>   count=1666675 us50=20182 us250=100482 diff=80300 cpu_MHz=4151.121 tsc50=42624768 tsc250=212217884 diff=101 rdtsc_MHz=2111.994
>   count=1666675 us50=20153 us250=100151 diff=79998 cpu_MHz=4166.792 tsc50=42562818 tsc250=211519326 diff=101 rdtsc_MHz=2112.009
>   count=1666675 us50=20281 us250=99803 diff=79522 cpu_MHz=4191.733 tsc50=42832298 tsc250=210784252 diff=100 rdtsc_MHz=2112.019
>   count=1666675 us50=20104 us250=99841 diff=79737 cpu_MHz=4180.431 tsc50=42461284 tsc250=210862414 diff=101 rdtsc_MHz=2111.957
>   count=1666675 us50=20088 us250=100287 diff=80199 cpu_MHz=4156.349 tsc50=42423974 tsc250=211807066 diff=101 rdtsc_MHz=2112.035
>   count=1666675 us50=20178 us250=99913 diff=79735 cpu_MHz=4180.536 tsc50=42614718 tsc250=211016124 diff=101 rdtsc_MHz=2112.014
>
> It's also what's used by sbc-bench to verify that CPU vendors are not
> cheating in the embedded world. It's very simple, and not arch-dependent
> nor compiler-dependent (at least as long as you don't build at -O0 :-)).

Seeing the rdtsc stuff, I was a bit skeptical, but it seems to work
fine on arm32 and rv64.

Unfortunately you forgot to add a LICENSE file ;-)

Alternatively, I can use the C version of BogoMIPS.  Which has its
own merits and reputation.

The nice thing about Dhrystones is that people still publish numbers
for comparison, also DMIPS/MHz and DMIPS/mW.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Dhrystone -- userland version

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1107 bytes --]

Hi!

> >   count=1666675 us50=20178 us250=99913 diff=79735 cpu_MHz=4180.536 tsc50=42614718 tsc250=211016124 diff=101 rdtsc_MHz=2112.014
> >
> > It's also what's used by sbc-bench to verify that CPU vendors are not
> > cheating in the embedded world. It's very simple, and not arch-dependent
> > nor compiler-dependent (at least as long as you don't build at -O0 :-)).
> 
> Seeing the rdtsc stuff, I was a bit skeptical, but it seems to work
> fine on arm32 and rv64.
> 
> Unfortunately you forgot to add a LICENSE file ;-)
> 
> Alternatively, I can use the C version of BogoMIPS.  Which has its
> own merits and reputation.

I'd expect BogoMIPS to be _much_ worse than Dhrystone. If all the
cores are same, it should be good enough, but...

> The nice thing about Dhrystones is that people still publish numbers
> for comparison, also DMIPS/MHz and DMIPS/mW.

...and there seems to be no replacement for that. Dhrystones seem to
become the standard in RISC-V community at least.

Best regards,
								Pavel
-- 
People of Russia, stop Putin before his war on Ukraine escalates.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Dhrystone -- userland version

[Geert Uytterhoeven]

Hi Willy,

On Mon, Jan 9, 2023 at 3:31 PM Willy Tarreau <w@1wt.eu> wrote:
> Then maybe you'd rather use this:
>
>    https://github.com/wtarreau/mhz

> It's also what's used by sbc-bench to verify that CPU vendors are not
> cheating in the embedded world. It's very simple, and not arch-dependent
> nor compiler-dependent (at least as long as you don't build at -O0 :-)).

Gcc 4.1.2 (still the userspace compiler on a few of my old systems)
doesn't like the aligned() attribute on functions.
In glibc 2.3, LLONG_MAX is defined conditionally on __USE_ISOC99.

Apart from that, the result is fairly accurate. Nice job!

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Dhrystone -- userland version

[Geert Uytterhoeven]

Hi Pavel,

On Mon, Jan 9, 2023 at 8:51 PM Pavel Machek <pavel@ucw.cz> wrote:
> > >   count=1666675 us50=20178 us250=99913 diff=79735 cpu_MHz=4180.536 tsc50=42614718 tsc250=211016124 diff=101 rdtsc_MHz=2112.014
> > >
> > > It's also what's used by sbc-bench to verify that CPU vendors are not
> > > cheating in the embedded world. It's very simple, and not arch-dependent
> > > nor compiler-dependent (at least as long as you don't build at -O0 :-)).
> >
> > Seeing the rdtsc stuff, I was a bit skeptical, but it seems to work
> > fine on arm32 and rv64.
> >
> > Unfortunately you forgot to add a LICENSE file ;-)
> >
> > Alternatively, I can use the C version of BogoMIPS.  Which has its
> > own merits and reputation.
>
> I'd expect BogoMIPS to be _much_ worse than Dhrystone. If all the
> cores are same, it should be good enough, but...

Both give different results when run on little vs. big cores.
That's why my in-kernel version prints the CPU number, and supports
running multiple benchmarks in parallel.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Dhrystone -- userland version

[Willy Tarreau]

Hi Geert,

On Mon, Jan 09, 2023 at 08:36:39PM +0100, Geert Uytterhoeven wrote:
> Seeing the rdtsc stuff, I was a bit skeptical, but it seems to work
> fine on arm32 and rv64.

Yes, I'm using it on arm32/64, mips, i386/x86_64, ppc64. It always
works fine. The rdtsc stuff you saw is not there to perform the
measurements, it's the opposite, the tool is measuring the TSC's
frequency when there is one :-)

> Unfortunately you forgot to add a LICENSE file ;-)

I really don't care. It can be public domain or MIT (since in many
places you can't drop your own rights and put something into public
domain). If you need anything from it, feel free to steal code from
it, I'm fine with this.

> Alternatively, I can use the C version of BogoMIPS.  Which has its
> own merits and reputation.

Yes, it's just that you have to apply your own ratios depending on
the build options and arch if you want to find the frequency.

> The nice thing about Dhrystones is that people still publish numbers
> for comparison, also DMIPS/MHz and DMIPS/mW.

Yes that's true, but the numbers can differ by orders of magnitude on
the same hardware depending on build options, it's hard to compare
anything using it these days.

> Gcc 4.1.2 (still the userspace compiler on a few of my old systems)
> doesn't like the aligned() attribute on functions.

Good to know, thank you. We could enclose it between ifdefs if needed.

> In glibc 2.3, LLONG_MAX is defined conditionally on __USE_ISOC99.

Yes or the usual (~0ULL)/2 should work and be equally portable.

> Apart from that, the result is fairly accurate. Nice job!

Actually it's not the tool that's accurate, it's gettimeofday() ;-)

Cheers,
Willy

Re: Dhrystone -- userland version

[Jan Engelhardt]

On Monday 2023-01-09 01:45, Theodore Ts'o wrote:
>On Sun, Jan 08, 2023 at 07:47:07PM +0100, Pavel Machek wrote:
>> > However, as this is not Linux-specific, how hard can it be to convince
>> > your distro to include https://github.com/qris/dhrystone-deb.git?
>> > Usually, when I have a full userspace available, I just clone the above,
>> > and debuild it myself.
>> 
>> Dunno. I'd not solve it if package was in Debian, but it is not.
>
>I would suspect the better long-term solution would be to get the
>package into Debian, since that will be easier for people to use.  I
>suspect the reason why most distros don't include it is because it
>really is a **terrible** benchmark for most use cases.

(All of) you could install boinc-client and just exercise
--run_cpu_benchmarks without (I think) attaching to any particular
computing project. The package should be in distros and you get
both Dhrystone and Whetstone.

Re: Dhrystone -- userland version

[Geert Uytterhoeven]

Hi Jan,

On Wed, Jan 11, 2023 at 2:32 PM Jan Engelhardt <jengelh@inai.de> wrote:
> On Monday 2023-01-09 01:45, Theodore Ts'o wrote:
> >On Sun, Jan 08, 2023 at 07:47:07PM +0100, Pavel Machek wrote:
> >> > However, as this is not Linux-specific, how hard can it be to convince
> >> > your distro to include https://github.com/qris/dhrystone-deb.git?
> >> > Usually, when I have a full userspace available, I just clone the above,
> >> > and debuild it myself.
> >>
> >> Dunno. I'd not solve it if package was in Debian, but it is not.
> >
> >I would suspect the better long-term solution would be to get the
> >package into Debian, since that will be easier for people to use.  I
> >suspect the reason why most distros don't include it is because it
> >really is a **terrible** benchmark for most use cases.
>
> (All of) you could install boinc-client and just exercise
> --run_cpu_benchmarks without (I think) attaching to any particular
> computing project. The package should be in distros and you get
> both Dhrystone and Whetstone.

Thanks, but I really want an in-kernel benchmark, as a full userland
may not be available during early development.

BTW, boinc doesn't seem to work under Wayland, spewing lots of
"Invalid MIT-MAGIC-COOKIE-1 key", before printing Dhrystone and
Whetstone numbers...

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Dhrystone -- userland version

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1158 bytes --]

Hi!

> >> > However, as this is not Linux-specific, how hard can it be to convince
> >> > your distro to include https://github.com/qris/dhrystone-deb.git?
> >> > Usually, when I have a full userspace available, I just clone the above,
> >> > and debuild it myself.
> >> 
> >> Dunno. I'd not solve it if package was in Debian, but it is not.
> >
> >I would suspect the better long-term solution would be to get the
> >package into Debian, since that will be easier for people to use.  I
> >suspect the reason why most distros don't include it is because it
> >really is a **terrible** benchmark for most use cases.
> 
> (All of) you could install boinc-client and just exercise
> --run_cpu_benchmarks without (I think) attaching to any particular
> computing project. The package should be in distros and you get
> both Dhrystone and Whetstone.

Hehe, nice trick, but it does not work for me:

boinc --run_cpu_benchmarks

does not actually seem to run the benchmarks. Anyway, it is 3MB
project for few kilobytes of benchmarks...

Best regards,
								Pavel
-- 
People of Russia, stop Putin before his war on Ukraine escalates.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Dhrystone -- userland version

[Jan Engelhardt]

On Wednesday 2023-01-11 16:30, Pavel Machek wrote:
>
>> (All of) you could install boinc-client and just exercise
>> --run_cpu_benchmarks without (I think) attaching to any particular
>> computing project. The package should be in distros and you get
>> both Dhrystone and Whetstone.
>
>Hehe, nice trick, but it does not work for me:
>
>boinc --run_cpu_benchmarks
>
>does not actually seem to run the benchmarks.

Maybe I misread the manpage.

# runuser -u boinc -- /bin/bash --login
$ cd ~
$ boinccmd --get_messages
11: 11-Jan-2023 17:23:50 (low) [] Processor: 16 AuthenticAMD AMD Ryzen 7 3700X 8-Core Processor [Family 23 Model 113 Stepping 0]
12: 11-Jan-2023 17:23:50 (low) [] Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sev sev_es
13: 11-Jan-2023 17:23:50 (low) [] OS: Linux openSUSE: openSUSE Tumbleweed [6.1.3|libc 2.36]
...
$ boinccmd --run_benchmarks
...
$ boinccmd --get_messages
30: 11-Jan-2023 17:24:31 (low) [] Running CPU benchmarks
31: 11-Jan-2023 17:24:31 (low) [] Suspending computation - CPU benchmarks in progress
32: 11-Jan-2023 17:25:02 (low) [] Benchmark results:
33: 11-Jan-2023 17:25:02 (low) [] Number of CPUs: 16
34: 11-Jan-2023 17:25:02 (low) [] 7136 floating point MIPS (Whetstone) per CPU
35: 11-Jan-2023 17:25:02 (low) [] 124780 integer MIPS (Dhrystone) per CPU
36: 11-Jan-2023 17:25:03 (low) [] Resuming computation

(This is using the openSUSE boinc-client package. It's modeled as a
daemon/service, so runs without X11 or Wayland, since Geert was wondering.)

>Thanks, but I really want an in-kernel benchmark

Can't have everything :)