All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH v2 1/2] package/coremark: new package
@ 2020-11-11 22:41 Chris Packham
  2020-11-11 22:41 ` [Buildroot] [PATCH v2 2/2] package/coremark-pro: " Chris Packham
                   ` (2 more replies)
  0 siblings, 3 replies; 7+ messages in thread
From: Chris Packham @ 2020-11-11 22:41 UTC (permalink / raw)
  To: buildroot

CoreMark is a simple, yet sophisticated benchmark that is designed
specifically to test the functionality of a processor core. Running
CoreMark produces a single-number score allowing users to make quick
comparisons between processors.

https://www.eembc.org/coremark/

Signed-off-by: Chris Packham <judge.packham@gmail.com>
---
Changes in v2:
- None

 DEVELOPERS                     |  1 +
 package/Config.in              |  1 +
 package/coremark/Config.in     |  8 ++++++++
 package/coremark/coremark.hash |  3 +++
 package/coremark/coremark.mk   | 21 +++++++++++++++++++++
 5 files changed, 34 insertions(+)
 create mode 100644 package/coremark/Config.in
 create mode 100644 package/coremark/coremark.hash
 create mode 100644 package/coremark/coremark.mk

diff --git a/DEVELOPERS b/DEVELOPERS
index 43cc1b55ae..ff823a2e3d 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -490,6 +490,7 @@ F:	configs/olimex_a13_olinuxino_defconfig
 F:	configs/orangepi_plus_defconfig
 
 N:	Chris Packham <judge.packham@gmail.com>
+F:	package/coremark/
 F:	package/gstreamer1/gst1-shark/
 F:	package/micropython/
 F:	package/micropython-lib/
diff --git a/package/Config.in b/package/Config.in
index 016a99ed1a..16fd95fe38 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -87,6 +87,7 @@ menu "Debugging, profiling and benchmark"
 	source "package/bonnie/Config.in"
 	source "package/cache-calibrator/Config.in"
 	source "package/clinfo/Config.in"
+	source "package/coremark/Config.in"
 	source "package/dacapo/Config.in"
 	source "package/dhrystone/Config.in"
 	source "package/dieharder/Config.in"
diff --git a/package/coremark/Config.in b/package/coremark/Config.in
new file mode 100644
index 0000000000..f0e13eda63
--- /dev/null
+++ b/package/coremark/Config.in
@@ -0,0 +1,8 @@
+config BR2_PACKAGE_COREMARK
+	bool "coremark"
+	help
+	  CoreMark is a benchmark that measures the performance of
+	  microcontrollers (MCUs) and central processing units (CPUs)
+	  used in embedded systems
+
+	  https://www.eembc.org/coremark/
diff --git a/package/coremark/coremark.hash b/package/coremark/coremark.hash
new file mode 100644
index 0000000000..57da3e998c
--- /dev/null
+++ b/package/coremark/coremark.hash
@@ -0,0 +1,3 @@
+#locally computed
+sha256  99c5a6d63af85a281b4e4d6ccb522c446653c435dfec9455ad73ef9e71f28bde  coremark-1.01.tar.gz
+sha256  d72094a3cb629dc7712ae09b7b777416903c6670930928b5530f2ddfb2296888  LICENSE.md
diff --git a/package/coremark/coremark.mk b/package/coremark/coremark.mk
new file mode 100644
index 0000000000..418fd5c8d3
--- /dev/null
+++ b/package/coremark/coremark.mk
@@ -0,0 +1,21 @@
+################################################################################
+#
+# CoreMark
+#
+################################################################################
+
+COREMARK_VERSION = 1.01
+COREMARK_SITE = $(call github,eembc,coremark,v$(COREMARK_VERSION))
+COREMARK_LICENSE = Apache-2.0
+COREMARK_LICENSE_FILES = LICENSE.md
+
+define COREMARK_BUILD_CMDS
+	$(TARGET_MAKE_ENV) $(MAKE) CC="$(TARGET_CC)" -C $(@D) \
+		PORT_DIR=linux$(if $(BR2_ARCH_IS_64),64) EXE= link
+endef
+
+define COREMARK_INSTALL_TARGET_CMDS
+	$(INSTALL) -D $(@D)/coremark $(TARGET_DIR)/usr/bin/coremark
+endef
+
+$(eval $(generic-package))
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 7+ messages in thread
* [Buildroot] [PATCH v2 2/2] package/coremark-pro: new package
  2020-11-11 22:41 [Buildroot] [PATCH v2 1/2] package/coremark: new package Chris Packham
@ 2020-11-11 22:41 ` Chris Packham
  2020-11-12 13:17   ` Matthew Weber
  2021-01-07 22:00   ` Arnout Vandecappelle
  2020-11-12 13:15 ` [Buildroot] [PATCH v2 1/2] package/coremark: " Matthew Weber
  2021-01-07 21:48 ` Arnout Vandecappelle
  2 siblings, 2 replies; 7+ messages in thread
From: Chris Packham @ 2020-11-11 22:41 UTC (permalink / raw)
  To: buildroot

CoreMark-Pro is a comprehensive, advanced processor benchmark that
works with and enhances the market-proven industry-standard EEMBC
CoreMark benchmark.

https://www.eembc.org/coremark-pro/

Signed-off-by: Chris Packham <judge.packham@gmail.com>
---
Changes in v2:
- Add depends on BR2_USE_MMU (thanks Matt)

 DEVELOPERS                             |  1 +
 package/Config.in                      |  1 +
 package/coremark-pro/Config.in         |  8 +++++
 package/coremark-pro/coremark-pro.hash |  3 ++
 package/coremark-pro/coremark-pro.mk   | 42 ++++++++++++++++++++++++++
 package/coremark-pro/coremark-pro.sh   | 36 ++++++++++++++++++++++
 6 files changed, 91 insertions(+)
 create mode 100644 package/coremark-pro/Config.in
 create mode 100644 package/coremark-pro/coremark-pro.hash
 create mode 100644 package/coremark-pro/coremark-pro.mk
 create mode 100755 package/coremark-pro/coremark-pro.sh

diff --git a/DEVELOPERS b/DEVELOPERS
index ff823a2e3d..291dd41286 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -491,6 +491,7 @@ F:	configs/orangepi_plus_defconfig
 
 N:	Chris Packham <judge.packham@gmail.com>
 F:	package/coremark/
+F:	package/coremark-pro/
 F:	package/gstreamer1/gst1-shark/
 F:	package/micropython/
 F:	package/micropython-lib/
diff --git a/package/Config.in b/package/Config.in
index 16fd95fe38..0011ccd684 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -88,6 +88,7 @@ menu "Debugging, profiling and benchmark"
 	source "package/cache-calibrator/Config.in"
 	source "package/clinfo/Config.in"
 	source "package/coremark/Config.in"
+	source "package/coremark-pro/Config.in"
 	source "package/dacapo/Config.in"
 	source "package/dhrystone/Config.in"
 	source "package/dieharder/Config.in"
diff --git a/package/coremark-pro/Config.in b/package/coremark-pro/Config.in
new file mode 100644
index 0000000000..dd6821056c
--- /dev/null
+++ b/package/coremark-pro/Config.in
@@ -0,0 +1,8 @@
+config BR2_PACKAGE_COREMARK_PRO
+	bool "coremark-pro"
+	depends on BR2_USE_MMU # perl uses fork()
+	select BR2_PACKAGE_PERL
+	help
+	  CoreMark-PRO is a comprehensive, advanced processor benchmark.
+
+	  https://www.eembc.org/coremark-pro/
diff --git a/package/coremark-pro/coremark-pro.hash b/package/coremark-pro/coremark-pro.hash
new file mode 100644
index 0000000000..16b121edb3
--- /dev/null
+++ b/package/coremark-pro/coremark-pro.hash
@@ -0,0 +1,3 @@
+#locally computed
+sha256  440223554134832efec4ad220795611769ded3869fbc3ce3c59b05b7ddeb9f6d  coremark-pro-1.1.2743.tar.gz
+sha256  0d7775bd793ea71fef3594c18164a08ed2f5f66c898958d0644f7a236a536f2b  LICENSE.md
diff --git a/package/coremark-pro/coremark-pro.mk b/package/coremark-pro/coremark-pro.mk
new file mode 100644
index 0000000000..d3e99fcdfd
--- /dev/null
+++ b/package/coremark-pro/coremark-pro.mk
@@ -0,0 +1,42 @@
+################################################################################
+#
+# CoreMark-PRO
+#
+################################################################################
+
+COREMARK_PRO_VERSION = 1.1.2743
+COREMARK_PRO_SITE = $(call github,eembc,coremark-pro,v$(COREMARK_PRO_VERSION))
+COREMARK_PRO_LICENSE = Apache-2.0
+COREMARK_PRO_LICENSE_FILES = LICENSE.md
+COREMARK_PRO_DEPENDENCIES = perl
+
+ifeq ($(BR2_ENDIAN),"BIG")
+COREMARK_PRO_MAKE_OPTS += PLATFORM_DEFINES='EE_BIG_ENDIAN=1 EE_LITTLE_ENDIAN=0'
+else
+COREMARK_PRO_MAKE_OPTS += PLATFORM_DEFINES='EE_BIG_ENDIAN=0 EE_LITTLE_ENDIAN=1'
+endif
+
+COREMARK_PRO_MAKE_OPTS += \
+	TARGET=linux$(if $(BR2_ARCH_IS_64),64) \
+	EXE=
+
+define COREMARK_PRO_BUILD_CMDS
+	$(TARGET_MAKE_ENV) $(MAKE) CC="$(TARGET_CC)" LD="$(TARGET_CC)" -C $(@D) \
+		$(COREMARK_PRO_MAKE_OPTS) build
+endef
+
+COREMARK_PRO_MARKS = cjpeg-rose7-preset core linear_alg-mid-100x100-sp loops-all-mid-10k-sp nnet_test parser-125k radix2-big-64k sha-test zip-test
+COREMARK_PRO_SCRIPTS = results_parser.pl cert_median.pl cert_mark.pl headings.txt
+
+define COREMARK_PRO_INSTALL_TARGET_CMDS
+	mkdir -p  $(TARGET_DIR)/usr/share/coremark-pro/logs
+	$(foreach m,$(COREMARK_PRO_MARKS),\
+		$(INSTALL) -D $(@D)/builds/linux$(if $(BR2_ARCH_IS_64),64)/gcc$(if $(BR2_ARCH_IS_64),64)/bin/$(m) $(TARGET_DIR)/usr/bin/$(m)$(sep) \
+		size $(TARGET_DIR)/usr/bin/$(m) > $(TARGET_DIR)/usr/share/coremark-pro/logs/$(m).size.log$(sep))
+	$(INSTALL) -D $(@D)/builds/linux$(if $(BR2_ARCH_IS_64),64)/gcc$(if $(BR2_ARCH_IS_64),64)/data/libbmp/Rose256.bmp $(TARGET_DIR)/usr/share/coremark-pro/Rose256.bmp
+	$(foreach s,$(COREMARK_PRO_SCRIPTS),\
+		$(INSTALL) -D $(@D)/util/perl/$(s) $(TARGET_DIR)/usr/share/coremark-pro/util/perl/$(s)$(sep))
+	$(Q)cp package/coremark-pro/coremark-pro.sh $(TARGET_DIR)/usr/bin/
+endef
+
+$(eval $(generic-package))
diff --git a/package/coremark-pro/coremark-pro.sh b/package/coremark-pro/coremark-pro.sh
new file mode 100755
index 0000000000..65ad9df645
--- /dev/null
+++ b/package/coremark-pro/coremark-pro.sh
@@ -0,0 +1,36 @@
+#!/bin/sh
+# SPDX-License-Identifier: Apache-2.0
+
+workloads="cjpeg-rose7-preset core linear_alg-mid-100x100-sp \
+    loops-all-mid-10k-sp nnet_test parser-125k radix2-big-64k sha-test zip-test"
+
+rm -rf /tmp/coremark-pro
+mkdir -p /tmp/coremark-pro
+cp /usr/share/coremark-pro/Rose256.bmp /tmp/coremark-pro
+cp /usr/share/coremark-pro/logs/*.size.log /tmp/coremark-pro
+
+cd /tmp/coremark-pro
+
+for contype in single best; do
+    cat /usr/share/coremark-pro/util/perl/headings.txt >>result.log
+    for wld in $workloads; do
+        [ "$contype" = "single" ] && XCMD="-c1 -w1"
+        [ "$contype" = "best" ] && XCMD="-c$(nproc) -w$(nproc)"
+        echo "#Results for verification run started at $(date +%y%j:%H:%M:%S) XCMD=$XCMD" >$wld-$contype-result.log
+        echo "Verification run for $wld"
+        $wld $XCMD -v1 > $wld.run.log
+        LC_ALL=C perl /usr/share/coremark-pro/util/perl/results_parser.pl $wld-$contype-result.log $wld.run.log
+        echo "#Results for performance runs started at $(date +%y%j:%H:%M:%S) XCMD=$XCMD" >>$wld-$contype-result.log
+        echo "Performance run for $wld"
+        for i in 1 2 3; do
+            $wld $XCMD -v0 > $wld.run.log
+            LC_ALL=C perl /usr/share/coremark-pro/util/perl/results_parser.pl $wld-$contype-result.log $wld.run.log
+        done
+        echo "#Median for final result $wld" >>$wld-$contype-result.log
+        LC_ALL=C perl /usr/share/coremark-pro/util/perl/cert_median.pl $wld-$contype-result.log $contype >>$wld-$contype-result.log
+        cat $wld-$contype-result.log >>result.log
+    done
+done
+LC_ALL=C perl /usr/share/coremark-pro/util/perl/cert_mark.pl -i result.log -s coremarkpro > result.mark
+cat result.mark
+cd - >/dev/null
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 7+ messages in thread
* [Buildroot] [PATCH v2 1/2] package/coremark: new package
  2020-11-11 22:41 [Buildroot] [PATCH v2 1/2] package/coremark: new package Chris Packham
  2020-11-11 22:41 ` [Buildroot] [PATCH v2 2/2] package/coremark-pro: " Chris Packham
@ 2020-11-12 13:15 ` Matthew Weber
  2021-01-07 21:48 ` Arnout Vandecappelle
  2 siblings, 0 replies; 7+ messages in thread
From: Matthew Weber @ 2020-11-12 13:15 UTC (permalink / raw)
  To: buildroot

Chris,

On Wed, Nov 11, 2020 at 4:44 PM Chris Packham <judge.packham@gmail.com> wrote:
>
> CoreMark is a simple, yet sophisticated benchmark that is designed
> specifically to test the functionality of a processor core. Running
> CoreMark produces a single-number score allowing users to make quick
> comparisons between processors.
>
> https://www.eembc.org/coremark/
>
> Signed-off-by: Chris Packham <judge.packham@gmail.com>

Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>

^ permalink raw reply	[flat|nested] 7+ messages in thread
* [Buildroot] [PATCH v2 2/2] package/coremark-pro: new package
  2020-11-11 22:41 ` [Buildroot] [PATCH v2 2/2] package/coremark-pro: " Chris Packham
@ 2020-11-12 13:17   ` Matthew Weber
  2021-01-07 22:00   ` Arnout Vandecappelle
  1 sibling, 0 replies; 7+ messages in thread
From: Matthew Weber @ 2020-11-12 13:17 UTC (permalink / raw)
  To: buildroot

Chris,

On Wed, Nov 11, 2020 at 4:45 PM Chris Packham <judge.packham@gmail.com> wrote:
>
> CoreMark-Pro is a comprehensive, advanced processor benchmark that
> works with and enhances the market-proven industry-standard EEMBC
> CoreMark benchmark.
>
> https://www.eembc.org/coremark-pro/
>
> Signed-off-by: Chris Packham <judge.packham@gmail.com>

Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>

^ permalink raw reply	[flat|nested] 7+ messages in thread
* [Buildroot] [PATCH v2 1/2] package/coremark: new package
  2020-11-11 22:41 [Buildroot] [PATCH v2 1/2] package/coremark: new package Chris Packham
  2020-11-11 22:41 ` [Buildroot] [PATCH v2 2/2] package/coremark-pro: " Chris Packham
  2020-11-12 13:15 ` [Buildroot] [PATCH v2 1/2] package/coremark: " Matthew Weber
@ 2021-01-07 21:48 ` Arnout Vandecappelle
  2 siblings, 0 replies; 7+ messages in thread
From: Arnout Vandecappelle @ 2021-01-07 21:48 UTC (permalink / raw)
  To: buildroot

 Hi Chris,

On 11/11/2020 23:41, Chris Packham wrote:
> CoreMark is a simple, yet sophisticated benchmark that is designed
> specifically to test the functionality of a processor core. Running
> CoreMark produces a single-number score allowing users to make quick
> comparisons between processors.
> 
> https://www.eembc.org/coremark/
> 
> Signed-off-by: Chris Packham <judge.packham@gmail.com>

 Matt tested the first iteration of this patch, so you should keep his Tested-by
tag (otherwise it gets lost...).

 With that, applied to master, thanks.

 Regards,
 Arnout

> ---
> Changes in v2:
> - None
[snip]

^ permalink raw reply	[flat|nested] 7+ messages in thread
* [Buildroot] [PATCH v2 2/2] package/coremark-pro: new package
  2020-11-11 22:41 ` [Buildroot] [PATCH v2 2/2] package/coremark-pro: " Chris Packham
  2020-11-12 13:17   ` Matthew Weber
@ 2021-01-07 22:00   ` Arnout Vandecappelle
  2021-01-08  8:04     ` Chris Packham
  1 sibling, 1 reply; 7+ messages in thread
From: Arnout Vandecappelle @ 2021-01-07 22:00 UTC (permalink / raw)
  To: buildroot



On 11/11/2020 23:41, Chris Packham wrote:
> CoreMark-Pro is a comprehensive, advanced processor benchmark that
> works with and enhances the market-proven industry-standard EEMBC
> CoreMark benchmark.
> 
> https://www.eembc.org/coremark-pro/
> 
> Signed-off-by: Chris Packham <judge.packham@gmail.com>
> ---
> Changes in v2:
> - Add depends on BR2_USE_MMU (thanks Matt)
> 
[snip]
> diff --git a/package/coremark-pro/coremark-pro.hash b/package/coremark-pro/coremark-pro.hash
> new file mode 100644
> index 0000000000..16b121edb3
> --- /dev/null
> +++ b/package/coremark-pro/coremark-pro.hash
> @@ -0,0 +1,3 @@
> +#locally computed

 (nitpick)

# Locally computed

> +sha256  440223554134832efec4ad220795611769ded3869fbc3ce3c59b05b7ddeb9f6d  coremark-pro-1.1.2743.tar.gz
> +sha256  0d7775bd793ea71fef3594c18164a08ed2f5f66c898958d0644f7a236a536f2b  LICENSE.md
> diff --git a/package/coremark-pro/coremark-pro.mk b/package/coremark-pro/coremark-pro.mk
> new file mode 100644
> index 0000000000..d3e99fcdfd
> --- /dev/null
> +++ b/package/coremark-pro/coremark-pro.mk
> @@ -0,0 +1,42 @@
> +################################################################################
> +#
> +# CoreMark-PRO
> +#
> +################################################################################
> +
> +COREMARK_PRO_VERSION = 1.1.2743
> +COREMARK_PRO_SITE = $(call github,eembc,coremark-pro,v$(COREMARK_PRO_VERSION))
> +COREMARK_PRO_LICENSE = Apache-2.0
> +COREMARK_PRO_LICENSE_FILES = LICENSE.md
> +COREMARK_PRO_DEPENDENCIES = perl
> +
> +ifeq ($(BR2_ENDIAN),"BIG")
> +COREMARK_PRO_MAKE_OPTS += PLATFORM_DEFINES='EE_BIG_ENDIAN=1 EE_LITTLE_ENDIAN=0'
> +else
> +COREMARK_PRO_MAKE_OPTS += PLATFORM_DEFINES='EE_BIG_ENDIAN=0 EE_LITTLE_ENDIAN=1'
> +endif
> +
> +COREMARK_PRO_MAKE_OPTS += \
> +	TARGET=linux$(if $(BR2_ARCH_IS_64),64) \
> +	EXE=
> +
> +define COREMARK_PRO_BUILD_CMDS
> +	$(TARGET_MAKE_ENV) $(MAKE) CC="$(TARGET_CC)" LD="$(TARGET_CC)" -C $(@D) \
> +		$(COREMARK_PRO_MAKE_OPTS) build
> +endef
> +
> +COREMARK_PRO_MARKS = cjpeg-rose7-preset core linear_alg-mid-100x100-sp loops-all-mid-10k-sp nnet_test parser-125k radix2-big-64k sha-test zip-test
> +COREMARK_PRO_SCRIPTS = results_parser.pl cert_median.pl cert_mark.pl headings.txt
> +
> +define COREMARK_PRO_INSTALL_TARGET_CMDS
> +	mkdir -p  $(TARGET_DIR)/usr/share/coremark-pro/logs
> +	$(foreach m,$(COREMARK_PRO_MARKS),\
> +		$(INSTALL) -D $(@D)/builds/linux$(if $(BR2_ARCH_IS_64),64)/gcc$(if $(BR2_ARCH_IS_64),64)/bin/$(m) $(TARGET_DIR)/usr/bin/$(m)$(sep) \
> +		size $(TARGET_DIR)/usr/bin/$(m) > $(TARGET_DIR)/usr/share/coremark-pro/logs/$(m).size.log$(sep))

 Why is this size.log needed? Would be nice to have mentioned that in the commit
message.

> +	$(INSTALL) -D $(@D)/builds/linux$(if $(BR2_ARCH_IS_64),64)/gcc$(if $(BR2_ARCH_IS_64),64)/data/libbmp/Rose256.bmp $(TARGET_DIR)/usr/share/coremark-pro/Rose256.bmp
> +	$(foreach s,$(COREMARK_PRO_SCRIPTS),\
> +		$(INSTALL) -D $(@D)/util/perl/$(s) $(TARGET_DIR)/usr/share/coremark-pro/util/perl/$(s)$(sep))
> +	$(Q)cp package/coremark-pro/coremark-pro.sh $(TARGET_DIR)/usr/bin/
> +endef
> +
> +$(eval $(generic-package))
> diff --git a/package/coremark-pro/coremark-pro.sh b/package/coremark-pro/coremark-pro.sh
> new file mode 100755
> index 0000000000..65ad9df645
> --- /dev/null
> +++ b/package/coremark-pro/coremark-pro.sh
> @@ -0,0 +1,36 @@
> +#!/bin/sh
> +# SPDX-License-Identifier: Apache-2.0
> +
> +workloads="cjpeg-rose7-preset core linear_alg-mid-100x100-sp \
> +    loops-all-mid-10k-sp nnet_test parser-125k radix2-big-64k sha-test zip-test"

 I think it would be better to set

workloads="@COREMARK_PRO_MARKS#@"

here, and substitute it from the .mk file. That way, changing the list has to be
done only in one place.

> +
> +rm -rf /tmp/coremark-pro
> +mkdir -p /tmp/coremark-pro
> +cp /usr/share/coremark-pro/Rose256.bmp /tmp/coremark-pro
> +cp /usr/share/coremark-pro/logs/*.size.log /tmp/coremark-pro
> +
> +cd /tmp/coremark-pro
> +
> +for contype in single best; do
> +    cat /usr/share/coremark-pro/util/perl/headings.txt >>result.log
> +    for wld in $workloads; do
> +        [ "$contype" = "single" ] && XCMD="-c1 -w1"
> +        [ "$contype" = "best" ] && XCMD="-c$(nproc) -w$(nproc)"
> +        echo "#Results for verification run started at $(date +%y%j:%H:%M:%S) XCMD=$XCMD" >$wld-$contype-result.log
> +        echo "Verification run for $wld"
> +        $wld $XCMD -v1 > $wld.run.log
> +        LC_ALL=C perl /usr/share/coremark-pro/util/perl/results_parser.pl $wld-$contype-result.log $wld.run.log> +        echo "#Results for performance runs started at $(date +%y%j:%H:%M:%S)
XCMD=$XCMD" >>$wld-$contype-result.log
> +        echo "Performance run for $wld"
> +        for i in 1 2 3; do
> +            $wld $XCMD -v0 > $wld.run.log
> +            LC_ALL=C perl /usr/share/coremark-pro/util/perl/results_parser.pl $wld-$contype-result.log $wld.run.log
> +        done
> +        echo "#Median for final result $wld" >>$wld-$contype-result.log
> +        LC_ALL=C perl /usr/share/coremark-pro/util/perl/cert_median.pl $wld-$contype-result.log $contype >>$wld-$contype-result.log

 Since this script supposedly generates user-readable output, I'm not sure if
the LC_ALL=C is appropriate. Similary, the timestamps should probably use
"+%X %x" as well. And maybe the other perl invocations too.

> +        cat $wld-$contype-result.log >>result.log
> +    done
> +done

 Shouldn't the /tmp/coremark-pro directory be cleaned up again afterwards?


 None of these are major concerns, so I've applied to master (with just the
small fix in the hash file). Feel free to post improvements as follow up
patches. Thanks!

 Regards,
 Arnout

> +LC_ALL=C perl /usr/share/coremark-pro/util/perl/cert_mark.pl -i result.log -s coremarkpro > result.mark
> +cat result.mark
> +cd - >/dev/null
> 

^ permalink raw reply	[flat|nested] 7+ messages in thread
* [Buildroot] [PATCH v2 2/2] package/coremark-pro: new package
  2021-01-07 22:00   ` Arnout Vandecappelle
@ 2021-01-08  8:04     ` Chris Packham
  0 siblings, 0 replies; 7+ messages in thread
From: Chris Packham @ 2021-01-08  8:04 UTC (permalink / raw)
  To: buildroot

On Fri, 8 Jan 2021, 11:00 AM Arnout Vandecappelle, <arnout@mind.be> wrote:

>
>
> On 11/11/2020 23:41, Chris Packham wrote:
> > CoreMark-Pro is a comprehensive, advanced processor benchmark that
> > works with and enhances the market-proven industry-standard EEMBC
> > CoreMark benchmark.
> >
> > https://www.eembc.org/coremark-pro/
> >
> > Signed-off-by: Chris Packham <judge.packham@gmail.com>
> > ---
> > Changes in v2:
> > - Add depends on BR2_USE_MMU (thanks Matt)
> >
> [snip]
> > diff --git a/package/coremark-pro/coremark-pro.hash
> b/package/coremark-pro/coremark-pro.hash
> > new file mode 100644
> > index 0000000000..16b121edb3
> > --- /dev/null
> > +++ b/package/coremark-pro/coremark-pro.hash
> > @@ -0,0 +1,3 @@
> > +#locally computed
>
>  (nitpick)
>
> # Locally computed
>

Noted


> > +sha256
> 440223554134832efec4ad220795611769ded3869fbc3ce3c59b05b7ddeb9f6d
> coremark-pro-1.1.2743.tar.gz
> > +sha256
> 0d7775bd793ea71fef3594c18164a08ed2f5f66c898958d0644f7a236a536f2b  LICENSE.md
> > diff --git a/package/coremark-pro/coremark-pro.mk
> b/package/coremark-pro/coremark-pro.mk
> > new file mode 100644
> > index 0000000000..d3e99fcdfd
> > --- /dev/null
> > +++ b/package/coremark-pro/coremark-pro.mk
> > @@ -0,0 +1,42 @@
> >
> +################################################################################
> > +#
> > +# CoreMark-PRO
> > +#
> >
> +################################################################################
> > +
> > +COREMARK_PRO_VERSION = 1.1.2743
> > +COREMARK_PRO_SITE = $(call
> github,eembc,coremark-pro,v$(COREMARK_PRO_VERSION))
> > +COREMARK_PRO_LICENSE = Apache-2.0
> > +COREMARK_PRO_LICENSE_FILES = LICENSE.md
> > +COREMARK_PRO_DEPENDENCIES = perl
> > +
> > +ifeq ($(BR2_ENDIAN),"BIG")
> > +COREMARK_PRO_MAKE_OPTS += PLATFORM_DEFINES='EE_BIG_ENDIAN=1
> EE_LITTLE_ENDIAN=0'
> > +else
> > +COREMARK_PRO_MAKE_OPTS += PLATFORM_DEFINES='EE_BIG_ENDIAN=0
> EE_LITTLE_ENDIAN=1'
> > +endif
> > +
> > +COREMARK_PRO_MAKE_OPTS += \
> > +     TARGET=linux$(if $(BR2_ARCH_IS_64),64) \
> > +     EXE=
> > +
> > +define COREMARK_PRO_BUILD_CMDS
> > +     $(TARGET_MAKE_ENV) $(MAKE) CC="$(TARGET_CC)" LD="$(TARGET_CC)" -C
> $(@D) \
> > +             $(COREMARK_PRO_MAKE_OPTS) build
> > +endef
> > +
> > +COREMARK_PRO_MARKS = cjpeg-rose7-preset core linear_alg-mid-100x100-sp
> loops-all-mid-10k-sp nnet_test parser-125k radix2-big-64k sha-test zip-test
> > +COREMARK_PRO_SCRIPTS = results_parser.pl cert_median.pl cert_mark.pl
> headings.txt
> > +
> > +define COREMARK_PRO_INSTALL_TARGET_CMDS
> > +     mkdir -p  $(TARGET_DIR)/usr/share/coremark-pro/logs
> > +     $(foreach m,$(COREMARK_PRO_MARKS),\
> > +             $(INSTALL) -D $(@D)/builds/linux$(if
> $(BR2_ARCH_IS_64),64)/gcc$(if $(BR2_ARCH_IS_64),64)/bin/$(m)
> $(TARGET_DIR)/usr/bin/$(m)$(sep) \
> > +             size $(TARGET_DIR)/usr/bin/$(m) >
> $(TARGET_DIR)/usr/share/coremark-pro/logs/$(m).size.log$(sep))
>
>  Why is this size.log needed? Would be nice to have mentioned that in the
> commit
> message.
>

I _think_ it's used as part of verifying the benchmarks. Doing it this way
saves having to depend on binutils for the target.


> > +     $(INSTALL) -D $(@D)/builds/linux$(if $(BR2_ARCH_IS_64),64)/gcc$(if
> $(BR2_ARCH_IS_64),64)/data/libbmp/Rose256.bmp
> $(TARGET_DIR)/usr/share/coremark-pro/Rose256.bmp
> > +     $(foreach s,$(COREMARK_PRO_SCRIPTS),\
> > +             $(INSTALL) -D $(@D)/util/perl/$(s)
> $(TARGET_DIR)/usr/share/coremark-pro/util/perl/$(s)$(sep))
> > +     $(Q)cp package/coremark-pro/coremark-pro.sh $(TARGET_DIR)/usr/bin/
> > +endef
> > +
> > +$(eval $(generic-package))
> > diff --git a/package/coremark-pro/coremark-pro.sh
> b/package/coremark-pro/coremark-pro.sh
> > new file mode 100755
> > index 0000000000..65ad9df645
> > --- /dev/null
> > +++ b/package/coremark-pro/coremark-pro.sh
> > @@ -0,0 +1,36 @@
> > +#!/bin/sh
> > +# SPDX-License-Identifier: Apache-2.0
> > +
> > +workloads="cjpeg-rose7-preset core linear_alg-mid-100x100-sp \
> > +    loops-all-mid-10k-sp nnet_test parser-125k radix2-big-64k sha-test
> zip-test"
>
>  I think it would be better to set
>
> workloads="@COREMARK_PRO_MARKS#@"
>
> here, and substitute it from the .mk file. That way, changing the list has
> to be
> done only in one place.
>

Yep agreed. For such things is it normal to have a .sh.in and "build" the
actual .sh file or just sed it in place?


> > +
> > +rm -rf /tmp/coremark-pro
> > +mkdir -p /tmp/coremark-pro
> > +cp /usr/share/coremark-pro/Rose256.bmp /tmp/coremark-pro
> > +cp /usr/share/coremark-pro/logs/*.size.log /tmp/coremark-pro
> > +
> > +cd /tmp/coremark-pro
> > +
> > +for contype in single best; do
> > +    cat /usr/share/coremark-pro/util/perl/headings.txt >>result.log
> > +    for wld in $workloads; do
> > +        [ "$contype" = "single" ] && XCMD="-c1 -w1"
> > +        [ "$contype" = "best" ] && XCMD="-c$(nproc) -w$(nproc)"
> > +        echo "#Results for verification run started at $(date
> +%y%j:%H:%M:%S) XCMD=$XCMD" >$wld-$contype-result.log
> > +        echo "Verification run for $wld"
> > +        $wld $XCMD -v1 > $wld.run.log
> > +        LC_ALL=C perl /usr/share/coremark-pro/util/perl/
> results_parser.pl $wld-$contype-result.log $wld.run.log> +        echo
> "#Results for performance runs started at $(date +%y%j:%H:%M:%S)
> XCMD=$XCMD" >>$wld-$contype-result.log
> > +        echo "Performance run for $wld"
> > +        for i in 1 2 3; do
> > +            $wld $XCMD -v0 > $wld.run.log
> > +            LC_ALL=C perl /usr/share/coremark-pro/util/perl/
> results_parser.pl $wld-$contype-result.log $wld.run.log
> > +        done
> > +        echo "#Median for final result $wld" >>$wld-$contype-result.log
> > +        LC_ALL=C perl /usr/share/coremark-pro/util/perl/cert_median.pl
> $wld-$contype-result.log $contype >>$wld-$contype-result.log
>
>  Since this script supposedly generates user-readable output, I'm not sure
> if
> the LC_ALL=C is appropriate.


I think LC_ALL=C is appropriate as it enables comparison between
boards/SoCs without subtle differences due to the current locale.

Similary, the timestamps should probably use
> "+%X %x" as well.


Yep that's reasonable.

And maybe the other perl invocations too.
>
> > +        cat $wld-$contype-result.log >>result.log
> > +    done
> > +done
>
>  Shouldn't the /tmp/coremark-pro directory be cleaned up again afterwards?
>

I kind of went the otherway in deleting it at the start of the script. My
thought was to leave the artifacts around so they could be manually
inspected or further processed.


>  None of these are major concerns, so I've applied to master (with just the
> small fix in the hash file). Feel free to post improvements as follow up
> patches. Thanks!
>

Thank you. I'm still on an extended break (southern hemisphere summer) so
I'll look at sending some of these updates when I'm in front of a real
keyboard.


>  Regards,
>  Arnout
>
> > +LC_ALL=C perl /usr/share/coremark-pro/util/perl/cert_mark.pl -i
> result.log -s coremarkpro > result.mark
> > +cat result.mark
> > +cd - >/dev/null
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20210108/a8dd09d2/attachment.html>

^ permalink raw reply	[flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-01-08  8:04 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-11 22:41 [Buildroot] [PATCH v2 1/2] package/coremark: new package Chris Packham
2020-11-11 22:41 ` [Buildroot] [PATCH v2 2/2] package/coremark-pro: " Chris Packham
2020-11-12 13:17   ` Matthew Weber
2021-01-07 22:00   ` Arnout Vandecappelle
2021-01-08  8:04     ` Chris Packham
2020-11-12 13:15 ` [Buildroot] [PATCH v2 1/2] package/coremark: " Matthew Weber
2021-01-07 21:48 ` Arnout Vandecappelle

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.