* [meta-oe][PATCH] add a storage I/O benchmark test suite
@ 2018-10-19 14:54 Anders Roxell
2018-10-20 23:28 ` Randy MacLeod
0 siblings, 1 reply; 8+ messages in thread
From: Anders Roxell @ 2018-10-19 14:54 UTC (permalink / raw)
To: openembedded-devel
Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
---
.../recipes-benchmark/s-suite/s-suite_git.bb | 30 +++++++++++++++++++
1 file changed, 30 insertions(+)
create mode 100644 meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
diff --git a/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
new file mode 100644
index 000000000000..77d172814065
--- /dev/null
+++ b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
@@ -0,0 +1,30 @@
+SUMMARY = "Small collection of benchmarks for storage I/O"
+LICENSE = "GPLv2"
+LIC_FILES_CHKSUM = "file://COPYING;md5=b529aaa6a0c50f15d29f89609b5c22f3"
+
+SRCREV = "79698f645bfb28d0d966484ddad3a1efb562246d"
+PV = "0.0+git${SRCPV}"
+SRC_URI = "git://github.com/Algodev-github/S.git;protocol=https;branch=master"
+
+S = "${WORKDIR}/git"
+
+# installing in /opt/S-suite since the package has
+# dependencies to the directory structure.
+do_install() {
+ install -d ${D}/opt/S-suite
+ for i in $(find ${S}/* -type d); do
+ install -d ${D}/opt/S-suite/$(basename $i)
+ install -m0755 -p ${S}/$(basename $i)/* ${D}/opt/S-suite/$(basename $i)
+ done
+
+ install -m0755 ${S}/config_params.sh ${D}/opt/S-suite
+ install -m0755 ${S}/def_config_params.sh ${D}/opt/S-suite
+}
+
+RDEPENDS_${PN} = "bash bc coreutils gawk g++ gcc fio libaio libaio-dev sysstat"
+
+FILES_${PN} = "/opt/S-suite/"
+
+# added to INSANE_SKIP since s-suite have an runtime
+# dependency (RDEPENDS) on libaio-dev.
+INSANE_SKIP_${PN} += "dev-deps"
--
2.19.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [meta-oe][PATCH] add a storage I/O benchmark test suite
2018-10-19 14:54 [meta-oe][PATCH] add a storage I/O benchmark test suite Anders Roxell
@ 2018-10-20 23:28 ` Randy MacLeod
2018-10-21 15:10 ` Khem Raj
0 siblings, 1 reply; 8+ messages in thread
From: Randy MacLeod @ 2018-10-20 23:28 UTC (permalink / raw)
To: Anders Roxell, openembedded-devel
On 10/19/18 10:54 AM, Anders Roxell wrote:
> Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
> ---
> .../recipes-benchmark/s-suite/s-suite_git.bb | 30 +++++++++++++++++++
> 1 file changed, 30 insertions(+)
> create mode 100644 meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
>
> diff --git a/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
> new file mode 100644
> index 000000000000..77d172814065
> --- /dev/null
> +++ b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
> @@ -0,0 +1,30 @@
> +SUMMARY = "Small collection of benchmarks for storage I/O"
> +LICENSE = "GPLv2"
> +LIC_FILES_CHKSUM = "file://COPYING;md5=b529aaa6a0c50f15d29f89609b5c22f3"
> +
> +SRCREV = "79698f645bfb28d0d966484ddad3a1efb562246d"
> +PV = "0.0+git${SRCPV}"
> +SRC_URI = "git://github.com/Algodev-github/S.git;protocol=https;branch=master"
> +
> +S = "${WORKDIR}/git"
> +
> +# installing in /opt/S-suite since the package has
> +# dependencies to the directory structure.
> +do_install() {
> + install -d ${D}/opt/S-suite
> + for i in $(find ${S}/* -type d); do
> + install -d ${D}/opt/S-suite/$(basename $i)
> + install -m0755 -p ${S}/$(basename $i)/* ${D}/opt/S-suite/$(basename $i)
> + done
> +
> + install -m0755 ${S}/config_params.sh ${D}/opt/S-suite
> + install -m0755 ${S}/def_config_params.sh ${D}/opt/S-suite
> +}
> +
> +RDEPENDS_${PN} = "bash bc coreutils gawk g++ gcc fio libaio libaio-dev sysstat"
> +
> +FILES_${PN} = "/opt/S-suite/"
> +
> +# added to INSANE_SKIP since s-suite have an runtime
> +# dependency (RDEPENDS) on libaio-dev.
> +INSANE_SKIP_${PN} += "dev-deps"
>
Thanks for the submission but from:
https://github.com/Algodev-github/S
"These benchmarks have been written just for internal use,
and are more a yard than a well-finished work."
That doesn't lead to a lot of confidence that these suite will
be maintained for years. What's the value to meta-oe users to add
this recipe compared to something that has been around for years
if not decades such as Bonnie++:
https://layers.openembedded.org/layerindex/recipe/864/
--
# Randy MacLeod
# Wind River Linux
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [meta-oe][PATCH] add a storage I/O benchmark test suite
2018-10-20 23:28 ` Randy MacLeod
@ 2018-10-21 15:10 ` Khem Raj
2018-10-23 13:33 ` Anders Roxell
0 siblings, 1 reply; 8+ messages in thread
From: Khem Raj @ 2018-10-21 15:10 UTC (permalink / raw)
To: Randy MacLeod; +Cc: openembeded-devel
On Sun, Oct 21, 2018 at 12:29 AM Randy MacLeod
<randy.macleod@windriver.com> wrote:
>
> On 10/19/18 10:54 AM, Anders Roxell wrote:
> > Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
> > ---
> > .../recipes-benchmark/s-suite/s-suite_git.bb | 30 +++++++++++++++++++
> > 1 file changed, 30 insertions(+)
> > create mode 100644 meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
> >
> > diff --git a/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
> > new file mode 100644
> > index 000000000000..77d172814065
> > --- /dev/null
> > +++ b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
> > @@ -0,0 +1,30 @@
> > +SUMMARY = "Small collection of benchmarks for storage I/O"
> > +LICENSE = "GPLv2"
> > +LIC_FILES_CHKSUM = "file://COPYING;md5=b529aaa6a0c50f15d29f89609b5c22f3"
> > +
> > +SRCREV = "79698f645bfb28d0d966484ddad3a1efb562246d"
> > +PV = "0.0+git${SRCPV}"
> > +SRC_URI = "git://github.com/Algodev-github/S.git;protocol=https;branch=master"
> > +
> > +S = "${WORKDIR}/git"
> > +
> > +# installing in /opt/S-suite since the package has
> > +# dependencies to the directory structure.
> > +do_install() {
> > + install -d ${D}/opt/S-suite
> > + for i in $(find ${S}/* -type d); do
> > + install -d ${D}/opt/S-suite/$(basename $i)
> > + install -m0755 -p ${S}/$(basename $i)/* ${D}/opt/S-suite/$(basename $i)
> > + done
> > +
> > + install -m0755 ${S}/config_params.sh ${D}/opt/S-suite
> > + install -m0755 ${S}/def_config_params.sh ${D}/opt/S-suite
> > +}
> > +
> > +RDEPENDS_${PN} = "bash bc coreutils gawk g++ gcc fio libaio libaio-dev sysstat"
> > +
> > +FILES_${PN} = "/opt/S-suite/"
> > +
> > +# added to INSANE_SKIP since s-suite have an runtime
> > +# dependency (RDEPENDS) on libaio-dev.
> > +INSANE_SKIP_${PN} += "dev-deps"
> >
>
> Thanks for the submission but from:
>
> https://github.com/Algodev-github/S
>
> "These benchmarks have been written just for internal use,
> and are more a yard than a well-finished work."
>
>
> That doesn't lead to a lot of confidence that these suite will
> be maintained for years. What's the value to meta-oe users to add
> this recipe compared to something that has been around for years
> if not decades such as Bonnie++:
> https://layers.openembedded.org/layerindex/recipe/864/
>
how I read this is that they have not made proper installation
targets and its dependencies or letting user know of the dependencies
which actually is not a problem with OE since both
of these would be taken care of in the recipe with
do_install and DEPENDS
>
> --
> # Randy MacLeod
> # Wind River Linux
> --
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [meta-oe][PATCH] add a storage I/O benchmark test suite
2018-10-21 15:10 ` Khem Raj
@ 2018-10-23 13:33 ` Anders Roxell
2018-10-25 17:46 ` Paolo Valente
0 siblings, 1 reply; 8+ messages in thread
From: Anders Roxell @ 2018-10-23 13:33 UTC (permalink / raw)
To: Khem Raj; +Cc: Paolo Valente, openembedded-devel
CCing in the package maintainer.
On Sun, 21 Oct 2018 at 17:10, Khem Raj <raj.khem@gmail.com> wrote:
>
> On Sun, Oct 21, 2018 at 12:29 AM Randy MacLeod
> <randy.macleod@windriver.com> wrote:
> >
> > On 10/19/18 10:54 AM, Anders Roxell wrote:
> > > Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
> > > ---
> > > .../recipes-benchmark/s-suite/s-suite_git.bb | 30 +++++++++++++++++++
> > > 1 file changed, 30 insertions(+)
> > > create mode 100644 meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
> > >
> > > diff --git a/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
> > > new file mode 100644
> > > index 000000000000..77d172814065
> > > --- /dev/null
> > > +++ b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
> > > @@ -0,0 +1,30 @@
> > > +SUMMARY = "Small collection of benchmarks for storage I/O"
> > > +LICENSE = "GPLv2"
> > > +LIC_FILES_CHKSUM = "file://COPYING;md5=b529aaa6a0c50f15d29f89609b5c22f3"
> > > +
> > > +SRCREV = "79698f645bfb28d0d966484ddad3a1efb562246d"
> > > +PV = "0.0+git${SRCPV}"
> > > +SRC_URI = "git://github.com/Algodev-github/S.git;protocol=https;branch=master"
> > > +
> > > +S = "${WORKDIR}/git"
> > > +
> > > +# installing in /opt/S-suite since the package has
> > > +# dependencies to the directory structure.
> > > +do_install() {
> > > + install -d ${D}/opt/S-suite
> > > + for i in $(find ${S}/* -type d); do
> > > + install -d ${D}/opt/S-suite/$(basename $i)
> > > + install -m0755 -p ${S}/$(basename $i)/* ${D}/opt/S-suite/$(basename $i)
> > > + done
> > > +
> > > + install -m0755 ${S}/config_params.sh ${D}/opt/S-suite
> > > + install -m0755 ${S}/def_config_params.sh ${D}/opt/S-suite
> > > +}
> > > +
> > > +RDEPENDS_${PN} = "bash bc coreutils gawk g++ gcc fio libaio libaio-dev sysstat"
> > > +
> > > +FILES_${PN} = "/opt/S-suite/"
> > > +
> > > +# added to INSANE_SKIP since s-suite have an runtime
> > > +# dependency (RDEPENDS) on libaio-dev.
> > > +INSANE_SKIP_${PN} += "dev-deps"
> > >
> >
> > Thanks for the submission but from:
> >
> > https://github.com/Algodev-github/S
> >
> > "These benchmarks have been written just for internal use,
> > and are more a yard than a well-finished work."
> >
> >
> > That doesn't lead to a lot of confidence that these suite will
> > be maintained for years. What's the value to meta-oe users to add
> > this recipe compared to something that has been around for years
> > if not decades such as Bonnie++:
> > https://layers.openembedded.org/layerindex/recipe/864/
> >
>
> how I read this is that they have not made proper installation
> targets and its dependencies or letting user know of the dependencies
> which actually is not a problem with OE since both
> of these would be taken care of in the recipe with
> do_install and DEPENDS
Do you mean DEPENDS or RDEPENDS?
Cheers,
Anders
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [meta-oe][PATCH] add a storage I/O benchmark test suite
2018-10-23 13:33 ` Anders Roxell
@ 2018-10-25 17:46 ` Paolo Valente
2018-10-25 18:17 ` Randy MacLeod
0 siblings, 1 reply; 8+ messages in thread
From: Paolo Valente @ 2018-10-25 17:46 UTC (permalink / raw)
To: Anders Roxell; +Cc: openembedded-devel
[RESENDING: FIRST ATTEMPT REJECTED BECAUSE I WAS NOT SUBSCRIBED TO THE LIST]
> Il giorno 23 ott 2018, alle ore 15:33, Anders Roxell <anders.roxell@linaro.org> ha scritto:
>
> CCing in the package maintainer.
>
Thanks Anders, hi everybody,
> On Sun, 21 Oct 2018 at 17:10, Khem Raj <raj.khem@gmail.com> wrote:
>>
>> On Sun, Oct 21, 2018 at 12:29 AM Randy MacLeod
>> <randy.macleod@windriver.com> wrote:
>>>
>>> On 10/19/18 10:54 AM, Anders Roxell wrote:
>>>> Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
>>>> ---
>>>> .../recipes-benchmark/s-suite/s-suite_git.bb | 30 +++++++++++++++++++
>>>> 1 file changed, 30 insertions(+)
>>>> create mode 100644 meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
>>>>
>>>> diff --git a/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
>>>> new file mode 100644
>>>> index 000000000000..77d172814065
>>>> --- /dev/null
>>>> +++ b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
>>>> @@ -0,0 +1,30 @@
>>>> +SUMMARY = "Small collection of benchmarks for storage I/O"
>>>> +LICENSE = "GPLv2"
>>>> +LIC_FILES_CHKSUM = "file://COPYING;md5=b529aaa6a0c50f15d29f89609b5c22f3"
>>>> +
>>>> +SRCREV = "79698f645bfb28d0d966484ddad3a1efb562246d"
>>>> +PV = "0.0+git${SRCPV}"
>>>> +SRC_URI = "git://github.com/Algodev-github/S.git;protocol=https;branch=master"
>>>> +
>>>> +S = "${WORKDIR}/git"
>>>> +
>>>> +# installing in /opt/S-suite since the package has
>>>> +# dependencies to the directory structure.
>>>> +do_install() {
>>>> + install -d ${D}/opt/S-suite
>>>> + for i in $(find ${S}/* -type d); do
>>>> + install -d ${D}/opt/S-suite/$(basename $i)
>>>> + install -m0755 -p ${S}/$(basename $i)/* ${D}/opt/S-suite/$(basename $i)
>>>> + done
>>>> +
>>>> + install -m0755 ${S}/config_params.sh ${D}/opt/S-suite
>>>> + install -m0755 ${S}/def_config_params.sh ${D}/opt/S-suite
>>>> +}
>>>> +
>>>> +RDEPENDS_${PN} = "bash bc coreutils gawk g++ gcc fio libaio libaio-dev sysstat"
>>>> +
>>>> +FILES_${PN} = "/opt/S-suite/"
>>>> +
>>>> +# added to INSANE_SKIP since s-suite have an runtime
>>>> +# dependency (RDEPENDS) on libaio-dev.
>>>> +INSANE_SKIP_${PN} += "dev-deps"
>>>>
>>>
>>> Thanks for the submission but from:
>>>
>>> https://github.com/Algodev-github/S
>>>
>>> "These benchmarks have been written just for internal use,
>>> and are more a yard than a well-finished work."
>>>
>>>
>>> That doesn't lead to a lot of confidence that these suite will
>>> be maintained for years.
Gosh. I put that disclaimer in the README when I first wrote it more
than 10 years ago. After then, thousands of changes have been made by
me and many other contributors. Unfortunately, I put the suite on
GitHub only recently. But as you can see from the log, there have
been 448 commits in (less than) the last five months.
In addition, the S suite is now used in a new Phoronix benchmark and
in a new mmtests benchmark.
So, believe me, the S suite ATM is definitely in good shape, and seems
to be here to stay.
I'll remove that line, right after finishing a non-trivial new commit
I'm working on.
>>> What's the value to meta-oe users to add
>>> this recipe compared to something that has been around for years
>>> if not decades such as Bonnie++:
>>> https://layers.openembedded.org/layerindex/recipe/864/
>>>
>>
The added value is that bonnie++ does not perform *any* of the benchmarks in the S suite, apart from that on throughput. I have made the S suite, many years ago, because I needed to run benchmarks that no suite provided. Here are some examples:
- start-up times of real applications under real background workloads
- playback quality (drop rate) of video and audio under real background workloads
- speed of code-development tasks (make, git checkout, git merge, git
grep) under real background workloads, plus measurement of system
responsiveness while one of these dev tasks is executed
- minimum per-client bandwidth guaranteed to a set of clients doing
any possible type of I/O
- maximum per-client latency guaranteed to a set of clients doing any
possible type of I/O
For any doubt, here I am.
Thanks,
Paolo
>> how I read this is that they have not made proper installation
>> targets and its dependencies or letting user know of the dependencies
>> which actually is not a problem with OE since both
>> of these would be taken care of in the recipe with
>> do_install and DEPENDS
>
> Do you mean DEPENDS or RDEPENDS?
>
> Cheers,
> Anders
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [meta-oe][PATCH] add a storage I/O benchmark test suite
2018-10-25 17:46 ` Paolo Valente
@ 2018-10-25 18:17 ` Randy MacLeod
2018-10-25 21:24 ` Khem Raj
2018-10-30 15:25 ` Paolo Valente
0 siblings, 2 replies; 8+ messages in thread
From: Randy MacLeod @ 2018-10-25 18:17 UTC (permalink / raw)
To: Paolo Valente, Anders Roxell; +Cc: openembedded-devel
On 10/25/18 1:46 PM, Paolo Valente wrote:
> [RESENDING: FIRST ATTEMPT REJECTED BECAUSE I WAS NOT SUBSCRIBED TO THE LIST]
>
>> Il giorno 23 ott 2018, alle ore 15:33, Anders Roxell <anders.roxell@linaro.org> ha scritto:
>>
>> CCing in the package maintainer.
>>
>
> Thanks Anders, hi everybody,
>
>> On Sun, 21 Oct 2018 at 17:10, Khem Raj <raj.khem@gmail.com> wrote:
>>>
>>> On Sun, Oct 21, 2018 at 12:29 AM Randy MacLeod
>>> <randy.macleod@windriver.com> wrote:
>>>>
>>>> On 10/19/18 10:54 AM, Anders Roxell wrote:
>>>>> Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
>>>>> ---
>>>>> .../recipes-benchmark/s-suite/s-suite_git.bb | 30 +++++++++++++++++++
>>>>> 1 file changed, 30 insertions(+)
>>>>> create mode 100644 meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
>>>>>
>>>>> diff --git a/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
>>>>> new file mode 100644
>>>>> index 000000000000..77d172814065
>>>>> --- /dev/null
>>>>> +++ b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
>>>>> @@ -0,0 +1,30 @@
>>>>> +SUMMARY = "Small collection of benchmarks for storage I/O"
>>>>> +LICENSE = "GPLv2"
>>>>> +LIC_FILES_CHKSUM = "file://COPYING;md5=b529aaa6a0c50f15d29f89609b5c22f3"
>>>>> +
>>>>> +SRCREV = "79698f645bfb28d0d966484ddad3a1efb562246d"
>>>>> +PV = "0.0+git${SRCPV}"
>>>>> +SRC_URI = "git://github.com/Algodev-github/S.git;protocol=https;branch=master"
>>>>> +
>>>>> +S = "${WORKDIR}/git"
>>>>> +
>>>>> +# installing in /opt/S-suite since the package has
>>>>> +# dependencies to the directory structure.
>>>>> +do_install() {
>>>>> + install -d ${D}/opt/S-suite
>>>>> + for i in $(find ${S}/* -type d); do
>>>>> + install -d ${D}/opt/S-suite/$(basename $i)
>>>>> + install -m0755 -p ${S}/$(basename $i)/* ${D}/opt/S-suite/$(basename $i)
>>>>> + done
>>>>> +
>>>>> + install -m0755 ${S}/config_params.sh ${D}/opt/S-suite
>>>>> + install -m0755 ${S}/def_config_params.sh ${D}/opt/S-suite
>>>>> +}
>>>>> +
>>>>> +RDEPENDS_${PN} = "bash bc coreutils gawk g++ gcc fio libaio libaio-dev sysstat"
>>>>> +
>>>>> +FILES_${PN} = "/opt/S-suite/"
>>>>> +
>>>>> +# added to INSANE_SKIP since s-suite have an runtime
>>>>> +# dependency (RDEPENDS) on libaio-dev.
>>>>> +INSANE_SKIP_${PN} += "dev-deps"
>>>>>
>>>>
>>>> Thanks for the submission but from:
>>>>
>>>> https://github.com/Algodev-github/S
>>>>
>>>> "These benchmarks have been written just for internal use,
>>>> and are more a yard than a well-finished work."
>>>>
>>>>
>>>> That doesn't lead to a lot of confidence that these suite will
>>>> be maintained for years.
>
> Gosh. I put that disclaimer in the README when I first wrote it more
> than 10 years ago. After then, thousands of changes have been made by
> me and many other contributors. Unfortunately, I put the suite on
> GitHub only recently. But as you can see from the log, there have
> been 448 commits in (less than) the last five months.
>
> In addition, the S suite is now used in a new Phoronix benchmark and
> in a new mmtests benchmark.
Ah, good to know.
>
> So, believe me, the S suite ATM is definitely in good shape, and seems
> to be here to stay.
>
> I'll remove that line, right after finishing a non-trivial new commit
> I'm working on.
>
>>>> What's the value to meta-oe users to add
>>>> this recipe compared to something that has been around for years
>>>> if not decades such as Bonnie++:
>>>> https://layers.openembedded.org/layerindex/recipe/864/
>>>>
>>>
>
> The added value is that bonnie++ does not perform *any* of the benchmarks in the S suite, apart from that on throughput. I have made the S suite, many years ago, because I needed to run benchmarks that no suite provided. Here are some examples:
> - start-up times of real applications under real background workloads
> - playback quality (drop rate) of video and audio under real background workloads
> - speed of code-development tasks (make, git checkout, git merge, git
> grep) under real background workloads, plus measurement of system
> responsiveness while one of these dev tasks is executed
> - minimum per-client bandwidth guaranteed to a set of clients doing
> any possible type of I/O
> - maximum per-client latency guaranteed to a set of clients doing any
> possible type of I/O
>
> For any doubt, here I am.
Thanks for the background Paolo.
Perhaps when you get to the clean-up work, you will simplify the
README based on your list above, move the details to another file
and mention it at the end of the README.
I withdraw my objection; this looks like a very useful
set of benchmarks. Long live Algodev-github/S. ;-)
../Randy
>
> Thanks,
> Paolo
>
>>> how I read this is that they have not made proper installation
>>> targets and its dependencies or letting user know of the dependencies
>>> which actually is not a problem with OE since both
>>> of these would be taken care of in the recipe with
>>> do_install and DEPENDS
>>
>> Do you mean DEPENDS or RDEPENDS?
>>
>> Cheers,
>> Anders
>
--
# Randy MacLeod
# Wind River Linux
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [meta-oe][PATCH] add a storage I/O benchmark test suite
2018-10-25 18:17 ` Randy MacLeod
@ 2018-10-25 21:24 ` Khem Raj
2018-10-30 15:25 ` Paolo Valente
1 sibling, 0 replies; 8+ messages in thread
From: Khem Raj @ 2018-10-25 21:24 UTC (permalink / raw)
To: Randy MacLeod; +Cc: openembeded-devel
On Thu, Oct 25, 2018 at 7:17 PM Randy MacLeod
<randy.macleod@windriver.com> wrote:
>
> On 10/25/18 1:46 PM, Paolo Valente wrote:
> > [RESENDING: FIRST ATTEMPT REJECTED BECAUSE I WAS NOT SUBSCRIBED TO THE LIST]
> >
> >> Il giorno 23 ott 2018, alle ore 15:33, Anders Roxell <anders.roxell@linaro.org> ha scritto:
> >>
> >> CCing in the package maintainer.
> >>
> >
> > Thanks Anders, hi everybody,
> >
> >> On Sun, 21 Oct 2018 at 17:10, Khem Raj <raj.khem@gmail.com> wrote:
> >>>
> >>> On Sun, Oct 21, 2018 at 12:29 AM Randy MacLeod
> >>> <randy.macleod@windriver.com> wrote:
> >>>>
> >>>> On 10/19/18 10:54 AM, Anders Roxell wrote:
> >>>>> Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
> >>>>> ---
> >>>>> .../recipes-benchmark/s-suite/s-suite_git.bb | 30 +++++++++++++++++++
> >>>>> 1 file changed, 30 insertions(+)
> >>>>> create mode 100644 meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
> >>>>>
> >>>>> diff --git a/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
> >>>>> new file mode 100644
> >>>>> index 000000000000..77d172814065
> >>>>> --- /dev/null
> >>>>> +++ b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
> >>>>> @@ -0,0 +1,30 @@
> >>>>> +SUMMARY = "Small collection of benchmarks for storage I/O"
> >>>>> +LICENSE = "GPLv2"
> >>>>> +LIC_FILES_CHKSUM = "file://COPYING;md5=b529aaa6a0c50f15d29f89609b5c22f3"
> >>>>> +
> >>>>> +SRCREV = "79698f645bfb28d0d966484ddad3a1efb562246d"
> >>>>> +PV = "0.0+git${SRCPV}"
> >>>>> +SRC_URI = "git://github.com/Algodev-github/S.git;protocol=https;branch=master"
> >>>>> +
> >>>>> +S = "${WORKDIR}/git"
> >>>>> +
> >>>>> +# installing in /opt/S-suite since the package has
> >>>>> +# dependencies to the directory structure.
> >>>>> +do_install() {
> >>>>> + install -d ${D}/opt/S-suite
> >>>>> + for i in $(find ${S}/* -type d); do
> >>>>> + install -d ${D}/opt/S-suite/$(basename $i)
> >>>>> + install -m0755 -p ${S}/$(basename $i)/* ${D}/opt/S-suite/$(basename $i)
> >>>>> + done
> >>>>> +
> >>>>> + install -m0755 ${S}/config_params.sh ${D}/opt/S-suite
> >>>>> + install -m0755 ${S}/def_config_params.sh ${D}/opt/S-suite
> >>>>> +}
> >>>>> +
> >>>>> +RDEPENDS_${PN} = "bash bc coreutils gawk g++ gcc fio libaio libaio-dev sysstat"
> >>>>> +
> >>>>> +FILES_${PN} = "/opt/S-suite/"
> >>>>> +
> >>>>> +# added to INSANE_SKIP since s-suite have an runtime
> >>>>> +# dependency (RDEPENDS) on libaio-dev.
> >>>>> +INSANE_SKIP_${PN} += "dev-deps"
> >>>>>
> >>>>
> >>>> Thanks for the submission but from:
> >>>>
> >>>> https://github.com/Algodev-github/S
> >>>>
> >>>> "These benchmarks have been written just for internal use,
> >>>> and are more a yard than a well-finished work."
> >>>>
> >>>>
> >>>> That doesn't lead to a lot of confidence that these suite will
> >>>> be maintained for years.
> >
> > Gosh. I put that disclaimer in the README when I first wrote it more
> > than 10 years ago. After then, thousands of changes have been made by
> > me and many other contributors. Unfortunately, I put the suite on
> > GitHub only recently. But as you can see from the log, there have
> > been 448 commits in (less than) the last five months.
> >
> > In addition, the S suite is now used in a new Phoronix benchmark and
> > in a new mmtests benchmark.
>
> Ah, good to know.
>
> >
> > So, believe me, the S suite ATM is definitely in good shape, and seems
> > to be here to stay.
> >
> > I'll remove that line, right after finishing a non-trivial new commit
> > I'm working on.
> >
> >>>> What's the value to meta-oe users to add
> >>>> this recipe compared to something that has been around for years
> >>>> if not decades such as Bonnie++:
> >>>> https://layers.openembedded.org/layerindex/recipe/864/
> >>>>
> >>>
> >
> > The added value is that bonnie++ does not perform *any* of the benchmarks in the S suite, apart from that on throughput. I have made the S suite, many years ago, because I needed to run benchmarks that no suite provided. Here are some examples:
> > - start-up times of real applications under real background workloads
> > - playback quality (drop rate) of video and audio under real background workloads
> > - speed of code-development tasks (make, git checkout, git merge, git
> > grep) under real background workloads, plus measurement of system
> > responsiveness while one of these dev tasks is executed
> > - minimum per-client bandwidth guaranteed to a set of clients doing
> > any possible type of I/O
> > - maximum per-client latency guaranteed to a set of clients doing any
> > possible type of I/O
> >
> > For any doubt, here I am.
>
> Thanks for the background Paolo.
>
> Perhaps when you get to the clean-up work, you will simplify the
> README based on your list above, move the details to another file
> and mention it at the end of the README.
>
> I withdraw my objection; this looks like a very useful
> set of benchmarks. Long live Algodev-github/S. ;-)
thanks for reviewing and feedback. Its appreciated. the patch is on its
path to master.
>
> ../Randy
>
> >
> > Thanks,
> > Paolo
> >
> >>> how I read this is that they have not made proper installation
> >>> targets and its dependencies or letting user know of the dependencies
> >>> which actually is not a problem with OE since both
> >>> of these would be taken care of in the recipe with
> >>> do_install and DEPENDS
> >>
> >> Do you mean DEPENDS or RDEPENDS?
> >>
> >> Cheers,
> >> Anders
> >
>
>
> --
> # Randy MacLeod
> # Wind River Linux
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [meta-oe][PATCH] add a storage I/O benchmark test suite
2018-10-25 18:17 ` Randy MacLeod
2018-10-25 21:24 ` Khem Raj
@ 2018-10-30 15:25 ` Paolo Valente
1 sibling, 0 replies; 8+ messages in thread
From: Paolo Valente @ 2018-10-30 15:25 UTC (permalink / raw)
To: Randy MacLeod; +Cc: openembedded-devel
> Il giorno 25 ott 2018, alle ore 20:17, Randy MacLeod <randy.macleod@windriver.com> ha scritto:
>
> On 10/25/18 1:46 PM, Paolo Valente wrote:
>> [RESENDING: FIRST ATTEMPT REJECTED BECAUSE I WAS NOT SUBSCRIBED TO THE LIST]
>>> Il giorno 23 ott 2018, alle ore 15:33, Anders Roxell <anders.roxell@linaro.org> ha scritto:
>>>
>>> CCing in the package maintainer.
>>>
>> Thanks Anders, hi everybody,
>>> On Sun, 21 Oct 2018 at 17:10, Khem Raj <raj.khem@gmail.com> wrote:
>>>>
>>>> On Sun, Oct 21, 2018 at 12:29 AM Randy MacLeod
>>>> <randy.macleod@windriver.com> wrote:
>>>>>
>>>>> On 10/19/18 10:54 AM, Anders Roxell wrote:
>>>>>> Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
>>>>>> ---
>>>>>> .../recipes-benchmark/s-suite/s-suite_git.bb | 30 +++++++++++++++++++
>>>>>> 1 file changed, 30 insertions(+)
>>>>>> create mode 100644 meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
>>>>>>
>>>>>> diff --git a/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
>>>>>> new file mode 100644
>>>>>> index 000000000000..77d172814065
>>>>>> --- /dev/null
>>>>>> +++ b/meta-oe/recipes-benchmark/s-suite/s-suite_git.bb
>>>>>> @@ -0,0 +1,30 @@
>>>>>> +SUMMARY = "Small collection of benchmarks for storage I/O"
>>>>>> +LICENSE = "GPLv2"
>>>>>> +LIC_FILES_CHKSUM = "file://COPYING;md5=b529aaa6a0c50f15d29f89609b5c22f3"
>>>>>> +
>>>>>> +SRCREV = "79698f645bfb28d0d966484ddad3a1efb562246d"
>>>>>> +PV = "0.0+git${SRCPV}"
>>>>>> +SRC_URI = "git://github.com/Algodev-github/S.git;protocol=https;branch=master"
>>>>>> +
>>>>>> +S = "${WORKDIR}/git"
>>>>>> +
>>>>>> +# installing in /opt/S-suite since the package has
>>>>>> +# dependencies to the directory structure.
>>>>>> +do_install() {
>>>>>> + install -d ${D}/opt/S-suite
>>>>>> + for i in $(find ${S}/* -type d); do
>>>>>> + install -d ${D}/opt/S-suite/$(basename $i)
>>>>>> + install -m0755 -p ${S}/$(basename $i)/* ${D}/opt/S-suite/$(basename $i)
>>>>>> + done
>>>>>> +
>>>>>> + install -m0755 ${S}/config_params.sh ${D}/opt/S-suite
>>>>>> + install -m0755 ${S}/def_config_params.sh ${D}/opt/S-suite
>>>>>> +}
>>>>>> +
>>>>>> +RDEPENDS_${PN} = "bash bc coreutils gawk g++ gcc fio libaio libaio-dev sysstat"
>>>>>> +
>>>>>> +FILES_${PN} = "/opt/S-suite/"
>>>>>> +
>>>>>> +# added to INSANE_SKIP since s-suite have an runtime
>>>>>> +# dependency (RDEPENDS) on libaio-dev.
>>>>>> +INSANE_SKIP_${PN} += "dev-deps"
>>>>>>
>>>>>
>>>>> Thanks for the submission but from:
>>>>>
>>>>> https://github.com/Algodev-github/S
>>>>>
>>>>> "These benchmarks have been written just for internal use,
>>>>> and are more a yard than a well-finished work."
>>>>>
>>>>>
>>>>> That doesn't lead to a lot of confidence that these suite will
>>>>> be maintained for years.
>> Gosh. I put that disclaimer in the README when I first wrote it more
>> than 10 years ago. After then, thousands of changes have been made by
>> me and many other contributors. Unfortunately, I put the suite on
>> GitHub only recently. But as you can see from the log, there have
>> been 448 commits in (less than) the last five months.
>> In addition, the S suite is now used in a new Phoronix benchmark and
>> in a new mmtests benchmark.
>
> Ah, good to know.
>
>> So, believe me, the S suite ATM is definitely in good shape, and seems
>> to be here to stay.
>> I'll remove that line, right after finishing a non-trivial new commit
>> I'm working on.
>>>>> What's the value to meta-oe users to add
>>>>> this recipe compared to something that has been around for years
>>>>> if not decades such as Bonnie++:
>>>>> https://layers.openembedded.org/layerindex/recipe/864/
>>>>>
>>>>
>> The added value is that bonnie++ does not perform *any* of the benchmarks in the S suite, apart from that on throughput. I have made the S suite, many years ago, because I needed to run benchmarks that no suite provided. Here are some examples:
>> - start-up times of real applications under real background workloads
>> - playback quality (drop rate) of video and audio under real background workloads
>> - speed of code-development tasks (make, git checkout, git merge, git
>> grep) under real background workloads, plus measurement of system
>> responsiveness while one of these dev tasks is executed
>> - minimum per-client bandwidth guaranteed to a set of clients doing
>> any possible type of I/O
>> - maximum per-client latency guaranteed to a set of clients doing any
>> possible type of I/O
>> For any doubt, here I am.
>
> Thanks for the background Paolo.
>
> Perhaps when you get to the clean-up work, you will simplify the
> README based on your list above, move the details to another file
> and mention it at the end of the README.
>
Just done that too, thank you very much for this suggestion.
Paolo
> I withdraw my objection; this looks like a very useful
> set of benchmarks. Long live Algodev-github/S. ;-)
>
> ../Randy
>
>> Thanks,
>> Paolo
>>>> how I read this is that they have not made proper installation
>>>> targets and its dependencies or letting user know of the dependencies
>>>> which actually is not a problem with OE since both
>>>> of these would be taken care of in the recipe with
>>>> do_install and DEPENDS
>>>
>>> Do you mean DEPENDS or RDEPENDS?
>>>
>>> Cheers,
>>> Anders
>
>
> --
> # Randy MacLeod
> # Wind River Linux
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2018-10-30 15:25 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-10-19 14:54 [meta-oe][PATCH] add a storage I/O benchmark test suite Anders Roxell
2018-10-20 23:28 ` Randy MacLeod
2018-10-21 15:10 ` Khem Raj
2018-10-23 13:33 ` Anders Roxell
2018-10-25 17:46 ` Paolo Valente
2018-10-25 18:17 ` Randy MacLeod
2018-10-25 21:24 ` Khem Raj
2018-10-30 15:25 ` Paolo Valente
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.