From: Shally Verma <shallyv@marvell.com>
To: "Trybula, ArturX" <arturx.trybula@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"Trahe, Fiona" <fiona.trahe@intel.com>,
"Dybkowski, AdamX" <adamx.dybkowski@intel.com>
Subject: Re: [dpdk-dev] [PATCH] app/test-compress-perf: report header improvement
Date: Thu, 27 Jun 2019 14:38:48 +0000 [thread overview]
Message-ID: <BN6PR1801MB20525330D91140C460238EDDADFD0@BN6PR1801MB2052.namprd18.prod.outlook.com> (raw)
In-Reply-To: <5B6D1C77E9D7034C93E97BD83D1D9F57077DBB98@HASMSX110.ger.corp.intel.com>
> -----Original Message-----
> From: Trybula, ArturX <arturx.trybula@intel.com>
> Sent: Thursday, June 27, 2019 5:17 PM
> To: Shally Verma <shallyv@marvell.com>; dev@dpdk.org; Trahe, Fiona
> <fiona.trahe@intel.com>; Dybkowski, AdamX
> <adamx.dybkowski@intel.com>
> Subject: [EXT] RE: [dpdk-dev] [PATCH] app/test-compress-perf: report
> header improvement
>
> External Email
>
> ----------------------------------------------------------------------
>
>
> -----Original Message-----
> From: Shally Verma [mailto:shallyv@marvell.com]
> Sent: Thursday, June 27, 2019 13:01
> To: Trybula, ArturX <arturx.trybula@intel.com>; dev@dpdk.org; Trahe, Fiona
> <fiona.trahe@intel.com>; Dybkowski, AdamX
> <adamx.dybkowski@intel.com>
> Subject: RE: [dpdk-dev] [PATCH] app/test-compress-perf: report header
> improvement
>
>
>
> > -----Original Message-----
> > From: Trybula, ArturX <arturx.trybula@intel.com>
> > Sent: Thursday, June 27, 2019 2:29 PM
> > To: Shally Verma <shallyv@marvell.com>; dev@dpdk.org; Trahe, Fiona
> > <fiona.trahe@intel.com>; Dybkowski, AdamX
> <adamx.dybkowski@intel.com>
> > Subject: [EXT] RE: [dpdk-dev] [PATCH] app/test-compress-perf: report
> > header improvement
> >
> > External Email
> >
> > ----------------------------------------------------------------------
> >
> >
> > -----Original Message-----
> > From: Shally Verma [mailto:shallyv@marvell.com]
> > Sent: Wednesday, June 26, 2019 19:04
> > To: Trybula, ArturX <arturx.trybula@intel.com>; dev@dpdk.org; Trahe,
> > Fiona <fiona.trahe@intel.com>; Dybkowski, AdamX
> > <adamx.dybkowski@intel.com>
> > Subject: RE: [dpdk-dev] [PATCH] app/test-compress-perf: report header
> > improvement
> >
> >
> >
> > > -----Original Message-----
> > > From: dev <dev-bounces@dpdk.org> On Behalf Of Artur Trybula
> > > Sent: Monday, June 3, 2019 3:54 PM
> > > To: dev@dpdk.org; fiona.trahe@intel.com; arturx.trybula@intel.com;
> > > adamx.dybkowski@intel.com
> > > Subject: [dpdk-dev] [PATCH] app/test-compress-perf: report header
> > > improvement
> > >
> > > This patch adds extra features to the compress performance test.
> > > Some important parameters (memory allocation, number of ops, number
> > > of
> > > segments) are calculated and printed out on the screen.
> > >
> > > Signed-off-by: Artur Trybula <arturx.trybula@intel.com>
> > > ---
> > > app/test-compress-perf/main.c | 105
> > > +++++++++++++++++++++++++++++++---
> > > 1 file changed, 98 insertions(+), 7 deletions(-)
> > >
> > ......
> > >
> > > -
> > > +static void
> > > +print_report_header(void)
> > > +{
> > > + uint32_t opt_total_segs = DIV_CEIL(tests_res.input_data_sz,
> > > + MAX_SEG_SIZE);
> > > +
> > > + if (tests_res.total_buffs > 1) {
> > > + printf("\nWarning: for the current input parameters number"
> > > + " of ops is higher than one, which may result"
> > > + " in sub-optimal performance.\n");
> > > + printf("To improve the performance (for the current"
> > > + " input data) following parameters are"
> > > + " suggested:\n");
> > > + printf(" • Segment size: %d\n", MAX_SEG_SIZE);
> > > + printf(" • Number of segments: %u\n", opt_total_segs);
> > > + } else if (tests_res.total_buffs == 1) {
> > > + printf("\nWarning: There is only one op with %u segments –"
> > > + " the compression ratio is the best.\n",
> > > + tests_res.segments_per_last_buff);
> > > + if (tests_res.segment_sz < MAX_SEG_SIZE)
> > > + printf("To reduce compression time, please use"
> > > + " bigger segment size: %d.\n",
> > > + MAX_SEG_SIZE);
> > > + else if (tests_res.segment_sz == MAX_SEG_SIZE)
> > > + printf("Segment size is optimal for the best"
> > > + " performance.\n");
> >
> > [Shally] Why these kind of warnings? If total_bufs > 1, then, how
> > behavior would change? Same question for segment size as well?
> > [Artur] I suppose you mean the second one "Warning: There is only one
> ...".
> > It's just to preserve the structure of the report for both variants.
> > But I agree, there is nothing wrong if there is only one op.
> > Please notice that each op is processed independently, in most cases
> > it implies lower compression ratio. The best situation is to have all
> > the input data aggregated into one op (even if partitioned across many
> mbufs).
> >
> [Shally] Then can we better covert this in terms of "stateless" and "stateful" .
> Example, Say it like:
> Since test app support stateless mode only, thus if input data is segmented
> across multiple buffers (i.e. op) Then each will be processed independently
> (i.e. in stateless mode) . Thus to gain better results, it might be more useful
> to put all data in one buffer.
> [Artur] We started from an idea of simple checking memory allocation. From
> my point of view we should avoid making it too complex unless there is a
> clear need. This is a "simple tool" for quick verification what is the effect of
> the input parameters. It's for developers familiar with the compression. I
> agree with you that in case of supporting both stateless and stateful
> compression such information should be printed out.
>
[Shally] Okay. I leave it to your comfort. But could you confirm if I understood it correct?
I mean keep num_bufs = 1, will result in better history thus better compression.
>
> > > + } else
> > > + printf("Warning: something wrong happened!!\n");
> > > +
> > > + printf("\nFor the current input parameters (segment size = %u,"
> > > + " segments number = %u):\n",
> > > + tests_res.segment_sz,
> > > + tests_res.segments_per_buff);
> > > + printf(" • Total number of segments: %d\n",
> > > + tests_res.total_segments);
> > > + printf(" • %u segments %u bytes long, last segment %u"
> > > + " byte(s) long\n",
> > > + tests_res.total_segments - 1,
> > > + tests_res.segment_sz,
> > > + tests_res.last_segment_sz);
> > > + printf(" • Number of ops: %u\n", tests_res.total_buffs);
> > > + printf(" • Total memory allocation: %u\n",
> > > + (tests_res.total_segments - 1) * tests_res.segment_sz
> > > + + tests_res.last_segment_sz);
> > > + if (tests_res.total_buffs > 1)
> > > + printf(" • %u ops: %u segments in each,"
> > > + " segment size %u\n",
> > > + tests_res.total_buffs - 1,
> > > + tests_res.segments_per_buff,
> > > + tests_res.segment_sz);
> > > + if (tests_res.segments_per_last_buff > 1) {
> > > + printf(" • 1 op %u segments:\n",
> > > + tests_res.segments_per_last_buff);
> > > + printf(" o %u segment size %u\n",
> > > + tests_res.segments_per_last_buff - 1,
> > > + tests_res.segment_sz);
> > > + printf(" o last segment size %u\n",
> > > + tests_res.last_segment_sz);
> > > + } else if (tests_res.segments_per_last_buff == 1) {
> > > + printf(" • 1 op (the last one): %u segment %u"
> > > + " byte(s) long\n\n",
> > > + tests_res.segments_per_last_buff,
> > > + tests_res.last_segment_sz);
> > > + }
> > > +}
> > >
> > > int
> > > main(int argc, char **argv)
> > > @@ -533,8 +622,9 @@ main(int argc, char **argv)
> > > else
> > > level = test_data->level.list[0];
> > >
> > > + print_report_header();
> > > +
> > [Shally] looks like we're printing input characteristics and possible
> > performance behavior. Is that the intention of this API?
> > [Artur] That was the idea to have a tool for verification how the data
> > partitioning affects the compression performance. Your description
> > Shally is very accurate: "... printing input characteristics and
> > possible performance behavior" and that is the intention of this API.
> [Shally] Then probably we can add this in function description.
> [Artur] To be precise it is not a typical API function. It's an internal (static)
> function of the perf test.
[Shally] Oh ya. Then can we change name .. print_report_header() sounds like printing report on test results but here intent is different. Something like print_test_dynamics() seem more applicable?
> >
> >
> > > printf("Burst size = %u\n", test_data->burst_sz);
> > > - printf("File size = %zu\n", test_data->input_data_sz);
> > >
> > > printf("%6s%12s%17s%19s%21s%15s%21s%23s%16s\n",
> > > "Level", "Comp size", "Comp ratio [%]", @@ -612,3 +702,4
> > @@
> > > main(int argc, char **argv)
> > > }
> > > return ret;
> > > }
> > > +
> > > --
> > > 2.17.1
next prev parent reply other threads:[~2019-06-27 14:39 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-03 10:24 [dpdk-dev] [PATCH] app/test-compress-perf: report header improvement Artur Trybula
2019-06-26 17:04 ` Shally Verma
2019-06-27 8:58 ` Trybula, ArturX
2019-06-27 11:00 ` Shally Verma
2019-06-27 11:46 ` Trybula, ArturX
2019-06-27 14:38 ` Shally Verma [this message]
2019-06-28 9:52 ` Trybula, ArturX
2019-07-05 6:30 ` [dpdk-dev] [PATCH v2 0/1] add extra features to test-compress-perf Artur Trybula
2019-07-05 6:30 ` [dpdk-dev] [PATCH v2 1/1] app/test-compress-perf: report header improvement Artur Trybula
2019-07-05 6:53 ` Jerin Jacob Kollanukkaran
2019-07-05 7:40 ` [dpdk-dev] [PATCH v3 0/1] add extra features to test-compress-perf Artur Trybula
2019-07-05 7:40 ` [dpdk-dev] [PATCH v3 1/1] app/test-compress-perf: report header improvement Artur Trybula
2019-07-12 10:43 ` [dpdk-dev] [PATCH v4 0/1] add extra features to test-compress-perf Artur Trybula
2019-07-12 10:43 ` [dpdk-dev] [PATCH v4 1/1] app/test-compress-perf: report header improvement Artur Trybula
2019-07-15 12:47 ` [dpdk-dev] [EXT] " Shally Verma
2019-07-15 14:01 ` Trybula, ArturX
2019-07-16 13:49 ` Trahe, Fiona
2019-07-17 7:21 ` Shally Verma
2019-07-17 11:02 ` [dpdk-dev] [PATCH v5 0/1] add extra features to test-compress-perf Artur Trybula
2019-07-17 11:02 ` [dpdk-dev] [PATCH v5 1/1] app/test-compress-perf: report header improvement Artur Trybula
2019-07-17 11:15 ` [dpdk-dev] [EXT] " Shally Verma
2019-07-17 11:26 ` Trybula, ArturX
2019-07-19 13:17 ` Akhil Goyal
2019-07-19 16:21 ` Thomas Monjalon
2019-07-19 17:37 ` Trahe, Fiona
2019-07-24 12:53 ` [dpdk-dev] [PATCH v6 0/1] add extra features to test-compress-perf Adam Dybkowski
2019-07-24 12:53 ` [dpdk-dev] [PATCH v6 1/1] app/test-compress-perf: provide more detailed report Adam Dybkowski
2019-07-24 13:55 ` [dpdk-dev] [PATCH v7 0/1] add extra features to test-compress-perf Adam Dybkowski
2019-07-24 13:55 ` [dpdk-dev] [PATCH v7 1/1] app/test-compress-perf: provide more detailed report Adam Dybkowski
2019-07-25 12:39 ` Trahe, Fiona
2019-07-26 12:41 ` Akhil Goyal
2019-07-17 11:16 ` [dpdk-dev] [EXT] [PATCH v5 0/1] add extra features to test-compress-perf Shally Verma
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BN6PR1801MB20525330D91140C460238EDDADFD0@BN6PR1801MB2052.namprd18.prod.outlook.com \
--to=shallyv@marvell.com \
--cc=adamx.dybkowski@intel.com \
--cc=arturx.trybula@intel.com \
--cc=dev@dpdk.org \
--cc=fiona.trahe@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).