From: Jinpu Wang <jinpu.wang@cloud.ionos.com>
To: Bart Van Assche <bvanassche@acm.org>
Cc: Jason Gunthorpe <jgg@ziepe.ca>, Jack Wang <jinpuwang@gmail.com>,
linux-block@vger.kernel.org, linux-rdma@vger.kernel.org,
Jens Axboe <axboe@kernel.dk>,
Christoph Hellwig <hch@infradead.org>,
Sagi Grimberg <sagi@grimberg.me>,
Leon Romanovsky <leon@kernel.org>,
Doug Ledford <dledford@redhat.com>,
Danil Kipnis <danil.kipnis@cloud.ionos.com>,
rpenyaev@suse.de
Subject: Re: [PATCH v5 00/25] RTRS (former IBTRS) rdma transport library and the corresponding RNBD (former IBNBD) rdma network block device
Date: Tue, 7 Jan 2020 11:56:01 +0100 [thread overview]
Message-ID: <CAMGffEm=1ai=vkxLV7Qp0z3M-98BDej4mpc4WnsOpSHUJWuPeA@mail.gmail.com> (raw)
In-Reply-To: <CAMGffEntYmFfGD8BGEtykoFXD_3DvNWQ8hrRmDoLtdr9ykTwug@mail.gmail.com>
On Mon, Jan 6, 2020 at 6:07 PM Jinpu Wang <jinpu.wang@cloud.ionos.com> wrote:
>
> On Fri, Jan 3, 2020 at 5:29 PM Bart Van Assche <bvanassche@acm.org> wrote:
> >
> > On 1/3/20 4:39 AM, Jinpu Wang wrote:
> > > Performance results for the v5.5-rc1 kernel are here:
> > > link: https://github.com/ionos-enterprise/ibnbd/tree/develop/performance/v5-v5.5-rc1
> > >
> > > Some workloads RNBD are faster, some workloads NVMeoF are faster.
> >
> > Thank you for having shared these graphs.
> >
> > Do the graphs in RNBD-SinglePath.pdf show that NVMeOF achieves similar
> > or higher IOPS, higher bandwidth and lower latency than RNBD for
> > workloads with a block size of 4 KB and also for mixed workloads with
> > less than 20 disks, whether or not invalidation is enabled for RNBD?
> Hi Bart,
>
> Yes, that's the result on one pair of Server with Connect X4 HCA, I
> did another test on another
> 2 servers with Connect X5 HCA, results are quite different, we will
> double-check the
> performance results also on old machines, will share new results later.
>
here are the results with ConnectX5 HCA MT4119 + Intel(R) Xeon(R) Gold
6130 CPU @ 2.10GHz, sorry no graph for now,
will prepare the next round.
disks 4k nvme dual 4k nvme single 4k rnbd dual 4k rnbd single
4k rnbd-inv dual 4k rnbd-inv single
x1 251637.436256 254312.068793 270886.311369 260934.306569
218632.336766 190800.519948
x2 460894.610539 463925.907409 496318.068193 466374.862514
418960.30397 372848.815118
x3 603263.673633 605004.49955 675073.892611 614552.144786
586223.077692 524221.977802
x4 731648.935106 733174.482552 850245.575442 743062.493751
744380.361964 656861.813819
x5 827732.326767 827444.855514 1026939.306069 840515.548445
897801.719828 762707.329267
x6 876705.329467 873963.0037 1142399.960004 876974.70253
1037773.522648 834073.892611
x7 893808.719128 893268.073193 1239282.471753 892728.027197
1135570.742926 871336.966303
x8 906589.741026 905938.006199 1287178.964207 906189.381062
1225040.9959 895292.070793
x9 912048.09519 912400.259974 1386211.878812 913885.311469
1302472.964884 910176.282372
x10 915566.243376 915602.739726 1442959.70403 916288.871113
1350296.325879 914966.40336
x11 917116.188381 916905.809419 1418574.942506 916264.373563
1370438.698083 915255.874413
x12 915852.814719 917710.128987 1423534.546545 916348.386452
1352357.364264 914966.656684
x13 919042.69573 918819.536093 1429697.830217 917631.036896
1378083.824558 916519.161677
x14 920000.49995 920031.59684 1443317.268273 917562.843716
1395023.56936 918935.706429
x15 920160.883912 920367.363264 1445306.425863 918278.472153
1440776.944611 916352.265921
x16 920652.869426 920673.832617 1454705.229477 917902.948526
1455708.2501 918198.001998
x17 916892.310769 916883.623275 1491071.841948 918936.706329
1436507.428457 917182.934132
x18 917247.775222 917762.523748 1612129.058036 918546.835949
1488716.583417 919521.095781
x19 920084.791521 920349.930014 1615690.87821 915371.496958
1406747.32954 918347.248577
x20 922339.232154 922208.058388 1591415.958404 917922.631526
1343806.744488 918903.393214
x21 923141.771646 923297.040592 1581547.169811 919371.025795
1342568.406347 919157.752944
x22 923063.787243 924072.385523 1574972.846162 919173.713143
1340318.639673 920577.995403
x23 923549.490102 924643.471306 1573597.440256 918705.060385
1333047.771337 917469.027431
x24 925584.483103 925224.955009 1578143.485651 921744.15117
1321494.708466 920001.498352
x25 926165.366927 926842.031594 1579288.271173 921845.392764
1319902.568202 920448.830702
x26 926852.729454 927382.123575 1585351.318945 922669.59912
1325670.791338 919137.796847
x27 928196.260748 928093.981204 1581427.557244 921379.155436
1325009.972078 919017.550858
x28 929330.433913 929606.778644 1581726.527347 924325.834833
1331721.074174 919557.761373
x29 929885.522895 929876.924615 1578317.436513 922977.240966
1333612.86783 921094.386736
x30 930635.972805 930599.520144 1583537.946205 922746.107784
1333446.651362 922821.171531
next prev parent reply other threads:[~2020-01-07 10:56 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-12-20 15:50 [PATCH v5 00/25] RTRS (former IBTRS) rdma transport library and the corresponding RNBD (former IBNBD) rdma network block device Jack Wang
2019-12-20 15:50 ` [PATCH v5 01/25] sysfs: export sysfs_remove_file_self() Jack Wang
2019-12-20 16:26 ` Jinpu Wang
2019-12-20 15:50 ` [PATCH v5 02/25] rtrs: public interface header to establish RDMA connections Jack Wang
2019-12-21 10:15 ` Leon Romanovsky
2019-12-21 14:27 ` Danil Kipnis
2019-12-22 7:36 ` Leon Romanovsky
2019-12-23 7:38 ` Jinpu Wang
2019-12-23 8:04 ` Leon Romanovsky
2019-12-23 10:31 ` Jinpu Wang
2019-12-20 15:50 ` [PATCH v5 03/25] rtrs: private headers with rtrs protocol structs and helpers Jack Wang
2019-12-20 15:50 ` [PATCH v5 04/25] rtrs: core: lib functions shared between client and server modules Jack Wang
2019-12-20 15:50 ` [PATCH v5 05/25] rtrs: client: private header with client structs and functions Jack Wang
2019-12-20 15:50 ` [PATCH v5 06/25] rtrs: client: main functionality Jack Wang
2019-12-20 15:50 ` [PATCH v5 07/25] rtrs: client: statistics functions Jack Wang
2019-12-20 15:50 ` [PATCH v5 08/25] rtrs: client: sysfs interface functions Jack Wang
2019-12-20 15:50 ` [PATCH v5 09/25] rtrs: server: private header with server structs and functions Jack Wang
2019-12-20 15:50 ` [PATCH v5 10/25] rtrs: server: main functionality Jack Wang
2019-12-20 15:50 ` [PATCH v5 11/25] rtrs: server: statistics functions Jack Wang
2019-12-20 15:50 ` [PATCH v5 12/25] rtrs: server: sysfs interface functions Jack Wang
2019-12-20 15:50 ` [PATCH v5 13/25] rtrs: include client and server modules into kernel compilation Jack Wang
2019-12-20 15:50 ` [PATCH v5 14/25] rtrs: a bit of documentation Jack Wang
2019-12-20 15:50 ` [PATCH v5 15/25] rnbd: private headers with rnbd protocol structs and helpers Jack Wang
2019-12-20 15:51 ` [PATCH v5 16/25] rnbd: client: private header with client structs and functions Jack Wang
2019-12-20 15:51 ` [PATCH v5 17/25] rnbd: client: main functionality Jack Wang
2019-12-20 15:51 ` [PATCH v5 18/25] rnbd: client: sysfs interface functions Jack Wang
2019-12-20 15:51 ` [PATCH v5 19/25] rnbd: server: private header with server structs and functions Jack Wang
2019-12-20 15:51 ` [PATCH v5 20/25] rnbd: server: main functionality Jack Wang
2019-12-20 15:51 ` [PATCH v5 21/25] rnbd: server: functionality for IO submission to file or block dev Jack Wang
2019-12-20 15:51 ` [PATCH v5 22/25] rnbd: server: sysfs interface functions Jack Wang
2019-12-23 8:14 ` Leon Romanovsky
2019-12-23 8:33 ` Jinpu Wang
2019-12-20 15:51 ` [PATCH v5 23/25] rnbd: include client and server modules into kernel compilation Jack Wang
2019-12-20 15:51 ` [PATCH v5 24/25] rnbd: a bit of documentation Jack Wang
2019-12-20 15:51 ` [PATCH v5 25/25] MAINTAINERS: Add maintainers for RNBD/RTRS modules Jack Wang
2019-12-22 9:55 ` Gal Pressman
2019-12-23 7:20 ` Jinpu Wang
2019-12-21 10:17 ` [PATCH v5 00/25] RTRS (former IBTRS) rdma transport library and the corresponding RNBD (former IBNBD) rdma network block device Leon Romanovsky
2020-01-02 18:18 ` Jason Gunthorpe
2020-01-03 12:39 ` Jinpu Wang
2020-01-03 16:28 ` Bart Van Assche
2020-01-06 17:07 ` Jinpu Wang
2020-01-07 10:56 ` Jinpu Wang [this message]
2020-01-16 16:41 ` Bart Van Assche
2020-01-16 16:46 ` Jinpu Wang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAMGffEm=1ai=vkxLV7Qp0z3M-98BDej4mpc4WnsOpSHUJWuPeA@mail.gmail.com' \
--to=jinpu.wang@cloud.ionos.com \
--cc=axboe@kernel.dk \
--cc=bvanassche@acm.org \
--cc=danil.kipnis@cloud.ionos.com \
--cc=dledford@redhat.com \
--cc=hch@infradead.org \
--cc=jgg@ziepe.ca \
--cc=jinpuwang@gmail.com \
--cc=leon@kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=rpenyaev@suse.de \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).