Linux-Block Archive on lore.kernel.org
 help / color / Atom feed
From: Jinpu Wang <jinpu.wang@cloud.ionos.com>
To: Bart Van Assche <bvanassche@acm.org>
Cc: Jason Gunthorpe <jgg@ziepe.ca>, Jack Wang <jinpuwang@gmail.com>,
	linux-block@vger.kernel.org, linux-rdma@vger.kernel.org,
	Jens Axboe <axboe@kernel.dk>,
	Christoph Hellwig <hch@infradead.org>,
	Sagi Grimberg <sagi@grimberg.me>,
	Leon Romanovsky <leon@kernel.org>,
	Doug Ledford <dledford@redhat.com>,
	Danil Kipnis <danil.kipnis@cloud.ionos.com>,
	rpenyaev@suse.de
Subject: Re: [PATCH v5 00/25] RTRS (former IBTRS) rdma transport library and the corresponding RNBD (former IBNBD) rdma network block device
Date: Tue, 7 Jan 2020 11:56:01 +0100
Message-ID: <CAMGffEm=1ai=vkxLV7Qp0z3M-98BDej4mpc4WnsOpSHUJWuPeA@mail.gmail.com> (raw)
In-Reply-To: <CAMGffEntYmFfGD8BGEtykoFXD_3DvNWQ8hrRmDoLtdr9ykTwug@mail.gmail.com>

On Mon, Jan 6, 2020 at 6:07 PM Jinpu Wang <jinpu.wang@cloud.ionos.com> wrote:
>
> On Fri, Jan 3, 2020 at 5:29 PM Bart Van Assche <bvanassche@acm.org> wrote:
> >
> > On 1/3/20 4:39 AM, Jinpu Wang wrote:
> > > Performance results for the v5.5-rc1 kernel are here:
> > >    link: https://github.com/ionos-enterprise/ibnbd/tree/develop/performance/v5-v5.5-rc1
> > >
> > > Some workloads RNBD are faster, some workloads NVMeoF are faster.
> >
> > Thank you for having shared these graphs.
> >
> > Do the graphs in RNBD-SinglePath.pdf show that NVMeOF achieves similar
> > or higher IOPS, higher bandwidth and lower latency than RNBD for
> > workloads with a block size of 4 KB and also for mixed workloads with
> > less than 20 disks, whether or not invalidation is enabled for RNBD?
> Hi Bart,
>
> Yes, that's the result on one pair of Server with Connect X4 HCA, I
> did another test on another
> 2 servers with Connect X5 HCA, results are quite different, we will
> double-check the
> performance results also on old machines, will share new results later.
>
here are the results with ConnectX5 HCA MT4119 + Intel(R) Xeon(R) Gold
6130 CPU @ 2.10GHz, sorry no graph for now,
will prepare the next round.

 disks   4k nvme dual  4k nvme single    4k rnbd dual  4k rnbd single
4k rnbd-inv dual  4k rnbd-inv single
   x1  251637.436256   254312.068793   270886.311369   260934.306569
  218632.336766       190800.519948
   x2  460894.610539   463925.907409   496318.068193   466374.862514
   418960.30397       372848.815118
   x3  603263.673633    605004.49955   675073.892611   614552.144786
  586223.077692       524221.977802
   x4  731648.935106   733174.482552   850245.575442   743062.493751
  744380.361964       656861.813819
   x5  827732.326767   827444.855514  1026939.306069   840515.548445
  897801.719828       762707.329267
   x6  876705.329467     873963.0037  1142399.960004    876974.70253
 1037773.522648       834073.892611
   x7  893808.719128   893268.073193  1239282.471753   892728.027197
 1135570.742926       871336.966303
   x8  906589.741026   905938.006199  1287178.964207   906189.381062
   1225040.9959       895292.070793
   x9   912048.09519   912400.259974  1386211.878812   913885.311469
 1302472.964884       910176.282372
  x10  915566.243376   915602.739726   1442959.70403   916288.871113
 1350296.325879        914966.40336
  x11  917116.188381   916905.809419  1418574.942506   916264.373563
 1370438.698083       915255.874413
  x12  915852.814719   917710.128987  1423534.546545   916348.386452
 1352357.364264       914966.656684
  x13   919042.69573   918819.536093  1429697.830217   917631.036896
 1378083.824558       916519.161677
  x14   920000.49995    920031.59684  1443317.268273   917562.843716
  1395023.56936       918935.706429
  x15  920160.883912   920367.363264  1445306.425863   918278.472153
 1440776.944611       916352.265921
  x16  920652.869426   920673.832617  1454705.229477   917902.948526
   1455708.2501       918198.001998
  x17  916892.310769   916883.623275  1491071.841948   918936.706329
 1436507.428457       917182.934132
  x18  917247.775222   917762.523748  1612129.058036   918546.835949
 1488716.583417       919521.095781
  x19  920084.791521   920349.930014   1615690.87821   915371.496958
  1406747.32954       918347.248577
  x20  922339.232154   922208.058388  1591415.958404   917922.631526
 1343806.744488       918903.393214
  x21  923141.771646   923297.040592  1581547.169811   919371.025795
 1342568.406347       919157.752944
  x22  923063.787243   924072.385523  1574972.846162   919173.713143
 1340318.639673       920577.995403
  x23  923549.490102   924643.471306  1573597.440256   918705.060385
 1333047.771337       917469.027431
  x24  925584.483103   925224.955009  1578143.485651    921744.15117
 1321494.708466       920001.498352
  x25  926165.366927   926842.031594  1579288.271173   921845.392764
 1319902.568202       920448.830702
  x26  926852.729454   927382.123575  1585351.318945    922669.59912
 1325670.791338       919137.796847
  x27  928196.260748   928093.981204  1581427.557244   921379.155436
 1325009.972078       919017.550858
  x28  929330.433913   929606.778644  1581726.527347   924325.834833
 1331721.074174       919557.761373
  x29  929885.522895   929876.924615  1578317.436513   922977.240966
  1333612.86783       921094.386736
  x30  930635.972805   930599.520144  1583537.946205   922746.107784
 1333446.651362       922821.171531

  reply index

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-20 15:50 Jack Wang
2019-12-20 15:50 ` [PATCH v5 01/25] sysfs: export sysfs_remove_file_self() Jack Wang
2019-12-20 16:26   ` Jinpu Wang
2019-12-20 15:50 ` [PATCH v5 02/25] rtrs: public interface header to establish RDMA connections Jack Wang
2019-12-21 10:15   ` Leon Romanovsky
2019-12-21 14:27     ` Danil Kipnis
2019-12-22  7:36       ` Leon Romanovsky
2019-12-23  7:38         ` Jinpu Wang
2019-12-23  8:04           ` Leon Romanovsky
2019-12-23 10:31             ` Jinpu Wang
2019-12-20 15:50 ` [PATCH v5 03/25] rtrs: private headers with rtrs protocol structs and helpers Jack Wang
2019-12-20 15:50 ` [PATCH v5 04/25] rtrs: core: lib functions shared between client and server modules Jack Wang
2019-12-20 15:50 ` [PATCH v5 05/25] rtrs: client: private header with client structs and functions Jack Wang
2019-12-20 15:50 ` [PATCH v5 06/25] rtrs: client: main functionality Jack Wang
2019-12-20 15:50 ` [PATCH v5 07/25] rtrs: client: statistics functions Jack Wang
2019-12-20 15:50 ` [PATCH v5 08/25] rtrs: client: sysfs interface functions Jack Wang
2019-12-20 15:50 ` [PATCH v5 09/25] rtrs: server: private header with server structs and functions Jack Wang
2019-12-20 15:50 ` [PATCH v5 10/25] rtrs: server: main functionality Jack Wang
2019-12-20 15:50 ` [PATCH v5 11/25] rtrs: server: statistics functions Jack Wang
2019-12-20 15:50 ` [PATCH v5 12/25] rtrs: server: sysfs interface functions Jack Wang
2019-12-20 15:50 ` [PATCH v5 13/25] rtrs: include client and server modules into kernel compilation Jack Wang
2019-12-20 15:50 ` [PATCH v5 14/25] rtrs: a bit of documentation Jack Wang
2019-12-20 15:50 ` [PATCH v5 15/25] rnbd: private headers with rnbd protocol structs and helpers Jack Wang
2019-12-20 15:51 ` [PATCH v5 16/25] rnbd: client: private header with client structs and functions Jack Wang
2019-12-20 15:51 ` [PATCH v5 17/25] rnbd: client: main functionality Jack Wang
2019-12-20 15:51 ` [PATCH v5 18/25] rnbd: client: sysfs interface functions Jack Wang
2019-12-20 15:51 ` [PATCH v5 19/25] rnbd: server: private header with server structs and functions Jack Wang
2019-12-20 15:51 ` [PATCH v5 20/25] rnbd: server: main functionality Jack Wang
2019-12-20 15:51 ` [PATCH v5 21/25] rnbd: server: functionality for IO submission to file or block dev Jack Wang
2019-12-20 15:51 ` [PATCH v5 22/25] rnbd: server: sysfs interface functions Jack Wang
2019-12-23  8:14   ` Leon Romanovsky
2019-12-23  8:33     ` Jinpu Wang
2019-12-20 15:51 ` [PATCH v5 23/25] rnbd: include client and server modules into kernel compilation Jack Wang
2019-12-20 15:51 ` [PATCH v5 24/25] rnbd: a bit of documentation Jack Wang
2019-12-20 15:51 ` [PATCH v5 25/25] MAINTAINERS: Add maintainers for RNBD/RTRS modules Jack Wang
2019-12-22  9:55   ` Gal Pressman
2019-12-23  7:20     ` Jinpu Wang
2019-12-21 10:17 ` [PATCH v5 00/25] RTRS (former IBTRS) rdma transport library and the corresponding RNBD (former IBNBD) rdma network block device Leon Romanovsky
2020-01-02 18:18 ` Jason Gunthorpe
2020-01-03 12:39   ` Jinpu Wang
2020-01-03 16:28     ` Bart Van Assche
2020-01-06 17:07       ` Jinpu Wang
2020-01-07 10:56         ` Jinpu Wang [this message]
2020-01-16 16:41           ` Bart Van Assche
2020-01-16 16:46             ` Jinpu Wang

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAMGffEm=1ai=vkxLV7Qp0z3M-98BDej4mpc4WnsOpSHUJWuPeA@mail.gmail.com' \
    --to=jinpu.wang@cloud.ionos.com \
    --cc=axboe@kernel.dk \
    --cc=bvanassche@acm.org \
    --cc=danil.kipnis@cloud.ionos.com \
    --cc=dledford@redhat.com \
    --cc=hch@infradead.org \
    --cc=jgg@ziepe.ca \
    --cc=jinpuwang@gmail.com \
    --cc=leon@kernel.org \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=rpenyaev@suse.de \
    --cc=sagi@grimberg.me \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-Block Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-block/0 linux-block/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-block linux-block/ https://lore.kernel.org/linux-block \
		linux-block@vger.kernel.org
	public-inbox-index linux-block

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-block


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git