From: Jinpu Wang <jinpu.wang@ionos.com>
To: Peter Xu <peterx@redhat.com>
Cc: "Daniel P. Berrangé" <berrange@redhat.com>,
"Markus Armbruster" <armbru@redhat.com>,
"Michael Galaxy" <mgalaxy@akamai.com>,
"Yu Zhang" <yu.zhang@ionos.com>,
"Zhijian Li (Fujitsu)" <lizhijian@fujitsu.com>,
"Elmar Gerdes" <elmar.gerdes@ionos.com>,
"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
"Yuval Shaia" <yuval.shaia.ml@gmail.com>,
"Kevin Wolf" <kwolf@redhat.com>,
"Prasanna Kumar Kalever" <prasanna.kalever@redhat.com>,
"Cornelia Huck" <cohuck@redhat.com>,
"Michael Roth" <michael.roth@amd.com>,
"Prasanna Kumar Kalever" <prasanna4324@gmail.com>,
"integration@gluster.org" <integration@gluster.org>,
"Paolo Bonzini" <pbonzini@redhat.com>,
"qemu-block@nongnu.org" <qemu-block@nongnu.org>,
"devel@lists.libvirt.org" <devel@lists.libvirt.org>,
"Hanna Reitz" <hreitz@redhat.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
"Thomas Huth" <thuth@redhat.com>,
"Eric Blake" <eblake@redhat.com>,
"Song Gao" <gaosong@loongson.cn>,
"Marc-André Lureau" <marcandre.lureau@redhat.com>,
"Alex Bennée" <alex.bennee@linaro.org>,
"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
"Beraldo Leal" <bleal@redhat.com>,
arei.gonglei@huawei.com, pannengyuan@huawei.com
Subject: Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling
Date: Tue, 7 May 2024 06:52:50 +0200 [thread overview]
Message-ID: <CAMGffE=Hcep90DaoJDmKY6ESMtr1fZHehv-UrcHJHA8b2KvjBw@mail.gmail.com> (raw)
In-Reply-To: <Zjj3GXsaUyCjjUnC@x1n>
Hi Peter, hi Daniel,
On Mon, May 6, 2024 at 5:29 PM Peter Xu <peterx@redhat.com> wrote:
>
> On Mon, May 06, 2024 at 12:08:43PM +0200, Jinpu Wang wrote:
> > Hi Peter, hi Daniel,
>
> Hi, Jinpu,
>
> Thanks for sharing this test results. Sounds like a great news.
>
> What's your plan next? Would it then be worthwhile / possible moving QEMU
> into that direction? Would that greatly simplify rdma code as Dan
> mentioned?
I'm rather not familiar with QEMU migration yet, from the test
result, I think it's a possible direction,
just we need to at least based on a rather recent release like
rdma-core v33 with proper 'fork' support.
Maybe Dan or you could give more detail about what you have in mind
for using rsocket as a replacement for the future.
We will also look into the implementation details in the meantime.
Thx!
J
>
> Thanks,
>
> >
> > On Fri, May 3, 2024 at 4:33 PM Peter Xu <peterx@redhat.com> wrote:
> > >
> > > On Fri, May 03, 2024 at 08:40:03AM +0200, Jinpu Wang wrote:
> > > > I had a brief check in the rsocket changelog, there seems some
> > > > improvement over time,
> > > > might be worth revisiting this. due to socket abstraction, we can't
> > > > use some feature like
> > > > ODP, it won't be a small and easy task.
> > >
> > > It'll be good to know whether Dan's suggestion would work first, without
> > > rewritting everything yet so far. Not sure whether some perf test could
> > > help with the rsocket APIs even without QEMU's involvements (or looking for
> > > test data supporting / invalidate such conversions).
> > >
> > I did a quick test with iperf on 100 G environment and 40 G
> > environment, in summary rsocket works pretty well.
> >
> > iperf tests between 2 hosts with 40 G (IB),
> > first a few test with different num. of threads on top of ipoib
> > interface, later with preload rsocket on top of same ipoib interface.
> >
> > jwang@ps401a-914.nst:~$ iperf -p 52000 -c 10.43.3.145
> > ------------------------------------------------------------
> > Client connecting to 10.43.3.145, TCP port 52000
> > TCP window size: 165 KByte (default)
> > ------------------------------------------------------------
> > [ 3] local 10.43.3.146 port 55602 connected with 10.43.3.145 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 3] 0.0000-10.0001 sec 2.85 GBytes 2.44 Gbits/sec
> > jwang@ps401a-914.nst:~$ iperf -p 52000 -c 10.43.3.145 -P 2
> > ------------------------------------------------------------
> > Client connecting to 10.43.3.145, TCP port 52000
> > TCP window size: 165 KByte (default)
> > ------------------------------------------------------------
> > [ 4] local 10.43.3.146 port 39640 connected with 10.43.3.145 port 52000
> > [ 3] local 10.43.3.146 port 39626 connected with 10.43.3.145 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 3] 0.0000-10.0012 sec 2.85 GBytes 2.45 Gbits/sec
> > [ 4] 0.0000-10.0026 sec 2.86 GBytes 2.45 Gbits/sec
> > [SUM] 0.0000-10.0026 sec 5.71 GBytes 4.90 Gbits/sec
> > [ CT] final connect times (min/avg/max/stdev) =
> > 0.281/0.300/0.318/0.318 ms (tot/err) = 2/0
> > jwang@ps401a-914.nst:~$ iperf -p 52000 -c 10.43.3.145 -P 4
> > ------------------------------------------------------------
> > Client connecting to 10.43.3.145, TCP port 52000
> > TCP window size: 165 KByte (default)
> > ------------------------------------------------------------
> > [ 4] local 10.43.3.146 port 46956 connected with 10.43.3.145 port 52000
> > [ 6] local 10.43.3.146 port 46978 connected with 10.43.3.145 port 52000
> > [ 3] local 10.43.3.146 port 46944 connected with 10.43.3.145 port 52000
> > [ 5] local 10.43.3.146 port 46962 connected with 10.43.3.145 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 3] 0.0000-10.0017 sec 2.85 GBytes 2.45 Gbits/sec
> > [ 4] 0.0000-10.0015 sec 2.85 GBytes 2.45 Gbits/sec
> > [ 5] 0.0000-10.0026 sec 2.85 GBytes 2.45 Gbits/sec
> > [ 6] 0.0000-10.0005 sec 2.85 GBytes 2.45 Gbits/sec
> > [SUM] 0.0000-10.0005 sec 11.4 GBytes 9.80 Gbits/sec
> > [ CT] final connect times (min/avg/max/stdev) =
> > 0.274/0.312/0.360/0.212 ms (tot/err) = 4/0
> > jwang@ps401a-914.nst:~$ iperf -p 52000 -c 10.43.3.145 -P 8
> > ------------------------------------------------------------
> > Client connecting to 10.43.3.145, TCP port 52000
> > TCP window size: 165 KByte (default)
> > ------------------------------------------------------------
> > [ 7] local 10.43.3.146 port 35062 connected with 10.43.3.145 port 52000
> > [ 6] local 10.43.3.146 port 35058 connected with 10.43.3.145 port 52000
> > [ 8] local 10.43.3.146 port 35066 connected with 10.43.3.145 port 52000
> > [ 9] local 10.43.3.146 port 35074 connected with 10.43.3.145 port 52000
> > [ 3] local 10.43.3.146 port 35038 connected with 10.43.3.145 port 52000
> > [ 12] local 10.43.3.146 port 35088 connected with 10.43.3.145 port 52000
> > [ 5] local 10.43.3.146 port 35048 connected with 10.43.3.145 port 52000
> > [ 4] local 10.43.3.146 port 35050 connected with 10.43.3.145 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 4] 0.0000-10.0005 sec 2.85 GBytes 2.44 Gbits/sec
> > [ 8] 0.0000-10.0011 sec 2.85 GBytes 2.45 Gbits/sec
> > [ 5] 0.0000-10.0000 sec 2.85 GBytes 2.45 Gbits/sec
> > [ 12] 0.0000-10.0021 sec 2.85 GBytes 2.44 Gbits/sec
> > [ 3] 0.0000-10.0003 sec 2.85 GBytes 2.44 Gbits/sec
> > [ 7] 0.0000-10.0065 sec 2.50 GBytes 2.14 Gbits/sec
> > [ 9] 0.0000-10.0077 sec 2.52 GBytes 2.16 Gbits/sec
> > [ 6] 0.0000-10.0003 sec 2.85 GBytes 2.44 Gbits/sec
> > [SUM] 0.0000-10.0003 sec 22.1 GBytes 19.0 Gbits/sec
> > [ CT] final connect times (min/avg/max/stdev) =
> > 0.096/0.226/0.339/0.109 ms (tot/err) = 8/0
> > jwang@ps401a-914.nst:~$ iperf -p 52000 -c 10.43.3.145 -P 16
> > [ 3] local 10.43.3.146 port 49540 connected with 10.43.3.145 port 52000
> > ------------------------------------------------------------
> > Client connecting to 10.43.3.145, TCP port 52000
> > TCP window size: 165 KByte (default)
> > ------------------------------------------------------------
> > [ 6] local 10.43.3.146 port 49554 connected with 10.43.3.145 port 52000
> > [ 8] local 10.43.3.146 port 49584 connected with 10.43.3.145 port 52000
> > [ 5] local 10.43.3.146 port 49552 connected with 10.43.3.145 port 52000
> > [ 20] local 10.43.3.146 port 49626 connected with 10.43.3.145 port 52000
> > [ 4] local 10.43.3.146 port 49606 connected with 10.43.3.145 port 52000
> > [ 9] local 10.43.3.146 port 49596 connected with 10.43.3.145 port 52000
> > [ 10] local 10.43.3.146 port 49604 connected with 10.43.3.145 port 52000
> > [ 26] local 10.43.3.146 port 49678 connected with 10.43.3.145 port 52000
> > [ 7] local 10.43.3.146 port 49556 connected with 10.43.3.145 port 52000
> > [ 25] local 10.43.3.146 port 49662 connected with 10.43.3.145 port 52000
> > [ 22] local 10.43.3.146 port 49636 connected with 10.43.3.145 port 52000
> > [ 11] local 10.43.3.146 port 49612 connected with 10.43.3.145 port 52000
> > [ 13] local 10.43.3.146 port 49618 connected with 10.43.3.145 port 52000
> > [ 23] local 10.43.3.146 port 49646 connected with 10.43.3.145 port 52000
> > [ 15] local 10.43.3.146 port 49688 connected with 10.43.3.145 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 11] 0.0000-10.0024 sec 2.28 GBytes 1.95 Gbits/sec
> > [ 23] 0.0000-10.0022 sec 2.28 GBytes 1.95 Gbits/sec
> > [ 20] 0.0000-10.0010 sec 2.28 GBytes 1.95 Gbits/sec
> > [ 8] 0.0000-10.0032 sec 2.28 GBytes 1.95 Gbits/sec
> > [ 26] 0.0000-10.0038 sec 2.28 GBytes 1.95 Gbits/sec
> > [ 10] 0.0000-10.0002 sec 2.28 GBytes 1.95 Gbits/sec
> > [ 7] 0.0000-10.0033 sec 2.28 GBytes 1.95 Gbits/sec
> > [ 15] 0.0000-10.0015 sec 2.27 GBytes 1.95 Gbits/sec
> > [ 4] 0.0000-10.0028 sec 2.28 GBytes 1.95 Gbits/sec
> > [ 6] 0.0000-10.0012 sec 2.28 GBytes 1.96 Gbits/sec
> > [ 13] 0.0000-10.0030 sec 2.28 GBytes 1.95 Gbits/sec
> > [ 25] 0.0000-10.0051 sec 2.28 GBytes 1.95 Gbits/sec
> > [ 5] 0.0000-10.0001 sec 2.28 GBytes 1.96 Gbits/sec
> > [ 9] 0.0000-10.0017 sec 2.28 GBytes 1.95 Gbits/sec
> > [ 22] 0.0000-10.0008 sec 2.27 GBytes 1.95 Gbits/sec
> > [ 3] 0.0000-10.0033 sec 2.28 GBytes 1.95 Gbits/sec
> > [SUM] 0.0000-10.0034 sec 36.4 GBytes 31.3 Gbits/sec
> > [ CT] final connect times (min/avg/max/stdev) =
> > 0.105/0.217/0.401/0.093 ms (tot/err) = 16/0
> > jwang@ps401a-914.nst:~$
> > LD_PRELOAD=/usr/lib/x86_64-linux-gnu/rsocket/librspreload.so iperf -p
> > 52000 -c 10.43.3.145 -P 16
> > ------------------------------------------------------------
> > Client connecting to 10.43.3.145, TCP port 52000
> > TCP window size: 128 KByte (default)
> > ------------------------------------------------------------
> > [ 3] local 10.43.3.146 port 48902 connected with 10.43.3.145 port 52000
> > [ 5] local 10.43.3.146 port 52777 connected with 10.43.3.145 port 52000
> > [ 9] local 10.43.3.146 port 42911 connected with 10.43.3.145 port 52000
> > [ 11] local 10.43.3.146 port 56354 connected with 10.43.3.145 port 52000
> > [ 15] local 10.43.3.146 port 43325 connected with 10.43.3.145 port 52000
> > [ 6] local 10.43.3.146 port 37041 connected with 10.43.3.145 port 52000
> > [ 7] local 10.43.3.146 port 58828 connected with 10.43.3.145 port 52000
> > [ 17] local 10.43.3.146 port 48858 connected with 10.43.3.145 port 52000
> > [ 13] local 10.43.3.146 port 49256 connected with 10.43.3.145 port 52000
> > [ 16] local 10.43.3.146 port 35652 connected with 10.43.3.145 port 52000
> > [ 8] local 10.43.3.146 port 48567 connected with 10.43.3.145 port 52000
> > [ 18] local 10.43.3.146 port 47394 connected with 10.43.3.145 port 52000
> > [ 19] local 10.43.3.146 port 48065 connected with 10.43.3.145 port 52000
> > [ 10] local 10.43.3.146 port 39788 connected with 10.43.3.145 port 52000
> > [ 4] local 10.43.3.146 port 46818 connected with 10.43.3.145 port 52000
> > [ 14] local 10.43.3.146 port 57174 connected with 10.43.3.145 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 14] 0.0000-10.0002 sec 2.30 GBytes 1.98 Gbits/sec
> > [ 6] 0.0000-10.0004 sec 2.31 GBytes 1.98 Gbits/sec
> > [ 5] 0.0000-10.0002 sec 2.31 GBytes 1.98 Gbits/sec
> > [ 8] 0.0000-10.0001 sec 2.31 GBytes 1.98 Gbits/sec
> > [ 11] 0.0000-10.0003 sec 2.31 GBytes 1.98 Gbits/sec
> > [ 18] 0.0000-10.0003 sec 2.31 GBytes 1.98 Gbits/sec
> > [ 3] 0.0000-10.0004 sec 2.31 GBytes 1.98 Gbits/sec
> > [ 4] 0.0000-10.0005 sec 2.30 GBytes 1.98 Gbits/sec
> > [ 17] 0.0000-10.0004 sec 2.31 GBytes 1.98 Gbits/sec
> > [ 15] 0.0000-10.0005 sec 2.31 GBytes 1.98 Gbits/sec
> > [ 19] 0.0000-10.0001 sec 2.30 GBytes 1.98 Gbits/sec
> > [ 7] 0.0000-10.0004 sec 2.31 GBytes 1.98 Gbits/sec
> > [ 13] 0.0000-10.0005 sec 2.31 GBytes 1.98 Gbits/sec
> > [ 10] 0.0000-10.0003 sec 2.30 GBytes 1.98 Gbits/sec
> > [ 9] 0.0000-10.0000 sec 2.31 GBytes 1.98 Gbits/sec
> > [ 16] 0.0000-10.0002 sec 2.31 GBytes 1.98 Gbits/sec
> > [SUM] 0.0000-10.0003 sec 36.9 GBytes 31.7 Gbits/sec
> > [ CT] final connect times (min/avg/max/stdev) =
> > 88.398/101.706/114.726/24.755 ms (tot/err) = 16/0
> > jwang@ps401a-914.nst:~$
> > LD_PRELOAD=/usr/lib/x86_64-linux-gnu/rsocket/librspreload.so iperf -p
> > 52000 -c 10.43.3.145 -P 1
> > ------------------------------------------------------------
> > Client connecting to 10.43.3.145, TCP port 52000
> > TCP window size: 128 KByte (default)
> > ------------------------------------------------------------
> > [ 3] local 10.43.3.146 port 49168 connected with 10.43.3.145 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 3] 0.0000-10.0000 sec 34.3 GBytes 29.5 Gbits/sec
> > jwang@ps401a-914.nst:~$
> > LD_PRELOAD=/usr/lib/x86_64-linux-gnu/rsocket/librspreload.so iperf -p
> > 52000 -c 10.43.3.145 -P 2
> > ------------------------------------------------------------
> > Client connecting to 10.43.3.145, TCP port 52000
> > TCP window size: 128 KByte (default)
> > ------------------------------------------------------------
> > [ 3] local 10.43.3.146 port 42096 connected with 10.43.3.145 port 52000
> > [ 4] local 10.43.3.146 port 58667 connected with 10.43.3.145 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 4] 0.0000-10.0001 sec 18.4 GBytes 15.8 Gbits/sec
> > [ 3] 0.0000-10.0000 sec 18.5 GBytes 15.9 Gbits/sec
> > [SUM] 0.0000-10.0001 sec 36.9 GBytes 31.7 Gbits/sec
> > [ CT] final connect times (min/avg/max/stdev) =
> > 38.155/38.997/39.839/39.839 ms (tot/err) = 2/0
> > jwang@ps401a-914.nst:~$
> > LD_PRELOAD=/usr/lib/x86_64-linux-gnu/rsocket/librspreload.so iperf -p
> > 52000 -c 10.43.3.145 -P 4
> > ------------------------------------------------------------
> > Client connecting to 10.43.3.145, TCP port 52000
> > TCP window size: 128 KByte (default)
> > ------------------------------------------------------------
> > [ 3] local 10.43.3.146 port 36100 connected with 10.43.3.145 port 52000
> > [ 5] local 10.43.3.146 port 55108 connected with 10.43.3.145 port 52000
> > [ 6] local 10.43.3.146 port 41039 connected with 10.43.3.145 port 52000
> > [ 7] local 10.43.3.146 port 34868 connected with 10.43.3.145 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 7] 0.0000-10.0000 sec 9.22 GBytes 7.92 Gbits/sec
> > [ 5] 0.0000-10.0000 sec 9.22 GBytes 7.92 Gbits/sec
> > [ 3] 0.0000-10.0000 sec 9.22 GBytes 7.92 Gbits/sec
> > [ 6] 0.0000-10.0001 sec 9.22 GBytes 7.92 Gbits/sec
> > [SUM] 0.0000-10.0001 sec 36.9 GBytes 31.7 Gbits/sec
> > [ CT] final connect times (min/avg/max/stdev) =
> > 51.401/53.644/56.015/30.487 ms (tot/err) = 4/0
> >
> > You can see with rsocket it reaches ~ 30 Gb/s with single stream,
> > while ipoib only 2.5 Gb/s (12 X), ipoib scales with more threads until
> > ~ 32 Gb/s, which is the link limit.
> >
> > With 100 G env, rsocket also out perform ipoib, see below:
> >
> >
> > jwang@ps404a-59.stg:~$
> > LD_PRELOAD=/usr/lib/x86_64-linux-gnu/rsocket/librspreload.so iperf -p
> > 52000 -c 10.43.48.58
> > ------------------------------------------------------------
> > Client connecting to 10.43.48.58, TCP port 52000
> > TCP window size: 128 KByte (default)
> > ------------------------------------------------------------
> > [ 3] local 10.43.48.59 port 40588 connected with 10.43.48.58 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 3] 0.0000-10.0000 sec 80.7 GBytes 69.4 Gbits/sec
> > jwang@ps404a-59.stg:~$
> > LD_PRELOAD=/usr/lib/x86_64-linux-gnu/rsocket/librspreload.so iperf -p
> > 52000 -c 10.43.48.58 -P 2
> > ------------------------------------------------------------
> > Client connecting to 10.43.48.58, TCP port 52000
> > TCP window size: 128 KByte (default)
> > ------------------------------------------------------------
> > [ 3] local 10.43.48.59 port 41813 connected with 10.43.48.58 port 52000
> > [ 5] local 10.43.48.59 port 60638 connected with 10.43.48.58 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 5] 0.0000-10.0000 sec 48.9 GBytes 42.0 Gbits/sec
> > [ 3] 0.0000-10.0000 sec 49.8 GBytes 42.8 Gbits/sec
> > [SUM] 0.0000-10.0000 sec 98.7 GBytes 84.8 Gbits/sec
> > [ CT] final connect times (min/avg/max/stdev) =
> > 6.962/7.764/8.567/8.567 ms (tot/err) = 2/0
> > jwang@ps404a-59.stg:~$
> > LD_PRELOAD=/usr/lib/x86_64-linux-gnu/rsocket/librspreload.so iperf -p
> > 52000 -c 10.43.48.58 -P 4
> > [ 6] local 10.43.48.59 port 58086 connected with 10.43.48.58 port 52000
> > [ 3] local 10.43.48.59 port 49335 connected with 10.43.48.58 port 52000
> > ------------------------------------------------------------
> > Client connecting to 10.43.48.58, TCP port 52000
> > TCP window size: 128 KByte (default)
> > ------------------------------------------------------------
> > [ 4] local 10.43.48.59 port 44593 connected with 10.43.48.58 port 52000
> > [ 5] local 10.43.48.59 port 60464 connected with 10.43.48.58 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 5] 0.0000-10.0000 sec 28.0 GBytes 24.0 Gbits/sec
> > [ 4] 0.0000-10.0000 sec 28.0 GBytes 24.0 Gbits/sec
> > [ 3] 0.0000-10.0000 sec 28.0 GBytes 24.1 Gbits/sec
> > [ 6] 0.0000-10.0000 sec 28.0 GBytes 24.1 Gbits/sec
> > [SUM] 0.0000-10.0001 sec 112 GBytes 96.3 Gbits/sec
> > [ CT] final connect times (min/avg/max/stdev) =
> > 7.344/9.619/12.199/5.271 ms (tot/err) = 4/0
> > jwang@ps404a-59.stg:~$
> > LD_PRELOAD=/usr/lib/x86_64-linux-gnu/rsocket/librspreload.so iperf -p
> > 52000 -c 10.43.48.58 -P 8
> > [ 3] local 10.43.48.59 port 43020 connected with 10.43.48.58 port 52000
> > [ 7] local 10.43.48.59 port 59720 connected with 10.43.48.58 port 52000
> > [ 4] local 10.43.48.59 port 52547 connected with 10.43.48.58 port 52000
> > [ 8] local 10.43.48.59 port 41712 connected with 10.43.48.58 port 52000
> > [ 10] local 10.43.48.59 port 53126 connected with 10.43.48.58 port 52000
> > ------------------------------------------------------------
> > Client connecting to 10.43.48.58, TCP port 52000
> > TCP window size: 128 KByte (default)
> > ------------------------------------------------------------
> > [ 6] local 10.43.48.59 port 60311 connected with 10.43.48.58 port 52000
> > [ 5] local 10.43.48.59 port 44103 connected with 10.43.48.58 port 52000
> > [ 9] local 10.43.48.59 port 49007 connected with 10.43.48.58 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 9] 0.0000-10.0001 sec 14.0 GBytes 12.0 Gbits/sec
> > [ 8] 0.0000-10.0000 sec 14.0 GBytes 12.0 Gbits/sec
> > [ 4] 0.0000-10.0001 sec 14.0 GBytes 12.0 Gbits/sec
> > [ 6] 0.0000-10.0000 sec 14.0 GBytes 12.0 Gbits/sec
> > [ 10] 0.0000-10.0000 sec 14.0 GBytes 12.0 Gbits/sec
> > [ 7] 0.0000-10.0000 sec 14.0 GBytes 12.0 Gbits/sec
> > [ 5] 0.0000-10.0001 sec 14.0 GBytes 12.0 Gbits/sec
> > [ 3] 0.0000-10.0001 sec 14.0 GBytes 12.0 Gbits/sec
> > [SUM] 0.0000-10.0001 sec 112 GBytes 96.3 Gbits/sec
> > [ CT] final connect times (min/avg/max/stdev) =
> > 6.942/12.361/18.109/4.872 ms (tot/err) = 8/0
> > jwang@ps404a-59.stg:~$ iperf -p 52000 -c 10.43.48.58 -P 8
> > ------------------------------------------------------------
> > Client connecting to 10.43.48.58, TCP port 52000
> > TCP window size: 165 KByte (default)
> > ------------------------------------------------------------
> > [ 4] local 10.43.48.59 port 58176 connected with 10.43.48.58 port 52000
> > [ 5] local 10.43.48.59 port 58180 connected with 10.43.48.58 port 52000
> > [ 3] local 10.43.48.59 port 58178 connected with 10.43.48.58 port 52000
> > [ 10] local 10.43.48.59 port 58226 connected with 10.43.48.58 port 52000
> > [ 11] local 10.43.48.59 port 58228 connected with 10.43.48.58 port 52000
> > [ 9] local 10.43.48.59 port 58212 connected with 10.43.48.58 port 52000
> > [ 7] local 10.43.48.59 port 58194 connected with 10.43.48.58 port 52000
> > [ 8] local 10.43.48.59 port 58198 connected with 10.43.48.58 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 9] 0.0000-10.0005 sec 15.8 GBytes 13.5 Gbits/sec
> > [ 4] 0.0000-10.0002 sec 15.8 GBytes 13.6 Gbits/sec
> > [ 3] 0.0000-10.0000 sec 15.8 GBytes 13.6 Gbits/sec
> > [ 5] 0.0000-10.0002 sec 15.8 GBytes 13.6 Gbits/sec
> > [ 8] 0.0000-10.0005 sec 7.89 GBytes 6.78 Gbits/sec
> > [ 10] 0.0000-10.0000 sec 15.8 GBytes 13.6 Gbits/sec
> > [ 11] 0.0000-10.0014 sec 7.94 GBytes 6.82 Gbits/sec
> > [ 7] 0.0000-10.0009 sec 15.8 GBytes 13.6 Gbits/sec
> > [SUM] 0.0000-10.0009 sec 111 GBytes 95.1 Gbits/sec
> > [ CT] final connect times (min/avg/max/stdev) =
> > 0.234/0.325/0.406/0.155 ms (tot/err) = 8/0
> > jwang@ps404a-59.stg:~$ iperf -p 52000 -c 10.43.48.58 -P 4
> > [ 3] local 10.43.48.59 port 42548 connected with 10.43.48.58 port 52000
> > ------------------------------------------------------------
> > Client connecting to 10.43.48.58, TCP port 52000
> > TCP window size: 165 KByte (default)
> > ------------------------------------------------------------
> > [ 4] local 10.43.48.59 port 42558 connected with 10.43.48.58 port 52000
> > [ 5] local 10.43.48.59 port 42560 connected with 10.43.48.58 port 52000
> > [ 6] local 10.43.48.59 port 42562 connected with 10.43.48.58 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 6] 0.0000-10.0000 sec 27.8 GBytes 23.9 Gbits/sec
> > [ 5] 0.0000-10.0001 sec 27.3 GBytes 23.5 Gbits/sec
> > [ 3] 0.0000-10.0001 sec 27.8 GBytes 23.9 Gbits/sec
> > [ 4] 0.0000-10.0001 sec 27.8 GBytes 23.9 Gbits/sec
> > [SUM] 0.0000-10.0001 sec 111 GBytes 95.1 Gbits/sec
> > [ CT] final connect times (min/avg/max/stdev) =
> > 0.295/0.340/0.390/0.201 ms (tot/err) = 4/0
> > jwang@ps404a-59.stg:~$ iperf -p 52000 -c 10.43.48.58 -P 2
> > ------------------------------------------------------------
> > Client connecting to 10.43.48.58, TCP port 52000
> > TCP window size: 165 KByte (default)
> > ------------------------------------------------------------
> > [ 4] local 10.43.48.59 port 44194 connected with 10.43.48.58 port 52000
> > [ 3] local 10.43.48.59 port 44186 connected with 10.43.48.58 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 3] 0.0000-10.0000 sec 48.3 GBytes 41.5 Gbits/sec
> > [ 4] 0.0000-10.0000 sec 41.3 GBytes 35.5 Gbits/sec
> > [SUM] 0.0000-10.0000 sec 89.7 GBytes 77.0 Gbits/sec
> > [ CT] final connect times (min/avg/max/stdev) =
> > 0.227/0.233/0.240/0.240 ms (tot/err) = 2/0
> > jwang@ps404a-59.stg:~$ pbkvm list
> > VM State PID Cores Mem VNC Migration
> > --------------------------------------------
> >
> > Total: 0 VMs, Running: 0
> > jwang@ps404a-59.stg:~$ iperf -p 52000 -c 10.43.48.58 -P 1
> > ------------------------------------------------------------
> > Client connecting to 10.43.48.58, TCP port 52000
> > TCP window size: 165 KByte (default)
> > ------------------------------------------------------------
> > [ 3] local 10.43.48.59 port 40364 connected with 10.43.48.58 port 52000
> > [ ID] Interval Transfer Bandwidth
> > [ 3] 0.0000-10.0000 sec 51.2 GBytes 44.0 Gbits/sec
> >
> > Thanks!
> >
> >
> > > Thanks,
> > >
> > > --
> > > Peter Xu
> > >
> >
>
> --
> Peter Xu
>
next prev parent reply other threads:[~2024-05-07 4:54 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-28 13:02 [PATCH-for-9.1 v2 0/3] rdma: Remove RDMA subsystem and pvrdma device Philippe Mathieu-Daudé
2024-03-28 13:02 ` [PATCH-for-9.1 v2 1/3] hw/rdma: Remove pvrdma device and rdmacm-mux helper Philippe Mathieu-Daudé
2024-03-28 17:51 ` Thomas Huth
2024-03-28 13:02 ` [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling Philippe Mathieu-Daudé
2024-03-28 14:18 ` Fabiano Rosas
2024-03-28 15:01 ` Peter Xu
2024-03-28 15:22 ` Thomas Huth
2024-03-28 19:04 ` Peter Xu
2024-03-29 1:53 ` Zhijian Li (Fujitsu) via
2024-03-29 10:28 ` Philippe Mathieu-Daudé
2024-03-29 19:44 ` Daniel P. Berrangé
2024-04-01 7:55 ` Zhijian Li (Fujitsu) via
2024-04-01 21:26 ` Yu Zhang
2024-04-02 21:23 ` Peter Xu
2024-04-08 14:07 ` Jinpu Wang
2024-04-08 16:18 ` Peter Xu
2024-04-09 7:32 ` Jinpu Wang
2024-04-09 19:46 ` Peter Xu
2024-04-10 2:28 ` Zhijian Li (Fujitsu) via
2024-04-10 13:49 ` Peter Xu
2024-04-11 14:20 ` Peter Xu
2024-04-11 16:36 ` Yu Zhang
2024-04-12 14:04 ` Peter Xu
2024-04-29 13:08 ` Michael Galaxy
2024-04-29 14:56 ` Peter Xu
2024-04-29 20:45 ` Yu Zhang
2024-04-29 20:56 ` Michael Galaxy
2024-04-30 7:15 ` Markus Armbruster
2024-04-30 8:00 ` Daniel P. Berrangé
2024-05-01 15:31 ` Peter Xu
2024-05-01 15:59 ` Daniel P. Berrangé
2024-05-01 16:16 ` Peter Xu
2024-05-02 13:22 ` Michael Galaxy
2024-05-02 13:30 ` Jinpu Wang
2024-05-02 16:19 ` Peter Xu
2024-05-02 17:10 ` Jinpu Wang
2024-05-03 6:40 ` Jinpu Wang
2024-05-03 14:33 ` Peter Xu
2024-05-06 10:08 ` Jinpu Wang
2024-05-06 15:28 ` Peter Xu
2024-05-07 4:52 ` Jinpu Wang [this message]
2024-05-08 10:06 ` Daniel P. Berrangé
2024-05-06 2:06 ` Gonglei (Arei) via
2024-05-06 15:18 ` Peter Xu
2024-05-07 1:50 ` Gonglei (Arei) via
2024-05-07 16:28 ` Peter Xu
2024-05-09 8:58 ` Zheng Chuan via
2024-05-09 14:13 ` Peter Xu
2024-05-13 7:30 ` Jinpu Wang
2024-05-14 15:19 ` Yu Zhang
2024-05-16 17:29 ` Michael Galaxy
2024-05-17 13:01 ` Yu Zhang
2024-05-21 22:15 ` Peter Xu
2024-05-13 18:52 ` Michael Galaxy
2024-04-11 14:42 ` Jinpu Wang
2024-04-09 9:00 ` Markus Armbruster
2024-03-28 13:02 ` [PATCH-for-9.1 v2 3/3] block/gluster: " Philippe Mathieu-Daudé
2024-03-28 17:54 ` Thomas Huth
2024-03-29 9:17 ` [PATCH-for-9.1 v2 0/3] rdma: Remove RDMA subsystem and pvrdma device Michael S. Tsirkin
2024-04-03 9:37 ` Philippe Mathieu-Daudé
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAMGffE=Hcep90DaoJDmKY6ESMtr1fZHehv-UrcHJHA8b2KvjBw@mail.gmail.com' \
--to=jinpu.wang@ionos.com \
--cc=alex.bennee@linaro.org \
--cc=arei.gonglei@huawei.com \
--cc=armbru@redhat.com \
--cc=berrange@redhat.com \
--cc=bleal@redhat.com \
--cc=cohuck@redhat.com \
--cc=devel@lists.libvirt.org \
--cc=eblake@redhat.com \
--cc=elmar.gerdes@ionos.com \
--cc=gaosong@loongson.cn \
--cc=hreitz@redhat.com \
--cc=integration@gluster.org \
--cc=kwolf@redhat.com \
--cc=lizhijian@fujitsu.com \
--cc=marcandre.lureau@redhat.com \
--cc=mgalaxy@akamai.com \
--cc=michael.roth@amd.com \
--cc=mst@redhat.com \
--cc=pannengyuan@huawei.com \
--cc=pbonzini@redhat.com \
--cc=peterx@redhat.com \
--cc=prasanna.kalever@redhat.com \
--cc=prasanna4324@gmail.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=thuth@redhat.com \
--cc=wainersm@redhat.com \
--cc=yu.zhang@ionos.com \
--cc=yuval.shaia.ml@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.