From: Sagi Grimberg <sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
To: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Cc: Christoph Hellwig <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Linux NFS Mailing List
<linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Subject: Re: [PATCH v1 3/9] xprtrdma: Introduce ro_unmap_sync method
Date: Tue, 24 Nov 2015 16:44:28 +0200 [thread overview]
Message-ID: <565477CC.5070309@dev.mellanox.co.il> (raw)
In-Reply-To: <4B2D7C66-31AC-44F3-A8CC-22CC7136015C-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Hey Chuck,
>
>> It is painful, too painful. The entire value proposition of RDMA is
>> low-latency and waiting for the extra HW round-trip for a local
>> invalidation to complete is unacceptable, moreover it adds a huge loads
>> of extra interrupts and cache-line pollutions.
>
> The killer is the extra context switches, I’ve found.
That too...
> I’ve noticed only a marginal loss of performance on modern
> hardware.
Would you mind sharing your observations?
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
WARNING: multiple messages have this Message-ID (diff)
From: Sagi Grimberg <sagig@dev.mellanox.co.il>
To: Chuck Lever <chuck.lever@oracle.com>
Cc: Christoph Hellwig <hch@infradead.org>,
linux-rdma@vger.kernel.org,
Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
Sagi Grimberg <sagig@mellanox.com>
Subject: Re: [PATCH v1 3/9] xprtrdma: Introduce ro_unmap_sync method
Date: Tue, 24 Nov 2015 16:44:28 +0200 [thread overview]
Message-ID: <565477CC.5070309@dev.mellanox.co.il> (raw)
In-Reply-To: <4B2D7C66-31AC-44F3-A8CC-22CC7136015C@oracle.com>
Hey Chuck,
>
>> It is painful, too painful. The entire value proposition of RDMA is
>> low-latency and waiting for the extra HW round-trip for a local
>> invalidation to complete is unacceptable, moreover it adds a huge loads
>> of extra interrupts and cache-line pollutions.
>
> The killer is the extra context switches, I’ve found.
That too...
> I’ve noticed only a marginal loss of performance on modern
> hardware.
Would you mind sharing your observations?
next prev parent reply other threads:[~2015-11-24 14:44 UTC|newest]
Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-23 22:13 [PATCH v1 0/9] NFS/RDMA client patches for 4.5 Chuck Lever
2015-11-23 22:13 ` Chuck Lever
[not found] ` <20151123220627.32702.62667.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2015-11-23 22:13 ` [PATCH v1 1/9] xprtrdma: Add a safety margin for receive buffers Chuck Lever
2015-11-23 22:13 ` Chuck Lever
[not found] ` <20151123221357.32702.59922.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2015-11-24 0:55 ` Tom Talpey
2015-11-24 0:55 ` Tom Talpey
[not found] ` <5653B586.705-CLs1Zie5N5HQT0dZR+AlfA@public.gmane.org>
2015-11-24 1:16 ` Chuck Lever
2015-11-24 1:16 ` Chuck Lever
[not found] ` <D946BAC3-26D5-4801-BD50-9F026EEF6551-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2015-11-24 1:22 ` Tom Talpey
2015-11-24 1:22 ` Tom Talpey
[not found] ` <5653BBCB.8010107-CLs1Zie5N5HQT0dZR+AlfA@public.gmane.org>
2015-11-24 1:44 ` Chuck Lever
2015-11-24 1:44 ` Chuck Lever
2015-11-23 22:14 ` [PATCH v1 2/9] xprtrdma: Move struct ib_send_wr off the stack Chuck Lever
2015-11-23 22:14 ` Chuck Lever
2015-11-23 22:14 ` [PATCH v1 3/9] xprtrdma: Introduce ro_unmap_sync method Chuck Lever
2015-11-23 22:14 ` Chuck Lever
[not found] ` <20151123221414.32702.87638.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2015-11-24 6:45 ` Christoph Hellwig
2015-11-24 6:45 ` Christoph Hellwig
[not found] ` <20151124064556.GA29141-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2015-11-24 7:28 ` Jason Gunthorpe
2015-11-24 7:28 ` Jason Gunthorpe
2015-11-24 10:59 ` Sagi Grimberg
2015-11-24 10:59 ` Sagi Grimberg
[not found] ` <565442F5.7080400-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-11-24 13:43 ` Tom Talpey
2015-11-24 13:43 ` Tom Talpey
[not found] ` <5654697E.1030809-CLs1Zie5N5HQT0dZR+AlfA@public.gmane.org>
2015-11-24 14:40 ` Sagi Grimberg
2015-11-24 14:40 ` Sagi Grimberg
2015-11-24 14:39 ` Chuck Lever
2015-11-24 14:39 ` Chuck Lever
[not found] ` <4B2D7C66-31AC-44F3-A8CC-22CC7136015C-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2015-11-24 14:44 ` Sagi Grimberg [this message]
2015-11-24 14:44 ` Sagi Grimberg
[not found] ` <565477CC.5070309-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-11-24 15:20 ` Chuck Lever
2015-11-24 15:20 ` Chuck Lever
2015-11-24 18:57 ` Jason Gunthorpe
2015-11-24 18:57 ` Jason Gunthorpe
2015-11-23 22:14 ` [PATCH v1 4/9] xprtrdma: Add ro_unmap_sync method for FRWR Chuck Lever
2015-11-23 22:14 ` Chuck Lever
2015-11-23 22:14 ` [PATCH v1 5/9] xprtrdma: Add ro_unmap_sync method for FMR Chuck Lever
2015-11-23 22:14 ` Chuck Lever
[not found] ` <20151123221430.32702.86114.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2015-11-24 0:57 ` Tom Talpey
2015-11-24 0:57 ` Tom Talpey
[not found] ` <5653B606.3070700-CLs1Zie5N5HQT0dZR+AlfA@public.gmane.org>
2015-11-24 6:52 ` Future of FMR support, was: " Christoph Hellwig
2015-11-24 6:52 ` Christoph Hellwig
[not found] ` <20151124065225.GB29141-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2015-11-24 7:12 ` Jason Gunthorpe
2015-11-24 7:12 ` Jason Gunthorpe
[not found] ` <20151124071215.GC23597-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-12-01 15:33 ` Chuck Lever
2015-12-01 15:33 ` Chuck Lever
[not found] ` <AE86B182-6000-4437-8502-9D2C5EC3B09D-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2015-12-02 12:27 ` Christoph Hellwig
2015-12-02 12:27 ` Christoph Hellwig
2015-11-24 12:36 ` Tom Talpey
2015-11-24 12:36 ` Tom Talpey
2015-11-24 21:54 ` santosh shilimkar
2015-11-24 21:54 ` santosh shilimkar
[not found] ` <5654DC7A.7080807-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2015-11-25 9:00 ` Christoph Hellwig
2015-11-25 9:00 ` Christoph Hellwig
[not found] ` <20151125090009.GA11255-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2015-11-25 17:09 ` santosh shilimkar
2015-11-25 17:09 ` santosh shilimkar
[not found] ` <5655EB40.8070508-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2015-11-25 18:22 ` Or Gerlitz
2015-11-25 18:22 ` Or Gerlitz
[not found] ` <CAJ3xEMgC5aa8pvvvs0y8O+h_WBk-em07gHvOgP=DFDrv4ydGAg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-11-25 19:17 ` santosh shilimkar
2015-11-25 19:17 ` santosh shilimkar
2015-11-25 19:28 ` Jason Gunthorpe
2015-11-25 19:28 ` Jason Gunthorpe
[not found] ` <20151125192824.GD3223-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-11-26 10:01 ` Sagi Grimberg
2015-11-26 10:01 ` Sagi Grimberg
2015-11-23 22:14 ` [PATCH v1 6/9] xprtrdma: Add ro_unmap_sync method for all-physical registration Chuck Lever
2015-11-23 22:14 ` Chuck Lever
2015-11-23 22:14 ` [PATCH v1 7/9] SUNRPC: Introduct xprt_commit_rqst() Chuck Lever
2015-11-23 22:14 ` Chuck Lever
[not found] ` <20151123221446.32702.24797.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2015-11-24 19:54 ` Anna Schumaker
2015-11-24 19:54 ` Anna Schumaker
[not found] ` <5654C079.7060507-ZwjVKphTwtPQT0dZR+AlfA@public.gmane.org>
2015-11-24 19:56 ` Chuck Lever
2015-11-24 19:56 ` Chuck Lever
2015-11-23 22:14 ` [PATCH v1 8/9] xprtrdma: Invalidate in the RPC reply handler Chuck Lever
2015-11-23 22:14 ` Chuck Lever
[not found] ` <20151123221454.32702.76062.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2015-11-24 1:01 ` Tom Talpey
2015-11-24 1:01 ` Tom Talpey
2015-11-23 22:15 ` [PATCH v1 9/9] xprtrdma: Revert commit e7104a2a9606 ('xprtrdma: Cap req_cqinit') Chuck Lever
2015-11-23 22:15 ` Chuck Lever
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=565477CC.5070309@dev.mellanox.co.il \
--to=sagig-ldsdmyg8hgv8yrgs2mwiifqbs+8scbdb@public.gmane.org \
--cc=chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org \
--cc=hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org \
--cc=linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.