From: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org> To: Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org> Cc: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org, bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org, axboe-b10kYP2dOMg@public.gmane.org, linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org Subject: Re: [PATCH 2/9] IB: add a proper completion queue abstraction Date: Sat, 14 Nov 2015 08:08:49 +0100 [thread overview] Message-ID: <20151114070849.GD27738@lst.de> (raw) In-Reply-To: <20151113182513.GB21808-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org> On Fri, Nov 13, 2015 at 11:25:13AM -0700, Jason Gunthorpe wrote: > For instance, like this, not fulling draining the cq and then doing: > > > + completed = __ib_process_cq(cq, budget); > > + if (completed < budget) { > > + irq_poll_complete(&cq->iop); > > + if (ib_req_notify_cq(cq, IB_POLL_FLAGS) > 0) { > > Doesn't seem entirely right? There is no point in calling > ib_req_notify_cq if the code knows there is still stuff in the CQ and > has already, independently, arranged for ib_poll_hander to be > guarenteed called. The code only calls ib_req_notify_cq if it knowns we finished earlier than our budget. > > + completed = __ib_process_cq(cq, IB_POLL_BUDGET_WORKQUEUE); > > + if (completed >= IB_POLL_BUDGET_WORKQUEUE || > > + ib_req_notify_cq(cq, IB_POLL_FLAGS) > 0) > > + queue_work(ib_comp_wq, &cq->work); > > Same comment here.. Same here - we only requeue the work item if either we processed all of our budget, or ib_req_notify_cq with IB_CQ_REPORT_MISSED_EVENTS told us that we need to poll again. > I understand several drivers are not using a hard irq context for the > comp_handler call back. Is there any way to exploit that in this new > API so we don't have to do so many context switches? Ie if the driver > already is using a softirq when calling comp_handler can we somehow > just rig ib_poll_handler directly and avoid the overhead? (Future) Let's say this API makes it possible. I still don't think moving the whole budget and rearm logic into the LLD is necessarily a good idea if we can avoid it. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html
WARNING: multiple messages have this Message-ID (diff)
From: Christoph Hellwig <hch@lst.de> To: Jason Gunthorpe <jgunthorpe@obsidianresearch.com> Cc: Christoph Hellwig <hch@lst.de>, linux-rdma@vger.kernel.org, sagig@dev.mellanox.co.il, bart.vanassche@sandisk.com, axboe@fb.com, linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 2/9] IB: add a proper completion queue abstraction Date: Sat, 14 Nov 2015 08:08:49 +0100 [thread overview] Message-ID: <20151114070849.GD27738@lst.de> (raw) In-Reply-To: <20151113182513.GB21808@obsidianresearch.com> On Fri, Nov 13, 2015 at 11:25:13AM -0700, Jason Gunthorpe wrote: > For instance, like this, not fulling draining the cq and then doing: > > > + completed = __ib_process_cq(cq, budget); > > + if (completed < budget) { > > + irq_poll_complete(&cq->iop); > > + if (ib_req_notify_cq(cq, IB_POLL_FLAGS) > 0) { > > Doesn't seem entirely right? There is no point in calling > ib_req_notify_cq if the code knows there is still stuff in the CQ and > has already, independently, arranged for ib_poll_hander to be > guarenteed called. The code only calls ib_req_notify_cq if it knowns we finished earlier than our budget. > > + completed = __ib_process_cq(cq, IB_POLL_BUDGET_WORKQUEUE); > > + if (completed >= IB_POLL_BUDGET_WORKQUEUE || > > + ib_req_notify_cq(cq, IB_POLL_FLAGS) > 0) > > + queue_work(ib_comp_wq, &cq->work); > > Same comment here.. Same here - we only requeue the work item if either we processed all of our budget, or ib_req_notify_cq with IB_CQ_REPORT_MISSED_EVENTS told us that we need to poll again. > I understand several drivers are not using a hard irq context for the > comp_handler call back. Is there any way to exploit that in this new > API so we don't have to do so many context switches? Ie if the driver > already is using a softirq when calling comp_handler can we somehow > just rig ib_poll_handler directly and avoid the overhead? (Future) Let's say this API makes it possible. I still don't think moving the whole budget and rearm logic into the LLD is necessarily a good idea if we can avoid it.
next prev parent reply other threads:[~2015-11-14 7:08 UTC|newest] Thread overview: 144+ messages / expand[flat|nested] mbox.gz Atom feed top 2015-11-13 13:46 add a proper completion queue abstraction Christoph Hellwig 2015-11-13 13:46 ` [PATCH 1/9] move blk_iopoll to limit and make it generally available Christoph Hellwig [not found] ` <1447422410-20891-2-git-send-email-hch-jcswGhMUV9g@public.gmane.org> 2015-11-13 15:23 ` Or Gerlitz 2015-11-13 15:23 ` Or Gerlitz [not found] ` <CAJ3xEMgj2ycv61K38ZOowTRbrri_UhQgBcaKT0ZnnMHiBrmL5A-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2015-11-14 7:02 ` Christoph Hellwig 2015-11-14 7:02 ` Christoph Hellwig [not found] ` <20151114070200.GA27738-jcswGhMUV9g@public.gmane.org> 2015-11-15 8:48 ` Sagi Grimberg 2015-11-15 8:48 ` Sagi Grimberg [not found] ` <564846E9.9070301-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> 2015-11-15 9:04 ` Or Gerlitz 2015-11-15 9:04 ` Or Gerlitz [not found] ` <CAJ3xEMgvttM1D3bePz0CWhZAZ3gCSQsf_qgmq9Ny4gzK5d0bXw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2015-11-15 13:16 ` Sagi Grimberg 2015-11-15 13:16 ` Sagi Grimberg 2015-11-15 12:51 ` Christoph Hellwig 2015-11-15 12:51 ` Christoph Hellwig 2015-11-13 19:19 ` Bart Van Assche 2015-11-13 19:19 ` Bart Van Assche 2015-11-14 7:02 ` Christoph Hellwig 2015-11-17 17:16 ` Bart Van Assche 2015-11-17 17:16 ` Bart Van Assche 2015-11-17 17:27 ` Bart Van Assche 2015-11-18 13:58 ` Christoph Hellwig 2015-11-13 13:46 ` [PATCH 2/9] IB: add a proper completion queue abstraction Christoph Hellwig 2015-11-15 9:40 ` Sagi Grimberg [not found] ` <564852F2.5080602-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> 2015-11-15 12:55 ` Christoph Hellwig 2015-11-15 12:55 ` Christoph Hellwig [not found] ` <20151115125501.GB2218-jcswGhMUV9g@public.gmane.org> 2015-11-15 13:21 ` Sagi Grimberg 2015-11-15 13:21 ` Sagi Grimberg [not found] ` <1447422410-20891-3-git-send-email-hch-jcswGhMUV9g@public.gmane.org> 2015-11-13 18:25 ` Jason Gunthorpe 2015-11-13 18:25 ` Jason Gunthorpe [not found] ` <20151113182513.GB21808-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org> 2015-11-13 19:57 ` Bart Van Assche 2015-11-13 19:57 ` Bart Van Assche 2015-11-13 22:06 ` Jason Gunthorpe [not found] ` <20151113220636.GA32133-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org> 2015-11-14 7:13 ` Christoph Hellwig 2015-11-14 7:13 ` Christoph Hellwig [not found] ` <20151114071344.GE27738-jcswGhMUV9g@public.gmane.org> 2015-11-23 20:37 ` Jason Gunthorpe 2015-11-23 20:37 ` Jason Gunthorpe 2015-11-23 21:04 ` Bart Van Assche 2015-11-23 21:04 ` Bart Van Assche 2015-11-23 21:28 ` Jason Gunthorpe 2015-11-23 21:54 ` Bart Van Assche 2015-11-23 22:18 ` Jason Gunthorpe 2015-11-23 22:33 ` Bart Van Assche 2015-11-23 23:06 ` Jason Gunthorpe [not found] ` <B24F4DDE-709A-4D2D-8B26-4E83325DBB1A@asomi.com> 2015-11-24 0:00 ` Jason Gunthorpe [not found] ` <20151124000011.GA9301-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org> 2015-11-24 0:34 ` Tom Talpey 2015-11-24 0:34 ` Tom Talpey [not found] ` <5653B0AD.7090402-CLs1Zie5N5HQT0dZR+AlfA@public.gmane.org> 2015-11-24 0:40 ` Jason Gunthorpe 2015-11-24 0:40 ` Jason Gunthorpe 2015-11-24 2:35 ` Caitlin Bestler 2015-11-24 2:35 ` Caitlin Bestler [not found] ` <5653CCF0.7050501-DpaxOq6QOWMAvxtiuMwx3w@public.gmane.org> 2015-11-24 7:03 ` Jason Gunthorpe 2015-11-24 7:03 ` Jason Gunthorpe [not found] ` <20151124070301.GA23597-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org> 2015-11-24 12:52 ` Tom Talpey 2015-11-24 12:52 ` Tom Talpey 2015-11-14 7:08 ` Christoph Hellwig [this message] 2015-11-14 7:08 ` Christoph Hellwig 2015-11-23 20:01 ` Jason Gunthorpe [not found] ` <20151123200136.GA5640-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org> 2015-11-23 20:57 ` Christoph Hellwig 2015-11-23 20:57 ` Christoph Hellwig 2015-11-17 17:52 ` Bart Van Assche 2015-11-17 17:52 ` Bart Van Assche [not found] ` <564B697A.2020601-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org> 2015-11-18 7:55 ` Sagi Grimberg 2015-11-18 7:55 ` Sagi Grimberg [not found] ` <564C2F01.6020407-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> 2015-11-18 18:20 ` Bart Van Assche 2015-11-18 18:20 ` Bart Van Assche 2015-11-20 10:16 ` Christoph Hellwig 2015-11-20 16:50 ` Bart Van Assche 2015-11-22 9:51 ` Sagi Grimberg 2015-11-22 10:13 ` Christoph Hellwig [not found] ` <20151122101308.GA12189-jcswGhMUV9g@public.gmane.org> 2015-11-22 10:36 ` Sagi Grimberg 2015-11-22 10:36 ` Sagi Grimberg [not found] ` <56519A90.5010502-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> 2015-11-22 13:23 ` Christoph Hellwig 2015-11-22 13:23 ` Christoph Hellwig [not found] ` <20151122132352.GA14154-jcswGhMUV9g@public.gmane.org> 2015-11-22 14:57 ` Sagi Grimberg 2015-11-22 14:57 ` Sagi Grimberg 2015-11-22 16:55 ` Bart Van Assche 2015-11-18 14:00 ` Christoph Hellwig 2015-11-18 14:00 ` Christoph Hellwig 2015-11-13 13:46 ` [PATCH 3/9] IB: add a helper to safely drain a QP Christoph Hellwig [not found] ` <1447422410-20891-4-git-send-email-hch-jcswGhMUV9g@public.gmane.org> 2015-11-13 16:16 ` Steve Wise 2015-11-13 16:16 ` Steve Wise [not found] ` <56460CC4.3030001-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org> 2015-11-14 7:05 ` Christoph Hellwig 2015-11-14 7:05 ` Christoph Hellwig 2015-11-18 11:32 ` Sagi Grimberg 2015-11-18 11:32 ` Sagi Grimberg [not found] ` <564C61C3.3050307-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> 2015-11-18 14:06 ` Christoph Hellwig 2015-11-18 14:06 ` Christoph Hellwig [not found] ` <20151118140645.GI18820-jcswGhMUV9g@public.gmane.org> 2015-11-18 15:21 ` Steve Wise 2015-11-18 15:21 ` Steve Wise 2015-11-15 9:34 ` Sagi Grimberg [not found] ` <564851BB.1020004-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> 2015-11-16 16:38 ` Steve Wise 2015-11-16 16:38 ` Steve Wise [not found] ` <564A067B.8030504-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org> 2015-11-16 18:30 ` Steve Wise 2015-11-16 18:30 ` Steve Wise 2015-11-16 18:37 ` Sagi Grimberg [not found] ` <564A2270.1040004-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> 2015-11-16 19:03 ` Steve Wise 2015-11-16 19:03 ` Steve Wise 2015-11-17 8:54 ` Sagi Grimberg 2015-11-17 8:54 ` Sagi Grimberg 2015-11-23 10:28 ` Sagi Grimberg 2015-11-23 10:28 ` Sagi Grimberg 2015-11-23 10:35 ` Sagi Grimberg 2015-11-23 14:33 ` 'Christoph Hellwig' [not found] ` <5652EC00.8010705-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> 2015-11-23 14:48 ` Steve Wise 2015-11-23 14:48 ` Steve Wise 2015-11-23 14:44 ` Steve Wise 2015-11-23 14:44 ` Steve Wise 2015-11-17 17:06 ` Bart Van Assche 2015-11-17 17:06 ` Bart Van Assche [not found] ` <564B5E7D.9030309-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org> 2015-11-18 7:59 ` Sagi Grimberg 2015-11-18 7:59 ` Sagi Grimberg [not found] ` <1447422410-20891-1-git-send-email-hch-jcswGhMUV9g@public.gmane.org> 2015-11-13 13:46 ` [PATCH 4/9] srpt: chain RDMA READ/WRITE requests Christoph Hellwig 2015-11-13 13:46 ` Christoph Hellwig 2015-11-18 1:17 ` Bart Van Assche 2015-11-18 1:17 ` Bart Van Assche 2015-11-18 9:15 ` Sagi Grimberg 2015-11-18 16:32 ` Bart Van Assche 2015-11-18 16:32 ` Bart Van Assche [not found] ` <564CA83B.4060403-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org> 2015-11-20 10:20 ` Christoph Hellwig 2015-11-20 10:20 ` Christoph Hellwig [not found] ` <564BD1AF.60200-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org> 2015-11-18 14:06 ` Christoph Hellwig 2015-11-18 14:06 ` Christoph Hellwig 2015-11-13 13:46 ` [PATCH 5/9] srpt: use the new CQ API Christoph Hellwig [not found] ` <1447422410-20891-6-git-send-email-hch-jcswGhMUV9g@public.gmane.org> 2015-11-17 18:22 ` Bart Van Assche 2015-11-17 18:22 ` Bart Van Assche 2015-11-17 19:38 ` Bart Van Assche 2015-11-17 19:38 ` Bart Van Assche [not found] ` <564B8248.7050407-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org> 2015-11-18 14:03 ` Christoph Hellwig 2015-11-18 14:03 ` Christoph Hellwig 2015-11-13 13:46 ` [PATCH 6/9] srp: " Christoph Hellwig [not found] ` <1447422410-20891-7-git-send-email-hch-jcswGhMUV9g@public.gmane.org> 2015-11-17 19:56 ` Bart Van Assche 2015-11-17 19:56 ` Bart Van Assche 2015-11-18 14:03 ` Christoph Hellwig 2015-11-13 13:46 ` [PATCH 7/9] IB/iser: Use a dedicated descriptor for login Christoph Hellwig [not found] ` <1447422410-20891-8-git-send-email-hch-jcswGhMUV9g@public.gmane.org> 2015-11-15 9:14 ` Or Gerlitz 2015-11-15 9:14 ` Or Gerlitz 2015-11-13 13:46 ` [PATCH 8/9] IB/iser: Use helper for container_of Christoph Hellwig 2015-11-13 13:46 ` [PATCH 9/9] IB/iser: Convert to CQ abstraction Christoph Hellwig [not found] ` <1447422410-20891-10-git-send-email-hch-jcswGhMUV9g@public.gmane.org> 2015-11-15 9:21 ` Or Gerlitz 2015-11-15 9:21 ` Or Gerlitz [not found] <20151124100839.48b52fb35c6f209c51bccbb9807b6df0.f113bf890f.wbe@email24.secureserver.net> [not found] ` <20151124100839.48b52fb35c6f209c51bccbb9807b6df0.f113bf890f.wbe-wCzC33v8tqnShzhksYgB+AejPw4fNl8p@public.gmane.org> 2015-11-24 17:52 ` [PATCH 2/9] IB: add a proper completion queue abstraction Jason Gunthorpe 2015-11-24 17:52 ` Jason Gunthorpe [not found] ` <56552132.7090701@asomi.com> [not found] ` <56552132.7090701-DpaxOq6QOWMAvxtiuMwx3w@public.gmane.org> 2015-11-25 6:21 ` Jason Gunthorpe 2015-11-25 6:21 ` Jason Gunthorpe
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20151114070849.GD27738@lst.de \ --to=hch-jcswghmuv9g@public.gmane.org \ --cc=axboe-b10kYP2dOMg@public.gmane.org \ --cc=bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org \ --cc=jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org \ --cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \ --cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \ --cc=linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \ --cc=sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.