archive mirror
 help / color / mirror / Atom feed
From: Dominique Martinet <>
To: Eric Van Hensbergen <>
Cc: Latchesar Ionkov <>,,,
	linux-kernel <>,,
	V9FS Developers <>,, David Miller <>
Subject: Re: [V9fs-developer] [PATCH] net: trans_rdma: remove unused function
Date: Fri, 26 Jul 2013 17:17:37 +0200	[thread overview]
Message-ID: <20130726151737.GA24757@nautica> (raw)
In-Reply-To: <>

Eric Van Hensbergen wrote on Fri, Jul 26, 2013 :
> It depends on the flags you use with 9p.  If you mount with nodevmap it'll
> treat them like normal files.  Alternatively you can use synthetics from
> fuse or even 9p file systems from Plan 9 port.

Ok, thank you for the explanation. Now I need to see if my server
supports this :)

> I need to go back and refresh more of my RDMA knowledge, I thought you
> could provide a better more direct coupling between the request and
> response buffers (similar to what we do in virtio).  In such a case, you'd
> simply need to reclaim the "skipped" buffer not shuffle everything around.
> Ultimately that would seem to work against the point of having "zero copy"

What I said next was wrong, but the queue really isn't something you can
control this finely. Once something is posted, you can't take it back,
and you can't know which buffer will come back for which reply (but it's
easy enough to read the tag from the reply and associate it back)

For flush, I'm honestly not sure on how to deal with the possibility of
the servers processing the original query and sending a reply after
we've tagged our own buffer as free again, but I think there might
actually be a problem even on sucessful flush if the next post is from a
different tag (that has its own rc buffer) -- as far as I understand it,
this one's reply will be considered duplicate reply in handle_recv.
But I really need to do some more precise testing, I'll try playing with
nodevmap, if I get the flush functions to be called I'll get this

> > I haven't found how to reproduce this perfectly yet, but a dd with
> > blocksize 1MB and one with blocksize 10B in parallel brought the
> > mountpoint down (and the whole server was completely unavailable for the
> > duration of the dd - TCP sessions timed out, I even got IO errors on the
> > local disk :D)
> >
> Sounds like you ran out of memory and things didn't degrade gracefully.

I think it's actually "just" in the debug, there are two p9_debug printk
in p9_client_cb which are called from an rdma irq handler, and it gets
messy once the kernel buffer is full and it wants to flush to the
I've got a short patch that adds an argument to p9_client_cb with an
indication that it shouldn't output messages, I'll send it to the v9fs
mailing list for comments if you think the idea is acceptable.

>  Which server are you using?

I know of two servers out there which handle 9P/RDMA - diod ( ) and nfs-ganesha ( )
I wrote the RDMA implementation for ganesha and am mostly running tests
on that, but both should somewhat work with their quirks :)

Dominique Martine

  parent reply	other threads:[~2013-07-26 15:17 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-07-22 12:59 [PATCH] net: trans_rdma: remove unused function Andi Shyti
2013-07-24 22:46 ` David Miller
2013-07-24 23:09   ` Paul Bolle
2013-07-24 23:45     ` David Miller
2013-07-25  6:14       ` [V9fs-developer] " Dominique Martinet
2013-07-25  6:48         ` Dominique Martinet
2013-07-25  8:27           ` Andi Shyti
2013-07-25  8:35             ` David Miller
     [not found]           ` <>
2013-07-25 19:05             ` Dominique Martinet
2013-07-26  7:01               ` Dominique Martinet
     [not found]               ` <>
2013-07-26 15:17                 ` Dominique Martinet [this message]
2013-07-25  8:54         ` [PATCH] 9p: client: remove unused code and any reference to "cancelled" function Andi Shyti
2013-07-25  8:57           ` Andi Shyti
2013-07-30 22:54           ` David Miller

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20130726151737.GA24757@nautica \ \ \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).