From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christoph Hellwig Subject: Re: a simple and scalable pNFS block layout server Date: Tue, 6 Jan 2015 18:56:11 +0100 Message-ID: <20150106175611.GA16413@lst.de> References: <1420561721-9150-1-git-send-email-hch@lst.de> <20150106173222.GF12067@fieldses.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Cc: linux-fsdevel@vger.kernel.org, linux-nfs@vger.kernel.org, Jeff Layton , xfs@oss.sgi.com To: "J. Bruce Fields" Return-path: Content-Disposition: inline In-Reply-To: <20150106173222.GF12067@fieldses.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: xfs-bounces@oss.sgi.com Sender: xfs-bounces@oss.sgi.com List-Id: linux-fsdevel.vger.kernel.org T24gVHVlLCBKYW4gMDYsIDIwMTUgYXQgMTI6MzI6MjJQTSAtMDUwMCwgSi4gQnJ1Y2UgRmllbGRz IHdyb3RlOgo+IAktIGRvIHdlIGhhdmUgZXZpZGVuY2UgdGhhdCB0aGlzIGlzIHVzZWZ1bCBpbiBp dHMgY3VycmVudCBmb3JtPwoKV2hhdCBpcyB5b3VyIHRocmVzaG9sZCBmb3IgdXNlZnVsbmVzcz8g IEl0IHBhc3NlcyB4ZnN0ZXN0cyBmaW5lLCBhbmQKc2hvd3MgbGluZWFyIHNjYWxhYmlsaXR5IHdp dGggbXVsdGlwbGUgY2xpZW50cyB0aGF0IGVhY2ggaGF2ZSAxMEdCCmxpbmtzLiAKCj4gCS0gYW55 IGFkdmljZSBvbiB0ZXN0aW5nPyAgSXMgdGhlcmUgd2FzIHNvbWUgc2ltcGxlIHZpcnR1YWwgc2V0 dXAKPiAJICB0aGF0IHdvdWxkIGFsbG93IGFueSBsb3NlciB3aXRoIG5vIHNwZWNpYWwgaGFyZHdh cmUgKGUuZy4sIG1lKQo+IAkgIHRvIGNoZWNrIHdoZXRoZXIgdGhleSd2ZSBicm9rZW4gdGhlIGJs b2NrIHNlcnZlcj8KClJ1biB0d28ga3ZtIFZNcyB0aGF0IHNoYXJlIHRoZSBzYW1lIGRpc2suICBD cmVhdGUgYW4gWEZTIGZpbGVzeXN0ZW0Kb24gdGhlIE1EUywgYW5kIGV4cG9ydCBpdC4gIElmIHRo ZSBjbGllbnQgaGFzIGJsa21hcGQgcnVubmluZyAob24gRGViaWFuCml0IG5lZWRzIHRvIGJlIHN0 YXJ0ZWQgbWFudWFsbHkpIGl0IHdpbGwgdXNlIHBORlMgZm9yIGFjY2Vzc2luZyB0aGUKZmlsZXN5 c3RlbS4gIFZlcmlmeSB0aGF0IHVzaW5nIHRoZSBwZXItb3BlcmF0aW9uIGNvdW50ZXJzIGluCi9w cm9jL3NlbGYvbW91bnN0YXRzLiAgUmVwZWF0IHdpdGggYWRkaXRpb25hbCBjbGllbnRzIGFzIG5l c3NlY2FyeS4KCkFsdGVybmF0aXZlbHkgc2V0IHVwIGEgc2ltcGxlIGlTQ1NJIHRhcmdldCB1c2lu ZyB0Z3Qgb3IgbGlvIGFuZApjb25uZWN0IHRvIGl0IGZyb20gbXVsdGlwbGUgY2xpZW50cy4KCj4g CS0gYW55IGRlYnVnZ2luZyBhZHZpY2U/ICBFLmcuLCBoYXZlIHlvdSBjaGVja2VkIGlmIGN1cnJl bnQKPiAJICB3aXJlc2hhcmsgY2FuIGhhbmRsZSB0aGUgTURTIHRyYWZmaWM/CgpUaGUgd2lyZXNo YXJlIHZlcnNpb24gSSd2ZSB1c2VkIGRlY29kZWQgdGhlIGdlbmVyaWMgcE5GUyBvcGVyYXRpb25z CmZpbmUsIGJ1dCBqdXN0IGR1bXBzIHRoZSBsYXlvdXQgc3BlY2lmaWNzIGFzIGhleCBkYXRhLgoK RW5hYmxlIHRoZSB0cmFjZSBwb2ludHMgYWRkZWQgaW4gdGhpcyBzZXJpZXMsIHRoZXkgdHJhY2sg YWxsIHN0YXRlaWQKaW50ZXJhY3Rpb25zIGluIHRoZSBzZXJ2ZXIuICBBZGRpdNGWb25hbGx5IHRo ZSBwbmZzIGRlYnVnIHByaW50a3Mgb24KY2xpZW50IGFuZCBzZXJ2ZXIgZHVtcCBhIGxvdCBvZiBp bmZvcm1hdGlvbi4KCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fCnhmcyBtYWlsaW5nIGxpc3QKeGZzQG9zcy5zZ2kuY29tCmh0dHA6Ly9vc3Muc2dpLmNvbS9t YWlsbWFuL2xpc3RpbmZvL3hmcwo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: linux-nfs-owner@vger.kernel.org Received: from verein.lst.de ([213.95.11.211]:57514 "EHLO newverein.lst.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755960AbbAFR4O (ORCPT ); Tue, 6 Jan 2015 12:56:14 -0500 Date: Tue, 6 Jan 2015 18:56:11 +0100 From: Christoph Hellwig To: "J. Bruce Fields" Cc: Jeff Layton , linux-nfs@vger.kernel.org, linux-fsdevel@vger.kernel.org, xfs@oss.sgi.com Subject: Re: a simple and scalable pNFS block layout server Message-ID: <20150106175611.GA16413@lst.de> References: <1420561721-9150-1-git-send-email-hch@lst.de> <20150106173222.GF12067@fieldses.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 In-Reply-To: <20150106173222.GF12067@fieldses.org> Sender: linux-nfs-owner@vger.kernel.org List-ID: On Tue, Jan 06, 2015 at 12:32:22PM -0500, J. Bruce Fields wrote: > - do we have evidence that this is useful in its current form? What is your threshold for usefulness? It passes xfstests fine, and shows linear scalability with multiple clients that each have 10GB links. > - any advice on testing? Is there was some simple virtual setup > that would allow any loser with no special hardware (e.g., me) > to check whether they've broken the block server? Run two kvm VMs that share the same disk. Create an XFS filesystem on the MDS, and export it. If the client has blkmapd running (on Debian it needs to be started manually) it will use pNFS for accessing the filesystem. Verify that using the per-operation counters in /proc/self/mounstats. Repeat with additional clients as nessecary. Alternatively set up a simple iSCSI target using tgt or lio and connect to it from multiple clients. > - any debugging advice? E.g., have you checked if current > wireshark can handle the MDS traffic? The wireshare version I've used decoded the generic pNFS operations fine, but just dumps the layout specifics as hex data. Enable the trace points added in this series, they track all stateid interactions in the server. AdditÑ–onally the pnfs debug printks on client and server dump a lot of information.