From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.2 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29909C10F0E for ; Thu, 18 Apr 2019 18:42:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EF0752064A for ; Thu, 18 Apr 2019 18:42:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1555612961; bh=MY2shcyZpXrEUui82Lz1MosyPFbg/iJbPkxPSB4EGnM=; h=From:To:Cc:Subject:Date:List-ID:From; b=CnVbV90kvm5Gp3RaRpAKUcq5CVcodI53IyTiCfIS50wjEKkqUPZxH2yAwVb/0EDdV enVxAOy2yibuHp72F/bNieC/NHEG2r5S0MbLYOfYSSlLnGL1eHtdZEo/aV0uEez6JA RRtkMnrTOVL/Qg84j6EtlwCHifgSumIcCqkdQXvI= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2389899AbfDRSmk (ORCPT ); Thu, 18 Apr 2019 14:42:40 -0400 Received: from mail.kernel.org ([198.145.29.99]:60562 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2389853AbfDRSmj (ORCPT ); Thu, 18 Apr 2019 14:42:39 -0400 Received: from localhost.localdomain (unknown [24.213.116.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1024A2064A; Thu, 18 Apr 2019 18:42:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1555612958; bh=MY2shcyZpXrEUui82Lz1MosyPFbg/iJbPkxPSB4EGnM=; h=From:To:Cc:Subject:Date:From; b=pdWROMxwDs21S0Ip8Vu9SwrHVK60O3koPJz/GgimiA7baItCxmic8QTbOtZBAIp/O nuEX9w4Lo/1ArwZbBw936KsNBurTsjyGGksF6XiVW973NEfhDyrzyp4RcWEj7jGNb3 nlL7gzDb+AMFIipGu9Z795ID2LNUh3UqA3FqpcV8= From: hubcap@kernel.org To: linux-fsdevel@vger.kernel.org, christoph@lameter.com, martin@omnibond.com Cc: Mike Marshall Subject: [RFC PATCH 00/22] Orangefs Through the Pagecache Date: Thu, 18 Apr 2019 14:40:52 -0400 Message-Id: <20190418184113.9152-1-hubcap@kernel.org> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: Mike Marshall Until now the Orangefs kernel module has not gone through the pagecache and has been a de facto O_DIRECT style module. This patch series adds pagecache support and greatly improves small IO. For example, xfstest generic/069 used to take several hours on our test VMs, and now finishes in seconds. Orangefs' kernel module shares a memory buffer with Orangefs' userspace parts for IO. Orangefs can best exploit parallelism when the buffer is significantly filled, and struggles when the buffer is nearly empty. Buffering reads and writes through the pagecache helps us to communicate with userspace less often with buffers that are more full. Reads can be tuned with "count" from read(2) to simulate page sizes more in line with Orangefs's buffer size (4 meg by default). Userspace will deposit "count" bytes in the shared buffer on a read and when the VFS calls orangefs_readpage, as many pagecache pages as possible will be filled at once. Writes are coalesced even across pages when possible up to the size of the buffer between the OrangeFS client core and the kernel. This is 4 MB by default. This means that an application writing with a block size less than 4 MB will have its writes upgraded. For sequential writes, this is a big improvement. Martin Brandenburg (19): orangefs: implement xattr cache orangefs: do not invalidate attributes on inode create orangefs: simplify orangefs_inode_getattr interface orangefs: update attributes rather than relying on server orangefs: hold i_lock during inode_getattr orangefs: set up and use backing_dev_info orangefs: let setattr write to cached inode orangefs: reorganize setattr functions to track attribute changes orangefs: remove orangefs_readpages orangefs: service ops done for writeback are not killable orangefs: migrate to generic_file_read_iter orangefs: implement writepage orangefs: do not return successful read when the client-core disappeared orangefs: move do_readv_writev to direct_IO orangefs: skip inode writeout if nothing to write orangefs: avoid fsync service operation on flush orangefs: write range tracking orangefs: implement writepages orangefs: add orangefs_revalidate_mapping Mike Marshall (3): orangefs: remember count when reading. orangefs: pass slot index back to readpage. orangefs: copy Orangefs-sized blocks into the pagecache if possible. fs/orangefs/acl.c | 4 +- fs/orangefs/file.c | 389 ++++++-------- fs/orangefs/inode.c | 908 +++++++++++++++++++++++++++++---- fs/orangefs/namei.c | 40 +- fs/orangefs/orangefs-bufmap.c | 13 + fs/orangefs/orangefs-bufmap.h | 2 + fs/orangefs/orangefs-debugfs.c | 4 +- fs/orangefs/orangefs-kernel.h | 56 +- fs/orangefs/orangefs-mod.c | 1 + fs/orangefs/orangefs-sysfs.c | 22 + fs/orangefs/orangefs-utils.c | 179 +++---- fs/orangefs/super.c | 39 +- fs/orangefs/waitqueue.c | 18 +- fs/orangefs/xattr.c | 106 +++- 14 files changed, 1293 insertions(+), 488 deletions(-) -- 2.20.1