From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 138D8C433F5 for ; Thu, 30 Sep 2021 14:39:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F1B1661A05 for ; Thu, 30 Sep 2021 14:39:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350898AbhI3Okx (ORCPT ); Thu, 30 Sep 2021 10:40:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:60056 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350863AbhI3Okw (ORCPT ); Thu, 30 Sep 2021 10:40:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633012749; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=2ly9W6CaHyQP3qe9RdMzB2/4zDGpIqL2pyl1uodDkAQ=; b=DdAdtpbIbBKbMMxsqFbOg2IxWLFRYsGUEpTzXuElvyPCZOEpThvHf7txQPNWBd65MH77l6 V7aJmC5ZzHj+MQouDzaVBoiE9QqFib7mNa8iCaVqaeVtFzBfxzIu1eDGTWFOkkCdKnmyUH TAdKrgVd13fXt/pD52rxGsHEWYJ1r+s= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-62-ioWxPXreOueJpjzdU2WR1g-1; Thu, 30 Sep 2021 10:39:08 -0400 X-MC-Unique: ioWxPXreOueJpjzdU2WR1g-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8126B835E04; Thu, 30 Sep 2021 14:39:07 +0000 (UTC) Received: from horse.redhat.com (unknown [10.22.16.146]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2A1255D6D5; Thu, 30 Sep 2021 14:39:07 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id B97FC220B02; Thu, 30 Sep 2021 10:39:06 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtio-fs@redhat.com, miklos@szeredi.hu, stefanha@redhat.com Cc: vgoyal@redhat.com, iangelak@redhat.com, jaggel@bu.edu, dgilbert@redhat.com Subject: [PATCH 0/8] virtiofs: Notification queue and blocking posix locks Date: Thu, 30 Sep 2021 10:38:42 -0400 Message-Id: <20210930143850.1188628-1-vgoyal@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Hi, As of now we do not support blocking remote posix locks with virtiofs. Well fuse client does not care but server returns -EOPNOTSUPP. There are couple of reasons to not support it yet. - If virtiofsd is single threaded or does not have a thread pool just to handle requests which can block for a long time, virtiofsd will stop processing new requests and virtiofs will come to a halt. To the extent that further unlock request will not make progress and deadlock will result. This can be taken care of by creating a custom thread pool in virtiofsd just to hanlde lock requests. - If client sends a blocking lock request and blocks, then it will consume descriptors in vring. If enough processes block, it is possible that vring does not have capacity to send more requests till some response comes back and descriptors are free. This can also lead to deadlock where an unlock request can't be sent to virtiofsd now. Also this will stop virtiofs operation as well as new filesystem requests can't be sent. To avoid this issue, idea was suggested thatn when a blocking lock request is sent by client, do not block it. Immediately send a reply saying client process should wait for a notification which will let it know once lock is available. This will make sure descriptors in virtqueue are not kept busy while we are waiting for lock and future unlock and other file system requests can continue to make progress. This first requires a notion of notification queue and virtiosfd being able to send notifications to client. This patch series implements that as well. As of now only one notification type has been implemented but now infrastructure is in place and other use cases should be easily add more type of notifications as need be. We don't yet have the capability to interrupt the process which is waiting for the posix lock. And reason for that is that virtiofs does not support capability to interrupt yet. That's a TODO item for later. Please have a look. Thanks Vivek Vivek Goyal (8): virtiofs: Disable interrupt requests properly virtiofs: Fix a comment about fuse_dev allocation virtiofs: Add an index to keep track of first request queue virtiofs: Decouple queue index and queue type virtiofs: Add a virtqueue for notifications virtiofs: Add a helper to end request and decrement inflight number virtiofs: Add new notification type FUSE_NOTIFY_LOCK virtiofs: Handle reordering of reply and notification event fs/fuse/virtio_fs.c | 438 ++++++++++++++++++++++++++++++--- include/uapi/linux/fuse.h | 11 +- include/uapi/linux/virtio_fs.h | 5 + 3 files changed, 412 insertions(+), 42 deletions(-) -- 2.31.1