From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.4 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B874C43603 for ; Tue, 17 Dec 2019 23:56:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4326220729 for ; Tue, 17 Dec 2019 23:56:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="BciJzHLB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725975AbfLQX4D (ORCPT ); Tue, 17 Dec 2019 18:56:03 -0500 Received: from mail-oi1-f196.google.com ([209.85.167.196]:44770 "EHLO mail-oi1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725940AbfLQX4D (ORCPT ); Tue, 17 Dec 2019 18:56:03 -0500 Received: by mail-oi1-f196.google.com with SMTP id d62so105416oia.11 for ; Tue, 17 Dec 2019 15:56:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=uSrcrrmPD2HZZfVbkp1SOmo6OrJr9Uy/MkJXxU1BuKQ=; b=BciJzHLBwtLM77jVMtBYhAevWZzXWU7epivW4diMgfngZD5Vc7l9XqN+WuFJg9o3Tn jh8qjHPMQVyV8LLDIESzfYhnKv/xXfJl0fNSPxzsLuAfBu8YBtDDvZo2R09K411Nxpy2 ZqMVpo5mhjucPyBtq7D8FkpdlpoO9mxewotIUcBolsdzdMWhuIu6bJY7IqLiJTyHB53g MJghTLXf6ylqgKj/izG3tKXTyfrBgZrXlp4tyoI/iv7J6m8eM+Y2wwBb1cAmHehemsMR TvNaWEzZc3U/7owuwb7QaP6GnvWkesYsF1LrVJIGX3d8lhSVV42ekjkn14xxPOp1QANR lN3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=uSrcrrmPD2HZZfVbkp1SOmo6OrJr9Uy/MkJXxU1BuKQ=; b=EX2WdoittSJKvjyOT6Dhzg3/VvLKA/1dbyHtpgETZR7zlbcfR2r+evAj6p+E5xamYC 5AuveketAjlak+qmDHNum3r9FQMyBzQweel/0INAPQSA0hFFosgFIwvhCwOVkZk2ZbfH zIA2YNH9Gfwm0XVIurAn072wl5FG3jEPQUXAgrQLdmbfFxwKKuS5y7laTd9Y2vAXfpnj OMl4NgwRFR7hpwFjSNdR0yL3MxVbphJtZsoKqt5QHak3Rg96aOdJIscUE0je4Tdz+IDn NJUnXRdiSihTpu3l4W5PXcHtRBdpiWCH9Om218BIGkLD1Hrv0fG/Zwi6PK8w/HpZ51AJ rsRA== X-Gm-Message-State: APjAAAVBtnv7e/3pXONUZqAekCuMEPbFo+ZS8XO1jt3Mm6YDfH8MCrI4 GhFx3A0XjompE7QJiR2sS2lfQSyQnHd51slAPv2LEw== X-Google-Smtp-Source: APXvYqwrOCbCg/ESp2zlljby9Ba2gr6DV3ew6j4AQUNR95y4mxT8z/4/uiPGM9Z+m/1zGaPMx0yOd28PYrfcxAJcqRg= X-Received: by 2002:aca:bb08:: with SMTP id l8mr3134569oif.47.1576626962574; Tue, 17 Dec 2019 15:56:02 -0800 (PST) MIME-Version: 1.0 References: <20191217225445.10739-1-axboe@kernel.dk> <20191217225445.10739-4-axboe@kernel.dk> In-Reply-To: <20191217225445.10739-4-axboe@kernel.dk> From: Jann Horn Date: Wed, 18 Dec 2019 00:55:36 +0100 Message-ID: Subject: Re: [PATCH 3/7] io_uring: don't wait when under-submitting To: Jens Axboe , Pavel Begunkov Cc: io-uring Content-Type: text/plain; charset="UTF-8" Sender: io-uring-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org On Tue, Dec 17, 2019 at 11:54 PM Jens Axboe wrote: > There is no reliable way to submit and wait in a single syscall, as > io_submit_sqes() may under-consume sqes (in case of an early error). > Then it will wait for not-yet-submitted requests, deadlocking the user > in most cases. > > In such cases adjust min_complete, so it won't wait for more than > what have been submitted in the current io_uring_enter() call. It > may be less than total in-flight, but that up to a user to handle. > > Signed-off-by: Pavel Begunkov > Signed-off-by: Jens Axboe [...] > if (flags & IORING_ENTER_GETEVENTS) { > unsigned nr_events = 0; > > min_complete = min(min_complete, ctx->cq_entries); > + if (submitted != to_submit) > + min_complete = min(min_complete, (u32)submitted); Hm. Let's say someone submits two requests, first an ACCEPT request that might stall indefinitely and then a WRITE to a file on disk that is expected to complete quickly; and the caller uses min_complete=1 because they want to wait for the WRITE op. But now the submission of the WRITE fails, io_uring_enter() computes min_complete=min(1, 1)=1, and it blocks on the ACCEPT op. That would be bad, right? If the usecase I described is valid, I think it might make more sense to do something like this: u32 missing_submissions = to_submit - submitted; min_complete = min(min_complete, ctx->cq_entries); if ((flags & IORING_ENTER_GETEVENTS) && missing_submissions < min_complete) { min_complete -= missing_submissions; [...] } In other words: If we do a partially successful submission, only wait as long as we know that userspace definitely wants us to wait for one of the pending requests; and once we can't tell whether userspace intended to wait longer, return to userspace and let the user decide. Or it might make sense to just ignore IORING_ENTER_GETEVENTS completely in the partial submission case, in case userspace wants to immediately react to the failed request by writing out an error message to a socket or whatever. This case probably isn't performance-critical, right? And it would simplify things a bit.