From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 381D4C6369E for ; Wed, 2 Dec 2020 20:21:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B6A6C2053B for ; Wed, 2 Dec 2020 20:21:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729130AbgLBUVe (ORCPT ); Wed, 2 Dec 2020 15:21:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727003AbgLBUVe (ORCPT ); Wed, 2 Dec 2020 15:21:34 -0500 Received: from mail-qv1-xf34.google.com (mail-qv1-xf34.google.com [IPv6:2607:f8b0:4864:20::f34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C62F1C0613D6 for ; Wed, 2 Dec 2020 12:20:53 -0800 (PST) Received: by mail-qv1-xf34.google.com with SMTP id g19so1365295qvy.2 for ; Wed, 02 Dec 2020 12:20:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:mime-version :content-disposition:in-reply-to; bh=aj7zikfociwcKr6VGphiYIZ8rc665LBf4u7f617m8kE=; b=n/zZJtm7n5sa6dwUDkqyAEHqr/PViZlkWneUEVRRE9J7qlT6SEGJmJvmqUdDW+JR+T s+cD1eXzNCyKAoElzdabttMfDbV7qG8GcwMTm12kwHRGuTHsWtZ+mia3KD1Tg534QKyT OopetEiPDs0dU8/pSCwxnWUxa5s3RmsH6gP97z/3GeGmfwq0RM8i7l4NuzKcVVNuRxMM +z3ovi+Ukh56I3ih9IYGzuMW6byxrQbgKjfjg1GfPeAG/xmpWIPQzuZldmWuCqOjXrZ9 /fxLSzgdoIgG1XKR9V6TpIocayT5u6dLnhhixywPtHfOLAzOQqxKrEURZQ6/zQqHN5eB Ezrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :mime-version:content-disposition:in-reply-to; bh=aj7zikfociwcKr6VGphiYIZ8rc665LBf4u7f617m8kE=; b=WWJWPbb75JqZJsBpEcDBQeULoOsFa3awCDju7bt2IL6hz4ywQvEruJ71Cgqn3CeOaW 5Cprrj+8Rap+PSx5tVHA/R/WA3r+E9qx8Z+ez4VynBc/j90AhvKKuvmvVnksSVmXysKK gZ3lB9Eozbv7K02OqTqVAKooWPsjnlG2Q5Iac+EmcqHr6CdrFtCSQIs8xRqBu6tJlyUW UEtj7+7AUTcvfvR5NT2GVK5kspGFxuXVZMTTQeqdeND1Vw8tdpZy+flk6EKv5PqLSRvA RABK6E+UdlxIBgji/I9vuUSfMdl6Ui0/oXAdJ7PPxiJwQuCjshnNNmvyBCtr2dS0iPlH J50w== X-Gm-Message-State: AOAM531TVkzeV2r3d9nqeVbVTPKa0rjrWEgqyC0Gy8Smd8nDKTWR0gfa CqcmPQAnRFGJvPXKgQnXQVA= X-Google-Smtp-Source: ABdhPJz7smZ4bg9M3d7RHvZOaE7BxC09JYYiK8er2jAUeG9o73jqv6IzmdWlxeHZ+hZk9lrN8zPRGA== X-Received: by 2002:a0c:a992:: with SMTP id a18mr4320506qvb.21.1606940452966; Wed, 02 Dec 2020 12:20:52 -0800 (PST) Received: from localhost ([2620:10d:c091:480::1:8dbd]) by smtp.gmail.com with ESMTPSA id v15sm2739138qti.92.2020.12.02.12.20.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Dec 2020 12:20:52 -0800 (PST) Sender: Tejun Heo Date: Wed, 2 Dec 2020 15:20:24 -0500 From: "tj@kernel.org" To: NeilBrown Cc: Hillf Danton , Trond Myklebust , "peterz@infradead.org" , "linux-kernel@vger.kernel.org" , Lai Jiangshan Subject: Re: [PATCH rfc] workqueue: honour cond_resched() more effectively. Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87ima1ndwz.fsf@notabene.neil.brown.name> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hello, Neil. (cc'ing Lai) On Fri, Nov 20, 2020 at 10:07:56AM +1100, NeilBrown wrote: > If the workqueue maintainers are unmovable in the position that a > CM-workitem must not use excessive CPU ever, and so must not call > cond_resched(), then I can take that back to the NFS maintainers and Oh, that's not my position at all. I'm completely movable and am pretty sure Lai would be too. > negotiate different workqueue settings. But as I've said, I think this > is requiring the decision to be made in a place that is not well > positioned to make it. Very true. A lot of the current workqueue code is a result of gradual evolution and has a lot of historical cruft. Things like CPU_INTENSIVE or long_wq were introduced partly to facilitate existing users to CM workqueue and definitely are silly interfaces as you mentioned in another reply. I can see two directions: 1. Keeping things mostly as-is - leave the selection of the specific workqueue type to users. We can improve things by adding debug / sanity checks to surfaces issues more proactively, adding more documentation and cleaning up old cruft. 2. Make things properly dynamic. By default, let workqueue core decide what type of execution constraints are gonna be applied and dynamically adjust according to the behavior of specific workqueues or work items. e.g. it can start most work items CM per-cpu by default and then release them as unbound when its cpu consumption exceeds certain threshold. #1 is easier. #2 is definitely more attractive long term but would need a lot of careful work. I was suggesting towards #1 mostly because it's a safer / easier direction but if you wanna explore #2, please be my guest. Thank you. -- tejun