From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ed1-f51.google.com (mail-ed1-f51.google.com [209.85.208.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1EA1AC8CE for ; Mon, 15 May 2023 15:45:12 +0000 (UTC) Received: by mail-ed1-f51.google.com with SMTP id 4fb4d7f45d1cf-50bc4bc2880so19836469a12.2 for ; Mon, 15 May 2023 08:45:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; t=1684165511; x=1686757511; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=XPEGaC+x0TYEuX7HHgeWtz7pyNM4DGEHBqF/DDq7Iw0=; b=gImqqK7Ez4ILwctNY0AcR7B6kOcnLZLrABJErDV0yJSdTCJO24RZwul28fa1YXP4J5 bKoHYDUdz0GrdUt1uvyJRzrL7wHWnEE7CFgoVdIxKph77hkZpY5hXoE8CN2AnKNyhvPE 1WD55DRKkWgfPnF1cu9nglLrdj/x+IQwngXX0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684165511; x=1686757511; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XPEGaC+x0TYEuX7HHgeWtz7pyNM4DGEHBqF/DDq7Iw0=; b=jgjAvqWc/5AWLkhH394b4K8L7XuJBjmfwjgu7cs6v51EX2Mr9xxwODpvnK3YpvWN/h Av+hX3L8FEBp5Ffa+Jzyjn1njRgkJwDNALFFt6tIyPgKUhfxZ5enTFE4jS9aMv8jUbPO n7uf73uJp+ObCGoKz+7KZzfyCoQSG9JUxBnuuhYCetaRYsztbHW+RFhyy65uifx8Wif+ B1XbNWESmiIh8ygRxTfNO5DOhhWBt/gXnonn8WOQzrjdzdiFfqU6WF/tS24gvXPxoOP9 ilWdNvjh8vWfqDSkByForVYSYcevweIvJx90afsFX2ekOK/dmd09RLM4SkcFY7GYv1MY NmWA== X-Gm-Message-State: AC+VfDwHn+b5gLJF15YkZu/oNFW9zQ72Ibo+iR6Xa5Hx7/L9wUXpRNSH o0A+0+PruXosr7jP9DcZtRYT4TSE6Q1D69ieMDZNPg== X-Google-Smtp-Source: ACHHUZ7mpZ/KVecAOcbHDfPHr47zJUMyjgmaO5qna5LTVzcYtnH2Zdzm612DJQ0L8SgdxUY8pU9x/w== X-Received: by 2002:a17:907:7212:b0:966:61b3:f630 with SMTP id dr18-20020a170907721200b0096661b3f630mr25690495ejc.9.1684165511034; Mon, 15 May 2023 08:45:11 -0700 (PDT) Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com. [209.85.218.53]) by smtp.gmail.com with ESMTPSA id n12-20020a1709065dac00b009663cf5dc2fsm9771381ejv.66.2023.05.15.08.45.09 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 15 May 2023 08:45:10 -0700 (PDT) Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-965d2749e2eso1954984466b.1 for ; Mon, 15 May 2023 08:45:09 -0700 (PDT) X-Received: by 2002:a17:907:70d:b0:94b:d57e:9d4b with SMTP id xb13-20020a170907070d00b0094bd57e9d4bmr30521770ejb.2.1684165509441; Mon, 15 May 2023 08:45:09 -0700 (PDT) Precedence: bulk X-Mailing-List: regressions@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <20230202232517.8695-1-michael.christie@oracle.com> <20230202232517.8695-9-michael.christie@oracle.com> <78c5e150-26cf-7724-74ee-4a0b16b944b1@oracle.com> <48842e92-835e-bc3f-7118-48b8f415f532@leemhuis.info> <20230515-vollrausch-liebgeworden-2765f3ca3540@brauner> In-Reply-To: <20230515-vollrausch-liebgeworden-2765f3ca3540@brauner> From: Linus Torvalds Date: Mon, 15 May 2023 08:44:52 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v11 8/8] vhost: use vhost_tasks for worker threads To: Christian Brauner Cc: Thorsten Leemhuis , Mike Christie , nicolas.dichtel@6wind.com, Linux kernel regressions list , hch@infradead.org, stefanha@redhat.com, jasowang@redhat.com, mst@redhat.com, sgarzare@redhat.com, virtualization@lists.linux-foundation.org, ebiederm@xmission.com, konrad.wilk@oracle.com, linux-kernel@vger.kernel.org, Jens Axboe Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, May 15, 2023 at 7:23=E2=80=AFAM Christian Brauner wrote: > > So I think we will be able to address (1) and (2) by making vhost tasks > proper threads and blocking every signal except for SIGKILL and SIGSTOP > and then having vhost handle get_signal() - as you mentioned - the same > way io uring already does. We should also remove the ingore_signals > thing completely imho. I don't think we ever want to do this with user > workers. Right. That's what IO_URING does: if (args->io_thread) { /* * Mark us an IO worker, and block any signal that isn't * fatal or STOP */ p->flags |=3D PF_IO_WORKER; siginitsetinv(&p->blocked, sigmask(SIGKILL)|sigmask(SIGSTOP= )); } and I really think that vhost should basically do exactly what io_uring doe= s. Not because io_uring fundamentally got this right - but simply because io_uring had almost all the same bugs (and then some), and what the io_uring worker threads ended up doing was to basically zoom in on "this works". And it zoomed in on it largely by just going for "make it look as much as possible as a real user thread", because every time the kernel thread did something different, it just caused problems. So I think the patch should just look something like the attached. Mike, can you test this on whatever vhost test-suite? I did consider getting rid of ".ignore_signals" entirely, and instead just keying the "block signals" behavior off the ".user_worker" flag. But this approach doesn't seem wrong either, and I don't think it's wrong to make the create_io_thread() function say that ".ignore_signals =3D 1" thing explicitly, rather than key it off the ".io_thread" flag. Jens/Christian - comments? Slightly related to this all: I think vhost should also do CLONE_FILES, and get rid of the whole ".no_files" thing. Again, if vhost doesn't use any files, it shouldn't matter, and looking different just to be different is wrong. But if vhost doesn't use any files, the current situation shouldn't be a bug either. Linus From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CB39CC7EE23 for ; Mon, 15 May 2023 15:45:18 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 1774541B31; Mon, 15 May 2023 15:45:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 1774541B31 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.a=rsa-sha256 header.s=google header.b=gImqqK7E X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qOTqGlr65Q_4; Mon, 15 May 2023 15:45:16 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 25E0341B19; Mon, 15 May 2023 15:45:16 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 25E0341B19 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id EEADDC0036; Mon, 15 May 2023 15:45:15 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 72676C002A for ; Mon, 15 May 2023 15:45:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 4CEAF60B12 for ; Mon, 15 May 2023 15:45:14 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 4CEAF60B12 Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.a=rsa-sha256 header.s=google header.b=gImqqK7E X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HR622yeRLK9q for ; Mon, 15 May 2023 15:45:13 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6038760AE6 Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [IPv6:2a00:1450:4864:20::532]) by smtp3.osuosl.org (Postfix) with ESMTPS id 6038760AE6 for ; Mon, 15 May 2023 15:45:13 +0000 (UTC) Received: by mail-ed1-x532.google.com with SMTP id 4fb4d7f45d1cf-50bc4bc2880so19836478a12.2 for ; Mon, 15 May 2023 08:45:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; t=1684165511; x=1686757511; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=XPEGaC+x0TYEuX7HHgeWtz7pyNM4DGEHBqF/DDq7Iw0=; b=gImqqK7Ez4ILwctNY0AcR7B6kOcnLZLrABJErDV0yJSdTCJO24RZwul28fa1YXP4J5 bKoHYDUdz0GrdUt1uvyJRzrL7wHWnEE7CFgoVdIxKph77hkZpY5hXoE8CN2AnKNyhvPE 1WD55DRKkWgfPnF1cu9nglLrdj/x+IQwngXX0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684165511; x=1686757511; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XPEGaC+x0TYEuX7HHgeWtz7pyNM4DGEHBqF/DDq7Iw0=; b=HoV3LBrwgGFAyBG0jsJdT7t3OVdDwQZV/vtzKVISWCWMHpi8Plvf7M2T17ddlFWgbv bc9nXygm2FJfCJST6fwC37RK8q69ztTTETYwywNz9f96t+2V4OfObFtTX/mzm0SGtnMO MX1JnGrXnUPEr6+eu/sxjE8Z+sg6wEzRcmcJbtG87XrVhrD67QYhdmeGBUsuNhFO63+H ecIcVhz2rwLILqoDXK9DMVBo3PnicRESqAZkW5AHzgpQ4rTgIl4zwoNMmj+TlEOX+HGJ c5KJPnjrchh68g6wxvHeq3tCA80d+ZlmWm2H+WR/qQO81rwtmLdzunCkm09JWEUJxM+n WO9A== X-Gm-Message-State: AC+VfDzL0JQmaHbU+t0k5yD2mPtqzoQYkaA0egHGZevvuVzbDGTyuhkX 9Cc91UomTaWHPDQcecnS2ItBiNBSMt9sUjLW87dDjAkg X-Google-Smtp-Source: ACHHUZ7TTRaXXbVxxry8seKbMqqGxyU4p3jJF4LGCJ9A1gs+EmUsACiZhn9cbkSIQYiwJXH16gyISw== X-Received: by 2002:a17:907:1614:b0:956:f4f8:23b6 with SMTP id hb20-20020a170907161400b00956f4f823b6mr36575670ejc.43.1684165511245; Mon, 15 May 2023 08:45:11 -0700 (PDT) Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com. [209.85.218.54]) by smtp.gmail.com with ESMTPSA id hg14-20020a1709072cce00b009662de1937dsm9680060ejc.131.2023.05.15.08.45.09 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 15 May 2023 08:45:10 -0700 (PDT) Received: by mail-ej1-f54.google.com with SMTP id a640c23a62f3a-965cc5170bdso1986312366b.2 for ; Mon, 15 May 2023 08:45:09 -0700 (PDT) X-Received: by 2002:a17:907:70d:b0:94b:d57e:9d4b with SMTP id xb13-20020a170907070d00b0094bd57e9d4bmr30521770ejb.2.1684165509441; Mon, 15 May 2023 08:45:09 -0700 (PDT) MIME-Version: 1.0 References: <20230202232517.8695-1-michael.christie@oracle.com> <20230202232517.8695-9-michael.christie@oracle.com> <78c5e150-26cf-7724-74ee-4a0b16b944b1@oracle.com> <48842e92-835e-bc3f-7118-48b8f415f532@leemhuis.info> <20230515-vollrausch-liebgeworden-2765f3ca3540@brauner> In-Reply-To: <20230515-vollrausch-liebgeworden-2765f3ca3540@brauner> From: Linus Torvalds Date: Mon, 15 May 2023 08:44:52 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v11 8/8] vhost: use vhost_tasks for worker threads To: Christian Brauner Cc: Jens Axboe , Linux kernel regressions list , mst@redhat.com, linux-kernel@vger.kernel.org, konrad.wilk@oracle.com, Thorsten Leemhuis , virtualization@lists.linux-foundation.org, hch@infradead.org, ebiederm@xmission.com, stefanha@redhat.com, nicolas.dichtel@6wind.com X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" T24gTW9uLCBNYXkgMTUsIDIwMjMgYXQgNzoyM+KAr0FNIENocmlzdGlhbiBCcmF1bmVyIDxicmF1 bmVyQGtlcm5lbC5vcmc+IHdyb3RlOgo+Cj4gU28gSSB0aGluayB3ZSB3aWxsIGJlIGFibGUgdG8g YWRkcmVzcyAoMSkgYW5kICgyKSBieSBtYWtpbmcgdmhvc3QgdGFza3MKPiBwcm9wZXIgdGhyZWFk cyBhbmQgYmxvY2tpbmcgZXZlcnkgc2lnbmFsIGV4Y2VwdCBmb3IgU0lHS0lMTCBhbmQgU0lHU1RP UAo+IGFuZCB0aGVuIGhhdmluZyB2aG9zdCBoYW5kbGUgZ2V0X3NpZ25hbCgpIC0gYXMgeW91IG1l bnRpb25lZCAtIHRoZSBzYW1lCj4gd2F5IGlvIHVyaW5nIGFscmVhZHkgZG9lcy4gV2Ugc2hvdWxk IGFsc28gcmVtb3ZlIHRoZSBpbmdvcmVfc2lnbmFscwo+IHRoaW5nIGNvbXBsZXRlbHkgaW1oby4g SSBkb24ndCB0aGluayB3ZSBldmVyIHdhbnQgdG8gZG8gdGhpcyB3aXRoIHVzZXIKPiB3b3JrZXJz LgoKUmlnaHQuIFRoYXQncyB3aGF0IElPX1VSSU5HIGRvZXM6CgogICAgICAgIGlmIChhcmdzLT5p b190aHJlYWQpIHsKICAgICAgICAgICAgICAgIC8qCiAgICAgICAgICAgICAgICAgKiBNYXJrIHVz IGFuIElPIHdvcmtlciwgYW5kIGJsb2NrIGFueSBzaWduYWwgdGhhdCBpc24ndAogICAgICAgICAg ICAgICAgICogZmF0YWwgb3IgU1RPUAogICAgICAgICAgICAgICAgICovCiAgICAgICAgICAgICAg ICBwLT5mbGFncyB8PSBQRl9JT19XT1JLRVI7CiAgICAgICAgICAgICAgICBzaWdpbml0c2V0aW52 KCZwLT5ibG9ja2VkLCBzaWdtYXNrKFNJR0tJTEwpfHNpZ21hc2soU0lHU1RPUCkpOwogICAgICAg IH0KCmFuZCBJIHJlYWxseSB0aGluayB0aGF0IHZob3N0IHNob3VsZCBiYXNpY2FsbHkgZG8gZXhh Y3RseSB3aGF0IGlvX3VyaW5nIGRvZXMuCgpOb3QgYmVjYXVzZSBpb191cmluZyBmdW5kYW1lbnRh bGx5IGdvdCB0aGlzIHJpZ2h0IC0gYnV0IHNpbXBseSBiZWNhdXNlCmlvX3VyaW5nIGhhZCBhbG1v c3QgYWxsIHRoZSBzYW1lIGJ1Z3MgKGFuZCB0aGVuIHNvbWUpLCBhbmQgd2hhdCB0aGUKaW9fdXJp bmcgd29ya2VyIHRocmVhZHMgZW5kZWQgdXAgZG9pbmcgd2FzIHRvIGJhc2ljYWxseSB6b29tIGlu IG9uCiJ0aGlzIHdvcmtzIi4KCkFuZCBpdCB6b29tZWQgaW4gb24gaXQgbGFyZ2VseSBieSBqdXN0 IGdvaW5nIGZvciAibWFrZSBpdCBsb29rIGFzIG11Y2gKYXMgcG9zc2libGUgYXMgYSByZWFsIHVz ZXIgdGhyZWFkIiwgYmVjYXVzZSBldmVyeSB0aW1lIHRoZSBrZXJuZWwKdGhyZWFkIGRpZCBzb21l dGhpbmcgZGlmZmVyZW50LCBpdCBqdXN0IGNhdXNlZCBwcm9ibGVtcy4KClNvIEkgdGhpbmsgdGhl IHBhdGNoIHNob3VsZCBqdXN0IGxvb2sgc29tZXRoaW5nIGxpa2UgdGhlIGF0dGFjaGVkLgpNaWtl LCBjYW4geW91IHRlc3QgdGhpcyBvbiB3aGF0ZXZlciB2aG9zdCB0ZXN0LXN1aXRlPwoKSSBkaWQg Y29uc2lkZXIgZ2V0dGluZyByaWQgb2YgIi5pZ25vcmVfc2lnbmFscyIgZW50aXJlbHksIGFuZCBp bnN0ZWFkCmp1c3Qga2V5aW5nIHRoZSAiYmxvY2sgc2lnbmFscyIgYmVoYXZpb3Igb2ZmIHRoZSAi LnVzZXJfd29ya2VyIiBmbGFnLgpCdXQgdGhpcyBhcHByb2FjaCBkb2Vzbid0IHNlZW0gd3Jvbmcg ZWl0aGVyLCBhbmQgSSBkb24ndCB0aGluayBpdCdzCndyb25nIHRvIG1ha2UgdGhlIGNyZWF0ZV9p b190aHJlYWQoKSBmdW5jdGlvbiBzYXkgdGhhdAoiLmlnbm9yZV9zaWduYWxzID0gMSIgdGhpbmcg ZXhwbGljaXRseSwgcmF0aGVyIHRoYW4ga2V5IGl0IG9mZiB0aGUKIi5pb190aHJlYWQiIGZsYWcu CgpKZW5zL0NocmlzdGlhbiAtIGNvbW1lbnRzPwoKU2xpZ2h0bHkgcmVsYXRlZCB0byB0aGlzIGFs bDogSSB0aGluayB2aG9zdCBzaG91bGQgYWxzbyBkbwpDTE9ORV9GSUxFUywgYW5kIGdldCByaWQg b2YgdGhlIHdob2xlICIubm9fZmlsZXMiIHRoaW5nLiBBZ2FpbiwgaWYKdmhvc3QgZG9lc24ndCB1 c2UgYW55IGZpbGVzLCBpdCBzaG91bGRuJ3QgbWF0dGVyLCBhbmQgbG9va2luZwpkaWZmZXJlbnQg anVzdCB0byBiZSBkaWZmZXJlbnQgaXMgd3JvbmcuIEJ1dCBpZiB2aG9zdCBkb2Vzbid0IHVzZSBh bnkKZmlsZXMsIHRoZSBjdXJyZW50IHNpdHVhdGlvbiBzaG91bGRuJ3QgYmUgYSBidWcgZWl0aGVy LgoKICAgICAgICAgICAgICAgICAgICAgTGludXMKX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX18KVmlydHVhbGl6YXRpb24gbWFpbGluZyBsaXN0ClZpcnR1YWxp emF0aW9uQGxpc3RzLmxpbnV4LWZvdW5kYXRpb24ub3JnCmh0dHBzOi8vbGlzdHMubGludXhmb3Vu ZGF0aW9uLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3ZpcnR1YWxpemF0aW9u