From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D03D2C10F1B for ; Tue, 27 Dec 2022 09:14:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229880AbiL0JOA (ORCPT ); Tue, 27 Dec 2022 04:14:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47574 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229766AbiL0JNu (ORCPT ); Tue, 27 Dec 2022 04:13:50 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F8388FE6 for ; Tue, 27 Dec 2022 01:13:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1672132386; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8Jx6o34uyqpu/PSUYQD9I3JF4UvYdpXWHh4RWvb/DkI=; b=jOWGCQHDmLJ1BPzyR3iquygrvg/87/Se39DTvKhIHpxraCY5xJU2tBUlH+Y9ReyrfpTx1V 8Eqv7IkKaqXTDvewQSuE9lor81IhwKoWR16ejiN6Un4nc3r8xGaNs+/wLpmXIiUGPjR2c1 Zz+o/NYpJEETVnaiCLM4NYXp2XTscog= Received: from mail-pj1-f69.google.com (mail-pj1-f69.google.com [209.85.216.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-493-tbQNy4_mMXGjGVlyRCgGmA-1; Tue, 27 Dec 2022 04:13:04 -0500 X-MC-Unique: tbQNy4_mMXGjGVlyRCgGmA-1 Received: by mail-pj1-f69.google.com with SMTP id mn20-20020a17090b189400b0021941492f66so9699789pjb.0 for ; Tue, 27 Dec 2022 01:13:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8Jx6o34uyqpu/PSUYQD9I3JF4UvYdpXWHh4RWvb/DkI=; b=E7U3rWELbxM4cgc2uWk2iem4/0HKMnRNaaGxv1TeAc/n4fYpUs7Cek+C1ItIJPR3WB kEiBisAhKVMx8bCCPZ1mpOlmww1xiMeId9IZBkZcNkofUSsTgSMnt0TCC8JEVekJQ7LR ykPqThjWN0twskn1zDPXpYYUNCnfOa4BNKrC6d2Gb8VTc8w9xhb0Y8eLubST6iDMLtEs q5cRQ5lg6ZlFYk2KeCrvUYVDXGTixKXW0x/KtOvbVDBhkUbP6LJeWLUiMUMIV1YOdpx4 wkROasMvazr8Nv5zAu3cMEf1iWd8KcRNycQKDBZBLrhCDD2eV0zcfQ+ICzmInaN4Gapr rWNQ== X-Gm-Message-State: AFqh2kqib5PWem62CcplYamftiQMaYr53yape7VqeMWFu+qIkaSM1RvW xzuWl0+UKpAa1Y9pbJXCRioJPnS6b89Y/TOojB7/VUDtkcpOl8hWMNSqBkuc9xrJgj9g/DXZsJ2 0ahVxxG7de9ykQxYcWQA/duRd X-Received: by 2002:aa7:960d:0:b0:56c:3fba:c5ca with SMTP id q13-20020aa7960d000000b0056c3fbac5camr38459828pfg.16.1672132383920; Tue, 27 Dec 2022 01:13:03 -0800 (PST) X-Google-Smtp-Source: AMrXdXs0uW6pvl7WTQFTWTYhpvNudCBZ3Vz1d8332wzs1ANd1EMyd8E6UvZETJjAdYG2CpgZ74SUjQ== X-Received: by 2002:aa7:960d:0:b0:56c:3fba:c5ca with SMTP id q13-20020aa7960d000000b0056c3fbac5camr38459804pfg.16.1672132383657; Tue, 27 Dec 2022 01:13:03 -0800 (PST) Received: from [10.72.13.143] ([43.228.180.230]) by smtp.gmail.com with ESMTPSA id q13-20020aa7960d000000b005752b9fec48sm8142551pfg.204.2022.12.27.01.13.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 27 Dec 2022 01:13:03 -0800 (PST) Message-ID: Date: Tue, 27 Dec 2022 17:12:58 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.6.1 Subject: Re: [PATCH 3/4] virtio_ring: introduce a per virtqueue waitqueue Content-Language: en-US To: "Michael S. Tsirkin" Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, maxime.coquelin@redhat.com, alvaro.karsz@solid-run.com, eperezma@redhat.com References: <20221226074908.8154-1-jasowang@redhat.com> <20221226074908.8154-4-jasowang@redhat.com> <20221226183705-mutt-send-email-mst@kernel.org> <20221227022255-mutt-send-email-mst@kernel.org> From: Jason Wang In-Reply-To: <20221227022255-mutt-send-email-mst@kernel.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 在 2022/12/27 15:33, Michael S. Tsirkin 写道: > On Tue, Dec 27, 2022 at 12:30:35PM +0800, Jason Wang wrote: >>> But device is still going and will later use the buffers. >>> >>> Same for timeout really. >> Avoiding infinite wait/poll is one of the goals, another is to sleep. >> If we think the timeout is hard, we can start from the wait. >> >> Thanks > If the goal is to avoid disrupting traffic while CVQ is in use, > that sounds more reasonable. E.g. someone is turning on promisc, > a spike in CPU usage might be unwelcome. Yes, this would be more obvious is UP is used. > > things we should be careful to address then: > 1- debugging. Currently it's easy to see a warning if CPU is stuck > in a loop for a while, and we also get a backtrace. > E.g. with this - how do we know who has the RTNL? > We need to integrate with kernel/watchdog.c for good results > and to make sure policy is consistent. That's fine, will consider this. > 2- overhead. In a very common scenario when device is in hypervisor, > programming timers etc has a very high overhead, at bootup > lots of CVQ commands are run and slowing boot down is not nice. > let's poll for a bit before waiting? Then we go back to the question of choosing a good timeout for poll. And poll seems problematic in the case of UP, scheduler might not have the chance to run. > 3- suprise removal. need to wake up thread in some way. what about > other cases of device breakage - is there a chance this > introduces new bugs around that? at least enumerate them please. The current code did: 1) check for vq->broken 2) wakeup during BAD_RING() So we won't end up with a never woke up process which should be fine. Thanks > > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 18D23C10F1B for ; Tue, 27 Dec 2022 09:13:18 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 5CB9460BE7; Tue, 27 Dec 2022 09:13:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 5CB9460BE7 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=bwwYu5C7 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cxyxVEnasodP; Tue, 27 Dec 2022 09:13:17 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 07B3460BD6; Tue, 27 Dec 2022 09:13:16 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 07B3460BD6 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id BD625C0032; Tue, 27 Dec 2022 09:13:16 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id AA84CC002D for ; Tue, 27 Dec 2022 09:13:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 7EBB081334 for ; Tue, 27 Dec 2022 09:13:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 7EBB081334 Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=bwwYu5C7 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bsIAjWyPd9Pu for ; Tue, 27 Dec 2022 09:13:14 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org B077381331 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id B077381331 for ; Tue, 27 Dec 2022 09:13:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1672132393; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8Jx6o34uyqpu/PSUYQD9I3JF4UvYdpXWHh4RWvb/DkI=; b=bwwYu5C7t/6cYE7x7iYl75j1Z5e12FoGCfXm7a1tvAxI2jL6p9rPqDkUA4lxXJWOGzmxl9 03zuCPlhiP/2xphedFVn2xluXB3KrO4S83ntsDEVEILFL+5q1zBe58wsdxloDjRSFcHiiW t1smLzztnMawwPQTUSSQ9nn00eLQuyc= Received: from mail-pj1-f71.google.com (mail-pj1-f71.google.com [209.85.216.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-54-gmCMmbepM8uh3_RsI7hGVQ-1; Tue, 27 Dec 2022 04:13:04 -0500 X-MC-Unique: gmCMmbepM8uh3_RsI7hGVQ-1 Received: by mail-pj1-f71.google.com with SMTP id ot18-20020a17090b3b5200b00219c3543529so9508109pjb.1 for ; Tue, 27 Dec 2022 01:13:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8Jx6o34uyqpu/PSUYQD9I3JF4UvYdpXWHh4RWvb/DkI=; b=ZVaWjWR8TW4P4qm/a5ct4KYKHhRZRIJp29y4wMG4ak2wG75Wqp87goVP+CJREX6o4k x3OQoypPQBqbEFcgJpn/BsRoQXI+AuBkO2AQ1pD2rz6Q2fSBFHC3/qEV5Q0Wv+vW1viK rBQpzohVm9bin8qECC0eVApqK/r4AV9rsxHI/bWJF5/vDlKTKjecNvmD3pH4IfNifdQc YkXbhkHaHpaHHYhseC41KcSoTu+l6lYWE05S14XPA4FxdjmGQqhkQ24ey9pEFf7ckRp3 Hd3cHmqpXBa/1vyRxfFpOGCHYnV9nX7UrZD9AvnJFHBQQ17OstiINEIZkdSZ2+AUBfM3 A8yQ== X-Gm-Message-State: AFqh2kpeUZmAwwgTYqhI50xMBQjRJMWGe0p54jLeMm5waFUVX5M53QY/ Q3FbfPvVMBhQHutiYQWMOz4PcnZnMR+3M+w5n4igosgnR8SdItfVnaQ9Hj98Z4eX2pZVFx/RZ2P AYMz9ZwG9CqzxaQBh1//YMUkBYZzQerPhBEZcd+oxBg== X-Received: by 2002:aa7:960d:0:b0:56c:3fba:c5ca with SMTP id q13-20020aa7960d000000b0056c3fbac5camr38459833pfg.16.1672132383949; Tue, 27 Dec 2022 01:13:03 -0800 (PST) X-Google-Smtp-Source: AMrXdXs0uW6pvl7WTQFTWTYhpvNudCBZ3Vz1d8332wzs1ANd1EMyd8E6UvZETJjAdYG2CpgZ74SUjQ== X-Received: by 2002:aa7:960d:0:b0:56c:3fba:c5ca with SMTP id q13-20020aa7960d000000b0056c3fbac5camr38459804pfg.16.1672132383657; Tue, 27 Dec 2022 01:13:03 -0800 (PST) Received: from [10.72.13.143] ([43.228.180.230]) by smtp.gmail.com with ESMTPSA id q13-20020aa7960d000000b005752b9fec48sm8142551pfg.204.2022.12.27.01.13.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 27 Dec 2022 01:13:03 -0800 (PST) Message-ID: Date: Tue, 27 Dec 2022 17:12:58 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.6.1 Subject: Re: [PATCH 3/4] virtio_ring: introduce a per virtqueue waitqueue To: "Michael S. Tsirkin" References: <20221226074908.8154-1-jasowang@redhat.com> <20221226074908.8154-4-jasowang@redhat.com> <20221226183705-mutt-send-email-mst@kernel.org> <20221227022255-mutt-send-email-mst@kernel.org> From: Jason Wang In-Reply-To: <20221227022255-mutt-send-email-mst@kernel.org> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, eperezma@redhat.com, edumazet@google.com, maxime.coquelin@redhat.com, kuba@kernel.org, pabeni@redhat.com, davem@davemloft.net X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" CuWcqCAyMDIyLzEyLzI3IDE1OjMzLCBNaWNoYWVsIFMuIFRzaXJraW4g5YaZ6YGTOgo+IE9uIFR1 ZSwgRGVjIDI3LCAyMDIyIGF0IDEyOjMwOjM1UE0gKzA4MDAsIEphc29uIFdhbmcgd3JvdGU6Cj4+ PiBCdXQgZGV2aWNlIGlzIHN0aWxsIGdvaW5nIGFuZCB3aWxsIGxhdGVyIHVzZSB0aGUgYnVmZmVy cy4KPj4+Cj4+PiBTYW1lIGZvciB0aW1lb3V0IHJlYWxseS4KPj4gQXZvaWRpbmcgaW5maW5pdGUg d2FpdC9wb2xsIGlzIG9uZSBvZiB0aGUgZ29hbHMsIGFub3RoZXIgaXMgdG8gc2xlZXAuCj4+IElm IHdlIHRoaW5rIHRoZSB0aW1lb3V0IGlzIGhhcmQsIHdlIGNhbiBzdGFydCBmcm9tIHRoZSB3YWl0 Lgo+Pgo+PiBUaGFua3MKPiBJZiB0aGUgZ29hbCBpcyB0byBhdm9pZCBkaXNydXB0aW5nIHRyYWZm aWMgd2hpbGUgQ1ZRIGlzIGluIHVzZSwKPiB0aGF0IHNvdW5kcyBtb3JlIHJlYXNvbmFibGUuIEUu Zy4gc29tZW9uZSBpcyB0dXJuaW5nIG9uIHByb21pc2MsCj4gYSBzcGlrZSBpbiBDUFUgdXNhZ2Ug bWlnaHQgYmUgdW53ZWxjb21lLgoKClllcywgdGhpcyB3b3VsZCBiZSBtb3JlIG9idmlvdXMgaXMg VVAgaXMgdXNlZC4KCgo+Cj4gdGhpbmdzIHdlIHNob3VsZCBiZSBjYXJlZnVsIHRvIGFkZHJlc3Mg dGhlbjoKPiAxLSBkZWJ1Z2dpbmcuIEN1cnJlbnRseSBpdCdzIGVhc3kgdG8gc2VlIGEgd2Fybmlu ZyBpZiBDUFUgaXMgc3R1Y2sKPiAgICAgaW4gYSBsb29wIGZvciBhIHdoaWxlLCBhbmQgd2UgYWxz byBnZXQgYSBiYWNrdHJhY2UuCj4gICAgIEUuZy4gd2l0aCB0aGlzIC0gaG93IGRvIHdlIGtub3cg d2hvIGhhcyB0aGUgUlROTD8KPiAgICAgV2UgbmVlZCB0byBpbnRlZ3JhdGUgd2l0aCBrZXJuZWwv d2F0Y2hkb2cuYyBmb3IgZ29vZCByZXN1bHRzCj4gICAgIGFuZCB0byBtYWtlIHN1cmUgcG9saWN5 IGlzIGNvbnNpc3RlbnQuCgoKVGhhdCdzIGZpbmUsIHdpbGwgY29uc2lkZXIgdGhpcy4KCgo+IDIt IG92ZXJoZWFkLiBJbiBhIHZlcnkgY29tbW9uIHNjZW5hcmlvIHdoZW4gZGV2aWNlIGlzIGluIGh5 cGVydmlzb3IsCj4gICAgIHByb2dyYW1taW5nIHRpbWVycyBldGMgaGFzIGEgdmVyeSBoaWdoIG92 ZXJoZWFkLCBhdCBib290dXAKPiAgICAgbG90cyBvZiBDVlEgY29tbWFuZHMgYXJlIHJ1biBhbmQg c2xvd2luZyBib290IGRvd24gaXMgbm90IG5pY2UuCj4gICAgIGxldCdzIHBvbGwgZm9yIGEgYml0 IGJlZm9yZSB3YWl0aW5nPwoKClRoZW4gd2UgZ28gYmFjayB0byB0aGUgcXVlc3Rpb24gb2YgY2hv b3NpbmcgYSBnb29kIHRpbWVvdXQgZm9yIHBvbGwuIEFuZCAKcG9sbCBzZWVtcyBwcm9ibGVtYXRp YyBpbiB0aGUgY2FzZSBvZiBVUCwgc2NoZWR1bGVyIG1pZ2h0IG5vdCBoYXZlIHRoZSAKY2hhbmNl IHRvIHJ1bi4KCgo+IDMtIHN1cHJpc2UgcmVtb3ZhbC4gbmVlZCB0byB3YWtlIHVwIHRocmVhZCBp biBzb21lIHdheS4gd2hhdCBhYm91dAo+ICAgICBvdGhlciBjYXNlcyBvZiBkZXZpY2UgYnJlYWth Z2UgLSBpcyB0aGVyZSBhIGNoYW5jZSB0aGlzCj4gICAgIGludHJvZHVjZXMgbmV3IGJ1Z3MgYXJv dW5kIHRoYXQ/IGF0IGxlYXN0IGVudW1lcmF0ZSB0aGVtIHBsZWFzZS4KCgpUaGUgY3VycmVudCBj b2RlIGRpZDoKCjEpIGNoZWNrIGZvciB2cS0+YnJva2VuCjIpIHdha2V1cCBkdXJpbmcgQkFEX1JJ TkcoKQoKU28gd2Ugd29uJ3QgZW5kIHVwIHdpdGggYSBuZXZlciB3b2tlIHVwIHByb2Nlc3Mgd2hp Y2ggc2hvdWxkIGJlIGZpbmUuCgpUaGFua3MKCgo+Cj4KCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fClZpcnR1YWxpemF0aW9uIG1haWxpbmcgbGlzdApWaXJ0 dWFsaXphdGlvbkBsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZwpodHRwczovL2xpc3RzLmxpbnV4 Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9saXN0aW5mby92aXJ0dWFsaXphdGlvbg==