From: "Michael S. Tsirkin" <mst@redhat.com> To: Greg KH <gregkh@linuxfoundation.org> Cc: xuanzhuo@linux.alibaba.com, herbert@gondor.apana.org.au, arnd@arndb.de, amit@kernel.org, Xianting Tian <xianting.tian@linux.alibaba.com>, marcel@holtmann.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-bluetooth@vger.kernel.org, Xianting Tian <tianxianting.txt@alibaba-inc.com>, linux-crypto@vger.kernel.org, luiz.dentz@gmail.com, davem@davemloft.net, johan.hedberg@gmail.com Subject: Re: [PATCH 1/3] virtio-crypto: fixup potential cpu stall when free unused bufs Date: Fri, 9 Jun 2023 12:02:36 -0400 [thread overview] Message-ID: <20230609115840-mutt-send-email-mst@kernel.org> (raw) In-Reply-To: <2023060924-onion-armhole-803e@gregkh> On Fri, Jun 09, 2023 at 04:05:57PM +0200, Greg KH wrote: > On Fri, Jun 09, 2023 at 09:49:39PM +0800, Xianting Tian wrote: > > > > 在 2023/6/9 下午9:41, Greg KH 写道: > > > On Fri, Jun 09, 2023 at 03:39:24PM +0200, Greg KH wrote: > > > > On Fri, Jun 09, 2023 at 09:18:15PM +0800, Xianting Tian wrote: > > > > > From: Xianting Tian <tianxianting.txt@alibaba-inc.com> > > > > > > > > > > Cpu stall issue may happen if device is configured with multi queues > > > > > and large queue depth, so fix it. > > > > > > > > > > Signed-off-by: Xianting Tian <xianting.tian@linux.alibaba.com> > > > > > --- > > > > > drivers/crypto/virtio/virtio_crypto_core.c | 1 + > > > > > 1 file changed, 1 insertion(+) > > > > > > > > > > diff --git a/drivers/crypto/virtio/virtio_crypto_core.c b/drivers/crypto/virtio/virtio_crypto_core.c > > > > > index 1198bd306365..94849fa3bd74 100644 > > > > > --- a/drivers/crypto/virtio/virtio_crypto_core.c > > > > > +++ b/drivers/crypto/virtio/virtio_crypto_core.c > > > > > @@ -480,6 +480,7 @@ static void virtcrypto_free_unused_reqs(struct virtio_crypto *vcrypto) > > > > > kfree(vc_req->req_data); > > > > > kfree(vc_req->sgs); > > > > > } > > > > > + cond_resched(); > > > > that's not "fixing a stall", it is "call the scheduler because we are > > > > taking too long". The CPU isn't stalled at all, just busy. > > > > > > > > Are you sure this isn't just a bug in the code? Why is this code taking > > > > so long that you have to force the scheduler to run? This is almost > > > > always a sign that something else needs to be fixed instead. > > > And same comment on the other 2 patches, please fix this properly. > > > > > > Also, this is a tight loop that is just freeing memory, why is it taking > > > so long? Why do you want it to take longer (which is what you are doing > > > here), ideally it would be faster, not slower, so you are now slowing > > > down the system overall with this patchset, right? > > > > yes, it is the similar fix with one for virtio-net > > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/net/virtio_net.c?h=v6.4-rc5&id=f8bb5104394560e29017c25bcade4c6b7aabd108 <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/net/virtio_net.c?h=v6.4-rc5&id=f8bb5104394560e29017c25bcade4c6b7aabd108> Well that one actually at least describes the configuration: For multi-queue and large ring-size use case, the following error occurred when free_unused_bufs: rcu: INFO: rcu_sched self-detected stall on CPU. So a similar fix but not a similar commit log, this one lacks Fixes tag and description of what the problem is and when does it trigger. > I would argue that this too is incorrect, because why does freeing > memory take so long? You are correct that even that one lacks detailed explanation why does the patch help. And the explanation why it takes so long is exactly that we have very deep queues and a very large number of queues. What the patch does is gives scheduler a chance to do some work between the queues. > And again, you are making it take longer, is that > ok? > > thanks, > > greg k-h _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
WARNING: multiple messages have this Message-ID (diff)
From: "Michael S. Tsirkin" <mst@redhat.com> To: Greg KH <gregkh@linuxfoundation.org> Cc: Xianting Tian <xianting.tian@linux.alibaba.com>, arei.gonglei@huawei.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, herbert@gondor.apana.org.au, davem@davemloft.net, amit@kernel.org, arnd@arndb.de, marcel@holtmann.org, johan.hedberg@gmail.com, luiz.dentz@gmail.com, linux-bluetooth@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Xianting Tian <tianxianting.txt@alibaba-inc.com> Subject: Re: [PATCH 1/3] virtio-crypto: fixup potential cpu stall when free unused bufs Date: Fri, 9 Jun 2023 12:02:36 -0400 [thread overview] Message-ID: <20230609115840-mutt-send-email-mst@kernel.org> (raw) In-Reply-To: <2023060924-onion-armhole-803e@gregkh> On Fri, Jun 09, 2023 at 04:05:57PM +0200, Greg KH wrote: > On Fri, Jun 09, 2023 at 09:49:39PM +0800, Xianting Tian wrote: > > > > 在 2023/6/9 下午9:41, Greg KH 写道: > > > On Fri, Jun 09, 2023 at 03:39:24PM +0200, Greg KH wrote: > > > > On Fri, Jun 09, 2023 at 09:18:15PM +0800, Xianting Tian wrote: > > > > > From: Xianting Tian <tianxianting.txt@alibaba-inc.com> > > > > > > > > > > Cpu stall issue may happen if device is configured with multi queues > > > > > and large queue depth, so fix it. > > > > > > > > > > Signed-off-by: Xianting Tian <xianting.tian@linux.alibaba.com> > > > > > --- > > > > > drivers/crypto/virtio/virtio_crypto_core.c | 1 + > > > > > 1 file changed, 1 insertion(+) > > > > > > > > > > diff --git a/drivers/crypto/virtio/virtio_crypto_core.c b/drivers/crypto/virtio/virtio_crypto_core.c > > > > > index 1198bd306365..94849fa3bd74 100644 > > > > > --- a/drivers/crypto/virtio/virtio_crypto_core.c > > > > > +++ b/drivers/crypto/virtio/virtio_crypto_core.c > > > > > @@ -480,6 +480,7 @@ static void virtcrypto_free_unused_reqs(struct virtio_crypto *vcrypto) > > > > > kfree(vc_req->req_data); > > > > > kfree(vc_req->sgs); > > > > > } > > > > > + cond_resched(); > > > > that's not "fixing a stall", it is "call the scheduler because we are > > > > taking too long". The CPU isn't stalled at all, just busy. > > > > > > > > Are you sure this isn't just a bug in the code? Why is this code taking > > > > so long that you have to force the scheduler to run? This is almost > > > > always a sign that something else needs to be fixed instead. > > > And same comment on the other 2 patches, please fix this properly. > > > > > > Also, this is a tight loop that is just freeing memory, why is it taking > > > so long? Why do you want it to take longer (which is what you are doing > > > here), ideally it would be faster, not slower, so you are now slowing > > > down the system overall with this patchset, right? > > > > yes, it is the similar fix with one for virtio-net > > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/net/virtio_net.c?h=v6.4-rc5&id=f8bb5104394560e29017c25bcade4c6b7aabd108 <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/net/virtio_net.c?h=v6.4-rc5&id=f8bb5104394560e29017c25bcade4c6b7aabd108> Well that one actually at least describes the configuration: For multi-queue and large ring-size use case, the following error occurred when free_unused_bufs: rcu: INFO: rcu_sched self-detected stall on CPU. So a similar fix but not a similar commit log, this one lacks Fixes tag and description of what the problem is and when does it trigger. > I would argue that this too is incorrect, because why does freeing > memory take so long? You are correct that even that one lacks detailed explanation why does the patch help. And the explanation why it takes so long is exactly that we have very deep queues and a very large number of queues. What the patch does is gives scheduler a chance to do some work between the queues. > And again, you are making it take longer, is that > ok? > > thanks, > > greg k-h
next prev parent reply other threads:[~2023-06-09 16:02 UTC|newest] Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-06-09 13:18 [PATCH 0/3] fixup potential cpu stall Xianting Tian 2023-06-09 13:18 ` [PATCH 1/3] virtio-crypto: fixup potential cpu stall when free unused bufs Xianting Tian 2023-06-09 13:39 ` Greg KH 2023-06-09 13:39 ` Greg KH 2023-06-09 13:41 ` Greg KH 2023-06-09 13:41 ` Greg KH 2023-06-09 13:49 ` Xianting Tian 2023-06-09 14:05 ` Greg KH 2023-06-09 14:05 ` Greg KH 2023-06-09 14:25 ` Xianting Tian 2023-06-09 16:02 ` Michael S. Tsirkin [this message] 2023-06-09 16:02 ` Michael S. Tsirkin 2023-06-09 13:57 ` fixup potential cpu stall bluez.test.bot 2023-06-09 15:57 ` [PATCH 1/3] virtio-crypto: fixup potential cpu stall when free unused bufs Michael S. Tsirkin 2023-06-09 15:57 ` Michael S. Tsirkin 2023-06-10 3:20 ` Xianting Tian 2023-06-10 6:51 ` Greg KH 2023-06-10 6:51 ` Greg KH 2023-06-09 15:58 ` Michael S. Tsirkin 2023-06-09 15:58 ` Michael S. Tsirkin 2023-06-09 13:18 ` [PATCH 2/3] virtio_console: " Xianting Tian 2023-06-09 16:05 ` Michael S. Tsirkin 2023-06-09 16:05 ` Michael S. Tsirkin 2023-06-10 3:19 ` Xianting Tian 2023-06-09 13:18 ` [PATCH 3/3] virtio_bt: " Xianting Tian 2023-06-22 11:59 ` [PATCH 0/3] fixup potential cpu stall Michael S. Tsirkin 2023-06-22 11:59 ` Michael S. Tsirkin 2023-06-23 0:58 ` Xianting Tian 2023-07-06 20:10 ` patchwork-bot+bluetooth
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20230609115840-mutt-send-email-mst@kernel.org \ --to=mst@redhat.com \ --cc=amit@kernel.org \ --cc=arnd@arndb.de \ --cc=davem@davemloft.net \ --cc=gregkh@linuxfoundation.org \ --cc=herbert@gondor.apana.org.au \ --cc=johan.hedberg@gmail.com \ --cc=linux-bluetooth@vger.kernel.org \ --cc=linux-crypto@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=luiz.dentz@gmail.com \ --cc=marcel@holtmann.org \ --cc=tianxianting.txt@alibaba-inc.com \ --cc=virtualization@lists.linux-foundation.org \ --cc=xianting.tian@linux.alibaba.com \ --cc=xuanzhuo@linux.alibaba.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.