All of lore.kernel.org
 help / color / mirror / Atom feed
From: Andres Freund <andres@anarazel.de>
To: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Xuan Zhuo <xuanzhuo@linux.alibaba.com>,
	Jason Wang <jasowang@redhat.com>,
	"David S. Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@google.com>,
	Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
	virtualization@lists.linux-foundation.org,
	netdev@vger.kernel.org,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Jens Axboe <axboe@kernel.dk>,
	James Bottomley <James.Bottomley@hansenpartnership.com>,
	"Martin K. Petersen" <martin.petersen@oracle.com>,
	Guenter Roeck <linux@roeck-us.net>,
	linux-kernel@vger.kernel.org,
	Greg KH <gregkh@linuxfoundation.org>,
	c@redhat.com
Subject: Re: upstream kernel crashes
Date: Mon, 15 Aug 2022 10:46:17 -0700	[thread overview]
Message-ID: <20220815174617.z4chnftzcbv6frqr@awork3.anarazel.de> (raw)
In-Reply-To: <20220815124748-mutt-send-email-mst@kernel.org>

Hi,

On 2022-08-15 12:50:52 -0400, Michael S. Tsirkin wrote:
> On Mon, Aug 15, 2022 at 09:45:03AM -0700, Andres Freund wrote:
> > Hi,
> > 
> > On 2022-08-15 11:40:59 -0400, Michael S. Tsirkin wrote:
> > > OK so this gives us a quick revert as a solution for now.
> > > Next, I would appreciate it if you just try this simple hack.
> > > If it crashes we either have a long standing problem in virtio
> > > code or more likely a gcp bug where it can't handle smaller
> > > rings than what device requestes.
> > > Thanks!
> > 
> > I applied the below and the problem persists.
> > 
> > [...]
>
> Okay!

Just checking - I applied and tested this atop 6.0-rc1, correct? Or did you
want me to test it with the 762faee5a267 reverted? I guess what you're trying
to test if a smaller queue than what's requested you'd want to do so without
the problematic patch applied...


> And just to be 100% sure, can you try the following on top of 5.19:

> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index 623906b4996c..6f4e54a618bc 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -208,6 +208,9 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>  		return ERR_PTR(-EINVAL);
>  	}
>  
> +	if (num > 1024)
> +		num = 1024;
> +
>  	info->msix_vector = msix_vec;
>  
>  	/* create the vring */
> 
> -- 

Either way, I did this, and there are no issues that I could observe. No
oopses, no broken networking. But:

To make sure it does something I added a debugging printk - which doesn't show
up. I assume this is at a point at least earlyprintk should work (which I see
getting enabled via serial)?

Greetings,

Andres Freund

WARNING: multiple messages have this Message-ID (diff)
From: Andres Freund <andres@anarazel.de>
To: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Jens Axboe <axboe@kernel.dk>,
	"Martin K. Petersen" <martin.petersen@oracle.com>,
	netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org,
	James Bottomley <James.Bottomley@hansenpartnership.com>,
	Eric Dumazet <edumazet@google.com>,
	Greg KH <gregkh@linuxfoundation.org>,
	c@redhat.com, Jakub Kicinski <kuba@kernel.org>,
	Paolo Abeni <pabeni@redhat.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	"David S. Miller" <davem@davemloft.net>,
	Guenter Roeck <linux@roeck-us.net>
Subject: Re: upstream kernel crashes
Date: Mon, 15 Aug 2022 10:46:17 -0700	[thread overview]
Message-ID: <20220815174617.z4chnftzcbv6frqr@awork3.anarazel.de> (raw)
In-Reply-To: <20220815124748-mutt-send-email-mst@kernel.org>

Hi,

On 2022-08-15 12:50:52 -0400, Michael S. Tsirkin wrote:
> On Mon, Aug 15, 2022 at 09:45:03AM -0700, Andres Freund wrote:
> > Hi,
> > 
> > On 2022-08-15 11:40:59 -0400, Michael S. Tsirkin wrote:
> > > OK so this gives us a quick revert as a solution for now.
> > > Next, I would appreciate it if you just try this simple hack.
> > > If it crashes we either have a long standing problem in virtio
> > > code or more likely a gcp bug where it can't handle smaller
> > > rings than what device requestes.
> > > Thanks!
> > 
> > I applied the below and the problem persists.
> > 
> > [...]
>
> Okay!

Just checking - I applied and tested this atop 6.0-rc1, correct? Or did you
want me to test it with the 762faee5a267 reverted? I guess what you're trying
to test if a smaller queue than what's requested you'd want to do so without
the problematic patch applied...


> And just to be 100% sure, can you try the following on top of 5.19:

> diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
> index 623906b4996c..6f4e54a618bc 100644
> --- a/drivers/virtio/virtio_pci_modern.c
> +++ b/drivers/virtio/virtio_pci_modern.c
> @@ -208,6 +208,9 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
>  		return ERR_PTR(-EINVAL);
>  	}
>  
> +	if (num > 1024)
> +		num = 1024;
> +
>  	info->msix_vector = msix_vec;
>  
>  	/* create the vring */
> 
> -- 

Either way, I did this, and there are no issues that I could observe. No
oopses, no broken networking. But:

To make sure it does something I added a debugging printk - which doesn't show
up. I assume this is at a point at least earlyprintk should work (which I see
getting enabled via serial)?

Greetings,

Andres Freund
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

  reply	other threads:[~2022-08-15 17:46 UTC|newest]

Thread overview: 62+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-08-14 21:26 upstream kernel crashes Guenter Roeck
2022-08-14 21:40 ` Linus Torvalds
2022-08-14 22:37   ` Andres Freund
2022-08-14 22:47     ` Linus Torvalds
2022-08-15  1:04       ` Jens Axboe
2022-08-15  1:36         ` Andres Freund
2022-08-15  3:18           ` Linus Torvalds
2022-08-15  7:11             ` Andres Freund
2022-08-15  7:29               ` Michael S. Tsirkin
2022-08-15  7:46                 ` Andres Freund
2022-08-15  7:53                   ` Michael S. Tsirkin
2022-08-15  8:02                   ` Michael S. Tsirkin
2022-08-15  8:02                     ` Michael S. Tsirkin
2022-08-15  7:51               ` Michael S. Tsirkin
2022-08-15  8:15                 ` Andres Freund
2022-08-15  8:28                   ` Michael S. Tsirkin
2022-08-15  8:34                     ` Andres Freund
2022-08-15 15:40                       ` Michael S. Tsirkin
2022-08-15 15:40                         ` Michael S. Tsirkin
2022-08-15 16:45                         ` Andres Freund
2022-08-15 16:45                           ` Andres Freund
2022-08-15 16:50                           ` Michael S. Tsirkin
2022-08-15 16:50                             ` Michael S. Tsirkin
2022-08-15 17:46                             ` Andres Freund [this message]
2022-08-15 17:46                               ` Andres Freund
2022-08-15 20:21                               ` Michael S. Tsirkin
2022-08-15 20:21                                 ` Michael S. Tsirkin
2022-08-15 20:53                                 ` Andres Freund
2022-08-15 20:53                                   ` Andres Freund
2022-08-15 21:04                                   ` Andres Freund
2022-08-15 21:04                                     ` Andres Freund
2022-08-15 21:10                                     ` Andres Freund
2022-08-15 21:10                                       ` Andres Freund
2022-08-15 21:32                                   ` Michael S. Tsirkin
2022-08-15 21:32                                     ` Michael S. Tsirkin
2022-08-16  2:45                                     ` Xuan Zhuo
2022-08-16  2:45                                       ` Xuan Zhuo
2022-08-17  6:13                                     ` Dmitry Vyukov
2022-08-17  6:13                                       ` Dmitry Vyukov via Virtualization
2022-08-17  6:36                                       ` Xuan Zhuo
2022-08-17  6:36                                         ` Xuan Zhuo
2022-08-17 10:53                                         ` Michael S. Tsirkin
2022-08-17 10:53                                           ` Michael S. Tsirkin
2022-08-17 15:58                                         ` Linus Torvalds
2022-08-17 15:58                                           ` Linus Torvalds
2022-08-18  1:55                                           ` Xuan Zhuo
2022-08-18  1:55                                             ` Xuan Zhuo
2022-08-15 20:45                             ` Guenter Roeck
2022-08-15 20:45                               ` Guenter Roeck
2022-08-15  6:36           ` Michael S. Tsirkin
2022-08-15  7:17             ` Andres Freund
2022-08-15  7:43               ` Michael S. Tsirkin
2022-08-15  1:17       ` Guenter Roeck
2022-08-15  1:29         ` Jens Axboe
2022-08-15  9:43 ` Michael S. Tsirkin
2022-08-15 15:49   ` Guenter Roeck
2022-08-15 16:01     ` Michael S. Tsirkin
2022-08-15 18:22       ` Guenter Roeck
2022-08-15 18:37         ` Linus Torvalds
2022-08-15 20:38           ` Guenter Roeck
2022-08-17 17:12 ` Linus Torvalds
2022-08-18  1:08   ` Andres Freund

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220815174617.z4chnftzcbv6frqr@awork3.anarazel.de \
    --to=andres@anarazel.de \
    --cc=James.Bottomley@hansenpartnership.com \
    --cc=axboe@kernel.dk \
    --cc=c@redhat.com \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=jasowang@redhat.com \
    --cc=kuba@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux@roeck-us.net \
    --cc=martin.petersen@oracle.com \
    --cc=mst@redhat.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=torvalds@linux-foundation.org \
    --cc=virtualization@lists.linux-foundation.org \
    --cc=xuanzhuo@linux.alibaba.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.