All of lore.kernel.org
 help / color / mirror / Atom feed
From: Mark McLoughlin <markmc@redhat.com>
To: Dustin Kirkland <kirkland@canonical.com>
Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org,
	Anthony Liguori <anthony@codemonkey.ws>,
	Scott Tsai <scottt.tw@gmail.com>
Subject: Re: [Qemu-devel] qemu-kvm-0.11 regression, crashes on older guests with virtio network
Date: Thu, 29 Oct 2009 09:16:43 +0000	[thread overview]
Message-ID: <1256807803.10825.39.camel@blaa> (raw)
In-Reply-To: <d9c105ea0910281222u76a61a2by3cc924e85d40a865@mail.gmail.com>

Hi Dustin,

On Wed, 2009-10-28 at 14:22 -0500, Dustin Kirkland wrote:
> I believe that we have identified a regression in qemu-kvm-0.11.0.

Regression versus which previous version of qemu-kvm?

> The kvm process crashes for older guests with virtio networking, when
> the guest's incoming network connection is saturated.  The subject
> guest is Ubuntu 8.04 Hardy, 2.6.24 kernel with virtio backports.

It'd be good to see your virtio_net code

e.g. it should not have the VIRTIO_NET_F_GUEST_TSO4 feature set if it
doesn't have the "big_packets" code in try_fill_recv()

> For your convenience, I have uploaded a stock, ~260MB Ubuntu 8.04 disk
> image (user/pw = ubuntu) that I'm using to reproduce the problem at:
>  * http://rookery.canonical.com/~kirkland/hardy.img.bz2
> 
> The command line is:
>  * sudo /usr/bin/kvm -m 512 -net nic,model=virtio -net
> tap,script=/home/kirkland/bin/bridge.sh -drive
> file=hardy.img,if=virtio,index=0,boot=on
> 
> The bridge script is:
>  * http://rookery.canonical.com/~kirkland/bridge.sh
> 
> I'm saturating the guest's incoming network connection, with:
>   user@guest$ nc -lp 1234 > /dev/null
> and
>   user@host$ cat /dev/urandom | nc -w 3 guest_ip 1234
> 
> In less than a second of sending, the vm crashes with:
>   virtio-net truncating packet

Assuming this is something like the virtio-net in 2.6.26, there was no
receivable buffers support so (as Scott points out) it must be that
we've read a packet from the tap device which is >1514 bytes (or >1524
bytes with IFF_VNET_HDR) but the guest has not supplied buffers which
are large enough to take it

One thing to check is that the tap device is being initialized by
qemu-kvm using TUNSETOFFLOAD with either zero or TUN_F_CSUM - i.e. GSO
should not be enabled, because the guest cannot handle large GSO packets

Another possibility is that the MTU on the bridge in the host is too
large and that's what's causing the large packets to be sent

I agree we shouldn't exit in this scenario, but let's figure out what's
causing it to occur

Cheers,
Mark.


WARNING: multiple messages have this Message-ID (diff)
From: Mark McLoughlin <markmc@redhat.com>
To: Dustin Kirkland <kirkland@canonical.com>
Cc: Scott Tsai <scottt.tw@gmail.com>,
	qemu-devel@nongnu.org, kvm@vger.kernel.org
Subject: Re: [Qemu-devel] qemu-kvm-0.11 regression, crashes on older guests with virtio network
Date: Thu, 29 Oct 2009 09:16:43 +0000	[thread overview]
Message-ID: <1256807803.10825.39.camel@blaa> (raw)
In-Reply-To: <d9c105ea0910281222u76a61a2by3cc924e85d40a865@mail.gmail.com>

Hi Dustin,

On Wed, 2009-10-28 at 14:22 -0500, Dustin Kirkland wrote:
> I believe that we have identified a regression in qemu-kvm-0.11.0.

Regression versus which previous version of qemu-kvm?

> The kvm process crashes for older guests with virtio networking, when
> the guest's incoming network connection is saturated.  The subject
> guest is Ubuntu 8.04 Hardy, 2.6.24 kernel with virtio backports.

It'd be good to see your virtio_net code

e.g. it should not have the VIRTIO_NET_F_GUEST_TSO4 feature set if it
doesn't have the "big_packets" code in try_fill_recv()

> For your convenience, I have uploaded a stock, ~260MB Ubuntu 8.04 disk
> image (user/pw = ubuntu) that I'm using to reproduce the problem at:
>  * http://rookery.canonical.com/~kirkland/hardy.img.bz2
> 
> The command line is:
>  * sudo /usr/bin/kvm -m 512 -net nic,model=virtio -net
> tap,script=/home/kirkland/bin/bridge.sh -drive
> file=hardy.img,if=virtio,index=0,boot=on
> 
> The bridge script is:
>  * http://rookery.canonical.com/~kirkland/bridge.sh
> 
> I'm saturating the guest's incoming network connection, with:
>   user@guest$ nc -lp 1234 > /dev/null
> and
>   user@host$ cat /dev/urandom | nc -w 3 guest_ip 1234
> 
> In less than a second of sending, the vm crashes with:
>   virtio-net truncating packet

Assuming this is something like the virtio-net in 2.6.26, there was no
receivable buffers support so (as Scott points out) it must be that
we've read a packet from the tap device which is >1514 bytes (or >1524
bytes with IFF_VNET_HDR) but the guest has not supplied buffers which
are large enough to take it

One thing to check is that the tap device is being initialized by
qemu-kvm using TUNSETOFFLOAD with either zero or TUN_F_CSUM - i.e. GSO
should not be enabled, because the guest cannot handle large GSO packets

Another possibility is that the MTU on the bridge in the host is too
large and that's what's causing the large packets to be sent

I agree we shouldn't exit in this scenario, but let's figure out what's
causing it to occur

Cheers,
Mark.

  parent reply	other threads:[~2009-10-29  9:18 UTC|newest]

Thread overview: 70+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-10-28 19:22 qemu-kvm-0.11 regression, crashes on older guests with virtio network Dustin Kirkland
2009-10-28 19:22 ` [Qemu-devel] " Dustin Kirkland
2009-10-28 19:29 ` Dustin Kirkland
2009-10-28 19:29   ` [Qemu-devel] " Dustin Kirkland
2009-10-29  3:12 ` [Qemu-devel] " Scott Tsai
2009-10-29  3:12   ` Scott Tsai
2009-10-29  9:16 ` Mark McLoughlin [this message]
2009-10-29  9:16   ` Mark McLoughlin
2009-10-29 12:00   ` Scott Tsai
2009-10-29 12:00     ` Scott Tsai
2009-10-29 12:16     ` Mark McLoughlin
2009-10-29 12:16       ` Mark McLoughlin
2009-10-29 12:21       ` Scott Tsai
2009-10-29 12:21         ` Scott Tsai
2009-10-29 14:11       ` Anthony Liguori
2009-10-29 14:11         ` Anthony Liguori
2009-10-29 14:25         ` Mark McLoughlin
2009-10-29 14:25           ` Mark McLoughlin
2009-10-29 14:34           ` Dustin Kirkland
2009-10-29 14:34             ` Dustin Kirkland
2009-10-29 14:46             ` Dustin Kirkland
2009-10-29 14:46               ` Dustin Kirkland
2009-10-29 14:50               ` Mark McLoughlin
2009-10-29 14:50                 ` Mark McLoughlin
2009-10-29 14:39           ` Anthony Liguori
2009-10-29 14:39             ` Anthony Liguori
2009-10-29 14:48             ` Mark McLoughlin
2009-10-29 14:48               ` Mark McLoughlin
2009-10-29 15:01               ` Dustin Kirkland
2009-10-29 15:01                 ` Dustin Kirkland
2009-10-29 15:01                 ` Mark McLoughlin
2009-10-29 15:01                   ` Mark McLoughlin
2009-10-29 15:13                   ` Dustin Kirkland
2009-10-29 15:13                     ` Dustin Kirkland
2009-10-29 15:15                     ` Mark McLoughlin
2009-10-29 15:15                       ` Mark McLoughlin
2009-10-29 15:34               ` [PATCH] whitelist host virtio networking features [was Re: qemu-kvm-0.11 regression, crashes on older ...] Dustin Kirkland
2009-10-29 15:34                 ` [Qemu-devel] " Dustin Kirkland
2009-10-30 21:15                 ` Dustin Kirkland
2009-10-30 21:15                   ` [Qemu-devel] " Dustin Kirkland
2009-11-02 14:38                   ` Mark McLoughlin
2009-11-02 14:38                     ` [Qemu-devel] " Mark McLoughlin
2009-11-02 15:42                     ` Anthony Liguori
2009-11-02 15:42                       ` [Qemu-devel] " Anthony Liguori
2009-11-02 15:52                       ` Jamie Lokier
2009-11-02 18:20                         ` Michael Tokarev
2009-11-02 18:20                           ` Michael Tokarev
2009-11-02 19:39                           ` Jamie Lokier
2009-11-02 19:39                             ` Jamie Lokier
2009-11-02 18:55                         ` Anthony Liguori
2009-11-02 19:25                           ` Dustin Kirkland
2009-11-02 19:25                             ` Dustin Kirkland
2009-11-02 20:50                             ` Anthony Liguori
2009-11-02 20:50                               ` Anthony Liguori
2009-11-05  5:06                               ` Jamie Lokier
2009-11-05  5:06                                 ` Jamie Lokier
2009-11-02 16:58                     ` Dustin Kirkland
2009-11-02 16:58                       ` [Qemu-devel] " Dustin Kirkland
2009-10-29 14:39     ` [Qemu-devel] qemu-kvm-0.11 regression, crashes on older guests with virtio network Dustin Kirkland
2009-10-29 14:39       ` Dustin Kirkland
2009-10-29 23:22       ` Scott Tsai
2009-10-29 23:22         ` Scott Tsai
2009-10-29 12:23   ` Michael S. Tsirkin
2009-10-29 12:23     ` [Qemu-devel] " Michael S. Tsirkin
2009-10-29 14:38     ` Avi Kivity
2009-10-29 14:38       ` [Qemu-devel] " Avi Kivity
2009-10-29 15:03       ` Michael S. Tsirkin
2009-10-29 15:03         ` [Qemu-devel] " Michael S. Tsirkin
2009-10-29 14:43   ` [Qemu-devel] " Dustin Kirkland
2009-10-29 14:43     ` Dustin Kirkland

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1256807803.10825.39.camel@blaa \
    --to=markmc@redhat.com \
    --cc=anthony@codemonkey.ws \
    --cc=kirkland@canonical.com \
    --cc=kvm@vger.kernel.org \
    --cc=qemu-devel@nongnu.org \
    --cc=scottt.tw@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.