All of lore.kernel.org
 help / color / mirror / Atom feed
* Wrong TCP checksum of packets sent by Linux guest (virtIO/vhost)
@ 2014-09-02 13:20 Franck BAUDIN
       [not found] ` <D84D5A6C1B26E448A0F35B539111D0E721D8C0-tjtCPadWEHp5RdiqnJbviwvhXHaN8uqmQQ4Iyu8u01E@public.gmane.org>
  0 siblings, 1 reply; 9+ messages in thread
From: Franck BAUDIN @ 2014-09-02 13:20 UTC (permalink / raw)
  To: dev-VfR2kkLFssw

Hello,

This is a repost of https://lists.01.org/pipermail/dpdk-ovs/2014-September/001788.html as the issue might be related to dpdk.org vhost code.

I am using dpdk-ovs 1.1.0 (latest release) as follow :

linux-guest (no DPKD) <---  virtIO ---> OVDK 1.1.0 (with latest DPDK [*]) < --- Niantic --- > linux-native

[*] commit 8fd8bebc051704d7caecfed8d8a065a79c022329
Author: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Date:   Mon Sep 1 12:31:11 2014 +0200

UDP/ICMP connectivity is fine, but TCP checksum of packet sent by the guest are incorrect, as showed with tcpdump on linux-native.

Is it supposed to work? If yes, what is the proper qemu version to use and its parameters?

I am using qemu version provided by dpdk-ovs (same issue with PDDK 1.7.0 and latest one) and I followed this documentation: https://github.com/01org/dpdk-ovs/blob/development/docs/04_Sample_Configurations/02_Userspace-vHost.md

Thanks!
Franck



This message and any attachments (the "message") are confidential, intended solely for the addressees. If you are not the intended recipient, please notify the sender immediately by e-mail and delete this message from your system. In this case, you are not authorized to use, copy this message and/or disclose the content to any other person. E-mails are susceptible to alteration. Neither Qosmos nor any of its subsidiaries or affiliates shall be liable for the message if altered, changed or falsified.

Ce message et toutes ses pièces jointes (ci-après le "message")sont confidentiels et établis à l'intention exclusive de ses destinataires. Si vous avez reçu ce message par erreur, merci d’en informer immédiatement son émetteur par courrier électronique et d’effacer ce message de votre système. Dans cette hypothèse, vous n’êtes pas autorisé à utiliser, copier ce message et/ou en divulguer le contenu à un tiers. Tout message électronique est susceptible d'altération. Qosmos et ses filiales déclinent toute responsabilité au titre de ce message s'il a été altéré, déformé ou falsifié.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Wrong TCP checksum of packets sent by Linux guest (virtIO/vhost)
       [not found] ` <D84D5A6C1B26E448A0F35B539111D0E721D8C0-tjtCPadWEHp5RdiqnJbviwvhXHaN8uqmQQ4Iyu8u01E@public.gmane.org>
@ 2014-09-02 13:29   ` Thomas Monjalon
  2014-09-02 14:37     ` Franck Baudin
  2014-09-03 10:00     ` Gray, Mark D
  0 siblings, 2 replies; 9+ messages in thread
From: Thomas Monjalon @ 2014-09-02 13:29 UTC (permalink / raw)
  To: Franck BAUDIN; +Cc: dev-VfR2kkLFssw

Hi Franck,

2014-09-02 13:20, Franck BAUDIN:
> I am using dpdk-ovs 1.1.0 (latest release) as follow :
> 
> linux-guest (no DPKD) <---  virtIO ---> OVDK 1.1.0 (with latest DPDK [*]) < --- Niantic --- > linux-native
> 
[...]
> UDP/ICMP connectivity is fine, but TCP checksum of packet sent by the guest are incorrect, as showed with tcpdump on linux-native.

Thanks for reporting.
Could you try virtio without vhost?


> This message and any attachments (the "message") are confidential, intended solely for the addressees. If you are not the intended recipient, please notify the sender immediately by e-mail and delete this message from your system. In this case, you are not authorized to use, copy this message and/or disclose the content to any other person. E-mails are susceptible to alteration. Neither Qosmos nor any of its subsidiaries or affiliates shall be liable for the message if altered, changed or falsified.
> 
> Ce message et toutes ses pièces jointes (ci-après le "message")sont confidentiels et établis à l'intention exclusive de ses destinataires. Si vous avez reçu ce message par erreur, merci d’en informer immédiatement son émetteur par courrier électronique et d’effacer ce message de votre système. Dans cette hypothèse, vous n’êtes pas autorisé à utiliser, copier ce message et/ou en divulguer le contenu à un tiers. Tout message électronique est susceptible d'altération. Qosmos et ses filiales déclinent toute responsabilité au titre de ce message s'il a été altéré, déformé ou falsifié.

This big disclaimer block has no sense on a public mailing list.
Please try to remove it.

Thanks
-- 
Thomas

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Wrong TCP checksum of packets sent by Linux guest (virtIO/vhost)
  2014-09-02 13:29   ` Thomas Monjalon
@ 2014-09-02 14:37     ` Franck Baudin
  2014-09-03 10:00     ` Gray, Mark D
  1 sibling, 0 replies; 9+ messages in thread
From: Franck Baudin @ 2014-09-02 14:37 UTC (permalink / raw)
  To: Thomas Monjalon; +Cc: dev-VfR2kkLFssw

Hi Thomas,

On 09/02/14 15:29, Thomas Monjalon wrote:
> UDP/ICMP connectivity is fine, but TCP checksum of packet sent by the guest are incorrect, as showed with tcpdump on linux-native.
> Thanks for reporting.
> Could you try virtio without vhost?
Nope, dpdk-ovs supports only vhost.

Thanks!
Franck

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Wrong TCP checksum of packets sent by Linux guest (virtIO/vhost)
  2014-09-02 13:29   ` Thomas Monjalon
  2014-09-02 14:37     ` Franck Baudin
@ 2014-09-03 10:00     ` Gray, Mark D
       [not found]       ` <738D45BC1F695740A983F43CFE1B7EA92D72308F-kPTMFJFq+rFP9JyJpTNKArfspsVTdybXVpNB7YpNyf8@public.gmane.org>
  1 sibling, 1 reply; 9+ messages in thread
From: Gray, Mark D @ 2014-09-03 10:00 UTC (permalink / raw)
  To: Thomas Monjalon, Franck BAUDIN, Xie, Huawei
  Cc: dev-VfR2kkLFssw, dpdk-ovs-hn68Rpc1hR1g9hUCZPvPmw

> 
> Hi Franck,
> 
> 2014-09-02 13:20, Franck BAUDIN:
> > I am using dpdk-ovs 1.1.0 (latest release) as follow :
> >
> > linux-guest (no DPKD) <---  virtIO ---> OVDK 1.1.0 (with latest DPDK [*]) < --
> - Niantic --- > linux-native
> >
> [...]
> > UDP/ICMP connectivity is fine, but TCP checksum of packet sent by the
> guest are incorrect, as showed with tcpdump on linux-native.
> 
> Thanks for reporting.
> Could you try virtio without vhost?
> 
> 

Looping in the dpdk-ovs list. 

* Does the new vhost API allow a user to know if all the relevant offloads have been
turned on/off for that interface? It seems that this is possible through the virtio_net
structure but it would be good to get some feedback from the relevant person
working on DPDK (Huawei?).

* If this is the case, then it is probably in the realm of the vswitch do the actual 
checksum (for VM-VM) or correctly configure the NIC when sending out through
the physical interface.

Comments?

Mark

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Wrong TCP checksum of packets sent by Linux guest (virtIO/vhost)
       [not found]       ` <738D45BC1F695740A983F43CFE1B7EA92D72308F-kPTMFJFq+rFP9JyJpTNKArfspsVTdybXVpNB7YpNyf8@public.gmane.org>
@ 2014-09-03 11:13         ` Xie, Huawei
       [not found]           ` <C37D651A908B024F974696C65296B57B0F284EE2-0J0gbvR4kThpB2pF5aRoyrfspsVTdybXVpNB7YpNyf8@public.gmane.org>
  0 siblings, 1 reply; 9+ messages in thread
From: Xie, Huawei @ 2014-09-03 11:13 UTC (permalink / raw)
  To: Gray, Mark D, Thomas Monjalon, Franck BAUDIN
  Cc: dev-VfR2kkLFssw, dpdk-ovs-hn68Rpc1hR1g9hUCZPvPmw



> -----Original Message-----
> From: Gray, Mark D
> Sent: Wednesday, September 03, 2014 6:01 PM
> To: Thomas Monjalon; Franck BAUDIN; Xie, Huawei
> Cc: dev@dpdk.org; dpdk-ovs@lists.01.org
> Subject: RE: [dpdk-dev] Wrong TCP checksum of packets sent by Linux guest
> (virtIO/vhost)
> 
> >
> > Hi Franck,
> >
> > 2014-09-02 13:20, Franck BAUDIN:
> > > I am using dpdk-ovs 1.1.0 (latest release) as follow :
> > >
> > > linux-guest (no DPKD) <---  virtIO ---> OVDK 1.1.0 (with latest DPDK [*]) < --
> > - Niantic --- > linux-native
> > >
> > [...]
> > > UDP/ICMP connectivity is fine, but TCP checksum of packet sent by the
> > guest are incorrect, as showed with tcpdump on linux-native.
> >
> > Thanks for reporting.
> > Could you try virtio without vhost?
> >
> >
> 
> Looping in the dpdk-ovs list.
> 
> * Does the new vhost API allow a user to know if all the relevant offloads have
> been
> turned on/off for that interface? It seems that this is possible through the
> virtio_net
> structure but it would be good to get some feedback from the relevant person
> working on DPDK (Huawei?).
> 
> * If this is the case, then it is probably in the realm of the vswitch do the actual
> checksum (for VM-VM) or correctly configure the NIC when sending out through
> the physical interface.
> 
> Comments?


Mark:
So far not supported. This is important as well in VxLan case. For the packet flow  
Guest A->  virtio -> ..->OVDK->.. -> Guest B.
1) If guest A and B are on different host machines, say A and B respectively,  and if the nic on A supports
vxlan checksum offload, then both guest and host needn't generate checksum, the nic will  
generate checksum for both inner and outer packet.
2) In VM2VM case, as it is trusted communication channel, could we negotiate with the guest tcp stack not to verify checksum 
for received packet? 

> 
> Mark

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Wrong TCP checksum of packets sent by Linux guest (virtIO/vhost)
       [not found]           ` <C37D651A908B024F974696C65296B57B0F284EE2-0J0gbvR4kThpB2pF5aRoyrfspsVTdybXVpNB7YpNyf8@public.gmane.org>
@ 2014-09-03 14:12             ` Franck Baudin
       [not found]               ` <540721E8.2010307-kQLz4VG1Jc7QT0dZR+AlfA@public.gmane.org>
  2014-09-03 16:15             ` Gray, Mark D
  1 sibling, 1 reply; 9+ messages in thread
From: Franck Baudin @ 2014-09-03 14:12 UTC (permalink / raw)
  To: Xie, Huawei, Gray, Mark D, Thomas Monjalon
  Cc: dev-VfR2kkLFssw, dpdk-ovs-hn68Rpc1hR1g9hUCZPvPmw

Hi,

On 09/03/14 13:13, Xie, Huawei wrote:
> Looping in the dpdk-ovs list.
>
> * Does the new vhost API allow a user to know if all the relevant offloads have
> been
> turned on/off for that interface? It seems that this is possible through the
> virtio_net
> structure but it would be good to get some feedback from the relevant person
> working on DPDK (Huawei?).
>
> * If this is the case, then it is probably in the realm of the vswitch do the actual
> checksum (for VM-VM) or correctly configure the NIC when sending out through
> the physical interface.
>
> Comments?
>
> Mark:
> So far not supported. This is important as well in VxLan case. For the packet flow
> Guest A->  virtio -> ..->OVDK->.. -> Guest B.
> 1) If guest A and B are on different host machines, say A and B respectively,  and if the nic on A supports
> vxlan checksum offload, then both guest and host needn't generate checksum, the nic will
> generate checksum for both inner and outer packet.
> 2) In VM2VM case, as it is trusted communication channel, could we negotiate with the guest tcp stack not to verify checksum
> for received packet?
The problem is that any TCP packet send by a vanilla Linux guest through 
vhost is incorrect (VM to anything, including other colocalied VMs). In 
other words, the VM cannot use TCP. QEMU options and ethtool -K csum off 
tso off ("TCP stack negociation") have no effect, maybe because the 
vhost backend is misbehaving.

Franck

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Wrong TCP checksum of packets sent by Linux guest (virtIO/vhost)
       [not found]               ` <540721E8.2010307-kQLz4VG1Jc7QT0dZR+AlfA@public.gmane.org>
@ 2014-09-03 16:07                 ` Gray, Mark D
  2014-09-04  2:54                 ` Xie, Huawei
  1 sibling, 0 replies; 9+ messages in thread
From: Gray, Mark D @ 2014-09-03 16:07 UTC (permalink / raw)
  To: Franck Baudin, Xie, Huawei, Thomas Monjalon
  Cc: dev-VfR2kkLFssw, dpdk-ovs-hn68Rpc1hR1g9hUCZPvPmw

> 
> Hi,
> 
> On 09/03/14 13:13, Xie, Huawei wrote:
> > Looping in the dpdk-ovs list.
> >
> > * Does the new vhost API allow a user to know if all the relevant
> > offloads have been turned on/off for that interface? It seems that
> > this is possible through the virtio_net structure but it would be good
> > to get some feedback from the relevant person working on DPDK
> > (Huawei?).
> >
> > * If this is the case, then it is probably in the realm of the vswitch
> > do the actual checksum (for VM-VM) or correctly configure the NIC when
> > sending out through the physical interface.
> >
> > Comments?
> >
> > Mark:
> > So far not supported. This is important as well in VxLan case. For the
> > packet flow Guest A->  virtio -> ..->OVDK->.. -> Guest B.
> > 1) If guest A and B are on different host machines, say A and B
> > respectively,  and if the nic on A supports vxlan checksum offload,
> > then both guest and host needn't generate checksum, the nic will generate
> checksum for both inner and outer packet.
> > 2) In VM2VM case, as it is trusted communication channel, could we
> > negotiate with the guest tcp stack not to verify checksum for received
> packet?
> The problem is that any TCP packet send by a vanilla Linux guest through
> vhost is incorrect (VM to anything, including other colocalied VMs). In other
> words, the VM cannot use TCP. QEMU options and ethtool -K csum off tso
> off ("TCP stack negociation") have no effect, maybe because the vhost
> backend is misbehaving.

This sounds like a DPDK issue. However, we did "clone and own" the vhost
code as it is a sample app. It may have changed since we integrated it.

We plan to use the vhost library interface when it is available on dpdk.org. So
that would pick up the latest version of the vhost code.

> 
> Franck

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Wrong TCP checksum of packets sent by Linux guest (virtIO/vhost)
       [not found]           ` <C37D651A908B024F974696C65296B57B0F284EE2-0J0gbvR4kThpB2pF5aRoyrfspsVTdybXVpNB7YpNyf8@public.gmane.org>
  2014-09-03 14:12             ` Franck Baudin
@ 2014-09-03 16:15             ` Gray, Mark D
  1 sibling, 0 replies; 9+ messages in thread
From: Gray, Mark D @ 2014-09-03 16:15 UTC (permalink / raw)
  To: Xie, Huawei, Thomas Monjalon, Franck BAUDIN
  Cc: dev-VfR2kkLFssw, dpdk-ovs-hn68Rpc1hR1g9hUCZPvPmw

> > >
> > > Hi Franck,
> > >
> > > 2014-09-02 13:20, Franck BAUDIN:
> > > > I am using dpdk-ovs 1.1.0 (latest release) as follow :
> > > >
> > > > linux-guest (no DPKD) <---  virtIO ---> OVDK 1.1.0 (with latest
> > > > DPDK [*]) < --
> > > - Niantic --- > linux-native
> > > >
> > > [...]
> > > > UDP/ICMP connectivity is fine, but TCP checksum of packet sent by
> > > > the
> > > guest are incorrect, as showed with tcpdump on linux-native.
> > >
> > > Thanks for reporting.
> > > Could you try virtio without vhost?
> > >
> > >
> >
> > Looping in the dpdk-ovs list.
> >
> > * Does the new vhost API allow a user to know if all the relevant
> > offloads have been turned on/off for that interface? It seems that
> > this is possible through the virtio_net structure but it would be good
> > to get some feedback from the relevant person working on DPDK
> > (Huawei?).
> >
> > * If this is the case, then it is probably in the realm of the vswitch
> > do the actual checksum (for VM-VM) or correctly configure the NIC when
> > sending out through the physical interface.
> >
> > Comments?
> 
> 
> Mark:
> So far not supported. This is important as well in VxLan case. For the packet

Ok 

> flow Guest A->  virtio -> ..->OVDK->.. -> Guest B.
> 1) If guest A and B are on different host machines, say A and B respectively,
> and if the nic on A supports vxlan checksum offload, then both guest and
> host needn't generate checksum, the nic will generate checksum for both
> inner and outer packet.

The vswitch would need to support this

> 2) In VM2VM case, as it is trusted communication channel, could we
> negotiate with the guest tcp stack not to verify checksum for received
> packet?

Perhaps

> 
> >
> > Mark

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Wrong TCP checksum of packets sent by Linux guest (virtIO/vhost)
       [not found]               ` <540721E8.2010307-kQLz4VG1Jc7QT0dZR+AlfA@public.gmane.org>
  2014-09-03 16:07                 ` Gray, Mark D
@ 2014-09-04  2:54                 ` Xie, Huawei
  1 sibling, 0 replies; 9+ messages in thread
From: Xie, Huawei @ 2014-09-04  2:54 UTC (permalink / raw)
  To: Franck Baudin, Gray, Mark D, Thomas Monjalon
  Cc: dev-VfR2kkLFssw, dpdk-ovs-hn68Rpc1hR1g9hUCZPvPmw



> -----Original Message-----
> From: Franck Baudin [mailto:franck.baudin@qosmos.com]
> Sent: Wednesday, September 03, 2014 10:13 PM
> To: Xie, Huawei; Gray, Mark D; Thomas Monjalon
> Cc: dev@dpdk.org; dpdk-ovs@lists.01.org
> Subject: Re: [dpdk-dev] Wrong TCP checksum of packets sent by Linux guest
> (virtIO/vhost)
> 
> Hi,
> 
> On 09/03/14 13:13, Xie, Huawei wrote:
> > Looping in the dpdk-ovs list.
> >
> > * Does the new vhost API allow a user to know if all the relevant offloads have
> > been
> > turned on/off for that interface? It seems that this is possible through the
> > virtio_net
> > structure but it would be good to get some feedback from the relevant person
> > working on DPDK (Huawei?).
> >
> > * If this is the case, then it is probably in the realm of the vswitch do the actual
> > checksum (for VM-VM) or correctly configure the NIC when sending out
> through
> > the physical interface.
> >
> > Comments?
> >
> > Mark:
> > So far not supported. This is important as well in VxLan case. For the packet
> flow
> > Guest A->  virtio -> ..->OVDK->.. -> Guest B.
> > 1) If guest A and B are on different host machines, say A and B respectively,
> and if the nic on A supports
> > vxlan checksum offload, then both guest and host needn't generate checksum,
> the nic will
> > generate checksum for both inner and outer packet.
> > 2) In VM2VM case, as it is trusted communication channel, could we negotiate
> with the guest tcp stack not to verify checksum
> > for received packet?
> The problem is that any TCP packet send by a vanilla Linux guest through
> vhost is incorrect (VM to anything, including other colocalied VMs). In
> other words, the VM cannot use TCP. QEMU options and ethtool -K csum off
> tso off ("TCP stack negociation") have no effect, maybe because the
> vhost backend is misbehaving.
> 
> Franck

Hi Franck:
I checked your original thread.
root at linux-native:~# tcpdump -vnei eth0
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
17:27:09.262926 00:1b:21:b9:9b:2c > 52:54:00:51:51:12, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 47743, offset 0, flags [DF], proto TCP (6), length 60)
   1.1.1.3.34272 > 1.1.1.2.5555: Flags [S], cksum 0x0435 (incorrect -> 0xb2dd), seq 1963818356, win 14600, options [mss 1460,sackOK,TS val 49530635 ecr 0,nop,wscale 7], length 0
17:27:09.263220 52:54:00:51:51:12 > 00:1b:21:b9:9b:2c, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 64, id 3367, offset 0, flags [DF], proto TCP (6), length 40)
    1.1.1.2.5555 > 1.1.1.3.34272: Flags [R.], cksum 0x1db4 (correct), seq 0, ack 1963818357, win 0, length 0

The packet from the guest, received on the native machine, has "correct" checksum 0x1db4. The TCP connection is refused is due to this packet has R flag. Could you check if it is the firewall that blocks the 
connection?

Besides, I did ssh tcp test between two guest VMs,  the ssh connection could be established correctly with tx checksum off. I didn't use OVDK, but set the arp table and route table manually.
The packet flow is
	Guest A(virtio)<->vhost example->Guest B(virtio)









^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2014-09-04  2:54 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-09-02 13:20 Wrong TCP checksum of packets sent by Linux guest (virtIO/vhost) Franck BAUDIN
     [not found] ` <D84D5A6C1B26E448A0F35B539111D0E721D8C0-tjtCPadWEHp5RdiqnJbviwvhXHaN8uqmQQ4Iyu8u01E@public.gmane.org>
2014-09-02 13:29   ` Thomas Monjalon
2014-09-02 14:37     ` Franck Baudin
2014-09-03 10:00     ` Gray, Mark D
     [not found]       ` <738D45BC1F695740A983F43CFE1B7EA92D72308F-kPTMFJFq+rFP9JyJpTNKArfspsVTdybXVpNB7YpNyf8@public.gmane.org>
2014-09-03 11:13         ` Xie, Huawei
     [not found]           ` <C37D651A908B024F974696C65296B57B0F284EE2-0J0gbvR4kThpB2pF5aRoyrfspsVTdybXVpNB7YpNyf8@public.gmane.org>
2014-09-03 14:12             ` Franck Baudin
     [not found]               ` <540721E8.2010307-kQLz4VG1Jc7QT0dZR+AlfA@public.gmane.org>
2014-09-03 16:07                 ` Gray, Mark D
2014-09-04  2:54                 ` Xie, Huawei
2014-09-03 16:15             ` Gray, Mark D

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.