From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59250C433DF for ; Wed, 17 Jun 2020 10:36:10 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 23A0C2098B for ; Wed, 17 Jun 2020 10:36:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="qlyMbUIM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 23A0C2098B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:49476 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jlVQP-0006nK-FP for qemu-devel@archiver.kernel.org; Wed, 17 Jun 2020 06:36:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56016) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jlVMb-00033C-SF for qemu-devel@nongnu.org; Wed, 17 Jun 2020 06:32:13 -0400 Received: from mail-wr1-x42a.google.com ([2a00:1450:4864:20::42a]:33250) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jlVMZ-00007K-Qm for qemu-devel@nongnu.org; Wed, 17 Jun 2020 06:32:13 -0400 Received: by mail-wr1-x42a.google.com with SMTP id l11so1791403wru.0 for ; Wed, 17 Jun 2020 03:32:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=a7dqyj5cuUATWqNomX1lzd1a0dS44usP0OrTroPvyR8=; b=qlyMbUIM8UHbamdlsDM1TkzEyVL8t5aAB979Cnd4MbV7ZkAOZzdsvWBXN23A4eFPxW 3wv62jhvE7J5bRcfR3OiKZ0aND369w/kLZs3wk3RMvbJlwUfT7Zb30ezwHADsVJ83laR ri8EuaZTjuqAUeyo6PLRt5B5izvpQnFzibxYL37F2koVuqeLKvmucS8D5Oxov3T0OR0K q4kQeWdgtzNh6B1h582p1uebyuxx2aNA94RfdQ9Kt2+XSz6U0DZ3jFu3vNlQ9Vrp35oJ Utt/Crgzuk5kwNaPaiyjDct8rM/7UyHDYpTBAgBheyGQtwRCrKdfNi0RVasmUitkR3xS UyZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=a7dqyj5cuUATWqNomX1lzd1a0dS44usP0OrTroPvyR8=; b=k97OelJ38VbhGa26IweIDJVdpjTcG5iBYRAcPumS0GPrIJIFhm3pvV7UZenaebsM4p HyBC/U8ChYr0oQGmfOKD8wTmyX/9m93cunyAs5Kx/4L2fbLxLCNeZg3ZulTloNqNxTby 8Oya9/VqkYZ50XHeKICSPS1i156gtThWmFgNbn4qerBJ8MAqjwQy+ozWGke2glP4bYR4 KyXozyccl5umgIwP2svswL8GXAsIOdU2T0rDFJFidDpftXv99v0OcFATvgIy3rtdIXUS 1CgvAGVP4MpTKa7ShAkux8qeh0U8MH7OEUQ7PKB4zU2LGEMA8+7C4Gjx6nR0B5JUFjDR ke9Q== X-Gm-Message-State: AOAM533apLzMYgdXbLErDW96l1VI0fWbBiUj67cFlDwBSISwSzTfy3FH k4EYbBUcnioTGFuzg7qjgubJ7EZHoNU= X-Google-Smtp-Source: ABdhPJwVJdSg7Z/mxghK4mYUIRgfQPbjXoJix6iRVvrNLYcfMhZGjUNx83MVvZDhjCKgbp39Li7/gQ== X-Received: by 2002:a05:6000:1202:: with SMTP id e2mr7373981wrx.231.1592389929545; Wed, 17 Jun 2020 03:32:09 -0700 (PDT) Received: from localhost ([51.15.41.238]) by smtp.gmail.com with ESMTPSA id z6sm32380903wrh.79.2020.06.17.03.32.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jun 2020 03:32:08 -0700 (PDT) Date: Wed, 17 Jun 2020 11:32:06 +0100 From: Stefan Hajnoczi To: Bug 1882241 <1882241@bugs.launchpad.net> Subject: Re: [Bug 1882241] [NEW] file transfer over cifs to 64bit guest corrupts large files Message-ID: <20200617103206.GA1728005@stefanha-x1.localdomain> References: <159136023930.32294.17616621945608188739.malonedeb@gac.canonical.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="zYM0uCDKw75PZbzx" Content-Disposition: inline In-Reply-To: <159136023930.32294.17616621945608188739.malonedeb@gac.canonical.com> Received-SPF: pass client-ip=2a00:1450:4864:20::42a; envelope-from=stefanha@gmail.com; helo=mail-wr1-x42a.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" --zYM0uCDKw75PZbzx Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jun 05, 2020 at 12:30:39PM -0000, timsoft wrote: > Public bug reported: >=20 > qemu 4.0 compiled fom source. > vm called by > qemu-system-x86_64 -cpu qemu64 -smp 4 -m 4G -drive file=3D/data/images/sl= ack14.2_64bit_test.qcow2,format=3Dqcow2 -cdrom /mnt/smb1/slackware/iso/slac= kware64-14.2-install-dvd.iso -boot c -net nic,macaddr=3D02:00:00:11:11:17,m= odel=3Di82551 -net bridge,br=3Dbr0 -enable-kvm -k en-gb -display vnc=3D:3 -= monitor telnet:localhost:7103,server,nowait,nodelay >=20 > copying large files eg 2.4gb or reading them on a cifs mount in the guest= causes corruption every time. For smaller files 40-60mb corruption is more= than 50% of the time. tested by md5sum on cifs server, or on host machine = vs. on guest vm. > corruption is seen only with 64bit guest using cifs with i82551 emulated = network device > ie. 32bit guest using cifs with i82551 emulated network device gives no c= orruption. >=20 > changing the emulated device to vmxnet3 removes the data corruption (see > below) >=20 > qemu-system-x86_64 -cpu qemu64 -smp 4 -m 4G -drive > file=3D/data/images/slack14.2_64bit_test.qcow2,format=3Dqcow2 -cdrom > /mnt/smb1/slackware/iso/slackware64-14.2-install-dvd.iso -boot c -net > nic,macaddr=3D02:00:00:11:11:17,model=3Dvmxnet3 -net bridge,br=3Dbr0 -ena= ble- > kvm -k en-gb -display vnc=3D:3 -monitor > telnet:localhost:7103,server,nowait,nodelay >=20 > this corruption is repeatable. ie. I created new vm, call using top examp= le, installed 64bit linux, mounted cifs share and copied 2.4gb file to /tmp= then run md5sum "filecopied" > the md5sum is different every time. copy same file to the host, or to a 3= 2bit guest with the same virtual network device and bridge and md5sums are = correct. The host pysical network adapter is > lspci|grep Ether > 1e:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168= /8411 PCI Express Gigabit Ethernet Controller (rev 11) >=20 > physically connected via gigabit ethernet to cifs server (via gigabit > switch) Not a solution but some comments: 1. As a sanity-check you could try "nc 1234 /tmp/file" in the guest. Netcat simply sends/receives data over a TCP connection (it's a much simpler test than CIFS). Is the checksum okay? 2. I don't know the CIFS network protocol, but if Wireshark can dissect it then you could compare the flows between the vmxnet3 and the i82551. This is only feasible if Wireshark can produce an unencrypted conversation and the CIFS protocol doesn't have many protocol header fields that differ between two otherwise identical sessions. 3. virtio-net is the most widely used and high-performance NIC model. Other emulated NIC models are mainly there for very old guests that lack virtio guest drivers. --zYM0uCDKw75PZbzx Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEhpWov9P5fNqsNXdanKSrs4Grc8gFAl7p8SYACgkQnKSrs4Gr c8jwHQf/bI/7/ZD0LN4eIFOG+yqe8Dh5gaJZGMl4q8djO2yT2ErT2WiBSKfbjZrn ERokOtxQ2Jsk9BngxoH48zj3q011z/xdKuHDN1HZb16aZMili/HwqXPn623/CVcG dfrDaX5NR2RMQfoLPj8wSy8FN6ulcUVOnsZCz0iIMom8ARn0rOtU6yF8VVjTYGvg ewPr4yN2zPawPJiSSsyewmYV2i8tGO4eDqj+IBz0A3lrgldln3tkKwunE/2JiOPF HNaLqVCNu8SBJopVjGIIUMpgadBVbbAHscrTNoB3+a24UEWcsTBt+3yFB3e3u3Vb cYZXwTu/ur5fb7rnWVzqG6wM7OwEvg== =s3Gs -----END PGP SIGNATURE----- --zYM0uCDKw75PZbzx-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.7 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C329C433E0 for ; Wed, 17 Jun 2020 10:41:41 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 70EC22082F for ; Wed, 17 Jun 2020 10:41:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 70EC22082F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=bugs.launchpad.net Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:54772 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jlVVk-0001OO-OS for qemu-devel@archiver.kernel.org; Wed, 17 Jun 2020 06:41:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57944) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jlVVA-0000or-3S for qemu-devel@nongnu.org; Wed, 17 Jun 2020 06:41:04 -0400 Received: from indium.canonical.com ([91.189.90.7]:60128) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jlVV6-0001mL-6g for qemu-devel@nongnu.org; Wed, 17 Jun 2020 06:41:02 -0400 Received: from loganberry.canonical.com ([91.189.90.37]) by indium.canonical.com with esmtp (Exim 4.86_2 #2 (Debian)) id 1jlVV4-00028P-Dn for ; Wed, 17 Jun 2020 10:40:58 +0000 Received: from loganberry.canonical.com (localhost [127.0.0.1]) by loganberry.canonical.com (Postfix) with ESMTP id 5FA042E802E for ; Wed, 17 Jun 2020 10:40:58 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Date: Wed, 17 Jun 2020 10:32:06 -0000 From: Stefan Hajnoczi <1882241@bugs.launchpad.net> To: qemu-devel@nongnu.org X-Launchpad-Notification-Type: bug X-Launchpad-Bug: product=qemu; status=New; importance=Undecided; assignee=None; X-Launchpad-Bug-Tags: i82551 X-Launchpad-Bug-Information-Type: Public X-Launchpad-Bug-Private: no X-Launchpad-Bug-Security-Vulnerability: no X-Launchpad-Bug-Commenters: stefanha tim-tree-of-life X-Launchpad-Bug-Reporter: timsoft (tim-tree-of-life) X-Launchpad-Bug-Modifier: Stefan Hajnoczi (stefanha) References: <159136023930.32294.17616621945608188739.malonedeb@gac.canonical.com> Message-ID: <20200617103206.GA1728005@stefanha-x1.localdomain> Subject: Re: [Bug 1882241] [NEW] file transfer over cifs to 64bit guest corrupts large files X-Launchpad-Message-Rationale: Subscriber (QEMU) @qemu-devel-ml X-Launchpad-Message-For: qemu-devel-ml Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="1cbd0aa39df153c901321817f9b57cf3f232b507"; Instance="production-secrets-lazr.conf" X-Launchpad-Hash: a1bc5039a7412011692b210df8b4a0416c216510 Received-SPF: none client-ip=91.189.90.7; envelope-from=bounces@canonical.com; helo=indium.canonical.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/17 06:40:58 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -58 X-Spam_score: -5.9 X-Spam_bar: ----- X-Spam_report: (-5.9 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Bug 1882241 <1882241@bugs.launchpad.net> Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Message-ID: <20200617103206.gX4QJ0pr5J5E5IfaeYqjbavV-wVmL4hV2vuNbiqOx2s@z> On Fri, Jun 05, 2020 at 12:30:39PM -0000, timsoft wrote: > Public bug reported: > = > qemu 4.0 compiled fom source. > vm called by > qemu-system-x86_64 -cpu qemu64 -smp 4 -m 4G -drive file=3D/data/images/sl= ack14.2_64bit_test.qcow2,format=3Dqcow2 -cdrom /mnt/smb1/slackware/iso/slac= kware64-14.2-install-dvd.iso -boot c -net nic,macaddr=3D02:00:00:11:11:17,m= odel=3Di82551 -net bridge,br=3Dbr0 -enable-kvm -k en-gb -display vnc=3D:3 -= monitor telnet:localhost:7103,server,nowait,nodelay > = > copying large files eg 2.4gb or reading them on a cifs mount in the guest= causes corruption every time. For smaller files 40-60mb corruption is more= than 50% of the time. tested by md5sum on cifs server, or on host machine = vs. on guest vm. > corruption is seen only with 64bit guest using cifs with i82551 emulated = network device > ie. 32bit guest using cifs with i82551 emulated network device gives no c= orruption. > = > changing the emulated device to vmxnet3 removes the data corruption (see > below) > = > qemu-system-x86_64 -cpu qemu64 -smp 4 -m 4G -drive > file=3D/data/images/slack14.2_64bit_test.qcow2,format=3Dqcow2 -cdrom > /mnt/smb1/slackware/iso/slackware64-14.2-install-dvd.iso -boot c -net > nic,macaddr=3D02:00:00:11:11:17,model=3Dvmxnet3 -net bridge,br=3Dbr0 -ena= ble- > kvm -k en-gb -display vnc=3D:3 -monitor > telnet:localhost:7103,server,nowait,nodelay > = > this corruption is repeatable. ie. I created new vm, call using top examp= le, installed 64bit linux, mounted cifs share and copied 2.4gb file to /tmp= then run md5sum "filecopied" > the md5sum is different every time. copy same file to the host, or to a 3= 2bit guest with the same virtual network device and bridge and md5sums are = correct. The host pysical network adapter is > lspci|grep Ether > 1e:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168= /8411 PCI Express Gigabit Ethernet Controller (rev 11) > = > physically connected via gigabit ethernet to cifs server (via gigabit > switch) Not a solution but some comments: 1. As a sanity-check you could try "nc 1234 /tmp/file" in the guest. Netcat simply sends/receives data over a TCP connection (it's a much simpler test than CIFS). Is the checksum okay? 2. I don't know the CIFS network protocol, but if Wireshark can dissect it then you could compare the flows between the vmxnet3 and the i82551. This is only feasible if Wireshark can produce an unencrypted conversation and the CIFS protocol doesn't have many protocol header fields that differ between two otherwise identical sessions. 3. virtio-net is the most widely used and high-performance NIC model. Other emulated NIC models are mainly there for very old guests that lack virtio guest drivers. -- = You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1882241 Title: file transfer over cifs to 64bit guest corrupts large files Status in QEMU: New Bug description: qemu 4.0 compiled fom source. vm called by qemu-system-x86_64 -cpu qemu64 -smp 4 -m 4G -drive file=3D/data/images/sl= ack14.2_64bit_test.qcow2,format=3Dqcow2 -cdrom /mnt/smb1/slackware/iso/slac= kware64-14.2-install-dvd.iso -boot c -net nic,macaddr=3D02:00:00:11:11:17,m= odel=3Di82551 -net bridge,br=3Dbr0 -enable-kvm -k en-gb -display vnc=3D:3 -= monitor telnet:localhost:7103,server,nowait,nodelay copying large files eg 2.4gb or reading them on a cifs mount in the guest= causes corruption every time. For smaller files 40-60mb corruption is more= than 50% of the time. tested by md5sum on cifs server, or on host machine = vs. on guest vm. corruption is seen only with 64bit guest using cifs with i82551 emulated = network device ie. 32bit guest using cifs with i82551 emulated network device gives no c= orruption. changing the emulated device to vmxnet3 removes the data corruption (see below) qemu-system-x86_64 -cpu qemu64 -smp 4 -m 4G -drive file=3D/data/images/slack14.2_64bit_test.qcow2,format=3Dqcow2 -cdrom /mnt/smb1/slackware/iso/slackware64-14.2-install-dvd.iso -boot c -net nic,macaddr=3D02:00:00:11:11:17,model=3Dvmxnet3 -net bridge,br=3Dbr0 -enable-kvm -k en-gb -display vnc=3D:3 -monitor telnet:localhost:7103,server,nowait,nodelay this corruption is repeatable. ie. I created new vm, call using top examp= le, installed 64bit linux, mounted cifs share and copied 2.4gb file to /tmp= then run md5sum "filecopied" the md5sum is different every time. copy same file to the host, or to a 3= 2bit guest with the same virtual network device and bridge and md5sums are = correct. The host pysical network adapter is lspci|grep Ether 1e:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168= /8411 PCI Express Gigabit Ethernet Controller (rev 11) physically connected via gigabit ethernet to cifs server (via gigabit switch) To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1882241/+subscriptions