From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Chen, Junjie J" <junjie.j.chen@intel.com>
Subject: Re: [PATCH] vhost: do deep copy while reallocate vq
Date: Tue, 16 Jan 2018 07:38:41 +0000
Message-ID: <AA85A5A5E706C44BACB0BEFD5AC08BF63132C4EF@SHSMSX101.ccr.corp.intel.com>
References: <1516015939-11266-1-git-send-email-junjie.j.chen@intel.com>
 <E182254E98A5DA4EB1E657AC7CB9BD2A8B0250EC@BGSMSX101.gar.corp.intel.com>
 <AA85A5A5E706C44BACB0BEFD5AC08BF63132C026@SHSMSX101.ccr.corp.intel.com>
 <E182254E98A5DA4EB1E657AC7CB9BD2A8B025827@BGSMSX101.gar.corp.intel.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Cc: "dev@dpdk.org" <dev@dpdk.org>
To: "Yang, Zhiyong" <zhiyong.yang@intel.com>, "yliu@fridaylinux.org"
 <yliu@fridaylinux.org>, "maxime.coquelin@redhat.com"
 <maxime.coquelin@redhat.com>
Return-path: <dev-bounces@dpdk.org>
Received: from mga06.intel.com (mga06.intel.com [134.134.136.31])
 by dpdk.org (Postfix) with ESMTP id 00E7412001
 for <dev@dpdk.org>; Tue, 16 Jan 2018 08:38:45 +0100 (CET)
In-Reply-To: <E182254E98A5DA4EB1E657AC7CB9BD2A8B025827@BGSMSX101.gar.corp.intel.com>
Content-Language: en-US
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

Hi
> > > > @@ -227,6 +227,7 @@ vhost_user_set_vring_num(struct virtio_net
> *dev,
> > > >  				"zero copy is force disabled\n");
> > > >  			dev->dequeue_zero_copy =3D 0;
> > > >  		}
> > > > +		TAILQ_INIT(&vq->zmbuf_list);
> > > >  	}
> > > >
> > > >  	vq->shadow_used_ring =3D rte_malloc(NULL, @@ -261,6 +262,9
> @@
> > > > numa_realloc(struct virtio_net *dev, int index)
> > > >  	int oldnode, newnode;
> > > >  	struct virtio_net *old_dev;
> > > >  	struct vhost_virtqueue *old_vq, *vq;
> > > > +	struct zcopy_mbuf *new_zmbuf;
> > > > +	struct vring_used_elem *new_shadow_used_ring;
> > > > +	struct batch_copy_elem *new_batch_copy_elems;
> > > >  	int ret;
> > > >
> > > >  	old_dev =3D dev;
> > > > @@ -285,6 +289,33 @@ numa_realloc(struct virtio_net *dev, int
> index)
> > > >  			return dev;
> > > >
> > > >  		memcpy(vq, old_vq, sizeof(*vq));
> > > > +		TAILQ_INIT(&vq->zmbuf_list);
> > > > +
> > > > +		new_zmbuf =3D rte_malloc_socket(NULL, vq->zmbuf_size *
> > > > +			sizeof(struct zcopy_mbuf), 0, newnode);
> > > > +		if (new_zmbuf) {
> > > > +			rte_free(vq->zmbufs);
> > > > +			vq->zmbufs =3D new_zmbuf;
> > > > +		}
> > >
> > > You need to consider how to handle the case  ( rte_malloc_socket
> > > return NULL).
> >
> > If it failed to allocate new_zmbuf, it uses old zmbufs, so as to keep
> > vhost alive.
>=20
> It sounds reasonable, another question is, for the 3 blocks of memory bei=
ng
> allocated,  If some succeed , others fails,  Does it mean that the code w=
ill
> run on different socket?  What's the perf impact if it happens.

The original code doesn't do deep copy and thus access memory on different =
socket, this patch is to mitigate this situation. It does access remote mem=
ory when one of above allocation failed.=20

I saw some performance improvement (24.8Gbits/s -> 26.1Gbit/s) on my dev ma=
chine when only reallocate for zmbufs, while I didn't see significant perfo=
rmance difference when allocating vring_used_elem=20
and batch_copy_elem.