From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752007AbdFUM3F (ORCPT ); Wed, 21 Jun 2017 08:29:05 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36226 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751004AbdFUM3D (ORCPT ); Wed, 21 Jun 2017 08:29:03 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 628274025D Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=mst@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 628274025D Date: Wed, 21 Jun 2017 15:28:56 +0300 From: "Michael S. Tsirkin" To: Wei Wang Cc: virtio-dev@lists.oasis-open.org, linux-kernel@vger.kernel.org, qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-mm@kvack.org, david@redhat.com, dave.hansen@intel.com, cornelia.huck@de.ibm.com, akpm@linux-foundation.org, mgorman@techsingularity.net, aarcange@redhat.com, amit.shah@redhat.com, pbonzini@redhat.com, liliang.opensource@gmail.com, riel@redhat.com, nilal@redhat.com Subject: Re: [virtio-dev] Re: [PATCH v11 6/6] virtio-balloon: VIRTIO_BALLOON_F_CMD_VQ Message-ID: <20170621151922-mutt-send-email-mst@kernel.org> References: <1497004901-30593-1-git-send-email-wei.w.wang@intel.com> <1497004901-30593-7-git-send-email-wei.w.wang@intel.com> <20170620190343-mutt-send-email-mst@kernel.org> <5949E7C0.3050106@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5949E7C0.3050106@intel.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 21 Jun 2017 12:29:03 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jun 21, 2017 at 11:28:00AM +0800, Wei Wang wrote: > On 06/21/2017 12:18 AM, Michael S. Tsirkin wrote: > > On Fri, Jun 09, 2017 at 06:41:41PM +0800, Wei Wang wrote: > > > - if (!virtqueue_indirect_desc_table_add(vq, desc, num)) { > > > + if (!virtqueue_indirect_desc_table_add(vq, desc, *num)) { > > > virtqueue_kick(vq); > > > - wait_event(vb->acked, virtqueue_get_buf(vq, &len)); > > > - vb->balloon_page_chunk.chunk_num = 0; > > > + if (busy_wait) > > > + while (!virtqueue_get_buf(vq, &len) && > > > + !virtqueue_is_broken(vq)) > > > + cpu_relax(); > > > + else > > > + wait_event(vb->acked, virtqueue_get_buf(vq, &len)); > > > > This is something I didn't previously notice. > > As you always keep a single buffer in flight, you do not > > really need indirect at all. Just add all descriptors > > in the ring directly, then kick. > > > > E.g. > > virtqueue_add_first > > virtqueue_add_next > > virtqueue_add_last > > > > ? > > > > You also want a flag to avoid allocations but there's no need to do it > > per descriptor, set it on vq. > > > > Without using the indirect table, I'm thinking about changing to use > the standard sg (i.e. struct scatterlist), instead of vring_desc, so that > we don't need to modify or add any new functions of virtqueue_add(). > > In this case, we will kmalloc an array of sgs in probe(), and we can add > the sgs one by one to the vq, which won't trigger the allocation of an > indirect table inside virtqueue_add(), and then kick when all are added. > > Best, > Wei And allocate headers too? This can work. API extensions aren't necessarily a bad idea though. The API I suggest above is preferable for the simple reason that it can work without INDIRECT flag support in hypervisor. I wonder which APIs would Nitesh find useful. -- MST From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: [virtio-dev] Re: [PATCH v11 6/6] virtio-balloon: VIRTIO_BALLOON_F_CMD_VQ Date: Wed, 21 Jun 2017 15:28:56 +0300 Message-ID: <20170621151922-mutt-send-email-mst@kernel.org> References: <1497004901-30593-1-git-send-email-wei.w.wang@intel.com> <1497004901-30593-7-git-send-email-wei.w.wang@intel.com> <20170620190343-mutt-send-email-mst@kernel.org> <5949E7C0.3050106@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: aarcange@redhat.com, virtio-dev@lists.oasis-open.org, riel@redhat.com, kvm@vger.kernel.org, qemu-devel@nongnu.org, amit.shah@redhat.com, liliang.opensource@gmail.com, dave.hansen@intel.com, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-mm@kvack.org, cornelia.huck@de.ibm.com, pbonzini@redhat.com, akpm@linux-foundation.org, nilal@redhat.com, mgorman@techsingularity.net To: Wei Wang Return-path: Content-Disposition: inline In-Reply-To: <5949E7C0.3050106@intel.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org List-Id: kvm.vger.kernel.org On Wed, Jun 21, 2017 at 11:28:00AM +0800, Wei Wang wrote: > On 06/21/2017 12:18 AM, Michael S. Tsirkin wrote: > > On Fri, Jun 09, 2017 at 06:41:41PM +0800, Wei Wang wrote: > > > - if (!virtqueue_indirect_desc_table_add(vq, desc, num)) { > > > + if (!virtqueue_indirect_desc_table_add(vq, desc, *num)) { > > > virtqueue_kick(vq); > > > - wait_event(vb->acked, virtqueue_get_buf(vq, &len)); > > > - vb->balloon_page_chunk.chunk_num = 0; > > > + if (busy_wait) > > > + while (!virtqueue_get_buf(vq, &len) && > > > + !virtqueue_is_broken(vq)) > > > + cpu_relax(); > > > + else > > > + wait_event(vb->acked, virtqueue_get_buf(vq, &len)); > > > > This is something I didn't previously notice. > > As you always keep a single buffer in flight, you do not > > really need indirect at all. Just add all descriptors > > in the ring directly, then kick. > > > > E.g. > > virtqueue_add_first > > virtqueue_add_next > > virtqueue_add_last > > > > ? > > > > You also want a flag to avoid allocations but there's no need to do it > > per descriptor, set it on vq. > > > > Without using the indirect table, I'm thinking about changing to use > the standard sg (i.e. struct scatterlist), instead of vring_desc, so that > we don't need to modify or add any new functions of virtqueue_add(). > > In this case, we will kmalloc an array of sgs in probe(), and we can add > the sgs one by one to the vq, which won't trigger the allocation of an > indirect table inside virtqueue_add(), and then kick when all are added. > > Best, > Wei And allocate headers too? This can work. API extensions aren't necessarily a bad idea though. The API I suggest above is preferable for the simple reason that it can work without INDIRECT flag support in hypervisor. I wonder which APIs would Nitesh find useful. -- MST From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-f198.google.com (mail-qk0-f198.google.com [209.85.220.198]) by kanga.kvack.org (Postfix) with ESMTP id 64B806B03EA for ; Wed, 21 Jun 2017 08:29:05 -0400 (EDT) Received: by mail-qk0-f198.google.com with SMTP id z22so21306090qka.4 for ; Wed, 21 Jun 2017 05:29:05 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id a195si2692905qkc.122.2017.06.21.05.29.04 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 21 Jun 2017 05:29:04 -0700 (PDT) Date: Wed, 21 Jun 2017 15:28:56 +0300 From: "Michael S. Tsirkin" Subject: Re: [virtio-dev] Re: [PATCH v11 6/6] virtio-balloon: VIRTIO_BALLOON_F_CMD_VQ Message-ID: <20170621151922-mutt-send-email-mst@kernel.org> References: <1497004901-30593-1-git-send-email-wei.w.wang@intel.com> <1497004901-30593-7-git-send-email-wei.w.wang@intel.com> <20170620190343-mutt-send-email-mst@kernel.org> <5949E7C0.3050106@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5949E7C0.3050106@intel.com> Sender: owner-linux-mm@kvack.org List-ID: To: Wei Wang Cc: virtio-dev@lists.oasis-open.org, linux-kernel@vger.kernel.org, qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-mm@kvack.org, david@redhat.com, dave.hansen@intel.com, cornelia.huck@de.ibm.com, akpm@linux-foundation.org, mgorman@techsingularity.net, aarcange@redhat.com, amit.shah@redhat.com, pbonzini@redhat.com, liliang.opensource@gmail.com, riel@redhat.com, nilal@redhat.com On Wed, Jun 21, 2017 at 11:28:00AM +0800, Wei Wang wrote: > On 06/21/2017 12:18 AM, Michael S. Tsirkin wrote: > > On Fri, Jun 09, 2017 at 06:41:41PM +0800, Wei Wang wrote: > > > - if (!virtqueue_indirect_desc_table_add(vq, desc, num)) { > > > + if (!virtqueue_indirect_desc_table_add(vq, desc, *num)) { > > > virtqueue_kick(vq); > > > - wait_event(vb->acked, virtqueue_get_buf(vq, &len)); > > > - vb->balloon_page_chunk.chunk_num = 0; > > > + if (busy_wait) > > > + while (!virtqueue_get_buf(vq, &len) && > > > + !virtqueue_is_broken(vq)) > > > + cpu_relax(); > > > + else > > > + wait_event(vb->acked, virtqueue_get_buf(vq, &len)); > > > > This is something I didn't previously notice. > > As you always keep a single buffer in flight, you do not > > really need indirect at all. Just add all descriptors > > in the ring directly, then kick. > > > > E.g. > > virtqueue_add_first > > virtqueue_add_next > > virtqueue_add_last > > > > ? > > > > You also want a flag to avoid allocations but there's no need to do it > > per descriptor, set it on vq. > > > > Without using the indirect table, I'm thinking about changing to use > the standard sg (i.e. struct scatterlist), instead of vring_desc, so that > we don't need to modify or add any new functions of virtqueue_add(). > > In this case, we will kmalloc an array of sgs in probe(), and we can add > the sgs one by one to the vq, which won't trigger the allocation of an > indirect table inside virtqueue_add(), and then kick when all are added. > > Best, > Wei And allocate headers too? This can work. API extensions aren't necessarily a bad idea though. The API I suggest above is preferable for the simple reason that it can work without INDIRECT flag support in hypervisor. I wonder which APIs would Nitesh find useful. -- MST -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56292) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dNeku-0001Op-M6 for qemu-devel@nongnu.org; Wed, 21 Jun 2017 08:29:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dNekq-0002e2-Pv for qemu-devel@nongnu.org; Wed, 21 Jun 2017 08:29:08 -0400 Received: from mx1.redhat.com ([209.132.183.28]:58014) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dNekq-0002dn-JV for qemu-devel@nongnu.org; Wed, 21 Jun 2017 08:29:04 -0400 Date: Wed, 21 Jun 2017 15:28:56 +0300 From: "Michael S. Tsirkin" Message-ID: <20170621151922-mutt-send-email-mst@kernel.org> References: <1497004901-30593-1-git-send-email-wei.w.wang@intel.com> <1497004901-30593-7-git-send-email-wei.w.wang@intel.com> <20170620190343-mutt-send-email-mst@kernel.org> <5949E7C0.3050106@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5949E7C0.3050106@intel.com> Subject: Re: [Qemu-devel] [virtio-dev] Re: [PATCH v11 6/6] virtio-balloon: VIRTIO_BALLOON_F_CMD_VQ List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Wei Wang Cc: virtio-dev@lists.oasis-open.org, linux-kernel@vger.kernel.org, qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-mm@kvack.org, david@redhat.com, dave.hansen@intel.com, cornelia.huck@de.ibm.com, akpm@linux-foundation.org, mgorman@techsingularity.net, aarcange@redhat.com, amit.shah@redhat.com, pbonzini@redhat.com, liliang.opensource@gmail.com, riel@redhat.com, nilal@redhat.com On Wed, Jun 21, 2017 at 11:28:00AM +0800, Wei Wang wrote: > On 06/21/2017 12:18 AM, Michael S. Tsirkin wrote: > > On Fri, Jun 09, 2017 at 06:41:41PM +0800, Wei Wang wrote: > > > - if (!virtqueue_indirect_desc_table_add(vq, desc, num)) { > > > + if (!virtqueue_indirect_desc_table_add(vq, desc, *num)) { > > > virtqueue_kick(vq); > > > - wait_event(vb->acked, virtqueue_get_buf(vq, &len)); > > > - vb->balloon_page_chunk.chunk_num = 0; > > > + if (busy_wait) > > > + while (!virtqueue_get_buf(vq, &len) && > > > + !virtqueue_is_broken(vq)) > > > + cpu_relax(); > > > + else > > > + wait_event(vb->acked, virtqueue_get_buf(vq, &len)); > > > > This is something I didn't previously notice. > > As you always keep a single buffer in flight, you do not > > really need indirect at all. Just add all descriptors > > in the ring directly, then kick. > > > > E.g. > > virtqueue_add_first > > virtqueue_add_next > > virtqueue_add_last > > > > ? > > > > You also want a flag to avoid allocations but there's no need to do it > > per descriptor, set it on vq. > > > > Without using the indirect table, I'm thinking about changing to use > the standard sg (i.e. struct scatterlist), instead of vring_desc, so that > we don't need to modify or add any new functions of virtqueue_add(). > > In this case, we will kmalloc an array of sgs in probe(), and we can add > the sgs one by one to the vq, which won't trigger the allocation of an > indirect table inside virtqueue_add(), and then kick when all are added. > > Best, > Wei And allocate headers too? This can work. API extensions aren't necessarily a bad idea though. The API I suggest above is preferable for the simple reason that it can work without INDIRECT flag support in hypervisor. I wonder which APIs would Nitesh find useful. -- MST