From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 822ECC43387 for ; Tue, 25 Dec 2018 10:09:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4A9F421773 for ; Tue, 25 Dec 2018 10:09:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725884AbeLYKJ0 (ORCPT ); Tue, 25 Dec 2018 05:09:26 -0500 Received: from mx1.redhat.com ([209.132.183.28]:56894 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725845AbeLYKJZ (ORCPT ); Tue, 25 Dec 2018 05:09:25 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 883AF83F3E; Tue, 25 Dec 2018 10:09:25 +0000 (UTC) Received: from [10.72.12.17] (ovpn-12-17.pek2.redhat.com [10.72.12.17]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 785AC5D717; Tue, 25 Dec 2018 10:09:21 +0000 (UTC) Subject: Re: [PATCH net-next 0/3] vhost: accelerate metadata access through vmap() To: "Michael S. Tsirkin" Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org References: <20181213101022.12475-1-jasowang@redhat.com> <20181213102315-mutt-send-email-mst@kernel.org> <9459e227-a943-8553-732b-d7f5225a0f22@redhat.com> <20181214072334-mutt-send-email-mst@kernel.org> <20181224131040-mutt-send-email-mst@kernel.org> From: Jason Wang Message-ID: <51fa034d-99ae-3820-c3a4-d9e6f2eefe34@redhat.com> Date: Tue, 25 Dec 2018 18:09:19 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <20181224131040-mutt-send-email-mst@kernel.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 25 Dec 2018 10:09:25 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2018/12/25 上午2:12, Michael S. Tsirkin wrote: > On Mon, Dec 24, 2018 at 04:32:39PM +0800, Jason Wang wrote: >> On 2018/12/14 下午8:33, Michael S. Tsirkin wrote: >>> On Fri, Dec 14, 2018 at 11:42:18AM +0800, Jason Wang wrote: >>>> On 2018/12/13 下午11:27, Michael S. Tsirkin wrote: >>>>> On Thu, Dec 13, 2018 at 06:10:19PM +0800, Jason Wang wrote: >>>>>> Hi: >>>>>> >>>>>> This series tries to access virtqueue metadata through kernel virtual >>>>>> address instead of copy_user() friends since they had too much >>>>>> overheads like checks, spec barriers or even hardware feature >>>>>> toggling. >>>>> Userspace accesses through remapping tricks and next time there's a need >>>>> for a new barrier we are left to figure it out by ourselves. >>>> I don't get here, do you mean spec barriers? >>> I mean the next barrier people decide to put into userspace >>> memory accesses. >>> >>>> It's completely unnecessary for >>>> vhost which is kernel thread. >>> It's defence in depth. Take a look at the commit that added them. >>> And yes quite possibly in most cases we actually have a spec >>> barrier in the validation phase. If we do let's use the >>> unsafe variants so they can be found. >> >> unsafe variants can only work if you can batch userspace access. This is not >> necessarily the case for light load. > > Do we care a lot about the light load? How would you benchmark it? > If we can reduce the latency that's will be more than what we expect. 1 byte TCP_RR shows 1.5%-2% improvement. >>>> And even if you're right, vhost is not the >>>> only place, there's lots of vmap() based accessing in kernel. >>> For sure. But if one can get by without get user pages, one >>> really should. Witness recently uncovered mess with file >>> backed storage. >> >> We only pin metadata pages, I don't believe they will be used by any DMA. > It doesn't matter really, if you dirty pages behind the MM back > the problem is there. Ok, but the usual case is anonymous pages, do we use file backed pages for user of vhost? And even if we use sometime, according to the pointer it's not something that can fix, RFC has been posted to solve this issue. Thanks From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jason Wang Subject: Re: [PATCH net-next 0/3] vhost: accelerate metadata access through vmap() Date: Tue, 25 Dec 2018 18:09:19 +0800 Message-ID: <51fa034d-99ae-3820-c3a4-d9e6f2eefe34@redhat.com> References: <20181213101022.12475-1-jasowang@redhat.com> <20181213102315-mutt-send-email-mst@kernel.org> <9459e227-a943-8553-732b-d7f5225a0f22@redhat.com> <20181214072334-mutt-send-email-mst@kernel.org> <20181224131040-mutt-send-email-mst@kernel.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, virtualization@lists.linux-foundation.org To: "Michael S. Tsirkin" Return-path: In-Reply-To: <20181224131040-mutt-send-email-mst@kernel.org> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org List-Id: netdev.vger.kernel.org Ck9uIDIwMTgvMTIvMjUg5LiK5Y2IMjoxMiwgTWljaGFlbCBTLiBUc2lya2luIHdyb3RlOgo+IE9u IE1vbiwgRGVjIDI0LCAyMDE4IGF0IDA0OjMyOjM5UE0gKzA4MDAsIEphc29uIFdhbmcgd3JvdGU6 Cj4+IE9uIDIwMTgvMTIvMTQg5LiL5Y2IODozMywgTWljaGFlbCBTLiBUc2lya2luIHdyb3RlOgo+ Pj4gT24gRnJpLCBEZWMgMTQsIDIwMTggYXQgMTE6NDI6MThBTSArMDgwMCwgSmFzb24gV2FuZyB3 cm90ZToKPj4+PiBPbiAyMDE4LzEyLzEzIOS4i+WNiDExOjI3LCBNaWNoYWVsIFMuIFRzaXJraW4g d3JvdGU6Cj4+Pj4+IE9uIFRodSwgRGVjIDEzLCAyMDE4IGF0IDA2OjEwOjE5UE0gKzA4MDAsIEph c29uIFdhbmcgd3JvdGU6Cj4+Pj4+PiBIaToKPj4+Pj4+Cj4+Pj4+PiBUaGlzIHNlcmllcyB0cmll cyB0byBhY2Nlc3MgdmlydHF1ZXVlIG1ldGFkYXRhIHRocm91Z2gga2VybmVsIHZpcnR1YWwKPj4+ Pj4+IGFkZHJlc3MgaW5zdGVhZCBvZiBjb3B5X3VzZXIoKSBmcmllbmRzIHNpbmNlIHRoZXkgaGFk IHRvbyBtdWNoCj4+Pj4+PiBvdmVyaGVhZHMgbGlrZSBjaGVja3MsIHNwZWMgYmFycmllcnMgb3Ig ZXZlbiBoYXJkd2FyZSBmZWF0dXJlCj4+Pj4+PiB0b2dnbGluZy4KPj4+Pj4gVXNlcnNwYWNlIGFj Y2Vzc2VzIHRocm91Z2ggcmVtYXBwaW5nIHRyaWNrcyBhbmQgbmV4dCB0aW1lIHRoZXJlJ3MgYSBu ZWVkCj4+Pj4+IGZvciBhIG5ldyBiYXJyaWVyIHdlIGFyZSBsZWZ0IHRvIGZpZ3VyZSBpdCBvdXQg Ynkgb3Vyc2VsdmVzLgo+Pj4+IEkgZG9uJ3QgZ2V0IGhlcmUsIGRvIHlvdSBtZWFuIHNwZWMgYmFy cmllcnM/Cj4+PiBJIG1lYW4gdGhlIG5leHQgYmFycmllciBwZW9wbGUgZGVjaWRlIHRvIHB1dCBp bnRvIHVzZXJzcGFjZQo+Pj4gbWVtb3J5IGFjY2Vzc2VzLgo+Pj4KPj4+PiBJdCdzIGNvbXBsZXRl bHkgdW5uZWNlc3NhcnkgZm9yCj4+Pj4gdmhvc3Qgd2hpY2ggaXMga2VybmVsIHRocmVhZC4KPj4+ IEl0J3MgZGVmZW5jZSBpbiBkZXB0aC4gVGFrZSBhIGxvb2sgYXQgdGhlIGNvbW1pdCB0aGF0IGFk ZGVkIHRoZW0uCj4+PiBBbmQgeWVzIHF1aXRlIHBvc3NpYmx5IGluIG1vc3QgY2FzZXMgd2UgYWN0 dWFsbHkgaGF2ZSBhIHNwZWMKPj4+IGJhcnJpZXIgaW4gdGhlIHZhbGlkYXRpb24gcGhhc2UuIElm IHdlIGRvIGxldCdzIHVzZSB0aGUKPj4+IHVuc2FmZSB2YXJpYW50cyBzbyB0aGV5IGNhbiBiZSBm b3VuZC4KPj4KPj4gdW5zYWZlIHZhcmlhbnRzIGNhbiBvbmx5IHdvcmsgaWYgeW91IGNhbiBiYXRj aCB1c2Vyc3BhY2UgYWNjZXNzLiBUaGlzIGlzIG5vdAo+PiBuZWNlc3NhcmlseSB0aGUgY2FzZSBm b3IgbGlnaHQgbG9hZC4KPgo+IERvIHdlIGNhcmUgYSBsb3QgYWJvdXQgdGhlIGxpZ2h0IGxvYWQ/ IEhvdyB3b3VsZCB5b3UgYmVuY2htYXJrIGl0Pwo+CgpJZiB3ZSBjYW4gcmVkdWNlIHRoZSBsYXRl bmN5IHRoYXQncyB3aWxsIGJlIG1vcmUgdGhhbiB3aGF0IHdlIGV4cGVjdC4KCjEgYnl0ZSBUQ1Bf UlIgc2hvd3MgMS41JS0yJSBpbXByb3ZlbWVudC4KCgo+Pj4+IEFuZCBldmVuIGlmIHlvdSdyZSBy aWdodCwgdmhvc3QgaXMgbm90IHRoZQo+Pj4+IG9ubHkgcGxhY2UsIHRoZXJlJ3MgbG90cyBvZiB2 bWFwKCkgYmFzZWQgYWNjZXNzaW5nIGluIGtlcm5lbC4KPj4+IEZvciBzdXJlLiBCdXQgaWYgb25l IGNhbiBnZXQgYnkgd2l0aG91dCBnZXQgdXNlciBwYWdlcywgb25lCj4+PiByZWFsbHkgc2hvdWxk LiBXaXRuZXNzIHJlY2VudGx5IHVuY292ZXJlZCBtZXNzIHdpdGggZmlsZQo+Pj4gYmFja2VkIHN0 b3JhZ2UuCj4+Cj4+IFdlIG9ubHkgcGluIG1ldGFkYXRhIHBhZ2VzLCBJIGRvbid0IGJlbGlldmUg dGhleSB3aWxsIGJlIHVzZWQgYnkgYW55IERNQS4KPiBJdCBkb2Vzbid0IG1hdHRlciByZWFsbHks IGlmIHlvdSBkaXJ0eSBwYWdlcyBiZWhpbmQgdGhlIE1NIGJhY2sKPiB0aGUgcHJvYmxlbSBpcyB0 aGVyZS4KCgpPaywgYnV0IHRoZSB1c3VhbCBjYXNlIGlzIGFub255bW91cyBwYWdlcywgZG8gd2Ug dXNlIGZpbGUgYmFja2VkIHBhZ2VzIApmb3IgdXNlciBvZiB2aG9zdD8gQW5kIGV2ZW4gaWYgd2Ug dXNlIHNvbWV0aW1lLCBhY2NvcmRpbmcgdG8gdGhlIHBvaW50ZXIgCml0J3Mgbm90IHNvbWV0aGlu ZyB0aGF0IGNhbiBmaXgsIFJGQyBoYXMgYmVlbiBwb3N0ZWQgdG8gc29sdmUgdGhpcyBpc3N1ZS4K ClRoYW5rcwoKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f ClZpcnR1YWxpemF0aW9uIG1haWxpbmcgbGlzdApWaXJ0dWFsaXphdGlvbkBsaXN0cy5saW51eC1m b3VuZGF0aW9uLm9yZwpodHRwczovL2xpc3RzLmxpbnV4Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9s aXN0aW5mby92aXJ0dWFsaXphdGlvbg==