From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E75F3C48BE0 for ; Fri, 11 Jun 2021 05:44:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C4C1361009 for ; Fri, 11 Jun 2021 05:44:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231132AbhFKFqJ (ORCPT ); Fri, 11 Jun 2021 01:46:09 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:20789 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230001AbhFKFqI (ORCPT ); Fri, 11 Jun 2021 01:46:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623390250; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5nlidzY7vVox2ieeNTmLbeG44JwkLPe3MMhpIq19x6g=; b=LOiTUDx60R9wOz9sP1ZGZtlwFXZen6zjRQvWMitayuXLMNIcTLGlDvaUF3WVODXI8I35q7 iC5GJros2ztPy4IZuHHeOWvIAZPoIshMuJIs4cISIPcMr8QCZIZeoMxJ19vNCCOGDXS0aU rlpnURci5sjJN8MvfCzJQE+fbqDvKZ0= Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-321-YXNFEZA0ON6mm0nD4C4AZQ-1; Fri, 11 Jun 2021 01:44:09 -0400 X-MC-Unique: YXNFEZA0ON6mm0nD4C4AZQ-1 Received: by mail-pg1-f200.google.com with SMTP id a10-20020a65418a0000b029021b78388f57so1085398pgq.15 for ; Thu, 10 Jun 2021 22:44:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=5nlidzY7vVox2ieeNTmLbeG44JwkLPe3MMhpIq19x6g=; b=Au6v3TGgT9vIRiIPpbgH/ZNCs0pprsoh/5EedVYx6h2Ym0BPFAMRa3fql6y6t3Q6ND 6Z6KH8karyLG5wPP2+NN9HMtEN/HfYQDxcDMG6N81uGDFj9aRmKsO0K0q+pMkUN+peTy kQCakiM//UUCx8fAFWajh6dGePEyKqYWb+FC9r0FzRdA2VI7OyoFRtt/9RzKShPkx7nK lGsIMMAux8SGdSq+neAQvzzAZ21KQpXATDlF4Sffn/kBgKUCmO4QR0EOD/4fw/xJs8gI vF3ZbrWxRZrwNk+z2/3Ccqaq/flnW1ZdB/ijOZ7OhPOQPA/JFP/JaWhuTfka0w8jNdBo Yw/Q== X-Gm-Message-State: AOAM530lPIuYAge2O2cOX342T5NrpfKtKsdn13yAkEDGskggyRR/mnmt vFkXu/1u6YDqt66AiNDhuhZLnNexH+AJAmOfqjyuN2xmLIIz7n6kifZVH78HL6pEQzuRdZAueqS URTmFXcvZ9OVwj7rpKGS0HhcO X-Received: by 2002:a63:6547:: with SMTP id z68mr1935901pgb.341.1623390248323; Thu, 10 Jun 2021 22:44:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyKB65M93ciDtRjp4dLL7boO1gINllpJ4guVwZKP5dhsZp4Ljb4TFphg7Htp0BodlrU9JnK0w== X-Received: by 2002:a63:6547:: with SMTP id z68mr1935883pgb.341.1623390248048; Thu, 10 Jun 2021 22:44:08 -0700 (PDT) Received: from wangxiaodeMacBook-Air.local ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id d8sm4085729pfq.198.2021.06.10.22.44.04 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 10 Jun 2021 22:44:07 -0700 (PDT) Subject: Re: [RFC] /dev/ioasid uAPI proposal To: Jason Gunthorpe Cc: "Tian, Kevin" , Lu Baolu , Liu Yi L , "kvm@vger.kernel.org" , Jonathan Corbet , "iommu@lists.linux-foundation.org" , LKML , "Alex Williamson (alex.williamson@redhat.com)\"\"" , David Woodhouse References: <05d7f790-870d-5551-1ced-86926a0aa1a6@redhat.com> <42a71462-1abc-0404-156c-60a7ee1ad333@redhat.com> <20210601173138.GM1002214@nvidia.com> <20210602172154.GC1002214@nvidia.com> <20210608132039.GG1002214@nvidia.com> <20210610114751.GK1002214@nvidia.com> From: Jason Wang Message-ID: Date: Fri, 11 Jun 2021 13:43:59 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <20210610114751.GK1002214@nvidia.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 在 2021/6/10 下午7:47, Jason Gunthorpe 写道: > On Thu, Jun 10, 2021 at 10:00:01AM +0800, Jason Wang wrote: >> 在 2021/6/8 下午9:20, Jason Gunthorpe 写道: >>> On Tue, Jun 08, 2021 at 09:10:42AM +0800, Jason Wang wrote: >>> >>>> Well, this sounds like a re-invention of io_uring which has already worked >>>> for multifds. >>> How so? io_uring is about sending work to the kernel, not getting >>> structued events back? >> >> Actually it can. Userspace can poll multiple fds via preparing multiple sqes >> with IORING_OP_ADD flag. > Poll is only a part of what is needed here, the main issue is > transfering the PRI events to userspace quickly. Do we really care e.g at most one more syscall in this case? I think the time spent on demand paging is much more than transferring #PF to userspace. What's more, a well designed vIOMMU capable IOMMU hardware should have the ability to inject such event directly to guest if #PF happens on L1. > >> This means another ring and we need introduce ioctl() to add or remove >> ioasids from the poll. And it still need a kind of fallback like a list if >> the ring is full. > The max size of the ring should be determinable based on the PRI > concurrance of each device and the number of devices sharing the ring This has at least one assumption, #PF event is the only event for the ring, I'm not sure this is the case. Thanks > > In any event, I'm not entirely convinced eliding the PRI user/kernel > copy is the main issue here.. If we want this to be low latency I > think it ends up with some kernel driver component assisting the > vIOMMU emulation and avoiding the round trip to userspace > > Jason >