From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.2 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87494C433DB for ; Wed, 17 Feb 2021 09:42:30 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2789764D9A for ; Wed, 17 Feb 2021 09:42:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2789764D9A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.86180.161525 (Exim 4.92) (envelope-from ) id 1lCJLO-0002Wt-1P; Wed, 17 Feb 2021 09:42:02 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 86180.161525; Wed, 17 Feb 2021 09:42:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lCJLN-0002Wm-UZ; Wed, 17 Feb 2021 09:42:01 +0000 Received: by outflank-mailman (input) for mailman id 86180; Wed, 17 Feb 2021 09:42:00 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lCJLM-0002Wh-30 for xen-devel@lists.xenproject.org; Wed, 17 Feb 2021 09:42:00 +0000 Received: from mail-wr1-x42e.google.com (unknown [2a00:1450:4864:20::42e]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 1611a42b-b7b4-4c0a-b152-a0ba389d5b12; Wed, 17 Feb 2021 09:41:58 +0000 (UTC) Received: by mail-wr1-x42e.google.com with SMTP id r21so16597736wrr.9 for ; Wed, 17 Feb 2021 01:41:58 -0800 (PST) Received: from ?IPv6:2a00:23c5:5785:9a01:71b0:bf69:5f0d:b70f? ([2a00:23c5:5785:9a01:71b0:bf69:5f0d:b70f]) by smtp.gmail.com with ESMTPSA id o13sm3968880wrs.45.2021.02.17.01.41.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 17 Feb 2021 01:41:57 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1611a42b-b7b4-4c0a-b152-a0ba389d5b12 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:reply-to:subject:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=H9YiCxPC3EWKAD/DFSNhs4Ry4EYfjJv5o1ZFvoYgAxU=; b=X8SNqkdKYWiEpgfGlmblSR1TfoWzv/Zw+itn4ZAw9ABm9hLvgspWpFUaGxzdiYgGKO cP/PB8nMsOnBtxuogTHBwjBrUmNOsHZueHl30HXHxrPg31/lft//ccAcJX3IjPhyakUy Et7S65k/fIJiL+HGvxeplAG1N20fOA0DobTp23sdgxgfGHVw6wQo6217tCeqOjlLjzhB niSy2VqU/eEthqhnzC3Pq8K26halit0KSqPConW7Ux6v6Ts9kBSRttOcvp+5C/132dpG YQ4xWi20s6jKFMWeN+GylMMa8R69J5lW2m5MFoL+gShVtsBFVRnmPo3FyhQpmvfAlq8s 33yA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:reply-to:subject:to:cc:references :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=H9YiCxPC3EWKAD/DFSNhs4Ry4EYfjJv5o1ZFvoYgAxU=; b=mI/jtJihQ4aE49VwBHx0SuNHrgIImtRYXoPDi2hvkv5JzgSyLG0CkWbplsNKVOFlRM B93V+XfKxmk9HsUkwciD/s7Chc7ASSkUgIQXQxBvPPiu/US74vFv31P3SVEXzeXzvIaV yT5hZsqiWyiMOXpPlAvPItmhS5KEjciBDT8aMT1gjkw4u/cHuHjrzDkbftYy+Ug1z2iw PRvJH2AGM+PVEJs17Q86fcowqQ2yIiy5Q8AtSd3WFQ2kqqe8UzCRw6Dy+PKuPoXRA/M9 +lsJO9AGhGU65gajvRSpNw85Vjf6jy8M0vv6Tzbq/WMx4cxbj/LMRhOL8cnNqtT1wB6R Jh0Q== X-Gm-Message-State: AOAM533hYluius0O1dY7gha9Q3G++Q2uVmH0AOIBBUd1o2MJUpDaj2Le J2B+a3ZAu5bNzEn1xw/ZK6fPJhe1kouPgQ== X-Google-Smtp-Source: ABdhPJzoVsEvclI2KTyTxhqm3DQdUCEZtjnNXr2qlaRSsp548D0t0p/l5wZODJjOi3+9Povuwcrw7w== X-Received: by 2002:a5d:4850:: with SMTP id n16mr28684852wrs.296.1613554917998; Wed, 17 Feb 2021 01:41:57 -0800 (PST) From: Paul Durrant X-Google-Original-From: Paul Durrant Reply-To: paul@xen.org Subject: Re: Ping: [PATCH v2 2/2] IOREQ: refine when to send mapcache invalidation request To: Jan Beulich Cc: 'Andrew Cooper' , 'Wei Liu' , =?UTF-8?B?J1JvZ2VyIFBhdSBNb25uw6kn?= , 'Julien Grall' , 'Stefano Stabellini' , 'George Dunlap' , xen-devel@lists.xenproject.org References: <0e7265fe-8d89-facb-790d-9232c742c3fa@suse.com> <03fb01d6fad7$c39087b0$4ab19710$@xen.org> <006bd542-e213-a6ad-7812-e91fed7093a3@suse.com> Message-ID: <56900eda-9718-f68a-8a05-99a8e713446d@xen.org> Date: Wed, 17 Feb 2021 09:41:35 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <006bd542-e213-a6ad-7812-e91fed7093a3@suse.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit On 17/02/2021 08:30, Jan Beulich wrote: > Paul (or others), thoughts? > > On 04.02.2021 12:24, Jan Beulich wrote: >> On 04.02.2021 10:26, Paul Durrant wrote: >>>> From: Jan Beulich >>>> Sent: 02 February 2021 15:15 >>>> >>>> XENMEM_decrease_reservation isn't the only means by which pages can get >>>> removed from a guest, yet all removals ought to be signaled to qemu. Put >>>> setting of the flag into the central p2m_remove_page() underlying all >>>> respective hypercalls as well as a few similar places, mainly in PoD >>>> code. >>>> >>>> Additionally there's no point sending the request for the local domain >>>> when the domain acted upon is a different one. The latter domain's ioreq >>>> server mapcaches need invalidating. We assume that domain to be paused >>>> at the point the operation takes place, so sending the request in this >>>> case happens from the hvm_do_resume() path, which as one of its first >>>> steps calls handle_hvm_io_completion(). >>>> >>>> Even without the remote operation aspect a single domain-wide flag >>>> doesn't do: Guests may e.g. decrease-reservation on multiple vCPU-s in >>>> parallel. Each of them needs to issue an invalidation request in due >>>> course, in particular because exiting to guest context should not happen >>>> before the request was actually seen by (all) the emulator(s). >>>> >>>> Signed-off-by: Jan Beulich >>>> --- >>>> v2: Preemption related adjustment split off. Make flag per-vCPU. More >>>> places to set the flag. Also handle acting on a remote domain. >>>> Re-base. >>> >>> I'm wondering if a per-vcpu flag is overkill actually. We just need >>> to make sure that we don't miss sending an invalidation where >>> multiple vcpus are in play. The mapcache in the emulator is global >>> so issuing an invalidate for every vcpu is going to cause an >>> unnecessary storm of ioreqs, isn't it? >> >> The only time a truly unnecessary storm would occur is when for >> an already running guest mapcache invalidation gets triggered >> by a remote domain. This should be a pretty rare event, so I >> think the storm in this case ought to be tolerable. >> >>> Could we get away with the per-domain atomic counter? >> >> Possible, but quite involved afaict: The potential storm above >> is the price to pay for the simplicity of the model. It is >> important to note that while we don't need all of the vCPU-s >> to send these ioreqs, we need all of them to wait for the >> request(s) to be acked. And this waiting is what we get "for >> free" using the approach here, whereas we'd need to introduce >> new logic for this with an atomic counter (afaict at least). >> >> Jan >> > Ok, let's take the patch as-is then. Reviewed-by: Paul Durrant