From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751318AbcGNNlE (ORCPT ); Thu, 14 Jul 2016 09:41:04 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41882 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751054AbcGNNlB (ORCPT ); Thu, 14 Jul 2016 09:41:01 -0400 Date: Thu, 14 Jul 2016 15:40:56 +0200 From: Radim =?utf-8?B?S3LEjW3DocWZ?= To: Suravee Suthikulpanit Cc: joro@8bytes.org, pbonzini@redhat.com, alex.williamson@redhat.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, sherry.hurwitz@amd.com Subject: Re: [PART2 PATCH v4 07/11] iommu/amd: Introduce amd_iommu_update_ga() Message-ID: <20160714134055.GD16130@potion> References: <1468416032-7692-1-git-send-email-suravee.suthikulpanit@amd.com> <1468416032-7692-8-git-send-email-suravee.suthikulpanit@amd.com> <20160713141457.GF21976@potion> <578757A8.3000200@amd.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <578757A8.3000200@amd.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Thu, 14 Jul 2016 13:41:00 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 2016-07-14 16:13+0700, Suravee Suthikulpanit: > On 7/13/16 21:14, Radim Krčmář wrote: >> 2016-07-13 08:20-0500, Suravee Suthikulpanit: >> > diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c >> > @@ -4461,4 +4461,69 @@ int amd_iommu_create_irq_domain(struct amd_iommu *iommu) >> > +int amd_iommu_update_ga(u32 vcpu_id, u32 cpu, u32 vm_id, >> > + u64 base, bool is_run) >> >> |2016-07-13 15:49+0700, Suravee Suthikulpanit: >> |> On 07/12/2016 01:59 AM, Radim Krčmář wrote: >> |>> Not just in this function does the interface between svm and iommu split >> |>> ga_tag into its two components (vcpu_id and ga_tag), but it seems that >> |>> the combined value could always be used instead ... >> |>> Is there an advantage to passing two values? >> |> >> |> Here, the amd_iommu_update_ga() takes the two separate value for input >> |> parameters. Mainly the ga_tag (which is really the vm_id) and vcpu_id. This >> |> allow IOMMU driver to decide how to encode the GATAG to be programmed into >> |> the IRTE. Currently, the actual GATAG is a 16-bit value, . >> |> This keeps the interface independent from how we encode the GATAG. >> >> I was thinking about making the IOMMU unaware how SVM or other Linux >> hypervisors use the ga_tag, i.e. passing the final u32 ga_tag. >> For example 32 bit hypervisor doesn't need to use lookup, because any >> pointer can used as the ga_tag directly. > > Ahh....... (w/ a big light bulb) > I get your point now. Let's just have SVM (or other hypervisor) define what > the tag should be and just pass-on the value to IOMMU. IOMMU can just simply > use this w/o knowing what it is. Sorry, I'm slow :) That is what I meant, but misunderstanding is a product of both participants. I didn't write it clearly on the first try. >> > + hash_for_each_possible(iommu->gatag_ir_hash, ir_data, hnode, >> > + AMD_IOMMU_GATAG(vm_id, vcpu_id)) { >> > + struct irte_ga *irte = (struct irte_ga *) ir_data->entry; >> >> |>> (The ga_tag check is missing here too.) >> |> >> |> Here, the intention is to update all interrupt remapping entries in the >> |> bucket w/ the same GATAG (i.e. vm_id + vcpu_id), where GATAG = >> |> AMD_IOMMU_GATAG(vm_id, vcpu_id). >> >> Which is why you need to check that >> AMD_IOMMU_GATAG(vm_id, vcpu_id) == entry->fields_vapic.ga_tag >> >> The hashing function can map two different vm_id + vcpu_id to the same >> bucket and hash_for_each_possible() would return both of them, but only >> one belongs to the VCPU that we want to update. >> >> (And shouldn't there be only one match?) > > Actually, with your suggestion above, the hask key would be (vm_id & > 0x3FFFFF << 8)| (vcpu_id & 0xFF). So, it should be unique for each vcpu of > each vm, or am I still missing something? [Reply in the followup mail.] > Also, since we will not be passing the vmid and vcpuid as separate value, > and just passing the (u32)ga_tag, we would not be able to do the check you > suggested here. There will be the u32 ga_tag argument, so you would still do ga_tag == entry->fields_vapic.ga_tag Because even if the ga_tag is unique for every vcpu, the hash table will mix various vcpus into one bucket and you need to filter them. >> > + update_irte_ga((struct irte_ga *)ir_data->ref, >> > + ir_data->irq_2_irte.devid, >> > + base, cpu, is_run); >> >> |>> (The lookup leading up to here is avoidable -- svm, the caller, has the >> |>> ability to map ga_tag into irte/ir_data directly with a pointer. > > I'm not sure about this optimization to avoid look up. > > The struct amd_ir_data is part of the IOMMU driver, and the SVM knows > nothing about it. I don't think it would be able to find out the pointer to > amd_ir_data/irte. Yeah, SVM would store it in a "void *" pointer, because it doesn't need to know anything else, but you still need to retrieve it from IOMMU, which could be done through vcpu_info argument to amd_ir_set_vcpu_affinity(). (I am not sure if it doesn't breach isolation of IOMMU, so we might not want to do it in any case ...) > Also, with the current design, each ga_tag can be mapped to different irte > since there could be multiple interrupts targeting a particular cpu. Here, > we would want to update all of the IRTEs with the same ga_tag. True, that design is good. SVM would need a list of pointers for each vcpu to cope with it ... >> |>> I'm not sure if the lookup is slow enough to pardon optimization, but >> |>> it might make the code simpler as well.) >> |> >> |> I might have mislead you up to this point. Not sure if the assumption here >> |> still hold with my explanation above. Sorry for confusion. >> >> SVM configures IOMMU with ga_tag, so IOMMU could return the pointer to >> ir_data/irte that was just configured. > > Also, IIUC, you want to use the pointer to ir_data/irte as the ga_tag value. > The issue would be ga_tag is a 32-bit value, and this would not work with > 64-bit address. I mean something slightly different. Instead of passing ga_tag into amd_iommu_update_ga(), just pass void * of whatever IOMMU provided back when SVM configured the interrupt. ga_tag will never come into play. (The vcpu lookup from ga_tag is necessary, when processing the queue of undelivered interrupts. ir_data lookup can be avoided.)