From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
To: "Radim Krčmář" <rkrcmar@redhat.com>
Cc: <joro@8bytes.org>, <pbonzini@redhat.com>,
<alex.williamson@redhat.com>, <kvm@vger.kernel.org>,
<linux-kernel@vger.kernel.org>, <sherry.hurwitz@amd.com>
Subject: Re: [PART2 PATCH v4 07/11] iommu/amd: Introduce amd_iommu_update_ga()
Date: Thu, 14 Jul 2016 16:13:12 +0700 [thread overview]
Message-ID: <578757A8.3000200@amd.com> (raw)
In-Reply-To: <20160713141457.GF21976@potion>
Hi Radim,
On 7/13/16 21:14, Radim Krčmář wrote:
> [I pasted v3 reviews prefixed with a pipe where I think they still apply.]
>
> 2016-07-13 08:20-0500, Suravee Suthikulpanit:
>> From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
>>
>> Introduces a new IOMMU API, amd_iommu_update_ga(), which allows
>> KVM (SVM) to update existing posted interrupt IOMMU IRTE when
>> load/unload vcpu.
>>
>> Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
>> ---
>> diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
>> @@ -4461,4 +4461,69 @@ int amd_iommu_create_irq_domain(struct amd_iommu *iommu)
>> +int amd_iommu_update_ga(u32 vcpu_id, u32 cpu, u32 vm_id,
>> + u64 base, bool is_run)
>
> |2016-07-13 15:49+0700, Suravee Suthikulpanit:
> |> On 07/12/2016 01:59 AM, Radim Krčmář wrote:
> |>> Not just in this function does the interface between svm and iommu split
> |>> ga_tag into its two components (vcpu_id and ga_tag), but it seems that
> |>> the combined value could always be used instead ...
> |>> Is there an advantage to passing two values?
> |>
> |> Here, the amd_iommu_update_ga() takes the two separate value for input
> |> parameters. Mainly the ga_tag (which is really the vm_id) and vcpu_id. This
> |> allow IOMMU driver to decide how to encode the GATAG to be programmed into
> |> the IRTE. Currently, the actual GATAG is a 16-bit value, <vm_id><vcpu_id>.
> |> This keeps the interface independent from how we encode the GATAG.
>
> I was thinking about making the IOMMU unaware how SVM or other Linux
> hypervisors use the ga_tag, i.e. passing the final u32 ga_tag.
> For example 32 bit hypervisor doesn't need to use lookup, because any
> pointer can used as the ga_tag directly.
Ahh....... (w/ a big light bulb)
I get your point now. Let's just have SVM (or other hypervisor) define
what the tag should be and just pass-on the value to IOMMU. IOMMU can
just simply use this w/o knowing what it is. Sorry, I'm slow :)
> And there are other viable algoritms for assigning the ga_tag --
> why isn't the vm_id 24 bits?
Good point! Actually, I am somehow limited to 30-bit hash value. So, the
VM_ID can be 22 bits, I'll make that change.
>
>> + unsigned long flags;
>> + struct amd_iommu *iommu;
>> +
>> + if (!AMD_IOMMU_GUEST_IR_VAPIC(amd_iommu_guest_ir))
>> + return 0;
>> +
>> + for_each_iommu(iommu) {
>> + struct amd_ir_data *ir_data;
>> +
>> + spin_lock_irqsave(&iommu->gatag_ir_hash_lock, flags);
>> +
>> + /* Note:
>> + * We need to update all interrupt remapping table entries
>> + * for targeting the specified vcpu. Here, we use gatag
>> + * as a hash key and iterate through all entries in the bucket.
>> + */
>> + hash_for_each_possible(iommu->gatag_ir_hash, ir_data, hnode,
>> + AMD_IOMMU_GATAG(vm_id, vcpu_id)) {
>> + struct irte_ga *irte = (struct irte_ga *) ir_data->entry;
>
> |>> (The ga_tag check is missing here too.)
> |>
> |> Here, the intention is to update all interrupt remapping entries in the
> |> bucket w/ the same GATAG (i.e. vm_id + vcpu_id), where GATAG =
> |> AMD_IOMMU_GATAG(vm_id, vcpu_id).
>
> Which is why you need to check that
> AMD_IOMMU_GATAG(vm_id, vcpu_id) == entry->fields_vapic.ga_tag
>
> The hashing function can map two different vm_id + vcpu_id to the same
> bucket and hash_for_each_possible() would return both of them, but only
> one belongs to the VCPU that we want to update.
>
> (And shouldn't there be only one match?)
Actually, with your suggestion above, the hask key would be (vm_id &
0x3FFFFF << 8)| (vcpu_id & 0xFF). So, it should be unique for each vcpu
of each vm, or am I still missing something?
Also, since we will not be passing the vmid and vcpuid as separate
value, and just passing the (u32)ga_tag, we would not be able to do the
check you suggested here.
>
>> +
>> + if (!irte->lo.fields_vapic.guest_mode)
>> + continue;
>> +
>> + update_irte_ga((struct irte_ga *)ir_data->ref,
>> + ir_data->irq_2_irte.devid,
>> + base, cpu, is_run);
>
> |>> (The lookup leading up to here is avoidable -- svm, the caller, has the
> |>> ability to map ga_tag into irte/ir_data directly with a pointer.
I'm not sure about this optimization to avoid look up.
The struct amd_ir_data is part of the IOMMU driver, and the SVM knows
nothing about it. I don't think it would be able to find out the pointer
to amd_ir_data/irte.
Also, with the current design, each ga_tag can be mapped to different
irte since there could be multiple interrupts targeting a particular
cpu. Here, we would want to update all of the IRTEs with the same ga_tag.
> |>> I'm not sure if the lookup is slow enough to pardon optimization, but
> |>> it might make the code simpler as well.)
> |>
> |> I might have mislead you up to this point. Not sure if the assumption here
> |> still hold with my explanation above. Sorry for confusion.
>
> SVM configures IOMMU with ga_tag, so IOMMU could return the pointer to
> ir_data/irte that was just configured.
Also, IIUC, you want to use the pointer to ir_data/irte as the ga_tag
value. The issue would be ga_tag is a 32-bit value, and this would not
work with 64-bit address.
> SVM would couple it with a VCPU
> (and hence a ga_tag) and when amd_iommu_update_ga() was needed, SVM
> would pass the ir_data/irte pointer directly, instead of looking it up
> though a ga_tag.
Please let me know if I am still missing any points.
Thanks,
Suravee
next prev parent reply other threads:[~2016-07-14 9:13 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-07-13 13:20 [PART2 PATCH v4 00/11] iommu/AMD: Introduce IOMMU AVIC support Suravee Suthikulpanit
2016-07-13 13:20 ` [PART2 PATCH v4 01/11] iommu/amd: Detect and enable guest vAPIC support Suravee Suthikulpanit
2016-07-13 13:20 ` [PART2 PATCH v4 02/11] iommu/amd: Move and introduce new IRTE-related unions and structures Suravee Suthikulpanit
2016-07-13 13:20 ` [PART2 PATCH v4 03/11] iommu/amd: Introduce interrupt remapping ops structure Suravee Suthikulpanit
2016-07-13 13:20 ` [PART2 PATCH v4 04/11] iommu/amd: Add support for multiple IRTE formats Suravee Suthikulpanit
2016-07-13 13:20 ` [PART2 PATCH v4 05/11] iommu/amd: Detect and initialize guest vAPIC log Suravee Suthikulpanit
2016-07-20 7:15 ` [lkp] " Fengguang Wu
2016-07-13 13:20 ` [PART2 PATCH v4 06/11] iommu/amd: Adding GALOG interrupt handler Suravee Suthikulpanit
2016-07-13 13:20 ` [PART2 PATCH v4 07/11] iommu/amd: Introduce amd_iommu_update_ga() Suravee Suthikulpanit
2016-07-13 14:14 ` Radim Krčmář
2016-07-14 9:13 ` Suravee Suthikulpanit [this message]
2016-07-14 9:33 ` Suravee Suthikulpanit
2016-07-14 13:45 ` Radim Krčmář
2016-07-14 13:40 ` Radim Krčmář
2016-07-13 13:20 ` [PART2 PATCH v4 08/11] iommu/amd: Implements irq_set_vcpu_affinity() hook to setup vapic mode for pass-through devices Suravee Suthikulpanit
2016-07-13 13:20 ` [PART2 PATCH v4 09/11] iommu/amd: Enable vAPIC interrupt remapping mode by default Suravee Suthikulpanit
2016-07-13 13:20 ` [PART2 PATCH v4 10/11] svm: Introduce AMD IOMMU avic_ga_log_notifier Suravee Suthikulpanit
2016-07-13 14:29 ` Radim Krčmář
2016-07-14 9:43 ` Suravee Suthikulpanit
2016-07-14 13:52 ` Radim Krčmář
2016-07-13 13:20 ` [PART2 PATCH v4 11/11] svm: Implements update_pi_irte hook to setup posted interrupt Suravee Suthikulpanit
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=578757A8.3000200@amd.com \
--to=suravee.suthikulpanit@amd.com \
--cc=alex.williamson@redhat.com \
--cc=joro@8bytes.org \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=pbonzini@redhat.com \
--cc=rkrcmar@redhat.com \
--cc=sherry.hurwitz@amd.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).