From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932956AbcDHMvu (ORCPT ); Fri, 8 Apr 2016 08:51:50 -0400 Received: from mail-bn1on0061.outbound.protection.outlook.com ([157.56.110.61]:21856 "EHLO na01-bn1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S932801AbcDHMuV (ORCPT ); Fri, 8 Apr 2016 08:50:21 -0400 Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=amd.com; From: Suravee Suthikulpanit To: , , , , , CC: , , , , Suravee Suthikulpanit Subject: [PART2 RFC v1 6/9] iommu/amd: Implements irq_set_vcpu_affinity hook to setup GA mode for pass-through devices Date: Fri, 8 Apr 2016 07:49:27 -0500 Message-ID: <1460119770-2896-7-git-send-email-Suravee.Suthikulpanit@amd.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1460119770-2896-1-git-send-email-Suravee.Suthikulpanit@amd.com> References: <1460119770-2896-1-git-send-email-Suravee.Suthikulpanit@amd.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [124.121.8.20] X-ClientProxiedBy: HKNPR06CA0022.apcprd06.prod.outlook.com (10.141.16.12) To SN1PR12MB0446.namprd12.prod.outlook.com (10.162.105.14) X-MS-Office365-Filtering-Correlation-Id: fb7ba553-e863-43c8-ba92-08d35fac56fc X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0446;2:YQK7DSd4FIKSNI23q1slaa6or1PTpPEM7TFC6c6jAu4DL909kJJuBqGXZO9S0rOgX4o3a8HJ4d0kNCoqXhAtUWi+WYdfqJ9YfljsqHZHqOf6pJFz5fXeSTD5v/Wf7LjDV0YTxkOru9Sh/t7ySvTa7RFhcHsQhknSPXU54gFQLr28vyZfpOWBfP+TgHJYRnIm;3:pfG7h6HvZVq3TiQ4LjWDHmgpFYIhZMc8hzkbS+GL3papQrApKjsFLGPOSz9/A5ThmDFza0YfTVqt1E2dwhQv2kTUIqkzsxCtNLMOQDcBq5biNLL/JGSmKNrfoW7SUAnB;25:hEZTkefsr/AW4atCZ9ipxQPW052tsgjThH/ze5S19xYz2FQoLBmbsW56CVoDkfSXM9yB/4ZILI5j/wU0BK8eqGJovqdlhexeqgopFps0l3Ep/lLSoHa3Xetdnvi15p6NzU5xVynDlX/wFNtZjfEwklpwpGU4brgoHcy/stinyfPT1LyVOkVijapdhf6Jp+OWTD/HIPzKZSTiQ2R7u9dvrzQlIm9pJiUNjkNGuT8v0sE8Dgu3hA6RFPBLOA6RISGLmAtZ68iDzWM97BzN0YiYFEDNHtmelIqmG1xt4SfzqPvDz3TVNDpfjrafp5nCRQ+1dtQYdcph04hNHZ2+oLzUlg== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:SN1PR12MB0446; X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0446;20:TOmNlnPyjrG+pRD8AFzHODVUFLw5A3ftJd+bFZ9OiKFxl4tPSUfDAYMqYNiZTaO8JUdOBdeILWKkTRMgey/wIkUVkWdJyhwPAA8xGKnAucmTPnejXAQWV2EoI4m3cQc5+tM3wAfsH8quxCqr2X6ZBryyPAyFR3rffuak87M7puHUJU6zWUVwDuo+cwEQ3etwvKVc0SzkACX7rPExlwzNXtXWiELjRgrFPuV71f+TBiyArMCxDPCITYp2sAaaJ9gW3g2pthrKaTNSa0YNQ86F8WOWWSSeV9wycppwsijsSOG8Ad7UPXjF3yPjXP8iKHYJ9AW8Fu/tyitqXvqEYifzGkaBPBTPfOvtwc3oj07E6PZU1XB2GUPKeNXf6M/oAhDdFOlZjLTwZnwc6dAYfDlh60+IQtEDlq/xCTweIeyFFrGW5QzO6Z9X5Nb0KiBgOXB4Qn4mNELAK4AxXSu07R6alzZ2HMmc+/g4y2h+e+InFLxLageGQV2srA5onuQzWDF/;4:iwY+NX9pJCVazWEncwlOyEnUD0gkTM717sGrv/pGjLnmDuOElYMOljTWbkzGHEjB/53UB3TqxtVhcvMvhKXkr7tF8nI6SxLjiFndXMH62sTH2ZobZFUUdKOxTxaUlcCGDI1Gp53QZyPJ7hBUB2qaMGhx6LOr6mMLZ516V1ICxyeTEdR4QeMcWXq2g9GD1w7U+WEInTsAibE/KSjKw1lMA644ftb1maPnmmXO5TwDbP0+RofOhndHzAQrcm3nL3WALmhaw6Z2bYA0k4d9Uh+S/NRy+TQsZnVcwkkEqLhtxG3HWJB5zNnLnoB8nt2kDQVXZXeksv1kqtxTI4ZALbA7FS8QxvzKpUIp9A9gQ/pR8b9ZTTUQkCIj1Xi0SGscKCfB X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046);SRVR:SN1PR12MB0446;BCL:0;PCL:0;RULEID:;SRVR:SN1PR12MB0446; X-Forefront-PRVS: 0906E83A25 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6069001)(6009001)(50466002)(48376002)(77096005)(47776003)(66066001)(229853001)(36756003)(86362001)(1096002)(42186005)(19580405001)(19580395003)(92566002)(6116002)(3846002)(586003)(5004730100002)(5890100001)(50986999)(76176999)(4326007)(189998001)(5003940100001)(2950100001)(2201001)(2906002)(5008740100001)(5001770100001)(81166005)(50226001);DIR:OUT;SFP:1101;SCL:1;SRVR:SN1PR12MB0446;H:localhost.localdomain;FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0446;23:0q/sTNFJ1zLW5LzJQofpAQbICf0KK8vQAFLzFECL4GNpNrSCIoiHTlwyH9XhnT0W5VBYGR8SYKD74chU1LiL0miDRQKCUEymbDAFcQqNPAvCDeLt4Rzt7IZ9BCBhURduFYSpCVgPM4U76hSQokY07ebSHtwCnlFVG0brb+f/hNiojsrqXg2XCJcdZWyhCy9OBLh8gCjWhkFWgL6GXk/8VkZQEnVqvBWhaGwSEahlG3Wih8POWrhfxTXcTe4con6n1eOwrgN3E6Vdsk0X/3y3aMY8dtoMy90C2N7UsLc85pRbJjkV9/91Pw10Akt67L1ZUKWfq+0cGCMaiJlg85fezVQdUTqY1jXobusbTNfpc4aRRkihEx70Q2tTCKCM5kRFvWo8RWEiX/fbHIB4lu9g8EIffLIsvfDAJf5CZjCF+yFScgMBGsLHe8m2lJ2eVbiHXWS0GFqoODeO2oDbYu1nx8imBG5e+fUzk78/qISuZhtqPo+tvtbV3fXxVsFEt2j1xn/A8BM2W1HteBZRWJze7fLfoaQxwKIXpv6HUpii39j/+uo5Zqq/uT/Qs/yCwZAv0Wskv+QlyvmUugTHb3ln6qTktPeI3N8JpAzR2eZr23TMqfJ28dcKuo7HpuFJp4+Wr8CoV6mw0GGhtWHCGWKQ14ELaQFW+R7beVM2jQVhgat7wrMbUtYcTwQ/QpeHnmYJl+JWL0Bdsd/rgHrjMIzrwxRorNXEBKKrCobJGKwds31h8xyPPFp6cg62ocEUCnKnlUYHps54kt0KHHnuvQg81mm38Bpvczlgr27JkkRadzCqjxllt8d/G5UpZCcUSaHDB0PdZjHSlmd1T+SE2hk203wTwmDrKO8zaqiTIa2mJwF29DVpuDwTqDJJ27oYpTsdV4N5Pm1dsPeicQ5xUyc7aeNE/8N7zUaGjKMwfeSp+5o= X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0446;5:+S/fZZBb+7Yn1JuExvTVSnNAeSeURezXXAdIBUjQytcWT6l7S9ZJpGhZJbYuRnMypkn0wewEsSQ9n96KrCqahuR/IhWz7QqTtkZmXGdmTgs/S40orFT29daxXy4v3BDrfh5uIHXwVyYPQ9OKRJ+oVw==;24:8bS/vPjjsdpRTeIZubj3EAddrtnG/PRCPfGVpOzELtgQqXhCLUygx4VzrHoFZ79fEuXU6ZfJxSGw3DO5H5AkmRFFKVR+DlPkssWkMBlrzfc=;20:6MxFhKnbqnaD9jZ44QvACzoJLTWSDb5D2BPrAhJAW6dfbO/WWwS8Fw9DEYb/QnkcjRZLSzyNuwfJ1AT38YHmQAOoAXWHnlkvKhY5yFWdQV+JIsJ9LvZcwomwWxufArPKxqLE9ZMiSdk/GxUfoxUieRK04v0VJe+5HOjv/moS3u9dnrn9IbEGmzjZ2DrSTlS8HrbsCq3EfiFUSZGpDuiQgdgNNuQHztbb4h7rDnnqGEBekC6YaGidv+U/AWFRLtUH X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2016 12:50:15.3412 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB0446 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Suravee Suthikulpanit This patch implements irq_set_vcpu_affinity function hook to set up interrupt remapping table entry with GA mode for pass-through devices. In case requirements for GA mode are not met, it falls back to set up the IRTE in legacy mode. Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd_iommu.c | 98 ++++++++++++++++++++++++++++++++++------- drivers/iommu/amd_iommu_init.c | 4 ++ drivers/iommu/amd_iommu_types.h | 27 ++++++++++++ include/linux/amd-iommu.h | 6 +++ 4 files changed, 119 insertions(+), 16 deletions(-) diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c index 1d17597..112a937 100644 --- a/drivers/iommu/amd_iommu.c +++ b/drivers/iommu/amd_iommu.c @@ -101,6 +101,7 @@ struct iommu_dev_data { bool pri_tlp; /* PASID TLB required for PPR completions */ u32 errata; /* Bitmap for errata to apply */ + u32 guest_mode; }; /* @@ -3145,6 +3146,10 @@ static void amd_iommu_detach_device(struct iommu_domain *dom, if (!iommu) return; + if ((amd_iommu_guest_ir >= AMD_IOMMU_GUEST_IR_GA) && + (dom->type == IOMMU_DOMAIN_UNMANAGED)) + dev_data->guest_mode = 0; + iommu_completion_wait(iommu); } @@ -3170,6 +3175,13 @@ static int amd_iommu_attach_device(struct iommu_domain *dom, ret = attach_device(dev, domain); + if (amd_iommu_guest_ir >= AMD_IOMMU_GUEST_IR_GA) { + if (dom->type == IOMMU_DOMAIN_UNMANAGED) + dev_data->guest_mode = 1; + else + dev_data->guest_mode = 0; + } + iommu_completion_wait(iommu); return ret; @@ -3698,20 +3710,6 @@ EXPORT_SYMBOL(amd_iommu_device_info); * *****************************************************************************/ -struct irq_2_irte { - u16 devid; /* Device ID for IRTE table */ - u16 index; /* Index into IRTE table*/ -}; - -struct amd_ir_data { - struct irq_2_irte irq_2_irte; - union irte irte_entry; - struct irte_ga irte_ga_entry; - union { - struct msi_msg msi_entry; - }; -}; - static struct irq_chip amd_ir_chip; #define DTE_IRQ_PHYS_ADDR_MASK (((1ULL << 45)-1) << 6) @@ -4067,6 +4065,7 @@ static void irq_remapping_prepare_irte(struct amd_ir_data *data, struct irq_2_irte *irte_info = &data->irq_2_irte; struct msi_msg *msg = &data->msi_entry; struct IO_APIC_route_entry *entry; + struct iommu_dev_data *dev_data = search_dev_data(devid); data->irq_2_irte.devid = devid; data->irq_2_irte.index = index + sub_handle; @@ -4086,7 +4085,8 @@ static void irq_remapping_prepare_irte(struct amd_ir_data *data, irte->lo.val = 0; irte->hi.val = 0; - irte->lo.fields_remap.guest_mode = 0; + irte->lo.fields_remap.guest_mode = dev_data ? + dev_data->guest_mode : 0; irte->lo.fields_remap.int_type = apic->irq_delivery_mode; irte->lo.fields_remap.dm = apic->irq_dest_mode; irte->hi.fields.vector = irq_cfg->vector; @@ -4259,6 +4259,70 @@ static struct irq_domain_ops amd_ir_domain_ops = { .deactivate = irq_remapping_deactivate, }; +static int amd_ir_set_vcpu_affinity(struct irq_data *data, void *vcpu_info) +{ + unsigned long flags; + struct amd_iommu *iommu; + struct amd_iommu_pi_data *pi_data = vcpu_info; + struct vcpu_data *vcpu_pi_info = pi_data->vcpu_data; + struct amd_ir_data *ir_data = data->chip_data; + struct irte_ga *irte = &ir_data->irte_ga_entry; + struct irq_2_irte *irte_info = &ir_data->irq_2_irte; + struct iommu_dev_data *dev_data = search_dev_data(irte_info->devid); + + /* Note: + * This device has never been set up for guest mode. + * we should not modify the IRTE + */ + if (!dev_data || !dev_data->guest_mode) + return 0; + + /* Note: + * SVM tries to set up for GA mode, but we are in + * legacy mode. So, we force legacy mode instead. + */ + if (amd_iommu_guest_ir < AMD_IOMMU_GUEST_IR_GA) { + pr_debug("AMD-Vi: %s: Fall back to using intr legacy remap\n", + __func__); + vcpu_pi_info = NULL; + } + + iommu = amd_iommu_rlookup_table[irte_info->devid]; + if (iommu == NULL) + return -EINVAL; + + spin_lock_irqsave(&iommu->ga_hash_lock, flags); + + if (vcpu_pi_info) { + /* Setting */ + irte->hi.fields.vector = vcpu_pi_info->vector; + irte->lo.fields_vapic.guest_mode = 1; + irte->lo.fields_vapic.ga_tag = + AMD_IOMMU_GATAG(pi_data->avic_tag, pi_data->vcpu_id); + + if (!hash_hashed(&ir_data->hnode)) + hash_add(iommu->ga_hash, &ir_data->hnode, + (u16)(irte->lo.fields_vapic.ga_tag)); + } else { + /* Un-Setting */ + struct irq_cfg *cfg = irqd_cfg(data); + + irte->hi.val = 0; + irte->lo.val = 0; + irte->hi.fields.vector = cfg->vector; + irte->lo.fields_remap.guest_mode = 0; + irte->lo.fields_remap.destination = cfg->dest_apicid; + irte->lo.fields_remap.int_type = apic->irq_delivery_mode; + irte->lo.fields_remap.dm = apic->irq_dest_mode; + + hash_del(&ir_data->hnode); + } + + spin_unlock_irqrestore(&iommu->ga_hash_lock, flags); + + return modify_irte_ga(irte_info->devid, irte_info->index, irte); +} + static int amd_ir_set_affinity(struct irq_data *data, const struct cpumask *mask, bool force) { @@ -4266,6 +4330,7 @@ static int amd_ir_set_affinity(struct irq_data *data, struct irq_2_irte *irte_info = &ir_data->irq_2_irte; struct irq_cfg *cfg = irqd_cfg(data); struct irq_data *parent = data->parent_data; + struct iommu_dev_data *dev_data = search_dev_data(irte_info->devid); int ret; ret = parent->chip->irq_set_affinity(parent, mask, force); @@ -4281,7 +4346,7 @@ static int amd_ir_set_affinity(struct irq_data *data, ir_data->irte_entry.fields.destination = cfg->dest_apicid; modify_irte(irte_info->devid, irte_info->index, ir_data->irte_entry); - } else { + } else if (!dev_data || !dev_data->guest_mode) { struct irte_ga *entry = &ir_data->irte_ga_entry; entry->hi.fields.vector = cfg->vector; @@ -4311,6 +4376,7 @@ static struct irq_chip amd_ir_chip = { .name = "AMD-IR-IRQ-CHIP", .irq_ack = ir_ack_apic_edge, .irq_set_affinity = amd_ir_set_affinity, + .irq_set_vcpu_affinity = amd_ir_set_vcpu_affinity, .irq_compose_msi_msg = ir_compose_msi_msg, }; diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c index 5b783af..4d933a7 100644 --- a/drivers/iommu/amd_iommu_init.c +++ b/drivers/iommu/amd_iommu_init.c @@ -1370,6 +1370,10 @@ static int iommu_init_pci(struct amd_iommu *iommu) !iommu_feature(iommu, FEATURE_GAM_VAPIC)) amd_iommu_guest_ir = AMD_IOMMU_GUEST_IR_LEGACY_GA; + if (amd_iommu_guest_ir >= AMD_IOMMU_GUEST_IR_GA) { + hash_init(iommu->ga_hash); + spin_lock_init(&iommu->ga_hash_lock); + } ret = iommu_init_ga_log(iommu); if (ret) diff --git a/drivers/iommu/amd_iommu_types.h b/drivers/iommu/amd_iommu_types.h index d528a46..1b640b2 100644 --- a/drivers/iommu/amd_iommu_types.h +++ b/drivers/iommu/amd_iommu_types.h @@ -22,10 +22,12 @@ #include #include +#include #include #include #include #include +#include /* * Maximum number of IOMMUs supported @@ -119,6 +121,14 @@ #define MMIO_STATUS_GALOG_OVERFLOW_MASK (1 << 9) #define MMIO_STATUS_GALOG_INT_MASK (1 << 10) +#define AMD_IOMMU_GA_HASH_BITS 16 +#define AMD_IOMMU_GA_HASH_MASK ((1U << AMD_IOMMU_GA_HASH_BITS) - 1) +#define AMD_IOMMU_GATAG(x, y) \ + ((((x & 0xFF) << 8) | (y & 0xFF)) & AMD_IOMMU_GA_HASH_MASK) + +#define GATAG_TO_AVICTAG(x) ((x >> 8) & 0xFF) +#define GATAG_TO_VCPUID(x) (x & 0xFF) + /* event logging constants */ #define EVENT_ENTRY_SIZE 0x10 #define EVENT_TYPE_SHIFT 28 @@ -556,6 +566,8 @@ struct amd_iommu { struct irq_domain *ir_domain; struct irq_domain *msi_domain; #endif + DECLARE_HASHTABLE(ga_hash, AMD_IOMMU_GA_HASH_BITS); + spinlock_t ga_hash_lock; }; struct devid_map { @@ -801,4 +813,19 @@ struct irte_ga { union irte_ga_hi hi; }; +struct irq_2_irte { + u16 devid; /* Device ID for IRTE table */ + u16 index; /* Index into IRTE table*/ +}; + +struct amd_ir_data { + struct hlist_node hnode; + struct irq_2_irte irq_2_irte; + union irte irte_entry; + struct irte_ga irte_ga_entry; + union { + struct msi_msg msi_entry; + }; +}; + #endif /* _ASM_X86_AMD_IOMMU_TYPES_H */ diff --git a/include/linux/amd-iommu.h b/include/linux/amd-iommu.h index e52cee5..f698900 100644 --- a/include/linux/amd-iommu.h +++ b/include/linux/amd-iommu.h @@ -22,6 +22,12 @@ #include +struct amd_iommu_pi_data { + u32 vcpu_id; + u32 avic_tag; + struct vcpu_data *vcpu_data; +}; + #ifdef CONFIG_AMD_IOMMU struct task_struct; -- 1.9.1