From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751479AbcBLQVk (ORCPT ); Fri, 12 Feb 2016 11:21:40 -0500 Received: from mail-bn1bon0065.outbound.protection.outlook.com ([157.56.111.65]:53753 "EHLO na01-bn1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750961AbcBLQVe (ORCPT ); Fri, 12 Feb 2016 11:21:34 -0500 Authentication-Results: amd.com; dkim=none (message not signed) header.d=none;amd.com; dmarc=none action=none header.from=amd.com; Subject: Re: [PART1 RFC 6/9] svm: Add interrupt injection via AVIC To: Paolo Bonzini , , , References: <1455285574-27892-1-git-send-email-suravee.suthikulpanit@amd.com> <1455285574-27892-7-git-send-email-suravee.suthikulpanit@amd.com> <56BE005D.1040905@redhat.com> CC: , , , From: Suravee Suthikulpanit Message-ID: <56BE0678.7060103@amd.com> Date: Fri, 12 Feb 2016 23:21:12 +0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 In-Reply-To: <56BE005D.1040905@redhat.com> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [124.121.8.20] X-ClientProxiedBy: HK2PR0201CA0005.apcprd02.prod.outlook.com (25.162.206.15) To BY1PR12MB0439.namprd12.prod.outlook.com (25.162.147.140) X-MS-Office365-Filtering-Correlation-Id: 5bae8e25-e941-4e5f-44ad-08d333c890c2 X-Microsoft-Exchange-Diagnostics: 1;BY1PR12MB0439;2:Tj9/fKJVqhM1fu0IMecHztAbFN78RpHbJ6dNm+oVAXb7P1sxqSkizLAr84WiPjLUZeIkDJzfVCzWQJobgQf52aj7Zm5PF6slWfv6BYnc2bTi7HQp8tRJX6XO1TtZojekuDs8wTaiw0gCSCNGK0ikGp9VQpurt1StueoMSjaQbpyp1RGCGImZeJCtU6bLefW2;3:nc+amLy+9tdBUSB69t+lFf3xkXbgURiw6EptqG3/mxWCZlfTu0DnE0kOh2SOs1WrbAu+VG8wPSO8r26WfI+Uy+6Y/2A1PK6M1X4lXv/Qt4ddaUZzyPy1R89lDp6BHRNa;25:kHDsgl1+c9WNVRMKhRpeEHc014/GlTtzFlpzQBj9ZJgKUjMg7QcNvckgvidERA/vVVSHh7VnLw/+RmTq4I61Uw5nTEH8Niumy45P8RY8Kb+XNxVz8QttTbBjntvYQ6UIJ6H/6fccpq1g9GUddaMIuEsPOZXKVt2PNSOwPdSHxN9VAJUyfe20kD6NKwDTw3dtCpVJgbmNU8jBhkku7EN5xh7vQmnm6LL+YvuGbQbl0UmArFrZJy1HW+UTp3ihP/lH/wQ8bizvp7MT1AqW1U9VVLJkMfCnTO3NfkfS0sGozaTqtrSvHgTiAXWOriY0Egs+ X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY1PR12MB0439; X-Microsoft-Exchange-Diagnostics: 1;BY1PR12MB0439;20:7jyr+bOKIPHqbWbL2X9AsUf6e43kU5Hok5Gtr4Yl9mdGbz0SVkc9MKKfDju64vfeTRn5px8V7LC6gMBr/a5KbKCyw5MfdtYaerwCvYcQZeRskVwlNkHoP7/9M34eTgQLstVG2NJUwg0CyaHxlXfUYFbyIPohgLOmX/mowPr52iefL8cbDqqEov1tNsaVrKj3PIjyhwxzou+U7XyyPOxLl52yEENCEHZ+MY9nzQQPmjtLB5b0XYYE5VbkX1bqwsS4akHKSjiBw8uy8j2gtBjxlG+AZHEE7e+ogv9pnDpH8vbzRetWJnFBhDj79B1J/k0cm6NqFc5yXvuNoUwJIQKHza8te25k98Uzt6h8dzP8lwP9DJxbpB9QG/woIwk1ZltqQL4WB/Ai/yHFl69hPbC1F1ueFsMAKZr1Qvdnh5IU2I73c0c10WTZYPKwqH39z8Qll2FRzM5aPW9eRKHlodbvbm5wmSIvS4s7C+Va2ty03y+voJku8/7Qi7z7uKeVhr0B;4:LkDBiYkpmVtRLVBvCj6oWpaxHjo5cBBxh/+pVJ//eeO7skUZCaR4jxjjghdKdTqjzcdZdPFrQHezK5R/EPXePmhs5MsT6HVk6B1VRFr4txjxgEIzSexrDSn/wHDMMhCenTiRfBX7BBDmFz4E+0jUVjsc/MgksWpnkF8eoTBPEn09EszxCY9VXWJfhgSCTFb5jCW7vRBMSYpPWbkqJ3A2BRQQ8IU5dYEhW27vXl1TFv3X54uPQN6JfdsSQENvN/2AtNITxdm0AzIWqYnqmA3y2Q3ikjuk22p2p9Yg2md2Pr1qnho2JkDhUV/C/X4MIUMsvnCxsNIPyz7J8/+U7c3DJNkWoLkXznd0B1NGHOvIMxE2be3uqmhP61gZCOZPrz3T X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001);SRVR:BY1PR12MB0439;BCL:0;PCL:0;RULEID:;SRVR:BY1PR12MB0439; X-Forefront-PRVS: 0850800A29 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6049001)(6009001)(377454003)(479174004)(24454002)(76104003)(164054003)(23746002)(65816999)(83506001)(54356999)(66066001)(50986999)(86362001)(230700001)(40100003)(122386002)(5004730100002)(65806001)(586003)(6116002)(1096002)(5001960100002)(4001350100001)(2906002)(92566002)(33656002)(3846002)(76176999)(77096005)(59896002)(4326007)(36756003)(117156001)(5008740100001)(50466002)(42186005)(189998001)(47776003)(5001770100001)(2950100001)(2201001)(87976001)(65956001);DIR:OUT;SFP:1101;SCL:1;SRVR:BY1PR12MB0439;H:[192.168.0.19];FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;BY1PR12MB0439;23:G8CQ6iUzlPj7AthhnUTdFrqBdvCEdFIGcV2ww?= =?Windows-1252?Q?KZLMi1aXMICx6fjZvfyjOgfTVgO+jlx64s0ZUGNYkbI99xdvs5ILilby?= =?Windows-1252?Q?pWgwgHoNXQEqP3Xw8Og2AHZmww/w8OLpZTNOpywo/VyH3GOknSpR0ZZ5?= =?Windows-1252?Q?NNkYN2TytWbfINtLNG+7Mh4ql9iki5V4rJg/E+XZAwbehFUI/etzBLBX?= =?Windows-1252?Q?+rvFrDiMewbcSvMA0sSLW3nm9xfkH7PwmQGK0H8hX3c51am6NmIBwRls?= =?Windows-1252?Q?MFV20X9aYfla4ZGuBRcjXhq2Czb22/AyS7xg0+OPSrI/WXbVxYU2/RD/?= =?Windows-1252?Q?RNpiWfGwNs+MqnwxFS+vYLfRHCEFcQ4zy/1ThKlqbKPg20tbQMO6eO5d?= =?Windows-1252?Q?R4wuoRnKPyZv7fP03A2eczc8EgBeCgYK/CbnC8/akFyHeFgAGQMwriLK?= =?Windows-1252?Q?WREDgD/7Ot8RN+du04MU71sceCrjqUJ0SFlZmK2joINdGfV0c3fMcmoQ?= =?Windows-1252?Q?AJhud5EDAFzrwTE/bmTft1lEwy4EXBpG/CHFI+TH/5VwcLBnVZuT5NaH?= =?Windows-1252?Q?lSz4uXSijFFPKc0R7xLkhvWGtkTndprgys0d5qm8+bDQhrwSFyRLw4cW?= =?Windows-1252?Q?RtpK+tKMg7AxVsZacSvqJPF7BzuW6wm7FcP651I9m0q77xEpfO4G15qi?= =?Windows-1252?Q?Q1u+8GALWdII07UVrV4PY7MQ8xKETkW1bWOiuxAs7SpVv7i2wRJCIqCM?= =?Windows-1252?Q?0Zshzhig0uPZiCEsUj7ntvMjekkvIWo1cFgfImJ/MLEdMnvLCsik+Fp2?= =?Windows-1252?Q?uYDvg3XBhf6wkbOhdVYJzvH4mWSEtdUIYEaM3QjtStBd1cs0Gq9v9YIx?= =?Windows-1252?Q?hnYTBKNO4NZHnKIQLUHwvO19zNtryqCkTM3xffc2rOC7kjwRmtfOovVL?= =?Windows-1252?Q?EsDHdX07c2D4lb8Y3UNZpQOEFo6+U2lGrzQKtwaK2TusMMXGuxblUK4s?= =?Windows-1252?Q?TgqjiSfhUVi+dzzE1tijlmh4XYQeHYujJpAVtHRvDJj52h9evvHJhnLB?= =?Windows-1252?Q?cxWRQLS8M0H2woAKn48SrwdsMZnypedJfEEVNLrKGOi3RoBxtYhLaEdr?= =?Windows-1252?Q?VlO/kblu5W+ctaZHOUGRgCJyB/CjZIdTc1WeW0+gHTO2OtyCzXiNu2Ka?= =?Windows-1252?Q?BNVVfC01+qp82OCa1noo+MDlpsYFqMfCea7TNexsLcMKxaQXrdXcYkiI?= =?Windows-1252?Q?/oS9nWinzqPkvMwXA=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;BY1PR12MB0439;5:U70/M2E7T5HhDFUGwGxCnZpaNtCMPBevQxhmHuGdnKg2rkYg0QCau5l0vE/xrlTCdDfMPwNxPU27xd5A5hISLXeKVHpHdudVq4GiiiNY62QKDxCbpG+LXwGvW1g7m/VBMfcpQi/+k1jEuCyvri114A==;24:1ln2UJUPvz96fPDhYgEzRZy8+eagET/VVMSd6sFvmlvlykpon3uqJiSfCAl33nvT5pcBpQLHEFlCb6ZuOLK8kAiYxG25ELx+8v/OT9Nb8uU=;20:jRl0BarX9im+t9XEKPkfDvwpA1UO5WP1RstDOAIcWpY8ui1r/3bF0/k19jUGjiOvgCux/ez/I+GYQG1MwzFMkip8s15AWEtNqelfG8/cOLslMW6tyNSKcXxoGAN48KERlgric2vaQGBUC4BhaePJW0qtvb1SQ8u9YEmjvGvTojdJAuK5Jsyx9be7vl4oXFhgX1gmRaycVwDdJQhXg6amXM3W1lIOJlmwnYkBPsEDp9rIO2CIm1WT8736Z4Vdb1GE X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Feb 2016 16:21:28.2154 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR12MB0439 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Paolo, On 02/12/2016 10:55 PM, Paolo Bonzini wrote: >> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c >> >index 4244c2b..2def290 100644 >> >--- a/arch/x86/kvm/x86.c >> >+++ b/arch/x86/kvm/x86.c >> >@@ -8087,7 +8087,9 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu) >> > if (is_guest_mode(vcpu) && kvm_x86_ops->check_nested_events) >> > kvm_x86_ops->check_nested_events(vcpu, false); >> > >> >- return kvm_vcpu_running(vcpu) || kvm_vcpu_has_events(vcpu); >> >+ return (kvm_vcpu_running(vcpu) || kvm_vcpu_has_events(vcpu) || >> >+ (kvm_x86_ops->apicv_intr_pending && >> >+ kvm_x86_ops->apicv_intr_pending(vcpu))); >> > } > I think this is not necessary. What you need is to make kvm_lapic's > regs field point to the backing page. Then when the processor writes to > IRR, kvm_apic_has_interrupt (called through kvm_vcpu_has_events) will > see it. > > avic_pending_cnt shouldn't be necessary either. > > Paolo So, the other thing I am using the avic_pending_cnt for is for the part 2 of the series (to enable AVIC support in IOMMU) that I am planning to send out later. However, it might be good to discuss this at this point. When the IOMMU cannot inject interrupts into the guest vcpu due to it is not running (therefore, it cannot doorbell the vcpu directly), it logs the interrupt in the GA log buffer. Then it generates interrupt to notify the IOMMU driver that it needs to handle the log entry. Here, the IOMMU driver will end up notifying the SVM to scheduling the VCPU in to process interrupt. Here, I have run into issue where the vcpu often goes into idle (i.e. scheduled out), and ended up causing IOMMU to generate a lot of the entries in the GA log. This really hurts device pass-through performance (e.g. for XGBE NIC). So, what I ended up experimenting with is to set the avic_pending_cnt to a larger value (i.e. avic_ga_log_threshold) whenever we processing the GA log entry. The intention is to delay the vcpu schedule out in expecting that there might be more interrupts coming in soon. I also make this threshold value tunable as a module_param. This actually works well in my experiment, where I can actually get about 5% speed up in my netperf test on XGBE NIC pass-through test. However, I am not sure if this is an acceptable approach. Actually, I think it's similar to the halt_poll_ns, but specifically for IOMMU GA log in this case. Let me know what you think. Thanks, Suravee