From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752887AbcHLRBq (ORCPT ); Fri, 12 Aug 2016 13:01:46 -0400 Received: from mail-by2nam03on0098.outbound.protection.outlook.com ([104.47.42.98]:55232 "EHLO NAM03-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752693AbcHLRBo (ORCPT ); Fri, 12 Aug 2016 13:01:44 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=waiman.long@hpe.com; Message-ID: <57AE00EE.8070904@hpe.com> Date: Fri, 12 Aug 2016 13:01:34 -0400 From: Waiman Long User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130109 Thunderbird/10.0.12 MIME-Version: 1.0 To: Dave Hansen CC: Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , , , Jiang Liu , Borislav Petkov , Andy Lutomirski , Prarit Bhargava , Scott J Norton , Douglas Hatch , Randy Wright , John Stultz Subject: Re: [RESEND PATCH v4] x86/hpet: Reduce HPET counter read contention References: <1470853770-37625-1-git-send-email-Waiman.Long@hpe.com> <57ACD2DE.6080306@intel.com> <57AD0898.7030506@hpe.com> <57AD18D1.1050107@intel.com> In-Reply-To: <57AD18D1.1050107@intel.com> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [72.71.243.96] X-ClientProxiedBy: CY1PR04CA0036.namprd04.prod.outlook.com (10.166.187.46) To AT5PR84MB0305.NAMPRD84.PROD.OUTLOOK.COM (10.162.138.27) X-MS-Office365-Filtering-Correlation-Id: 16366ca6-fa93-46d4-959b-08d3c2d254e2 X-Microsoft-Exchange-Diagnostics: 1;AT5PR84MB0305;2:eQSIZzC7MAqqsjZ6JA+B2SBP09sdW2GoeLx8hnw0T5vZE0eNbXcuARMi8+OjfwnZe/dsWHxQQf4tM4ZdudHYa3vNeCXEA4AfllXAtohNU7xKJ/i+2ngm/GFgLLzG8pNVNtci8hmYp+6S2ICSuvgaB/DTf/7Lo5R+b5JSIPYx90PCsDMRk0B9Y/A00rehhGpj;3:WSguogT/fpQJcMK/JdKH3XSFwIHES37DMjdbcVQ9++Dmd6SGRfzXa/LB2Dyl1VeN1OF2z2slAyGWCmUzh2hQOgZptrlWz3UNRrx8qoOq50S55dT4gq0/Q/FC5FRW/NRz;25:1FamKSSW11EgQTd2ME/UJ2QXesnuO63rl1lwmJqK6Kfh4Bfue/IubFZ/EdTwxHTeUzdxI8Ve2Q1XUmcTaEojDvCoW4UQMmAVZn8Hj5vmwOe37S4GnvXlE+4y6Q0U0LcvxrdcsNdqPs6cN/AMMWdqBUyL/iLUVxq2ynIpwrQFtLT9iATv3yuWMtCINlnjcDpWhnmZ91NeLwT/fSGqSaFqNtM4/UXcZD6IhdKuRGxh3GcKjtFfgoW/INzrnJL3vqXbg7D97g2tcMKx4k435deSjY6BuNAhufKb4exEzUI8MyZL7cCvS6vaZSWnIuwBncMFH9X2kLOeGBtipUVM70Z7DwyhksP2O9qrDKu+nocUnVBiSy+tAX1rDDTVQ4ZqCqRe0uxloOLxOmyEEbEWnKtJiG+/j//iLscgRBSR/kHyQRk= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AT5PR84MB0305; X-Microsoft-Exchange-Diagnostics: 1;AT5PR84MB0305;31:tsNOSRqZeNi2kUvfTJQPbG+3UEQOF2PXGcbdzqxvjxQ3g+Ah4LFJ+kfzMozBab/dEqkPVxMRiezj1NdPEIOW5uqe3GfheYSZDlNQjLCLc1jdo/3RqKXjJYt/MmYze8d7J1iLKMJF4y4Uk0Nd3FBtMBgMGNceI69iUHSukJ9mVNAz8YSIR8alReXotKu7a6GEQnt58pZUPzNHVY9fuCe/nxnqJmhVxfc8LH7V43Q6a3c=;20:MxXJ9JWhkEcDfax/dzfhoyqoFplYnHao3FKmx/+Kyu5AO9CUiKjg6qqsTbwrJKo8VS6uO7yERe9ZffgFF3ZTbOPQ0arC87PN3YA1qsbZHKyb+CyWVpe/gc1WKBWCOjm+wItpKStF9L2RgnRspMKuEA/KgZIc5kuWs8r8IZ6Her8Up2LdOv5L8s8YWbs8N5zyrE1vkPPgCF/yTIE5O8zaLNkzyjWjBFo7n3h+fOfEuDMv4kAJjEoRxIUX4SOKRIZ2L+Z13GU5IvbYGJ9ZewxVqabAP8FqW8xBCURBcCeClwara23cMULCPYhGh/zncfRKijAbELr/kiFkOvTYtIm9yJHopr4ve1srO1cHRrIMb321WTvK7SGq3DhljJg/T+vkWFtEpgxVFqYebooJwpgPwhaXAfcqPmNjczSfo7KNrsxMizqZhHqXGrTX3o6g4FQ6lsrapZQ72P8FivwaWIw3w6lEqaSEwHZFdV8mFy2+y348lKRgzg3Tnf4TD2FWvFkA X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6055026);SRVR:AT5PR84MB0305;BCL:0;PCL:0;RULEID:;SRVR:AT5PR84MB0305; X-Microsoft-Exchange-Diagnostics: 1;AT5PR84MB0305;4:2rNw7qDNVmnpefZnpi1/uesSANU5eLzZO7ir6M1MaOjU4BKF2waMLLTShl4KxfPnW7mDnBzUMshMkE1gWyVrV33w1v414sU3+2604ANdwhfoRolnxJBrGVsb7Q6mNmIpncoKzt4Jl+JgKQ94goCVBXSqZAI+w4gsXnnk03VO9rvLSUrS/ktlh9/PkrFjyr4mUn3mFQgfDKdE2wc1jhv4zQ0N6lXUBn+bsJxUD4gnXb/66U6IjE3EVkuN6SbAWt9v5sz9LY1q0RPftnAw0cHHTC1gQwRe747L3+sJBh04FV9H4sMp9K7tPOTen36eZX8t5L/rZ/NwquCdw/Q9cDA6mcxFoLenw2BkQMM5j9fdPf+g5aP8/xiXBgVKX0XppRjv18PezzQmc9ObrttSbBfz3w== X-Forefront-PRVS: 003245E729 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6049001)(6009001)(7916002)(199003)(189002)(24454002)(377454003)(83506001)(80316001)(64126003)(33656002)(230700001)(65816999)(50466002)(54356999)(87266999)(76176999)(50986999)(106356001)(86362001)(101416001)(105586002)(92566002)(77096005)(4001350100001)(2906002)(93886004)(117156001)(81166006)(7846002)(8676002)(81156014)(7416002)(305945005)(4326007)(97736004)(110136002)(189998001)(23746002)(68736007)(586003)(2950100001)(7736002)(47776003)(65956001)(6116002)(3846002)(42186005)(59896002)(65806001)(66066001)(36756003)(217873001);DIR:OUT;SFP:1102;SCL:1;SRVR:AT5PR84MB0305;H:[192.168.142.185];FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;AT5PR84MB0305;23:XdTUDFOnUG+3BreKYTjrV6X1DC8LansyvXVRo?= =?Windows-1252?Q?TKcqj2INjAHGNWpdkHS9fpAQDM001A91NiIxF3V68olttPZ09x5UMAfo?= =?Windows-1252?Q?4+vwHbPwSx/Xg1okhjdgK6NPd5m92/ZyRoE3QnRUCbkEr0baEP+UuJvQ?= =?Windows-1252?Q?9jVUSt9WWOAtY2aokwwDN9wp9jP2ROrrjrS9mKjhkHzTvvPATg8fMfwe?= =?Windows-1252?Q?GDGAL3v16UthC3JiCC/iBMDZ4Dy4IPpn/ryMZ25ux8x3740QV+PPnq1J?= =?Windows-1252?Q?6UneUxlWsxzMbF+oFettAx5cfYZRR5tI8C3xxw41Z/lRdFDWH4mACLGB?= =?Windows-1252?Q?51MMoP7k8HZ2Pz2bSQVIk1rYnSmE48a8yHL0E7G/9nmPS8aMAFncVvB5?= =?Windows-1252?Q?L/CVrEWemLhzqMA/wsevov4n4vyp4q7IdTqYsSVtcyCdG+it3KZ3eyGb?= =?Windows-1252?Q?lss8W+p8ttYEBpF/U6u72SGjnrGGgiprma1fdfYCm0njH2Oa9HVUixzq?= =?Windows-1252?Q?Ge/Zw7JWgFT4gtE6vbpoB1EXQFrZK0iy3TB1p/2s5tn2PUjQWiTg9/nh?= =?Windows-1252?Q?ctcxWW6c8KIN/5sE4ehcSceZDi/DGiqMAc5qhQy6LnafTMxtkXC2NeeN?= =?Windows-1252?Q?b9a8wIBLSboZ34leww/ohr6SLCG2aMCUkEh7/QEa5mFb+zXRO5P7yJSR?= =?Windows-1252?Q?4COXo35tGVsQzR8WcxJhOEiRmVIV7AGbMhr/AV5QPTcV37duWg4rDLxk?= =?Windows-1252?Q?ltpdvqBmw+eX0PKYxOPUulF1CzQXhIlPXXH7+9uhV+lKhPiVDDh/5YCI?= =?Windows-1252?Q?gRs9qAAotz7cW+EpRgTMFA/+4VFu++zQzXik2xXvcuhfTNe/cD46RPYJ?= =?Windows-1252?Q?eQ69YGLwvDKeNKbv/TIzvUFrtJ+Fn32Cl7KotDyZbiCqhekRRDh2r9fr?= =?Windows-1252?Q?HHxstHfltjNHFrScVIyFNaFH33i34wIA939yXeCpAqywXjzjIrC1OuIX?= =?Windows-1252?Q?lXQ0e9edb1rbkBOrwCBnvRa5Jnk1qT+w7R271PpADOwc6IhXI+jVCqS5?= =?Windows-1252?Q?ee151x5vWu19zFKucE0YGU9CGzJXMQV2yWWQNOmU6WnSJGcFW+RA5hke?= =?Windows-1252?Q?ZReQRrqB9zdzq1JLO9HyKJWhzbwuQ2M4sm9hOO1KWEZLNQppsvi/3p+b?= =?Windows-1252?Q?trxIvIf87trKQO1DK6L4v+gqqpRIaPKLHZ3jg9C1JnKPti7DU4P1GXVb?= =?Windows-1252?Q?S45B3ry8UADZE2TYdRb81y+5n97mtawmCA0eVB4fWygkKtDh4I9pXzys?= =?Windows-1252?Q?zAT3unOtggm+MN5g4PSXosbVPcxcnao8r/hCOP8TuSUQeE5aB39MB40L?= =?Windows-1252?Q?1PIMJ3GFlV1vaF/qWWpg8z919WiCVmVqvP6HqEM9ljb5PXg0uxDDA4?= =?Windows-1252?Q?=3D?= X-Microsoft-Exchange-Diagnostics: 1;AT5PR84MB0305;6:SBSgtJbX7ErKgwG9Z/QL7C4axqJtIdvBGNrNiCz+Bbs7jrjM4lslBLT1gOro/IlarVUrukcj2duaP38+n1JgPztasKxQbDRlyu5EB9RTvakxGjnShJUvzfm6CXbpQcRBOCzUCC1Wdu6viV7QP4Z1p+4hScvmslnsp3S2H9ZyB/NivZZtT1VAbH+lAvGsfZgiRopaTmd+iJ+Mp2bsfZWQYZM+NZXhjO0VOqNy6t6jWTnZqnttYQjGEr2mvgln8dizz0jR9uH5fP/Rggpa5+WDWlIIMzovk3AtXis/vF2xzp8EfcglbQRLzwAKnCJ7X2E3aMEU66CFOW4drrWO8lDKmA==;5:zwvojH4hnPSQr5vFz2PDct1nqz6cn8nh1a+8vmAZwvysJP4CoacI6Iml3/LIvMKD8YOgjvgYIhqyP0ixX5F/N8kwjWj8datKrP7MXRj/GGNr9aFUTbbIBbtbK0G5VNCaIfKFwo38G6QzVDD0c7oXuQ==;24:DYpSmh36xTs0AWdpzQAdfOaY2wZli1VitdBmAX5ItD5r0MpjHN3yyz0VzXBjvJ/CCHatVyqjlzZ+Zxch12A83KCx+1+qtt8KFqNn4ewI0lQ=;7:tqUxyuznLWcgoRWIB26luaSAeRutfsR5qKNGN4tYWgkhblTFCzD0qWKl0r+UAS1ir1MTXHcw1FR0ZD+a7xK+4NZlnVEBmua/cxGMc5Uv7WsiN2fTptcF/EQGIJeczA9wR46FVtGHGdvSO7TCk8ysVja17Rx9/XOpoZMQw+8W8y/yBQ78uWKyJUPWAMIkypDqlcdHpN2i5zAGwFhlaKRQ9+sye6MBEla1r+aGrnxqmxMcyYGo4kTcPnQ8ZRrCLI1Y SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: hpe.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2016 17:01:39.2489 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: AT5PR84MB0305 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 08/11/2016 08:31 PM, Dave Hansen wrote: > On 08/11/2016 04:22 PM, Waiman Long wrote: >> On 08/11/2016 03:32 PM, Dave Hansen wrote: >>> It's a real bummer that this all has to be open-coded. I have to wonder >>> if there were any alternatives that you tried that were simpler. >> What do you mean by "open-coded"? Do you mean the function can be inlined? > I just mean that it's implementing its own locking instead of being able > to use spinlocks or seqlocks, or some other existing primitive. The reason for using a special lock is that I want both sequence number update and locking to be done together atomically. They can be made separate as is in the seqlock. However, that will make the code more complex to make sure that all the threads see a consistent set of lock state and sequence number. >>> Is READ_ONCE()/smp_store_release() really strong enough here? It >>> guarantees ordering, but you need ordering *and* a guarantee that your >>> write is visible to the reader. Don't you need actual barriers for >>> that? Otherwise, you might be seeing a stale HPET value, and the spin >>> loop that you did waiting for it to be up-to-date was worthless. The >>> seqlock code, uses barriers, btw. >> The cmpxchg() and smp_store_release() act as the lock/unlock sequence >> with the proper barriers. Another important point is that the hpet value >> is visible to the other readers before the sequence number. This is >> what the smp_store_release() is providing. cmpxchg is an actual barrier, >> even though smp_store_release() is not. However, the x86 architecture >> will guarantee the writes are in order, I think. > The contended case (where HPET_SEQ_LOCKED(seq)) doesn't do the cmpxchg. > So it's entirely relying on the READ_ONCE() on the "reader" side and > the cmpxchg/smp_store_release() on the "writer". This probably works in > practice, but I'm not sure it's guaranteed behavior. > It is true that the latency where the sequence number change becomes visible to others can be unpredictable. All the code in the writer side is doing is to make sure that the new HPET value is visible before the sequence number change. Do you know of a way to reduce the latency without introducing too much overhead, like changing the smp_store_release() to smp_store_mb(), maybe? Cheers, Longman