From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933475AbcINWP6 (ORCPT ); Wed, 14 Sep 2016 18:15:58 -0400 Received: from mail-sn1nam01on0097.outbound.protection.outlook.com ([104.47.32.97]:55030 "EHLO NAM01-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1765900AbcINWPx (ORCPT ); Wed, 14 Sep 2016 18:15:53 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=waiman.long@hpe.com; Message-ID: <57D9CC0D.7010608@hpe.com> Date: Wed, 14 Sep 2016 18:15:41 -0400 From: Waiman Long User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130109 Thunderbird/10.0.12 MIME-Version: 1.0 To: Linus Torvalds CC: "Theodore Ts'o" , Arnd Bergmann , Greg Kroah-Hartman , Linux Kernel Mailing List , Scott J Norton , Douglas Hatch Subject: Re: [PATCH] random: Fix kernel panic due to system_wq use before init References: <1473879781-23819-1-git-send-email-Waiman.Long@hpe.com> <57D9A1A9.8050506@hpe.com> <57D9A633.2030904@hpe.com> In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [72.71.243.249] X-ClientProxiedBy: CY1PR14CA0032.namprd14.prod.outlook.com (10.163.13.170) To CS1PR84MB0309.NAMPRD84.PROD.OUTLOOK.COM (10.162.189.158) X-MS-Office365-Filtering-Correlation-Id: 9831a78a-599f-4479-7467-08d3dcecad53 X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0309;2:/+nTpLJgUqB4Z9tA18PrQCC9g75dj0mXKi0HxfHQ75s56Laabyo4KMmiLAvYw3tGut4ifHdgZgRR72RevGrQPAo/AlxPS3GF50fDf9QyEXUEPQimJf7jf6GMZ5TovGNpkmlgkx6OCD/tdRwRlwmJrhBwi4Jw0JknNfB48xnpAzeAsDpHuSkyBNmylgncsvrk;3:Ig/YNTggTvwna3sz79UMKhd1B6fz1NbcySNR5bap1UXwJJ8cBUaExEWvuvZy5jzH+Gj/JKndtQkESgriv1ViUKGL/R7gmmKsXSpusoY/tExwYCuib+ZOMieE20odfZmd;25:N+ewZapAM4X4LvyyzLlHrPSrHILMVaSCLBlLT/17oeQzLboBPnOVtNexIigtfnAPM2Wqz9WGwDf36haK1bQ+z7TvaQfCYeZ7XBopz3F35f+jFdiwCKsYybciFNpoRCAHQ8uFiBIqz2KypPxrmov1GXHPT0wT90qowjTMnWedud02WIcbaGJEehLfPy6tM5dMnuhYvOgFsNtQe9nbyi13wQ+s9sWWBH1C7SvPt9wZgL5AnKhjDA0s4CuZn8zCWbUcDY4ZdITB+JbYQcICH/ozaIVUZ78ywOynUkv/7AvKtgfuXoS15z75KeWtASHLf2xBSyE6qhjyKfjqiRLB23M7YGhZ6anLiDQL5/0eaRiup1yO6taBr8FOT2O3RmGvC77zEP+K43qtGlswlaiL7KCzoBH7h+SyNuKwMMnb8SgM2m0= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CS1PR84MB0309; X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0309;31:/Who8E+2VOdXgxH4rOXYomGc1lh1PLQbIic1IGHPzY/T+Ig3QKRezNTwCaifs4tOXCbfIYXuImNLQ3tjGEi5gCbBLbzU+kDo5gGP87Nf9GVAmlYJpLWTsxmeL03IIS8z6FkZzU8YI9lhK8jV0BN0thA17AlS60MhrA7J3+SsebYYEJUhjMtkSha8Nfb6G/JPFeKFeURf2Ihd79NU9Nu18aqghDbYBlEP0M4f6DIXotM=;20:NVQW+nATmZ5YIF6DDRZ3oZdMoO4WrFnbizSVczUK4SRPeZu+8qI5b+MDP/ZGEQzVr3RXz4xS3jHEkG3WgdpJ+TglGUsGdaUvIQrTmlDI52oJfOgR1Rnxu6USB1YDe/ZR/00ZGptECf42141EfhyNAdbRURMpOszmGFNYTs929cQc3iH/V1+HRQNRruMnTLh1s2Evd3jq0QvsJWx0UH0aXmmarJOO0qcNlKsEUFwsDzO6g6UZXbbFF3CvbgjkW74d49cxgc9MVYMTLVvpeRWF4rFjgY2oBz/5MPzkyJq4G3EeMUsFaUZg3PCLPDLrA5IEB3HS8M8qwBCiUPMNIapzz5CL/i0bjvagdz1do2d4/s075j5uB8Ijf/8RW+ZAlBjBCXGpsHHucBHrMOHk6P6GeVHzbH3zIg7qjl08Kpng8dskBrWM05RtuFHB5ROTnTFXTGfBVL+9abuZNN143Df8ISwef4/Gdj8LBLfocYay3LK7HMvjQUCmcCfAvrio2ssd X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(227479698468861); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6055026);SRVR:CS1PR84MB0309;BCL:0;PCL:0;RULEID:;SRVR:CS1PR84MB0309; X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0309;4:yyhMFZNYvLiYRs9BvA9gYywrLoBf2SXMfEvQl1yqBj3QMBqguionluZVlZcvm0oiE9Qe31F9txaCkNfi6p6n5lhAk9Va4mjFr+6IK6az8/UoTXx1SU5V1AD+13wUyUwak19loxxBGkq1NPqgwOakVe3w8i5Q/v6/KVxAyg/X+e0Elqt0Mmcu3h4ETvA4QXChSdCc0FnrUvHFaQFr2R3dQVJ7vzNPIQEj1LBUb/2DPgap0eiyKZca3CO+2gQhO9onbB235rYyrR1kmI914f2sbkXoVN8WWR8+MZMF1fC7d74rX4y35yGa/vNq3tvFXFvzgodQ0C7xyAcyyjlQBk94W76PlxvpbH1krOG0ua8vFMhH9RmEsg+57CDh9c6tpYkd4DdZJqknZRQDECH5eW6FdULt9rtWpRCWiavy/IvbzPRu+P8UsdCE9B+PlvzWVmBXOmR8fa3kdiPCHzFm3kVN2w== X-Forefront-PRVS: 006546F32A X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6049001)(6009001)(7916002)(377454003)(24454002)(199003)(189002)(230700001)(86362001)(80316001)(101416001)(36756003)(19580405001)(586003)(6116002)(3846002)(106356001)(77096005)(47776003)(65956001)(66066001)(305945005)(50466002)(4326007)(65806001)(68736007)(42186005)(2906002)(5660300001)(64126003)(105586002)(117156001)(33656002)(59896002)(2950100001)(4001350100001)(97736004)(19580395003)(83506001)(189998001)(81166006)(7846002)(50986999)(93886004)(65816999)(76176999)(92566002)(7736002)(8676002)(87266999)(110136003)(54356999)(81156014)(23676002);DIR:OUT;SFP:1102;SCL:1;SRVR:CS1PR84MB0309;H:[192.168.142.204];FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDUzFQUjg0TUIwMzA5OzIzOnZRY2h3MHlMOHlPVUNFVXBSUHpBV2UrMjJD?= =?utf-8?B?RDJHNDY3S1Z2YlVXRVRFa25yWFgzZWxva2RneEVZa1M5STVHSWExVGVWTUpF?= =?utf-8?B?WFJVeFZBTlJTOEYvWTlFRWNOeTQ3YzByc3laSm9qb01YWGFDcTNXNEwyaXJ2?= =?utf-8?B?ZTZrU0ZGMkNvbkNsYzhBOFpYOXlmR2grRUNGUkJrMjFoODdTUzZPUXJlYXFT?= =?utf-8?B?b3U3dzRBN0R4UkIwdTlUMVQ5c3BObU95SFZiMVpub051TUlQQzNJRm1tMFIy?= =?utf-8?B?cWpwK2JhcTJmVUowVWpiYk1jdjJLOEdhNFZUclRMR3d1c3I3elJGNjh5Qmt3?= =?utf-8?B?Znd5eVVSR2VpUlZwT3NyaG9IVG9NdUd1OUgzcjVyYjRLN0V2anl3eHRWa01q?= =?utf-8?B?cDlyUjJXNWsydjZnTW9iREVFOFhPdjlrK2VRR056QTlQNEpUcTNWbDkyOWZq?= =?utf-8?B?YUVPd0RQNUVtaHB0Q2NIejdQSXQyU2ZkKzRPMVRBL3JJNkxtVnhBZXVvRDBy?= =?utf-8?B?eWFsUzh3eHg2ck5nYkNJSE4xdGc4bkFmRDBhck83OVVVdVJCMUdLVWY0U3Jz?= =?utf-8?B?R1FBU3h0a0NaRGw4ZTU4NDU5dXhsblU1dm1BMURvbjdpSkdZRzloKys4VWRT?= =?utf-8?B?a0h0eWlwL2tobWIrS0pOL2Qyc25HNytBMTVyaloreGVUTzh1SWtUY0xxelY5?= =?utf-8?B?di9qYlpDNTYycUErbzdzam9aYUViVlJHZ1NxVk5RWlBpWkxlcEhzQ3lEcWx1?= =?utf-8?B?c25lM1FVa2RpS2FHWFU1WCt0UEpkU1JZbm1kTUdmUU1ncnc2NWQ5UHdTVi9u?= =?utf-8?B?TGVRNnVGL0JudUpBQ2FIaXJsbzU3Uk1QbTNmWUtoQmtmMUVsclNyOWNNMjNy?= =?utf-8?B?NlFEWTUxRkRrbjk1YjZaS0VWUEVvTjRPRmhWbDVJamlQZ0g1Tks3N0htNlNa?= =?utf-8?B?dmNENkErYUFBSUlEdGQyU1g5Q1o3V3FBQ0l6UGJqaWFneG9icU1Cdk5ySlhH?= =?utf-8?B?MWR0U2JMVFBUT0hkRmt4WC9aV1JVNFBWYk01V21ET3Q1V0lXTUNlbVNQR3RX?= =?utf-8?B?OTlmOVlHeHlraWNPNDcrV2FsbFk3ekJQVkd0T2MvQ0JMK3h0dmZ3Tmh5T0Vp?= =?utf-8?B?Q0x6dG13VzNpeUlhRXNmVUJncDI3UHpkY2lXdmNzQjNMcFpwM0Nqa1IrL0ZO?= =?utf-8?B?RVEyMWcxK0NiSFg5SUVXYVgzaERoV0g3M1RFczRhNGg5TFpSR1QzSllxN1N1?= =?utf-8?B?SzJ5UHUxcnpRMFcyR1A4NjlNSU9LWVdaWkkwYkNVc0tyeW1pVzNkaTl1d25H?= =?utf-8?B?dTdlMk9HZC9XdWdDQ3VZb1RjNFRFZ2pLL0QxTjB4REtmS1ZOdWlMRmRDbEJC?= =?utf-8?B?c2ZGUmpzTUFSVDdqaDN2UDE2ZTZYZDZNa3FJeWVHenZvNnNhejlzekhISnBx?= =?utf-8?B?LytYUHhJd1RuOEEvRG1ReWhPdHVrZUdTRVFpdFUybjd1cyt5REZyalBWL0xl?= =?utf-8?B?OERUNk16Ti92Mi8xMU1WWERWbWc0ZlEvZThHNXJha2lpSzgvcDlCZEc0LzBx?= =?utf-8?B?VmJaU2REQ05NTDBwV3RUV2xOa2tncVpKVzcvOTE2QlgxT2ZUMjN0ZXFMZnIx?= =?utf-8?B?eXU1dGlpVHVycDJuc3lZeFhPeTlnR01XcUdPa3dyQmU0d29tR1J4YXVLZWFu?= =?utf-8?B?TVlPL0FsZ0NyZlB3ZlVBWUw1MlZ6dHNQSWhWSFdsOVVENVd3SVVOMWZ4N2xX?= =?utf-8?B?SzgxWlNiQTdGczlxSFU2SUJ4Mm90dFRuNWdpdk1DUUg0SWlXYlc4aHJuNy8x?= =?utf-8?B?bnp3VzVoa3pidURxNUh2cEROREFra29TYmdYUm9IbG4xSGc9PQ==?= X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0309;6:ttYFKdG1ONAAVpRkyLhBDw/PCVaOoNzk/ix8uXATWhMxBqvFDEYfmkno8n8h9Zc4kZJy42xtKgyrx4HNwgk8oQvtQyCVPZXDWdq+2YpMUJTau6JP+xSKksKfmOjzB2yQzrawuvTZ2JnhT5ApHz8MuRTE5IMm4SynzOjedM3u4Dt7tgv3Dk55O9qxwqHqT+cb+N3ZwOvH/hhtFzrwSa4l4PC4AHojGKCHoMqFBTLue+mi+FArVJUi43s9PbkrSphnupY3nCHrdAHIja5d56RNR7UP4cTracoSYmY1+RPs86fGhXwSLy5MQlNV5aQIn4gU30Kryc/wAPW8QxNqiY7fjg==;5:uM/HfcxW9xtWBTR7SQhKDpDRG0pBiD2RnkByz9dqT93W0d2vpddBkvy/mfhr93xgqUYqFCvr4RbxFclhuNDZ/11bP9YJbRvEdNbEPJawetqdHK1N0E9Tz9BHsJ1/eAk5h5z/y4DfyA2NM2bwOFEIVA==;24:tTuaMQypv8jcTRzatB1sbiL10BB035w1HfEXZxdMq2dB1tKtYByVbVYxwj6W9UTe/CohTs3iDZam8lbIvqAncL8eA3Y9uw1kQSub8hASkQs=;7:cZOXufslWmIeqzxuGqlY6qFMI09GvrlQBq2II0VTzCow3PUzP7hwOJUu3Jt8MkbD8+jA7rf2BMrwTVFOMPJzKfKzMLtQq0zvfDjq3w8+QTt+Q0rIsgDPLwky4ShNjVBnxxNa50hPKeoxUR/3pc3536/yIn6xd8xq6vxYUyTswqZO6GMQ9KVHo5HAvLVuPmbdoyClThifg0XbMT1j6bAbnDL4e/0qJEm2S5zU8tvMx0aWfm7SaxEsySjSGC5BPQrd SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: hpe.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2016 22:15:46.0085 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CS1PR84MB0309 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 09/14/2016 05:06 PM, Linus Torvalds wrote: > On Wed, Sep 14, 2016 at 12:34 PM, Waiman Long wrote: >> I can try, but the 16-socket system that I have at the moment takes a long >> time (more than an hour) for one shutdown-reboot cycle. It may not be really >> more interrupts in 4.8, it may be that the random driver just somehow run >> very slow on my test machine as it seems to have a major rewrite in the 4.8 >> cycle. > Looking at the random driver updates since 4.7, the only thing I see > is that .crng_fast_load() for the chacha20 randomness. And that should > trigger only until it's been initialized, so the cost looks like it > should be limited. > > Is there some fundamental reason you think it's the random driver? > Other than the oops? Because I'd be more inclined to suspect just some > apic issue or something, where an actual interrupt line ends up > screaming or whatever. Is this UV? There's also the CPU hotplug state > machine changes etc. Yes, it is because of the oops that I suspect the random driver may be the cause. > But a few rounds of bisecting should hopefully cut down on the > suspects a lot. A *full* bisect might be 16-17 rounds, but if you can > do just four or five rounds of bisection, that should still cut it > down from 14k commits to "only" several hundred.. > > Linus Yes, I will do a few rounds to see if we can isolate the problem. In the mean time, I will also reconfigure the system with less sockets to see if it is reproduced in a smaller configuration. Cheers, Longman