From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933063AbbIYTIF (ORCPT ); Fri, 25 Sep 2015 15:08:05 -0400 Received: from mail-ig0-f177.google.com ([209.85.213.177]:33582 "EHLO mail-ig0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932890AbbIYTID (ORCPT ); Fri, 25 Sep 2015 15:08:03 -0400 Subject: Re: [PATCH 1/3] Make /dev/urandom scalable To: "Theodore Ts'o" , Jeff Epler , Andi Kleen , linux-kernel@vger.kernel.org, kirill.shutemov@linux.intel.com, herbert@gondor.apana.org.au, Andi Kleen References: <1442963767-14945-1-git-send-email-andi@firstfloor.org> <5603004A.20801@gmail.com> <20150923232841.GK1747@two.firstfloor.org> <5603E083.8020004@gmail.com> <20150924131235.GB6841@thunk.org> <56041E2C.2030602@gmail.com> <20150924165204.GA2835@unpythonic.net> <56044ADB.5050102@gmail.com> <20150924201413.GA3989@thunk.org> <560532FC.1070601@gmail.com> From: Austin S Hemmelgarn Message-ID: <56059B8A.2010402@gmail.com> Date: Fri, 25 Sep 2015 15:07:54 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <560532FC.1070601@gmail.com> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms030604050306050906020801" X-Antivirus: avast! (VPS 150925-0, 2015-09-25), Outbound message X-Antivirus-Status: Clean Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This is a cryptographically signed message in MIME format. --------------ms030604050306050906020801 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable On 2015-09-25 07:41, Austin S Hemmelgarn wrote: > On 2015-09-24 16:14, Theodore Ts'o wrote: >> On Thu, Sep 24, 2015 at 03:11:23PM -0400, Austin S Hemmelgarn wrote: >>>> That is a startling result. Please say what architecture, kernel >>>> version, dieharder version and commandline arguments you are using t= o >>>> get 10% WEAK or FAILED assessments from dieharder on /dev/urandom. >>> >>> I do not remember what exact dieharder version or command-line argume= nts >>> (this was almost a decade ago), except that I compiled it from source= >>> myself, I do remember it was a 32-bit x86 processor (as that was >>> sadly all I >>> had to run Linux on at the time), and an early 2.6 series kernel >>> (which if I >>> remember correctly was already EOL by the time I was using it). >> >> It might have been nice if you had said this from the beginning >> instead of making an unqualified statement with the assumption that it= >> was applicable to kernels likely to be used today in non-obsolete >> systems. Otherwise it risks generating a click-bait article on >> Phoronix that would get people really worried for no good reason... > I sincerely apologize about this, I should have been more specific righ= t > from the beginning (I need to get better about that when talking to > people, I'm so used to dealing with some of my friends who couldn't > event tell you the difference between RAM and a hard drive, think a bus= > is only something you use for transportation, and get confused when I > try to properly explain even relatively simple CS and statistics concep= ts). >> >> There was a bug a long, long time ago (which where we weren't doing >> sufficient locking and if two processes raced reading from >> /dev/urandom at the same time, it was possible that the two processes >> would get the same value read out from /dev/urandom). This was fixed >> a long time ago, though, and in fact the scalability problem which >> Andi is trying to fix was caused by that extra locking that was >> added. :-) >> >> It's possible that is what you saw. I don't know, since there was no >> reproduction information to back up your rather startling claim. > I don't think this was what I hit, I'm pretty sure I had serialized the= > dieharder runs. >> >> If you can reproduce consistent Dieharder failures, please do let us >> know with detailed reproduction instructures. > Will do. OK, just started a couple of runs in parallel using different generators = using the following command line: dieharder -a -m 32 -k 1 -Y 1 -g XXX with one each for: /dev/urandom (502) AES_OFB (205) glibc random() (039) mt19937 (013) The above command line will run all dieharder tests with 12800 psamples, = using a higher than default precision, and re-running tests that return=20 WEAK until it gets a PASS or FAIL. Even on the relatively fast (at=20 least, fast for a desktop) system I'm running them on, I expect it will=20 take quite some time to finish (although regardless of that I'm probably = not going to be getting back to it until Monday). Interestingly, based on what dieharder is already saying about=20 performance, /dev/urandom is slower than AES_OFB (at least, on this=20 particular system, happy to provide hardware specs if someone wants). --------------ms030604050306050906020801 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Brgwgga0MIIEnKADAgECAgMRLfgwDQYJKoZIhvcNAQENBQAweTEQMA4GA1UEChMHUm9vdCBD QTEeMBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0IFNp Z25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2VydC5vcmcwHhcN MTUwOTIxMTEzNTEzWhcNMTYwMzE5MTEzNTEzWjBjMRgwFgYDVQQDEw9DQWNlcnQgV29UIFVz ZXIxIzAhBgkqhkiG9w0BCQEWFGFoZmVycm9pbjdAZ21haWwuY29tMSIwIAYJKoZIhvcNAQkB FhNhaGVtbWVsZ0BvaGlvZ3QuY29tMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA nQ/81tq0QBQi5w316VsVNfjg6kVVIMx760TuwA1MUaNQgQ3NyUl+UyFtjhpkNwwChjgAqfGd LIMTHAdObcwGfzO5uI2o1a8MHVQna8FRsU3QGouysIOGQlX8jFYXMKPEdnlt0GoQcd+BtESr pivbGWUEkPs1CwM6WOrs+09bAJP3qzKIr0VxervFrzrC5Dg9Rf18r9WXHElBuWHg4GYHNJ2V Ab8iKc10h44FnqxZK8RDN8ts/xX93i9bIBmHnFfyNRfiOUtNVeynJbf6kVtdHP+CRBkXCNRZ qyQT7gbTGD24P92PS2UTmDfplSBcWcTn65o3xWfesbf02jF6PL3BCrVnDRI4RgYxG3zFBJuG qvMoEODLhHKSXPAyQhwZINigZNdw5G1NqjXqUw+lIqdQvoPijK9J3eijiakh9u2bjWOMaleI SMRR6XsdM2O5qun1dqOrCgRkM0XSNtBQ2JjY7CycIx+qifJWsRaYWZz0aQU4ZrtAI7gVhO9h pyNaAGjvm7PdjEBiXq57e4QcgpwzvNlv8pG1c/hnt0msfDWNJtl3b6elhQ2Pz4w/QnWifZ8E BrFEmjeeJa2dqjE3giPVWrsH+lOvQQONsYJOuVb8b0zao4vrWeGmW2q2e3pdv0Axzm/60cJQ haZUv8+JdX9ZzqxOm5w5eUQSclt84u+D+hsCAwEAAaOCAVkwggFVMAwGA1UdEwEB/wQCMAAw VgYJYIZIAYb4QgENBEkWR1RvIGdldCB5b3VyIG93biBjZXJ0aWZpY2F0ZSBmb3IgRlJFRSBo ZWFkIG92ZXIgdG8gaHR0cDovL3d3dy5DQWNlcnQub3JnMA4GA1UdDwEB/wQEAwIDqDBABgNV HSUEOTA3BggrBgEFBQcDBAYIKwYBBQUHAwIGCisGAQQBgjcKAwQGCisGAQQBgjcKAwMGCWCG SAGG+EIEATAyBggrBgEFBQcBAQQmMCQwIgYIKwYBBQUHMAGGFmh0dHA6Ly9vY3NwLmNhY2Vy dC5vcmcwMQYDVR0fBCowKDAmoCSgIoYgaHR0cDovL2NybC5jYWNlcnQub3JnL3Jldm9rZS5j cmwwNAYDVR0RBC0wK4EUYWhmZXJyb2luN0BnbWFpbC5jb22BE2FoZW1tZWxnQG9oaW9ndC5j b20wDQYJKoZIhvcNAQENBQADggIBADMnxtSLiIunh/TQcjnRdf63yf2D8jMtYUm4yDoCF++J jCXbPQBGrpCEHztlNSGIkF3PH7ohKZvlqF4XePWxpY9dkr/pNyCF1PRkwxUURqvuHXbu8Lwn 8D3U2HeOEU3KmrfEo65DcbanJCMTTW7+mU9lZICPP7ZA9/zB+L0Gm1UNFZ6AU50N/86vjQfY WgkCd6dZD4rQ5y8L+d/lRbJW7ZGEQw1bSFVTRpkxxDTOwXH4/GpQfnfqTAtQuJ1CsKT12e+H NSD/RUWGTr289dA3P4nunBlz7qfvKamxPymHeBEUcuICKkL9/OZrnuYnGROFwcdvfjGE5iLB kjp/ttrY4aaVW5EsLASNgiRmA6mbgEAMlw3RwVx0sVelbiIAJg9Twzk4Ct6U9uBKiJ8S0sS2 8RCSyTmCRhJs0vvva5W9QUFGmp5kyFQEoSfBRJlbZfGX2ehI2Hi3U2/PMUm2ONuQG1E+a0AP u7I0NJc/Xil7rqR0gdbfkbWp0a+8dAvaM6J00aIcNo+HkcQkUgtfrw+C2Oyl3q8IjivGXZqT 5UdGUb2KujLjqjG91Dun3/RJ/qgQlotH7WkVBs7YJVTCxfkdN36rToPcnMYOI30FWa0Q06gn F6gUv9/mo6riv3A5bem/BdbgaJoPnWQD9D8wSyci9G4LKC+HQAMdLmGoeZfpJzKHMYIE0TCC BM0CAQEwgYAweTEQMA4GA1UEChMHUm9vdCBDQTEeMBwGA1UECxMVaHR0cDovL3d3dy5jYWNl cnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0IFNpZ25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcN AQkBFhJzdXBwb3J0QGNhY2VydC5vcmcCAxEt+DANBglghkgBZQMEAgMFAKCCAiEwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTUwOTI1MTkwNzU0WjBPBgkq hkiG9w0BCQQxQgRAIDlQAK75E8/7lcj1S1eqcgHzpWT4iW9Th5DZq/DKLHDe4Bed9vc0F/Bj 3c4yTnXDR7/8fOUY/yI+UCf9cRQmZDBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGRBgkrBgEEAYI3EAQxgYMwgYAweTEQMA4GA1UE ChMHUm9vdCBDQTEeMBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlD QSBDZXJ0IFNpZ25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2Vy dC5vcmcCAxEt+DCBkwYLKoZIhvcNAQkQAgsxgYOggYAweTEQMA4GA1UEChMHUm9vdCBDQTEe MBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0IFNpZ25p bmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2VydC5vcmcCAxEt+DAN BgkqhkiG9w0BAQEFAASCAgAoi+mw/XBe6HlEX2fg/GQbDWqbXnPYehIeGUUCZInUZHfLHFOS wT6oLPA8dzjJqkXfPlCtPNYfYGa/lHKuOWc18UKZdaRnzRw3jYQdbUMNINnTsLqD5gz2TLI7 t8xZPWXfi7VK92LQNSF2hGSZolfnF3xKo00uDktJS7ZCfLuVTtNVOw0eCFf8JqxjJ2kcCUHT 3WoUf9w6I1PP7bPNPG9zUmSm/jqupHyL5ZNGx7S6iVj41XotxCxgnTMWd1LH5ME2GcUQ1XvS s3LslilPpsRWPkxYanmyImbzyMSfSDppvbx16bIMq6xQ3avRoVhCIeabVkBbpaXvu4NR1yNm xCQkDLpKSBk3OoK88BQCQwfl2fDJ2H1f9DTxre1z9s8bhh8j+SkNaSnaIXQ8J7lkHoYvjRRx f6FbHSLez9ecLrrxFEpWXcIQUgUgnwLxZlGj6VNtstxQ/4mQRM12wRe16Fbtb9li9Ptu5Ebk 67owSZDZ3depL0an+zxvDTkopFLlaS0Lhte57QqH85PfYDSvbxrlhbFjhrxG54u2D0tT6N3E bZwU0UEtXTCPbzLiew7re3eGFpowgzvgM9LYbgP/G8bEzCWDWvmp6Dx1NwG1a28DHSIDSAJ8 wk0kQ2Y/K1i+wOG2o/ig50FralHGl37+cGHAaPCGQyEpr/ZsdiK8hWje0AAAAAAAAA== --------------ms030604050306050906020801--