From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S964882AbbI2L5n (ORCPT ); Tue, 29 Sep 2015 07:57:43 -0400 Received: from mail-ig0-f171.google.com ([209.85.213.171]:37181 "EHLO mail-ig0-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S964789AbbI2L5f (ORCPT ); Tue, 29 Sep 2015 07:57:35 -0400 Subject: Re: [PATCH 1/3] Make /dev/urandom scalable To: "Theodore Ts'o" , Jeff Epler , Andi Kleen , linux-kernel@vger.kernel.org, kirill.shutemov@linux.intel.com, herbert@gondor.apana.org.au, Andi Kleen References: <1442963767-14945-1-git-send-email-andi@firstfloor.org> <5603004A.20801@gmail.com> <20150923232841.GK1747@two.firstfloor.org> <5603E083.8020004@gmail.com> <20150924131235.GB6841@thunk.org> <56041E2C.2030602@gmail.com> <20150924165204.GA2835@unpythonic.net> <56044ADB.5050102@gmail.com> <20150924201413.GA3989@thunk.org> <560532FC.1070601@gmail.com> <56059B8A.2010402@gmail.com> From: Austin S Hemmelgarn Message-ID: <560A7CAB.6060105@gmail.com> Date: Tue, 29 Sep 2015 07:57:31 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <56059B8A.2010402@gmail.com> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms020109040606000907010704" X-Antivirus: avast! (VPS 150929-0, 2015-09-29), Outbound message X-Antivirus-Status: Clean Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This is a cryptographically signed message in MIME format. --------------ms020109040606000907010704 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable On 2015-09-25 15:07, Austin S Hemmelgarn wrote: > On 2015-09-25 07:41, Austin S Hemmelgarn wrote: >> On 2015-09-24 16:14, Theodore Ts'o wrote: >>> On Thu, Sep 24, 2015 at 03:11:23PM -0400, Austin S Hemmelgarn wrote: >>>>> That is a startling result. Please say what architecture, kernel >>>>> version, dieharder version and commandline arguments you are using = to >>>>> get 10% WEAK or FAILED assessments from dieharder on /dev/urandom. >>>> >>>> I do not remember what exact dieharder version or command-line >>>> arguments >>>> (this was almost a decade ago), except that I compiled it from sourc= e >>>> myself, I do remember it was a 32-bit x86 processor (as that was >>>> sadly all I >>>> had to run Linux on at the time), and an early 2.6 series kernel >>>> (which if I >>>> remember correctly was already EOL by the time I was using it). >>> >>> It might have been nice if you had said this from the beginning >>> instead of making an unqualified statement with the assumption that i= t >>> was applicable to kernels likely to be used today in non-obsolete >>> systems. Otherwise it risks generating a click-bait article on >>> Phoronix that would get people really worried for no good reason... >> I sincerely apologize about this, I should have been more specific rig= ht >> from the beginning (I need to get better about that when talking to >> people, I'm so used to dealing with some of my friends who couldn't >> event tell you the difference between RAM and a hard drive, think a bu= s >> is only something you use for transportation, and get confused when I >> try to properly explain even relatively simple CS and statistics >> concepts). >>> >>> There was a bug a long, long time ago (which where we weren't doing >>> sufficient locking and if two processes raced reading from >>> /dev/urandom at the same time, it was possible that the two processes= >>> would get the same value read out from /dev/urandom). This was fixed= >>> a long time ago, though, and in fact the scalability problem which >>> Andi is trying to fix was caused by that extra locking that was >>> added. :-) >>> >>> It's possible that is what you saw. I don't know, since there was no= >>> reproduction information to back up your rather startling claim. >> I don't think this was what I hit, I'm pretty sure I had serialized th= e >> dieharder runs. >>> >>> If you can reproduce consistent Dieharder failures, please do let us >>> know with detailed reproduction instructures. >> Will do. > OK, just started a couple of runs in parallel using different generator= s > using the following command line: > dieharder -a -m 32 -k 1 -Y 1 -g XXX > with one each for: > /dev/urandom (502) > AES_OFB (205) > glibc random() (039) > mt19937 (013) > The above command line will run all dieharder tests with 12800 psamples= , > using a higher than default precision, and re-running tests that return= > WEAK until it gets a PASS or FAIL. Even on the relatively fast (at > least, fast for a desktop) system I'm running them on, I expect it will= > take quite some time to finish (although regardless of that I'm probabl= y > not going to be getting back to it until Monday). > > Interestingly, based on what dieharder is already saying about > performance, /dev/urandom is slower than AES_OFB (at least, on this > particular system, happy to provide hardware specs if someone wants). > Apologies for not replying yesterday like I said I would. I actually didn't get a chance to run the tests to completion as the=20 wifi card in the system I was running the tests on lost it's mind about=20 55 hours in and I had to cold reboot the system to reset it. I would=20 give the results here, except that I have a feeling that people probably = don't want 110kb of data in the e-mail body, and thunderbird is for some = reason choking on trying to attach files. In general, the results were=20 pretty typical of a good PRNG, performance differences not withstanding. = In other words, don't use /dev/urandom except for seeding other=20 PRNG's, but because of the speed, not the quality. --------------ms020109040606000907010704 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Brgwgga0MIIEnKADAgECAgMRLfgwDQYJKoZIhvcNAQENBQAweTEQMA4GA1UEChMHUm9vdCBD QTEeMBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0IFNp Z25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2VydC5vcmcwHhcN MTUwOTIxMTEzNTEzWhcNMTYwMzE5MTEzNTEzWjBjMRgwFgYDVQQDEw9DQWNlcnQgV29UIFVz ZXIxIzAhBgkqhkiG9w0BCQEWFGFoZmVycm9pbjdAZ21haWwuY29tMSIwIAYJKoZIhvcNAQkB FhNhaGVtbWVsZ0BvaGlvZ3QuY29tMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA nQ/81tq0QBQi5w316VsVNfjg6kVVIMx760TuwA1MUaNQgQ3NyUl+UyFtjhpkNwwChjgAqfGd LIMTHAdObcwGfzO5uI2o1a8MHVQna8FRsU3QGouysIOGQlX8jFYXMKPEdnlt0GoQcd+BtESr pivbGWUEkPs1CwM6WOrs+09bAJP3qzKIr0VxervFrzrC5Dg9Rf18r9WXHElBuWHg4GYHNJ2V Ab8iKc10h44FnqxZK8RDN8ts/xX93i9bIBmHnFfyNRfiOUtNVeynJbf6kVtdHP+CRBkXCNRZ qyQT7gbTGD24P92PS2UTmDfplSBcWcTn65o3xWfesbf02jF6PL3BCrVnDRI4RgYxG3zFBJuG qvMoEODLhHKSXPAyQhwZINigZNdw5G1NqjXqUw+lIqdQvoPijK9J3eijiakh9u2bjWOMaleI SMRR6XsdM2O5qun1dqOrCgRkM0XSNtBQ2JjY7CycIx+qifJWsRaYWZz0aQU4ZrtAI7gVhO9h pyNaAGjvm7PdjEBiXq57e4QcgpwzvNlv8pG1c/hnt0msfDWNJtl3b6elhQ2Pz4w/QnWifZ8E BrFEmjeeJa2dqjE3giPVWrsH+lOvQQONsYJOuVb8b0zao4vrWeGmW2q2e3pdv0Axzm/60cJQ haZUv8+JdX9ZzqxOm5w5eUQSclt84u+D+hsCAwEAAaOCAVkwggFVMAwGA1UdEwEB/wQCMAAw VgYJYIZIAYb4QgENBEkWR1RvIGdldCB5b3VyIG93biBjZXJ0aWZpY2F0ZSBmb3IgRlJFRSBo ZWFkIG92ZXIgdG8gaHR0cDovL3d3dy5DQWNlcnQub3JnMA4GA1UdDwEB/wQEAwIDqDBABgNV HSUEOTA3BggrBgEFBQcDBAYIKwYBBQUHAwIGCisGAQQBgjcKAwQGCisGAQQBgjcKAwMGCWCG SAGG+EIEATAyBggrBgEFBQcBAQQmMCQwIgYIKwYBBQUHMAGGFmh0dHA6Ly9vY3NwLmNhY2Vy dC5vcmcwMQYDVR0fBCowKDAmoCSgIoYgaHR0cDovL2NybC5jYWNlcnQub3JnL3Jldm9rZS5j cmwwNAYDVR0RBC0wK4EUYWhmZXJyb2luN0BnbWFpbC5jb22BE2FoZW1tZWxnQG9oaW9ndC5j b20wDQYJKoZIhvcNAQENBQADggIBADMnxtSLiIunh/TQcjnRdf63yf2D8jMtYUm4yDoCF++J jCXbPQBGrpCEHztlNSGIkF3PH7ohKZvlqF4XePWxpY9dkr/pNyCF1PRkwxUURqvuHXbu8Lwn 8D3U2HeOEU3KmrfEo65DcbanJCMTTW7+mU9lZICPP7ZA9/zB+L0Gm1UNFZ6AU50N/86vjQfY WgkCd6dZD4rQ5y8L+d/lRbJW7ZGEQw1bSFVTRpkxxDTOwXH4/GpQfnfqTAtQuJ1CsKT12e+H NSD/RUWGTr289dA3P4nunBlz7qfvKamxPymHeBEUcuICKkL9/OZrnuYnGROFwcdvfjGE5iLB kjp/ttrY4aaVW5EsLASNgiRmA6mbgEAMlw3RwVx0sVelbiIAJg9Twzk4Ct6U9uBKiJ8S0sS2 8RCSyTmCRhJs0vvva5W9QUFGmp5kyFQEoSfBRJlbZfGX2ehI2Hi3U2/PMUm2ONuQG1E+a0AP u7I0NJc/Xil7rqR0gdbfkbWp0a+8dAvaM6J00aIcNo+HkcQkUgtfrw+C2Oyl3q8IjivGXZqT 5UdGUb2KujLjqjG91Dun3/RJ/qgQlotH7WkVBs7YJVTCxfkdN36rToPcnMYOI30FWa0Q06gn F6gUv9/mo6riv3A5bem/BdbgaJoPnWQD9D8wSyci9G4LKC+HQAMdLmGoeZfpJzKHMYIE0TCC BM0CAQEwgYAweTEQMA4GA1UEChMHUm9vdCBDQTEeMBwGA1UECxMVaHR0cDovL3d3dy5jYWNl cnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0IFNpZ25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcN AQkBFhJzdXBwb3J0QGNhY2VydC5vcmcCAxEt+DANBglghkgBZQMEAgMFAKCCAiEwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTUwOTI5MTE1NzMxWjBPBgkq hkiG9w0BCQQxQgRA99yZR/LNFoqnjHdcfr9bpexJrqJZncwuKA09g6iLOZM3evcGH84Xb6sA m5ReCSYcwT2jKbo71cjdPv0jDAiudDBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGRBgkrBgEEAYI3EAQxgYMwgYAweTEQMA4GA1UE ChMHUm9vdCBDQTEeMBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlD QSBDZXJ0IFNpZ25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2Vy dC5vcmcCAxEt+DCBkwYLKoZIhvcNAQkQAgsxgYOggYAweTEQMA4GA1UEChMHUm9vdCBDQTEe MBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0IFNpZ25p bmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2VydC5vcmcCAxEt+DAN BgkqhkiG9w0BAQEFAASCAgB8QfAs99h0fotOVF1N0B4qanbbIg7Vz+O5xJYG1l+C8VKjwBB9 1O/nFP+l4QYMQIV4iOhLiff/bDkPzNbl+CMc9m1g2SY/Gw6lqV+D0jHxFsxD62kcfDgID8y0 +Z7mCRk/zr0iojAHmPVZ92F+84NbKtBvQGN+RDNObbK3rpExIQI1Gz+/IDkftrX5fGqj3lbU 6I2wAvMLpUDwygankM3Eg9POLG9kyfKME04lhbplREOU/lxfEWYZJ0A/lB/J+fdYPhJ0XM3E 1y69oACJpvyJtCh6GDj6X9PIwsJ82YwFgRss6RSzHhTM+tq6jTRbjtnegM0UGMcNcYGCqrrr 8Zx/Qx0ldO/MFvDuMPPLeXslm8UKpmbKno5Ez4bri/8pmuMNVHcYUwzN3Jhn3prTt9KH5Zvs +8V+e4UlfdUx0ABbI3ohoLwp3lvA8KxL9yJ0en7lGVMTjAqLhY9yyZgh+wLck0shTquj+AXt hUoo0jTkL49N9Hje1dgWFonHJXSorHM7y84woDJSedM5BvYweRThyYq6WgYcGaVshRSaKrKI knwn/xgt/lP22ggqxsongSesDkLxSUVsJbrEI+dm7gctmJVGjRpkrvyHStAKSF+eJLSKzyPe YaiH55myDSDBECq2S0JLZvrpSkoSn33XwhjsY+lZbDTTeEZZIx2HqpfokQAAAAAAAA== --------------ms020109040606000907010704--