From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754147AbdEQMPe (ORCPT ); Wed, 17 May 2017 08:15:34 -0400 Received: from mail-he1eur01on0127.outbound.protection.outlook.com ([104.47.0.127]:22400 "EHLO EUR01-HE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753719AbdEQMP3 (ORCPT ); Wed, 17 May 2017 08:15:29 -0400 Authentication-Results: lge.com; dkim=none (message not signed) header.d=none;lge.com; dmarc=none action=none header.from=virtuozzo.com; Subject: Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to reduce memory consumption To: , Andrew Morton References: <1494897409-14408-1-git-send-email-iamjoonsoo.kim@lge.com> CC: Alexander Potapenko , Dmitry Vyukov , , , , Thomas Gleixner , Ingo Molnar , "H . Peter Anvin" , , Joonsoo Kim From: Andrey Ryabinin Message-ID: Date: Wed, 17 May 2017 15:17:13 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <1494897409-14408-1-git-send-email-iamjoonsoo.kim@lge.com> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-Originating-IP: [195.214.232.6] X-ClientProxiedBy: DB6PR1001CA0043.EURPRD10.PROD.OUTLOOK.COM (10.168.69.157) To DB5PR0801MB2726.eurprd08.prod.outlook.com (10.166.176.22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b1a2559a-45fc-4e3a-34a0-08d49d1e6597 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(201703131423075)(201703031133081);SRVR:DB5PR0801MB2726; X-Microsoft-Exchange-Diagnostics: 1;DB5PR0801MB2726;3:JZ/KLlmcTBV1kYBxWHzEGIR21i+FSYBaKLgBkgp42Vcwga7Xnkzi9m7Uso0AT9TBzLeZ/lkkixItl9Lsz+cln3qzRb5/V0uR8NSCztsJA/ryy4No5WxPcz/ijYaOp2Sby2g811Oy4TomrMbcVjaGPLwm0hynrQPogRwJLgi4EiJL5jjumYYQl4xkbHdtDeSqqZ9SCG/yEDe+KR+xEdDV6p2zennG1o7OpHGlTSlyORvq4zO+5LoueCBT+F5xZAcEdCejl+H1zSlhEfKDXF5WGN5QJmPRnTwio2SF5J7CXaYwYENQHyG4gXbZQ9Rh7DrELMFMvrBVVi85larpdMnlXg==;25:ofZSZCV1A/vGQndJog1XPwa8AXkIwf714pGT64DhWfheFMuhO+dle6enSVhw1HVXewYB3QvQ5anFgqCHmtuDAK1N263Gf3nd/onR3Cs0z6fFLB6Be1BgCeuXUFfhaTIq932gPes53et7xLG7+92rUUAMrT49GzJBlvr4DGZgx7SR4Afywvoc7kbHWQOnqeDLLYECG9pwSDp4VIMjNUdp6fccj+dW3gHAZsx3YEYYaYaJs61oewn7KvNiZan24YriohKbOCjyYUEfynXPJsvoGEbgPnJawzxDK7GTzzEMgSe4HFfpoQ2LX/qKuJbp7ofSqJ7HRg9TKVKHXJUDtBJjdEnJfK8/sCOVGHvBqDruOh8Kv2ewzHxlg6dUoUUpTIi4qlWdLJU85KwoeA2nl8LFZyHb2cRifkCQRR+i9Q03LYF4dypXKgekXU38OxkP0uYF5j0TT5vON+P/+jtyvNInF1Ft089nWCvIeeyQqAEeizE= X-Microsoft-Exchange-Diagnostics: 1;DB5PR0801MB2726;31:S5tyvu43q0o/Nor18kGxyS4aHxVwI5ynil0bu0yAcHsjwkNTo3MHZbkum35NUK3YNDBWJvLp50/2MH2o2+piuFVzFcsWUsiG8BufyvT5JjykBROjhByq6A1DVf5Uh8+RDyQfP5rGVyVYoHycWK5AAWEpCrRYVVBHT54GbKRDnl84PXvsON3N1Qgz5SzP/Tjdq6qwkh+mmL0zk+UEhz+tbOG1g0A/3ZFaEqwnQ2cZRI8YQRQmcSWZRBO93lh4OHEqqwhB89DCfJR2+m/PZp37Sg==;20:rZUizSAlxO5KmM0KaI9GHF6I/LkJnrnvWAA3gDw5Q2Gq19sGy/vuzRLrV7VFgdZxbH6kTgNqmVB7uc5cpA6Hd5m+mJL8efwQHXsILf+jNOnvdwlYzSehFIjf5IdhGCNaztZqPLstX9pGkyYdnZfOsTDZiWhhDVHt+zwg9gCvQZn3oHDJbmZ7Ewf6gB+lcfPl9LJmbH14Ngr9E5BfnztHWBOYpNsfAR1u0DRWlS+mC36zFXiN80JPSLDEeLhBkprRLezr7ywDgI+83i22HXpN1MzX6zwRGS1/TxS/rlpcO3rarlr8vStfHmP8nBI48Vm3QTEnyfqzET5sVTKkEV8ZkNmw2eoVpSLFP/Q/98vjF9PO4Z33WnGx03KqlZ36FdFD26OmfJ21SAP/lUdwSw6hBaHMWGwoKpmyxqSFw6NvM50= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040450)(601004)(2401047)(8121501046)(5005006)(93006095)(93001095)(3002001)(10201501046)(6041248)(20161123560025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123555025)(20161123558100)(6072148);SRVR:DB5PR0801MB2726;BCL:0;PCL:0;RULEID:;SRVR:DB5PR0801MB2726; X-Microsoft-Exchange-Diagnostics: 1;DB5PR0801MB2726;4:2DeneKdB0XW6C2/YRFFvfomJ9sZU4mbeLA9ULopa1PLcdWR9xsn9R3PF13K7k0XwZ891/sv0AGg/Uw5/Er6W9j0yrEVvlJjnIs1dwehmMp1GtQPLB0h4JJy6tJJqFloERR/ohW0yHEYre9A/UIu/0eGBoqYvxmWIuMv7O7zAfZ8fFhJch0HziEgbnGcdd91vIxvHtWdkLCGYxZgnAzuehWVrALKvfqKdOc0dy2SyJH5NAo/jUof1AzRth1Hqg+j8SqWeh9ott+Itlrf3atNB34YvWMnu1+eXg2wqMfQXmetMw8LtGdynHAjOEmDsgvlKu8ttlJ7/iYOYFgPlr3b5S4Sz/7zOrQYdcQKJNf/nXad55lh7lmfinbFv0UYcUUj1OmzESB1n2uTrhIYb7tRa4pl3z9A5ZHWwDRvtLCY4NeuvQ+cIrVbU4MsKVV/4fd8T0+Pewq5wUMSY/Qpu+FXgOcxZGmersd6wmpqvPxOVnuXZ4boif2VYD+SU0M500fLWb7Lov87UPZsmD47bHNStXHvE+7OFwcVK0DD+7dynA875t8Cxuf+/IOS1v6tuOnkCGixrN64C0XjROJFoxXuVvpWdvy/ROCgKJ0SKwZ4K2NYWisY6Z0nrnoALMbUb6d4VUTm3qcpoHF34aaxAefp7YfAq8tV+QOOqPlzHED6szCEj2IA3LeXHbO0Q4zTFOJ4y4357egBCXdFfeLG63l0sEM66XU/dj8258w/jXoqsEZj/arHTOWxtROm4KZ/WyHFMr2SPabrSt3m+adN0HLJEG3YXdmw+9xxnXID1REZcT88= X-Forefront-PRVS: 0310C78181 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(6049001)(39840400002)(39450400003)(39400400002)(39410400002)(377454003)(51444003)(24454002)(4001350100001)(83506001)(86362001)(189998001)(8676002)(33646002)(81166006)(6666003)(77096006)(7736002)(5660300001)(3846002)(230700001)(31696002)(305945005)(2950100002)(2906002)(6116002)(478600001)(6486002)(7416002)(54906002)(229853002)(50466002)(54356999)(50986999)(38730400002)(53936002)(64126003)(53546009)(6246003)(76176999)(4326008)(25786009)(23746002)(47776003)(36756003)(42186005)(31686004)(65806001)(66066001);DIR:OUT;SFP:1102;SCL:1;SRVR:DB5PR0801MB2726;H:[172.16.25.12];FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;DB5PR0801MB2726;23:RMK888pvUc8sZzO3ad07ssug544RwiJpDSV?= =?Windows-1252?Q?YRAePxHgBgk2Dk06+vecSu+LMdir/RXZZswPZDyNnzmCPCm1r6BNowaB?= =?Windows-1252?Q?2LnCCoMh7wV2+3Uk0KaOnF2oi9ciTi9sQHVz2nl/ry2dLFR6aiUOgzu7?= =?Windows-1252?Q?RrBRu7aYZhgy4wyuDLKzlSM4HaMbXOXI/a+EeS80zQ4IbTnPc+8h6DCD?= =?Windows-1252?Q?4UB+56AOHqF/GJd7YEKaZ/jrX19lEld3OUX9Rfwtx7Rw5uTU857GSgWk?= =?Windows-1252?Q?b32nCT+bjDpnTlLDiaee+Zy+Ur0kasTTSnuerd8NptpCzXX/ZgqwJ8DU?= =?Windows-1252?Q?fdU7awP1T94kYhE5yEzD7Q9enU7RIIZ3dm7HPA/pLAff+cB99l5IcYss?= =?Windows-1252?Q?xmVm4PddTv+S6x86QonPpjSsuTPKn02NAtw2enWn2oVG6/6bbomYPjNI?= =?Windows-1252?Q?uqF2uhrZ6VrtcOWECP5Utqr1G6Iom/ZBYqnrp1pseFlRty6PH7J6ix2y?= =?Windows-1252?Q?wy/4sgOhd/7PeEt5wNJg8HfolgWg4mfR7SPkBNplMAS1Uu4AW9HbH4qq?= =?Windows-1252?Q?kRQuGDNImaBvvjBi6aCHKy8XSaG0IlWiJ1rLguNSfS+y/hmxvMJwMIMk?= =?Windows-1252?Q?88QpYotIxuI1YMOxXEjRS4oQSE13tUcXpeXz8OzgC9jT02qlkbwu57Rl?= =?Windows-1252?Q?jMxOYYSyLfk9t0qwJbLBAEdjkMkFL0JGyA7A6oJf/A3SaX4GlskGiDzf?= =?Windows-1252?Q?744pJYM8H49jWee4U6A3iJvD9izsqZb0OaLynjGMATX2U5QXmJyL2soj?= =?Windows-1252?Q?1lwdH42z7EgwRRodFpoWcCm+8Pblt+NF2pcnVSedAq/p87NdsddicfCe?= =?Windows-1252?Q?d0yUnWQBc9f0XKHNZ3PkXbMEFydK/BLGQ6RTLchaa2L3iS4jqfCd5qgO?= =?Windows-1252?Q?7ktPetUoInCI/tygK4hDAdrx04tB96Uw4J+NXnA4w1RJ7AhZx1g7jzzD?= =?Windows-1252?Q?vJhma8wKAWuVUz5ucm/GqSDgmDYAZSsH/Qm1aL3Z+RcS8/TMkK2OiQqE?= =?Windows-1252?Q?yPC6ox2D61OSF6Fd051krZh6MFWeaAYC/tPlwZYSC4cYAcy1e1Xf2ItP?= =?Windows-1252?Q?uLiyCJOCo1saw13ZQzXSTFyLjRuUv9Q67lhqMb7K/fKuuQMZ6O1GYh5w?= =?Windows-1252?Q?bciMgb4nKWvc4XfcP21986xAxnNU2cN3HC8v3/H15wZR0Q/k5jk9bUrn?= =?Windows-1252?Q?8uFmyKNBWIzhfwH03RDJ5fk+0woTWETJacLjDNGUW3COBI1/++o3tTAY?= =?Windows-1252?Q?s2muM0YTqWIkZQSe2diAIHLkQVXo8l9X3IBmcn+hF1fPaJBOUWZPR8ZT?= =?Windows-1252?Q?E2QWUMuWI+SdZ?= X-Microsoft-Exchange-Diagnostics: 1;DB5PR0801MB2726;6:Fhs866n2TY05lWIQPdxNjHP/DxVLO0DbaWdIK9ImfAi3Pv1vvLakFQt77Bw2R2JYBfQlLHRfsfUn56XVpaXjKa865BgxbkNDM8Hh4RUnQoBYeKqYpKo/8YFWIzlmqiKGjxLHitu/mn3I9t6wOcxi3QLlUpZTwbc3pZIaQCum0T4rmZGFtCWt5siURXnsdFxGrFCfSCFv/+ihrL678Sm0ZRQa7q6h4tMGIiH9XuSTRz9ivQ9KB+ajNXWqvpjhMTPvRvPaDy7zYMUBXti5QPsFvjKBdSMx9kGFACqdYXqzat4VMLZO7dyLaW0z6cpQQGjunUGw0pa6htyxMrl2Jv7KGdCOsBOIRTJBojkmD6iRtxfn4ASBUb5edFwTLNjy0u0dRX6/IzAOB69J5ZCxIycxrifBotG+qFxsX+izEueoURtrqU10zuyhP2n2wdBUJLycb+6Ktn3/Lguqt1rLjSPpZ+StbfHsn4NlhMC2+Ea8Kk7jeFnsgmddXW9Dorjm8hJqYrYdxoXVUwJxFV6MLBSL3g==;5:CHNZnEZf/UHYPBTMMpOjGOq0N9KdGhJEnN1Z4YEEyGL+Qv/BooOFT7H9aE+ZoIUSrPJmTJfKMI7l/QbFqKYwIuyvdBVzyFs5E8iA+IfAKmddrYTQGmOgR+cCfWniK7meLfEBn/prcWTmG7cpvyDvRw==;24:8IeCPgbeOwvesTjrJN4+/wpbJ4Zk+0kgqz0e/EU0BDRBoi6OK4YQYRYmofGnc8DlabLtdEEh45i9+wv5/JBMGet7FuBbw6ociAkE/cfh1UM= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;DB5PR0801MB2726;7:1mIYN1hTMHx9NB/BYCXIpY+PkqtvQm99Mk9cdj8GNVziBy9J2ck98FWdyO0T+SgpY2PNuCaiy4CPEkXhZZZWbyUsCpklI1fsowOVpMc+b3y7dna5y1KFdPvt4NvhEWmkcbNqIuQtqxKB0tWiZs3WRMHod/J0BCthsDJuCfvyHpq+VYNeSeYwx6CyY+0BSh+9eGqB6/8738WYfZ5ETtvQefAWGB8DLkALqrnjXq3gg+eJsi3nOlZfxZlZTFU6Al8zEYEsyFO/aO1V2PO9o1Y893O9h6E3J9eyVPSPX0DDt1CAK0wIizohJnGaDnTeLFi0ciw7ObUW0g8zV5eJ/uUbLw==;20:vYLnDRDo+8sHJx+cAZ6l0xxVeFat0A0BFl8o9p2R9lHoDliA94saw2+da1Q7Fk6o12ICVVLpKNl2sLiTwN7+r01WjhDjiFw/J1SKVWeaKQnRmRyKNUJQ8jGs2/GfK6w6OkWWRlFwfdC+n5UuZM9M6xZjCtAQ8QQmhhBHF/0U3lI= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2017 12:15:23.6013 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR0801MB2726 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/16/2017 04:16 AM, js1304@gmail.com wrote: > From: Joonsoo Kim > > Hello, all. > > This is an attempt to recude memory consumption of KASAN. Please see > following description to get the more information. > > 1. What is per-page shadow memory > > This patch introduces infrastructure to support per-page shadow memory. > Per-page shadow memory is the same with original shadow memory except > the granualarity. It's one byte shows the shadow value for the page. > The purpose of introducing this new shadow memory is to save memory > consumption. > > 2. Problem of current approach > > Until now, KASAN needs shadow memory for all the range of the memory > so the amount of statically allocated memory is so large. It causes > the problem that KASAN cannot run on the system with hard memory > constraint. Even if KASAN can run, large memory consumption due to > KASAN changes behaviour of the workload so we cannot validate > the moment that we want to check. > > 3. How does this patch fix the problem > > This patch tries to fix the problem by reducing memory consumption for > the shadow memory. There are two observations. > I think that the best way to deal with your problem is to increase shadow scale size. You'll need to add tunable to gcc to control shadow size. I expect that gcc has some places where 8-shadow scale size is hardcoded, but it should be fixable. The kernel also have some small amount of code written with KASAN_SHADOW_SCALE_SIZE == 8 in mind, which should be easy to fix. Note that bigger shadow scale size requires bigger alignment of allocated memory and variables. However, according to comments in gcc/asan.c gcc already aligns stack and global variables and at 32-bytes boundary. So we could bump shadow scale up to 32 without increasing current stack consumption. On a small machine (1Gb) 1/32 of shadow is just 32Mb which is comparable to yours 30Mb, but I expect it to be much faster. More importantly, this will require only small amount of simple changes in code, which will be a *lot* more easier to maintain. I'd start from implementing this on the kernel side only. With KASAN_OUTLINE and disabled stack instrumentation (--param asan-stack=0) it's doable without any changes in gcc. ... > base vs patched > > MemTotal: 858 MB vs 987 MB > runtime: 0 MB vs 30MB > Net Available: 858 MB vs 957 MB > > For 4096 MB QEMU system > > MemTotal: 3477 MB vs 4000 MB > runtime: 0 MB vs 50MB > > base vs patched (2048 MB QEMU system) > 204 s vs 224 s > Net Available: 3477 MB vs 3950 MB > > Thanks. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-f70.google.com (mail-pg0-f70.google.com [74.125.83.70]) by kanga.kvack.org (Postfix) with ESMTP id 57DE86B02C4 for ; Wed, 17 May 2017 08:15:31 -0400 (EDT) Received: by mail-pg0-f70.google.com with SMTP id x64so8203654pgd.6 for ; Wed, 17 May 2017 05:15:31 -0700 (PDT) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0111.outbound.protection.outlook.com. [104.47.0.111]) by mx.google.com with ESMTPS id d8si1950376plj.104.2017.05.17.05.15.29 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 17 May 2017 05:15:29 -0700 (PDT) Subject: Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to reduce memory consumption References: <1494897409-14408-1-git-send-email-iamjoonsoo.kim@lge.com> From: Andrey Ryabinin Message-ID: Date: Wed, 17 May 2017 15:17:13 +0300 MIME-Version: 1.0 In-Reply-To: <1494897409-14408-1-git-send-email-iamjoonsoo.kim@lge.com> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: js1304@gmail.com, Andrew Morton Cc: Alexander Potapenko , Dmitry Vyukov , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Thomas Gleixner , Ingo Molnar , "H . Peter Anvin" , kernel-team@lge.com, Joonsoo Kim On 05/16/2017 04:16 AM, js1304@gmail.com wrote: > From: Joonsoo Kim > > Hello, all. > > This is an attempt to recude memory consumption of KASAN. Please see > following description to get the more information. > > 1. What is per-page shadow memory > > This patch introduces infrastructure to support per-page shadow memory. > Per-page shadow memory is the same with original shadow memory except > the granualarity. It's one byte shows the shadow value for the page. > The purpose of introducing this new shadow memory is to save memory > consumption. > > 2. Problem of current approach > > Until now, KASAN needs shadow memory for all the range of the memory > so the amount of statically allocated memory is so large. It causes > the problem that KASAN cannot run on the system with hard memory > constraint. Even if KASAN can run, large memory consumption due to > KASAN changes behaviour of the workload so we cannot validate > the moment that we want to check. > > 3. How does this patch fix the problem > > This patch tries to fix the problem by reducing memory consumption for > the shadow memory. There are two observations. > I think that the best way to deal with your problem is to increase shadow scale size. You'll need to add tunable to gcc to control shadow size. I expect that gcc has some places where 8-shadow scale size is hardcoded, but it should be fixable. The kernel also have some small amount of code written with KASAN_SHADOW_SCALE_SIZE == 8 in mind, which should be easy to fix. Note that bigger shadow scale size requires bigger alignment of allocated memory and variables. However, according to comments in gcc/asan.c gcc already aligns stack and global variables and at 32-bytes boundary. So we could bump shadow scale up to 32 without increasing current stack consumption. On a small machine (1Gb) 1/32 of shadow is just 32Mb which is comparable to yours 30Mb, but I expect it to be much faster. More importantly, this will require only small amount of simple changes in code, which will be a *lot* more easier to maintain. I'd start from implementing this on the kernel side only. With KASAN_OUTLINE and disabled stack instrumentation (--param asan-stack=0) it's doable without any changes in gcc. ... > base vs patched > > MemTotal: 858 MB vs 987 MB > runtime: 0 MB vs 30MB > Net Available: 858 MB vs 957 MB > > For 4096 MB QEMU system > > MemTotal: 3477 MB vs 4000 MB > runtime: 0 MB vs 50MB > > base vs patched (2048 MB QEMU system) > 204 s vs 224 s > Net Available: 3477 MB vs 3950 MB > > Thanks. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org