From mboxrd@z Thu Jan 1 00:00:00 1970 From: Topi Miettinen Subject: Re: [PATCH 00/14] Present useful limits to user (v2) Date: Fri, 15 Jul 2016 16:35:33 +0000 Message-ID: <41b6ca51-1358-0fd7-b45a-dc29a1344865@gmail.com> References: <1468578983-28229-1-git-send-email-toiwoton@gmail.com> <20160715130458.GB21685@350D> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20160715130458.GB21685@350D> Sender: owner-linux-mm@kvack.org To: bsingharora@gmail.com Cc: linux-kernel@vger.kernel.org, Jonathan Corbet , Tony Luck , Fenghua Yu , Alexander Graf , Paolo Bonzini , =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" , Doug Ledford , Sean Hefty , Hal Rosenstock , Mike Marciniszyn , Dennis Dalessandro , Christian Benvenuti , Dave Goodell , Sudeep Dutt , Ashu List-Id: linux-rdma@vger.kernel.org On 07/15/16 13:04, Balbir Singh wrote: > On Fri, Jul 15, 2016 at 01:35:47PM +0300, Topi Miettinen wrote: >> Hello, >> >> There are many basic ways to control processes, including capabilities, >> cgroups and resource limits. However, there are far fewer ways to find out >> useful values for the limits, except blind trial and error. >> >> This patch series attempts to fix that by giving at least a nice starting >> point from the highwater mark values of the resources in question. >> I looked where each limit is checked and added a call to update the mark >> nearby. >> >> Example run of program from Documentation/accounting/getdelauys.c: >> >> ./getdelays -R -p `pidof smartd` >> printing resource accounting >> RLIMIT_CPU=0 >> RLIMIT_FSIZE=0 >> RLIMIT_DATA=18198528 >> RLIMIT_STACK=135168 >> RLIMIT_CORE=0 >> RLIMIT_RSS=0 >> RLIMIT_NPROC=1 >> RLIMIT_NOFILE=55 >> RLIMIT_MEMLOCK=0 >> RLIMIT_AS=130879488 >> RLIMIT_LOCKS=0 >> RLIMIT_SIGPENDING=0 >> RLIMIT_MSGQUEUE=0 >> RLIMIT_NICE=0 >> RLIMIT_RTPRIO=0 >> RLIMIT_RTTIME=0 >> >> ./getdelays -R -C /sys/fs/cgroup/systemd/system.slice/smartd.service/ >> printing resource accounting >> sleeping 1, blocked 0, running 0, stopped 0, uninterruptible 0 >> RLIMIT_CPU=0 >> RLIMIT_FSIZE=0 >> RLIMIT_DATA=18198528 >> RLIMIT_STACK=135168 >> RLIMIT_CORE=0 >> RLIMIT_RSS=0 >> RLIMIT_NPROC=1 >> RLIMIT_NOFILE=55 >> RLIMIT_MEMLOCK=0 >> RLIMIT_AS=130879488 >> RLIMIT_LOCKS=0 >> RLIMIT_SIGPENDING=0 >> RLIMIT_MSGQUEUE=0 >> RLIMIT_NICE=0 >> RLIMIT_RTPRIO=0 >> RLIMIT_RTTIME=0 > > Does this mean that rlimit_data and rlimit_stack should be set to the > values as specified by the data above? My plan is that either system administrator, distro maintainer or even upstream developer can get reasonable values for the limits. They may still be wrong, but things would be better than without any help to configure the system. > > Do we expect a smart user space daemon to then tweak the RLIMIT values? Someone could write an autotuning daemon that checks if the system has changed (for example due to upgrade) and then run some tests to reconfigure the system. But the limits are a bit too fragile, or rather, applications can't handle failure, so I don't know if that would really work. -Topi > > Balbir Singh. > -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Subject: Re: [PATCH 00/14] Present useful limits to user (v2) To: bsingharora@gmail.com References: <1468578983-28229-1-git-send-email-toiwoton@gmail.com> <20160715130458.GB21685@350D> Cc: linux-kernel@vger.kernel.org, Jonathan Corbet , Tony Luck , Fenghua Yu , Alexander Graf , Paolo Bonzini , =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" , Doug Ledford , Sean Hefty , Hal Rosenstock , Mike Marciniszyn , Dennis Dalessandro , Christian Benvenuti , Dave Goodell , Sudeep Dutt , Ashutosh Dixit , Alex Williamson , Alexander Viro , Tejun Heo , Li Zefan , Johannes Weiner , Peter Zijlstra , Alexei Starovoitov , Arnaldo Carvalho de Melo , Alexander Shishkin , Markus Elfring , "David S. Miller" , Nicolas Dichtel , Andrew Morton , Konstantin Khlebnikov , Jiri Slaby , Cyrill Gorcunov , Michal Hocko , Vlastimil Babka , Dave Hansen , Greg Kroah-Hartman , Dan Carpenter , Michael Kerrisk , "Kirill A. Shutemov" , Marcus Gelderie , Vladimir Davydov , Joe Perches , Frederic Weisbecker , Andrea Arcangeli , "Eric W. Biederman" , Andi Kleen , Oleg Nesterov , Stas Sergeev , Amanieu d'Antras , Richard Weinberger , Wang Xiaoqiang , Helge Deller , Mateusz Guzik , Alex Thorlton , Ben Segall , John Stultz , Rik van Riel , Eric B Munson , Alexey Klimov , Chen Gang , Andrey Ryabinin , David Rientjes , Hugh Dickins , Alexander Kuleshov , "open list:DOCUMENTATION" , "open list:IA64 (Itanium) PLATFORM" , "open list:KERNEL VIRTUAL MACHINE (KVM) FOR POWERPC" , "open list:KERNEL VIRTUAL MACHINE (KVM)" , "open list:LINUX FOR POWERPC (32-BIT AND 64-BIT)" , "open list:INFINIBAND SUBSYSTEM" , "open list:FILESYSTEMS (VFS and infrastructure)" , "open list:CONTROL GROUP (CGROUP)" , "open list:BPF (Safe dynamic programs and tools)" , "open list:MEMORY MANAGEMENT" From: Topi Miettinen Message-ID: <41b6ca51-1358-0fd7-b45a-dc29a1344865@gmail.com> Date: Fri, 15 Jul 2016 16:35:33 +0000 MIME-Version: 1.0 In-Reply-To: <20160715130458.GB21685@350D> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: On 07/15/16 13:04, Balbir Singh wrote: > On Fri, Jul 15, 2016 at 01:35:47PM +0300, Topi Miettinen wrote: >> Hello, >> >> There are many basic ways to control processes, including capabilities, >> cgroups and resource limits. However, there are far fewer ways to find out >> useful values for the limits, except blind trial and error. >> >> This patch series attempts to fix that by giving at least a nice starting >> point from the highwater mark values of the resources in question. >> I looked where each limit is checked and added a call to update the mark >> nearby. >> >> Example run of program from Documentation/accounting/getdelauys.c: >> >> ./getdelays -R -p `pidof smartd` >> printing resource accounting >> RLIMIT_CPU=0 >> RLIMIT_FSIZE=0 >> RLIMIT_DATA=18198528 >> RLIMIT_STACK=135168 >> RLIMIT_CORE=0 >> RLIMIT_RSS=0 >> RLIMIT_NPROC=1 >> RLIMIT_NOFILE=55 >> RLIMIT_MEMLOCK=0 >> RLIMIT_AS=130879488 >> RLIMIT_LOCKS=0 >> RLIMIT_SIGPENDING=0 >> RLIMIT_MSGQUEUE=0 >> RLIMIT_NICE=0 >> RLIMIT_RTPRIO=0 >> RLIMIT_RTTIME=0 >> >> ./getdelays -R -C /sys/fs/cgroup/systemd/system.slice/smartd.service/ >> printing resource accounting >> sleeping 1, blocked 0, running 0, stopped 0, uninterruptible 0 >> RLIMIT_CPU=0 >> RLIMIT_FSIZE=0 >> RLIMIT_DATA=18198528 >> RLIMIT_STACK=135168 >> RLIMIT_CORE=0 >> RLIMIT_RSS=0 >> RLIMIT_NPROC=1 >> RLIMIT_NOFILE=55 >> RLIMIT_MEMLOCK=0 >> RLIMIT_AS=130879488 >> RLIMIT_LOCKS=0 >> RLIMIT_SIGPENDING=0 >> RLIMIT_MSGQUEUE=0 >> RLIMIT_NICE=0 >> RLIMIT_RTPRIO=0 >> RLIMIT_RTTIME=0 > > Does this mean that rlimit_data and rlimit_stack should be set to the > values as specified by the data above? My plan is that either system administrator, distro maintainer or even upstream developer can get reasonable values for the limits. They may still be wrong, but things would be better than without any help to configure the system. > > Do we expect a smart user space daemon to then tweak the RLIMIT values? Someone could write an autotuning daemon that checks if the system has changed (for example due to upgrade) and then run some tests to reconfigure the system. But the limits are a bit too fragile, or rather, applications can't handle failure, so I don't know if that would really work. -Topi > > Balbir Singh. > -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x241.google.com (mail-wm0-x241.google.com [IPv6:2a00:1450:400c:c09::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3rrdVB3D68zDqF6 for ; Sat, 16 Jul 2016 02:35:46 +1000 (AEST) Received: by mail-wm0-x241.google.com with SMTP id i5so2839027wmg.2 for ; Fri, 15 Jul 2016 09:35:46 -0700 (PDT) Subject: Re: [PATCH 00/14] Present useful limits to user (v2) To: bsingharora@gmail.com References: <1468578983-28229-1-git-send-email-toiwoton@gmail.com> <20160715130458.GB21685@350D> Cc: linux-kernel@vger.kernel.org, Jonathan Corbet , Tony Luck , Fenghua Yu , Alexander Graf , Paolo Bonzini , =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" , Doug Ledford , Sean Hefty , Hal Rosenstock , Mike Marciniszyn , Dennis Dalessandro , Christian Benvenuti , Dave Goodell , Sudeep Dutt , Ashutosh Dixit , Alex Williamson , Alexander Viro , Tejun Heo , Li Zefan , Johannes Weiner , Peter Zijlstra , Alexei Starovoitov , Arnaldo Carvalho de Melo , Alexander Shishkin , Markus Elfring , "David S. Miller" , Nicolas Dichtel , Andrew Morton , Konstantin Khlebnikov , Jiri Slaby , Cyrill Gorcunov , Michal Hocko , Vlastimil Babka , Dave Hansen , Greg Kroah-Hartman , Dan Carpenter , Michael Kerrisk , "Kirill A. Shutemov" , Marcus Gelderie , Vladimir Davydov , Joe Perches , Frederic Weisbecker , Andrea Arcangeli , "Eric W. Biederman" , Andi Kleen , Oleg Nesterov , Stas Sergeev , Amanieu d'Antras , Richard Weinberger , Wang Xiaoqiang , Helge Deller , Mateusz Guzik , Alex Thorlton , Ben Segall , John Stultz , Rik van Riel , Eric B Munson , Alexey Klimov , Chen Gang , Andrey Ryabinin , David Rientjes , Hugh Dickins , Alexander Kuleshov , "open list:DOCUMENTATION" , "open list:IA64 (Itanium) PLATFORM" , "open list:KERNEL VIRTUAL MACHINE (KVM) FOR POWERPC" , "open list:KERNEL VIRTUAL MACHINE (KVM)" , "open list:LINUX FOR POWERPC (32-BIT AND 64-BIT)" , "open list:INFINIBAND SUBSYSTEM" , "open list:FILESYSTEMS (VFS and infrastructure)" , "open list:CONTROL GROUP (CGROUP)" , "open list:BPF (Safe dynamic programs and tools)" , "open list:MEMORY MANAGEMENT" From: Topi Miettinen Message-ID: <41b6ca51-1358-0fd7-b45a-dc29a1344865@gmail.com> Date: Fri, 15 Jul 2016 16:35:33 +0000 MIME-Version: 1.0 In-Reply-To: <20160715130458.GB21685@350D> Content-Type: text/plain; charset=windows-1252 List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On 07/15/16 13:04, Balbir Singh wrote: > On Fri, Jul 15, 2016 at 01:35:47PM +0300, Topi Miettinen wrote: >> Hello, >> >> There are many basic ways to control processes, including capabilities, >> cgroups and resource limits. However, there are far fewer ways to find out >> useful values for the limits, except blind trial and error. >> >> This patch series attempts to fix that by giving at least a nice starting >> point from the highwater mark values of the resources in question. >> I looked where each limit is checked and added a call to update the mark >> nearby. >> >> Example run of program from Documentation/accounting/getdelauys.c: >> >> ./getdelays -R -p `pidof smartd` >> printing resource accounting >> RLIMIT_CPU=0 >> RLIMIT_FSIZE=0 >> RLIMIT_DATA=18198528 >> RLIMIT_STACK=135168 >> RLIMIT_CORE=0 >> RLIMIT_RSS=0 >> RLIMIT_NPROC=1 >> RLIMIT_NOFILE=55 >> RLIMIT_MEMLOCK=0 >> RLIMIT_AS=130879488 >> RLIMIT_LOCKS=0 >> RLIMIT_SIGPENDING=0 >> RLIMIT_MSGQUEUE=0 >> RLIMIT_NICE=0 >> RLIMIT_RTPRIO=0 >> RLIMIT_RTTIME=0 >> >> ./getdelays -R -C /sys/fs/cgroup/systemd/system.slice/smartd.service/ >> printing resource accounting >> sleeping 1, blocked 0, running 0, stopped 0, uninterruptible 0 >> RLIMIT_CPU=0 >> RLIMIT_FSIZE=0 >> RLIMIT_DATA=18198528 >> RLIMIT_STACK=135168 >> RLIMIT_CORE=0 >> RLIMIT_RSS=0 >> RLIMIT_NPROC=1 >> RLIMIT_NOFILE=55 >> RLIMIT_MEMLOCK=0 >> RLIMIT_AS=130879488 >> RLIMIT_LOCKS=0 >> RLIMIT_SIGPENDING=0 >> RLIMIT_MSGQUEUE=0 >> RLIMIT_NICE=0 >> RLIMIT_RTPRIO=0 >> RLIMIT_RTTIME=0 > > Does this mean that rlimit_data and rlimit_stack should be set to the > values as specified by the data above? My plan is that either system administrator, distro maintainer or even upstream developer can get reasonable values for the limits. They may still be wrong, but things would be better than without any help to configure the system. > > Do we expect a smart user space daemon to then tweak the RLIMIT values? Someone could write an autotuning daemon that checks if the system has changed (for example due to upgrade) and then run some tests to reconfigure the system. But the limits are a bit too fragile, or rather, applications can't handle failure, so I don't know if that would really work. -Topi > > Balbir Singh. >