* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer @ 2021-01-25 10:24 Xinpeng Liu 2021-01-25 10:24 ` [LTP] [PATCH 2/2] syscalls/ioctl: ioctl_sg01.c:TBROK: Test killed! Xinpeng Liu ` (2 more replies) 0 siblings, 3 replies; 25+ messages in thread From: Xinpeng Liu @ 2021-01-25 10:24 UTC (permalink / raw) To: ltp Kernel version is 5.4.81+,the available RAM is less than free,as follow: [root@liuxp mywork]# head /proc/meminfo MemTotal: 198101744 kB MemFree: 189303148 kB MemAvailable: 188566732 kB So use available RAM to avoid OOM killer. Signed-off-by: Xinpeng Liu <liuxp11@chinatelecom.cn> --- lib/tst_memutils.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/lib/tst_memutils.c b/lib/tst_memutils.c index dd09db4..21df9a8 100644 --- a/lib/tst_memutils.c +++ b/lib/tst_memutils.c @@ -10,14 +10,33 @@ #define TST_NO_DEFAULT_MAIN #include "tst_test.h" +#include "tst_safe_stdio.h" #define BLOCKSIZE (16 * 1024 * 1024) +static unsigned long get_available_ram(void) +{ + char buf[60]; /* actual lines we expect are ~30 chars or less */ + unsigned long available_kb = 0; + FILE *fp; + + fp = SAFE_FOPEN("/proc/meminfo","r"); + while (fgets(buf, sizeof(buf), fp)) { + if (sscanf(buf, "MemAvailable: %lu %*s\n", &available_kb) == 1){ + break; + } + } + SAFE_FCLOSE(fp); + + return 1024 * available_kb; +} + void tst_pollute_memory(size_t maxsize, int fillchar) { size_t i, map_count = 0, safety = 0, blocksize = BLOCKSIZE; void **map_blocks; struct sysinfo info; + unsigned long available_ram = get_available_ram(); SAFE_SYSINFO(&info); safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); @@ -26,15 +45,19 @@ void tst_pollute_memory(size_t maxsize, int fillchar) if (info.freeswap > safety) safety = 0; + /*"MemAvailable" field maybe not exist, or freeram less than available_ram*/ + if(available_ram == 0 || info.freeram < available_ram) + available_ram = info.freeram; + /* Not enough free memory to avoid invoking OOM killer */ - if (info.freeram <= safety) + if (available_ram <= safety) return; if (!maxsize) maxsize = SIZE_MAX; - if (info.freeram - safety < maxsize / info.mem_unit) - maxsize = (info.freeram - safety) * info.mem_unit; + if (available_ram - safety < maxsize / info.mem_unit) + maxsize = (available_ram - safety) * info.mem_unit; blocksize = MIN(maxsize, blocksize); map_count = maxsize / blocksize; -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [LTP] [PATCH 2/2] syscalls/ioctl: ioctl_sg01.c:TBROK: Test killed! 2021-01-25 10:24 [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Xinpeng Liu @ 2021-01-25 10:24 ` Xinpeng Liu 2021-01-27 4:41 ` Li Wang 2021-01-27 4:27 ` [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Li Wang 2021-01-27 9:39 ` Cyril Hrubis 2 siblings, 1 reply; 25+ messages in thread From: Xinpeng Liu @ 2021-01-25 10:24 UTC (permalink / raw) To: ltp [root@test ioctl]# ./ioctl_sg01 tst_test.c:1263: TINFO: Timeout per run is 0h 05m 00s ioctl_sg01.c:81: TINFO: Found SCSI device /dev/sg2 Test timeouted, sending SIGKILL! Test timeouted, sending SIGKILL! Test timeouted, sending SIGKILL! Test timeouted, sending SIGKILL! tst_test.c:1308: TINFO: If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1 tst_test.c:1309: TBROK: Test killed! (timeout?) [root@test ioctl]# lscpu |grep Model Model: 85 Model name: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz [root@test ioctl]# free -h total used free shared buff/cache available Mem: 502G 2.7G 499G 82M 820M 497G Swap: 0B 0B 0B 512G RAM is too common,so add timeout 600 seconds,result is TPASS. Signed-off-by: Xinpeng Liu <liuxp11@chinatelecom.cn> --- testcases/kernel/syscalls/ioctl/ioctl_sg01.c | 1 + 1 file changed, 1 insertion(+) diff --git a/testcases/kernel/syscalls/ioctl/ioctl_sg01.c b/testcases/kernel/syscalls/ioctl/ioctl_sg01.c index 94b30dc..d1f2d3c 100644 --- a/testcases/kernel/syscalls/ioctl/ioctl_sg01.c +++ b/testcases/kernel/syscalls/ioctl/ioctl_sg01.c @@ -123,6 +123,7 @@ static void run(void) } static struct tst_test test = { + .timeout = 600, .test_all = run, .setup = setup, .cleanup = cleanup, -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [LTP] [PATCH 2/2] syscalls/ioctl: ioctl_sg01.c:TBROK: Test killed! 2021-01-25 10:24 ` [LTP] [PATCH 2/2] syscalls/ioctl: ioctl_sg01.c:TBROK: Test killed! Xinpeng Liu @ 2021-01-27 4:41 ` Li Wang 0 siblings, 0 replies; 25+ messages in thread From: Li Wang @ 2021-01-27 4:41 UTC (permalink / raw) To: ltp On Wed, Jan 27, 2021 at 11:33 AM Xinpeng Liu <liuxp11@chinatelecom.cn> wrote: > [root@test ioctl]# ./ioctl_sg01 > tst_test.c:1263: TINFO: Timeout per run is 0h 05m 00s > ioctl_sg01.c:81: TINFO: Found SCSI device /dev/sg2 > Test timeouted, sending SIGKILL! > Test timeouted, sending SIGKILL! > Test timeouted, sending SIGKILL! > Test timeouted, sending SIGKILL! > tst_test.c:1308: TINFO: If you are running on slow machine, try exporting > LTP_TIMEOUT_MUL > 1 > tst_test.c:1309: TBROK: Test killed! (timeout?) > > [root@test ioctl]# lscpu |grep Model > Model: 85 > Model name: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz > > [root@test ioctl]# free -h > total used free shared buff/cache available > Mem: 502G 2.7G 499G 82M 820M 497G > Swap: 0B 0B 0B > > 512G RAM is too common,so add timeout 600 seconds,result is TPASS. > > Signed-off-by: Xinpeng Liu <liuxp11@chinatelecom.cn> > --- > testcases/kernel/syscalls/ioctl/ioctl_sg01.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/testcases/kernel/syscalls/ioctl/ioctl_sg01.c > b/testcases/kernel/syscalls/ioctl/ioctl_sg01.c > index 94b30dc..d1f2d3c 100644 > --- a/testcases/kernel/syscalls/ioctl/ioctl_sg01.c > +++ b/testcases/kernel/syscalls/ioctl/ioctl_sg01.c > @@ -123,6 +123,7 @@ static void run(void) > } > > static struct tst_test test = { > + .timeout = 600, > It's hard to say 600s is proper for other huge systems. What about setting .timeout = -1 to disable it like what we did for the oom0* series test: https://github.com/linux-test-project/ltp/blob/master/lib/tst_test.c#L1251 -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/f4afeaf0/attachment.htm> ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-25 10:24 [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Xinpeng Liu 2021-01-25 10:24 ` [LTP] [PATCH 2/2] syscalls/ioctl: ioctl_sg01.c:TBROK: Test killed! Xinpeng Liu @ 2021-01-27 4:27 ` Li Wang 2021-01-27 5:42 ` liuxp11 2021-01-27 6:54 ` liuxp11 2021-01-27 9:39 ` Cyril Hrubis 2 siblings, 2 replies; 25+ messages in thread From: Li Wang @ 2021-01-27 4:27 UTC (permalink / raw) To: ltp Hi Xinpeng, On Wed, Jan 27, 2021 at 11:28 AM Xinpeng Liu <liuxp11@chinatelecom.cn> wrote: > Kernel version is 5.4.81+,the available RAM is less than free,as follow: > [root@liuxp mywork]# head /proc/meminfo > MemTotal: 198101744 kB > MemFree: 189303148 kB > MemAvailable: 188566732 kB > > So use available RAM to avoid OOM killer. > > Signed-off-by: Xinpeng Liu <liuxp11@chinatelecom.cn> > --- > lib/tst_memutils.c | 29 ++++++++++++++++++++++++++--- > 1 file changed, 26 insertions(+), 3 deletions(-) > > diff --git a/lib/tst_memutils.c b/lib/tst_memutils.c > index dd09db4..21df9a8 100644 > --- a/lib/tst_memutils.c > +++ b/lib/tst_memutils.c > @@ -10,14 +10,33 @@ > > #define TST_NO_DEFAULT_MAIN > #include "tst_test.h" > +#include "tst_safe_stdio.h" > > #define BLOCKSIZE (16 * 1024 * 1024) > > +static unsigned long get_available_ram(void) > +{ > + char buf[60]; /* actual lines we expect are ~30 chars or less */ > + unsigned long available_kb = 0; > + FILE *fp; > + > + fp = SAFE_FOPEN("/proc/meminfo","r"); > + while (fgets(buf, sizeof(buf), fp)) { > + if (sscanf(buf, "MemAvailable: %lu %*s\n", &available_kb) > == 1){ > + break; > + } > + } > + SAFE_FCLOSE(fp); > + > + return 1024 * available_kb; > +} > + > void tst_pollute_memory(size_t maxsize, int fillchar) > { > size_t i, map_count = 0, safety = 0, blocksize = BLOCKSIZE; > void **map_blocks; > struct sysinfo info; > + unsigned long available_ram = get_available_ram(); > LTP provides SAFE_READ_MEMINFO() macro to be used in /proc/meminfo reading. See: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85 > > SAFE_SYSINFO(&info); > safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); > @@ -26,15 +45,19 @@ void tst_pollute_memory(size_t maxsize, int fillchar) > if (info.freeswap > safety) > safety = 0; > > + /*"MemAvailable" field maybe not exist, or freeram less than > available_ram*/ > + if(available_ram == 0 || info.freeram < available_ram) > + available_ram = info.freeram; > + > /* Not enough free memory to avoid invoking OOM killer */ > - if (info.freeram <= safety) > + if (available_ram <= safety) > return; > > if (!maxsize) > maxsize = SIZE_MAX; > > - if (info.freeram - safety < maxsize / info.mem_unit) > - maxsize = (info.freeram - safety) * info.mem_unit; > + if (available_ram - safety < maxsize / info.mem_unit) > + maxsize = (available_ram - safety) * info.mem_unit; > > blocksize = MIN(maxsize, blocksize); > map_count = maxsize / blocksize; > -- > 1.8.3.1 > > > -- > Mailing list info: https://lists.linux.it/listinfo/ltp > > -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/e340dd17/attachment.htm> ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 4:27 ` [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Li Wang @ 2021-01-27 5:42 ` liuxp11 2021-01-27 6:54 ` liuxp11 1 sibling, 0 replies; 25+ messages in thread From: liuxp11 @ 2021-01-27 5:42 UTC (permalink / raw) To: ltp ok?thanks for your direction! From: Li Wang Date: 2021-01-27 12:27 To: Xinpeng Liu CC: LTP List Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Hi Xinpeng, On Wed, Jan 27, 2021 at 11:28 AM Xinpeng Liu <liuxp11@chinatelecom.cn> wrote: Kernel version is 5.4.81+,the available RAM is less than free,as follow: [root@liuxp mywork]# head /proc/meminfo MemTotal: 198101744 kB MemFree: 189303148 kB MemAvailable: 188566732 kB So use available RAM to avoid OOM killer. Signed-off-by: Xinpeng Liu <liuxp11@chinatelecom.cn> --- lib/tst_memutils.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/lib/tst_memutils.c b/lib/tst_memutils.c index dd09db4..21df9a8 100644 --- a/lib/tst_memutils.c +++ b/lib/tst_memutils.c @@ -10,14 +10,33 @@ #define TST_NO_DEFAULT_MAIN #include "tst_test.h" +#include "tst_safe_stdio.h" #define BLOCKSIZE (16 * 1024 * 1024) +static unsigned long get_available_ram(void) +{ + char buf[60]; /* actual lines we expect are ~30 chars or less */ + unsigned long available_kb = 0; + FILE *fp; + + fp = SAFE_FOPEN("/proc/meminfo","r"); + while (fgets(buf, sizeof(buf), fp)) { + if (sscanf(buf, "MemAvailable: %lu %*s\n", &available_kb) == 1){ + break; + } + } + SAFE_FCLOSE(fp); + + return 1024 * available_kb; +} + void tst_pollute_memory(size_t maxsize, int fillchar) { size_t i, map_count = 0, safety = 0, blocksize = BLOCKSIZE; void **map_blocks; struct sysinfo info; + unsigned long available_ram = get_available_ram(); LTP provides SAFE_READ_MEMINFO() macro to be used in /proc/meminfo reading. See: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85 SAFE_SYSINFO(&info); safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); @@ -26,15 +45,19 @@ void tst_pollute_memory(size_t maxsize, int fillchar) if (info.freeswap > safety) safety = 0; + /*"MemAvailable" field maybe not exist, or freeram less than available_ram*/ + if(available_ram == 0 || info.freeram < available_ram) + available_ram = info.freeram; + /* Not enough free memory to avoid invoking OOM killer */ - if (info.freeram <= safety) + if (available_ram <= safety) return; if (!maxsize) maxsize = SIZE_MAX; - if (info.freeram - safety < maxsize / info.mem_unit) - maxsize = (info.freeram - safety) * info.mem_unit; + if (available_ram - safety < maxsize / info.mem_unit) + maxsize = (available_ram - safety) * info.mem_unit; blocksize = MIN(maxsize, blocksize); map_count = maxsize / blocksize; -- 1.8.3.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/1e9d01f7/attachment-0001.htm> ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 4:27 ` [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Li Wang 2021-01-27 5:42 ` liuxp11 @ 2021-01-27 6:54 ` liuxp11 2021-01-27 7:58 ` Li Wang 1 sibling, 1 reply; 25+ messages in thread From: liuxp11 @ 2021-01-27 6:54 UTC (permalink / raw) To: ltp Hi Li, Have a question about using macro SAFE_READ_MEMINFO get MemAvailable value, Some old kernels maybe not privode "MemAvailable" field, which will broken. From: Li Wang Date: 2021-01-27 12:27 To: Xinpeng Liu CC: LTP List Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Hi Xinpeng, On Wed, Jan 27, 2021 at 11:28 AM Xinpeng Liu <liuxp11@chinatelecom.cn> wrote: Kernel version is 5.4.81+,the available RAM is less than free,as follow: [root@liuxp mywork]# head /proc/meminfo MemTotal: 198101744 kB MemFree: 189303148 kB MemAvailable: 188566732 kB So use available RAM to avoid OOM killer. Signed-off-by: Xinpeng Liu <liuxp11@chinatelecom.cn> --- lib/tst_memutils.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/lib/tst_memutils.c b/lib/tst_memutils.c index dd09db4..21df9a8 100644 --- a/lib/tst_memutils.c +++ b/lib/tst_memutils.c @@ -10,14 +10,33 @@ #define TST_NO_DEFAULT_MAIN #include "tst_test.h" +#include "tst_safe_stdio.h" #define BLOCKSIZE (16 * 1024 * 1024) +static unsigned long get_available_ram(void) +{ + char buf[60]; /* actual lines we expect are ~30 chars or less */ + unsigned long available_kb = 0; + FILE *fp; + + fp = SAFE_FOPEN("/proc/meminfo","r"); + while (fgets(buf, sizeof(buf), fp)) { + if (sscanf(buf, "MemAvailable: %lu %*s\n", &available_kb) == 1){ + break; + } + } + SAFE_FCLOSE(fp); + + return 1024 * available_kb; +} + void tst_pollute_memory(size_t maxsize, int fillchar) { size_t i, map_count = 0, safety = 0, blocksize = BLOCKSIZE; void **map_blocks; struct sysinfo info; + unsigned long available_ram = get_available_ram(); LTP provides SAFE_READ_MEMINFO() macro to be used in /proc/meminfo reading. See: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85 SAFE_SYSINFO(&info); safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); @@ -26,15 +45,19 @@ void tst_pollute_memory(size_t maxsize, int fillchar) if (info.freeswap > safety) safety = 0; + /*"MemAvailable" field maybe not exist, or freeram less than available_ram*/ + if(available_ram == 0 || info.freeram < available_ram) + available_ram = info.freeram; + /* Not enough free memory to avoid invoking OOM killer */ - if (info.freeram <= safety) + if (available_ram <= safety) return; if (!maxsize) maxsize = SIZE_MAX; - if (info.freeram - safety < maxsize / info.mem_unit) - maxsize = (info.freeram - safety) * info.mem_unit; + if (available_ram - safety < maxsize / info.mem_unit) + maxsize = (available_ram - safety) * info.mem_unit; blocksize = MIN(maxsize, blocksize); map_count = maxsize / blocksize; -- 1.8.3.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/bed16118/attachment-0001.htm> ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 6:54 ` liuxp11 @ 2021-01-27 7:58 ` Li Wang 2021-01-27 8:05 ` Li Wang 2021-01-27 8:49 ` liuxp11 0 siblings, 2 replies; 25+ messages in thread From: Li Wang @ 2021-01-27 7:58 UTC (permalink / raw) To: ltp Hi Xinpeng, I sent to you the case swapping01 solving this(via FILE_LINES_SCANF) already, feel free to take an reference: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85 On Wed, Jan 27, 2021 at 2:54 PM liuxp11@chinatelecom.cn < liuxp11@chinatelecom.cn> wrote: > Hi Li, > Have a question about using macro SAFE_READ_MEMINFO get MemAvailable > value, > Some old kernels maybe not privode "MemAvailable" field, which will > broken. > > > *From:* Li Wang <liwang@redhat.com> > *Date:* 2021-01-27 12:27 > *To:* Xinpeng Liu <liuxp11@chinatelecom.cn> > *CC:* LTP List <ltp@lists.linux.it> > *Subject:* Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 > invoked oom-killer > Hi Xinpeng, > > On Wed, Jan 27, 2021 at 11:28 AM Xinpeng Liu <liuxp11@chinatelecom.cn> > wrote: > >> Kernel version is 5.4.81+,the available RAM is less than free,as follow: >> [root@liuxp mywork]# head /proc/meminfo >> MemTotal: 198101744 kB >> MemFree: 189303148 kB >> MemAvailable: 188566732 kB >> >> So use available RAM to avoid OOM killer. >> >> Signed-off-by: Xinpeng Liu <liuxp11@chinatelecom.cn> >> --- >> lib/tst_memutils.c | 29 ++++++++++++++++++++++++++--- >> 1 file changed, 26 insertions(+), 3 deletions(-) >> >> diff --git a/lib/tst_memutils.c b/lib/tst_memutils.c >> index dd09db4..21df9a8 100644 >> --- a/lib/tst_memutils.c >> +++ b/lib/tst_memutils.c >> @@ -10,14 +10,33 @@ >> >> #define TST_NO_DEFAULT_MAIN >> #include "tst_test.h" >> +#include "tst_safe_stdio.h" >> >> #define BLOCKSIZE (16 * 1024 * 1024) >> >> +static unsigned long get_available_ram(void) >> +{ >> + char buf[60]; /* actual lines we expect are ~30 chars or less */ >> + unsigned long available_kb = 0; >> + FILE *fp; >> + >> + fp = SAFE_FOPEN("/proc/meminfo","r"); >> + while (fgets(buf, sizeof(buf), fp)) { >> + if (sscanf(buf, "MemAvailable: %lu %*s\n", &available_kb) >> == 1){ >> + break; >> + } >> + } >> + SAFE_FCLOSE(fp); >> + >> + return 1024 * available_kb; >> +} >> + >> void tst_pollute_memory(size_t maxsize, int fillchar) >> { >> size_t i, map_count = 0, safety = 0, blocksize = BLOCKSIZE; >> void **map_blocks; >> struct sysinfo info; >> + unsigned long available_ram = get_available_ram(); >> > > LTP provides SAFE_READ_MEMINFO() macro to be used in /proc/meminfo reading. > See: > https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85 > > > >> >> SAFE_SYSINFO(&info); >> safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * >> 1024); >> @@ -26,15 +45,19 @@ void tst_pollute_memory(size_t maxsize, int fillchar) >> if (info.freeswap > safety) >> safety = 0; >> >> + /*"MemAvailable" field maybe not exist, or freeram less than >> available_ram*/ >> + if(available_ram == 0 || info.freeram < available_ram) >> + available_ram = info.freeram; >> + >> /* Not enough free memory to avoid invoking OOM killer */ >> - if (info.freeram <= safety) >> + if (available_ram <= safety) >> return; >> >> if (!maxsize) >> maxsize = SIZE_MAX; >> >> - if (info.freeram - safety < maxsize / info.mem_unit) >> - maxsize = (info.freeram - safety) * info.mem_unit; >> + if (available_ram - safety < maxsize / info.mem_unit) >> + maxsize = (available_ram - safety) * info.mem_unit; >> >> blocksize = MIN(maxsize, blocksize); >> map_count = maxsize / blocksize; >> -- >> 1.8.3.1 >> >> >> -- >> Mailing list info: https://lists.linux.it/listinfo/ltp >> >> > > -- > Regards, > Li Wang > > -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/d99fe5a0/attachment.htm> ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 7:58 ` Li Wang @ 2021-01-27 8:05 ` Li Wang 2021-01-27 9:24 ` Cyril Hrubis 2021-01-27 8:49 ` liuxp11 1 sibling, 1 reply; 25+ messages in thread From: Li Wang @ 2021-01-27 8:05 UTC (permalink / raw) To: ltp Li Wang <liwang@redhat.com> wrote: Hi Xinpeng, > > I sent to you the case swapping01 solving this(via FILE_LINES_SCANF) > already, feel free to take an reference: > > https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85 > Or, maybe we can extract this process into a function and put it in tst_memutils.h, to convinently reuse by other testcases too? void tst_get_MemAvailable(void); -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/3b37c2f4/attachment.htm> ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 8:05 ` Li Wang @ 2021-01-27 9:24 ` Cyril Hrubis 2021-01-27 10:03 ` Li Wang 0 siblings, 1 reply; 25+ messages in thread From: Cyril Hrubis @ 2021-01-27 9:24 UTC (permalink / raw) To: ltp Hi! > > I sent to you the case swapping01 solving this(via FILE_LINES_SCANF) > > already, feel free to take an reference: > > > > https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85 > > > > Or, maybe we can extract this process into a function and put it in > tst_memutils.h, to convinently reuse by other testcases too? > > void tst_get_MemAvailable(void); Please do not use CamelCase. It should be tst_get_mem_available(void) and it should also return unsigned long. -- Cyril Hrubis chrubis@suse.cz ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 9:24 ` Cyril Hrubis @ 2021-01-27 10:03 ` Li Wang 0 siblings, 0 replies; 25+ messages in thread From: Li Wang @ 2021-01-27 10:03 UTC (permalink / raw) To: ltp On Wed, Jan 27, 2021 at 5:23 PM Cyril Hrubis <chrubis@suse.cz> wrote: > Hi! > > > I sent to you the case swapping01 solving this(via FILE_LINES_SCANF) > > > already, feel free to take an reference: > > > > > > > https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85 > > > > > > > Or, maybe we can extract this process into a function and put it in > > tst_memutils.h, to convinently reuse by other testcases too? > > > > void tst_get_MemAvailable(void); > > Please do not use CamelCase. > +1 Sorry, I just pasted the name by the editor, and YES, we should avoid that. > It should be tst_get_mem_available(void) and it should also return > unsigned long. > Absolutely right. liuxp11@chinatelecom.cn <liuxp11@chinatelecom.cn> wrote: In this testcase,we first check MemAvailable. If MemAvailable doesn't > exist,then use info.freeram. > Maybe not other cases need do these. > Yes, but we also could make use of tst_get_mem_available() here because, in the patch, you're trying to avoid that (MemFree > MemAvailable) situation. If the fix is correct, we just need to get both and choose the smaller one to use, isn't it? > Have a question about using macro SAFE_READ_MEMINFO get MemAvailable >> value, >> Some old kernels maybe not privode "MemAvailable" field, which will >> broken. >> > The most different of SAFE_* macro is that will exit with TBROK if not get it expected. As Cyril proposes another one FILE_LINES_SCANF has no such concern. -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/22cc467d/attachment-0001.htm> ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 7:58 ` Li Wang 2021-01-27 8:05 ` Li Wang @ 2021-01-27 8:49 ` liuxp11 2021-01-27 9:50 ` Cyril Hrubis 1 sibling, 1 reply; 25+ messages in thread From: liuxp11 @ 2021-01-27 8:49 UTC (permalink / raw) To: ltp In this testcase,we first check MemAvailable. If MemAvailable doesn't exist,then use info.freeram. Maybe not other cases need do these. From: Li Wang Date: 2021-01-27 15:58 To: liuxp11@chinatelecom.cn CC: ltp Subject: Re: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Hi Xinpeng, I sent to you the case swapping01 solving this(via FILE_LINES_SCANF) already, feel free to take an reference: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85 On Wed, Jan 27, 2021 at 2:54 PM liuxp11@chinatelecom.cn <liuxp11@chinatelecom.cn> wrote: Hi Li, Have a question about using macro SAFE_READ_MEMINFO get MemAvailable value, Some old kernels maybe not privode "MemAvailable" field, which will broken. From: Li Wang Date: 2021-01-27 12:27 To: Xinpeng Liu CC: LTP List Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Hi Xinpeng, On Wed, Jan 27, 2021 at 11:28 AM Xinpeng Liu <liuxp11@chinatelecom.cn> wrote: Kernel version is 5.4.81+,the available RAM is less than free,as follow: [root@liuxp mywork]# head /proc/meminfo MemTotal: 198101744 kB MemFree: 189303148 kB MemAvailable: 188566732 kB So use available RAM to avoid OOM killer. Signed-off-by: Xinpeng Liu <liuxp11@chinatelecom.cn> --- lib/tst_memutils.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/lib/tst_memutils.c b/lib/tst_memutils.c index dd09db4..21df9a8 100644 --- a/lib/tst_memutils.c +++ b/lib/tst_memutils.c @@ -10,14 +10,33 @@ #define TST_NO_DEFAULT_MAIN #include "tst_test.h" +#include "tst_safe_stdio.h" #define BLOCKSIZE (16 * 1024 * 1024) +static unsigned long get_available_ram(void) +{ + char buf[60]; /* actual lines we expect are ~30 chars or less */ + unsigned long available_kb = 0; + FILE *fp; + + fp = SAFE_FOPEN("/proc/meminfo","r"); + while (fgets(buf, sizeof(buf), fp)) { + if (sscanf(buf, "MemAvailable: %lu %*s\n", &available_kb) == 1){ + break; + } + } + SAFE_FCLOSE(fp); + + return 1024 * available_kb; +} + void tst_pollute_memory(size_t maxsize, int fillchar) { size_t i, map_count = 0, safety = 0, blocksize = BLOCKSIZE; void **map_blocks; struct sysinfo info; + unsigned long available_ram = get_available_ram(); LTP provides SAFE_READ_MEMINFO() macro to be used in /proc/meminfo reading. See: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85 SAFE_SYSINFO(&info); safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); @@ -26,15 +45,19 @@ void tst_pollute_memory(size_t maxsize, int fillchar) if (info.freeswap > safety) safety = 0; + /*"MemAvailable" field maybe not exist, or freeram less than available_ram*/ + if(available_ram == 0 || info.freeram < available_ram) + available_ram = info.freeram; + /* Not enough free memory to avoid invoking OOM killer */ - if (info.freeram <= safety) + if (available_ram <= safety) return; if (!maxsize) maxsize = SIZE_MAX; - if (info.freeram - safety < maxsize / info.mem_unit) - maxsize = (info.freeram - safety) * info.mem_unit; + if (available_ram - safety < maxsize / info.mem_unit) + maxsize = (available_ram - safety) * info.mem_unit; blocksize = MIN(maxsize, blocksize); map_count = maxsize / blocksize; -- 1.8.3.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp -- Regards, Li Wang -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/f3f4d4d7/attachment.htm> ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 8:49 ` liuxp11 @ 2021-01-27 9:50 ` Cyril Hrubis 0 siblings, 0 replies; 25+ messages in thread From: Cyril Hrubis @ 2021-01-27 9:50 UTC (permalink / raw) To: ltp > Signed-off-by: Xinpeng Liu <liuxp11@chinatelecom.cn> > --- > lib/tst_memutils.c | 29 ++++++++++++++++++++++++++--- > 1 file changed, 26 insertions(+), 3 deletions(-) > > diff --git a/lib/tst_memutils.c b/lib/tst_memutils.c > index dd09db4..21df9a8 100644 > --- a/lib/tst_memutils.c > +++ b/lib/tst_memutils.c > @@ -10,14 +10,33 @@ > > #define TST_NO_DEFAULT_MAIN > #include "tst_test.h" > +#include "tst_safe_stdio.h" > > #define BLOCKSIZE (16 * 1024 * 1024) > > +static unsigned long get_available_ram(void) > +{ Can we prefix this function with tst_ and make it non-static? I guess that there may be other tests that may use it later on. > + char buf[60]; /* actual lines we expect are ~30 chars or less */ > + unsigned long available_kb = 0; > + FILE *fp; > + > + fp = SAFE_FOPEN("/proc/meminfo","r"); > + while (fgets(buf, sizeof(buf), fp)) { > + if (sscanf(buf, "MemAvailable: %lu %*s\n", &available_kb) == 1){ > + break; > + } > + } > + SAFE_FCLOSE(fp); Just use FILE_LINES_SCANF() instead. Also we should fall back to something as 90% of (MemFree + Cached) here if MemAvailable is not present so that the function returns sensible number on older kernels as well. > + return 1024 * available_kb; Can we just return kilobytes instead? It will be less likely to overflow if we do all the calculations in kilobytes instead. -- Cyril Hrubis chrubis@suse.cz ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-25 10:24 [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Xinpeng Liu 2021-01-25 10:24 ` [LTP] [PATCH 2/2] syscalls/ioctl: ioctl_sg01.c:TBROK: Test killed! Xinpeng Liu 2021-01-27 4:27 ` [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Li Wang @ 2021-01-27 9:39 ` Cyril Hrubis 2021-01-27 9:53 ` Martin Doucha 2021-01-27 10:04 ` liuxp11 2 siblings, 2 replies; 25+ messages in thread From: Cyril Hrubis @ 2021-01-27 9:39 UTC (permalink / raw) To: ltp Hi! > MemTotal: 198101744 kB > MemFree: 189303148 kB > MemAvailable: 188566732 kB This sounds really strange, usually MemFree is smaller than MemAvailable since MemFree does not include page cache. Is this freshly booted system? Can you send the whole meminfo here please? -- Cyril Hrubis chrubis@suse.cz ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 9:39 ` Cyril Hrubis @ 2021-01-27 9:53 ` Martin Doucha 2021-01-27 10:07 ` liuxp11 2021-01-27 10:04 ` liuxp11 1 sibling, 1 reply; 25+ messages in thread From: Martin Doucha @ 2021-01-27 9:53 UTC (permalink / raw) To: ltp On 27. 01. 21 10:39, Cyril Hrubis wrote: > Hi! >> MemTotal: 198101744 kB >> MemFree: 189303148 kB >> MemAvailable: 188566732 kB > > This sounds really strange, usually MemFree is smaller than MemAvailable > since MemFree does not include page cache. > > Is this freshly booted system? > > Can you send the whole meminfo here please? Please also send the contents of /proc/sys/vm/min_free_kbytes. I suspect that that's where the OOM issues are actually coming from. -- Martin Doucha mdoucha@suse.cz QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 9:53 ` Martin Doucha @ 2021-01-27 10:07 ` liuxp11 0 siblings, 0 replies; 25+ messages in thread From: liuxp11 @ 2021-01-27 10:07 UTC (permalink / raw) To: ltp [root@test-env-nm05-compute-14e5e72e38 ~]# w 18:05:00 up 15 days, 1:06, 3 users, load average: 0.39, 0.42, 0.47 From: Martin Doucha Date: 2021-01-27 17:53 To: Cyril Hrubis; Xinpeng Liu CC: ltp Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer On 27. 01. 21 10:39, Cyril Hrubis wrote: > Hi! >> MemTotal: 198101744 kB >> MemFree: 189303148 kB >> MemAvailable: 188566732 kB > > This sounds really strange, usually MemFree is smaller than MemAvailable > since MemFree does not include page cache. > > Is this freshly booted system? > > Can you send the whole meminfo here please? Please also send the contents of /proc/sys/vm/min_free_kbytes. I suspect that that's where the OOM issues are actually coming from. -- Martin Doucha mdoucha@suse.cz QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/437569b4/attachment.htm> ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 9:39 ` Cyril Hrubis 2021-01-27 9:53 ` Martin Doucha @ 2021-01-27 10:04 ` liuxp11 2021-01-27 11:28 ` Martin Doucha 2021-03-04 7:52 ` Li Wang 1 sibling, 2 replies; 25+ messages in thread From: liuxp11 @ 2021-01-27 10:04 UTC (permalink / raw) To: ltp [root@test-env-nm05-compute-14e5e72e38 ~]# cat /proc/meminfo MemTotal: 526997420 kB MemFree: 520224908 kB MemAvailable: 519936744 kB Buffers: 0 kB Cached: 2509036 kB SwapCached: 0 kB Active: 906868 kB Inactive: 2398084 kB Active(anon): 816396 kB Inactive(anon): 77236 kB Active(file): 90472 kB Inactive(file): 2320848 kB Unevictable: 610056 kB Mlocked: 610056 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 0 kB Writeback: 0 kB AnonPages: 1406336 kB Mapped: 118628 kB Shmem: 84400 kB KReclaimable: 193752 kB Slab: 703668 kB SReclaimable: 193752 kB SUnreclaim: 509916 kB KernelStack: 12672 kB PageTables: 14056 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 263498708 kB Committed_AS: 10263760 kB VmallocTotal: 34359738367 kB VmallocUsed: 651756 kB VmallocChunk: 0 kB Percpu: 62464 kB HardwareCorrupted: 88 kB AnonHugePages: 0 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB FileHugePages: 0 kB FilePmdMapped: 0 kB CmaTotal: 0 kB CmaFree: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 0 kB DirectMap4k: 1371808 kB DirectMap2M: 49661952 kB DirectMap1G: 486539264 kB [root@test-env-nm05-compute-14e5e72e38 ~]# cat /proc/sys/vm/min_free_kbytes 90112 From: Cyril Hrubis Date: 2021-01-27 17:39 To: Xinpeng Liu CC: ltp Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Hi! > MemTotal: 198101744 kB > MemFree: 189303148 kB > MemAvailable: 188566732 kB This sounds really strange, usually MemFree is smaller than MemAvailable since MemFree does not include page cache. Is this freshly booted system? Can you send the whole meminfo here please? -- Cyril Hrubis chrubis@suse.cz -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/d67a18b3/attachment.htm> ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 10:04 ` liuxp11 @ 2021-01-27 11:28 ` Martin Doucha 2021-01-27 11:41 ` liuxp11 2021-03-04 7:52 ` Li Wang 1 sibling, 1 reply; 25+ messages in thread From: Martin Doucha @ 2021-01-27 11:28 UTC (permalink / raw) To: ltp On 27. 01. 21 11:04, liuxp11@chinatelecom.cn wrote: > [root@test-env-nm05-compute-14e5e72e38 ~]# cat ?/proc/sys/vm/min_free_kbytes > 90112 Yep, there it is. min_free_kbytes is 90MB but we only leave 64MB safety margin. I'll prepare a patch that'll increase safety margin to 2*min_free_kbytes if needed. -- Martin Doucha mdoucha@suse.cz QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 11:28 ` Martin Doucha @ 2021-01-27 11:41 ` liuxp11 2021-01-27 11:46 ` Martin Doucha 0 siblings, 1 reply; 25+ messages in thread From: liuxp11 @ 2021-01-27 11:41 UTC (permalink / raw) To: ltp safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); now safety margin is 128MB,not 64MB. Right? From: Martin Doucha Date: 2021-01-27 19:28 To: liuxp11@chinatelecom.cn; Cyril Hrubis CC: ltp Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer On 27. 01. 21 11:04, liuxp11@chinatelecom.cn wrote: > [root@test-env-nm05-compute-14e5e72e38 ~]# cat /proc/sys/vm/min_free_kbytes > 90112 Yep, there it is. min_free_kbytes is 90MB but we only leave 64MB safety margin. I'll prepare a patch that'll increase safety margin to 2*min_free_kbytes if needed. -- Martin Doucha mdoucha@suse.cz QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/43f122fb/attachment.htm> ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 11:41 ` liuxp11 @ 2021-01-27 11:46 ` Martin Doucha 2021-01-27 11:50 ` liuxp11 0 siblings, 1 reply; 25+ messages in thread From: Martin Doucha @ 2021-01-27 11:46 UTC (permalink / raw) To: ltp On 27. 01. 21 12:41, liuxp11@chinatelecom.cn wrote: > safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); > now safety margin is 128MB,not 64MB. Right? Yes, right, sorry. -- Martin Doucha mdoucha@suse.cz QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 11:46 ` Martin Doucha @ 2021-01-27 11:50 ` liuxp11 0 siblings, 0 replies; 25+ messages in thread From: liuxp11 @ 2021-01-27 11:50 UTC (permalink / raw) To: ltp available memory can avoid to oom-killer. #man free available Estimation of how much memory is available for starting new applications, without swapping. Unlike the data provided by the cache or free fields, this field takes into account page cache and also that not all reclaimable memory slabs will be reclaimed due to items being in use (MemAvailable in /proc/meminfo, available on kernels 3.14, emulated on kernels 2.6.27+, otherwise the same as free) [root@bogon ltp]# cat my.diff commit eb28176a3351c6854620aaa8248bf17edea210ae Author: Xinpeng Liu <liuxp11@chinatelecom.cn> Date: Mon Jan 25 20:58:20 2021 +0800 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Kernel version is 5.4.81+,the available RAM is less than free,as follow: [root@liuxp mywork]# head /proc/meminfo MemTotal: 198101744 kB MemFree: 189303148 kB MemAvailable: 188566732 kB So use available RAM to avoid OOM killer. diff --git a/include/tst_memutils.h b/include/tst_memutils.h index 91dad07..3fd70b2 100644 --- a/include/tst_memutils.h +++ b/include/tst_memutils.h @@ -6,6 +6,8 @@ #ifndef TST_MEMUTILS_H__ #define TST_MEMUTILS_H__ +unsigned long tst_get_mem_available(void); + /* * Fill up to maxsize physical memory with fillchar, then free it for reuse. * If maxsize is zero, fill as much memory as possible. This function is diff --git a/lib/tst_memutils.c b/lib/tst_memutils.c index dd09db4..9408b37 100644 --- a/lib/tst_memutils.c +++ b/lib/tst_memutils.c @@ -13,11 +13,21 @@ #define BLOCKSIZE (16 * 1024 * 1024) +unsigned long tst_get_mem_available(void) +{ + unsigned long available_kb = 0; + + FILE_LINES_SCANF("/proc/meminfo", "MemAvailable: %lu", &available_kb); + + return available_kb; +} + void tst_pollute_memory(size_t maxsize, int fillchar) { size_t i, map_count = 0, safety = 0, blocksize = BLOCKSIZE; void **map_blocks; struct sysinfo info; + unsigned long available_ram; SAFE_SYSINFO(&info); safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); @@ -26,15 +36,22 @@ void tst_pollute_memory(size_t maxsize, int fillchar) if (info.freeswap > safety) safety = 0; + available_ram = 1024 * tst_get_mem_available(); + available_ram /= info.mem_unit; + + /*"MemAvailable" field maybe not exist, or freeram less than available_ram*/ + if(available_ram == 0 || info.freeram < available_ram) + available_ram = info.freeram; + /* Not enough free memory to avoid invoking OOM killer */ - if (info.freeram <= safety) + if (available_ram <= safety) return; if (!maxsize) maxsize = SIZE_MAX; - if (info.freeram - safety < maxsize / info.mem_unit) - maxsize = (info.freeram - safety) * info.mem_unit; + if (available_ram - safety < maxsize / info.mem_unit) + maxsize = (available_ram - safety) * info.mem_unit; blocksize = MIN(maxsize, blocksize); map_count = maxsize / blocksize; From: Martin Doucha Date: 2021-01-27 19:46 To: liuxp11@chinatelecom.cn; Cyril Hrubis CC: ltp Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer On 27. 01. 21 12:41, liuxp11@chinatelecom.cn wrote: > safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); > now safety margin is 128MB,not 64MB. Right? Yes, right, sorry. -- Martin Doucha mdoucha@suse.cz QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/182cafa3/attachment.htm> ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-01-27 10:04 ` liuxp11 2021-01-27 11:28 ` Martin Doucha @ 2021-03-04 7:52 ` Li Wang 2021-03-05 5:52 ` liuxp11 1 sibling, 1 reply; 25+ messages in thread From: Li Wang @ 2021-03-04 7:52 UTC (permalink / raw) To: ltp Hi Xinpeng, [root@test-env-nm05-compute-14e5e72e38 ~]# cat /proc/meminfo > MemTotal: 526997420 kB > MemFree: 520224908 kB > MemAvailable: 519936744 kB > Buffers: 0 kB > Cached: 2509036 kB > SwapCached: 0 kB > ... > SwapTotal: 0 kB > SwapFree: 0 kB > ... > CommitLimit: 263498708 kB > Committed_AS: 10263760 kB > > [root@test-env-nm05-compute-14e5e72e38 ~]# cat > /proc/sys/vm/min_free_kbytes > 90112 > After looking back on this problem, I prefer to think the reasons were caused by lower CommitLimit. CommitLimit: 263498708 kB < MemAvailable: 519936744 kB If you try to enable all swap-disk or reset to a high ratio in overcommit_ratio to make it larger than MemAvailable, probably no OOM occurs anymore. Btw, I also observed that ioctl_sg01 almost being killed by OOM every time on an aarch64 with little swap space, but if I add more swap or set a high value of overcommit_ratio, the problem is gone. (I manually tried with another x86_64 to confirm this too) total used free shared buff/cache available Mem: 259828 5365 247383 68 7079 231296 Swap: 4095 55 4040 --- MemTotal: 266063872 kB MemFree: 253320768 kB MemAvailable: 236848064 kB Buffers: 1472 kB Cached: 6755456 kB SwapCached: 12160 kB ... CommitLimit: 137226176 kB Committed_AS: 1206912 kB --- The previous method in the patch[1] seems not good enough, but that can help to verify if OOM disappears when resetting the overcommit_ratio. [1] http://lists.linux.it/pipermail/ltp/2021-February/020907.html Hence, another improvement way based on the above is to allocate proper memory-size according to CommitLimit value when detecting the value of CommitLimit is less than MemAvailable. That will make the test happy with a little swap-space size system. Any thoughts, or comments? --- a/lib/tst_memutils.c +++ b/lib/tst_memutils.c @@ -36,6 +36,13 @@ void tst_pollute_memory(size_t maxsize, int fillchar) if (info.freeram - safety < maxsize / info.mem_unit) maxsize = (info.freeram - safety) * info.mem_unit; + /* + * To respect CommitLimit to prevent test invoking OOM killer, + * this may appear on system with a smaller swap-disk (or disabled). + */ + if (SAFE_READ_MEMINFO("CommitLimit:") < SAFE_READ_MEMINFO("MemAvailable:")) + maxsize = SAFE_READ_MEMINFO("CommitLimit:") * 1024 - (safety * info.mem_unit); + blocksize = MIN(maxsize, blocksize); map_count = maxsize / blocksize; map_blocks = SAFE_MALLOC(map_count * sizeof(void *)); ======================== About the MemAvailable < MemFree, I think that is correct behavior on your system and not the OOM root-cause. Generally, we assumed the MemAvailable higher than MemFree, but we sometimes also allow situations to break that. We'd better count all of the different free watermarks from /proc/zoneinfo, then add the sum of the low watermarks to MemAvailable, if get a value larger than MemFree, that should be OK from my perspective. ----- # echo 675840 > /proc/sys/vm/min_free_kbytes # cat /proc/meminfo |grep -i mem MemTotal: 5888584 kB MemFree: 4518064 kB MemAvailable: 3692008 kB Shmem: 21128 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB # cat /proc/zoneinfo |grep low -B 3 ... pages free 3840 min 440 low 550 -- Node 0, zone DMA32 pages free 355602 min 79706 low 99632 -- Node 0, zone Normal pages free 0 min 0 low 0 -- Node 0, zone Movable pages free 0 min 0 low 0 -- Node 0, zone Device pages free 0 min 0 low 0 -- Node 1, zone DMA pages free 0 min 0 low 0 -- Node 1, zone DMA32 pages free 0 min 0 low 0 -- nr_kernel_misc_reclaimable 0 pages free 769192 min 88812 low 111015 (111015+99632+550)*4 + 3692008(MemAvailable) > 5888584(MemFree) Btw the formula to count MemAvailable is: available = MemFree - totalreserve_pages + pages[LRU_ACTIVE_FILE] + pages[LRU_INACTIVE_FILE] - min(pagecache / 2, wmark_low) -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210304/07089c65/attachment-0001.htm> ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-03-04 7:52 ` Li Wang @ 2021-03-05 5:52 ` liuxp11 2021-03-05 9:02 ` Li Wang 0 siblings, 1 reply; 25+ messages in thread From: liuxp11 @ 2021-03-05 5:52 UTC (permalink / raw) To: ltp --- a/lib/tst_memutils.c +++ b/lib/tst_memutils.c @@ -36,6 +36,13 @@ void tst_pollute_memory(size_t maxsize, int fillchar) if (info.freeram - safety < maxsize / info.mem_unit) maxsize = (info.freeram - safety) * info.mem_unit; ==>Thanks,but the maxsize original code need to be deleted,Right? + /* + * To respect CommitLimit to prevent test invoking OOM killer, + * this may appear on system with a smaller swap-disk (or disabled). + */ + if (SAFE_READ_MEMINFO("CommitLimit:") < SAFE_READ_MEMINFO("MemAvailable:")) + maxsize = SAFE_READ_MEMINFO("CommitLimit:") * 1024 - (safety * info.mem_unit); + blocksize = MIN(maxsize, blocksize); map_count = maxsize / blocksize; map_blocks = SAFE_MALLOC(map_count * sizeof(void *)); Thanks? From: Li Wang Date: 2021-03-04 15:52 To: liuxp11@chinatelecom.cn CC: Cyril Hrubis; ltp; Martin Doucha Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Hi Xinpeng, [root@test-env-nm05-compute-14e5e72e38 ~]# cat /proc/meminfo MemTotal: 526997420 kB MemFree: 520224908 kB MemAvailable: 519936744 kB Buffers: 0 kB Cached: 2509036 kB SwapCached: 0 kB ... SwapTotal: 0 kB SwapFree: 0 kB ... CommitLimit: 263498708 kB Committed_AS: 10263760 kB [root@test-env-nm05-compute-14e5e72e38 ~]# cat /proc/sys/vm/min_free_kbytes 90112 After looking back on this problem, I prefer to think the reasons were caused by lower CommitLimit. CommitLimit: 263498708 kB < MemAvailable: 519936744 kB If you try to enable all swap-disk or reset to a high ratio in overcommit_ratio to make it larger than MemAvailable, probably no OOM occurs anymore. Btw, I also observed that ioctl_sg01 almost being killed by OOM every time on an aarch64 with little swap space, but if I add more swap or set a high value of overcommit_ratio, the problem is gone. (I manually tried with another x86_64 to confirm this too) total used free shared buff/cache available Mem: 259828 5365 247383 68 7079 231296 Swap: 4095 55 4040--- MemTotal: 266063872 kB MemFree: 253320768 kB MemAvailable: 236848064 kB Buffers: 1472 kB Cached: 6755456 kB SwapCached: 12160 kB ... CommitLimit: 137226176 kB Committed_AS: 1206912 kB --- The previous method in the patch[1] seems not good enough, but that can help to verify if OOM disappears when resetting the overcommit_ratio. [1] http://lists.linux.it/pipermail/ltp/2021-February/020907.html Hence, another improvement way based on the above is to allocate proper memory-size according to CommitLimit value when detecting the value of CommitLimit is less than MemAvailable. That will make the test happy with a little swap-space size system. Any thoughts, or comments? --- a/lib/tst_memutils.c +++ b/lib/tst_memutils.c @@ -36,6 +36,13 @@ void tst_pollute_memory(size_t maxsize, int fillchar) if (info.freeram - safety < maxsize / info.mem_unit) maxsize = (info.freeram - safety) * info.mem_unit; + /* + * To respect CommitLimit to prevent test invoking OOM killer, + * this may appear on system with a smaller swap-disk (or disabled). + */ + if (SAFE_READ_MEMINFO("CommitLimit:") < SAFE_READ_MEMINFO("MemAvailable:")) + maxsize = SAFE_READ_MEMINFO("CommitLimit:") * 1024 - (safety * info.mem_unit); + blocksize = MIN(maxsize, blocksize); map_count = maxsize / blocksize; map_blocks = SAFE_MALLOC(map_count * sizeof(void *)); ======================== About the MemAvailable < MemFree, I think that is correct behavior on your system and not the OOM root-cause. Generally, we assumed the MemAvailable higher than MemFree, but we sometimes also allow situations to break that. We'd better count all of the different free watermarks from /proc/zoneinfo, then add the sum of the low watermarks to MemAvailable, if get a value larger than MemFree, that should be OK from my perspective. ----- # echo 675840 > /proc/sys/vm/min_free_kbytes # cat /proc/meminfo |grep -i mem MemTotal: 5888584 kB MemFree: 4518064 kB MemAvailable: 3692008 kB Shmem: 21128 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB # cat /proc/zoneinfo |grep low -B 3 ... pages free 3840 min 440 low 550 -- Node 0, zone DMA32 pages free 355602 min 79706 low 99632 -- Node 0, zone Normal pages free 0 min 0 low 0 -- Node 0, zone Movable pages free 0 min 0 low 0 -- Node 0, zone Device pages free 0 min 0 low 0 -- Node 1, zone DMA pages free 0 min 0 low 0 -- Node 1, zone DMA32 pages free 0 min 0 low 0 -- nr_kernel_misc_reclaimable 0 pages free 769192 min 88812 low 111015 (111015+99632+550)*4 + 3692008(MemAvailable) > 5888584(MemFree) Btw the formula to count MemAvailable is: available = MemFree - totalreserve_pages + pages[LRU_ACTIVE_FILE] + pages[LRU_INACTIVE_FILE] - min(pagecache / 2, wmark_low) -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210305/35aaf9fc/attachment-0001.htm> ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-03-05 5:52 ` liuxp11 @ 2021-03-05 9:02 ` Li Wang 2021-03-24 9:26 ` liuxp11 0 siblings, 1 reply; 25+ messages in thread From: Li Wang @ 2021-03-05 9:02 UTC (permalink / raw) To: ltp Hi Xinpeng, On Fri, Mar 5, 2021 at 1:52 PM liuxp11@chinatelecom.cn < liuxp11@chinatelecom.cn> wrote: > --- a/lib/tst_memutils.c > +++ b/lib/tst_memutils.c > @@ -36,6 +36,13 @@ void tst_pollute_memory(size_t maxsize, int fillchar) > if (info.freeram - safety < maxsize / info.mem_unit) > maxsize = (info.freeram - safety) * info.mem_unit; > > ==>Thanks,but the maxsize original code need to be deleted,Right? > No, the maxsize code is also useful, it needs to define the value in most common situations(i.e. CommitLimit > MemAvailable). But I'm still hesitating to use ComitLimit as the threshold for 'maxsize'. Because according to the Linux document, it says that only take effort when overcommit_memory is setting to 2. But our test system all set 0 by default. "This limit is only adhered to if strict overcommit accounting is enabled (mode 2 in 'vm.overcommit_memory')." see: https://github.com/torvalds/linux/blob/master/Documentation/filesystems/proc.rst Seems to use CommitLimit looks a bit strict and strange to test. And I even think the way to use MemAvailable is acceptable if MemFree > MemAvailable, just like what you did in your last patch. I'm still not very sure so far~ (But one thing I can confirm that MemAvailable < MemFree is correct behavior sometimes) > > + /* > + * To respect CommitLimit to prevent test invoking OOM killer, > + * this may appear on system with a smaller swap-disk (or > disabled). > + */ > + if (SAFE_READ_MEMINFO("CommitLimit:") < > SAFE_READ_MEMINFO("MemAvailable:")) > + maxsize = SAFE_READ_MEMINFO("CommitLimit:") * 1024 - > (safety * info.mem_unit); > + > blocksize = MIN(maxsize, blocksize); > map_count = maxsize / blocksize; > map_blocks = SAFE_MALLOC(map_count * sizeof(void *)); > -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210305/d6f661f7/attachment.htm> ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-03-05 9:02 ` Li Wang @ 2021-03-24 9:26 ` liuxp11 2021-04-11 5:01 ` Li Wang 0 siblings, 1 reply; 25+ messages in thread From: liuxp11 @ 2021-03-24 9:26 UTC (permalink / raw) To: ltp Hi Li Wang, I think your patch is good. 1.CommitLimit is the memory that can be allocated by application. 2.ioctl_sg01 in serveral machines with your patch,the result is passed. Thanks! From: Li Wang Date: 2021-03-05 17:02 To: liuxp11@chinatelecom.cn CC: Cyril Hrubis; ltp; mdoucha Subject: Re: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Hi Xinpeng, On Fri, Mar 5, 2021 at 1:52 PM liuxp11@chinatelecom.cn <liuxp11@chinatelecom.cn> wrote: --- a/lib/tst_memutils.c +++ b/lib/tst_memutils.c @@ -36,6 +36,13 @@ void tst_pollute_memory(size_t maxsize, int fillchar) if (info.freeram - safety < maxsize / info.mem_unit) maxsize = (info.freeram - safety) * info.mem_unit; ==>Thanks,but the maxsize original code need to be deleted,Right? No, the maxsize code is also useful, it needs to define the value in most common situations(i.e. CommitLimit > MemAvailable). But I'm still hesitating to use ComitLimit as the threshold for 'maxsize'. Because according to the Linux document, it says that only take effort when overcommit_memory is setting to 2. But our test system all set 0 by default. "This limit is only adhered to if strict overcommit accounting is enabled (mode 2 in 'vm.overcommit_memory')." see: https://github.com/torvalds/linux/blob/master/Documentation/filesystems/proc.rst Seems to use CommitLimit looks a bit strict and strange to test. And I even think the way to use MemAvailable is acceptable if MemFree > MemAvailable, just like what you did in your last patch. I'm still not very sure so far~ (But one thing I can confirm that MemAvailable < MemFree is correct behavior sometimes) + /* + * To respect CommitLimit to prevent test invoking OOM killer, + * this may appear on system with a smaller swap-disk (or disabled). + */ + if (SAFE_READ_MEMINFO("CommitLimit:") < SAFE_READ_MEMINFO("MemAvailable:")) + maxsize = SAFE_READ_MEMINFO("CommitLimit:") * 1024 - (safety * info.mem_unit); + blocksize = MIN(maxsize, blocksize); map_count = maxsize / blocksize; map_blocks = SAFE_MALLOC(map_count * sizeof(void *)); -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210324/39164c88/attachment.htm> ^ permalink raw reply [flat|nested] 25+ messages in thread
* [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer 2021-03-24 9:26 ` liuxp11 @ 2021-04-11 5:01 ` Li Wang 0 siblings, 0 replies; 25+ messages in thread From: Li Wang @ 2021-04-11 5:01 UTC (permalink / raw) To: ltp Hi all, Seems this ioctl_sg01 problem is caused by the kernel commit: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=8c7829b04c523cdc732cb77f59f03320e09f3386 I start a new thread for tracking it in Linux-MM just now: https://lists.linux.it/pipermail/ltp/2021-April/021903.html liuxp11@chinatelecom.cn <liuxp11@chinatelecom.cn> wrote: Hi Li Wang, > I think your patch is good. > 1.CommitLimit is the memory that can be allocated by application. > 2.ioctl_sg01 in serveral machines with your patch,the result is passed. > > Thank you Xinpeng, That is just an eclectic workaround but not the key point of the OOM occurs issue. I'd not suggest merging my patch to LTP:). -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210411/44f80803/attachment.htm> ^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2021-04-11 5:01 UTC | newest] Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-01-25 10:24 [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Xinpeng Liu 2021-01-25 10:24 ` [LTP] [PATCH 2/2] syscalls/ioctl: ioctl_sg01.c:TBROK: Test killed! Xinpeng Liu 2021-01-27 4:41 ` Li Wang 2021-01-27 4:27 ` [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Li Wang 2021-01-27 5:42 ` liuxp11 2021-01-27 6:54 ` liuxp11 2021-01-27 7:58 ` Li Wang 2021-01-27 8:05 ` Li Wang 2021-01-27 9:24 ` Cyril Hrubis 2021-01-27 10:03 ` Li Wang 2021-01-27 8:49 ` liuxp11 2021-01-27 9:50 ` Cyril Hrubis 2021-01-27 9:39 ` Cyril Hrubis 2021-01-27 9:53 ` Martin Doucha 2021-01-27 10:07 ` liuxp11 2021-01-27 10:04 ` liuxp11 2021-01-27 11:28 ` Martin Doucha 2021-01-27 11:41 ` liuxp11 2021-01-27 11:46 ` Martin Doucha 2021-01-27 11:50 ` liuxp11 2021-03-04 7:52 ` Li Wang 2021-03-05 5:52 ` liuxp11 2021-03-05 9:02 ` Li Wang 2021-03-24 9:26 ` liuxp11 2021-04-11 5:01 ` Li Wang
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.