* [LTP] [PATCH v4] make pid_list dynamically sized with memory
@ 2011-01-06 18:40 czhang
2011-01-06 18:42 ` [LTP] Final version [Re: [PATCH v4] make pid_list dynamically sized with memory] Caspar Zhang
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: czhang @ 2011-01-06 18:40 UTC (permalink / raw)
To: ltp-list; +Cc: Caspar Zhang
From: Caspar Zhang <czhang@redhat.com>
We get segfaults during testing mtest01 on a 5TB memory machine, the
problem was traced to the array pid_list[] went to overflow and
corrupted memory. This fix makes pid_list dynamically sized with
correct memory size to avoid overflow.
v2: not split into different bits when allocating pid_list.
v3: 1) fix optargs: -b and -p should not be used at the same time.
2) pre_mem maybe not initialized before using if -p option not used.
fix it by moving it outside ``if(maxpercent)'' block.
v4: fix code format.
Signed-off-by: Caspar Zhang <czhang@redhat.com>
---
testcases/kernel/mem/mtest01/mtest01.c | 409 +++++++++++++++++---------------
1 files changed, 215 insertions(+), 194 deletions(-)
diff --git a/testcases/kernel/mem/mtest01/mtest01.c b/testcases/kernel/mem/mtest01/mtest01.c
index aee0d51..94b429a 100644
--- a/testcases/kernel/mem/mtest01/mtest01.c
+++ b/testcases/kernel/mem/mtest01/mtest01.c
@@ -44,223 +44,244 @@
#include "test.h"
+#define FIVE_HUNDRED_KB (unsigned long long)(500*1024*1024)
+#define ONE_MEGABYTE (unsigned long long)(1024*1024*1024)
+#define THREE_MEGABYTES (unsigned long long)(3*ONE_MEGABYTE)
+
char *TCID = "mtest01";
int TST_TOTAL = 1;
-
int pid_count = 0;
void handler(int signo)
{
- pid_count++;
+ pid_count++;
}
-int main(int argc, char* argv[]) {
- char* mem;
- float percent;
- unsigned int maxpercent=0, dowrite=0, verbose=0, j, c;
- unsigned long bytecount, alloc_bytes;
- unsigned long long original_maxbytes,maxbytes=0;
- unsigned long long pre_mem, post_mem;
- extern char* optarg;
- int chunksize = 1024*1024; /* one meg at a time by default */
- struct sysinfo sstats;
- int i,pid_cntr;
- pid_t pid,pid_list[1000];
- struct sigaction act;
-
- act.sa_handler = handler;
- act.sa_flags = 0;
- sigemptyset(&act.sa_mask);
- sigaction(SIGRTMIN, &act, 0);
-
- for (i=0;i<1000;i++)
- pid_list[i]=(pid_t)0;
-
- while ((c=getopt(argc, argv, "c:b:p:wvh")) != EOF) {
- switch((char)c) {
- case 'c':
- chunksize = atoi(optarg);
- break;
- case 'b':
- maxbytes = atoll(optarg);
- break;
- case 'p':
- maxpercent = atoi(optarg);
- if (maxpercent <= 0) {
- tst_resm(TFAIL, "ERROR: -p option requires number greater than 0");
- exit(1);}
- if (maxpercent > 99) {
- tst_resm(TFAIL, "ERROR: -p option cannot be greater than 99");
- exit(1);}
- break;
- case 'w':
- dowrite = 1;
- break;
- case 'v':
- verbose = 1;
- break;
- case 'h':
- default:
- printf("Usage: %s [-c <bytes>] [-b <bytes>|-p <percent>] [-v]\n", argv[0]);
- printf("\t-c <num>\tsize of chunk in bytes to malloc on each pass\n");
- printf("\t-b <bytes>\tmaximum number of bytes to allocate before stopping\n");
- printf("\t-p <bytes>\tpercent of total memory used at which the program stops\n");
- printf("\t-w\t\twrite to the memory after allocating\n");
- printf("\t-v\t\tverbose\n");
- printf("\t-h\t\tdisplay usage\n");
- exit(-1);
- }
- }
-
- sysinfo(&sstats);
- if (maxpercent) {
+int main(int argc, char* argv[])
+{
+ char* mem;
+ float percent;
+ unsigned int maxpercent = 0, dowrite = 0, verbose=0, j, c;
+ unsigned long bytecount, alloc_bytes, max_pids;
+ unsigned long long original_maxbytes, maxbytes = 0;
+ unsigned long long pre_mem, post_mem;
unsigned long long total_ram, total_free, D, C;
- percent=(float)maxpercent/100.00;
-
- total_ram=sstats.totalram;
- total_ram=total_ram+sstats.totalswap;
-
- total_free=sstats.freeram;
- total_free=total_free+sstats.freeswap;
-
- /* Total memory used needed to reach maxpercent */
- D = percent*(sstats.mem_unit*total_ram);
- tst_resm(TINFO, "Total memory used needed to reach maxpercent = %llu kbytes", D/1024);
-
- /* Total memory already used */
- C = sstats.mem_unit*(total_ram-total_free);
- tst_resm(TINFO, "Total memory already used on system = %llu kbytes", C/1024);
+ extern char* optarg;
+ int chunksize = 1024*1024; /* one meg at a time by default */
+ struct sysinfo sstats;
+ int i, pid_cntr;
+ pid_t pid, *pid_list;
+ struct sigaction act;
+ act.sa_handler = handler;
+ act.sa_flags = 0;
+ sigemptyset(&act.sa_mask);
+ sigaction(SIGRTMIN, &act, 0);
+
+ while ((c = getopt(argc, argv, "c:b:p:wvh")) != EOF)
+ {
+ switch((char)c)
+ {
+ case 'c':
+ chunksize = atoi(optarg);
+ break;
+ case 'b':
+ if (maxpercent != 0) {
+ tst_resm(TFAIL, "ERROR: -b option cannot be used with -p option at the same time");
+ exit(1);
+ }
+ maxbytes = atoll(optarg);
+ break;
+ case 'p':
+ if (maxbytes != 0) {
+ tst_resm(TFAIL, "ERROR: -p option cannot be used with -b option at the same time");
+ exit(1);
+ }
+ maxpercent = atoi(optarg);
+ if (maxpercent <= 0) {
+ tst_resm(TFAIL, "ERROR: -p option requires number greater than 0");
+ exit(1);
+ }
+ if (maxpercent > 99) {
+ tst_resm(TFAIL, "ERROR: -p option cannot be greater than 99");
+ exit(1);
+ }
+ break;
+ case 'w':
+ dowrite = 1;
+ break;
+ case 'v':
+ verbose = 1;
+ break;
+ case 'h':
+ default:
+ printf("Usage: %s [-c <bytes>] [-b <bytes>|-p <percent>] [-v]\n", argv[0]);
+ printf("\t-c <num>\tsize of chunk in bytes to malloc on each pass\n");
+ printf("\t-b <bytes>\tmaximum number of bytes to allocate before stopping\n");
+ printf("\t-p <bytes>\tpercent of total memory used at which the program stops\n");
+ printf("\t-w\t\twrite to the memory after allocating\n");
+ printf("\t-v\t\tverbose\n");
+ printf("\t-h\t\tdisplay usage\n");
+ exit(-1);
+ }
+ }
+
+ sysinfo(&sstats);
+ total_ram = sstats.totalram + sstats.totalswap;
+ total_free = sstats.freeram + sstats.freeswap;
/* Total Free Pre-Test RAM */
- pre_mem = sstats.mem_unit*total_free;
-
- /* Are we already using more than maxpercent? */
- if (C>D) {
- tst_resm(TFAIL, "More memory than the maximum amount you specified is already being used");
- exit(1);
+ pre_mem = sstats.mem_unit * total_free;
+ max_pids = total_ram / (unsigned long)FIVE_HUNDRED_KB + 1;
+
+ if ((pid_list = malloc(max_pids * sizeof(pid_t))) == NULL)
+ {
+ tst_resm(TBROK|TERRNO, "malloc failed.");
+ exit(1);
}
- else
- pre_mem = sstats.mem_unit*total_free;
+ memset(pid_list, 0, max_pids * sizeof(pid_t));
+
+ /* Currently used memory */
+ C = sstats.mem_unit * (total_ram - total_free);
+ tst_resm(TINFO, "Total memory already used on system = %llu kbytes", C/1024);
+
+ if (maxpercent)
+ {
+ percent = (float)maxpercent / 100.00;
- /* set maxbytes to the extra amount we want to allocate */
- maxbytes = D-C;
- tst_resm(TINFO, "Filling up %d%% of ram which is %llu kbytes", maxpercent, maxbytes/1024);
- }
- original_maxbytes=maxbytes;
- i=0;
- pid_cntr=0;
- pid=fork();
- if (pid != 0)
- pid_cntr++;
- pid_list[i]=pid;
+ /* Desired memory needed to reach maxpercent */
+ D = percent * (sstats.mem_unit * total_ram);
+ tst_resm(TINFO, "Total memory used needed to reach maxpercent = %llu kbytes", D/1024);
+
+ /* Are we already using more than maxpercent? */
+ if (C > D)
+ {
+ tst_resm(TFAIL, "More memory than the maximum amount you specified is already being used");
+ free(pid_list);
+ exit(1);
+ }
+
+ /* set maxbytes to the extra amount we want to allocate */
+ maxbytes = D - C;
+ tst_resm(TINFO, "Filling up %d%% of ram which is %llu kbytes", maxpercent, maxbytes/1024);
+ }
+ original_maxbytes = maxbytes;
+ i = 0;
+ pid_cntr = 0;
+ pid = fork();
+ if (pid != 0)
+ pid_cntr++;
+ pid_list[i] = pid;
#if defined (_s390_) /* s390's 31bit addressing requires smaller chunks */
-#define FIVE_HUNDRED_KB (500*1024*1024)
-#define ONE_MEGABYTE (1024*1024*1024)
-#define THREE_MEGABYTES (3*ONE_MEGABYTE)
- while (pid != 0 && maxbytes > FIVE_HUNDRED_KB)
- {
- i++;
- maxbytes -= FIVE_HUNDRED_KB;
- pid = fork();
- if (pid != 0) {
- pid_cntr++;
- pid_list[i] = pid;
+ while (pid != 0 && maxbytes > FIVE_HUNDRED_KB)
+ {
+ i++;
+ maxbytes -= FIVE_HUNDRED_KB;
+ pid = fork();
+ if (pid != 0)
+ {
+ pid_cntr++;
+ pid_list[i] = pid;
+ }
}
- }
- if (maxbytes > FIVE_HUNDRED_KB)
- alloc_bytes FIVE_HUNDRED_KB;
- else
- alloc_bytes = (unsigned long) maxbytes;
-
+ if (maxbytes > FIVE_HUNDRED_KB)
+ alloc_bytes = FIVE_HUNDRED_KB;
+ else
+ alloc_bytes = (unsigned long) maxbytes;
+
#elif __WORDSIZE==32
- while (pid != 0 && maxbytes > ONE_MEGABYTE)
- {
- i++;
- maxbytes -= ONE_MEGABYTE;
- pid = fork();
- if (pid != 0) {
- pid_cntr++;
- pid_list[i]=pid;
+ while (pid != 0 && maxbytes > ONE_MEGABYTE)
+ {
+ i++;
+ maxbytes -= ONE_MEGABYTE;
+ pid = fork();
+ if (pid != 0)
+ {
+ pid_cntr++;
+ pid_list[i]=pid;
+ }
}
- }
- if (maxbytes > ONE_MEGABYTE)
- alloc_bytes = ONE_MEGABYTE;
- else
- alloc_bytes = (unsigned long)maxbytes;
-
+ if (maxbytes > ONE_MEGABYTE)
+ alloc_bytes = ONE_MEGABYTE;
+ else
+ alloc_bytes = (unsigned long)maxbytes;
+
#elif __WORDSIZE==64
- while (pid!=0 && maxbytes > THREE_MEGABYTES)
- {
- i++;
- maxbytes -= THREE_MEGABYTES;
- pid=fork();
- if (pid != 0) {
- pid_cntr++;
- pid_list[i] = pid;
- }
- }
- if (maxbytes > THREE_MEGABYTES)
- alloc_bytes = THREE_MEGABYTES;
- else
- alloc_bytes = maxbytes;
-#endif
-
- if (pid == 0) /** CHILD **/
- {
- bytecount=chunksize;
- while (1) {
- if ((mem = (char*)malloc(chunksize)) == NULL) {
- tst_resm(TINFO, "stopped at %lu bytes", bytecount);
- exit(1);
- }
- if (dowrite)
- for (j=0; j<chunksize; j++)
- *(mem+j)='a';
- if (verbose)
- tst_resm(TINFO, "allocated %lu bytes chunksize is %d", bytecount, chunksize);
- bytecount+=chunksize;
- if (alloc_bytes && (bytecount >= alloc_bytes))
- break;
+ while (pid!=0 && maxbytes > THREE_MEGABYTES)
+ {
+ i++;
+ maxbytes -= THREE_MEGABYTES;
+ pid = fork();
+ if (pid != 0)
+ {
+ pid_cntr++;
+ pid_list[i] = pid;
+ }
}
- if (dowrite)
- tst_resm(TINFO, "... %lu bytes allocated and used.", bytecount);
+ if (maxbytes > THREE_MEGABYTES)
+ alloc_bytes = THREE_MEGABYTES;
else
- tst_resm(TINFO, "... %lu bytes allocated only.", bytecount);
- kill(getppid(),SIGRTMIN);
- while (1)
- sleep(1);
- }
- else /** PARENT **/
- {
-
- i=0;
- sysinfo(&sstats);
+ alloc_bytes = maxbytes;
+#endif
- if (dowrite)
+ if (pid == 0) /** CHILD **/
{
- /* Total Free Post-Test RAM */
- post_mem = (unsigned long long)sstats.mem_unit*sstats.freeram;
- post_mem = post_mem+((unsigned long long)sstats.mem_unit*sstats.freeswap);
-
- while ((((unsigned long long)pre_mem - post_mem) < (unsigned long long)original_maxbytes) &&
- (pid_count < pid_cntr) )
- {
- sleep(1);
- sysinfo(&sstats);
- post_mem = (unsigned long long)sstats.mem_unit*sstats.freeram;
- post_mem = post_mem+((unsigned long long)sstats.mem_unit*sstats.freeswap);
- }
+ bytecount = chunksize;
+ while (1)
+ {
+ if ((mem = malloc(chunksize)) == NULL)
+ {
+ tst_resm(TBROK|TERRNO, "stopped at %lu bytes", bytecount);
+ free(pid_list);
+ exit(1);
+ }
+ if (dowrite)
+ for (j = 0; j < chunksize; j++)
+ *(mem+j) = 'a';
+ if (verbose)
+ tst_resm(TINFO, "allocated %lu bytes chunksize is %d", bytecount, chunksize);
+ bytecount += chunksize;
+ if (alloc_bytes && bytecount >= alloc_bytes)
+ break;
+ }
+ if (dowrite)
+ tst_resm(TINFO, "... %lu bytes allocated and used.", bytecount);
+ else
+ tst_resm(TINFO, "... %lu bytes allocated only.", bytecount);
+ kill(getppid(), SIGRTMIN);
+ while (1)
+ sleep(1);
}
- while (pid_list[i]!=0)
+ else /** PARENT **/
{
- kill(pid_list[i],SIGKILL);
- i++;
+ i = 0;
+ sysinfo(&sstats);
+
+ if (dowrite)
+ {
+ /* Total Free Post-Test RAM */
+ post_mem = (unsigned long long)sstats.mem_unit * sstats.freeram;
+ post_mem = post_mem + (unsigned long long)sstats.mem_unit * sstats.freeswap;
+
+ while ((((unsigned long long)pre_mem - post_mem) < (unsigned long long)original_maxbytes) &&
+ (pid_count < pid_cntr))
+ {
+ sleep(1);
+ sysinfo(&sstats);
+ post_mem = (unsigned long long)sstats.mem_unit * sstats.freeram;
+ post_mem = post_mem + (unsigned long long)sstats.mem_unit * sstats.freeswap;
+ }
+ }
+ while (pid_list[i] != 0)
+ {
+ kill(pid_list[i], SIGKILL);
+ i++;
+ }
+ if (dowrite)
+ tst_resm(TPASS, "%llu kbytes allocated and used.", original_maxbytes/1024);
+ else
+ tst_resm(TPASS, "%llu kbytes allocated only.", original_maxbytes/1024);
}
- if (dowrite)
- tst_resm(TPASS, "%llu kbytes allocated and used.", original_maxbytes/1024);
- else
- tst_resm(TPASS, "%llu kbytes allocated only.", original_maxbytes/1024);
- }
- exit(0);
-}
\ No newline at end of file
+ free(pid_list);
+ exit(0);
+}
--
1.7.3.4
------------------------------------------------------------------------------
Learn how Oracle Real Application Clusters (RAC) One Node allows customers
to consolidate database storage, standardize their database environment, and,
should the need arise, upgrade to a full multi-node Oracle RAC database
without downtime or disruption
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [LTP] Final version [Re: [PATCH v4] make pid_list dynamically sized with memory]
2011-01-06 18:40 [LTP] [PATCH v4] make pid_list dynamically sized with memory czhang
@ 2011-01-06 18:42 ` Caspar Zhang
2011-01-11 9:26 ` Caspar Zhang
2011-01-16 11:05 ` [LTP] [PATCH v4] make pid_list dynamically sized with memory Caspar Zhang
2011-01-18 8:29 ` Caspar Zhang
2 siblings, 1 reply; 8+ messages in thread
From: Caspar Zhang @ 2011-01-06 18:42 UTC (permalink / raw)
To: Garrett Cooper; +Cc: ltp-list
Hi Garrett, this is the *final* version, please review.
On 01/07/2011 02:40 AM, czhang@redhat.com wrote:
> From: Caspar Zhang <czhang@redhat.com>
>
> We get segfaults during testing mtest01 on a 5TB memory machine, the
> problem was traced to the array pid_list[] went to overflow and
> corrupted memory. This fix makes pid_list dynamically sized with
> correct memory size to avoid overflow.
>
> v2: not split into different bits when allocating pid_list.
> v3: 1) fix optargs: -b and -p should not be used at the same time.
> 2) pre_mem maybe not initialized before using if -p option not used.
> fix it by moving it outside ``if(maxpercent)'' block.
> v4: fix code format.
>
> Signed-off-by: Caspar Zhang <czhang@redhat.com>
> ---
> testcases/kernel/mem/mtest01/mtest01.c | 409 +++++++++++++++++---------------
> 1 files changed, 215 insertions(+), 194 deletions(-)
>
> diff --git a/testcases/kernel/mem/mtest01/mtest01.c b/testcases/kernel/mem/mtest01/mtest01.c
> index aee0d51..94b429a 100644
> --- a/testcases/kernel/mem/mtest01/mtest01.c
> +++ b/testcases/kernel/mem/mtest01/mtest01.c
> @@ -44,223 +44,244 @@
>
> #include "test.h"
>
> +#define FIVE_HUNDRED_KB (unsigned long long)(500*1024*1024)
> +#define ONE_MEGABYTE (unsigned long long)(1024*1024*1024)
> +#define THREE_MEGABYTES (unsigned long long)(3*ONE_MEGABYTE)
> +
> char *TCID = "mtest01";
> int TST_TOTAL = 1;
> -
> int pid_count = 0;
>
> void handler(int signo)
> {
> - pid_count++;
> + pid_count++;
> }
>
> -int main(int argc, char* argv[]) {
> - char* mem;
> - float percent;
> - unsigned int maxpercent=0, dowrite=0, verbose=0, j, c;
> - unsigned long bytecount, alloc_bytes;
> - unsigned long long original_maxbytes,maxbytes=0;
> - unsigned long long pre_mem, post_mem;
> - extern char* optarg;
> - int chunksize = 1024*1024; /* one meg at a time by default */
> - struct sysinfo sstats;
> - int i,pid_cntr;
> - pid_t pid,pid_list[1000];
> - struct sigaction act;
> -
> - act.sa_handler = handler;
> - act.sa_flags = 0;
> - sigemptyset(&act.sa_mask);
> - sigaction(SIGRTMIN, &act, 0);
> -
> - for (i=0;i<1000;i++)
> - pid_list[i]=(pid_t)0;
> -
> - while ((c=getopt(argc, argv, "c:b:p:wvh")) != EOF) {
> - switch((char)c) {
> - case 'c':
> - chunksize = atoi(optarg);
> - break;
> - case 'b':
> - maxbytes = atoll(optarg);
> - break;
> - case 'p':
> - maxpercent = atoi(optarg);
> - if (maxpercent <= 0) {
> - tst_resm(TFAIL, "ERROR: -p option requires number greater than 0");
> - exit(1);}
> - if (maxpercent > 99) {
> - tst_resm(TFAIL, "ERROR: -p option cannot be greater than 99");
> - exit(1);}
> - break;
> - case 'w':
> - dowrite = 1;
> - break;
> - case 'v':
> - verbose = 1;
> - break;
> - case 'h':
> - default:
> - printf("Usage: %s [-c <bytes>] [-b <bytes>|-p <percent>] [-v]\n", argv[0]);
> - printf("\t-c <num>\tsize of chunk in bytes to malloc on each pass\n");
> - printf("\t-b <bytes>\tmaximum number of bytes to allocate before stopping\n");
> - printf("\t-p <bytes>\tpercent of total memory used at which the program stops\n");
> - printf("\t-w\t\twrite to the memory after allocating\n");
> - printf("\t-v\t\tverbose\n");
> - printf("\t-h\t\tdisplay usage\n");
> - exit(-1);
> - }
> - }
> -
> - sysinfo(&sstats);
> - if (maxpercent) {
> +int main(int argc, char* argv[])
> +{
> + char* mem;
> + float percent;
> + unsigned int maxpercent = 0, dowrite = 0, verbose=0, j, c;
> + unsigned long bytecount, alloc_bytes, max_pids;
> + unsigned long long original_maxbytes, maxbytes = 0;
> + unsigned long long pre_mem, post_mem;
> unsigned long long total_ram, total_free, D, C;
> - percent=(float)maxpercent/100.00;
> -
> - total_ram=sstats.totalram;
> - total_ram=total_ram+sstats.totalswap;
> -
> - total_free=sstats.freeram;
> - total_free=total_free+sstats.freeswap;
> -
> - /* Total memory used needed to reach maxpercent */
> - D = percent*(sstats.mem_unit*total_ram);
> - tst_resm(TINFO, "Total memory used needed to reach maxpercent = %llu kbytes", D/1024);
> -
> - /* Total memory already used */
> - C = sstats.mem_unit*(total_ram-total_free);
> - tst_resm(TINFO, "Total memory already used on system = %llu kbytes", C/1024);
> + extern char* optarg;
> + int chunksize = 1024*1024; /* one meg at a time by default */
> + struct sysinfo sstats;
> + int i, pid_cntr;
> + pid_t pid, *pid_list;
> + struct sigaction act;
>
> + act.sa_handler = handler;
> + act.sa_flags = 0;
> + sigemptyset(&act.sa_mask);
> + sigaction(SIGRTMIN, &act, 0);
> +
> + while ((c = getopt(argc, argv, "c:b:p:wvh")) != EOF)
> + {
> + switch((char)c)
> + {
> + case 'c':
> + chunksize = atoi(optarg);
> + break;
> + case 'b':
> + if (maxpercent != 0) {
> + tst_resm(TFAIL, "ERROR: -b option cannot be used with -p option at the same time");
> + exit(1);
> + }
> + maxbytes = atoll(optarg);
> + break;
> + case 'p':
> + if (maxbytes != 0) {
> + tst_resm(TFAIL, "ERROR: -p option cannot be used with -b option at the same time");
> + exit(1);
> + }
> + maxpercent = atoi(optarg);
> + if (maxpercent <= 0) {
> + tst_resm(TFAIL, "ERROR: -p option requires number greater than 0");
> + exit(1);
> + }
> + if (maxpercent > 99) {
> + tst_resm(TFAIL, "ERROR: -p option cannot be greater than 99");
> + exit(1);
> + }
> + break;
> + case 'w':
> + dowrite = 1;
> + break;
> + case 'v':
> + verbose = 1;
> + break;
> + case 'h':
> + default:
> + printf("Usage: %s [-c <bytes>] [-b <bytes>|-p <percent>] [-v]\n", argv[0]);
> + printf("\t-c <num>\tsize of chunk in bytes to malloc on each pass\n");
> + printf("\t-b <bytes>\tmaximum number of bytes to allocate before stopping\n");
> + printf("\t-p <bytes>\tpercent of total memory used at which the program stops\n");
> + printf("\t-w\t\twrite to the memory after allocating\n");
> + printf("\t-v\t\tverbose\n");
> + printf("\t-h\t\tdisplay usage\n");
> + exit(-1);
> + }
> + }
> +
> + sysinfo(&sstats);
> + total_ram = sstats.totalram + sstats.totalswap;
> + total_free = sstats.freeram + sstats.freeswap;
> /* Total Free Pre-Test RAM */
> - pre_mem = sstats.mem_unit*total_free;
> -
> - /* Are we already using more than maxpercent? */
> - if (C>D) {
> - tst_resm(TFAIL, "More memory than the maximum amount you specified is already being used");
> - exit(1);
> + pre_mem = sstats.mem_unit * total_free;
> + max_pids = total_ram / (unsigned long)FIVE_HUNDRED_KB + 1;
> +
> + if ((pid_list = malloc(max_pids * sizeof(pid_t))) == NULL)
> + {
> + tst_resm(TBROK|TERRNO, "malloc failed.");
> + exit(1);
> }
> - else
> - pre_mem = sstats.mem_unit*total_free;
> + memset(pid_list, 0, max_pids * sizeof(pid_t));
> +
> + /* Currently used memory */
> + C = sstats.mem_unit * (total_ram - total_free);
> + tst_resm(TINFO, "Total memory already used on system = %llu kbytes", C/1024);
> +
> + if (maxpercent)
> + {
> + percent = (float)maxpercent / 100.00;
>
> - /* set maxbytes to the extra amount we want to allocate */
> - maxbytes = D-C;
> - tst_resm(TINFO, "Filling up %d%% of ram which is %llu kbytes", maxpercent, maxbytes/1024);
> - }
> - original_maxbytes=maxbytes;
> - i=0;
> - pid_cntr=0;
> - pid=fork();
> - if (pid != 0)
> - pid_cntr++;
> - pid_list[i]=pid;
> + /* Desired memory needed to reach maxpercent */
> + D = percent * (sstats.mem_unit * total_ram);
> + tst_resm(TINFO, "Total memory used needed to reach maxpercent = %llu kbytes", D/1024);
> +
> + /* Are we already using more than maxpercent? */
> + if (C > D)
> + {
> + tst_resm(TFAIL, "More memory than the maximum amount you specified is already being used");
> + free(pid_list);
> + exit(1);
> + }
> +
> + /* set maxbytes to the extra amount we want to allocate */
> + maxbytes = D - C;
> + tst_resm(TINFO, "Filling up %d%% of ram which is %llu kbytes", maxpercent, maxbytes/1024);
> + }
> + original_maxbytes = maxbytes;
> + i = 0;
> + pid_cntr = 0;
> + pid = fork();
> + if (pid != 0)
> + pid_cntr++;
> + pid_list[i] = pid;
>
> #if defined (_s390_) /* s390's 31bit addressing requires smaller chunks */
> -#define FIVE_HUNDRED_KB (500*1024*1024)
> -#define ONE_MEGABYTE (1024*1024*1024)
> -#define THREE_MEGABYTES (3*ONE_MEGABYTE)
> - while (pid != 0 && maxbytes > FIVE_HUNDRED_KB)
> - {
> - i++;
> - maxbytes -= FIVE_HUNDRED_KB;
> - pid = fork();
> - if (pid != 0) {
> - pid_cntr++;
> - pid_list[i] = pid;
> + while (pid != 0 && maxbytes > FIVE_HUNDRED_KB)
> + {
> + i++;
> + maxbytes -= FIVE_HUNDRED_KB;
> + pid = fork();
> + if (pid != 0)
> + {
> + pid_cntr++;
> + pid_list[i] = pid;
> + }
> }
> - }
> - if (maxbytes > FIVE_HUNDRED_KB)
> - alloc_bytes FIVE_HUNDRED_KB;
> - else
> - alloc_bytes = (unsigned long) maxbytes;
> -
> + if (maxbytes > FIVE_HUNDRED_KB)
> + alloc_bytes = FIVE_HUNDRED_KB;
> + else
> + alloc_bytes = (unsigned long) maxbytes;
> +
> #elif __WORDSIZE==32
> - while (pid != 0 && maxbytes > ONE_MEGABYTE)
> - {
> - i++;
> - maxbytes -= ONE_MEGABYTE;
> - pid = fork();
> - if (pid != 0) {
> - pid_cntr++;
> - pid_list[i]=pid;
> + while (pid != 0 && maxbytes > ONE_MEGABYTE)
> + {
> + i++;
> + maxbytes -= ONE_MEGABYTE;
> + pid = fork();
> + if (pid != 0)
> + {
> + pid_cntr++;
> + pid_list[i]=pid;
> + }
> }
> - }
> - if (maxbytes > ONE_MEGABYTE)
> - alloc_bytes = ONE_MEGABYTE;
> - else
> - alloc_bytes = (unsigned long)maxbytes;
> -
> + if (maxbytes > ONE_MEGABYTE)
> + alloc_bytes = ONE_MEGABYTE;
> + else
> + alloc_bytes = (unsigned long)maxbytes;
> +
> #elif __WORDSIZE==64
> - while (pid!=0 && maxbytes > THREE_MEGABYTES)
> - {
> - i++;
> - maxbytes -= THREE_MEGABYTES;
> - pid=fork();
> - if (pid != 0) {
> - pid_cntr++;
> - pid_list[i] = pid;
> - }
> - }
> - if (maxbytes > THREE_MEGABYTES)
> - alloc_bytes = THREE_MEGABYTES;
> - else
> - alloc_bytes = maxbytes;
> -#endif
> -
> - if (pid == 0) /** CHILD **/
> - {
> - bytecount=chunksize;
> - while (1) {
> - if ((mem = (char*)malloc(chunksize)) == NULL) {
> - tst_resm(TINFO, "stopped at %lu bytes", bytecount);
> - exit(1);
> - }
> - if (dowrite)
> - for (j=0; j<chunksize; j++)
> - *(mem+j)='a';
> - if (verbose)
> - tst_resm(TINFO, "allocated %lu bytes chunksize is %d", bytecount, chunksize);
> - bytecount+=chunksize;
> - if (alloc_bytes && (bytecount >= alloc_bytes))
> - break;
> + while (pid!=0 && maxbytes > THREE_MEGABYTES)
> + {
> + i++;
> + maxbytes -= THREE_MEGABYTES;
> + pid = fork();
> + if (pid != 0)
> + {
> + pid_cntr++;
> + pid_list[i] = pid;
> + }
> }
> - if (dowrite)
> - tst_resm(TINFO, "... %lu bytes allocated and used.", bytecount);
> + if (maxbytes > THREE_MEGABYTES)
> + alloc_bytes = THREE_MEGABYTES;
> else
> - tst_resm(TINFO, "... %lu bytes allocated only.", bytecount);
> - kill(getppid(),SIGRTMIN);
> - while (1)
> - sleep(1);
> - }
> - else /** PARENT **/
> - {
> -
> - i=0;
> - sysinfo(&sstats);
> + alloc_bytes = maxbytes;
> +#endif
>
> - if (dowrite)
> + if (pid == 0) /** CHILD **/
> {
> - /* Total Free Post-Test RAM */
> - post_mem = (unsigned long long)sstats.mem_unit*sstats.freeram;
> - post_mem = post_mem+((unsigned long long)sstats.mem_unit*sstats.freeswap);
> -
> - while ((((unsigned long long)pre_mem - post_mem) < (unsigned long long)original_maxbytes) &&
> - (pid_count < pid_cntr) )
> - {
> - sleep(1);
> - sysinfo(&sstats);
> - post_mem = (unsigned long long)sstats.mem_unit*sstats.freeram;
> - post_mem = post_mem+((unsigned long long)sstats.mem_unit*sstats.freeswap);
> - }
> + bytecount = chunksize;
> + while (1)
> + {
> + if ((mem = malloc(chunksize)) == NULL)
> + {
> + tst_resm(TBROK|TERRNO, "stopped at %lu bytes", bytecount);
> + free(pid_list);
> + exit(1);
> + }
> + if (dowrite)
> + for (j = 0; j < chunksize; j++)
> + *(mem+j) = 'a';
> + if (verbose)
> + tst_resm(TINFO, "allocated %lu bytes chunksize is %d", bytecount, chunksize);
> + bytecount += chunksize;
> + if (alloc_bytes && bytecount >= alloc_bytes)
> + break;
> + }
> + if (dowrite)
> + tst_resm(TINFO, "... %lu bytes allocated and used.", bytecount);
> + else
> + tst_resm(TINFO, "... %lu bytes allocated only.", bytecount);
> + kill(getppid(), SIGRTMIN);
> + while (1)
> + sleep(1);
> }
> - while (pid_list[i]!=0)
> + else /** PARENT **/
> {
> - kill(pid_list[i],SIGKILL);
> - i++;
> + i = 0;
> + sysinfo(&sstats);
> +
> + if (dowrite)
> + {
> + /* Total Free Post-Test RAM */
> + post_mem = (unsigned long long)sstats.mem_unit * sstats.freeram;
> + post_mem = post_mem + (unsigned long long)sstats.mem_unit * sstats.freeswap;
> +
> + while ((((unsigned long long)pre_mem - post_mem) < (unsigned long long)original_maxbytes) &&
> + (pid_count < pid_cntr))
> + {
> + sleep(1);
> + sysinfo(&sstats);
> + post_mem = (unsigned long long)sstats.mem_unit * sstats.freeram;
> + post_mem = post_mem + (unsigned long long)sstats.mem_unit * sstats.freeswap;
> + }
> + }
> + while (pid_list[i] != 0)
> + {
> + kill(pid_list[i], SIGKILL);
> + i++;
> + }
> + if (dowrite)
> + tst_resm(TPASS, "%llu kbytes allocated and used.", original_maxbytes/1024);
> + else
> + tst_resm(TPASS, "%llu kbytes allocated only.", original_maxbytes/1024);
> }
> - if (dowrite)
> - tst_resm(TPASS, "%llu kbytes allocated and used.", original_maxbytes/1024);
> - else
> - tst_resm(TPASS, "%llu kbytes allocated only.", original_maxbytes/1024);
> - }
> - exit(0);
> -}
> \ No newline at end of file
> + free(pid_list);
> + exit(0);
> +}
--
Quality Assurance Associate (Kernel) in
Red Hat Software (Beijing) Co., R&D Branch
TEL: +86-10-62608150
WEB: http://www.redhat.com/
--
C-A-S-P-A-R, Caspar /kæspɑ:/ ;-)
------------------------------------------------------------------------------
Learn how Oracle Real Application Clusters (RAC) One Node allows customers
to consolidate database storage, standardize their database environment, and,
should the need arise, upgrade to a full multi-node Oracle RAC database
without downtime or disruption
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [LTP] Final version [Re: [PATCH v4] make pid_list dynamically sized with memory]
2011-01-06 18:42 ` [LTP] Final version [Re: [PATCH v4] make pid_list dynamically sized with memory] Caspar Zhang
@ 2011-01-11 9:26 ` Caspar Zhang
0 siblings, 0 replies; 8+ messages in thread
From: Caspar Zhang @ 2011-01-11 9:26 UTC (permalink / raw)
To: ltp-list
On 01/07/2011 02:42 AM, Caspar Zhang wrote:
> Hi Garrett, this is the *final* version, please review.
Hi, anybody have a chance to look at this patch? Thanks!
Caspar
>
> On 01/07/2011 02:40 AM, czhang@redhat.com wrote:
>> From: Caspar Zhang <czhang@redhat.com>
>>
>> We get segfaults during testing mtest01 on a 5TB memory machine, the
>> problem was traced to the array pid_list[] went to overflow and
>> corrupted memory. This fix makes pid_list dynamically sized with
>> correct memory size to avoid overflow.
>>
>> v2: not split into different bits when allocating pid_list.
>> v3: 1) fix optargs: -b and -p should not be used at the same time.
>> 2) pre_mem maybe not initialized before using if -p option not used.
>> fix it by moving it outside ``if(maxpercent)'' block.
>> v4: fix code format.
>>
>> Signed-off-by: Caspar Zhang <czhang@redhat.com>
>> ---
>> testcases/kernel/mem/mtest01/mtest01.c | 409 +++++++++++++++++---------------
>> 1 files changed, 215 insertions(+), 194 deletions(-)
>>
>> diff --git a/testcases/kernel/mem/mtest01/mtest01.c b/testcases/kernel/mem/mtest01/mtest01.c
>> index aee0d51..94b429a 100644
>> --- a/testcases/kernel/mem/mtest01/mtest01.c
>> +++ b/testcases/kernel/mem/mtest01/mtest01.c
>> @@ -44,223 +44,244 @@
>>
>> #include "test.h"
>>
>> +#define FIVE_HUNDRED_KB (unsigned long long)(500*1024*1024)
>> +#define ONE_MEGABYTE (unsigned long long)(1024*1024*1024)
>> +#define THREE_MEGABYTES (unsigned long long)(3*ONE_MEGABYTE)
>> +
>> char *TCID = "mtest01";
>> int TST_TOTAL = 1;
>> -
>> int pid_count = 0;
>>
>> void handler(int signo)
>> {
>> - pid_count++;
>> + pid_count++;
>> }
>>
>> -int main(int argc, char* argv[]) {
>> - char* mem;
>> - float percent;
>> - unsigned int maxpercent=0, dowrite=0, verbose=0, j, c;
>> - unsigned long bytecount, alloc_bytes;
>> - unsigned long long original_maxbytes,maxbytes=0;
>> - unsigned long long pre_mem, post_mem;
>> - extern char* optarg;
>> - int chunksize = 1024*1024; /* one meg at a time by default */
>> - struct sysinfo sstats;
>> - int i,pid_cntr;
>> - pid_t pid,pid_list[1000];
>> - struct sigaction act;
>> -
>> - act.sa_handler = handler;
>> - act.sa_flags = 0;
>> - sigemptyset(&act.sa_mask);
>> - sigaction(SIGRTMIN, &act, 0);
>> -
>> - for (i=0;i<1000;i++)
>> - pid_list[i]=(pid_t)0;
>> -
>> - while ((c=getopt(argc, argv, "c:b:p:wvh")) != EOF) {
>> - switch((char)c) {
>> - case 'c':
>> - chunksize = atoi(optarg);
>> - break;
>> - case 'b':
>> - maxbytes = atoll(optarg);
>> - break;
>> - case 'p':
>> - maxpercent = atoi(optarg);
>> - if (maxpercent <= 0) {
>> - tst_resm(TFAIL, "ERROR: -p option requires number greater than 0");
>> - exit(1);}
>> - if (maxpercent > 99) {
>> - tst_resm(TFAIL, "ERROR: -p option cannot be greater than 99");
>> - exit(1);}
>> - break;
>> - case 'w':
>> - dowrite = 1;
>> - break;
>> - case 'v':
>> - verbose = 1;
>> - break;
>> - case 'h':
>> - default:
>> - printf("Usage: %s [-c <bytes>] [-b <bytes>|-p <percent>] [-v]\n", argv[0]);
>> - printf("\t-c <num>\tsize of chunk in bytes to malloc on each pass\n");
>> - printf("\t-b <bytes>\tmaximum number of bytes to allocate before stopping\n");
>> - printf("\t-p <bytes>\tpercent of total memory used at which the program stops\n");
>> - printf("\t-w\t\twrite to the memory after allocating\n");
>> - printf("\t-v\t\tverbose\n");
>> - printf("\t-h\t\tdisplay usage\n");
>> - exit(-1);
>> - }
>> - }
>> -
>> - sysinfo(&sstats);
>> - if (maxpercent) {
>> +int main(int argc, char* argv[])
>> +{
>> + char* mem;
>> + float percent;
>> + unsigned int maxpercent = 0, dowrite = 0, verbose=0, j, c;
>> + unsigned long bytecount, alloc_bytes, max_pids;
>> + unsigned long long original_maxbytes, maxbytes = 0;
>> + unsigned long long pre_mem, post_mem;
>> unsigned long long total_ram, total_free, D, C;
>> - percent=(float)maxpercent/100.00;
>> -
>> - total_ram=sstats.totalram;
>> - total_ram=total_ram+sstats.totalswap;
>> -
>> - total_free=sstats.freeram;
>> - total_free=total_free+sstats.freeswap;
>> -
>> - /* Total memory used needed to reach maxpercent */
>> - D = percent*(sstats.mem_unit*total_ram);
>> - tst_resm(TINFO, "Total memory used needed to reach maxpercent = %llu kbytes", D/1024);
>> -
>> - /* Total memory already used */
>> - C = sstats.mem_unit*(total_ram-total_free);
>> - tst_resm(TINFO, "Total memory already used on system = %llu kbytes", C/1024);
>> + extern char* optarg;
>> + int chunksize = 1024*1024; /* one meg at a time by default */
>> + struct sysinfo sstats;
>> + int i, pid_cntr;
>> + pid_t pid, *pid_list;
>> + struct sigaction act;
>>
>> + act.sa_handler = handler;
>> + act.sa_flags = 0;
>> + sigemptyset(&act.sa_mask);
>> + sigaction(SIGRTMIN, &act, 0);
>> +
>> + while ((c = getopt(argc, argv, "c:b:p:wvh")) != EOF)
>> + {
>> + switch((char)c)
>> + {
>> + case 'c':
>> + chunksize = atoi(optarg);
>> + break;
>> + case 'b':
>> + if (maxpercent != 0) {
>> + tst_resm(TFAIL, "ERROR: -b option cannot be used with -p option at the same time");
>> + exit(1);
>> + }
>> + maxbytes = atoll(optarg);
>> + break;
>> + case 'p':
>> + if (maxbytes != 0) {
>> + tst_resm(TFAIL, "ERROR: -p option cannot be used with -b option at the same time");
>> + exit(1);
>> + }
>> + maxpercent = atoi(optarg);
>> + if (maxpercent <= 0) {
>> + tst_resm(TFAIL, "ERROR: -p option requires number greater than 0");
>> + exit(1);
>> + }
>> + if (maxpercent > 99) {
>> + tst_resm(TFAIL, "ERROR: -p option cannot be greater than 99");
>> + exit(1);
>> + }
>> + break;
>> + case 'w':
>> + dowrite = 1;
>> + break;
>> + case 'v':
>> + verbose = 1;
>> + break;
>> + case 'h':
>> + default:
>> + printf("Usage: %s [-c <bytes>] [-b <bytes>|-p <percent>] [-v]\n", argv[0]);
>> + printf("\t-c <num>\tsize of chunk in bytes to malloc on each pass\n");
>> + printf("\t-b <bytes>\tmaximum number of bytes to allocate before stopping\n");
>> + printf("\t-p <bytes>\tpercent of total memory used at which the program stops\n");
>> + printf("\t-w\t\twrite to the memory after allocating\n");
>> + printf("\t-v\t\tverbose\n");
>> + printf("\t-h\t\tdisplay usage\n");
>> + exit(-1);
>> + }
>> + }
>> +
>> + sysinfo(&sstats);
>> + total_ram = sstats.totalram + sstats.totalswap;
>> + total_free = sstats.freeram + sstats.freeswap;
>> /* Total Free Pre-Test RAM */
>> - pre_mem = sstats.mem_unit*total_free;
>> -
>> - /* Are we already using more than maxpercent? */
>> - if (C>D) {
>> - tst_resm(TFAIL, "More memory than the maximum amount you specified is already being used");
>> - exit(1);
>> + pre_mem = sstats.mem_unit * total_free;
>> + max_pids = total_ram / (unsigned long)FIVE_HUNDRED_KB + 1;
>> +
>> + if ((pid_list = malloc(max_pids * sizeof(pid_t))) == NULL)
>> + {
>> + tst_resm(TBROK|TERRNO, "malloc failed.");
>> + exit(1);
>> }
>> - else
>> - pre_mem = sstats.mem_unit*total_free;
>> + memset(pid_list, 0, max_pids * sizeof(pid_t));
>> +
>> + /* Currently used memory */
>> + C = sstats.mem_unit * (total_ram - total_free);
>> + tst_resm(TINFO, "Total memory already used on system = %llu kbytes", C/1024);
>> +
>> + if (maxpercent)
>> + {
>> + percent = (float)maxpercent / 100.00;
>>
>> - /* set maxbytes to the extra amount we want to allocate */
>> - maxbytes = D-C;
>> - tst_resm(TINFO, "Filling up %d%% of ram which is %llu kbytes", maxpercent, maxbytes/1024);
>> - }
>> - original_maxbytes=maxbytes;
>> - i=0;
>> - pid_cntr=0;
>> - pid=fork();
>> - if (pid != 0)
>> - pid_cntr++;
>> - pid_list[i]=pid;
>> + /* Desired memory needed to reach maxpercent */
>> + D = percent * (sstats.mem_unit * total_ram);
>> + tst_resm(TINFO, "Total memory used needed to reach maxpercent = %llu kbytes", D/1024);
>> +
>> + /* Are we already using more than maxpercent? */
>> + if (C > D)
>> + {
>> + tst_resm(TFAIL, "More memory than the maximum amount you specified is already being used");
>> + free(pid_list);
>> + exit(1);
>> + }
>> +
>> + /* set maxbytes to the extra amount we want to allocate */
>> + maxbytes = D - C;
>> + tst_resm(TINFO, "Filling up %d%% of ram which is %llu kbytes", maxpercent, maxbytes/1024);
>> + }
>> + original_maxbytes = maxbytes;
>> + i = 0;
>> + pid_cntr = 0;
>> + pid = fork();
>> + if (pid != 0)
>> + pid_cntr++;
>> + pid_list[i] = pid;
>>
>> #if defined (_s390_) /* s390's 31bit addressing requires smaller chunks */
>> -#define FIVE_HUNDRED_KB (500*1024*1024)
>> -#define ONE_MEGABYTE (1024*1024*1024)
>> -#define THREE_MEGABYTES (3*ONE_MEGABYTE)
>> - while (pid != 0 && maxbytes > FIVE_HUNDRED_KB)
>> - {
>> - i++;
>> - maxbytes -= FIVE_HUNDRED_KB;
>> - pid = fork();
>> - if (pid != 0) {
>> - pid_cntr++;
>> - pid_list[i] = pid;
>> + while (pid != 0 && maxbytes > FIVE_HUNDRED_KB)
>> + {
>> + i++;
>> + maxbytes -= FIVE_HUNDRED_KB;
>> + pid = fork();
>> + if (pid != 0)
>> + {
>> + pid_cntr++;
>> + pid_list[i] = pid;
>> + }
>> }
>> - }
>> - if (maxbytes > FIVE_HUNDRED_KB)
>> - alloc_bytes FIVE_HUNDRED_KB;
>> - else
>> - alloc_bytes = (unsigned long) maxbytes;
>> -
>> + if (maxbytes > FIVE_HUNDRED_KB)
>> + alloc_bytes = FIVE_HUNDRED_KB;
>> + else
>> + alloc_bytes = (unsigned long) maxbytes;
>> +
>> #elif __WORDSIZE==32
>> - while (pid != 0 && maxbytes > ONE_MEGABYTE)
>> - {
>> - i++;
>> - maxbytes -= ONE_MEGABYTE;
>> - pid = fork();
>> - if (pid != 0) {
>> - pid_cntr++;
>> - pid_list[i]=pid;
>> + while (pid != 0 && maxbytes > ONE_MEGABYTE)
>> + {
>> + i++;
>> + maxbytes -= ONE_MEGABYTE;
>> + pid = fork();
>> + if (pid != 0)
>> + {
>> + pid_cntr++;
>> + pid_list[i]=pid;
>> + }
>> }
>> - }
>> - if (maxbytes > ONE_MEGABYTE)
>> - alloc_bytes = ONE_MEGABYTE;
>> - else
>> - alloc_bytes = (unsigned long)maxbytes;
>> -
>> + if (maxbytes > ONE_MEGABYTE)
>> + alloc_bytes = ONE_MEGABYTE;
>> + else
>> + alloc_bytes = (unsigned long)maxbytes;
>> +
>> #elif __WORDSIZE==64
>> - while (pid!=0 && maxbytes > THREE_MEGABYTES)
>> - {
>> - i++;
>> - maxbytes -= THREE_MEGABYTES;
>> - pid=fork();
>> - if (pid != 0) {
>> - pid_cntr++;
>> - pid_list[i] = pid;
>> - }
>> - }
>> - if (maxbytes > THREE_MEGABYTES)
>> - alloc_bytes = THREE_MEGABYTES;
>> - else
>> - alloc_bytes = maxbytes;
>> -#endif
>> -
>> - if (pid == 0) /** CHILD **/
>> - {
>> - bytecount=chunksize;
>> - while (1) {
>> - if ((mem = (char*)malloc(chunksize)) == NULL) {
>> - tst_resm(TINFO, "stopped at %lu bytes", bytecount);
>> - exit(1);
>> - }
>> - if (dowrite)
>> - for (j=0; j<chunksize; j++)
>> - *(mem+j)='a';
>> - if (verbose)
>> - tst_resm(TINFO, "allocated %lu bytes chunksize is %d", bytecount, chunksize);
>> - bytecount+=chunksize;
>> - if (alloc_bytes && (bytecount >= alloc_bytes))
>> - break;
>> + while (pid!=0 && maxbytes > THREE_MEGABYTES)
>> + {
>> + i++;
>> + maxbytes -= THREE_MEGABYTES;
>> + pid = fork();
>> + if (pid != 0)
>> + {
>> + pid_cntr++;
>> + pid_list[i] = pid;
>> + }
>> }
>> - if (dowrite)
>> - tst_resm(TINFO, "... %lu bytes allocated and used.", bytecount);
>> + if (maxbytes > THREE_MEGABYTES)
>> + alloc_bytes = THREE_MEGABYTES;
>> else
>> - tst_resm(TINFO, "... %lu bytes allocated only.", bytecount);
>> - kill(getppid(),SIGRTMIN);
>> - while (1)
>> - sleep(1);
>> - }
>> - else /** PARENT **/
>> - {
>> -
>> - i=0;
>> - sysinfo(&sstats);
>> + alloc_bytes = maxbytes;
>> +#endif
>>
>> - if (dowrite)
>> + if (pid == 0) /** CHILD **/
>> {
>> - /* Total Free Post-Test RAM */
>> - post_mem = (unsigned long long)sstats.mem_unit*sstats.freeram;
>> - post_mem = post_mem+((unsigned long long)sstats.mem_unit*sstats.freeswap);
>> -
>> - while ((((unsigned long long)pre_mem - post_mem) < (unsigned long long)original_maxbytes) &&
>> - (pid_count < pid_cntr) )
>> - {
>> - sleep(1);
>> - sysinfo(&sstats);
>> - post_mem = (unsigned long long)sstats.mem_unit*sstats.freeram;
>> - post_mem = post_mem+((unsigned long long)sstats.mem_unit*sstats.freeswap);
>> - }
>> + bytecount = chunksize;
>> + while (1)
>> + {
>> + if ((mem = malloc(chunksize)) == NULL)
>> + {
>> + tst_resm(TBROK|TERRNO, "stopped at %lu bytes", bytecount);
>> + free(pid_list);
>> + exit(1);
>> + }
>> + if (dowrite)
>> + for (j = 0; j < chunksize; j++)
>> + *(mem+j) = 'a';
>> + if (verbose)
>> + tst_resm(TINFO, "allocated %lu bytes chunksize is %d", bytecount, chunksize);
>> + bytecount += chunksize;
>> + if (alloc_bytes && bytecount >= alloc_bytes)
>> + break;
>> + }
>> + if (dowrite)
>> + tst_resm(TINFO, "... %lu bytes allocated and used.", bytecount);
>> + else
>> + tst_resm(TINFO, "... %lu bytes allocated only.", bytecount);
>> + kill(getppid(), SIGRTMIN);
>> + while (1)
>> + sleep(1);
>> }
>> - while (pid_list[i]!=0)
>> + else /** PARENT **/
>> {
>> - kill(pid_list[i],SIGKILL);
>> - i++;
>> + i = 0;
>> + sysinfo(&sstats);
>> +
>> + if (dowrite)
>> + {
>> + /* Total Free Post-Test RAM */
>> + post_mem = (unsigned long long)sstats.mem_unit * sstats.freeram;
>> + post_mem = post_mem + (unsigned long long)sstats.mem_unit * sstats.freeswap;
>> +
>> + while ((((unsigned long long)pre_mem - post_mem) < (unsigned long long)original_maxbytes) &&
>> + (pid_count < pid_cntr))
>> + {
>> + sleep(1);
>> + sysinfo(&sstats);
>> + post_mem = (unsigned long long)sstats.mem_unit * sstats.freeram;
>> + post_mem = post_mem + (unsigned long long)sstats.mem_unit * sstats.freeswap;
>> + }
>> + }
>> + while (pid_list[i] != 0)
>> + {
>> + kill(pid_list[i], SIGKILL);
>> + i++;
>> + }
>> + if (dowrite)
>> + tst_resm(TPASS, "%llu kbytes allocated and used.", original_maxbytes/1024);
>> + else
>> + tst_resm(TPASS, "%llu kbytes allocated only.", original_maxbytes/1024);
>> }
>> - if (dowrite)
>> - tst_resm(TPASS, "%llu kbytes allocated and used.", original_maxbytes/1024);
>> - else
>> - tst_resm(TPASS, "%llu kbytes allocated only.", original_maxbytes/1024);
>> - }
>> - exit(0);
>> -}
>> \ No newline at end of file
>> + free(pid_list);
>> + exit(0);
>> +}
>
>
--
Quality Assurance Associate (Kernel) in
Red Hat Software (Beijing) Co., R&D Branch
TEL: +86-10-62608150
WEB: http://www.redhat.com/
--
C-A-S-P-A-R, Caspar /kæspɑ:/ ;-)
------------------------------------------------------------------------------
Gaining the trust of online customers is vital for the success of any company
that requires sensitive data to be transmitted over the Web. Learn how to
best implement a security strategy that keeps consumers' information secure
and instills the confidence they need to proceed with transactions.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [LTP] [PATCH v4] make pid_list dynamically sized with memory
2011-01-06 18:40 [LTP] [PATCH v4] make pid_list dynamically sized with memory czhang
2011-01-06 18:42 ` [LTP] Final version [Re: [PATCH v4] make pid_list dynamically sized with memory] Caspar Zhang
@ 2011-01-16 11:05 ` Caspar Zhang
2011-01-18 8:29 ` Caspar Zhang
2 siblings, 0 replies; 8+ messages in thread
From: Caspar Zhang @ 2011-01-16 11:05 UTC (permalink / raw)
To: Garrett Cooper; +Cc: ltp-list
On 01/07/2011 02:40 AM, czhang@redhat.com wrote:
> We get segfaults during testing mtest01 on a 5TB memory machine, the
> problem was traced to the array pid_list[] went to overflow and
> corrupted memory. This fix makes pid_list dynamically sized with
> correct memory size to avoid overflow.
>
> v2: not split into different bits when allocating pid_list.
> v3: 1) fix optargs: -b and -p should not be used at the same time.
> 2) pre_mem maybe not initialized before using if -p option not used.
> fix it by moving it outside ``if(maxpercent)'' block.
> v4: fix code format.
>
> Signed-off-by: Caspar Zhang <czhang@redhat.com>
Hi Garrett, would you please review this patch? Thanks!
Caspar
--
Quality Assurance Associate (Kernel) in
Red Hat Software (Beijing) Co., R&D Branch
TEL: +86-10-62608150
WEB: http://www.redhat.com/
--
C-A-S-P-A-R, Caspar /kæspɑ:/ ;-)
------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand
malware threats, the impact they can have on your business, and how you
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list
^ permalink raw reply [flat|nested] 8+ messages in thread
* [LTP] [PATCH v4] make pid_list dynamically sized with memory
2011-01-06 18:40 [LTP] [PATCH v4] make pid_list dynamically sized with memory czhang
2011-01-06 18:42 ` [LTP] Final version [Re: [PATCH v4] make pid_list dynamically sized with memory] Caspar Zhang
2011-01-16 11:05 ` [LTP] [PATCH v4] make pid_list dynamically sized with memory Caspar Zhang
@ 2011-01-18 8:29 ` Caspar Zhang
2011-01-18 8:33 ` Garrett Cooper
2 siblings, 1 reply; 8+ messages in thread
From: Caspar Zhang @ 2011-01-18 8:29 UTC (permalink / raw)
To: ltp-list
[-- Attachment #1: Type: text/plain, Size: 923 bytes --]
On 01/07/2011 02:40 AM, czhang@redhat.com wrote:
> From: Caspar Zhang <czhang@redhat.com>
>
> We get segfaults during testing mtest01 on a 5TB memory machine, the
> problem was traced to the array pid_list[] went to overflow and
> corrupted memory. This fix makes pid_list dynamically sized with
> correct memory size to avoid overflow.
>
> v2: not split into different bits when allocating pid_list.
> v3: 1) fix optargs: -b and -p should not be used at the same time.
> 2) pre_mem maybe not initialized before using if -p option not used.
> fix it by moving it outside ``if(maxpercent)'' block.
> v4: fix code format.
>
> Signed-off-by: Caspar Zhang <czhang@redhat.com>
Patch attached.
Thanks,
Caspar
--
Quality Assurance Engineer (Kernel) in
Red Hat Software (Beijing) Co., R&D Branch
TEL: +86-10-62608150
WEB: http://www.redhat.com/
--
C-A-S-P-A-R, Caspar /kæspɑ:/ ;-)
[-- Attachment #2: 0001-make-pid_list-dynamically-sized-with-memory.patch --]
[-- Type: text/plain, Size: 14749 bytes --]
From ca9ec16d1a80e0dbbe8819de03cc39f33c01f9e4 Mon Sep 17 00:00:00 2001
From: Caspar Zhang <czhang@redhat.com>
Date: Fri, 7 Jan 2011 02:35:34 +0800
Subject: [PATCH v4] make pid_list dynamically sized with memory
We get segfaults during testing mtest01 on a 5TB memory machine, the
problem was traced to the array pid_list[] went to overflow and
corrupted memory. This fix makes pid_list dynamically sized with
correct memory size to avoid overflow.
v2: not split into different bits when allocating pid_list.
v3: 1) fix optargs: -b and -p should not be used at the same time.
2) pre_mem maybe not initialized before using if -p option not used.
fix it by moving it outside ``if(maxpercent)'' block.
v4: fix code format.
Signed-off-by: Caspar Zhang <czhang@redhat.com>
---
testcases/kernel/mem/mtest01/mtest01.c | 409 +++++++++++++++++---------------
1 files changed, 215 insertions(+), 194 deletions(-)
diff --git a/testcases/kernel/mem/mtest01/mtest01.c b/testcases/kernel/mem/mtest01/mtest01.c
index aee0d51..94b429a 100644
--- a/testcases/kernel/mem/mtest01/mtest01.c
+++ b/testcases/kernel/mem/mtest01/mtest01.c
@@ -44,223 +44,244 @@
#include "test.h"
+#define FIVE_HUNDRED_KB (unsigned long long)(500*1024*1024)
+#define ONE_MEGABYTE (unsigned long long)(1024*1024*1024)
+#define THREE_MEGABYTES (unsigned long long)(3*ONE_MEGABYTE)
+
char *TCID = "mtest01";
int TST_TOTAL = 1;
-
int pid_count = 0;
void handler(int signo)
{
- pid_count++;
+ pid_count++;
}
-int main(int argc, char* argv[]) {
- char* mem;
- float percent;
- unsigned int maxpercent=0, dowrite=0, verbose=0, j, c;
- unsigned long bytecount, alloc_bytes;
- unsigned long long original_maxbytes,maxbytes=0;
- unsigned long long pre_mem, post_mem;
- extern char* optarg;
- int chunksize = 1024*1024; /* one meg at a time by default */
- struct sysinfo sstats;
- int i,pid_cntr;
- pid_t pid,pid_list[1000];
- struct sigaction act;
-
- act.sa_handler = handler;
- act.sa_flags = 0;
- sigemptyset(&act.sa_mask);
- sigaction(SIGRTMIN, &act, 0);
-
- for (i=0;i<1000;i++)
- pid_list[i]=(pid_t)0;
-
- while ((c=getopt(argc, argv, "c:b:p:wvh")) != EOF) {
- switch((char)c) {
- case 'c':
- chunksize = atoi(optarg);
- break;
- case 'b':
- maxbytes = atoll(optarg);
- break;
- case 'p':
- maxpercent = atoi(optarg);
- if (maxpercent <= 0) {
- tst_resm(TFAIL, "ERROR: -p option requires number greater than 0");
- exit(1);}
- if (maxpercent > 99) {
- tst_resm(TFAIL, "ERROR: -p option cannot be greater than 99");
- exit(1);}
- break;
- case 'w':
- dowrite = 1;
- break;
- case 'v':
- verbose = 1;
- break;
- case 'h':
- default:
- printf("Usage: %s [-c <bytes>] [-b <bytes>|-p <percent>] [-v]\n", argv[0]);
- printf("\t-c <num>\tsize of chunk in bytes to malloc on each pass\n");
- printf("\t-b <bytes>\tmaximum number of bytes to allocate before stopping\n");
- printf("\t-p <bytes>\tpercent of total memory used at which the program stops\n");
- printf("\t-w\t\twrite to the memory after allocating\n");
- printf("\t-v\t\tverbose\n");
- printf("\t-h\t\tdisplay usage\n");
- exit(-1);
- }
- }
-
- sysinfo(&sstats);
- if (maxpercent) {
+int main(int argc, char* argv[])
+{
+ char* mem;
+ float percent;
+ unsigned int maxpercent = 0, dowrite = 0, verbose=0, j, c;
+ unsigned long bytecount, alloc_bytes, max_pids;
+ unsigned long long original_maxbytes, maxbytes = 0;
+ unsigned long long pre_mem, post_mem;
unsigned long long total_ram, total_free, D, C;
- percent=(float)maxpercent/100.00;
-
- total_ram=sstats.totalram;
- total_ram=total_ram+sstats.totalswap;
-
- total_free=sstats.freeram;
- total_free=total_free+sstats.freeswap;
-
- /* Total memory used needed to reach maxpercent */
- D = percent*(sstats.mem_unit*total_ram);
- tst_resm(TINFO, "Total memory used needed to reach maxpercent = %llu kbytes", D/1024);
-
- /* Total memory already used */
- C = sstats.mem_unit*(total_ram-total_free);
- tst_resm(TINFO, "Total memory already used on system = %llu kbytes", C/1024);
+ extern char* optarg;
+ int chunksize = 1024*1024; /* one meg at a time by default */
+ struct sysinfo sstats;
+ int i, pid_cntr;
+ pid_t pid, *pid_list;
+ struct sigaction act;
+ act.sa_handler = handler;
+ act.sa_flags = 0;
+ sigemptyset(&act.sa_mask);
+ sigaction(SIGRTMIN, &act, 0);
+
+ while ((c = getopt(argc, argv, "c:b:p:wvh")) != EOF)
+ {
+ switch((char)c)
+ {
+ case 'c':
+ chunksize = atoi(optarg);
+ break;
+ case 'b':
+ if (maxpercent != 0) {
+ tst_resm(TFAIL, "ERROR: -b option cannot be used with -p option at the same time");
+ exit(1);
+ }
+ maxbytes = atoll(optarg);
+ break;
+ case 'p':
+ if (maxbytes != 0) {
+ tst_resm(TFAIL, "ERROR: -p option cannot be used with -b option at the same time");
+ exit(1);
+ }
+ maxpercent = atoi(optarg);
+ if (maxpercent <= 0) {
+ tst_resm(TFAIL, "ERROR: -p option requires number greater than 0");
+ exit(1);
+ }
+ if (maxpercent > 99) {
+ tst_resm(TFAIL, "ERROR: -p option cannot be greater than 99");
+ exit(1);
+ }
+ break;
+ case 'w':
+ dowrite = 1;
+ break;
+ case 'v':
+ verbose = 1;
+ break;
+ case 'h':
+ default:
+ printf("Usage: %s [-c <bytes>] [-b <bytes>|-p <percent>] [-v]\n", argv[0]);
+ printf("\t-c <num>\tsize of chunk in bytes to malloc on each pass\n");
+ printf("\t-b <bytes>\tmaximum number of bytes to allocate before stopping\n");
+ printf("\t-p <bytes>\tpercent of total memory used at which the program stops\n");
+ printf("\t-w\t\twrite to the memory after allocating\n");
+ printf("\t-v\t\tverbose\n");
+ printf("\t-h\t\tdisplay usage\n");
+ exit(-1);
+ }
+ }
+
+ sysinfo(&sstats);
+ total_ram = sstats.totalram + sstats.totalswap;
+ total_free = sstats.freeram + sstats.freeswap;
/* Total Free Pre-Test RAM */
- pre_mem = sstats.mem_unit*total_free;
-
- /* Are we already using more than maxpercent? */
- if (C>D) {
- tst_resm(TFAIL, "More memory than the maximum amount you specified is already being used");
- exit(1);
+ pre_mem = sstats.mem_unit * total_free;
+ max_pids = total_ram / (unsigned long)FIVE_HUNDRED_KB + 1;
+
+ if ((pid_list = malloc(max_pids * sizeof(pid_t))) == NULL)
+ {
+ tst_resm(TBROK|TERRNO, "malloc failed.");
+ exit(1);
}
- else
- pre_mem = sstats.mem_unit*total_free;
+ memset(pid_list, 0, max_pids * sizeof(pid_t));
+
+ /* Currently used memory */
+ C = sstats.mem_unit * (total_ram - total_free);
+ tst_resm(TINFO, "Total memory already used on system = %llu kbytes", C/1024);
+
+ if (maxpercent)
+ {
+ percent = (float)maxpercent / 100.00;
- /* set maxbytes to the extra amount we want to allocate */
- maxbytes = D-C;
- tst_resm(TINFO, "Filling up %d%% of ram which is %llu kbytes", maxpercent, maxbytes/1024);
- }
- original_maxbytes=maxbytes;
- i=0;
- pid_cntr=0;
- pid=fork();
- if (pid != 0)
- pid_cntr++;
- pid_list[i]=pid;
+ /* Desired memory needed to reach maxpercent */
+ D = percent * (sstats.mem_unit * total_ram);
+ tst_resm(TINFO, "Total memory used needed to reach maxpercent = %llu kbytes", D/1024);
+
+ /* Are we already using more than maxpercent? */
+ if (C > D)
+ {
+ tst_resm(TFAIL, "More memory than the maximum amount you specified is already being used");
+ free(pid_list);
+ exit(1);
+ }
+
+ /* set maxbytes to the extra amount we want to allocate */
+ maxbytes = D - C;
+ tst_resm(TINFO, "Filling up %d%% of ram which is %llu kbytes", maxpercent, maxbytes/1024);
+ }
+ original_maxbytes = maxbytes;
+ i = 0;
+ pid_cntr = 0;
+ pid = fork();
+ if (pid != 0)
+ pid_cntr++;
+ pid_list[i] = pid;
#if defined (_s390_) /* s390's 31bit addressing requires smaller chunks */
-#define FIVE_HUNDRED_KB (500*1024*1024)
-#define ONE_MEGABYTE (1024*1024*1024)
-#define THREE_MEGABYTES (3*ONE_MEGABYTE)
- while (pid != 0 && maxbytes > FIVE_HUNDRED_KB)
- {
- i++;
- maxbytes -= FIVE_HUNDRED_KB;
- pid = fork();
- if (pid != 0) {
- pid_cntr++;
- pid_list[i] = pid;
+ while (pid != 0 && maxbytes > FIVE_HUNDRED_KB)
+ {
+ i++;
+ maxbytes -= FIVE_HUNDRED_KB;
+ pid = fork();
+ if (pid != 0)
+ {
+ pid_cntr++;
+ pid_list[i] = pid;
+ }
}
- }
- if (maxbytes > FIVE_HUNDRED_KB)
- alloc_bytes FIVE_HUNDRED_KB;
- else
- alloc_bytes = (unsigned long) maxbytes;
-
+ if (maxbytes > FIVE_HUNDRED_KB)
+ alloc_bytes = FIVE_HUNDRED_KB;
+ else
+ alloc_bytes = (unsigned long) maxbytes;
+
#elif __WORDSIZE==32
- while (pid != 0 && maxbytes > ONE_MEGABYTE)
- {
- i++;
- maxbytes -= ONE_MEGABYTE;
- pid = fork();
- if (pid != 0) {
- pid_cntr++;
- pid_list[i]=pid;
+ while (pid != 0 && maxbytes > ONE_MEGABYTE)
+ {
+ i++;
+ maxbytes -= ONE_MEGABYTE;
+ pid = fork();
+ if (pid != 0)
+ {
+ pid_cntr++;
+ pid_list[i]=pid;
+ }
}
- }
- if (maxbytes > ONE_MEGABYTE)
- alloc_bytes = ONE_MEGABYTE;
- else
- alloc_bytes = (unsigned long)maxbytes;
-
+ if (maxbytes > ONE_MEGABYTE)
+ alloc_bytes = ONE_MEGABYTE;
+ else
+ alloc_bytes = (unsigned long)maxbytes;
+
#elif __WORDSIZE==64
- while (pid!=0 && maxbytes > THREE_MEGABYTES)
- {
- i++;
- maxbytes -= THREE_MEGABYTES;
- pid=fork();
- if (pid != 0) {
- pid_cntr++;
- pid_list[i] = pid;
- }
- }
- if (maxbytes > THREE_MEGABYTES)
- alloc_bytes = THREE_MEGABYTES;
- else
- alloc_bytes = maxbytes;
-#endif
-
- if (pid == 0) /** CHILD **/
- {
- bytecount=chunksize;
- while (1) {
- if ((mem = (char*)malloc(chunksize)) == NULL) {
- tst_resm(TINFO, "stopped at %lu bytes", bytecount);
- exit(1);
- }
- if (dowrite)
- for (j=0; j<chunksize; j++)
- *(mem+j)='a';
- if (verbose)
- tst_resm(TINFO, "allocated %lu bytes chunksize is %d", bytecount, chunksize);
- bytecount+=chunksize;
- if (alloc_bytes && (bytecount >= alloc_bytes))
- break;
+ while (pid!=0 && maxbytes > THREE_MEGABYTES)
+ {
+ i++;
+ maxbytes -= THREE_MEGABYTES;
+ pid = fork();
+ if (pid != 0)
+ {
+ pid_cntr++;
+ pid_list[i] = pid;
+ }
}
- if (dowrite)
- tst_resm(TINFO, "... %lu bytes allocated and used.", bytecount);
+ if (maxbytes > THREE_MEGABYTES)
+ alloc_bytes = THREE_MEGABYTES;
else
- tst_resm(TINFO, "... %lu bytes allocated only.", bytecount);
- kill(getppid(),SIGRTMIN);
- while (1)
- sleep(1);
- }
- else /** PARENT **/
- {
-
- i=0;
- sysinfo(&sstats);
+ alloc_bytes = maxbytes;
+#endif
- if (dowrite)
+ if (pid == 0) /** CHILD **/
{
- /* Total Free Post-Test RAM */
- post_mem = (unsigned long long)sstats.mem_unit*sstats.freeram;
- post_mem = post_mem+((unsigned long long)sstats.mem_unit*sstats.freeswap);
-
- while ((((unsigned long long)pre_mem - post_mem) < (unsigned long long)original_maxbytes) &&
- (pid_count < pid_cntr) )
- {
- sleep(1);
- sysinfo(&sstats);
- post_mem = (unsigned long long)sstats.mem_unit*sstats.freeram;
- post_mem = post_mem+((unsigned long long)sstats.mem_unit*sstats.freeswap);
- }
+ bytecount = chunksize;
+ while (1)
+ {
+ if ((mem = malloc(chunksize)) == NULL)
+ {
+ tst_resm(TBROK|TERRNO, "stopped at %lu bytes", bytecount);
+ free(pid_list);
+ exit(1);
+ }
+ if (dowrite)
+ for (j = 0; j < chunksize; j++)
+ *(mem+j) = 'a';
+ if (verbose)
+ tst_resm(TINFO, "allocated %lu bytes chunksize is %d", bytecount, chunksize);
+ bytecount += chunksize;
+ if (alloc_bytes && bytecount >= alloc_bytes)
+ break;
+ }
+ if (dowrite)
+ tst_resm(TINFO, "... %lu bytes allocated and used.", bytecount);
+ else
+ tst_resm(TINFO, "... %lu bytes allocated only.", bytecount);
+ kill(getppid(), SIGRTMIN);
+ while (1)
+ sleep(1);
}
- while (pid_list[i]!=0)
+ else /** PARENT **/
{
- kill(pid_list[i],SIGKILL);
- i++;
+ i = 0;
+ sysinfo(&sstats);
+
+ if (dowrite)
+ {
+ /* Total Free Post-Test RAM */
+ post_mem = (unsigned long long)sstats.mem_unit * sstats.freeram;
+ post_mem = post_mem + (unsigned long long)sstats.mem_unit * sstats.freeswap;
+
+ while ((((unsigned long long)pre_mem - post_mem) < (unsigned long long)original_maxbytes) &&
+ (pid_count < pid_cntr))
+ {
+ sleep(1);
+ sysinfo(&sstats);
+ post_mem = (unsigned long long)sstats.mem_unit * sstats.freeram;
+ post_mem = post_mem + (unsigned long long)sstats.mem_unit * sstats.freeswap;
+ }
+ }
+ while (pid_list[i] != 0)
+ {
+ kill(pid_list[i], SIGKILL);
+ i++;
+ }
+ if (dowrite)
+ tst_resm(TPASS, "%llu kbytes allocated and used.", original_maxbytes/1024);
+ else
+ tst_resm(TPASS, "%llu kbytes allocated only.", original_maxbytes/1024);
}
- if (dowrite)
- tst_resm(TPASS, "%llu kbytes allocated and used.", original_maxbytes/1024);
- else
- tst_resm(TPASS, "%llu kbytes allocated only.", original_maxbytes/1024);
- }
- exit(0);
-}
\ No newline at end of file
+ free(pid_list);
+ exit(0);
+}
--
1.7.3.4
[-- Attachment #3: Type: text/plain, Size: 372 bytes --]
------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand
malware threats, the impact they can have on your business, and how you
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
[-- Attachment #4: Type: text/plain, Size: 155 bytes --]
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [LTP] [PATCH v4] make pid_list dynamically sized with memory
2011-01-18 8:29 ` Caspar Zhang
@ 2011-01-18 8:33 ` Garrett Cooper
2011-01-18 8:45 ` Caspar Zhang
2011-01-18 9:14 ` [LTP] [PATCH v5] " Caspar Zhang
0 siblings, 2 replies; 8+ messages in thread
From: Garrett Cooper @ 2011-01-18 8:33 UTC (permalink / raw)
To: Caspar Zhang; +Cc: ltp-list
On Tue, Jan 18, 2011 at 12:29 AM, Caspar Zhang <czhang@redhat.com> wrote:
> On 01/07/2011 02:40 AM, czhang@redhat.com wrote:
>> From: Caspar Zhang <czhang@redhat.com>
>>
>> We get segfaults during testing mtest01 on a 5TB memory machine, the
>> problem was traced to the array pid_list[] went to overflow and
>> corrupted memory. This fix makes pid_list dynamically sized with
>> correct memory size to avoid overflow.
>>
>> v2: not split into different bits when allocating pid_list.
>> v3: 1) fix optargs: -b and -p should not be used at the same time.
>> 2) pre_mem maybe not initialized before using if -p option not used.
>> fix it by moving it outside ``if(maxpercent)'' block.
>> v4: fix code format.
>>
>> Signed-off-by: Caspar Zhang <czhang@redhat.com>
>
> Patch attached.
Doesn't patch:
Patching file testcases/kernel/mem/mtest01/mtest01.c using Plan A...
Hunk #1 failed at 44.
1 out of 1 hunks failed--saving rejects to
testcases/kernel/mem/mtest01/mtest01.c.rej
Hmm... Ignoring the trailing garbage.
done
Have you updated your sources recently?
Thanks,
-Garrett
------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand
malware threats, the impact they can have on your business, and how you
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [LTP] [PATCH v4] make pid_list dynamically sized with memory
2011-01-18 8:33 ` Garrett Cooper
@ 2011-01-18 8:45 ` Caspar Zhang
2011-01-18 9:14 ` [LTP] [PATCH v5] " Caspar Zhang
1 sibling, 0 replies; 8+ messages in thread
From: Caspar Zhang @ 2011-01-18 8:45 UTC (permalink / raw)
To: Garrett Cooper; +Cc: ltp-list
On 01/18/2011 04:33 PM, Garrett Cooper wrote:
> Doesn't patch:
>
> Patching file testcases/kernel/mem/mtest01/mtest01.c using Plan A...
> Hunk #1 failed at 44.
> 1 out of 1 hunks failed--saving rejects to
> testcases/kernel/mem/mtest01/mtest01.c.rej
> Hmm... Ignoring the trailing garbage.
> done
>
> Have you updated your sources recently?
Oh... sorry I didn't. I'll work out a updated version shortly.
Thanks,
Caspar
--
Quality Assurance Engineer (Kernel) in
Red Hat Software (Beijing) Co., R&D Branch
TEL: +86-10-62608150
WEB: http://www.redhat.com/
--
C-A-S-P-A-R, Caspar /kæspɑ:/ ;-)
------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand
malware threats, the impact they can have on your business, and how you
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list
^ permalink raw reply [flat|nested] 8+ messages in thread
* [LTP] [PATCH v5] make pid_list dynamically sized with memory
2011-01-18 8:33 ` Garrett Cooper
2011-01-18 8:45 ` Caspar Zhang
@ 2011-01-18 9:14 ` Caspar Zhang
1 sibling, 0 replies; 8+ messages in thread
From: Caspar Zhang @ 2011-01-18 9:14 UTC (permalink / raw)
To: Garrett Cooper; +Cc: ltp-list
[-- Attachment #1: Type: text/plain, Size: 24 bytes --]
v5: resolve conflict.
[-- Attachment #2: 0001-make-pid_list-dynamically-sized-with-memory.patch --]
[-- Type: text/plain, Size: 14817 bytes --]
From 3a2d333293f0b4387f67641dd5b409a6c3b32e37 Mon Sep 17 00:00:00 2001
From: Caspar Zhang <czhang@redhat.com>
Date: Tue, 18 Jan 2011 17:10:31 +0800
Subject: [PATCH v5] make pid_list dynamically sized with memory
We get segfaults during testing mtest01 on a 5TB memory machine, the
problem was traced to the array pid_list[] went to overflow and
corrupted memory. This fix makes pid_list dynamically sized with
correct memory size to avoid overflow.
v2: not split into different bits when allocating pid_list.
v3: 1) fix optargs: -b and -p should not be used at the same time.
2) pre_mem maybe not initialized before using if -p option not used.
fix it by moving it outside ``if(maxpercent)'' block.
v4: fix code format.
v5: resolve conflict.
Signed-off-by: Caspar Zhang <czhang@redhat.com>
---
testcases/kernel/mem/mtest01/mtest01.c | 412 +++++++++++++++++---------------
1 files changed, 215 insertions(+), 197 deletions(-)
diff --git a/testcases/kernel/mem/mtest01/mtest01.c b/testcases/kernel/mem/mtest01/mtest01.c
index a57fa55..c5fc2df 100644
--- a/testcases/kernel/mem/mtest01/mtest01.c
+++ b/testcases/kernel/mem/mtest01/mtest01.c
@@ -44,226 +44,244 @@
#include "test.h"
+#define FIVE_HUNDRED_KB (unsigned long long)(500*1024*1024)
+#define ONE_MEGABYTE (unsigned long long)(1024*1024*1024)
+#define THREE_MEGABYTES (unsigned long long)(3*ONE_MEGABYTE)
+
char *TCID = "mtest01";
int TST_TOTAL = 1;
-
int pid_count = 0;
void handler(int signo)
{
- pid_count++;
+ pid_count++;
}
-int main(int argc, char* argv[]) {
- char* mem;
- float percent;
- unsigned int maxpercent=0, dowrite=0, verbose=0, j, c;
- unsigned long bytecount, alloc_bytes;
- unsigned long long original_maxbytes,maxbytes=0;
- unsigned long long pre_mem, post_mem;
- extern char* optarg;
- int chunksize = 1024*1024; /* one meg at a time by default */
- struct sysinfo sstats;
- int i,pid_cntr;
- pid_t pid,pid_list[1000];
- struct sigaction act;
-
- act.sa_handler = handler;
- act.sa_flags = 0;
- sigemptyset(&act.sa_mask);
- sigaction(SIGRTMIN, &act, 0);
-
- pre_mem = post_mem = 0;
-
- for (i=0;i<1000;i++)
- pid_list[i]=(pid_t)0;
-
- while ((c=getopt(argc, argv, "c:b:p:wvh")) != EOF) {
- switch((char)c) {
- case 'c':
- chunksize = atoi(optarg);
- break;
- case 'b':
- maxbytes = atoll(optarg);
- break;
- case 'p':
- maxpercent = atoi(optarg);
- if (maxpercent <= 0) {
- tst_resm(TFAIL, "ERROR: -p option requires number greater than 0");
- exit(1);}
- if (maxpercent > 99) {
- tst_resm(TFAIL, "ERROR: -p option cannot be greater than 99");
- exit(1);}
- break;
- case 'w':
- dowrite = 1;
- break;
- case 'v':
- verbose = 1;
- break;
- case 'h':
- default:
- printf("Usage: %s [-c <bytes>] [-b <bytes>|-p <percent>] [-v]\n", argv[0]);
- printf("\t-c <num>\tsize of chunk in bytes to malloc on each pass\n");
- printf("\t-b <bytes>\tmaximum number of bytes to allocate before stopping\n");
- printf("\t-p <bytes>\tpercent of total memory used at which the program stops\n");
- printf("\t-w\t\twrite to the memory after allocating\n");
- printf("\t-v\t\tverbose\n");
- printf("\t-h\t\tdisplay usage\n");
- exit(-1);
- }
- }
-
- sysinfo(&sstats);
- if (maxpercent) {
+int main(int argc, char* argv[])
+{
+ char* mem;
+ float percent;
+ unsigned int maxpercent = 0, dowrite = 0, verbose=0, j, c;
+ unsigned long bytecount, alloc_bytes, max_pids;
+ unsigned long long original_maxbytes, maxbytes = 0;
+ unsigned long long pre_mem = 0, post_mem = 0;
unsigned long long total_ram, total_free, D, C;
- percent=(float)maxpercent/100.00;
-
- total_ram=sstats.totalram;
- total_ram=total_ram+sstats.totalswap;
-
- total_free=sstats.freeram;
- total_free=total_free+sstats.freeswap;
-
- /* Total memory used needed to reach maxpercent */
- D = percent*(sstats.mem_unit*total_ram);
- tst_resm(TINFO, "Total memory used needed to reach maxpercent = %llu kbytes", D/1024);
-
- /* Total memory already used */
- C = sstats.mem_unit*(total_ram-total_free);
- tst_resm(TINFO, "Total memory already used on system = %llu kbytes", C/1024);
+ extern char* optarg;
+ int chunksize = 1024*1024; /* one meg at a time by default */
+ struct sysinfo sstats;
+ int i, pid_cntr;
+ pid_t pid, *pid_list;
+ struct sigaction act;
+ act.sa_handler = handler;
+ act.sa_flags = 0;
+ sigemptyset(&act.sa_mask);
+ sigaction(SIGRTMIN, &act, 0);
+
+ while ((c = getopt(argc, argv, "c:b:p:wvh")) != EOF)
+ {
+ switch((char)c)
+ {
+ case 'c':
+ chunksize = atoi(optarg);
+ break;
+ case 'b':
+ if (maxpercent != 0) {
+ tst_resm(TFAIL, "ERROR: -b option cannot be used with -p option at the same time");
+ exit(-1);
+ }
+ maxbytes = atoll(optarg);
+ break;
+ case 'p':
+ if (maxbytes != 0) {
+ tst_resm(TFAIL, "ERROR: -p option cannot be used with -b option at the same time");
+ exit(-1);
+ }
+ maxpercent = atoi(optarg);
+ if (maxpercent <= 0) {
+ tst_resm(TFAIL, "ERROR: -p option requires number greater than 0");
+ exit(-1);
+ }
+ if (maxpercent > 99) {
+ tst_resm(TFAIL, "ERROR: -p option cannot be greater than 99");
+ exit(-1);
+ }
+ break;
+ case 'w':
+ dowrite = 1;
+ break;
+ case 'v':
+ verbose = 1;
+ break;
+ case 'h':
+ default:
+ printf("Usage: %s [-c <bytes>] [-b <bytes>|-p <percent>] [-v]\n", argv[0]);
+ printf("\t-c <num>\tsize of chunk in bytes to malloc on each pass\n");
+ printf("\t-b <bytes>\tmaximum number of bytes to allocate before stopping\n");
+ printf("\t-p <bytes>\tpercent of total memory used at which the program stops\n");
+ printf("\t-w\t\twrite to the memory after allocating\n");
+ printf("\t-v\t\tverbose\n");
+ printf("\t-h\t\tdisplay usage\n");
+ exit(-1);
+ }
+ }
+
+ sysinfo(&sstats);
+ total_ram = sstats.totalram + sstats.totalswap;
+ total_free = sstats.freeram + sstats.freeswap;
/* Total Free Pre-Test RAM */
- pre_mem = sstats.mem_unit*total_free;
-
- /* Are we already using more than maxpercent? */
- if (C>D) {
- tst_resm(TFAIL, "More memory than the maximum amount you specified is already being used");
- exit(1);
+ pre_mem = sstats.mem_unit * total_free;
+ max_pids = total_ram / (unsigned long)FIVE_HUNDRED_KB + 1;
+
+ if ((pid_list = malloc(max_pids * sizeof(pid_t))) == NULL)
+ {
+ tst_resm(TBROK|TERRNO, "malloc failed.");
+ exit(1);
}
- else
- pre_mem = sstats.mem_unit*total_free;
-
- /* set maxbytes to the extra amount we want to allocate */
- maxbytes = D-C;
- tst_resm(TINFO, "Filling up %d%% of ram which is %llu kbytes", maxpercent, maxbytes/1024);
- }
- original_maxbytes=maxbytes;
- i=0;
- pid_cntr=0;
- pid=fork();
- if (pid != 0)
- pid_cntr++;
- pid_list[i]=pid;
+ memset(pid_list, 0, max_pids * sizeof(pid_t));
+
+ /* Currently used memory */
+ C = sstats.mem_unit * (total_ram - total_free);
+ tst_resm(TINFO, "Total memory already used on system = %llu kbytes", C/1024);
+
+ if (maxpercent)
+ {
+ percent = (float)maxpercent / 100.00;
-#define FIVE_HUNDRED_KB (500*1024*1024)
-#define ONE_MEGABYTE (1024*1024*1024)
-#define THREE_MEGABYTES (3*ONE_MEGABYTE)
+ /* Desired memory needed to reach maxpercent */
+ D = percent * (sstats.mem_unit * total_ram);
+ tst_resm(TINFO, "Total memory used needed to reach maxpercent = %llu kbytes", D/1024);
+
+ /* Are we already using more than maxpercent? */
+ if (C > D)
+ {
+ tst_resm(TFAIL, "More memory than the maximum amount you specified is already being used");
+ free(pid_list);
+ exit(1);
+ }
+
+ /* set maxbytes to the extra amount we want to allocate */
+ maxbytes = D - C;
+ tst_resm(TINFO, "Filling up %d%% of ram which is %llu kbytes", maxpercent, maxbytes/1024);
+ }
+ original_maxbytes = maxbytes;
+ i = 0;
+ pid_cntr = 0;
+ pid = fork();
+ if (pid != 0)
+ pid_cntr++;
+ pid_list[i] = pid;
#if defined (_s390_) /* s390's 31bit addressing requires smaller chunks */
- while (pid != 0 && maxbytes > FIVE_HUNDRED_KB)
- {
- i++;
- maxbytes -= FIVE_HUNDRED_KB;
- pid = fork();
- if (pid != 0) {
- pid_cntr++;
- pid_list[i] = pid;
+ while (pid != 0 && maxbytes > FIVE_HUNDRED_KB)
+ {
+ i++;
+ maxbytes -= FIVE_HUNDRED_KB;
+ pid = fork();
+ if (pid != 0)
+ {
+ pid_cntr++;
+ pid_list[i] = pid;
+ }
}
- }
- if (maxbytes > FIVE_HUNDRED_KB)
- alloc_bytes FIVE_HUNDRED_KB;
- else
- alloc_bytes = (unsigned long) maxbytes;
-
+ if (maxbytes > FIVE_HUNDRED_KB)
+ alloc_bytes = FIVE_HUNDRED_KB;
+ else
+ alloc_bytes = (unsigned long) maxbytes;
+
#elif __WORDSIZE==32
- while (pid != 0 && maxbytes > ONE_MEGABYTE)
- {
- i++;
- maxbytes -= ONE_MEGABYTE;
- pid = fork();
- if (pid != 0) {
- pid_cntr++;
- pid_list[i]=pid;
+ while (pid != 0 && maxbytes > ONE_MEGABYTE)
+ {
+ i++;
+ maxbytes -= ONE_MEGABYTE;
+ pid = fork();
+ if (pid != 0)
+ {
+ pid_cntr++;
+ pid_list[i]=pid;
+ }
}
- }
- if (maxbytes > ONE_MEGABYTE)
- alloc_bytes = ONE_MEGABYTE;
- else
- alloc_bytes = (unsigned long)maxbytes;
-
+ if (maxbytes > ONE_MEGABYTE)
+ alloc_bytes = ONE_MEGABYTE;
+ else
+ alloc_bytes = (unsigned long)maxbytes;
+
#elif __WORDSIZE==64
- while (pid!=0 && maxbytes > THREE_MEGABYTES)
- {
- i++;
- maxbytes -= THREE_MEGABYTES;
- pid=fork();
- if (pid != 0) {
- pid_cntr++;
- pid_list[i] = pid;
- }
- }
- if (maxbytes > THREE_MEGABYTES)
- alloc_bytes = THREE_MEGABYTES;
- else
- alloc_bytes = maxbytes;
-#endif
-
- if (pid == 0) /** CHILD **/
- {
- bytecount=chunksize;
- while (1) {
- if ((mem = (char*)malloc(chunksize)) == NULL) {
- tst_resm(TINFO, "stopped at %lu bytes", bytecount);
- exit(1);
- }
- if (dowrite)
- for (j=0; j<chunksize; j++)
- *(mem+j)='a';
- if (verbose)
- tst_resm(TINFO, "allocated %lu bytes chunksize is %d", bytecount, chunksize);
- bytecount+=chunksize;
- if (alloc_bytes && (bytecount >= alloc_bytes))
- break;
+ while (pid!=0 && maxbytes > THREE_MEGABYTES)
+ {
+ i++;
+ maxbytes -= THREE_MEGABYTES;
+ pid = fork();
+ if (pid != 0)
+ {
+ pid_cntr++;
+ pid_list[i] = pid;
+ }
}
- if (dowrite)
- tst_resm(TINFO, "... %lu bytes allocated and used.", bytecount);
+ if (maxbytes > THREE_MEGABYTES)
+ alloc_bytes = THREE_MEGABYTES;
else
- tst_resm(TINFO, "... %lu bytes allocated only.", bytecount);
- kill(getppid(),SIGRTMIN);
- while (1)
- sleep(1);
- }
- else /** PARENT **/
- {
-
- i=0;
- sysinfo(&sstats);
+ alloc_bytes = maxbytes;
+#endif
- if (dowrite)
+ if (pid == 0) /** CHILD **/
{
- /* Total Free Post-Test RAM */
- post_mem = (unsigned long long)sstats.mem_unit*sstats.freeram;
- post_mem = post_mem+((unsigned long long)sstats.mem_unit*sstats.freeswap);
-
- while ((((unsigned long long)pre_mem - post_mem) < (unsigned long long)original_maxbytes) &&
- (pid_count < pid_cntr) )
- {
- sleep(1);
- sysinfo(&sstats);
- post_mem = (unsigned long long)sstats.mem_unit*sstats.freeram;
- post_mem = post_mem+((unsigned long long)sstats.mem_unit*sstats.freeswap);
- }
+ bytecount = chunksize;
+ while (1)
+ {
+ if ((mem = malloc(chunksize)) == NULL)
+ {
+ tst_resm(TBROK|TERRNO, "stopped at %lu bytes", bytecount);
+ free(pid_list);
+ exit(1);
+ }
+ if (dowrite)
+ for (j = 0; j < chunksize; j++)
+ *(mem+j) = 'a';
+ if (verbose)
+ tst_resm(TINFO, "allocated %lu bytes chunksize is %d", bytecount, chunksize);
+ bytecount += chunksize;
+ if (alloc_bytes && bytecount >= alloc_bytes)
+ break;
+ }
+ if (dowrite)
+ tst_resm(TINFO, "... %lu bytes allocated and used.", bytecount);
+ else
+ tst_resm(TINFO, "... %lu bytes allocated only.", bytecount);
+ kill(getppid(), SIGRTMIN);
+ while (1)
+ sleep(1);
}
- while (pid_list[i]!=0)
+ else /** PARENT **/
{
- kill(pid_list[i],SIGKILL);
- i++;
+ i = 0;
+ sysinfo(&sstats);
+
+ if (dowrite)
+ {
+ /* Total Free Post-Test RAM */
+ post_mem = (unsigned long long)sstats.mem_unit * sstats.freeram;
+ post_mem = post_mem + (unsigned long long)sstats.mem_unit * sstats.freeswap;
+
+ while ((((unsigned long long)pre_mem - post_mem) < (unsigned long long)original_maxbytes) &&
+ (pid_count < pid_cntr))
+ {
+ sleep(1);
+ sysinfo(&sstats);
+ post_mem = (unsigned long long)sstats.mem_unit * sstats.freeram;
+ post_mem = post_mem + (unsigned long long)sstats.mem_unit * sstats.freeswap;
+ }
+ }
+ while (pid_list[i] != 0)
+ {
+ kill(pid_list[i], SIGKILL);
+ i++;
+ }
+ if (dowrite)
+ tst_resm(TPASS, "%llu kbytes allocated and used.", original_maxbytes/1024);
+ else
+ tst_resm(TPASS, "%llu kbytes allocated only.", original_maxbytes/1024);
}
- if (dowrite)
- tst_resm(TPASS, "%llu kbytes allocated and used.", original_maxbytes/1024);
- else
- tst_resm(TPASS, "%llu kbytes allocated only.", original_maxbytes/1024);
- }
- exit(0);
-}
\ No newline at end of file
+ free(pid_list);
+ exit(0);
+}
--
1.7.4.rc1
[-- Attachment #3: Type: text/plain, Size: 372 bytes --]
------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand
malware threats, the impact they can have on your business, and how you
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
[-- Attachment #4: Type: text/plain, Size: 155 bytes --]
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list
^ permalink raw reply related [flat|nested] 8+ messages in thread
end of thread, other threads:[~2011-01-18 9:13 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-01-06 18:40 [LTP] [PATCH v4] make pid_list dynamically sized with memory czhang
2011-01-06 18:42 ` [LTP] Final version [Re: [PATCH v4] make pid_list dynamically sized with memory] Caspar Zhang
2011-01-11 9:26 ` Caspar Zhang
2011-01-16 11:05 ` [LTP] [PATCH v4] make pid_list dynamically sized with memory Caspar Zhang
2011-01-18 8:29 ` Caspar Zhang
2011-01-18 8:33 ` Garrett Cooper
2011-01-18 8:45 ` Caspar Zhang
2011-01-18 9:14 ` [LTP] [PATCH v5] " Caspar Zhang
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.