All of lore.kernel.org
 help / color / mirror / Atom feed
* [LTP] [PATCH 1/7 ver2 w/attachment] Updated pounder's documentation
@ 2011-08-15 18:56 Lucy Liang
  2011-08-15 18:56 ` [LTP] [PATCH 6/7 ver2 w/attachment] Modified pounder install and run scripts Lucy Liang
  0 siblings, 1 reply; 8+ messages in thread
From: Lucy Liang @ 2011-08-15 18:56 UTC (permalink / raw)
  To: ltp-list

[-- Attachment #1: Type: text/plain, Size: 1112 bytes --]


Updated the README and SCHEDULER doc files. Merged QUICKSTART
with README. README gives an overview of building/running pounder
and how its files/subdirectories are organized. SCHEDULER goes
into detail about how to create, modify, build, and run tests and
test schedulers. Also added a new CONFIGURATION document that
explains pounder's environment variables. CONFIGURATION and
SCHEDULER are now located in a newly created doc/ directory.

Signed-off-by: Lucy Liang <lgliang@linux.vnet.ibm.com>
---
 tools/pounder21/CHANGELOG         |   59 ++++++
 tools/pounder21/QUICKSTART        |   20 --
 tools/pounder21/README            |  272 ++++++++++++++++++++---------
 tools/pounder21/SCHEDULER         |  150 ----------------
 tools/pounder21/doc/CONFIGURATION |  107 +++++++++++
 tools/pounder21/doc/SCHEDULER     |  352 +++++++++++++++++++++++++++++++++++++
 6 files changed, 711 insertions(+), 249 deletions(-)
 delete mode 100644 tools/pounder21/QUICKSTART
 delete mode 100644 tools/pounder21/SCHEDULER
 create mode 100644 tools/pounder21/doc/CONFIGURATION
 create mode 100644 tools/pounder21/doc/SCHEDULER


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Updated-pounder-s-documentation.patch --]
[-- Type: text/x-patch; name="0001-Updated-pounder-s-documentation.patch", Size: 44131 bytes --]

diff --git a/tools/pounder21/CHANGELOG b/tools/pounder21/CHANGELOG
index c248d96..4543f3a 100644
--- a/tools/pounder21/CHANGELOG
+++ b/tools/pounder21/CHANGELOG
@@ -1,5 +1,64 @@
 This is a log of changes to pounder21.
 
+pounder30-2011-08-09
+- Created new documentation CONFIGURATION and moved it and SCHEDULER
+  into a newly created doc/ directory
+- Deleted the test-all test scheduler
+- Created /schedulers directory and moved the remaining test schedulers there
+- Removed option to specify "NONE" when asked to unpack test scheduler during build
+- Removed check for existing kernel directory in /tmp in test_scripts/build_kernel
+  since it appears that some files get lost after running build_kernel once; Instead
+  just untar the kernel each time we run the test script to be on the safe side
+- ltp test script would pass even if it didn't build currently, fixed this in
+  test_scripts/ltp
+- changed ltp build_script to install ltp to $POUNDER_TMPDIR
+- removed QUICKSTART and included it in README instead
+- removed trailing "/" from POUNDER_LOGLOCAL export in libpounder.sh
+- Added functionality for automatic skipping of subtests (see README)
+- Created xterm_stress build script and merged 00xbonkers with it
+- Created ide_cdrom_copy build script and merged 00check_cdrom_presence with it
+- Merged nasm and schedutils build scripts with the lame build script
+- Merged time_test build script with the time_consistency and time_drift build
+  scripts
+- Created test_repo/ directory
+- Uncommented a piece of code in time_drift that allowed it to always pass
+- Added pounder -c option for creating new test schedulers
+- Modified POUNDER_VERSION in libpounder.sh
+
+pounder30-2011-07-21
+- Updated bonnie++, ipmitool, kernel (used in build_kernel), and memtest script to latest versions
+- Updated memtest build scripts and $POUNDER_HOME/src/memtest.patch
+- Added functionality for skipping of subtests
+  - Added functionality for automating the skipping of subtests (see README)
+- Removed unnecessary 00checklatest test
+- Moved checking for system requirements from test run to build phase
+  - Affects bonnie++, memtest, cpufreq, and ide_cdrom_copy
+- Added environment variable MAX_FAILURES that, if defined, sets
+  an upper bound on the number of failures a looped test will allow
+  before aborting the test altogether (see SCHEDULER)
+- Added functionality for removing and re-adding subtests to the test scheduler (see SCHEDULER)
+- Updated README, SCHEDULER, and config files
+
+pounder21-2011-04-08:
+- LTP: Updated to LTP 20101031 release.
+- Build kernel testcase - Updated kernel from 2.6.18 to 2.6.38.
+- Updated 2.6.38 kernel source tar in pounder cache.
+- Did corresponding kernel changes i.e for 2.6.38 in "memtest" testcase too.
+- Files modified are:-
+  -$POUNDER_HOME/test_scripts/memtest.
+  -$POUNDER_HOME/test_scripts/build_kernel .
+  -$POUNDER_HOME/build_scripts/memtest.
+  -$POUNDER_HOME/build_scripts/build_kernel.
+  -$POUNDER_HOME/opt/memtest.sh. [Actually this file need to get changed in tux1 cache].
+
+pounder21-2011-04-12
+-Integrated bash-memory testcase in pounder21
+-Files added/modified are:-
+ -Copied bash-memory test case tar to pounder cache.
+ -Added file $POUNDER_HOME/build_scripts/bash-memory
+ -Added file $POUNDER_HOME/test_scripts/bash_memory
+ -Added file $POUNDER_HOME/tests/T10single/T03bashmemory
+
 pounder21-2006-11-07:
 - Fix a bug in randasys on x86-64 where we had insufficient random bits and
   would truncate whatever we got, leading to all 0 arguments by simply
diff --git a/tools/pounder21/QUICKSTART b/tools/pounder21/QUICKSTART
deleted file mode 100644
index d2a0ef4..0000000
--- a/tools/pounder21/QUICKSTART
+++ /dev/null
@@ -1,20 +0,0 @@
-Quick and Dirty Guide to Setting Up Pounder
-Copyright (C) 2003-2006 IBM.
-
-0. Install system.  gcc and related development packages are required by
-   pounder; for extra stress testing, enable X too.
-1. Download and unpack the LTP tarball.  You've already done this.
-2. cd testcases/pounder21/.  You've already done this too.
-3. Set up a NFS server to export "/pounder".
-4. Add the following to "config":
-
-export DO_X_TESTS=1
-export NFS_SERVER=<your NFS server here>
-export NTP_SERVER=pool.ntp.org
-
-5. ./Install
-6. ./pounder
-
-This should take approximately two days to run to completion.  If your machine
-hangs, you can use the magic SysRq key (if you built it into the kernel) to
-obtain debugging information, kdumps, etc.
diff --git a/tools/pounder21/README b/tools/pounder21/README
index fd5a252..6621a14 100644
--- a/tools/pounder21/README
+++ b/tools/pounder21/README
@@ -1,46 +1,211 @@
-This is pounder21, as of 2006-01-24.  Copyright (C) 2003-2006 IBM.
-The above line is used to detect the pounder release version.  If
-you change the line, be sure to update build_scripts/00checklatest.
+This is pounder30 as of 2011-08-09.  Copyright (C) 2003-2011 IBM.
+(Do not delete top line. It is used for version checking.)
+
+It would be a good idea to read this README file and the SCHEDULER and
+CONFIGURATION files (in the doc/ directory) prior to dabbling with pounder!
 
 Licensing
 =========
 All files in this tarball are licensed under the GPL.  Please see
 the file COPYING for more details.
 
-Overview
+Contents
 ========
-This package is a system stress test.
-
-Unlike the original pounder, pounder21 *does* report pass/fail and
-it is NOT infinite.  There is a -k option to kill the tests manually,
-though ^C in the pounder terminal works.  With the new scheduler,
-one can enforce an order in which tests are run, and even control
-which ones get run in parallel.  See SCHEDULER for details.
-
-There are only two prerequisites:
-
-    - Get a CD with some data on it and put in the drive.
-      This is used to test optical drives, which are typically
-      the only IDE devices on a server.
-    - Make sure you can mount an NFS server.  See libpounder.sh for
-      config details.
+1. Overview
+2. Getting Started
+3. Files and Directories
+4. The Install Script
+5. Configuration
+6. The Pounder Script
+7. Credits
 
-If you downloaded the source tarball, then do this to get started:
+Overview
+========
+Pounder provides a framework for building, running, and logging results
+for user-defined sets of tests.  Almost any test or test suite may be run
+as a subtest from within this framework, including the LTP test suite.
+(For more guidelines on building, scheduling, and running user-defined
+subtests, see doc/SCHEDULER)
+
+Getting Started
+===============
+
+Some sample test "schedules" comprised of various publically available
+tests, including LTP, are provided. The default test schedule illustrates
+how one might use pounder and is also a useful general purpose stress test.
+
+The following steps describe how to build and run the default schedule:
+
+	0. Install your operating system.  gcc and related development packages are
+		required to build pounder.  Missing dependencies will be identified at
+		build time. X development packages are needed for the included video test.
+
+	1. Download and unpack the LTP tarball.  You've already done this.
+
+	2. cd tools/pounder21/.  You've already done this too.
+
+	3. (optional) Set up a NFS server to export "/pounder21" (unless you wish 
+		to skip nfs tests).
+
+	4. (optional) Modify any variables in "config" (see doc/CONFIGURATION for details).
+
+	5. Run "make install" to build tests for your machine
+		The Install script will attempt to build all the subtests in the
+		build_scripts folder. It will prompt you for the test scheduler
+		you want to unpack. Go ahead of type "default" or simply press
+		enter. It will then ask if you want to automate skipping of
+		failed subtests. If you enter "y", the script will automatically
+		delete any subtests from the test scheduler that fail to build.
+		If you enter "n", the script will prompt you each time a test
+		fails to build on whether or not to skip the failed test.
+       
+	6. Run "./pounder" to start the tests (run "./pounder -h" for usage options).
+
+	7. Press ^C or run "./pounder -k" to stop the tests
+		The default scheduler runs tests for 48 hours, but you can set a new
+		duration in seconds by modifying config (see doc/CONFIGURATION for details)
+		or by using the -d option when starting pounder (./pounder -d <duration in seconds>)
+
+	8. Run "make mrclean" to restore everything to the state before the tarball 
+		was unpacked (running this command will of course require you to 
+		rebuild with "make install" for the next pounder run)
+
+See doc/SCHEDULER for details on defining the order in which tests are run, and whether they
+are run serially or in parallel.
+
+A few of the sample subtests have prerequisites:
+
+	- ide_cdrom_copy: Requires a CD with some data on it to be put in the drive.
+
+	- nfs, ping_nfs: Make sure you can mount an NFS server. Specify NFS in config
+		or run "./pounder -n ipaddr"
+
+	- xterm_stress: Make sure you can start X sessions. Enable X testing by setting
+		the DO_X_TESTS flag in config or run "./pounder -x"
+
+These tests can be skipped during the build phase if reqs aren't met though.
+
+Files and Directories
+=====================
+Below are brief descriptions of the files and directories found under the pounder/
+directory.
+
+Files:
+
+	CHANGELOG
+		- A log of changes made to pounder
+	COPYING
+		- GNU general public license info
+	Install
+		- The script used to build pounder
+	Makefile
+		- Makefile for pounder
+	debug.c
+		- Debugging routines used for logging pounder results
+	infinite_loop.c, timed_loop.c, fancy_timed_loop.c
+		- Procedures used to run tests repeatedly (see doc/SCHEDULER for more
+		information)
+	config
+		- Environment variables used for customizing pounder run are defined
+		here (see doc/CONFIGURATION for details)
+	libpounder.sh
+		- More environment variables defined here. Unlike the ones in config,
+		these are not meant to be modified by the user. (see doc/CONFIGURATION
+		for details)
+	nfs_logging
+		- Script that sets up user-defined NFS server for logging pounder output.
+		This script is executed when pounder is run with $NFS_LOGGING enabled in
+		config (see doc/CONFIGURATION) or when "pounder -r" is used. Normally when
+		running pounder, test output will be directly logged to $POUNDER_LOGLOCAL,
+		but with NFS logging enabled, output will instead be logged to user-specified
+		remote directory of an NFS server, $NFS_LOGSERVER:$NFS_LOGDIR.
+		See doc/CONFIGURATION for more information on these variables.
+	pounder
+		- Script used to run pounder. (see "The Pounder Script" section below
+		for details)
+	proclist.c
+		- Manages list of processes during pounder run.
+	README
+		- This file, which gives an overview of pounder's structure and how to
+		build and start pounder.
+	run.c
+		- Program to run the tests in the test scheduler.
+
+Directories:
+
+	build_scripts/
+		- Scripts to build your subtests go here. (see doc/SCHEDULER for details)
+	doc/
+		- Contains the SCHEDULER file, which describes how to create, build,
+		schedule, and run your own tests with pounder.
+		- Contains the CONFIGURATION file, which describes pounder's environment
+		variables.
+	schedulers/
+		- Test scheduler tarballs are in here. (see doc/SCHEDULER for details)
+	src/
+		- Sources packaged with pounder are in here.
+	test_repo/
+		- This directory is a copy of the default test scheduler. It provides an
+		example of what an test scheduler should look like after unpacking.
+	test_scripts/
+		- Scripts to run your subtests go here. (see doc/SCHEDULER for details)
+	tests/
+		- Symlinks to run the tests in a particular order. (see doc/SCHEDULER for
+		details)
+
+After running "make install," you will see three additional directories:
+
+	opt/
+		- Third party packages (LTP, kernel, etc) go here.
+	tmp/
+		- Temporary directory to hold files that a test needs.
+	log/
+		- Logs of output from pounder runs go here.
+
+Note that for the provided tests, third party test packages (bonnie, kernel, etc) aren't
+packaged with pounder. The build scripts should download them to opt/ (stored in
+$POUNDER_OPTDIR) and build them as necessary. The use of a cache might come in handy here
+(see doc/CONFIGURATION for details regarding the $POUNDER_CACHE variable).
 
-    1. Run "./Install" - this builds tests for your machine
-       Proceed to step 2.
+The Install Script
+==================
+The Install script has no options.  Run it to build whatever tests have been
+imported into the pounder package.
 
-If you downloaded a binary tarball, then do this to start testing:
+Configuration
+=============
+See doc/CONFIGURATION documentation file for details.
 
-    2. Run "./pounder" (see pounder script for usage options) to 
-        start the tests
+The Pounder Script
+==================
+The pounder script has the following syntax:
 
-    3. When you want to stop the tests press ^C or run "./pounder -k"
+Usage: ./pounder [-g logdir] [-x] [-d duration] [-n ipaddr] [-m max_failures] [-f] [-h|-u|-r|-k|-l|-e subtests|-i subtests|-c scheduler] [-s]
+
+-h              Brings up this menu
+-c scheduler    Creates a new test scheduler called scheduler-tests.tar.gz in the pounder/schedulers folder.
+                All subtests to be packaged with this scheduler must first be placed in the pounder/tests folder.
+-x              Enable X stress tests.
+-d duration     Run pounder for duration seconds.
+-n ipaddr       Use ipaddr for NFS tests.
+-f              Remove pounder pid file before running.
+-u              Unmount NFS log storage.
+-r              Remount NFS log storage.
+-g logdir       Use logdir as the log directory. (You probably want -s too.)
+-s              Store logs locally.
+-l              List (both included and excluded) subtests that came with the test scheduler
+-e subtests     Exclude subtests from next pounder run
+-i subtests     Include previously excluded subtests in the next pounder run
+-k              Kill pounder.
+
+run "./pounder" to run all subtests
+run "./pounder subtest" to run just one particular subtest
+        (example: ./pounder tests/T90ramp/D02build_kernel)
 
 Credits
 =======
 o Inspired by Sammy Benjamin (sammy@us.ibm.com).  None of his code remains
-  in pounder21 today.
+  in this version of pounder today.
 o Modifications and additions by members of the LTC xSeries Kernel Team:
     Darrick Wong (djwong@us.ibm.com)
     Chris McDermott (lcm@us.ibm.com)
@@ -56,6 +221,7 @@ o Modifications and additions by members of the LTC xSeries Kernel Team:
     Pradeep Kumar (pradeepkumars@in.ibm.com)
     Bhaskar Rangaswamy (bharanga@in.ibm.com)
     Manikandan Chidambaram (cmanikandan@in.ibm.com)
+    Lucy Liang (lgliang@us.ibm.com)
 o Other contributers:
 
 Also utilizes:
@@ -71,55 +237,3 @@ o schedutils, to test CPU affinity (with lame)
 
 (note that the above packages are not distributed with pounder
  and are simply installed by the installer script)
-
-Improvements in pounder2
-========================
-This iteration of pounder you're looking at has been reworked a bit.
-First, pounder environment variables are defined in libpounder.sh.
-There are a lot more of them than there used to be.
-
-Second, I've attempted to make test integration a bit easier than it
-used to be.  There are several new directories; they are detailed below.
-
-build_scripts/   Put a script to build your test in here.
-config/          Put config files for your test in here.
-src/             Sources packaged with pounder go in here.
-opt/             Third party packages (LTP, kernel, etc) go here.
-tmp/             Temporary directory for files that a test needs.
-test_scripts/    Put a script to run your test in here.
-tests/           Symlinks to run the tests in a particular order.
-                 See the SCHEDULER documentation file for details.
-log/             Logs from pounder runs go here.
-
-Like before, all the tests are run in parallel.
-
-Note that third party test packages (LTP, kernel, etc) aren't packaged
-with pounder; the build scripts should download them and build them
-as necessary.
-
-The Install Script
-==================
-The Install script has no options.  Run it to build whatever tests have been
-imported into the pounder package.
-
-Configuration
-=============
-See the Configuration section of the SCHEDULER documentation file for details.
-See libpounder.sh for the actual configuration variables.
-
-The pounder Script
-==================
-The pounder script has the following syntax:
-
-Usage: ./pounder [-l logdir] [-x] [-d duration] [-n ipaddr] [-f] [-u|-r|-k] [-s]
-
--x      Enable X stress tests.
--d      Run pounder for duration seconds.
--n      Use ipaddr for NFS tests.
--f      Remove pounder pid file before running.
--u      Unmount NFS log storage.
--r      Remount NFS log storage.
--l      Use logdir as the log directory. (You probably want -s too.)
--s      Store logs locally.
--k      Kill pounder.
-
diff --git a/tools/pounder21/SCHEDULER b/tools/pounder21/SCHEDULER
deleted file mode 100644
index 92ace7b..0000000
--- a/tools/pounder21/SCHEDULER
+++ /dev/null
@@ -1,150 +0,0 @@
-This document describes the operation and configuration of the test scheduling
-framework in the pounder2 package.  This document reflects pounder21 as of
-2004-12-20, though the scheduler isn't likely to change much.
-
-Author: Darrick Wong <djwong@us.ibm.com>
-Copyright (C) 2004 IBM.
-
-Overview
-========
-The scheduler in the original pounder release was too simplistic--it would kick
-off every test at once, simultaneously.  There was no attempt to ramp up the
-machine's stress levels test by test, or to run only certain combinations, or
-even run the tests one by one before beginning the real load testing.
-
-In addition, the test scripts had a very simple pass/fail mechanism--failure
-was defined by a kernel panic/oops/bug, and passing was defined by the lack of
-that condition.  There was no attempt to find soft failures--situations where
-a test program would fail, but without bringing the machine down.  The test
-suite would not alert the user that these failures had occurred.
-
-Consequently, Darrick Wong rewrote the test scheduling framework to achieve
-several goals--first, to separate the test automation code from the tests
-themselves, to allow for more intelligent scheduling of tests, to give better
-summary reports of what passed (and what didn't), and finally to improve the
-load testing that this suite could do.
-
-Configuration File
-==================
-The test suite's configuration file is $POUNDER_HOME/libpounder.sh.  That file
-defines several variables that are referenced throughout the rest of this 
-document.  Obviously, these variables should be customized for your particular
-machine's set up.
-
-Files
-=====
-Each test has to provide at a bare minimum, two files:
-
-build_scripts/<testname> and test_scripts/<testname>.
-
-Temporary files should go in $POUNDER_TMPDIR; source and binaries should go
-in $POUNDER_OPTDIR.
-
-Build Scripts
-=============
-As the name implies, the script in build_scripts is in charge of downloading
-and building whatever bits of code are necessary to make the test run.  The
-variable $POUNDER_CACHE defines a web-accessible directory containing cached
-tarballs of whatever it is you're trying to build.
-
-Should there be a failure in the build script that is essential to the ability
-to run a test, the build script should return 0 to halt the main build process
-immediately.
-
-Also, be aware that distributing pre-built binary tarballs is not always a good
-idea because distros are not always good at ABI/library path compatibility,
-despite the efforts of LSB, FHS, etc.  These will go away in the (very) long
-term, however.
-
-Anatomy of a Test Script
-========================
-The requirements on test scripts are pretty light.  First, the building of the
-test ought to go in the build script unless it's absolutely necessary to build
-a test component at run time.
-
-Second, the script must catch SIGTERM and clean up after itself.  SIGTERM is
-used by the test scheduler to stop tests.
-
-The third requirement is much more stringent: Return codes.  The script should
-return 0 to indicate success, 1-254 to indicate failure (the common use is to
-signify the number of failures), and -1 or 255 to indicate that the there was
-a failure that cannot be fixed.
-
-Note: If a test is being run in a timed or infinite loop, returning -1 or 255
-has the effect of cancelling all subsequent loops.
-
-Quick map of return codes to what gets reported:
-0             = "PASS"
--1            = "ABORT"
-255           = "ABORT"
-anything else = "FAIL"
-
-Also note: If a test is killed by an unhandled signal, the test is reported as
-failing.
-
-Scheduling Tests
-================
-The current test scheduler borrows a System V rc script-like structure for
-specifying how and when tests should be run.  Everything under the tests/
-directory is used for scheduling purposes; files under that tree should have
-names that follow the following standard:
-
-   [type][sequence number][name]
-
-"type" is the type of test.  Currently, there are two types, 'D' and 'T'.  'T'
-signifies a test, which means that the scheduler starts the test, waits for the
-test to complete, and reports on its exit status.  'D' signifies a daemon
-"test", which is to say that the scheduler will start the test, let it run in
-the background, and kill it when it's done running all the tests in that
-directory.
-
-The "sequence number" dictates the order in which the test are run.  00 goes
-first, 99 goes last.  Tests with the same number are started simultaneously,
-regardless of the type.
-
-"name" is just a convenient mnemonic to distinguish between tests.
-
-File system objects under the tests/ directory can be nearly anything--
-directories, symbolic links, or files.  The test scheduler will not run
-anything that doesn't have the execute bit set.  If a FS object is a
-directory, then the contents of the directory are executed sequentially.
-
-Running Tests Repeatedly
-========================
-Two helper programs are provided to run tests repeatedly:
-
-    timed_loop duration command [arguments]
-
-This program will run "command" repeated until the number of seconds given
-as "duration" has passed.
-
-    infinite_loop command [arguments]
-
-This program runs "command" repeatedly until sent SIGTERM.
-
-Examples
-========
-
-Let's examine the following test scheduler hierarchy:
-
-tests/
-    D00stats
-    T01foo
-    T01bar
-    T02dir/
-        T00gav -> ../../test_scripts/gav
-        T01hic -> ../../test_scripts/hic
-    T03lat
-
-Let's see how the tests are run.  The test scheduler will start off by scanning
-the tests/ directory.  First it spawns D00stats and lets it run in the
-background.  Next, T01foo and T01bar are launched at the same time; the
-scheduler will wait for both of them to complete before proceeding.  Since T01foo
-is a file and not just a symbolic link, there is a fair chance that T01foo runs
-some test in a loop for a certain amount of time.  In any case, the scheduler
-next sees T02dir and proceeds into it.
-
-In the T02dir/, we find two test scripts.  First T00gav runs, followed by
-T01hic.  Now there are no more tests to run in T02dir/, so the scheduler heads
-back up to the parent directory.  T03lat is forked and allowed to run to
-completion, after which D00stats is killed, and the test suite exits.
diff --git a/tools/pounder21/doc/CONFIGURATION b/tools/pounder21/doc/CONFIGURATION
new file mode 100644
index 0000000..8557ea9
--- /dev/null
+++ b/tools/pounder21/doc/CONFIGURATION
@@ -0,0 +1,107 @@
+This document describes the environment variables found in the pounder30 package
+as of 2011-8-09.
+
+Author:
+Lucy Liang <lgliang@us.ibm.com>
+	
+Copyright (C) 2011 IBM.
+
+Contents
+========
+1. The libpounder.sh File
+2. The config File
+
+The libpounder.sh File
+======================
+The "libpounder.sh" file defines most of the environment variables used in the test
+suite and referenced throughout the documentation. These variables are not
+intended to be modified by the user, although they can be if customization is desired.
+Below is a brief description of these variables (see libpounder.sh for details):
+
+	DATE			- the current date, used for logging
+	DEFAULT_SCHEDPACK	- name of the default test scheduler, "default"
+	TESTS			- list of scripts from the test_scripts/ directory
+	BUILDS			- list of scripts from the build_scripts/ directory
+	POUNDER_HOME		- the pounder/ directory
+	POUNDER_PIDFILE		- pid file created when running pounder
+	POUNDER_LOGLOCAL	- the log/ directory where output of ALL pounder runs
+				get logged
+	POUNDER_LOGDIR		- the log/$DATE directory where output of only the
+				current pounder run get logged
+	POUNDER_TMPDIR		- the tmp/ directory used for storing temporary files
+				used for test runs
+	POUNDER_OPTDIR		- the opt/ directory used for storing third party packages
+				used by subtests, which can be fetched from web or from
+				a user-created cache (see $POUNDER_CACHE below)
+	POUNDER_SRCDIR		- the src/ directory containing source files packaged with
+				pounder
+	POUNDER_VERSION		- the pounder version
+	NR_CPUS			- the number of cpus on the system
+
+The config File
+===============
+The "config" file defines a few environment variables that ARE intended to be modified
+by the user for customizing pounder runs. The variables are described below:
+
+	DURATION		- Time in seconds for pounder to run. Setting this variable
+				to 0 will not put an upper bound on pounder run time.
+
+	MAX_FAILURES		- Maximum number of test failures allowed for each subtest
+				using infinite_loop or timed_loop (see the "Running Tests Repeatedly"
+				section in SCHEDULER for more info on these two procedures) before aborting.
+				Setting this variable to 0 will not put an upper bound on any
+				subtest failures.
+
+
+	NFS_LOGGING             - Enables/disables NFS logging. Setting this variable to
+				1 will enable NFS logging of pounder output; pounder will
+                             	log output to remote directory on NFS server specified
+                                by $NFS_LOGDIR and  $NFS_LOGSERVER (see below), which
+                                will be mounted on $POUNDER_LOGLOCAL (see libpounder.sh).
+                                Setting this variable to 0 will disable this feature; all
+				output for pounder runs will be stored locally directly in
+				$POUNDER_LOGLOCAL instead.
+
+	NFS_LOGSERVER           - IP address of the NFS server to use for logging pounder results.
+				NFS_LOGGING should be enabled to use this feature.
+
+	NFS_LOGDIR		- Path to the log directory on $NFS_LOGSERVER; If $NFS_LOGGING
+				is enabled, pounder will attempt to mount $NFS_LOGSERVER:$NFS_LOGDIR/
+				on $POUNDER_LOGLOCAL (see libpounder.sh).
+
+	POUNDER_CACHE  		- Address of the cache to use for fetching outside packages,
+                        	The cache is a user-created web-accessible directory
+                        	containing cached tarballs/scripts/etc. used for
+                        	the various tests you intend to build. This is optional
+                        	but useful for saving download time and keeping everything in one place.
+
+				For instance, the build_kernel subtest requires downloading a
+				linux kernel tarball during build time (see build_scripts/build_kernel).
+				Instead of calling "wget http://www.kernel.org/pub/linux/kernel/v2.6/$TARNAME"
+				to retrieve the tarball, we can pre-download and store it in a user-created online
+				directory, then call "wget ${POUNDER_CACHE}${TARNAME}," where POUNDER_CACHE
+				is the address of the directory. Other provided subtests: bonnie++, lame, ipmitool,
+				and memtest also make use of this cache.
+
+				Examples of some things you may want to include in your cache for building
+				the provided tests:
+					bonnie++-1.03e.tgz	(for the bonnie++ subtest)
+					linux-2.6.39.tar.gz	(for the build_kernel subtest)
+					ipmitool-1.8.9.tar.gz	(for the ipmitool subtest)
+					...
+
+				These can be found in $POUNDER_OPTDIR after you run "make install" on the
+				default package.
+
+	[These variables below are used by specific subtests contained in the provided default
+	test scheduler, but they can be incorporated into other user-defined subtests as well.]
+
+	DO_X_TESTS		- 0 disables X system testing, 1 enables X system testing.
+				Used by the xterm_stress subtest.
+
+
+	NFS_SERVER=0		- IP address of the NFS server to use for nfs and ping_nfs
+				subtests. Setting this variable to 0 disables nfs testing.
+
+	NTP_SERVER		The NTP server to use. By default, it's set to pool.ntp.org.
+				Used by the time_drift subtest.
diff --git a/tools/pounder21/doc/SCHEDULER b/tools/pounder21/doc/SCHEDULER
new file mode 100644
index 0000000..2df082a
--- /dev/null
+++ b/tools/pounder21/doc/SCHEDULER
@@ -0,0 +1,352 @@
+This document describes the operation of the test scheduling framework in
+the pounder30 package.  This document reflects pounder30 as of 2011-8-09.
+
+Authors:
+Darrick Wong <djwong@us.ibm.com>
+Lucy Liang <lgliang@us.ibm.com>
+	
+Copyright (C) 2011 IBM.
+
+Contents
+========
+1. Overview
+2. Test Files
+3. Build Scripts
+4. Test Scripts
+5. Scheduling Tests
+6. Running Tests Repeatedly
+7. The Provided Test Schedulers
+8. Creating Your Own Test Scheduler
+9. Including and Excluding Tests
+
+Overview
+========
+The scheduler in the original pounder release was too simplistic--it would kick
+off every test at once, simultaneously.  There was no attempt to ramp up the
+machine's stress levels test by test, or to run only certain combinations, or
+even run the tests one by one before beginning the real load testing.
+
+In addition, the test scripts had a very simple pass/fail mechanism--failure
+was defined by a kernel panic/oops/bug, and passing was defined by the lack of
+that condition.  There was no attempt to find soft failures--situations where
+a test program would fail, but without bringing the machine down.  The test
+suite would not alert the user that these failures had occurred.
+
+Consequently, Darrick Wong rewrote the test scheduling framework to achieve
+several goals--first, to separate the test automation code from the tests
+themselves, to allow for more intelligent scheduling of tests, to give better
+summary reports of what passed (and what didn't), and finally to improve the
+load testing that this suite could do.
+
+Test Files
+==========
+Each test should only need to provide three files:
+
+1) build_scripts/<testname>
+	- The build_scripts/ directory contains scripts that take care of checking for
+	system requirements, downloading the relevant packages and binaries, and building
+	any code necessary to run the subtests. See the "Build Scripts" section below for
+	more information.
+
+2) test_scripts/<testname>
+	- The test_script/ directory contains scripts that take care of running the actual tests.
+	See the "Test Scripts" section below for more information.
+
+3) tests/.../[T|D]XX<testname>
+	- The tests/ directory represents our unpackaged "test scheduler" (if your tests/
+	directory is empty, that means you haven't unpacked any test schedulers yet and will
+	need run "make install" to unpack a scheduler - see "The Provided Test Schedulers"
+	section for more information. The test_repo/ directory also provides an example of what
+	an unpacked test scheduler should look like). The files in the tests/ directory are
+	usually symlinks that point to files in test_scripts/. The order in which the subtests are
+	run upon starting pounder depends on how the files in tests/ are named and organized.
+	See the "Scheduling Tests" section below for more information.
+
+Note: <testname> should be the same in the build_scripts/, test_scripts/, and tests/ folders.
+(Example: build_scripts/subtest1, test_scripts/subtest1, and tests/D99subtest1 would be valid.
+build_scripts/subtest1, test_scripts/subtest1_different, and tests/D99subtest1 would not.)
+See "Scheduling Tests" below for a detailed description of naming rules for files in the tests/
+directory.
+
+Build Scripts
+=============
+As the name implies, a script in build_scripts/ is in charge of downloading
+and building whatever bits of code are necessary to make the test run.
+
+Temporary files needed to run a test should go in $POUNDER_TMPDIR. Third party source,
+packages, binaries should go in $POUNDER_OPTDIR. Third party packages can be fetched
+from the web or from a user-created cache, a web-accessible directory containing
+cached tarballs and files used for whatever it is you'll need to build.
+(see "$POUNDER_CACHE" in doc/CONFIGURATION for more information)
+
+Should there be a failure in the build script that is essential to the ability
+to run a test, the build script should exit with error to halt the main build 
+process immediately.
+
+Also, be aware that distributing pre-built binary tarballs is not always a good
+idea. Though one could cache pre-built binary tarballs rather than source, it may 
+not be a good idea because distros are not always good at ABI/library path compatibility,
+despite the efforts of LSB, FHS, etc.  It is always safest to build your
+subtests from source on your target system.
+
+The build_scripts/ directory provides some examples.
+
+Test Scripts
+============
+A script in test_scripts/ is in charge of running the actual test.
+
+The requirements on test scripts are pretty light.  First, the building of the
+test ought to go in the build script unless it's absolutely necessary to build
+a test component at run time. Any checking for system requirements should also
+go in the build script.
+
+Second, the script must catch SIGTERM and clean up after itself.  SIGTERM is
+used by the test scheduler to stop tests.
+
+The third requirement is much more stringent: Return codes.  The script should
+return 0 to indicate success, 1-254 to indicate failure (the common use is to
+signify the number of failures), and -1 or 255 to indicate that the there was
+a failure that cannot be fixed.
+
+Note: If a test is being run in a timed or infinite loop (see the
+"Running Tests Repeatedly" section below for details), returning -1 or 255
+has the effect of cancelling all subsequent loops.
+
+Quick map of return codes to what gets reported:
+0             = "PASS"
+-1            = "ABORT"
+255           = "ABORT"
+anything else = "FAIL"
+
+Also note: If a test is killed by an unhandled signal, the test is reported as
+failing.
+
+Put any temporary files created during test run in $POUNDER_TMPDIR.
+
+The test_scripts/ directory provides some examples.
+
+Scheduling Tests
+================
+Everything under the tests/ directory is used for scheduling purposes. The current
+test scheduler borrows a System V rc script-like structure for specifying how and
+when tests should be run. Files under tests/ should have names that follow the this
+standard:
+
+   [type][sequence number][name]
+
+"type" is the type of test. Currently, there are two types, 'D' and 'T'.  'T'
+signifies a test, which means that the scheduler starts the test, waits for the
+test to complete, and reports on its exit status.  'D' signifies a daemon
+"test", which is to say that the scheduler will start the test, let it run in
+the background, and kill it when it's done running all the tests in that
+directory.
+
+The "sequence number" dictates the order in which the test are run. 00 goes
+first, 99 goes last.  Tests with the same number are started simultaneously,
+regardless of the type.
+
+"name" is just a convenient mnemonic to distinguish between tests. However,
+it should be the same as the corresponding name using in build_scripts and
+test_scripts. (A test with build script "build_scripts/subtest" and
+test script "test_scripts/subtest" should be defined as  something like
+"tests/T00subtest" as opposed to "tests/T00whatever_i_feel_like")
+
+Test names must be unique!
+
+File system objects under the tests/ directory can be nearly anything--
+directories, symbolic links, or files.  The test scheduler will not run
+anything that doesn't have the execute bit set.  If a FS object is a
+directory, then the contents of the directory are executed sequentially.
+
+Example:
+
+Let's examine the following test scheduler hierarchy:
+
+tests/
+    D00stats
+    T01foo
+    T01bar
+    T02dir/
+        T00gav -> ../../test_scripts/gav
+        T01hic -> ../../test_scripts/hic
+    T03lat
+
+Let's see how the tests are run.  The test scheduler will start off by scanning
+the tests/ directory.  First it spawns D00stats and lets it run in the
+background.  Next, T01foo and T01bar are launched at the same time; the
+scheduler will wait for both of them to complete before proceeding.  Since T01foo
+is a file and not just a symbolic link, there is a fair chance that T01foo runs
+some test in a loop for a certain amount of time.  In any case, the scheduler
+next sees T02dir and proceeds into it.
+
+In the T02dir/, we find two test scripts.  First T00gav runs, followed by
+T01hic.  Now there are no more tests to run in T02dir/, so the scheduler heads
+back up to the parent directory.  T03lat is forked and allowed to run to
+completion, after which D00stats is killed, and the test suite exits.
+
+Running Tests Repeatedly
+========================
+Two helper programs are provided to run tests repeatedly, timed_loop and infinite_loop.
+(This version of pounder currently also includes a fancy_timed_loop.c file, but it's only
+meant to be used for the random_syscall and will most likely be merged with timed_loop.c
+in the future, so we will ignore it here for now.)
+
+1. timed_loop
+
+    timed_loop [-m max_failures] duration_in_seconds command [arguments]
+
+This program will run "command" with the given arguments repeated
+until the number of seconds given as "duration" has passed or the
+command has failed a total of "max_failures" times, whichever comes first.
+If the $MAX_FAILURES variable is set (defined in config, see CONFIGURATION
+for details), then the program will run until command has failed a total of
+$MAX_FAILURES time (as long as it's not overridden by the -m option).
+
+2. infinite_loop
+
+    infinite_loop [-m max_failures] command [arguments]
+
+This program runs "command" repeatedly until sent SIGTERM or the
+command has failed a total of "max_failures" times. If the $MAX_FAILURES
+variable is set (defined in config, see CONFIGURATION for details), then
+the program will run until command has failed a total of $MAX_FAILURES time
+(as long as it's not overridden by the -m option).
+
+Examples:
+
+1. test_repo/T90ramp/D02build_kernel contains the following line:
+
+	"$POUNDER_HOME/infinite_loop $POUNDER_HOME/test_scripts/build_kernel"
+
+	which will run the build_kernel test script repeatedly until sent SIGTERM
+	or until it has failed a total of $MAX_FAILURES times.
+
+	"$POUNDER_HOME/infinite_loop -m 10 $POUNDER_HOME/test_scripts/build_kernel"
+
+	would run the build_kernel test script repeatedly until sent SIGTERM or
+	until it has failed 10 times, regardless of what $MAX_FAILURES is.
+
+2. test_scripts/time_drift contains the following line:
+       
+	"$POUNDER_HOME/timed_loop 900 "$POUNDER_SRCDIR/time_tests/drift-test.py" $NTP_SERVER $FREQ"
+
+	which will run the drift-test.py script ($NTP_SERVER and $FREQ are some args passed to drift-test.py)
+	for 15 minutes or until it has failed a total of $MAX_FAILURES times.
+
+	"$POUNDER_HOME/timed_loop -m 10 900 "$POUNDER_SRCDIR/time_tests/drift-test.py" $NTP_SERVER $FREQ"
+
+	would run the drift-test.py script for 15 minutes or until it has failed 10 times, regardless of
+	what $MAX_FAILURES is.
+
+The Provided Test Schedulers
+============================
+This version of pounder provides 3 test schedulers: the "default," "fast," and "test" test schedulers.
+The tarred versions can be found in the schedulers/ directory as default-tests.tar.gz, fast-tests.tar.gz,
+and test-tests.tar.gz respectively.
+
+To unpack a test scheduler, run "make install" in the pounder/ directory and enter the name of the
+scheduler you would like to unpack at the first prompt.
+
+Example of unpacking the "fast" test scheduler:
+
+	# make install
+	./Install
+	Looking for tools...make g++ lex gcc python wget sudo diff patch egrep rm echo test which cp mkdir .
+	All tools were found.
+	WHICH TEST SCHEDULER SETUP DO YOU WANT TO UNPACK?
+	[Choose from:
+	default-tests.tar.gz
+	fast-tests.tar.gz
+	test-tests.tar.gz]
+	[Or simply press ENTER for the default scheduler]
+	Scheduler selection: fast
+
+Descriptions of the provided test schedulers:
+
+1. default - provides a general purpose stress test, runs for 48 hours unless the -d option
+		is used when starting pounder.
+2. fast - basically the same as default, except it runs for 12 hours by default.
+3. test - provides a set of useless tests. Each test simply passes, fails, aborts, or sleeps for
+		some period of time. They don't do anything useful but can be used to see how
+		the test scheduling setup works.
+
+Creating Your Own Test Schedulers
+=================================
+From the pounder directory, place the desired tests in the tests/ directory according to
+the rules described in the "Scheduling Tests" section above. Then run the following command:
+
+./pounder -c name_of_scheduler
+
+to create a new test scheduler, which will be tarred as name_of_scheduler-tests.tar.gz and
+placed in the schedulers/ directory.
+
+Example Usage:
+
+	# ls ./schedulers
+	default-tests.tar.gz  fast-tests.tar.gz     test-tests.tar.gz
+
+	# ls ./tests
+	T00hwinfo
+
+	# ./pounder -c new_sched
+
+	# ls ./schedulers
+	default-tests.tar.gz  fast-tests.tar.gz     new_sched-tests.tar.gz      test-tests.tar.gz
+
+	After unpacking the "new_sched" test scheduler during install, the tests/ directory should
+	contain the T00hwinfo subtest along with a tests/excluded/ directory (see the "Including and
+	Excluding Tests" section below for details regarding the tests/excluded directory).
+
+Including and Excluding Tests
+=============================
+After unpacking the test scheduler and building each individual test, running
+"./pounder" will automatically run every test included in the tests folder. If you
+would like to run only ONE test, run "./pounder ./tests/<some subtest>". If you would
+like to run a portion of tests, you can use the "./pounder -e" option to exclude
+certain subtests from subsequent pounder runs:
+
+Example:
+
+Suppose you have already ran "make install" and unpacked the default test scheduler.
+The tests/ directory should now contain the subtests to be run
+
+1) ./pounder -l
+	- lists all of the subtests that came with the currently active test scheduler.
+	The output should look something like:
+
+	------------------	
+	#./pounder -l
+	Included subtests:
+	...
+	.../ltp-full-xxxxxxxx/tools/pounder/tests/T10single/T00xterm_stress
+	.../ltp-full-xxxxxxxx/tools/pounder/tests/T00hwinfo
+	...
+
+	Excluded subtests:
+	[NONE]
+	------------------
+
+2) ./pounder -e "tests/T10single/T00xterm_stress tests/T00hwinfo"
+	- will exclude T00xterm_stress and T00hwinfo from any subsequent pounder runs.
+	This command essentially moves the two tests from the "tests" folder to the
+	"tests/excluded" folder for temporary storage, where they will remain until
+	re-included back into the test scheduler (this is also why all test names
+	should be unique). A file "tests/excluded/testlist" keeps track of which tests
+	have been excluded from the test scheduler and what their original paths were.
+
+3) ./pounder -l
+	- should now output something like:
+
+	------------------
+	#./pounder -l
+	Included subtests:
+	...
+
+	Excluded subtests:
+	T00xterm_stress
+	T00hwinfo
+	------------------
+
+4) ./pounder -i "T00xterm_stress T00hwinfo" - will re-include these subtests back into
+	the test scheduler. They will be moved from the tests/excluded folder back into
+	the tests folder under their original paths.

[-- Attachment #3: Type: text/plain, Size: 351 bytes --]

------------------------------------------------------------------------------
uberSVN's rich system and user administration capabilities and model 
configuration take the hassle out of deploying and managing Subversion and 
the tools developers use with it. Learn more about uberSVN and get a free 
download at:  http://p.sf.net/sfu/wandisco-dev2dev

[-- Attachment #4: Type: text/plain, Size: 155 bytes --]

_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

^ permalink raw reply related	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2011-11-02 17:33 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <mailman.342816.1314268291.22889.ltp-list@lists.sourceforge.net>
2011-08-25 17:26 ` [LTP] [PATCH 4/7 ver2 w/attachment] Changes to pounder's build scripts (Cyril Hrubis) James Takahashi
2011-08-26 11:40   ` Cyril Hrubis
     [not found] ` <20110825174602.GB6398@us.ibm.com>
2011-08-26 11:38   ` [LTP] [PATCH 6/7 ver2 w/attachment] Modified pounder install and run scripts Cyril Hrubis
2011-09-08 11:01     ` Cyril Hrubis
     [not found]       ` <20110908225757.GA31906@us.ibm.com>
2011-09-09 12:31         ` Cyril Hrubis
     [not found]           ` <20110909175843.GA4276@us.ibm.com>
     [not found]             ` <20111027172945.GB11634@us.ibm.com>
2011-11-02 17:39               ` Cyril Hrubis
2011-08-15 18:56 [LTP] [PATCH 1/7 ver2 w/attachment] Updated pounder's documentation Lucy Liang
2011-08-15 18:56 ` [LTP] [PATCH 6/7 ver2 w/attachment] Modified pounder install and run scripts Lucy Liang
2011-08-25 10:17   ` Cyril Hrubis

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.