[PATCH 1/2][RFC] Add README_cyclicload

[PATCH 1/2][RFC] Add README_cyclicload

[Priyanka Jain]

Cyclicload program is designed to simulate load
at regular intervals in form of one or two threads.

Signed-off-by: Priyanka Jain <Priyanka.Jain@freescale.com>
---
 src/cyclictest/README_cyclicload |  147 ++++++++++++++++++++++++++++++++++++++
 1 files changed, 147 insertions(+), 0 deletions(-)
 create mode 100644 src/cyclictest/README_cyclicload

diff --git a/src/cyclictest/README_cyclicload b/src/cyclictest/README_cyclicload
new file mode 100644
index 0000000..f1e99bf
--- /dev/null
+++ b/src/cyclictest/README_cyclicload
@@ -0,0 +1,147 @@
+DESCRIPTION
+-------------
+
+The cyclicload program is developed above existing cyclictest application.
+It is basically designed to simulate specified load at regular intervals
+in addition to cyclictest functionality.
+It can simulate one or two load threads.
+
+
+Why it is required?
+---------------------
+It is required to test system performance under a specified load
+along with tracking latency of simulated load thread.
+
+
+Example use case
+--------------------
+For products like LTE, L2 layer runs in form of two threads.
+-MAC layer thread runs at highest RT priority producing a
+fixed load at regular intervals and
+-second thread run at lower RT priority or in non-rt priority
+ producing some load in each interval depending upon availability of CPU.
+Requirement is to test system under this load as well as to
+track latency of highest priority RT thread which is MAC thread
+in example usecase.
+
+
+What does it do?
+------------------
+It creates one or two load generating thread/s.
+1)load1 thread (timer thread)
+2)load2 thread
+priority, nice value, load % as input via command line
+
+-cyclictest funcationality like latency measurement
+
+
+How does it work
+-----------------
+It uses generate_load loop function to simualte load.
+First it caliberate required loop_count per unit time per CPU.
+It stores this data in some file.
+For subsequent runs, it uses this caliberated data to generate load
+if form of one or two threads .
+It keeps on tracking the latency of RT thread.
+More in Design Overview section.
+
+
+Recommended Settings
+----------------------
+-First run is recommended to be run with no or least load for accuracy.
+-should be run with sudo or root permission.
+-caliberation routine produces calinerate_ount file in runnign directory.
+	If one don't have permission in that dir, path should be changed in
+	FILENAME or one can exploit shared memory method.
+-Atleast one thread should be of RT priority. This thread takes priority of
+	cyclictest as its priority.
+-cyclictest applcication should be run with SCHED_OTHER policy.
+-recommended to run in quiet mode (-q) in background.
+TODO: add option to take filepath from cmd line
+
+
+Cmd line usage/examples
+------------------------
+New command line arguments:
+ "-x       --load_t1         load in percentage for t1 thread\n"
+ "-X       --load_t2         load in percentage for t2 thread\n"
+ "-z       --priority_t2     priority of t2 thread\n"
+ "-Z       --nice_t2         nice value of t2 thread\n"
+
+If both load_t1 and load_t2 are zero, it behaves as default cyclictest application
+
+For uniprocessor:
+	#sudo ./cyclictest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600&
+
+For multiprocessor:
+	#sudo ./cyclcitest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600 -S&
+
+
+
+Future Enhancements
+-------------------
+Maintain statistics of average load produces.
+invalidate cache in each interval to make it close to actual scenario.
+can be scalable to produce n number of load threads.
+
+
+Design Overview
+--------------
+---------------
+
+The logic to simulate load has been added above existing cyclictest application.
+
+Threads
+--------
+cyclicload : main process
+--------------------------
+-parse input arguments.
+
+-for first run : create caliberate thread.
+	store caliberated count in caliberate_count file
+-for subsequent runs:
+	read caliberated count from file and use that count to simulate defined load.
+
+-create t load1threads (t depending on cmdlime args, for smp system = num of cores, one thread per core)
+
+-update stats periodically while !shutdown
+-print stats periodically depending on cmdline args while !shutdown
+
+
+caliberate thread
+------------------
+-is created only once for the first run of cyclicload
+-run at highest RT priority.
+-affine itself turn by turn to each cpu. (for all cpus for multicore system)
+-caliberate count per unit (ms by default) per cpu
+-store per cpu data in caliberate_count_array
+-recommended to be run with no or least load.
+
+
+load1 thread(timer thread)
+---------------------------
+-run at priority parsed in main routine
+-recommended to run at highest RT priority
+-creates load2_thread (optional only if on load2 is nonzero)
+-calculate number of loops to execute to generate defined load by
+	using caliberate_count_array and load percentage parsed in main routine
+-reduced interval = interval(window) - duration for load_t1
+-while loop until !shutdown
+	-generate load_t1 load
+	-sleep for reduced interval
+	-calculate latency (cosumes around 1%-2% cpu)
+	-discard off remaining load if window expires
+	-set next_window_started flag to 1
+	-signal load2_thread about next window
+-overhead: consumes 1-2% cpu for latency maintainenance, etc
+-variation in produced load : 0%-2% of full cpu utilization
+
+
+load2_thread
+------------
+generate load_t2 load
+wait for next window to start
+if window expires before generate load_t2 finishes,
+	discard off remaining load
+	restart generate load_t2 load
+variation in produced load : 0%-1% of full cpu utilization
-- 
1.7.4.1

RE: [PATCH 1/2][RFC] Add README_cyclicload

[Jain Priyanka-B32167]

Hi,

Waiting for the comments on this tool.

Thanks
Priyanka

-----Original Message-----
From: Jain Priyanka-B32167 
Sent: Wednesday, August 01, 2012 10:37 AM
To: linux-rt-users@vger.kernel.org; williams@redhat.com; dvhart@linux.intel.com
Cc: rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; Aggrwal Poonam-B10812; Jain Priyanka-B32167
Subject: [PATCH 1/2][RFC] Add README_cyclicload

Cyclicload program is designed to simulate load at regular intervals in form of one or two threads.

Signed-off-by: Priyanka Jain <Priyanka.Jain@freescale.com>
---
 src/cyclictest/README_cyclicload |  147 ++++++++++++++++++++++++++++++++++++++
 1 files changed, 147 insertions(+), 0 deletions(-)  create mode 100644 src/cyclictest/README_cyclicload

diff --git a/src/cyclictest/README_cyclicload b/src/cyclictest/README_cyclicload
new file mode 100644
index 0000000..f1e99bf
--- /dev/null
+++ b/src/cyclictest/README_cyclicload
@@ -0,0 +1,147 @@
+DESCRIPTION
+-------------
+
+The cyclicload program is developed above existing cyclictest application.
+It is basically designed to simulate specified load at regular 
+intervals in addition to cyclictest functionality.
+It can simulate one or two load threads.
+
+
+Why it is required?
+---------------------
+It is required to test system performance under a specified load along 
+with tracking latency of simulated load thread.
+
+
+Example use case
+--------------------
+For products like LTE, L2 layer runs in form of two threads.
+-MAC layer thread runs at highest RT priority producing a fixed load at 
+regular intervals and -second thread run at lower RT priority or in 
+non-rt priority  producing some load in each interval depending upon 
+availability of CPU.
+Requirement is to test system under this load as well as to track 
+latency of highest priority RT thread which is MAC thread in example 
+usecase.
+
+
+What does it do?
+------------------
+It creates one or two load generating thread/s.
+1)load1 thread (timer thread)
+2)load2 thread
+priority, nice value, load % as input via command line
+
+-cyclictest funcationality like latency measurement
+
+
+How does it work
+-----------------
+It uses generate_load loop function to simualte load.
+First it caliberate required loop_count per unit time per CPU.
+It stores this data in some file.
+For subsequent runs, it uses this caliberated data to generate load if 
+form of one or two threads .
+It keeps on tracking the latency of RT thread.
+More in Design Overview section.
+
+
+Recommended Settings
+----------------------
+-First run is recommended to be run with no or least load for accuracy.
+-should be run with sudo or root permission.
+-caliberation routine produces calinerate_ount file in runnign directory.
+	If one don't have permission in that dir, path should be changed in
+	FILENAME or one can exploit shared memory method.
+-Atleast one thread should be of RT priority. This thread takes priority of
+	cyclictest as its priority.
+-cyclictest applcication should be run with SCHED_OTHER policy.
+-recommended to run in quiet mode (-q) in background.
+TODO: add option to take filepath from cmd line
+
+
+Cmd line usage/examples
+------------------------
+New command line arguments:
+ "-x       --load_t1         load in percentage for t1 thread\n"
+ "-X       --load_t2         load in percentage for t2 thread\n"
+ "-z       --priority_t2     priority of t2 thread\n"
+ "-Z       --nice_t2         nice value of t2 thread\n"
+
+If both load_t1 and load_t2 are zero, it behaves as default cyclictest 
+application
+
+For uniprocessor:
+	#sudo ./cyclictest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600&
+
+For multiprocessor:
+	#sudo ./cyclcitest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600 -S&
+
+
+
+Future Enhancements
+-------------------
+Maintain statistics of average load produces.
+invalidate cache in each interval to make it close to actual scenario.
+can be scalable to produce n number of load threads.
+
+
+Design Overview
+--------------
+---------------
+
+The logic to simulate load has been added above existing cyclictest application.
+
+Threads
+--------
+cyclicload : main process
+--------------------------
+-parse input arguments.
+
+-for first run : create caliberate thread.
+	store caliberated count in caliberate_count file -for subsequent runs:
+	read caliberated count from file and use that count to simulate defined load.
+
+-create t load1threads (t depending on cmdlime args, for smp system = 
+num of cores, one thread per core)
+
+-update stats periodically while !shutdown -print stats periodically 
+depending on cmdline args while !shutdown
+
+
+caliberate thread
+------------------
+-is created only once for the first run of cyclicload -run at highest 
+RT priority.
+-affine itself turn by turn to each cpu. (for all cpus for multicore 
+system) -caliberate count per unit (ms by default) per cpu -store per 
+cpu data in caliberate_count_array -recommended to be run with no or 
+least load.
+
+
+load1 thread(timer thread)
+---------------------------
+-run at priority parsed in main routine -recommended to run at highest 
+RT priority -creates load2_thread (optional only if on load2 is 
+nonzero) -calculate number of loops to execute to generate defined load 
+by
+	using caliberate_count_array and load percentage parsed in main 
+routine -reduced interval = interval(window) - duration for load_t1 
+-while loop until !shutdown
+	-generate load_t1 load
+	-sleep for reduced interval
+	-calculate latency (cosumes around 1%-2% cpu)
+	-discard off remaining load if window expires
+	-set next_window_started flag to 1
+	-signal load2_thread about next window
+-overhead: consumes 1-2% cpu for latency maintainenance, etc -variation 
+in produced load : 0%-2% of full cpu utilization
+
+
+load2_thread
+------------
+generate load_t2 load
+wait for next window to start
+if window expires before generate load_t2 finishes,
+	discard off remaining load
+	restart generate load_t2 load
+variation in produced load : 0%-1% of full cpu utilization
--
1.7.4.1

Re: [PATCH 1/2][RFC] Add README_cyclicload

[Clark Williams]

[-- Attachment #1: Type: text/plain, Size: 6824 bytes --]

Priyanka,

For some reason all of us that work on rt-tests were busy last week.
I'll apply these patches and try out cyclicload this week.

Clark

On Mon, 6 Aug 2012 04:00:15 +0000
Jain Priyanka-B32167 <B32167@freescale.com> wrote:

> Hi,
> 
> Waiting for the comments on this tool.
> 
> Thanks
> Priyanka
> 
> -----Original Message-----
> From: Jain Priyanka-B32167 
> Sent: Wednesday, August 01, 2012 10:37 AM
> To: linux-rt-users@vger.kernel.org; williams@redhat.com; dvhart@linux.intel.com
> Cc: rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; Aggrwal Poonam-B10812; Jain Priyanka-B32167
> Subject: [PATCH 1/2][RFC] Add README_cyclicload
> 
> Cyclicload program is designed to simulate load at regular intervals in form of one or two threads.
> 
> Signed-off-by: Priyanka Jain <Priyanka.Jain@freescale.com>
> ---
>  src/cyclictest/README_cyclicload |  147 ++++++++++++++++++++++++++++++++++++++
>  1 files changed, 147 insertions(+), 0 deletions(-)  create mode 100644 src/cyclictest/README_cyclicload
> 
> diff --git a/src/cyclictest/README_cyclicload b/src/cyclictest/README_cyclicload
> new file mode 100644
> index 0000000..f1e99bf
> --- /dev/null
> +++ b/src/cyclictest/README_cyclicload
> @@ -0,0 +1,147 @@
> +DESCRIPTION
> +-------------
> +
> +The cyclicload program is developed above existing cyclictest application.
> +It is basically designed to simulate specified load at regular 
> +intervals in addition to cyclictest functionality.
> +It can simulate one or two load threads.
> +
> +
> +Why it is required?
> +---------------------
> +It is required to test system performance under a specified load along 
> +with tracking latency of simulated load thread.
> +
> +
> +Example use case
> +--------------------
> +For products like LTE, L2 layer runs in form of two threads.
> +-MAC layer thread runs at highest RT priority producing a fixed load at 
> +regular intervals and -second thread run at lower RT priority or in 
> +non-rt priority  producing some load in each interval depending upon 
> +availability of CPU.
> +Requirement is to test system under this load as well as to track 
> +latency of highest priority RT thread which is MAC thread in example 
> +usecase.
> +
> +
> +What does it do?
> +------------------
> +It creates one or two load generating thread/s.
> +1)load1 thread (timer thread)
> +2)load2 thread
> +priority, nice value, load % as input via command line
> +
> +-cyclictest funcationality like latency measurement
> +
> +
> +How does it work
> +-----------------
> +It uses generate_load loop function to simualte load.
> +First it caliberate required loop_count per unit time per CPU.
> +It stores this data in some file.
> +For subsequent runs, it uses this caliberated data to generate load if 
> +form of one or two threads .
> +It keeps on tracking the latency of RT thread.
> +More in Design Overview section.
> +
> +
> +Recommended Settings
> +----------------------
> +-First run is recommended to be run with no or least load for accuracy.
> +-should be run with sudo or root permission.
> +-caliberation routine produces calinerate_ount file in runnign directory.
> +	If one don't have permission in that dir, path should be changed in
> +	FILENAME or one can exploit shared memory method.
> +-Atleast one thread should be of RT priority. This thread takes priority of
> +	cyclictest as its priority.
> +-cyclictest applcication should be run with SCHED_OTHER policy.
> +-recommended to run in quiet mode (-q) in background.
> +TODO: add option to take filepath from cmd line
> +
> +
> +Cmd line usage/examples
> +------------------------
> +New command line arguments:
> + "-x       --load_t1         load in percentage for t1 thread\n"
> + "-X       --load_t2         load in percentage for t2 thread\n"
> + "-z       --priority_t2     priority of t2 thread\n"
> + "-Z       --nice_t2         nice value of t2 thread\n"
> +
> +If both load_t1 and load_t2 are zero, it behaves as default cyclictest 
> +application
> +
> +For uniprocessor:
> +	#sudo ./cyclictest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600&
> +
> +For multiprocessor:
> +	#sudo ./cyclcitest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600 -S&
> +
> +
> +
> +Future Enhancements
> +-------------------
> +Maintain statistics of average load produces.
> +invalidate cache in each interval to make it close to actual scenario.
> +can be scalable to produce n number of load threads.
> +
> +
> +Design Overview
> +--------------
> +---------------
> +
> +The logic to simulate load has been added above existing cyclictest application.
> +
> +Threads
> +--------
> +cyclicload : main process
> +--------------------------
> +-parse input arguments.
> +
> +-for first run : create caliberate thread.
> +	store caliberated count in caliberate_count file -for subsequent runs:
> +	read caliberated count from file and use that count to simulate defined load.
> +
> +-create t load1threads (t depending on cmdlime args, for smp system = 
> +num of cores, one thread per core)
> +
> +-update stats periodically while !shutdown -print stats periodically 
> +depending on cmdline args while !shutdown
> +
> +
> +caliberate thread
> +------------------
> +-is created only once for the first run of cyclicload -run at highest 
> +RT priority.
> +-affine itself turn by turn to each cpu. (for all cpus for multicore 
> +system) -caliberate count per unit (ms by default) per cpu -store per 
> +cpu data in caliberate_count_array -recommended to be run with no or 
> +least load.
> +
> +
> +load1 thread(timer thread)
> +---------------------------
> +-run at priority parsed in main routine -recommended to run at highest 
> +RT priority -creates load2_thread (optional only if on load2 is 
> +nonzero) -calculate number of loops to execute to generate defined load 
> +by
> +	using caliberate_count_array and load percentage parsed in main 
> +routine -reduced interval = interval(window) - duration for load_t1 
> +-while loop until !shutdown
> +	-generate load_t1 load
> +	-sleep for reduced interval
> +	-calculate latency (cosumes around 1%-2% cpu)
> +	-discard off remaining load if window expires
> +	-set next_window_started flag to 1
> +	-signal load2_thread about next window
> +-overhead: consumes 1-2% cpu for latency maintainenance, etc -variation 
> +in produced load : 0%-2% of full cpu utilization
> +
> +
> +load2_thread
> +------------
> +generate load_t2 load
> +wait for next window to start
> +if window expires before generate load_t2 finishes,
> +	discard off remaining load
> +	restart generate load_t2 load
> +variation in produced load : 0%-1% of full cpu utilization
> --
> 1.7.4.1
> 
> 
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

RE: [PATCH 1/2][RFC] Add README_cyclicload

[Jain Priyanka-B32167]

Thanks Clark

Waiting for the results at your end.


Regards
Priyanka

-----Original Message-----
From: Clark Williams [mailto:williams@redhat.com] 
Sent: Monday, August 06, 2012 7:29 PM
To: Jain Priyanka-B32167
Cc: linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; Aggrwal Poonam-B10812
Subject: Re: [PATCH 1/2][RFC] Add README_cyclicload

Priyanka,

For some reason all of us that work on rt-tests were busy last week.
I'll apply these patches and try out cyclicload this week.

Clark

On Mon, 6 Aug 2012 04:00:15 +0000
Jain Priyanka-B32167 <B32167@freescale.com> wrote:

> Hi,
> 
> Waiting for the comments on this tool.
> 
> Thanks
> Priyanka
> 
> -----Original Message-----
> From: Jain Priyanka-B32167
> Sent: Wednesday, August 01, 2012 10:37 AM
> To: linux-rt-users@vger.kernel.org; williams@redhat.com; 
> dvhart@linux.intel.com
> Cc: rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; 
> Aggrwal Poonam-B10812; Jain Priyanka-B32167
> Subject: [PATCH 1/2][RFC] Add README_cyclicload
> 
> Cyclicload program is designed to simulate load at regular intervals in form of one or two threads.
> 
> Signed-off-by: Priyanka Jain <Priyanka.Jain@freescale.com>
> ---
>  src/cyclictest/README_cyclicload |  147 
> ++++++++++++++++++++++++++++++++++++++
>  1 files changed, 147 insertions(+), 0 deletions(-)  create mode 
> 100644 src/cyclictest/README_cyclicload
> 
> diff --git a/src/cyclictest/README_cyclicload 
> b/src/cyclictest/README_cyclicload
> new file mode 100644
> index 0000000..f1e99bf
> --- /dev/null
> +++ b/src/cyclictest/README_cyclicload
> @@ -0,0 +1,147 @@
> +DESCRIPTION
> +-------------
> +
> +The cyclicload program is developed above existing cyclictest application.
> +It is basically designed to simulate specified load at regular 
> +intervals in addition to cyclictest functionality.
> +It can simulate one or two load threads.
> +
> +
> +Why it is required?
> +---------------------
> +It is required to test system performance under a specified load 
> +along with tracking latency of simulated load thread.
> +
> +
> +Example use case
> +--------------------
> +For products like LTE, L2 layer runs in form of two threads.
> +-MAC layer thread runs at highest RT priority producing a fixed load 
> +at regular intervals and -second thread run at lower RT priority or 
> +in non-rt priority  producing some load in each interval depending 
> +upon availability of CPU.
> +Requirement is to test system under this load as well as to track 
> +latency of highest priority RT thread which is MAC thread in example 
> +usecase.
> +
> +
> +What does it do?
> +------------------
> +It creates one or two load generating thread/s.
> +1)load1 thread (timer thread)
> +2)load2 thread
> +priority, nice value, load % as input via command line
> +
> +-cyclictest funcationality like latency measurement
> +
> +
> +How does it work
> +-----------------
> +It uses generate_load loop function to simualte load.
> +First it caliberate required loop_count per unit time per CPU.
> +It stores this data in some file.
> +For subsequent runs, it uses this caliberated data to generate load 
> +if form of one or two threads .
> +It keeps on tracking the latency of RT thread.
> +More in Design Overview section.
> +
> +
> +Recommended Settings
> +----------------------
> +-First run is recommended to be run with no or least load for accuracy.
> +-should be run with sudo or root permission.
> +-caliberation routine produces calinerate_ount file in runnign directory.
> +	If one don't have permission in that dir, path should be changed in
> +	FILENAME or one can exploit shared memory method.
> +-Atleast one thread should be of RT priority. This thread takes priority of
> +	cyclictest as its priority.
> +-cyclictest applcication should be run with SCHED_OTHER policy.
> +-recommended to run in quiet mode (-q) in background.
> +TODO: add option to take filepath from cmd line
> +
> +
> +Cmd line usage/examples
> +------------------------
> +New command line arguments:
> + "-x       --load_t1         load in percentage for t1 thread\n"
> + "-X       --load_t2         load in percentage for t2 thread\n"
> + "-z       --priority_t2     priority of t2 thread\n"
> + "-Z       --nice_t2         nice value of t2 thread\n"
> +
> +If both load_t1 and load_t2 are zero, it behaves as default 
> +cyclictest application
> +
> +For uniprocessor:
> +	#sudo ./cyclictest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600&
> +
> +For multiprocessor:
> +	#sudo ./cyclcitest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600 -S&
> +
> +
> +
> +Future Enhancements
> +-------------------
> +Maintain statistics of average load produces.
> +invalidate cache in each interval to make it close to actual scenario.
> +can be scalable to produce n number of load threads.
> +
> +
> +Design Overview
> +--------------
> +---------------
> +
> +The logic to simulate load has been added above existing cyclictest application.
> +
> +Threads
> +--------
> +cyclicload : main process
> +--------------------------
> +-parse input arguments.
> +
> +-for first run : create caliberate thread.
> +	store caliberated count in caliberate_count file -for subsequent runs:
> +	read caliberated count from file and use that count to simulate defined load.
> +
> +-create t load1threads (t depending on cmdlime args, for smp system = 
> +num of cores, one thread per core)
> +
> +-update stats periodically while !shutdown -print stats periodically 
> +depending on cmdline args while !shutdown
> +
> +
> +caliberate thread
> +------------------
> +-is created only once for the first run of cyclicload -run at highest 
> +RT priority.
> +-affine itself turn by turn to each cpu. (for all cpus for multicore
> +system) -caliberate count per unit (ms by default) per cpu -store per 
> +cpu data in caliberate_count_array -recommended to be run with no or 
> +least load.
> +
> +
> +load1 thread(timer thread)
> +---------------------------
> +-run at priority parsed in main routine -recommended to run at 
> +highest RT priority -creates load2_thread (optional only if on load2 
> +is
> +nonzero) -calculate number of loops to execute to generate defined 
> +load by
> +	using caliberate_count_array and load percentage parsed in main 
> +routine -reduced interval = interval(window) - duration for load_t1 
> +-while loop until !shutdown
> +	-generate load_t1 load
> +	-sleep for reduced interval
> +	-calculate latency (cosumes around 1%-2% cpu)
> +	-discard off remaining load if window expires
> +	-set next_window_started flag to 1
> +	-signal load2_thread about next window
> +-overhead: consumes 1-2% cpu for latency maintainenance, etc 
> +-variation in produced load : 0%-2% of full cpu utilization
> +
> +
> +load2_thread
> +------------
> +generate load_t2 load
> +wait for next window to start
> +if window expires before generate load_t2 finishes,
> +	discard off remaining load
> +	restart generate load_t2 load
> +variation in produced load : 0%-1% of full cpu utilization
> --
> 1.7.4.1
> 
> 
> 

RE: [PATCH 1/2][RFC] Add README_cyclicload

[Jain Priyanka-B32167]

Hello Clark,

Did you have the chance to test the patch.

Thanks
Priyanka

-----Original Message-----
From: linux-rt-users-owner@vger.kernel.org [mailto:linux-rt-users-owner@vger.kernel.org] On Behalf Of Jain Priyanka-B32167
Sent: Tuesday, August 07, 2012 9:32 AM
To: Clark Williams
Cc: linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; Aggrwal Poonam-B10812
Subject: RE: [PATCH 1/2][RFC] Add README_cyclicload

Thanks Clark

Waiting for the results at your end.


Regards
Priyanka

-----Original Message-----
From: Clark Williams [mailto:williams@redhat.com] 
Sent: Monday, August 06, 2012 7:29 PM
To: Jain Priyanka-B32167
Cc: linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; Aggrwal Poonam-B10812
Subject: Re: [PATCH 1/2][RFC] Add README_cyclicload

Priyanka,

For some reason all of us that work on rt-tests were busy last week.
I'll apply these patches and try out cyclicload this week.

Clark

On Mon, 6 Aug 2012 04:00:15 +0000
Jain Priyanka-B32167 <B32167@freescale.com> wrote:

> Hi,
> 
> Waiting for the comments on this tool.
> 
> Thanks
> Priyanka
> 
> -----Original Message-----
> From: Jain Priyanka-B32167
> Sent: Wednesday, August 01, 2012 10:37 AM
> To: linux-rt-users@vger.kernel.org; williams@redhat.com; 
> dvhart@linux.intel.com
> Cc: rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; 
> Aggrwal Poonam-B10812; Jain Priyanka-B32167
> Subject: [PATCH 1/2][RFC] Add README_cyclicload
> 
> Cyclicload program is designed to simulate load at regular intervals in form of one or two threads.
> 
> Signed-off-by: Priyanka Jain <Priyanka.Jain@freescale.com>
> ---
>  src/cyclictest/README_cyclicload |  147 
> ++++++++++++++++++++++++++++++++++++++
>  1 files changed, 147 insertions(+), 0 deletions(-)  create mode 
> 100644 src/cyclictest/README_cyclicload
> 
> diff --git a/src/cyclictest/README_cyclicload 
> b/src/cyclictest/README_cyclicload
> new file mode 100644
> index 0000000..f1e99bf
> --- /dev/null
> +++ b/src/cyclictest/README_cyclicload
> @@ -0,0 +1,147 @@
> +DESCRIPTION
> +-------------
> +
> +The cyclicload program is developed above existing cyclictest application.
> +It is basically designed to simulate specified load at regular 
> +intervals in addition to cyclictest functionality.
> +It can simulate one or two load threads.
> +
> +
> +Why it is required?
> +---------------------
> +It is required to test system performance under a specified load 
> +along with tracking latency of simulated load thread.
> +
> +
> +Example use case
> +--------------------
> +For products like LTE, L2 layer runs in form of two threads.
> +-MAC layer thread runs at highest RT priority producing a fixed load 
> +at regular intervals and -second thread run at lower RT priority or 
> +in non-rt priority  producing some load in each interval depending 
> +upon availability of CPU.
> +Requirement is to test system under this load as well as to track 
> +latency of highest priority RT thread which is MAC thread in example 
> +usecase.
> +
> +
> +What does it do?
> +------------------
> +It creates one or two load generating thread/s.
> +1)load1 thread (timer thread)
> +2)load2 thread
> +priority, nice value, load % as input via command line
> +
> +-cyclictest funcationality like latency measurement
> +
> +
> +How does it work
> +-----------------
> +It uses generate_load loop function to simualte load.
> +First it caliberate required loop_count per unit time per CPU.
> +It stores this data in some file.
> +For subsequent runs, it uses this caliberated data to generate load 
> +if form of one or two threads .
> +It keeps on tracking the latency of RT thread.
> +More in Design Overview section.
> +
> +
> +Recommended Settings
> +----------------------
> +-First run is recommended to be run with no or least load for accuracy.
> +-should be run with sudo or root permission.
> +-caliberation routine produces calinerate_ount file in runnign directory.
> +	If one don't have permission in that dir, path should be changed in
> +	FILENAME or one can exploit shared memory method.
> +-Atleast one thread should be of RT priority. This thread takes priority of
> +	cyclictest as its priority.
> +-cyclictest applcication should be run with SCHED_OTHER policy.
> +-recommended to run in quiet mode (-q) in background.
> +TODO: add option to take filepath from cmd line
> +
> +
> +Cmd line usage/examples
> +------------------------
> +New command line arguments:
> + "-x       --load_t1         load in percentage for t1 thread\n"
> + "-X       --load_t2         load in percentage for t2 thread\n"
> + "-z       --priority_t2     priority of t2 thread\n"
> + "-Z       --nice_t2         nice value of t2 thread\n"
> +
> +If both load_t1 and load_t2 are zero, it behaves as default 
> +cyclictest application
> +
> +For uniprocessor:
> +	#sudo ./cyclictest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600&
> +
> +For multiprocessor:
> +	#sudo ./cyclcitest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600 -S&
> +
> +
> +
> +Future Enhancements
> +-------------------
> +Maintain statistics of average load produces.
> +invalidate cache in each interval to make it close to actual scenario.
> +can be scalable to produce n number of load threads.
> +
> +
> +Design Overview
> +--------------
> +---------------
> +
> +The logic to simulate load has been added above existing cyclictest application.
> +
> +Threads
> +--------
> +cyclicload : main process
> +--------------------------
> +-parse input arguments.
> +
> +-for first run : create caliberate thread.
> +	store caliberated count in caliberate_count file -for subsequent runs:
> +	read caliberated count from file and use that count to simulate defined load.
> +
> +-create t load1threads (t depending on cmdlime args, for smp system = 
> +num of cores, one thread per core)
> +
> +-update stats periodically while !shutdown -print stats periodically 
> +depending on cmdline args while !shutdown
> +
> +
> +caliberate thread
> +------------------
> +-is created only once for the first run of cyclicload -run at highest 
> +RT priority.
> +-affine itself turn by turn to each cpu. (for all cpus for multicore
> +system) -caliberate count per unit (ms by default) per cpu -store per 
> +cpu data in caliberate_count_array -recommended to be run with no or 
> +least load.
> +
> +
> +load1 thread(timer thread)
> +---------------------------
> +-run at priority parsed in main routine -recommended to run at 
> +highest RT priority -creates load2_thread (optional only if on load2 
> +is
> +nonzero) -calculate number of loops to execute to generate defined 
> +load by
> +	using caliberate_count_array and load percentage parsed in main 
> +routine -reduced interval = interval(window) - duration for load_t1 
> +-while loop until !shutdown
> +	-generate load_t1 load
> +	-sleep for reduced interval
> +	-calculate latency (cosumes around 1%-2% cpu)
> +	-discard off remaining load if window expires
> +	-set next_window_started flag to 1
> +	-signal load2_thread about next window
> +-overhead: consumes 1-2% cpu for latency maintainenance, etc 
> +-variation in produced load : 0%-2% of full cpu utilization
> +
> +
> +load2_thread
> +------------
> +generate load_t2 load
> +wait for next window to start
> +if window expires before generate load_t2 finishes,
> +	discard off remaining load
> +	restart generate load_t2 load
> +variation in produced load : 0%-1% of full cpu utilization
> --
> 1.7.4.1
> 
> 
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH 1/2][RFC] Add README_cyclicload

[John Kacur]

On Mon, 13 Aug 2012, Jain Priyanka-B32167 wrote:

> Hello Clark,
> 
> Did you have the chance to test the patch.
> 
> Thanks
> Priyanka
> 
> -----Original Message-----
> From: linux-rt-users-owner@vger.kernel.org [mailto:linux-rt-users-owner@vger.kernel.org] On Behalf Of Jain Priyanka-B32167
> Sent: Tuesday, August 07, 2012 9:32 AM
> To: Clark Williams
> Cc: linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; Aggrwal Poonam-B10812
> Subject: RE: [PATCH 1/2][RFC] Add README_cyclicload
> 
> Thanks Clark
> 
> Waiting for the results at your end.
> 
> 
> Regards
> Priyanka
> 
> -----Original Message-----
> From: Clark Williams [mailto:williams@redhat.com] 
> Sent: Monday, August 06, 2012 7:29 PM
> To: Jain Priyanka-B32167
> Cc: linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; Aggrwal Poonam-B10812
> Subject: Re: [PATCH 1/2][RFC] Add README_cyclicload
> 
> Priyanka,
> 
> For some reason all of us that work on rt-tests were busy last week.
> I'll apply these patches and try out cyclicload this week.
> 
> Clark
> 
> On Mon, 6 Aug 2012 04:00:15 +0000
> Jain Priyanka-B32167 <B32167@freescale.com> wrote:
> 
> > Hi,
> > 
> > Waiting for the comments on this tool.
> > 
> > Thanks
> > Priyanka
> > 
> > -----Original Message-----
> > From: Jain Priyanka-B32167
> > Sent: Wednesday, August 01, 2012 10:37 AM
> > To: linux-rt-users@vger.kernel.org; williams@redhat.com; 
> > dvhart@linux.intel.com
> > Cc: rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; 
> > Aggrwal Poonam-B10812; Jain Priyanka-B32167
> > Subject: [PATCH 1/2][RFC] Add README_cyclicload
> > 
> > Cyclicload program is designed to simulate load at regular intervals in form of one or two threads.
> > 
> > Signed-off-by: Priyanka Jain <Priyanka.Jain@freescale.com>
> > ---
> >  src/cyclictest/README_cyclicload |  147 
> > ++++++++++++++++++++++++++++++++++++++
> >  1 files changed, 147 insertions(+), 0 deletions(-)  create mode 
> > 100644 src/cyclictest/README_cyclicload
> > 
> > diff --git a/src/cyclictest/README_cyclicload 
> > b/src/cyclictest/README_cyclicload
> > new file mode 100644
> > index 0000000..f1e99bf
> > --- /dev/null
> > +++ b/src/cyclictest/README_cyclicload
> > @@ -0,0 +1,147 @@
> > +DESCRIPTION
> > +-------------
> > +
> > +The cyclicload program is developed above existing cyclictest application.
> > +It is basically designed to simulate specified load at regular 
> > +intervals in addition to cyclictest functionality.
> > +It can simulate one or two load threads.
> > +
> > +
> > +Why it is required?
> > +---------------------
> > +It is required to test system performance under a specified load 
> > +along with tracking latency of simulated load thread.
> > +
> > +
> > +Example use case
> > +--------------------
> > +For products like LTE, L2 layer runs in form of two threads.
> > +-MAC layer thread runs at highest RT priority producing a fixed load 
> > +at regular intervals and -second thread run at lower RT priority or 
> > +in non-rt priority  producing some load in each interval depending 
> > +upon availability of CPU.
> > +Requirement is to test system under this load as well as to track 
> > +latency of highest priority RT thread which is MAC thread in example 
> > +usecase.
> > +
> > +
> > +What does it do?
> > +------------------
> > +It creates one or two load generating thread/s.
> > +1)load1 thread (timer thread)
> > +2)load2 thread
> > +priority, nice value, load % as input via command line
> > +
> > +-cyclictest funcationality like latency measurement
> > +
> > +
> > +How does it work
> > +-----------------
> > +It uses generate_load loop function to simualte load.
> > +First it caliberate required loop_count per unit time per CPU.
> > +It stores this data in some file.
> > +For subsequent runs, it uses this caliberated data to generate load 
> > +if form of one or two threads .
> > +It keeps on tracking the latency of RT thread.
> > +More in Design Overview section.
> > +
> > +
> > +Recommended Settings
> > +----------------------
> > +-First run is recommended to be run with no or least load for accuracy.
> > +-should be run with sudo or root permission.
> > +-caliberation routine produces calinerate_ount file in runnign directory.
> > +	If one don't have permission in that dir, path should be changed in
> > +	FILENAME or one can exploit shared memory method.
> > +-Atleast one thread should be of RT priority. This thread takes priority of
> > +	cyclictest as its priority.
> > +-cyclictest applcication should be run with SCHED_OTHER policy.
> > +-recommended to run in quiet mode (-q) in background.
> > +TODO: add option to take filepath from cmd line
> > +
> > +
> > +Cmd line usage/examples
> > +------------------------
> > +New command line arguments:
> > + "-x       --load_t1         load in percentage for t1 thread\n"
> > + "-X       --load_t2         load in percentage for t2 thread\n"
> > + "-z       --priority_t2     priority of t2 thread\n"
> > + "-Z       --nice_t2         nice value of t2 thread\n"
> > +
> > +If both load_t1 and load_t2 are zero, it behaves as default 
> > +cyclictest application
> > +
> > +For uniprocessor:
> > +	#sudo ./cyclictest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600&
> > +
> > +For multiprocessor:
> > +	#sudo ./cyclcitest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600 -S&
> > +
> > +
> > +
> > +Future Enhancements
> > +-------------------
> > +Maintain statistics of average load produces.
> > +invalidate cache in each interval to make it close to actual scenario.
> > +can be scalable to produce n number of load threads.
> > +
> > +
> > +Design Overview
> > +--------------
> > +---------------
> > +
> > +The logic to simulate load has been added above existing cyclictest application.
> > +
> > +Threads
> > +--------
> > +cyclicload : main process
> > +--------------------------
> > +-parse input arguments.
> > +
> > +-for first run : create caliberate thread.
> > +	store caliberated count in caliberate_count file -for subsequent runs:
> > +	read caliberated count from file and use that count to simulate defined load.
> > +
> > +-create t load1threads (t depending on cmdlime args, for smp system = 
> > +num of cores, one thread per core)
> > +
> > +-update stats periodically while !shutdown -print stats periodically 
> > +depending on cmdline args while !shutdown
> > +
> > +
> > +caliberate thread
> > +------------------
> > +-is created only once for the first run of cyclicload -run at highest 
> > +RT priority.
> > +-affine itself turn by turn to each cpu. (for all cpus for multicore
> > +system) -caliberate count per unit (ms by default) per cpu -store per 
> > +cpu data in caliberate_count_array -recommended to be run with no or 
> > +least load.
> > +
> > +
> > +load1 thread(timer thread)
> > +---------------------------
> > +-run at priority parsed in main routine -recommended to run at 
> > +highest RT priority -creates load2_thread (optional only if on load2 
> > +is
> > +nonzero) -calculate number of loops to execute to generate defined 
> > +load by
> > +	using caliberate_count_array and load percentage parsed in main 
> > +routine -reduced interval = interval(window) - duration for load_t1 
> > +-while loop until !shutdown
> > +	-generate load_t1 load
> > +	-sleep for reduced interval
> > +	-calculate latency (cosumes around 1%-2% cpu)
> > +	-discard off remaining load if window expires
> > +	-set next_window_started flag to 1
> > +	-signal load2_thread about next window
> > +-overhead: consumes 1-2% cpu for latency maintainenance, etc 
> > +-variation in produced load : 0%-2% of full cpu utilization
> > +
> > +
> > +load2_thread
> > +------------
> > +generate load_t2 load
> > +wait for next window to start
> > +if window expires before generate load_t2 finishes,
> > +	discard off remaining load
> > +	restart generate load_t2 load
> > +variation in produced load : 0%-1% of full cpu utilization
> > --
> > 1.7.4.1
> > 

Okay, the patches apply cleanly, and it builds cleanly.
I admit I didn't look closely at what you're trying to do, but I'm a 
little skeptical that this is a real load instead of just a task spinning 
at an rt prio, but I could be wrong.

However, the major rule when doing stuff in rt-tests is don't break 
cyclictest!

When I tried to run cyclictest with your patches applied, I immediately 
observed a lot of broken behaviour, even when I tried your -x0 and -X0 
(although that shouldn't be necessary).

Basically everything moved at a crawl. So, I'm not going to look any 
closer at this until cyclictest works exactly as before. Sorry.

I know you are trying to simulate a specific load and not a generic one, 
however you might want to take a look at rteval.

rteval basically runs some standard benchmarks such as hackbench and then 
runs cyclictest at the same time to see if we can break the behaviour of 
cyclictest. Have a look at that too please and see if it might meet your 
needs.

You can fetch it here.
git://git.kernel.org/pub/scm/linux/kernel/git/clrkwllms/rteval.git>

Thanks
John 

Re: [PATCH 1/2][RFC] Add README_cyclicload

[Frank Rowand]

On 08/05/12 21:00, Jain Priyanka-B32167 wrote:
> Hi,
> 
> Waiting for the comments on this tool.
> 

< snip >

I would suggest not adding this to cyclictest, but to instead
make it a stand alone program.

The patch adds complexity to an already complex program.

-Frank

RE: [PATCH 1/2][RFC] Add README_cyclicload

[Jain Priyanka-B32167]

Thanks John and Frank for going through it.

As Frank has suggested to make it separate application and  as John also pointed out that it is breaking some of cyclic-test features, and also the targeted use-case is different, so I think it's better to maintain it as separate tool. Please comments on this. 

Other replies in-lined.

Thanks
Priyanka

-----Original Message-----
From: John Kacur [mailto:jkacur@gmail.com] On Behalf Of John Kacur
Sent: Monday, August 13, 2012 9:19 PM
To: Jain Priyanka-B32167
Cc: Clark Williams; linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; Aggrwal Poonam-B10812
Subject: RE: [PATCH 1/2][RFC] Add README_cyclicload



On Mon, 13 Aug 2012, Jain Priyanka-B32167 wrote:

> Hello Clark,
> 
> Did you have the chance to test the patch.
> 
> Thanks
> Priyanka
> 
> -----Original Message-----
> From: linux-rt-users-owner@vger.kernel.org 
> [mailto:linux-rt-users-owner@vger.kernel.org] On Behalf Of Jain 
> Priyanka-B32167
> Sent: Tuesday, August 07, 2012 9:32 AM
> To: Clark Williams
> Cc: linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; 
> rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; 
> Aggrwal Poonam-B10812
> Subject: RE: [PATCH 1/2][RFC] Add README_cyclicload
> 
> Thanks Clark
> 
> Waiting for the results at your end.
> 
> 
> Regards
> Priyanka
> 
> -----Original Message-----
> From: Clark Williams [mailto:williams@redhat.com]
> Sent: Monday, August 06, 2012 7:29 PM
> To: Jain Priyanka-B32167
> Cc: linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; 
> rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; 
> Aggrwal Poonam-B10812
> Subject: Re: [PATCH 1/2][RFC] Add README_cyclicload
> 
> Priyanka,
> 
> For some reason all of us that work on rt-tests were busy last week.
> I'll apply these patches and try out cyclicload this week.
> 
> Clark
> 
> On Mon, 6 Aug 2012 04:00:15 +0000
> Jain Priyanka-B32167 <B32167@freescale.com> wrote:
> 
> > Hi,
> > 
> > Waiting for the comments on this tool.
> > 
> > Thanks
> > Priyanka
> > 
> > -----Original Message-----
> > From: Jain Priyanka-B32167
> > Sent: Wednesday, August 01, 2012 10:37 AM
> > To: linux-rt-users@vger.kernel.org; williams@redhat.com; 
> > dvhart@linux.intel.com
> > Cc: rostedt@goodmis.org; tglx@linutronix.de; Srivastava 
> > Rajan-B34330; Aggrwal Poonam-B10812; Jain Priyanka-B32167
> > Subject: [PATCH 1/2][RFC] Add README_cyclicload
> > 
> > Cyclicload program is designed to simulate load at regular intervals in form of one or two threads.
> > 
> > Signed-off-by: Priyanka Jain <Priyanka.Jain@freescale.com>
> > ---
> >  src/cyclictest/README_cyclicload |  147
> > ++++++++++++++++++++++++++++++++++++++
> >  1 files changed, 147 insertions(+), 0 deletions(-)  create mode
> > 100644 src/cyclictest/README_cyclicload
> > 
> > diff --git a/src/cyclictest/README_cyclicload
> > b/src/cyclictest/README_cyclicload
> > new file mode 100644
> > index 0000000..f1e99bf
> > --- /dev/null
> > +++ b/src/cyclictest/README_cyclicload
> > @@ -0,0 +1,147 @@
> > +DESCRIPTION
> > +-------------
> > +
> > +The cyclicload program is developed above existing cyclictest application.
> > +It is basically designed to simulate specified load at regular 
> > +intervals in addition to cyclictest functionality.
> > +It can simulate one or two load threads.
> > +
> > +
> > +Why it is required?
> > +---------------------
> > +It is required to test system performance under a specified load 
> > +along with tracking latency of simulated load thread.
> > +
> > +
> > +Example use case
> > +--------------------
> > +For products like LTE, L2 layer runs in form of two threads.
> > +-MAC layer thread runs at highest RT priority producing a fixed 
> > +load at regular intervals and -second thread run at lower RT 
> > +priority or in non-rt priority  producing some load in each 
> > +interval depending upon availability of CPU.
> > +Requirement is to test system under this load as well as to track 
> > +latency of highest priority RT thread which is MAC thread in 
> > +example usecase.
> > +
> > +
> > +What does it do?
> > +------------------
> > +It creates one or two load generating thread/s.
> > +1)load1 thread (timer thread)
> > +2)load2 thread
> > +priority, nice value, load % as input via command line
> > +
> > +-cyclictest funcationality like latency measurement
> > +
> > +
> > +How does it work
> > +-----------------
> > +It uses generate_load loop function to simualte load.
> > +First it caliberate required loop_count per unit time per CPU.
> > +It stores this data in some file.
> > +For subsequent runs, it uses this caliberated data to generate load 
> > +if form of one or two threads .
> > +It keeps on tracking the latency of RT thread.
> > +More in Design Overview section.
> > +
> > +
> > +Recommended Settings
> > +----------------------
> > +-First run is recommended to be run with no or least load for accuracy.
> > +-should be run with sudo or root permission.
> > +-caliberation routine produces calinerate_ount file in runnign directory.
> > +	If one don't have permission in that dir, path should be changed in
> > +	FILENAME or one can exploit shared memory method.
> > +-Atleast one thread should be of RT priority. This thread takes priority of
> > +	cyclictest as its priority.
> > +-cyclictest applcication should be run with SCHED_OTHER policy.
> > +-recommended to run in quiet mode (-q) in background.
> > +TODO: add option to take filepath from cmd line
> > +
> > +
> > +Cmd line usage/examples
> > +------------------------
> > +New command line arguments:
> > + "-x       --load_t1         load in percentage for t1 thread\n"
> > + "-X       --load_t2         load in percentage for t2 thread\n"
> > + "-z       --priority_t2     priority of t2 thread\n"
> > + "-Z       --nice_t2         nice value of t2 thread\n"
> > +
> > +If both load_t1 and load_t2 are zero, it behaves as default 
> > +cyclictest application
> > +
> > +For uniprocessor:
> > +	#sudo ./cyclictest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600&
> > +
> > +For multiprocessor:
> > +	#sudo ./cyclcitest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600 -S&
> > +
> > +
> > +
> > +Future Enhancements
> > +-------------------
> > +Maintain statistics of average load produces.
> > +invalidate cache in each interval to make it close to actual scenario.
> > +can be scalable to produce n number of load threads.
> > +
> > +
> > +Design Overview
> > +--------------
> > +---------------
> > +
> > +The logic to simulate load has been added above existing cyclictest application.
> > +
> > +Threads
> > +--------
> > +cyclicload : main process
> > +--------------------------
> > +-parse input arguments.
> > +
> > +-for first run : create caliberate thread.
> > +	store caliberated count in caliberate_count file -for subsequent runs:
> > +	read caliberated count from file and use that count to simulate defined load.
> > +
> > +-create t load1threads (t depending on cmdlime args, for smp system 
> > += num of cores, one thread per core)
> > +
> > +-update stats periodically while !shutdown -print stats 
> > +periodically depending on cmdline args while !shutdown
> > +
> > +
> > +caliberate thread
> > +------------------
> > +-is created only once for the first run of cyclicload -run at 
> > +highest RT priority.
> > +-affine itself turn by turn to each cpu. (for all cpus for 
> > +multicore
> > +system) -caliberate count per unit (ms by default) per cpu -store 
> > +per cpu data in caliberate_count_array -recommended to be run with 
> > +no or least load.
> > +
> > +
> > +load1 thread(timer thread)
> > +---------------------------
> > +-run at priority parsed in main routine -recommended to run at 
> > +highest RT priority -creates load2_thread (optional only if on 
> > +load2 is
> > +nonzero) -calculate number of loops to execute to generate defined 
> > +load by
> > +	using caliberate_count_array and load percentage parsed in main 
> > +routine -reduced interval = interval(window) - duration for load_t1 
> > +-while loop until !shutdown
> > +	-generate load_t1 load
> > +	-sleep for reduced interval
> > +	-calculate latency (cosumes around 1%-2% cpu)
> > +	-discard off remaining load if window expires
> > +	-set next_window_started flag to 1
> > +	-signal load2_thread about next window
> > +-overhead: consumes 1-2% cpu for latency maintainenance, etc 
> > +-variation in produced load : 0%-2% of full cpu utilization
> > +
> > +
> > +load2_thread
> > +------------
> > +generate load_t2 load
> > +wait for next window to start
> > +if window expires before generate load_t2 finishes,
> > +	discard off remaining load
> > +	restart generate load_t2 load
> > +variation in produced load : 0%-1% of full cpu utilization
> > --
> > 1.7.4.1
> > 

Okay, the patches apply cleanly, and it builds cleanly.
I admit I didn't look closely at what you're trying to do, but I'm a little skeptical that this is a real load instead of just a task spinning at an rt prio, but I could be wrong.
[Priyanka]: Yes John, intention is to create a real cpu load so that we can test the overall system behavior (performance, latency) in presence of this load.

However, the major rule when doing stuff in rt-tests is don't break cyclictest!

When I tried to run cyclictest with your patches applied, I immediately observed a lot of broken behaviour, even when I tried your -x0 and -X0 (although that shouldn't be necessary).

Basically everything moved at a crawl. So, I'm not going to look any closer at this until cyclictest works exactly as before. Sorry.
[Priyanka]Yes, Sorry for this . It might have broken some of cyclic-test functionalities. I hadn't done much testing around them as I was concentrating more on the target use-case and basic cyclictest functionality.  I hope basic commands like ./cyclictest -S -p 99 -c 1 -d 0 is working fine. Did you face any issues in this also ?
I will do more testing around it, but if suggestion is to maintain it as separate application then basic functionality should be sufficient. Please comment.


I know you are trying to simulate a specific load and not a generic one, however you might want to take a look at rteval.

rteval basically runs some standard benchmarks such as hackbench and then runs cyclictest at the same time to see if we can break the behaviour of cyclictest. Have a look at that too please and see if it might meet your needs.

You can fetch it here.
git://git.kernel.org/pub/scm/linux/kernel/git/clrkwllms/rteval.git>
[Priyanka] : Thanks for the link. I will explore this tool .


Thanks
John 

RE: [PATCH 1/2][RFC] Add README_cyclicload

[Jain Priyanka-B32167]

-----Original Message-----
From: linux-rt-users-owner@vger.kernel.org [mailto:linux-rt-users-owner@vger.kernel.org] On Behalf Of Jain Priyanka-B32167
Sent: Tuesday, August 14, 2012 2:06 PM
To: John Kacur; Frank Rowand
Cc: Clark Williams; linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; Aggrwal Poonam-B10812
Subject: RE: [PATCH 1/2][RFC] Add README_cyclicload


Thanks John and Frank for going through it.

As Frank has suggested to make it separate application and  as John also pointed out that it is breaking some of cyclic-test features, and also the targeted use-case is different, so I think it's better to maintain it as separate tool. Please comments on this. 

Other replies in-lined.

Thanks
Priyanka

-----Original Message-----
From: John Kacur [mailto:jkacur@gmail.com] On Behalf Of John Kacur
Sent: Monday, August 13, 2012 9:19 PM
To: Jain Priyanka-B32167
Cc: Clark Williams; linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; Aggrwal Poonam-B10812
Subject: RE: [PATCH 1/2][RFC] Add README_cyclicload



On Mon, 13 Aug 2012, Jain Priyanka-B32167 wrote:

> Hello Clark,
> 
> Did you have the chance to test the patch.
> 
> Thanks
> Priyanka
> 
> -----Original Message-----
> From: linux-rt-users-owner@vger.kernel.org
> [mailto:linux-rt-users-owner@vger.kernel.org] On Behalf Of Jain
> Priyanka-B32167
> Sent: Tuesday, August 07, 2012 9:32 AM
> To: Clark Williams
> Cc: linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; 
> rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; 
> Aggrwal Poonam-B10812
> Subject: RE: [PATCH 1/2][RFC] Add README_cyclicload
> 
> Thanks Clark
> 
> Waiting for the results at your end.
> 
> 
> Regards
> Priyanka
> 
> -----Original Message-----
> From: Clark Williams [mailto:williams@redhat.com]
> Sent: Monday, August 06, 2012 7:29 PM
> To: Jain Priyanka-B32167
> Cc: linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; 
> rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; 
> Aggrwal Poonam-B10812
> Subject: Re: [PATCH 1/2][RFC] Add README_cyclicload
> 
> Priyanka,
> 
> For some reason all of us that work on rt-tests were busy last week.
> I'll apply these patches and try out cyclicload this week.
> 
> Clark
> 
> On Mon, 6 Aug 2012 04:00:15 +0000
> Jain Priyanka-B32167 <B32167@freescale.com> wrote:
> 
> > Hi,
> > 
> > Waiting for the comments on this tool.
> > 
> > Thanks
> > Priyanka
> > 
> > -----Original Message-----
> > From: Jain Priyanka-B32167
> > Sent: Wednesday, August 01, 2012 10:37 AM
> > To: linux-rt-users@vger.kernel.org; williams@redhat.com; 
> > dvhart@linux.intel.com
> > Cc: rostedt@goodmis.org; tglx@linutronix.de; Srivastava 
> > Rajan-B34330; Aggrwal Poonam-B10812; Jain Priyanka-B32167
> > Subject: [PATCH 1/2][RFC] Add README_cyclicload
> > 
> > Cyclicload program is designed to simulate load at regular intervals in form of one or two threads.
> > 
> > Signed-off-by: Priyanka Jain <Priyanka.Jain@freescale.com>
> > ---
> >  src/cyclictest/README_cyclicload |  147
> > ++++++++++++++++++++++++++++++++++++++
> >  1 files changed, 147 insertions(+), 0 deletions(-)  create mode
> > 100644 src/cyclictest/README_cyclicload
> > 
> > diff --git a/src/cyclictest/README_cyclicload
> > b/src/cyclictest/README_cyclicload
> > new file mode 100644
> > index 0000000..f1e99bf
> > --- /dev/null
> > +++ b/src/cyclictest/README_cyclicload
> > @@ -0,0 +1,147 @@
> > +DESCRIPTION
> > +-------------
> > +
> > +The cyclicload program is developed above existing cyclictest application.
> > +It is basically designed to simulate specified load at regular 
> > +intervals in addition to cyclictest functionality.
> > +It can simulate one or two load threads.
> > +
> > +
> > +Why it is required?
> > +---------------------
> > +It is required to test system performance under a specified load 
> > +along with tracking latency of simulated load thread.
> > +
> > +
> > +Example use case
> > +--------------------
> > +For products like LTE, L2 layer runs in form of two threads.
> > +-MAC layer thread runs at highest RT priority producing a fixed 
> > +load at regular intervals and -second thread run at lower RT 
> > +priority or in non-rt priority  producing some load in each 
> > +interval depending upon availability of CPU.
> > +Requirement is to test system under this load as well as to track 
> > +latency of highest priority RT thread which is MAC thread in 
> > +example usecase.
> > +
> > +
> > +What does it do?
> > +------------------
> > +It creates one or two load generating thread/s.
> > +1)load1 thread (timer thread)
> > +2)load2 thread
> > +priority, nice value, load % as input via command line
> > +
> > +-cyclictest funcationality like latency measurement
> > +
> > +
> > +How does it work
> > +-----------------
> > +It uses generate_load loop function to simualte load.
> > +First it caliberate required loop_count per unit time per CPU.
> > +It stores this data in some file.
> > +For subsequent runs, it uses this caliberated data to generate load 
> > +if form of one or two threads .
> > +It keeps on tracking the latency of RT thread.
> > +More in Design Overview section.
> > +
> > +
> > +Recommended Settings
> > +----------------------
> > +-First run is recommended to be run with no or least load for accuracy.
> > +-should be run with sudo or root permission.
> > +-caliberation routine produces calinerate_ount file in runnign directory.
> > +	If one don't have permission in that dir, path should be changed in
> > +	FILENAME or one can exploit shared memory method.
> > +-Atleast one thread should be of RT priority. This thread takes priority of
> > +	cyclictest as its priority.
> > +-cyclictest applcication should be run with SCHED_OTHER policy.
> > +-recommended to run in quiet mode (-q) in background.
> > +TODO: add option to take filepath from cmd line
> > +
> > +
> > +Cmd line usage/examples
> > +------------------------
> > +New command line arguments:
> > + "-x       --load_t1         load in percentage for t1 thread\n"
> > + "-X       --load_t2         load in percentage for t2 thread\n"
> > + "-z       --priority_t2     priority of t2 thread\n"
> > + "-Z       --nice_t2         nice value of t2 thread\n"
> > +
> > +If both load_t1 and load_t2 are zero, it behaves as default 
> > +cyclictest application
> > +
> > +For uniprocessor:
> > +	#sudo ./cyclictest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600&
> > +
> > +For multiprocessor:
> > +	#sudo ./cyclcitest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600 -S&
> > +
> > +
> > +
> > +Future Enhancements
> > +-------------------
> > +Maintain statistics of average load produces.
> > +invalidate cache in each interval to make it close to actual scenario.
> > +can be scalable to produce n number of load threads.
> > +
> > +
> > +Design Overview
> > +--------------
> > +---------------
> > +
> > +The logic to simulate load has been added above existing cyclictest application.
> > +
> > +Threads
> > +--------
> > +cyclicload : main process
> > +--------------------------
> > +-parse input arguments.
> > +
> > +-for first run : create caliberate thread.
> > +	store caliberated count in caliberate_count file -for subsequent runs:
> > +	read caliberated count from file and use that count to simulate defined load.
> > +
> > +-create t load1threads (t depending on cmdlime args, for smp system 
> > += num of cores, one thread per core)
> > +
> > +-update stats periodically while !shutdown -print stats 
> > +periodically depending on cmdline args while !shutdown
> > +
> > +
> > +caliberate thread
> > +------------------
> > +-is created only once for the first run of cyclicload -run at 
> > +highest RT priority.
> > +-affine itself turn by turn to each cpu. (for all cpus for 
> > +multicore
> > +system) -caliberate count per unit (ms by default) per cpu -store 
> > +per cpu data in caliberate_count_array -recommended to be run with 
> > +no or least load.
> > +
> > +
> > +load1 thread(timer thread)
> > +---------------------------
> > +-run at priority parsed in main routine -recommended to run at 
> > +highest RT priority -creates load2_thread (optional only if on
> > +load2 is
> > +nonzero) -calculate number of loops to execute to generate defined 
> > +load by
> > +	using caliberate_count_array and load percentage parsed in main 
> > +routine -reduced interval = interval(window) - duration for load_t1 
> > +-while loop until !shutdown
> > +	-generate load_t1 load
> > +	-sleep for reduced interval
> > +	-calculate latency (cosumes around 1%-2% cpu)
> > +	-discard off remaining load if window expires
> > +	-set next_window_started flag to 1
> > +	-signal load2_thread about next window
> > +-overhead: consumes 1-2% cpu for latency maintainenance, etc 
> > +-variation in produced load : 0%-2% of full cpu utilization
> > +
> > +
> > +load2_thread
> > +------------
> > +generate load_t2 load
> > +wait for next window to start
> > +if window expires before generate load_t2 finishes,
> > +	discard off remaining load
> > +	restart generate load_t2 load
> > +variation in produced load : 0%-1% of full cpu utilization
> > --
> > 1.7.4.1
> > 

Okay, the patches apply cleanly, and it builds cleanly.
I admit I didn't look closely at what you're trying to do, but I'm a little skeptical that this is a real load instead of just a task spinning at an rt prio, but I could be wrong.
[Priyanka]: Yes John, intention is to create a real cpu load so that we can test the overall system behavior (performance, latency) in presence of this load.

However, the major rule when doing stuff in rt-tests is don't break cyclictest!

When I tried to run cyclictest with your patches applied, I immediately observed a lot of broken behaviour, even when I tried your -x0 and -X0 (although that shouldn't be necessary).

Basically everything moved at a crawl. So, I'm not going to look any closer at this until cyclictest works exactly as before. Sorry.
[Priyanka]Yes, Sorry for this . It might have broken some of cyclic-test functionalities. I hadn't done much testing around them as I was concentrating more on the target use-case and basic cyclictest functionality.  I hope basic commands like ./cyclictest -S -p 99 -c 1 -d 0 is working fine. Did you face any issues in this also ?
I will do more testing around it, but if suggestion is to maintain it as separate application then basic functionality should be sufficient. Please comment.


I know you are trying to simulate a specific load and not a generic one, however you might want to take a look at rteval.

rteval basically runs some standard benchmarks such as hackbench and then runs cyclictest at the same time to see if we can break the behaviour of cyclictest. Have a look at that too please and see if it might meet your needs.

You can fetch it here.
git://git.kernel.org/pub/scm/linux/kernel/git/clrkwllms/rteval.git>
[Priyanka] : Thanks for the link. I will explore this tool .
[Priyanka]: Had a look at the tool. It is creating load on the system. But that load looks-like is not fixed.
I have a requirement of creating fixed % load. Like a system having 20% of load of priority 99, 30% load of priority 75, then test system behavior under this load. Is this possible with this tool? 


Thanks
John 


--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2][RFC] Add README_cyclicload

[Clark Williams]

[-- Attachment #1: Type: text/plain, Size: 13064 bytes --]


My comments inline.

On Tue, 14 Aug 2012 10:15:28 +0000
Jain Priyanka-B32167 <B32167@freescale.com> wrote:

> 
> 
> -----Original Message-----
> From: linux-rt-users-owner@vger.kernel.org [mailto:linux-rt-users-owner@vger.kernel.org] On Behalf Of Jain Priyanka-B32167
> Sent: Tuesday, August 14, 2012 2:06 PM
> To: John Kacur; Frank Rowand
> Cc: Clark Williams; linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; Aggrwal Poonam-B10812
> Subject: RE: [PATCH 1/2][RFC] Add README_cyclicload
> 
> 
> Thanks John and Frank for going through it.
> 
> As Frank has suggested to make it separate application and  as John also pointed out that it is breaking some of cyclic-test features, and also the targeted use-case is different, so I think it's better to maintain it as separate tool. Please comments on this. 

What I'd *really* like to do is pull the common routines out of
cyclictest.c and put them in separate object files, so that we could
create cyclictest-like-tests such as cyclicload without having to cram
the new logic into cyclictest.  As Frank mentioned, cyclictest is
complicated enough and somewhat fragile, so adding new logic tends to
just add new bugs as well. 

John (and Frank and the test of the CC list) what do you think?


> 
> Other replies in-lined.
> 
> Thanks
> Priyanka
> 
> -----Original Message-----
> From: John Kacur [mailto:jkacur@gmail.com] On Behalf Of John Kacur
> Sent: Monday, August 13, 2012 9:19 PM
> To: Jain Priyanka-B32167
> Cc: Clark Williams; linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; Aggrwal Poonam-B10812
> Subject: RE: [PATCH 1/2][RFC] Add README_cyclicload
> 
> 
> 
> On Mon, 13 Aug 2012, Jain Priyanka-B32167 wrote:
> 
> > Hello Clark,
> > 
> > Did you have the chance to test the patch.
> > 
> > Thanks
> > Priyanka
> > 
> > -----Original Message-----
> > From: linux-rt-users-owner@vger.kernel.org
> > [mailto:linux-rt-users-owner@vger.kernel.org] On Behalf Of Jain
> > Priyanka-B32167
> > Sent: Tuesday, August 07, 2012 9:32 AM
> > To: Clark Williams
> > Cc: linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; 
> > rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; 
> > Aggrwal Poonam-B10812
> > Subject: RE: [PATCH 1/2][RFC] Add README_cyclicload
> > 
> > Thanks Clark
> > 
> > Waiting for the results at your end.
> > 
> > 
> > Regards
> > Priyanka
> > 
> > -----Original Message-----
> > From: Clark Williams [mailto:williams@redhat.com]
> > Sent: Monday, August 06, 2012 7:29 PM
> > To: Jain Priyanka-B32167
> > Cc: linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; 
> > rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; 
> > Aggrwal Poonam-B10812
> > Subject: Re: [PATCH 1/2][RFC] Add README_cyclicload
> > 
> > Priyanka,
> > 
> > For some reason all of us that work on rt-tests were busy last week.
> > I'll apply these patches and try out cyclicload this week.
> > 
> > Clark
> > 
> > On Mon, 6 Aug 2012 04:00:15 +0000
> > Jain Priyanka-B32167 <B32167@freescale.com> wrote:
> > 
> > > Hi,
> > > 
> > > Waiting for the comments on this tool.
> > > 
> > > Thanks
> > > Priyanka
> > > 
> > > -----Original Message-----
> > > From: Jain Priyanka-B32167
> > > Sent: Wednesday, August 01, 2012 10:37 AM
> > > To: linux-rt-users@vger.kernel.org; williams@redhat.com; 
> > > dvhart@linux.intel.com
> > > Cc: rostedt@goodmis.org; tglx@linutronix.de; Srivastava 
> > > Rajan-B34330; Aggrwal Poonam-B10812; Jain Priyanka-B32167
> > > Subject: [PATCH 1/2][RFC] Add README_cyclicload
> > > 
> > > Cyclicload program is designed to simulate load at regular intervals in form of one or two threads.
> > > 
> > > Signed-off-by: Priyanka Jain <Priyanka.Jain@freescale.com>
> > > ---
> > >  src/cyclictest/README_cyclicload |  147
> > > ++++++++++++++++++++++++++++++++++++++
> > >  1 files changed, 147 insertions(+), 0 deletions(-)  create mode
> > > 100644 src/cyclictest/README_cyclicload
> > > 
> > > diff --git a/src/cyclictest/README_cyclicload
> > > b/src/cyclictest/README_cyclicload
> > > new file mode 100644
> > > index 0000000..f1e99bf
> > > --- /dev/null
> > > +++ b/src/cyclictest/README_cyclicload
> > > @@ -0,0 +1,147 @@
> > > +DESCRIPTION
> > > +-------------
> > > +
> > > +The cyclicload program is developed above existing cyclictest application.
> > > +It is basically designed to simulate specified load at regular 
> > > +intervals in addition to cyclictest functionality.
> > > +It can simulate one or two load threads.
> > > +
> > > +
> > > +Why it is required?
> > > +---------------------
> > > +It is required to test system performance under a specified load 
> > > +along with tracking latency of simulated load thread.
> > > +
> > > +
> > > +Example use case
> > > +--------------------
> > > +For products like LTE, L2 layer runs in form of two threads.
> > > +-MAC layer thread runs at highest RT priority producing a fixed 
> > > +load at regular intervals and -second thread run at lower RT 
> > > +priority or in non-rt priority  producing some load in each 
> > > +interval depending upon availability of CPU.
> > > +Requirement is to test system under this load as well as to track 
> > > +latency of highest priority RT thread which is MAC thread in 
> > > +example usecase.
> > > +
> > > +
> > > +What does it do?
> > > +------------------
> > > +It creates one or two load generating thread/s.
> > > +1)load1 thread (timer thread)
> > > +2)load2 thread
> > > +priority, nice value, load % as input via command line
> > > +
> > > +-cyclictest funcationality like latency measurement
> > > +
> > > +
> > > +How does it work
> > > +-----------------
> > > +It uses generate_load loop function to simualte load.
> > > +First it caliberate required loop_count per unit time per CPU.
> > > +It stores this data in some file.
> > > +For subsequent runs, it uses this caliberated data to generate load 
> > > +if form of one or two threads .
> > > +It keeps on tracking the latency of RT thread.
> > > +More in Design Overview section.
> > > +
> > > +
> > > +Recommended Settings
> > > +----------------------
> > > +-First run is recommended to be run with no or least load for accuracy.
> > > +-should be run with sudo or root permission.
> > > +-caliberation routine produces calinerate_ount file in runnign directory.
> > > +	If one don't have permission in that dir, path should be changed in
> > > +	FILENAME or one can exploit shared memory method.
> > > +-Atleast one thread should be of RT priority. This thread takes priority of
> > > +	cyclictest as its priority.
> > > +-cyclictest applcication should be run with SCHED_OTHER policy.
> > > +-recommended to run in quiet mode (-q) in background.
> > > +TODO: add option to take filepath from cmd line
> > > +
> > > +
> > > +Cmd line usage/examples
> > > +------------------------
> > > +New command line arguments:
> > > + "-x       --load_t1         load in percentage for t1 thread\n"
> > > + "-X       --load_t2         load in percentage for t2 thread\n"
> > > + "-z       --priority_t2     priority of t2 thread\n"
> > > + "-Z       --nice_t2         nice value of t2 thread\n"
> > > +
> > > +If both load_t1 and load_t2 are zero, it behaves as default 
> > > +cyclictest application
> > > +
> > > +For uniprocessor:
> > > +	#sudo ./cyclictest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600&
> > > +
> > > +For multiprocessor:
> > > +	#sudo ./cyclcitest -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600 -S&
> > > +
> > > +
> > > +
> > > +Future Enhancements
> > > +-------------------
> > > +Maintain statistics of average load produces.
> > > +invalidate cache in each interval to make it close to actual scenario.
> > > +can be scalable to produce n number of load threads.
> > > +
> > > +
> > > +Design Overview
> > > +--------------
> > > +---------------
> > > +
> > > +The logic to simulate load has been added above existing cyclictest application.
> > > +
> > > +Threads
> > > +--------
> > > +cyclicload : main process
> > > +--------------------------
> > > +-parse input arguments.
> > > +
> > > +-for first run : create caliberate thread.
> > > +	store caliberated count in caliberate_count file -for subsequent runs:
> > > +	read caliberated count from file and use that count to simulate defined load.
> > > +
> > > +-create t load1threads (t depending on cmdlime args, for smp system 
> > > += num of cores, one thread per core)
> > > +
> > > +-update stats periodically while !shutdown -print stats 
> > > +periodically depending on cmdline args while !shutdown
> > > +
> > > +
> > > +caliberate thread
> > > +------------------
> > > +-is created only once for the first run of cyclicload -run at 
> > > +highest RT priority.
> > > +-affine itself turn by turn to each cpu. (for all cpus for 
> > > +multicore
> > > +system) -caliberate count per unit (ms by default) per cpu -store 
> > > +per cpu data in caliberate_count_array -recommended to be run with 
> > > +no or least load.
> > > +
> > > +
> > > +load1 thread(timer thread)
> > > +---------------------------
> > > +-run at priority parsed in main routine -recommended to run at 
> > > +highest RT priority -creates load2_thread (optional only if on
> > > +load2 is
> > > +nonzero) -calculate number of loops to execute to generate defined 
> > > +load by
> > > +	using caliberate_count_array and load percentage parsed in main 
> > > +routine -reduced interval = interval(window) - duration for load_t1 
> > > +-while loop until !shutdown
> > > +	-generate load_t1 load
> > > +	-sleep for reduced interval
> > > +	-calculate latency (cosumes around 1%-2% cpu)
> > > +	-discard off remaining load if window expires
> > > +	-set next_window_started flag to 1
> > > +	-signal load2_thread about next window
> > > +-overhead: consumes 1-2% cpu for latency maintainenance, etc 
> > > +-variation in produced load : 0%-2% of full cpu utilization
> > > +
> > > +
> > > +load2_thread
> > > +------------
> > > +generate load_t2 load
> > > +wait for next window to start
> > > +if window expires before generate load_t2 finishes,
> > > +	discard off remaining load
> > > +	restart generate load_t2 load
> > > +variation in produced load : 0%-1% of full cpu utilization
> > > --
> > > 1.7.4.1
> > > 
> 
> Okay, the patches apply cleanly, and it builds cleanly.
> I admit I didn't look closely at what you're trying to do, but I'm a little skeptical that this is a real load instead of just a task spinning at an rt prio, but I could be wrong.
> [Priyanka]: Yes John, intention is to create a real cpu load so that we can test the overall system behavior (performance, latency) in presence of this load.
> 
> However, the major rule when doing stuff in rt-tests is don't break cyclictest!
> 
> When I tried to run cyclictest with your patches applied, I immediately observed a lot of broken behaviour, even when I tried your -x0 and -X0 (although that shouldn't be necessary).
> 
> Basically everything moved at a crawl. So, I'm not going to look any closer at this until cyclictest works exactly as before. Sorry.
> [Priyanka]Yes, Sorry for this . It might have broken some of cyclic-test functionalities. I hadn't done much testing around them as I was concentrating more on the target use-case and basic cyclictest functionality.  I hope basic commands like ./cyclictest -S -p 99 -c 1 -d 0 is working fine. Did you face any issues in this also ?
> I will do more testing around it, but if suggestion is to maintain it as separate application then basic functionality should be sufficient. Please comment.
> 
> 
> I know you are trying to simulate a specific load and not a generic one, however you might want to take a look at rteval.
> 
> rteval basically runs some standard benchmarks such as hackbench and then runs cyclictest at the same time to see if we can break the behaviour of cyclictest. Have a look at that too please and see if it might meet your needs.
> 
> You can fetch it here.
> git://git.kernel.org/pub/scm/linux/kernel/git/clrkwllms/rteval.git>
> [Priyanka] : Thanks for the link. I will explore this tool .
> [Priyanka]: Had a look at the tool. It is creating load on the system. But that load looks-like is not fixed.
> I have a requirement of creating fixed % load. Like a system having 20% of load of priority 99, 30% load of priority 75, then test system behavior under this load. Is this possible with this tool? 

No, rteval is focused on measuring realtime performance with a heavy
SCHED_OTHER load running on the system. So it runs a parallel kernel
make with numcpus*2 parallel jobs, then kicks off the hackbench program
to elevate the context switch rate.  I don't really see a good way to
create a precise system load with rteval. 

Clark

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [PATCH 1/2][RFC] Add README_cyclicload

[Frank Rowand]

On 08/14/12 08:30, Clark Williams wrote:
> 
> My comments inline.
> 
> On Tue, 14 Aug 2012 10:15:28 +0000 Jain Priyanka-B32167
> <B32167@freescale.com> wrote:
> 
>> 
>> 
>> -----Original Message----- From:
>> linux-rt-users-owner@vger.kernel.org
>> [mailto:linux-rt-users-owner@vger.kernel.org] On Behalf Of Jain
>> Priyanka-B32167 Sent: Tuesday, August 14, 2012 2:06 PM To: John
>> Kacur; Frank Rowand Cc: Clark Williams;
>> linux-rt-users@vger.kernel.org; dvhart@linux.intel.com;
>> rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330;
>> Aggrwal Poonam-B10812 Subject: RE: [PATCH 1/2][RFC] Add
>> README_cyclicload
>> 
>> 
>> Thanks John and Frank for going through it.

I didn't look at the load portion of the patch (the whole point of the
proposal) - I only looked at how it was breaking cyclictest.


>> 
>> As Frank has suggested to make it separate application and  as John
>> also pointed out that it is breaking some of cyclic-test features,
>> and also the targeted use-case is different, so I think it's better
>> to maintain it as separate tool. Please comments on this.
> 
> What I'd *really* like to do is pull the common routines out of 
> cyclictest.c and put them in separate object files, so that we could 
> create cyclictest-like-tests such as cyclicload without having to
> cram the new logic into cyclictest.  As Frank mentioned, cyclictest
> is complicated enough and somewhat fragile, so adding new logic tends
> to just add new bugs as well.
> 
> John (and Frank and the test of the CC list) what do you think?

If there is a set of common routines and several programs that need
to use them, then that would make sense.  If the only other use case
is cyclicload then it seems like a lot of pain for little gain.

There is already rt-tests/src/lib/ that contains common code...

< snip >

- Frank

Re: [PATCH 1/2][RFC] Add README_cyclicload

[John Kacur]

On Tue, 14 Aug 2012, Clark Williams wrote:

> 
> My comments inline.
> 
> On Tue, 14 Aug 2012 10:15:28 +0000
> Jain Priyanka-B32167 <B32167@freescale.com> wrote:
> 
> > 
> > 
> > -----Original Message-----
> > From: linux-rt-users-owner@vger.kernel.org [mailto:linux-rt-users-owner@vger.kernel.org] On Behalf Of Jain Priyanka-B32167
> > Sent: Tuesday, August 14, 2012 2:06 PM
> > To: John Kacur; Frank Rowand
> > Cc: Clark Williams; linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; Aggrwal Poonam-B10812
> > Subject: RE: [PATCH 1/2][RFC] Add README_cyclicload
> > 
> > 
> > Thanks John and Frank for going through it.
> > 
> > As Frank has suggested to make it separate application and  as John also pointed out that it is breaking some of cyclic-test features, and also the targeted use-case is different, so I think it's better to maintain it as separate tool. Please comments on this. 
> 
> What I'd *really* like to do is pull the common routines out of
> cyclictest.c and put them in separate object files, so that we could
> create cyclictest-like-tests such as cyclicload without having to cram
> the new logic into cyclictest.  As Frank mentioned, cyclictest is
> complicated enough and somewhat fragile, so adding new logic tends to
> just add new bugs as well. 
> 
> John (and Frank and the test of the CC list) what do you think?
> 

Yes, that is what we want to do, and we've started doing so, although it 
is still pretty bare bones. This code is in src/lib. We should keep 
enhancing this.

But it's too much of course to ask someone to do this who just wants to 
get a new tool into the code. So, I would require this.

1. The new tool doesn't break cyclictest, this is rule number one.
2. The new tool does what it claims to do, and is useful.
3. Doesn't unncessarily complicate cyclictest.

I think you automatically lower the bar if you can do this separately 
because you won't be breaking cyclictest. Of course, it doesn't seem very 
smart to just go and duplicate a lot of code though, it might be good 
enough for a prototype. Is a prototype good enough for rttests though?

I think the answer to that depends on meeting requirement number 2.

My first impression running cyclicload was, that it needs a bit of work.
One example of many, using the command line for smp he gave in his README
I got

(gdb) r  -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600 -S
Program received signal SIGFPE, Arithmetic exception.
[Switching to Thread 0x7ffff7fcc700 (LWP 21506)]
0x00000000004042c5 in timerthread (param=0x60c490)
    at src/cyclictest/cyclictest.c:1032
1032					((stat->done_t1 * 
100)/stat->num_t1))/ 
glibc-2.15-51.fc17.x86_64 numactl-libs-2.0.7-6.fc17.x86_64
(gdb) print stat->num_t1
$1 = 0
 
It has to be at a certain level before we're interested in reviewing it more 
closely. I'm not sure if Jain was just showing us a prototype or he 
thought it was ready to go in?

I'm not willing to review it until it's polished some more.

 --->o snip

RE: [PATCH 1/2][RFC] Add README_cyclicload

[Jain Priyanka-B32167]

-----Original Message-----
From: John Kacur [mailto:jkacur@gmail.com] On Behalf Of John Kacur
Sent: Wednesday, August 15, 2012 2:05 AM
To: Clark Williams
Cc: Jain Priyanka-B32167; John Kacur; Frank Rowand; linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; Aggrwal Poonam-B10812
Subject: Re: [PATCH 1/2][RFC] Add README_cyclicload



On Tue, 14 Aug 2012, Clark Williams wrote:

> 
> My comments inline.
> 
> On Tue, 14 Aug 2012 10:15:28 +0000
> Jain Priyanka-B32167 <B32167@freescale.com> wrote:
> 
> > 
> > 
> > -----Original Message-----
> > From: linux-rt-users-owner@vger.kernel.org 
> > [mailto:linux-rt-users-owner@vger.kernel.org] On Behalf Of Jain 
> > Priyanka-B32167
> > Sent: Tuesday, August 14, 2012 2:06 PM
> > To: John Kacur; Frank Rowand
> > Cc: Clark Williams; linux-rt-users@vger.kernel.org; 
> > dvhart@linux.intel.com; rostedt@goodmis.org; tglx@linutronix.de; 
> > Srivastava Rajan-B34330; Aggrwal Poonam-B10812
> > Subject: RE: [PATCH 1/2][RFC] Add README_cyclicload
> > 
> > 
> > Thanks John and Frank for going through it.
> > 
> > As Frank has suggested to make it separate application and  as John also pointed out that it is breaking some of cyclic-test features, and also the targeted use-case is different, so I think it's better to maintain it as separate tool. Please comments on this. 
> 
> What I'd *really* like to do is pull the common routines out of 
> cyclictest.c and put them in separate object files, so that we could 
> create cyclictest-like-tests such as cyclicload without having to cram 
> the new logic into cyclictest.  As Frank mentioned, cyclictest is 
> complicated enough and somewhat fragile, so adding new logic tends to 
> just add new bugs as well.
> 
> John (and Frank and the test of the CC list) what do you think?
> 

Yes, that is what we want to do, and we've started doing so, although it is still pretty bare bones. This code is in src/lib. We should keep enhancing this.

But it's too much of course to ask someone to do this who just wants to get a new tool into the code. So, I would require this.

1. The new tool doesn't break cyclictest, this is rule number one.
2. The new tool does what it claims to do, and is useful.
3. Doesn't unncessarily complicate cyclictest.

I think you automatically lower the bar if you can do this separately because you won't be breaking cyclictest. Of course, it doesn't seem very smart to just go and duplicate a lot of code though, it might be good enough for a prototype. Is a prototype good enough for rttests though?

I think the answer to that depends on meeting requirement number 2.

My first impression running cyclicload was, that it needs a bit of work.
One example of many, using the command line for smp he gave in his README I got

(gdb) r  -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600 -S Program received signal SIGFPE, Arithmetic exception.
[Switching to Thread 0x7ffff7fcc700 (LWP 21506)]
0x00000000004042c5 in timerthread (param=0x60c490)
    at src/cyclictest/cyclictest.c:1032
1032					((stat->done_t1 * 
100)/stat->num_t1))/
glibc-2.15-51.fc17.x86_64 numactl-libs-2.0.7-6.fc17.x86_64
(gdb) print stat->num_t1
$1 = 0
 
It has to be at a certain level before we're interested in reviewing it more closely. I'm not sure if Jain was just showing us a prototype or he thought it was ready to go in?

[Priyanka]: It is a RFC version . I am working on to make it a robust one. I will send the polished version soon. My major concern for the polished version is to propose this as separate tool or as part of cyclictest . Also, as you have mentioned above that cyclic-load need a bit of work, can you please mentioned those rework required liked the one you highlighted. It will help me in correcting the tool faster. Thanks a lot, Looking forward for your support.

I'm not willing to review it until it's polished some more.

 --->o snip

Re: [PATCH 1/2][RFC] Add README_cyclicload

[John Kacur]

On Fri, Aug 17, 2012 at 1:04 PM, Jain Priyanka-B32167
<B32167@freescale.com> wrote:
>
>
> -----Original Message-----
> From: John Kacur [mailto:jkacur@gmail.com] On Behalf Of John Kacur
> Sent: Wednesday, August 15, 2012 2:05 AM
> To: Clark Williams
> Cc: Jain Priyanka-B32167; John Kacur; Frank Rowand; linux-rt-users@vger.kernel.org; dvhart@linux.intel.com; rostedt@goodmis.org; tglx@linutronix.de; Srivastava Rajan-B34330; Aggrwal Poonam-B10812
> Subject: Re: [PATCH 1/2][RFC] Add README_cyclicload
>
>
>
> On Tue, 14 Aug 2012, Clark Williams wrote:
>
>>
>> My comments inline.
>>
>> On Tue, 14 Aug 2012 10:15:28 +0000
>> Jain Priyanka-B32167 <B32167@freescale.com> wrote:
>>
>> >
>> >
>> > -----Original Message-----
>> > From: linux-rt-users-owner@vger.kernel.org
>> > [mailto:linux-rt-users-owner@vger.kernel.org] On Behalf Of Jain
>> > Priyanka-B32167
>> > Sent: Tuesday, August 14, 2012 2:06 PM
>> > To: John Kacur; Frank Rowand
>> > Cc: Clark Williams; linux-rt-users@vger.kernel.org;
>> > dvhart@linux.intel.com; rostedt@goodmis.org; tglx@linutronix.de;
>> > Srivastava Rajan-B34330; Aggrwal Poonam-B10812
>> > Subject: RE: [PATCH 1/2][RFC] Add README_cyclicload
>> >
>> >
>> > Thanks John and Frank for going through it.
>> >
>> > As Frank has suggested to make it separate application and  as John also pointed out that it is breaking some of cyclic-test features, and also the targeted use-case is different, so I think it's better to maintain it as separate tool. Please comments on this.
>>
>> What I'd *really* like to do is pull the common routines out of
>> cyclictest.c and put them in separate object files, so that we could
>> create cyclictest-like-tests such as cyclicload without having to cram
>> the new logic into cyclictest.  As Frank mentioned, cyclictest is
>> complicated enough and somewhat fragile, so adding new logic tends to
>> just add new bugs as well.
>>
>> John (and Frank and the test of the CC list) what do you think?
>>
>
> Yes, that is what we want to do, and we've started doing so, although it is still pretty bare bones. This code is in src/lib. We should keep enhancing this.
>
> But it's too much of course to ask someone to do this who just wants to get a new tool into the code. So, I would require this.
>
> 1. The new tool doesn't break cyclictest, this is rule number one.
> 2. The new tool does what it claims to do, and is useful.
> 3. Doesn't unncessarily complicate cyclictest.
>
> I think you automatically lower the bar if you can do this separately because you won't be breaking cyclictest. Of course, it doesn't seem very smart to just go and duplicate a lot of code though, it might be good enough for a prototype. Is a prototype good enough for rttests though?
>
> I think the answer to that depends on meeting requirement number 2.
>
> My first impression running cyclicload was, that it needs a bit of work.
> One example of many, using the command line for smp he gave in his README I got
>
> (gdb) r  -p 99 -c 1 -d 0 -x 40 -X 30 -q -D 600 -S Program received signal SIGFPE, Arithmetic exception.
> [Switching to Thread 0x7ffff7fcc700 (LWP 21506)]
> 0x00000000004042c5 in timerthread (param=0x60c490)
>     at src/cyclictest/cyclictest.c:1032
> 1032                                    ((stat->done_t1 *
> 100)/stat->num_t1))/
> glibc-2.15-51.fc17.x86_64 numactl-libs-2.0.7-6.fc17.x86_64
> (gdb) print stat->num_t1
> $1 = 0
>
> It has to be at a certain level before we're interested in reviewing it more closely. I'm not sure if Jain was just showing us a prototype or he thought it was ready to go in?
>
> [Priyanka]: It is a RFC version . I am working on to make it a robust one. I will send the polished version soon. My major concern for the polished version is to propose this as separate tool or as part of cyclictest . Also, as you have mentioned above that cyclic-load need a bit of work, can you please mentioned those rework required liked the one you highlighted. It will help me in correcting the tool faster. Thanks a lot, Looking forward for your support.
>

You should make it into a separate tool in it's own directory just
like cyclictest and company. ( rt-tests/src/cyclicload ) For now you
can copy the code over. If you get cyclicload working well enough and
we accept it, then we can do refractoring work later to use common
functions and so on.

Thanks

John