On Sun, 21 Mar 2021, Jonathan Schwender wrote:

> 
> On 2/22/21 5:20 PM, Ahmed S. Darwish wrote:
> > On Mon, Feb 22, 2021 at 04:28:30PM +0100, Jonathan Schwender wrote:
> >> Hi John,
> >>
> >> This patch adds the option --mainaffinity to specify the affinity of
> >> the main pid.
> >> This is mainly useful if you want to bind the main thread to a
> >> different (e.g. housekeeping ) CPU than the measurement threads.
> >>
> > Do you also have any numbers showing different results with/without
> > "--mainaffinity"?
> 
> Sorry for the delay. I do have some numbers now, and there is a benefit to
> using this option if CPU isolation + CAT are used. Otherwise it's not really
> visible.
> 
> Rendered Markdown:
> https://gist.github.com/jschwe/d4c46026aec57b10a2b0e6f72258b96e
> 
> # Testing proposed cyclictest --mainaffinity option
> 
> System information:
> - 2 socket Intel Xeon E5-2643 v4 @ 3.40Ghz
> - Turbo boost is disabled.
> - Fedora 33 with kernel 5.10.1-rt20 with small patch (see cmdline isolcpus)
> - Cmdline: `nosmt isolcpus=domain,managed_irq,wq,rcu,misc,kthread,3,5,7,9,11
> rcu_nocbs=3,5,7,9,11 irqaffinity=0,2,4 maxcpus=12 rcu_nocb_poll nowatchdog
> tsc=nowatchdog processor.max_cstate=1 intel_idle.max_cstate=0
> systemd.unified_cgroup_hierarchy=0`
>     - The additional isolcpus arguments set the HK_FLAG with the corresponding
>     name.
>       This cmdline adds all HK_FLAGs usually set by nohz_full, except the
>       actual
>       nohz flags `tick` and `timer`. This improves cyclictest latencies on my
>       system.
> - Rteval is running on all CPUs from node 0 + CPU 1, but not on the isolated
> CPUs.
> - L3 Cache is reserved for the isolated CPUs via `resctrl` (CPU based
> allocation)
> - Test duration 24 hours, interval 200 µs
> 
> 
> ## Test 1: 5 cyclictest instance with main pid on same cpu as the measurement
> thread
> 
> This test simply starts 5 cyclictest instances (via numactl) with one
> measurement thread each and bound to a single
> CPU via `--affinity`, so that the main thread is also bound to the same CPU.
> 
> ![Figure: 5 cyclictest instances with main pid pinned to same CPU as
> measurement
> thread](https://gist.githubusercontent.com/jschwe/d4c46026aec57b10a2b0e6f72258b96e/raw/e27c3f284cf4bbeecded84865dfee5676b47fe88/2021-03-11.png)
> 
> ## Test 2: Single cyclictest instance with --mainaffinity=1 for isolated CPUs
> 3,5,7,9,11
> The main thread was placed on CPU 1 via `--mainaffinity` and
> `--refresh_on_max` was added for good measure to keep the
> logfile small.
> 
> ![Figure: Single cyclictest instance with --mainaffinity=1 for isolated CPUs
> 3,5,7,9,11](https://gist.githubusercontent.com/jschwe/d4c46026aec57b10a2b0e6f72258b96e/raw/afd81b2a70a3e88bdebc46615d0f60e24238b405/2021-03-19.png)
> 
> >
> > Thanks,
> >
> > --
> > Ahmed S. Darwish
> Jonathan
> 
> 

Alright, I can imagine how this could be useful. Would you respin the 
patches against the latest upstream and resend to me?

Thanks

John