From: Yury Norov <yury.norov@gmail.com>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: Pavel Machek <pavel@ucw.cz>, Alex Belits <abelits@marvell.com>,
"nitesh@redhat.com" <nitesh@redhat.com>,
"frederic@kernel.org" <frederic@kernel.org>,
Prasun Kapoor <pkapoor@marvell.com>,
"linux-api@vger.kernel.org" <linux-api@vger.kernel.org>,
"davem@davemloft.net" <davem@davemloft.net>,
"trix@redhat.com" <trix@redhat.com>,
"mingo@kernel.org" <mingo@kernel.org>,
"catalin.marinas@arm.com" <catalin.marinas@arm.com>,
"rostedt@goodmis.org" <rostedt@goodmis.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"peterx@redhat.com" <peterx@redhat.com>,
"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
"mtosatti@redhat.com" <mtosatti@redhat.com>,
"will@kernel.org" <will@kernel.org>,
"peterz@infradead.org" <peterz@infradead.org>,
"leon@sidebranch.com" <leon@sidebranch.com>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
"pauld@redhat.com" <pauld@redhat.com>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>
Subject: Re: [PATCH v5 0/9] "Task_isolation" mode
Date: Fri, 11 Dec 2020 10:08:09 -0800 [thread overview]
Message-ID: <20201211180809.GA397355@yury-ThinkPad> (raw)
In-Reply-To: <87h7oz96o6.fsf@nanos.tec.linutronix.de>
On Sun, Dec 06, 2020 at 12:25:45AM +0100, Thomas Gleixner wrote:
> Pavel,
>
> On Sat, Dec 05 2020 at 21:40, Pavel Machek wrote:
> > So... what kind of guarantees does this aim to provide / what tasks it
> > is useful for?
> >
> > For real time response, we have other approaches.
>
> Depends on your requirements. Some problems are actually better solved
> with busy polling. See below.
>
> > If you want to guarantee performnace of the "isolated" task... I don't
> > see how that works. Other tasks on the system still compete for DRAM
> > bandwidth, caches, etc...
>
> Applications which want to run as undisturbed as possible. There is
> quite a range of those:
>
> - Hardware in the loop simulation is today often done with that crude
> approach of "offlining" a CPU and then instead of playing dead
> jumping to a preloaded bare metal executable. That's a horrible hack
> and impossible to debug, but gives them the results they need to
> achieve. These applications are well optimized vs. cache and memory
> foot print, so they don't worry about these things too much and they
> surely don't run on SMI and BIOS value add inflicted machines.
>
> Don't even think about waiting for an interrupt to achieve what
> these folks are doing. So no, there are problems which a general
> purpose realtime OS cannot solve ever.
>
> - HPC computations on large data sets. While the memory foot print is
> large the access patterns are cache optimized.
>
> The problem there is that any unnecessary IPI, tick interrupt or
> whatever nuisance is disturbing the carefully optimized cache usage
> and alone getting rid of the timer interrupt gained them measurable
> performance. Even very low single digit percentage of runtime saving
> is valuable for these folks because the compute time on such beasts
> is expensive.
>
> - Realtime guests in KVM. With posted interrupts and a fully populated
> host side page table there is no point in running host side
> interrupts or IPIs for random accounting or whatever purposes as
> they affect the latency in the guest. With all the side effects
> mitigated and a properly set up guest and host it is possible to get
> to a zero exit situation after the bootup phase which means pretty
> much matching bare metal behaviour.
>
> Yes, you can do that with e.g. Jailhouse as well, but you lose lots
> of the fancy things KVM provides. And people care about these not
> just because they are fancy. They care because their application
> scenario needs them.
>
> There are more reasons why people want to be able to get as much
> isolation from the OS as possible but at the same time have a sane
> execution environment, debugging, performance monitoring and the OS
> provided protection mechanisms instead of horrible hacks.
>
> Isolation makes sense for a range of applications and there is no reason
> why Linux should not support them.
One good client for the task isolation is Open Data Plane. There are
even some code stubs supposed to enable isolation where needed.
> > If you want to guarantee performnace of the "isolated" task... I don't
> > see how that works. Other tasks on the system still compete for DRAM
> > bandwidth, caches, etc...
My experiments say that typical delay caused by dry IPI or syscall is
2000-20000 'ticks'. Typical delay caused by cache miss is 3-30 ticks.
To guarantee cache / memory bandwidth, one can use resctrl. Linux has
implementation of it for x86 only, but arm64 has support for for
resctrl on CPU side.
Thanks,
Yury
> Thanks,
>
> tglx
prev parent reply other threads:[~2020-12-11 19:44 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-23 17:42 [PATCH v5 0/9] "Task_isolation" mode Alex Belits
2020-11-23 17:56 ` [PATCH v5 1/9] task_isolation: vmstat: add quiet_vmstat_sync function Alex Belits
2020-11-23 21:48 ` Thomas Gleixner
2020-11-23 17:56 ` [PATCH v5 2/9] task_isolation: vmstat: add vmstat_idle function Alex Belits
2020-11-23 21:49 ` Thomas Gleixner
2020-11-23 17:56 ` [PATCH v5 3/9] task_isolation: userspace hard isolation from kernel Alex Belits
2020-11-23 22:01 ` Thomas Gleixner
2020-11-23 17:57 ` [PATCH v5 4/9] task_isolation: Add task isolation hooks to arch-independent code Alex Belits
2020-11-23 22:31 ` Thomas Gleixner
2020-11-23 17:57 ` [PATCH v5 5/9] task_isolation: Add driver-specific hooks Alex Belits
2020-12-02 14:18 ` Mark Rutland
2020-12-04 0:43 ` [EXT] " Alex Belits
2020-11-23 17:58 ` [PATCH v5 6/9] task_isolation: arch/arm64: enable task isolation functionality Alex Belits
2020-12-02 13:59 ` Mark Rutland
2020-12-04 0:37 ` [EXT] " Alex Belits
2020-12-07 11:57 ` Mark Rutland
2020-11-23 17:58 ` [PATCH v5 7/9] task_isolation: don't interrupt CPUs with tick_nohz_full_kick_cpu() Alex Belits
2020-11-23 22:13 ` Frederic Weisbecker
2020-11-23 22:35 ` Alex Belits
2020-11-23 22:36 ` Thomas Gleixner
2020-12-02 14:20 ` Mark Rutland
2020-12-04 0:54 ` [EXT] " Alex Belits
2020-12-07 11:58 ` Mark Rutland
2020-11-23 17:58 ` [PATCH v5 8/9] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Alex Belits
2020-11-23 17:58 ` [PATCH v5 9/9] task_isolation: kick_all_cpus_sync: don't kick isolated cpus Alex Belits
2020-11-23 22:29 ` Frederic Weisbecker
2020-11-23 22:39 ` [EXT] " Alex Belits
2020-11-23 23:21 ` Frederic Weisbecker
2020-11-25 3:20 ` Alex Belits
2021-01-22 15:00 ` Marcelo Tosatti
2020-11-24 16:36 ` [PATCH v5 0/9] "Task_isolation" mode Tom Rix
2020-11-24 17:40 ` [EXT] " Alex Belits
2020-12-02 14:02 ` Mark Rutland
2020-12-04 0:39 ` Alex Belits
2020-12-05 20:40 ` Pavel Machek
2020-12-05 23:25 ` Thomas Gleixner
2020-12-11 18:08 ` Yury Norov [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201211180809.GA397355@yury-ThinkPad \
--to=yury.norov@gmail.com \
--cc=abelits@marvell.com \
--cc=catalin.marinas@arm.com \
--cc=davem@davemloft.net \
--cc=frederic@kernel.org \
--cc=leon@sidebranch.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-arch@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=mtosatti@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=nitesh@redhat.com \
--cc=pauld@redhat.com \
--cc=pavel@ucw.cz \
--cc=peterx@redhat.com \
--cc=peterz@infradead.org \
--cc=pkapoor@marvell.com \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=trix@redhat.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).