From: Alexander Duyck <alexander.duyck@gmail.com>
To: "Michael S. Tsirkin" <mst@redhat.com>
Cc: David Hildenbrand <david@redhat.com>,
Nitesh Narayan Lal <nitesh@redhat.com>,
kvm list <kvm@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
linux-mm <linux-mm@kvack.org>,
Paolo Bonzini <pbonzini@redhat.com>,
lcapitulino@redhat.com, pagupta@redhat.com, wei.w.wang@intel.com,
Yang Zhang <yang.zhang.wz@gmail.com>,
Rik van Riel <riel@surriel.com>,
dodgen@google.com, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
dhildenb@redhat.com, Andrea Arcangeli <aarcange@redhat.com>
Subject: Re: [RFC][Patch v9 0/6] KVM: Guest Free Page Hinting
Date: Wed, 6 Mar 2019 15:12:33 -0800 [thread overview]
Message-ID: <CAKgT0UeN=o0NratKSmhBhBBmwQ06qo9tiU9sqvpq_u8uMZPM1w@mail.gmail.com> (raw)
In-Reply-To: <20190306165223-mutt-send-email-mst@kernel.org>
On Wed, Mar 6, 2019 at 2:18 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Wed, Mar 06, 2019 at 10:40:57PM +0100, David Hildenbrand wrote:
> > On 06.03.19 21:32, Michael S. Tsirkin wrote:
> > > On Wed, Mar 06, 2019 at 07:59:57PM +0100, David Hildenbrand wrote:
> > >> On 06.03.19 19:43, Michael S. Tsirkin wrote:
> > >>> On Wed, Mar 06, 2019 at 01:30:14PM -0500, Nitesh Narayan Lal wrote:
> > >>>>>> Here are the results:
> > >>>>>>
> > >>>>>> Procedure: 3 Guests of size 5GB is launched on a single NUMA node with
> > >>>>>> total memory of 15GB and no swap. In each of the guest, memhog is run
> > >>>>>> with 5GB. Post-execution of memhog, Host memory usage is monitored by
> > >>>>>> using Free command.
> > >>>>>>
> > >>>>>> Without Hinting:
> > >>>>>> Time of execution Host used memory
> > >>>>>> Guest 1: 45 seconds 5.4 GB
> > >>>>>> Guest 2: 45 seconds 10 GB
> > >>>>>> Guest 3: 1 minute 15 GB
> > >>>>>>
> > >>>>>> With Hinting:
> > >>>>>> Time of execution Host used memory
> > >>>>>> Guest 1: 49 seconds 2.4 GB
> > >>>>>> Guest 2: 40 seconds 4.3 GB
> > >>>>>> Guest 3: 50 seconds 6.3 GB
> > >>>>> OK so no improvement.
> > >>>> If we are looking in terms of memory we are getting back from the guest,
> > >>>> then there is an improvement. However, if we are looking at the
> > >>>> improvement in terms of time of execution of memhog then yes there is none.
> > >>>
> > >>> Yes but the way I see it you can't overcommit this unused memory
> > >>> since guests can start using it at any time. You timed it carefully
> > >>> such that this does not happen, but what will cause this timing on real
> > >>> guests?
> > >>
> > >> Whenever you overcommit you will need backup swap.
> > >
> > > Right and the point of hinting is that pages can just be
> > > discarded and not end up in swap.
> > >
> > >
> > > Point is you should be able to see the gain.
> > >
> > > Hinting patches cost some CPU so we need to know whether
> > > they cost too much. How much is too much? When the cost
> > > is bigger than benefit. But we can't compare CPU cycles
> > > to bytes. So we need to benchmark everything in terms of
> > > cycles.
> > >
> > >> There is no way
> > >> around it. It just makes the probability of you having to go to disk
> > >> less likely.
> > >
> > >
> > > Right and let's quantify this. Does this result in net gain or loss?
> >
> > Yes, I am totally with you. But if it is a net benefit heavily depends
> > on the setup. E.g. what kind of storage used for the swap, how fast, is
> > the same disk also used for other I/O ...
> >
> > Also, CPU is a totally different resource than I/O. While you might have
> > plenty of CPU cycles to spare, your I/O throughput might already be
> > limited. Same goes into the other direction.
> >
> > So it might not be as easy as comparing two numbers. It really depends
> > on the setup. Well, not completely true, with 0% CPU overhead we would
> > have a clear winner with hinting ;)
>
> I mean users need to know about this too.
>
> Are these hinting patches a gain:
> - on zram
> - on ssd
> - on a rotating disk
> - none of the above
> ?
>
> If users don't know when would they enable hinting?
>
> Close to one is going to try all possible configurations, test
> exhaustively and find an optimal default for their workload.
> So it's our job to figure it out and provide guidance.
Right. I think for now I will stick to testing on what I have which is
a SSD for swap, and no-overcommit for the "non of the above" case.
BTW it looks like this patch set introduced a pretty heavy penalty for
the no-overcommit case. For a 32G VM with no overcommit a 32G memhog
test is now taking over 50 seconds whereas without the patch set I can
complete the test in around 20 seconds.
> >
> > >
> > >
> > >> If you assume that all of your guests will be using all of their memory
> > >> all the time, you don't have to think about overcommiting memory in the
> > >> first place. But this is not what we usually have.
> > >
> > > Right and swap is there to support overcommit. However it
> > > was felt that hinting can be faster since it avoids IO
> > > involved in swap.
> >
> > Feels like it, I/O is prone to be slow.
> >
> >
> > --
> >
> > Thanks,
> >
> > David / dhildenb
>
> OK so should be measureable.
>
> --
> MST
next prev parent reply other threads:[~2019-03-06 23:12 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-03-06 15:50 [RFC][Patch v9 0/6] KVM: Guest Free Page Hinting Nitesh Narayan Lal
2019-03-06 15:50 ` [RFC][Patch v9 1/6] KVM: Guest free page hinting support Nitesh Narayan Lal
2019-03-06 23:43 ` Alexander Duyck
2019-03-07 19:32 ` Nitesh Narayan Lal
2019-03-06 15:50 ` [RFC][Patch v9 2/6] KVM: Enables the kernel to isolate guest free pages Nitesh Narayan Lal
2019-03-07 18:30 ` Alexander Duyck
2019-03-07 19:23 ` Nitesh Narayan Lal
2019-03-07 19:30 ` David Hildenbrand
2019-03-07 21:32 ` Alexander Duyck
2019-03-07 21:40 ` David Hildenbrand
2019-03-07 22:35 ` Alexander Duyck
2019-03-08 2:28 ` Michael S. Tsirkin
2019-03-08 2:32 ` Michael S. Tsirkin
2019-03-08 18:06 ` Alexander Duyck
2019-03-08 18:59 ` Michael S. Tsirkin
2019-03-08 19:10 ` Nitesh Narayan Lal
2019-03-08 19:25 ` Alexander Duyck
2019-03-08 19:38 ` Nitesh Narayan Lal
2019-03-08 21:39 ` Alexander Duyck
2019-03-12 19:46 ` Nitesh Narayan Lal
2019-03-12 21:13 ` Alexander Duyck
2019-03-12 21:53 ` David Hildenbrand
2019-03-12 22:56 ` Alexander Duyck
2019-03-13 11:54 ` Nitesh Narayan Lal
2019-03-13 12:17 ` David Hildenbrand
2019-03-13 13:08 ` Nitesh Narayan Lal
2019-03-13 16:37 ` Alexander Duyck
2019-03-13 16:39 ` David Hildenbrand
2019-03-13 22:54 ` Alexander Duyck
2019-03-13 23:18 ` David Hildenbrand
2019-03-06 15:50 ` [RFC][Patch v9 3/6] KVM: Enables the kernel to report isolated pages Nitesh Narayan Lal
2019-03-06 21:30 ` Alexander Duyck
2019-03-07 13:23 ` Nitesh Narayan Lal
2019-03-06 15:50 ` [RFC][Patch v9 4/6] KVM: Reporting page poisoning value to the host Nitesh Narayan Lal
2019-03-06 15:50 ` [RFC][Patch v9 5/6] KVM: Enabling guest free page hinting via static key Nitesh Narayan Lal
2019-03-06 15:50 ` [RFC][Patch v9 6/6] KVM: Adding tracepoints for guest free page hinting Nitesh Narayan Lal
2019-03-06 15:52 ` [RFC][QEMU Patch] KVM: Enable QEMU to free the pages hinted by the guest Nitesh Narayan Lal
2019-03-06 23:49 ` Alexander Duyck
2019-03-07 0:35 ` Alexander Duyck
2019-03-07 12:23 ` Nitesh Narayan Lal
2019-03-06 16:09 ` [RFC][Patch v9 0/6] KVM: Guest Free Page Hinting Michael S. Tsirkin
2019-03-06 18:07 ` Nitesh Narayan Lal
2019-03-06 18:12 ` Michael S. Tsirkin
2019-03-06 18:30 ` Nitesh Narayan Lal
2019-03-06 18:38 ` Michael S. Tsirkin
2019-03-06 18:40 ` Nitesh Narayan Lal
2019-03-06 18:43 ` Alexander Duyck
2019-03-06 18:43 ` Michael S. Tsirkin
2019-03-06 18:59 ` David Hildenbrand
2019-03-06 19:08 ` Alexander Duyck
2019-03-06 19:18 ` David Hildenbrand
2019-03-06 19:24 ` Alexander Duyck
2019-03-06 20:31 ` Nitesh Narayan Lal
2019-03-06 20:32 ` Michael S. Tsirkin
2019-03-06 21:40 ` David Hildenbrand
2019-03-06 22:18 ` Michael S. Tsirkin
2019-03-06 23:12 ` Alexander Duyck [this message]
2019-03-14 16:42 ` Nitesh Narayan Lal
2019-03-14 16:58 ` Alexander Duyck
2019-03-18 15:57 ` Nitesh Narayan Lal
2019-03-19 13:33 ` David Hildenbrand
2019-03-19 16:04 ` Nitesh Narayan Lal
2019-03-19 17:38 ` Alexander Duyck
2019-03-19 17:59 ` Nitesh Narayan Lal
2019-03-20 13:18 ` Nitesh Narayan Lal
2019-03-25 14:27 ` Nitesh Narayan Lal
2019-03-25 15:37 ` Michael S. Tsirkin
2019-03-25 15:42 ` Nitesh Narayan Lal
2019-03-06 18:00 ` Alexander Duyck
2019-03-06 19:07 ` Nitesh Narayan Lal
2019-03-06 22:05 ` Alexander Duyck
2019-03-07 13:09 ` Nitesh Narayan Lal
2019-03-07 18:45 ` Alexander Duyck
2019-03-07 18:53 ` Michael S. Tsirkin
2019-03-07 19:27 ` David Hildenbrand
2019-03-08 2:24 ` Michael S. Tsirkin
2019-03-08 11:53 ` David Hildenbrand
2019-03-07 21:14 ` Alexander Duyck
2019-03-07 21:28 ` David Hildenbrand
2019-03-07 22:19 ` Alexander Duyck
2019-03-07 19:45 ` Nitesh Narayan Lal
2019-03-07 19:49 ` David Hildenbrand
2019-03-07 18:46 ` Michael S. Tsirkin
2019-03-12 19:58 ` David Hildenbrand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAKgT0UeN=o0NratKSmhBhBBmwQ06qo9tiU9sqvpq_u8uMZPM1w@mail.gmail.com' \
--to=alexander.duyck@gmail.com \
--cc=aarcange@redhat.com \
--cc=david@redhat.com \
--cc=dhildenb@redhat.com \
--cc=dodgen@google.com \
--cc=konrad.wilk@oracle.com \
--cc=kvm@vger.kernel.org \
--cc=lcapitulino@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mst@redhat.com \
--cc=nitesh@redhat.com \
--cc=pagupta@redhat.com \
--cc=pbonzini@redhat.com \
--cc=riel@surriel.com \
--cc=wei.w.wang@intel.com \
--cc=yang.zhang.wz@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).